Computer Vision. Colorado School of Mines. Professor William Hoff Dept of Electrical Engineering &Computer Science.
|
|
- Bertram Reeves
- 5 years ago
- Views:
Transcription
1 Professor William Hoff Dept of Electrical Engineering &Computer Science
2 Pattern Recognition
3 Pattern Recognition The process by which patterns in data are found, recognized, discovered Usually aims to classify data (patterns) based on either a priori knowledge or on statistical information extracted from the patterns The patterns to be classified are observations, defining points in a multidimensional space Classification is usually based on a set of patterns that have already been classified (e.g., by a person) This set of patterns is termed the training set The learning strategy is called supervised Learning can also be unsupervised In this case there is no training set Instead it establishes the classes itself based on the statistical regularities of the patterns Resources Statistical Toolbox in Matlab Journals include Pattern Recognition, IEEE Trans. Pattern Analysis & Machine Intelligence Good book: Pattern Recognition and Machine Learning by Bishop 3
4 Approaches Statistical Pattern Recognition We assume that the patterns are generated by a probabilistic system The data is reduced to vectors of numbers and statistical techniques are used for classification Structural Pattern Recognition The process is based on the structural interrelationships of features The data is converted to a discrete structure (such as a grammar or a graph) and classification techniques such as parsing and graph matching are used Neural The model simulates the behavior of biological neural networks 4
5 Unsupervised Pattern Recognition The system must learn the classifier from unlabeled data It s related to the problem of trying to estimate the underlying probability density function of the data Approaches to unsupervised learning include clustering (e.g., k means, mixture models, hierarchical clustering) techniques for dimensionality reduction (e.g., principal component analysis, independent component analysis, non negative matrix factorization, singular value decomposition) 5
6 k means Clustering Given a set of n dimensional vectors Also specify k (number of desired clusters) The algorithm partitions the vectors into clusters, such that it minimizes the sum, over all clusters, of the within cluster sums of point to clustercentroid distances 6
7 k means Algorithm. Given a set of vectors {x i }. Randomly choose a set of k means {m i } as the center of each cluster 3. For each vector x i compute distance to each m i ; assign x i to the closest cluster 4. Update the means to get a new set of cluster centers 5. Repeat steps 3 and 4 until there is no more change in cluster centers k means is guaranteed to terminate, but may not find the global optimum in the least squares sense 7
8 Example: Indexed Storage of Color Images If an image uses 8 bits for each of R,G,B, there are ^4 possible colors Most images don t use the entire color space of possible values we can get by with fewer We ll use k means clustering to find the reduced set of colors Image using the full color space Image using only 3 discrete colors 8
9 Indexed Storage of Color Images For each image, we find a set of colors that are a good approximation of the entire set of pixels in the image; and put those into a colormap Then for each pixel, we just store the indices into the colormap Image of indices (0..63) Color image, using 64 colors 9
10 Indexed storage of Color Images Use a colormap Image f(x,y) stores indices into a lookup table (colormap) Colormap specifies RGB for each index Display system will display these values of RGB [img,cmap] = imread( kids.tif ); imshow(img,cmap); Also see rgbind, indrgb >> cmap(:0,:) ans = Index = : 0
11 clear all close all % Read image RGB = imdouble(imread('peppers.png')); RGB = imresize(rgb, 0.5); %RGB = imdouble(imread('pears.png')); RGB = imresize(rgb, 0.5); %RGB = imdouble(imread('tissue.png')); RGB = imresize(rgb, 0.5); figure, imshow(rgb); % Convert 3-dimensional array (M,N,3) array to D (MxN,3) X = reshape(rgb, [], 3); k=6; % Number of clusters to find % Call kmeans. It returns: % IDX: for each point in X, which cluster (..k) it was assigned to % C: the k cluster centers [IDX,C] = kmeans(x,k,... 'EmptyAction', 'drop'); % if a cluster becomes empty, drop it % Reshape the index array back to a -dimensional image I = reshape(idx, size(rgb,), size(rgb,)); % Show the reduced color image figure, imshow(i, C); % Plot pixels in color space figure hold on for i=:0:size(x,) plot3(x(i, ), X(i, ), X(i, 3),... '.', 'Color', C(IDX(i),:)); end % Also plot cluster centers for i=:k plot3(c(i,), C(i,), C(i,3), 'ro', 'MarkerFaceColor', 'r'); end xlabel('red'), ylabel('green'), zlabel('blue'); axis equal axis vis3d
12
13 Supervised Statistical Methods A class is a set of objects having some important properties in common We might have known description for each We might have set of samples for each A feature extractor is a program that inputs the data (image) and extracts features that can be used in classification; these values are put into a feature vector A classifier is a program that inputs the feature vector and assigns it to one of a set of designated classes or to the reject class We will look at these classifiers: Decision tree Nearest class mean Another powerful classifier is support vector machines 3
14 Feature Vector Representation A feature vector is a vector x=[x, x,, x n ], where each x j is a real number The elements x j may be object measurements For example, x j may be a count of object parts or properties Example: an object region can be represented by [#holes, #strokes, moments, ] from Shapiro & Stockman 4
15 Possible features for Char Recognition from Shapiro & Stockman 5
16 Discriminant functions Functions f(x, K) perform some computation on feature vector x Knowledge K from training or programming is used Final stage determines class from Shapiro & Stockman 6
17 Decision Trees Strength: Easy to understand Weakness: Overtraining from Shapiro & Stockman Class #holes #strokes Best axis Mom of inertia A 3 90 Med B 90 Large Med Large 0 90 Low W Large X 0? Large * 0 0? Large Low / 0 60 Low 7
18 Entropy Based Automatic Decision Tree Construction Training Set S x=(f,f, fm) x=(f,f, fm).. xn=(fn,f, fm) Node What feature should be used? What values? Choose the feature which results in the most information gain, as measured by the decrease in entropy from Shapiro & Stockman 8
19 Entropy Given a set of training vectors S, if there are c classes, Entropy(S) = -p i log (p i ) where p i is the proportion of category i examples in S. If all examples belong to the same category, the entropy is 0. If the examples are equally mixed (/c examples of each class), the entropy is a maximum at.0. c i= e.g. for c=, -.5 log log.5 = -.5(-) -.5(-) = from Shapiro & Stockman 9
20 Decision Tree Classifier Uses subsets of features in sequence Feature extraction may be interleaved with classification decisions Can be easy to design and efficient in execution from Shapiro & Stockman 0
21 Matlab demo See Decision Trees in Matlab help Statistics and Machine Learning toolbox Fisher's iris data consists of measurements on the sepal length, sepal width, petal length, and petal width of 50 iris specimens. There are 50 specimens from each of three species. clear all close all % Loads: % meas(50,4) - each row is a pattern (a 4-dimensional vector) % species{50} - each element is the name of a flower load fisheriris % Create a vector of class numbers. We know that the input data is grouped % so that..50 is the st class, is the nd class, is the % 3rd class. y(:50,) = ; % class 'setosa' y(5:00,) = ; % class 'versico' y(0:50,) = 3; % class 'virginica' X = meas(:, :); % just use first features (easier to visualize)
22 % We will just use the first features, since it is easier to visualize. % However, when we do that there is a chance that some points will be % duplicated (since we are ignoring the other features). If so, just keep % the first point. indicestokeep = true(size(x,),); for i=:size(x,) % See if we already have the ith point. if any((x(i,)==x(:i-,)) & (X(i,)==X(:i-,))) indicestokeep(i) = false; % Skip this point end end X = X(indicesToKeep, :); y = y(indicestokeep); % Plot the feature vectors. figure hold on plot(x(y==,), X(y==,), '*r'); plot(x(y==,), X(y==,), '*g'); plot(x(y==3,), X(y==3,), '*b'); xlabel('sepal length'), ylabel('sepal width'); Sepal width
23 Choose a test point to classify % Specify a test vector to classify. xtest = [5.6, 3.]; plot(xtest(), xtest(), 'ok'); % the black circle (k) is the test point hold off Sepal width 3
24 Construct a decision (or classification) tree % A "classification" tree produces classification decisions that are % "nominal" (ie, names). A "regression" tree produces classification % decisions that are numeric. ctree = ClassificationTree.fit(X,y,... 'MinParent', 0); % default is 0 view(ctree); % Prints a text description view(ctree, 'mode', 'graph'); % Draws a graphic description of the tree 4
25 Use decision tree to classify a vector In our example, x = 5.6 and x = 3. % Classify a test vector, using the decision tree. class = predict(ctree, xtest); fprintf('test vector is classified as %d\n', class); 5
26 View the whole feature space % Visualize entire feature space, and what class each vector belongs to. xmin = min(x(:,)); xmax = max(x(:,)); ymin = min(x(:,)); ymax = max(x(:,)); hold on; dx = (xmax-xmin)/40; dy = (ymax-ymin)/40; for x=xmin:dx:xmax for y=ymin:dy:ymax class = predict(ctree,... [x y]); if class== plot(x,y,'.r'); elseif class== plot(x,y,'.g'); else plot(x,y,'.b'); end end end hold off; Note that some training vectors are incorrectly classified 6
27 The input parameter MinParent has default value 0. Setting 'MinParent = will cause the decision tree to split (make a new node) if there are any instances that are still not correctly labeled. 'MinParent = 7
28 Generalization Making the tree accurately classify every single training point leads to overfitting. If you have lots of data, some training points may be noisy We don t want the tree to learn the noise We want the tree to generalize from the training data It should learn the general, underlying rules It s ok to misclassify a few training points 8
29 MinParent = Sepal width Sepal length
30 MinParent =
31 MinParent =
32 Classification using nearest class mean Compute the Euclidean distance between feature vector X and the mean of each class. x x [ i] x[ i x ] i, d Choose closest class, if close enough (reject otherwise) Low error rate (intersection) from Shapiro & Stockman 3
33 Scaling Distance Using Standard Deviations Scale distance to the mean of class c, according to the measured standard deviation i in each direction i x x c x[ i] xc[ i] / i Otherwise, a point near the top of class 3 will be closer to the class mean from Shapiro & Stockman 33
34 If ellipses are not aligned with axes Instead of using standard deviation along each separate axis, use the covariance matrix C Variance (of a single variable x) is defined as Covariance (of two variables, x and y) is x x x x x x x x x x x x x x x x o o o o o o o o o o o N i x i N yy yx xy xx C xy y i N i x i yx xy N i y i yy N i x i xx y x N y N x N 34
35 Examples C = C = Notes Off diagonal values are small if variables are independent Off diagonal values are large if variables are correlated (they vary together) Matlab cov function 35
36 Probability Density Let s assume that errors are Gaussian The probability density for an dimensional error vector x is T x x C x p exp C x x x - axis x - axis 3 36
37 Probability Density Look at where the probability is a constant. This is where the exponent is a constant: x This is the equation of an ellipse. T C x x For example, with uncorrelated errors this reduces to z x xx y yy z Can choose z to get desired probability. For z=3, the cumulative probability is about 97%. 37
38 Plotting Contours of constant probability 3 3 x - axis 0 x - axis x - axis x - axis 38
39 % Show covariance of two variables clear all close all Matlab code randn('state',0); yp = randn(40,); xp = 0.5 * randn(40,); % xp = randn(40,); % yp = xp + 0.5*randn(40,); plot(xp,yp, '+'), axis equal; axis([ ]); C = cov(xp,yp) Cinv = inv(c); detcsqrt = sqrt(det(c)); % Plot the probability density, % p(x,y) = (/(pi det(c)^0.5))exp(-x Cinv x/) L = 3.0; delta = 0.; [x,x] = meshgrid(-l:delta:l,-l:delta:l); for i=:size(x,) for j=:size(x,) x = [x(i,j); x(i,j)]; fx(i,j) = (/(*pi*detcsqrt)) * exp( -0.5*x'*Cinv*x ); end end hold on % meshc(x,x,fx); % this does a surface plot contour(x,x,fx); % this does a contour plot xlabel('x - axis'); ylabel('x - axis'); 39
40 Example flower data from Matlab % This loads in the measurements: % meas(n,4) are the feature values % species{n} are the species names load fisheriris; % There are three classes X = meas(strmatch('setosa', species), 3:4); % use features 3,4 X = meas(strmatch('virginica', species), 3:4); X3 = meas(strmatch('versicolor', species), 3:4); hold on plot( X(:,), X(:,), '.r' ); plot( X(:,), X(:,), '.g' ); plot( X3(:,), X3(:,), '.b' ); m = sum(x)/length(x); m = sum(x)/length(x); m3 = sum(x3)/length(x3); plot( m(), m(), '*r' ); plot( m(), m(), '*g' ); plot( m3(), m3(), '*b' );
41 Overlaying probability contours % Plot the contours of equal probability [f,f] = meshgrid( min(meas(:,3)):0.:max(meas(:,3)),... min(meas(:,4)):0.:max(meas(:,4)) ); C = cov(x); Cinv = inv(c); detcsqrt = sqrt(det(c)); for i=:size(f,) for j=:size(f,) x = [f(i,j) f(i,j)]; fx(i,j) = (/(*pi*detcsqrt)) * exp( -0.5*(x-m)*Cinv*(x-m)' ); end end contour(f,f,fx); C = cov(x); Cinv = inv(c); detcsqrt = sqrt(det(c)); for i=:size(f,) for j=:size(f,) x = [f(i,j) f(i,j)]; fx(i,j) = (/(*pi*detcsqrt)) * exp( -0.5*(x-m)*Cinv*(x-m)' ); end end contour(f,f,fx); C = cov(x3); Cinv = inv(c); detcsqrt = sqrt(det(c)); for i=:size(f,) for j=:size(f,) x = [f(i,j) f(i,j)]; fx(i,j) = (/(*pi*detcsqrt)) * exp( -0.5*(x-m3)*Cinv*(x-m3)' ); end end contour(f,f,fx); 4
42 Mahalanobis distance Given an unknown feature vector x, which class is it closest to? Assume you know the class centers (centroids) z i, and their covariances C i We find the class that has the smallest distance from its center to the point in feature space The distance is weighted by the covariance this is called the Mahalanobis distance For example, the Mahalanobis distance of feature vector x to the i th class is di T x z C x z i where C i is the covariance matrix of the feature vectors in the i th class i i 4
43 Summary / Questions In pattern recognition, we classify patterns (usually in the form of vectors) into classes. Training of the classifier can be supervised (i.e., we have to provide labeled training data) or unsupervised. k means clustering is an example of unsupervised learning Approaches to classification include Statistical Structural Neural Name some statistical pattern recognition methods. 43
Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Image Segmentation Some material for these slides comes from https://www.csd.uwo.ca/courses/cs4487a/
More informationIntroduction to Artificial Intelligence
Introduction to Artificial Intelligence COMP307 Machine Learning 2: 3-K Techniques Yi Mei yi.mei@ecs.vuw.ac.nz 1 Outline K-Nearest Neighbour method Classification (Supervised learning) Basic NN (1-NN)
More informationIntroduction to Pattern Recognition Part II. Selim Aksoy Bilkent University Department of Computer Engineering
Introduction to Pattern Recognition Part II Selim Aksoy Bilkent University Department of Computer Engineering saksoy@cs.bilkent.edu.tr RETINA Pattern Recognition Tutorial, Summer 2005 Overview Statistical
More informationSupervised vs unsupervised clustering
Classification Supervised vs unsupervised clustering Cluster analysis: Classes are not known a- priori. Classification: Classes are defined a-priori Sometimes called supervised clustering Extract useful
More informationBL5229: Data Analysis with Matlab Lab: Learning: Clustering
BL5229: Data Analysis with Matlab Lab: Learning: Clustering The following hands-on exercises were designed to teach you step by step how to perform and understand various clustering algorithm. We will
More informationColorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Neural Networks in MATLAB This is a good resource on Deep Learning for papers and code: https://github.com/kjw612/awesome
More informationComputational Statistics The basics of maximum likelihood estimation, Bayesian estimation, object recognitions
Computational Statistics The basics of maximum likelihood estimation, Bayesian estimation, object recognitions Thomas Giraud Simon Chabot October 12, 2013 Contents 1 Discriminant analysis 3 1.1 Main idea................................
More informationCluster Analysis. Mu-Chun Su. Department of Computer Science and Information Engineering National Central University 2003/3/11 1
Cluster Analysis Mu-Chun Su Department of Computer Science and Information Engineering National Central University 2003/3/11 1 Introduction Cluster analysis is the formal study of algorithms and methods
More informationClustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
More informationINF 4300 Classification III Anne Solberg The agenda today:
INF 4300 Classification III Anne Solberg 28.10.15 The agenda today: More on estimating classifier accuracy Curse of dimensionality and simple feature selection knn-classification K-means clustering 28.10.15
More informationData clustering & the k-means algorithm
April 27, 2016 Why clustering? Unsupervised Learning Underlying structure gain insight into data generate hypotheses detect anomalies identify features Natural classification e.g. biological organisms
More informationStatistical Methods in AI
Statistical Methods in AI Distance Based and Linear Classifiers Shrenik Lad, 200901097 INTRODUCTION : The aim of the project was to understand different types of classification algorithms by implementing
More informationUnsupervised Learning
Unsupervised Learning Unsupervised learning Until now, we have assumed our training samples are labeled by their category membership. Methods that use labeled samples are said to be supervised. However,
More informationCluster Analysis and Visualization. Workshop on Statistics and Machine Learning 2004/2/6
Cluster Analysis and Visualization Workshop on Statistics and Machine Learning 2004/2/6 Outlines Introduction Stages in Clustering Clustering Analysis and Visualization One/two-dimensional Data Histogram,
More informationStats 170A: Project in Data Science Exploratory Data Analysis: Clustering Algorithms
Stats 170A: Project in Data Science Exploratory Data Analysis: Clustering Algorithms Padhraic Smyth Department of Computer Science Bren School of Information and Computer Sciences University of California,
More informationAn Introduction to Cluster Analysis. Zhaoxia Yu Department of Statistics Vice Chair of Undergraduate Affairs
An Introduction to Cluster Analysis Zhaoxia Yu Department of Statistics Vice Chair of Undergraduate Affairs zhaoxia@ics.uci.edu 1 What can you say about the figure? signal C 0.0 0.5 1.0 1500 subjects Two
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationClustering & Classification (chapter 15)
Clustering & Classification (chapter 5) Kai Goebel Bill Cheetham RPI/GE Global Research goebel@cs.rpi.edu cheetham@cs.rpi.edu Outline k-means Fuzzy c-means Mountain Clustering knn Fuzzy knn Hierarchical
More informationK-means Clustering & PCA
K-means Clustering & PCA Andreas C. Kapourani (Credit: Hiroshi Shimodaira) 02 February 2018 1 Introduction In this lab session we will focus on K-means clustering and Principal Component Analysis (PCA).
More informationKTH ROYAL INSTITUTE OF TECHNOLOGY. Lecture 14 Machine Learning. K-means, knn
KTH ROYAL INSTITUTE OF TECHNOLOGY Lecture 14 Machine Learning. K-means, knn Contents K-means clustering K-Nearest Neighbour Power Systems Analysis An automated learning approach Understanding states in
More informationClustering CS 550: Machine Learning
Clustering CS 550: Machine Learning This slide set mainly uses the slides given in the following links: http://www-users.cs.umn.edu/~kumar/dmbook/ch8.pdf http://www-users.cs.umn.edu/~kumar/dmbook/dmslides/chap8_basic_cluster_analysis.pdf
More informationCluster Analysis for Microarray Data
Cluster Analysis for Microarray Data Seventh International Long Oligonucleotide Microarray Workshop Tucson, Arizona January 7-12, 2007 Dan Nettleton IOWA STATE UNIVERSITY 1 Clustering Group objects that
More informationDimension Reduction CS534
Dimension Reduction CS534 Why dimension reduction? High dimensionality large number of features E.g., documents represented by thousands of words, millions of bigrams Images represented by thousands of
More informationUnsupervised Learning : Clustering
Unsupervised Learning : Clustering Things to be Addressed Traditional Learning Models. Cluster Analysis K-means Clustering Algorithm Drawbacks of traditional clustering algorithms. Clustering as a complex
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationChapter 4. The Classification of Species and Colors of Finished Wooden Parts Using RBFNs
Chapter 4. The Classification of Species and Colors of Finished Wooden Parts Using RBFNs 4.1 Introduction In Chapter 1, an introduction was given to the species and color classification problem of kitchen
More informationData Mining - Data. Dr. Jean-Michel RICHER Dr. Jean-Michel RICHER Data Mining - Data 1 / 47
Data Mining - Data Dr. Jean-Michel RICHER 2018 jean-michel.richer@univ-angers.fr Dr. Jean-Michel RICHER Data Mining - Data 1 / 47 Outline 1. Introduction 2. Data preprocessing 3. CPA with R 4. Exercise
More informationJMP Book Descriptions
JMP Book Descriptions The collection of JMP documentation is available in the JMP Help > Books menu. This document describes each title to help you decide which book to explore. Each book title is linked
More informationRadial Basis Function (RBF) Neural Networks Based on the Triple Modular Redundancy Technology (TMR)
Radial Basis Function (RBF) Neural Networks Based on the Triple Modular Redundancy Technology (TMR) Yaobin Qin qinxx143@umn.edu Supervisor: Pro.lilja Department of Electrical and Computer Engineering Abstract
More informationStatistical Pattern Recognition
Statistical Pattern Recognition Features and Feature Selection Hamid R. Rabiee Jafar Muhammadi Spring 2013 http://ce.sharif.edu/courses/91-92/2/ce725-1/ Agenda Features and Patterns The Curse of Size and
More informationClassification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
More informationPerformance Measure of Hard c-means,fuzzy c-means and Alternative c-means Algorithms
Performance Measure of Hard c-means,fuzzy c-means and Alternative c-means Algorithms Binoda Nand Prasad*, Mohit Rathore**, Geeta Gupta***, Tarandeep Singh**** *Guru Gobind Singh Indraprastha University,
More informationCOSC160: Detection and Classification. Jeremy Bolton, PhD Assistant Teaching Professor
COSC160: Detection and Classification Jeremy Bolton, PhD Assistant Teaching Professor Outline I. Problem I. Strategies II. Features for training III. Using spatial information? IV. Reducing dimensionality
More informationStatistical Pattern Recognition
Statistical Pattern Recognition Features and Feature Selection Hamid R. Rabiee Jafar Muhammadi Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda Features and Patterns The Curse of Size and
More informationUnsupervised Learning
Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover
More informationNearest Neighbor Classification
Nearest Neighbor Classification Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms January 11, 2017 1 / 48 Outline 1 Administration 2 First learning algorithm: Nearest
More informationK-Means Clustering 3/3/17
K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data points. We want to find underlying structure in the data. Examples: Identify groups of similar data points. Clustering
More informationUsing Machine Learning to Optimize Storage Systems
Using Machine Learning to Optimize Storage Systems Dr. Kiran Gunnam 1 Outline 1. Overview 2. Building Flash Models using Logistic Regression. 3. Storage Object classification 4. Storage Allocation recommendation
More informationUniversity of Florida CISE department Gator Engineering. Clustering Part 2
Clustering Part 2 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville Partitional Clustering Original Points A Partitional Clustering Hierarchical
More informationRecognition Part I: Machine Learning. CSE 455 Linda Shapiro
Recognition Part I: Machine Learning CSE 455 Linda Shapiro Visual Recognition What does it mean to see? What is where, Marr 1982 Get computers to see Visual Recognition Verification Is this a car? Visual
More informationIBL and clustering. Relationship of IBL with CBR
IBL and clustering Distance based methods IBL and knn Clustering Distance based and hierarchical Probability-based Expectation Maximization (EM) Relationship of IBL with CBR + uses previously processed
More informationArtificial Intelligence. Programming Styles
Artificial Intelligence Intro to Machine Learning Programming Styles Standard CS: Explicitly program computer to do something Early AI: Derive a problem description (state) and use general algorithms to
More informationImage processing & Computer vision Xử lí ảnh và thị giác máy tính
Image processing & Computer vision Xử lí ảnh và thị giác máy tính Detection and Recognition 2D et 3D Alain Boucher - IFI Introduction In this chapter, we introduce some techniques for pattern detection
More informationEPL451: Data Mining on the Web Lab 5
EPL451: Data Mining on the Web Lab 5 Παύλος Αντωνίου Γραφείο: B109, ΘΕΕ01 University of Cyprus Department of Computer Science Predictive modeling techniques IBM reported in June 2012 that 90% of data available
More informationUnsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2008 CS 551, Spring 2008 c 2008, Selim Aksoy (Bilkent University)
More informationMATH5745 Multivariate Methods Lecture 13
MATH5745 Multivariate Methods Lecture 13 April 24, 2018 MATH5745 Multivariate Methods Lecture 13 April 24, 2018 1 / 33 Cluster analysis. Example: Fisher iris data Fisher (1936) 1 iris data consists of
More informationIntroduction to Machine Learning Prof. Anirban Santara Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur
Introduction to Machine Learning Prof. Anirban Santara Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture 14 Python Exercise on knn and PCA Hello everyone,
More informationClassification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
More informationComputer Vision Group Prof. Daniel Cremers. 6. Boosting
Prof. Daniel Cremers 6. Boosting Repetition: Regression We start with a set of basis functions (x) =( 0 (x), 1(x),..., M 1(x)) x 2 í d The goal is to fit a model into the data y(x, w) =w T (x) To do this,
More informationComputer Vision Group Prof. Daniel Cremers. 8. Boosting and Bagging
Prof. Daniel Cremers 8. Boosting and Bagging Repetition: Regression We start with a set of basis functions (x) =( 0 (x), 1(x),..., M 1(x)) x 2 í d The goal is to fit a model into the data y(x, w) =w T
More informationCIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.
More informationIntroduction to Machine Learning. Xiaojin Zhu
Introduction to Machine Learning Xiaojin Zhu jerryzhu@cs.wisc.edu Read Chapter 1 of this book: Xiaojin Zhu and Andrew B. Goldberg. Introduction to Semi- Supervised Learning. http://www.morganclaypool.com/doi/abs/10.2200/s00196ed1v01y200906aim006
More informationData Mining: Exploring Data. Lecture Notes for Chapter 3
Data Mining: Exploring Data Lecture Notes for Chapter 3 1 What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include
More informationHsiaochun Hsu Date: 12/12/15. Support Vector Machine With Data Reduction
Support Vector Machine With Data Reduction 1 Table of Contents Summary... 3 1. Introduction of Support Vector Machines... 3 1.1 Brief Introduction of Support Vector Machines... 3 1.2 SVM Simple Experiment...
More informationLast week. Multi-Frame Structure from Motion: Multi-View Stereo. Unknown camera viewpoints
Last week Multi-Frame Structure from Motion: Multi-View Stereo Unknown camera viewpoints Last week PCA Today Recognition Today Recognition Recognition problems What is it? Object detection Who is it? Recognizing
More informationFuzzy Segmentation. Chapter Introduction. 4.2 Unsupervised Clustering.
Chapter 4 Fuzzy Segmentation 4. Introduction. The segmentation of objects whose color-composition is not common represents a difficult task, due to the illumination and the appropriate threshold selection
More informationGrundlagen der Künstlichen Intelligenz
Grundlagen der Künstlichen Intelligenz Unsupervised learning Daniel Hennes 29.01.2018 (WS 2017/18) University Stuttgart - IPVS - Machine Learning & Robotics 1 Today Supervised learning Regression (linear
More informationGene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
More informationNote Set 4: Finite Mixture Models and the EM Algorithm
Note Set 4: Finite Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine Finite Mixture Models A finite mixture model with K components, for
More informationThe exam is closed book, closed notes except your one-page cheat sheet.
CS 189 Fall 2015 Introduction to Machine Learning Final Please do not turn over the page before you are instructed to do so. You have 2 hours and 50 minutes. Please write your initials on the top-right
More informationData Exploration and Preparation Data Mining and Text Mining (UIC Politecnico di Milano)
Data Exploration and Preparation Data Mining and Text Mining (UIC 583 @ Politecnico di Milano) References Jiawei Han and Micheline Kamber, "Data Mining, : Concepts and Techniques", The Morgan Kaufmann
More informationUnsupervised Data Mining: Clustering. Izabela Moise, Evangelos Pournaras, Dirk Helbing
Unsupervised Data Mining: Clustering Izabela Moise, Evangelos Pournaras, Dirk Helbing Izabela Moise, Evangelos Pournaras, Dirk Helbing 1 1. Supervised Data Mining Classification Regression Outlier detection
More informationRegion-based Segmentation
Region-based Segmentation Image Segmentation Group similar components (such as, pixels in an image, image frames in a video) to obtain a compact representation. Applications: Finding tumors, veins, etc.
More informationEnsemble Methods, Decision Trees
CS 1675: Intro to Machine Learning Ensemble Methods, Decision Trees Prof. Adriana Kovashka University of Pittsburgh November 13, 2018 Plan for This Lecture Ensemble methods: introduction Boosting Algorithm
More informationCS 1675 Introduction to Machine Learning Lecture 18. Clustering. Clustering. Groups together similar instances in the data sample
CS 1675 Introduction to Machine Learning Lecture 18 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem:
More informationNetwork Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Introduction Often, we have only a set of features x = x 1, x 2,, x n, but no associated response y. Therefore we are not interested in prediction nor classification,
More informationInterpolation is a basic tool used extensively in tasks such as zooming, shrinking, rotating, and geometric corrections.
Image Interpolation 48 Interpolation is a basic tool used extensively in tasks such as zooming, shrinking, rotating, and geometric corrections. Fundamentally, interpolation is the process of using known
More informationStatistical Pattern Recognition
Statistical Pattern Recognition Features and Feature Selection Hamid R. Rabiee Jafar Muhammadi Spring 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Agenda Features and Patterns The Curse of Size and
More informationCS 195-5: Machine Learning Problem Set 5
CS 195-5: Machine Learning Problem Set 5 Douglas Lanman dlanman@brown.edu 26 November 26 1 Clustering and Vector Quantization Problem 1 Part 1: In this problem we will apply Vector Quantization (VQ) to
More informationData Mining: Exploring Data. Lecture Notes for Chapter 3. Introduction to Data Mining
Data Mining: Exploring Data Lecture Notes for Chapter 3 Introduction to Data Mining by Tan, Steinbach, Kumar What is data exploration? A preliminary exploration of the data to better understand its characteristics.
More informationSupervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
More informationStatistical Analysis of Metabolomics Data. Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte
Statistical Analysis of Metabolomics Data Xiuxia Du Department of Bioinformatics & Genomics University of North Carolina at Charlotte Outline Introduction Data pre-treatment 1. Normalization 2. Centering,
More informationUnsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
More informationClustering analysis of gene expression data
Clustering analysis of gene expression data Chapter 11 in Jonathan Pevsner, Bioinformatics and Functional Genomics, 3 rd edition (Chapter 9 in 2 nd edition) Human T cell expression data The matrix contains
More informationData Mining: Exploring Data. Lecture Notes for Data Exploration Chapter. Introduction to Data Mining
Data Mining: Exploring Data Lecture Notes for Data Exploration Chapter Introduction to Data Mining by Tan, Steinbach, Karpatne, Kumar 02/03/2018 Introduction to Data Mining 1 What is data exploration?
More informationLab # 2 - ACS I Part I - DATA COMPRESSION in IMAGE PROCESSING using SVD
Lab # 2 - ACS I Part I - DATA COMPRESSION in IMAGE PROCESSING using SVD Goals. The goal of the first part of this lab is to demonstrate how the SVD can be used to remove redundancies in data; in this example
More informationMachine Learning and Data Mining. Clustering (1): Basics. Kalev Kask
Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of
More informationHard clustering. Each object is assigned to one and only one cluster. Hierarchical clustering is usually hard. Soft (fuzzy) clustering
An unsupervised machine learning problem Grouping a set of objects in such a way that objects in the same group (a cluster) are more similar (in some sense or another) to each other than to those in other
More informationInput: Concepts, Instances, Attributes
Input: Concepts, Instances, Attributes 1 Terminology Components of the input: Concepts: kinds of things that can be learned aim: intelligible and operational concept description Instances: the individual,
More informationLecture 4 Face Detection and Classification. Lin ZHANG, PhD School of Software Engineering Tongji University Spring 2018
Lecture 4 Face Detection and Classification Lin ZHANG, PhD School of Software Engineering Tongji University Spring 2018 Any faces contained in the image? Who are they? Outline Overview Face detection Introduction
More informationWhat to come. There will be a few more topics we will cover on supervised learning
Summary so far Supervised learning learn to predict Continuous target regression; Categorical target classification Linear Regression Classification Discriminative models Perceptron (linear) Logistic regression
More informationUnsupervised learning
Unsupervised learning Enrique Muñoz Ballester Dipartimento di Informatica via Bramante 65, 26013 Crema (CR), Italy enrique.munoz@unimi.it Enrique Muñoz Ballester 2017 1 Download slides data and scripts:
More informationRemote Sensing & Photogrammetry W4. Beata Hejmanowska Building C4, room 212, phone:
Remote Sensing & Photogrammetry W4 Beata Hejmanowska Building C4, room 212, phone: +4812 617 22 72 605 061 510 galia@agh.edu.pl 1 General procedures in image classification Conventional multispectral classification
More informationUnsupervised Learning
Outline Unsupervised Learning Basic concepts K-means algorithm Representation of clusters Hierarchical clustering Distance functions Which clustering algorithm to use? NN Supervised learning vs. unsupervised
More information10/14/2017. Dejan Sarka. Anomaly Detection. Sponsors
Dejan Sarka Anomaly Detection Sponsors About me SQL Server MVP (17 years) and MCT (20 years) 25 years working with SQL Server Authoring 16 th book Authoring many courses, articles Agenda Introduction Simple
More informationUnsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)
More informationLarge Scale Data Analysis Using Deep Learning
Large Scale Data Analysis Using Deep Learning Machine Learning Basics - 1 U Kang Seoul National University U Kang 1 In This Lecture Overview of Machine Learning Capacity, overfitting, and underfitting
More informationLecture-17: Clustering with K-Means (Contd: DT + Random Forest)
Lecture-17: Clustering with K-Means (Contd: DT + Random Forest) Medha Vidyotma April 24, 2018 1 Contd. Random Forest For Example, if there are 50 scholars who take the measurement of the length of the
More informationContents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation
Contents Machine Learning concepts 4 Learning Algorithm 4 Predictive Model (Model) 4 Model, Classification 4 Model, Regression 4 Representation Learning 4 Supervised Learning 4 Unsupervised Learning 4
More informationFinding Clusters 1 / 60
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-means Density Based Clustering, e.g. DBScan Grid Based Clustering 1 / 60
More informationData Informatics. Seon Ho Kim, Ph.D.
Data Informatics Seon Ho Kim, Ph.D. seonkim@usc.edu Clustering Overview Supervised vs. Unsupervised Learning Supervised learning (classification) Supervision: The training data (observations, measurements,
More informationAn Introduction to PDF Estimation and Clustering
Sigmedia, Electronic Engineering Dept., Trinity College, Dublin. 1 An Introduction to PDF Estimation and Clustering David Corrigan corrigad@tcd.ie Electrical and Electronic Engineering Dept., University
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationCSCI567 Machine Learning (Fall 2014)
CSCI567 Machine Learning (Fall 2014) Drs. Sha & Liu {feisha,yanliu.cs}@usc.edu September 9, 2014 Drs. Sha & Liu ({feisha,yanliu.cs}@usc.edu) CSCI567 Machine Learning (Fall 2014) September 9, 2014 1 / 47
More informationIntro to Artificial Intelligence
Intro to Artificial Intelligence Ahmed Sallam { Lecture 5: Machine Learning ://. } ://.. 2 Review Probabilistic inference Enumeration Approximate inference 3 Today What is machine learning? Supervised
More informationMachine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves
Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves
More informationLearning to Learn: additional notes
MASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science 6.034 Artificial Intelligence, Fall 2008 Recitation October 23 Learning to Learn: additional notes Bob Berwick
More informationClustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More informationDATA MINING INTRODUCTION TO CLASSIFICATION USING LINEAR CLASSIFIERS
DATA MINING INTRODUCTION TO CLASSIFICATION USING LINEAR CLASSIFIERS 1 Classification: Definition Given a collection of records (training set ) Each record contains a set of attributes and a class attribute
More information