Segmentation: Clustering, Graph Cut and EM
|
|
- Madison Alexander
- 6 years ago
- Views:
Transcription
1 Segmentation: Clustering, Graph Cut and EM Ying Wu Electrical Engineering and Computer Science Northwestern University, Evanston, IL /29
2 Outline Motivations and Applications Image Segmentation by Clustering K-Means Algorithm Self-Organizing Map Image Segmentation by Graph Cut Basic Idea Block-diagonalization Segmentation by Expectation-Maximization Missing Data Problem E-M iteration Issues Remained 2/29
3 Segmentation is a Fundamental Problem To group up similar components such as image pixels, image regions or even video clips. It is an ill-posed problem! How do we define the similarity measurement? 3/29
4 Background Subtraction Video surveillance applications Separate the foreground from background Assume fixed camera Subtract background from images Adaptive scheme by selecting w a, w i and w c. B n+1 = w af + i w ib n i w c 4/29
5 Object Modeling Represent an object by regions First step for recognition Represent a scene by a set of layers Motion segmentation Image segmentation v.s. motion segmentation 5/29
6 Basic Approaches Segmentation by clustering Segmentation by graph cut Segmentation by EM algorithm 6/29
7 Outline Motivations and Applications Image Segmentation by Clustering K-Means Algorithm Self-Organizing Map Image Segmentation by Graph Cut Basic Idea Block-diagonalization Segmentation by Expectation-Maximization Missing Data Problem E-M iteration Issues Remained 7/29
8 K-Means Clustering Assume the number of clusters, K, is given. Use the center of each clusters C i to represent each cluster. How do we determine the identity of a data point? Need to define a distance measurement, D(x, y). e.g., D(x, y) = x y 2. Winner takes all: l k (x k ) = arg min D(x k, C i ) = arg min x k C i 2 i i where l k is the label for the data point x k. K-means finds the clusters to minimize the total distortion. φ(x, C) = x j C i 2 i C j i th cluster 8/29
9 K-Means Clustering To minimize φ, K-means algorithm iterates between two steps: Labelling: assume the p-th iteration ends up with a set of cluster centers C (p) i, i = 1,...,K. We label each data point based on such a set of cluster centers, i.e., x k, find l (p+1) k (x k ) = min x k C (p) i i 2 and group data points belong to the same cluster Ω j = {x k : l k (x k ) = C j } Re-centering: re-calculating the centers: C (p+1) x i = k Ω i x k Ω i Iterates between labelling and re-centering until it converges. 9/29
10 Self-Organizing Map (SOM) SOM can be used for visualizing high-dim data Map to a low-dim space based on competitive learning A two-layer neural network outputs ξ 1 ξ 2 ξ m-1 ξ m weights x 1 x 2 x 3 inputs The # of neuron in the input layer is the same as the dimension of the input vector. Connection weights W k for each output neuron. 10/29
11 Competitive Learning For an input x, all neurons compete against each other The winner is the one whose weight is the closest to the input: y i = arg min i D(x i, W i ) The index of the winner is taken as the output of SOM. Adjust the weight of the winner Train the neurons nearby, and counter-train those far away. A window function Λ( y yk ) and the Hebbian learning rule: W(t + 1) = W(t) + η(t)λ( y y k )(x k W y k (t)) Intuition: the input data point will attract the neuron inside the window to its location, but push those neuron outside the window far away. Relation to vector quantization (VQ) and K-means clustering? 11/29
12 Outline Motivations and Applications Image Segmentation by Clustering K-Means Algorithm Self-Organizing Map Image Segmentation by Graph Cut Basic Idea Block-diagonalization Segmentation by Expectation-Maximization Missing Data Problem E-M iteration Issues Remained 12/29
13 Adjacency Graph and Affinity Matrix We can represent the set of data {x 1,...,x N } by a graph G = {V,E} Each vertex represents an individual data point Each edge represents the adjacency of two data points And the weight of the edge represents the affinity of the two points For example A ij = exp { x i x j 2 } 2σ 2 i.e., the similarity of two points Thus, the data set can be viewed as a weighted adjacency graph More importantly, it can also be viewed as an affinity matrix A 13/29
14 Block-diagonalization: Idea If the data are grouped, then the affinity matrix is pretty much block-diagonalized Now, clustering can be treated as the task of finding the best re-permutation to block-diagonalize A More specifically, the summation of the affinity values of those off-diagonal block matrices is minimized or the sum of diagonal block matrices is maximized 14/29
15 Block-diagonalization: Formulation Introduce an association vector (i.e., a projection) for each cluster component w k, w k = w k1 w k2. w kn where w ki is the association of x i to the cluster k. Positive w ki indicates that x i is in cluster k to some extent, and negative otherwise Usually, such projection vector is normalized, i.e., we have: w T k w k = 1, Now we can formulate the problem as w k = arg max w k k = 1,...,K s.t. w T k w k = 1 w T k Aw k 15/29
16 Spectral Analysis The solution is easy The Lagrangian L = wk T Aw k + λ(1 wk T w k) It is clear that L w k = 2Aw k 2λw k = 0 Aw k = λw k What is this! w k, an eigenvector, indicates the association of data with cluster k The size of the cluster is given by the eigenvalue λ More significantly, we don t need to know K in advance! The significant λs tell K 16/29
17 A Problem Ideally, we can check the values of w ki for grouping But life is always complicated Suppose A has two identical eigenvalues Aw 1 = λw 1, and Aw 2 = λw 2 It is easy to see any linear combination of w 1 and w 2 also gives a valid eigenvector A(a 1 w 1 + a 2 w 2 ) = λ(a 1 w 1 + a 2 w 2 ) This means that we cannot simply use the values of w = a 1 w 1 + a 2 w 2 for grouping now Instead of using the 1-D subspace, we need to go to the 2-D subspace spanned by {w 1,w 2 } If all the K clusters are more or less of the same size, we ll have K similar eigenvalues. Then we have to go to K-d subspace. This is the worse case. 17/29
18 Graph Cut We may view the problem from another point of view: graph cut We still represent the data set by the affinity graph Suppose we want to divide the data set into two clusters, we need to find the set of weakest links between the subgraphs, each of which corresponds to one cluster A set of edges in a graph is called a cut Now, we need to find a minimum cut for the weakest links But we have singularity here: the separation of the isolated vertex gives the minimum cut In other words, the cut does not balance the sizes of the clusters 18/29
19 Normalized Cut So, the cut needs to be normalized. Suppose we partition V into A and B. z { 1, 1} N is the indicator. z i = 1 if x i in A, and -1 otherwise. Let d i = j A ij be the total connection from x i to all others Define normalized cut NCut(A, B) = = cut(a, B) cut(b, A) + asso(a, V) asso(b, V) A ij x i x j x i >0,x j <0 + d i x i >0 x i <0,x j >0 x i <0 A ij x i x j Denote x D = diag{d 1,...,d N }, k = i >0 d i i d, b = k i 1 k d i 19/29
20 Normalized Cut Define y = (1 + x) b(1 x) Shi & Malik (1997) gave a nice formulation 1 y T (D A)y minncut(x) = min x y y T Dy { yi {1, b} s.t. y T D1 = 0 This is to solve a generalized EVD under constraints (D A)y = λdy The showed that the eigenvector associated with the 2nd smallest eigenvalue is able to bipartite the graph 1 J. Shi and J. Malik, Normalized Cuts and Image Segmentation, CVPR 97 20/29
21 Outline Motivations and Applications Image Segmentation by Clustering K-Means Algorithm Self-Organizing Map Image Segmentation by Graph Cut Basic Idea Block-diagonalization Segmentation by Expectation-Maximization Missing Data Problem E-M iteration Issues Remained 21/29
22 Generative Model and Missing Data Assume each image pixel is produced by a probability density associated with one of the g image segments. The data generation process: we first choose an image segment, and then generate the pixel based on: p(x) = i p(x θ i )π i where π is the prior for the i-th image segment, and θ i is the parameter. We can use Gaussian for each component: p(x θ i ) G(µ i, Σ i ) Associate a label l k for each x k for its identity This mixture model is a generative model. The data labels are missing. 22/29
23 Formulation So, our task is to do the inverse. Given a set of data point (image pixels) X = {x k, k = 1,...,N}, we need to estimate those parameters θ i, π i, and estimating the labels for all the data points by: lj = arg maxp(l j = k x j, Θ), x j k which gives the posteriori probability of x j. Maximum Likelihood Estimation. The likelihood of the data set can be written by: p(x Θ) = g ( p(x j θ i )π i ) j Usually, we use log likelihood: log p(x Θ) = j i=1 log( g p(x j θ i )π i ) But this is very ugly (why?) and intractable! 23/29 i=1
24 Missing Data and Indicator Variable Introduce an indicator variable z: z 1 z 2 z =. If a data point x is drawn from the k-th component, then z k = 1, and all other z i k = 0. This indicator variable tells the identity of a data point It is the missing part! Why do we need it? z g 24/29
25 Good News! Let s form the complete data: y k = [ xk And the complete data set is Y = {y k, k = 1,...,N}. The likelihood of the complete data point y k : g p(y k ) Θ) = z ki p(x k θ i ) log p(y k Θ) = i=1 z k ] g z ki log p(x k θ i ) i=1 So, for the whole data set, we have N g p(y Θ) = z ki p(x k θ i ) k=1 i=1 25/29
26 Good News and Bad News And thus: log p(y Θ) = = N g log( z ki p(x k θ i )) k=1 N k=1 i=1 i=1 g z ki log p(x k θ i ) Because we eliminate the summation terms inside log, the ML estimation becomes easier: Θ = arg maxlog p(y Θ) Θ However, the bad news is that the indicator variable z k make the ML difficult, since we do not know z k. 26/29
27 Expectation-Maximization Iteration Fortunately, life won t be too bad. A quite interesting phenomenon: if we know such zk, i.e., we know the identities for each data point, we can easily estimate the density parameters Θ based on ML, without any doubt. At the same time, if we know the density parameters, we can easily solve such indicator variables z k based on MAP. This phenomenon suggest an iterative procedure: E-step: computing an expected value of the complete data, here only E[z k ]; M-step: maximizing the the log likelihood of the complete data to estimate Θ. It converges to a local maximum of the likelihood. 27/29
28 EM for Image Segmentation let s apply EM to image segmentation: E-step: E[z ki ] = 1 p(kth pixel comes from ith component) M-step + 0 p(kth pixel doesn t come from ith component) = p(kth pixel comes from ith component) π i p(x k θ i ) = g j=1 π jp(x k θ j ) π i = 1 r µ i = r p(i x l,θ) l=1 r l=1 x lp(i x l,θ) r l=1 p(i x l,θ) Σ i = r l=1 p(i x l,θ)[(x l µ i )(x l µ i ) T ] r l=1 p(i x l,θ) 28/29
29 Issues Remained Structural parameters EM assumes a known number of components A common problem in clustering What if we don t know it? Minimum Description Length (MDL) principle in theory Cross-validation in practice Curse of dimensionality What if the dimensionality of x is very high? Too many parameters to estimate Requires a huge amount of training data Otherwise, the estimation is heavily biased 29/29
CS 534: Computer Vision Segmentation II Graph Cuts and Image Segmentation
CS 534: Computer Vision Segmentation II Graph Cuts and Image Segmentation Spring 2005 Ahmed Elgammal Dept of Computer Science CS 534 Segmentation II - 1 Outlines What is Graph cuts Graph-based clustering
More information6.801/866. Segmentation and Line Fitting. T. Darrell
6.801/866 Segmentation and Line Fitting T. Darrell Segmentation and Line Fitting Gestalt grouping Background subtraction K-Means Graph cuts Hough transform Iterative fitting (Next time: Probabilistic segmentation)
More informationThe goals of segmentation
Image segmentation The goals of segmentation Group together similar-looking pixels for efficiency of further processing Bottom-up process Unsupervised superpixels X. Ren and J. Malik. Learning a classification
More informationSGN (4 cr) Chapter 11
SGN-41006 (4 cr) Chapter 11 Clustering Jussi Tohka & Jari Niemi Department of Signal Processing Tampere University of Technology February 25, 2014 J. Tohka & J. Niemi (TUT-SGN) SGN-41006 (4 cr) Chapter
More informationImage Segmentation continued Graph Based Methods
Image Segmentation continued Graph Based Methods Previously Images as graphs Fully-connected graph node (vertex) for every pixel link between every pair of pixels, p,q affinity weight w pq for each link
More informationUnsupervised Learning
Networks for Pattern Recognition, 2014 Networks for Single Linkage K-Means Soft DBSCAN PCA Networks for Kohonen Maps Linear Vector Quantization Networks for Problems/Approaches in Machine Learning Supervised
More informationTargil 12 : Image Segmentation. Image segmentation. Why do we need it? Image segmentation
Targil : Image Segmentation Image segmentation Many slides from Steve Seitz Segment region of the image which: elongs to a single object. Looks uniform (gray levels, color ) Have the same attributes (texture
More informationVisual Representations for Machine Learning
Visual Representations for Machine Learning Spectral Clustering and Channel Representations Lecture 1 Spectral Clustering: introduction and confusion Michael Felsberg Klas Nordberg The Spectral Clustering
More informationLecture 11: E-M and MeanShift. CAP 5415 Fall 2007
Lecture 11: E-M and MeanShift CAP 5415 Fall 2007 Review on Segmentation by Clustering Each Pixel Data Vector Example (From Comanciu and Meer) Review of k-means Let's find three clusters in this data These
More informationCS Introduction to Data Mining Instructor: Abdullah Mueen
CS 591.03 Introduction to Data Mining Instructor: Abdullah Mueen LECTURE 8: ADVANCED CLUSTERING (FUZZY AND CO -CLUSTERING) Review: Basic Cluster Analysis Methods (Chap. 10) Cluster Analysis: Basic Concepts
More informationPattern Recognition. Kjell Elenius. Speech, Music and Hearing KTH. March 29, 2007 Speech recognition
Pattern Recognition Kjell Elenius Speech, Music and Hearing KTH March 29, 2007 Speech recognition 2007 1 Ch 4. Pattern Recognition 1(3) Bayes Decision Theory Minimum-Error-Rate Decision Rules Discriminant
More informationBig Data Analytics. Special Topics for Computer Science CSE CSE Feb 11
Big Data Analytics Special Topics for Computer Science CSE 4095-001 CSE 5095-005 Feb 11 Fei Wang Associate Professor Department of Computer Science and Engineering fei_wang@uconn.edu Clustering II Spectral
More informationClustering. So far in the course. Clustering. Clustering. Subhransu Maji. CMPSCI 689: Machine Learning. dist(x, y) = x y 2 2
So far in the course Clustering Subhransu Maji : Machine Learning 2 April 2015 7 April 2015 Supervised learning: learning with a teacher You had training data which was (feature, label) pairs and the goal
More informationClustering: Classic Methods and Modern Views
Clustering: Classic Methods and Modern Views Marina Meilă University of Washington mmp@stat.washington.edu June 22, 2015 Lorentz Center Workshop on Clusters, Games and Axioms Outline Paradigms for clustering
More informationClustering. Subhransu Maji. CMPSCI 689: Machine Learning. 2 April April 2015
Clustering Subhransu Maji CMPSCI 689: Machine Learning 2 April 2015 7 April 2015 So far in the course Supervised learning: learning with a teacher You had training data which was (feature, label) pairs
More informationCS 534: Computer Vision Segmentation and Perceptual Grouping
CS 534: Computer Vision Segmentation and Perceptual Grouping Ahmed Elgammal Dept of Computer Science CS 534 Segmentation - 1 Outlines Mid-level vision What is segmentation Perceptual Grouping Segmentation
More informationLecture 7: Segmentation. Thursday, Sept 20
Lecture 7: Segmentation Thursday, Sept 20 Outline Why segmentation? Gestalt properties, fun illusions and/or revealing examples Clustering Hierarchical K-means Mean Shift Graph-theoretic Normalized cuts
More informationSegmentation and low-level grouping.
Segmentation and low-level grouping. Bill Freeman, MIT 6.869 April 14, 2005 Readings: Mean shift paper and background segmentation paper. Mean shift IEEE PAMI paper by Comanici and Meer, http://www.caip.rutgers.edu/~comanici/papers/msrobustapproach.pdf
More informationAnnouncements. Image Segmentation. From images to objects. Extracting objects. Status reports next Thursday ~5min presentations in class
Image Segmentation Announcements Status reports next Thursday ~5min presentations in class Project voting From Sandlot Science Today s Readings Forsyth & Ponce, Chapter 1 (plus lots of optional references
More informationFitting D.A. Forsyth, CS 543
Fitting D.A. Forsyth, CS 543 Fitting Choose a parametric object/some objects to represent a set of tokens Most interesting case is when criterion is not local can t tell whether a set of points lies on
More informationClustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
More informationCS325 Artificial Intelligence Ch. 20 Unsupervised Machine Learning
CS325 Artificial Intelligence Cengiz Spring 2013 Unsupervised Learning Missing teacher No labels, y Just input data, x What can you learn with it? Unsupervised Learning Missing teacher No labels, y Just
More informationSegmentation. Bottom Up Segmentation
Segmentation Bottom up Segmentation Semantic Segmentation Bottom Up Segmentation 1 Segmentation as clustering Depending on what we choose as the feature space, we can group pixels in different ways. Grouping
More informationNon-exhaustive, Overlapping k-means
Non-exhaustive, Overlapping k-means J. J. Whang, I. S. Dhilon, and D. F. Gleich Teresa Lebair University of Maryland, Baltimore County October 29th, 2015 Teresa Lebair UMBC 1/38 Outline Introduction NEO-K-Means
More informationMixture Models and the EM Algorithm
Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine c 2017 1 Finite Mixture Models Say we have a data set D = {x 1,..., x N } where x i is
More informationCSCI-B609: A Theorist s Toolkit, Fall 2016 Sept. 6, Firstly let s consider a real world problem: community detection.
CSCI-B609: A Theorist s Toolkit, Fall 016 Sept. 6, 016 Lecture 03: The Sparsest Cut Problem and Cheeger s Inequality Lecturer: Yuan Zhou Scribe: Xuan Dong We will continue studying the spectral graph theory
More informationCS 664 Slides #11 Image Segmentation. Prof. Dan Huttenlocher Fall 2003
CS 664 Slides #11 Image Segmentation Prof. Dan Huttenlocher Fall 2003 Image Segmentation Find regions of image that are coherent Dual of edge detection Regions vs. boundaries Related to clustering problems
More informationIntroduction to Machine Learning
Introduction to Machine Learning Clustering Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB CSE 474/574 1 / 19 Outline
More informationUnsupervised Learning
Unsupervised Learning Learning without Class Labels (or correct outputs) Density Estimation Learn P(X) given training data for X Clustering Partition data into clusters Dimensionality Reduction Discover
More information( ) =cov X Y = W PRINCIPAL COMPONENT ANALYSIS. Eigenvectors of the covariance matrix are the principal components
Review Lecture 14 ! PRINCIPAL COMPONENT ANALYSIS Eigenvectors of the covariance matrix are the principal components 1. =cov X Top K principal components are the eigenvectors with K largest eigenvalues
More informationSpectral Clustering on Handwritten Digits Database
October 6, 2015 Spectral Clustering on Handwritten Digits Database Danielle dmiddle1@math.umd.edu Advisor: Kasso Okoudjou kasso@umd.edu Department of Mathematics University of Maryland- College Park Advance
More informationImage Segmentation continued Graph Based Methods. Some slides: courtesy of O. Capms, Penn State, J.Ponce and D. Fortsyth, Computer Vision Book
Image Segmentation continued Graph Based Methods Some slides: courtesy of O. Capms, Penn State, J.Ponce and D. Fortsyth, Computer Vision Book Previously Binary segmentation Segmentation by thresholding
More informationRandom projection for non-gaussian mixture models
Random projection for non-gaussian mixture models Győző Gidófalvi Department of Computer Science and Engineering University of California, San Diego La Jolla, CA 92037 gyozo@cs.ucsd.edu Abstract Recently,
More informationMultiple cosegmentation
Armand Joulin, Francis Bach and Jean Ponce. INRIA -Ecole Normale Supérieure April 25, 2012 Segmentation Introduction Segmentation Supervised and weakly-supervised segmentation Cosegmentation Segmentation
More informationClustering K-means. Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, Carlos Guestrin
Clustering K-means Machine Learning CSEP546 Carlos Guestrin University of Washington February 18, 2014 Carlos Guestrin 2005-2014 1 Clustering images Set of Images [Goldberger et al.] Carlos Guestrin 2005-2014
More informationMachine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme, Nicolas Schilling
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme, Nicolas Schilling Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim,
More informationMultiple Model Estimation : The EM Algorithm & Applications
Multiple Model Estimation : The EM Algorithm & Applications Princeton University COS 429 Lecture Dec. 4, 2008 Harpreet S. Sawhney hsawhney@sarnoff.com Plan IBR / Rendering applications of motion / pose
More informationNormalized cuts and image segmentation
Normalized cuts and image segmentation Department of EE University of Washington Yeping Su Xiaodan Song Normalized Cuts and Image Segmentation, IEEE Trans. PAMI, August 2000 5/20/2003 1 Outline 1. Image
More informationEE 701 ROBOT VISION. Segmentation
EE 701 ROBOT VISION Regions and Image Segmentation Histogram-based Segmentation Automatic Thresholding K-means Clustering Spatial Coherence Merging and Splitting Graph Theoretic Segmentation Region Growing
More informationNote Set 4: Finite Mixture Models and the EM Algorithm
Note Set 4: Finite Mixture Models and the EM Algorithm Padhraic Smyth, Department of Computer Science University of California, Irvine Finite Mixture Models A finite mixture model with K components, for
More informationCS839: Probabilistic Graphical Models. Lecture 10: Learning with Partially Observed Data. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 10: Learning with Partially Observed Data Theo Rekatsinas 1 Partially Observed GMs Speech recognition 2 Partially Observed GMs Evolution 3 Partially Observed
More informationClustering. Department Biosysteme Karsten Borgwardt Data Mining Course Basel Fall Semester / 238
Clustering Department Biosysteme Karsten Borgwardt Data Mining Course Basel Fall Semester 2015 163 / 238 What is Clustering? Department Biosysteme Karsten Borgwardt Data Mining Course Basel Fall Semester
More informationMachine Learning for OR & FE
Machine Learning for OR & FE Unsupervised Learning: Clustering Martin Haugh Department of Industrial Engineering and Operations Research Columbia University Email: martin.b.haugh@gmail.com (Some material
More informationK-means and Hierarchical Clustering
K-means and Hierarchical Clustering Xiaohui Xie University of California, Irvine K-means and Hierarchical Clustering p.1/18 Clustering Given n data points X = {x 1, x 2,, x n }. Clustering is the partitioning
More informationhuman vision: grouping k-means clustering graph-theoretic clustering Hough transform line fitting RANSAC
COS 429: COMPUTER VISON Segmentation human vision: grouping k-means clustering graph-theoretic clustering Hough transform line fitting RANSAC Reading: Chapters 14, 15 Some of the slides are credited to:
More informationContent-based image and video analysis. Machine learning
Content-based image and video analysis Machine learning for multimedia retrieval 04.05.2009 What is machine learning? Some problems are very hard to solve by writing a computer program by hand Almost all
More informationLecture 8: The EM algorithm
10-708: Probabilistic Graphical Models 10-708, Spring 2017 Lecture 8: The EM algorithm Lecturer: Manuela M. Veloso, Eric P. Xing Scribes: Huiting Liu, Yifan Yang 1 Introduction Previous lecture discusses
More informationIntroduction to spectral clustering
Introduction to spectral clustering Denis Hamad LASL ULCO Denis.Hamad@lasl.univ-littoral.fr Philippe Biela HEI LAGIS Philippe.Biela@hei.fr Data Clustering Data clustering Data clustering is an important
More informationImage Analysis - Lecture 5
Texture Segmentation Clustering Review Image Analysis - Lecture 5 Texture and Segmentation Magnus Oskarsson Lecture 5 Texture Segmentation Clustering Review Contents Texture Textons Filter Banks Gabor
More informationIMAGE RESTORATION VIA EFFICIENT GAUSSIAN MIXTURE MODEL LEARNING
IMAGE RESTORATION VIA EFFICIENT GAUSSIAN MIXTURE MODEL LEARNING Jianzhou Feng Li Song Xiaog Huo Xiaokang Yang Wenjun Zhang Shanghai Digital Media Processing Transmission Key Lab, Shanghai Jiaotong University
More informationClustering web search results
Clustering K-means Machine Learning CSE546 Emily Fox University of Washington November 4, 2013 1 Clustering images Set of Images [Goldberger et al.] 2 1 Clustering web search results 3 Some Data 4 2 K-means
More informationADAPTIVE LOW RANK AND SPARSE DECOMPOSITION OF VIDEO USING COMPRESSIVE SENSING
ADAPTIVE LOW RANK AND SPARSE DECOMPOSITION OF VIDEO USING COMPRESSIVE SENSING Fei Yang 1 Hong Jiang 2 Zuowei Shen 3 Wei Deng 4 Dimitris Metaxas 1 1 Rutgers University 2 Bell Labs 3 National University
More informationCIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, :59pm, PDF to Canvas [100 points]
CIS 520, Machine Learning, Fall 2015: Assignment 7 Due: Mon, Nov 16, 2015. 11:59pm, PDF to Canvas [100 points] Instructions. Please write up your responses to the following problems clearly and concisely.
More informationInference and Representation
Inference and Representation Rachel Hodos New York University Lecture 5, October 6, 2015 Rachel Hodos Lecture 5: Inference and Representation Today: Learning with hidden variables Outline: Unsupervised
More informationAarti Singh. Machine Learning / Slides Courtesy: Eric Xing, M. Hein & U.V. Luxburg
Spectral Clustering Aarti Singh Machine Learning 10-701/15-781 Apr 7, 2010 Slides Courtesy: Eric Xing, M. Hein & U.V. Luxburg 1 Data Clustering Graph Clustering Goal: Given data points X1,, Xn and similarities
More informationSpectral Clustering. Presented by Eldad Rubinstein Based on a Tutorial by Ulrike von Luxburg TAU Big Data Processing Seminar December 14, 2014
Spectral Clustering Presented by Eldad Rubinstein Based on a Tutorial by Ulrike von Luxburg TAU Big Data Processing Seminar December 14, 2014 What are we going to talk about? Introduction Clustering and
More informationK-Means and Gaussian Mixture Models
K-Means and Gaussian Mixture Models David Rosenberg New York University June 15, 2015 David Rosenberg (New York University) DS-GA 1003 June 15, 2015 1 / 43 K-Means Clustering Example: Old Faithful Geyser
More informationMSA220 - Statistical Learning for Big Data
MSA220 - Statistical Learning for Big Data Lecture 13 Rebecka Jörnsten Mathematical Sciences University of Gothenburg and Chalmers University of Technology Clustering Explorative analysis - finding groups
More information55:148 Digital Image Processing Chapter 11 3D Vision, Geometry
55:148 Digital Image Processing Chapter 11 3D Vision, Geometry Topics: Basics of projective geometry Points and hyperplanes in projective space Homography Estimating homography from point correspondence
More informationDATA MINING LECTURE 7. Hierarchical Clustering, DBSCAN The EM Algorithm
DATA MINING LECTURE 7 Hierarchical Clustering, DBSCAN The EM Algorithm CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related)
More informationExpectation Maximization!
Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University and http://www.stanford.edu/class/cs276/handouts/lecture17-clustering.ppt Steps in Clustering Select Features
More informationCS 231A Computer Vision (Autumn 2012) Problem Set 1
CS 231A Computer Vision (Autumn 2012) Problem Set 1 Due: Oct. 9 th, 2012 (2:15 pm) 1 Finding an Approximate Image asis EigenFaces (25 points) In this problem you will implement a solution to a facial recognition
More informationS(x) = arg min s i 2S kx
8 Clustering This topic will focus on automatically grouping data points into subsets of similar points. There are numerous ways to define this problem, and most of them are quite messy. And many techniques
More informationMachine Learning A W 1sst KU. b) [1 P] Give an example for a probability distributions P (A, B, C) that disproves
Machine Learning A 708.064 11W 1sst KU Exercises Problems marked with * are optional. 1 Conditional Independence I [2 P] a) [1 P] Give an example for a probability distribution P (A, B, C) that disproves
More informationChapter 7: Competitive learning, clustering, and self-organizing maps
Chapter 7: Competitive learning, clustering, and self-organizing maps António R. C. Paiva EEL 6814 Spring 2008 Outline Competitive learning Clustering Self-Organizing Maps What is competition in neural
More informationCS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu HITS (Hypertext Induced Topic Selection) Is a measure of importance of pages or documents, similar to PageRank
More informationCLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS
CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS CHAPTER 4 CLASSIFICATION WITH RADIAL BASIS AND PROBABILISTIC NEURAL NETWORKS 4.1 Introduction Optical character recognition is one of
More informationCS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu SPAM FARMING 2/11/2013 Jure Leskovec, Stanford C246: Mining Massive Datasets 2 2/11/2013 Jure Leskovec, Stanford
More informationClustering Lecture 8. David Sontag New York University. Slides adapted from Luke Zettlemoyer, Vibhav Gogate, Carlos Guestrin, Andrew Moore, Dan Klein
Clustering Lecture 8 David Sontag New York University Slides adapted from Luke Zettlemoyer, Vibhav Gogate, Carlos Guestrin, Andrew Moore, Dan Klein Clustering: Unsupervised learning Clustering Requires
More informationMachine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany
More informationImage Segmentation. April 24, Stanford University. Philipp Krähenbühl (Stanford University) Segmentation April 24, / 63
Image Segmentation Philipp Krähenbühl Stanford University April 24, 2013 Philipp Krähenbühl (Stanford University) Segmentation April 24, 2013 1 / 63 Image Segmentation Goal: identify groups of pixels that
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More informationCluster Analysis. Jia Li Department of Statistics Penn State University. Summer School in Statistics for Astronomers IV June 9-14, 2008
Cluster Analysis Jia Li Department of Statistics Penn State University Summer School in Statistics for Astronomers IV June 9-1, 8 1 Clustering A basic tool in data mining/pattern recognition: Divide a
More informationEnergy Minimization for Segmentation in Computer Vision
S * = arg S min E(S) Energy Minimization for Segmentation in Computer Vision Meng Tang, Dmitrii Marin, Ismail Ben Ayed, Yuri Boykov Outline Clustering/segmentation methods K-means, GrabCut, Normalized
More informationFisher vector image representation
Fisher vector image representation Jakob Verbeek January 13, 2012 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.11.12.php Fisher vector representation Alternative to bag-of-words image representation
More informationImproving Image Segmentation Quality Via Graph Theory
International Symposium on Computers & Informatics (ISCI 05) Improving Image Segmentation Quality Via Graph Theory Xiangxiang Li, Songhao Zhu School of Automatic, Nanjing University of Post and Telecommunications,
More informationImage Segmentation. Srikumar Ramalingam School of Computing University of Utah. Slides borrowed from Ross Whitaker
Image Segmentation Srikumar Ramalingam School of Computing University of Utah Slides borrowed from Ross Whitaker Segmentation Semantic Segmentation Indoor layout estimation What is Segmentation? Partitioning
More informationConvexization in Markov Chain Monte Carlo
in Markov Chain Monte Carlo 1 IBM T. J. Watson Yorktown Heights, NY 2 Department of Aerospace Engineering Technion, Israel August 23, 2011 Problem Statement MCMC processes in general are governed by non
More informationUnit 3 : Image Segmentation
Unit 3 : Image Segmentation K-means Clustering Mean Shift Segmentation Active Contour Models Snakes Normalized Cut Segmentation CS 6550 0 Histogram-based segmentation Goal Break the image into K regions
More informationMultiple Model Estimation : The EM Algorithm & Applications
Multiple Model Estimation : The EM Algorithm & Applications Princeton University COS 429 Lecture Nov. 13, 2007 Harpreet S. Sawhney hsawhney@sarnoff.com Recapitulation Problem of motion estimation Parametric
More informationRobust Kernel Methods in Clustering and Dimensionality Reduction Problems
Robust Kernel Methods in Clustering and Dimensionality Reduction Problems Jian Guo, Debadyuti Roy, Jing Wang University of Michigan, Department of Statistics Introduction In this report we propose robust
More informationToday s lecture. Clustering and unsupervised learning. Hierarchical clustering. K-means, K-medoids, VQ
Clustering CS498 Today s lecture Clustering and unsupervised learning Hierarchical clustering K-means, K-medoids, VQ Unsupervised learning Supervised learning Use labeled data to do something smart What
More informationSocial-Network Graphs
Social-Network Graphs Mining Social Networks Facebook, Google+, Twitter Email Networks, Collaboration Networks Identify communities Similar to clustering Communities usually overlap Identify similarities
More informationComputer Vision 5 Segmentation by Clustering
Computer Vision 5 Segmentation by Clustering MAP-I Doctoral Programme Miguel Tavares Coimbra Outline Introduction Applications Simple clustering K-means clustering Graph-theoretic clustering Acknowledgements:
More informationClustering. SC4/SM4 Data Mining and Machine Learning, Hilary Term 2017 Dino Sejdinovic
Clustering SC4/SM4 Data Mining and Machine Learning, Hilary Term 2017 Dino Sejdinovic Clustering is one of the fundamental and ubiquitous tasks in exploratory data analysis a first intuition about the
More informationMixture Models and EM
Table of Content Chapter 9 Mixture Models and EM -means Clustering Gaussian Mixture Models (GMM) Expectation Maximiation (EM) for Mixture Parameter Estimation Introduction Mixture models allows Complex
More informationObject Recognition Using Pictorial Structures. Daniel Huttenlocher Computer Science Department. In This Talk. Object recognition in computer vision
Object Recognition Using Pictorial Structures Daniel Huttenlocher Computer Science Department Joint work with Pedro Felzenszwalb, MIT AI Lab In This Talk Object recognition in computer vision Brief definition
More informationContrast adjustment via Bayesian sequential partitioning
Contrast adjustment via Bayesian sequential partitioning Zhiyu Wang, Shuo Xie, Bai Jiang Abstract Photographs taken in dim light have low color contrast. However, traditional methods for adjusting contrast
More informationApplication of Principal Components Analysis and Gaussian Mixture Models to Printer Identification
Application of Principal Components Analysis and Gaussian Mixture Models to Printer Identification Gazi. Ali, Pei-Ju Chiang Aravind K. Mikkilineni, George T. Chiu Edward J. Delp, and Jan P. Allebach School
More informationIntroduction to Machine Learning
Department of Computer Science, University of Helsinki Autumn 2009, second term Session 8, November 27 th, 2009 1 2 3 Multiplicative Updates for L1-Regularized Linear and Logistic Last time I gave you
More informationMining Social Network Graphs
Mining Social Network Graphs Analysis of Large Graphs: Community Detection Rafael Ferreira da Silva rafsilva@isi.edu http://rafaelsilva.com Note to other teachers and users of these slides: We would be
More informationCS 664 Segmentation. Daniel Huttenlocher
CS 664 Segmentation Daniel Huttenlocher Grouping Perceptual Organization Structural relationships between tokens Parallelism, symmetry, alignment Similarity of token properties Often strong psychophysical
More informationLast week. Multi-Frame Structure from Motion: Multi-View Stereo. Unknown camera viewpoints
Last week Multi-Frame Structure from Motion: Multi-View Stereo Unknown camera viewpoints Last week PCA Today Recognition Today Recognition Recognition problems What is it? Object detection Who is it? Recognizing
More informationA Course in Machine Learning
A Course in Machine Learning Hal Daumé III 13 UNSUPERVISED LEARNING If you have access to labeled training data, you know what to do. This is the supervised setting, in which you have a teacher telling
More informationHomework #4 Programming Assignment Due: 11:59 pm, November 4, 2018
CSCI 567, Fall 18 Haipeng Luo Homework #4 Programming Assignment Due: 11:59 pm, ovember 4, 2018 General instructions Your repository will have now a directory P4/. Please do not change the name of this
More informationMarkov Random Fields and Segmentation with Graph Cuts
Markov Random Fields and Segmentation with Graph Cuts Computer Vision Jia-Bin Huang, Virginia Tech Many slides from D. Hoiem Administrative stuffs Final project Proposal due Oct 27 (Thursday) HW 4 is out
More informationSpectral Clustering X I AO ZE N G + E L HA M TA BA S SI CS E CL A S S P R ESENTATION MA RCH 1 6,
Spectral Clustering XIAO ZENG + ELHAM TABASSI CSE 902 CLASS PRESENTATION MARCH 16, 2017 1 Presentation based on 1. Von Luxburg, Ulrike. "A tutorial on spectral clustering." Statistics and computing 17.4
More informationImage Segmentation. Selim Aksoy. Bilkent University
Image Segmentation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Examples of grouping in vision [http://poseidon.csd.auth.gr/lab_research/latest/imgs/s peakdepvidindex_img2.jpg]
More informationImage Segmentation. Selim Aksoy. Bilkent University
Image Segmentation Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr Examples of grouping in vision [http://poseidon.csd.auth.gr/lab_research/latest/imgs/s peakdepvidindex_img2.jpg]
More informationArtificial Neural Networks Unsupervised learning: SOM
Artificial Neural Networks Unsupervised learning: SOM 01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011 01101011 01110101 01110000 01101001 01101110 01100001
More information