UVA CS 6316/4501 Fall 2016 Machine Learning. Lecture 19: Unsupervised Clustering (I) Dr. Yanjun Qi. University of Virginia
|
|
|
- Joseph Stevens
- 5 years ago
- Views:
Transcription
1 Dr. Yanjun Qi / UVA CS 66 / f6 UVA CS 66/ Fall 6 Machine Learning Lecture 9: Unsupervised Clustering (I) Dr. Yanjun Qi University of Virginia Department of Computer Science
2 Dr. Yanjun Qi / UVA CS 66 / f6 Where are we? è major secjons of this course q Regression (supervised) q ClassificaJon (supervised) q Feature selecjon q Unsupervised models q Dimension ReducJon (PCA) q Clustering (K-means, GMM/EM, Hierarchical ) q Learning theory q Graphical models
3 Dr. Yanjun Qi / UVA CS 66 / f6 An unlabeled Dataset X a data matrix of n observations on p variables x,x, x p Unsupervised learning = learning from raw (unlabeled, unannotated, etc) data, as opposed to supervised data where a classificajon label of examples is given Data/points/instances/examples/samples/records: [ rows ] Features/aributes/dimensions/independent variables/covariates/predictors/regressors: [ columns]
4 Dr. Yanjun Qi / UVA CS 66 / f6 Today: What is clustering? Are there any groups? What is each group? How many? How to idenjfy them?
5 Dr. Yanjun Qi / UVA CS 66 / f6 What is clustering? Find groups (clusters) of data points such that data points in a group will be similar (or related) to one another and different from (or unrelated to) the data points in other groups Intra-cluster distances are minimized Inter-cluster distances are maximized
6 Dr. Yanjun Qi / UVA CS 66 / f6 What is clustering? Clustering: the process of grouping a set of objects into classes of similar objects high intra-class similarity low inter-class similarity It is the commonest form of unsupervised learning A common and important task that finds many applicajons in Science, Engineering, informajon Science, and other places, e.g. Group genes that perform the same funcjon Group individuals that has similar polijcal view Categorize documents of similar topics Ideality similar objects from pictures 6
7 Dr. Yanjun Qi / UVA CS 66 / f6 Toy Examples People Images Language species 7
8 Dr. Yanjun Qi / UVA CS 66 / f6 Issues for clustering What is a natural grouping among these objects? DefiniJon of "groupness" What makes objects related? DefiniJon of "similarity/distance" RepresentaJon for objects Vector space? NormalizaJon? How many clusters? Fixed a priori? Completely data driven? Avoid trivial clusters - too large or small Clustering Algorithms ParJJonal algorithms Hierarchical algorithms Formal foundajon and convergence 8
9 Dr. Yanjun Qi / UVA CS 66 / f6 Today Roadmap: clustering DefiniJon of "groupness DefiniJon of "similarity/distance" RepresentaJon for objects How many clusters? Clustering Algorithms ParJJonal algorithms Hierarchical algorithms Formal foundajon and convergence 9
10 Dr. Yanjun Qi / UVA CS 66 / f6 What is a natural grouping among these objects?
11 Another example: clustering is subjecjve A B A B B B Two possible SoluJons A B B A A A Dr. Yanjun Qi / UVA CS 66 / f6
12 Dr. Yanjun Qi / UVA CS 66 / f6 Today Roadmap: clustering DefiniJon of "groupness DefiniJon of "similarity/distance" RepresentaJon for objects How many clusters? Clustering Algorithms ParJJonal algorithms Hierarchical algorithms Formal foundajon and convergence
13 Dr. Yanjun Qi / UVA CS 66 / f6 What is Similarity? Hard to define! But we know it when we see it The real meaning of similarity is a philosophical quesjon. We will take a more pragmajc approach Depends on representajon and algorithm. For many rep./alg., easier to think in terms of a distance (rather than similarity) between vectors.
14 Dr. Yanjun Qi / UVA CS 66 / f6 What properjes should a distance measure have? D(A,B) = D(B,A) Symmetry D(A,A) = Constancy of Self-Similarity D(A,B) = IIf A= B Posi=vity Separa=on D(A,B) <= D(A,C) + D(B,C) Triangular Inequality
15 Dr. Yanjun Qi / UVA CS 66 / f6 IntuiJons behind desirable properjes of distance measure D(A,B) = D(B,A) Symmetry Otherwise you could claim "Alex looks like Bob, but Bob looks nothing like Alex" D(A,A) = Constancy of Self-Similarity Otherwise you could claim "Alex looks more like Bob, than Bob does" D(A,B) = IIf A= B Posi=vity Separa=on Otherwise there are objects in your world that are different, but you cannot tell apart. D(A,B) <= D(A,C) + D(B,C) Triangular Inequality Otherwise you could claim "Alex is very like Bob, and Alex is very like Carl, but Bob is very unlike Carl"
16 Dr. Yanjun Qi / UVA CS 66 / f6 Distance Measures: Minkowski Metric Suppose two object x and y both have p features x = (x,x,!,x p ) y = ( y!, y,!, y p ) The Minkowski metric is defined by Most Common Minkowski Metrics d(x, y)=! r p i= xi yi r, r = (Euclidean distance ) d(x, y)= xi yi, r = (Manhattan distance) d(x, y)= xi yi, r = + ("sup" distance ) d(x, y)= max i p p i= p i= xi yi 6
17 Dr. Yanjun Qi / UVA CS 66 / f6 An Example x y : : : Euclidean distance: Manhattan distance: "sup"distance: + =. + = 7. max{, } =. 7
18 Dr. Yanjun Qi / UVA CS 66 / f6 Hamming distance: binary features Manhanan distance is called Hamming distance p when all features are binary. GeneA GeneB d(x, y)=! Hamming Distance: #( ) + #( ) = + =. xi yi E.g., Gene Expression Levels Under 7 CondiJons (-High,-Low) i= 6 7 8
19 Dr. Yanjun Qi / UVA CS 66 / f6 Similarity Measures: CorrelaJon Coefficient Expression Level Expression Level Gene A Gene B Gene B Gene A Time Time CorrelaJon is unit independent; Expression Level Gene B If you scale one of the objects ten Jmes, you will get different euclidean distances and same correlajon distances. Time Gene A 9
20 Dr. Yanjun Qi / UVA CS 66 / f6 Similarity Measures: CorrelaJon Coefficient Pearson correlajon coefficient s( x, y) = where x = p i= ( x x)( y p p ( xi x) i= i= p p i= x s( x, y) i i and y = i y) ( y i y) p p i= y. i Measuring the linear correlamon between two sequences, x and y, giving a value between + and inclusive, where is total posijve correlamon, is no correlamon, and is total negajve correlamon. Special case: cosine distance CorrelaJon is unit independent!! x y s ( x, y) =!! x y
21 Dr. Yanjun Qi / UVA CS 66 / f6 Edit Distance: A generic technique for measuring similarity To measure the similarity between two objects, transform one of the objects into the other, and measure how much effort it took. The measure of effort becomes the distance measure. The distance between Pany and Selma. Change dress color, point Change earring shape, point Change hair part, point D(Pany,Selma) = The distance between Marge and Selma. Change dress color, point Add earrings, point Decrease height, point Take up smoking, point Lose weight, point D(Marge,Selma) = Marge Pany Selma This is called the Edit distance or the TransformaJon distance
22 Dr. Yanjun Qi / UVA CS 66 / f6 Today Roadmap: clustering DefiniJon of "groupness DefiniJon of "similarity/distance" RepresentaJon for objects How many clusters? Clustering Algorithms ParJJonal algorithms Hierarchical algorithms Formal foundajon and convergence
23 Dr. Yanjun Qi / UVA CS 66 / f6 Clustering Algorithms ParJJonal algorithms Usually start with a random (parjal) parjjoning Refine it iterajvely K means clustering Mixture-Model based clustering Hierarchical algorithms Bonom-up, agglomerajve Top-down, divisive
24 Dr. Yanjun Qi / UVA CS 66 / f6 Today Roadmap: clustering DefiniJon of "groupness DefiniJon of "similarity/distance" RepresentaJon for objects How many clusters? Clustering Algorithms ParJJonal algorithms Hierarchical algorithms Formal foundajon and convergence
25 Dr. Yanjun Qi / UVA CS 66 / f6 Hierarchical Clustering Build a tree-based hierarchical taxonomy (dendrogram) from a set of objects, e.g. organisms, documents. With backbone Without backbone Note that hierarchies are commonly used to organize informajon, for example in a web portal. Yahoo! hierarchy is manually created, we will focus on automajc creajon of hierarchies in data mining.
![] Number Number of Possible of Leafs Dendrograms.](/docs-images/81/84080608/images/26-4.jpg "..,9, Bottom-Up (agglomerative): Starting with each item in its own cluster, find the")
26 Dr. Yanjun Qi / UVA CS 66 / f6 (How-to) Hierarchical Clustering The number of dendrograms with n leafs = (n -)!/[( (n -) ) (n -)!] Number Number of Possible of Leafs Dendrograms...,9, Bottom-Up (agglomerative): Starting with each item in its own cluster, find the best pair to merge into a new cluster. Repeat until all clusters are fused together. A greedy local opjmal solujon Clustering: the process of grouping a set of objects into classes of similar objects è high intra-class similarity low inter-class similarity 6
27 We begin with a distance matrix which contains the distances between every pair of objects in our database. Dr. Yanjun Qi / UVA CS 66 / f D(, ) = 8 D(, ) = 7
28 Bottom-Up (agglomerative): Starting with each item in its own cluster, find the best pair to merge into a new cluster. Repeat until all clusters are fused together. Dr. Yanjun Qi / UVA CS 66 / f6 Consider all possible merges Choose the best 8
29 Bottom-Up (agglomerative): Starting with each item in its own cluster, find the best pair to merge into a new cluster. Repeat until all clusters are fused together. Dr. Yanjun Qi / UVA CS 66 / f6 Consider all possible merges Choose the best Consider all possible merges Choose the best 9
30 Bottom-Up (agglomerative): Starting with each item in its own cluster, find the best pair to merge into a new cluster. Repeat until all clusters are fused together. Dr. Yanjun Qi / UVA CS 66 / f6 Consider all possible merges Choose the best Consider all possible merges Choose the best Consider all possible merges Choose the best
31 Bottom-Up (agglomerative): Starting with each item in its own cluster, find the best pair to merge into a new cluster. Repeat until all clusters are fused together. Dr. Yanjun Qi / UVA CS 66 / f6 Consider all possible merges Consider all possible merges Choose the best But how do we compute distances between clusters rather than objects? Choose the best Consider all possible merges Choose the best
32 Dr. Yanjun Qi / UVA CS 66 / f6 How to decide the distances between clusters? Single-Link Nearest Neighbor: their closest members. Complete-Link Furthest Neighbor: their furthest members. Average: average of all cross-cluster pairs.
33 Computing distance between clusters: Single Link Dr. Yanjun Qi / UVA CS 66 / f6 cluster distance = distance of two closest members in each class - Potentially long and skinny clusters
34 Dr. Yanjun Qi / UVA CS 66 / f6 Computing distance between clusters: : Complete Link cluster distance = distance of two farthest members + tight clusters
35 Dr. Yanjun Qi / UVA CS 66 / f6 Computing distance between clusters: Average Link cluster distance = average distance of all pairs the most widely used measure Robust against noise
36 Example: single link Dr. Yanjun Qi / UVA CS 66 / f6 6
37 Example: single link Dr. Yanjun Qi / UVA CS 66 / f6 7
38 Example: single link (,) (,) 8 min{ 9,8} }, min{ 9 min{,9} }, min{ 6,} min{ }, min{,, (,),,, (,),,, (,), = = = = = = = = = d d d d d d d d d Dr. Yanjun Qi / UVA CS 66 / f6 8
39 Example: single link 7 (,,) (,,) (,) (,) min{ 8,} }, min{ 7 min{ 9,7} }, min{, (,), (,,),, (,), (,,), = = = = = = d d d d d d Dr. Yanjun Qi / UVA CS 66 / f6 9
40 Example: single link 7 (,,) (,,) (,) (,) }, min{ (,,), (,,),,,),(,) ( = = d d d Dr. Yanjun Qi / UVA CS 66 / f6
41 ParJJons by cutng the dendrogram at a desired level: each connected component forms a cluster. Dr. Yanjun Qi / UVA CS 66 / f6 Single linkage Height represents distance between objects / clusters Average linkage
42 Dr. Yanjun Qi / UVA CS 66 / f6 Hierarchical Clustering Bonom-Up AgglomeraJve Clustering Starts with each object in a separate cluster then repeatedly joins the closest pair of clusters, unjl there is only one cluster. The history of merging forms a binary tree or hierarchy (dendrogram) Top-Down divisive StarJng with all the data in a single cluster, Consider every possible way to divide the cluster into two. Choose the best division And recursively operate on both sides.
43 Dr. Yanjun Qi / UVA CS 66 / f6 ComputaJonal Complexity In the first iterajon, all HAC methods need to compute similarity of all pairs of n individual instances which is O(n p). In each of the subsequent n merging iterajons, compute the distance between the most recently created cluster and all other exisjng clusters. For the subsequent steps, in order to maintain an overall O(n ) performance, compujng similarity to each other cluster must be done in constant Jme. Else O(n log n) or O(n ) if done naively
44 Dr. Yanjun Qi / UVA CS 66 / f6 Summary of Hierarchal Clustering Methods No need to specify the number of clusters in advance. Hierarchical structure maps nicely onto human intuijon for some domains They do not scale well: Jme complexity of at least O(n ), where n is the number of total objects. Like any heurisjc search algorithms, local opjma are a problem. InterpretaJon of results is (very) subjecjve.
45 Dr. Yanjun Qi / UVA CS 66 / f6 Hierarchical Clustering Task Clustering Representation n/a Score Function Search/Optimization No clearly defined loss greedy bottom-up (or top-down) Models, Parameters Dendrogram (tree)
46 Dr. Yanjun Qi / UVA CS 66 / f6 References q HasJe, Trevor, et al. The elements of sta=s=cal learning. Vol.. No.. New York: Springer, 9. q Big thanks to Prof. Eric CMU for allowing me to reuse some of his slides q Big thanks to Prof. Ziv CMU for allowing me to reuse some of his slides 6
UVA CS 4501: Machine Learning. Lecture 22: Unsupervised Clustering (I) Dr. Yanjun Qi. University of Virginia. Department of Computer Science
UVA CS : Machine Learning Lecture : Unsupervised Clustering (I) Dr. Yanjun Qi University of Virginia Department of Computer Science Where are we? è major secjons of this course q Regression (supervised)
10601 Machine Learning. Hierarchical clustering. Reading: Bishop: 9-9.2
161 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview Clustering - Hierarchical, semi-supervised learning Graphical models - Bayesian networks, HMMs, Reasoning under
Lecture 15 Clustering. Oct
Lecture 15 Clustering Oct 31 2008 Unsupervised learning and pattern discovery So far, our data has been in this form: x 11,x 21, x 31,, x 1 m y1 x 12 22 2 2 2,x, x 3,, x m y We will be looking at unlabeled
Unsupervised learning, Clustering CS434
Unsupervised learning, Clustering CS434 Unsupervised learning and pattern discovery So far, our data has been in this form: We will be looking at unlabeled data: x 11,x 21, x 31,, x 1 m x 12,x 22, x 32,,
CSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Cluster Analysis: Basic Concepts and Methods Huan Sun, CSE@The Ohio State University 09/25/2017 Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han 2 Chapter 10.
10701 Machine Learning. Clustering
171 Machine Learning Clustering What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally, finding natural groupings among
CSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Cluster Analysis: Basic Concepts and Methods Huan Sun, CSE@The Ohio State University Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han 2 Chapter 10. Cluster
Data Mining. Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Department of Computer Science
Data Mining Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology Department of Computer Science 06 07 Department of CS - DM - UHD Road map Cluster Analysis: Basic
Clustering CE-324: Modern Information Retrieval Sharif University of Technology
Clustering CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2014 Most slides have been adapted from: Profs. Manning, Nayak & Raghavan (CS-276, Stanford) Ch. 16 What
Based on Raymond J. Mooney s slides
Instance Based Learning Based on Raymond J. Mooney s slides University of Texas at Austin 1 Example 2 Instance-Based Learning Unlike other learning algorithms, does not involve construction of an explicit
5/15/16. Computational Methods for Data Analysis. Massimo Poesio UNSUPERVISED LEARNING. Clustering. Unsupervised learning introduction
Computational Methods for Data Analysis Massimo Poesio UNSUPERVISED LEARNING Clustering Unsupervised learning introduction 1 Supervised learning Training set: Unsupervised learning Training set: 2 Clustering
Clustering and Dimensionality Reduction. Stony Brook University CSE545, Fall 2017
Clustering and Dimensionality Reduction Stony Brook University CSE545, Fall 2017 Goal: Generalize to new data Model New Data? Original Data Does the model accurately reflect new data? Supervised vs. Unsupervised
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-31-017 Outline Background Defining proximity Clustering methods Determining number of clusters Comparing two solutions Cluster analysis as unsupervised Learning
CS 1675 Introduction to Machine Learning Lecture 18. Clustering. Clustering. Groups together similar instances in the data sample
CS 1675 Introduction to Machine Learning Lecture 18 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem:
BBS654 Data Mining. Pinar Duygulu. Slides are adapted from Nazli Ikizler
BBS654 Data Mining Pinar Duygulu Slides are adapted from Nazli Ikizler 1 Classification Classification systems: Supervised learning Make a rational prediction given evidence There are several methods for
Clustering. CE-717: Machine Learning Sharif University of Technology Spring Soleymani
Clustering CE-717: Machine Learning Sharif University of Technology Spring 2016 Soleymani Outline Clustering Definition Clustering main approaches Partitional (flat) Hierarchical Clustering validation
Hard clustering. Each object is assigned to one and only one cluster. Hierarchical clustering is usually hard. Soft (fuzzy) clustering
An unsupervised machine learning problem Grouping a set of objects in such a way that objects in the same group (a cluster) are more similar (in some sense or another) to each other than to those in other
CS 2750 Machine Learning. Lecture 19. Clustering. CS 2750 Machine Learning. Clustering. Groups together similar instances in the data sample
Lecture 9 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem: distribute data into k different groups
Distances, Clustering! Rafael Irizarry!
Distances, Clustering! Rafael Irizarry! Heatmaps! Distance! Clustering organizes things that are close into groups! What does it mean for two genes to be close?! What does it mean for two samples to
Slides by Eamonn Keogh
Slides by Eamonn Keogh Also called unsupervised learning, sometimes called classification by statisticians and sorting by psychologists and segmentation by people in marketing Organizing data into classes
Clustering. Partition unlabeled examples into disjoint subsets of clusters, such that:
Text Clustering 1 Clustering Partition unlabeled examples into disjoint subsets of clusters, such that: Examples within a cluster are very similar Examples in different clusters are very different Discover
Data Informatics. Seon Ho Kim, Ph.D.
Data Informatics Seon Ho Kim, Ph.D. seonkim@usc.edu Clustering Overview Supervised vs. Unsupervised Learning Supervised learning (classification) Supervision: The training data (observations, measurements,
Unsupervised Learning
Outline Unsupervised Learning Basic concepts K-means algorithm Representation of clusters Hierarchical clustering Distance functions Which clustering algorithm to use? NN Supervised learning vs. unsupervised
Machine Learning and Data Mining. Clustering (1): Basics. Kalev Kask
Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of
Linear Regression QuadraJc Regression Year
Linear Regression Regression Given: n Data X = x (1),...,x (n)o where x (i) 2 R d n Corresponding labels y = y (1),...,y (n)o where y (i) 2 R September Arc+c Sea Ice Extent (1,000,000 sq km) 9 8 7 6 5
UVA CS 6316/4501 Fall 2016 Machine Learning. Lecture 15: K-nearest-neighbor Classifier / Bias-Variance Tradeoff. Dr. Yanjun Qi. University of Virginia
UVA CS 6316/4501 Fall 2016 Machine Learning Lecture 15: K-nearest-neighbor Classifier / Bias-Variance Tradeoff Dr. Yanjun Qi University of Virginia Department of Computer Science 11/9/16 1 Rough Plan HW5
Unsupervised Learning. Presenter: Anil Sharma, PhD Scholar, IIIT-Delhi
Unsupervised Learning Presenter: Anil Sharma, PhD Scholar, IIIT-Delhi Content Motivation Introduction Applications Types of clustering Clustering criterion functions Distance functions Normalization Which
Hierarchical Clustering 4/5/17
Hierarchical Clustering 4/5/17 Hypothesis Space Continuous inputs Output is a binary tree with data points as leaves. Useful for explaining the training data. Not useful for making new predictions. Direction
What is Unsupervised Learning?
Clustering What is Unsupervised Learning? Unlike in supervised learning, in unsupervised learning, there are no labels We simply a search for patterns in the data Examples Clustering Density Estimation
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-25-2018 Outline Background Defining proximity Clustering methods Determining number of clusters Other approaches Cluster analysis as unsupervised Learning Unsupervised
Cluster Analysis: Agglomerate Hierarchical Clustering
Cluster Analysis: Agglomerate Hierarchical Clustering Yonghee Lee Department of Statistics, The University of Seoul Oct 29, 2015 Contents 1 Cluster Analysis Introduction Distance matrix Agglomerative Hierarchical
Clustering algorithms
Clustering algorithms Machine Learning Hamid Beigy Sharif University of Technology Fall 1393 Hamid Beigy (Sharif University of Technology) Clustering algorithms Fall 1393 1 / 22 Table of contents 1 Supervised
Supervised and Unsupervised Learning (II)
Supervised and Unsupervised Learning (II) Yong Zheng Center for Web Intelligence DePaul University, Chicago IPD 346 - Data Science for Business Program DePaul University, Chicago, USA Intro: Supervised
UVA CS 4501: Machine Learning. Lecture 10: K-nearest-neighbor Classifier / Bias-Variance Tradeoff. Dr. Yanjun Qi. University of Virginia
UVA CS 4501: Machine Learning Lecture 10: K-nearest-neighbor Classifier / Bias-Variance Tradeoff Dr. Yanjun Qi University of Virginia Department of Computer Science 1 Where are we? è Five major secfons
Cluster Analysis. Mu-Chun Su. Department of Computer Science and Information Engineering National Central University 2003/3/11 1
Cluster Analysis Mu-Chun Su Department of Computer Science and Information Engineering National Central University 2003/3/11 1 Introduction Cluster analysis is the formal study of algorithms and methods
Data Mining Techniques
Data Mining Techniques CS 6220 - Section 3 - Fall 2016 Lecture 8 Jan-Willem van de Meent (credit: Yijun Zhao, Carla Brodley, Eamonn Keogh) Classification Wrap-up Classifier Comparison Data Nearest Neighbors
Hierarchical Clustering
What is clustering Partitioning of a data set into subsets. A cluster is a group of relatively homogeneous cases or observations Hierarchical Clustering Mikhail Dozmorov Fall 2016 2/61 What is clustering
Gene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
K-Means Clustering 3/3/17
K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data points. We want to find underlying structure in the data. Examples: Identify groups of similar data points. Clustering
Data Mining Cluster Analysis: Basic Concepts and Algorithms. Slides From Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Slides From Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining
Chapter DM:II. II. Cluster Analysis
Chapter DM:II II. Cluster Analysis Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained Cluster Analysis DM:II-1
CHAPTER 4: CLUSTER ANALYSIS
CHAPTER 4: CLUSTER ANALYSIS WHAT IS CLUSTER ANALYSIS? A cluster is a collection of data-objects similar to one another within the same group & dissimilar to the objects in other groups. Cluster analysis
Tree Models of Similarity and Association. Clustering and Classification Lecture 5
Tree Models of Similarity and Association Clustering and Lecture 5 Today s Class Tree models. Hierarchical clustering methods. Fun with ultrametrics. 2 Preliminaries Today s lecture is based on the monograph
Cluster analysis. Agnieszka Nowak - Brzezinska
Cluster analysis Agnieszka Nowak - Brzezinska Outline of lecture What is cluster analysis? Clustering algorithms Measures of Cluster Validity What is Cluster Analysis? Finding groups of objects such that
What to come. There will be a few more topics we will cover on supervised learning
Summary so far Supervised learning learn to predict Continuous target regression; Categorical target classification Linear Regression Classification Discriminative models Perceptron (linear) Logistic regression
DATA MINING LECTURE 7. Hierarchical Clustering, DBSCAN The EM Algorithm
DATA MINING LECTURE 7 Hierarchical Clustering, DBSCAN The EM Algorithm CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related)
Lesson 3. Prof. Enza Messina
Lesson 3 Prof. Enza Messina Clustering techniques are generally classified into these classes: PARTITIONING ALGORITHMS Directly divides data points into some prespecified number of clusters without a hierarchical
CS570: Introduction to Data Mining
CS570: Introduction to Data Mining Scalable Clustering Methods: BIRCH and Others Reading: Chapter 10.3 Han, Chapter 9.5 Tan Cengiz Gunay, Ph.D. Slides courtesy of Li Xiong, Ph.D., 2011 Han, Kamber & Pei.
Clustering Part 3. Hierarchical Clustering
Clustering Part Dr Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville Hierarchical Clustering Two main types: Agglomerative Start with the points
Chapter 6: Cluster Analysis
Chapter 6: Cluster Analysis The major goal of cluster analysis is to separate individual observations, or items, into groups, or clusters, on the basis of the values for the q variables measured on each
Unsupervised Learning
Unsupervised Learning A review of clustering and other exploratory data analysis methods HST.951J: Medical Decision Support Harvard-MIT Division of Health Sciences and Technology HST.951J: Medical Decision
Statistics 202: Data Mining. c Jonathan Taylor. Clustering Based in part on slides from textbook, slides of Susan Holmes.
Clustering Based in part on slides from textbook, slides of Susan Holmes December 2, 2012 1 / 1 Clustering Clustering Goal: Finding groups of objects such that the objects in a group will be similar (or
Lecture Notes for Chapter 7. Introduction to Data Mining, 2 nd Edition. by Tan, Steinbach, Karpatne, Kumar
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 7 Introduction to Data Mining, 2 nd Edition by Tan, Steinbach, Karpatne, Kumar Hierarchical Clustering Produces a set
INF4820. Clustering. Erik Velldal. Nov. 17, University of Oslo. Erik Velldal INF / 22
INF4820 Clustering Erik Velldal University of Oslo Nov. 17, 2009 Erik Velldal INF4820 1 / 22 Topics for Today More on unsupervised machine learning for data-driven categorization: clustering. The task
Clustering. Unsupervised Learning
Clustering. Unsupervised Learning Maria-Florina Balcan 03/02/2016 Clustering, Informal Goals Goal: Automatically partition unlabeled data into groups of similar datapoints. Question: When and why would
Lecture 25: Review I
Lecture 25: Review I Reading: Up to chapter 5 in ISLR. STATS 202: Data mining and analysis Jonathan Taylor 1 / 18 Unsupervised learning In unsupervised learning, all the variables are on equal standing,
Clustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
Foundations of Machine Learning CentraleSupélec Fall Clustering Chloé-Agathe Azencot
Foundations of Machine Learning CentraleSupélec Fall 2017 12. Clustering Chloé-Agathe Azencot Centre for Computational Biology, Mines ParisTech chloe-agathe.azencott@mines-paristech.fr Learning objectives
Big Data Analytics! Special Topics for Computer Science CSE CSE Feb 9
Big Data Analytics! Special Topics for Computer Science CSE 4095-001 CSE 5095-005! Feb 9 Fei Wang Associate Professor Department of Computer Science and Engineering fei_wang@uconn.edu Clustering I What
Cluster analysis formalism, algorithms. Department of Cybernetics, Czech Technical University in Prague.
Cluster analysis formalism, algorithms Jiří Kléma Department of Cybernetics, Czech Technical University in Prague http://ida.felk.cvut.cz poutline motivation why clustering? applications, clustering as
Clustering CS 550: Machine Learning
Clustering CS 550: Machine Learning This slide set mainly uses the slides given in the following links: http://www-users.cs.umn.edu/~kumar/dmbook/ch8.pdf http://www-users.cs.umn.edu/~kumar/dmbook/dmslides/chap8_basic_cluster_analysis.pdf
Cluster Analysis. Angela Montanari and Laura Anderlucci
Cluster Analysis Angela Montanari and Laura Anderlucci 1 Introduction Clustering a set of n objects into k groups is usually moved by the aim of identifying internally homogenous groups according to a
Introduction to Data Mining
Introduction to Data Mining Lecture #14: Clustering Seoul National University 1 In This Lecture Learn the motivation, applications, and goal of clustering Understand the basic methods of clustering (bottom-up
Midterm Examination CS540-2: Introduction to Artificial Intelligence
Midterm Examination CS540-2: Introduction to Artificial Intelligence March 15, 2018 LAST NAME: FIRST NAME: Problem Score Max Score 1 12 2 13 3 9 4 11 5 8 6 13 7 9 8 16 9 9 Total 100 Question 1. [12] Search
Introduction to Machine Learning. Xiaojin Zhu
Introduction to Machine Learning Xiaojin Zhu jerryzhu@cs.wisc.edu Read Chapter 1 of this book: Xiaojin Zhu and Andrew B. Goldberg. Introduction to Semi- Supervised Learning. http://www.morganclaypool.com/doi/abs/10.2200/s00196ed1v01y200906aim006
Lecture 10: Semantic Segmentation and Clustering
Lecture 10: Semantic Segmentation and Clustering Vineet Kosaraju, Davy Ragland, Adrien Truong, Effie Nehoran, Maneekwan Toyungyernsub Department of Computer Science Stanford University Stanford, CA 94305
Clustering. Unsupervised Learning
Clustering. Unsupervised Learning Maria-Florina Balcan 11/05/2018 Clustering, Informal Goals Goal: Automatically partition unlabeled data into groups of similar datapoints. Question: When and why would
Clustering Distance measures K-Means. Lecture 22: Aykut Erdem December 2016 Hacettepe University
Clustering Distance measures K-Means Lecture 22: Aykut Erdem December 2016 Hacettepe University Last time Boosting Idea: given a weak learner, run it multiple times on (reweighted) training data, then
Hierarchical Clustering
Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram A tree like diagram that records the sequences of merges or splits 0 0 0 00
Understanding Clustering Supervising the unsupervised
Understanding Clustering Supervising the unsupervised Janu Verma IBM T.J. Watson Research Center, New York http://jverma.github.io/ jverma@us.ibm.com @januverma Clustering Grouping together similar data
Big Data Infrastructure CS 489/698 Big Data Infrastructure (Winter 2017)
Big Data Infrastructure CS 489/698 Big Data Infrastructure (Winter 2017) Week 9: Data Mining (4/4) March 9, 2017 Jimmy Lin David R. Cheriton School of Computer Science University of Waterloo These slides
Exploratory Analysis: Clustering
Exploratory Analysis: Clustering (some material taken or adapted from slides by Hinrich Schutze) Heejun Kim June 26, 2018 Clustering objective Grouping documents or instances into subsets or clusters Documents
Data Mining Clustering
Data Mining Clustering Jingpeng Li 1 of 34 Supervised Learning F(x): true function (usually not known) D: training sample (x, F(x)) 57,M,195,0,125,95,39,25,0,1,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0 0
Today s lecture. Clustering and unsupervised learning. Hierarchical clustering. K-means, K-medoids, VQ
Clustering CS498 Today s lecture Clustering and unsupervised learning Hierarchical clustering K-means, K-medoids, VQ Unsupervised learning Supervised learning Use labeled data to do something smart What
Clustering Lecture 3: Hierarchical Methods
Clustering Lecture 3: Hierarchical Methods Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced
Clustering and Dimensionality Reduction
Clustering and Dimensionality Reduction Some material on these is slides borrowed from Andrew Moore's excellent machine learning tutorials located at: Data Mining Automatically extracting meaning from
Data Mining Concepts & Techniques
Data Mining Concepts & Techniques Lecture No 08 Cluster Analysis Naeem Ahmed Email: naeemmahoto@gmailcom Department of Software Engineering Mehran Univeristy of Engineering and Technology Jamshoro Outline
CS490W. Text Clustering. Luo Si. Department of Computer Science Purdue University
CS490W Text Clustering Luo Si Department of Computer Science Purdue University [Borrows slides from Chris Manning, Ray Mooney and Soumen Chakrabarti] Clustering Document clustering Motivations Document
An Unsupervised Technique for Statistical Data Analysis Using Data Mining
International Journal of Information Sciences and Application. ISSN 0974-2255 Volume 5, Number 1 (2013), pp. 11-20 International Research Publication House http://www.irphouse.com An Unsupervised Technique
Mining di Dati Web. Lezione 3 - Clustering and Classification
Mining di Dati Web Lezione 3 - Clustering and Classification Introduction Clustering and classification are both learning techniques They learn functions describing data Clustering is also known as Unsupervised
High throughput Data Analysis 2. Cluster Analysis
High throughput Data Analysis 2 Cluster Analysis Overview Why clustering? Hierarchical clustering K means clustering Issues with above two Other methods Quality of clustering results Introduction WHY DO
Unsupervised Learning and Data Mining
Unsupervised Learning and Data Mining Unsupervised Learning and Data Mining Clustering Supervised Learning ó Decision trees ó Artificial neural nets ó K-nearest neighbor ó Support vectors ó Linear regression
Unsupervised Learning
Harvard-MIT Division of Health Sciences and Technology HST.951J: Medical Decision Support, Fall 2005 Instructors: Professor Lucila Ohno-Machado and Professor Staal Vinterbo 6.873/HST.951 Medical Decision
Machine Learning and Data Mining. Clustering. (adapted from) Prof. Alexander Ihler
Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Overview What is clustering and its applications? Distance between two clusters. Hierarchical Agglomerative clustering.
Statistics 202: Data Mining. c Jonathan Taylor. Week 8 Based in part on slides from textbook, slides of Susan Holmes. December 2, / 1
Week 8 Based in part on slides from textbook, slides of Susan Holmes December 2, 2012 1 / 1 Part I Clustering 2 / 1 Clustering Clustering Goal: Finding groups of objects such that the objects in a group
Machine Learning. Unsupervised Learning. Manfred Huber
Machine Learning Unsupervised Learning Manfred Huber 2015 1 Unsupervised Learning In supervised learning the training data provides desired target output for learning In unsupervised learning the training
Cluster Analysis. Prof. Thomas B. Fomby Department of Economics Southern Methodist University Dallas, TX April 2008 April 2010
Cluster Analysis Prof. Thomas B. Fomby Department of Economics Southern Methodist University Dallas, TX 7575 April 008 April 010 Cluster Analysis, sometimes called data segmentation or customer segmentation,
ECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
CS7267 MACHINE LEARNING
S7267 MAHINE LEARNING HIERARHIAL LUSTERING Ref: hengkai Li, Department of omputer Science and Engineering, University of Texas at Arlington (Slides courtesy of Vipin Kumar) Mingon Kang, Ph.D. omputer Science,
ECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/004 1
Clustering. Unsupervised Learning
Clustering. Unsupervised Learning Maria-Florina Balcan 04/06/2015 Reading: Chapter 14.3: Hastie, Tibshirani, Friedman. Additional resources: Center Based Clustering: A Foundational Perspective. Awasthi,
DS504/CS586: Big Data Analytics Big Data Clustering Prof. Yanhua Li
Welcome to DS504/CS586: Big Data Analytics Big Data Clustering Prof. Yanhua Li Time: 6:00pm 8:50pm Thu Location: AK 232 Fall 2016 High Dimensional Data v Given a cloud of data points we want to understand
Cluster Analysis. Ying Shen, SSE, Tongji University
Cluster Analysis Ying Shen, SSE, Tongji University Cluster analysis Cluster analysis groups data objects based only on the attributes in the data. The main objective is that The objects within a group
CS 2750: Machine Learning. Clustering. Prof. Adriana Kovashka University of Pittsburgh January 17, 2017
CS 2750: Machine Learning Clustering Prof. Adriana Kovashka University of Pittsburgh January 17, 2017 What is clustering? Grouping items that belong together (i.e. have similar features) Unsupervised:
Multivariate Analysis
Multivariate Analysis Cluster Analysis Prof. Dr. Anselmo E de Oliveira anselmo.quimica.ufg.br anselmo.disciplinas@gmail.com Unsupervised Learning Cluster Analysis Natural grouping Patterns in the data
CPSC 425: Computer Vision
1 / 31 CPSC 425: Computer Vision Instructor: Jim Little little@cs.ubc.ca Department of Computer Science University of British Columbia Lecture Notes 2016/2017 Term 2 2 / 31 Menu March 16, 2017 Topics:
Network Traffic Measurements and Analysis
DEIB - Politecnico di Milano Fall, 2017 Introduction Often, we have only a set of features x = x 1, x 2,, x n, but no associated response y. Therefore we are not interested in prediction nor classification,
SYDE Winter 2011 Introduction to Pattern Recognition. Clustering
SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned
Clustering Gene Expression Data: Acknowledgement: Elizabeth Garrett-Mayer; Shirley Liu; Robert Tibshirani; Guenther Walther; Trevor Hastie
Clustering Gene Expression Data: Acknowledgement: Elizabeth Garrett-Mayer; Shirley Liu; Robert Tibshirani; Guenther Walther; Trevor Hastie Data from Garber et al. PNAS (98), 2001. Clustering Clustering