Introduction to Computer Science
|
|
|
- Bryan Montgomery
- 6 years ago
- Views:
Transcription
1 DM534 Introduction to Computer Science Clustering and Feature Spaces
2 Richard Roettger: About Me Computer Science (Technical University of Munich and thesis at the ICSI at the University of California at Berkeley) PhD at the Max Planck Institute for Computer Science in Saarbrücken Since 2014: Assistant Professor at SDU Research Interests: Bioinformatics Machine Learning Clustering Biological Networks Slides are taken from Arthur Zimek 2
3 Clustering & Feature Spaces Learning Objectives Understand the problem of clustering in general Learn about k-means Understand the importance of feature spaces and object representation Understand the influence of distance functions 3
4 Clustering & Feature Spaces Lecture Content Clustering Clustering in General Partitional Clustering Visualization: Algorithmic Differences Summary Feature Spaces Distances Features for Images Summary 4
5 Purpose of Clustering Identify a finite number of categories (classes, groups: clusters) in a given dataset Similar objects shall be grouped in the same cluster, dissimilar objects in different clusters similarity is highly subjective, depending on the application scenario 5
6 How Many Clusters? Image taken from: 6
7 How Many Clusters? Everitt, Brian S., et al. Cluster Analysis, 5th Edition (2011). 7
8 How Many Clusters? Everitt, Brian S., et al. Cluster Analysis, 5th Edition (2011). 8
9 A Seemingly Simple Problem Each dataset can be clustered in many meaningful ways Highly problem depended Not known by the algorithm a priori Figure from Tan et al. [2006]. 9
10 About Clustering Clustering is the unsupervised machine-learning task of grouping or segmenting a collection of objects into subsets or clusters such that those within each cluster are more closely related to one another than objects assigned to different clusters. What is related? For example Customers? Age? Behavior? Kinship? Treatment of Outliers? Ill-posed Problem... That means there exist multiple solutions What is the best? 10
11 Overview of a Cluster Analysis 11
12 Overview of a Cluster Analysis 12
13 Clustering & Feature Spaces Lecture Content Clustering Clustering in General Partitional Clustering Visualization: Algorithmic Differences Summary Feature Spaces Distances Features for Images Summary 13
14 Steps to Automatization: Cluster Criteria Cohesion: how strong are the cluster objects connected (how similar, pairwise, to each other)? Separation: how well is a cluster separated from other clusters? small within cluster distance large between cluster distance 14
15 Steps to Automatization: Cluster Criteria Cohesion: how strong are the cluster objects connected (how similar, pairwise, to each other)? Separation: how well is a cluster separated from other clusters? There exist many other criteria, e.g., areas with the same density. It is important to choose a criterion which fits the data! small within cluster distance large between cluster distance 15
16 Optimization Partitional clustering algorithms partition a dataset into k clusters, typically minimizing some cost function no overlaps all points must be part of a cluster (compactness criterion), i.e., optimizing cohesion. 16
17 Assumptions for Partitioning Clustering Central assumptions for approaches in this family are typically: number k of clusters known (i.e., given as input) clusters are characterized by their compactness compactness measured by some distance function (e.g., distance of all objects in a cluster from some cluster representative is minimal) criterion of compactness typically leads to convex or even spherically shaped clusters 17
18 Construction of Central Points: Basics objects are points x = x 1,, x d in Euclidean vector space R d dist = Euclidean distance (L 2 ) I centroid μ C : mean vector of all points in cluster C 18
19 Construction of Central Points: Basics objects are points x = x 1,, x d in Euclidean vector space R d dist = Euclidean distance (L 2 ) I centroid μ C : mean vector of all points in cluster C 19
20 Cluster Criteria Measure for compactness: (sum of squares) TD 2 C = p C dist p, μ C 2 Measure of compactness for a clustering: TD 2 C 1,, C k k = TD 2 (C i ) i=1 20
21 Cluster Criteria Measure for compactness: (sum of squares) TD 2 C = p C dist p, μ C 2 Measure of compactness for a clustering: TD 2 C 1,, C k k = TD 2 (C i ) i=1 21
22 Basic Algorithm: Clustering by Minimization of Variance [Forgy, 1965, Lloyd, 1982] start with k (e.g., randomly selected) points as cluster representatives (or with a random partition into k clusters ) repeat: 1. assign each point to the closest representative 2. compute new representatives based on the given partitions (centroid of the assigned points) until there is no change in assignment 22
23 k-means k-means [MacQueen, 1967] is a variant of the basic algorithm: A centroid is immediately updated when some point changes its assignment k-means has very similar properties, but the result now depends on the order of data points in the input file Note: The name k-means is often used indifferently for any variant of the basic algorithm, in particular also for the Algorithm shown before [Forgy, 1965, Lloyd, 1982]. 23
24 Clustering & Feature Spaces Lecture Content Clustering Clustering in General Partitional Clustering Visualization: Algorithmic Differences Summary Feature Spaces Distances Features for Images Summary 24
25 k-means Clustering Lloyd/Forgy Algorithm 25
26 k-means Clustering Lloyd/Forgy Algorithm 26
27 k-means Clustering Lloyd/Forgy Algorithm 27
28 k-means Clustering Lloyd/Forgy Algorithm 28
29 k-means Clustering Lloyd/Forgy Algorithm 29
30 k-means Clustering Lloyd/Forgy Algorithm 30
31 k-means Clustering Lloyd/Forgy Algorithm 31
32 k-means Clustering Lloyd/Forgy Algorithm 32
33 k-means Clustering Lloyd/Forgy Algorithm 33
34 k-means Clustering MacQueen Algorithm 34
35 k-means Clustering MacQueen Algorithm 35
36 k-means Clustering MacQueen Algorithm 36
37 k-means Clustering MacQueen Algorithm 37
38 k-means Clustering MacQueen Algorithm 38
39 k-means Clustering MacQueen Algorithm 39
40 k-means Clustering MacQueen Algorithm 40
41 k-means Clustering MacQueen Algorithm 41
42 k-means Clustering MacQueen Algorithm 42
43 k-means Clustering MacQueen Algorithm 43
44 k-means Clustering MacQueen Algorithm 44
45 k-means Clustering MacQueen Algorithm 45
46 k-means Clustering MacQueen Algorithm 46
47 k-means Clustering MacQueen Algorithm 47
48 k-means Clustering MacQueen Algorithm Alternative Ordering 48
49 k-means Clustering MacQueen Algorithm 49
50 k-means Clustering MacQueen Algorithm 50
51 k-means Clustering MacQueen Algorithm 51
52 k-means Clustering MacQueen Algorithm 52
53 k-means Clustering MacQueen Algorithm 53
54 k-means Clustering MacQueen Algorithm 54
55 k-means Clustering MacQueen Algorithm 55
56 k-means Clustering MacQueen Algorithm 56
57 k-means Clustering MacQueen Algorithm 57
58 k-means Clustering MacQueen Algorithm 58
59 k-means Clustering MacQueen Algorithm 59
60 k-means Clustering MacQueen Algorithm 60
61 k-means Clustering Quality 61
62 k-means Clustering Quality 62
63 k-means Clustering Quality 63
64 Clustering & Feature Spaces Lecture Content Clustering Clustering in General Partitional Clustering Visualization: Algorithmic Differences Summary Feature Spaces Distances Features for Images Summary 64
65 k-means: Pros Efficient: O(k n) per iteration, number of iterations is usually in the order of 10. Easy to implement, thus very popular Only one parameter, easy to understand Well understood and researched Different variants exists Fuzzy clustering Variants without n-dimensional embedding 65
66 k-means: Disadvantages k-means converges towards a local minimum k-means (MacQueen-variant) is order-dependent Deteriorates with noise and outliers (all points are used to compute centroids) Clusters need to be convex and of (more or less) equal extension Number k of clusters is hard to determine Strong dependency on initial partition (in result quality as well as runtime) 66
67 What to do? How can we tackle the initialization problem? 67
68 What to do? How can we tackle the initialization problem? Repeated runs Furthest-first initialization Subset Furthest-first initialization 68
69 Insertion: Furthest First Initialization Select a random point as start For each point, the minimum of its distances to the selected centers is maintained. While less than k points selected, repeat: Selected point p with the maximum distance to the existing centers Remove p from the not-yet-selected points and add it to the center points For each remaining not-yet-selected point q, replace the distance stored for q by the minimum of its old value and the distance from p to q. 69
70 Furthest First Initialization: Visualization 70
71 Furthest First Initialization: Visualization 71
72 Furthest First Initialization: Visualization 72
73 Furthest First Initialization: Visualization 73
74 Furthest First Initialization: Visualization Update the minimum distances 74
75 Furthest First Initialization: Visualization 75
76 Furthest First Initialization: Visualization 76
77 Furthest First Initialization: Visualization 77
78 Furthest First Initialization: Visualization 78
79 Learnings of this Section What is Clustering? Basic idea for identifying good partitions into k clusters Selection of representative points Iterative refinement Local optimum k-means variants [Forgy, 1965, Lloyd, 1982, MacQueen, 1967] Different initialization methods 79
80 Clustering & Feature Spaces Lecture Content Clustering Clustering in General Partitional Clustering Visualization: Algorithmic Differences Summary Feature Spaces Distances Features for Images Summary 80
81 Recall: Clustering as a Workflow 81
82 Similarities Similarity (as given by some distance measure) is a central concept in data mining, e.g.: Clustering: group similar objects in the same cluster, separate dissimilar objects to different clusters Outlier detection: identify objects that are dissimilar (by some characteristic) from most other objects Definition of a suitable distance measure is often crucial for deriving a meaningful solution in the data mining task Images CAD objects Proteins Texts... 82
83 Spaces and Distance Functions 83
84 Spaces and Distance Functions 84
85 Clustering & Feature Spaces Lecture Content Clustering Clustering in General Partitional Clustering Visualization: Algorithmic Differences Summary Feature Spaces Distances Features for Images Summary 85
86 Categories of Feature Descriptors for Images Distribution of colors Texture Shapes (contours) Many more 86
87 Color Histogram A histogram represents the distribution of colors over the pixels of an image Definition of an color histogram: Choose a color space (RGB, HSV, HLS,... ) Choose number of representants (sample points) in the color space Possibly normalization (to account for different image sizes) 87
88 Impact of Number of Representants 88
89 Impact of Number of Representants 89
90 Impact of Number of Representants 90
91 Impact of Number of Representants 91
92 Impact of Number of Representants 92
93 Impact of Number of Representants 93
94 Impact of Number of Representants 94
95 Impact of Number of Representants 95
96 Distances for Color Histograms 96
97 Distances for Color Histograms 97
98 Distances for Color Histograms 98
99 Clustering & Feature Spaces Lecture Content Clustering Clustering in General Partitional Clustering Visualization: Algorithmic Differences Summary Feature Spaces Distances Features for Images Summary 99
100 Your Choice of a Distance Measure There are hundreds of distance functions [Deza and Deza, 2009]. For time series: DTW, EDR, ERP, LCSS,... For texts: Cosine and normalizations For sets based on intersection, union,... (Jaccard) For clusters (single-link, average-link, etc.) For histograms: histogram intersection, Earth movers distance, quadratic forms with color similarity For proteins: Edit distance, structure, With normalization: Canberra, Quadratic forms / bilinear forms: d(x, y) = x T My for some positive (usually symmetric) definite matrix M. 100
101 Learnings of this Section Distances (L p -norms, weighted, quadratic form) Color histograms as feature (vector) descriptors for images Impact of the granularity of color histograms on similarity measures 101
Clustering. CE-717: Machine Learning Sharif University of Technology Spring Soleymani
Clustering CE-717: Machine Learning Sharif University of Technology Spring 2016 Soleymani Outline Clustering Definition Clustering main approaches Partitional (flat) Hierarchical Clustering validation
Olmo S. Zavala Romero. Clustering Hierarchical Distance Group Dist. K-means. Center of Atmospheric Sciences, UNAM.
Center of Atmospheric Sciences, UNAM November 16, 2016 Cluster Analisis Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster)
CSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Cluster Analysis: Basic Concepts and Methods Huan Sun, CSE@The Ohio State University 09/25/2017 Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han 2 Chapter 10.
University of Florida CISE department Gator Engineering. Clustering Part 2
Clustering Part 2 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville Partitional Clustering Original Points A Partitional Clustering Hierarchical
CSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Cluster Analysis: Basic Concepts and Methods Huan Sun, CSE@The Ohio State University Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han 2 Chapter 10. Cluster
BBS654 Data Mining. Pinar Duygulu. Slides are adapted from Nazli Ikizler
BBS654 Data Mining Pinar Duygulu Slides are adapted from Nazli Ikizler 1 Classification Classification systems: Supervised learning Make a rational prediction given evidence There are several methods for
SYDE Winter 2011 Introduction to Pattern Recognition. Clustering
SYDE 372 - Winter 2011 Introduction to Pattern Recognition Clustering Alexander Wong Department of Systems Design Engineering University of Waterloo Outline 1 2 3 4 5 All the approaches we have learned
Kapitel 4: Clustering
Ludwig-Maximilians-Universität München Institut für Informatik Lehr- und Forschungseinheit für Datenbanksysteme Knowledge Discovery in Databases WiSe 2017/18 Kapitel 4: Clustering Vorlesung: Prof. Dr.
Cluster Analysis. Ying Shen, SSE, Tongji University
Cluster Analysis Ying Shen, SSE, Tongji University Cluster analysis Cluster analysis groups data objects based only on the attributes in the data. The main objective is that The objects within a group
Gene Clustering & Classification
BINF, Introduction to Computational Biology Gene Clustering & Classification Young-Rae Cho Associate Professor Department of Computer Science Baylor University Overview Introduction to Gene Clustering
Cluster analysis. Agnieszka Nowak - Brzezinska
Cluster analysis Agnieszka Nowak - Brzezinska Outline of lecture What is cluster analysis? Clustering algorithms Measures of Cluster Validity What is Cluster Analysis? Finding groups of objects such that
Classification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
Clustering CS 550: Machine Learning
Clustering CS 550: Machine Learning This slide set mainly uses the slides given in the following links: http://www-users.cs.umn.edu/~kumar/dmbook/ch8.pdf http://www-users.cs.umn.edu/~kumar/dmbook/dmslides/chap8_basic_cluster_analysis.pdf
Unsupervised Learning
Outline Unsupervised Learning Basic concepts K-means algorithm Representation of clusters Hierarchical clustering Distance functions Which clustering algorithm to use? NN Supervised learning vs. unsupervised
Data Mining Cluster Analysis: Basic Concepts and Algorithms. Slides From Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Slides From Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining
K-Means. Oct Youn-Hee Han
K-Means Oct. 2015 Youn-Hee Han http://link.koreatech.ac.kr ²K-Means algorithm An unsupervised clustering algorithm K stands for number of clusters. It is typically a user input to the algorithm Some criteria
Unsupervised Data Mining: Clustering. Izabela Moise, Evangelos Pournaras, Dirk Helbing
Unsupervised Data Mining: Clustering Izabela Moise, Evangelos Pournaras, Dirk Helbing Izabela Moise, Evangelos Pournaras, Dirk Helbing 1 1. Supervised Data Mining Classification Regression Outlier detection
K-Means Clustering With Initial Centroids Based On Difference Operator
K-Means Clustering With Initial Centroids Based On Difference Operator Satish Chaurasiya 1, Dr.Ratish Agrawal 2 M.Tech Student, School of Information and Technology, R.G.P.V, Bhopal, India Assistant Professor,
Working with Unlabeled Data Clustering Analysis. Hsiao-Lung Chan Dept Electrical Engineering Chang Gung University, Taiwan
Working with Unlabeled Data Clustering Analysis Hsiao-Lung Chan Dept Electrical Engineering Chang Gung University, Taiwan chanhl@mail.cgu.edu.tw Unsupervised learning Finding centers of similarity using
Applications. Foreground / background segmentation Finding skin-colored regions. Finding the moving objects. Intelligent scissors
Segmentation I Goal Separate image into coherent regions Berkeley segmentation database: http://www.eecs.berkeley.edu/research/projects/cs/vision/grouping/segbench/ Slide by L. Lazebnik Applications Intelligent
Unsupervised Learning Partitioning Methods
Unsupervised Learning Partitioning Methods Road Map 1. Basic Concepts 2. K-Means 3. K-Medoids 4. CLARA & CLARANS Cluster Analysis Unsupervised learning (i.e., Class label is unknown) Group data to form
CHAPTER 4: CLUSTER ANALYSIS
CHAPTER 4: CLUSTER ANALYSIS WHAT IS CLUSTER ANALYSIS? A cluster is a collection of data-objects similar to one another within the same group & dissimilar to the objects in other groups. Cluster analysis
Supervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
Hard clustering. Each object is assigned to one and only one cluster. Hierarchical clustering is usually hard. Soft (fuzzy) clustering
An unsupervised machine learning problem Grouping a set of objects in such a way that objects in the same group (a cluster) are more similar (in some sense or another) to each other than to those in other
ECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
Classification. Vladimir Curic. Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University
Classification Vladimir Curic Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University Outline An overview on classification Basics of classification How to choose appropriate
COSC 6339 Big Data Analytics. Fuzzy Clustering. Some slides based on a lecture by Prof. Shishir Shah. Edgar Gabriel Spring 2017.
COSC 6339 Big Data Analytics Fuzzy Clustering Some slides based on a lecture by Prof. Shishir Shah Edgar Gabriel Spring 217 Clustering Clustering is a technique for finding similarity groups in data, called
Hierarchical Clustering
Hierarchical Clustering Hierarchical Clustering Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram A tree-like diagram that records the sequences of merges
Overview of Clustering
based on Loïc Cerfs slides (UFMG) April 2017 UCBL LIRIS DM2L Example of applicative problem Student profiles Given the marks received by students for different courses, how to group the students so that
Unsupervised Learning. Presenter: Anil Sharma, PhD Scholar, IIIT-Delhi
Unsupervised Learning Presenter: Anil Sharma, PhD Scholar, IIIT-Delhi Content Motivation Introduction Applications Types of clustering Clustering criterion functions Distance functions Normalization Which
Unsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2008 CS 551, Spring 2008 c 2008, Selim Aksoy (Bilkent University)
COSC 6397 Big Data Analytics. Fuzzy Clustering. Some slides based on a lecture by Prof. Shishir Shah. Edgar Gabriel Spring 2015.
COSC 6397 Big Data Analytics Fuzzy Clustering Some slides based on a lecture by Prof. Shishir Shah Edgar Gabriel Spring 215 Clustering Clustering is a technique for finding similarity groups in data, called
Data Mining Cluster Analysis: Basic Concepts and Algorithms. Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/004 1
Data Informatics. Seon Ho Kim, Ph.D.
Data Informatics Seon Ho Kim, Ph.D. seonkim@usc.edu Clustering Overview Supervised vs. Unsupervised Learning Supervised learning (classification) Supervision: The training data (observations, measurements,
Unsupervised Learning. Supervised learning vs. unsupervised learning. What is Cluster Analysis? Applications of Cluster Analysis
7 Supervised learning vs unsupervised learning Unsupervised Learning Supervised learning: discover patterns in the data that relate data attributes with a target (class) attribute These patterns are then
Cluster Analysis. Summer School on Geocomputation. 27 June July 2011 Vysoké Pole
Cluster Analysis Summer School on Geocomputation 27 June 2011 2 July 2011 Vysoké Pole Lecture delivered by: doc. Mgr. Radoslav Harman, PhD. Faculty of Mathematics, Physics and Informatics Comenius University,
ECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
INF4820. Clustering. Erik Velldal. Nov. 17, University of Oslo. Erik Velldal INF / 22
INF4820 Clustering Erik Velldal University of Oslo Nov. 17, 2009 Erik Velldal INF4820 1 / 22 Topics for Today More on unsupervised machine learning for data-driven categorization: clustering. The task
Lecture-17: Clustering with K-Means (Contd: DT + Random Forest)
Lecture-17: Clustering with K-Means (Contd: DT + Random Forest) Medha Vidyotma April 24, 2018 1 Contd. Random Forest For Example, if there are 50 scholars who take the measurement of the length of the
[7.3, EA], [9.1, CMB]
![[7.3, EA], [9.1, CMB]](/thumbs/76/73876791.jpg "[7.3, EA], [9.1, CMB]")
K-means Clustering Ke Chen Reading: [7.3, EA], [9.1, CMB] Outline Introduction K-means Algorithm Example How K-means partitions? K-means Demo Relevant Issues Application: Cell Neulei Detection Summary
Clustering: Classic Methods and Modern Views
Clustering: Classic Methods and Modern Views Marina Meilă University of Washington mmp@stat.washington.edu June 22, 2015 Lorentz Center Workshop on Clusters, Games and Axioms Outline Paradigms for clustering
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-25-2018 Outline Background Defining proximity Clustering methods Determining number of clusters Other approaches Cluster analysis as unsupervised Learning Unsupervised
Lecture on Modeling Tools for Clustering & Regression
Lecture on Modeling Tools for Clustering & Regression CS 590.21 Analysis and Modeling of Brain Networks Department of Computer Science University of Crete Data Clustering Overview Organizing data into
Unsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)
Clustering: K-means and Kernel K-means
Clustering: K-means and Kernel K-means Piyush Rai Machine Learning (CS771A) Aug 31, 2016 Machine Learning (CS771A) Clustering: K-means and Kernel K-means 1 Clustering Usually an unsupervised learning problem
Data Mining Algorithms
for the original version: -JörgSander and Martin Ester - Jiawei Han and Micheline Kamber Data Management and Exploration Prof. Dr. Thomas Seidl Data Mining Algorithms Lecture Course with Tutorials Wintersemester
K-Means. Algorithmic Methods of Data Mining M. Sc. Data Science Sapienza University of Rome. Carlos Castillo
K-Means Class Program University Semester Slides by Algorithmic Methods of Data Mining M. Sc. Data Science Sapienza University of Rome Fall 2017 Carlos Castillo http://chato.cl/ Sources: Mohammed J. Zaki,
MultiDimensional Signal Processing Master Degree in Ingegneria delle Telecomunicazioni A.A
MultiDimensional Signal Processing Master Degree in Ingegneria delle Telecomunicazioni A.A. 205-206 Pietro Guccione, PhD DEI - DIPARTIMENTO DI INGEGNERIA ELETTRICA E DELL INFORMAZIONE POLITECNICO DI BARI
CS 2750 Machine Learning. Lecture 19. Clustering. CS 2750 Machine Learning. Clustering. Groups together similar instances in the data sample
Lecture 9 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem: distribute data into k different groups
Clustering. Shishir K. Shah
Clustering Shishir K. Shah Acknowledgement: Notes by Profs. M. Pollefeys, R. Jin, B. Liu, Y. Ukrainitz, B. Sarel, D. Forsyth, M. Shah, K. Grauman, and S. K. Shah Clustering l Clustering is a technique
k-means Clustering Todd W. Neller Gettysburg College Laura E. Brown Michigan Technological University
k-means Clustering Todd W. Neller Gettysburg College Laura E. Brown Michigan Technological University Outline Unsupervised versus Supervised Learning Clustering Problem k-means Clustering Algorithm Visual
An Unsupervised Technique for Statistical Data Analysis Using Data Mining
International Journal of Information Sciences and Application. ISSN 0974-2255 Volume 5, Number 1 (2013), pp. 11-20 International Research Publication House http://www.irphouse.com An Unsupervised Technique
k-means Clustering Todd W. Neller Gettysburg College
k-means Clustering Todd W. Neller Gettysburg College Outline Unsupervised versus Supervised Learning Clustering Problem k-means Clustering Algorithm Visual Example Worked Example Initialization Methods
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 01-31-017 Outline Background Defining proximity Clustering methods Determining number of clusters Comparing two solutions Cluster analysis as unsupervised Learning
Clustering: Overview and K-means algorithm
Clustering: Overview and K-means algorithm Informal goal Given set of objects and measure of similarity between them, group similar objects together K-Means illustrations thanks to 2006 student Martin
Image Processing. Image Features
Image Processing Image Features Preliminaries 2 What are Image Features? Anything. What they are used for? Some statements about image fragments (patches) recognition Search for similar patches matching
Unsupervised Learning : Clustering
Unsupervised Learning : Clustering Things to be Addressed Traditional Learning Models. Cluster Analysis K-means Clustering Algorithm Drawbacks of traditional clustering algorithms. Clustering as a complex
Statistics 202: Statistical Aspects of Data Mining
Statistics 202: Statistical Aspects of Data Mining Professor Rajan Patel Lecture 11 = Chapter 8 Agenda: 1)Reminder about final exam 2)Finish Chapter 5 3)Chapter 8 1 Class Project The class project is due
CBioVikings. Richard Röttger. Copenhagen February 2 nd, Clustering of Biomedical Data
CBioVikings Copenhagen February 2 nd, Richard Röttger 1 Who is talking? 2 Resources Go to http://imada.sdu.dk/~roettger/teaching/cbiovikings.php You will find The dataset These slides An overview paper
Road map. Basic concepts
Clustering Basic concepts Road map K-means algorithm Representation of clusters Hierarchical clustering Distance functions Data standardization Handling mixed attributes Which clustering algorithm to use?
2D image segmentation based on spatial coherence
2D image segmentation based on spatial coherence Václav Hlaváč Czech Technical University in Prague Center for Machine Perception (bridging groups of the) Czech Institute of Informatics, Robotics and Cybernetics
Segmentation Computer Vision Spring 2018, Lecture 27
Segmentation http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 218, Lecture 27 Course announcements Homework 7 is due on Sunday 6 th. - Any questions about homework 7? - How many of you have
COMP 551 Applied Machine Learning Lecture 13: Unsupervised learning
COMP 551 Applied Machine Learning Lecture 13: Unsupervised learning Associate Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551
Clustering. K-means clustering
Clustering K-means clustering Clustering Motivation: Identify clusters of data points in a multidimensional space, i.e. partition the data set {x 1,...,x N } into K clusters. Intuition: A cluster is a
Machine Learning for Signal Processing Clustering. Bhiksha Raj Class Oct 2016
Machine Learning for Signal Processing Clustering Bhiksha Raj Class 11. 13 Oct 2016 1 Statistical Modelling and Latent Structure Much of statistical modelling attempts to identify latent structure in the
Unsupervised Learning. Andrea G. B. Tettamanzi I3S Laboratory SPARKS Team
Unsupervised Learning Andrea G. B. Tettamanzi I3S Laboratory SPARKS Team Table of Contents 1)Clustering: Introduction and Basic Concepts 2)An Overview of Popular Clustering Methods 3)Other Unsupervised
Lecture 6: Unsupervised Machine Learning Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 6: Unsupervised Machine Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 23 November 2018 Overview Unsupervised Machine Learning overview Association
Lecture Notes for Chapter 8. Introduction to Data Mining
Data Mining Cluster Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 8 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/8/4 What
Knowledge Discovery in Databases. Part III Clustering
Knowledge Discovery in Databases Erich Schubert Winter Semester 7/8 Part III Clustering Clustering : / Overview Basic Concepts and Applications Distance & Similarity Clustering Approaches Overview Hierarchical
DATA MINING LECTURE 7. Hierarchical Clustering, DBSCAN The EM Algorithm
DATA MINING LECTURE 7 Hierarchical Clustering, DBSCAN The EM Algorithm CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related)
Unsupervised Learning
Unsupervised Learning Unsupervised learning Until now, we have assumed our training samples are labeled by their category membership. Methods that use labeled samples are said to be supervised. However,
Exploratory data analysis for microarrays
Exploratory data analysis for microarrays Jörg Rahnenführer Computational Biology and Applied Algorithmics Max Planck Institute for Informatics D-66123 Saarbrücken Germany NGFN - Courses in Practical DNA
Hierarchical Clustering
What is clustering Partitioning of a data set into subsets. A cluster is a group of relatively homogeneous cases or observations Hierarchical Clustering Mikhail Dozmorov Fall 2016 2/61 What is clustering
Data Mining: Concepts and Techniques. Chapter March 8, 2007 Data Mining: Concepts and Techniques 1
Data Mining: Concepts and Techniques Chapter 7.1-4 March 8, 2007 Data Mining: Concepts and Techniques 1 1. What is Cluster Analysis? 2. Types of Data in Cluster Analysis Chapter 7 Cluster Analysis 3. A
Clustering: Overview and K-means algorithm
Clustering: Overview and K-means algorithm Informal goal Given set of objects and measure of similarity between them, group similar objects together K-Means illustrations thanks to 2006 student Martin
Data Mining Chapter 9: Descriptive Modeling Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 9: Descriptive Modeling Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Descriptive model A descriptive model presents the main features of the data
9/17/2009. Wenyan Li (Emily Li) Sep. 15, Introduction to Clustering Analysis
Introduction ti to K-means Algorithm Wenan Li (Emil Li) Sep. 5, 9 Outline Introduction to Clustering Analsis K-means Algorithm Description Eample of K-means Algorithm Other Issues of K-means Algorithm
Lecture 10: Semantic Segmentation and Clustering
Lecture 10: Semantic Segmentation and Clustering Vineet Kosaraju, Davy Ragland, Adrien Truong, Effie Nehoran, Maneekwan Toyungyernsub Department of Computer Science Stanford University Stanford, CA 94305
数据挖掘 Introduction to Data Mining
数据挖掘 Introduction to Data Mining Philippe Fournier-Viger Full professor School of Natural Sciences and Humanities philfv8@yahoo.com Spring 2019 S8700113C 1 Introduction Last week: Association Analysis
Clustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
Redefining and Enhancing K-means Algorithm
Redefining and Enhancing K-means Algorithm Nimrat Kaur Sidhu 1, Rajneet kaur 2 Research Scholar, Department of Computer Science Engineering, SGGSWU, Fatehgarh Sahib, Punjab, India 1 Assistant Professor,
Clustering (Basic concepts and Algorithms) Entscheidungsunterstützungssysteme
Clustering (Basic concepts and Algorithms) Entscheidungsunterstützungssysteme Why do we need to find similarity? Similarity underlies many data science methods and solutions to business problems. Some
Introduction to Mobile Robotics
Introduction to Mobile Robotics Clustering Wolfram Burgard Cyrill Stachniss Giorgio Grisetti Maren Bennewitz Christian Plagemann Clustering (1) Common technique for statistical data analysis (machine learning,
Meta-Clustering. Parasaran Raman PhD Candidate School of Computing
Meta-Clustering Parasaran Raman PhD Candidate School of Computing What is Clustering? Goal: Group similar items together Unsupervised No labeling effort Popular choice for large-scale exploratory data
INF4820 Algorithms for AI and NLP. Evaluating Classifiers Clustering
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Murhaf Fares & Stephan Oepen Language Technology Group (LTG) September 27, 2017 Today 2 Recap Evaluation of classifiers Unsupervised
Hierarchical Clustering Lecture 9
Hierarchical Clustering Lecture 9 Marina Santini Acknowledgements Slides borrowed and adapted from: Data Mining by I. H. Witten, E. Frank and M. A. Hall 1 Lecture 9: Required Reading Witten et al. (2011:
CS 1675 Introduction to Machine Learning Lecture 18. Clustering. Clustering. Groups together similar instances in the data sample
CS 1675 Introduction to Machine Learning Lecture 18 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem:
Flat Clustering. Slides are mostly from Hinrich Schütze. March 27, 2017
Flat Clustering Slides are mostly from Hinrich Schütze March 7, 07 / 79 Overview Recap Clustering: Introduction 3 Clustering in IR 4 K-means 5 Evaluation 6 How many clusters? / 79 Outline Recap Clustering:
Clustering. Chapter 10 in Introduction to statistical learning
Clustering Chapter 10 in Introduction to statistical learning 16 14 12 10 8 6 4 2 0 2 4 6 8 10 12 14 1 Clustering ² Clustering is the art of finding groups in data (Kaufman and Rousseeuw, 1990). ² What
CLUSTERING. CSE 634 Data Mining Prof. Anita Wasilewska TEAM 16
CLUSTERING CSE 634 Data Mining Prof. Anita Wasilewska TEAM 16 1. K-medoids: REFERENCES https://www.coursera.org/learn/cluster-analysis/lecture/nj0sb/3-4-the-k-medoids-clustering-method https://anuradhasrinivas.files.wordpress.com/2013/04/lesson8-clustering.pdf
Information Retrieval and Web Search Engines
Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 4th, 2014 Wolf-Tilo Balke and José Pinto Institut für Informationssysteme Technische Universität Braunschweig The Cluster
Clustering in Data Mining
Clustering in Data Mining Classification Vs Clustering When the distribution is based on a single parameter and that parameter is known for each object, it is called classification. E.g. Children, young,
DS504/CS586: Big Data Analytics Big Data Clustering Prof. Yanhua Li
Welcome to DS504/CS586: Big Data Analytics Big Data Clustering Prof. Yanhua Li Time: 6:00pm 8:50pm Thu Location: AK 232 Fall 2016 High Dimensional Data v Given a cloud of data points we want to understand
Clustering Part 2. A Partitional Clustering
Universit of Florida CISE department Gator Engineering Clustering Part Dr. Sanja Ranka Professor Computer and Information Science and Engineering Universit of Florida, Gainesville Universit of Florida
CHAPTER 4 K-MEANS AND UCAM CLUSTERING ALGORITHM
CHAPTER 4 K-MEANS AND UCAM CLUSTERING 4.1 Introduction ALGORITHM Clustering has been used in a number of applications such as engineering, biology, medicine and data mining. The most popular clustering
Understanding Clustering Supervising the unsupervised
Understanding Clustering Supervising the unsupervised Janu Verma IBM T.J. Watson Research Center, New York http://jverma.github.io/ jverma@us.ibm.com @januverma Clustering Grouping together similar data
Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.
Professor William Hoff Dept of Electrical Engineering &Computer Science http://inside.mines.edu/~whoff/ 1 Image Segmentation Some material for these slides comes from https://www.csd.uwo.ca/courses/cs4487a/
K-Means Clustering Using Localized Histogram Analysis
K-Means Clustering Using Localized Histogram Analysis Michael Bryson University of South Carolina, Department of Computer Science Columbia, SC brysonm@cse.sc.edu Abstract. The first step required for many
Machine Learning : Clustering, Self-Organizing Maps
Machine Learning Clustering, Self-Organizing Maps 12/12/2013 Machine Learning : Clustering, Self-Organizing Maps Clustering The task: partition a set of objects into meaningful subsets (clusters). The
http://www.xkcd.com/233/ Text Clustering David Kauchak cs160 Fall 2009 adapted from: http://www.stanford.edu/class/cs276/handouts/lecture17-clustering.ppt Administrative 2 nd status reports Paper review