Association Rule Mining and Clustering
|
|
- Alannah Chandler
- 6 years ago
- Views:
Transcription
1 Association Rule Mining and Clustering Lecture Outline: Classification vs. Association Rule Mining vs. Clustering Association Rule Mining Clustering Types of Clusters Clustering Algorithms Hierarchical: agglomerative, divisive Non-hierarchical: k-means Reading: Chapters 3.4, 3.9, 4.5, 4.8, 6.6 Witten and Frank, 2nd ed. Chapter 14, Foundations of Statistical Language Processing, C.D. Manning & H. Schütze, MIT Press, 1999 COM3250 /
2 Classification vs. Association Rule Mining vs. Clustering So far we have primarily focused on classification: Given: a set of training examples represented as pairs of attribute value vectors (instance representations) + a designated target class Learn: how to predict the target class of an unseen instance Example: learn to distinguish edible/poisonous mushrooms; credit-worthy loan applicants Works well if we understand which attributes are likely to predict others and/or we have a clear-cut classification task in mind. However in other cases there may be no distinguished class attribute. COM3250 /
3 Classification vs. Association Rule Mining vs. Clustering So far we have primarily focused on classification: Given: a set of training examples represented as pairs of attribute value vectors (instance representations) + a designated target class Learn: how to predict the target class of an unseen instance Example: learn to distinguish edible/poisonous mushrooms; credit-worthy loan applicants Works well if we understand which attributes are likely to predict others and/or we have a clear-cut classification task in mind. However in other cases there may be no distinguished class attribute. We may want to learn association rules capturing regularities underlying a dataset: Given: set of training examples represented as attribute value vectors Learn: if-then rules expressing significant associations between attributes Example: learn associations between items consumers buy at the supermarket COM3250 / a
4 Classification vs. Association Rule Mining vs. Clustering So far we have primarily focused on classification: Given: a set of training examples represented as pairs of attribute value vectors (instance representations) + a designated target class Learn: how to predict the target class of an unseen instance Example: learn to distinguish edible/poisonous mushrooms; credit-worthy loan applicants Works well if we understand which attributes are likely to predict others and/or we have a clear-cut classification task in mind. However in other cases there may be no distinguished class attribute. We may want to learn association rules capturing regularities underlying a dataset: Given: set of training examples represented as attribute value vectors Learn: if-then rules expressing significant associations between attributes Example: learn associations between items consumers buy at the supermarket We may want discover clusters in our data either to understand the data or to train classifiers Given: set of training examples + a similarity measure Learn: a set of clusters capturing significant groupings amongst instances Example: cluster documents returned by a search engine COM3250 / b
5 Association Rule Mining Could use rule learning methods studied earlier: consider each possible attribute + value and each possible combination of attributes + values as a potential consequent (RHS) of an if-then rule run a rule induction process to induce rules for each such consequent then prune resulting association rules by coverage number of instances rules correctly predicts (also called support); and accuracy proportion of instances to which the rule applies which it correctly predicts (also called confidence) COM3250 /
6 Association Rule Mining Could use rule learning methods studied earlier: consider each possible attribute + value and each possible combination of attributes + values as a potential consequent (RHS) of an if-then rule run a rule induction process to induce rules for each such consequent then prune resulting association rules by coverage number of instances rules correctly predicts (also called support); and accuracy proportion of instances to which the rule applies which it correctly predicts (also called confidence) However, given the combinatorics such an approach is computationally infeasible... COM3250 / a
7 Association Rule Mining (cont) Instead, assume we are only interested in rules with some minimum coverage Look for combinations of attribute-value pairs with pre-specified minimum coverage called item sets, where an item is an attribute-value pair (terminology borrowed from market basket analyis, where associations are sought between items customers buy) This approach followed by the Apriori association rule miner in Weka (Agrawal et al.) Sequentially generate all 1-item, 2-item, 3-item,... n-item sets that have minimum coverage This can be done efficiently by observing that an n-item set can achieve minimum coverage only if all of the n 1-item sets which are subsets of the n-item set have minimum coverage Example: in the PlayTennis dataset the 3-item set {humidity=normal, windy=false, play=yes} has coverage 4 (i.e. these three attribute value pairs are true of 4 instances). COM3250 /
8 Association Rule Mining (cont) Next, form rules by considering for each minimum coverage item set all possible rules containing 0 or more attribute value pairs from the item set in the antecedent and one or more attribute value pairs from the item set in the consequent. From the 3-item set {humidity=normal, windy=false, play=yes} generate 7 rules: Association Rule IF humidity=normal windy=false THEN play=yes 4/4 IF humidity=normal play=yes THEN windy=false 4/6 IF windy=false play=yes THEN humidity=normal 4/6 IF humidity=normal THEN windy=false play=yes 4/7 IF windy=false THEN humidity=normal play=yes 4/8 IF play=yes THEN humidity=normal windy=false 4/9 IF -- THEN humidity=normal windy=false play=yes 4/12 Keep only those rules that meet pre-specified desired accuracy e.g. in this example only first rule kept if accuracy of 100% is specified For PlayTennis dataset there are: 3 rules with coverage 4 and accuracy 100% 5 rules with coverage 3 and accuracy 100% 50 rules with coverage 2 and accuracy 100% COM3250 / Accuracy
9 Types of Clusters Approaches to clustering can be characterised in various ways One characterisation is by the type of clusters produced clusters may be: 1. partitions of the instance space each instance is assigned to exactly one cluster 2. overlapping subsets of the instance space instances may belong to more than one cluster 3. probabilities of cluster membership associated with instance each instance has some probability of belonging to each cluster 4. hierarchical structures any given cluster may consist of subclusters or instances or both (1) (2) e d c d e a k g j h i f b a k j g h c i b f (3) (4) a b c d e f g h g a c i e d k b j f COM3250 /
10 Clustering Algorithms: A Taxonomy Hierarchical clustering Agglomerative: bottom up start with individual instances and group the most similar Divisive: top down start with all instances in a single cluster and divide into groups so as to maximize within group similarity Mixed: start with individual instances and either add to existing cluster or form new cluster, possibly merging or splitting instances in existing clusters (CobWeb) Non-hierarchical ( flat ) clustering Partitioning approaches hypothesise k clusters, randomly pick cluster centres, and iteratively assign instances to centres and recompute centres until stable Probabilistic approaches hypothesise k clusters each with an associated (initially guessed) probability distribution of attribute values for instances in the cluster, then iteratively compute cluster probabilities for each instance and recompute cluster parameters until stability Incremental vs batch clustering: are clusters computed dynamically as instances become available (CobWeb) or statically on presumption whole instance set is available? COM3250 /
11 Hierarchical Clustering: Agglomerative Clustering Given: a set X = {x 1,...,x n } of instances + a function sim: 2 X 2 X R for i = 1 to n endfor c i {x i } C {c 1,...,c n } j n+1 while C > 1 return C (c n1,c n2 ) argmax (cu,c v ) C Csim(c u,c v ) c j c n1 c n2 C C \ {c n1,c n2 } {c j } j j+ 1 (Manning & Schütze, p. 502) Start with a separate cluster for each instance Repeatedly determine two most similar clusters and merge them together Terminate when a single cluster containing all instances has been formed COM3250 /
12 Hierarchical Clustering: Divisive Clustering Given: a set X = {x 1,...,x n } of instances + a function coh: 2 X R + a function split: 2 X 2 X 2 X C {X} (= {c 1 }) j 1 while c i C s.t. c i > 1 return C c u argmin cv C coh(c v ) (c j+1,c j+2 ) = split(c u ) C C \ {c u } {c j+1,c j+2 } j j+ 2 (Manning & Schütze, p. 502) Start with a single cluster containing all instances Repeatedly determine least coherent cluster and split into two subclusters Terminate when no cluster contains more than one instance COM3250 /
13 Similarity Functions used in Clustering (1) Single Link: similarity between two clusters = similarity of the two most similar members Complete Link: similarity between two clusters = similarity of the two least similar members Group average: similarity between two clusters = average similarity between members 5 a b c d 4 3 d 2d 2 1 3/2 d e f g h Mannning & Schütze, pp COM3250 /
14 Similarity Functions used in Clustering (2) Best initial move is to merge a/b, c/d, e/ f and g/h since the similarities between these objects are greatest (assume similarity reciprocally related to distance) 5 a b c d 4 3 d 2d 2 1 3/2 d e f g h Mannning & Schütze, pp COM3250 /
15 Similarity Functions used in Clustering (3) Using single link clustering the clusters {a,b} and {c,d} and also {e, f } and {g,h} are merged next since the pairs b/c and f/g are closer than other pairs not in the same cluster (e.g. than b/ f or c/g) members Single link clustering results in elongated clusters ( chaining effect ) that are locally coherent in that close objects are in same cluster However, may have poor global quality a is closer to e than to d, but a and d are in same cluster while a and e are not. 5 a b c d 4 3 d 2d 2 1 3/2 d e f g h Mannning & Schütze, pp COM3250 /
16 Similarity Functions used in Clustering (4) Complete link clustering avoids this problem by focusing on global rather than local quality similarity of two clusters is the similarity of their two most dissimilar members Results in tighter clusters in the example than single link similarity minimally similar pairs for complete link clusters (a/ f or b/e) closer than minimally similar pairs for single link clusters (a/d) 5 a b c d 4 3 d 2d 2 1 3/2 d e f g h Mannning & Schütze, pp COM3250 /
17 Similarity Functions used in Clustering (5) Unfortunately complete link clustering has time complexity O(n 3 ) single link clustering is O(n 2 ). Group average clustering is a compromise that is O(n 2 ) but avoids the elongated clusters of single link clustering. The average similarity between vectors in a cluster c is defined as: S(c) = 1 c ( c 1) x c sim( x, y) x y c At each iteration the clustering algorithm picks two clusters c u and c v that maximize S(c u c v ) To carry out group average clustering efficiently care must be taken to avoid recomputing average similarities from scratch after each of the merging steps can avoid doing this representing instances as length-normalised vectors in m-dimensional real-valued space and using cosine similarity measure given this approach average similarity of a cluster can be computed in constant time from the average similarity of its two children see Manning and Schütze for details COM3250 /
18 Similarity Functions used in Clustering (6) In top down hierarchical clustering a measure is needed for cluster coherence and an operation to split clusters must be defined The similarity measures already defined for bottom up clustering can be used for these tasks Coherence can be defined as the smallest similarity in the minimum spanning tree for the cluster (tree connecting all instances the sum of whose edge lengths is minimal) according to the single link similarity measure the smallest similarity between any two instances in the cluster, according to the complete link measure the average similarity between objects in the cluster, according to the group average measure Once the least coherent cluster is identified it needs to be split Splitting can be seen as a clustering task find two subclusters of a given cluster any clustering algorithm can be used for this task COM3250 /
19 Non-hierarchical Clustering Non-hierarchical clustering algorithms typically start with a partition based on randomly selected seeds and then iteratively refine this partition by reallocating instances to current best cluster contrast with hierarchical algorithms which typically require only one pass COM3250 /
20 Non-hierarchical Clustering Non-hierarchical clustering algorithms typically start with a partition based on randomly selected seeds and then iteratively refine this partition by reallocating instances to current best cluster contrast with hierarchical algorithms which typically require only one pass Termination occurs when according to some measure of goodness clusters are no longer improving measures of goodness include: group average similarity; mutual information between adjacent clusters; likelihood of data given the clustering model COM3250 / a
21 Non-hierarchical Clustering Non-hierarchical clustering algorithms typically start with a partition based on randomly selected seeds and then iteratively refine this partition by reallocating instances to current best cluster contrast with hierarchical algorithms which typically require only one pass Termination occurs when according to some measure of goodness clusters are no longer improving measures of goodness include: group average similarity; mutual information between adjacent clusters; likelihood of data given the clustering model How many clusters? may have some prior knowledge about right number of clusters can try various cluster numbers n and see how measures of cluster goodness compare or if there is a reduction in the rate of increase of goodness for some n can use Minimum Description Length to minimize sum of lengths of encodings of instances in terms of distance from clusters + encodings of clusters COM3250 / b
22 Non-hierarchical Clustering Non-hierarchical clustering algorithms typically start with a partition based on randomly selected seeds and then iteratively refine this partition by reallocating instances to current best cluster contrast with hierarchical algorithms which typically require only one pass Termination occurs when according to some measure of goodness clusters are no longer improving measures of goodness include: group average similarity; mutual information between adjacent clusters; likelihood of data given the clustering model How many clusters? may have some prior knowledge about right number of clusters can try various cluster numbers n and see how measures of cluster goodness compare or if there is a reduction in the rate of increase of goodness for some n can use Minimum Description Length to minimize sum of lengths of encodings of instances in terms of distance from clusters + encodings of clusters Hierarchical clustering does not require number of clusters to be determined however full hierarchical clusterings are rarely usable and tree must be cut at some point to specifying a number of clusters COM3250 / c
23 K-means Clustering Given: a set X = { x 1,..., x n } R m + a distance measure: d : R m R m R + a function for computing the mean µ : P(R) R m select k initial centres f 1,..., f k while stopping criterion is not true for all clusters c j c j = { x i f l d( x i, f j ) d( x i, f l )} end for for all means f j f j = µ(c j ) end for end while (Manning & Schütze, p. 516) The algorithm picks k initial centres and forms clusters by allocating each instance to its nearest centre Centres for each cluster are recomputed as the centroid or mean of the cluster s members: µ= (1/ c j ) x c j x and instances are once again allocated to their nearest centre The algorithm iterates until stability or some measure of goodness is attained COM3250 /
24 K-means Clustering Movement of Cluster Centres mount/projects/kmeans/images/centers.gif COM3250 /
25 Probability-based Clustering and The EM Algorithm In probability-based clustering an instance is not placed categorically in a single cluster, but rather is assigned a probability of belonging to every cluster COM3250 /
26 Probability-based Clustering and The EM Algorithm In probability-based clustering an instance is not placed categorically in a single cluster, but rather is assigned a probability of belonging to every cluster Basis of statistical clustering is the finite mixture model mixture of k probability distributions representing k clusters each distribution gives probability that an instance would have a certain set of attribute values if it were known to be a member of that cluster COM3250 / a
27 Probability-based Clustering and The EM Algorithm In probability-based clustering an instance is not placed categorically in a single cluster, but rather is assigned a probability of belonging to every cluster Basis of statistical clustering is the finite mixture model mixture of k probability distributions representing k clusters each distribution gives probability that an instance would have a certain set of attribute values if it were known to be a member of that cluster Clustering problem is to take a set of instances and a pre-specified number of clusters and work out each cluster s mean and variance and the population distribution between clusters COM3250 / b
28 Probability-based Clustering and The EM Algorithm In probability-based clustering an instance is not placed categorically in a single cluster, but rather is assigned a probability of belonging to every cluster Basis of statistical clustering is the finite mixture model mixture of k probability distributions representing k clusters each distribution gives probability that an instance would have a certain set of attribute values if it were known to be a member of that cluster Clustering problem is to take a set of instances and a pre-specified number of clusters and work out each cluster s mean and variance and the population distribution between clusters EM Expectation-Maximisation is an algorithm for doing this Like k-means start with guess for parameters governing clusters Use these parameters to calculate cluster probabalities for each instance (expectation of class values) Use these cluster probabilities for each instance to re-estimate cluster parameters (maximisation of the likelihood of the distributions given the data) Terminate when some goodness measure is met usually when increase in log likelihood that data came from the finite mixture model is negligible between iterations COM3250 / c
29 Summary While classification learning to predict an instance s class given a set of attribute values is central to machine learning it is not the only task of interest/value COM3250 /
30 Summary While classification learning to predict an instance s class given a set of attribute values is central to machine learning it is not the only task of interest/value When there is no clearly distinguished class attribute we may want to learn association rules reflecting regularities underlying the data discover clusters in the data COM3250 / a
31 Summary While classification learning to predict an instance s class given a set of attribute values is central to machine learning it is not the only task of interest/value When there is no clearly distinguished class attribute we may want to learn association rules reflecting regularities underlying the data discover clusters in the data Association rules can be learned by a procedure which identifies sets of attribute-value pairs which occur together sufficiently often to be of interest proposes rules relating these attribute-value pairs whose accuracy over the data set is sufficiently high as to be useful COM3250 / b
32 Summary While classification learning to predict an instance s class given a set of attribute values is central to machine learning it is not the only task of interest/value When there is no clearly distinguished class attribute we may want to learn association rules reflecting regularities underlying the data discover clusters in the data Association rules can be learned by a procedure which identifies sets of attribute-value pairs which occur together sufficiently often to be of interest proposes rules relating these attribute-value pairs whose accuracy over the data set is sufficiently high as to be useful Clusters which can hard or soft, hierarchical or non-hierarchical can be discovered using a variety of algorithms including: for hierarchical clusters: agglomerative or divisive clustering for non-hierarchical clusters: k-means or EM COM3250 / c
INF4820. Clustering. Erik Velldal. Nov. 17, University of Oslo. Erik Velldal INF / 22
INF4820 Clustering Erik Velldal University of Oslo Nov. 17, 2009 Erik Velldal INF4820 1 / 22 Topics for Today More on unsupervised machine learning for data-driven categorization: clustering. The task
More informationCOMS 4771 Clustering. Nakul Verma
COMS 4771 Clustering Nakul Verma Supervised Learning Data: Supervised learning Assumption: there is a (relatively simple) function such that for most i Learning task: given n examples from the data, find
More informationBased on Raymond J. Mooney s slides
Instance Based Learning Based on Raymond J. Mooney s slides University of Texas at Austin 1 Example 2 Instance-Based Learning Unlike other learning algorithms, does not involve construction of an explicit
More informationINF4820, Algorithms for AI and NLP: Evaluating Classifiers Clustering
INF4820, Algorithms for AI and NLP: Evaluating Classifiers Clustering Erik Velldal University of Oslo Sept. 18, 2012 Topics for today 2 Classification Recap Evaluating classifiers Accuracy, precision,
More informationCHAPTER 4: CLUSTER ANALYSIS
CHAPTER 4: CLUSTER ANALYSIS WHAT IS CLUSTER ANALYSIS? A cluster is a collection of data-objects similar to one another within the same group & dissimilar to the objects in other groups. Cluster analysis
More informationCS 2750 Machine Learning. Lecture 19. Clustering. CS 2750 Machine Learning. Clustering. Groups together similar instances in the data sample
Lecture 9 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem: distribute data into k different groups
More informationLecture 6: Unsupervised Machine Learning Dagmar Gromann International Center For Computational Logic
SEMANTIC COMPUTING Lecture 6: Unsupervised Machine Learning Dagmar Gromann International Center For Computational Logic TU Dresden, 23 November 2018 Overview Unsupervised Machine Learning overview Association
More informationhttp://www.xkcd.com/233/ Text Clustering David Kauchak cs160 Fall 2009 adapted from: http://www.stanford.edu/class/cs276/handouts/lecture17-clustering.ppt Administrative 2 nd status reports Paper review
More informationMachine Learning. Unsupervised Learning. Manfred Huber
Machine Learning Unsupervised Learning Manfred Huber 2015 1 Unsupervised Learning In supervised learning the training data provides desired target output for learning In unsupervised learning the training
More informationMIA - Master on Artificial Intelligence
MIA - Master on Artificial Intelligence 1 Hierarchical Non-hierarchical Evaluation 1 Hierarchical Non-hierarchical Evaluation The Concept of, proximity, affinity, distance, difference, divergence We use
More informationInformation Retrieval and Web Search Engines
Information Retrieval and Web Search Engines Lecture 7: Document Clustering December 4th, 2014 Wolf-Tilo Balke and José Pinto Institut für Informationssysteme Technische Universität Braunschweig The Cluster
More informationCluster Analysis. Ying Shen, SSE, Tongji University
Cluster Analysis Ying Shen, SSE, Tongji University Cluster analysis Cluster analysis groups data objects based only on the attributes in the data. The main objective is that The objects within a group
More informationINF4820 Algorithms for AI and NLP. Evaluating Classifiers Clustering
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal & Stephan Oepen Language Technology Group (LTG) September 23, 2015 Agenda Last week Supervised vs unsupervised learning.
More informationIBL and clustering. Relationship of IBL with CBR
IBL and clustering Distance based methods IBL and knn Clustering Distance based and hierarchical Probability-based Expectation Maximization (EM) Relationship of IBL with CBR + uses previously processed
More informationIntroduction to Mobile Robotics
Introduction to Mobile Robotics Clustering Wolfram Burgard Cyrill Stachniss Giorgio Grisetti Maren Bennewitz Christian Plagemann Clustering (1) Common technique for statistical data analysis (machine learning,
More informationUnsupervised Learning
Outline Unsupervised Learning Basic concepts K-means algorithm Representation of clusters Hierarchical clustering Distance functions Which clustering algorithm to use? NN Supervised learning vs. unsupervised
More informationClustering. Partition unlabeled examples into disjoint subsets of clusters, such that:
Text Clustering 1 Clustering Partition unlabeled examples into disjoint subsets of clusters, such that: Examples within a cluster are very similar Examples in different clusters are very different Discover
More informationUnsupervised: no target value to predict
Clustering Unsupervised: no target value to predict Differences between models/algorithms: Exclusive vs. overlapping Deterministic vs. probabilistic Hierarchical vs. flat Incremental vs. batch learning
More informationData Informatics. Seon Ho Kim, Ph.D.
Data Informatics Seon Ho Kim, Ph.D. seonkim@usc.edu Clustering Overview Supervised vs. Unsupervised Learning Supervised learning (classification) Supervision: The training data (observations, measurements,
More information10701 Machine Learning. Clustering
171 Machine Learning Clustering What is Clustering? Organizing data into clusters such that there is high intra-cluster similarity low inter-cluster similarity Informally, finding natural groupings among
More informationClustering. CE-717: Machine Learning Sharif University of Technology Spring Soleymani
Clustering CE-717: Machine Learning Sharif University of Technology Spring 2016 Soleymani Outline Clustering Definition Clustering main approaches Partitional (flat) Hierarchical Clustering validation
More informationCSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Cluster Analysis: Basic Concepts and Methods Huan Sun, CSE@The Ohio State University Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han 2 Chapter 10. Cluster
More informationClustering Lecture 5: Mixture Model
Clustering Lecture 5: Mixture Model Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics
More informationCSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Cluster Analysis: Basic Concepts and Methods Huan Sun, CSE@The Ohio State University 09/25/2017 Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han 2 Chapter 10.
More informationUnsupervised Learning
Unsupervised Learning Unsupervised learning Until now, we have assumed our training samples are labeled by their category membership. Methods that use labeled samples are said to be supervised. However,
More informationSupervised and Unsupervised Learning (II)
Supervised and Unsupervised Learning (II) Yong Zheng Center for Web Intelligence DePaul University, Chicago IPD 346 - Data Science for Business Program DePaul University, Chicago, USA Intro: Supervised
More informationInformation Retrieval and Web Search Engines
Information Retrieval and Web Search Engines Lecture 7: Document Clustering May 25, 2011 Wolf-Tilo Balke and Joachim Selke Institut für Informationssysteme Technische Universität Braunschweig Homework
More informationClustering. Supervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
More informationCluster Evaluation and Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University
Cluster Evaluation and Expectation Maximization! adapted from: Doug Downey and Bryan Pardo, Northwestern University Kinds of Clustering Sequential Fast Cost Optimization Fixed number of clusters Hierarchical
More informationCS 1675 Introduction to Machine Learning Lecture 18. Clustering. Clustering. Groups together similar instances in the data sample
CS 1675 Introduction to Machine Learning Lecture 18 Clustering Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square Clustering Groups together similar instances in the data sample Basic clustering problem:
More informationCHAPTER 4 K-MEANS AND UCAM CLUSTERING ALGORITHM
CHAPTER 4 K-MEANS AND UCAM CLUSTERING 4.1 Introduction ALGORITHM Clustering has been used in a number of applications such as engineering, biology, medicine and data mining. The most popular clustering
More informationCHAPTER 3 A FAST K-MODES CLUSTERING ALGORITHM TO WAREHOUSE VERY LARGE HETEROGENEOUS MEDICAL DATABASES
70 CHAPTER 3 A FAST K-MODES CLUSTERING ALGORITHM TO WAREHOUSE VERY LARGE HETEROGENEOUS MEDICAL DATABASES 3.1 INTRODUCTION In medical science, effective tools are essential to categorize and systematically
More informationData Mining. 3.3 Rule-Based Classification. Fall Instructor: Dr. Masoud Yaghini. Rule-Based Classification
Data Mining 3.3 Fall 2008 Instructor: Dr. Masoud Yaghini Outline Using IF-THEN Rules for Classification Rules With Exceptions Rule Extraction from a Decision Tree 1R Algorithm Sequential Covering Algorithms
More informationHierarchical Clustering Lecture 9
Hierarchical Clustering Lecture 9 Marina Santini Acknowledgements Slides borrowed and adapted from: Data Mining by I. H. Witten, E. Frank and M. A. Hall 1 Lecture 9: Required Reading Witten et al. (2011:
More informationClustering and Visualisation of Data
Clustering and Visualisation of Data Hiroshi Shimodaira January-March 28 Cluster analysis aims to partition a data set into meaningful or useful groups, based on distances between data points. In some
More informationWhat to come. There will be a few more topics we will cover on supervised learning
Summary so far Supervised learning learn to predict Continuous target regression; Categorical target classification Linear Regression Classification Discriminative models Perceptron (linear) Logistic regression
More informationCLUSTERING. JELENA JOVANOVIĆ Web:
CLUSTERING JELENA JOVANOVIĆ Email: jeljov@gmail.com Web: http://jelenajovanovic.net OUTLINE What is clustering? Application domains K-Means clustering Understanding it through an example The K-Means algorithm
More informationData Mining Part 5. Prediction
Data Mining Part 5. Prediction 5.4. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Using IF-THEN Rules for Classification Rule Extraction from a Decision Tree 1R Algorithm Sequential Covering Algorithms
More informationLesson 3. Prof. Enza Messina
Lesson 3 Prof. Enza Messina Clustering techniques are generally classified into these classes: PARTITIONING ALGORITHMS Directly divides data points into some prespecified number of clusters without a hierarchical
More informationMultiDimensional Signal Processing Master Degree in Ingegneria delle Telecomunicazioni A.A
MultiDimensional Signal Processing Master Degree in Ingegneria delle Telecomunicazioni A.A. 205-206 Pietro Guccione, PhD DEI - DIPARTIMENTO DI INGEGNERIA ELETTRICA E DELL INFORMAZIONE POLITECNICO DI BARI
More informationText Documents clustering using K Means Algorithm
Text Documents clustering using K Means Algorithm Mrs Sanjivani Tushar Deokar Assistant professor sanjivanideokar@gmail.com Abstract: With the advancement of technology and reduced storage costs, individuals
More informationAdministrative. Machine learning code. Supervised learning (e.g. classification) Machine learning: Unsupervised learning" BANANAS APPLES
Administrative Machine learning: Unsupervised learning" Assignment 5 out soon David Kauchak cs311 Spring 2013 adapted from: http://www.stanford.edu/class/cs276/handouts/lecture17-clustering.ppt Machine
More informationClustering. SC4/SM4 Data Mining and Machine Learning, Hilary Term 2017 Dino Sejdinovic
Clustering SC4/SM4 Data Mining and Machine Learning, Hilary Term 2017 Dino Sejdinovic Clustering is one of the fundamental and ubiquitous tasks in exploratory data analysis a first intuition about the
More informationUnsupervised Learning : Clustering
Unsupervised Learning : Clustering Things to be Addressed Traditional Learning Models. Cluster Analysis K-means Clustering Algorithm Drawbacks of traditional clustering algorithms. Clustering as a complex
More informationK-Means Clustering 3/3/17
K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data points. We want to find underlying structure in the data. Examples: Identify groups of similar data points. Clustering
More informationBBS654 Data Mining. Pinar Duygulu. Slides are adapted from Nazli Ikizler
BBS654 Data Mining Pinar Duygulu Slides are adapted from Nazli Ikizler 1 Classification Classification systems: Supervised learning Make a rational prediction given evidence There are several methods for
More informationINF4820 Algorithms for AI and NLP. Evaluating Classifiers Clustering
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Murhaf Fares & Stephan Oepen Language Technology Group (LTG) September 27, 2017 Today 2 Recap Evaluation of classifiers Unsupervised
More informationClustering: Overview and K-means algorithm
Clustering: Overview and K-means algorithm Informal goal Given set of objects and measure of similarity between them, group similar objects together K-Means illustrations thanks to 2006 student Martin
More informationIntro to Artificial Intelligence
Intro to Artificial Intelligence Ahmed Sallam { Lecture 5: Machine Learning ://. } ://.. 2 Review Probabilistic inference Enumeration Approximate inference 3 Today What is machine learning? Supervised
More informationClustering. CS294 Practical Machine Learning Junming Yin 10/09/06
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means,
More informationMachine learning - HT Clustering
Machine learning - HT 2016 10. Clustering Varun Kanade University of Oxford March 4, 2016 Announcements Practical Next Week - No submission Final Exam: Pick up on Monday Material covered next week is not
More informationClustering CS 550: Machine Learning
Clustering CS 550: Machine Learning This slide set mainly uses the slides given in the following links: http://www-users.cs.umn.edu/~kumar/dmbook/ch8.pdf http://www-users.cs.umn.edu/~kumar/dmbook/dmslides/chap8_basic_cluster_analysis.pdf
More informationUnsupervised Learning: Clustering
Unsupervised Learning: Clustering Vibhav Gogate The University of Texas at Dallas Slides adapted from Carlos Guestrin, Dan Klein & Luke Zettlemoyer Machine Learning Supervised Learning Unsupervised Learning
More informationClustering Results. Result List Example. Clustering Results. Information Retrieval
Information Retrieval INFO 4300 / CS 4300! Presenting Results Clustering Clustering Results! Result lists often contain documents related to different aspects of the query topic! Clustering is used to
More informationUnsupervised Learning and Clustering
Unsupervised Learning and Clustering Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2009 CS 551, Spring 2009 c 2009, Selim Aksoy (Bilkent University)
More informationECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
More informationClustering & Bootstrapping
Clustering & Bootstrapping Jelena Prokić University of Groningen The Netherlands March 25, 2009 Groningen Overview What is clustering? Various clustering algorithms Bootstrapping Application in dialectometry
More informationData Mining. Covering algorithms. Covering approach At each stage you identify a rule that covers some of instances. Fig. 4.
Data Mining Chapter 4. Algorithms: The Basic Methods (Covering algorithm, Association rule, Linear models, Instance-based learning, Clustering) 1 Covering approach At each stage you identify a rule that
More information4. Ad-hoc I: Hierarchical clustering
4. Ad-hoc I: Hierarchical clustering Hierarchical versus Flat Flat methods generate a single partition into k clusters. The number k of clusters has to be determined by the user ahead of time. Hierarchical
More informationData Mining. Lesson 9 Support Vector Machines. MSc in Computer Science University of New York Tirana Assoc. Prof. Dr.
Data Mining Lesson 9 Support Vector Machines MSc in Computer Science University of New York Tirana Assoc. Prof. Dr. Marenglen Biba Data Mining: Content Introduction to data mining and machine learning
More informationClustering algorithms
Clustering algorithms Machine Learning Hamid Beigy Sharif University of Technology Fall 1393 Hamid Beigy (Sharif University of Technology) Clustering algorithms Fall 1393 1 / 22 Table of contents 1 Supervised
More informationMachine Learning: Symbol-based
10c Machine Learning: Symbol-based 10.0 Introduction 10.1 A Framework for Symbol-based Learning 10.2 Version Space Search 10.3 The ID3 Decision Tree Induction Algorithm 10.4 Inductive Bias and Learnability
More informationIntroduction to Information Retrieval
Introduction to Information Retrieval http://informationretrieval.org IIR 6: Flat Clustering Wiltrud Kessler & Hinrich Schütze Institute for Natural Language Processing, University of Stuttgart 0-- / 83
More informationECLT 5810 Clustering
ECLT 5810 Clustering What is Cluster Analysis? Cluster: a collection of data objects Similar to one another within the same cluster Dissimilar to the objects in other clusters Cluster analysis Grouping
More informationNatural Language Processing
Natural Language Processing Machine Learning Potsdam, 26 April 2012 Saeedeh Momtazi Information Systems Group Introduction 2 Machine Learning Field of study that gives computers the ability to learn without
More informationData Clustering. Danushka Bollegala
Data Clustering Danushka Bollegala Outline Why cluster data? Clustering as unsupervised learning Clustering algorithms k-means, k-medoids agglomerative clustering Brown s clustering Spectral clustering
More informationINF4820, Algorithms for AI and NLP: Hierarchical Clustering
INF4820, Algorithms for AI and NLP: Hierarchical Clustering Erik Velldal University of Oslo Sept. 25, 2012 Agenda Topics we covered last week Evaluating classifiers Accuracy, precision, recall and F-score
More informationCS490W. Text Clustering. Luo Si. Department of Computer Science Purdue University
CS490W Text Clustering Luo Si Department of Computer Science Purdue University [Borrows slides from Chris Manning, Ray Mooney and Soumen Chakrabarti] Clustering Document clustering Motivations Document
More informationClustering CE-324: Modern Information Retrieval Sharif University of Technology
Clustering CE-324: Modern Information Retrieval Sharif University of Technology M. Soleymani Fall 2014 Most slides have been adapted from: Profs. Manning, Nayak & Raghavan (CS-276, Stanford) Ch. 16 What
More informationFinding Clusters 1 / 60
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-means Density Based Clustering, e.g. DBScan Grid Based Clustering 1 / 60
More informationSupervised vs. Unsupervised Learning
Clustering Supervised vs. Unsupervised Learning So far we have assumed that the training samples used to design the classifier were labeled by their class membership (supervised learning) We assume now
More informationDATA MINING LECTURE 7. Hierarchical Clustering, DBSCAN The EM Algorithm
DATA MINING LECTURE 7 Hierarchical Clustering, DBSCAN The EM Algorithm CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related)
More informationChapter 4: Algorithms CS 795
Chapter 4: Algorithms CS 795 Inferring Rudimentary Rules 1R Single rule one level decision tree Pick each attribute and form a single level tree without overfitting and with minimal branches Pick that
More informationINTRODUCTION TO DATA MINING. Daniel Rodríguez, University of Alcalá
INTRODUCTION TO DATA MINING Daniel Rodríguez, University of Alcalá Outline Knowledge Discovery in Datasets Model Representation Types of models Supervised Unsupervised Evaluation (Acknowledgement: Jesús
More informationHierarchical Clustering 4/5/17
Hierarchical Clustering 4/5/17 Hypothesis Space Continuous inputs Output is a binary tree with data points as leaves. Useful for explaining the training data. Not useful for making new predictions. Direction
More informationCSE 255 Lecture 6. Data Mining and Predictive Analytics. Community Detection
CSE 255 Lecture 6 Data Mining and Predictive Analytics Community Detection Dimensionality reduction Goal: take high-dimensional data, and describe it compactly using a small number of dimensions Assumption:
More informationECG782: Multidimensional Digital Signal Processing
ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting
More informationHard clustering. Each object is assigned to one and only one cluster. Hierarchical clustering is usually hard. Soft (fuzzy) clustering
An unsupervised machine learning problem Grouping a set of objects in such a way that objects in the same group (a cluster) are more similar (in some sense or another) to each other than to those in other
More informationMSA220 - Statistical Learning for Big Data
MSA220 - Statistical Learning for Big Data Lecture 13 Rebecka Jörnsten Mathematical Sciences University of Gothenburg and Chalmers University of Technology Clustering Explorative analysis - finding groups
More informationKeywords Clustering, Goals of clustering, clustering techniques, clustering algorithms.
Volume 3, Issue 5, May 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Survey of Clustering
More informationClustering and Dissimilarity Measures. Clustering. Dissimilarity Measures. Cluster Analysis. Perceptually-Inspired Measures
Clustering and Dissimilarity Measures Clustering APR Course, Delft, The Netherlands Marco Loog May 19, 2008 1 What salient structures exist in the data? How many clusters? May 19, 2008 2 Cluster Analysis
More informationMachine Learning. B. Unsupervised Learning B.1 Cluster Analysis. Lars Schmidt-Thieme
Machine Learning B. Unsupervised Learning B.1 Cluster Analysis Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University of Hildesheim, Germany
More informationA Comparative study of Clustering Algorithms using MapReduce in Hadoop
A Comparative study of Clustering Algorithms using MapReduce in Hadoop Dweepna Garg 1, Khushboo Trivedi 2, B.B.Panchal 3 1 Department of Computer Science and Engineering, Parul Institute of Engineering
More informationLecture-17: Clustering with K-Means (Contd: DT + Random Forest)
Lecture-17: Clustering with K-Means (Contd: DT + Random Forest) Medha Vidyotma April 24, 2018 1 Contd. Random Forest For Example, if there are 50 scholars who take the measurement of the length of the
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Clustering and EM Barnabás Póczos & Aarti Singh Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K-
More informationLecture 7: Segmentation. Thursday, Sept 20
Lecture 7: Segmentation Thursday, Sept 20 Outline Why segmentation? Gestalt properties, fun illusions and/or revealing examples Clustering Hierarchical K-means Mean Shift Graph-theoretic Normalized cuts
More informationFlat Clustering. Slides are mostly from Hinrich Schütze. March 27, 2017
Flat Clustering Slides are mostly from Hinrich Schütze March 7, 07 / 79 Overview Recap Clustering: Introduction 3 Clustering in IR 4 K-means 5 Evaluation 6 How many clusters? / 79 Outline Recap Clustering:
More informationCSE 7/5337: Information Retrieval and Web Search Document clustering I (IIR 16)
CSE 7/5337: Information Retrieval and Web Search Document clustering I (IIR 16) Michael Hahsler Southern Methodist University These slides are largely based on the slides by Hinrich Schütze Institute for
More informationMulti-label classification using rule-based classifier systems
Multi-label classification using rule-based classifier systems Shabnam Nazmi (PhD candidate) Department of electrical and computer engineering North Carolina A&T state university Advisor: Dr. A. Homaifar
More informationCOMP 551 Applied Machine Learning Lecture 13: Unsupervised learning
COMP 551 Applied Machine Learning Lecture 13: Unsupervised learning Associate Instructor: Herke van Hoof (herke.vanhoof@mail.mcgill.ca) Slides mostly by: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551
More informationIntroduction to Machine Learning. Xiaojin Zhu
Introduction to Machine Learning Xiaojin Zhu jerryzhu@cs.wisc.edu Read Chapter 1 of this book: Xiaojin Zhu and Andrew B. Goldberg. Introduction to Semi- Supervised Learning. http://www.morganclaypool.com/doi/abs/10.2200/s00196ed1v01y200906aim006
More informationClustering. Bruno Martins. 1 st Semester 2012/2013
Departamento de Engenharia Informática Instituto Superior Técnico 1 st Semester 2012/2013 Slides baseados nos slides oficiais do livro Mining the Web c Soumen Chakrabarti. Outline 1 Motivation Basic Concepts
More informationClustering. Informal goal. General types of clustering. Applications: Clustering in information search and analysis. Example applications in search
Informal goal Clustering Given set of objects and measure of similarity between them, group similar objects together What mean by similar? What is good grouping? Computation time / quality tradeoff 1 2
More informationOlmo S. Zavala Romero. Clustering Hierarchical Distance Group Dist. K-means. Center of Atmospheric Sciences, UNAM.
Center of Atmospheric Sciences, UNAM November 16, 2016 Cluster Analisis Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster)
More informationINF 4300 Classification III Anne Solberg The agenda today:
INF 4300 Classification III Anne Solberg 28.10.15 The agenda today: More on estimating classifier accuracy Curse of dimensionality and simple feature selection knn-classification K-means clustering 28.10.15
More informationCMPUT 391 Database Management Systems. Data Mining. Textbook: Chapter (without 17.10)
CMPUT 391 Database Management Systems Data Mining Textbook: Chapter 17.7-17.11 (without 17.10) University of Alberta 1 Overview Motivation KDD and Data Mining Association Rules Clustering Classification
More informationIntroduction to Information Retrieval
Introduction to Information Retrieval http://informationretrieval.org IIR 16: Flat Clustering Hinrich Schütze Institute for Natural Language Processing, Universität Stuttgart 2009.06.16 1/ 64 Overview
More informationUnsupervised Data Mining: Clustering. Izabela Moise, Evangelos Pournaras, Dirk Helbing
Unsupervised Data Mining: Clustering Izabela Moise, Evangelos Pournaras, Dirk Helbing Izabela Moise, Evangelos Pournaras, Dirk Helbing 1 1. Supervised Data Mining Classification Regression Outlier detection
More informationUniversity of Florida CISE department Gator Engineering. Clustering Part 2
Clustering Part 2 Dr. Sanjay Ranka Professor Computer and Information Science and Engineering University of Florida, Gainesville Partitional Clustering Original Points A Partitional Clustering Hierarchical
More informationClustering Web Documents using Hierarchical Method for Efficient Cluster Formation
Clustering Web Documents using Hierarchical Method for Efficient Cluster Formation I.Ceema *1, M.Kavitha *2, G.Renukadevi *3, G.sripriya *4, S. RajeshKumar #5 * Assistant Professor, Bon Secourse College
More information