Data Mining: Clustering

Transcription

1 Bi-Clustering COMP Seminar Spring 011 Data Mining: Clustering k t 1 K-means clustering minimizes Where ist ( x, c i t i c t ) ist ( x m j 1 ( x ij i, c c t ) tj )

2 Clustering by Pattern Similarity (p-clustering) The micro-array raw ata shows 3 genes an their values in a multi-imensional space Parallel lcoorinates Plots Difficult to fin their patterns non-traitional clustering 3 Clusters Are Clear After Projection 4

3 Motivation E-Commerce: collaborative filtering Viewer Viewer Viewer Viewer Viewer Motivation rating viewer 1 viewer viewer 3 viewer 4 viewer 5 movie e 1 movie e movie e 3 movie e44 movie e 5 movie e 6 movie e 7 6

4 Motivation Viewer Viewer Viewer Viewer Viewer Motivation 8 ra ating 6 4 viewer 1 viewer 3 viewer 4 0 movie 1 movie movie 4 movie 6 8

5 9 Motivation DNA microarray analysis CH1I CH1B CH1D CHI CHB CTFC VPS EFB SSA FUN SP MDM CYS DEP NTG Motivation stre ength CH1I CH1D CHB conition 10

6 Motivation Strong coherence exhibits by the selecte objects on the selecte attributes. They are not necessarily close to each other but rather bear a constant shift. Object/attribute bias bi-cluster 11 Challenges The set of objects an the set of attributes are usually unknown. Different objects/attributes may possess ifferent biases an such biases may be local to the set of selecte objects/attributes are usually unknown in avance May have many unspecifie entries 1

7 Previous Work Subspace clustering Ientifying a set of objects an a set of attributes such that the set of objects are physically close to each other on the subspace forme by the set of attributes. Collaborative filtering: i Pearson R Only consiers global offset of each object/attribute. ( o1 o1 )( o o) ( o o1 ) ( o ( o 1 ) 13 bi-cluster Consists of a (sub)set of objects an a (sub)set of attributes Correspons to a submatrix Occupancy threshol Each object/attribute has to be fille by a certain percentage. Volume: number of specifie entries in the submatrix Base: average value of each object/attribute (in the bi-cluster) 14

8 bi-cluster 15 CH1I CH1B CH1D CHI CHB Obj base CTFC3 VPS EFB SSA1 FUN14 SP07 MDM10 CYS DEP1 NTG1 Attr base bi-cluster 16 Perfect -cluster ij ij ij ij Ij Ij ij Imperfect -cluster Resiue: r ij ij 0, ij ij Ij Ij IJ IJ IJ IJ, ij ij ij IJ Ij ij is specifie is unspecifie

9 bi-cluster The smaller the average resiue, the stronger the coherence. Objective: ientify -clusters with resiue smaller than a given threshol 17 Cheng-Church h h Algorithm Fin one bi-cluster. Replace the ata in the first bi-cluster with ranom ata Fin the secon bi-cluster, an go on. The quality of the bi-cluster egraes (smaller volume, higher resiue) ue to the insertion of ranom ata. 18

10 The FLOC algorithm Generating initial clusters Determine the best action for each row an each column Perform the best action of each row an column sequentially Improve? Y N 19 The FLOC algorithm Action: the change of membership of a row(or column) with respect to a cluster N=3 row column M= M+N actions are Performe at each iteration

11 The FLOC algorithm Gain of an action: the resiue reuction incurre by performing the action Orer of action: Fixe orer Ranom orer Weighte ranom orer g j gi p ( i, j ) 0.5 R Complexity: O((M+N)MNkp) ) 1 The FLOC algorithm Aitional features Maximum allowe overlap among clusters Minimum coverage of clusters Minimum volume of each cluster Can be enforce by temporarily blocking certain action uring the mining process if such action woul violate some constraint.

12 Performance Microarray ata: 884 genes, 17 conitions 100 bi-clusters with smallest resiue were returne. Average resiue = The average resiue of clusters foun via the state of the art metho in computational biology fiel is 1.54 The average volume is 5% bigger The response time is an orer of magnitue faster 3 Conclusion Remark The moel of bi-cluster is propose to capture coherent objects with incomplete ata set. base resiue Many aitional features can be accommoate (nearly for free). 4

13 Coherent Cluster Want to accommoate noise but not outliers 5 Coherent Cluster Coherent cluster Subspace clustering pair-wise isparity For a (sub)matrix consisting of objects {x, y} an attributes {a, b} 6 D xa ya xb yb ( xa ya ) ( xb yb) mutual bias of attribute a mutual bias of attribute b x z y xa ya a b attribute xb yb

14 Coherent Cluster A (sub)matrix is a -coherent cluster if its D value is less than or equal to. An mn matrix X is a -coherent cluster if every submatrix of X is -coherent cluster. A -coherent cluster is a maximum -coherent cluster if it is not a submatrix of any other -coherent cluster. Objective: given a ata matrix an a threshol, fin all maximum -coherent e e clusters. s 7 Coherent Cluster Challenges: Fining subspace clustering base on istance itself is alreay a ifficult task ue to the curse of imensionality. The (sub)set of objects an the (sub)set of attributes that form a cluster are unknown in avance an may not be ajacent to each other in the ata matrix. The actual values of the objects in a coherent cluster may be far apart from each other. Each object or attribute in a coherent cluster may bear some relative bias (that are unknown in avance) an such bias may be local to the coherent cluster. 8

15 Coherent Cluster Compute the maximum coherent attribute sets for each pair of objects Two-way Pruning Construct the lexicographical tree Post-orer traverse the tree to fin maximum coherent clusters 9 Coherent Cluster Observation: Given a pair of objects {o 1, o } an a (sub)set of attributes {a 1, a,, a k }, the k submatrix is a -coherent cluster iff, for every attribute a i, the mutual bias ( o1ai oai ) oes not iffer from each other by more than a 1 a a 3 a 4 a [, 3.5] If = , o then {a 1 1,a,a 3,a 4,a 5 } is a coherent attribute set (CAS) o of (o 1,o ). 30

16 Coherent Cluster Observation: given a subset of objects {o 1, o,, o l } an a subset of attributes {a 1, a,, a k }, the lk submatrix is a -coherent cluster iff {a 1, a,,a a k }is a coherent attribute set for every pair of objects (o i,o j ) where 1 i, j l. a 1 a a 3 a 4 a 5 a 6 a 7 o 1 o o 3 o 4 o 5 o 6 31 Coherent Cluster Strategy:finthemaximum the coherent attribute sets for each pair of objects with respect to the given threshol r 1 5 r r r a 1 a a 3 a 4 a 5 a a 4 a 5 a 1 a 3 = The maximum coherent attribute sets efine the search space for maximum coherent clusters.

17 Two Way Pruning a0 a1 a o0 1 4 o1 5 5 o o (o0,o) o) (a0,a1,a) (a0a1a) (a0,a1) a1) (o0,o1,o) (o0o1o) (o1,o) (a0,a1,a) (a0,a) (o1,o,o3) (a1,a) (o1,o,o4) (a1,a) (o0,o,o4) o (o0,o) (a0,a1,a) (o1,o) (a0,a1,a) elta=1 nc =3 nr = 3 (a0,a1) (o0,o1,o) (a0,a) (o1,o,o3) (a1,a) (o1,o,o4) (a1,a) (o0,o,o4) MCAS MCOS 33 Coherent Cluster Strategy: grouping object pairs by their CAS an, for each group, fin the maximum clique(s). Implementation: using s a lexicographical tree to organize the object pairs an to generate all maximum coherent clusters with a single post-orer orer traversal of the tree. objects attributes 34

18 a 0 a 1 a a 3 o o o o o assume = 1 (o 0,o 1 ) : {a 0,a 1 }, {a,a 3 } (o 0,o ) : {a 0,a 1 1,,a,,a 3 3} (o 0,o 4 ) : {a 1,a } (o 1,o ) : {a 0,a 1,a }, {a,a 3 } (o 1,oo 3 ) :{a 0,aa } (o 1,o 4 ) : {a 1,a } (o 1,o ) (o,o 3 ) : {a 0,a } a 3 35 (o,o 4 ) : {a 1,a } 3 {a 0,a 1 } : (o 0,o 1 ) (o 1,o ) (o 0,o ) {a 0,a } : (o 1,o 3 ),(o,o 3 ) (o 1,o ) (o 0,o ) {a 1,aa }:(o 0,oo 4 )(o ),(o 1,oo 4 )(o ),(o,oo 4 ) (o 1,oo ) (o 0,oo ) {a,a 3 } : (o 0,o 1 ),(o 1,o ) (o 0,o ) {a 0,a 1,a } : (o 1,o ) (o 0,o ) {a 0,a 1,a,a 3 }:(o 0,o ) a 0 a 1 a a 1 a a a 3 a (o,o 3 ) (o 0,o 1 ) (o 1,o 3 ) (o 0,o ) (o 0,o 4 ) (o 1,o 4 ) (o,o 4 4) (o 0,o 1 ) (o 1,o ) {o 0,o } {a 0,a 1,a,a 3 } {o 1,oo } {a 0,aa 1,aa } {o 0,o 1,o } {a 0,a 1 } {o 1,o,o 3 } {a 0,a } {o 0,o,o 4 } {a 1,a } {o 1,o,o 4 } {a 1,a } {o 0,o 1,o } {a,a 3 } a 0 a 1 a a 3 a 1 a a a 3 a 3 36 (o 0,o ) (o 0,o 1 ) (o 1,o 3 ) (o 0,o 4 ) (o 0,o 1 ) a (o 1,o ) (o,o 3 ) (o 1,o 4 ) (o 1,o ) (o 0,o ) (o 1,o ) (o,o 4 ) (o 0,o ) (o (o 1,o ) a a 0,o ) (o 3 a 1,o ) 3 3 (o 0,o ) (o 0,o ) a (o 0,o ) 3 (o 0,o ) (o 0,o ) (o 0,o ) (o 0,o )

19 Coherent Cluster High expressive power The coherent cluster can capture many interesting an meaningful patterns overlooke by previous clustering methos Efficient an highly scalable 4000 Wie applications Gene expression analysis Collaborative filtering (se c ) sponse tim e a v e r a g e r e s subspace cluster coherent cluster number of conitions 37 Remark Comparing to Bicluster Can well separate noises an outliers No ranom ata insertion an replacement Prouce optimal solution 38