Cluster Analysis in Data Mining

Transcription

1 Cluster Analysis in Data Mining Part II Andrew Kusiak 9 Seamans Center Iowa City, IA - 7 andrew-kusiak@uiowa.edu Tel. 9-9 Cluster Analysis Decomposition Aggregation Grouping Page

2 Models Matrix formulation Mathematical programming formulation Graph hformulation Cluster Representation G B A C D H E F Mutually exclusive regions Page

3 Cluster Representation G B A E C D F H Overlapping regions Cluster Representation A B C E F D G H Hierarchy Page

4 Cluster Representation Clusters A... B... C... D... Objects E...7 F...7 G... H... Fuzzy clusters Two types of data Object with decisions Object without decisions Page

5 Clustering Data with Decisions Method Ignore decisions Treat the objects as data without decisions Method Group objects according to the decisions Treat each group of objects with decisions as a separate data set Solving the Clustering Problem: Binary Matrix Formulation Similarity il it coefficient i methods Sorting based algorithms Bond energy algorithm Cost-based method Cluster identification algorithm Extended cluster identification algorithm Page

6 Clustering Algorithms Desired Properties Low computational time complexity Structure enhancement Enhancement of human interaction and simplification of interface with software Generalizability Similarity Coefficient Method n δ(aik, ajk ) k = sij = n δ(aik, ajk ) k = if aik = ajk where δ (aik, ajk ) = { 0 otherwise Generalization to qualitative features Page

7 δ (aik, ajk ) = { 0 if aik = ajk otherwise n = the number of features Consider GO- { Yes Yes GO- { Page 7 7

8 s = s = = 0. s = = 0. δ (aik, ajk ) = { δ (aik, ajk ) = { if aik = ajk 0 otherwise 0 if aik = ajk otherwise 0 s = s = s = = 0 Yes GO- { Yes GO- { Tree Representing Similarity Coefficients Object number Sim milarity (%) GO- GO- GO - GF- Yes Yes Page 8 8

9 Cluster Identification Algorithm Decomposition Problem Decompose an object-feature incidence matrix into mutually separable submatrices (groups of objects and groups of features) with the minimum number of overlapping objects (or features) subject to the following constraints: Constraint C: Constraint C: Empty groups of objects or features are not allowed. The number of objects in a group does not exceed an upper limit, b (or alternatively, the number of features in a group does not exceed, d). Page 9 9

10 Cluster Identification Algorithm Step 0. Set iteration number k =. Step. Select row i of incidence matrix [aij](k) and draw a horizontal line hi through it ([aij](k) is read: matrix [aij] at iteration k ). Step. For each entry of * crossed by the horizontal line hi draw a vertical line vj. Step. For each entry of * crossed-once by the vertical line vj draw a horizontal line hk. Step. Repeat steps and until there are no more crossed-once entries of * in [aij](k). All crossed-twice entries * in [aij](k) form row cluster RC-k and column cluster CC-k. Step. Transform the incidence matrix [aij](k) into [aij](k+) by removing rows and columns corresponding to the horizontal and vertical lines drawn in steps through. Step. If matrix [aij](k+) = 0 (where 0 denotes a matrix with all empty elements ), stop; otherwise set k = k + and go to step. Example Incidence matrix Object 7 Feature 7 8 * * Page 0 0

11 7 7 8 * * Next Step Delete all double-crossed elements Page

12 7 8 * 7 7 * * Resultant Matrix 7 * Page

13 7 Iteration * h h v v Iteration 7 * h h v v7 Final decomposition result RC- RC- RC- 7 CC- CC- CC- 8 7 * * * Page

14 Three types of matrices (a) (b) (c) a b c d e f g h a b c d e f g h a b c d e f * * * * * * * * * * * * 7 * (a) decomposable matrix (b) non-decomposable matrix with overlapping features (c) non-decomposable matrix with objects Extended CI Algorithm Step 0. Set iteration number k =. (k) Step. Select those objects (rows of matrix [aij] ) that based on the user's expertise, are potential candidates for inclusion in machine cell GO-k. Draw a horizontal line hi through each row of matrix [aij] (k) corresponding to these objects. In the absence of the user's expertise any machine can be selected. (k) Step. For each column in [aij] corresponding to entry, single-crossed by any of the horizontal lines hi, draw a vertical line vj. (k) Step. For each row in [aij] corresponding to the entry, single-crossed by the vertical line vj, drawn in Step, draw a horizontal line hi. Based on the objects corresponding to all the horizontal lines drawn in Step and Step, a temporary machine cell GO-k, is formed. Page

15 If the user's expertise indicates that some of the objects cannot be included in the temporary group GO-k, (k) erase the corresponding horizontal lines in the matrix [aij]. Removal of these horizontal lines results in group GO'-k. (k ) Delete from matrix [aij] features (columns) that are contained in at least one of the objects already included in GO-k. Place these features on a separate list. Draw a vertical line vj through each single-crossed entry in [aij](k) which does not involve any other objects than those included in GO-k. (k) Step.For all the double-crossed entries in [aij], form a cluster GO-k and a group GF-k Step. Transform the incidence matrix [aij] (k+) into [aij] by removing all the rows and columns included in GO-k and GF-k, respectively. (k +) Step. If matrix [aij] = 0 (where 0 denotes a matrix with all elements equal to zero), stop; otherwise set k = k + and go to step. (k) Page

16 Extended CI Algorithm Constraints: Max GO = Objects and in the cluster 7 Feature number Object number Step 0. Set iteration number k =. Step. Since user's expertise indicates that objects and should be included cluster GO-, two horizontal lines h and h are drawn. Feature number h h Page

17 Step. For columns,,,, and 7 crossed by the horizontal lines h and h, five vertical lines, v, v, v, v, and v7 are drawn. Feature number h h v v v v v7 Step. Three horizontal lines, h, h, and h7 are drawn through rows,, and 7 corresponding of the single-crossed elements. Feature number h h h h h7 v v v v v7 Page 7 7

18 A temporary machine cell GO'- with objects {,,,, 7} is formed. objects and are excluded from in GO'- as they include less double-crossed (committed) elements than object 7. The horizontal lines h and h are erased h h h7 v v v v7 Step. The double-crossed entries of matrix indicate: Object cluster GO- = {,, 7} and Feature cluster GF- = {,,, 7} Step. Matrix is transformed into the matrix below. 8 0 Page 8 8

19 Step. Set k = k + = and go to Step. The second iteration (k = ) results in: Object cluster GO- = {,,, } and Feature cluster GF- = {, 8, 0, } 8 0 v v8 v0 v h h h h The final result GO - GO - 7 GF - GF Page 9 9

20 Notation m = number of objects n = number of features p = number of clusters if feature i belongs to xij = { cluster j 0 otherwise dij = distance between objects i and j n n min dij xij i = j = subject to: n xij = for all i =,..., n j = n xjj = p j = xij <= xjj for all i =,..., n, j =,..., n xij = 0, for all i =,..., n, j =,..., n Page 0 0

21 m dij = d(aki, akj j) k = Two objects Hamming distance Hamming Distance 0 Blue Blue O 0 = O = 0 0 d = = Decision variable xij Object (or feature) number Object (or feature) number Cluster number Page

22 Given p = m dij = d(aki, akj ) k = [dij] = Object number Feature number Feature number Feature number MIN X+X+X+X+X+X+X+X+ X+X+X+X+X+X+X+X n SUBJECT TO xij = for all i =,..., n j = X+X+X+X+X= X+X+X+X+X= Feature number X+X+X+X+X= X+X+X+X+X= X+X+X+X+X=[dij] = X+X+X+X+X= Feature number Page

23 X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0X< X-X<=0 X-X<=0 X-X<=0 Constraint xij <= xjj X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0 X-X<=0 END INTEGER Solution x =, x = x =, x =, x = Based on the definition of xij, two groups of features are formed: GF- = {, } GF- = {,, } GO- = {, } GO- = {, } Feature number GF - GF- Object number GO- GO- Page

24 GO- { GO- { Solution: x=, x = x =, x =, x= Page