NIVA: A Robust Cluster Validity

Transcription

1 2th WSEAS Internatonal Conference on COMMUNICATIONS, Heralon, Greece, July 23-25, 2008 NIVA: A Robust Cluster Valdty ERENDIRA RENDÓN, RENE GARCIA, ITZEL ABUNDEZ, CITLALIH GUTIERREZ, EDUARDO GASCA, FEDERICO DEL RAZO, ADRIAN GONZALEZ Dvsón de Estudos de Postgrado e Investgacón Insttuto Tecnológco de Toluca Ex. Rancho la Vrgen, Metepec, Edo. de Méxco MÉXICO Abstract: Clusterng ams at extractng hdden structures n datasets. Many valdty ndces have been proposed to evaluate clusterng results; some of them wor well when clusters have dfferent denstes and szes and others wth dfferent shapes. They usually have a tendency to consder one or two characterstcs smultaneously. In ths paper, we present a cluster valdty ndex that taes advantage of the densty, sze and shape of cluster characterstcs. The proposed ndex s expermentally compared wth PS, CS and S_Dbw ndces usng 2 synthetc datasets. Our proposed ndex mproves others ndces. Key-Words: Cluster valdty, cluster algorthm, connectvty and compactness. Introducton The goal of clusterng s to determne the ntrnsc groupng n a set of unlabeled data, where the obects n each group are ndstngushable under some crteron of smlarty. Clusterng s an unsupervsed classfcaton process fundamental to data mnng (one of the most mportant tass n data analyss). It fnds applcaton n several areas le bonformatcs [4], web data analyss [3], text mnng [7] and scentfc data exploraton []. Clusterng refers to unsupervsed learnng and, for that reason, t has no a pror data set nformaton. However, to get good results, the clusterng algorthm depends on nput parameters. For nstance, -means [6] and CURE [5] algorthms requre the number of clusters () to be created. In ths sense, the queston s: What s the optmal number of clusters? Currently, cluster valdty ndces research has drawn attenton as a means to gve a soluton [7]. Many dfferent cluster valdty methods have been proposed [8] [0] wthout any a pror class nformaton. Clusterng valdaton s a technque to fnd a set of clusters that best fts natural parttons (number of clusters) wthout any class nformaton. In ths paper, we present an analyss of the ndces offered by Chow (PS, CS ndces) [8, 0] and Hald (S_Dbw ndex) [7]. We also offer a soluton to address ther drawbacs. For ths purpose, we frst show a novel ndex valdaton (NIVA) that uses connectvty amount ponts to capture the shape cluster; secondly, we present a comparatve study wth CS, PS; SD_bw valdaton ndces. The rest of the paper s organzed as follows: secton 2 presents surveys of related wors. Secton 3 offers a lght analyss about some ndex valdaton. Secton 4 contans detals about the ndex proposed. Secton 5 provdes the expermental results of our ndex and dscusses some fndngs from these results. Fnally, we conclude by brefly showng our contrbutons and further wors. 2 Prevous wors Almost every clusterng algorthm depends on the characterstcs of the dataset and on the nput parameters. Incorrect nput parameters may lead to clusters that devate from those n the dataset. In order to determne the nput parameters that lead to clusters that best ft a gven dataset, we need relable gudelnes to evaluate the clusters; clusterng valdty ndces have been recently employed. In general, clusterng valdty ndces are usually defned by combnng compactness and separablty.. Compactness: Ths measures closeness of cluster elements. A common measure of compactness s varance. 2. Separablty: Ths ndcates how dstnct two clusters are. It computes the dstance between two dfferent clusters. The dstance between representatve obects of two clusters s a good example. Ths measure has been wdely used due to ts computatonal effcency and effectveness for hypersphere-shaped clusters. There are three approaches to study cluster valdty []. The frst s based on external crtera. Ths mples that we evaluate the results of a clusterng algorthm based on a pre-specfed ISSN: ISBN:

2 2th WSEAS Internatonal Conference on COMMUNICATIONS, Heralon, Greece, July 23-25, 2008 structure, whch s mposed on a dataset,.e. external nformaton that s not contaned n the dataset. The second approach s based on nternal crtera. We may evaluate the results of a clusterng algorthm usng nformaton that nvolves the vectors of the datasets themselves. Internal crtera can roughly be subdvded nto two groups: the one that assesses the ft between the data and the expected structure and others that focus on the stablty of the soluton [2]. The thrd approach of clusterng valdty s based on relatve crtera, whch conssts of evaluatng the results (clusterng structure) by comparng them wth other clusterng schemes. Many dfferent cluster valdty measures have been proposed n the past [2]. In general, valdty ndces can be grouped nto two man categores: the frst category conssts of valdty measures that evaluate the propertes of the crsp structures mposed on the data by the clusterng algorthm [3]. The second wors wth measures that use membershp degrees obtaned by fuzzy clusterng algorthm. [3].The thrd category conssts of valdty measures that tae nto account not only the membershp degrees but also the data themselves. In recent tmes, many ndces have been proposed n the lterature, whch are used to measure the ftness of the parttons produced by clusterng algorthm [2]. The Dunn ndex [2] measures the rato between the smallest cluster dstance and the largest ntra-cluster n a parttonng; several varatons of Dunn have been proposed [4][5]. DB measures the average smlarty between each cluster and the one that most resembles t. [6]. The SD ndex [7] s defned based on the concepts of the average scatterng for clusterng and total separaton amount clusters. The S_Dbw ndex s very smlar to SD ndex; ths ndex measures the ntra-cluster varance and nter-cluster varance. The ndex PS [8] used a nonmetrc dstance based on the concept of pont symmetry [9], and measures the total average symmetry wth respect to the cluster centers. [0] proposes the CS ndex that obtans good clusterng results when the denstes and szes are dfferent, but ts computatonal cost s elevated. 3 Analyss of ndces In ths secton, we offer an overvew of the CS, PS, SD_bw valdty ndces, snce our ndex s based on ther dsadvantages. 3. SD_bw valdty ndex M. Hald proposed the SD_bw ndex [7]. Ths ndex s based on cluster compactness and separaton consderng the densty of clusters. In other words, the SD_bw ndex measures the ntra-cluster varance and the nter-cluster varance. The ntra cluster varance measures the mean scatterng of clusters and t s descrbed by Eq.. The nter-cluster densty s defned by the Eq. 2. n c σ ( v ) Scatt = () n = σ ( X ) Where: c σ ( v ) s the varance of cluster c and σ (X ) s the varance of the set data n c n c Dens_ bw= nc ( nc ) = =, max denstyu ( ) ( denstyv ( ), densty( v ) ) Where: u s the mddle pont of the lne segment defned by the v and v clusters centers. The densty functon around a pont u (2) s defned as follows: t counts the number of ponts n a hyper-sphere whose radus s equal to the average standard devaton of clusters. The standard devaton of clusters s defned as n c stdev = σ ( v ) n c = The S_Dbw ndex s defned as below: S _ Dbw = Scatt + Dens _ bw 3.2 PS valdty Index PS ndex was proposed by Chen-Hsng Chou [8]; ths ndex dentfes clusters of dfferent forms smultaneously, when there s dversty of forms n the cluster. In order to do ths, t uses the dstance proposed by Su [9]. The general concept of pont symmetry dstance [9] s defned as follows: ( x c) + ( x c) d s = ( x, c) = mn (4) =,.. N x c + x c and 3.3 CS valdty Index Chen-Hsng Chou proposed the CS valdty ndex []. Ths ndex evaluates clusterng results when denstes and szes are dfferent. The CS ndex s defned as follows: c max c = A x A x A CS( c) c mn = c, Where: { d( x, x )} { d( v, v )} c max = A = x A x A c mn = c, = { d( x, x )} { d( v, v )} (5) ISSN: ISBN:

3 2th WSEAS Internatonal Conference on COMMUNICATIONS, Heralon, Greece, July 23-25, 2008 d s a dstance functon, max( d ( x, x )) measures the rado of the sum of wthn-cluster scatter. mn{ ( v, v )} measures the between-cluster separaton. Thus CS ndex cluster valdty measures scatterng as a functon to between-cluster separaton. 4 The novel cluster valdty ndex In ths secton, we descrbe a novel valdty ndex called NIVA. 4. NIVA valdty ndex defnton We frst need to ntroduce the basc prncples. Consder a partton of the data set C = { c =,.., N} and the center of each cluster v ( =,2,..., N), where N s the number cluster from C. The cluster valdty ndex NIVA wors n two stages: Frst stage: nown as local evaluaton, t carres out a sub-clusterng of obects belongng to clusters c, obtanng l groups, as seen n Fg.. For = up to N Calculate l groups of set c, usng the OSB clusterng algorthm. Calculate average compact of l groups usng Eq. 8. Calculate average separaton of l groups of, usng Eq. 9. c End Fg.. Steps of frst NIVA stage. Second stage: conssts of calculatng the NIVA ndex of partton C. Thus, the NIVA valdaton ndex s defned as follows: Compac( C) NIVA ( C) = (6) SepxG( C) - Compac(C) : Average of compactness product ( Esp( c ), Eq. 8) of c groups and separablty between them ( SepxS( c ), Eq. 9). N Compac( C) = Esp( c )* SepxS( c ) N = (7) Where: l n ESp( c ) = d x x + l = n ( (, )) (8) = and SepxS( c ) = l l { max{ ( d( sv, sv ) } = l = subclusters number of c n = data number sv, sv = they are the p from subcluster clusters center x = s the nearest neghbor from + x from c - SepxG(C) : Average separablty of C groups. It s calculated usng Eq. 0. N SepxG( C) = mn{ d( v, v )} (0) N = C The smaller value NIVA(C) ndcates that a vald optmal partton to the dfferent gven parttons was found. The clusterng algorthm used to fnd c subgroups was called OSB Algorthm clusterng OSB The clusterng algorthm uses the clusterng crtera n order to detect connected components, for whch an obect x belongs to a cluster, f only there s an obect x, h (nearest neghbor from x ) such that the Eucldean dstance between the two obects s greater than a calculated threshold, called smlarty threshold st. In order to calculate the smlarty threshold, we use a heurstc, whch conssts of calculatng an average of dstances between x and x, h each tme an obect enters cluster. x, h 5. Expermental Results In ths secton, NIVA s expermentally tested usng the K-means algorthm. We used 2 synthetc data sets (see Tables and 2). These data sets were used by Mara Hald [7] and Chen-Hsng Chou [8][0]. We have used these sets because we also compared ther valdty ndces [7][8][0]. 5. The best partton To fnd the best partton, we have used the K-means algorthm wth ts nput parameters rangng between 2 and 8 and, to verfy that we really found the best (9) ISSN: ISBN:

4 2th WSEAS Internatonal Conference on COMMUNICATIONS, Heralon, Greece, July 23-25, 2008 partton, we appled our valdaton ndex to the labeled set. Table Result of NIVA valdty ndex Data sets K Labeled data set (5) (4).2648 (2 ó 3).888 (7) (2) (2) (2) ISSN: ISBN:

5 2th WSEAS Internatonal Conference on COMMUNICATIONS, Heralon, Greece, July 23-25, 2008 Table 2. Data sets synthetcs (a) DataSet (b) DataSet 2 (c)dataset 3 (d) DataSet 4 (e)dataset 5 (f) DataSet 6 (g) DataSet 7 (h) DataSet 8 ISSN: ISBN:

6 2th WSEAS Internatonal Conference on COMMUNICATIONS, Heralon, Greece, July 23-25, 2008 Table 3. Data sets synthetcs (a) DataSet 9 (b) DataSet 0 (c)dataset (d) DataSet 2 Table shows NIVA values of the results obtaned wth the clusterng found by K-means. In both cases, NIVA faled (Datasets 8 and 9). On the other hand, to prove the veracty of our results, we appled NIVA to the labeled datasets. The values obtaned are depcted n table. Last column of Table shows that, n all cases, NIVA found the best partton from datasets. In other words, NIVA was able to fnd the best partton between the results of the K-means algorthm and the labeled datasets. 5.2 Comparson wth other valdty ndces We used the nown valdty ndces proposed n the lterature [7][8][0], such as CS, PS and SD_bw. For comparson purposes, we used datasets very smlar to those used by PS, CS and SD_bw ndces. Table 4 presents the results summary of CS, PS, SD_bw and NIVA valdaton ndces. For our study, we used the results of the algorthm K-means wth ther nput values, rangng between 2 and 8 and labeled datasets. Le n the last experment we used 2 datasets (see Tables 2 and 3). We can see that the PS ndex made four mstaes, CS made two mstaes, SD_bw made sx mstaes and NIVA found the correct cluster number n all cases. It s mportant to say that our ndex faled n datasets 8 and 9 (see table ) when t was run to the clusterng result of the K-means. But when the labeled datasets are ncluded, NIVA obtaned the optmal K (clusters number) n all cases. The followng tests were addtonally run: For clusters wth dfferent geometrcal shapes (NIVA vs. PS), the comparson wth ndex PS was carred out; for ths purpose, datasets DataSet2 and DataSet3 were used n ths test. Both ndces obtaned the correct number of clusters (5 and 3). For groups wth dfferent denstes and szes, as well as separablty between them (NIVA vs. CS), the comparson between ndex CS and datasets DataSet4 and DataSet6 was carred out; NIVA found the correct number of clusters beng 3 and 4. For clusters wth dfferent compactness and separablty between them (NIVA vs. SD_bw), the comparson of the NIVA ndex wth ndex S_dbw was carred out wth datasets DataSet7 and DataSet8, whch have compact and well-separated clusters. It s mportant to pont out that NIVA found the correct number of clusters (2) of dataset DataSet7. For dataset DataSet8, t does not fnd the correct number of clusters obtaned by the K-means; however, when the clustered dataset s ncluded, NIVA fnds the correct number of clusters (7). 6. Conclusons and further wor In ths paper, we have defned a novel valdty cluster called NIVA to fnd the best partton, the results of clusterng algorthms. The NIVA ndex fnds groups wth dfferent denstes, szes and shapes. To do that, the compactness of the dataset s measured usng the connectvty among data from clusters, whereas the separaton among clusters was measured by mnmzng the dstance between center clusters. We used 2 datasets to carry out the experments. It s ISSN: ISBN:

7 2th WSEAS Internatonal Conference on COMMUNICATIONS, Heralon, Greece, July 23-25, 2008 mportant to say that we used the same datasets that were used by Mara Hald [7] and Chen-Hsng Chou [8][0] to compare wth ther ndces. The results obtaned by the NIVA ndex were encouragng, because t found the best partton n every case. The performance of the NIVA ndex was compared wth three popular valdaton ndces. When our ndex was compared wth other ndces, t always obtaned better results. Fnally, we hope we can mprove the results obtaned by mang more experments and usng clusterng results as Cure. DataSet s Table 4. Results of comparsons PS CS S_Dbw NIVA K K K K # correct References: [] Jan, A. K., Murty, M.N. Flynn, P.J. Data clusterng: A revew. ACM Computer. Surveys 3, 999, pp [2] Dunn, J. C., 973. A Fuzzy relatve of the ISODATA process and ts use n detectng compact well-separated clusters. J. Cyber. 3, 973, pp [3] Bouguessa M., Wang S. and Sun H An Obectve Approach to Cluster Valdaton. Pattern Recognton Letters, Vol. 27, Issue 3, pp [4] Pal N. R. and Bswas J., 997. Cluster Valdaton usng graph theoretc concepts, Pattern Recognton, Vol.30, No. 6. pp [5] Bezde J. C. Pal N.R., 998. Some new ndexes of cluster valdty. IEEE Trans. Syst.Man, Cyber. Part B 28,pp [6 ] Davs D. L., Bouldn D.W A cluster separaton measure. IEEE Trans. Pattern Anal. Mach.Intel. (PAMI) (2), pp [7] Hald M., Vazrganns, M., Qualty scheme assessment n the clusterng process. In Proc. PKDD (Prncples and Practce of Knowledge n databases). Lyon, France. Lecture Notes n Artfcal Intellgence. Sprng Verlag Gmbh, vol.90, pp [8] Chow C.H, Su M.C and La Eugene 2002.Symmetry as A new measure for Cluster Valdty. 2 th WSEAS Int.Conf. scentfc Computaton and Soft Computng, Crete, Greece, pp [9] Su M.C, Chow C.H, 200. A Modfed Verson of the K-Means Algorthm wth a Dstance Based on Cluster Symmetry. IEEE Trans. Pattern Anal. And Machne Intellgence, vol.23. No. 6, pp [0] Chow C.H, Su M.C and La Eugene A new Valdty Measure for Clusters wth Dfferent Denstes. Pattern Anal. Applcatons, 7, pp [] Theodords, S., Koutroubas, K. (999). Pattern Recognton, Academc Press, USA [2] Voler Roth, Tlman Lange, Mo Braun, and Joachn Buhmann. A Resamplng Approch to Cluster Valdaton. 2002, Proceedng n Computatonal Statstcs COMPSTAT Physa Verlag, pp [3] Athena Vaal, Jaroslav Poorný and Theodore Dalamagas. An Overvew of Web Data Clusterng Practces. 2005, Lecture Notes Computer Scence, Vol. 3268, pp [4] M.J.L. Hoon, S. Imoto, J. Nolan and S. Myano. 2004, Open source clusterng software. Vol. 20 No. 9, pp [5] Guha Sudpto, Rastog Raeev, Shm Kyuseo. CURE: An Effcent Clusterng Algorthm for Large DataBases. In Proceedngs of the CAM SIGMOD Conference on Magnagement of Data, Seatle, Washngton, U.S.,0-04 Jun., pp , 998. ISSN: ISBN:

8 2th WSEAS Internatonal Conference on COMMUNICATIONS, Heralon, Greece, July 23-25, 2008 [6] J.B. MacQueen. Some Methods for classfcaton and Analyss of Multvarable Observatons, Proceedng of 5 th Bereley on Mathematcal Statstcs and Probablty, 967, Unversty of Calforna Press, pp [7] Bernd Drewes. Some Industral Applcatons of Text Mnng. 2005, Knowledge Mnng, Sprnger Berln, Vol. 85, pp ISSN: ISBN: