New Version of Davies-Bouldin Index for Clustering Validation Based on Cylindrical Distance

Transcription

1 New Version of Davies-Boulin Inex for lustering Valiation Base on ylinrical Distance Juan arlos Roas Thomas Faculta e Informática Universia omplutense e Mari Mari, España correoroas@gmail.com Abstract This paper presents a new version of Davies- Boulin inex for clustering valiation through the use of a new istance base on ensity. This new istance is use as a similarity measurement between the means of the clusters, with the purpose of overcoming the limitations of the Eucliean istance. The new istance propose allows consiering the istribution of the ata set an to approximate in a more accurate way the separation between clusters through the estimation of the ensities along the line segments that connect the centrois. Keywors lustering; Davies-Boulin Inex; Density; ylinrical Distance. Introuction The process of clustering consists on classifying in an unsupervise way a set of patterns (observations or ata) into groups (clusters) []. In general, the clustering methos shoul search for clusters whose members are close to each other (in other wors have a high egree of similarity) an well separate [2]. One of the most important issues in cluster analysis is the evaluation of clustering results to fin the partition that best fits the unerlying ata. This is the main subect of clustering valiation [2]. In general, there are three approaches to investigate clustering valiation: external criteria, internal criteria an relative criteria [3]. lustering valiation approaches, which are base on relative criteria, aim at fining the best clustering scheme that a clustering algorithm can efine uner certain assumptions an parameter. Here the basic iea is the evaluation of a clustering structure by comparing it to other clustering schemes, resulting by the same algorithm but with ifferent parameter values [4]. The Davies-Boulin inex falls into the latter category. Such inexes are use when the partitions generate by the applie clustering algorithm are no overlapping, meaning by this that each ata belongs strictly to an only one class [4]. The Davies-Boulin inex is base on the approximately estimation of the istances between clusters an their ispersions to obtain a final value that represents the quality of the partition. One of the statistics use to estimate the istances between the clusters is the Eucliean istance between the means. The problem with this statistic is that it oesn't consier the geometry of the clusters; instea it reuces the estimation to the Eucliean istance between representative points (the means). As a result, two clusters with means very close each other will be consiere very close even if their ata are not. In orer to overcome these limitations, this paper proposes a special type of istance, the cylinrical istance, which is use to calculate the istance between the means. This istance tries to capture the ata ensity along the straight lines that connect the means, an through this, to estimate how close are the clusters as a whole. This technique is valiate by comparing its results with the original version of the inex over real atasets. This paper presents the following structure: first a escription of the original inex an its limitations an relate works. Then, the cylinrical istance an the new version of the inex propose are explaine. Afterwor is presente the comparative performance of the original inex an the new version propose, an finally the conclusions an future works. I. DAVIES-BOULDIN INDEX A. Definition This inex (DB) is base on the iea that for a goo partition inter cluster separation as well as intra cluster homogeneity an compactness shoul be high [5]. Then, to efine the DB inex, we nee to efine the ispersion measure an the cluster similarity measure [6]. In [] the ispersion S i of i cluster () an the separation D between ith an th clusters (2) are efine as: i x p p D x, c i, p 0 i () Where i is the number of ata points in cluster i an c i is the center of cluster i, an: D v l il v l t t, t (2)

2 Where v i an v are the centrois of clusters i an, respectively. Then, the DB inex is efine as: V k DB k i Where k is the number of clusters an is efine as: max R i Where R is the similarity measure between clusters i an, an is efine as: R S D nce the goal is to achieve minimum within-cluster ispersion an maximum between-cluster separation, the number of clusters c that minimizes V DB is taken as the optimal value of c [5]. B. Limitations of the Original Proposal Essentially, the main limitation of the original proposal is that none of its terms consiers the geometry of the spatial istribution of the clusters. This limitation is irectly relate to the use of the means to measure the istances between the clusters. Specifically, the use of means as a measure of istance between clusters fails when they have ifferences variances, or the variances are not equal in all axes. Using the Eucliean istance between the means to calculate the istances between the clusters can lea to situations where two clusters whose geometries are close but its means are far from each other will be consiere more istant than two clusters whose geometries are more istant but its means are closer. This situation is illustrate in Fig.. Another limitation of the use of the Eucliean istance between the means is its inability to istinguish when it is measuring istances between centrois that represent genuine clusters, from situations where it is in front of centrois that represent a partition that it is really iviing a genuine one. The Fig. 2 gives an example of two ifferent configurations where the Eucliean istance between the means are equal, but actually in only one of them the centrois represents genuine well separate clusters. (3) (4) (5) a) b) Fig. 2. The images show two configurations where the istance between the centrois are equals, but in one of them (a) the centrois belong to the same cluster, an in the other (b) the centrois belong to a well separate clusters. A final example of the limitations of the inex are ilustrate in Fig. 3. These images show two iferent configurations of clusters. In the first configuration (a) the clusters have a maximum variance vector along the horizontal axe. The secon one (b) is the result of rotating the maximum variance vectors of the first configuration to a vertical position. As a result, both configurations have the same means an the same ispersions, an as a consecuence the value of the Davies-Boulin inex will be the same for both, even if in the secon configuration (b) the clusters are more well separate than in the first one(a). a) b) Fig. 3. The images show two configurations of clusters with the same means an ispersion, but with iferent orientations of the maximum variance vectors. Both have the same Davies-Boulin inex value. Fig.. The image shows two clusters with closer geometries but more istan means (blue an green clusters) an the opositte situation (blue an brown clusters). II. RELATED WORKS In [5] the Davies-Boulin inex is generalize through the use of graphs. Specifically, they use Minimal Spanning Tree (MST), Relative Neighborhoo Graph (RNG) an Gabriel Graph (GG). However, these graphs are use only to obtain the

3 ispersion of each cluster, while the measurement of istance between clusters is still the istance between the means. For this reason, those approaches are not consiere equivalent to this proect, because they tackle another problem of the Davies- Boulin inex. Instea, they are consiere complementary, an will not be use in the evaluation of the new version. Respect to istances base on ensity, there is a group of clustering algorithms bases on ensity, where clusters are efine as ense regions separate by low-ensity regions [6]. Examples of these algorithms are DBSAN [9] an BRIDGE [0]. In these algorithm the concept of ensity is use for etermine a relation of connectivity of ata points that belong to the same cluster rather than to efine a new measure of istance for replacing the Eucliean one. III. THE PROPOSAL The proposal presente in this paper consists in a new istance, calle cylinrical istance, which is use to measure the istance between the means, instea of the Eucliean one. The main iea behin this istance is to capture the ata ensity in a limite region of the space aroun the straight line that connects the means, as it is illustrate in Fig. 4. This region is etermine by a parameter, the raius. All ata that are locate in a istance, from the straight line that connects the means, equal or less that the raius, an whose proections to this segment is perpenicular, are consiere to belong to this region, an use in the final calculation of the cylinrical istance. This process is illustrate in Fig 5. A. Definition of ylinrical Distance Let D be the ataset of n imensions. Let be any ata point that belongs to the ataset D. Let p, p 2 be the ata points of which the istance is wante to be etermine. Let L(p, p 2 ) be the length of the line segment that has p an p 2 as its extremes. Let E(, p, p 2 ) be the Eucliean istance between the ata point an the line segment etermine by the ata points p an p 2. Let R(r, p, p 2 ) be the region that is etermine by the parameters p, p 2 an r, where p an p 2 correspons to the beginning an the ening of a line segment, an r the raius that etermines the limits of the region R. Let P(,p,p 2 ) be the angle between the line segment etermine by the ata points p an p 2, an the shortest line that connects the ata point with that line segment. Let be the subset of ata points that belong to the region R, efines as:, p, p / 2 E, p p D P 2, 2 r (6) Let ρ(r) the relative ensity of the region R(r, p, p 2 ), efine as: R L( p, p 2 ) (7) Where correspons to the carinality of the subset. Then the cylinrical istance θ(r, p, p 2 ) is efine as: r, p, R (8) Fig. 4. The image shows an example of a region R generate to measure the cilynrical istance between two centrois (re an black circles). a) b) p Meaning that enser the region s, closer will be the ata points consiere. As a result of the efinition of the relative ensity of R, ρ(r), if this region oes not contain any ata point, then the value of the cylinrical istance is equal to the value of the Eucliean istance between p an p 2. c) r Fig. 5. The images show graphically the process of creation of the cylinrical region R, first (a) the two ata points of which the istance is to be calculate are selecte (p an ). Then (b) the line segment that connects both ata points is built, an finally (c) using the parameter r (raius) the region is built an the ata points that fall insie are use for the istance estimation (). ) B. Determination of the set of Data Points in R To obtain the ata points that belong to the region s necessary to etermine first, which ata points have a perpenicular proection over the line segment etermine by p an p 2, the ata points of which the istance is to be calculate. To perform this, it is use the law of the cosines. The use of this law has the avantage that allows avoiing the complexities of working with geometries of high imensions, an at the same time, it generalizes the algorithm to ifferent features spaces. To apply this law a triangle whose vertices are p, p 2 an the ata point, whose membership to the region s to be etermine, is built, as illustrate in Fig. 6. Then, the lengths of the sies of the triangle are calculate, an finally, the law of the cosines is applie with the purpose of obtaining the

4 measures of the inner angles of the triangle. If the two angles between the line segment p p an the two sies of the triangle 2 p an are not greater than π/2, then the ata point has a perpenicular proection over the central line segment of the region R. If it is the case, the secon conition to belong to this region it is the length of that proection (the line segment conforme by the ata point an the nearest point of the central line segment) is not greater than the parameter r, the raius of the region R. p α β α p β Fig. 6. The images show the two ifferent configurations of the triangles mae by the ata points of which the istance is to be calculate (p an ) an a ata point belonging to the ata set (). When the ata point has a perpenicular proection over the base of the triangle, the inner angles of the base of the triangle are always less or equal than π/2 (left image). When it is not the case, one of these angles will be always greater than π/2 (right image).. ylinrical Distance Algorithm Input: ata points p,, raius r, ataset D Begin n (Number of ata points belonging to R) = 0 alculate the length of p For each Data Point in D Do alculate the length of p an alculate the angles α an β If α<=π/2 an β<=π/2 then alculate E(, p, p 2 ), the Eucliean istance between an p If E(, p, p 2 )<=r Then n=n+ En If En If En For θ=l(p,p 2 )/(n+) En D. Definition of the New Inex The new inex, calle the cylinrical version of Davies- Boulin inex, DB, is efine as: k DB Ri (9) k i Where k is the number of clusters an R i is efine as: max R (0) i Where R is the similarity measure between clusters i an, an is efine as: S R () r, v, v i Where θ(r, v i, v ) correspons to the cylinrical istance between the v i, the centroi of the cluster i, an v, the centroi of the cluster, an r to the raius of the cylinrical region. The ispersion S i of i cluster is efine as usual in the traitional Davies-Boulin inex. IV. RESULTS A series of experiments on ifferent sets of test were mae to compare the performance of the propose version (DB ) with the original version of the Davies-Boulin inex. In the process of evaluation the Ran inex was use, which allows to measure the level of similitue between two partitions, with values ranging from zero (minimal similitue) to one (maximal similitue) [6]. Another coefficient that was use is the Pearson correlation coefficient. This can take values from - to +. A value of + show that the variables are perfectly linear relate by an increasing relationship, a value of - shows that the variables are perfectly linear relate by an ecreasing relationship, an a value of 0 shows that the variables are not linear relate by each other. It is consiere a strong correlation if the correlation coefficient is greater than 0.8 an a weak correlation if the correlation coefficient is less than 0.5 [7]. The methoology use was the following: first, the ranges of values of the features of each ataset were scale to a range of 0 to 00. Then, applying the clustering algorithm k-means with variable parameters were obtaine 20 ifferent partitions on each ataset use. Then, for each partition obtaine, were calculate the original version of the Davies-Boulin inex (DB), the propose version of the inex (DB ) using 5 ifferent values for the raius (3, 5, 8, 0 an 5) an the Ran inex, with the obective to see the similitue between the partition generate an the original classes of the ata set. Finally was obtaine the Pearson correlation between each version of the DB inex an the Ran inex consiering the 20 experiments. This process was applie over 8 ifferent atasets. They are the IRIS Dataset (3 classes, 4 features an 50 instances) [8], Ecoli Data Set (8 classes, 7 features an 336 instances) [8], Wine Data Set (3 classes, 3 features an 78 instances) [8], Vertebral olumn Data Set (3 classes, 6 features an 30 instances) [8], limate Moel mulation rashes Data Set (2 classes, 8 features, 540 instances) [8], Glass Ientification Data Set (6 classes, 9 features, 24 instances) [8], Page Block lassification Data Set (5 classes, 0 features, 5473 instances) [8] an an Artificial Dataset of Gaussian istributions (3 classes, 2 features, 3500 instances). The Table shows the values of Pearson coefficient obtaine by each of the inex evaluate. Because the obective of the Davies-Boulin inex an its erivatives is to be minimize, a high negative value in the Pearson coefficient inicates a goo performance of the inex. Those values which are highlighte inicate when the DB inex ha the best performance. V. ONLUSIONS AND FUTURE WORKS In four of the atasets the propose inex ha better results than the original version (Ecoli, Iris, Page an Artificial), showing a very goo performance. In other two atasets where the original Davies-Boulin inex showe a goo performance too (Wine an limate) the results were equivalent, even with

5 the ifferent raius use in the new version. An in the remaining two atasets where the original version of the inex showe a ba performance (olumn an Glass) the new version was not able to improve it. As a conclusion, the results are very promising, but, at the same time, they suggest that a more eep analysis is necessary for unerstaning the behavior of the inex, the new version an the original one, their relations with the structural characteristics of the atasets an the influence of the clustering algorithm use for generating the partitions. An explanation of the occasions where the new inex showe the same goo performance than the original one coul be that sometimes the structural characteristics of the atasets (an the partitions generate) make the consierations of the ensities irrelevant, an in the occasions where the new inex couln't improve the ba performance of the original coul be ue to the structural characteristics of the atasets make impossible for the Davies-Boulin inex to obtain goo results, an it woul be a limitation to any new version base on it. As a future works, it is possible to obtain better results aing moification to the cylinrical istance, with the obective of taking a more accurate measurement of the ensities involve, for example, moifying the region R or optimizing the raius value. TABLE I. DB DB Raius Wine Ecoli Iris olumn limate Glass Page Artificial REFERENES [] A.K. Jain, M.N. Murty, O.J. Flynn Data lustering: a review, AM omputing Surveys, Vol. 3, No. 3, September 999 [2] M. Halkii, Y. Batistakis, M. Vazirgiannis, "On lustering Valiation Techniques", Intelligent Information Systems Journal, Kluwer Pulishers, 7(2-3): 07-45, 200 [3] M. Halkii, Y. Batistakis, M. Vazirgiannis. "luster Valiity Methos: Part I", SIGMOD Recor, June 2002 [4] M. Halkii, Y. Batistakis, M. Vazirgiannis. "luster Valiity Methos: Part II", SIGMOD Recor, September [5] Nikhil R. Pal, J. Biswas: luster valiation using graph theoretic concepts. Pattern Recognition 30(6): (997) [6] Guoun Gan, haoqun Ma, Jianhong Wu, Data lustering Theory, algorithms an applications SIAM, Society for Inustrial an Applie Mathematics (May 30, 2007) [7] Sorana-Daniela BOLBOAĂ, Lorentz JÄNTSHI, Pearson versus Spearman, Kenall's Tau orrelation Analysis onstructure-activity Relationships of Biologic Active ompouns, Leonaro Journal of Sciences, Issue 9, July-December 2006, p [8] Frank, A. & Asuncion, A. (200). UI Machine Learning Repository [ Irvine, A: University of alifornia, School of Information an omputer Science. [9] Ester, M., Kriegel, H.-P., Saner, J., an Xu, X A ensity-base algorithm for iscovering clusters in large spatial atabases with noise. Proc. 2n Int. onf. on Knowlege Discovery an Data Mining. Portlan, OR, pp [0] Dash, M., Liu, H. an Xu, X. (200), "+>2: Merging Distance an Density Base lustering", In Proceeing of the 7th International onference on Database Systems for Avance Applications (DASFAA '0), pages Hong Kong. IEEE omputer Society Publisher.