A New Measure of Cluster Validity Using Line Symmetry *

Transcription

1 JOURAL OF IFORMATIO SCIECE AD EGIEERIG 30, (014) A ew Measure of Cluster Valdty Usng Lne Symmetry * CHIE-HSIG CHOU 1, YI-ZEG HSIEH AD MU-CHU SU 1 Department of Electrcal Engneerng Tamkang Unversty Tamhsu, 51 Tawan Department of Computer Scence and Informaton Engneerng atonal Central Unversty Chungl, 30 Tawan E-mal: muchun@cse.ncu.edu.tw Many real-world and man-made obects are symmetry, therefore, t s reasonable to assume that some knd of symmetry may exst n data clusters. In ths paper a new cluster valdty measure whch adopts a non-metrc dstance measure based on the dea of lne symmetry s presented. The proposed valdty measure can be appled n fndng the number of clusters of dfferent geometrcal structures. Several data sets are used to llustrate the performance of the proposed measure. Keywords: cluster valdty, clusterng algorthm, lne symmetry, cluster analyss, smlarty measure, unsupervsed learnng 1. ITRODUCTIO Cluster analyss s one of the basc tools for explorng the underlyng structure of a gven data set and plays an mportant role n many applcatons [1-6]. In cluster analyss, two crucal problems requred to be solved are (1) the determnng of the smlarty measure based on whch patterns are assgned to the correspondng clusters and () the determnng of the optmal number of clusters. Whle the determnng of the smlarty measure s the so-called data clusterng problem, the estmaton of the number of clusters n the data set s the cluster valdty problem. In order to mathematcally dentfy clusters n a data set, t s usually necessary to frst defne a measure of smlarty or proxmty whch wll establsh a rule for assgnng patterns to the doman of a partcular cluster center. Recently, several dfferent clusterng algorthms have been proposed to deal wth clusters wth varous geometrc shapes. These algorthms can detect compact clusters [7], straght lnes [8], shells [9-11], contours wth polygonal boundares [1] or well-separated non-convex clusters [13]. One thng that should be emphaszed s that there s no cluster algorthm whch can tackle all knds of clusters. Some comprehensve overvew of clusterng algorthms can be found n the lterature [1-4]. In fact, f cluster analyss s to make a sgnfcant contrbuton to engneerng applcatons, much more attenton must be pad to cluster valdty ssues that are concerned wth determnng the optmal number of clusters and checkng the qualty of clusterng results. For the partton-based clusterng algorthm [7-13], the cluster number should be decded n pror. Bascally, there are three dfferent approaches to the determnaton of Receved January 11, 01; revsed July 3, 01; accepted September 11, 01. Communcated by Vncent S. Tseng. * Ths paper was partly supported by the atonal Scence Councl, Tawan, under SC E Y3 and for the Center for Dynamcal Bomarkers and Translatonal Medcne, atonal Central Unversty, SC I , and SC 10-1-E

2 444 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU the cluster number of a data set. The frst approach s to use a certan global valdty measure to valdate clusterng results for a range of cluster numbers [14-19]. The second approach s based on the dea of performng progressve clusterng [0-]. The thrd approach s the proecton-based approach [3-9]. Proecton algorthms allow us to vsualze hgh-dmensonal data as a two-dmensonal or three-dmensonal scatter plot. The focus of the paper s not to argue whch approach s the best one. If one decdes to choose the frst approach then we try to provde a new measure to valdate clusterng results more effectvely than exstng measures. The goal of ths paper s to propose a new valdty measure that s able to deal wth clusters wth lne symmetry structure. Many dfferent cluster valdty measures have been proposed [14-19, 30-38], such as the Dunn s separaton measure [14], the Bezdek s partton coeffcent [15], the Xe- Ben s separaton measure [16], Daves-Bouldn s measure [17], the Gath-Geva s measure [18], the CS measure [19] etc. Some of these valdty measures assume a certan geometrcal structure n cluster shapes. For example, the Gath-Geva s valdty measure that uses the value of fuzzy hypervolume as a measure s a good choce for compact hyperellpsodal clusters. However, t s a bad choce for shell clusters snce the decson as to whether t s a well or badly recognzed ellpsodal shell should be ndependent of the rad or the volume of ellpses. A mnmzaton of the fuzzy hypervolume makes no sense for the recognton of ellpsodal shells. Hence, some specal valdty measures (such as Dave s fuzzy shell covarance matrx [30] and shell thckness [31]) are proposed for shell clusters. Dependng on the desred results, a partcular valdty measure should be chosen for the respectve applcaton. In many cases, several dfferent geometrcal structures (e.g. compact clusters and shell clusters) may smultaneously exst n a data set. Furthermore, clusters can be of arbtrary shapes. Examples can be found n varous chromosome mages and mages of obects wth dfferent geometrcal structures. Most of the common crtera for cluster valdty cannot work well n complex real data wth a large varablty of cluster shapes. Snce clusters can be of arbtrary shapes and szes, we have to seek a more flexble valdty measure to deal wth the problem of geometrcal shapes. The laws of nature gve symmetry or near-symmetry to ther products. Lookng around us, we get the mmedate mpresson that almost every nterestng real-world obects (e.g. flowers, starfsh, human faces, etc) or man-made obects (e.g. archtectures, rose wndows, 3C products, etc.) own some knd of generalzed form of symmetry (e.g. pont symmetry, radal symmetry, reflectve symmetry or lne symmetry, etc.). In addton to the shapes of obects, symmetry s also an mportant parameter n physcal and chemcal processes. Snce symmetry s so common n nature, t s reasonable to assume some knd of symmetry exst n the structures of data clusters. Therefore, we may assgn data to a cluster f they present a symmetrcal structure wth respect to the correspondng cluster center. Those aforementoned symmetry operators [39-43] are effcent and affectve n mage processng for the detecton of obects wth some knd of symmetry; however, t seems that there s no smple way to generalze them to cluster hgh-dmensonal data. Although obects wth pont symmetry s very wdespread, lne symmetry s the most common type of symmetry around us. In ths paper, we try to explore the possblty of usng lne symmetrcal propertes as a new valdty measure to deal wth data wth dfferent geometrcal structures. The proposed valdty measure can be appled n fndng the number of clusters of dfferent

3 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY 445 geometrcal structures. Several data sets are used to llustrate the performance of the proposed measure. The organzaton of the rest of the paper s as follows. In Secton, we brefly revewed some popular valdty measure. In Secton 3, we frst ntroduced the dea of lne symmetry dstance measures. Then the proposed valdty measure employng the lne symmetry dstance was fully dscussed. Several examples were used to demonstrate the effectveness of the new valdty measure. Secton 4 presents the smulaton results. Fnally, Secton 5 presents the concluson.. CLUSTER VALIDITY MEASURES Cluster valdaton refers to procedures that evaluate the clusterng results n a quanttatve and obectve fashon. Some knds of valdty measures are usually adopted to measure the adequacy of a structure recovered through cluster analyss. Determnng the correct number of clusters n a data set has been, by far, the most common applcaton of cluster valdty. Bensad et al. [44] grouped valdty measures nto three categores. The frst category conssts of valdty measures that evaluate the propertes of the crsp structure mposed on the data by the clusterng algorthm [14, 17]. The second category conssts of measures that use the membershp degrees produced by the correspondng fuzzy clusterng algorthms (e.g. the FCM algorthm) [3, 15], where the membershp degree represents the possblty of a data pattern belongng to the specfed cluster. The thrd category conssts of valdty measures that take nto account not only the membershp degrees but also the data themselves [16, 18, 31]. Each has ts own consderatons and lmtatons. A comparatve examnaton of thrty valdty measures s presented n [33] and an overvew of the varous measures can be found n [45]. Snce t s not feasble to attempt a comprehensve comparson of our proposed valdty measure wth many others, we ust chose three of the popular measures for comparsons. These three measures have dfferent ratonales and propertes. Consder a partton of the data set X = {x, = 1,,, ) and the center of each cluster v ( = 1,,, c), where s the data number and c represents the cluster number. Partton coeffcent (PC) [15]: Bezdek desgns the partton coeffcent (PC) to measure the amount of overlap between clusters. He defnes the partton coeffcent (PC) as follows: c ( u PC( c) ) (1) where u ( = 1,,, c; = 1,,, ) s the membershp of data pattern n cluster. The closer ths value s to unty the better the data are classfed. In the case of a hard partton, we obtan the maxmum value PC(c) = 1. If we are lookng for a good partton, we am at a partton wth a maxmum partton coeffcent. Ths knd of partton yelds the most unambguous assgnment. The dsadvantages of the partton coeffcent are ts monotonc decreasng wth c and the lack of drect connecton to some propertes of the data themselves.

4 446 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU Classfcaton entropy (CE) [3]: The classfcaton entropy measure strongly resembles the partton coeffcent; however, t s related on Shannon s nformaton theory. The classfcaton entropy s defned as follows: c 1 CE() c u log( u ). () 1 1 If we have a crsp partton we have the most nformaton (.e. the mnmum entropy). Consequently, a partton wth the mnmum entropy s regarded as a good partton. Although ths valdty measure s based on Shannon s nformaton theory, t can be vewed as a measure for the fuzzness of cluster partton, whch s very smlar to the partton coeffcents. Bezdek proves that the relaton 0 1 PC(c) CE(c) holds for all probablstc cluster parttons c. The lmtaton of the classfcaton entropy can be attrbuted to ts apparent monotony and to an extent, to the heurstc nature of the ratonale underlyng ts formulaton. Separaton measure (S) [16]: Xe and Ben ntroduce a valdty measure, smlar to the Dunn s separaton measure [14]. The separaton measure S s defned as follows: c c u x v u x v Sc () mn { v v } mn, 1,, c and mn m d n mn (3) where x v denotes the Eucldean dstance between the pattern, x, and the cluster center, v, and d mn represents the mnmum Eucldean dstance between cluster centers. The separaton measure S s based on fuzzy compactness and separaton. Whle the numerator of Eq. (3) measures the total varance (or the compactness) of each fuzzy cluster, the denomnator measures the separaton defned as the mnmum dstance between cluster centers. The smallest value of S(c) ndcates a vald optmal partton because u wll be hgh when x v s low and well separated clusters wll produce a hgh value of d mn. Before ntroducng the proposed measure, we frst use the example shown n Fg. 1 (a) to llustrate the motvaton of the new measure. There are three clusters n the dataset contanng a sphercal cluster and two lnear clusters, and two lnear clusters are close to each other. Frst, we use the FCM algorthm [3] to group the data set nto clusters (shown n Fg. 1 (b)) and 3 clusters (shown n Fg. 1 (c)). For the case of clusters, the membershp degrees for the two patterns, x 1 and x (shown n Fg. 1 (b)), are u 11 = 0.991, u 1 = 0.009, u 1 = and u = As for the case of 3 clusters, the membershp degrees become u 11 = 0.544, u 1 = 0.033, u 31 = 0.43, u 1 = 0.909, u = 0.05, and u 3 = By examnng these clusterng results, we have the followng observatons:

5 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY 447 (a) (b) (c) Fg. 1. (a) The data set contanng a combnaton of a sphercal cluster and two lnear clusters. The clusterng result acheved by the FCM algorthm at (b) c = ; (c) c = Whle the membershp degree, u 1, of the data pattern, x, ncreases from (for the case of clusters) to (for the case of 3 clusters); the membershp degree, u 11, of the data pattern, x 1, decreases from (for the case of clusters) to (for the case of 3 clusters). For data patterns located at the upper regon of cluster 1 (e.g. x ) or the rght regon of cluster 3, the ncrement of ther membershp degrees nduced by parttonng the data set nto 3 clusters ndcates that t s approprate to partton the data set nto 3 clusters. On the contrary, the decrement of the membershp degrees for data patterns located at the lower regon of cluster 1 (e.g. x 1 ) or the left regon of cluster 3 ndcates that the partton of 3 clusters s not a good choce. Because the total decrement of membershp degrees s larger than the total ncrement of membershp degrees due to the ncreases of the number of clusters, the value of the PC measure decreases from 0.93 to Understandably, the PC measure favors the partton of two clusters n ths example. A smlar reason can also be appled to explan why the classfcaton entropy measure CE can not fnd the rght number of clusters for ths data set ether.. ot only dose the value of the numerator n Eq. (3) changes from to when the number of clusters changes from to 3, but also the mnmum Eucldean dstance between cluster centers, d mn, changes from 4.51 to.17. Whle the decrement of the numerator ndcates that the degree of compactness ncreases, the decrement of the denomnator ndcates the degree of separaton decreases. Fnally, the value of S(c) changes from to Therefore, the S(c) measure prefers the partton of clusters. 3. The two measures, PC and CE, use the basc heurstc rule that good clusters are not fuzzy. Consequently, they rely solely upon the membershps to determne the fuzzness of the partton. Ths example confrms what one already knows: valdty measures that use only the fuzzy membershp grades usually lack drect connecton to some geometrcal propertes of the data themselves. 4. The separaton measure S uses c u x v 1 1 to measure the compactness. For compact sphercal clusters, ths knd of measure may be effectve; however, for some clusters wth dfferent geometrcal structures (e.g. lnear clusters, shells, etc.), t may not work well. In ths example, the data set ncludes not only a sphercal

6 448 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU cluster but also two lnear clusters so that t s not very surprsng when we fnd out that the separaton measure S can not fnd the correct partton. 5. If we move two lnear clusters closer to the sphercal cluster (e.g. the dstances between each other do not vary too much), the measures, PC, CE, and S, can fnd the correct number of clusters. Each valdty measure mplctly mposes a certan structure on the data set, and f the data set happens to exhbt the structures, the valdty measure can fnd the correct partton. o matter how good a valdty measure s, there always are lmtatons assocated wth t snce clusters can be of arbtrary shapes and szes. Hence, there s no sngle best valdty measure whch can work well for any knd of clusterng results. These observatons motvate us to explore whether there s a valdty measure whch can deal wth the valdaton of clusters wth dfferent geometrc shapes under some certan assumptons. In our prevous work [46], we fnd that lne symmetry dstance s very effectve n clusterng data wth dfferent structures. Consequently, we try to propose a new valdty measure whch s based on the dea of lne symmetry. Ths new measure consders not only the dstrbutons of the data set but also the degree of symmetry n each cluster. 3. THE LIE SYMMETRY DISTACE AD THE VALIDITY MEASURE USIG LIE SYMMETRY What s lne symmetry? For a -dmenstonal fgure, f t can be folded n such a way that one-half of t les exactly on the other half s sad to have lne symmetry. The dea of lne symmetry s very clear and smple but an mmedate problem s how to fnd a metrc to measure lne symmetry. A knd of lne symmetry dstance was proposed n [47-49]. In ths approach, the symmetrcal lne of a data set s defned by a center vector and an angle between the maor axs of the data set and the x axs. The nformaton of the maor axs of the data ponts belongng to a class or a cluster s computed by the moment of order (p + q) method. Then the maor axs s treated as the symmetrcal lne of that class or cluster. Furthermore, the Kd-tree based nearest neghbor search s used to reduce the complexty of computng the lne dstance. It s not clear how the proposed lne symmetry dstance can be generalzed to hgh-dmensonal space. In ther approach, the symmetrcal lne s defned by a pont and an angle between the maor axs and the x axs. As we know, a pont combned wth an angle can defne one and only one lne n -dmensonal space but wll produce an nfnte number of lnes n hgh-dmensonal space. In addton, the authors themselves stated that the performance of ther lne symmetry dstance depends on the selecton of the number of nearest neghbors used n the computaton of lne symmetry dstance. The authors n [47-49] proposed another smlar defnton of lne symmetry n [50]. They assumed that a data set s symmetrcal then t should be also be symmetrc wth respect to the frst prncpal axs of the data set [50]. Ths assumpton may not be hold for many stuatons. There are obects whch s symmetrcal wth respectve to ther second prncpal axs but not symmetrcal to ther frst prncpal axs. Therefore, a new lne symmetry dstance whch can deal wth hgh-dmensonal data ponts s developed by our prevous work [46].

7 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY The Lne Symmetry Dstance In one of our prevous work, a so-called pont symmetry dstance (symmetry about a cluster center) was proposed n [51]. In [5], Bandyopadhyay and Saha proposed a new pont symmetry-based dstance measure. Ther algorthm apples GA and Kd-tree based nearest neghbor search to reduce the complexty of fndng the closest symmetrc pont. Although obect wth pont symmetry s very wdespread, lne symmetry s the most common type of symmetry around us. Based on the pont symmetry dstance, a new lne symmetry dstance s developed n our another work [46]. Before we present the defnton of the proposed lne symmetry dstance, we brefly revew the defnton of the pont symmetry dstance. A -dmensonal fgure s wth pont symmetry f t can be rotated 180 degrees about a pont onto tself. To generalze ths dea of pont symmetry to measure the degree of pont symmetry for a set of hgh-dmensonal data patterns, a pont symmetry dstance s defned as follows [51]. Gven a data set X contanng patterns, x, = 1,,, and a reference vector c (e.g. a cluster center), the pont symmetry dstance between a pattern x and the reference vector c s defned as ( x c) ( x c) dps ( x, c) mn. ( x c x c ) 1,, and (4) ote that Eq. (4) s mnmzed (.e., d ps (x, c) = 0) f there s a pattern x ps * = (c x ) exsts n the data set (see Fg. ). The pattern x ps * s then denoted as the pont symmetrcal data pattern relatve to x wth respect to c as shown n Fg.. Fg.. An example for llustratng the bas problem ncurred by the orgnal verson of the pont symmetry dstance. By further analyzng the pont symmetry dstance defned n Eq. (4), we fnd that the dstance measure has a bas for data patterns wth a larger dstance from the center c even f they are wth the same dstance to the pont symmetrcal data pattern x ps *. For example, two data patterns, x 1 and x, are wth the same dstance to the symmetrcal data pattern x ps * as shown n Fg.. Accordng to the defnton of Eq. (4), we fnd that ( x c) ( x1 c) ( x c) ( x c) even f these two patterns are wth the same ( x c x1 c ) ( x c x c ) dstances to the pont symmetrcal data pattern x ps *. To amend ths flaw, we modfy the pont symmetry dstance as follows:

8 450 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU ( x c) ( x c) dps ( x, c) mn. ( x c x c x x ) 1,, and * * (5) (a) (b) Fg. 3. The dstance contours generated by the two versons of the pont symmetry dstance; (a) By Eq. (4). (b) By Eq. (5). Fg. 3 shows examples of the dstance functons defned n Eqs. (4) and (5) for the case of x = (1, 0), c = (0, 0) T, and x ps * = (1, 0) T (correspondng to three small crcle ponts n the fgure). For each pont n the pcture plane, the dstance s used for the ntensty. Here, small dstance gets a dark color and large dstance gets a lght grey color. From Fg. 3, we fnd that Fg. 3 (b) presents concentrc contours wth center at the pont symmetrcal data pattern x ps * but Fg. 3 (a) presents non-concentrc contours. Therefore, the bas problem has been fxed by modfyng the denomnator of Eq. (4). Followng the defnton of a fgure wth pont symmetry, we may pont out that the lne symmetrcal data pattern relatve to x wth respect to a center c and a unt drecton vector e s the data pattern x ls * as shown n Fg. 4 where the pont symmetrcal data pattern relatve to x wth respect to a center c s denoted as x ps *. Smlar to the defnton of the modfed verson of the pont symmetry dstance defned n Eq. (5), the defnton of the lne symmetry dstance s gven as follows. Gven a reference vector c and a unt drecton vector e, the lne symmetry dstance of a pattern x n the data set X wth respectve to a reference vector c and a unt drecton vector e s defned as d ls ( x, c, e) ( x p) ( x p) mn 1,, ls ls ( x p x p x x ) (6) and * where the data pattern p s the normal proecton of the data pattern x onto the lne formed by the data pattern c and the unt drecton vector e. As for how to fnd the three vectors, c, p and e from the data set X, the computatonal procedure wll be explaned as follows. Frst of all, the mean vector c and the covarance matrx Cov can be approxmated from the data patterns by * c 1 x 1, (7)

9 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY 451 Fg. 4. A geometrcal explanaton about the defntons of pont symmetry and lne symmetry. 1 T T Cov xx cc. (8) 1 Snce the covarance matrx Cov s real and symmetrc, we can fnd a set of n orthonormal egenvectors from the matrx. One of the n orthonormal egenvector wll be chosen to be the unt drecton vector e. The vector e s then regarded as the symmetrc lne of the data set. For each cluster, the egenvector wth the smallest amount of lne symmetry dstances (e.g., the * th egenvector n ths case) s chosen to be the symmetrcal lne of that cluster based on the followng mnmum-value crteron:, * Arg mn d ls ( x, c k, e ) 1 (9) n S k k where S k represents the data set consstng of data ponts belongng to cluster k and e k represents the th egenvector computed from the covarance matrx correspondng to the data set S k. The normal proected data pattern p can be computed by p = c + c p e = c + (x c) T e e. (10) After we have computed the normal proected data pattern p, we can fnd the lne symmetrcal data pattern, x ls *, relatve to x wth respect to the center c and the unt drecton vector e by the followng equaton: x ls * x x p x x c ( p x ) p c ( p x ) (11) x x c T ( x c) e ( p x ).

10 45 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU 3. The Valdty Measure Usng Lne Symmetry The proposed valdty measure s referred to as LS measure and s computed as follows. Consder a partton of the data set X = {x ; = 1,,, } and each data pattern x s assgned to ts correspondng cluster by a partcular clusterng algorthm. In order to calculate lne symmetry dstance, we need re-compute the cluster center v (.e. mean vector) and the covarance matrx Cov by usng the followng equaton: 1 v x (1) x S 1 T T Cov x x v v (13) x S where S s the set whose elements are the data patterns assgned to the th cluster and s the number of elements n S. ote that we assgn data patterns to the correspondng clusters usng the maxmum membershp grade crteron f the clusterng result s acheved by fuzzy clusterng algorthms. Then we compute the degree of lne symmetry of cluster by 1 k 1 k LS d c ( x, v, e* ) ( dls ( x, v, e* ) d 0 ) d e ( x, v ) x S x S (14) where the dstance, d c (x, v, e k *), represents the composte lne symmetry dstance defned n Eq. (14), d e (x, v ) represents the Eucldean dstance between x and v, and d 0 s a small valued postve constant. The reason why we use the composte lne symmetry dstance, d c (x, v, e k *), rather than the lne symmetry dstance tself, d ls (x, v, e k *), s as follows. The lne symmetry dstance tself may not work for stuatons where clusters themselves are lne symmetrc. A possble soluton to overcome ths lmtaton s to combne the lne symmetrc dstance wth the Eucldean dstance n such a way that f data patterns are relatvely close, then the lne symmetry s more mportant. On the other hand, f the data patterns are very far, then the Eucldean dstance s more mportant. The smaller the value of LS s the larger the degree of lne symmetry of cluster has. The separaton of clusters s defned as the mnmum dstance between clusters d mn d ( v, v ). mn mn, 1,, c and mn e m n (15) Fnally, the LS measure s obtaned by averagng the rato of the degree of lne symmetry of the cluster to the separaton over all clusters, more explctly c 1 LS c 1 LS() c d mn

11 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY 453 c 1 1 c 1 x S c 1 1 c 1 x S k dc( x, v, e* ) d mn k ( dls( x, v, e* ) d0) de( x, v). d mn (16) Whle the numerator of Eq. (16) measures the average degree of lne symmetry of entre clusters, the denomnator measures the separaton defned as the mnmum dstance between cluster centers. Ths measure favors the creaton of a set of clusters that are maxmally separated from one another and each cluster has the degree of lne symmetry as large as possble. Therefore, a partton wth the mnmum LS(c) s regarded as a good partton. 4. EXPERIMETAL RESULTS We llustrate the effectveness of the proposed valdty measure by testng some data sets wth dfferent geometrcal structures. For the comparson purpose, these data sets were also tested by the three popular valdty measures the partton coeffcent (PC), the classfcaton entropy (CE) and the Xe-Ben s separaton measure (S). The FCM algorthm or the Gustafson-Kessel (GK) algorthm [7] s appled to cluster these data sets at each cluster number c from c = to c = 10. Accordng the expermental results, we notce that the FCM algorthm s not sutable to be appled for ellpsodal clusters, but the GK algorthm can be used to cluster sphercal and ellpsodal clusters. Therefore, n examples to 5, the GK algorthm s appled to cluster the data patterns. A value of the fuzzy exponent m = was chosen for the FCM algorthm and the GK algorthm. The parameter d 0 was chosen to be for the composte lne symmetry dstance. Example 1: The data set shown n Fg. 1 (a) conssts of a combnaton of a sphercal cluster and two lnear clusters. We generated the data patterns randomly wth unform dstrbuton. The total number of data patterns s 400. In ths example, the FCM algorthm s appled to cluster the data patterns. The total number of data patterns s 400. The performance of each valdty measure s tabulated n Table 1. In Table 1 and others to follow, the hghlghted (bold and shaded) entres correspond to optmal values of the measures. ote that only the LS valdty measure fnds the optmal cluster s three, but the PC, CE and S valdty measures choose two clusters as the optmal partton. The clusterng result acheved by the FCM algorthm at c = 3 s shown n Fg. 1 (c). Table 1. umercal values of the valdty measures for Example 1. C PC CE S LS

12 454 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU Example : We randomly generated a mxture of sphercal and ellpsodal clusters wth Gaussan dstrbuton. Ths data set conssts of 850 data patterns dstrbuted on fve clusters, as shown n Fg. 5 (a). The clusterng results acheved by the GK and FCM algorthms at c = 5 are shown n Fgs. 5 (b) and (c). We notce that the FCM algorthm s not sutable to be appled for ellpsodal clusters; the two non-overlapped compact clusters are not clustered correctly. Therefore, n ths example, the GK algorthm s appled to cluster the data patterns. The performance of each valdty measure s gven n Table. Both the S and LS valdty measures fnd that the optmal cluster number c s at c = 5, but both the PC and CE valdty measures ndcate two clusters as the optmal partton. Table. umercal values of the valdty measures for Example. C PC CE S LS (a) (b) (c) Fg. 5. (a) The data set used n example ; (b) the clusterng result acheved by the GK algorthm at c = 5; (c) the clusterng result acheved by the FCM algorthm at c = 5. Example 3: Ths data set conssts of a combnaton of a rng-shaped cluster, a rectangular compact cluster and a lnear cluster as shown n Fg. 6 (a). We generated the data patterns randomly wth unform dstrbuton. The total number of data patterns s 400. In ths example, the GK algorthm s appled to cluster the data patterns. Ths example s consdered to show that the valdty measures, PC, CE and S, are not ndcatve of a good detecton of clusters wth dfferent geometrc shapes. The performance of each valdty measure s gven n Table 3. Only the LS valdty measure fnds the optmal cluster c at c = 3. The PC, CE and S valdty measures fal to choose the correct number of clusters, and select c = as the optmal partton. The clusterng results acheved by the GK and FCM algorthms at c = 3 are shown n Fgs. 6 (b) and (c).

13 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY 455 Table 3. umercal values of the valdty measures for Example 3. C PC CE S LS (a) (b) (c) Fg. 6. (a) The data set used n example 3; (b) the clusterng result acheved by the GK algorthm at c = 3; (c) the clusterng result acheved by the FCM algorthm at c = 3. Example 4: Ths example demonstrates an applcaton of the LS valdty measure to detect the number of obects n an mage. In mage processng, t s very mportant to fnd obects n mages. In ths example, these obects have dfferent geometrc shapes. Fg. 7 (a) shows a real mage consstng of a moble phone, a doll, and an obect of crescent. Frst, we apply the thresholdng technque to extract the obects from the orgnal mage (see Fg. 7 (b)). Then we transfer the obect pxels to be the data patterns. The GK algorthm s used to cluster the data set. Table 4 shows the performance of each valdty measure. The LS valdty measure fnds that the optmal cluster number c s at c = 3. However, the PC, CE and S valdty measures fnd the optmal cluster number at c =. Once agan, ths example demonstrates that the proposed LS valdty measure can work well for a set of clusters of dfferent geometrcal shapes. The clusterng results acheved by the GK and FCM algorthms at c = 3 are shown n Fgs. 7 (c) and (d). Table 4. umercal values of the valdty measures for Example 4. C PC CE S LS

14 456 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU (a) (b) (c) (d) Fg. 7. (a) The orgnal mage contanng a moble phone, a toll, and an obect of crescent; (b) the bnary mage by applyng threshold to the orgnal mage; (c) the clusterng result acheved by the GK algorthm at c = 3; (d) the clusterng result acheved by the FCM algorthm at c = 3. Example 5: In ths example, there are four obects wth dfferent geometrc shapes n Fg. 8 (a), whch conssts of an adhesve tape, a knfe, a note and a cover of a teacup. The threshold technque s appled to extract the obects from the orgnal mage (see Fg. 8 (b)), and then the obect pxels are transferred to be the data patterns. The GK algorthm s used to cluster the data set. Table 5 shows the performance of each valdty measure. The LS valdty measure fnds that the optmal cluster number c s at c = 4. However, the PC and CE valdty measures fnd the optmal cluster number at c =, and the S valdty measure fnds the optmal cluster number at c = 3. The clusterng results acheved by the GK and FCM algorthms at c = 4 are shown n Fgs. 8 (c) and (d). Table 5. umercal values of the valdty measures for example 5. C PC CE S LS DISCUSSIO AD COCLUSIO Based on the lne symmetry dstance, a new measure LS s then proposed for cluster valdaton. The smulaton results reveal the nterestng observatons about the valdty measures dscussed n ths paper. The S valdty measure fals f not n one example, at

15 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY 457 (a) (b) (c) (d) Fg. 8. (a) The orgnal mage contanng an adhesve tape, a knfe, a note and a cover of a teacup; (b) the bnary mage by applyng threshold to the orgnal mage; (c) the clusterng result acheved by the GK algorthm at c = 4; (d) the clusterng result acheved by the FCM algorthm at c = 4. least n the other, and the PC and CE valdty measures fal n all the examples. The proposed LS valdty measure shows that consstency for these and several other examples that are tred. Because the lne symmetry dstance s extended from the pont symmetry dstance, one may nterest the performance f the cluster valdty measure s based on pont symmetry. Accordng our smulatons, f the clusters wth pont symmetry structure (e.g. examples and 3), the valdty measure based on pont symmetry should also perform well for these cases. However, f the geometrcal structure of clusters s lne symmetry (e.g. example 4), t not a good choce to use pont symmetry as valdty measure. Although these smulatons show that the new measure outperforms the other three measures, we want to emphasze that there are also lmtatons assocated wth ths new measure. Frst, we need to assume that clusters are lne symmetrcal structures. If the data set does not follow the assumpton, the measure may not work well. Second, for large data sets, the determnaton of the measure s very computatonally expensve. The prce pad for the flexblty n detectng cluster number s the ncrease of computatonal complexty. The computng complexty s O( ), where ndcates the data number. In fact, a lot of future work can be done to mprove not only the lne symmetry dstance but also the LS measure. REFERECES 1. A. K. Jan and R. C. Dubes, Algorthms for Clusterng Data, Englewood Clffs, Prentce Hall, J, 1988.

16 458 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classfcaton, Wley, Y, J. Bezdek, Pattern Recognton wth Fuzzy Obectve Functon Algorthms, Plenum, Y, F. Höppner, F. Klawonn, R. Kruse, and T. Runkler, Fuzzy Cluster Analyss-Methods for Classfcaton, Data Analyss and Image Recognton, John Wley & Sons, Ltd., C.-S. Fahn and Y.-C. Lo, On the clusterng of head-related transfer functons used for 3-D sound localzaton, Journal of Informaton Scence and Engneerng, Vol. 19, 003, pp P.-C. Wang and J.-J. Leou, ew fuzzy herarchcal clusterng algorthms, Journal of Informaton Scence and Engneerng, Vol. 9, 1993, pp D. E. Gustafson and W. C. Kessel, Fuzzy clusterng wth a fuzzy covarance matrx, n Proceedngs of IEEE Conference on Decson and Control, 1979, pp R.. Dave, Use of the adaptve fuzzy clusterng algorthm to detect lnes n dgtal mages, Intellgent Robots and Computer Vson VIII, Vol. 119, 1989, pp R.. Dave, Fuzzy shell-clusterng and applcaton to crcle detecton n dgtal mages, Internatonal Journal General Systems, Vol. 16, 1990, pp Y. Man and I. Gath, Detecton and separaton of rng-shaped clusters usng fuzzy clusterng, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 16, 1994, pp R.. Dave and K. Bhaswan, Adaptve fuzzy c-shells clusterng and detecton of ellpses, IEEE Transactons on eural etworks, Vol. 3, o. 5, 199, pp F. Höppner, Fuzzy shell clusterng algorthms n mage processng: fuzzy c-rectangular and -rectangular shells, IEEE Transactons on Fuzzy Systems, Vol. 5, 1997, pp M. C. Su and Y. C. Lu, A new approach to clusterng data wth arbtrary shapes, Pattern Recognton, Vol. 38, 005, pp J. C. Dunn, Well separated clusters and optmal fuzzy parttons, Journal Cybern, Vol. 4, 1974, pp J. C. Bezdek, umercal taxonomy wth fuzzy sets, Journal of Mathematcal Bology, Vol. 1, 1974, pp X. L. Xe and G. Ben, A valdty measure for fuzzy Clusterng, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 13, 1991, pp D. L. Daves and D. W. Bouldn, A cluster separaton measure, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 1, 1979, pp I. Gath and A. B. Geva, Unsupervsed optmal fuzzy clusterng, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 11, 1989, pp C. H. Chou, M. C. Su, and E. La, A new cluster valdty measure and ts applcaton to mage compresson, Pattern Analyss and Applcatons, Vol. 7, 004, pp T. L. Huntsherger, C. L. Jacobs, and R. L. Cannon, Iteratve fuzzy mage segmentaton, Pattern Recognton, Vol. 18, 1985, pp R.. Dave and K. J. Patel, Progressve fuzzy clusterng algorthms for characterstc shape recognton, n Proceedngs of orth Amercan Fuzzy Informaton Processng Socety, Vol. 1, 1990, pp J. M. Jolon, P. Meer, and S. Bataouche, Robust clusterng wth applcatons n computer vson, IEEE Transactons on Pattern Analyss and Machne, Vol. 13,

17 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY , pp J. W. Shammon, A nonlnear mappng for data structure analyss, IEEE Transactons on Computers, Vol. 18, 1969, pp M. A. Kraaveld, J. Mao, and A. K. Jan, A nonlnear proecton method based on Kohonen s topology preservng maps, IEEE Transactons on eural etworks, Vol. 6, 1995, pp J. Mao and A. K. Jan, Artfcal neural networks for feature extracton and multvarate data proecton, IEEE Transactons on eural etworks, Vol. 6, 1995, pp M. C. Su,. DeClars, and T. K. Lu, Applcaton of neural networks n cluster analyss, n Proceedngs of IEEE Internatonal Conference on Systems, Man, and Cybernetcs, 1997, pp M. C. Su and H. C. Chang, A new model of self-organzng neural networks and ts applcaton n data proecton, IEEE Transactons on eural etworks, Vol. 1, 001, pp H. Yn, Data vsualzaton and manfold mappng usng the VSOM, eural etworks, Vol. 15, 00, pp M. C. Su, S. Y. Su, and Y. X. Zhao, A swarm-nspred proecton algorthm, Pattern Recognton, Vol. 4, 009, pp R.. Dave, ew measures for evaluatng fuzzy parttons nduced through c-shells clusterng, n Proceedngs of SPIE Conference on Intellgent Robot Computer Vson X, Vol. 1670, 1991, pp R.. Dave, Valdatng fuzzy parttons obtaned through c-shells clusterng, Pattern Recognton Letters, Vol. 17, 1996, pp R. Krshnapuram, H. Frogu, and O. asraou, Fuzzy and possblstc shell clusterng algorthms and ther applcaton to boundary detecton and surface approxmaton Part 1&, IEEE Transactons on Fuzzy Systems, Vol. 3, 1995, pp M. P. Wndham, Cluster valdty for fuzzy clusterng algorthms, Fuzzy Sets and Systems, Vol. 5, 1981, pp J. C. Bezdek and. R. Pal, Some new ndexes of cluster valdty, IEEE Transactons on System, Man, and Cybernetcs, Vol. 8, 1998, pp V. S. Tseng and C.-P. Kao, An effcent approach to dentfyng and valdatng clusters n multvarate datasets wth applcatons n gene expresson analyss, Journal of Informaton Scence and Engneerng, Vol. 0, 004, pp J.-S. Wang and J.-C. Chang, A cluster valdty measure wth outler detecton for support vector clusterng, IEEE Transactons on Systems, Man, and Cybernetcs, Part B, Vol. 38, 008, pp R. Jan and A. Koronos, Innovaton n the cluster valdatng technques, Fuzzy Optmzaton and Decson Makng, Vol. 7, 008, pp A. K. Jan, M.. Murthy, and P. J. Flynn, Data clusterng: A revew, ACM Computng Surveys, Vol. 31, 1999, pp D. Resfeld, H. Wolfsow, and Y. Yeshurun, Context-free attentonal operators: the generalzed symmetry transform, Internatonal Journal of Computer Vson, Vol. 14, 1995, pp D. Resfeld and Y. Yeshurun, Preprocessng of face mages: Detecton of features and pose normalsaton, Computer Vson and Image Understandng, Vol. 71, 1998,

18 460 CHIE-HSIG CHOU, YI-ZEG HSIEH AD MU-CHU SU pp R. K. K. Yp, A hough transform technque for the detecton of reflectonal symmetry and skew-symmetry, Pattern Recognton Letters, Vol. 1, 000, pp G. Loy and A. Zelnsky, Fast radal symmetry for detectng ponts of nterest, IEEE Transactons on Pattern Analyss and Machne, Vol. 5, 003, pp G. Loy and J. Eklundh, Detectng symmetry and symmetrc constellatons of features, n Proceedngs of European Conference on Computer Vson, LCS 395, 006, pp A. M. Bensad, L.O. Hall, J. C. Bezdek, C. P. Clarke, M. L. Slbger, J. A. Arrngton, and R. F. Murtagh, Valdty-guded (Re) clusterng wth applcatons to mage segmentaton, IEEE Transactons on Fuzzy Systems, Vol. 4, 1996, pp R. Dubes and A. K. Jan, Valdty studes n clusterng methodologes, Pattern Recognton, Vol. 11, 1979, pp Y. Z. Hseh, M. C. Su, C. H. Chou, and P. C. Wang, Detecton of lne-symmetry clusters, Internatonal Journal of Innovatve Computng, Informaton and Control, Vol. 7, 011, pp S. Saha, S. Bandyopadhyay, and C. T. Sngh, A new lne symmetry dstance based pattern classfer, n Proceedngs of Internatonal Jont Conference on eural etwork, 008, pp S. Saha and S. Bandyopadhyay, A new lne symmetry dstance and ts applcaton n data clusterng, Journal of Computer Scence and Technology, Vol. 4, 009, pp S. Saha and S. Bandyopadhyay, On prncple axs based lne symmetry clusterng technques, Memetc Computng, Vol. 3, 011, pp S. Bandyopadhyay and S. Saha, A new prncpal axs based lne symmetry measurement and ts applcaton to clusterng, The Proceedngs of ICOIP, LCS 5507, 009, pp M. C. Su and C. H. Chou, A modfed verson of the K-means algorthm wth a dstance based on cluster symmetry, IEEE Transactons on Pattern Analyss and Machne Intellgence, Vol. 3, 001, pp S. Bandyopadhyay and S. Saha, GAPS: A new symmetry based genetc clusterng technque, Pattern Recognton, Vol. 40, 007, pp Chen-Hsng Chou () receved the B.S. and M.S. degrees from the Department of Electrcal Engneerng, Tamkang Unversty, Tawan, n 1997 and 1999, respectvely, and the Ph.D. degree at the Department of Electrcal Engneerng from Tamkang Unversty, Tawan, n 003. He s currently an Assstant Professor of Electrcal Engneerng at Tamkang Unversty, Tawan. Hs research nterests nclude mage analyss and recognton, moble phone programmng, machne learnng, document analyss and recognton, and clusterng analyss.

19 A EW MEASURE OF CLUSTER VALIDITY USIG LIE SYMMETRY 461 Y-Zeng Hseh () receved the Ph.D. degree n Computer Scence and Informaton Engneerng from atonal Central Unversty, Taoyuan, Tawan, respectvely n 01. Hs current research nterests nclude neural networks, pattern recognton, mage processng. Mu-Chun Su () receved the B.S. degree n Electroncs Engneerng from atonal Chao Tung Unversty, Tawan, n 1986, and the M.S. and Ph.D. degrees n Electrcal Engneerng from Unversty of Maryland, College Park, n 1990 and 1993, respectvely. He was the IEEE Frankln V. Taylor Award recpent for the most outstandng paper co-authored wth Dr.. DeClars and presented to the 1991 IEEE SMC Conference. He s currently a Professor of Computer Scence and Informaton Engneerng at atonal Central Unrsty, Tawan. He s a senor member of the IEEE Computatonal Intellgence Socety and Systems, Man, and Cybernetcs Socety. Hs current research nterests nclude neural networks, fuzzy systems, assstve technologes, swarm ntellgence, effectve computng, pattern recognton, physologcal sgnal processng, and mage processng.