Research Article A Self-Adaptive Fuzzy c-means Algorithm for Determining the Optimal Number of Clusters

Transcription

1 Computatioal Itelligece ad Neurosciece Volume 216, Article ID , 12 pages Research Article A Self-Adaptive Fuzzy c-meas Algorithm for Determiig the Optimal Number of Clusters Mi Re, 1,2,3 Peiyu Liu, 1,3 Zhihao Wag, 1,3 adjigyi 1,3 1 School of Iformatio Sciece ad Egieerig, Shadog Normal Uiversity, Jia, Shadog, Chia 2 SchoolofMathematicadQuatitativeEcoomics,ShadogUiversityofFiaceadEcoomics,Jia,Shadog,Chia 3 Shadog Provicial Key Laboratory for Distributed Computer Software Novel Techology, Jia, Shadog, Chia Correspodece should be addressed to Peiyu Liu; sdkeylab@163.com Received 29 Jue 216; Accepted 17 October 216 Academic Editor: Elio Masciari Copyright 216 Mi Re et al. This is a ope access article distributed uder the Creative Commos Attributio Licese, which permits urestricted use, distributio, ad reproductio i ay medium, provided the origial work is properly cited. For the shortcomig of fuzzy c-meas algorithm (FCM) eedig to kow the umber of clusters i advace, this paper proposed a ew self-adaptive method to determie the optimal umber of clusters. Firstly, a desity-based algorithm was put forward. The algorithm, accordig to the characteristics of the dataset, automatically determied the possible maximum umber of clusters istead of usig the empirical rule ad obtaied the optimal iitial cluster cetroids, improvig the limitatio of FCM that radomly selected cluster cetroids lead the covergece result to the local miimum. Secodly, this paper, by itroducig a pealty fuctio, proposed a ew fuzzy clusterig validity idex based o fuzzy compactess ad separatio, which esured that whe the umber of clusters verged o that of objects i the dataset, the value of clusterig validity idex did ot mootoically decrease ad was close to zero, so that the optimal umber of clusters lost robustess ad decisio fuctio. The, based o these studies, a self-adaptive FCM algorithm was put forward to estimate the optimal umber of clusters by the iterative trial-ad-error process. At last, experimets were doe o the UCI, KDD Cup 1999, ad sythetic datasets, which showed that the method ot oly effectively determied the optimal umber of clusters, but also reduced the iteratio of FCM with the stable clusterig result. 1. Itroductio Cluster aalysis has a log research history. Due to its advatage of learig without a priori kowledge, it is widely applied i the fields of patter recogitio, image processig, web miig, spatiotemporal database applicatio, busiess itelligece, ad so forth. Clusterig is ofte as usupervised learig ad aims to partitio objects i a dataset ito several atural groupigs, amely, the so-called clusters, such that objects withi a cluster ted to be similar while objects belogig to differet clusters are dissimilar. Geerally, the datasets from differet applicatio fields vary i feature, ad the purposes of clusterig are multifarious. Therefore, the best method of cluster aalysis depeds o datasets ad purposes of use. There is o uiversal clusterig techology that ca be widely applicable to the diverse structures preseted by various datasets [1]. Accordig to the accumulatio rules of objects i clusters ad the methods of applyig these rules, clusterig algorithms are divided ito may types. However, for most clusterig algorithms icludig partitioal clusterig ad hierarchical clusterig, the umber of clusters is a parameter eedig to be preset, to which the quality of clusterig result is closely related. I practical applicatio, it usually relies o users experiece or backgroud kowledge i related fields. But i mostcases,theumberofclustersisukowtousers.ifthe umber is assiged too large, it may result i more complicated clusterig results which are difficult to be explaied. O the cotrary, if it is too small, a lot of valuable iformatio i clusterig result may be lost [2]. Thus, it is still a fudametal problem i the research of cluster aalysis to determie the optimal umber of clusters i a dataset [3]. Cotributios of this paper are as follows. (1) A desitybased algorithm is proposed, which ca quickly ad succictly geerate high-quality iitial cluster cetroids istead of choosig radomly, so as to stabilize the clusterig result ad also quicke the covergece of the clusterig algorithm. Besides, this algorithm ca automatically estimate the

2 2 Computatioal Itelligece ad Neurosciece maximumumberofclustersbasedothefeaturesofa dataset, hereby determiig the search rage for estimatig the optimal umber of clusters ad effectively reducig the iteratios of the clusterig algorithm. (2) Based o the features of compactess withi a cluster ad separatio betwee clusters, a ew fuzzy clusterig validity idex (CVI) is defied i this paper avoidig its value close to zero alog with the umber of clusters tedig to the umber of objects ad obtaiig the optimal clusterig result. (3) A self-adaptive method that iteratively uses improved FCM algorithm to estimate the optimal umber of clusters was put forward. 2. Related Work The easiest method of determiig the umber of clusters is data visualizatio. For the dataset that ca be effectively mapped to a 2-dimesioal Euclidea space, the umber of clusters ca be ituitively acquired through the distributio graph of data poits. However, for high-dimesioal ad complicated data, this method is userviceable. Rodriguez ad Laio proposed a clusterig algorithm based o desity peaks, declarig that it was able to detect ospherical clusters ad to automatically fid the true umber of clusters [4]. But, i fact, the umber of cluster cetroids still eeds to be selected artificially accordig to the decisio graph. Next, relevat techologies to determie the optimal umber of clusters are summarized below Clusterig Validity Idex Based Method. Clusterig validity idex is used to evaluate the quality of partitios o a dataset geerated by the clusterig algorithm. It is a effective method to costruct a appropriate clusterig validity idex to determie the umber of clusters. The idea is to assig differet values of the umber of clusters c withi a certai rage, the ru fuzzy clusterig algorithm o the dataset, ad fially to evaluate the results by clusterig validity idex. Whe the value of clusterig validity idex is of the maximum or the miimum or a obvious iflectio poit appears, the correspodig value of c is the optimal umber of clusters c opt. So far, researchers have put forward lots of fuzzy clusterig validity idices, divided ito the followig two types. (1) Clusterig Validity Idex Based o Fuzzy Partitio. These idices are accordig to such a poit of view that for a wellseparated dataset, the smaller the fuzziess of fuzzy partitio is,themorereliabletheclusterigresultis.basedoit,zadeh, the fouder of fuzzy sets, put forward the first clusterig validity idex degree of separatio i 1965 [5]. But its discrimiatig effect is ot ideal. I 1974, Bezdek put forward the cocept of partitio coefficiet (PC) [6] which was the first practical geeric fuctio for measurig the validity of the fuzzy clusterig ad subsequetly proposed aother clusterig validity fuctio partitio etropy (PE) [7] closely related to partitio coefficiet. Later, Widham defied proportio expoet by use of the maximum value of a fuzzy membership fuctio [8]. Lee put forward a fuzzy clusterig validity idex usig the distiguishableess of clusters measured by the object proximities [9]. Based o the Shao etropy ad fuzzy variatio theory, Zhag ad Jiag put forward a ew fuzzy clusterig validity idex takig accout of the geometry structure of the dataset [1]. Saha et al. put forward a algorithm based o differetial evolutio for automatic cluster detectio, which well evaluated the validity of the clusterig result [11]. Yue et al. partitioed the origial data space ito a grid-based structure ad proposed a cluster separatio measure based o grid distaces [12]. The clusterig validity idex based o fuzzy partitio is oly related to the fuzzy degree of membership ad has the advatages of simpleess ad small calculatig amout. But it lacks direct relatio with some structural features of the dataset. (2) Clusterig Validity Idex Based o the Geometry Structure of the Dataset. These idices are based o such a poit of view that for a well-separated dataset, every cluster should be compact ad separated from each other as far as possible. The ratio of compactess ad separatio is used as the stadard of clusterig validity. This type of represetative clusterig validity idices iclude Xie-Bei idex [13], Besaid idex [14],adKwoidex[15].Suetal.proposedaewvalidity idex based o a liear combiatio of compactess ad separatio ad ispired by Rezaee s validity [16]. Li ad Yu defied ew compactess ad separatio ad put forward a ew fuzzy clusterig validity fuctio [17]. Based o fuzzy graulatio-degraulatio, Saha ad Badyopadhyay put forward a fuzzy clusterig validity fuctio [18]. Zhag et al. adopted Pearso correlatio to measure the distace ad put forward a validity fuctio [19]. Kim et al. proposed a clusterig validity idex for GK algorithm based o the average value of the relative degrees of sharig of all possible pairs of fuzzy clusters [2]. Rezaee proposed a ew validity idex for GK algorithm to overcome the shortcomigs of Kim s idex [21]. Zhag et al. proposed a ovel WGLI to detect the optimal cluster umber, usig global optimum membership as the global property ad modularity of bipartite etwork as the local idepedet property [22]. The clusterig validity idex based o the geometric structure of the dataset cosiders both the fuzzy degree of membership ad the geometric structure, but its membership fuctio is quite complicated with large calculatig amout. Based o clusterig validity idex, the optimal umber of clusters is determied through exhaustive search. I order to icrease the efficiecy of estimatig the optimal umber of clusters c opt,thesearchrageofc opt must be set; that is, c max is the maximum umber of clusters, assiged to meet the coditio c opt c max. Most researchers used the empirical rule c max,where is the umber of data i the dataset. For this problem, theoretical aalysis ad example verificatio were coducted i [23], which idicated that it was reasoable i a sese. However, apparetly, this method has the followig disadvatages. (1) Each c must be tried i tur, which will cause a huge calculatio. (2) For each c,itcaot be guarateed that the clusterig result is the globally optimal solutio. (3) Whe oise ad outliers are existig, the reliability of clusterig validity idex is weak. (4) For some datasets like FaceImage [24], if c, theempiricalrulewill be ivalid. Researches showed that due to the diversified data types ad structures, o uiversal fuzzy clusterig validity

3 Computatioal Itelligece ad Neurosciece 3 idexcabeapplicabletoalldatasets.theresearchisad will still be carried o urgetly Heuristic Method. Some ew clusterig algorithms have bee proposed i successio recetly. The mai idea is to use some criteria to guide the clusterig process, with the umber of clusters beig adjusted. I this way, while the clusterig is completed, the appropriate umber of clusters ca be obtaied as well.for example,x-meas algorithm [25] based o the split hierarchical clusterig is represetative. Cotrary to the aforesaid process, RCA [26] determied the actual umber of clusters by a process of competitive agglomeratio. Combiig the sigle-poit iterative techology with hierarchical clusterig, a similarity-based clusterig method (SCM) [27] was proposed. A mercer kerel-based clusterig [28] estimated the umber of clusters by the eigevectors of a kerel matrix. A clusterig algorithm based o maximal θdistat subtrees [29] detected ay umber of well-separated clusters with ay shapes. I additio, Frey ad Dueck put forward a affiity propagatio clusterig algorithm (AP) [24] which geerated high-quality cluster cetroids via the message passig betwee objects to determie the optimal umber of clusters of large-scale data. Shihog et al. proposed a Gerschgori disk estimatio-based criterio to estimate the true umber of clusters [3]. José-García ad Gómez- Flores preseted a up-to-date review of all major atureispired metaheuristic algorithms used thus far for automatic clusterig, determiig the best estimate of the umber of clusters[31].allthesehavewideedthethoughtsforrelevat researches. 3. Traditioal FCM for Determiig the Optimal Number of Clusters Sice Ruspii first itroduced the theory of fuzzy sets ito cluster aalysis i 1973 [32], differet fuzzy clusterig algorithms have bee widely discussed, developed, ad applied i various areas. FCM [33] is oe of the most commoly used algorithms. FCM was first preseted by Du i Subsequetly, Bezdek itroduced weighted idex to the fuzzy degree of membership [34], further developig FCM. FCM divides the dataset X ito c fuzzy clusters. This algorithm holds that each object belogs to a certai cluster with a differet degree of membership, that is, a cluster is cosidered as a fuzzy subset o the dataset. Assume that X={x 1,x 2,...,x } is a fiite dataset, where x i = (x i1,x i2,...,x il ) is a l-dimesioal object ad x ij is the jth property of the ith object. C = {C 1,C 2,...,C c } deotes c cluster(s). V = {V 1, V 2,...,V c } represets c l- dimesioal cluster cetroid(s), where V i =(V i1, V i2,...,v il ). U=(u ik ) ( c) is a fuzzy partitio matrix, ad u ik is the degree of membership of the ith object i the kth cluster, where c k=1 u ik = 1, i = 1,...,. The objective fuctio is the quadratic sum of weighed distaces from the samples to the cluster cetroid i each cluster; that is, c J m (U, V) = u m ik d2 ik, (1) k=1 i=1 where d ik = x i V k shows the Euclidea distace betwee the ith object ad the kth cluster cetroid. m (m [1, )) is a fuzziess idex, cotrollig the fuzziess of the memberships.as the value of m becomes progressively higher, the resultig memberships become fuzzier [35]. Pal ad Bezdek advised that m should be betwee 1.5 ad 2.5, ad usually let m=2if without ay special requiremet [36]. Accordig to the clusterig criterio, appropriate fuzzy partitio matrix U ad cluster cetroid V are obtaied to miimize the objective fuctio J m.basedothelagrage multiplier method, U ad V are, respectively, calculated by the formulas below: 1 u ik =, c j=1 (d ik/d jk ) 2/(m 1) (2) V k = i=1 um ik x i. (3) i=1 um ik FCM algorithm is carried out through a iterative process of miimizig the objective fuctio J m,withtheupdateof U ad V. The specific steps are as follows. Step 1. Assigtheiitialvalueoftheumberofclustersc, fuzziess idex m, maximum iteratios I max,adthreshold ξ. Step 2. Iitialize the fuzzy partitio U () radomly accordig to the costraits of the degree of membership. Step 3. At the t-step, calculate c cluster cetroids V (t) accordig to (3). Step 4. Calculate the objective fuctio J (t) m accordig to (1). If J (t) m J(t 1) m <ξor t>i max, the stop; otherwise cotiue to Step 5. Step 5. Calculate U (t+1) accordig to (2) ad retur to Step 3. At last, each object ca be arraged ito oe cluster i accordace with the priciple of the maximum degree of membership. The advatages of FCM may be summarized as simple algorithm, quick covergece, ad easiess to be exteded. Its disadvatages lie i the selectio of the iitial cluster cetroids, the sesitivity to oise ad outliers, ad the settig of the umber of clusters, which have a great impact o the clusterig result. As the radom selectio of the iitial cluster cetroids caot esure the fact that FCM coverges to a optimal solutio, differet iitial cluster cetroids are used for multiple ruig of FCM; otherwise they are determied by usig of other fast algorithms. The traditioal method to determie the optimal umber of clusters of FCM is to set the search rage of the umber of clusters, ru FCM to geerate clusterig results of differet umber of clusters, select a appropriate clusterig validity idex to evaluate clusterig results, ad fially obtai the optimal umber of clusters accordig to the evaluatio result. The method is composed of the followig steps.

4 4 Computatioal Itelligece ad Neurosciece Step 1. Iput the search rage [c mi,c max ]; geerally, c mi =2 ad c max =. Step 2. For each iteger k [c mi,c max ]. Step 2.1. Ru FCM. Step 2.2. Calculate clusterig validity idex. Step 3. Compare all values of clusterig validity idex. k correspodig to the maximum or miimum value is the optimal umber of clusters c opt. Step 4. Output c opt,theoptimalvalueofclusterigvalidity idex, ad clusterig result. 4. The Proposed Methods 4.1. Desity-Based Algorithm. Cosiderig the large ifluece of the radomly selected iitial cluster cetroid o the clusterig result, a desity-based algorithm is proposed to select the iitial cluster cetroids, ad the maximum umber of clusters ca be estimated at the same time. Some related terms will be defied at first. Defiitio 1 (local desity). The local desity ρ i of object x i is defied as ρ i = j=1 e d2 ij /dc2, (4) where d ij is the distace betwee two objects x i ad x j ad dc is a cutoff distace. The recommeded approach is to sort the distace betwee ay two objects i descedig order ad the assig dc as the value correspodig to the first p% of thesorteddistace(appropriatelyp [2, 5]). It shows that the more objects with a distace from x i less tha dc there are,thebiggerthevalueofρ i is. The cutoff distace fially ca decide the umber of iitial cluster cetroids, amely, the maximum umber of clusters. The desity-based algorithm is robust with respect to the choice of dc. Defiitio 2 (core poit). Assume that there is a object x i X,ifx i C k ad ρ i = max xj C k (ρ j ), k=1,...,c, j=1,...,, the x i is a core poit. Defiitio 3 (directly desity-reachable). Assume that there are two objects x i, x j X,iftheirdistaced ij <dc,thex j is directly desity-reachable to x i ad vice versa. Defiitio 4 (desity-reachable). Assume that X is a dataset ad objects x s,x s+1,...,x t 1,x t belog to it. If, for each i [s, t 1], x i+1 is directly desity-reachable to x i,thex s is desity-reachable to x t. Defiitio 5 (eighbor). Neighbors of a object x i are those who are directly desity-reachable or desity-reachable to it, deoted as Neighbor (x i ). E dc C A D B F Figure 1: A example. Poit A is of the highest local desity. If A does ot belog to ay cluster, the A is a core poit. Poits B, C, D, ad E are directly desity-reachable to poit A. Poit F is desityreachable to poit A. Poit H is a border poit. Defiitio 6 (border poit). A object is called a border poit if it has o eighbors. The example is show i Figure 1. The selectio priciple of iitial cluster cetroids of desity-based algorithm is that, usually, a cluster cetroid is a object with higher local desity, surrouded by eighbors with lower local desity tha it, ad has a relatively large distace from other cluster cetroids [4]. Desity-based algorithm ca automatically select the iitial cluster cetroids ad determie the maximum umber of clusters c max accordig to local desity. The pseudocode is show i Algorithm 1. These cluster cetroids obtaied are sorted i descedig order accordig to local desity. Figure 2 demostrates the process of desity-based algorithm Fuzzy Clusterig Validity Idex. Firstly based o the geometric structure of the dataset, Xie ad Bei put forward the Xie-Bei fuzzy clusterig validity idex [13], which tried tofidabalacepoitbetweethefuzzycompactessad separatio so as to acquire the optimal cluster result. The idex is defied as dc V xie (U, V) = (1/) c i=1 j=1 um ij H dc mi i=j V i V j V 2 i x j 2. (5) I the formula, the umerator is the average distace from various objects to cetroids, used to measure the compactess, ad the deomiator is the miimum distace betwee ay two cetroids, measurig the separatio. However, Besaid et al. foud that the size of each cluster hadalargeiflueceoxie-beiidexadputforward a ew idex [14], which was isesitive to the umber of objects i each cluster. Besaid idex is defied as V B (U, V) = c k=1 i=1 um ik x i V k 2 k c j=1 V j V k 2, (6)

5 Computatioal Itelligece ad Neurosciece 5 Iput:Datasetadp Output: V ad c max D=(d ij ) ; dc = cutoffdis (p); / Calculate the cutoff distace dc. / ρ=(ρ i ) ; idx = arg(sort(ρ, descet)); / The umber of object correspodig to sorted ρ. / k=1; cl = ( 1) 1 ; / The cluster umber that each object belogs to, with iitial value 1. / for i=1to 1do if cl idxi!= 1&&#{Neighbor(x idxi )>1}the / x idxi dose ot belog to ay cluster ad the mber of its eighbors is greater tha 1. / V k =x idxi ; for j Neighbor(x idxi ) do cl j =k; k=k+1; c max =k; Algorithm 1: Desity-based algorithm. where k is the fuzzy cardiality of the kth cluster ad defied as k = i=1 u ik. (7) i=1 um ik x i V k 2 shows the variatio of the kth fuzzy cluster. The, the compactess is computed as 1 u m ik k i=1 x i V k 2. (8) c j=1 V j V k 2 deotes the separatio of the kth fuzzy cluster, defied as the sum of the distaces from its cluster cetroid to the cetroids of other c 1clusters. Because lim c x i V k 2 =, (9) this idex is the same as the Xie-Bei idex. Whe c,the idex value will be mootoically decreased, close to, ad will lose robustess ad judgmet fuctio for determiig the optimal umber of clusters. Thus, this paper improves Besaid idex ad proposes a ew idex V R : V R (U, V) = c k=1 (1/ k ) i=1 um ik x i V k 2 + (1/c) V k V 2 (1/ (c 1)) c j=1 V j V k 2. (1) The umerator represets the compactess of the kth cluster, where k is its fuzzy cardiality. Its secod item, a itroduced puishig fuctio, deotes the distace from the cluster cetroid of the kthclustertotheaverageofall cluster cetroids, which ca elimiate the mootoically decreasig tedecy as the umber of clusters icreases to. The deomiator represets the mea distace from the kth cluster cetroid to other cluster cetroids, which is used for measurig the separatio. The ratio of the umerator ad the deomiator thereof represets the clusterig effect of the kth cluster. The clusterig validity idex is defied as the sum of the clusterig effect (the ratio) of all clusters. Obviously, the smaller the value is, the better the clusterig effect of the dataset is, ad the correspodig c to the miimum value is the optimal umber of clusters Self-Adaptive FCM. I this paper, the iterative trial-aderror process [16] is still used to determie the optimal umber of clusters by self-adaptive FCM (SAFCM). The pseudocode of SAFCM algorithm is described i Algorithm Experimet ad Aalysis This paper selects 8 experimetal datasets, amog which 3 datasets come from UCI datasets, respectively, Iris, Wie, ad Seeds, a dataset (SubKDD) is radomly selected from KDD Cup 1999 show i Table 1, ad the remaiig 4 datasets aresytheticdatasetsshowifigure3.subkddicludes ormal, 2 kids of Probe attack (ipsweep ad portsweep) ad 3 kids of DoS attack (eptue, smurf, ad back). The first sythetic dataset (SD1) cosists of 2 2-dimesioal Gaussia distributio data with 1 samples. Their covariace matrixes are secod-order uit matrix I 2. The structural feature of the dataset is that the distace betwee ay two clusters is large, ad the umber of classes is greater tha. Thesecod sythetic dataset (SD2) cosists of 4 2-dimesioal Gaussia distributio data. The cluster cetroids are, respectively, (5,5), (1,1), (15,15), ad (2,2), each cotaiig 5 samples ad thecovariacematrixofeachbeig2i 2. The structural feature

6 6 Computatioal Itelligece ad Neurosciece (a) Iitial data (c) Iteratio (b) Iteratio (d) Iteratio 3 Figure 2: Demostratio of the process of desity-based algorithm. (a) is the iitial data distributio of a sythetic dataset. The dataset cosists of two pieces of 2-dimesioal Gaussia distributio data with cetroids, respectively, as (2, 3) ad (7, 8). Each class has 1 samples. I (b), the blue circle represets the highest desity core poit as the cetroid of the first cluster, ad the red plus sig represets the object belogig to the first cluster. I (c), the red circle represets the core poit as the cetroid of the secod cluster, ad the blue asterisk represets the object belogig to the secod cluster. I (d), the purple circle represets the core poit as the cetroid of the third cluster, the gree times sig represets the object belogig to the third cluster, ad the black dot represets the fial border poit which does ot belog to ay cluster. Accordig to a certai cutoff distace, the maximum umber of clusters is 3. If calculated i accordace with the empirical rule, the maximum umber of clusters should be 14. Therefore, the algorithm ca effectively reduce the iteratio of FCM algorithm operatio. Table 1: The data type ad distributio of SubKDD. Attack behavior Number of samples ormal 2 ipsweep 5 portsweep 5 eptue 2 smurf 3 back 5 of the dataset is of short itercluster distace with a small overlap. The third sythetic dataset (SD3) has a complicated structure. The fourth sythetic dataset (SD4) is a ocovex oe. I this paper, the umeric value x ij is ormalized as x ij = x ij x j s j, (11) where x j ad s j deote the mea of the jth attribute value ad its stadard deviatio, respectively; the x j = 1 i=1 x ij, s j = 1 (x 1 ij x j ) 2. i=1 (12)

7 Computatioal Itelligece ad Neurosciece 7 Iput: V, c max, I max ad ξ Output: Jbest, c opt, Ubest for k = 2 to c max do t=; / The curret iteratio. / V (t) k =(V i) k ; / The first k cluster cetroids of V. / U (t) k =(u(t) ik ) k; J (t) k =V R(U (t) k,v(t) k ); while t== t<imax&& J (t) k J(t 1) k >ξdo Update V (t) k ; Update U (t) k ; Update J (t) k ; t=t+1; Jmi k = mi t i=1 (J(i) k ); idx = arg(mi t i=1 (J(i) k )); Umi k =U (idx) k ; Jbest = mi cmax k=2 (Jmi k); / The best value of clusterig validity idex. / c opt = arg(mi cmax k=2 (Jmi k)); Ubest = Umi copt ; Algorithm 2: SAFCM. Table 2: c max estimated by several methods. is the umber of objects i the dataset, c is the actual umber of clusters, c ER is the umber of clusters estimated by the empirical rule, that is,, c AP is the umber of clusters obtaied by AP algorithm, ad c DBA is the umber of clusters obtaied by desity-based algorithm. c DBA Dataset c c ER c AP p=1 p=2 p=3 p=5 Iris Wie Seeds SubKDD SD SD SD SD For the categorical attribute of SubKDD, the simple matchig is used for the dissimilarity measure, that is, for idetical values ad 1 for differet values Experimet of c max Simulatio. I [37], it was proposed that the umber of clusters geerated by AP algorithm could be selected as the maximum umber of clusters. So this paper estimates the value of c max by the empirical rule, AP algorithm, ad desity-based algorithm. The specific experimetal results are show i Table 2. Let p=p m i AP algorithm. dc is, respectively, selected as the distace of the first1%,2%,3%,ad5%. Experimet results show that, for the dataset with the true umber of clusters greater tha, itisobviously icorrect to let c max =. I other words, the empirical rule is ivalid. The umber of clusters fially obtaied by AP algorithm is close to the actual umber of clusters. For the dataset with the true umber of clusters smaller tha, the umber of clusters geerated by AP algorithm ca be used as a appropriate c max, but, sometimes, the value is greater tha the value estimated by the empirical rule, which elarges the search rage of the optimal umber of clusters. If c max is estimated by the proposed desity-based algorithm, the results i most cases are appropriate. The method is ivalidolyosd1whethecutoffdistaceisthefirst5% distace. Whe the cutoff distace is selected as the first 3% distace, c max geerated by the proposed algorithm is much smaller tha. Itisclosertothetrueumberofclusters ad greatly arrows the search rage of the optimal umber of clusters. Therefore, the cutoff distace is selected as the distace of the first 3% i the later experimets Experimet of Ifluece of the Iitial Cluster Cetroids o the Covergece of FCM. To show that the iitial cluster cetroids obtaied by desity-based algorithm ca quicke the covergece of clusterig algorithm, the traditioal FCM is adopted for verificatio. The umber of clusters is assiged asthetruevalue,withthecovergecethresholdbeig1 5. Because the radom selectio of iitial cluster cetroids has a large ifluece o FCM, the experimet of each dataset

8 8 Computatioal Itelligece ad Neurosciece (a) SD (c) SD (b) SD (d) SD4 Figure 3: Four sythetic datasets. Table3:ComparisoofiteratiosofFCMalgorithm.Method1uses the radom iitial cluster cetroids, ad Method 2 uses the cluster cetroids obtaied by desity-based algorithm. Dataset Method 1 Method 2 Iris Wie Seeds SubKDD SD SD SD SD is doe repeatedly for 5 times, ad the roud umbers of the mea of algorithmic iteratios are compared. The specific experimetal result is show i Table 3. As show i Table 3, the iitial cluster cetroids obtaied by desity-based algorithm ca effectively reduce the iteratio of FCM. Particularly, o SD1, the iteratio of FCM is far smaller tha that with radomly selected iitial cluster cetroids whose miimum iteratio is 24 ad maximum iteratio reaches 63, with ustable clusterig results. Therefore, theproposedmethodcaotolyeffectivelyquickethe covergece of FCM, but also obtai a stable clusterig result Experimet of Clusterig Accuracy Based o Clusterig Validity Idex. For 3 UCI datasets ad SubKDD, clusterig accuracy r is adopted to measure the clusterig effect, defied as r= k i=1 b i. (13) Here b i is the umber of objects which co-occur i the ith cluster ad the ith real cluster, ad is the umber of objects i the dataset. Accordig to this measuremet, the higher the clusterig accuracy is, the better the clusterig result of FCM is. Whe r=1,theclusterigresultoffcmistotally accurate. I the experimets, the true umber of clusters is assiged to each dataset ad the iitial cluster cetroids are obtaied by desity-based algorithm. The, experimetal results of clusterig accuracy are show i Table 4.

9 Computatioal Itelligece ad Neurosciece (a) SD (c) SD (b) SD (d) SD4 Figure 4: Clusterig results of two sythetic datasets. Table 4: Clusterig accuracy. Dataset Iris Wie Seeds SubKDD Clusterig accuracy 84.% 96.63% 91.9% 94.35% Apparetly, clusterig accuracies of the datasets Wie, Seeds, ad SubKDD are high, while that of the dataset Iris is relatively low for the reaso that two clusters are oliearly separable. The clusterig results of the proposed clusterig validity idex o 4 sythetic datasets are show i Figure 4, which depict the good partitios o the datasets SD1 ad SD2 ad the slightly poor partitio o SD3 ad SD4 because of the complexity of their structure or ocovexity. Whe the umber of clusters is set as 4, SD3 is divided ito 4 groups that meas the right ad the left each have 2 groups Experimet of the Optimal Number of Clusters. At last, Xie-Bei idex, Besaid idex, Kwo idex, ad the proposed idex are, respectively, adopted for ruig of SAFCM so as to determie the optimal umber of clusters. The results Table 5: Optimal umber of clusters estimated by several clusterig validity idices. Dataset V XB V B V K V R Iris Wie Seeds SubKDD SD SD SD SD are show i Table 5. V XB, V B, V K,adV R represet Xie-Bei idex, Besaid idex, Kwo idex, ad the proposed idex, respectively. It shows that, for sythetic datasets SD1 ad SD2 with simple structure, these idices ca all obtai the optimal umber of clusters. For 3 UCI datasets, SubKDD ad SD4, Besaid idex caot obtai a accurate umber of clusters except SD3. For the dataset Wie, both Xie-Bei idex ad

10 1 Computatioal Itelligece ad Neurosciece Table 6: The value of clusterig validity idex o Iris Table 7: The value of clusterig validity idex o Wie Table 8: The value of clusterig validity idex o Seeds Kwo idex ca obtai the accurate umber of clusters, while for the datasets Iris, Seeds, SubKDD, SD3, ad SD4, they oly obtai the result approximate to the true umber of cluster. The proposed idex obtais the right result o the datasets Wie ad SD4 ad a better result compared to Xie-Bei idex ad Kwo idex o the datasets SubKDD ad SD3, while it obtais the same result of the two idices o the datasets Iris ad Seeds. There are two reasos. First, the importace degree of each property of the dataset is ot cosidered i the clusterig process but assumed to be the same, thereby affectig the experimetal result. Secod, SAFCM has a weak capacity to deal with overlappig clusters or ocovexity dataset. These are the authors subsequet research cotets. Tables 6 13 list the detailed results of 4 idices o the experimetal datasets i each iteratio of SAFCM. For the dataset Wie, whe the umber of clusters is 4, 5, or 6, the result based o Xie-Bei idex or Kwo idex is of o covergece. However, the proposed idex ca achieve the desired results. 6. Coclusios FCMiswidelyusedilotsoffields.Butiteedstopreset the umber of clusters ad is greatly iflueced by the iitial cluster cetroids. This paper studies a self-adaptive method for determiig the umber of clusters by usig of FCM algorithm. I this method, a desity-based algorithm is put Table 9: The value of clusterig validity idex o SubKDD Table 1: The value of clusterig validity idex o SD Table 11: The value of clusterig validity idex o SD Table 12: The value of clusterig validity idex o SD forward at first, which ca estimate the maximum umber of clusters to reduce the search rage of the optimal umber, especially beig fit for the dataset o which the empirical rule is ioperative. Besides, it ca geerate the high-quality

11 Computatioal Itelligece ad Neurosciece 11 Table 13: The value of clusterig validity idex o SD iitial cluster cetroids so that the the clusterig result is stable ad the covergece of FCM is quick. The, a ew fuzzy clusterig validity idex was put forward based o fuzzy compactess ad separatio so that the clusterig result is closer to global optimum. The idex is robust ad iterpretable whe the umber of clusters teds to that of objects i the dataset. Fially, a self-adaptive FCM algorithm is proposed to determie the optimal umber of clusters ru i the iterative trial-ad-error process. The cotributios are validated by experimetal results. However, i most cases, each property plays a differet role i the clusterig process. I other words, the weight of properties is ot the same. This issue will be focused o i the authors future work. Competig Iterests The authors declare that they have o competig iterests. Ackowledgmets This work is partially supported by the Natioal Natural Sciece Foudatio of Chia ( ad ), Shadog Provice Natural Sciece Foudatio (ZR214FL1), Sciece Foudatio of Miistry of Educatio of Chia (14YJC 8642), Shadog Provice Outstadig Youg Scietist AwardFud(BS213DX33),ShadogProviceSocialScieces Project (15CXWJ13 ad 16CFXJ5), ad Project of Shadog Provice Higher Educatioal Sciece ad Techology Program (o. J15LN2). Refereces [1] S. Sambasivam ad N. Theodosopoulos, Advaced data clusterig methods of miig web documets, Issues i Iformig Sciece ad Iformatio Techology,vol.3,pp ,26. [2] J. Wag, S.-T. Wag, ad Z.-H. Deg, Survey o challeges i clusterig aalysis research, Cotrol ad Decisio,vol.3,212. [3] C.-W. Bog ad M. Rajeswari, Multi-objective ature-ispired clusterig ad classificatio techiques for image segmetatio, Applied Soft Computig Joural, vol.11,o.4,pp , 211. [4] A.RodriguezadA.Laio, Clusterigbyfastsearchadfidof desity peaks, Sciece, vol. 344, o. 6191, pp , 214. [5] L.A.Zadeh, Fuzzysets, Iformatio ad Computatio,vol.8, o. 3, pp , [6] J. C. Bezdek, Numerical taxoomy with fuzzy sets, Joural of Mathematical Biology,vol.1,o.1,pp.57 71,1974. [7] J. C. Bezdek, Cluster validity with fuzzy sets, Joural of Cyberetics,vol.3,o.3,pp.58 73,1974. [8] M. P. Widham, Cluster validity for fuzzy clusterig algorithms, Fuzzy Sets ad Systems,vol.5,o.2,pp ,1981. [9] M. Lee, Fuzzy cluster validity idex based o object proximities defied over fuzzy partitio matrices, i Proceedigs of the IEEE Iteratioal Coferece o Fuzzy Systems ad IEEE World Cogress o Computatioal Itelligece (FUZZ-IEEE 8), pp , Jue 28. [1] X.-B. Zhag ad L. Jiag, A ew validity idex of fuzzy c-meas clusterig, i Proceedigs of the IEEE Iteratioal Coferece o Itelliget Huma-Machie Systems ad Cyberetics (IHMSC 9),vol.2,pp ,Hagzhou,Chia,August29. [11] I. Saha, U. Maulik, ad S. Badyopadhyay, A ew differetial evolutio based fuzzy clusterig for automatic cluster evolutio, i Proceedigs of the IEEE Iteratioal Advace Computig Coferece (IACC 9), pp , IEEE, Patiala, Idia, March 29. [12] S. Yue, J.-S. Wag, T. Wu, ad H. Wag, A ew separatio measure for improvig the effectiveess of validity idices, Iformatio Scieces,vol.18,o.5,pp ,21. [13] X. L. Xie ad G. Bei, A validity measure for fuzzy clusterig, IEEE Trasactios o Patter Aalysis ad Machie Itelligece, vol. 13, o. 8, pp , [14] A. M. Besaid, L. O. Hall, J. C. Bezdek et al., Validity-guided (re)clusterig with applicatios to image segmetatio, IEEE Trasactios o Fuzzy Systems,vol.4,o.2,pp ,1996. [15] S. H. Kwo, Cluster validity idex for fuzzy clusterig, Electroics Letters, vol. 34, o. 22, pp , [16] H. Su, S. Wag, ad Q. Jiag, FCM-based model selectio algorithms for determiig the umber of clusters, Patter Recogitio,vol.37,o.1,pp ,24. [17] Y. Li ad F. Yu, A ew validity fuctio for fuzzy clusterig, i Proceedigs of the Iteratioal Coferece o Computatioal Itelligece ad Natural Computig (CINC 9),vol.1,pp , IEEE, Jue 29. [18] S. Saha ad S. Badyopadhyay, A ew cluster validity idex based o fuzzy graulatio-degraulatio criteria, i Proceedigs of the IEEEIteratioal Coferece o Advaced Computig ad Commuicatios (ADCOM 7), pp , IEEE, Guwahati, Idia, 27. [19] M. Zhag, W. Zhag, H. Sicotte, ad P. Yag, A ew validity measure for a correlatio-based fuzzy c-meas clusterig algorithm, i Proceedigs of the 31st IEEE Aual Iteratioal Coferece of the IEEE Egieerig i Medicie ad Biology Society (EMBC 9), pp , Mieapolis, Mi, USA, September 29. [2] Y.-I.Kim,D.-W.Kim,D.Lee,adK.H.Lee, Aclustervalidatio idex for GK cluster aalysis based o relative degree of sharig, Iformatio Scieces, vol. 168, o. 1 4, pp , 24. [21] B. Rezaee, A cluster validity idex for fuzzy clusterig, Fuzzy Sets ad Systems,vol.161,o.23,pp ,21. [22] D. Zhag, M. Ji, J. Yag, Y. Zhag, ad F. Xie, A ovel cluster validity idex for fuzzy clusterig based o bipartite modularity, Fuzzy Sets ad Systems,vol.253,pp ,214. [23] J. Yu ad Q. Cheg, Search rage of the optimal cluster umber i fuzzy clusterig, Sciece i Chia (Series E), vol. 32, o. 2, pp , 22.

12 12 Computatioal Itelligece ad Neurosciece [24] B. J. Frey ad D. Dueck, Clusterig by passig messages betwee data poits, Sciece, vol.315,o.5814,pp , 27. [25] D. Pelleg, A. W. Moore et al., X-meas: extedig K-meas with efficiet estimatio of the umber of clusters, i Proceedigs of the 17th Iteratioal Coferece o Machie Learig (ICML ),vol.1,pp ,2. [26] H. Frigui ad R. Krishapuram, A robust competitive clusterig algorithm with applicatios i computer visio, IEEE Trasactios o Patter Aalysis ad Machie Itelligece, vol. 21,o.5,pp ,1999. [27] M.-S. Yag ad K.-L. Wu, A similarity-based robust clusterig method, IEEE Trasactios o Patter Aalysis ad Machie Itelligece, vol. 26, o. 4, pp , 24. [28] M. Girolami, Mercer kerel-based clusterig i feature space, IEEE Trasactios o Neural Networks, vol.13,o.3,pp , 22. [29] L. Yujia, A clusterig algorithm based o maximal θ-distat subtrees, Patter Recogitio, vol. 4, o. 5, pp , 27. [3] Y. Shihog, H. Ti, ad W. Peglog, Matrix eigevalue aalysis-based clusterig validity idex, Joural of Tiaji Uiversity (Sciece ad Techology),vol.8,article6,214. [31] A. José-García ad W. Gómez-Flores, Automatic clusterig usig ature-ispired metaheuristics: a survey, Applied Soft Computig Joural,vol.41,pp ,216. [32] E. H. Ruspii, New experimetal results i fuzzy clusterig, Iformatio Scieces, vol. 6, pp , [33] J. C. Du, Well-separated clusters ad optimal fuzzy partitios, Joural of Cyberetics,vol.4,o.1,pp.95 14,1974. [34] J. C. Bezdek, Patter Recogitio with Fuzzy Objective Fuctio Algorithms, Spriger Sciece & Busiess Media, 213. [35] R. E. Hammah ad J. H. Curra, Fuzzy cluster algorithm for the automatic idetificatio of joit sets, Iteratioal Joural of Rock Mechaics ad Miig Scieces, vol.35,o.7,pp , [36] N. R. Pal ad J. C. Bezdek, O cluster validity for the fuzzy c- meas model, IEEE Trasactios o Fuzzy Systems, vol. 3, o. 3, pp , [37] S. B. Zhou, Z. Y. Xu, ad X. Q. Tag, Method for determiig optimal umber of clusters based o affiity propagatio clusterig, Cotrol ad Decisio, vol. 26, o. 8, pp , 211.

13 Joural of Advaces i Idustrial Egieerig Multimedia The Scietific World Joural Applied Computatioal Itelligece ad Soft Computig Iteratioal Joural of Distributed Sesor Networks Advaces i Fuzzy Systems Modellig & Simulatio i Egieerig Submit your mauscripts at Joural of Computer Networks ad Commuicatios Advaces i Artificial Itelligece Iteratioal Joural of Biomedical Imagig Advaces i Artificial Neural Systems Iteratioal Joural of Computer Egieerig Computer Games Techology Advaces i Advaces i Software Egieerig Iteratioal Joural of Recofigurable Computig Robotics Computatioal Itelligece ad Neurosciece Advaces i Huma-Computer Iteractio Joural of Joural of Electrical ad Computer Egieerig