An adaptive multi-objective genetic algorithm with fuzzy c-means for automatic data clustering

Transcription

1 An adaptve ult-objetve genet algorth wth fuzzy -eans for autoat data lusterng Ze Dong, Hao Ja, and Mao Lu Departent of Autoaton, North Chna Eletr Power Unversty, Baodng 0700, Chna, Hebe Engneerng Researh Center of Sulaton & Optzed Control for Power Generaton Correspondene should be addressed to Hao Ja; Abstrat: Ths paper presents a fuzzy lusterng ethod based on ult-objetve genet algorth. The ADNSGA- algorth was developed to solve the lusterng proble by obnng the fuzzy lusterng algorth () wth the ult-objetve genet algorth (NSGA-II) and ntrodung an adaptve ehans. The algorth does not need to gve the nuber of lusters n advane. After the nuber of ntal lusters and the enter oordnates are gven randoly, the optal soluton set s found by the ult-objetve evolutonary algorth. After deternng the optal nuber of lusters by ajorty vote ethod, the J value s ontnuously optzed through the obnaton of Canonal Genet Algorth and, and fnally the best lusterng result s obtaned. By usng standard UCI dataset verfaton and oparng wth exstng sngle-objetve and ult-objetve lusterng algorths, the effetveness of ths ethod s proved. Key words: Fuzzy lusterng, Mult-objetve genet algorth (NSGA-II), Adaptve ethod, Fuzzy -eans. Introduton Clusterng s a oon unsupervsed learnng ethod n the feld of ahne learnng. It has been wdely used n any felds, suh as data nng, pattern reognton, nforaton retreval and other felds. In these felds, ost of the data s unlabeled data wth ultple attrbutes. In these hgh-densonal spae, t s very dffult for us to get the requred nforaton through the graph. Therefore, the ultate goal of lusterng s to aheve unsupervsed lassfaton of oplex data where there s lttle or no tagged on those data. Fuzzy C-eans algorth () s a wdely used lusterng algorth n the feld of ahne learnng. It was proposed by Bezdek et al n 98 []. It s dfferent fro the tradtonal hard lusterng ethod s that nstead of the tradtonal hard lusterng ethod of non-a or B lusterng. By ntrodung the fuzzy ebershp atrx, the fuzzy C-eans algorth allows data ponts to belong to ultple lasses aordng to ther fuzzy ebershp degree. We hoose the lass wth the hghest value of the urrent data pont n the fuzzy ebershp atrx as the fnal lusterng result. Ths ethod solves the proble of lusterng overlap of tradtonal hard lusterng algorths (suh as k-eans ethod []), and ore n lne wth the atual stuaton n data lusterng. However, has two serous shortongs. Frstly, t s easly fall nto loal na. Seondly, t s neessary to spefy the nuber of lusters and the algorth s very senstve to the ntal enter [-]. To overoe the frst shortong, soe global optzaton tehnques have been ntrodued to deal wth data lusterng probles n the past years, for exaple, sulated annealng (SA)-based [5], partle swar optzaton (PSO)-based [6 8], genet algorths (GA)-based [9 ], quantu genet algorths (QGA)-based []. In reent years, genet algorths have been used to autoatally deterne the nuber of lusters by usng varable-length strngs [-]. In [5], obned wth genet algorth, whh has been suessfully appled to reote sensng agery n [6]. Most optzaton-based lusterng algorths are sngle-objetve optzaton algorths, beause only one valdty easure of effetveness s optzed. Noted that a sngle valdty easure an only reflet soe of the nherent segentaton attrbutes. For exaple, the opatness of lusters, the spatal separaton between the lusters, and the luster s syetry. If there are several lasses of geoetr shapes are present n the sae dataset, the lusterng algorths that use a sngle lusterng valdty ndex wll not be able to proess suh datasets. Therefore, t s neessary to optze several lusterng valdty ndexes that an apture dfferent data features at the sae te. Based on ths onsderaton, data lusterng should be onsdered as a ult-objetve optzaton proble. In reent years, ult-objetve optzaton probles have reeved extensve attenton. Many

2 sholars have onduted extensve researhes on ult-objetve evolutonary algorths and have aheved extensve applatons n feature extraton, data lassfaton, and lusterng. In [7], a novel nteratve evolutonary algorth (IEA) for ult-objetve optzaton probles (MOPs) wth nterval paraeters was then proposed based on the theory of preferene polyhedron that nterats wth a deson aker (DM) durng the optzaton proess to obtan the ost preferred soluton. In [8], an EA norporatng wth a deson-aker (DM)'s preferenes was presented to obtan a Pareto-optal subset that eets the DM's preferenes. In [9], a set-based genet algorth s proposed to solve the nterval any-objetve optzaton probles nvolvng ore than three objetves and at least one subjeted to nterval unertanty. In [0], a novel any-objetve evolutonary algorth usng a one-by-one seleton strategy s proposed. Ths ethod effetvely balanes the onvergene and dversty n the hgh-densonal objetve spae, effetvely solve the dffultes enountered by ult-objetve evolutonary algorths n solvng ult-objetve optzaton probles. In [], a new ethod of lassfaton feature extraton s proposed. A probablty-based enodng tehnology and an effetve hybrd operator, together wth the deas of the rowdng dstane, the external arhve, and the Pareto donaton relatonshp, are appled to PSO. By usng ths way to prove the searh apablty of the algorth. The experental oparson proves the effetveness of the algorth. In [], a new ult-label feature seleton algorth s proposed to use an proved ult-objetve partle swar optzaton (PSO), wth the purpose of searhng for a Pareto set of non-donated solutons (feature subsets). Proposed two new operators to prove the perforane of the proposed PSO-based algorth. Fnally, the effetveness of the algorth s verfed by experents. Mult-objetve evolutonary algorths (MOEAs) have proven to provde prosng solutons to the proble of sngle-objetve lusterng algorths that provde effent searh perforane []. In [], a ult-objetve lusterng tehnque, MOCK, s proposed to reognze the approprate parttonng fro the data sets that ontan ether hyperspheral shaped lusters or well-separated lusters. In [5], a ult-objetve lusterng tehnque s proposed, alled VAMOSA. The algorth optzes two lusterng valdty ndes sultaneously, so that the algorth an evolve proper parttonng fro the lusterng data set wth any shape, sze or onvexty. In [6], a fuzzy lusterng algorth naed MOoDEFC based on proved ult-objetve dfferental evoluton was proposed. By usng XB Index [7] and easure(j) as objetve funtons, the algorth an optze both the opatness and separaton of lusters sultaneously. It also proved the lusterng effet. Based on the above onsderaton, we developed a fuzzy lusterng algorth by usng the ult-objetve optzaton fraework, obned the knowledge of and general genet algorth. The algorth s aed to aheve the funtons as follows: () autoatally deterne the nuber of lusters; () prove lusterng perforane. The rest of ths paper s arranged as follows. Seton ntrodues the related theores, nludng the algorth and the ult-objetve genet algorth NSGA-II. Seton ntrodues the proved ult-objetve optzaton fraework and the adaptve ult-objetve dyna fuzzy lusterng algorth ADNSGA-. In Seton, experents results were arred out by usng soe standard UCI datasets. The experental results were opared wth any lusterng algorths n detal. Fnally, onlusons are drawn n Seton 5.. Theoretal bass. Fuzzy C-eans algorth Suppose that fuzzy -eans () parttons a set of n data objets X { x, x, x n } n p nto ( n) fuzzy lusters, where eah objet has p attrbutes. Let V { v, v, v } p be a set of luster enters. Let U [ uk ] n be a n atrx of ebershp degrees n whh u k s the ebershp degree of kth objet to the th luster enter. The atrx satsfes the ondtons: uk, uk [0,] () uk 0 The algorth uses the objetve funton to solve the optal lusterng, whh s a lear dfferene fro the hard lusterng algorth. The objetve funton J of an be defned as follows: n k k k J u x v () In Eq.(), s the fuzzfaton oeffent, represents the fuzzy degree of lusterng. Defne. x v eans the Euldean dstane between the kth data k ponts and the frst th luster enters, whh represents the n-lass slarty. A good lusterng algorth should ensure that the dstane between slar ponts n the lusterng result are as opat as possble. The standard uses the J as a ost funton to be nzed. The nzaton of J an be aheved by Lagrange ultpler ethod under onstrant u, k,, n, whle the ebershp degrees atrx k U and luster enters are updated aordng to Eq.() and Eq.(). xk v uk ( ) () x v t k t

3 v n ( u x ) k k k n uk k () By teraton, the algorth ends when ondton J ( t) J ( t ) s satsfed, where s a sall postve nuber representng the end of teraton threshold.. Mult-objetve Optzaton Based on Genet Algorth Unlke sngle-objetve optzaton algorths, the ult-objetve optzaton algorth optzes ultple objetve funtons sultaneously. Beause t s neessary to optze ultple onfltng objetves sultaneously, t s often dffult to fnd a soluton to ake all the objetve funtons reah the optu sultaneously. For ult-objetve optzaton algorths, eah objetve funton s onsdered equally portant when the relatve portane of the goals s unknown. Therefore, the ult-objetve optzaton proble s not to optze one soluton, but to optze one soluton set, whh s haraterzed by provng any objetve funton wthout parng other objetve funtons. We all ths soluton a nondonated solutons or a Pareto optal solutons, whh s defned as follows [8]: For nzng the ult-objetve proble, a vetor of n target oponents f (,, n) : f ( X ) ( f ( X), f ( X), f ( X)) (5) Where Xu U s the deson varable. If X u s the Pareto optal soluton, t needs to be satsfed: only f X v U, there s no deson varable v f ( X v) ( v, v,, v ), donatng n u f ( X u) ( u, u,, un). There are dfferent approahes to solvng ult-objetve optzaton probles [8-9], e.g., aggregatng, populaton based non-pareto, and Pareto-based tehnques. Vetor evaluated genet algorth (VEGA) s a tehnque n the populaton-based non-pareto approah n whh dfferent subpopulatons are used for the dfferent objetves. Multple objetve GA (MOGA), non-donated sortng GA (NSGA), and nhed Pareto GA (NPGA) onsttute a nuber of tehnques under the Pareto-based non-eltst approahes [9].NSGA-II [0], SPEA [], and SPEA [] are soe reently developed ult-objetve eltst tehnques. As a ult-objetve genet algorth, NSGA-II algorth s a ature ult-objetve elte seleton algorth. Copared wth the NSGA, the NSGA-II has been proved n three aspets: () When onstrutng the Pareto optal soluton set, the te oplexty of the algorth s redued fro O( N ) to O( N ) by adoptng a new rank-based fast non-donated sortng ethod. () The eltst reservaton ehans s proposed. n After seleton, offsprng fro breedng ndvduals opete wth ther parents to produe the next generaton. The new optal ndvdual reservaton ehans an not only prove the perforane of ult-objetve evolutonary algorth (MOEA) but also effetvely prevent the loss of the optal soluton and prove the overall evolutonary level of the populaton. () In order to albrate the ftness values of dfferent eleents at the sae level after rapd non-donated sortng and to ake the ndvduals n the Pareto fronter extend to the front of the entre fronter Pareto, the rowded dstane oparson operator s used nstead of the orgnal ftness sharng ethod. The present paper uses NSGA-II as the underlyng ult-objetve algorth for developng the proposed fuzzy lusterng ethod.. Dyna Fuzzy Clusterng Method Based on Adaptve NSGA-II. Chroosoe representaton In general, there are two knds of hroosoe odng shees to solve the lusterng proble by usng genet algorth:() nueral odng based on the lusterng enter ;() enodng based on the partton atrx U []. Sne the genet operator n ths paper uses the varable hroosoe length operaton, the frst hroosoe odng shee s adopted. Defnton Q denotes a hroosoe that represents ( n ) luster enters wth d densonal attrbute spae. The odng for an be expressed as: Q [,,,,,, g g gd g g gd g g gd ] Fg. shows an exaple of a hroosoe oprsng fve enters {C, C, C, C, C5} n two densons. hroosoe C C C C C C C C C5 C5 C C C C C5 Cluster enter {C,C,C,C,C5} n -densonal features Fg. hroosoe representaton It uses the sequene for of real value to desrbe the hroosoe, avods the oplex enodng for of bnary for, and an dsplay the pratal sgnfane of the representaton ore ntutvely.. Populaton ntalzaton The seleton of ntal luster enters wll have a great pat on the fnal lusterng results. However, due to the rossover operator that dynaally hanges the hroosoe length, the fxed ntal luster enters are not onduve to antanng the dversty of the populaton. Therefore, ths paper uses the ost oon ethod of rando gven ntal luster enters to ntalze the populaton. Note that: for the saple datasets, the range of d attrbute values ay not be the sae, whh an have a sgnfant pat on the alulaton of the NSGA-II

4 algorth. Therefore, t s neessary to standardze the saple data set, Max-Mn Noralzaton frst needs to be perfored on the saple dataset to redue the possble error. The Max-Mn Noralzaton s defned as follows: x Mn xnoralzaton (6) Max Mn. Seleton of ftness funton The perforane of ult-objetve optzaton s hghly dependent on the hoe of objetve funton, whh an produe good results by reasonably seletng the objetve funton. The seleton of objetve funtons should be suh so that they an balane eah other rtally and are possbly ontradtory n nature. Contradton n the objetve funtons s benefal sne t gudes to global optu soluton. It also ensures that no sngle lusterng objetve s optzed leavng the other probable sgnfant objetves unnoted. In ths paper, two knds of ftness funtons, DB Index and Index I, are used as objetve funtons for NSGA-II algorth. The two ftness funtons are desrbed n detal below. A. Daves-Bouldng(DB)Index DB Index [] s a oonly used luster valdty ndes. Ths ndex s the rato funton of the su of wthn-luster satter to between-luster separaton. Defne the satter of the th lass as where j S x z C q / q { j } (7) C j x denotes the data pont n the th lass and denotes the enter of the th lass; C represents the nuber of data ponts n the th lass; q s an ndex value. The dstane between luster enter z and z j, s defned as M z z. The slarty between th j j luster and jth luster, s defned as S Sj Rj ax, j{ } (8) M The Daves-Bouldn (DB) ndex s then defned as j N Rj N DB (9) The objetve s to nze the DB ndex for ahevng proper lusterng. B. Index I Index I [5] s another oonly used luster valdty ndes. E I( ) ( D ) P E (0) where s the nuber of lusters. Here, E stand for wthn-luster satter, defned as: n E u x z () j j j D stand for between-luster separaton, defned as: z ax D z z () j, j E and p are orrelaton oeffents. The power p s used to ontrol the ontrast between the dfferent luster onfguratons, n general, p. In ths artle, we have taken p. E s a onstant for a gven dataset, noralzed to avod the nu value of the ndator. The value of K for whh I () s axzed s onsdered to be the orret nuber of lusters. The goal n ths paper s to nze DB and / I ( ) sultaneously. At the sae te, pay attenton to adjust the orrelaton oeffent E n I (). By adjustng the paraeters, the values of DB and / I ( ) are n the sae order of agntude, avodng the seleton error aused by too large target value. At the sae te, n the use of the algorth, t an be found that wth the nrease of the nuber of lusters, the value of / I ( ) begns to derease, and the value of DB begns to nrease, whh onfors to the onfltng requreents of the two objetve funtons entoned earler.. Genet anpulaton A. Seleton The two ndvduals are randoly seleted to play a tournaent and the wnner s seleted by the rowded oparson operator. Ths operator takes nto aount two attrbutes of the non-donant rank and the rowded dstane. If two ndvduals are at dfferent levels, the lower level s preferred. If both ndvduals are at the sae level, hoose a soluton that has less rowded regon. B. Crossover After seleton, the seleted hroosoes are plaed n the atng pool. The perforane of rossover operator wll deterne the perforane of genet anpulaton to a great extent. Beause of the varable-length enodng used n hroosoe odng, the onventonal one-pont rossover approah does not apply to the urrent stuaton. In ths paper, the followng two rossover ethods are used to perfor rossover operaton wth the sae probablty. ) Based on the nearest neghbor athng rossover operaton Let two parents S [ a, a, a ] and S [ b, b, b ] denote parent solutons wth and luster enters. Suppose n the ase of,selet the d gene strng a ( a, a, a ) n turn, whh represents eah luster enter n S, and selet the nearest dstane d a strng b ( b, b, b ) fro S to ath the. Already pared gene strngs are no longer nvolved n parng. Reorderng the prevous gene strngs n S, hoose a pont randoly fro wthn 0~ d. For S and S, tradtonal rossover operatons are used to

5 generate new offsprng S and S. By usng ths rossover ethod, the offsprng antan the sae nuber of luster enters as ther parents, and antan the stablty of the populaton. Usng gene rearrangeents before rossover an ake the dfferent hroosoes have the ost slar luster enters n the sae poston, avodng the generaton of poor offsprng when rossng, and then lead to populaton degradaton. The rossover operaton an be llustrated n Fg.. Two parents a a a... b b b... ad a a bd b b rossover ethod Two hldren a a b... b b a... bd b b Fg. Crossover ethod ) Based on the trunaton and stthng ross operaton Dfferent fro the frst ethod, the rossover operaton based on trunaton and stthng wll produe the offsprng whh are dfferent fro the nuber of the parent luster enters, so as to antan the dversty of the populaton. In ths rossover operaton, the strng representng eah luster enter s ndvsble and an only be rossed at dfferent gene strngs. The operaton s desrbed as follows: S and S are two parent ndvduals, where S ( d) ( a, a, a a, a ) t t and S ( d) ( b, b, b b, b ). t t Suppose that the nterseton ponts of S and S are t and t respetvely. The offsprng S and S generated after rossng an be expressed as: S (( t t ) d) ( a, a, a b, b ) t t and S (( t t ) d) ( b, b, b a, a ). t t The nuber of luster enters represented by S and S s ( t t) and ( t t), respetvely. The rossover operaton an be llustrated n Fg.. Two parents a a a a b b b b rossover ethod a Two hldren b b b a a a Fg. Crossover ethod C. Mutaton Indvduals are utated aordng to gene lo, and rando varaton s usually ade aordng to the varaton probablty P. If the hroosoe s seleted for utaton, the loaton of the utated gene wll be seleted randoly. After utatng, the floatng pont nuber at the gene ste s replaed by another unfor rando nuber. D. Adaptve operaton By usng the adaptve strategy of rossover probablty P and utaton probablty P, the two paraeters an be autoatally hanged aordng to the a ad a a ftness of the urrent populaton. For the whole populaton, when the ftness value of the populaton tends to be onsstent or tends to loal optu, the P and P nrease approprately; when the ftness value s dspersed, the P are approprately redued. For an ndvdual n a populaton, when ts ftness s hgher than the average ftness of the populaton, the lower P values,ake t ore lkely to enter the next generaton; when the urrent ftness value s lower than the average ftness value, the hgher P and P values wll be gven to ake t ore lkely to be elnated. Thus, the adaptve strategy an provde the best P for the soluton [6]. P are alulated as follows: * ( p 0.6)( fax f ) * f fean P fax fean () * p f fean ( p 0.00)( fax f ) f fean P f ax fean () p f fean * where f s the larger ftness value of two ndvduals to be ross-operated, f s the ftness value of the urrent ndvdual, f ax s the axu ftness value of the urrent generaton, f s the average ftness of the ean urrent generaton, p 0.9, p 0.. It should be noted that the ftness value entoned here s the su of two objetve funton values. When an ndvdual's ftness value s the axu ftness value of a onteporary populaton, we set ts P to 0.6 and 0.00, respetvely. E. Seletng a soluton fro the non-donated set In ths paper, the ajorty votng ethod s used to deterne the nuber of lusters.that s to say, n the donant set, the nuber of ourrenes of a luster n the whole donatng luster s ore than 50% of the total nuber of ourrenes, and the sae nuber ontnuously appears ore than 5 generatons, we thnk t s the optal luster nuber. If the algorth stll an t hoose the optal luster nuber at the spefed axu nuber of teratons, the orrespondng to the best ndvdual n the fnal generaton s taken as the optal luster nuber. F. Deterne the fnal lusterng result After the nuber of lusters s deterned, all the ndvduals whose populaton nuber s equal to are seleted to for a new populaton for lusterng. The ethod s to use a obnaton of Canonal Genet Algorth (CGA) and algorths. The rossover operaton used here only uses the nearest neghbor athng ross operaton entoned above, so t wll not hange the nuber of lusters. By obnng the global optzaton algorth wth, ths an effetvely overoe the proble that the algorth an only

6 obtan the loal optal soluton. Fnally, the algorth wll ternate after the objetve funton value J no longer hanges obvously, and the obtaned result s the optal lusterng result. At ths pont, the relevant onepts of the algorth have been desrbed. Algorth shows the steps of the ADNSGA- algorth. Algorth. ADNSGA- nput: Dataset. Intalze paraeters and NSGA-II nludng populaton sze Pop, terax,,, p, p, Tax.. Rando to selet ntal nuber of lusters and rando to generate ntal luster enters to reate a ntalze populaton P(0).. Deode eah ndvdual to obtan the luster enters, and alulate the ebershp degrees U usng Eq.().. Calulate new luster enters V of eah ndvdual usng Eq.() based on U and V 5. Calulate the J of eah ndvdual usng Eq.() based on U and V 6. Calulate ftness values f and f of eah ndvdual usng Eq.(9) and (0).Calulate f f f,store f ax and f ean at eah teraton. 7. Non-donated sortng and rowdng dstane operaton for populaton. 8. Usng the rowded oparson operator to selet. 9. Calulate P usng Eq.() and (). 0. Generate offsprng usng genet operaton.. Reobnaton urrent generaton and offsprng to selet next generaton usng elts operaton.. Usng ajorty votng tehnque to deterne the nuber of luster.. If the nuber satsfes the seleton ondton, go to step ; else go to step.. Fnd all hroosoes whose luster nubers are equal to fro the populaton n step.the new populaton s oposed of these hroosoes. 5. Deode eah ndvdual to obtan the luster enters, and alulate the ebershp degrees U usng Eq.(). 6. Calulate new luster enters V of eah ndvdual usng Eq.() based on U and V 7. Calulate the J of eah ndvdual usng Eq.() based on U and V 8. Calulate ftness values f and f of eah ndvdual usng Eq.(9) and (0).Calulate f f f,store f ax and f ean at eah teraton. 9. Non-donated sortng and rowdng dstane operaton for populaton. 0. Usng the rowded oparson operator to selet.. Calulate P usng Eq.() and ().. Generate offsprng usng genet operaton.. Reobnaton urrent generaton and offsprng to selet next generaton usng elts operaton.. If ADNSGA- has not et the stoppng rteron ( f ( t ) f ( t) and t < Tax ), else t=t+ and go to step. 5. Return the best ndvdual (n J () t )..5 Te oplexty The ADNSGA- algorth has a worse-ase O ( GPN axd ) te oplexty, where G denotes the nuber of generatons, P s populaton sze, N s the sze of data, ax s axu nuber of lusters and d s data densons. () In the ntal stage of populaton, the te requred s ax d and eah strng ontans d densonal features untl the populaton sze P s full. Therefore, ths onstruton requres O( Pax d ). () In lusterng for eah ndvdual, suppose the nuber of data n the urrent data set s N, both proedures of ebershp assgnent and updatng of enter values take Nax d te. For the populaton, the te oplexty s O( PN axd ). () The te oplexty of the two objetve funtons DB Index and Index I are both ON ( ). () The te oplexty of eah exeuton of rossover and utaton operators s O( Pax d ). (5) The non-donated sortng n NSGA-II needs MP te for eah soluton to opare wth every other soluton to fnd f t s donated. M s the nuber of objetves and a axu nuber of the non-donated solutons equals the populaton sze P. The oparson for all populaton ebers therefore requres O( MP ) te, where M. (6) In the label assgnent for eah non-donated soluton, Nax d te s requred to assgn label for every data pont. To selet the best soluton fro P non-donated solutons, ths yelds O( PN axd ) te. It an be seen that the te oplexty of the algorth s worse, and the oplexty of eah generaton n the worst ase s O( PN d ). Assung that the algorth runs G generaton, the te oplexty s O( GPN d ). ax. Experent Study In ths paper, for the purpose of verfyng the perforane of the ethod proposed n ths paper (ADNSGA-), soe lusterng algorths are hosen for extensve oparatve analyss. The soft subspae lusterng (SSC) algorth based on sngle objetve evoluton (ESSC) [7], the SSC algorth based on ult-objetve evoluton (MOEASSC) [8], the rsp verson of lusterng ethod: the kernel-based attrbute-weghted algorths VKCM-KLP [9], the fuzzy verson of VKCM-KLP algorth, VK-K-LP [0], the kernel - based autoat attrbute-weght ult - objetve lusterng algorth MOKCW [] and the NSGA-II- ethod, t s the non-adaptve verson of ADNSGA- and used fxed paraeters.. Datasets and paraeter settng For the purpose of oparson, there are two groups ax

7 of data sets, artfal and real-lfe data sets. The three artfal data sets are Square, Square and Szes5 fro []. The sx real-lfe data sets obtaned fro the UCI Mahne Learnng Repostory [], naely Irs, Wne, Newthyrod, Vertebral, Iage, Abalone. As shown n Table the data sets onsdered are brefly desrbed, where C s the true nuber of lasses, d and n are, respetvely, the nuber of features and objets. For ost SSC algorths, the experents are onduted on the data sets standardzed nto the nterval [0,], whh an allevate the uneven pat of dfferent attrbutes ranges on updatng the weghts. Therefore, the standardzaton s based on the nu and axu values of eah attrbute. Table The haraters of datasets. DataSets C d n Square Square Szes5 Irs Wne 78 Newthyrod 5 5 Vertebral 6 0 Iage Abalone 8 77 The paraeters of the ADNSGA- algorth are set as shown n Table, the paraeters of other algorths are set as shown n Table. Table Paraeter settngs for the ADNSGA- algorth. Paraeter Settng Populaton sze 0 Max nuber of generatons 00 C n C ax n p 0.9 p 0. Table Paraeter settngs for other algorths Algorth Paraeter settng ESSC n( nd, ), 00, 0. n( nd, ) VKCM-K-LP VK-K-LP MOEASSC P 0.5, P K / D, 0 MOKCW P 0., 0. NSGA-II- P 0.9, P Experent result and analyss In the frst experent, the above nne data sets 7 (Square, Square, Szes5, Irs, Wne, Newthyrod, Vertebral, Iage, Abalone) and the adjusted Rand Index (ARI) [] are used here to evaluate the lusterng qualty of ADNSGA- algorth, whh s to be axzed. Table suarzes the result obtaned fro ADNSGA-. The real nuber of lusters an be obtaned by the ADNSGA- algorth n nne datasets. Fro the ARI value, the ADNSGA- algorth has a bg dfferene to the nne data sets. Ths s anly beause the ADNSGA- algorth uses the fuzzy -eans () ethod as the lusterng ethod, but the algorth s sutable for alloatng the data of the spheral lusters. Therefore,for sx datasets of Square, Square, Szes5, Irs, Wne and Newthyrod, the effet s better, and the effet of the other three data sets s poor. It also an be observed fro Table that, n all data sets, the optal lusterng result s obtaned by the algorth. Table Adjusted Rand ndex (ARI) and the nuber of the lusters (C) obtaned by ADNSGA-. DataSet Atual C ADNSGA- Square Square Szes5 Irs C ARI Wne Newthyrod 0.8 Vertebral 0.6 Iage Abalone 0.60 Fgs.- opare the results fro the three synthet data sets (Square, Square and Szes5) of data parttonng obtaned by ADNSGA- wth enter arkngs and the true data parttonng. The algorth perfors well under the well-separated strutures of Square (Fg. ) and Square (Fg. ) as well as the unequally-szed lusters of Szes5 (Fg. ). The overlappng and unequally-szed haraterst auses ore slassfaton of the data ponts whh are at the borderlne between lusters. In order to evaluate the perforane of the lusterng result of seven algorths, three well-known external CVIs auray (A), rand ndex (RI) [] and noralzed utual nforaton (NMI)[7] are adopted here. They all take ther values fro the nterval [0,], n whh eans the best ath between the result and the true partton, whereas 0 eans the worst result. In ths experent, all algorths are exeuted 0 tes ndependently, and ther perforanes are opared n ters of the best ase of A, RI, NMI shown n Tables 5. Aong the, the best result s expressed n bold.

8 Fg.. Square(C=): (a) true soluton and (b) data partton usng ADNSGA- (ARI=0.97). Fg.. Square(C=): (a) true soluton and (b) data partton usng ADNSGA- (ARI=0.80). Fg.. Szes5 (C=): (a) true soluton and (b) data partton usng ADNSGA- (ARI=0.88). It an be frstly observed fro Table 5 that, n all data sets, the optal lusterng result s obtaned by the ult-objetve algorth. Ths result an proves that the ult-objetve lusterng algorth has soe advantages opared wth the sngle-objetve lusterng algorth. For ost data sets, the ADNSGA- algorth proposed n ths paper an obtan the best results. For the data set Irs and Vertebral, the kernel based ult-objetve lusterng algorth MOKCW an aheve the best results. Copared wth MOKCW algorth and VKCM-K-LP algorth, the two results are slar, and the result of ADNSGA- s worse. For the Vertebral data set, the ADNSGA- algorth obtans the best A value. For the Wne data set, the MOEASSC algorth an aheve the best effet, and the ADNSGA- effet s very lose to t. It shows that two knds of ult-objetve lusterng ethods based on evolutonary oputaton an obtan the best global results on Wne datasets. In the three datasets of Newthyrod, Iage and Abalone, ADNSGA- proposed n ths paper has obvous advantages over other algorths.

9 Table 5 The result of all algorths on A, RI and NMI. DataSet ESSC VKCM-K-LP VK-K-LP MOEASSC MOKCW NSGA-II- ADNSGA- A Irs RI NMI A Wne RI NMI A Newthyrod RI NMI A Vertebral RI NMI A Iage RI NMI A Abalone RI NMI Fro Table 5 we an also see that the effet of the adopton of an adaptve ehans s effetve. ADNSGA- algorth usng the adaptve Table 6 Average perforane rankngs of dfferent algorths on ehans s sgnfantly better than the NSGA-II- all datasets regardng A, RI and NMI. algorth. Exept that the two ndators are the sae as Algorths A RI NMI ADNSGA- algorth, the other ndators of ESSC 5.(7) 5.(6).8(6) NSGA-II- are qute dfferent fro ADNSGA- VKCM-K-LP.().8().() algorth. More obvously, n the Iage dataset, the VK-K-LP.0().().7() NSGA-II- algorth annot obtan the orret MOEASSC.8(6) 5.(6).8(6) nuber of lusterng by 0 ndependent exeutons. MOKCW.().().8() NSGA-II-.(5).5(5).(5) Through areful analyss, we onlude that the adaptve ADNSGA-.7().8().0() ehans akes the ADNSGA- algorth fnally fnd the orret nuber of lusters. Ths s anly due to the fat that the adaptve ehans effetvely ontrols the speed of rossover and utaton of genet algorths. Beause the NSGA-II- algorth does not adopt the adaptve ehans, t leads to ts preature onvergene to the loal optal soluton, whh leads to the fnal lusterng nuber s wrong. Lookng at the other fve data sets. Beause the paraeter values of NSGA-II- are fxed, ths leads to the fat that the algorth does not ake full use of data nforaton n the optzaton proess, and the rate of onvergene s too fast. Although the orret nuber of lusters was eventually found, the lusterng effet was poor. Wth the nreasng nuber of data and attrbutes n the data set, Fg.. Mean values of A, RI, and NMI usng dfferent ths trend s even ore obvous. Fro ths set of algorths n the 6 real-lfe datasets experents, we an see that usng an adaptve Table 6 shows the average perforane rankngs of ehans does prove the lusterng effet. all algorths on the 6 datasets regardng A, RI, NMI Fro ths result, t s easy to thnk that beause the oputed fro Table 5, akng a ore evdent lusterng proble laks pror knowledge of the data set, oparson. Fro Table 6, we an see that the and the genet algorth s also a rando searh ADNSGA- algorth proposed n ths paper ranks algorth, t s dffult to gve sutable rossover frst n A and RI, rankng seond on NMI, anly due probablty and utaton probablty. However, adoptng to the fat that NMI ndators are not onsstent wth A an adaptve ehans here an avod gvng fxed and RI ndators. On A, the ADNSGA- global paraeters dretly. algorth has a greater advantage than the seond Fro the above analyss, we beleve that the algorth. On RI, the ADNSGA- algorth

10 perfors slghtly better than the seond algorth. In NMI, ADNSGA- algorth s worse than MOKCW algorth, but t s not uh dfferent. It shows that the ADNSGA- algorth has soe advantages over the other 6 algorths on the three ndexes of A, RI and NMI, and better lusterng results an be obtaned. Fg. shows the hstogra of ean values of the three ndes n oparson for dfferent algorths. As an be observed fro Table5, Table6 and Fg., the perforane of our proposed ethod has obvous advantages n the A ndex, and has a slght advantage n the RI ndex, whh s not as good as the MOKCW algorth n the NMI ndex. The fnal Pareto optal front obtaned by ADNSGA- lusterng tehnque on the real-lfe data sets, Irs, Wne, Newthyrod, Vertebral, Iage, and Abalone are llustrated fro Fgs. 5 7, respetvely. Fg.5. Pareto optal front obtaned by the proposed ADNSGA- algorth for Irs data set and Wne data set. Fg.6. Pareto optal front obtaned by the proposed ADNSGA- algorth for Newthyrod data set and Vertebral data set. Fg.7. Pareto optal front obtaned by the proposed ADNSGA- algorth for Iage data set and Abalone data set

11 5. Conluson Ths paper presents a fuzzy lusterng ethod based on ult-objetve genet algorth. The ADNSGA- algorth was developed to solve the lusterng proble by obnng the fuzzy lusterng algorth () wth the ult-objetve genet algorth (NSGA-II) and ntrodung an adaptve ehans. In ths paper, NSGA-II algorth uses two luster valdty ndexes of Index I and DB Index as ts objetve funton, so as to ontrol ult-objetve optzaton. The algorth does not need to gve the nuber of lusters n advane. After the nuber of ntal lusters and the enter oordnates are gven randoly, the optal soluton set s found by the ult-objetve evolutonary algorth. After deternng the optal nuber of lusters by ajorty vote ethod, the J value s ontnuously optzed through the obnaton of Canonal Genet Algorth and, and fnally the best lusterng result s obtaned. In addton to the bas fraework of ult-objetve genet algorth, the approprate objetve funton s also one of the suess fators of ADNSGA- algorth. Ths paper does not use a sngle luster evaluaton ndex, but uses two oprehensve evaluaton ndators. These two ndexes take nto aount both the wthn-luster satter and the between-luster separaton. The experental results show that the ult-objetve lusterng ethod s better than the sngle-objetve lusterng ethod, and the better lusterng results an be obtaned by hoosng a reasonable objetve funton. Although the ADNSGA- algorth perfors well, t also has soe nherent probles. Sne the algorth adopts the NSGA-II fraework, the ult-objetve genet algorth an only oprose aong ultple objetve funtons, so the ethod an only approah the real Pareto front. Beause the NAGA-II algorth s a knd of genet algorth, there are strong randoness, we an fnd the optal soluton through the randoness, also ay be due to rando unable to fnd the optal soluton, so an't guarantee the optal lusterng soluton s absolutely rght. In the followng work, we hope to prove the seleton and lusterng auray of the optal lusterng results. Aknowledgents Ths work was supported by the Natonal Natural Fund of Chna (Grant No ); the Fundaental Researh Funds for the Central Unverstes (Grant No.07XS5, 08QN096). Referene [] J.C. Bezdek, R. Ehrlh, W. Full, : The fuzzy -eans lusterng algorth, Coput. Geos. 0 (98) 9 0. [] J.B. MaQueen, Soe Methods for lassfaton and Analyss of Multvarate Observatons, n: Pro. 5-Th Berkeley Syp. Math. Stat. Probab., 967: pp [] E.R. Hrushka, R.J.G.B. Capello, A.A. Fretas, A.C.P.L.F. de Carvalho, A survey of evolutonary algorths for lusterng, IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 9 (009) 55. [] R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classfaton, nd edton, Wley, 00. [5] S. Bandyopadhyay, Sulated annealng usng a Reversble Jup Markov Chan Monte Carlo algorth for fuzzy lusterng, IEEE Trans. Knowl. Data Eng. 7 (005) [6] A. Hatalou, M. Hatalou, PSOHS: An effent two-stage approah for data lusterng, Meet Coput. 5 (0) [7] R.J. Kuo, Y.J. Syu, Z.-Y. Chen, F.C. Ten, Integraton of partle swar optzaton and genet algorth for dyna lusterng, Inf. S. (Ny). 95 (0) 0. [8] T.M. Slva Flho, B.A. Pentel, R.M.C.R. Souza, A.L.I. Olvera, Hybrd ethods for fuzzy lusterng based on fuzzy -eans and proved partle swar optzaton, Expert Syst. Appl. (05) [9] U. Maulk, S. Bandyopadhyay, Genet algorth-based lusterng tehnque, Pattern Reognt. (000) [0] U. Maulk, S. Bandyopadhyay, Fuzzy parttonng usng a real-oded varable-length genet algorth for pxel lassfaton, IEEE Trans. Geos. Reote Sens. (00) [] C.D. Nguyen, K.J. Cos, GAKREM: A novel hybrd lusterng algorth, Inf. S. (Ny). 78 (008) [] A. Ye, Y. Jn, I. Engneerng, Z. Noral, A Fuzzy C-Means Clusterng Algorth Based on Iproved Quantu Genet Algorth, Int. J. Database Theory Appl. 9 (06) 7 6. [] G. Gara, B. Chaudhur, A novel genet algorth for autoat lusterng, Pattern Reognt. Lett.5 (00) [] D.-X. Chang, X.-D. Zhang, C.-W. Zheng, A genet algorth wth gene rearrangeent for K-eans lusterng, Pattern Reognt. (009) 0. [5] S. Saha, S. Bandyopadhyay, A fuzzy genet lusterng tehnque usng a new syetry based dstane for autoat evoluton of lusters, n: Pro. - Int. Conf. Coput. Theory Appl. ICCTA 007, 007: pp. 09. [6] S. Bandyopadhyay, S. Saha, A pont syetry-based lusterng tehnque for autoat evoluton of lusters, IEEE Trans. Knowl. Data Eng. 0 (008) 57. [7] D.Gong, J.Sun, X.J. Evolutonary algorths wth preferene polyhedron for nterval ult-objetve optzaton probles. Inforaton Senes (0): -6. [8] D.Gong, X.J, J.Sun, X.Sun, Interatve evolutonary algorths wth deson-aker s preferenes for solvng nterval ult-objetve optzaton probles. Neurooputng 7 (0) -5. [9] D. Gong, J. Sun and Z. Mao. A set-based genet algorth for nterval any-objetve optzaton probles. IEEE Transatons on Evolutonary Coputaton. (06) [0] Y.Lu, D.Gong, J.Sun, Y.Jn. A any-objetve evolutonary algorth usng a one-by-one seleton strategy. IEEE Transatons on Cybernets, 7 (07) [] Y. Zhang, D. Gong, J. Cheng, Mult-Objetve Partle Swar Optzaton Approah for Cost-Based Feature Seleton n Classfaton, IEEE/ACM Transatons on Coputatonal Bology and Bonforats, (07) [] Y. Zhang, D. Gong, X. Sun, Y. Guo. A PSO-based ult-objetve ult-label feature seleton ethod n lassfaton. Sentf Reports 7.(07):76. [] J. Handl, J. Knowles, Evolutonary Multobjetve Clusterng, Let. Notes Coput. S. (00) [] J. Handl, J. Knowles, An evolutonary approah to ultobjetve lusterng, IEEE Trans. Evol. Coput.

12 (007) [5] S. Saha, S. Bandyopadhyay, A syetry based ultobjetve lusterng tehnque for autoat evoluton of lusters, Pattern Reognt. (00) [6] I. Saha, U. Maulk, D. Plewzynsk, A new ult-objetve tehnque for dfferental fuzzy lusterng, n: Appl. Soft Coput. J., 0: pp [7] X.L. Xe, G. Ben, A valdty easure for fuzzy lusterng, IEEE Trans. Pattern Anal. Mah. Intell. (99) [8] C. a. C. Coello, A Coprehensve Survey of Evolutonary-Based Multobjetve Optzaton Tehnques, Knowl. Inf. Syst. (999) [9] K. Deb, Multobjetve Optzaton Usng Evolutonary Algorths. Chhester, U.K.: Wley, 00. [0] K. Deb, A. Pratap, S. Agarwal, T. Meyarvan, A fast and eltst ultobjetve genet algorth: NSGA-II, IEEE Trans. Evol. Coput. 6 (00) [] E. Ztzler, L. Thele, An Evolutonary Algorth for Multobjetve Optzaton : The Strength Pareto Approah, TIK-Report. (998). [] E. Ztzler, M. Lauanns, L. Thele, SPEA: Iprovng the Strength Pareto Evolutonary Algorth, Evol. Methods Des. Opt. Control wth Appl. to Ind. Probl. (00) [] Zhao L,Tsujura Y,Gen M, Genet Algorth for Fuzzy Clusterng, Pro. IEEE Int. Conf. Evol. Coput. (996). [] D.L. Daves, D.W. Bouldn, A Cluster Separaton Measure, IEEE Trans. Pattern Anal. Mah. Intell. PAMI- (979) 7. [5] U. Maulk, S. Bandyopadhyay, Perforane evaluaton of soe lusterng algorths and valdty ndes, IEEE Trans. Pattern Anal. Mah. Intell. (00) [6] M. Srnvas, L.M. Patnak, Adaptve Probabltes of Crossover and Mutaton n Genet Algorths, IEEE Trans. Syst. Man Cybern. (99) [7] Z. Deng, K.S. Cho, F.L. Chung, S. Wang, Enhaned soft subspae lusterng ntegratng wthn-luster and between-luster nforaton, Pattern Reognt. (00) [8] H. Xa, J. Zhuang, D. Yu, Novel soft subspae lusterng wth ult-objetve evolutonary approah for hgh-densonal data, Pattern Reognt. 6 (0) [9] M.R.P. Ferrera, F.D.A.T. De Carvalho, E.C. Sões, Kernel-based hard lusterng ethods wth kernelzaton of the etr and autoat weghtng of the varables, Pattern Reognt. 5 (06) 0. [0] M.R.P. Ferrera, F.D.A.T. De Carvalho, Kernel fuzzy -eans wth autoat varable weghtng, Fuzzy Sets Syst. 7 (0) 6. [] Z. Zhou, S. Zhu, Kernel-based ultobjetve lusterng algorth wth autoat attrbute weghtng, Soft Coput. (07) :-5 [] K. Bahe, M. Lhan, UCI Mahne Learnng Repostory, Unv. Calf. Irvne Sh. Inf. 008 (0) 0. [] J.M. Santos, M. Ebrehts, On the use of the adjusted rand ndex as a etr for evaluatng supervsed lassfaton, n: Let. Notes Coput. S. (Inludng Subser. Let. Notes Artf. Intell. Let. Notes Bonforats), 009: pp [] J.Z. Huang, M.K. Ng, H. Rong, Z. L, Autoated varable weghtng n k-eans type lusterng, IEEE Trans. Pattern Anal. Mah. Intell. 7 (005)