A Clustering Algorithm using Cellular Learning Automata based Evolutionary Algorithm

Transcription

1 A Clusterg Algorthm usg Cellular Learg Automata based Evolutoary Algorthm R. RASTEGAR, M. RAHMATI, Member, IEEE, M. R. MEYBODI Comuter Eg. Deartmet, Amrkabr Uversty, Tehra, Ira {rrastegar, rahmat, Abstract -- I ths aer, a ew clusterg algorthm based o CLA-EC s roosed. The CLA-EC s a model obtaed by combg the cocets of cellular learg automata ad evolutoary algorthms. The CLA-EC s used to search for cluster ceters such a way that mmzes the squared-error crtero. The smulato results dcate that the roosed algorthm roduces clusters wth accetable qualty wth resect to squared-error crtera ad rovdes a erformace that s sgfcatly sueror to that of the K-meas algorthm. Idex Terms -- Clusterg, Cellular Learg Automata, CLA-EC, Otmzato. C I. INTRODUCTION lusterg s a mortat usuervsed classfcato method used detfyg some heret structure reset a set of data. The urose of clusterg s to grou data to subsets that have useful meag the cotext of a artcular roblem []. Varous clusterg methods have bee develoed whch may be classfed to the followg categores: herarchcal clusterg, learg etwork clusterg, mxture model clusterg, obectve-fuctobased clusterg, ad artto clusterg, etc [5][6]. The clusterg roblem ca be stated as fdg the clusters such that the betwee-grou scatter s maxmzed ad wthgrou scatter s mmzed. May heurstc techques for clusterg exst the lteratures, whch address the global mmzato of squared-error crtero fucto, Geetc Algorthms (GA) [][3][4][7] ad Smulated Aealg (SA) [6] are two of these techques. The Cellular Learg Automata (CLA), whch s troduced for the frst tme [], s a mathematcal model for modelg dyamcal comlex systems that cossts of large umber of smle comoets. The smle comoets of CLA, whch have learg caabltes, act together to solve a artcular roblem. Ths model has bee aled to several roblems such as mage rocessg [0], chael assgmet cellular moble system [2], fucto otmzato [3], modelg of rumor dffuso [8], VLSI Placemet [9], ad modelg commerce etworks [9]. I [3], CLA ad evolutoary comutg are combed to obta a ew model called CLA-EC for otmzato roblems. Ths model s caable of erformg search comlex, large ad multmodal ladscae. I ths aer, a ew clusterg algorthm based o CLA-EC s roosed. The CLA-EC s used to search for cluster ceters such a way that mmzes the squared-error crtero. Due to arallel ature of CLA-EC model, the roosed algorthm s arorate for clusterg large data set. I order to demostrate the effectveess of the roosed CLA-ECclusterg, 6 dfferet two-dmesoal data sets ad IRIS data set are cosdered. Our exermetal results of clusterg dcate that the CLA-EC based clusterg algorthm rovdes a erformace that s sgfcatly sueror to that of the K-meas algorthm. The rest of the aer s orgazed as follows. Secto 0II brefly resets the clusterg roblem. Secto III gves a bref revew of learg automata, cellular learg automata ad CLA-EC model. The roosed clusterg algorthm s descrbed secto IV. Secto V resets the smulato results for dfferet data sets ad the last secto (VI) s the cocluso. II. CLUSTERING I a clusterg roblem, a data set, N-dmesoal Eucldea sace, S={x,,x M } s gve, where x R N ad M s the umber of data. Cosderg K clusters, rereseted by C,,C K,the clusters should satsfy the followg codtos, C φ, =,..., K, C S, K U = = C C = φ,,, =,..., K. Amog varous clusterg methods, the K-meas method s more attractve tha others ractcal alcatos [5]. The K-meas clusterg algorthm s oe of the well-kow clusterg methods, whch s based o a teratve hllclmbg algorthm. Oe of the most mortat dsadvatages of ths algorthm s that t s very sestve to the tal cofgurato ad may be traed a local mmum [5]. Therefore, several aroxmate methods such as Smulated Aealg (SA) [6] ad Geetc Algorthm (GA) [][3][4][7] have bee develoed to solve the above roblem. III. THE CLA-EC MODEL Learg Automata: Learg Automata are adatve decso-makg devces oeratg o ukow radom evromets. The Learg Automato has a fte set of actos ad each acto has a certa robablty (ukow for the automato) of gettg rewarded by the evromet of the automato. The am s to lear choosg the otmal acto (.e. the acto wth the hghest robablty of beg rewarded) through reeated teracto o the system. If the learg algorthm s chose roerly, the the teratve rocess of teractg o the evromet ca be made to result selecto of the otmal acto. Fgure llustrates

2 how a stochastc automato works feedback coecto wth a radom evromet. Learg Automata ca be classfed to two ma famles: fxed structure learg automata ad varable structure learg automata (VSLA). I the followg the varable structure learg automata s descrbed. A VSLA s a qutule < α, β,, T(α,β,) >, where α, β, are a acto set wth s actos, a evromet resose set ad the robablty set cotag s robabltes, each beg the robablty of erformg every acto the curret teral automato state, resectvely. Fucto T s the reforcemet algorthm, whch modfes the acto robablty vector wth resect to the erformed acto ad the receved resose. Let a VSLA oerate a evromet wth β={0,}. Let N be the set of oegatve tegers that rereset stace of teratos. A geeral lear schema for udatg acto robabltes ca be rereseted as follows. Let acto be erformed at stace. If β()=0 (reward), ( + ) = ( ) + a[ ( )] ( + ) = ( a) ( ) If β()=(ealty), ( + ) = ( b) ( ) ( + ) = ( b s ) + ( b) ( ) Where a ad b are reward ad ealty arameters. Whe a=b, the automato s called L RP. If b=0 the automato s called L RI ad f 0<b<<a<, the automato s called L RεP. For more Iformato about learg automata the reader may refer to [2] [7]. Fg.. The teracto betwee learg automata ad evromet Cellular Learg Automata: The Cellular Learg Automata (CLA) [][20] s a mathematcal model for dyamcal comlex systems that cossts of large umber of smle comoets. The smle comoets, whch have learg caabltes, act together to roduce comlcated behavoral atters. A CLA s a cellular automata whch learg automato (or multle learg automato) s assged to ts every cell. The learg automato resdg a artcular cell determes ts state (acto) o the bass of ts acto robablty vector. There s a rule that CLA oerate uder t. The rule of CLA ad the actos selected by the eghborg LAs of ay artcular LA determes the reforcemet sgal to that LA (multle LA). I CLA, the eghborg LAs of ay artcular LA costtute ts local evromet. The local evromet s o-statoary because t vares as acto robablty vector of eghborg LAs vary. The oerato of cellular learg automata ca be descrbed as follows: At the frst ste, the teral state of every cell s secfed. The teral state of every cell s determed o the bass of acto robablty vectors of learg automato (automata) resdg that cell. The tal value may be chose o the bass of ast exerece or at radom. I the secod ste, the rule of cellular learg automata determes the reforcemet sgal to each learg automato (automata) resdg that cell. Fally, each learg automato udates ts acto robablty vector o the bass of the suled reforcemet sgal ad the chose acto. Ths rocess cotues utl the desred result s obtaed. The model of CLA-EC (Cellular Learg Automata based Evolutoary Comutg) [3]: The CLA-EC model [3] s obtaed by combg cellular learg automata ad evolutoary comutg. Ths model s caable of erformg search comlex, large ad multmodal ladscae. I CLA-EC, smlar to other evolutoary algorthms, the arameters of the search sace are ecoded the form of geomes. Each geome has two comoets, model geome ad strg geome. Model geome s a set of learg automata. The set of actos selected by ths set of learg automata determes the secod comoet of geome (strg geome). Based o a local rule, a reforcemet sgal vector s geerated ad gve to the set of learg automata. Accordg to the learg algorthm, each learg automato the set of learg automata udates ts teral state accordg to a learg algorthm. The each learg automata a cell chooses oe of ts actos usg ts robablty vector. The set of actos chose by the set of automata resdg a cell determes a caddate strg geome that may relace the curret strg geome. The ftess of ths strg geome s the comared wth the ftess of the strg geome resdg that cell. If the ftess of the geerated geome s better tha the qualty of the stg geome of the cell, the geerated strg geome becomes the strg geome of that cell. The rocess of geeratg strg geome by the cells of the CLA-EC s reeated utl a termato codto s satsfed. I order to have a effectve algorthm, the desger of the algorthm must be careful about determg a sutable geome reresetato, ftess fucto for the roblem at had, the arameters of CLA such as the umber of cells (oulato sze), the toology, ad the tye of the learg automata for each cell. Assume f be a real fucto that s to be mmzed. m m{ f ( ξ ) ξ B } where B m s {0,} m. I order to use CLA-EC for otmzato of fucto f, frst a set of learg automata wll be assocated to each cell of CLA-EC. The umber of learg automata assocated to a cell of CLA-EC s the umber of bts the strg geome reresetg ots of the search sace of f. Each automato has two actos: 0 ad. The CLA-EC terates the followg stes utl the termato codto s met. Ste: every automato cell chooses oe of ts actos usg ts acto robablty vector. Ste 2: cell geerates a ew strg geome, η, by combg the actos chose by the set of learg automata of cell. The ewly geerated strg geome s obtaed by cocateatg the actos of the automata (0 or ) assocated to that cell. Ste 3: Every cell comutes the ftess value of strg geome η ; f the ftess of ths strg geome s better tha the oe the cell, the the ew strg geome η becomes the strg geome of that cell. That s

3 ξ ξ + = η where ξ ad f ( ξ ) f ( η+ ) f ( ξ ) > f ( η ) + + η reset the strg geome ad the ew strg geome of cell at stace. Ste 4: Se cells of the eghborg cells of the cell are selected. Ths selecto s based o the ftess value of the eghborg cells accordg to trucato strategy [8]. Ste 5: Based o the selected eghborg cells a reforcemet vector s geerated. Ths vector becomes the ut to the set of learg automata assocated to the cell. Let Ne() be set selected eghbors of cell. Defe, N l, ( k) = δ ( ξ ), k, l Ne( ) = where, f ex s true δ (ex) = 0 otherwse β,, the reforcemet sgal gve to learg automato of cell, s comuted as follows,, u( N () (0) ) = 0,, N, f ξ t β =, u( N, (0) N, ()) f ξ t = where u(.) s a ste fucto. The overall oerato of CLA- EC s summarzed the algorthm of fgure 2. Whle ot doe do For each cell CLA do arallel Geerate a ew strg geome; Evaluate the ew strg geome; If f(ew strg geome) < f(old strg geome) the Accet the ew strg geome Ed f Select Se cells from eghbors of cell ; Geerate the reforcemet sgal vector; Udate teral state LAs of cell Ed arallel for Ed whle Fg. 2. Pseudocode of CLA-EC = max m where x, s the th comoets of x. Secod, a ew search sace where we deote t by R whch s R = [0, ] [0, N ] s defed. where s Cartesa roduct sg. B. The CLA-EC Phase I the CLA-EC hase, clusters are otmzed wth resect to the squared error crtero. The characterstcs of the aled CLA-EC are as follows. Strg geome reresetato: Each strg geome s rereseted by a bary strg cosstg of M N arts where each art s a reresetato of a ecoded real umber. Let λ, be ( N + ) th art of strg geome where s the dmeso of the ceter of cluster R. If bary reresetato of λ, has w bts the a N-dmesoal sace wth K clusters, the legth of a strg geome s m=σσw. Ftess fucto (total sum of dstaces): To comute the ftess value of ξ, at frst, we comute λ, by decodg ξ, ad set λ, to be (λ, +m ). The ftess value of geome s comuted as follows: where, M * 2,..., K ) = ( Α ) = f ( ξ ) = f ( λ λ, = m Α * Α, = x λ Ad, K Parameters of CLA: A oe dmesoal CLA wth wra aroud coecto ad wth the eghborhood show fgure 3a s used. The eghbors of cell are cell - ad cell +. The archtecture of each cell s show fgure 3b. Each cell s equed wth m learg automata. The strg geome determer comares the ew strg geome wth the strg geome resdg the cell. The strg wth the hgher qualty relaces the strg geome of the cell. Deedg o the eghborg strg geomes ad the strg geome of the cell, a reforcemet sgal wll be geerated by the sgal geerator. Α IV. A CLA-EC BASED CLUSTERING ALGORITHM We roose to use the CLA-EC model to determe the K cluster ceters of the data set R N ; thereby clusterg the set of M ots of S={x,,x M }. The sum of the squared dstaces of the ots from ther resectve cluster ceters s take as the clusterg metrc, ad deotes t by f. The am s to search the cluster ceters such a way that fucto f be mmzed. The roosed algorthm cossts of three hases: rerocessg hase, the CLA-EC hase ad the clusterg hase. A. Prerocessg Phase The urose of rerocessg hase s to reduce the sze of the search sace o whch CLA-EC wll oerate. To reduce the sze of the search sace, at frst the largest ad the smallest values of each dmeso of data set s foud as follows: m = m { x, }, max = max{ x, } M M Fg. 3. The toology of the CLA-EC used ths aer. Termato Crtera: CLA-EC stos after a resecfed umber of teratos. The best strg geome

4 foud the last terato s the soluto to the clusterg roblem. For the exermetatos that follow the maxmum umber of terato s set to 200. C. The Clusterg Phase I ths hase, the clusters are created usg ther ceters, whch are ecoded the best strg geome reorted by the ervous hase. Ths s doe by assgg each ot x, = M, to oe of the clusters C k wth ceter λ k such that, = arg m Α, C k, K where Α, = x λ All tes are resolved arbtrary. Fg. 4. Data Data 2. Fg. 5. Data 3 Data 4. Fg. 6. Data 5 Data 6. V. SIMULATION RESULTS Several smulatos are coducted order to evaluate the effectveess of the roosed method. The results are the comared wth the results obtaed for K-meas algorthm. Smulatos are coducted for seve dfferet data sets, whch we call them Data, Data 2, Data 3, Data 4, Data 5, Data 6 ad IRIS Data set. The characterstcs of these data sets are gve below. Data : a two-dmesoal data set wth 4 clusters ad 59 ots as show fgure 4a. Data 2: a two-dmesoal data set wth 4 clusters ad 25 ots as show fgure 4b. Data 3: a two-dmesoal data set wth 4 clusters ad 70 ots as show fgure 5a. Data 4: a two-dmesoal data set wth 5 clusters ad 28 ots as show fgure 5b. Data 5: a two-dmesoal data set wth 5 clusters ad 00 ots as show fgure 6a. Data 6: a two-dmesoal data set wth 3 clusters ad 35 ots as show fgure 6b. Irs data: Ths data set reresets dfferet categores of rses havg four feature values. The four feature values rereset the seal legth, seal wdth, etal legth ad the etal wdth cetmeter. It has 3 clusters wth 50 ots. For the sake of coveece resetato, we use CLA- EC(automata(a,b), se,q) to refer to the CLA-EC algorthm wth q cells, the umber of selected cell Se ad whe usg learg automata automata wth reward arameter a ad ealty arameter b. Exermet I: I ths exermet we study the effectveess (qualty of clusters foud) of the roosed clusterg algorthm wth resect to CLA-EC arameters such as ealty ad reward arameters, the umber of cells, ad the umber of selected cells. Tables of [2] shows the results of the CLA-EC-clusterg algorthm for data set Data for dfferet values of the arameters of the CLA-EC such as the tye of the learg automata, the ealty ad the reward arameters of the learg automata, the umber of selected cells ad the umber of cells. For each smulato maxmum umber of teratos of CLA-EC s take to be 200. It s clear that as the mea ad the stadard devato decrease the qualty of the clusterg becomes better. By careful secto of the results reorted Table [2] t s foud that as the umber of cells creases, the mea ad the stadard devato of the result decreases. Also, t has bee foud that, better results are obtaed whe each automato uses L RP or L RεP learg algorthm ad whe Se s set to. Fgure 7 shows the effect of the umber of cells CLA- EC(L RP (0.0,0.0),,-) o clusterg IRIS data set. It s show that as the umber of cells creases the qualty of clusterg becomes better. Fgure 8 shows the ftess of the best geome (sold le) ad the mea of the ftess of geomes (dashed le) for each terato whe usg CLA- EC(L RP (0.0,0.0),,5) for Data ad Data 4. Exermet 2: I ths exermetato we comare the results of the roosed algorthm wth that of K-meas algorthm. For ths exermetato CLA-EC has 5 cells, each automato uses L RP learg algorthm wth a=b=0., Se s ad the maxmum umber of teratos s set to be 200. The results of 50 smulatos for Data 2 ad Data 3 are show fgure 9. For Data t s foud that the CLA-ECclusterg algorthm rovdes the otmal value of % of the rus whereas K-meas algorthm attas ths value 8% of the rus. Both algorthms get traed at a local mmum for the other rus. For Data 2, CLA-ECclusterg attas the best value of all the rus. K- meas, o the other had, attas ths value for 28% of the rus, whle other rus t gets stuck at some of ts local mma (such as , ad ). For Data 3,

5 Data 4, Data 5, Data 6, ad IRIS data set the CLA-ECclusterg attas the best values of , 873.7, , , ad %, 00%, 0%, 40%, %30 of the rus, resectvely. The best values attaed by the K-meas algorthm for these data sets are , 873.7, , , ad %, 30%, 2%, 30%, ad %24 of rus, resectvely. Table 2 shows the summary of results of ths exermet. By careful secto of the results t s foud that the CLA-EC(L RP (0.,0.),, 5) erforms better tha the K-meas method for Data, Data 2, Data 3, Data 4, Data 4, Data5, Data 6, ad IRIS data set. Fg. 7. Effect of the umber of cells o the qualty of clusterg for IRIS data set whe usg CLA-EC(L RP(0.0,0.0),, -) -- shows mea ad shows the stadard devato over 50 rus. VI. CONCLUSION I ths aer, a ew clusterg algorthm based o CLA- EC, called CLA-EC-clusterg, was develoed. The CLA- EC fds the cluster ceters, such a way that the squarederror crtero be mmzed. I order to demostrate the effectveess of the CLA-EC-clusterg, 6 dfferet twodmesoal data sets ad IRIS data set were cosdered. The results of smulatos showed that the CLA-ECclusterg algorthm rovdes a erformace that s sgfcatly sueror to that of the K-meas algorthm. Due to the arallel ature of CLA-EC, the roosed algorthm s very sutable for clusterg large data set. [2] Begy, H., ad Meybod, M. R., A Self-Orgazg Chael Assgmet Algorthm: A Cellular Leag Automata Aroach, Vol of Srger-Verlag Lecture Notes Comuter Scece, PP. 9-26, Srger-Verlag, [3] Bhuya, J. N., Raghava, V. V., ad Elayavall, V. K., Geetc Algorthm for Clusterg wth a Ordered Reresetato, Iteratoal Coferece o Geetc Algorthms 9, PP , 99. [4] Gara, G., ad Ghaudhur, B. B., A Novel Geetc Algorthm for Automatc Clusterg, Patter Recogto Letters, No. 25, PP , [5] Ja, A. K., Du, R. P. W., ad Mao, J., Statstcal Patter Recogto: A Revew, IEEE Trasacto o Patter Aalyss ad Mache Itellgece, Vol 22, PP. 4-37, [6] Kle, R., ad Dubes, R., Exermets Proecto ad Clusterg by Smulated Aealg, Patter Recogto, Vol 22, PP , 989. [7] Maulk, U., ad Badyoadhyay, S., Geetc Algorthm-Based Clusterg Techque, Patter Recogto, No. 33, PP , [8] Meybod, M. R., ad Taherkha, M., Alcato of Cellular Learg Automata to Modelg of Rumor Dffuso, Proceedgs of 9th Coferece o Electrcal Egeerg, Power ad Water sttute of Techology, Tehra, Ira, PP. 02-0, May 200. [9] Meybod, M. R., ad Khoaste, M. R., Alcato of Cellular Learg Automata Modelg of Commerce Networks, Proceedgs of 6 th Aual Iteratoal Comuter Socety of Ira Comuter Coferece CSICC200, Isfaha, Ira, PP , 200. [0] Meybod, M. R., ad, Kharazm, M. R., Image Restorato Usg Cellular Learg Automata, Proceedgs of the Secod Iraa Coferece o Mache Vso, Image Processg ad Alcatos, Tehra, Ira, PP , [] Meybod, M. R., Beyg, H., ad Taherkha, M., Cellular Learg Automata, Proceedgs of 6 th Aual Iteratoal Comuter Socety of Ira Comuter Coferece CSICC200, Isfaha, Ira, PP , 200. [2] Naredra, K. S., ad Thathachar, M. A. L., Learg Automata: A Itroducto, Prtce-Hall Ic, 989. [3] Rastegar, R., ad Meybod, M. R., A New Evolutoary Comutg Model based o Cellular Learg Automata, Acceted IEEE CIS 2004, Sgaore, [4] Saheb Zama, M., Mehdour, M., ad Meybod, M. R., "Imlemetato of Cellular Learg Automata o Recofgurable Comutg Systems", IEEE CCGEI 2003 Coferece, Motreal, Caada, May [5] Selm, S. Z., ad Ismal, M. A., K-meas-tye Algorthm: Geeralzed Covergece Theorem ad Characterzato of Local Otmalty, IEEE Trasacto o Patter Aalyss ad Mache Itellgece, Vol 6, PP 8-87, 984. [6] Yag, M. S., ad Wu, K. L., A Smlarty-Based Robust Clusterg Method, IEEE Trasacto o Patter Aalyss ad Mache Itellgece, Vol. 26, No. 4, Arl [7] Thathachar, M. A. L., Sastry, P. S., Varetes of Learg Automata: A Overvew, IEEE Trasacto o Systems, Ma, ad Cyberetcs-Part B: Cyberetcs, Vol. 32, No. 6, PP , [8] Goldberg, D. E., Geetc Algorthms Search, Otmzato ad Mache Learg, Addso-Wesley, New York, 989. [9] Meybod, M. R., ad Mehdour, F., VLSI Placemet Usg Cellular Learg Automata, Proceedgs of 8 th Aual Iteratoal Comuter Socety of Ira Comuter Coferece CSICC200, Mashhad, Ira, PP , [20] Begy, H. ad M. R. Meybod, A Mathematcal Framework for Cellular Learg Automata, Advaces Comlex Systems, to aear. [2] Rastegar, R., Rahmat, M., ad Meybod, M. R., A New Clusterg Algorthm based O CLA-EC, Techcal Reort, Comuter Eg. Deartmet, Amrkabr Uversty, REFERENCES [] Badyoadhyay, S., ad Maulk, U., A Evolutoary Techque based o K-meas Algorthm for Otmal Clusterg R N, Iformato Sceces, No. 46, PP , 2002.

6 Fg. 8. The ftess of the best geome (sold le) ad the mea of the ftess of geomes (dashed le) for each terato whe usg CLA- EC(L RP (0.0,0.0),,5) for a) Data b) Data 4 Fg. 9. Comarso of the K-meas ad the CLA-EC(L RP(0.,0.),, 5) shows the total sum of dstaces obtaed for K-meas ad CLA- EC(L RP(0.,0.),, 5) over 50 dfferet rus for Data 2- shows the total sum of dstaces obtaed for K-meas ad CLA-EC(L RP (0.,0.),, 5) over 50 dfferet rus for Data 3. The sold le s for the CLA- EC(L RP (0.,0.),, 5) ad the dashed le s for the K-meas algorthm. TABLE. The results of the CLA-EC(L RP (0.,0.),, 5) algorthm (maxmum 200 teratos) ad the K-meas algorthm for Data,2,3,4,5,6, IRIS - Colums Mea ad Std show the mea ad stadard devato over 50 rus. Data Set (CLA-EC-clusterg) Mea (CLA-EC-clusterg) Std (Kmeas) Mea (Kmeas) Std Irs