A New Evolutionary Computing Model based on Cellular Learning Automata

Transcription

1 Proceedgs of the 2004 IEEE Coferece o Cyberetcs ad Itellget Systems Sgaore, 1-3 December, 2004 A New Evolutoary Comutg Model based o Cellular Learg Automata R. Rastegar Soft Comutg Lab. Comuter Egeerg Deartmet Amrkabr Uversty Tehra, Ira rrastegar@ce.aut.ac.r M. R. Meybod Soft Comutg Lab. Comuter Egeerg Deartmet Amrkabr Uversty Tehra, Ira meybod@ce.aut.ac.r Abstract: I ths aer, a ew evolutoary comutg model, called CLA-EC, s roosed. Ths ew model s a combato of a model called cellular learg automata (CLA) ad the evolutoary model. I ths ew model, each geome s assged to a cell of cellular learg automata to each of whch a set of learg automata s assged. The set of actos selected by the set of automata assocated to a cell determes the geome s strg for that cell. Based o a local rule, a reforcemet sgal vector s geerated ad gve to the set learg automata resdg the cell. Based o the receved sgal, each learg automato udates ts teral structure accordg to a learg algorthm. The rocess of acto selecto ad udatg the teral structure s reeated utl a redetermed crtero s met. Ths model ca be used to solve otmzato roblems. To show the effectveess of the roosed model t has bee used to solve several otmzato roblems such as real valued fucto otmzato ad clusterg roblems. Comuter smulatos have show the effectveess of ths model. 1. INTRODUCTION Evolutoary algorthms form a class of radom search algorthms whch rcles of atural evoluto are regarded as rules for otmzato. They are ofte aled to otmzato roblem where secalzed techques such as gradet based algorthms, lear rogrammg, dyamc rogrammg, ad etc, are ot avalable or stadard methods fal to gve reasoable aswers due to multmodalty, odfferetablty or dscotue of the roblem at had. The oor behavors of evolutoary algorthms such as geetc algorthms some roblems, whch the desged oerators of crossover ad mutato do ot guaratee that the buldg block hyothess s reserved, have led to roose other aroaches. Toward the develomet of a more robust evolutoary algorthm three ma aroaches have bee take to revet buldg blocks dsruto, the frst aroach; researchers have focused o evolvg a roblem s geome reresetato coucto wth ts soluto [1]. Others have attemted to evolve recombato oerators usg selfadatato mechasms [2]. Ad others have tred to relace the cocet of recombato by exlctly modelg of good solutos search sace [3][4][5]. I [6], Cellular learg automata (CLA) whch a combato of the cellular automata (CA) ad learg automata (LAs) s troduced.. Ths model s sueror to CA because of ts ablty to lear ad also s sueror to sgle LA because t s a collecto of LAs, whch ca teract wth each other toward solvg a artcular roblem. The basc dea of CLA, whch s suer class of stochastc CA, s to use learg automata to adust the state trasto robablty of stochastc CA [7]. So far, CLA have bee used may alcatos such as VLSI lacemet [8], rumor dffuso [9], chael assgmet cellular moble systems [10], call admsso cotrol cellular moble system [11], cooerato mutaget systems [12]. For more formato about CLA ad ts mathematcal studes the reader may refer to [7]. I ths aer a ew model, called cellular learg automata based evolutoary comutg (CLA-EC), whch s the combato of cellular learg automata (CLA) ad evolutoary comutg model s reseted. I ths model, each geome s assged to a cell of cellular learg automata to each of whch a set of learg automata s assged. The set of actos selected by the set of automata assocated to a cell determes the geome s strg for that cell. Based o a local rule, a reforcemet sgal vector s geerated ad gve to the set learg automata resdg the cell. Based o the receved sgal, each learg automato udates ts teral structure accordg to a learg algorthm. The rocess of acto selecto ad udatg the teral structure s reeated utl a redetermed crtero s met. Ths model ca be used to solve otmzato roblems. CLA-EC s caable of solvg roblems wth very comlex ladscae. Oe of the advatages of CLA-EC lke ts couterart, CLA, s ts heret arallelsm. The rest of ths aer s orgazed as follows. Secto 2 descrbes the learg automata ad cellular learg automata. Secto 3 troduces the CLA-EC algorthm. The exermetal results of CLA-EC algorthm are roosed Secto 4. Fally we draw cocluso Secto CELLULAR LEARNING AUTOMATA Learg Automata [13][14] are adatve decso-makg devces oeratg o ukow radom evromets. The Learg Automato has a fte set of actos ad each acto has a certa robablty (ukow for the automato) of /04/$ IEEE 433

2 gettg rewarded by the evromet of the automato. The am s to lear to choose the otmal acto (.e. the acto wth the hghest robablty of beg rewarded) through reeated teracto o the system. If the learg algorthm s chose roerly, the the teratve rocess of teractg o the evromet ca be made to result selecto of the otmal acto. s esecally sutable for modelg atural systems that ca be descrbed as massve collectos of smle obect teractg locally wth each other. Cellular automato has ot oly a smle structure for modelg comlex systems, but also t ca be mlemeted easly o SIMD rocessors. Therefore t has bee used evolutoary comutg frequetly. Mush lteratures are avalable o cellular automata ad ts alcato to evolutoary comutg, ad the terested reader s referred to [16][15]. Fg. 1 The teracto betwee learg automata ad evromet Fgure 1 llustrates how a stochastc automato works feedback coecto wth a radom evromet. Learg Automata ca be classfed to two ma famles: fxed structure learg automata ad varable structure learg automata (VSLA) [19][28]. I the followg, the varable structure learg automata s descrbed. A VSLA s a qutule <α,β,,t(α,β,)>, where α, β, are a acto set wth s actos, a evromet resose set ad the robablty set cotag s robabltes, each beg the robablty of erformg every acto the curret teral automato state, resectvely. The fucto of T s the reforcemet algorthm, whch modfes the acto robablty vector wth resect to the erformed acto ad receved resose. Let a VSLA oerate a evromet wth β={0,1}. Let N be the set of oegatve tegers. A geeral lear schema for udatg acto robabltes ca be rereseted as follows. Let acto be erformed at stace. If β()=0, ( + 1) = + a[1 ] (1) ( + 1) = (1 a) If β()=1, ( + 1) = (1 b) (2) ( + 1) = ( b s 1) + (1 b) Where a ad b are reward ad ealty arameters. Whe a=b, automato s called L RP. If b=0 ad 0<b<<a<1, the automato s called L RI ad L RεP, resectvely. Fgure 2 show the workg mechasm of learg automata. Oe of the models that are used to develo cellular evolutoary algorthm s a cellular automato (CA). A cellular automato s a abstract model that cossts of large umbers of smle detcal comoets wth a local teracto. CA s o-lear dyamcal systems whch sace ad tme are dscrete. It called cellular, because t s made u cells lke ots the lattce or lke squares of the checker boards ad t s called automata, because t follows a smle rule [15]. The smle comoets act together to roduce comlcate atters of behavor. CA erforms comlex comutato wth hgh degree of effcecy ad robustess. It Italze to [1/s,1/s,,1/s] where s s the umber of actos Whle ot doe Select a acto based o the robablty vector Evaluate acto ad retur a reforcemet sgal β Udate robablty vector usg learg rule. Ed Whle Fg. 2 Pseudocode of varable-structure learg automato Cellular Learg Automata s a mathematcal model for dyamcal comlex systems that cossts of large umber of smle comoets. The smle comoets, whch have learg caabltes, act together to roduce comlcated behavoral atters. A CLA s a CA whch learg automato (multle learg automato) s assged to ts every cell. The learg automato resdg artcular cell determes ts state (acto) o the bass of ts acto robablty vector. Lke CA, there s a rule that CLA oerate uder t. The rule of CLA ad the actos selected by eghborg LAs of ay artcular LA determe the reforcemet sgal to the LA resdg that cell. I CLA, the eghborg LAs of ay artcular LA costtute ts local evromet, whch s ostatoary because t vares as acto robablty vector of eghborg LAs vary. The oerato of cellular learg automata could be descrbed as follows: At the frst ste, the teral state of every cell secfed. The state of every cell s determed o the bass of acto robablty vectors of learg automata resdg that cell. The tal value may be chose o the bass of exerece or at radom. I the secod ste, the rule of cellular automata determes the reforcemet sgal to each learg automato resdg that cell. Fally, each learg automato udates ts acto robablty vector o the bass of suled reforcemet sgal ad the chose acto. Ths rocess cotues utl the desred result s obtaed [7]. 3. CELLULAR LEARNING AUTOMATA BASED EVOLUTIONARY COMPUTING I ths secto, Cellular Learg Algorthm Based Evolutoary Comutg, called CLA-EC s troduced as a ew arallel model for evolutoary comutg. I CLA-EC, smlar to other evolutoary algorthms, the arameters of the search sace are ecoded the form of geomes. Each geome has two comoets, model geome ad strg geome. Model geome s a set of learg automata. The set of actos selected by the set of learg automata determes the secod comoet of the geome called strg geome. For each cell, based o a local rule, a reforcemet sgal 434

3 vector s geerated ad gve to the set of learg automata resdg that cell. Each learg automato based o the receved sgal udate ts teral structure accordg to a learg algorthm. The, each cell CLA-EC geerates a ew strg geome ad comares ts ftess wth the ftess of the strg geome of the cell. If the ftess of the geerated geome s better tha the qualty of the stg geome of the cell, the geerated strg geome becomes the strg geome of that cell. Ths rocess of geeratg strg geome by the cells of the CLA-EC s terated utl a termato codto s satsfed. The ma ssue volved desgg a CLA-EC for a roblem s fdg a sutable geome reresetato ad ftess fucto, ad the arameters of CLA such as the umber of cells (oulato sze), toology ad the tye of the learg automata for each cell. Evolutoary algorthms as the oe descrbed algorthm ths aer ca be used ay arbtrary fte dscrete search sace. To smlfy the algorthm, we assume that sght search sace s a bary fte search sace. So the otmzato roblem ca be reseted as follows. Assume f:{0,1} m R be a real fucto that s to be mmzed. I order to use CLA- EC for the otmzato fucto f frst a set of learg automata s assocated to each cell of CLA-EC. The umber of learg automata assocated to a cell of CLA-EC s the umber bts the strg geome reresetg ots of the search sace of f. Each automato has two actos called acto 0 ad 1. The the followg stes wll be reeated utl a termato crtero s met. 1- Every automata a cell chooses oe of ts actos usg ts acto robablty vector 2- Cell geerates a ew strg geome, ew, by combg the actos chose by the learg automata of cell. The ewly geerated strg geome s obtaed by cocateatg the actos of the automata (0 or 1) assocated to that cell. Ths secto of algorthm s equvalet to learg from revous self-exereces. 3- Every cell comutes the ftess value of strg geome ew ; f the ftess of ths strg geome s better tha the oe the cell the the ew strg geome ew becomes the strg geome of that cell. That s ξ f ( ξ ) f ( ew + ) 1 ξ + 1 = (3) ew+ 1 f ( ξ ) > f ( ew+ 1) 4- Se cells of the eghborg cells of the cell are selected. Ths Selecto s based o the ftess value of the eghborg cells accordg to trucato strategy. Ths rocess s equvalet to matg the ature. Note that matg the cotext of roosed algorthm s ot recrocal,.e., a cell selects aother cell for matg but ecessarly vse versa. 5- Based o selected eghborg cells a reforcemet vector s geerated. Ths vector becomes the ut for the set of learg automata assocated to the cell. Ths secto of algorthm s equvalet to learg from exereces of others. Let N s () be set selected eghbors of cell. Defe, l, N ( k) = δ ( ξ = ), (4) k, l N ( ) s Where, 1 ex s true δ (ex) = 0 otherwse (5) β,, the reforcemet sgal gve to learg automato of cell, s comuted as follows,, u( N (1) (0) ) = 0,, N, f ξ β =, u( N, (0) N, (1)) f ξ = 1 (6) Where u(.) s a ste fucto. The overall oerato of CLA- EC s summarzed the algorthm of fgure of 3. Italze. Whle ot doe do For each cell CLA do arallel Geerate a ew strg geome Evaluate the ew strg geome If f(ew strg geome)> f(old strg geome) the Accet the ew strg geome Ed f Select Se cells from eghbors of cell Geerate the reforcemet sgal vector Udate LAs of cell Ed arallel for Ed whle Fg. 3 Pseudocode of CLA-EC 4. SIMULATION RESULTS Ths secto resets smulato results for fve fucto otmzato roblems ad the comarso of these results wth the results obtaed usg Smle Geetc Algorthm (SGA), terms of soluto qualty, ad the umber of fucto evaluatos take by the algorthm to coverge comletely for a gve oulato sze. The CLA-EC used for the exermets has a lear toology wth wra aroud coecto as show fgure 4a. The eghbors of cell are cell -1 ad cell +1. The archtecture of each cell s show fgure 4b. Each cell s equed wth m learg automata. The strg geome determer comares the ew strg geome wth the strg geome resdg the cell. The strg wth the hgher qualty relaces the strg geome of the cell. Deedg o the eghborg strg geomes ad the strg geome of the cell, a reforcemet sgal wll be geerated by the sgal geerator. 435

4 Fg. 4 a) A oe-dmesoal (lear) cellular automato wth eghborhood radus oe (r=1) ad wra aroud coecto ad q cells. b) The structure of a cell Each quatty of the results reorted s the average take over 20 rus. The oulato sze (umber of cells CLA- EC) vares from 3 to 49 wth cremets of two. The roosed algorthm s tested for learg algorthm L RP. For the sake of coveece resetato, we use CLA- EC(automata(a,b),r,se,q) to refer to the CLA-EC algorthm wth q cells, eghborhood radus r, the umber of selected cell Se whe usg learg automata automata wth reward arameter a ad ealty arameter b. The algorthm termates whe all learg automata coverge comletely. The roosed algorthms are tested fve dfferet stadard fucto mmzato roblems. These fuctos that are gve below are borrowed from referece [17]. F ( X) 3 2 F 1( X ) = = x x ( x x ) + (1 x ) x 2 = F 5 F ( X ) = = teger( x ) x ( X ) = = x + (0,1) Gauss x A smle geetc algorthm [18] that uses two-touramet selecto wthout relacemet ad uform crossover wth exchage robablty 0.5 s used our exermets. Mutato s ot used ad crossover s aled wth robablty oe. I ths aer globally covergece s cosdered as termato codto for smle geetc algorthm. For fuctos F 1, F 2, F 4, F 5, we set Se to 2 because smulatos have show that value 2 for arameter Se s the most arorate value for these fuctos. For F 3 fucto we set Se to 3. The results of comarsos are reorted fgures 5 trough 9, whch show the suerorty of the roosed algorthm. Fg. 5 CLA-EC(LRP(0.01,0.01),1, -,5) ad Smle GA for fucto F 1 a) Obectve value b) fucto evaluatos 346

5 Fg. 6 CLA-EC(LRP(0.01,0.01),1, -,5) ad Smle GA for fucto F 2 a) Obectve value b) fucto evaluatos Fg. 7 CLA-EC(LRP(0.01,0.01),1, -,5) ad Smle GA for fucto F 3 a) Obectve value b) fucto evaluatos Fg. 8 CLA-EC(LRP(0.01,0.01),1, -,5) ad Smle GA for fucto F 4 a) Obectve value b) fucto evaluato 437

6 Fg. 9 CLA-EC(LRP(0.01,0.01),1, -,5) ad Smle GA for fucto F 5 a) Obectve value b) fucto evaluatos 5. CONCLUSION I ths aer, the Cellular Learg Automata model s exteded by combg wth Evolutoary Comutg Model ad a ew evolutoary model called CLA-EC roosed. The CLA-EC has a umber of roertes that make t sueror over other evolutoary models. A hghly degree of dversty s aaret the early geeratos created by havg the robabltes tally radom ad oly slghtly based the early terato. I other had wth resect to the fact that teractos betwee cells (geomes) are local the robablty of stuck local otma ca be decreased. REFERENCE [1] Hark, G., Learg Lkage to Effcetly Solve Problems of Bouded Dffculty Usg Geetc Algorthms, Illos Geetc Algorthm Reort, No , Illos Uversty, Illos, USA, [2] Smth, J., ad Fogarty, T. C., Self Adatato of Mutato Rates a Steady State Geetc Algorthm, I Proc. 3 rd IEEE Cof. o Evolutoary Com. IEEE Press, [3] Balua, S., Caruaa, R., "Removg The Geetcs from The Stadard Geetc Algorthm", I Proceedgs of ICML 95, PP , Morga Kaufma Publshers, Palo Alto, CA, [4] Balua, S., ad Daves, S., Usg Otmal Deedecy Trees for Combatoral Otmzato: Learg the Structure of Search Sace, Techcal Reort CMU-CS , Carege Mello Uversty, Pttsburgh, Pesylvaa, [5] Mühlebe, H., ad Pelka, M., The Bvarate Margal Dstrbuto Algorthm, Advaces Soft Comutg-Egeerg Desg ad Maufacturg, PP , [6] Meybod, M. R., Beyg, H., ad Taherkha, M., Cellular Learg Automata, Proceedgs of 6 th Aual Iteratoal Comuter Socety of Ira Comuter Coferece CSICC2001, Isfaha, Ira, PP , [7] Begy, H., ad Meybod, M. R., A Mathematcal Framework for Cellular Learg Automata, Advaced Comlex Systems, to aear. [8] Meybod, M. R., ad Mehdour, F., VLSI Placemet Usg Cellular Learg Automata, Proceedgs of 8 th Aual Iteratoal Comuter Socety of Ira Comuter Coferece CSICC2001, Mashhad, Ira, PP , [9] Meybod, M. R., ad Taherkha, M., Alcato of Cellular Learg Automata to Modelg of Rumor Dffuso, Proceedgs of 9th Coferece o Electrcal Egeerg, Power ad Water sttute of Techology, Tehra, Ira, PP , May [10] Begy, H., ad Meybod, M. R., A Self-Orgazg Chael Assgmet Algorthm: A Cellular Leag Automata Aroach, Vol of Srger- Verlag Lecture Notes Comuter Scece, PP , Srger-Verlag, [11] Baradarahashem, A, Begy, H., ad Meybod, M. R., "Dyamc Call Access Cotrol for Cellular Moble Networks, Proceedgs of 9th Aual CSI Comuter Coferece, Comuter Egeerg Deartmet, Sharf Uversty, Tehra, Ira, , Feb [12] Khoasteh, M. R. ad Meybod, M. R. Cooerato Mult-Aget Systems Usg Learg Automata", Iraa Joural of Electrcal ad Comuter Egeerg, Vol. 1, No. 2,.81-91, [13] Thathachar, M. A. L., Sastry, P. S., Varetes of Learg Automata: A Overvew, IEEE Trasacto o Systems, Ma, ad Cyberetcs-Part B: Cyberetcs, Vol. 32, No. 6, PP , [14] Naredra, K. S., ad Thathachar, M. A. L., Learg Automata: A Itroducto, Prtce-Hall Ic, [15] Wolfram, S., Cellular Automata ad Comlexty, Perseus Books Grou, [16] Alba, E., ad Troya, J. M., Aalyzg Sychroous ad Asychroous Parallel Dstrbuted Geetc Algorthms, Future Geerato Comuter Systems, Vol. 17, PP , [17] De Jog, K. A., The Aalyss of the behavor of a class of geetc adatve systems Ph.D. dssertato, Uversty of Mchga, A Arbor, [18] Goldberg, D. E., Geetc Algorthms Search, Otmzato ad Mache Learg, Addso-Wesley, New York,