Parameter Tuning of Evolutionary Algorithms: Generalist vs. Specialist

Transcription

1 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist S.K. Smit ad A.E. Eibe Vrije Uiversiteit Amsterdam The Netherlads Abstract. Fidig appropriate parameter values for Evolutioary Algorithms (EAs) is oe of the persistet challeges of Evolutioary Computig. I recet publicatios we showed how the REVAC (Relevace Estimatio ad VAlue Calibratio) method is capable to fid good EA parameter values for sigle problems. Here we demostrate that REVAC ca also tue a EA to a set of problems (a whole test suite). Hereby we obtai robust, rather tha problem-tailored, parameter values ad a EA that is a geeralist, rather tha a specialist. The optimized parameter values prove to be differet from problem to problem ad also differet from the values of the geeralist. Furthermore, we compare the robust parameter values optimized by REVAC with the supposedly robust covetioal values ad see great differeces. This suggests that traditioal settigs might be far from optimal, eve if they are meat to be robust. Key words: parameter tuig, algorithm desig, test suites, robustess 1 Backgroud ad Objectives Fidig appropriate parameter values for evolutioary algorithms (EA) is oe of the persistig grad challeges of the evolutioary computig (EC) field. As explaied by Eibe et al. i [8] this challege ca be addressed before the ru of the give EA (parameter tuig) or durig the ru (parameter cotrol). I this paper we focus o parameter tuig, that is, we are seekig good parameter values off-lie ad use these values for the whole EA ru. I today s practice, this tuig problem is usually solved by covetios (mutatio rate should be low), ad hoc choices (why ot use uiform crossover), ad experimetal comparisos o a limited scale (testig combiatios of three differet crossover rates ad three differet mutatio rates). Util recetly, there were ot may workable alteratives. However, by the developmets over last couple of years ow there are a umber of tuig methods ad correspodig software packages that eable EA practitioers to perform tuig without much effort. I particular, REVAC [10, 13] ad SPOT [3, 5, 4] are well developed ad documeted. The mai objective of this paper is to illustrate the advatage of usig tuig algorithms i terms of improved EA performace. To this ed, we will select a set

2 2 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist Table 1 problem solvig parameter tuig Method at work evolutioary algorithm tuig procedure Search space solutio vectors parameter vectors Quality fitess utility Assessmet evaluatio testig of test fuctios ad compare a bechmark EA (with robust parameter values set by commo wisdom ) with a EA whose parameters are tued for this set of fuctios. A secod objective is to compare specialist EAs (that are tued o oe of our test fuctios) with a geeralist EA (that is tued o the whole set of test fuctios). For this compariso we will look at the performace of the EAs as well as the tued parameter values. Furthermore, we wat to show what kid of problems rise whe tuig evolutioary algorithms. At the ed we hope to provide a covicig showcase justifyig the use of a tuig algorithm ad to obtai ovel isights regardig the good parameter values. 2 Parameters, Tuers, ad Utility Ladscapes To obtai a detailed view o parameter tuig we distiguish three layers: the applicatio layer, the algorithm layer, ad the desig or tuig layer. The lower part of this three-tier hierarchy cosists of a problem o the applicatio layer (e.g., the travelig salesma problem) ad a EA (e.g., a geetic algorithm) o the algorithm layer tryig to fid a optimal solutio for this problem. Simply put, the EA is iteratively geeratig cadidate solutios (e.g., permutatios of city ames) seekig oe with maximal fitess. The upper part of the hierarchy cotais a tuig method that is tryig to fid optimal parameter values for the EA o the algorithm layer. Similarly to the lower part, the tuig method is iteratively geeratig parameter vectors seekig oe with maximal quality, where the quality of a give parameter vector is based o the performace of the EA usig the values of it. To avoid cofusio we use the term utility, rather tha fitess, to deote the quality of parameter vectors. Table 1 provides a quick overview of the related vocabulary. Usig this omeclature we ca defie the utility ladscape as a abstract ladscape where the locatios are the parameter vectors of a EA ad the height reflects utility, based o ay appropriate otio of EA performace. It is obvious that fitess ladscapes commoly used i EC have a lot i commo with utility ladscapes as itroduced here. To be specific, i both cases we have a search space (cadidate solutios vs. parameter vectors), a quality measure (fitess vs. utility) that is coceptualized as height, ad a method to assess the quality of a poit i the search space (evaluatio vs. testig). Fially, we have a search method (a evolutioary algorithm vs. a tuig procedure) that is seekig for a poit with maximum height.

3 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist 3 3 Geeralist EAs vs. Specialist EAs Studyig algorithm performace o differet problems has led to the o-freeluch theorem statig that algorithms (of a certai geeric type) performig well o oe type of problem, will perform worse o aother [14]. This also holds for parameter values i the sese that a parameter vector that performs good o oe type of problems, is likely to perform worse o aother. However, very little effort is spet o studyig the relatio betwee problem characteristics ad optimal parameter values. It might be argued that this situatio is ot a matter of igorace, but a cosequece of a attitude favorig robust parameter values that perform good o a wide rage of differet problems. Note that the term robust is also used i the literature to idicate a low variace i outcomes whe performig multiple repetitios of a ru o the same problem with differet radom seeds. To avoid cofusio, we will use the term geeralist to deote parameter values that perform good o a wide rage of problems. The opposite of such a geeralist is the a specialist, amely a parameter set that shows excellet performace o oe specific type of problems. The otio of a geeralist raises a umber of issues. First, a true geeralist would perform good o all possible test fuctios. However, this is impossible by the o-free-luch theorem. So, i practice, oe eeds to restrict the set of test fuctios a geeralist must solve well ad formulate the claims accordigly. For istace, a specific test suite {F 1,..., F } ca be used to support such claims. The secod problem is related to the defiitio of utility. I simplest case, the utility of a parameter vector #» p is the performace of the EA usig the values of #» p o a give test fuctio F. This otio is sufficiet to fid specialists for F. However, for geeralists, a collectio of fuctios {F 1,..., F } should be used. This meas that the utility is ot a sigle umber, but a vector of utilities correspodig to each of the test fuctios. Hece, fidig a good geeralist is a multi-objective problem, for which each test-fuctio is oe objective. I this ivestigatio we address this issue i a straightforward way, by defiig the utility o a set {F 1,..., F } as the average of utilities o the fuctios F i. 4 Experimetal Setup ad System Descriptio As described earlier, the experimetal setup cosist of a three layer architecture. O the applicatio layer, we have chose a widely used set of 10 dimesioal test-fuctios to be solved, amely: Ackley, Griewak, Sphere, Rastrigi, ad Rosebrock. For Ackley, Griewak ad Rosebrock, the Evolutioary Algorithm is allowed for fuctio evaluatios. O Rastrigi it is allowed for evaluatios, ad o the Sphere fuctio oly This is a rather limited set of problems, but due to a large rutime more exhaustive ad complex test suites are ot yet feasible. O the algorithm layer, we have chose a simple geetic algorithm usig N-poit crossover, bitflip mutatio, k-touramet paret selectio, ad determiistic survivor selectio. These choices require 6 parameters to be defied as

4 4 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist Table 2: Parameters to be tued, ad their rages Parameter Mi Max Populatio size Offsprig size Mutatio probability 0 1 # crossover poits Crossover probability 0 1 Touramet size described i Table 2. the allowed values for most of the parameters a defie by either the ulatio size or geome legth (150). For ulatio size, we have chose a maximum value of 200, which we believe is big eough for this geome size ad allowed umber of evaluatios. The offsprig size determies the umber of idividuals that are bor every geeratio. These ewbor idividual replace the worst idividuals i the ulatio. If the offsprig size is bigger tha the ulatio size, the the whole ulatio is replaced by the ew group of idividuals, causig a icrease i ulatio size. N-poit crossover is chose to allow for a whole rage of crossover operators, such as the commoly used 1-poit crossover (N=1) ad 2-poit crossover (N=2). The same holds for k-touramet selectio. The commoly used radom uiform selectio (k=1), determiistic selectio (k ulatio-size) ad everythig i betwee ca be used by meas of selectig k accordigly. Because the test-fuctios require 10 dimesioal real-valued strigs as iput, a 15-bit Gray codig is used to trasform the biary strig of legth 150, ito a real-valued strig of legth 10. O the desig layer, REVAC [9] is used for tuig the parameters of the Evolutioary Algorithm. Techically, REVAC is a heuristic geerate-ad-test method that is iteratively searchig for the set of parameter vectors of a give EA with a maximum performace. I each iteratio a ew parameter vector is geerated ad its performace is tested. Testig a parameter vector is doe by executig the EA with the give parameter values ad measurig the EA performace. EA performace ca be defied by ay appropriate performace measure ad the results will reflect the utility of the parameter vector i questio. Because of the stochastic ature of EAs, i geeral a umber of rus is advisable to obtai better statistics. A detailed explaatio of REVAC ca be foud i [13] ad [11]. REVAC itself has some parameters too, which eed to be specified. The REVAC-parameter values used i these experimets are the default settigs, ad ca be foud i Table Huma Expert Tuig a algorithm requires a lot of computer power, while some people argue that this is a waste of time. Geeral rules of thumb as a ulatio size of

5 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist 5 Table 3: REVAC Parameters Populatio Size 80 Best Size 40 Smoothig coefficiet 10 Repetitios per vector 10 Maximum umber of vectors tested ad low mutatio probabilities are supposed to perform reasoably well. The questio rises how beeficial tuig, ad more specific automated tuig, is eve to experieced practitioers. For quick assessmet of the added value of algorithmic tuig, we have tested the EA usig parameter values defied by commo wisdom (Table 4). 4.2 Performace Measures The quality of a certai parameter vector, is measured two times. First, the estimated utility is used to asses a performace to the parameter-vector durig the tuig procedure. This estimated utility is calculated by takig the Mea Best Fitess of 10 idepedet rus usig these parameter-values. Secodly, after the tuig procedure is fiished, a validated utility is calculated for the 5 parameter vectors with the best estimated utility. This validated utility is based o 25 idepedet rus istead of 10, ad is therefore supposed to be a better estimate of the true utility of the parameter vector. After this validatio step, we defie the best parameter values as the parameter vector with the highest validated utility. Furthermore, we tried to idicate good rages for each of the parameters. Such a rage is defied by takig the value of.25 ad.75 quatile of the 25 parameter vectors with the highest estimated utility. This removes outliers that are caused by parameters vectors that were lucky, ad received a estimated utility that is much higher tha their true utility. 4.3 System Descriptio The complete experimet is defied i MOBAT[12] (Meta-Heuristic Optimizer Bechmark ad Aalysis Toolbox), a toolbox for defiig, tuig ad evaluatig Evolutioary Algorithms o a distributed system. The default package of MO- BAT cotais all the compoets for composig the evolutioary algorithm used i these experimets, the test-fuctios ad REVAC. MOBAT is ope source ad freely available via SourceForge.et. The experimets are ra o a sigle 2.93 GHz Itel Core 2 Duo processor with 4GB of memory. A specialist tuig-sessio took o average 8 hours to fiish, while the geeralist experimet o our testsuite of 5 test fuctios fiished i 40 hours.

6 6 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist 5 Results 5.1 Performace I this sectio we preset the outcomes of our experimets. As explaied i sectio 3, our goal is to fid the best parameter vectors for both specialists ad geeralists. Furthermore, we stated that a good geeralist, is the oe with the best average performace. From the results i Table 4 we ca immediately see the effect of this choice. The Rosebrock fuctio appeared to be much harder to solve tha the other fuctios, causig big differeces i utility. Therefore the best geeralist, was the oe that performed very well o the Rosebrock fuctio, without losig too much performace o the other fuctios. From Table 5, it is eve clear that there is o sigificat differece i multi-fuctio performace betwee the three best performig istaces o the Rosebrock fuctio (based o a t-test with α = 0.1). Furthermore, we ca observe that the geeralist is oly outperformed by a specialist o the Sphere fuctio. However, focusig more o the sphere fuctio makes hardly ay differece i the average performace, due to the small fuctio values o this problem. The parameter values chose by commo wisdom are, except o the Sphere fuctio, sigificatly outperformed by the other parameter vectors. Whe lookig at the specialists, we ca observe some iterestig pheomea. It is apparetly very easy to tue parameters i such a way that they are purely specialized o the Sphere fuctio. This specialist is the oly oe that solves its problem perfectly, but o the dowside, it performs very bad o the others fuctios. The Ackley specialist, o the other had, does ot oly perform best o its ow fuctio, but also outperforms most others o the Rastrigi fuctio. Iterestigly, the Rosebrock ad Griewak specialists show very similar behavior o all fuctios, however it is remarkable that the Griewak specialist has oly a average performace o the fuctio it is tued to. Estimated Utility vs Validated Utility: Oe of the causes of such suboptimal performace o its ow fuctio, is a differece betwee the estimated utility, that is used durig tuig, ad the validated utility, as show i the results. I some cases, these validated utilities are twice as bad as the estimated utility. These differeces ca be explaied by lookig at the fitess ladscapes of the fuctios. I most cases, a fitess ladscape cosists of multiple local optima ad a global optimum, with certai bases of attractio. It is likely that a evolutioary algorithm will termiate with a value that is (close to) a local or global optimum, because it will cotiue to climb the hill it is curretly o, usig small mutatios util it gets stuck. The utility vector, therefore cosists of values that are all close to a optima. For example, the utility vector of the commo wisdom parameter values o the Ackley fuctio has 58% of the values close to the global optimum. 35 % of the values is betwee 3 ad 4, which are exactly the values o the first rig of local optima surroudig the global optimum. Fially 7 % of the fitess values at termiatio, is betwee 5 ad 7, which is equal to value of the secod

7 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist 7 Table 4: Best Parameter Values ad their Mea Best Fitess (to be miimized). Stadard deviatios show withi brackets. Commo Geeralist Specialist Ackley Griewak Sphere Rastrigi Rosebrock Pop. size Offsprig size Mutatio prob N-poit crossover Crossover prob Touramet size Ackley (1.747) (0.539) (0.013) (0.542) (2.312) (0.753) (0.782) Griewak (0.038) (0.023) (0.040) (0.032) (0.056) (0.037) (0.030) Sphere (0.028) (0.015) (0.056) 0.01 (0.012) (0.000) (0.003) (0.022) Rastrigi (10.48) 6.92 (4.70) 7.28 (3.65) (5.97) (11.71) 7.60 (4.99) 9.85 (4.91) Rosebrock (395.0) 64.2 (110.4) (129.4) 68.4 (126.5) (229.8) (195.2) 62.5 (123.4) Average Table 5: The specialists/geeralist (colums) that show sigificatly better performace (α = 0.1) o a certai problem(rows), based o the results from Table 4 Commo Geeralist Specialist Ackley Griewak Sphere Rastrigi Rosebrock (1) (2) (3) (4) (5) (6) (7) Ackley 1,5 1,5,6,7 1,5 1,5 1,5 Griewak All Sphere 2,3,7 3,7 2,3,7 All 1,2,3,4,7 3 Rastrigi 1,4,5,7 1,4,5,7 1,5 1,4,5,7 1,5 Rosebrock 1,3,5 1,3,5 1,3,5 Average 1,3,5 1,3,5 1,3,5 rig of local optima. Such a distributio ca disturb the tuig ru heavily, for example i 7.5% of the cases the estimated utility, based o 10 rus, of this set of parameter values will be lower tha 0.2, which is ie times lower tha the true utility. Such a lucky oe ca therefore steer evolutio ito the wrog directio. 5.2 Best Parameter Values To get better isight ito the best parameter rages, we have chose ot to use the sigle best solutio, but the 1% of the best performig parameters values durig the tuig phase. Figure 1 shows the.25 ad.75 quatile of those values for each of the specialists ad the geeralist relative to the parameter rages. The most obvious coclusio that ca be draw from these charts, is that mutatio should be low, i order to get a good performace. However, there are also some less stadard coclusios. Oe of the most iterestig results is that, although

8 8 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist (a) Geeralist (b) Ackley (c) Griewak (d) Sphere (e) Rastrigi (f) Rosebrock Fig. 1: The good parameter rages for each of the parameters o each test fuctio, ad the combied set. The parameter rages from Table 2 are scaled to [0, r] the rages for ulatio-size are quite differet for each of the fuctios, the guesstimate of ulatio-size equals 100 is ot that bad after all. To be more specific, o all five problems, a ulatio-size of 100 lies withi the.25 ad.75 quatile. However, most other parameter values are completely differet tha the commo wisdom oes. For example the N parameter of N-Poit crossover is much higher tha 2, which idicated that uiform crossover like crossover, outperforms commo 1 or 2-poit crossover o these problems. This is probably due to the separable (Sphere, Rastrigi ad Ackley) or partially separable ature of the test fuctios. Furthermore, we ca observe a much higher selectio pressure tha ormally used. Touramet sizes are almost equal to the ulatio size, causig the evolutioary algorithm to rapidly coverge towards the best idividuals i the ulatio. However, such behavior ca be explaied by the limited umber of evaluatios that the evolutioary algorithm was allowed to perform. The questio rises, if such a fast-covergig algorithm is always preferred over a slow-but-accurate istace. I some cases it is, while i other cases it might ot be the preferred behavior. Therefore, we emphasize that the best parameter values preseted here are highly related to the choices that are made i the experimetal setup.

9 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist 9 6 Coclusios ad Outlook I this paper we have show that REVAC is ot oly capable to fid good EA parameter values for a sigle problem (test fuctio), but also for a set of problems (test fuctios). The parameter values we foud for our geeralist, differ greatly from the commo wisdom values that are supposed to be robust. The optimal selectio pressure for these problems, appears to be much higher tha commoly used. Furthermore, a may-poit crossover operator outperforms the commoly used 1 -ad 2-poit crossover o all five problems. O the other had, a ulatio-size of 100 tured out to be ot that bad after all. The scope of these coclusios is limited, we do ot advocate them as beig the ew best geeral parameters, because differet test-suites, aggregatio fuctios ad performace measures will lead to differet optimal parameters. The best geeralist will therefore always deped upo the choices that are made to defie it. Based o the defiitio of geeralist i this paper, our geeralist performed quite good o most problems. However, the results o the Sphere fuctio cofirm that the o-free-luch theorem also holds o a parameter vector level. Furthermore, the experimets revealed some major issues for parameter tuig i geeral. Estimatig the utility has a key role i parameter tuig, both for specialists ad geeralists. Our experimets revealed how the umber of rus per vector ca ifluece the outcome of the experimets. Too may rus lead to high computatioal costs, while too few lead to a iaccurate estimated utility ad therefore iaccurate results. Therefore we advocate the use of racig [2, 6, 15, 13] ad sharpeig [4, 13] to deal with this issue. This, o the oe had sharpes the estimate of the utility, ad o the other had reduces the computatioal effort. Tuig for a geeralist raises specific problems. I geeral, it is ot clear how a good geeralist should be defied. I the area of multi-objective optimizatio several approaches are kow. Oe approach is to use aggregatio methods, like the simple average that we used here. From the results we ca observe the effect of such choices; it is more effective to focus o the hard problems that ca lead to high deviatio i the average utility, rather tha searchig for a geeralist that performs good o all fuctios. Whe defiig tuig sessios, oe have to be aware of the fact that a tuer will optimize o the problem that is defied, rather tha the problem they wished to be solved. Future work ca overcome this issue, by usig a approach kow from multi-objective optimizatio, amely searchig for the Pareto frot. Rather tha aggregatig the results based o choices made beforehad, such a approach allows the researcher to study the effect of a certai desig choice afterwards. I [7] such a approach is used to show which parameter values are optimal, whe comparig o both algorithm speed ad algorithm accuracy at the same time, without specifyig weights for those two objective beforehad. This approach ca easily be exteded to multi-fuctio optimizatio, which ca give isight ito the the rages of parameter-values that are most efficiet o a certai problem, a class of problems, or o a whole test suite. By meas of oe extesive ru, oe ca idetify specialists, class-specialist, or a true geeralist without defiig those terms beforehad. Based o such

10 10 Parameter Tuig of Evolutioary Algorithms: Geeralist vs. Specialist studies, oe o loger has to rely o commo wisdom i order to choose their parameter values wisely but ca select oe that fits their eeds. Refereces 1. Proceedigs of the 2009 IEEE Cogress o Evolutioary Computatio, Trodheim, May IEEE Press. 2. Prasaa Balaprakash, Mauro Birattari, ad Thomas Stützle. Improvemet strategies for the f-race algorithm: Samplig desig ad iterative refiemet. I Hybrid Metaheuristics, pages , T. Bartz-Beielstei, C.W.G. Lasarczyk, ad M. Preuss. Sequetial parameter optimizatio. I David Core et al., editors, Proceedigs of the 2005 IEEE Cogress o Evolutioary Computatio IEEE Cogress o Evolutioary Computatio, volume 1, pages Vol.1, Ediburgh, UK, Sept IEEE Press. 4. T. Bartz-Beielstei, K.E. Parsopoulos, ad M.N. Vrahatis. Aalysis of Particle Swarm Optimizatio Usig Computatioal Statistics. I Chalkis, editor, Proceedigs of the Iteratioal Coferece of Numerical Aalysis ad Applied Mathematics (ICNAAM 2004), pages 34 37, Thomas Bartz-Beielstei ad Sador Marko. Tuig search algorithms for realworld applicatios: A regressio tree based approach. Techical Report of the Collaborative Research Cetre 531 Computatioal Itelligece CI-172/04, Uiversity of Dortmud, March M. Birattari, T. Stützle, L. Paquete, ad K. Varretrapp. A racig algorithm for cofigurig metaheuristics. I W.B. Lagdo, editor, GECCO 2002: Proceedigs of the Geetic ad Evolutioary Computatio Coferece, pages 11 18, Sa Fracisco CA, Morga Kaufma. 7. Joha Dréo. Usig performace frots for parameter settig of stochastic metaheuristics. I Fraz Rothlauf, editor, Proceedigs of the Geetic ad Evolutioary Computatio Coferece (GECCO-2009), pages ACM, A.E. Eibe, R. Hiterdig, ad Z. Michalewicz. Parameter Cotrol i Evolutioary Algorithms. IEEE Trasactios o Evolutioary Computatio, 3(2): , V. Nae ad A. E. Eibe. Relevace Estimatio ad Value Calibratio of Evolutioary Algorithm Parameters. I Mauela M. Veloso, editor, Proceedigs of the 20th Iteratioal Joit Coferece o Artificial Itelligece (IJCAI), pages , V. Nae ad A.E. Eibe. A Method for Parameter Calibratio ad Relevace Estimatio i Evolutioary Algorithms. I M. Keijzer, editor, Proceedigs of the Geetic ad Evolutioary Computatio Coferece (GECCO-2006), pages Morga Kaufma, Sa Fracisco, V. Nae, S.K. Smit, ad A.E. Eibe. Costs ad beefits of tuig parameters of evolutioary algorithms. I Güter Rudolph et al., editors, PPSN, volume 5199 of Lecture Notes i Computer Sciece, pages Spriger, S.K. Smit. MOBAT S.K. Smit ad A.E. Eibe. Comparig parameter tuig methods for evolutioary algorithms. I [1]. 14. David H. Wolpert ad William G. Macready. No free luch theorems for optimizatio. IEEE Trasactio o Evolutioary Computatio, 1(1):67 82, B. Yua ad M. Gallagher. Combiig Meta-EAs ad Racig for Difficult EA Parameter Tuig Tasks. I F.G. Lobo, C.F. Lima, ad Z. Michalewicz, editors, Parameter Settig i Evolutioary Algorithms, pages Spriger, 2007.