The aims of this tutorial

Size: px

Start display at page:

Download "The aims of this tutorial"

Barnaby Peter Townsend
5 years ago
Views:

1 A.E. Eiben Free University Amsterdam (google: eiben) The aims of this tutorial Creating awareness of the tuning issue Providing guidelines for tuning EAs Presenting a vision for future development CEC 2009 tutorial 1

2 By the end of this talk You will certainly NOT have: Simple answers like Crossover rate should be 0.8 Clear recipes like Method X can tune your EA perfectly You will hopefully have: A new look on EA parameter calibration Motivation to adjust your practice Inspiration to do research on EA parameter calibration The basic EC metaphor EVOLUTION Environment Individual Fitness PROBLEM SOLVING Problem Candidate Solution Quality Fitness chances for survival and reproduction Quality chance for seeding new solutions CEC 2009 tutorial 2

3 The two main pillars of EAs There are two competing forces active Increasing population diversity by genetic operators mutation recombination Decreasing population diversity by selection of parents of survivors Push towards novelty Push towards quality Proper balance of these forces is essential This is regulated through the parameters Algorithm design and parameters Calibration of EAs Design of EAs Configuration of EAs Parameter optimization of EAs Given: Required: an algorithmic framework instantiation to a specific algorithm with good quality CEC 2009 tutorial 3

4 Problems How to find good parameter values? Many parameters, unknown effects, unknown, nonlinear interactions How to vary parameter values? EA is a dynamic, staged, process optimal parameter values may vary during a run Brief historical account 1970/80ies GA is a robust method 1970ies+ ESs self adapt mutation stepsize σ 1986 meta GA for optimizing GA parameters 1990ies EP adopts self adaptation of σ as standard 1990ies some papers on changing parameters on the fly 1999 Eiben Michalewicz Hinterding paper proposes cleartaxonomy & terminology CEC 2009 tutorial 4

5 Taxonomy PARAMETER CALIBRATION PARAMETER TUNING (before the run) PARAMETER CONTROL (during the run) DETERMINISTIC ADAPTIVE SELF-ADAPTIVE (time dependent) (feedback from search) (coded in chromosomes) Google Scholar index > 550 (>700 with the conference version) Parameter tuning Parameter tuning: testing and comparing different values before the real run Problems: users mistakes in settings can be sources of errors or sub-optimal performance costs much time parameters interact: exhaustive search is not practicable good values may become bad during the run CEC 2009 tutorial 5

6 Parameter control Parameter control: setting values on-line, during the actual run, e.g. predetermined time-varying schedule p = p(t) using (heuristic) feedback from the search process encoding parameters in chromosomes and rely on natural selection Problems: finding optimal p is hard, finding optimal p(t) is harder still user-defined feedback mechanism, how to optimize? when would natural selection work for algorithm parameters? Example Task to solve: min f(x 1,,x, n ) L i x i U i for i = 1,,n bounds g i (x) 0 for i = 1,,q inequality constraints h i (x) = 0 for i = q+1,,m equality constraints Algorithm: EA with real-valued l representation ti (x 1,,x n ) arithmetic averaging crossover Gaussian mutation: x i = x i + N(0, σ) standard deviation σ is called mutation step size CEC 2009 tutorial 6

7 Varying mutation step size: option 1 Replace the constant σ by a function σ(t) σ (t) = t T is the current generation number Features: changes in σ are independent from the search progress strong user control of σ by the above formula σ is fully predictable a given σ acts on all individuals of the population t T Varying mutation step size: option 2 Replace the constant σ by a function σ(t). The value is updated after every n steps by Rechenberg s 1/5 success rule σ (t t n ) / c if ps >1/5 σ ( t) = σ ( t n) c if ps < 1/5 σ ( t n) otherwise where p s is the % of successful mutations, c is a parameter (0.8 < c < 1) Features: changes in σ are based on feedback from the search progress some user control of σ by the above formula σ is not predictable a given σ acts on all individuals of the population CEC 2009 tutorial 7

8 Varying mutation step size: option 3 Assign a personal σ to each individual Incorporate this σ into the chromosome: (x 1,, x n, σ) Apply variation operators to x i s and σ Features: N ( 0, τ) σ = σ e x i = xi + N ( 0, σ ) changes in σ are results of natural selection (almost) no user control of σ σ is not predictable a given σ acts on one individual Varying mutation step size: option 4 Assign a personal σ to each variable in each individual Incorporate σ s into the chromosomes: (x 1,, x n, σ 1,, σ n ) Apply variation operators to x i s and σ i s Features: N ( 0, τ) σ i = σi e x i = xi + N ( 0, σ i) changes in σ i are results of natural selection (almost) no user control of σ i σ i is not predictable a given σ i acts on one variable of one individual CEC 2009 tutorial 8

9 Control WHAT? Practically any EA component can be parameterized and thus controlled on-the-fly: representation evaluation function variation operators selection operator (parent or mating selection) replacement operator (survival or environmental selection) population (overlap, size, topology) Control HOW? Three major types of parameter control: deterministic: some rule modifies strategy parameter without feedback from the search (based on some counter, typically time or no of search steps) adaptive: feedback rule, i.e., heuristic, based on some measure monitoring search progress self-adaptative: parameter values evolve along with solutions; encoded onto chromosomes they undergo variation and selection CEC 2009 tutorial 9

10 Notes on parameter control Parameter control offers the possibility to use appropriate values in various stages of the search Adaptive and self adaptive control can liberate users from tuning reduces need for EA expertise for a new application Assumption: control heuristic is less parameter sensitive than the EA BUT State of the art is a mess: literature is a potpourri, no generic knowledge, no principled approaches to developing control heuristics (deterministic or adaptive), no solid testing methodology WHAT ABOUT TUNING? Historical account (cont d) Last decade: More & more work on parameter control Traditional parameters: mutation and xover Non traditional parameters: selection and population size All parameters parameterless EAs (name!?) Hardly any work on parameter tuning, i.e., Nobody reports on tuning efforts behind their EA published Just a handful papers on tuning methods / algorithms CEC 2009 tutorial 10

11 Control flow of EA calibration / design User Design layer Meta-GA optimizes GP Algorithm layer GA optimizes Symbolic regression Application layer One-max Information flow of EA calibration / design Design layer Algorithm quality Algorithm layer Solution quality Application layer CEC 2009 tutorial 11

12 Lower level of EA calibration / design EA Searches Evaluates Decision variables Problem parameters Candidate solutions Application Space of solution vectors Lower level of EA calibration / design Searches EA Evaluates The whole field of EC is about this Decision variables Problem parameters Candidate solutions Application Space of solution vectors CEC 2009 tutorial 12

13 Upper level of EA calibration / design Design method Searches Design variables, Algorithm parameters, Strategy parameters Evaluates EA Space of parameter vectors Parameter performance landscape All parameters together span a (search) space One point one EA instance Height of point = performance of EA instance on a given problem Parameter performance landscape or utility landscape for each { EA + problem instance + performance measure } This landscape is unlikely to be trivial, e.g., unimodal, separable If there is some structure in the utility landscape, then we can do better than random or exhaustive search CEC 2009 tutorial 13

14 Ontology Terminology LOWER PART UPPER PART METHOD EA Tuner SEARCH SPACE Solution vectors Parameter vectors QUALITY Fitness Utility ASSESSMENT Evaluation Test Fitness objective function value Utility =? MBF utility, AES utility, SR utility, combined utility, robustness utility, Off line vs. on line calibration / design Design / calibration method Off line parameter tuning On line parameter control Why focus on tuning (first)? Easier Most immediate need of users Control strategies have parameters too need tuning themselves Knowledge about tuning (utility landscapes) can help the design of good control strategies There are indications that good tuning works better than control High impact R&D programme: principled approaches to EA tuning CEC 2009 tutorial 14

15 Tuning by generate and test EA tuning is a search problem itself Straightforward hf approach: generate and test Generate parameter vectors Test parameter vectors Terminate Testing parameter vectors Run EA with these parameters on the given problem or problems Record EA performance in that run e.g., by Solution quality = best fitness at termination Speed = time used to find required solution quality EAs are stochastic repetitions are needed for reliable evaluation we get statistics, e.g., Average performance by solution quality, speed (MBF, AES, AEB) Robustness = variance in those averages Success rate = % runs ending with success Big issue: how many repetitions of the test CEC 2009 tutorial 15

16 Numeric parameters E.g., population size, xover rate, tournament size, Domain is subset of R, Z, N (finite it or infinite) it Sensible distance metric searchable EA perfo rmance EA perfo rmance Parameter value Relevant parameter Parameter value Irrelevant parameter Symbolic parameters E.g., xover_operator, elitism, selection_method Finitedomain, e.g., {1 point, uniform, averaging}, {Y, N} No sensible distance metric non searchable in general EA pe erformance A B C D E F G H EA pe erformance B D C A H F G E Parameter value Parameter value Non-searchable ordering Searchable ordering CEC 2009 tutorial 16

17 Notes on parameters A value of a symbolic parameter can introduce a numeric parameter, e.g., Selection = tournament tournament size Populations_type = overlapping generation gap Parameters can have a hierarchical, nested structure Number of EA parameters is not defined in general Cannot simply denote the design space / tuning search space by S = Q 1 x Q m x R 1 x x R n with Q i / R j as domains of the symbolic/numeric parameters What is an EA? ALG 1 ALG 2 ALG 3 ALG 4 SYMBOLIC PARAMETERS Representation Bit string Bit string Real valued Real valued Overlapping pops N Y Y Y Survivor selection Tournament Replace worst Replace worst Parent selection Roulette wheel Uniform determ Tournament Tournament Mutation Bit flip Bit flip N(0,σ) N(0,σ) Recombination Uniform xover Uniform xover Discrete recomb Discrete recomb NUMERIC PARAMETERS Generation gap Population size Tournament size Mutation rate Mutation stepsize Crossover rate CEC 2009 tutorial 17

18 What is an EA? (cont d) Make a principal distinction between EAs and EA instances and place the border between them by: Option 1 There is only one EA, the generic EA scheme Previous table contains 1 EA and 4 EA instances Option 2 An EA = particular configuration of the symbolic parameters Previous table contains 3 EAs, with 2 instances for one of them Option 3 An EA = particular configuration of parameters Notions of EA and EA instance coincide Previous table contains 4 EAs / 4 EA instances Generate and test under the hood Generate initial parameter vectors Test p.v. s Terminate Select p.v. s Fixed-set search Smart fixed-set search Population-based search Generate p.v. s CEC 2009 tutorial 18

19 Tuning effort Total amount of computational work is determined by A = number of vectors tested B = number of tests per vector C = number of fitness evaluations per test Tuning methods can be positioned by their rationale: To optimize A (population based search) To optimize B (smart fixed set search) To optimize A and B (combination) To optimize C (non existent) Optimize A = optimally use A Applicable only to numeric parameters Number of tested vectors not fixed, A is the maximum (stop cond.) Initialization with N << A vectors + iterations Per iteration: generating, testing, selecting Meta EA (Greffenstette 86) Generate: usual crossover and mutation of p.v. s SPO O( (Bartz Beielstein e et al. 05) Generate: uniform random sampling!!! of p.v. s REVAC (Nannen & Eiben 06) Generate: usual crossover and distribution based mutation of p.v. s CEC 2009 tutorial 19

REVAC illustration Optimize B = reduce B Applicable to symbolic and numeric parameters Number of tested

random method random sampling exhaustive method enumeration Complete testing vs.

of tests per vector varies, B Idea: Execute tests in a breadth first fashion, all vectors X < B times Stop

20 REVAC illustration Optimize B = reduce B Applicable to symbolic and numeric parameters Number of tested vectors (A) fixed at initialization Set of tested vectors can be created by regular method grid search random method random sampling exhaustive method enumeration Complete testing vs. selective testing Complete testing: nr. of tests per vector = B (thus, not optimizing) Selective testing: nr. of tests per vector varies, B Idea: Execute tests in a breadth first fashion, all vectors X < B times Stop testing vectors with statistically significant poorer utility Well known methods ANOVA (Scheffer 89) Racing (Maron & Moore 97) CEC 2009 tutorial 20

21 Optimize A & B Existing work: Meta EA with racing (Yuan & Gallagher 04) New trick: sharpening (Smit & Eiben 2009) destilled from SPO Idea: test vectors X < B times and increase X over time during the run of a population based tuner see talk on Tuesday (S2 1 at ) Ongoing research Meta EA with sharpening Mt Meta EA with racing & sharpening REVAC with racing REVAC with sharpening REVAC with racing & sharpening Which tuning method? Differences between tuning algorithms Maximum utility reached Computational costs Number of their own parameters overhead costs Insights offered about EA parameters (probability distribution, interactions, relevance, explicit model ) Similarities between tuning algorithms Nobody is using them Can find good parameter vectors Solid comparison is missing ongoing CEC 2009 tutorial 21

22 Preliminary comparison Meta EA REVAC SPO Human A Human B Optimum Model + ++ Value One single problem (Rastrigin function) Meta EA = CMA ES REVAC is untuned Human A = 20 years experience Human B = ½ year experience Used also add ons (racing, sharpening) see presentation during this CEC Tuning world champion EAs Evolution Strategy (G-CMA-ES) Tuned by Best Median Worst G-CMA-ES REVAC CEC-2005 champion Differential Evolution (SADE) Tuned by Best Median Worst G-CMA-ES REVAC CEC-2005 champion Main conclusion: if they had asked us. CEC 2009 tutorial 22

GP, optimization, evolutionary robotics, etc Note our web based quiz check your random sampling score And the biggest

23 Practice of tuning Present practice: no principles, the main approaches are ad hoc choices, (mis)beliefs, conventions, limited grid search New, improved practice needs Solid tuning method(s) Working EA framework, shell (e.g., ECJ) Integrated toolbox New attitude We can improve any EA with our tuning; working on evidence incl. GP, optimization, evolutionary robotics, etc Note our web based quiz check your random sampling score And the biggest challenge is CEC 2009 tutorial 23

CEC 2009, IEEE Press, 2009 The big picture: methodology Experimental methodology as a

24 Recommended reading S.K. Smit and A.E. Eiben, Using Entropy for Parameter Analysis of Evolutionary Algorithms, in Bartz Beielstein et al. (eds.), Empirical Methods for the Analysis of Optimization Algorithms, Springer, 2009 (to appear) S.K. Smit and A.E. Eiben, Comparing Parameter Tuning Methods for Evolutionary Algorithms, Proceedings of the CEC 2009, IEEE Press, 2009 The big picture: methodology Experimental methodology as a whole is weak Eiben and Jelasity paper (CEC 2002) Eiben and Smith book (Springer 2003), Chapter 14 Context: objectives & user priorities Performance measures Statistics Test problems relevance?! misadapting community? Test problems relevance?! misadapting community? Scoping! CEC 2009 tutorial 24

25 Culture change? Fast and good tuning can lead to radically new attitude Past & present: robust EAs preferred (the expert vs. tuner game is actually unfair) Future: problem specific EAs preferred Old question: what is better the GA or the ES? New question: what symbolic configuration is best? Black box with 3 layers inside and a single START button Assumption: tuner level less sensitive to its parameters Never mind the No Free Lunch theorems The (near) future of automated tuning Hybrid methods combining population based search and selective testing Well funded EA performance measures, multi objective formulation multi objective tuner algorithms (Statistical) models of the utility landscape more knowledge about parameters Open source toolboxes Distributed execution Good testbeds Adoption by the EC community Rollout to other heuristic methods with parameters THUS: MORE THAN ENOUGH WORK TO DO CEC 2009 tutorial 25

Parameter Control of Genetic Algorithms by Learning and Simulation of Bayesian Networks

Submitted Soft Computing Parameter Control of Genetic Algorithms by Learning and Simulation of Bayesian Networks C. Bielza,*, J.A. Fernández del Pozo, P. Larrañaga Universidad Politécnica de Madrid, Departamento