Automatic Algorithm Configuration

Transcription

1 Automatic Algorithm Configuration Thomas Stützle RDA, CoDE, Université Libre de Bruxelles Brussels, Belgium iridia.ulb.ac.be/~stuetzle Outline 1. Context 2. Automatic algorithm configuration 3. Applications 4. Concluding remarks TPNC 2017, Prague, Czech Republic 2

2 The algorithmic solution of hard optimization problems is one of the CS/OR success stories! Exact (systematic search) algorithms Branch&Bound, Branch&Cut, constraint programming,... powerful general-purpose software available guarantees on optimality but often time/memory consuming Approximate algorithms heuristics, local search, metaheuristics, hyperheuristics... typically special-purpose software rarely provable guarantees but often fast and accurate Much active research on hybrids between exact and approximate algorithms! TPNC 2017, Prague, Czech Republic 3 Design choices and parameters everywhere Todays high-performance optimizers involve a large number of design choices and parameter settings exact solvers design choices include alternative models, pre-processing, variable selection, value selection, branching rules... many design choices have associated numerical parameters example: SCP solver (fastest non-commercial MP solver) has more than 200 relevant parameters that influence the solver s search mechanism approximate algorithms design choices include solution representation, operators, neighborhoods, pre-processing, strategies,... many design choices have associated numerical parameters example: multi-objective ACO algorithms with 22 parameters (plus several still hidden ones) TPNC 2017, Prague, Czech Republic 4

3 LS design choices and numerical parameters choice of constructive procedures random vs. greedy construction static vs. adaptive construction how many initial solutions? perturbation type of perturbation fixed vs. variable size of destruction random vs. biased perturbation acceptance criterion strength of bias towards best solutions use of history information and if yes, how local search... many choices... numerical parameters perturbation size parameters associated type of perturbation parameters related to acceptance criterion TPNC 2017, Prague, Czech Republic 5 Parameter types categorical parameters choice of constructive procedure, choice of recombination operator, choice of branching strategy,... ordinal parameters neighborhoods, lower bounds,... numerical parameters design design tuning, calibration integer or real-valued parameters weighting factors, population sizes, temperature, hidden constants,... numerical parameters may be conditional to specific values of categorical or ordinal parameters Design and configuration of algorithms involves setting categorical, ordinal, and numerical parameters TPNC 2017, Prague, Czech Republic 6

4 Designing optimization algorithms Challenges many alternative design choices nonlinear interactions among algorithm components and/or parameters performance assessment is di cult Traditional design approach trial and error design guided by expertise/intuition prone to over-generalizations, implicit independence assumptions, limited exploration of design alternatives Can we make this approach more principled and automatic? TPNC 2017, Prague, Czech Republic 7 Towards automatic algorithm configuration Automated algorithm configuration apply powerful search techniques to design algorithms use computation power to explore design spaces assist algorithm designer in the design process free human creativity for higher level tasks TPNC 2017, Prague, Czech Republic 8

5 O ine configuration and online parameter control O ine configuration configure algorithm before deploying it configuration on training instances related to algorithm design Online parameter control adapt parameter setting while solving an instance typically limited to a set of known crucial algorithm parameters related to parameter calibration O ine configuration techniques can be helpful to configure (online) parameter control strategies TPNC 2017, Prague, Czech Republic 9 O ine configuration Typical performance measures maximize solution quality (within given computation time) minimize computation time (to reach optimal solution) TPNC 2017, Prague, Czech Republic 10

6 Configurators TPNC 2017, Prague, Czech Republic 11 Approaches to configuration experimental design techniques e.g. CALBRA [Adenso Díaz, Laguna, 2006], [Ridge&Kudenko, 2007], [Coy et al., 2001], [Ruiz, Stützle, 2005] numerical optimization techniques e.g. MADS [Audet&Orban, 2006], various [Yuan et al., 2012] heuristic search methods e.g. meta-ga [Grefenstette, 1985], ParamLS [Hutter et al., 2007, 2009], gender-based GA [Ansótegui at al., 2009], linear GP [Oltean, 2005], REVAC(++) [Eiben et al., 2007, 2009, 2010]... model-based optimization approaches e.g. SPO [Bartz-Beielstein et al., 2005, 2006,.. ], SMAC [Hutter et al., 2011,..], GGA++ [Ansótegui, 2015] sequential statistical testing e.g. F-race, iterated F-race [Birattari et al, 2002, 2007,...] General, domain-independent methods required: (i) applicable to all variable types, (ii) multiple training instances, (iii) high performance, (iv) scalable TPNC 2017, Prague, Czech Republic 12

7 Approaches to configuration experimental design techniques e.g. CALBRA [Adenso Díaz, Laguna, 2006], [Ridge&Kudenko, 2007], [Coy et al., 2001], [Ruiz, Stützle, 2005] numerical optimization techniques e.g. MADS [Audet&Orban, 2006], various [Yuan et al., 2012] heuristic search methods e.g. meta-ga [Grefenstette, 1985], ParamLS [Hutter et al., 2007, 2009], gender-based GA [Ansótegui at al., 2009], linear GP [Oltean, 2005], REVAC(++) [Eiben et al., 2007, 2009, 2010]... model-based optimization approaches e.g. SPO [Bartz-Beielstein et al., 2005, 2006,.. ], SMAC [Hutter et al., 2011,..], GGA++ [Ansótegui, 2015] sequential statistical testing e.g. F-race, iterated F-race [Birattari et al, 2002, 2007,...] General, domain-independent methods required: (i) applicable to all variable types, (ii) multiple training instances, (iii) high performance, (iv) scalable TPNC 2017, Prague, Czech Republic 13 The racing approach i start with a set of initial candidates consider a stream of instances sequentially evaluate candidates discard inferior candidates as su cient evidence is gathered against them...repeat until a winner is selected or until computation time expires TPNC 2017, Prague, Czech Republic 14

8 The F-Race algorithm Statistical testing 1. family-wise tests for di erences among configurations Friedman two-way analysis of variance by ranks 2. if Friedman rejects H0, perform pairwise comparisons to best configuration apply Friedman post-test TPNC 2017, Prague, Czech Republic 15 terated race Racing is a method for the selection of the best configuration and independent of the way the set of configurations is sampled terated race sample configurations from initial distribution While not terminate() apply race modify sampling distribution sample configurations TPNC 2017, Prague, Czech Republic 16

9 terated race: sampling { x 1 x 2 x 3 { x 1 x 2 x 3 TPNC 2017, Prague, Czech Republic 17 terated racing: sampling distributions Numerical parameter X d 2 [x d, x d ] ) Truncated normal distribution N (µ z d, i d ) 2 [x d, x d ] µ z d =valueofparameterd in elite configuration z i d =decreaseswiththenumberofiterations Categorical parameter X d 2 {x 1, x 2,...,x nd } ) Discrete probability distribution x 1 x 2... x nd Pr z {X d = x j } = x 1 x 2 x 3 Updated by increasing probability of parameter value in elite configuration Other probabilities are reduced TPNC 2017, Prague, Czech Republic 18

10 The irace package Manuel López-báñez, Jérémie Dubois-Lacoste, Thomas Stützle, and Mauro Birattari. The irace package, terated Race for Automatic Algorithm Configuration. Technical Report TR/RDA/ , RDA, Université Libre de Bruxelles, Belgium, 2011; extended version: Operations Research Perspectives, The irace Package: User Guide, 2016, Technical Report TR/RDA/ implementation of terated Racing in R Goal 1: flexible Goal 2: easy to use but no knowledge of R necessary parallel evaluation (MP, multi-cores, grid engine.. ) capping for run-time optimization irace has shown to be e ective for configuration tasks with several hundred of variables TPNC 2017, Prague, Czech Republic 19 The irace package: usage Training instances Parameter space Configuration scenario calls with θ,i irace irace returns c(θ,i) targetrunner TPNC 2017, Prague, Czech Republic 20

11 Other tools: ParamLS, SMAC ParamLS iterated local search in configuration space requires discretization of numerical parameters SMAC surrogate model assisted search process encouraging results for large configuration spaces capping: e ective speed-up technique for configuration target run-time TPNC 2017, Prague, Czech Republic 21 Applications of automatic configuration tools configuration of black-box solvers e.g. mixed integer programming solvers, continuous optimizers supporting tool in algorithm engineering e.g. metaheuristics for probabilistic TSP, re-engineering PSO bottom-up generation of heuristic algorithms e.g. heuristics for SAT, FSP, etc.; metaheuristic framework design configurable algorithm frameworks e.g. Satenstein, MOACO, UACOR TPNC 2017, Prague, Czech Republic 22

12 Example, configuration of black-box solvers Mixed-integer programming solvers TPNC 2017, Prague, Czech Republic 23 Mixed integer programming (MP) solvers [Hutter, Hoos, Leyton-Brown, Stützle, 2009, Hutter, Hoos Leighton-Brown, 2010] MP modelling widely used for tackling optimization problems powerful commercial (e.g. CPLEX) and non-commercial (e.g. SCP) solvers available large number of parameters (tens to hundreds) Benchmark set Default Configured Speedup Regions (11.4 ± 0.9) 6.8 Conic.SCH (2.4 ± 0.29) 2.51 CLS (327 ± 860) MK (301 ± 948) QP (827 ± 306) 1.85 FocusedLS, 10 runs, 2 CPU days, 63 parameters TPNC 2017, Prague, Czech Republic 24

13 Example, bottom-up generation of algorithms Automatic design of hybrid SLS algorithms TPNC 2017, Prague, Czech Republic 25 Approach TPNC 2017, Prague, Czech Republic 26

14 Main approaches Top-down approaches develop flexible framework following a fixed algorithm template with alternatives apply high-performing configurators Examples: Satenstein, MOACO, MOEA, MP Solvers?(!) Bottom-up approaches flexible framework implementing algorithm components define rules for composing algorithms from components e.g. through grammars frequently usage of genetic programming, grammatical evolution etc. TPNC 2017, Prague, Czech Republic 27 Automatic design of hybrid SLS algorithms [Marmion, Mascia, Lópes-báñez, Stützle, 2013] Approach decompose single-point SLS methods into components derive generalized metaheuristic structure component-wise implementation of metaheuristic part mplementation present possible algorithm compositions by a grammar instantiate grammer using a parametric representation allows use of standard automatic configuration tools shows good performance when compared to, e.g., grammatical evolution [Mascia, Lópes-báñez, Dubois-Lacoste, Stützle, 2013] TPNC 2017, Prague, Czech Republic 28

15 General Local Search Structure: LS s 0 :=initsolution s := ls(s 0 ) repeat s 0 :=perturb(s,history) s 0 :=ls(s 0 ) s :=accept(s,s 0,history) until termination criterion met many SLS methods instantiable from this structure abilities hybridization recursion problem specific implementation at low-level TPNC 2017, Prague, Czech Republic 29 Grammar <algorithm> ::= <initialization> <ils> <initialization> ::= random <pbs_initialization> <ils> ::= LS(<perturb>, <ls>, <accept>, <stop>) <perturb> ::= none <initialization> <pbs_perturb> <ls> ::= <ils> <descent> <sa> <rii> <pii> <vns> <ig> <pbs_ls> <accept> ::= alwaysaccept improvingaccept <comparator> prob(<value_prob_accept>) probrandom <metropolis> threshold(<value_threshold_accept>) <pbs_accept> <descent> ::= bestdescent(<comparator>, <stop>) firstmprdescent(<comparator>, <stop>) <sa> ::= LS(<pbs_move>, no_ls, <metropolis>, <stop>) <rii> ::= LS(<pbs_move>, no_ls, probrandom, <stop>) <pii> ::= LS(<pbs_move>, no_ls, prob(<value_prob_accept>), <stop>) <vns> ::= LS(<pbs_variable_move>, firstmprdescent(improvingstrictly), improvingaccept(improvingstrictly), <stop>) <ig> ::= LS(<deconst-construct_perturb>, <ls>, <accept>, <stop>) <comparator> ::= improvingstrictly improving <value_prob_accept> ::= [0, 1] <value_threshold_accept> ::= [0, 1] <metropolis> ::= metropolisaccept(<init_temperature>, <final_temperature>, <decreasing_temperature_ratio>, <span>) <init_temperature> ::= {1, 2,..., 10000} <final_temperature> ::= {1, 2,..., 100} <decreasing_temperature_ratio> ::= [0, 1] <span> ::= {1, 2,..., 10000} TPNC 2017, Prague, Czech Republic 30

16 Grammar <algorithm> ::= <initialization> <ils> <initialization> ::= random <pbs_initialization> <ils> ::= LS(<perturb>, <ls>, <accept>, <stop>) <perturb> ::= none <initialization> <pbs_perturb> <ls> ::= <ils> <descent> <sa> <rii> <pii> <vns> <ig> <pbs_ls> <accept> ::= alwaysaccept improvingaccept <comparator> prob(<value_prob_accept>) probrandom <metropolis> threshold(<value_threshold_accept>) <pbs_accept> <descent> ::= bestdescent(<comparator>, <stop>) firstmprdescent(<comparator>, <stop>) <sa> ::= LS(<pbs_move>, no_ls, <metropolis>, <stop>) <rii> ::= LS(<pbs_move>, no_ls, probrandom, <stop>) <pii> ::= LS(<pbs_move>, no_ls, prob(<value_prob_accept>), <stop>) <vns> ::= LS(<pbs_variable_move>, firstmprdescent(improvingstrictly), improvingaccept(improvingstrictly), <stop>) <ig> ::= LS(<deconst-construct_perturb>, <ls>, <accept>, <stop>) <comparator> ::= improvingstrictly improving <value_prob_accept> ::= [0, 1] <value_threshold_accept> ::= [0, 1] <metropolis> ::= metropolisaccept(<init_temperature>, <final_temperature>, <decreasing_temperature_ratio>, <span>) <init_temperature> ::= {1, 2,..., 10000} <final_temperature> ::= {1, 2,..., 100} <decreasing_temperature_ratio> ::= [0, 1] <span> ::= {1, 2,..., 10000} TPNC 2017, Prague, Czech Republic 31 Grammar <algorithm> ::= <initialization> <ils> <initialization> ::= random <pbs_initialization> <ils> ::= LS(<perturb>, <ls>, <accept>, <stop>) <perturb> ::= none <initialization> <pbs_perturb> <ls> ::= <ils> <descent> <sa> <rii> <pii> <vns> <ig> <pbs_ls <accept> ::= alwaysaccept improvingaccept <comparator> prob(<value_prob_accept>) probrandom <metropolis> threshold(<value_threshold_accept>) <pbs_accept> <descent> ::= bestdescent(<comparator>, <stop>) firstmprdescent(<comparator>, <stop>) <sa> ::= LS(<pbs_move>, no_ls, <metropolis>, <stop>) <rii> ::= LS(<pbs_move>, no_ls, probrandom, <stop>) <pii> ::= LS(<pbs_move>, no_ls, prob(<value_prob_accept>), <stop>) <vns> ::= LS(<pbs_variable_move>, firstmprdescent(improvingstrictly), improvingaccept(improvingstrictly), <stop>) <ig> ::= LS(<deconst-construct_perturb>, <ls>, <accept>, <stop>) <comparator> ::= improvingstrictly improving <value_prob_accept> ::= [0, 1] <value_threshold_accept> ::= [0, 1] <metropolis> ::= metropolisaccept(<init_temperature>, <final_temperature>, <decreasing_temperature_ratio>, <span>) <init_temperature> ::= {1, 2,..., 10000} <final_temperature> ::= {1, 2,..., 100} <decreasing_temperature_ratio> ::= [0, 1] <span> ::= {1, 2,..., 10000} TPNC 2017, Prague, Czech Republic 32

17 Flow-shop problem with makespan objective Automatic configuration: max. three levels of recursion biased / unbiased grammar resulting in 262 and 502 parameters, respectively budget: trials of n m 0.03 seconds 95% confidence limits ARPD Grs Gtb irace1 irace2 irace3 irace4 Algorithms Results are clearly superior to state-of-the-art TPNC 2017, Prague, Czech Republic 33 Flow-shop problem with total tardiness objective Automatic configuration: max. three levels of recursion biased / unbiased grammar resulting in 262 and 502 parameters, respectively budget: trials of n m 0.03 seconds Relative Deviation ndex TSM63 TSMe63 irace4 Results are clearly superior to state-of-the-art TPNC 2017, Prague, Czech Republic 34

18 Conclusions Results design and analysis of (hybrid) metaheuristics is automated not a silver bullet, but needs right components, especially low-level problem-specific ones better or equal performance to state-of-the-art for PFSP-WT, UBQP, TSP-TW directly extendible for unbiased comparisons of metaheuristics Future work extensions to other methods and templates dealing with complexity of hybrid algorithms increase generality, tackling wide problem classes TPNC 2017, Prague, Czech Republic 35 Why automatic algorithm configuration? improvement over manual, ad-hoc methods for tuning reduction of development time and human intervention increase number of considerable degrees of freedom empirical studies, comparisons of algorithms support for end users of algorithms... and it has become feasible due to increase in computational power! TPNC 2017, Prague, Czech Republic 36

19 Configuring configurators What about configuring automatically the configurator?... and configuring the configurator of the configurator? can be done (example, see [Hutter et al., 2009]), but... it is costly and iterating further leads to diminishing returns TPNC 2017, Prague, Czech Republic 37 Towards a shift of paradigm in algorithm design TPNC 2017, Prague, Czech Republic 38

20 Towards a shift of paradigm in algorithm design TPNC 2017, Prague, Czech Republic 39 Towards a shift of paradigm in algorithm design TPNC 2017, Prague, Czech Republic 40

21 Conclusions Status using automatic configuration tools is rewarding in terms of development time and algorithm performance interactive usage of configurators allows humans to focus on creative part of algorithm design many application opportunities also in other areas than optimization Future work more powerful configurators more and more complex applications best practice TPNC 2017, Prague, Czech Republic 41