Automatic Algorithm Configuration based on Local Search
|
|
|
- Donna Horn
- 5 years ago
- Views:
Transcription
1 Automatic Algorithm Configuration based on Local Search Frank Hutter 1 Holger Hoos 1 Thomas Stützle 2 1 Department of Computer Science University of British Columbia Canada 2 IRIDIA Université Libre de Bruxelles Belgium
2 Motivation for automatic algorithm configuration Want to design best algorithm to solve a problem Many design choices need to be made Some choices deferred to later: free parameters of algorithm Set parameters to maximise empirical performance Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 2
3 Motivation for automatic algorithm configuration Want to design best algorithm to solve a problem Many design choices need to be made Some choices deferred to later: free parameters of algorithm Set parameters to maximise empirical performance Finding best parameter configuration is non-trivial Many parameters, discrete & continuous Dependencies between parameters Many test instances needed to generalize Many runs per instance needed for randomised algorithms Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 2
4 Motivation for automatic algorithm configuration Want to design best algorithm to solve a problem Many design choices need to be made Some choices deferred to later: free parameters of algorithm Set parameters to maximise empirical performance Finding best parameter configuration is non-trivial Many parameters, discrete & continuous Dependencies between parameters Many test instances needed to generalize Many runs per instance needed for randomised algorithms Algorithm configuration / tuning still often done manually, using ad-hoc methods tedious and time-consuming, sub-optimal results big incentive for automation Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 2
5 Real-world example: Application: Solving SAT-encoded software verification problems Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 3
6 Real-world example: Application: Solving SAT-encoded software verification problems Tune 26 parameters of new DPLL-type SAT solver (Spear) 7 categorical, 3 boolean, 12 continuous, 4 integer parameters Variable/value heuristics, clause learning, restarts,... Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 3
7 Real-world example: Application: Solving SAT-encoded software verification problems Tune 26 parameters of new DPLL-type SAT solver (Spear) 7 categorical, 3 boolean, 12 continuous, 4 integer parameters Variable/value heuristics, clause learning, restarts,... Minimize expected run-time Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 3
8 Real-world example: Application: Solving SAT-encoded software verification problems Tune 26 parameters of new DPLL-type SAT solver (Spear) 7 categorical, 3 boolean, 12 continuous, 4 integer parameters Variable/value heuristics, clause learning, restarts,... Minimize expected run-time Problems: default settings 300 seconds / run good performance on a few instances may not generalise Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 3
9 Standard algorithm configuration approach Choose a representative benchmark set for tuning Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 4
10 Standard algorithm configuration approach Choose a representative benchmark set for tuning Perform iterative manual tuning: start with some parameter configuration repeat modify a single parameter if results on tuning set improve then keep new configuration until no more improvement possible (or good enough ) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 4
11 Problems: cost for evaluating configuration depends on number and hardness of problem instances constraints on tuning time (per iteration and overall) typically use few and fairly easy instances, few iterations Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 5
12 Problems: cost for evaluating configuration depends on number and hardness of problem instances constraints on tuning time (per iteration and overall) typically use few and fairly easy instances, few iterations manual search = iterative improvement (hill climbing) finds local optimum only Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 5
13 Problems: cost for evaluating configuration depends on number and hardness of problem instances constraints on tuning time (per iteration and overall) typically use few and fairly easy instances, few iterations manual search = iterative improvement (hill climbing) finds local optimum only slow and tedious, requires significant human time procedure often performed in ad-hoc way Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 5
14 Problems: cost for evaluating configuration depends on number and hardness of problem instances constraints on tuning time (per iteration and overall) typically use few and fairly easy instances, few iterations manual search = iterative improvement (hill climbing) finds local optimum only slow and tedious, requires significant human time procedure often performed in ad-hoc way Solution: automate process use more powerful search method Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 5
15 Related work Search approaches [Minton 1993, 1996], [Hutter 2004], [Cavazos & O Boyle 2005], [Adenso-Diaz & Laguna 2006], [Audet & Orban 2006] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 6
16 Related work Search approaches [Minton 1993, 1996], [Hutter 2004], [Cavazos & O Boyle 2005], [Adenso-Diaz & Laguna 2006], [Audet & Orban 2006] Racing algorithms/bandit solvers [Birattari et al. 2002], [Smith et al. 2004, 2006] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 6
17 Related work Search approaches [Minton 1993, 1996], [Hutter 2004], [Cavazos & O Boyle 2005], [Adenso-Diaz & Laguna 2006], [Audet & Orban 2006] Racing algorithms/bandit solvers [Birattari et al. 2002], [Smith et al. 2004, 2006] Stochastic Optimisation [Kiefer & Wolfowitz 1952], [Spall 1987] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 6
18 Related work Search approaches [Minton 1993, 1996], [Hutter 2004], [Cavazos & O Boyle 2005], [Adenso-Diaz & Laguna 2006], [Audet & Orban 2006] Racing algorithms/bandit solvers [Birattari et al. 2002], [Smith et al. 2004, 2006] Stochastic Optimisation [Kiefer & Wolfowitz 1952], [Spall 1987] Learning approaches Regression trees [Bartz-Beielstein et al. 2004] Response surface models, DACE [Bartz-Beielstein et al ] Lots of work on per-instance / reactive tuning orthogonal to the approach followed here Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 6
19 Outline 1. Introduction 2. Iterated local search over parameter configurations 3. The FocusedILS algorithm 4. Sample applications and performance results 5. Conclusions and future work Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 7
20 ILS in parameter configuration space (ParamILS): choose initial parameter configuration θ perform subsidiary local search on s Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 8
21 ILS in parameter configuration space (ParamILS): choose initial parameter configuration θ perform subsidiary local search on s While tuning time left: θ := θ perform perturbation on θ perform subsidiary local search on θ Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 8
22 ILS in parameter configuration space (ParamILS): choose initial parameter configuration θ perform subsidiary local search on s While tuning time left: θ := θ perform perturbation on θ perform subsidiary local search on θ based on acceptance criterion, keep θ or revert to θ := θ Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 8
23 ILS in parameter configuration space (ParamILS): choose initial parameter configuration θ perform subsidiary local search on s While tuning time left: θ := θ perform perturbation on θ perform subsidiary local search on θ based on acceptance criterion, keep θ or revert to θ := θ with probability p restart randomly pick new θ performs biased random walk over local minima Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 8
24 Details on ParamILS: subsidiary local search: iterative first improvement, change on parameter in each step Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 9
25 Details on ParamILS: subsidiary local search: iterative first improvement, change on parameter in each step perturbation: change 3 randomly chosen parameters Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 9
26 Details on ParamILS: subsidiary local search: iterative first improvement, change on parameter in each step perturbation: change 3 randomly chosen parameters acceptance criterion: always select better configuration Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 9
27 Details on ParamILS: subsidiary local search: iterative first improvement, change on parameter in each step perturbation: change 3 randomly chosen parameters acceptance criterion: always select better configuration initialisation: pick best of default + R random configurations Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 9
28 Evaluation of a parameter configuration θ (based on N runs) Sample N instances from given set (with repetitions) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 10
29 Evaluation of a parameter configuration θ (based on N runs) Sample N instances from given set (with repetitions) For each of the N instances, execute algorithm with configuration θ. Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 10
30 Evaluation of a parameter configuration θ (based on N runs) Sample N instances from given set (with repetitions) For each of the N instances, execute algorithm with configuration θ. Record scalar cost of each of the N runs: sc 1,..., sc n (run-time, solution quality,... ) empirical cost distribution ĈD Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 10
31 Evaluation of a parameter configuration θ (based on N runs) Sample N instances from given set (with repetitions) For each of the N instances, execute algorithm with configuration θ. Record scalar cost of each of the N runs: sc 1,..., sc n (run-time, solution quality,... ) empirical cost distribution ĈD Compute scalar statistic ĉ N (θ) of ĈD (mean, median,... ) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 10
32 Evaluation of a parameter configuration θ (based on N runs) Sample N instances from given set (with repetitions) For each of the N instances, execute algorithm with configuration θ. Record scalar cost of each of the N runs: sc 1,..., sc n (run-time, solution quality,... ) empirical cost distribution ĈD Compute scalar statistic ĉ N (θ) of ĈD (mean, median,... ) Note: For large N, ĉ N (θ) approaches true cost c(θ) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 10
33 Solution quality over time achieved by ParamILS Runlength (median, 10% & 90% quantiles) 2.6 x 104 BasicILS(100) performance on training set CPU time [s] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 11
34 Question: How many runs/instances? too many evaluating a configuration is very expensive optimisation process is very slow Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 12
35 Question: How many runs/instances? too many evaluating a configuration is very expensive optimisation process is very slow too few very noisy approximations ĉ N (θ) poor generalisation to independent test runs Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 12
36 Generalisation to independent test runs (1) Runlength (median, 10% & 90% quantiles) x 10 4 BasicILS(100) performance on test set BasicILS(100) performance on training set CPU time [s] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 13
37 Generalisation to independent test runs (2) Runlength (median, 10% & 90% quantiles) 6 x BasicILS(1) performance on test set BasicILS(1) performance on training set CPU time [s] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 14
38 The FocusedILS algorithm Given a budget of available CPU time for tuning, use different numbers of runs, N(θ), for each configuration θ Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 15
39 The FocusedILS algorithm Given a budget of available CPU time for tuning, use different numbers of runs, N(θ), for each configuration θ Idea: Use high N(θ) only for good θ start with N(θ) = 0 for all θ increment N(θ) whenever θ is visited additional runs upon finding new, better configuration θ Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 15
40 The FocusedILS algorithm Theorem: As number of FocusedILS iterations, it converges to true optimal configuration θ Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 16
41 The FocusedILS algorithm Theorem: As number of FocusedILS iterations, it converges to true optimal configuration θ Key ideas in proof: 1. For N(θ), N(θ ), comparisons between θ, θ become precise. Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 16
42 The FocusedILS algorithm Theorem: As number of FocusedILS iterations, it converges to true optimal configuration θ Key ideas in proof: 1. For N(θ), N(θ ), comparisons between θ, θ become precise. 2. Underlying ILS eventually reaches any configuration θ. Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 16
43 Performance of FocusedILS vs BasicILS (Test performance on SAPS-QWH) Median runlength of SAPS [steps] 2.2 x BasicILS(100) CPU time for ParamILS [s] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 17
44 Performance of FocusedILS vs BasicILS (Test performance on SAPS-QWH) Median runlength of SAPS [steps] 2.2 x BasicILS(100) BasicILS(10) CPU time for ParamILS [s] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 17
45 Performance of FocusedILS vs BasicILS (Test performance on SAPS-QWH) Median runlength of SAPS [steps] 2.2 x BasicILS(100) BasicILS(1) BasicILS(10) CPU time for ParamILS [s] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 17
46 Performance of FocusedILS vs BasicILS (Test performance on SAPS-QWH) Median runlength of SAPS [steps] 2.2 x FocusedILS BasicILS(100) BasicILS(1) BasicILS(10) CPU time for ParamILS [s] Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 17
47 Sample applications and performance results Experiment: Comparison to CALIBRA Scenario Default CALIBRA(100) BasicILS(100) FocusedILS GLS-GRID ɛ = ± ± ± Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 18
48 Sample applications and performance results Experiment: Comparison to CALIBRA Scenario Default CALIBRA(100) BasicILS(100) FocusedILS GLS-GRID ɛ = ± ± ± SAPS-QWH 85.5K steps 10.7K ± 1.1K 10.9K ± 0.6K 10.6K ± 0.5K Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 18
49 Sample applications and performance results Experiment: Comparison to CALIBRA Scenario Default CALIBRA(100) BasicILS(100) FocusedILS GLS-GRID ɛ = ± ± ± SAPS-QWH 85.5K steps 10.7K ± 1.1K 10.9K ± 0.6K 10.6K ± 0.5K SAPS-SW 5.60 s ± ± ± Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 18
50 Sample applications and performance results Experiment: Comparison to CALIBRA Scenario Default CALIBRA(100) BasicILS(100) FocusedILS GLS-GRID ɛ = ± ± ± SAPS-QWH 85.5K steps 10.7K ± 1.1K 10.9K ± 0.6K 10.6K ± 0.5K SAPS-SW 5.60 s ± ± ± SAT4J-SW 7.02 s (too many param.) 1.19 ± ± 0.2 Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 18
51 Speedup obtained by automated tuning (SAPS default vs tuned on test set SW-GCP) run time [s], auto tuned parameters run time [s], default parameters Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 19
52 Two real-world applications New DPLL-type SAT solver Spear 26 parameters Software verification: 500-fold speedup Hardware verification: 4.5-fold speedup New state of the art for those instances Hutter, Babić, Hoos & Hu: FMCAD 07 (to appear) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 20
53 Two real-world applications New DPLL-type SAT solver Spear 26 parameters Software verification: 500-fold speedup Hardware verification: 4.5-fold speedup New state of the art for those instances Hutter, Babić, Hoos & Hu: FMCAD 07 (to appear) New replica exchange Monte Carlo algorithm for protein structure prediction 3 parameters 2-fold improvement New state of the art for 2D/3D protein structure prediction Thachuk, Shmygelska & Hoos (under review) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 20
54 Conclusions ParamILS: Simple and efficient framework for automatic parameter optimization Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 21
55 Conclusions ParamILS: Simple and efficient framework for automatic parameter optimization FocusedILS: converges provably towards optimal configuration, no over-confidence excellent performance in practice (outperforms BasicILS, CALIBRA) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 21
56 Conclusions ParamILS: Simple and efficient framework for automatic parameter optimization FocusedILS: converges provably towards optimal configuration, no over-confidence excellent performance in practice (outperforms BasicILS, CALIBRA) Huge speedups: 100 for SAPS (local search) on graph colouring 500 for Spear (tree search) on software verification Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 21
57 Conclusions ParamILS: Simple and efficient framework for automatic parameter optimization FocusedILS: converges provably towards optimal configuration, no over-confidence excellent performance in practice (outperforms BasicILS, CALIBRA) Huge speedups: 100 for SAPS (local search) on graph colouring 500 for Spear (tree search) on software verification Publically available at: Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 21
58 Future work Continuous parameters (currently discretised) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 22
59 Future work Continuous parameters (currently discretised) Statistical tests (cf. racing algorithms) Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 22
60 Future work Continuous parameters (currently discretised) Statistical tests (cf. racing algorithms) Per-instance tuning Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 22
61 Future work Continuous parameters (currently discretised) Statistical tests (cf. racing algorithms) Per-instance tuning Automatic algorithm design Hutter, Hoos, Stützle: Automatic Algorithm Configuration based on Local Search 22
Automatic Algorithm Configuration based on Local Search
Automatic Algorithm Configuration based on Local Search Frank Hutter 1 Holger Hoos 1 Thomas Stützle 2 1 Department of Computer Science University of British Columbia Canada 2 IRIDIA Université Libre de
Automated Configuration of MIP solvers
Automated Configuration of MIP solvers Frank Hutter, Holger Hoos, and Kevin Leyton-Brown Department of Computer Science University of British Columbia Vancouver, Canada {hutter,hoos,kevinlb}@cs.ubc.ca
Sequential Model-based Optimization for General Algorithm Configuration
Sequential Model-based Optimization for General Algorithm Configuration Frank Hutter, Holger Hoos, Kevin Leyton-Brown University of British Columbia LION 5, Rome January 18, 2011 Motivation Most optimization
Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms
Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms Frank Hutter 1, Youssef Hamadi 2, Holger Hoos 1, and Kevin Leyton-Brown 1 1 University of British Columbia, Vancouver,
ParamILS: An Automatic Algorithm Configuration Framework
Journal of Artificial Intelligence Research 36 (2009) 267-306 Submitted 06/09; published 10/09 ParamILS: An Automatic Algorithm Configuration Framework Frank Hutter Holger H. Hoos Kevin Leyton-Brown University
Problem Solving and Search in Artificial Intelligence
Problem Solving and Search in Artificial Intelligence Algorithm Configuration Nysret Musliu Database and Artificial Intelligence Group, Institut of Logic and Computation, TU Wien Motivation Metaheuristic
Automated Configuration of Mixed Integer Programming Solvers
Automated Configuration of Mixed Integer Programming Solvers Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown University of British Columbia, 2366 Main Mall, Vancouver BC, V6T 1Z4, Canada {hutter,hoos,kevinlb}@cs.ubc.ca
MO-ParamILS: A Multi-objective Automatic Algorithm Configuration Framework
MO-ParamILS: A Multi-objective Automatic Algorithm Configuration Framework Aymeric Blot, Holger Hoos, Laetitia Jourdan, Marie-Éléonore Marmion, Heike Trautmann To cite this version: Aymeric Blot, Holger
A Modular Multiphase Heuristic Solver for Post Enrolment Course Timetabling
A Modular Multiphase Heuristic Solver for Post Enrolment Course Timetabling Marco Chiarandini 1, Chris Fawcett 2, and Holger H. Hoos 2,3 1 University of Southern Denmark, Department of Mathematics and
Boosting Verification by Automatic Tuning of Decision Procedures
Boosting Verification by Automatic Tuning of Decision Procedures Frank Hutter, Domagoj Babić, Holger H. Hoos, and Alan J. Hu Department of Computer Science, University of British Columbia {hutter, babic,
Automated Design of Metaheuristic Algorithms
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Automated Design of Metaheuristic Algorithms T. Stützle and M. López-Ibáñez IRIDIA
Automatic Algorithm Configuration
Automatic Algorithm Configuration Thomas Stützle RDA, CoDE, Université Libre de Bruxelles Brussels, Belgium stuetzle@ulb.ac.be iridia.ulb.ac.be/~stuetzle Outline 1. Context 2. Automatic algorithm configuration
Automatically Configuring Algorithms. Some recent work
Automatically Configuring Algorithms Some recent work Thomas Stützle, Leonardo Bezerra, Jérémie Dubois-Lacoste, Tianjun Liao, Manuel López-Ibáñez, Franco Mascia and Leslie Perez Caceres IRIDIA, CoDE, Université
Continuous optimization algorithms for tuning real and integer parameters of swarm intelligence algorithms
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Continuous optimization algorithms for tuning real and integer parameters of swarm
SpySMAC: Automated Configuration and Performance Analysis of SAT Solvers
SpySMAC: Automated Configuration and Performance Analysis of SAT Solvers Stefan Falkner, Marius Lindauer, and Frank Hutter University of Freiburg {sfalkner,lindauer,fh}@cs.uni-freiburg.de Abstract. Most
Note: In physical process (e.g., annealing of metals), perfect ground states are achieved by very slow lowering of temperature.
Simulated Annealing Key idea: Vary temperature parameter, i.e., probability of accepting worsening moves, in Probabilistic Iterative Improvement according to annealing schedule (aka cooling schedule).
Meta-algorithmic techniques in SAT solving: Automated configuration, selection and beyond
Meta-algorithmic techniques in SAT solving: Automated configuration, selection and beyond Holger H. Hoos BETA Lab Department of Computer Science University of British Columbia Canada SAT/SMT Summer School
Sequential Model-Based Parameter Optimization: an Experimental Investigation of Automated and Interactive Approaches
Sequential Model-Based Parameter Optimization: an Experimental Investigation of Automated and Interactive Approaches Frank Hutter Thomas Bartz-Beielstein Holger H. Hoos Kevin Leyton-Brown Kevin P. Murphy
Computer-Aided Design of High-Performance Algorithms
University of British Columbia Department of Computer Science Technical Report TR-2008-16 Computer-Aided Design of High-Performance Algorithms Holger H. Hoos University of British Columbia Department of
a local optimum is encountered in such a way that further improvement steps become possible.
Dynamic Local Search I Key Idea: Modify the evaluation function whenever a local optimum is encountered in such a way that further improvement steps become possible. I Associate penalty weights (penalties)
Data Science meets Optimization. ODS 2017 EURO Plenary Patrick De Causmaecker EWG/DSO EURO Working Group CODeS KU Leuven/ KULAK
Data Science meets Optimization ODS 2017 EURO Plenary Patrick De Causmaecker EWG/DSO EURO Working Group CODeS KU Leuven/ KULAK ODS 2017, Sorrento www.fourthparadigm.com Copyright 2009 Microsoft Corporation
Parallel Algorithm Configuration
Parallel Algorithm Configuration Frank Hutter, Holger H. Hoos and Kevin Leyton-Brown University of British Columbia, 366 Main Mall, Vancouver BC, V6T Z4, Canada {hutter,hoos,kevinlb}@cs.ubc.ca Abstract.
Université Libre de Bruxelles
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Improvement Strategies for the F-Race algorithm: Sampling Design and Iterative
arxiv: v1 [cs.ai] 7 Oct 2013
![arxiv: v1 [cs.ai] 7 Oct 2013](/thumbs/95/123077987.jpg "arxiv: v1 [cs.ai] 7 Oct 2013")
Bayesian Optimization With Censored Response Data Frank Hutter, Holger Hoos, and Kevin Leyton-Brown Department of Computer Science University of British Columbia {hutter, hoos, kevinlb}@cs.ubc.ca arxiv:.97v
On the Run-time Behaviour of Stochastic Local Search Algorithms for SAT
From: AAAI-99 Proceedings. Copyright 1999, AAAI (www.aaai.org). All rights reserved. On the Run-time Behaviour of Stochastic Local Search Algorithms for SAT Holger H. Hoos University of British Columbia
Improvement Strategies for thef-racealgorithm: Sampling Design and Iterative Refinement
Improvement Strategies for thef-racealgorithm: Sampling Design and Iterative Refinement Prasanna Balaprakash, Mauro Birattari, and Thomas Stützle IRIDIA, CoDE, Université Libre de Bruxelles, Brussels,
Effective Stochastic Local Search Algorithms for Biobjective Permutation Flow-shop Problems
Effective Stochastic Local Search Algorithms for Biobjective Permutation Flow-shop Problems Jérémie Dubois-Lacoste, Manuel López-Ibáñez, and Thomas Stützle IRIDIA, CoDE, Université Libre de Bruxelles Brussels,
AI-Augmented Algorithms
AI-Augmented Algorithms How I Learned to Stop Worrying and Love Choice Lars Kotthoff University of Wyoming larsko@uwyo.edu Warsaw, 17 April 2019 Outline Big Picture Motivation Choosing Algorithms Tuning
Simple mechanisms for escaping from local optima:
The methods we have seen so far are iterative improvement methods, that is, they get stuck in local optima. Simple mechanisms for escaping from local optima: I Restart: re-initialise search whenever a
A Portfolio Solver for Answer Set Programming: Preliminary Report
A Portfolio Solver for Answer Set Programming: Preliminary Report M. Gebser, R. Kaminski, B. Kaufmann, T. Schaub, M. Schneider, and S. Ziller Institut für Informatik, Universität Potsdam Abstract. We propose
Programming by Optimisation:
Programming by Optimisation: Towards a new Paradigm for Developing High-Performance Software Holger H. Hoos BETA Lab Department of Computer Science University of British Columbia Canada ICAPS 2013 Rome,
Non-deterministic Search techniques. Emma Hart
Non-deterministic Search techniques Emma Hart Why do local search? Many real problems are too hard to solve with exact (deterministic) techniques Modern, non-deterministic techniques offer ways of getting
n Informally: n How to form solutions n How to traverse the search space n Systematic: guarantee completeness
Advanced Search Applications: Combinatorial Optimization Scheduling Algorithms: Stochastic Local Search and others Analyses: Phase transitions, structural analysis, statistical models Combinatorial Problems
Testing the Impact of Parameter Tuning on a Variant of IPOP-CMA-ES with a Bounded Maximum Population Size on the Noiseless BBOB Testbed
Testing the Impact of Parameter Tuning on a Variant of IPOP-CMA-ES with a Bounded Maximum Population Size on the Noiseless BBOB Testbed ABSTRACT Tianjun Liao IRIDIA, CoDE, Université Libre de Bruxelles
Automatic solver configuration using SMAC
Automatic solver configuration using SMAC How to boost performance of your SAT solver? Marius Lindauer 1 University of Freiburg SAT Industrial Day 2016, Bordeaux 1 Thanks to Frank Hutter! Ever looked into
CS-E3200 Discrete Models and Search
Shahab Tasharrofi Department of Information and Computer Science, Aalto University Lecture 7: Complete and local search methods for SAT Outline Algorithms for solving Boolean satisfiability problems Complete
Continuous optimization algorithms for tuning real and integer parameters of swarm intelligence algorithms
Swarm Intell (2012) 6:49 75 DOI 10.1007/s11721-011-0065-9 Continuous optimization algorithms for tuning real and integer parameters of swarm intelligence algorithms Zhi Yuan MarcoA.MontesdeOca Mauro Birattari
Iterated Robust Tabu Search for MAX-SAT
Iterated Robust Tabu Search for MAX-SAT Kevin Smyth 1, Holger H. Hoos 1, and Thomas Stützle 2 1 Department of Computer Science, University of British Columbia, Vancouver, B.C., V6T 1Z4, Canada {hoos,ksmyth}@cs.ubc.ca
Parallel Algorithm Configuration
Parallel Algorithm Configuration Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown University of British Columbia, 366 Main Mall, Vancouver BC, V6T Z4, Canada {hutter,hoos,kevinlb}@cs.ubc.ca Abstract.
Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms
Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms Frank Hutter, Youssef Hamadi, Holger H. Hoos, and Kevin Leyton-Brown University of British Columbia, 366 Main Mall, Vancouver
Hydra-MIP: Automated Algorithm Configuration and Selection for Mixed Integer Programming
Hydra-MIP: Automated Algorithm Configuration and Selection for Mixed Integer Programming Lin Xu, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown Department of Computer Science University of British
Gradient Descent. 1) S! initial state 2) Repeat: Similar to: - hill climbing with h - gradient descent over continuous space
Local Search 1 Local Search Light-memory search method No search tree; only the current state is represented! Only applicable to problems where the path is irrelevant (e.g., 8-queen), unless the path is
underlying iterative best improvement procedure based on tabu attributes. Heuristic Optimization
Tabu Search Key idea: Use aspects of search history (memory) to escape from local minima. Simple Tabu Search: I Associate tabu attributes with candidate solutions or solution components. I Forbid steps
Université Libre de Bruxelles
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle F-Race and iterated F-Race: An overview Mauro Birattari, Zhi Yuan, Prasanna Balaprakash,
AI-Augmented Algorithms
AI-Augmented Algorithms How I Learned to Stop Worrying and Love Choice Lars Kotthoff University of Wyoming larsko@uwyo.edu Boulder, 16 January 2019 Outline Big Picture Motivation Choosing Algorithms Tuning
Tuning Algorithms for Tackling Large Instances: An Experimental Protocol.
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Tuning Algorithms for Tackling Large Instances: An Experimental Protocol. Franco
Identifying Key Algorithm Parameters and Instance Features using Forward Selection
Identifying Key Algorithm Parameters and Instance Features using Forward Selection Frank Hutter, Holger H. Hoos and Kevin Leyton-Brown University of British Columbia, 66 Main Mall, Vancouver BC, V6T Z,
Handbook of Constraint Programming 245 Edited by F. Rossi, P. van Beek and T. Walsh c 2006 Elsevier All rights reserved
Handbook of Constraint Programming 245 Edited by F. Rossi, P. van Beek and T. Walsh c 2006 Elsevier All rights reserved Chapter 8 Local Search Methods Holger H. Hoos and Edward Tsang Local search is one
Local Search for CSPs
Local Search for CSPs Alan Mackworth UBC CS CSP February, 0 Textbook. Lecture Overview Domain splitting: recap, more details & pseudocode Local Search Time-permitting: Stochastic Local Search (start) Searching
The Mesh Adaptive Direct Search Algorithm for Discrete Blackbox Optimization
The Mesh Adaptive Direct Search Algorithm for Discrete Blackbox Optimization Sébastien Le Digabel Charles Audet Christophe Tribes GERAD and École Polytechnique de Montréal ICCOPT, Tokyo 2016 08 08 BBO:
Ordered Racing Protocols for Automatically Configuring Algorithms for Scaling Performance
Ordered Racing Protocols for Automatically Configuring Algorithms for Scaling Performance James Styles University of British Columbia jastyles@cs.ubc.ca Holger Hoos University of British Columbia hoos@cs.ubc.ca
2. Blackbox hyperparameter optimization and AutoML
AutoML 2017. Automatic Selection, Configuration & Composition of ML Algorithms. at ECML PKDD 2017, Skopje. 2. Blackbox hyperparameter optimization and AutoML Pavel Brazdil, Frank Hutter, Holger Hoos, Joaquin
Massively Parallel Seesaw Search for MAX-SAT
Massively Parallel Seesaw Search for MAX-SAT Harshad Paradkar Rochester Institute of Technology hp7212@rit.edu Prof. Alan Kaminsky (Advisor) Rochester Institute of Technology ark@cs.rit.edu Abstract The
Automatically Improving the Anytime Behaviour of Optimisation Algorithms
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Automatically Improving the Anytime Behaviour of Optimisation Algorithms Manuel
Stochastic Local Search Methods for Dynamic SAT an Initial Investigation
Stochastic Local Search Methods for Dynamic SAT an Initial Investigation Holger H. Hoos and Kevin O Neill Abstract. We introduce the dynamic SAT problem, a generalisation of the satisfiability problem
Integrating Probabilistic Reasoning with Constraint Satisfaction
Integrating Probabilistic Reasoning with Constraint Satisfaction IJCAI Tutorial #7 Instructor: Eric I. Hsu July 17, 2011 http://www.cs.toronto.edu/~eihsu/tutorial7 Bottom Line Up Front (BLUF) Thesis: The
WalkSAT: Solving Boolean Satisfiability via Stochastic Search
WalkSAT: Solving Boolean Satisfiability via Stochastic Search Connor Adsit cda8519@rit.edu Kevin Bradley kmb3398@rit.edu December 10, 2014 Christian Heinrich cah2792@rit.edu Contents 1 Overview 1 2 Research
Parameter Adjustment Based on Performance Prediction: Towards an Instance-Aware Problem Solver
Parameter Adjustment Based on Performance Prediction: Towards an Instance-Aware Problem Solver Frank Hutter Department of Computer Science University of British Columbia 2366 Main Mall Vancouver, BC, V6T
The COMPSET Algorithm for Subset Selection
The COMPSET Algorithm for Subset Selection Yaniv Hamo and Shaul Markovitch {hamo,shaulm}@cs.technion.ac.il Computer Science Department, Technion, Haifa 32000, Israel Abstract Subset selection problems
Parallel Stochastic Gradient Descent
University of Montreal August 11th, 2007 CIAR Summer School - Toronto Stochastic Gradient Descent Cost to optimize: E z [C(θ, z)] with θ the parameters and z a training point. Stochastic gradient: θ t+1
An Adaptive Noise Mechanism for WalkSAT
From: AAAI-02 Proceedings. Copyright 2002, AAAI (www.aaai.org). All rights reserved. An Adaptive Noise Mechanism for WalkSAT Holger H. Hoos University of British Columbia Computer Science Department 2366
arxiv: v2 [cs.ai] 11 Oct 2018
![arxiv: v2 [cs.ai] 11 Oct 2018](/thumbs/90/102214913.jpg "arxiv: v2 [cs.ai] 11 Oct 2018")
Journal of Artificial Intelligence Research X (XXX) X-XX Submitted X/XX; published X/XX Pitfalls and Best Practices in Algorithm Configuration Katharina Eggensperger Marius Lindauer Frank Hutter University
Automatic Configuration of Sequential Planning Portfolios
Automatic Configuration of Sequential Planning Portfolios Jendrik Seipp and Silvan Sievers and Malte Helmert University of Basel Basel, Switzerland {jendrik.seipp,silvan.sievers,malte.helmert}@unibas.ch
The MAX-SAX Problems
STOCHASTIC LOCAL SEARCH FOUNDATION AND APPLICATION MAX-SAT & MAX-CSP Presented by: Wei-Lwun Lu 1 The MAX-SAX Problems MAX-SAT is the optimization variant of SAT. Unweighted MAX-SAT: Finds a variable assignment
Predicting the runtime of combinatorial problems CS6780: Machine Learning Project Final Report
Predicting the runtime of combinatorial problems CS6780: Machine Learning Project Final Report Eoin O Mahony December 16, 2010 Abstract Solving combinatorial problems often requires expert knowledge. These
The Impact of Initial Designs on the Performance of MATSuMoTo on the Noiseless BBOB-2015 Testbed: A Preliminary Study
The Impact of Initial Designs on the Performance of MATSuMoTo on the Noiseless BBOB-2015 Testbed: A Preliminary Study Dimo Brockhoff INRIA Lille Nord Europe Bernd Bischl LMU München Tobias Wagner TU Dortmund
From Stochastic Search to Programming by Optimisation:
From Stochastic Search to Programming by Optimisation: My Quest for Automating the Design of High-Performance Algorithms Holger H. Hoos Department of Computer Science University of British Columbia Canada
Outline. TABU search and Iterated Local Search classical OR methods. Traveling Salesman Problem (TSP) 2-opt
TABU search and Iterated Local Search classical OR methods Outline TSP optimization problem Tabu Search (TS) (most important) Iterated Local Search (ILS) tks@imm.dtu.dk Informatics and Mathematical Modeling
TABU search and Iterated Local Search classical OR methods
TABU search and Iterated Local Search classical OR methods tks@imm.dtu.dk Informatics and Mathematical Modeling Technical University of Denmark 1 Outline TSP optimization problem Tabu Search (TS) (most
Pre-scheduled and adaptive parameter variation in MAX-MIN Ant System
Pre-scheduled and adaptive parameter variation in MAX-MIN Ant System Michael Maur, Manuel López-Ibáñez, and Thomas Stützle Abstract MAX-MIN Ant System (MMAS) is an ant colony optimization (ACO) algorithm
Simulated Annealing. Slides based on lecture by Van Larhoven
Simulated Annealing Slides based on lecture by Van Larhoven Iterative Improvement 1 General method to solve combinatorial optimization problems Principle: Start with initial configuration Repeatedly search
Programming by Optimisation:
Programming by Optimisation: A Practical Paradigm for Computer-Aided Algorithm Design Holger H. Hoos & Frank Hutter Department of Computer Science University of British Columbia Canada Department of Computer
Off-line vs. On-line Tuning: A Study on MAX MIN Ant System for the TSP
Off-line vs. On-line Tuning: A Study on MAX MIN Ant System for the TSP Paola Pellegrini 1,ThomasStützle 2, and Mauro Birattari 2 1 Dipartimento di Matematica Applicata Università Ca Foscari Venezia, Venezia,
Ant Colony Optimization Exercises
Outline DM6 HEURISTICS FOR COMBINATORIAL OPTIMIZATION Lecture 11 Ant Colony Optimization Exercises Ant Colony Optimization: the Metaheuristic Application Examples Connection between ACO and other Metaheuristics
Empirical Methods for the Analysis of Optimization Heuristics
Empirical Methods for the Analysis of Optimization Heuristics Marco Chiarandini Department of Mathematics and Computer Science University of Southern Denmark, Odense, Denmark www.imada.sdu.dk/~marco www.imada.sdu.dk/~marco/comisef08
Outline of the module
Evolutionary and Heuristic Optimisation (ITNPD8) Lecture 2: Heuristics and Metaheuristics Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/ Computing Science and Mathematics, School of Natural Sciences University
CS264: Beyond Worst-Case Analysis Lecture #20: Application-Specific Algorithm Selection
CS264: Beyond Worst-Case Analysis Lecture #20: Application-Specific Algorithm Selection Tim Roughgarden March 16, 2017 1 Introduction A major theme of CS264 is to use theory to obtain accurate guidance
Université Libre de Bruxelles
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Adaptive Sample Size and Importance Sampling in Estimation-based Local Search
Designing a Portfolio of Parameter Configurations for Online Algorithm Selection
Designing a Portfolio of Parameter Configurations for Online Algorithm Selection Aldy Gunawan and Hoong Chuin Lau and Mustafa Mısır Singapore Management University 80 Stamford Road Singapore 178902 {aldygunawan,
A Brief Look at Optimization
A Brief Look at Optimization CSC 412/2506 Tutorial David Madras January 18, 2018 Slides adapted from last year s version Overview Introduction Classes of optimization problems Linear programming Steepest
Machine Learning on Physical Robots
Machine Learning on Physical Robots Alfred P. Sloan Research Fellow Department or Computer Sciences The University of Texas at Austin Research Question To what degree can autonomous intelligent agents
Classical Gradient Methods
Classical Gradient Methods Note simultaneous course at AMSI (math) summer school: Nonlin. Optimization Methods (see http://wwwmaths.anu.edu.au/events/amsiss05/) Recommended textbook (Springer Verlag, 1999):
Stochastic greedy local search Chapter 7
Stochastic greedy local search Chapter 7 ICS-275 Winter 2016 Example: 8-queen problem Main elements Choose a full assignment and iteratively improve it towards a solution Requires a cost function: number
Outline. No Free Lunch Theorems SMTWTP. Outline DM812 METAHEURISTICS
DM812 METAHEURISTICS Outline Lecture 9 Marco Chiarandini 1. Department of Mathematics and Computer Science University of Southern Denmark, Odense, Denmark 2. Outline 1. 2. Linear permutations
Single Candidate Methods
Single Candidate Methods In Heuristic Optimization Based on: [3] S. Luke, "Essentials of Metaheuristics," [Online]. Available: http://cs.gmu.edu/~sean/book/metaheuristics/essentials.pdf. [Accessed 11 May
CS227: Assignment 1 Report
1 CS227: Assignment 1 Report Lei Huang and Lawson Wong April 20, 2008 1 Introduction Propositional satisfiability (SAT) problems have been of great historical and practical significance in AI. Despite
Reactive Stochastic Local Search Algorithms for the Genomic Median Problem
Reactive Stochastic Local Search Algorithms for the Genomic Median Problem Renaud Lenne 1,2, Christine Solnon 1, Thomas Stützle 2, Eric Tannier 3, and Mauro Birattari 2 1 LIRIS, UMR CNRS 5205, Université
SLS Algorithms. 2.1 Iterative Improvement (revisited)
SLS Algorithms Stochastic local search (SLS) has become a widely accepted approach to solving hard combinatorial optimisation problems. An important characteristic of many recently developed SLS methods
Sudoku is a logic puzzle where one has to fill a grid with
1 A Hybrid Heuristic Approach for the Sudoku problem Nysret Musliu and Felix Winter Vienna University of Technology corresponding author Abstract Sudoku is not only a popular puzzle but also an interesting
Ant Colony Optimization
DM841 DISCRETE OPTIMIZATION Part 2 Heuristics Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. earch 2. Context Inspiration from Nature 3. 4. 5.
Automatic Configuration of MIP Solver: Case Study in Vertical Flight Planning
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle Automatic Configuration of MIP Solver: Case Study in Vertical Flight Planning
ReACT: Real-time Algorithm Configuration through Tournaments
ReACT: Real-time Algorithm Configuration through Tournaments Tadhg Fitzgerald and Yuri Malitsky and Barry O Sullivan Kevin Tierney Insight Centre for Data Analytics Department of Business Information Systems
Discovery of the Source of Contaminant Release
Discovery of the Source of Contaminant Release Devina Sanjaya 1 Henry Qin Introduction Computer ability to model contaminant release events and predict the source of release in real time is crucial in
INF Biologically inspired computing Lecture 1: Marsland chapter 9.1, Optimization and Search Jim Tørresen
INF3490 - Biologically inspired computing Lecture 1: Marsland chapter 9.1, 9.4-9.6 2017 Optimization and Search Jim Tørresen Optimization and Search 2 Optimization and Search Methods (selection) 1. Exhaustive
EECS 219C: Computer-Aided Verification Boolean Satisfiability Solving. Sanjit A. Seshia EECS, UC Berkeley
EECS 219C: Computer-Aided Verification Boolean Satisfiability Solving Sanjit A. Seshia EECS, UC Berkeley Project Proposals Due Friday, February 13 on bcourses Will discuss project topics on Monday Instructions
Université Libre de Bruxelles
Université Libre de Bruxelles Institut de Recherches Interdisciplinaires et de Développements en Intelligence Artificielle An Incremental Particle Swarm for Large-Scale Continuous Optimization Problems:
Satisfiability. Michail G. Lagoudakis. Department of Computer Science Duke University Durham, NC SATISFIABILITY
Satisfiability Michail G. Lagoudakis Department of Computer Science Duke University Durham, NC 27708 COMPSCI 271 - Spring 2001 DUKE UNIVERSITY Page 1 Why SAT? Historical Reasons The first NP-COMPLETE problem
Learning to Choose Instance-Specific Macro Operators
Learning to Choose Instance-Specific Macro Operators Maher Alhossaini Department of Computer Science University of Toronto Abstract The acquisition and use of macro actions has been shown to be effective
algorithms, i.e., they attempt to construct a solution piece by piece and are not able to offer a complete solution until the end. The FM algorithm, l
The FMSAT Satisfiability Solver: Hypergraph Partitioning meets Boolean Satisfiability Arathi Ramani, Igor Markov framania, imarkovg@eecs.umich.edu February 6, 2002 Abstract This report is intended to present
EECS 219C: Formal Methods Boolean Satisfiability Solving. Sanjit A. Seshia EECS, UC Berkeley
EECS 219C: Formal Methods Boolean Satisfiability Solving Sanjit A. Seshia EECS, UC Berkeley The Boolean Satisfiability Problem (SAT) Given: A Boolean formula F(x 1, x 2, x 3,, x n ) Can F evaluate to 1