Effective Stochastic Local Search Algorithms for Biobjective Permutation Flow-shop Problems

Effective Stochastic Local Search Algorithms for Biobjective Permutation Flow-shop Problems Jérémie Dubois-Lacoste, Manuel López-Ibáñez, and Thomas Stützle IRIDIA, CoDE, Université Libre de Bruxelles Brussels, Belgium stuetzle@ulb.ac.be iridia.ulb.ac.be/ stuetzle

Outline 1. Stochastic local search 2. Simple SLS method: IG Iterated Greedy for PFSP 3. Multi-objective Optimization Hybrid TPLS+PLS for biobjective PFSPs 4. Automatic Algorithm Configuration Automatic Configuration of Hybrid TPLS+PLS Algorithm

Combinatorial optimisation problems Examples finding minimum cost schedule to deliver goods finding optimal sequence of jobs in production line finding best allocation of flight crews to airplanes finding a best routing for Internet data packets... and many more Few facts arise in many real-world applications many have high computational complexity (N P-hard) in research, often abstract versions of real-world problems are treated

Search paradigms Systematic search traverse search space of instances in systematic manner complete: guaranteed to find optimal solution in finite amount of time (plus proof of optimality) Local search start at some initial solution iteratively move from search position to neighbouring one incomplete: not guaranteed to find optimal solutions

Search paradigms Perturbative (local) search search space = complete candidate solutions search step = modification of one or more solution components example: 2-opt algorithm for TSP

Search paradigms Constructive (local) search search space = partial candidate solutions search step = extension with one or more solution components example: nearest neighbor heuristic for TSP

Stochastic local search global view vertices: candidate solutions (search positions) edges: connect neighbouring positions s s: (optimal) solution c c: current search position

Stochastic local search local view s next search position is selected from local neighbourhood based on local information.

Stochastic local search (SLS) SLS algorithm defined through search space S set of solutions neighborhood relation finite set of memory states initialization function step function termination predicate evaluation function (for a formal definition see SLS:FA, Hoos & Stützle, 2005)

A simple SLS algorithm Iterative improvement start from some initial solution iteratively move from the current solution to an improving neighbouring one as long as such one exists Main problem getting stuck in local optima Solution general purpose SLS methods (aka metaheuristics) that direct the search and allow escapes from local optima

SLS methods (metaheuristics) modify neighbourhoods variable neighbourhood search accept occasionally worse neighbours simulated annealing tabu search modify evaluation function dynamic local search generate new (starting) solutions (for local search) EAs / memetic algorithms ant colony optimization iterated local search iterated greedy

Iterated Greedy Key Idea: iterate over greedy construction heuristics through destruction and construction phases Motivation: start solution construction from partial solutions to avoid reconstruction from scratch keep features of the best solutions to improve solution quality if few construction steps are to be executed, greedy heuristics are fast adding a subsidiary local search phase may further improve performance

Iterated Greedy (IG): While termination criterion is not satisfied: generate candidate solution s using greedy constructive search While termination criterion is not satisfied: r := s apply solution destruction on s perform greedy constructive search on s perform local search on s based on acceptance criterion, keep s or revert to s := r Note: local search after solution reconstruction can substantially improve performance

IG main issues destruction phase fixed vs. variable size of destruction stochastic vs. deterministic destruction uniform vs. biased destruction construction phase not every construction heuristic is necessarily useful typically, adaptive construction heuristics preferable speed of the construction heuristic is an issue acceptance criterion determines tradeoff diversification intensification of the search

IG has been re-invented several times; names include simulated annealing, ruin and recreate, iterative flattening, iterative construction search, large neighborhood search,.. close relationship to iterative improvement in large neighbourhoods for some applications so far excellent results can give lead to effective combinations of tree search and local search heuristics

Permutation flow-shop problem (PFSP) M 1 J 1 J 2 J 3 J 4 J 5 M 2 J 1 J 2 J 3 J 4 J 5 M 3 J 1 J 2 J 3 J 4 J 5 0 5 10 15 20 time n jobs are to be processed on m machines (in canonical order of machines) input data: processsing times for each job on each machine and due dates of each job otherwise: usual PFSP characteristics

IG for PFSP Initial solution construction NEH heuristic Destruction heuristic randomly remove d jobs from sequence Construction heuristic follow the NEH heuristic considering jobs in random order Acceptance criterion Metropolis condition with fixed temperature

Iterative improvement for PFSP A B C D E F A C B D E F φ φ' transpose neighbourhood A B C D E F A E C D B F φ φ' exchange neighbourhood A B C D E F A C D B E F φ φ' insert neighbourhood best choice: insert; profits from speed-ups

IG for PFSP, example 7 3 4 1 8 2 5 6 Initial NEH solution, C max = 8564 3 2 ---DESTRUCTION PHASE --- 1 7 3 4 1 8 2 5 6 Choose d (3) jobs at random 7 5 3 8 2 6 1 4 Partial sequence to reconstruct Jobs to reinsert ---CONSTRUCTION PHASE --- 7 3 8 5 2 6 After reinserting job 5, C max = 7589 7 3 8 5 2 1 6 After reinserting job 1, C max = 8243 7 3 8 5 2 1 6 4 After reinserting job 4, C max = 8366

when combined with local search, IG is a state-of-the-art algorithm for permutation flow-shop scheduling Avrg. Relative Percentage Deviation (RPD) 6 5 4 3 2 1 0 Means and 95.0 Percent LSD Intervals GA_AA GA_MIT GA_RMA IG_RS IG_RS LS NEHT SA_OP GA_CHEN GA_REEV HGA_RMA ILS M-MMAS PACO SPIRIT Algorithm

Multi-objective Optimization Multiobjective Combinatorial Optimization Problems (MCOPs) many real-life problems are multiobjective timetabling and scheduling logistics and transportation telecommunications and computer networks... and many others example: objectives in PFSP makespan sum of flowtimes total weighted or unweighted tardiness

Pareto optimization multiple objective functions f(x) = (f 1 (x),..., f Q (x)) no a priori knowledge Pareto-optimality

Main SLS approaches to Pareto optimization SLS algorithms based on dominance criterion component-wise acceptance criterion example: Pareto local search (PLS) based on solving scalarizations convert MCOPs into single-objective problems min x X Q λ i f i (x) i=1 for obtaining many solution: vary weight vector λ example: two-phase local search (TPLS) hybrids of the two search models

CWAC Search Model input: candidate solution x Add x to Archive repeat Choose x from Archive X N = Neighbors(x) Add X N to Archive Filter Archive until all x in Archive are visited return Archive cost time

SAC Search Model input: weight vectors Λ for each λ Λ do x is a candidate solution x = SolveSAC(x, λ) Add x to Archive Filter Archive return Archive cost time

Hybrid Search Model input: weight vectors Λ for each λ Λ do x is a candidate solution x = SolveSAC(x, λ) X = CW(x ) Add X to Archive Filter Archive return Archive cost time

Hybrid TPLS+PLS for biobjective PFSPs Engineering an effective TPLS+PLS algorithm context: development of effective SLS algorithms MCOPS example problem: bi-objective flow-shop problems (bpfsps) steps followed: 1. knowledge of state-of-the-art 2. development of powerful single-objective algorithms 3. experimental study of TPLS components 4. experimental study of PLS components 5. design of a hybrid algorithm 6. detailed comparison to state of the art

bi-objective permutation flow-shop problem permutation flow-shop problem n jobs are to be processed on m machines (in canonical order of machines) input data: processsing times for each job on each machine and due dates of each job otherwise: usual PFSP characteristics objective functions makespan sum of flowtimes total weighted or unweighted tardiness tackle all bi-objective problems for any combination of objectives

Step 2: IG for other single-objective problems Recall: we have state-of-the-art IG algorithm for PFSP with makespan criterion (part of step 1) main adaptations for other objectives constructive heuristics to provide good initial solutions neighborhood operators for local search step number of jobs to remove acceptance criterion: formula, temperature Remark: parameters have been fine-tuned using I/F-Race.

Evaluation of IG Sum of flowtimes comparison with the state-of-the-art (Tseng and Lin, 2009): Short runs size of instance R.D. Best R.D. Mean 20x5 0.016-0.142 20x10 0.000-0.002 20x20 0.000-0.033 50x5 0.117 0.164 50x10 0.012-0.065 50x20-0.196-0.278 100x5 0.186 0.135 100x10-0.003-0.149 100x20-0.502-0.625 Average -0.041-0.110 Long runs size of instance R.D. Best R.D. Mean 50x5-0.026-0.044 50x10-0.034-0.135 50x20-0.071 0.201 Average on all sizes -0.044-0.126 more than 50 new best known solutions for instance set of 90 instances state-of-the-art results

Evaluation of IG-Weighted tardiness few studies on this criterion in the literature more than 90% of best known solutions improved for a benchmark set of 540 instances (for total tardiness), in production mode

Experimental analysis Better relation [Jaszkiewicz and Hansen 1998] cost cost time time Blue is better than Red Blue is incomparable to Red

Experimental analysis Attainment functions [Grunert da Fonseca et al. 2001] AF : Probability that an outcome an arbitrary point EAF : How many runs an outcome an arbitrary point Is EAF Blue significantly different from EAF Red? Permutation tests with Smirnov distance as test statistic

Experimental analysis Visualization of differences [Paquete 2005] EAF Blue EAF Red positive differences negative differences

Step 3: Effective TPLS algorithm Two-phase local search Phase 1: generate high quality solution for single objective problem Phase 2: solve sequence of scalarizations use solution found for previous scalarization as initial solution for the next one

Studied TPLS components f 1 1. Search strategy 2. Number of scalarizations 3. Intensification mechanism f 2 2-phase Generate (i) high quality solution for f 1 and (ii) sequence of solutions

Studied TPLS components f 1 1. Search strategy 2. Number of scalarizations 3. Intensification mechanism f 2 Restart Independent runs of SLS algorithms using different weights

Studied TPLS components f 1 1. Search strategy 2. Number of scalarizations 3. Intensification mechanism f 2

Studied TPLS components f 1 1. Search strategy 2. Number of scalarizations 3. Intensification mechanism f 2 Higher solution quality returned by SLS algorithm

Adaptive Anytime TPLS Idea: dynamically generate weights to adapt to the shape of the Pareto front focus search on the largest gap in Pareto front seed new scalarizations with solutions from previous similar scalarizations weight generation inspired from dichotomic scheme

Adaptive Anytime TPLS vs. Restart C i 6700 6800 6900 7000 7100 T i 3.9e+05 4e+05 4.1e+05 [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) 6700 6800 6900 7000 7100 C i Two Phase Restart

Step 4: Effective PLS algorithm Pareto local search iterative improvement algorithm that directly follows the CWAC search model (dominance-based acceptance criterion) studied PLS components neighborhood operators seeding the algorithm with different quality solutions

PLS: Neighborhhood operators C i 3.8e+05 3.86e+05 3.92e+05 3.98e+05 w i T i 5e+05 6e+05 7e+05 8e+05 [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) 3.8e+05 3.86e+05 3.92e+05 3.98e+05 C i Exchange Exchange+Insertion

PLS: Seeding C j 380000 400000 420000 Objectives: C max and C j 6500 6700 6900 7100 C max random set heuristic set IG set heuristic seeds IG seeds w j T j 6e+05 8e+05 1e+06 Objectives: C j and w j T j 380000 400000 420000 C j random set heuristic set IG set heuristic seeds IG seeds

Step 5: Hybrid algorithm, TPLS+PLS 1: TPLS uses roughly half of the overall time each objective and combination of objectives uses a dedicated Iterated Greedy 2: PLS both exchange and insertion operators bounded in time

Step 6: Comparison to state-of-the-art recent review (2008) by Minella et al.tests 23 algorithms for three biobjective PFSP problems. They also provide reference sets measured across all 23 algorithms often the median or even worst attainment surfaces of TPLS+PLS dominate reference sets! two state-of-the-art algorithms identified by Minella et al. multi-objective Simulated Annealing (MOSA) by Varadharajan & Rajendran, 2005 multi-objective Genetic Local Search (MOGLS) by Arroyo & Armentano, 2004 comparison to re-implementations of both algorithms; 10 runs per instance

Comparison to state-of-the-art: PFSP-(C max, SFT) nxm TPLS+PLS MOSA 20x5 4.75 6.11 20x10 3.15 9.57 20x20 0 1.7 50x5 91.99 0 50x10 78.57 0 50x20 82.88 0 100x5 85.06 0 100x10 74.5 0.03 100x20 75.91 0 200x10 26.36 0 200x20 32.84 0 Given: Percentage of times an outcome of an algorithm outperforms an outcome of the other one.

Comparison to state-of-the-art: PFSP-(C max, TT) nxm TPLS+PLS MOSA 20x5 5.74 1.36 20x10 0 0.29 20x20 0.4 1.57 50x5 84.75 0 50x10 67.35 0 50x20 69.55 0 100x5 84.61 0 100x10 73.29 0 100x20 61.99 0 200x10 37.14 0 200x20 33.04 0 Given: Percentage of times an outcome of an algorithm outperforms an outcome of the other one.

Comparison to state-of-the-art: PFSP-(SFT, TT) nxm TPLS+PLS MOSA 20x5 8.46 28 20x10 0.19 0.4 20x20 3.6 5.19 50x5 98.37 0 50x10 94.88 0 50x20 5.13 0 100x5 98.56 0.6 100x10 98.9 0 100x20 97.25 0 200x10 98.34 0.36 200x20 89.13 0.92 Given: Percentage of times an outcome of an algorithm outperforms an outcome of the other one.

Comparison to state-of-the-art: PFSP-(C max, SFT) C max 6300 6400 6500 6600 6700 6800 6900 C i 3.75e+05 3.9e+05 4e+05 [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) 6300 6450 6600 6750 6900 C max TP+PLS MOSA

Comparison to state-of-the-art: PFSP-(C max, WT) C max 6300 6400 6500 6600 6700 6800 6900 7000 w i T i 1.2e+05 1.4e+05 1.6e+05 [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) 6300 6500 6700 6900 C max TP+PLS MOSA

Comparison to state-of-the-art: PFSP-(SFT, WT) C i 3.69e+05 3.72e+05 3.75e+05 3.78e+05 w i T i 1.2e+05 1.24e+05 1.3e+05 3.69e+05 3.72e+05 3.75e+05 3.78e+05 C i TP+PLS [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) MOSA

Summary of results for PFSP single-objective problems new best-known solutions for 50 out of 90 instances from PFSP-flowtime benchmarks new best-known solutions of 90% of available benchmarks for PFSP-total-tardiness multi-objective problems hybrid algorithms clearly outperforms the two previous state-of-the-art algorithms hybrid algorithms usually outperforms the non-dominated obtained from the best results in an extensive computational study of 23 algorithms

Configuration of SLS algorithms SLS algorithm components categorical parameters type of construction method in IG choice of cross-over operator in evolutionary algorithms numerical parameters destruction strength operator application probability Configuration/design problem given an application scenario, choose categorical and numerical parameters to optimize some performance criterion finding a good configuration can be very time-consuming

Main configuration approaches Offline configuration configure algorithm before deploying it configuration done on training instances Online tuning (parameter control) adapt parameter setting while solving an instance typically limited to a set of known crucial algorithm parameters We focus on offline tuning

Importance of the configuration problem improvement over manual, ad-hoc methods for tuning reduction of development time and human intervention increase number of considerable degrees of freedom empirical studies, comparisons of algorithms support for end users of algorithms Methods for automated algorithm configuration are an important tool for engineering SLS algorithms

The configuration problem (Our) Configuration problem Given: (finite) set of candidate configurations set of training instances optimization criterion: solution quality, run-time Goal: find a best configuration (for future instances in production-mode)

The racing approach Θ start with a set of initial candidates consider a stream of instances sequentially evaluate candidates discard inferior candidates as sufficient evidence is gathered against them... repeat until a winner is selected or until computation time expires i

The F-Race algorithm Statistical testing 1. family-wise tests for differences among configurations Friedman two-way analysis of variance by ranks 2. if Friedman rejects H 0, perform pairwise comparisons to best configuration apply Friedman post-test

Sampling configurations F-race is a method for the selection of the best configuration and independent of the way the set of configurations is sampled Sampling configurations and F-race full factorial design random sampling design iterative refinement of a sampling model (iterative F-race) (Balaprakash, Birattari, Stützle, 2007; Birattari et al. 2010, López-Ibáñez et al. 2011)

Iterative F-race: an illustration sample configurations from initial distribution While not terminate() 1. apply F-Race 2. modify the distribution 3. sample configurations with selection probability

Automatic configuration of multi-objective optimizers Goal: find the best parameter settings of multi-objective optimizer to solve unseen instances of a problem, given a flexible framework for the multi-objective optimizer a set of training instances representative of the same problem. a maximum budget (number of experiments / time limit) automatic configuration tool: I/F-Race designed for single-objective optimization. I/F-Race + hypervolume = multi-objective automatic configuration

Hypervolume measure

Automatic configuration of TPLS+PLS TPLS+PLS framework multi-objective part is modular and problem-independent TPLS+PLS framework can be easily parameterized Parameter name Type Domain tpls ratio ordered {0.1, 0.2,..., 0.9, 1} init scal ratio ordered {1, 1.5, 2, 3, 4, 6, 8, 10} nb scal integer [0, 30] two seeds categorical {yes, no} restart categorical {yes, no} theta real [0, 0.5] pls operator categorical {ex, ins, exins}

Hypervolume statistics, size 50x20 conf hand conf tun rnd conf tun ic mean sd mean sd mean sd (C max, SFT) 0.974 0.036 0.982 0.038 0.984 0.034 (C max, TT) 0.999 0.039 1.005 0.038 1.002 0.035 (C max, WT) 1.037 0.026 1.045 0.024 1.045 0.023 (SFT, TT) 0.954 0.038 0.955 0.039 0.96 0.04 (SFT, WT) 1.022 0.028 1.024 0.03 1.029 0.026

Hypervolume statistics, size 100x20 conf hand conf tun rnd conf tun ic mean sd mean sd mean sd (C max, SFT) 0.943 0.058 0.968 0.056 0.971 0.058 (C max, TT) 1.005 0.043 1.008 0.045 1.012 0.038 (C max, WT) 1.013 0.043 1.028 0.039 1.025 0.04 (SFT, TT) 0.621 0.129 0.755 0.117 0.761 0.133 (SFT, WT) 0.951 0.037 0.922 0.051 0.962 0.048

Comparison hand-tuned vs. automatically configured C max 3650 3750 3850 3950 4050 4150 4250 C max 3800 3900 4000 4100 4200 4300 4400 w iti 4e+04 8e+04 1.2e+05 1.6e+05 [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) 3650 3750 3850 3950 4050 4150 4250 4e+04 8e+04 1.2e+05 1.6e+05 w iti w iti 5e+04 1e+05 1.5e+05 [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) 3800 3900 4000 4100 4200 4300 4400 5e+04 1e+05 1.5e+05 w iti C max hand tuning 1 C max hand tuning 1 C max 3800 4000 4200 4400 C max 3750 3850 3950 4050 4150 4250 w iti 4e+04 8e+04 1.2e+05 1.6e+05 [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) 3800 4000 4200 4400 4e+04 8e+04 1.2e+05 1.6e+05 w iti w iti 2e+04 6e+04 1e+05 1.4e+05 [0.8, 1.0] [0.6, 0.8) [0.4, 0.6) [0.2, 0.4) [0.0, 0.2) 3750 3850 3950 4050 4150 4250 2e+04 6e+04 1e+05 1.4e+05 w iti C max hand tuning 1 C max hand tuning 1

Conclusions automatic configuration of multi-objective optimizers well feasible new state-of-the-art algorithms for biobjective PFSPs have been obtained significant room for further research

Acknowledgments Special thanks to Mauro (F-Race), Luis (multi-objective part) for providing some of the graphics used in this talk