GECCO 2007 Tutorial / Evolutionary Multiobjective Optimization. Eckart Zitzler ETH Zürich. weight = 750g profit = 5.

Transcription

1 Tutorial / Evolutionary Multiobjective Optimization Tutorial on Evolutionary Multiobjective Optimization Introductory Example: The Knapsack Problem weight = 75g profit = 5 weight = 5g profit = 8 weight = 3g profit = 7 weight = g profit = 3 Eckart Zitzler, Kalyanmoy Deb Computer Engineering (TIK), ETH Zurich, Switzerland Single objective:? Multiobjective: choose subset that maximizes overall profit w.r.t. a weight limit choose subset that maximizes overall profit minimizes overall weight Computer Engineering and Networks Laboratory The Search Space The Trade-off Front Observations: some solutions ( ) are better than others ( ) profit profit g g 5g 2g 25g 3g 35g weight 5g g 5g 2g 25g 3g 35g weight Copyright is held by the author/owner(s). GECCO 7, July 7, 27, London, England, United Kingdom. ACM /7/

2 The Trade-off Front Tutorial / Evolutionary Multiobjective Optimization The Trade-off Front Observations: there is no single optimal solution, but some solutions ( ) are better than others ( ) Observations: there is no single optimal solution, but some solutions ( ) are better than others ( ) profit profit 2 2 selecting a solution 5 finding the good solutions 5 finding the good solutions 5 5 5g g 5g 2g 25g 3g 35g weight 5g g 5g 2g 25g 3g 35g weight Decision Making: Selecting a Solution Decision Making: Selecting a Solution Approaches: profit more important than cost (ranking) Approaches: profit more important than cost (ranking) weight must not exceed 24g (constraint) profit profit too heavy 5 5 5g g 5g 2g 25g 3g 35g weight 5g g 5g 2g 25g 3g 35g weight 3793

3 When to Make the Decision Tutorial / Evolutionary Multiobjective Optimization When to Make the Decision Before Optimization: Before Optimization: ranks objectives, defines constraints, ranks objectives, defines constraints, searches for one (green) solution searches for one (green) solution 2 profit 5 too heavy 5 weight 5g g 5g 2g 25g 3g 35g When to Make the Decision Outline Before Optimization: ranks objectives, defines constraints, searches for one (green) solution After Optimization: searches for a set of (green) solutions selects one solution considering constraints, etc.. Introduction: Why multiple objectives make a difference 2. Basic Principles: 3. Algorithm Design: Do it yourself 4. Performance Assessment: decision making often easier EAs well suited 5. Applications Domains: Where EMO is useful 6. Further Information: 3794

4 Optimization Problem: Definition Tutorial / Evolutionary Multiobjective Optimization Objective Functions A general optimization problem is given by a quadruple (X, Z, f, rel) X denotes the decision space containing the elements among which the best is sought; elements of are called decision vectors or simply solutions; Z denotes the objective space, the space within which the decision vectors are evaluated and compared to each other; elements of Z are denoted as objective vectors; f represents a function f: X Z vector a corresponding objective vector; f is usually neither injective nor surjective; rel is a binary relation over Z, i.e., rel Z Z, which represents a partial order over Z. Usually, consists of one or several functions,..., f n assign each solution a real number. Such a function f i : X cost, size, execution time, etc. In the case of a single objective function (n=), the problem is denoted as a single-objective optimization problem; a multiobjective optimization problem involves several (n 2) objective functions: single objective performance cost multiple objectives performance Comparing Objective Vectors Preference Structures The pair (Z, rel) forms a partially ordered set, i.e., for any two objective vectors a, b Z a and b are equal: a rel b and b rel a a is better than b: a rel b and not (b rel a) a is worse than b: not (a rel b) and b rel a a and b are incomparable: neither a rel b nor b rel a Example: Z = 2, (a, a 2 ) rel (b, b 2 ) : a b a 2 b 2 incomparable better worse incomparable Often, (Z, rel) is a totally ordered set, i.e., for all a, b Z either a rel b or b rel a or both holds (no incomparable elements). The function f together with the partially ordered set (Z, rel) a preference structure on the decision space X solutions the decision maker / user prefers to other solutions: x prefrel x 2 : f(x ) rel f(x 2 ) One says: Two solutions x, x 2 are equal iff = x 2 ; A solution is indifferent to a solution iff and prefrel x and x x 2 ; A solution x is preferred to a solution x 2 iff x prefrel x 2 ; A solution x is strictly preferred to a solution x 2 iff x prefrel x 2 and not ( ); A solution x is incomparable to a solution iff neither x 2 nor x 2 prefrel x. 3795

5 The Notion of Optimality Tutorial / Evolutionary Multiobjective Optimization Pareto Dominance A solution x X S X no solution x S is strictly preferred to x, i.e., for all x S: x prefrel x x prefrel x. In other words, f(x) is a minimal element of f(s) regarding the partially ordered set (Z, rel). optimal solutions w.r.t. X x x 2 f minimal element 4 Assumption: n f i : X Z = n all objectives are to be maximized Usually considered relation: weak Pareto dominance (X, n, (f,..., f n ), ) weak Pareto dominance: Pareto dominance: strict version of weak Pareto dominance x 3 x 4 f 6 Illustration of Pareto Optimality Decision and Objective Space Maximize (,,, yk) = f(x, x2,, xn) Pareto set non-optimal decision vector Pareto front non-optimal objective vector Pareto optimal = not dominated x2 decision space objective space better incomparable dominated worse incomparable x (x, x2,, xn) f (,,, yk) Pareto(-optimal) set search evaluation 3796

6 Pareto Set Approximations Tutorial / Evolutionary Multiobjective Optimization What Is the Optimization Goal? Pareto set approximation (algorithm outcome) = set of incomparable solutions performance B D A C cheapness A C A B is better than B = not worse in all objectives and sets not equal dominates D = better in at least one objective strictly dominates C = better in all objectives is incomparable to C = neither set weakly better Find all Pareto-optimal solutions? Impossible in continuous search spaces How should the decision maker handle solutions? Find a representative subset of the Pareto set? Many problems are NP-hard What does representative actually mean? Find a good approximation of the Pareto set? What is a good approximation? How to formalize intuitive understanding: close to the Pareto front well distributed Preference Information Outline Preference information (here) = any additional information that refines the dominance relation on approximation sets (partial order total order). Introduction: Example: optimization goal = maximize size S of dominated objective space S(A) S(A) = 6% A 2. Basic Principles: 3. Algorithm Design: 4. Performance Assessment: Note: about the decision maker s preferences (limited memory, selection) 5. Applications Domains: 6. Further Information: 3797

7 Design Choices Tutorial / Evolutionary Multiobjective Optimization Ranking Solutions representation fitness assignment mating selection aggregation-based criterion-based dominance-based weighted sum VEGA SPEA2 parameters environmental selection variation operators parameter-oriented scaling-dependent set-oriented scaling-independent Example: VEGA Aggregation-Based Ranking select according to shuffle [Schaffer 985] multiple objectives (,,, yk) parameters transformation single objective y f T f 2 T 2 Example: weighting approach M f 3 T 3 M (w, w2,, wk) f k- T k- y = w + + wkyk f k T k population k separate selections mating pool Note: weights need to be varied during the run

8 Example: Weighted Sum Tutorial / Evolutionary Multiobjective Optimization Example: Multistart Constraint Method x 2 f x 2 f 2 Underlying concept: Convert all objectives except of one into constraints Adaptively vary constraints x Aggregation x maximize f f x 2 f x 2 f x 2 x x x.75 f +.25 f 2.5 f +.5 f 2.25 f +.75 f 2 feasible region constraint Example: Multistart Constraint Method Example: Multistart Constraint Method Underlying concept: Convert all objectives except of one into constraints Adaptively vary constraints Underlying concept: Convert all objectives except of one into constraints Adaptively vary constraints maximize f maximize f feasible region feasible region constraint constraint feasible region constraint 3799

9 Example: Multistart Constraint Method (Cont d) Tutorial / Evolutionary Multiobjective Optimization Dominance-Based Ranking [Laumanns et al. 26] f is the objective to optimize The boxes are defined by constraints on f 2 and f 3 Types of information: dominance rank dominance count dominance depth individual dominated? how many individuals does an individual dominate? at which front is an individual located? Examples: MOGA, NPGA NSGA/NSGA-II SPEA/SPEA2 dominance count + rank Example: MOGA and SPEA2 Refining Rankings 3 MOGA [Fonseca, Fleming 993] R (raw fitness) = #dominating solutions 2 SPEA2 [Zitzler et al. 22] S (strength) = #dominated solutions R (raw fitness) = strengths of dominators ranks pure dominance rank refined ranking no selection pressure within equivalence classes density information based on Euclidean distance modified dominance relation 38

10 Methods Based On Euclidean Distance Tutorial / Evolutionary Multiobjective Optimization Computation Effort Versus Accuracy Density estimation techniques: [Silverman 986] Kernel Nearest neighbor Histogram MOGA PAES density estimate = sum of f values where f is a function of the distance density estimate = volume of the sphere defined by the nearest neighbor density estimate = number of solutions in the same box Two Nearest Neighbor Variants Objective-Wise Euclidean Distance NSGA-II SPEA2 f f f faster good for 2 objectives slower good for 3 objectives and more [Deb et al. 22] [Zitzler et al. 22] The Problem of Deterioration Refinement of Dominance Relations Observation: The use of Euclidean distance can lead to deterioration Integration of Goals, Priorities, Constraints: [Fonseca, Fleming 998] A is preferable over B Continuous dominance relations : I + (A,B) = min i f i (A) f i (B) A Knapsack problem [Laumanns et al. 22] I + (A,B) and I + (B,A) A dominates B (binary additive epsilon quality indicator) B 38

11 Example: IBEA Tutorial / Evolutionary Multiobjective Optimization Example: IBEA (Cont d) Question: How to continuous dominance relations for fitness Künzli 24] function I (binary quality indicator) with A dominates B I(A, B) < I(B, A) Idea: measure for loss in quality if A is removed Fitness:...corresponds to continuous extension of dominance rank...blurrs influence of dominating and dominated individuals Fitness assignment: O(n 2 ) Fitness: parameter is problem- and indicator-dependent no additional diversity preservation mechanism Mating selection: O(n) binary tournament selection, fitness values constant Environmental selection: O(n 2 ) iteratively remove individual with lowest fitness update fitness values of remaining individuals after each deletion Further Design Aspects Constraint Handling & Multiple Objectives Constraint handling: How to integrate constraints into fitness assignment? Archiving / environmental selection: How to keep a good approximation? Hybridization: How to integrate, e.g., local search in a multiobjective EA? penalty functions Add penalty term to fitness constraints as objectives Introduce additional objective(s) modified dominance extend to infeasible solutions Preference articulation: How to focus the search on interesting regions? Robustness and uncertainty: How to account for variations in the objective function values? Data structures: How to support, e.g., fast dominance checks? overall constraint violation constraints treated separately [Michalewicz 992] [Wright, Loosemore 2]? [Coello 2] [Deb 2] [Fonseca, Fleming 998] 382

12 Archiving / Environmental Selection Tutorial / Evolutionary Multiobjective Optimization Outline Variant : Variant 2: old population offspring old population offspring archive. Introduction: 2. Basic Principles: new population new population new archive 3. Algorithm Design: deterministic truncation archive = only nondominated solutions Additional selection criteria: Density information / other preferences Time Chance 4. Performance Assessment: 5. Applications Domains: 6. Further Information: Once Upon a Time... Performance Assessment: Approaches... multiobjective EAs were mainly compared visually: Theoretically (by analysis): difficult Limit behavior (unlimited run-time resources) Running time analysis Empirically (by simulation): standard Problems: Issues: randomness, multiple objectives quality measures, statistical testing, visualization, benchmark problems, parameter settings, ZDT6 benchmark problem: IBEA, SPEA2, NSGA-II 383

13 Two Approaches for Empirical Studies Tutorial / Evolutionary Multiobjective Optimization Empirical Attainment Functions Attainment function approach: Applies statistical tests directly to the samples of approximation sets Gives detailed information about how and where performance differences occur Quality indicator approach: First, reduces each approximation set to a single value of quality Applies statistical tests to the samples of quality values three runs of two multiobjective optimizers frequency of attaining regions Attainment Plots Attainment Function Analysis 5% attainment surface for IBEA, SPEA2, NSGA2 (ZDT6) A Kolmogorov-Smirnov test examines the maximum difference between two cumulative distribution functions.35.3 A KS-like test can be used to probe differences between the empirical attainment functions of a pair of optimizers, A B.25.2 The null hypothesis is that the attainment functions of A and B are identical The alternative hypothesis is that the distributions differ somewhere [Fonseca et al. 2] 384

14 Statistical Assessment Tutorial / Evolutionary Multiobjective Optimization Quality Indicator Approach Goal: compare two Pareto set approximations A and B ZDT6 IBEA NSGA-II significant difference (p=) IBEA SPEA2 significant difference (p=) Knapsack IBEA NSGA-II no significant difference IBEA SPEA2 no significant difference B A A B hypervolume distance diversity spread cardinality 6 5 A better SPEA2 NSGA-II significant difference (p=) SPEA2 NSGA-II no significant difference Comparison method C = quality measure(s) + Boolean function A, B quality R n Boolean measure function reduction interpretation statement Example: -Quality Indicator Power of Unary Quality Indicators Two solutions: E(a,b) = max i n min f i ( ) f i ( ) Two approximations: E(A,B) = max b B a A ab Important: [Zitzler et al. 23] E(a,b) = 2 E(b,a) = ½ E(A,B) = 2 E(, ) = ½ 2 a b 2 A B Unary quality indicator: [Zitzler et al. 23] strictly better not weakly better not better weakly better 385

15 Tutorial / Evolutionary Multiobjective Optimization Example: Box Plots Example: Box Plots epsilon indicator hypervolume R indicator epsilon indicator hypervolume R indicator IBEA NSGA-II SPEA2 IBEA NSGA-II SPEA2 IBEA NSGA-II SPEA2 IBEA NSGA2 SPEA2 IBEA NSGA2 SPEA2 IBEA NSGA2 SPEA2 DTLZ DTLZ Knapsack Knapsack ZDT ZDT [Fonseca et al. 25] [Fonseca et al. 25] Statistical Assessment (Kruskal( Test) Performance Assessment Tools is better than IBEA NSGA2 SPEA2 IBEA ZDT6 Epsilon NSGA2 ~ SPEA2 ~ ~ Overall p-value = e-7. Null hypothesis rejected (alpha.5) is better than IBEA NSGA2 SPEA2 IBEA DTLZ2 R NSGA2 ~ ~ SPEA2 ~ Overall p-value = e-7. Null hypothesis rejected (alpha.5) Knapsack/Hypervolume: H = No significance of any differences Reference set calculation Attainment function calculation Indicators Statistical testing procedures eaf eaf-test runs bound normalize indicators (eps, hyp, r) Surface Plots Comparison filter statistics (fisher, kruskal, mann, wilcoxon) Population Plots Box Plots Comparison 386

16 Outline Tutorial / Evolutionary Multiobjective Optimization Design Space Exploration. Introduction: Specification Optimization Evaluation Implementation 2. Basic Principles: 3. Algorithm Design: Mutation Mutation Überleben Überleben Stromverbrauch Latenz f x 2 4. Performance Assessment: Rekombination Rekombination x Fortpflanzung Fortpflanzung Kosten 5. Applications Domains: 6. Further Information: Examples: computer design, biological experiment design, etc. Example Applications Application: Genetic Programming Architecture exploration: min. cost max. performance min. power consumption Problem: Trees grow rapidly Premature convergence Overfitting of training data Multiobjective approach: Optimize both error and size error Genetic marker selection: min. cost max. sensitivity [Hubley, Zitzler, Roach 23] SNPs Heuristic SPEA2 # SPEA2 #2 SPEA2 #3 SPEA2 #4 [Eisenring, Thiele, Zitzler 2] b b 2 b 3 b 4 b 5 b 6 b 7 b 8 b 9 b f 98. f2 6 snp#: 4 f 6.4 f2 7 snp#: 35 f f2 29 snp#: 36 f 5.6 f2 62 snp#: 43 f f2 8 snp#: 85 Common approaches: Constraint (tree size limitation) Penalty term (parsimony pressure) Objective ranking (size post-optimization) Structure-based (ADF, etc.) tree size Keep and optimize small trees (potential building blocks) 387

17 Multiobjective GP: Results Tutorial / Evolutionary Multiobjective Optimization Outline Multiobjective approach (SPEA2) can find a correct solution with higher probability a correct solution slightly faster more compact (correct) solutions than alternative approaches on even-parity problem. [Bleuler et al. 2]. Introduction: Why multiple objectives make a difference 2. Basic Principles: Terms you need to know 3. Algorithm Design: Do it yourself 4. Performance Assessment: Once upon a time 5. Applications Domains: Where EMO is useful 6. Further Information: What else The Concept of PISA PISA: Implementation SPEA2 Algorithms Applications knapsack shared file file system NSGA-II TSP selector process text files variator process PAES text-based Platform and programming language independent Interface [Bleuler et al.: 23] network design application independent: mating / environmental selection individuals are described by IDs and objective vectors handshake protocol: state / action individual IDs objective vectors parameters application dependent: variation operators stores and manages individuals 388

18 PISA Website Tutorial / Evolutionary Multiobjective Optimization PISA Example knapsack_nsga2. variator: knapsack selector: nsga2 generation: all runs The EMO Community References Links: EMO mailing list: EMO bibliography: Events: Conference on Evolutionary Multi-Criterion Optimization (EMO 29 to be held in France) Books: Multi-Objective Optimization using Evolutionary Algorithms Kalyanmoy Deb, Wiley, 2 Evolutionary Algorithms for Solving Multi Evolutionary Algorithms for Solving Multi-Objective Problems Objective Problems, Carlos A. Coello Coello, David A. Van Veldhuizen & Gary B. Lamont, Kluwer, 22 Bleuler, S., Brack, M.,Thiele, L., Zitzler, E. (2). Multiobjective Genetic Programming: Reducing Bloat Using SPEA2. CEC-2, pp Bleuler, S., Laumanns, M., Thiele, L., Zitzler, E.: PISA - A Platform and Programming Language Independent Interface for Search Algorithms. EMO 23, pp Coello, C. (2). Treating Constraints as Objectives for Single-Objective Evolutionary Optimization, Engineering Optimization, 32(3): Deb, K. (2). Multi -Objective Optimization using Evolutionary Algorithms.Wiley. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T. (22). A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, Volume 6, Issue 2, pp Eisenring, M., Thiele, L., Zitzler, E. (2). Handling Conflicting Criteria in Embedded System Design. IEEE Design & Test of Computers, Vol. 7, No. 2, pp Fonseca, C. M., Fleming, P. J. (993). Genetic algorithms for multiobjective optimization: Formulation, discussion and generalization. In S. Forrest (Ed.), ICGA Proceedings, pp Fonseca, C. M., Fleming, P. J. (998). Multiobjective optimization and multiple constraint handling with evolutionary algorithms part i: A unified formulation. IEEE Transactions on Systems, Man, and Cybernetics 28(), pp Fonseca, C., Knowles, J., Thiele, L., Zitzler, E. (25). A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. EMO 25. Grunert da Fonseca, V., Fonseca, C., Hall, A. (2). Inferential Performance Assessment of Stochastic Optimisers and the Attainment Function. EMO 2, pp Hubley, R., Zitzler, E., Roach, J. (23). Evolutionary algorithms for the selection of single nucleotide polymorphisms. BMC Bioinformatics, Vol. 4, No. 3. Laumanns, M., Thiele, L., Deb, K., Zitzler, E. (22). Combining Convergence and Diversity in Evolutionary Multi-objective Optimization. Evolutionary Computation, Vol., No. 3, pp Laumanns, M., Thiele, L., Zitzler, E. (26). An Efficient Metaheuristic Based on the Epsilon-Constraint Method. European Journal of Operational Research, Volume 69, Issue 3, pp Michalewicz, Z. (992). Genetic Algorithms + Data Structures = Evolution Programs. Springer. Schaffer, J. D. (985). Multiple objective optimization with vector evaluated genetic algorithms. In J. J. Grefenstette (Ed.), ICGA Proceedings, pp. 93. Silverman, B. W. (986). Density estimation for statistics and data analysis, Chapman and Hall, London. Wright, J., Loosemore, H. (2). An Infeasibility Objective for Use in Constrained Pareto Optimization. EMO 2, pp Zitzler, E., Künzli, S. (24). Indicator-Based Selection in Multiobjective Search. PPSN VIII, pp Zitzler, E., Laumanns, M,, Thiele, L. (22). SPEA2: Improving the Strength Pareto Evolutionary Algorithm for Multiobjective Optimization. Evolutionary Methods for Design, Optimisation, and Control, CIMNE, Barcelona, Spain, pages 95-. Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C., Grunert da Fonseca, V.: Performance Assessment of Multiobjective Optimizers: An Analysis and Review. IEEE Transactions on Evolutionary Computation, Vol. 7, No. 2, pages