Scalarization versus Indicator-based Selection in Multi-Objective CMA Evolution Strategies

Transcription

1 Scalarization versus Indicator-based Selection in Multi-Objective CMA Evolution Strategies Thomas Voß Institut für Neuroinformatik Ruhr-Universität Bochum 780 Bochum, Germany Günter Rudolph Lehrstuhl für Algorithm Engineering Technische Universität Dortmund Dortmund, Germany Nicola Beume Lehrstuhl für Algorithm Engineering Technische Universität Dortmund Dortmund, Germany Christian Igel Institut für Neuroinformatik Ruhr-Universität Bochum 780 Bochum, Germany Abstract While scalarization approaches to multicriteria optimization become infeasible in the case of many objectives, for few objectives the benefits of populationbased methods compared to a set of independent singleobjective optimization trials on scalarized functions are not obvious. The multi-objective covariance matrix adaptation evolution strategy (MO-CMA-ES) is a powerful algorithm for real-valued multi-criteria optimization. This populationbased approach combines mutation and strategy adaptation from the elitist CMA-ES with multi-objective selection. We empirically compare the steady-state MO-CMA-ES with different scalarization algorithms, in which the elitist CMA-ES is used as single-objective optimizer. Although only bicriteria benchmark problems are considered, the MO-CMA-ES performs best in the overall comparison. However, if the scalarized problems have a structure that can easily be exploited by the CMA-ES and that is less apparent in the vector-valued fitness function, the CMA- ES with scalarization outperforms the population-based approach. I. INTRODUCTION The goal of multi-objective optimization (MOO) can be defined as efficiently finding a set of solutions that diversely and accurately approximates the set of Pareto-optimal solutions. Evolutionary algorithms (EAs) are particularly well-suited for MOO. This may have several reasons, but one of the most intriguing arguments is that the search in MOO for a set of solutions perfectly matches the concept of populations in EAs. In this study, we empirically investigate the role of the population dynamics in a recently proposed evolutionary MOO algorithm. Many established deterministic approaches to MOO consider the scalarization of the vector-valued problem. To find a diverse approximation of the set of Pareto-optimal solutions, the vector-valued problem is reduced to a number of different scalar-valued optimization problems, which favor different trade-offs between objectives. Prominent examples of scalarization methods are the basic weighted-sum approach and the Tchebycheff method []. In contrast, most state-of-the-art evolutionary methods for solving MOO problems more directly rely on the notion of Pareto-dominance for finding good approximations of the Pareto-optimal set. This does not explicitly need information regarding the importance of single criteria. However, selection in general requires a ranking of the individuals in the current population including the comparison of solutions that are not related in terms of Pareto-dominance. This ranking may be viewed as some kind of scalarization, which is, however, dynamic and usually for each solution dependent on the other solutions represented in the current population. In the population-based algorithm considered here, this ranking is induced by the hypervolume in the objective space covered by the solutions in the population: Selection aims at preserving those solutions that maximize the covered hypervolume. Scalarization approaches as outlined above are only feasible if there are only very few objectives, as the number of combinations of weighting coefficients expressing relative preferences for objectives grows exponentially with the number of objectives and so scales the number of single-objective optimization trials in simple scalarization methods. However, for two or three objectives the benefits of directly evolving a set of solutions compared to independent (parallel) singleobjective optimization trials on scalarized functions are not obvious. Note that there exists scalarization methods (e.g., based on the Tchebycheff metric) that do not suffer from the often mentioned problems of weightedsum approaches with non-convex MOO problems. In this study, we consider the covariance matrix

2 adaptation evolution strategy (CMA-ES [], []), which is an elaborate algorithm for real-valued optimization performing well in real-world applications as well as in benchmark scenarios [], and its multi-objective variant, the MO-CMA-ES [], []. The latter features a population-based approach combining the methods for mutation and strategy adaptation from the elitist CMA-ES with multi-objective selection. This makes the algorithm ideal for studying the influence of the population dynamics in the case of few objectives, because we can directly compare the MO-CMA-ES with singleobjective optimizations using the elitist CMA-ES. To this end, we formulate hybrid algorithms combining established scalarization methods with the CMA-ES. The remainder of this paper is organized as follows. First, two basic methods for scalarization are presented (Sec. II). In Sec. III, the single- and multi-objective elitist CMA-ESs are described and hybrid algorithms using scalarization and the CMA-ES are introduced. TheexperimentalsetupisgiveninSec.IVandresultsof the performance assessment are presented. The paper closes with conclusions and ideas for future work. II. SCALARIZATION OF MULTI-OBJECTIVE OPTIMIZATION PROBLEMS This section introduces the weighted-sum method and the method of weighted metrics for the scalarization of vector-valued optimization problems. Moreover, the ability of these methods to find Pareto-optimal solutions is briefly discussed. Real-valued, Ñ-objective optimization problems Ê Ê Ñ Ü Üµ Ñ Üµµ to be minimized in each objective are considered. A. The Weighted-Sum Method The weighted-sum method (e.g., [7], [8]) assigns a weighting coefficient to each objective function and minimizes the weighted sum of the objectives. Thus, the vector-valued optimization problem is transformed into a scalar optimization problem of the following form: ÑÑÞ Ü¾Ê Ûµ ܵ Ñ Û Üµ where Û is a weighting-vector with Û ¼ for all Ñ and È Ñ Û. B. The Method of Weighted Metrics In the method of weighted metrics, the distance between some reference point and the feasible objective regionis minimized.most often,the utopianpoint Ù is used as a reference and Ä Ô -metrics are used to measure distance. Moreover, each single objective function is See the References to CMA-ES Applications: tu-berlin.de/user/niko/cmaapplications.pdf f u f Fig.. Illustration of the Tchebycheff-method for the scalarization of a multi-objective optimization problem in the objective space. All valid points () are reachable by a rectangle centered around an utopian point Ù. Thus, even concave regions of the Pareto front (bold line) can be obtained. associated with a weighting coefficient. Thus, the multiobjective optimization problem is replaced with the following single-objective optimization problem, called the weighted Ä Ô -problem: ÑÑÞ Ü¾Ê Ä ÛÙ µ Ô Üµ Z Ñ Û Üµ Ù Ô Ô with Ô, Û ¼ for all È Ñ and Ñ Û. In the case of Ô, the metric is also called the Tchebycheff metric and the weighted Ä - or the Tchebycheff problem is of the form: ÑÑÞ Ü¾Ê ÛÙ µ ܵ ÑÜ Û Üµ Ñ Ù The utopian point Ù Þ can be set via the ideal point Þ with Þ Ñ Üµ Ü ¾ Ê and an offset, where ¾ Ê and is the Ñ-dimensional vector of all ones. C. Properties of Scalarization Methods Now we review some properties of the methods introduced previously regarding their ability to find every Pareto-optimal solution of a vector-valued optimization problem. Recall that a function Ê Ê is convex if for all Ü Ü ¾ ¾ Ê and ¼ the following equation holds Ü µü ¾ µ Ü µ µ Ü ¾ µ A set Ë Ê is convex if Ü Ü ¾ ¾ Ë implies that Ü µü ¾ ¾ Ë with ¼. A multi-objective optimization problem is convex, if all the objective functions and the feasible region are convex. Let Ê Ê Ñ be a multi-objective optimization problem with the feasible region Ë Ê. As stated in [], if is convex and Ü ¾ Ë is Pareto-optimal, then

3 there exists a weighting-vector Û ¾ Ê Ñ such that Ü is a solution of Ûµ. For the Tchebycheff-method a stronger result holds. It is able to find every Pareto-optimal solution regardless of the vector-valued optimization problem being convex. That is, let Ü ¾ Ë be Pareto-optimal. Then there exists a weighting vector Û ¾ Ê Ñ such that Ü is a solution of ÛÙ. III. COVARIANCE MATRIX ADAPTATION EVOLUTION STRATEGIES IN MULTI-OBJECTIVE OPTIMIZATION In this section, we describe the covariance matrix adaptation evolution strategy (CMA-ES). First, we review the elitist single-objective CMA-ES, and then we outline the steady-state multi-objective CMA-ES. The key idea of CMA-ES algorithms is to alter the mutation distribution such that the probability to reproduce steps in the search space that led to the actual population(i.e., produced offspring that were selected) is increased. This enables the algorithm to detect correlations between object variables and to become invariant under affine transformations of the search space. The algorithms implement several important concepts for adapting search distributions efficiently. The first one is known as derandomization meaning that the mutation distribution is adapted in a deterministic way. The second principle is cumulation, which refers to taking the search path of the population over the past generations into account, where the influence of previous steps decays exponentially. A. Nomenclature In the elitist CMA-ES, an individual µ in generation is a -tuple µ Ü µ Ô µ succ µ Ô µ µ comprising its candidate solution vector Ü µ ¾ Ê, an averaged success rate Ô µ Ù ¾ ¼, the global step size µ ¾ Ê, an evolution path Ô µ ¾ Ê, and the covariance matrix µ ¾ Ê. For the multiobjective CMA-ES, an individual µ denotes the -th individual in generation. Additionally, the following nomenclature is used: Ê Ê Ü Üµ is the scalar fitness function to be minimized. For the MO-CMA-ES, Ê Ê Ñ Ü Üµ is the vector-valuedfitness function. Æ Ñ µ is a multi-variate normal distribution with mean vector Ñ and covariance matrix. The notation Ü Æ Ñ µ denotes that random variable Ü is distributed according to Æ Ñ µ. Ü µ Ü µ ¾ Ê is the best point from that is, are broken at random. Ü µ Ü µ Ü µ Ó, for all. Ties B. Elitist Single-Objective CMA Evolution Strategy The (+)-CMA-ES combines success-based step size adaptation and elitist selection [9], [0], [] with the covariance matrix adaptation presented in []. The algorithm is described within three routines. In the main routine, (+)-CMA-ES, the new candidate solution is sampled and the parent solution µ is updateddependingon whether the new solution µ is better than µ Algorithm : (+)-CMA-ES ¼, initialize µ repeat µ µ Ü µ Ü ¼ µ Æ µ¾ µ -update µ Á Ü ¼ µ if Ü ¼ µ Ü µ then Ü µ Ü ¼ µ -update µ Ü µ Ü µ µ until stopping criterion is met Here the indicator function Á µ Ü µ is one if its argument is true and zero otherwise. Thus, Á Ü ¼ µ Ü µ is one if the last mutation has been successful and zero otherwise. After sampling the new candidate solution, the step size is updated based on the success with a learning rate Ô (¼ Ô ) using a target success rate Ô target succ. Procedure -update( Ü Ô succ Ô Ô succ ) Ô Ù ÜÔ Ô µ Ô Ù Ô Ô succ Ô succ Ô target succ Ô target succ Ô succ µ This update rule is rooted in the -success-rule proposed by[9]andis anextensionfromthe rule proposed by []. If the best new candidate solution was better than the parent individual (see main routine), the covariance matrix is updated as in the original CMA-ES (see []). Procedure -update( Ü Ô succ Ô Ü step ) if Ô succ Ô thresh then Ô Ô µô c ¾ c µ Ü step T cov µ cov Ô Ô else Ô µô T cov µ cov Ô Ô c ¾ c µ The update of the evolution path Ô depends on the value of Ô Ù. If the smoothed success rate Ô Ù is high, that is, above Ô thresh ¼, the update of the evolution path Ô is stalled. This prevents a too fast increaseof axesof when the step size is fartoo small, for example, in a linear surrounding. The constants and cov (¼ cov ) are learning rates for the evolution path and the covariance matrix, respectively.

4 Ô The factor c ¾ c µ normalizes the variance of Ô viewed as a random variable (see []). The evolution path Ô is used to update the covariance matrix. The new covariance matrix is a weighted mean of the old covariance matrix and the outer product of Ô. In the second case (line ), the second summand in the update of Ô is missing and the length of Ô shrinks. Although of minor relevance, the term c ¾ c µ (line ) compensates for this shrinking in. The (external) strategy parameters are the population size, target success probability Ô ØÖØ Ù, step size damping, success rate averaging parameter Ô, cumulation timehorizonparameter c,andcovariancematrixlearning rate cov. Default values, as given in [] and used in this paper, are: Ô ØÖØ Ù ¾ Ô ¾ Ô Ô ØÖØ Ù ¾ ÔØÖØ Ù c ¾ ¾µ cov ¾ ¾ µ Ô thresh ¼ The elements of the initial individual ¼µ are set to Ô Ù Ô target succ, Ô ¼, and Á, where Á is the identity matrix. The initial candidate solution Ü ¼µ ¾ Ê and the initial ¾ Ê must be chosen problem dependent. The optimum should presumably be within the cube Æ Ü ¼µ ¾ Ü ¼µ ¾. C. Multi-Objective CMA Evolution Strategy In the following, we briefly outline the multiobjective covariance matrix adaptation evolution strategy (MO-CMA-ES) according to [], for a detailed description and a performance evaluation on benchmark functions (in particular a comparison with real-valued NSGA-II []) we refer to []. The MO-CMA-ES relies on the non-dominated sorting selection scheme suggested by []. As in the SMS-EMOA [], the hypervolume-indicator serves as second-level sorting criterion to rank individuals on the same level of non-dominance. Let be a population, and let, ¼ be two individuals in. Let the non-dominated solutions in be denoted by ÓÑ µ ¾ ¼ ¾ ¼ where denotes the Pareto-dominance relation. The elements in ÓÑ µ are assigned a level of nondominance of. The other ranks of non-dominance are defined recursively by considering the set without the solutions with lower ranks ([]). Formally, let ÓÑ ¼ µ ÓÑ Ð µ ÓÑ Ð µ ÓÑ Ð µ, and ÓÑ Ð µ ÓÑ ÓÑ Ð µµ for Ð. For ¾ we define the level of non-dominance Ö µ to be iff ¾ ÓÑ µ. Let Ë µ be the contributing hypervolume of with respect to ¼ ¾ Ö ¼ µ Ö µ []. Moreover, let ÓØ µ be the contribution rank of with respect to its contributing hypervolume. Then the following relation is defined. ¼ Ö µ Ö ¼ µ Ö µ Ö ¼ µ ÓØ µ ÓØ ¼ µµ The standard version of the steady-state MO-CMA- ES is given in Algorithm. Algorithm : Steady-State MO-CMA-ES Ó ¼, initialize É µ repeat µ Í ÓÑ É µ µ µ É µ ¼ µ µ Ü ¼ µ Ü µ Æ µ¾ µ É µ É µ Ó ¼ µ ܼ µ ¼ µ -update -update -update µ Á ¼ µ Á É µ É µ É µ until stopping criterion is met Ü µ µ ¼ µ µ É µ ¼ µ É µ µ The parent of the single offspring is selected uniformly at random among the non-dominated solutions in the current population and an offspring is generated (line 7). The step size of the parent and its offspring are updated depending on whether the mutation was successful, that is, whether the offspring is better than the parent according to the relation É µ. Both step size and covariance matrix update are the same as in the single-objective (+)-CMA-ES described above. The best individuals in É µ sorted by É µ form the next parent generation (line, where É µ is the -th best offspring in É µ w.r.t. É µ). D. Hybrid MO-CMA-ES In this section we introduce the canonical version of an algorithm for solving vector-valued optimization problems by using the scalarization methods discussed in Sec. II and the single-objective (+)-CMA-ES. Depending on the scalarization method, the algorithm is denoted by (+)-CMA-ES or (+)-CMA-ES, where the superscript indicates whether the Tchebycheff () or the weighted sum approach () is used. a) Choosing the weight vector: We restrict ourselves to bicriteria optimization problems Ê Ê ¾ that

5 allow for a simple and efficient scheme to systematically select the weighting-vectors. Given a number of weight steps Ð a set of weighting-vectors Ï ««µ «Ð µ ¼ Ð is generated. b) Calculating the utopian point: The Tchebycheffmethod requires a utopian point Ù. Unfortunately, this point is not known in advance given a multiobjective optimization problem. Thus, we rely on the (+)-CMA-ES to calculate an estimate of the ideal point Þ and set Ù Þ for some ¼ and the Ñ-dimensional vector of ones. Let be a bicritera optimization problem, and let Ü, ¾ Ü ¾ be the objective values of the final points found by the (+)- CMA-ES running on and ¾, respectively. Then we choose Þ Ü and Þ ¾ ¾ Ü ¾. c) Hybrid algorithms: If Tchebycheff scalarization is used, the algorithm reads: Algorithm : (+)-CMA-ES initialize Ü ¼µ Ü Ü µµ (+)-CMA-ES on Ü µ Ù Ü µ initialize Ü ¼µ Ü Ü µµ (+)-CMA-ES on ¾ Ü µ Ù ¾ ¾ Ü µ Ï ««µ«ð µ ¼ Ð foreach Û ¾ Ï do initialize Ü ¼µ Ü Ü µµ (+)-CMA-ES on ÛÙ µ Ü µ In lines 9 a utopian point is estimated based on an estimate of the optimum for each objective. È ¾ Then for each Û ¾ Ï, Û ¼ ¾ and Û, the (+)-CMA-ES is run on ÛÙ µ for a given number of generations (line ), producing a solution Ü and the corresponding objective vector Ü µ. If the weighted-sum approach is used, ÛÙ µ is simplyreplacedby Ûµ inthealgorithmabove.inthe weighted sum approach, an estimate of a utopian point is not required and lines 9 can be replaced by if we set Ï ««µ«ð µ ¼ Ð. For each single-objective optimization of a scalarized function, all parameters of the (+)-CMA-ES are reinitialized, in particular the global step-size. However, given a weighting-vector Û and the corresponding solution Ü Û, Rudolph et al. suggested in [] to use Ü Û as initial search point in solving the single-objective problem corresponding to Û ¼ Û Ð µ Ð µµ. Apart from the initial search point Ü ¼µ, all parameters of the (+)-CMA-ES are reinitialized. The modified algorithms are denoted by (+)-CMA-ES and (+)-CMA-ES, respectively. For example, the (+)-CMA-ES algorithm reads: Algorithm : (+)-CMA-ES, initialize Ü ¼µ Ï ««µ«ð µ ¼ Ð foreach Û ¾ Ï do Ü Ü µµ (+)-CMA-ES on Ûµ Ü µ Ü ¼µ Ü IV. EXPERIMENTS The experimental setup for the performance evaluation has been chosen similar to the one in []. The results of both variants of the (+)-CMA- ES and (+)-CMA-ES were compared to results of the steady-state MO-CMA-ES using the Ë-measure as second-level sorting criterion. Our evaluation is based on two performance indicators, the - and the hypervolume-indicator. We use the evaluation module of PISA presented in [] for parts of the performance assessment. We briefly outline the evaluation process below and refer to [7], [8] for a detailed description of the methods. Before performance indicators are computed, the data are normalized. We want to compare algorithms on aparticularoptimizationproblem after fitness evaluations and we assume that we have conducted Ø trials. We consider the non-dominated individuals of the union of all Ø populations after evaluations. These individuals make up the reference set Ê. Their objective vectors are normalized by an affine linear transformation such that for every objective the smallest and largest objective function values are mapped to and, respectively. The higher the value for the unary hypervolumeindicator the more volume is covered by the final set of solution and the better the performance. In contrast, a lower value of the binary -indicator means better performance. The -indicator is computed with respect to the reference set Ê, and the smaller the value the closer is the result to Ê. A. Benchmark Functions Several benchmark functions have been chosen from the literature. The constrained functions ZDT, ZDT, ZDT, and ZDT [9] and their rotated variants termed IHR, IHR and IHR [] were selected. Additionally, PISA stands for Platform and Programming Language Independent Interface for Search Algorithms: Note that in [], [], [] the hypervolume is compared to a reference set and therefore a lower value indicates better performance.

6 the unconstrained and rotated functions ELLI, ELLI ¾, CIGTAB, and CIGTAB ¾ proposed in [] were used. The benchmark function WFG introduced in [0] has been chosen to illustrate the ability of a scalarization method to find solutions in concave regions of the Pareto-optimal front. In Table I the definitions of the benchmark functions are summarized. B. Parameter Setup For each of the algorithms, 0 independent trials with 00,000 fitness function evaluations were carried out. For the hybrid algorithms a number of weight steps Ð ¼¼ with the single-objective CMA-ES running for 000 generations was chosen. For the MO-CMA-ES, a population size of ¼¼ and an offspring population size of was chosen. Accordingly, the number of generations was set to 00,000. All other parameters were set to standard values. C. Results Table II shows the results of the performance evaluation after 0 trials with 00,000 fitness evaluations. After 0,000 fitness evaluations the results are similar and therefore not shown here. Some differences are so small that they are not apparent in the result table, although they are statistically significant. For the function ELLI ¾, where the results are very close, we additionally provide boxplots of the indicator values in Fig.. D. Discussion The MO-CMA-ES performs significantly better than the (+)-CMA-ES and the (+)-CMA-ES on all benchmark functions except for ELLI and ELLI ¾. On nine test problems, (+)-CMA-ES performs significantly better than (+)-CMA-ES, while (+)- CMA-ES is only significantly better than (+)- CMA-ES onihr.thus,thewarmstartsinthe(+)- CMA-ES improve the algorithms The function WFG has a non-convex Pareto front, and thus it does not come as a surprise that the weighted-sum approach performs badly on WFG. The good results of the (+)-CMA-ES algorithms on the functions CIGTAB, CIGTAB ¾, ELLI, and ELLI ¾ are striking. But these are ideal test functions for this method. First, as all objective functions are convex, the weighted-sum approach is in principle able to find all Pareto-optimal points. Second, for fixed weighting coefficients, these functions reduce to quadratic optimization problems (i.e., for these two functions Û Üµ Û µ ¾ ܵ are elliptic paraboloids), and exactly these problems are ideal for the CMA-ES. That is, on CIGTAB, CIGTAB ¾, ELLI, and ELLI ¾, the algorithms (+)-CMA-ES and (+)-CMA-ES decompose the multi-objective optimization problem into subproblems perfectly suited for the CMA-ES, and this results in superior performance on these benchmark functions. V. CONCLUSIONS Scalarization approaches to multi-objective optimization (MOO) as considered in this paper suffer from the curse of dimensionality and are not suited for many objectives. However, for bicriteria optimization single-objective evolutionary algorithms applied to a set of scalarized fitness functions may be competitive to population-based approaches, in which the population ofasingletrialisconsideredasthesolutiontothemoo problem. Here we compared the multi-objective CMA evolution strategy (MO-CMA-ES), which is based on the elitist CMA-ES, with application of the elitist CMA- ES to the optimization of scalarized fitness functions. To this end, we introduced new hybrid algorithms combining established scalarization methods with the CMA-ES. These algorithms have similar invariance properties as the MO-CMA-ES. On most standard benchmark functions, the MO- CMA-ES outperformed the elitist CMA-ES combined with scalarization. But when the scalarization produced perfect quadratic fitness functions, the powerful strategy adaptation of the CMA-ES could exploit this ideal structure and the CMA-ES combined with weightedsum scalarization performed on par with the truly population-based MO-CMA-ES. Because of the known problems of the weighted-sum approach with nonconvexity, the hybrid algorithm using the Tchebycheff metric performed better compared to weighted aggregation on non-convex MOO problems. Future work will include the evaluation of the proposed algorithms on more than two objectives with the hypothesis that the MO-CMA-ES will outperform the hybrid methods even more clearly. Further, we will compare the performance in scenarios where the number of fitness evaluations is strictly limited. REFERENCES [] K. Miettinen, Nonlinear Multiobjective Optimization, ser. Kluwer s International Series in Operations Research & Management Science. Kluwer Academic Publishers, 999, vol.. [] N. Hansen and A. Ostermeier, Completely derandomized selfadaptation in evolution strategies, Evolutionary Computation, vol. 9, no., pp. 9 9, 00. [] N. Hansen, The CMA evolution strategy: A comparing review, in Towards a new evolutionary computation. Advances on estimation of distribution algorithms., I. I. J.A. Lozano, P. Larraãga and E. Bengoetxea, Eds. Springer-Verlag, 00. [] A. Auger and N. Hansen, A restart CMA evolution strategy with increasing population size, in Proceedings of the IEEE Congress on Evolutionary Computation (CEC 00). IEEE Press, 00, pp [] C. Igel, N. Hansen, and S. Roth, Covariance matrix adaptation for multi-objective optimization, Evolutionary Computation, vol., no., 00. [] C. Igel, T. Suttorp, and N. Hansen, Steady-state selection and efficient covariance matrix update in the multi-objective cma-es, in Fourth International Conference on Evolutionary Multi-Criterion Optimization (EMO 007), ser. LNCS, vol. 0. Springer-Verlag, 007.

7 TABLE I BENCHMARK PROBLEMS TO BE MINIMIZED, Ý ÇÜ, WHERE Ç ¾ Ê IS AN ORTHOGONAL MATRIX, AND Ý MAX ÑÜ Ó µ. FOR THE DEFINITION OF,, AND WE REFER TO [9]. FOR THE ROTATED BENCHMARK FUNCTIONS, WITH ¼¼¼, ¼¼, Ý ÇÜ, AND Þ Ç¾Ü, WHERE Ç AND Ǿ ARE ORTHOGONAL MATRICES.FOR THE BENCHMARK FUNCTIONWFG, THE INPUT VECTOR Ü IS TRANSFORMED SEVERAL TIMES RESULTING IN THE VECTOR Ø WITH ¾. WE REFER TO [0] FOR THE DEFINITION OF THE TRANSFORMATIONS. Problem Variable Objective Optimal bounds function solution IHR 0 ܵ Ý Ý ¾ ¼ Ý Ô max ¾ ܵ ݵ ݵ ݵ Ý ¼ ݵ ¾ Ý µ µ ¾ IHR 0 ܵ Ý Ý ¾ ¼ Ý Ô max ݵ ¾ ܵ ݵ È Ýµ ݵ ݵ ¼Ý µ Ý ¼ ݵ ¾ Ý µ µ ¾ IHR 0 ܵ ÜÔ Ýµµ ݵ Ý ¾ Ý max Ýmax ¾ ܵ ݵ ܵ ݵµ ¾ È Ý ¼ ݵ ¾ ¼¾ Ý µ µ ¾ WFG 0 ¼ ܵ Ó ¼Ø µ Ü ¾ ¼ ¾ ܵ Ø Ó ¾ Ø ¾µµ ¾ µ ELLI 0 ¼ ¼ ܵ ¾ ¾ ܵ Unconstrained Benchmark Functions ¾ Ý ¾ ¾ ELLI¾ 0 ¼ ¼ ܵ ¾ ¾ ܵ ¾ CIGTAB 0 ¼ ¼ ܵ ¾ ¾ ܵ ¾ CIGTAB¾ 0 ¼ ¼ ܵ ¾ ¾ ܵ ¾ Ý Ý ¾ Ý ¾µ ¾ Ý ¾ ¼ ¾ ¾ Ý ¾ ¾ Þ ¾µ ¾ Ý ¾ È ¾ ݾ ¾ Ý ¾ Ý Ý Ý ¾µ ¾ È ¾ Ý ¾µ ¾ ¾ Ý ¾µ ¾ Ý ¾ ¼ ¾ Ý ¾ È ¾ ݾ ¾ Ý ¾ Þ ¾µ ¾ È ¾ Þ ¾µ ¾ ¾ Þ ¾µ ¾ ¼ (+)-CMA-ES (+)-CMA-ES (+)-CMA-ES (+)-CMA-ES ¼ hypervolume-indicator ¼ -indicator ¼ ¼ MO-CMA-ES (+)-CMA-ES (+)-CMA-ES MO-CMA-ES (+)-CMA-ES (+)-CMA-ES Fig.. Boxplots of theresultsof theperformance assessmentfor thefitnessfunctionelli¾.theleftplot showsthevaluesof thehypervolumeindicator (minus ¾), the right plot shows the -indicator values on logarithmic scale. The boxes refer to the th and 7th percentile. In each plot, from left to right, results of the MO-CMA-ES, (+)-CMA-ES, (+)-CMA-ES, (+)-CMA-ES, (+)-CMA-ES are shown.

8 TABLE II RESULTS ON BENCHMARK PROBLEMS.THE UPPER AND LOWER PART OF EACH TABLE SHOWS THE MEDIAN OF 0 TRIALS AFTER 00,000 FITNESS EVALUATIONS OF THE HYPERVOLUME-INDICATOR(THE HIGHER THE BETTER) AND THE -INDICATOR(THE LOWER THE BETTER), RESPECTIVELY.THE BEST VALUE IN EACH ROW IS UNDERLINED, THE WORST IS PRINTED IN BOLD. THE SUPERSCRIPTSI, II, III, IV AND V INDICATE WHETHER AN ALGORITHM IS STATISTICALLY SIGNIFICANTLY BETTER THAN THE MO-CMA-ES,(+)-CMA-ES, (+)-CMA-ES, (+)-CMA-ES AND (+)-CMA-ES, RESPECTIVELY(TWO-SIDEDWILCOXON RANK SUM TEST, Ô ¼¼¼, SLANTED SUPERSCRIPTS REFER TO A SIGNIFICANCE LEVEL OF Ô ¼¼). MO-CMA-ES (+)-CMA-ES (+)-CMA-ES (+)-CMA-ES (+)-CMA-ES Hypervolume-Indicator CIGTAB.8 II,III,IV,V II.9 II,III.8 II,III,IV CIGTAB¾.000 II,III,IV,V.000 IV,V.000 II,IV,V IV ELLI.08.0 I.08 I.0 I,II,III.089 I,II,III,IV ELLI¾ I,II,IV,V.000 I,II.000 I,II,IV IHR II,IV,V II,IV IHR.7 II,IV I,II,IV.98 II.9809 I,II,III,IV IHR.80 II,III,IV,V III II,III 0.00 II,III,IV WFG ¼¾¼ II,III,IV,V ¼¾ IV,V ¼ IV,V ¼ ¼¼¼ V ¼¼ ZDT.089 II,III,IV,V..07 II,IV,V. II.8 II,IV ZDT.7 II,III,IV,V II,IV,V.0798 II.0878 II,IV ZDT.7 II,III,IV,V II,IV,V.08 II.08 II,IV ZDT.07 II,III,IV,V.99 IV,V.99 II,IV,V Indicator CIGTAB ¾¼ ¼ II,III,IV,V ¼¾¾¼ ¼ ¼ ¼ II ¾ ¼ II,III ¼ II,III,IV CIGTAB¾ ¼ ¼ II,III,IV ¼ ¼ ¼ ¼ II,IV ¾ ¼ ¼ II ¼¾ ¼ II,III,IV ELLI ¾¼ ¼ ¼ I ¼ I ¼¾¼ ¼ I,II,III ¾ ¼ I,II,III,IV ELLI¾ ¼ ¾¾ ¼ I ¾ ¼ I,II ¼ ¼ I,II,III ¼¼ ¼ I,II,III,IV IHR ¼ ¼ ¾ ¼ ¼ ¾IV ¾¼¼ ¼ ¾II,IV,V ¾¼ ¼ ¾ ¼ ¼ ¾II,IV IHR ¼¼ ¼ ¾II,IV ¼¼ ¼ ¾ ¾¼ ¼ ¾I,II,IV ¾ ¼ ¾II ¾ ¼ ¾I,II,III,IV IHR ¼ ¼ II,III,IV,V ¾¼¼ ¼ III ¼¼ ¼ ¼ ¼ II,III ¼ ¼ II,III,IV WFG ¼ ¼ ¾II,III,IV,V ¼ ¼ ¾III,IV,V ¼ ¼ ¾IV,V ¾ ¼ ¼ ¾ ¼ ¼ ¾IV ZDT ¼ ¼ ¾II,III,IV,V ¼ ¼ ¾ ¼¾ ¼ ¾II,IV,V ¼ ¼ ¾II ¼¼¼ ¼ ¾II ZDT ¼¼ ¼ II,III,IV,V ¼¼ ¼ ¼¼ ¼ II,IV,V ¼¼ ¼ II ¼ ¼ ¼ II,IV ZDT ¼ ¼ ¾II,III,IV,V ¾¼ ¼ ¾ ¾ ¼ ¾II,IV,V ¼¾ ¼ ¼ ¾II ¾¼ ¼ ¾II,IV ZDT ¾ ¼ ¾II,III,IV,V ¼ ¼ ¾IV,V ¼ ¼ ¾II,IV,V ¼ ¾V ¼ ¾ [7] S. Gass and T. Saaty, The Computational Algorithm for the Parametric Objective Function, Naval Research Logistics Quarterly, vol., no., pp. 9, 9. [8] L. Zadeh, Optimality and Non-Scalar-Valued Performance Criteria, IEEE Transactions on Automatic Control, vol. 8, no. 8, pp. 9 0, 9. [9] I. Rechenberg, Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Frommann- Holzboog, 97. [0] H.-P. Schwefel, Evolution and Optimum Seeking, ser. Sixth- Generation Computer Technology Series. John Wiley & Sons, 99. [] H.-G. Beyer and H.-P. Schwefel, Evolution strategies: A comprehensive introduction, Natural Computing, vol., no., pp., 00. [] S. Kern, S. D. Müller, N. Hansen, D. Büche, J. Ocenasek, and P. Koumoutsakos, Learning probability distributions in continuous evolutionary algorithms a comparative review, Natural Computing, vol., pp. 77, 00. [] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Transactions on Evolutionary Computation, vol., pp. 8 97, 00. [] N. Beume, B. Naujoks, and M. Emmerich, SMS-EMOA: Multiobjective selection based on dominated hypervolume, European Journal of Operational Research, vol. 8, no., pp. 9, 007. [] G. Rudolph, B. Naujoks, and M. Preuss, Capabilities of MOEA to Detect and Preserve Equivalent Pareto Subsets, in Proceedings of the th International Conference on Evolutionary Multi-Criterion Optimization (EMO 007), S. Obayashi, Ed. Springer-Verlag, 007, pp. 0. [] J. Knowles, L. Thiele, and E. Zitzler, A tutorial on the performance assessment of stochastic multiobjective optimizers, Computer Engineering and Networks Laboratory (TIK), ETH Zurich, Switzerland,, 00, revised version. [7] E. Zitzler and L. Thiele, Multiobjective optimization using evolutionary algorithms a comparative case study, in Fifth International Conference on Parallel Problem Solving from Nature (PPSN-V), A. E. Eiben, T. Bäck, M. Schoenauer, and H.-P. Schwefel, Eds. Berlin, Germany: Springer-Verlag, 998, pp [8] E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and V. Grunert da Fonseca, Performance assesment of multiobjective optimizers: An analysis and review, IEEE Transactions on Evolutionary Computation, vol. 7, no., pp. 7, 00. [9] E. Zitzler, K. Deb, and L. Thiele, Comparison of multiobjective evolutionary algorithms: Empirical results, Evolutionary Computation, vol. 8, no., pp. 7 9, 000. [0] S. Huband, P. Hingston, L. Barone, and L. While, A review of multi-objective test problems and a scalable test problem toolkit, in IEEE Transactions on Evolutionary Computation, vol. 0, 007, pp