Himanshu Jain and Kalyanmoy Deb, Fellow, IEEE

Transcription

1 An Evolutionary Many-Objective Optimization Algorithm Using Reerence-point Based Non-dominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach Himanshu Jain and Kalyanmoy Deb, Fellow, IEEE Abstract In the precursor paper [], a many-objective optimization method (NSGA-III), based on the NSGA-II ramework, was suggested and applied to a number o unconstrained (with box constraints alone) test and practical problems. In this paper, we extend NSGA-III to solve generic constrained many-objective optimization problems. In the process, we also suggest three types o constrained test problems that are scalable to any number o objectives and may provide dierent types o challenges to a many-objective optimizer. A previously suggested MOEA/D algorithm is also extended to solve constrained problems. Results using constrained NSGA-III and constrained MOEA/D show an edge o the ormer particularly in solving problems having a large number o objectives. Further, the NSGA-III algorithm is made adaptive in deleting and including new reerence points on the ly. The resulting adaptive NSGA-III is shown to provide a denser representation o the Pareto-optimal ront compared to the original NSGA-III or an identical computational eort. This and the original NSGA-III paper together suggest and amply test aviableevolutionarymany-objectiveoptimizationalgorithm or handling constrained and unconstrained problems. These studies should encourage researchers to use and pay urther attention to the growing interest in evolutionary many-objective optimization. Index Terms Many-objective optimization, evolutionary computation, large dimension, NSGA-III, non-dominated sorting, multi-criterion optimization. I. INTRODUCTION Evolutionary multi-objective optimization (EMO) methodologies suggested since early nineties have amply demonstrated the use o evolutionary algorithms in solving optimization problems having mostly two and three objectives [], [3], [4]. The main reason or their popularity was their ability to ind multiple trade-o solutions in a single simulation run and their ease and lexibility to ocus on any part o the Paretooptimal rontier. Besides their academic use, EMO has also been practiced in industries, mainly due to the availability o a number o commercial EMO sotwares. EMO has also been diversiied to hybridize with its contemporary ields, H. Jain is with Eaton Corporation, Pune 44, India, himanshu.j689@gmail.com. K. Deb is with Department o Electrical and Computer Engineering. Michigan State University, 48 S. Shaw Lane, EB, East Lansing, MI 4884, USA, kdeb@egr.msu.edu (see kdeb). Pro. Deb is also a visiting proessor at Aalto University School o Business, Finland and University o Skövde, Sweden. Copyright (c) IEEE. Personal use o this material is permitted. However, permission to use this material or any other purposes must be obtained rom the IEEE by sending a request to pubs-permissions@ieee.org. such as in multiple criterion decision making (MCDM) and in mathematical multi-objective optimization studies. With all these all-round developments, despite a ew studies, one aspect mainly remained unexplored: the issue o handling a large number o objectives. It has been clearly shown in the EMO literature that an EMO methodology that works so well or two or three objectives has the curse o dimensionality in solving more than three-objective problems [5], [6]. The main reasons or their poor working domination principle being too weak to provide an adequate selection pressure, a large population size requirement, etc. were not unknown to the EMO researchers, but were diicult to alleviate in an adequate manner. Although a ew many-objective evolutionary algorithms have been suggested in the past [6], [7], [8], [9], [], [], [], there is still a need or more eicient algorithms or many-objective optimization, similar to popular two or three-objective EMO methods, such as NSGA-II [3], SPEA [4] and others. In the precursor study [], we suggested an evolutionary many-objective optimization algorithm by extending the NSGA-II ramework. Realizing the computational challenge associated with a population-based optimization algorithm in converging to Pareto-optimal ront and simultaneously spread its population along the entire ront, in NSGA-III, the latter task is aided by supplying a set o pre-deined reerence points. The algorithm was then expected to ocus its search to ind an associated Pareto-optimal solution or each reerence point. Keeping NSGA-II s emphasis on nondominated solutions intact, its elitist selection mechanism was modiied to incorporate three new operations normalization o objective vectors and the supplied reerence points so as to have both sets within a single range, association o every population member with a particular reerence point based on a proximity measure, and niching o accepted population members in order to ensure a diverse set o solutions. The results on several test problems and practical problems have amply demonstrated NSGA-III s useulness in solving two to 5-objective unconstrained problems with speciied variable bounds. Since the supplied reerence points were chosen as a diverse set, the obtained trade-o solutions were also likely to be diverse. Since multiple Pareto-optimal points were targeted to be ound simultaneously in a single simulation run, NSGA- III provided an eicient parallel search. In the earlier paper, NSGA-III was restricted to solve prob- Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

2 lems having box constraints alone. In this paper, we extend NSGA-III to solve constrained many-objective optimization problems o the ollowing type: Minimize ( (x), (x),..., M (x)), subject to g j (x), j =,,...,J, h k (x) =, k =,,...,K, x i x (U) i, i =,,...,n. x (L) i One advantage o using an evolutionary algorithm or solving the above problem is that the box constraints (the last set o constraints on variables alone) can be handled automatically by initializing all population members satisying the bounds and by ensuring that the creation o ospring solutions are always within the speciied lower and upper bounds. Thus, a procedure to handle inequality and equality constraints remains to be incorporated with the NSGA-III algorithm. A linear equality constraint, i present in a problem, can be used to eliminate one variable using the constraint. Thus, linear constraints, in general, help reduce the dimensionality o the search space. In this paper, we modiy certain operators o NSGA-III to emphasize easible solutions more than the ineasible solutions in a population. Two main changes are suggested in the original algorithm or this purpose. The modiications suggested still keep the overall algorithm parameter-less (besides the need o usual genetic parameters). Another aspect o the extension is that i all population members are easible or an unconstrained problem is supplied, the constrained NSGA- III reduces to the original unconstrained NSGA-III algorithm. To evaluate its perormance, the proposed constrained NSGA- III procedure is applied to a number o many-objective test problems, suggested here or the irst time, and two practical many-objective problems. The constrained NSGA-III approach is also applied with a ew preerred reerence points to ind a handul o solutions on apreerredregiononthepareto-optimalset.onatestproblem and on a practical problem, the approach is able to ind ive or trade-o solutions corresponding ive or supplied preerred reerence points. During the course o the earlier study [] and this study on constraint-handling using NSGA-III, we have realized that in certain problems, not all speciied reerence points will correspond to a Pareto-optimal solution. In such a scenario, the processing o these non-useul reerence points causes a waste o computations. In this paper, we rectiy this diiculty by suggesting an adaptive NSGA-III that identiies non-useul reerence points and adaptively deletes and includes new reerence points in addition to the supplied reerence points. Simulation results on a number o many-objective test problems and practical problems support the modiications made and demonstrate the useulness o the proposed procedure. In the remainder o this paper, we provide a brie overview o existing many-objective constraint handling procedures in Section II. Thereater, the constrained NSGA-III is described in detail by irst providing a brie description o the original NSGA-III in Section III. As an alternative algorithm, we extend the MOEA/D-DE algorithm proposed in [5] to solve () constrained problems in the next section. The resulting C- MOEA/D and proposed constrained NSGA-III algorithms are then applied to three types o scalable constrained test problems in Section V. This section also applies the constrained NSGA-III algorithm to two engineering design problems. Next, to show NSGA-III s ability to be hybridized with a decision-making technique, NSGA-III is applied with a ew preerred reerence points. Results on two problems are shown in Section VI. Thereater, in Section VII, we have proposed an adaptive NSGA-III algorithm in detail and applied it to solve many-objective test problems and practical problems. Conclusions o the extensive study are then drawn in Section IX. The appendix contains the optimization problem ormulations o two engineering design problems considered in this study. II. EXISTING MANY-OBJECTIVE CONSTRAINT-HANDLING PROCEDURES There is not enough literature on handling constraints in a many-objective optimization algorithm, as most existing many-objective EA studies handled unconstrained problems only. MOEA/D, ater its suggestion [7] in 8, was extended to include the dierential evolution (DE) operator [6], and later suggested to address constraints using the MOEA/D-DE approach [5]. We briely describe the procedure below. The constrained MOEA/D-DE algorithm [5] is dierent rom its unconstrained version in the ollowing ways: (i) it uses apenaltyunctiontohandleconstraints,butintroducestwo penalty parameters s and s or the purpose, (ii) it restricts the number o reerence directions a new child solution can be associated with by introducing a limiting parameter n r, (iii) it chooses a mating partner o a solution based on a probability distribution involving a parameter δ,and(iv)ituses the dierential evolution [7] to create new solutions which involves two parameters CR and p m.theresultswerereported to depend on the choice o the penalty parameters. Moreover, ixing six parameters adequately or a problem is the main drawback o the above constrained MOEA/D-DE approach. In Section IV, we suggest a dierent constrained version o MOEA/D based on the principles o our proposed approach that may remain as a viable pragmatic extension o MOEA/D or handling constraints. The constrained handling approaches proposed by Fonseca and Fleming [8] and by Deb et al. [9], [3] do not require any additional parameters. By making pairwise comparisons between population members, easible and less constraintviolated solutions were emphasized. Although these methods were suggested or multi-objective optimization problems, they can very well be tried or solving many-objective optimization problems. Our proposed NSGA-III approach, described next, uses these ideas or handling constraints. III. PROPOSED NSGA-III WITH A CONSTRAINT HANDLING APPROACH Beore we describe the constraint handling procedure, we present a brie outline o recently proposed many-objective NSGA-II procedure described in the original paper []. Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

3 3 NSGA-III starts with description o a set o reerence points Z.ThecurrentparentpopulationP t (at generation t) isusedto create an ospring population Q t by using genetic operations. The combined population R t = P t Q t is sorted into dierent levels o non-domination. All population members up to the last ront (F l ) that could not be ully accommodated are saved in a set S t and remaining members o R t are rejected. Members in S t \F l are already selected or the next generation and the remaining population slots are selected rom F l.inthe original NSGA-II, the last ront members having the largest crowding distance values (providing widest diversity) were chosen. The crowding distance operation does not work well or many-objective problems [] and here we modiy the selection mechanism by perorming a more systematic analysis o members o S t with respect to the supplied reerence points. Objective values and supplied reerence points are irst normalized so that they have an identical range. This way, the ideal point o the set is the zero vector. Each member o S t is then associated with a reerence point depending on the proximity o the member with a reerence line obtained by joining the ideal point with the reerence point. This procedure helps determine the number and indices o population members associated with each supplied reerence point in S\F l.thereater,aniching procedure is used to select population members rom F l that are not well represented in S t \F l using the outcome o the above association procedure. The reerence points that have the least number o association in S\F l population are looked or an associated point in F l set. Such F l members are then added one at a time to ill the population. Such a careul selection strategy is ound to have aslightlylargercomputationalcomplexityoo(n log N) compared to O(N(log N) M ) complexity o NSGA-II, but NSGA-III helped solve problems having a large number o objectives. We now propose an extension o the above NSGA-III procedure to handle generic equality and/or inequality constraints. We discuss the modiications one by one. A. Modiications in the Elitist Selection Operator Recall that the combined population R t needs to be sorted according to dierent non-domination levels. For unconstrained problems, the objective unction values alone are considered or the domination check between two solutions. But in the presence o constraints, we ollow the constraintdomination principle adopted in NSGA-II [3] using the ideas rom [8], [9]: Deinition A solution x () is said to constraint-dominate another solution x (),ianyoneotheollowingconditions is true: ) i x () is easible and x () is ineasible, ) i x () and x () are ineasible and x () has a smaller constraint violation value, or 3) i x () and x () are easible and x () dominates x () with the usual domination principle [], []. For calculating the constraint violation value (CV (x)) o a solution x, we suggest normalizing all constraints by dividing the constraint unctions by the constant present (that is, or g j (x) b j, the normalized constraint unction becomes ḡ j (x) =g j (x)/b j and similarly h k (x) can also be normalized) and then using the ollowing measure: CV (x) = J K ḡ j (x) + h k (x), () j= k= where the bracket operator α returns the negative o α, i α< and returns zero, otherwise. The population R t o size N can be sorted into dierent non-domination levels according to the above constraintdomination principle. I every population member is ineasible, the non-domination sorting procedure will assign the solution having the smallest CV in the irst ront, the solution with the next smallest CV in the second ront and so on. Thus, there will be a total on ronts, unless there exists two solutions having an identical CV value. On the other hand, i all population members are easible, the non-domination sorting will be identical to that obtained by the usual domination principle. In most cases, the population may have some easible solutions (set F)andsomeineasiblesolutions(setI). In this case, the above sorting procedure will arrange easible solutions according to their non-domination levels in the top o the sorted levels and the ineasible solutions will occupy the next levels one (in most cases) in each ront starting with the least constraint-violated solution. Once the combined population R t is sorted according to constraint-domination, the number o easible solutions N in R t is counted. I N N, meaningthatthereareatmostn easible points in R t,wedeinitelyselectalleasiblesolutions or P t+ and the remaining (N P t+ ) population slots are illed with top levels o the ineasible solutions (having smaller CV values). However, i N >N,meaningthatthereare more easible solutions in R t than required, we do not consider the ineasible solutions at all and ollow the unconstrained NSGA-III selection procedure with easible solution set R t \I. In either case, we then update the population ideal (z min ) and nadir points (z max )usingtheobjectivevaluesoeasible solutions or the normalization procedure, discussed in the original NSGA-III paper. B. Modiication in Creation o Ospring Population The original NSGA-III algorithm suggested to use a population size (N) almostequaltothenumberoreerencepoints (H). ( The parameter H is derived rom a combinatorial value (M+p ) ) p or a given p. Thepopulationsizeisrecommended to be the smallest multiple o our, greater than H. Thus, every population member is likely to be associated with a dierent reerence point and at the end it is desired that there will be at least one Pareto-optimal solution associated with every reerence point. Due to this one-member-to-onereerence-point expectation, no additional tournament selection was applied to the parent population P t to create the ospring population Q t.however,inthepresenceoineasible solutions in the population, there is a need or bringing back the tournament selection operator particularly or emphasizing Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

4 4 aeasiblesolutionoveranineasiblesolutionandasmallcv solution over a large CV solution. For this purpose, we select two members rom P t at random, and a binary tournament selection is applied, as ollows, to select a better solution: Deinition The modiied tournament selection operation between solutions p and p is deined as ollows: ) i p is easible and p is ineasible, select p else i p is easible and p is ineasible, select p, ) i p and p are ineasible then i p has a smaller constraint violation CV, select p, else i p has a smaller constraint violation CV,selectp,and 3) i both p and p are easible then p or p is chosen at random. The above conditions or choosing a solution over another is similar to that used in deining constraint-domination, except that when both solutions are easible, a random solution is now chosen, thereby implying that there is no tournament selection perormed in this case. Similarly, another pair o members are randomly selected rom population P t and the above modiied tournament selection is applied to select the second parent. Thereater, crossover and mutation operators are applied on both parents to produce two ospring solutions as usual. This process is continued till N ospring are created to orm the population Q t. The overall procedure is presented in a pseudo-code in Algorithm. Notice how the procedure becomes similar to the unconstrained NSGA-III selection operator (described in the original study []) when there is no ineasible population member or when there is equality or inequality constraints speciied in the optimization problem ormulation. Algorithm Tournament Selection(p,p ) procedure Require: p,p Ensure: p : i easible(p )=TRUEandeasible(p )=FALSEthen : p = p 3: else i easible(p )=FALSEandeasible(p )=TRUE then 4: p = p 5: else i easible(p )=FALSEandeasible(p )=FALSE then 6: i CV (p ) >CV(p ) then 7: p = p 8: else i CV (p ) <CV(p ) then 9: p = p : else : p = random(p,p ) : end i 3: else 4: p = random(p,p ) 5: end i The rest o the NSGA-III procedure described in the original paper [] remains the same. A careul analysis will reveal that the above constrained NSGA-III approach does not introduce any new parameter or handling constraints. This remains as a hallmark eature o our proposed constraint handling approach. IV. PROPOSED CONSTRAINT-MOEA/D METHOD (C-MOEA/D) The original MOEA/D approach [7] was extended to include the DE operator to develop MOEA/D-DE approach [6], and subsequently suggested a constrained MOEA/D-DE approach to handle constraints [5], but as discussed in Section II, the constrained MOEA/D-DE approach is based on a penalty unction concept that requires two penalty parameters. In addition, the approach also requires our other parameters that are needed to be set right in solving an arbitrary problem. In the original NSGA-III study [], we have reported that MOEA/D approach, in principle, can perorm well in solving many-objective optimization problems, which the developers o MOEA/D did not demonstrate. Here, we modiy the MOEA/D-DE approach with a similar constrained handling approach as described above, hoping that the proposed C-MOEA/D can also become a competing and alternate algorithm or constrained many-objective problem solving. We make the ollowing modiications to the original MOEA/D-DE approach [5], [6]. When a child solution y is compared with a randomly picked member x rom its neighborhood, instead o replacing the member just based on perormance metric (PBI or Tchebyche), the constraint violation, i any, o both solutions are checked. Following our scenarios can occur: ) Solution x is easible while solution y is ineasible. Then, x is not replaced by y. NocomputationoPBI or Tchebyche metric is needed or any o these two solutions. ) Solution x is ineasible while solution y is easible. Then, x is replaced by y. 3) Both solutions x and y are ineasible. I x has a larger constraint violation than y (that is, CV (x) >CV(y), then x is replaced by y. 4) Both solutions x and y are easible. Here, we propose the use o a perormance measure. I PBI (or Tchebyche) metric value o x is worse than that o y, thenx is replaced by y. The above modiications are in tune with that adopted in constrained NSGA-III algorithm and should provide an adequate emphasis or easible and small-cv solutions in the population. Importantly, no new parameter is introduced in the algorithm. Moreover, in the original study [], the DE operator or creating ospring solutions did not perorm well with the rest o the MOEA/D algorithm. Based on that study, here, we do not use the DE operator, instead a real-coded GA with SBX and polynomial mutation operators are used or creating the ospring population. We also propose the use o PBI metric (instead o the Tchebyche metric), as PBI metric was ound to work better in the original study []. We name this version o MOEA/D as constrained MOEA/D or simply C-MOEA/D. Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

5 5 V. RESULTS In this section, we present simulation results o the proposed constrained NSGA-III and C-MOEA/D approaches. For this purpose, we use a number o constrained test problems having three to 5 objectives, designed to introduce dierent types o diiculties to an algorithm. The problems are scalable both in the number o objectives and in the number o variables. For each problem, dierent runs with dierent initial populations are carried out and the best, median and the worst IGD perormance values (which can only be computed or a test problem with a known Pareto-optimal ront) are reported. To compute IGD value, irst, we compute the targeted points (Z) ontheknownpareto-optimalrontromthesuppliedreerence points or directions in the normalized objective space. Then, or an algorithm, we obtain the inal non-dominated points (set A) intheobjectivespace.now,wecomputethe IGD metric value as the average Euclidean distance o points in set Z with their nearest members o all points in set A: IGD(A, Z) = Z Z i= A min d(z i, a j ), (3) j= where d(z i, a j ) = z i a j. For both algorithms, the population members rom the inal generation are presented and used or computing the above IGD metric. The number o reerence points, population size and other parameters are kept in agreement with the original study [] and are tabulated in Tables I and II. In case o C-MOEA/D, two parameters δ TABLE I NUMBER OF REFERENCE POINTS/DIRECTIONS AND CORRESPONDING POPULATION SIZES USED IN CONSTRAINED NSGA-III AND C-MOEA/D ALGORITHMS. No. o Re. pts./ NSGA-III MOEA/D objectives Re. dirn. popsize popsize (M) (H) (N) (N ) TABLE II PARAMETER VALUES USED IN CONSTRAINED NSGA-III AND C- MOEA/D. n IS THE NUMBER OF VARIABLES. Parameters NSGA-III MOEA/D SBX probability [3], p c Polynomial mutation prob. [], p m /n /n η c [3] 3 η m[3] (probability with which the parent solutions are selected rom neighborhood) and n r (maximal number o solutions replaced by an ospring solution) are set as.9 and, respectively, as suggested by the developers [5]. In contrast, the proposed constrained-handling NSGA-III does not require setting o any new parameter. A. Constrained Problems o Type- In Type- constrained problems, the original Pareto-optimal ront is still optimal, but there is an ineasible barrier in approaching the Pareto-optimal ront. This is achieved by adding a constraint to the original problem. The barrier provides ineasible regions in the objective space that an algorithm must learn to overcome, thereby providing a diiculty in converging to the true Pareto-optimal ront. DTLZ and DTLZ3 problems [4] are modiied according to this principle in this study. For the type constrained DTLZ (or C-DTLZ), only a part o objective space that is close to Pareto-ront is made easible, as shown in Figure. The objective unctions are kept the same as they were in the original DTLZ problem, while the ollowing constraint is now added: c(x) = M (x).6 M i= i (x).5. (4) The easible region and the Pareto-optimal ront are shown or a two-objective C-DTLZ problem in Figure. In all simulations, we use k =5variables or the original g-unction [4], thereby making a total o (M +4)variables to the M- objective C-DTLZ problem. In the case o C-DTLZ3 problem, a band o ineasible space is introduced adjacent to the Pareto-optimal ront, as shown in Figure. Again, the objective unctions are kept the same as in original DTLZ3 problem [4], while the ollowing constraint is added: ( M )( M ) c(x) = i (x) 6 i (x) r. (5) i= i= where, r = {9,.5,.5, 5, 5} is the radius o the hypersphere or M = {3, 5, 8,, 5}. ForC-DTLZ3,weusek =, sothattotalnumberovariablesare(m +9) in a M- objective problem Paretoront Feasible Fig.. Two objective version o C-DTLZ problem Paretoront Ineasible Fig.. Two objective version o C-DTLZ3 problem. Both algorithms (NSGA-III and C-MOEA/D) are tested on three to 5 objective versions o above two problems. Figure 3 shows that in the case o three-objective C-DTLZ problem, NSGA-III is able to reach the easible region and ind awell-distributedsetopointsontheentirepareto-optimal ront. C-MOEA/D is also able to ind a nice distribution o points (Figure 4). However, as it is evident rom Table III, in most o the cases or the C-DTLZ problem, NSGA- III perorms better than C-MOEA/D in terms o the IGD metric. Interestingly, the best perormance o C-MOEA/D is in most cases better than that o NSGA-III. However, as the Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

6 6 TABLE III BEST, MEDIAN AND WORST IGD AND GD METRIC VALUES OBTAINED FOR NSGA-III AND C-MOEA/D ON M -OBJECTIVE C-DTLZ AND C-DTLZ3 PROBLEMS.BEST PERFORMANCE IS SHOWN IN BOLD. INCASESWHEREALGORITHMGOTSTUCKINLOCALPARETO-FRONT THE CORRESPONDING IGD VALUE IS NOT SHOWN INSTEAD THE NUMBER OF SUCCESSFUL RUNS OUT OF ARE SHOWN IN BRACKETS. Problem M MaxGen NSGA-III C-MOEA/D IGD GD IGD GD C-DTLZ C-DTLZ (8) (3) (8) (5) () (4) (6) (8) (9) number o objectives increase ( ad 5-objective problems), the perormance o NSGA-III is clearly better. Additionally, we compute the GD metric value or NSGA- III solutions and tabulate the best, median and worst values in Table III. Small GD values indicate that NSGA-III solutions are close to the true Pareto-optimal ronts in each case. Corresponding GD metric values o MOEA/D with PBI approach are also presented. GD metric values or both methods are similar, although interestingly in most cases whichever algorithm produced a better IGD value also made a smaller GD value. It is important to highlight here that GD metric indicates the convergence property o an algorithm, but cannot reveal the diversity in the solutions. On the other hand, the IGD metric indicates a combined measure o both diversity and convergence and is a more reliable metric or comparing multi-objective optimization algorithms. Figures 5 and 6 show Pareto-optimal ronts (corresponding to median IGD value) obtained by NSGA-III and C-MOEA/D, respectively, or the three-objective C-DTLZ3 problem. It is clear that while NSGA-III was able to reach the global Pareto-ront, C-MOEA/D (the median-perormed run) was not able to cross the ineasibility barrier and instead got stuck in the ineasible region. This problem is a diicult problem and Table III shows that in case o three, ive and objectives, although C-MOEA/D inds a better best IGD value but the success rate is less or C-MOEA/D as compared to NSGA- III. In all the cases or C-MOEA/D success rate is less than 5% while or NSGA-III it is more than 6% in all the cases except or 5 objectives where it is 45% and there C-MOEA/D was unable to reach the Pareto-ront in any o the runs. This problem was ound to be diicult or both algorithms, but the perormance o NSGA-III was ound to be somewhat better than C-MOEA/D. B. Eect o η c on C-MOEA/D In the above simulations with C-MOEA/D, η c =is used, simply because o the preerence o this value by the original developers o MOEA/D on unconstrained problems. Since, a somewhat higher value (η c =3)isusedwithNSGA-IIIto have a higher probability o creating ospring solutions close to parent solutions in a higher-dimensional space, we rerun C-MOEA/D algorithm with η c =3or C-DTLZ problem and tabulate results in Table IV. C-MOEA/D results are not signiicantly dierent rom previous C-MOEA/D results obtained with η c =. C. Constrained Problems o Type- While type constrained problems introduced diiculties in arriving at the entire Pareto-optimal ront, type- constrained problems are designed to introduce ineasibility to a part othe Pareto-optimal ront. Such problems will test an algorithm s ability to deal with disconnected Pareto-optimal ronts. To accomplish this, DTLZ [4] and the convex DTLZ problems [] are modiied. Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

7 Fig. 3. Obtained solutions using NSGA-III on three-objective C-DTLZ problem. Fig. 4. Obtained solutions using C-MOEA/D approach on three-objective C-DTLZ problem Pareto ront Ineasible Space Fig. 5. Obtained solutions using NSGA-III on three-objective constrained C-DTLZ3 problem. Fig. 6. Obtained solutions using C-MOEA/D on three-objective constrained C-DTLZ3 problem. In C-DTLZ problem, only the region o objective space that lies inside each o the M + hyper-spheres o radius r is made easible. O (M +) hyper-spheres, M are placed at the corners o unit hyper-plane and the (M +)-th is placed at the intersection o the equally-angled line with objective axes and the original Pareto-optimal ront. This way, the Paretooptimal ront is disconnected, as shown in Figure 7. Objective r Feasible Fig. 7. Two-objective version o C-DTLZ problem Ineasible Fig. 8. Two-objective version o convex C-DTLZ problem. unctions are calculated in the same way as in the original r DTLZ problem, except that a constraint is now introduced: M M c(x) = max max ( i(x) ) + j r, i= j=,j i [ M ( i(x) / ]} M) r, i= where r =.4, orm =3and.5, otherwise. For an M- objective C-DTLZ problem, k =is used, thereby having atotalo(m +9)variables. For the convex C-DTLZ described in [], we construct adierenteasibleregion.theregionintheobjectivespace lying inside a hyper-cylinder with (,,...,) T as the axis and radius r is kept ineasible, thereby creating an ineasible hole through the objective space. This also produces a hole on the Pareto-optimal ront, as demonstrated or a two-objective version o convex C-DTLZ problem in Figure 8. The objective unctions are kept the same as beore, while the ollowing constraint is added: c(x) = M ( i (x) λ) r, (6) i= Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

8 8 TABLE IV IGD AND GD VALUES WITH C-MOEA/D ALGORITHM WITH η c =3. Problem M MaxGen NSGA-III C-MOEA/D IGD GD IGD GD C-DTLZ C-DTLZ C3-DTLZ where λ = M M i= i(x) and the radius r = {.5,.5,.6,.6,.7} or M = {3, 5, 8,, 5}. Total number o variables or this problem are (M +9). We now present results o both algorithms on these two problems. Since in these problems only a part o Paretooptimal ront is easible, there may exist some reerence points/directions or which there is no corresponding easible point on Pareto-optimal ront. So while calculating the IGD metric, only the Pareto-optimal points (Z) correspondingto useul reerence points/directions are used. It could be that some reerence points have more than one Pareto-optimal points associated with it. For IGD metric value computation and or plotting the inal population, we have considered only one Pareto-optimal point that has the smallest perpendicular distance rom the extended reerence line corresponding to a reerence point/direction. Table V clearly shows that in the case o C-DTLZ problem, both NSGA-III and C-MOEA/D give similar perormance, however in all the cases (3-5 objectives) C-MOEA/D could not come close to the true Pareto-optimal ront in all runs (which is indicated by a large value o worst IGD values). However, despite the best perormance o NSGA-III is not better than C-MOEA/D, NSGA-III perorms well in all runs. The table also shows the number o useul reerence points out o total supplied reerence points or constrained NSGA-III procedure. The rest o the reerence points do not associate with any easible Pareto-optimal solution, hence not solution is ound by constrained NSGA-III or them. GD metric values are calculated and presented in the table. Interestingly, the algorithm perorming better in terms o IGD metric also shows a better GD metric value. Figures 9 and show that both algorithms are able to ind Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

9 Fig. 9. Obtained solutions using NSGA-III on three-objective C-DTLZ problem. Fig.. Obtained solutions using C-MOEA/D approach on three-objective C-DTLZ problem Fig.. Obtained solutions using NSGA-III on three-objective constraint convex C-DTLZ problem. Fig.. Obtained solutions using C-MOEA/D approach on three-objective constraint convex C-DTLZ problem. the disconnected Pareto-optimal regions or the three-objective C-DTLZ problem. Column 3 o Table V shows the number o reerence points out o total supplied reerence points are able to ind at least one Pareto-optimal solution. We discuss more about this in Section VII. However, in the case o convex C-DTLZ problem, as can be seen rom Table V, NSGA-III outperorms C-MOEA/D or three to 5-objective versions o the problem. Figure shows that on a three-objective convex C-DTLZ problem, NSGA-III is able to ind the easible Pareto-optimal points. No point in the intermediate ineasible portion o the ront is ound. However, as shown in Figure, although intermediate ineasible points were not ound by C-MOEA/D, it could not ind the points on the ront boundary. A similar observation was also made while solving the convex DTLZ problem using the unconstrained MOEA/D algorithm []. D. Constrained Problems o Type-3 Type-3 problems involve multiple constraints and the entire Pareto-optimal ront o the unconstrained problem need not be optimal any more, rather portions o the added constraint suraces constitute the Pareto-optimal ront. We modiy DTLZ and DTLZ4 problems or this purpose here by adding M dierent constraints. In the case o C3-DTLZ problem, objective unctions are same as in the original ormulation [4], however, ollowing M linear constraints are added: M c j (x) = j (x)+ i(x), j =,,...,M..5 i=,i j (7) For C3-DTLZ problem, k = 5 is used in the original g-unction, thereby making a total o (M +4) variables. Figure 3 shows constraints and easible region or the twoobjective C3-DTLZ problem. Notice how the unconstrained Pareto-optimal ront is now ineasible by the presence o two constraints. Similarly, problem DTLZ4 is modiied by adding M quadratic constraints o the type: c j (x) = j M 4 + i (x), j =,,...,M. i=,i j (8) Another diiculty posed by DTLZ4 is that it introduces bias or creating solutions in certain parts o the objective space. For this problem, we have used n = M +4 variables. Figure 3 shows the respective constraints and resulting Pareto-optimal Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

10 TABLE V BEST, MEDIAN AND WORST IGD AND GD METRIC VALUES OBTAINED FOR NSGA-III AND C-MOEA/D ON M -OBJECTIVE C-DTLZ AND CONVEX C-DTLZ PROBLEMS.BEST PERFORMANCE IS SHOWN IN BOLD. U IS THE NUMBER OF USEFUL REFERENCE POINTS THAT GENERATED A PARETO-OPTIMAL SOLUTION AND H IS THE TOTAL NUMBER OF SUPPLIED REFERENCE POINTS. Problem M U/H MaxGen NSGA-III C-MOEA/D IGD GD IGD GD C-DTLZ / / / / / C-DTLZ Convex 3 47/ / / / / Paretoront Ineasible Fig. 3. Two objective version o C3-DTLZ problem..5.5 Paretoront Ineasible.5.5 Fig. 4. Two objective version o C3-DTLZ4 problem. ront or the C3-DTLZ4 problem. For C3-DTLZ problem, Table VI shows that NSGA- III outperorms C-MOEA/D mostly in the case o higherobjective problems. Figures 5 and 6 show that both algorithms are able to ind a airly uniormly distributed set o points over the entire Pareto-optimal ront or three-objective C3-DTLZ problem. It is clear how the three constraints take their share o the Pareto-optimal ront in this problem. The GD metric value or three to eight-objective C3-DTLZ problems is better or C-MOEA/D algorithm, whereas or higher objective problems GD metric value is better or NSGA-III. In the case o C3-DTLZ4, Figure 7 shows the distribution o points ound by NSGA-III or three-objective C3-DTLZ4 problem corresponding to the run having the median IGD value. Clearly, NSGA-III is able to locate points on all three quadratic constraint suraces, thereby inding points on the entire Pareto-optimal ront. On the contrary, C-MOEA/D is not able to ind a single point on two o the three constraint suraces, as shown in Figure 8. Table VI shows that NSGA-III has perormed well on all objective versions o this problem in terms o both IGD and GD metric values. C-MOEA/D, like in C3-DTLZ problem, is able to solve three and iveobjective version or some runs, but it is not able to solve higher-objective versions o the problem very well. E. Discussion o the Results Based on above results on three types o constrained manyobjective optimization problems, it can be concluded that the proposed NSGA-III perorms airly well in all types o problems considered in this study. It is also important to note that the successul application o NSGA-III has come without having to ix any additional parameter other than the usual genetic parameters, such as population size, operator probabilities, etc. A careul handling o ineasible and easible solutions through the constraint-domination principle and in creating ospring population emphasize easible and lessviolated ineasible solutions. NSGA-III s non-dominated sorting, updated normalization process o objectives, association mechanism or linking a population member with a reerence Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

11 TABLE VI BEST, MEDIAN AND WORST IGD AND GD METRIC VALUES OBTAINED FOR NSGA-III AND C-MOEA/D ON M -OBJECTIVE C3-DTLZ AND C3-DTLZ4 PROBLEMS.BEST PERFORMANCE IS SHOWN IN BOLD. Problem M MaxGen NSGA-III C-MOEA/D IGD GD IGD GD C3-DTLZ C3-DTLZ Fig. 5. Obtained solutions using NSGA-III on three-objective C3-DTLZ problem. Fig. 6. Obtained solutions using C-MOEA/D approach on three-objective C3-DTLZ problem. point, and the niching operator to careully choose members rom the last accepted non-dominated ront are all able to produce a correct signal or providing an adequate emphasis or easible and ineasible solutions in the population and help in progressing towards the Pareto-optimal ront on all three types o constrained search regions. NSGA-III has repeatedly shown its successul perormance on three to 5-objective versions o these challenging problems. Based on the principles used or emphasizing easible and ineasible population members in constrained NSGA-III, we have also suggested a constrained MOEA/D algorithm (C- MOEA/D) that has also been ound to perorm well on most o these test problems. However, in more challenging problems, particularly having a large number o objectives (M >5), it has not been able to perorm as well as NSGA-III. It is worth noting here that C-MOEA/D requires our extra parameters to be set properly in a problem. In this study, we have used the values suggested in the original MOEA/D study [5], but a parametric study is needed to determine i some other values would allow the proposed C-MOEA/D to perorm better Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

12 Fig. 7. Obtained solutions using NSGA-III on three-objective C3-DTLZ4 problem. Fig. 8. Obtained solutions using C-MOEA/D approach on three-objective C3-DTLZ4 problem. on these challenging problems. For brevity, we belabor this parametric study or C-MOEA/D and leave urther study with the C-MOEA/D approach and investigate how the proposed NSGA-III algorithm will perorm on some practical manyobjective optimization problems in the next section. F. Engineering Constrained Optimization Problems Having tested NSGA-III s ability in solving various kinds o constrained test problems, it is now applied to two engineering design optimization problems. The irst problem has three objectives and ten constraints, while the second one has ive objectives and seven constraints. ) Car Side Impact Problem: This problem aims at minimizing the weight o car and at the same time minimize the pubic orce experienced by a passenger and the average velocity o the V-Pillar responsible or withstanding the impact load. All the three objectives are conlicting, thereore, a three-dimensional trade-o ront is expected. There are ten constraints involving limiting values o abdomen load, pubic orce, velocity o V-Pillar, rib delection, etc. There are design variables describing thickness o B-Pillars, loor, crossmembers, door beam, roo rail, etc. Mathematical ormulation or the problem is given in the Appendix. For this problem, p = 6 is chosen so that there are H = ( ) or 53 reerence points in total. The reerence points are initialized on the entire normalized hyper-plane on the three-objective space. NSGA-III is applied with 56 population members and run or 5 generations. Other parameters are kept the same as beore. Out o 53 reerence points 95 unique solutions corresponding to 95 reerence points are ound. These points are shown in Figure 9. No solution with an association to other 58 reerence points is ound, meaning that these reerence points may not correspond to any easible trade-o points. These results are next tested against a classical generative method. For this purpose, 6,6 reerence points are created (by taking p =). Thereater, the ideal and the nadir points are estimated using the 95 solutions obtained by NSGA-III. The ideal and the nadir points are then used to normalize the objectives. Now, corresponding to each o 6,6 reerence Fig unique solutions are ound on the entire ront on the threeobjective car-side impact problem. points, the PBI metric is minimized using Matlab s mincon routine, which uses a classical single-objective constrained optimization method. The resulting 6,6 points are then collected and dominated points are removed. Remaining nondominated points are then shown in Figure 9 in small dots. The igure clearly shows that all the points ound by NSGA- III are nicely distributed over the entire surace ormed by the classical generative procedure. To investigate the closeness o NSGA-III points with that obtained by the classical generative procedure, the convergence metric (average distance o NSGA- III points rom the closest mincon optimized points) is computed and the minimum, median and maximum convergence metric values are ound to be 9.8( 4 ),.( 3 ),and.3( 3 ), respectively. These values are small and they clearly indicate that NSGA-III is able to converge close to the true trade-o ront o this problem. The spread o solutions is demonstrated visually through Figure 9. We have observed that in solving the constrained test problems o Type (reer Table V), not all reerence points resulted in a easible solution. We observed a similar phenomenon occurring in a practical optimization problem in the original study [] as well. Applying an algorithm on careully designed Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

13 3 test problems help evaluate the algorithm s perormance on challenging problems beore they are applied to a practical problem. Since NSGA-III perormed airly well on such test problems, it is interesting to note that NSGA-III is able to solve a practical problem exhibiting a similar challenge. However, i many problems in practice have such a property, some o NSGA-III s eort would go waste in trying to ind a solution corresponding to a reerence point that do not end up creating any easible solution. We address this issue later in Section VII and suggest an adaptive NSGA-III algorithm or automatically identiying such non-useul reerence points. ) Water Problem: This is a ive objective problem taken rom literature [], [5]. There are three design variables and seven constraints. Mathematical ormulation or the problem is given in the appendix. reerence points are created using p =6and NSGA-III algorithm (with N =)isranor,generationsor this problem. To show a ive-objective trade-o ront, we irst identiy the ideal and nadir points rom a set o NSGA-III points obtained with dierent runs rom dierent initial populations. Then, the objective values are normalized and presented on a value path plot in Figure. The igure shows that NSGA-III is able to ind well-distributed set o tradeo points. Objective Value Objective No. Fig.. A value path plot or the ive-objective water problem showsthat NSGA-III is able to ind a well-distributed set o trade-o solutions. To investigate the near-optimality o constrained NSGA-III solutions, we solve the same problem using Matlab s mincon method in a generative manner or 4,845 reerence points one at a time. O them, 4,53 obtained trade-o solutions are ound to be easible. These mincon solutions and the constrained NSGA-III solutions are shown in the scatter matrix plot in Figure. The lower let plots are shown or mincon results and the upper right plots are or constrained NSGA-III solutions. For convenience o comparison o two similar plots, the (i, j)-th (i >j)shouldbecomparedwith(j, i)-th plot, or which the axes are interchanged or an easier viewing. Notice that the solutions ound by the constrained NSGA-III are widely distributed on the entire Pareto-optimal ront. VI. CONSTRAINED NSGA-III WITH PREFERRED REFERENCE POINTS So ar, we have demonstrated constrained NSGA-III s ability to ind a well-distributed set o points on the entire Paretooptimal ront. For this purpose, we started with a set o reerence points that are uniormly distributed on the normalized hyper-plane using Das and Dennis s structured approach [6]. However, in some practical scenario and or the purpose o Fig.. A scatter plot showing constrained NSGA-III results (top-right plots) vis-a-vis classical generative results obtained using mincon (bottomlet plots). decision-making, only a ew solutions ( or so) may be desired to be ound on a preerred part o the Pareto-optimal ront. We demonstrate here the constrained NSGA-III s ability or such a purpose. In the case o inding a preerred set, the user will supply a set o preerred reerence points (H p )intheregionohis/her preerence. In addition, we include M extreme reerence points (,,,...,) T, (,,,...,) T and so on, to make the normalization process to work well and make a total o H p + M reerence points (set H). These extreme points are needed to ensure the ideal and nadir points o the population members are properly calculated or the normalization purpose in the NSGA-III algorithm. We then run the NSGA-III algorithm as it is. First, we solve the constrained type DTLZ problem (C- DTLZ) introduced in this paper. Recall that this problem introduces diiculty or an algorithm to approach the Paretooptimal ront, as only the region close to the Pareto-optimal ront is easible. Only ive reerence points (H p )arechosen in the middle o the normalized hyper-plane, as shown in Figure. Three more extreme points are added to make a total o eight reerence points (set H). The igure shows a typical outcome o the NSGA-III procedure run with 48 population members or 75 generations and using the reerence set H. Ater eight solutions are obtained, only the ones corresponding to the supplied preerred reerence points (H p )arereported. However, i the user would like to know the extreme Paretooptimal points, they can also be reported, as these solutions already exist in the inal population. Since C-DTLZ problem is scalable in terms o number o objectives, next we try a -objective version o C- DTLZ problem. In this case, only preerred reerence points are randomly generated in the intermediate portion ( i [.4,.6]). extreme points are added to the set in creating the reerence point set H and constrained NSGA-III procedure is applied with a population size and run or Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

14 Fig.. A preerred set o reerence points ind corresponding Pareto-optimal solutions or C-DTLZ problem. TABLE VII BEST, MEDIAN AND WORST IGD VALUES FOR C-DTLZ PROBLEM WITH RANDOMLY SUPPLIED REFERENCE POINTS. Problem M N MaxGen NSGA-III C-DTLZ ,5 generations. Table VII shows the IGD values obtained using the expected Pareto-optimal points rom the reerence points and the NSGA-III obtained points. Small IGD values indicate that near-pareto-optimal points are obtained in the case o -objective problem. Next, we apply the preerence-based NSGA-III procedure to the car side impact problem discussed in Subsection V-F. In this case as well, we speciy ive reerence points at the intermediate portion o the normalized hyper-plane ( i [.4,.6]). The constrained NSGA-III procedure is run with 8 population members or 5 generations and with a total o (5+3) or 8 reerence points. Figure 3 shows the obtained solutions on the trade-o ront obtained using Matlab s mincon procedure (discussed earlier). It can be seen that obtained solutions lie on the trade-o rontier. Fig. 3. Five preerred set o reerence points ind ive corresponding tradeo solutions or the car side impact problem. Points shown with dots are obtained earlier using classical mincon method. NSGA-III is designed to ind Pareto-optimal points that are closer to each o these reerence points in the sense that their perpendicular distance rom the extended reerence line is minimum. Now, in many constrained or even unconstrained problems, not every extended reerence line may intersect with the Pareto-optimal ront. Thus, there will be some reerence points with no Pareto-optimal point associated with them while others will have more than one point associated with them, and hence NSGA-III may not end up distributing all population members uniormly over the entire Pareto-optimal ront. We have witnessed this in certain problems in the original study [] while solving the practical problems and also in Section V-F o this study. To illustrate urther, let us consider the three-objective inverted DTLZ problem (which we describe later in Subsection VIII-A) shown in Figure 4. The corresponding Pareto- VII. A-NSGA-III: AN ADAPTIVE APPROACH TO NSGA-III NSGA-III requires a set o reerence points to be supplied beore the algorithm can be applied. We have suggested the ollowing in the original NSGA-III study. I the user has no particular preerence on a particular part o the Pareto-optimal ront, a structured set o points can be created by using Das and Dennis s approach [6] on a normalized hyper-plane a hyper-plane that is equally inclined to all objective axes and intersects each axis at one. For a three-objective problem, this means that the supplied reerence points are spread uniormly over the triangle with its apexes at (,, ) T, (,, ) T and (,, ) T.Areerencelineoreachreerencepointcanbe deined as a line joining the origin and the reerence point. Fig. 4. Only 8 out o 9 reerence points ind a Pareto-optimal solution. optimal ront is shown as a shaded triangle. I we use a set o structured reerence points (shown with open circles), they will lie uniormly on the normalized triangle (hyper-plane). It is clear that the reerence lines originating rom many o these reerence points do not intersect with the Pareto-optimal ront. Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

15 5 When optimized, these reerence points will end up associated with no Pareto-optimal point. We call these reerence points as useless reerence points and those that will generate a Pareto-optimal points are called useul reerence points, or our discussions here. To discuss urther, we use 9 reerence points (with p = )asshowninthefigure4,andrun the original NSGA-III procedure [] with a population size o 9 to ind the Pareto-optimal points. As shown in the igure, 8 Pareto-optimal points (shown in big solid circles) are ound rom 8 dierent useul reerence points, but the remaining 63 reerence points could not associate a Paretooptimal solution. The solutions marked in small open circles are duplicate solutions to the 8 useul reerence points. Since location o these solutions are not used in any signiicant way in the algorithm, their distribution is somewhat random. The presence o these random points makes the distribution o the inal population non-uniorm, but importantly it causes a waste in computational eorts in processing these solutions. One possible remedy to this problem is to increase the number o supplied reerence points (H) byincreasingp, so that relatively more points can now appear on the Paretooptimal ront, but this is particularly not a viable suggestion, as this will require a larger population size yielding more computational eorts and still there will be several population members which are just randomly distributed over Paretooptimal ront. Ideally, it would be good to allocate all reerence points in a such a manner so as to generate a uniormly distributed set o Pareto-optimal points, but the knowledge o which reerence points will create Pareto-optimal solutions in an arbitrary problem is not known a priori. To alleviate the diiculty, we suggest here an adaptive NSGA-III procedure to adaptively identiy non-useul reerence points and re-allocate them in the hope o creating a Pareto-optimal solution or each o them at the end. There are two modiications made on the NSGA-III procedure ater the new population P t+ o size N is created: ) Addition o new reerence points, and ) Deletion o existing reerence points. We describe the criterion and modus operandi o each o these two operations in the ollowing subsections. A. Addition o Reerence Points Note that ater the niching operation, P t+ population is created and the niche count ρ j (the number o population members that are associated with j-th reerence point) or each reerence point is updated. Recall that the population size (N) iskeptmoreorlessequaltothenumberochosen reerence points (H). Thus, it is expected that i all reerence points are useul in inding a non-dominated point, ρ j = or all reerence points. But i ρ j is observed or any (j-th) reerence point (considered crowded), this has probably happened at the expense o some other reerence point (say k- th one) or which ρ k =.Ik-th reerence point is supposed to be a useul one, the NSGA-III procedure will eventually ind an associated population member or it. But i k-th reerence point is useless, then NSGA-III will never ind an associated population member. It is then better to replace the k reerence point with a new reerence point close to the crowded j-th reerence point. However, we do not have a priori knowledge about the eventual useulness o the k-th reerence point. In this case, we simply add a set o reerence points centering around the crowded j-th reerence point. The procedure is described in Figure 5. Fig. 5. Addition o reerence points. ( We simply introduce a simplex o M points (obtained using M+p ) p with p =)havingadistancebetweenthemsame as the distance between two consecutive reerence points on the original simplex. For example, or M =3objectives, three new points will be added around the j-th reerence point, as shown in the igure. I there are more than one reerence points or which ρ j, theaboveinclusionstepisexecutedor each o these reerence points. Beore a new reerence point is accepted, two checks are made: (i) i it does not lie on the irst quadrant, it is not accepted, and (ii) i it already exists in the set o reerence points, it is not accepted. The j-th reerence point is not allowed to have another inclusion operation until all original reerence points have a chance to be operated or inclusion as above. As it may have become clear rom above discussion that in many cases it might happen that too many reerence points are added and many o them eventually become non-useul, that is no population member is associated with them. Also i too many reerence points exist, they will slow down the algorithm. Thus, we consider the possibility o deleting non-useul reerence points, as described in the ollowing subsection. B. Deletion o Reerence Points Ater the inclusion operation is perormed, the niche count o all reerence points are updated. Note that H j= ρ j = N. Now, i there exists exactly N reerence points having ρ j =, that is, we have a scenario in which N o the reerence points have one associated member rom P t+,wehaveaperectscenario in which points are well-distributed among the reerence points. We then arrange to delete all included reerence points (excluding the original reerence points) having ρ j =.Thus, the original reerence points are always kept (even i their Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

16 6 niche count is zero) and all those included reerence points that have an niche count exactly one. The inclusion and deletion operations adaptively relocate reerence points based on the niche count values o the respective reerence points. We now show the results o this adaptive NSGA-III algorithm to a number o challenging problems including a couple o practical optimization problems. VIII. RESULTS WITH A-NSGA-III First, we consider two test problems the irst problem is an unconstrained inverted DTLZ problem and the second problem is a constrained type problem. For both problems, three and ive-objective versions are tried and the hypervolume indicator is used as a perormance measure. A. Inverted DTLZ Problem The DTLZ problem is modiied so that the corresponding Pareto-optimal ront is inverted. The problem is such that minimum o each objective unction has a unique solution. It is called an inverted unction because it is in disagreement with our deined hyper-plane or which the maximum (and not minimum) point o each objective among all points on the hyper-plane is a unique point. One eature o this problem is that many reerence points on created on the normalized hyper-plane will not have an associated Pareto-optimal point. Thereore, the use o A-NSGA-III may turn out to be an useul algorithm. The objective unctions are calculated using the original ormulation o DTLZ problem, however ater calculating the objective unction values, the ollowing transormation is made: i (x).5( + g(x)) i (x), (9) where g(x) was deined in DTLZ ormulation [4]. Three and ive-objective versions o this problem are solved using A-NSGA-III. To show the useulness o the adaptive method, i any, we also compare it with the original NSGA- III procedure, where no update o reerence points is made. In case o three-objective inverted DTLZ problem, NSGA- III is able to ind only 8 well-distributed points (as shown in Figure 6). Here, or visual clarity, only the closest population member or each useul reerence point (having ρ > ) is shown, while rest members which are just randomly distributed are not shown. Although 9 reerence points were supplied, only 8 o them could ind a representative Paretooptimal point and hence out o 9 population members only 8 are well distributed. Figure 7 shows the distribution o solutions with A-NSGA-III. 8 well-distributed points are now ound. This is a remarkable perormance o our proposed adaptive procedure. Next, we show the results on ive-objective inverted DTLZ problem using the hyper-volume metric. Table VIII shows the best, median and worst hyper-volume values obtained in runs or both the algorithms. Clearly, the use o the adaptive approach is able to ind an increased hyper-volume value in both three and ive-objective cases. Both NSGA-III and A- NSGA-III approaches use an identical number o population TABLE VIII BEST, MEDIAN AND WORST HYPER-VOLUME VALUE OBTAINED FOR NSGA-III AND A-NSGA-III ON M -OBJECTIVE INVERTED DTLZ AND C-DTLZ PROBLEMS.BEST PERFORMANCE IS SHOWN IN BOLD. M MaxGen NSGA-III A-NSGA-III Inverted DTLZ Problem Constrained DTLZ Problem members or computing the hyper-volume metric. In the case o NSGA-III, some reerence points may have more than one associated population members. Since only the one closest to each reerence point is used or the niching purpose, other associated population members may not be well-distributed on their own. On the other hand, A-NSGA-III reallocates nonuseul reerence points in a structured manner so that each reerence point can ind an associated population member or obtaining a better diversity o points. Thus, the hyper-volume metric value is better or A-NSGA-III. B. Type- constrained DTLZ Problem (C-DTLZ) As mentioned in Section V-C, in C-DTLZ problem, the Pareto-optimal ront is disconnected, that is, there are natural gaps in Pareto-optimal ront. Thus, there may exist some original reerence points with no associated Pareto-optimal point. In such a case, not all points obtained by NSGA-III will be well-spread, however as discussed above, with the adaptive approach a better distribution o points can be achieved. Aseto9reerencepointsaresuppliedinitially.Figure9 clearly shows that on three-objective C-DTLZ problem, A- NSGA-III inds 9 well-distributed set o points on the easible part o the Pareto-optimal ront, whereas the constrained NSGA-III inds only 58 such points (Figure 8). The superior perormance o A-NSGA-III is also clear rom the larger hyper-volume value depicted in Table VIII or three and iveobjective inverted DTLZ and constrained DTLZ problems. C. Two Problems rom Practice Having shown the improved perormance o A-NSGA-III on two three and ive-objective test problems, we now test the method to a couple o practical optimization problems. ) Crash-worthiness in Design o Vehicles: This is a threeobjective unconstrained problem considered in []. NSGA- III results are re-plotted here or convenience in Figure 3. Although 9 reerence points were chosen, only 4 o them are able to ind associated trade-o solutions. We now apply A-NSGA-III with an identical parameter setting. As shown in Figure 3, 83 unique solutions are now ound. A comparison Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

17 Fig. 6. Obtained solutions using NSGA-III on three-objective inverted DTLZ problem (only the closest solution or every useul reerence point is shown). Fig. 7. Obtained solutions using adaptive reerence points basednsga- III approach on three-objective inverted DTLZ problem (only the closest solution or every useul reerence point is shown) Fig. 8. Obtained solutions using NSGA-III on three-objective C-DTLZ problem (only the closest representative easible point or each reerence point is shown). Fig. 9. Obtained solutions using A-NSGA-III approach on three-objective C-DTLZ problem (only the closest representative easible point or each reerence point is shown). o the two igures reveal that the points ound by A-NSGA-III are more dense and describes the nature o the trade-o ront more clearly than that with NSGA-III mainly due to ormer s ability to re-allocate reerence points more uniormly on the ly. It is worth mentioning that A-NSGA-III does not require any additional parameter than what are needed in NSGA- III. A-NSGA-III adaptively re-allocates the supplied reerence points so that more easible trade-o points can be discovered. ) Car-side Impact Problem: This is a constrained optimization problem and was discussed in Section V-F. The tradeo ront obtained earlier had a dierent shape than the chosen normalized reerence plane on which the reerence points are supplied. The obtained ront is reproduced here in Figure o 56 reerence points could ind a easible trade-o solution using NSGA-III. Identical parameter values are used with A-NSGA-III and obtained trade-o ront is shown in Figure 33. Now, all 56 reerence points ind an associated trade-o point. A comparison o the two igures reveals that A-NSGA-III points are more dense and provide a better picture o the trade-o ront than that obtained using NSGA-III. Importantly, all 56 reerence points were considered in NSGA-III in all generations and the procedure has resulted in only 95 easible and well-distributed trade-o points, whereas by processing 56 reerence points, A-NSGA-III is able to ind 56 dierent and well-distributed set o trade-o points. These results amply show the useulness o A-NSGA-III approach or solving practical many-objective optimization problems. Above results on the test problems and two practical problems clearly show the eicacy o the adaptive NSGA- III approach. With an increase in number o objectives, realworld problems are likely to possess complicated ronts, as it is evident rom several practical test problems used in the original study [] and in this study. Thus, in such cases and in cases where no inormation about the shape, orientation, discontinuity, convexity etc. o the Pareto-optimal ront is known beorehand, the A-NSGA-III approach may be ound to be useul. IX. CONCLUSIONS In this paper, we have extended the recently proposed NSGA-III approach or solving many-objective optimization Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

18 Fig. 3. Obtained solutions using NSGA-III on the three-objective crashworthiness problem. Fig. 3. Obtained solutions using A-NSGA-III approach on the threeobjective crash-worthiness problem Fig. 3. Obtained solutions using NSGA-III on three-objective car-side impact problem. Fig. 33. Obtained solutions using adaptive reerence points basednsga- III approach on three-objective car-side impact problem. problems with box constraints to generic many-objective constrained optimization problems. The constraint-domination principle, instead o the usual domination principle, has been suggested or classiying population members into dierent non-dominated ronts. Furthermore, a modiied tournament selection operator has been applied along with recombination and mutation operators or generating the ospring population. The constrained algorithm is such that when there is no constraint in a problem or when all population members are easible in a particular generation, the approach is identical to the original NSGA-III approach developed or solving box-constrained problems in []. Like the original NSGA- III approach, the proposed constrained NSGA-III algorithm does not require any additional parameter. This remains as a hallmark property o the proposed algorithm. Additionally, this study has suggested an eicient extension o MOEA/D algorithm or handling constraints. Although our proposed constraint handling part o the algorithm does not require any parameter, C-MOEA/D s other operations require our parameters. To evaluate both constraint handling algorithms, this paper has also suggested three types o test problems providing three dierent kinds o challenges to any many-objective constraint handling algorithm. Both algorithms have shown their ability to solve most o the test problems involving three to 5 objectives, but the proposed C-MOEA/D algorithm has shown its weakness in solving problems having a large number o objectives. A parametric study is then needed to improve its perormance. However, on most problems the proposed constrained NSGA-III has been able to converge and ind a well-distributed set o points up to 5- objective problems. As a by-product o the development o the above two algorithms, this paper has also suggested three dierent types o scalable constrained test problems or many-objective optimization. Hopeully, these test problems and their subsequent modiications will oer adequate challenges to many-objective EMO algorithms in the years to come. Based on the success o NSGA-III on test problems, it is then applied to two engineering design problems involving three and ive objectives. In both cases, NSGA-III is able to ind a well-distributed set o trade-o solutions. The constrained NSGA-III algorithm has also been shown to perorm satisactorily on problems where a ew preerred reerence points are supplied, thereby suggesting the practical useulness o the constrained NSGA-III algorithm. It has been observed that reerence point or reerence Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

19 9 direction based approaches may, in some problems, ail to ind auniqueassociatedpareto-optimalsolutionoreachsupplied reerence points, particularly i theoretically there does not exist such a Pareto-optimal point or a reerence point. This eature has been ound to be commonly present in realworld many-objective constrained problems. We have then suggested an adaptive NSGA-III approach which adaptively adds and deletes reerence points depending on the crowding o population members on dierent parts o the current nondominated ront. The approach has been tested on two dierent types o problems having three and ive objectives. In all problems, the proposed A-NSGA-III has been ound to discover more and well-distributed points on the Pareto-optimal ronts. An application o A-NSGA-III to two engineering design problems have also shown the useulness o the proposed adaptive approach. With the two-part study ([] and this paper) presenting the development and application o an evolutionary manyobjective optimization algorithm based on the ramework o NSGA-II, we believe that we have addressed a long-awaited issue in the area o evolutionary multi-objective optimization (EMO). Testing on a number o unconstrained problems having box constraints alone in [] and on generic constrained problems in this study amply suggest that evolutionary methods can be useul as well to solve many-objective optimization problems (shown up to 5 objectives in both studies). Although urther improvements are possible, the proposed adaptive NSGA-III approach should also remain as a useul algorithm or adaptively relocating reerence points in the relevant part o the Pareto-optimal ront. Importantly, these two extensive studies have paved directions or uture research and should motivate EMO researchers to develop urther and better algorithms or many-objective optimization in the near uture. ACKNOWLEDGMENT The authors acknowledges the support provided by Michigan State University, East Lansing, USA or their visit during which this work was initiated. REFERENCES [] K. Deb and H. Jain, An improved NSGA-II procedure or manyobjective optimization Part I: Problems with box constraints, Indian Institute o Technology Kanpur, Tech. Rep. 9,. [] K. Deb, Multi-objective optimization using evolutionary algorithms. Chichester, UK: Wiley,. [3] C. A. C. Coello, D. A. VanVeldhuizen, and G. Lamont, Evolutionary Algorithms or Solving Multi-Objective Problems. Boston, MA:Kluwer,. [4] K. C. Tan, E. F. Khor, and T. H. Lee, Multiobjective Evolutionary Algorithms and Applications. London, UK: Springer-Verlag, 5. [5] K. Deb and D. Saxena, Searching or Pareto-optimal solutions through dimensionality reduction or certain large-dimensional multi-objective optimization problems, in Proceedings o the World Congress on Computational Intelligence (WCCI-6), 6,pp [6] E. J. Hughes, Evolutionary many-objective optimisation: Many once or one many? in IEEE Congress on Evolutionary Computation (CEC- 5), 5,pp. 7. [7] Q. Zhang and H. Li, MOEA/D: A multiobjective evolutionary algorithm based on decomposition, Evolutionary Computation, IEEE Transactions on, vol.,no.6,pp.7 73,7. [8] D. K. Saxena, J. A. Duro, A. Tiwari, K. Deb, and Q. Zhang, Objective reduction in many-objective optimization: Linear and nonlinear algorithms, IEEE Transactions on Evolutionary Computation, inpress. [9] D. Hadka and P. Reed, Borg: An auto-adaptive many-objective evolutionary computing ramework, Evolutionary Computation, p. in press,. [] J. A. López and C. A. C. Coello, Some techniques to deal with manyobjective problems, in Proceedings o the th Annual Conerence Companion on Genetic and Evolutionary Computation Conerence. New York: ACM, 9, pp [] H. Ishibuchi, N. Tsukamoto, and Y. Nojima, Evolutionary manyobjective optimization: A short review, in Proceedings o Congress on Evolutionary Computation (CEC-8), 8,pp [] K. Sindhya, K. Miettinen, and K. Deb, A hybrid ramework or evolutionary multi-objective optimization, IEEE Transactions on Evolutionary Computation, inpress. [3] K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan, A ast and elitist multi-objective genetic algorithm: NSGA-II, IEEE Transactions on Evolutionary Computation, vol.6,no.,pp.8 97,. [4] E. Zitzler, M. Laumanns, and L. Thiele, SPEA: Improving the strength Pareto evolutionary algorithm or multiobjective optimization, in Evolutionary Methods or Design Optimization and Control with Applications to Industrial Problems, K.C.Giannakoglou,D.T.Tsahalis, J. Périaux, K. D. Papailiou, and T. Fogarty, Eds. Athens, Greece: International Center or Numerical Methods in Engineering (CIMNE),, pp. 95. [5] M. A. Jan and Q. Zhang, MOEA/D or constrained multiobjective optimization: Some preliminary experimental results, in Proceedings o UK Workshop on Computational Intelligence,,pp. 6. [6] H. Li and Q. Zhang, Multiobjective optimization problems with complicated Pareto sets, MOEA/D and NSGA-II, IEEE Trans on Evolutionary Computation, vol.,no.,pp.84 3,9. [7] R. Storn and K. Price, Dierential evolution A ast and eicient heuristic or global optimization over continuous spaces, Journal o Global Optimization, vol.,pp ,997. [8] C. M. Fonseca and P. J. Fleming, Multiobjective optimization and multiple constraint handling with evolutionary algorithms Part II: Application example, IEEE Transactions on Systems, Man, and Cybernetics: Part A: Systems and Humans, vol.8,no.,pp.38 47,998. [9] K. Deb, A. Pratap, and T. Meyarivan, Constrained test problems or multi-objective evolutionary optimization, in Proceedings o the First International Conerence on Evolutionary Multi-Criterion Optimization (EMO-),,pp [] S. Kukkonen and K. Deb, Improved pruning o non-dominated solutions based on crowding distance or bi-objective optimization problems, in Proceedings o the Congress on Evolutionary Computation (CEC-6), 6,pp [] V. Chankong and Y. Y. Haimes, Multiobjective Decision Making Theory and Methodology. New York: North-Holland, 983. [] K. Miettinen, Nonlinear Multiobjective Optimization. Boston: Kluwer, 999. [3] K. Deb and R. B. Agrawal, Simulated binary crossover or continuous search space, Complex Systems, vol.9,no.,pp.5 48,995. [4] K. Deb, L. Thiele, M. Laumanns, and E. Zitzler, Scalable test problems or evolutionary multi-objective optimization, in Evolutionary Multiobjective Optimization, A. Abraham, L. Jain, and R. Goldberg, Eds. London: Springer-Verlag, 5, pp [5] T. Ray, K. Tai, and K. C. Seow, An evolutionary algorithm or multiobjective optimization, Engineering Optimization, vol.33,no.3, pp ,. [6] I. Das and J. Dennis, Normal-boundary intersection: A new method or generating the Pareto surace in nonlinear multicriteria optimization problems, SIAM Journal o Optimization, vol. 8, no. 3, pp , 998. Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.

20 APPENDIX CAR-SIDE IMPACT PROBLEM FORMULATION Mathematical ormulation o the three-objective problem is given below. All objectives are to be minimized. (x) = x +6.67x +6.98x 3 +4.x x 5 +.x x 7, (x) =F (x) =.5(V MBP + V FD ) g (x) =.6.377x x 4.998x 3 Himanshu Jain Himanshu Jain was born in Nehtaur, India, on December 6, 989. He received his dual Bachelor s and Master s degree in Mechanical Engineering rom Indian Institute o Technology Kanpur in. He was a member o the Kanpur Genetic Algorithms Laboratory (KanGAL) rom 9. He is currently working as an Engineer at Eaton Corporation, Pune. His research interests include evolutionary computation and machine learning. g (x) =.6.59x x.6486x.9x x x 3 x x 6.3 g 3 (x) = x x.3568x +.399x x 6.8x x x x 3.364x 5 x 6.8x.3 g 4 (x) =.74.6x.396x 3.387x 7 +.7x.3 g 5 (x) = x 3 4.x x +.796x x 7 3 g 6 (x) = x x x 3.795x 3.443x g 7 (x) = x 4.455x 3 g 8 (x) F =4.7.5x 4.9x x 3 4 g 9 (x) V MBP = x x.6775x 9.9 g (x) V FD = x 3 x 7.843x 5 x Variable bounds are given given as ollows:.5 x.5,.45 x.35,.5 x 3.5,.5 x 4.5,.875 x 5.65,.4 x 6.,.4 x 7. WATER PROBLEM FORMULATION Mathematical ormulation o the ive-objective problem is given below. All objectives are to be minimized. (x) =678.37(x + x 3 ) (x) =3.x (x) = x /(.6 89.).65 4 (x) =5. 89.exp( 39.75x +9.9x ) 5 (x) =5.((.39/(x x )) x 3 8.) g (x) =.39/(x x )+4.94x 3.8 g (x) =.36/(x x )+.8x g 3 (x) =.37/(x x ) x g 4 (x) =.98/(x x ) x g 5 (x) =.38/(x x ) x g 6 (x) =.47(x x )+7.6x g 7 (x) =.64/(x x )+63.3x x.45. x,x 3.. Kalyanmoy Deb Kalyanmoy Deb is the Koenig Endowed Chair Proessor at the Department o Electrical and Computer Engineering at Michigan State University (MSU), East Lansing, USA. He is also aproessorocomputerscienceandengineering, and Mechanical Engineering at MSU. Prior to this position, he was at Indian Institute o Technology Kanpur in India. Pro. Deb also has a visiting position at Aalto University School o Economics, Finland and University o Skövde, Sweden. Pro. Deb s main research interests are in evolutionary optimization algorithms and their application in optimization and machine learning. He is largely known or his seminal research in Evolutionary Multi- Criterion Optimization. He was awarded the prestigious Inosys Prize in, TWAS Prize in Engineering Sciences in, CajAstur Mamdani Prize in, JC Bose National Fellowship in, Distinguished Alumni Award rom IIT Kharagpur in, Edgeworth-Pareto award in 8, Shanti Swarup Bhatnagar Prize in Engineering Sciences in 5, Thomson Citation Laureate Award rom Thompson Reuters. His IEEE-TEC NSGA-II paper is now judged as the Most Highly Cited paper and acurrent Classic by Thomson Reuters having more than 4,+ citations. He is a ellow o IEEE. He has written two text books on optimization and more than34 international journal and conerence research papers. He is in the editorial board on 8 major international journals, including IEEE TEC. Copyright (c) 3 IEEE. Personal use is permitted. For any other purposes, permission must be obtained rom the IEEE by ing pubs-permissions@ieee.org.