Chapter 10 Multobjectve Optmzaton All men seek one goal: success or happness. The only way to acheve true success s to express yourself completely n servce to socety. Frst, have a defnte, clear, practcal deal a goal, an objectve. Second, have the necessary means to acheve your ends: wsdom, money, materals, and methods. Thrd, adjust all your means to that end. Arstotle Multobjectve optmzaton caters to achevng multple goals, subject to a set of constrants, wth a lkelhood that the objectves wll conflct wth each other. Multobjectve optmzaton can also be explaned as a multcrtera decson-makng process, n whch multple objectve functons have to be optmzed smultaneously. In many cases, optmal decsons may requre tradeoffs between conflctng objectves. Tradtonal optmzaton schemes use a weght vector to specfy the relatve mportance of each objectve and then combne the objectves nto a scalar cost functon. Ths strategy reduces the complexty of solvng a multobjectve problem by convertng t nto a sngle-objectve problem. Soluton technques for multobjectve optmzaton nvolve a tradeoff between model complexty and accuracy. Examples of multobjectve optmzaton can be found n economcs (settng monetary polcy), fnance (rsk return analyss), engneerng (process control, desgn tradeoff analyss), and many other applcatons n whch conflctng objectves must be obtaned. One of the prerequstes of multobjectve optmzaton s to determne whether one soluton s better than another. However, no smple method exsts for reachng such a concluson. Instead, multobjectve optmzaton methods commonly adopt a set of Pareto optmal solutons (also called nondomnated solutons), whch are alternatves wth dfferent tradeoffs between the varous objectves. In the soluton defned by a Pareto optmal set, one objectve cannot be mproved wthout degradng at least one other objectve n the set. It s up to the decson maker to select the Pareto optmal soluton that best fts preferred polcy or gudelnes. Pareto graphs llustrate the attrbutes of the tradeoff between dstnct objectves. The soluton can be represented n the shape of a curve, or a three-dmensonal surface that trades off dfferent zones n the multobjectve space. Ths chapter dscusses machne learnng methodologes for solvng Pareto-based multobjectve optmzaton problems, usng an evolutonary approach. The goal s to fnd a set of nondomnated solutons wth the mnmum dstance to the Pareto front n each generaton. Successve solutons are bult as part of the evolutonary process, n whch one set of selected ndvdual solutons gves rse to another set for the next generaton. Solutons wth hgher ftness measures are more lkely to be selected to the matng pool, on the assumpton that they wll produce a ftter soluton n the next generaton (next run), whereas solutons wth weaker ftness measures are more lkely to be dscarded. Such solutons possess several attrbutes that make them sutable for problems nvolvng (1) a large and complex search space and (2) mutually conflctng objectves. 185
Formal Defnton A multobjectve optmzaton problem deals wth a fnte number of objectve functons. In an optmzaton problem wth n objectves of equal mportance, all need to be mnmzed (or maxmzed) to serve a performance crteron. Mathematcally, the problem can be expressed as a vector of objectves f (x) that must be traded off n some manner, Fx ( ) = mn éë f 1 ( x), f 2 ( x), f 3 ( x),, fm( x) xîx ù û, (10-1) where X (see Equaton 10-2) s a set of n decson vectors (a decson space) that represents parameters for the values selected to satsfy constrants and optmze a vector functon, X = [,,,, ] T x x x x n (10-2) 1 2 3 low hgh x x x =1, 23,,, n. (10-3) The relatve sgnfcance of these objectves cannot be determned untl the tradeoffs between them are dstnctly understood. Because F(x) s a vector, competng objectve functons wll prevent t from achevng a unque soluton. You can assocate each soluton x n a decson space X wth a pont n objectve space Y, such that T f( x) Y y, y, y, 1 2 3 y m,. (10-4) = =[ ] In multobjectve optmzaton the sets X and Y are known as decson varable space and objectve functon space, respectvely. Fgure 10-1 llustrates the mappng of the search space to the objectve space. Every teraton of search space leads to a set of objectve vectors that defnes the objectve space, n whch several optmal objectve vectors may represent dfferent tradeoffs between the objectves. 186
Fgure 10-1. Multobjectve optmzaton problem: mappng the search space to the objectve space Pareto Optmalty Pareto optmalty s a concept bult on multobjectve optmzaton that facltates optmzaton of a vector of multple goals through tradeoffs between combnatons of multple objectves. Tradeoffs are formulated to mprove the performance of one objectve at the cost of one or more other objectves. As dsplayed n Fgure 10-1, each pont n the objectve space represents a unque set of model varables, such that Pareto optmalty categorzes multple Pareto optmal solutons. The term honors Vlfredo Pareto (1848 1923), an Italan economst who demonstrated that ncome follows a power law probablty dstrbuton. For an deal case the optmal soluton of a multobjectve problem s generally denoted as a Pareto set X * Í X. The correspondng outcome, or objectve vector, s represented by a Pareto front Y * + f ( X * ) Í Y. In practce an deal soluton s nonexstent, and solvng multobjectve optmzaton does not typcally produce an optmally unque soluton. Instead, we use Pareto optmal solutons, n whch one objectve cannot be mproved wthout degradng at least one of the other objectves. Therefore, when usng evolutonary technques, knowledge of the optmal Pareto set (X*, Y*) asssts n fndng a best-compromse soluton. Domnance Relatonshp A soluton x 1 domnates another soluton ( x x ) x f the followng condtons are met: 1 2 2 1. For all objectves, soluton x 1 s better than or equal to x 2, such that f( x1) f( x2) " Î123,,,, m. 2. For at least one objectve, soluton x 1 s strctly better than x 2, such that fj( x1) < fj( x2) $ jî123,,,, m. 187
If ether of these condtons s volated, then x 1 does not (Pareto) domnate the soluton x 2. The domnance relatonshp s nonsymmetrcal. For example, f the soluton x 1 does not domnate the soluton x2 ( x1 / x2), that does not mply that x 2 domnates x1( x2 x1); therefore, both solutons can be nondomnated. However, the domnance relatonshp s also transtve. For nstance, f x1 x2 and x2 x3, then x1 x3. Ths property allows us to dentfy whch solutons are not domnated (Xˆ ) by any member of the soluton set X. These nondomnated sets (Xˆ ) of the entre feasble search space are called globally Pareto-optmal sets. Generatng a Pareto set can be computatonally expensve. Therefore, you need to select a computatonally effcent method for determnng the Pareto-optmal set of a multobjectve optmzaton algorthm. Although you may employ many dfferent approaches to solve a multobjectve optmzaton problem, much work has been done n the area of evolutonary multobjectve optmzaton on the approxmaton of the Pareto set. Performance Measure To evaluate the performance of a soluton, t s essental to develop a measurement scheme that quantfes the qualty of the nondomnant Pareto front. The general performance crtera for multobjectve optmzaton algorthms can be summarzed as follows: 1. Convergence (g): Estmates the proxmty of the canddate nondomnated (Pareto) solutons to the best-known predcton or known set of Pareto optmal solutons. For each soluton obtaned usng an algorthm, you can use the mnmum Eucldan dstance (Deb, Pratap, and Agarwal 2002) to the Pareto optmal front. The average dstance can be used as the convergence measure. A smaller g value ndcates a better convergence. 2. Dversty (D): Provdes a decson maker wth effcent choces. Because you are nterested n the soluton that covers the entre Pareto-optmal regon, you need to evaluate the degree of spread between the solutons obtaned. 3. Dsplacement (D): In the case of algorthmc approxmatons or the presence of a dscontnuous Pareto-optmal front, only a porton of true optmal front may be reflected. Dsplacement s used to overcome ths lmtaton. Dsplacement measures the relatve proxmty of the canddate soluton set to a known set of Pareto-optmal solutons. Mathematcally, dsplacement can be expressed as * P 1 Q D = mn [ d (, j) ], P * J = 1 å = 1 (10-5) 188 where, P* = Unformly spaced solutons from the true Pareto-optmzed front Q = Fnal soluton d(i, j) = Eucldean dstance between the th soluton of P* and jth soluton of Q A lower dsplacement value represents better convergence and coverage. Each algorthm may select one or more performance crtera to test the qualty of a soluton. In many cases, the performance crtera may depend on the avalablty (or nonavalablty) of a known collecton of Pareto-optmal sets. The rest of ths chapter looks at varous multobjectve optmzaton solutons based on evolutonary learnng methodologes.
Machne Learnng: Evolutonary Algorthms Chapter 10 Multobjectve Optmzaton Generatng the Pareto set can be computatonally expensve, because multobjectve optmzaton problems no longer have a sngle optmal soluton, but a whole set of potental solutons. Classcal optmzers (Marler and Arora 2004) nclude weghted-sum approaches, perturbaton methods, Tchybeshev methods, goal programmng, and mn max methods. Although these methods can be used for multcrtera optmzaton, you can only obtan a sngle soluton for each smulaton run; smulaton needs to execute multple tmes, wth an expectaton that one of the solutons may lead to the Pareto-optmal soluton. Evolutonary algorthms (EAs) are well suted to solvng multobjectve optmzaton problems, because they mmc natural processes that are nherently multobjectve; a number of Pareto-optmal solutons can be captured n a sngle smulaton run. Addtonally, EAs are less senstve to the shape or contnuty of the Pareto front. These algorthms have been successfully appled to a wde range of combnaton problems, n whch nformaton from multple sources s brought together to acheve an optmal soluton. Such algorthms are partcularly useful n applcatons nvolvng desgn and optmzaton, n whch there are a large number of varables and n whch procedural algorthms are ether nonexstent or extremely complcated. Generally, evolutonary methods are populaton-based, metaheurstc optmzaton algorthms that mmc the prncples of natural evoluton. These methods use the ntal populaton of a soluton and update n each generaton to converge to a sngle optmal soluton. Although EAs do not guarantee a true optmal soluton, they attempt to fnd a good approxmaton, representng a near-pareto-optmal soluton. EAs are typcally classfed nto four major categores: (1) genetc algorthms (GAs), (2) genetc programmng (GP), (3) evolutonary programmng (EP), and (4) evoluton strategy (ES). Although these algorthms employ dfferent approaches, they all derve nspraton from the prncple of natural selecton. Fundamental processes nvolved n EAs are selecton, mutaton, and crossover. The frst stage of an EA entals applyng a ftness factor to evaluate the populaton n the objectve space (whch represents the qualty of the soluton). Next, a matng pool s created by selectng the populaton from prevous step, usng a random selecton or lkelhood-based selecton crteron. Once the matng pool s organzed, t s subjected to recombnaton and mutaton, whch produce a new populaton set. The recombnaton process performs an n-pont crossover, wth a confgurable probablty that allows fragments of one parent to combne wth fragments of another parent to create an entrely new chld populaton. Matng selecton s a crtcal step n the EA process, nasmuch as t attempts to select promsng solutons, on the assumpton that future matng pools derved as a consequence of a hgh-qualty selecton tend to be superor. A mutaton operator modfes ndvduals by makng small changes to the assocated vectors, accordng to a gven mutaton rate. Gven the probablstc nature of the matng and mutaton processes, certan populatons may not undergo any varaton and smply replcate to the next generaton. Analogous to natural evoluton, ndvduals represent possble solutons, and a set of ndvduals (or possble solutons) s called a populaton. Each ndvdual s encoded, usng a problem-specfc encodng scheme that can be decoded and evaluated by a ftness functon. The matng process terates through the process of modfyng an exstng populaton va recombnaton and mutaton to evolve a new populaton. Each loop teraton s called a generaton, whch represents a tmelne n the evolutonary process. Early work n the area of multobjectve EAs s credted to Davd Schaffer, who mplemented the vectorevaluated GA (VEGA) (Schaffer 1985). Goldberg(1989) proposed calculatng ndvdual ftness accordng to Pareto domnance. Many varants of multobjectve EAs have snce been suggested (of whch ths chapter consders some of the more popular). 189
Fgure 10-2. Basc flow of a GA Genetc Algorthm GAs follow the prncple of natural selecton (see Fgure 10-2), n whch each soluton s represented as a bnary (or real) coded strng (chromosomes) and an assocated ftness measure. Successve solutons are bult as part of the evolutonary process, n whch one set of selected ndvdual solutons gves rse to another set for the next generaton. Indvduals wth a hgh ftness measure are more lkely to be selected to the matng pool, on the assumpton that they wll produce a ftter soluton n the next generaton. Solutons wth the weaker ftness measures are naturally dscarded. Typcally, you can use roulette-wheel selecton to smulate natural selecton, n whch elmnaton of solutons wth a hgher functonal ftness s, although possble, less lkely. In ths method each possble selecton s assgned a porton of the wheel that s proportonal to ts ftness value, followed by a random selecton, analogous to spnnng a roulette wheel. A small lkelhood also exsts that some weaker solutons wll survve the selecton process, because they may nclude components (genes) that prove useful after the crossover process. Mathematcally, the lkelhood of selectng a potental soluton s gven by P = F N å j= 0 F j, (10-6) where P represents the lkelhood of th soluton s beng selected for the matng pool, F stands for the operatng ftness of th ndvdual soluton, and N s the total number of soluton elements n a populaton. GAs have proven useful n solvng complex problems wth large search spaces that are less understood by reason of lttle doman knowledge. The chromosomes of a GA represent the buldng blocks (alleles) of a soluton to the problem that s sutable for the genetc operators and the ftness functon. Canddate solutons undergo modfcaton, usng crossover and mutaton functons, and result n new canddate solutons that undergo evaluaton for canddacy n new matng pools. 190
Genetc Programmng Chapter 10 Multobjectve Optmzaton GP s an evolutonary technque that expands the genetc learnng paradgm nto an autonomous synthess of computer programs that, when executed, lead to canddate solutons. Unlke GAs, n whch populatons are fxed-length encoded character strngs representng canddate solutons, n GP, populatons are programs represented by syntax trees (also called parse trees). GP teratvely evolves the populatons of programs, transformng one set of programs nto another set by exercsng the genetc operatons crossover and mutaton. Crossover functon s mplemented by exchangng subtrees at a random crossover pont of two parent ndvduals (selected accordng to ftness crtera) n the populaton. Crossover creates an offsprng by replacng the subtree at the crossover pont of the frst parent wth the subtree of the second parent. In subtree mutaton (the most commonly used form of mutaton) the subtree of a randomly selected mutaton pont s replaced by the subtree of a randomly generated tree. Fgure 10-3 demonstrates the general flow and crossover operaton of a GP methodology usng two varables x and y and prefx notaton to express mathematcal operators. Parent 1 [+(*(x, y),2)] crosses over wth parent 2 [*(+(x,1),/(y,2))] and produces an offsprng represented by [+(/(y,2)),2)]. It s customary to use such prefx notaton to represent expressons n GP. Fgure 10-3. Basc flow of GP wth crossover operatons; after selectng random crossover ponts on both parents, a porton of parent 1 attaches to a porton of parent 2 to create an offsprng 191
Multobjectve Optmzaton: An Evolutonary Approach In sngle-objectve optmzaton, to evaluate the qualty of the soluton, you smply measure the value of the objectve functon. In the case of multobjectve optmzaton, t may not be possble to evaluate the qualty of the soluton relatve to optmal Pareto approxmatons, because you may not possess the relevant nformaton, wth respect to objectve space or coverage, and thus may not be able to defne the qualty of soluton, n terms of closeness to the optmal Pareto set and dversty of coverage. Even f one soluton domnates the other soluton, you may stll not be able to quantfy the relatve mprovement, because relatve dstance and dversty alone are not suffcent to quantfy the Pareto set approxmaton. Ths brngs us to the fundamental requrements for defnng the strategy for mplementng multobjectve EAs. These requrements can be summarzed as follows: Ftness: Gudng the soluton closer to the Pareto set. Ths requres constructng a scalar ftness functon that fulflls multple optmzaton crtera. Dversty mprovement: Improvng coverage by selectng a dverse set of nondomnated solutons. Ths avods a stuaton n whch dentcal solutons exst, relatve to objectve space and decson space. Eltsm: Preventng nondomnated solutons from beng elmnated. Most EAs dffer n the manner n whch they handle ftness, dversty, and eltsm. Lsted here are some of the most popular multobjectve EA (MOEA) approaches: Weghted-Sum approach Vector-Evaluated GA (VEGA) (Schaffer 1985) Multobjectve GA (MOGA) (Fonseca and Flemng 1993) Nched Pareto GA (NPGA) (Horn, Nafplots, and Goldberg 1994) Nondomnated sortng GA (NSGA) (Ndamarth and Deb 1994) Strength Pareto EA (SPEA) (Ztzler and Thele 1999) Strength Pareto EA II (SPEA-II) (Ztzler, Laumanns, and Thele 2001) Pareto archved evolutonary strategy (PAES) (Knowles and Corne 1999) Pareto envelope-based selecton algorthm (PESA) (Corne, Knowles, and Oates 2000) Pareto envelope-based selecton algorthm II (PESA-II) (Corne et al. 2001) Eltst nondomnated sortng GA (NSGA-II) (Deb, Pratap, and Agarwal 2002) These approaches are presented n turn n the followng sectons. Weghted-Sum Approach The weghted-sum method for multobjectve optmzaton delvers multple soluton ponts by varyng the weghts consstently. Dfferent objectves are merged nto a sngle objectve, and the composte functon s mnmzed, usng confgurable weghts. Mathematcally, the weghted-sum approach can be represented as m å F= w. f ( x) for w³ 0 and w = 1. = 1 m å (10-7) 192
For postve weghts, mnmzng F can result n a Pareto optmal soluton. Although ths method s computatonally effcent, the major drawback s that t cannot determne the weghts that can optmally scale the objectve functons for a problem wth lttle or no nformaton. Vector-Evaluated Genetc Algorthm VEGA s a populaton-based algorthm that extends the selecton operator of a smple GA (SGA), such that each generaton produces a number of dsjont subpopulatons, as a result of a proportonal selecton scheme, and s governed by dfferent objectves. For a problem wth m objectves and a total populaton of sze N, m subpopulatons of sze N / m are generated by ther respectve ftness functons. As depcted n Fgure 10-4, these subpopulatons are shuffled together to generate a new populaton of sze N. The scheme s effcent and easy to mplement, because only the selecton method of SGA s modfed. Fgure 10-4. Basc flow of a VEGA Because of proportonal selecton, the shufflng and mergng operatons of all the subpopulatons n VEGA result n an aggregatng approach. The drawback of ths scheme s ts nablty to fnd a large number of ponts on the Pareto optmal front because each soluton executes ts own objectve functon. VEGA s prone to fndng extreme solutons, owng to the parallel search drectons of the axes n the objectve space or smultaneous executon of multple-objectve functons. 193
Multobjectve Genetc Algorthm MOGA s another varant of SGA, dfferng n the way ftness s assgned to a soluton. In ths scheme, rank R s assgned to each soluton, usng the expresson Rx (, t) = 1 + n ( t), (10-8) where n s the number of solutons that domnate the th soluton x n generaton t. Once the rankng process s completed, the ftness of ndvduals s assgned by nterpolatng between the best rank (1) and the worst rank ( m) va a user-defned functon. The ftness of ndvduals of the same rank s averaged, allowng samplng at the smlar rate, whle mantanng selecton pressures. The ftness of certan ndvduals may degrade more than others, dependng on the sze of the ranked populaton. Rankng gudes the search to converge only on global optma. Solutons exhbtng good performance n many objectve dmensons are more lkely to partcpate n the matng process. Although the rankng process assgns the nondomnated solutons the correct ftness, t does not always guarantee samplng unformty n the Pareto set. When dealng wth multple objectves, genetc drft trggers a suboptmal behavor, n whch a large number of solutons tend to converge on a lesser number of objectves, owng to an mperfect selecton process. To prevent premature convergence and to dversfy the populaton, a nche-formaton method s adopted to dstrbute the populaton over the Pareto regon, n the objectve space. If the ftness of two ndvduals s closer than a certan nchng dstance, they are consdered part of same nche (.e., sharng the same ftness). Nche formaton dscourages convergence to a sngle regon of the ftness functon by ntroducng compettve pressures among nches that reduce the ftness of such locally optmal solutons. Nche formaton leads to dscovery of dverse regons of the ftness landscape. In nature a nche s regarded as an organsm s task n the envronment, and a speces s the collecton of organsms wth the same features. Nchng segments the GA populaton nto dsjont sets n such a manner that at least one member n each regon of ftness functon covers more than one local optmal. In one such method, you defne a parameter nche radus (s radus ). Any two ndvduals closer than ths dstance are consdered part of the same nche, sharng the same ftness value. Nchng lets the GA operate on the new shared ftness nstead of on the orgnal ftness of an ndvdual. Nchng reduces nterspeces competton and helps synthesze a stable subpopulaton around dfferent nches. In multobjectve optmzaton problems, a nche s ordnarly represented by the locale of each optmum n the search space, wth ftness as the resource of that nche. Nched Pareto Genetc Algorthm NPGA s a tournament selecton scheme based on Pareto domnance, n whch a comparson set of randomly selected ndvduals partcpates to determne the wnner between two canddate solutons. Each of the canddates s tested to determne domnance. The canddate that s nondomnated by the comparson set s selected for the matng pool. If both canddates are ether domnated or nondomnated by the comparson set, then they are lkely to belong to the same equvalence class. As shown n Fgure 10-5, for a gven nche radus (s share ) the selecton for the matng pool s determned by the nche class count. Canddates wth the least number of ndvduals n the equvalence class (least nche count) have the best ftness. In ths example, because both canddates are nondomnated, Canddate 1 s selected to the matng pool, on the bass of lower nche class count. 194
Fgure 10-5. Equvalence class sharng; canddate 1 (nche class count = 3) s a better ft than canddate 2 (nche class count = 4) MOGA and NPGA suffer from smlar drawbacks; both methods are hghly senstve to selecton of nche radus (s share ). Nondomnated Sortng Genetc Algorthm NSGA s another Pareto-based noneltst approach that dffers from SGA n the manner n whch the selecton operator s used. All the nondomnant solutons are selected frst and classfed as the frst nondomnant front n the populaton. To determne the members of the second nondomnant front, members of the frst nondomnant front are elmnated from the evaluaton process, and the search for nondomnance contnues wth the remanng populaton. Ths process of level elmnaton and nondomnance search wthn a shrnkng populaton contnues untl all the ndvduals of the populaton have been categorzed to a level of nondomnance. Levels of nondomnance range from 1 to p. Ftness s assgned to each category of the subpopulaton proportonally to the populaton sze. Solutons belongng to the lower levels of nondomnance have hgher ftness than those belongng to hgher levels. Ths mechansm mantans the selecton pressure to select ndvduals to the matng pool wth hgher ftness (members of lower levels of nondomnance), n a drecton toward the Pareto-optmal front. In the frst step the ntal dummy ftness, equal to the populaton sze, s assgned to ndvduals n the frst level of the nondomnance front. Based on the number of neghborng solutons (nche class count for a gven nche radus s share ) sharng the same front and the same level, the ftness value of an ndvdual s reduced by a factor of the nche count, and a new shared ftness value s recomputed for each ndvdual n ths level. For the ndvduals n the second nondomnance level, a dummy ftness smaller than the lowest shared ftness of the frst nondomnance level s assgned. Smlarly, ndvduals that are members of the thrd and all subsequent levels are assgned ftnesses n decreasng order, relatve to the lowest ftness of the lower levels. Ths guarantees that the ftness of ndvduals belongng to hgher levels of nondomnance s 195
always lower than that of ndvduals n the lower levels. Ths process contnues untl all ndvduals n the entre populaton have been assgned ther shared ftness. Once all the ftness values have been assgned, tradtonal GA processes related to selecton, crossover, and mutaton apply. Mathematcally, ths process can be explaned as follows: for k ndvduals wth a dummy ftness of f p and nche count of m p, as part of p nondomnance level, the shared ftness of each ndvdual can be calculated as ˆf p fp = p (10-9) m dummy ftness for ndvduals n the subsequent nondomnance level s gven as k f mn fˆ p-1 = ( )-e, p = 1 (10-10) where e s a small postve number. NSGA shares the same drawback as other algorthms n ths category: hgh senstvty to the nche radus s share. Strength Pareto Evolutonary Algorthm SPEA mplements eltsm and nondomnance by mergng several features of prevous mplementatons of multobjectve EAs. Eltst selecton prevents the qualty of good solutons from degradng, from one generaton to the next. In one of ts varants, the best ndvduals from the current generaton are carred to the next, wthout alteraton. Ztzler et al. (2001) defned the characterstcs of SPEA by referencng the followng attrbutes: 1. Creates an external and contnuously updatng nondomnated populaton set by archvng prevously found nondomnated solutons. At each generaton the nondomnated solutons are coped to the external nondomnated set. Unlke other EAs, n SPEA the relatve domnance of one soluton by other solutons wthn the populaton s rrelevant. 2. Apples external nondomnated solutons from step 1 to the selecton process by evaluatng an ndvdual s ftness, based on the strength of ts solutons that domnate the canddate soluton. 3. Preserves populaton dversty, usng the Pareto domnance relatonshp. Ths EA does not requre a dstance parameter (such as nche radus). 4. Incorporates a clusterng procedure to prune the nondomnated external set wthout destroyng ts characterstcs. As stated, ths algorthm mplements eltsm explctly by mantanng an external nondomnant populaton set ( P ). The algorthm flow conssts of the followng steps: 1. Intalze the populaton P of sze n. 2. Intalze an empty populaton P representng an external nondomnant soluton set archve. 3. Copy the nondomnated solutons of P to P. 4. Remove solutons contaned n P that are covered by other members of P (or domnated solutons). 196
5. If the number of solutons n P exceeds a gven threshold, prune P, usng clusterng. 6. Compute the ftness of each member of P and the strength of each member of P. 7. Perform bnary tournament selecton (wth replacement) to select ndvduals for the matng pool from the multset unon of P and P (P+ P). Tournament selecton creates selecton pressure by holdng a tournament among randomly selected ndvduals from the current populaton (P+ P). The wnner of each tournament (the ndvdual wth the best ftness) s nducted nto the matng pool. The matng pool has hgher average ftness, compared wth the average populaton ftness, and helps buld selecton pressure, whch mproves the average ftness of successve generatons. 8. Apply problem-specfc mutaton and crossover operators, as usual. 9. Go to step 3, and repeat (unless termnaton crtera are reached). Strength of P Solutons Each soluton s assgned a strength S Î[ 01),. S s proportonal to the number of ndvduals jî P, such that domnates j. The ftness of the soluton n an external nondomnated set P s gven by f n = S = N +1, (10-11) where n s the number of ndvduals n P domnated by, and N s the total populaton of P. Ftness of P Solutons The ftness of soluton jî P s calculated by summng the strength of all external nondomnated solutons ÎP ( ) that cover (or domnate) j. The ftness of a soluton n set P s gven by f j å = 1 + S, ÎP, j wth 1 added to the ftness to mantan better ftness of the external nondomnant soluton. Because the ftness s mnmzed, lower ftness results n a hgher lkelhood of beng selected to the matng pool. (10-12) Clusterng In SPEA the sze of the external nondomnated soluton set ( ) P s key to the success of the algorthm. Because of ts partcpaton n the selecton process, an extremely large nondomnated soluton set may reduce selecton pressure and slow down the search. Yet, unbalanced dstrbuton n the populaton may bas the solutons toward certan regons of the search space. Therefore, a prunng process s needed 197
to elmnate ndvduals n the external nondomnated populaton set, whle mantanng ts dversty. Ztzler, Laumanns, and Thele (2001) used the average lnkage method (Morse 1980) to prune the external nondomnated soluton set. The clusterng steps are as follows: 1. Intalze a cluster C, such that each ndvdual ( ÎP) n the external nondomnated soluton set s a member of a dstnct cluster. 2. Calculate the dstance between all possble pars of clusters. Let d m,n be the dstance between two clusters c m and c C ; then, n Î d mn, 1 =. å m -n, c. c m n m, n (10-13) where mîcm, nîcn, m -n s the Eucldan dstance between the objectve space of two ndvduals, and c k s the populaton of cluster c k. 3. Merge two clusters wth mnmum dstance d m,n nto the larger cluster. 4. Identfy the ndvdual n each cluster set wth the mnmum average dstance to all other ndvduals n the cluster. 5. Cycle steps 2 4 untl reachng a threshold of maxmum number of allowed clusters ( C N). SPEA ntroduces eltsm nto evolutonary multobjectve optmzaton. One advantage that stands out s that ths algorthm s not dependent on nche dstance (s radus ), as are MOGA or NSGA. The success of SPEA largely depends on the ftness assgnment methodology, based on the strength of the archve members. In the worst-case scenaro, f the archve contans a sngle member, then every member of P wll have the same rank. The clusterng process also remans the crtcal consderaton for the success of the algorthm. Although essental for mantanng dversty, ths technque may not be able to preserve boundary solutons, whch can lead to nonunform spread of nondomnated solutons. Strength Pareto Evolutonary Algorthm II SPEA-II s an enhanced verson of SPEA. In SPEA-II each ndvdual n both the man populaton and the eltst archve s assgned a strength value (S ) representng the number of solutons t domnates, S = ( j j Î ( P + P )) Ù j. (10-14) On the bass of the strength value S, the raw ftness value R s calculated by summng the strengths of the ndvduals that domnate the exstng one, R = å Sj, (10-15) j where jî ( P+ P), j. Unlke SPEA, n whch ftness s determned only by the cumulatve strength of the domnatng archve members, n SPEA-II, ftness s determned by the cumulatve strength of the domnatng members n both the archve and the populaton. Because the ftness s mnmzed, a hgher ftness value sgnfes that the canddate ndvdual s domnated by a large number of ndvduals. 198
To dstngush ndvduals wth dentcal raw ftness scores, SPEA uses the k nearest neghbors (k-nn) method (Slverman 1986) for estmatng addtonal densty nformaton for each ndvdual. Here, k s calculated as the square root of the combned sample sze of P and P. Each ndvdual measures, stores, and sorts ts dstance n objectve space, relatve to all other ndvduals j n the archve and the populaton. The kth element (dstance) of the sorted lst, n ncreasng order, s represented by s k. Densty D s gven by D 1 = 2 + s, (10-16) k where D 1. Fnally, addng R (raw ftness) and D yelds the ftness of ndvdual, represented by F = R + D. (10-17) Unlke SPEA, SPEA-II mantans a constant number of ndvduals n the archve. After the ftness evaluaton s completed, the next step s to copy all nondomnated ndvduals from archve ( P t ) and populaton (P t ) to the archve of the next generaton ( ), P t+1 { } P = Î t P + + t P Ù t F < 1 ( ) 1. (10-18) If the number of nondomnated solutons s less than the threshold N, then the N- P t+1 best-domnated solutons ( F >1 ) from the sorted lst of the prevous archve ( P t ) and populaton (P t ) are moved to the new archve ( P t+1 ). If, however, the number of nondomnated solutons exceeds the threshold N, then the truncaton process takes place by removng Pt+ 1 - N ndvduals wth mnmum dstance, relatve to each other. In the case of a te, the second-smallest dstances are consdered, and so on. Also unlke SPEA, n whch bnary tournament selecton (wth replacement) selects ndvduals for the matng pool from the multset populaton of P t and P t, SPEA-II selects ndvduals from the archve populaton P t+1 only. Pareto Archved Evolutonary Strategy PAES s a smple multobjectve EA capable of generatng dverse Pareto-optmal solutons. It s a sngleparent, sngle-chld EA that resembles (1+1)-Evolutonary Strategy. PAES uses bnary representaton and btwse mutaton operators to fulfll local search and create offsprng. A btwse mutaton operator flps the bts (genes) of the bnary coded soluton (chromosomes) wth a fxed probablty, thereby creatng a new soluton. A reference archve stores and updates the best nondomnated solutons found n prevous generatons. The best soluton s the one that ether domnates or remans nondomnated n a less crowded regon n the parameter space. Ths archve s used for rankng the domnance of all the resultng solutons. Frst, a chld s created, and ts objectve functons are computed. Next, the chld s compared wth the parent. If the chld domnates the parent, the chld s accepted as a parent for the next generaton, and ts copy s added to the archve. If the parent domnates the chld, the chld s dscarded, and a new mutated soluton s generated from the parent. In the event that the parent and the chld are nondomnatng, wth respect to each other, then both are compared wth the archve of best solutons to make an approprate selecton. If any member of the archve domnates the chld, the chld s dscarded, and a new mutated soluton s generated from the parent. If the chld domnates any member of the archve, the chld s accepted as a parent for the next generaton, and all domnated solutons n the archve are elmnated. If the chld does not domnate any soluton n 199
the reference archve, then the chld s checked for ts proxmty to the solutons n the archve. The chld s accepted as a parent n the next generaton f t resdes n a less crowded regon n the parameter space. A copy of the chld s also added to the archve. The PAES algorthm conssts of the followng steps: 1. Intalze a parent, evaluate ts objectve functon, and add t to the archve. 2. Mutate the parent, generate a chld, and evaluate ts objectve functon. 3. Compare the parent and chld. a. If the parent domnates the chld, dscard the chld, and go to step 2. b. If the chld domnates the parent, accept the chld as a parent for the next generaton, and add t to the archve. 4. Compare the chld wth members n the archve. a. If any member of the archve domnates the chld, dscard the chld, and go to step 2. b. If the chld domnates any member of the archve, accept the chld as a parent for the next generaton, add t to the archve, and remove all domnated solutons n the archve. 5. If the chld does not domnate any soluton n the reference archve, then check the chld for proxmty to the solutons n the archve; accept the chld as a parent n next generaton f t resdes n a less crowded regon n the parameter space. Copy the chld to the archve. 6. Go to step 2, and repeat untl a predefned number of generatons s reached. Pareto Envelope-Based Selecton Algorthm PESA s a multobjectve EA that uses features from both SPEA and PAES. The dfference s attrbuted to the part of the algorthm n whch PESA ntegrates selecton and dversty, usng a hypergrd-based crowdng scheme. Lke SPEA, PESA employs a smaller nternal populaton and larger external populaton. Whereas the external populaton archves the exstng Pareto front approxmaton, the nternal populaton comprses new canddates competng for ncluson n the external archve. Smlar to PAES, to mantan dversty, PESA uses the hypergrd dvson of objectve space to measure the scale of crowdng n dstnct regons of the external archve. Lke PAES and SPEA, PESA s soluton replacement scheme (archvng the best nondomnated solutons) for the external archve s based on the crowdng measure; however, unlke PAES (whch uses parent mutaton) and SPEA (whch uses the ftness measure, based on the strength of the domnatng solutons), the selecton scheme n PESA s also based on the crowdng measure. The PESA algorthm uses two populaton sets: P I representng the nternal populaton and P E representng the external populaton (also called archve populaton). The steps of PESA are as follows: 1. Intalze the external populaton (P E ) to an empty set. 2. Intalze the nternal populaton ( P I =f ). 3. Evaluate each ndvdual n the nternal populaton. 200
4. Update the external populaton archve P E. a. Copy the nondomnated soluton (n P I and any member of P E ) of P I nto P E. b. Remove the soluton of P E that s domnated by the newly added nondomnated soluton of P I. c. If the soluton of P I nether domnates nor s domnated by P E, then add the soluton to P E. d. If P E exceeds a threshold, randomly choose a soluton from the most crowded hypergrds to be removed. 5. Check the termnaton crtera. a. IF a termnaton crteron has been reached, STOP; return P E. b. OTHERWISE, 1. Delete the nternal populaton P I =f. 2. Repeat (untl a new P I s generated). a. Select two parents from P E, from the less crowded hypergrd (based on the densty nformaton). b. Create new offsprng, based on crossover and mutaton. 6. Go to to step 3, and repeat. The crowdng methodology n PESA forms a hypergrd that dvdes objectve space nto hyperboxes. Each ndvdual n the external archve s assocated wth a partcular hyperbox n objectve space. An attrbute defned as the squeeze factor represents the total number of other ndvduals that resde n the same hyperbox. The squeeze factor narrows down the choce of solutons from among randomly selected solutons (from the external archve) by pckng the ones wth lower squeeze factors. The squeeze factor drves the search toward an emergng Pareto front by selectng members of the under represented populaton. The squeeze factor s also used to regulate the populaton of the external archve. When the archve populaton P E exceeds a certan threshold, a random ndvdual from the regon wth a maxmum squeeze factor s chosen to be removed. Pareto Envelope-Based Selecton Algorthm II PESA-II s an extenson of PESA that exercses a regon-based selecton approach, n whch the selecton crtera are satsfed usng a hyperbox nstead of random ndvduals n the hyperbox. A sparsely populated hyperbox has a hgher lkelhood of beng selected than a crowded one. Once the cell s selected, ndvduals wth the cell are randomly selected to partcpate n the matng and mutaton processes. Although ths algorthm s computatonally effcent, t requres pror nformaton about the objectve space to tune the grd sze. Eltst Nondomnated Sortng Genetc Algorthm NSGA-II mproves the noneltst nature of NSGA wth a crowded tournament selecton scheme that uses crowdng dstance to facltate selecton. In NSGA-II, once the populaton s ntalzed, ndvduals n the populaton undergo nondomnated sortng and rankng, as n NSGA. To fnd the frst nondomnated front, each ndvdual n the populaton s compared wth every other ndvdual n the populaton to fnd f that ndvdual s domnated. The nondomnated ndvduals n the frst front are removed from the populaton and placed n temporary (level 1) storage. To fnd the next front, the procedure s repeated wth 201
the remander of the populaton. The process contnues untl all the members of the populaton are assgned a front. In the worst-case scenaro, each front contans only one soluton. Each ndvdual n each front s gven a ftness value (or rank), based on the front t belongs to; for nstance, an ndvdual n the nth front s gven a ftness of n. Addtonally, crowdng dstance s measured for each ndvdual. Crowdng dstance represents the measure of an ndvdual s proxmty to ts neghbors, whch drves the populaton toward better dversty. Parents are admtted nto the matng pool, usng bnary tournament selecton, based on rank and crowdng dstance. On completon of the nondomnated sort, a crowdng dstance value s assgned to each ndvdual. If two solutons are compared durng tournament selecton, the wnnng soluton s selected, based on the followng crtera: If the solutons belong to two dfferent ranks, the soluton wth the better rank wns the selecton. If the solutons belong to the same rank, the soluton wth the hgher crowdng dstance (or lesser crowdng regon) wns. Once the matng pool s populated, crossover and mutaton operators are appled to generate the offsprng populaton. To mplement eltsm, the parent and chld populatons are combned, and the nondomnated ndvduals from the combned populaton are propagated to the next generaton. The NSGA-II algorthm s summarzed as follows: Intalzaton 1. Intalze a random populaton P 0 of sze N. 2. Sort and rank the populaton by creatng nondomnaton fronts. 3. Assgn ftness, accordng to the ranks of the populaton. 4. Create offsprng Q 0 of sze N, usng crossover and mutaton operators. Selecton 5. The start of each generaton has a combned populaton of Rt () = Pt ( -1) ÈQt ( -1) sze 2N. 6. Sort and rank the populaton by creatng nondomnaton fronts (F 1 (t), F 2 (t), F 3 (t),,f n (t)). 7. Select fronts F 1 (t) to F n (t) untl the sum of the combned populaton of selected fronts exceeds N. 8. Copy the entre populatons of selected fronts F 1 (t) to F n-1 (t) to the matng pool of the next generaton. 9. Sort the populaton of the last selected front F n (t) n decreasng order, by crowdng dstance. 10. Select the best ndvduals from the last front F n (t) needed to fll the matng pool slot of N. 11. The matng pool now comprses the entre populaton of fronts F 1 to F n-1 and the partal populaton (sorted by crowdng dstance) of front F n to create a parent populaton (matng pool) of populaton N. 12. Use crossover and mutaton operators to create N offsprng. 13. Go to step 5, and repeat. 202
Fgure 10-6. NSGA-II procedure: the nondomnated fronts F 1 (t), F 2 (t), and F 3 (t) are ncluded fully n the matng pool P(t + 1); the crowdng dstance sorted front F 4 (t) s ncluded partally n the matng pool P(t + 1) The crowdng dstance gudes the selecton process toward a unformly spread-out Pareto optmal front. The crowdng dstance of the th soluton D[] s calculated as the sum of ndvdual dstance values correspondng to each objectve m. Each objectve functon s normalzed before calculatng the crowdng dstance. The followng steps summarze the crowdng dstance computaton of all solutons n a nondomnated set I: 1. l = I Number of solutons 2. I = {0} Intalze all solutons to 0 ---------[For all objectves k = 1 to k = m]---------- 3. I = Sort (I,k ) Sort by the kth objectve 4. D[] = D[] l = --------[For = 2 to (l 1) ]----------------------- ( [ ] - [ - ] ) I + 1. k I 1. k 5. D[] = D[] + max mn f - f m max f max I 1. m, I 2. m, I 3. m, I N. m m m = ( [] [ ] [ ] [ ] ) (10-19) mn f mn I 1. m, I 2. m, I 3. m, I N. m m = ( [] [ ] [ ] [ ] ) Step 5 s a recursve operaton, n whch each successve teraton evaluates the crowdng dstance of the sorted solutons, based on the objectve ftness. Step 5 s nvoked for each objectve ftness. Here, I[].k represents the kth objectve functon value of the th ndvdual n the set I. The crowdng dstance s the Eucldan dstance between each ndvdual n the m dmensonal hyperspace. The ndvduals n 203
the boundary are always selected, because they have nfnte dstance assgnment. To ensure eltsm, the offsprng and parent populatons of the current generaton are combned to select the matng pool of the next generaton. The populaton s subsequently sorted by nondomnaton. As llustrated n Fgure 10-6, the new populaton for the matng pool s generated by fllng the populatons of each front F j (low to hgh) untl the populaton sze exceeds a threshold sze of the parent populaton. If by ncludng ndvduals n the front F j the total populaton exceeds N, then ndvduals n the front F j are selected n descendng order of crowdng dstance untl the populaton of sze N s reached. Ths concludes the creaton of the matng pool for the next generaton. In the fgure the nondomnated fronts F 1 (t), F 2 (t), and F 3 (t) are ncluded fully, and the crowdng dstance sorted front F 4 (t) s ncluded partally, n the matng pool P(t + 1). Example: Multobjectve Optmzaton Cloud computng allows us to host workloads wth varable resource requrements and servce-level objectves or performance guarantees. Furthermore, the cloud enables us to share resources effcently, thereby reducng operatonal costs. These shared resources prmarly relate to compute, memory, nput/ output (I/O), and storage. Varablty of resources creates thermal mbalances, over- or underprovsonng, performance loss, and relablty ssues. If these problems reman unchecked, ther cumulatve effect can ncrease the cost of runnng a datacenter as well as degrade workload performance, owng to unplanned provsonng and unantcpated demands. The soluton for effcent datacenter management rests n satsfyng multdmensonal constrants that may be dynamc n nature and mutually conflctng. Envronmental stresses vary from tme to tme and create resource pressures, whch may be ether global or regonal, creatng dynamc constrants that result n revsed goals that need to be acheved. As llustrated n Fgure 10-7, the operatonal constrants n ths example can be classfed as four objectve functons: Fgure 10-7. Multobjectve optmzaton n a datacenter wth four objectve functons, related to power, performance, temperature, and usage 204
1. Reducng thermal stresses (F T ): Thermal stresses occur when one or more devces approach ther throttlng lmt or generate hot or cold spots, relatve to other devces (or clusters of systems). Thermal stresses can be releved by regulatng fan speed, nput arflow, or resource utlzaton. 2. Meetng power targets (F P ): Power targets are set by an external management agent, accordng to far usage and avalablty of maxmum power. System power targets can be regulated by resource utlzaton; fan speed; or hostng workloads that do not exceed power demands and that are nonnosy, relatve to other workloads already runnng on dfferent cores. 3. Meetng performance guarantees (F S ): Performance guarantees are the ftness matrces defned by applcatons to measure servce-level objectves (SLOs). For example, query response tme s a measure that can quantfy the qualty of servce when hosted on a system or cluster of systems. Performance guarantees are delvered va regulated resource utlzaton or by hostng workloads that are nonnosy, relatve to other workloads runnng on dfferent cores. 4. Meetng resource utlzaton targets (F U ): Resource utlzaton targets are enforced to maxmze the server usage n a unt volume of rack space, leadng to a reducton n dle perods. In some cases, resource utlzaton s regulated to delver servce assurance or reduce thermal hot spots. Resource utlzaton enforcement s generally realzed by usng an approprate dstrbuton of workloads n dfferent cores n a manner that ultmately leads to the most effcent resource utlzaton wth the least amount of mutual nose (cache, prefetchng) or contenton. The multobjectve optmzaton problem can be represented as a functon of these four objectves, Fx ( ) mn f F ( x), F ( x), F ( x), F ( x), = ( ) (10-20) T P s U where x represents the parameters for the values selected to satsfy thermal, power, utlzaton, and performance constrants. These parameters can be summarzed as follows: Fan speed ( x 1 ) Central processng unt (CPU) power lmt ( x 2 ) Memory power lmt ( x 3 ) Input arflow ( x 4 ) Workload type ID ( x 5 ) Number of CPU cores ( x 6 ) These parameters x = (x 1, x 2, x 3, x 4, x 5, x 6 ) regulate the operatng states of the resources, whch result n envronmental as well as system-specfc perturbatons that may need to be corrected as part of explorng a true Pareto-optmal front or stable system. 205
Objectve Functons Indvdual objectve functons measure the qualty of solutons. Multobjectve optmzaton methods trade off the performance between varous objectves. Here, a sutable tradeoff between power, thermal, performance, and utlzaton objectves s sought, usng the EAs. Equatons 10-21 10-24 represent the objectve functons for these objectves. Each of the objectves s contngent on the values of parameters (decson vectors) that defne the search space to satsfy the vectors of multple goals through tradeoffs between combnatons of multple objectves, F F T U N 1 d d d d d d = å f ( T, TT ) for T = ft ( x) ; 0 f ( T, TT ) 1 (10-21) N d= 1 F = f ( PP, ) forp= f ( x) ; 0 f ( P, P ) 1 (10-22) P T P T F = f ( QQ, ) forq= f ( x) ; 0 f ( Q, Q ) 1 (10-23) s T Q T N 1 d d d d d d = å f ( U, UT ) foru = fu ( x) ; 0 f ( U, UT ) 1 (10-24) N d= 1 where T d and U d are temperature and utlzaton of devce d, respectvely; T d T and U d T are the respectve temperature and utlzaton thresholds; P s the current power consumpton of the complete system; and Q s the servce-level agreement (SLA), or performance score, of the workload runnng on the system. The soluton x mpacts the process output, represented by the correspondng functons d ( ft ( x), fp( x), fq( x), f d U ( x), ) whch nfluence the output of the objectve functons. (Note that power and performance are system specfc and not devce specfc n ths context.) The soluton x evolves by maneuverng multple dmensons of the decson space and antcpatng an optmal tradeoff between all four objectves. Fornstance, settng a hgher fan speed (x 1 ) wll mprove coolng (F T ) but ncrease power consumpton, thereby degradng F P. Smlarly, the CPU power lmt (x 1 ) may regulate power consumpton but degrade performance (F S ). Therefore, the goal of the EAs s to synthesze a nearoptmal soluton that attempts to fulfll the nherent and often conflctng constrants of all the objectves. Solutons should reflect optmal decsons n the presence of tradeoffs between the four objectves. These decson vectors match certan workloads on specfc systems, such that there s the least amount of conflct between objectves. Addtonal controls regulate the fan speed, CPU and memory power lmts, nput arflow, and allocaton (or deallocaton) of addtonal CPU cores. Fgure 10-8 dsplays the process of selectng the best compute node (from among a large number of nodes) for workload hostng. 206