Performance Study of Mode-Pursuing Sampling Method

Transcription

1 Performance Study of Mode-Pursung Samplng Method X. Duan 1, G.G. Wang *, X. Kang 1, Q. Nu 1, G. Naterer 3, Q. Peng 1 Abstract Snce the publcaton of our recently developed mode-pursng samplng (MPS) method, questons have been ased on ts performance as compared wth tradtonal global optmzaton methods such as genetc algorthm (GA), and when to use MPS as opposed to GA. Ths wor ams to provde an answer to these questons. Smlartes and dstnctons between MPS and GA are presented. Then MPS and GA are compared va testng wth benchmar functons and practcal engneerng desgn problems. These problems can be categorzed from dfferent perspectves such as dmensonalty, contnuous / dscrete varables, or the amount of computatonal tme for evaluatng the objectve functon. It s found that both MPS and GA demonstrate great effectveness n dentfyng the global optmum. In general, MPS needs much less functon evaluatons and teratons than GA, whch maes MPS sutable for expensve functons. But GA s more effcent than MPS for nexpensve functons. In addton, MPS s lmted by the computer memory when the total number of sample ponts reaches a certan extent. Ths wor serves a purpose of postonng the new MPS n the context of drect optmzaton and provdes gudelnes for users of MPS. It s also antcpated that the smlartes n concepts, dstnctons n phlosophy and methodology, and effectveness as drect search methods for both MPS and GA wll nspre the development of new drect optmzaton methods. Keywords: Mode-Pursung Samplng, Genetc Algorthm, Global Optmzaton 1. Introducton Practcal engneerng desgn problems are usually hghly nonlnear and nvolve many contnuous and/or dscrete varables. Often t s dffcult to defne a desgn problem or to express t as a mathematcal model. In addton, the ncreasngly wde use of fnte element analyss (FEA) and computatonal flud dynamcs (CFD) tools brngs new challenges to optmzaton. FEA and CFD smulatons nvolve a large number of smultaneous equatons and therefore are consdered computatonally expensve and also blac-box 1 Dept. of Mechancal and Manufacturng Engneerng, Unversty of Mantoba, Wnnpeg, MB, Canada R3T5V6 School of Engneerng Scence, Smon Fraser Unversty, Ave., Surrey, BC, Canada V3T03A, Tel , Fax , Emal: gary_wang@sfu.ca; *Correspondng author 3 Faculty of Engneerng and Appled Scence, Unversty of Ontaro Insttute of Technology, 000 Smcoe Street North, Oshawa, Ontaro 1

2 functons. The gradents computed from FEA and CFD, whch requre extra computng resources, are often not relable (Hafta et al. 1998). Therefore, t s dffcult to apply tradtonal gradent-based optmzaton methods on these blac-box functons, thereby nhbtng solutons of many practcal desgn problems. Metamodelng based desgn optmzatons (MBDO) have emerged as a promsng soluton for the expensve blac-box problems. Its essence s to use a computatonally smpler model to approxmate the orgnal expensve blac-box model. Ths approxmaton s realzed by samplng n the desgn space and performng model fttng for a chosen metamodel type. The metamodel can then be optmzed. Current research n MBDO focuses on developng better samplng methods, approxmaton models, or the whole global optmzaton strategy (Wang and Shan 007). Recently, Wang et al. (004) developed a new Mode Pursung Samplng based global optmzaton (MPS) method for blac-box functons. Its dscrete varable verson was n (Sharf et al. 008). In several applcatons (Wang et al. 004, Sharf et al. 008, Lao and Wang 008), the MPS was found to be effectve and effcent. Genetc Algorthm (GA), on the other hand, has been wdely used n engneerng for global optmzaton (Goldberg 1989). The development of Genetc Algorthms (GA) was nspred by the prncples of genetcs and evoluton. GA employs the prncpal of survval of the fttest n ts search process to generate and select chromosomes (desgn solutons) composed of genes (desgn varables). Chromosomes that are more adaptve to ther envronment (desgn objectves/constrants) are more lely to be chosen. Over a number of generatons (teratons), desrable trats (desgn characterstcs) wll evolve and reman n the populaton (set of desgn solutons generated at each teraton). GA begns ts search from a randomly generated populaton of desgn. Three operators are often used to propagate the populatons from one generaton to another to search for the optmum soluton, namely, selecton, crossover, and mutaton. GA can be used for problems that are not well-defned, dffcult to model mathematcally, or blac-box. It can also be used when the objectve functon s dscontnuous, hghly nonlnear, stochastc, or t has unrelable or undefned dervatves. The lmtaton of GA s that t usually demands a large number of functon evaluatons. When analyzng the performance of MPS, t s found that there are many smlartes between GA and MPS: (1) the ntal populaton s randomly created n GA and the samplng ponts are randomly generated n MPS; () n GA, the chromosomes are chosen by a stochastc process and the probablty of a chromosome to be selected s determned by ts ftness value, whle n MPS, the ponts are sampled accordng to a probablty determned by ts objectve functon value; (3) both GA and MPS are based on a set of solutons, or populaton, and render themselves well suted for parallel computaton; (4) both methods use no gradent nformaton and thus are called dervatve-free methods, or drect methods.

3 The dstnctons between GA and MPS are also manfold. Frst they are based on dfferent phlosophes. GA has ts root n evolutonary processes, whle MPS s based on dscrmnatve samplng. Second, GA explores the space through operatons of genes; t s ntutvely a bottom-up approach from the perspectve of explorng the entre desgn space. On the other hand, MPS s le a top-down approach as the entre space s explored at every teraton. Specfc smlartes and dstnctons between the two methods are summarzed n Table 1. Table 1 Comparson of features of GA and MPS algorthms Smlartes Mechansm Features Capabltes GA MPS Generate more ponts around current best pont; Uses probablty n searchng/samplng; Statstcally cover the entre desgn space Random process; Dervatve free; Populaton-based search Perform a "global" optmzaton; Supportng parallel computaton Dstnctons For dscrete problem Input parameters Effcency Form of functon values Robustness Dscrete n nature; Easly handle contnuous problems Many parameters are set by the user, e.g. populaton sze, reproducton operators, crossover probablty, and mutaton probablty. The parameters are problem dependent and could be senstve. Normally a large number of functon evaluatons needed. Usually coded form of the functon values rather than the actual values Premature convergence may happen f the operators are not set properly. Needs dfferent treatments for contnuous and dscrete problems Fewer parameters need to be changed and not very senstve. Developed specally for expensve functons to mnmze the number of functon evaluatons Actual functon values For the problems wth a large quantty of local optma, t may be trapped n a local optmum. To better understand the performance and the problems that most amenable to MPS, ths wor studes the performance of MPS n parallel wth GA. The selecton GA s because of the smlartes between the two algorthms and because GA s well-nown and wdely used by practtoners, whch maes the performance study of MPS more relevant to real practce. The man purpose s thus to poston MPS n the context of engneerng global optmzaton n order to provde some gudelnes for MPS users. The comparson between MPS and GA s thus not to determne whch algorthm s the wnner, but rather to shed lghts on characterstcs and performance behavour of MPS n reference to GA. 3

4 . Overvew of MPS MPS entals ts contnuous varable verson, C-MPS (Wang et al. 004), and ts dscrete varable verson D-MPS (Sharf et al. 008). Ths overvew wll focus on ts orgnal contnuous-varable verson, and provde a bref descrpton of the new dscrete-varable verson..1 Contnuous-varable Mode-Pursung Samplng (C-MPS) Method The essence of MPS s the ntegraton of metamodelng and a novel dscrmnatve samplng method, whch generates more sample ponts n the neghbourhood of the functon mode (local optmal) and fewer ponts n other areas as guded by a specal samplng gudance functon. Fundamental to MPS s the Fu and Wang s algorthm (Fu and Wang 00), whch s used to generate a sample of an asymptotc dstrbuton of a gven Probablty Densty Functon (PDF). Gven a d- dmensonal PDF g(x) wth compact support S d ( g) R, Fu and Wang s algorthm (Fu and Wang 00) conssts of three steps. In the frst step, the dscretzaton step, a dscrete space S N (g) s generated consstng of N unformly dstrbuted base ponts n S(g). Usually N s large and should be larger f the dmenson of g(x), d, s hgher. These unform base ponts may be generated usng ether determnstc or stochastc procedures. In the second step, the contourzaton step, the base ponts of S N (g) are grouped nto K contours {E 1, E,, E K } wth equal sze accordng to the relatve heght of the functon g(x). For example, the frst contour E 1 contans the [N/K] ponts havng the hghest functon values among all base ponts, whereas the last contour E K contans the [N/K] ponts havng the lowest functon values. Also n ths step, a dscrete dstrbuton {P 1, P,, P K } over the K contours s constructed, whch s proportonal to the average functon values of the contours. Fnally, a sample s drawn from the set of all base ponts S N (g) accordng to the dscrete dstrbuton {P 1, P,, P K } and the dscrete unform dstrbuton wthn each contour. As has been shown n the reference (Fu and Wang 00), the sample drawn accordng to ther algorthm s ndependent and has an asymptotc dstrbuton g(x). The approxmaton gets better for larger values of N and K. For optmzaton, we wsh to mnmze an n-dmensonal blac-box functon f(x) over a compact set S n ( f ) R. Followng the conventon of engneerng optmzaton, we refer to the mnmum as the functon mode. To smplfy notaton, assume that S(f) = [a, b] n, where 4 < a < b < are nown, and

5 f(x) s postve on S(f) and contnuous n a neghbourhood of the global mnmum. In general, f f(x) s negatve for some x S(f), then we can always add a postve number to f(x), so that t becomes postve on S(f). Note that mnmzng f(x) s equvalent to maxmzng f(x). The MPS algorthm conssts of the followng four steps: Step 1. Generate m ntal ponts x (1), x (),, x (m) that are unformly dstrbuted on S(f) (m s usually small). Step. Use the m functon values f(x (1) ), f(x () ),, f(x (m) ) to ft a lnear splne functon m f ˆ ( ) ( x) = α x x, (1) = 1 ˆ ) ( ) ( such that f ( x ) = f ( x ), = 1,,, m, where stands for the Eucldean norm. Step 3. Defne g( x) = c ˆ( 0 f x), where c 0 s any constant such that c f ˆ( ), for all x n S(f). Snce 0 x g(x) s nonnegatve on S(f), t can be vewed as a PDF, up to a normalzng constant, whose modes are located at those x () s where the functon values are the lowest among {f(x () )}. Then apply the samplng algorthm of Fu and Wang [7] to draw a random sample x (m+1), x (m+),, x (m) from S(f) accordng to g(x). These sample ponts have the tendency to concentrate about the maxmum of g(x), whch corresponds to the mnmum of f ˆ( x ). Step 4. Combne the sample ponts obtaned n Step 3 wth the ntal ponts n Step 1 to form the set x (1), x (),, x (m) and repeat Steps 3 untl a certan stoppng crteron s met. For ease of understandng, the MPS method s llustrated wth the well-nown sx-hump camel-bac (SC) problem (Brann and Hoo 197). The mathematcal expresson of SC s f sc ( x) = 4x1 x1 + x1 + x1x 4x + 4x, ( x1, x ) [, ]. () 10 3 A contour plot of the SC functon s shown n Fgure 1, where the H s represent local optma. H and H 5 are two global optma at ponts (-0.090, 0.713) and (0.090, ), respectvely, wth an equal functon value f mn =

6 H 1 H H 3 H 4 H 5 H 6 Fgure 1 Contour plot of the SC functon. In the frst step of the MPS algorthm, we start wth m = 6 ntal random ponts x (1), x (),, x (6) [, ]. Then ( ) f ˆ x s computed by fttng Eq. 1 to f(x (1) ), f(x () ),, f(x (6) ). Further, the functon g(x) s obtaned by usng the maxmum of {f(x () ), =1,, 6} as c 0. Now Fu and Wang s algorthm s appled to draw a sample as follows. Frst, N = 10 4 unformly dstrbuted base ponts are generated to form S N (g), the dscretzed verson of the sample space [, ]. Note that the base ponts n S N (g) are cheap ponts, n contrast to the orgnal m = 6 expensve ponts used to buld f ˆ( x ). Further, wthout loss of generalty, suppose the ponts n SN (g) are sorted n ascendng order of the values of functon f ˆ( x ). The sequence of correspondng functon values of f ˆ ( x) s plotted n Fgure (a), whereas the functon g(x) s plotted n Fgure (b). 6

7 (a) fˆ Model Ponts (b) (c) g g ~ Model Ponts Pont Contours (d) G Pont Contours (e) Ĝ Pont Contours Fgure A screen shot of raned pont dstrbuton of fˆ, g, g ~, G and G ) for the SC problem. Accordng to Fu and Wang s method (Fu and Wang 00), the ordered 10 4 base ponts are grouped nto K = 10 contours {E 1, E,, E 100 }, wth each havng N/K = 100 ponts. For example, the frst contour E 1 contans the 100 ponts at whch the values of functon f ˆ( x ) are the lowest, whereas the last contour E100 contans the 100 ponts at whch the values of f ˆ( x ) are the hghest. Let g ~ ( ) be the average of g (x) over E, = 1,,, 100. The functon g ~ ( ), = 1,,, 100 s plotted n Fgure (c) and ts cumulatve dstrbuton functon G() s dsplayed n Fgure (d). Fnally, m = 6 contours are drawn wth replacement accordng to dstrbuton {G()} and, f the contour E occurs m > 0 tmes n these draws, then m ponts are randomly drawn from E. All such ponts form the new sample x (m+1), x (m+),, x (m). As one can see from Fgure (d), the contours from E 80 ~E 100 (correspondng to hgh fˆ values) have lower selecton probabltes for further samplng than other contours, snce the G curve s relatvely flat 7

8 n ths area. However, such a probablty for each contour s always larger than zero. On the other hand, t s generally desred to ncrease the probablty of the frst few contours as they correspond to low fˆ values. To better control the samplng process, a speed control factor s ntroduced [3]. Fgure (e) shows { G ) () }, whch s obtaned by applyng the speed control factor to { G () } n Fgure (d). From Fgure (e), one can see that the frst few contours have hgh selecton probabltes for next-step samplng, whle the contours from E 40 ~E 100 have low probabltes. Ths curve shows an aggressve samplng step, as many more new sample ponts are close to the current mnmum of f(x) as compared to the samplng based on Fgure (d). The whole procedure s repeated eght tmes, so that a total of 48 sample ponts are generated. Fgure 3 shows these 48 sample ponts, where the crcles ndcate attractve desgn ponts havng a functon value less than 0.5. Even wth only 48 sample ponts, many attractve ponts have already shown up around H and H 5. It can also be seen that ponts spread out n the desgn space wth a hgh densty around functon mode H (global mnmum). In the mode-pursung samplng step, every pont has a postve probablty of beng drawn, so that the probablty of excludng the global optmum s zero. As the teraton process contnues, more and more sample ponts wll be generated around the mnmum of functon f(x).. Dscrete-varable MPS (D-MPS) D-MPS nherts the metamodelng and dscrmnatve samplng deas from C-MPS. For the dscrete varable space, sample ponts are mapped from a contnuous space to the dscrete varable space. The man dfference between C-MPS and D-MPS les on ther convergence strateges. In C-MPS (Wang et al. 004), a local quadratc metamodel s employed to adaptvely dentfy a sub-area, wth whch a local optmzaton s called to search for the local optmum. For a dscrete varable space, there lacs of a contnuous sub-area to be approxmated. Even f one could buld a contnuous functon n a dscrete space, the local optmum on a contnuous functon mght not be a vald or optmal soluton n the dscrete space. Therefore the local quadratc metamodelng does not apply to dscrete varable optmzaton problems. 8

9 H H 5 Fgure 3 Sample ponts of the SC problem generated by the MPS method, where o ndcates ts functon value less than 0.5; and H and H 5 are the locatons of two global optma. For the dscrete problems, a Double Sphere method s used. Ths method ncludes two areas (or spheres ) of dynamcally changng radus. One sphere controls the exploraton, and the other controls explotaton. Recall that the C-MPS s composed of three man steps,.e., generatng cheap ponts, approxmaton, and dscrmnatve samplng. In D-MPS, these man steps wll be performed on the domans provded by the double-sphere. For the objectve functon f(x) on doman S[f], the double-sphere D1 f strategy dynamcally provdes a doman D S[ ]; D 1 s the doman nsde the smaller hypersphere; and D s the doman between the smaller hyper-sphere and bgger hyper-sphere. The three man steps of D-MPS are performed on both D 1 and D. The dscrmnatve samplng s performed ndependently n the two spheres untl the optmum s found or the maxmum number of teratons s reached. D-MPS thus does not call any exstng local optmzaton routne and there s no local metamodelng. The optmum s found by samplng alone (Sharf et al. 008). 9

10 3. Testng functons and problems Gven the focus on examnng the performance of MPS, n ths wor a real-value GA s employed. The mplementaton from Ref. (Houc et al. 1995) s selected for a number of reasons. Frst, ths mplementaton has commonly-used operators and t demonstrates good performance. Second, t s based on Matlab TM, whch renders a common bass for computatonal cost (CPU tme) comparsons because MPS s also based on Matlab TM. Last, snce there exsts a large number of GA mplementatons and algorthms, ths wor s not ntended to draw any general concluson n regard to GA as a whole. It s ntended to obtan certan qualtatve nsghts nto MPS and provde gudelnes to users of MPS, whch should largely be nsenstve to the choce of specfc GA mplementaton. The selected mplementaton (Houc et al. 1995) forces the desgn varable to only tae values wthn ts upper and lower bounds to ensure the feasblty of solutons. It uses three selecton operators, namely, the normgeom selecton (a ranng selecton based on the normalzed geometrc dstrbuton), the roulette selecton (a tradtonal selecton wth the probablty of survvng equal to the ftness of an ndvdual over the sum of the ftness of all), and the tournament selecton (choosng the wnner of the tournaments as the new populaton). It also has three crossover operators, namely, the smple crossover, the heurstc crossover, and the arthmetc crossover. Fnally, t uses four optonal mutaton operators, called the boundary mutaton, mult-non-unform mutaton, non-unform mutaton, and unform mutaton. Detals of these operators are n the reference (Houc et al. 1995). Two termnaton crtera are used n the GA mplementaton. The frst s based on the maxmum number of generatons, whch s sutable for problems wthout nowng the analytcal optmum a pror. The second termnaton crteron occurs when the analytcal optmum s reached. The GA optmzaton process wll be termnated whenever one of these two crtera s met. For constraned optmzaton problems, the dscard method was used n order to satsfy the constrants. Three constrant checs are performed on the ntal populaton, wth new chldren generated from crossover, as well as from mutaton operatons at each teraton. Any chromosome that does not pass the constrant chec wll be dscarded from the populaton. For dscrete varable problems, f all of the varables are ntegers, a smple roundng functon s used to round a number to ts closest nteger. If the dscrete varable values are pced from a set, for example, x [16., 17.1, 18.5, 19.3], we frst ndex the possble values, e.g., ndex = {1,,3,4}, and then generate a random number and round t to the closest ndex. For example, f ndex=3, then we use x=18.5. For mxed-varable problems, only the dscrete varables are handled as descrbed above. 10

11 Nne problems are tested for a performance comparson of the chosen GA mplementaton and MPS optmzaton algorthms. The characterstcs of these problems are summarzed n Table. In Table, only optmzaton problems wth constrants other than bounds are referred as constraned problems. Table Characterstcs of test functons and problems. Functon (n: number of varables) Sx-hump camel-bac (SC) (n=) Corana (CO) functon (n=) Hartman (HN) functon (n=6) Gear tran (GT) problem (n=4) A hgh dmensonal functon (F16) (n=16) Rosenbroc functon (R10) (n=10) Insulaton layer (IL) desgn (n=) Pressure vessel (PV) desgn (n=4) Fxture and jonng poston desgn (FJP) (n=4) Characterstcs Multple local optma Flat bottom, multple local optma Multple local optma Dscrete, multple local optma Hgh dmenson Flat bottom, relatvely hgh dmenson, multple local optma Constraned Engneerng Problem Constraned Engneerng Problem Constraned, expensve, blac-box functon As one can see from Table, the test functons and problems present dfferent challenges to the optmzaton algorthms. Ths secton brefly descrbes each test problem. Sx-hump camel-bac (SC) problem The SC problem s a well-nown benchmar test problem. It appears early n Ref. (Brann and Hoo 197) and t was used by many other researchers. The mathematcal expresson s shown n Eq. (). The SC functon has sx local optma. A contour plot of the SC functon s shown n Fgure 1. The Corana (CO) functon The Corana functon s a well nown benchmar test functon (Corana et al. 1987). It was also tested n Ref. (Humphrey and Wlson 000). The Corana functon s a parabolod wth axes parallel to the coordnate drecton, except a set of open, dsjont, rectangular flat pocets. Let the doman of the functon f (x) n an n-dmensonal space be: D f + = n n { x R : a x a ; a R, 1,,... n} Let D represent the set of pocets n D : m f 11

12 1 } 1,,..., ;, ; ; : {,..., 1 n s t R s t Ζ t s x t s D x d n f n = < + < < = + 0,0,...,0. 1,...,, 1 d d D n m n = U L The Corana functon can then be defned as: = + 0 ),...,,(,,, ) ( 1,..., 1 1 n r n n m f d x z d c R d D D x x d x f n (3) where > = < + = 0, 0, 0 0, t s t s z The parameters n ths test are set to n=4, 1) ( 0.15, 0.05, 0., 100, = = = = = r e d c t s a. A less extreme weght d s used as compared to the standard representaton, 1) ( 10. The searchng space s x [-100, 100]. The four-dmensonal Corana functon has ts global mnmum f*=0 at x*=(0, 0, 0, 0). A dffculty arses from the fact that the optmum s located n a deep valley, and several deep flat bottoms exst along the space. Fgure 4 shows a -D Corana functon n the range of [-0.5, 0.5]. Fgure 4 -D corana functon n [-0.5, 0.5]

13 Hartman (HN) functon The Hartman functon wth n=6 was tested n Ref. [3]. It can be expressed as f HN whereα j and p j are lsted below. 4 n ( x) = c exp α j ( x j pj ), x [ 0,1], = 1,..., n (4) = 1 j= 1 α j, j=1,...,6 c p j, j=1,..., Compound gear tran (GT) desgn problem Ths s a dscrete varant problem nvolvng a compound gear tran desgn (Fu et al. 1991, Ca and Therauf 1996), as shown n Fgure 5. It s desred to produce a gear rato as close as possble to 1/ For each gear, the number of teeth must be an nteger between 14 and 60. The nteger desgn varables are the numbers of teeth, x=[t d, T b, T a, T f ] T =[x 1, x, x 3, x 4 ] T. The optmzaton problem s formulated as: Mnmze = 1 x x ( 1 X ) x3x4 GT (5) Subject to 14 x 60, = 1,,3, 4 13

14 A functon of 16 varables (F16) Fgure 5 Compound gear tran Ths hgh dmensonal problem was descrbed n Ref. (Wang et al. 004). It can be expressed by F16 ( x) = aj ( x + x + 1)( x j + x j + 1) = 1 j= 1 f,, j =1,,, 16, (6) Ten-dmensonal Rosenbroc functon (R10) The Rosenbroc functon s a wdely used benchmar problem for testng optmzaton algorthms such as Refs. (Bouvry et al. 000, Auguglaro et al. 00). It can be expressed as n 1 = 1 f ( x) = (100( x (7) RO The Rosenbroc functon has ts global mnmum + 1 x ) + ( x 1) ) f mn 14 =0 at x*= (1,1,1). Whle attemptng to fnd a global mnmum, a dffculty arses from the fact that the optmum s located n a deep parabolc valley wth a flat bottom. In the present study, the 10-dmensonal Rosenbroc functon wll be used as another hgh dmensonal benchmar problem, n addton to F16. The searchng range for each varable s [-5, 5]. Two layer nsulaton desgn (IL) problem Ths s a problem nvolvng the desgn of a two-layer nsulated steam ppe lne, as shown n Fgure 6. The objectve s to desgn r 1 and r to mnmze the total cost per unt length of ppe ($/m) n one year of

15 operaton. The total cost functon ncludes the cost of heat loss, the cost of nsulaton materals, and the cost of fxed charges ncludng nterest and deprecaton (wth a rate of 15%). The parameters n the desgn are (1) Inner radus r 0 = 73mm, () Steam temperature T f = 400 deg. C, and the temperature of the ambent ar T a =5 deg. C, (3) The conductve resstance through the wall s neglected, (4) The convectve heat transfer rate at the nner surface has h f =55W/m K, (5) The frst layer of nsulaton s roc wool, wth a thermal conductvty 1 =0.06W/mK and thcness δ 1 = r 1 -r 0, (6) The second nsulaton layer s calcum slcate wth =0.051W/mK and thcness of δ =r -r 1, (7) The outsde convectve heat transfer coeffcent s h 0 =10W/m K, (8) The cost of heat s C h =0.0$/Wh; cost of roc wool, C 1 =146.7$/m 3 ; cost of calcum slcate C =336$/m 3. The constrants are determned by the nsulaton requrement and other practcal consderatons, whch are descrbed n Ref. (Za and Al-Tur 000). Fgure 6 A steam ppe wth two-layer nsulaton The total cost per unt length of ppe can be expressed as: Mnmze Subject to: f ( r, r 0.001( T f Ta ) IL 1 ) = τ Ch + fπ [ C ( r1 r0 ) + C ( r r1 )] 1 (8) 1 ln( r1 / r0 ) ln( r / r1 ) πr h π π πr h 0 f 1 o 38mm<δ<5r 0 ; r 1 >r 0 ; r >r 1 ; T a ( T f Ta ) 1 + <60 (9) 1 ln( r1 / r0 ) ln( r / r1 ) πr ho πr h π π πr h 0 f 1 o The characterstc of ths problem s ts complexty n both the objectve functon and the last constrant. The optmum soluton s f*= 5.536$/m, at r 1 =0.311m and r =0.349m. Pressure vessel desgn (PV) problem 15

16 The desgn of a pressure vessel was used as a test problem n Ref. (Wang et al. 004) and t s shown n Fgure 7. There are four desgn varables: radus, R, and length, L, of the cylndrcal shell, shell thcness, T s, and sphercal head thcness, T h, all of whch are n nches. They have the followng ranges of nterest: 5 R 150, 1.0 T s 1.375, 5 L 40, and 0.65 T h 1.0. Fgure 7 Pressure vessel (adapted from Ref. (Wang et al. 004)) The desgn objectve s to mnmze the total system cost, whch s a combnaton of weldng, materal and formng costs. The optmzaton model s then expressed as: Mnmze f ( R, T, T, L) PV s h =.64T s RL T R T L +.84T s R h s 19 (10) 4 3 Subject to: Ts ; Th R 0 ; π R L + πr 1.96E6 0 (11) The optmum contnuous soluton s f * = , occurrng at R*= n., T s = 1.0 n., L*=84.579n., and T h *=0.65n. Fxture and jonng postons (FJP) optmzaton problem Smultaneous optmzaton of fxture and jonng postons for a non-rgd sheet metal assembly was studed by Lao and Wang (008). Ths s a blac-box functon problem nvolvng fnte element analyss, whch maes t computatonally expensve. The optmzaton problem can be descrbed as follows. In the presence of part varaton and fxture varaton, as well as the constrans from the assembly process and desgned functon requrements, fnd the best locatons of fxtures and jonng ponts so that the non-rgd sheet metal assembly can acheve the mnmal assembly varaton. An assembly of two dentcal flat sheet metal components by lap jonts shown n Fgure 8 s optmzed. Assumng that these two components are manufactured under the same condtons, ther fabrcaton varatons are expected to be the same. The sze of each flat sheet metal part s mm, wth Young s modulus E =.6e+9 N/mm, and Poson s rato ν = 0.3. The fnte element computatonal model of the assembly s created n ANSYS TM. The 3 16

17 element type s SHELL63. The number of elements and the number of nodes are 150 and 135, respectvely. Fgure 8 An assembly of two sheet metal parts (note: The symbol p ndcates the fxture locaton and s ndcates the jont postons) The mathematcal optmzaton model for ths specfc example can be wrtten as follows, U 1 + z U Mnmze Θ ( x) = abs( ( x)) abs( ( x)) (1) Subject to (x 1 -x 3 ) + (x -x 4 ) 100; 0 x 1, x, x 3, x 4 80; 10 x 5 40 (13) where U s the deformaton of crtcal ponts n an assembly. It s obtaned by modelng the assembly deformaton usng the fnte element analyss through ANSYS TM. In ths study, ANSYS TM and Matlab TM are ntegrated to mplement the optmzaton on a FEA process for both MPS and GA. 4. Results and dscussons 4.1 Performance crtera Two man performance crtera are used to evaluate the two algorthms, namely effectveness and effcency. The effectveness ncludes the robustness of the algorthm and the accuracy of the dentfed global optmum. For robustness, 10 ndependent runs are carred out for each problem and each algorthm, except for the expensve FJP problem, where only 5 runs are performed. The range of varaton and medan value of the optma are recorded and compared aganst the analytcal or nown optmum. If the nown soluton of a problem s not zero, the accuracy of an algorthm s quantfed by z soluton nown soluton Q sol = 1 (14) nown soluton 17

18 If the nown soluton s zero, the devaton of the soluton from zero s examned. When there s no analytcal or nown optmum, the optma found by GA and MPS are compared by the values. The second crteron for comparson s effcency. In ths study, the effcency of an algorthm n solvng a problem s evaluated by the number of teratons, nt, and number of functon evaluatons, nfe, as well as the CPU tme requred to solve the problem. Agan, the average (arthmetc mean) and medan values of these values n 10 (or 5) runs are used. 4. Effects of tunng parameters As dscussed prevously, there are several optmzaton operators to be set n the GA program. In testng the same problems, these factors were found to have sgnfcant effects on the effcency. Fgure 9 shows the varaton of GA performance under dfferent crossover rates, Pc, when solvng the SC functon (wth a populaton sze of 100 and a mutaton rate of Pm=0.01). A general trend can be observed n Fgure 9, whereby ncreasng Pc mples that the number of functon evaluatons, nfe, ncreases and the number of teratons, nt, decreases, whle the computaton tme remans at a smlar level. Wth a small Pc, the GA needs more generatons to establsh the optmum, therefore more teratons are needed. When Pc s set to be very large, nfe ncreases sharply because almost all parents are replaced and few good solutons are nherted. Performance evaluaton nfe nt 10 tme Crossover Rate Fgure 9 Performance evaluaton wth dfferent crossover rates n solvng the SC problem. 18

19 Smlarly, the effect of mutaton rate, Pm, on GA s performance n solvng the SC problem (wth a populaton sze of 100 and crossover rate of Pc=0.6) s studed. It s observed that the mnmum of nfe occurs at Pm=0.1 and nfe ncreases wth Pm>0.1. In the MPS program, there s a tuneable parameter (dfference coeffcent, ), whch was also found to have notceable effect on performance of the MPS. The dfference coeffcent,, (denoted as cd n Ref. [3]) s used for a convergence chec n a quadratc model detecton. A smaller dfference coeffcent maes the MPS have a more accurate soluton, but t may cause more computatonal expense, as shown n Table 3. By examnng the MPS performance n several problems n ths study, t s found that =0.01 s a good choce for most problems. However, for the dscrete GT problem and the R10 problem, a larger coeffcent ( > 0.5) must be used to establsh an acceptable soluton. SC problem Table 3 Effcency of MPS wth dfferent values on the SC and PV problems Coeffcent, Average CPU tme (s) average of nt Average of nfe PV problem Coeffcent, Average CPU tme (s) Average of nt Average of nfe Both the random characterstc and the effects of ther tunng parameters necesstate the method of usng results of multple runs n ths comparson study. Specfcally for the comparson n ths wor, we used crossover rates of 0.6 or 0.7, mutaton rates of 0.05 or 0.1 for GA. For MPS, we used the dfference coeffcent of 0.01 for most problems but a coeffcent larger than 0.5 for the GT problem and the R10 19

20 problem. For both MPS and GA, dfferent combnatons of parameters for a problem are frst tested; and the set of parameters that yelds the best soluton s then used for comparson. Hence for each test problem, the best results of GA are compared wth the best results of MPS. 4.3 Effectveness comparson of GA and MPS For those problems wth nown optma of non-zero (SC, HN, F16, IL, PV), the qualtes of the solutons (usng the medan soluton) are compared based on Eq. (14). The results are shown n Fgure 10. It can be seen that for all these fve problems, both GA and MPS methods can fnd solutons of hgh qualty close to 100%. Only a lower qualty of 98.7% was obtaned by GA on the HN problem. 100 Qualty of solutons % SC HN F16 IL PV Functons/problems GA MPS Fgure 10 Mean qualty ratngs of solutons obtaned by GA and MPS for 5 testng problems. The two algorthms, however, show great dfferences n solvng the other problems. For the CO problem (analytcal optmum of zero), GA fnds very good solutons rangng from 0 to 0.003, whle MPS only fnds solutons from 34.5 to 5, For the ten-dmensonal R10 functon (analytcal optmum of zero), GA also outperforms MPS. The solutons obtaned by GA range from 0 to 0.013, whle those obtaned by MPS range from 73.9 to It was found n solvng the R10 functon MPS reaches the computer memory lmt when the number of functon evaluatons reaches about,500. Ths s because all of the 500 ponts partcpate n the constructon of the metamodel defned by Eq. (1). GA, on the other hand, can afford around 3,000,000 functon evaluatons. In a later comparson, we let both GA and MPS run for,500 ponts and found that the results are smlar wth those of the MPS, as shown n Table 4. 0

21 Table 4 Comparson of effectveness of GA and MPS for R10 and CO problems (*: solutons of GA for 500 nfe s) Mnmum Functon N Space GA MPS Range of varaton Medan Range of varaton Medan CO 4 [-100,100] [ 0.000, 0.003] 0 [34.56, ] R10 10 [-5,5] [0, 0.013] [63.641,07.665]* * [73.96, ] A partcular nterest n ths study s to compare the effectveness of GA and MPS n solvng dscrete problems. In ths study, a well nown gear tran (GT) problem s solved and the solutons are compared. Table 5(a) shows the solutons of the GT problem wth dfferent optmzaton algorthms. The MPS outperforms most of the other approaches, ncludng GA. Table 5(b) shows the solutons wth 10 runs of both GA and MPS. The MPS outperforms GA for the dscrete problem. Approach Table 5 Solutons of the gear tran desgn problem (a) Best results Optmzaton soluton GT mn PFA [11] 14,9,47, ES [1] 18,15,3, GA 0,0,47, MPS 16, 19, 43, (b) Solutons of 10 runs Algorthm Solutons Range of varaton Medan Average GA [ , ] MPS [ , ] e

22 Another partcular problem, the FJP problem, also deserves more explanaton. It s an expensve blacbox functon problem whch nvolves fnte element analyss (FEA). When testng ths problem wth GA, the maxmum number of generatons s set as 60; populaton sze s 100 wth the normgeomselect parameter, as well as the 3 crossover (Pc=0.7) and 4 mutaton methods (Pm=0.1) as descrbed n Secton. For MPS, =0.01. The results are shown n Table 5. Both GA and MPS gve good solutons n all fve ndependent runs. But MPS provdes a wder range of varaton of the solutons and t gves a better optmum of Table 6 Comparson of optmal solutons of the FJP problem by GA and MPS (a) Wth 5 runs Algorthm Solutons Range of varaton Medan Average GA [0.177, ] MPS [0.1515, ] (b) Best results Algorthm Optmzaton soluton Θ mn MPS P1= (75.153,31.709), P= (68.117, ), S= GA P1= (63.944, ), P= (0.043, 7.741), S= Effcency comparson of GA and MPS The effcences are compared n Fgure 11 n terms of the number of functon evaluatons, the number of teratons, and CPU tmes used to solve the problems. The number of functon evaluatons and teratons needed by GA are dramatcally larger than those needed by MPS for most problems, except the HN problem. In Fgure 11 (a), note that the nfe of GA s scaled by 0.1 for better llustraton. It can also be found that GA needs more functon evaluatons and teratons for hgh dmensonal problems and complex desgn problems, such as the F16, PV and IL problems. The lower nfe of MPS occurs because of

23 ts dscrmnatve samplng and metamodelng formulaton n the algorthms. Only the expensve ponts are calculated usng the objectve functon, whle the cheap ponts are calculated usng an approxmate lnear splne functon or quadratc functon. Snce only several expensve ponts are generated n MPS, nfe s very low. However, for GA, each ndvdual n the populaton s calculated usng the objectve functon, so nfe s very hgh. As for the CPU tme used for solvng the problems, MPS method uses more CPU tme than GA n all of the fve problems, especally the HN problem, where MPS spends 100 tmes more CPU tme than the GA, as shown n Fgure 11 (c). The reason s that MPS needs to generate ten thousand ponts per teraton, and calculate the functon value of each pont from the metamodel, thereby leadng to a large computng load and hgh CPU tme. But ths has not been counted as a functon evaluaton snce the fundamental assumpton of MPS s that the objectve functon evaluaton s at least one magntude more expensve than evaluaton wth a metamodel n MPS. For the other functons / problems, the computatonal expenses by GA and MPS are lsted n Table 7. Generally GA performs better than MPS on the hgh dmensonal problems wth nexpensve functons. The effcency advantage of MPS shows n the expensve functon problem, the FJP problem, n whch MPS uses only half of the computatonal efforts of GA and fnds better solutons. 3

24 No.of Functon Evaluatons MPS,nef GA,nef/10 No.of Iteratons MPS GA 0 SC HN F16 IL PV Functons/problems 0 SC HN F16 IL PV Functons/problems (a) (b) MPS GA CPU tme (s) SC HN F16 IL PV Functons/problems (c) Fgure 11 Computatonal expenses of MPS and GA for the soluton of fve testng problems (a) numbers of functon evaluatons (b) numbers of teratons and (c) CPU tmes 4

25 Table 7 Effcency of the GA and MPS algorthms n some optmzaton problems Functon or Problem Number of functon evaluatons, nfe Number of teratons, nt CPU tme(s) GA MPS GA MPS GA MPS Average Medan Average Medan Average Medan Average Medan Average Medan Average Medan SC HN F IL PV CO GT R FJP

26 5. Summary and Remars Ths paper studes the performance of mode-pursung samplng (MPS) method, n reference to wdely used Genetc Algorthm (GA). Based on qualtatve analyss of MPS and GA and quanttatve comparsons on test problems, the followng observatons are made: 1. MPS can robustly and accurately dentfy the global optmum for a majorty of test problems, ncludng both contnuous and dscrete varable problems. It meets ts lmtaton, however, when the number of functon evaluatons requred for convergence s larger than a certan value that exceeds the computer memory. From ths regard, MPS s best suted for low dmensonal problems and hgh dmensonal problems wth smple functons. For hgh-dmensonal complex or expensve functons, large-memory computers are needed or a better memory-management strategy needs to be developed for MPS.. MPS s recommended for global optmzaton of expensve functons. MPS s developed for expensve functons and t therefore does not bear advantages over GA for global optmzaton on nexpensve functons. 3. The dfference coeffcent,, s a senstve parameter for MPS. It s recommended to set =0.01 f the user has no a pror nowledge of the optmzaton problem. 4. Common features of MPS and GA, such as the group-based (or populaton-based) samplng and selectve generaton of new samples may be found n other recognzed global optmzaton methods, e.g. smulated annealng, ant colony optmzaton, partcle swamp optmzaton, etc. The unque phlosophy behnd MPS, namely, the top-down exploraton and dscrmnatve samplng, may nspre the development of future algorthms. Future research on MPS s to enhance ts capablty for hgh dmensonal problems. One possble method s to employ a more economcal metamodelng method to avod usng all of the evaluated ponts n model fttng whle stll provdng an overall gude for dscrmnatve samplng. References Auguglaro, A., Dusonchet, L., and Sanseverno, E.R., 00. An Evolutonary Parallel Tabu Search Approach for Dstrbuton Systems Renforcement Plannng. Advanced Engneerng Informatcs, 16, Bouvry, P., Arbab, F., and Seredyns, F., 000. Dstrbuted Evolutonary Optmzaton n Manfold: the Rosenbroc's Functon Case Study. Informaton Scences, 1, Brann, F.H. and Hoo, S.K., 197. A method for fndng multple extrema of a functon of n varables. In: Lootsma, F., edtor. Numercal methods for non-lnear optmzaton. New Yor: Academc Press,

27 Ca, J., and Therauf, G., Evoluton Strateges for Solvng Dscrete Optmzaton Problems. Advances n Engneerng Software, 5, Corana, A., Marches, M., Martn, C., and Rdella, S., Mnmzng Multmodal Functons of Contnuous Varables wth the "Smulated Annealng" Algorthm. ACM Transactons on Mathematcal Software, 13, Fu, J., Fenton, R.G., and Cleghorn, W.L., A Mxed Integer Dscrete-contnuous Programmng Method and Its Applcaton to Engneerng Desgn Optmzaton. Engneerng Optmzaton, 17, Fu, J.C., and Wang, L., 00. A random-dscretzaton based Monte Carlo samplng method and ts applcatons. Methodology and Computng n Appled Probablty, 4, 5-5. Goldberg, G., 1989, Genetc Algorthms n Search, Optmzaton and Machne Learnng. Readng, MA: Addson-Wesley. Hafta, R.T., Scott, E.P., Cruz, J.R., Optmzaton and Experments: A Survey. Appled Mechancs Revew, 51(7), Houc, C., Jones, J., and Kay, M., A Genetc Algorthm for Functon Optmzaton: A Matlab Implementaton, Techncal Report NCSU-IE-TR-95-09, North Carolna State Unversty, Ralegh, NC. Humphrey, D.G., and Wlson, J.R., 000. A Revsed Smplex Search Procedure for Stochastc Smulaton Response Surface Optmzaton. INFORMS Journal on Computng, 1(4), Lao, X., and Wang, G.G., 008. Smultaneous Optmzaton of Fxture and Jont Postons for Non-rgd Sheet Metal Assembly. Internatonal Journal of Advanced Manufacturng Technology, 36, Sharf, B., Wang, G.G., and ElMeawy, T., 008. Mode Pursung Samplng Method for Dscrete Varable Optmzaton on Expensve Blac-Box Functons. Transactons of ASME, Journal of Mechancal Desgn, 130(), Wang, G.G., and Shan, S., 007. Revew of Metamodelng Technques n Support of Engneerng Desgn Optmzaton. Transactons of ASME, Journal of Mechancal Desgn, 19(4), Wang, L., Shan, S., and Wang, G.G., 004. Mode-Pursung Samplng Method for Global Optmzaton on Expensve Blac-box Functons. Journal of Engneerng Optmzaton, 36(4), Za, G.M., and Al-Tur, A.M., 000. Optmzaton of Multlayer Thermal Insulaton for Ppelnes. Heat Transfer Engneerng, 1,