A Self-adaptive Similarity-based Fitness Approximation for Evolutionary Optimization

A Self-adaptve Smlarty-based Ftness Approxmaton for Evolutonary Optmzaton Je Tan Dvson of Industral and System Engneerng, Tayuan Unversty of Scence and Technology, Tayuan, 34 Chna College of Informaton Technology, Shandong Womens Unversty, Jnan, 53 Chna tanje3@outlook.com Chaol Sun Department of Computer Scence and Technology, Tayuan Unversty of Scence and Technology, Tayuan, 34 Chna State Key Laboratory of Synthetcal Automaton for Process Industres, Northeastern Unversty, Shenyang, 4 chaol.sun.cn@gmal.com Yaochu Jn Department of Computer Scence, Unversty of Surrey, Guldford, GU 7XH, UK Department of Computer Scence and Technology, Tayuan Unversty of Scence and Technology, Tayuan, 34 Chna yaochu.jn@surrey.ac.uk Yn Tan Dvson of Industral and System Engneerng, Tayuan Unversty of Scence and Technology, Tayuan, 34, Chna tanyng965@gmal.com Janchao Zeng School of Computer Scence and Control Engneerng, North Unversty of Chna, Tayuan, 34 Chna zengjanchao@63.net Abstract Evolutonary algorthms used to solve complex optmzaton problems usually need to perform a large number of ftness functon evaluatons, whch often requres huge computatonal overhead. Ths paper proposes a self-adaptve smlarty-based surrogate model as a ftness nhertance strategy to reduce computatonally expensve ftness evaluatons. Gaussan smlarty measurement, whch consders the ruggedness of the landscape, s proposed to adaptvely regulate the smlarty n order to mprove the accuracy of the nhertance ftness values. Emprcal results on three tradtonal benchmark problems wth 5,,, and 3 decson varables and on the CEC 3 test functons wth 3 decson varables demonstrate the hgh effcency and effectveness of the proposed algorthm n that t can obtan better or compettve solutons compared to the state-of-the-art algorthms under a lmted computatonal budget. Keywords Ftness nhertance; Ftness estmaton; smlarty; Computatonally expensve optmzaton; Partcle swarm optmzaton I. INTRODUCTION Evolutonary algorthms (EAs) have much strength for optmzaton, for example, no requrement on dfferentablty of the objectve functons, ease mplementaton, and better global search capablty, compared to tradtonal sngle-pont optmzaton algorthms. For these reasons, EAs have been appled to many real-world applcatons and have emerged as well-establshed and powerful optmzaton tools, especally n the feld of engneerng desgn. However, many real-world engneerng optmzaton problems nvolve complex Ths work was supported n part by Natonal Natural Scence Foundaton of Chna (Grant No. 64769, 6437 and 6437), and the State Key Laboratory of Synthetcal Automaton for Process Industres, Northeastern Unversty, Chna. computatonal smulatons, whch may take a few mnutes, hours or even days to perform one smulaton [-4], such as computatonal flud dynamcs [5], fnte element method computatons [6], and aerodynamc shape optmzaton [7], just to name a few. As EAs usually need a large number of ftness evaluatons to obtan an acceptable soluton, t s essental to reduce the number of ftness evaluatons due to the constrant of lmted computatonal resources. To tackle computatonally expensve optmzaton problems, use of computatonally cheap ftness approxmaton to replace the real expensve ftness evaluatons s a common approach [8-]. The most popular method of ftness approxmaton s surrogate models (also called meta-models) [-4]. Commonly used surrogate models nclude the polynomal method [5], the krgng method (Gaussan process) [6], artfcal neural networks (ANN) [7], radal bass functons (RBF) [, 8], and support vector machnes (SVM) [9]. A comprehensve descrpton of these methods can be found n [8-]. Among varous ftness approxmaton methods, ftness nhertance technques are a generc approach that does not rely on the use of surrogate models. One clear beneft of ftness nhertance s that ts computatonal expense s much less than that of tranng a surrogate model [9]. In ths paper, we propose to use a relatvely smple ftness approxmaton method to determne whch ndvdual wll be evaluated usng the real objectve functon, and whch one wll be relably estmated usng ftness nhertance. The dea of ftness nhertance was orgnally proposed by Smth []. It s an ntutve dea assumng that the ftness of an ndvdual can be derved from ts parents. Two types of ftness nhertance strateges have been proposed, one s known as the averaged nhertance and the other weghted nhertance. Salam and Hendtlass [] ntroduced a fast evolutonary algorthm, n

whch a ftness value and an assocated relablty value were assgned to each new ndvdual, and the ndvdual s evaluated usng the real ftness functon only when ts relablty value s below a threshold. As mentoned n [], n many real-world applcatons, the ftness of offsprng cannot be relably estmated from the ftness of ther parents, as they are not necessarly smlar to ther parents. Instead, a newly generated offsprng ndvdual may be smlar to those n ts vcnty. For example n a partcle swarm optmzaton (PSO) algorthm, Cu [3] proposed a ftness nhertance model based on the smlarty or credblty to an ndvdual s prevous poston. In [4], a ftness estmaton strategy was proposed for partcle swarm optmzaton, called FESPSO, based on the postonal relatonshp between two ndvduals at same teraton. To further reduce the number of ftness evaluatons for FESPSO, Sun et al [5] ntroduced a smlarty n the ftness estmaton strategy method later. Km [6] used a clusterng technque to group smlar ndvduals and the ndvduals that are closest to cluster center wll be evaluated usng the real objectve functon, whle the ftness of the rest ndvduals wll be nherted from those wth ther ftness beng evaluated usng the real objectve functon. Reyes-Serra and Coello [7] suggested a ftness nhertance model that also used the average method accordng to the Eucldean dstances between ndvduals n multobjectve partcle swarm optmzaton, whch was shown to have reduced the tme of optmzaton and mproved the performance n solvng the classcal test functons. Gomde [8] utlzed a fuzzy clusterng method wth fuzzy adaptve clusterng of c-means. In each cluster, the ftness of the most representatve ndvdual was calculated usng the real ftness functon and the ftness of the other ndvduals s estmated usng a ftness nhertance method weghted by the Eucldean dstance and the membershp degree. Fonseca et al. [9] ntroduced a smlarty-based surrogate model based upon the k-nearest neghbors method (KNN). The smlarty proposed by Fonseca s also based on the Eucldean dstance among ndvduals. Subsequently, Fonseca et al [3] appled three dfferent types of adaptve value nhertance models, ncludng the averaged nhertance model, weghted nhertance model, and parent nhertance model, n a real-coded genetc algorthm for comparsons. Jn and Sendhoff [3] also used the k-means method appled to group the ndvduals of a populaton nto a number of clusters. For each cluster, only the ndvdual whch closest to the cluster center s evaluated and the ftness of other ndvduals are estmated usng a neural network ensemble. The man ssue that needs to be addressed when employng the ftness nhertance method s the accuracy. In most ftness nhertance strateges, the ftness of an ndvdual s estmated based on that of other ndvduals accordng to the postonal relatonshp of these two ndvduals n the decson space usng the Eucldean dstance, and the smaller Eucldean dstance between two ndvduals postons s, the hgher the degree of smlarty between them. However, ftness landscape analyss proved that the dstance metrcs wll become less meanngful when the dmensonalty of the decson space sgnfcantly ncreases [3]. For example, the ftness dstance correlaton (FDC) [33] ndcated that there s no smple relatonshp between the ftness values of two ndvduals and the dstance between two ndvduals. Dfferent from the above-mentoned approaches, Davarynejad [34-36] proposed a method of adaptve fuzzy ftness granulaton usng the ftness of the ndvduals n the pool for ftness estmaton. When the center of the granule has a hgh ftness, the radus of the granules wll be small, and the new ndvdual s ftness s more lkely to be computed n ths granule. On the other hand, more ndvduals wll be estmated usng ftness nhertance n the granule when ts center has a poor ftness. Nonetheless, for fuzzy granule rules, an ndvdual may fnd tself n more than one granule, and granules wth Gaussan measures of smlarty was proposed to adjust ts the wdth so that the new ndvdual wll nhert the ftness from a granule wth a poor ftness yet far from ths ndvdual, whle not nhert the ftness from a granule wth good ftness but near the new ndvdual[37]. Therefore, the above-mentoned approaches cannot guarantee the accuracy n ftness estmaton. In order to obtan more accurate estmaton usng a ftness nhertance method, the mpact of the ftness landscape on the smlarty between two ndvduals must be consdered. In ths paper, an adaptve smlarty-based surrogate model s proposed to reduce the computatonal cost for tmeconsumng optmzaton problems, whch takes nto account the smlarty on both smooth ftness landscapes (wth small ftness changes n the neghborhood) and rough ftness landscapes (wth hgh ftness varatons n the neghborhood). The degree of smlarty between ndvduals wll be decreased when the local ftness landscape s beleved to be rugged. By utlzng an adaptve smlarty-based surrogate models n order to adjust the frequency of usng ftness estmaton, we am to acheve a balance between the frequency and the accuracy of ftness estmaton, thereby ensurng the estmaton accuracy whle reducng the requred ftness evaluatons usng the real ftness functon. The remander of ths paper s organzed as follows. Secton II gves a detaled descrpton of the proposed selfadaptve smlarty-based surrogate model and the surrogateasssted PSO. Expermental results on two groups of test problems are presented and dscussed n Secton III. Secton IV gves a summary of ths paper and some dscussons for future work. II. A SELF-ADAPTIVE SIMILARITY-BASED SURROGATE MODEL As prevously dscussed, the man ssue wth utlzng a ftness nhertance method s to enhance the accuracy of ftness estmaton. In the followng, we frst provde an llustratve example showng why most exstng ftness nhertance methods usng the Eucldean dstance as a measure for smlarty may fal. After that, we ntroduce a new ftness nhertance method. Fg. gves an llustratve example to show the problem n ftness estmaton when a smlarty degree s utlzed n a onedmensonal Mchalewcz functon. In ths example, there are three solutons x, x and x 3, n whch the ftness value of x and x 3 are known and evaluated usng the real ftness functon. x s a new canddate soluton whose ftness s to

be estmated. The smlarty between x and x s s, and the smlarty between x 3 and x s s 3. Gven a smlarty threshold s, x wll be estmated f and only f s or s 3 s greater than s. Accordng to the Eucldean dstance, a smaller dstance between two ponts means a hgher degree of smlarty. If s < s < s 3, the ftness of x (y ) can be nherted from that of x 3. However, as we can see from Fg., the ftness landscape s smoother between x and x than between x 3 and x. Therefore, n order to relably estmate the ftness of x, a new smlarty metrc s needed, whch takes the ruggedness of the local landscape n addton to the Eucldean dstance, so that the ftness of x wll be used for estmatng the ftness of x, then the ftness estmaton error wll be smaller than the ftness of x 3 used for estmatng the ftness of x. y. -. -.4 -.6 -.8 X:.5 Y: -4.868e-3 x x Mchalewcz,D= X:.5 Y: -.7 x 3 X: Y: -.37-3 x Fg.. An example of three solutons n a one-dmensonal Mchalewcz functon To address the above ssue n ftness estmaton, ths paper proposes a Gaussan smlarty measure that s able to adjust the degree of smlarty accordng to the local ruggedness of the ftness landscape. It s assumed that the smlarty and the local ruggedness are negatvely correlated. A. Self-adaptve smlarty-based ftness approxmaton strategy Wthout loss of generalty, we consder the followng mnmzaton problem: Mnf(x) Subject to : x l x x u () where s the feasble soluton set, n denotes the dmensonalty of the search space. Below are the man steps of our proposed method for PSO algorthms. Step :Intalze a populaton, perform ftness evaluaton usng the real objectve functon for all ndvduals. Step :Determne the personal best poston and global best poston of PSO. If the ftness of the personal best poston or global best poston s estmated, then re-evaluate ther ftness usng real objectve functon. Step 3: Update the velocty and poston of the swarm. For each ndvdual (=,,,n), set a neghborhood sze of x, and select k neghborng ndvduals. If k =, evaluate the ftness of x usng the real objectve functon, and go to Step 6; otherwse go to Step 4. Step 4:Calculate the Gaussan smlarty ¹ r between x and ts neghbor x r. Step 5:Estmate the ftness value of x usng the ftness nhertance strategy. Step 6:Repeat steps -5 f the stop condton s not satsfed, otherwse stop and ext. B. Gaussan Smlarty Gaussan smlarty, whch uses a Gaussan functon, s a smlarty matrx that commonly be used as a clusterng calculaton method for smlarty models, s(; r) = exp( d (x ;x r ) ¾ ) () In (), d (x ; x r ) s the Eucldean dstance between two samples and r, and ¾ s the metrc parameter. The most mportant ssue n the Gaussan smlarty s the selecton of metrcs. Dfferent metrc parameters wll lead to dfferent clusterng results. In ths paper, we ncorporate the roughness of the local ftness landscape to help adjust the metrc of the Gaussan smlarty. The smlarty wll be self-adaptvely calculated accordng to the changes n the local roughness, whch wll reduce the estmaton errors n ftness nhertance. In the followng, we wll elaborate the proposed self-adaptve smlarty-based surrogate model n detal. C. Defnng a measure for local roughness ± It s concevable that the local roughness where ndvdual s located wll nfluence the accuracy of ftness estmaton much more than the global roughness of the ftness landscape. In ths paper, the local roughness wll be consdered when the ftness of ndvdual s to be estmated. Suppose the radus of neghborhood of ndvdual s R, the neghborng U(x ; R ) s defned by [x R ; x + R ]. Apparently, the sze of radus R wll drectly affects the number of ndvduals n the neghborhood. If the neghborhood s too small, there mght not be enough neghborng ndvduals around ndvdual, and the Gaussan smlarty cannot be constructed. If the neghborhood s approprately set, there wll be suffcent neghbors and the ftness of ndvdual wll be nherted from ts neghbors. At the earler generatons, the global search ablty s pad more attenton to the algorthm, so a relatvely large radus R s adopted. Then, R wll be decreased to mprove the accuracy of ftness estmaton at later generatons. In ths paper, an adaptve radus s proposed accordng to the number of the ndvduals dstrbuted n the neghborhood of ndvdual. The upper and lower bounds of the neghborhood on j-th dmenson of ndvdual are defned as cx jmax and cx j mn, that s: cx jmax = max (fx j ; = ; ; ; :::; ng ) (3) cx j mn = mn (fx j ; = ; ; ; :::; ng ) (4) where n s the populaton sze. Correspondngly, the value of R on j-th dmenson s set as: R j = jcx j max cx j mn j (5)

In(5), s called the convergence factor. Apparently, R = fr j ; j = ; ; : : : ; lg s dfferent on each dmenson, l s the number of varables. Suppose that there are ndvduals whose ftness s evaluated usng the real objectve functon, we save ther postons and correspondng ftness nto an archve X E = fx ev ; f(x ev ); ev = ;:::; mg, whch wll enable ndvduals to fnd ther neghbors n ther neghborng space U(x ; R ). The neghbors of each ndvdual wll be also saved n an archve X r = fx r ; f(x r ); r = ; :::; kg. x r and f(x r ) represent the poston and ftness of r-th neghbor of ndvdual, respectvely. Eq.(6) defnes the method to calculate the local roughness ± around ndvdual. Obvously, f the ftness varaton among the neghbors s large, the local roughness ± wll be relatvely large, and vce versa. max (f (x r)) mn (f (x r)) rf;;;:::;kg rf;;;:::;kg ± = max (f (x ev)) mn (f (x ev)) (6) evf;;;:::;mg evf;;;:::;mg We can easly fnd that ± [; ]. D. Defnng metrc parameter ¾ The metrc parameter ¾ plays an mportant role n the Gaussan smlarty. We frstly consder the value of the proper metrc parameter ¾ on one-dmensonal problems when the ftness of an ndvdual wll be estmated. The smlarty ¹ r between the nterested ndvdual and ts neghborng ndvduals r s defned accordng to (). ¹ r = e (xr x ) ¾ ; r = ; :::; k (7) The next step s to select those neghbors wth ther smlarty ¹ r hgher than the threshold µ. All neghbors, whch are called estmaton neghbors, wll be used to estmate the ftness of x usng the nhertance method. The values of both parameters, ¾ and µ, are key factors n selectng estmaton neghbors. We dvde ndvduals around ndvdual nto three parts. Accordng to the 3σ prncple of Gaussan functon and the smlarty defned n (7), the poston of x s the center of the neghborhood, and ndvduals n the range [x ¾;x +¾] are called core neghbors, ndvduals located n the range [x ¾; x + ¾] are called outskrt neghbors, and those located n the range [x 3¾; x + 3¾] are called weak outskrt neghbors. When the core neghbors are used to estmate the ftness of ndvdual, the threshold for the smlarty level s set to µ = e ¼ :6. Then the value of ¾ wll drectly affects the range of the core neghbors [x ¾;x +¾]. Therefore, the value of ¾ value wll correspondngly affects the smlarty among ndvduals. In ths paper, a local roughness of the ftness functon ± s proposed to be ncorporated n the settng of metrc parameters, whch ams to adjust the sze of the neghborhood the core neghbors are located. The sze of ths space wll be shrunken as the local roughness ncreases. Obvously, f the roughness s low, the core neghbors wll be selected from a wde range of ndvduals around ndvdual, as a result, the ftness of ndvdual wll be more lkely to be estmated; otherwse, the ftness of ndvdual wll be more lkely to be evaluated usng the real objectve functon. Eq. (8) gves the defnton of ¾ proposed n ths paper: ¾ = ± R (8) where s the convergence factor, and ± represents the local roughness of the ftness functon. Because ± [; ], we can deduce that when the varaton on ftness of neghbors s small n the neghborng space, t s possble for ± to be near zero wth ¾ approachng the nfnty. Therefore, we gve an upper bound of ¾, ¾ R, to set ¾ n a lmted range. Then we can get ¾ = ± R R whch we can derve that ± ; as a result, when ±, we have ± =,and ± [ ; ], and subsequently, ¾ [ R ; R ]. When the ndvduals are n mult-dmensonal space, ¾ and R are n dmensonal vectors wth the same number of dmensons as x, and n the jth dmenson: ¾ j = ± R j (9) The smlarty ¹ r j between ndvdual and ts neghbor r on j-th dmenson s gven n the followng: ¹ r j = e ( xrj x ¾ j j) ; r = ; :::; k () The smlarty ¹ r between x and x r, then, wll be the average of the smlarty on all dmenson: P lj= ¹ r ¹ r j = ; r = ; :::; k () l If the core neghbors are used for estmatng the ftness of x n a weghed manner, the smlarty threshold should be µ = e ¼ :6. We suppose that ndvduals wth a hgher ftness value are commonly surrounded by more ndvduals, so when the average ftness value of the neghborng ndvduals s relatvely hgh, the ndvdual of nterest s more lkely to have a hgher ftness value. Especally at the end of evoluton, the populaton converges gradually, and the ndvdual whose ftness s to be estmated s often located n an area of hgh ftness. As a result, the number k s relatvely hgh n the core neghbors archve X kr = fx kr ; f(x kr ); r = ; :::; k g. Therefore, the ftness of the neghbors should be also consdered when selectng neghbors for ftness estmaton. In ths paper, the threshold of smlarty µ s constructed based on the averaged ftness value of the core neghbors, denoted as mean(f (x kr )), and the threshold µ s adapted accordng to the value of mean(f (x r )): P k kr= mean(f(x kr )) = f(x kr) k ; kr = ; :::; k () mean (f (x kr )) mn (f (x ev)) evf;;;:::;ng µ = :9 :3 max (f (x ev)) mn (f (x ev)) evf;;;:::;ng evf;;;:::;ng (3)

From (3) we can see that µ [:6; :9]. By neghborng clusters from the core ndvduals, those neghbors used for ftness estmaton are selected based on ther merts accordng to mean(f (x kr )). When the average ftness mean(f (x kr )) of the ndvduals s weak n the neghborhood, the ftness of the neghbors of x s undesrable overall, and µ wll be reduced accordngly, makng the nterested ndvdual more lkely to be estmated. On the other hand, when the average ftness mean(f (x kr )) of the ndvduals s large n the neghborhood, the value of µ wll be ncreased, thus ncreasng the lkelhood for the ndvdual s ftness to be evaluated usng the real objectve functon. E. Ftness nhertance strategy The ndvduals n the set X r that satsfy ¹ r > µ consttute the set SN = fsn ; :::sn t g. The smlarty between and ndvduals from the set SN s f¹ ; :::¹ t g. Suppose that there are potentally more good ndvduals around the current best ndvdual, there should be the hstorcally best ndvduals found prevously n the set SN, the new ndvdual s also consdered to potentally locate at the best poston. Thus, the ftness of ndvdual s obtaned wth the followng procedure: If the hstorcally best ndvduals are n the set SN, the ftness of ndvdual s evaluated usng the real objectve functon. If t T, P tl= (¹ l ) f(sn l ) f (x ) = P tl= (¹ l ) ; t T (4) If t > T,the smlartes between ndvduals and current ndvdual are ranked accordng to a decreasng order of ther ftness, and T ndvduals were selected to assgn f (x ) value n a weghted manner. T s the maxmum number of neghborng ndvduals used for ftness estmaton. P Tl= (¹ l ) f(sn l ) f (x ) = P Tl= (¹ l ) ; t > T (5) III. EXPERIMENTAL STUDIES In order to examne the performance of the proposed selfadaptve smlarty-based ftness approxmaton for partcle swarm optmzaton, called, for computatonally expensve problems, the algorthm s tested on two sets of wdely used benchmark problems. In addton, the effcency and effectveness of s also demonstrated by comparng to a few exstng ftness nhertance strateges proposed for PSO. A. Tradtonal Optmzaton Benchmark Functons To nvestgate the effcency of our proposed approach, three tradtonal optmzaton benchmark functons suggested n [35] and 8 CEC functons are adopted. Four sets of dmensons are utlzed for three tradtonal optmzaton benchmark functons:d = 5; ; and3. The characterstcs of the three benchmark functons (Grewank, Rastrgn and Ackley), referrng to [35] for more detals. These three benchmark functons are scalable and are commonly used to assess the performance of optmzaton algorthms. They have some ntrgung features whch most optmzaton algorthms fnd hard to deal wth. All the compared algorthms are run for ndependent tmes on each test problem n Matlab R4a. The code for the tradtonal PSO s provded by [38]. In ths paper, The PSO and the k-nearest neghbors method for PSO, called, are adopted to compare on the performance wth under the same fxed ftness evaluatons. In, we select k as the number of the closest neghbors to the ndvduals to be estmated for performng the calculaton of ftness nhertance. The parameters for the method are set as follows: k s the number of the closest neghbors n radus R, whch s the same as the radus set for. From (5) and (6) wth = :5, ftness nhertance s performed for the ndvdual of nterest when À k, but only the closest neghbors to x are selected for estmatng the ftness of ndvdual usng the nhertance strategy when k. The parameters of and are set as follows n the experment: The sze of the swarm s, the cogntve and socal parameters are both set to.5, the velocty v has the same range as the poston x on each dmenson, and the maxmum number of real ftness evaluatons s set to ME = D, whch s the same as those used n [35]. Fgs. -4 shows the results obtaned by PSO,, and wth the same number of ftness evaluatons. The algorthm stops when the ftness evaluatons reach the maxmum value of ftness evaluatons usng the real objectve functon. A number of experments were conducted and showed that f the value of n (5) s set too small, the sze of the neghborhood wll be small too, resultng n an nsuffcent number of the neghborng ndvduals. If the value of n (9) s set too small, the number of the core ndvduals wll be small too, whch wll then greatly reduce the number of ftness estmatons. However, f the value of s too large, there wll be an excessve number of neghborng ndvduals nvolved n ftness evaluatons, whch wll ncrease the tme of ftness approxmaton. In ths paper, ¾ [ R ; R ] was derved when defnng ¾, f the value of s set too large, the range of ¾ s varaton wll be much small, and t wll not be able to properly reflect the mpact of the local roughness on the selecton of neghborng ndvduals used for ftness estmaton. In addton, f the range for selectng core neghbors s too large, t wll lower the accuracy of the ftness estmaton. The values of the parameters n (5) and n (9) are both set by trals and errors. Therefore, n ths experment, = :5, and = :3. Fonseca [9] chose the dfferent largest neghborng number of ndvduals T = ;;5;;5 to verfy the results. The results obtaned when T = ;5 are superor to those obtaned when T = ; ; 5. To mprove the accuracy of ftness nhertance and the computatonal effcency, the maxmum number of neghborng ndvduals nvolved n ftness estmaton T s set to be.

Mean (Log) Mean (Log) Mean (Log) Mean (Log) Mean (Log) Mean (Log) Mean (Log) Mean (Log) Mean (Log) Mean (Log) Mean (Log) Mean (Log) 5 5 5 (a) Ackley, (5-D) SS-based FAPSO (c) Ackley, (-D) SS-based FAPSO 5 5 5 5 (b) Ackley, (-D) SS-based FAPSO (d) Ackley, (3-D) SS-based FAPSO Fg.., and SS-based FAPSO Box plot of the best ftness n the fnal generaton for Ackley functon 8 6 4 5 5 (a) Grewank, (5-D) SS-based FAPSO (c) Grewank, (-D) SS-based FAPSO 4 5 5 (b) Grewank, (-D) SS-based FAPSO (d) Grewank, (3-D) SS-based FAPSO Fg. 3., and SS-based FAPSO Boxplots of the best ftness n the fnal generaton for Rastrgn functon 3 8 6 4 (a) Rastrgn, (5-D) SS-based FAPSO (c) Rastrgn, (-D) SS-based FAPSO 8 6 4 8 6 4 (b) Rastrgn, (-D) SS-based FAPSO (d) Rastrgn, (3-D) SS-based FAPSO Fg. 4., and SS-based FAPSO Box plot of the best ftness n the fnal generaton for Grewank functon To further demonstrate the performance of ftness nhertance proposed n our method, Fgs. -4 gve the boxplots of the best ftness n the fnal generaton of, and SS-based FAPSO on the three functons. Fg. 5 plots the convergence profle of PSO,, and on benchmark problems (Grewank, Rastrgn and Ackley). As we can see from Fgs. -4, the proposed method has fewer outlers compared to the other two methods n ndependent runs, ndcatng that t performs more robust than the other two algorthms. It can be see that our proposed method obtaned best results nearly n all nstances n terms of the best, the worst and the mean results of the optmal results averaged over ten ndependently runs on three functons of dfferent dmensons compared to PSO and. From Fg. 5, we can see that the proposed SS- Based FAPSO method converges much faster than the other two methods. Although, ntally, the method showed slghtly worse or comparable performance on converge (e.g., n the Rastrgn functon n 3 dmensons), t shows much better convergence capablty as the number of ftness evaluatons ncreases (.e. n the later stages)..4..8.6.4. Ackley, (5-D) 3 4 5 Ackley, (-D).35.3.5..5. 5 5 Grewank, (5-D).5.5 -.5 3 4 5 3.5 Grewank, (-D).5 5 5 Rastrgn, (5-D).8.6.4..8 3 4 5 Rastrgn, (-D).5.4.3.. 5 5.4.3...9.8.7 Ackley, (-D) 4 6 8 Ackley3, (3-D).36.34.3.3.8.6.4 5 5 5 3 Grewank, (-D).8.6.4..5.5.5 4 6 8 Grewank 3-D 3 5 5 5 3 Rastrgn, (-D)..8.6.4 4 6 8 Rastrgn3, (3-D).7.65.6.55.5.45 5 5 5 3 Fg. 5. Convergence profles of the, and on the three benchmark problems B. CEC 3 Functon Tests TABLE I. THE DETAILS OF FUNCTIONS ON WHICH FESPSO AND SS- BASED FAPSO HAVE YIELDED BEST PERFORMANCE. Functon number Detal F Sphere Functon(f) F Rotated Bent Cgar Func Functon(f3) F3 Rotated Rosenbrock s Functon(f6) F4 Rotated Schaffers F7 Functon F5 Composton Functon (n=5,rotated)(f) F6 Composton Functon 8 (n=5,rotated) To further compare the performance of these dfferent methods, the ftness estmaton strategy for partcle swarm

optmzaton proposed by Sun et al. (3), called FESPSO, was adopted for comparson. Table II presents the expermental results on CEC 3 testng functons of a dmenson of 3 obtaned by PSO,, and FESPSO, wth a maxmum of, real ftness evaluatons. Table I lsts the detals of the test functons, where we can see that F to F are unmodal functons, F3 to F4 are mult-modal functons, and F5 to F6 are complex combnatons of unmodal and mult-modal functons. TABLE II. COMPARATIVE RESULTS ON THE 8 CEC 3 TEST FUNCTIONS Func Approach Best_ft Worst_ft Ft_mean std_ft opt PSO 7.344E+ 4.34E+3.467E+3.63E+3 F.668E+3 3.959E+3.7568E+3 7.466E+ -.4 FESPSO 6.776E+.454E+4 8.5867E+3 7.434E+3 E+3 -.373E+3 -.468E+3 -.947E+3 6.734E+ F F3 F4 F5 F6 PSO.36E+ 3.354E+.75E+ 6.5599E+9.686E+ 4.76E+.54E+ 9.343E+9 FESPSO 4.3694E+.76E+4.89E+3 6.8765E+3.36E+9.747E+ 7.7893E+9 4.67E+9 PSO -6.964E+ -4.39E+ -5.8E+ 5.48E+ -6.443E+ -3.863E+ -5.436E+ 8.3366E+ FESPSO -7.8E+ 7.39E+.796E+ 4.635E+ -8.3849E+ -7.49E+ -7.9E+ 3.8836E+ PSO -6.9753E+ -5.7398E+ -6.6374E+ 3.538E+ -7.87E+ -6.98E+ -6.67E+.7688E+ FESPSO -5.43E+ 7.854E+4 6.89E+3.57E+4-7.658E+ -5.578E+ -6.55E+ 6.3E+ PSO.5673E+3.877E+3.8976E+3.48E+.546E+3.675E+3.67E+3 4.45E+ FESPSO.9E+3.53E+3.73E+3.766E+ 9.994E+.9E+3.98E+3 6.656E+ PSO.9499E+3 3.869E+3 3.393E+3.746E+ 3.84E+3 4.38E+3 3.5554E+3 3.63E+ FESPSO 4.73E+3 5.46E+3 4.7597E+3 4.54E+.84E+3 3.845E+3.548E+3 4.56E+ -. E+3-9. E+ - 8. E+ 7. E+.4 E+3 As the results shown n Table II ndcate, method has obtaned better results than the compared algorthms on both unmodal and mult-modal test functons. shows sgnfcantly better overall performance on F. In F4 and F6, the standard devaton obtaned by was slghtly nferor to the compared methods. IV. CONCLUSION AND FUTURE WORK Ths paper proposed a new self-adaptve smlarty-based surrogate model for ftness estmaton n PSO to solvng computatonally expensve problems. In ths model, the radus of the neghborhood to be selected for estmatng the ftness of the ndvduals s self-adaptve. The proposed method requres a lower number of ftness calculatons usng the real ftness functon whle a hgh accuracy n ftness estmatons. Our future work ncludes the development of a more accurate method n evaluatng local roughness of the ftness landscape. Furthermore, the adaptve smlarty-based surrogate model proposed n ths study can be seen as a local surrogate model. Hence, our future work wll consder the stuatons where the condton for ftness nhertance s not satsfed. In these cases, we could combne global surrogate models wth the local estmaton method proposed n ths work to provde more relable ftness estmatons. V. APPENDIX In the canoncal PSO, when searchng n a D-dmensonal hyperspace, each partcle has a velocty V = [v ;v ; :::; v D ] and a poston X = [x ;x ;:::;x D ]. The vectors V and X are ntalzed randomly and then updated by (6) and (7) through the gudance of ts personal best poston P, and the global best poston P n : V (t+) =!V (t) X (t+) + c r (P (t) = X (t) X (t) ) + c r (P (t) n X (t) +V (t+) ) (6) (7) where X (t) and V (t) are the poston and velocty of partcle at generaton t respectvely., The Coeffcents! s called the nerta weght, c and c are acceleraton parameters whch are commonly set to.. r and r are two dagonal matrx whose dagonal elements are random numbers unformly generated wthn the range of [, ]. ACKNOWLEDGMENT Ths work was supported n part by Natonal Natural Scence Foundaton of Chna (Grant No. 64769, 6437 and 6437), and the State Key Laboratory of Synthetcal Automaton for Process Industres, Northeastern Unversty, Chna. REFERENCES [] D. Lm, Y. Jn, Y.-S. Ong, and B. Sendhoff, "Generalzng surrogateasssted evolutonary computaton, Evolutonary Computaton, IEEE Transactons on, vol. 4,, pp. 39-355. [] D. Lm, Y.-S. Ong, Y. Jn, and B. Sendhoff, "Trusted evolutonary algorthm, n Evolutonary Computaton, 6. CEC 6. IEEE Congress on, 6, pp. 49-56. [3] Y. S. Ong, P. Nar, A. Keane, and K. Wong, Surrogate-asssted evolutonary optmzaton frameworks for hgh-fdelty engneerng desgn problems, n Knowledge Incorporaton n Evolutonary Computaton, ed: Sprnger, 5, pp. 37-33. [4] Y. S. Ong, P. B. Nar, and A. J. Keane, Evolutonary optmzaton of computatonally expensve problems va surrogate modelng, AIAA journal, vol. 4, 3, pp. 687-696. [5] M. N. Le, Y. S. Ong, S. Menzel, Y. Jn, and B. Sendhoff, Evoluton by adaptng surrogates, Evolutonary computaton, vol., 3, pp. 33-34. [6] K. S. Won and T. Ray, A framework for desgn optmzaton usng surrogates, Engneerng optmzaton, vol. 37, 5, pp. 685-73. [7] M. K. Karakass, A. P. Gots, and K. C. Gannakoglou, Inexact nformaton aded, low cost, dstrbuted genetc algorthms for aerodynamc shape optmzaton, Internatonal Journal for Numercal Methods n Fluds, vol. 43, 3,pp. 49-66. [8] Y. Jn, A comprehensve survey of ftness approxmaton n evolutonary computaton, Soft computng, vol. 9, 5, pp. 3-. [9] L. Sh and K. Rasheed, A survey of ftness approxmaton methods appled n evolutonary algorthms, n Computatonal ntellgence n expensve optmzaton problems, ed: Sprnger,, pp. 3-8. [] Y. Jn, Surrogate-asssted evolutonary computaton: Recent advances and future challenges, Swarm and Evolutonary Computaton, vol.,, pp. 6-7. [] E. Sanchez, S. Pntos, and N. V. Quepo, Toward an optmal ensemble of kernel-based approxmatons wth engneerng applcatons, Structural and multdscplnary optmzaton, vol. 36, 8, pp. 47-6. [] D. Lm, Y.-S. Ong, Y. Jn, and B. Sendhoff, A study on metamodelng technques, ensembles, and mult-surrogates n evolutonary computaton, n Proceedngs of the 9th annual conference on Genetc and evolutonary computaton, 7, pp. 88-95.

[3] A. Samad, K.-Y. Km, T. Goel, R. T. Haftka, and W. Shyy, Multple surrogate modelng for axal compressor blade shape optmzaton, Journal of Propulson and Power, vol. 4, 8, pp. 3-3. [4] C. Sun, Y. Jn, J. Zeng, and Y. Yu, A two-layer surrogate-asssted partcle swarm optmzaton algorthm, Soft Computng, vol. 9, 5, pp. 46-475. [5] I. Paenke, J. Branke, and Y. Jn, Effcent search for robust solutons by means of evolutonary algorthms and ftness approxmaton, Evolutonary Computaton, IEEE Transactons on, vol., 6, pp. 45-4. [6] B. Lu, Q. Zhang, and G. G. Gelen, A Gaussan process surrogate model asssted evolutonary algorthm for medum scale expensve optmzaton problems, Evolutonary Computaton, IEEE Transactons on, vol. 8, 4,pp. 8-9. [7] S. Ferrar and R. F. Stengel, Smooth functon approxmaton usng neural networks, Neural Networks, IEEE Transactons on, vol. 6, 5,pp. 4-38. [8] Y. Jn, M. Olhofer, and B. Sendhoff, A framework for evolutonary optmzaton wth approxmate ftness functons, Evolutonary Computaton, IEEE Transactons on, vol. 6,,pp. 48-494. [9] X. Lu, K. Tang, and X. Yao, Classfcaton-asssted dfferental evoluton for computatonally expensve problems, n Evolutonary Computaton (CEC), IEEE Congress on,, pp. 986-993. [] R. E. Smth, B. A. Dke, and S. Stegmann, Ftness nhertance n genetc algorthms, n Proceedngs of the 995 ACM symposum on Appled computng, 995, pp. 345-35. [] M. Salam and T. Hendtlass, A fast evaluaton strategy for evolutonary algorthms, Appled Soft Computng, vol., 3,pp. 56-73. [] E. Ducheyne, B. De Baets, and R. De Wulf, Is ftness nhertance useful for real-world applcatons?, n Evolutonary Mult-Crteron Optmzaton, 3, pp. 3-4. [3] Z. Cu, J. Zeng, and G. Sun, A fast partcle swarm optmzaton, Internatonal Journal of Innovatve Computng, Informaton and Control, vol., 6,pp. 365-38. [4] Z. J. Sun C, Pan J, et al., A new ftness estmaton strategy for partcle swarm optmzaton, Informaton Scences,vol., 3,pp.355-37. [5] C. Sun, J. Zeng, J. Pan, and Y. Jn, Smlarty-based evoluton control for ftness estmaton n partcle swarm optmzaton, n Computatonal Intellgence n Dynamc and Uncertan Envronments (CIDUE), 3 IEEE Symposum on, 3, pp. -8. [6] H.-S. Km and S.-B. Cho, An effcent genetc algorthm wth less ftness evaluaton by clusterng, n Evolutonary Computaton,. Proceedngs of the Congress on,, pp. 887-894. [7] M. Reyes-Serra and C. A. C. Coello, A study of ftness nhertance and approxmaton technques for mult-objectve partcle swarm optmzaton, n Evolutonary Computaton, 5. The 5 IEEE Congress on, 5, pp. 65-7. [8] F. Gomde, Fuzzy clusterng n ftness estmaton models for genetc algorthms and applcatons, n Fuzzy Systems, 6 IEEE Internatonal Conference on, 6, pp. 388-395. [9] L. Fonseca, H. Barbosa, and A. Lemonge, A smlarty-based surrogate model for enhanced performance n genetc algorthms, Opsearch, vol. 46, 9,pp. 89-7. [3] L. G. Fonseca, A. C. Lemonge, and H. J. Barbosa, A study on ftness nhertance for enhanced effcency n real-coded genetc algorthms, n Evolutonary Computaton (CEC), IEEE Congress on,, pp. - 8. [3] Y. Jn and B. Sendhoff, Reducng Ftness Evaluatons Usng Clusterng Technques and Neural Network Ensembles, n Genetc and Evolutonary Computaton GECCO 4: Genetc and Evolutonary Computaton Conference, Seattle, WA, USA, June 6-3, 4. Proceedngs, Part I, K. Deb, Ed., ed Berln, Hedelberg: Sprnger Berln Hedelberg, 4, pp. 688-699. [3] Y. Sun, S. K. Halgamuge, M. Krley, and M. A. Munoz, On the selecton of ftness landscape analyss metrcs for contnuous optmzaton problems, n Informaton and Automaton for Sustanablty (ICIAfS), 4 7th Internatonal Conference on, 4, pp. -6. [33] T. Jones and S. Forrest, Ftness Dstance Correlaton as a Measure of Problem Dffculty for Genetc Algorthms, n ICGA, 995, pp. 84-9. [34] M.-R. Akbarzadeh-T, M. Davarynejad, and N. Parz, Adaptve fuzzy ftness granulaton for evolutonary optmzaton, Internatonal Journal of Approxmate Reasonng, vol. 49, 8, pp. 53-538. [35] M. Davarynejad, C. Ahn, J. Vrancken, J. van den Berg, and C. C. Coello, Evolutonary hdden nformaton detecton by granulaton-based ftness approxmaton, Appled Soft Computng, vol.,,pp. 79-79. [36] M. Davarynejad, J. Vrancken, J. van den Berg, and C. A. C. Coello, A ftness granulaton approach for large-scale structural desgn optmzaton, n Varants of Evolutonary Algorthms for Real-World Applcatons, ed: Sprnger,, pp. 45-8. [37] I. Cruz-Vega, M. Garca-Lmon, and H. J. Escalante, Adaptvesurrogate based on a neuro-fuzzy network and granular computng, n Proceedngs of the 4 conference on Genetc and evolutonary computaton, 4, pp. 76-768. [38] J. Lang, B. Qu, P. Suganthan, and A. G. Hernández-Díaz, Problem defntons and evaluaton crtera for the CEC 3 specal sesson on real-parameter optmzaton," Computatonal Intellgence Laboratory, Zhengzhou Unversty, Zhengzhou, Chna and Nanyang Technologcal Unversty, Sngapore, Techncal Report, vol., 3.