Duality Search: A novel simple and efficient meta-heuristic algorithm. *Mohsen Shahrouzi 1)

Transcription

1 Dualty Search: A novel smple and effcent meta-heurstc algorthm *Mohsen Shahrouz 1) 1) Engneerng Department, Kharazm Unversty, Tehran, Iran 1) shahruz@khu.ac.r ABSTRACT In ths artcle a novel meta-heurstc s proposed based on the concept of dualty n nature. Many real world creatures apply ths concept snce they are found n couples:.e. famles of male and female wth ther chldren. Males are usually powerful n some features that females are weak n them and vce versa. Perfect performance of such a couple s domnated by cooperaton of both these dual ndvduals. The present method apples ths concept by subdvdng the populaton of search agents nto the prmary ndvduals and ther duals. Dual of an ndvdual s determned by a ftness rankng procedure. A smple and effcent algorthm s thus proposed utlzng specal movements of the artfcal couples to search the desgn space. Several features of the proposed Dualty Search ncludng ts memory-less structure, robustness and effcency n dfferent search spaces wthout ntrnsc parameter tunng has made t very compettve to some other lterature methods. A varety of problems ncludng dfferent test functons and engneerng examples are treated to llustrate performance of the proposed method n global optmzaton. Keywords: dualty search, meta-heurstc, global optmzaton, dversty measure 1. INTRODUCTION Many real-world applcatons requre fndng ther best soluton n the form of optmzaton problems. In ths regard, zero-order optmzaton methods have receved consderable attenton as they do not requre evaluatng and approxmatng gradent of the objectve functon (Arora, 4). Most of the meta-heurstc algorthms fall n ths category as general-purpose optmzaton tools for varous felds of engneerng problems (Yang et al. 1). They usually smulate some cultural, bologcal or physcal phenomena n nature n order to acheve optmal soluton; wthn a practcal tme (Kaveh, 14; Yang, 13). The need to search for global optma n the non-convex, mult-modal and constraned desgn spaces has rased several attempts to develop new algorthms 1) Ph.D., Faculty of Engneerng, Kharazm Unversty.

2 whch can work n contnuous and dscrete problems wth dfferent complextes. Some of the most recent ones are P (Shahrouz, 11a), GBMO (Abdeshr et al. 13), RMO (Rahman and Rubyah, 14), WWO (Zheng, 15), DFO (Shahrouz and Kaveh, 15), ALO (Mrjall, 15), CSA (Akbarzadeh, 16) and TWO (Kaveh and Zolghadr, 16). Among them, populaton-based methods are generally more powerful than sngle-agent ones n global search. Parameter tunng s a challenge n practcal applcaton of many meta-heurstc algorthms. Due to no-free-lunch theory (Wolpert and Macready, 5), such parameters should be re-tuned for every other problem to truly acheve ts best performance. The tunng process s usually performed va several tral runs wth the charge of extra computatonal effort for mplementaton of an algorthm to effcently seek global optmum and to avod premature convergence. Therefore, fewer control parameters s more nterestng from practcal pont of vew. Some recent attempts have addressed ths matter ncludng CBO (Kaveh and Mahdav, 14, 15), TWO (Kaveh and Zolghadr, 16), CSA (Akbarzadeh, 16), and TLBO (Rao et al. 11, Rao and Patel, 13). The most successful attempts have no ntrnsc parameters rather than populaton sze and number of teratons (Rao, 16). The present work ntroduces a new parameter-less meta-heurstc called Dualty Search,. It apples a dualty dfference between capabltes of such males from females n the ftness scope. Consequently, the populaton of search agents s subdvded to prmary ndvduals and ther duals. The man algorthm steps rely on specal dual walks toward better postons. The method has also stmulated a chld ndvdual that shares features of ts parents n the correspondng famly unt. A number of llustratve examples are treated to evaluate performance of the proposed method va comparson wth some other well-known meta-heurstcs.. DUALITY SEARCH CONCEPTS Many creatures n the real-world are dvded nto male and female types. These two can be consdered dual to each other based on ther dfferent capabltes n a specfc feature. For example, male agents are usually more powerful n hard-workng and competton wth the others. In the other hand, females are more capable for chld treatment than males. Survval of the speces n the nature s preserved by cooperatve acton of males and females n couples rather than by sngle acton of each. Every such a couple s called a famly unt ncludng one male and one female. The dea s extended here to develop an effcent yet smple optmzaton method called Dualty Search. In the termnology, artfcal ndvduals n a populaton of search agents are subdvded nto prmary ones and ther duals. Such a dualty s defned usng the ftness rank of agents so that the fttest member of the populaton s taken dual to the least ft one. Settng asde these two from the current group of ndvduals, another couple of duals s dentfed n the same way and the process s repeated untl no further ndvdual s remaned for the last set. The populaton sze s essentally even n ths process so that fnally t can be dvded nto subpopulatons;.e. prmary and dual. Another concept s dfference of walk types n the search space for an ndvdual wth ts dual. In a sngle walk, ether a prmary ndvdual or ts dual (not both) can move

3 toward a specfc poston; e.g. the fttest locaton that has already found by the algorthm. Besdes, when an ndvdual s ftter than ts dual they can get farther from each other. Note that the recent defnton of dualty uses ftness rankng and dffers from defntons n the opposton-based algorthms whch apply desgn varables scope (Tzhoosh, 5; Rahnamayan et al. 8; Han and He, 7). Another feature s property sharng between a prmary ndvdual and ts dual by a thrd type of search agent called a chld. Such a chld provdes extra drecton of movement n addton to those revealed ether by the prmary or dual agents. Ths dea smulates crossover between chromosomes n the Evolutonary Algorthms (Back, 1996; He and Wang, 7; Shahrouz, 11a), however, ts combnaton wth dual walks s a specal feature of the proposed. In addton, the generated chld s not added to the next populaton so that populaton sze remans fxed durng teratons of the search. The present method does not requre any auxlary memory of the prevous ndvduals. Snce the current study ams to fnd the best objectve functon, the aforementoned walks are desgnated regardng the best poston already found as a member of the current populaton. Such movements are smlar to that n the Dfferental Evoluton, (Prce et al., 5). As the present method uses ftness-based rankng n order to dstngush couple of dual agents t s expected that when a prmary ndvdual move from ts poston, ts dual s smultaneously changed. Hence, search walks n any teraton are only performed for ether prmary or dual subpopulaton rather than for the entre populaton. As can be realzed further, ths specal feature has provded extra computatonal effcency for. 3. A DUALITY SEARCH ALGORITHM Regardng the aforementoned concepts, a algorthm s ntroduced for the ftness maxmzng problem va the followng steps: 3.1 Intaton Set the number of total ndvduals as the prescrbed PopSze. Randomly generate the entre populaton of ndvduals and evaluate ther ftness. 3. Selecton of subpopulaton Gbest Sort the populaton n ascendng order of ftness and dentfy the fttest ndvdual X as the global best soluton. Then determne the ndex of dual for each ndvdual usng the followng relaton. DualInd PopSze 1, (1) Dvde the entre populaton nto prmary and dual subpopulatons. Choose ether prmary or dual subpopulaton to modfy postons of ther ndvduals. 3.3 Inner Loop

4 Repeat the followng steps for every th ndvdual subpopulaton. X and ts dual D X, n the selected Dualty-based walk Choose ether X or ts dual ndvdual: D X and ntate the search drecton to move the Gbest S1 X Y, () Introduce addtonal movng drecton by: S X X (3) D, 3.3. Walk toward the Chld For each couple produce ther chld X Ch. It can be generated by a unform crossover over the parents that are dual of each other. Identfy an extra drecton vector as: Chld S3 X Y, (4) Selecton and replacement Generate the canddate soluton and evaluate ts ftness. Cand X X V,(5) V s the velocty vector constructed by vector-sum of each walk drecton after ts producton by a random generator. Replace the ndvdual canddate s ftter than the current ndvdual. X wth Cand X f such a 3.4 Termnaton or repeat the outer loop Go back to step and repeat untl termnaton crteron s satsfed. Termnaton may occur when sum of velocty norms n the current actve subpopulaton tends to zero; that means no further movement of ndvduals wll occur from ther last postons. However, an alternate termnaton crteron s more common: to repeat untl the teraton number reaches a prescrbed value Iter. max 3.5 Results announcement Gbest At ths fnal step, announce the updated X as the optmum. Fg.1 reveals Gbest pseudo-code of the proposed. Snce X s updated as a member of every populaton no extra memory s needed to save t.

5 4. ILLUSTRATIVE EXAMPLES Some examples of unmodal and multmodal test functons (Yang, 1) are selected for llustratve purposes. The results are treated by and also compared wth two other meta-heurstcs; Partcle Swarm Optmzaton, PSO (Kennedy and Eberhart, 1) and Teachng-Learnng-based-Optmzaton, TLBO. PSO s wdely used n many engneerng applcatons and has a very smple algorthmc structure. As the second method, TLBO s selected due to ts parameter-less structure (Rao, 14). For the sake of true comparson n each case, the ntal populaton and the number of teratons s taken smlar. All the algorthms are run wth the same PopSze of 1 whch s a relatvely small number to test ther capablty n searchng wth such a few agents. However, the number of objectve functon evaluatons s taken the base of comparson n the ftness vew. Other control parameters of PSO are taken C 1, C C ; mplemented by the followng relaton: new Pbest Gbest V C V rand C ( X X ) rand C ( X X ), (6) Inertal Cogntve Socal Inertal Cogntve Socal Applyng termnaton crteron of as V the resulted number of teratons for each example are derved and used to run PSO and TLBO, as well. The proposed TLBO s mplemented wth an eltst strategy whch saves the global best ndvdual durng the search by an auxlary memory. As TLBO has no ntrnsc parameters; t s run just wth the same ntal populaton and number of teratons as. Regardng dversty varaton as the search progresses, trace of V s drawn to study behavor of the treated algorthms. In addton, total CPU tme on the same platform s compared between the treated methods to evaluate ther computatonal effcency. Test functon mnmzaton s transformed here to a ftness maxmzng problem: Maxmze Ftness( X ) f( X ), LB X x,..., x, x x x 1 n j UB (7) Pror to ftness evaluaton, each varable of the desgn vector s forced to fall between LB UB x, x as ts lower and upper bounds, respectvely. LB UB x mn(max( x, x ), x ), (8)

6 The number of desgn varables n s taken n the followng examples for llustratve purposes. Fg.1 Pseudo-code of the proposed algorthm 4.1 Test problem 1: sphere functon Mnmzng ths benchmark s selected as an example of a convex unmodal problem. It s defned by the followng relaton: n ( ) x j, j 1 f X (9) LB UB Whereas feasble hyper-cube s lmted to x 5.1, x 5.1. It has no local optma * but one global mnmum obvously located at the orgn: X resultng n f *. Performance of s compared wth PSO n Fg.3. Accordng to the Pseudo-code of Fg.1, the number of ftness evaluaton calls n every teraton of PSO s twce that of. Therefore, true comparson of convergence should be performed va functon calls.

7 As depcted n Fg.3a, t can be realzed that convergence s compettve to PSO usng the same ntal populaton and consequent startng elte ftness. Applyng.1, the proposed could acheve * f( X ).849 wthn only 56 teratons. It s * whle PSO obtaned f( X ) The reason s declared when comparng dversty trace of these methods. Fg.3b shows that despte PSO, has successfully reduced V to refne ts search toward optmum n ths convex example. Fg. Sphere test functon Accordng to Fg.4, TLBO has more rapd teraton-wse decrease n the dversty than ; however, the proposed method has overrdden ts convergence curve n early functon calls. In another word, has shown proper balance between ntensfcaton and dversfcaton n ths example. Note that TLBO apples PopSze # teratons ftness evaluatons whle ths number s PopSze # teratons for PSO and 1 PopSze # teratons for. Therefore s expected to be the most effcent among these methods. The matter s confrmed regardng elapsed tme of the algorthms n every comparatve run for a couple of methods. Accordng to Fg3.c, the percentage of computatonal tme n a sngle run s dstrbuted as 4% for and 6% for PSO. Ths rato s more sever n Fg.4c as 3% for vs. 77% for TLBO. Whle the number of functon calls n the same teraton for s 5% of TLBO and 5% of PSO, the recent results of Fg.3c and Fg.4c, gve extra nformaton about tme complexty of the algorthms.

8 Sum of Velocty Norm BestSoFar Ftness Sphere PSO Functon Calls 8 7 PSO (a) Sphere Iteraton (b) 6% 4% (c) Fg.3 Comparson of wth PSO: (a) ftness (b) dversty and (c) computatonal tme percentage for the sphere functon

9 Sum of Velocty Norm BestSoFar Ftness Sphere TLBO Functon Calls 35 3 (a) Sphere TLBO Iteraton (b) 3% 77% (c) Fg.4 Comparson of wth TLBO: (a) ftness (b) dversty and (c) computatonal tme percentage for the sphere functon

10 Fg.5 Shubert s test functon 4. Test problem : Shubert s functon The prevous tests are repeated here treatng an example of a complex multmodal benchmark wth several local optma and 18 global optma as depcted n Fg.5. It s defned by the followng relaton: n 5 f ( X ) k(cos[( k 1) x j k)], (1) j 1 k 1 LB UB The varable lmts are taken x 5.1, x 5.1. Value of the Shubert s functon at ts global optma s the same as f * (Karm and Sarry, 1). Applyng the same threshold of.1 as the termnaton crteron, number of teratons s rased by to more than 3 tmes than the prevous example. Besdes, accordng to Fg.6 and Fg.7 t s realzed that trend of dversty decrease wth teraton has got slower. Note that ths example has several local and global optma and s more complex than the prevous convex functon. It s evdent from Fg.6 that despte, the employed PSO has not been successful n dversty balance as the search progresses; because t has led to premature convergence. PSO has acheved f whch s qute dfferent from global optmum captured by as * f In another word, has better capablty of escapng from local optma n such a complex desgn space wth respect to PSO. Accordng to Fg6.c and Fg3.c, the tme complexty rato s slghtly altered wth respect to prevous example from 4% to 37% comparng over PSO tme.

11 Sum of Velocty Norm BestSoFar Ftness Shubert PSO Functon Calls (a) Shubert PSO Iteraton (b) 63% 37% (c) Fg.6 Comparson of wth PSO: (a) ftness (b) dversty and (c) computatonal tme percentage for the Shubert s functon

12 Sum of Velocty Norm BestSoFar Ftness Shubert TLBO Functon Calls 45 4 TLBO (a) Shubert Iteraton (b) 1% 79% (c) Fg.7 Comparson of wth TLBO: (a) ftness (b) dversty and (c) computatonal tme percentage for the Shubert s functon

13 Regardng Fg.7a, has agan shown comparable convergence wth TLBO n * obtanng fnal value of f , n optmzng ths multmodal functon. has captured such a global optmum by 15 functon calls va 4 teratons. TLBO has acheved t wthn 183 teratons but by 365 ftness evaluatons. Comparson of the total elapsed tme up to total 548 teratons, declares tme complexty of s 1% of TLBO. Hence, t s concluded that s not only robust but also effcent n capturng global optma n such a complex multmodal search space. Fg.8 Langermann s test functon 4.3 Test problem 3: Langermann s functon Ths example s a sample of a non-convex functon wth an asymmetrc multmodal desgn space, defned by the followng Langermann s functon (Fg.8): ( x a ) f ( X ) c exp cos ( x a ), j n j j j (11) The fxed parameters are gven by: T T A, C 1 5 3, (1) LB UB Lmts on the varables are set to x, x 1. Accordng to Fg.9a, can exhbt stable convergence toward optmum wthout premature convergence. The reason s

14 Sum of Velocty Norm BestSoFar Ftness behnd proper robustness of by keepng more dversty than TLBO durng early teratons of the search n such a complex problem (See Fg.9b) Langermann TLBO Functon Calls 4 35 TLBO (a) Langermann Iteratons (b) 19% 81% (c) Fg.9 Comparson of wth TLBO: (a) ftness (b) dversty and (c) computatonal tme percentage for the Langermann s functon

15 In ths example could capture the optmum of f (.7934,1.597) by 19% total tme of TLBO whch resulted n f (1.4165, ) 4.19 after 1 teratons. (a) (b) Fg.1 (a) The structural model and (b) desgn space of the 3-bar truss problem 4.4 Test problem 4: 3-bar truss desgn Ths s an example of a constraned engneerng problem that s chosen because ts desgn space s llustratable (Shahrouz and Kaveh, 15). It has only desgn varables X ( x1, x) as secton areas of the truss members, shown n Fg.1a. The

16 objectve s to mnmze materal consumpton subjected to structural behavour constrants where desgn varables are lmted wth n [,1] doman. The problem s formulated as follows wth fxed parameters of l 1cm, kn / cm and P kn as the vertcal load: Mnmze f ( X ) l ( x x ), (13) 1 Subject to g x x P 1 1 x1 x1 x (14) g x P x1 x1x (15) g 3 1 P x x 1 (16) It s treated n the followng unconstraned form. The penalty coeffcent of k p 5 s employed n ths study. Maxmze Ftness( X ) f ( X ) (1 k max(,g ( X ))), (17) 3 p 1 Accordng to Fg.11, t s realzed that TLBO have the most rapd dversty decrease. In contrary, PSO dversty fluctuates about a nearly constant hgher value. trend s between PSO and TLBO so that t has a hgh dversty at early stages of optmzaton whch gradually decreases as progressng to the end. Numercal results of, PSO and TLBO are compared n Table 1 for ths constraned structural optmzaton problem. It s observed that the mnmal weght result of s between those of PSO and TLBO wth a dfference of less than 1%. However, has obtaned such a result wthn nearly one forth of TLBO tme and half of PSO tme. Average measure of velocty norm for n these sample runs, has been less than % of PSO and 19% of TLBO among the entre teratons. Table.1 Comparatve results of PSO, TLBO and n 3-bar truss desgn Method: PSO TLBO Cost functon Infeasblty Elapsed tme Rato 6% 46% 1% Mean Sum of Velocty Norm

17 Sum of Velocty Norm ThreeBarTrussDesgn PSO TLBO Functon Calls Fg.11 Dversty comparson between PSO, TLBO and n the 3-bar truss desgn 5. CONCLUSIONS A new meta-heurstc algorthm was ntroduced regardng dualty of search agents n populaton based stochastc optmzaton. apples specal dualty measure after dynamc sortng of ndvduals based on ther ftness rank at every teraton. The algorthm takes beneft of both the prmary ndvdual and ts dual as a couple of agents va ts specal walks. Snce an ether prmary or dual part of the populaton s actvated, s run wth low functon calls for the same number of teratons. Addtonal walk of the dual parents toward ther chld has provded an enhanced capablty for the proposed algorthm. Although that the number of ftness evaluatons s reduced compared to many other algorthms, showed compettve performance n seekng global optma not only n smple unmodal test functons but also n complex non-convex symmetrc or asymmetrc desgn spaces. The proposed method was successfully tested n unconstraned and constraned problems tracng ts dversty n comparson wth PSO as a vector-sum optmzaton and TLBO as a parameter-less algorthm. It was concluded that can robustly alter ts trend of dversty varaton wthout requrng rgorous parameter tunng as the complexty of the desgn space changes. Applyng a relatvely small populaton sze n addton to actvatng half of t at any

18 teraton; s proven to be very effcent and compettve search algorthm n the treated problems. It requres consderably less computatonal effort and functon calls to acheve smlar results to other treated algorthms. In the lght of the current study, the proposed dualty search can be recommended as a smple method wth outstandng effcency and robust dversty control n optmzaton. The parameter-less and memory-less archtecture of the proposed has made t nterestng for practcal mplementaton n engneerng problems. REFERENCES Abdechr, M., Meybod, M.R., Bahram, H. (13), Gases brownan moton optmzaton: An algorthm for optmzaton (GBMO). Appl. Soft Comput., 13(5), Akbarzadeh, A. (16), A novel metaheurstc method for solvng constraned engneerng optmzaton problems: Crow Search Algorthm, Comput. Struct., 16, 1-1. Arora J.S. (4), Introducton to optmum desgn, Elsever, US. Back, T. (1996), Evolutonary Algorthms n Theory and Practce, Oxford Unversty Press, US. Karm, A., Sarry, P. (1), Global smplex optmzaton: a smple and effcent metaheurstc for contnuous optmzaton, Eng. Appl. Artf. Intell., 5, Kennedy J. and Eberhart R. (1), Swarm Intellgence, Academc Press, UK. Kaveh A. (14), Advances n Metaheurstc Algorthms for Optmal Desgn of Structures, Sprnger Internatonal Publshng, Swtzerland. Kaveh, A., Mahdav, M.R.(14), Colldng Bodes Optmzaton method for optmum dscrete desgn of truss structures, Comput. Struct., 139, Kaveh, A., Mahdav, M.R.(15), Colldng Bodes Optmzaton: extenson and applcatons, Sprnger Internatonal Publshng, Swtzerland. Kaveh A, Zolghadr, A. (16), A novel meta-heurstc algorthm: Tug of War Optmzaton, Int. J. Optm. Cvl Eng.,6(4), Han, L., He, X.S. (7), A novel opposton-based partcle swarm optmzaton for nosy problems, Thrd Int. Conf. Natural Comput., Hakou, Chna, He, Q., Wang, L. (7), An effectve co-evolutonary partcle swarm optmzaton for constraned engneerng desgn problems, Eng. Appl. Artf. Intell,, Mrjall, S. (15), The Ant Lon Optmzer, Adv. Eng soft, 83, Prce, K., Storn, R.M., Lampnen, J.A. (5), Dfferental Evoluton: A Practcal Approach to Global Optmzaton, Natural Computng Seres, Sprnger; 1st Ed. Rahman, R., Rubyah, Y. (14), A new smple, fast and effcent algorthm for global optmzaton over contnuous search-space problems: Radal Movement Optmzaton, Appl. Math. Comput., 48, Rahnamayan, S., Tzhoosh, H.R., Salama, M.M.A. (8), Opposton versus Randomness n Soft Computng Technques, Appl. Soft Comput., 8, Rao, RV., Savsan, VJ., Vakhara, DP. (11), Teachng-learnng-based optmzaton: A novel method for constraned mechancal desgn optmzaton problems, Computer-Aded Des., 43(3),

19 Rao, R.V., Patel, V. (13), An mproved teachng-learnng-based optmzaton algorthm for solvng unconstraned optmzaton problems, Scenta Iranca, (3), Rao R.V. (16), Teachng Learnng Based Optmzaton Algorthm And Its Engneerng Applcatons, Sprnger Internatonal Publshng, Swtzerland. Shahrouz, M. (11a), A new hybrd genetc and swarm optmzaton for earthquake accelerogram scalng, Int. J. Opt. Cvl Eng., 1(1), Shahrouz, M. (11b), Pseudo-random drectonal search: a new heurstc for optmzaton, Int. J. Opt. Cvl Eng., 1(), Shahrouz, M., Kaveh, A. (15), Dynamc Fuzzy-membershp Optmzaton: An Enhanced Meta-heurstc Search, Asan J. Cvl Eng., 4(3), Tzhoosh, H.R. (5), Opposton-Based Learnng: A New Scheme for Machne Intellgence, Int. Conf. Comput. Intell. Modellng Control Automaton (CIMCA 5), Venna, Austra, I, Wolpert, D.H., Macready, W.G. (5), Co-evolutonary free lunches, IEEE Trans. Evolut. Comput., 9(6), Yang X.-S. (1), Test problems n optmzaton, n: Engneerng Optmzaton: An Introducton wth Metaheurstc Applcatons (Eds Xn-She Yang), John Wley & Sons. Yang, X.-S. (13), Bat algorthm: lterature revew and applcatons, Int. J. Bo- Inspred Computaton, 5 (3), Yang, X-S., Gandom, A.H., Talatahar, S., Alav, S. (1), Metaheurstcs n water, geotechncal and transport engneerng, Elsever, US. Zheng, Y.J. (15), Water wave optmzaton: A new nature-nspred metaheurstc, Comput. Oper. Res., 55, 1-11.