Continuous Ant Colony System and Tabu Search Algorithms Hybridized for Global Minimization of Continuous Multi-minima Functions

Transcription

1 Cotiuous At Coloy System ad Tabu Search Algorithms Hybridized for Global Miimizatio of Cotiuous Multi-miima Fuctios Akbar Karimi Departmet of Aerospace Egieerig, Sharif Uiversity of Techology, P.O. Box: , Tehra, Ira. Hadi Nobahari Departmet of Aerospace Egieerig, Sharif Uiversity of Techology, P.O. Box: , Tehra, Ira. Patrick Siarry (Correspodig Author) Uiversité Paris Val-de-Mare, Laboratoire Images, Sigaux et Systèmes Itelligets (LiSSi), 9400 Créteil, Frace

2 Abstract A ew hybrid optimizatio method, combiig Cotiuous At Coloy System (CACS) ad Tabu Search (TS) is proposed for miimizatio of cotiuous multimiima fuctios. The ew algorithm icorporates the cocepts of promisig list, tabu list ad tabu balls from TS ito the framework of CACS. This eables the resultat algorithm to avoid bad regios ad to be guided toward the areas more likely to cotai the global miimum. New strategies are proposed to dyamically tue the radius of the tabu balls durig the executio ad also to hadle the variable correlatios. The promisig list is also used to update the pheromoe distributio over the search space. The parameters of the ew method are tued based o the results obtaied for a set of stadard test fuctios. The results of the proposed scheme are also compared with those of some recet at based ad o-at based meta-heuristics, showig improvemets i terms of accuracy ad efficiecy. Keywords At Coloy Optimizatio, Tabu Search, Hybrid Meta-heuristics, Global Optimizatio, Cotiuous Optimizatio

3 Itroductio The global miimizatio of cotiuous multi-miima fuctios cosists i fidig the global miima without beig trapped ito ay of the local miima. Various metaheuristic approaches have bee developed to solve these problems, such as Simulated Aealig, Geetic Algorithms (GA), Tabu Search, At Coloy Optimizatio ad so o. A big challege i developig global optimizatio approaches is to compromise the cotradictory requiremets, icludig accuracy, robustess ad computatio time. It is difficult to meet all these requiremets by cocetratig o a sole meta-heuristic. I recet years, there has bee a up-growig iterest i hybridizatio of differet metaheuristics to provide more efficiet algorithms. I this paper a hybridizatio of Tabu Search ad At Coloy Optimizatio meta-heuristics is proposed. Tabu Search (TS) was origially developed by Glover [,]. This meta-heuristic has bee successfully applied to a variety of combiatorial optimizatio problems. The extesio of TS to cotiuous optimizatio problems has bee ivestigated otably i [3-8]. The proposed algorithm i [6], called Cotiuous Tabu Search (CTS), starts from a radomly geerated iitial solutio, called the iitial curret solutio. From this curret solutio, a set of eighbors are geerated. To avoid the appearace of a cycle, the eighbors of the curret solutio, which belog to a subsequetly defied tabu list, are systematically elimiated. The obective fuctio is evaluated for each eighbor. The best eighbor becomes the ew curret solutio, eve if it is worse tha the iitial curret solutio. This allows escapig from the local miima of the obective fuctio. The cosequetly geerated curret solutios are put ito a circular list of tabu solutios, called tabu list. Whe the tabu list becomes full, it is updated by removig the first solutio etered. The a ew iteratio is performed. The previous procedure is repeated by startig from the ew curret poit, util some stoppig coditio is reached. Usually, the algorithm stops after a give umber of iteratios for which o improvemet of the obective fuctio occurs. To improve the accuracy of CTS, Chelouah ad Siarry [7,8] proposed a variat of TS, called Ehaced Cotiuous Tabu Search (ECTS). This scheme divides the optimizatio process ito two sequetial phases, amely diversificatio ad itesificatio. I diversificatio, the algorithm scas the whole solutio space ad detects the areas, which are likely to cotai a global miimum. The ceters of these

4 promisig areas are stored i a list, called the promisig list. Whe diversificatio eds, the itesificatio starts. I this phase, the search is cocetrated withi the most promisig area by makig the search domai smaller ad gradually reducig the eighborhood structure. This strategy improves the performace of the algorithm ad allows exploitig the most promisig area with more accuracy [7, 8]. The other meta-heuristic, utilized i this paper, is At Coloy Optimizatio (ACO), which was first proposed by Marco Dorigo ad colleagues [9,0] as a multi-aget approach to solve difficult combiatorial optimizatio problems. The first algorithm ispired from the at coloy fuctioig, is At System (AS) [9,], which is the base of may other approaches such as Max-Mi AS (MMAS) [], At Coloy System (ACS) [3], At-Q [4] ad ANTCOL [5]. The mai idea utilized i all of these algorithms has bee adopted from the ats' pheromoe trails-layig behavior, which is a idirect form of commuicatio mediated by modificatios of the eviromet. The applicatio of at algorithms to optimizatio problems ca be divided ito two mai categories, icludig discrete ad cotiuous problems. Most of the primary algorithms were developed to solve discrete optimizatio problems such as Travelig Salesma Problem, Quadratic Assigmet Problem, Job-Shop schedulig, Vehicle Routig, Sequetial Orderig, Graph Colorig, Routig i Commuicatios Networks ad so o [6,7]. However, there have bee several attempts to adapt ACO for cotiuous optimizatio problems. Bilchev ad Parmee [9] ad Wodrich ad Bilchev [8] proposed a method called Cotiuous At Coloy Optimizatio (CACO), which uses at coloy framework to perform local searches whereas the global search is hadled by a geetic algorithm. Aother idea, kow as API method [0], was proposed based o a kid of recruitmet process. This method was ispired by a primitive at's recruitmet behavior. It performs a tadem-ruig which ivolves two ats ad leads them to gather o a same hutig site. The authors use this particular recruitmet techique to make the populatio proceed towards the optimum solutio, by selectig the best poit amog those evaluated by the ats. A recet research o modelig ats' behavior [,] has show that it is possible to start a recruitmet sequece eve without takig pheromoe trails ito accout. I this model, the stigmergic process is cosidered oitly with iter-idividual relatioships. The model is ispired from the flow of ats exitig the est after the etry of a scout who has discovered a ew food source. To differetiate this process from the

5 recruitmet, the authors have called it mobilizatio. This algorithm is called Cotiuous Iteractig At Coloy (CIAC). Aother approach to solve cotiuous optimizatio problems is by covertig them to discrete form so that the discrete versios of at algorithms ca be used [3,4]. Oe pure pheromoe based method for global miimizatio of cotiuous optimizatio problems, called Cotiuous At Coloy System (CACS), was proposed by Pourtakdoust ad Nobahari [5]. To deal with a cotiuous fuctio, the pheromoe distributio over the search space was modeled i the form of a ormal Probability Distributio Fuctio (PDF), the ceter of which is the last best global solutio, foud from the begiig of the trial, ad its variace depeds o the aggregatio of the other promisig areas aroud the best oe. I this cotext a radom geerator with ormal probability distributio fuctio is utilized as the state trasitio operator. I aother work, published cocurretly by Socha ad Dorigo (first i [6] ad the i [7,8]), a combiatio of several ormal PDF was utilized istead of the discrete PDF used i the origial ACO formulatios. They called this method as At Coloy Optimizatio for Real domais (ACO R ). I recet years, CACS has bee successfully applied to several practical optimizatio problems such as fuzzy rule learig ad shape optimizatio [9-3]. It should also be oted that amog aforemetioed at-related algorithms, CACO, API ad CIAC all have coceptual differeces with the origial ACO formulatios, regardig the operators they use. Therefore, as discussed i [8], they do ot qualify to be extesios of ACO. I this work the authors hybridize their previously proposed methods to itroduce a ew hybrid optimizatio scheme. The proposed algorithm is called Tabu Cotiuous At Coloy System (TCACS). The basic structure of TCACS is very similar to the origial CACS, while it icorporates the cocepts of tabu ad promisig lists, used i CTS ad ECTS, ito the framework of CACS to improve the performace. The use of the promisig list improves the covergece rate while the utilizatio of the so called tabu balls guides the ats toward the solutios far eough away from the worst regios of the search space. O the other had, the use of promisig ad tabu lists allows to dyamically adust the size of the tabu balls durig the executio of the algorithm. Furthermore, to hadle the udesired correlatios betwee the optimizatio variables

6 ad have a more effective samplig, all pheromoe calculatios are performed i a separate coordiate system whose orietatio is specified i a o-determiistic way. The paper is orgaized as follows: sectio is devoted to a detailed descriptio of the ew optimizatio method. The experimetal results are provided i sectio 3. Here, the parameters of TCACS are tued based o the results obtaied for a set of stadard test fuctios ad the performace of the ew algorithm is compared with those of some other meta-heuristics, amely four at-based (ACO R, CIAC, CACO, ad CACS) ad four o at-based methods (CGA, CHA, ECTS, ad ESA). The fial coclusio is made i sectio 4. Tabu Cotiuous At Coloy System It is desired to fid the global miimum of the fuctio f, withi a give iterval [ a, b], i which the miimum occurs at a poit x s. I geeral, f ca be a multi-variable fuctio, defied o a subset of R delimited by itervals ai, bi ], i,, [ =.. Geeral Settig Out of the Algorithm Fig. shows the geeral iterative structure of TCACS. A high level descriptio of the sequetial steps is show i this figure. I the followig subsectios, these steps are discussed i detail. As i CACS, a cotiuous pheromoe model is used to gradually guide the ats toward the global miimum poit. This pheromoe model is i fact a strategy to assig a cotiuous probability distributio to the whole solutio space ad to methodically update it as the algorithm progresses. Durig ay iteratio, ats move from their curret positios to the ew destiatios accordig to the curret pheromoe distributio. The destiatios are chose usig a radom geerator with ormal probability distributio fuctio. The values of the obective fuctio are calculated i these ew poits ad some kowledge about the problem is acquired, which is used to update the pheromoe distributio. Like CACS, for each dimesio of the solutio space a ormal probability distributio fuctio is used to model the pheromoe aggregatio aroud the last best poit foud from the begiig of the trial. Therefore, for x ( x, x,, ) = beig a x

7 * * * * arbitrary poit withi the solutio space, ad x ( x x,, x ) = beig the last best, poit, the pheromoe itesities are give by ormal distributio fuctios i the followig form: * ( x i x ) τ x i = e ( ) i σ i () where σ i is the variace of the ormal distributio correspodig to the i-th dimesio of the solutio space. The algorithm updates x * ad σ as it proceeds ad the pheromoe distributio over the whole solutio space gradually chages. A maor differece betwee TCACS ad CACS is the use of tabu ad promisig lists. Simply a specified umber of the best poits foud from the begiig of the algorithm up to the curret iteratio form the promisig list. Likewise, a specified umber of the worst poits foud so far costitute the tabu list. Each member of the tabu list is the ceter of a tabu ball, the size of which is updated durig iteratios. Tabu balls specify circular, spherical ad i geeral hyper-spherical regios withi the solutio space. The ats are ot allowed to select ay poit iside the tabu balls, while they are choosig their ew destiatios. I other words, a acceptable movemet is made whe the ew radomly selected destiatio does ot lie withi ay of the tabu balls. Fig. lists the pseudo code of TCACS algorithm.. Iitializatio The ew algorithm has some cotrol parameters, defied i the ext subsectios, which must be set before the executio of the algorithm. As it will be discussed later, pheromoe distributio is computed i a temporary coordiate system, which is iitially the same as the origial coordiate system. Moreover, both the tabu ad promisig lists must be iitially empty..3 Movemet of the ats Each iteratio starts by the movemet of the ats to ew locatios x ( x, x,, x ), =,, k =. Chose radomly withi the solutio space, these ew poits should ot belog to ay of the tabu balls. It should be oted that if a

8 radom geerated value for the i-th compoet of x lies outside the specified Iterval [ a i, bi ], the a ew oe is geerated ad the process is repeated util a acceptable value is obtaied. I order to be able to use the coordiate correlatio hadlig routie described i sectio.7, we describe ad calculate the pheromoe distributio i a separate coordiate system, deoted by Z, whose orietatio is updated at ay iteratio based o the curret distributio of the idividuals. Thus, to move each at to a ew locatio, a vector of radom icremets i Z axes is geerated based o the curret pheromoe distributio. This vector is the trasformed ito the origial coordiate system, X, ad added to the best foud solutio as follows: x = x * + R XZ N (0, σ ) () ew Where * x is the best solutio foud so far, R XZ is the rotatio matrix from Z to X axes, ad N ( 0,σ ) is a vector of radom values geerated accordig to a ormal probability distributio with mea of zero ad stadard deviatios specified i the vector σ. As metioed before, σ states the pheromoe distributios accordig to Z axes. After selectio of all destiatios, the obective fuctio is evaluated at these poits. The results are stored i a vector y, where y = f ( x ), =,, k. At the first iteratio, both the tabu ad promisig lists are empty, therefore the iitial positio of the ats is selected usig a uiform radom geerator, whereas for all subsequet iteratios, the Eq. () is used to select the radom poits..4 Updatig Tabu List, Promisig List ad Tabu Ball Radius Tabu ad promisig lists are k matrices that store the worst ad the best poits visited from the begiig of the algorithm up to the curret iteratio. The parameters k ad are the umber of ats ad the dimesio of the problem, respectively. Therefore, each row of the tabu or promisig lists represets a poit withi the solutio space.

9 I order to update the tabu ad promisig lists, the ew poits geerated i the curret iteratio ad the curret cotets of the tabu ad promisig lists are first combied to form a set of 3 k idividuals. The all idividuals fallig outside a hyper-rectagular eighborhood of the curret best solutio ( x * ), defied as Eq (3), are discarded from the set. It should be oted that the area outside of this regio is ot effectively covered by the samplig process ad hece it is ot useful to have ay tabu balls withi it. { σ} { σ} x 3 max x x + 3 max, i =,..., (3) * * i Now the k umber of the best ad the worst idividuals amog the ew set are specified as the ew promisig ad tabu poits, respectively. The promisig list is filled i the first iteratio, after the iitial locatios of the ats have bee selected. It is updated i the later iteratios ad is used to update pheromoe distributios. Moreover, it should be oted that the tabu list is empty i the first iteratio ad is filled i the secod iteratio after the algorithm has evaluated k poits withi the solutio space. The tabu list is used i the optimizatio process from the third iteratio to the ed. Durig ay iteratio, the radius of the tabu balls is calculated as half of the smallest distace betwee the tabu poits ad the promisig poits ( d mi ) as follows: r T = d mi k = mi = x T x P (4) where subscripts P ad T represet the promisig ad the tabu lists, respectively. The parameter r specifies the miimum Euclidea distace that must exist betwee each T radom poit ad tabu poits. It is importat to ote that usig this scheme, the size of the tabu balls dyamically chages based o the distributio of the promisig ad the tabu poits i the solutio space..5 Updatig Pheromoe Distributios TCACS utilizes the pheromoe updatig rule of CACS as the basic structure. However some modificatios are made to improve the performace. Pheromoe updatig rule of CACS ca be stated as follows: the value of the obective fuctio is evaluated for the ew selected poits by the ats. The the best poit foud from the begiig of the

10 trial is assiged to * x. Also, σ is updated based o the evaluated poits durig the last iteratio ad the aggregatio of those poits aroud x *. To satisfy simultaeously the fitess ad aggregatio criteria, CACS uses the cocept of weighted variace as follows: k * [ x * i x ] i = y y * σ i =, for all i which y y k (5) = y y * However, i TCACS, the members of the promisig list are used istead of the curret positios of the ats, ad some ew elemets are itroduced. The pheromoe updatig rule of TCACS is stated as follows: k ( γ w f + ( γ ) wd ) * i P i = * σ i =, for all i which yp y (6) k ( γ w f + ( γ ) wd ) = [( x ) x ] where w f ad w d are two weightig idices for the -th promisig poit. The first idex, w f is a measure of the optimality of the poit ad w d is a measure of how far the poit is from the curret optimal poit. We have cosidered two ways for calculatig these idices: the rak method ad the roulette method. Whe the rak method is used, the obective fuctio values correspodig to the curret poits, except the curret best poit, are first sorted i descedig order. The, the rak of each poit withi the list is assiged to miimum obective fuctio value will receive a w f. Therefore, the poit with the w f of k-. The poits are agai sorted accordig to their distace from the curret best poit ad the rak of each poit withi the list is assiged to of k-. Whe Roulette method is used, w d. This time, the farthest poit from the best oe receives a w f ad w d are calculated as follows: w d w f = ( yp ) ( y ) k ( max P yp ) = max y P, for all i which y P y * (7)

11 w d = d ( dp ) ( d ) k ( dp mi P ) = P mi, for all i which y P y * (8) where the curret best poit is agai discarded from the promisig list. I the later equatio, d P represets the distace of the -th poit of the list from the curret best poit, ad d P is a vector cotaiig the distace values associated with the list. Regardless of the method used to calculate weightig idices, the collective weight values ca be obtaied by bledig these two idices usig a weightig factor γ as i Eq. (6). I fact, the poits with better obective fuctio values are preferred while at the same time choosig the poits far from the curret best solutio, allows a more distributed promisig list ad helps the algorithm to avoid premature covergeces..6 Stoppig Coditios Various stoppig coditios ca be applied to the algorithm. The algorithm may stop whe a maximum umber of evaluatios, a miimum value of the weighted variace or a maximum umber of iteratios, without ay sigificat improvemet i the obective fuctio, is reached. I this paper, we have two experimetal setups, each with its ow specific stoppig coditio. I the first setup, the algorithm stops whe the Euclidea distace betwee the best foud solutio ad the other idividuals falls below a certai threshold. I the secod setup, the algorithm stops whe a predefied level of accuracy is reached. The details of both criteria are described i subsectio Variable Correlatio Hadlig Sometimes the variables of the problem are highly correlated, which is maily due to the iheret rotatio of the fuctio ladscape with respect to the referece coordiate system. I such cases, it is required to perform samplig i differet directios accordig to the distributio of the idividuals. As metioed earlier, i this paper a temporary coordiate system, Z, is utilized to describe the pheromoe distributios while samplig ew poits. The durig ay iteratio, it is eeded to determie the orietatio of Z axes so that they coicide with the pricipal directios amog the curret populatio. Pricipal Compoet Aalysis (PCA) is a well-kow method i

12 this regard. However, Socha ad Dorigo [7,8] have stated that the use of PCA i ACO R has ot prove to be successful ad ofte leads to stagatio. Therefore they propose their ow method, which performs the similar task i a stochastic maer [7,8]. Ufortuately, the details of this method are ot reported completely. I this work, the authors first employed PCA to determie the curret pricipal coordiate system; however, like i ACO R, it did ot prove to be a efficiet approach, ad the same problem of premature covergece was experieced, as Socha ad Dorigo [7,8] reported. Therefore, a alterative stochastic method is desiged, which may have similarities with that used i [7,8]. This method cosists of the followig steps:. The origi of the coordiate system is traslated to the mea of the curret idividuals.. Oe of the preset idividuals, x, is radomly selected, with a selectio probability proportioal to x m, where x is the Euclidea orm of the -th idividual ad m is a parameter of the algorithm. The positio vector of the selected idividual is the cosidered as the first axis of the ew coordiate system, deoted by v. 3. For remaiig - idividuals, the equatio set ( ) = 0 x + α is solved for the values of α. The ew idividuals, x + α, obtaied i this way, represet the proectio of the positio vectors to the vector v. x o the plae perpedicular 4. Give the ew idividuals, x + α, the secod selectio is made with a selectio probability proportioal to x m + α. The secod axis v, which is ormal to the first axis, is obtaied i this way. 5. For remaiig - idividuals, the equatio set ( ) 0 solved for the values of β. The third axis x + α + β = is 3 v, is the selected with a selectio probability proportioal to x m + α + β. 6. This patter is repeated util - pricipal axes are obtaied. 7. The -th axis is specified usig the orthogoality coditios:

13 = = = (9) However, oe more equatio is eeded for the system of Eq. (9) to be determied. So the sum of the elemets of the -th vector is assumed to be equal to a arbitrary value s ad form the followig matrix equatio: = s 0 0 (0) Solvig Eq. (0) gives the -th axis v. The vectors v v v,,, are the ormalized to form the rotatio matrix R XZ. 3 Results ad discussio I this sectio, the parameters of the algorithm ad their suggested optimal values are itroduced i the first subsectio. A compariso of the algorithm with some other methods is preseted i the ext subsectio, ad a graphical demostratio of the algorithm makes up the last subsectio. 3. Parameter Settig TCACS has four cotrol parameters to set, the umber of ats (k), the weightig strategy which ca be either Roulette or Rak, the weightig factor (γ), ad the parameter m used i correlatio hadlig method. To fid the optimal values of these parameters, the solutios obtaied for a set of stadard test fuctios (as listed i the appedix) with differet settigs were studied. For each settig, the algorithm was applied to the test fuctios i 00 differet rus. Fially, the optimal parameter settig was obtaied as i Table, where there are give two optimal sets for the parameters of the algorithm based o the problem dimesio,.

14 3. Compariso with other methods The ew algorithm was tested over a set of stadard test fuctios, listed i the appedix. Sice i the literature the performace of the proposed algorithms has bee aalyzed i differet ways, i this paper the experimetal setup was divided ito two categories. Each category cotais the results of a set of competig algorithms (icludig TCACS), for which the performace aalysis is performed i a similar way. I the first set of experimets, TCACS is compared with six other meta-heuristics, two at based ad four o-at based methods, as listed i Table. Table 3 presets the results obtaied by these methods. Dash sigs correspod to cases for which there is o reported value available i the literature. The results for TCACS are obtaied usig 00 differet executios of the algorithm. The algorithm stops whe the Euclidea distace betwee the best foud solutio ad the other idividuals falls below a certai threshold, which is typically 0-4. For each problem, three umbers are reported: the rate of successful miimizatios i a sese defied shortly, ad the average umber of obective fuctio evaluatios ad the average error betwee the best foud solutio ad the kow global optimum computed for successful miimizatios oly. Followig the literature, a miimizatio ca be cosidered successful whe the followig iequality holds: F F ε F + ε ob aal rel aal abs () where F ob is the value of the obective fuctio correspodig to the best foud solutio ad F aal is the kow aalytical optimum. However, a problem arises here, where differet values have bee used for ε abs. I fact, i all works, ε rel is set to 0-4, while the value of ε abs i some articles is 0-6 ad i others it is 0-4. This makes it difficult to compare success rate values especially for fuctios with zero miimum values. I case of TCACS, we have used values of ε rel ad ε abs equal to 0-4 ad 0-6, respectively. Aother problem with the results reported i the literature is a icosistecy betwee the successfuless criterio metioed above ad the error values reported. For istace, whe a success rate of 00% is reported for a fuctio with a zero miimum value, oe expects the average error to be less tha or equal to ε abs ; while for

15 may of the problems, this is ot the case (eve whe assumig ε abs =0-4 ) ad the average error is of a cosiderably higher order tha ε abs (See Table 3). Despite these problems, sice for most methods both the average umber of obective fuctio evaluatios ad the average error are available i Table 3, it is still possible to compare the algorithms; although ot much credit ca be give to the success rate values. Sice each method uses its ow stoppig coditio allowig differet orders of accuracy, to make a fair compariso, oe has to simultaeously take both the umber of evaluatios ad the average error values ito accout. From Table 3 it is clear that TCACS exhibits superior results ad for most problems its performace seems to be better tha those of the competig methods. It reaches more accurate solutios with less umber of fuctio evaluatios. However, there are also areas of weakess. For example, i case of R 0, the algorithm was ot able to preset the level of accuracy required by the successfuless coditio used, so the authors have reported the average of all 00 rus regardless of the quality of the results. I this problem, it is observed that the average error of TCACS is higher tha those of the other methods. If the competig algorithms were cosistet i their defiitio of the successful miimizatio, it could also be possible to compare the robustess of TCACS with the other methods. However, as discussed above, it is clearly see by comparig the reported average error ad success rate values for competig methods that they are icosistet with the defiitio of the successful miimizatio metioed i the correspodig papers. I the secod set of experimets, TCACS is compared with CACS [5] ad ACO R [7,8]. It should be oted that the results of ACO R, reported i [7,8], are obtaied with the successfuless criterio of Eq (0) employed as its stoppig coditio, which is ot a commo practice sice all the competig methods use stoppig coditios idepedet of the quality of the curret solutio. Therefore the comparisos made i those papers seem to be ufair. This is especially true whe we ote that oly the umber of obective fuctio evaluatios is reported ad there is o idicatio of the quality of the solutios provided by each method. Furthermore, the domai of defiitio for the test fuctios reported i [7,8] are i some cases differet to those available i the curret literature. Nevertheless, i order to provide a fair compariso, TCACS was applied to the problems for which the results of ACO R were available ad with the same search domais ad stoppig coditio, i.e. the algorithm stops after Eq.

16 () is held. The values of ε abs ad ε rel were both set to 0-4 followig the values used i [7,8]. I additio, the results reported for CACS were too limited ad icomplete; therefore, sice the authors had access to the origial code for CACS, it was applied to the related problems with the same stoppig coditio ad search domai. I this case we examied differet umbers of ats equal to 0, 0, 50, ad 00. The results of the three methods are preseted i Table 4. Reported are the average (arithmetic mea) umber of obective fuctio evaluatios ad the rate of successful miimizatios displayed i paretheses. Both for TCACS ad CACS we have examied 00 idepedet rus of the algorithms. Iterestig observatios ca be made by comparig TCACS with CACS. As see, almost for all problems, the success rate values of TCACS are cosiderably higher tha those of CACS. I case of R 5, beig a highly correlated problem, CACS completely fails to fid the global miimum, while TCACS tackles it with a good performace. This ca be directly attributed to the variable correlatio hadlig scheme implemeted i TCACS. It is also observable that the 0 umber of ats reported i [5] as the optimal umber of ats, proves to be optimal also for this set of test fuctios. Thus, cosiderig the results of CACS with 0 ats, we ca coclude that almost i all test cases TCACS is faster ad more robust tha CACS. This ca be liked to the use of tabu ad promisig balls. A clear evidece is the far superior performace of TCACS i solvig Easom problem that offers a extremely uiform ladscape with a very arrow valley withi, which makes it a hard oe. The use of tabu balls blocks large portio of this uiform area ad icreases the chace of hittig the arrow valley. Comparig TCACS with ACO R, it ca be see that for almost all test fuctios used, TCACS is faster tha or at least as fast as ACO R. This is especially true for the fuctios of fewer dimesios, where TCACS reaches the desired solutio with about half of the umber of fuctio evaluatios eeded by ACO R. O the other had, it is observable that ACO R geerally shows higher success rates tha TCACS, particularly i cases of fuctios H 6,4 ad GR 0, where a cosiderable differece is see. Therefore, it ca be cocluded that TCACS is a faster algorithm tha ACO R, while the latter shows better robustess. This is also i accordace with the trade-off which always exists betwee opposig characteristics, efficiecy ad robustess of the algorithms.

17 3.3 Demostratio of the Algorithm I this sectio a graphical represetatio, o how the algorithm progresses, is demostrated. Fig. 3 shows the variatio of a -variable fuctio B as defied i appedix versus x ad x, where i this demostratio have bee limited betwee - ad. Fig. 4 represets the arragemet of the tabu balls, the promisig poits ad the curret locatios of the ats durig iteratios 3, 4, 5, ad 6. The first iterestig observatio is that as algorithm progresses, tabu ad promisig poits gather i the worst ad the best regios of the search space, respectively. Also it ca be observed how the radius of tabu balls varies betwee iteratios. Geerally, the size of tabu balls decreases with the progress of the search ad aggregatio of the ats aroud the global optimum. However, based o the arragemet of the tabu ad promisig poits, it ca also be icreased from oe iteratio to the ext. This prevets the ats to choose their destiatios withi the tabu areas ad pushes them toward the promisig poits. 4 Coclusio I this paper a ew hybrid optimizatio method, combiig Cotiuous At Coloy System (CACS) ad Tabu Search (TS) was proposed for miimizatio of cotiuous multi-miima fuctios. The basic structure of the proposed scheme, called Tabu Cotiuous At Coloy System (TCACS), is similar to CACS while it borrows the cocepts of tabu ad promisig lists from TS. The so called tabu balls prevet the ats to choose their destiatio withi the tabu regios. A ew strategy was also proposed to dyamically tue the radius of the tabu balls ad also to hadle the variable correlatios. TCACS was tested over a set of stadard test fuctios to tue its cotrol parameters ad to compare its results with those of other meta-heuristics. The overall results show that the use of tabu ad promisig lists is beeficial regardig the accuracy ad robustess of the method. A graphical demostratio o the performace of TCACS was also made. It shows that as the algorithm progresses, the members of the tabu ad promisig poits gather i the worst ad the best regios, respectively. Also it ca be observed how the radius

18 of tabu balls varies betwee iteratios. This prevets the ats to choose their destiatios withi the tabu areas ad pushes them toward the promisig poits. Refereces [] F. Glover, Tabu Search: Part I. ORSA Joural o Computig, 3: (989). [] F. Glover, Tabu Search: Part II. ORSA Joural o Computig, : 4-3 (990). [3] N. Hu, Tabu Search Method with Radom Moves for Globally Optimal Desig. Iteratioal Joural for Numerical Methods i Egieerig, 35: (99). [4] D. Cviovic ad J. Kliowski, Taboo Search: A Approach to the Multiple Miima Problem. Sciece, 667: (995). [5] R. Battiti ad G. Tecchiolli, The Cotiuous Reactive Tabu search: Bledig Combiatorial Optimizatio ad Stochastic Search for Global Optimizatio. Aals of Operatios Research, 63: (996). [6] P. Siarry ad G. Berthiau, Fittig of Tabu Search to Optimize Fuctios of Cotiuous Variables. Iteratioal Joural for Numerical Methods i Egieerig, 40: (997). [7] R. Chelouah ad P. Siarry, Ehaced Cotiuous Tabu Search: A Algorithm for the Global Optimizatio of Multimiima Fuctios. i Meta-Heuristics, Advaces ad Treds i Local Search Paradigms for Optimizatio (S. Voss, S. Martello, I. H. Osma ad C. Roucairol Eds), Kluwer Academic Publishers, 4: 49 6 (999). [8] R. Chelouah ad P. Siarry, Tabu Search Applied to Global Optimizatio. Europea Joural of Operatioal Research, 3: (000). [9] M. Dorigo, Optimizatio, Learig ad Natural Algorithms. Ph.D. thesis, Uiv. of Mila, Mila, Italy, 99. [0] A. Colori, M. Dorigo ad V. Maiezzo, Distributed Optimizatio by At Coloies. Proceedigs of the First Europea Coferece o Artificial Life, Elsevier Sciece Publisher, 34 4 (99). [] M. Dorigo, V. Maiezzo ad A. Colori, The At System: Optimizatio by a Coloy of Cooperatig Agets. IEEE Trasactios o Systems, Ma, ad Cyberetics Part B, : 9-4 (996).

19 [] T. Stutzle ad H. Hoos, The MAX MIN At System ad Local Search for the Travelig Salesma Problem. Proceedigs of IEEE Iteratioal Coferece o Evolutioary Computatio ad Evolutioary Programmig, (997). [3] M. Dorigo ad L. M. Gambardella, At Coloy System: A Cooperative Learig Approach to the Travelig Salesma Problem. IEEE Trasactios o Evolutioary Computatio, : (997). [4] L. M. Gambardella ad M. Dorigo, At-Q: A Reiforcemet Learig Approach to the Travelig Salesma Problem. Proceedigs of the Twelfth Iteratioal Coferece o Machie Learig, Palo Alto, 5 60 (995). [5] D. Costa ad A. Hertz, Ats Ca Colour Graphs. Joural of the Operatioal Research Society, 48: (997). [6] M. Dorigo, G. D. Caro ad L. M. Gambardella, At Algorithms for Discrete Optimizatio. Artificial Life, 3: 37-7 (999). [7] M. Dorigo, E. Boabeau ad G. Theraulaz, At Algorithms ad Stigmergy. Future Geeratio Computer Systems, 6: (000). [8] M. Wodrich ad G. Bilchev, Cooperative Distributed Search: The Ats Way. Cotrol ad Cyberetics, 6 (3): (997). [9] G. Bilchev ad I. C. Parmee, The At Coloy Metaphor for Searchig Cotiuous Desig Spaces. Lecture Notes i Computer Sciece, 993: 5-39 (995). [0] N. Momarché, G. Veturii ad M. Slimae, O How Pachycodyla apicalis Ats Suggest a New Search Algorithm. Future Geeratio Computer Systems, 6: (000). [] J. Dréo ad P. Siarry, Cotiuous Iteractig At Coloy Algorithm Based o Dese Heterarchy. Future Geeratio Computer Systems, 0: (004). [] J. Dréo ad P. Siarry, A New At Coloy Algorithm Usig the Heterarchical Cocept Aimed at Optimizatio of Multi-miima Cotiuous Fuctios. Lecture Notes i Computer Sciece, 463: 6- (00). [3] C. Lig, S. Jie, O. Lig ad C. Hogia, A Method for Solvig Optimizatio Problems i Cotiuous Space Usig At Coloy Algorithm. Lecture Notes i Computer Sciece, 463: (00).

20 [4] L. Y. Ju ad W. T. Ju, A Adaptive At Coloy System Algorithm for Cotiuous-Space Optimizatio Problems. Joural of Zheiag Uiversity Sciece, : (003). [5] S. H. Pourtakdoust ad H. Nobahari, A Extesio of At Coloy System to Cotiuous Optimizatio Problems. Lecture Notes i Computer Sciece, 37: (004). [6] K. Socha, ACO for Cotiuous ad Mixed-Variable Optimizatio. Lecture Notes i Computer Sciece, 37: 5-36 (004). [7] K. Socha ad M. Dorigo, At Coloy Optimizatio for Cotiuous Domais. IRIDIA Techical Report, TR/IRIDIA/ [8] K. Socha ad M. Dorigo, At Coloy Optimizatio for Cotiuous Domais. Europea Joural of Operatioal Research, 85: (008). [9] H. Nobahari ad S. H. Pourtakdoust, Optimizatio of Fuzzy Rule Bases Usig Cotiuous At Coloy System. Proceedigs of the First Iteratioal Coferece o Modelig, Simulatio ad Applied Optimizatio, Sharah, U.A.E., Paper No. 43, 005. [30] H. Nobahari ad S. H. Pourtakdoust, Optimal Fuzzy CLOS Guidace Law Desig Usig At Coloy Optimizatio. Lecture Notes i Computer Sciece, 3777: (005). [3] H. Nobahari, S. Y. Nabavi ad S. H. Pourtakdoust, Aerodyamic Shape Optimizatio of Uguided Proectiles Usig At Coloy Optimizatio. Proceedigs of ICAS 006, Hamburg, Germay, 3-8 Sept [3] R. Chelouah ad P. Siarry, A Cotiuous Geetic Algorithm Desiged for the Global Optimizatio of Multimodal Fuctios. Joural of Heuristics, 6: 9-3 (000). [33] P. Siarry, G. Berthiau, F. Durbi, ad J. Haussy, Ehaced Simulated Aealig for Globally Miimizig Fuctios of May Cotiuous Variables. ACM Trasactios o Mathematical Software, 3 (): 09 8 (997). [34] R. Chelouah ad P. Siarry, Geetic ad Nelder Mead Algorithms Hybridized for a More Accurate Global Optimizatio of Cotiuous Multimiima Fuctios. Europea Joural of Operatioal Research, 48: (003).

21 APPENDIX: LIST OF TEST FUNCTIONS Brai RCOS (RC) ( variables): 5 5 RC ( x, x ) x 6 0 cos( ) 0 ( 4 ) x π π x = + + 8π x + search domai: -5 < x < 0, 0 < x < 5; o local miimum; 3 global miima: (x,x ) * = (-π,.75),( π,.75),(9.4478,.475); RC((x,x ) * ) = B ( variables): ( 3π x ) 0.4 cos( 4π ) 0. 7 B( x, x ) = x + x 0.3cos x + search domai: -00<x <00; =, ; several local miima (exact umber uspecified i usual literature); global miimum: (x,x ) * = (0,0); B((x,x ) * ) = 0. Easom (ES) ( variables): ES( x, x ) = cos( x )cos( x )exp x π + x π ( (( ) ( ) )) search domai: -00<x <00; =, ; several local miima (exact umber uspecified i usual literature); global miimum: (x,x ) * = ( π, π); ES((x,x ) * ) = -. Goldstei ad Price (GP) ( variables): GP ( x, x) = + ( x+ x + ) *( 9 4x+ 3x 4x + 6xx + 3x) 30 + ( x 3 x) *( 8 3x+ x 48x 36xx + 7x) search domai: - < x <, =, ; 4 local miima; global miimum: (x,x ) * = (-,0); GP((x,x ) * ) = 3. Shubert (SH) ( variables): 5 5 SH ( x, x) = cos ( + ) x+ cos ( + ) x + = = search domai: -0 < x < 0, =, ; 760 local miima; 8 global miima: SH((x,x ) * ) = De Joug (DJ) (3 variables): DJ ( x, x, x3) = x + x + x3 search domai: -5.<x <5.; =, 3; sigle miimum (local ad global): (x,x,x 3 ) * = (0,0,0); DJ((x,x,x 3 ) * ) = 0. Hartma (H 3,4 ) (3 variables): 4 3 H 3,4 x = ci exp ai x i= = ( ) ( p ) i

22 search domai: 0<x <, =,,3; 4 local miima: p i =(p i, p i, p i3 ) = ith local miimum approximatio; f((p i )) -c i ; global miimum: x * = (0.,0.555,0.855); H 3,4 (x * ) = i a i c i p i Shekel (S4;) (4 variables): T T 4, ( ) = ( i) ( i) + i = ( 3 4 ) i= T i = i i i i S x x a x a c ; x x, x, x, x ; 3 4 (,,, ) a a a a a 3 fuctios S 4, were cosidered: S 4,5, S 4,7 ad S 4,0 ; search domai: 0<x <0, =,..,4; T local miima ( = 5, 7 or 0): a i = ith local miimum approximatio; S 4, ((a T i )) - /c i ; S 4,5 =5 5 miima with global miimum: S 4,5 (x * ) = S 4,7 =7 7 miima with global miimum: S 4,5 (x * ) = S 4,0 =0 0 miima with global miimum: S 4,5 (x * ) = i a i c i Hartma (H6;4/ (6 variables): 4 6 H6,4 ( x) = ciexp ai( x pi) i= = search domai: 0<x <; =,,6; 4 local miima: p i = (p i,,p i6 ) = ith local miimum approximatio; f(p i ) -c i ; global miimum: H 6,4 (x * ) = i a i c i p i Rosebrock (R) ( variables): R x = 00 x x+ + x ( ) ( ) ( ) =

23 3 fuctios were cosidered: R ; R 5 ; R 0 ; search domai: -5<x <0, =,..,, several local miima (exact umber uspecified i usual literature); global miimum: x * =(,,), R (x * ) = 0. Zakharov (Z) ( variables): 4 Z( x) = x x x = = = 3 fuctios were cosidered: Z ; Z 5 ; Z 0 ; search domai: -5<x <0, =,..,, several local miima (exact umber uspecified i usual literature); global miimum: x * = (0,,0), Z (x * ) = 0. Marti & Gaddy (MG) ( variables) ( ) ( ) = ( ) + ( + ) MG x x x x x 0 / 3 search domai: -0<x <0, =,; global miimum: x * =(5,5), MG(x * )=0. Sphere Model (SM) (6 variables) SM x ( ) x = = search domai: -5.<x <5., =,,6; global miimum: x * =(5,5), MG(x * )=0. Griewagk (Gr ) ( variables) ( ( )) 0. / 4000 cos / = i= ( ) ( ) ( ) i Gr x = + x x i + i i Gr 0 were used; search domai: -5.<x i <5., i=,,0; global miimum at origi; Gr 0 (x * )= ;

24 Table. Optimal parameter settigs for TCACS. Problem Dimesio k Weightig Strategy γ m <4 0 Rak 4 5 Roulette 0.5

25 Table. The algorithms preset i the first experimetal category. Abbr. Complete Name Type Referece TCACS Tabu Cotiuous At Coloy System Hybrid: At Coloy + Tabu Search This work CGA Cotiuous Geetic Algorithm Geetic Algorithm [3] CHA Cotiuous Hybrid Algorithm Hybrid: Geetic Algorithm Simplex Search [34] ECTS Ehaced Cotiuous Tabu Search Tabu Search [7] ESA Ehaced Simulated Aealig Simulated Aealig [33] CIAC Cotiuous Iteractig At Coloy At Coloy [] CACO Cotiuous At Coloy Algorithm At Coloy [9]

26 Table 3. Compariso of TCACS with other meta-heuristics. Reported are the average umber of fuctio evaluatios (topmost umber), the average error betwee the solutio ad the kow optimum value (middle umber), ad the rate of successful miimizatios (bottom umber i parethesis). Uavailable data have bee represeted by dash sigs. Test fuctio TCACS CGA CHA ECTS ESA CIAC CACO RC B ES GP MG SH R Z DJ H 3,4 S 4,5 S 4,7 S 4,0 R 5 Z 5 SM 6 H 6,4 R 0 Z e-7 (00%) e-9 (96%) e-0 (00%) e-5 (97%) e-008 (00%) e-6 (8%) e-8 (8%) e-8 (00%) e-0 (00%) e-6 (00%) e-6 (59%) e-5 (68%) 3.399e-5 (74%) e-9 (87%) e-7 (00%) e-007 (00%) e-6 (55%) (00%) * e-8 (97%) 60 e-4 (00%) 430 3e-4 (00%) 504 e-3 (00%) 40 e-3 (00%) 95 e-4 (00%) 3 e-7 (00%) 95 e-3 (00%) 59 e-3 (00%) (00%) (00%) 3 e-3 (00%) e-8 (00%) e- (00%) e-3 (00%) 960 4e-3 (00%) 60 3e-6 (00%) 750 e-4 (00%) 58 5e-3 (00%) (76%) (83%) (8%) (00%) 350 4e-4 (00%) 345 5e-3 (00%) 459 4e-4 (00%) 5 3e-6 (00%) 37 e-4 (00%) 49 5e-3 (00%) 698 9e-3 (85%) (85%) (85%) (00%) 950 6e-5 (00%) 370 e-3 (00%) (00%) 95 e-7 (00%) 338 3e-8 (00%) (00%) (75%) (80%) (75%) (00%) 54 4e-6 (00%) (56%) (0%) (00%) (00%) e-3 (00%) (00%) e-3 (00%) e- (54%) e- (54%) e- (50%) (00%) (80%) 699 e-6 (00%) 930 8e-3 (00%) e-3 (83%) 49 e-6 (00%) (00%) (85%) 4630 e-7 (00%) (5%) e-3 (00%) e- (00%) e- (00%) e- (00%) GR e-006 (6%) * Successfuless coditio igored (See sectio 3.) e-0 (00%) (00%) (5%) (00%)

27 Table 4. Compariso of the results obtaied by TCACS with ACO R ad CACS. Reported are the average umber of fuctio evaluatios (topmost umber) ad the rate of successful miimizatios (bottom umber i parethesis). Dash sigs represet the cases where o solutio was foud with the required accuracy. Test Fuctio TCACS ACO R CACS Na=0 CACS Na=0 CACS Na=50 CACS Na=00 RC 39 (00%) 735 (00%) 3 (00%) (00%) 437 (00%) 87 (00%) B 38 (94%) 560 (00%) 5 (79%) 358 (97%) 74 (00%) 98 (00%) ES 87 (99%) 77 (98%) 63 (90%) 765 (97%) 5777 (96%) 860 (%) GP 67 (98%) 39 (00%) 5 (83%) 98 (89%) 407 (99%) 684 (00%) MG 57 (00%) 345 (00%) 64 (00%) 3 (00%) 47 (00%) 736 (00%) R 06 (00%) 86 (00%) 74 (00%) 307 (00%) 55 (00%) 93 (00%) Z 38 (00%) 9 (00%) 4 (00%) 88 (00%) 343 (00%) 576 (00%) DJ 94 (00%) 400 (00%) 95 (00%) 94 (00%) 573 (00%) 003 (00%) H 3,4 59 (00%) 34 (00%) 36 (94%) 07 (00%) 404 (00%) 695 (00%) S 4,5 768 (63%) 793 (57%) 43 (35%) 84 (3%) 367 (34%) 574 (45%) S 4,7 684 (74%) 748 (79%) 44 (43%) 856 (47%) 9 (60%) 490 (68%) S 4,0 738 (75%) 75 (8%) 47 (%) 79 (3%) 37 (49%) 4898 (6%) R (9%) 487 (97%) - (0%) - (0%) - (0%) - (0%) Z (00%) 77 (00%) 86 (00%) 80 (00%) 437 (00%) 49 (00%) SM (00%) 78 (00%) 540 (00%) 767 (00%) 568 (00%) 80 (00%) H 6,4 6 (7%) 7 (00%) 405 (6%) 568 (64%) 59 (5%) 47 (58%) GR (37%) 390 (6%) 908 (7%) 49 (7%) 575 (5%) 4890 (8%)

28 Iitializatio Move ats to ew Locatios Update tabu list, promisig list ad the best solutio foud Update Pheromoe Distributio Stop YES Stoppig Coditio reached? NO Fig. Geeral Flowchart of TCACS.

29 procedure TCACS() iitialize() while (termiatio coditio ot satisfied) if (iteratio_umber=) uiformly select iitial positio of ats else move_ats_to_ew_locatios() update tabu list ed if update promisig list update tabu ball size update coordiates trasformatio matrix update_pheromoe_distributio() ed while ed procedure procedure iitialize() set the parameters of algorithm set tabu ad promisig lists to empty tables set curret coordiates system to the origial axes ed procedure procedure move_ats_to_ew_locatios() for i=, k repeat for =, choose the ew x for the i-th at usig a ormal radom geerator ed for util (x is ot icluded i a tabu ball) ed for ed procedure procedure update_pheromoe_distributio() update the globally best poit for =, update the value of σ ed for ed procedure Fig. Pseudo-code of TCACS.

30 4 B (X,X ) X X Fig. 3. Graphical represetatio of fuctio B.

31 Iteratio 3 Iteratio Iteratio 5 Iteratio 6 Fig. 4. A demostratio for fuctio B o how the tabu (O) ad promisig (Δ) poits ad the curret (+) solutios are updated as TCACS progresses.