Variance as a Stopping Criterion for Genetic Algorithms with Elitist Model

Transcription

1 Fudameta Iformaticae 120 (2012) DOI /FI IOS Press Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model Diabadhu Bhadari, C. A. Murthy, Sakar K. Pal Ceter for Soft Computig Research Idia Statistical Istitute Kolkata , Idia diabadhu.bhadari@gmail.com, murthy@isical.ac.i, sakar@isical.ac.i Abstract. Geetic Algorithm (GA) has ow become oe of the leadig mechaisms i providig solutio to complex optimizatio problems. Although widely used, there are very few theoretical guidelies for determiig whe to stop the algorithm. This article establishes theoretically that the variace of the best fitess values obtaied i the iteratios ca be cosidered as a measure to decide the termiatio criterio of a GA with elitist model (EGA). The criterio automatically takes ito accout the iheret characteristics of the objective fuctio. Implemetatio issues of the proposed stoppig criterio are explaied. Its differece with some other stoppig criteria is also critically aalyzed. Keywords: Geetic Algorithm with Elitist Model, Stoppig Criterio, Markov Chai, Variace 1. Itroductio Geetic Algorithms (GAs) are stochastic search methods based o the priciples of atural geetic systems. They perform a multidimesioal search i providig a optimal solutio for evaluatio (fitess) fuctio of a optimizatio problem. Ulike the covetioal search methods, GAs deal simultaeously with multiple solutios ad use oly the fitess fuctio values. Populatio members are represeted by strigs, correspodig to chromosomes. Search starts with a populatio of radomly selected strigs, ad, from these, the ext geeratio is created by usig geetic operators. At each Address for correspodece: Ceter for Soft Computig Research, Idia Statistical Istitute, Kolkata , Idia

2 146 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model iteratio idividual strigs are evaluated with respect to a performace criteria ad assiged a fitess value. Strigs are radomly selected usig these fitess values to either survive or to mate to produce offsprig for the ext geeratio. All such strigs are subject to mutatio. GAs are theoretically ad empirically foud to provide global ear optimal solutios of various complex optimizatio problems i the fields of operatios research, VLSI desig, patter recogitio, image processig, machie learig etc. [6, 11, 16, 15, 17, 9, 12, 10]. Amog the may global search methods available, GA has bee cosidered to be a viable ad promisig optio for exploratio. This evolutioary techique is populatio orieted, successive populatios of feasible solutios are geerated i stochastic maer followig laws of atural selectio. I this approach, multiple stochastic trajectories approach simultaeously towards oe or more regios of the search space providig importat clues about the global structure of the fuctio. Various theoretical aspects of geetic algorithms have bee studied i the literature. Researchers have tried to establish the theoretical basis of the use of simple (yet difficult to model) operatios. Attempts have bee made to fid the GAs amazig search ability by aalyzig the evolutio of strigs geerated by the crossover ad mutatio operatios. Geetic algorithms have bee successfully modeled as Markov chais [9, 2, 3, 18, 14, 21, 20]. Vose [21], ad Davis ad Pricipe [3] have ot preserved the kowledge of the previous best i their model. Bhadari et. al. [2], Rudolph [18] ad Suzuki [20] preserved the kowledge of the previous best i their model ad proved the covergece of GAs to the optimal strig. I [13], Murthy et. al. have tried to provide the optimal stoppig time for a GA i terms of ɛ-optimal stoppig time. They have provided a estimate for the umber of iteratios that a GA has to ru to obtai a ɛ-optimal global solutio. All of the steps of a GA are well defied except the stoppig criterio. So far the practitioers use stoppig criteria based o time or fitess value. Time based stoppig criteria are maily of two kids. The popular oe is to decide upfrot the umber of iteratios to be executed. Aother is based o executio time of the algorithm. Algorithm is ru for a predetermied period ad gets the result. Though these criteria are very simple to implemet, determiig the time is agai a challege. I the first, the process is executed for a fixed umber of iteratios ad the best strig, obtaied so far, is take to be the optimal oe. While i the other, the algorithm is termiated if o further improvemet i the fitess value for the best strig is observed for a fixed umber of iteratios. However, it is ot clear how to fix the umber of iteratios required for the executio of a GA. I this article, we propose a ew stoppig criterio based o the variace of the best fitess values obtaied over the geeratios. The proposed criterio uses oly the fitess fuctio values ad takes ito accout the iheret properties of the objective fuctio. User does ot eed to study the characteristics of the objective fuctio ad the geetic parameters to be used i the algorithm. The criterio based o the variace of the fitess fuctio values obtaied over the geeratios oly eeds a sufficietly small values of the boud. It has bee show theoretically that the variace teds to zero whe the umber of geeratios teds to ifiity with probability of obtaiig the global optimal solutio teds to uity.

3 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 147 The basic priciples of geetic algorithms ad a descriptio of GAs with elitist model (EGAs) are provided i the ext sectio. The mathematical modelig of EGAs ad the issues of covergece are discussed i Sectios 3. Sectios 4 ad 5 deal with the optimal stoppig time of EGAs. Sectio 6 presets experimetal results demostratig the effectiveess of the proposed criterio for a umber of objective fuctios. A detailed comparative study amog differet stoppig criteria is provided i Sectio 7. Sectio 8 cocludes the article. 2. Basic Priciples of Geetic Algorithms We describe the basic priciple of GAs i this sectio, cosiderig a problem of maximizig a fuctio f(x), x D where D is a fiite set. The problem here is to fid x such that f(x ) f(x); x D. Note that D is a discrete domai sice it is fiite. While solvig a optimizatio problem usig GAs, each solutio is coded as a strig (called chromosome ) of fiite legth (say, L). Each strig or chromosome is cosidered as a idividual. A collectio of M (fiite) such idividuals is called a populatio. GAs start with a radomly geerated populatio. I each iteratio, a ew populatio of same size is geerated from the curret populatio usig three basic operatios o the idividuals of the populatio. The operators are (i) Reproductio/Selectio, (ii) Crossover ad (iii) Mutatio. To use Geetic algorithms i searchig the global optimal solutio, the very first step is to defie a mechaism to represet the solutios i a chromosomal form. The pioeerig work of Hollad proposed to represet a solutio as a strig of legth L over a fiite set of alphabet A. Each strig S correspods to a value x D. The GA with A = {0, 1} is termed as biary coded geetic algorithm (BCGA) or simple geetic algorithm (SGA). The strig represetatio limits the algorithm to search a fiite (though users ca achieve their required approximatio by icreasig the strig legth) domai ad provides the best solutio amog the 2 L possible optios. To take ito accout the cotiuous domai, real valued strigs are cosidered as the chromosomal represetatio by suitably maipulatig the geetic operators ad is termed as real coded geetic algorithm (RCGA). However, it is agai quite difficult to cosider all the real values cosiderig the limitatio of the computer i storig irratioal values. Heceforth, throughout this article, we shall take A = {0, 1} as this paper deals with the SGAs ad ca be easily exteded to GAs defied over a fiite set of Alphabet or over RCGAs. Geerally a radom sample of size M is draw from 2 L possible strigs to geerate a iitial populatio. GAs leverage a populatio of solutios to geerate a ew populatio with the expectatio that the ew populatio will provide better solutio i terms of the fitess values. I every iteratio, we evaluate each chromosome of the populatio usig fitess fuctio f it. Evaluatio or fitess fuctio fit for a strig S is equivalet to the fuctio f: fit(s) = f(x) where, S correspods to x. Without loss of geerality, let us assume that fit(s) > 0 for all S i S where, S is the set of all possible strigs. Selectio is a process i which idividual strigs of the curret populatio are copied ito a matig pool with respect to the empirical probability distributio based o their fitess fuctio values.

4 148 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model Crossover exchages iformatio betwee two potetial strigs ad geerates two offsprig for the ext populatio. Mutatio is a occasioal radom alteratio of a character. Mutatio itroduces some extra variability ito the populatio. Though it is performed usually with very low probability q(> 0), it has a importat role i the exploratio process. Every character i each chromosome (geerated after crossover) has a equal chace to udergo mutatio. Note that, ay strig ca be geerated from ay give strig by mutatio operatio [13]. The mutatio probability q is take to be i the rage of (0, 0.5]. It may be due to the fact that, ituitively, the probability of mutatig i bit positios is more tha that of mutatig i + 1 bit positios, i.e., q i (1 q) L i > q i+1 (1 q) L i 1 i = 0, 1, 2,..., L 1 which results i q 0.5. Hece the miimum probability of obtaiig ay strig from ay give strig is q L, that is, mutatio eeds to be performed at every character positio of the give strig. The kowledge about the best strig obtaied so far is preserved either (i) i a separate locatio outside the populatio or (ii) withi the populatio; i that way the algorithm would report the best value foud, amog all possible coded solutios obtaied durig the whole process. GAs with this strategy of retaiig the kowledge of the best strig obtaied so far as geetic algorithms with elitist model or EGAs. The ew populatio obtaied after selectio, crossover ad mutatio is the used to geerate aother populatio. Note that the umber of possible populatios is always fiite sice M is fiite. This paper deals with the GAs with the elitist model (EGA) of selectio of De Jog [4], where the best strig obtaied i the previous iteratio is copied ito the curret populatio if the fitess fuctio values of all strigs are less tha the previous best. Note that the values for the parameters L, M, p ad q have to be chose properly before performig those operatios. The populatio size M is take as a eve iteger so that strigs ca be paired for crossover. The probability (p) of performig crossover operatio is take to be ay value betwee 0.0 ad 1.0. Usually i GAs, p is assumed to be a value i the iterval [0.25, 1] [6]. The mutatio probability q is take to be very low [0.001, 0.01] [6], however, it ca be take i the iterval (0, 0.5]. Mutatio plays a importat role i the covergece of GAs to the global optimal solutio. The followig sectio presets the approach Bhadari et. al. [2] provided to prove the covergece of GAs as that is the fudametal buildig block i proposig the variace of the best fitess value obtaied so far as the stoppig criterio of a GA. 3. Covergece of Geetic Algorithms Various theoretical aspects of geetic algorithms have bee studied i the literature. Researchers have tried to establish the theoretical basis of the use of simple (yet difficult to model) operatios. Attempts have bee made to fid the GAs amazig search ability by aalyzig the evolutio of strigs geerated by the crossover ad mutatio operatios. Geetic algorithms have bee successfully modeled as Markov chais [2, 3, 18, 14, 21, 20]. Vose [21], ad Davis ad Pricipe [3] have ot preserved the kowledge of the previous best i their model. Bhadari et. al. [2], Rudolph [18] ad Suzuki [20] preserved the kowledge of the previous best i their models ad proved the covergece of GAs to the optimal strig. A extesive theoretical study regardig the dyamics of evolutioary algorithms may be foud i [9]. Geetic algorithms search over a space S of 2 L strigs ad evetually provide the best with respect to the fitess fuctio fit. The strigs ca be classified ito a set of s classes depedig o their fitess

5 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 149 fuctio values. The classes are defied as S i = {S : fit(s) = F i } where F i deotes the ith highest fitess fuctio value. Thus, F 1 > F 2 > > F s. Let us also assume without loss of geerality that F s > 0. A populatio Q is a multi-set of M strigs of legth L geerated over a fiite alphabet A ad is defied as follows: Q = {S 1, S 1,, (r 1 times), S 2, S 2,, (r 2 times),, S m, S m,, (r m times) where, S i S; S i1 S i2 i 1 i 2 ad r i 1 fori = 1, 2,, m; m = M}. Let Q deote the set of all populatios of size M. The umber of populatios or states i the Markov chai is fiite. The fitess fuctio value fit(q) of a populatio is defied as fit(q) = max S Q fit(s). The the populatios are partitioed ito s sets. E i = {Q : Q Q ad fit(q) = F i } is a set of populatios havig the same fitess fuctio value F i. I a iteratio, the geetic operators (selectio, crossover ad mutatio) create a populatio Q kl E k ; l = 1, 2,, e k ad k = 1, 2,, s; from some Q ij E i where e k is the umber of elemets i E k. The geeratio of a populatio Q kl from Q ij is cosidered as a trasitio from Q ij to Q kl ad let p ij.kl deotes this trasitio probability. The the probability of trasitio from Q ij to ay populatio i E k ca be calculated as p ij.k = e k l=1 p ij.kl ; j = 1, 2,, e i ; k = 1, 2,, s. For all j = 1, 2,, e i ad i = 1, 2,, s oe obtais p ij.k > 0 if k i = 0 otherwise by costructio. This meas that oce GAs reach a populatio Q E k they will always be i some populatio Q E k for k i. I particular, oce GAs reach a populatio Q E 1 they will ever go out of E 1. Let p () ij.kl be the probability that GA results i Q kl at the th step give that the iitial state is Q ij. Let p () ij.k deote the probability of reachig oe of the populatios i E k from Q ij at the th step. The p () ij.k = e k To show the evetual covergece of a GA with elitist model to a global optimal solutio the followig theorem has bee proved i [2] ad is ot preseted here. l=1 p () ij.kl. Theorem 1. For a EGA with the probability of mutatio q [0, 1 2 ], lim p() ij.k = 0 for 2 k s; j = 1, 2,, e i ad i = 1, 2,, s. Hece lim p() ij.1 = 1 j = 1, 2,, e i ad i = 1, 2,, s.

6 150 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 4. Stoppig Criteria Proof of covergece of a algorithm to a optimal solutio is very importat as it assures the optimal solutio i ifiite iteratios. Whe a algorithm does ot assure or guaratee the optimal solutio eve after ruig for ifiite iteratios, it s utility is i doubt. Oce the covergece of a algorithm is assured, the focus turs ito the exploratio of stoppig time or stoppig criterio of the algorithm. Today, the biggest challege i the implemetatio of GAs is to decide whe to stop the algorithm keepig i mid that there is o a-prior iformatio regardig the objective fuctio. Attempts have bee made to provide stoppig criteria of a GA based o time the algorithm is beig executed ad obtaied objective fuctio values or their distributio [11, 8, 12]. Time based stoppig criteria are maily of two kids. The popular oe is to decide upfrot the umber of iteratios to be executed. Aother is based o executio time of the algorithm. Algorithm is ru for a predetermied period ad gets the result. Though these criteria are very simple to implemet but determiig the time is agai a challege. This would require a good kowledge about the global optimal solutio, which is ot always available. I the first, the process is executed for a fixed umber of iteratios ad the best strig, obtaied so far, is take to be the optimal oe. While i the other, the algorithm is termiated if o further improvemet i the fitess value for the best strig is observed for a fixed umber of iteratios. Though these criteria are easy to implemet, they do ot guaratee the covergece of the GAs to the global optimal solutio as they are termiated after a fiite umber of iteratios. The criteria based o objective fuctio values use the uderlyig fitess fuctio values to calculate auxiliary values as a measure of the state of the covergece of the GA. Subsequetly, the ruig mea, stadard deviatio of the populatio uder cosideratio, Best Worst, Phi, Kappa were defied as a covergece measure. I [8], Jai et. al. have tried to provide a cluster based stoppig criteria called ClusTerm. This cocept takes ito accout the iformatio about the objective values as well as the spatial distributio of idividuals i the search space i order to termiate a GA. Safe et. al. [19] preseted a critical aalysis of various aspects associated with the specificatio of termiatio coditios for simple geetic algorithms. The study, which is based o the use of Markov chais, idetifies the mai difficulties that arise whe oe wishes to set meaigful upper bouds for the umber of iteratios required to guaratee the covergece of such algorithms with a give cofidece level. Greehalgh et. al. [7] have discussed covergece properties based o the effect of mutatio ad also obtaied a upper boud for the umber of iteratios ecessary to esure the covergece. I [1], authors have tried to provide a stoppig criteria together with a estimatio for the stoppig time. I [13], Murthy et. al have provided ɛ-optimal stoppig criteria for GAs with elitist model ad also derived the ɛ-optimal stoppig time. Pessimistic ad optimistic stoppig times were derived with respect to the mutatio probability. However, it is to be oted that with existig operatios, whatever value for the umber of iteratios N is decided, there is always a positive probability of ot obtaiig the global optimal solutio at that stage ad is stated below as a lemma. Lemma 1. With existig operatios, whatever value for the umber of iteratios N is decided, there is always a positive probability of ot obtaiig a optimal solutio. Proof: I geeral, the umber of solutios i the solutio space Ω is much higher compare to the umber of optimal solutios. Moreover, the populatio size cosidered i the implemetatio of geetic algorithms

7 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 151 is very low compare to that of solutio space. Therefore, it is clear that the probability of startig with a populatio (Q) cotaiig o optimal solutio is positive. Let P (1) Q.Q be the probability of obtaiig the same populatio i oe iteratio. It is obvious that P (1) Q.Q > 0 as the probability of mutatio δ is assumed to be < 1. Now, P (2) Q.Q = P (1) Q.Q.P (1) Q.Q + P (1) (1) Q.Q.P Q.Q Q Q (1).P Q.Q > P (1) Q.Q > 0. Similarly, P () Q.Q, the probability remai i Q i iteratio, is > 0. The theoretical study brigs out a umber of limitatios i defiig a stoppig criterio. Some desirable properties of a good stoppig criterio are give below. Easy to implemet. Able to provide stoppig time automatically for ay fitess fuctio. Should lead to good / satisfactory result. Total umber of strigs searched should ot exceed 2 L. However, the fudametal task of a stoppig criterio is to provide a guidelie i termiatig the algorithm so that the solutio obtaied at that time ad the optimal solutio are close to each other. I other words, the stoppig criterio provides the user a guidelie i stoppig the algorithm with a acceptable solutio close to the optimal solutio. Mathematically, the closeess may be judged i various ways. Oe way is to show that the probability of ot reachig optimal at that time is less tha a small predefied quatity. Aother way is to measure the distace betwee the optimal ad the curret solutio, ad show that the distace is less tha a small predefied quatity. If stoppig criterio is ot mathematically (i.e., heuristically) defied, the amout of error (the value of probability i first case ad distace i the secod case) i acceptig the solutio would ot be kow. I this cotext, a ew stoppig criterio based o the variace of the best solutios obtaied up to the iteratio i had is defied here. This is easily implemetable ad does ot eed ay auxiliary iformatio other tha the fitess fuctio values obtaied so far. The theoretical study give below reveals its properties ad stregths i searchig for the global optimal solutio. 5. Proposed Stoppig Criterio Let a i be the best fitess fuctio value obtaied at the ed of ith iteratio of a EGA. The, a 1 a 2 a 3 a F 1, as F 1 is the global optimal value of the fitess fuctio (defied i sectio 3). Let a = 1 a i be the average of the a i s ad a 2 = 1 a 2 i be the average of the a 2 i s up to the th iteratio, the variace of the best fitess values obtaied up to the th iteratio, defied by b, is b = 1 (a i a ) 2 = 1 a 2 i a 2 = a 2 2 a

8 152 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model b ca be used as a stoppig criterio for a GA. A GA is stopped or termiated after N iteratios whe b N < ɛ, where ɛ(> 0) is a user defied small quatity. Give below are the basic steps of the geetic algorithm with elitist model where variace of the best solutios obtaied i the geeratios is cosidered as a stoppig criterio. 1. A populatio of radom solutios is created ad a ɛ is defied. 2. Each solutio is evaluated o the basis of the fitess fuctio. 3. Store the best solutio if it is better tha previous best. 4. Calculate the variace of the best solutios obtaied so far. 5. If the variace is greater tha the predefied value (ɛ), go to the ext step, else stop the algorithm 6. A ew geeratio of solutios is created from the old geeratio usig selectio, crossover ad mutatio. 7. Steps 2 to 6 above are repeated util the coditio i step 5 is satisfied. Now, we will theoretically establish that whe the umber of geeratios teds to ifiity, the probability of obtaiig the global optimal solutio teds to 1, ad the variace of the best fit solutios obtaied i the geeratios approaches to 0. I Sectio 3, we have discussed the covergece of GAs with elitist model. I the covergece theorem, we have see that lim p() ij.1 = 1 j = 1, 2,..., e i ad i = 1, 2,..., s. The covergece theorem i tur implies that the probability of obtaiig a global optimal solutio (F 1 )is 1 as umber of iteratios goes to ifiity ca be stated as the followig lemma. Lemma 2: For each ɛ 1 > 0, lim P rob( a F 1 > ɛ 1 ) = 0. I other words, for each ɛ 0 > 0 ad ɛ 1 > 0, there exists N 0 such that for > N 0, Proof: Trivial. 1 P rob( a F 1 ɛ 1 ) < ɛ 0 or P rob( a F 1 ɛ 1 ) > 1 ɛ 0 for > N 0 (1) With the help of the above lemma, we shall ow show that the variace of the best solutios obtaied i the geeratios approaches to 0 whe umber of iteratios teds to. Theorem 2: P rob( 1 (a i a ) 2 ɛ) 1 as for each ɛ > 0.

9 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 153 Proof: Now for > N 0, 1 (a i a ) 2 = 1 = 1 1 [(a i F 1 ) ( a F 1 )] 2 (a i F 1 ) 2 ( a F 1 ) 2 (a i F 1 ) 2 (2) 1 (a i F 1 ) 2 = 1 N 0 (a i F 1 ) i=n 0 +1 (a i F 1 ) 2 Sice F s is the miimum value of the fuctio f(x) (defied i sectio 3), we have, 1 N 0 (a i F 1 ) 2 1 N 0 = N 0 Oe ca always fid a N 1 (> N 0 ) such that for each ɛ 2 (> 0), Therefore, for > N 1 > N 0 (F s F 1 ) 2 (as F s a i F 1 i) (F s F 1 ) 2 (3) N 0 N 1 (F s F 1 ) 2 < ɛ 2 (4) 1 N 0 As a 1 a 2 a 3 a i a i+1 F 1, 1 i=n 0 +1 (a i F 1 ) 2 N 0 (F s F 1 ) 2 (a i F 1 ) 2 1 N 0 N 1 (F s F 1 ) 2 ɛ 2 from (4) i=n 0 +1 (a N0 +1 F 1 ) 2 = N 0 1 (a N0 +1 F 1 ) 2 (a N0 +1 F 1 ) 2 (5) (6) From (1), we have for > N 0, Therefore, P rob( a F 1 ɛ 1 ) > 1 ɛ 0 P rob((a N0 +1 F 1 ) 2 ɛ 2 1 ) > 1 ɛ 0 P rob((a N0 +1 F 1 ) 2 ɛ 1 ) > 1 ɛ 0, as ɛ 1 << 1. (7)

10 154 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model Now, for each ɛ = ɛ 1 + ɛ 2, P rob( 1 (a i F 1 ) 2 ɛ) = P rob( 1 N 0 (a i F 1 ) (a i F 1 ) 2 ɛ) i=n 0 +1 P rob(ɛ (a i F 1 ) 2 ɛ) (from 5) = P rob( 1 i=n 0 +1 i=n 0 +1 (a i F 1 ) 2 ɛ ɛ 2 ) > 1 ɛ 0, (from 7), where ɛ 1 = ɛ ɛ 2. (8) Therefore, we ca coclude that for each ɛ 0 > 0, there exists a N 1 such that for > N 1 P rob( 1 (a i F 1 ) 2 < ɛ) > 1 ɛ 0 I other words, P rob( 1 (a i F 1 ) 2 ɛ) 1 as for each ɛ > 0. This completes the proof of theorem 2. The followig remarks ca be made regardig the proposed algorithm with variace as stoppig criterio. Remarks: 1. The variace is calculated from the fitess values obtaied over the geeratios, which implicitly takes ito accout the characteristics of the objective fuctio. 2. ɛ sigifies a measure of error, the differece betwee the fitess value of the best solutio obtaied so far ad the global optimal solutio Implemetatio Details Some of the saliet features of the stoppig criterio are discussed below regardig its implemetatio. The user eeds to choose oly the boud for variace for implemetig this criterio. Naturally, less the value of the boud, more is the chace of obtaiig a solutio close to global optima. A user ca decide the value of ɛ based o the accuracy required. Due to the radomess ivolved i the algorithm, it may so happe that there may ot be ay chage (improvemet) i the best fitess value over a umber of cosecutive iteratios which will result i 0 variace. It is importat to select a sigificat umber of iteratios from which the fitess values will be cosidered i calculatig the variace so that the algorithm gets eough opportuity to yield improved (better) solutio.

11 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 155 Variace ca be iteratively calculated. Variace b +1 at the ed of ( + 1)th iteratio ca be calculated as follows: or b +1 = 1 +1 (a i a +1 ) 2 = a 2 i 2 a +1 b +1 = (( a 2 + a 2 +1) (a + a +1 ) 2 ) (9) This idicates that oly the average of the fitess fuctio values ad their squares of the previous iteratios are required to evaluate the variace for the + 1th geeratio. This iterative feature of variace calculatio ideed makes the algorithm easier to implemet. The proposed variace based criterio is that it is ot scale ivariat. That meas it is sesitive to trasformatios of the fitess fuctio. The algorithm may eed differet umber of iteratios for f(x) ad g(x) = k f(x), where k is a costat. Sometimes, scale ivariace is a desirable property but ot always. There are several measures that are ot scale ivariat, e.g., mea, variace, co-variace, momets. However, oe ca easily avoid the impact of the scalig effect by a simple trasformatio of the fitess fuctio. Oe such trasformatio is give i equatio (10). g(x) = f(x) f (1) max (10) where, f (1) max is the maximum value of the fitess fuctio obtaied i the first iteratio. Let us ow try to uderstad the impact of the above metioed trasformatio i the selectio of the value of ɛ. We have, b = 1 (a i a ) 2 (11) Now, let b (g) be the variace of the best fitess values obtaied up to the th iteratio for the fuctio g(x). The, it is clear that b (g) = 1 (a i a ) 2 2 (12) f (1) max It is ow obvious that the user who assumed ɛ f as the value of ɛ for the fuctio f ca assume ɛ f f (1) max2 as the value of ɛ for the fuctio g. This simply implies that the user has to adjust the value of ɛ for the applied trasformatio. It may be coveiet for the user to select the value of ɛ after the trasformatio. Therefore, the user eeds to kow his/her desire of makig the stoppig criterio scale ivariat based o the coveiece for selectig the value of ɛ. The experimetal results are beig discussed i the followig sectio.

12 156 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 6. Experimetal Results The effectiveess of the proposed stoppig criterio is demostrated i searchig for global optimal solutios of some complex fuctios of multiple variables. A umber of typical objective fuctios (give below) are used i the experimet [8]. f 2 (x) = log(1 + f 4 (x) = f 3 (x) = 11 + f 1 (x) = 6 + si(x) whe 0 x 2π = 6 + 2si(x) whe 2π < x 4π = 6 + 3si(x) whe 4π < x 6π = 6 + 4si(x) whe 6π < x 8π = 6 + 5si(x) whe 8π < x 10π = 6 + si(x) whe 10π < x 32 5 [x i ] x i ), where [x] is the itegral part of x 1, where [x] is the itegral part of x 5 [x i ] [x i ] si(, where [x] is the largest iteger x [x i ] ) The pictorial presetatios of the fuctios are show i fig 1 (a-d). f 1 is a uivariate fuctio while the remaiig 3 are multi-variate (umber of variable is cosidered as 5 here). Fuctios f 2 ad f 3 are multimodal with symmetrical distributed plateaus of idetical size ad havig multiple global maxima. f 1 ad f 4 are uimodal with spatially distat local maxima ad sigle global maxima. Differet search spaces are cosidered for differet fuctios to exploit the typical features of the fuctios. As metioed earlier, there may be o chage i the fitess fuctio value for a umber of cosecutive iteratios. Therefore, the variace would become 0 ad may result i premature termiatio of the algorithm. The user should cosider a sigificat umber of iteratios i calculatig the variace. I our experimet the miimum umber of iteratios cosidered to geerate variace are 50 for f 1 ad 200 for other fuctios. The geetic parameters used i the executio of the algorithm are as follows: Populatio size = 10 for f 1 ad 50 for others Strig legth = 20 for f 1 ad 100 for others Crossover probability = 0.8 Mutatio probability = varyig from 0.2 to 0.45 To obtai statistical sigificat results, oe test ru comprises 100 rus for a particular ɛ value for each fuctio. Differet seeds are beig supplied to brig i the radomess i geeratig iitial populatios

13 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model (a) (b) (c) (d) Figure Pictorial presetatio of the fuctios ad performig other geetic operatios. Cosiderig the importace of mutatio i the covergece process of the GAs, the mutatio probability is made variable. It is cosidered as high as 0.45 i the iitial iteratios ad beig mootoically reduced to 0.1 towards the fial iteratios. Fig. 2 depicts the tred of the variace with the iteratio umber. It is clearly o-decreasig. As the algorithm explores better solutio with higher fitess value the variace icreases. Table 1 depicts the average umber of iteratios required i order to coverge the algorithm for a give. The results shows that for a low value of, the algorithm produces satisfactory performace for all the fuctios. I fact, the algorithm produces global optimal solutio i most of the cases for = Note that the umber of iteratios to attai the give boud differs for differet fuctios depedig o the characteristics of the fuctio. Also the percetage of covergece to the global optimum solutio for f3 is much higher whereas it is lower for f4 (i fact, with values > 10 4 of, o ru could produce the global optimal solutio). This is due to the fact that the presece of multiple global optima of f3 results i faster covergece while the sigle optima of f4 is hard to attai. This clearly demostrates the effectiveess of the criterio to take ito accout the iheret properties of the objective uctio. I some cases though the stoppig criterio is satisfied, the algorithm does ot coverge to the global optimal value of the objective fuctio. This is i lie with the fact that GAs do ot guaratee the global optimal solutio i a fiite umber of iteratios. However, with the reductio i value the chace of obtaiig the global optimal solutio icreases.

14 158 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model Figure 2. Chage i variace with iteratio 7. Differeces with Other Stoppig Criteria Three widely used stoppig criteria are (i) the variace of the fitess evolved i the whole populatio is less tha a predefied small quatity (ɛ p, say), (ii) umber of iteratios is greater tha or equal to a fixed umber (N), decided a-priori, ad (iii) o improvemet i the best fitess value through a fixed umber of iteratios (say K). Let us refer them as POP-Var, AN-it ad K-it respectively i the successive discussio Pop-var I this framework, the algorithm stops whe the variace of fitess values of all the strigs i the curret populatio is less tha a predefied threshold (say, ɛ p ). Usually, ɛ p is take to be a small value, close to 0. It is primarily based o the assumptio that after sigificatly may iteratios the fitess values of the strigs preset i the populatio are all close to each other, thereby makig the variace of the fitess values close to 0. I geeral, this assumptio is ot true due to the followig reasos: (i) Usually i elitist model oly the best strig is preserved, (ii) ay populatio cotaiig a optimal strig is sufficiet for the covergece of the algorithm ad (iii) there is a positive probability of obtaiig a populatio after ifiitely may iteratios with exactly oe optimal strig ad others are beig ot optimal. This is further illustrated usig a example take from [2]. I this example, a maximizatio problem is cosidered i the domai D = {0, 1, 2, 3}. We have cosidered M = 2, L = 2 ad A = {0, 1}. The best strig of the previous populatio is copied ito the curret oe if the fitess values of all offsprig are less tha the previous best. The strigs represetig x are s 1 = 11, s 2 = 10, s 3 = 01 ad s 4 = 00. The fitess fuctio values are take to be fit(s 1 ) = 1fit(s 2 ) = 4fit(s 3 ) = 2 adfit(s 4 ) = 3. The strigs ca be classified ito four classes which are S 1 = {s 2 }, S 2 = {s 4 }, S 3 = {s 3 } ad S 4 = {s 1 }.

15 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 159 Fuctio ɛ Average umber % of cases whe Number of strigs Number of of Iteratios global solutio is reached i search space strigs searched f , , , , 072 f , , , 186, , 014, 741 f , , , 808, , 999, 049 f , , , , 620, 305 Table 1. Average iteratios for various ɛ ( 2 2 ) The umber of populatios or states is ( ) = 10 ad they are 2 Q 1 = {10, 10}, Q 2 = {10, 00}, Q 3 = {10, 01}, Q 4 = {10, 11} Q 5 = {00, 00}, Q 6 = {00, 01}, Q 7 = {00, 11}, Q 8 = {01, 01}, Q 9 = {01, 11}, Q 10 = {11, 11}. The partitio over the populatios is give below. E 1 = {Q 1, Q 2, Q 3, Q 4 }, E 2 = {Q 5, Q 6, Q 7 }, E 3 = {Q 8, Q 9 }, E 4 = {Q 10 }.. We are represetig here the trasitio probabilities as p i.j where i, j = 1, 2,, 10 for coveiece. The -step trasitio probability matrices for = 1 ad 1024, are give below for p = 0.5 ad q = P (1) =

16 160 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model P (1024) = The figures i the above matrices are give upto 6 decimal places. It ca be easily see from the matrix P (1024) that p (1024) i.j are early zero for all j 5 ad for all i = 1, 2,, 10. It ca also be see that all the rows are almost idetical i P (1024). From the literature o Markov chais [5], it is ecessary that all the rows of P () are idetical for sufficietly large, i.e., p () i,j are idepedet of i for sufficietly large. This fact esures the covergece of the GA for the cosidered model. The POP-Var criterio uses the variace of the populatio as the basis i the termiatio of the algorithm. This criterio will be effective whe the populatio becomes homogeeous (havig the same or almost same fitess values of all the strigs). For the give example, the algorithm obtais Q 1, Q 5, Q 8 or Q 10 ( Q 1 beig the most desired populatio) so that the algorithm coverges to the global optimal solutio satisfyig the POP-Var criterio. It is clear from P (1024) that the probability of obtaiig Q 1 after 1024 iteratios is maximum but there is a sigificat chace i obtaiig Q 2, Q 3 or Q 4 after those may iteratios. The elemets of secod, third ad fourth colums of P () are positive (> 0) eve after > Therefore, the algorithm does ot guaratee the maiteace of the same populatio over the iteratios. Moreover, there is a high chace of premature termiatio AN-it I this stoppig criterio, the umber of iteratios to be executed is decided a-priori. Let us deote the fixed value by N. It has bee proved that as the umber of iteratios, we shall obtai a populatio cotaiig a optimal strig with probability 1. Note that ay value N, that is fixed to a fiite umber, whereas the boud o the umber of iteratios, i.e.,, is ifiite. Oe would like to fix, i geeral, a value so that the fixed value is closed to the limitig value. Here, we eed to make the umber of iteratios to be ifiite to obtai the optimum value. Ay fiite value, which is fixed, will have ifiite differece with. Thus, though higher the value of N, the better would be the cofidece o the obtaied result, the differece betwee ad the fixed value will remai. Ideally, oe would like to fix a value so that the differece betwee optimal ad the fixed value ca be measured ad is small. These two properties are upheld by the proposed stoppig criterio K-it I this criterio, if there is o chage i the best fitess value for K cosecutive iteratios, the algorithm is termiated. The value of K is to be fixed by the user. So the basic assumptio, the user is makig here is that it is impossible to obtai a better strig after this K cosecutive iteratios. Actually, followig lemma 1, oe ca show that there is always a positive probability of obtaiig K cosecutive equal suboptimal solutios, whatever may be the value of K, where K is fiite. However, if K, for a elitist model, the probability of reachig a optimal solutio will also ted to 1. As i AN-it, the upper boud of K is ad we ca ot get a fiite differece betwee K ad.

17 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 161 Note that startig with a sub-optimal populatio Q (Q does ot cotai ay optimal strig), there is always a positive probability to remai i the same sub-optimal populatio Q after K iteratios. This probability will be higher if Q represets a sub-optimal populatio as well as a local optimal populatio (a populatio cotaiig a local optimal solutio). I this situatio, oe eeds to cotiue the algorithm further to make it go out of the local optimal populatio. Additioally, the same value of K eed ot hold good for every local optimal populatio. Oe eeds to device a techique where, the algorithm takes ito accout these situatio automatically. The proposed criterio partially does this job Proposed Criterio The proposed criterio fixes a boud (ɛ) o the variace of the best fitess values obtaied through a umber of iteratios, ad the algorithm stops whe the variace is less tha the boud. It has bee show i this article that the variace goes to 0 with probability 1 as umber of iteratios goes to. Thus ɛ, which is equal to ɛ 0, ca be viewed as the differece betwee the optimal ad the best solutio foud so far. This measuremet of the differece could ot be doe with AN-it or K-it. Secodly, by takig ɛ close to 0, we ca improve upo the best fitess value. Suppose, the geetic algorithm got stuck at a local optimal strig/populatio. I this sceario, it is highly probable that for may cosecutive iteratios the best fitess value will remai the same. It may be oted that i order to reach the local optimal populatio, the algorithm would have goe through some iteratios previously, because of which there would be some fitess values which are less tha that of the curret best fitess value. This eables the variace to be positive (> 0). By makig the value of ɛ to be very small, the proposed criterio allows the algorithm to execute more umber of iteratios compare to the umber of iteratios allowed by K-it; thereby icreasig the probability of goig out of the local optimal populatio with the proposed criterio. Let us cosider two examples, Example 1: fit(s) = 2 whe, S = S 1 = 1 otherwise Example 2: fit(s) = 3 whe, S = S 1 = 2 whe, S = S 2 = 1 otherwise I case of Example 1, whe the strig legth is quite large (say > 100), it is likely that the startig populatio will ot cotai the optimal strig S 1 ad the probability of explorig the optimal strig i sigificat umber of iteratios will be very low. I this sceario, the chace of obtaiig the strig with same fitess value over a sigificat umber of iteratios is very high. As a cosequece the variace of the best fitess values obtaied over the iteratios remai 0 ad the proposed criterio may ot be applicable. Same will be the performace of Pop-var, AN-it ad K-it. Similar result may be observed for the secod example also. Moreover, oce the secod best strig S 2 is explored, sice the variace for the proposed criterio will be positive (> 0), it will allow the algorithm to ru for a loger period thereby will icrease the chace of explorig the optimal strig S 1 cotrary to to AN-it ad K-it. Though the characteristics of POP-Var, AN-it ad K-it seem to be similar to the proposed variace based criterio, there are quite a few differeces as metioed below.

18 162 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 1. The proposed criterio ot oly takes ito accout the iformatio extracted i a fixed umber (say, K) of iteratios but also cosiders the fitess values observed prior to those K iteratios. 2. oe may ote that both the iteratio based criteria (AN-it ad K-it), metioed above, are based o the fact that the algorithm coverges as the umber of iteratio teds to, whereas the proposed criterio is based o the fact that the differece betwee the global optimal value of the fuctio ad the fitess fuctio value teds to It is ituitive to assume the value of N or K for AN-it or K-it respectively depedig o the legth of the strig legth L. Whe L is low oe cosiders a low value for N or K. O the other had, for a larger L, oe will cosider a higher value for K or N. While i the case of proposed criterio, oe does ot eed to chage the value of ɛ. 4. I AN-it ad K-it, oe has to defie the umber at the start of the algorithm. While for the proposed algorithm, oe does ot eed to defie the umber of iteratios to be executed. 5. For the iteratio based criteria, oe decides the umber of iteratios keepig i mid that the algorithm will provide a satisfactory result after those may iteratios. This is purely heuristic without cosiderig the characteristics of the fuctio. The iheret characteristics of the objective fuctios are automatically take ito accout for a sufficietly small value of the boud for variace i the proposed criterio. 6. The proposed criterio clearly gives a fiite measure regardig the closeess of the obtaied solutio to a optimum solutio. 7. While POP-Var have used the iformatio from the curret populatio, the proposed criterio maitais a elite preservig mechaism over the geeratios ad use them as the basis of the criterio. 8. Other researchers [1, 8] have computed their olie stoppig criteria at each geeratio whereas the proposed criterio is estimated for geeratios to reduce the possibility of covergece to a local optima. Though, K-it criterio is easy to implemet ad computatioally less expesive, the probability of resultig to a local optimum i K-it criterio is higher tha that of the proposed variace based criterio. This is due to the fact that K-it criterio takes ito accout the iformatio obtaied oly i K iteratios ot the iformatio prior to that. I favorable coditios, K-it criterio may stop the algorithm early ad reduce the computatio, but i geeral, ad i the worst case sceario, the proposed criterio would allow the algorithm to execute more umber of iteratios ad permit it to coverge to a global optimum solutio. 8. Coclusio ad Scope for Future Work A ew stoppig criterio for EGA has bee suggested here. It has bee show that the variace of the best solutios obtaied so far teds to 0 as. I practice, a user eeds to suggest a appropriate value for the upper boud of the variace for his/her problem. It is experimetally foud that differet

19 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model 163 problems with the same size of search space eed differet bouds for variace to obtai global optimal solutio. For better accuracy, the user eeds to choose sufficietly small value for ɛ (boud for variace). No automatic way of choosig the value ɛ is suggested here. The choice of ɛ depeds upo the accuracy the user desires. It may also be desirable for the user to kow the umber of iteratios for obtaiig ɛ accuracy i variace before performig the experimet. This is a extremely challegig problem ad it is the matter for further research. Ackowledgemet The authors sicerely thak the reviewers for their ivaluable commets ad suggestios that helped to improve the article to a great extet. The work was doe whe Prof. Sakar K. Pal held J. C. Bose Natioal Fellowship of the Govermet of Idia. Refereces [1] Aytug, H., Koehler, G. J.: New stoppig criteria for Geetic Algorithms, Europea Joural of Operatioal Research, 126, 2000, [2] Bhadari, D., Murthy, C. A., Pal, S. K.: Geetic Algorithms with Elitist Model ad its Covergece, Iteratioal Joural of Patter recogitio ad Artificial Itelligece, 10(6), 1996, [3] Davis, T. E., Pricipe, C. J.: A simulated aealig like covergece theory for the simple geetic algorithm, Proceedigs of 4th it. cof. o geetic algorithms, Morga Kaufma, Los Altos, CA, [4] Dejog, K. A.: A aalysis of the behaviour of a class of geetic adaptive systems, Ph.D. Thesis, Departmet of Computer ad Commuicatio Sciece, Uiv. of Michiga, A Arbor, [5] Feller, W.: A Itroductio to Probability Theory ad its Applicatios (Vol. I), Wiley Easter Pvt. Ltd., New Delhi, [6] Goldberg, D. E.: Geetic Algorithms: Search,Optimizatio ad Machie Learig, Addiso-Wesley, Readig, MA, [7] Greehalgh, D., Marshall, S.: Covergece Criteria for Geetic Algorithms, SIAM Joural o Computig, 30(1), 2000, [8] Jai, B. J., Pohlheim, H., Wegeer, J.: O Termiatio Criteria of Evolutioary Algorithms, GECCO Proceedigs of the Geetic ad Evolutioary Computatio Coferece., Morga Kauffma, Sa Fracisco, CA, [9] Kallel, L., Naudts, B., Rogers, A., Eds.: Theoretical aspects of evolutioary computig, Spriger, Heidelberg, Heidelberg, [10] Maity, S. P., Kudu, M. K.: Geetic algorithms for optimality of data hidig i digital images, Soft Computig, 13(4), 2009, [11] Michalewicz, Z.: Geetic Algorithms + Data Structure = Evolutio programs, Spriger Verlag, [12] Muteau, C., Rosa, A.: Gray-Scale Image Ehacemet as a Automatic Process Drive by Evolutio, IEEE Trasactios o Systems, Ma ad Cyberetics-Part B: Cyberetics, 34(2), 2004, [13] Murthy, C. A., Bhadari, D., Pal, S. K.: ɛ-optimal Stoppig Time for Geetic Algorithms, Fudameta Iformaticae, 35(1-4), 1998,

20 164 D. Bhadari et. al. / Variace as a Stoppig Criterio for Geetic Algorithms with Elitist Model [14] Nix, A. E., Vose, M. D.: Modelig geetic algorithms with markov chais, Aals of Mathematics ad Artificial Itelligece, 5, 1992, [15] Pal, S. K., Bhadari, D.: Selectio of optimal set of weights i a layered etwork usig Geetic algorithms, Iformatio Scieces, 80(3-4), 1994, [16] Pal, S. K., Bhadari, D., Kudu, M. K.: Geetic algorithms for optimal image ehacemet, Patter Recogitio Letters, 15, 1994, [17] Pal, S. K., Ghosh, A., Kudu, M. K., Eds.: Soft Computig for Image Processig, Physica Verlag, Heidelberg, [18] Rudolph, G.: Covergece aalysis of caoical geetic algorithm, IEEE Trasactios o Neural etworks, 5(1), 1994, [19] Safe, M., Carballido, J. A., Pozoi, I., Brigole, N. B.: O stoppig criteria for geetic algorithms, Proc. of Advaces i artificial itelligecesbia 2004, 17th Brazilia symposium o artificial itelligece (A. L. C. Bazza, S. Labidi, Eds.), Spriger, Berli, [20] Suzuki, J.: A Markov chai aalysis o a geetic algorithm, IEEE Trasactios o Systems, Ma ad Cyberetics, 25(4), 1995, [21] Vose, M. D.: Modelig simple geetic algorithms, Foudatios of geetic algorithms II (D. Whitley, Ed.), Morga Kaufma Publishers, 1993.