Contents Introduction Algorithm denition 3 3 Problem denition 6 4 The ridge functions using (; ){ES 9 4. The distance to the ridge axis, r

Transcription

1 Convergence Behavior of the ( + ; ) Evolution Strategy on the Ridge Functions A. Irfan Oyman Hans{Georg Beyer Hans{Paul Schwefel Technical Report December 3, 997 University of Dortmund Department of Computer Science Systems Analysis Research Group D-44 Dortmund oyman@ls.informatik.uni-dortmund.de This research is funded by a PhD scholarship of DAAD (German Academic Exchange Service).

2 Contents Introduction Algorithm denition 3 3 Problem denition 6 4 The ridge functions using (; ){ES 9 4. The distance to the ridge axis, r The progress rate ' The success rate sr() and the success probability P s ; The ridge functions using ( + ){ES 6 5. The distance to the ridge axis, R () ; 5. The success probability P s + ; 5.3 The progress rate ' ; 6 The parabolic ridge with more ospring: Increasing 7 The parabolic ridge with (+){ES for r () = 6 8 The sharp ridge 9 8. The performance of the ( + ){ES The rectangular corridor model 35 The cylindrical corridor model 37. With nonlethal ospring per generation Comparing both corridor models without normalization Conclusion and Outlook 4 References 43

3 Abstract The convergence behavior of ( + ; ){ES is investigated at parabolic ridge, sharp ridge, and at the general case of the ridge functions. The progress rate, the distance to the ridge axis, the success rate, and the success probability are used in the analysis. For the parabolic ridge, we observed in our simulations with large values using (; ){ES a larger progress rate asymptote than the theoretically computed one. The strong dependency of the (+){ES to the initial conditions is shown using parabolic ridge test function when low distances to the ridge axis are chosen as the start value. The progress rate curve and the success probability curve of the sharp ridge is explained quite exactly using a simple local model. Two members of the corridor model family are compared to the ridge functions (with large ), and they do not seem to be the limit case of the ridge function family according to our measures for convergence behavior. Keywords Evolution Strategy, progress rate, convergence analysis, ridge functions, corridor model Introduction In this work, we analyze the convergence behavior of the ( + ; ){ES on dierent ridge functions. The whole work mainly consists of simulation results, which are mostly supported by theoretical results. A dierent seed value is used for each simulation run. The derivation of the missing parts of the theoretical results is an essential part of our future work. The underlying work is based on [OBS97], in which we analyzed the parabolic ridge function, which is a member of the ridge family. Here, we continue our analysis on the general case. We give the simulation results on the sharp ridge test function. However, most of the underlying work consists of the results obtained for other members of the ridge function family (for > ). The algorithm used is described in Section. The denitions of the Evolution Strategy, the (; ){ES, and the (+){ES are given in the same section. The ridge function as well as other functions are dened in Section 3. We explain there the functions which are used or referred to in this work (parabolic ridge, sharp ridge, corridor model, rectangular corridor, cylindrical corridor, etc.). The measures used in the analysis of the convergence behavior are the expected mean distance to the progress axis (R () ), the progress rate ('), the success rate (sr()), and the success probability (P s + ; ). We used these measures already for the parabolic

4 analysis of the ridge. In this report, we will use them for dierent members of the ridge family. In Section 4, we explain the convergence behavior of (; ){ES at the family of ridge functions ( ). The analysis is repeated in Section 5 for the ( + ){ES, where we also compare the results for the two selection strategies. We compare the progress behavior of the ( + ; ){ES at the parabolic ridge in Section 6, for greater number of ospring ( = and = 5). In all the other sections of this work, we use =. Section 7 is dedicated for the worst initial condition of the ( + ){ES at the ridge functions: The ridge axis. We give some simulation results at the parabolic ridge for r () = at =, and also for r () < R () +. We summarize the simulation results obtained for the ( + ; ){ES at the sharp ridge in Section 8; the progress rate formula for (; ){ES is derived analytically. The two corridor model functions, namely rectangular corridor model and cylindrical corridor model, are investigated in Section 9 and Section, respectively. These functions are supposed to be the limit case of the ridge functions (as! ). Mainly the progress rates of these corridor model functions are compared with the ' + ; values of the ridge functions with high. Section concludes this work and gives a short summary of the results and of the research planned for the near future.

5 Algorithm denition The algorithm used in the simulations is shown in Figure below. It belongs to the class of Evolutionary Algorithms, which imitate the search operators of the natural evolution. These operators are (among others) mutation, crossover (or recombination), and selection. The evaluation is done by the environment, in our case the objective function is used for this purpose. A more detailed description of the imitation of this biological phenomenon is described in [Rec7, Rec94, Hol75, Gol89, Sch95, Fog95, Bey96, Rud96]. The Evolution Strategies (ES) is the name of the Evolutionary Algorithm \species" developed in 96s in the Technical University of Berlin [Sch95]. It imitates the mutation, selection, and recombination processes in the nature, by using (mostly normal) probability distribution functions for mutation, a deterministic mechanism for selection (which chooses just the best set of ospring individuals for the next generation), and a broad repertoire of recombination operators. The principle in ES is to generate an excess number of ospring from the current generation. With excess we mean that the number of ospring produced is larger than the size of the population in the next generation. In early 97s, the following notation was introduced for this case [Sch95]: represents the number of parents, the number of ospring (therefore > is always true). The parent individual (or individuals for the recombination) are selected arbitrarily from the current generation. The simplest case here is =, but then we have a parent individual and not a parent population. The simplest case for the mutation is used in this work, namely isotropic mutation, which has equal properties along all directions. (See Equation for the realization of the mutations.) In ES, the normal distribution is used mostly for generating mutations, but this is not a must for the algorithm. We can also use dierent mutation strengths for each axis (which will make together a mutation vector). For a more complex (and capable) model, we can use a correlation matrix for the mutations in order to adjust the mutation distribution to the local topology independent of the coordinate axes. In these three cases, the set of strategy parameters consists of a single mutation strength, of an N{dimensional vector (where N stands for the number of dimensions), of N (N +) covariances (since the matrix is symmetric), respectively. The selection scheme used in ES is dierent from the tournament or proportional selection schemes described in [Gol89]. ES uses a deterministic selection scheme, which selects the best individuals as the individuals of the next generation. In the elitist case, these individuals are selected from the union set of parents and ospring. This is called plus strategy, and noted as ( + ) in the literature. If the parent individuals do not take part in the selection, then the individuals of the next generation are selected from the ospring, which is named as comma strategy and notated as (; ) in the literature. We use the notation ( + ; ) to subsume both selection schemes. Recombination means the usage of the \genetic" information of more than one parent in the generation of each descendant, where we name the complete set of parameters of an individual as its genetic information. For recombination, several schemes can be 3

6 used [BS93]. Along with the discrete or intermediate (weighted) recombination, where both of which can be considered on global or local scale, other recombination strategies also exist. The variable setting (or the \genetic information") of an individual consists of its N{ dimensional variable vector (which gives the geometrical position of the individual in the N{dimensional search space), and its strategy parameters. As a characteristic dierence from Genetic Algorithms [Hol75, Gol89], the strategy parameters are also considered as part of the evolution process. This means that also the strategy parameters evolve during the process, they adapt to the (local) conditions. This is called the control of the mutation size, and is the rst step towards \selfadaptation". Even in its earliest development stage, ES contained self-adaptation methods for the strategy parameters (such as mutation strength). We did not consider self-adaptation in the underlying work. The capabilities of the ES models with mutation vector or mutation matrix beyond the ES model with isotropic mutations become more evident when we apply self-adaptation mechanisms. The results obtained using self-adaptation will be summarized in the next report. In the most general case, an aging scheme is introduced [SRB95]. The idea is to combine the relative merits of the comma and plus selection strategies. The parents take part in the selection as candidates, and best individuals are selected for the next generation. However, each individual has a limited lifetime, and after that it cannot be selected even if it may be the best individual ( ). The two extreme conditions, = and =, represent the comma and plus selection strategies, respectively. In this work we use a simple (; ){ES. It is outlined in Figure. Since we use just one parent, we do not actually have a population. The entity P (g) refers to the parent individual at the generation number g, and P () to the starting point. P (g) refers to the individuals generated from the parent P (g). The (; )-ES Algorithm g := initialize P () evaluate P () while not terminate do P (g) := mutate P (g) evaluate P (g) P (g+) := select (;) P (g) g := g + od Figure : The (; ){ES Algorithm without self-adaptation In Figure, subroutines of the algorithm are written in italic. First, we initialize the parent P (). The starting point can be chosen arbitrarily, however, in order to be able to compare the results with dierent mutation strengths, we initialize all the P () in 4

7 the same manner The evaluate function calculates the objective function value(s) of the parameter setting represented in the individual(s). The mutate operator is shown in Equation. This operator uses normally distributed pseudo-random variables, Z N N (; ) (with zero mean and variance one) to generate the individuals in P (g). New pseudo-random samples are used for each individual and for each variable of all individuals. We call the individuals of the ospring P (g) as I i such that P (g) = fi i ji = ; : : : ; g. The N variables of an individual I i are called a j; I i = fa jjj = ; : : : ; N?g, and of P (g) (the only parent) as P (g) = fa j jj = ; : : : ; N?g, using the j subscript. The mutation strength is not considered to be in the variable setting, since it is treated as a constant in the analysis. The probability distribution function for each a j is therefore N (a j ; ): a j := a j + Z Nj () The mutation strength is given by the user at the beginning of the simulation, i.e. it is an exogenous constant for this work. The simulation lasts for a xed number of generations given by the user, therefore the terminate function is also simple. The select operator records the individual having the highest objective function value among the ospring, and selects it as the parent of the next generation. The parent of the ospring does not take part at in the selection if we have the comma strategy. In case of the plus strategy, we denote the selection operator as select (+) P (g) [ P (g). At the last step, the generation counter is incremented, and the loop restarts for the next generation. 5

8 3 Problem denition Several test functions are analyzed in the simulations of this work. In this section, we will list these test functions, as well as the denitions of some related test functions. In this report, we analyze the ridge functions, and later the rectangular and cylindrical corridor model. The hyperplane test function and the sphere model are used in the comparisons made during the analysis. All the test functions given in this section are to be maximized by the optimization algorithm, which is in our case the ( + ; ){Evolution Strategy. The sphere model test function [Rec73, p.5] is dened as the general case of the function family with spherically distributed quality function values. The concentric hyperspheres are ordered around the optimum, which is chosen to be the origin to make the denition of the function simpler. The quality function value increases as the distance to the optimum decreases. In the general case, the sphere model can be expressed as F (x) =?f vu t X N i= x i A ; () and f(:) stands for any nondecreasing monotonous function. sphere model functions is A special case of the F (x) =? NX i= x i ; (3) which is used mostly in the theoretical analysis due to its analytically convenient structure. The general case of the hyperplane test function is dened as F (x) = a + NX i= a i x i ; a IR; a i IR + : (4) The progress rate ' is measured for the hyperplane test function in the direction of the normal vector of the plane. After a simple coordinate rotation, the coordinate axis x gives the progress direction. The general case for the ridge functions is F (x) = x? d N? X i= x i! ; (5) 6

9 where IR; d IR +. The positive factor d is used to scale the eect of the quadratic part on the convergence behavior. For = (or d = ) we get the hyperplane, for = the sharp ridge, and for = the parabolic ridge test function. The parabolic ridge is considered in [OBS97]. The success region of the ridge functions is not bounded. The subscript count starts with zero in order to emphasize the x {axis as the progress direction. The optimizer of this function is given as ^x = +; ^x i = ; i N? : (6) By adding a simple constraint, x ^x < +, we get a nite optimizer vector. The square root of the sum of the squares is named as the distance to the progress axis, i.e. r = vu ux t N? i= x i : (7) We also analyze the convergence behavior of the Evolution Strategy in the corridor model test functions in this work. The common property of the corridor model functions is the narrow corridor which is obtained using restrictions in all directions except one: the progress direction. We investigated two corridor models, which are named after the shape of the corridor walls: The rectangular corridor model [Sch95, p.34, p.35] and the cylindrical corridor model [Sch95, p.36]. In the problem catalog of [Sch95], the restrictions and the objective function are given for the general case. We will simplify them so that we have the x axis as the progress axis, and the constraint boundaries parallel to the x axis. Therefore, the rectangular corridor model is given as F (x) = cx if all G j (x) fullled? otherwise, with G j (x) : jx j j b; b IR + ; for j = ; : : : ; N? : (9) (8) The cylindrical corridor model is stated after the coordinate rotation as F (x) = G(x) : cx if G(x) is fullled? otherwise, with vu ux t N? i= () x i b ; b IR + : () 7

10 We selected c = for our simulations (in general, c IR + ; c 6= ). For c 6=, the relation between the two progress measures is as follows: Q ; = c' (Q ; gives the progress in terms of the quality function values; ' is the expected progress rate in the domain of the variable vector. See [Bey96] for the formal denitions.). The coordinate rotation does not change the nature of the problem for the Evolution Strategy. Since we use isotropic mutations, the convergence behavior (and in particular progress rate) is not aected [Sal96]. Using rotation, we can obtain problem formulations which make the theoretical analysis easier. 8

11 4 The ridge functions using (; ){ES The simulation results for the parabolic ridge were summarized in [OBS97]. In this section, we will give the simulation results for the general case of the ridge functions (for f; 3; 4; 5; 8g) using (; ){ES, with =, d = :, and N =. The simulation length is chosen as G = generations; generations are reserved additionally at the beginning of the simulations to avoid inuences from the transient case. The results for the measures of the convergence behavior are reported in separate sections. We summarize the simulation results for the distance to the ridge axis (R () ) in Section 4. (We use the symbol R () ; if we need to distinguish between the R() values of the comma and plus strategies). The results on the convergence rate (' ) are given in Section 4.. Lastly, in Section 4.3, the success probability (P s ; ) and success rate (sr()) values observed are reported. 4. The distance to the ridge axis, r This section gives the simulation results for the Efrg values of some ridge function family members with. The r value is dened in Equation 7. The expected mean of r during the simulation runs (i.e. R () for ) uctuates (with a small sample error) around R () fsmg, for the suciently large mutation strength. The value of R () fsmg is obtained from the sphere model theory [Bey93, p.86]: (N? ) R () := Efrg R () fsmg (N? ) = () c ; The value of c ; can be obtained by simulations, or from the Tables, e.g. in [Rec94, p.36-4]. As an example, c ; : The quantity R () fsmg gives the expected distance to the optimum of a (; ){ES with xed mutation strength as the simulation length G goes to innity. The formula for R () for = (parabolic ridge case) was derived analytically in [OBS97]. The validity of this formula was shown in the same report. In this section, we will show the condition for the validity of approximation for the Efrg ( > ). R () fsmg We use the normalization R () = d? (N? )R () (3) in order to be able to compare the simulation results for dierent. We normalize the mutation strength according to Equation. We made simulations for ` f; 3; 4; 5; 8g'; the results for ` = ; 4; 8' are given in Figure. For c ; 3, we can use the approximation Efrg R () fsmg. The 9

12 R (), [normalized] R (), [normalized] frag replacements = 3 = = 4 = R () fsmg = 3 = = 4 = R () fsmg Figure : R () {versus{, =. Normalized distance to the ridge axis versus normalized mutation strength. Three members of the ridge family are indicated in the gure ( ). The Efrg value of the ridge functions with 3 can be approximated by R () fsmg for c ; 3. dependence of the progress rate ' to the mutation strength will be investigated in the next section. We will use R () fsmg in the progress rate formula, and obtain accurate results (for c ; ). For < c ;, the simulation results for the mean r should be used. Therefore, for smaller values of the mutation strength, we need a more accurate theoretical formula for Efrg. Finding an accurate analytic formula for Efrg for small values of is a part of our research in the near future. The Efrg value for < will also be investigated, for =, Efrg is given in [OBS97, p.9] as R () ; = = R () fsmg vu u t v uut! 3 4 c ;? dr () N? : (4) fsmg 4. The progress rate ' In this section, we will give the simulation results for the progress rate ' of some members of the ridge family. We use an approximation with rst order partial derivatives for the theoretical ' to justify the simulation results. This quantity is obtained using the local model at the equilibrium condition, i.e. when the r (g) has a value around R () (The dependency of r to and is explained in Section 4.). The rst estimate to the ' can be obtained after using a nonlinear normalization scheme. After this normalization, ' can be calculated using,, and the constant c ;. This simple estimate for ' is valid for larger values. It yields quite useful values for 3. This normalization scheme is also used in order to analyze the other performance measures, i.e. r, P s ;, sr(), and Q ;. We start the analysis with substituting r dened in Equation 7 into the ridge function denition in Equation 5, yielding

13 F (x ; r) = x? dr : (5) The gradient vector a is therefore @r A =?dr? : (6) Using the quantity a, we get the rst basic estimate for the progress rate ' as [OBS97, p.56] ' ridge c ; kak = c ; q + (dr? ) (7) rr () fsmg = s + d c ; (N?) c ;? : (8) The approximation \r R () fsmg " is valid for c ;, as shown in Section 4.. The accurate ' ; for = is [OBS97, p.] ' ; = = vu u t c ; + + c ; r + c ;! : (9) At this stage, we can introduce a nonlinear normalization scheme which allows us to compare the progress rates of dierent ridge functions. This scheme is actually a generalization of the scheme introduced for the parabolic ridge test function: = d? (N? ) () ' = d? (N? )' () Using this scheme, the approximate formula for the progress rate reduces to ' ridge c ; : () r +? c ;

14 ', progress rate [normalized].6 c ;.5.5 = = 3 = 4 = Figure 3: ' {versus{, =. Normalized progress rate versus normalized mutation strength. Five members of the ridge family are indicated in the gure ( ). The theoretical ' curves for parabolic ridge ( = ) and hyperplane function ( = ) are also indicated for comparison, as well as the horizontal asymptote c ;. In Figure 3, we give the normalized progress rate curves for normalized mutation strength. Firstly, all of the ridge functions (like other function classes) have progress rate values less than ' hyperplane (This can also be seen from Equation 8, since the denominator is never smaller than one). The value ' ridge is always nonnegative, since there is always a component of the mutation vectors in the direction towards optimum. Secondly, the progress rates of all ridge functions for > start to decrease, for suciently large values. For 3, this decrease starts at <. The ' value for the cases = " (" <<, i.e. for values very near to two) are investigated using Equation. We also made simulation runs for = : and = :5, and obtained results supporting this claim. Therefore, the limit value of ' using the (; ){ES for increasing mutation strength is lim! ' ridge < = ; lim! ' ridge > = : (3) For =, we have c ; as the limit value (see Figure 3 or Equation 9; and Section 6 for details). After explaining the progress behavior of the ridge family, we will show how good the approximate progress rate formula is for nite. We consider the two extremes from Figure 3, namely = and (We obtained also accurate results for the observed values in between). In Figure 4, we see that the approximate formula in Equation 7 gives very good results for = using the r values obtained from the simulations. We want to test the same idea for. As we see in Figure 5, we can compute the ' value by inserting the mean of the r values from the simulations in Equation 7 (and applying the normalization afterwards) quite accurately. We nd this result quite surprising, since this simple estimate for ' only uses rst order partial derivatives.

15 ', progress rate [normalized] ', progress rate [normalized].6 c ;.5.5 theo theo r = theo Figure 4: ' {versus{, =, =. The simple estimate from Equation 8 (with the approximation r R () fsmg, indicated as theo) diers much from the simulation results ( = ); however, it is asymptotically correct. If we use the approximation in Equation 7, and insert the mean value of r obtained from the simulations, we get the curve \theo r". The analytically derived formula for ' (Equation 9) is indicated as \theo". 3.6 c ;.5.5 theo theo r Figure 5: ' {versus{,. The gure legend is the same as in Figure 4. Again, the simulation results are almost exactly explained by \theo r". We expect this method to work also for much larger values. The remaining step is nding a theoretical (if possible an analytically driven) equation for R (), i.e. the expected mean value of r for a given. The ' {versus{ plots for < have as their asymptotes simple lines with a positive slope. The slopes of the asymptotes increase up to c ; as is decreased down to zero (remember that ' hyperplane = c ; ). For <, the simulation results coincide with the theoretical progress rate curve of the hyperplane. Therefore, we can say that the parabolic ridge test function is the only member of the ridge function family with a horizontal asymptote (c ; ), which is somewhat obvious after considering Equation. 3

16 sr(), success rate 4.3 The success rate sr() and the success probability P s ; The success behavior is an important part for the convergence behavior. The \success" is measured using two measures: The probability of having a descendant which has a better quality function value than its parent is called success rate (sr()). The probability that at least one of the ospring will have a better quality function value than its parent is called success probability (P s ; ). The sr() and P s ; values were investigated in [OBS97] for the parabolic ridge case ( = ). Here we will give the simulation results for ` = ; 4; 8'. We also made simulation runs for = 3 and = 5. Since the success curves changed gradually as is increased, we removed these two intermediary curves from the pictures. = 3 = = = 4 theo = Figure 6: sr(){versus{ for =. The success rate curves versus normalized mutation strength for dierent members of the ridge family. The analytically obtained curve for the parabolic ridge is also indicated in the gure (theo = ): It coincides with the simulation results for =. The asymptote (?c ; ) is reached faster at larger values. We obtain sr(; ) (?c ; ) (4) and P s ;? [(c ; )] (5) from the gures (sr(; ) :6933 and P s ; :47355). Therefore, the limits for the parabolic ridge seem also to be valid for the other members of the ridge family (See [OBS97, p.3] for the analytical derivation of sr() and P s ; for the parabolic ridge). The limit value is attained faster (i.e. at smaller values) for larger values of. We also obtained the same lower limits for the sharp ridge ( = ). 4

17 P s ;, success probability = 3 = = = 4 theo = Figure 7: P s ; {versus{ for =. The success probability curves versus normalized mutation strength for dierent members of the ridge family. The analytically obtained curve for the parabolic ridge is also indicated in the gure (theo = ). The asymptote? [(c ; )] is reached faster at larger values. The limit value of P s ; in Equation 5 changes very little as is increased. For = 3 it is around :485, for = it decreases to :47. For = and we get the values :448 and 44, respectively. 5

18 R (), [normalized] R (), [normalized] 5 The ridge functions using ( + ){ES We reported the simulation results obtained using (; ){ES for in Section 4. We will report here the simulation results for the convergence measures R () (Section 5.), P s + ; (Section 5.), and ' (Section 5.3). In these sections, we will compare the (; ){ES and ( + ){ES, and investigate the limit behaviors of these strategies. The simulation runs were done for f; 3; 4; 5; 8; 64g, d = :, N =, and =. In order to make the diagrams more readable, we do not indicate the results results for all these values. Each simulation run lasted G = generations, and we reserved additional generations for the transient phase. 5. The distance to the ridge axis, R () + ; At our simulations for the parabolic ridge, we observed smaller R () rates at the plus strategy, which meant smaller P s + rate than P s ;. This was an acceptable explanation for the lower progress rate values using the plus strategy. Here we will investigate the dependence of the R () values on + ;. Furthermore, we will show the dependence of R () on and the selection strategy. The + R() values are ; + ; normalized according to Equation 3. frag replacements = 64 comma plus comma plus comma plus = 64 comma plus Figure 8: R () {versus{ for = (see Equation 3 for the denition of R () ). Normalized distance to the ridge axis versus normalized mutation strength. The R () values are compared for the two selection strategies at (gure left) and = 64 (gure right). The plus strategy has a lower R () value for larger. In Figure 8, we compare the R () values of plus and comma strategies. For = 64 and < c ; 3, this value is the same at both selection strategies. On the gure left, the same comparison is done at, and the curves separate from each other at lower values than for = 64. In both gures, we see that the R () + value is smaller than the R () ; value. For small values, the R () value gets larger for larger : This is more clear in Figure 9. For the comma selection strategy, we compared the R () values for increasing (Figure ). As we have seen in that picture, the R () ; values are very near to R() fsmg for 6

19 P s + ;, success probability R (), [normalized] R (), [normalized] = 64 = 5 = 5 = 4 = 3 = 3 = ( + ){ES = 64 = 4 = = 64 = 5 = 5 = 4 = 3 = 3 = 5 5 (; ){ES = 64 = 4 = Figure 9: R () {versus{ for =. Normalized distance to the ridge axis versus normalized mutation strength. The R () + and R() ; values are shown next to each other (gure left and gure right, respectively). The R () fsmg values are also shown in both pictures to ease the comparison. In general, for > c ; 3, the R () value is larger for the comma strategy. 3 and > c ; 3. In Figure 9, we compare the R () values (i.e. for plus and comma strategies). The R () value increases in both cases for < c ; if we increase, where the = 64 curves seem to be very near to the limit (R () (c ; )). 5. The success probability P s + ; The success probability P s + ; (for the comma and plus strategies) is investigated in [OBS97, p.45, p.49] at the parabolic ridge. In Section 4.3, the simulation results obtained using (; ){ES are given for several values. Here we will compare the P s + ; results for f; 3; 4; 5; 8; 64g. The results for = 3 and = 5 are not indicated in the Figure, in order to reduce the number of curves per diagram. We used = in our simulations... = 64 comma plus Figure : P s + ; {versus{ for =. The success probability curves versus normalized mutation strength for = 64. The limit value for P s ; obtained at the parabolic ridge is also valid for = 64. On the other hand, the P s + value continues to decrease as is increased. The success probability P s + ; for = 64 is shown in Figure. The curves for both 7

20 P s +, success probability P s ;, success probability P s +, success probability ( + ){ES (; ){ES selection strategies dier from each other essentially for > c ; 3; and the P s + continues to decrease as P s ; goes to the limit shown in Equation 5 (Page 4). = 64 = 64 (; ){ES.9 = = 4 = 5..8 = 64 = 5 ( + ){ES = 4 = 3 = 3 =. = = 4 = = 5 = 5 = 4 = 3 = 3 = Figure : P s + ; {versus{ for =. The success probability curves versus normalized mutation strength for dierent members of the ridge family. The P s + curves (gure left) and the P s ; curves (gure right) are plotted next to each other for dierent values of. The tendency at the comma strategy is easier to see: for very large values of, P s ; decreases suddenly near = c ; 3 to the asymptotic limit. The P s + value, however, decreases enormously for increasing. The success probability curves dier enormously for plus and comma selection. In Figure, we observe the dierence for = 64; in Figure, we can see the dierence for several increasing values. The P s ; curve has an asymptotic lower limit, which is attained almost by c ; 3 for = 64. The P s + curve, however, goes to zero for larger. 5.3 The progress rate ' + ; In this section, we will compare the progress rates of the ridge function family (with ) for plus and comma selection strategy. This convergence measure was compared for the = case (parabolic ridge) in [OBS97, p.44]. In Section 4., it was investigated for 8 using (; ){ES. The ' curves are compared for = 3 and = 5 in Figure. The comma strategy + ; has better progress rates for any value at = 3. For = 4, the peak values of these selection strategies are very near to each other; however, the comma strategy shows larger progress rates for any. For = 5 (Figure right), the ^' + value is slightly larger than ^' ;. The ' + is larger in the interval :6 < < 4:5. The shape of the ' + curve does change only a little as is increased from three to ve; however, the change at the ' ; curve foretells us that the plus strategy will give comparably better progress rates for larger. The progress rate of plus and comma strategies for = 64 are compared in Figure 3. For small mutation strengths, both strategies give almost same results; otherwise (for > :4), the ' + value is denitely better. The drastic dierence at = c ; 3:8, where the plus strategy is more than 8

21 , progress rate [normalized], progress rate [normalized], progress rate [normalized] frag replacements = 5 comma plus ' + ; = 3 comma plus = 3 comma plus ' + ; = 5 comma plus Figure : ' + ; {versus{ for =. Normalized progress rate versus normalized mutation strength. The comparison of the ridge functions for = 3 and = 5. The value c ; is also shown in the pictures. The progress rate of the plus strategy surpasses the ' ; and also the ^' ; for = 5 (gure right) = 64 plus comma ' + ; Figure 3: ' + ; {versus{ for =. Normalized progress rate versus normalized mutation strength for the ridge function with = 64. The value c ; is also shown in the picture. For < :4, both selection strategies give similar results; otherwise, the plus strategy is better. ve times better than the comma selection strategy, will be examined more thoroughly. The P s + ; values are almost the same at this value (P s ; :5, P s + :53, Figure ). The R () + ; values are also almost the same (the R() ; is about ve per cent greater than R () +, Figure 8). Neither the success probability gures nor the R() gures can explain the progress rate dierence between these two strategies. Figure 4 is used for the explanation successfully. The '{versus{r () plots for and = 64 are given in Figure 4. Both axes are unnormalized. For = 64, the progress rate values decrease after R () =, and we see that ' ; < ' + for the same R () value (we have to note that same R () does not necessarily mean same ). For, ^' is attained at a larger + ; R() value, and the plus strategy shows smaller progress rates than the comma strategy for larger R () values. As a conclusion, we see that the ^' value depends on the + ; R() value (to be more precisely, it depends on the value at which the critical R () magnitude is reached). The value indicates how large the eect of R () is in the objective function, 9

22 ' +, progress rate [normalized] ' ;, progress rate [normalized].4.5 frag replacements ' + ;.3. comma plus ' + ;..5. comma plus = 64 = R () + ; R () + ; frag replacements Figure 4: ' + ; {versus{r() for and = 64, =. The progress rate versus distance to + ; the ridge axis. Both axes are PSfrag unnormalized. replacements The ' values are compared for the selection strategies at (gure left) and = 64 (gure right). The ' + value is greater than ' ; on the gure right, for R () >. rate [normalized] ' e.g. the ^' +, progress rate [normalized] value is reached around R () for = 64. This explains us why the ' ( + ){ES ; {versus{ curve decreases suddenly (Figure 3). The decrease in ' + was not so (; ){ES sharp since the R () does not increase in + as fast as R () ; (see Figure 8). = 64 = ( + ){ES (; ){ES = 64 = 64 = 5 = 4 = 4.5 =.5 = = 5 = 4 = 3 = 3 = = 5 = 5 = 4 = 3 = 3 = Figure 5: ' + ; {versus{ for =. Normalized progress rate versus normalized mutation strength, for dierent values of. The ' + values are shown on the gure left. The ' ; values on the gure right decrease to zero suddenly after > c ; 3. The normalized progress rates ' and + ' ; are plotted next to each other in Figure 5 for dierent values of. The case = is discussed in depth in [OBS97, pp.48]. The results for comma strategy are given already in Section 4.. In Figure 5, we can compare the shapes and the asymptotic of the progress rate gures for the plus and comma strategy. For 5, the plus strategy gives better results (since IR, we have to note here that the exact value for the change where ^' becomes larger than + ^' ; should be between four and ve). The negative r term in the objective function makes the progress in the x direction for the comma strategy very hard (See Equation 6 on Page 6), since R () ; increases faster than R() +. The plus strategy prots from the lower R () values, and ' + decreases more slowly.

23 6 The parabolic ridge with more ospring: Increasing We want to verify the theoretical formulae describing the convergence behavior of the (; )-ES at the parabolic ridge test function using a larger value for the number of descendants ( ). The formulae for ', ; R(), and P ; s ; are derived analytically in [OBS97]. The derivations do not force a specic value for ; however, since it is assumed that N (the number of dimensions) to be suciently large, we doubt the formula to be valid for higher values of, keeping N constant. In this section, we will compare the simulation results obtained for three dierent values (,, and 5) at the (; ){ES with the theoretical formulae, and with the results obtained using ( + ){ES. We keep d and N constant (d = : and N = ). The simulation length is G = for the (; ){ES and G = 5 for the others, reserving additional generations before collecting data. We do not have any theoretical results for the plus strategy, since it is not possible to evaluate the respective integrals analytically. The limit value for the progress rate at the parabolic ridge test function is lim! ' ; = c ;: (6) Therefore, in order to be able to compare the results for dierent, we divided the normalized progress rates by c ;. Therefore, the horizontal asymptote becomes simply. This horizontal line is also indicated in the gure for the plus strategy. The and ' are normalized according to Equations and (Page )... frag replacements ' ; =c ; (; ){ES (5) () () ' + =c ; ( + ){ES (5) () () ' + =c ; ( + ){ES ' ; =c ; (; ){ES Figure 6: ' + ; =c ; versus. The normalized progress rates divided by c ; are plotted against the normalized mutation strength. The values used for are,, and 5 (d = :, N =, ( + ; ){ES). The simulation results for the comma strategy are shown in the gure left, and for the plus strategy on the gure right, respectively. The curves obtained from the theoretical formula are also indicated for the (; ){ES case, the simulation results diverge from these values for = and = 5. The ' ; values exceed the c ; value for larger values of. The peak value of the progress rate for the plus strategy ( ^' + =c ; ) increases as is increased. In Figure 6, the progress rate values for both selection methods are given next to

24 each other. We have chosen d = : and N =, and for the values, and 5. The simulation results for the comma strategy are in the gure left, for the plus strategy in the gure right. The ' values for = are almost equal for both strategies for < (approximately). If is increased further to and 5, this range increases further to < 3:5 and < 6, respectively. The ' ; =c value obtained at large ; values increases as is increased, i.e. the horizontal asymptote is not valid for these large values (N = ). The progress rate value goes to a greater limit. An explanation of this limit behavior for large will be investigated in the future. However, the progress rate curve obtained from the simulations with large does not dier much from the N-independent theoretical formula for larger values of N. For instance, in a simulation run with N = and = 5, we observed that the horizontal asymptote c ; was again valid. This indicates that a more accurate ' ; formula must be developed which incorporates an N-dependence. Also the values for ' + decreased for this case. However, the progress attained by increasing the simulation length 5 times (using = ) is much more than by increasing from to 5. The simulations for the plus strategy (Figure 6 right) showed us that the peak value for the progress rate ^' + increases as is increased. The mutation strength at which this optimum progress rate is attained (^ ) also increases. However, the ^' value + does not seem to surpass the c ; value. Some values for comparing these two strategies are given in Table. This table summarizes Figure 6 and Figure 8. c ; c ; ^' ; ^ + sr(^ ; ) + ^' + ^' + =c ; ^' = + ^' ; :539 :368 :368 :5 :9 :59 :67 :67 :58 6:88 6:54 4:4 : 4:8 :77 :74 5 3:37 9: 9:9 6: : 7:73 :84 :78 Table : Comparing the ( + ; ){ES for large. The value of ^ ; is innity (). The table is generated using Figures 6 and 8. The values for ^' ; ( = and = 5) are obtained by averaging the simulation results for > 4. The table entries which have one or two digits after the decimal point are obtained empirically. Several valuable informations are repeated in Table. The entries with one or two digits after the decimal point are approximate, and are obtained from the simulations. In summary, we can see that ^ increases as is increased. The ratio + ^' = + ^' ; increases at the same time. The success rate at ^ + decreases as is increased. The distance to the progress axis (R () ) is another quantity which is used in describing the convergence behavior of the ridge functions. In Figure 7, we investigate this value for the higher values of () for both selection methods. We multiplied the simulation results by c ; in order to be able to compare the results for dierent in the same picture. As the number of descendants increased, the plus strategy attained the same R () values up to larger values of as the comma strategy. To give an example, we have R () ;5 R () +5 for 5, whereas R () ; R () + for. The strange observation is ' ;5 > ' +5 for 6, i.e. also when R () ; R(). +

25 sr(), success rate sr(), success rate ; R () ; c ; R () + c (; ){ES ( + ){ES 8 8 frag replacements c ; R () R sm (5) () () c ; R () ; 6 4 R sm (5) () () ( + ){ES 5 5 (; ){ES 5 5 Figure 7: c ; R () + ; versus. The distances to the ridge axis (multiplied by c ; ) are plotted against the normalized mutation strength. The values used for are,, and 5 (d = :, N =, ( + ; ){ES). The line labeled by \Rsm" stands for c ; R () fsmg = (N? )=. The simulation results for the comma strategy are shown in the gure left, and for the plus strategy on the gure right, respectively. The R () ; value does not dier much from the R() for + 6 (approximately), although the ' values for both selection methods dier remarkably even for = 5 (See Figure 6). This peculiarity is more clear for = 5: R () ;5 R() for +5 5, and ' ;5 ' +5 for. The progress rate values do not seem to be related to R () values for higher values of. As a result, we can say that the ' values are not necessarily related to the R () values. For the case with large number of descendants, we obtained hereby a result conicting to the explanation in [OBS97, p.48]. The success probability P s ; gives us the necessary explanation for the dierence between the progress rate values of these two selection methods. frag replacements ( + ){ES (; ){ES () () (5) e (; ){ES ( + ){ES () () (5) e Figure 8: sr() versus. The success rate values are plotted against the normalized mutation strength. The values used for are,, and 5 (d = :, N =, ( + ; ){ES). The simulation results for the comma strategy are shown in the gure left, and for the plus strategy on the gure right, respectively. The theoretical sr() curve and the theoretical asymptote are indicated for the comma strategy. The success rate values for the plus strategy case continue to decrease as increases, and they are always smaller than the values obtained by the comma selection method. Success rate value is a measure on the local curvature, i.e. it gives us the ratio of the success volume to the total local volume. The success volume consists of points having 3

26 a better quality function value than the parent, and which are within the distance of the normalized mutation strength to the parent individual. The analytical formula for the success rate (using the comma selection strategy) is [OBS97, p.33] 6 sr() + c ; 3? 3 4 c; + s + 5A 7 5 (7) with the theoretical limit given as lim sr() = (?c ;) (8)! The success rate curves for high values of (,, and 5) are shown in Figure 8. For the curves obtained using (; ){ES, we see that the asymptotic value is valid for = ; however, for = and = 5, it seems to be an unsharp lower limit. The theoretical curve for sr(5) is also not correct, either. However, it describes the tendency correctly. For the plus strategy (gure right), the curves for these three values do not deviate much from each other. For 8, we can say that the larger the the smaller the sr() value. This result is somewhat surprising, but we should remember that the success rate is also a function of, and the equilibrium condition changes if is changed. The sr() values are the same for both selection strategies at small values, to be more precise, for, 4, and 5 for =, =, and = 5, respectively. An interesting result obtained from Figures 6 and 8 is the following: The progress rate of the plus strategy cannot be greater than the one of the comma strategy at the parabolic ridge test function. The success rate is always less than for the ^ value where ^' ; is reached (for any ). The plus strategy has a success rate values which are less than or equal to the ones of the comma strategy. For values less than ^, the + plus strategy could not surpass the comma strategy. As the number of ospring goes to innity, the plus strategy will attain progress rate values closer to ^' ; ; however, it cannot surpass. The success probability P s + ; is dened as the probability to have at least one descendant in ospring which has a better tness value than the parent individual. The analytical formula for the success probability P s ; is [OBS97, p.3] P s ;? 8 >< >: 6 + c ; 3 9? 3 4 c; + s + 5A 7 >= 5 >; (9) 4

27 P s ;, success probability P s +, success probability frag replacements ess probability ( + ){ES (5) () () limit PSfrag (; replacements ){ES P s ;, success probability (; ){ES ( + ){ES (5) () () Figure 9: P s + ; versus. The success probability values are plotted against the normalized mutation strength. The values used for are,, and 5 (d = :, N =, ( + ; ){ES). The simulation results for the comma strategy are shown in the gure left, and for the plus strategy on the gure right, respectively. The gure on the left side has a non-logarithmic scale on the vertical axis, whereas a logarithmic scale was necessary on the gure right. The theoretical P s ; curve and the theoretical asymptote are indicated in the gure left. It diers from the simulation results for larger values. The P s ; values have the same asymptote for increasing. The success probability values for the plus strategy case (P s + ) continue to decrease as increases, and they are always smaller than the P s ; values. with the theoretical limit given as lim P s ; =? [(c ; )] (3)! The success probability curves for dierent values of (,, and 5) are shown in Figure 9. The theoretical curve obtained for P s ; is not correct for = and = 5. All three curves seem to have almost equal asymptotic values, which is a remarkable dierence to the plus strategy (gure right). Moreover, the P s ; values are much larger than P s + values for larger values of. The progress deciency of the plus strategy observed in Figure 6 can be explained by using Figure 9. We will point here to a special case: = 5 with < < 5. The ' +5 values are smaller than the ' ;5 values for this interval of the mutation strength, although the R () values are almost the same. The explanation for the lower progress rate is lower P s + rates. The plus strategy has a dierent convergence behavior, and since it does not accept any ospring with a lower quality function value as the parent of the next generation, the progress is also slower. 5

28 7 The parabolic ridge with (+){ES for r () = In this section, we will investigate the eect of the starting position in the search space to the progress behavior of the plus strategy. We used the ( + ){ES algorithm for this purpose. The parabolic ridge is used as the test function (d = :, N = ). In [OBS97, pp.48], we compared the two selection strategies (( + ; ){ES) at the parabolic ridge, and explained the progress deciency of the plus strategy. We concluded that the low R () and P + s + values caused the low ' rates (for + > ). In Section 6, we have seen that the latter of these values (P s + ) gives more information on the progress behavior. In the simulations done, we used r () = R () fsmg, which is a good estimate for the comma strategy, and we observed R () < + R() ; at the end of the simulation. We reserved a period of generations before starting to collect statistics, and observed later that this was a sucient length for reaching the steady state for r (g) for both selection strategies. The aim here was to start collecting data in the steady state of both algorithms. The question remaining unanswered was the eect of a much smaller r () for the ( + )-ES (The comma strategy is not sensitive to the starting position; for the problem parameters used, the period is suciently long to reach the steady state R () ; : See [OBS97, pp.59]). In order to have a fair comparison, the starting position in the search space (the initial conditions) should not favor either of the algorithms compared [Bey96, p.3]. We want to show here how the initial conditions can aect the comparison results. Our rst test aimed to verify the existence of R () + for dierent values of ; i.e. we have chosen dierent values for r () (very large values as well as values smaller than R () fsmg ). For very large values (up to r() = R () fsmg ), the same R() + value is obtained, it took just a longer time in generations to reach the steady state (e.g. for = and r () = R () fsmg, generations were sucient to reach R() ). + However, for r () < R (), the results were a bit dierent. The + r(g) values uctuate around the same R () value; however, an + r() value chosen smaller than the lower end of this uctuation interval results in a very long adapting time. Several generations pass by without a single movement in the search space if r () is chosen too small. For =, we made some experiments for r () < R () (d = :, N =, G = + 56 ), which has R () 5. If we chose + r() 8, the rst move in the search space (g M in short) was the 348th, and after 3 moves (at g = 95683) the vicinity of the stationary state was reached (r 4). In short, more than two million generations were necessary for the rst move in the search space, and thereafter, about 6 5 generations to reach the vicinity of the steady state. If we started at r () 98, \only" generations were necessary to reach r 4, i.e. at least an order of magnitude faster. On the other hand, for r () 65, more than four million generations were necessary for the rst move. Therefore, we advise to choose a large start value r () (such as R () fsmg ) as long as the is unknown. Otherwise, the ( + ){ES needs too many generations to reach the R () + stationary state for r. 6