A HYBRID APPROACH TO GLOBAL OPTIMIZATION USING A CLUSTERING ALGORITHM IN A GENETIC SEARCH FRAMEWORK VIJAYKUMAR HANAGANDI. Los Alamos, NM 87545, U.S.A.

Transcription

1 A HYBRID APPROACH TO GLOBAL OPTIMIZATION USING A CLUSTERING ALGORITHM IN A GENETIC SEARCH FRAMEWORK VIJAYKUMAR HANAGANDI MS C2, Los Alamos National Laboratory Los Alamos, NM 87545, U.S.A. and MICHAEL NIKOLAOU Chemical Engineering Department Texas A&M University College Station, TX , U.S.A. Submitted to Computers and Chemical Engineering, April 995. Abstract. The concern of this work is global optimization using genetic algorithms (GAs). In this work we propose a synergy between the cluster analysis technique, popular in classical stochastic global optimization, and the GA to accomplish global optimization. This synergy minimizes redundant searches around local optima and enhances the capability of the GA to explore new areas in the search space. The proposed methodology demonstrates superior performance when compared with the simple GA on benchmark cases. We also report our solution of the optimal pump congurations synthesis problem. *Author to whom all correspondence should be addressed. Tel.: (55) , FAX: (55) , vmh@lanl.gov

2 INTRODUCTION Multi-modal objective functions are common in engineering applications. Also, very little or no a priori information about the analytical properties of the objective function in some optimization applications in engineering stress the need to be able to search for the global optimum. Existing optimization techniques that seek the global optimum can be broadly classied into the following two categories: deterministic methods and stochastic methods. Here we focus on stochastic methods of optimization. Many classical stochastic optimization methods rely on the so called multistart algorithm. In this algorithm one starts with a set of uniformly random points and does local searches starting from a selected number of points from this set. A variety of algorithms stemmed out of the above basic scheme and some are very successful. Torn (978; 977) introduced an eective technique to prevent redundant objective function evaluations at local optima in a multistart algorithm. This technique uses a clustering algorithm. Experiences of various researchers (Kan and Timmer, 984; Boender et al., 982; Gomulka, 978a; 978b) on using Torn's clustering algorithm have shown that it is a valuable technique. The review paper by Kan and Timmer (984) discusses the salient features of several successful variations of the multistart algorithm and also gives a comparison between these techniques on some benchmark problems. The collection of papers edited by Dixon and Szego (978) is a citation classic in the eld of stochastic optimization. Among the non-traditional stochastic global optimization techniques, genetic algorithms (GAs) and simulated annealing (SA) have shown some promise. Genetic algorithms (GAs) are attracting a lot of attention in recent optimization literature. They are emerging as powerful alternatives to traditional optimization methods which are too restrictive and CPU intensive. A GA manipulates 2

3 populations of candidate solutions whereas a gradient-based method manipulates a single solution. This helps the GA to search for the global optimum. Also, since the search procedure is stochastic, the possibilities of a GA getting stuck in a local optimum are less. Moreover, GAs manipulate populations based solely on the individual objective function values and do not need any other information regarding the objective function. Theoretically, the simple genetic algorithm (SGA) locates the global optimum of a multi-modal function in nitely many generations (trials). In an SGA run, when a population is stuck in a local optimum, the mutation operator is supposed to provide the necessary diversity to the population to search in dierent areas. In spite of these arguments, it is demonstrated that an SGA in inecient (i.e. it takes a large number of trials) in locating the global optimum in a multi-modal function (Goldberg, 989) and that there is scope for improvement. De Jong (975) concluded that the elitist strategy (explained in the sequel) in an SGA brings premature convergence. In order to tackle this problem he proposed a modication that increases diversity in a population. His modication, called the crowding scheme, has seen some success. Androulakis and Vankatasubramanian (99) reported a modied genetic search which manipulates the search direction and called it an extended genetic search (EGS) algorithm. In this work we propose to use Torn's clustering algorithm in the genetic search procedure to identify and dissolve a niche (or a cluster) of species in a given population to increase population diversity. We demonstrate the eectiveness of our hybrid methodology on benchmark problems given in Dixon and Szego (978). Also, we report our solution of the multi-modal pump congurations design problem studied by Westerlund et al. (994). In the sections that follow, rst we give a background on genetic algorithms (GAs), then we discuss the motivation for our modication to the SGA. Subse-

4 quently we discuss methodology, simulation results and our solution of the optimal pump congurations synthesis problem using the modied GA. Conclusions appear in the last section. 2 BACKGROUND ON GENETIC ALGORITHMS GAs accomplish the task of optimization by starting with a random \population" of values for the parameters of an optimization problem, and thereafter producing new \generations" of improved values that combine the best \parts" of values from previous populations. In order to achieve this, it is very crucial to represent the objective variables in a non ambiguous way. In our case studies we use the binary representation of real numbers and integers and manipulate these strings to achieve reproduction, crossover and mutation. Parent representations which are used to produce ospring are selected in a random manner but with a bias toward selecting candidates with high tness values. This step is called reproduction. Ospring are produced by randomly cutting and swapping parts of the two parent representations. This is called the crossover operation. Mutation is achieved by screening the bits of the ospring and changing each binary bit to the opposite kind ( to or to ) with a very low probability. Mutation helps to introduce diversity in the population thus preventing the GA from getting saturated with solutions in local optima. Figs. and 2 illustrate the crossover and mutation operations using binary representations of the real and integer optimization variables. In each generation a limited number of candidates get to reproduce and have ospring which go into the next generation. When the problem is of objective function maximization the tness f i of candidate i in a given population is the normalized 4

5 Parent Representations Offspring Figure : Illustration of the crossover operation. The ospring are a result of cutting and swapping parts indicated between the arrows in the parent representations. Flip Figure 2: Illustration of mutation. Each bit is ipped to the opposite kind with a very low probability. 5

6 Table : The SGA pseudocode - Main algorithm. START generation count= initialize population by random strings repeat generation count=generation count+ create next generation until (generation count max gen) STOP objective function given by f i = f i? f min f max? f min where f i is the objective function value of candidate i and f max and f min are the maximum and minimum values respectively of f i encountered in that population. In a minimization problem the same procedure is followed but with a sign reversal. Elitism, in which the best candidate encountered thus far is reintroduced in each generation, can also be employed. Tables, 2 and give the pseudocode for a simple GA (SGA). In the pseudocode, the array tness contains the tness values of all the members of the current population. sumtness is the sum of the tness values of all members of the population and the function random() returns a uniform random number on the real interval [,]. One run of the roulette wheel selection routine gives two parents which can now be used to perform crossover and mutation as demonstrated in Figs. and 2. For further details about the theory and implementation of GAs we refer the reader to well known literature (Goldberg, 989; Davis, 99). The following points summarize the advantages of using GAs: As a GA proceeds randomly (yet systematically) in its search, it does not re- 6

7 Table 2: The SGA pseudocode - Renement of create next generation. START STOP j= repeat select two members by roulette wheel selection perform crossover perform mutation introduce ospring into the new population j=j+2 until(j population size) Table : The SGA pseudocode - Renement of select two members by roulette wheel selection. START STOP partial sum=. j= rand=random()sumtness repeat j=j+ partial sum=partial sum+tness(j) until(partial sum > rand or j=population size) selected individual=j 7

8 quire smoothness, derivability, continuity etc. in the objective function. The only requirement is that for a set of values for the optimization parameters, one must assign a tness value. GAs can eciently handle highly nonlinear and noisy objective functions encountered in stochastic processes where traditional gradient based methods are inecient. GAs are amenable to parallel processing. Unlike gradient search algorithms, in GA the objective function evaluation for one parameter set is independent of that for all others in the same generation. This facilitates the use of parallel computers for the search procedure. Motivation Many valuable lessons are learned by observing the nature closely. Holland (975) made arguments concerning the distribution of individuals in a natural environment. He argued that, in nature, when like individuals begin to dominate a niche, increased competition for limited resources decreases life expectancy and birth rates among the members of that niche. Less crowded niches experience less pressure and achieve life expectancy and birth rates much closer to their actual potential. In other words, each individual has a certain survival probability and reproduction capability associated with its tness in its environment. If a particular individual belongs to a niche which is dominated by like individuals then its survival and reproduction chances are diminished because it is more likely to compete with the members of its own community (niche) for limited resources whereas for an individual belonging to a less crowded niche these rates are not altered. To incorporate these aspects of nature in articial genetics, De Jong used 8

9 the so called crowding model. In this model, whenever an individual is born, it is compared with a xed number of individuals in the current population and it replaces an individual which is closest to itself. He called the xed number of individuals compared as the crowding factor. This helps prevent the duplication of like individuals and helps to maintain diversity to some extent. When multimodal function optimization is of concern, the issue of niches being formed has yet another dimension. Consider a situation where we start with a random initial population of parameters and do a few steps of gradient search starting from each one of these points. In this case there is a high probability that elements of a particular niche (which is nothing but a group of like individuals) tend toward the same local optimum if the gradient-based search is continued. Instead, it makes perfect sense to select the optimal member of this niche and do a gradient-based search starting from only this member. A similar concept was used by Torn (977; 978) in conjunction with the multistart algorithm- a stochastic optimization technique. Torn's algorithm, known as Algorithm LC (multiple local searches with clustering), is given below:. Choose uniformly random points. 2. Use a local search algorithm for a few steps.. Find clusters by using a cluster analysis technique. 4. Take a sample point from each cluster. Return to step 2. We propose to use this idea in the context of the genetic search procedure. We use Torn's clustering technique to identify and dissolve clusters in the parameter space so that redundant function evaluations of candidates leading to the same local 9

10 5 z 5 x Figure : The two-dimensional Shekel's function. y optimum are minimized and diversity is introduced in the population. In other words, by the cluster dissolving idea we explicitly impose the natural pressure on crowded niches and cause them to perish. When the performance of a simple genetic algorithm (SGA) and that of a simple genetic algorithm with De Jong's crowding model (SGA-CM) are compared we noticed that niches are not eectively dissolved for any value of the crowding factor. On the other hand our methodology explicitly identies and breaks the clusters. We use a two dimensional Shekel's function to demonstrate this. This test function has six prominent local maxima. Details of this function are given in the next subsection. Fig. graphically shows this function. Fig. 4 shows a snap shot of the parameter values in the twentieth generation of an SGA. Clusters are pointed out by arrows. Fig. 5 shows the corresponding snapshot of an SGA-CM run. Note that the clusters are still present. Similar behavior is noted for several generations and during several dierent runs. Fig. 6 shows that the GA with a clustering algorithm (SGA-CL) eectively dissolves the clusters. In our method, as soon as niches are

11 8 6 x x Figure 4: The distribution of the variables after 2 generations in an SGA run for the 2-D Shekel's function. 8 6 x x Figure 5: The distribution of the variables after 2 generations in an SGA-CM run for the 2-D Shekel's function.

12 8 x x Figure 6: The distribution of the variables after 2 generations in an SGA-CL run for the 2-D Shekel's function. formed they are identied. The best members of those niches are retained and the rest are randomized. This introduces diversity in the population as shown in the spread of the parameters in Fig. 6. We show the same aspects again with the Goldstein and Price function in Figs. 7, 8, 9 and. 4 Methodology and Simulation Results Although it eliminates redundant calculations, Torn's algorithm (see previous section) lacks the ability to search in new directions which is crucial for multi-modal function optimization. To remedy this problem we propose a hybrid methodology which relies on the strength of GAs to search in new directions and on the ability of the clustering technique to reduce redundant calculations. The steps in the proposed GA with clustering (GA-CL) are as follows:. Start with an initial population. 2. Evaluate the tness of each member of the population. 2

13 z x y Figure 7: The Goldstein and Price function. 2 x x Figure 8: The distribution of the variables after 2 generations in an SGA run for the Goldstein and Price function.

14 2.5 x s R x Figure 9: The distribution of the variables after 2 generations in an SGA-CM run for the Goldstein and Price function. x x Figure : The distribution of the variables after 2 generations in an SGA-CL run for the Goldstein and Price function. 4

15 . Apply a clustering algorithm to identify clusters. 4. If any clusters are identied, from each cluster select the best member and randomize the rest. 5. Evaluate tness of the new members whose tness has not been evaluated in step Apply reproduction, crossover and mutation. 7. Generate new population. 8. Stop if the number of generations exceeds the maximum allowable generations or if convergence is satisfactory. Else, go to step 2. The breaking of clusters reduces redundant objective function evaluations and the randomization introduces diversity in the population. In the GA we use here, we employ elitism. Elitism accelerates local search. Using elitism in between cluster dissolving operations is analogous to doing a local search between cluster dissolving operations as suggested by Torn in his algorithm LC. The clustering algorithm we use here aims at selecting a seed point and growing the cluster by enlarging the hypersphere around the seed point as long as the density of the cluster remains greater than the average density of the space in which all the points are distributed. This method does not require the user to specify the number of clusters a priori. Details of this algorithm are given in APPENDIX A. We report our study on six benchmark problems and one design problem. The design problem is given in the next section. The benchmark problems are taken from Dixon and Szego (978). Details of these problems are given in Dixon and Szego (978). Table 4 summarizes the properties of these problems. Table 5 gives a comparison between the success probabilities of dierent approaches. These 5

16 Table 4: Details of the benchmark optimization problems GP, H, H6, S5, S7 and S. Goldstein and Price (GP): max f? [( + (x + x 2 + ) 2 (9? 4x + x 2? 4x 2 + 6x x 2 + x 2 2] [ + (2x? x 2 ) 2 (8? 2x + 2x x 2? 6x x x 2 2 ]g Range:?2: x ;2 2: Four local maxima. Global maximum at: x = :, x 2 =?:7 with the value -. Hartman's Family (H: variables, H6: 6 variables) max f P m i= c iexp(? P n j= a ij(x j? p ij ) 2 g, n= or n=6 Range: : x i : Four local maxima. Global maxima at: H:(.,.556,.85) with the value.86 H6:(.29,.42,.47,.265,.9,.66) with the value.4 Shekel's Family (S5: m=5, S7: m=7, S: m=): max f P m i= Pn Range: : x i : Global maxima at: j= (x i?a ij )2 +ci g, n=4 S5: (4.,.998,.99,.996) with the value.422 S7: (4.9, 4., 4., 4.7) with the value.99 S: (.995, 4.5, 4.,.996) with the value.5 6

17 Table 5: Comparison between success probabilities of dierent algorithms. Algorithms Problem SGA SGA-CM EGS SGA-CL Goldstein and Price (GP) Hartman (H) Hartman 6 (H6) Shekel 5 (S5) Shekel 7 (S7) Shekel (S) probabilities were computed after one hundred independent runs for each problem. The parameters for the various algorithms compared are given in Table 6. The parameters were adjusted by trial and error to get the best performance of the SGA and SGA-CM approaches. The simple genetic algorithm (SGA) gives the base case for comparison. De Jong's crowding model (SGA-CM) shows marginal to no improvement over the performance of the SGA. The SGA with clustering approach (SGA-CL) showed superior performance in all cases. Figs. through 6 show the convergence proles of SGA and SGA-CL approaches for dierent problems. The dotted lines on all these plots indicate the global optimum of the function being investigated. Since a genetic search-based algorithm is a stochastic search, a single run does not give the true convergence prole. To present a reasonably true prole the convergence plots are made by averaging fty independent runs. The improvement of the clustering approach over the SGA is clear from these plots too. In terms of objective function computations, the SGA tops the list as it is based on non overlapping populations (in the transition from one generation to the next, the whole population is replaced by the ospring.) 7

18 Table 6: GA parameters. Population size : Number of ospring in each generation : for SGA, 2 for SGA-CM and 2 for SGA-CL Mutation rate :. Crossover probability :. Elitism : O for SGA, O for SGA-CM and On for SGA-CL Crowding factor : (only applicable for SGA-CM) Chromosome length per variable : 5-5 SGA-CL F(x) SGA Generation Figure : The objective function vs. generation number for GP (average of fty independent runs). 8

19 SGA-CL.855 F(x) SGA Generations Figure 2: The objective function vs. generation number for H (average of fty independent runs) SGA-CL.24 F(x).22.2 SGA Generations Figure : The objective function vs. generation number for H6 (average of fty independent runs). 9

20 9 SGA-CL 8 F(x) SGA Generations Figure 4: The objective function vs. generation number for S5 (average of fty independent runs). 9 SGA-CL F(x) SGA Generations Figure 5: The objective function vs. generation number for S7 (average of fty independent runs). 2

21 9 SGA-CL F(x) SGA Generations Figure 6: The objective function vs. generation number for S (average of fty independent runs). 5 Optimization of Pump Congurations as an MINLP Problem In this section we solve the problem of optimal pump conguration synthesis (Westerlund et al., 994). The optimization problem is cast as a constrained MINLP problem which is outlined next. Centrifugal pumps are available at variable speeds (rpm). The total required pressure rise and the total ow are specied. The aim is to select the best seriesparallel combination of the pumps that minimizes total cost. The cost is a function of the number of parallel pumping lines (N p;i ) in each level, the number of pumps (N s;i ) in series in every line and the power (P i ) of each pump. The power required for each pump is a function of the pump speed (! i ) and the volumetric ow in the pump ( V _ i ). Fig. 7 gives a schematic of a two level pump conguration. Problem details are given in Table 7. We use the three level pump conguration (L = ) in our application. Westerlund et al. (994) have reported that this is a nonconvex 2

22 Table 7: Details of the pump congurations problem. Objective Variables N p;i number of parallel pumping lines N s;i number of pumps in each line P i power of pump i x i volumetric fraction of Vtot _ in i! i rpm of pump i Constraints p = 629(! 295 )2 + :696(! ) V _ 295? :6 V _ 2 p 2 = 25(! )2 + 2:95(! 2 ) V _ 295 2? :5 V _ 2 2 p = 6(! 295 )2 + :5(! ) V _ 295? :946 V _ 2! i 295 P Objective Function L J = i=(c i + Ci P i)n p;i N s;i where L =, Ci = 8, C = 629: C 2 = 2489:, C = 67:89, _V tot = 5, p tot = 4 P = (! )(9:9(! )2 + :6(! ) V _? :56 V_ P 2 = (! 2 )(:2(! )2 + :644(! 2 ) V _ 295 2? :564 _ P = (! )(6:52(! )2 + :2(! ) V _ 295 2? :22 V _ 2 _V i = x i Vtot _ p i = N p;i ptot N s;i ) V 2 2 ) ) 22

23 Level 2 Level L=2, N =2, i=,2 pi N =2, i=,2 si Figure 7: Schematic of a two level pump conguration. Each level is shown with two parallel lines with three pumps in each line. problem with thirty seven local minima. They used the extended cutting plane (ECP) method and the software DICOPT++ to solve the MINLP problem. They had to change the problem formulation to improve the solution and achieve near global optimum results. Also, proper selection of the software related parameters was found to be critical. We found that the GA-CL could solve this MINLP problem and it reached the global optimum in a reasonable number of generations. The objective function augmented with the weighted constraints was used to evaluate tness of each candidate solution. Fig. 8 shows the convergence prole. Fig. 9 gives the globally optimal pump conguration. Table 8 gives a comparison of the dierent solutions. An advantage of using GAs for optimization is that in the last population, a variety of near optimal solutions are present from which the designer can choose in case the optimal solution has some undesirable properties or if the suboptimal solution has some additional desirable properties. In one of the suboptimal designs 2

24 -5 -e+6 -.5e+6?J -2e+6-2.5e+6 -e+6 -.5e Generation Figure 8:.-J vs. generation for the optimal pump congurations synthesis problem. Level 2 Level Figure 9: The globally optimal pump conguration. 24

25 Table 8: Comparison of results for the pump congurations problem. Design# Conguration Cost Remarks N p; = 2, N p;2 =, N p; = Global N s; =, N s;2 = 2, N s; = optimum x = :94, x 2 = :86, x = : (Westerlund et al.,! = 2855,! 2 = 295,! = 994) 2 N p; =, N p;2 =, N p; = 2 54 ECP method N s; =, N s;2 =, N s; = 2 solution x = :449, x 2 = :, x = :55 (Westerlund et al.,! = 286,! 2 =,! = ) N p; =, N p;2 =, N p; = 557 DICOPT++ N s; =, N s;2 =, N s; = 2 solution x = :, x 2 = :, x = (Westerlund et al.,! =,! 2 =,! = ) 4 N p; = 2, N p;2 =, N p; = GA solution N s; =, N s;2 = 2, N s; = x = :94, x 2 = :86, x = :! = 2855,! 2 = 295,! = 5 N p; = 2, N p;2 =, N p; = 2578 GA suboptimal N s; =, N s;2 =, N s; = solution. x = :, x 2 = :, x = : Note that!! = 296,! 2 =,! = violates the maximum rpm limit for a pump 25

26 Level Figure 2: The suboptimal pump conguration. (design#5 in Table 8) the GA suggests that if pumps of 296 rpm ( rpm more than the maximum limit) are used we can get a better design consisting of only one level with two parallel lines containing one pump in each line as shown in Fig Conclusions In stochastic search techniques like GAs, formation of niches around local optima is detrimental to the convergence to global optimum. In this paper we presented a modication to the simple genetic search which is based on a clustering algorithm and demonstrated its eciency in solving benchmark multi-modal optimization problems. This algorithm aims at identifying and dissolving the clusters so that diversity is preserved in the population. The superiority of the proposed methodology over the simple genetic search and extended genetic search is clear. Our methodology was able to nd the globally optimal conguration in the optimal pump congurations synthesis problem. In this problem, apart from the glob- 26

27 ally optimal conguration, the modied GA was also able to locate a suboptimal, simple and trivial conguration. 27

28 References Androulakis, I. P., and V. Venkatasubramanian, \A Genetic Algorithmic Framework for Process Design and Optimization," Computers and Chem. Eng., 5, 27 (99). Boender, C. G. E., A. H. G. R. Kan, and G. T. Timmer, \A Stochastic Method for Global Optimization," Mathematical Programming, 22, 25 (982). Davis, L, 99, Handbook of Genetic Algorithms, Van Nostrand, New York. De Jong, K. A., An Analysis of the Behavior of a Class of Genetic Adaptive Systems, Ph.D. Dissertation, University of Michigan, Ann Arbor, MI (975). Dixon, L. C. W., and G. P. Szego, Towards Global Optimisation 2, North-Holland, Amsterdam (978). Goldberg, D. E., 989, Genetic Algorithms in Search Optimization and Machine Learning, Addison Wesley, Reading, MA. Gomulka, J., \Deterministic Versus Probabilistic Approaches to Global Optimization," in Towards Global Optimisation 2, L. C. W. Dixon and G. P. Szego (Eds.), North-Holland, Amsterdam (978a). Gomulka, J., \A User's Experience with Torn's Algorithm," in Towards Global Optimisation 2, L. C. W. Dixon and G. P. Szego (Eds.), North-Holland, Amsterdam (978b). Holland, J. H., Adaptation in Natural and Articial Systems, The University of Michigan Press, Ann Arbor, MI (975). 28

29 Kan, A. H. G. R., and G. T. Timmer, \A Stochastic Approach to Global Optimization," in Numerical Optimization, SIAM, Philadelphia, PA (984). More, J. J., and S. J. Wright, Optimization Software Guide, SIAM, Philadelphia, PA (99). Torn, A. A., \A Search-Clustering Approach to Global Optimization," in Towards Global Optimisation 2, L. C. W. Dixon and G. P. Szego (Eds.), North- Holland, Amsterdam (978). Torn, A. A., \Cluster Analysis Using Seed Points and Density-Determined Hyperspheres as an Aid to Global Optimization," IEEE Trans. Sys. Man. Cyber., 7, 6 (977). Westerlund, T., F. Pettersson, and I. E. Grossmann, \Optimization of Pump Congurations as a MINLP Problem," Computers and Chem. Eng., 8, 845 (994). 29

30 APPENDIX A T ORN'S CLUSTERING ALGORITHM In what follows we describe briey the clustering algorithm used by Torn in his algorithm LC. The basic idea is to grow clusters from seed points to include all the neighboring points. There are two main issues in identifying clusters: Location of the seed points. When to stop the growth of a cluster. In other words, deciding about a threshold distance r for stopping the growth of a cluster. The choice of a threshold distance is critical because if r is large, all points will be assigned to a single cluster and if r is small, each point will be form an isolated cluster. In a multi-modal objective function minimization problem, consider a collection of N points in which we want to identify clusters. The assumption is made that the local minima would be good seed points to use and that the point with the smallest function value may be used as an estimate of the local minimum. The cluster growing is started from this point. When this cluster stops growing and a new seed point is needed, the point, among those not yet classied, with the least function value is chosen. This procedure is repeated until all the N points are classied. A cluster is grown by enlarging a hypersphere around the seed point as long as the point density in the hypersphere remains greater than the uniform point density = N V where V is the volume of the space containing all the N points and is given by V = N i=4 2 i :

31 i are the eigenvalues of the covariance matrix S and s ij = N? ( NX k= x ik x jk? n nx nx x ik k= k= In the above equation, x k is the kth parameter vector and x ik is the ith component of x k. Torn's original clustering algorithm includes only the parameters of optimization in the vectors x k for cluster identication, but we include the objective function value along with the parameters of optimization in the vectors x k. This modication gives better classication (cluster identication) particularly for functions with multiple local minima, each having dierent objective function values, which are crowded in a particular search space. x jk )

32 List of Figures Illustration of the crossover operation. The ospring are a result of cutting and swapping parts indicated between the arrows in the parent representations. : : : : : : : : : : : : : : : : : : : : : : : : 5 2 Illustration of mutation. Each bit is ipped to the opposite kind with a very low probability. : : : : : : : : : : : : : : : : : : : : : 5 The two-dimensional Shekel's function. : : : : : : : : : : : : : : : 4 The distribution of the variables after 2 generations in an SGA run for the 2-D Shekel's function. : : : : : : : : : : : : : : : : : : 5 The distribution of the variables after 2 generations in an SGA- CM run for the 2-D Shekel's function. : : : : : : : : : : : : : : : : 6 The distribution of the variables after 2 generations in an SGA-CL run for the 2-D Shekel's function. : : : : : : : : : : : : : : : : : : 2 7 The Goldstein and Price function. : : : : : : : : : : : : : : : : : : 8 The distribution of the variables after 2 generations in an SGA run for the Goldstein and Price function. : : : : : : : : : : : : : : 4 9 The distribution of the variables after 2 generations in an SGA- CM run for the Goldstein and Price function. : : : : : : : : : : : 4 The distribution of the variables after 2 generations in an SGA-CL run for the Goldstein and Price function. : : : : : : : : : : : : : : 5 The objective function vs. generation number for GP (average of fty independent runs). : : : : : : : : : : : : : : : : : : : : : : : : 9 2 The objective function vs. generation number for H (average of fty independent runs). : : : : : : : : : : : : : : : : : : : : : : : : 9 2

33 The objective function vs. generation number for H6 (average of fty independent runs). : : : : : : : : : : : : : : : : : : : : : : : : 2 4 The objective function vs. generation number for S5 (average of fty independent runs). : : : : : : : : : : : : : : : : : : : : : : : : 2 5 The objective function vs. generation number for S7 (average of fty independent runs). : : : : : : : : : : : : : : : : : : : : : : : : 2 6 The objective function vs. generation number for S (average of fty independent runs). : : : : : : : : : : : : : : : : : : : : : : : : 2 7 Schematic of a two level pump conguration. Each level is shown with two parallel lines with three pumps in each line. : : : : : : : 2 8.-J vs. generation for the optimal pump congurations synthesis problem. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 2 9 The globally optimal pump conguration. : : : : : : : : : : : : : 25 2 The suboptimal pump conguration. : : : : : : : : : : : : : : : : 27

34 List of Tables The SGA pseudocode - Main algorithm. : : : : : : : : : : : : : : 6 2 The SGA pseudocode - Renement of create next generation. : : : 7 The SGA pseudocode - Renement of select two members by roulette wheel selection. : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7 4 Details of the benchmark optimization problems GP, H, H6, S5, S7 and S. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7 5 Comparison between success probabilities of dierent algorithms. 8 6 GA parameters. : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8 7 Details of the pump congurations problem. : : : : : : : : : : : : 24 8 Comparison of results for the pump congurations problem. : : : 26 4