MIED INTEGER-DISCRETE-CONTINUOUS OPTIMIZATION BY DIFFERENTIAL EVOLUTION Part : the optmzaton method Joun Lampnen Unversty of Vaasa Department of Informaton Technology and Producton Economcs P. O. Box 700 FIN-650 Vaasa Fnland phone: +358-6-3248 fax: +358-6-3248 467 E-mal: Joun.Lampnen@UWasa.f Ivan Zelnka Techncal Unversty of Brno Faculty of Technology (Zlín) Department of Automatc Control nám. T.G.M. 275 762 72 Zlín Czech Republc phone: +420-67-72 5 2 fax: +420-67-72 5 2 E-mal: Zelnka@zln.vutbr.cz Abstract: Ths artcle dscusses solvng non-lnear programmng problems contanng nteger dscrete and contnuous varables. The Part of the artcle descrbes a novel optmzaton method based on Dfferental Evoluton algorthm. The requred handlng technques for nteger dscrete and contnuous varables are descrbed ncludng the technques needed to handle boundary constrants as well as those needed to smultaneously deal wth several non-lnear and non-trval constrant functons. In Part 2 of the artcle a mechancal engneerng desgn related numercal example desgn of a col sprng s gven to llustrate the capabltes and the practcal use of the method. It s demonstrated that the descrbed approach s capable of obtanng hgh qualty solutons. The novel method s relatvely easy to mplement and use effectve effcent and robust whch makes t as an attractve and wdely applcable approach for solvng practcal engneerng desgn problems. Keywords: evolutonary algorthms dfferental evolutonon-lnear optmzaton engneerng desgn Introducton In general when dscussng non-lnear programmng the varables of the obect functon are usually assumed to be contnuous. However practcal engneerng desgn work the problems n whch some or all of the desgn varables have dscrete or nteger values are very common. They are commonly dscrete because the avalable values are lmted to a set of standard szes. For example the thckness of a steel plate the dameter of a copper ppe the sze of a screw the module (or dametral ptch) of a gear tooth the sze of a roller bearng etc. are often lmted to a set of commercally avalable standard szes. Respectvely nteger varables are commonly used to express a number of dentcal elements used n the desgn. Examples of nteger varables are the number of teeth of a gear the number of bolts or rvets needed to fx a structure the number of heat exchanger tubes the number of coolng fns of a heat snk the number of parallel V-belts used for transmsson the number of cols of a sprng etc. It s clear that a large fracton of engneerng desgn optmzaton problems fall nto the category of mxed nteger-dscrete-contnuouson-lnear programmng problems. Despte of that mostly solvng contnuous problems are dscussed n the lterature. In practce the problems contanng nteger or dscrete varables are usually solved as a contnuous problem and the nearest avalable dscrete value s then selected. In ths case the result s often qute far from optmum. The reason for ths approach s that t s stll commonly consdered that no really satsfactory non-lnear programmng method appear to be avalable whch s able to handle all nteger dscrete and contnuous varables and whch s also effcent effectve robust and easy to use. Generally contnuous problems are consdered to be easer to solve than dscrete ones suggestng that the presence of any noncontnuous varables consderably ncreases the dffculty of fndng a soluton. Another source of dffcultes n practcal engneerng desgn optmzaton s handlng of constrants. Often multple constrants are nvolved and often the nature of the constrants s non-lnear and non-trval too. The feasble solutons may be only a small subset of the desgn space. Many dfferent methods have been proposed for solvng mxed nteger-dscrete-contnuouson-lnear programmng problems. Some engneerng desgn related works were collected n Table. Recently many softcomputng based methods have been under nvestgatons and were found to be hghly potental. Howevero sngle approach seems to be totally satsfyng when all the nvolved aspects are consdered. Wthout pontng to any sngle method t can be stated that all of them suffer from at least one of the followng shortcomngs: complexty of mplementaton and usage lack of flexblty hgh computatonal cost poor robustness lmted constrant handlng capabltes poor capabltes n fndng feasble hgh qualty solutons. In order to response these demands of practcal engneerng desgn optmzaton a novel approach for solvng mxed nteger-dscrete-contnuouson-lnear engneerng desgn optmzaton problems has been developed based on the recently ntroduced Dfferental Evoluton (DE) algorthm [SP95a]. Ths nvestgaton descrbes the technques needed to handle boundary constrants as well as those needed to smultaneously deal wth several
non-lnear and non-trval constrant functons. After ntroducng these technques an llustratve and practcal numercal example s gven n Part 2 of ths artcle whch the desgn of a col sprng s optmzed by DE. The mxed-varable methods used to solve the example problem are dscussed n detal and compared wth publshed results obtaned wth other optmzaton methods for the same problem. Mxed Integer-Dscrete-Contnuous Non-Lnear Optmzaton Reported by Proposed soluton technque Reference Sandgren Branch & Bound usng Sequental Quadratc programmng [San90] Fu Fenton & Gleghorn Integer-Dscrete-Contnuous Non-Lnear Programmng [FFG9] Loh & Papalambros Sequental Lnearzaton Algorthm [LP9a LP9b] Zhang & Wang Smulated Annealng [ZW93] Chen & Tsao Genetc Algorthm [CT93] L & Chow Non-Lnear Mxed-dscrete Programmng [LC94] Wu & Chow Meta-Genetc Algorthm [WC95] Ln Zhang & Wang Modfed Genetc Algorthm [LZW95] Therauf & Ca Two-level parallel Evoluton Strategy [TC97] Cao & Wu Evolutonary Programmng [CW97] Lampnen & Zelnka Dfferental Evoluton Ths artcle Table : Some proposed methods for solvng engneerng desgn related mxed nteger-dscretecontnuous non-lnear optmzaton problems. 2 Mxed Integer-Dscrete-Contnuous Non-Lnear Programmng A mxed nteger-dscrete-contnuouson-lnear programmng problem can be expressed as follows: Fnd = ( L) x ( ) ( ) ( d ) ( c) { x x x x } = [ ] to mnmze f ( ) subect to constrants g ( ) 0 =... m x x Ü 2 3 andsubect to boundary constrants ( U ) ( d ) n = d Ü ( c) c Ü T () () (d) and (c) denote feasble subsets of nteger dscrete and contnuous varables respectvely. The above formulaton s general and bascally the same for all types of varables. Only the structure of the desgn doman dstngushes one problem from another. However t s worth to notce here the prncpal dfferences between nteger and dscrete varables. Whle both nteger and dscrete varables have a dscrete nature only dscrete varables can assume floatng-pont values. In practce the dscrete values of the feasble set are often unevenly spaced. These are the man reasons why nteger and dscrete varables requres dfferent handlng. 3 Dfferental Evoluton Prce and Storn frst ntroduced the Dfferental Evoluton (DE) algorthm a few years ago [SP95a]. DE can be categorzed nto a class of floatng-pont encoded evolutonary optmzaton algorthms. Currently there are several varants of DE. The partcular varant used throughout ths nvestgaton s the DE/rand//bn scheme. Ths scheme wll be dscussed here only brefly snce more detaled descrptons are provded n [SP95a SP97a]. Snce the DE algorthm was orgnally desgned to work wth contnuous varables the optmzaton of contnuous problems s dscussed frst. Handlng of nteger and dscrete varables are to be explaned later. Generally the functon to be optmzed f s of the form: f ( ) : Ü 3 Ü (2)
The optmzaton target s to mnmze the value of ths obectve functon f() mn ( f ( )) (3) by optmzng the values of ts eters: ( x x ) Ü = x n (4) denotes a vector composed of n obectve functon eters. Usually the eters of the obectve functon are also subect to lower and upper boundary constrants x (L) and x (U) respectvely: (5) L) ( U ) x x x = ( As wth all evolutonary optmzaton algorthms DE works wth a ulaton of solutonsot wth a sngle soluton for the optmzaton problem. Populaton P of generaton G contans n soluton vectors called ndvduals of the ulaton. Each vector represents potental soluton for the optmzaton problem. (6) ( G) ( G) P = = G = G So the ulaton P of generaton G contans n ndvduals each contanng n eters (chromosomes of ndvduals): (7) G G P ( ) ( ) ) = = x = = ( G In order to establsh a startng pont for optmum seekng the ulaton must be ntalzed. Often there s no more knowledge avalable about the locaton of a global optmum than the boundares of the problem varables. In ths case a natural way to ntalze the ulaton P (0) (ntal ulaton) s to seed t wth random values wthn the gven boundary constrants: (8) U L P ( 0) ( 0) ( ) ( ) = x = r ( x x ) + x = = L r denotes to a unformly dstrbuted random value wthn range [0.0.0[. The ulaton reproducton scheme of DE s dfferent from the other evolutonary algorthms. From the st generaton forward the ulaton of the followng generaton P (G+) s created n the followng way on bass of the current ulaton P (G). Frst a temporary (tral) ulaton for the subsequent generaton P (G+) s generated as follows: (9) ( ) ( ) ( ) ( ) ( ) G G G G+ xc + F x x f r Cr = D A B x' = ( G) x otherwse = D = A = C r = B = [ 0] F [ 02] r [ 0[ C = max A B A B and C are three randomly chosen ndexes referrng to three randomly chosen ndvduals of ulaton. They are mutually dfferent from each other and also dfferent from the runnng ndex. New random values for A B and C are assgned for each value of ndex (for each ndvdual). A new value for random number r s assgned for each value of ndex (for each chromosome). The ndex D refers to a randomly chosen chromosome and t s used to ensure that at least one chromosome of each ndvdual vector (G+) dffers from ts counterpart n the prevous generaton (G). A new random (nteger) value s assgned to D for each value of ndex (for each ndvdual). F and C r are DE control eters. Both values reman constant durng the search process. As well the thrd control eter (ulaton sze) reman constant too. F s a real-valued factor n range [0.02.0] that controls the amplfcaton of dfferental varatons and C r s a real-valued crossover factor n range [0.0.0] controllng the probablty to choose mutated value for x nstead of ts current value. Generally both F and C r C
affect the convergence velocty and robustness of the search process. Ther optmal values are dependent both on obectve functon f() characterstcs and on the ulaton sze n. Usually sutable values for F C r and n can be found by tral-and-error after a few tests usng dfferent values. Practcal advce on how to select control eters n F and C r can be found n [SP95a Sto96a SP97a]. The selecton scheme of DE also dffers from the other evolutonary algorthms. On bass of the current ulaton P (G) and the temporary ulaton P (G+) the ulaton of the next generaton P (G+) s created as follows: (0) ( G+ ) ( G+ ) ( G) ( G+ ) ' f fcost ( ' ) fcost ( ) = ( G) otherwse Thus each ndvdual of the temporary (tral) ulaton s compared wth ts counterpart n the current ulaton. The one wth the lower value of cost functon f cost () (to be mnmzed) wll survve to the ulaton of the next generaton. As a result all the ndvduals of the next generaton are as good or better than ther counterparts n the current generaton. The nterestng pont concernng DE s selecton scheme s that a tral vector s not compared aganst all the ndvduals n the current ulaton but only aganst one ndvdual aganst ts counterpart n the current ulaton. 4 Constrant Handlng 4. Boundary constrants It s mportant to notce that the reproducton operaton of DE s able to extend the search outsde of the ntalzed range of the search space (Equaton 8 and 9). It s also worthwhle to notce that sometmes ths s benefcal property n problems wth no boundary constrants because t s possble to fnd the optmum that s located outsde of the ntalzed range. However boundary constraned problems t s essental to ensure that eter values le nsde ther allowed ranges after reproducton. A smple way to guarantee ths s to replace eter values that volate boundary constrants wth random values generated wthn the feasble range: () ( U ) ( L) ( L) ( G ) ( L) ( G ) ( U ) ( G ) + + + r ( x x ) + x f x' x x' x x' < > = ( G+ ) x' otherwse = = Ths s the method that was used for ths work. Another smple method s to reproduce the boundary constrant volatng values accordng Equaton 9 as many tmes as s necessary to satsfy the boundary constrants. 4.2 Constrant functons In ths nvestgaton a soft-constrant (penalty) approach was appled for handlng of the constrant functons. The constrant functon ntroduces a dstance measure from the feasble regon but s not used to reect unfeasble solutons as t s n the case of hard-constrants. One possble soft-constrant approach s to formulate the cost-functon as follows: (2) f b mn ( f ( ) + a) ( ) = c ( f ( )) + a > 0 =.0 + s g( ) f g( ) > 0 c = otherwse s cost The constant a s used to ensure that only non-negatve values wll be assgned to f cost. When the value of a s set hgh enough t does not otherwse affect the search process. Constant s s used for approprate scalng of the constrant functon value. The exponent b modfes the shape of the optmzaton surface. Generally hgher values of s and b are used when the range of the constrant functon g() s expected to be low. Often settng m b
s= and b= works satsfactorly and only f one of the constrant functons g () remans volated after the optmzaton run t wll be necessary to use hgher values for s or/and b. In many real-world engneerng desgn optmzaton problems the number of constrant functons s relatvely hgh and the constrants are often non-trval. It s possble that the feasble solutons are only a small subset of the search space. Feasble solutons may also be dvded nto separated slands around the search space. Furthermore the user may easly defne totally conflctng constrants so that no feasble solutons exst at all. For example f two or more constrants conflct so that no feasble soluton exsts DE s stll able to fnd the nearest feasble soluton. In the case of non-trval constrants the user s often able to udge whch of the constrants are conflctng on the bass of the nearest feasble soluton. It s then possble to reformulate the cost-functon or reconsder the problem settng tself to resolve the conflct. Another beneft of the soft-constrant approach s that the search space remans contnuous. Multple hard constrants often splt the search space nto many separated slands of feasble solutons. Ths dscontnuty ntroduces stallng ponts for some genetc searches and also rases the possblty of new locally optmal areas near the sland borders. For these reasons a soft-constrant approach s consdered essental. It should be mentoned that many tradtonal optmzaton methods are only able to handle hard-constrants. For evolutonary optmzaton the soft-constrant approach was found to be a natural approach. 5 Handlng of Integer and Dscrete Varables In ts canoncal form the Dfferental Evoluton algorthm s only capable of handlng contnuous varables. Extendng t for optmzaton of nteger varables however s rather easy. Only a couple of smple modfcatons are requred. Frst for evaluaton of cost-functonteger values should be used. Despte of that the DE tself may stll work nternally wth contnuous floatng-pont values. Thus (3) fcost ( y ) = n x for contnuous varables y = INT( x ) for nteger varables x INT() s a functon for convertng a real value to an nteger value by truncaton. Truncaton s performed here only for purposes of cost-functon value evaluaton. Truncated values are not else assgned. Thus DE works wth a ulaton of contnuous varables regardless of the correspondng obect varable type. Ths s essental for mantanng the dversty of the ulaton and the robustness of the algorthm. Second case of nteger varablestead of Equaton 8 the ulaton should be ntalzed as follows: (4) U L L P ( 0) ( 0) ( ) ( ) ( ) = x = r ( x x + ) + x = = Addtonallystead of Equaton the boundary constrant handlng for nteger varables should be performed as follows: (5) ( U ) ( L) ( L) ( G ) ( L) ( G ) ( U ) ( G ) + + + r ( x x + ) + x f INT( x' ) x INT( x' ) x x' < > = ( G+ ) x' otherwse = = Dscrete values can also be handled n a straghtforward manner. Suppose that the subset of dscrete varables (d) contans l elements that can be assgned to varable x: (6) ( d ) ( d ) = x = l ( d ) x < x ( d ) + Instead of the dscrete value x tself we may assgn ts ndex to x. Now the dscrete varable can be handled as an nteger varable that s boundary constraned to range l. To evaluate of the obectve functon the dscrete value x s used nstead of ts ndex. In other wordsstead of optmzng the value of the dscrete varable drectly we optmze the value of ts ndex. Only durng evaluaton s the ndcated dscrete value used. Once
the dscrete problem has been converted nto an nteger one the prevously descrbed methods for handlng nteger varables can be appled (Equatons 3 5). 6. Conclusons The descrbed method s relatvely smple easy to mplement and easy to use. Despte of that t s capable of optmzng all nteger dscrete and contnuous varables and capable of handlng non-lnear obectve functons wth multple non-trval constrants. A soft-constrant (penalty) approach s appled for handlng of constrant functons. Some optmzaton methods requre a feasble ntal soluton as a startng pont for a search. Preferably ths soluton should be rather close to a global optmum to ensure convergence to t nstead of a local optmum. If non-trval constrants are mposed t may be dffcult or mpossble to provde a feasble ntal soluton. The effcency effectveness and robustness of many methods are often hghly dependent on the qualty of the startng pont. The combnaton of DE wth the soft-constrant approach does not requre any ntal soluton but t can stll take advantage of a hgh qualty ntal soluton f one s avalable. For example ths ntal soluton can be used for ntalzaton of the ulaton n order to establsh an ntal ulaton that s based towards the feasble regon of the search space. If there are no feasble solutons n the search space as s the case for totally conflctng constrants DE wth the soft-constrant approach s stll able to fnd the nearest feasble soluton. Ths s mportant n practcal engneerng desgn work because often many non-trval constrants are nvolved. The descrbed approach was targeted to fll the gap n the feld of mxed dscrete-nteger-contnuous optmzaton no really satsfactory methods appeared to be avalable. The Part 2 of ths artcle wll demonstrate wth a practcal example that the attempts were at least farly successful. Despte beng n ts nfancy the descrbed approach have great potental to become a wdely used multpurpose optmzaton tool for solvng a broad range of practcal engneerng desgn problems. References [CW97] Cao Y. J. and Wu Q. H. (997). Mechancal desgn optmzaton by mxed-varable evolutonary programmng. Proceedngs of the 997 IEEE Conference on Evolutonary Computaton pp. 443 446. [CT93] Chen J. L. and Tsao Y. C. (993). Optmal desgn of machne elements usng genetc algorthms. Journal of the Chnese Socety of Mechancal Engneers 4(2):93 99 993. [FFG9] Fu J.-F. Fenton R. G. and Cleghorn W. L. (99). A mxed nteger-dscrete-contnuous programmng method and ts applcaton to engneerng desgn optmzaton. Engneerng Optmzaton 7(4):263 280 99. ISSN 0305-254 [LC94] L H.-L. and Chou C.-T. (994). A global approach for nonlnear mxed dscrete programmng n desgn optmzaton. Engneerng Optmzaton 22():09 22 994. [LZW95] Ln Shu-Shun Zhang Chun and Wang Hsu-Pn (993). On mxed-dscrete nonlnear optmzaton problems: A comparatve study. Engneerng Optmzaton 23(4):287 300 995. ISSN 0305-25 [LP9a] Loh Han Tong and Papalambros P. Y. (99). A sequental lnearzaton approach for solvng mxed-dscrete nonlnear desgn optmzaton problems. Transactons of the ASME Journal of Mechancal Desgn 3(3):325 334 September 99. [LP9b] Loh Han Tong and Papalambros P. Y. (99). Computatonal mplementaton and tests of a sequental lnearzaton algorthm for mxed-dscrete nonlnear desgn optmzaton. Transactons of the ASME Journal of Mechancal Desgn 3(3):335 345 September 99. [Sto96a] Storn Raner (996). On the usage of dfferental evoluton for functon optmzaton. NAFIPS 996 Berkeley pp. 59-523. [SP95a] Storn Raner and Prce Kenneth (995). Dfferental evoluton - a smple and effcent adaptve scheme for global optmzaton over contnuous spaces. Techncal Report TR-95-02 ICSI March 995. (Avalable va ftp from ftp.cs.berkeley.edu/pub/techreports/995/tr-95-02.ps.z) [SP97a] Storn Raner and Prce Kenneth (997). Dfferental Evoluton A smple evoluton strategy for fast optmzaton. Dr. Dobb's Journal Aprl 97 pp. 8 24 and p. 78. [San90] Sandgren E. (990). Nonlnear nteger and dscrete programmng n mechancal desgn optmzaton. Transactons of the ASME Journal of Mechancal Desgn 2(2):223 229 June 990. ISSN 0738-0666 [TC97] Therauf G. and Ca J. (997). Evoluton strateges parallelsaton and applcaton n engneerng optmzaton. In B.H.V. Toppng (ed.) (997). Parallel and dstrbuted processng for computatonal mechancs. Saxe-Coburg Publcatons Ednburgh (Scotland). ISBN -874672-03-2 [WC95] Wu S.-J. and Chow P.-T. (995). Genetc algorthms for nonlnear mxed dscrete-nteger optmzaton problems va meta-genetc eter optmzaton. Engneerng Optmzaton 24(2):37 59 995. ISSN 0305-25 [ZW93] Zhang Chun and Wang Hsu-Pn (993). Mxed-dscrete nonlnear optmzaton wth smulated annealng. Engneerng Optmzaton 2(4):277 29 993. ISSN 0305-25
Reference data for ths document: [LZ99b] Joun Lampnen Ivan Zelnka (999). Mxed Integer-Dscrete-Contnuous Optmzaton By Dfferental Evoluton Part : the optmzaton method. In: Ošmera Pavel (ed.) (999). Proceedngs of MENDEL'99 5th Internatonal Mendel Conference on Soft Computng June 9. 2. 999 Brno Czech Republc. Brno Unversty of Technology Faculty of Mechancal Engneerng Insttute of Automaton and Computer Scence Brno (Czech Republc) pp. 7 76. ISBN 80-24-3-7.