Optmzaton of ntegrated crcuts by means of smulated annealng Jernej Olenšek, Janez Puhan, Árpád Bűrmen, Sašo Tomažč, Tadej Tuma Unversty of Ljubljana, Faculty of Electrcal Engneerng, Tržaška 25, Ljubljana, Slovena Key words: parametrc optmzaton, smulated annealng, desgn of ntegrated crcuts. Abstract The purpose of ths paper s to test the effcency of the modfed orthogonal smulated annealng algorthm. The method s compared wth the COMPLEX method on a set of mathematcal functons. The method s then used on three real-world cases of ntegrated crcuts and compared wth a modfed COMPLEX method that uses ntellgent ntal ponts selecton. Optmzacja ntegrranh vezj z algortmom smulranega ohlajanja Ključne besede: parametrčna optmzacja, smulrano ohlajanje, načrtovanje ntegrranh vezj. Povzetek Namen prspevka je prezkust učnkovtost modfcranega ortogonalnega smulranega ohlajanja. Prmerjamo ga z metodo COMPLEX na skupn matematčnh funkcj. Metoda je nato uporabljena na treh realnh prmerh ntegrranh vezj n prmerjana z modfcrano metodo COMPLEX, k uporablja pametno zbranje začetnh točk.
1 Introducton Optmzaton problems arse n vrtually every feld of engneerng, scence, and busness. The parametrc optmzaton problems are usually presented n the followng form: mn f f : R x x R n ( x) n R [ L, U ] (1) where f s the so-called cost functon (CF) and x s a vector of parameter values. L and U are vectors of lower and upper parameter bounds, respectvely. Unfortunately analytcal solutons to (1) can only be obtaned for some very smple and small problems. Most practcal problems are complex and often nclude smulatons and measurements, whch are very expensve and tme consumng. The complexty of the optmzaton problem depends on the dmensonalty (.e. the number of optmzaton parameters) and on the shape of the CF. The sze of the soluton space ncreases exponentally wth the problem dmensonalty, so locatng good solutons becomes ncreasngly more dffcult. But the real challenge arses from the CF tself. In most real-world applcatons the CF s nonlnear and has many local mnma. Often the value of the CF s a result of numercal smulatons or measurements that ntroduce nose to the CF. Nose makes the fast determnstc gradent based methods useless and dervatve free drect methods become more attractve. Drect methods are usually dvded n two major groups. Determnstc methods always produce the same fnal soluton when they start wth the same ntal guess. One method from ths group s the smplex method whch s well known and popular due to ts smplcty and speed. But the smplex method s a local downhll search method and ts soluton greatly depends on the ntal guess. Stochastc methods, on the
other hand, ntroduce randomness to the search process and are capable of escapng from the local mnma n order to fnd better solutons. Smulated annealng s a stochastc method. In ths paper we descrbe a recent verson of smulated annealng referred to as Orthogonal Smulated Annealng (OSA) [1] and compare t wth a modfed smplex method also known as COnstraned smplex (COMPLEX) [2]. The comparson s done on a set of mathematcal test functons. OSA and modfed COMPLEX methods are then used on three real-world ntegrated crcut (IC) desgn problems. The purpose of comparson s to establsh the feasblty of crcut optmzaton wth OSA. The paper s organzed as follows. In secton 2 a bref descrpton of the basc smulated annealng algorthm s gven and n secton 3 the OSA algorthm s descrbed n detal. Secton 4 compares the algorthm wth the COMPLEX method on a set of mathematcal test functons. In secton 5 OSA s compared wth a modfed COMPLEX methods on three cases of IC desgn. Secton 6 gves the conclusons. 2 Smulated annealng algorthm Downhll methods can easally get trapped n local mnma. To escape from a local mnmum uphll moves must be allowed from tme to tme to gve the algorthm a chance to move to unexplored parts of the soluton space. Smulated annealng [3] was developed for ths purpose. It always accepts downhll moves but occasonally uphll moves are also accepted. The basc features of the algorthm come from the analogy wth the movement of atoms n metal. When metal s heated up to a very hgh temperature, atoms can move freely even to a state wth hgher energy. When the materal s cooled down slowly, atoms are more lkely to move to low energy states. If the annealng s slow enough, the resultng metal has an unform structure
wth very few defects and mnmal free energy. The smulated annealng method mmcs ths process by ntroducng an artfcal parameter to the search process often referred to as the temperature (T ) whch controlls the acceptance probablty for the uphll moves. At the begnnng of the search t s set to a hgh value and most transtons to hgher CF values are accepted. As the search progresses, the temperature s slowly decreased so that the uphll moves become less frequent. If the annealng s done n a suffcently slow manner, the fnal solutons reached by the algorthm are near the global mnmum of the CF. The CF s an analogy of the free energy of the atoms n a metal. The basc steps of the smulated annealng algorthm are: 1. ntalze - set algorthm parameters, ntal pont 2. generate new pont - generaton mechansm 3. acceptance crteron transton 4. contnue wth 2 untl end of temperature stage 5. annealng - coolng schedule (decrease temperature) 6. contnue wth 2 untl stoppng condton s met These steps must be chosen carefully n order to ensure the probablstc convergence to the global optmum. The obtaned algorthms, however, are not very effcent n practce because the requred coolng schedules are too slow or the generaton mechansms are too neffcent to get any useful result n a reasonable amount of tme. That s why most practcal versons of the algorthm use modfed generaton mechansms and coolng schedules. Ths way the convergence proofs (.e. [5]) no longer apply but good solutons can stll be obtaned n a reasonable amount of tme.
3 Orthogonal smulated annealng algorthm (OSA) Recently a new verson of the smulated annealng algorthm was developed [1], takng advantage of a carefully desgned set of experments at every teraton that helps to choose a good pont for the next teraton. Snce the results reported n [1] were encouragng, ths method was chosen for mplementaton and testng. All steps of the algorthm are descrbed n ths secton. 3.1. Intalzaton In the ntalzaton step basc algorthm parameters are set. For ths purpose several ponts (n our case 100) are randomly chosen and evaluated. The best of these ponts s set as the startng pont for the algorthm. The ntal value of the temperature parameter T s set to the standard devaton of CF values at these ponts. Our method dffers slghtly from the orgnal one [1]. Instead of usng the same parameter (temperature) for the acceptance crteron and the generaton meachansm we use a separate parameter for generaton of random moves. The allowed ntervals [ L, U ] for dfferent optmzaton varables can vary consderably so the generaton mechansm must use a separate parameter for each varable. For ths reason we ntroduce another vector parameter referred to as the range ( R ). The ntal values of the components of R are set to allowed nterval wdths of optmzaton varables. Another parameter that needs to be set s the number of moves at each temperature stage N. In theory t must be large enough for the algorthm to reach thermal t equlbrum n every temperature stage. In our case N t s set to 10.
3.2. Generaton mechansm The algorthm uses orthogonal expermental desgn (OED) to choose good canddates for the next teraton. A carefully desgned set of experments allows for an effcent factor analyss. The man dea s to evaluate a small number of ponts n order to estmate factor effects on the gven CF. The selecton of these ponts s done wth the help of orthogonal arrays. In ths context orthogonal means statstcally ndependent so that the estmaton of the effect of one factor does not affect the estmaton of the effects of others. An example of such an expermental desgn for three factors and three levels per factor s gven n table 1, whch contans the generated orthogonal array and fgure 1, whch shows the dstrbuton of the correspondng expermental ponts. -- Table 1 -- -- Fg 1 -- N f There s a total of Q = 3 3 = 27 combnatons of factor levels for ths example, where Q s the number of levels per factor and N f s the number of factors. In our case only nne expermental ponts have to be evaluated and the use of the orthogonal array assures that these ponts are evenly spread around the search space. The algorthm for generaton of orthogonal arrays can be found n [4]. The use of orthogonal arrays also has drawbacks. The method works very well when there are no nteractons between dfferent factors. Unfortunately ths s usually not the case. Furthermore optmzaton problems often nclude many optmzaton varables so the number of requred experments for an effcent factor analyss s large. Therefore the N v varables are randomly grouped nto N f factors. The two most extreme cases are when N = N and N = 1. In the former case there are f v f
many factors but because of the nteracton effects between factors, the estmaton of factor effects s less accurate. In the latter case there s only one factor and the estmated effect s accurate, but the optmzaton requres more teratons. A compromse s needed. The formula for determnng the number of factors for a gven problem s: log ( 3 3 (2 1) 1) N v + / 2 N = (2) f At the begnnng of every teraton the varables are randomly dvded nto N f groups and every group s consdered as one factor. Then a random perturbaton vector s generated accordng to a specfed probablty dstrbuton (n our case the Cauchy dstrbuton). For every optmzaton parameter x the probablty dstrbuton of perturbaton dx s: R p 1,2,... N ( dx ) =, = π 2 2 v ( R + dx ) (3) where R s the range parameter of the -th varable n the current temperature stage. To generate a random varable from ths dstrbuton the nverson method s used: ( ( U 1/ 2) ), 1,2 N v dx = R tan π =,... (4) where U s an unformly dstrbuted random number from the nterval [0,1]. After generatng the perturbaton vector dx, the three levels for every optmzaton varable are determned as: x x x 0 1 2 = x = x c c = x c + dx, = 1,2,... N (5) dx v where c x s the current value of the -th varable. If x 1 or x 2 volates boxconstrants[ L, ], ts value s chosen randomly from ths nterval. Snce varables are U
grouped nto factors, settng one factor to some level means settng all the varables x from that factor to the correspondng level ( x, x or x ). Ths way the orthogonal 0 1 2 array s converted nto expermental ponts, whch are then evaluated. The man effects of all factor levels are obtaned by the followng formula: S, = y F (6) j k t t t where S, denotes the effect of the k -th level of the j -th factor and yt s the value of j k the CF from the t -th experment. F t has only two possble values. It s 1 f n the t -th experment the j -th factor has k -th level. Otherwse F t s 0. The new canddate soluton can now be generated. For every factor ( j ) the level ( k ) wth the mnmum S, s chosen. The CF of the new canddate soluton s then evaluated. The best of j k all the expermental ponts and the canddate soluton s then submtted to the acceptance crteron as a potental soluton for the next teraton. Ths process s repeated N t tmes n every temperature stage. 3.3. Acceptance crteron Most versons of the smulated annealng algorthm use the same transton acceptance crteron whch s known as the Metropols crteron. Downhll transtons are allways accepted. Uphll transtons are accepted wth the probablty: P = e y' y T (7) where y ' and y are the CF values at the new and the current pont, respectvely, and T s the value of the temperature parameter. At hgh temperatures almost all transtons are accepted but when the temperature s close to zero most of the uphll moves are rejected and the algorthm acts as a downhll method.
3.4. Annealng The next step of the algorthm s the coolng schedule. Several schedules have been developed but the best known and also very popular s the orgnal schedule of Krkpatrck [3]. The temperature decreases exponentally: ( k) = T ( k 1) α, α [ 0,1] T (8) where k s the temperature stage ndex. Large values of α mean slow convergence but more relable search for the global optmum whereas smaller values mean fast convergence wth the ncreased rsk of gettng trapped n a local mnmum. The emprcally chosen value for α was 0.99. At the end of every temperature stage the number of moves N t n a temperature stage s also decreased by α : ( k) = N ( k 1) α N t t (9) The probalty dstrbuton for random moves must also be adapted. The range parameter R s reduced at the end of every temperature stage: t ( k) Rt ( k 1 ), = 1,2 N v R = α,... (10) 3.5. Stoppng crteron Several stoppng crtera can be used. In our case the algorthm stops when the 6 temperature reaches user specfed mnmal value T mn (n our case10 ) or when the number of CF evaluatons exceeds the maxmum allowed number of evaluatons. 4 Optmzaton of mathematcal test functons Orthogonal smulated annealng was compared wth the COMPLEX method [2] on a set of mathematcal functons. The set ncludes unmodal functons, functons wth a
small number of local mnma (consdered as moderatelly dffcult problems) and dffcult problems wth many local mnma, nose, nonlnearty and strong nteractons between varables. All of the tested functons can be found n [4]. The optmzaton was repeated 50 tmes for every functon wth randomly chosen ntal ponts. Both methods had the same lmted number of CF evaluatons (50000 and 70000 for problems wth N = 30 and N = 100, respectvely). The optmzaton results are gven v v n table 2. -- Table 2 -- The results show that the OSA method s promsng when compared to COMPLEX. The COMPLEX method exhbts very fast convergence but gets stuck n a local mnmum n almost every tested case. It outperforms the OSA method n some cases of unmodal functons and functons wth strong nose. The latter s not unexpected snce the method mantans a populaton of ponts between teratons. Smulated annealng, on the other hand, starts every teraton from a sngle pont. On multmodal functons OSA outperformed the COMPLEX method n terms of global search capabltes. Due to the modfed generaton mechansm and coolng schedule the algorthm was not able to locate the global mnmum n all optmzaton runs, but the solutons that were found were stll farly good when compared to the global mnmum. 5 Optmzaton of ntegrated crcuts Snce the OSA performance on mathematcal functons was very promsng, the next step was to test t on real-world electronc crcut desgn problems and compare ts performance wth the COMPLEX method. For ths purpose SPICE OPUS crcut smulator was used [6]. In SPICE OPUS a modfed COMPLEX method s already
ntegrated as one of the avalable optmzaton methods. Snce the orgnal method has very fast convergence, restart wth ntellgent ntal ponts selecton s conducted every tme the basc method reaches ts stoppng crteron [7]. Ths process s repeated untll the gven lmt of CF evaluatons s reached. The fnal result s the best soluton of all the runs. The OSA method had to be mplemented n C langunage and added as one of the avalable optmzaton methods n SPICE OPUS. Three cases of electronc crcut desgn were consdered. The frst crcut was a smple delay element, the second an operatonal amplfer, and the thrd a rather complex amplfer crcut. Crcut topologes for all three cases are gven n fgures 2, 3, and 4. The key propertes of all three optmzaton problems are gven n table 3. -- Fg2 -- -- Fg3 -- -- Fg4 -- -- Table 3 -- Optmzaton parameters were resstances, capactances and transstor channel lengths, wdths, and multpler factors. For every crcut several desgn goals were set. A sngle CF s constructed as a combnaton of all the desgn goals [8]. Optmzaton s conducted across several corner ponts to account for dfferent envrenmental condtons (supply voltage, temperature, process parameter varatons,...). Snce every CF evaluaton requres a separate crcut smulaton for each corner pont, a large number of smulatons s expected resultng n very long run tmes. Therefore every crcut was optmzed only once. The results of the optmzaton are gven n table 4. -- Table 4 --
Snce the modfed COMPLEX method uses restarts and can explore several local solutons wthn the gven number of CF evaluatons, the number of the run n whch a soluton was found s gven n brackets. The number of CF evaluatons after whch the COMPLEX method was manually stopped, s also gven. The OSA method stopped automatcally when the temperature reached ts fnal value. Both tested methods were compared n terms of the soluton qualty and the number of CF evaluatons (FE). The frst case s the most smple of the three cases consdered. It only has a few desgn goals and does not nclude corner ponts. It also has the least optmzaton varables. All ths makes the soluton space smaller and the CF less complex. For ths case the modfed COMPLEX method performed consderably better than OSA. It dd however requre more CF evaluatons and several restarts to reach a good soluton. In the second case multple corner ponts and more desgn goals were consdered. OSA outperformed the modfed COMPLEX method n terms of soluton qualty and number of requred CF evaluatons. The thrd case has the largest number of optmzaton varables, desgn goals, and corner ponts. In ths case OSA was also more successfull than the modfed COMPLEX method. These results show that for smpler cases the modfed COMPLEX method clearly s a better choce. But when t comes to complex crcuts, many desgn goals, and, above all, a large number of corner ponts, t does not perform as good as OSA. Not even restarts helped the COMPLEX method to fnd a better solutons than the one OSA found n a sngle run. 6 Conclusons A recently developed optmzaton method called Orthogonal Smulated Annealng (OSA) s descrbed and compared aganst a verson of the smplex algorthm
(COMPLEX method). Both methods are frst tested on a set of mathematcal test functons. The results showed that OSA performs better when the CF has many local mnma. On the other hand, the COMPLEX method s a good choce when fndng a local mnmum quckly s more mportant than fndng a global mnmum. OSA and modfed COMPLEX method were then tested on three IC desgn cases. The results showed that on the smpler case the modfed COMPLEX method usng restarts outperformed the OSA method. As the problem complexty ncreased, the ablty of the OSA to explore the search space more thoroughly resulted n better performance (compared to the modfed COMPLEX method). But n order to obtan a good soluton n a reasonable amount of tme, probablstc global convergence of the algorthm had to be sacrfced (modfed generaton mechansm and coolng schedule). Therefore there s no guarantee as to when and f the global mnmum wll actually be found. Nevertheless OSA s well suted to IC optmzaton and desgn, partcularly for problems wth many varables and corner ponts. 7 Acknowledgment The research has been supported by the Mnstry of Hgher Educaton, Scence and Technology of Republc of the Slovena wthn programme P2-0246 - Algorthms and optmzaton methods n telecommuncatons. 8 References [1] L-Sun Shu, Shnn-Yng Ho, A Novel Orthogonal Smulated Annealng Algorthm for Optmzaton of Electromagnetc Problems, IEEE transactons on magnetcs, Vol. 40, No. 4, pp. 1790-1795, July 2004.
[2] M.J. Box, A new method of constraned optmzaton and a comparson wth other methods, Computer Journal, Vol. 8, pp. 42-52, 1965. [3] S. Krkpatrck, C.D. Gelatt, M.P. Vecch, Optmzaton by smulated annealng, Scence, Vol. 220, pp. 1277-1292,1983. [4] Yu-Wng Leung, An orthogonal Genetc Algorthm wth Quantzaton for Global Numercal Optmzaon, IEEE transactons on evolutonary computaton, Vol. 5, No. 1, pp. 41-53, 2001. [5] R.L. Yang, Convergence of smulated annealng algorthm for contnuous global optmzaton, Journal of optmzaton theory and applcatons, Vol. 104, No. 3, pp. 691-716, 2000. [6] SPICE OPUS crcut smulator homepage: URL: http://www.fe.un-lj.s/spce/, Faculty of Electrcal Engneerng, Electronc Desgn Automaton Laboratory: URL: http://www.fe.un-lj.s/edalab/\hspace{1mm}. [7] Puhan J, Bűrmen A, Tuma T. Analogue ntegrated crcut szng wth several optmzaton runs usng heurstcs for settng ntal ponts, Canadan journal of electrcal and computer engneerng, Vol. 28 (3-4): pp. 105-111 JUL-OCT 2003. [8] Bűrmen A, Strle D, Bratkovč F, Puhan J, Fajfar I, Tuma T. Automated robust desgn and optmzaton of ntegrated crcuts by means of penalty functons, AEU- Internatonal journal of electroncs and communcatons, Vol. 57 (1), pp. 47-56, 2003.
unv. dpl. ng. el. Jernej Olenšek Unverza v Ljubljan Fakulteta za elektrotehnko Tržaška 25, SI-1000 Ljubljana E-mal: jernej.olensek@fe.un-lj.s Tel: (01) 4768 724 doc. dr. Janez Puhan Unverza v Ljubljan Fakulteta za elektrotehnko Tržaška 25, SI-1000 Ljubljana E-mal: janez.puhan@fe.un-lj.s Tel: (01) 4768 322 doc. dr. Árpád Bűrmen Unverza v Ljubljan Fakulteta za elektrotehnko Tržaška 25, SI-1000 Ljubljana E-mal: arpadb@fdes.fe.un-lj.s Tel: (01) 4768 322 prof. dr. Sašo Tomažč Unverza v Ljubljan Fakulteta za elektrotehnko Tržaška 25, SI-1000 Ljubljana E-mal: saso.tomazc@fe.un-lj.s Tel: (01) 4768 432 zr. prof. dr. Tadej Tuma Unverza v Ljubljan Fakulteta za elektrotehnko Tržaška 25, SI-1000 Ljubljana E-mal: tadej.tuma@fe.un-lj.s Tel: (01) 4768 329
experment number factor 1 level factor 2 level 1 1 1 1 2 1 2 2 3 1 3 3 4 2 1 2 5 2 2 3 6 2 3 1 7 3 1 3 8 3 2 1 9 3 3 2 Table 1: Orthogonal array for 3 factors and 3 levels per factor. factor 3 level Fg. 1: Dstrbuton of expermental ponts when the orthogonal array from Table 1 s used.
N COMPLEX OSA global v mnmum f 30-9596.2-12569.5-12569.5 1-3012.3-12569.5 f 30 6.9709 3.5527 10-14 0 2 224.67 1.9899 f 30 1.6714 10-2 1.5234 10-7 0 3 3.93447 2.7337 10-7 f 30 2.2362 10-5 1.7375 10-13 0 4 3.04840 1.6971 10-1 f 30 2.1584 10-5 1.6672 10-16 0 5 1.59433 2.0732 10-1 f 30 1.9551 10-5 3.1601 10-15 0 6 6.99148 1.0987 10-2 f 100-61.124-98.861-99.2784 7-29.261-97.860 f 100-70.408-78.332-78.33236 8-63.311-78.331 f 100 113.13 283.59 0 10 196.95 795.04 f 30 2.5981 10-8 5.4955 10-14 0 11 5.3587 10-5 8.2124 10-13 f 30 4.1618 10-3 2.4360 10-2 0 12 2.0489 10-2 1.5660 10-1 f 30 1.9705 10-1 1.8169 10-7 0 13 3.6034 8.2303 10-7 f 30 8.9943 10-2 102.83 0 14 39.777 1928.0 f 30 1.8026 1.2809 10-5 0 15 9.4910 8.4080 10-1 Table 2. Table shows the best and the worst soluton found n 50 optmzaton runs. The functons used are defned n [4].
Fg. 2: Topology of the frst crcut. Fg 3: Topology of the second crcut.
Fg. 4: Topology of the thrd crcut. case N v desgn goals corner ponts 1 12 7 1 2 15 14 14 3 17 32 17 Table 3: Summary of the optmzaton cases: number of optmzaton parameters, number of desgn goals, and number of corner ponts.
case mdfed COMPLEX OSA 1 FE untl cost < 100 10 3 FE untl cost < 20 10 3 best cost FE untl best cost fnal FE 253 (1) 660 (1) 6.39 10 3 21409 (28) > 100 000 1011 12758 11.3 10 3 14703 16122 2 FE untl cost < 50 FE untl cost < 10 best cost FE untl best cost fnal FE 124 (1) 2483 (2) 8.07 96912 (69) > 100 000 58 33595 7.37 47605 47768 3 FE untl cost < 10 FE untl cost < 1 best cost FE untl best cost fnal FE 3672 (3) 26688 (21) 0.282 41131 (32) > 45 000 32014 34248 0.088 43877 44164 Table 4: Optmzaton results: number of functon evaluatons (FE) to fnd a soluton of the gven qualty, and fnal solutons. For modfed COMPLEX method the number of the run n whch a soluton was found, s also gven n brackets.