New Bundle Methods for Solving Lagrangian Relaxation Dual Problems 1

Transcription

1 JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 113, No. 2, pp , May 2002 ( 2002) New Bundle Methods for Solvng Lagrangan Relaxaton Dual Problems 1 X. ZHAO 2 AND P. B. LUH 3 Communcated by W. B. Gong Abstract. Bundle methods have been used frequently to solve nonsmooth optmzaton problems. In these methods, subgradent drectons from past teratons are accumulated n a bundle, and a tral drecton s obtaned by performng quadratc programmng based on the nformaton contaned n the bundle. A lne search s then performed along the tral drecton, generatng a serous step f the functon value s mproved by ( or a null step otherwse. Bundle methods have been used to maxmze the nonsmooth dual functon n Lagrangan relaxaton for nteger optmzaton problems, where the subgradents are obtaned by mnmzng the performance ndex of the relaxed problem. Ths paper mproves bundle methods by makng good use of near-mnmum solutons that are obtaned whle solvng the relaxed problem. The bundle nformaton s thus enrched, leadng to better search drectons and less number of null steps. Furthermore, a smplfed bundle method s developed, where a fuzzy rule s used to combne lnearly drectons from near-mnmum solutons, replacng quadratc programmng and lne search. When the smplfed bundle method s specalzed to an mportant class of problems where the relaxed problem can be solved by usng dynamc programmng, fuzzy dynamc programmng s developed to obtan effcently near-optmal solutons and ther weghts for the lnear combnaton. Ths method s then appled to job shop schedulng problems, leadng to better performance than prevously reported n the lterature. Key Words. Lagrangan relaxaton, bundle methods, nonsmooth optmzaton, manufacturng schedulng. 1 Ths work was supported n part by the Natonal Scence Foundaton under Grants DMI and ECS Member of the Advanced Techncal Staff, I2 Technologes, Cambrdge, Massachusetts. 3 Professor, Department of Electrcal and Computer Engneerng, School of Engneerng, Unversty of Connectcut, Storrs, Connectcut Plenum Publshng Corporaton

2 374 JOTA: VOL. 113, NO. 2, MAY Introducton Integer optmzaton problems are generally dffcult to solve because of ther nherent combnatoral complexty, and Lagrangan relaxaton has been a powerful approach to obtan near-optmal solutons. In Lagrangan relaxaton, certan constrants are frst relaxed through the ntroducton of Lagrangan multplers. The relaxed problem s easer than the orgnal one and can be solved effcently f t belongs to Class P. Multplers are then teratvely adjusted based on the level of constrant volaton. The dual functon s maxmzed n ths multpler updatng process, and the values of the dual functon serve as lower bounds to the optmal feasble cost (Ref. 1). At the termnaton of such updatng teratons, smple heurstcs are appled to adjust the relaxed problem solutons to form a feasble result satsfyng all the constrants. A major challenge n Lagrangan relaxaton s to maxmze effectvely the dual functon, whch s concave, pecewse lnear, and conssts of many facets. The subgradent method s commonly used, where a subgradent can be obtaned by mnmzng the relaxed problem, and the multplers are updated along ths drecton (Ref. 2). However, the multplers often zgzag across rdges (ntersectons of two or more facets) of the dual functon, and requre many teratons to reach an optmum. Recently, the surrogate subgradent method (SSG) was developed, where only an approxmate optmzaton of the performance ndex of the relaxed problem s needed to obtan a proper surrogate subgradent drecton to update the multplers (Refs. 3 4). Compared wth methods that take effort to fnd good drectons, ths method obtans drectons wth much less effort and provdes a new approach for solvng large problems. Bundle methods represent a qute dfferent approach for nonsmooth optmzaton, and am to fnd an (-ascent drecton along whch a functon value can ncrease by at least ( (Ref. 5). In these methods, subgradents from past teratons are accumulated n a bundle, and a tral drecton s obtaned by quadratc programmng based on the bundle nformaton. Lne search s then performed along the tral drecton, generatng a serous step f the functon value s mproved by ( or a null step otherwse. Snce a serous step s generated at the cost of quadratc programmng and lne search, wth the possblty of havng multple null steps n-between two serous steps, much computaton s requred. The Lagrangan relaxaton framework and the methods for solvng the nonsmooth dual problem as dscussed above wll be presented brefly n Secton 2. Based on the nsghts obtaned from these methods, (-mnmum solutons of the relaxed problem and (-surrogate subgradents are ntroduced, and the mproved bundle method s developed n Secton 3. The key

3 JOTA: VOL. 113, NO. 2, MAY dea s to make good use of all the nformaton obtaned durng the mnmzaton of the performance ndex of the relaxed problem, not just the mnmum soluton but also near-mnmum solutons. The bundle nformaton s thus enrched, leadng to better search drectons and less number of null steps. To reduce the computatonal requrements n solvng large problems, a smplfed bundle method s developed n Secton 4 to decrease the dstance to the optmal soluton nstead of requrng the functon value to be ncreased by (. In ths way, the quadratc programmng and lne search requred by tradtonal bundle methods are no longer necessary, and a fuzzy rule s establshed to combne lnearly the subgradents assocated wth the near-optmal solutons of the relaxed problem. When the smplfed bundle method s specalzed to an mportant class of problems where the relaxed problem can be solved by usng dynamc programmng (DP), fuzzy dynamc programmng (FZDP) s developed to obtan effcently near-optmal solutons and ther weghts for the lnear combnaton. The convergence of the method s proved, and the testng result presented n Secton 5 on a benchmark problem shows that the smplfed bundle method generates a smlar result but wth less computatonal requrements as compared to results prevously obtaned by usng tradtonal bundle methods. Fuzzy dynamc programmng s then appled to job shop schedulng problems n Secton 6, where the complexty of FZDP can be reduced further by explotng the specal structure of the relaxed problem. Testng results show that the smplfed bundle FZDP method leads to better performance as compared wth result prevously reported n the lterature. Ths method s generc for separable nteger or mxed nteger optmzaton problems beyond job shop schedulng, and provdes a powerful approach to solve large-scale dual problems wthn the Lagrangan relaxaton framework. 2. Problem Descrpton and Formulaton The Lagrangan relaxaton framework and several methods to solve the dual problem are ntroduced n ths secton Lagrangan Relaxaton. An nteger optmzaton problem can be descrbed as follows: (IP) mn x J(x), (1) s.t. g(x) 0 and x X. (2)

4 376 JOTA: VOL. 113, NO. 2, MAY 2002 Here, x s an nb1 decson varable belongng to the nteger space XGZ n. The constrant g(x) s an mb1 functon, and the cost J(x) s a scalar functon. In the Lagrangan relaxaton approach, the constrants g(x) 0 are relaxed by ntroducng the mb1 multpler vector λ and the Lagrangan functon L (x, λ) J(x)Cλ T g(x). (3) The relaxed problem s to mnmze L (x, λ) over X, resultng n the concave nonsmooth dual functon L(λ), L(λ) mn L (x, λ). (4) x X The dual problem s to maxmze the dual functon (Ref. 6), max L(λ), (5) λ 0 wth the optmal dual soluton denoted by λ * and the optmal dual value denoted by L* GL(λ*). In most cases, the optmzaton n (5) s performed teratvely, and at the termnaton of such teratons, a smple heurstcs s appled to adjust the relaxed problem solutons so as to form a feasble result satsfyng all the constrants Methods for the Dual Problem. Subgradent Method. Snce the dual problem s nondfferentable for nteger optmzaton problems, the subgradent method s commonly used to maxmze the dual functon. In order to get a subgradent drecton, a mnmum soluton for the relaxed problem s obtaned, x k Garg mn L (λ k, x), (6) x X where k s the teraton ndex. A subgradent s then calculated based on ths mnmum soluton, g k Gg(x k ). (7) In the subgradent method, the multplers are updated along the subgradent drecton, λ kc1 Gλ k Cs k g k, (8) where the stepsze s k satsfes 0Fs k F2(L*AL k ) g k 2. (9) The method requres the mnmzaton of the performance ndex of the relaxed problem to obtan a subgradent and wll reduce the dstance to the

5 JOTA: VOL. 113, NO. 2, MAY optmal pont step-by-step (Ref. 7). However, t may suffer from the dffculty of zgzaggng across rdges of the dual functon (Ref. 2). Surrogate Subgradent Method. To overcome the above-mentoned dffcultes and to solve more effcently large nteger optmzaton problems, the surrogate subgradent method was recently developed. It provdes a new approach to speed up convergence by reducng the effort to obtan a drecton. Instead of mnmzng the performance ndex of the relaxed problem to obtan an optmal soluton, only an approxmate mnmzaton s performed where the new terate x(λ k ) should satsfy the followng condton for the gven set of multplers λ k (Ref. 3): x(λ k ) {x L (λ k, x)fl (λ k, x ka1 ), x X}. (10) The surrogate subgradent drecton g k s calculated based on an approxmate soluton x k, g kgg(x k ), (11) and the multplers are updated along the surrogate subgradent drecton. It has been proved that the dstance to the optmal pont s reduced stepby-step under the followng stepsze rule: 0Fs k F(L*AL k) g k 2. (12) Note that the upper bound n (12) for the surrogate subgradent method s half of the bound n (9) for the gradent method, ndcatng that, wthout the accurate x k, the stepszng rule should be more conservatve. Nevertheless, wth the approxmate mnmzaton employed n the relaxed problem, the computatonal complexty to obtan a drecton can be much reduced. For example, when the relaxed problem can be decomposed nto N subproblems, only one subproblem needs to be solved to satsfy (10). Ths specalzed verson s the nterleaved dea of Ref. 4, and the effort to obtan a drecton s 1 N of that requred by a tradtonal subgradent method. Bundle Method. The bundle method has been a powerful approach for maxmzng nonsmooth concave functons (Ref. 5). It employs a concept called the (-subdfferental, defned as ( L(λ) {g R m L(λ ) L(λ)C g, λ Aλ C(, λ R m }. (13) Elements n ( L(λ) are called (-subgradents. Correspondngly, the (-drectonal dervatve along the drecton d at λ s defned as L ( (λ, d) sup [L(λCtd)AL(λ)A(] t. (14) th0

6 378 JOTA: VOL. 113, NO. 2, MAY 2002 It has been shown that L ( (λ, d)g nf g d. (15) g (L(λ) From (15), f a drecton d can be found such that L ( (λ, d)h0, then the dual cost can be ncreased by at least (. Therefore, t s desrable to select a search drecton d*such that the drectonal dervatve s maxmzed,.e., d*garg {max L ( (λ, d)} d G1 Garg {max Garg { Garg { nf d G1 g (L(λ) nf g (L(λ) d G1 nf g (L(λ) g d} max g d} g }. (16) Therefore, ths d* s the(-subgradent wth the smallest norm. Generally, snce the (-subdfferental s very dffcult to obtan, the dea of the bundle method s to accumulate subgradents of the past terates n a bundle BG{g 1, g 2,...,g b } and to approxmate ( L(λ) by the convex hull of the bundle elements, P b G g gg b G1 α g, g B,0 α, b G1 where e s the lnearzaton error for element, α G1, b G1 α e (, (17) e GL(λ )C g, λaλ AL(λ). (18) A drecton n P b that has the smallest norm s obtaned by usng quadratc programmng, and a lne search s then performed along ths tral drecton. If a pont n the tral drecton leads to an (-ascent of the functon value, λ k s updated to the new pont, resultng n a serous step. If P b s not adequate enough to approxmate ( L(λ), the tral drecton may not be an (-ascent drecton. In ths case, λ k s not updated and the new pont s added to the bundle, resultng n a null step. Null steps generate more subgradents near λ k so that P b becomes closer to ( L(λ); however, several null steps may be requred before a serous step s obtaned. The approach descrbed above s sometmes referred to as the dual form of the bundle methods. There are also prmal forms of the bundle methods derved from stablzed cuttng-plane methods (Ref. 8). Bundle methods have been used to maxmze nonsmooth Lagrangan dual functons

7 JOTA: VOL. 113, NO. 2, MAY so as to provde better drectons than those of the subgradent method (Ref. 9). The quadratc programmng and lne search nvolved, however, requre much computaton. Recently, second-order nformaton was explored for bundle methods so as to mprove the convergence rate at the cost of even more computaton (Ref. 10). 3. Improved Bundle Methods Bundle methods are developed based on the assumpton that one subgradent can be obtaned for a gven λ. When ths method s appled to maxmze the dual functon wthn the Lagrangan relaxaton framework, the dual value and the subgradent are obtaned by mnmzng the performance ndex of the relaxed problem (6). However, solvng the relaxed problem s problem dependent and can be qute tme consumng. For example, t has been reported that around 80% of the total CPU tme s spent on solvng the relaxed problem n job shop schedulng (Ref. 11). In addton, multple null steps may be needed to accumulate neghborhood subgradents f P b s not a good approxmaton of ( L(λ). Snce the relaxed problem must be solved at least once for every null step, t s desrable to approxmate ( L(λ) wth as small number of null steps as possble. Wthn the Lagrangan relaxaton framework, t wll be shown that near-optmal solutons of the relaxed problem contan valuable nformaton, whch can be used to mprove the approxmaton of ( L(λ), thereby reducng the number of null steps needed. To be precse, defne the set of (-optmal solutons for the relaxed problem as X Z (λ, () {x L (λ, x )AL(λ) (, x Z n }, (19) where x s an (-optmal soluton. Usually, these (-optmal solutons are byproducts when the relaxed problem s solved and can be obtaned wthout much effort. Accordngly, the (-surrogate subdfferental s defned as L ((λ) α g(x ) x X Z (λ, (), and α H0, α G1. (20) In the followng theorem, t wll be shown that (-surrogate subgradents belong to (-subdfferentals. Any (-surrogate subgradent belongs to the (-subdffer- Theorem 3.1. ental,.e., L ((λ) ( L(λ). (21)

8 380 JOTA: VOL. 113, NO. 2, MAY 2002 Proof. satsfes Based on the defnton of (19), an (-optmal soluton x L (λ, x )AL(λ) (. (22) From the defnton of the dual functon n (4), we have that, for any λ, L(λ )Gmn x Z n {J(x)Cλ Tg(x)} J(x )Cλ Tg(x ). (23) From the defnton of the relaxed problem n (3), we have L (λ, x )GJ(x )Cλ T g(x )GJ(x )Cλ Tg(x )A(λ Aλ) T g(x ), (24) whch can be rewrtten as J(x )Cλ Tg(x )GL (λ, x )C(λ Aλ) T g(x ). (25) Combnng (22), (23), (25), we have L(λ ) L (λ, x )C(λ Aλ) T g(x ) L(λ)C(λ Aλ) T g(x )C(. (26) Thus, the drecton g(x ) related to an (-optmal soluton x satsfes L(λ ) L(λ)C g(x ), λ Aλ C(, λ. (27) Based on the defnton of ( L(λ) n (13), consequently, g(x ) ( L(λ); gg α g(x ) L ( (λ). Theorem 3.1 states that (-optmal solutons provde valuable nformaton and that the related (-surrogate subgradents belong to ( L(λ). Snce the bundle method uses P b to approxmate ( L(λ), L ((λ) can be added to P b to obtan a better approxmaton. In fact, for convex problems wth real decson varables (.e., x R n as opposed to x Z n ) and lnear constrants, t can be proved that P b tself s contaned n L ((λ). Theorem 3.2. lnear constrants, Proof. For a convex problem wth real decson varables and P b L ((λ). (28) gg b From the defnton of P b n (17), gven any g P b, we have G1 α g, b α G1, (29) G1 b α e (. (30) G1

9 JOTA: VOL. 113, NO. 2, MAY For each bundle element g, there s a correspondng λ, and the assocated mnmum soluton x satsfes g Gg(x ), (31) L(λ )GJ(x )C g, λ. (32) Accordng to (18), we have e GL(λ )C g, λaλ AL(λ). (33) Combnng (32) and (33), we have e GJ(x )C g, λ C g, λaλ AL(λ) GJ(x )C g, λ AL(λ). (34) Ths can be rewrtten as J(x )C g, λ GL(λ)Ce. (35) Snce J(x) s convex and g(x) s lnear, the followng s true for α x : L α x, λ GJ α x Cg α x T λ Combned wth (35) and (30), we have α J(x )C α g, λ. (36) L α x, λ α (J(x )C g, λ )G α (L(λ)Ce ) L(λ)C(. (37) For the case wth real decson varables, the set of (-optmal soluton s defned as X R (λ, () {x L (λ, x)al(λ) (, x R n }, (38) and the correspondng (-surrogate subdfferental s L ((λ) {g(x), x X R (λ, ()}. (39) From (37), α x X R (λ, (), therefore, (28) s proved. gg α g Gg α x L ((λ);

10 382 JOTA: VOL. 113, NO. 2, MAY 2002 Fgure 1. Relatonshp among bundle P b, (-surrogate subdfferental L ((λ), and (-subdfferental L ( (λ). From the above theorems, the relatonshp among the bundle P b, (- surrogate subdfferental L ((λ), and (-subdfferental L ( (λ) s summarzed n Fg. 1. Snce t s not guaranteed that α x X Z (λ, (), Theorem 3.2 s not true for general nteger optmzaton problems. For nteger optmzaton, the dual functon s a pecewse lnear concave functon wth many facets, and each facet corresponds to a soluton of the relaxed problem. A near-mnmum soluton for a gven λ k s assocated wth a facet that s close to the facet correspondng to the mnmum soluton as llustrated n Fg. 2. In ths fgure, x 3 s the mnmum soluton gven λ k and x 2 s a near-mnmum soluton. Ther correspondng facets are close to each other around λ k, and the gradent assocated wth x 2 belongs to L ((λ k ). When P b s not a good approxmaton of L ( (λ), the bundle methods accumulate subgradents from nearby facets wth several null steps. Snce most neghborhood subgradents are n fact already contaned n L ((λ), some null steps may not be necessary f the nformaton n L ((λ) s utlzed fully. Therefore, by combnng L ((λ) wth P b, the bundle method can be enrched, leadng to better tral drectons and less number of null steps. Fgure 2. Near-mnmum soluton and nearby ntersectng facet.

11 4. Smplfed Bundle Method JOTA: VOL. 113, NO. 2, MAY Relaxed Convergence Condton. Compared wth tradtonal bundle methods, the mproved bundle method provdes better drectons by enrchng the bundle wth (-surrogate subdfferental. Quadratc programmng and lne search, however, are stll requred to guarantee the algorthm convergence. Ths s because both methods try to mprove the objectve functon by at least ( for each serous step. Instead of mprovng the functon value, another approach s to reduce the dstance between the current terate to an optmal soluton step-by-step. It wll be shown n ths secton that the convergence condtons for such an algorthm wll be much relaxed. Gven the current terate λ, defne the optmal drecton to be the drecton emanatng from λ to λ *,.e., g*(λ) λ*aλ. We have the followng theorem. Theorem 4.1. Any drecton g P b or g L ((λ) s at an acute angle wth the optmal drecton f ( s suffcently small;.e., f 0F[L*AL (λ)] 2Fg T (λ*aλ), (40) (F[L*AL (λ)] 2. (41) Proof. For any g P b, gg b G1 α g. From (35), wthn the proof of Theorem 3.2, we have L (λ, x )GJ(x )C g, λ GL(λ)Ce, (42) where x s the mnmum soluton for λ assocated wth the bundle element g. Snce a mnmzaton s performed n dervng L(λ), L (λ, x ) s greater than or equal to the dual;.e., L (λ) L (λ, x ), x and λ. (43) The above s also true at λ *,.e., L*GL (λ*) L (λ*, x ). (44) Then, from (42), we have L*AL (λ)ae GL*AL (λ, x ) L (λ*, x )AL (λ, x ), (45)

12 384 JOTA: VOL. 113, NO. 2, MAY 2002 whch from (3) can be wrtten as L*AL (λ)ae g ( x ) T (λ*aλ). (46) Combnng (30), (41), (46) one obtans 0F[L*AL (λ)] 2FL*AL (λ)a( F α (L*AL (λ)ae ) F α g T (λ*aλ). (47) Equaton (40) s thus proved for g P b. For any g L ((λ), gg α g ( x ), where x s an (-optmal soluton satsfyng L (λ, x )AL(λ) (. (48) Gven (39) and (41), we have L (λ, x ) (CL(λ) [L*AL(λ)] 2CL (λ)g[l*cl (λ)] 2. (49) From (41), (49), (4), we have 0F[L*AL (λ)] 2 L*AL (λ, x ) L (λ*, x )AL (λ, x ), (50) whch can be rewrtten as 0F[L*AL (λ)] 2 g(x ) T (λ*aλ); (51) therefore, (40) s proved for g L ((λ). Theorem 4.1 states that any drecton g P b or g L ((λ) s at an acute angle wth the drecton pontng to λ *,f( s suffcently small as llustrated n Fg. 3. In the followng, t wll be shown that the dstance to the optmal λ * can be reduced step-by-step. Fgure 3. Surrogate subgradent formng an acute angle wth the optmal drecton.

13 wth and f Theorem 4.2. JOTA: VOL. 113, NO. 2, MAY If the multplers are updated as follows: λ kc1 Gλ k Cs k g k, (52) g k L ((λ k ) or g k P b, (53a) 0Fs k F(L*AL (λ k )) g k 2, then the multplers move closer to an optmal λ * step-by-step;.e., (53b) λ*aλ kc1 F λ*aλ k, for all k. (54) Proof. From (52), λ*aλ kc1 2 G λ*aλ k 2 A2s k (λ*aλ k ) T g k C(s k ) 2 g k 2. (55) Combnng wth (40), the above yelds [for brevty, we set L k L(λ k )] λ*aλ kc1 2 λ*aλ k 2 As k (L*AL k )C(s k ) 2 g k 2. (56) It can be rewrtten as λ*aλ kc1 2 λ*aλ k 2 As k [(L*AL k )As k g k 2 ]. (57) For the range of stepszes n (53), the term n the par of brackets s greater than zero; thus, (54) s proved Smplfed Bundle Method. Smlar to the convergence proof of subgradent methods, the above theorems demonstrate that quadratc programmng and lne search are not necessary to guarantee algorthm convergence. Therefore, t s possble to develop smpler and more flexble algorthms. For example, we can use the stepszng rule (53) nstead of the lne search. Furthermore, nstead of obtanng the drecton wth the mnmum norm by usng quadratc programmng, t wll be much easer to get a drecton wth a small norm by usng some sensble rules. In fact, any drecton g P b or g L ((λ) s elgble. One ntutvely appealng dea s to form a fuzzy set of near-optmal solutons and defne the membershp ᾱ of a near-optmal soluton x to be the closeness of L (x, λ) tol(λ), e.g., ᾱ [L(λ)C(AL (x, λ)] (, wth x X Z (λ, (), (58a) ᾱ 0, otherwse. (58b) A fuzzy gradent s then obtaned by combnng lnearly the gradents of all the elements n the fuzzy set, wth the weght of an element equal to ts

14 386 JOTA: VOL. 113, NO. 2, MAY 2002 normalzed fuzzy membershp,.e., gg α g(x ), wth α ᾱ ᾱ. (59) Combnng the deas obtaned n Sectons 3 and 4, a smplfed bundle method can be constructed. In ths method, the drectons L ((λ) from the near-optmal solutons form the bundle, and a smple fuzzy rule such as (59) s used to obtan a search drecton. For smplcty of computaton, the stepszng rule (53) s appled nstead of a lne search to update the multplers Fuzzy Dynamc Programmng. The above smplfed bundle method provdes a framework to utlze (-optmal solutons of the relaxed problem, assumng that these solutons are avalable. In certan cases, t may not be easy to obtan (-optmal solutons f the relaxed problem s complcated. In the followng, we wll present how to mplement the fuzzy dea for an mportant class of problems where the relaxed problem can be solved by usng dynamc programmng (Ref. 12). Ths s done by explotng the closedloop nature of DP solutons where near-optmal solutons are obtanable wthout much addtonal effort than that needed for obtanng an optmal soluton. To be precse, the standard DP s frst ntroduced. Dynamc Programmng. Suppose that the relaxed problem s a dscrete-tme and dscrete-state optmal control problem descrbed by x {t}c1 Gf t (x t, u t ), for tg0,...,ta1, wth the ntal state x 0 gven. The objectve functon to be mnmzed s LGg T (x T )C TA1 g t (x t, u t ). tg0 In the above, t s the stage or tme ndex, x t the state, and u t the control that governs the state transton; g t (x t, u t ) s the stagewse cost and g T (x T ) s the termnal cost. The structure of such a problem s llustrated n Fg. 4, where the optmal states are represented by gray nodes and the optmal controls are marked wth weght 1. A standard backward DP begns at the last stage wth the termnal cost V T (x T )Gg T (x T ) computed. It then moves backward to the prevous stage. For a gven state, the cost-to-go functon for each possble control s calculated as Ṽ t (x t, u t )Gg t,(x t, u t )CV tc1 ( f t,(x t, u t )), (60)

15 JOTA: VOL. 113, NO. 2, MAY Fgure 4. Structure of dynamc programmng. where the cost-to-go functon Ṽ t (x t, u t ) s the sum of the stagewse cost and the assocated mnmum cost-to-go startng from the next stage. The mnmum cost-to-go functon V t (x t ) s then obtaned by mnmzng Ṽ t (x t, u t ) over the set of all possble controls U t (x t ),.e., V t (x t )G mn Ṽ t (x t, u t ) u t U t(x t) G mn {g t (x t, u t )CV tc1 ( f t (x t, u t ))}. (61) u t U t(x t) The backward calculaton proceeds from the last stage to the frst stage, and the mnmum cost-to-go at the frst stage for the gven ntal state x 0 s the mnmum cost. A forward sweep s then used to fnd the optmal path from the frst stage for the gven x 0 to the last stage as follows: u* t (x t )Garg mn Ṽ t (x t, u t ), tg0,...,ta1, (62) u t U t(x t) x* tc1 Gf t (x* t, u* t (x* t )), wth x* 0 Gx 0, tg0,...,ta1. (63) Based on u* 0 (x 0 ) for the gven x 0, t proceeds to the state at the next stage. Ths process contnues untl the last stage s reached. Fuzzy Dynamc Programmng. Usually, only one optmal path s obtaned n DP, and these x* t and u* t are used to calculate a subgradent. However, n general, there could be many near-optmal or optmal paths, each assocated wth an (-mnmum soluton or a mnmum soluton. In vew of the closed-loop nature of DP, many (-optmal paths can be obtaned as byproducts when the DP procedure s performed wthout much addtonal efforts. Fuzzy dynamc programmng (FZDP) s thus to assgn weghts to states and controls assocated wth these (-optmal paths accordng to fuzzy rules, and to compute the correspondng fuzzy gradent to utlze fully the

16 388 JOTA: VOL. 113, NO. 2, MAY 2002 Fgure 5. Structure of fuzzy dynamc programmng. nformaton contaned n near-optmal solutons. FZDP can be mplemented effcently by properly modfyng the DP procedure as llustrated n Fg. 5, where proper weghts are assgned to near-optmal controls. In FZDP, the backward calculaton of the mnmum costs-to-go s the same as that n DP. The forward sweep begns at the ntal state wth the state weght w(x 0 )G1. Gven a state wth a postve weght, all possble transtons to the next stage are assgned control weghts w(x t, u t ) followng (58), [V(x w (x t, u t )G t )C( AṼ(x t, u t )] (, fṽ(x t, u t )FV(x t )C(, (64) 0, otherwse, w(x t, u t )Gw (x t, u t ) w (x t, u t ), tg0,...,ta1. (65) U t(x t) To obtan (-mnmum solutons for the relaxed problem, the parameter ( n (64) s arbtrarly set to be ( G( T, where ( s equally dvded among the T stages. Gven a state weght w(x ta1 ) at stage ta1 and the assocated control weght w(u ta1 ), the state weght at stage t can be calculated as w(x t )G (w(x ta1, u ta1 ) w(x ta1 )), (66a) s.t. x t Gf ta1 (x ta1, u ta1 ). (66b) The process then repeats untl the termnal stage s reached. Consequently, states related to near-optmal paths are assgned wth state weghts {w(x t )} and controls wth control weghts {w(x t, u t )}. Such nformaton can be used to calculate a fuzzy gradent followng (59), and an example on job shop schedulng wll be presented n Secton 6. Compared to DP, addtonal computatons are requred n the forward sweep of FZDP. Assumng that there are K S states per stage and there are K C controls per state, the complexty of the forward sweep n the worst case

17 JOTA: VOL. 113, NO. 2, MAY s O(K S K C T), whch s smlar to the complexty of the backward calculaton. Snce there may be many states wth zero weghts, the addtonal computaton s usually nsgnfcant as compared to the tme-consumng backward sweep. In FZDP, any path wth a postve weght s guaranteed to be an (- optmal soluton. However, not all (-optmal solutons are obtaned from the above process n vew of the equal dvson of ( nto T parts, one for each stage. The addtonal complexty to fnd all the remanng (-optmal solutons n DP s nonetheless prohbtve. The key of FZDP s to consder a sgnfcant number of (-optmal solutons and to reasonably assgn ther weghts n a computatonally effcent manner Comparson of Methods. For the smplfed bundle method wth fuzzy dynamc programmng (SB FZDP), the relaxed problem s solved by usng FZDP for a gven set of multplers, and a fuzzy gradent s obtaned to be the search drecton by consderng near-mnmum solutons. The dfferences between SB FZDP and other methods are hghlghted n Table 1. In a subgradent method, a mnmum soluton s selected wth weght 1, and all other solutons are gnored. For the surrogate subgradent method, only a near-optmal soluton obtaned by approxmate mnmzaton s selected wth weght 1. The bundle methods obtan a good search drecton by combnng the subgradents of many nearby ponts. However, the relaxed problem may have to be solved many tmes to obtan the needed subgradents for a serous step, and the combnaton of the ndvdual subgradents s determned by quadratc programmng. Therefore, the computatonal requrements for bundle methods are hgh. The mproved bundle method s smlar to the bundle methods, except that fewer null steps are Table 1. Comparson of methods. Improved SG SSG Bundle bundle SB FZDP Relaxed problem Once FOnce a Several FSeveral Once mnmzed tmes tmes b Neghborhood nformaton No No Yes Yes Yes consdered (-optmal solutons used Ignored Utlzed Ignored Utlzed Utlzed Lne search No No Yes Yes No Weght assgnment N A N A QP QP Smple rule Drecton Not good Not good Good Best Good Complexty Low Lowest Hghest Hgh Low a Approxmate mnmzaton s performed once. b Number of null steps for the mproved bundle method s less than that for the bundle method.

18 390 JOTA: VOL. 113, NO. 2, MAY 2002 Fgure 6. Complexty comparson of varous methods. requred n vew of the ncluson of the (-surrogate subgradents. The smplfed bundle method, on the contrary, does not requre a lne search nor quadratc programmng. It makes good use of the subgradents of nearoptmal solutons, whch are generally avalable when the relaxed problem s solved, and these drectons are combned lnearly by usng a smple rule. Computatonal requrements are thus relatvely small. The complexty of one teraton of the above methods s depcted n the ascendng order n Fg. 6. However, the overall performance of a method s a tradeoff between the complexty per teraton versus the number of teratons requred. 5. Numercal Testng In ths secton, two examples are presented. Example 5.1 s to solve a smple dual problem to show that the smplfed bundle method can reduce the soluton zgzaggng. Example 5.2 s a benchmark problem, and the performance of the smplfed bundle method s compared wth that of a tradtonal bundle method. Example 5.1. As mentoned earler, the subgradent drectons often cause the multplers to zgzag across sharp rdges. However, for the smplfed bundle method snce the drecton s obtaned by a weghted combnaton of the gradents of nearby facets, zgzaggng s sgnfcantly reduced as llustrated below by maxmzng the followng pecewse lnear dual functon: L(λ 1, λ 2 )Gmn{ λ 1 A3λ 2 C300,Aλ 1 Cλ 2 C1, 2λ 1 Aλ 2 C1}. The parameter ( for the smplfed bundle method s set to be and (G(L*AL) 2, sg(l*al) 2 g 2 (67)

19 JOTA: VOL. 113, NO. 2, MAY s used followng (53). The rules (58) and (59) are appled to obtan a combned drecton based on L ((λ). For the subgradent method, sg(l*al) 2 g 2 s also used. For both methods, L*G25(11 12) s assumed to be known wthn ther stepszng rule formulas. The multpler trajectores for the frst ten teratons of both methods are shown n Fg. 7. Compared wth the subgradent method, the smplfed bundle method sgnfcantly reduces zgzaggng. Ths s because, when the multplers are close to a rdge, the search drecton of the smplfed bundle method s a combnaton of the gradents of nearby facets, generatng a much smoother trajectory. Fgure 7. Reducton of soluton zgzaggng: (a) dual functon, (b) trajectory obtaned by usng the subgradent method, and (c) trajectory obtaned by usng the smplfed bundle method.

20 392 JOTA: VOL. 113, NO. 2, MAY 2002 Example 5.2. Benchmark Problem. Bundle methods were developed for general nonsmooth optmzaton, and many benchmark problems have been reported n the lterature. However, the mproved bundle method and the smplfed bundle method are developed wthn the Lagrangan relaxaton framework so as to maxmze the dual problem, whch s a specal class of nonsmooth optmzaton problems. Nevertheless, the concepts of near-optmal solutons and (-surrogate subdfferental are generc, and the results n Sectons 3 and 4 can be extended to lnear mn-max problems such as mn f(x) mn {max ( f (x): G1,...,I)}, (68) x where {f (x)} are lnear functons. In the lterature, the Goffn polyhedral problem s such a lnear mn-max benchmark problem, f(x)gn max {x : G1, N}A N G1 x. (69) The problem dmenson s NG50, and the optmal cost s f(x*)g0. The ntal pont s gven as x 0 GA(NC1) 2, for G1,...,N, wth cost f(x 0 )G1225. It s reported that bundle methods can get to the optmal soluton wth 51 teratons (34 serous steps and 17 null steps n Ref. 13). The results obtaned by the subgradent method and the smplfed bundle method are summarzed n Table 2 for a few selected number of teraton ndces. It can be seen that the performance of the smplfed bundle method (SB) s much better than that of the subgradent method (SG). The performance of SB s also comparable wth those obtaned by bundle methods, havng a smlar number of teratons or functon evaluatons. However, the computaton complexty n SB s much reduced as compared to tradtonal bundle methods, because nether quadratc programmng nor lne search s requred. Table 2. Testng results for the Goffn polyhedral problem. Iteratons f(x) SG SB

21 JOTA: VOL. 113, NO. 2, MAY Job Shop Schedulng wth Fuzzy Dynamc Programmng In ths secton, fuzzy dynamc programmng s appled to job shop schedulng problems Problem Formulaton and Soluton Methodology. In a job shop, each part has ts due date and weght or prorty, and requres a seres of operatons for ts completon. Each operaton s to be performed on a machne of a specfed type for a gven perod of tme. The processng may start only after ts precedng operatons have been completed, satsfyng the operaton precedence constrant. The number of operatons assgned to a machne type may not exceed the number of machnes avalable at any tme, satsfyng the machne capacty constrants. The problem s to determne the operaton begnnng tmes so that the total weghted part earlness and tardness penalty s mnmzed. Through proper selecton of the decson varables, these constrants are formulated n addtve forms n Ref. 14. Unlke the prevalent formulatons n the lterature, the key feature here s ts separablty. Wthn the Lagrangan relaxaton framework, machne capacty constrants are relaxed by usng Lagrange multplers (capacty multplers). For a gven set of multplers, the relaxed problem can be decomposed nto decoupled part subproblems. Each subproblem represents the schedulng of a part so as to mnmze ts tardness and earlness penaltes and the costs for utlzng the machnes (reflected by the values of the multplers for the requred machne types at scheduled tme slots). Each subproblem s a multstage optmzaton problem, and can be solved effcently by usng dynamc programmng (DP) wth polynomal complexty (Ref. 11). A typcal DP structure s shown n Fg. 8. Wth stages Fgure 8. Dynamc programmng for a part subproblem.

22 394 JOTA: VOL. 113, NO. 2, MAY 2002 correspondng to operatons and states correspondng to operaton begnnng tmes, the backward DP algorthm starts wth the last stage and computes the tardness penaltes and machne utlzaton costs. As the algorthm moves backward, the cumulatve costs of the ndvdual states belongng to a partcular stage are computed based on the stagewse costs and the mnmal costs-to-go for the succeedng stage, subject to the allowable state transtons as delneated by the operaton precedence constrants. The optmal subproblem cost s then obtaned as the mnmum of the cumulatve costs at the frst stage, and the optmal begnnng tmes for the ndvdual operatons can be obtaned by forward tracng the stages. Each state node n Fg. 8 represents an operaton begnnng tme and mples the utlzaton of a machne of a partcular type for a specfed perod of tme. Based on the optmal begnnng tmes obtaned from DP, the machne utlzaton by all the operatons for each machne type at dfferent tme slots can be calculated. The subgradent s then a long vector of the dfference between machne utlzaton and machne capacty for all machnes and for all tme perods. Iteratve updatng of the multplers along a proper drecton, repeated resolutons of subproblems, and the fnal heurstc adjustment of the subproblem solutons lead to the near-optmal solutons of the orgnal problem. The cost of the feasble schedule J from the heurstc s an upper bound on the optmal feasble cost J*. On the other hand, the optmal dual D* s a lower bound on J*. Snce t s usually dffcult to fnd J* and D*, the pseudodualty gap (J-D) D s often used as a measure of the qualty of the feasble schedule, where D s the hghest dual cost obtaned over the teratons. Only one optmal begnnng tme s usually obtaned for each operaton n a tradtonal DP. In FZDP, near-optmal begnnng tmes are obtaned, each wth an assocated state weght. The fuzzy machne utlzaton by the operaton s calculated based on a weghted combnaton of these nearoptmal begnnng tmes. A fuzzy gradent can then be derved as the dfference between the total fuzzy machne utlzaton and the machne capacty. Snce near-optmal solutons are consdered n FZDP, the resultng fuzzy gradent drectons are much better than the subgradent drectons. The complexty of FZDP can be reduced further by explotng the specal structure of the relaxed problem Testng Results. Several practcal job shop schedulng problems were tested by usng the smplfed bundle fuzzy dynamc programmng (SB FZDP) and the subgradent dynamc programmng method (SG DP) wthn the Lagrangan relaxaton framework on a Pentum 400 MHz PC. However, tradtonal bundle methods were not tested here because ther

23 JOTA: VOL. 113, NO. 2, MAY sgnfcant computatonal requrements to obtan good search drectons hnder ther ablty for handlng large problems (Ref. 11). For both methods tested, the same heurstcs was used to obtan feasble solutons. For SG DP, the nput to the heurstcs s the optmal begnnng tme for each operaton, and for SB FZDP, the nput s the begnnng tme assocated wth the largest weght for each operaton. Both algorthms are stopped after the same amount of CPU tme, and the results are summarzed n Table 3. The testng shows sgnfcant performance mprovement by usng SB FZDP, as t can effcently provde good drectons. As mentoned early, the surrogate subgradent method combned wth dynamc programmng (SSG DP) was developed recently, and wth ts nterleaved dea, t generates better results than the subgradent methods or bundle methods for very large job shop schedulng problems (Ref. 3 4). In order to further mprove SSG DP, the nterleaved dea s combned wth FZDP, and the resultng SSG FZDP s compared wth SSG DP. Twentyfve data sets were randomly generated wth operaton processng tmes, machne types requred, and part due dates unformly dstrbuted wthn approprate ntervals. For each data set, there are 100 parts each wth tardness weght equal to 1, ten operatons per part, and ten machne types n total. There are 10,000 multplers n the dual problem. The dualty gaps obtaned by both methods are compared, and the mprovement of the dual values n percentage by usng SSG FZDP s shown n Fg. 9 for all the 25 cases. It can be seen that the dual cost of SSG FZDP s better than that of SSG DP for most cases, and the average mprovement s about 0.5%. In addton, the average dualty gap s reduced by more than 10%. Ths much sgnfcant mprovement n the dualty gap s probably caused by the more sensble operaton begnnng tmes obtaned by usng fuzzy state weghts n FZDP than the operaton begnnng tmes obtaned by usng crsp DP. Table 3. Comparson of SB FZDP va SG DP. Optmzaton Dual Prmal Dualty CPU Prmal dmensons MT P O method cost cost gap tme SB FZDP % 30 SG DP % SB FZDP % 120 SG DP % SB FZDP % 600 SG DP % 600 CPU tme n seconds. MT P O represents the number of machne types (MT), number of parts (P), and number of operatons (O).

24 396 JOTA: VOL. 113, NO. 2, MAY 2002 Fgure 9. Testng results for SSG FDP and SSG DP. 7. Conclusons Bundle methods are advanced methods developed for general nonsmooth optmzaton. When they are appled to optmze the dual functons wthn the Lagrangan relaxaton framework, they can be mproved and smplfed, leadng to the mproved bundle method and the smplfed bundle method wth fuzzy dynamc programmng as presented n ths paper. These methods can utlze effcently the valuable nformaton contaned n nearmnmum solutons, whch many tmes are byproducts when the relaxed problem s solved. Thus, they provde a new approach wth good search drectons and small computaton requrements to solve large dual problems.

25 JOTA: VOL. 113, NO. 2, MAY References 1. GEOFFRION, M., Lagrangan Relaxaton for Integer Programmng, Mathematcal Programmng Studes, North Holland, Amsterdam, Netherlands, pp , BERTSEKAS, D. P., Nonlnear Programmng, 2nd Edton, Athena Scentfc, Belmont, Massachusetts, ZHAO, X., LUH, P. B., and WANG, J., Surrogate Gradent Algorthm for Lagrangan Relaxaton, Journal of Optmzaton Theory and Applcatons, Vol. 100, pp , KASKAVELIS, C. A., and CARAMANIS, M. C., Effcent Lagrangan Relaxaton Algorthms for Industry Sze Job-Shop Schedulng Problems, IIE Transactons, Vol. 30, pp , HIRIART-URRUTY, J. B., and LEMARECHAL, C., Conûex Analyss and Mnmzaton Algorthms, Vols. 1 and 2, Sprnger Verlag, Berln, Germany, NEMHAUSER, G., and WOLSEY, L., Integer and Combnatoral Optmzaton, John Wley and Sons, New York, NY, HELD, M., WOLFE, P., and CROWDER, H., Valdaton of Subgradent Optmzaton, Mathematcal Programmng, Vol. 6, pp , KIWIEL, K.C.,Restrcted Step and Leûenberg Marquardt Technques n Proxmal Bundle Methods for Nonconûex Nondfferentable Optmzaton, SIAM Journal on Optmzaton, Vol. 6, pp , KOHL, N. and MADSEN, O. B. G., An Optmzaton Algorthm for the Vehcle Routng Problem wth Tme Wndows Based on Lagrangan Relaxaton, Operatons Research, Vol. 45, pp , MIFFLIN, R., SUN, D., and QI, L., Quas-Newton Bundle-Type Methods for Nondfferentable Conûex Optmzaton, SIAM Journal on Optmzaton, Vol. 8, pp , WANG, J., LUH, P. B., ZHAO, X., and WANG, J., An Optmzaton-Based Algorthm for Job Shop Schedulng, Sadhana, Vol. 22, pp , BELLMAN, R., Appled Dynamc Programmng, Prnceton Unversty Press, Prnceton, New Jersey, KIWIEL, K. C., Proxmal Leûel Bundle Methods for Conûex Nondfferentable Optmzaton, Saddle-Pont Problems, and Varatonal Inequaltes, Mathematcal Programmng, Vol. 69, pp , HOITOMT, D. J., LUH, P. B., and PATTIPATI, K.R.,A Practcal Approach to Job Shop Schedulng Problems, IEEE Transactons on Robotcs and Automaton, Vol. 9, pp. 1 13, 1993.