5 The Primal-Dual Method

Size: px

Start display at page:

Download "5 The Primal-Dual Method"

Robyn Johnston
5 years ago
Views:

1 5 The Prmal-Dual Method Orgnally desgned as a method for solvng lnear programs, where t reduces weghted optmzaton problems to smpler combnatoral ones, the prmal-dual method (PDM) has receved much attenton over the last years, as t can be generalzed to more complex optmzaton settngs and can be used to derve approxmaton schemes for NP-hard problems. PDM s a vast topc, and we can only gve a very basc dea here. There are many flavors, versons and extensons to t. 1 In ts most basc form, the man prncple s to mprove a feasble dual soluton untl the prmal satsfes complementary slackness condtons, ndcatng optmalty n cases where strong dualty holds, or an approxmate soluton where t doesn t. 5.1 The prmal-dual method for lnear programs Assume we have a prmal mnmzaton and a dual maxmzaton problem n standard form,.e. and mn x n max y m c x Ax b x 0 b y A y c y 0 Recall the property of complementary slackness for optmal solutons x, y n LP: Prmal complementary slackness (PCS) : At least one of x j = 0 or a j y = c j must hold. Dual complementary slackness (DCS) : At least one of y = 0 or j a j x j = b must hold. The central observaton s that f strong dualty holds for all constrants n the prmal and the dual for some y, that y s n fact an optmal soluton. Hence, CS can be used as a certfcate for optmalty. Ths leads to the orgnal verson of the PDM for LPs, whch can be summarzed thusly: 1 If you are eager to fnd out more, there s an extensve recent revew ( document/ /), wth a free preprnt (.e. not peer-revewed) verson ( 5429). 1. Fnd some feasble dual soluton y. 2. Gven y, fnd some x that mnmzes the volaton of complementary slackness n the prmal. 3. If CS holds, y s optmal, and PDM termnates. 4. Otherwse, change y so as to mprove the dual objectve b y, and go to 2. Note that at ths pont, the soluton x obtaned n step 2. s not necessarly feasble and mght volate prmal constrants! Obvously, we requre some way to fnd x n step 2., and a way to measure whether complementary slackness holds, and f not, to what degree t s volated. Gven some feasble dual soluton y, let I := y = 0 be the set of all ndces for whch the dual varables are zero, and m J := j a j y = c j =1 the set of all ndces for whch the dual constrants are bndng. Obvously, I serves as an ndex for those prmal constrants for whch DCS holds because ther assocated dual varable y s zero, and J denotes the dual constrants that are bndng for a gven y. The complements of those sets are denoted by I and J. J s the set of ndces j for whch the dual constrants are not bndng, and hence x j would have to be zero for PCS to hold. Lkewse, I s the set of ndces for whch y > 0, and ths the prmal constrants would have to be bndng n order for DCS to hold. The dea s therefore to construct a new optmzaton problem called the restrcted prmal, n whch we try to reduce the slackness n the prmal constrants and the non-zeroness of the prmal varables x j, j J as much as we can. If and only f they are both zero, complementary slackness holds and y was an optmal soluton. Usng slack varables s to capture prmal constrant volatons, the restrcted prmal (RP) s defned as f RP = mn s + x j I j J I : j a j x j b I : j a j x j s = b s 0 x :33 1

2 If the soluton to the RP s zero, we know that complementary slackness holds, and, due to strong dualty, we have an optmal soluton for the prmal and the dual. In case the RP has a non-zero optmal value, we want to mprove the dual soluton b y. One way to do ths s to fnd a vector z such that b z > 0, whch leads to b y + b z = b (y + z) > b y To fnd such a z, we can use the followng observaton: the dual of the restrcted prmal, the restrcted dual, has the same objectve value as the restrcted prmal, due to strong dualty, and wll be strctly postve. We hence know that we can use the optmal soluton z of the restrcted prmal to mprove y. We derve the restrcted dual (RD) by our usual scheme, Ths leads to 0 x J mn 1 1 x J s x J x J s 0 z I A IJ A IJ O b I max b z z I A I J A I J E = b I f RD = max b z j J : a jz 0 j J : a jz 1 I : z 0 I : z 1 Notce that the last constrant does not seem to appear n the scheme; whle the prmal equalty constrant mposes no restrctons on y for I, due to E we also have z 1 and therefore z 1. However, whle z s guaranteed to mprove our dual soluton, the newly formed vector mght be volatng constrants A (y + z ) c, y + z 0 Ths can be salvaged by scalng z wth a small enough constant ε > 0 such that A (y + εz ) c, y + εz 0, Ths constant s easly derved: f I, then y + εz 0 for any ε 0. The same f true for those I for whch z 0. The only problematc case s for negatve z,.e. whenever I and 1 z < 0. To guarantee non-negatvty, we use y + εz 0 εz y ε y z and then take the mnmum across all those cases, ε mn y I z z < 0. Lkewse, for j J, the constrant a j (y + εz ) c holds snce and a j (y + εz ) = a j z 0 a j y + ε a j z The same holds for j J whenever a jz 0, hence the only problematc case s whenever j J and 0 < a jz 1. Then, a j y + ε a j z c j ε a j z c j a j y :33 2

3 and therefore for ε mn j J ε c j a j y a jz c j a j y a jz a j z > 0 the dual constrants are preserved. Choosng the smaller of those two ε-values takes us from a feasble dual soluton y to a better feasble dual soluton y + εz. The full verson of the prmal-dual method for LP s then obtaned as follows 1. Fnd some feasble dual soluton y. 2. Gven y, formulate the restrcted prmal and fnd the mnmum value of ts objectve f RP. 3. If f RP = 0, complementary slackness holds and y s optmal. Return. 4. Otherwse, formulate the restrcted dual. Determne the best ε and change y to y + εz so as to mprove the dual objectve b y, and go to step 2. One of the reasons to employ ths sort of algorthm s that the cost c vanshes. Ths turns a weghted problem nto an unweghted problem, and step 2 can potentally be solved usng effcent combnatoral optmzaton algorthms that do not rely on lnear programmng. 5.2 PDM for approxmaton schemes The prmal dual method can be used to derve approxmaton schemes for NP-hard problems. As a motvatng example, we consder the followng par of dscrete optmzaton problem: Let G = (V, E) be an undrected graph wth vertex set V and edge set E. The vertex cover problem asks to fnd the smallest subset V opt V such that each edge n G s ncdent to at least one node n V opt. Its ILP formulaton s straghtforward: mn v V x v (v, w) E : x v + x w 1 x v {0, 1} Its dual s the maxmum matchng problem, whch asks to fnd a maxmum set of edges such that no two edges share a node. It can be wrtten as max e E v V : w:(v,w) E y vw 1 y e {0, 1} If we were to wrte ths as a scheme, the matrx A would be the transpose of the nodeedge ncdence matrx. A node or edge s selected f ts varable s 1, and deselected f t s 0. In bpartte graphs, A s TUM, and thus we can solve both vertex cover as well as maxmum matchng n polynomal tme. Specfcally, maxmum matchng can be seen as a specal case of s-t-flow, by addng source and snk nodes as well as puttng drectonalty on the edges such that they all pont towards the second vertex partton. Strong dualty therefor provdes a smple proof for the followng mportant result: Theorem 1 (Kőng s theorem). In a bpartte graph, the number of edges n a maxmum matchng equals the number of nodes n a mnmum vertex cover. As a non-mandatory exercse, try and see how the PDM for LPs apples n ths case. Unfortunately, for general graphs, vertex cover s an NP-hard problem, whereas maxmum matchng s solvable n polynomal tme, e.g. usng Edmond s algorthm. We cannot expect strong dualty to hold. As we want to cover as many edges wth as few nodes as possble, there s an obvous greedy heurstc we could try n order to fnd a good approxmaton. Let s call t the naïve heurstc: 1. Pck the node v wth the hghest degree deg v,.e. the hghest number of ncdent edges. 2. Add v to the soluton, then delete v and all ts ncdent edges. 3. Repeat untl no edges are left. The queston arses whether the naïve heurstc comes wth any performance guarantees. To that end, let us assume w.l.o.g. that the nodes and edges are numbered n the order n whch they are selected, so v 1 s selected before v 2, and e 1 before e 2. v k and ts ncdent edges are selected n the k-th teraton. We are tryng to mnmze the the total number of selected nodes, hence our objectve functon s a cost functon, wth optmal value. Selectng a node adds 1 to the total cost. Equvalently, we can dstrbute the cost of selectng a node equally among all remanng edges ncdent to that node, hence n teraton k, each edge that gets deleted ncurs a cost of y e 1 deg(v k ) :33 3

4 Assume there are n nodes and m edges. Assume edge e j s ncdent to v k and removed durng the k-th teraton. We want to obtan a bound on how much cost s ncurred by removng e j. Snce larger node degrees means lower edge costs, we want to fnd a lower bound on the largest degree stll n the graph: Before enterng the k-th teraton, there are at least m j + 1 uncovered edges left n the graph. Obvously, f we could solve vertex cover optmally, we would requre no more than nodes to cover these remanng edges. We therefore have to dstrbute m j + 1 edges among (at most) nodes. Ths leads to a very basc but mportant result from Ramsey theory: f we were to have n pgeonholes and m pgeons, what s the mnmum number of pgeons n the most crowded hole? Theorem 2 (Pgeonhole prncple). In any partton of a set of m elements nto n blocks, there exsts a block wth at least m n elements. Ths means that somewhere n the graph, there exsts a node v wth m j + 1 deg(v) m j + 1, so the cost of removng e j s at most m j + 1. Over all m teratons, the cost ncurred by the naïve heurstc, f N H, s therefore bounded as m m f N H m j + 1 = = H m j j=1 where H m s the m-th harmonc number. Snce 2 H m O{log m} O{log(n 2 )} = O{2 log n} = O{log n}, the naïve heurstc s an O(log n)-approxmaton of vertex cover. That s not very exctng news: even for moderately szed problems wth 1000 nodes, the approxmaton rato can be almost seven! Even worse, the rato grows wth the problem sze. Ths case may serve as an example that sometmes the obvous approach s not as good a choce as we mght thnk. Instead, we wll turn to usng the PDM to derve a much better heurstc. For approxmaton schemes for problems lke vertex cover, we don t have strong dualty, and complementary slackness does not hold for a prmal and ts dual smultaneously. Instead, the PDM s modfed so that only prmal complementary slackness s enforced, whereas dual complementary slackness s relaxed: nstead of 2 We use O{f } for the set of functons growng not faster than f, and O(f ) for an element from ths set. In practce, the two are often not dstngushed. j=1 requrng constrants for non-zero varables to hold wth equalty, one can requre the slack to be wthn a certan bounded nterval. Ths leads to the followng relaxed complementary slackness condtons (stated here for a prmal mnmzaton problem n standard form): Relaxed PCS : For all j some α 1, at least one of x j = 0 or c j α a j y c j must hold. Relaxed DCS : For all and some β 1, at least one of y = 0 or β b j a j x j b must. So we have as well as c α A y c c x αy Ax b Ax βb y Ax βb y αy Ax αβb y As before, for a prmal mnmzaton problem, weak dualty mples that b y c x. Combnng the above yelds b y c x αβb y In other words, f relaxed complementary slackness holds, the optmal prmal soluton s between the optmal dual soluton and a constant factor of that soluton, hence αβ serves as an approxmaton rato. The prmal-dual method for ILP approxmatons works as follows: 1. Set x = y = Rase some of the y wthout volatng any dual constrants. 3. Whenever a dual constrant becomes bndng, freeze the dual varables occurrng n that constrant, and rase the assocated prmal varables. 4. Repeat from 2. untl all dual constrants become tght. In the begnnng, PCS s guaranteed by x j = 0. Makng the j-th dual constrant bndng frees up x j to take on a postve value wthout volatng PCS. Furthermore, we mantan dual feasblty by not rasng any frozen y, or ncreasng non-frozen ones by too much. Snce for general ILP strong dualty does not hold, we do not mantan DCS, but only ts relaxed form :33 4

5 We now have a prncpled way to derve an approxmaton heurstc for vertex cover: The prmary complementary slackness condton nvolves prmary node varables and dual constrants, so f x v > 0, then PCS s enforced by requrng w:(v,w) E y vw = 1 On the other hand, dual complementary slackness, nvolvng the dual edge varables and prmal constrants, says that f y e > 0 for some e = (v, w), we must have 3. Whenever a dual constrant becomes tght,.e. (u,w) δ(u) y vw = c u (the sum of ncdent edges equals the node weght), select that node (rase the prmal varable) and freeze the edges (fx values of dual varables, snce rasng them later would volate the constrant). 4. Repeat from 2. untl all edges are frozen. Check for yourself that ths s stll a 2-approxmaton n the weghted case! x v + x w = 1 However, as we sad before, n the prmal dual method, only the PCS s enforced, but the DCS s relaxed f necessary. The PDM proceeds lke ths: 1. Set y = 0, except for one entry y e = 1. Ths s a feasble matchng, as t obeys all dual constrants. Also, set x = 0; note ths s not a feasble prmal soluton, as t s not a vertex cover and volates prmal constrants. 2. We now have x e = 1 > 0 for some e := (v, w). Ths means that the dual constrants v, w are bndng (α = 1) and ther PCS s satsfed. Ths allows us to set x v = x w = 1, as they don t have to be zero anymore for PCS to hold. We thereby decrease the number of volated prmal constrants, as we added coverng nodes and x v + x w 1 now holds. However, DCS holds only n ts relaxed form, as x v + x w = 2 1, so β = 2. Ths means the algorthm s a 2-approxmaton of vertex cover! 3. If no more edge can be added, return. 4. Otherwse, pck another edge that does not share a node wth one we selected before. Ths mproves the dual objectve, snce the matchng gets larger. The prmal soluton mght stll not be a vertex cover and thus be nfeasble. Go to step 2. For the weghted mnmum vertex cover, the dual does not have a straghtforward nterpretaton such as matchng anymore. Stll, the PDM works: 1. Set all nodes and edges to Pck a non-frozen edge (u, v) (.e. dual varable), and rase ts value such that the sum of edges ncdent to node u does not exceed the cost of node u,.e. (u,w) δ(u) y vw c u, and lkewse for v :33 5

GSLM Operations Research II Fall 13/14

GSLM Operations Research II Fall 13/14 GSLM 58 Operatons Research II Fall /4 6. Separable Programmng Consder a general NLP mn f(x) s.t. g j (x) b j j =. m. Defnton 6.. The NLP s a separable program f ts objectve functon and all constrants are