Lecture Notes on Integer Linear Programming

Transcription

1 Lecture Notes o Iteger Liear Programmig Roel va de Broek October 15, 2018 These otes supplemet the material o (iteger) liear programmig covered by the lectures i the course Algorithms for Decisio Support. History 26 October, 2017 First versio 15 October, 2018 Added missig ote o dictioaries Liear Programmig I a optimizatio problem we typically have to select the best solutio from the set of all solutios, the solutio space, that satisfy the costraits of the problem. The evaluatio of the quality of a solutio is based o a objective fuctio that we have to either miimize or maximize. A simple example of a optimizatio problem is the Healthy Diet problem. Bob is a Computer Sciece studet who, surprisigly, is ot particularly fod of goig to the gym. As Bob still wats to followig a somewhat healthy lifestyle, he decides to compose a diet that meets the daily referece itake of vitamis with the miimal amout of calories. Ufortuately for Bob, he ca oly get pizzas ad burritos ear his place. The utritioal values of a slice of pizza ad a burrito are show below, i Table 1. A C D Calories Pizza Burrito Itake I this example, a solutio to the optimizatio problem is a meal of pizzas ad burritos. The variables of a diet are the umber of slices of pizza, x p 0, ad the umber of burritos, x b 0. As lowcalorie meals are preferred, the objective will be the miimizatio of calories i the diet, Table 1: The utritioal values of a slice of pizza ad a burrito, as well as the required daily itake of vitamis. Please ote that the umbers used i this example are fictioal. miimize 600x p + 300x b. Based o this objective fuctio, the optimal solutio would be to eat othig at all. However, to achieve the daily referece itake, ay

2 lecture otes o iteger liear programmig 2 meal has to respect the costraits o the amout of vitamis, 225x p + 600x b x p + 100x b x p + 75x b 600 (A), (C), (D). As ot eatig violates the vitami costraits, the empty meal is a ifeasible solutio. A feasible solutio satisfies all costraits of the optimizatio problem. Figure 1 shows the costraits as well as the area cotaiig feasible solutios, called the feasible regio 1. Furthermore, we ca see i Figure 1 that the optimal meal 2 cosists of eatig four burritos ad oe ad a half slices of pizza each day, for a total of 2100 calories. x p 1 The feasible regio is empty if o solutio satisfies all costraits, i which case the optimizatio problem itself is ifeasible. 2 Disclaimer you should ot take this optimal solutio as soud dietary advise. Ay reliace you place o these diets is strictly at your ow risk x p + 600x b objective 600xp + 300x b Figure 1: The colored area cotais the feasible solutio to the Healthy Diet problem. The gradiet shows the value of the objective fuctio i the solutio space x p + 100x b x p + 75x b x b The formulatio of the Healthy Diet problem is a example of Liear Programmaig (LP), also kow as Liear Optimizatio. I liear programmig, a solutio is represeted of oe or more variables, which are called decisio variables, ad the domai of each variable is a iterval o the real lie. Furthermore, both the objective ad the costraits are liear 3 i the variables. The geeral liear programmig formulatio of a miimizatio 4 3 The liearity property of liear programs meas that, with two variables x ad y, the objective ad costraits ca cotai expressios such as x + y ad y x, but, for example, ot x y, x2 or x y, as the latter three are o-liear expressios. 4 Maximizig the objective fuctio f (x) is equivalet to miimizig f (x).

3 lecture otes o iteger liear programmig 3 problem is miimize subject to a i1 x i b 1 i=1.. a im x i b m i=1 c i x i i=1 x i 0 i {1,..., } (domai), where solutios are ecoded by decisio variables, x 1 to x, with associated costs c 1 to c, ad the objective is to miimize the total cost. The decisio variables are subject to m costraits of the form 5 i=1 a ijx i b j, ad domai costraits, x i 0. A optimal solutio is ay solutio that satisfies the costraits ad has miimal cost 6. For may applicatios, it is easier to use the matrix form of a LP istead of the sum formulatio show i (1). Deote vectors x = (x 1,..., x ), c = (c 1,..., c ), b = (b 1,..., b m ) ad let a 1,1 a 1,2 a 1, a 2,1 a 2,2 a 2, A = a m,1 a m,2 a m, The the liear program (1) ca be writte i matrix form 7 as miimize c T x (1) 5 Liear costraits such as a i x i = b or a i x i b ca be rewritte to this form as well. Strict iequalities such as a i x i < b are usually ot allowed i a liear program, sice the optimal solutio to the LP might ot be well-defied. For example, i the sigle variable LP max x x < 1 o solutio is maximal. x 0, 6 A optimizatio problem is called ubouded it is feasible ad we ca fid a arbitrarily good feasible solutio, i.e. the costraits i the problem do ot produce a upper boud o the goodess of feasible solutios. 7 The otatio x 0 idicates that all elemets x i of x should be at least zero. subject to Ax b x 0. (2) Techiques to fid the optimal solutio of a liear program is ot covered i the lecture otes. Examples are show o the lecture slides ad i the first two chapters of Chvatal 8. 8 Vasek Chvatal. Liear Programmig. Macmilla, 1983 Modelig Liear programmig is a flexible techique that ca be applied to may real-world problems. A major advatage of modelig a problem as a LP is that liear programs are efficietly solvable. That is, the computatio time of a LP is polyomial 9 i the umber of 9 I complexity theory we would deote this property as LP P. More o this i a few lectures.

4 lecture otes o iteger liear programmig 4 variables ad costraits. With the curret state-of-the-art of commercial ad ope-source solvers, it is rarely ecessary or beeficial to implemet your ow custom solver. The major challege of liear programmig is i the problem modelig: how do we traslate a optimizatio problem to a liear program that ca be processed efficietly by a solver? What decisio variables will we use to ecode the solutios of the problem, ad how ca we rewrite the problem costraits to liear equatios? To further complicate matters, there are problems for which we caot formulate liear programs 10. As there are o algorithms available that decide how, ad if, a optimizatio problem ca be modeled as a LP, modelig ofte has to be doe maually. I the ext sectios, we will look at several examples of optimizatio problems, ad show you how they ca be modeled as liear programs. 10 Fortuately, extesios to liear programmig allow us to model a much broader class of optimizatio problems, albeit at the cost of computatio time i solvers. Assigmet Problem I the Assigmet problem, we have jobs that eed to be performed; each job takes T time to complete. As we obviously do ot wat to do the work ourselves, we hire workers to perform the jobs. Each worker ca be hired for at most T time uits. The cost of hirig a certai worker depeds o both the employmet duratio ad the job he/she has to perform; if worker i speds a fractio a of its time o job j, it will cost us a times C ij. The objective of the Assigmet problem is to assig the workers to jobs such that all jobs are completed with the lowest total cost. To model this problem as a liear program, we eed to have decisio variables that ecode all possible solutios, i.e. the differet assigmets. A solutio to the Assigmet problem should state for each worker the time 11 it speds o each job. As the processig time of a job equals the maximum hirig duratio of a worker, this is equivalet to statig the fractio of job j completed by worker i. Therefore, we itroduce for each worker-job pair (i, j) the variable x ij [0, 1] that idicates the fractio of job j that is performed by worker i. The objective the becomes the miimizatio of the sum of all variables x ij multiplied by their cost C ij. To esure that each job j is completed, we have the costrait that the sum of the fractio of work sped by all workers o job j is precisely oe. Similarly, the work load of each worker i, the total fractio of their time allocated to all jobs, should be o more tha oe. The LP formulatio of this model is show below, (3). 11 Note that the problem descriptio does ot limit a worker to a sigle job; workers ca switch betwee jobs.

5 lecture otes o iteger liear programmig 5 mi i=1 i=1 j=1 C ij x ij (3a) x ij 1 j {1,..., } (work load) (3b) x ij = 1 i {1,..., } (job completio) (3c) j=1 x ij [0, 1] i, j {1,..., } (domai) (3d) I the curret problem statemet, workers are allowed to be assiged to multiple jobs, the oly requiremet is that they should ot perform more tha a sigle jobs worth of work. I practice, it might be iefficiet if a worker has to switch jobs. Therefore, the costrait that each job is completed by a sigle worker is preferable. That is, worker i performs job j either etirely or ot at all. To model this costrait we restrict the domai of the x ij variables, costrait (3d), to biary values, x ij {0, 1} i, j {1,..., } (biary domai). (3d ) The meaig of the variables stays the same, 1 if worker i performs job j, x ij = 0 otherwise. Iteger Liear Programmig The program described by (3) with the additioal costraits (3d ) is a example of Iteger Liear Programmig, abbreviated as ILP or IP, where each variable is restricted to iteger values 12. Iteger liear programmig is a importat tool i combiatorial optimizatio, as may problems feature discrete decisios that ca be modeled i a ILP. We will examie a few examples of such problems i these lecture otes. I our first example of a LP, the Healty Diet problem, suppose that Bob wats to avoid meals that cosist of partial burritos or pizza slices, as he does ot like to eat leftovers of the previous day. Similar to the Assigmet problem, Bob oly has to set the domai of the variables, x p ad x b, to itegers i his model to get the desired result: 12 Models that cotai both iteger ad cotiuous variables are kow i literature as Mixed Iteger (Liear) Programs or MI(L)Ps.

6 lecture otes o iteger liear programmig 6 mi 600x p + 300x b (4a) 225x p + 600x b 1800 (A) (4b) 100x p + 100x b 550 (C) (4c) 200x p + 75x b 600 (D) (4d) x p, x b N 0 (domai) (4e) Figure 2 shows the resultig feasible regio of the ew iteger liear program. Note that each feasible solutio to the ILP is also feasible i the origial LP, but ot vice versa. I the ILP formulatio, we ow have three optimal solutios (x p, xb ), amely (2, 4), (1, 6) ad (0, 8), each worth 2400 calories. x p objective 600xp + 300x b Figure 2: The colored dots are the feasible solutio of the Healthy Diet problem without partial meals, the color of each dot shows the objective value x b Kapsack I the classical Kapsack problem we have valuable items, ad we wat to maximize our profit by sellig some of them. However, the carryig capacity B of our kapsack is limited, so we have to decide

7 lecture otes o iteger liear programmig 7 which items we will take with us. Item i has a value of c i ad weighs a i uits. Breakig items ito smaller parts makes them worthless, so we are ot allowed to take a fractioal part of a item. A solutio to the Kapsack problem cosists of a set of items. We ca ecode the solutios by itroducig a variable x i for every item i, idicatig whether we take the item or ot, 1 if we put item i i the kapsack, x i = 0 otherwise. These variables are biary, hece we are creatig a iteger liear program. Sice a optimal solutio should maximize the total profit of the selected items, ad the mai costrait of the problem is the capacity of kapsack, we ca use the ILP model max c i x i i=1 a i x i B i=1 x i {0, 1} (capacity) i {1,..., }. (5) Maximum Idepedet Set Problem A example of a optimizatio problem o graphs is the Maximum Idepedet Set problem. A idepedet set of a graph G = (V, E) is a subset of vertices S V with the property that o two vertices i S are adjacet i graph G. The Maximum Idepedet Set problem cosists of fidig a largest idepedet set i the graph. Similar to the Kapsack problem, a atural solutio represetatio is to associate a biary variable to each vertex i the graph, modelig the decisio o whether the vertex is icluded i the idepedet set or ot. Suppose that graph G has vertices, v 1 to v. The we associate a biary variable x i to each vertex, such that 1 if v i is i the idepedet set, x i = 0 otherwise. Figure 3: The colored vertices form a idepedet set of the graph, as o eighborig vertices are colored. Sice this graph does ot admit a larger idepedet set, the colored vertices are a maximum idepedet set. With this choice of variables, the objective simply becomes the maximizatio of the sum of all variables. To defie the costraits, we ca observe that at most oe vertex of each pair of eighborig vertices i the graph ca be icluded i the idepedet set. This results i the followig iteger liear program,

8 lecture otes o iteger liear programmig 8 max x i i=1 (6a) x i + x j 1 (v i, v j ) E (o-adjacet) (6b) x i {0, 1} v i V (domai). (6c) Although modelig the Maximum Idepedet Set problem as a ILP is rather straightforward, fidig the largest idepedet set is still a difficult task. I cotrast to liear programs, which are all solvable to optimality i polyomial time (i the umber of variables ad costraits), there is o polyomial boud o the computatio time kow for may iteger liear programs 13. Facility Locatio Problem Aother classical example i combiatorial optimizatio is the Capacitated Facility Locatio problem. A maufacturig compay has to meet the demad of m customers, with the demad of customer i {1,..., m} equal to D i 0. The compay wats to ope a umber of ew productio facilities, such that the costs of trasportig the maufactured commodity to the customers is miimal. There are possible buildig sites for the facilities, ad the cost of trasportig of oe uit of the commodity from locatio j {1,..., } to customer i is c ij. Of course, costructig a facility costs moey as well, the fixed costructio cost at locatio j beig F j. Furthermore, the productio capacity at locatio j is limited to C j. I this problem we have to make two types of decisios: 13 For the Maximum Idepedet Set problem, it is ulikely that such a boud exists, as it is a kow N P-hard problem. For more iformatio, see the lectures o complexity theory. where to ope the facilities, ad how customers obtai their goods. I lie with this distictio, we defie two types of variables for locatio j {1,..., } ad customer i {1,..., m}, amely the locatio variable 1 if we ope a facility at locatio j y j = 0 otherwise, Figure 4: I the Facility Locatio problem, there are locatios, show i as circles, available for the facilities. These facilities have to serve m customers, the black dots. ad the trasportatio variable q ij = the quatity of goods trasported from locatio j to i. Oe way of modelig the problem is the mixed iteger liear

9 lecture otes o iteger liear programmig 9 program (7): mi m j=1 i=1 c ij q ij + j=1 F j y j (7a) q ij = D i i {1,..., m} (demad) (7b) j=1 q ij C j y j j {1,..., } (capacity) (7c) i=1 q ij 0 i {1,..., m}, j {1,..., } (7d) y j {0, 1} j {1,..., }. (7e) I this formulatio, Equatio (7b) esures that each costumer is served. Equatio (7c) limits the total amout of commodity shipped from a locatio to its capacity if a facility has bee built, or 0 if there is o facility at the locatio. The objective, Equatio (7a), is to miimize the total cost of facility costructio ad trasportatio. The domais of the variables are bouded by Equatios (7d) ad (7e). I a real-world settig, it is usually preferable that customers receive receive their ordered goods from a sigle facility, as it simplifies the shippig process. I that case, we are o loger iterested i what fractio of the demad we set from a facility to a customer, as this will always be either zero or the etire demad. We ca model this with the biary decisio variables 1 if customer i receives its demad from locatio j, x ij = 0 otherwise. Figure 5: A possible solutio to the Facility Locatio problem show i Figure 4. The locatios with a facility are colored. The arcs show by which facilities each customer is served. This solutio does ot satisfy the costrait that each customer should be supplied by exactly oe facility. Substitutig the x ij variables for the fractioal trasportatio variables q ij i model (7) multiplyig with D i where ecessary yields the iteger liear program (8): mi m j=1 i=1 c ij D i x ij + j=1 F j y j x ij = 1 i {1,..., m} (demad) j=1 D i x ij C j y j j {1,..., } (capacity) i=1 x ij {0, 1} i {1,..., m}, j {1,..., } y j {0, 1} j {1,..., }. (8)

10 lecture otes o iteger liear programmig 10 Towards solvig a ILP: LP-relaxatio I our example of the Healty Diet problem, we restricted the domai of the decisio variables to iteger values to fid a meal without leftovers. The solutio space of the resultig iteger liear program was a subset of the set of feasible solutios of the origial problem, with the itegral optimum cotaiig slightly more calories tha the best fractioal solutio. I geeral, restrictig the domais of the variables will ever lead to a better solutio. The coverse also holds: relaxig the domai of a variable from itegers to a cotiuous real iterval that ecompasses the origial itegral values ever results i a worse optimum. The liear program obtaied by relaxig the itegrality costraits of a ILP is kow as the LPrelaxatio of the origial problem. As liear programs ca be solved more efficietly tha iteger liear programs, LP-relaxatios provide a efficiet procedure to fid a boud 14 o the optimum of a ILP: 1. Relax the itegrality costraits. 2. Solve the resultig liear program. If the optimal solutio to the LP-relaxatio happes to be itegral, the you do ot eve have to solve the ILP itself, because the boud guaratees that it will ot fid a better solutio. Eve if the LP-relaxatio has a fractioal optimal solutio, the boud it provides is crucial i solvig the ILP, as we will see i a few sectios. I the example of the Capacitated Facility Locatio problem, we have see that there ca be multiple models of the same problem, ad the optimal objective value of these models should be the same. However, the LP-relaxatios of the differet models do ot have ot result i the same boud o the optimum, as their solutios spaces ca be differet. Due to the importace of the LP-relaxatio bouds, we usually wat to fid a model with strog boud o the ILP. I the ext sectios, we will see some examples of problems with multiple models, for which we ca prove that the LP-relaxatio of oe model gives a better boud tha the other. 14 The LP-relaxatio provides a lower boud o the optimal objective value for miimizatio problems, ad a upper boud i case of maximizatio. The LP-relaxatio of the iteger liear program mi Ax b c T x x N 0 is the liear program mi Ax b x 0. c T x Facility Locatio Problem (Revisited) A commo variat of the Facility Locatio problem is to remove the capacity costraits of the facilities, i.e. each facility ca produce eough to supply all customers. Furthermore, i this Ucapacitated Facility Locatio problem we require each customer to be served by exactly oe facility. We ca model this problem with a small modificatio to the ILP formulatio show i Equatio (8). The capacity costrait is o loger eeded. However, we do have to

11 lecture otes o iteger liear programmig 11 esure that customer i is oly served from locatio j if we selected j as the costructio site for a facility. Usig the otatio d ij = c ij D i, we get the ILP formulatio mi m j=1 i=1 d ij x ij + j=1 F j y j (9a) x ij = 1 i {1,..., m} (9b) j=1 x ij y j i {1,..., m}, j {1,..., } (9c) x ij {0, 1} i {1,..., m}, j {1,..., } (9d) y j {0, 1} j {1,..., }. (9e) Equatio (9c) creates m costraits, oe for each of trasportatio variable, to prevet customers beig served by o-existig facilities. We ca costruct a more compact model by observig that if all trasportatio variables x 1j to x mj of locatio j are at most y j, the the sum of the variables x ij will ot be more tha my j. Therefore, we ca combie the m separate trasportatio costraits ito aggregated trasportatio costraits per locatio: mi m j=1 i=1 d ij x ij + j=1 F j y j (10a) x ij = 1 i {1,..., m} (10b) j=1 m x ij my j j {1,..., } (10c) i=1 x ij {0, 1} i {1,..., m}, j {1,..., } (10d) y j {0, 1} j {1,..., }. (10e) Both the separated model, (9), ad the aggregated model, (10), have the same optimal value, yet the latter ILP has fewer costraits, suggestig that the compact formulatio might be the preferred model for this problem. However, a compariso of the LP-relaxatios of both models obtaied by replacig the domai costraits i the ILPS with 0 x ij 1 ad 0 y j 1 will show that the separated model provides a stroger boud o the optimal solutio of the ILP. Theorem 1. The lower boud o the optimum value of the Ucapacitated Facility Locatio problem obtaied from the LP-relaxatio

12 lecture otes o iteger liear programmig 12 of separated model, Equatio (9), is at least as high as the boud of the LPrelaxatio of the aggregated model, Equatio (10). Proof. Let P ILP, P LPS ad P LPA be the sets of feasible solutios of respectively the ILP model 15, the LP-relaxatio of the separated model (9) ad the LP-relaxatio of the aggregated model (10). Every feasible solutio to the ILP is feasible for the two LP-relaxatios as well, hece 15 The solutio space ad optimal value of models (9) ad (10) is the same. P ILP P LPS ad P ILP P LPA. Recall our earlier observatio that, for ay j {1,..., }, i {1,..., m} : x ij y j = m x ij my j. i=1 y 1 = 1 2 x 11 = 1 C 1 Sice the two models differ oly i costraits (9c) respectively (10c), each solutio i P LPS is also a feasible solutio to the aggregated model: P LPS P LPA. L 1 x 12 = x 21 = 0 I cotrast, ot all feasible solutios to the LP-relaxatio of the aggregated model satisfy the costraits i (9c). Let s cosider the followig example of two customers, c 1 ad c 2, ad two locatios, l 1 ad l 2. A possible solutio to this problem is y 1 = y 2 = 1/2, x 11 = x 22 = 1 ad x 12 = x 21 = 0, as is show i Figure 6. Each customer is fully served i this solutio ad, sice L 2 y 2 = 1 2 x 22 = 1 C 2 Figure 6: A fractioal solutio to the Ucapacitated Facility Locatio problem with two locatios ad two customers. x 11 + x 21 = = 2 y 1 ad x 12 + x 22 = = 2 y 2, this solutio is feasible for the LP-relaxatio of the aggregated model i Equatio (10). However, the solutio is ot i P LPS, as it does ot satisfy all costraits i the LP-relaxatio of the separated model: x 11 = 1 > 1 2 = y 1. Therefore, we have the followig relatio o the feasible solutio sets: P ILP P LPS P LPA. Let the optimal values of the three solutio sets be Z ILP, Z LPS ad Z LPA, the the relatio above implies that Z ILP Z LPS Z LPA, showig that the boud of the LP-relaxatio of the separated model is at least as good as the boud obtaied from the aggregated model of the Ucapacitated Facility Locatio problem.

13 lecture otes o iteger liear programmig 13 Miimum Spaig Tree A commo task whe workig o graphs is to fid a miimum spaig tree. A tree T of a coected, udirected graph G = (V, E) is a subset of the edges E that is both coected ad without cycles. Tree T is a spaig tree if it coects all vertices V i G. Let c e be the cost of icludig edge e E i the spaig tree, the a miimum spaig tree is a spaig tree of miimum total cost of the icluded edges. A example is show i Figure 7. Miimum spaig trees are usually costructed with a optimal, greedy algorithm, as it is much more efficiet tha kow ILP models. Nevertheless, formulatig the Miimum Spaig Tree problem as a iteger liear program allows us to look at some useful modelig patters ad provides aother opportuity to compare LP-relaxatios. We will formulate two differet models of the Miimum Spaig Tree problem i this sectio. Both models use the same biary variables, oe for each edge e E i the graph: 1 if e is icluded i the spaig tree, x e = 0 otherwise. E A 0 D C 1 1 Figure 7: A udirected graph with edge costs. The edges of a miimum spaig tree are colored. B We use two basic properties of a spaig tree T to create the first model: 1. T = V 1, i.e. the size of the tree is precisely oe less tha the umber of vertices i the graph T has o cycles. 16 The proof of this property is left as a exercise to the reader. Have fu. The first property ca be formulated directly as a costrait. To model the secod property, we use the (equivalet) property that ay subset S V coected by S or more edges cotais a cycle. Let E(S) = {e = (v, w) E v, w S}, be the set of edges betwee vertices i S V, the the spaig tree T has o cycle i the set of vertices S if the sum of all x e with e E(S) is at most S 1. This type of costrait is kow as a subtour elimiatio costrait. To esure that the etire spaig tree does ot cotai a cycle, we have to add such a costrait for every subset S of V, as ca be see i the first ILP model i Equatio (11),

14 lecture otes o iteger liear programmig 14 mi c e x e e E (11a) x e = V 1 (11b) e E e E(S) x e S 1 S V (subtour) (11c) x e {0, 1} e E. (11d) Our secod model does ot traslate the acyclicity property directly to a set of costraits. Istead of subtour elimiatio, it uses the property that i ay partitio (S, V \ S) 17 of the graph G = (V, E) 17 A partitio (S, V \ S) of a graph G = the spaig tree has at least oe edge from S to V \ S. Let the cut set δ(s) of S V be the set of edges coectig S to V \ S, δ(s) = {e = (v, w) E v S, w V \ S}, (V, E) splits the graph ito two sets, S V ad V \ S, removig all edges that do ot coect a vertex i S to a vertex i V \ S. That is, the partitio (S, V \ S) is the graph G = (V, E ) with the the cut set model is E = {e = (v, w) E v S, w V \ S}. mi c e x e e E (12a) x e = V 1 (12b) e E e δ(s) x e 1 S V (cut set) (12c) x e {0, 1} e E. (12d) Ufortuately, the umber of costraits i the two models grows expoetially with the size of the graph due to Equatio (11b) ad (12c). Although this makes both models impractical for large graphs, we ca prove that the LP-relaxatio of the subtour model gives a stroger lower boud o the optimum. Theorem 2. The lower boud o the optimal value of the Miimum Spaig Tree problem obtaied from the LP-relaxatio of the subtour model, Equatio (11), is as least as high as the boud of the LP-relaxatio of the cut set model, Equatio (12). Proof. Let P LPS ad P LPC be the feasible regios of the LP-relaxatios of respectively the subtour model ad the cut set model. We start by showig that each feasible solutio to relaxed subtour model is feasible for the relaxed cut set models as well. First ote that, for ay subset S of V, we have that E(S) δ(s) E(V \ S) = E, (13)

15 lecture otes o iteger liear programmig 15 as each edge e E coects either two vertices i S, two vertices i V \ S or a vertex i S to a vertex i V \ S. Let x = {x e e E} P LPS, i.e. a feasible solutio to the LPrelaxatio of the subtour model, the for ay vertex set S V V 1 = x e e E = x e + e E(S) S 1 + e δ(s) e δ(s) = V 2 + x e, e δ(s) which we ca rewrite to x e + e E(V\S) x e + V S 1 by Equatio (11b) x e by Equatio (13) by Equatio (11c) x e 1. (14) e δ(s) As this holds for every subset of vertices i V, ay feasible solutio to the LP-relaxatio of model (11) satisfies the cut set costraits i Equatio (12c); hece, x belogs to P LPC, the feasible regio of the LP-relaxatio of the cut set model. The example i Figure 8 shows that coverse does ot hold, as some solutios i P LPC are ot feasible with respect to the LPrelaxatio of the subtour model. We coclude that E A 1 D 1/2 1/2 C 1/2 1/2 Figure 8: A fractioal, feasible solutio to the LP-relaxatio of the cut set model, Equatio (12), with the values of variables x e o the edges. The total cost of this solutio is 3/2. The colored edges do ot satisfy the subtour costrait i Equatio (11c). 1 B P LPS P LPC, ad, aalogue to the proof of Theorem 1, that the lower boud of subtour model is at least as good as the lower boud of the cut set model. Modelig the Miimum Spaig Tree problem as a ILP might seem like a poitless exercise, as both models cotai a expoetial umber of costraits, ad faster algorithms are available for this problem. However, the two costrait types show i this sectio are applicable to other, more difficult problems as well. For example, a solutio to the classical Travellig Sales Perso problem, where we have to fid a shortest tour that visits all vertices i a graph, should ot cotai ay cycles smaller tha the umber of vertices i the graph. A ILP model with subtour costraits is much more iterestig 18 for such a computatioally hard problem, tha it is for the Miimum Spaig Tree problem. Solvig a ILP: brach-ad-boud 18 We still have the mior problem of the expoetial umber of subtour costraits. However, it turs out that, i practice, you rarely eed all of them to fid a solutio without subtours. A commo strategy is to start without (may) subtour costraits, ad iteratively solve the model, addig ew costraits for ay subtours i the resultig solutio, util a solutio without subtours is foud. We have see examples of modelig optimizatio problems as iteger liear programs, ad showed that we ca get a boud o the optimal

16 lecture otes o iteger liear programmig 16 objective value by relaxig the itegrality costraits. However, this does ot provide us with a algorithm to actually solve the ILP models. We might be fortuate eough to obtai a itegral solutio from the LP-relaxatio, but this will ot happe for all problem types ad istaces. The stadard framework used to solve a ILP is brach-ad-boud, where we repeatedly divide the problem i smaller subproblems. This framework creates a tree structure, with at the root the origial problem. Each paret ode i the tree splits, or braches, ito multiple child odes by creatig copies of the paret problem with additioal costraits, such that the child solutio spaces partitio the paret solutio space. If we would oly brach, the the leaves of the tree would correspod to subproblems that either are ifeasible or have all their variables fixed. By costructig the etire tree, we eumerate all feasible solutios to the origial problem, ad are thus guarateed to fid the optimum. However, as the size of the tree teds to grow expoetially with the umber of variables i the model, it will take a log time to actually fid the optimal solutio. To avoid complete eumeratio of solutio space, we brig the boudig part of brach-ad-boud ito play. It uses the LP-relaxatio of the iteger liear program of the problem at each ode to obtai a lower boud i case of a miimizatio problem 19 ad a heuristic that produces a upper boud o the optimal value by costructig a good, itegral solutio 20. Istead of simply brachig at each possible ode, we use the bouds o the optimal value of the subproblems to elimiate or boud upromisig braches early. There are the four differet outcomes at each ode: 1. The subproblem of the brach is ifeasible. I this case, we do ot split the subproblem i smaller parts. This ode becomes a leaf. 19 I a maximizatio problem, the LPrelaxatio gives the upper boud ad the costructio heuristic the lower boud. 20 May heuristics ca be used to fid feasible, but ot ecessarily optimal, solutios to the ILP. A simple example is to roud a fractioal solutio of the LP-relaxatio to feasible iteger values. 2. The solutio of the LP-relaxatio is itegral. We have foud the optimal solutio of this ode, so further brachig o this subproblem will ot result i a better solutio. This ode becomes a leaf. We update the best solutio see so far if ecessary. 3. The lower boud o the curret subproblem is at least as high as the objective value of the best solutio foud so far. Sice we have already discovered a solutio that is better tha the best this brach has to offer, we gai othig by explorig it ay further. This ode becomes a leaf. 4. The lower boud o the curret subproblem is lower tha the objective value of the best solutio foud so far. If we have a heuristic available, we will compute a upper boud o this subproblem,

17 lecture otes o iteger liear programmig 17 ad update the best solutio see so far if eeded. We cotiue brachig o this ode. As this procedure oly prues upromisig braches, we get a tremedous speed up of the search process without losig the optimal solutio. However, brach-ad-boud does ot provide ay guaratees o the computatio time, ad is heavily affected by the implemetatio. As metioed earlier, brach-ad-boud is a framework, meaig that may parts have to be filled i by the user: Model Clearly, the model has a large ifluece o the search process. It determies the variables o which we ca brach as well as the boud of the LP-relaxatio. The better the boud of the LPrelaxatio, the more likely it is that we ca elimiate bad braches early. Iteger heuristic Similar to the LP-relaxatio boud, a good heuristic for itegral solutios allows early pruig of upromisig braches, reducig the solutio space. Search strategy Sice we are essetially costructig a tree ode by ode durig brach-ad-boud, we eed to decide durig the search, whe the tree is ot yet costructed fully, which ode of the tree we wish to evaluate ad brach o. Possible approaches are depth-first, breadth-first or selectio strategies that expad the most promisig ode. Brachig strategy I additio to selectig a ode to brach o, we eed to decide how we will brach. That is, how will we split the solutio space of the selected ode. A commo approach whe brachig o decisio variables is to split its domai ito two parts. For example, if we have a biary decisio variable x i {0, 1}, we ca brach ito x i = 0 ad x i = 1. I case of a itegral decisio variable x j 0, we might create two 21 braches, splittig the domai i 0 x j 4 ad x j 5. If the subproblem cotais more tha oe decisio variable o which we ca brach, we eed to select oe of them. For example, a simple strategy for decisio variables is to select the variable with the most fractioal 22 value. Brachig o other aspects, such as x i + x j 6 ad x i + x j 7, is possible as well, as log as at least oe of the braches preserves the optimal solutio of the paret ode i the brach-ad-boud tree. 21 Similar to a biary decisio variable, if a itegral decisio variable has a upper boud of k, with k ot too large, we brach ito k child odes, oe for each possible value of the variable. 22 The ratioale behid this strategy is that fixig a highly fractioal close to 1/2 biary variable is likely to have a big impact o the overall solutio, which hopefully allows us to prue the brach with wrog decisio early i the search. Kapsack (Revisited) To illustrate the brach-ad-boud framework, let us cosider the followig example of the Kapsack problem. Our kapsack has a

18 lecture otes o iteger liear programmig 18 capacity B of 15, ad we have five items that we could carry i the backpack. The values c i ad weights a i are show i Table 2. Usig these data as iput for ILP formulatio (5) of the Kapsack problem gives us model (15): max 8x x 2 + 7x x x 5 4x 1 + 8x 2 + 3x 3 + 6x 4 + 5x 5 15 x 1, x 2, x 3, x 4, x 5 {0, 1}. (15) To use the brach-ad-boud framework, we eed to specify the search ad brachig strategies. The simple breadth-first-search strategy is used to explore the brach-ad-boud tree, evaluatig the odes level by level. We base our brachig strategy o the ituitio that items with a large value-to-weight ratio are more favorable to put i the kapsack tha low value, high weight items i the Kapsack problem. Therefore, it is likely that the exclusio of a item with a high value-to-weight ratio will lead to a bad solutio, allowig us to cut off the correspodig brach i the tree early i the search process. The brachig strategy that exploits this idea is to brach o the biary decisio variables i o-icreasig value-to-weight ratio: x 5 x 4 x 3 x 1 x 2. Lower bouds ca be obtaied at each ode by roudig dow the fractioal part of the optimal solutio 23 to the LP-relaxatio. Figure 9 shows the brach-ad-boud tree of our example. The exploratio order of the odes of the brach-ad boud tree is show below: N 0. The root ode correspodig to the origial ILP model show i(15). The lower boud heuristic costructs a iitial solutio; the global lower boud becomes 34. We brach o decisio variable x 5. i c i a i c i/a i 2 11/2 21/3 21/2 22/5 Table 2: The values c i, weights a i ad value-to-weight ratios of the five items i Kapsack problem. 23 A optimal solutio to the LPrelaxatio ca be costructed efficietly by greedily selectig items with the highest value-to-weight ratio util the kapsack is full. N 1. The upper boud o the optimal value of this ode is less tha the curret best solutio. We elimiate this brach. N 2. The optimal solutio to the LP-relaxatio is equal to the solutio at the root. We brach o x 4. N 3. The upper boud of this ode is less tha the curret best solutio. We elimiate this brach. N 4. The optimal solutio of the LP-relaxatio is equal to the solutio at the root. We brach o x 3. N 5. The optimal solutio of the LP-relaxatio is itegral, ad is better tha the curret best solutio; the global lower boud becomes 35. As the solutio to the LP-relaxatio is optimal for the ILP as well, we do ot brach o this ode.

19 lecture otes o iteger liear programmig 19 N 0 1/4, 0, 1, 1, x 5 = 0 x 5 = 1 1, 1/4, 1, 1, 0 1/4, 0, 1, 1, 1 N N 3 N x 4 = 0 1, 3/8, 1, 0, 1 N /2 x 3 = 0 1, 0, 0, 1, 1 N N 7 x 4 = 1 1/4, 0, 1, 1, x 3 = 1 1/4, 0, 1, 1, 1 N x 1 = 0 0, 1/8, 1, 1, 1 N /2 x 1 = 1 1, 0, 1, 1, 1 ifeasible Figure 9: A brach-ad-boud tree for the istace of Kapsack show i Table 2. Each ode shows at the top a optimal solutio to the LP-relaxatio of the subproblem, ad at the bottom right the correspodig objective value. This is a upper boud o the optimal solutio to the ILP. A lower boud, obtaied by roudig dow fractioal decisio variables i the LP solutio, ca be foud i the bottom left of a ode. The labels at the arcs show brachig choices. A lower boud is uderlied if it improves the curret best lower boud. Bold values i the solutio idicate variables that are fixed by brachig. The brach-ad-boud tree shows that 35 is the optimal value of this Kapsack istace. x 2 = 0 x 2 = 1 N 9 0, 0, 1, 1, N 10 0, 1, 1, 1, 1 ifeasible

20 lecture otes o iteger liear programmig 20 N 6. Th optimal solutio of the LP-relaxatio is equal to the solutio at the root. We brach o x 1. N 7. We brach o x 2. N 8. The ode is ifeasible, because the total weight of the fixed items, 18, exceed the maximum capacity of the kapsack. We do ot brach o this ode. N 9. The upper boud of this ode is less tha the curret best solutio. We elimiate this brach. N 10. The ode is ifeasible, because the total weight of the fixed items, 22, exceed the maximum capacity of the kapsack. We do ot brach o this ode. The optimal solutio is the the best solutio that we have see over all odes. I this case, the solutio cosists of items 1, 4 ad 5, with a total value of 35. Valid Iequalities Brach-ad-boud relies heavily o a strog LP-relaxatio boud o the optimum of each ode i the search tree. I previous sectios we have see that modelig choices i the ILP affect the stregth of the boud. However, a good model is ot always sufficiet to fid the itegral optimum i reasoable time. I that case, we ca stregthe the LP-relaxatio by itroducig additioal costraits, called valid iequalities or cuttig plaes. These costraits cut off part of the LP solutio space to tighte the boud o the itegral optimum, without removig itegral solutios. By preservig the ILP space, we esure that the stregtheed model remais valid with regard to the origial iteger liear program. Ehacig brach-ad-boud by addig the valid iequalities at each ode to cut off fractioal solutios is kow as brach-ad-cut. Gomory s cuts Suppose that we have a iteger liear program with decisio variables x 1,..., x 0, ad a costrait of the form a i x i = b. i The, for ay itegral solutio that satisfies this costrait, the iequality obtaied by roudig dow the costat ad the coefficiets i the costrait, a i x i b, i

21 lecture otes o iteger liear programmig 21 holds as well, sice the decisio variables are all o-egative. As the iequality does ot cut off itegral solutios, it is valid for the ILP model. This type of valid iequality is kow as Gomory s cut. Oe of the advatages of Gomory cuts is that we ca geerate ew cuttig plaes efficietly from the fial dictioary 24 of the LPrelaxatio. Cosider the followig example ILP, max 2x 1 + x 2 x 1 x 2 1 2x 1 + 2x 2 7 x 1, x 2 N 0, depicted i Figure 10. We tur a iequality costraits to a equality with a slack variable, modelig the gap betwee the left- ad right-had side of the origial iequality. Itroducig slack variables x 3 ad x 4 i our model yields max 2x 1 + x 2 x 1 x 2 + x 3 = 1 (16) (17) 24 The dictioary format of a solutio to a liear program expresses all o-zero decisio variables (after itroducig slack variables) i the remaiig zerovalued decisio variables. See the first two chapters of Chvatal for more o this topic x 2 2x 1 + 2x 2 7 x 1 x objective Figure 10: The feasible regio of the LP-relaxatio of model (16). x x 1 + x2 2x 1 + 2x 2 + x 4 = 7 x 1, x 2, x 3, x 4 N 0. The optimal solutio of the LP-relaxatio of this model is x 1 = 21/3, x 2 = 11/4, x 3 = x 4 = 0, with a value of 53/4. I the dictioary of this solutio, the o-zero decisio variables x 1 ad x 2 are expressed i the other, zero-valued decisio variables, x 1 = x x 4 (18) x 2 = x x 4. (19) We ca derive a Gomory cut from the costrait of ay fractioal decisio variable. For example, as x 2 = 1 1 4, rewritig Equatio (19) to x x x 4 = 1 1 4, ad roudig dow gives the Gomory cut x x x 4 = x 2 x = 1. (20) 4 Not oly are Gomory cuts geerated efficietly from the fial dictioary of a LP-relaxatio, these cuttig plaes are also effective. I

22 lecture otes o iteger liear programmig 22 our example, the curret optimal LP-relaxatio solutio does ot satisfy iequality (20), sice > 1. The property of cuttig off the curret optimal solutio of the LPrelaxatio holds for ay Gomory cut geerated from the costrait of a fractio decisio variable. This property is a direct result of the expressig the o-zero decisio variable i terms of the zero-valued decisio variables: x k = b k + a i x i = b k > b k if x i is fractioal. i:x i =0 4 x 2 Gomory cut 12 Therefore, addig the valid iequality i Equatio (20) as a costrait to our example model model (17) ad solvig the LP-relaxatio yields a ew optimum of 51/2 at x 1 = 2, x 2 = 11/2, as is illustrated i Figure 11. Sice the optimum of the LP-relaxatio of the exteded model is fractioal as well, we ca repeat the process of addig Gomory cuts ad solvig the resultig LP-relaxatio, util a itegral solutio is foud. This procedure, the cuttig plae algorithm, is guarateed to coverge to a itegral solutio, although the umber of cuttig plaes added to the model ca be expoetial. I brach-ad-cut, we do ot ecessarily have to cotiue util a itegral solutio is foud, as a good boud o the objective of a ode i the search tree is sufficiet i may cases objective x 1 0 2x 1 + x2 Figure 11: The feasible regio of the LP-relaxatio of model (16) with the Gomory cut i Equatio (20). The gray area is removed by the Gomory cut. Problem specific cuts Although the previous valid iequalities are geerally applicable, cuttig plaes that exploit the structure of a problem are ofte able to stregthe the boud of a LP-relaxatio much more efficietly. I this sectio we will look at two of these problem specific valid iequalities. The first example relates to the Maximum Idepedet Set problem. It is based o the observatio that at most half of the vertices i a cycle i a graph ca be icluded i a idepedet set. This results for model (6) i the valid iequality i C x i C 1, 2 where C V is a odd cycle i the graph. The odd cycle 25 iequality elimiates fractioal solutios such as the oe show i Figure 12. Aother type of problem specific valid iequalities ca be derived for the Kapsack problem. Suppose that we have a subset of items 1/2 1/2 1/2 1/2 1/2 Figure 12: The fractioal solutio to the Maximum Idepedet Set problem that assigs 1/2 to each vertex i the odd cycle is optimal, with a value of 21/2, but does ot satisfy the odd cycle iequality. 25 Of course, a similar iequality exists for eve cycles as well. However, as feasible fractioal solutio satisfy the eve cycle iequality, we do ot stregthe the LP-relaxatio boud by addig it to our model.

23 lecture otes o iteger liear programmig 23 I {1,..., } with a total weight larger tha the capacity of our kapsack. The we kow that we caot carry all items i the subset with us, i.e. every feasible solutio i model (5) should satisfy x i I 1. i I This type of cuttig plae, kow as a cover iequality, is applicable to may other problems as well, ad is commoly used strategy i moder solvers to tighte the boud of the LP-relaxatio. As a example of its usage, take the subset of items I = {1, 3, 4, 5} for the problem istace listed i Table 2. Sice a i = > 15, i I the set I covers our kapsack. The correspodig cover iequality x 1 + x 3 + x 4 + x 5 3 cuts off the optimal solutio to the LP-relaxatio, x 3 = x 4 = x 5 = 1, x 1 = 1/4. Refereces Vasek Chvatal. Liear Programmig. Macmilla, 1983.