Integer Programming. Leo Liberti. March 13, Abstract

Transcription

1 Integer Programming Leo Liberti LIX, École Polytechnique, F-9118 Palaiseau, France March 1, 6 Abstract A short introduction to Integer Programming (IP). Problems leading to IP models. Some modelling tricks and reformulations. Geometry of linear IP. TUM matrices. Brief notes on polyhedral analysis. Separation theory. Chvatal cut hierarchy, Gomory cuts, Disjunctive cuts, RLT cut hierarchy. Iterative methods: Branch-and-Bound, Cutting Plane, Branch-and-Cut, Branch-and-Price. Lower bounds: Lagrangian relaxation and subgradient method. Upper bounds: heuristics. Contents 1 Introduction 1.1 Canonical and Standard Forms Modelling Tricks Geometrical and Structural Properties of ILP.1 Totally Unimodular Matrices Polyhedral Analysis 7.1 Separation Theory Chvátal Cut Hierarchy Gomory Cuts Gomory Cutting Plane Algorithm: Example Disjunctive Cuts Lifting RLT Cut Hierarchy Enumerative Methods for Integer Programming 1.1 Branch-and-Bound Example Branch-and-Cut

2 1 INTRODUCTION. Branch-and-Price Heuristics and Bounds Lower Bounds: Lagrangian Relaxation and Subgradient Method Conclusion 17 1 Introduction Many real-world problems naturally require solutions of a discrete type. We might be asked how many containers we should route from a set of manufacturing plants to a number of customer sites, subject to production limits and delivery requirements, so that the transportation costs are minimized. In another setting, we might have to decide where to install some servicing facilities so that every neighbouring town is guaranteed service, at minimum cost. The first is an abstract description of a transportation problem, which calls for discrete variables (the number of containers routed from plant i to customer j). The second describes a set covering problem, which relies on binary variables which tell whether a servicing facility will be built on a geographical region i. A more abstract description of the set covering problem is that, given a set X and a subset cost function c : P R, we want to find a subset system S P(X) so that S = X and s S c(s) is minimized. The fact that integer decision variables are required to solve these problems makes it impossible to model them by linear programming (LP) methods alone, as appears from their formulations below. 1.1 Example Transportation Problem. Let x ij be the (discrete) number of product units transported from plant i to customer j. We model the problem as follows: m n max x c ij x ij i=1 j=1 n i m x ij l i (1) j=1 m j n x ij d j i=1 i=1 where m is the number of plants, n is the number of customers, c ij is the unit transportation cost from plant i to customer j, l i are the production limits at the plants and d j are the demands at the customer sites. 1. Example Set Covering Problem. Let x i = 1 if a servicing facility will be built on geographical region i (out of m possible regions), and otherwise. We model the problem as follows: m min x c i x i i=1 m () j n a ij x i 1 where m is the number of regions, n is the number of towns to be serviced, c i is the cost of building a facility on region j and a ij = 1 if a facility on region i can serve town j, and otherwise.

3 1 INTRODUCTION This essay describes the standard techniques for modelling Integer Programming problems (IP) and solving them. Notice that we deal almost exclusively with integer linear programs (ILP). The rest of this section is devoted to basic definitions, modelling tricks and reformulation. Section deals with the geometrical and structural properties of an integer program. In Section we give brief notes about polyhedral analysis. Section is a resumé of the most popular iterative methods for solving ILPs. Finally, in section 5 we discuss techniques for finding lower and upper bounds to integer programs. Most of the material has been taken from [Wol98, NW88, PS98, Fis99, KV]. 1.1 Canonical and Standard Forms An ILP is in canonical form if it is modelled as follows: min x s.t. c x Ax b x x Z n, () where x Z n are the decision variables, c R n is a cost vector, A is an m n matrix with integral entries, and b an integral vector in Z m. An ILP is in standard form if it is modelled as follows: min x s.t. c x Ax = b x x Z n, () again, x Z n are the decision variables, c R n is a cost vector, A is an m n matrix with integral entries (assume rank(a) = m), and b an integral vector in Z m. As in LP theory, a canonical ILP problem can be put into standard form via the introduction of slack variables (one per constraint). Notice that an ILP is nonlinear. In fact, every integer variable x Z can be replaced by a continuous variable and an integrality enforcing constraint sin(πx) =. A problem where both integer and continuous variables appear is called a mixed-integer linear problem (MILP) or mixed-integer nonlinear problem (MINLP). Since the -Sat problem (finding a feasible solution of a conjunctive normal form system of boolean formulas with variables) is N P-complete and -Sat can be modelled as an ILP problem, it follows that the class of ILP problems is N P-hard. 1. Modelling Tricks Whereas continuous LPs are used to model situations where quantitative decisions must be taken, ILPs express a qualitative type of decision. 1. To impose the fact that exactly one choice x i is to be taken among many possibilities x 1,...,x n, use a constraint as follows: n x i = 1. Similar constraints hold for at least one choice ( 1) and at most one choice ( 1) i=1. The disjunctive constraint either a 1 x b 1 or a x b

4 GEOMETRICAL AND STRUCTURAL PROPERTIES OF ILP can be formulated as follows: a 1 x b 1 + ym 1 a x b + (1 y)m, where M 1,M are upper bounds on the values attained by a 1 x, a x. Basically, where y = the first constraint is active and the second one is inactive, due to the big M constant. When y = 1 the situation is reversed.. If-then type constraints can be reformulated to disjunctive constraints by observing that in first order logic we have φ ψ is the same as φ ψ. So, to express the condition if a 1 x b 1 then a x b, we use disjunctive constraints reformulation for a 1 x b 1 and a x b.. An integer variable product x 1 x can be linearized exactly as follows: substitute the product x 1 x with a (continuous) variable y and introduce the following constraints: y x 1 y x y x 1 + x 1 y [,1]. Geometrical and Structural Properties of ILP Since by relaxing the integrality constraints of an ILP we obtain an LP problem, it should come to no surprise that the basic geometrical definitions concerning ILPs are much the same as those of LP. The solutions of an ILP in form () or () are the integer vectors in Z n inside the polyhedron P R n defined by the problem constraints..1 Definition A set S R n is convex if for any two points x,y S the segment between them is wholly contained in S, that is, λ [,1] (λx + (1 λ)y S). A linear equation a x = b where a R n defines a hyperplane in R n. The corresponding linear inequality a x b defines a closed half-space. Both hyperplanes and closed half-spaces are convex sets. Since any intersection of convex sets is a convex set, the subset of R n defined by the system of hyperplanes Ax b,x is convex.. Definition A subset S R n defined by a finite number of closed halfspaces is called a polyhedron. A bounded, non-empty polyhedron is a polytope.. Definition The set conv(s) of all convex combinations n i=1 λ ix i (with λ i for all i and n i=1 λ i = 1) of a finite set S of points {x i i n} in R n is called the convex hull of S. A convex combination is strict if for all i n we have < λ i < 1.. Definition Consider a polyhedron P and a closed half-space H in R n. Let H P be the affine closure of H P and d = dim(h P). If d < n then H P is a face of P. If d =, H P is called a vertex of P, and if d = 1, it is called an edge. If d = n 1 then H P is a facet of P.

5 GEOMETRICAL AND STRUCTURAL PROPERTIES OF ILP 5.1 Totally Unimodular Matrices Consider the following LP problem: min x s.t. c x Ax = b x x R n, where c,a,b are as in problem (). This problem is the continuous relaxation of the ILP (). By LP theory, the solution of problem (5) is a basic feasible solution x R n corresponding to a square, invertible m m matrix B composed of a set of basic columns of A, and we have that x = (B 1 b,). Furthermore, x is a vertex of the polyhedron P s = {x Ax = b}. The conditions for x to be a solution of () are that x Z n, that is, every component of B 1 b must be integer. Since b is an integer vector, A is an integer matrix, B is a submatrix of A, and B 1 = 1 det(b) C where C is an integer matrix whose entries are integral functions of the entries of B, a condition for B 1 b to have integral components is that det(b) = ±1..5 Definition An m n integral matrix A whose m m invertible square submatrices all have determinant ±1 is called unimodular. (5) We have just proved the following proposition..6 Proposition If A is unimodular, the vertices of the polyhedron P s = {x Ax = b} all have integral components Thus, any solution of the LP relaxation (5) is a solution of the ILP problem (). A similar condition can be imposed on A for a polyhedron P c = {x Ax b} corresponding to the ILP in canonical form ()..7 Definition An m n integral matrix A where all invertible square submatrices of any size have determinant ±1 is called totally unimodular (TUM). The following propositions give some characterizations of TUM matrices..8 Proposition A matrix A is TUM A is TUM (A,I) is TUM..9 Proposition An m n matrix A is TUM if: (a) for all i m, j n we have a ij {,1, 1}; (b) each column of A contains at most nonzero coefficients; (c) there is a partition R 1,R of the set of rows such that for all columns j having exactly nonzero coefficients, we have i R 1 a ij i R a ij =. Proof. Suppose that conditions (a), (b), (c) hold but A is not TUM. Let B be the smallest square submatrix of A such that det(b) {,1, 1}. B cannot contain a column with just one nonzero entry, otherwise B would not be minimal (just eliminate the row and column of B containing that single nonzero entry). Hence B must contain nonzero entries in every column. By condition (c), adding the rows in R 1 to those in R gives the zero vector, and hence det(b) =, contradicting the hypothesis. In the following lemma and theorem, we show why TUM matrices are important.

6 GEOMETRICAL AND STRUCTURAL PROPERTIES OF ILP 6.1 Lemma Let x P = {x Ax b}. Then x is a vertex of P if and only if for any pair of distinct points x 1,x P, x cannot be a strict convex combination of x 1,x. Proof. ( ) Suppose x is a vertex of P and there are points x 1,x P such that there is a λ (,1) with x = λx 1 + (1 λ)x. Since x is a vertex, there is a a halfspace H = {x R n h x d} such that H P = {x }. Thus x 1,x H and h x 1 > d, h x > d. Hence h x = h (λx 1 + (1 λ)x ) > d, whence x H, a contradiction. ( ) Assume that x cannot be a strict convex combination of any pair of distinct points x 1,x of P, and suppose that x is not a vertex of P. Since x is not a vertex, there is a ball S of radius ε > and center x wholly contained in P. Let x 1 S such that x 1 x = ε. Let x = (x x 1 ) + x. By construction, x is the point symmetric to x 1 on the line x 1 x. Thus, x x = ε, x x 1, x P, and x = 1 x x, which is a strict convex combination of x 1,x, a contradiction..11 Theorem If A is totally unimodular, the vertices of the polyhedron P c = {x Ax b} all have integral components. Proof. Let x be a vertex of P c, and s = Ax b. Then (x,s ) is a vertex of the standardized polyhedron P c = {(x,s) Ax s = b}: suppose it were not, then by Lemma.1 there would be two distinct points (x 1,s 1 ), (x,s ) in P c such that (x,s ) is a strict convex combination of (x 1,s 1 ), (x,s ), i.e. there would be a λ (,1) such that (x,s ) = λ(x 1,s 1 ) + (1 λ)(x,s ). Since s 1 = Ax 1 b and s = Ax b, both x 1 and x are in P c, and thus x = λx 1 + (1 λ)x is not a vertex of P c since < λ < 1, which is a contradiction. Now, since A is TUM, (A, I) is unimodular, and (x,s ) is an integral vector. The transportation problem (1) has a totally unimodular matrix, so that solving a continuous relaxation of the problem always yields an integral solution vector..1 Example Network Flow. One important class of problems with a TUM matrix is that of network flow problems. We have a network on a digraph G = (V,A) with a source node s, a destination node t, and capacities u ij on each arc (i,j). We have to determine the maximum amount of material flow that can circulate on the network from s to t. The variables x ij, defined for each arc (i,j) in the graph, denote the quantity of flow units. max x x si i δ + (s) i V,i s,i t x ij = x ji (6) j δ + (i) j δ (i) (i,j) A x ij u ij. Notice that the network flow problem is not strictly an ILP, since there is no integrality requirement on the amount of flow routed in the network. However, it does have a TUM matrix, and hence, provided the capacities u ij are integral, the solution is an integral vector. This class of problem is especially important as many common graph problems can be modelled by network flow constraints (e.g. shortest path problem). Even when this is not possible, sometimes it helps to model part of a problem with flow constraints, so that some of the integer variables in the resulting formulation can be relaxed to be continuous [Mac]. Note that whereas a TUM constraint matrix implies a polyhedron with integral vertices, the converse is not necessarily true. In particular, a sufficient condition for A to be TUM is that the polyhedron {x Ax = b} has integral vertices for all integral vectors b Z m.

7 POLYHEDRAL ANALYSIS 7 Polyhedral Analysis The feasible region of any ILP problem can be expressed as the convex hull of its feasible vectors. Thus, if the integrality constraints of the ILP are relaxed, solving its continuous relaxation by LP methods still yields the optimal solution of the ILP. Since LP is in P, there is a definite advantage in being able to express the feasible region of any ILP as the convex hull of its feasible space. Unfortunately, given a feasible region in canonical form Ax b,x, finding the convex hull P of the feasible vectors is a hard task. Even though any polyhedron (hence even P) can be expressed by a finite number of inequalities, the required number might be exponential in the size of the instance, thus rendering the whole exercise useless. Nonetheless, this concept spawned a lot of research, from the 197s and onwards, based on finding the inequalities defining P. This field is called polyhedral analysis. Consider an ILP min{c x x Z n P } where P = {x Ax b} and such that P is the convex hull of its feasible integral solutions (thus min{c x x P } has the same solution as the original ILP). We are interested in finding explicit linear constraints defining the facets of P. The polyhedral analysis approach is usually expressed as follows: given an integral vector set X Z n and a valid inequality h x d for X, show that the inequality is a facet of conv(x). We present here two approaches. 1. Find n points x 1,...,x n X such that h x i = d i n and show that these points are affinely independent (i.e. that the n 1 directions x x 1,..., x n x 1 are linearly independent).. Select t n points x 1,...,x t satisfying the inequality. Suppose that all these points are on a generic hyperplane µ x = µ. Solve the equation system k t n x kj µ j = µ j=1 in the t + 1 unknowns (µ j,µ ). If the solution is (λh j,λd) with λ then the inequality h x d is facet-defining. There are various theoretical approaches to show that a given polyhedron is a convex hull of an integral vector set; however most of these approaches are difficult to generalize. One of the most stringent limits of polyhedral analysis is that the theoretical devices used to find facets for a given ILP problem are often unique to that very problem..1 Separation Theory Finding all the facets of P is overkill, however. Since the optimization direction points us towards a specific region of the feasible polyhedron, what we really need is an oracle that, given a point x P, tells us that x P or else produces a separating hyperplane h x = d such that for all h x d for all x P and h x > d (adding the constraint h x d to the ILP problem formulation then tightens the feasible region P). The problem of finding a valid separating hyperplane is the separation problem. It is desirable that the separation problem for a given NP-hard problem should have polynomial complexity. The classic example of a polynomial separation problem for an NP-hard problem is the TSP. Consider the undirected TSP of finding the shortest Hamiltonian cycle in an indirected complete weighted graph K n (with weights c ij on the edges {i,j}) with n vertices. Since a Hamiltonian tour is also a -matching, the following ILP model has a feasible region which contains the set of Hamiltonian tours as a proper subset: min s.t. c ij x ij x ij = i V i j V i j V x ij {,1} i j V. (7)

8 POLYHEDRAL ANALYSIS 8 Some of the feasible solutions are given by disjoint cycles. However, we can exclude them from the feasible region by requesting that each cut should have cardinality at least, as follows: S V ( x ij ). i S,j S There is an exponential number of such constraints (one for each subset S of V ), however we do not need them all: we can add them iteratively by identifying cuts δ(s) where {i,j} δ(s) x ij is minimum. We can see this as the problem of finding a cut of minimum capacity in a flow network (so the current values x ij are used as arc capacities each edge is replaced by opposing arcs). The problem is the LP dual of finding the maximum flow in the network, and we can solve that problem in with a polynomial algorithm (O(n ) if we use the Goldberg-Tarjan push-relabel algorithm). So if each cut has capacity at least, the solution is an optimal Hamiltonian cycle. Otherwise, supposing that δ(s) has capacity K <, the following is a valid cut: x ij, {i,j} δ(s) which can be added to the formulation. The problem is then re-solved iteratively to get a new current solution until the optimality conditions are satisfied. In the following sections (.-.6) we shall give examples of four different cut families which can be used to separate the incumbent (fractional) solution from the convex hull of the integral feasible set.. Chvátal Cut Hierarchy A cut hierarchy is a finite sequence of cut families that generate tighter and tighter relaxed feasible sets P = P P 1... P k = P, where P = conv(x). Given the relaxed feasible region P = {x Ax = b} in standard form, define the Chvátal first-level closure of P as P 1 = {x Ax = b, u R n + ua ub }. Observe that although formally the number of inequalities defining P 1 is infinite, in fact it can be shown that these can be reduced to a finite number. We can proceed in this fashion by transforming the Chvátal first-level inequalities to equations via the introduction of slack variables and reiterating the process on P 1 to obtain the Chvátal second-level closure P of P, and so on. It can be shown that any fractional vertex of P can be cut by a Chvátal inequality.. Gomory Cuts Gomory cuts are in fact a special kind of Chvátal cuts. Their fundamental property is that they can be inserted in the simplex tableau very easily, and thus are very efficient in cutting plane algorithms, which generate cuts iteratively in function of the current incumbent. Suppose x is the optimal solution found by the simplex algorithm running on a continuous relaxation of the ILP. Assume x h is fractional. Since x h, column h is a basic column; thus there corresponds a row t in the simplex tableau: x h + ā tj x j = b t, (8) j ν where ν are the nonbasic variable indices, ā tj is a component of B 1 A (B is the nonsingular square matrix of the current basic columns of A) and b t is a component of B 1 b. Since ā tj ā tj for each row index t and column index j, x h + j ν ā tj x j b t.

9 POLYHEDRAL ANALYSIS 9 Furthermore, since the LHS must be integer, we can restrict the RHS to be integer too: x h + j ν ā tj x j b t. (9) We now subtract (9) from (8) to obtain the Gomory cut: ( ā tj ā tj )x j ( b t b t ). (1) j ν We can subsequently add a slack variable to the Gomory cut, transform it to an equation, and easily add it back to the current dual simplex tableau as the last row with the slack variable in the current basis...1 Gomory Cutting Plane Algorithm: Example In this section we illustrate the application of Gomory cut in an iterative fashion in a Cutting Plane algorithm. The cutting plane algorithm works by solving a continuous relaxation at each step. If the continuous relaxation solution fails to be integral, a separating cutting plane (a valid Gomory cut) is generated and added to the formulation, and the process is repeated. The algorithm terminates when the continuous relaxation solution is integral. Let us solve the following IP in standard form: min x 1 x t.c. x 1 + 6x + x = 9 x 1 + x + x = x, x 1,x Z In this example, x,x can be seen as slack variables added to an original problem in canonical form with inequality constraints expressed in x 1,x only. We identify x B = (x,x ) as an initial feasible basis; we apply the simplex algorithm obtaining the following tableaux sequence, where the pivot element is framed. x 1 x x x x 1 x x x x 1 x x x The solution of the continuous relaxation is x = (, 5 ), where x = x =.

10 POLYHEDRAL ANALYSIS 1 x (1) () We derive a Gomory cut from the first row of the optimal tableau: x x + 5 x = 5. The cut is formulated as follows: x i + j N ā ij x j b i, (11) where N is the set of nonbasic variable indices and i is the index of the chosen row. In this case we obtain the constraint x. We introduce this Gomory cut in the current tableau. Note that if we insert a valid cut in a simplex tableau, the current basis becomes primal infeasible, thus a dual simplex iteration is needed. First of all, we express x in terms of the current nonbasic variables x,x. We subtract the i-th optimal tableau row from (11), obtaining: x i + j N ā ij x j b i j N( ā ij ā ij )x j ( b i b i ) 1 1 x 5 x 1. Recall that the simplex algorithm requires the constraints in equation (not inequality) form, so we introduce a slack variable x 5 in the problem: 1 1 x 5 x + x 5 = 1. We add this constraint as the bottom row of the optimal tableau. We now have a current tableau with an added row and an added column, corresponding to the new cut and the new slack variable (which is in the basis): x 1 x x x x

11 POLYHEDRAL ANALYSIS 11 The new row corresponds to the Gomory cut x (labelled constraint () in the figure below). x x () (1) () We carry out an iteration of the dual simplex algorithm using this modified tableau. The reduced costs are all non-negative, but b = 1 < implies that x 5 = b has negative value, so it is not primal feasible (as x 5 is now a valid problem constraint). We pick x 5 to exit the basis. The variable j entering the basis is given by: j = argmin{ c j ā ij j n ā ij < }. In this case, j = argmin{, 1 }, corresponding to j =. Thus x enters the basis replacing x 5 (the pivot element is indicated in the above tableau). The new tableau is: x 1 x x x x The optimal solution is x = (,). Since this solution is not integral, we continue. We pick the second tableau row: x 1 1 x + x 5 =, to generate a Gomory cut which, written in terms of the variables x 1,x is x 1 x + x 5, x 1 + 5x 7. This cut can be written as: x 1 x 5. The new tableau is:

12 POLYHEDRAL ANALYSIS 1 x 1 x x x x 5 x The pivot is framed; exiting row è was chosen as b <, entering column 5 as Pivoting, we obtain: c ā = 1 < 1 = c5 ā 5 ). x 1 x x x x 5 x This tableau has optimal solution x = (1,), which is integral (and hence optimal). Figure below shows the optimal solution and the last Gomory cut to be generated. x () (1) x x () (). Disjunctive Cuts We saw in Section 1., point, that disjunctive constraints can be modelled by resorting to ILP techniques. Suppose we are working in a feasible region consisting of a disjunction of two polyhedra P 1,P, and consider two inequalities h 1 x d 1 (valid in P 1 ), h x d (valid in P ). Choose h such that for all i n h i min(h 1i,h i ), and d such that d max(d 1,d ). Then the inequality h x d is valid for P 1 P (it is called disjunctive cut).

13 ENUMERATIVE METHODS FOR INTEGER PROGRAMMING 1.5 Lifting It is possible to derive valid inequalities for a given ILP min{cx Ax b x {,1} n } (1) by lifting lower-dimensional inequalities to a higher space. If the inequality n j= π jx j π (with π j for all j {,...,n}) is valid for the restricted ILP n min{ c j x j j= n a ij x j b i i m x j {,1} j {,...,n}}, j= then by letting π 1 = max (π n π j x j ) j= n a ij x j b i a i1 i m j= x j {,1} j {,...,n}. we can find the inequality n j=1 π jx j π which is valid for (1), as long as the above problem is not infeasible; if it is, then x 1 = is a valid inequality. Of course the lifting process can be carried out with respect to any variable index (not just 1)..6 RLT Cut Hierarchy This cut hierarchy, described in [SD], is a nonlinear lifting. RLT stands for Reformulation and Linearization Technique. One advantage of this cut hierarchy with respect to the Chvátal one is that the RLT can be applied to Mixed-Integer Linear Programs (MILP) very effectively. From each constraint a i x b i we derive the i-th constraint factor γ(i) = a i x b i (i m). We then form the bound products x(j 1,J ) = i J 1 x i i J (1 x i ) for all partitions J 1,J of {1,...,d}. The RLT d-th level closure is obtained by multiplying each constraint factor γ(i) by all the bound products. For all i m, for all J 1,J partitioning d, we derive the following valid inequality x(j 1,J )γ(i). (1) Notice that these inequalities are nonlinear. After having reduced all occurrences of x i via the valid integer equation x i = x i, we now linearize the inequalities (1) by lifting them in higher-dimensional space: substitute all products i J x i by a new added variable w J = i J x i, to obtain linearized cuts of the form l i J 1,J (x,w), (1) where l i J 1,J is a linear function of x,w. We define P d = {x i m and J 1,J partitioning d, l i J 1,J (x,w) }. This cut hierarchy has two main shortcomings: (a) the high number of added variables and (b) the huge number of generated constraints. It is sometimes used heuristically to generate tight continuous relaxations of the ILP problem to be used in Branch-and-Bound algorithms. Enumerative Methods for Integer Programming In this section we shall very briefly describe some of the most popular solution methods for the solution of ILP problems.

14 ENUMERATIVE METHODS FOR INTEGER PROGRAMMING 1.1 Branch-and-Bound The first iterative method developed for solving ILPs is the Branch-and-Bound algorithm, which relies on a list of relaxed subproblems to be solved. Initially, the list is initialized to contain the continuous reformulation of the ILP. In a typical iteration, the subproblem in the list having the lowest lower bound is chosen, removed from the list, and solved by LP methods, yielding a solution x and a lower bounding objective function value z. If x is integral, the problem is fathomed and a new incumbent solution is found, and every subproblem in the list having lower bound greater than the incumbent objective function value is removed from the list and discarded as it cannot contain the optimum. Otherwise, the variable x i with fractional value x i x i nearest to 1 is chosen for branching, and two new subproblems are created: one with an added constraint x i x i and the other with x i x i ; both are assigned the lower bound z and inserted in the list. The algorithm terminates when the list of subproblems is empty. This algorithm is guaranteed to terminate in a finite number of steps..1.1 Example Initially, we let x = (,) e f =. Let L be the unsolved problem list, initially containing the continuous relaxation of the original IP N 1 : max x 1 + x x 1 + x 6x 1 + 8x 15 x 1,x Z +. The solution of the continuous relaxation is P (intersection of line (1) x 1 +x = with () 6x 1 +8x = 15), as depicted in the figure below. 15/8 / () f 1 Q P () /8 R 1 / 5/ () (1) We obtain an upper bound 1 f = 1 corresponding to a fractionary solution x = P = (, ). We choose the fractionary component x 1 as the branching variable, and form two subproblems N,N ; we add the constraint x 1 1 (labelled ()) to N and x 1 (labelled ()) to N. Finally, we push N and N on the problem list L. We choose N and remove it from L. The solution of the continuous relaxation of N is at the intersection Q of (1) and (): we obtain therefore x = Q = (1,1) and f = 5; since the solution is integral, we do not branch; further, since f > f we update x = x and f = f. 1 Notice we are solving a maximization problem, so lower bound is replaced by upper bound.

15 5 HEURISTICS AND BOUNDS 15 Now, we choose N and remove it from L. The solution of the continuous relaxation of N is at the intersection R of () and (): we obtain x = R = (, 8 ) and f = 1 8. Since the upper bound is 1/8, and 1 8 = 5, each integral solution found in the feasible region of N will never attain a better objective function value than f ; hence, again, we do not branch. Since the list L is now empty, the algorithm terminates with solution x = (1,1). The algorithmic process can be summarized by the following tree: N 1 x 1 1 x 1 N N. Branch-and-Cut Branch-and-Cut algorithms are a mixture of Branch-and-Bound and Cutting Plane algorithms. In practice, they follow a Branch-and-Bound iterative scheme where valid cuts are generated at each subproblem.. Branch-and-Price Branch-and-Price might be described as a mixture of Branch-and-Bound and Column Generation algorithms. Extremely large LP models are solved at each Branch-and-Bound iteration by column generation. The branching rules are sometimes modified to take into account certain numerical difficulties arising from the employment of the column generation algorithm to solve each subproblem. 5 Heuristics and Bounds When optimal methods take too long to solve a difficult ILP, we must resort to heuristic methods. These methods do not guarantee optimality, yet they strive to build a good solution in an acceptable amount of time. The trade-off between computational time spent searching for a solution and solution quality can often be controlled by fine-tuning the algorithmic parameters. As heuristic algorithms are not guaranteed to find the optimal solution, they are often constructed with a special eye to the implementation practicalities, so that they run fast. A posteriori analysis and modification of these algorithms is also specially useful. The heuristic construction process is often: (a) think of a possible heuristic, (b) implement it, (c) see how it performs on a certain number of test instances and record its failures, and (d) improve it, learning from experience. For example, a possible heuristic for the set covering problem () is to iteratively choose the region covering the largest number of towns. This type of heuristic is called greedy. Since the primary reason to employ a heuristic algorithm is that we are not able to solve the problem to optimality, estimating the heuristic solution quality is a hard challenge in itself. In certain cases (as with Christofides algorithm for the Symmetric Travelling Salesman Problem) we are able to prove that the heuristic solution is within a certain, pre-defined range from the optimal solution. The techniques for these proofs all belong to a field called Approximation Theory. As in Polyhedral Analysis, a different approach is called for each heuristic and problem. In the absence of approximation results, the only way to assess the heuristic solution quality is to find a lower bound to the optimal solution, and compare it with the heuristic results. The tighter the bound, the best our quality assessment will be.

16 5 HEURISTICS AND BOUNDS 16 Another reason why heuristics and lower bounds are important in ILP methods is that heuristics provide an upper bound to the solution. Thus, Branch-and-Bound convergence can be accelerated by finding good incumbents via heuristic methods. Tight lower bounding techniques also help speed up convergence considerably. 5.1 Lower Bounds: Lagrangian Relaxation and Subgradient Method The classic way to obtain a lower bound for an ILP is to solve its continuous relaxation, as has been said above. Other techniques exist, however, which make use of some problem structure to improve the bound. Consider a nearly decomposable problem as follows: min x,y c 1 x + c y A 1 x b 1 A y b Bx + Dy d x Z n1 +, y Z n + (15) where x,y are two sets of integer variables, c 1,c,b 1,b,d are vectors and A 1,A,B,D are matrices of appropriate sizes. Observe that were it not for the complicating constraints Bx + Dy d we might decompose problem (15) into two smaller independet subproblems and solve each separately (usually a much easier task). A tight lower bound for this kind of problem can be obtained by solving a Lagrangian relaxation of the problem. The complicating constraints become part of the objective function (each constraint weighted by a Lagrange multiplier): L(λ) = min x,y c 1 x + c y + λ(bx + Dy d) A 1 x b 1 A y b x Z n1 +, y Z n + (16) For each real-valued vector λ, L(λ) is a lower bound to the solution of the original problem. Since we want to find the tightest possible lower bound, we maximize L(λ) with respect to the Lagrange multipliers λ: max L(λ). (17) λ The lower bound obtained by solving (17) is guaranteed to be at least as tight as that obtained by solving the continuous relaxation of the original problem. Indeed, there exists a stronger result about the strength of the Lagrangian relaxation. For simplicity of notation, let us consider the following problem: min x c x Ax b x X, where X is a discrete but finite set X = {x 1,...,x }. Consider the lagrangian relaxation (18) max min λ x X c x + λ(ax b), (19) and let p be the solution of (19). By definition of X, (19) can be written as: max λ,u u u c x t + λ(ax t b) t T λ.

17 6 CONCLUSION 17 The above problem is linear, so its dual: min v T t=1 v t(c x t ) T t=1 v t(a i x t b i ) i m T t=1 v t = 1 v t t T (where A i is the i-the row of A) has the same optimal objective function value p. Notice now that T t=1 v tx t subject to T t=1 v t = 1 and v t for all t T is the convex hull of {x t t T }, and so the above dual program can be written as: min x c x Ax b x conv(x), where conv(x) is the convex hull of X. Thus, the bound given by the Lagrangian relaxation is at least as strong as that given by the continuous relaxation restricted to the convex hull of the original discrete domain. For problems with the integrality property, the Lagrangian relaxation is no tighter than the continuous relaxation. On the other hand, the above result also suggests that Lagrangian relaxations are usually very tight. Solving (17) is not straightforward, as L(λ) is a piecewise linear (but not differentiable) function. The most popular method for solving such problems is the subgradient method. Its description is very simple, although the implementation details are not. 1. Start with an initial vector λ.. If a pre-set termination condition is verified, exit.. Evaluate L(λ ) by solving the decomposable problem (16) with fixed λ, yielding a solution x,y.. Choose a vector d in the subgradient of L with respect to λ, and a step length t, and set λ = max{,λ + td}. 5. Update λ = λ and go back to step. Choosing appropriate subgradient vectors and step lengths at each iteration to accelerate convergence is a non-trivial task. 6 Conclusion In this essay we have presented some of the most well-known tools for the solution of ILP problems. The topics discussed in the essay range from modelling concepts to geometrical insights, to algorithmic descriptions. References [Fis99] M. Fischetti. Lezioni di Ricerca Operativa (in Italian). Edizioni Libreria Progetto, Padova, [KV] B. Korte and J. Vygen. Combinatorial Optimization, Theory and Algorithms. Springer-Verlag, Berlin,.

18 REFERENCES 18 [Mac] N. Maculan. Integer programming problems using a polynomial number of variables and constraints for combinatorial optimization problems in graphs. In N. Mladenović and Dj. Dugošija, editors, SYM-OP-IS Conference Proceedings, Herceg-Novi, pages 6, Beograd, September. Mathematical Institute, Academy of Sciences. [NW88] G.L. Nemhauser and L.A. Wolsey. Integer and Combinatorial Optimization. Wiley, New York, [PS98] [SD] C.H. Papadimitriou and K. Steiglitz. Combinatorial Optimization: Algorithms and Complexity. Dover, New York, H. Sherali and P. Driscoll. Evolution and state-of-the-art in integer programming. Journal of Computational and Applied Mathematics, 1:19,. [Wol98] L.A. Wolsey. Integer Programming. Wiley, New York, 1998.