Combinatorial Optimization

Transcription

1 Combinatorial Optimization MA4502 Michael Ritter This work is licensed under a Creative Commons Attribution- ShareAlike 4.0 International license. last updated July 14, 2015

2

3 Contents 1 Introduction and Prerequisites Recap: Linear & Integer Linear Programs Recap: Polyhedra, Dimension, Faces and Facets Combinatorial Optimization Problems Basic Complexity Theory Polyhedral Combinatorics Valid Inequalities Generating Valid Inequalities Inequalities for Stable Set The Matching Polytope Separation of Valid Inequalities The Ellipsoid Algorithm Separating Odd Cycle Inequalities for Stable Set Lifting: Strengthening Valid Inequalities Knapsack Cover and Extended Knapsack Cover Inequalities Lifting Cover Inequalities The General Lifting Procedure Approximation Algorithms Constant Factor Approximation Approximating Knapsack Approximating the Traveling Salesman Problem Approximation Schemes Review: A Dynamic Programming Algorithm for Knapsack An Approximation Scheme for Knapsack Column Generation and Decomposition Recap: Linear Programming Duality Column Generation for the Cutting Stock Problem Benders Decomposition Fixed Charge Transportation Problem General Procedure iii

4 Contents Benders Decomposition Applied to Fixed Charge Transportation Branch and Bound The General Framework Branch & Bound for Integer Linear Programming Branch & Bound for Special Ordered Sets of Type Bibliography 99 iv

5 Chapter 1 Introduction and Prerequisites For a lot of (hard) combinatorial problems, linear programming techniques are a great way of understanding, analyzing and solving the problems. We will hence frequently use these techniques. In fact, a large part of the lecture will be devoted to polyhedral combinatorics, where we try to gain some insight into the geometry of the polyhedra associated with certain combinatorial problems. Let us start with a brief review of linear programming terminology. 1.1 Recap: Linear & Integer Linear Programs Integer Linear Programs A linear and integer linear program, respectively, is usually specified by some constraint matrix A, an objective vector c and an optimization direction. Geometrically, the constraints can be interpreted as halfspaces and their intersection is a polyhedron (a polytope, if it is also bounded). max c T x Ax b x 0 x Z n P := { x R n : Ax b, x 0 } P (A,b) := { x R n : Ax b } I (P) := conv(p Z n ) 1

6 Chapter 1 Introduction and Prerequisites c x 2 P x I (P) x 1 Figure 1.1: A polyhedron and its associated integer hull. The linear program (without the integrality constraints) is often called relaxation of the ILP. While P is the polyhedron associated with this relaxation, the polyhedron I (P) is associated to the integer problem and called integer hull of P. See figure 1.1 for an illustration of these concepts. 1.2 Recap: Polyhedra, Dimension, Faces and Facets In this section, we will give a brief review of the most important notions related to polyhedra in combinatorial optimization. You should already know this material from courses such as Fundamentals of Convex Optimization or Discrete Optimization. If you feel you need to recap these materials in more detail, please refer to the literature. A comprehensive book on the topic is Gritzmann [Gri13] (in German), Ziegler [Zie95] or the classic Grünbaum [Grü03], a brief summary of the important facts can be found in Korte and Vygen [KV12]. Definition 1.1 (polyhedra and polytopes) A polyhedron is the intersection of finitely many halfspaces, i. e. a set of the form P := { x R n : Ax b } for some matrix AnR m n and some vector b R m. If A and b are rational, then P is called a rational polyhedron. In this case, we will usually assume that A and b are integer (by multiplication with a sufficiently large constant). A bounded polyhedron is called a polytope. 2

7 1.2 Recap: Polyhedra, Dimension, Faces and Facets Definition 1.2 For a nonempty set X R n, the dimension of X is the dimension of its affine hull, i. e. dim(x ) := dim aff(x ). A set X R n is called full-dimensional, if dim(x ) = n. Definition 1.3 (faces, vertices, edges, facets) Let P R n be a non-empty polyhedron. 1. Let H (a,α ) = { x R n : a T x = α } a hyperplane with normal vector a R n and α R and H (a,α ) := { x R n : a T x α } the associated halfspace. Then H (a,α ) is called a supporting hyperplane for P, if P H (a,α ) and P H (a,α ). 2. A subset of F P of P is called a face of P, if F = P or F = P H (a,α ) for some supporting hyperplane H (a,α ) of P. 3. A face of dimension 0 is called a vertex and a face of dimension 1 is called an edge of P. 4. A face of dimension dim(p) 1 is called a facet of P. examples: supporting hyperplane, vertex, facet Theorem 1.4 Let P = {x R n : Ax b} be a polyhedron and F P. The following are equivalent: 1. F is a face of P. 3

8 Chapter 1 Introduction and Prerequisites 2. There exists a vectorc R n such thatγ := max { c T x : x P } is finite and F = { x P : c T x = γ }. (We say F is induced/defined by the inequality c T x γ, or the inequality is face-defining for F.) 3. F = {x P : A x = b } for some subsystem A x b of Ax b. 1.3 Combinatorial Optimization Problems Discrete optimization problems can be represented in an abstract sense by incidence sets. The following definition is pretty general and not of much practical use, but we will state it to be able to define incidence polytopes, which provide a geometric abstraction of a discrete optimization problem. Gaining insight into this geometric representation is a key tool for many combinatorial problems and we will be concerned with this topic for quite some time.. Definition 1.5 Let S be a finite ground set, T 2 S a collection of feasible subsets and c : S R a valuation of the elements of S. We call (S, T ) a discrete incidence structure. The corresponding discrete optimization problem is given by min / max c(t ) T T Definition 1.6 (Incidence Polytope) Let (S, T ) be an incidence structure and let S = {s 1, s 2,..., s m } be some arbitrary, but fixed enumeration of S. For a feasible set T T, we define the incidence vector or characteristic vector 1 T {0, 1} m through (1 T ) i := 1, s i T 0, otherwise where i [m]. Depending on the set S, we may sometimes use the elements of S directly as vector indices (e. g. when S consists of arc or vertices of a graph). The incidence polytope corresponding to (S, T ) is defined as P (S, T ) := conv {1 T : T T }. The study of combinatorial problems via their incidence polytopes is at the very heart of polyhedral combinatorics, a topic which we will cover in more depth in the following lessons. Many of the advancements that were made in combinatorial optimization are due to that seemingly simple idea of studying the geometric objects related to a combinatorial problem and different relaxations. 4

9 1.3 Combinatorial Optimization Problems What makes the problem hard, of course, is the question of representation: While the incidence polytope is defined by its V-representation, we usually need an H -representation for common algorithms. Also, we do not usually have access to the V-representation in practice, because the feasible sets of the incidence system are not given explicitly, but just through some implicit description (often, T is of exponential size, so an explicit listing is not even desirable). The art of polyhedral combinatorics is therefore finding a good relaxation and then refining that relaxation as necessary through valid inequalities. Next, we will look at some common combinatorial problems and their incidence polytopes. Some of these problems are algorithmically easy (and may have been treated in Algorithmische Diskrete Mathematik ) and we will see whether that fact is reflected in the polytopal structure. Most of the problems we will be concerned with in this course, however, are hard. As we will frequently use the easy problems as building blocks or relaxations for the hard ones, we will start with those, nevertheless. Problem 1.7 (Maximum Network Flow) Input: a connected digraph G = (V, E), an edge capacity c : E R, distinct nodes s, t V Task: find a feasible s-t flow of maximum value Maximum Network Flow 1 3 s t 2 4 5

10 Chapter 1 Introduction and Prerequisites Problem 1.8 (Shortest Path Problem, SPP) Input: a connected (di-)graph G = (V, E), an edge valuation c : E R, distinct nodes s, t V Task: find a shortest path connecting s and t in G with respect to c Shortest Path 1 3 s t 2 4 6

11 1.3 Combinatorial Optimization Problems Problem 1.9 (Minimum Spanning Tree, MST) Input: a connected graph G = (V, E) and an edge valuation c : E R. Task: find a spanning tree T E of minimal total weight c(t ) Minimum Spanning Tree

12 Chapter 1 Introduction and Prerequisites Problem 1.10 ((Perfect) Matching) Input: a graph G = (V, E) Task: find a (perfect) matching in G Matching

13 1.3 Combinatorial Optimization Problems Problem 1.11 (Knapsack) Input: n N, valuations and weights v,w N n, a bound b N c : E R. Task: find a subset K [n] of maximum valuation v(k) such that w(k) b Knapsack

14 Chapter 1 Introduction and Prerequisites Problem 1.12 (Maximum Stable Set, STAB) Input: a graph G = (V, E), node weights c : V R 0 Task: find a stable set, i. e. a subset S V such that no two nodes in in S are connected by an edge, of maximum size/weight c(s) Maximum Stable Set

15 1.3 Combinatorial Optimization Problems Problem 1.13 (Traveling Salesman, TSP) Input: a connected graph G = (V, E) and an edge valuation c : E R Task: find a Hamilton circuit T in G of minimal total weight c(t ) or assert that G does not contain a Hamilton circuit Traveling Salesman

16 Chapter 1 Introduction and Prerequisites 1.4 Basic Complexity Theory In this section, we will give a brief and somewhat informal review of basic notions from complexity theory. We will elaborate on this, but just do enough for understanding encoding length, polynomial time algorithms, N P hardness and (in some later chapter) the notion of (fully) polynomial time approximation schemes. Definition 1.14 (encoding length) Let Σ be a finite set of cardinality at least 2 called the alphabet (often {0, 1} or {0, 1,..., 9}) and let Σ denote the set of all finite strings consisting of letters from Σ. The size or encoding length of a string x Σ is defined to be the number of letters in x. Definition 1.15 (decision problem) An instance of a decision problem is a subset Π Σ. To decide Π is the following problem: Input: x Σ Task: decide whether x Π A decision problem is the set of all instances of a decision problem. Example 1.17 (stable set problem) An instance of Stable Set consists of all words (a,b) of the following form: a encodes a graph G = (V, E) (e. g., in the form of an adjacency list) b encodes an integer k [0, V ] G contains a stable set of size at least k Hence the problem is, given a graph and an integer k, decide whether G contains a stable set of size at least k. If we encode the graph by an adjacency list, we need size(m) + size(n) + n + 2m (encode the integers m and n, the set V with n entries and one entry for each endnode of an edge for the adjacency list, yielding 2m entries). In addition, size(k) is required to encode the number k. Definition 1.19 (P and N P) The class P consists of all problems which can be decided in polynomial time, i. e. there exists an algorithm whose running time is bounded by some polynomial of the encoding length of the input. The class N P consists of all problems Π Σ such that there is a problem Π P and a polynomial p such that for each x Σ the following property holds: x Π if and only if there exists y Σ with size(y) p(size(x)) and (x,y) Π 12

17 1.4 Basic Complexity Theory In other words: For every YES instance of Π (i. e., x Π), there is a certificate y polynomially bounded in size that can be employed to prove that x is indeed a YES instance in polynomial time (through Π ). Example 1.20 The problem Stable Set is in N P. A YES instance has a stable set of size at least k, and we can simply use this stable set and an algorithm that, given the set and the graph, can check whether it is indeed stable and has cardinality at least k in polynomial time. Definition 1.21 (polynomial time reduction) A polynomial time reduction of a problem Π to a problem Π is a function f, that can be computed in polynomial time, with the property We say Π is (polynomial time) reducible to Π. x Π f (x) Π. If some problem Π can be reduced to a different problem Π, then Π is easier than Π : If we could solve Π, then the reduction f would readily yield an algorithm for solving Π: Take the instance, transform it and then simply use the algorithm for Π. The converse is not necessarily true, because we might not be able to transform an instance of Π into an instance of Π, so an algorithm for Π is not immediately helpful for solving Π. In that sense, Π is easier than Π. Definition 1.22 A decision problem is called N P hard, if every problem in N P can be reduced to it (i. e., if we could solve it, then we could solve every N P problem). It is called N P complete, if it is N P hard and contained in N P. The N P complete problem include the decision versions of Stable Set, Traveling Salesman, Knapsack, Bin Packing, Integer Linear Programming and many others. 13

18

19 Chapter 2 Polyhedral Combinatorics In polyhedral combinatorics, our goal is to get some insight into the geometry of the polyhedra related to combinatorial problems. Ultimately, we would like to fully characterize that geometry, i. e. we would like to know which inequalities define facets of the polyhedron. If there a too many (i. e. exponentially many) facets, we would also like to have a way of finding valid inequalities (facets, if possible) that are violated by some given point outside of the polyhedron, i. e. we would like to be able to solve the separations problem efficiently. This chapter will deal with with these problems and introduce important concepts and techniques using some example problems. In addition, we will deal with the task of strenghtening valid inequalities if we cannot get a facet, can we at least obtain something as strong as possible given the limited knowledge that we have? Throughout this whole chapter, when talking about graphs and graph-related problems, n will generally be used to denote the number of vertices, while m will denote the number of edges/arcs (unless explicitly specified otherwise). 2.1 Valid Inequalities Definition 2.1 (valid inequalities) Let S R n and let w R n, ω R define the inequality w T x ω. We say w T x ω is a valid inequality for S if S { x R n : w T x ω }. In particular, for a polyhedron P an inequality w T x ω is valid for P if and only if w T x ω holds for all x P. The set F := P { x R n : w T x = ω } is then a face of P, called the face (vertex/edge/facet) induced/defined by w T x ω. Ideally, we would like to find facet-defining inequalities for the polyhedra we are concerned with. However, in many cases we will have to settle for less than that. 15

20 Chapter 2 Polyhedral Combinatorics x 2 2x 1 + x 2 3 x 1 5/2 F P x 1 Figure 2.1: Valid inequalities for the polytope P. Example We consider the polyhedron P = P (A, b) defined by Figure 2.1 shows some valid inequalities for P A := , b := Generating Valid Inequalities A general method for generating valid inequalities is the idea of rounding cuts. Most of the inequalities we will encounter are special cases of rounding cuts, so we will briefly review the idea here. Definition 2.2 (integer hull, rounding cut) Let P = P (A,b) be a rational polyhedron, i. e. A Z m n. Then I (P) := conv(p Z n ) denotes the integer hull of P (sometimes also denoted as P I ). 16

21 2.2 Generating Valid Inequalities Let u R m 0 and let a 1,..., a n denote the columns of A, then the inequality n u T a j x u T b is called the rounding cut with respect to the cut vector u. j=1 Example of a Rounding Cut Example 2.3 Consider the polytope P (A, b) defined by A := , b := and the cut vector u = (0, 1/2, 0, 0, 1/2, 0) T. 17

22 Chapter 2 Polyhedral Combinatorics x 2 I (P) P x 1 Figure 2.2: Illustration of example 2.3 Theorem 2.5 Let P = (A,b) R n 0 be a rational polyhedron and u Rm 0. Then the rounding cut with respect to u is a valid inequality for I (P). Proof 18

23 2.3 Inequalities for Stable Set Definition 2.6 Let P = P (A,b) R n 0 with A Zm n. Then n P := x P : u T a j x u T b for all u R m 0 j=1 is called the first Chvátal-Gomory closure of P. The k-th Chvátal-Gomory closure of P is defined recursively as P (k) := (P (k 1)) Remark 2.7 It can be shown that the Chvátal-Gomory closure can be obtained by applying finitely many cuts, thus P is again a polyhedron. Furthermore, there is some finite k such that P (k ) = I (P), meaning that a finite number of rounds of applying finitely many cuts will eventually yield the integer hull of any rational polyhedron. The smallest such number k is called the Chvátal rank of P. This theory is one of the most prominent subjects of the course Discrete Optimization. 2.3 Inequalities for Stable Set As a first example we will consider the stable set polytope and devise some facts and a valid inequality. The main purpose of this section is demonstrate some proof techniques that we will subsequently apply to other polyhedra. Definition 2.8 (stable set polytope) The incidence polytope of the stable set problem for a graph G = (V, E) is called stable set polytope P STAB (G). Thus, the stable set polytope is the convex hull of incidence vectors of stable sets. We usually just write P STAB if the corresponding graph is clear from the context. Using the ILP formulation from problem 1.12 and denoting the number of nodes of the graph G = (V, E) by n, we get the following relaxation of P STAB R n : x i + x j 1 x 1 x 0 {i, j} E Lemma 2.9 The stable set polytope is full-dimensional, i. e. dim(p STAB ) = n. Remark 2.10 (dimension proof technique) The dimension of a polyhedron is usually shown by exhibiting a large enough number (dim +1) of affinely independent points that are contained in the polyhedron and by bounding the dimension from above. 19

24 Chapter 2 Polyhedral Combinatorics Proof 20

25 2.3 Inequalities for Stable Set Theorem 2.11 (Clique Inequalities) Let G = (V, E) be a graph and C V a clique in G (i. e., there is an edge in G between any two nodes in C). The following inequality is called the clique inequality for the clique C: x (C) = x v 1 1. Each clique inequality is a valid inequality for the stable set polytope P STAB (G). v C 2. The clique inequality for a clique C of G defines a facet of the stable set polytope P STAB (G) if any only if C is a maximal clique (with respect to set inclusion) in G. Remark 2.12 (facet proof technique) To show that some inequality a T x α defines a facet F = { x P : a T x = α } of P you can often use the following approach: 1. Show that a T x α is a valid inequality. 2. Find dim(p) affinely independent points in P that are also contained in F. Proof 21

26 Chapter 2 Polyhedral Combinatorics Theorem 2.13 (odd cycle inequalities) Let K V be an odd cycle in the graph G = (V, E). Then the following odd cycle inequality is valid for the stable set polytope P STAB : v K x v K 1 2 Odd cycle inequalities have an appealing combinatorial interpretation: Along a cycle, at most every second node may be chosen for a stable set, thus at most half of the nodes can be in the stable set. If the cycle has odd length, then this number is fractional and we have to round it down (i. e. leave a hole of one node). Proof see tutorials 22

27 2.4 The Matching Polytope 2.4 The Matching Polytope In this section, we will introduce the perfect matching polytope and the matching polytope that represent the set of all (perfect) matchings in a graph. The matching polytope is a good example of a polytope where a complete characterization of the facets is known. In fact, it was a breakthrough in polyhedral combinatorics, because it was the first complete characterization of a combinatorial polytope that does not simply follow through total unimodularity. While the proof was originally derived from an algorithmics approach to the matching problem (around 1965), we will give a different proof here. Our approach starts with the perfect matching polytope and subsequently generalizes the characterization to the matching polytope. Definition 2.15 ((perfect) matching polytope) For a graph G = (V, E), the (perfect) matching polytope, denoted by P P M (G) and P M (G), respectively, is the incidence polytope of the set of (perfect) matchings on G. Our goal is to prove the following characterization due to Jack Edmonds, one of the theorems that started the field of polyhedral combinatorics. Theorem 2.16 (Edmonds Theorem for the Matching Polytope) The matching polytope P M (G) on a graph G = (V, E) is equal to the set of feasible points described by the following inequalities: x e 1 for all v V (degree constraints) (2.1) e δ (v ) e E(U ) x e U 1 2 for all U V of odd cardinality (odd set constraints) (2.2) x e 0 for all e E (nonnegativity constraints) (2.3) To achieve this goal, we will take two intermediary steps: We will first show that the fractional matching polytope without the odd set inequalities is already half-integral, i. e. all its vertices have coordinates in {0, 1/2, 1}. We will then proceed to prove equality of the perfect matching polytope and its fractional counterpart. In the last step, we will extend this to prove the characterization of the matching polytope. Lemma 2.17 Let G = (V, E) be a graph and let P (G) be the polytope defined by the following inequalities: x e = 1 for all v V (degree constraints) (2.4) e δ (v ) x e 0 for all e E (nonnegativity constraints) (2.5) 23

28 Chapter 2 Polyhedral Combinatorics Then a feasible point x P (G) is a vertex of P (G) if and only if x e {0, 1/2, 1} for all e E and the set {e E : x e = 1/2} consists of node disjoint odd cycles. Proof. We will first prove that any solution satisfying the stated requirements is indeed a vertex of P (G). Let x P (G) be a feasible point such that x {0, 1/2, 1} m and such that the set { e E : x e = 1/2 } consists of node disjoint odd cycles. We define an inequality w T x 0 by setting w e := 1, if x e = 0, 0, if x e > 0. As w T x 0 for all x 0, the inequality is obviously valid for P (G). We consider the face F := { x P (G) : w T x = 0 }. By definition of w, we know x F, and we will show that x is in fact the only point contained in F, thereby proving that x is a vertex of P (G). To see this, let x F be an arbitrary point in F. Then 0 = W T x = x e 0, e E:x e =0 as x 0. Thus x e = 0 for all edges e where x e = 0. Let us consider an edge e E where x e = 1/2. By assumption, this edge is contained in an odd cycle C of edges of the same type, and none of the nodes in this cycle can be connected to an edge with x e = 1. By the degree constraints, the value of x along that cycle must therefore alternate between two values ξ [0, 1] and 1 ξ. As the cycle is odd, all of these values must in fact be equal, thus the only possible choice is x e = 1/2 along C. This shows that x = x and therefore F = {x }. To show the converse, assume that x P (G) is a vertex of P (G). As P (G) is a rational polytope, there is some objective vector c R m such that x is the unique solution of the optimization problem max { c T x : x P (G) }. We construct a bipartite graph H = (V H, E H ) from G = (V, E) as follows: 1. replace each node v V with two nodes v,v V H 2. replace each edge {u,v} by two edges {u,v } and {u,v } 3. extend the weight function c : E Z by setting c H (u,v ) := c H (u,v ) := c(u,v) 4. map each feasible point x P (G) to a feasible point x H P (H ) by setting x H (u,v ) := x H (u,v ) := x (u,v) Observe that for each x P (G) the corresponding x H P (H ) has exactly twice the objective value: c T H x H = 2 c T x. Also, for each feasible x H P (G) we can define a feasible point x G by setting x G (u,v) := 1/2(x H (u,v ) + x H (u,v )), and then c T x G = 1/2x T H x H. 24

29 2.4 The Matching Polytope /2 1/ / Figure 2.3: Illustration of the transformation G H in the proof of lemma Consider the problem max { c T H x H : x H P (H ) }. As H is a bipartite graph, its(node-edge) incidence matrix is totally unimodular, thus the polytope P (H ) is integer and the problem has an optimal integral solution xh. Using the above transformation we define x G as x G (u,v) := 1/2(x H (u,v ) + xh (u,v )), then xg is an optimal solution to max { c T x : x P } and it is halfintegral. As c was chosen so that x would be the unique optimal solution, we can conclude x = xg, thus x is half-integral. Now consider a half-integral vertex x P (G) and set {e E : x e = 1/2}. This set must be composed of node disjoint cycles, because every node that meets edges with x e = 1/2 must meet exactly two of those. Suppose the set contained an even cycle. Then we could obtain a new feasible solution by substracting some 1/2 > ε > 0 form every other edge and adding the same ε to the remaining edges. This way, we could obtain two distinct feasible solutions whose convex hull would contain x, contradicting the assumption that x is a vertex (and thus an extreme point). Hence the set is composed only of odd cycles. We are now ready to prove Edmonds Theorem for the perfect matching polytope: Theorem 2.18 (Edmonds Theorem for the Perfect Matching Polytope, cf. [Sch03]) The perfect matching polytope P P M (G) on a graph G = (V, E) is equal to the set of feasible points described by the following inequalities: x e = 1 for all v V (degree constraints) (2.6) e δ (v) e δ (U ) x e 1 for all U V of odd cardinality (odd set constraints) (2.7) x e 0 for all e E (nonnegativity constraints) (2.8) Proof. Let x P P M be a vertex of P P M, in particular x {0, 1} m. As x is the incidence vector of a perfect matching, equations (2.6) and (2.8) are satisfied by definition. Also, for any set U V 25

30 Chapter 2 Polyhedral Combinatorics of odd cardinality, the inequality equation (2.7) is satisfied at x, because otherwise one of the nodes in U could not be contained in a matching edge, contradicting the fact that x represents a perfect matching. This shows that P P M is a subset of the feasible set describes by equations (2.6) to (2.8). Suppose that there is a point x satisfying the above inequalities equations (2.6) to (2.8) that is not contained in P P M. Among all graphs that allow for such a counterexample, choose a graph that minimizes m + n. We may assume that x is a vertex of the polytope described by equations (2.6) to (2.8) (if there was no vertex contradicting the theorem, then there could be no other points by convexity). Let us start by excluding some easy cases: By minimality of our counterexample we know that 0 < x e < 1 for all e E: If x e = 0, we could simply remove the edge e and end up with a smaller counterexample, if x e = 1 we could remove e and both of its end nodes and again get a smaller counterexample. By the degree constraints, we then know that every vertex of G has at least degree 2, therefore m n. If equality holds, each node has precisely degree 2, meaning that G is made up of odd and even circuits. By the odd set constraints, odd circuits are not possible, so the graph only has even circuits. By the degree constraints, the values of x e alternate between at most two different values in (0, 1) within all even circuits. In this case, x is clearly a convex combination of two perfect matchings, thus it is contained in P P M. We know that m > n and also that n is even (otherwise, the odd set constraint for U = V could not be satisfied). As we have chosen x as a vertex there are m linearly independent constraints among equations (2.6) to (2.8) that are satisfied with equality at x. We already know that none of the nonnegativity constraints hold at equality, and we have n degree constraints (that are always satisfied at equality), so as m > n there must be an odd subset U V with 3 U n 3 whose odd set inequality x (δ (U )) = 1 holds at equality. Define U := V \ U, then this set is also of odd cardinality (otherwise, G could not contain a perfect matching, as the total number of nodes was odd). We consider two graphs that will be obtained from G by shrinking U and U, respectively: G U is obtained from G by shrinking U into a single supernode U, deleting all edges within U and replacing all edges {v,u} δ (U ) with u U by new edges {v,u }, keeping all parallel edges. G U is obtained from G in the same way by shrinking U into a single supernode U. Let x U and x U denote the projections of x onto these contracted graphs G U. and G U, respectively. By construction, we know x U (δ (U )) = x U (δ (U )) = 1 and x U (δ (U )) = x U (δ (U )) = 1, so x U and x U satisfy equations (2.6) to (2.8) with respect to G U and G U, respectively. As the new graphs are strictly smaller in the number of nodes and edges than G, they cannot be counterexamples to our theorem, and thus x U P P M (G U ) and x U P P M (G U ). 26

31 2.4 The Matching Polytope This means we can write both x U and x U as convex combinations of perfect matchings of G U and G U : x U = 1 k x U = 1 k k 1 M U, i i=1 k 1 M U. i i=1 We have used two assumptions in this representation: We can use the same number of matchings, namely k, for both convex combinations. This may be done by repeating matchings if necessary. We have assumed a special form where all coefficients are the same (1/k). This can be done as x is known to be a rational point (because it as a vertex of a rational polytope), and thus x U and x U are also rational. We can therefore rewrite any convex combination using a common denominator and repeating summands as necessary. Now consider an edge e δ (U ). Then the number of indices i {1,..., k} with e Mi U is equal to kxe U = kx e = kxe U. This is equal to the number of indices i with e Mi U. Thus, we can order the Mi U, Mi U such that Mi U and Mi U share an edge in δ (U ) (any one of these matchings has exactly one edge from δ (U )). This means that we can take the union M i := Mi U Mi U to get a perfect matching of our original graph G. This yields x = 1 k 1 Mi, k and thus x belongs to the perfect matching polytope P P M (G) of G. i=1 Now we are ready to prove the main result of this section. Let us state it once more: Theorem 2.19 (Edmonds Theorem for the Matching Polytope) The matching polytope P M (G) on a graph G = (V, E) is equal to the set of feasible points described by the following inequalities: x e 1 for all v V (degree constraints) (2.9) e δ (v ) e E (U ) x e U 1 2 for all U V of odd cardinality (2.10) x e 0 for all e E (nonnegativity constraints) (2.11) 27

32 Chapter 2 Polyhedral Combinatorics 1 U supernode U supernode U 28 Figure 2.4: Illustration of the shrinking steps in the proof of theorem 2.18.

33 2.4 The Matching Polytope The idea of the proof is this: 1. duplicate the graph G 2. add edges between each node and its corresponding copy 3. define a weight function y on the edges that replicates x on the two copies of the original graph 4. define y on the new edges such that y is contained in the perfect matching polytope 5. use Edmonds theorem for the perfect matching polytope and apply the result to the original subgraph Proof. Let x be a vertex of the matching polytope, then x is an incidence vector of a matching in G. In particular, the degree constraints and the nonnegativity constraints are satisfied. Consider some odd subset U V, then the following holds: e E (U ) x e v U e δ (v ) vinu = U 2 The odd set inequality follows from the fact that the left hand side is integer and rounding down. Now let x satisfy the inequalities from the theorem. Define a graph H = (V H, E H ) by duplicating G in the following way: Define y [0, 1] E H as follows: x e V H := {(v, 1) : v V } {(v, 2) : v V } E H := {{(v, 1), (w, 1)} : {v,w} E} {{(v, 2), (w, 2)} : {v,w} E} {{(v, 1), (v, 2)} : v V } y {(v,1), (w,1) } := x {v,w } y {(v,2), (w,2) } := x {v,w } y {(v,1), (v,2) } := 1 e δ (v) x e for {v,w} E for {v,w} E for v V We will show that y belongs to the perfect matching polytope P P M (H ) using theorem In fact, by the degree constraints and the nonnegativity constraints on x we know thaty [0, 1] E H 29

34 Chapter 2 Polyhedral Combinatorics and by our definition of y we also have e δ H (v ) y e = 1 for all v V H. It remains to show the odd set inequalities, so consider some odd subset U V H. We define the following sets: A := {(v V : (v, 1) U, (v, 2) U )} B := {(v V : (v, 1) U, (v, 2) U )} C := {(v V : (v, 1) U, (v, 2) U )} As U = A + 2 B + C, precisely one of either A or C must have odd cardinality. Let us w. l. o. g. assume that A is odd (the proof is analogous for the other case). We denote by A 1, A 2 and B 1, B 2 the partitions of A and B into nodes of the form (v, 1) and (v, 2), respectively. Then the following holds: y e y e 2 y e y e + y e e δ H (U ) v A 1 e δ H (v ) e E H (A 1 ) e δ H A 1,B 1 e δ h (B 2,A 2 ) = y e 2 v A 1 e δ H (v ) = 1 2 v A 1 e E G (A) 1 ( A 1) v A 1 =1 e E G (A) x e x e (degree constraints) (odd set constraints) This shows that y is contained in the perfect matching polytope and thus it is a convex combination of perfect matchings. If we constrain these perfect matchings to the subgraph of H that is isomorphic to the original graph G, we get matchings of G such that x is a convex combination of these matchings. This proves the theorem. 2.5 Separation of Valid Inequalities Up to this point, our focus was on finding and characterizing classes of valid inequalities for different combinatorial optimization problems. In some cases, these inequalities easily yield a better description (a tighter relaxation) of some incidence polytope and can simply be added to the relaxation. In other cases, the classes we obtained were exponentially large (think of the odd cycle inequalities for Stable Set or the subtour elimination constraints for Traveling Salesman) and thus just adding them all is not an option for problems of practically relevant size. To overcome this obstacle, note that we might not need all of these inequalities: In an optimal vertex of some polytope P, only dim(p) many inequalities are active and uniquely determine the vertex. Of course, if we knew which inequalities were active in an optimal solution the 30

35 2.5 Separation of Valid Inequalities 4 1/4 3 1/2 1/4 (4, 1) 1 2 1/4 3/4 (4, 2) 1/4 (3, 1) 1/2 1/4 0 1/2 1/4 (3, 2) 1/2 1/4 (1, 1) (2, 1) 1/4 (1, 2) (2, 2) 1/4 1/4 Figure 2.5: Illustration of the graph duplication in the proof to theorem

36 Chapter 2 Polyhedral Combinatorics U A A 1 A 2 B B 1 B 2 C {(v, 1) : v V } {(v, 2) : v V } Figure 2.6: Illustration of the A/B/C partition of the duplicated graph in the proof to theorem

37 2.5 Separation of Valid Inequalities whole task of optimization could be reduced to solving one linear equation system, so thing will usually not be that easy. However, there is some hope that we will not need to add all valid inequalities of a large class, but that a limited number might do. To be a little more precise, we would like to use a procedure such as this: 1. Start with some relaxation of the original problem. 2. Solve the current relaxation to obtain an optimum x. 3. Find some valid inequality that is not part of the current relaxation and that is violated by x or confirm that x is a feasible optimal point of the original (unrelaxed) problem. 4. If a new valid inequality has been found, add it to the current relaxation and start over The Ellipsoid Algorithm We already know from the tutorials (see xercise 1.3 on sheet 1) that the ability to find a feasible point for a polyhedron is sufficient to determine an optimal solution. Thus we need to solve the following problem repeatedly: Definition 2.20 (separation problem) The following task is called the separation problem: Input: A polytope P and a point z. Task: Either assert that z P or find a valid inequality a T x αfor P that cuts off z, i. e. a T z > α. An algorithm that solves the separation problem for some given combinatorial optimization problem is called a separation oracle. The hope is that we do not have to add a very large number of inequalities until we obtain a feasible solution. A famous result by Khachiyan (1979) in fact justifies this approach: Theorem 2.21 (Khachiyan) If there is a separation oracle that can be implemented in polynomial running time for some feasibility problem then the feasibility problem can be solved in polynomial time. The proof relies on the ellipsoid algorithm that provided a major theoretical advance in the 70 s. We will not provide a proof of the theorem here, as that would require a lot of technical details, but we will give a brief overview of the main ideas of the ellipsoid algorithm. 33

38 Chapter 2 Polyhedral Combinatorics Figure 2.7: An ellipsoid E(D, z). Definition 2.22 (ellipsoid) Let D R n n be a symmetric, positive definite matrix and let z R n. The set E(D, z) := { x R n : (x z) T D 1 (x z) 1 } is called the ellipsoid described by D, centered in z. Algorithm 2.23 (ellipsoid algorithm) The following algorithm uses a separation oracle to find a feasible point in a polytope P. 1. start with a sufficiently large ellipsoid E 0 centered at z 0 := 0 such that P E Repeat the following steps starting at i = 0: a) Use the separation oracle to decide whether z i P. b) If the answer is yes, stop with output z i. c) If the answer is no, obtain a valid inequality (a i ) T x b i for P which is violated by z i, i. e. (a i ) T z i > b i. d) Define E i+1 to be the minimum volume ellipsoid that contains E i { x : (a i ) T x (a i ) T z i } and let z i+1 be the center of that ellipsoid. As (a i ) T x b i < (a i ) T z i we know that P E i { x : (a i ) T x (a i ) T z i }, thus P E i+1. It can be shown that the minimum volume ellipsoid and its center can be computed explicitly once the separating inequality is known. Furthermore, the decrease in volume of subsequent ellipsoids is bounded by a fixed factor strictly less than one. Therefore, the algorithm will terminate before 34

39 2.5 Separation of Valid Inequalities E i P z i+1 z i Figure 2.8: One step of the ellipsoid algorithm. the volume of the ellipsoids drops below the volume of P and a point in P will be found. The proof of Khachiyan s theorem is heavily based on the following fact: Lemma 2.24 (minimum volume ellipsoid) The minimum volume ellipsoid containing E(D, z) { x R n : a T x a T z } is given by E(D, z ) where ( D := n2 n 2 D 2 ) 1 n + 1 DaaT D a T, Da z := z 1 n + 1 Da a T Da. Furthermore, we have vol(e(d, z )) vol(e(d, z)) e 1 2n+2 < 1 This result inspires our quest for polynomial time separation algorithms. Alas, for most problems we will not be able to separate over all facet defining inequalities in polynomial time (in fact, if we can do this for a problem, then it is contained in P). But in many cases we will at least be able to separate over some class of valid inequalities (or even facets), and with some luck that might already be enough or at least get us way closer towards a solution for our problem. 35

40 Chapter 2 Polyhedral Combinatorics Separating Odd Cycle Inequalities for Stable Set Recall the following ILP formulation for Stable Set and its relaxation: x i + x j 1 {i, j} E x i + x j 1 {i, j} E x {0, 1} n 0 x 1 We have already shown that the odd cycle inequalities v K x v K 1 2 are valid for the stable set polyhedron P STAB (cf. theorem 2.13). Of course, there may be an exponential number of odd cycles in the graph, so we would like to have a method to separate violated inequalities for some given solution x [0, 1] n of the relaxation. The other classes of inequalities in the relaxation are polynomial in number, so we will just separate the odd cycle inequalities. We start by reformulating the separation problem into an optimization problem: Problem 2.25 (separation of odd cycle inequalities for Stable Set) Input: An instance of Stable Set, a feasible solution x of the LP relaxation Task: Find an odd cycle K V such that the odd cycle inequality corresponding to K is violated, i. e. xv > K 1 2 or assert that no such cycle exists. v K It is usually convenient to state the separation problem as an optimization problem with the left hand side of the inequality as the objective function. Then, we simply maximize the left hand side (over all odd cycles in the graph) and compare the objective value with the right hand side to decide whether the solution of our optimization problem yields a violated inequality. If it does not, then we immediately get a proof that none of the odd cycle inequalities can be violated, because there is no odd cycle with a larger left hand side. The following lemma provides a convenient alternative formulation for the objective function at hand, using edge weights instead of node weights. (Try converting between edge and node weights if you do not see a different way to tackle the problem sometimes thing are easier to see that way.) 36

41 2.5 Separation of Valid Inequalities Lemma 2.26 Let G = (V, E) be a graph on n nodes and let x [0, 1] n be a feasible solution to the LP relaxation of Stable Set. Define y ij := 1 xi x j, then the odd cycle inequality for an odd cycle K V is violated if and only if y ij < 1 (where E(K) E denotes the set of cycle edges). {i, j } E (K ) Proof 37

42 Chapter 2 Polyhedral Combinatorics Figure 2.9: The graph transformation for the odd cycle separation routine. Thus, if we consider {i, j } E (K ) y ij the weight of the cycle K, then we need to find an odd cycle with minimum weight. We will do this by translating the task of identifying minimum weight cycles into a problem of finding shortest paths by transforming the original graph into a bipartite graph (a technique which you should be pretty much familiar with by now): 1. Start with the original graph G = (V, E) and construct a new bipartite graph G = (V + V, E ) by doubling the node set and replacing each edge with two edges connecting the nodes in different set of the node set bipartition. 2. Set V + := {i + : i V } and V := {i : i V }. 3. Set E := {i, j } E ({i +, j } {j +, i }). 4. Define a weight function y : E R by setting y (i +, j ) := y (j +, i ) := y ij. Notice that an even cycle in the original graph G corresponds to a cycle starting at some node i + and ending at the same node i + in G, while an odd cycle corresponds to a path starting at some node i +, but ending at the corresponding node i. Thus we obtain the following separation routine: 1. Solve the LP relaxation of Stable Set and define the weights y and the bipartite auxiliary graph G as outlined above. 2. For each i V, compute the shortest path from i + to i in G. This path corresponds to an odd cycle of the same weight in the orginal graph G. 38

43 2.5 Separation of Valid Inequalities 3. If the weight of this cycle is strictly less than 1, add the violated inequality to the relaxation and start over. 4. If none of the cycles has a weight strictly less than 1, then no violated odd cycle inequalities exist. Notice that this separation routine can be implemented in polynomial time: The graph G itself is clearly polynomial in the size of the original instance, iterating over all nodes is polynomial and there are polynomial algorithms for computing a shortest path (like the algorithms of Floyd-Warshall or Bellman-Ford). Also note that we only need to solve the LP relaxation (which can be done in polynomial time), not the integer linear program in each step. (In fact, this is very important for theoretical purposes: If we needed to spend exponential time in each step of our separation routine, the whole point of getting around the exponential number of inequalities is somehow void. However, in practice such a separation routine might still be very useful, if the number of times it is called is far less than the number of inequalities that are separated.) Question So, can we use that separation routine and the ellipsoid algorithm to obtain a polynomial time algorithm for Stable Set? Why? 39

44 Chapter 2 Polyhedral Combinatorics Remark 2.27 There is a polynomial time separation algorithm for the odd set constraints of the matching polyope as well. As by Edmond s theorem we know that these inequalities actually suffice to characterize the matching polytope, this separation routine yields a polynomial time algorithm for the matching problem. Many other polynomial time separation algorithms are known, but usually the only yield some of the facets of a combinatorial incidence polyhedron. Also, it may be a good idea to use separation heuristics instead of exact algorithms in practice if they perform a lot better. The exact approach can then either be used as a fallback or one simply is satisfied with separating just a few of the valid inequalities. 2.6 Lifting: Strengthening Valid Inequalities So far, we have devised a few methods to generate valid inequalities for combinatorial optimization problems. Some of those are already facets and hence as good as they can be in a certain sense. However, for those inequalities which are not facets, we can try to improve upon the inequality even further. In this section, we will explore the lifting procedure which does just that. One note of warning: Do not confuse this procedure with the method of lift and project while this is also a method that may be employed for combinatorial optimization problems, it is completely different otherwise! As an example we will use the knapsack problem and introduce a new class of valid inequalities for this problem, called the knapsack cover inequalities Knapsack Cover and Extended Knapsack Cover Inequalities In this section, we will consistently use the following notation for a knapsack problem on n N items: max c T x a T x β x {0, 1} n Here, c N n is the values vector, a N n are the weights and β N denotes the maximum total weight. Definition 2.28 (Knapsack Cover) A set C [n] is called a cover for the knapsack problem, if a(c) = i C a i > β. A cover is minimal, if no proper subset of C is also a cover. 40

45 2.6 Lifting: Strengthening Valid Inequalities Theorem 2.29 (Knapsack Cover Inequalities) Let C [n] be a cover for the knapsack problem. Then the following knapsack cover inequality is valid for the knapsack polytope: x (C) = x i C 1 i C Proof 41

46 Chapter 2 Polyhedral Combinatorics If the knapsack cover inequality is not itself a facet, it may be possible to add variables to the inequality without loosing validity. As a first idea, we consider extended knapsack cover. Theorem 2.30 Let C [n] be cover for the knapsack problem and define the extended cover E(C) := Then the extended knapsack cover inequality x (E(C)) = is valid for the knapsack polytope. { } i [n] : a i max a j C. j C i E(C) x i C 1 Proof 42

47 2.6 Lifting: Strengthening Valid Inequalities Lifting Cover Inequalities The extended knapsack cover inequalities are based on a very simple idea: Find variable that does not yet occur in some valid inequality and see whether it may be added without loosing validity. We will now investigate this idea a little more systematically, starting with an example. Example 2.31 (Lifted Knapsack Cover) Consider the knapsack polytope defined by the weight vector a = (11, 6, 6, 5, 5, 4, 1) T and the weight limit β = 19, i. e. P KNAP (a, β) = conv { x {0, 1} 7 : 11x 1 + 6x 2 + 6x 3 + 5x 4 + 5x 5 + 4x 6 + x 7 19 }. The set C := {3, 4, 5, 6} has aggregated weight a(c) = = 20, thus C is a (minimal) cover and the inequality x 3 + x 4 + x 5 + x 6 3 is valid for P KNAP (a, β). However, the inequality does not contain the variable x 1 at all let us try to find out whether it is possible to include it. Specifically, We would like to find the maximum possible value of w 1 such that w 1 x 1 + x 3 + x 4 + x 5 + x 6 3 is still a valid inequality for P KNAP (a, β). Let x P KNAP, then we consider two cases: x 1 x 1 = 0: In this case, our inequality is equivalent to the cover inequality and thus valid by our previous deliberations. = 1: In this case, the inequality w 1 x 1 + x 3 + x 4 + x 5 + x 6 3 is valid for any w! where w 1 3 max {x 3 + x 4 + x 5 + x 6 : x P KNAP and x 1 = 1}. Specifically, we need to solve the optimization problem max x 3 + x 4 + x 5 + x 6 6x 2 + 6x 3 + 5x 4 + 5x 5 + 4x 6 + x = 8 x {0, 1} 7 It can be seen easily that the optimal objective value for this problem is 1, thus w = 2. The new lifted inequality is then 2x 1 + x 3 + x 4 + x 5 + x

48 Chapter 2 Polyhedral Combinatorics example continued Let us try to lift the remaining variables in the same way, starting with x 2. 44

49 2.6 Lifting: Strengthening Valid Inequalities The General Lifting Procedure This leads us to the general lifting procedure. We will only formulate the theorem for the first variable for ease of notation, other variables should then be straightforward. Theorem 2.32 (general lifting procedure for x 1 ) Let F {0, 1} n be a feasible set for some n N and let n i=2 a i x i a 0 be a valid inequality of F 0. Consider the sets then the following holds: 1. If F 1 =, then x 1 0 is valid for F. F 0 := F { x {0, 1} n : x 1 = 0 } and F 1 := F { x {0, 1} n : x 1 = 1 }, 2. If F 1, then a 1 x 1 + n i=2 a i x i a 0 is valid for F for any a 1 a 0 z, where z := max n a i x i : x F 1. i=2 3. Suppose n i=2 a i x i a 0 defines a face of dimension at least k of conv(f 0 ). If F 1 and a 1 = a 0 z with z as above, then the inequality a 1 x 1 + n i=2 a i x i a 0 defines a face of conv(f ) of dimension at least k + 1. Proof 45