Investigating Mixed-Integer Hulls using a MIP-Solver

Investigating Mixed-Integer Hulls using a MIP-Solver Matthias Walter Otto-von-Guericke Universität Magdeburg Joint work with Volker Kaibel (OvGU) Aussois Combinatorial Optimization Workshop 2015

Outline 1 Polyhedral Combinatorics 1 Introduction 2 Usual Approach 3 Limitations 2 Vision 3 Facets 1 Polarity 2 Target Cuts 4 Affine Hull 5 Minimizing the 1-Norm of Basis Vectors 1 Problem 2 2 Vectors: Exact Approach 3 2 Vectors: Heuristic Approach Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 2 / 22

Setup Problems in Question We consider mixed-integer programs with rational data: max c, x s.t. Ax b x i Z i I [n] Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 3 / 22

Setup Problems in Question We consider mixed-integer programs with rational data: max c, x s.t. Ax b x i Z i I [n] Denote by R = {x R n : Ax b} the relaxation polyhedron, by L = {x R n : x i Z i I } the integrality restrictions, and by P = conv.hull(r L) the mixed-integer hull. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 3 / 22

Setup Problems in Question We consider mixed-integer programs with rational data: max c, x s.t. Ax b x i Z i I [n] Denote by R = {x R n : Ax b} the relaxation polyhedron, by L = {x R n : x i Z i I } the integrality restrictions, and by P = conv.hull(r L) the mixed-integer hull. Facts P is a polyhedron again. For most (e.g., NP-hard) problems, P has many facets. MIP-solvers are really fast these days. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 3 / 22

Polyhedral Combinatorics Another Fact The time a MIP-solver needs for solving depends on the strength of the relaxation, i.e., how well P is approximated by R. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 4 / 22

Polyhedral Combinatorics Another Fact The time a MIP-solver needs for solving depends on the strength of the relaxation, i.e., how well P is approximated by R. Strengthening a Relaxation Generic cutting planes: GMI, MIR, CG, Lift & Project,... Problem-specific inequalities: Problem-dependent Goals of Polyhedral Combinatorics Given a MIP-model for a problem, find inequalities valid for P (but not for R), develop algorithms (exact or heuristics) to separate these inequalities if there are too many, determine the dimension of P, i.e., find valid equations, and prove if/when the inequalities define facets of P. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 4 / 22

Usual Approach for IPs Step 1: Find all feasible points (a) By hand / handcrafted software (b) Some tool, e.g. PORTA s vint functionality or azove Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 5 / 22

Usual Approach for IPs Step 1: Find all feasible points (a) By hand / handcrafted software (b) Some tool, e.g. PORTA s vint functionality or azove Step 2: Compute Outer Description There are quite a few tools (PORTA, Polymake, azove) and several algorithms: The Beneath-and-Beyond method The Double-Description method Lexicographic Reverse Search Pyramid decomposition (mixture of beneath-and-beyond and Fourier-Motzkin) Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 5 / 22

Usual Approach for IPs Step 1: Find all feasible points (a) By hand / handcrafted software (b) Some tool, e.g. PORTA s vint functionality or azove Step 2: Compute Outer Description There are quite a few tools (PORTA, Polymake, azove) and several algorithms: The Beneath-and-Beyond method The Double-Description method Lexicographic Reverse Search Pyramid decomposition (mixture of beneath-and-beyond and Fourier-Motzkin) Step 3: Generalize Inequalities There s only one main tool here: The mathematician. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 5 / 22

Limitations Memory and Time The dominant of the cut polytope (corresponds to MinCut problem) has among others a facet per disjoint union of cycles joined together by any spanning tree! Alevras ( 99) enumerated the facets for this polyhedron for the complete graph on 8 nodes, including 2 billions of the above type. Continuous Variables The mixed-integer case is usually harder for enumeration tools. Symmetry Often, facets have a lot of symmetry and it is only sometimes possible to exploit (parts of it) during the convex-hull computation. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 6 / 22

Limitations Memory and Time The dominant of the cut polytope (corresponds to MinCut problem) has among others a facet per disjoint union of cycles joined together by any spanning tree! Alevras ( 99) enumerated the facets for this polyhedron for the complete graph on 8 nodes, including 2 billions of the above type. Continuous Variables The mixed-integer case is usually harder for enumeration tools. Symmetry Often, facets have a lot of symmetry and it is only sometimes possible to exploit (parts of it) during the convex-hull computation. Specific Objective Functions Which are the facets useful when optimizing specific objective functions? Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 6 / 22

Vision Fact Reminder PORTA & friends need a moment for a 50-dimensional polytope, while a MIP with 50 variables is usually solved within a second! Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 7 / 22

Vision Fact Reminder PORTA & friends need a moment for a 50-dimensional polytope, while a MIP with 50 variables is usually solved within a second! Goal of this work: Use MIP-solvers to determine facets! (and the affine hull) Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 7 / 22

Assumptions Definition (MIP-oracle Pair) A MIP-oracle pair for a rational MIP R L with relaxation R and integrality restrictions L consists of an oracle which solves max { c, x : x R L} for any objective c Q n, and an oracle which can separate over R, i.e., decides for given ˆx, whether ˆx R holds, and if not, yields a separating inequality. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 8 / 22

Assumptions Definition (MIP-oracle Pair) A MIP-oracle pair for a rational MIP R L with relaxation R and integrality restrictions L consists of an oracle which solves max { c, x : x R L} for any objective c Q n, and an oracle which can separate over R, i.e., decides for given ˆx, whether ˆx R holds, and if not, yields a separating inequality. Remark We can use an ordinary MIP solver and apply rational reconstruction techniques to the floating-point solution Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 8 / 22

Assumptions Definition (MIP-oracle Pair) A MIP-oracle pair for a rational MIP R L with relaxation R and integrality restrictions L consists of an oracle which solves max { c, x : x R L} for any objective c Q n, and an oracle which can separate over R, i.e., decides for given ˆx, whether ˆx R holds, and if not, yields a separating inequality. Remark We can use an ordinary MIP solver and apply rational reconstruction techniques to the floating-point solution (and hope for correctness). Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 8 / 22

Facets Finding Facets Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 9 / 22

Polarity - Basics Definition Let P R n be a polyhedron containing the origin O. Then the set {y R n : y, x 1 x P} is called the polar dual of P. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 10 / 22

Polarity - Basics Definition Let P R n be a polyhedron containing the origin O. Then the set {y R n : y, x 1 x P} is called the polar dual of P. Since the polar dual has a lineality space if P is not full-dimensional, let P := {y lin.hull (P) : y, x 1 x P}. denote the truncated polar dual of P. Proposition (see Schrijver 86) (P ) = P x is a point in (vertex of) P if and only if x, y 1 defines an inequality for (facet of) P. P is bounded if and only if P contains O in its relative interior. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 10 / 22

Using Polarity to Obtain Facets Suppose we know a point o in the relative interior of P. 1 Solve max x R c, x and max x P c, x (2 oracle calls!). If objective values coincide, R approximates P well in direction c. 2 If not, we obtain a point ˆx R \ P. Let s find a facet separating ˆx! 4 By polarity, we want a vertex y of (P o) with ˆx o, y > 1. For this we solve max ˆx o, y s.t. x o, y 1 x P y lin.hull(p o) using the simplex method. 5 The separation problem for the first constraint is again by polarity the problem to maximize ŷ over P (Many oracle calls!). Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 11 / 22

Using Polarity to Obtain Facets Suppose we know a point o in the relative interior of P. 1 Solve max x R c, x and max x P c, x (2 oracle calls!). If objective values coincide, R approximates P well in direction c. 2 If not, we obtain a point ˆx R \ P. Let s find a facet separating ˆx! 3 Restrict to facets that are tight at points x 1,..., x k. 4 By polarity, we want a vertex y of (P o) with ˆx o, y > 1. For this we solve max ˆx o, y s.t. x o, y 1 x P using the simplex method. y lin.hull(p o) x i o, y = 1 i = 1,..., k 5 The separation problem for the first constraint is again by polarity the problem to maximize ŷ over P (Many oracle calls!). Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 11 / 22

Using Polarity to Obtain Facets Suppose we know a point o in the relative interior of P. 1 Solve max x R c, x and max x P c, x (2 oracle calls!). If objective values coincide, R approximates P well in direction c. 2 If not, we obtain a point ˆx R \ P. Let s find a facet separating ˆx! 3 Restrict to facets that are tight at points x 1,..., x k. 4 By polarity, we want a vertex y of (P o) with ˆx o, y > 1. For this we solve max ˆx o, y s.t. x o, y 1 x P using the simplex method. y lin.hull(p o) x i o, y = 1 i = 1,..., k 5 The separation problem for the first constraint is again by polarity the problem to maximize ŷ over P (Many oracle calls!). Not new: Target Cuts invented by Buchheim, Liers & Oswald, 08 Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 11 / 22

Choice of the Interior Point Proposition The objective value of a facet a, x β of P equals a, ˆx a, o. β a, o ˆx x o maximize Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 12 / 22

Choice of the Interior Point Proposition The objective value of a facet a, x β of P equals a, ˆx a, o. β a, o ˆx x o o maximize Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 12 / 22

Affine Hull Computing Affine Hull & Dimension Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 13 / 22

Computing the Affine Hull Input: MIP-oracle pair for = P = conv.hull(r L) R n. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 14 / 22

Computing the Affine Hull Input: MIP-oracle pair for = P = conv.hull(r L) R n. Output: Dimension d of P (d + 1)-many affinely independent points in P (n d)-many irredundant equations Cx = d valid for P Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 14 / 22

Computing the Affine Hull Input: MIP-oracle pair for = P = conv.hull(r L) R n. Output: Dimension d of P (d + 1)-many affinely independent points in P (n d)-many irredundant equations Cx = d valid for P Ideas: 1 Maintain known equations Cx = d and points x 1, x 2,..., x l P. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 14 / 22

Computing the Affine Hull Input: Output: Ideas: MIP-oracle pair for = P = conv.hull(r L) R n. Dimension d of P (d + 1)-many affinely independent points in P (n d)-many irredundant equations Cx = d valid for P 1 Maintain known equations Cx = d and points x 1, x 2,..., x l P. 2 Repeatedly find a useful direction c R n, and compute z + := max x P c, x and z := min x P c, x (2 oracle calls!). 3 If we are lucky and z + = z holds, then c, x = z + is a valid equation. To make progress, we want c to be linear independent of C s rows. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 14 / 22

Computing the Affine Hull Input: Output: Ideas: MIP-oracle pair for = P = conv.hull(r L) R n. Dimension d of P (d + 1)-many affinely independent points in P (n d)-many irredundant equations Cx = d valid for P 1 Maintain known equations Cx = d and points x 1, x 2,..., x l P. 2 Repeatedly find a useful direction c R n, and compute z + := max x P c, x and z := min x P c, x (2 oracle calls!). 3 If we are lucky and z + = z holds, then c, x = z + is a valid equation. To make progress, we want c to be linear independent of C s rows. 4 If we also choose c to be orthogonal to aff.hull(x 1,..., x l ) (that is, c, x i = c, x 1 for all i = 2,..., l), the result z + > z yields a point x l+1 affinely independent of x 1,..., x l. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 14 / 22

Computing the Affine Hull (cont.) Reminder: x 1,..., x l P R n are affinely independent points found so far. Cx = d are the equations found so far. Algebra: Let D R (l 1) n be the matrix whose rows are x i x 1 for i = 2,..., l. We can choose c (ker C ker D) \ O. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 15 / 22

Computing the Affine Hull (cont.) Reminder: x 1,..., x l P R n are affinely independent points found so far. Cx = d are the equations found so far. Algebra: Let D R (l 1) n be the matrix whose rows are x i x 1 for i = 2,..., l. We can choose c (ker C ker D) \ O. Drawback: It works but we modeled linear independence by orthogonality! Conclusion: Consider a basis c 1,..., c k of ker D and use each c i as a direction (or skip c i if it linearly depends on C s rows). Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 15 / 22

Computing the Affine Hull (cont.) Reminder: x 1,..., x l P R n are affinely independent points found so far. Cx = d are the equations found so far. Algebra: Let D R (l 1) n be the matrix whose rows are x i x 1 for i = 2,..., l. We can choose c (ker C ker D) \ O. Drawback: It works but we modeled linear independence by orthogonality! Conclusion: Consider a basis c 1,..., c k of ker D and use each c i as a direction (or skip c i if it linearly depends on C s rows). Open Question: Can one do better than 2n oracle calls? Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 15 / 22

Minimize 1-Norm Making Equations and Facets Pretty Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 16 / 22

1-Norm Minimization Motivation Readability of produced equations & facets! a, x β with a Z n and β Z where (a, β) is a primitive vector. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 17 / 22

1-Norm Minimization Motivation Readability of produced equations & facets! a, x β with a Z n and β Z where (a, β) is a primitive vector. If dim(p) n, facet representations are not unique and equation normals are an arbitrary basis of aff.hull (P). Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 17 / 22

1-Norm Minimization Motivation Readability of produced equations & facets! a, x β with a Z n and β Z where (a, β) is a primitive vector. If dim(p) n, facet representations are not unique and equation normals are an arbitrary basis of aff.hull (P). Goal: Consider pairs of vectors and replace one by a nice linear combination. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 17 / 22

1-Norm Minimization Motivation Readability of produced equations & facets! a, x β with a Z n and β Z where (a, β) is a primitive vector. If dim(p) n, facet representations are not unique and equation normals are an arbitrary basis of aff.hull (P). Goal: Consider pairs of vectors and replace one by a nice linear combination. Problem (1-Norm Minimization in Dimension Two) Input are two linearly independent vectors u, v Z n. Find λ, µ Q with λ 0 and λu + µv Z n minimizing λu + µv 1. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 17 / 22

2 Vectors, Step 1: Lattice Basis Problem (1-Norm Minimization in Dimension Two) Input are two linearly independent vectors u, v Z n. Find λ, µ Q with λ 0 and λu + µv Z n minimizing λu + µv 1. Lemma ( ) u By computing the Hermite-Normal-Form of Z 2 n, we can replace u, v by a lattice basis u, v of lin.hull (u, v) Z n...... such that λu + µv is a multiple of v if and only if λ = 0. v Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 18 / 22

2 Vectors, Step 1: Lattice Basis Problem (1-Norm Minimization in Dimension Two) Input are two linearly independent vectors u, v Z n. Find λ, µ Q with λ 0 and λu + µv Z n minimizing λu + µv 1. Lemma ( ) u By computing the Hermite-Normal-Form of Z 2 n, we can replace u, v by a lattice basis u, v of lin.hull (u, v) Z n...... such that λu + µv is a multiple of v if and only if λ = 0. v Problem (Restricted 1-Norm Minimization in Dimension Two) Given a lattice basis u, v Z n, find λ, µ Z with λ > 0 minimizing λu + µv 1. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 18 / 22

2 Vectors, Step 2: Case Distinction Problem (Restricted 1-Norm Minimization in Dimension Two) Given a lattice basis u, v Z n, find λ, µ Z with λ > 0 minimizing λu + µv 1. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 19 / 22

2 Vectors, Step 2: Case Distinction Problem (Restricted 1-Norm Minimization in Dimension Two) Given a lattice basis u, v Z n, find λ, µ Z with λ > 0 minimizing λu + µv 1. Let s consider all sign-patterns of λu + µv in order to resolve the 1-norm! Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 19 / 22

2 Vectors, Step 2: Case Distinction Problem (Restricted 1-Norm Minimization in Dimension Two) Given a lattice basis u, v Z n, find λ, µ Z with λ > 0 minimizing λu + µv 1. Let s consider all sign-patterns of λu + µv in order to resolve the 1-norm! Observation: Sign of λu i + µv i depends on µ/λ compared to u i /v i. Lemma Let u, v Z n be lin. independent. { Let = q 0 < q 1 } <... < q k 1 < q k = be the sorted elements of Q := u i v i : i [n], v i 0 {± }. Then the sign-pattern of λu + µv is constant over all multiplier pairs λ, µ with with λ > 0 for which λ/µ is in any fixed interval [q j 1, q j ]. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 19 / 22

2 Vectors, Step 2: Case Distinction Problem (Restricted 1-Norm Minimization in Dimension Two) Given a lattice basis u, v Z n, find λ, µ Z with λ > 0 minimizing λu + µv 1. Let s consider all sign-patterns of λu + µv in order to resolve the 1-norm! Observation: Sign of λu i + µv i depends on µ/λ compared to u i /v i. Lemma Let u, v Z n be lin. independent. { Let = q 0 < q 1 } <... < q k 1 < q k = be the sorted elements of Q := u i v i : i [n], v i 0 {± }. Then the sign-pattern of λu + µv is constant over all multiplier pairs λ, µ with with λ > 0 for which λ/µ is in any fixed interval [q j 1, q j ]. Final Step: In such an interval the 1-norm is linear and integer programming in dimension 2 can be solved efficiently (see Eisenbrand & Laue, 03). Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 19 / 22

2 Vectors: Heuristic Approach Observation Let u, v Z n be lin. independent. { Let = q 0 < q 1 } <... < q k 1 < q k = be the sorted elements of Q := u i v i : i [n], v i 0 {± }. Then the only multiplier pairs λ, µ Z for which λu + µv has zero entries not present in u nor v are those with λ/µ Q. Heuristic: 1 Let w := u. 2 For all i [n] with v i 0 do: 1 Let w (i) := v i u u i v. 2 Divide w (i) by the g.c.d. of its entries. 3 If w (i) 1 < w 1, replace w by w (i). 3 Return w. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 20 / 22

Live Demo Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 21 / 22

Questions? Problems to be tackled by help of MIP-solvers: Compute all valid equations / the dimension d. Find facets useful when optimizing certain objective functions, together with d independent points proving that it is a facet. Compute the dimension of other faces, e.g. the optimal face. Compute the conflict graph for 0/1-variables. To be done: Finish basic code :-) Improve cutting plane procedure (stabilize, aging of cuts,... ). Carry out computational study. Find some nice facets for interesting polytopes. Investigate dimensions of non-facets. Matthias Walter Investigating Mixed-Integer Hulls using a MIP-Solver Aussois 2015 22 / 22