Exact solutions to mixed-integer linear programming problems

Exact solutions to mixed-integer linear programming problems Dan Steffy Zuse Institute Berlin and Oakland University Joint work with Bill Cook, Thorsten Koch and Kati Wolter November 18, 2011

Mixed-Integer Programming Mixed-Integer Linear Program. max s.t. c T x Ax b x R n 1 Z n 2

Floating-point Computation for MIP What can go wrong? Suboptimal solution returned Infeasible solution returned Feasible problem declared infeasible

Deviation in Integer Programming Software Results Default Settings Presolve, cuts and heur. disabled Example SCIP CPLEX SCIP CPLEX aim-200-1 6-yes1-3 200 200 200 177 alu10 7 91 86 84 83 alu8 9 2.341e5 91 2.341e5 90 neos-585467 399.373 399.373 397.363 392.805 neos-619167 [1.643, 2.611] 2.141 2.141 1.664 ns1637403 46 46 46 14 ns2122603 [94, 7e5] infeasible [3.3, ] [1.7, ] opti 157 0 8.593e3 infeas./unbd. infeasible infeasible opti 24 1 8.210e2 infeas./unbd. infeasible infeasible ran14x18.disj-8 [3595, 3714] 3761 [3574, 3735] [3618, 3797] transportmoment infeasible infeas./unbd. unbounded [-5e10, -3e9] Results of FP based solvers (2 hour time limit) Numerically difficult instances (from MIPLIB 2010 and others)

Necessity of Exact Solutions Where are exact MIP results needed? VLSI chip design verification Combinatorial auctions Numerically difficult problems Verifying results of test libraries

Rational Arithmetic for Linear Programming Applegate, Cook, Dash and Espinoza [2007] tested rational simplex implementation. http://gmplib.org It was hundreds or thousands of times slower than floating-point code.

Exact Linear Programming QSopt ex: Exact Rational LP Solver 1 Simplex method performed limited/fixed precision Final basic solution checked exactly Precision increased if needed Only 2-5x slower than FP-codes 1 Developed by Applegate, Cook, Dash and Espinoza [2007]

Branch-and-Bound Procedure max

Branch-and-Bound Procedure max f

Branch-and-Bound Procedure max x

Exact Mixed Integer Programming What happens if we use a fast exact LP solver at every node?

Exact Mixed Integer Programming What happens if we use a fast exact LP solver at every node? Applegate, Dash, Cook, Espinoza [2007] found this considerably slower than FP code. Warm starting the LPs in branch-and-bound means lots of LPs with less pivots per LP.

Exact Mixed Integer Programming Hybrid Approach for Exact MIP: Branch & Bound

Exact Mixed Integer Programming Hybrid Approach for Exact MIP: Branch & Bound 1 Store exact representation of problem Perform many operations on approximation or relaxation Exact or safe methods must be used for: Computing feasible solutions Computing LP bounds 1 see Cook, Koch, Steffy and Wolter [2011]

Exact Mixed Integer Programming Hybrid Approach for Exact MIP: Branch & Bound 1 Store exact representation of problem Perform many operations on approximation or relaxation Exact or safe methods must be used for: Computing feasible solutions use exact LP solver Computing LP bounds many choices here 1 see Cook, Koch, Steffy and Wolter [2011]

Valid Dual Bounds: Exact LP and Basis Verification Exact LP Solve each node LP exactly using QSopt ex Produces tightest possible bound

Valid Dual Bounds: Exact LP and Basis Verification Exact LP Solve each node LP exactly using QSopt ex Produces tightest possible bound Basis Verification Recompute basic sol. from floating-point LP solver exactly If dual feasible return valid dual bound Otherwise return infinite bound, branch (Hopefully branching can resolve numerical troubles)

Valid Dual Bounds: Primal-Bound-Shift Linear Program Primal: max s.t. Dual: min s.t. c T x Ax b l x u b T y l T z l + u T z u A T y Iz l + Iz u = c y, z l, z u 0 Idea: Use dual variables from primal bounds to correct approximate dual solution Applegate et al. [2006], Neumaier and Shcherbina [2004]

Valid Dual Bounds: Primal-Bound-Shift Linear Program Primal: max s.t. Dual: min s.t. c T x Ax b l x u b T y l T z l + u T z u A T y Iz l + Iz u = c y, z l, z u 0 Rigorous objective bound: Let ŷ, ẑ l, ẑ u 0 be an approximate dual solution Set r = c A T ŷ + ẑ l ẑ u (y, z l, z u ) = (ŷ, ẑ l + r +, ẑ u + r ) is a valid dual solution

Valid Dual Bounds: Primal-Bound-Shift Linear Program Primal: max c T x s.t. Ax b l x u Dual: min b T y l T z l + u T z u s.t. A T y Iz l + Iz u = c y, z l, z u 0 Good: Bad: Bound is trivial to compute Floating-point computation with directed rounding can be used Strength of the bound depends on tightness of primal variable bounds

Valid Dual Bounds: Project-and-Shift Linear Program Primal: max c T x s.t. Ax b Dual: min b T y s.t. A T y = c y 0 A more general procedure:

Valid Dual Bounds: Project-and-Shift Linear Program Primal: max c T x s.t. Ax b Dual: min b T y s.t. A T y = c y 0 A more general procedure: Main Idea: Find approximate dual solution ỹ Project ỹ to satisfy A T y = c Maintaining feasibility, shift to satisfy y 0

Valid Dual Bounds: Project-and-Shift Linear Program Primal: max c T x s.t. Ax b Dual: min b T y s.t. A T y = c y 0 Comments: Under some assumptions, most expensive computations can be done only once at root node and reused through the tree More general than Primal-Bound-Shift (but not entirely general) Some exact computation still required

Problem Structure in Tree Root Primal: max c T x s.t. Ax b Root Dual: min b T y s.t. A T y = c y 0 Node Primal: max c T x s.t. Ax b Āx b Node Dual: min b T y + b T z s.t. A T y + Ā T z y, z 0 = c

Valid Dual Bounds: Project-and-Shift Components of Project-and-Shift for MIP Root Node: Setup Phase Compute exact LU factorization of A T Determine (exact) corrector point y A T y = c, y > 0

Valid Dual Bounds: Project-and-Shift Components of Project-and-Shift for MIP Root Node: Setup Phase Compute exact LU factorization of A T Determine (exact) corrector point y A T y = c, y > 0 Node Bound Computation Start with approximate solution Project into equality space (using LU = A T ) Take convex combination with y to ensure feasibility

Implementation Implementation of Hybrid Branch-and-Bound Method Branch and bound framework: SCIP 1.2.0.8 Exact LP solver: QSopt ex 2.5.5 Floating-point LP solver: CPLEX 12.10 Multiple precision arithmetic package: GMP

Computational Tests Dual Bounding Methods Exact LP Basis Verification Primal-Bound-Shift Project-and-Shift Combination of above methods (automatic selection at each node)

Computational Tests Dual Bounding Methods Exact LP Basis Verification Primal-Bound-Shift Project-and-Shift Combination of above methods (automatic selection at each node) Pure Branch-and-Bound First Fractional Branching Results reported on 57 test problems selected from the MIPLIB and Mittelmann test libraries that were solved by (floating-point) SCIP branch-and-bound within 1 hour

Overall Computation Time # solved in 24h. Nodes Time (s.) Inexact 57 18,030 59.4 Exact LP 51 18,076 550.7 VerifyBasis 51 18,078 461.8 BoundShift 43 24,994 110.4 ProjectShift 49 18,206 369.3 Automatic 55 18,226 91.4 Nodes and time reported are the geometric means over 37/57 problems solved by all methods within 24h. time limit

Future Plans Some Future Plans: Cutting planes and pre-solve Certification of results Solve test library problems

East Coast Computer Algebra Day 2012 ECCAD 2012 Where: Oakland University, Rochester, Michigan When: Saturday May 12, 2012... more details to come

Exact solutions to mixed-integer linear programming problems Dan Steffy Zuse Institute Berlin and Oakland University Joint work with Bill Cook, Thorsten Koch and Kati Wolter November 18, 2011