Integer Optimization: Mathematics, Algorithms, and Applications

Transcription

1 Integer Optimization: Mathematics, Algorithms, and Applications Sommerschool Jacobs University, July 2007 DFG Research Center Matheon Mathematics for key technologies Thorsten Koch Zuse Institute Berlin

2 Joint work with Thorsten Koch, Zuse Institute Berlin (ZIB) 2 /213 Tobias Achterberg Timo Berthold Benjamin Hiller Sebastian Orlowski Marc Pfetsch Andreas Tuchscherer

3 Thorsten Koch, Zuse Institute Berlin (ZIB) 3 /213 The Problem Solving Cycle in Modern Applied Mathematics The Real Problem Hardware Software Data GUI Practitioner Simulation Modelling Specialist Implementation in Practice Numerical Solution Optimization Algorithmic Implementation Design of Good Solution Algorithms Quick Check: Heuristics Simulation The Application Driven Approach Education Mathematical Model Mathematical Theory Computer Science Pure Mathematics

4 Modeling Cycle Thorsten Koch, Zuse Institute Berlin (ZIB) 4 /213

5 typical optimization problems Thorsten Koch, Zuse Institute Berlin (ZIB) 5 /213 max f ( x) or min f ( x) g ( x) = 0, i= 1, 2,..., k i h ( x) 0, j = 1, 2,..., m j n x ( and x S) n n R ( x R ) T min c x Ax = a T min c x Ax = a Bx b Bx b x 0 x 0 n ( x ) some x n n k x { } ( 0,1 ) j Z general (nonlinear) program NLP linear program LP program = optimization problem (linear) 0/1- mixedinteger program IP, MIP

6 Linear Programming Thorsten Koch, Zuse Institute Berlin (ZIB) 6 /213 max cx + cx cx subject to a x + a x a x = b n n 1 a x + a x a x = b n n 2 a x + a x a x = b m1 1 m2 2 mn n m 1 2. x, x,..., x 0 n n n max Ax x T cx = 0 b linear program in standard form

7 Linear Programming Thorsten Koch, Zuse Institute Berlin (ZIB) 7 / L. V. Kantorovitch: Foundations of linear programming (Nobel Prize 1975) 1947 G. B. Dantzig: Invention of the simplex algorithm T max c x Ax= b x 0 Today: From an economic point of view, linear programming is one of the most important developments of mathematics in the 20th century.

8 Stiglers Diet Problem : The first linear program Min x1 + x2 2x1 + x2 3 x1 + 2x2 3 x1 0 x2 0 costs protein carbohydrates potatoes beans minimizing the cost of food George J. Stigler Nobel Prize in economics 1982 Thorsten Koch, Zuse Institute Berlin (ZIB) 8 /213

9 Diet Problem Sets n nutrients / calorie thousands, protein grams, calcium grams, iron milligrams vitamin-a thousand ius, vitamin-b1 milligrams, vitamin-b2 milligrams, niacin milligrams, vitamin-c milligrams / f foods / wheat, cornmeal, cannedmilk, margarine, cheese, peanut-b, lard liver, porkroast, salmon, greenbeans, cabbage, onions, potatoes spinach, sweet-pot, peaches, prunes, limabeans, navybeans / Parameter b(n) required daily allowances of nutrients / calorie 3, protein 70, calcium.8, iron 12 vitamin-a 5, vitamin-b1 1.8, vitamin-b2 2.7, niacin 18, vitamin-c 75 / Table a(f,n) nutritive value of foods (per dollar spent) calorie protein calcium iron vitamin-a vitamin-b1 vitamin-b2 niacin vitamin-c (1000) (g) (g) (mg) (1000iu) (mg) (mg) (mg) (mg) wheat cornmeal cannedmilk margarine cheese peanut-b lard liver porkroast salmon greenbeans cabbage onions potatoes spinach sweet-pot peaches prunes limabeans navybeans Positive Variable x(f) dollars of food f to be purchased daily (dollars) Free Variable cost total food bill (dollars) Equations nb(n) nutrient balance (units), cb cost balance (dollars) ; nb(n).. sum(f, a(f,n)*x(f)) =g= b(n); cb.. cost=e= sum(f, x(f)); Model diet stiglers diet problem / nb,cb /; Thorsten Koch, Zuse Institute Berlin (ZIB) 9 /213

10 Solution of the Diet Problem Thorsten Koch, Zuse Institute Berlin (ZIB) 10 /213 Goal: find the cheapest combination of foods that will satisfy the daily requirements of a person motivated by the army s desire to meet nutritional requirements of the soldiers at minimum cost Army s problem had 77 unknowns and 9 constraints. Stigler solved problem using a heuristic: $39.93/year (1939) Laderman (1947) used simplex: $39.69/year (1939 prices) first large-scale computation took 120 man days on hand operated desk calculators (10 human computers )

11 George Dantzig and Ralph Gomory founding fathers ~1950 linear programming ~1960 integer programming Thorsten Koch, Zuse Institute Berlin (ZIB) 11 /213

12 George Dantzig and Ralph Gomory Thorsten Koch, Zuse Institute Berlin (ZIB) 12 /213 ISMP Atlanta 2000 the fathers of Linear Programming and Integer Programming

13 Dantzig and Bixby Thorsten Koch, Zuse Institute Berlin (ZIB) 13 /213 George Dantzig and Bob Bixby at the International Symposium on Mathematical Programming, Atlanta, August 2000

14 special simple combinatorial optimization problems Thorsten Koch, Zuse Institute Berlin (ZIB) 14 /213 Finding a minimum spanning tree shortest path maximum matching maximal flow through a network cost-minimal flow solvable in polynomial time by special purpose algorithms

15 Optimizers dream: Duality theorems Thorsten Koch, Zuse Institute Berlin (ZIB) 15 /213 Max-Flow Min-Cut Theorem The maximal (s,t)-flow in a capacitated network is equal to the minimal capacity of an (s,t)-cut. The Duality Theorem of linear programming max Ax x T cx b 0 = min T y A y 0 T y b c T

16 special hard combinatorial optimization problems Thorsten Koch, Zuse Institute Berlin (ZIB) 16 /213 travelling salesman problem location und routing set-packing, partitioning, -covering max-cut linear ordering scheduling (with a few exceptions) node and edge colouring NP-hard (in the sense of complexity theory) The most successful solution techniques employ linear programming.

17 1954, the Beginning of IP G. Dantzig, D.R. Fulkerson, S. Johnson Thorsten Koch, Zuse Institute Berlin (ZIB) 17 /213 USA 49 cities 1146 variables 1954

18 The Simplex Method Thorsten Koch, Zuse Institute Berlin (ZIB) 18 /213 Dantzig, 1947: primal Simplex Method Lemke, 1954; Beale, 1954: dual Simplex Method Dantzig, 1953: revised Simplex Method. Underlying Idea: Find a vertex of the set of feasible LP solutions (polyhedron) and move to a better neighbouring vertex, if possible.

19 The Simplex Method: an example Thorsten Koch, Zuse Institute Berlin (ZIB) 19 /213 min/max + x1 + 3x2 (1) - x2 <= 0 (2) - x1 - x2 <=-1 (3) - x1 + x2 <= 3 (4) + x1 <= 3 (5) + x1 + 2x2 <= 9 (3) (5) (4) (2) (1)

20 The Simplex Method: an example Thorsten Koch, Zuse Institute Berlin (ZIB) 20 /213 min/max + x1 + 3x2 (1) - x2 <= 0 (2) - x1 - x2 <=-1 (3) - x1 + x2 <= 3 (4) + x1 <= 3 (5) + x1 + 2x2 <= 9 (3) (5) (4) (2) (1)

21 computationally important idea of the Simplex Method Let a (m,n)-matrix A with full row rank m, an m-vector b and an n-vector c with m<n be given. For every vertex y of the polyhedron of feasible solutions of the LP, max Ax x T cx = 0 b there is a non-singular (m,m)-submatrix B (called basis) of A representing the vertex y (basic solution) as follows y = B 1 b, y = 0 B A = B N Many computational consequences: Update-formulas, reduced cost calculations, number of non-zeros of a vertex, Thorsten Koch, Zuse Institute Berlin (ZIB) 21 /213 N

22 cutting plane technique for integer and mixed-integer programming Thorsten Koch, Zuse Institute Berlin (ZIB) 22 /213 Feasible integer solutions Objective function Convex hull LP-based relaxation Cutting planes

23 Which LP solvers are used in practice? Thorsten Koch, Zuse Institute Berlin (ZIB) 23 /213 Fourier-Motzkin: hopeless Ellipsoid Method: total failure primal Simplex Method: good dual Simplex Method: better Barrier Method: for large LPs frequently even better For LP relaxations of MIPs: dual Simplex Method

24 PDS Models, Courtesy Bob Bixby Thorsten Koch, Zuse Institute Berlin (ZIB) 24 /213 Patient Distribution System : Carolan, Hill, Kennington, Niemi, Wichmann, An empirical evaluation of the KORBX algorithms for military airlift applications, Operations Research 38 (1990), pp MODEL ROWS pds pds pds pds pds pds pds pds pds CPLEX CPLEX CPLEX SPEEDUP Primal Simplex Dual Simplex Dual Simplex

25 Courtesy Bob Bixby Thorsten Koch, Zuse Institute Berlin (ZIB) 25 / Not just faster -- Growth with size: Quadratic then & Linear now! CPLEX 1.0 seconds CPLEX 8.0 seconds # Time Periods: PDS02 -- PDS70

26 Computational history: Thorsten Koch, Zuse Institute Berlin (ZIB) 26 / Dantzig, Fulkerson, S. Johnson: 42-city TSP Solved to optimality using cutting planes and LP 1957 Gomory Cutting plane algorithm: A complete solution 1960 Land, Doig; 1965 Dakin Branch-and-bound (B&B) 1971 MPSX/370, Benichou et al UMPIRE, Forrest, Hirst, Tomlin (and Beale) Good B&B remained the state-of-the-art in commercial codes, in spite of 1973 Padberg 1974 Balas (disjunctive programming) 1983 Crowder, Johnson, Padberg: PIPX, pure 0/1 MIP 1987 Van Roy and Wolsey: MPSARX, mixed 0/1 MIP Grötschel, Padberg, Rinaldi TSP (120, 666, 2392 city models solved)

27 1998 A New Generation of MIP Codes Thorsten Koch, Zuse Institute Berlin (ZIB) 27 /213 Presolve Numerous small ideas Linear programming Stable, robust dual simplex Variable/node selection Influenced by traveling salesman problem Cutting planes Gomory, knapsack covers, flow covers, mix-integer rounding, cliques, GUB covers, implied bounds, path cuts Heuristics 11 different tried at root Retried based upon success

28 Solving MIPs Thorsten Koch, Zuse Institute Berlin (ZIB) 28 /213 Branching Decompose problem in smaller subproblems Solve subproblems recursively branching tree Bounding LP relaxation lower bound on objective function Cutting planes Tighten LP relaxation improved lower bounds Column generation Dynamic generation of problem variables

29 Branching (MIP + CP + SAT) Thorsten Koch, Zuse Institute Berlin (ZIB) 29 /213 Current solution is infeasible

30 Branching (MIP + CP + SAT) Thorsten Koch, Zuse Institute Berlin (ZIB) 30 /213 Decomposition into subproblems removes infeasible solution

31 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 31 /213 Root node defines global problem

32 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 32 /213 c * 2 LP relaxation yields lower bound Branching decomposes problem into subproblems

33 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 33 /213 c * 2 LP relaxation yields lower bound Branching decomposes problem into subproblems LP relaxation is solved for subproblems

34 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 34 /213 c * 2 c * 8 LP relaxation yields lower bounds

35 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 35 /213 c * 2 c * 8 LP relaxation yields lower bounds

36 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 36 /213 c * 2 c * 8 c * 4 LP relaxation yields lower bounds

37 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 37 /213 c * 2 c * 8 c * 4 LP relaxation yields lower bounds

38 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 38 /213 c * 2 c * 8 c * 4 c * 10 LP relaxation yields lower bounds

39 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 39 /213 c * 2 c * 8 c * 4 c * 10 LP relaxation yields lower bounds

40 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 40 /213 c * 2 c * 8 c * 4 c * 5 c * 10 LP relaxation yields lower bounds

41 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 41 /213 c * 2 c * 8 c * 4 c * 5 c * 10 LP relaxation yields lower bounds

42 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 42 /213 c * 2 c * 8 c * 4 c * 5 c * 10 c = 8 LP relaxation yields lower bounds Primal solution yields upper bound

43 Bounding (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 43 /213 c * 2 c * 8 c * 4 c * 5 c * 10 c = 8 LP relaxation yields lower bounds Primal solution yields upper bound Subproblems cannot contain better solution

44 Cutting Planes (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 44 /213 Current solution is infeasible

45 Cutting Planes (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 45 /213 Infeasible solution is separated by cutting plane

46 Cutting Planes Example (MIP) Thorsten Koch, Zuse Institute Berlin (ZIB) 46 /213 Binary knapsack problem: LP solution: max 3x 1 + 5x 2 + 5x 3 + 4x 4 + x 5 s.t. 3x 1 + 6x 2 + 7x 3 + 6x 4 + 2x 5 18 x 1, x 2, x 3, x 4, x 5 {0,1} x 1 = x 2 = x 3 = 1, x 4 = 1/3, x 5 = 0 Knapsack cover cut: New LP solution: x 2 + x 3 + x 4 2 x 1 = x 2 = x 3 = 1, x 4 = 0, x 5 = 1

47 Branch-and-Bound Thorsten Koch, Zuse Institute Berlin (ZIB) 47 /213 Branching Select suitable candidate from C={x i* Z} Bounding Preprocessing/Domain Propagation Solve LP relaxation Create two subproblems with x i x i* and x i x i* list of subproblems Select next subproblem yes yes yes relaxation infeasible? no lower upper bound? no relaxation integral? no tighten relaxation (cuts) primal heuristics (upper bounds)

48 Branching Algorithms Thorsten Koch, Zuse Institute Berlin (ZIB) 48 /213

49 Branching Trees Thorsten Koch, Zuse Institute Berlin (ZIB) 49 /213

50 Branching Trees Thorsten Koch, Zuse Institute Berlin (ZIB) 50 /213

51 A Branching Tree Thorsten Koch, Zuse Institute Berlin (ZIB) 51 /213 Applegate Bixby Chvátal Cook

52 LP as subroutine of MIP Thorsten Koch, Zuse Institute Berlin (ZIB) 52 /213 The Simplex algorithm is the core engine of every Branch-and-Bound based MIP solver At least one LP is solved at each node in the branching tree, sometimes even more than 10. SoPlex is a composite Simplex. It features primal and dual algorithms for column and row basis.

53 Why some MIPs are difficult Thorsten Koch, Zuse Institute Berlin (ZIB) 53 /213

54 TSP model Thorsten Koch, Zuse Institute Berlin (ZIB) 54 /213

55 Extension Thorsten Koch, Zuse Institute Berlin (ZIB) 55 /213

56 Strengthened Extension Thorsten Koch, Zuse Institute Berlin (ZIB) 56 /213

57 Results CPLEX 10 V l =13 Thorsten Koch, Zuse Institute Berlin (ZIB) 57 /213

58 Times for permuted instances CPLEX (D) CPLEX (S) SCIP (D) 1 21,82 9,79 137, ,07 15,89 188, ,01 16,01 185, ,41 17,15 200, ,26 17,31 156, ,74 20,38 149, ,57 21,17 167, ,28 23,10 124, ,49 23,75 177, ,52 25,26 177, ,61 25,95 123, ,84 153,00 178, ,42 153,51 146, ,33 189,87 119, ,70 216,81 148,46 Average 161,53 62,54 159,28 Factor 19,04 22,15 2,66 Thorsten Koch, Zuse Institute Berlin (ZIB) 58 /213

59 SCIP Thorsten Koch, Zuse Institute Berlin (ZIB) 59 /213 constraint integer programming integrates constraint programming (CP) mixed integer programming (MIP) SAT solving techniques SCIP can be used as black box solver for MIP and SAT framework for constraint integer programming, including branch-cut-and-price available in source code lines of C source code free to use for non-commercial purposes

60 SCIP as Standalone MIP Solver Thorsten Koch, Zuse Institute Berlin (ZIB) 60 /213 reads MPS or LP file format directly reads ZIMPL models built-in MIP specific components: branching rules (reliability, strong, most infeasible,...) primal heuristics (rounding, diving, feas. pump,...) node selectors (depth-first, best-first with plunging) presolving (dual fixing, probing,...) cut separators (clique, impl. bounds, c-mir, Gomory, strong CG, lifted knapsack cover)

61 SCIP as CIP Framework Thorsten Koch, Zuse Institute Berlin (ZIB) 61 /213 C interface with C++ wrapper classes infrastructure to support user plugins subproblem and branching tree management global cut pool and cut selection pricing column management event mechanism (bound changes, new solutions,...) lots of statistical data about the solving process efficient memory management all existing MIP components are implemented as user plugins interface is powerful enough for most applications

62 Benchmarks (commercial) MIPLIB2003 / Mittelmann / Lodi test set Thorsten Koch, Zuse Institute Berlin (ZIB) 62 /213 SCIP using CPLEX 10.0 as LP solver: CPLEX 10.0 CPLEX SCIP 0.82 nodes (total) nodes (geom) time (total) time (geom) SCIP comparable to CPLEX factor 2 slower than CPLEX 10.0

63 Thorsten Koch, Zuse Institute Berlin (ZIB) 63 /213 SCIP using Soplex as LP solver: Benchmarks (free software) results from Hans Mittelmann's web site CBC GLPK MINTO SYMPHONY SCIP geometric mean of solving time (max: 7200 s)

64 Algebraic Modelling Language Thorsten Koch, Zuse Institute Berlin (ZIB) 64 /213

65 Example Thorsten Koch, Zuse Institute Berlin (ZIB) 65 /213

66 Extended functions Thorsten Koch, Zuse Institute Berlin (ZIB) 66 /213

67 Extended functions Thorsten Koch, Zuse Institute Berlin (ZIB) 67 /213

68 Expansion of extended functions Thorsten Koch, Zuse Institute Berlin (ZIB) 68 /213

69 Zimpl Thorsten Koch, Zuse Institute Berlin (ZIB) 69 /213

70 The Programs Thorsten Koch, Zuse Institute Berlin (ZIB) 70 /213 SCIP Solving Constraint Integer Programs SoPlex Sequential Object-oriented simplex Zimpl Zuse Institute Mathematical Programming Language and friends perplex - Rational arithmetic basis verification Porta - POlyhedron Representation Transformation Algorithm MCF - Min Cost Flow

71 Thank you very much! Thorsten Koch, Zuse Institute Berlin (ZIB) 71 /213

72 Questions? Thorsten Koch, Zuse Institute Berlin (ZIB) 72 /213