Fundamentals of Integer Programming Di Yuan Department of Information Technology, Uppsala University January 2018
Outline Definition of integer programming Formulating some classical problems with integer programming Linear programming Solution methods for integer programming (Introduction to modeling language and solver) Impact of modeling on problem solving 2
Definition of Integer Programming Optimization and Programming Mathematical programming: to find the best solution from a set of alternatives A generic formulation: max f(x) min s. t. x 2 X Variables Objective function Solution set The variables (their values) represent problem solution The solution set is specified by equations and inequalities 3
Definition of Integer Programming Binary Knapsack A set of items, each with a value and weight A knapsack with a weight capacity limit Select items to maximize the total value of the knapsack, without exceeding the weight limit $12 2 kg $24 3 kg $11 2.5 kg $10 1 kg $30 4 kg $25 5 kg 4
Definition of Integer Programming Types of Optimization Models max f(x) min s. t. g i (x)» b i ; i = 1; : : : ; m Linear programming Integer (linear) programming Nonlinear programming Integer nonlinear programming 5
Definition of Integer Programming Generic Model of Mixed Integer (Linear) Programming (MIP) Special case: binary variables 6
Modeling Some Classical Problems with Integer Programming Coloring Assign a color to each vertex (node); two adjacent vertexes must use different colors Minimize the total number of colors used 7
Modeling Some Classical Problems with Integer Programming Coloring (cont d) How to define the variables and constraints? De ne color set C ² x ic = 1 if node i has color c ² y c = 1 if color c is used (by any node) 1 2 8
Modeling Some Classical Problems with Integer Programming Set Covering Candidate service centers and demand points Each service center has a deployment cost, and can serve a subset of the demand points Select a subset of service centers at minimum total cost, to cover all demand points 9
Modeling Some Classical Problems with Integer Programming Set Covering (cont d) ² J: Set of demand points ² I: Candidate set of service centers ² I j : Subset of service centers covering point j 2 J ² c i : Deployment cost, i 2 I ² x i = 1 if i 2 I is deployed X min c i x i s. t. X i2i x i 1; 8j 2 J i2i j x 2 f0; 1g 10
Modeling Some Classical Problems with Integer Programming Uncapacitated Facility Location A set of candidate facility (e.g., warehouse) locations A set of customers Opening a facility has a fixed charge Transportation cost between facility locations and customers 11
Modeling Some Classical Problems with Integer Programming Uncapacitated Facility Location (cont d) ² I: Candidate set of facility locations ² J: Set of customers ² f i : Fixed charge, i 2 I ² c ij : Transportation cost, i 2 I; j 2 J ² x ij = 1 if facility i 2 I serves customer j ² y i = 1 if facility i 2 I is deployed 12
Modeling Some Classical Problems with Integer Programming Traveling Salesman Problem Find a tour visiting each node in a graph exactly once with minimum length 13
Modeling Some Classical Problems with Integer Programming Traveling Salesman Problem (cont d) How to formulate this problem by integer programming? ² x ij = 1 if city j is visited immediately after city i (i 6= j) X X min c ij x ij i2n j2n:j6=i X s. t. x ij = 1; 8i 2 N j2n:j6=i x 2 f0; 1g Have we overlooked anything? X X i2s j2nns x ij 1; 8S ½ N Potential drawback of the formulation? 14
Linear Programming A Small Example x 2 5 x 1 5 15
Linear Programming Linear Programming Relaxation Relaxation: removal of some constraints/restrictions In general, the linear programming relaxation is an approximation of the integer model; the solution of the former may be fractional Which one is easier to solve? 16
Linear Programming Solving a Linear Programming Model Fundamental property: Optimum is located at one of the extreme/corner points (why?) This is used by the Simplex Method for solving linear programs (visiting a sequence of objective-improving extreme points) There are other efficient, interior-point methods 17
Linear Programming The Convex Hull Convex hull: The minimum convex set containing the solution space Integer programming = linear programming on the convex hull of the integer points Convex hull exists, but its description is hard to derive in general 18
Solution Methods for Integer Programming Computing the Global Optimum General-purpose method: Linear programming relaxation + Iterative improvement in approximating the convex hull, a.k.a. cutting planes Divide-and-conquer, a.k.a. branch-and-bound LP optimum LP optimum Optimality gap: The (relative) difference between the objective value of the best known integer solution and that of the best ( optimistic ) LP bound so far 19
Solution Methods for Integer Programming Cutting Planes: An Example Knapsack problem instance: LP optimum: x 1 = x 5 = x 6 = x 7 = 1, x 4 = 0:167, x 2 = x 3 = 0 Can we pack items 1, 4, 6, and 7 all in the knapsack? (5+6+6+5=22) The above inequality (referred as a cover cut ) is valid for integer solutions, but violated by the LP relaxation optimum 20
Solution Methods for Integer Programming Cutting Planes: An Example (cont d) Adding the cut to the linear programming relaxation: LP optimum: x 4 = x 5 = x 6 = x 7 = 1, x 1 = x 2 = x 3 = 0 Challenge: To time-efficiently find valid and useful cuts 21
Solution Methods for Integer Programming Branch-and-Bound: An Example Same knapsack problem instance: max 6x 1 + 4x 2 + 6x 3 + 7x 4 + 5x 5 + 9x 6 + 8x 7 s. t. 5x 1 + 6x 2 + 8x 3 + 6x 4 + 4x 5 + 6x 6 + 5x 7 21 x 2 f0; 1g LP optimum: x 1 = x 5 = x 6 = x 7 = 1, x 4 = 0:167, x 2 = x 3 = 0 Rounding down x 4 to zero gives an integer solution of value 28 ) Integer optimum is at least 28 22
Solution Methods for Integer Programming Branch-and-Bound: An Example (cont d) Branching generates a search tree Opt. gap =? L P = 2 8. 6 6 x 2 = 0 : 1 6 7 L P = 2 8. 7 5 x 3 = 0 : 1 2 5 x 4 = 0 x 4 = 1 x 3 = 0 x 3 = 1...... L P = 29. 16 Opt. gap =? x 4 = 0 : 167 ( o n e f r a c t i o n a l v a r i a b l e ) L P = 2 5. 5 x 5 = 0 : 5 Integer, value 29 Opt. gap =? We can stop branching here if the integer solution of value 29 is known (why?) Can we stop branching here because of an integer solution of value 28? 23
Introduction to Modeling Language and Solver Optimization Solver Solver: software implementing methods for solving optimization models (here: integer programming models) Interface + optimization engine knapsack.lp 24
Introduction to Modeling Language and Solver Another Sample of Solver Log for Integer Programming 25
Introduction to Modeling Language and Solver Modeling Language: Separation between Model and Data knapsack.mod knapsack6.dat 26
Introduction to Modeling Language and Solver Command Script: An Example 27
Introduction to Modeling Language and Solver Modeling System + Solver Command script - Model and data specification - Solver options - Display options - etc. Model Modeling system Data - Sets - Parameters - Variables - Objective function - Constraints Solver - Set elements - Parameter values Result analysis Feasible solution? Optimal solution? Optimum and variable value Solution time etc. 28
Introduction to Modeling Language and Solver AMPL Basics: Sets Simple sets (numbers or symbols) Indexed collection of sets 29
Introduction to Modeling Language and Solver AMPL Basics: Parameters Scalar parameter and parameters for set elements Bounds and default value Symbolic parameters 30
Introduction to Modeling Language and Solver AMPL Basics: Variables Similar to declaration of numerical parameters May have value and/or type restrictions 31
Introduction to Modeling Language and Solver AMPL Basics: Objective Function and Constraints Integer linear programming: The objective is a linear expression of the variables Single constraint Indexed collections of constraints (with condition) 32
Introduction to Modeling Language and Solver AMPL Basics: A Complete Model for Set Covering X min s. t. X i2i c i x i x i 1; 8j 2 J i2i j x 2 f0; 1g 33
Impact of Modeling on Problem Solving Uncapacitated Facility Location Can we reduce the model size? 34
Impact of Modeling on Problem Solving Uncapacitated Facility Location (cont d) Facilities Customers Aggregated Model Disaggregate Model (seconds) (seconds) 20 200 1.19 0.39 30 50 70 300 500 700 5.55 69.67 2362.67 1.6 19.42 764.30 AMPL Version 20060626 (Linux 2.6.9-5.EL)... Node log... Best integer = 1.433808e+05 Node = 0 Best node = 1.687869e+04 Best integer = 2.293498e+04 Node = 0 Best node = 1.687869e+04 Heuristic still looking. Heuristic still looking. Heuristic complete. Best integer = 2.276127e+04 Node = 828 Best node = 1.831542e+04 Best integer = 2.267411e+04 Node = 1000 Best node = 1.838685e+04 Implied bound cuts applied: 2292 Flow cuts applied: 18... Times (seconds): Solve = 2362.67 CPLEX 10.1.0: optimal integer solution within mipgap or absmipgap; objective 22674.11 1048766 MIP simplex iterations 16865 branch-and-bound nodes AMPL Version 20060626 (Linux 2.6.9-5.EL)... Node log... Best integer = 4.704366e+04 Node = 0 Best node = 2.203318e+04 Best integer = 2.295632e+04 Node = 0 Best node = 2.203318e+04 Heuristic still looking. Best integer = 2.267411e+04 Node = 0 Best node = 2.203529e+04 Heuristic complete. Gomory fractional cuts applied: 3 Using devex. Times (seconds): Solve = 764.304 CPLEX 10.1.0: optimal integer solution; objective 22674.11 157501 MIP simplex iterations 36 branch-and-bound nodes 35
Impact of Modeling on Problem Solving Coloring De ne color set C ² x ic = 1 if node i has color c ² y c = 1 if color c is used (by any node) 1 2 - Symmetry: Solution search becomes inefficient - Alternative model of set-covering type Is this a good formulation? Other formulations? 36
Impact of Modeling on Problem Solving Coloring (cont d) Sample results for an extension of graph coloring: Nodes Model I (previous slide) Model II (not shown) Best Solution Time Best Solution Time 10 10 0.1s 10 0.1s 20 16 1s 16 3s 30 21 10h 21 7s 40 15 10h 15 32s 50 28 10h 23 1m19s 60 31 10h 26 4m31s It may matter (a lot) which model you use 37