Heuristics in MILP Group 1 D. Assouline, N. Molyneaux, B. Morén Supervisors: Michel Bierlaire, Andrea Lodi Zinal 2017 Winter School 0 / 23
Primal heuristics Original paper: Fischetti, M. and Lodi, A. (2011). Heuristics in mixed integer programming. Wiley Encyclopedia of Operations Research and Management Science LP-based heuristics Feasibility pump Pivoting methods Rounding and diving OCTANE MILP-based heuristics Local branching RINS Evolutionary algorithms 1 / 23
Pivoting methods 1 Pivoting methods 2 Relaxation Induced Neighborhood Search (RINS) 3 An evolutionary algorithm within MILP 4 Conclusion 2 / 23
Pivoting methods Pivoting Methods Balas, E. and Martin, C. H. (1980). Pivot and complement a heuristic for 0-1 programming. Management Science MILP with binary and continuous variables, no integer variables MILP formulation min x c T x s. t. Ax b x j binary, j B x j continuous, j C 3 / 23
Pivoting methods Pivoting Methods Given a solution to the LP-relaxation Non-basic variables: at lower or upper bound, x i = l i or x i = u i Basic variables: between lower and upper bound, l i < x i < u i (nondegeneracy) Binary variables Either on lower or on upper bound If all binary variables are non-basic, solution is feasible Pivoting methods tries to pivot binary variables out of basis 4 / 23
Pivoting methods Pivoting Methods - Pivot steps Pivot types 1 Pivot binary basic variable out of basis, maintaining primal feasibility. 2 Reduce the binary infeasibility i B min{x i x i, x i x i } 3 Pivot binary basic variable out of basis, allowing primal infeasibility. 5 / 23
Pivoting methods Pivoting Methods - Extension Pivot and shift - integer variables and improving solution Combined with local branching Combined with tabu search Pivot, cut, and dive Pivot and Gomory cut 6 / 23
Relaxation Induced Neighborhood Search (RINS) 1 Pivoting methods 2 Relaxation Induced Neighborhood Search (RINS) 3 An evolutionary algorithm within MILP 4 Conclusion 7 / 23
Relaxation Induced Neighborhood Search (RINS) Relaxation Induced Neighborhood Search (RINS) Part of the MILP-based heuristics. Improvement heuristic: start with reference feasible solution x of an MILP, and try to find a better one. Main idea: Uses the local neighborhood search principle, using information from the LP relaxation solution to build the neighborhood. Observation: often the incumbent solution x of an MILP and the optimal LP-relaxation x have a few of variables set to the same values. These values form a partial solution of good objective value. Neighborhood exploration is then formulated as a sub-milp. 8 / 23
Relaxation Induced Neighborhood Search (RINS) RINS algorithm At each node of the branch-and-cut tree, we have the incumbent x, and x from the LP-relaxation at the node, and do the following: 1 Fix the integer variables that have the same value in x and x ; 2 Set an objective cutoff based on the objective value of the current incumbent; 3 Solve the associated sub-milp on the remaining variables (a simplified and smaller in size MILP!); 4 If incumbent solution is improved, x is updated in the rest of the tree. Sub-MILP in RINS min x c T x s. t. Ax b c T x (1 ɛ) c T x for some ɛ > 0 x j = x j j I, with x j = x j x j integer, j I 9 / 23
Relaxation Induced Neighborhood Search (RINS) RINS in practice Each sub-milp might be large and difficult to solve: the neighborhood exploration if often truncated, by setting a limited number of nodes nl. RINS can be invoked at every node of the branch-and-cut tree: very suitable for integration within solvers. (The LP relaxation is potentially different at each node, so neighborhood diversification is well performed) Yet, neighborhoods from consecutive nodes might be similar: we don t invoke RINS at each node, but every f node (f 1). Another way to control the frequence of RINS use is to impose a minimum ratio of variables mfr to fix allowing to invoke RINS. It is embedded in CPLEX 12.6! 10 / 23
Relaxation Induced Neighborhood Search (RINS) RINS results and discussion (a) Small spread Figure: Taken from [Danna et al., 2005] 1 (b) Large spread 1 Danna, E., Rothberg, E., and Le Pape, C. (2005). Exploring relaxation induced neighborhoods to improve mip solutions. Mathematical Programming, 102(1):71 90 11 / 23
Relaxation Induced Neighborhood Search (RINS) RINS discussion with LB The main difference between Local Branching and RINS is that if the former uses soft-fixing (with addition of a linear inequality), the latter uses hard-fixing (by considering the LP relaxation into account). That constitutes the biggest advantage of RINS: the sub-milp are smaller. Also, RINS can be invoked more often since x changes between the nodes. RINS theoretically leads to faster improvements, but lacks the flexibility of LB... 12 / 23
Relaxation Induced Neighborhood Search (RINS) RINS variants DINS: Distance Induced Neighborhood Search Idea: the most promising solutions are those close to x. The neighborhood is defined by the distance inequality: j I xj xj j I xj xj If xj xj <0.5, xj = x j, otherwise we set the bounds of the variables so that the distance with x cannot increase. RENS: Relaxation Enforced Neighborhood Search Idea: define a reduced MILP as the set of all integer solutions which can be obtained by rounding a relaxed solution x. It is basically a sophisticated rounding heuristic, and does not require an incumbent solution to work with. 13 / 23
An evolutionary algorithm within MILP 1 Pivoting methods 2 Relaxation Induced Neighborhood Search (RINS) 3 An evolutionary algorithm within MILP 4 Conclusion 14 / 23
An evolutionary algorithm within MILP Evolutionary algorithm within MILP Evolutionary algorithms are the parents of genetic algorithms (GA) The four important steps of GA are found in this heuristic: Population: a pool of feasible solutions (hopefully good ones) Combination: take two (or more) solutions and merge into a child Mutation: randomly change part of the solution Selection: procedure for selecting solutions to perform previous actions Unlike DNA, simply changing "bits" of an MILP solution will not generate a new feasible solution need to define a method for adressing this issue 15 / 23
An evolutionary algorithm within MILP Evolutionary algorithm: ensuring feasibility Combination and mutation steps follow a similar procedure: select some variables to change (permute if binary) solve the sub-milp using RINS (or other heuristics) For mutation, an adaptive approach for choosing the fraction of variables to change is given. (a) Combination process (b) Mutation process Figure: Taken from [Rothberg, 2007] 2 2 Rothberg, E. (2007). An evolutionary algorithm for polishing mixed integer programming solutions. INFORMS Journal on Computing, 19(4):534 541 16 / 23
An evolutionary algorithm within MILP Evolutionary algorithm: selection process When selecting solutions for combination & mutation a preference should be given to "good" solutions, but selecting systematically the best solutions will drastically reduce the diversity Suggested selection process: randomly pick first solution from pool, then second solution is selected from sub-pool of better solutions 17 / 23
An evolutionary algorithm within MILP Evolutionary algorithm: results Computationally expensive use for improving already acceptable solution (found with RINS, local branching and guided diving) Compare results from (i) default CPLEX, (2) RINS, local branching and guided diving and (3) polishing (evolutionary algorithm) Figure: Taken from [Rothberg, 2007] 18 / 23
An evolutionary algorithm within MILP Evolutionary algorithm: discussion Improves solutions for some problems, but not all lack of diversity or neighborhood size General framework applicable to many different problems Computationally expensive: solves many (60 for example) sub-milps cannot rely sollely on this Implemented in CPLEX 12.4 (and following) 19 / 23
Conclusion 1 Pivoting methods 2 Relaxation Induced Neighborhood Search (RINS) 3 An evolutionary algorithm within MILP 4 Conclusion 20 / 23
Conclusion Conclusion Various heuristics used for different stages of a problem solving need to choose one/some methods All heuristics have pros and cons, and should be used for an adapted problem. We are just scratching the surface" in terms of all existing heuristics. 21 / 23
Conclusion Thank you for you attention Questions? 22 / 23
Conclusion References Balas, E. and Martin, C. H. (1980). Pivot and complement a heuristic for 0-1 programming. Management Science. Danna, E., Rothberg, E., and Le Pape, C. (2005). Exploring relaxation induced neighborhoods to improve mip solutions. Mathematical Programming, 102(1):71 90. Fischetti, M. and Lodi, A. (2011). Heuristics in mixed integer programming. Wiley Encyclopedia of Operations Research and Management Science. Rothberg, E. (2007). An evolutionary algorithm for polishing mixed integer programming solutions. INFORMS Journal on Computing, 19(4):534 541. 23 / 23