Column Generation Based Primal Heuristics C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck University Bordeaux 1 & INRIA team RealOpt
Outline 1 Context Generic Primal Heuristics The Branch-and-Price Approach 2 Column Generation based Primal Heuristics Specificity of the B-a-P context Literature Review 3 Diving Heuristic An implementation with Limited Discrepancy Search Numerical tests 4 Discussion
Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]
Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]
Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]
Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]
Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]
Context Generic LP based heuristics min{ j c jx j : j a ijx j b i i, l j x j u j j} Rounding: Iteratively select a var x j and bound/fix it least fractional: argmin j {min{x j x j, x j x j }} least infeasibility: argmin j {up-lock j + down-lock j } where up-lock j = # constraint i with a ij < 0 and down-lock j = # constraint i with a ij > 0 guided search: argmin j { x j x inc j } least pseudo-cost: argmin j {LP val increase} greedy: argmin j { c j Pi a } ij Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule that of exact branch-and-bound sub-miping: rounding + MIP sol of the residual prob.
Context Generic LP based heuristics min{ j c jx j : j a ijx j b i i, l j x j u j j} Rounding: Iteratively select a var x j and bound/fix it least fractional: argmin j {min{x j x j, x j x j }} least infeasibility: argmin j {up-lock j + down-lock j } where up-lock j = # constraint i with a ij < 0 and down-lock j = # constraint i with a ij > 0 guided search: argmin j { x j x inc j } least pseudo-cost: argmin j {LP val increase} greedy: argmin j { c j Pi a } ij Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule that of exact branch-and-bound sub-miping: rounding + MIP sol of the residual prob.
Context Generic LP based heuristics min{ j c jx j : j a ijx j b i i, l j x j u j j} Rounding: Iteratively select a var x j and bound/fix it least fractional: argmin j {min{x j x j, x j x j }} least infeasibility: argmin j {up-lock j + down-lock j } where up-lock j = # constraint i with a ij < 0 and down-lock j = # constraint i with a ij > 0 guided search: argmin j { x j x inc j } least pseudo-cost: argmin j {LP val increase} greedy: argmin j { c j Pi a } ij Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule that of exact branch-and-bound sub-miping: rounding + MIP sol of the residual prob.
Context Generic LP based heuristics min{ j c jx j : j a ijx j b i i, l j x j u j j} Rounding: Iteratively select a var x j and bound/fix it least fractional: argmin j {min{x j x j, x j x j }} least infeasibility: argmin j {up-lock j + down-lock j } where up-lock j = # constraint i with a ij < 0 and down-lock j = # constraint i with a ij > 0 guided search: argmin j { x j x inc j } least pseudo-cost: argmin j {LP val increase} greedy: argmin j { c j Pi a } ij Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule that of exact branch-and-bound sub-miping: rounding + MIP sol of the residual prob.
Context Heuristic search in branch-and-bound tree Diving sub-miping
Context The Branch-and-Price Approach min c 1 x 1 + c 2 x 2 +... + c K x K D x 1 + D x 2 +... + D x K d B x 1 b B x 2 b...... B x K x 1 N n, x 2 N n,... x K N n. Relax Dx d = decomposition: subproblem {B x b, x N n } and a reformulation solved by Branch-and-Price: b min P S S cx s λ s P s S Dx s λ s d P S S λs = K λ N S Solve Master LP Pricing Problem y := X k x k = X S S cx s λ s Solve Master LP Solve Master LP Pricing Problem Pricing Problem
Context The Branch-and-Price Approach min c 1 x 1 + c 2 x 2 +... + c K x K D x 1 + D x 2 +... + D x K d B x 1 b B x 2 b...... B x K x 1 N n, x 2 N n,... x K N n. Relax Dx d = decomposition: subproblem {B x b, x N n } and a reformulation solved by Branch-and-Price: b min P S S cx s λ s P s S Dx s λ s d P S S λs = K λ N S Solve Master LP Pricing Problem y := X k x k = X S S cx s λ s Solve Master LP Solve Master LP Pricing Problem Pricing Problem
Context The Branch-and-Price Approach min c 1 x 1 + c 2 x 2 +... + c K x K D x 1 + D x 2 +... + D x K d B x 1 b B x 2 b...... B x K x 1 N n, x 2 N n,... x K N n. Relax Dx d = decomposition: subproblem {B x b, x N n } and a reformulation solved by Branch-and-Price: b min P S S cx s λ s P s S Dx s λ s d P S S λs = K λ N S Solve Master LP Pricing Problem y := X k x k = X S S cx s λ s Solve Master LP Solve Master LP Pricing Problem Pricing Problem
Column Generation based Primal Heuristics Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
Column Generation based Primal Heuristics Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
Column Generation based Primal Heuristics Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
Column Generation based Primal Heuristics Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue
Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue
Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue
Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue
Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue
Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master (generic, often leads to infeasibility) min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy (application specific) iteratively select column with best pseudo-cost / red-cost 3 Rounding (application specific) round down master LP solution + iterative round-up complete heuristically 4 Local search (application specific) delete columns from incumbent rebuild heuristically genericity & feasibility issue
Diving Heuristic Diving Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at heuristically set values: λ s λ s Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: + diversification procedure: limited backtracking
Diving Heuristic Diving Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at heuristically set values: λ s λ s Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: Stop. + diversification procedure: limited backtracking
Diving Heuristic Diving Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at heuristically set values: λ s λ s Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: Backtrack. + diversification procedure: limited backtracking
Diving Heuristic Diving plus diversification Depth-first search: select column: λ s λ s update master and SP apply preprocessing Backtracking: Limited Discrepancy Search MaxDiscrepancy = 2, MaxDepth = 3
Diving Heuristic Numerical tests: Cutting Stock & Vertex Coloring 2 1 3 5 4 pure B-a-P DH w LDS problem size #inst. #nodes time gap time CSP 50 10 245 26 0% 2 CSP 80 10 417 192 0% 14 VCP 11 211 18 17 116 0% 26
Diving Heuristic Numerical tests: Bin Packing With Conflicts conflict conflict conflict 20 inst. b-a-p b-a-p + diving w LDS class not opt. av. time not opt. av. time max. time t60 0 1.7 0 1.6 7.2 t120 1 156.0 0 14.9 846.3 t249 2 334.2 0 32.1 172.1 t501 0 245.9 0 174.9 793.5 u120 0 3.4 0 3.6 36.2 u250 0 23.9 0 14.6 30.9 u500 1 318.0 0 74.8 136.3 u1000 0 1401.2 0 552.8 5359.6
Diving Heuristic Numerical tests: Bin Packing With Conflicts conflict conflict conflict Tabu Search Diving H DH LDS class gap time gap time gap time t60 0.45% 37.5 0.11% 1.3 0% 1.5 t120 0.62% 40.0 0.66% 2.5 0% 5.9 t249 0.39% 51.9 0.35% 5.1 0% 32.1 t501 0.21% 58.9 0.16% 10.3 0% 174.9 u120 0.10% 22.4 0.18% 2.6 0% 3.3 u250 0.21% 52.1 0.07% 5.4 0% 14.6 u500 0.20% 69.7 0.03% 10.7 0% 74.8 u1000 0.22% 107.8 0.01% 21.5 0.002% 514.1
Diving Heuristic Numerical tests: Vehicle Routing Diving H problem size #inst. # unsolved gap time VRP An32-80 27 17 5.25% 70
Discussion Feedback on experimentation Rounding/Diving based on fixing master var. λ works when 1 Sufficiently many columns in the solution: s λ s = K with K 1 2 Objective function is flat : many solutions of the same costs 3 Most of the combinatorial difficulty is in the subproblem
Discussion Further research Exact Branch-and-Price with heuristic oracle First experiment on CSP with greedy knapsack solver Seems no good when most of the difficulty is in the SP Sub-MIPing Fixing original variables in aggregate space