Column Generation Based Primal Heuristics

Transcription

1 Column Generation Based Primal Heuristics C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck University Bordeaux 1 & INRIA team RealOpt

2 Outline 1 Context Generic Primal Heuristics The Branch-and-Price Approach 2 Column Generation based Primal Heuristics Specificity of the B-a-P context Literature Review 3 Diving Heuristic An implementation with Limited Discrepancy Search Numerical tests 4 Discussion

3 Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]

4 Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]

5 Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]

6 Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]

7 Context Generic Primal Heuristics for MIPs good feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold 06] 1 Relaxation Induced Neighborhood Search [Dana al 05] 2 Local Branching [Fischetti al 03] 3 Feasibility Pump [Fischetti al 05]

8 Context Generic LP based heuristics min{ j c jx j : j a ijx j b i i, l j x j u j j} Rounding: Iteratively select a var x j and bound/fix it least fractional: argmin j {min{x j x j, x j x j }} least infeasibility: argmin j {up-lock j + down-lock j } where up-lock j = # constraint i with a ij < 0 and down-lock j = # constraint i with a ij > 0 guided search: argmin j { x j x inc j } least pseudo-cost: argmin j {LP val increase} greedy: argmin j { c j Pi a } ij Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule that of exact branch-and-bound sub-miping: rounding + MIP sol of the residual prob.

9 Context Generic LP based heuristics min{ j c jx j : j a ijx j b i i, l j x j u j j} Rounding: Iteratively select a var x j and bound/fix it least fractional: argmin j {min{x j x j, x j x j }} least infeasibility: argmin j {up-lock j + down-lock j } where up-lock j = # constraint i with a ij < 0 and down-lock j = # constraint i with a ij > 0 guided search: argmin j { x j x inc j } least pseudo-cost: argmin j {LP val increase} greedy: argmin j { c j Pi a } ij Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule that of exact branch-and-bound sub-miping: rounding + MIP sol of the residual prob.

10 Context Generic LP based heuristics min{ j c jx j : j a ijx j b i i, l j x j u j j} Rounding: Iteratively select a var x j and bound/fix it least fractional: argmin j {min{x j x j, x j x j }} least infeasibility: argmin j {up-lock j + down-lock j } where up-lock j = # constraint i with a ij < 0 and down-lock j = # constraint i with a ij > 0 guided search: argmin j { x j x inc j } least pseudo-cost: argmin j {LP val increase} greedy: argmin j { c j Pi a } ij Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule that of exact branch-and-bound sub-miping: rounding + MIP sol of the residual prob.

11 Context Generic LP based heuristics min{ j c jx j : j a ijx j b i i, l j x j u j j} Rounding: Iteratively select a var x j and bound/fix it least fractional: argmin j {min{x j x j, x j x j }} least infeasibility: argmin j {up-lock j + down-lock j } where up-lock j = # constraint i with a ij < 0 and down-lock j = # constraint i with a ij > 0 guided search: argmin j { x j x inc j } least pseudo-cost: argmin j {LP val increase} greedy: argmin j { c j Pi a } ij Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule that of exact branch-and-bound sub-miping: rounding + MIP sol of the residual prob.

12 Context Heuristic search in branch-and-bound tree Diving sub-miping

13 Context The Branch-and-Price Approach min c 1 x 1 + c 2 x c K x K D x 1 + D x D x K d B x 1 b B x 2 b B x K x 1 N n, x 2 N n,... x K N n. Relax Dx d = decomposition: subproblem {B x b, x N n } and a reformulation solved by Branch-and-Price: b min P S S cx s λ s P s S Dx s λ s d P S S λs = K λ N S Solve Master LP Pricing Problem y := X k x k = X S S cx s λ s Solve Master LP Solve Master LP Pricing Problem Pricing Problem

14 Context The Branch-and-Price Approach min c 1 x 1 + c 2 x c K x K D x 1 + D x D x K d B x 1 b B x 2 b B x K x 1 N n, x 2 N n,... x K N n. Relax Dx d = decomposition: subproblem {B x b, x N n } and a reformulation solved by Branch-and-Price: b min P S S cx s λ s P s S Dx s λ s d P S S λs = K λ N S Solve Master LP Pricing Problem y := X k x k = X S S cx s λ s Solve Master LP Solve Master LP Pricing Problem Pricing Problem

15 Context The Branch-and-Price Approach min c 1 x 1 + c 2 x c K x K D x 1 + D x D x K d B x 1 b B x 2 b B x K x 1 N n, x 2 N n,... x K N n. Relax Dx d = decomposition: subproblem {B x b, x N n } and a reformulation solved by Branch-and-Price: b min P S S cx s λ s P s S Dx s λ s d P S S λs = K λ N S Solve Master LP Pricing Problem y := X k x k = X S S cx s λ s Solve Master LP Solve Master LP Pricing Problem Pricing Problem

16 Column Generation based Primal Heuristics Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)

17 Column Generation based Primal Heuristics Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)

18 Column Generation based Primal Heuristics Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)

19 Column Generation based Primal Heuristics Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)

20 Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue

21 Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue

22 Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue

23 Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue

24 Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy iteratively select column with best pseudo-cost / red-cost 3 Rounding round down master LP solution + iterative round-up complete heuristically 4 Local search delete columns from incumbent rebuild heuristically genericity & feasibility issue

25 Column Generation based Primal Heuristics Review of column generation based heuristics 1 Restricted Master (generic, often leads to infeasibility) min{ S c sλ s : s Dx s λ s d, s λ s = K, λ s N s S} restricted set build either heuristically / from master LP 2 Greedy (application specific) iteratively select column with best pseudo-cost / red-cost 3 Rounding (application specific) round down master LP solution + iterative round-up complete heuristically 4 Local search (application specific) delete columns from incumbent rebuild heuristically genericity & feasibility issue

26 Diving Heuristic Diving Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at heuristically set values: λ s λ s Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: + diversification procedure: limited backtracking

27 Diving Heuristic Diving Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at heuristically set values: λ s λ s Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: Stop. + diversification procedure: limited backtracking

28 Diving Heuristic Diving Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at heuristically set values: λ s λ s Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: Backtrack. + diversification procedure: limited backtracking

29 Diving Heuristic Diving plus diversification Depth-first search: select column: λ s λ s update master and SP apply preprocessing Backtracking: Limited Discrepancy Search MaxDiscrepancy = 2, MaxDepth = 3

30 Diving Heuristic Numerical tests: Cutting Stock & Vertex Coloring pure B-a-P DH w LDS problem size #inst. #nodes time gap time CSP % 2 CSP % 14 VCP % 26

31 Diving Heuristic Numerical tests: Bin Packing With Conflicts conflict conflict conflict 20 inst. b-a-p b-a-p + diving w LDS class not opt. av. time not opt. av. time max. time t t t t u u u u

32 Diving Heuristic Numerical tests: Bin Packing With Conflicts conflict conflict conflict Tabu Search Diving H DH LDS class gap time gap time gap time t % % 1.3 0% 1.5 t % % 2.5 0% 5.9 t % % 5.1 0% 32.1 t % % % u % % 2.6 0% 3.3 u % % 5.4 0% 14.6 u % % % 74.8 u % % % 514.1

33 Diving Heuristic Numerical tests: Vehicle Routing Diving H problem size #inst. # unsolved gap time VRP An % 70

34 Discussion Feedback on experimentation Rounding/Diving based on fixing master var. λ works when 1 Sufficiently many columns in the solution: s λ s = K with K 1 2 Objective function is flat : many solutions of the same costs 3 Most of the combinatorial difficulty is in the subproblem

35 Discussion Further research Exact Branch-and-Price with heuristic oracle First experiment on CSP with greedy knapsack solver Seems no good when most of the difficulty is in the SP Sub-MIPing Fixing original variables in aggregate space