The Gurobi Optimizer. Bob Bixby

Transcription

1 The Gurobi Optimizer Bob Bixby

2 Outline Gurobi Introduction Company Products Benchmarks Gurobi Technology Rethinking MIP MIP as a bag of tricks 8-Jul Gurobi Optimization 2

3 Gurobi Optimization Incorporated July, 2008 Founders: Zonghao Gu, Ed Rothberg, Bob Bixby Product Releases: Version 1.0: May 2009 Performance roughly equal to CPLEX 11.0 Version 2.0: October 2009 Version 3.0: April 2010 Version 4.0: November 2010 Version 4.5: April

4 The Gurobi Products Gurobi Solvers Mixed-Integer Programming (MIP and MIQP) Deterministic, parallel Linear and Quadratic Programming Dual and primal simplex Parallel Barrier APIs Simple command-line interface Python interactive interface C, C++, Java,.NET, Python callable libraries All standard modeling languages Commercial and Academic Licenses Licensing options and pricing available at 4

5 Company Business Model Focus on math programming solvers Our business is helping our customers succeed with math programming solutions Provide the best possible support from people who know math programming Be a flexible partner Licensing models that meet user needs Clear, upfront pricing Be the technology leader Build the best available math programming technology 5

6 Expanding the Reach of Optimization 6

7 Expanding the Reach of Optimization Multi-core Parallelism at no extra charge Gurobi was the first Free Trial Licenses Automated download and license generation 500 variable by 500 constraint size limit Academic Program Free single-user licenses for academics Automated download and license generation Pay-as-you-go Licenses Amazon EC2 Pay-by-the-day licenses 7

8 Academic Licenses Over 8400 free academic licenses issued Over 60% outside the US Over 50% outside the OR community 8

9 MIP Performance Benchmarks 9

10 MIP Performance Gurobi Internal Test Set Version-to-version improvements: (Geometric mean runtime over ~800 models in our internal model set that take more than 100s to solve) Gurobi 1.0 -> 2.0: 2.2X Gurobi 2.0 -> 3.0: 2.9X (6.4X) Gurobi 3.0 -> 4.0: 1.3X (8.3X) Gurobi 4.0 -> 4.5: 1.8X (14.9X) Continued improvement in our ability to solve difficult MIP models 10

11 MIP/MIQP Performance Mittelmann Benchmarks Gurobi 4.5 vs. Competition: Solve times (> 1.0 means Gurobi faster) Benchmark CPLEX XPRESS P=1 P=4 P=12 P=1 P=4 P=12 MIPLIB X 1.19X 1.19X 1.01X 1.21X 1.16X Feasibility 3.57X X - - Infeasibility X X Pathological X X MIQP X X - 11

12 Gurobi Technology

13 Basic MIP Technology: A Fresh Look 13

14 Building From the Ground Up Start from a clean slate Things have changed: Two particular examples: Sub-MIP as a pervasive approach Ubiquitous parallel processing 14

15 Sub-MIP As A New Paradigm Key recent insight for heuristics: Can use MIP solver recursively Solve a related model: Hopefully smaller and simpler Examples: Local cuts [Applegate, Bixby, Chvatal & Cook, 2001] Local branching [Fischetti & Lodi, 2003] RINS [Danna, Rothberg, Le Pape, 2005] Solution polishing [Rothberg, 2007] 15

16 Rethinking MIP Tree Search 16

17 Branch-and-Bound Each node in branch-and-bound is a new MIP Original model, plus several variable fixings Can view search tree as a tree-oftrees The nature of a sub- MIP can change dramatically 17

18 Tree-of-Trees Gurobi MIP search tree manager built to handle multiple related trees Can transform any node into the root node of a new tree Maintains a pool of nodes from all trees No need to dedicate the search to a single subtree 18

19 Tree-of-Trees Each tree has its own relaxation and its own strategies... Presolved model for each subtree Cuts specific to that subtree Pseudo-costs for that subtree only Symmetry detection on that submodel Etc. Captures structure that is often not visible in the original model 19

20 Heuristics 20

21 Summary of Heuristics 5 heuristics prior to solving root LP 5 different variable orders, fix variables in this order 15 heuristics within tree (9 primary, several variations) RINS, RINS diving, rounding, fix and dive (LP), fix and dive (Presolve), fix and dive (simple), Lagrangian approach, pseudo costs, Hail Mary (set objective to 0) 3 solution improvement heuristics Applied whenever a new integer feasible is found Key tool: Bound strengthening 21

22 MIP is a bag of tricks! 22

23 Improving a MIP Solver Improvements can be plotted on two axes: Speedup Big speedups on a few models Big speedups on lots of models Modest speedups on lots of models Generality 23

24 New Ideas Speedup Nine out of ten ideas end up here Experience generally keeps you away from here Generality 24

25 Gurobi 2.0 Speedup Node files SOS improvements Dual Simplex improvements Zero-half cuts Parallel improvements Generality 25

26 Gurobi 3.0 Speedup Symmetry New presolve red. SubMIP cuts Network cuts Barrier Cut tuning Generality 26

27 Disjoint Subtrees

28 Disjoint Subtrees Basic principle of branching: Feasible regions for child nodes after a branch should be disjoint Not always the case Simple example integer complementarity: x 10 b y 10 (1-b) x, y non-negative ints, x 10, y 10, b binary Branch on b: x=y=0 feasible in both children 28

29 Removing Overlap Simplest way to remove overlap: Modify variable bound in one subtree b=0 child: x = 0, 10 y 0 b=1 child: y = 0, 10 x 1 29

30 Performance Impact Overlap present in several models 35 out of 510 models in our test set Performance impact can be huge Model neos goes from seconds to 0.01s Median improvement for affected models is ~1% 30

31 A Presolve Reduction

32 A Presolve Reduction Consider a x + b y = c x, y are integer variables a, b and c are integers, a > 1 Assume gcd(a,b) = 1 Otherwise a Euclidean reduction is possible Observation: Then x(mod b) and y(mod a) are constants. Reduction Substitute y = a z + d (d is easy to compute). z has a smaller search space than y General application Can easily be extended to general all integer constraints. 32

33 Pseudo-Cost Adjustment 33

34 Pseudo-Costs What s a good branching variable? Superb: fractional variable infeasible in both branch directions Great: infeasible in one direction Good: both directions move the objective Expensive to predict which branches lead to infeasibility or big objective moves Strong branching Truncated LP solve for every possible branch at every node Rarely cost effective Need a quick estimate 34

35 Pseudo-Costs Use historical data to predict impact of a branch: Record cost x = obj / x for each branch Need a scheme for infeasible branches too Store results in a pseudo-cost table Two entries per integer variable Average (or max) down cost Average (or max) up cost Use table to predict cost of a future branch 35

36 Pseudo-Cost Initialization What do you do when there is no history? E.g., at the root node Initialize pseudo-costs [Linderoth & Savelsbergh, 1999] Always compute up/down cost (using strong branching) for new fractional variables Initialize pseudo-costs for every fractional variable at root Reliability branching [Achterberg, Koch, & Martin, 2002] Don t rely on historical data until pseudo-cost for a variable has been recomputed r times 36

37 Pseudo-Cost Adjustment Gurobi 4.5 adjusts pseudo-costs in several ways Implied pseudo-cost bounds Ancestor adjustment 37

38 Implied Pseudo-Cost Bounds Consider constraint: x i = 1 Computed objective bounds: x 1 =1 -> obj 100 x 2 =0 -> obj 110 Stronger bound for x 1 =1: x 1 =1 -> x 2 =0 -> obj 110 In general: If x=a -> y=b, then obj x=a obj y=b Often violated: Strong branching uses an iteration limit 38

39 Implied Pseudo-Cost Bounds Consider x i = 1 again Computed objective bounds: x 1 =0 -> obj 100 x j =1 -> obj 110 for all j!= 1 Stronger bound for x 1 =0: x 1 =0 -> x j =1 for some j!= 1 -> obj 110 In general: If x i =a -> x j =b for some j!= i Then obj xi=a min(obj xj=b ) 39

40 Ancestor Adjustment Objective move isn t entirely the result of most recent branch variable Depends on ancestors as well Particularly true for infeasible nodes Adjust pseudocosts for ancestor branches 40

41 Ancestor Adjustment Need an adjustment strategy Empirically, adjustment should decrease as you move up the tree Our approach: Exponential backoff ½ for parent, ¼ for grandparent, etc. 41

42 Product Roadmap

43 Projected Gurobi Roadmap Version 4.X November 2011 LP & MIP performance release Version 5.0 April 2012 SOCP and MISOCP MIP performance enhancements Version 6.0 New features under discussion MIP performance enhancements 43

44 Thank You