Recent Developments in the Gurobi Optimizer

Transcription

1 Recent Developments in the Gurobi Optimizer

2 Overview Introduction (10 minutes) Ed Rothberg Gurobi 7.5 and 8.0 (15 minutes) Ed Rothberg Improvements in Compute Server and Instant Cloud (30 minutes) Michel Jaczynski MIP is not an Algorithm (30 minutes) Ed Rothberg Performance Improvements (10 minutes) Ed Rothberg Wrap-up and Q&A Copyright 2018, Gurobi Optimization, LLC 2

3 A Few Facts Founded July 2008 Zonghao Gu, Ed Rothberg, Bob Bixby Focused exclusively on math programming No competing priorities The Gurobi team 27 strong ~50% development and support ~50% have PhD s History of continuing, significant performance improvements ~2X average improvement with each major release 92x cumulative improvement since first version in 2009 Copyright 2018, Gurobi Optimization, LLC 3

4 Recent News Copyright 2018, Gurobi Optimization, LLC 4

5 What Does It Mean? No change in focus Optimization is our sole focus Broader community is taking notice of Operations Research More resources Additional employees Greater investment in R&D More outreach to people who aren t aware of the benefits of optimization Advice from business experts Company better able to push the market and the technology forward Copyright 2018, Gurobi Optimization, LLC 5

6 New Gurobi team members in 2018 Miranda Gould Licensing Admin KJ Gundersen VP of Finance Robert Luce Developer Duke Perrucci Chief Revenue Officer Mark Reynaud US Central Sales Pano Santos Sr. Technical Content Manager Isela Warner Director of Marketing Copyright 2018, Gurobi Optimization, LLC Yuriy Zinchenko Support Engineer

7 Why Do Companies Choose Gurobi? Technology Leadership The most experienced development team focused 100% on delivering the best solver and user experience Outstanding Support Direct access to PhD-level optimization experts for fast answers to your questions No Surprises Flexible licensing and straightforward pricing to meet a full range of business situations Copyright 2018, Gurobi Optimization, LLC 7

8 Over 1,600 companies have made Gurobi their solver of choice

9 Strong track record of innovation Commercial First cloud offering Client-server support Distributed algorithms Academic Free academic licenses Free site licenses Free online-course licenses Free distributed optimization Take Gurobi with You program Copyright 2018, Gurobi Optimization, LLC 9

10 Gurobi Products Overview Gurobi Solvers Mixed-Integer Programming (MIP, MIQP, MIQCP) Deterministic, parallel Linear and Quadratic Programming Dual and primal simplex; Parallel Barrier Piecewise-linear simplex Second-Order Cone Programming Parallel Barrier APIs Simple command-line interface Python interactive interface Python modeling interface C, C++, Java,.NET, Python programming interfaces R and MATLAB matrix interfaces All standard modeling languages Extensions Piecewise-linear objectives General constraints Multi-objective optimization Commercial and Academic Licenses Licensing options and pricing available at Gurobi Compute Server Cloud offering Gurobi Instant Cloud Gurobi Cloud for AWS Perpetual licenses cloud deployable Copyright 2018, Gurobi Optimization, LLC 10

11 Gurobi 7.5 and 8.0 Dr. Edward Rothberg Copyright 2018, Gurobi Optimization, LLC 11

12 What's New? New features in Gurobi 7.5 (highlights) Improved Python interface Multi-objective improvements JavaDoc documentation New features in Gurobi 8.0 Significant enhancements to Compute Server and Instant Cloud Enhanced MATLAB and R interfaces Support for MATLAB problem-based modeling Partition heuristic Multiple MIP starts Support for.net Core 2.0 Performance improvements Copyright 2018, Gurobi Optimization, LLC 12

13 Python Modeling Enhancements Copyright 2018, Gurobi Optimization, LLC 13

14 Python Modeling With Gurobi Gurobi Python modeling interface provides the best of both worlds Convenient, expressive mathematical modeling constructs Within a widely-used, well-designed, well-supported programming language Simple constructs for creating variables, building linear expressions, adding constraints, etc. m.addconstrs(outflow[city] == transport.sum(city, '*') for city in Cities)

15 Supporting More Constraint Types More concise constraint declarations Old: model.addrange( x+y, 1.0, 2.0, c2 ) model.addgenconstrindicator( x, True, y+z == 1.0, c3 ) model.addgenconstrmax( x, [y,z], 2.0, c4 ) New: model.addconstr( x+y == [1.0,2.0], c2 ) model.addconstr((x == 1) >> (y+z == 1.0), c3 ) model.addconstr( x == max_(y,z,2.0), c4 ) Can be used in model.addconstrs() as well Copyright 2018, Gurobi Optimization, LLC 15

16 Enhanced Multi-Objective API Copyright 2018, Gurobi Optimization, LLC 16

17 Enhanced Multi-Objective API Easier definition of multiple objectives We introduced model.setobjectiven(linexpr, index,...) Old (Gurobi 7.0): coef1 = [0, 1, 2, 3, 4] coef2 = [4, 3, 2, 1, 0] x = model.addvars(5) model.numobj = 2 model.params.objnumber = 0 model.objnpriority = 2 model.objnweight = 1 model.setattr(grb.attr.objn, x, coef1) model.params.objnumber = 1 model.objnpriority = 1 model.objnweight = 1 model.setattr(grb.attr.objn, x, coef2) New (Gurobi 7.5): x = model.addvars(5) model.setobjectiven(x[1]+2*x[2]+3*x[3]+4*x[4], 0, priority=2, weight=1) model.setobjectiven(4*x[0]+3*x[1]+2*x[2]+x[3], 1, priority=1, weight=1) Copyright 2018, Gurobi Optimization, LLC 17

18 Enhanced MATLAB and R interfaces Copyright 2018, Gurobi Optimization, LLC 18

19 Completing the MATLAB and R Interfaces Previously missing features: Multi-objective optimization General constraints Several other (smaller) features Now available in MATLAB and R Also now available: Full set of Gurobi examples for MATLAB and R Copyright 2018, Gurobi Optimization, LLC 19

20 MATLAB Problem-Based Modeling Problem-based modeling: new feature in MATLAB 2017b Optimization Toolbox A more structured approach to building an optimization model x = optimvar('x', 'Type','integer','LowerBound',0,'UpperBound',1); y = optimvar('y', 'Type','integer','LowerBound',0,'UpperBound',1); z = optimvar('z', 'Type','integer','LowerBound',0,'UpperBound',1); prob = optimproblem('objectivesense','maximize'); prob.objective = x + y + 2 * z; prob.constraints.cons1 = x + 2 * y + 3 * z <= 4; prob.constraints.cons2 = x + y >= 1; sol = solve(prob, options) You can now call Gurobi through the MATLAB solve() function Copyright 2018, Gurobi Optimization, LLC 20

21 Partition Heuristic Copyright 2018, Gurobi Optimization, LLC 21

22 User-Specified Local Improvement Heuristic RINS is our most effective heuristic It is a sub-mip heuristic: Fix a subset of the variables to incumbent values Solve the resulting MIP (recursively) Reoptimizes over just that portion of the problem Sub-MIP heuristics extremely effective in general How to choose the sub-problem to reoptimize? RINS chooses automatically New feature allows user to make the choice Example sub-problems: All decisions related to a single time period All decisions related to a single machine All decisions related to physical sub-regions (e.g., Western US, Eastern US, etc.) Copyright 2018, Gurobi Optimization, LLC 22

23 Multiple MIP Starts Copyright 2018, Gurobi Optimization, LLC 23

24 Multiple MIP Starts User can now provide multiple MIP starts Useful when you have multiple partial solutions MIP solver will try to complete them, and will store the ones it finds For distributed MIP, MIP starts will be evaluated on different machines Copyright 2018, Gurobi Optimization, LLC 24

25 .NET Core 2.0 Copyright 2018, Gurobi Optimization, LLC 25

26 .NET Core 2.0 Gurobi users can now write.net programs that run on Windows, Linux, and Mac Using new Microsoft.NET Core 2.0 Copyright 2018, Gurobi Optimization, LLC 26

27 Compute Server and Instant Cloud Dr. Michel Jaczynski Copyright 2018, Gurobi Optimization, LLC 27

28 Compute Server Overview Client Queuing Load Balancing Remote Services Seamlessly offload optimization jobs to one or more servers Common use cases: Access more powerful machines Share resources across users, teams and applications Ensure high-availability with multiple nodes Build modern microservice architectures 2018 Gurobi Optimization

29 Improved Compute Server Architecture Client Remote Services (grb_rs) Remote Services (grb_rs) HTTP/HTTPS Remote Services (grb_rs) Join/leave Communication using HTTP or secured HTTPS Easier integration with firewalls and proxies Standard security and data encryption Better robustness to handle intermittent network issues Remote Services (grb_rs) Clustering Nodes can join or leave the cluster dynamically Jobs are load-balanced on the different nodes Client can connect to any node in the cluster 2018 Gurobi Optimization

30 Setting up the Cluster New Packaging New separated server package (grb_rs and server tools) New Remote Services Reference Manual Configure nodes Set configuration properties in grb_rs.cnf file Configure protocol (HTTP, HTTPS) and port (80, 443, or other) How to get help? $ grb_rs --help $ grb_rs properties Start the Remote Services Agent (grb_rs) as a service (Linux, Mac, Windows) $ grb_rs start 2018 Gurobi Optimization

31 New Command Line Tool grbcluster (1/3) Getting help? $ grbcluster --help $ grbcluster <command> --help Submit jobs using libraries or gurobi_cl $ gurobi_cl --server=server1 glass4.mps Compute Server job ID: d933fff7-ac0b-41a c4d92b4395 Capacity available on server1' - connecting... Established HTTP unencrypted connection [ ] List cluster status: grbcluster nodes $ grbcluster --server=server1 nodes ADDRESS STATUS TYPE LICENSE #Q #R JL IDLE %MEM %CPU server1 ALIVE COMPUTE VALID m server2 ALIVE COMPUTE VALID m List running and queued jobs: grbcluster jobs $ grbcluster --server=server1 jobs JOBID ADDRESS STATUS #Q STIME USER PRIO d933fff7 server1 RUNNING :40:37 user Gurobi Optimization

32 New Command Line Tool grbcluster (2/3) List recently processed jobs: grbcluster recent $ grbcluster --server=server1 recent JOBID ADDRESS STATUS STIME USER OPT API 8a8cd65e server1 COMPLETED :44:54 user1 OPTIMAL Python ecfabdeb server2 COMPLETED :05:25 user1 INFEASIBLE Java fe60d00b server1 IDLETIMEOUT :01:35 user1 OPTIMAL Python 977eb86d server2 DISCONNECTED :51:49 user1 Display job log: grbcluster log <JOBID> $ grbcluster --server=server1 log MIR: 3 Flow cover: 43 Explored nodes ( simplex iterations) in seconds Thread count was 2 (of 2 available processors) Solution count 10: e e e e+09 Optimal solution found (tolerance 1.00e-04) Display job parameters: grbcluster params <JOBID> $ gurobi_cl --server=server1 TimeLimit=120 glass4.mps $ grbcluster --server=server1 params e88a496f TimeLimit= Gurobi Optimization

33 New Command Line Tool grbcluster (3/3) List licenses: grbcluster licenses $ grbcluster --server=server1 --password=admin licenses ADDRESS STATUS TYPE KEY EXP ORG USER APP VER CS DL ERROR server1 VALID NODE Gurobi Inc. 8 true 100 server2 VALID NODE Gurobi Inc. 8 true 100 Abort jobs: grbcluster abort <JOBID> $ grbcluster --server=server1 --password=admin abort 8f9b15d9 $ grbcluster --server=server1 recent JOBID ADDRESS STATUS STIME USER 8f9b15d9 server1 ABORTED :41:33 user1 Change some configuration parameters: grbcluster config $ grbcluster --server=server1 --password=admin config --job-limit=4 $ grbcluster --server=server1 nodes ADDRESS STATUS TYPE LICENSE #Q #R JL IDLE %MEM %CPU server1 ALIVE COMPUTE VALID m server2 ALIVE COMPUTE VALID m Gurobi Optimization

34 New REST API Programmatically monitor and manage your cluster Monitor cluster nodes, jobs, licenses Abort jobs 2018 Gurobi Optimization

35 Instant Cloud Overview Automated Provisioning Cloud Manager Client Queuing Load Balancing Remote Services Dedicated Cloud Machines Seamlessly offload optimization jobs to the cloud Common use cases: Solve challenging models Spikes in demand Periodic optimization needs Cloud-based solution Cloud-based failover 2018 Gurobi Optimization

36 Instant Cloud Manager (cloud.gurobi.com) Download license files Customize machine Pools Region: N. Virginia, N. California, Frankfurt, Tokyo, Sydney Machine type, up to 32 cores and 244GB memory Number of servers Monitor machines and jobs 2018 Gurobi Optimization

37 Improved Architecture Automated Provisioning Cloud Manager Client Remote Services Dedicated Cloud Machines HTTPS Communication using secured HTTPS Easier integration with firewalls and proxies (no custom port to open) Standard security and data encryption (certificate validation, TLS encryption) Better robustness to handle intermittent network issues (retry and reconnect) 2018 Gurobi Optimization

38 New Job List 3 categories: Main jobs Worker jobs (sub-jobs submitted for distributed optimization or tuning) Queued job Jobs can be aborted Includes recently processed jobs (completed, failed, aborted) 2018 Gurobi Optimization

39 New Job Dashboard Access to model information, parameters, charts and log Gurobi Optimization

40 New Job History Optional feature, keep history over the last 90 days: Display Job status, metadata Download log file 2018 Gurobi Optimization

41 New Machine Usage Metrics Monitor machine CPU and memory usage 2018 Gurobi Optimization

42 New Pool Scaling Scaling up: Start additional machines that will take jobs from the queue Scaling down: Remove machines from the pool, drain existing jobs that are already running 2018 Gurobi Optimization

43 Extended REST API Programmatically monitor and manage your cloud machines New APIs for jobs and pool scaling 2018 Gurobi Optimization

44 MIP is not an Algorithm Dr. Ed Rothberg Copyright 2018, Gurobi Optimization, LLC 44

45 MIP is not an Algorithm MIP is typically thought of as an algorithm Relaxation-based branch-and-bound A limiting point of view Better to think of it as a declarative framework for stating optimization problems Backed by a rich mathematical foundation Linear programming, duality, polyhedral theory, etc. Allows for a variety of algorithms to be applied A giant bag of tricks Applied systematically and automatically Only when considered together do you get a robust and efficient method Copyright 2018, Gurobi Optimization, LLC 45

46 Customer Applications of MIP ( ) Accounting Advertising Agriculture Airlines ATM provisioning Compilers Defense Electrical power Energy Finance Food service Forestry Gas distribution Government Internet applications Logistics/supply chain Medical Mining National research labs Online dating Portfolio management Railways Recycling Revenue management Semiconductor Shipping Social networking Sourcing Sports betting Sports scheduling Statistics Steel Manufacturing Telecommunications Transportation Utilities Workforce Management Copyright 2018, Gurobi Optimization, LLC 46

47 Expressiveness Versus Tractability

48 Problem and Algorithms Trade-off between expressiveness and tractability Efficient algorithms for very specific problems Tractability Expressiveness State problems you can t solve Copyright 2018, Gurobi Optimization, LLC 48

49 A Few Examples Graph algorithms Regression Linear systems of equations Tractability SAT Expressiveness Constraint programming Non-linear programming Copyright 2018, Gurobi Optimization, LLC 49

50 A Few Examples Graph algorithms Regression Linear systems of equations Tractability SAT MIP Expressiveness Constraint programming Non-linear programming Copyright 2018, Gurobi Optimization, LLC 50

51 Important Properties of MIP Declarative No need to rewrite algorithm to accommodate additional constraints Expressive Adding side constraints typically leaves the problem type unchanged Regular problem structure (Ax=b) Allows for fairly simple recognition of useful/special structure Copyright 2018, Gurobi Optimization, LLC 51

52 MIP Improvements and Where They Come From

53 MIP Improvements in Gurobi 7.5 (by Category) Presolve - 7.8% 8 improvements, largest 2.0% Node presolve - 4.2% 5 improvements, largest 1.3% Symmetry - 7.0% 3 improvements, largest 3.1% LP % Many small improvements Cutting planes % 6 improvements, largest 6.3% Branching - 2.8% Heuristics - 4.6% 5 improvements, largest 1.4% Copyright 2018, Gurobi Optimization, LLC 53

54 Opening the Bag of Tricks MIP draws from many different disciplines A few examples: Graph algorithms: Bi-connected components Number theory Modular multiplicative inverse Meta-heuristics Relaxation Induced Neighborhood Search (RINS) Copyright 2018, Gurobi Optimization, LLC 54

55 Bi-Connected Components

56 Disconnected Components A = As a single model, runtime grows as #nodes n for n components When split, runtime grows as #nodes*n Copyright 2018, Gurobi Optimization, LLC 56

57 Nearly Disconnected Components A = Identify variables that disconnect the model to create disconnect components Copyright 2018, Gurobi Optimization, LLC 57

58 Identifying Variables that Disconnect the Model Problem is equivalent to finding a vertex separator in the intersection graph of A Intersection graph: One node for each variable An edge (i,j) whenever variables i and j ever appear in the same constraint Vertex separator A set of vertices whose removal disconnects the graph Copyright 2018, Gurobi Optimization, LLC 58

59 Intersection Graph V1 V2 S No edges between vertices in V1 and V2 implies No pair of variables in V1 and V2 participate in the same constraint Copyright 2018, Gurobi Optimization, LLC 59

60 Bi-Connected Components Articulation point Identify a single vertex that disconnects the graph Algorithm by Lipton and Tarjan Linear time in # of edges in graph Identifies all vertices that disconnect graph V1 V2 S Copyright 2018, Gurobi Optimization, LLC 60

61 Bi-Connected Components Articulation point If articulation point is a binary variable Fix variable to 1 solve remaining pieces independently Fix variable to 0 Exploits bi-connected structure to create disconnected components Often much faster than solving whole problem V1 V2 S Copyright 2018, Gurobi Optimization, LLC 61

62 Number Theory Copyright 2018, Gurobi Optimization, LLC

63 Modular Inverse Reduction Consider a x + b y = c (possibly after aggregation) x, y are integer variables a, b and c are integers, a > 1 Assume GCD(a,b) = 1 Divide through by GCD otherwise Compute modular multiplicative inverse of a Integer m such that a m = 1 (mod b) Computed using extended Euclidean algorithm Transformation a x + b y = c m a x + m b y = m c x = m c (mod b) Copyright 2018, Gurobi Optimization, LLC 63

64 Modular Inverse Reduction - Example 5 x x x x 4 = 31 (all integer variables) Substitute y = x x x 4 5 y + 3 x 3 = 31 Modular inverse of 3 mod 5 is 2 (2 * 3 = 1 (mod 5)) 10 y + 6 x 3 = 62 (mod 5) x 3 = 2 (mod 5) x 3 = { 2, 7, 12, 17, } Exploiting this observation In MIP search tree, if x 3 = 3.5 in relaxation Normal branch: x <= 3 or x >= 4 Better branch: x <= 2 or x >= 7 Impact Runtime drops from s to 0.01s for one family of models Often a significant win when it can be applied Copyright 2018, Gurobi Optimization, LLC 64

65 Meta-Heuristics Copyright 2018, Gurobi Optimization, LLC

66 Sub-MIP Heuristics Meta-heuristics play a big role in MIP Basic structure: local search through sub-mips Fix a set of variables Solve a MIP on the remainder Recursive (sub-mips explored in remainder model too) A form of Very Large Neighborhood Search (VLNS) Population-based meta-heuristics MIP tree search naturally finds multiple solutions Can use approaches that combine solutions Most effective meta-heuristic in MIP: Relaxation Induced Neighborhood Search (RINS) Copyright 2018, Gurobi Optimization, LLC 66

67 Relaxation Induced Neighborhood Search A population of two Current incumbent solution integer feasible but (probably) not optimal Current relaxation solution optimal but not integer feasible Simple sub-mip Fix variables that agree in two solutions Diversified Different relaxation at each branch-and-bound node Adaptive Neighborhood is large when optimality gap is large, small when gap is small Often the best approach to find a good solution is Find a bad solution and improve it Copyright 2018, Gurobi Optimization, LLC 67

68 Meta-meta-heuristics Feed domain-specific heuristic solutions to MIP solver MIP solver works to improve those solutions Systematic exploration of space, using cutoff from provided solution Additional input to MIP meta-heuristics Copyright 2018, Gurobi Optimization, LLC 68

69 Conclusions Copyright 2018, Gurobi Optimization, LLC

70 Always Try MIP Copyright 2018, Gurobi Optimization, LLC 70

71 Gurobi 8.0 Performance Improvements Dr. Ed Rothberg Copyright 2018, Gurobi Optimization, LLC 71

72 Two Kinds of Benchmarks Internal benchmarks Most important: compare Gurobi version-over-version Based on internal library of 4538 models External, competitive benchmarks Conducted by Hans Mittelmann, Arizona State University For MIP largely based upon MIPLIB 2010 Copyright 2018, Gurobi Optimization, LLC 72

73 Internal Benchmarks Copyright 2018, Gurobi Optimization, LLC 73

74 Gurobi MIP Library 1E+09 (4538 models) ,000,000 Columns , Rows Copyright 2018, Gurobi Optimization, LLC 74

75 Performance Improvements: 7.5 to 8.0 Problem Class >1s >100s # Wins Losses Speedup # Wins Losses Speedup LP: concur x x primal x x dual x x barrier x x MIP x x MIQP x x MIQCP x x Gurobi 7.5 vs. 8.0: > 1.00x means that Gurobi 8.0 is faster than Gurobi 7.5 Copyright 2018, Gurobi Optimization, LLC 75

76 Performance Improvements: 7.0 to 8.0 Problem Class >1s >100s # Wins Losses Speedup # Wins Losses Speedup LP: concur x x primal x x dual x x barrier x x MIP x x MIQP x x MIQCP x x Gurobi 7.0 vs. 8.0: > 1.00x means that Gurobi 8.0 is faster than Gurobi 7.0 Copyright 2018, Gurobi Optimization, LLC 76

77 Continual Performance Improvements Nearly a 2x average improvement per major release 3.00 V-V Speedup Cumulative Speedup 90.0 ~92x improvement (8+ years) Version-to-Version Speedup Cummulative Speedup > > > > > > > 8.0* - Time limit: sec. Intel Xeon CPU E GHz 4 cores, 8 hyper-threads 32 GB RAM Gurobi Version-to-Version Pairs MIP test set has 3740 models: discarded due to inconsistent answers discarded that none of the versions can solve - speed-up measured on >100s bracket: 1227 models Copyright 2018, Gurobi Optimization, LLC 77

78 Wrap-Up and Q&A Copyright 2018, Gurobi Optimization, LLC 78

79 While You re At INFORMS Software Tutorial: Gurobi Compute Server and Instant Cloud Dr. Michel Jaczynski Monday, 11:30am 12:20pm Falkland Drop by the Gurobi Booth #28 at the Exhibit Hall Copyright 2018, Gurobi Optimization, LLC 79

80 We're Hiring Support Engineer Worldwide Stop by our booth or visit our website for more information Copyright 2018, Gurobi Optimization, LLC 80

81 Thank You Questions?