GLOBAL OPTIMIZATION WITH BRANCH-AND-REDUCE

Transcription

1 GLOBAL OPTIMIZATION WITH BRANCH-AND-REDUCE Nick Sahinidis Department of Chemical Engineering Carnegie Mellon University EWO seminar, 23 October 27

2 THE MULTIPLE-MINIMA DIFFICULTY IN OPTIMIZATION f Classical optimality conditions are necessary but not sufficient Classical optimization provides the local minimum closest to the starting point used 2

3 COMMON FUNCTIONS IN MODELING 3

4 COMMON FUNCTIONS IN MODELING 4

5 AUTOMOTIVE REFRIGERANT DESIGN (Joback and Stephanopoulos, 99) Higher enthalpy of vaporization (ΔH ve ) reduces the amount of refrigerant Lower liquid heat capacity (C pla ) reduces amount of vapor generated in epansion valve Maimize ΔH ve / C pla, subject to: ΔH ve 8.4, C pla

6 FUNCTIONAL GROUPS CONSIDERED 6

7 PROPERTY PREDICTION 7

8 BRANCH-AND-BOUND 8

9 MOLECULAR DESIGN AFTER 5 CPU HOURS IN 995 One feasible solution identified Optimality not proved First attempt: IBM RS/6 43P with 28 MB RAM Second attempt: IBM SP/2 Single Processor with 2 GB RAM 9

10 MOLECULAR DESIGN IN 2 In 3 CPU minutes

11 BREAST CANCER DIAGNOSIS 2, cases diagnosed in the U.S. a year 4, deaths a year Most breast cancers are first diagnosed by the patient as a lump in the breast Majority of breast lumps are benign Available diagnosis methods: Mammography (68% to 79% correct) Surgical biopsy (% correct but invasive and costly) Fine needle aspirate (FNA)» With visual inspection: 65% to 98% correct» Automated diagnosis: 95% correct Linear programming techniques Mangasarian and Wolberg in 99s

12 WISCONSIN DIAGNOSTIC BREAST CANCER (WDBC) DATABASE From Wolberg, Street, & Mangasarian, patients 9 cytological characteristics: Clump thickness Uniformity of cell size Uniformity of cell shape Marginal adhesion Single epithelial cell size Bare nuclei Bland chromatin Normal nucleoli Mitoses Biopsy classified these 653 patients in two classes: Benign Malignant 2

13 BILINEAR (IN-)SEPARABILITY OF TWO SETS IN R n Requires the solution of three nonconve bilinear programs 3

14 CHALLENGES IN GLOBAL OPTIMIZATION min s.t. f (, y) g(, y) n R, y Z p f (, y) f (, y) f (, y) Multimodal objective Integrality conditions Nonconve constraints NP-HARD PROBLEM 4

15 GLOBAL OPTIMIZATION ALGORITHMS Stochastic and deterministic algorithms Branch-and-Bound Bound problem over successively refined partitions» Falk and Soland, 969» McCormick, 976 Conveification Outer-approimate with increasingly tighter conve programs Tuy, 964 Sherali and Adams, 994 Horst and Tuy, Global Optimization: Deterministic Approaches, 996 Over 8 citations Our approach Branch-and-Reduce» Ryoo and Sahinidis, 995, 996» Shectman and Sahinidis, 998 Constraint Propagation & Duality-Based Reduction» Ryoo and Sahinidis, 995, 996» Tawarmalani and Sahinidis, 22 Conveification» Tawarmalani and Sahinidis, 2, 22, 24, 25 Tawarmalani and Sahinidis, Conveification and Global Optimization in Continuous and Mied-Integer Nonlinear Programming, 22 5

16 6 BOUNDING SEPARABLE PROGRAMS s.t. min = = f

17 7 BOUNDING SEPARABLE PROGRAMS s.t. min = = f

18 BOUNDING FACTORABLE PROGRAMS Introduce variables for intermediate quantities whose envelopes are not known 8

19 TIGHT RELAXATIONS f () f () Concave over-estimator Concave envelope Conve under-estimator Conve envelope f () Conve/concave envelopes often finitely generated 9

20 RATIO: TRADITIONAL RELAXATION 2

21 RATIO: THE GENERATING SET 2

22 DIFFERENCE BETWEEN ENVELOPE AND TRADITIONAL RELAXATION Traditional 22

23 ENVELOPES OF MULTILINEAR FUNCTIONS Multilinear function over a bo M pt = t t i= (,..., ) a, < L U < +, i =, K, n n i i i i Generating set n vert [ Li, U i= Polyhedral conve encloser follows trivially from polyhedral representation theorems i ] 23

24 POLYHEDRAL OUTER-APPROXIMATION Local NLP solvers essential for local search Linear programs can be solved very efficiently Outer-approimate conve relaation by polyhedron Tawarmalani and Sahinidis (Math. Progr., 24, 25) Quadratically convergent sandwich algorithm Cutting planes for functional compositions 24

25 RECURSIVE FUNCTIONAL COMPOSITIONS Consider h=g(f), where g and f are multivariate conve functions g is non-decreasing in the range of each nonlinear component of f h is conve Two outer approimations of the composite function h: S: a single-step procedure that constructs supporting hyperplanes of h at a predetermined number of points S2: a two-step procedure that constructs supporting hyperplanes for g and f at corresponding points Two-step is sharper than one-step If f is affine, S2=S In general, the inclusion is strict 25

26 OUTER APPROXIMATION OF 2 +y

27 MARGINALS-BASED RANGE REDUCTION Relaed Value Function z U L L U If a variable goes to its upper bound at the relaed problem solution, this variable s lower bound can be improved 27

28 REDUCTION VIA CONSTRAINT PROPAGATION a. b. c. d. e. f. 28

29 FINITE VERSUS CONVERGENT BRANCH-AND-BOUND ALGORITHMS Finite sequences A potentially infinite sequence 29

30 FINITE BRANCHING RULE f() * Variable selection: Typically, select variable with largest underestimating gap Occasionally, select variable corresponding to largest edge Point selection: Typically, at the midpoint (ehaustiveness) When possible, at the best currently known solution Finite isolation of global optimum Finite termination in many cases Concave minimization over polytopes 2-Stage stochastic integer programming 3

31 BRANCH-AND-REDUCE START Multistart search and reduction Nodes? Y Select Node N STOP Preprocess Lower Bound Feasibility-based reduction Inferior? N Upper Bound Y Delete Node Postprocess Optimality-based reduction Y Reduced? N Branch 3

32 Branch-And-Reduce Optimization Navigator Components Modeling language Preprocessor Data organizer I/O handler Range reduction Solver links Interval arithmetic Sparse matri routines Automatic differentiator IEEE eception handler Debugging facilities Capabilities Core module Application-independent Epandable Fully automated MINLP solver Application modules Multiplicative programs Indefinite QPs Fied-charge programs Mied-integer SDPs Solve relaations using CPLEX, MINOS, SNOPT, OSL, SDPA, Available under GAMS and AIMMS Available on NEOS server 32

33 26 PROBLEMS FROM globallib AND minlplib Constraints Variables Discrete variables Minimum 2 4 Without cuts Maimum With cuts Average 5 EFFECT OF CUTTING PLANES % reduction Nodes 23,3, , Nodes in memory 622,339 3, CPU hrs

34 POOLING PROBLEM: p-formulation 34

35 POOLING PROBLEM: q-formulation 35

36 POOLING PROBLEM: pq-formulation 36

37 37 PRODUCT DISAGGREGATION Consider the function: Let Then = = = n k k k n k k k n y b b y a a y y ),, ; φ( K ], [ ], [ U k L k n k U L y y H Π = = = = = n k k k y y n k k k H y b b y a a U L U k L k ], [ ], [ ) ( convenv convenv φ Disaggregated formulations are tighter

38 38 LOCAL SEARCH WITH CONOPT Infeasible rt97-75 haverly3-4 haverly2-4 haverly foulds foulds foulds3-6 - foulds bental5 bental adhya adhya3 adhya adhya pq-formulation objective q-formulation objective Problem

39 39 GLOBAL SEARCH WITH BARON rt97 3 haverly3 7 haverly2 25 haverly - >2 >389 foulds5 - >2 >326 foulds4 5 - >2 >348 foulds3-6 6 foulds2 - >2 >6445 bental5.5.5 bental4 >2 >629 adhya4.5 3 >2 >9248 adhya adhya adhya CPU sec Nodes CPU sec Nodes pq-formulation p-formulation Problem

40 ONGOING DEVELOPMENT OF BARON Structural Bioinformatics Systems biology X-ray imaging Portfolio optimization U E(r) 4

41 BARON IN APPLICATIONS Development of new Runge-Kutta methods for partial differential equations Ruuth and Spiteri, SIAM J. Numerical Analysis, 24 Energy policy making Manne and Barreto, Energy Economics, 24 Design of metabolic pathways Grossmann, Domach and others, Computers & Chemical Engineering, 25 Model estimation for automatic control Bemporand and Ljung, Automatica, 24 Agricultural economics Cabrini et al., Manufacturing and Service Operations Management, 25 4

42 GLOBAL/MINLP SOFTWARE AlphaECP Eploits pseudoconveity BARON Branch-And-Reduce BONMIN Integer programming technology (CMU/IBM) DICOPT Decomposition GlobSol Interval arithmetic Interval Solver (Frontline) Interval solver; Ecel LaGO Lagrangian relaations (COIN/OR) LGO Stochastic search; black-bo optimization LINGO Trigonometric functions; IF-THEN-ELSE; MSNLP, OQNLP Stochastic search SBB Simple branch-and-bound NLP/MINLP NLP MINLP 42

43 COMPARISONS ON MINLPLIB BARON 43

44 GAMS SALES Commercial and academic users % 9% 8% 7% 6% 5% 4% 3% 2% % % Global (BARON, LGO, MSNLP, OQNLP) MINLP (DICOPT, SBB) Local NLP (CONOPT, KNITRO, MINOS, PATH, SNOPT) Data courtesy of Ale Meeraus 44

45 Range Reduction Finiteness Conveification * BRANCH-AND-REDUCE Engineering design Management and Finance Chem-, Bio-, Medical Informatics 45