The Branch-and-Sandwich Algorithm for Mixed-Integer Nonlinear Bilevel Problems

Transcription

1 The Branch-and-Sandwich Algorithm for Mixed-Integer Nonlinear Bilevel Problems Polyxeni-M. Kleniati and Claire S. Adjiman MINLP Workshop June 2, 204, CMU Funding Bodies EPSRC & LEVERHULME TRUST

2 OUTLINE INTRODUCTION 2 PROPOSED METHOD 3 BOUNDING PROBLEMS: INITIAL NODE 4 BRANCHING & BOUNDING ON SUBDOMAINS 5 NUMERICAL RESULTS 6 CONCLUSIONS IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 2 / 9

3 OUTLINE INTRODUCTION 2 PROPOSED METHOD 3 BOUNDING PROBLEMS: INITIAL NODE 4 BRANCHING & BOUNDING ON SUBDOMAINS 5 NUMERICAL RESULTS 6 CONCLUSIONS IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 2 / 9

4 BILEVEL OPTIMISATION PROBLEM PROBLEM FORMULATION A two-person, non-cooperative game in which the play is sequential The Mixed-Integer Nonlinear Bilevel Problem is min x i,x c,y i,y c F(x i, x c, y i, y c) s.t. G(x i, x c, y i, y c) 0 (y i, y c) arg min y i Y I,y c Y C {f (x i, x c, y i, y c) s.t. g(x i, x c, y i, y c) 0} x i X I ZZ n, x c X C IR n n y i Y I ZZ m, y c Y C IR m m (MINBP) Subscripts i and c stand for integer and continuous, respectively Typical assumptions apply, such as continuity of all functions and compactness of the host sets Assume also twice differentiability of the continuous relaxations of the functions No special class or convexity assumptions are made IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 3 / 9

5 OPTIMAL VALUE EQUIVALENT REFORMULATION Define the inner optimal value function w(x i, x c) = min y i,y c {f (x i, x c, y i, y c) s.t. g(x i, x c, y i, y c) 0, y i Y I, y c Y C} Then problem (e.g. Dempe and Zemkoho, 20): min x i,x c,y i,y c F(x i, x c, y i, y c) s.t. G(x i, x c, y i, y c) 0 g(x i, x c, y i, y c) 0 f (x i, x c, y i, y c) w(x i, x c) x i X I, x c X C y i Y I, y c Y C is equivalent to MINBP without any convexity assumptions whatsoever USEFUL PROPERTY A restriction of the inner problem yields a relaxation of the overall problem and vice versa (Mitsos and Barton, 2006) IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 4 / 9

6 OPTIMAL VALUE EQUIVALENT REFORMULATION Define the inner optimal value function w(x i, x c) = min y i,y c {f (x i, x c, y i, y c) s.t. g(x i, x c, y i, y c) 0, y i Y I, y c Y C} Then problem (e.g. Dempe and Zemkoho, 20): min x i,x c,y i,y c F(x i, x c, y i, y c) s.t. G(x i, x c, y i, y c) 0 g(x i, x c, y i, y c) 0 f (x i, x c, y i, y c) w(x i, x c) x i X I, x c X C y i Y I, y c Y C is equivalent to MINBP without any convexity assumptions whatsoever USEFUL PROPERTY A restriction of the inner problem yields a relaxation of the overall problem and vice versa (Mitsos and Barton, 2006) IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 4 / 9

7 PROPOSED APPROACHES FOR DISCRETE BILEVEL PROGRAMMING INFLUENTIAL WORKS BY BARD ET AL., DEMPE ET AL. AND VICENTE ET AL., e.g. The Mixed Integer Linear Bilevel Programming Problem, Moore & Bard (990) 2 An Algorithm for the Mixed-Integer Nonlinear Bilevel Program. Prob., Edmunds & Bard (992) 3 The Discrete Linear Bilevel Programming Problem, Vicente, Savard & Judice (996) 4 Practical Bilevel Optimization, Bard (998) 5 Foundations of Bilevel Programming, Dempe (2002) 6 Discrete Bilevel Programming, Dempe, Kalashnikov & Ríos-Mercado (2005) 7 Bilevel programming with discrete lower level problems, Fanghänel & Dempe (2009) ADVANCES ON MORE GENERAL CLASSES Global Optimization of Mixed-Integer Bilevel Program. Prob., Gümüş and Floudas (2005) 2 Global Solution of Nonlinear Mixed-Integer Bilevel Programs, Mitsos (200) IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 5 / 9

8 CHALLENGES : ILLUSTRATIVE EXAMPLE (MOORE AND BARD, 990) min x [0,8] s.t. x 0y x ZZ y arg min{y s.t. y Y(x), y ZZ} y [0,4] 4 25x + 20y 30 Y(x) : Inner Feasible Region 3 x + 2y 0 y 2 (2, 2) (2, 22) (f(x,y),f(x,y)) 2x 0y 5 (, 3) (, 4) (, 5) (, 6) (, 7) (, 8) 2x y x IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

9 CHALLENGES : ILLUSTRATIVE EXAMPLE (MOORE AND BARD, 990) min x [0,8] s.t. x 0y x ZZ y arg min{y s.t. y Y(x), y ZZ} y [0,4] 4 25x + 20y 30 Y(x) : Inner Feasible Region 3 x + 2y 0 y 2 (2, 2) (2, 22) (f(x,y),f(x,y)) 2x 0y 5 (, 3) (, 4) (, 5) (, 6) (, 7) (, 8) 2x y x IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

10 CHALLENGE : CONSTRUCTING A CONVERGENT LOWER BOUNDING PROBLEM min x,y s.t. x 0y y Y(x) x [0, 8], y [0, 4] x, y ZZ 4 25x + 20y 30 (4, 42) Y(x) : Inner Feasible Region 3 (3, 32) (3, 33) x + 2y 0 (3, 34) y 2 (2, 2) (2, 22) (2, 23) (2, 24) (2, 25) (2, 26) (f(x,y),f(x,y)) 2x 0y 5 (, 3) (, 4) (, 5) (, 6) (, 7) (, 8) 2x y x IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

11 CHALLENGE 2 : PARTITIONING THE INNER REGION min x [0,8] s.t. x 0y x ZZ y arg min{y s.t. y Y(x), y ZZ} y [0,] min x [0,8] s.t. x 0y x ZZ y arg min{y s.t. y Y(x), y ZZ} y [2,4] 4 25x + 20y 30 Y(x) : Inner Feasible Region 3 x + 2y 0 y 2 (2, 2) (2, 22) (2, 23) (2, 24) (2, 25) (2, 26) y 2 (f(x,y),f(x,y)) y (, 3) (, 4) (, 5) (, 6) (, 7) (, 8) 2x 0y 5 2x y x IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

12 OUTLINE INTRODUCTION 2 PROPOSED METHOD 3 BOUNDING PROBLEMS: INITIAL NODE 4 BRANCHING & BOUNDING ON SUBDOMAINS 5 NUMERICAL RESULTS 6 CONCLUSIONS IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

13 BRANCH-AND-SANDWICH ALGORITHM,2 CONNECTION TO EARLIER WORK A deterministic global optimisation algorithm for bilevel programs with twice continuously differentiable functions 2 continuous decision variables 3 nonconvex inner problem satisfying regularity The Branch-and-Sandwich algorithm was proved to be ε-convergent based on exhaustiveness and the general convergence theory (Horst and Tuy, 996) Branch-and-Sandwich was tested on 33 small problems with promising numerical results Kleniati, P. M. and Adjiman, C. S., 20, Proceedings of the 2 st European Symposium on Computer-Aided Process Engineering, Computer-Aided Chemical Engineering, 29, , 204, Parts I & II, JOGO, DOI 0.007/s ; DOI 0.007/s IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 7 / 9

14 WE EXTEND THE BRANCH-AND-SANDWICH ALGORITHM TO TACKLE MIXED-INTEGER NONLINEAR BILEVEL PROBLEMS CONVERGENT LOWER BOUNDING PROBLEM Employ the optimal value reformulation and replace the inner optimal value function w(x i, x c) by a constant upper bound the relaxed problem has one constraint (cut) only the right hand side of the cut is updated when appropriate PARTITIONING THE INNER REGION Branching scheme that allows branching on the inner variables consider all inner subregions where (inner) global optima may lie SUMMARY OF FEATURES Generation of two sets of upper and lower bounds for the inner and outer objective values the resulting bounding problems are MINLPs Tree management with auxiliary lists of nodes as well as inner and outer fathoming rules exploration of two decision spaces using a single branch-and-bound tree IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 8 / 9

15 OUTLINE INTRODUCTION 2 PROPOSED METHOD 3 BOUNDING PROBLEMS: INITIAL NODE 4 BRANCHING & BOUNDING ON SUBDOMAINS 5 NUMERICAL RESULTS 6 CONCLUSIONS IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 8 / 9

16 INNER PROBLEM BOUNDING SCHEME INNER LOWER BOUND CONSIDER NO BRANCHING YET The inner bounding scheme is based on finding valid lower and upper bounds on w(x i, x c) for all values of the outer vector variable (x i, x c) The inner lower bounding problem is f L = min x i,x c,y i,y c f (x i, x c, y i, y c) s.t. g(x i, x c, y i, y c) 0 x i X I, x c X C, y i Y I, y c Y C A convex relaxation can be derived for problem class considered use of techniques suitable for MINLPs, e.g. Adjiman et al. (2000), Tawarmalani and Sahinidis (2004) f = min f (xi, x x i,x c,y i,y c, y i, y c) c s.t. ğ(x i, x c, y i, y c) 0 x i X I, x c X C, y i Y I, y c Y C IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

17 INNER PROBLEM BOUNDING SCHEME INNER UPPER BOUND CONSIDER NO BRANCHING YET The inner bounding scheme is based on finding valid lower and upper bounds on w(x i, x c) for all values of the outer vector variable (x i, x c) The inner upper bounding problem is the robust counterpart of the inner problem f U = min y i,y c,t s.t. f (x i, x c, y i, y c) t (x i, x c) X I X C g(x i, x c, y i, y c) 0 (x i, x c) X I X C y i Y I, y c Y C t An upper bound can be derived by using SIP techniques, e.g. Bhattacharjee et al. (2005), & interval arithmetic extended to mixed-integer problems, e.g. Apt and Zoeteweij (2007), Berger and Granvilliers (2009) f = min y i,y c,t s.t. t f (XI, X C, y i, y c) t ḡ(x I, X C, y i, y c) 0 y i Y I, y c Y C IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 0 / 9

18 INNER PROBLEM BOUNDING SCHEME (CONT.) MOTIVATION : WE MUST HAVE VALID INNER BOUNDS OVER THE CURRENT SUBDOMAIN FIGURE: Inner lower bound FIGURE: Inner upper bound 6 5 x = 0 x = 0.5 x = Plot of f(x,y) = x 2 y + sin(y) 6 4 Plot of f(x,y) = y 2 /x x =, F * = 3 x = 2, F * = 6 x = 4, F * = f(x,y) (x,y) = (,) (x *,y * ) = (0,4.724) y f(x,y) (x *,y * ) = (,2) (x,y) = (2,2) y IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP / 9

19 OUTER PROBLEM BOUNDING SCHEME CONSIDER NO BRANCHING YET The proposed lower bounding problem is min x i,x c,y i,y c F(x i, x c, y i, y c) s.t. G(x i, x c, y i, y c) 0 g(x i, x c, y i, y c) 0 f (x i, x c, y i, y c) f x i X I, x c X C, y i Y I, y c Y C to tighten, add the inner KKT conditions with respect to the continuous inner variables y c based on regularity being satisfied for all the parameter values any feasible solution in the (MINBP) is feasible in the proposed relaxation For (x i, x c) = ( x i, x c), the upper bounding problem is (Mitsos et al., 2008) min F( x y i,y i, x c, y i, y c) c s.t. G( x i, x c, y i, y c) 0 g( x i, x c, y i, y c) 0 f ( x i, x c, y i, y c) w( x i, x c) + ε f y i Y I, y c Y C any feasible solution in the upper bounding problem is feasible in the (MINBP) set F UB := min{ F, } to express the incumbent IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 2 / 9

20 OUTER PROBLEM BOUNDING SCHEME CONSIDER NO BRANCHING YET The proposed lower bounding problem is min x i,x c,y i,y c F(x i, x c, y i, y c) s.t. G(x i, x c, y i, y c) 0 g(x i, x c, y i, y c) 0 f (x i, x c, y i, y c) f x i X I, x c X C, y i Y I, y c Y C to tighten, add the inner KKT conditions with respect to the continuous inner variables y c based on regularity being satisfied for all the parameter values any feasible solution in the (MINBP) is feasible in the proposed relaxation For (x i, x c) = ( x i, x c), the upper bounding problem is (Mitsos et al., 2008) min F( x y i,y i, x c, y i, y c) c s.t. G( x i, x c, y i, y c) 0 g( x i, x c, y i, y c) 0 f ( x i, x c, y i, y c) w( x i, x c) + ε f y i Y I, y c Y C any feasible solution in the upper bounding problem is feasible in the (MINBP) set F UB := min{ F, } to express the incumbent IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 2 / 9

21 OUTLINE INTRODUCTION 2 PROPOSED METHOD 3 BOUNDING PROBLEMS: INITIAL NODE 4 BRANCHING & BOUNDING ON SUBDOMAINS 5 NUMERICAL RESULTS 6 CONCLUSIONS IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 2 / 9

22 LIST MANAGEMENT AUXILIARY LISTS L: is the classical list of open nodes, corresponding to the outer problem L I : is the list of exclusively inner open nodes L Xp : is the list of (outer & inner) nodes that cover the whole Y for a subdomain X p of X, p n p Number n p equals the number of partition sets X p X s.t. : for each subdomain X p, the whole Y is maintained Two lists L Xp, L Xp2 are called independent if L Xp L Xp2 = two lists with common nodes, e.g. S X p and S 2 X p, are sublists of L Xp C 0 C D B A 2 B 2 C X A B Y X 0 A Y B X 0 A Y X 0 A B Y (a) L [,] (b) L [,] (c) L [,] (d) L [,0], L [0,] For each p, best inner upper bound lowest over Y, but largest over X p: f UB X p = max{ min { f (j) },..., min { f (j) }}. j S Xp j S Xp s IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 3 / 9

23 MODIFIED BOUNDING PROBLEMS ALLOW BRANCHING & CONSIDER A NODE k OUTER UPPER BOUND k := arg min j L Xp w (j) ( x i, x c ) F (k ) = L Xp min F( x i, x y i,y c, y i, y c) c s.t. G( x i, x c, y i, y c) 0 g( x i, x c, y i, y c) 0 f ( x i, x c, y i, y c) w (k ) ( x i, x c) + ε f y i Y (k ) I, y c Y (k ) C INNER LOWER BOUND f (k) = min f (k) (x i, x x i,x c,y i,y c, y i, y c) c s.t. ğ (k) (x i, x c, y i, y c) 0 x i X (k) I, x c X (k) C y i Y (k) I, y c Y (k) C OUTER LOWER BOUND F (k) = min x i,x c,y i,y c F(x i, x c, y i, y c) s.t. G(x i, x c, y i, y c) 0 g(x i, x c, y i, y c) 0 INNER UPPER BOUND f (k) = min y i,y c,t s.t. t f (x i, x c, y i, y c) fx UB p x i X (k) I, x c X (k) C y i Y (k) I, y c Y (k) C f (X (k) I ḡ(x (k) I y i Y (k) I, X (k) C, y i, y c) t, X (k) C, y i, y c) 0, y c Y (k) C IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 4 / 9

24 NODE FATHOMING RULES NODE k L L Xp INNER FATHOMING RULES IF f (k) = or 2 f (k) > fx UB p then fathom, i.e. delete from L (or L I ) and L Xp. OUTER FATHOMING RULES IF F (k) = or 2 F (k) F UB ε F then OUTER FATHOM, i.e. move from L to L I. Hence, k L I L Xp. IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 5 / 9

25 NODE FATHOMING RULES NODE k L L Xp INNER FATHOMING RULES IF f (k) = or 2 f (k) > fx UB p then fathom, i.e. delete from L (or L I ) and L Xp. OUTER FATHOMING RULES IF F (k) = or 2 F (k) F UB ε F then OUTER FATHOM, i.e. move from L to L I. Hence, k L I L Xp. IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 5 / 9

26 OUTLINE INTRODUCTION 2 PROPOSED METHOD 3 BOUNDING PROBLEMS: INITIAL NODE 4 BRANCHING & BOUNDING ON SUBDOMAINS 5 NUMERICAL RESULTS 6 CONCLUSIONS IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 5 / 9

27 ILLUSTRATIVE EXAMPLE REVISITED : EXTENDED-TREE VERSION min { x 0y s.t. x Z, y arg min{y s.t. y Y(x), y Z}} WITH F = 22 AT (x, y ) = (2, 2) x [0,8] y [0,4] 8 X Y f () =, f () =4, F () = 42, x () =2, w( x)=2, F () = 22 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

28 ILLUSTRATIVE EXAMPLE REVISITED : EXTENDED-TREE VERSION min { x 0y s.t. x Z, y arg min{y s.t. y Y(x), y Z}} WITH F = 22 AT (x, y ) = (2, 2) x [0,8] y [0,4] X f () =, f UB X =4, F () = 42, F UB = Y y y f (2) =, f (2) =4, F (2) = 8 f (3) =2, f (3) =4, F (3) = 42, x (3) =2, w (3) ( x)=2 (lowest), F (3) = 22 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

29 ILLUSTRATIVE EXAMPLE REVISITED : EXTENDED-TREE VERSION min { x 0y s.t. x Z, y arg min{y s.t. y Y(x), y Z}} WITH F = 22 AT (x, y ) = (2, 2) x [0,8] y [0,4] X f () =, f UB X =4, F () = 42, F UB = Y y y f (2) =, f (2) =4, F (2) = 8 f (3) =2, f (3) =4, F (3) = 42, x (3) =2, w (3) ( x)=2 (lowest), F (3) = 22 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

30 ILLUSTRATIVE EXAMPLE REVISITED : EXTENDED-TREE VERSION min { x 0y s.t. x Z, y arg min{y s.t. y Y(x), y Z}} WITH F = 22 AT (x, y ) = (2, 2) x [0,8] y [0,4] X X f () =, f UB X =4, F () = 42, F UB = Y y y 2 2 F (2) = 8 f (3) =2, F (3) = 42 3 x 3 x 4 x 3 x f (6) =, f UB X =4 f (7) =, f UB X 2 = f (4) =2, f (4) =4 f (5) =2 > f UB X 2 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

31 ILLUSTRATIVE EXAMPLE REVISITED : EXTENDED-TREE VERSION min { x 0y s.t. x Z, y arg min{y s.t. y Y(x), y Z}} WITH F = 22 AT (x, y ) = (2, 2) x [0,8] y [0,4] X f () =, f UB X =4, F () = 42, F UB = Y y y 2 2 F (2) = 8 f (3) =2, F (3) = 42 3 x 3 x 4 x 3 x f (6) =, f UB X =4 f (7) =, f UB X 2 = f (4) =2, f (4) =4, F (4) = 42 f (5) =2 > f UB X 2 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

32 ILLUSTRATIVE EXAMPLE REVISITED : EXTENDED-TREE VERSION min { x 0y s.t. x Z, y arg min{y s.t. y Y(x), y Z}} WITH F = 22 AT (x, y ) = (2, 2) x [0,8] y [0,4] X2 f () =, f UB X =4, F () = 42, F UB = X y y 2 Y 2 F (2) = 8 f (3) =2, F (3) = 42 3 x 3 x 4 x 3 x 4 6 f (6) = 7 4 f (4) =2, F (4) = 42 5 x x 2 f (7) =, f UB X 2 = x x 2 f (5) =2 > f UB X f (0) = f () =, f () =4 f (8) =2, f UB X =4 F (8) = 2 f (9) =2, f UB X 2 =2, F (9) = 22 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

33 ILLUSTRATIVE EXAMPLE REVISITED : EXTENDED-TREE VERSION min { x 0y s.t. x Z, y arg min{y s.t. y Y(x), y Z}} WITH F = 22 AT (x, y ) = (2, 2) x [0,8] y [0,4] X2 f () =, f UB X =4, F () = 42, F UB = y y 2 Y 2 F (2) = 8 f (3) =2, F (3) = 42 3 x 3 x 4 x 3 x 4 6 f (6) = 7 4 f (4) =2, F (4) = 42 5 x x 2 f (7) =, f UB X 2 = x x 2 f (5) =2 > f UB X f (0) = f () =, f () =4 f (8) =2 F (8) = 2 f (9) =2, f UB X 2 =2, F (9) = 22 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

34 ILLUSTRATIVE EXAMPLE REVISITED : EXTENDED-TREE VERSION min { x 0y s.t. x Z, y arg min{y s.t. y Y(x), y Z}} WITH F = 22 AT (x, y ) = (2, 2) x [0,8] y [0,4] f () =, f UB X =4, F () = 42, F UB = y y 2 Y 2 F (2) = 8 f (3) =2, F (3) = 42 3 x 3 x 4 x 3 x 4 6 f (6) = 7 4 f (4) =2, F (4) = 42 5 x x 2 f (7) =, f UB X 2 = x x 2 f (5) =2 > f UB X f (0) = f () =, f () =4 f (8) =2 F (8) = 2 f (9) =2, f UB X 2 =2, F (9) = 22 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 6 / 9

35 PRELIMINARY NUMERICAL RESULTS WITH ε f = 0 5 AND ε F = 0 3 Number of variables No. Source Problem x i xc y i yc F UB Nodes Moore & Bard (990) Example Moore & Bard (990) Example Edmunds & Bard (992) Equation Sahin & Ciric (998) Example Dempe (2002) Equation Mitsos (200) am Mitsos (200) am Mitsos (200) am Mitsos (200) am Mitsos (200) am IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 7 / 9

36 OUTLINE INTRODUCTION 2 PROPOSED METHOD 3 BOUNDING PROBLEMS: INITIAL NODE 4 BRANCHING & BOUNDING ON SUBDOMAINS 5 NUMERICAL RESULTS 6 CONCLUSIONS IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 7 / 9

37 CONCLUDING REMARKS Branch-and-Sandwich is a deterministic global optimisation algorithm that can be applied to mixed-integer nonlinear bilevel problems Key features : encompasses implicitly two branch-and-bound trees 2 introduces simple bounding problems, always obtained from the bounding problems of the parent node 3 allows branch-and-bound with respect to x and y, but at the same time it keeps track of the partitioning of Y for successively refined subdomains of X Performance is linked to the tightness of the inner upper bounds f UB X p Numerical results appear promising Implementation & computational experience to investigate alternative choices in the way each step of the proposed algorithm is performed different branching strategies IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 8 / 9

38 ACKNOWLEDGMENTS IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

39 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS PROBLEM FORMULATION : CONTINUOUS CASE The optimistic bilevel problem is a LEADER-follower game The leader (outer) problem is: min F(x, y) s.t. G(x, y) 0, (x, y) X Y, y Y(x) (BPP) x,y Y(x) is the global optimal solution set of the follower (inner) problem: Y(x) = arg min f (x, y) g(x, y) 0 y Y Common assumptions should apply, such as continuity of all functions and compactness of X and Y Assume also twice differentiability of all functions For the inner problem, assume constraint qualifications No convexity assumption is made IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

40 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS CONSTRAINT QUALIFICATION FOR THE INNER PROBLEM RECALL THE INNER PROBLEM min{f (x, y) g(x, y) 0} y Y Assume that a constraint qualification holds for all values of x Regularity ensures that the KKT conditions can be employed and are necessary If we replace y Y by the corresponding bound constraints y L y y U, the KKT conditions of the inner problem define the set below: yf (x, y) + µ yg(x, y) λ + ν = 0, Ω KKT = (x, y, µ, λ, ν) µ T g(x, y) = 0, µ 0, λ T (y L. y) = 0, λ 0, ν T (y y U ) = 0, ν 0. Ω KKT contains all points satisfying the KKT conditions of the inner problem If the inner problem is convex with a unique optimal solution for all values of x the KKT conditions are also sufficient IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

41 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS MOTIVATING EXAMPLE (MITSOS AND BARTON, 200) min y [,] y s.t. y arg min 6y 4 + 2y 3 8y 2 3/2y + /2 y [,] 9 Plot of f(y) = 6y 4 + 2y 3 8y 2 3/2y + /2 9 Plot of f(y) = 6y 4 + 2y 3 8y 2 3/2y + / f(y) 4 f(y) y (e) y 0 0 y * = y (f) 0 y IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

42 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS INNER PROBLEM BOUNDING SCHEME CONSIDER NO BRANCHING YET The auxiliary relaxed inner problem is: f L = The auxiliary restricted inner problem is: f U = max min f (x, y) s.t. g(x, y) 0 x X,y Y min x X y Y f (x, y) s.t. g(x, y) 0 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

43 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS INNER PROBLEM BOUNDING SCHEME (CONT.) CONSIDER NO BRANCHING YET The auxiliary relaxed inner problem is: f = min fx,y(x, y) s.t. ğ x X,y Y x,y(x, y) 0 Relaxation using convex underestimators f x,y(x, y) and ğ x,y(x, y) (e.g. Floudas, 2000, Tawarmalani and Sahinidis, 2002) The auxiliary restricted inner problem is: f = max x 0,x,y,µ,λ,ν x 0, s.t. x 0 f (x, y) 0, g(x, y) 0, (x, y) X Y, (x, y, µ, λ, ν) Ω KKT. Relaxation using the KKT-approach (Still, 2004, Stein and Still, 2002) IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

44 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS OUTER PROBLEM BOUNDING SCHEME CONSIDER NO BRANCHING YET The proposed lower bounding problem is: F = min x,y,µ,λ,ν F(x, y), s.t. G(x, y) 0, g(x, y) 0, f (x, y) f, (x, y) X Y, (x, y, µ, λ, ν) Ω KKT, any feasible solution in the BPP is feasible in the proposed relaxation need to solve to global optimality For x = x, the upper bounding problem is (Mitsos et al., 2008): F = min y Y F( x, y) s.t. G( x, y) 0, g( x, y) 0, f ( x, y) w( x) + ε f In this work, w( x) = min fy( x, y) s.t. ğ y Y y( x, y) 0 Any feasible solution ȳ in the restricted problem is feasible in the BPP: f ( x, ȳ) ε f w( x) w( x) f ( x, ȳ) + ε f IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

45 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS OUTER UPPER BOUNDING PROBLEM REQUIRES PARTITIONING OF THE INNER SPACE Y Convexifying the inner problem for fixed x requires some form of refinement of Y in order to compute tighter and tighter approximations of the inner problem over refined subregions of Y Subdivision of Y is usually applied to semi-infinite programs, but no branching with respect to y the whole Y is always considered in subproblems e.g. Bhattacharjee et al. (2005a;b), Floudas and Stein (2007), Mitsos et al. (2008a) We use partitioning of Y no distinction between the inner and outer decision spaces during branching possible to consider only some subregions of Y and eliminate others via fathoming all Y subregions where global optima may lie are considered IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

46 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS AFTER FATHOMING : A USEFUL PRELIMINARY THEORETICAL RESULT Every independent list L Xp, p {,..., n p}, still contains all promising subregions of Y where global optimal solutions may lie for any x X p Define the set of fathomed Y domains for X p as follows: F Xp := { d Y (d) Y (d) Y deleted for all x X p}. Then, we prove by contradiction that Y(x) F Xp = x X p The sets F Xp, p =,..., n p, are infeasible in the BPP X p Y (d) IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

47 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS THE BRANCH-AND-SANDWICH ALGORITHM IS ε-convergent AT TERMINATION, AN ε-optimal SOLUTION OF THE BILEVEL PROBLEM IS COMPUTED Convergent best inner upper bound lim q f UB X pq = min j S x { f (j) } = min j S x min y Y (j){f ( x, y) s.t. g( x, y) 0} = w( x) ε f -finite Convergent inner B-&-B scheme Certain-in-the-limit fathoming by outer infeasibility rule f ( x, ȳ) fx UB } pq fx UB pq w( x) ε f q q f ( x, ȳ) w( x) + ε f ȳ / Y εf ( x) f ( x, ȳ) > w( x) + ε f Consistent bounding scheme: F (kq) and F (kq) are identical in the limit Bound-improving selection operation The Branch-and-Sandwich algorithm is ε-convergent by Horst and Tuy (996) IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

48 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS PRELIMINARY NUMERICAL RESULTS WITH ε f = 0 5 AND ε F = 0 3 FOR ALL PROBLEMS, EXCEPT NO. 20 WHERE ε F = 0 No. NC Inner n m r F UB Nodes Yes No No Yes Yes Yes Yes No No Yes 0 3 Yes Yes Yes Yes Yes Yes No. NC Inner n m r F UB Nodes 8 Yes Yes Yes Yes Yes Yes Yes Yes Yes No No Yes No No Yes Yes Yes IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9

49 INTRODUCTION BOUNDING SCHEME: INITIAL NODE CONVERGENCE NUMERICAL RESULTS BRANCH-AND-BOUND TREE FOR PROBLEM NO. f () = 52.50, f UB [0.,] =0.57, F() = 0.50, F () = y [,0] y [0,] 2 f (2) = 24, f (2) =0.57, F (2) = 3 f (3) = 22.60, f UB [0.,] = 0.0, F (3) =0.50, F (3) = x [0.,0.55] 6 f (6) = 4.50, f (6) =0.3 y [, 0.5] x [0.55,] 7 f (7) = 6, f (7) =0.57 y [ 0.5,0] x [0.,0.55] 4 f (4) = 4.20, f (4) = 0.0, F (4) =0.50, F (4) = y [0,0.5] x [0.55,] 5 f (5) = 5.40, f [0.55,] UB = 0.55, F (5) =0.50, F (5) = y [0.5,] 0 f (0) = 4.60, f (0) =0 f () =.70, f () =0.57 f (8) = 2, f (8) = 0.55, F (8) =0.50, F UB = f (9) = 4.70, f (9) = 0.55, F (9) =0.50, F (9) =0.50 IMPERIAL COLLEGE LONDON BRANCH-AND-SANDWICH ALGORITHM MINLP WORKSHOP 9 / 9