13. Cones and semidefinite constraints

Transcription

1 CS/ECE/ISyE 524 Introduction to Optimization Spring Cones and semidefinite constraints ˆ Geometry of cones ˆ Second order cone programs ˆ Example: robust linear program ˆ Semidefinite constraints Laurent Lessard (

2 What is a cone? ˆ A set of points C R n is called a cone if it satisfies: αx C whenever x C and α > 0. x + y C whenever x C and y C. ˆ Similar to a subspace, but α > 0 instead of α R. (this is a critical difference!) ˆ Simple examples: x y and y

3 What is a cone? ˆ A slice of a cone is its intersection with a subspace. ˆ We are interested in convex cones (all slices are convex). ˆ Can be polyhedral, ellipsoidal, or something else

4 What is a cone? Polyhedral cone recipe: 1. Begin with your favorite polyhedron Ax b where x R n 2. {Ax bt, t 0} is a polyhedral cone in (x, t) R n+1 3. The slice t = 1 is the original polyhedron

5 What is a cone? Ellipsoidal cone recipe: 1. Ellipsoid x T Px + q T x + r 0 where P 0 and x R n 2. Complete the square Ax + b c 3. { Ax + bt ct} is an ellipsoidal cone in (x, t) R n+1 4. The slice t = 1 is the original ellipsoid

6 Second-order cone constraint A second-order cone constraint is the set of points x R n : Ax + b c T x + d Every SOC constraint is a slice (set t = 1) of the cone Ax + bt c T x + dt. It s not always a cone itself! Special cases: ˆ If A = 0, we have a linear inequality (hyperplane) ˆ If c = 0, it s a slice of an ellipsoidal cone Every SOC constraint describes a convex set. 13-6

7 Second-order cone constraint A second-order cone constraint is the set of points x R n : Ax + b c T x + d If you square both sides... Ax + b c T x + d { Ax + b 2 (c T x + d) 2 c T x + d 0 The quadratic inequality is: x T (A T A cc T )x + 2(b T A dc T )x + (b T b d 2 ) 0 This may be nonconvex! 13-7

8 Second-order cone constraint A second-order cone constraint is the set of points x R n : Ax + b c T x + d Example: If A = [ 1 0 ] and c = x y Squaring both sides leads to: [ ] 0 and b = d = 0: 1 x 2 y 2 0 and y

9 Special case: rotated second-order cone A rotated second-order cone is the set x R n, y, z R: With n = 1, this looks like: x T x yz, y 0, z

10 Special case: rotated second-order cone A rotated second-order cone is the set x R n, y, z R: Can put into standard form: x T x yz, y 0, z 0 4x T x 4yz 4x T x + y 2 + z 2 4yz + y 2 + z 2 4x T x + (y z) 2 (y + z) 2 4x T x + (y z) 2 y + z [ ] 2x y + z y z 13-10

11 SOCPs A second-order cone program (SOCP) has the form: minimize x c T x subject to: A i x + b i ci T x + d i for i = 1,..., m ˆ Every LP is an SOCP (just make each A i = 0) ˆ Every convex QP and QCQP is an SOCP convert quadratic cost to epigraph form (add a variable) convert quadratic constraints to SOCP (complete square) 13-11

12 Implementation details A second-order cone program (SOCP) has the form: minimize x c T x subject to: A i x + b i ci T x + d i for i = 1,..., m ˆ In JuMP, you can specify SOCP norm(a*x+b) <= dot(c,x)+d) works with ECOS, SCS, Mosek, Gurobi, Ipopt. ˆ Can also specify rotated cones directly in Mosek, Ipopt

13 Example: robust LP Consider a linear program with each linear constraint separately written out: maximize x c T x subject to: ai T x b i for i = 1,..., m Suppose there is uncertainty in some of the a i vectors. Say for example that a i = ā i + ρu where ā i is a nominal value and u is the uncertainty. ˆ box constraint: u 1 ˆ ball constraints: u

14 Robust LP with box constraint Substituting a i = ā i + ρu into a T i x b i, obtain: ā T i x + ρu T x b i for all uncertain u box constraint: If this must hold for all u with u 1, then it holds for the worst-case u. Therefore: u T x = n u i x i n u i x i n x i = x 1 i=1 i=1 i=1 Then we have ā T i x + ρ x 1 b i 13-14

15 Robust LP with box constraint With a box constraint a i = ā i + ρu with u 1 maximize x c T x subject to: ai T x b i for i = 1,..., m Is equivalent to the optimization problem maximize x c T x subject to: āi T x + ρ x 1 b i for i = 1,..., m 13-15

16 Robust LP with box constraint With a box constraint a i = ā i + ρu with u 1 maximize x c T x subject to: ai T x b i for i = 1,..., m... which is equivalent to the linear program: maximize x,t subject to: c T x n āi T x + ρ t j b i for i = 1,..., m j=1 t j x j t j for j = 1,..., n 13-16

17 Robust LP with box constraint a T i x b i a T i x x 1 b i ˆ New region is smaller, still a polyhedron ˆ More robust to uncertain constraints 13-17

18 Robust LP with ball constraint Substituting a i = ā i + ρu into a T i x b i, obtain: ā T i x + ρu T x b i for all uncertain u ball constraint: If this must hold for all u with u 2 1, then it holds for the worst-case u. Using Cauchy-Schwarz inequality: u T x u 2 x 2 x 2 Then we have āi T x + ρ x 2 b i (a second-order cone constraint!) 13-18

19 Robust LP with ball constraint With a ball constraint a i = ā i + ρu with u 2 1 maximize x c T x subject to: ai T x b i for i = 1,..., m Is equivalent to the optimization problem maximize x c T x subject to: āi T x + ρ x 2 b i for i = 1,..., m which is an SOCP 13-19

20 Robust LP with ball constraint a T i x b i a T i x x 2 b i ˆ New region is smaller, no longer a polyhedron ˆ More robust to uncertain constraints 13-20

21 Matrix variables Sometimes, the decision variable is a matrix X. ˆ Can always just think of X R m n as x R mn. ˆ Linear functions: ˆ Linear program: m i=1 mn k=1 c k x k = c T x n C ij X ij = trace(c T X ) j=1 maximize X trace(c T X ) subject to: trace(a T i X ) b i for i = 1,..., k 13-21

22 Matrix variables If a decision variable is a symmetric matrix X = X T R n n, we can represent it as a vector x R n(n+1)/2. x 1 x 2 x 3 x 2 x 4 x 5 x 3 x 5 x 6 x 1 x 2 x 3 x 4 x 5 x 6 The constraint X 0 is called a semidefinite constraint. What does it look like geometrically? 13-22

23 The PSD cone The set of matrices X 0 are a convex cone in R n(n+1)/2 [ ] x y Example: The set 0 of points in R y z 3 satisfy: xz y 2, x 0, z 0 This is a rotated second-order cone! Equivalent to: [ ] 2y x + z x z 13-23

24 More complicated example The set of (x, y, z) satisfying [ 1 x ] y x 1 z y z 1 0 is the solution of: { X R 3 3, X 0, X 11 = 1, X 22 = 1, X 33 = 1 } 13-24

25 Spectrahedra ˆ Two common set representations: variables x1,..., x k, constants Q i = Q T i, and constraint: Q 0 + x 1 Q x k Q k 0 (linear matrix inequality) variable X 0 and the constraints: trace(a T i X ) b i (linear constraint form) ˆ These sets are called spectrahedra. ˆ Very rich set, lots of possible shapes

26 Semidefinite program (SDP) Standard form #1: (looks like the standard form for an LP) maximize X trace(c T X ) subject to: trace(a T i X ) b i for i = 1,..., m X 0 Standard form #2: maximize x c T x subject to: Q 0 + m x i Q i 0 i=

27 Relationship with other programs Every LP is an SDP: [ ] [ ] a11 a 12 x1 a 21 a 22 x 2 is the same as: [ ] a11 0 x 1 0 a 21 [ b1 b 2 [ ] a x 2 0 a 22 (polyhedra are special cases of spectrahedra) ] [ ] b1 0 0 b

28 Relationship with other programs Every SOCP is an SDP: Ax + b c T x + d is the same as: [ ] (c T x + d)i Ax + b (Ax + b) T c T 0 x + d This isn t obvious proof requires use of Schur complement. (second-order cones are special cases of spectrahedra) 13-28