Computational Optimization. Constrained Optimization

Transcription

1 Computational Optimization Constrained Optimization

2 Easiest Problem Linear equality constraints min f( ) f R s.. t A b A R, b R n m n m

3 Null Space Representation Let * be a feasible point, A*b. Any other feasible point can be written as *+P where Ap0 The feasible region { : *+p p N(A)} where N(A) is null space of A

4 min Eample ( ) + + Solve by substitution becomes 3 st min 3 ( ) s.. t (( ) ) min

5 Null Space Method min ( ) st * [4 0 0] Z *+v becomes v 4v 0 0 v v + v 0 0 v (( ) v ) v + v + v3 min

6 Variable Reduction Method Let A[B N] for * (a basic feasible solution with at most m nonzero variables corresponding to columns of B) A R B R N R I R Z B N I m n m m m ( n m) ( n m) ( n m) assumes m < n is a basis matri for null space of A B B A [ ] r AAr BN I

7 Where did Z come from? A[ 3 4] * [4 0 0] A[B N] B[] N [3 4] Z *[3 4] 3 4 B N 0 0 I 0 0

8 General Method There eists a Null Space Matri n r Z R r n m The feasible region is: { * + Zv} Equivalent Reduced Problem min ( * ) v f + Zv

9 Practice Problem min ( ) st

10 Optimality Conditions Assume feasible point and convert to null space formulation gv () f( * + Zv) g( v) Z' f( * + Zv) Z' f( y) 0 wherez * + Zv gv () Z' f( * + ZvZ ) Z' f() yz

11 Lemma 4. Necessary Conditions (Nash + Sofer) If * is a local min of f over { Ab}, and Z is a null matri Z' f( *) 0 and Z f Z is p s d ' ( *)... Or equivalently use KKT Conditions f(*) A' λ 0 has a solution A* b Z f Z is psd ' ( *)...

12 Lemma 4. Sufficient Conditions (Nash + Sofer) If * satisfies (where Z is a basis matri for Null(A)) A* b Z ' f( *) 0 Z ' f( *) Z is pd.. then * is a strict local minimizer

13 Lemma 4. Sufficient Conditions (KKT form) If (*,λ*) satisfies (where Z is a basis matri for Null(A)) A* b f(*) A' 0 λ Z ' f( *) Z is pd.. then * is a strict local minimizer

14 Lagrangian Multiplier λ* is called the Lagrangian Multiplier It represents the sensitivity of solution to small perturbations of constraints f( ˆ) f( *) + ( ˆ *) ' f( *) f (*) + ( ˆ *)' A' λ * by KKT OC Nowlet Aˆ b + δ * f(*) + δ ' λ* f(*) + δiλi m i

15 Optimality conditions Consider min ( +4y )/ s.t. -y0 f( ) A' λ 0 A b 4y λ y 0 * λ* 8, y*,

16 Optimality conditions Find KKT point Check SOSC f( ) A' λ 0 A b 4y λ y 0 * λ* 8, y*, Z ' [] ( ) f Z f Z is pd ' ( ).. So SOSC satisfied Or we could just observe that it is a conve program so FONC are sufficient

17 ( ) [ ] 0 4 A 4 f() b A ) ( 0 s.t. 4 min + A f T λ λ Linear Equality Constraints - I

18 Linear Equality Constraints - II KKT point 8 *, 8 * Solve : λ λ λ

19 [ ] [ ] ) ( ) ( Z - A SOSC > Z f Z f T so SOSC satisfied, and * is a strict local minimum Objective is conve, so KKT conditions are sufficient. Linear Equality Constraints - III

20 Handy ways to compute Null Space Variable Reduction Method Orthogonal Projection Matri QR factorization (best numerically) ZNull(A) in matlab

21 Orthogonal Projection Method Use optimization. Minimize distance between given point c and null space of A. min p p c st.. Ap 0 f( p*) A' λ Ap* 0 or equivalently ( p* c) A' λ Ap* 0

22 Orthogonal Projection Method Optimality conditions give us the solution FONC is ( p* c) A' λ Ap* 0 Ap * Ac AA' λ λ AA' ( ) Ac ( ) ( ) p* A' λ + c A' AA' Ac + c ( I A' AA' A) c

23 Orthogonal Projection Method Final result is: ( ) ( I A' AA' A) Null Matrices of A Note null space matri is not unique Try it in Matlab for A [ 3 5; 4 -] Compare with Null(A) Null( A,r)

24 Get Lagrangian Multipliers for free! The matri A A' AA' where AA AA' AA' I r ( ) ( ) is the right inverse matri for A. For general problems r min f ( ) st.. A λ ' r * A f( *) b

25 Let s try it For min f( ) st Projection matri Z I A' AA' A ( ) [ ] [ ]

26 Solve FONC for Optimal Point FONC f( ) A' λ λ

27 Check Optimality Conditions For * []/ 4 f( *) []/ 4 A* b Z f(*) * Using Lagrangian ( ) Ar A' AA' []' 4 λ A f(*) /4 r Clearly f(*) A' λ

28 For min f( ) ' C st.. A b You try it C A b Find projection matri Confirm optimality conds are Z C*0, A* b Find * Compute Lagrangian multipliers Check Lagrangian form of the multipliers.

29 Variable Reduction Method Let A[B N] A is m by n B is m by m assume m < n Z B N I is a basis matri for null space of A B B A [ ] r AAr BN I

30 Try on our eample Take for eample first two columns for B Then A [ B N] Z A r Condition number of Z CZ 58 better but not great

31 QR Factorization Use Gram-Schmidt algorithm to make orthogonal factorize A QR with Q orthogonal and R upper triangular R A' QR [ Q Q] 0 where A m n, Q n m, Q n ( n m), R m m Z Q A QR T r

32 QR on problem Use matlab command QR [Q R ] qr(a ) Q Q(:,3:4) Cond(Q *C*Q) 9.79

33 In Class Practice Find optimal solution and verify FONC and SOSC of the following: Let the perimeter of a rectangle be fied to 4. Find the shape of the rectangle with largest area. Solve the problem ma s.t