A primal-dual Dikin affine scaling method for symmetric conic optimization

Transcription

1 A primal-dual Dikin affine scaling method for symmetric conic optimization Ali Mohammad-Nezhad Tamás Terlaky Department of Industrial and Systems Engineering Lehigh University July 15, 2015 A primal-dual Dikin affine scaling method for SCO (1 of 33)

2 Outline Dikin affine scaling Primal-dual Dikin affine scaling Extension to symmetric cones Numerical results A primal-dual Dikin affine scaling method for SCO (2 of 33)

3 Outline Dikin affine scaling Primal-dual Dikin affine scaling Extension to symmetric cones Numerical results A primal-dual Dikin affine scaling method for SCO (3 of 33)

4 Semidefinite optimization The first idea of IPMs goes back to the work of Frisch (1955). IPMs for nonlinear optimization by Fiacco and McCormic in 1960s. Affine scaling direction introduced by Dikin in 1960s. Karmarkar (1984) revived the interest in IPMs. Primal-dual affine scaling methods were proposed in 1990s. Monteiro, Adler, and Resende (1990). Jansen, Roos, and Terlaky (1996). A primal-dual Dikin affine scaling method for SCO (4 of 33)

5 Ilya I. Dikin ( ) A primal-dual Dikin affine scaling method for SCO (5 of 33)

6 Primal affine scaling method Originally a primal method for linear optimization (LO). At each step minimizes the objective over an ellipsoid. The idea is to replace the nonnegativity constraint x 0 by (x x) T diag( x) 2 (x x) 1, where x denotes an interior feasible solution. Dikin proved that under nondegeneracy assumption, this method converges. The polynomial complexity of the method has still not been proved. A primal-dual Dikin affine scaling method for SCO (6 of 33)

7 Outline Dikin affine scaling Primal-dual Dikin affine scaling Extension to symmetric cones Numerical results A primal-dual Dikin affine scaling method for SCO (7 of 33)

8 Illustration of the primal-dual Dikin method Jansen, Roos and Terlaky (1996); A primal-dual Dikin affine scaling method. The method has a worst-case iteration complexity O(nL). A primal-dual Dikin affine scaling method for SCO (8 of 33)

9 Primal-dual Dikin affine scaling method At each step, the search directions ( x, s) are derived by solving min x T s + s T x s.t. A x = 0, A T y + s = 0, x 1 x + s 1 s 1, where x 1 x and s 1 s are coordinate-wise products. Let v = xs, d = x s, dx = d 1 x, d s = d s, and d y = y. The scaled subproblem in the v-space is represented by min v T (d x + d s) s.t. Ād x = 0, Ā T d y + d s = 0, v 1 (d x + d s) 1, where Ā = ADiag(d). A primal-dual Dikin affine scaling method for SCO (9 of 33)

10 Extension to semidefinite optimization (SDO) De Klerk, Roos, and Terlaky (1998) generalized the method to SDO. The search directions (D X, D S ) are derived by solving min trace(v (D X + D S )) s.t. trace(āid X ) = 0, i = 1,..., m m i=1 Ā i (d y) i + D S = 0, V 1 2 (D X + D S )V 1 2 1, where V is the NT scaling matrix. D X and D S are defined in a similar way as in LO. A primal-dual Dikin affine scaling method for SCO (10 of 33)

11 Outline Dikin affine scaling Primal-dual Dikin affine scaling Extension to symmetric cones Numerical results A primal-dual Dikin affine scaling method for SCO (11 of 33)

12 Extension to symmetric conic optimization (SCO) Goal: To extend the primal-dual Dikin method to SCO. Symmetric cone generalizes R n +, Ln, and S n +. Euclidean Jordan Algebra (EJA) is a major tool for the analysis of SCO. EJA provides a unified framework for IPMs over all symmetric cones. A primal-dual Dikin affine scaling method for SCO (12 of 33)

13 Symmetric cone Let J be a vector space over the field of real numbers; K J be a closed pointed convex cone; K + be the interior of K. x, s is the inner product of x, s J. The dual cone of K is K = {s : x, s 0, for all x K}, K is self-dual if K = K. K is referred to as a homogeneous cone if: an invertible map A exists so that A(x) = s and A(K) = K for all x, s K +. K is symmetric if it is both self-dual and homogeneous. Special cases: R n +, Ln, and S n + A primal-dual Dikin affine scaling method for SCO (13 of 33)

14 Standard form of SCO (P ) min{ c, x : Ax = b, x K}, (D) max{b T y : A y + s = c, s K, y R m }. K is a symmetric cone. b R m, and c, a i J for i = 1,..., m. a 1, x. Ax =. ; A is assumed to be full row rank.. a m, x Interior point condition holds. A primal-dual Dikin affine scaling method for SCO (14 of 33)

15 Optimality conditions and Jordan product Optimality conditions Primal and dual feasibility: Ax = b, A y + s = c, x, s K Optimality: b T y = c, x x, s = 0. Jordan product Optimal iff x s = 0. x s = 0 is the complementarity condition. x s = L(x)s is a bilinear map, where L(x) is a symmetric matrix. Properties: L(x)e = x, L(x)x 1 = e, and L(x)x = x 2. Quadratic representation of x: P (x) = 2L 2 (x) L(x 2 ). (J, ) is a commutative algebra with rank r and dimension n. A primal-dual Dikin affine scaling method for SCO (15 of 33)

16 Second-order cone Example The second-order cone is represented as L n = {x R n : x 0 x 1:n 1 }. e n 1 = [1, 0,..., 0] T [ x L(x) := 0 x 1:n 1 x T ] 1:n 1. x 0 I n 1 [ x s = x T ] s. x 0 s 1:n 1 + s 0 x 1:n 1 A primal-dual Dikin affine scaling method for SCO (16 of 33)

17 Scaled primal-dual problem The NT scaling point of x and s is defined as w = P (s 1 2 )(P (s 1 2 )x) 1 2. For x, s K +, there exists a unique w K 0 so that v = P (w) 1 2 x = P (w) 1 2 s. The scaled primal-dual system is given by Ād x = 0, Ā d y + d s = 0, where d x = P (w) 1 2 x, d s = P (w) 1 2 s, d y = y, Ā = AP (w) 1 2. The complementary gap is x, s = P (w) 1 2 v, P (w) 1 2 v = v, v. A primal-dual Dikin affine scaling method for SCO (17 of 33)

18 Extension of Dikin search directions The generalization of the Dikin ellipsoid is written as P (w) 1 2 x 1 P (w) 1 2 x + P (w) 1 2 s 1 P (w) 1 2 s F 1, which is equivalent to v 1 (d x + d s) F 1. The Dikin search directions are derived by solving min trace(v (d x + d s)) s.t. Ād x = 0, Ā d y + d s = 0, v 1 (d x + d s) F 1. (*) A primal-dual Dikin affine scaling method for SCO (18 of 33)

19 Proximity to the central path and feasibility The Dikin method keeps the iterates close to the central path. The measure of proximity is defined as κ(v 2 ) = λmax(v2 ) λ min (v 2 ), where λ min (v 2 ) and λ max(v 2 ) are the min and max eigenvalues of v 2. Note: κ(v 2 ) 1, with equality iff v 2 = µe. A primal-dual Dikin affine scaling method for SCO (19 of 33)

20 The primal-dual Dikin method Input An interior feasible feasible solution (x 0, y 0, s 0 ) Parameters Proximity measure τ > 1 so that κ(x 0 s 0 ) τ Steplength α with default value 1 τ r Accuracy parameter ɛ x := x 0, s := s 0 Repeat Obtain ( x, s) by solving (*) Set x := x + α x Set s := s + α s Until trace(x s) ɛ A primal-dual Dikin affine scaling method for SCO (20 of 33)

21 Iteration complexity of the Dikin method Theorem Let ɛ > 0, α = 1 τ r and τ > 1 so that κ(x0 s 0 ) τ. The method terminates after at most τr log trace(x0 s 0 ) iterations. ɛ It yields a feasible solution (x, s) such that κ(v 2 ) τ and trace(x s) ɛ. A primal-dual Dikin affine scaling method for SCO (21 of 33)

22 Outline Dikin affine scaling Primal-dual Dikin affine scaling Extension to symmetric cones Numerical results A primal-dual Dikin affine scaling method for SCO (22 of 33)

23 Numerical results We compare the Dikin affine scaling method and SeDuMi solver. 13 second-order conic problems from Dimacs library. SeDuMi was modified to implement the Dikin affine scaling method. Centering steps are introduced as safeguard. A primal-dual Dikin affine scaling method for SCO (23 of 33)

24 Numerical results Dikin-type affine scaling SeDuMi Instance CPU/Iter Obj relinf Cent CPU/Iter Obj relinf nql30n 3.6/ E / E-08 nql30o 7.2/ E / E-09 nql60n 12.7/ E / E-08 nql60o 44.3/ E / E-08 nql180n 447.7/ E / E-06 nb 4.7/ E / E-12 nb-l1 4.4/ E / E-09 nb-l2 11.0/ E / E-09 nb-l2-b 3.8/ E / E-11 qssp30n 11.8/ E / E-08 qssp30o 40.8/ E / qssp60n 82.7/ E / E-08 qssp60o 332.5/ E / A primal-dual Dikin affine scaling method for SCO (24 of 33)

25 Conclusion Generalized the primal-dual Dikin affine scaling method for SCO. Established polynomial complexity of the method. EJA provides a powerful tool for the analysis of the method for SCO. The method is viable by the numerical results. A primal-dual Dikin affine scaling method for SCO (25 of 33)

26 Thank you for your attention Any questions? A primal-dual Dikin affine scaling method for SCO (26 of 33)

27 Euclidean Jordan algebra (EJA) Definition Let J be a vector space over R n with bilinear map (x, y) x y. (J, ) is referred to as EJA if for all x, y J 1. x y = y x, 2. x (x 2 y) = x 2 (x y) where x 2 = x x, 3. there exists an inner product so that x y, s = x, y s. There exists a unique element e such that x e = e x = x for all x J. EJA is commutative over the field of real numbers. EJA is power associative; that is x p+q = x p x q. A primal-dual Dikin affine scaling method for SCO (27 of 33)

28 Cone of squares Definition The cone of squares of (J, ) is defined as K(J ) = {x 2 : x J }, where x 2 = x x. K(J ) is a closed pointed convex cone with nonempty interior. Theorem A cone is symmetric iff it is the cone of squares of some EJA. A primal-dual Dikin affine scaling method for SCO (28 of 33)

29 Rank and characteristic polynomial Let r be the smallest integer so that {e, x, x 2,, x r } is linearly dependent. r is the degree of x denoted by deg(x) for x J. The rank of J is defined as rk(j ) = max x J {deg(x)}. x is regular if deg(x) = rk(j ). Suppose that x is a regular element of J. The characteristic polynomial of x is given by λ r a 1 (x)λ r ( 1) r a r(x), where a 1 (x),..., a r(x) are real numbers so that x r a 1 (x)x r ( 1) r a r(x)e = 0. A primal-dual Dikin affine scaling method for SCO (29 of 33)

30 Eigenvalues, trace and determinant Definition Let λ 1,..., λ r be the roots of the characteristic polynomial. λ 1,..., λ r are called the eigenvalues of x J. trace(x) = λ λ r. det(x) = λ 1 λ 2...λ r. x, y = trace(x y). x F = λ λ2 r. x 2 = max i λ i. A primal-dual Dikin affine scaling method for SCO (30 of 33)

31 Spectral decomposition Definition A Jordan frame is a set of elements {q 1,, q k } of J so that q i cannot be represented by the sum of two other elements. qi 2 = q i for i = 1,..., k. q i q j = 0 for all i j. q q k = e. Theorem Let J be an EJA with rank r. Each x J can be uniquely represented as x = λ 1 q λ rq r, where the eigenvalues are real numbers. A primal-dual Dikin affine scaling method for SCO (31 of 33)

32 Second-order cone Example Let x L n. It can be shown that x 2 2x 0 x + (x 2 0 x 1:n 1 2 )e = 0. r = 2 for this EJA. The spectral decomposition is given by x = λ 1 q 1 + λ 2 q 2, where λ 1 = x 0 x 1:n 1, λ 2 = x 0 + x 1:n 1, ( ) ( ) q 1 = x 1:n 1, q 2 = 1 1 x 2 1:n 1. x 1:n 1 x 1:n 1 A primal-dual Dikin affine scaling method for SCO (32 of 33)

33 Spectral decomposition In particular: x = λ 2 1 q λ 2 k qr, x 1 = λ 1 1 q λ 1 k qr. x is invertible if λ 1,..., λ r are nonzero. x K (K + ) if λ 1,..., λ r are nonnegative (positive). A primal-dual Dikin affine scaling method for SCO (33 of 33)