GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 1

Transcription

1 GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 1

2 Quadratic approximation of some convex optimization problems using the arithmetic-geometric mean iteration François Glineur UCL/CORE (Center for Operations Research and Econometrics) UCL/INMA (Département d ingénierie mathématique) November 20, 2009 GeoLMI Toulouse GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

3 Overview 1. Motivation Convex optimization: problem classes Approximations: direct vs. extended formulations 2. Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Application to approximations 3. Generalizations and conclusions Matrix version Conclusions Addendum GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

4 Motivation Convex optimization: problem classes Overview 1. Motivation Convex optimization: problem classes Approximations: direct vs. extended formulations 2. Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Application to approximations 3. Generalizations and conclusions Matrix version Conclusions Addendum GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

5 Motivation Convex optimization: problem classes Convex optimization Nonlinear optimization min f 0(x) such that f i (x) 0 for all i I and f i (x) = 0 for all i E x R n Variables: finite-dimensional vector x R n Constraints: finite number of (in)equalities, indexed by sets I and E Problem is convex when objective function f 0 is convex functions f i defining inequalities f i (x) 0 are convex for all i I functions f i defining equalities f i (x) = 0 are affine for all i E GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 1

6 Motivation Convex optimization: problem classes Well-known classes of convex problems min f 0(x) such that f i (x) 0 for all i I and f i (x) = 0 for all i E x R n 1. Linear optimization (LO): f 0 and f i are affine for all i E I f i (x) = a T i x b i 2. Quadratically constrained quadratic optimization (QCQO): f 0 and f i are convex quadratic for all i I f i (x) = x T Q i x + r T i x + s i with Q i 0 (equalities f i, if present, must still be affine for i E) We call these problems structured convex problems GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 2

7 Motivation Convex optimization: problem classes Other well-known classes of convex problems Conic optimization (over symmetric cones) 3. Second-order cone optimization (SOCO) involves constraints such as (ci1 a T i1x, c i2 a T i2x,..., c in a T inx) ci0 a T i0x 4. Semidefinite optimization (over symmetric real matrices, SDO) C + i x i A i 0 QCQO is a special case of SOCO (i.e. QCQO problems have an equivalent formulation as SOCO problems, although no proof of strict inclusion yet) Both LO and SOCO are special cases of (real) SDO, as well as complex/hermitian SDO GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 3

8 Motivation Convex optimization: problem classes More classes of well-known convex problems 4. Geometric optimization (GO): f 0 and f i are posynomials (in exponential form) for all i I f i (x) = c i + j M i exp(a T j x b j ) Each term in the sum is the composition of exponential and affine scalar function 5. Optimization with powers: l p -norm optimization (l p O): f 0 linear, f i are affine plus sum of convex powers with affine scalar arguments for all i I f i (x) = c T i x d i + j M i a T j x b j pj with p j 1 GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 4

9 Motivation Convex optimization: problem classes Even more classes of well-known convex problems 6. Optimization with norms: sum-of-norm optimization (SNO): f 0 (and f i for all i I, if any) are convex norms with affine arguments f i (x) = c i + j M i A T j x b j pj with p j 1 with y p = ( x 1 p + x 2 p + + x n p) 1 p 7. Entropy optimization (EO): f 0 is a sum of entropy terms, f i are affine for all i E f 0 (x) = i x i log x i (implicitly implying x 0) 8. Analytic centering (AC): f 0 is a sum of logarithmic terms, f i are affine for all i I E f 0 (x) = j M 0 log(a T j x b j ) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 5

10 Motivation Convex optimization: problem classes Solving convex problems in practice Although each class can be tackled by a general-purpose nonlinear solver, better performance is expected from dedicated solvers Such solvers exist for all problems classes described (and others) but typically only handle one (or a few) problem class at a time i.e. there are dedicated solvers for linear optimization, quadratic optimization, geometric optimization, etc. Typically some problem classes are (much) easier to solve than others Note however that in theory, all problems can be solved by unified class of interior-point algorithms, but no single unified efficient solver seems to exist in practice Currenly most versatile class of solvers: mixed linear-second-order-semidefinite, such as SeDuMi or SDPT3 GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 6

11 Motivation Approximations: direct vs. extended formulations Overview 1. Motivation Convex optimization: problem classes Approximations: direct vs. extended formulations 2. Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Application to approximations 3. Generalizations and conclusions Matrix version Conclusions Addendum GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

12 Motivation Approximations: direct vs. extended formulations Approximations We focus on approximating one class of problems P with another class of problems A if possible with arbitrarily high accuracy (but then typically at the cost of a growing size for the approximating problem) Examples Linear approximations of quadratic problems Previous work: very nice construction of Ben-Tal and Nemirovski (2001) Quadratic approximations of nonlinear, nonquadratic convex functions This work: based on the concept of arithmetic-geometric mean iteration GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 7

13 Motivation Approximations: direct vs. extended formulations Why approximate problems P by problems A? Useful in a few situations, such as algorithms for A are significantly faster than algorithms for P hope to obtain an approximate solution to P, possibly with very high accuracy, in less time than required to solve it exactly from a more practical point of view: need to solve problem P but only have access to solver for A in particular, when dealing with the following type of problems 1. problem to be solved is discrete, such as (mixed) integer programming 2. its continuous relaxation belongs to class P 3. available branch-and-bound type solvers only work with subproblems of type A e.g. quadratic integer optimization using a commercial and highly optimized linear (mixed) integer optimization solver GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 8

14 Motivation Approximations: direct vs. extended formulations Direct vs. extended formulations Assume we want linear (polyhedral) approximation of convex set S R n 1. Direct approximation Look for polytope in R n approximating S, i.e. P = {y R n A T y c} such that P S 2. Approximation based on extended formulation, i.e. a lifting Look for polytope in higher-dimensional space R n+p such that its projection on R n approximates S E = {(y, u) R n R p A T y + B T u c} such that E y = Proj y E = {y (y, u) E for some u} S Optimizing over projection E y is not more difficult than on E: min f(y) s.t. y S min f(y) s.t. y E y min f(y) s.t. (y, u) E GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 9

15 Motivation Approximations: direct vs. extended formulations Direct vs extended: example Assume we want polyhedral approximation of disc {(x, y) x 2 + y 2 1} 1. Direct approximation Use m linear inequalities based on tangents (in R 2 ) m-sided approximations π2 accuracy 2m 2 very expensive (m > 2000 for ε = 10 6 ) 2. Approximation based on extended formulation Construction by Ben-Tal and Nemirovsky: Explicit description of polytope E R 2+p with p + 1 inequalities with a projection on R 2 with 2 p sides π2 accuracy 2 p+1 cheap (p = 24 for ε = 10 6 ) Has been successfully applied to mechanical engineering problems (limit analysis) and the resolution of integer quadratic problems with linear integer programming solvers GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 10

16 Motivation Approximations: direct vs. extended formulations Ben-Tal-Nemirovsky construction (details) (u, v) approximated disc in R 2 y R m (u, v, y) P where P is a polytope in the higher-dimensional space R 2+m Let q 1 a positive parameter and consider the following system { αi+1 = α i cos π 2 i + β i sin π 2 i β i+1 α i sin π 2 i + β i cos π 2 i, 0 i < q { β q 2 sin π 2 q 1 = α q cos π 2 q + β q sin π 2 q Its projection on (α 0, β 0 ) is a regular 2 q -sided polygon, at the cost of m = 2q + 1 inequalities and 2q additional variables GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 11

17 Motivation Approximations: direct vs. extended formulations C B 0.8 B M 0.4 M A 0 0 A C C D B 0.8 D B M 0.4 M E 0 A 0 E 0 A D B C GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 12

18 Motivation Approximations: direct vs. extended formulations Our goal in this talk Generalize this quadratic by linear approximation result with arbitrary accuracy: Discover new types of extended formulations to approximate convex nonlinear (transcendental) functions and related optimization problems with arbitrary accuracy using only convex quadratic (second-order cone) inequalities Main tool: the arithmetic-geometric mean iteration GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 13

19 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Overview 1. Motivation Convex optimization: problem classes Approximations: direct vs. extended formulations 2. Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Application to approximations 3. Generalizations and conclusions Matrix version Conclusions Addendum GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

20 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Arithmetic-geometric mean iteration Let a 0 = α, b 0 = β with α > β > 0 and define the iteration a n+1 = a n + b n 2 and b n+1 = a n b n Inequality between arithmetic and geometric means implies α > a 1 > > a n > a n+1 > > b n+1 > b n > > b 1 > β so that sequences {a n } and {b n } must admit a joint finite limit AG(α, β), called the arithmetic-geometric mean of α and β ; already known from Gauss, but revisited recently by the Borwein brothers (late 80 s) Since AG(α, β) is (positively) homogeneous, i.e. AG(λα, λβ) = λag(α, β) for any λ > 0, one only has to consider AG(1, β), which is a concave function. GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 14

21 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Arithmetic-geometric mean β AG(1, β) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 15

22 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Arithmetic-geometric mean β AG(1, β) (between arithmetic and geometric means) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 16

23 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Properties of AGM iterates Denote AG k (1, β) = a k, i.e. the k th AGM iterate starting from (1, β): AG k (α, β) AG(1, β) where convergence is quadratic i.e. the error is squared after each iteration, which implies 0 AG k (α, β) AG(1, β) C 2k Function AG(1, β) can be defined in terms of a complete elliptic integral of the first kind AG(1, β) = π 2I(1, β) with I(α, β) = π 2 0 dθ α 2 cos 2 θ + β 2 sin 2 θ for which there exists no closed form in terms of elementary functions GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 17

24 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Properties of elliptic integral I(α, β) I(α, β) = π 2 0 dθ α 2 cos 2 θ + β 2 sin 2 θ = Some careful but straightforward computations show that I( a + b 2, ab) = I(a, b) 0 dx (x 2 + a 2 )(x 2 + b 2 ) which implies (since I(a n, b n ) must be independent from n) and therefore I(a 0, b 0 ) = I(a n, b n ) = lim n I(a n, b n ) = I(L, L) = π 2L AG(1, β) = π 2I(1, β) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 18

25 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration From an algorithm to an extended formulation Computing the AGM only requires linear and quadratic operations But our goal is not to compute the AGM for a given value of β, but for any value of β (β is potentially a variable in an optimization model) The computation of AG k (1, β) must therefore be embedded into the optimization model, which leads to an extended formulation How to convert an algorithm into an extended formulation? operations become equalities intermediate results become additional variables every operation must preserve convexity, i.e. equalities can only be linear unless you can prove they can be relaxed into inequalities and those inequalities can only be in the form convex concave The end result is an extended formulation for the epigraph/hypograph of the (convex/concave) function you want to compute (approximately) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 19

26 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration A quadratic extended formulation for the AGM For each value of k, function β AG k (1, β) is concave and its hypograph H k = {(β, t) AG k (1, β) t}, a convex set, admits the following quadratic extended formulation a 0 = 1, b 0 = β a n = a n 1 + b n 1, b n a n 1 b n 1 1 n k 2 a k = t Each quadratic inequality X Y Z is convex and corresponds to a second-order cone: X 2 Y Z X 2 + ( Y Z 2 ) 2 ( Y +Z 2 ) 2 (slightly more general than convex quadratic inequalities, but still convex) Hypograph of AG(1, β) can be approximated with arbitrary accuracy using only quadratic inequalities ; similarly for convex epigraph of I(1, β) (using one additional quadratic inequality π 2a k t) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 20

27 Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Arithmetic-geometric mean β AG(1, β) and approximations β AG k (1, β) for k = 1, 2, 3, GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 21

28 Quadratic approximations of convex optimization problems Application to approximations Overview 1. Motivation Convex optimization: problem classes Approximations: direct vs. extended formulations 2. Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Application to approximations 3. Generalizations and conclusions Matrix version Conclusions Addendum GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

29 Quadratic approximations of convex optimization problems Application to approximations First example: computation of π One can show that when (α, β) = (1, 1 2 ), AGM iterates satisfy 2a 2 k 1 k i=0 2i c 2 i π when k (where c i = a i b i ) 2 Therefore arbitrary accuracy approximations of π can be computed with a rational (quadratic) second-order cone optimization problem k approximation correct digits π = Three inequalities suffice for near double-precision floating-point accuracy GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 22

30 Quadratic approximations of convex optimization problems Application to approximations Second application: transcendental functions One can show that I(1, x) gives a good approximation of log( 4 x ) near the origin [ lim log ( 4 ) ] I(1, x) = 0 x 0 + x Moreover, for each m 3, one has for all 0 < x < 1 log x ( I(1, 10 m ) I(1, x10 m ) ) < m10 2(m 1) i.e. log x can be approximated very accurately using function I(1, β), which can itself be cheaply approximated by a few quadratic inequalities. cheap and accurate quadratic approximation of the logarithm GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 23

31 Quadratic approximations of convex optimization problems Application to approximations Quadratic extended formulation for the logarithm Main result: the hypograph of the logarithm {(x, t) R ++ R s.t. log x t} can be approximated with arbitrary accuracy using the following k-step quadratic extended formulation for sufficiently large values of m and k { (x, a 0, b 0, a 1, b 1,..., a k, b k, u, t) R ++ R 2k+4 s.t. a 0 = 1 and b 0 = x10 m a n = a n 1 + b n 1 1 n k 2 b n a n 1 b n 1 1 n k π (a k + b k )u } I(1, 10 m ) u = t where the k + 1 inequalities can be formulated with second-order cones GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 24

32 Quadratic approximations of convex optimization problems Application to approximations log vs. approximation for k = 3 iterations and m = 2 (loglinear plot) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 25

33 Quadratic approximations of convex optimization problems Application to approximations Approximation error for k = 3 iterations and m = 2 (loglog plot) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 26

34 Quadratic approximations of convex optimization problems Application to approximations Approximation for k = 5 iterations and m = 4 (loglinear plot) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 27

35 Quadratic approximations of convex optimization problems Application to approximations Approximation error for k = 5 iterations and m = 4 (loglog plot) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 28

36 Quadratic approximations of convex optimization problems Application to approximations Approximation error for k = 7 iterations and m = 7 (loglog plot) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 29

37 Quadratic approximations of convex optimization problems Application to approximations Logarithm brings many other transcendental functions Since constraint log x t can be approximated, so can exponential e t x (inverse has same graph), its conic hull log( x u ) t u and entropy u log u t (using conic hull with x = 1) Convex powers can also obtained: x p t (with p 1) is equivalent to p log x log t, itself equivalent to the pair of constraints x log x and u/x log(t/x) (another lifting) u p 1 Hyperbolic and inverse functions, such as cosh x = 1 2 (ex + e x ) and cosh 1 z = log(z + z 2 1) can also be similarly approximated (e.g. e x t 1, e x t 2 and t 1 + t 2 = 2t) ; another useful example is the Lambert W function (inverse of xe x, with no closed-form) A whole class of convex optimization problems involving powers and exponentials can be approximated (including geometric optimization, l p norm-optimization, entropy optimization, analytic centering, etc.) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 30

38 Generalizations and conclusions Matrix version Overview 1. Motivation Convex optimization: problem classes Approximations: direct vs. extended formulations 2. Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Application to approximations 3. Generalizations and conclusions Matrix version Conclusions Addendum GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

39 Generalizations and conclusions Matrix version Possible generalizations Arithmetic-geometric mean iteration can be generalized to different settings: A complex variant of the iteration can be used to approximate trigonometric functions (via exponential with imaginary argument): can this approach also be translated into convex quadratic inequalities? I don t know A matrix variant of the iteration can be used to approximate the matrix logarithm/exponential function (on positive definite matrices): can this approach also be translated into convex quadratic inequalities? linear matrix inequalities? quadratic matrix inequalities? I know how to do it with (lifted) LMIs GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 31

40 Generalizations and conclusions Matrix version Third application: matrix logarithm The arithmetic-geometric mean iteration is well defined for (commuting) symmetric positive semidefinite matrices! Let A 0 = A and B 0 = B symmetric positive definite commuting matrices ; define the iteration A n+1 = A n + B n 2 and B n+1 = A n B n (note square-root is uniquely defined over symmetric positive definite matrices, and commutativity is preserved at each iteration) The nonlinear B n+1 iteration can be relaxed as a semidefinite constraint B n+1 ( ) An B A n B n n+1 0 B n+1 B n GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 32

41 Generalizations and conclusions Matrix version Integral representation and matrix logarithm Sequences {A n } and {B n } admit a joint finite limit AG M (A, B), called the matrix arithmetic-geometric mean of A and B As in the scalar case, one has AG M (A, B) = π 2 IM (A, B) 1 with (using matrix square root and inverse) I M (A, B) = 0 dx (x2 I + A 2 )(x 2 2 I + B 2 ) and I M can now be used to approximate the matrix logarithm ( ) log M X I(1, 10 m ) I I M (I, 10 m X) (where I is the identity matrix) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 33

42 Generalizations and conclusions Matrix version Example Simply computing the matrix logarithm of a given matrix M can be done using max tr T such that log M X T where the nonlinear constraint can be approximated by A 0 = I and B 0 = 10 m X A n = A n 1 + B n 1 ( ) 2 An B n n k B n+1 B n ( ) Ak + B k I 1 0 I π U I(1, 10 m ) I U = T 1 n k GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 34

43 Generalizations and conclusions Matrix version A (recorded) MATLAB demo Output of a 10-line MATLAB script using the YALMIP toolbox (Löfberg) and SeDuMi solver (AdvOL-Mcmaster) Approximation of 5 x 5 matrix logarithm with 6-step matrix AGM (param m=5): M = truelog = aproxlog = err = e-004 GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 35

44 Generalizations and conclusions Matrix version Detailed GeoLMI timing Toulouse, (sec) November 2009 Approximating convex problems with AGM iterations 36 >> M=randn(5,5)*2;log_demo(M *M,6); SeDuMi 1.21 by AdvOL, and Jos F. Sturm, Alg = 2: xz-corrector, theta = 0.250, beta = Put 150 free variables in a quadratic cone eqs m = 180, order n = 63, dim = 752, blocks = 8 nnz(a) = , nnz(ada) = 28350, nnz(l) = it : b*y gap delta rate t/tp* t/td* feas cg cg prec 0 : 1.41E : 4.93E E E : 4.80E E E : 3.51E E E : 2.80E E E : 2.42E E E : 2.19E E E : 2.05E E E : 1.55E E E : 1.54E E E : 1.54E E E : 1.54E E E : 1.54E E E-010 iter seconds digits c*x b*y Inf e e+000 Ax-b = 2.1e-009, [Ay-c]_+ = 2.8E-010, x = 9.7e+002, y = 1.6e+000

45 Generalizations and conclusions Matrix version function log_demo(m, k) n=size(m,2); A0=M/10^5;B0=eye(n); for i=1:k A{i}=sdpvar(n,n); B{i}=sdpvar(n,n); end cons = set(a{1}==(a0+b0)/2) + set([a0 B{1};B{1} B0]>0); for i=2:k cons = cons + set(a{i}==(a{i-1}+b{i-1})/2) + set([a{i-1} B{i};B{i} B{i-1}]>0); end L = sdpvar(n); cons = cons + [(A{k}+B{k})/pi eye(n) ; eye(n) *eye(n)-L] solvesdp(cons, -trace(l)); disp(sprintf( \napproximation of %d x %d matrix logarithm with %d-step matrix AGM:, M truelog=logm(m) aproxlog=double(l) err=norm(truelog-aproxlog) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 37

46 Generalizations and conclusions Conclusions Overview 1. Motivation Convex optimization: problem classes Approximations: direct vs. extended formulations 2. Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Application to approximations 3. Generalizations and conclusions Matrix version Conclusions Addendum GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

47 Generalizations and conclusions Conclusions Concluding remarks Further research needed: Compare with other proposed approximation techniques for elementary functions (e.g. CORDIC, Padé, Brent, etc.) Perform computational experiments with off-the-shelf quadratic solvers Test applicability to geometric optimization and (mixed) integer geometric optimization Is it possible to guarantee accuracy of the solution (e.g. error on the objective function)? Implication on the difficulty to compute exact solutions to systems of convex quadratic inequalities? Adapt Ben-Tal-Nemirovsky construction to obtain other efficient linear approximations GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 38

48 Generalizations and conclusions Addendum Overview 1. Motivation Convex optimization: problem classes Approximations: direct vs. extended formulations 2. Quadratic approximations of convex optimization problems Arithmetic-geometric mean iteration Application to approximations 3. Generalizations and conclusions Matrix version Conclusions Addendum GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations

49 Generalizations and conclusions Addendum A small addendum Huge gap in class of convex sets efficiently representable or approximable by quadratic inequalities: LMI-representable sets (i.e. solve (approximately) semidefinite optimization with second-order cone optimization) What we can do exactly with quadratic inequalities? ( ) a b 2 by 2 matrices: 0 a + c (a c, 2b) b c x 0 x 1 x n x 1 x 0 arrow matrices: x 0 (x 1,, x n ) x n x 0 1 a b and...? What about the Cayley cubic: a 1 c 0? b c 1 GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 39

50 Generalizations and conclusions Addendum Cayley cubic and generalizations After trying for a while without success, I started to think it was impossible... until Y. Nesterov showed me how to do it: 1 a b a 1 c 0 a 2 + b 2 + c 2 2abc 1 and 1 a, b, c 1 b c 1 (a + b)2 1 + c + (a b)2 1 c 2 and 1 c 1 (a + b) 2 (1 + c)u, (a b) 2 (1 c)v, 1 c 1 and u + v = 2 ( ) ( ) 1 + c a + b 1 c a b 0, 0 and u + v = 2 a + b u a b u (this is an extended formulation - the two others were direct) GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 40

51 Generalizations and conclusions Addendum Can the Cayley cubic representation be generalized? Non constant diagonal elements: α a b α a b I can handle a β c 0 but not a β c 0 b c β b c γ (but maybe somebody can show me how to do the latter...) Higher dimensions: Question: Why is there a trick for Cayley and (apparently) not for a general 3 3 symmetric matrix? My (current) answer: Cayley and the first example above works ( because) they admit a 2 2 minor with constant eigenvectors β c always admits (1, 1) and ( 1, 1) as eigenvectors, c β and hence is diagonalizable in a constant basis can be generalized to handle larger matrices with that constant-basis structure augmented by one row/column GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 41

52 Generalizations and conclusions Addendum Thank you for your attention! GeoLMI Toulouse, November 2009 Approximating convex problems with AGM iterations 42