Convex Optimization Euclidean Distance Geometry 2ε

Transcription

1 Convex Optimization Euclidean Distance Geometry 2ε In my career, I found that the best people are the ones that really understand the content, and they re a pain in the butt to manage. But you put up with it because they re so great at the content. And that s what makes great products; it s not process, it s content. Steve Jobs, Overview 21 2 Convex geometry Convex set Vectorized-matrix inner product Hulls Halfspace, Hyperplane Subspace representations Extreme, Exposed Cones Cone boundary Positive semidefinite (PSD) cone Conic independence (c.i.) When extreme means exposed Convex polyhedra Dual cone & generalized inequality Geometry of convex functions Convex function Practical norm functions, absolute value Powers, roots, and inverted functions Affine function Epigraph, Sublevel set Gradient Convex matrix-valued function Quasiconvex Salient properties

2 10 CONVEX OPTIMIZATION EUCLIDEAN DISTANCE GEOMETRY 2ε 4 Semidefinite programming Conic problem Framework Rank reduction Rank-constrained semidefinite program Constraining cardinality Cardinality and rank constraint examples Constraining rank of indefinite matrices Convex Iteration rank Euclidean Distance Matrix EDM First metric properties fifth Euclidean metric property EDM definition Invariance Injectivity of D & unique reconstruction Embedding in affine hull Euclidean metric versus matrix criteria Bridge: Convex polyhedra to EDMs EDM-entry composition EDM indefiniteness List reconstruction Reconstruction examples Fifth property of Euclidean metric Cone of distance matrices Defining EDM cone Polyhedral bounds EDM cone is not convex EDM definition in 11 T Correspondence to PSD cone S+ N Vectorization & projection interpretation A geometry of completion Dual EDM cone Theorem of the alternative Postscript Proximity problems First prevalent problem: Second prevalent problem: Third prevalent problem: Conclusion

3 CONVEX OPTIMIZATION EUCLIDEAN DISTANCE GEOMETRY 2ε 11 A Linear algebra 513 A.1 Main-diagonal δ operator, λ, tr, vec A.2 Semidefiniteness: domain of test A.3 Proper statements of positive semidefiniteness A.4 Schur complement A.5 Eigenvalue decomposition A.6 Singular value decomposition, SVD A.7 Zeros B Simple matrices 547 B.1 Rank-one matrix (dyad) B.2 Doublet B.3 Elementary matrix B.4 Auxiliary V -matrices B.5 Orthomatrices C Some analytical optimal results 563 C.1 Properties of infima C.2 Trace, singular and eigen values C.3 Orthogonal Procrustes problem C.4 Two-sided orthogonal Procrustes C.5 Nonconvex quadratics D Matrix calculus 577 D.1 Directional derivative, Taylor series D.2 Tables of gradients and derivatives E Projection 603 E.1 Idempotent matrices E.2 I P, Projection on algebraic complement E.3 Symmetric idempotent matrices E.4 Algebra of projection on affine subsets E.5 Projection examples E.6 Vectorization interpretation, E.7 Projection on matrix subspaces E.8 Range/Rowspace interpretation E.9 Projection on convex set E.10 Alternating projection F Notation, Definitions, Glossary 663 Bibliography 679 Index 703

4 12 CONVEX OPTIMIZATION EUCLIDEAN DISTANCE GEOMETRY 2ε

5 List of Figures 1 Overview 21 1 Sigma delta quantizer Room geometry estimation by first acoustic reflections Orion nebula Application of trilateration is localization Molecular conformation Face recognition Swiss roll USA map reconstruction Honeycomb, Hexabenzocoronene molecule Robotic vehicles Reconstruction of David David by distance geometry Convex geometry Slab Open, closed, convex sets Intersection of line with boundary Tangentials Inverse image Inverse image under linear map Tesseract Linear injective mapping of Euclidean body Linear noninjective mapping of Euclidean body Convex hull of a random list of points Hulls Two Fantopes Nuclear Norm Ball Convex hull of rank-1 matrices A simplicial cone Hyperplane illustrated H is a partially bounding line Hyperplanes in R

6 14 LIST OF FIGURES 30 Affine independence {z C a T z = κ i } Hyperplane supporting closed set Minimizing hyperplane over affine set in nonnegative orthant Maximizing hyperplane over convex set Closed convex set illustrating exposed and extreme points Two-dimensional nonconvex cone Nonconvex cone made from lines Nonconvex cone is convex cone boundary Union of convex cones is nonconvex cone Truncated nonconvex cone X Cone exterior is convex cone Not a cone Minimum element, Minimal element K is a pointed polyhedral cone not full-dimensional Exposed and extreme directions Positive semidefinite cone Convex Schur-form set Projection of truncated PSD cone Circular cone showing axis of revolution Circular section Polyhedral inscription Conically (in)dependent vectors Pointed six-faceted polyhedral cone and its dual Minimal set of generators for halfspace about origin Venn diagram for cones and polyhedra Range form polyhedron Simplex Two views of a simplicial cone and its dual Two equivalent constructions of dual cone Dual polyhedral cone construction by right angle K is a halfspace about the origin Iconic primal and dual objective functions Orthogonal cones Blades K and K Membership w.r.t K and orthant Shrouded polyhedral cone Simplicial cone K in R 2 and its dual Monotone nonnegative cone K M+ and its dual Monotone cone K M and its dual Two views of monotone cone K M and its dual First-order optimality condition

7 LIST OF FIGURES 15 3 Geometry of convex functions Convex functions having unique minimizer Minimum/Minimal element, dual cone characterization norm ball B 1 from compressed sensing/compressive sampling Cardinality minimization, signed versus unsigned variable norm variants Affine function Supremum of affine functions Epigraph Quadratic bowl gradient in R 2 evaluated on grid Quadratic function convexity in terms of its gradient Contour plot of convex real function at selected levels Iconic quasiconvex function Quasiconcave monotonic function xu Arbitrary magnitude analog filter design Semidefinite programming Venn diagram of convex program classes Visualizing positive semidefinite cone in high dimension Primal/Dual transformations Projection versus convex iteration Trace heuristic Sensor-network localization lattice of sensors and anchors for localization example lattice of sensors and anchors for localization example lattice of sensors and anchors for localization example lattice of sensors and anchors for localization example Uncertainty ellipsoids orientation and eccentricity lattice localization solution lattice localization solution lattice localization solution lattice localization solution lattice localization solution randomized noiseless sensor localization randomized sensors localization Nonnegative spectral factorization Regularization curve for convex iteration norm heuristic Sparse sampling theorem Signal dropout Signal dropout reconstruction Simplex with intersecting line problem in compressed sensing Geometric interpretations of sparse-sampling constraints Permutation matrix column-norm and column-sum constraint max cut problem

8 16 LIST OF FIGURES 114 Shepp-Logan phantom MRI radial sampling pattern in Fourier domain Aliased phantom Neighboring-pixel stencil on Cartesian grid Differentiable almost everywhere Eternity II Eternity II game-board grid Eternity II demo-game piece illustrating edge-color ordering Eternity II vectorized demo-game-board piece descriptions Eternity II difference and boundary parameter construction Sparsity pattern for composite Eternity II variable matrix MIT logo One-pixel camera One-pixel camera - compression estimates Straight line through three direction vectors by midpoint fit Euclidean Distance Matrix Convex hull of three points Complete dimensionless EDM graph Fifth Euclidean metric property Fermat point Arbitrary hexagon in R Kissing number Trilateration This EDM graph provides unique isometric reconstruction Two sensors and three anchors Two discrete linear trajectories of sensors Coverage in cellular telephone network Contours of equal signal power Depiction of molecular conformation Square diamond Orthogonal complements in S N abstractly oriented Elliptope E Elliptope E 2 interior to S Smallest eigenvalue of V T N DV N Some entrywise EDM compositions Map of United States of America Largest ten eigenvalues of V T N OV N Relative-angle inequality tetrahedron Nonsimplicial pyramid in R

9 LIST OF FIGURES 17 6 Cone of distance matrices Relative boundary of cone of Euclidean distance matrices Example of V X selection to make an EDM Vector V X spirals Three views of translated negated elliptope Halfline T on PSD cone boundary Vectorization and projection interpretation example Intersection of EDM cone with hyperplane Neighborhood graph Trefoil knot untied Trefoil ribbon Orthogonal complement of geometric center subspace EDM cone construction by flipping PSD cone Decomposing member of polar EDM cone Ordinary dual EDM cone projected on S 3 h Proximity problems Pseudo-Venn diagram for EDM Elbow placed in path of projection Convex envelope A Linear algebra Geometrical interpretation of full SVD B Simple matrices Four fundamental subspaces for any dyad Four fundamental subspaces for doublet Four fundamental subspaces for elementary matrix Gimbal D Matrix calculus Convex quadratic bowl in R 2 R E Projection Action of pseudoinverse Nonorthogonal projection of x R 3 on R(U)= R Biorthogonal expansion of point x aff K Linear regression versus principal component analysis Dual interpretation of projection on convex set Projection on orthogonal complement Projection on dual cone Projection product on convex set in subspace

10 18 LIST OF FIGURES 183 von Neumann-style projection of point b Alternating projection on two halfspaces Distance, feasibility, optimization Alternating projection on nonnegative orthant and hyperplane Geometric convergence of iterates in norm Distance between PSD cone and iterate in A Dykstra s alternating projection algorithm Polyhedral normal cones Normal cone to elliptope Normal-cone progression

11 List of Tables 2 Convex geometry Table , rank versus dimension of S 3 + faces 107 Table , maximum number of c.i. directions 122 Cone Table Cone Table S 167 Cone Table A 168 Cone Table 1* Semidefinite programming Faces of S 3 + corresponding to faces of S Euclidean Distance Matrix Précis 5.7.2: affine dimension in terms of rank 401 B Simple matrices Auxiliary V -matrix Table B D Matrix calculus Table D.2.1, algebraic gradients and derivatives 595 Table D.2.2, trace Kronecker gradients 596 Table D.2.3, trace gradients and derivatives 597 Table D.2.4, logarithmic determinant gradients, derivatives 599 Table D.2.5, determinant gradients and derivatives 600 Table D.2.6, logarithmic derivatives 600 Table D.2.7, exponential gradients and derivatives 601 Dattorro, Convex Optimization Euclidean Distance Geometry 2ε, Mεβoo, v