Convex Optimization Euclidean Distance Geometry 2ε

Transcription

1 Convex Optimization Euclidean Distance Geometry 2ε 1 Overview 19 2 Convex geometry Convex set Vectorized-matrix inner product Hulls Halfspace, Hyperplane Subspace representations Extreme, Exposed Cones Cone boundary Positive semidefinite (PSD) cone Conic independence (c.i.) When extreme means exposed Convex polyhedra Dual cone & generalized inequality Geometry of convex functions Convex real and vector-valued function Practical norm functions, absolute value Powers, roots, and inverted functions Affine function Epigraph, Sublevel set Gradient First-order convexity condition, real function First-order convexity condition, vector-valued Second-order convexity condition, vector-valued Convex matrix-valued function First-order convexity condition, matrix-valued Epigraph of matrix-valued function, sublevel sets Second-order convexity condition, matrix-valued Quasiconvex Salient properties Semidefinite programming Conic problem Framework Rank reduction

2 10 CONVEX OPTIMIZATION EUCLIDEAN DISTANCE GEOMETRY 2ε 4.4 Cardinality reduction Rank constraint by Convex Iteration Constraining cardinality Cardinality and rank constraint examples Quantum optimization Constraining rank of indefinite matrices Convex Iteration rank Euclidean Distance Matrix EDM First metric properties fifth Euclidean metric property EDM definition Invariance Injectivity of D & unique reconstruction Embedding in affine hull Euclidean metric versus matrix criteria Bridge: Convex polyhedra to EDMs EDM-entry composition EDM indefiniteness List reconstruction Reconstruction examples Fifth property of Euclidean metric Cone of distance matrices Defining EDM cone Polyhedral bounds EDM cone is not convex EDM definition in 11 T Correspondence to PSD cone S+ N Vectorization & projection interpretation A geometry of completion Dual EDM cone Theorem of the alternative Postscript Proximity problems First prevalent problem: Second prevalent problem: Third prevalent problem: Conclusion A Linear algebra 487 A.1 Main-diagonal δ operator, λ, tr, vec,, A.2 Semidefiniteness: domain of test A.3 Proper statements of positive semidefiniteness A.4 Schur complement A.5 Eigenvalue decomposition A.6 Singular value decomposition, SVD A.7 Zeros

3 CONVEX OPTIMIZATION EUCLIDEAN DISTANCE GEOMETRY 2ε 11 B Simple matrices 517 B.1 Rank-1 matrix (dyad) B.2 Doublet B.3 Elementary matrix B.4 Auxiliary V -matrices B.5 Orthomatrices B.6 Arrow matrix C Some analytical optimal results 535 C.1 Properties of infima C.2 Trace, singular and eigen values C.3 Orthogonal Procrustes problem C.4 Two-sided orthogonal Procrustes C.5 Quadratics D Matrix calculus 549 D.1 Gradient, Directional derivative, Taylor series D.2 Tables of gradients and derivatives E Projection 573 E.1 Idempotent matrices E.2 I P, Projection on algebraic complement E.3 Symmetric idempotent matrices E.4 Algebra of projection on affine subsets E.5 Projection examples E.6 Vectorization interpretation E.7 Projection on matrix subspaces E.8 Range, Rowspace interpretation E.9 Projection on convex set E.10 Alternating projection F Notation, Definitions, Glossary 629 Bibliography 645 Index 667

4 List of Tables 2 Convex geometry Table , rank versus dimension of S 3 + faces 97 Table , maximum number of c.i. directions 111 Cone Table Cone Table S 152 Cone Table A 153 Cone Table 1* Semidefinite programming Faces of S 3 + corresponding to faces of S Quantum impulse 328 Quantum step 330 Quantum and function Euclidean Distance Matrix Précis 5.7.2: affine dimension in terms of rank 383 B Simple matrices Auxiliary V -matrix Table B D Matrix calculus Table D.2.1, algebraic gradients and derivatives 565 Table D.2.2, trace Kronecker gradients 566 Table D.2.3, trace gradients and derivatives 567 Table D.2.4, logarithmic determinant gradients, derivatives 569 Table D.2.5, determinant gradients and derivatives 570 Table D.2.6, logarithmic derivatives 570 Table D.2.7, exponential gradients and derivatives 571 Dattorro, Convex Optimization Euclidean Distance Geometry 2ε, Mεβoo, v

5 List of Figures 1 Overview 19 1 Sigma delta quantizer Room geometry estimation by first acoustic reflections Orion nebula Application of trilateration is localization Molecular conformation Facial recognition Swiss roll USA map reconstruction Honeycomb, Hexabenzocoronene molecule Robotic vehicles Reconstruction of David David by distance geometry Convex geometry Slab Open, closed, convex sets Intersection of line with boundary Tangentials Inverse image Inverse image under linear map Tesseract Linear injective mapping of Euclidean body Linear noninjective mapping of Euclidean body Convex hull of a random list of points Hulls Two Fantopes Nuclear Norm Ball Convex hull of rank-1 matrices A simplicial cone Hyperplane illustrated H is a partially bounding line Hyperplanes in R Affine independence {z C a T z = κ i } Hyperplane supporting closed set Minimizing hyperplane over affine subset in nonnegative orthant Maximizing hyperplane over convex set Closed convex set illustrating exposed and extreme points

6 14 LIST OF FIGURES 36 Two-dimensional nonconvex cone Nonconvex cone made from lines Nonconvex cone is convex cone boundary Union of convex cones is nonconvex cone Truncated nonconvex cone X Cone exterior is convex cone Not a cone Minimum element, Minimal element K is a pointed polyhedral cone not full-dimensional Exposed and extreme directions Positive semidefinite cone Convex Schur-form set Projection of truncated PSD cone Circular cone showing axis of revolution Circular section Polyhedral inscription Conically (in)dependent vectors Pointed six-faceted polyhedral cone and its dual Minimal set of generators for halfspace about origin Venn diagram for cones and polyhedra Range form polyhedron Simplex Two views of a simplicial cone and its dual Two equivalent constructions of dual cone Dual polyhedral cone construction by right angle Orthogonal cones Blades K and K K is a halfspace about the origin Iconic primal and dual objective functions Membership w.r.t K and orthant Shrouded polyhedral cone Simplicial cone K in R 2 and its dual Monotone nonnegative cone K M+ and its dual Monotone cone K M and its dual Two views of monotone cone K M and its dual First-order optimality condition Geometry of convex functions Convex functions having unique minimizer Minimum/Minimal element, dual cone characterization Norm balls norm ball B 1 from compressed sensing/compressive sampling Cardinality minimization, phase transition, signed versus unsigned variable norm variants Affine function Supremum of affine functions Epigraph Log function constraint Quadratic bowl and 1-norm gradients in R 2 evaluated on grid Quadratic function convexity in terms of its gradient Contour plot of convex real function at selected levels

7 LIST OF FIGURES Tangent hyperplane to nonconvex surface Taxicab distance on nonuniform rectangular grid Iconic quasiconvex function Quasiconcave monotonic function xu Operational Amplifier implementation of third-order filter having a zero Mason flowgraph for operational amplifier arbitrary magnitude filter circuit Bisection method linearity Arbitrary magnitude analog filter design Sum of convex functions Semidefinite programming Venn diagram of convex program types Visualizing positive semidefinite cone in high dimension Primal/Dual transformations Projection versus convex iteration Trace heuristic Sensor-network localization lattice of sensors and anchors for localization example lattice of sensors and anchors for localization example lattice of sensors and anchors for localization example lattice of sensors and anchors for localization example Uncertainty ellipsoids orientation and eccentricity lattice localization solution lattice localization solution lattice localization solution lattice localization solution lattice localization solution randomized noiseless sensor localization randomized sensors localization Nonnegative spectral factorization Regularization curve for convex iteration norm heuristic Sparse sampling theorem Signal dropout Signal dropout reconstruction Simplex with intersecting line problem in compressed sensing Geometric interpretations of sparse-sampling constraints Permutation matrix column-norm and column-sum constraint max cut problem Shepp-Logan phantom MRI radial sampling pattern in Fourier domain Aliased phantom Neighboring-pixel stencil on Cartesian grid Differentiable almost everywhere Eternity II Eternity II game-board grid Eternity II demo-game piece illustrating edge-color ordering Eternity II vectorized demo-game-board piece descriptions Eternity II difference and boundary coefficient β construction Eternity II composite variable matrix sparsity pattern Eternity II problem visualization in three dimensions

8 16 LIST OF FIGURES 134 Eternity II permutation polyhedron vertices visualization on sphere Chimera topology for D:Wave 1152-qubit chip D:Wave Chimera chip layout MIT logo One-pixel camera One-pixel camera - compression estimates Convergence of Singular Value Decomposition by Convex Iteration Straight line through three direction vectors by midpoint fit Euclidean Distance Matrix Convex hull of three points Complete dimensionless EDM graph Fifth Euclidean metric property Fermat point Arbitrary hexagon in R Kissing number Trilateration This EDM graph provides unique isometric reconstruction Two sensors and three anchors Two discrete linear trajectories of sensors Coverage in cellular telephone network Contours of equal signal power Depiction of molecular conformation Square diamond Orthogonal complements in S N abstractly oriented Elliptope E Elliptope E 2 interior to S Smallest eigenvalue of V T N DV N Some entrywise EDM compositions Map of United States of America Largest ten eigenvalues of V T N OV N Relative-angle inequality tetrahedron Nonsimplicial pyramid in R Cone of distance matrices Relative boundary of cone of Euclidean distance matrices Example of V X selection to make an EDM Vector V X spirals Three views of translated negated elliptope Halfline T on PSD cone boundary Vectorization and projection interpretation example Intersection of EDM cone with hyperplane Neighborhood graph Trefoil knot untied Trefoil ribbon Orthogonal complement of geometric center subspace EDM cone construction by flipping PSD cone Decomposing member of polar EDM cone Ordinary dual EDM cone projected on S 3 h

9 LIST OF FIGURES 17 7 Proximity problems Pseudo-Venn diagram for EDM Elbow placed in path of projection Convex envelope A Linear algebra Geometrical interpretation of full SVD B Simple matrices Four fundamental subspaces for any dyad Four fundamental subspaces for doublet Four fundamental subspaces for elementary matrix Antireflection Gimbal Arrow matrix D Matrix calculus Convex quadratic bowl in R 2 R E Projection Action of pseudoinverse Nonorthogonal projection of x R 3 on R(U)= R Biorthogonal expansion of point x aff K Linear regression versus principal component analysis Dual interpretation of projection on convex set Projection on orthogonal complement Projection on dual cone Projection product on convex set in subspace von Neumann-style projection of point b Alternating projection on two halfspaces Distance, feasibility, optimization Alternating projection on nonnegative orthant and hyperplane Geometric convergence of iterates in norm Distance between PSD cone and iterate in A Dykstra s alternating projection algorithm Polyhedral normal cones Normal cone to elliptope Normal-cone progression