Manifold Learning Theory and Applications

Transcription

1 Manifold Learning Theory and Applications Yunqian Ma and Yun Fu CRC Press Taylor Si Francis Group Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business

2 Contents List of Figures List of Tables Preface Editors Contributors xi xvii xix xxi xxiii 1 Spectral Embedding Methods for Manifold Learning 1 Alan Julian Izenman 1.1 Introduction Spaces and Manifolds Topological Spaces Topological Manifolds Riemannian Manifolds Curves and Geodesies Data on Manifolds Linear Manifold Learning Principal Component Analysis Multidimensional Scaling Nonlinear Manifold Learning Isomap Local Linear Embedding Laplacian Eigenmaps Diffusion Maps Hessian Eigenmaps Nonlinear PCA Summary Acknowledgment 32 Bibliography ' 32 v

3 vi Contents 2 Robust Laplacian Eigenmaps Using Global Information 37 Shounak Roychowdhury and Joydeep Ghosh 2.1 Introduction Graph Laplacian Definitions Laplacian of Graph Sum Global Information of Manifold Laplacian Eigenmaps with Global Information Experiments LEM Results GLEM Results Summary Bibliographical and Historical Remarks 53 Bibliography 54 3 Density Preserving Maps 57 Arkadas Ozakin, Nikolaos Vasiloglou II, Alexander Gray 3.1 Introduction The Existence of Density Preserving Maps Moser's Theorem and Its Corollary on Density Preserving Maps Dimensional Reduction to IR d Intuition on Non-Uniqueness Density Estimation on Submanifolds Introduction Motivation for the Submanifold Estimator Statement of the Theorem Curse of Dimensionality in KDE Preserving the Estimated Density: The Optimization Preliminaries The Optimization Examples Summary Bibliographical and Historical Remarks 69 Bibliography 71 4 Sample Complexity in Manifold Learning 73 Hariharan Narayanan 4.1 Introduction : Sample Complexity of Classification on a Manifold Preliminaries Remarks Learning Smooth Class Boundaries Volumes of Balls in a Manifold Partitioning the Manifold Constructing Charts by Projecting onto Euclidean Balls Proof of Theorem Sample Complexity of Testing the Manifold Hypothesis 83

4 Contents vii 4.5 Connections and Related Work Sample Complexity of Empirical Risk Minimization 85 ' Bounded Intrinsic Curvature Bounded Extrinsic Curvature Relating Bounded Curvature to Covering Number Class of Manifolds with a Bounded Covering Number Fat-Shattering Dimension and Random Projections Minimax Lower Bounds on the Sample Complexity Algorithmic Implications fc-means Fitting Piecewise Linear Curves Summary 91 Bibliography 92 5 Manifold Alignment 95 Chang Wang, Peter Krafft, and Sridhar Mahadevan 5.1 Introduction Problem Statement Overview of the Algorithm Formalization and Analysis Loss Functions Optimal Solutions The Joint Laplacian Manifold Alignment Algorithm Variants of Manifold Alignment Linear Restriction Hard Constraints Multiscale Alignment Unsupervised Alignment Application Examples Protein Alignment Parallel Corpora. Ill Aligning Topic Models Summary Bibliographical and Historical Remarks Acknowledgments Bibliography 119 / 6 Large-Scale Manifold Learning 121 Ameet Talwalkar, Sanjiv Kumar, Mehryar Mohri, Henry Rowley 6.1 Introduction Background Notation Nystrom Method Column Sampling Method Comparison of Sampling Methods Singular Values and Singular Vectors Low-Rank Approximation Experiments Large-Scale Manifold Learning 129

5 viii Contents Manifold Learning Approximation Experiments Large-Scale Learning Manifold Evaluation Summary Bibliography and Historical Remarks 140 Bibliography Metric and Heat Kernel 145 Wei Zeng, Jian Sun, Ren Guo, Feng Luo, and Xianfeng Gu 7.1 Introduction Theoretic Background Laplace-Beltrami Operator Heat Kernel Discrete Heat Kernel Discrete Laplace-Beltrami Operator Discrete Heat Kernel Main Theorem Proof Outline Rigidity on One Face Rigidity for the Whole Mesh Heat Kernel Simplification Numerical Experiments Applications Summary Bibliographical and Historical Remarks 163 Bibliography Discrete Ricci Flow for Surface and 3-Manifold 167 Xianfeng Gu, Wei Zeng, Feng Luo, and Shing-Tung Yau 8.1 Introduction Theoretic Background Conformal Deformation Uniformization Theorem Yamabe Equation Ricci Flow Quasi-Conformal Maps Surface Ricci Flow Derivative Cosine Law Circle Pattern Metric Discrete Metric Surface Discrete Ricci Flow Discrete Ricci Energy Quasi-Conformal Mapping by Solving Beltrami Equations Manifold Ricci Flow Surface and 3-Manifold Curvature Flow Hyperbolic 3-Manifold with Complete Geodesic Boundaries Discrete Hyperbolic 3-Manifold Ricci Flow Applications 194

6 Contents ix 8.6 Summary Bibliographical and Historical Remarks 202 ' Bibliography D and 3D Objects Morphing Using Manifold Techniques 209 Chafik Samir, Pierre-Antoine Absil, and Paul Van Dooren 9.1 Introduction Fitting Curves on Manifolds Morphing Techniques Morphing Using Interpolation Interpolation on Euclidean Spaces Aitken-Neville Algorithm on De Casteljau Algorithm on R m Example of Interpolations on M Generalization of Interpolation Algorithms on a Manifold M Aitken-Neville on M De Casteljau Algorithm on M Interpolation on SO(m) Aitken-Neville Algorithm on SO(m) De Casteljau Algorithm on SO(m) Example of Fitting Curves on SO(3) Application: The Motion of a Rigid Object in Space Interpolation on Shape Manifold Geodesic between 2D Shapes Geodesic between 3D Shapes Examples of Fitting Curves on Shape Manifolds D Curves Morphing D Face Morphing Summary 229 Bibliography Learning Image Manifolds from Local Features 233 Ahmed Elgammal and Marwan Torki 10.1 Introduction Joint Feature-Spatial Embedding Objective Function Intra-Image Spatial Structure Inter-Image Feature Affinity Solving the Out-of-Sample Problem Populating the Embedding Space From Feature Embedding to Image Embedding.... : Applications Visualizing Objects View Manifold What the Image Embedding Captures Object Categorization Object Localization Unsupervised Category Discovery Multiple Set Feature Matching Summary 247

7 x Contents 10.7 Bibliographical and Historical Remarks 247 Bibliography 248 r 11 Human Motion Analysis Applications of Manifold Learning 253 Ahmed Elgammal and Chan Su Lee 11.1 Introduction Learning a Simple Motion Manifold Case Study: The Gait Manifold Learning the Visual Manifold: Generative Model Solving for the Embedding Coordinates Synthesis, Recovery, and Reconstruction Factorized Generative Models Example 1: A Single Style Factor Model Example 2: Multifactor Gait Model Example 3: Multifactor Facial Expressions Generalized Style Factorization Style-Invariant Embedding Style Factorization Solving for Multiple Factors Examples Dynamic Shape Example: Decomposing View and Style on Gait Manifold Dynamic Appearance Example: Facial Expression Analysis Summary Bibliographical and Historical Remarks 273 Acknowledgment 274 Bibliography 275 Index 281