Image Similarities for Learning Video Manifolds. Selen Atasoy MICCAI 2011 Tutorial

Similar documents
Non-linear dimension reduction

Robust Pose Estimation using the SwissRanger SR-3000 Camera

Locality Preserving Projections (LPP) Abstract

RDRToolbox A package for nonlinear dimension reduction with Isomap and LLE.

Data fusion and multi-cue data matching using diffusion maps

Locality Preserving Projections (LPP) Abstract

Large-Scale Face Manifold Learning

The Analysis of Parameters t and k of LPP on Several Famous Face Databases

Technical Report. Title: Manifold learning and Random Projections for multi-view object recognition

Manifold Clustering. Abstract. 1. Introduction

Dimension Reduction CS534

SELECTION OF THE OPTIMAL PARAMETER VALUE FOR THE LOCALLY LINEAR EMBEDDING ALGORITHM. Olga Kouropteva, Oleg Okun and Matti Pietikäinen

Isometric Mapping Hashing

Globally and Locally Consistent Unsupervised Projection

Global versus local methods in nonlinear dimensionality reduction

Remote Sensing Data Classification Using Combined Spectral and Spatial Local Linear Embedding (CSSLE)

CIE L*a*b* color model

Hyperspectral image segmentation using spatial-spectral graphs

Selecting Models from Videos for Appearance-Based Face Recognition

Differential Structure in non-linear Image Embedding Functions

Locally Linear Landmarks for large-scale manifold learning

Manifold Learning for Video-to-Video Face Recognition

Sparse Manifold Clustering and Embedding

Learning a Manifold as an Atlas Supplementary Material

Non-linear CCA and PCA by Alignment of Local Models

Sensitivity to parameter and data variations in dimensionality reduction techniques

Global versus local methods in nonlinear dimensionality reduction

Non-Local Manifold Tangent Learning

DIMENSION REDUCTION FOR HYPERSPECTRAL DATA USING RANDOMIZED PCA AND LAPLACIAN EIGENMAPS

Generalized Principal Component Analysis CVPR 2007

Réduction de modèles pour des problèmes d optimisation et d identification en calcul de structures

COMPRESSED DETECTION VIA MANIFOLD LEARNING. Hyun Jeong Cho, Kuang-Hung Liu, Jae Young Park. { zzon, khliu, jaeypark

Lecture Topic Projects

Assessment of Dimensionality Reduction Based on Communication Channel Model; Application to Immersive Information Visualization

School of Computer and Communication, Lanzhou University of Technology, Gansu, Lanzhou,730050,P.R. China

Head Frontal-View Identification Using Extended LLE

Stratified Structure of Laplacian Eigenmaps Embedding

Advanced Machine Learning Practical 2: Manifold Learning + Clustering (Spectral Clustering and Kernel K-Means)

Iterative Non-linear Dimensionality Reduction by Manifold Sculpting

Video sequences association for people re-identification across multiple non-overlapping cameras

Keywords: Linear dimensionality reduction, Locally Linear Embedding.

Appearance Manifold of Facial Expression

Alternative Model for Extracting Multidimensional Data Based-on Comparative Dimension Reduction

Manifold Learning for Image-Based Breathing Gating with Application to 4D Ultrasound

Learning Appearance Manifolds from Video

Dimension reduction for hyperspectral imaging using laplacian eigenmaps and randomized principal component analysis

Towards Multi-scale Heat Kernel Signatures for Point Cloud Models of Engineering Artifacts

Dimension Reduction of Image Manifolds

Dynamic Facial Expression Recognition Using A Bayesian Temporal Manifold Model

A Stochastic Optimization Approach for Unsupervised Kernel Regression

Modelling and Visualization of High Dimensional Data. Sample Examination Paper

Dimension reduction for hyperspectral imaging using laplacian eigenmaps and randomized principal component analysis:midyear Report

Topology-Invariant Similarity and Diffusion Geometry

Texture Representations Using Subspace Embeddings

Multidimensional scaling Based in part on slides from textbook, slides of Susan Holmes. October 10, Statistics 202: Data Mining

Reconstruction of Images Distorted by Water Waves

Spectral Clustering X I AO ZE N G + E L HA M TA BA S SI CS E CL A S S P R ESENTATION MA RCH 1 6,

A Supervised Non-linear Dimensionality Reduction Approach for Manifold Learning

Application of Spectral Clustering Algorithm

A Taxonomy of Semi-Supervised Learning Algorithms

Semi-supervised Learning by Sparse Representation

A Discriminative Non-Linear Manifold Learning Technique for Face Recognition

Classification Performance related to Intrinsic Dimensionality in Mammographic Image Analysis

LEARNING ON STATISTICAL MANIFOLDS FOR CLUSTERING AND VISUALIZATION

Clustering and Dimensionality Reduction

Learning Manifolds in Forensic Data

Data Mining: Data. Lecture Notes for Chapter 2. Introduction to Data Mining

Recognizing Handwritten Digits Using the LLE Algorithm with Back Propagation

Discovering Shared Structure in Manifold Learning

Robust Locally Linear Embedding

Non-Local Estimation of Manifold Structure

Alternative Statistical Methods for Bone Atlas Modelling

Proximal Manifold Learning via Descriptive Neighbourhood Selection

Research Article Chaotic Neural Network for Biometric Pattern Recognition

A Geometric Perspective on Machine Learning

Non-Local Estimation of Manifold Structure

A Course in Machine Learning

Local Discriminant Embedding and Its Variants

Representation Discovery in Planning using Harmonic Analysis

Evgeny Maksakov Advantages and disadvantages: Advantages and disadvantages: Advantages and disadvantages: Advantages and disadvantages:

Learning Multiple Tasks using Manifold Regularization

Dimensionality reduction and visualisation of geoscientific images via locally linear embedding 1

Face Recognition using Tensor Analysis. Prahlad R. Enuganti

Image Processing. Image Features

Supervised Isomap with Dissimilarity Measures in Embedding Learning

Learning Semantic Features for Action Recognition via Diffusion Maps

Linear and Non-linear Dimentionality Reduction Applied to Gene Expression Data of Cancer Tissue Samples

A Framework for Automated Measurement of the Intensity of Non-Posed Facial Action Units

Locality Condensation: A New Dimensionality Reduction Method for Image Retrieval

Population Modeling in Neuroscience Using Computer Vision and Machine Learning to learn from Brain Images. Overview. Image Registration

Face Recognition Using Wavelet Based Kernel Locally Discriminating Projection

Overview. Spectral Processing of Point- Sampled Geometry. Introduction. Introduction. Fourier Transform. Fourier Transform

Using Graph Model for Face Analysis

Generating Different Realistic Humanoid Motion

A kernel-based active learning strategy for content-based image retrieval

Finding Structure in CyTOF Data

Visualizing breast cancer data with t-sne

Angular Decomposition

Graph based machine learning with applications to media analytics

Enhanced Multilevel Manifold Learning

Transcription:

Image Similarities for Learning Video Manifolds Selen Atasoy MICCAI 2011 Tutorial

Image Spaces

Image Manifolds Tenenbaum2000 Roweis2000 Tenenbaum2000 [Tenenbaum2000: J. B. Tenenbaum, V. Silva, J. C. Langford: A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2000.] [Roweis2000: S. T. Roweis, L. K. Saul: Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500), 2000]

Video Manifolds Pless2003 Atasoy2010 [Pless2003: R. Pless: Using Isomap to Explore Video Sequences: ICCV, 2003.] [Atasoy2010: S. Atasoy, D. Mateus, J. Lallemand, A. Meining, G.Z. Yang, N. Navab: Endoscopic Video Manifolds, MICCAI, 2010.] [Atasoy2011: S. Atasoy, D. Mateus, A. Meining, G.Z. Yang, N. Navab: Targeted Optical Biopsies for Surveillance Endoscopies, MICCAI, 2011.]

Theoretical Background

High dimensional data points lying on or near a manifold Manifold Learning Low dimensional representation Find a mapping that best preserves???...

1. Define a matrix based on the relations between data points Manifold Learning A General Recipe 2. Compute the eigenvectors & eigenvalues 3. Embed each sample

Manifold Learning A General Recipe Method Operator/Matrix Preserved Objective Function PCA Covariance matrix Variance of the dataset / Euclidean distances between data points Laplacian Eigenmaps Graph Laplacian Distances within the local neighbourhood of each data point ISOMAP Geodesic distance matrix Geodesic distances between data points LLE Reconstruction weights Reconstruction weights within the local neighbourhood of each data point

Rayleigh-Ritz Theorem: Manifold Learning Why does it work? eigenvalues eigenvectors Recall: Scalar product: Scalar product in H: Norm: Norm in H:

Manifold Learning Why does it work? Discrete Domain vectors Continuous Domain functions Schwarz s Kernel Theorem: Each linear operator is given as an integration against a unique kernel. That kernel is the impulse response of the linear system to an impulse (a delta function).

Manifold Learning Why does it work? Discrete Domain vectors Continuous Domain functions

Manifold Learning Why does it work? Discrete Domain vectors Continuous Domain functions The matrix H defines: which operator is applied which (Hilbert) space we are working in which quantity will be conserved

Laplacian Eigenmaps

Solve Manifold Learning Laplacian Eigenmaps Find the eigenvectors of the graph Laplacian Equivalent to solving the Helmholtz Equation [Belkin2003: M. Belkin, P. Niyogi: Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation, 15(6), 1373-1396. MIT Press, 2003]

Manifold Learning Laplacian Eigenmaps - Interpretation [Chladni1787] [Levy2010] [Levy2010] [Chladni1787: E. Chladni: Discoveries in the Theory of Sound, 1787.] [Levy2010: B. Levy: Spectral Geometry Processing: ACM SIGGRAPH Course Notes, 2010.]

Non-linear Manifold Learning Laplacian Eigenmaps - Interpretation Manifold learning as bending, stretching without cutting or creating wholes Vibrational modes are preserved while bending the manifold

Endoscopic Video Manifolds (EVMs)

Endoscopic Video Manifolds Challenges Clustering Uninformative Frames

Endoscopic Video Manifolds Clustering Uninformative Frames

Endoscopic Video Manifolds Clustering Uninformative Frames Informative frame & power spectrum Uninformative frame & power spectrum Informative frame Uninformative frame [Atasoy2010: S. Atasoy, D. Mateus, J. Lallemand, A. Meining, G.Z. Yang, N. Navab: Endoscopic Video Manifolds, MICCAI, 2010.]

Endoscopic Video Manifolds Clustering Uninformative Frames [Atasoy2010: S. Atasoy, D. Mateus, J. Lallemand, A. Meining, G.Z. Yang, N. Navab: Endoscopic Video Manifolds, MICCAI, 2010.]

Significant change in endoscope viewpoint Endoscopic Video Manifolds Challenges Small overlap between frames showing the same scene Scenes do not necessarily contain distinctive features

Endoscopic Video Manifolds Clustering Endoscopic Scenes Euclidean Distance Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Cluster 9 Cluster 10 0.04 0.02 0-0.02-0.04-0.06-0.08 0.1 0.05 Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Cluster 9 Cluster 10-0.1-0.12-0.06-0.04-0.02 0 0.02 0.04 0.06 0.08-0.1 0-0.05 338 frames 150 frames 137 frames 102 frames 98 frames 78 frames 71 frames 71 frames 44 frames 38 frames [Belkin2003: M. Belkin, P. Niyogi: Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation, 15(6), 1373-1396. MIT Press, 2003] [Atasoy2010: S. Atasoy, D. Mateus, J. Lallemand, A. Meining, G.Z. Yang, N. Navab: Endoscopic Video Manifolds, MICCAI, 2010.]

Endoscopic Video Manifolds Clustering Endoscopic Scenes Euclidean Distances Cluster 3 0.04 0.02 0-0.02-0.04-0.06-0.08 0.1 0.05 Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Cluster 9 Cluster 10-0.1-0.12-0.06-0.04-0.02 0 0.02 0.04 0.06 0.08-0.1 0-0.05

Endoscopic Video Manifolds Clustering Endoscopic Scenes - NCC Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Cluster 9 Cluster 10 0.04 0.02 0-0.02-0.04-0.06-0.08-0.1-0.12 0.1 Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Cluster 9 Cluster 10 0.05 0-0.05-0.1-0.08-0.06-0.04-0.02 0 0.02 0.04 389 frames 137 frames 103 frames 98 frames 85 frames 82 frames 81 frames 64 frames 44 frames 44 frames [Atasoy2010: S. Atasoy, D. Mateus, J. Lallemand, A. Meining, G.Z. Yang, N. Navab: Endoscopic Video Manifolds, MICCAI, 2010.]

Endoscopic Video Manifolds Clustering Endoscopic Scenes - NCC Euclidean Distance Normalized Cross Correlation [Atasoy2010: S. Atasoy, D. Mateus, J. Lallemand, A. Meining, G.Z. Yang, N. Navab: Endoscopic Video Manifolds, MICCAI, 2010.]

Endoscopic Video Manifolds Clustering Endoscopic Scenes - NCC

Endoscopic Video Manifolds Clustering Endoscopic Scenes with Temporal Constraints Change the adjacency matrix to include temporal constraints [Atasoy2011: S. Atasoy, D. Mateus, A. Meining, G.Z. Yang, N. Navab: Targeted Optical Biopsies for Surveillance Endoscopies, MICCAI, 2011]

Endoscopic Video Manifolds Clustering Endoscopic Scenes with Temporal Constraints Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Cluster 9 Cluster 10 0.04 0.02 0-0.02-0.04-0.06 Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 Cluster 9 Cluster 10-0.08 0-0.05-0.1 0.1 0.08 0.06 0.04 0.02 0-0.02-0.04 0.15 0.1 0.05 344 frames 143 frames 126 frames 120 frames 112 frames 88 frames 55 frames 53 frames 43 frames 43 frames [Atasoy2011: S. Atasoy, D. Mateus, A. Meining, G.Z. Yang, N. Navab: Targeted Optical Biopsies for Surveillance Endoscopies, MICCAI, 2011]

Acknowledgements Prof. Nassir Navab Prof. Guang-Zhong Yang Prof. Alexander Meining Dr. Diana Mateus Thank you for your attention!!!

2024 © technodocbox.com Privacy Policy | Terms of Service | Feedback