Non-rigid shape correspondence by matching semi-local spectral features and global geodesic structures

Transcription

1 Non-rigid shape correspondence by matching semi-local spectral features and global geodesic structures Anastasia Dubrovina Technion Israel Institute of Technology

2 Introduction Correspondence detection in contet of 3D shape processing Shape processing: recognition, retrieval, morphing, similarity and self-similarity detection, etc. Correspondence detection The Stanford 3D scanning repository TOSCA high resolution database 2

3 Introduction Rigid vs. non-rigid world Rigid transformations: rotation, translation, reflections Non-rigid isometric transformations: Rigid transformations + bending + twisting, without stretching or tearing 3

4 Introduction Correspondence detection, the shapes we want to match : correspondence function: for each its corresponding point is? 4

5 Introduction Non-rigid correspondence detection: previous work Matching by metric structure comparison Matching shape embeddings in some canonical domain Memoli 05 Memoli 07 Elad 03, Jain 07, Rustamov 07, Mateus 08, Lipman 09 Matching by computing set of rigid transformations Zhang 08, Huang 09 Bronstein 06 Tevs 09 Descriptor-based matching The proposed method Hybrid approaches Leordeanu 05, Torresani 08, Hu 09, Thorstensen 09 Similarity and symmetry detection Raviv 07, Ovsjanikov 08 Sun 08 Zaharescu 09 5

6 Problem formulation Previous work: Close-up Matching by metric structure comparison Anguelov 04 Memoli 05 Memoli 07 Bronstein 06 Tevs 09 Bronstein 09 Decriptor-based matching Zhang 08 Sun 08 Zaharescu 09 6

7 Problem formulation Metric structure: definition With each, associate distance measure d : d, y y d y, y d, How to compare two metric spaces and d?, 7

8 Problem formulation Metric structure comparison: previous work Memoli 05 Memoli 07 Bronstein 06 Compare metrics using Gromov-Hausdorf distance (Gromov 81) GH, inf H, d d f g f: g : f, g,? NP-hard combinatorial problem (MS) or not conve continuous formulation (BBK) Find initialization using pointwise surface descriptors! 8

9 Problem formulation Metric structure comparison: proposed formulation Map onto : d, y y,, d y y d The mapping distortion: d, d, Distortion, 9

10 Problem formulation Descriptor-based matching Zhang 08, Sun 08, Zharescu 09 Descriptor eamples: teture, curvature, Mesh HoG, Heat Kernel Signatures, GPS :, f y, f Pointwise distortion: f Distortion f 10

11 Problem formulation Metric structure vs. descriptor-based matching Metric structure-based matching: Preserves global pairwise structure Requires good initialization Descriptor-based matching: Fast Local, does not preserve global structure Combine into one distortion measure:,, Distortion f f d d, Berg 05, Torresani 08, Hu 09, Bronstein 10 11

12 Problem formulation The proposed framework: Discrete setting 1 : y 1 N points P N points Represent correspondence by a binary matri P P ij 1, yj corresponds to 0, otherwise i N N 12

13 Problem formulation The proposed framework: Discrete setting,, Distortion f f d d, f i f yj d i, m d y j, yn Distortion P P P P ij ij mn i i, m y y, y j j n Denote each correspondence single inde : i, j by Pij pk k f i f yj d i, m d y j, yn b k Q kl Integer Quadratic Programming T T Distortion p b p p Qp 13

14 Problem formulation The proposed framework: Schematic view Descriptors Distance measure Combined distortion measure Distortion T b T p p p Qp,,,,,,, Application dependent can be used with d any descriptors d y y and metric! f, f y,, y y y We need isometry invariant features and distance measure Minimize to obtain the correspondence s.. t p 1 * P * : 14

15 Problem formulation Distance measure Geodesic distance d, y y d y, y Other distance measures: diffusion distance (Berard 94, Coiffman 06), commute time distance (Qui 07) 15

16 Surface descriptors Isometry invariant surface descriptors Use the Laplace-Beltrami operator M (generalization of the Laplacian for surfaces) Defined by intrinsic geometry of M Isometry invariant M admits orthogonal discrete eigendecomposition M

17 Surface descriptors The proposed descriptors Similar to the GPS signature (Rustamov 07), we use f 1, 2,..., K, K f K y Eigenfunctions are defined up to a sign: (Jain 07, Mateus 08, Ovsjanikov 08) 1 y 2 y 3 y [ + ] f y s y s y s y s,,...,,,? K K K 17

18 Surface descriptors The proposed descriptors: properties Ovsjanikov et al. 08: An intrinsic symmetry of an object induces an etrinsic symmetry (either rotation or reflection) of its GPS embedding. Therefore: intrinsic symmetry of implies etrinsic symmetry of eigenfunctions (corresponding to eigenvalues with unit multiplicity) or, symm Symmetric Anti-symmetric 18

19 Surface descriptors How many sign sequences are there? Primary correspondence s (1) = [ + ] s (2) = [ + ] Symmetric correspondence We want to estimate all possible sign sequences

20 Surface descriptors Sign sequence estimation algorithm I. Find mied set of correspondences: both primary and symmetrical f Absolute values f f f II. Cluster correspondences into (# of symmetries + 1) groups C 2 C 1 III. Find sign sequence induced by each cluster C 1 s 1,, C 2 s 2,, 20

21 Algorithm outline The proposed framework Descriptors based on the Laplace-Beltrami operator j, f, f y,, y j Geodesic distance (can t distinguish between different correspondences),,,,,,, d d y y y y Combined distortion measure Distortion T b T p p p Qp Minimize to obtain the correspondence s.. t p 1 * P * : 21

22 IQP compleity Integer Quadratic Programming NP-hard We approimate its solution using MIQP solver by Bemporad 04 Compleity: eponential in number of correspondences Calculation time, in seconds, for 20 correspondences between different models from the TOSCA database, on a laptop with Intel Core 2 Duo T7500 processor and 2GBytes memory. Future research: reduce compleity by eploring surface connectivity information 22

23 Results Test cases Rigid transformations 100% accuracy Approimately isometric transformations Intrinsically symmetry shapes Different sampling and triangulation Scaling SHREC database 23

24 Results Different sampling and triangulation Original (53K) vs. 25K Original (53K) vs. 5K 24

25 Results Approimately isometric transformations 25

26 Results Scaling 26

27 Results SHREC database Bronstein et al

28 Summary We proposed a framework for shape matching based on: Metric structures Pointwise surface descriptors Formulated matching as Integer Quadratic Programming Showed that intrinsic symmetries imply eistence of more than one possible correspondence Showed how to find all those correspondences Future research: efficient optimization, other descriptors and metrics 28

29 Thank you! Questions 29