Data-Driven Shape Analysis --- Joint Shape Matching I. Qi-xing Huang Stanford University

Transcription

1 Data-Driven Shape Analysis --- Joint Shape Matching I Qi-xing Huang Stanford University 1

2 Shape matching Affine

3 Applications Shape reconstruction Transfer information Aggregate information Protein docking

4 Pair-wise matching is unreliable State-of-the-art techniques: Blended intrinsic maps [Kim et al. 11] Learning-based graph matching [Leordeanu et al. 12]

5 Piece assembly 5

6 Ambiguous matches 6

7 Additional data helps

8 Additional data helps Blended intrinsic maps [Kim et al. 11] Intermediate object Composite

9 Cycle-consistency Maps are consistent along cycles Consistent

10 Cycle-consistency Maps are consistent along cycles Inconsistent

11 Cycle-consistency Blended intrinsic maps [Kim et al. 11] Inconsistent Composite

12 Cycle-consistency Blended intrinsic maps [Kim et al. 11] Direct Consistent Composite

13 Joint matching formulation Input: Shapes Pair-wise maps (existing algorithms)

14 Joint matching formulation Input: Shapes Pair-wise maps (existing algorithms) Output: Cycle-consistent Close to the input maps NP-complete [Huber 2002]

15 Outline Three existing approaches Spanning tree based Inconsistent cycle detection Spectral techniques Convex optimization framework

16 Spanning tree based Automatic Three-dimensional Modeling from Reality, PhD thesis, D. Huber, Robotics Institute, Carnegie Mellon University, 2002

17 Spanning tree based

18 Spanning tree based Issue: A single incorrect match can destroy everything

19 Detecting inconsistent cycles large for inconsistent cycles maximize subject to Disambiguating visual relations using loop constraints, C. Zach, M. Klopscjotz, and M. POLLEFEYS, CVPR 10

20 Spectral techniques The resulting soft map From paths of length k Diffusion Multiplication and aggregation of mapping matrices An Optimization Approach for Extracting and Encoding Consistent Maps in a Shape Collection, Q. Huang, G. Zhang, L. Gao, S. Hu, A. Bustcher, L. Guibas, SIGGRAPH ASIA 12

21 Spectral techniques Compute fuzzy correspondence based on diffusion distance in graph represented by initial correspondences Exploring collections of 3d models using fuzzy correspondences, V. G. Kim, W. Li, N. Mitra, S. Diverdi, and T. Funkhouser, SIGGRAPH 12

22 Approaches we have discussed Spanning tree optimization [Huber and Hebert 03] Detecting Inconsistent Cycles [Zach et al. 10, Nguyen et al. 11] Cons: --- Many parameters to set --- Does it converge? --- Run quickly? Spectral techniques [Kim et al.12, Pachauri et al.13]

23 Outline Three existing approaches Spanning tree based Inconsistent cycle detection Spectral techniques Convex optimization framework

24 Convex optimization framework Parameter-free! Near-optimal! Efficient!

25 Basic setting G n objects, each object has m points

26 Matrix representation of maps S X= S X12 = Im X12 X1n.. T X12 Im X(n 1);n.. T T X1n X(n 1);n Im Diagonal blocks are identity matrices Off diagonal blocks are permutation matrices Symmetric

27 Cycle-consistency constraint X º0 (Positive) semidefiniteness T X Xij = Xj1 i1 2 3 Im h i 6. 7 X = 4. 5 Im Xn1 XTn1

28 Parameterization X= Xii = Im; 1 i n Xij 1 = 1; XTij 1 = 1; Xº0 X 2 f0; 1gnm 1 i<j n

29 Parametrizing maps X= Xii = Im; 1 i n Xij 1 = 1; XTij 1 = 1; Xº0 X 0 1 i<j n

30 Objective function input Xij L1-norm: P (i;j)2e Xij input kxij Xij k1

31 Convex program minimize subject to P (i;j)2e input kxij Xii = Im; Xij k1 1 i n Xij 1 = 1; XTij 1 = 1; 1 i<j n Xº0 X 0 ADMM [Boyd et al.11]

32 Convex program minimize subject to P (i;j)2e input kxij Xii = Im; Xij k1 1 i n Xij 1 = 1; XTij 1 = 1; 1 i<j n Xº0 X 0 ADMM [Boyd et al.11]

33 Deterministic guarantee Exact recovery condition: #incorrect corres. per point < algebraic-connectivity(g)/4

34 Complete input graph G 25% incorrect correspondences Worst-case scenario Two clusters of objects of equal size Wrong correspondences between objects of different clusters only (50%)

35 Randomized setting Erdős Rényi model G(n, pobs, ptrue) : pobs : the probability that two objects connect; ptrue : the probability that a pair-wise map is correct; Incorrect maps are random permutations; Theorem: The ground truth maps can be recovered w.h.p if 2n log ptrue > c pnp obs

36 Phase transition Numerical optimization: ADMM [Wen et al. 10, Boyd et al. 11] m = 16 Red: never recovers Blue: always recovers

37 Versus RPCA [Candes et al. 11] 2 3 Im h i 6. 7 X = 4. 5 Im X1n XT1n SDP RPCA

38 CMU hotel Input: 102 images (30 points per image) RANSAC [Fisher 81] Each image connects with 10 random images SDP Running time: 6m19s (3.2GHZ, single core) Input SDP RPCA Leordeanu et al % 100% 90.1% 94.8%

39 CMU house Input: 110 images (30 points per image) RANSAC [Fisher 81] Each image connects with 10 random images SDP Running time: 7m12s (3.2GHZ, single core) Input SDP RPCA Leordeanu et al % 100% 92.2% 99.8%

40 Constraint set X=6 4 minimize Xii = Im; " m 1T 1 X X 0 # 1 i n º0 #Distinctive points Im1 X12 X1n X21 Im2 X2n Xn1 Imn P (i;j)2e input kxij Xii = Im; Xij k1 1 i n Xij 1 = 1; XTij 1 = 1; 1 i<j n Xº0 X 0 SDP relaxation of MAP [kumar et al. 09]

41 Partial similarity Step I: Estimate m (#distinctive points) -- Gap in the spectrum of the input map matrix (In the same spirit as [Pachauri et al. 13]) Step II: minimize subject to P input (i;j)2e kxij Xii = Imi ; " m 1T 1 X X 0 # Xij k1 1 i n º0

42 Random model Extended model: an universe of m elements: For each set Si, each point s is included in Si with probability pset; pobs : the probability that two objects connect ptrue : the probability that a pair-wise map is correct Incorrect maps satisfy

43 Similar theoretical guarantee Theorem: The size of the universe m and the ground truth maps can be recovered w.h.p if 2n log ptrue > C1 p npobs p2 set

44 Chair 16 images Clustered SIFT features + RANSAC ( points per image) SDP Running time: 2m19s (3.2GHZ, single core) Input output

45 Building Input Output 16 images Clustered SIFT features + RANSAC ( points per image) SDP Running time: 5m07s (3.2GHZ, single core)

46 3D Benchmark Armadillo, Fish, Fourleg, Human and Hand 20 objects, 128 points per object Blended intrinsic maps as input [Kim et al. 11]

47 Next lecture Rotations [Wang and Singer 13] Functional maps [Huang et al. 14]

48 Next lecture Super-objects = = =

49 References Automatic Three-dimensional Modeling from Reality, PhD thesis, D. Huber, Robotics Institute, Carnegie Mellon University, 2002 Reassembling fractured objects by geometric matching. Q. Huang, S. Flory, N. Gelfand, M. Hofer, and H. Pottmann, SIGGRAPH 06 Disambiguating visual relations using loop constraints, C. Zach, M. Klopscjotz, and M. POLLEFEYS, CVPR 10 An optimization approach to improving collections of shape maps, A. Nguyen, M. Ben-Chen, K. Welnicka, Y. Ye, and L. Guibas, SGP 11 Exploring collections of 3d models using fuzzy correspondences, V. G. Kim, W. Li, N. Mitra, S. Diverdi, and T. Funkhouser, SIGGRAPH 12 An Optimization Approach for Extracting and Encoding Consistent Maps in a Shape Collection, Q. Huang, G. Zhang, L. Gao, S. Hu, A. Bustcher, L. Guibas, SIGGRAPH ASIA 12 Consistent Shape Maps via Semidefinite Programming, Q. Huang and L. Guibas, SGP 13 Matching Partially Similar Objects via Matrix Completion, Y. Chen, L. Guibas and Q. Huang, ICML 14