Data-Driven Geometry Processing Map Synchronization I. Qixing Huang Nov. 28 th 2018

Transcription

1 Data-Driven Geometry Processing Map Synchronization I Qixing Huang Nov. 28 th 2018

2 Shape matching Affine

3 Applications Shape reconstruction Transfer information Aggregate information Protein docking

4 Pair-wise matching is unreliable State-of-the-art techniques: Blended intrinsic maps [Kim et al. 11] Learning-based graph matching [Leordeanu et al. 12]

5 5 Piece assembly

6 6 Ambiguous matches

7 Additional data helps

8 Additional data helps Blended intrinsic maps [Kim et al. 11] Intermediate object Composite

9 Cycle-consistency Maps are consistent along cycles Consistent

10 Cycle-consistency Maps are consistent along cycles Inconsistent

11 Cycle-consistency Blended intrinsic maps [Kim et al. 11] Inconsistent Composite

12 Cycle-consistency Blended intrinsic maps [Kim et al. 11] Direct Consistent Composite

13 Input: Joint matching formulation Shapes Pair-wise maps (existing algorithms)

14 Input: Joint matching formulation Shapes Pair-wise maps (existing algorithms) Output: Cycle-consistent Close to the input maps NP-complete [Huber 2002]

15 Outline Three existing approaches Spanning tree based Inconsistent cycle detection Spectral techniques Convex optimization framework

16 Spanning tree based Automatic Three-dimensional Modeling from Reality, PhD thesis, D. Huber, Robotics Institute, Carnegie Mellon University, 2002

17 Spanning tree based

18 Spanning tree based Issue: A single incorrect match can destroy everything

19 Detecting inconsistent cycles large for inconsistent cycles maximize subject to Disambiguating visual relations using loop constraints, C. Zach, M. Klopscjotz, and M. POLLEFEYS, CVPR 10

20 Spectral techniques The resulting soft map From paths of length k Diffusion Multiplication and aggregation of mapping matrices An Optimization Approach for Extracting and Encoding Consistent Maps in a Shape Collection, Q. Huang, G. Zhang, L. Gao, S. Hu, A. Bustcher, L. Guibas, SIGGRAPH ASIA 12

21 Spectral techniques Compute fuzzy correspondence based on diffusion distance in graph represented by initial correspondences Exploring collections of 3d models using fuzzy correspondences, V. G. Kim, W. Li, N. Mitra, S. Diverdi, and T. Funkhouser, SIGGRAPH 12

22 Approaches we have discussed Spanning tree optimization [Huber and Hebert 03] Cons: --- Many parameters to set --- Does it converge? --- Run quickly? Detecting Inconsistent Cycles [Zach et al. 10, Nguyen et al. 11] Spectral techniques [Kim et al.12, Pachauri et al.13]

23 Outline Three existing approaches Spanning tree based Inconsistent cycle detection Spectral techniques Convex optimization framework

24 Convex optimization framework Parameter-free! Near-optimal! Efficient!

25 Basic setting G n objects, each object has m points

26 Matrix representation of maps S S X 12 = X = I m X 12 X 1n X12 T I m X (n 1);n X1n T. X (n 1);n T I m Diagonal blocks are identity matrices Off diagonal blocks are permutation matrices Symmetric

27 Cycle-consistency constraint X º 0 (Positive) semidefiniteness X ij = X T j1 X i1 X = I ṃ h Im X n1 i X T n1

28 Parameterization X = X ii = I m ; 1 i n X ij 1 = 1; X T ij 1 = 1; 1 i < j n X º 0 X 2 f0; 1g nm

29 Parametrizing maps X = X ii = I m ; 1 i n X ij 1 = 1; X T ij 1 = 1; 1 i < j n X º 0 X 0

30 Objective function X input ij L1-norm: P (i;j)2e kx input ij X ij k 1 X ij

31 Convex program minimize subject to P (i;j)2e kx input ij X ij k 1 X ii = I m ; 1 i n X ij 1 = 1; X T ij 1 = 1; 1 i < j n X º 0 X 0 ADMM [Boyd et al.11]

32 Convex program minimize subject to P (i;j)2e kx input ij X ij k 1 X ii = I m ; 1 i n X ij 1 = 1; X T ij 1 = 1; 1 i < j n X º 0 X 0 ADMM [Boyd et al.11]

33 Deterministic guarantee Exact recovery condition: #incorrect corres. per point < algebraic-connectivity(g)/4

34 Complete input graph G 25% incorrect correspondences Worst-case scenario Two clusters of objects of equal size Wrong correspondences between objects of different clusters only (50%)

35 Randomized setting Erdős Rényi model G(n, p obs, p true ) : p obs : the probability that two objects connect; p true : the probability that a pair-wise map is correct; Incorrect maps are random permutations; Theorem: The ground truth maps can be recovered w.h.p if p true > c log2 n p npobs

36 Phase transition Numerical optimization: ADMM [Wen et al. 10, Boyd et al. 11] m = 16 Red: never recovers Blue: always recovers

37 Versus RPCA [Candes et al. 11] X = I ṃ. X T 1n h Im X 1n i SDP RPCA

38 CMU hotel Input: 102 images (30 points per image) RANSAC [Fisher 81] Each image connects with 10 random images SDP Running time: 6m19s (3.2GHZ, single core) Input SDP RPCA Leordeanu et al % 100% 90.1% 94.8%

39 CMU house Input: 110 images (30 points per image) RANSAC [Fisher 81] Each image connects with 10 random images SDP Running time: 7m12s (3.2GHZ, single core) Input SDP RPCA Leordeanu et al % 100% 92.2% 99.8%

40 Constraint set X = I m1 X 12 X 1n X 21 I m2 X 2n X n1 I mn minimize P (i;j)2e kx input ij X ij k 1 X ii = I m ; 1 i n " # m 1 T º 0 1 X X 0 #Distinctive points X ii = I m ; 1 i n X ij 1 = 1; X T ij 1 = 1; 1 i < j n X º 0 X 0 SDP relaxation of MAP [kumar et al. 09]

41 Partial similarity Step I: Estimate m (#distinctive points) -- Gap in the spectrum of the input map matrix (In the same spirit as [Pachauri et al. 13]) Step II: minimize subject to P (i;j)2e X ii = I mi ; 1 i n " # m 1 T º 0 1 X X 0 kx input ij X ij k 1

42 Random model Extended model: an universe of m elements: For each set S i, each point s is included in S i with probability p set ; p obs : the probability that two objects connect p true : the probability that a pair-wise map is correct Incorrect maps satisfy

43 Similar theoretical guarantee Theorem: The size of the universe m and the ground truth maps can be recovered w.h.p if p true > C 1 log 2 n p npobs p 2 set

44 Chair 16 images Clustered SIFT features + RANSAC ( points per image) SDP Running time: 2m19s (3.2GHZ, single core) Input output

45 Building Input 16 images Clustered SIFT features + RANSAC ( points per image) Output SDP Running time: 5m07s (3.2GHZ, single core)

46 3D Benchmark Armadillo, Fish, Fourleg, Human and Hand 20 objects, 128 points per object Blended intrinsic maps as input [Kim et al. 11]

47 Next lecture Rotations [Wang and Singer 13] Functional maps [Huang et al. 14]

48 Next lecture Super-objects = = =

49 References Automatic Three-dimensional Modeling from Reality, PhD thesis, D. Huber, Robotics Institute, Carnegie Mellon University, 2002 Reassembling fractured objects by geometric matching. Q. Huang, S. Flory, N. Gelfand, M. Hofer, and H. Pottmann, SIGGRAPH 06 Disambiguating visual relations using loop constraints, C. Zach, M. Klopscjotz, and M. POLLEFEYS, CVPR 10 An optimization approach to improving collections of shape maps, A. Nguyen, M. Ben- Chen, K. Welnicka, Y. Ye, and L. Guibas, SGP 11 Exploring collections of 3d models using fuzzy correspondences, V. G. Kim, W. Li, N. Mitra, S. Diverdi, and T. Funkhouser, SIGGRAPH 12 An Optimization Approach for Extracting and Encoding Consistent Maps in a Shape Collection, Q. Huang, G. Zhang, L. Gao, S. Hu, A. Bustcher, L. Guibas, SIGGRAPH ASIA 12 Consistent Shape Maps via Semidefinite Programming, Q. Huang and L. Guibas, SGP 13 Matching Partially Similar Objects via Matrix Completion, Y. Chen, L. Guibas and Q. Huang, ICML 14