RELATIVE SCALE ESTIMATION AND 3D REGISTRATION OF MULTI-MODAL GEOMETRY USING GROWING LEAST SQUARES

Transcription

1 RELATIVE SCALE ESTIMATION AND 3D REGISTRATION OF MULTI-MODAL GEOMETRY USING GROWING LEAST SQUARES Nicolas Mellado Matteo Dellepiane Université de Toulouse University College London Visual Computing Lab, ISTI-CNR, Pisa Roberto Scopigno

2 2 PROBLEM STATEMENT Align acquired multi-modal 3D data Account for variations of: Scale factor Sampling Details Noise Photogrammetry Lidar

3 3 PROBLEM STATEMENT Align acquired multi-modal 3D data Output may be refined by local registration (ICP)

4 4 RELATED WORK Multi-modal registration Robustness to Bbox Fitting Game-Theoric framework [26] SIFT [17,18] Corsini et al. [27] Keyscale [28] Our method Density variation Noise Scalability Point-clouds Low overlap [26] Rodola et al., A scale independent selection process for 3d object recognition in cluttered scenes, 2013 [17] Li&Guskov, Multi-scale features for approximate alignment of point-based surfaces, 2005 [18] Skelly&Sclaroff, Improved feature descriptors for 3d surface matching, 2007 [27] Corsini et al., Fully automatic registration of image sets on approximate geometry, 2013 [28] Lin et al., Scale alignment of 3d point clouds with different scales, 2014

5 5 OVERVIEW Point-wise relative scale factor estimation multi-scales analysis 2

6 6 OVERVIEW Point-wise relative scale factor estimation Multi-Modal registration 2-clicks (supervised) Unsupervised multi-scales analysis 2

7 7 POINT-WISE SCALE ESTIMATION CHALLENGES 1. Consider geometric properties at multiple scales scale torso chest hairs

8 8 POINT-WISE SCALE ESTIMATION CHALLENGES 1. Consider geometric properties at multiple scales 2. Track changes due to scale factor scale

9 9 POINT-WISE SCALE ESTIMATION CHALLENGES 1. Consider geometric properties at multiple scales 2. Track changes due to scale factor scale

10 10 POINT-WISE SCALE ESTIMATION CHALLENGES 1. Consider geometric properties at multiple scales 2. Track changes due to scale factor scale

11 11 POINT-WISE SCALE ESTIMATION CHALLENGES 1. Consider geometric properties at multiple scales 2. Track changes due to scale factor scale

12 12 POINT-WISE SCALE ESTIMATION REQUIREMENTS 1. Shape descriptor 1. Robust 2. Multiscale 3. Characterize pertinent shapes scale

13 13 POINT-WISE SCALE ESTIMATION REQUIREMENTS 1. Shape descriptor 1. Robust 2. Multiscale 3. Characterize pertinent shapes scale 2. Scale-space transformation 1. Related to scale factor 2. Easy to measure Analysis scale = Diffusion time [Sun2009;Bronstein2010;Patané2013; ]

14 14 POINT-WISE SCALE ESTIMATION GROWING LEAST SQUARES [Mellado2009] Efficient Algebraic sphere fitting APSS [Guennebaud2007] x; S u x = 0

15 15 POINT-WISE SCALE ESTIMATION GROWING LEAST SQUARES [Mellado2009] Efficient Algebraic sphere fitting APSS [Guennebaud2007] x; S u x = 0 Meaningful parametrization τ = s u (p) η = s u(p) s u (p) κ = 2u q γ = residuals

16 16 POINT-WISE SCALE ESTIMATION GROWING LEAST SQUARES [Mellado2009] Efficient Algebraic sphere fitting APSS [Guennebaud2007] x; S u x = 0 Meaningful parametrization τ = s u (p) η = s u(p) s u (p) κ = 2u q γ = residuals Multiscale point-wise comparison c-wise distance Δ p, p = 1 b a b s=a σ p, s, p, s

17 17 POINT-WISE SCALE ESTIMATION OUR APPROACH Use logarithmic scale sampling (basis m) [Bronstein&Kokkinos2010]

18 18 POINT-WISE SCALE ESTIMATION OUR APPROACH Use logarithmic scale sampling (basis m) [Bronstein&Kokkinos2010] Find optimal offset h Relative scale factor s e = 1 m h

19 19 POINT-WISE SCALE ESTIMATION OUR APPROACH New multiscale point-wise comparison Δ p, p, h = 1 2I I i= I σ p, h i, p, h 2I Original formulation: Δ p, p = 1 b a b s=a σ p, s, p, s

20 20 POINT-WISE SCALE ESTIMATION OUR APPROACH Optimal offset h is defined as Δ p, p, h = 1 2I I i= I σ p, h i, p, h arg max h Δ p, p, h

21 21 POINT-WISE SCALE ESTIMATION EVALUATION Sensitivity to analysis scale range Sensitivity to noise/sampling Spatial consistency

22 22 OVERVIEW Point-wise relative scale estimation Multi-Modal registration 2-clicks (supervised) Fully automatic multi-scales analysis x 2,08

23 23 MULTIMODAL REGISTRATION 2-CLICKS (SUPERVISED) 1. Let the user choose similar points, and scale (min/max) 2. Compute GLS and find h interactively 3. Align the two clouds Scale is given by h Translation is defined by input points Rotation is defined by spatial differentiation η d κ1 Principal curvature direction η d κ1

24 24 MULTIMODAL REGISTRATION 2-CLICKS (SUPERVISED) 1. Let the user choose similar points, and scale (min/max) 2. Compute GLS and find h interactively 3. Align the two clouds Video

25 25 MULTIMODAL REGISTRATION UNSUPERVISED Overview Sub-sample input clouds RANSAC-Based registration scheme Use scale estimation for filtering

26 26 MULTIMODAL REGISTRATION UNSUPERVISED Overview Sub-sample input clouds RANSAC-Based registration scheme Use scale estimation for filtering Prioritize vertices using GLS geometric variation ν ν unstable pertinent scale

27 27 MULTIMODAL REGISTRATION UNSUPERVISED - RESULTS Input clouds Target scale factor: 4.25 Registration result (env. 7 minutes) Estimated scale factor: 4.11 After ICP refinement Estimated scale factor : 4.24

28 28 MULTIMODAL REGISTRATION UNSUPERVISED - RESULTS Input clouds Registration result After ICP refinement LIDAR Modelled Target scale factor: 0.23 Estimated scale factor: 0.2 Timing: 2m50s Estimated scale factor: 0.21 LIDAR Estimated scale factor: 1.09 Timing: 10m15s Estimated scale factor: 1.09 Spherical Target scale factor: 1.09

29 29 MULTIMODAL REGISTRATION UNSUPERVISED - COMPARISONS Real Blocks Small Blocks (with change) Small Blocks (no change) Ours + ICP [1] Ours Scale Ratio ICP [28] ICP [61] Keyscale [60] Mesh Resolution [59] -50% 0% 50% Relative Estimation Error Ours Ours+ICP [1] [1] Du et al., Scaling iterative closest point algorithm for registration of m-d point sets, 2010 [28] Lin et al., Scale alignment of 3d point clouds with different scales, 2014 [61] Besl&McKay, A method for registration of 3d shapes, 1992 [60] Tamaki et al., Scale matching of 3d point clouds by finding keyscales with spin images, 2010 [59] Johnson&Hebert, Using spin images for efficient object recognition in cluttered 3d scenes, 1999

30 30 RELATIVE SCALE ESTIMATION AND 3D REGISTRATION TAKE HOME MESSAGE Multi-modal data are difficult to register Sampling, noise, details variations, unknown scale factor Proposed approach Use multiscale extrinsic descriptor to Build and compare robust point-based signatures Retrieve point-wise relative scale factor Design systems for automatic and assisted registration Applied on real-world data Compatible to existing workflows Future work Automatic detection of the analysis scale range Registration algorithm: use scale information from the start

31 31 Thank you for your attention RELATIVE SCALE ESTIMATION AND 3D REGISTRATION OF MULTI-MODAL GEOMETRY USING GROWING LEAST SQUARES Nicolas Mellado Matteo Dellepiane Université de Toulouse University College London Fundings Visual Computing Lab, ISTI-CNR, Pisa Roberto Scopigno ANR Mapstyle project ERC Starting Grant Smart Geometry EU FP7 project ICT FET Harvest4D