Structured Light II. Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov

Transcription

1 Structured Light II Johannes Köhler Thanks to Ronen Gvili, Szymon Rusinkiewicz and Maks Ovsjanikov

2 Introduction Previous lecture: Structured Light I Active Scanning Camera/emitter systems Geometric calibration Today: Structured Light II Gamma calibration Mesh registration 1/18/2012 Lecture 3D Computer Vision 2

3 Gamma function 1/18/2012 Lecture 3D Computer Vision 3

4 Gamma function Phase shifting heavily relies on the different gray values being transmitted correctly Information A Information B Intensity encoding Intensity decoding Physical transmission 4 1/18/2012 Lecture 3D Computer Vision 4

5 Gamma function Problem: Usually projector and camera distort the values being sent / received Result is a non-linear gamma curve of the system x Axis: Intensity values sent by the projector [0 255] y Axis: Intensity values captured by the camera [0 255] 1/18/2012 Lecture 3D Computer Vision 5

6 Gamma function Non-linearity causes large errors in the reconstructions 1/18/2012 Lecture 3D Computer Vision 6

7 Gamma function Solution: Distort the values being sent by the projector in a way, that once they are received, this happens in a linear manner Find function f best describing the systems response curve Exponential function Sigmoid function Use inverse f^{-1} of f and send f^{-1}(x) instead of x Received value: f(f^{-1}(x)) = x 1/18/2012 Lecture 3D Computer Vision 7

8 Gamma function By linearizing the gamma response, the quality of the reconstructions is highly improved 1/18/2012 Lecture 3D Computer Vision 8

9 Matching data + 1/18/2012 Lecture 3D Computer Vision 9

10 Motivation Usually whole objects should be reconstructed from all sides A structured light system can only reconstruct the parts of the objects which are seen by both the camera and the projector Problem: Complex objects 1/18/2012 Lecture 3D Computer Vision 10

11 Motivation Not all details can be captured in one scan Problem: Each scan has its own coordinate system Question: How to align scans? 1/18/2012 Lecture 3D Computer Vision 11

12 Aligning 3D data If the correct correspondences are known, we can find correct relative rotation/translation 1/18/2012 Lecture 3D Computer Vision 12

13 Aligning 3D data How to find correspondences: User input? Feature detection? Signatures? Alternative: Assume closest point 1/18/2012 Lecture 3D Computer Vision 13

14 Aligning 3D data Iterate to find alignment => Algorithm: Iterative Closest Points (ICP) [Besl & McKay 92] Problem: Converges only if starting position is close enough Closest point is often a bad approximation for the corresponding point 1/18/2012 Lecture 3D Computer Vision 14

15 ICP method Extended formulation of the ICP method 1. Selecting source points (from one or both meshes) 2. Matching to points in the other mesh 3. Weighting the correspondences 4. Rejecting outlier point pairs 5. Assigning an error metric to the current transform 6. Minimizing the error metric w.r.t. transformation 1/18/2012 Lecture 3D Computer Vision 15

16 ICP method 1. Step: Selecting source points Use all points Uniform sampling Random sampling Normal-space sampling Ensure that samples have normals distributed as uniformly as possible Uniform sampling Normal-space sampling 1/18/2012 Lecture 3D Computer Vision 16

17 Random vs. normal-space sampling Random sampling Normal-space sampling Normal-space sampling better for mostly-smooth areas with sparse features 1/18/2012 Lecture 3D Computer Vision 17

18 ICP method 2. Step: Matching Closest point => Expensive For range images we can simply project point [Blais 95] Slightly worse performance per iteration 1/18/2012 Lecture 3D Computer Vision 18

19 ICP method 3. Step: Assigning weights Constant weight Assigning lower weights to pairs with greater point-to-point distance : Weight = 1 Dist( p Dist 1, p2) max Weighting based on compatibility of normals Weight = n 1 n 2 Scanner uncertainty 1/18/2012 Lecture 3D Computer Vision 19

20 ICP method 4. Step: Rejecting outliers Point to point distance higher than a given threshold Pairs containing points on end vertices 1/18/2012 Lecture 3D Computer Vision 20

21 ICP method 4. Step: Rejecting outliers Rejection of pairs that are not consistent with their neighboring pairs [Dorai 98] (p1,q1), (p2,q2) are inconsistent <=> Dist( p1, p2) Dist( q1, q2) > τ q1 q2 p2 p1 1/18/2012 Lecture 3D Computer Vision 21

22 ICP method 5. & 6. Step: Error metric and minimization Minimize e.g. f ([ R t]) = 1 N S N S i= 1 qi [ R t] 1 p i 2 Sequential minimization Minimize 1 and 2, then 2 and 3, => Accumulated error (loop closing problem) 1/18/2012 Lecture 3D Computer Vision 22

23 ICP method 5. & 6. Step: Error metric and minimization Simultaneous minimization Simultaneously minimize the various transformations [R i t i ] Diffusively distribute the alignment error over all overlaps of each range images Large computational cost. 1/18/2012 Lecture 3D Computer Vision 23

24 ICP method How to find good initial position? Try to find and match features in both meshes Closely related to image features and matching (=> see lecture 9: Wide baseline matching) but in 3D Therefore: Use neighborhood geometric information Descriptors should be Invariant under transform Local Cheap Improves correspondence searching and matching 1/18/2012 Lecture 3D Computer Vision 24

25 ICP method Features: 0.20 Inherent smoothing Related to mean curvature Robust to noise 1/18/2012 Lecture 3D Computer Vision 25

26 ICP method Feature identification Pick as features points with rare descriptor values Results in few but relatively robust correspondences # occurences Features Descriptor value 1/18/2012 Lecture 3D Computer Vision 26

27 ICP method Feature matching Match features with similar descriptor values Use e.g. RANSAC (see lecture 5: Parameter estimation) to filter out wrong correspondences P Q 1/18/2012 Lecture 3D Computer Vision 27

28 ICP method Use features to roughly align the various reconstructions Then use classical ICP for refinement using the good starting approximation Input: 10 scans Alignment Refined by ICP 1/18/2012 Lecture 3D Computer Vision 28

29 SL reconstruction example 29

30 References Besl, McKay A Method for Registration of 3D Shapes; in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, 1992 Rusinkiewicz, Levoy Efficient Variants of the ICP Algorithm, In Proceedings of the Third Intl. Conf. on 3D Digital Imaging and Modeling (2001), pp Dorai, Weng, Jain Registration and Integration of Multiple Object Views for 3D Model Construction, Trans. PAMI, Vol. 20, No. 1, 1998 Gelfand, Mitra, Guibas, Pottmann Robust global registration, in Proceedings of 3 rd Eurographics Symposium on Geometry Processing, /18/2012 Lecture 3D Computer Vision 30