3D Photography: Active Ranging, Structured Light, ICP

Transcription

1 3D Photography: Active Ranging, Structured Light, ICP Kalin Kolev, Marc Pollefeys Spring

2 Schedule (tentative) Feb 18 Feb 25 Mar 4 Mar 11 Mar 18 Mar 25 Apr 1 Apr 8 Apr 15 Apr 22 Apr 29 May 6 May 13 May 20 May 27 Introduction Lecture: Geometry, Camera Model, Calibration Lecture: Features, Tracking/Matching Project Proposals by Students Lecture: Epipolar Geometry Lecture: Stereo Vision Easter Short lecture Stereo Vision (2) + 2 papers Project Updates Short lecture Active Ranging, Structured Light + 2 papers Short lecture Volumetric Modeling + 2 papers Short lecture Mesh-based Modeling + 2 papers Short lecture SfM, SLAM + 2 papers Pentecost / White Monday Final Demos

3 Structured light and active ranging techniques

4 Today s class Obtaining depth maps / range images unstructured light structured light time-of-flight Registering range images (some slides from Szymon Rusinkiewicz, Brian Curless)

5 Taxonomy 3D modeling passive active stereo structured/ unstructured shape from light silhouettes photometric stereo laser scanning

6 Unstructured light project texture to disambiguate stereo

7 Space-time stereo Davis, Ramamoothi, Rusinkiewicz, CVPR 03

8 Space-time stereo Davis, Ramamoothi, Rusinkiewicz, CVPR 03

9 Space-time stereo Zhang, Curless and Seitz, CVPR 03

10 Space-time stereo results Zhang, Curless and Seitz, CVPR 03

11 Light Transport Constancy Davis, Yang, Wang, ICCV05

12 Triangulation Light / Laser Camera Peak position in image reveals depth

13 Triangulation: Moving the Camera and Illumination Moving independently leads to problems with focus, resolution Most scanners mount camera and light source rigidly, move them as a unit, allows also for (partial) pre-calibration

14 Triangulation: Moving the Camera and Illumination

15 Triangulation: Extending to 3D Alternatives: project dot(s) or stripe(s) Object Laser Camera

16 Triangulation Scanner Issues Accuracy proportional to working volume (typical is ~1000:1) Scales down to small working volume (e.g. 5 cm. working volume, 50 µm. accuracy) Does not scale up (baseline too large ) Two-line-of-sight problem (shadowing from either camera or laser) Triangulation angle: non-uniform resolution if too small, shadowing if too big (useful range: )

17 Triangulation Scanner Issues Material properties (dark, specular) Subsurface scattering Laser speckle Edge curl Texture embossing Where is the exact (subpixel) spot position?

18

19 Space-time analysis Curless 95

20 Space-time analysis Curless 95

21 Projector as camera

22 Multi-Stripe Triangulation To go faster, project multiple stripes But which stripe is which? Answer #1: assume surface continuity e.g. Eyetronics ShapeCam

23 Kinect Infrared projector Infrared camera Works indoors (no IR distraction) invisible for human Depth Map: note stereo shadows! Color Image (unused for depth) IR Image

24 Kinect Projector Pattern strong texture Correlation-based stereo between IR image and projected pattern possible stereo shadow Bad SNR / too close Homogeneous region, ambiguous without pattern

25 Multi-Stripe Triangulation To go faster, project multiple stripes But which stripe is which? Answer #2: colored stripes (or dots)

26 Multi-Stripe Triangulation To go faster, project multiple stripes But which stripe is which? Answer #3: time-coded stripes

27 Time-Coded Light Patterns Assign each stripe a unique illumination code over time [Posdamer 82] Time Space

28 Better codes Gray code Neighbors only differ one bit => Structured light presented afterwards

29 Poor man s scanner Bouguet and Perona, ICCV 98

30 Pulsed Time of Flight Basic idea: send out pulse of light (usually laser), time how long it takes to return d 1 = c t 2

31 Pulsed Time of Flight Advantages: Large working volume (up to 100 m.) Disadvantages: Not-so-great accuracy (at best ~5 mm.) Requires getting timing to ~30 picoseconds Does not scale with working volume Often used for scanning buildings, rooms, archeological sites, etc.

32 Depth cameras 2D array of time-of-flight sensors e.g. Canesta s CMOS 3D sensor jitter too big on single measurement, but averages out on many (10,000 measurements 100x improvement)

33 3D modeling Aligning range images Pairwise Globally (some slides from S. Rusinkiewicz, J. Ponce, )

34 Aligning 3D Data If correct correspondences are known (from feature matches, colors, ), it is possible to find correct relative rotation/translation

35 X 1 Aligning 3D Data X 2 T e.g. Kinect motion X 1 X 2 X i = T X i For T as general 4x4 matrix: Linear solution from 5 corrs. T is Euclidean Transform: 3 corrs. (using quaternions) [Horn87] Closed-form solution of absolute orientation using unit quaternions

36 Aligning 3D Data How to find corresponding points? Previous systems based on user input, feature matching, surface signatures, etc.

37 Spin Images [Johnson and Hebert 97] Signature that captures local shape Similar shapes similar spin images

38 Computing Spin Images Start with a point on a 3D model Find (averaged) surface normal at that point Define coordinate system centered at this point, oriented according to surface normal and two (arbitrary) tangents Express other points (within some distance) in terms of the new coordinates

39 Computing Spin Images Compute histogram of locations of other points, in new coordinate system, ignoring rotation around normal: α = p nˆ β = p nˆ radial dist. elevation

40 Computing Spin Images elevation radial dist.

41 Spin Image Parameters Size of neighborhood Determines whether local or global shape is captured Big neighborhood: more discriminative power Small neighborhood: resilience to clutter Size of bins in histogram: Big bins: less sensitive to noise Small bins: captures more detail

42 Alignment with Spin Image Compute spin image for each point / subset of points in both sets Find similar spin images => potential correspondences Compute alignment from correspondences Same problems as with image matching: - Robustness of descriptor vs. discriminative power - Mismatches => robust estimation required

43 Solving 3D puzzles with VIPs SIFT features Extracted from 2D images Variation due to viewpoint VIP features Extracted from 3D model Viewpoint invariant (Wu et al., CVPR08) 43

44 Aligning 3D Data Alternative: assume closest points correspond to each other, compute the best transform

45 Aligning 3D Data and iterate to find alignment Iterated Closest Points (ICP) [Besl & McKay 92] Converges if starting position close enough

46 ICP Variant Point-to-Plane Error Metric Using point-to-plane distance instead of point-topoint lets flat regions slide along each other more easily [Chen & Medioni 92]

47 Finding Corresponding Points Finding closest point is most expensive stage of ICP Brute force search O(n) Spatial data structure (e.g., k-d tree) O(log n) Voxel grid O(1), but large constant, slow preprocessing

48 Finding Corresponding Points For range images, simply project point [Blais/Levine 95] Constant-time, fast Does not require precomputing a spatial data structure

49 Efficient ICP Efficient Variants of the ICP algorithm [Rusinkiewicz & Levoy, 3DIM 2001] => Presented afterwards

50 Presentations [Scharstein/Szeliski 03]: High Accuracy Stereo Depth Maps using Structured Light [Rusinkiewicz/Levoy 01]: Efficient Variants of the ICP Algorithm