Active Light Time-of-Flight Imaging

Transcription

1 Active Light Time-of-Flight Imaging

2 Time-of-Flight Pulse-based time-of-flight scanners [Gvili03] NOT triangulation based short infrared laser pulse is sent from camera reflection is recorded in a very short time frame (ps) results in depth profile (intensity image)

3 Time-of-Flight Pulse-based time-of-flight scanner examples accuracy 1-2 cm in a range of 4 7 m applications: "depth keying" replaces chroma keying 3D interaction large scale 3D scanning (LIDAR light detection and ranging)

4 Canesta/3DV Kinect 2 (?)

5 PMDs Photonic Mixer Devices Also called multi-bucket sensors Chip layout Schematic view Electric symbol Fast on-chip modulation [Luan 01]

6 PMDs Photonic Mixer Devices Working principle: Measure phase difference of emitted, modulated signal and received one [Metrilus] Current hardware: ~20MHz modulation frequency in practice square wave

7 [slide by Felix Heide] Photonic Mixer Device (PMD) sensor: P PMD Sensor dt

8 [slide by Felix Heide] Single-path Time-of-Flight depth imaging: P PMD Sensor dt

9 [slide by Felix Heide] Single-path Time-of-Flight depth imaging: P PMD Sensor dt distance-dependent correlation

10 Time-of-Flight Cameras LIDAR University of Washington NASA PMD Panasonic Fotonic ARTTS MESA See [Kolb et al. 10], EG STAR for more details

11 Active Light Transient Imaging

12 [slide by Felix Heide] What is Transient Imaging? a.k.a. Light-in-Flight Imaging Ultrafast imaging of nonstationary light distribution in scenes Tracking of wavefronts of light as they propagate in the scene Equivalent to imaging ultrashort pulses of light t =

13 [slide by Felix Heide] Applications of Transient Imaging Many applications by analyzing a transient image! Understanding light transport: Surface reflectance capture: Decomposing light transport: Reconstructing hidden object geometry (and motion): Wu et al 2012 Velten et al 2013 Naik et al 2011 Velten et al 2012 Pandharkar et al 2011

14 Visualizing Light in Motion Repetitive Event, 1012 FPS, 672 x 1000 pixel resolution, 1 ns+ capture time Light moves 0.6 mm per frame Ability to see light transport [Velten et al. 13]

15 Streak Cameras Picosecond time resolution 1D: 1x672 pixels Result as 2D image ( Streak Photo ) Hamamatsu [Velten et al. 13]

16 Experimental Setup Ti:Sapphire Laser [Velten et al. 13] Synchronization Camera Object Scanning Unit

17 Actual Setup [Velten et al. 13]

18 Streak Camera Picture Time [Velten et al. 13] Space

19 [Velten et al. 13]

20 [Velten et al. 13]

21 Example Result [Velten et al. 13]

22 [Velten et al. 13]

23 Scattering in Real Scenes [Velten et al. 13]

24 [Velten et al. 13]

25 Transient Imaging With PMD Devices linked to the multipath or mixed pixel problem PMD Panasonic Fotonic ARTTS MESA

26 [slides by Felix Heide] Multi-path contributions: P2 PMD Sensor P1 dt

27 [slides by Felix Heide] Recovering multi-path contributions: P2 PMD Sensor P1 dt

28 [slides by Felix Heide] Recovering multi-path contributions: P2 PMD Sensor P1 dt

29 [slides by Felix Heide] Recovering multi-path contributions: P2 PMD Sensor P1 dt

30 [slides by Felix Heide] Image formation model: Relation of Transient Image to PMD measurement: Linear system is ill-posed: assume model for temporal response (signal sparsity) Mixture Model: Gaussian + Exponential and spatial smoothness solve non-linear system multiple measurements Spatial coherence Temporal model

31 [slides by Felix Heide] PMDTechnologies CamBoard nano 2 Too slow LED unit Fixed frequency PLL 25MHz Control FPGA PMD sensor ADC

32 [slides by Felix Heide] Modifications to the hardware Prototype: Max mod. frequency 180 MHz, stable up to 110 MHz Novatech DDS9m Laser unit ic-hg + LPC826 2-ch. function generator MHz Control FPGA Control MOD PMD sensor TRIG ADC

33 Prototype setup MOD in Used for results in paper Camera Power TRIG out Laser unit Function generator

34 [slides by Felix Heide] Result Discoball : Color coding of strongest component:

35 [slides by Felix Heide] Results Discoball : Different time-steps:

36 [slides by Felix Heide] Results Discoball : Different time-steps:

37 [slides by Felix Heide] Results Discoball : Different time-steps:

38 [slides by Felix Heide] Results Discoball : Different time-steps:

39 [slides by Felix Heide] Results Discoball : Different time-steps:

40 [slides by Felix Heide] Result Bottles : Color coding of strongest component:

41 [slides by Felix Heide] Emerging Technologies prototype Camera unit Light source AD9958 DDS (function generator) MHz Control FPGA Control MOD PMD sensor TRIG ADC

42 [slides by Felix Heide] Camera Light source

43 [slides by Felix Heide] Results: People

44 References Curless, Levoy, Better optical triangulation through spacetime analysis, CVPR 1995 Levoy et al., The digital Michelangelo project: 3D scanning of large statues, SIGGRAPH 2000 [Gvili03] R. Gvili, A. Kaplan, E. Ofek, G. Yahav, "Depth Keying", SPIE ISOE 5006, 2003, pp Luan, Experimental Investigation of Photonic Mixer Device and Development of TOF 3D Ranging Systems Based on PMD Technology, PhD Thesis, Siegen University, 2001 Velten et al. Femto-photography: capturing and visualizing the propagation of light, SIGGRAPH 2013 Heide et al. Low-budget Transient Imaging using Photonic Mixer Devices., SIGGRAPH 2013