TRANSIENT IMAGING. Freshman Imaging Project 2014

Transcription

1 1 TRANSIENT IMAGING Freshman Imaging Project 2014

2 2 Outline What is transient imaging? Previous work: MIT, UBC Possible applications of transient imaging Theory behind transient imaging Our goals Technology Work breakdown structure Project schedule

3 3

4 4 Previous Research MIT School of Architecture and Planning, Program in Media Arts and Sciences Ramesh Raskar, Ahmed Kirmani UBC Department of Computer Science, Imager Lab Felix Heide, Matthias Hullin

5 5 UBC s RESEARCH used Photonic Mixer Device (Camboard Nano) in conjunction with laser diodes ca/labs/imager/tr/2013/tran sientpmd/

6 6 Theory Behind Transient Imaging

7 7 What is Time-of-Flight? Time of flight (ToF) measures the time required for pulses of laser light to travel from an emitter and reflect back to an image sensor

8 8 What is a PMD? A Photonic Mixer Device Does not measure specific time, measures the crosscorrelation of a reference signal and a modulated light source Mixes the signals

9 9 Setup

10 10

11 11 Raw Data Needed System collects intensity and time for each pixel The intensity of each pixel at a given frequency and at a given time is the impulse response of light The impulse response is used to predict the passage of light through the scene ~100 frames per capture (3-5s) will be reconstructed using these predictions

12 12 Our System Data

13 13 The Basic Principle of Our System Our capture is an estimation MIT used streak cameras and femtosecond lasers We are using a ToF sensor and modulated light source to construct our own capture via our graphs of intensity over time

14 14 Components of Light P=D+S+I D: direct reflection S: subsurface scattering I: interreflections and longer range scattering

15 15 Software To break up the interactions and see them intermittently over time, we will be using the code provided by UBC which uses advanced optimization algorithms to solve for each component of our image

16 16 Our Goals Replicate UBC s system PMDtechnologies Camboard Pico Microsoft Kinect for Windows v2 Advancements Microscopy Multiple sensors

17 17 Light Source - Kinect System Internal 860 nm according to Microsoft nm found with spectrometer External 850 nm laser diodes Signal generator Oscilloscope

18 18

19 19 Light Source - Kinect System Internal 860 nm according to Microsoft nm found with spectrometer External 850 nm laser diodes Signal generator Oscilloscope

20 20 Sensor - Kinect System Resolution 524x424 pixels Modulation Frequency MHz (Average 80 MHz) Field of View 70 (H) x 60 (V) degrees Depth Uncertainty < 0.5% of range Distance Range meters Operating Wavelength 860 nm Frame Rate Max 60 fps (Normally 30 fps)

21 Sensor - Kinect System 21

22 22 Processing - Kinect Sensor Software Kinect for Windows Software Development Kit 2.0 (SDK) MetriCam Software Development Kit Processing Write an app for the Kinect

23 23 PMD System Light source Internal 850nm Sensor IR lens Accepts 850 nm Can t be removed Must use different light source than UBC

24 24 Sensor - PMD System Resolution Modulation Frequency Field of View Depth Uncertainty Distance Range Operating Wavelength 160x120 pixels average of 80 MHz 82 x66 < 0.5meters meters 850nm

25 25 Sensor - PMD System

26 26 Processing - PMD System Software C/C++ SDK, Matlab SDK Coding Code from UBC, however we will need to make sure that we have the data that we will run through the code is in the correct format

27 Preliminary Tests Running all the code with UBC s data Testing illumination system with signal generator and oscilloscope and a photodiode 27

28 28 Budget PMD $700 Kinect (x 2) $150 each Illumination System $140 Graphics Card $50

29 29 Work Breakdown Structure

30 PROJECT SCHEDULE 10 PRELIM DESIGN REVIEW ILLUMINATION SYSTEM Build Illumination System Verify Illumination Performance KINECT-BASED SYSTEM SDK Modulate Internal Illum. SDK Extract Relevant Data Render Using Internal Illum Data Operate w/ External Illumination Render Using External Illum Data CRITICAL DESIGN REVIEW PMD-BASED SYSTEM Operate w/ External Illumination Render Using External Illum Data ADVANCED PERFORMANCE Conceptual Design Work Implement Advanced Concepts IMAGINERIT EXHIBIT

31 31 Support/ Resources CIS PMDtec Microsoft Kinect Support University of British Columbia

32 32 Questions?

33 21 Propagation - Kinect System

34 8 Uses for Transient Imaging

35 34 Citations [1] F. Heide et al., Low-budget Transient Imaging using Photonic Mixer Devices, Vancouver: The University of British Columbia. [2] M. Hullin. (2014, August). Building a Transient Camera. [Online]. Available: [3] R. Schwarte et al., A new electro-optical mixing and correlating sensor: Facilities and Applications of the Photonic Mixer Device, in Proc. SPIE 3100, 1997 SPIE. doi: /

36 31