Using a Raster Display Device for Photometric Stereo

Transcription

1 DEPARTMEN T OF COMP UTING SC IENC E Using a Raster Display Device for Photometric Stereo Nathan Funk & Yee-Hong Yang CRV 2007 May 30, 2007

2 Overview 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1. Background 2. Model 3. Experiments 4. Conclusions 2

3 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1. Background Knowledge about lighting simplifies shape from shading Controlling lighting helps Controlling lighting is not easy 3

4 Motivation 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Idea: Use a display to control lighting! Use it to perform photometric stereo 4

5 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Related Work Zongker (1999), Schechner (2003) Use displays as light source Clark (CRV 2006) Photometric Stereo with Nearby Planar Distributed Illuminants Equivalent light source for an image Only offers theoretical analysis 5

6 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Photometric Stereo Proposed by Woodham (1978) 2 Steps: 1 Photometric Stereo 2 Depth from Surface Normals Input Images Surface Normals Surface Depth 6

7 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Photometric Stereo For a single Lambertian surface point r R = " max( 0, L! nˆ) Radiance Albedo Light source vector Normal r r r Simplified R = L! N where N =! nˆ Known: R, L r Unknown: N r Can not uniquely determine N r from single R 7

8 8 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 1 BACKGROUND Photometric Stereo!!! " # $ $ $ % &!!! " # $ $ $ % & =!!! " # $ $ $ % & z y x nx nx nx z y x n N N N L L L L L L R R M M M M R N = L r r Radiance Light vectors Scaled normal Need 3 or more radiance values for each normal

9 2. Model 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Screen Scene Camera 9

10 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 2. Model Need R Radiance values (inferred from images) L Light vectors (incl. intensity) Challenges Screens are not distant light sources Screen s light can be directional (LCDs) Inverse square law is significant 10

11 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 2. Model Distant illumination not achievable Instead: 50x50 squares of pixels 6 sources 6 images 11

12 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Screen Position Calibration Display (showing a calibration pattern) Mirror Camera Mirror calibration pattern Alternative method: Francken (CRV 07) 12

13 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Lighting Model Display cross-section 1 Unattenuated RU Screen directionality function 2 Attenuation R (",!) R f (",!) P = U 3 Inverse square law I S = R P!," ( ) r 2 13

14 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Depth from Surface Normals n 2 Camera n 1 Scene z 1 z 2 Constraints form a large homogeneous linear system Coefficient matrix r = 0 r z H Depth values Solve system with sparse matrix routines 14

15 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 3. Experiments Enclosure LCD Screen Camera Scene 15

16 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS 3. Experiments Quantitative evaluation on synthetic and real images Objects Sphere (Ping-Pong ball) Stanford Bunny (printed on a 3D printer) 16

17 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Captured Images Images displayed on screen Processed captured images 17

18 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Results on Synthetic Images 18

19 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Results on Real Images 19

20 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Results on Real Images 20

21 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Results on Real Images 21

22 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Results on Real Images 22

23 Future work 2. MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Use more advanced methods Allow surface specularity Increase precision Examine different displays and cameras Reduce image noise Integration with other methods E.g. combine with multiple view vision 23

24 2. MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Potential Shape-from-shading Capture objects, faces in front of home computer; assist in face recognition Lighting estimation, Image relighting, Image based rendering, Surface reflectance measurement 24

25 2. MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Acknowledgements D E P A R T M E N T O F COMPUTIN G SCIENCE 25

26 Questions? 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Slides available at: singularsys.com/research 26

27 2 MODEL. 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Screen Directionality Calibration 27

28 2 MODEL 3 EXPERIMENTS 4 CONCLUSIONS 5 QUESTIONS Results on Synthetic Images Sphere Stanford Bunny 28