Introduction to Computer Vision. Week 8, Fall 2010 Instructor: Prof. Ko Nishino

Transcription

1 Introduction to Computer Vision Week 8, Fall 2010 Instructor: Prof. Ko Nishino

2 Midterm

3 Project 2 without radial distortion correction with radial distortion correction

4 Light Light Light! How do you recover the true color of a scene? " Recovering Lightness! How do you describe reflection? " Radiometry and Reflectance

5 Recovering Lightness

6 Land s Experiment (1959)! Cover all patches except a blue rectangle! Make it look gray by changing illumination! Uncover the other patches Color Constancy We filter out illumination variations

7 Color Cube Illusion D. Purves, R. Beau Lotto, S. Nundy Why We See What We do, American Scientist

8 Checker Shadow Illusion by Ted MIT

9 Lightness Recovery and Retinex Theory! Recover surface reflectance/color in varying illumination conditions " Use tools developed before " Sensing: intensity/color " Image processing: Fourier transform and convolution " Edge operators " Iterative techniques

10 Image Brightness (Intensity) scene irradiance scene reflectance scene radiance sensor response intensity Assume monochromatic light and b' ( x, y) = r' ( x, y)e' ( x,y) Reflectance as a multiplier to irradiance is often called lightness Note: our analysis can be applied to colored light (we ll simply have filters in front of the sensor) Also, assume everything is flat

11 Lightness Recovery b' ( x, y) = r' ( x, y)e' ( x,y) Can we recover and from? We need assumptions to constrain the solution!! Sharp changes in reflectance! Smooth changes in illumination Frequency spectrum (Fourier transform) We want to filter out e

12 Lightness Recovery Take logarithm Use Laplacian logb' x,y b' ( x, y) = r' ( x, y)e' ( x, y) ( ) = log r' ( x, y) + loge' ( x,y) ( ) = r( x, y) + e( x, y) b x, y d = " 2 b = " 2 r + " 2 e! Sharp changes in reflectance " 2 = #2 #x 2 + #2 #y 2 has 2 infinite spikes near edges and! Smooth changes in illumination elsewhere everywhere

13 Lightness Recovery (Retinex Scheme) Image: Laplacian : Thresholding : Reconstruction : (lightness) (reflectance)

14 Solving the Inverse Problem Image Laplacian Thresholding Lightness Find lightness l(x,y) from t(x,y) Poisson s Equation # % $ " 2 "x 2 + "2 "y 2 & ( l x, y ' ( ) = t( x, y) We have to find g(x,y) which satisfies ( ) = t u,v l x, y $ # "# $ # "# ( )g( x " u,y " v)dudv l( x, y) = t( x, y) " g( x, y)

15 Solving Poisson s Equation We have " 2 l( x, y) = t( x, y) and l( x, y) = t( x, y) " g( x, y) So "( u 2 + v 2 )L u,v Fourier transform? = T( u,v) and ( ) = T( u,v) and L( u,v) = T( u,v)g( u,v) L( u,v) = T( u,v)g( u,v) Thus G( u,v) = " 1 u 2 + v 2 g( x, y) = 1 2" log x 2 + y 2 + c See Robot Vision page 123,124

16 Lightness Recovery Log of image Which means: Normalize: Assume maximum value of Then: ( ) = k + r( x, y) l x, y (lightness) (reflectance) l' x, y l' x, y r' ( x, y) = (reflectance) l max ( ) = kr' ( x, y) ( ) corresponds to b' x, y e' ( x, y) = (illumination) r' x, y g( x, y) = 1 2" log x 2 + y 2 ( ) ( )

17 Computing Lightness (Discrete Case) Log of image basically inverse of Laplacian mask But, inverse of a discrete approximation of the Laplacian would not be an exact inverse of the Laplacian Solve directly

18 Computing Lightness (Discrete Case) "20l i, j + 4( l i+1, j + l i, j "1 + l i"1, j + l ) i, j "1 + ( l i+1, j +1 + l i"1, j +1 + l i"1, j "1 + l ) i+1, j "1 = 6t i, j Solve iteratively n l +1 i, j = 1 ( 5 l n + l n i+1, j i, j+1 + l n i"1, j + l ) n i, j"1 + 1 ( 20 l n + l n i+1, j+1 i"1, j+1 + l n i"1, j"1 + l ) n i+1, j"1 " 3 10 t i, j Use a discrete approximation to the inverse of Laplacian to obtain initial estimate of l

19 Lightness from Multiple Images taken under Varying Illumination Use spatial statistics of edges Illumination is not smooth Derivative operator responses are sparse ( ) = "r( x, y) + "e( x, y,t) "b x, y,t ( ) # median( "b( x,y,t) ) "r x, y t Y. Weiss ICCV 2001

20 Lightness from Multiple Images taken under Varying Illumination Y. Weiss ICCV 2001

21 Lightness from Multiple Images taken under Varying Illumination Y. Weiss ICCV 2001

22 Using Lightness w/o decomposition w/ decomposition Y. Weiss ICCV 2001

23 Using Lightness Y. Matsushita and K. Nishino CVPR 2003

24 Comments on Lightness Recovery! Not applicable to smooth reflectance variations! Not applicable to curved objects In general Intensity=f( Shape, Reflectance, Illumination )

25 Appearance! How do you describe reflection? " Radiometry and Reflectance! Can we recover geometry from shading? When can we do that? " Photometric Stereo

26 Radiometry and Reflectance

27 Radiometry and Image Formation To interpret image intensities, we need to understand Radiometric Concepts and Reflectance Properties " Image Intensities: Overview " Radiometric Concepts: " Radiant Intensity " Irradiance " Radiance " Bidirectional Reflectance Distribution Function (BRDF) " Image Formation Radiometry vs. Photometry?

28 Light!! Geometric Optics " Light rays " Emission, Reflection/Refraction, Absorption! Wave Optics " Electromagnetic wave (Maxwell s equations) " Diffraction, Interference, Polarization! Quantum Optics " Photons " Fluorescence, Phosphorescence

29 Geometric Optics heat Emission Reflection Refraction Absorption

30 Snell s law for air

31 Image Intensities source surface normal sensor image intensity surface element Image intensity understanding is an under-constrained problem!

32 Radiometric Concepts Solid Angle : Solid Angle subtended by : Foreshortened Area What is the solid angle subtended by a hemisphere?

33 Radiometric Concepts flux Radiant Intensity Light flux (power) emitted per unit solid angle : Flux Surface Irradiance Light flux (power) incident per unit surface area Does not depend on where the light is coming from

34 Radiometric Concepts Surface Radiance (Brightness) Light flux (power) emitted per unit foreshortened area per unit solid angle L = d" ( dacos# r )d$ foreshortened area ( W/m 2 str) : Flux L depends on direction Surface can radiate into whole hemisphere L is proportional to irradiance E Depends on reflectance properties of surface

35 Sun Example Total flux: 3.91x10 26 W" Surface area: 6.07x10 18 m 2 " Radiance by Steve Marschner

36

37 Sun Example Total flux: 3.91x10 26 W" Surface area: 6.07x10 18 m 2 " Radiance by Steve Marschner

38 Sun Example! Same flux on Earth and Mars?

39 Sun Example! Same flux on Earth and Mars? Radiance does not change along the ray! fall off of flux

40 Radiometric Image Formation (Scene Radiance to Image Irradiance) image plane lens scene Image Irradiance: Scene Radiance: Solid angles:

41 Radiometric Image Formation (Scene Radiance to Image Irradiance) image plane lens scene Image Irradiance: Scene Radiance: Solid Angle subtended by lens

42 Radiometric Image Formation (Scene Radiance to Image Irradiance) image plane lens scene Image Irradiance: Scene Radiance: Flux received by lens from = Flux projected onto (image) Image Irradiance

43 Radiometric Image Formation (Scene Radiance to Image Irradiance) We have Solid Angles: Solid Angle subtended by lens: Flux: Image Irradiance: We get image irradiance scene radiance Image irradiance is proportional to scene radiance Telephoto lenses have narrow field of view effects of are small