Applications of Light Polarization in Vision

Transcription

1 Applications of Light Polarization in Vision Lecture #18 Thanks to Yoav Schechner et al, Nayar et al, Larry Wolff, Ikeuchi et al

2 Separating Reflected and Transmitted Scenes Michael Oprescu, Reconstructing Shape of Transparent Objects Removing Specularities Removing Haze and Underwater Scattering Effects

3 Separation of Diffuse and Specular Reflections Diffuse surfaces : No (or minimal) Polarization All light depolarized due to many random scattering events inside object. Specular Surfaces: Strong Polarization (even though partially polarized) Smooth/Rough Surfaces: The degree of polarization decreases with roughness.

4 Active Illumination Completely remove specular reflections using polarized light when the filters are 90 degrees apart. Commonly used in industrial settings.

5 Passive Illumination Most illumination from sources (sun, sky, lamps) is unpolarized. Merely using a polarizer will not remove specular reflections completely.

6 I_min is not equal to I_d (diffuse component)

7 Polarization Measurements polarizer camera θ max I max Polarization vector determination 3 general measurements suffice I min θ max 180 o

8 Determining the Polarization Cosine Curve Using Vector Notation: Three measurements suffice to determine the cosine curve.

9 Degree of Polarization Varies between 0 and 1. If zero, then there is no polarization Only diffuse component present. If one, only specular component present. If degree of polarization does not change as polarizer is rotated, then there is no guarantee that specular component is completely removed (I_sc may still be present).

10 Fresnel Ratio I_sc and I_sv depend on refractive index and angle of incidence. I_sc and I_sv are related to fresnel coefficients: is fresnel coefficient perpendicular to plane of incidence is fresnel coefficient parallel to plane of incidence

11 Fresnel Ratio Metals Dielectrics Hard to separate diffuse and specular parts for metals. Brewster angle Easier for dielectrics (good for non-normal incidences).

12 Dichromatic Model for Removing Specularities Completely Specularities are only reduced in intensity using polarization. They are removed completely only for the Brewster angle of incidence. Nayar et al. use additional color constraints in dichromatic model to remove reflections completely. Assume a local patch where the highlight and its surrounding area have the same diffuse component.

13 Semi-Reflections Michael Oprescu, Both Reflected and Transmitted light are polarized. But they are polarized differently. They depend on the orientation of the transparent layer. Reflections are removed completely only at Brewster Angle of Incidence.

14 Transparent Layers window camera

15 Semi-Reflections transmitted scene I T reflected scene I R ϕ ϕ window camera

16 Yoav Schechner, Joseph Shamir, Nahum Kiryati 99 reflected scene I r transmitted scene I t ϕ ϕ r I r polarizer 0.8 window r I r camera r r Optical coding ϕ

17 Yoav Schechner, Joseph Shamir, Nahum Kiryati 99 reflected scene I r transmitted scene I t ϕ ϕ t I t r I r polarizer 0.8 t=1-r window t I t r I r camera r r Optical coding ϕ

18 Experiment reflected scene transmitted scene I t I r window observed image Yoav Schechner, Joseph Shamir, Nahum Kiryati 99

19 Optical coding reflected scene transmitted scene I t I r observed image window I I = [ r + t I ]/ 2 I r t Yoav Schechner, Joseph Shamir, Nahum Kiryati 99

20 Optical coding reflected scene transmitted scene I t I r observed image window I I = [ r I r + I t t ] / 2 Yoav Schechner, Joseph Shamir, Nahum Kiryati 99

21 Digital decoding 2 Linear equations I = [ I r r + I t t ]/ 2 I = [ I r t I t r + ]/ 2 Solve for 2 unknowns: I R, I T Window at 27 o I I Reflected I R 2 _ 2 r _ r r _ 2 2 r r r I I Transmitted I T 2 r _ r r _ 2 r _ r r Yoav Schechner, Joseph Shamir, Nahum Kiryati 99

22 17 o 27 o 37 o Transmitted I T Reflected I R negative crosstalk positive crosstalk _ 0.4 _ 0.8 The inclination of an invisible surface ϕ

23 Imaging through Haze clear day moderate haze very hazy Recover: Object + haze layers Scene structure Info about the aerosols Previous work Pure image processing Grewe & Brooks 98, Kopeika 98 Oakley & Satherley 98 Physics based Nayar & Narasimhan 99 Polarization filtering Shurcliff & Ballard 64 Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

24 Imaging through Haze Airlight A object radiance R camera direct transmission T scattering Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

25 Airlight A object radiance R camera direct transmission T scattering total I ( x, y) = T( x, y) + A( x, y) 1 1 z is a function of (x,y) 0 e βz R( x, y ) z 0 1 e βz z A Color Multiplicative & additive models - similar dependence

26 Polarization and Haze polarizer A camera A direct transmission Plane of rays determines airlight components A A _ A Airlight degree of polarization p A + A > A A =0 A = A polarized unpolarized p=1 p=0 Along the line of sight, polarization state is distance invariant Assume: The object is unpolarized T / all orientations Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

27 Trivial case I I Life is tough still, there is a dominant polarization Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

28 Experiment Best polarized image I = T/2 + A Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

29 Experiment Worst polarized image I = T/2 + A Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

30 Model airlight A object radiance R camera transmission T 2 input images: I = T / 2 + A I = T / 2 + A transmission airlight T = R e βz = βz A A 1 - e polarization degree p A A _ + A A Recovery depth radiance e β z I + I I I R = ( ) ( )/ βz e for known ( = I I )/p 1 A p, A p

31 Model camera saturated airlight A airlight polarization p 2 input images: I = T / 2 + A I = T / 2 + A transmission airlight T = R e βz = βz A A 1 - e polarization degree p A A _ + A A Recovery depth radiance e β z I + I I I R = ( ) ( )/ βz e for known ( = I I )/p 1 A p, A p

32 Dehazing Experiment I Best polarized image Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

33 Dehazing Experiment R Dehazed image Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

34 Range map depth e β z = 1 ( I I ) pa log βz( x, y) component images I ( x,y), I ( x,y) Airlight saturation polarization A p

35 Dehazing Experiment I Best polarized image Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

36 Dehazing Experiment R Dehazed image Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

37 Range map Instant Dehazing: Yoav Schechner, Srinivasa Narasimhan, Shree Nayar

38 veiling light B signal S I total (, xy) = Sxy (, ) + Bxy (, ) 1 1 z is a function of (x,y) e ηz Leffective (, xy) object + 1 e η z B Color 0 z 0 z - object with blur

39 Hypothesis, 4 Decades Old Lythgoe & Hemmings, 1967 (Nature) : Many invertebrates are able to distinguish the plane of polarized light. Does this enable them to see further underwater? Lythgoe, 1972 (Handbook Sensory Physiol) : there is a strong possibility that it [polarization] could be useful for improving the visibility of distant objects, especially under water.

40 Hypothesis, 4 Decades Old Lythgoe & Hemmings, 1967 (Nature) : Many invertebrates are able to distinguish the plane of polarized light. Does this enable them to see further underwater? when the [polarizing] screen was oriented to exclude the maximum spacelight fishes stood out in greater contrast against their background. simple polarizing screen will be less versatile than the system found in Octopus, where there is the intra-ocular ability to distinguish light polarized in one plane from that polarized in another. Lythgoe, 1972 (Handbook Sensory Physiol) : there is a strong possibility that it [polarization] could be useful for improving the visibility of distant objects, especially under water.

41 Polarization of Veiling Light B B Veiling light is partially polarized Y. Schechner & N. Karpel, polarization-based recovery

42 24 Image Components veiling light B signal S scattering L Veiling light = Spacelight = Path radiance = Backscatter Schechner, Karpel, underwater vision

43 Signal Polarization Rough surfaces : naturally depolarize Specular reflection : weaker than in air Multiple scattering Signal decreases with distance / veiling-light increases At large distance: signal polarization has a negligible effect (Supported by Shashar, Sabbah & Cronin 2004) Y. Schechner & N. Karpel, polarization-based recovery

44 Polarization Photography 25

45 Past Polarization-Based Methods I min I max I min I max I min I max I min I max Raw images Polarization-difference imaging Degree of polarization

46 Model 2 input images: camera Recovery

47 2 input images: camera Recovery

48 Aqua-polaricam Y. Schechner & N. Karpel, underwater imaging

49 Experiments

50 Experiment Eilat, 26m underwater I min Best polarization image Y. Schechner & N. Karpel, underwater imaging

51 Naive White Balancing 26m underwater Y. Schechner & N. Karpel, underwater imaging

52 Y. Schechner & N. Karpel, underwater imaging 26m underwater

53 Range Map Attenuation Image components backscatter Y. Schechner & N. Karpel, underwater imaging

54 Shape Reconstruction of Transparent Objects Miyazaki et al Incident light is completely unpolarized. Index of refraction is given. Exploit relation between degree of polarization and angle of incidence (Surface normal).

55 Relationship between DOP and Angle of Incidence Two-way ambiguity in recovered angle of incidence: Manually disambiguate, use multiple views or use prior knowledge (convex, concave, etc).

56 Recovered Shape

57 NEXT WEEK Volumetric Scattering and its Applications to Computer Vision and Computer Graphics Lectures #18, #19, #20