Physics-based Methods in Vision

Transcription

1 LIGHT AND COLOR The slides are from several sources through James Hays (Brown); Srinivasa Narasimhan (CMU); Bill Freeman and Antonio Torralba (MIT), including their own slides.

2 Physics-based Methods in Vision Camera Lighting Computer Physical Models Scene We need to understand the relation between the lighting, surface reflectance and medium and the image of the scene.

3 Light and Shadows

4

5

6 Reflections

7

8

9

10 Refractions

11

12

13 Interreflections

14

15 Scattering

16

17

18

19 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection? λ light source

20 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source ex. sunglasses absorpt some of the wavelengths

21 A photon s life choices Absorption Diffuse Reflection Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source in every direction is the same ex. frost glass, microscopically rough

22 A photon s life choices Absorption Diffusion Specular Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source just in one direction ex. glossy paints have both reflection

23 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source ex. glass window passes light

24 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection e.g. the straw in a glass λ light source

25 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ 2 λ 1 light source emission at different wavelengths ex. minerals emit visible light exposed to ultraviolet light

26 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source scattering interaction with material ex. realistic rendering of marble, skin, milk etc.

27 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection t=n t=1 light source re-emit in time, 1...n... ex. clock dials that glow

28 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source specular interreflection here

29 Surface Appearance source sensor normal surface element Image intensities = f ( normal, surface reflectance, illumination ) Surface reflection depends on both the viewing and illumination directions.

30 BRDF: Bidirectional Reflectance Distribution Function spherical coordinate system x z φ θ y source incident direction θ, ) θ r, φ ) ( i φi normal surface element ( r viewing direction surface E surface L ( θi, φi ) θ, φ ) ( r r Irradiance at Surface in direction ( θ i, φi ) Radiance of Surface in direction θ r, φ ) ( r BRDF : f ( θ, φ ; θ, φ ) = i i r r L E surface surface ( θr, φr ) ( θ, φ ) i i out ---- in

31 BRDFs can be incredibly complicated

32 Important Properties of BRDFs x z φ θ y source incident direction θ, ) θ r, φ ) ( i φi normal surface element ( r viewing direction Conservation of Energy: f hemisphere ( θ, φ ; θ, φ ) dω 1 i i r r i

33 Important Properties of BRDFs x z φ θ y source incident direction θ, ) θ r, φ ) ( i φi normal surface element ( r viewing direction Helmholtz Reciprocity: (follows from 2 nd Law of Thermodynamics) BRDF does not change when source and viewing directions are swapped. f ( θ i, φi; θr, φr ) = f ( θr, φr; θi, φi )

34 Important Properties of BRDFs x z φ θ y source incident direction θ, ) θ r, φ ) ( i φi normal surface element ( r viewing direction Rotational Symmetry (Isotropy): BRDF does not change when surface is rotated about the normal. Can be written as a function of 3 variables : f θ, θ, φ φ ) ( i r i r

35 Mechanisms of Reflection source incident direction surface reflection body reflection surface Body Reflection: Diffuse Reflection Matte Appearance Non-Homogeneous Medium Clay, paper, etc Surface Reflection: Specular Reflection Glossy Appearance Highlights Dominant for Metals Image Intensity = Body Reflection + Surface Reflection

36 Diffuse and Specular Reflection diffuse specular diffuse+specular

37 example: Whiteout Snow and Overcast Skies Lambertian BRDF = a constant diffuse reflection can't perceive the shape of the snow covered terrain but can perceive shape in regions lit by the street lamp now has more specular components

38 The Eye Iris Ii - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris Lens - changes shape by using ciliary muscles (to focus on objects tdiff at different tdi t distances) ) Retina - photoreceptor cells Slide by Steve Seitz

39 Rods and cones cone pigment pg molecules rod Rods are responsible for intensity, cones for color perception Rods and cones are non-uniformly distributed on the retina Fovea - Small region (1 or 2 ) at the center of the visual field containing the highest density of cones (and no rods) Slide by Steve Seitz

40 Rod / Cone sensitivity Why can t we read in the dark? Slide by A. Efros

41 Electromagnetic spectrum Human Human Luminance Luminan Sensitivity Function

42 Visible Light Why do we see light of these wavelengths? because that s where the Sun radiates EM energy Stephen E. Palmer, 2002

43 Physiology of Color Vision Three kinds of cones: nm. RELATIVE ABSORBAN NCE (%) 100 S M L WAVELENGTH (nm.) Ratio of L to M to S cones: approx. 10:5:1 Almost no S cones in the center of the fovea Stephen E. Palmer, 2002

44 Spectra of some real-world surfaces metamers

45 The Physics of Light Some examples of the reflectance spectra of surfaces % Photons Reflected Red Yellow Blue Purple Wavelength (nm) Stephen E. Palmer, 2002

46 Color Images in Matlab Images represented as a matrix 3x(NxM) Suppose we have a NxM RGB image called im im(1,1,1) = top-left pixel value in R-channel im(y, x, b) = y pixels down, x pixels to right in the b th channel im(n, M, 3) = bottom-right pixel in B-channel imread(filename) returns a uint8 image (values 0 to 255) Convert to double format (values 0 to 1) with im2double row column G R B

47 Color spaces How can we represent color? here additive combination

48 Color spaces: RGB Default color space 0,1,0 R (G=0,B=0) 1,0,0 G (R=0,B=0) 0,0,1 Some drawbacks Strongly correlated channels Non-perceptual B (R=0,G=0) Image from:

49 Linear color space CIE XYZ from RGB Primaries are imaginary, but matching functions are everywhere positive, not in RGB The Y parameter corresponds to brightness or luminance of a color 2D visualization: draw (x,y), where x = X/(X+Y+Z), y = Y/(X+Y+Z) Matching functions Z Y X

50 Forsyth & Ponce xy-axes for a constant brightness

51 Pure wavelength in chromaticity diagram Blue: big value of Z, therefore x and y small x=x/(x+y+z) y=y/(x+y+z)

52 Pure wavelength in chromaticity diagram Then y increases x=x/(x+y+z) y=y/(x+y+z)

53 Pure wavelength in chromaticity diagram Green: y is big x=x/(x+y+z) y=y/(x+y+z)

54 Pure wavelength in chromaticity diagram Yellow: x & y are equal x=x/(x+y+z) y=y/(x+y+z)

55 Pure wavelength in chromaticity diagram Red: big x, but y is not null x=x/(x+y+z) y=y/(x+y+z)

56 Color spaces: L*a*b* Perceptually uniform color space Nonlinear transformations of XYZ space. and L*u*v* L (a=0,b=0) a (L=65,b=0) distances quasi Euclidean b (L=65,a=0)

57 A. CIE VI. CIE COLOUR SPACES CIE, the International Commission on Illumination - abbreviated as CIE from its French title Commission Internationale de l Eclairage - is an organization devoted to international cooperation and exchange of information among its member countries on all matters relating to the science and art of lighting [2]. In 1931 CIE laid down the CIE 1931 standard colorimetric observer. This is the data on the ideal observer on which all colorimetry is based [5, page 131]. B. CIE XYZ CIE standardized the XY Z values as tristimulus values that can describe any colour that can be percepted by an average human observer (the CIE 1931 standard colorimetric observer). These primaries are nonreal, i.e. they cannot be realized by actual colour stimuli [5, page 138]. This colour space is chosen in such a way that every perceptible visual stimulus is described with positive XY Z values. A very important attribute of the CIE XYZ colour space is that it is device independent. Every colour space that has a transformation from the CIE XYZ colour space (RGB 709, CIELab, CIELuv) can also be regarded as being device independent. The CIE XYZ colour space is usually used as a reference colour space and is as such an intermediate device-independent colour space. C. CIE Luv and CIE Lab colour spaces In 1976 the CIE proposed two colour spaces (CIELuv and CIELab) whose main goal was to provide a perceptually equal space. This means that the Euclidian distance between two colours in the CIELuv/CIELab colour space is strongly correlated with the human visual perception. To achieve this property there were two main constraints to take into account: chromatic adaptation non-linear visual response The main difference between the two colour spaces is in the chromatic adaptation model implemented. The CIE Lab colour space normalizes its values by the division with the white point while the CIELuv colour space normalizes its values by the subtraction of the white point. The transformation from CIE XYZ to CIE Luv is performed with the following equations ( ) 1 Y L 3 = Y n u = 13L (u u n) v = 13L (v v n) for Y Y n > 0.01, otherwise the following L formulae is used L = Y Y n u 4X n n = X n + 15Y n + 3Z n v 9Y = X + 15Y + 3Z v n 9Y n = X n + 15Y n + 3Z n The tristimulus values X n, Y n, Z n are those of the nominally white object-colour stimulus. The transformation from CIE XYZ to CIE Lab is performed with the following equations ( Y L = 116 a = 500 b = 200 [ ( X X n [ ( Y Y n Y n ) 1 ) 1 ) ( ) 1 ] 3 Y 3 Y n ( ) 1 ] 3 Z 3 Z n The perceptually linear colour difference formulaes between two colours are E ab = ( L ) 2 + ( a ) 2 + ( b ) 2 E uv = ( L ) 2 + ( u ) 2 + ( v ) 2 VII. CONCLUSION In this paper we have presented an overview of colour spaces used in image processing. We have tried to stress the importance of the historical and perceptual background that has led to the introduction of these colour spaces. REFERENCES [1] Symon D O. Cotton, Colour, colour spaces and the human visual system, School of Computer Science, University of Birmingham, England, Technical Report, B15-2TT, May [2] CIE home page [3] Charles Poynton, A Guided Tour of Color Space, New Foundations for Video Technology (Proceedings of the SMTPE Advanced Television and Electronic Imaging Conference), 1995, pages [4] Charles Poynton, Frequently Asked Questions about Color, poynton, [5] Gunter Wyszecki, W.S. Stiles, Color Science Concepts and Methods, Quantitative Data and Formulae, John Wiley and Sons, Inc, [6] Mark D. Fairchild, Color Appearance Models, Addison Wesley, [7] Henryk Palus, Colour spaces, Chapmann and Hall, [8] Adrian Ford and Alan Roberts, Colour space conversions, Westminster University, London, L between 0 and 100 u* between -134 and 200 v* between -140 and 122 The quantities u, v and u n, v n are calculated from u = 4X X + 15Y + 3Z

58 L*u*v* the inner solid

59 not completely uniform (s.d. x10)

60 If you chose only chrominance, say, a and b... Only color shown constant intensity

61 ...if you chose only luminance L. Only intensity shown constant color

62 Most information in intensity... Original image

63 ...so back to grayscale intensity