Light. Computer Vision. James Hays

Transcription

1

2

3 Light Computer Vision James Hays

4 Projection: world coordinatesimage coordinates Camera Center (,, ) z y x X... f z y ' ' v u x. v u z f x u * ' z f y v * ' 5 2 ' 2* u 5 2 ' 3* v If X = 2, Y = 3, Z = 5, and f = 2 What are U and V? 3 5-2? z y f v '

5 Interlude: why does this matter?

6 Relating multiple views

7 X x K I z y x f f v u w K Slide Credit: Savarese Projection matrix Intrinsic Assumptions Unit aspect ratio Optical center at (,) No skew Extrinsic Assumptions No rotation Camera at (,,) X x

8 Remove assumption: optical center at origin X x K I z y x v f u f v u w Intrinsic Assumptions Unit aspect ratio No skew Extrinsic Assumptions No rotation Camera at (,,)

9 Remove assumption: square pixels X x K I z y x v u v u w Intrinsic Assumptions No skew Extrinsic Assumptions No rotation Camera at (,,)

10 Remove assumption: non-skewed pixels X x K I z y x v u s v u w Intrinsic Assumptions Extrinsic Assumptions No rotation Camera at (,,) Note: different books use different notation for parameters

11 Oriented and Translated Camera R j w t X k w O w x i w

12 Allow camera translation X t x K I z y x t t t v u v u w z y x Intrinsic Assumptions Extrinsic Assumptions No rotation

13 3D Rotation of Points Rotation around the coordinate axes, counter-clockwise: 1 cos sin sin cos ) ( cos sin 1 sin cos ) ( cos sin sin cos 1 ) ( z y x R R R p p y z Slide Credit: Saverese

14 Allow camera rotation X t x K R z y x t r r r t r r r t r r r v u s v u w z y x

15 Degrees of freedom X t x K R z y x t r r r t r r r t r r r v u s v u w z y x 5 6

16 Reminder: read your book Lectures have assigned readings Szeliski 2.1 and especially cover the geometry of image formation

17 Field of View (Zoom, focal length)

18 Things to remember Vanishing points and vanishing lines Vanishing line Vanishing point Vertical vanishing point (at infinity) Vanishing point Pinhole camera model and camera projection matrix Homogeneous coordinates x K R tx

19 Image Formation Digital Camera Film The Eye

20 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection? λ light source

21 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source

22 A photon s life choices Absorption Diffuse Reflection Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source

23 A photon s life choices Absorption Diffusion Specular Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source

24 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source

25 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source

26 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ 2 λ 1 light source

27 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection λ light source

28 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection t=n t=1 light source

29 A photon s life choices Absorption Diffusion Reflection Transparency Refraction Fluorescence Subsurface scattering Phosphorescence Interreflection (Specular Interreflection) λ light source

30 Lambertian Reflectance In computer vision, surfaces are often assumed to be ideal diffuse reflectors with no dependence on viewing direction.

31 Digital camera A digital camera replaces film with a sensor array Each cell in the array is light-sensitive diode that converts photons to electrons Two common types Charge Coupled Device (CCD) CMOS Slide by Steve Seitz

32 Sensor Array CMOS sensor

33 Sampling and Quantization

34 Interlace vs. progressive scan Slide by Steve Seitz

35 Progressive scan Slide by Steve Seitz

36 Interlace Slide by Steve Seitz

37 Rolling Shutter

38 The Eye The human eye is a camera! Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris What s the film? photoreceptor cells (rods and cones) in the retina Slide by Steve Seitz

39 Aside: why do we care about human vision in this class? We don t, necessarily.

40 Ornithopters

41 Why do we care about human vision? We don t, necessarily. But cameras necessarily imitate the frequency response of the human eye, so we should know that much. Also, computer vision probably wouldn t get as much scrutiny if biological vision (especially human vision) hadn t proved that it was possible to make important judgements from 2d images.

42 Does computer vision understand images? "Can machines fly?" The answer is yes, because airplanes fly. "Can machines swim?" The answer is no, because submarines don't swim. "Can machines think?" Is this question like the first, or like the second? Source: Norvig

43 The Retina Cross-section of eye Cross section of retina Ganglion axons Ganglion cell layer Bipolar cell layer Pigmented epithelium Receptor layer

44 What humans don t have: tapetum lucidum Human eyes can reflect a tiny bit and blood in the retina makes this reflection red.

45 Two types of light-sensitive receptors Cones cone-shaped less sensitive operate in high light color vision Rods rod-shaped highly sensitive operate at night gray-scale vision Stephen E. Palmer, 22

46 Rod / Cone sensitivity

47 . Distribution of Rods and Cones # Receptors/mm2 15, 1, 5, 8 Rods 6 Cones 4 Fovea 2 Blind Spot Rods Cones Visual Angle (degrees from fovea) Night Sky: why are there more stars off-center? Averted vision: Stephen E. Palmer, 22

48 Wait, the blood vessels are in front of the photoreceptors??

49

50 Eye Movements Saccades Can be consciously controlled. Related to perceptual attention. 2ms to initiation, 2 to 2ms to carry out. Large amplitude. Microsaccades Involuntary. Smaller amplitude. Especially evident during prolonged fixation. Function debated. Ocular microtremor (OMT) involuntary. high frequency (up to 8Hz), small amplitude. Smooth pursuit tracking an object

51 Electromagnetic Spectrum Human Luminance Sensitivity Function

52 Visible Light Why do we see light of these wavelengths? because that s where the Sun radiates EM energy Stephen E. Palmer, 22

53 The Physics of Light Any patch of light can be completely described physically by its spectrum: the number of photons (per time unit) at each wavelength 4-7 nm. # Photons (per ms.) Wavelength (nm.) Stephen E. Palmer, 22

54 . The Physics of Light Some examples of the spectra of light sources A. Ruby Laser B. Gallium Phosphide Crystal Wavelength (nm.) D. Normal Daylight # Photons # Photons Wavelength (nm.) C. Tungsten Lightbulb # Photons # Photons Stephen E. Palmer, 22

55 % Photons Reflected The Physics of Light Some examples of the reflectance spectra of surfaces Red Yellow Blue Purple Wavelength (nm) Stephen E. Palmer, 22

56 The Psychophysical Correspondence There is no simple functional description for the perceived color of all lights under all viewing conditions, but... A helpful constraint: Consider only physical spectra with normal distributions mean # Photons area variance Wavelength (nm.) Stephen E. Palmer, 22

57 # Photons The Psychophysical Correspondence Mean Hue blue green yellow Wavelength Stephen E. Palmer, 22

58 # Photons The Psychophysical Correspondence Variance Saturation hi. high med. low medium low Wavelength Stephen E. Palmer, 22

59 # Photons The Psychophysical Correspondence Area Brightness B. Area Lightness bright dark Wavelength Stephen E. Palmer, 22

60 . Physiology of Color Vision Three kinds of cones: nm. RELATIVE ABSORBANCE (%) 1 S M L WAVELENGTH (nm.) Why are M and L cones so close? Why are there 3? Stephen E. Palmer, 22

61 Tetrachromatism Bird cone responses Most birds, and many other animals, have cones for ultraviolet light. Some humans, mostly female, seem to have slight tetrachromatism.

62 More Spectra metamers

63 Practical Color Sensing: Bayer Grid Estimate RGB at G cells from neighboring values Slide by Steve Seitz

64 Color Image R G B

65 Images in Matlab Images represented as a matrix Suppose we have a NxM RGB image called im im(1,1,1) = top-left pixel value in R-channel im(y, x, b) = y pixels down, x pixels to right in the b th channel im(n, M, 3) = bottom-right pixel in B-channel imread(filename) returns a uint8 image (values to 255) Convert to double format (values to 1) with im2double row column G R B

66 Color spaces How can we represent color?

67 Color spaces: RGB Default color space,1, R (G=,B=) 1,, G (R=,B=),,1 Some drawbacks Strongly correlated channels Non-perceptual B (R=,G=) Image from:

68 Color spaces: HSV Intuitive color space H (S=1,V=1) S (H=1,V=1) V (H=1,S=)

69 Color spaces: YCbCr Fast to compute, good for compression, used by TV Y= Y=.5 Y (Cb=.5,Cr=.5) Cr Cb Y=1 Cb (Y=.5,Cr=.5) Cr (Y=.5,Cb=5)

70 Color spaces: L*a*b* Perceptually uniform * color space L (a=,b=) a (L=65,b=) b (L=65,a=)

71 If you had to choose, would you rather go without luminance or chrominance?

72 If you had to choose, would you rather go without luminance or chrominance?

73 Most information in intensity Only color shown constant intensity

74 Most information in intensity Only intensity shown constant color

75 Most information in intensity Original image

76 Back to grayscale intensity

77 Next week Convolution, Filtering, Image Pyramids, Frequencies, Project 1