Understanding Variability

Transcription

1 Understanding Variability Why so different?

2 Light and Optics Pinhole camera model Perspective projection Thin lens model Fundamental equation Distortion: spherical & chromatic aberration, radial distortion Reflection and Illumination: color, Lambertian and specular surfaces, Phong, BRDF Sensing Light Conversion to Digital Images Sampling Theorem Other Sensors: frequency, type,.

3 Basic Radiometry Radiometry is the part of image formation concerned with the relation among the amounts of light energy emitted from light sources, reflected from surfaces, and registered by sensors. Light Source CCD Array Optics p i n e P L(P,d) Surface

4 Lecture Assumptions Typical imaging scenario: visible light ideal lenses standard sensor (e.g. TV camera) opaque objects Goal To create 'digital' images which can be processed to recover some of the characteristics of the 3D world which was imaged.

5 Steps World Optics Sensor Signal Digitizer Digital Representation World reality Optics focus {light} from world on sensor Sensor converts {light} to {electrical energy} Signal representation of incident light as continuous electrical energy Digitizer converts continuous signal to discrete signal Digital Rep. final representation of reality in computer memory

6 Factors in Image Formation Geometry concerned with the relationship between points in the three-dimensional world and their images Radiometry concerned with the relationship between the amount of light radiating from a surface and the amount incident at its image Photometry concerned with ways of measuring the intensity of light Digitization concerned with ways of converting continuous signals (in both space and time) to digital approximations

7 Image Formation Light (Energy) Source Surface Imaging Plane Pinhole Lens World Optics Sensor Signal B&W Film Color Film TV Camera Silver Density Silver density in three color layers Electrical

8 Geometry Geometry describes the projection of: three-dimensional (3D) world two-dimensional (2D) image plane. Typical Assumptions Light travels in a straight line Optical Axis: the perpendicular from the image plane through the pinhole (also called the central projection ray) Each point in the image corresponds to a particular direction defined by a ray from that point through the pinhole. Various kinds of projections: - perspective - oblique - orthographic - isometric - spherical

9 Basic Optics Two models are commonly used:" Pin-hole camera" Optical system composed of lenses" Pin-hole is the basis for most graphics and vision" Derived from physical construction of early cameras" Mathematics is very straightforward" Thin lens model is first of the lens models" Mathematical model for a physical lens" Lens gathers light over area and focuses on image plane."

10 Pinhole Camera Model Image Plane Optical Axis f Pinhole lens World projected to 2D Image Image inverted Size reduced Image is dim No direct depth information f called the focal length of the lens Known as perspective projection

11 Pinhole camera image Amsterdam Photo by Robert Kosara,

12 Equivalent Geometry Consider case with object on the optical axis: z f More convenient with upright image: z - f Projection plane z = 0 Equivalent mathematically

13 Thin Lens Model Rays entering parallel on one side converge at focal point." Rays diverging from the focal point become parallel." f i o IMAGE PLANE LENS OPTIC AXIS 1 = THIN LENS LAW f i o

14 Coordinate System Simplified Case: Origin of world and image coordinate systems coincide Y-axis aligned with y-axis X-axis aligned with x-axis Z-axis along the central projection ray Image Coordinate System p(x,y) y (0,0,0) X Y Y World Coordinate System Z P(X,Y,Z) x (0,0) X Z

15 Perspective Projection Compute the image coordinates of p in terms of the world coordinates of P. p(x, y) y P(X,Y,Z) Z=-f x Z = 0 Z Look at projections in x-z and y-z planes

16 X-Z Projection - f Z x X By similar triangles: x f = X Z+f x = fx Z+f

17 Y-Z Projection y Y - f Z y By similar triangles: = f Y Z+f y = fy Z+f

18 Perspective Equations Given point P(X,Y,Z) in the 3D world The two equations: x = fx Z+f y = fy Z+f transform world coordinates (X,Y,Z) into image coordinates (x,y)

19 Reverse Projection Given a center of projection and image coordinates of a point, it is not possible to recover the 3D depth of the point from a single image. P(X,Y,Z) can be anywhere along this line p(x,y) All points on this line have image coordinates (x,y). In general, at least two images of the same point taken from two different locations are required to recover depth.

20 Stereo Geometry P(X,Y,Z) Object point p l Central Projection Rays p r Vergence Angle Depth obtained by triangulation Correspondence problem: p l and p r must correspond to the left and right projections of P, respectively.

21 Variability in appearance Consequences of image formation geometry for computer vision What set of shapes can an object take on? rigid non-rigid planar non-planar SIFT features Sensitivity to errors.

22 Radiometry Brightness: informal notion used to describe both scene and image brightness. Image brightness: related to energy flux incident on the image plane: IRRADIANCE Illuminance Scene brightness: brightness related to energy flux emitted (radiated) from a surface. RADIANCE luminance

23 Light and Surfaces Reflection mirrors highlights specularities Scattering Lambertian matte diffuse

24 Light sources Point source Extended source Single wavelength Multi-wavelength Uniform Non-uniform

25 Linearity of Light Linearity definition For extended sources For multiple wavelengths Across time

26 Light watts radiated into 4 steradians! watts d" r = d sphere d R = Point Source Radiant Intensity = d (of source) Watts/unit solid angle (steradian)

27 Irradiance Light falling on a surface from all directions. How much? d da Irradiance: power per unit area falling on a surface. d Irradiance E = d watts/m 2

28 Inverse Square Law Relationship between point source radiance (radiant intensity) and irradiance da! watts d" r d = da r 2 d E = d R: Radiant Intensity E: Irradiance d R = d r 2 d = = d r 2 E : Watts : Steradians E = R r 2

29 Surface acts as light source Radiates over a hemisphere Surface Radiance (Extended source) Surface Normal d! R: Radiant Intensity E: Irradiance L: Surface radiance da f Surface Radiance: power per unit foreshortened area emitted into a solid angle L = d da d f (watts/m 2 steradian)

30 Pseudo-Radiance Consider two definitions: Radiance: power per unit foreshortened area emitted into a solid angle Pseudo-radiance power per unit area emitted into a solid angle Why should we work with radiance rather than pseudoradiance? Only reason: Radiance is more closely related to our intuitive notion of brightness.

31 Lambertian Surfaces A particular point P on a Lambertian (perfectly matte) surface appears to have the same brightness no matter what angle it is viewed from. Piece of paper Matte paint Doesn t depend upon incident light angle. What does this say about how they emit light?

32 Lambertian Surfaces Equal Amounts of Light Observed From Each Vantage Point Theta Area of black box = 1 Area of orange box = 1/cos(Theta) Foreshortening rule.

33 Lambertian Surfaces Relative magnitude of light scattered in each direction. Proportional to cos (Theta).

34 Lambertian Surfaces Equal Amounts of Light Observed From Each Vantage Point Theta Area of black box = 1 Area of orange box = 1/cos(Theta) Foreshortening rule. Radiance = 1/cos(Theta)*cos(Theta) = 1

35 The BRDF The bidirectional reflectance distribution function.

36 Geometry Goal: Relate the radiance of a surface to the irradiance in the image plane of a simple optical system. das " Lens Diameter d : Solid angle of patch da s : Area on surface da i : Area in image i! e! dai

37 Light at the Surface E = flux incident on the surface (irradiance) = d d i = incident angle e = emittance angle g = phase angle " = surface reflectance d#! watts Incident Ray r da s " i e g N (surface normal) Emitted ray We need to determine d and da

38 Reflections from a Surface I da da = da s cos i {foreshortening effect in direction of light source} d d = flux intercepted by surface over area da da subtends solid angle d = da s cos i / r 2 d = R d = R da s cos i / r 2 E = d / da s Surface Irradiance: E = R cos i / r 2

39 Reflections from a Surface II Now treat small surface area as an emitter.because it is bouncing light into the world How much light gets reflected? Incident Ray da s i e g N (surface normal) Emitted Ray E is the surface irradiance L is the surface radiance = luminance They are related through the surface reflectance function: L s E = (i,e,g, ) May also be a function of the wavelength of the light

40 das Power Concentrated in Lens Lens Diameter d "! Image Plane! dai L = s d 2 da d s Z -f Luminance of patch (known from previous step) What is the power of the surface patch as a source in the direction of the lens? d 2 = L s da d s

41 das Through a Lens Darkly Lens Diameter d! e " In general: L is a function of the angles i and e. s Lens can be quite large Hence, must integrate over the lens solid angle to get d d = da s L s d

42 Simplifying Assumption Lens diameter is small relative to distance from patch Surface area of patch in direction of lens d = da d = da s L s s L s d d L s is a constant and can be removed from the integral Solid angle subtended by lens in direction of patch = da s cos e = Area of lens as seen from patch (Distance from lens to patch) 2 = 2 (d/2) cos (z / cos ) 2

43 Putting it Together d = da s L s d = da cos e L s s 2 (d/2) cos (z / cos ) 2 Power concentrated in lens: d d = L s da cos e cos 3 s 4 Z 2 Assuming a lossless lens, this is also the power radiated by the lens as a source.

44 das Lens Diameter d Through a Lens Darkly "! Image Plane! dai Image irradiance at da = Z -f i d da i =E i E = i L s da s 2 3 da i 4 d Z cos e cos ratio of areas

45 das Lens Diameter d Patch ratio "! Image Plane! dai Z -f The two solid angles are equal da s cos e da i cos (Z / cos ) 2 = da s cos (-f / cos ) 2 = da i cos e Z -f 2

46 The Fundamental Result Source Radiance to Image Sensor Irradiance: da s da i = cos cos e Z -f 2 E = i L s da s 2 3 da i 4 d Z cos e cos E = i L s cos cos e Z -f 2 4 d Z 2 3 cos e cos E = i L s 4 d -f 2 4 cos

47 Radiometry Final Result E = i L s 4 d -f 2 4 cos Image irradiance is a function of: Scene radiance L s Focal length of lens f Diameter of lens d f/d is often called the 'effective focal length' of the lens Off-axis angle

48 4 Cos Light Falloff Lens Center Top view shaded by height y /2 /2 /2 x

49 Limitation of Radiometry Model Surface reflection can be a function of viewing and/or illumination angle Surface normal N i g! i! e e (i,e,g,, e ) = i dl(e, ) e de(i, ) i Arbitrary reference axis may also be a function of the wavelength of the light source Assumed a point source (sky, for example, is not)

50 Lambertian Surfaces The BRDF for a Lambertian surface is a constant (i,e,g,, ) = k e i function of cos e due to the foreshortening effect k is the 'albedo' of the surface Good model for diffuse surfaces Other models combine diffuse and specular components (Phong, Torrance-Sparrow, Oren-Nayar)

51 BRDF Ron Dror s thesis

52 Reflection parameters

53 Real World Light Variation

54 Fake Light and Real Light

55 Simple and Complex Light

56 BRDF again

57 Likelihood of material

58 Classifying Under Uncertainty

59 More Fake Light

60 Illumination Maps

61 Classification

62 Classification Continued

63 Lighting direction vs. Viewer direction For Lambertian surface: Viewer direction is irrelevant Lighting direction is very relevant

64