Perception of Shape from Shading

Transcription

1 1

2 Percetion of Shae from Shading Continuous image brightness variation due to shae variations is called shading Our ercetion of shae deends on shading Circular region on left is erceived as a flat disk Circular region on right has a varying brightness and is erceived as a shere

3 From Image to Shae Four main factors Geometry of the scene Reflectance of the visible surfaces Piel brightness Illumination direction and distribution Viewoint Can we comute scene geometry from distribution of iel brightness in scene image? Only in very simle situations Too many unknowns in general Viewoint Illumination Scene geometry 3

4 How Do We Do It? Humans have to make assumtions about illumination: bum (left) is erceived as hole (right) when uside down 4 Illumination direction is unknown. It is assumed to come from above

5 Does Shading Play a Central Role? Contour lays a more imortant role Variations in intensity are same on both shaes Uer region is erceived as comosed of three cylindrical ieces illuminated from above Lower region is erceived as sinusoidal, illuminated from one side Note the ambiguities of the surface ercetions, deending on assumed illumination direction ossible illumination hyotheses 5

6 Psychohysics (Percetion of Solid Shae from Shading, Mingolla & Todd, 1986) What assumtions do eole make about surface reflectance? Is an estimate of illumination direction necessary? Stimuli: Shaded ellisoids with varying Elongations Directions of light source Reflectance Cast shadows Test: judge direction of light and surface orientations at discrete oints 6

7 Results Task is hard: errors 15 to 0 degrees No effect of glossiness, no Lambertian surface assumtion No correlation between judgement of light directions and shae No rior estimate of light direction Poor discrimination between elongated and rounded ellisoids Qualitative information 7

8 Human Shading Interretation Is it metric or ordinal? Metric: deth Ordinal: deth order Answer: Ordinal, ualitative Magnitude of shading gradient is not imortant 8

9 Quantitative Shae Recovery Orthograhic rojection We have gray levels at iels (, y) We want to recover the orientations of the normals at oints (, y, z) By integration, we want to obtain z f (,y) Image lane (, y) Orthograhic rojection z θ (, y, z) z f (,y) 9

10 From Normals to Surface Shae Fit a surface that is locally erendicular to the normals 10

11 Review: Radiance Radiance L ( θ 1 ) is ower emitted er unit area (flu) into a cone having unit solid angle Area used is foreshortened area in direction θ 1 L ( θ 1 ) d P / (da cos θ 1 dω), in W/m /sr θ 1 dω Scene, da 11

12 Review: Reflectance Reflection is characterized by reflectance Reflectance is ratio radiance/irradiance Described by a function called Bidirectional Reflectance Distribution Function BRDF BRDF f (θ i,φ i, θ e,φ e ) L (θ e,φ e )/ de(θ i,φ i ) n (θ i,φ i ) (θ e,φ e ) θ e φ e 1

13 Review: Lambertian Surfaces If BRDF is a constant K, surface is called a Lambertian surface de L cos θ 0 dω k L cos θ 0 L K de K 1 L cos θ 0 Radiance is same in all directions and is roortional to cos θ 0 L θ 0 n θ 1 L Scene, da 13

14 14 Piel Brightness and Scene Brightness ) cos / ( cos cos / ( cos α θ α α Z da f da ) cos cos f Z da da θ α θ α π cos cos 4 3 Z D LdA dp θ α π cos cos 4 3 Z D da da L da dp E D f da W a L f D E α π 4 cos 4 da dp LdA Ω cosθ Z L E k

15 Simle Radiometric Modeling Piel Brightness is roortional to radiance of corresonding scene atch Radiance of scene atch is indeendent of viewoint Radiance of scene atch is roortional to cosine of angle between normal to atch and direction of illumination source Therefore iel brightness is roortional to cosine of angle between normal to atch and direction of illumination source 15

16 Normals to z f (, y) We intersect surface zf(,y) with red lane and blue lane We find tangents to red curve and blue curve We write that normal is erendicular to tangents and is in direction of cross-roduct 0 y 0 z f (, y 0 ) Normal 16 ( f /, f / y, y z f ( 0, y) z Red tangent ( 1, 0, f / ) Blue tangent ( 0, 1, f / y) 1)

17 Gradient Sace Orientations of normal ( f /, f / y, 1) can be reresented by arameters f f / d dy The comonents and are called the gradient sace coordinates of the normal / Any direction (a, b, c) can be reresented by (-a/c, -b/c, -1), and by a oint with comonents ( -a/c, -b/c) in the same D gradient sace Eamle: direction of light source can be written ( s, s ) 17

18 Geometric Interretation of Gradient Sace A direction (a, b, c) can be reresented by a oint on the lane Z -1 by constructing the intersection between the vector of same direction (drawn from the origin) and the lane (,, -1) Plane Z -1 (a, b, c) Origin Z0 Z 18

19 Reflectance Ma A reflectance ma is a D looku table that gives the iel brightness as a function of the orientation of the scene surface in camera coordinates 19

20 0 Reflectance Ma for Point Light Source and Lambertian Surface Piel brightness at iel (, y) is roortional to cosine of angle between normal to atch and direction of illumination source For a given iel brightness, the locus of ossible normals (,) in gradient sace is a conic 1 1),, ( 1 1),, ( ) cos( ), ( k k y I s s s s θ ' ) /, ( k k y I s s s s

21 Locus of Iso-Brightness in Reflectance Ma Surface normals that roduce a given brightness are at a constant angle with resect to direction of illumination The directions belong to a cone The locus corresonding to each brightness in the reflectance ma is the intersection of the cone with the lane Z -1 ( s, s ) (, ) Plane Z -1 For a given light source, maimum brightness occurs when (, ) ( s, s ) Z0 Z 1

22 Reflectance Ma Obtained by Calibration Object A shere can be used as a calibration object 1. Find distance of iel to center of shere. If distance < radius, comute direction of normal to shere surface, and (, ) for iel 3. At osition (, ) of reflectance ma, store iel value Useful only for scene material similar to shere Image lane (, y) R y, R R y y

23 Using Reflectance Ma to Find Normals We are on the image at a iel where we know the direction of the normal, a oint in the reflectance ma Find Gradient 1 at iel Find Gradient at reflectance ma oint Move in image by Gradient Then the corresonding oint in reflectance ma is moved by Gradient 1 Image y Nose ti Reflectance Ma 3

24 4 Proof Gradient 1 in image Gradient in reflectance ma If (d, dy) Gradient, Then (d, d) Gradient 1 y I I, R R, y I y R y R d I R R d dy d dy y d d

25 From Normals to Surface Shae Ste by ste Global least suare formulation leads to eression for Lalacian of z z + y z( + d, y + dy) z(, y) + dz z d + z dy d + y dz dy Second order differential euation 5

26 Alication to Face Recognition (Zhao and Chellaa) Aearance of faces changes when viewing and lighting directions change Face databases use front views and frontal lighting If we can reconstruct 3D face shae, we can convert any face image into a front-view with frontal lighting and comare to the database faces Use shae from shading and symmetry of face Or assume generic shae, but varying albedo, and remove unknown albedo by using symmetry of face Synthetic faces for 4 angles and illuminations 6

27 Photometric Stereo Viewoint Move light source at different known ositions to obtain different shadings of object with unknown geometry Find geometry from shading information 3 1 Scene geometry 7

28 Photometric Stereo Different illumination conditions lead to different reflectance mas Each reflectance ma can be comuted if we know osition of oint light source Intersection of iso-brightness contours corresonding to same brightness rovides ossible normal directions for iels having that brightness value Three mas give unambiguous normals for each iel I 19 Reflectance Ma 8

29 Assumtions of Shae from Shading Surfaces with constant albedo Orthograhic rojection Distant oint sources Absence of cast shadows Insignificance of secondary illumination This one is a real roblem: inter-reflections are everywhere 9

30 Inter-Reflections Illumination Gray levels with black surface Accurate comutation of shae from shading is unlikely to succeed in real world Shae from shading may be used as a comlementary rocess Edges are more reliable indicators of shae Gray levels with white surface 30

31 The Real World Diffuse light sources (overcast sky) Interreflections between surfaces generate secondary light sources Surfaces have varying light absortion (albedo) Surface reflections range from Lambertian to secular Surfaces cast shadows on each other 31

32 Conclusions Accurate comutation of shae from shading is unlikely to succeed in the real world Edges are more reliable indicators of shae Shae from shading may be used as a comlementary rocess in combination with shae inference from edges There is still a lot of research activity in this area, so it is useful to have an idea of the terminology and the techniues (reflectance ma, etc.) 3

33 References A Guided Tour of Comuter Vision, Vishvjit S. Nalwa, AT&T Press, 1993 Robot Vision, B.K.P. Horn, MIT Press Perceiving Shae from Shading, V.S. Ramachandran, Scientific American, 1988, SFS Based View Synthesis for Robust Face Recognition, W.Y. Zhao and R. Chellaa, 33