Perception of Shape from Shading. How Do We Do It? From Image to Shape. Does Shading Play a Central Role?

Transcription

1 Percetion of Shae from Shading Continuou image brightne variation due to hae variation i called hading Our ercetion of hae deend on hading Circular region on left i erceived a a flat dik Circular region on right ha a varying brightne and i erceived a a here 1 Four main factor From Image to Shae Geometry of the cene Reflectance of the viible urface Piel brightne Illumination direction and ditribution Viewoint Can we comute cene geometry from ditribution of iel brightne in cene image? Only in very imle ituation Too many unknown in general Viewoint Illumination Scene geometry 3 How Do We Do It? Human have to make aumtion about illumination: bum (left) i erceived a hole (right) when uide down 4 Illumination direction i unknown. It i aumed to come from above Doe Shading Play a Central Role? Contour lay a more imortant role Variation in intenity are ame on both hae Uer region i erceived a comoed of three cylindrical iece illuminated from above Lower region i erceived a inuoidal, illuminated from one ide Note the ambiguitie of the urface ercetion, deending on aumed illumination direction oible illumination hyothee 5 Pychohyic (Percetion of Solid Shae from Shading,Mingolla & Todd, 1986) What aumtion do eole make about urface reflectance? I an etimate of illumination direction neceary? Stimuli: Shaded ellioid with varying Elongation Direction of light ource Reflectance Cat hadow Tet: judge direction of light and urface orientation at dicrete oint 6 1

2 Reult Tak i hard: error 15 to 0 degree No effect of gloine, no Lambertian urface aumtion No correlation between judgement of light direction and hae No rior etimate of light direction Poor dicrimination between elongated and rounded ellioid Qualitative information Human Shading Interretation I it metric or ordinal? Metric: deth Ordinal: deth order Anwer: Ordinal, ualitative Magnitude of hading gradient i not imortant 7 8 Quantitative Shae Recovery Orthograhic rojection We have gray level at iel (, y) We want to recover the orientation of the normal at oint (, y, z) By integration, we want to obtain z = f (,y) Image lane (, y) Orthograhic rojection z θ (, y, z) From Normal to Surface Shae Fit a urface that i locally erendicular to the normal z = f (,y) 9 10 Review: Radiance Radiance L ( θ 1 ) i ower emitted er unit area (flu) into a cone having unit olid angle Area ued i forehortened area in direction θ 1 L ( θ 1 ) = d P / (da co θ 1 dω), in W/m /r Review: Reflectance Reflection i characterized by reflectance Reflectance i ratio radiance/irradiance Decribed by a function called Bidirectional Reflectance Ditribution Function BRDF BRDF = f (θ i,φ i, θ e,φ e ) = L (θ e,φ e )/ de(θ i,φ i ) θ 1 dω n (θ i,φ i ) (θ e,φ e ) Scene, da 11 θ e φ e 1

3 Review: Lambertian Surface If BRDF i a contant K, urface i called a Lambertian urface de = L coθ 0 dω = k L co θ 0 L = K de = K 1 L co θ 0 Radiance i ame in all direction and i roortional to coθ 0 L θ 0 n θ 1 Scene, da 13 L Piel Brightne and Scene Brightne da f D W da coα dacoθ da coα Z = = ( f /coα ) ( Z /coα ) da coθ f π D 3 dp = LdA Ωcoθ dp = LdA co α coθ 4 Z dp daπ D 3 D E = = L co α coθ E π = co 4 α L da da 4 Z 4 f E = k L a Z da 14 Simle Radiometric Modeling Piel Brightne i roortional to radiance of correonding cene atch Radiance of cene atch i indeendent of viewoint Radiance of cene atch i roortional to coine of angle between normal to atch and direction of illumination ource Therefore iel brightne i roortional to coine of angle between normal to atch and direction of illumination ource 15 We interect urface z=f(,y) with red lane and blue lane We find tangent to red curve and blue curve We write that normal i erendicular to tangent and i in direction of cro-roduct Normal to z = f (, y) z 0 z = f (, y 0 ) y 0 Red tangent Blue tangent Normal 16 ( f /, f /, y z = f ( 0, y) ( 1, 0, f / ) ( 0, 1, f / ) 1) Gradient Sace Orientation of normal ( f /, f /, 1) can be rereented by arameter = f / d = f / dy The comonent and are called the gradient ace coordinate of the normal Any direction (a, b, c) can be rereented by (-a/c, -b/c, -1), and by a oint with comonent (= -a/c, = -b/c) in the ame D gradient ace 17 Eamle: direction of light ource can be written (, ) Geometric Interretation of Gradient Sace A direction (a, b, c) can be rereented by a oint on the lane Z= -1 by contructing the interection between the vector of ame direction (drawn from the origin) and the lane (,, -1) Plane Z= -1 (a, b, c) Origin Z=0 Z 18 3

4 Reflectance Ma A reflectance ma i a D looku table that give the iel brightne a a function of the orientation of the cene urface in camera coordinate 19 Reflectance Ma for Point Light Source and Lambertian Surface Piel brightne at iel (, y) i roortional to coine of angle between normal to atch and direction of illumination ource (,, 1) (,, 1) I(, y) = k co( θ ) = k I(, y)/ k = k' = For a given iel brightne, the locu of oible normal (,) in gradient ace i a conic 0 Locu of Io-Brightne in Reflectance Ma Surface normal that roduce a given brightne are at a contant angle with reect to direction of illumination The direction belong to a cone The locu correonding (, ) to each brightne in the reflectance ma i the (, ) interection of the cone with the lane Z = -1 Plane Z= -1 For a given light ource, maimum brightne occur when (, ) = (, ) Z=0 Z 1 Reflectance Ma Obtained by Calibration Object A here can be ued a a calibration object 1. Find ditance of iel to center of here. If ditance < radiu, comute direction of normal to here urface, and (, ) for iel 3. At oition (, ) of reflectance ma, tore iel value Ueful only for cene material imilar to here Image lane (, y) R = =, R y y R y Uing Reflectance Ma to Find Normal We are on the image at a iel where we know the direction of the normal, a oint in the reflectance ma Find Gradient 1 at iel Find Gradient at reflectance ma oint Move in image by Gradient Image Then the correonding oint in reflectance ma i moved by Gradient 1 y Noe ti Reflectance Ma 3 Proof I I Gradient 1 in image =, Gradient in reflectance ma =, If (d, dy) = Gradient, d = d + dy = d + dy I d = + = I d = + = 4 Then (d, d) = Gradient 1 4

5 From Normal to Surface Shae z( + d, y + dy) = z(, y) + dz Ste by te z z dz = d + dy = d + dy Global leat uare formulation lead to ereion for Lalacian of z z = + Second order differential euation 5 Alication to Face Recognition (Zhao and Chellaa) Aearance of face change when viewing and lighting direction change Face databae ue front view and frontal lighting If we can recontruct 3D face hae, we can convert any face image into a front-view with frontal lighting and comare to the databae face Ue hae from hading and ymmetry of face Or aume generic hae, but varying albedo, and remove unknown albedo by uing ymmetry of face Synthetic face for 4 angle and illumination 6 Photometric Stereo Photometric Stereo Move light ource at different known oition to obtain different hading of object with unknown geometry Find geometry from hading information Viewoint 3 Scene geometry 7 1 Different illumination condition lead to different reflectance ma Each reflectance ma can be comuted if we know oition of oint light ource Interection of io-brightne contour correonding to ame brightne rovide oible normal direction for iel having that brightne value Three ma give unambiguou normal for each iel I =19 Reflectance Ma 8 Aumtion of Shae from Shading Surface with contant albedo Orthograhic rojection Ditant oint ource Abence of cat hadow Inignificance of econdary illumination Thi one i a real roblem: inter-reflection are everywhere 9 Illumination Inter-Reflection Gray level with black urface Gray level with white urface Accurate comutation of hae from hading i unlikely to ucceed in real world Shae from hading may be ued a a comlementary roce Edge are more reliable indicator of hae 30 5

6 The Real World Concluion Diffue light ource (overcat ky) Interreflection between urface generate econdary light ource Surface have varying light abortion (albedo) Surface reflection range from Lambertianto ecular Surface cat hadow on each other 31 Accurate comutation of hae from hading i unlikely to ucceed in the real world Edge are more reliable indicator of hae Shae from hading may be ued a a comlementary roce in combination with hae inference from edge There i till a lot of reearch activity in thi area, o it i ueful to have an idea of the terminology and the techniue (reflectance ma, etc.) 3 Reference A Guided Tour of Comuter Viion, Vihvjit S. Nalwa, AT&T Pre, 1993 Robot Viion, B.K.P. Horn, MIT Pre Perceiving Shae from Shading, V.S. Ramachandran, Scientific American, 1988, SFS Baed View Synthei for Robut Face Recognition, W.Y. Zhao and R. Chellaa,