A Practical Method for Estimation of Point Light-Sources

Transcription

1 Practical Method for Estimatio of Poit Light-Sources Marti Weber ad Roberto Cipolla epartmet of Egieerig, Uiversity of Cambridge Cambridge, CB2 1PZ, UK bstract We itroduce a geeral model for poit light-sources ad show how the parameters of a source at fiite distace ca be estimated by shadig o a object with kow geometry ad Lambertia reflectace. The parameters we estimate iclude ot oly the directio but also the locatio of the source. Furthermore, we argue that the system itroduced here ca be used i arbitrarily complex backgroud illumiatio, provided oe ca switch o ad off the light-source that is to be estimated. 1 Itroductio The effect of a light-source ca be see as a map that supplies each poit i space with a radiace value. The correct estimatio of illumiatio plays a importat role i may braches of Computer Visio. The importace of light estimatio is explicitly apparet i all applicatios which use photometric measuremets (itesities) which obviously deped o the illumiatio. owever, the estimatio of light ca also be crucial for techiques such as texture based visio which are ot explicitly itesity depedet. Cocrete applicatios iclude all recostructio problems that use photometric measuremets (such as shape from shadig [4, 3], photometric stereo [11, 13]), but also for istace ugmeted Reality applicatios [5] where virtual objects have to be redered ito a scee with correct lightig ad shadow. We focus our attetio o recostructio problems while keepig the set-up geeral eough ot to exclude other applicatios. 2 Previous Work I most of the early work doe o recostructio [4, 3], a poit light-source at ifiite distace is assumed [8], i some cases i special aligmet with the camera. Comparably much effort was made i some cases [12], to calibrate the positio ad itesity of light-sources maually to the required degree of accuracy. Some authors also address the related questio of reflectace estimatio i their frameworks [1, 14]. More recet work discusses methods that were developed to estimate complex illumiatio distributios i ifiite distace approximatio. I the literature, oe fids two differet approaches to model the illumiatio distributio over the hemisphere of icidet directios: either as a fuctioal expasio [6] or as a discretisatio by sub-dividig the hemisphere [9]. * Supported by the EPSRC, the Cambridge Europea Trust ad the (Germay).

2 3 Motivatio ad pplicatios s we will demostrate, the stadard assumptios of ifiitely far ad/or sources aliged with the camera, are ot ecessary. Furthermore, we show how to estimate the lightsource reliably, solely based o the shadig observed i oe image. Our approach to the estimatio of light-sources is motivated by the recostructio problem (i.e. the image based recovery of shape ad other properties of viewed objects). The recostructio process ca be viewed as the iverse of the physical process that geerates the image. The physical process, that is simulated by Computer Graphics systems, cosists of a iterplay of the followig igrediets which we have to model. Light-source (Emitter): We itroduce the geeral model of a poit light-source i sectio 4. Surface (Reflector): We assume a kow geometry (objects such as cubes, cyliders or spheres) with homogeous Lambertia reflectace ad covex shape to exclude iter-reflectios. Camera (Receiver): The model of a perspective camera is employed ad pixels are coverted ito radiace values usig the iverse respose fuctio determied i radiometric calibratio. ssume that we have a camera that is geometrically ad photometrically calibrated 1. For the process of geometric calibratio, we refer the reader to [2]. Now, assume further, that we have a sigle image of a kow object which has a homogeeous surface of Lambertia reflectace. The it is coceivable that the set of all possible light-sources that ca geerate the viewed image is severely restricted. Ituitively, it should be possible to determie the directio of the icidet light, provided oe observes the reflectio o at least three poits with liearly idepedet surface-orietatios. s will be demostrated, this is ideed the case. Beyod this ituitio we will demostrate that apart from the directio of the source, other parameters (e.g. its distace) ca be estimated too. I the followig, we will assume that the illumiatio ca be viewed as origiatig from oe poit-like source. We give the followig reasos to justify this assumptio: May sources ca be approximated as poit-like if their distace to the viewed object is much greater tha their extesios. reflector ca be approximated as a poit light-source with a virtual origi lyig behid the actual reflector. The effect of arbitrary complex backgroud illumiatio ca be elimiated by subtractig images take with the source to be estimated switched o ad off. 2 I order to check the result of the estimatio with the reality, we simulated the image formig process ad compared images redered usig the virtual source, object ad camera with the real images. 1 s was poited out i [7], ay camera ca be calibrated photometrically provided the pixel-exposure ca be varied. 2 Note that i order to subtract, we first eed to covert pixel values ito radiace values usig the iverse respose fuctio of the camera.

3 5 4 % % 4 Geeral Poit Light-Source Model 4.1 The assumptios Poit light-source: Light origiates from oe poit. Trasparet media: No sigificat atteuatio of light by the media (air). irect illumiatio: The chage of the light field from iter-reflectios is eglected. -rotatioal symmetry: We assume that there exists at least oe axis of rotatio which leaves the distributio of radiated light ivariat. 4.2 The model Give these assumptios we ca state the most geeral model for the light-source as a map that assigs a radiace to each poit (i a empty space): "!$ &%'(*)' (1) With %'+-,/ :. Where ;9: represets the positio of the source, 5<9=.132 the directio ad )=9>? parameters of the agular depedet stregth. (For a multispectral source, oe would have to replace ad by fuctios depedig also o the wavelegth.) For later referece, we summarise the (first) derivatives of with respect to the model ad 5 : 4.3 Proof of the claims CB )LN 5 B EFG B IF!I )LN IK& B K,53K&7LIK& B MEFG! 5 B IK (2) (3) (4) We have to covice ourselves that usig the above assumptios ay model is of the stated form. For this purpose, imagie a sphere of radius cotaiig the light-source as its cetre (say the origi). Sice we assumed a -symmetry, we ca fid a directio 59 O'P "VXWZY! with the property RQ/TSU536 O P (with some fuctio Q ) ad sice,537 GSU&5[&" we have proved that is of the form claimed. Now cosider a surface patch 5'\ o!. ssumig that the media is trasparet, we ca deduce that the power radiated i the directio is idepedet of the distace while the size of the patch icreases quadratically 3. This proves the geerality of the stated form, ad the derivatives follow by stadard differetiatio. 3 The surface area of a sphere is ]6^6_ P[`Xa<bcXd ]e_ P3`Xab f(g'cxd

4 u j ` ) 4.4 Examples I this paper we restrict ourselves to model sources with hi of the form &%Z6)8 % if % else $k %ml ad a agular depedecy (where % l 9o such that G%*)'qp is esured) 4 which icludes two importat cases: Isotropic source: 5r, is idepedet of the directio. simple spot light-source: 5ts, poits i the directio of maximal itesity ad ecodes the extet of the spot-ature i its absolute legth. % l limits the agle i which the spot radiates (or ca be approximated by this simple agular depedece) 5. 5 Parameter Estimatio from Shadig I the followig two sectios we gather the iformatio eeded to estimate the parameters of the poit light-source solely from shadig. 5.1 Model evaluatio Give ay guess of the parameters, we eed to be able to evaluate the model ad predict the measuremets. s stated earlier, we assumed the camera to be fully calibrated ad the object to be kow. ssumig Lambertia reflectace we ca the predict the irradiace u observed at a give poit vw9o! i the image to be x3ezy where is projected by the camera to v poit (where it is differetiable). 5.2 epedecy o chages i the parameters "h{ (5) ad h is the (outward) ormal of the object at We ca use the derivatives derived i 4.2 to determie the overall chage of the predictio: 5 u 5 $ y "h {} (6) 3E~ y < 6h{ 5Z &<! h B,&< "hz7l B 5ZCB&ƒ < B < ere u is viewed as fuctio of the model ad ) ad hece 5 u B m 5' B m 55 B " 2Eˆ Š ˆ N m 5Z) N ad e has bee calculated i sectio 4 Note that Ž ca have discotiuities for bi 6 ad that we have to esure that the estimatio is restricted to values of o-vaishig Ž. ` 5 Note, that s has its maximum value i the directio of the spot give by. For other directios s declies as cosie of the agle multiplied by the costat

5 5.3 Gauss-Newto method If oe assumes ormally distributed errors i the measuremets, u the optimal parameters are estimated as solutio of a o-liear least square problem. We solve this problem usig the Gauss-Newto method which is comparably fast ad coverges rapidly (as our experimets cofirm). The merit of this method to our problem was also tested idepedetly by data simulated with Gaussia oise. Besides the estimated optimum parameter combiatio, the method also delivers! which we use i the stadard way to estimate the agreemet of the data to our model. 6 The Estimatio Process I this sectio we describe the full estimatio process i some detail. The iput for our method are two images of a kow object take of the idetical scee, oe with the poit light-source switched o ad oe with it tured off. Figure 1 demostrates the process we outlie i the followig. Figure 1: Estimatio process: Show (left to right) are the effective oe source image, a image idicatig the geometric calibratio as well as the area where data is acquired for the estimatio, ad fially the sythetic image redered usig the estimated parameters. Effective oe source image: I order to elimiate effects of possibly complex backgroud illumiatio, we use the iverse respose fuctio [7] to trasform the iput images ito radiace images which we ca the subtract from each other to give a effective oe source image. Geometric calibratio: To proceed further, we eed to fid the pose of the kow covex Lambertia object viewed. I this paper we use the model of a cube which aturally defies a coordiate system: we defie the origi to be the ier corer viewed ad use the three eighbourig corers to itroduce the uit base vectors: The first uit vector as leadig from the origi alog the edge to the top-right corer, the secod uit vector as leadig to the top-left corer ad fially the third uit vector as leadig to the bottom corer. Note that here we itroduced a legth scale. The calibratio is performed by maually clickig at the seve visible corers ad the iitial calibratio ca be refied usig sakes [1]. The perspective projectio matrix ca be calculated from the correspodig poits i the real world ad i the image. t this stage, we kow how to project our model ito the image geometrically but do ot yet kow about the correct illumiatio.

6 Object Light-model directio distace ormalised cube (view 1) ifiite distace $ [ [ Z1[ $ ' $ 'œ [ $ fiite distace 8 Table 1: Compariso of the ifiite distace estimatio with the fiite distace estimatio:! ormalised is expected to be approximately oe. Clearly the fiite distace model is much more cosistet with the data. Samplig of radiace values: Usig the kow pose of the cube we ca record the radiace value at each pixel ad associate it with the correspodig surfacepoit ad -ormal. s idicated i Figure 1, we also use the geometric calibratio to segmet eigbourhoods of edges i the image. We exclude these regios from the light estimatio to avoid errors due to imperfectios of the real cube ear the edges 6. Note that each o-excluded pixel o the cube delivers data which ca be used for the light estimatio ad we have a large data set. Light estimatio: Our aim is to estimate the parameters of a isotropic poit light-source. To iitialise the o-liear fiite distace model we first solve the liear least square problem for the ifiite-distace light model. Fially the Gauss- Newto method is used to estimate the parameters. Table 1 demostrates the improvemets achieved usig the fiite distace model. Note that the directio estimated i the fiite distace model is differet from the oe i the ifiite model. Compariso with the origial image: s is illustrated i Figure 1, we ca ow proceed to reder a sythetic versio (usig ray-tracig) ad compare it with the actual iput image 7. Figure 2 highlights the differeces betwee origial ad sythetic images ad the histogram quatifies the quality of the estimatio. 7 Experimetal Results 7.1 Experimetal set-up We use a iexpesive desktop haloge lamp (2W) as the light-source. s object of kow geometry, we use two cubes which were covered with paper of the same type to approximate homogeous Lambertia reflectace. The approximate dimesios of the cubes are 9.5cm (referred to as cube) ad 13cm (referred to as CUBE). The cubes are i tur placed o a tur-table with a dark surface to avoid iter-reflectios. (We used the tur-table later (sectio 7.3) for a cosistecy check of positios estimated at differet rotatios.) s recordig system, we used the video-camera (Soy C-77RR-CE ad amplifier XC-77RR-CE) which ca produce a gamma-corrected sigal so that we did ot eed to calibrate the camera radiometrically, although this could be easily doe as described i [7]. We estimated the ucertaity of the pixel-values, which is assumed i our modelig) empirically to approximately 8 (of 256). 6 Recall that the ormal field has discotiuities at the edges. 7 More precisely with the effective oe-source radiace image.

7 d Ÿ œ œ N=14349 µ=.27 σ= Figure 2: Compariso of origial ad sythetic images: The first image shows the absolute value of radiace differeces multiplied by a factor of 3. I the ideal case the cube should be black. The secod image shows the radiace differeces multiplied by a factor of 2 ad offset by the radiace value correspodig to the displayed square. I the ideal case the cube should have the itesity of the square. The histogram shows the distributio of the differeces. The x-axis correspods to the radiace differece ad is ormalised such that a value of 1 correspods to the maximum radiace observed. Object View! ormalised Z 2 $L Zœ L ' $ $ [ [ [ Z L 3L Z $ [ [ [ L [L Z $ [ [ Z$ [ [ $L [L $ '$ [ [ [ ZZ CUBE L [ L $ $ [ [ Z [L Z [L ' $ Z$ [ '$ [ ' L 3L $ [ [œ [ ' Z [L œ L $ [ ' [ ' Table 2: Results of the estimatio for the various images.! ormalised approximately oe. Note that the variatios i, which is correlated to ad hece the stregth œ œ Z Z Z Z [ Z [ is expected to be are much larger the variatios i 2! caot be estimated very accurately. 7.2 Estimatio of parameters ad compariso with origial images We performed the aalysis outlied i sectio 6 for four differet views of both cubes. Each time the tur-table was rotated approximately Z. The full estimatio process (coded i C++) takes about 1 secods per image o a Petium-III 9Mz machie without ay particular optimisatio. The results are displayed i Table 2 8 ad Figures 3, 4. Note that the histograms cofirm our assumptio o early Gaussia errors ad the small value of aroud 2% for the stadard deviatio meas that the model predicts the itesities successfully. 8 The larger value of ormalised ad origiates possibly from errors i the o-perfectly flat surfaces of CUBE.

8 2.6 N=14349 µ=.27 σ= N=15381 µ=.13 σ= N=14229 µ= 1.8ε 6 σ= N=13839 µ=.25 σ= Figure 3: Estimatio Results (cube): The first row shows the origial images of the cube from the four differet views. I the secod row, the differeces of sythetic ad real radiace values are displayed (multiplied by a factor of 2 ad offset by the radiace of the square area). The histograms show the distributio of radiace differeces alog with the total umber of pixels that were evaluated ( ), the mea (ª ) ad the stadard deviatio («). 2.6 N=29445 µ=.45 σ= N=311 µ=.42 σ= N=2994 µ=4ε 5 σ= N=271 µ=.37 σ= Figure 4: Estimatio Results (CUBE): (Images as i Figure 3 but for CUBE). The structure that ca be see i the subtractio images (secod row) ad the presece of tails i some of the histograms (third row) idicate that the surfaces are ot perfectly flat. This iterpretatio is i agreemet with the larger values for! ormalised i Table 2.

9 4 4 ³ 4 4 & cube & [ 32E $ $ 2 $ 8 $ Z Z $ $ $ 8 $ [ [ $ $ [ $ 8 [ [ $ $ $ $ [ $ ' ª $ $ $ 8 «[ Z 3 $ $ [ [ CUBE & 32E & Zœ * 8 $ $ $ 2 13 [ ' & $ Z [* ' $ $ 13 [ ZZ & * * Z $ $ [ ZZ $ Z $* $ $ [ [ ZZ ª $ $ [ [ $ ZZ «$ 'œ [ [ [ œ œ [ Table 3: greemet of the estimatios i differet views: Light positios estimated for differet views are trasformed ito the same coordiate system ad deoted by &. Values i each colum should agree. I the lower half, mea ad stadard deviatio of the above quatities are displayed. 7.3 Rotatioal cosistecy check ad accuracy I this sectio we ivestigate if the estimatios performed idepedetly for each view agree. Movig the object i frot of the fixed light-source is equivalet to movig the light-source with the object fixed. To that extet we ca use the projectio matrices to extract the relative motio ad this should eable us to trasform all estimates ito a commo coordiate system i which the positio of the light-source should be ivariat. First, from each projectio matrix, we ca extract a rigid-body trasformatio 9 : "±<²U± with ²µ9o '( 9i ³ Next, we ca express the relative motio as BN B¹ N which trasforms coordiates of view º ito coordiates of view». Table 3 shows the result of trasformig all light-estimatios ito the coordiates of the first image. Furthermore, we ca combie the estimatios to get a improved estimate o the lightsource ad to estimate the ucertaity (table 3). Note that as aticipated, the ucertaity i distace is much larger tha i orietatio. 8 Future Work ad Coclusio We aim to exted the aalysis doe usig cubes to other objects like cyliders ad spheres ad also iclude o-homogeous ad o-lambertia objects. I a further step, we pla to implemet the estimatio to o-isotropic sources as itroduced i sectio 4.2. Fially, it might be possible to exted the estimatio to situatios where more tha oe but a small umber of sources are to be estimated. We have preseted a model for poit light-sources which allowed us to successfully recover the positio of a source ad its parameters. s we have demostrated, it is possible ad worthwhile for may applicatios to work with a fiite distace model. The method is practical i that it does ot demad ay special imagig system or eviromet ad just oe image with a kow object is sufficiet for the estimatio. The direct compariso of real images with images redered uderlie the power of the method preseted. 9 We use the stadard method which correspods essetially to a QR-decompositio.

10 Refereces [1]. Blake ad M. Isard. ctive Cotours. Spriger, New York, [2] R. Cipolla ad P. Gibli. Visual Motio of Curves ad Surfaces. Cambridge Uiversity Press, Cambridge, 2. [3] B.K.P. or ad M.. Brooks. The variatioal approach to shape from shadig. Computer Visio, Graphics, ad Image Processig, 33(2):17428, [4] K. Ikeuchi ad B.K.P. or. Numerical shape from shadig ad occludig boudaries. rtificial Itelligece, 17:141184, [5] C. Loscos, M. Frasso, G. rettakis, B. Walter, X. Graier, ad P. Pouli. Iteractive virtual relightig ad remodelig of real scees. I Rederig Techiques 99 (1th EG workshop o Rederig). Spriger, Wie, ue [6] S.R. Marscher ad.p. Greeberg. Iverse lightig for photography. I Proceedigs of IS&T/SI Fifth Color Imagig Coferece, pages , November [7] S.K. Nayar ad T. Mitsuaga. Radiometric self calibratio. Proceedigs of IEEE Coferece o Computer Visio ad Patter Recogitio, I:37438, ue [8].P. Petlad. Fidig the illumiatio directio.. Opt. So. m., 72(4):448455, [9] I. Sato, Y. Sato, ad K. Ikeuchi. Illumiatio distributio from brightess i shadows: daptive estimatio of illumiatio distributio with ukow reflectace properties i shadow regios. I Proceedigs of IEEE ICCV 99, volume 2, pages , [1] P. Siha ad E.. delso. Recoverig reflectace ad illumiatio i a world of paited polyhedra. Proc 4th It. Cof. o Comp. Vis., pages , [11] R.. Woodham. Photometric method for determiig surface orietatio from multiple images. Optical Egieerig, 19(1):139144, 198. [12] R.. Woodham. Gradiet ad curvature from the photometric-stereo method, icludig local cofidece estimatio. OS, 11:35368, [13] P.L. Worthigto ad E.R. acock. New costraits o data-closeess ad eedle map cosistecy for shape-from-shadig. IEEE-PMI, 21: , [14] Y. Yu, P. ebevec,. Malik, ad T. awkis. Iverse global illumiatio: Recoverig reflectace models of real scees from photographs. SIGGRP99 coferece proceedigs, pages , 1999.