Object centered stereo: displacement map estimation using texture and shading

Transcription

1 Object centered stereo: dsplacement map estmaton usng texture and shadng Nel Brkbeck Dana Cobzas Martn Jagersand Computng Scence Unversty of Alberta Edmonton, AB T6G2E8, Canada Abstract Input Data Output Data We consder the problem of recoverng 3D surface dsplacements usng both shadng and mult-vew stereo cues. In contrast to tradtonal dsparty or depth map representatons, the object centered dsplacement map representaton enables the recovery of complete 3D objects whle also ensurng the reconstructon s not based towards a partcular mage. Although dsplacement mappng requres a base surface, ths base mesh s easly obtaned usng tradtonal computer vson technques e.g., shape-from-slhouette or structure-from-moton. Our method explots shadng varaton due to object rotaton relatve to the lght source, allowng the recovery of dsplacements n both textured and textureless regons n a common framework. In partcular, shadng cues are ntegrated nto a mult-vew stereo photoconsstency functon through the surface normals that are mpled by the dsplacement map. The analytc gradent of ths photo-consstency functon s used to drve a multresoluton conjugate gradent optmzaton. We demonstrate the geometrc qualty of the reconstructed dsplacements on several example objects ncludng a human face. 1. Introducton The automatc computaton of 3D geometrc and appearance models from mages s one of the most challengng and fundamental problems n computer vson. Whle a more tradtonal pont-based method provdes accurate results for camera geometry, a surface representaton s requred for modelng and vsualzaton applcatons. One of the most popular classes of surface-based reconstructon methods are mult-vew stereo approaches that represent the surface as a depth or dsparty map wth respect to one reference mage [17] or multple mages [11]. One of the man dsadvantage of those methods, referred to as mage-centered approaches, s that the reconstructon wll be based by the chosen reference mage. As a consequence, the results are often alased due to the lmted depth resoluton and they do not reconstruct parts of the object that were occluded n the reference mage. An object-centered model s therefore more sutable for mult-vew reconstructon. Examples of Input Images & Calbraton Base Mesh Dsplacement Albedo Fgure 1: An overvew of our method, whch recovers the surface dsplacement and albedo gven a set of nput mages, a base mesh, and calbraton data. object-centered surface representatons nclude voxels [18], level-sets [8], and meshes [10, 7]. We propose a surface reconstructon method that nvestgates a less explored object-centered representaton - a depth feld regstered wth a base mesh see Fg. 1. Our model s nspred by computer graphcs dsplacement maps [5] that nowadays have effcent HW mplementatons e.g., [15]. Compared to a deformable mesh-based representaton, our method s more stable as dspartes are constraned to move along the normal drectons w.r.t. the base mesh. For a purely mesh-based representaton, the vertexes are allowed to move freely, so the dsplacement drecton can become unstable. The method can also be regarded as an extenson of the stereo-based methods, where depth s calculated wth respect to an object-centered base mesh as opposed to the mage plane. Therefore reconstructon methods used for stereo can be easly generalzed to our representaton. One can, for example, make use of the effcent dscrete methods lke graph cuts or belef propagaton that cannot be used wth some other object-centered representatons e.g., mesh, level-sets. For our case, as the cost functon s based on shadng and uses the surface normals, we chose a contnuous representaton where the normals are mpled by the surface. We formulate the surface reconstructon of Lambertan scenes as an optmzaton of a photo-consstency functon that ntegrates stereo cues for textured regons wth shape 1

2 from shadng cues for texture-less regons. The cost functon s calculated over a dscretzaton of the base mesh. Even though the method assumes an a-pror base mesh, ths can be easly obtaned n practce usng shape-fromslhouette or trangulaton of structure-from-moton ponts. Due to the hgh dmensonalty, reconstructon can be dffcult and slow, whle requrng a substantal amount of mage data. To amelorate these problems, we propose a multresoluton algorthm. There exst other approaches that combne stereo for textured regons wth shape from shadng cues for texture-less regons [10, 14], but, n those works, the two scores are separate terms n the cost functon and the combnaton s acheved ether usng weghts [10] or by parttonng the surface nto regons [14]. Lke photometrc stereo, our method s able to reconstruct the surface of spatally varyng or unform materal objects by assumng that the object s movng relatve to the lght source. A smlar constrant was used by Zhang et al. [25] and Weber et al. [22] but wth a dfferent surface representaton. To summarze, the man contrbutons of the paper are: We propose a method that reconstructs surfaces dscretzed as a depth feld wth respect to a base mesh dsplacement map; the representaton s sutable for both closed and open surfaces and, unlke tradtonal stereo, reconstructs whole objects; We desgned a photo-consstency functon sutable for surfaces wth textured and unform Lambertan reflectance by ntegratng shadng cues on mpled surface normals; We desgned a practcal setup that provdes the necessary lght varaton, camera and lght calbraton and requres only commonly avalable hardware: a lght source, a camera, and a glossy whte sphere. 2 Related Work To our knowledge, the work of Vogatzs et al. [21] s the only source that deals wth reconstructon of depth from a base mesh. Most prevous approaches reconstruct depth from stereo and then ft planes to the resultng 3D ponts [6]. In contrast, Vogatzs et al. [21] estmate the dsplacement for sample ponts on the base mesh usng a belef propagaton technque. Ther method assumes there s no llumnaton varaton n the mages the scene s fxed w.r.t. llumnaton and the cost functon s just a varance of the colors that a sample pont project to. The method s therefore smlar to mult-vew stereo reconstructon, requrng scenes wth good texture. A related representaton was proposed by Zeng et al. [23] but, n ther case, the base geometry s a collecton of planes ftted to a set of 3D ponts gven a-pror that do not necessarly form a mesh. The depth of the surface patch along the plane normal s reconstructed usng a globally-optmal graph cut optmzaton that ensures that the patch surface goes through any neghborng 3D ponts. A smlar surface patch approach was used to determne voxel consstency n a more recent work by Zheng and hs colleagues [24]. Other approaches use parametrc models for local depth representaton e.g., planar dsks [9], quadratc patches [13]. The patches are then ntegrated and nterpolated to form the fnal surface. The mesh-based methods share some smlartes to the chosen representaton, but they operate by teratvely evolvng and refnng an ntal mesh untl t fts the set of mages [10, 7, 12, 2]. In these approaches large and complex meshes have to be mantaned n order to represent fne detal. In contrast we assume a fxed base mesh and compute surface detal as a heght feld wth respect to t. The chosen representaton s an extenson of the tradtonal dsparty map where the depth s regstered wth respect to a base geometry nstead of the reference mage plane. Thus t has smlartes to the mult-vew stereo reconstructon technques. Lke n the global stereo methods e.g., [20, 11] we formulated the problem as mnmzng a global energy functon that takes nto account both matchng and smoothness costs. But, unlke stereo methods, n our case the dscretzaton and regularzaton s vewpont ndependent. Addtonally, our approach s able to reconstruct complete scenes as every pont that s vsble n at least one mage wll be reconstructed n the case of stereo ponts that are occluded n the reference mage are not reconstructed. 3 Problem Defnton and Formulaton Base Mesh True Surface sample ponts dsplaced sample pont ˆp = p + µ d sample pont p Fgure 2: An overvew of the representaton and notaton used n ths paper. We assume that we are gven a set of n mages, I, the correspondng calbraton matrces, P, the correspondng llumnaton parameters, L, and a base mesh consstng of a set of vertces V = {v R 3 } and a trangulaton of these vertces T = {v 1, v 2, v 3 v V }. The shape reconstructon problem s then cast as an optmzaton problem, 2

3 that recovers a set of dsplacements µ j along a drecton d j from a dscrete set of m sample ponts, p j, taken on the surface of each trangle. A pont on the dsplaced surface wll be denoted as ˆp j = p j + µ j d j, whch s a functon of the current estmate of the dsplacement µ j see Fg. 2. The dsplacements, µ j, are then related to the mage measurements through a photo-consstency functon. The photo-consstency functon measures the smlarty between the reflectance of a pont on the surface and the mages n whch t s observed. The goal s then to fnd a set of dsplacements, µ j, and the reflectance parameters, α j, of the sample ponts that mnmze the value of the followng cost functon: F data = j = j 1 f µ j, α j V j V j 1 1 R ˆp j, α j I ΠP ˆp j 2 V j V j 2 In the above cost functon, V j denotes the set of mages that observe dsplaced pont ˆp j, and R denotes the renderng functon that produces a color value for the dsplaced pont observed under the llumnaton and vewng parameters of camera. In other words, the cost functon expresses the dfference between the rendered surface representaton and the nput mages. We used a smlar cost functon n our mesh-based mplementaton [2] although n ths work we currently take no precautonary measures to flter out specular hghlghts or address mage samplng ssues. Agan, nstead of usng the entre set of cameras that observe a sample pont, a subset of the vsble cameras closest to the medan camera are used. Ths adjustment partally accounts for mage samplng problems e.g., cameras that vew the surface at a grazng vew. We assume that the nput mages and llumnaton nformaton s suffcent for determnng the reflectance parameters α j. That s, f we fx the surface dsplacements, there exsts a closed form soluton for reflectance parameters that mnmze Eq. 4. If ths s the case, then ths mnmzaton process can be done for each α j ndependent of the other ones lke n photometrc stereo. In our specfc work, we assume that the surface s Lambertan, mplyng that the surface reflectance parameters at a pont are smply the albedo e.g., α j s a RGB color. Furthermore, we assume that the llumnaton for each frame s represented as a drectonal lght source plus an ambent term 1. We use l to denote the lght drecton, l to denote the lght color, and a to denote the ambent color. Under these condtons, the renderng functon for mage s smply the Lambertan shadng model for the case of a sngle lght source: 1 Actually, we model the lght as a pont lght source, mplyng that the drecton to the lght source s dependent on the 3D poston of the pont. n j R = α j l l + a where n j s the surface normal at dsplaced pont j. We use the mpled surface normal, obtaned as a functon of the sample ponts that are wthn a neghborhood of sample pont j, denoted N j. Unlke tradtonal bnocular stereo, the use of the mplct surface normal n the above cost functon acts a regularzer, but the mnmzaton s stll senstve to nose. As s the case n other mult-vew stereo reconstructons, addtonal regularzaton s necessary. In ths work, we take the approach often used n bnocular methods, where the regularzaton s added as a separate term n the mnmzaton, gvng 3 F = F data + λf smooth 4 Currently, we use a smple quadratc regularzer that ensures that neghborng sample ponts, have smlar dsplacements: F smooth = µ j µ k 2 5 j m k N j where N j s a neghborhood around sample pont j. Another potental smoothness term would be to use the Eucldean dstance between neghborng ponts, as was done by Vogatzs et al. [21]. An alternatve approach s to keep the regularzaton mplct multplcatve regularzer by consderng the weghted mnmal surface lke n the level set approaches e.g., Faugeras and Kerven [8]. However, researchers e.g., Soatto et al. [19] made the observaton that the mplct level set approach mght suffer from over-smoothness due to hgh order dervatves nvolved nto unknowns and also could change the mage varablty dependng on the surface. Therefore, for our system, we used an addtve regularzer, gvng a more precse and easly nterpretable effect. 4 Samplng & Optmzaton We now detal the procedures requred n mnmzng the cost functon presented n Secton 3. Frst we dscuss how we dscretze the base mesh nto sample ponts, and defne a specfc neghborhood functon for the sample ponts. Wth these detals n place we then present a closed form soluton for the reflectance parameters for a set of surface ponts, followed by detals of the optmzaton procedure. 4.1 Samplng Clearly, t s desrable to have a set of sample ponts that are evenly dstrbuted on the base mesh. The resoluton of the mesh should be chosen so that the dstance between sam- 3

4 ple ponts roughly corresponds to one pxel n mage space. One smple soluton would be to choose the sample ponts on a regular grd wthn each base trangle. However, for vsblty and neghborhood purposes t s useful to have a trangulaton of the base ponts. Unfortunately, obtanng a water tght trangulaton of these regularly spaced ponts may become complcated. Moreover, the samplng rate may be dfferent on the edges of the base mesh. To allevate these samplng ssues we nstead use a subdvson approach. In ths approach, startng from the base mesh and gven a desred sample spacng, any edges n the mesh that are longer than twce the desred sample spacng are splt n two. Splttng an edge turns the two trangles that contan the volatng edge nto four smaller trangles. Ths operaton s performed untl all edges n the mesh are less than twce the sample spacng e.g., splttng an edge would produce two sample ponts that closer than the desred sample spacng. Smply usng ths approach may produce sample ponts wth an rregular valence.e., an rregular number of neghbors and may create trangles wth poor aspect ratos. To crcumvent these problems, we use the topologcal operators of Lachaud and Montanvert [16]. These operators ensure that edge lengths are wthn a certan range and that non-neghborng vertces are suffcently spaced. The vertces of the subdvded mesh are the sample ponts p j, and the neghborhood functon N j s defned as those sample ponts that share an edge n the subdvded mesh. Ths approach ensures that the neghborhood functon extends across base trangles. Fnally, we take the dsplacement drecton d j of a sample pont to be the correspondng nterpolated surface normal at the poston p j on the base mesh. As our current mplementaton does not account for self-ntersecton durng the refnement, usng the nterpolated normal opposed to the base plane normal reduces the occurance of these self-ntersectons. 4.2 Reflectance Parameters Recall that we requred a closed form soluton for the reflectance parameters for any partcular choce of dsplacements. The Lambertan shadng functon n Eq. 3 s dependent on the mpled surface normal, n j, for surface pont j. In ths work, we compute the mpled surface normal, n j, as an area weghted average of the trangle normals of the sample pont trangulaton, whch s gven below n unnormalzed form: n j = ˆp k ˆp j ˆp ˆp j 6 j,k, p j where p j denotes the trangles that contan p j. Notce that the mpled surface normal s a functon of the dsplaced surface ponts. Usng the above defnton of a surface normal for a partcular nstantaton of surface dsplacements, we can compute a closed form soluton for the α j. For each partcular surface pont, we collect all mage observatons of the pont nto a system of equatons: n l j l 1 1 n j + a 1 n l j l 2 2 n j + a 2.. l m n j l m n j + a m α j = I 1 ΠP 1 ˆp j I 2 ΠP 2 ˆp j. I n ΠP n ˆp j 7 A j α j = I j 8 As both the left and rght hand sdes are n 1 vectors, the least squares soluton for the albedo s easly obtaned as: α j = A j I j A j A j 4.3 Optmzaton = A j I j A j A j 9 One possble method for optmzaton would be to dscretze the dsplacement values nto a set of dscrete labels and then use a combnatoral optmzaton technque to refne the labels e.g., smlar to the method of Vogatzs et al. [21]. As the surface normals rely on a current estmate of the neghborng dsplacements, ths approach can not drectly mnmze our cost functon see Secton 4.4.1, for more detals about a drect dscretzaton of the cost functon. Instead, we perform a contnuous optmzaton of the dsplacements. As both the memory and computatonal cost for computng a Hessan of the objectve functon s prohbtve, we chose the conjugate gradent method. We use the analytc dervatves of the cost functon n the optmzaton. These dervatves are summarzed below. For convenence we defne the ndvdual terms of the cost metrc for a gven sample pont n a gven vew: c j = 2 n j l α j l + a I ΠP ˆp j The value of the objectve functon for sample pont j n mage e.g., c j s dependent only on ts dsplacement, µ j, and the dsplacements of the neghbors N j p j. Therefore, c j = 0 f k j&k / N j p j 10 c j n j l = 2 α j l + a I ΠP ˆp j n j l αj n j l l + a + α j l I P ˆp j The dervatve of the least squares computaton for the 4

5 α j albedo,, s straghtforward and can be expressed n terms of. These dervatons and those for the nj l n j I ΠP ˆp j term can be found n Appendx A. 4.4 Mult-Resoluton The local nature of the conjugate gradent optmzaton s senstve to the startng poston and s lkely to fall nto a local mnma f ths startng poston s far from the global mnmum. Followng other mage-based approaches [10], we use a mult-resoluton to lessen the dependence on a good startng poston. We start the optmzaton on downsampled mages and a correspondng down-sampled mesh. After convergence the mesh resoluton s ncreased and the conjugate descent optmzaton s run agan. After ths step converges the mage resoluton s ncreased. These steps are terated untl the true resoluton for the nput mages has been used. We use OpenGL depth bufferng to compute vsblty of the sample ponts. Ths operaton can be expensve, so the vsblty of sample ponts s only updated every 100 teratons or when the resoluton changes Intalzaton To further avod local mnma and because our method requres a good ntalzaton for convergence we provde an approxmate ntalzaton at the lowest resoluton. Specfcally, we use a samplng technque that s smlar to the other dsplacement recovery methods [21]. That s, the cost functon s evaluated at a dscrete set of dsplacements for each sample pont ndependently. There s no noton of a current surface n ths approach, so we must approxmate some measurements. Frst, we approxmate vsblty durng ths stage by usng the vsblty of the base mesh. Furthermore, each dscrete sample of the cost functon for each sample pont requres a surface normal. As there s no mpled surface at ths stage, the best we can do s assume that the normal can be arbtrary. Therefore, for each sample pont and each dscrete depth label we must ft a surface normal that reduces the F data cost. The resdual of ths fttng s the approxmate cost functon. The optmzaton problem s now dscrete, wth the goal of recoverng a set of depth labels that reduce the approxmated cost functon. Ths sampled cost functon s easly ncorporated wth the smoothness term and optmzed usng exstng combnatoral optmzaton technques, such as graphcuts or loopy belef propagaton. In our case, we adapted the expanson graph-cut proposed by Boykov et al. [3] for the dsplacement map representaton the orgnal algorthm estmates a dsparty map. To summarze ths secton, our algorthm takes a low resoluton base mesh along wth the mages and calbraton data as nput. The downsampled mages are created, and the correspondng low resoluton sample ponts are obtaned Fgure 3: An mage of the capture setup used n the experments. The lght s placed nearby the camera. We used a calbraton pattern to calbrate the camera and a whte specular sphere to calbrate the lght poston. usng the subdvson method that was outlned n Secton 4.1. The sample dsplacements on the low resoluton mesh are then ntalzed wth the method outlned above Secton Fnally, the mult-resoluton optmzaton s performed. The algorthm outputs the recovered dsplacements and the albedo at each sample pont on the base mesh. 5 System We now dscuss the practcal elements requred n provdng the nput to our system. Recall that we need a base mesh, mage calbraton matrces, and a lght drecton for each frame. Furthermore, the lght drectons need to vary suffcently durng the capture to ensure that the reflectance parameters can be ft to each dsplaced surface pont. To provde these requrements we use a turntable based setup, whch s llumnated by a regular desk lamp located by the camera see Fg. 3. Wthn ths setup we use a planar dot-based pattern for the calbraton [1]. A blue-screenng approach s used to obtan slhouettes that are then used as nput to shape-from-slhouette to provde an ntal shape. Other computer vson technques for sparse structure could be substtuted for shape-from-slhouette. As shape-fromslhouette may gve a farly dense mesh, we then use the smplfcaton method of Cohen-Stener et al. to obtan a low polygon base mesh [4]. Notce that the camera s fxed relatve to the lght postons, mplyng that the lght s actually movng wth respect to the turntable coordnate system. To avod shadows the lght source s postoned near the camera, and we capture two full rotatons at dfferent heghts to ensure that there s suffcent lght varaton. A glossy whte sphere rotates on the turntable and s used to calbrate the the exact poston of the lght source.e., our desk lamp cannot be postoned ex- 5

6 actly at the camera center as well as to estmate the color of the source. Several spheres are not necessary as the rotaton of the sphere gves several temporal vews. We trangulate the poston of the lght source usng the specular hghlght on the sphere. From ths pont lght source we can obtan the drecton to the source for any 3D pont n the scene to use n the renderng functon. Assumng that the sphere s whte, after we know the poston of the sphere and the poston of the lght source, each non-specular pxel observaton of the sphere gves an equaton n the 2 unknowns ambent lght and lght source color for each color channel. 6 Experments In a frst experment we demonstrate that the method s accurate on synthetc objects wth varyng degrees of texture Fgure 4. We have smulated the capture setup usng a synthetc object texture mapped wth two dfferent textures. In each case the concavtes of the object were accurately recovered. We have also computed the dstance from the recovered dsplacements to the ground truth dsplacements we used ray tracng to recover the ground truth dsplacements from a trangulated mesh. A color mapped verson of ths dstance on the ground truth s also portrayed n Fgure 4. The reconstructon s qute good wth an average geometrc error s about unts on an object of sze 2 unts 1.6% accuracy. The results of the refnement on two real objects are shown n Fgure 5 2. Notce that n each case the base mesh and the ntal dsplacements usng the method descrbed n Secton are coarse approxmatons of the true object. After the dsplacements are refned, the fne scale detal of the 3D geometry becomes apparent. Notce the method performs well on the face sequence, whch would be consdered hard to capture wth tradtonal methods. The face also exhbts a surface reflectance that devates from the Lambertan model, but our Lambertan cost functon s successful at recoverng vsually appealng dsplacements. 7 Conclusons We have presented a method for reconstructng the dsplacements from a base mesh usng mage data. The method couples the surface normals n the cost functon, mplyng that surface shadng cues nfluence the reconstructon. The experments valdated the effectveness of ths approach on objects wth unform and non-unform Lambertan surface propertes. Expermental evdence also suggests that our method s capable of dealng wth slght reflectance devatons from the Lambertan model e.g., the face example. 2 The human head data set was obtaned by placng the calbraton pattern upsde down on the subject s head whle the subject rotated on an offce char. Currently, one of the man lmtatons of ths approach s the nfluence that the base mesh has on the fnal results. For example, the recovered mesh s restrcted to have the same topology as the base mesh. Also, t s possble that the true surface cannot be approxmated by dsplacement mappng the base mesh. In future work, we would lke to address these problems by refnng the base mesh durng the optmzaton. A Gradent Terms I ΠP ˆp j = 0 for k j 11 P = p 00 p 10 p 01 p 11 p 02 p 12 p 03 p 13 p 20 p 21 p 22 p I ΠP ˆp j = I ΠP ˆp j ΠP ˆp j [ ] p = I 00 d j,x + p 01d j,y + p 02d j,z /w p 10 d j,x + p 11 d j,y + p 12 d j,z/w [ up I 20 d j,x + p 21 d j,y + p 22 d ] j,z/w 2 vp 20d j,x + p 21d j,y + p 22d j,z /w 2 wth [u, v, w] = P ˆp j. In practce a Gaussan blurred verson of the mage gradent s substtuted for I. nj l = 1 n j l 1 + n j l nj n j Recall that: n j = = µ k n j l j,k, p j n j l 3 µ k n j n j ˆp k ˆp j ˆp ˆp j Lettng e 1 = ˆp k ˆp j = x k x j, y k y j, z k z j and e 2 = ˆp ˆp j = x x j, y y j, z z j, e 1 e 2 = e 1ye 2z e 2y e 1z e 1x e 2z + e 2x e 1z = e 1x e 2y e 2x e 1y y k z y k z j y j z y z k + y z j + y j z k x k z x k z j x j z x z k + x z j + x j z k x k y x k y j x j y x y k + x y j + x j y k Therefore µj n j = j,k, p j y y k d j,z + z k z d j,y z z k d j,x + x k x d j,z x x k d j,y + y k y d j,x 6

7 Fgure 4: A synthetc object wth dfferent textures used n the experments. From left to rght: an nput mage, the recovered shaded model, the recovered textured model, and dstance to ground truth dark means close, whte means far. The average dstance of the recovered dsplacement to the ground truth was and for the untextured and textured sequences respectvely the sze of the whole object s around 2 unts. The dervaton of µk n j and µ n j are smlar. Fnally, usng the defnton of A j,i j and α j from Equatons 8 and 9, we have α j = 1 A j A j A j I j A j A j 2 A j I j A j A j A j I j = = I ΠP ˆp j l A j A j = = nj l n j l + l n j l l + a I ΠP ˆp j n j l l n j l 2l l + a References + a I ΠP ˆp j nj l + a 2 [1] Adam Baumberg, Alex Lyons, and Rchard Taylor. 3D S.O.M. - a commercal software soluton to 3d scannng. In Vson, Vdeo, and Graphcs VVG 03, pages 41 48, July [2] Nel Brkbeck, Dana Cobzas, Peter Sturm, and Martn Jagersand. Varatonal shape and reflectance estmaton under changng lght and vewponts. In ECCV2006, [3] Yur Boykov, Olga Veksler, and Ramn Zabh. Fast approxmate energy mnmzaton va graph cuts. IEEE Trans. Pattern Anal. Mach. Intell., 2311: , [4] Davd Cohen-Stener, Perre Allez, and Matheu Desbrun. Varatonal shape approxmaton. ACM Trans. Graph., 233: , [5] Robert L. Cook. Shade trees. In SIGGRAPH 84: Proceedngs of the 11th annual conference on Computer graphcs and nteractve technques, pages , New York, NY, USA, ACM Press. [6] Paul E. Debevec, Camllo J. Taylor, and Jtendra Malk. Modelng and renderng archtecture from photographs. In SIGGRAPH, [7] Y. Duan, L. Yang, H. Qn, and D. Samaras. Shape reconstructon from 3d and 2d data usng pde-based deformable surfaces. In ECCV, [8] O. Faugeras and R. Kerven. Varatonal prncples, surface evoluton, pde s, level set methods and the stereo problem. IEEE Trans. Image Processng, 73: , [9] P. Fua. Reconstructng complex surfaces from multple stereo vews. In ICCV, pages , [10] P. Fua and Y. Leclerc. Object-centered surface reconstructon: combnng mult-mage stereo shadng. In Image Understandng Workshop, pages ,

8 Fgure 5: The results of the dsplacement map estmaton for a model house made from wood and bark and a human face. From left to rght, an nput mage, the base mesh, the low-resoluton startng pont usng dscrete optmzaton, two shaded vews of the object, and a textured lt renderng of the recovered object. [11] Pau Gargallo and Peter Sturm. Bayesan 3d modelng from mages usng multple depth maps. In CVPR, pages , [12] C. Hernández and F. Schmtt. Slhouette and stereo fuson for 3d object modelng. Computer Vson and Image Understandng, specal ssue on Model-based and magebased 3D Scene Representaton for Interactve Vsualzaton, 963: , December [13] Wllam A. Hoff and Narendra Ahuja. Surfaces from stereo: Integratng feature matchng, dsparty estmaton, and contour detecton. IEEE Trans. Pattern Anal. Mach. Intell., 112: , [14] H. Jn, A. Yezz, and S. Soatto. Stereoscopc shadng: Integratng shape cues n a varatonal framework. In CVPR, pages , [15] Jan Kautz and Hans-Peter Sedel. Hardware accelerated dsplacement mappng for mage based renderng. In Proccedngs of Graphcs nterface, pages 61 70, [16] J.-O. Lachaud and A. Montanvert. Deformable meshes wth autom. topology changes for coarse-to-fne 3D surface extracton. Medcal Im. Anal., 32: , [17] Danel Scharsten and Rchard Szelsk. A taxonomy and evaluaton of dense two-frame stereo correspondence algorthms. Int. J. Comput. Vson, 471-3:7 42, [18] Gregory G. Slabaugh, W. Bruce Culbertson, Thomas Malzbender, and Ronald W. Schafer. A survey of methods for volumetrc scene reconstructon from photographs. In Internatonal Workshop on Volume Graphcs, [19] S. Soatto, A. Yezz, and H. Jn. Tales of shape and radance n mult-vew stereo. In ICCV, [20] C. Strecha, T. Tuytelaars, and L.J. Van Gool. Dense matchng of multple wde-baselne vews. In ICCV, pages , [21] George Vogatzs, Phlp Torr, Steve Setz, and Roberto Cpolla. Reconstructng relef surfaces. In BMVC, [22] Martn Weber, Andre Blake, and Roberto Cpolla. Towards a complete dense geometrc and photometrc reconstructon under varyng pose and llumnaton. In BMVC, [23] G. Zeng, S. Pars, L. Quan, and M. Lhuller. Surface reconstructon by propagatng 3d stereo data n multple 2d mages. In ECCV, [24] Gang Zeng, Sylvan Pars, Long Quan, and Francos Sllon. Progressve surface reconstructon from mages usng a local pror. In ICCV, pages , [25] L. Zhang, B. Curless, A. Hertzmann, and S. M. Setz. Shape and moton under varyng llumnaton: Unfyng structure from moton, photometrc stereo, and mult-vew stereo. In ICCV,