Lighting Analysis and Texture Modification of 3D Human Face Scans

Transcription

1 Digitl Imge Computing Techniques n Applictions Lighting Anlysis n Texture Moifiction of 3D Humn Fce Scns Xiozheng Zhng, Snqing Zho n Yongsheng Go Computer Vision n Imge Processing Lbortory, Griffith School of Engineering, Griffith University, 170 Kessels Ro, Nthn Ql 4111, Austrli {x.zhng, s.zho, yongsheng.go}@griffith.eu.u Abstrct The 3D scnning of humn fces is n importnt tool to cquire ccurte shpe n texture informtion of hes n fces for both nimtion n recognition purposes. One of the most populr scnning tools is Cyberwre he n fce 3D colour scnner. Currently, 3D fce tbses scnne using the Cyberwre he n fce 3D colour scnner tbses re publicly vilble n hve been use in mny fce recognition system s the trining t, one of which is USF humn ID 3-D tbse. The mechnism of fcil texture cpturing, however, remins uncler ue to the specil settings of the light sources uring scnning. The irect ppliction of the cquire texture intensity s the lbeo of the fce surfces les to suboptiml moelling results. This pper nlyses the lighting configurtion of the scnner n proposes n pproprite reflectnce moel to escribe the reflecting mechnism when scnning. The texture moifictions cn be crrie on in the existing scnning results without ny further scns. Experiment results show tht the propose reflectnce moel cn better pproximte the reflecting sitution of scnning so tht the cquire textures cn be irectly use s iffuse reflectnce coefficients n chieve better renering results. Key wors: 3D fce moel, 3D scnner, fcil textures, lighting conition, reflection moel. 1. Introuction The 3D scnner (e.g., Cyberwre he n fce 3D colour scnner [9]) provies ccurte 3D shpe informtion of humn fces. While cpturing shpe informtion by lser bem, the he of the subject is lso illuminte by movble light sources instlle on the scnner s he for the cmer to cpture texture mp of the fce surfce. The synchronistion of shpe n texture cquisition is criticl so tht the point-wise corresponence coul be estblishe uring scnning. For the Cyberwre he n fce 3D colour scnner, white light sources re then instlle on the scnning he ner the 3D cmer. During scnning, the region being scnne for shpe informtion is immeitely illuminte by the lmp n cpture for texture informtion. In this wy, both shpe n texture informtion of the sme points on the fce surfce is mesure t lmost the sme time to prevent the movement of the subject n the consequent mislignment. Then, the shpe n texture informtion cn be use in vrious pplictions such s re-renering uner new illumintion n viewing conitions. In computer vision n pttern recognition, they re lso use s iscriminting fetures for humn fce recognisers to perform fce recognition. Since the light source in the scnner is movble, the lighting conition for ifferent points on the surfce my be ifferent. Consequently, the irect texture-mpping of the originl cpture imge intensity to the object shpe cnnot synthesise correct D imges uner orinry fixe light source(s). In computer vision, they re not correctly extrcte from the subject to reflect genuine intrinsic fetures for clssifictions. This pper proposes new lighting moel to ress this texture justment n reestimtion problem. Uner Lmbertin reflectnce moel, the cpture texture mp cn be juste using the propose texture moifiction scheme to resemble Lmbertin iffuse reflectnce coefficient mp (i.e., biirectionl reflectnce istribution function, BRDF) /07 $ IEEE DOI /DICTA

2 . The reflectnce moel for the 3D scnning This pper is motivte by the use of lrge tset of fce scns in the synthesis of 3D fces [] n fce recognition cross vrition in pose [3]. In these stuies, 00 3D fce scns were use s prior knowlege of humn fces for builing 3D morphble fce moel bse on principle component nlysis n for estblishing set of intrinsic prmeters for recognition. Both of the geometric n texturl informtion re cquire using the Cyberwre scnner. The scnne texture informtion ws irectly store in the tset s the trining t for the principle component nlysis. Then the principle components of fcil textures were extrcte n use in Eqution 6 of [] s the iffuse reflectnce coefficients of Phong moel [7]. Vetter n Blnz [8] expline the use of scnne texture s iffuse reflectnce coefficient tht the movble light source cn be pproximte s bckgroun lights uniformly istribute in the environment n the iffuse reflectnce coefficient n the mbient reflectnce coefficient were interchngeble in Lmbertin moel. The substitution of mbient reflectnce coefficient by iffuse reflectnce coefficient is vli uner convexity ssumption. The pproximtion of the movble light source of the Cyberwre scnner to be mbient light source, however, is fr from relity, since the light source is slim lmps which is incpble to provie lrge re light n hence cnnot be pproximte s uniformly istribute bckgroun light. This pper crefully exmines the lighting conitions of the scnning process of humn fces for the Cyberwre scnner n proposes novel n specific lighting moel to resemble the scnning conition. Using this lighting moel, the textures of scnne humn fces cn be moifie n then use s iffuse reflectnce coefficients s ws one in [] n [3]. The propose texture moifiction scheme consiers the reltionship of incient light sources n the surfce geometries uring scnning n then is more pproprite thn the rough pproximtion of scnner s lights s mbient light..1. Reflectnce moels The reflecte intensity (i.e., the imge intensity) in Phong reflectnce moel is expresse s υ I = ( I + I ( N L )) k + I k ( R V ), (1) r s where I n I re the mbient n irectionl light intensities respectively, N is the surfce norml irection, L is the irection of the irectionl light I, k n k s re the iffuse n speculr reflectnce coefficients respectively, R is the reflecting irection which cn be expresse s R = ( N L ) L, () n V is the viewing irection pointing from the surfce point to the cmer, n υ is the speculr reflectnce exponent. If the surfce is ull mtte, it oesn t hve speculr reflection of the incient lights. The reflection cn be pproximte well by Lmbertin moel s I r = I ( N L ) k. (3) In the bove eqution, mbient light n reflection is lso neglecte. The mbient light is often pproximte s uniformly istribute incient re light from the environment n the reflection of mbient light (i.e., mbient reflection) cn be moelle using n mbient reflectnce coefficient k s I r, mb = Ik. (4) Uner Lmbertin moel, k cn be substitute by for convex objects. Although Lmbertin moel is not perfect moel for humn skin, it is wiely use ue to the simplistic form of expressions in computer grphics n computer vision incluing fce nimtion n recognition such s in [1, 5, 6]. In this reserch, humn fces re ssume to be Lmbertin surfces s ws one in [] n [3]. The moelling of humn skins s non-lmbertin surfces will be n interesting problem, since humn fces o reflect speculrly. The extrction of speculr reflectnce coefficients my require itionl lighting n cquisition equipments uring the scnning process n my not be incorporte into the existing tsets.. Anlysis of the lighting conitions Cyberwre He & Fce Colour 3D Scnner is wiely use for cquisition of the shpe n texture informtion of 3D humn fces. While scnning, the scnner projects low-intensity lser bems to the fce surfce. Two cmers cptures the lser profile in two ifferent viewpoints n the shpe informtion cn then be estimte using stereo vision. At the sme time, k 403

3 lmps instlle in the scnner s he provie light sources for texture cquisition (see the two lmps on the moving scnner he in Figure 1.) Figure. While scnning the surfce point P, the scnner cmer is locte in the line connecting the centre point n P. The lmps re locte in the right n left sies of the scnner s he symmetriclly n let α enote the ngle between the incient light n the centre line (note this irection is not necessrily the norml irection). As the heights on the humn fce re vrible ue to ifferent fcil components (e.g., the nose tip is higher thn the lips), α is not constnt. Provie the scnne surfce shpe n the configurtion of the scnner, it is possible to clculte exct α vlues for every point. In this pper, however, we ssume constnt α vlue to simplify the texture nlysis since the vritions of fce shpe from cyliner re reltively smll compre to the istnce of the two lmps to the he. Fig. 1. The Cyberwre He & Fce Colour 3D Scnner [9] n the scnning process. Two of the lmps re instlle on the scnner s he to provie light sources for texture cquisition. Since the light source is movble, it genertes specil lighting conition which is ifferent to the existing lighting conition with fixe light sources. In [, 3], this light source is pproximte s mbient light, which generlise the two slim lights s the sptilly uniform istribute re light within the whole hemisphere. Then the iffuse reflectnce coefficient (lbeo) k which is equivlent to mbient reflectnce coefficient k cn be estimte using Eqution 3. This rough ssumption, however, results in suboptiml cquire fcil texture evient tht the regions with fine structures re reltively rker thn the smoother region. To correct this texture cquisition, we propose better lighting moel for the he n fce scnner, which is to improve the texture cquisition. In this pper, the incient light is moelle s movble light sources from infinity s shown in Fig.. The humn fce scnning using Cyberwre He & Fce Colour 3D Scnner. Uner Lmbertin ssumption, then, the cquire texture intensity cn be expresse s Ir = I [ ( N L1 ) + ( N L )] k, (5) where L 1 n L re the irections from the two lmps respectively. Suppose the ngles between the two lmps n the norml irection N re γ 1 n γ, respectively. We hve cosγ 1 = ( N L 1), (6) n 404

4 cosγ = ( N L ). (7) Surfce norml N is clculte using B-spline surfce pproximtion on 16 neighbour points in fcil re s N = Q( x, y) Q( x, y), (8) x y where Qxy (, ) is the B-spline surfce mtrix [4]. Suppose the zimuth n zenith ngles between surfce norml N n centre line re θ n φ respectively, the ngles γ 1 n γ cn be expresse s cosφ cos( α θ ) cosγ 1 =, (9) cos( α θ ) n cosφ cos( α + θ ) cosγ =. (10) cos( α + θ ) Therefore, Eqution 5 becomes cos ( α θ ) cos ( α + θ ) Ir = I cosφ + k.(11) cos( α θ ) cos( α + θ ) Then the cquire texture shoul be moifie by iviing by fctor cos( α θ) cos( α + θ ) cosφ + (1) cos( α θ ) cos( α + θ) to better resemble iffuse reflectnce coefficients. The verge texture cn lso be moifie in the sme wy since it is liner principle component nlysis. The results re shown in Figure 5 n 6. Fig. 3. The shpe n texture of person in the USF humn ID 3-D tbse () before texture moifiction, n (b) fter the propose texture moifiction. The moifie texture T ' cn then be clculte s T T ' =, (13) cos( α θ) cos ( α + θ) cosφ + cos( α θ ) cos( α + θ) where T is the originl cquire texture using the scnner. 3. Texture moifictions n results In the experiment, we set α = 30. The USF humn ID 3-D tbse ws processe with the clculte shpe n norml informtion n propose texture moifiction (Eqution 13). The iniviul fcil textures were moifie using Eqution 10 n the results re shown in Figure 3 n 4. The IDs of these two fces in the tbse re n 0371, respectively. Fig. 4. The shpe n texture of person 0371 in the USF humn ID 3-D tbse () before texture moifiction, n (b) fter the propose texture moifiction. 405

5 correcte n there re lso few show effects on the surfces. The texture moifiction pproch coul be use in lter fce recognition n fce moelling process n better result coul be expecte becuse of the vliity of lighting conition pproximtion. 4. Conclusion Fig. 5. The verge fce shpe n texture in the frontl view () before texture moifiction, n (b) fter the propose texture moifiction. Fig. 6. The verge fce shpe n texture from rotte viewing ngle () before texture moifiction, n (b) fter the propose texture moifiction. From the bove texture moifiction results, the moifie texture cn be use s the iffuse reflectnce coefficient in the 3D fce moels while the originlly cquire textures re suboptiml ue to the inccurte moelling of the lighting conitions of the scnning environment. Especilly on the occlue region of the fces such s the region ner eyes, the originl textures ten to be rk ue to the lck of consiertion of the structure n the big ngle between norml irection n lighting irections. The moifie textures re then Cyberwre He n Fce Colour 3D Scnner is one of the most wiely use 3D scnners for moelling 3D fce shpes n textures in the fce nimtion n recognition reserch n ppliction community. Through creful exmintion of the lighting conition of the scnning process of the Cyberwre He n Fce Colour 3D Scnner, this pper propose novel lighting moel n texture moifiction scheme to improve the cquire texture informtion so tht it cn be use s iffuse reflectnce coefficient uner Lmbertin reflectnce moel. The existing moelling of the lighting conition s mbient light is not correct ue to the presence of two slim lmps in the scnner s he. The propose light moel consiers the positions n moving conitions of the two lmps n therefore better resembles the scnning process. Then moifiction scheme is propose bse on the new light moel uner Lmbertin reflection moel. In experiment, 3D fce moels in public vilble tbse were processe n their textures were moifie using the propose pproch. The experiment result shows tht the propose pproch improves the cquisition of textures for the scnner n the use of this pproch coul subsequently improve the performnce of fce recognition n moelling lgorithms which use the 3D fce tbse. The future work of this reserch my lie in the incorporting non- Lmbertin reflectnce moels into the texture cquisition n/or moifiction, which coul enble the estimtion of both iffuse n speculr reflectnce coefficients of the fce surfce uring scnning. 5. References [1] D. Beymer n T. Poggio, "Fce recognition from one exmple view," Proceeings of Interntionl Conference on Computer Vision, pp , Cmbrige, MA, USA, [] V. Blnz n T. Vetter, "A morphble moel for the synthesis of 3D fces," Proceeings of SIGGRAPH 99 (Computer Grphics), pp , New York, NY, USA, [3] V. Blnz n T. Vetter, "Fce recognition bse on fitting 3D morphble moel," IEEE Trnsctions on Pttern Anlysis n Mchine Intelligence, vol. 5, pp ,

6 [4] J. D. Foley, A. vn Dm, S. K. Feiner, n J. F. Hughes, Computer grphics: principles n prctice, n e. Aison- Wesley Publishing Compny, Inc., [5] A. S. Georghies, P. N. Belhumeur, n D. J. Kriegmn, "From few to mny: Illumintion cone moels for fce recognition uner vrible lighting n pose," IEEE Trnsctions on Pttern Anlysis n Mchine Intelligence, vol. 3, pp , 001. [6] R. Gross, I. Mtthews, n S. Bker, "Appernce-bse fce recognition n light-fiels," IEEE Trnsctions on Pttern Anlysis n Mchine Intelligence, vol. 6, pp , 004. [7] B. T. Phong, "Illumintion for computer generte pictures," Communictions of the ACM, vol. 18, pp , [8] T. Vetter n V. Blnz, "Estimting coloure 3D fce moels from single imges: An exmple bse pproch," Proceeings of Computer Vision - ECCV, Freiburg, Germny, [9] Cyberwre, "He & Fce Color 3D Scnner, " ccesse on June 3,