3D Novel Face Sample Modeling for Face Recognition

Transcription

1 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER D ovel Face Sample Modelng for Face Recognton Yun Ge, Yanfeng Sun, Baoca Yn, Henglang Tang Beng Key Laboratory of Multmeda and Intellgent Software Technology College of Computer Scence and Technology, BJUT, Beng, Chna Emal: Abstract In ths paper, we present a method to generate novel realstc 3D face model usng a model traned from real 3D face data. 3D face samples are an mportant data platform for model tranng, algorthm desgn. Subect to the constrant of acquston technology, szes of current 3D face databases are relatvely small and nsuffcent. The presented method s used to solve ths problem. Frst real 3D face data are collected as tranng sample and a parametrc global model s learned based on them. Then a local model s establshed based on surface qultng. We use the global model to buld a novel coarse face model. Then, we condton the local model wth the global model. Wth approprate choces of local and global models t s possble to relably generate novel realstc 3D face data that do not correspond to any ndvdual n the tranng data. Fnally we apply our model to face recognton to examne the meanng of our work. Index Terms 3D face model; surface sttchng; 3D texture sttchng; face recognton I. ITRODUCTIO Face recognton has been one of the most challengng topcs n both pattern recognton and computer vson. Because of ts non-ntrusve nature and potental to dentfy ndvduals at a dstance wthout subect cooperaton, face recognton has been ntegrated nto many applcatons. In the past several decades, most of the research works are based on the 2D ntensty or color mages. The two manly exploted face recognton approaches are geometrc feature-based approaches [] and appearancebased methods [2]. Geometrc feature-based methods extract the relatve poston and other parameters of dstnctve facal features as features for the recognton purpose. The appearance-based methods drectly manpulate on the gray level values of the facal mage pxels and employ statstcal tools to extract features for subsequent classfcaton. 2D face recognton technques usng deas from 2D mage analyss are nherently lmted by varablty n magng factors such as llumnaton and pose. It appears that 2D face recognton technques are nfluenced by varatons of pose, expressons, and llumnatons varaton. Compared wth 2D facal mage, 3D face sample can Correspondng Author: Yun Ge provde more cues for recognton such as spatal nformaton and pose parameters whch are nherent property of human faces and are robust to uncontrollable envronment, where the appearance of 2D mage can be affected largely. Usng the 3D nformaton for face recognton s promsng to overcome the dffcultes of mage-based face recognton approaches and has potental possblty to mprove the performance of recognton system. Therefore, the commonly accepted clam about face recognton s that 3D s superor to 3D. Recently, wth the development of 3D acquston technology, 3D face recognton has attracted more and more nterest and a great deal of research effort has devoted to ths topc. 3D face recognton s tryng to nterpret the 3D sensng capablty of the human percepton system. Because of the dfference between 3D face and 2D facal mage, 3D face data acqurng, 3D face representng, 3D face modelng, 3D face feature extracton and 3D face recognton need to be dscussed/researched further [3], [4], and [5]. Some earler research on curvature analyss has been proposed for face recognton, whch can characterze delcate features [6].Chua et al. [7] use Gaussan dstrbuton to extract rgd parts of facal surface for matchng. They recognze 3D face usng a local descrptor, named Pont Sgnature, combned wth votng method. Beumer et al. [8] propose two methods for 3D face recognton usng central/lateral profles to compare two nstances. Bronsten et al. [9] assume that the facal surface n the presence of expresson s an sometrc surface whch s not stretched by expressons. All 3D facal models are transformed nto a canoncal form for recognton, whch s nvarant to the sometrc deformaton. Chang et al. [0] use prncpal component analyss (PCA) on both 2D ntensty mages and 3D depth mages, and fuse 2D and 3D results to obtan the fnal performance. Ther results show that appearance based methods such as PCA can also gve a good performance for 3D face recognton. Blanz et al. [] present a method for face recognton across large changes n vewpont. It s based on a Morphable Model of 3D faces that represented face-specfc nformaton extracted from a dataset of 3D scans. For the non-frontal face recognton n 2D stll mages, the Morphable Model s used to estmate the 3D shape of novel faces from the non-frontal nput mages, and generate the frontal vews. Lee et al. [2] present a technque to generate an 20 ACADEMY PUBLISHER do:0.4304/mm

2 468 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER 20 llumnaton subspace for arbtrary 3D faces based on the statstcs of measured llumnatons under varable lghtng condtons from many subects. A blnear model based on the hgher-order sngular value decomposton s used to create a compact llumnaton subspace gven arbtrary shape parameters from a parametrc 3D face model and reconstruct a shape-specfc llumnaton subspace from a sngle photograph. The reconstructed llumnaton subspace s appled to varous face recognton systems. Wllam et al. [3] present a novel shape-from-shadng algorthm used for llumnaton nsenstve face recognton. The algorthm uses prncpal geodesc analyss to model the varaton n surface orentaton across a face and s used to recover accurate facal shape and albedo from real world mages. The recovered shape nformaton s used to generate llumnaton normalzed prototype mages on whch recognton can be performed. Zhang et al. [4] propose a novel method for face recognton under arbtrary unknown lghtng by usng sphercal harmoncs llumnaton representaton based on the result that the set of mages of a convex Lambertan obect obtaned under a wde varety of lghtng condtons could be approxmated accurately by a low dmensonal lnear subspace. The statstcal models are bult drectly n 3D spaces by combnng the sphercal harmonc llumnaton representaton and a 3D morphable model of human faces to recover bass mages from mages across both poses and llumnatons for face recognton. Accordng to the above analyss, we can easly found that all these works have a very great dependence on tranng data. However subect to constran of acquston technology, data sze n current face database are relatve small and can not satsfy the research demands. It s necessary to collect a well-dstrbuted tranng set to ensure the algorthm's accuracy and robustness. Snce the 3D cameras are not as common as 2D cameras, t s expensve to buld a publc 3D face database, whch brngs the dffculty to valdate the proposed methods n a unform platform. Compared wth great deal of efforts on algorthm desgn, very ltter attenton has been pad to the optmal use of tranng set. In ths paper we present a novel 3D modelng way for generatng new novel 3D face based on surface qultng whch can expandng the coverage of exstng data. Except that, synthess of novel faces has many applcatons ncludng creatng crmnal models, expandng data coverage. Snce our method s based surface qultng, t can generate new samples that are both realstc and novel. We demonstrate that our model can generate completely novel faces. Remander of the paper s organzed as follows: Secton 2 presents some related work. The normalzaton procedure s ntroduced n the followng secton. In secton 4, we ntroduce the detal of modelng way. Expermental results are outlned n secton 5. Conclusons are presented n secton 6. II. RELATED WORK Because of the mportant applcaton value of 3D face sample, lots of nsttutes establshed ther own 3D face database. At present, there exst some publshed 3D face databases for dfferent expermentaton purpose: robust face recognton, face modelng. The CMU's FIA [6] was bult based on mult-angle 3D geometry nformaton, ncludng 80 3D data. The 3D-RAM [7] s bult based on structured lght. It contans 20 ndvduals. The GavabDB [8] contans 427 3D face samples correspondng to 6 persons (45 male and 6 female), and there are 7 dfferent nstances for each person. Each sample s obtaned by Mnolta VI-700-dgtal converter and represented by a 3D mesh surface. There are systematc varatons over the pose and facal expresson for every person. In the UOY 3D face database [9], all of the samples correspond to 97 ndvduals. It contans 0 captures for each ndvdual ncludng dfferent poses. However, only two of these samples present face expressons (happness and frown), and one presents face occluson. The XM2VTS database [20] s a large multmodal database supported by European ACTS proects' Mult Modal Verfcaton for Tele servces and Securty applcatons. It ncludes lots of face mages, face vdeos and 3D face data correspondng to 295 persons. These samples were generated by usng an actve stereo system and were recorded n VRML format. However, ths s a commercal database. Several years ago, our team bult a large-scale Chnese 3D face database BJUT 3D face Database [2].It s the largest Chnese 3D face database n chna. The samples used n ths paper are selected from ths database. Accordng to above presentaton, we fnd that number of the samples n these 3D face databases can not satsfy the needs for model tranng. For LDA face recognton algorthm, t s a fundament onal to collect a welldstrbuted tranng set face samples for proecton matrx establshment. If the account of samples s too few, nverse of between-class scatter matrx may become sngular matrx whch means we can not establsh LDA classfer based on these samples. To solve the problem, we proposed a framework for generatng new faces based on resamplng. Resamplng s one of basc ssues n statstcs. The theoretcal foundatons of resamplng technques are presented n [22]. We can generate a subset from exstng samples accordng to ths way. At present, the most popular resamplng methods are ackknfe [23], baggng [24], and arcng [25]. The ackknfe s superor for small datasets. The basc dea of ackknfe s randomly selectng one sample for testng and the other for tranng classfer. Ths procedure s carred out n tmes for a set. Baggng generates several tranng sets from an orgnal tranng set and then trans a component classfer from each of those tranng sets. In contrast, arcng s to adaptvely resample so that the sample weghts durng the resamplng are ncreased for those who often be msclassfed. In the feld of face recognton and face detecton, Sung and Poggo [26] proposed the bootstrap to obtan more non-face examples durng tranng. Lu and Jan [27] utlzed resamplng technques to generate several sample subsets from the orgnal tranng set. Krby and Sroych [28] used the Karhunen-Loeve reconstructed method to buld face 20 ACADEMY PUBLISHER

3 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER sample. Tore [29] presented a dfferent face mage synthesze way to generate new face mages wth dfferent accessores by combnng dfferent accessores wth normal face mage. Both of them have not changed the sample's dentty nformaton. In 2004 ChenJe [30] use the way of exchange some organ s regon of the face mage to synthess new face mage, In 2007 Umar Mohammed [3] use texture qultng method to get a vrtual 2D facal sample. Inspred by these works, we propose a new way for generatng mult new samples base on exstng samples. At present, there s a large body of lterature concernng 3D face modelng whch can be categorzed nto three types based on parametrcal model, based on generc model deformaton and based on 3D morphable model. In the method of parametrcal model, people use parametrcal method to buld a 3D face model. Compare wth the former way, t s easy to mplement and has a better result. The most dffculty s feature detecton whch s also a dffcult work n computer vson feld. By far the most successful approach s morphable model [5]. Matchng the model to gven facal mages, even a sngle mage, realstc 3D face model of the person can be syntheszed automatcally by adustng combnaton parameters of prototypc faces. The method based on the morphable model s automatc and has good results. However, all these works are amed to reconstruct 3D face nformaton based on 2D nput mage. Reconstruct 3D nformaton from 2D mage s ll-problem and the result are usually unrealstc. In ths paper we want to generate novel samples whch dose not exstng n real world. Ths work s manly nspred by the strands work n mage qultng. Gven a small sample of texture, a large and naturally new texture can be generated. These methods synthesze novel textures by pastng pxel, patches or regons derved from the orgnal sample nto the new mage such that they are n local agreement. In contrast to ther methods, we buld a novel model through 3D surface qultng whch are sampled from real 3D face data n correspond poston. In order to keep the over consstence we buld a global parametrc model based on whch a coarse model can be generated. The coarse model s not good at detal. The detal nformaton s added by substtuted each patch wth patches whch are derved from real data. In order to keep consstence of these, we propose a deformaton method for surface sttchng. Based on the proposed method a novel realty 3D face data can be generated. III. 3D FACE REGULATIO To perform the modelng work, we need a set of prototypcal 3D face samples derved from exstng database. These samples employed n ths paper are derved from BJUT-3D face database [32]. Ths database s composed of over 500 3D faces rangng n age from 6 to 49 years. Half of them are male and the other s female. Each data n ths database contans 2560 vertces and trangle patches. All faces were wthout makeup, accessores, and facal har. Fgure. BJUT 3D Faces. To ensure the consstency of 3D face samples, we need to transform these samples nto a unfed coordnate system. The upward drecton s z-axs and orented drecton s y-axs. Fgure 2. Corrected face. Due to the dfferences between samples, all selected ones need to be algned. For example, the arrangement of ponts has nothng to do wth human facal feature and t s dffcult to represent them n unfed way. Face regulaton s representng face data n form of vector. The regulated face data have dense correspondence whch s establsh pont to pont correspondence between dfferent samples n terms of facal feature. After regulaton, all these samples have dense correspondng. Gven a nasal tp pont on one sample, pont on other samples can be located n terms of the dense correspondence. Actually, t s very dffcult to establsh dense correspondence on 3D data. Due to characterstc dfference between people, geometry nformaton of 3D face data between samples has a large dfference. Besdes that the texture nformaton should also be taken nto account. Snce the acqured data through cylnder scannng, we expanded the 3D face nto 2D manfold. Wth the help of the correspondence on 2D mage, we can easly establshed 3D correspondence on 3D data. Fgure 3. Face Dense Correspondence In order to facltate the operaton, we dvde face model n to 22 patches n terms of the facal organ dstrbuton. Each classc organ can be covered by fnte patches. If we want to locate an organ, we can use the patch nformaton to do the locaton work. Then the vrtual faces can be generated patch by patch. 20 ACADEMY PUBLISHER

4 470 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER 20 Fgure 4. Dvde the 3D Face nto Patches. IV. SURFACE QUILTIG In the process of mage qultng the frst step s to extract all possble patches of a gven sze from the nput texture to form a patch lbrary. The syntheszed mage wll consst of a regular grd of these lbrary patches such that each overlaps ts neghbors by a few pxels. It starts from top-left of ths grd to the bottom-rght. At each poston, a lbrary patch s chosen such that t s vsually consstent wth the patches that have prevously been placed above and to the left. The new patches can then be blended together usng a varety of technques. We adapt the mage qultng way to face model. Frst very face samples have been dvded nto 22 patches accordng former secton. These patches have dense correspond between samples. In each patch locaton, we buld ts own patch lbrares whch are used to provde patch database for sample selecton. In ths paper, the novel face s generated by randomly pcked out a patch n each locaton s database and sttch them together. To ensure the consstency between patches, we do the work by the order of selecton from top-left to bottom. For each current poston, the canddate patches should ensure vsual consstency wth extng neghbors. The frst patch s randomly chosen. Ths randomness prevents the algorthm repeatng one of the samples n the database. 3D face sample conssts of two parts nformaton: geometry and texture. So the qultng work for each patch nvolves two works: shape sttchng and texture sttchng. To convenent the operaton, we bnd the texture nformaton to the pont. The geometry nformaton and texture nformaton are attached to the pont. So we can drectly swap them. Because of the bg dfference n pose and scale between samples, we can not drectly swap the correspondng patch. In order to generate qualfed samples, we need to sew the new organ patch wth the target face sample. The sttchng work about shape and texture are performed separately. Surface Sttchng The local model of novel face generatng s to ensure the consstency between adacent patches. For the geometry part of the 3D face, the current selected patch should consstency wth exstng one. Ths purpose can be acheved through surface deformaton. The obect s to fnd an nterpolaton way whch can transform the correspondng boundary ponts between current patch and ts neghbors nto a same poston. The other ponts move Fgure 5. TPS Deformaton Result. accordng to these boundary ponts. Ths work can be mplementng through surface deformaton. Currently the most popular deformaton methods can be dvded nto two types: the global nterpolaton way and gradent doman based technques whch operate drectly on the gradent feld of a surface. TPS was presented by Duchon n 976[33]. It can transform the shape of surface non-rgd n the sense of mnmum ntegral bendng energy. It represents a natural parametrc generalzaton from rgd to mld non rgd deformatons. Gven a set of control ponts{ w =, 2,, K}, Thn-plate-splne functon bascally defnes a spatal mappng whch maps pont X nto a new locaton represented by, K f ( X) = cϕ ( x w ) () = c s a set of mappng coeffcents. The kernel functon s the thn splne ϕ ( r) = r 2 log r. However, ths method s where denotes the Eucldean norm and { } global deformaton way and dd not consder detal nformaton of surface. In extremely case, a source surface can be transformed nto a mode whch s exactly the same as target patch gven enough control ponts. The extremely result can be seen n the Fg 5. In Fg 5 the left column s a 3D template face data. The faces n the mddle column are raw data. The samples n rght column are the deformed ones whch are formed by 000 tmes TPS deformaton. Obvously, n each TPS deformaton steps the detal nformaton of the template face lost more or less. Instead of deformng the spatal coordnates of a surface mesh, gradent based technques manpulate the mesh gradent feld and derves the surface matchng the deformed gradent feld by solvng a lnear Posson system. Gven a trangle mesh G wth n vertces { v, v2,, vn} and m trangles { t, t2,, t m }. Each mesh can be regarded as three scalar felds defned on a common doman that s actually an abstract structure. Let p( ) be the scalar value attached to vertex v on doman mesh G. A mesh scalar fled on G s defned to be a pecewse lnear combnaton: 20 ACADEMY PUBLISHER

5 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER n ( ) φ ( ) p x = x p (2) = Where φ () s pece wse lnear hat bass functons defned on the doman mesh valued at vertex v and 0 at all other vertces. The dscrete gradent of p( ) s expressed as: ( ) φ ( ) n (3) = p x = p x Gven a pecewse constant vector feld w, whch has constant value n each trangle of G, the dscrete dvergence of w at vertex v s defned as: ( )( ) ( ) φ ( ) dvw v : = w t x A (4) t t t t( v) where A t denotes the area of trangle t. Therefore, the dscrete Laplacan operator on doman mesh G s p ( v) = ( cot α + cot β )( p p) (5) 2 v V where α and β are the two angles opposte to the edge ( v, v ). Fnally, the dscrete Posson equaton s expressed as f dv( f ) dvw (6) wth specfed boundary condtons, the above equaton can be reformulated as a sparse lnear system Ax = b where the unknown vector x represents coordnates to be reconstructed. The coeffcent matrx A s determned by Eq. (4) and depends on mesh G. The vector b corresponds to known vector feld as well as the boundary condtons. The fundamental of ths algorthm s a Posson shape nterpolator. As a dfferental property, the gradent can be modfed locally, whch allows the local analyss and nterpolaton to be carred out n a more canoncal way. However t has less global constrant. To generate a good sttchng result, we combne these two methods. Frst we use TPS get a roughly surface deformaton. Because of ts global deformaton characterstc, ponts on the boundary may have some dstorton. Then we use the Posson system them to spread the dscrepancy along the boundary. In ths paper, overlapped ponts on the boundary are regarded as the control ponts. The nterpolaton functon s learned from the movements of the boundary ponts. To facltate the descrpton, the notatons are llustrated n Fg. 6: Let F S, a closet patch of 3D face, be the new face organ of the 3D face sample, and lett be the target 3D face sample. Let S be the boundary of new face organ patch. Fgure 6. Interpolaton otaton. Fgure 7. Surface Sttchng Result. The correspondng ponts on face patch are (,,, ) U = u u2 u m S and V T T V = ( v 2 ) and, v,, v m, where U, we need to establsh correspondences between other non feature ponts. uk and k v denote the k th correspondng ponts, m s the total number of correspondng ponts, and S,T are two surfaces. The nterpolaton functon f, whch maps pont set U to V subect to perfect algnments,s gven by the followng condtons: ( ) = + x + y + z + ( ) = T ( ) ( ) f x, y, z c a x a y a z ws P x, y 7 where c, a x, 2 2 ( ) log a y, s u = u u az and n w are TPS parameters, In order to mprove the sttchng result, we separate the rgd transformaton and non rgd movement. The non rgd movements of ponts are computed accordng to warpng functon F, and the rgd the transform s computed through coordnate algnment. Frst we pck a mddle pont on the boundary as the algned pont. Then we compute the movng vector between the algn pont and orgnal pont n the coordnate. At last the organ surface and face model move n terms of the movng vector. The sttchng synthess result s shown n Fg 7. Although they are consstency between adacency patches, the appearance seems unrealstc. The sttchng method only consders the local consstency and pays lttle attenton to global consstency. As we can see from Fg 7, these faces are asymmetry and ther eyes are dfferent. To solve ths problem we must ensure that the later patches are consstent wth the prevously pasted ones. In order to solve ths problem, we buld a global 20 ACADEMY PUBLISHER

6 472 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER 20 model as the global constrant and enforce the local consstency comply wth global model. Texture Sttchng Here, we present a 3D texture sttchng method based on dfferental coordnates for face model. Ths method s constructed based on dfferental coordnates and can solve the sttchng problem for texture elements on geometry surface. It thoroughly consders the characterstc of the texture and the correlaton between texture elements and surface geometry. Frst, to descrbe the correlaton of texture elements between ponts, we devse a new dfferental operator whch can reflect the nterrelaton between texture elements and surface geometry. Then an nterpolaton framework s proposed based on PDE. Because of the unsatsfactory nterpolaton result, a gudance vector s ntroduced nto the mnmzaton problem. Purpose of texture sttchng s to merge dfferent texture patches at the boundary and keep ther local structure at the same tme. Tradtonal sttchng methods are to construct a harmonc nterplant that smoothly spreads the dscrepancy along the boundary to the entre source patch. The key pont of harmonc nterplant s to fnd an approprate operator whch can fully represent local structure nformaton of texture elements. However, compare wth a great deal of efforts on mage sttchng, few of them pay attenton to the feld 3D texture sttchng. For the pxel based edtng stuaton, the structure nformaton s computed accordng to the dversty nformaton on each pont. But the adacency relatonshp n geometry surface s totally dfferent from the 2D mage. The number of adon pont are not fxed and the geometry characterstc vares from each other. The new operator should consder all the factors at the same tme. Inspred by the Yaron s[34] work, we proposed a operator based on dfferental coordnates. The dfferental coordnates can represent the detals and are defned by a lnear transformaton of the mesh vertces. The smplest form of dfferental coordnates s Laplacan coordnates. The powerful propertes of Laplacan coordnates for mesh representaton have been exploted n varous ways. t can effectvely used for morphng and free-form modelng and would be more sutable to constran under a global deformaton of the mesh. On the geometry surface, dfferent ponts have dfferent normal and the dstance between each other are not same. All these dfference wll have mpact on the dstrbuton of texture elements on t. After thoroughly analyss based on statstcal, we found that the smlarty of texture nformaton between ponts s hgh when the normal dscrepancy and dstance between them are all small. On the contrary, the smlarty decreases ether the normal dscrepancy become wde of dstance long. Accordng to above analyss, we desgn the new operator. Let G = ( V, E, T) be a 3D mesh, where V denotes the set of vertces, E denotes the set of edges and T denotes the set of texture on pont. t s defned as the texture nformaton on vertex. So the approxmaton of t s defned by lnear combnaton of ts adon ponts: t a t (8) supp(), l Where supp( ) denotes the set of vertex ndces that belong to set of adon ponts of v and a s defned as by the followng equaton: a = (9) wd + w n n 2 where w and w 2 are experence value, d s the dstance between v and v, n n s the normal dscrepancy. Then new operator can be defned as followng: D( t) = t at = δ (0) (, ) E where δ s used to descrbe the texture structure on geometry surface. D s defned as dfferental representaton of the geometry texture. In ths secton we explan n detal how dfferental operator can be used to perform seamless 3D texture sttchng. To sttch two texture patches, the sttchng crtera should be constructed n advance. Usually we sttch two patches by changng both of ther boundary values. However, t s too complex and unstable to change both of them at the same tme. In ths paper, we set one of them as target patch and other as source patch. In that case, the complex sttchng work s change to transform source patch so that these two patches can be seamless sttched. To merge source texture patch wth the target patch, we should let the boundary texture value of source patch agree wth the target patch. Snce we have defned the new operator, dscrepancy of texture elements are spread 3 accord to ths new operator. Let GS R be the doman 3 of the source geometry texture patch and GT R be the target patch. We would lke to sttch G S seamless wth G. Typcally the sttchng work s performed by T fndng an nterpolatng functon whch can make the transformed patch had a smooth varaton accordng to the new boundary value. The nterpolatng functon s computed by solvng the followng mnmzaton problem: () 2 * δ δ () = δ () ( ) mn Dt wth t t t G GS GT where G s the boundary of mesh G and δ s gudance vector derved from the source patch. Its dscrete soluton satsfes the followng smultaneous lnear equatons: * t a t = a t + δ (2) GS GS 20 ACADEMY PUBLISHER

7 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER Fgure 8. Texture Sttchng Result.Frst row s prmary texture of face sample. Other rows are novel sample wth new eye. * where t s the texture value of the -pont on target patch. Accordng to the above equatons, we can get a harmonc nterpolaton whch cans seamless sttch the source patch onto target patch. Ths equaton s solved n the least-squares sense. Except that the source patch can also keep ts local structure. Global Model The global parametrc model descrbes face data we used s 3D morphable model. Ths model s smlar to prncpal component analyss, but s fully probablstc. To construct the morphable model, prototypc 3D faces are acqured frstly. The morphable model s a type of lnear model, thus the prototypes must be algned to have operaton of lnear combnaton. Intutvely the lnear operaton of faces s based on the operaton of ther ponts. The lnear operaton of faces s mplemented by the operaton of the correspondng ponts. Every prototypc face can be represented by a shape vector and a texture vector n format: 3n S = ( x, y, z, x 2, y 2, z2,, xn, yn, zn ) R 3 3n T = ( r, g, b, r 2, g2, b 2,, rn, gn, bn ) R where =, 2,,, s the number of 3D faces, n s the number of vertexes on 3D face and ( r, g, b ) s the correspondng R, G, B color values of vertex ( x, y, z ). From these shape vectors, a novel face wll be represented by lnear combnaton operaton of these faces as follows: Snew = as T = bt (4) where = a = = b = As a number of hgh dense vertex faces lead too much computaton and there s correlaton among the prototypes, technque of PCA (prncpal component analyss) s used to gve the fnal morphable model format: m S = S+ α s T = T + βt mod el model = = where S = S = T = m T = ( 5) ( ) Fgure 9. ovel 3D Faces. are average shape and texture. s, t s man components n descendng order accordng to ther egenvalues. α = ( α, α2,, αm ) and β = β β β are shape and texture combnaton ( ),,, m 2 coeffcents. Once the coeffcents are gven, correspondng 3D face can be obtaned. The sample generated by ths model s coarse and not good at local detal whch s a coarse model. But t can provde a global constran for face modelng. Based on ths coarse model, we refne t patch by patch through replace the patch on coarse model wth the correspondng one derved from the real sample model. Because these patches are derved from real sample, so the refned model must have enough detal nformaton and stratfy face topologcal at the same tme. V. EXPERIMETS In Sectons 4, we presented two 3D face modelng methods wth complementary propertes. In ths secton we combne them together to generate more realstc sample. Frst, a coarse sample s generated by morphable model. Because morphable model s a lnear model, the generated one has less detal nformaton. To add detal nformaton, we then synthesze a model usng the local model method that s consstent wth ths global model. In probablstc terms, we condton the local model on the result of the global parametrc model. In practce ths condtonng s mplemented as follows. As before, patches are chosen such that they are vsually consstent wth the patches above and left. However, we also requre vsual consstency wth the results of the global model. Patch choce s now determned usng a weghted sum of these two constrants. In order to seamless sttch the new patches wth coarse model; we use non-rgd transformaton method as sttchng way. At last, the new novel 3D faces are generated by our proposed method. The experment results are shown n Fg. 9. From Fg. 9, we can see that the generated faces results have more characterstc feature and topologcal structure meets face structure demandng. In order to completely examne the novel face, we dsplay them n three drectons. Compare wth Fg. 7, we can fnd that topologcal structure of the generated model are guaranteed by consstency constran. As we can see, eyes of the generated model are not the same as random generated one whch have obvous dfference. The consstency between canddate patch and coarse model are calculated accordng to the error dfference between 20 ACADEMY PUBLISHER

8 474 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER 20 the two correspondng patches. Because the scale and shape dfferences can be drectly reflect on the patch area, the proposed smlarty calculaton tool s more effectve. If two patches are not the same, ther area must not the same. Besdes that the dfference computng are based on feature correspondng whch are establshed accordng to secton 3, the stuaton that two dfferent organ have the same area can be avod. To evaluate the performance of the proposed method n data expandng, we compare face recognton rates based on Prncpal component analyss (PCA) traned on dfferent tranng sets. Once we start the recognton work, the PCA-bass should be traned based on a tranng set at frst. Then all samples n the gallery and probe are proected nto the bass to obtan ther proecton parameters. To dentfy one sample n probe set, one should the compare ts parameter vector wth all samples parameter vector n gallery set. In ths paper, the match metrc we used for face recognton s the Eucldean Dstance between parameter vectors. The closest one n gallery set s the recognton result. Typcally, a bass bult on a good tranng set wll have a strong express ablty and the good tranng set usually have more samples and wder data coverage. For our experment, these samples used n ths paper are also derved from BJUT-3D face database. 420 ndvduals are used n our experments, and each has two neutral samples. All the samples are normalzed by the method mentoned n Secton3. The same gallery and probe set are used n each experment. Here the frst tranng set s derved from the orgnal sample set. The other tranng sets are derved from dfferent generaton populaton. Each generaton roughly contans more 420 ndvduals than ancent one. Dfferent bass are then traned on these dfferent sample set. The face recognton work s carred out several tmes based on dfferent bass. Table show the verfcaton rates based on dfferent bass. One can fnd that we obtan a recognton rate of 78.25% usng a bass traned on the ntal samples set and the rate gradually grows up wth the ncreasng of sample num. Obvously, the performance of a bass traned on chld generaton set s better than traned on ts former one. We beleved that ths performance s result from the optmzaton of tranng samples. VI. COCLUSIO AD FUTURE WORK In ths paper, we present a novel modelng method whch can have us get novel 3D face model and these novel face model don t lke any of the real sample model n the database. Ths can greatly help us to expandng our 3D face database and propose a new 3D face model way whch can be used n 3D face recognton n the feature. The shortcomng of our method s on patch sttchng. To assure the new patch can be seamless merge wth coarse face model, t must be rotaton and transformed. From the experment result, we can see that t can only assure the nternal consstency but have bad result on the edge. In the future, we wll pay more attenton on mprovng the way of patch sttchng. Tranng set Recognt on rate TABLE I. FACE RECOGITIO RATE Orgnal Extende Extende Extende set d set d set2 d set % 80.9% 83.33% 83.33% ACKOWLEDGMET The authors would lke to thank the anonymous revewers for ther constructve comments on mprovng ths paper. Ths paper s supported by the atonal Basc Research Program of Chna (973 Program) ( 20CB ),the atonal atural Scence Foundaton of Chna ( , , ,U ). REFERECES [] R. Brunell and T. Poggo, Face recognton: feature vs templates, IEEE Trans. Pattern Anal. Mach. Intell, 993, 5(0): [2] A. M. Martnez and A. C. Kak, PCA vs LDA, IEEE Trans. Pattern Anal.Mach. Intell, 200, 23(2): [3] J. Kttler, A. Hlton, M. Hamouz and J. Illngworth, 3D Asssted Face Recognton: A Survey of 3D Imagng, Modellng, Recognton Approaches, Proceedngs of the 2005 IEEE Computer Socety Conference on Computer Vson, Pattern Recognton (CVPR 05), 2005: 4-2. [4] Xaoguang Lu and Anl K. Jan, Automatc Feature Extracton for Multvew 3D Face Recognton, 7th Internatonal Conference on Automatc Face, Gesture Recognton (FGR 06), 2006: [5] K. W. Bowyer, K. Chang and P. J. Flynn, An evaluaton of mult-modal 2d+3d face bometrcs, IEEE PAMI, 2005, 27(4): [6] G. Gordon, Face recognton based on depth maps, surface curvature, SPIE, 99: [7] C. Chua, F. Han and Y. Ho, 3D human face recognton usng pont sgnature, IEEE Inetrnatonal Conference on Automatc Face, Gesture Recognton, 2000: [8] C. Beumer and M. Acheroy, Face verfcaton from 3d, grey level cues, Pattern Recognton Letters, 200, 22: [9] A. M. Bronsten, M. M. Bronsten and R. Kmmel, Threedmensonal face recognton, Internatonal Journal of Computer Vson, 2005, 64(): [0] K. I. Chang, K. W. Bowyer and P. J. Flynn, An evaluaton of multmodal 2d+3d face bometrcs, IEEE Transactons on Pattern Analyss, Machne Intellgence, 2005, 27(4): [] J. Yang and J. J. Yang, From mage vector to matrx: a straghtforward mage proecton technque-impca vs. PCA, Pattern Recognton, 2002, 35(9): [2] V. Blanz, P. Grother, P. J. Phllps and T. Vetter, Face Recognton Based on Frontal Vews Generated from on- Frontal Images, IEEE Computer Socety Conference on Computer Vson, Pattern Recognton (CVPR 05), 2005, 2: [3] J. Lee, B. Moghaddam, H. Pfster and R. Machrau, A Blnear Illumnaton Model for Robust Face Recognton, Tenth IEEE Internatonal Conference on Computer Vson (ICCV 05), 2005, 2: ACADEMY PUBLISHER

9 JOURAL OF MULTIMEDIA, VOL. 6, O. 5, OCTOBER [4] Le Zhang and Dmtrs Samaras, Face Recognton from a Sngle Tranng Image under Arbtrary Unknown Lghtng Usng Sphercal Harmoncs, IEEE Transactons on Pattern Analyss, Machne Intellgence, 2006, 29(3): [5] Blanz V., Vetter T., 999. A morphable model for the synthess of 3D faces, ACM Trans. SIGGRAPH 99, pp [6] Goh R., Lu L., Lu X., Chen, T., The CMU Face In Acton (FIA) Database, Proc. ICCV 2005 Workshop on Analyss and Modelng of Face and Gestures, [7] Charles B., D-RMA 3D face database, [8] Moreno A., Sanchez A., Gavab: A 3D Face Database, In: Second COST Workshop on Bometrcs on the Internet, [9] UOY 3D face database, abase.html [20] Messer K., Matas J., Kttler J., Matre G., 999. XM2VTSDB: The Extended M2VTS Database. In Conf. on Audo and Vdeo-based Bometrc Person Authentcaton, [2] BJUT 3D face database, database.htm [22] Efron B., 987. The Jackknfe, the Bootstrap and Other Resamplng Plans, Socety of Industral and Appled Mathematcs. [23] Davson A.C., Hnkley D.V., 997. Bootstrap Methods and Ther Applcaton, Cambrdge Unversty Press. [24] Breman L., 996. Baggng predctors, Machne Learnng. [25] Freund Y., Robert E.S., 996. Experments wth a new boostng algorthm,in: Conf. on Machne Learnng, [26] Tan X., Chen S., Zhou Z.H., Zhang F., Face recognton from a sngle mage per person: a survey, J.Sc. Pattern Recognton. 39, [27] Lu X., Jan A.K., Resamplng for face recognton, n Proc. on Audo and Vdeo-Based Bometrc Person Authentcaton, [28] Krby M., Srovch L.,990. Applcaton of the karhunen- Love procedure for the characterzaton of human faces, IEEE Trans. Pattern Analyss and Machne Intellgence, 2, [29] Torre F., Gross R., Baker S., Kumar V., Representatonal orented component analyss (ROCA) for face recognton wth one sample mage per tranng class, n: IEEE Conf. on Computer Vson and Pattern Recognton, [30] Chen J., Chen X.L., Gao W., Expand tranng set for face detecton by GA resamplng,n: IEEE Conf. on Automatc Face and Gesture Recognton, [3] Umar M., Smon J.D.P., Jan K., Vso-lzaton generatng novel facal mages ACM Trans Graphc, 28(3). [32] BJUT-3D Face Database, mul-lab/3dface/ facedatabase.htm [33] J. Duchon, 976. Splnes mnmzng rotaton nvarant semnorms n sobolev spaces, constructve theory of functons of several varables,, pp 85-. [34] LIPMA Y.,SORKIE O.,COHE-OR D.,LEVI D, Dfferental coordnates for nteractve mesh edtng. In Internatonal Conference on Shape Modelng and Applcatons,8 90. Yun Ge Ph.D. canddate at College of Computer Scence and Technology, Beng Unversty of Technology. Hs research nterest covers face modelng and face anmaton. Yanfeng Sun Professor at College of Computer Scence and Technology, Beng Unversty of Technology. She receved her Ph.D. degree from Dalan Unversty of Technology n 993. She s the membershp of Chna Computer Federaton Her research nterest covers mult-functonal percepton and mage process. Baoca Yn Professor at College of Computer Scence and Technology, Beng Unversty of Technology. He receved hs ph.d. degree from Dalan Unversty of Technology n 993. He s the membershp of Chna Computer Federaton. Hs research nterest covers multmeda, mult-functonal percepton, vrtual realty and computer graphcs. Henglang Tang Ph.D. canddate at College of Computer Scence and Technology, Beng Unversty of Technology. Hs research nterest covers 3D face recognton. 20 ACADEMY PUBLISHER