Viseme aware realistic 3D face modeling from range images

Transcription

1 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl Vseme aware realstc 3D face modelng from range mages Ken Yano and Koch Harada, Graduate School of Engneerng Hroshma Unversty, Hroshma, Japan Summary In ths paper, we propose an example based realstc face modelng method wth vseme control. he vseme descrbes the partcular facal and oral postons and movements that occur alongsde the vocng of phonemes. In facal anmaton such as speech anmaton and talkng head, etc, a face model wth open mouth s often used and the model s anmated along wth the speech sound by synchronzng the change of mouth shape to that of the phoneme. We clam that vsemes as well as facal expressons are dosyncratc by nature, so they have to be modeled specfc to the subect to generate a truthful facal anmaton for the subect. he proposed method tres to automate the creaton of a realstc face model wth vseme control from a set of scanned data. Key words: Face modelng, surface reconstructon, morphng. Introducton Creatng face models that look and move realstcally s an mportant problem n computer graphcs. It s also one of the most dffcult ones, as even the mnute changes n facal expresson can reveal complex moods and emotons. he presence of very convncng characters n recent flms makes a strong case that these dffcultes can be overcome wth the ad of hghly sklled anmators. Because of the sheer amount of work requred to create such models, there s a clear need for more automated technques. A vseme s a supposed basc unt of speech n the vsual doman. It descrbes the partcular facal and oral postons and movements that occur alongsde the vocng of phonemes. We propose an example based realstc face modelng technque especally to create a morphable face model wth expressve vseme control. he nput to our system s a set of range and color mages of the specfc subect scanned by the scanner []. Note that at each pxel coordnate of the range mage contans a measured 3D coordnate and each pxel of the color mage contans RGB color ntensty. A par of the two mages s scanned smultaneously and has natural pxel-to-pxel" correspondence between them. A statc 3D face model s generated n two folds. Frst a general 2D facal template mesh s ftted to the mages of the subect usng RBF (Radal Bass Functon) method. he fttng s processed n 2D space, the mage plane, usng the constrants set by the user. he constrant s a set of fducal marker ponts, such as the tp of the nose and corners of the eyes, etc. After the RBF fttng, a 3D coordnate s sampled at each vertex of the deformed mesh from the range mage as well as a texture coordnate from the color mage. he connectvty of the sampled ponts s naturally gven by the template mesh. he face models generated offer "pont-to-pont" correspondence among them rrespectve of the knd of the vseme of the subect. hs feature makes t possble to statstcally analyze the geometrc changes by vseme. In facal modelng, sculptng expressve open mouths requres dauntng task even for a sklled modeler as well as expressve eyes. oreover, artculatng an open mouth n a realstc way s one of the dffcultes that an anmator encounters for facal anmaton. In order to estmate the geometrc changes of a face model due to vseme, especally the shape of a open mouth, we frst prepare the 3D face models for a set of vsemes as well as the base model (closed mouth poston). Gven a set of 3D face models for each vseme and for the closed mouth, the geometrc changes due to each vseme are estmated by subtractng the base model from the face model of the vseme. In order to acqure relable moton vectors, the geometrc changes, we create a stencl mask to remove undesred moton vectors for the upper part of the face. Gven the geometrc changes for each vseme, we construct a morphable face model wth expressve vseme control. wo types of morphng are expermented. he frst s the blend shapes n whch a novel face model s create as a convex lnear combnaton of $n$ bass vectors, each vector beng one of the blendshapes, e.g., the face tself. In the second method, we apply PCA (Prncpal Component Analyss) to the face space. PCA s a statstcal tool that transforms the space such that the covarance matrx s dagonal (.e., t decorrelates the data). anuscrpt receved Aprl 5, 2009 anuscrpt revsed Aprl 20, 2009

2 246 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl 2009 he morphable model by PCA provdes an another alternatve to generate a novel face model wth expressve vseme, however n most cases the bass vectors of the decomposed face space do not have clear semantcs, so t requres the user some tmes to fgure out what effects each bass vector mght have for the deformed face model. he man contrbuton of ths paper s to create a realstc morphable face model wth vseme control from a set of scanned data. he proposed method does not requre specal modelng or anmaton sklls and proceeds wth as lttle as user nterventons. ost of the relevant works pay lttle attenton or no attenton about the change of facal geometry due to the open mouth postons or the vsemes. he results ensure that our method provdes a novel way to generate a realstc face model wth expressve vseme control and the generated models can be used for further processng for downstream applcatons. 2. Related Works Correspondent face models are often used n graphcs and vson communty. Blanz et al [0] create a morphable face model by takng range mages of several faces usng a 3D scanner and puttng them nto "one-to-one" correspondence by expressng each shape usng the same mesh structure. Usng the morphable model, t s possble to group changes n vertex poston together for representng common changes n shape among several surfaces. Usng prncpal component analyss, they succeeded to fnd a bass for expressng shape changes between faces. odel wth full correspondence was also studed by Praun et al [9]. In order to establsh full correspondences between models, they create a base doman that s shared between models, and then apply a consstent parameterzaton. hey search for topologcally equvalent patch boundares to create base doman mesh. Wang et al [8] propose a novel approach for facal expresson analyss from mages. hey decompose facal expresson by Hgher-Order Sngular Value Decomposton (HOSVD), a natural generalzaton of matrx SVD and learn the expresson subspace. Several papers propose the method to learn facal expresson from vdeo [7] []. hose works typcally nvolves vsual trackng of: facal movement such as contours, optcal flow, or transent features such as furrows and wrnkles. Facal expressons are learned and recognzed n relaton to Facal Acton Codng System (FACS) [6], a popular representaton that codes vsually dstngushable facal movement n small unts. hese acton unts descrbe qualtatve measures such as "pullng lp corners", or "furrow deepenng". A facal expresson s descrbed as a combnaton of these unts. Vlasc et al [5] proposes a method for expressng changes n face shape usng a mult-lnear model, accountng for shape changes not only based on the subect's dentty but also based on varous expressons and vsemes. Yong et al [2] presents a novel approach to produce facal expresson anmaton for a new model. Instead of creatng new facal anmaton from scratch for each new model created, they take advantage of exstng anmaton data n the form of vertex moton vector. hey call ths process "expresson clonng" and t provdes an alternatve way for creatng facal anmaton for character models. 3. Realstc 3D face modelng he technques of realstc face modelng and anmaton have been spurred by emergence of 3D scannng devces at relatvely low cost. Snce range data contans artfacts such as spkes and holes, t needs to be repared for further processng. Furthermore only facal part of whole scanned data s relevant, unnecessary porton of the data has to be removed wth great care and sklls. he proposed modelng method tres to automate such lengthy operatons and produce a realstc face model for specfc subect. he detals of our method are descrbed as follows. 3. ethod - 2D emplate fttng he nput to our system s range and color mages of the subect scanned by range scanner []. he two mages are taken smultaneously by one scan so there s a natural correspondence between each par of the pxels of the two mages. At each pxel locaton of the range data contans a measured 3D coordnate (x,y,z) from the vew of the scanner. o dsplay range mages, the depth nformaton (z coordnate) of each pxel s converted to gray scale ntensty. Fgure : From left to rght; (a) A 2D template mesh s shown wth facal feature ponts. he template mesh contans 44 vertces and 804 faces. (b) he user s asked to select feature ponts nteractvely on the color mage. he selected feature ponts are shown n red. Each selected feature pont corresponds to the feature pont defned on the 2D template mesh.

3 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl Frst, a 2D template mesh (wth zero depth) shown n Fgure s prepared on whch a set of fducal feature ponts ( FP ) s defned. wenty feature ponts n total are emprcally defned. he user s asked to select facal feature ponts ( F P ) nteractvely on color mage (Fgure ). As a result, there s a one-to-one mappng, Φ, from the ponts set ( FP ) to the pont set ( F P ). After establshng the mappng, Φ, the template mesh s deformed and ftted to the face mage usng the mappng as the constrants. he fttng s done usng RBF (Radal Bass Functon) technque, a well known scattered data nterpolaton technque, or some authors call t PS (hn Plate Splne) nterpolaton. he detal of the algorthm as follows. Gven a par of pont patterns wth known correspondences U = ( u u,..., ) and ( ), 2 V = v, v2,..., v m, where U ( FP), V ( FP ), and m = 20 n our case, we need to establsh correspondences between other ponts on the template mesh to the pxel locaton on the facal mage. he warpng functon, F, whch warps the pont set U to the pont set V subect to perfect algnment, s gven by the condtons F ( u ) = v for =,2,..., m. he nterpolaton deformaton model s gven n terms of the warpng functon F (u), wth m F( u) = A( u) + = λ φ( u u ), u R 2,where φ ( u ) = r log( r) s a radcally symmetrc bass functon, whch s known as thn plane splne functon, λ s a real-valued weght, and u u s Eucldean dstance. he functon, A (u), s a degree one polynomal that accounts for the lnear and constant porton of F (u) whch s defned as A ( u) = a0 + a* u x + a2* u y for 2D nterpolaton. Solvng for the weghts λ and the coeffcents of A (u) subect to the gven constrants yelds a functon that both 2 u m nterpolates the constrants and mnmzes the Equaton above n least-square sense. In order to solve for the unknown weghts λ and the coeffcents of A (u), the constrants become v m = A( u ) + = λ φ( u u Snce the equaton s lnear wth respect to the unknowns, λ and the coeffcents of A (u), t can be formulated as a lnear system. Fgure 2: Fttng the 2D template mesh to the mage. From left to rght; (a) range mage (b) deformed 2D template mesh (low resoluton) (c) deformed 2D template mesh (hgh resoluton) he resultng functon F (u) exactly nterpolates the gven facal features ( F P ) and s not subect to approxmaton or dscretzaton errors. Note that the number of weghts to be determned does not grow wth the number of vertces n the template mesh; rather t s only dependent on the number of constrants. he 2D template mesh s deformed by the functon, F (u), and t s shown n Fgure 2. As the fgure ndcates, the template mesh s ncely deformed and ftted to the facal area on the mage wth all the feature ponts exactly nterpolated. In order to generate a 3D face model, 3D coordnate at each vertex poston of the deformed 2D template mesh s calculated from the range mage. As mentoned before, at each pxel coordnate of the range mage contans measured 3D coordnate of the subect, then the 3D coordnate s calculated as a blnear nterpolaton of the 3D coordnates at the four neghborng pxel coordnate of the vertex poston. Note that each vertex of the deformed mesh should not necessarly resde on the grd pxel coordnate of the mage; we use blnear nterpolaton to correctly calculate the 3D coordnates. One of the well known ssues pertanng to 3D scanned data s that there are holes or mssng data. For varous )

4 248 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl 2009 reasons reflected laser beam may be obscured or dspersed so that the sensors see no range or color data at some surface ponts. Especally for frontal face, those mssng data commonly occur on the undersde of the aw, the head har, the eyebrows, open mouth, and pupls of the eyes, etc. here are some technques to fll n those mssng data. For example [3] ntroduces the followng algorthm [Step 0] he range data s converted to a floatng pont array wth the mssng values set to 0.0 [Step ] A second matte array s created and set to.0 where vald range data exst and to 0.0 where gaps exst. [Step 2] In the regons around 0.0 matte value, a small blurrng flter kernel s appled to both the matte data and the surface data [Step 3] Where the blurred matte value ncreases above a threshold value, the surface sample s replaced by the blurred surface value dvded by blurred matte value. [Step 4] he flterng n recursvely appled untl no matte values are below the threshold We have ntegrated the above algorthm to our system and appled to the range mage before samplng the 3D coordnates. Note that ths algorthm only flls n the small gaps and cannot cope wth large mssng area such as the head har and the background. he resoluton of the ntal 2D template mesh s consdered to be low, so as the resoluton of the generated 3D face model. In order to ncrease the resoluton, we apply the one-to-four subdvson method to the deformed 2D template mesh before samplng 3D coordnates. he subdvson method subdvdes each trangle face nto four trangle faces, thus ncreases the resoluton by the factor of four. Fgure 2 shows the deformed template mesh before and after the subdvson. Snce the color mage and the range mage have "one-toone" pxel correspondence between them, as we re-sample 3D coordnates from the range mage, we can also obtan texture coordnate at each vertex of the 3D face model. Fgure 3 shows the generated 3D face model n varous renderng modes for a subect. Fgure 3: Generated 3D face model. From left to rght; (a) shadng model (b) texture model (c) wre frame model 4. odelng of vseme odelng a realstc face wth open mouth s a challengng task. In order to create a fully functonal face model, t s common that face, teeth and tongue, etc are modeled separately and are placed later at approprate postons of the entre face model. Snce we only concern about the dynamcs of facal geometry due to vseme, we do not dscuss the ssues of teeth, tongue, etc. he dynamcs of mouth shape s dosyncratc as well as the facal expresson, so we clam that vsemes should dffer from person to person. Our modelng method of vsemes s an example based and the changes of the geometry of the mouth are learned and modeled from a set of scanned data taken from the same subect from whom we model the base (closed mouth) face model n Secton 3. he detals of the method are descrbed as follows. 4. ethod Obtan a scanned data for each vseme from the subect. he subect s asked to make natural mouth shape n fve postons one by one by pronouncng these fve words then haltng the mouth poston at the underlned two characters (. car. man. she v. eel v. too) Step: Generate a 3D face model for each vseme usng the same technque n Secton 3. However we use a dfferent set of feature ponts to handle the deformaton of the open mouth. he new set of the feature ponts and the result of the fttng s shown n Fgure 4.

5 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl Fgure 4: Fttng the 2D template mesh to the mage of open mouth. From left to rght; (a) color mage wth markers. A new set of markers are defned to model open mouth (b) deformed 2D template mesh (low resoluton) (c) deformed 2D template mesh ( hgh resoluton) Step2: Estmate 3D moton vectors from the base model (closed mouth) to the specfc model of the vseme. Snce the two models are generated usng the same template mesh, there s a natural correspondence between each par of the vertces. In order to obtan the 3D moton vectors, we frst algn the two model (base model and the vseme specfc model) usng ICP (Iteratve Closest Pont) method [4]. Step3: he ntal correspondences of the ponts of the two models are gven by the user. As shown n Fgure 5, sx pars of ponts are selected for each model. hese ponts are selected because they are not lkely to be transformed due to the deformaton of the mouth shapes and sx ponts are enough to calculate the rotaton and the translaton of the rgd transformaton between the two models. Fgure 6 shows the algnment of the two models before and after ICP method s appled. Fgure 5: Pont correspondence between the two model for ICP he 3D moton vectors can be obtaned from the algned models by subtractng the base model from the specfc model of each vseme. However, we do not want undesred effects of the moton vectors over the upper face due to the changes of the mouth shape; we suppress them by makng a stencl mask over the face. Fgure 6: Rgd algnment of the two model by ICP. Frst row: ntal pose of the two models, second row: algned poses after ICP s appled he stencl mask s created n these steps.. Generate seeds ponts for the stencl mask. he seeds ponts are generated by tracng the shortest path between selected two ponts. he seeds ponts are all the vertces along the shortest path generated. In order to make a vald mask, a set of fve shortest paths s generated usng Dkstra algorthm. he start and the goal pont of each shortest path are defned emprcally. hese steps are llustrated n Fgure After obtanng the seed ponts, we make vrtual spheres centered at each seed pont then assgn every vertces whch s nsde of one of the spheres to the stencl mask. he radus of the sphere s decded as 20mm for the subect. Fgure 7 (d) shows the generated fnal stencl mask (vertces n orange color). 3. he 3D moton vectors under the stencl mask are suppressed to zero vectors. Compared wth the base model, the vseme specfc model can reveal nosy measured data n the open mouth area. hs s because at the area around the open mouth, the reflected laser beam may be obscured n the hollow area between the lps or dspersed from the teeth. Fgure 8 shows unprocessed mesh models from range data and the mesh models generated by our method.

6 250 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl 2009 to get the face model. Fgure 0 shows a set of generated face models for each vseme. he procedure s appled to two dfferent subects and the results are shown n Fgure 7 Fgure 7: ask generaton to extract 3D moton vector. From left to rght, (a) seed ponts from the frst path (b) seed ponts from the second and the thrd paths (c) seed ponts from the fourth and ffth paths (d) generated mask. he mask s the vertces n orange. he uncovered vertces are colored n black. In order to smooth the nosy 3D moton vectors, we use Laplacan smoothng, e.g., a sgnal processng approach proposed by [2]. Laplacan smoothng removes hgh frequency nose. he formula s gven as Fgure 9: he result of deformaton. arget model (blue), deformed model (green) d v ) / t = wδd( v ) ( Δd ( v ) = N( v ) v N ( v ) ( d( v ) d( v )),where d ( v ) s a moton vector at the vertex v, N ( v ) s the -rng neghbors of the vertex v and w s a scalng factor 0 w. We set w = 0. and the smoothng operaton s terated thrty tmes to get the fnal 3D moton vectors. Fgure 0: he result of deformaton for all targets. From left to rght (a) vseme( car ) (b) vseme( she ) (c) vseme( man ) (d) vseme( eel ) (e) vseme( too) 5. orphable face model In ths secton example-based morphable 3D face models are created. wo methods are expermented. he frst method represents the morphable model as a blendng of example models. A lnear combnaton of the examples models of specfc vseme descrbes a realstc change of vsemes. he second method apples PCA to the face space. PCA defnes a bass transformaton from the set of example models to a bass that has two propertes relevant to our morphable face model. Frst, the bass vectors are ordered accordng to the varances n the dataset along each vector. Second the bass vectors are orthogonal to each other. he detals of two methods are descrbed as follows. Fgure 8: Nosy measured data around the open mouth area. Frst row shows the unprocessed 3D model obtaned from the range data. Second row shows the generated 3D model usng the template fttng. hose undesred nosy samplng data around the mouth s the result of nosy scannng data. 4.2 Generated models Fgure 9 shows generated face model of specfc vseme. he moton vectors obtaned are appled to the base model 5. orphng by Blend-Shapes In order to create a morphable face model, a set of the vseme models generated n Secton 4 s algned to the base model usng ICP method. We let S0 be the shape vector of the base model and S be the shape vectors of the model of the vseme, the morphable face model s represented as

7 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl S = w S + (.0 w ) S () = = 0, where w are non-negatve blendng weghts and (= 6) s the number of vseme poses. he shape vectors S are formed from the 3D coordnates of surface ponts: S =..., x, y, z ) ( x, y, z, n n Fgure shows the gradual deformaton from the base model to the models of three specfc vsemes. Each deformed model s created by changng the weght relevant to the specfc vseme and other weghts are set to zero. n Fgure 2: Random blendng of varous vsemes. Left: 0.6 * vseme(man) * vseme(eel) + 0. * base, Center: 0.5 * vseme(too) * vseme(car) * base, Rght: 0.8 * vseme(too) + 0. * vseme(eel) + 0. * vseme(car) 5.2 orphng by PCA Let a set of algned 3D face model represented as a matrx X, where [ S S S ] X = S S of each vseme be,and X s of dmenson 3 n, where n s the number of vertces. he dfference from the average face model (the sample mean) S s the matrx : [( S S )( S S )( S S )..( S S )] X = 0 = S S S S Prncpal component analyss seeks a set of orthogonal vectors, e, whch best descrbes the dstrbuton of the nput data n least-square sense,.e., the Eucldean proecton error s mnmzed. he typcal method of computng the prncpal components s to fnd the egenvectors of the covarance matrxc, where 2 Fgure : Gradual deformaton of each vseme. he base model s gradually deformed to the target vseme model by changng the weght of the target (w=0.3: left column, w=0.6: center column, and w=0.9: rght column). arget vseme model; frst row (vseme(car)), second row (vseme(eel)), thrd row (vseme(too)) he effect of blendng of multple vseme examples s shown n Fgure 2. C = = 0 S S = X X s 3 n 3n. hs wll normally be a huge matrx, and a full egenvector calculaton s mpractcal. Fortunately, there are only nonzero egenvalues, and they can be computed effcently wth an egenvector calculaton. It s easy to show the relatonshp between the two. he egenvectors e and egenvalues λ of C are such that Ce = λ e

8 252 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl 2009 hese are related to the egenvectors ê and egenvalues μ of the matrx D = X X n the followng way: Deˆ = μ eˆ, X CX eˆ = μ X eˆ, X eˆ = μ eˆ, X X C( X eˆ ) = μ ( X eˆ ), X eˆ = μ X eˆ Ce = λ e, showng that the egenvectors and egenvalues of $C$ can be computed as e = ( X ˆ ), e λ = μ In other words, the egenvectors of the large matrx C are equal to the egenvectors of the much smaller matrx D, premultpled by the matrx X. he nonzero egenvalues of C are equal to the egenvalues of D. Once the egenvectors of C are found, they are sorted accordng to ther correspondng egenvalues; a larger egenvalue means that more of the varance n the data s captured by the egenvector. Egenvectors correspondng to the top K evenvalues defne the face space of the vsemes. he result of PCA s descrbed as follows. Fgure 3 shows the average model, S, and Fgure 4 shows top fve prncpal egen face models. Fgure 3: Average face model he CR (Contrbuton Rate) and CCR (Cumulatve Contrbuton Rate) of a egen face model, e l, are defned as CR λ l = =, CCR = = λ l = λ λ able shows the egenvalue, CR and CCR of top fve prncpal egen models. From these results, the CCR of the top three prncpal models s over 0.95 and the last two prncpal models are less mportant. From ths analyss, we can conclude that only the top three prncpal models may be used to defne the face space thus compressng the entre model space. able : he result of PCA analyss Egen value CR CCR Frst egen model Second egen model hrd egen model Fourth egen model Ffth egen model able 2: PCA coeffcents of each vseme model ω ω 2 ω ω 3 4 ω 5 Base Vseme(too) Vseme(man) Vseme(she) Vseme(eel) Vseme(car) In order to proect each face model to ths egen face space, we frst subtract the average face model from the face model then proect t nto the egen face space as followng ( w w... w ) = ( e e2... e ) ( S ) 2 k k S, where w s the PCA coeffcent of the face model for Fgure 4: A set of prncpal egen face models. From left to rght; (a) frst egen model (b) second egen model (c) thrd egen model (d) fourth egen model (e) ffth egen model (he egen models are actually moton vectors, here we add them to the mean face model for dsplay) the th prncpal face model e. able 2 shows the PCA coeffcents of each face model and the results are plotted n Fgure 5 graphcally. From ths result, we can assume that the frst prncpal face model descrbes natural open mouth by aw rotaton, the second prncpal model descrbes puckered mouth and the thrd prncpal model descrbes mouth pulled out sdeways.

9 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl able 3: orphable model by PCA. he values are the blendng parameters, α, n Equaton 2. α α 2 α 3 Blend model Blend model Blend model Fgure 5: PCA Coeffcents of each vseme model. Graphcal dsplay of able 3 hanks to the face space decomposed by PCA, t gves the user an alternatve method to anmate the face model or to blend the prncpal face models. For nstance, exaggerated open puckered mouth can be ntutvely generated by usng the frst and second prncpal face model, whch s otherwse dffcult to be made by usng the method descrbed n Subsecton 5.. Compared wth the Equaton, the morphable face model by PCA s formulated as K S = α λ e + S ( 3 α 3) (2) = λ s the standard devaton for the th prncpal face model. Fgure 6 shows example face models generated by blendng the top three prncpal face models. he parameters of the blendng are shown n able 3. Fgure 6: Blendng face models generated from the morphable face model by PCA. he parameters of the blendng are descrbed n able 3. From left to rght; blend model, blend model2 and blend model3 he modelng by PCA offers better controllablty than the former method. Snce the bass shapes, the prncpal face models, are orthogonal wth each other, the blendng of one bass shape does not nterfere wth the blendng of the other bass shapes. However there s a drawback, e.g., the prncpal component does not have a clear semantcs of the shape tself and requre the user some tme to get the feelng of the effects that the prncpal component mght have on the result of the deformaton. 6. Concluson In ths paper, we propose an example based face modelng technque especally to create a realstc face model wth vseme control. odelng and anmatng a realstc human face requres sklls and consderable tme. he proposed method lessens the costs requred by utlzng the scanned data taken from the subect. he process runs almost automatc except that the user needs to select a set of fducal markers on the facal mages provded by the scanner. In preparaton, we generate a set of face models for the base (closed mouth) and for each vseme. he face models generated have "pont-to-pont" correspondence among them and make t straght forward to extract the moton vectors, the dsplacement vectors from the base model to the specfc model of the vseme. In order to generate a morphable model, two methods are expermented. he frst method uses the correspondent face model as the bass of the face space. A novel face model s created by a convex lnear combnaton of the bass vectors. he other method apples PCA to the face space and a novel face model s created by usng the average face model and a set of prncpal face models. he statstcal modelng method by PCA provdes another alternatve to generate a novel face whch cannot be made by the former method. he proposed method can also be used to extract expresson from a subect and generate a morphable face model wth realstc expressons. We plan to extend the morphable face model by ncorporatng the facal expressons to the face model wth vseme control, thus provdes much greater varetes of facal deformaton.

10 254 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No.4, Aprl 2009 [2] Ulrch, Jun Yong and Noh. Expresson clonng th annual conference on Computer graphcs and nteractve technques. pp Ken Yano s a PhD student of the Graduate School of Engneerng at Hroshma Unversty after workng at Sharp Corporaton. He receved the BE n 992 from Waseda Unversty and S n 998 from Central chgan Unversty n U.S. Hs current research s manly n the area of computer graphcs and computer vson. Specal nterests nclude 2D and 3D data processng and machne learnng. He s a member of AC, IEEE, IPS of Japan. Fgure 7: Realstc face models wth expressve vseme control for two subects. Subect: frst two rows, Subect2: second two rows. From the left most column to the rght most column { base model, vseme(car), vseme(eel), vseme(man), vseme(she), vseme(too) } References [] INOLA, KONIKA. Vvd 700 : Non-contact 3D laser scanner. [2] aubn, Gabrel. A Sgnal Processng Approach to Far Surface Desgn nd annual conference on Computer graphcs and nteractve technques. [3] L.Wllams. Performance drven facal anmaton. 990, Computer Graphcs, pp [4] ckay, Paul J. Besl and Nel D. A ethod for Regstraton of 3-D Shapes. 992, IEEE ransactons on Pattern Analyss and achne Intellgence, pp [5] Popovc, Danel Vlasc and atthew Brand and Hanspeter Pfster and Jovan. Face ransfer wth ultlnear odels. 2005, AC ransactons on Graphcs, pp [6] Paul Ekman, Wallace V.Fressen. Facal Acton Codng System. [7] Blake, B Bascle. Separablty of pose and expresson n facal trackng and anmaton Int. Conf. Computer Vson. pp [8] Ahua,H.Wang. Facal expresson decomposton. IEEE Internatonal Conference on Computer Vson. pp [9] Schroder, Eml Praun and Wm Sweldens and Peter.Consstent esh Parameterzatons Computer graphcs and nteractve technques. pp [0] Vetter, Volker Blanz and homas. Face Recognton Based on Fttng a 3D orphable odel. 2003, IEEE ransactons on Pattern Analyss and achne Intellgence, pp [] Bregler, E S Chuang and H Deshpande. Facal expresson space learnng Pacfc Conference on Computer Graphcs and Applcatons. pp Koch Harada s a professor of the Graduate School of Engneerng at Hroshma Unversty. He receved the BE n 973 from Hroshma Unversty, and S and PhD n 975 and 978, respectvely, from okyo Insttute of echnology. Hs current research s manly n the area of computer graphcs. Specal nterests nclude man-machne nterface through graphcs; 3D data nput technques, data converson between 2D and 3D geometry, effectve nteractve usage of curved surfaces. He s a member of AC, IPS of Japan, and IEICE of Japan.