3D Mesh Simplification for Deformable Human Body Mesh Using Deformation Saliency

Size: px

Start display at page:

Download "3D Mesh Simplification for Deformable Human Body Mesh Using Deformation Saliency"

Lillian Hawkins
6 years ago
Views:

1 3D Mesh Smplfcaton for Deformable Human Body Mesh Usng Deformaton Salency Tanhao ZHAO R301, SHB, CUHK Shatn, N.T , Hong Kong Kng Ng NGAN R304, SHB, CUHK Shatn, N.T , Hong Kong Songnan LI R301, SHB, CUHK Shatn, N.T , Hong Kong ABSTRACT 3D mesh of human body s the foundaton of many hot research topcs, such as 3D body pose trackng. In ths topc, the deformaton of the human body mesh has to be taken nto account because of varous poses of the human body. Consderng the tme cost of the body deformaton, however, t s mpractcal to adopt a hgh resoluton body mesh generated from scannng systems for the real-tme trackng. Mesh smplfcaton s a soluton to reduce the sze of body meshes and accelerate the deformaton process. In ths paper, we propose a mesh smplfcaton algorthm usng deformaton salency for such deformable human body meshes. Ths algorthm s based on quadrc edge contracton. The deformaton salency s computed from a set of meshes wth varous poses. Wth ths salency, our algorthm can smplfy the 3D mesh non-unformly. Experment shows that usng our algorthm can mprove the accuracy of body pose smulaton n the smplfed resoluton compared to usng classcal quadrc edge contracton methods. Keywords Mesh smplfcaton, Deformable human body mesh, Deformaton salency. 1 INTRODUCTION Nowadays, 3D human body trackng s a hot research topc [Bog15, Baa13, We12, Gan10]. In ths topc, 3D body mesh s a basc structure. Generally, the nput of 3D body trackng s a sequence of depth mages captured by depth cameras. A body mesh n an ntal pose s selected as a template mesh. The template mesh s deformed to dfferent poses accordng to the real-tme nput frames. The body mesh s defned as the combnaton of a pont cloud and a set of trangulated faces. The faces cover all the ponts and there s no overlay between any two faces. Typcally the 3D mesh s obtaned from a multple depth-camera scannng system or a laser scannng system. The mesh generated by these scannng systems s n very hgh resoluton, contanng tens of thousands of vertces and faces, such as the CAESAR dataset. Nevertheless, 3D human body trackng s requred to be real-tme, so the body mesh n such hgh resoluton s not sutable for the real-tme trackng. Reducng the mesh resoluton,.e., the number of vertces and faces of the mesh s necessary. Permsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. To copy otherwse, or republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. In ths paper, we focus on the mesh smplfcaton for such deformable human body object. We adopt the deformable human body model proposed by [Ang05]. The body meshes of SCAPE model are from the CAESAR dataset. The mesh standng n the "A" pose (shown n Fg.1) s selected as the template mesh and other meshes contan varous poses of the same person. All the meshes have the same number of vertces, whch have been regstered. In dfferent meshes, the topologes of trangulated faces are also the same. The human body s parttoned nto 16 parts, contanng head, chest, torso, arms, hands, legs and feet. The moton model of SCAPE descrbes two knds of deformatons of human bodes: the rgd transformaton and non-rgd deformaton. The rgd transformaton matrces show the global transformaton of an ntegral body part. For all the vertces n the same body part, ther rgd transformaton matrces of a certan pose are the same. The non-rgd deformaton matrces ndcate the change of muscle, skn and lgament n dfferent poses. These matrces are unque for each face. Apparently, there are much more non-rgd deformatons at jonts or muscular regons than other body regons. Therefore, n smplfcaton more vertces n these specal regons should be preserved to smulate the non-rgd deformaton more accurately. However, there s a drawback of the tradtonal mesh smplfcaton algorthms when they are appled to such deformable meshes: they always smplfy a 3D mesh

2 unformly or based on geometrc features, whch means that more vertces wll be kept n some geometry-salent regons, lke fngers and face of the human. Ths may lead to very few vertces remanng n the hghly nonrgd deformng regons. Hence, our purpose s to dstngush the hghly non-rgd deformng regons, and keep more vertces n these regons when smplfyng the body mesh. Based on the quadrc edge contracton (Qslm) [Gar97], we developed a mesh smplfcaton algorthm guded by deformaton salency. In our algorthm, we compute the deformaton salency for every vertex n the template mesh from the SCAPE pose database. Besdes the deformaton salency, we also ntroduce a dstance balancng tem to avod the over-contracton. Our method can smplfy meshes regon-specfcally accordng to the non-rgd deformaton levels. Expermental result shows that usng our smplfcaton method, the deformed result s more accurate than that usng the Qslm. Ths paper s organzed as followng. Secton 2 ntroduces the related work. Secton 3 dscusses our methodology. Secton 4 analyzes expermental results and compares the proposed method wth the pror studes. Concluson s drawn n Secton 5. 2 RELATED WORK Mesh smplfcaton ams to reduce the number of vertces of a 3D mesh, meanwhle preservng the shape of the mesh as accurately as possble. To ths end, the Quadrc Edge Collapse Decmaton (Qslm) was proposed by [Gar97]. In ther paper, each vertex of the mesh was assocated wth a quadrc error matrx, whch was defned as the squared dstances from ths vertex to the planes of ts adjacent trangulated faces. Every edge had a contracton cost based on ts potental contracton-target vertex and the quadrc error matrces of ts two endponts. Iteratvely the edges wth the mnmum contracton costs were contracted to the optmal target vertex whch could mnmze the contracton costs. The contracton costs for all related vertces would be updated n each teraton. One trend to mprove the mesh smplfcaton method s to reduce the runnng tme. Usng GPU s a common way to speed up the computaton. [Sho13] proposed a CPU-GPU combned algorthm, whch has the lowest computatonal cost compared to all the other methods just runnng on CPU. Some methods lke [Cam13] consdered both accuracy and smplcty to obtan the optmal smplfcaton result. Another multlevel refnement method was proposed n [Mor14], whch combned a Laplacan flow to the hgh resoluton mesh. Ths method can locate the most approprate regons to be contracted n dfferent refnement levels, whch took nto account both accuracy and speed. Another trend to mprove the smplfcaton s to keep more detals. Mesh features are used wdely n the related work. More vertces n the regons wth promnent features wll be preserved. For example, mesh curvature s a common feature used n the pror studes [All03, Lee05, Wan11, Yao15]. [All03] ntroduced curvature drectons to represent the ntrnsc ansotropy of the mesh geometry. In [Lee05], mesh feature was defned as a center-surround operator on Gaussan-weghted mean curvatures. In [Wan11], by measurng the curvature, the authors proposed a method to perform coarse smplfcaton n flat regons and fne smplfcaton near creases and corners respectvely. [Yao15] adopted the dscrete curvature to modfy the Qslm method. Besdes curvature, there are many other salences used for mesh smplfcaton. [Tol08] ntroduced acceleraton and deceleraton of vertces as salency for smplfcaton of dynamc meshes. [Zha12] dentfed vsually mportant regons by ponts samplng method to keep the mesh salency. [Pey14] reduced the number of vertces by Posson dsk samplng based on features such as sharp edges or corners, and re-meshed vertces along the detected feature lnes. In [Pel14], Pellern et al. appled Centrodal Vorono optmzaton to smplfy the mesh and merged features for the mesh wth complex contacts. A B-rep feature-based model was proposed n [Km14]. [Ng14] developed a method called half-edge collapsed scheme. They dentfed vald decmated edges by the length of the edges and the dfference between every two adjacent faces. Another dstnctve method was proposed n [Van15], whch adopted outgong radance functons of the mesh surface as the mesh feature. In [Ďur15], shape dameter functon was adopted as mesh salency to extract skeleton, whch can also be used n mesh smplfcaton. 3 METHODOLOGY Our purpose s to smplfy the human body mesh whle preservng more vertces at jonts and other regons wth promnent non-rgd deformatons. Gven a set of meshes M of varous poses and a template mesh T of the SCAPE data set, we compute the deformaton salency along wth a balancng weght for each vertex. Then we teratvely contract a vald edge on the template mesh to generate a new mesh T n lower resoluton. The ndces of the preserved vertces are recorded. For the other meshes, the vertces wth the same ndces wll be kept, so that the topologes of the trangulated faces of these meshes are the same as that of the smplfed template mesh T. 3.1 Deformaton salency We defne the deformaton salency as the Eucldean dstance of the correspondng vertces between the

3 Fgure 2: The red edge denotes the edge wth the mnmum deformaton weght and contracton cost; t s contracted to a new vertex (the red pont) whch can mnmze the contracton cost; other related vertces are re-connected to the new vertex. Fgure 1: The heat maps of salency values on the template mesh. Red means the hghest value, and blue means the lowest value rgdly transformed template mesh and the mesh n the target pose, whch s also descrbed by the non-rgd deformaton n the SCAPE model. Frst we estmate the rgd transformaton between the template mesh T and each mesh M X of pose X n the pose dataset, whch means to calculate the ntegral rotaton matrx for each body part between mesh T and M X n the global coordnates. In the SCAPE database, vertces have been regstered across dfferent meshes and parttoned nto 16 body parts. For each body part k, the rotaton matrx and translaton vector are denoted as R k and t k, respectvely. To estmate R k and t k, we mnmze the followng energy functon: E trans f orm = argmn R k,t k v T,vM X part(k) R k v T +t k v M X 2 2 (1) In ths least square functon, v T s the th vertex n the mesh T and v M X s the th vertex n the mesh M X. By vectorzng R k and t k, Eq. (1) can be turned nto a system of lnear equatons, whch can be solved analytcally. Wth the rgd transformaton, we can transform each body part of the template mesh va ths equaton: v T X = R k v T +t k,v part(k) (2) where T X s the rgdly transformed template mesh correspondng to the mesh M X. After generatng the rgdly transformed meshes T X n all poses X, we calculate the Eucldean dstance error between every vertex of T X and ts correspondng vertex of M X. For each vertex, we add up ts dstance errors calculated from all the poses X. Then we normalze the total error and take t as the deformaton salency for each vertex. Fg.1 shows the salency values around the human body. The vertex wth hgh value of deformaton salency means there s promnent non-rgd deformaton, so n the smplfcaton ts possblty to be preserved should be larger than other vertces. As the Qslm algorthm llustrated [Gar97], every vertex holds a quadrc matrx Q, whch represents the entre set of planes adjacent to ths vertex. For an edge (v 1,v 2 ), the contracton cost assocated to ts potental target vertex v t s defned as: E contract = v T t (Q 1 + Q 2 )v t (3) In ths equaton, Q 1 and Q 2 are the quadrc matrces of v 1 and v 2, respectvely. v t s the optmal target vertex whch can mnmze the contracton cost E contract. The contracton cost ndcates the sum of squared dstances from v t to the plane set represented by Q 1 and Q 2. Usng a couple of weghts w 1, w 2 to represent the deformaton salences of v 1, v 2, we update the edge contracton cost as: E contract = w 1 + w 2 2 v T t (Q 1 + Q 2 )v t (4) The deformaton weght of an edge s defned as the average weght of ts two attached endponts. However, an extreme result s that too many edges may be contracted n the regons where vertces have low deformaton weghts. In ths case, very long edges may be generated; meanwhle few edges are contracted n the regons where vertces have hgh deformaton weghts. To solve ths dlemma, we ntroduce a balancng weght for each edge, whch s equal to the length of the edge (v 1,v 2 ). The effect of ths balancng weght s to reduce the contractng prorty of long edges. By denotng ths balancng weght as d, the edge contracton cost s rewrtten as: E contract = d(w 1 + w 2 ) 2 v T t (Q 1 + Q 2 )v t (5) Iteratvely, the edge wth the least contracton cost s contracted, generatng the smplfed template mesh T. Fg.2 llustrates edge contracton n a sngle teraton, and Fg.3 compares the smplfed results wth and wthout the balancng tem.

Fgure 3: From left to rght: front sde of the smplfed template mesh wth and wthout balancng tem; back sde of the smplfed template mesh wth and wthout balancng tem.

Ths proves that the balancng tem can prevent too long edges occurrng effectvely. 3.

4 Fgure 3: From left to rght: front sde of the smplfed template mesh wth and wthout balancng tem; back sde of the smplfed template mesh wth and wthout balancng tem. Edges of the meshes are shown explctly. Wth the balancng tem, the varance of edge length s 1.53e-04; wthout t, the varance of edge length s 3.08e-02. Ths proves that the balancng tem can prevent too long edges occurrng effectvely. 3.2 Greedy least dstance mappng Accordng to the SCAPE, to buld the body deformaton model, vertex parttons and face topologes of all the meshes should be the same, and all the meshes should be regstered. So we use a greedy least dstance mappng from T to T to keep vertces regstered across dfferent smplfed meshes. Intally, we defne the smplfed canddates as all vertces of T and the orgnal canddates as all vertces of T. Iteratvely, between the two canddates we fnd the mappng vertex par wth the least Eucldean dstance, record the vertex ndex of ths par, and take them out from the canddates. All vertces of T are mapped to dstnctve vertces of T. Wth the mappng record, we can smplfy other meshes va preservng the vertces wth the same ndces as n the record. As a consequence, all smplfed meshes stll contan the regstered vertces and the same body partton. 4 EXPERIMENTS We conducted the experments on a subset of the SCAPE database. We chose 70 body meshes n dfferent poses of the same person as the pose dataset. Each of them has vertces and trangulated faces. In the 70 meshes, the frst mesh standng n the "A" pose was selected as the template mesh. Based on the pose set, deformaton salency was computed. Wth our method and the compared methods, the template mesh and other meshes were smplfed. The number of faces and vertces were reduced from to 5000 and from to 2500, respectvely. Then the template mesh was deformed to other poses. The errors between the deformed template mesh and the mesh of the target pose were computed to evaluate the proposed methods. Fgure 4: The area heat maps of the smplfed template mesh usng our method (2500 vertces, 5000 faces). Fgure 5: The area heat maps of the smplfed template mesh usng Qslm (2500 vertces, 5000 faces).

5 4.1 Area heat map of the smplfed result The vsualzed comparson between the results of the proposed method and the Qslm method s drawn n Fg.4 and Fg.5, whch are referred to as the heat maps of the area of the trangulated faces. In both fgures, the faces are colorzed accordng to the sze of ther area: smaller faces have warmer color, whle larger faces have cooler color. That s to say, red regons have the hghest vertex densty, and blue regons have the lowest vertex densty. Yellow and green regons have medan vertex densty. By observng the two heat maps, we can fnd that vertex denstes are qute dfferent n the same regon across the results produced by dfferent methods. We know the head, hands, and feet typcally have more geometrc detals. Therefore, the Qslm method preserves more vertces n these regons. On the contrary, t s clear that we preserve more vertces at shoulders, knees, elbows, even n chest and thghs; meanwhle we preserve fewer vertces n the regons lke hands, head and feet. Ths means that we have more vertces to smulate the nonrgd deformaton at jonts and muscular regons. 4.2 Deformaton errors In ths secton, we measure the deformaton errors between the deformed template mesh and the mesh of the target pose. Less deformaton error means that the smplfcaton method s more sutable for such deformable human body mesh to change the body pose. The detal of the body pose deformaton can be referred to [Ang05]. Besdes the 70 meshes smplfed by our method and the Qslm, we also smplfy 10 meshes usng the method provded n the open-source CGAL lbrary for comparson. The CGAL ( s a wdely-used lbrary of geometry algorthms. We ntroduce two crtera to measure the deformaton errors: 1. Vertex-to-face error (v2f) The vertex-to-face error measures the average squared dstance between the deformed template mesh and the smplfed mesh of the target pose, from a vertex to the closest face. It s also adopted by the Qslm method [Gar97]. The vertex-to-face error E = (M,M n ) between the deformed template M and ground-truth M n s defned as: E = 1 X + X n ( ) d1(v,m 2 n ) + d1(v,m 2 ) v X v X n (6) where X, X n are the pont clouds n the mesh M and M n respectvely. The dstance functon d 1 (v,m) = mn p M v p s the mnmum Eucldean dstance from the vertex v to the closest face p on the mesh M. The measurement unt of ths error s meter 2. Method v2f error v2v error Ours(70 poses) 1.08e e-05 Qslm(70 poses) 1.12e e-05 CGAL(10 poses) 1.10e e-05 Table 1: The average deformaton errors of our method, Qslm and CGAL. 2. Vertex-to-vertex error (v2v) The vertex-to-vertex error measures the average squared dstance from a vertex of the deformed template mesh to the closest vertex of the smplfed mesh of the target pose. Ths metrc s adopted n [Bog15]. Based on the prevous notatons, the vertex-to-vertex error E = (M,M n ) s defned as: E = 1 X v X d 2 2(v,M n ) (7) The dstance functon d 2 (v,m) = mn u M v u s the mnmum Eucldean dstance from the vertex v to the closest vertex u on the mesh M. The measurement unt s also meter 2. The comparson results between our method and other methods are shown n Table 1. The average errors show that our method can reduce the deformaton errors n both metrcs. The reducton of the v2v error s more sgnfcant, whch approaches about 8 percent compared to the Qslm. The results ndcate that usng our method, the smplfed human mesh can be deformed more accurately than usng the Qslm and CGAL. Fg.6 shows some smplfed results of dfferent denttes wth the same pose, and Fg.7 shows the pose deformaton results after smplfcaton for several poses. 5 CONCLUSION In ths paper, we propose a mesh smplfcaton method usng deformaton salency for 3D human pose deformng. Based on SCAPE dataset, we compute the vertex dstance error caused by non-rgd deformaton for each vertex as the deformaton salency. In the template body mesh we contract the edges wth the lowest salency n pror. We also add a balancng weght to avod generatng too long edges caused by over-contracton. Our method shows better performance of body pose deformaton compared to the Qslm and CGAL algorthms. Smlarly, our method can be also appled to smplfyng the meshes of other deformable objects for pose deformaton. 6 REFERENCES [Bog15] Bogo F, Black M J, Loper M, et al. Detaled Full-Body Reconstructons of Movng People from Monocular RGB-D Sequences. Proceedngs of the IEEE Internatonal Conference on Computer Vson, pp , 2015.

Fgure 6: The frst row shows several examples of the hgh resoluton meshes from the SCAPE database (12500 vertces and 25000 faces); the second row shows our smplfed meshes (2500 vertces and 5000 faces).

6 Fgure 6: The frst row shows several examples of the hgh resoluton meshes from the SCAPE database (12500 vertces and faces); the second row shows our smplfed meshes (2500 vertces and 5000 faces). [Baa13] Baak A, Muller M, Bharaj G, et al. A datadrven approach for real-tme full body pose reconstructon from a depth camera. Consumer Depth Cameras for Computer Vson, pp.71-98, [We12] We X, Zhang P, Cha J. Accurate realtme full-body moton capture usng a sngle depth camera. ACM Trans. Graph., 31(6), pp.188, [Gan10] Ganapath V, Plagemann C, Koller D, et al. Real tme moton capture usng a sngle tmeof-flght camera. Computer Vson and Pattern Recognton, pp , [Ang05] Anguelov D, Srnvasan P, Koller D, et al. SCAPE: shape completon and anmaton of people. ACM Trans. Graph., 24(3), pp , [Gar97] Garland M, Heckbert P S. Surface smplfcaton usng quadrc error metrcs. Proceedngs of the 24th annual conference on Computer graphcs and nteractve technques, pp , [Sho13] Shontz S M, Nstor D M. CPU-GPU algorthms for trangular surface mesh smplfcaton. Proceedngs of the 21st nternatonal meshng roundtable, pp , [Cam13] Campomanes-Alvarez B R, Cordon O, Damas S. Evolutonary mult-objectve optmzaton for mesh smplfcaton of 3D open models. 20(4), pp , [Mor14] Morg S, Rucc M. Multlevel mesh smplfcaton. The Vsual Computer, 30(5), pp , [All03] Allez P, Cohen-Stener D, Devllers O, et al. Ansotropc polygonal remeshng. ACM Trans. Graph., 22(3), pp , [Lee05] Lee C H, Varshney A, Jacobs D W. Mesh salency. ACM Trans. Graph., 24(3), pp , [Wan11] Wang J, Wang L, L J, et al. A feature preserved mesh smplfcaton algorthm. Journal of Engneerng and Computer Innovatons, 6, pp , [Yao15] Yao L, Huang S, Xu H, et al. Quadratc Error Metrc Mesh Smplfcaton Algorthm Based on Dscrete Curvature. Mathematcal Problems n Engneerng, [Tol08] Tolgay A. Anmated mesh smplfcaton based on salency metrcs. bilkent unversty, [Pey14] Peyrot J L, Payan F, Antonn M. Alasng-free smplfcaton of surface meshes. Internatonal Conference on Image Processng, pp , [Pel14] Pellern J, Levy B, Caumon G, et al. Automatc surface remeshng of 3D structural models at specfed resoluton: A method based on Vorono dagrams. Computers and Geoscences, 62, pp , [Km14] Km B C, Mun D. Feature-based smplfcaton of boundary representaton models usng sequental teratve volume decomposton. Computers and Graphcs, 38, pp , [Ng14] Ng K W, Low Z W. Smplfcaton of 3D Trangular Mesh for Level of Detal Computaton. Computer Graphcs, Imagng and Vsualzaton, pp.11-16, [Van15] Vanhoey K, Sauvage B, Kraemer P, et al. Smplfcaton of meshes wth dgtzed radance. The Vsual Computer, 31(6-8), pp , [Zha12] Zhao Y, Lu Y, Song R, et al. A salency detecton based method for 3d surface smplfcaton. Acoustcs, Speech and Sgnal Processng, pp , [Ďur15] Ďurkovč R, Madaras M. Controllable Skeleton-Sheets Representaton Va Shape Dameter Functon. Mathematcal Progress n Expressve Image Synthess II, pp.79-90, 2015.

7 Fgure 7: The frst row shows some orgnal meshes of dfferent target poses; the second row shows the smplfed results of these target poses; the thrd row shows the deformed results from the smplfed template mesh to these target poses.

Computer Animation and Visualisation. Lecture 4. Rigging / Skinning

Computer Animation and Visualisation. Lecture 4. Rigging / Skinning Computer Anmaton and Vsualsaton Lecture 4. Rggng / Sknnng Taku Komura Overvew Sknnng / Rggng Background knowledge Lnear Blendng How to decde weghts? Example-based Method Anatomcal models Sknnng Assume