SCAPE: Shape Completion and Animation of People By Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, James Davis From SIGGRAPH 2005 Presentation for CS468 by Emilio Antúnez nez 1
Motivation It is difficult to get high resolution body scans It is even harder at video rates By building up a human model, you could synthesize a high resolution scan from sparse/incomplete data Accurate model is most easily created by learning from sample scans 2
Pre existing Work in Deformable Human Models I Deformations described relative to a template shape Pose deformations given relative to local joints in an articulated model Body shape deformations described using displacement vectors from PCA 3
Pre existing Work in Deformable Human Models II Pose and shape deformations rarely addressed together Most similar work by Sumner and Popović Retargets pose deformation to another mesh Does not learn a model 4
Paper Contributions Learning an affine deformation model for both pose and shape Shape completion for scan of an arbitrary human target Body shape manipulation for motion capture animation 5
Presentation Overview Data Acquisition Learning the Human Model Applications Shape Completion Motion Capture Animation Limitations 6
Data Format / Assumptions Each input model is a deformation of a fixed topology triangle mesh Models divided into three categories One template model Template subject in different poses Different people in (roughly) same pose Articulated skeleton assigned to each mesh 7
Data Acquisition and Processing 8
Learning the Human Model Pose and shape deformations described per triangle using linear transformations Pose transformations learned from template subject in different poses Body shape transformations learned by comparing different subjects to template 9
Pose Deformation I Rigid (skeletal) deformations are represented separately from non rigid ones Transformations are given in relative coordinate system where one of the corners is fixed at the origin final triangle O R l[k] template triangle Q k 10
Pose Deformation II Triangle edges are not forced to be consistent Final synthesized mesh reduces the least squares error between mesh points and triangle deformations final triangle O R l[k] template triangle Q k 11
Learning Pose Deformation Model I Rigid rotation is known from skeleton Non rigid transformation is underdefined Q matrix is computed by requiring adjacent triangles non rigid transformations to be similar 12
Learning Pose Deformation Model II Non rigid deformation modeled as an affine function of adjacent joint angles In practice, some of the degrees of freedom are removed for constrained joings 13
Pose Deformation Learning Results 14
Body Shape Deformation Body shape is modeled as an additional linear transform, S S is underdetermined (like Q) Again, solved using a smoothness constraint 15
Learning the Shape Deformation Model The matrix coefficients for all body shape transformations are vectorized Principal component analysis is used to parameterize the shape transform vectors 16
Shape Deformation Learning Results 17
Shape Completion I Assuming you know some of the node positions, estimate the others Must estimate pose and body shape This optimization is highly nonlinear in the pose Empirically found that optimizing over all variables at once produces bad results Instead, SCAPE iterates solving 18
Shape Completion II Empirically found that optimizing over all variables at once produces bad results Instead, SCAPE iterates, solving each of these in order: Pose Mesh estimate Body shape Results in a completed mesh and a predicted mesh 19
Partial View Completion Skeletal and point correspondences may be off if too much data is missing Iterate between the shape completion algorithm previously described and remapping the point correspondences 20
Partial View Completion Results 21
Motion Capture Animation Motion capture data provides the pose data Body shape parameters can be set arbitrarily Since markers are generally placed on body surface (not in the bones), mesh is constrained to lie in the space of body shapes encoded by the model 22
Motion Capture Animation Results 23
Limitations Assumes that pose deformation and body shape are mostly independent Models only pose deformations from skeletal motion 24
Conclusion SCAPE learns simple body model which distinguishes pose and body shape deformations Creates reasonable shape completions, even when large features are missing Allows for flexible reconstruction of moving model from motion capture data 25