Pose Space Deformation A unified Approach to Shape Interpolation and Skeleton-Driven Deformation J.P. Lewis Matt Cordner Nickson Fong Presented by 1
Talk Outline Character Animation Overview Problem Statement Background Skeleton-Subspace Deformation Shape Interpolation Pose Space Deformation Deformation Model Evaluation Related Works 2
Character Animation Overview Animation: To give a soul to a lifeless character 3
Character Animation Problem Statement Main Components: 1. Character Model 2. Deformation model 4
Character Animation Deformation Model Complexity: vertex: 3 DoFs z y => Reduction of DoFs x 5
Character Animation Deformation Model Mapping: Parameters => Deformation 6
Joint rotations of a skeleton Character Animation Deformation Model 7
Character Animation Deformation Model Emotional Axes X=Sad/Happy Y=Relaxed/ Excited 8
Character Animation Deformation Model Reduction of DoFs Reduced Deformation Freedom Pathological Defects Movement Deficiencies Lack of realism Difficulty to control the deformations 9
Character Animation Deformation Model Reduction of DoFs Reduced Deformation Freedom Pathological Defects Movement Deficiencies Lack of realism Difficulty to control the deformations Pose Space Deformation solves these problems retaining few DoFs 10
Skeleton-Subspace Deformation also called skinning, enveloping Deformation Model Skeleton => Vertex Position 11
Skeleton-Subspace Deformation Initial pose 1. Local coordinates v,v 2. Vertex-joint weights: w`, w`` y v v v x 12
Skeleton-Subspace Deformation New pose y v = w` T (v`)+ w`` T (v``) T from local to world coordinates v v v x 13
Pros Simple Smooth deformations Low memory requirements Real-time deformations Skeleton-Subspace Deformation 14
Skeleton-Subspace Deformation Cons Indirect control on deformations through weights Deformation Subspace is limited => Pathological Defects (complex Joint configurations, Collapsing Elbow ) 15
Skeleton-Subspace Deformation Collapsing Elbow 16
Skeleton-Subspace Deformation Initial pose Frame 1 Frame 2 17
Skeleton-Subspace Deformation New pose Frame 1 Frame 2 18
Skeleton-Subspace Deformation Cons Indirect control on deformations through weights PSD: direct sculpting Deformation Subspace is limited => Pathological Defects (complex Joints, Collapsing Elbow ) PSD: interpolation of a vast range of deformations 19
Shape Interpolation also called shape blending, multi-target morphing Algorithm Input: key shapes S k (same topology) Superposition Anger Disgust Fear S = k w k S k Joy Sadness Surprise 20
Shape Interpolation + => neutral smirk half smirk 21
Shape Interpolation ½( + )= neutral smirk half smirk 22
Shape Interpolation ½( + )= neutral smirk half smirk 23
Tony de Peltrie 1985 24
Shape Interpolation Pros Direct manipulation of desired expressions Skin deformation for facial animation (Used in the movies) 25
Shape Interpolation Cons Not suited for skeleton-based deformations 26
Shape Interpolation Cons Not suited for skeleton-based deformations Storage expensive 27
Shape Interpolation Cons Not suited for skeleton-based deformations Storage expensive Conflicting Key Shapes 28
Shape Interpolation Conflicting Key Shapes Key Shapes are not independent Superposition a KS Happy + b KS Surprise 29
Shape Interpolation Conflicting Key Shapes Key Shapes are not independent Superposition a KS Happy + b KS Surprise + c KS Fear 30
Shape Interpolation Conflicting Key Shapes Key Shapes are not independent Superposition a KS Happy + b KS Surprise + c KS Fear 31
Shape Interpolation Conflicting Key Shapes Key Shapes are not independent Superposition a KS Happy + b KS Surprise + c KS Fear PSD: interpolation between key Shapes 32
Shape Interpolation Cons Not suited for skeleton-based deformations Storage expensive Conflicting Key Shapes Key Shape <=> Animation Parameter 33
Shape Interpolation Key Shape Anim. Parameter If new expression is needed => New Key shape & New Parameter S = k w k S k 34
Shape Interpolation Key Shape Anim. Parameter If new expression is needed => New Key shape & New Parameter S = k w k S k PSD: no additional parameter freedom in the design of parameter space 35
Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Memory High requirements Low requirements Smoothness of Interpolation Not always Yes Direct manipulation Yes No Field of application Facial Deformation Body (skeletoninfluenced) Deformation 36
Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Smoothness of Interpolation Not always Yes Direct manipulation Yes No Field of application Facial Deformation Body (skeletoninfluenced) Deformation 37
Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Low requirements Smoothness of Interpolation Not always Yes Direct manipulation Yes No Field of application Facial Deformation Body (skeletoninfluenced) Deformation 38
Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Low requirements Smoothness of Interpolation Not always Yes If needed Direct manipulation Yes No Field of application Facial Deformation Body (skeletoninfluenced) Deformation 39
Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Low requirements Smoothness of Interpolation Not always Yes If needed Direct manipulation Yes No Yes Field of application Facial Deformation Body (skeletoninfluenced) Deformation 40
Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Low requirements Smoothness of Interpolation Not always Yes If needed Direct manipulation Yes No Yes Field of application Facial Deformation Body (skeletoninfluenced) Deformation Facial and Body Deformations 41
Character Animation Deformation Model Mapping: Parameters => Deformation 42
Character Animation Deformation Model Mapping: Parameters => Deformation Pose Space Deformation Interpolat from deformation examples 43
Character Animation Deformation Model Mapping: Parameters => Deformation Pose Space => Displacements Δx Pose Space Deformation Interpolat from deformation examples 44
Pose Space Deformation Face Animation 1. Pose Space Ex: Emotion Space 2. Initial Face Model 3. Sculpting of examples with Pose Space Coordinates [Δx: between initial model and examples] 4. Interpolant 45
Pose Space Deformation Face Animation 46
Pose Space Deformation Body Animation 1. Pose Space Ex: Skeleton Parameters 2. Skeleton & Model 3. Sculpting of examples with Pose Space Coordinates [Δx: between SSD and examples] 4. Interpolant 47 SSD PSD
Pose Space Deformation Body Animation 48
Interpolant: Interpolat from Examples Pose Space => Displacements Δx from {(PSCoord 1, Δx 1 ),..., (PSCoord N, Δx N )} Scattered Data Interpolation: Interpolation of a set of irregularly located data points Approaches: -Shepard s Method - Radial Basis Function Interpolation 49
Shepard s Method dˆ( x) N k = 1 = N w k = 1 k w ( x) d k ( x) k w p ( x) = x, p > k x k 0 d,, K 1 d N are the given data points at x, K 1,x N 50
Weight function 51
Flat zones 52
Average at far points 53
Shepard s Method Properties: p Singular at data points ( ) Infinitely differentiable except at data points Partial derivatives are zero at data points => flat zones around data points Far from data points converges to the N N average value dˆ( w x k w ( ) d k k k = 1 k = 1 ) = = N k = 1 k w ( ) ( x) = x with p > 0 k x k N d k x k 54
55 Radial Basis Functions also called Hardy s multiquadratics from at solving a system of linear equations: ( ) = = N k k w k d 1 - ) ( ˆ x x x ψ w N w,, 1 K N d d,, 1 K N x,x, 1 K ( ) ( ) ( ) ( ) = N N N N N k N w w d d M L M O M L M 1 1 1 1 1 - - - - x x x x x x x x ψ ψ ψ ψ d Ψ w =
Choice of Radial Basis Function ψ () r = r 3 ψ () r = r ψ 2 2 α () r = ( r + σ ), α > 0 linear cubic localized r = x - x k ψ 2 () r = r ln( r) Thin-plate spline function ψ 2 2 β () r = ( r + σ ),0 < β < 1 56
Gaussian Radial Basis ψ () r r = exp 2 2σ 2 r = x - x k => dˆ( x) = 2 x - xk w k exp 2 = 1 2σ N k 57
Gaussian Basis Function 58
Gaussian Radial Basis 59
Properties: - smooth - localized: σ ψ Gaussian Radial Basis ( r) = 0 for r => width of falloff is adjustable - relatively fast to compute (table) 2 r () ψ r = exp 2 2σ 60
Deformation model: Interpolant: Pose Space Deformation [Summary] Pose Space => Displacements Δx from {(PSCoord 1, Δx 1 ),..., (PSCoord N, Δx N )} dˆ ( x) = N k = 1 w k ψ ( x) 61
Workflow l 62
For N poses: Preprocessing phase: - NxN Matrix Ψ must be inverted - Matrix-vector multiplication w = Ψ ( component of displaced vertex) Animation phase: dˆ ( x) - Interpolated table lookup d = Ψ w = Ψ = N k = 1 Performance ψ ψ w k ( x - x ) L ψ ( x - x ) ( x - x ) L ψ ( x - x ) ψ 1 1 M 1 k N d ( x - x ) k O N N M 1 N 63
Memory Requirements For N poses: Every vertex stores Nx3 weights (N weights component of a vertex) N pose space coordinates of ex. dˆ ( x) = N k = 1 w k ψ ( x - x ) k 64
Memory Requirements For N poses: Every vertex stores Nx3 weights (N weights component of a vertex) N pose space coordinates of ex. dˆ ( x) = N k = 1 w k ψ ( x - x ) k 65
Memory Requirements For N poses: Every vertex stores Nx3 weights (N weights component of a vertex) N pose space coordinates of ex. dˆ ( x) = N k = 1 w k ψ ( x - x ) k 66
Pros & Cons Pros: Wide range of deformations Arbitrary Pose Space Axes Real-time synthesis Relatively simple to implement Control on interpolation ( σ) Direct manipulation (Iterative layered refinement) 67
Pros & Cons Cons: Accuracy is reliant on the modeler/animator σ is manually tuned Performance Memory requirements 68
Critiques ( T ) 1 d T Least Square Problem w = Ψ Ψ Ψ? 1 w = Ψ d with Ψ regular If N poses then 3 NxN matrices must be inverted for each control vertex? Only one NxN matrix must be inverted Close coordinates in PS results in a numerically unstable matrix Ψ (regularization, TSVD) Ψ 69
Related Papers Shape from Examples Sloan, Rose, Cohen (I3D conference 2001) EigenSkin Kry, James, Pai (SIGGRAPH 2002) Skinning Mesh Animation James, Twigg (SIGGRAPH 2005) 70
Shape by Examples Sloan, Rose, Cohen Paradigm: Design by Examples Interpolation in Lagrangian form Radial Basis Functions (B-Splines) and Polynomials Interpolation & Extrapolation Reparameterization Applied also to Textures Preprocessing phase: one linear system per example Animation phase: twice faster 71
Shape by Examples Sloan, Rose, Cohen 72
Paradigm: Design by Examples Articulated characters Deformation field for each Joint support EigenDeformations (Deformation basis) EigenSkin Kry, James, Pai Mapping: Joint => Eigendeformation coordinates 1-D interpolation with Radial Basis Functions No non-linear joint-joint coupling effects Graphical Hardware Optimization Reduction of Memory Requirements Real-time rendering 73
EigenSkin Kry, James, Pai 74
Skinning Mesh Animation James, Twigg Approximation of sequence of input meshes No Pose Space Skinning algorithm: Proxy bone Transformations (+ weights) automatically approximated Displacement field in rest pose (TSVD) Real-time rendering (Graphical HW support)... more to come 75
Question Time 76