Pose Space Deformation A unified Approach to Shape Interpolation and Skeleton-Driven Deformation

Transcription

1 Pose Space Deformation A unified Approach to Shape Interpolation and Skeleton-Driven Deformation J.P. Lewis Matt Cordner Nickson Fong Presented by 1

2 Talk Outline Character Animation Overview Problem Statement Background Skeleton-Subspace Deformation Shape Interpolation Pose Space Deformation Deformation Model Evaluation Related Works 2

3 Character Animation Overview Animation: To give a soul to a lifeless character 3

4 Character Animation Problem Statement Main Components: 1. Character Model 2. Deformation model 4

5 Character Animation Deformation Model Complexity: vertex: 3 DoFs z y => Reduction of DoFs x 5

6 Character Animation Deformation Model Mapping: Parameters => Deformation 6

7 Joint rotations of a skeleton Character Animation Deformation Model 7

8 Character Animation Deformation Model Emotional Axes X=Sad/Happy Y=Relaxed/ Excited 8

9 Character Animation Deformation Model Reduction of DoFs Reduced Deformation Freedom Pathological Defects Movement Deficiencies Lack of realism Difficulty to control the deformations 9

10 Character Animation Deformation Model Reduction of DoFs Reduced Deformation Freedom Pathological Defects Movement Deficiencies Lack of realism Difficulty to control the deformations Pose Space Deformation solves these problems retaining few DoFs 10

11 Skeleton-Subspace Deformation also called skinning, enveloping Deformation Model Skeleton => Vertex Position 11

12 Skeleton-Subspace Deformation Initial pose 1. Local coordinates v,v 2. Vertex-joint weights: w`, w`` y v v v x 12

13 Skeleton-Subspace Deformation New pose y v = w` T (v`)+ w`` T (v``) T from local to world coordinates v v v x 13

14 Pros Simple Smooth deformations Low memory requirements Real-time deformations Skeleton-Subspace Deformation 14

15 Skeleton-Subspace Deformation Cons Indirect control on deformations through weights Deformation Subspace is limited => Pathological Defects (complex Joint configurations, Collapsing Elbow ) 15

16 Skeleton-Subspace Deformation Collapsing Elbow 16

17 Skeleton-Subspace Deformation Initial pose Frame 1 Frame 2 17

18 Skeleton-Subspace Deformation New pose Frame 1 Frame 2 18

19 Skeleton-Subspace Deformation Cons Indirect control on deformations through weights PSD: direct sculpting Deformation Subspace is limited => Pathological Defects (complex Joints, Collapsing Elbow ) PSD: interpolation of a vast range of deformations 19

20 Shape Interpolation also called shape blending, multi-target morphing Algorithm Input: key shapes S k (same topology) Superposition Anger Disgust Fear S = k w k S k Joy Sadness Surprise 20

21 Shape Interpolation + => neutral smirk half smirk 21

22 Shape Interpolation ½( + )= neutral smirk half smirk 22

23 Shape Interpolation ½( + )= neutral smirk half smirk 23

24 Tony de Peltrie

25 Shape Interpolation Pros Direct manipulation of desired expressions Skin deformation for facial animation (Used in the movies) 25

26 Shape Interpolation Cons Not suited for skeleton-based deformations 26

27 Shape Interpolation Cons Not suited for skeleton-based deformations Storage expensive 27

28 Shape Interpolation Cons Not suited for skeleton-based deformations Storage expensive Conflicting Key Shapes 28

29 Shape Interpolation Conflicting Key Shapes Key Shapes are not independent Superposition a KS Happy + b KS Surprise 29

30 Shape Interpolation Conflicting Key Shapes Key Shapes are not independent Superposition a KS Happy + b KS Surprise + c KS Fear 30

31 Shape Interpolation Conflicting Key Shapes Key Shapes are not independent Superposition a KS Happy + b KS Surprise + c KS Fear 31

32 Shape Interpolation Conflicting Key Shapes Key Shapes are not independent Superposition a KS Happy + b KS Surprise + c KS Fear PSD: interpolation between key Shapes 32

33 Shape Interpolation Cons Not suited for skeleton-based deformations Storage expensive Conflicting Key Shapes Key Shape <=> Animation Parameter 33

34 Shape Interpolation Key Shape Anim. Parameter If new expression is needed => New Key shape & New Parameter S = k w k S k 34

35 Shape Interpolation Key Shape Anim. Parameter If new expression is needed => New Key shape & New Parameter S = k w k S k PSD: no additional parameter freedom in the design of parameter space 35

36 Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Memory High requirements Low requirements Smoothness of Interpolation Not always Yes Direct manipulation Yes No Field of application Facial Deformation Body (skeletoninfluenced) Deformation 36

37 Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Smoothness of Interpolation Not always Yes Direct manipulation Yes No Field of application Facial Deformation Body (skeletoninfluenced) Deformation 37

38 Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Low requirements Smoothness of Interpolation Not always Yes Direct manipulation Yes No Field of application Facial Deformation Body (skeletoninfluenced) Deformation 38

39 Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Low requirements Smoothness of Interpolation Not always Yes If needed Direct manipulation Yes No Field of application Facial Deformation Body (skeletoninfluenced) Deformation 39

40 Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Low requirements Smoothness of Interpolation Not always Yes If needed Direct manipulation Yes No Yes Field of application Facial Deformation Body (skeletoninfluenced) Deformation 40

41 Comparison Table Shape Interpolation Skeletonsubspace Deformation Pose Space Deformation Performance Real-time Real-time Real-time Memory High requirements Low requirements Low requirements Smoothness of Interpolation Not always Yes If needed Direct manipulation Yes No Yes Field of application Facial Deformation Body (skeletoninfluenced) Deformation Facial and Body Deformations 41

42 Character Animation Deformation Model Mapping: Parameters => Deformation 42

43 Character Animation Deformation Model Mapping: Parameters => Deformation Pose Space Deformation Interpolat from deformation examples 43

44 Character Animation Deformation Model Mapping: Parameters => Deformation Pose Space => Displacements Δx Pose Space Deformation Interpolat from deformation examples 44

45 Pose Space Deformation Face Animation 1. Pose Space Ex: Emotion Space 2. Initial Face Model 3. Sculpting of examples with Pose Space Coordinates [Δx: between initial model and examples] 4. Interpolant 45

46 Pose Space Deformation Face Animation 46

47 Pose Space Deformation Body Animation 1. Pose Space Ex: Skeleton Parameters 2. Skeleton & Model 3. Sculpting of examples with Pose Space Coordinates [Δx: between SSD and examples] 4. Interpolant 47 SSD PSD

48 Pose Space Deformation Body Animation 48

49 Interpolant: Interpolat from Examples Pose Space => Displacements Δx from {(PSCoord 1, Δx 1 ),..., (PSCoord N, Δx N )} Scattered Data Interpolation: Interpolation of a set of irregularly located data points Approaches: -Shepard s Method - Radial Basis Function Interpolation 49

50 Shepard s Method dˆ( x) N k = 1 = N w k = 1 k w ( x) d k ( x) k w p ( x) = x, p > k x k 0 d,, K 1 d N are the given data points at x, K 1,x N 50

51 Weight function 51

52 Flat zones 52

53 Average at far points 53

54 Shepard s Method Properties: p Singular at data points ( ) Infinitely differentiable except at data points Partial derivatives are zero at data points => flat zones around data points Far from data points converges to the N N average value dˆ( w x k w ( ) d k k k = 1 k = 1 ) = = N k = 1 k w ( ) ( x) = x with p > 0 k x k N d k x k 54

55 55 Radial Basis Functions also called Hardy s multiquadratics from at solving a system of linear equations: ( ) = = N k k w k d 1 - ) ( ˆ x x x ψ w N w,, 1 K N d d,, 1 K N x,x, 1 K ( ) ( ) ( ) ( ) = N N N N N k N w w d d M L M O M L M x x x x x x x x ψ ψ ψ ψ d Ψ w =

56 Choice of Radial Basis Function ψ () r = r 3 ψ () r = r ψ 2 2 α () r = ( r + σ ), α > 0 linear cubic localized r = x - x k ψ 2 () r = r ln( r) Thin-plate spline function ψ 2 2 β () r = ( r + σ ),0 < β < 1 56

57 Gaussian Radial Basis ψ () r r = exp 2 2σ 2 r = x - x k => dˆ( x) = 2 x - xk w k exp 2 = 1 2σ N k 57

58 Gaussian Basis Function 58

59 Gaussian Radial Basis 59

60 Properties: - smooth - localized: σ ψ Gaussian Radial Basis ( r) = 0 for r => width of falloff is adjustable - relatively fast to compute (table) 2 r () ψ r = exp 2 2σ 60

61 Deformation model: Interpolant: Pose Space Deformation [Summary] Pose Space => Displacements Δx from {(PSCoord 1, Δx 1 ),..., (PSCoord N, Δx N )} dˆ ( x) = N k = 1 w k ψ ( x) 61

62 Workflow l 62

63 For N poses: Preprocessing phase: - NxN Matrix Ψ must be inverted - Matrix-vector multiplication w = Ψ ( component of displaced vertex) Animation phase: dˆ ( x) - Interpolated table lookup d = Ψ w = Ψ = N k = 1 Performance ψ ψ w k ( x - x ) L ψ ( x - x ) ( x - x ) L ψ ( x - x ) ψ 1 1 M 1 k N d ( x - x ) k O N N M 1 N 63

64 Memory Requirements For N poses: Every vertex stores Nx3 weights (N weights component of a vertex) N pose space coordinates of ex. dˆ ( x) = N k = 1 w k ψ ( x - x ) k 64

65 Memory Requirements For N poses: Every vertex stores Nx3 weights (N weights component of a vertex) N pose space coordinates of ex. dˆ ( x) = N k = 1 w k ψ ( x - x ) k 65

66 Memory Requirements For N poses: Every vertex stores Nx3 weights (N weights component of a vertex) N pose space coordinates of ex. dˆ ( x) = N k = 1 w k ψ ( x - x ) k 66

67 Pros & Cons Pros: Wide range of deformations Arbitrary Pose Space Axes Real-time synthesis Relatively simple to implement Control on interpolation ( σ) Direct manipulation (Iterative layered refinement) 67

68 Pros & Cons Cons: Accuracy is reliant on the modeler/animator σ is manually tuned Performance Memory requirements 68

69 Critiques ( T ) 1 d T Least Square Problem w = Ψ Ψ Ψ? 1 w = Ψ d with Ψ regular If N poses then 3 NxN matrices must be inverted for each control vertex? Only one NxN matrix must be inverted Close coordinates in PS results in a numerically unstable matrix Ψ (regularization, TSVD) Ψ 69

70 Related Papers Shape from Examples Sloan, Rose, Cohen (I3D conference 2001) EigenSkin Kry, James, Pai (SIGGRAPH 2002) Skinning Mesh Animation James, Twigg (SIGGRAPH 2005) 70

71 Shape by Examples Sloan, Rose, Cohen Paradigm: Design by Examples Interpolation in Lagrangian form Radial Basis Functions (B-Splines) and Polynomials Interpolation & Extrapolation Reparameterization Applied also to Textures Preprocessing phase: one linear system per example Animation phase: twice faster 71

72 Shape by Examples Sloan, Rose, Cohen 72

73 Paradigm: Design by Examples Articulated characters Deformation field for each Joint support EigenDeformations (Deformation basis) EigenSkin Kry, James, Pai Mapping: Joint => Eigendeformation coordinates 1-D interpolation with Radial Basis Functions No non-linear joint-joint coupling effects Graphical Hardware Optimization Reduction of Memory Requirements Real-time rendering 73

74 EigenSkin Kry, James, Pai 74

75 Skinning Mesh Animation James, Twigg Approximation of sequence of input meshes No Pose Space Skinning algorithm: Proxy bone Transformations (+ weights) automatically approximated Displacement field in rest pose (TSVD) Real-time rendering (Graphical HW support)... more to come 75

76 Question Time 76