计算机图形学. Computer Graphics 刘利刚.

Transcription

1 计算机图形学 Computer Graphics 刘利刚

2 Computer Animation Skinning and Enveloping The slide are from Durand from MIT.

3 Before getting started One more word about rotations 2

4 Extracting rotations We have seen that rotation is best interpolated with axisangle representation or, better, quaternions But often, we are simply given 3x3 or 4x4 matrices that contains rotations, translations, scale, shear, etc. It would be useful to extract the rotation part, do the (complicated) right thing, and interpolate the rest linearly MIT Durand 3

5 Non-orthonormal 3x3 matrix Polar decomposition breaks arbitrary matrix M into Rotation Q (+potential reflection) Symmetric positive definite S (anisotropic scale) Extract rotation part Q, interpolate it using quaternions Interpolate S using some other method. Shoemake, K. and Duff, T Matrix animation and polar decomposition. In Proceedings of the Conference on Graphics interface '92, pp MIT Durand 4

6 Polar Decomposition Algorithm Basic idea: rotations are the matrices such that M -1 =M T Massage the matrix until it satisfies this MIT Durand 5

7 Polar Decomposition Algorithm Given 3x3 Matrix M Compute the rotation factor Q by averaging the matrix with its inverse transpose until convergence: Set Q0 = M, then Qi+1 = 1/2(Qi+ Qi T ) until Qi+1 Qi 0. MIT Durand 6

8 Questions? MIT Durand 7

9 Traditional Animation Draw each frame by hand great control, but tedious Reduce burden with cel animation Layer, keyframe, inbetween, Example: Cel panoramas (Disney s Pinocchio) ACM 1997 Multiperspective panoramas for cel animation 8

10 Traditional Animation Principles The in-betweening, was once a job for apprentice animators. Splines accomplish these tasks automatically. However, the animator still has to draw the keyframes. This is an art form and precisely why the experienced animators were spared the inbetweening work even before automatic techniques. The classical paper on animation by John Lasseter from Pixar surveys some the standard animation techniques: "Principles of Traditional Animation Applied to 3D Computer Graphics, SIGGRAPH'87, pp See also 9

11 Example: Squash and stretch Squash: flatten an object or character by pressure or by its own power Stretch: used to increase the sense of speed and emphasize the squash by contrast 10

12 Example: Timing Timing affects weight: Light object move quickly Heavier objects move slower Timing completely changes the interpretation of the motion. 11

13 See also 12

14 Computer Animation How do we describe and generate motion of objects in the scene? Two very different contexts: Production (offline) Can be hardcoded, entire sequence know beforehand Interactive (e.g. games, simulators) 13

15 Computer Animation How do we describe and generate motion of objects in the scene? Two very different contexts: Production (offline) Can be hardcoded, entire sequence know beforehand Interactive (e.g. games, simulators) Needs to react to user interaction, sequence not known 14

16 Plan Types of Animation (overview) Keyframing (Procedural) Physically-based Animation Controls Character animation using skinning/enveloping CERN 15

17 Types of Animation: Keyframing Specify scene only at some instants of time Generate in-betweens automatically t 4 t 1 t 2 t3 16

18 Types of Animation: Keyframing Specify scene only at some instants of time Generate in-betweens automatically 17

19 Types of Animation: Procedural Describes the motion algorithmically Express animation as a function of small number of parameters Example a clock with second, minute and hour hands express the clock motions in terms of a seconds variable the clock is animated by changing this variable Another example: Grass in the wind, tree canopies, etc. 18

20 Types of Animation: Physically-Based Assign physical properties to objects Masses, forces, etc. Also procedural forces (like wind) Simulate physics by solving equations of motion Rigid bodies, fluids, plastic deformation, etc. Realistic but difficult to control v 0 m g 19

21 Example: Water Simulation Losasso, F., Talton, J., Kwatra, N. and Fedkiw, R., "Two-way Coupled SPH and Particle Level Set Fluid Simulation", IEEE TVCG 14, (2008). 20

22 Another Example Physically-Based Character Animation Specify keyframes, solve for physically valid motion that interpolates them by spacetime optimization Anthony C. Fang and Nancy S. Pollard, Efficient Synthesis of Physically Valid Human Motion, ACM Transactions on Graphics 22(3) , Proc. SIGGRAPH spacetime.html 21

23 Because we are Lazy... Animation is (usually) specified using some form of low-dimensional controls as opposed to remodeling the actual geometry for each frame. Can you think of examples? 22

24 Because we are Lazy... Animation is (usually) specified using some form of low-dimensional controls as opposed to remodeling the actual geometry for each frame. Example: The joint angles (bone transformations) in a hierarchical character determine the pose Example: A rigid motion is represented by changing the object-to-world transformation (rotation and translation). t 1 t 2 t3 t 4 23

25 Because we are Lazy... Animation is (usually) specified using some form of low-dimensional controls as opposed to remodeling the actual geometry for each frame. Example: The joint angles (bone transformations) in a hierarchical character determine the pose Example: A rigid motion is represented by changing the object-to-world transformation (rotation and translation). Blendshapes are keyframes that are just snapshots of the entire geometry. t 1 t 2 t3 t 4 24

26 Example of Higher-Level Controls Ken Perlin s facial expression applet experiments/facedemo/ Lower-level controls are mapped to semantically meaningful higher-level ones Frown/smile etc. 25

27 Building 3D models and their animation controls is a major component of every animation pipeline. Building the controls is called rigging. 26

28 Articulated Character Models Forward kinematics describes the positions of the body parts as a function of joint angles Body parts are usually called bones Angles are the lowdimensional control. Inverse kinematics specifies constraint locations for bones and solves for joint angles. 27

29 Skinning Characters Embed a skeleton into a detailed character mesh Animate bones Epic Games Change the joint angles over time Keyframing, procedural, etc. Bind skin vertices to bones Animate skeleton, skin will move with it 28

30 Skinning Characters Embed a skeleton into a detailed character mesh Animate bones Epic Games Change the joint angles over time Keyframing, procedural, etc. Bind skin vertices to bones Animate skeleton, skin will move with it 29

31 Motion Capture Usually uses optical markers and multiple high-speed cameras Triangulate to get marker 3D position (Again, structure from motion and projective geometry, i.e., homogeneous coordinates) Captures style, subtle nuances and realism But need ability to record someone 30

32 Example: Facial Motion Capture tigerucap07_080806_01.html 31

33 Motion Capture Motion capture records 3D marker positions But character is controlled using animation controls that affect bone transformations! Marker positions must be translated into character controls ( retargeting ) A kind of inverse kinematics!? 32

34 ILM / Walt Disney Pictures 33

35 ILM / Walt Disney Pictures 34

36 Questions? 35

37 Skinning/Enveloping Gears of War 2 Epic Games / ign.com 36

38 Skinning We know how to animate a bone hierarchy Epic Games Change the joint angles, i.e., bone transformations, over time (keyframing) 37

39 Skinning We know how to animate a bone hierarchy Change the joint angles, i.e., bone transformations, over time (keyframing) Epic Games Embed a skeleton into a detailed character mesh Bind skin vertices to bones Animate skeleton, skin will move with it But how? 38

40 Skinning/Enveloping Need to infer how skin deforms from bone transformations. Most popular technique: Skeletal Subspace Deformation (SSD), or simply Skinning Other aliases vertex blending matrix palette skinning linear blend skinning From wikipedia 39

41 SSD / Skinning Each bone has a deformation of the space around it (rotation, translation) 40

42 SSD / Skinning Each bone has a deformation of the space around it (rotation, translation) What if we attach each vertex of the skin to a single bone? 41

43 SSD / Skinning Each bone has a deformation of the space around it (rotation, translation) What if we attach each vertex of the skin to a single bone? Skin will be rigid, except at joints where it will stretch badly 42

44 SSD / Skinning Each bone has a deformation of the space around it (rotation, translation) What if we attach each vertex of the skin to a single bone? Skin will be rigid, except at joints where it will stretch badly Let s attach a vertex to many bones at once! In the middle of a limb, the skin points follow the bone rotation (near-) rigidly At a joint, skin is deformed according to a weighted combination of the bones 43

45 Examples James & Twigg 2005 Colored triangles are attached to 1 bone Black triangles are attached to more than 1 Note how they are near joints 44

46 Examples James & Twigg 2005 Colored triangles are attached to 1 bone Black triangles are attached to more than 1 Note how they are near joints 45

47 Vertex Weights We ll assign a weight wij for each vertex pi for each bone Bj. How much vertex i should move with bone j 46

48 Vertex Weights We ll assign a weight wij for each vertex pi for each bone Bj. How much vertex i should move with bone j wij = 1 means pi is rigidly attached to Bj. 47

49 Vertex Weights We ll assign a weight wij for each vertex pi for each bone Bj. How much vertex i should move with bone j wij = 1 means pi is rigidly attached to bone j. Wang & Phillips

50 Vertex Weights We ll assign a weight wij for each vertex pi for each bone Bj. How much vertex i should move with bone j wij = 1 means pi is rigidly attached to bone j. Weight properties Usually want weights to be non-negative 49

51 Vertex Weights We ll assign a weight wij for each vertex pi for each bone Bj. How much vertex i should move with bone j wij = 1 means pi is rigidly attached to bone j. Weight properties Usually want weights to be non-negative Also, want the sum over all bones to be 1 for each vertex This means translation independence. (Again, a partition of unity remember splines basis functions?) 50

52 Vertex Weights cont d We ll assign a weight wij for each vertex pi for each bone Bj. How much vertex i should move with bone j wij = 1 means pi is rigidly attached to bone j. We ll limit the number of bones N that can influence a single vertex N=4 bones/vertex is a usual choice Why? 51

53 Vertex Weights cont d We ll assign a weight wij for each vertex pi for each bone Bj. How much vertex i should move with bone j wij = 1 means pi is rigidly attached to bone j. We ll limit the number of bones N that can influence a single vertex N=4 bones/vertex is a usual choice Why? You most often don t need very many. Also, storage space is an issue. In practice, we ll store N (bone index j, weight wij) pairs per vertex. 52

54 How to compute vertex positions? 53

55 Linear Blend Skinning Basic Idea 1: Transform each vertex pi with each bone as if it was tied to it rigidly. 54

56 Linear Blend Skinning Basic Idea 1: Transform each vertex pi with each bone as if it was tied to it rigidly. Basic Idea 2: Then blend the results using the weights. 55

57 Computing Vertex Positions Basic Idea 1: Transform each vertex pi with each bone as if it was tied to it rigidly. Basic Idea 2: Then blend the results using the weights. p ij = T j p i p i = j w ij p ij p ij is the vertex i transformed using bone j. Tj is the current transformation of bone j. p i is the new skinned position of vertex i. 56

58 Computing Vertex Positions Rest ( bind ) pose Bone 1: T1 Bone 2: T2 Vertex p0 has weights w01=0.5, w02=0.5 p 0 Skin 57

59 Computing Vertex Positions Rest ( bind ) pose Bone 1: T1 After rotations Bone 1: T 1 p 0 Bone 2: T2 Skin Bone 2: T 2 Vertex p0 has weights w01=0.5, w02=0.5 Transform by T 1 and T 2 yields p 01, p 02 p 01 p 02 58

60 Computing Vertex Positions Rest ( bind ) pose Bone 1: T1 After rotations Bone 1: T 1 p 0 Bone 2: T2 Skin Bone 2: T 2 Vertex p0 has weights w01=0.5, w02=0.5 Transform by T 1 and T 2 yields p 01, p 02 the new position is p 0= 0.5*p 1 + p 0 p 01 p *p 2 59

61 Computing Vertex Positions Rest ( bind ) pose Bone 1: T1 Bone 2: T2 After rotations Bone 1: T 1 p 0 p 0 Skin Bone 2: T2 Skin p 01 p 02 Vertex p0 has weights w01=0.5, w02=0.5 Transform by T 1 and T 2 yields p 01, p 02 the new position is p 0= 0.5*p *p 2 60

62 SSD is Not Perfect q 0 After rotations p 0 61

63 Questions? Remedy Entertainment, Microsoft Game Studios, IGN.com 62

64 Bind Pose We are given a skeleton and a skin mesh in a default pose Called bind pose Undeformed vertices pi are given in the object space of the skin a global coordinate system, no hierarchy 63 okino.com

65 Bind Pose We are given a skeleton and a skin mesh in a default pose Called bind pose Undeformed vertices pi are given in the object space of the skin Previously we conveniently forgot that in order for p ij = Tj pi to make sense, coordinate systems must match up. 64 okino.com

66 Coordinate systems Undeformed vertices pi are given in the object space of the skin Tj is in local bone coordinate system according to skeleton hierarchy 65

67 Bind Pose cont d In the rigging phase, we line the skeleton up with the undeformed skin. This gives some rest pose bone transformations Bj from local bone coordinates to global Bj concatenates all hierarchy matrices from node j up to the root B n... B 2 B 1 66 okino.com

68 Bind Pose cont d When we animate the model, the bone transformations Tj change. 67 okino.com

69 Bind Pose cont d When we animate the model, the bone transformations Tj change. What is Tj? It maps from the local coordinate system of bone j to world space. again, concatenates hierarchy matrices 68 okino.com

70 Bind Pose cont d When we animate the model, the bone transformations Tj change. What is Tj? It maps from the local coordinate system of bone j to world space. To be able to deform pi according to Tj, we must first express pi in the local coordinate system of bone j. This is where the bind pose bone transformations Bj come in. 69 okino.com

71 Bind Pose cont d To be able to deform pi according to Tj, we must first express pi in the local coordinate system of bone j. This is where the bind pose bone transformations Bj come in. p ij = T j B 1 p j i This maps pi from bind pose to the local coordinate system of bone j using B -1 j, and then to world space using Tj. 70 okino.com

72 Bind Pose cont d p ij = T j B 1 p j i This maps pi from bind pose to the local coordinate system of bone j using B -1 j, and then to world space using Tj. What is Tj B -1 j? It is the relative change between the bone transformations between the current and the bind pose. 71

73 Bind Pose cont d p ij = T j B 1 p j i What is the transformation when the model is still in bind pose? This maps pi from bind pose to the local coordinate system of bone j using B -1 j, and then to world space using Tj. What is Tj B -1 j? It is the relative change between the bone transformations between the current and the bind pose. 72

74 Bind Pose cont d p ij = T j B 1 p j i This maps pi from bind pose to the local coordinate system of bone j using B -1 j, and then to world space using Tj. What is Tj B -1 j? It is the relative change between the bone transformations between the current and the bind pose. What is the transformation when the model is still in bind pose? The identity! 73

75 Questions? 74

76 Bind pose & weights We then figure out the vertex weights wij. How? Usually paint by hand! We ll look at much cooler methods in a while. B n... B 2 B 1 75 okino.com

77 Questions? 76

78 Skinning Pseudocode Do the usual forward kinematics maybe quaternion interpolation for rotations get a matrix T j (t) per bone (full transformation from local to world) For each skin vertex p i p i = j w ij T j (t)b 1 p j i 77

79 Skinning Pseudocode Do the usual forward kinematics maybe quaternion interpolation for rotations get a matrix T j (t) per bone (full transformation from local to world) For each skin vertex p i p i = j w ij T j (t)b 1 p j i Do you remember how to treat normals? 78

80 Skinning Pseudocode Do the usual forward kinematics maybe quaternion interpolation for rotations get a matrix T j (t) per bone (full transformation from local to world) For each skin vertex p i p i = j w ij T j (t)b 1 p j i Inverse transpose for normals! n i = j w ij T j (t)b 1 j Tni 79

81 Skinning Pseudocode Do the usual forward kinematics For each skin vertex p i p i = j w ij T j (t)b 1 p j i Note that the weights & bind pose vertices are constant over time Only matrices change (small number of them, one per bone) This enables implementation on GPU vertex shaders (little information to update for each frame) 80

82 Questions? 81

83 Hmmh... This is what we do to get deformed positions p i = j w ij T j (t)b 1 p j i 82

84 Hmmh... This is what we do to get deformed positions p i = j w ij T j (t)b 1 p j i But wait... p i = j w ij T j (t)b 1 j p i 83

85 Hmmh... This is what we do to get deformed positions p i = j w ij T j (t)b 1 p j i But wait... p i = j w ij T j (t)b 1 j p i This is exactly what I warned you of on Thursday: blending matrices MIT entry-by-entry EECS Durand (!!!) 84

86 Indeed... Limitations Rotations really need to be combined differently (quaternions!) From: Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation J. P. Lewis, Matt Cordner, Nickson Fong 85

87 Usual Solution Have the artists deal with it O:-) In practice, build a rig that has extra bones near joints that move in sync to counter the artifacts. Tedious, but this is what people often do. Need sophisticated animation controls to drive this. Cool paper that does this automatically: Kavan, Collins, O Sullivan: Automatic Linearization of Nonlinear Skinning, I3D

88 Back to quaternions! Dual quaternions! ToG 2008 Fredo Durand MIT EECS

89 Dual numbers kind of like complex numbers, vector space a dual number â=a0+εaε where a0 and aε are scalar and ε 2 =0 then you can deduce multiplication rules by linearity Fredo Durand MIT EECS

90 Dual quaternions ^ q=q0+εqε 8 dimensional! Can represent rigid transformations naturally rotation + translation Does the right interpolation when the axis of rotation translates Fredo Durand MIT EECS

91 People Have Thought of This

92 Real-time enveloping with rotational regression Wang, Pulli, Popovic We learn a fast model from exported examples. Black Box Simulation Exported Examples (skeleton-mesh pairs) Our method Fast Model Slide from Rob Wang

93 Figuring out the Weights Usual approach: Paint them on the skin. Can also find them by optimization from example poses and deformed skins. Wang & Phillips, SCA

94 Super Cool: Automatic Rigging When you just have some reference skeleton animation (perhaps from motion capture) and a skin mesh, figure out the bone transformations and vertex weights! Ilya Baran, Jovan Popovic: Automatic Rigging and Animation of 3D Characters, SIGGRAPH ~ibaran/autorig/ 93

95 First fully automatic rigging algorithm Input Processing Output 3D Character Rigged Character Generic Skeleton Pinocchio Slide from Ilya Baran

96 Fredo Durand MIT EECS

97 The Other Direction When you have no skeleton, but a source animation for the full mesh (not so common) 96

98 97

99 Q&A