3D Modeling: Skinning

Transcription

1 3D Modeling: Skinning CITS3003 Graphics & Animation Thanks to both Richard McKenna and Marco Gillies for permission to use their slides as a base. The second half of this lecture was extracted from the MIT EECS Computer Graphics course. Thanks to MIT for providing their teaching materials online.

2 Motivation non-rigid transformations Defining complex 3D objects as a hierarchical data structure allows us to define hierarchical transformations. Hierarchies of transformations allow parts of complex objects to move relative to each other. All vertices in a basic part are transformed the same way Basic parts retain their shape We generally have rigid transformations This works well for objects like tanks with rigid parts But what about human characters and other objects that have parts that rotate and move, but with surfaces that are flexible rather than rigid? 2

3 What is Rigging? Animation is usually specified using some form of lowdimensional controls as opposed to remodeling the actual geometry for each frame. E.g., The joint angles (known as bone transformations) in a hierarchical character determine the pose. Building the controls is called rigging. 3

4 What is Skinning? Skinning provides a means to animate a model while reducing the "folding" and "creasing" of polygons as the model animates A model is represented in two parts: Skin (this is described by the triangular mesh) Skeleton (the underlying bones of the model) 4

5 How is Skinning Implemented? Define a bone hierarchy in a model The mesh (triangle) data is defined separately. The bone hierarchy is linked to the model's mesh, so that animating the bone hierarchy animates the mesh's vertices. Note that multiple bones can affect each vertex in the skin mesh. So, we can have a weighting value for each bone to determine how much influence it has in proportion to the others This allows smooth transitions around bone joints 5

6 Skinning Coloured triangles are attached to one bone (different colour means different bone) Black triangles are attached to more than one bone Note that black triangles are near the joints ACM. All rights reserved. This content is excluded from our Creative Commonslicense. For more information, see James & Twigg, Skinning Mesh Animations, 2005, used with permission from ACM, Inc. 6

7 Smooth Skinning & Bone Weights To make deformations appear smooth, we need to average between the effects of different bones close to joints. Bone weights can be set using modelling software. They are displayed using colours in Blender in weight paint mode: red is for close to 1, blue for close to 0. There are a number of standard ways of setting weights. Automatic geometric weighting (as in the Project) Manual weight-painting (Often used for tweaking) Bone Envelopes which specify a distance that each bone affects. 7

8 Skinning A Character Embed a skeleton into a detailed character mesh Animate the bones by changing the joint angles in each keyframe, then interpolating the keyframes. Bind the skin vertices to bones So, animating skeleton will move the skin with it. Courtesy of Blender Foundation. License CC-BY. This content is excluded from our Creative Commons license. For more information, see 8 Courtesy Robert C. Duvall, Duke University. License CC BY-NC-SA.

9 Hierarchical Model Bind the skin vertices to bones 9

10 Blender Blender is a software application that provides a good interface for 3D modelling, skinning, and rigging. The skeletal model and the polygonal mesh can be defined in Blender. Key poses for animation can be added and exported to files in several formats. Manual weight painting can also be done in Blender. A gingerbread man and the underlying skeletal model. This is the rest pose. 10

11 Bone transformations How are bones moved? each bone has a 4 4 matrix that specifies how it transforms the points and vectors it influences for the current pose the bone transformations are defined relative to a rest pose Transforming a bone has what effect? transforms all children bones (and their children, etc.) transforms corresponding vertices following the bone weights transforms the normal vectors similarly So, to calculate the position and normal for a vertex: the bone transformations affecting the vertex are averaged according to the bone weights, then the resultant transformation is applied to both the position and normal. 11

12 Bone Weights For each vertex p i we ll assign a weight w ij for each bone B j. o Q: How much should vertex p i move with bone j? o w ij = 1 means p i is rigidly attached to bone j. o We can interpret w ij to be the weight being exerted by bone B j on vertex p i thus the term bone weight. Desirable properties for the weight w ij o Usually want weights to be non-negative, i.e., w ij 0 o Also, want the sum over all bones to be 1 for each vertex p i, i.e., we want N w ij = 1 j=1. 12

13 Bone Weights We ll limit the number of bones N that can influence a single vertex o N = 4 bones/vertex is a usual choice o Why 4? You most often don t need many. o Also, storage space is an issue. o In practice, we ll store N pairs of numbers (bone index j, weight w ij ) per vertex. 13

14 Computing Vertex Positions Basic step 1: Transform each vertex p i with each bone j as if it was tied to it rigidly. = T j p i p ij Basic step 2: Then blend the results using the weights. p i = N j=1 w ij p ij where p ij is the vertex i transformed using bone j; T j is the current transformation of bone B j ; p i is the new skinned position of vertex p i. 14

15 Data Structure Required for Skinning The necessary data to define relationship between bone hierarchy and the skin are: Number of vertices Number of bones Vertex Weights Vertex Indices Inverse Matrices Values are extracted from the modeller when loading an animation 15

16 Inverse Bone Matrices (InvBMs) Used to transform the skin data from the rest pose object space (also called skin space) to bone space, i.e., to coordinates relative to the position and orientation of a particular bone. An InvBM is the inverse of the transformation matrix from the root bone in the hierarchy to a particular bone in the hierarchy in the rest/default pose. In the project these are provided by the Open Asset Importer as: bone->moffsetmatrix (see the file gnatidread2.h) 16

17 High-level Skinning Algorithm 1. Transforms vertex positions from their object space for the rest pose (a.k.a., skin space) into bone space using the InvBMs. 2. Performs the key-frame based animation for that frame, which transform the vertex positions back into skin space in its animated pose. 3. The model is then rendered. 17

18 Smooth Skinning On a GPU Smooth Skinning is well suited to implementing as a vertex shader You deform vertices one by one based on the joint transforms Use uniform variables for the transformations of the bones in the current pose and change them each time the model is drawn. You need to get the bone weights for each vertex to the GPU Use attributes, i.e., vertex shader in (or varying ) variables These don t change after the model has been loaded. For the best performance, normally only a small number of bones (e.g., N = 4 as mentioned before) are allowed to affect each vertex. 18

19 Skinning GLSL vertex shader // in variables set for each vertex from the static mesh data in ivec4 BoneIDs; // Integer IDs of the 4 bones affecting this vertex in vec4 BoneWeights; // The corresponding 4 weights between 0.0 and 1.0 const int MAX_BONES = 32; // Adjust based on expected model complexity // the current bone transformations, set each time the model is drawn uniform mat4 BoneTransforms[MAX_BONES]; void main() { // Calculate a weighted average of the given 4 bones transformations mat4 BoneTransform = BoneTransforms[BoneIDs[0]] * BoneWeights[0] + BoneTransforms[BoneIDs[1]] * BoneWeights[1] + vec4 btposition = BoneTransform * vposition; // Transform the position vec4 btnormal = BoneTransform * vnormal; // and the normal // The rest of the vertex shader should use btposition and btnormal 19

20 Skinning C++ initialisation gnatidread2.h for (unsigned int boneid = 0 ; boneid < mesh->mnumbones ; boneid++) { for (unsigned int j = 0 ; j < mesh->mbones[boneid]->mnumweights ; j++) { int VertexID = mesh->mbones[boneid]->mweights[j].mvertexid; float Weight = mesh->mbones[boneid]->mweights[j].mweight; // Insertion sort, keep the 4 largest weights for(int slotid=0; slotid < 4; slotid++) { if(boneweights[vertexid][slotid] < Weight) { for(int shuff=3; shuff>slotid; shuff--) { boneweights[vertexid][shuff] = boneweights[vertexid][shuff-1]; boneids[vertexid][shuff] = boneids[vertexid][shuff-1]; boneweights[vertexid][slotid] = Weight; boneids[vertexid][slotid] = boneid; break; 20

21 Skinning C++ bone transforms // rotindex is for the rotation key just before or at currenttime If (rotindex+1 == channel-> mnumrotationkeys) else { gnatidread2.h currotation = channel->mrotationkeys[rotindex].mvalue; float t0 = channel->mrotationkeys[rotindex].mtime; float t1 = channel->mrotationkeys[rotindex+1].mtime; float wgt1 = (currenttime-t0)/(t1-t0); // Interpolate using quaternions aiquaternion::interpolate(currotation, channel->mrotationkeys[rotindex].mvalue, currotation = currotation.normalize(); channel->mrotationkeys[rotindex+1].mvalue, wgt1); // now build a transformation matrix from it. First rotation, then position. aimatrix4x4 trafo = aimatrix4x4(currotation.getmatrix()); trafo.a4 = curposition.x; trafo.b4 = curposition.y; trafo.c4 = curposition.z; targetnode->mtransformation = trafo; // assign this transformation to the node

22 C++ bone transform composition mat4 bonetransforms[mesh->mnumbones]; for(unsigned int a=0; a<mesh->mnumbones; a++) { const aibone* bone = mesh->mbones[a]; // find the node, looking recursively through the hierarchy for the name ainode* node = scene->mrootnode->findnode(bone->mname); aimatrix4x4 b = bone->moffsetmatrix; // start with the mesh-to-bone matrix // then add the bone s transformation for the current pose by composing all // nodes pose transformations up the parent chain const ainode* tempnode = node; while( tempnode) { A while loop that applies all the parent b = tempnode->mtransformation * b; note transformations tempnode = tempnode->mparent; gnatidread2.h Inverse bone matrix (see slide 16) bonetransforms[a] = mat4(vec4(b.a1, b.a2, b.a3, b.a4), vec4( ), ); gluniformmatrix4fv(ubonetransforms, mesh->mnumbones, GL_TRUE, (const GLfloat *)bonetransforms ); 22

23 Blender and DirectX In Blender, the animated character and its movements can be exported to several formats, including DirectX. 23

24 The DirectX File The DirectX file generated by Blender contains the hierarchical skeletal model and animation of the character. The data structure loaded into OpenGL looks like this (e.g., the.x file for monkey head): Frame Root {... Frame Armature { FrameTransformMatrix {... Frame Armature_torso { FrameTransformMatrix {... Frame Armature_head { FrameTransformMatrix {... Definition of the hierarchical skeletal model Transformation matrices of the different bones of the skeletal model for the rest pose. Note that the definition for head is at one indentation level deeper than torso. torso is the parent bone of head. 24

25 The DirectX File (cont.) Frame Cube { FrameTransformMatrix {... Mesh { 19968; ; ; ;, ;... MeshNormals { 19968;... MeshTextureCoords {... Texture coordinates SkinWeights { "Armature_shoulder_L"; 3314; 256,... Total number of 3D points in the mesh Total number of polygons in the mesh Total number of vertex normals vertex normals x, y, and z components of the 1 st point Same as bone weights When the left shoulder bone moves, 3314 vertices would be affected. The indices of these vertices are listed underneath. The first vertex index is

26 AnimationSet Global { Animation { {Armature_torso AnimationKey { // Rotation The DirectX File (cont.) 0; 80; 0;4; , , , ;;, 1; ;... AnimationKey { // Scale 0; 80; 0; unit quaternion The last part of the DirectX file describes the animation. In this examples, there are 81 frames. The first frame index is 0. The last frame index is 80. The animation is given by the rotation, scaling, translation, etc. of each bone. The first frame index The last frame index 26

27 The DirectX File (cont.) Exercise: Inspect the.x file of your animation created by blender for part 2 of the project. The.x file is a text file. Its content can be viewed using any text editor. 27

28 Blender Learning Resources Making a person Rigging the person (defining bones) While bone weights can be automatically assigned (see above), fine tuning can be done by manual weight painting. Weight Painting 28