Term Project Final Report for CPSC526 Statistical Models of Poses Using Inverse Kinematics

Transcription

1 Term Project Final Report for CPSC526 Statistical Models of Poses Using Inverse Kinematics Department of Computer Science The University of British Columbia Abstract In this project, we built the inverse kinematics(ik) system which can generate some novel and natural poses. Our system consists of 2 components. The first component is the offline training process: we started with reading motion capture data, followed by Principle Component Analysis(PCA) to reduce the dimension of the input data. Then the Gaussian Mixture Model(GMM) is learned to fit the data on the latent space in order to compute the Probability Distribution Function(PDF). Finally, for each pose in the original pose space, there is an associated PDF value for it. The second component of the system is the online optimization process which can be repeatedly used for different PDF s to generate new poses. This system can generate the most likely poses which are similar to the training data and fulfill the user constraints. Different types of input data will lead to different styles of inverse kinematics. We also demonstrat our system in the context of interactive character posing. Keywords: Inverse Kinematics, Motion Capture, Poses, Machine Learning, Principle Component Analysis, Gaussian Mixture Model, Constrained Nonlinear Optimization 1 Introduction Inverse kinematics (IK) is the process of computing the human poses subject to some given data constraints. Without other conditions, this problem is ill-posed in most cases. For example, given the positions of the hands and feet of a character, there are many possible poses that satisfy this constraints. However, some poses are more likely than others. Generally speaking, the likelihood of human poses is relevant to the constrains and the style of motion, which means it s difficult and impractical to manually design the likely human poses. In this project, we present an IK system based on learning from previously-recorded poses. We pose the IK as an optimization problem, which targets to find a maximum of the object function given by the Probability Distribution Function(PDF) of Gaussian Mixture Model(GMM). There are 2 components in our system, the machine learning component and the optimization component. In the machine learning process, we compute GMM model and its PDF based on the input poses data sequence. We firstly read motion capture data to get the skeleton tree structure as well as the related positions and joint angles. Since GMM model doesn t work well on high dimensional space(e.g. 69 dimensions), we used the PCA method to reduce the original mocap data to 2D/3D space. We then learned a GMM over the low dimensional data. This mapping give us the PDF over the vectors representing the different poses. We call this is the offline training process, which means this machine learning process is a preprocess to generate object function. In the optimization process, the objective function describes how desirable the pose is. The bigger the probability, the more likely the new pose is. Our task is to optimize the objective function subject to the constraints given by some user interaction(e.g. fix some joints 3D position, move some joints 3D position). Note that both optimization objective function and the constraints are nonlinear. We call this process as the online optimization process. Our system mostly focuses on the study and research of statistical model and optimization. We proved that our system is able to generate the most likely poses based on the input motion capture data in the context of interactive character posing. Since there are lots of Matlab resources related to motion capture data and optimization algorithm on the internet, we used Matlab to implement our system. There are several limitations and possible improvements to our styled-based IK system. First, our system is not userfriendly enough. Secondly, we cannot provide the IK solution in real time. Thirdly, display of human pose is simple. 2 Related Work There are some early work study on the basic IK problem for finding the pose which satisfies the constraints (e.g. [Bodenheimer et al. 1997] [Girard and Maciejewski 1985]). As

2 mentioned above, most IK problems are ill-posed because the solutions are not unique. Unfortunately, most poses which satisfy the constraints appear unnatural and irregular. In practice, animators usually need to specify many constraints in order to generate a desirable pose. Another category of approaches to keep the animated character natural are based on biomechanics(e.g. measuring the contribution of individual joints, minimizing energy consumption, mass displacement from some initial pose). However, this is still a not good way for describing poses. The most related approach is generating realistic poses from examples (e.g. warp an existing animation [Witkin and Popovic 1995], interpolate between sequences [Rose et al. 1998]). Our system is mainly based on [Grochow et al. 2004]. This article presented an IK system based on a learned probability model of human poses.the main difference between their work and ours is that we used simpler GMM model instead of the Scaled Gaussian Process Latent Variable Model (SGPLVM) to formulate the objective function. [wang et al. 2005] provides good understanding of GPDM model. [Redner and Walker 1984] talks about the optimization problem. They are both very useful to our project. 3 Methods 3.1 Character Model In this section, we presented the structure for our character model. We represented the pose as an skeleton model. This skeleton model is a tree-like model, where we can calculate 3D positions of the joints based on the parent and child information. In our project, the skeleton has 27 joints(figure 1). The skeleton model represents each pose as a 69- dimension vector, which includes the the 3D position of the root(hip), the offset at each joint angle. When calculating the 3D position of one joint, we backtrack to the parent of the joint to get the position and rotation matrix of the parent. We do it recursively until we get to the root. The input of the offline training process is a sequence of certain human motion such as running and walking. Figure 1: Skeleton tree structure. Each node represents one joint, which is labeled by the joint name and joint number. In order to calculate the 3D position of one node, we need its parent node s position, offset and joint angles. 3.2 Pipeline (Figure 2) shows the pipeline of our system. Once again, our system consists of two phases: the offline training process and online optimization process. 3.3 PCA As mentioned above, the poses in the training data set are represented as high dimensional vectors. Before fitting an Gaussian Mixture Model(GMM), we applied PCA as a preprocess on the high dimension data. There are several reasons why we did this. Figure 2: The architecture of our system: top box is the offline training process. We read the motion capture data, applied PCA dimensional reduction on them, then learned to get the GMM model and its PDF; bottom box is the online optimization process. The input is the objective function (based on the GMM model) and the user-specified constraints. The output is the trained vector representing the new pose. 2

3 3.5 Optimization and Pose Synthesis Learning a GMM gives us the objective function which is represented as a PDF of GMM. In order to synthesize a likely pose, the remaining work is to optimize the objective function subject to the constraints. Here is the general form of our optimization statement: Figure 3: Top left: PCA estimated the 2D positions associated with each training pose. Top right The 3 Gaussians generated during the learning process. Bottom left: The likelihood is represented as the 2D GMM probability distribution function. The points in the latent space are superimposed. Bottom right: The 3D illustration of the GMM probability distribution function. First, GMM works well only for low dimensional data. We did experiments on learning GMM on high dimension(69d, 67D) space, and we found that there were lots of artifacts in the final likelihood. Secondly, the optimization process, which is detailed in section 3.5, is more efficient only if the dimensionality of the latent space is low. Thirdly, 2D is more convenient for users to visualize and edit the data. In our system, we reduced the training data to 2D using the PCA functions from the PMTK MATLAB Package. 3.4 Learning a Gaussian Mixture Model Given the data points in the 2D latent space, we used GMM to model the likelihood of certain kind of human motions. More specifically, the model is represented as the probability distribution G that is a linear combination of several Gaussian distributions. G(x) = n a i G i (x; µ i, σ i ) i=1 where G i (x; µ i, σ i ) s are Gaussian probability distribution functions, and a i s are the associated weights. We applied cross validation to decide for each dataset the number of Gaussians in the GMM. We used the well-known Expectation-Maximization (EM) to compute the parameters (a i,σ i,µ i ) of the model. We modified the conventional EM method to compute maximum a posteriori (MAP) estimates for Bayesian inference. Applying MAP helps to prevent the singularity problem. In our system, we used the GMM functions from the PMTK MATLAB Package. s.t. arg max P IK (P CA(q)) q Ceq i (x, y, z) = 0 As for the objective function, the parameter q is a 69- dimension vector in the pose space, which we projected to 2D latent space using PCA. P IK is the GMM PDF over the latent space generated in offline training process. Several constraints comes from the 3D positions of certain joints specified by the users. For example, if the user gives the joint position of LeftHand, then for such 3D position constraint input (xc lh, yc lh, zc lh ), we find its ancestors upward recursively until we reach the root. Then we get the expression of the constraint: Ceq i = x lh xc lh = 0 Ceq i+1 = y lh yc lh = 0 Ceq i+2 = z lh zc lh = 0 For each joint constraint which user specified, we can get 3 constraints equations for x,y and z coordinates. In our experiment, 3-5 constraints should be enough to synthesize a human pose. More constraints are needed if we are given less training data. It this project, we are facing one optimization problem with nonlinear objective function and nonlinear constraints. Since it s difficult to compute the derivative formula of the object function, Steepest Descent or Newton s method cannot work. In our system, we used the FMINCON from the Matlab Optimization Toolbox, to optimize the objective function with nonlinear hard constrains. We used Sequence Quadratic Programming(SQP) algorithm here, which is supported by FMINCON and works well for hard constraints. 4 Results In order to test the effectiveness of the styled-base IK, we tested on the application of interactive character posing. Given a set of motion capture training data, an initial pose and a set of constraints, our system is able to provide the most likely pose which satisfies these constraints (Figure 4). Also, training the model on different input data leads to different style of IK. Since we used the Sequence Quadratic Programming optimization method instead of the gradientbased method mentioned in [Grochow et al. 2004], our method cannot response in real time. In most cases, it takes less than 30 seconds to generate a result. 3

4 5 Discussion and Future Work We have presented one inverse kinematics system based on a learned probability model of human pose. Given a input poses sequence, our system can produce the most likely pose satisfies the user-specified constraints. We demonstrated our system in the context of interactive character posing. we have shown that different style of input motion capture data leads to different pose. Our system runs well over different data sets and we get some good results. The implementation is in Matlab environment, we integrated many techniques and also got some good and smooth intermediate procedure (e.g. PCA, GMM) results. There are some limitations and future work of our project. The user interaction interface is limited by the flexibility of Matlab environment and therefore is not userfriendly. If the system can be built in development environment such as VC++, users can interact with our system more conveniently. The optimization process in our system is not in real time. The SGPLVM model introduced in [Grochow et al. 2004] is more efficient than the GMM we used. The display is simple. In our system, the human pose is displayed as the skeleton. We can improve it by integrating more decent models instead of drawing the poses with simple skeleton lines and nodes. To sum up, the system fulfills the purpose of study on statistical model for inverse kinematics and has a broad range of applications. But There are still possible improvements to our work. Acknowledgements Figure 4: Interactive character posing. Animators can interactively define a character by giving the constraints. Left column: The initial poses and user specified constraints. Right Column: The resulting poses. Top row: We used the running motion capture data. Middle row: We used the walking motion capture data. Bottom row: We used the skipping motion capture data. In each case, the poses are constrained by the positions of limbs and hip. We use red lines and red dots to represent the constraints. Thanks a lot to Prof. Michiel van de Panne for detailed discussion. He also gave us many good ideas. Thanks to Prof. Neil Lawrence s motion capture toolbox for reading and displaying for the motion capture data. The motion capture data we used are from ACCAD motion capture lab. We also want to thank to Prof. Kevin Murphy for its PMTK package as well as his machine learning course. References BODENHEIMER, B., ROSE, C., ROSENTHAL, S., AND PELLA, J The process of motion capture - dealing with the data. In Computer Animation and Simulation 97, GIRARD, M., AND MACIEJEWSKI, A. A Computational modeling for the computer animation of legged figures. In Computer graphics(proc. of SIGGRAPH 85) 19,

5 GROCHOW, K., MARTIN, S. L., HERTZMANN, A., AND POPOVIC, Z Style-based inverse kinematics. ACM Transactions on Graphics (TOG), Proceedings of ACM SIGGRAPH , 3, REDNER, R. A., AND WALKER, H. F Mixture densities, maximum likelihood and the em algorithm. SIAM Review 26, 2, ROSE, C., COHEN, M. F., AND BODENHEIMER, B Verbs and adverbs: Mutidimensional motion interpolation. IEEE Computer Graphics and Applications 18, 5, WANG, J. M., FLEET, D. J., AND HERTZMANN, A Gaussian process dynamical models. In Proc. NIPS 2005, WITKIN, A., AND POPOVIC, Z Motion warping. Proceedings of SIGGRAPH 95,