A statistical model for coupled human shape and motion synthesis

Transcription

1 A statistical model for coupled human shape and motion synthesis Alina Kuznetsova 1, Nikolaus F. Troje 2, Bodo Rosenhahn 1 1 Institute for Information Processing, Leibniz University Hanover, Germany 2 Bio Motion Lab, Queen s University, Kingston, Canada {kuznetso, rosenhahn}@tnt.uni-hannover.de, troje@queensu.ca Keywords: Abstract: Animation, Shape and motion synthesis, Statistical modeling Due to rapid development of virtual reality industry, realistic modeling and animation is becoming more and more important. In the paper, we propose a method to synthesize both human appearance and motion given semantic parameters, as well as to create realistic animation of still meshes and to synthesize appearance based on a given motion. Our approach is data-driven and allows to correlate two databases containing shape and motion data. The synthetic output of the model is evaluated quantitatively and in terms of visual plausibility. 1 INTRODUCTION Emerging interest in 3D technologies and virtual reality introduced a need for realistic computer modeling and animation. It is a rapidly developing area and yet there are many unresolved problems, such as fast and realistic character creation. In this paper, we propose a statistical model to solve the latter problem. In general, character creation is a challenging task. The character generation problem is addressed either by manual editing and 3D modeling or by data-driven approaches; a well-known example of appearance creation is the SCAPE model (Anguelov et al., 25). Motion simulation is usually considered as a separate topic, where three types of methods exist: manual motion editing, physics-based approaches and data-driven approaches. The oldest approach is manual motion editing, such as key frame animation. However, manual editing is not able to provide enough level of motion detailization and is very time-consuming, therefore data-driven approaches emerged recently, as well as physics based and control-based methods. Unfortunately, datadriven approaches usually produce physically incorrect results and the question of fitting generated motion to the concrete character (motion retargeting) is not solved completely; physics-based approaches are usually very computationally-intensive and complex, do not provide enough variability and, as experiments showed, physical fidelity of motion does not imply visual plausibility. Another way to animate a new character is to transfer motion from another character, but when the characters have different parameters (such as proportions, height, etc), such transfer can produce unrealistic results (so-called retargeting problem), as shown on the Fig 1. Here, the same shape was animated using different motions, one of which comes from a person with similar biometric parameters and the other is the motion of the person with completely different biometric parameters. Even from a sequence of images it is possible to see a mismatch between the shape and the motion. The other disadvantage of the approaches described above is that they are difficult to apply, when the task is to animate many characters at once. In this work, we propose a model for character appearance creation and animation by combining a statistical model of human motion with one for character appearance. In this way we address the problems described above. We are able to generate both the character s appearance and motion simultaneously, therefore avoiding the retargeting problem and extensive computations, while still producing visually plausible animations. Semantic parameters, such as weight, height and proportions of the character, serve as a link between shape and motion, and are integrated in our model, allowing excellent control over the generation process. Since our model is stochastic, it can be used for random character generation. Our paper is organized in the following way. In Section 2, we give a short overview of existing methods for character generation and animation. In Section 3, we give technical details about data collection and processing. In Section 4 we explain the model we use for generation. In Section 5, we provide an evaluation of our approach in terms of visual perception

2 (a) Figure 1: Example of motion retargeting problem: (a) red figure unrealistically bends backwards, while green figure has correct balance; (b) red figure has its shoulders set backwards, as is typical for slimmer person. (b) of generated characters and their motions, as well as a quantitative assessment of an appearance-to-motion fit. Motion simulation techniques Many techniques address the motion simulation problem. A purely data-driven approach for motion simulation was introduced by (Wang et al., 28) and is based on Gaussian process models for motion generation; (Li et al., 22) proposed to use linear dynamic systems with distribution of the dynamic system parameters to control the generation; (Brand and Hertzmann, 2) used style machines to create variations in one type of motion. Finally, (Troje, 22) used Principal Component Analysis (PCA) analysis to build a manifold of cyclic motions. Another approach to motion synthesis is in combining already existing motions without usage of any statistical models, for example, in (Sidenbladh et al., 22), no motion is synthesized, but the closest real motion in the database of motions is found based on predefined metrics. Physics-based models of human motion, such as proposed in (Liu et al., 25), are used to make motion physically plausible, which is often required for animation of interactions. More frequently, however, different types of controllers are used to refine already generated motion, for example as proposed in (da Silva et al., 28; Lee et al., 29; Popovic and Witkin, 1999; Sok et al., 27). 2 RELATED WORKS In this section, we revise already existing works on motion and shape modeling. Since standard approaches to this problem separate creation of character and creation of its motion, we first provide a short review of methods proposed for both problems. Mesh simulation techniques The traditional way of creating character appearance is manual modeling. Recently, automated approaches were presented, such as the SCAPE model (Anguelov et al., 25), that is now state-of-the-art technique. A slightly different statistical model is presented in (Hasler et al., 29). Here, the use of semantic handles is also proposed to achieve variability of shapes. Our contribution In contrast to the above mentioned works, where mesh and motion are clearly separated, we propose to combine appearance creation with motion simulation, therefore avoiding many problems of motion retargeting. To our knowledge, there are no previous works bringing motion simulation and shape creation together. We apply the model to generate joint realistic human shape and motion. We also see our contribution in developing a method to find dependencies in two separate databases and building a model, that allows to unite the data from both databases, using the dependencies found. In general, we can imagine many applications of the model, for example, crowd simulation, automatic animation of already existing characters or appearance-from-motion reconstruction. In mathematical sense, our model is inspired by variational models, proposed in (Cootes et al., 21). However, we include a random component and therefore allow for more loose coupling of motion and shape, as well as for variability in sampled characters.

3 (a) Weight (b) Height (c) Age (d) Weight (e) Height (f) Age Figure 2: Distribution of the semantic parameters (weight, height, age) of females in the databases. The upper row shows parameter distribution for the shapes dataset; the lower row shows parameter distribution for the motion dataset. 3 DATA PREPARATION 3.1 Data preprocessing Since our model is purely data-driven, we first give insights of data preprocessing. To train the model we need two sources of data: body scans to learn human appearance, or shape, and motion data, collected during recording session. These two databases contain shapes and motions, recorded from the different people, although the distribution of their semantic parameters are approximately the same. The shape database (Hasler et al., 29) contains body scans of 114 different subjects. Body scans were previously registered, such that aligned meshes, containing N = 12 vertices, were produced. Into each mesh we embed a skeleton (see Fig. 4), consisting of K = 15 joints, and morph meshes in such a way, that the skeletons of these meshes are exactly in the same pose. This is done to exclude variability in shapes due to slight differences in pose. We compute the skeleton s joints positions as a linear combination of nearby vertices: s k = N i=1 ω ik v i, (1) where weighs ω ik were derived manually based on the procedure, that was used to place markers on the body during the creation of the motion database. As a source of motion data, we used the motion database, described in (Troje, 22); the database contains recordings of gait motion of 1 individuals. Firstly, motion was acquired using MoCap system with 45 markers. From 45 physical markers K = 15 virtual markers were derived; these markers correspond to the placement of skeleton joints inside the mesh. For each motion 4 so-called eigenpostures are extracted. Each motion is then represented as a linear combination of eigenpostures, where the coefficients are periodical functions, that depend on time. The walk cycle parameter is individual for each person and therefore is stored separately. That means, that all together each motion is represented with K 3 (4 + 1) + 1 coordinates. Figure 4: Alignment of a skeleton and a mesh. Moreover, for each person semantic data, such as gender, weight, heights, age, was stored. We denote semantic attributes, connected to each recorded shape s or motion m, as v(s) or v(m). Firstly, we separated male from female subjects in both databases, since variation between male and female meshes is much

4 matching scale PCA regression Gaussian model Figure 3: Block diagram of the algorithm; starting from the initial sets of motions (M) and shapes (S), Gaussian model is built. greater then the variation within the female (or male) part of the database itself and such separation makes our model able to better capture variation within both groups. Each group has the same number of members, i.e 55 females and 59 males in the mesh database and 5 females and 5 males in the motion database. The distributions of the semantic parameters of females are presented in Fig. 2. For our experiments, we took weight and height as the most varying parameters across our data sets, but in general any set of parameters could have been taken, as long as it has significant influence on shape appearance and on motion. 3.2 Binding a skeleton to a mesh Since the motion data we obtained is represented using positions of joints depending on time, we chose to use a skeleton-based approach to animate the model. Several methods are proposed to solve the problem of skeleton-based character animation. As mentioned in the previous subsection, we embedded a skeleton into each mesh. To morph a mesh in accordance with a given skeleton position, we chose to use linear skinning (Jacka et al., 27), since the technique is both fast and robust, while providing sufficiently good results. To summarize the implementation, we first assign weights w i j to each vertex of the mesh, that characterize how much bone b j influences the vertex v i. Let a coordinate system be attached to each bone, then transformation matrix T j transforms vertex coordinates from bone coordinate system to world coordinate system. For the new position of the bone, ˆT j denotes the new transformation matrix. Then the new position of the vertex is given by the formula: ˆv i = K j=1 w i j ˆT j T 1 j v i, N i=1 w i j = 1 (2) where K = K 1 is the number of bones (see Fig. 4). The weights are generated using the solver for the heat equation, where heat is propagated from each of the bones (Baran and Popović, 27) until heat equilibrium is reached, and weights are set equal to the normalized temperatures. 4 Appearance morphable model In this section, we explain our shape-motion model and its constraints and limitations. The algorithm for model preparation and training is given in the Fig. 3. The main idea of our model is to correlate shape and motion based on semantic parameters. To achieve better alignment and to avoid the retargeting problem, we first match and align the meshes with the motions. A rough match is done based on semantic parameters, i.e. we create a set of shape-motion pairs (s p,m q ), such that the values of the semantic parameters for each pair differ less than a given threshold ε : P = {(s p,m q ), v(s p ) v(m q ) < ε} (this procedure corresponds to the first block in the Fig. 3). In the next step, we morph each shape s p in the way, that its skeleton corresponds exactly to the skeleton of the motion m q. To implement scaling, we modify Eq. (2) by adding a transformation matrix that represents relative scaling: ˆv i = K j=1 w i j ˆT j S j T 1 j v i, N i=1 w i j = 1, (3) S j = s j s j (4) s j where s j is a scaling factor. After scaling, we fit the skeleton again using Eq. (1). We iterate between these two steps several times before the bone lengths of the newly fitted skelton converge to the desired values. Since the scaling coefficients, i.e. ratio between corresponding bone lengths, are in the interval of [.85,1.15], it does not affect the realistic look of the shapes. After scaling, the bone lengths of the mesh skeleton correspond exactly to those of motion skeleton. Since the dimensionality of the data is still high after this preprocessing step, we first reduce the dimensions by performing Principal Component Analysis (PCA) on both shape and motion coefficients separately and converting our data into PCA space of smaller dimensions. PCA transformation consists of finding orthogonal directions in space (called basis), corresponding to the maximal variance of the sample. Then, given a set of vectors x j X R l, N x size of

5 the set X, and the found basis U x = [u 1 x,...,u k x], the original vectors can represented as: x j = U x ˆx j + µ(x), j = 1...N x (5) where µ(x) = 1 N x N x i=1 X i is the mean of the set X and ˆx j is a vector of coordinates of x j in PCA space, dimensionality of ˆx j is smaller than the dimensionality of x j. Then, ˆx j = U T x (x j µ(x)) (6) is the reverse transformation from ˆx j to x j. To apply PCA on our data, we firstly stretch the matrices, representing shapes and motions, into vectors; therefore we produce two sets: the set of vectors representing shapes S R 3N and the set of vectors representing motion M R 15K+1. Than we perform PCA as described above (see the third block in the Fig 3). We denote the PCA coordinates of smaller dimensionality as ˆm R ˆN and ŝ R ˆK accordingly. Dimensionality of the space is chosen in such a way, that leaves 95% of variance of the sample in both cases. Now we bind mesh and motion coordinates to learn a joint Gaussian distribution over a set [ŝ, ˆm], depending on the semantic parameters v. In this sense, our model is close to Active Appearance Models (AAM, (Cootes et al., 21)), although in contrast to AAM, the relation between two sets of PCA coordinates in our model is probabilistic, i.e. they are not firmly coupled together. Since each pair has common semantic parameters, i.e. to each pair of PCA coordinates ŷ = (ŝ, ˆm) Ŷ R ˆN+ ˆK a vector of semantic parameters v is assigned, we can now derive control handles over our model. For that, we use linear regression in the space of joint PCA coordinates. Since not all coordinates are correlated with semantic parameters, we first perform standard correlation significance analysis and find significant coefficients: I(Ŷ ) r = {i [1,..., ˆN + ˆK] : P(ρ(ŷ i,v) = ) < γ}, where ρ(ŷ i,v) is thecorrelation between the i-th coordinate of vector y and semantic values, γ =.5 is p-value for testing the hypothesis that no correlation exists (Kendall and Stuart, 1973). We then use coordinates from I(Ŷ ) r to build the joint regression model: ŷ I(Ŷ )r = Θv + ε, (7) where Θ is a matrix of regression coefficients and ε N (,Σ I ) is a normally distributed random variable with covariance matrix Σ I. For the rest of the coordinates we assume a joint Gaussian distribution N (,Σ f ), where Σ f is the joint covariance and can be easily estimated from the data. The full joint model (the construction of the joint (a) Figure 5: Two examples of conditional sampling of motion: (a) motion sampled with correlation analysis; (b) motion sampled without correlation analysis. model corresponds to the last two blocks in Fig. 3) is described by the following equations: (b) ŷ = µŷ + Σξ, ξ N (,I), (8) { (Θv) k, i I(Ŷ ), (µŷ ) i = (9), otherwise Here (Θv) k denotes the corresponding coordinate, obtained from Eq. (7). Without loosing generality, we can reorder indices in such a way, that indices in the sets I(Ŝ) r are put in the beginning, followed by indices of the elements, which are independent from semantic parameters. Then, the full covariance matrix can be written as: ( ) ΣI Σ = (1) Σ f Using the proposed model for generating a character based on semantic parameters is straightforward: firstly, semantic parameters are chosen to determine the parameters of Gaussian distribution using Eq (7),(8),(1); then a random point in PCA space is sampled according to a normal distribution with determined parameters. Finally, the coordinates of the point are converted into shape-motion space using Eq (5). Another useful property of the model is the ability to animate a given mesh, i.e. create the corresponding motion, given as well as create a well-fitting appearance, i.e. mesh, based on a motion. Let an input shape s be given. Firstly, the mesh is transformed in PCA space. Secondly, we derive a sampling distribution in the PCA space of ŝ by applying Bayes theorem. We assume normal distributions and therefore are able to sample from the conditional distribution to derive the mesh coordinates, corresponding to the motion and

6 vice verse: P( ˆm ŝ) = P( ˆm,ŝ) P(ŝ). (11) Since we assume a parametrized normal distributions we can write: µ ˆm ŝ = µ ˆm + Σ ˆmŝ Σ 1 ˆm ˆm (ŝ µ ŝ) (12) Σ ˆm ŝ = Σŝŝ Σŝ ˆm Σ 1 ˆm ˆm Σ ˆmŝ (13) ˆm = µ ˆm ŝ + Σ ˆm ŝ ξ, ξ N (,I) (14) Afterwards, we convert the PCA coordinates of the generated motion to the original motion space. One can apply the similar procedure to derive the mesh coordinates from a given motion. As mentioned above, we separate coordinates, correlated with semantic parameters. We do it to avoid false dependencies between coordinates and to be able to use free coordinates to our model to introduce more variability into the generation process. To show the importance of separating correlated coordinates, we perform the following experiment: we divide the databases into two disjoint parts - test and train parts, train our model using train parts of each of the databases and then perform motion sampling conditioned on a mesh from the test part of the mesh database using the model with correlation analysis and without correlation analysis. As can be seen in the Fig. 5, the model that uses all coordinates for the regression (i.e. the model trained without correlation analysis) fails to generate suitable motion, while the model with correlation analysis produces realistic motion. We explain that by the fact, that coordinates of the mesh from the test set possibly are far from the coordinates of the meshes in train set, and therefore false dependencies introduced by regression on all coordinates have strong influence and produce implausible results. 5 EVALUATION As stated above, our model allows to generate highly realistic shape-motion correlation with relatively small afford. In this section, we evaluate simulated sequences in terms of motion fidelity, and also provide quantitative evaluation to show how our approach allows to avoid the motion retargeting problem. There is a number of methods for visual fidelity evaluation proposed in the literature. They can be divided into interrogation approaches and automatic approaches. Interrogation-based approaches can be applied on very different problems. In (Reitsma and Pollard, our model selected mesh selected motion Experiment 1 pair 1 (4, 169) (53, 173) (86, 176) pair 2 (7, 155) (44.7, 15) (75.2, 186) pair 3 (11, 17) (12, 176) (49, 168) Experiment 2 question 1 (9, 176) (12, 176) (86.3, 176) question 2 (45, 169) (49, 169) ( ) question 3 (77.9, 179) (77.9, 176) (77.9, 181) question 4 (75, 178) (75.6, 178) (74.5, 179) Table 1: Semantic parameters (weight in kg., height in cm.) of the samples used in the experiments. 23), experiments with human observers were conducted to determine the sensitivity of human perception to physical errors in motions and (Hodgins et al., 1998) investigated, which types of anomalies added to the motion disturb human perception the most. In (Pražák and O Sullivan, 211) and (McDonnell et al., 28) human crowd variety was investigated and influence of either motion or shape clones on the whole crowd perception was investigated. As an example of the automated motion evaluation approaches, (Ren et al., 25) proposed and evaluated automated data-driven method for assessing unnaturalness of the human motion; in (Ahmed, 24) motion is evaluated in terms of feet sliding and general smoothness, while in (Jang et al., 28) physical plausibility of motion was evaluated. Unfortunately, as also mentioned by (Ren et al., 25), these approaches depend a lot on the type of distortion, applied to motion, and therefore are directed on the evaluation of a specific type of motion generation. Interrogation-based approaches provide more general results and are more reliable and more suitable for our goal. Therefore, we performed two experiments designed as questionnaires and one additional experiment for quantitative evaluation. 13 male and 1 female subjects took part in our experiment. The experimental setup is described below. 5.1 Interrogation-based evaluation Experiment 1 In the first experiment, we asked the participants to compare two animations, one of which was generated using the model and the second one was generated by choosing motion and mesh randomly from our databases. We prepared 3 pairs of videos of 1 s. length. The parameters of the generated (or selected) characters are given in the Table 1. To diminish bias due to greater attractiveness of

7 pair 1 pair 2 pair 3 average positive (%) neutral (%) negative (%) Table 2: Percentage of the participants, that accepted the sequence generated with our model as more realistic, less realistic and percentage of people, having difficulty to evaluate one of the motions as more natural. The perspective and surroundings for the animated figures, as well as textures of the figures were the same. The participants were asked to indicate, which animation from the pairs looked more realistic; they also had the opportunity to state, that both animations looked equally bad or equally good. The results of the experiment are summarized in the Table 2. As the results show, the mesh-to-motion fit delivers in general better results then just combining mesh and motion without taking into account parameters of combined mesh and motion. We explain failure in the second pair of sequences due to generally lesser attractiveness of the appearance sampled from the model (see Fig. 6). In general, we observed, that mismatch between mesh and motion due to difference in weight is a lot easier to detect then mismatch due to difference in height, because in the latter case proportions of the person, unless some extreme cases are taken (e.g. a six-year child and a tall adult) are the same and therefore motion can fit quite well. Some more examples of subjects, sampled using our model, are given in the Fig 7. Taking into account, that during artificial character animation it is usually difficult to find real motion data that fits exactly to the animated shape, our approach is beneficial in terms of delivered result. one figure in comparison to another, we chose appearances, that have approximately the same semantic parameters, e.g. compared two fat humans, or two thin humans etc. (a) weight: 4kg, height: 169cm (b) weight: 61kg, height: 171cm (c) weight: 77kg, height: 178cm (d) weight: 11kg, height: 17cm Figure 7: Examples of simulated characters. (a) Figure 6: Shapes used in the second pair of sequences in experiment 1 for comparison of visual fidelity (a) mesh sampled from the model (b) mesh from the original database. (b) Experiment 2 In the second experiment, we asked our participants to evaluate several sets of videos. In each set, four videos were presented. The videos in each set were produced as follows: A 1 : an animation was generated by matching mesh and motion based on semantic parameters; A 2 : an animation was generated by sampling mesh and motion from the model with the same semantic parameters; A 3 : a mesh was taken from the database of meshes and a motion was sampled from our model given the mesh; A 4 : an animation, where a motion was taken from the database of motion and a mesh was sampled from our model given the motion. The parameters of the samples are given in Table 1. The participants should grade each video with marks from 1 to 4, where 1 means very visually plausible and 4 mean completely unrealistic. In Table 3, the results are given in the form of percentage of participants, that evaluated sequences with 1, 2, 3 or 4 respectively.

8 alg average note A A A A Table 3: Percentage of the participants, given marks from 1 to 4 and mean note for each algorithm. The notes can vary in the interval [1,4], where 1 means very realistic, and 4 means not visually plausible. As the results show, that our model produces slightly better results as match of semantic parameters, which is a doos result given that the two data bases were only matched via semantic parameters during training.the second most realistic animation is delivered by motion sampling, conditioned on the mesh, while appearance sampling (i.e. mesh sampling) based on motion generates the same or even slightly lower results, then semantic matching. We attribute this to the fact that motion still does not provide enough information to sample appearance accurately. We explain the superiority of our sampling methods with several arguments: firstly, mesh and motion sampled with our model are aligned better to each other, and secondly, due to smoothing on the PCA stage, small artifacts, appearing occasionally in matched sequences, are smoothed away, and therefore visually plausible result can be produced. 5.2 Quantitative evaluation There exist several approaches to solve the motion retargeting problem, and none of them is perfect. However, we can avoid this problem by generating mesh and motion simultaneously. We evaluate the meshmotion fit by comparing bone lengths of the mesh skeleton b s j and corresponding bone lengths of the motion skeleton b m j in terms of amount of scaling needed: s j = 1 bs j b m j (15) Here j corresponds to a body part and s j denotes the amount of scaling required to fit mesh to motion. For the motion and mesh from the same person, i.e. when no scaling is required, the amount of scaling equals zero. As shown in the Figure 8, the model captures dependencies in bone lengths in mesh with bone lengths in motion and allows to generate pairs of mesh and motion already scaled properly, so that no scaling is required afterwards. We also evaluate mesh-motion fit in terms of conditional mesh sampling, when motion is given, and conditional motion sampling, when mesh is given. We create pairs from the real data using semantic matching and for the same range of semantic parameters we sample pairs from the model. The results (Fig. 8) confirm, that mesh and motion in the pairs, sampled from the model, are already properly aligned to each other s is scale factor for the bones model gen. data, on sem. param model gen. data, cond. on mesh model gen. data, cond. on motion sem. matched real data hand forearm shoulder torco crus thigh pelvis Figure 8: The mean amount of scaling required for each of the body parts; green color corresponds to the pairs sampled from the model, red color corresponds to the matched pairs. When no scaling is required, the amount of scaling equals. 5.3 Using the model in crowd simulation As mentioned above, we also propose to use the model for automatic character appearance and motion simulation of crowds. We leave out of scope of this paper the question of crowd behavior simulation, since it is an active area of research and numerous approaches exist (see, for example, (Guy et al., 21), (Narain et al., 29)). By controlling the distribution of the semantic parameters, it is possible to generate a set of character meshes, that is specific to the scene. In our example, we generate the crowd by randomly placing sampled characters in space. For creating the crowd in Fig. 9 and 1, we used a uniform distribution of the semantic parameters weight and height. However, one can consider more advanced ways of setting semantic parameters, if some specific distribution of shapes and motions is required. More demonstrations are provided in the supplementary material. 6 Future work As our experiments have shown, there are several limitations of our model. First of all, our model was

9 Figure 9: Example of crowd generation with semantic parameters weight and height uniformly distributed on [4,7] and [15,18] accordingly. The animation itself can be viewed in additional materials. Figure 1: Example of crowd generation with semantic parameters weight and height uniformly distributed on [5,1] and [15,2] accordingly. The animation itself can be viewed in additional materials. applied to a cyclical motion, so its extension to arbitrary type of motion can represent some difficulties. Secondly, the model can be used to generate samples based on semantic parameters, when the semantic parameters are not very far from the range of the parameters used for training. However, when it is not the case, produced results can look unrealistic. We hope, that increasing size and variability of the datatbases will help to generate characters with a wider range of semantic parameters. Therefore, in future we are planning to extend our work on more complex motions, as well as achieve better mesh-motion fitting in the model training phase. More complex motions should firstly be aligned using techniques such as time warping (Hsu et al., 27) and then the similar procedures as in (Troje, 22) should be applied to produce vectorbased representation of motions, suitable for analysis. Another interesting topic is the use of more advanced stochastic models for mesh-motion coordinates coupling to be able to capture non-linear dependencies. 7 CONCLUSION In our work, we proposed a parametrized model combining shape and motion, that can be applied to generate realistic characters together with appropriate motion. While providing enough variability, the model allows tight control over the generation process with the help of semantic parameters, and has a probabilistic formulation. Although the usage of two independent databases can be seen as a weakness of our approach, we want to stress that the method was suggested to find dependencies in the initially unrelated databases and use them to bridge these databases. Such an approach can be advantageous when it is not possible or difficult to collect motion and shape data together from the same subjects. Our model can be used to generate completely new meshes and motion, as well as to generate a specific motion for a given mesh or a specific shape using an existing motion. We also evaluated our model in three experiments, that showed superiority of the proposed model in terms of visual plausibility of created samples in comparison to simple matching based on semantic parameters. We furthermore avoid the retargeting problem, since our model already contains necessary dependencies in it. The model can be easily extended to bigger datasets, since all the algorithms used here in general are computationally inexpensive. Possible applications of our model include animating existing characters, creating appearance based

10 on motion, as well as creating a visually plausible crowd. REFERENCES Ahmed, A. A. H. (24). Parametric synthesis of human animation. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. (25). Scape: shape completion and animation of people. ACM Trans. Graph, 24: Baran, I. and Popović, J. (27). Automatic rigging and animation of 3d characters. In ACM SIGGRAPH 27 papers, SIGGRAPH 7, New York, NY, USA. ACM. Brand, M. and Hertzmann, A. (2). Style machines. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, SIG- GRAPH, pages , New York, NY, USA. ACM Press/Addison-Wesley Publishing Co. Cootes, T. F., Edwards, G. J., and Taylor, C. J. (21). Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell., 23(6): da Silva, M., Abe, Y., and Popović, J. (28). Interactive simulation of stylized human locomotion. In ACM SIGGRAPH 28 papers, SIGGRAPH 8, pages 82:1 82:1, New York, NY, USA. ACM. Guy, S. J., Chhugani, J., Curtis, S., Dubey, P., Lin, M., and Manocha, D. (21). Pledestrians: a least-effort approach to crowd simulation. In Proceedings of the 21 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA 1, pages , Airela-Ville, Switzerland, Switzerland. Eurographics Association. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., p. Seidel, H., and Informatik, M. (29). A statistical model of human pose and body shape. computer graphics forum28. Hodgins, J. K., O Brien, J. F., and Tumblin, J. (1998). Perception of human motion with different geometric models. IEEE Transactions on Visualization and Computer Graphics, 4(4): Hsu, E., da Silva, M., and Popović, J. (27). Guided time warping for motion editing. In Proceedings of the 27 ACM SIGGRAPH/Eurographics symposium on Computer animation, SCA 7, pages 45 52, Aire-la- Ville, Switzerland, Switzerland. Eurographics Association. Jacka, D., Merry, B., and Reid, A. (27). A comparison of linear skinning techniques for character animation. In In Afrigraph, pages ACM. Jang, W.-S., Lee, W.-K., Lee, I.-K., and Lee, J. (28). Enriching a motion database by analogous combination of partial human motions. Vis. Comput., 24(4): Kendall, G. and Stuart, A. (1973). The Advanced Theory of Statistics. Vol.2: Inference and: Relationsship. Griffin. Lee, Y., Lee, S. J., and Popović, Z. (29). Compact character controllers. In ACM SIGGRAPH Asia 29 papers, SIGGRAPH Asia 9, pages 169:1 169:8, New York, NY, USA. ACM. Li, Y., Wang, Y. L. T., and yeung Shum, H. (22). Motion texture: A two-level statistical model for character motion synthesis. In ACM Transactions on Graphics, pages Liu, C. K., Hertzmann, A., and Popovic, Z. (25). Learning physics-based motion style with nonlinear inverse optimization. ACM Trans. Graph, 24: McDonnell, R., Larkin, M., Dobbyn, S., Collins, S., and O Sullivan, C. (28). Clone attack! perception of crowd variety. In ACM SIGGRAPH 28 papers, SIG- GRAPH 8, pages 26:1 26:8, New York, NY, USA. ACM. Narain, R., Golas, A., Curtis, S., and Lin, M. C. (29). Aggregate dynamics for dense crowd simulation. In ACM SIGGRAPH Asia 29 papers, SIGGRAPH Asia 9, pages 122:1 122:8, New York, NY, USA. ACM. Popovic, Z. and Witkin, A. P. (1999). Physically based motion transformation. In SIGGRAPH, pages Pražák, M. and O Sullivan, C. (211). Perceiving human motion variety. In Proceedings of the ACM SIG- GRAPH Symposium on Applied Perception in Graphics and Visualization, APGV 11, pages 87 92, New York, NY, USA. ACM. Reitsma, P. S. A. and Pollard, N. S. (23). Perceptual metrics for character animation: sensitivity to errors in ballistic motion. In ACM SIGGRAPH 23 Papers, pages Ren, L., Patrick, A., Efros, A. A., Hodgins, J. K., and Rehg, J. M. (25). A data-driven approach to quantifying natural human motion. ACM Trans. Graph, 24: Sidenbladh, H., Black, M. J., and Sigal, L. (22). Implicit probabilistic models of human motion for synthesis and tracking. In In European Conference on Computer Vision, pages Sok, K. W., Kim, M., and Lee, J. (27). Simulating biped behaviors from human motion data. ACM Trans. Graph., 26(3). Troje, N. F. (22). Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. In Journal of Vision, volume 2, pages Wang, J. M., Fleet, D. J., and Hertzmann, A. (28). Gaussian process dynamical models for human motion. IEEE Trans. Pattern Anal. Mach. Intell., 3(2): APPENDIX We would like to thank Prof D. Fleet for the very helpful discussions and good advice.