THE PROBLEMATICS OF HUMAN PROTOTYPING AND ANIMATION

Transcription

1 THE PROBLEMATICS OF HUMAN PROTOTYPING AND ANIMATION Nadia Magnenat-Thalmann Centre Universitaire d'informatique Université de Genève, Suisse Daniel Thalmann Laboratoire d'infographie Ecole Polytechnique Fédérale de Lausanne, Suisse Abstract Several ideas and experiments are presented for the creation and realistic animation of three-dimensional scenes involving human beings conscious of their environment. The various approaches should allow the intelligent creation of human beings using prototypes and generate their animation based on mechanics, A. I. and robotics. This paper discusses the problems involved in three major steps of the simulation of human beings: the creation of the human shapes, the motion of the human skeleton, the deformation of the surfaces. Several examples are presented illustrating positional constraints, dynamics, behavioral animation and finite element theory. 1. Introduction One of the main challenges for the next few years is the creation and realistic animation of three-dimensional scenes involving human beings conscious of their environment. This problem should be solved using an interdisciplinary approach and an integration of methods from animation, mechanics, robotics, physiology, psychology, and artificial intelligence. The following objectives should be achieved:. automatic production of computer-generated human beings with natural behavior. improvement of the complexity and the realism of motion; realism of motion needs to be improved not only from the joint point-of-view as for robots, but also in relation to the deformations of bodies, hands and faces during animation.. reduction of the complexity of motion description This paper discusses the problems involved in three major steps of the simulation of human beings:. the creation of the human shapes. the motion of the human skeleton. the deformation of the surfaces

2 2 2. Creation of shapes 2.1 The traditional approach The problem of constructing human characters from a geometrical point of view is mainly a problem of entering free-form shapes. Essentially, two general approaches have been used until now: digitizing methods and parametric-surface approaches. The first class of methods is time-consuming and suffers of a lack of creativity. The second class is convenient for creating human characters 1 except when these characters have to be like well-known personalities. The most direct 3D digitizing technique is simply to enter the 3D coordinates using a 3D digitizer. Another common way of creating 3D objects is by 3D reconstruction from 2D information: e.g. 3D reconstruction from several photographs and 3D reconstruction from a set of serial cross sections. 2.2 New trends: local deformation, inbetweening and 3D portrait-robots The situation may be improved by introducing tools for the creation. Three approaches are possible:. Modification and edition of an existing synthetic actor using local transformations. Generation of new synthetic actors obtained by interpolation between two existing actors. Creation of a synthetic actor by composition of different parts Local transformations of existing synthetic actors A local transformation 2 is a transformation applied to a part of a figure and not the whole as a global transformation. Generally a local transformation consists in 2 steps. First, a region is determined by a selection operation; examples of selections are selection by vertex number, selection inside a volume, selection using color, selection based on set-theory operations applied to selected regions. Then, transformations are applied to regions, e.g. attraction by a vertex, general translation, scale according to a plane. Fig.1 shows how a face may be deformed using such local transformations. Fig.2 shows a synthetic actor improved by local transformations. Fig. 1 Local transformations Fig. 2 A synthetic actor improved by local transformations

3 Shape interpolation This method 2 is mainly used for the creation of human faces. It consists of generating an inbetween human face from two given human faces. The main problem of this method is that both original faces may have different numbers of facets and vertices; because of this, we reorganize both figures by creating a new structure of facets common to both human faces. The technique consists of extracting profiles of a digitized object from selected planes and generating a grid which corresponds to the original object. A correspondence is established between the profiles, then a correspondence between the parallel sections is found using a similar method; the correspondence between points is straightforward. Finally, an inbetween human face is just obtained by linear interpolation D portrait-robots This approach 3 is based on the composition of irregular surfaces stored in a database in one coherent figure; for example, one human face is obtained from a selection of noses, cheeks, mouths, chins etc. Generally figures to be composed do not have the right position, orientation or size for a direct assembly. This implies the use of common elements to both figures to allow them to be assembled. For example, common elements are vertices located on the border of both figures, where they are to be joined; these vertices are called brothers. Because of the shading process, each vertex on the border of the first figure must have a brother on the border of the other figure; otherwise, shading discontinuities will appear. The figures to be composed do not necessarily have the same border. To perform the composition operation, it is necessary to define a master figure and a slave figure and to indicate three non-linear vertices on both borders. The result of the composition of both figures is a new figure which has all vertices of the master figure and all vertices of the slave figure except the vertices of the border, which are replaced by the brothers of the master figure. Translation, rotation and scale operations are performed to make the three non-linear points of both figures correspond. 3. Motion control 3.1 The traditional approach: inbetweening The most common method in computer-assisted animation, called keyframe animation 4, consists of the automatic generation of intermediate frames, called inbetweens, obtained by interpolating a set of key-frames supplied by the animator. A way of producing better images is to interpolate parameters of the model instead of the object itself. This is a popular technique called parametric keyframe animation 5. In a parameter model, the animator creates keyframes by specifying the appropriate set of parameter values, parameters are then interpolated and images are finally individually constructed from the interpolated parameters. For example, to bend an arm of a synthetic actor, it is necessary to enter into the computer the elbow angle at different selected times. Then the software is able to find any angle at any time. Inbetween values are generally calculated using cubic splines.

4 4 3.2 Three steps to automatic motion control In future animation systems, based on synthetic actors, motion control will be automatically performed using A.I. and robotics techniques. In particular, motion will be planned at a task level and computed using physical laws. The simplest automatic control of motion is based on inverse kinematics First step: positional constraints and inverse kinematics Consider the important problem of limb positioning, e.g.: what are the angle values for the shoulder, elbow and wrist if the hand has to reach a certain position and orientation in space? The problem involves the determination of the joint variables given the position and the orientation of the end of the hand with respect to the reference coordinate system. This is the key problem, because independent variables in a human being are joint variables; this problem is well-known in robotics and is called the inverse-kinematics problem. In a typical animation system based on inverse kinematics, the animator specifies discrete positions and motions for end parts; then the system computes the necessary joint angles and orientations for other parts of the body to put the specified parts in the desired positions and through the desired motions. Such an approach works well for simple linkages. However, the inverse kinematic solutions to a particular position become numerous and complicated, when the number of linkages increases. Let us have an example, it is not difficult to determine how much to bend an elbow and a wrist to reach an object with the hand. It is much more difficult if we bring into play the rotation of the shoulder and the flexion of fingers. The transformation problem from Cartesian coordinates has no closed-form solution in general. However, there are a number of special arrangements of the joint axes for which closed-form solutions have been suggested in the context of animation For example to make a synthetic actor sit down on a chair, for example, it is necessary to specify the relevant constraints on the feet, on the pelvis and on the hands. Badler et al. 11 have introduced an iterative algorithm for solving multiple constraints using inverse kinematics. A simple algorithm solving the positional constraint problem has been implemented in our system 12. The animator may impose constraints at the hands, the feet and the pelvis levels. The position and orientation of the hand or the feet may be specified in the local coordinate system attached to the limb (arm or leg), or in the actor system or the world system. A constraint may be a fixed position/orientation or a 6D trajectory. Tools are available for constructing constraints as functions of the actor environment and his envelop (e.g. contact of the foot and the floor). In order to solve the constraints, the system makes use of the position and orientation of the pelvis and the trunk angles (vertebrae and clavicles) for finding the origin of the hips and the shoulders. It then calculates the limb angles required to reach the intended position. In the case where no solution exists, the intended position is projected on the volume of moving of the arm (leg). The skeleton has seven degrees of freedom at the arm (leg) level and the constraint has six degrees (position/orientation). Since the model is redundant from a kinematics point-of-view, this implies the existence of an infinity of solutions to reach the intended position. One solution consists of minimizing the angle variation of the angle between the leg (arm) and the foot (hand). It is also possible to have the user select the solution by giving an opening parameter. The position/orientation/opening constraint allows to select a unique solution from the arm's (or the leg's) seven degrees of freedom. Other criteria such as the collision of the limb with an object may play a role in the selection of the solution.

5 5 Fig. 3 shows an example of animation based on positional constraints. Fig.3 Animation based on positional constraints Second step: motion control using dynamics A more complex, but more realistic approach is based on dynamics. The motion of a synthetic actor is governed by forces and torques applied to limbs. Two problems may be considered: the direct-dynamics problem and the inverse-dynamics problem. The direct-dynamics problem consists of finding the trajectories of some point as the end effector with regard to the forces or the torques that cause the motion. The inverse-dynamics problem is much more useful and may be stated as follows: given a trajectory as well as the forces to be exerted at the manipulator tip, find the torques to be exerted at the joints so as to move it in the desired manner. For a synthetic actor, it is possible to compute the time sequence of joint torques required to achieve the desired time sequence of positions, velocities and accelerations using various methods. As discussed by Arnaldi et al. 13, three main factors lead to introduce dynamics in animation control :. dynamics frees the animator from the description of the motion due to the physical properties of the solid objects.. reality of natural phenomena is better rendered.. bodies can react automatically to internal and external environmental constraints : fields, collisions, forces and torques. In order to apply dynamic analysis, each link of a multibody mechanical system has shape, mass, center of gravity and a matrix of inertia. At each joint between two links, a link moves relative to the other one, using one to six translational or rotational degrees of freedom. Dynamic behaviors can be associated to each degree of freedom : springs and/or dampers which exert internal forces or torques within these joints, actuators which move a body along the corresponding degrees of freedom. Joints may also have limits which keep the degrees of freedom within some points. All these behaviors and limits which act on or react to the motions of the links, are expressed as works that exert upon the system. Techniques based on dynamics have already been used in computer animation , but only for simplified and rigid articulated bodies with few joints, geometrical bodies (cylinders) and without any deformation. Our mechanical approach 13 treats open and closed chains indifferently and it is based on the principle of virtual work 19. In order to save a lot of CPU time, most calculations (Jacobian matrix) are performed using a symbolical way in a preprocessing step. Then, equations are numerically solved for each frame using a simple Newton-Raphson algorithm. For the animation, the motion control is performed by acting on force, torque, spring damper and thrust parameters. Motion is solved using direct dynamics. The drawbacks of this technique are the following :

6 6. It is difficult to adjust the parameters used to simulate the equivalent forces and torques produced by muscle contractions and tensions in an animate figure.. The animator has to adjust the different parameters step by step, after each new set of frames, until he gets the good motion. For example, a comparison has been made between a kinematic-based and a dynamic-based simulation of the writing of a letter. In a kinematics-based method, the hand trajectory may be introduced under the form of kinematic constraints, as described previously. But how can we control the dynamic behavior of the whole arm and the realism of its motion? When we write, it is not the hand trajectory which controls the arm motion but the arm muscles which move the hand. To solve these difficulties, different ways can be explored 7 :. Inverse dynamics with accelerations obtained from keyframing techniques, inverse kinematics or motion recording.. Addition of constraints to the direct dynamics resolution; e.g. energy minimizing or continuity conditions. Application of automatic control theory 20. The use of the dynamics in an animation system of articulated bodies like the human body, provide several important disadvantages:. The animator does not think in terms of forces or torques to apply to a limb or the body in order to perform a motion. The design of a specific user interface is essential.. Dynamics-based animation requires a lot of CPU time to solve the motion equations of a complex articulated body using numerical methods. It considerably reduces the possibility of interaction of the system with the user.. Although dynamics-based motions are more realistic, they are too regular, because they do not take into account the personality of the characters Third step: impact of the environment Adaptive motion control of an actor means that the environment has an impact on the actor motion and conversely. Informations about the environment and the actor must be available during the control process. The purpose of adaptive control motion is to decrease the amount of information entered into the computer by the animator. This is done by using existing informations about the scene and the actor. The system should also have an efficient representation of the geometry of the objects in order to automatically plan tasks as well as prevent collisions. Girard 7 gives a good example of this type of control applied to the motion of humans and animals on a flat terrain. At the low level, the animation is performed on a sequence of key positions of the limbs which define angle trajectories (direct kinematics) or Cartesian positions (inverse kinematics). These trajectories are calculated using optimizing criteria with kinematics or dynamics constraints. The trajectory planning problem is classical and was extensively studied in robotics and Artificial Intelligence. For example, given the starting position of the actor hand

7 7 and objects on a table, the problem is to find the trajectory to follow in order to avoid obstacles. For a synthetic actor, the problem is more complex due to the non-rigidity of the actor. We are currently work on the problem of walking without collision among obstacles. One strategy used is based on Lozano-Perez algorithm 21. A 2D trajectory for the actor is obtained by projecting the octagonal cylinders surrounding the obstacles on the floor and constructing the visibility graph. Then, the obstacles are grown according to a selected projection of the actor. The result trajectory is then used as input to a positional constraint solver based on inverse kinematics Fourth step: behavioral animation Behavioral animation as defined by Reynolds 22 (1987) corresponds to modeling the behavior of characters, from a path planning to complex emotional interactions between characters. The animator is responsible for the design of these behaviors; " his job is somewhat like that of a theatrical director: the character's performance is the indirect result of the director's instructions to the actor. Due to the personality of the character, his reactions may sometimes cause surprises. In an ideal implementation of a behavioral animation, it is almost impossible (as in a theatrical scene) to exactly play the same scene twice. You cannot walk exactly the same way from the same bar to home twice. One current experiment is the design of an animation scene consisting of a small group of people walking together. The problem mainly consists of finding trajectories and using them as inputs to a positional constraint solver based on inverse kinematics. The trajectories are obtained by a behavioral animation module. As in 21, positions, velocities and orientations of the actors are known from the system at any time. The animator may control several global parameters:. weight of the obstacle avoidance component. weight of the convergence to the goal. weight of the centering of the group. weight of the velocity equality. maximum velocity. maximum acceleration. minimum distance between actors 4. Motion-based shape deformation 4.1 The traditional approach: surface deformations Most of the animation of articulated bodies does not take into account the deformation of bodies during motion. However, several authors proposed ways of deforming human faces deformation of human faces for the generation of facial expressions. One way of solving this problem is the abstraction of muscle actions, supported by Abstract Muscle Action 28 procedures (AMA procedures). These procedures are more complex that the single parameter approach and a general muscle approach, because they are very specialized. However, better control of the results may be obtained. AMA procedures work on specific regions of the human face, which must be defined when the face is constructed. AMA procedures are not independent; which means that the order of action of each AMA procedure is very important. In fact, each AMA procedure is responsible for a facial parameter corresponding approximately to a muscle: e.g. vertical jaw, close upper lip, close lower lip, lip raiser etc.

8 8 For the deformation of the bodies, the mapping of surfaces onto the skeleton may be based on the concept of Joint-dependent Local Deformation (JLD) operators 29, which are specific local deformation operators depending on the nature of the joints. These JLD operators control the evolution of surfaces and may be considered as operators on these surfaces. Each JLD operator will be applicable to some uniquely defined part of the surface which may be called the domain of the operator. The value of the operator itself will be determined as a function of the angular values of the specific set of joints defining the operator. Fig.4 shows deformations using JLD operators. Fig. 4 Deformations using JLD operators 4.2 New trends: finite elements and local deformations The environment of characters is made up of physical objects, which should act as if they had a mind. They should react to applied forces such as gravity, pressure and contact. The models recently developed by Terzopoulos et al 30 are for example implemented using the Finite Difference Method, and collisions between elastic objects are simulated by creating potential energy around each object, i.e. intersections between deformable bodies are avoided by surrounding the object surfaces with a repulsive collision force. Platt and Barr 10 also use repulsive forces and discuss constraint methods in terms of animator tools for physical model control. We developed a finite element method 31 to model the deformations of human flesh due to flexion of members and/or contact with objects. The method is able to deal with penetrating impacts and true contacts. For this reason, we prefer to consider true contact forces with possibilities of sliding and sticking rather than only repulsive forces. Simulation of impact with penetration can be used to model the grasping of ductile objects, and requires decomposition of objects into small geometrically simple objects. All the advantages of physical modeling of objects can also be transferred to human flesh 32. For example, we expect the hand grasp of an object to lead to realistic flesh deformation as well as an exchange of information between the object and the hand which will not only be geometrical. This exchange of information using acting and reacting forces is significant for a good and realistic grip and can influence the behavior of the hand and of the arm skeleton. When a deformable object is grasped, the contact forces that act on it and on the fingertips will lead both to deformation of the object and of the fingertips, giving reacting forces which provide significant information about the object and more generally about the environment of the synthetic human body. Once the various kinds of elements are defined, the modeled object shape is obtained by composition. Each element is linked to other elements at nodal points. In continuum mechanics, the equilibrium of a body presenting the same shape can be expressed by using the principle of stationarity of the total potential or the principle of virtual displacements. The equilibrium relation is applied to each element and the whole body is obtained by composing all elements.

9 9 There are several ways to exploit the intrinsic properties of the finite element method:. The decomposition approach can be exploited for modeling penetrating shocks between two or more deformable objects. Each object is subdivided in many deformable sub-objects which are able themselves to interact each other in their turn because each of them brought with him its own properties.. The composition approach can be used for modeling contacts without penetration between two or more objects. In this case, objects can be considered as subobjects evolving independently until a contact is detected and a global object is composed after contact. This process works if we take into account the contact forces, that prevent overlapping, into the problem 33. Fig.5 shows deformations based on finite element theory. Conclusion Fig. 5 Deformations based on finite element theory Several ideas and experiments have been presented for the creation and realistic animation of three-dimensional scenes involving human beings conscious of their environment. Fig. 6 shows the functions of an Intelligent Human Synthesis System based on these principles. Such an approach should allow the intelligent creation of human beings using prototypes and generate their animation based on mechanics, A. I. and robotics.

10 10 PROTOTYPE human prototyping 3D portrait-robots facial inbetweening INTELLIGENT behavioral animation implicit animation task planning INTELLIGENT HUMAN SYNTHESIS ROBOTICS object grasping locomotion obstacle avoidance path planning MECHANICS kinematics dynamics Acknowledgments Fig.6 An approach to the Intelligent Human Synthesis The authors would like to thank all people who participated to these projects. They are especially grateful to the following people: Dr. G. Hégron, Dr. B. Arnaldi and G. Dumont from INRIA, Rennes, Dr. J.P. Gourret from Ecole Nationale Superieure de Physique de Marseille and D. Boisvert, from University of Montreal. References 1 Nahas M, Huitric H, Saintourens M (1987) Animation of a B-spline Figure, The Visual Computer, Vol.3, No4 2 Magnenat-Thalmann N, Thalmann D, Hong MT, de Angelis M (1989) Design, Transformation and Animation of Human Faces, The Visual Computer (to appear) 3 Magnenat-Thalmann N, Thalmann D, Construction and Animation of a Synthetic Actress, Proc. EUROGRAPHICS '88, Nice, France, 1988, pp Burtnyk N, Wein M (1971) Computer-generated Key-frame Animation, Journal of SMPTE, 80, pp

11 11 5 Parke FI (1974) A Parametric Model for Human Faces, PhD dissertation, University of Utah, department of Computer Science 6 Badler NI, Korein JD, Korein JU, Radack GM and Brotman LS (1985) Positioning and Animating Human Figures in a Task-Oriented Environment, The Visual Computer, Vol.1, No4, pp Girard M and Maciejewski AA, Computational Modeling for the Computer animation of Legged Figures, Proc. SIGGRAPH '85, pp Girard M, Interactive Design of 3D Computer-Animated Legged Animal Motion, IEEE CG&A, June 1987, pp Korein JU, BBadler NI, Techniques for Generating the Goal-directed Motion of Articulated Structures, IEEE CG&A, Nov.1982, pp Forsey D and Wilhelms J, Techniques for Interactive Manipulation of Articulated Bodies Using Dynamics Analysis, Proc. Graphics Interface '88, pp Badler NI, Manoochehri KH and Walters G, Articulated Figure Positioning by Multiple Constraints, IEEE CG&A, June 1987, pp Boisvert D, Magnenat-Thalmann N, Thalmann D (1989) An Integrated View of Synthetic Actors, Proc. Computer Graphics International '89 (to appear) 13 Arnaldi B., Dumont G., Hégron G., Magnenat-Thalmann N. and Thalmann D. (1989) Animation Control with Dynamics in: State-of-the-Art in Computer Animation, Springer, Tokyo (to appear) 14 Armstrong WW and Green MW (1985) Dynamics for Animation of Characters with Deformable Surfaces in: N.Magnenat-Thalmann and D.Thalmann (Eds) Computer-generated Images, Springer, pp Armstrong WW and Green M (1985b) The Dynamics of Articulated Rigid Bodies for Purposes of Animation, The Visual Computer, Vol.1, No4, pp

12 12 16 Wilhelms J and Barsky B (1985) Using Dynamic Analysis to Animate Articulated Bodies such as Humans and Robots, in: N.Magnenat-Thalmann and D.Thalmann (Eds) Computer-generated Images, Springer, pp Wilhelms J (1987) Towards Automatic Motion Control, IEEE Computer Graphics and Applications, Vol.7, No 4, pp Isaacs PM and Cohen MF (1987) Controlling Dynamic Simulation with Kinematic Constraints, Bahvior Functions and Inverse Dynamics, Proc. SIGGRAPH'87, Computer Graphics, Vol.21, No4, pp Arnaldi B (1988) Conception du Noyau d'un Système d'animation de Scènes 3D intégrant les Lois de la Mécanique, PhD thesis, University of Rennes I 20 Samson C, Espiau B, Le Borgne M. Robot Control, Oxford University Press, Lozano-Perez T and Wesley MA (1979) An Algorithm for Planning Collision- Free Paths Among Polyhedral Obstacles, Comm.ACM, Vol.22, No10, pp Reynolds C (1987) Flocks, Herds, and Schools: A Distributed Behavioral Model, Proc.SIGGRAPH '87, Computer Graphics, Vol.21, No4, pp Hill DR, Pearce A, Wyvill B (1987) Animating Speech: an Automated Approach Using Speech Synthesised by Rules, The Visual Computer, Vol.3, No6 24 Lewis JP, Parke FI (1987) Automated Lip-synch and Speech Synthesis for Character Animation, Proc. CHI '87 and Graphics Interface '87, Toronto, pp Parke FI (1982) Parameterized Models for Facial Animation, IEEE Computer Graphics and Applications, Vol.2, No9, pp Platt S, Badler N (1981) Animating Facial Expressions, Proc. SIGGRAPH '81, pp

13 13 27 Waters K (1987) A Muscle Model for Animating Three-Dimensional Facial Expression, Proc. SIGGRAPH '87, Vol.21, No4, pp Magnenat-Thalmann N, Primeau E, Thalmann D (1987) Abstract Muscle Action Procedures for Human Face Animation, The Visual Computer, Springer, Vol.3, No 6 29 Magnenat-Thalmann N, Thalmann D (1983) The Direction of Synthetic Actors in the film Rendez-vous à Montréal, IEEE Computer Graphics and Applications, Vol. 7, No Terzopoulos D, Platt J, Barr A, fleischer K (1987) Elastically Deformable Models, Proc.SIGGRAPH'87, Computer Graphics, Vol 21 No 4, pp GourretJP, Magnenat-Thalmann N and Thalmann D (1989) The Use of Finite Element Theory for Simulating Object and Human Body Deformations and Contacts, proc. EUROGRAPHICS '89 (to appear) 32 Gourret JP (1988) Modeling 3D contacts and deformations using finite element theory in synthetic human tactile perception, in: Course notes on synthetic actors by D.Thalmann, N.Magnenat-Thalmann, B.Wyvill and D.Zeltzer, SIGGRAPH '88, pp , 33 Bohm J (1987) A comparison of different contact algorithms with applications, Comp. Structures, Vol 26 No 1-2 pp