3D Character Animation Synthesis From 2D Sketches

Transcription

1 3D Character Animation Synthesis From 2D Sketches Yi Lin University of Waterloo Abstract Traditional character animation has superiority in conveying stylized information about characters and events, but producing it requires a lot of labor and time. Computer generated animation improves greatly on efficiency, but is poor in expressing stylized motion. In this paper, we propose a sketching-based animation synthesis system. The system contains an interface for the user to draw the sketches or load sketch images. Then the system extracts 2D pose from the input strokes and maps the 2D pose to 3D pose that is in a motion capture database. During the mapping, a series of matching 3D pose candidates are found. The user can select the most satisfying candidate as the 3D key pose for the later animation synthesis. The system synthesizes an animation based on the 3D key poses by finding a path in the motion capture database and generating transition motions if needed. CR Categories: H.5.2 [Information Systems]: Information Interfaces and Presentation User Interfaces I.3.6 [Computing Methodologies]: Computer Graphics Methodology and Techniques; I.3.7 [Computing Methodologies]: Computer Graphics Three-Dimensional Graphics and Realism; Keywords: sketching-based interface, animation synthesis 1 Introduction In traditional hand-drawn character animation, animators can draw natural or exaggerate motions freely. They often quickly draw some thumbnails of a character as the key poses to capture the characters overall motions [Blair 1994]. The characters are drawn as stick figures, simple rectangular or ellipsoidal volumes. Once a coarse version is on paper, they refine the key poses and fill in the in-between poses to eventually produce the final animation. This coarse-to-fine process is also common in 3D computer animation [Lasseter 1994]. If the key poses and the motion path are given, most of work can be done automatically by computers. The user can interact with any part of the animation. But Setting up appropriate poses is timeconsuming. It is very hard for current animation systems to create any free-style motion. Creating free-style 3D character animations, like cartoons, is timeconsuming and difficult for unexperienced animators who only have an idea and some sketches. Without considering enough details, it is likely that the final animation is not what is in their minds. If this happens, they may have to give up most of work. This is one of the biggest frustrations for new animators. They prefer to get an animation containing their sketches, while not considering details y9lin@cs.uwaterloo.ca Copyright 2006 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) or permissions@acm.org. GRAPHITE 2006, Kuala Lumpur, Malaysia, November 29 December 02, ACM /06/0011 $ as current animation systems requiring they to do. Based on this animation, they can make further refinement as they like. This is an easy and fast way for these kinds of users to create free-style animation. In this paper, we introduce such a system that produces an animation based on a few sketches with a little of user interaction. We present an interface that allows the user to draw their sketches or to load the sketch images. The system uses the sketches to their corresponding 3D poses of an articulated figure. Since sketches of arbitrary style would be very difficult to automatically parse, our interface requires the user to indicate the skeleton of the sketch by drawing specific strokes. We call this skeleton a 2D key pose. In order to generate natural animation, we use motion capture data from a motion capture database as the source to synthesize the animation. We develop an efficient searching strategy to find the 3D pose in the database that looks most similar to the 2D key pose. We called these frames 3D key poses. We connect all the 3D key poses in time sequence by finding an optimal path in the motion capture database. The contribution of the paper is that we develop a complete system that generates 3D character animation from 2D sketches. It takes the advantage of 2D sketching animation and 3D motion capture data. Compared with previous work, our system does not require the 2D and 3D skeletons have the same structure. That means only motion styles are transferred. The system provides an easy-using sketching interface for both ordinary users and experienced animators. The processing procedure is consistent with traditional animation procedure. It can be used in the real animation projects. 2 Related Work Traditionally-drawn keyframing method is to create animations from sequentiallydrawn sketches of a character. With appropriate constraints, a 3D character pose can be inferred for each handdrawn frame [Davis et al. 2003]. The 2D stylus-based input aims to exploit drawing skills and not acting skills. Igarashi et al. [Igarashi et al. 1999; Igarashi and Hughes 2003] presented a sketching interface for designing freeform models such as stuffed animals and other rotund objects.the system inflates the sketch contour to a 3D rotund model, which does not need sketches from many views. Rademacher [Rademacher 1999] proposed an method to reconstruct a 3D model from an arbitrary viewpoint given a base model, a set of key deformations (deformed versions of the base model), and a set of corresponding key viewpoints. Based on this idea, the complete system is introduced by Chaudhuri et al. [Chaudhuri et al. 2004]. The system contains a camera recovery engine and a deformation engine. The notion of sketching a motion is less well-defined than that of sketching an object. Nevertheless, a number of approaches have been explored. Early work explores an animation-by-example approach for a single point and fits splines to an input stream of 3D points in order to produce a smooth version of the acted trajectory [Balaguer and Gobbetti 1995]. More recently, a more sophisticated animation-by-example approach has been proposed [Popovic et al. 2003], wherein the trajectory (position and orientation over time)

2 of a rigid body can be specified by examples using a 3D tracker and then cleaned up automatically to synthesize the physically-based motion that best fits the sketched motion. Walking motions can be easily created by drawing a desired path on the ground plane for the character to follow. The drawing governs the walking or running speed. Given the path and the path timing, the character motion can be implemented in many different ways [Arikan and Forsyth 2002; Girard 1987; Kovar et al. 2002; Park et al. 2002; van de Panne 1997]. The drawn input of the doodle system [Thorne et al. 2004] serves as a static visual record of the motion. Labanotation [Hutchinson and Balanchine 1987] is an example of a written motion notation system for dance choreography, which can be automatically translated into 3D human figure animations [Wilke et al. 2003]. 3 System Overview The input of the system is some sketches drawn by the user, a template 2D skeleton whose default value is provided by the system, and a motion capture database with variant types of motions in it. The output of the system is an 3D character animation keyframed by the user-drawn sketches. The operation starts with the user drawing the sketch or loading the sketch image. The user indicates the 2D pose by editing the parts of the provided template skeleton on top of the sketch. The system searches 3D poses (frames) in the database that are most similar to the 2D pose. During this process, the system automatically culls invalid poses based on the joint moving limitations. Then the user can select the most satisfying one as the 3D key pose. Repeating this process for more sketches, more 3D key poses are found. To synthesize the animation, the system connects all the key poses match by finding a path in the database. If the user changes one sketch or is not satisfied with one key pose, the system has a feedback cycle which allows the user to edit the 2D poses and resynthesize the animation by trying to influence on other frames as few as possible. This considers both the user convenience and the system efficiency. The system structure is shown in Figure 2. 4 User Interface and Pose Matching 4.1 User Interface The primary challenge in creating a 3D animation from 2D sketches is that many 3D poses may be mapped to a given 2D stick figure. Multiple poses match the drawing exactly. The imprecise nature of hand drawings compounds this difficulty since poses that approximately match the drawing should be considered as well. Since our goal is to aid animators to initially design an animation, a completely automated pose reconstruction system is not appropriate. However, manually posing an articulated figure by specifying the location of each joint is tedious. Instead, we desire a semiautomated method that allows the artist to influence and control the resulting animation. So the system first provides a series of 3D pose candidates after culling the invalid poses. The user can select the most satisfying one from the candidates. The system is also a feedback system, which allows the user to re-edit the sketches, 3D key poses. The system will re-synthesize the animation in an efficient way. Figure 1: System Overview. The operations that the user need to do is the following. The user first indicates a sequence of drawn keyframes that represent the desired motion by wedging the skeleton. The system provides an template skeleton with the predefined structure for the user. The structure of the template skeleton can be edited by the user in advance. Since the exact 3D pose matching each 2D pose is ambiguous, the user will guide a semi-automated process to the correct reconstruction. For one 2D pose, the system usually maps it to a series of 3D pose candidates. By listing these candidates according to their similarity to the 2D pose, the system allows the user to select the most satisfying candidate as the 3D key pose for the later animation synthesis (Figure 3). 4.2 Pose Matching We suppose the points of view in the sketching window and the motion display window are known and aligned. We project 3D poses to X-Y plane under orthographic projection. To search for best matches, each motion frame is defined as a vector (θ 1,,θ n ) T with the orientation θ i of each of the ith joint. The distance function for pairs of sketch s i and motion capture frame f j is computed as the weighted sum of distances between the orientations of matching body parts. Since the structure of the 2D skeleton may be different from the 3D skeleton. we use a transform matrix A to match the 3D bone structure to the 2D skeleton structure. That is, (s 1,s 2,,s m ) T = A( f 1, f 2,, f n ) T. To remove coordinate-frame differences, we normalize by aligning the roots in the start frames of each window. The distance is then defined as: m D(s, f ) = w i θ i (s) Aθ i ( f ), i=1 where the weight w i scales the angular distances for each joint i. We assign high weights to the trunk part and lower ones for the limbs, because we found the differences in the positions of the limbs can be more easily modified when computing the transition motion. To increase the speed of searching, we preprocess the database to find unique frames based on a difference tolerance and only compare these frames. Once we find the closest frame, we test all possible frames in the surrounding interval to find the best match. 94

3 Figure 3: Black strips: key pose frames; Blue regions: frames to be compared to find the transition point. Figure 4: Black strips: key pose frames; Red strips: transition points. Blue arrows show the frame sequence. 6 Results and Conclusion Figure 2: Top: 2D pose; Bottom: corresponding 3D pose candidates. 5 Path Finding and Animation Synthesis Now we have key poses f 1, f 2,, f k and c 1,c 2,,c k are the motion segments that the key poses belong to. It is possible that c i = c j where i j. That means key poses f i and f j are in the same segment. We need to find the transition point after the position f i in the segment c i before the position f i+1 in the segment c i+1. This region is shown in Figure 4. We compare each frame in the blue region. The pair of frames with smallest difference is the transition point. This is illustrated in Figure 5. Suppose T i and T i+1 are the corresponding transition points in c i and c i+1. The transition from T i to T i+1 may not be smooth and natural enough. There have been many techniques dealing with this problem. For small gap, we can insert frames using interpolation. For big gap, we can use inverse kinematics or physical simulation. It is possible that the synthesized motion has some artifacts, such as feet interpose into the ground. In such cases, the post-processing is needed to optimize the motion. Since these techniques have been sophisticated and discussed a lot in other work, we will not go into detail here. Our system is developed in Java platform. The sketches are drawn using an tablet pen. We test three sketches that are drawn as poses of walking, running and kicking a ball. The skeletons of these sketches are shown in Figure 6(a). The selected frames of the final 3D animation is shown in Figure 6(b). In this paper, we introduce a 3D character animation synthesis method from 2D sketches. We provide an interface for the user to draw the sketches and to extract the skeletons of the sketches. Then the system can semi-automatically find the matching 3D poses. Based on the key poses, the system automatically finds a motion path in the motion database and generate transition motion if needed. The final motion is smooth and natural, and contains key poses the user want. Compared with previous work, our system allows the user to create any kind of character animation with a little of user interaction. The interface is straightforward and the system is easy to use for both novices and experts. One constraint of the work is that we suppose body parts of the character are rigid, i.e., the system does not consider the model deformation. Since deformation is often required to create fancy and vivid animation, we will extend current system to work on nonrigid character bodies. GPUs, the high-performance data-parallel processors found in video accelerators, although designed for 3D rendering, can be applied to numerous problems in general-purpose numerical computation. They often have an order of magnitude higher performance than the host CPU. So we may exploit GPU acceleration in the future. References 95

4 References ARIKAN, O., AND FORSYTH, D Interactive motion generation from examples. In Proceedings of ACM SIGGRAPH 2002, BALAGUER, J., AND GOBBETTI, E Sketching 3d animations. Computer Graphics Forum 14, BLAIR, P Cartoon Animation. Walter Foster Publishing, Laguna Hills, CA, USA. CHAUDHURI, P., KALRA, P., AND BANERJEE, S A system for view-dependent animation. Graph Forum 23, DAVIS, J., CHUANG, E., POPOVIC, Z., AND SALESIN, D A sketching interface for articulated figure animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, GIRARD, M Interactive design of computer-animated legged animal motion. IEEE Computer Graphics and Applications 7, HUTCHINSON, A., AND BALANCHINE, G Labanotation: The system of analyzing and recording movement. In Theatre Arts Books. IGARASHI, T., AND HUGHES, J Smooth meshes for sketchbased freeform modeling. In Proceedings of ACM SIGGRAPH 2003 Symposium on Interactive 3D Graphics, IGARASHI, T., MATSUOKA, S., AND TANAKA, H A sketching interface for 3d freeform design. In Proceedings of ACM SIGGRAPH 1999, KOVAR, L., GLEICHER, M., AND PIGHIN, F Motion graphs. In Proceedings of ACM SIGGRAPH 2002, LASSETER, J Tricks to animating characters with a computer. In Proceedings of SIGGRAPH 1994 Course Notes No 1. PARK, S., SHIN, H., AND SHIN, S On-line locomotion generation based on motion blending. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, POPOVIC, J., SEITZ, S., AND ERDMANN, M Motion sketching for control of rigid-body simulations. ACM Transactions on Graphics 22, RADEMACHER, P View-dependent geometry. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, THORNE, M., BURKE, D., AND VAN DE PANNE, M Motion doodles: an interface for sketching character motion. ACM Transactions on Graphics 23, VAN DE PANNE, M From footprints to animation. Computer Graphics Forum 16, WILKE, L., CALVERT, T., RYMAN, R., AND FOX, I Animating the dance archives. In Proceedings of the 4th International Symposium on Virtual Reality, Archeology, and Intelligent Cultural Heritage., Figure 5: (a): key 2D skeletons representing walking running and kicking a ball.(b): selected frames of the final synthesized animation. 96

5 3D Character Animation Synthesis From 2D Sketches Yi Lin Figure 4: Black strips: key pose frames; Blue regions: frames to be compared to find the transition point. Figure 5: Black strips: key pose frames; Red strips: transition points. Blue arrows show the frame sequence. Efficient Animation of Water Flow on Irregular Terrains Marcelo M. Maes, Tadahiro Fujimoto, Norishige Chiba Figure 1. Water flowing on irregular terrain. Figure 8. Animation frame: fountain 20m 20m 5m. 492

6

7