MOTION capture is a technique and a process that

Transcription

1 JOURNAL OF L A TEX CLASS FILES, VOL. 6, NO. 1, JANUARY Automatic estimation of skeletal motion from optical motion capture data xxx, Member, IEEE, Abstract Utilization of motion capture techniques is becoming more popular in the pipeline of articulated character animation. Based upon captured motion data, defining accurate joints position and joints orientation for the movement of a hierarchical human-like character without using a pre-defined skeleton is still a potential concern for motion capture studio. In this paper, we present a method for automatically estimating hierarchical human skeleton from optical motion capture data. Our approach considers the marker group and the human biomechanical information to determine the topology of the character skeleton. The output of motion data from a hierarchical skeleton is able to be used for further character motion editing and retargeting. Index Terms Motion capture, Motion tracking, Character mapping, Character animation I. INTRODUCTION MOTION capture is a technique and a process that digitally record the movements of a live performer in a specified 3D space, such as human motion or the movement from an animal. The recorded motion data can be replicated into a computer generated character for manipulation. Since motion capture techniques is the most cost effective means to achieve realistic actions from a performer, it has been widely used in the computer games, movies that are heavily involved in the human-like character animation, and other areas that focus on the analysis of movement from the creature. It is known that captured optical motion data only record a set of three dimensional Cartesian points that describe the trajectories of markers attached on the live performer, but the recorded three-dimension motion data can not be used for manipulating a character directly because the motion data output format does not explicitly construct a hierarchical skeleton being captured. The difficulty for using optical motion capture data for a character animation is to transfer the three dimensional position data in Cartesian space to an articulated skeleton character motion that is defined by a world coordinate transformation at the root joint and other relative joints angles in its hierarchy. To convert the markers Cartesian position data into a hierarchical skeleton movement in joints space is the most important procedure for dealing with motion capture data. After the mapping of markers movements to an articulated skeleton s joints rotation, this conversion process simplifies and formats the motion of the performer that is editable and controllable. Such a procedure makes further motion data editing more convenient and shortens the production pipeline of computer animation. Usually, Autodesk Motionbuilder [2], a popular commercial software package, use a pre-defined skeleton for mapping the Manuscript revised January 11, optical motion capture data to a hierarchical character motion. During the calibration procedure, motion capture producer manually adjust the prototype of the pre-defined character and group the location of markers for data mapping. The quality of output motion data is based on the experience of the producer. Manual adjustment of the pre-defined skeleton for data mapping is a tedious procedure. Whereas, in this paper, we describe a method that will automatically group marker sets to find accurate joints location in the hierarchy. Since we use signal-based active motion capture system, there is less markers attached on a performer than passive optical system. In order to find accurate joint position and obtain more precise joint orientation, our method highly relies on biomechanical information from human motion [10][20]. Following automatic human character skeleton fitting from marker group, an inverse kinematics method is used to compute the joint rotation for character motion [19][21]. II. PREVIOUS WORK Motion capture system are able to record realistic and detailed human movement. However, in order to use motion capture data efficiently for character animation, there are several technical concerns in the motion data post-processing, and numerous researchers focus on the issues that related with the techniques, e.g., motion data mapping, motion editing, motion retargeting, motion blending etc. In computer animation, animator usually use existing human motion data to drive his own personal designed virtual character, e.g., a monster that has different skeleton prototype or has same prototype but different skeleton length from human being. In order to mapping human motion data to a custom designed virtual character, some researchers focus on the motion retargeting to solve the prototype problem [6][9]. A motion blending technique has been presented to create a long, continuous motion from multi motion clips [8]. Recently, some researchers in computer graphics are capable of abstracting the characteristic motion, a basis motion data, from an amount of motion database. They focus on modifying and editing motion data at the behaviour level through statistics based method [5][12][13][17]. Several other researchers took physics into account to modify motion capture data and synthesize convincing and realistic character movement [1][4][11][15][16]. These efforts aimed at finding physically convinced character movement in terms of required additional criteria. The techniques described above are based upon the fact that the skeleton motion are already in hierarchical joint space. To manipulate hierarchical skeleton for further motion editing, it is important and necessary to obtain joint rotation angle in

2 JOURNAL OF L A TEX CLASS FILES, VOL. 6, NO. 1, JANUARY a hierarchical skeleton from recorded motion capture data. Several researchers have presented their solutions for this specific problem. Bodenheimer et al. described a detailed procedure for transforming magnetic motion capture data to an articulated human skeleton [3]. O Brien et al. also estimated the skeleton from magnetic motion capture system in which each marker has both position and orientation information to deal with [14]. By comparison, the data captured from optical motion capture system only record position trajectory for each marker. The optical motion data are non-rotational, three-dimension Cartesian position. To construct an articulated skeleton from a camera-based multi-marker optical motion capture system, Silaghi et al. presented a method in which the joint position is inferred based upon adjacent joint link information [18]. Kirk and his colleagues groups the markers in different cluster according to image processing technique, a fitting method is used for estimating the joint position [7]. Considering the fact, the motion capture system usually has been used to capture live performer, e.g. human being, animals etc. Zordan et al. assume that all the markers attached on an actor are driven by force separately. For testing his assumption and result, he mapped the marker data into a fixed human character. A physical based forward dynamic model has been processed for motion data mapping. Then joint angle information of a fixed limb-length skeleton is obtained from the simulation of the equilibrium state [22]. III. MOTION CAPTURE AND SKELETON FITTING Currently, passive and active optical motion capture systems are the mainstream for human-like character motion tracking. The main difference between these two system is that in a passive system each marker has to be identified continuously during capture session, whereas in an active system the marker can communicate through pre-defined module to achieve individual identification. Among these optical motion capture systems, there are two types of devices: one is the camera based optical motion capture system that use imageprocessing technique to define the location of each marker, the other use signal-processing technique that transmit and detect a set of LED markers pulse to ascertain each marker s position. Unlike the camera based motion capture system, signal-processing based system is able to use less markers for capturing. Comparing with the processing techniques used in camera based motion capture system, Ascension ReActor2 is a signalprocessing based active optical system for motion data capturing. Each of the markers is attached at the related joint position based on the hierarchical skeleton of human being. There are 28 infrared markers used for tracking a live performance. According to the calibration of the system, the detectors will be placed at the specified position in 3D space. They will receive light flashed from each marker and determine the location of the markers. The recorded marker position are discrete three-dimension position points in time domain. For an ideal motion capture, we assume that the marker attached on the body at the same position during the motion capture session. The marker position represents the best estimation of joint position according to the anatomical topology Fig. 1. Topology of human character and some markers position showed with black dot. of a human being. Different from the previous researches were to map marker groups to a pre-defined rigid body skeleton. In our method, we define and estimate joint position from biomechanical information [10]. Figure 1 is the front view of the human character model with skeleton beneath skin and some markers position for motion capture arrangement. Our skeleton identification procedure includes two stages: a) identify each marker and group markers for individual rigid body part; b) calculate the actual joint rotation centre based upon the anatomy knowledge of human being. A. Marker Identification We know the difference between passive and active motion capture system is that how the marker works. Unlike passive system, the marker in active system is able to communicate with tracking devices, e.g., signal detectors. The communication is successful by means of LED pulse. The LED marker signal may be affected more or less by other environment lighting resource during the motion capture session. The noise would affect the quality of motion capture. Reduce or delete the noise affection is more important for obtaining precise marker position. There are wide range of frequencies in the electromagnetic spectrum. For infrared, the spectrum range covers from Hz to Hz. Since the frequency spectrum of infrared is known, the removal of the noise could be solved by means of bandpass digital filtering or other technology. To control LED marker flash respectively, a linear signal transmitter is able to transmit infrared light to the detectors according to the temporal constraint. As long as sufficient electricity power supplied to the LED marker, there are continuous pulse transmitted to the tracking devices. As a linear signal detector, it is able to receive the signal constantly in each time domain and identify individual marker

3 JOURNAL OF LATEX CLASS FILES, VOL. 6, NO. 1, JANUARY all the time. B. Fitting human character skeleton To construct hierarchical human skeleton, the essential task is to find the proper joint position that connect adjacent body segments. Before motion capture session, motion capture producer usually attach the marker on a performer in the position that could be regarded as a reference of performer s joint. After the detector has received the LED marker pulse, the precise marker position has been recorded that describes the performer s movement in three dimensional space. As the above assumption, the distance between each two markers which attached on the same body part would be the same all the time. This means the two adjacent markers should define one rigid body link in the hierarchical skeleton. From the biomechanical information of human motion [10][20], the actual human joint location beneath skin which related with referential marker group should keep the constant distance during the movement all the time. To guarantee the joint location in constant for motion data mapping, we define and minimize a joint penalty function to determine the accurate joint position that related with its marker group (Equation 1). k Pi k Pr k k ε k (1) k Pi is the k th joint position, Pr is the referential marker group position which related with k th joint, ε is the distance between the joint and the referential marker. Following up the solution of joint position, the next stage is to find the rigid body segment. Theoretically, the body segment between two adjacent joints should keep the same length within all frames. Constant skeleton length for each rigid body segment can be computed over time by standard s n P 1 deviation l = n 1 (li l)2. l, li and l are length, length i=1 at ith frame and mean of length respectively for each body segment. Once a hierarchical rigid body skeleton has been fitted according to joint position and rigid body segment, inverse kinematics is used to find joint rotation [19][21]. IV. I MPLEMENTATION AND RESULTS During the implementation, at the marker group stage, we automatically set markers group according to the prototype of human skeleton. For example, the four markers around the performer s waist will be used to define the root joint of skeleton and the hip joints in the hierarchy. The constructed hierarchical skeleton includes 20 revolution joints. Each of them has one to three degree of freedoms, and the root joint has six degree of freedom which includes three rotations and three translations. After calculation and determination of the joint position and the length of each body part, our method is able to estimate and construct joint orientation over time by inverse kinematics. In Figure 2, we show the original raw motion data and the constructed human skeleton. As we described early, most passive optical systems use more markers to define skeleton rigid link and joints. We noticed that the calculated position of Fig. 2. Row 1 and row3 are the raw optical motion data based on each individual marker set (black square shows the marker position) captured from Reactor2 and Vicon system respectively. Row 2 and row 4 are the constructed skeleton (green sphere displays joint, blue bar is the rigid body segment, and yellow dot shows the centre of mass of each rigid body). centre of mass of each individual rigid body segments is very much related with the extra marker used in Vicon system. The method we presented for mapping optical motion data is not the particular case for Reactor2 system, it is able to fit other optical motion capture data as well. To test our method, we chose various optical motion data ranging from subtle motion to more dynamic movement, e.g., dancing and marshal art. To infer the joint position, our method relies on the biomechanical information of human motion. For a standard human skeleton, we are able to find a experiential formula for the calculation of the body segment length according to human height [20]. In Figure 3, the calculated body segment length from two males and one female vs standard anthropometry are shown. And one male result (marked as ) are obtained according to the marker set of Vicon system. There are obvious offset between each data set. The average relative error among the samples is 4 5%. The biggest relative error is about 25% in calculating the length of a female s upper leg, which is 10

4 JOURNAL OF L A TEX CLASS FILES, VOL. 6, NO. 1, JANUARY centimeter longer than the standard measurement. In fact, the female does has very longer thigh compare with her height. For other calculation, there is a 1 4 centimeter difference from the anthropometry. And we did know how these difference came from. For example, the marker shape in Reactor2 system is a 3 centimeter diameter disk. In order to prevent the marker moving from the touch point, motion capture producer did put the wrist crossbar and the ankle marker away from actual joint position about 3 4 centimeter to guarantee the marker position stable. For the final character animation, the skeleton motion will be edited for scene organisation, and the further motion retargeting is able to compensate the calculation error. To obtain less error, we also could attach the marker to the joint as close as possible. So, for achieving the motion data mapping results in consistent, the error due to the marker arrangement is acceptable. recorded 28 markers position. Since our method for joint angle calculation is per-frame based technology, there is a limitation for less marker motion capture system. If one of the marker is lost or blocked by other object, the signal detector would not record the marker position. The output of the skeleton motion would produce more unrealistic effect. In order to achieve satisfied motion data, motion capture producer need either recapture motion data or clean up data after capture session. The more information supplied from optical marker, the more accuracy of skeleton mapping achieved. After we tested our method by other optical motion capture data, we noticed that the different naming convention among other optical system would limit our automatic method. For a common facility, it would be ideal to set up a marker set and do the marker name mapping manually. Our approach is helpful for mapping optical motion data to a same skeleton character. As we know from test result, the process is able to play on line. By simply scaling skeleton and mapping joint orientation to a new character, it is not a precise way for online performance. Much more artifacts would appear after the motion data mapping, e.g., penetration. For this special online issue, our mapping approach would consider joint limitation and other physics-based approach for skeleton retargeting. ACKNOWLEDGMENT We thank for their valuable comments and criticism. We would like to thank AccessMocap motion capture studio allow us to use the motion capture devices and donate the motion capture data. We also like to thank ATC who donate the Vicon system C3D motion data. Fig. 3. Human body segment fraction of human height, + and from two male performers, from a female. All the tests were run on a 3.4Gz Pentium IV with 2GB of main memory. Since our skeleton joint orientation estimation is per frame based technology, the computation time for marker mapping and the skeleton joint calculation is 12ms, and for simply stick character graphic drawing is 4ms. In respect of a complex geometric shape character, the time for graphics drawing may increase. Even though we haven t test complicated character, we believe this offline automatic human-like skeleton joints estimation processing can play on real time if the network streaming is fast enough. V. DISCUSSION AND CONCLUSION Motion capture technique is becoming more demanding in the pipeline of an articulated character animation. In order to transform three-dimension positional marker data to a useful skeleton joint rotational angle, apart from the commercial software by which animator manually adjust a pre-defined character for marker mapping, in this paper, we present an automatic method for marker mapping and skeleton joints estimation from optical motion capture data. The presented method is easy to map a hierarchical human skeleton from optical motion capture data. And the accuracy of skeleton joint position and orientation heavily relies on REFERENCES [1] Y. Abe, C. K. Liu and Z. Popovic. Momentum-based parameterization of dynamic character motion. In Proceedings of Eurographics/ACM SIGGRAPH Symposium on Computer Animation [2] Autodesk. Autodesk Motionbuilder. WWW Site, [3] B. Bodenheimer, C. Rose, S. Rosenthal, and J. PellaThe process of motion capture: Dealing with the data. In Computer Animation and Simulation 97, pages Eurographics, Springer-Verlag, Sept [4] A. C. Fang and N. S. Pollard. Efficient Synthesis of Physically Valid Human Motion. In Proceedings of ACM Transactions on Graphics, pages , July [5] P. Glardon, R. Boulic, and D. Thalmann PCA-based walking engine using motion capture data. CGI 04: Proceedings of the Computer Graphics International. pages , [6] M. Gleicher. Retargetting motion to new characters. In Computer Graphics, pages 33-42, July In Proceedings of SIGGRAPH [7] A. Kirk, J. F. O Brien and D. A. Forsyth. Skeletal parameter estimation from optical motion capture data. Computer Vision and Pattern Recognition CVPR [8] L.Kovar and M.Gleicher. Flexible automatic motion blending with Registration Curves. Eurographics [9] J.S.Monzani, P.Baerlocher, R.Boulic and D.Thalmann. Using an intermediate skeleton and inverse kinematics for motion retargeting. Eurographics [10] B. F. LeVeau. Williams and Lissner s Biomechanics of human motion.3rd edition. W.B. Saunders Company [11] C. K. Liu, And Z. Popovic. Synthesis of complex dynamic character motion from simple animations. SIGGRAPH 02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pages , [12] T. Mukai and S. Kuriyama. Geostatistical motion interpolation. ACM Transactions on Graphics. Proceedings of ACM SIGGRAPH 2005, volume24, pages

5 JOURNAL OF L A TEX CLASS FILES, VOL. 6, NO. 1, JANUARY [13] M. Muller, T. Roder, Efficient content-based retrieval of motion capture data. ACM Transations on Graphics. Proceedings ACM SIGGRAPH 2005 volume 24, pages [14] J. F. O Brien, R. E. Bodenheimer, G. J. Brostow, and J. K. Hodgins. Automatic joint parameter estimation from magnetic motion capture data. In Proceedings of Graphics Interface 2000, pages 53-60, May [15] Z. Popovic and A. Witkin. Physically based motion transformation. In Proceedings of SIGGRAPH, pages 11-20, August [16] C. Rose, B. Guenter, B. Bodenheimer and M. Cohen. Efficient generation of motion transitions using spacetime constrains. In proceedings of SIGGRAPH, pages , August [17] A. Safonova, Jessica K. Hodgins and Nancy S. Pollard. Synthesing pgysically realistic human motion in low-imensional, behaviour-specific spaces. ACM Transactions on Graphics, volume 23, pages , [18] M. C. Silaghi, R. Plankers, R. Boulic, P. Fua and D. Thalmann. Local and global skeleton fitting techniques for optical motion capture. CAPTECH 98; Modeling and Motion Capture for Virtual Environments, pages 26-40, [19] L. C. T. Wang and C. C. Chen. A combined optimization method for solving the inverse kinematics problems of mechanical manipulators. IEEE Transactions on Robotics and Automation, volume 7, papges , 1991 [20] D. A. Winter. Biomechanics and Motor Control of Human Movement. Wiley, New York, second edition, [21] J. Zhao and N. Badler. Inverse kinematics positioning using nonlinear programming for highly articulated figure. ACM Transactions on Graphics, volume 13, pages , [22] Victor B. Zordan and Nicholas C. Van De Horst. Mapping optical motion capture data to skeleton motion using a physical model. SCA 03: Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages , 2003