THE relationship between humans and machines will show

Transcription

1 58 IEEE TRANSACTIONS ON ROBOTICS, VOL. 21, NO. 1, FEBRUARY 2005 Somatosensory Computation for Man Machine Interface From Motion-Capture Data and Musculoskeletal Human Model Yoshihiko Nakamura, Member, IEEE, Katsu Yamane, Member, IEEE, Yusuke Fujita, and Ichiro Suzuki Abstract In this paper, we discuss the computation of somatosensory information from motion-capture data. The efficient computational algorithms previously developed by the authors for multibody systems, such as humanoid robots, are applied to a musculoskeletal model of the human body. The somatosensory information includes tension, length, and velocity of the muscles, tension of the tendons and ligaments, pressure of the cartilages, and stress of the bones. The inverse dynamics of the musculoskeletal human model is formulated as an optimization problem subject to equality and inequality conditions. We analyzed the solutions obtained by linear and quadratic programming methods, and showed that linear programming has better performance. The technological development aims to define a higher dimensional man machine interface and to open the door to the cognitive-level communication of humans and machines. Index Terms Forward dynamics, inverse dynamics, man machine interface, musculoskeletal human model, somatosensory information. I. INTRODUCTION THE relationship between humans and machines will show qualitative differences through technical developments of communications and interactions. Such communications and interactions would imply not only explicit and conscious signal exchange, but also an implicit and subconscious one. In this paper, we discuss the computation of somatosensory information from motion-capture data. Somatosensory information includes all the stimuli sensed by the sensory organs located both outside and inside the body. We are particularly interested in the internal somatosensory information such as tension, length, and velocity of the muscles, tension of the tendons and ligaments, pressure of the cartilages, and stress of the bones. The efficient computational algorithms previously developed by the authors for multibody systems, such as humanoid robots, Manuscript received October 21, 2003; revised March 26, This paper was recommended for publication by Associate Editor J. P. Merlet and Editor H. Arai upon evaluation of the reviewers comments. This work was supported in part by the CREST program, in part by the Japan Science and Technology Corporation, and in part by the Information-Technology Promotion Agency (IPA) of Japan. This paper was presented in part at the International Symposium on Adaptive Motion of Animals and Machines, Kyoto, Japan, March Y. Nakamura and K. Yamane are with the Department of Mechano-Informatics, University of Tokyo, Tokyo , Japan ( nakamura@ynl.t.u-tokyo.ac.jp; yamane@ynl.t.u-tokyo.ac.jp). Y. Fujita is with Nomura Research Institute, Yokohama , Japan ( fujita@ynl.t.u-tokyo.ac.jp). I. Suzuki is with Nintendo Corporation, Kyoto , Japan ( ichiro@ynl.t.u-tokyo.ac.jp). Digital Object Identifier /TRO are applied to a high-dimensional musculoskeletal model of the human body. The somatosensory information sensed deep inside the human body has not has much emphasis put on it for applications other than medicine and rehabilitation. Motion-capture systems have been used to capture only the superficial changes of the human body. By combining the motion-capture system, the musculoskeletal model of the human body, and the efficient computation of kinematics and dynamics, the authors are exploring the possibility of realtime computation of the somatosensory information. We know from our daily experiences that we subconsciously compute and estimate the somatosensory information of others, even when we do not explicitly try to communicate with others in verbal or nonverbal ways. For example, we can imagine whether a person is tired or not by observing his/her walking motion. We can also estimate which part of the body is the most stressed in a lifting motion. It is also known in brain science that a human uses the motor cortex not only for controlling the muscles, but also for recognizing motions of others. Such information is used in our cognitive process to understand the other s intention, mental state, and relationship with us. When the man machine interface obtains access to the somatosensory information, machines would make the first step to understand humans. Such implicit communication between humans and machines could be termed as a cognitive-level communication. The technology will enable us to build robots that naturally reach out to humans in need. The same information would also realize more realistic virtual humans [1], [2] by providing a more detailed internal state. A body of previous research on motion analysis and simulation of a musculoskeletal human model has been conducted in the field of sports science and medicine [3], [4]. Their main purposes are to use the human somatosensory data to improve motion patterns of athletes, and to monitor the recovery of those who are in rehabilitation of mobility. Most of them adopt either a simplified model of the whole body or a detailed model limited to a small part of the body, due to the high computational cost of detailed whole-body human models. The computation of the whole-body somatosensory information was investigated in [5] [7], although the efficiency of computational algorithms was not paid much attention. The authors developed an efficient algorithm for forward-dynamics computation of structure-varying kinematic chains to handle articulated multibody systems with the highest generality [8]. The algorithm was recently improved to deliver /$ IEEE

2 NAKAMURA et al.: SOMATOSENSORY COMPUTATION FOR MAN MACHINE INTERFACE 59 Fig. 2. Model of a simple muscle. 3) Tendon: Passive constrictive wire coupled with a muscle to transmit its power. 4) Ligament: Passive constrictive wire that connects multiple bones to constrain their relative movement. 5) Cartilage: Passive linear spring with zero nominal length that connects multiple bones and constrains their relative movement. Fig. 1. Musculoskeletal human model. order- efficiency with a single processor [9], and orderefficiency with as many processors as order- [10], where is the number of bodies. A near-realtime optical motion-capture system was also developed by Kurihara et al. [11] in the authors group, and will be used to capture the target behaviors. We constructed the musculoskeletal human model so that the dynamics-computation algorithms for multibody systems can be applied in a straightforward way, as we explain in the following section in detail. Each of the muscles, tendons, and ligaments is represented by a single wire which has two ends and any number of via-points and generates a tension. Muscles can also actively change their length. We assume that the tension is uniform throughout the wire, and that the via-points change only the direction of the wire without friction. Cartilages are modeled as linear springs. The forward/inverse-dynamics computation of musculoskeletal human models using motion-capture data allows us to compute the somatosensory information. Applications of this technology include sports science, rehabilitation, and medicine. Being integrated with realtime motion-capture techniques and efficient algorithms, the technology would also find applications in the higher dimensional man machine interface and serve as the foundation of cognitive-level communication between humans and machines. II. MUSCULOSKELETAL HUMAN MODEL A. Elements and Attributes We designed the detailed human model shown in Fig. 1. It is comprised of the skeleton and the musculotendon network. The skeleton is a set of bones grouped into a suitable number of body parts. The musculotendon network is a set of muscles, tendons, and ligaments spreading and straining among bones. Each attribute of these elements is modeled as follows. 1) Bone: Rigid-link element with mass and inertia. 2) Muscle: Active constrictive wire actuator. B. Skeleton Model We used polygon geometric data of the bones commercially available for computer graphics [14]. We also produced another set of polygon data by reconstructing the geometry from computer tomography (CT) cross-sections of a human skeleton model. Both models are based on the standard body of European male. In the beginning, for an experimentation of the whole-body dynamics, we grouped the bones into a smaller number of rigid links for simplification. For example, each of the head and hands is counted as one rigid link. A foot is grouped into two bodies, the toe and the rest. Thus, most of the facial muscles and short musculotendons on the hands and feet are omitted. We applied this simplification and grouped about 200 bones into 53 rigid links. C. Musculotendon Model There are various types of muscles, tendons, and ligaments. For example, there are large muscles like triangular muscles, long muscles winding bones, and muscles with furcation, like biceps. However, we can classify them into the following patterns: 1) one to be replaced by a simple wire that connects an origin and an end; 2) one to be replaced by a wire that has an origin, viapoints, and an end; 3) one to be replaced by several wires forming a fork at a virtual bone; 4) one to be replaced by a set of simple wires; 5) one to be replaced by a combination of (1) (4). We describe the detail of the above patterns with typical examples in the following paragraphs. 1) Simple wire model: Most of the muscles, tendons, and ligaments are modeled as a wire which has only an origin and an end. Fig. 2 is the model of M. Teres Minor, which ends at Humerus.

3 60 IEEE TRANSACTIONS ON ROBOTICS, VOL. 21, NO. 1, FEBRUARY 2005 Fig. 3. Muscle with via-points. Fig. 5. Model of M. Biceps Brachii. Fig. 4. Two models of a muscle with furcation. 2) Wire model with via-points: Some muscles, tendons, and ligaments wreathe around bones and cannot be modeled by simple wires connecting two points. We represent a curved path of a muscle, tendon, or ligament by segmented lines. Namely, we place via-points if a wire wreathes around a bone or if the tendons are sheaved by a synovial sheath. A path of segmented lines has an origin, an end, and any number of corners in the middle, called via-points. We assume that each via-point is fixed to a bone, at which the muscle, tendon, or ligament can slide without friction. Also assumed is that a muscle, tendon, or ligament has a uniform magnitude of tension throughout it, and only changes the direction of tension at the via-points. An example is shown in Fig. 3. 3) Wires connected to virtual links: Muscles, tendons, and ligaments are not always connected in series, but may have furcations. Fig. 4 shows two simple cases with furcations. Although the two models are equivalent, the right-hand case adopts the concept of virtual link, which allows more flexible representation of the muscle, tendon, and ligament networks. A virtual link is modeled as a small link without mass. Fig. 5 is the model of M. Biceps Brachii, which has a bifurcation and a via-point. Two muscles are connected to a tendon at the end of M. Biceps Brachii. The tendon separates into two branches attached to two different bones. Therefore, the furcation model of the tendon is essential to completely model the function of M. Biceps Brachii. Virtual links are needed to model the furcations under the fundamental rule that a wire can have only one pair of origin and end points. 4) Muscle model comprised of multiple wires: We modeled some wide-spanned muscles using several simple Fig. 6. Fig. 7. Model of M. Pectoralis Major. Mechanism of elbow joint. wires. A typical example is M. Pectoralis Major, shown in Fig. 6. 5) Combined model: Some complex parts require a combination of the above-mentioned models. Fig. 7 is the example of the ligaments on the elbow. The anular ligament of the radius forms a ring, allowing independent pronation and supination of the radius while constraining the radius and ulna to interlock at flexion and extension of the elbow. We use a virtual link and a number of via-points to model this function, as shown in Fig. 8. D. Musculoskeletal Model Consulting an anatomy text [15], we built the musculotendon network model. The model covers most of the parts of the mus-

4 NAKAMURA et al.: SOMATOSENSORY COMPUTATION FOR MAN MACHINE INTERFACE 61 forward-dynamics algorithm simulates the whole-body motion from the given wire tensions of the muscles. The computational algorithm to be presented in this section is based on the method proposed in [8], which was originally developed for general multibody systems, including closed kinematic chains, and even for those changing the structure in time. The forward dynamics computation of musculoskeletal models includes the following three steps. Step 1) Compute, the Jacobian matrix of the wire and spring lengths with respect to the generalized coordinates, defined as Fig. 8. Model of ligaments on elbow joint. (1) TABLE I COMPLEXITY OF MUSCULOSKELETAL MODEL. THE 53 BONE GROUPS ARE CONNECTED BY TWO 6-DOF BONES (HIP AND CHEST), TWO 1-DOF BONES (TOES), TWO 0-DOF BONES (KNEECAPS), AND 47 3-DOF BONES (THE OTHERS). EACH VIRTUAL LINK HAS 6 DOFs culotendon networks except for those in the head, hands, and feet. The complexity of the model is summarized in Table I. The 200 bones are divided into 53 bone groups, most of which are connected to the neighboring bones by 3-degree-of-freedom (DOF) spherical joints. The exceptions are to cover complex mobility due to constraints such as surface contacts and closed loops. The skeletal system has 155 DOFs. In addition, we introduce 28 virtual links, which have DOFs in total. The DOF of the kinematic chain is, therefore, 323. We used 366, 56, 91, and 34 wires to model the muscles, tendons, ligaments, and cartilages, respectively. Accordingly, we have 547 wires, in total. Our model should be able to approximate the function of most types of muscles and tendons. However, we do not consider the following phenomena that might occur in the real human body: realtime addition or removal of via-points: moving the joints may introduce new via-points due to new contacts between a muscle and bone, or eliminate a via-point by taking off a muscle from a bone; interwire contacts: we do not model the contact forces between multiple wires, although we do consider those between wires and bones by using the via-points. III. DYNAMICS COMPUTATION OF MUSCULOSKELETAL MODEL A. Forward Dynamics Forward dynamics is the computation to estimate the motion, namely accelerations, of the generalized coordinates for the given generalized forces. In our musculoskeletal model, the where is the length of the wires and springs, is the generalized coordinate, and and are the number of wires and springs and DOF of the model, respectively. and in our model. Step 2) Map the given wire tensions into the generalized force using the Jacobian matrix obtained above. Step 3) Compute the accelerations of the generalized coordinates from the generalized force using the method in [10] and integrate them to obtain the whole-body motion. For the definition of the generalized coordinates and the generalized force, and their determination for complex constrained multibody systems, please refer to [8]. The following paragraphs give more detailed descriptions of the procedures. Step 1) Computation of the Jacobian matrix: In order to compute the Jacobian matrix of (1), we first consider, the Jacobian matrix of the length of the th wire with respect to the generalized coordinates. The matrix satisfies the following relationship: (2) where is the length of the th wire. We suppose that the th wire is comprised of points, including the via-points and the origin and end points. Let the length between the th and th via-points be. Also let the Jacobian matrix of with respect to the generalized coordinates be. Then can be represented by the sum of as follows: Using wire, (3), the position of the th via-point of the th can be described as follows: (4) Differentiating (4) with respect to the generalized coordinates yields the following equation: (5)

5 62 IEEE TRANSACTIONS ON ROBOTICS, VOL. 21, NO. 1, FEBRUARY 2005 where denotes the Jacobian matrix of with respect to the generalized coordinates, and can be calculated using well-known methods such as [16]. We first compute by (3) and (6), and then obtain by copying to the th row of. Step 2) Mapping wire tensions into joint torques: We compute the joint torques from the given wire and spring tensions of muscles, tendons, ligaments, and cartilages. The mapping is linear and represented by the following equation: Step 3) Solving for accelerations: Without losing the generality of [8], we improved the efficiency of computation in [9] and [10]. We use these algorithms to solve for the accelerations of the generalized coordinates. For an articulated multibody system consisting of bodies, the computational complexity of the algorithm is. If the algorithm is implemented on a parallel processing system with processors, the computational time is reduced to. The same algorithm is used for parallel computation on any number of processors for higher efficiency, and even for serial computation on a single processor, as well, without modification. B. Inverse Dynamics With Force Distribution Inverse dynamics computes the tensile strength of the muscles, tendons, and ligaments and spring forces of the cartilages that realizes the motion obtained through, for example, motion capturing. The computation requires the following two steps. Step 1) Apply the standard Newton Euler inverse dynamics computation to the given motion data, and compute the joint torques. We can treat the mechanism as a tree-structure kinematic chain with a fixed base, assuming that the chain is cut open if it has closed loops, and also assuming that free joints are actively driven. This concept originates from the basic idea of [8]. Step 2) Map the joint torques to the wire tensions. For this computation, we first determine the generalized coordinates that may vary in time due to the change of constraints. We use, the Jacobian matrix of the wires with respect to the generalized coordinates, and, the Jacobian matrix of all the joints of the serial tree-structure kinematic chain with respect to the generalized coordinates for the mapping [8]. The mapping problem is not straightforward because the number of the wires are much greater than the DOFs of the whole system. We use mathematical programming methods to solve the problem with redundancy. In the following sections, we first formulate the optimization problem for computing the muscle tensions, and then provide two methods for solving the problem. (6) (7) C. Optimization of the Tensions The relationship between the generalized forces and the wire tensions is represented by (7). On the other hand, the relationship between the joint torques and the generalized forces is provided by the following equation: (8) The generalized forces are computed by simply substituting the joint torques computed in Step 1 into the above equation. The remaining problem is, therefore, to solve (7) for using the generalized force. Since the number of wires (547) is greater than the total DOFs (323), which is even greater than the DOFs of the skeleton system (155), computing from is a highly redundant problem. The wire tensions of the muscles, tendons, and ligaments only work in a contractive direction, while spring tensions of the cartilages work in both contractive and expansive directions. The characteristics are represented by the following constraint condition: where is a matrix which extracts the tensions of muscles, tendons, and ligaments from. It consists of as many unit row vectors as the total number of the muscles, tendons, and ligaments. Each row vector has one at the column corresponding to a muscle, tendon, or ligament. Technically, solving (7) is a problem of actuation redundancy [12] subject to an equality condition of (7) and an inequality condition of (9). The goal of the inverse-dynamics computation of musculoskeletal human models is to estimate the activity of muscles. To meet this goal, we have to carefully set the performance index for the actuation redundancy problem. We will also have to investigate how humans use the redundancy and formulate the performance index to incorporate the knowledge. In order to proceed with the discussion of inverse dynamics computation, we now assume that such knowledge is represented in the following form of nominal use of muscles: (10) where is the muscle tensions derived from biology-based methods, is the joint variable, and represents the time. may be obtained from electromyographic (EMG) data measured simultaneously with the motion data, or from a database describing how humans use the actuation redundancy during a particular task. Vector includes all the sensory information required to compute, such as EMG data and contact forces. The goal is to find that satisfies the dynamics (7) and minimizes the difference from. Our problem can now be represented by the following expression. Find that minimizes (9) (11) (12) (13)

6 NAKAMURA et al.: SOMATOSENSORY COMPUTATION FOR MAN MACHINE INTERFACE 63 1) Solving With Quadratic Programming: Solving (11) (13) forms a quadratic programming (QP) problem, which is commonly represented as the following optimization. Find an extreme value of the objective function (14) under linear equality and inequality constraints for. If the matrix is symmetric positive semidefinite, the problem is called a convex QP problem, for which efficient algorithms are known. In our case, is an identity matrix from (11) and, therefore, positive definite. We may also introduce the weighting ratios among the components of as long as the matrix remains positive definite. Note that in (11). Also note that the third term in (11),, is independent to and, therefore, can be disregarded for optimization. 2) Solving With Linear Programming: Although many efficient algorithms are known for QP, they are still computationally expensive for our large problem. For applications where the computational efficiency is more important than the accuracy, we can also apply linear programming (LP) as an approximating solver of the original problem. By defining (15) (16) we can equivalently reformulate the problem of (11) (13) as follows. Find that minimizes (17) (18) (19) The above formulation is still a QP problem. We simplify it to form a LP problem by introducing the variable vector as follows. For constant vector with positive components, find and that minimize (20) (21) (22) (23) (24) If all components of are one, (20) approximates (17), while with uneven positive components approximates the quadratic performance index with a weighted norm. We apply (20) instead Fig. 9. Example of forward dynamics. of, because LP does not allow square root in the cost function. We can apply LP to the problem because the constraint conditions of (21) (24) and the performance index of (20) are all linear. The simplex method is known as an efficient algorithm for LP. IV. EXAMPLES OF COMPUTATION A. Forward Dynamics The forward dynamics computes the motion of the whole body for a given set of muscle tensions. The tendons, ligaments, and cartilages can be modeled by hard springs and dampers. However, it was not easy to design a practical controller to compute the muscle tensions that produce a relevant example motion by the forward-dynamics computation. The example we adopted here is for a preliminary test which simulates a free motion with zero muscle forces under the gravity while the head position is constrained. Fig. 9 shows several snapshots from the simulation, where the whole body gradually stretches down due to the gravity. The simulation was executed on a Pentium IV 2-GHz machine, and the computation time was 0.5 s for each step. It is also possible to apply the muscle tensions obtained by the inverse-dynamics computation, but it would not yield the same motion as the reference because of the following two reasons. As shown in the results of inverse-dynamics computation, we cannot compute the muscle tensions exactly equivalent to the joint torques required to perform the reference motion, due to the limitation of our musculoskeletal model. Even with a complete model, discrete-time integration and collisions/contacts will also cause the discrepancy. It is, therefore, essential to include a robust controller to control the model in the virtual world. B. Inverse Dynamics In this section, we show the results of inverse-dynamics computation. In inverse dynamics, as we saw in the previous section,

7 64 IEEE TRANSACTIONS ON ROBOTICS, VOL. 21, NO. 1, FEBRUARY 2005 determination of is the fundamental and essential problem. It requires extensive experimental research to build a database of how the human uses the muscles. Such a database will be extremely important for further research of somatosensory computation. We plan to continue the research and carry out experiments for accumulating the data in our future work. In this particular paper, we assume and apply LP and QP to focus on the computational problems. We captured a kicking motion using the motion-capture system developed in our group [11]. The data contain 243 frames captured every 30 ms for 7.29 s. The skeletal system has 155 DOFs and 323 DOFs, including the virtual links, driven by the 366 muscle wires. Through computation of the example, however, we found that the driving inputs are not redundant, and not even sufficient for producing arbitrary whole-body motions. In fact, there was no solution of for the generalized force computed from the captured kicking motion. This was particularly prominent for the joint torques of the vertebras in the neck, which has seven 3-DOF spherical joints, but not so many muscle wires. The results suggest that the forces acting on the whole body cannot be approximated only by the simple wire model of major muscles. In the real human body, the tissue and organs, which are not included in our model, passively generate the distributed forces and support the skeletal system. It would be necessary to build a more precise model taking account of the effects of the tissue and organs. To avoid the essential problem in the current computation, we introduce another variable vector and replace the equality constraint of (21) with an inequality constraint as follows. For constant vectors and with positive components, find,, and that minimize (25) (26) (27) (28) (29) (30) This replacement is conducted to relax the problem and to ensure the existence of a solution by allowing an error which represents the permissible range of error in the joint torques. The permissible error is intuitively tuned by setting the ratio of the magnitudes of and. In our implementation, each of and was set to have uniform components, and we simply select the ratio of their magnitudes. Note that this formulation is effective even if the model is complete and an analytically exact solution exists, because the equation may lose the solution due to singular configurations. For comparison, we also reformulated the QP problem of (11) (13) as follows. Find that minimizes (31) Fig. 10. Comparison of computational time between LP and QP for inverse dynamics. Fig. 11. Comparison of j 0 J f j between LP and QP for inverse dynamics. (32) where indicates the weighting parameter similar to. Note that the above QP problem has 547 variables to optimize, which is the dimension of, while the LP problem of (25) (30) has 1417 variables of,, and. For the computation of convex QP problems, we used the primal-dual interior-point algorithm routines of the free object-oriented software package for solving convex QP problem (OOQP [17]). For the computation of LP problems, we used the simplex method routines of the GNU free LP library (GLPK [18]). All the computation was executed on a PC with a single Pentium IV 2-GHz processor. The results of computation are summarized by Figs We selected the weighting parameters as and. Since the performance indexes have the dimension of in QP and in LP, we cannot simply compare the two weighting parameters. Fig. 10 shows that LP takes 2.98 s for each frame, on average, while QP takes 14.2 s. LP is 4.77 times faster in computation than QP, in spite of the fact that LP has 2.59 times more variables than QP.

8 NAKAMURA et al.: SOMATOSENSORY COMPUTATION FOR MAN MACHINE INTERFACE 65 In the inverse-dynamics computation, we have determined the muscular force by using mathematical optimization methods. We have not yet started the physiological study to make a database of how we take advantage of the redundancy of our actuation system. The computational methods need to be extended to integrate the physiological knowledge, because the computed forces without such knowledge lack the consistency with the actual human muscular forces. The contribution of this paper is the formulation and implementation of a mathematical method that would realize the somatosensory computation with physiological knowledge. Fig. 12. Comparison of jf j between LP and QP for inverse dynamics. Fig. 13. Example of inverse dynamics. Fig. 11 represents, the squared magnitude of the error of constraints. The error becomes very large for LP around frames 50 and 110, the latter of which corresponds to the end of the kick. In general, we can reduce the magnitude of the error by using larger or. In this experiment, however, increase of could not reduce the error of LP, while increase of could reduce that of QP. This difference would be intuitively explained by the fact that LP assumes that the optimal solution is obtained at one of the vertices of the space formed by the inequality conditions. Fig. 12 shows for both LP and QP. Since we assumed, the two curves are equivalent to the squared magnitudes of. Although increase of tended to reduce the error of equality constraints as mentioned above, it admitted the increase of. The large fluctuation of the QP curve is due to the relatively smaller weight for. In other words, the fluctuation can be reduced at the cost of the error of equality constraints. In this particular example, the comparison of the computational time and the two graphs suggest that the LP problem described in (25) (30) is an efficient formulation of the inverse-dynamics problem in comparison with the QP problem. The result of LP computation is visualized in Fig. 13, where the colors of muscle wires change from yellow to red (from white to gray) as their tensions increase. The colors of other passive wires for tendons and ligaments also change from light blue to dark blue (from white to gray) as their tensions increase. The animation of this motion is available at along with other motions, including walk and squat. V. CONCLUSION We proposed a method for modeling the human musculoskeletal dynamics, where the dynamics-computation algorithms developed for kinematic chains are extended to compute the muscle and tendon tensions. We constructed a detailed human model based on this modeling method. We developed computational algorithms for inverse and forward dynamics of a wire-based human model on the basis of efficient multibody dynamics-computation algorithms [9], [10]. The computational algorithms were successfully implemented and used for numerical experiments to show their performance. The computational problem of inverse dynamics is reduced to an optimization problem. The problem was formulated in two forms, namely, as a QP problem and as a LP problem. The two formulations were compared from the viewpoint of computational time, accuracy, and stability of the solution. The results presented in this paper suggested that the LP is more efficient, accurate, and stable. The optimization criteria needs to be specifically and appropriately chosen to simulate how the human uses the muscle redundancy. We are currently searching for the criteria and modeling it. Concurrent use of EMG at motion capturing would be a promising approach. The forward and inverse dynamics computations of the musculoskeletal human model using motion-capture data would enable the computation of the somatosensory information. Applications of this technology include sports science, rehabilitation, and medicine, as it was originally targeted. Integrated with realtime motion-capture techniques and efficient algorithms, the technology would also find its applications in the higher dimensional man machine interface and serve as the foundation of cognitive-level communication between humans and machines. REFERENCES [1] J. Gratch, J. Rickel, E. Andre, J. Cassell, E. Petajan, and N. I. Badler, Creating interactive virtual humans: Some assembly required, IEEE Intell. Syst. Mag., vol. 17, pp , Apr [2] E. de Sevin, M. Kallmann, and D. Thalmann, Toward real time virtual human life simulations, Comput. Graph. Int., pp , [3] M. Kaya, H. Minamitani, K. Hase, and N. Yamazaki, Motion analysis of optimal rowing form by using biomechanical model, Proc. IEEE Engineering in Medicine and Biology Conf., pp , [4] T. Komura, Y. Shinagawa, and T. L. Kunii, Creating and retargetting motion by the musculoskeletal human body model, Visual Computer, vol. 16, no. 5, pp , [5] S. L. Delp and J. P. Loan, A software system to develop and analyze models of musculoskeletal structures, Comput. Biol. Med., vol. 25, pp , 1995.

9 66 IEEE TRANSACTIONS ON ROBOTICS, VOL. 21, NO. 1, FEBRUARY 2005 [6], A computational framework for simulating and analyzing human and animal movement, IEEE Comput. Sci. Eng. Mag., vol. 2, pp , Jan [7] N. Suzuki and A. Takatsu, 3D and 4D visualization of morphological and functional information from the human body using noninvasive measurement data, Atlas of Visualization, vol. 3, pp , [8] Y. Nakamura and K. Yamane, Dynamics computation of structure-varying kinematic chains and its application to human figures, IEEE Trans. Robot. Automat., vol. 16, pp , Apr [9] K. Yamane and Y. Nakamura, O(N) forward dynamics computation of open kinematic chains based on the principle of virtual work, in Proc. IEEE Int. Conf. Robotics, Automation, vol. 3, 2001, pp [10], Efficient parallel dynamics computation of human figures, in Proc. IEEE Int. Conf. Robotics, Automation, vol. 1, 2002, pp [11] K. Kurihara, S. Hoshino, K. Yamane, and Y. Nakamura, Optical motion capture system with pan-tilt camera tracking and realtime data processing, in Proc. IEEE Int. Conf. Robotics, Automation, vol. 2, 2002, pp [12] Y. Nakamura, Advanced Robotics: Redundancy and Optimization. Reading, MA: Addison-Wesley, [13] F. D. Carlson and D. R. Wilkie, Muscle Physiology. Englewood Cliffs, NJ: Prentice-Hall, [14] (2002) The Viewpoint Medical Products: Viewpoint Corporation. [Online]. Available: viewpoint.com/ [15] W. Kahle, H. Leonhardt, and W. Platzer, Taschenatlas der Anatomie, Band 1: Bewegungsapparat. Stuttgart, Germany/New York: Verlag, [16] D. E. Orin and W. W. Schrader, Efficient computation of the Jacobian for robot manipulators, Int. J. Robot. Res., vol. 3, no. 4, pp , [17] Object-oriented software for quadratic programming (OOQP). [Online]. Available: [18] Free Software Foundation. GNU linear programming kit (GLPK). [Online]. Available: Yoshihiko Nakamura (M 87) received the B.S., M.S., and Ph.D. degrees in precision engineering from Kyoto University, Kyoto, Japan, in 1977, 1978, and 1985, respectively. He was an Assistant Professor at the Automation Research Laboratory, Kyoto University, from 1982 to He joined the Department of Mechanical and Environmental Engineering, University of California, Santa Barbara, in 1987 as an Assistant Professor, and became an Associate Professor in Since 1991, he has been with Department of Mechano-Informatics, University of Tokyo, Tokyo, Japan, and is currently a Professor. His fields of research include redundancy in robotic mechanisms, nonholonomy of robotic mechanisms, kinematics and dynamics algorithms for computer graphics, nonlinear dynamics and brain-like information processing, and robotic systems for medical applications. He was the principal investigator of Brain-like Information Processing for Humanoid Robots under the CREST project of the Japan Science and Technology Corporation. He is the author of the textbook, Advanced Robotics: Redundancy and Optimization (Reading, MA: Addison-Wesley, 1991). Dr. Nakamura received excellent paper awards from the Society of Instrument and Control Engineers (SICE) in 1985 and from the Robotics Society of Japan in 1996 and He was also a recipient of the King-Sun Fu Memorial Best Transactions Paper Award, IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, in 2001 and He is a member of the ASME, the SICE, the Robotics Society of Japan, the Japan Society of Mechanical Engineers, the Institute of Systems, Control, and Information Engineers, and the Japan Society of Computer-Aided Surgery. Katsu Yamane (M 00) received the B.S., M.S., and Ph.D. degrees in mechanical engineering from University of Tokyo, Tokyo, Japan, in 1997, 1999, and 2002, respectively. He was a Research Fellow of the Japan Society for the Promotion of Science from 2000 to 2002, and a Postdoctoral Fellow at the Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, from 2002 to Since 2003, he has been an Assistant Professor with the Department of Mechano-Informatics, University of Tokyo. His research interests include dynamics simulation of human figures, control of humanoid robots, and physically-based animation in computer graphics. Dr. Yamane received the King-Sun Fu Memorial Best Transactions Paper Award and an Early Academic Career Award from the IEEE Robotics and Automation Society in 2001 and 2004, respectively. He is a member of the Robotics Society of Japan and ACM SIGGRAPH. Yusuke Fujita received the B.S degree in mechanical engineering and the M.S. degree in information science and technology from the University of Tokyo, Tokyo, Japan, in 2002 and 2004, respectively. His research interests include motion planning and dynamics computation of human figures. Mr. Fujita is a member of the Robotics Society of Japan. Ichiro Suzuki received the B.S. and M.S. degrees in mechanical engineering from the University of Tokyo, Tokyo, Japan, in 2001 and 2003, respectively. He is now with Nintendo Corporation, Kyoto, Japan. Mr. Suzuki received a Research Promotion Prize from the Robotics Society of Japan in He is a member of the Robotics Society of Japan.