Movement Primitives, Principal Component Analysis, and the Efficient Generation of Natural Motions

Transcription

1 Movement Primitives, Principal Component Analysis, and the Efficient Generation of Natural Motions Bokman Lim, Syungkwon Ra and F.C. Park School of Mechanical and Aerospace Engineering Seoul National University Seoul , Korea Abstract We propose a framework for robot movement coordination and learning that combines elements of movement storage, dynamic models, and optimization, with the ultimate objective of efficiently generating natural, humanlike motions. One of the novel features of our approach is that each movement primitive is represented and stored as a set of joint trajectory basis functions; these basis functions are extracted via a principal component analysis of human motion capture data. By representing arbitrary movements as a linear combination of these basis functions, and by taking advantage of recently developed geometric optimization algorithms for multibody systems, dynamics-based optimization can be more efficiently performed. Case studies with a diverse set of arm movements demonstrate the feasibility of our approach. Index Terms Motion optimization, movement primitive, principal component analysis, joint angle trajectory I. INTRODUCTION Roboticists are once again turning to our understanding of human movement coordination in an effort to develop better ways of coordinating movements for robots. While a comprehensive theory of human movement is still far away, it is true that great progress has been made in the last few decades, with important contributions coming from researchers engaged in robotics (for a recent overview see the paper by Mussa-Ivaldi and Bizzi [1]). By now there is general agreement that humans learn a great variety of movements, and that these movements are stored in some form in our memories. What still remains unresolved is the exact nature of these building blocks, or modules, for generating movements, and how these modules are used to generate new movements. Since our immediate goal in this paper is to develop efficient methods for generating natural, or physically meaningful, motions for a wide variety of robots, we begin by summarizing some of the observations drawn from human movement research that we feel are particularly relevant. First, in the course of practicing and refining certain motions, there is strong evidence that some sort of optimization is taking place, usually with respect to some physical criterion, e.g., minimum metabolic energy, or the effort dedicated toward control and sensing. It has also been known since the early work of Sherrington [2] that feedback plays a central role in human movement coordination. More recent is the evidence suggesting that inverse dynamics-based feedforward computations are also an integral part of human movement planning, particularly for fast movements [3]. The notion of memory-based computations for movement generation, in particular, the idea that inverse dynamics-based trajectories are stored in table form for each movement, has by now been dispelled on computational arguments; however, it is also clear that the human movement generation paradigm is fundamentally driven by the central nervous system s remarkable capacity for storage and retrieval of large amounts of data. Many researchers have attempted to take these lessons from human movement research and apply them to robot movement coordination. The generation of minimum torque motions based on dynamic models of robots has been investigated by Park, Bobrow, and colleagues [4], where it was verified through various lifting and gymnastics case studies that, indeed, such motions closely resemble their human motion counterparts. Mombaur et al [5] have also successfully demonstrated that stable open-loop controllers for performing somersaults can be obtained, through the numerical solution of a large-scale optimal control problem. From a practical perspective, the main drawback with these approaches is that computationally they are prohibitively expensive, making real-time movement generation effectively impossible. Yamane and Nakamura [6] suggested dynamic filter for online motion generation for human figure. They have used motion capture data as reference motion and efficiently generated physically consistent motion for environment. But to generate movement more versatilely, we should focus on movement adaptation and reuse. To emulate the human capability of storing and reusing movement modules, recently researchers have extended the paradigm of movement primitives to a robotics context. Mataric and colleagues have, through the analysis of human motions, extraced movement primitives as joint trajectories using principal component analysis (PCA), and used them to generate and classify motions for robots [7], [8]. Schaal and colleagues have also developed movement primitives using tools from nonlinear dynamical systems theory, and applied these tools to endow robots with the capability to learn motions by imitating humans [9], [1]. It should be emphasized that these approaches are exclusively kinematics-based; the problem of control, specifically, of determining the required input torques necessary to generate the kinematically specified motion, is not considered.

2 Our main motivation in this paper is to develop a method for generating natural (in the sense of being optimal with respect to a physical criterion) motions in an efficient manner, and at the dynamics level. More specifically, we seek to overcome the two main disadvantages of the dynamics-based motion optimization approach in [4]: the computational inefficiency, and the lack of modularity and reusability. The innovative features of our approach are (i) to select and classify a set of movement primitives, and to store these as a family of basis functions; (ii) to extract these basis functions using principal component analysis of observed human movements, and (iii) to perform dynamicsbased motion optimization using these basis functions to simultaneously improve computational performance and the resemblance to the original human motions. To the best of our knowledge, the representation of these movement primitives as basis functions, and performing local optimization in terms of these basis functions, has not been attempted elsewhere. These basic ideas were first outlined in [11] in the context of simple arm-lifting motions; in this paper we more firmly establish this conceptual framework, and validate our approach with a diverse set of case study motions. In the following section we explain the high-level framework, together with the details of extracting the basis functions using PCA, and the optimization procedure based on these basis functions. In later sections we report on results obtained for various humanoid movements, e.g., reaching and handshaking. II. MOVEMENT GENERATION FRAMEWORK A. High-Level Description Fig. 1. Proposed movement generation framework. Our proposed framework for movement generation is shown in Fig. 1. The kinematic and dynamic features of the robot are first defined together with the environment model. In addition, a video database of motion capture data is assumed available. Given a high-level description of a desired task, the task parser interprets this description and generates a corresponding sequence of desired movements, also in symbolic form. The movement compiler then takes this sequence as input and, drawing upon the database of available movement primitives, constructs an appropriate movement sequence plan this sequence plan provides a trajectory description, suitable for input to the control law generator. In generating these trajectories, the movement compiler performs local optimization and learning, motion interpolation, and motion blending and smoothing, all the while ensuring that any physical and user-specified motion constraints are satisfied. Prior to the optimization, the movement compiler will have extracted from the high-level description a suitable objective function. The control law generator, which also has access to the kinematic and dynamic model and the environment model, performs a basic analysis of e.g., reachability, stability, and gain tuning to ensure that the trajectory sequence is in fact feasible. Once feasibility is confirmed, and the feedback parameters are optimally tuned, an appropriate control law containing both feedforward and feedback terms is then generated as the final output. B. Representation of Movement Primitives The movement primitives are stored as a family of basis functions, with each basis function representing a joint trajectory. Just as an arbitrary continuous function can be represented a linear combination of trigonometric functions using Fourier series, our premise is that arbitrary joint trajectories should be represented as a linear combination of an appropriate set of basis functions. Rather than use, e.g., Fourier series or other standard basis functions from approximation theory, we believe that employing a customized set of basis functions for movement generation is both more economical while leading to more physically desirable motions. For illustration purposes, suppose arm motions are classified into, e.g., reaching, swinging, lifting light objects, and lifting heavy objects; each of these classifications is then regarded as a movement primitive. For each movement primitive we extract a set of k basis functions φ 1 (t),..., φ k (t) for each joint. Clearly the φ i (t) do not form a basis in the strict mathematical sense; rather, they are intended as a rich set from which a large number of joint trajectories can be closely approximated. In our approach each movement primitive is stored as a family of basis functions {φ 1 (t),..., φ k (t)} for each joint. An arbitrary trajectory for that particular joint is then represented as the linear combination k φ(t) = c i φ i (t) (1) with the c i scalar constants to be determined. In the next section we describe how to extract the basis functions φ i (t) from a video database. C. Extraction of Movement Primitives: PCA Like [8], the movement primitives are extracted via a principal component analysis of motion capture data. For

3 example, to construct the basis functions corresponding to an arm reaching primitive, we first obtain motion capture data of a human subject repeatedly performing such motions; the motions should be similar enough to all be classified as reaching motions, yet diverse enough to be able to represent a range of different reaching motions, e.g., reaching upward, downward, to the sides, etc. Once the motion capture data is available, we then perform a principal component analysis. To illustrate the procedure, consider the vector time series data {x[1], x[2],..., x[n]}, with each x[i] R p regarded as a sample of the random vector x R p (for our specific application each x[i] should be regarded as a one degreeof-freedom joint trajectory sampled at p uniform time intervals; N such measured joint trajectories are available). The sample mean x R p is then obtained from the formula x = 1 N N x[i]. (2) The sample covariance matrix S R p p in turn is obtained from the formula S = 1 N (x[i] x)(x[i] x) T. (3) N Let {(λ 1, e 1 ),..., (λ n, e n )} represent the eigenvalueeigenvector pairs for S, where the eigenvectors {e i } are orthonormal, and the eigenvalues are ordered so that λ 1 λ 2 λ n. The eigenvectors {e 1,..., e n } then correspond to the principal components; that is, e 1 R p represents the most dominant joint trajectory (or most representative of the measured data), whereas e n R p represents the least representative joint trajectory. An arbitrary joint trajectory x can thus be represented in discrete form as a linear combination of the e i : n x = c i e i (4) with the c i scalar weighting coefficients. The procedure outlined above is performed for each degree of freedom. An obvious advantage of the above approach is that the resulting trajectories x are forced to resemble those used as measurements in the PCA procedure; this will be even more true if we use just a subset of the most dominant principal component trajectories e i. The obvious disadvantage is that the set of all trajectories x representable as a linear combination of the e i may necessarily be limited, diminishing the richness of possible trajectories generated. We shall see later that the above procedure also leads to important computational advantages during dynamics-based optimization. D. Motion Optimization The equations of motion for our systems, which are modeled as a set of coupled rigid bodies, are of the form M(q) q + C(q, q) q + N(q, q) = τ (5) where M(q) R n n is the mass matrix, C(q, q) R n n is the Coriolis matrix, and N(q, q) R n includes gravity and other forces. We will be interested in minimizing cost functionals of the form J(τ) = Φ(q, q, t f ) + tf L(q, q, τ, t) (6) subject to Equation (5) and appropriate boundary conditions, where the terminal criterion function Φ penalizes deviations from the desired final condition. In [4] a local solution to the optimal control problem is found by assuming that the joint coordinates q(t) in (5) are parameterized by B-splines, and varying these parameters in the following manner. The B-spline curve depends on the blending, or basis, functions B i (t), and the control points P = {p 1,..., p m }, with p i R n. The joint trajectories then have the form q = q(t, P ), with m q(t, P ) = B i (t)p i (7) By parameterizing the trajectory in terms of B-splines, the original optimal control reduces to a parameter optimization problem of the form Min P J(P ) = Φ(P, t f ) + tf L(P, t) (8) subject to q p i q, i = 1... m (9) Here τ = τ(p, t); q, q, and q are all given functions of t and P from (7) and its time-derivatives, and τ becomes an explicit function of the spline parameters through (5). By a proper choice of spline basis functions at both ends of the joint trajectory, the path boundary conditions can be satisfied. In our movement generation framework we replace the B-spline basis functions by the basis functions obtained from principal component analysis (or, more specifically, their continuous-time polynomial approximations). Parameterizing each joint angle trajectory as q(t) = x 1 P C 1 (t) x 4 P C 4 (t) + x 5 (1) and the joint velocities and accelerations by their corresponding derivatives, the minimum torque problem now reduces to a classical parameter optimization problem. In closing this section, we remark that by sufficiently limiting the number of principal components used, we effectively reduce the number of optimization problems, even eventually eliminating the optimization altogether. In this case we have a means of straightforward joint space interpolation in terms of the principal component basis functions. III. CASE STUDY: HUMAN ARM MOVEMENTS To demonstrate the computational feasibility of our approach, we compare the arm movements generated using our approach with that obtained through straightforward dynamics-based optimization. Gains in computational efficiency are empirically assessed, and the degree to which

4 our proposed method generates more natural arm motions is qualitatively evaluated. A. Principal Component Analysis of Human Arm Movements A principal component analysis of typical human arm movements is first performed. The human arm is kinematically modeled as a four dof serial chain, consisting of a three dof shoulder joint (represented as a serial 3-revolute joint with intersecting orthogonal axes) and a one dof elbow joint. The movements of a human subject repeatedly performing various arm movements (i.e., raising, reaching, pulling, twisting, and swinging) are then captured and, following data cleansing and smoothing, and the solution of the inverse kinematics from the motion capture data (given by marker position data) stored in the form of joint trajectories. To ensure that the same number of sample data are stored for each motion, all joint trajectories are time-scaled to the same time interval; a principal component analysis of this data is then performed. As evident from Table I, four principal components are sufficient to represent over 99% of the captured arm movement data. TABLE I PERCENTAGE EXPLAINED BY EACH PRINCIPAL COMPONENT Joint shoulder1 shoulder2 shoulder3 elbow PC % 87.76% 89.29% 92.38% PC2 5.18% 9.21% 9.19% 6.58% PC3.53% 2.24% 1.14%.86% PC4.8%.53%.2%.1% Sum 99.91% Each principal component and its first and second derivatives are stored in table form; this will be useful for implementing fast lookup in the dynamics-based optimization procedure. Fig. 2 shows all the principal component trajectories of the shoulder and elbow joints, and their first and second derivatives. These trajectories are obtained by cubic spline interpolation of the discrete principal component table values. Our movement primitive database is shown in Table II. B. Motion Interpolation with Principle Components The simplest way of generating arbitrary motions is to use the principal components as interpolating functions. Specifically, given a starting and ending arm configuration, we parameterize an arbitrary arm motion as a linear combination of the principal components, and determine the coefficients such that the boundary conditions of the desired arm motion are satisfied. While obviously these motions are not optimal with respect to any physical criterion (e.g., minimum torque), and thus are not particularly useful when the movement must be adapted to, e.g., lift a heavy object, they are nevertheless effective for rapidly generating Task Fig. 2. Principal components of each joint. TABLE II MOVEMENT PRIMITIVE DATABASE Number of motion capture Average time (sec.) raise hand put down hand reach out hand spread arms revolve arms forward revolve arms backward raise hand with 7kg dumbbell put down hand with 7kg dumbbell hold out hand for handshaking handshaking 1.81 restore hand after handshaking small hand waving big hand waving simple reaching motions. In fact, there is justification in the human movement control literature for such an approach to generating simple arm reaching movements, e.g., [13] argues that human arm movements can be described by a low-dimensional superposition of principal components. Using three principal components, and assuming that joint position and velocity values are given at the endpoints, the interpolating joint angle trajectory can be easily ob-

5 tained by solving the following system of linear equations: q(t) = x 1 P C 1 (t) + x 2 P C 2 (t) + x 3 P C 3 (t) + x 4 (11) P C 1 (t ) P C 2 (t ) P C 3 (t ) 1 x 1 q(t ) P C 1 (t f ) P C 2 (t f ) P C 3 (t f ) 1 dp C 1 (t ) dp C 2 (t ) dp C 3 (t ) x 2 x 3 = q(t f ) x 4 dp C 1(t f ) dp C 2(t f ) dp C 3(t f ) dq(t ) dq(t f ) Other boundary conditions, e.g., joint angle accelerations given at the endpoints, can also be accommodated in a similar fashion by using more principal components. Some sample arm reaching motions generated using this approach are shown in in Fig. 3(a) and Fig. 3(b). Because of the simplicity of the computations, the movements are easily generated in real-time, and the resulting movements closely resemble typical human arm movements. variables (x 5 denotes the coefficient for the constant function). Therefore, by using nine control points for each joint degree of freedom, we obtain the same number of degrees of freedom as using four principal components. For the nonlinear optimization we use the following simple stopping criterion J k+1 (q) J k (q) < ɛ (12) where ɛ is 1 1. The optimizations were executed using Matlab version 6.5, running on a Pentium 4 (2.4 GHz) personal computer. As you know, we have local minima problem. But by constructing constant function x 5 in (1) as sum of sample mean x in (2), we can start optimization from initial sample mean motion (if we select initial optimization values as zeros, q(t) = x). This is helpful to avoid unreasonable optimization result. TABLE IV OPTIMIZATION RESULT: ARM REACHING PCA-based approach (a) Arm reaching (b) Arm raising Fig. 3. Sample motions generated by interpolation with principal components. Number of variables Number of PCs 2x4dof 3x4dof 4x4dof 5x4dof Number of iteration Cost function value Time (sec.) C. Dynamic Optimization Using Principal Components We now generate movements by performing dynamicsbased optimization with the principal components. We focus exclusively on generating minimum torque arm motions. Table III lists the dynamic properties of the human arm model. TABLE III MASSES AND MOMENTS OF INERTIA OF HUMAN ARM MODEL Link Mass(kg) Moment of inertia(kgm 2 ) Chest (.73,.63,.32) Upper arm 2.97 (.25,.25,.5) Lower arm 1.21 (.5,.54,.12) To benchmark, we compare our proposed PCA-based method with the more generic method of parameterizing joint trajectories by cubic B-splines. We first consider a simple arm reaching motion, in which the initial pose is standing at attention (all joint angles are assumed set to zero in this pose). For the B-spline parametrization, the input variables to the optimization are the control points. For each joint, the first and final two control points are used to determine the initial and final pose. Bounds are placed on all the control point locations so as to satisfy joint angle limits. Alternatively, by parameterizing joint trajectories as a linear combination of four principal components, the coefficients x 1, x 2, x 3, x 4, x 5 in (1) become the optimization TABLE V OPTIMIZATION RESULT: ARM REACHING B-spline based approach Number of variables Number of control points 7x4dof 8x4dof 9x4dof 1x4dof Number of iteration Cost function value Time (sec.) As shown in Fig. 4, our results indicate clear advantages in computational efficiency for the PCA-based approach. Note that the final objective function values achieved for the two approaches are similar. However, the final optimized motions for the two cases show notable differences, as illustrated in Fig. 5 and 6. Irrespective of the number of control points used, motions generated using the B- spline approach appear more unnatural than those of the PCA-based approach. For the B-spline motion, the elbow is bent at a sharper angle to utilize the kinematic singularity (thereby reducing the total applied torque), and shows fast movement in the latter phase of the reaching movement. Also, because the principal components are extracted from observations of human arm movement, it comes as no surprise that PCA-based motions appear much closer in shape and profile to human arm movements than their B- spline counterparts. We can also generate other types of minimum torque arm motions using the same set of principal components. Fig. 7

6 shows a forehand tennis swing motion, while Fig. 8 shows a handshaking motion. In both cases the motions appear quite natural, despite the fact that the principal components used were extracted from a wide repertoire of arm movements. Fig. 7. Forehand stroke motion Fig. 8. Handshaking motion Fig. 4. Computational performance comparison for arm reaching these basis functions, and by taking advantage of recently developed geometric optimization algorithms for multibody systems, dynamics-based optimization now can be forged into a more practical tool for natural motion generation. Clearly the results suggest that further improvements in computational efficiency are necessary for real-time movement generation. Current research is focused on extending the class of movement primitives, and incorporating feedback (e.g., potential field models) into current movement generation paradigm. Fig. 5. (a) Front view Optimized arm reaching motion comparison (b) Side view Fig. 6. Comparison of optimized arm reaching motion (top: B-spline, bottom: PCA) IV. CONCLUSION In this paper we have proposed a high-level framework for robot movement coordination and learning that combines elements of movement storage, dynamic models, and optimization, with the ultimate objective of efficiently generating natural, human-like motions. An innovative feature of our approach is that each movement primitive is represented and stored as a set of joint trajectory basis functions; these basis functions are extracted via a principal component analysis of human motion capture data. By representing arbitrary movements as a linear combination of REFERENCES [1] F.A. Mussa-Ivaldi, and E. Bizzi, Motor learning through the combination of primitives, Philosophical Transactions of the Royal Society: Biological Sciences, pp , 2. [2] C. Scherrington, Flexion-seflex of the limb, crossed extension reflex stepping and standing, Journal of Physiology, vol. 4, pp , 191. [3] N. Schweighofer, M.A. Arbib, and M. Kawato, Role of the cerebellum in reaching movements in humans. I. Distributed inverse dynamics control, European Journal of Neuroscience, Vol.1, pp , [4] J.E. Bobrow, B. Martin, G. Sohl, E.C. Wang, F.C. Park, and Junggon Kim, Optimal robot motions for physical criteria, Journal of Robotic Systems, vol. 18, no. 12, pp , 21. [5] K.D. Mombaur, H. G. Bock, J.P. Schlöder, and R.W. Longman, Self-stabilizing somersaults, IEEE Transaction on Robotics, unpublished. [6] K. Yamane, Y. Nakamura, Dynamic filter: Concept and implementation of online motion generator for human figure, in Proceedings of the IEEE ICRA. vol.1, pp , 2. [7] O.C. Jenkins, and M. Matari`c, Deriving action and behavior primitives from human motion data, IEEE International Conference on Intelligent Robots and Systems, pp , September 22. [8] A. Fod, M. Matari`c, and O.C. Jenkins, Automated derivation of primitives for movement classification, Autonomous Robots, vol. 12 no. 1, pp , January 22. [9] A.J. Ijspeert, J. Nakanishi, and S. Schaal, Learning attractor landscapes for learning motor primitives, Advances in Neural Information Processing Systems 15, MIT Press, 23. [1] S. Schaal, Dynamic movement primitives - A framework for motor control in humans and humanoid robotics, The International Symposium on Adaptive Motion of Animals and Machines, March 23. [11] F.C. Park, and K. Jo, Movement primitives and principal component analysis, in Advances in Robot Kinematics, J. Lenarcic and C. Galletti, Eds., Kluwer, 24. [12] S. Lee, J. Kim, F.C. Park, Munsang Kim, and J.E. Bobrow, Newtontype algorithms for dynamics-based robot motion optimization, IEEE Transactions on Robotics, 24 (accepted for publication). [13] T.D. Sanger, Human arm movements described by a lowdimensional superposition of principal components, The Journal of Neuroscience, 2(3), pp , February 2.