Action Recognition Through an Action Generation Mechanism

Transcription

1 Action Recognition Through an Action Generation Mechanism Barış Akgün Doruk Tunaoğlu Kovan Research Lab. Computer Engineering Department, Middle East Technical University, Ankara / Turkey Erol Şahin erol@ceng.metu.edu.tr Abstract In this paper, we studied how actions of others can be recognized by the very same mechanisms that generate similar actions. We used action generation mechanisms to simulate actions and compare resulting trajectories with the observed one as the action unfolds. Specifically, we modified Dynamic Movement Primitives (DMP) for online action recognition and used them as our action generation mechanism. A human demonstrator applied three different actions on two different objects. Recordings of these actions were used to train DMPs and test our approach. We showed that our system is capable of online action recognition within approximately the first one-third of the observed action with a success rate of over %90. Online capabilities of the recognition system were also tested in an interactive game with the robot. 1. Introduction Action recognition, the understanding of what a human (or another robot) is doing, is an essential competence for a robot to interact and communicate with humans. Nature might have already provided an elegant way for action recognition through mirror neurons. These neurons, discovered in the area F5 (and some other areas) of the premotor cortex of macaque monkeys, fire both during the generation and the observation of goal-directed actions (Rizzolatti et al., 1996, Gallese et al., 1996). Grasping is a prominent example of these actions. Grasping related mirror neurons of a macaque monkey are active both when the monkey grasps a piece of food and when a human or another monkey does the same thing. In this context, goal-oriented means that the action is directed towards an object being the food in the previous example. These neurons do not respond when the observed agent tries to grasp the air where there is no object. The dual response characteristics is attributed to action recognition hypothesis of mirror neurons; action generation and recognition shares the very same neural circuitry and that motor system is activated during action recognition. This paper proposes an online action recognition scheme using an action generation mechanism taking its inspiration from mirror neurons. 2. Related Work There have been various modelling efforts and studies related to integrated action generationrecognition approaches after the discovery of mirror neurons. Some computational models are developed to explain and/or mimic the mirror neuron system and some other models to directly be used in robotics (action generation etc.). From the robotics side, these models mostly deal with action generation, action learning and imitation, mentioning recognition as a side study. A review regarding the computational models of imitation learning is given in (Schaal et al., 2003a). In addition, (Oztop et al., 2006) presents a review about imitation methods and their relations to mirror neurons. This literature survey will present computational models that could be used as integrated action generation-recognition mechanisms. Recurrent Neural Network with Parametric Biases (RNNPB) is a modified Jordan-Type recurrent neural network with some special neurons in its input layer that can act both as input and output, according to context. These neurons constitute the parametric bias (PB) vector (Tani, 2003). RNNPBs are used to learn, generate and recognize actions (Tani et al., 2004). PB vectors are calculated per action but weights of the network are shared across different actions. For generation, learned PB vectors are input to the network according to the desired action. However, goal setting is not possible. Recognition mode is similar to learning mode; PB vectors are calculated from the observed action. Then the

2 calculated PB vector is compared with the existing vectors for recognition. PB vectors are used as output in the learning and recognition mode and used as input in the generation mode. Mental State Inference (MSI) (Oztop et al., 2005) model is a motor control model augmented with an action recognition scheme. Action generation mode incorporates a forward model, to compensate for sensory processing delays. This forward model is exploited for action recognition. Actor is assumed to make a certain action and a mental simulation of that action is carried out by the observer, using the forward model. The simulated and observed action trajectories are compared and according to the error, assumed action is updated. Modular Selection and Identification for Control (MOSAIC) (Wolpert and Kawato, 1998, Haruno et al., 2001) is an architecture for adaptive motor control. Given a desired trajectory, this architecture creates necessary motor commands to achieve it. The controller-predictor pairs compete and cooperate to handle a given control task in action generation mode. The contribution of a pair to the total control is proportional to its predictor s performance. Predictors performances are represented with responsibility signals. For action recognition, an observed trajectory is fed to the system as the desired trajectory and the responsibility signals are calculated which are then compared with the ones in the observer s memory. Hierarchical Attentive Multiple Models for Execution and Recognition (HAMMER) architecture is first used for imitation in (Demiris and Hayes, 2002) and recognition in (Demiris and Khadhouri, 2006). It consists of multiple forward and inverse models, as in MOSAIC. However, only a single pair handles the control task which is determined by its forward model s prediction performance, called the confidence value. For recognition, observed trajectory is given to the architecture and the confidence values are calculated. The action which has the highest confidence value is the one recognized. Dynamic Movement Primitives (DMP) (Ijspeert et al., 2001, Schaal et al., 2003b) are nonlinear dynamic systems which are used for imitation learning, action generation and recognition. The imitation learning consists of estimating parameters of a function approximator and these parameters are stored to be used during action generation. For action generation, goal and corresponding learned parameters are fed to the system. Recognition with DMPs are done by treating the observed action as a new one to be learned (Ijspeert et al., 2003). Then the calculated parameters are compared with the ones in observer s repertoire. There are some drawbacks of the aforementioned architectures for action recognition. RNNPBs are limited to offline recognition since PB vectors are calculated iteratively and all of the action needs to be observed. DMPs also need to observe the whole action to calculate its parameters and use them for recognition. Depending on implementation, MO- SAIC can suffer from a similar problem. MOSAIC and MSI methods use instantaneous error to update responsibility signals and action estimates respectively which is prone to noise. MOSAIC, HAMMER and MSI architectures do not provide a way for imitation learning and recognition is done on the assumption that the observer s and the actor s actions are similar. Goal-setting is not available for the RN- NPB approach, which limits action recognition on a single goal. 3. System Architecture In this section, we first describe the DMP framework and then present how we extended DMPs for online action recognition. 3.1 Dynamical Movement Primitives A DMP generates a trajectory x(t) by defining a differential equation for ẍ(t). There are more than one DMP formulation. The one that we are interested is; ẍ = K(g x) Dẋ + K f(s, w), (1) where K is the spring constant, D is the damping coefficient, g is the goal position, s is the phase variable, w is the parameters of the function approximator f and x, ẋ and ẍ are respectively the position, velocity and acceleration in the task space. The phase variable s is defined to decay from 1 to 0 in time as: s 0 = 1, (2) ṡ = αs, (3) = s = e α(t t0), (4) where α is calculated by forcing s = s f at the end of the demonstrated behavior. Equation 1 defines a DMP for a one dimensional system. For a multi-dimensional system, a separate DMP is learned for each dimension, bound via the common phase variable s. Note that DMP s can be defined in the joint space (with joint angles being the variable) as well as in the Cartesian space. We can analyze the behavior of the system in two parts: The canonical part (K(g x) Dẋ) which is inspired by mass-spring-damper equations and the non-linear part (K f(s, w)) which perturbs the massspring-damper system. The canonical part has a global asymptotic stable equilibrium point which is {x, ẋ} = {g, 0}. In other words, given any initial condition the system is guaranteed to converge to g with zero velocity as

3 time goes to. Moreover, if we ensure D = 2 K then the system will be critically damped and it will converge to the equilibrium point as quickly as possible without any oscillation. For a multi-dimensional system, the canonical part generates a simple linear trajectory between the starting point and the goal point in the task space. The non-linear part perturbs the canonical part in order to generate complex trajectories. For this, the perturbing function f(s, w) learns to approximate the additional input to the dynamics generated by the canonical part necessary to follow a trajectory, through its parameters w which are estimated from demonstration. 3.2 Learning by Demonstration For our case, learning by demonstration (LbD) refers to teaching a robot an action by demonstrating that action (as opposed to programming it). The presented recognition scheme requires that the actor and the observer have similar action generation mechanisms and LbD is used to ensure this. For DMPs, LbD is done by calculating the parameters w from a demonstrated action. A demonstrated action has different trajectories for each of its dimension. Given trajectories, x(t), for each of the dimensions, w of each dimension can be calculated as follows: First, the phase variable s(t) is calculated from equation 4 where α is calculated by forcing a final value (s f ) to the phase variable. Then for each of the dimensions, f(t) is calculated as: f(t) = ẍ(t) K (g x(t)) + Dẋ(t). (5) K Since s(t) and f(t) are calculated, and since s(t) is a monotonically decreasing function between 1 and 0, f(s) is also known for s [0 1]. Finally w for each dimension is estimated by applying linear regression using radial basis functions where s values are treated as inputs and the corresponding f(s) values are treated as targets. 3.3 Extending DMPs for Online Recognition There are some major problems of DMPs for online recognition, mostly because of the non-linear part: 1. The lack of an online recognition scheme. 2. The use of a phase variable that runs in an openloop fashion (i.e., not related to x or ẋ). 3. A consequence of the second problem: α in equations 3 and 4 has to be calculated from the end of the motion which prevents the recognition to be done in online fashion. 4. Another consequence of the second problem: s depends on starting time of an action (Equation 4) which may not be observed or estimated by an observer since it is an internal parameter of the actor. In order to circumvent these problems, we replaced f(s, w) with f( z, w) in equation 1, where vector z consists of variables that are calculated from the current state only. We chose to represent our state in Cartesian space instead of joint space. The reasoning behind is that it is easier to observe the end-effector of the actor in Cartesian space than the angular positions of his joints. We used object-centered Cartesian positions and velocities as the variables of z. The objectcentered representation implicitly sets g as zero for all dimensions. Such a choice of hand-object relations is inspired from (Oztop and Arbib, 2002, Oztop et al., 2005). Using hand-object relations for generation and recognition greatly reduces actorobserver differences and allows seamless use of generation mechanisms in recognition. As stated before, our hand-object relations are defined as relative position and velocity of the end-effector with respect to the object. Learning the non-linear part means learning a mapping from z to f where elements of f are the f value for each DOF. Given N recorded trajectories x i (t) for an action, each trajectory is translated to an object-centered reference frame. Then, x i (t) and x i (t) are calculated through simple differentiation. f i (t) is calculated using equation 5, and z i (t) is constructed by concatenating x i (t) and x i (t). Finally, a feed-forward neural network is trained by feeding all z values and the corresponding f values as inputs and target values respectively. 3.4 Recognition The aim of the presented approach is to recognize the observed action, and the object being acted upon, in an online manner. For recognition, initial state observations are given to action generation systems as initial conditions and future trajectories are simulated. These are then compared with the observed trajectory by calculating the cumulative error; err i (t c ) = t c t=t 0 x o (t) x i (t), (6) where err i is the i th behavior s cumulative error, t 0 is the observed initial time, t c is the current time, x o is the observed position, x i is the i th behavior s simulated position. Then, to have a quantitative measure of similarity

4 Figure 1: Recognition architecture flow diagram. Motion capture system tracks possible objects and the end-effector from which state variables are calculated. When recognition starts, the observed state is used as initial values to the learned behaviors (not shown). The learned behaviors (i.e., generation systems) then simulate future trajectories. As the action unfolds, observed and simulated trajectories are compared and responsibility signals are calculated. From these signals, recognition decision is made according to threshold. between actions, we define recognition signals as: rs i (t c ) = j e erri(tc) e errj(tc). (7) Recognition signals could be interpreted as the likelihood of the observed motion to be the corresponding simulated action. These are similar to the responsibility signals in (Haruno et al., 2001). However, instantaneous error is used in responsibility signal calculation, which is prone to noise and the authors also point this out to be problematic. During the recognition phase, if the ratio of the highest recognition signal and the second highest gets above a certain threshold (Υ), the recognition decision is made and the behavior corresponding to the highest signal is chosen. In case there are multiple objects in the environment, behaviors are simulated for each object, by calculating hand-object relations accordingly. If we have m behaviors and n objects in the scene, then m n behaviors are simulated and compared. This also allows to recognize which object is being acted upon. We do not learn a separate network for the same behavior on different objects. We use the same neural network for learning and generating the trajectory of one behavior on all objects; only changing inputs according to the object. The flow-chart of the overall recognition architecture is depicted in figure Experimental Framework 4.1 Setup There are two objects, a motion capture system, an actor (human) and an observer (robot) in our setup. The actor and the observer face each other and the objects are hanging in between as shown in figure 2. The motion capture system is used for tracking the centers of the objects and the end-effector of the ac-

5 tor. We used three reaching behaviors: R(X): Reaching object X from right D(X): Reaching object X directly L(X): Reaching object X from left which can be applied to one of the two objects: LO: Left object (wrt. the actor) RO: Right object (wrt. the actor) The use of multiple behaviors on multiple objects aimed to show that the proposed method can learn and recognize different behaviors for different goals. These behaviors and objects are depicted in figure 3. The actor can apply one of the behaviors on one of the objects. Thus, the recognition decision has to be made among six choices. Note that we do not use a different neural network for all behavior-object pairs but use a different one for each behavior. The inputs given to the same network changes between objects since hand-object relations are different. Figure 3: Depiction of behaviors and approximate geometric information from top view of the setup hand trajectories and the object centers are recorded with the motion capture system. The beginning and the end of the recordings are truncated automatically to capture the start and the end of the actual motion, using a velocity threshold (0.05m/s). Samples from the recorded trajectories are shown in figure 4. 60% of the processed recordings are used for training and the rest are left for testing. For different threshold (Υ) values, we have recorded decision times for each testing sample and measured the success rates. We have selected K = 10 and D = 2 10 empirically. The simulate-observe-compare loop of the approach is run with the same frequency as the motion capture system, 30Hz. Figure 2: Experimental setup We used the VisualeyezTM VZ motion capture system (Phoenix Technologies Incorporated). This device measures the 3D position of active markers, which are attached to the points of interest. In our setup, the markers are attached to the centers of the objects and to the right wrist (the end-effector) of the actor as shown in figure 2. The wrist is chosen to minimize occlusion during tracking. The frequency of the sensor is set to 30Hz per marker. 4.2 Experiments The actor performed 50 repetitions of each behavior on each object (a total of 300 repetitions) and the 1 Figure 4: A subset of recorded human trajectories 5. Results and Discussions We plotted the time evolution of recognition signals for nine different recordings in figure 5. In all plots, recognition signals start from the same initial value, 1 6, corresponding to 6 possible choices (3 actions 2 object). As the action unfolds recognition signal of the corresponding action goes to one while suppressing others. Although there may be a confusion in the beginning parts of an action (e.g. top right plot), our system recognizes it correctly as it unfolds. According to the chosen threshold value Υ, there

6 Figure 5: The time evolution of recognition signals. X-axis is time in seconds and Y-axis is recognition signal magnitude. Figure 6: Recognition rate and decision time (percentage completed) vs. threshold value Υ. Percentage completed is defined as decision time divided by action completion time. is a trade-off between the decision time and the success rate. To optimize Υ, we recorded success rates and mean decision times (for correct decisions) for different Υ values which are shown in figure 6. We chose Υ as 1.9 to obtain 90% recognition rate. On the average, the system makes a decision when 33% of the observed action is completed with this chosen Υ value. Figure 7 plots the histogram of the decision times for this Υ value. Table 1 shows the confusion matrix for the chosen Υ value. Cases where the object acted upon is not correctly decided is low: 25% of wrong decisions and 2.5% of all decisions. Since we want to make recognition before the action is completed, there are Figure 7: Distribution of decision times (percentage completed) for Υ = 1.9 confusions between reaching right of the left object and reaching left of the right object (See figure 3). Also, there is a confusion between reaching directly and reaching from left for the left object. These are expected since human motion has noise and variance between repetitions and a demonstrator may not give the initial curvature expected from the action every time (see figure 4). 6. Robot Demonstration To demonstrate that our system works in an online fashion, we tested it in an interactive game with the humanoid robot icub. The robot s head and eyes have a total of 6 DOF and arms have 7 DOF each, excluding the hands (Metta et al., 2007). It also has

7 Figure 8: Demonstrations with the robot: Each row shows a different demonstration. The first column shows the starting point of the actions. The second column shows the point where the system recognizes the action (indicated by the eye-blink). The third column is the point where the demonstrator finishes his action and the last column is the point where the robot finishes its action. Table 1: Confusion matrix for Υ = 1.9 (Recognition rate = 90%) Object LO RO Object Behavior R D L R D L R LO D L R RO D A facial expressions implemented by leds and moving eyelids. For our demonstration, robot was seated in front of the demonstrator, on the other side of the objects, as shown in figure 2. The interactive game is as follows: The actor applies one of the actions to one of the objects. The robot immediately raises its eyebrows and blinks when it recognizes the action and reacts by turning his head to the predicted object. The robot also makes a hand-coded counter action which is defined as bringing its hand on the opposite side of the object. The reason for using hand-coded actions is that action generation is out of the scope of this paper. DMPs have already been shown to have good generation characteristics. Snapshots from a captured video of the game can be seen in figure 8. This figures show that our action recognition method can be used for online interaction with robots. 7. Conclusion We demonstrated an online recognition architecture which can recognize an action before it is completed. Our recognition architecture is based on a generation system where we have modified DMPs to overcome some of their shortcomings. Our modifications remove the dependence of DMPs on the initial position and time (i.e., internal parameters). Also, we have changed recognition approach in (Ijspeert et al., 2003) to allow for online recognition. We have defined recognition signals to have a quantitative way for measuring similar actions. The temporal profile of our recognition signals are similar to mirror neuron responses, although more experiments and comparison should be done to claim functional similarity. Although seemingly similar, our work differs from HAMMER and MOSAIC in major aspects. We are working in trajectory space and our controller and trajectory generator are decoupled; i.e., we do not need to calculate motor commands for recognition. Our architecture has the ability to learn actions by demonstration, which is absent in HAMMER and MOSAIC. Both in HAMMER and MOSAIC, it is assumed that the actor and the observer have sim-

8 ilar internal models, which cannot be guaranteed. HAMMER is more suited for recognizing straight line trajectories. Although a solution is proposed in (Demiris and Simmons, 2006) by using via-points, everything needs to be calculated offline. Our recognition metric, recognition signals, is robust against noise since it uses the cumulative error unlike MO- SAIC and MSI. Acknowledgements This research was supported by European Commission under the ROSSI project(fp ) and by TÜBİTAK under the project 109E033. Barış Akgün acknowledges the support of the TÜBİTAK 2228 graduate student fellowship and Doruk Tunaoğlu acknowledges the support of the TÜBİTAK 2210 graduate student fellowship. References Demiris, J. and Hayes, G. (2002). Imitation as a dual-route process featuring predictive and learning components; a biologically-plausible computational model. In Dautenhahn, K. and Nehaniv, C., Imitation in animals and artifacts. MIT Press. Demiris, Y. and Khadhouri, B. (2006). Hierarchical attentive multiple models for execution and recognition of actions. Robotics and Autonomous Systems, 54: Demiris, Y. and Simmons, G. (2006). Perceiving the unusual: Temporal properties of hierarchical motor representations for action perception. Neural Networks, 19: Gallese, V., Fadiga, L., Fogassi, L., and Rizzolatti, G. (1996). Action recognition in the premotor cortex. Brain, 119(2): Haruno, M., Wolpert, D. M., and Kawato, M. M. (2001). Mosaic model for sensorimotor learning and control. Neural Computation, 13(10): Ijspeert, A., Nakanishi, J., and Schaal, S. (2001). Trajectory formation for imitation with nonlinear dynamical systems. Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180), pages Ijspeert, A. J., Nakanishi, J., and Schaal, S. (2003). Learning attractor landscapes for learning motor primitives. In Becker, S., Thrun, S., and Obermayer, K., (Eds.), Advances in Neural Information Processing Systems, volume 15, pages MIT-Press. Metta, G., Sandini, G., Vernon, D., Natale, L., and Nori, F. (2007). The icub cognitive humanoid robot: An open-system research platform for enactive cognition. In 50 Years of Artificial Intelligence, pages Springer Berlin / Heidelberg. Oztop, E. and Arbib, M. A. (2002). Schema design and implementation of the grasp-related mirror neuron system. Biological Cybernetics, 87: Oztop, E., Kawato, M., and Arbib, M. (2006). Mirror neurons and imitation: A computationally guided review. Neural Networks, 19: Oztop, E., Wolpert, D., and Kawato, M. (2005). Mental state inference using visual control parameters. Cognitive Brain Research, 22: Rizzolatti, G., Fadiga, L., Gallese, V., and Fogassi, L. (1996). Premotor cortex and the recognition of motor actions. Cognitive brain research, 3: Schaal, S., Ijspeert, A., and Billard, A. (2003a). Computational approaches to motor learning by imitation. Philosophical Transactions: Biological Sciences, 358: Schaal, S., Peters, J., Nakanishi, J., and Ijspeert, A. (2003b). Learning movement primitives. In International Symposium on Robotics Research. Springer. Tani, J. (2003). Learning to generate articulated behavior through the bottom-up and the topdown interaction processes. Neural Networks, 16: Tani, J., Ito, M., and Sugita, Y. (2004). Selforganization of distributedly represented multiple behavior schemata in a mirror system : reviews of robot experiments using RNNPB. Neural Networks, 17: Wolpert, D. and Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11: