Cooperative Movements through Hierarchical Database Search

Transcription

1 Proceedings of the th International Conference on Advanced Robotics (ICAR) Hong Kong, China, Jul 217 Cooperative Movements through Hierarchical Database Search Miha Deniša, Bojan Nemec and Aleš Ude Abstract The paper tackles the problem of snthesizing robot movements for human robot collaboration. The proposed approach emplos a dual hierarchical database, which encodes multiple demonstrated human-robot collaborative movements. The primar database encodes demonstrated human movements and is enhanced with a directed weighted graph. It is used for human movement recognition. After recognition, the secondar database, encoding corresponding robot demonstrations, is used to snthesize appropriate collaborative movement. The proposed approach is evaluated through comparison to Interactive Primitives, a popular approach for snthesizing human robot collaborative tasks. Different sets from a database of two-dimensional human movements are used as eample sets for evaluation. I. INTRODUCTION The vast majorit of robots are still used in structured and controlled industr environments, where their tasks are preprogrammed b hand and repeated multiple times over a long period. In the past ears, the need and desire to move robots into unstructured environments, where the would need to perform a variet of changing tasks, is on the rise. Varing tasks and conditions make programming a robot b hand unsuitable. Alternativel, new servomotor knowledge can be gained through human demonstration b using programming b demonstration (PbD) [1], [2], [3]. Human movement can be observed with multiple approaches: optical or magnetic marker-based sstem [4], [5]; vision sstems based on RGB-D sensors [6] or stereo cameras [7]; or kinaesthetic guidance, where a human phsicall guides the robot [8], [9]. Robot trajectories can be learned using a single demonstration and, for eample, be encoded b dnamic movement primitives (DMPs) [1], [11]. Alternativel, multiple demonstrations can be used to gain a single robot trajector. A set of recorded movements was used to snthesize robot trajectories adapted to varing tasks within the training space [12], [13], [14]. Other representations of movement primitives that can be used with multiple demonstrations include Hidden Markov models [15], Gaussian miture models [16], [17], Probabilistic Movement Primitives [18], etc. Human robot collaboration is another important aspect of introducing robots into home environments and industr settings of small to medium enterprises (SMEs). The first step in a successful human robot collaboration is recognizing human intention, i.e., recognizing and/or classifing human movements. In the second step, appropriate collaborative Miha Deniša, Bojan Nemec and Aleš Ude are with Humanoid and Cognitive Robotics Lab, Dept. of Automatics, Biocbernetics and Robotics, Jožef Stefan Institute, Ljubljana, Slovenia {miha.denisa,bojan.nemec,ales.ude}@ijs.si movement must be snthesized and eecuted an a robot. Various authors tackled human robot collaboration using Hidden Markov Models (HMMs). Lee et al. [19] learned responsive robot behaviors using hierarchical HMMs. A path-map HMM was used [2] to model interaction between two demonstrators while eecuting a collaborative task. An etension of DMPs, called Interaction Primitives (), was proposed b Ben Amor et al. [21]. The used a demonstrated set of a single collaborative movement with probabilistic approaches to predict DMP distributions of the interactive primitive. The work most related to our proposed approach, was done b Yamane et al. [22], [23]. Similarl to our work, the used a binar tree database to recognize human movement and then snthesized an appropriate robot reaching movement. We propose an approach based on an etended hierarchical database of demonstrated cooperative movements. A single hierarchical database was previousl used to snthesize new movements from a set of eample trajectories [24]. The single database was later etended in order to generate new compliant movements [25]. While kinematic trajectories were encoded in the primar database, corresponding torques were stored in the secondar part of the hierarchical database. The rest of the paper is structured as follows. While hierarchical database construction and its etension is presented in Section 2, Section 3 tackles motion recognition and cooperative movement snthesis. Evaluation and comparison to is given in Section 4. Concluding remarks are presented towards the end of the paper. II. HIERARCHICAL DATABASE The process of building the hierarchical database () begins with capturing a set of n S demonstrated cooperative movements, S D = {D 1, D 2,...,D ns }, (1) where each demonstration consists of human and corresponding robot movements. Each demonstration consist of n D state vectors sampled at a given discrete time t D, where each state vectors, D i =[ 1, 2,..., nd ], (2) i =[ h i, r i ] T, (3) includes state vectors i h belonging to human trajectories and state vectors i r from the captured robot movement. If the set consist of demonstrations with different durations, then the number of state vectors n D will var between them. Both human and robot state vectors can be defined differentl /17/$ IEEE 4

2 Fig. 1. A simple representation of an eample. The primar part of the database can be seen on the left. It is used for construction and motion recognition and stores human movements. The secondar part, which encodes corresponding robot movements, can be seen on the right. State vectors for end-effector trajectories can, for eample, be specified in Cartesian space and defined as i =[p i, ṗ i,p i, ṗ i,p zi, ṗ zi ], (4) where p ji and ṗ ji, j =,, z, denote the position and velocit at time t i. If the robot trajectories are, for eample, given in joint space, then state vectors can be defined as i =[q 1i, q 1i,q 2i, q 2i,...,q di, q di ], (5) where the j-th joint angle and its velocit at time t i are denoted b q ji and q ji, and d is the number of the robot degrees of freedom (DOFs). In the net step all human state vectors belonging to all of the demonstrated trajectories incorporated into the database are concatenated in a human sample motion matri, Y h =[ h 1, h 2,..., h n Y ], (6) where n Y denotes the sum number of state vectors in all n S demonstrated trajectories. The sample motion matri is used to build the primar part of a binar tree like. See Fig. 1 for a simple representation of an eample. While onl the human state vectors are used to build the primar part of the, the robot state vectors are represented in the secondar part. The human sample motion matri represents the root node and is divided via clustering into 2 child nodes, which represent the 2 nd level. While a variet of clustering algorithms is available, the k-means algorithm proved to be best suited for building a [24]. In order to gain nodes at the 3 rd level, clustering is preformed on the two child nodes in 2 nd level. The clustering continues at each level until all nodes meet a stop criterion. This criterion is based on the variabilit of state vectors associated with a node. In order to represent all the data on each level of the, each branch is etended to the last level b coping the leaf nodes. For each node v the mean of associated state vectors h v is calculated and stored. As all the demonstrated data is clustered, the time component and the demonstrated sequence of state vectors is lost. The original sequence is substituted through directed weighted graphs. This transition graph is constructed at each level of the (see Fig. 1). It represents all transitions between nodes at that level, where the graphs edge weights represent the probabilit of transition from one node to another. The weights are based on the number of transitions between nodes observed in the original demonstrated data. The details regarding construction, including on the stop criterion and transition graphs, are omitted in this paper. The reader is referred to [24], [25]. With the primar part of the constructed, the secondar part, which encodes robot state vectors, is build. At each level the nodes are copied from the primar part. This means that each node in the secondar database encodes robot state vectors recorded at same sample times as their counterparts in the human part of the. Again, the mean of associated state vectors r v is calculated and stored for each node v. See Fig. 1 for a simple representation of both parts of the database. 41

3 III. COOPERATIVE MOVEMENTS To successfull snthesize appropriate cooperative robot movements, the current eecuted human movement must first be recognized. While the first part of this section focuses on movement recognition, the second part tackles cooperative movement snthesis. A. Movement Recognition Human movement recognition is done b hierarchical search through the primar, which encodes demonstrated human movements. It starts b updating the sliding window, i.e., the sequence of last n W state vectors observed in the current captured human movement { o 1, o 2,..., o n W }, (7) where o denotes observed state vectors. The size of the sliding window n W is a compromise between the speed of recognition and the confidence of the result. Throughout the evaluation presented in this paper size n W =4was used. Recognition is done b traversing through the levels of the basic, with multiple steps done at each level l: 1) Establish considered nodes v c at current level l. These nodes are children of all the nodes that were below the cut-off range at the previous level. At the first level of the used for recognition, usuall level 3, all the nodes are denoted as considered nodes. 2) Matri of considered nodes at current level P l is build, where each row contains a permutation of considered nodes v c and has the length of the sliding window n W. The matri includes all possible permutations. 3) The recognition score for each permutation set is calculated, R = n W i=1 n W 1 d(i o,vi c )+ τ(vi c,vi+1), c (8) i=1 where d(i o,vc i ) denotes Euclidian distance between the observed human state vector i o and the mean state vector corresponding to the considered node vi c. The second term, τ(vi c,vc i+1 ), denotes the transition probabilit between the considered nodes, which is derived from the transition graph at the current level. 4) Determine the nodes belonging to permutations with recognition score R above the cut-off range, which is a compromise between the speed of recognition and the confidence of the result. Onl the children of these nodes are considered as we move to the net level of the. When the last level is reached, the permutation of nodes with the highest recognition score is considered as the recognized human sequence, i.e., nodes relating to the observed human movement in the current sliding window. B continuing from this sequence through the transition graph, following the highest weighted edges, the most probable subsequent sequence of human nodes, i.e., human path, can be inferred. See Fig. 2 for a simple representation. B. Cooperative Movement Snthesis The most probable human path, found b traversing the primar part of, can now be transferred into the secondar part of the, encoding cooperative robot movements. We denote these corresponding sequence of nodes as a robot path. The time component of demonstrated movements was lost in the process of building the. In order to enhance the robot path with time stamps, a duration t v of a single node v is estimated as t v = n v m v t D, (9) where n v denotes the number of state vectors clustered in node v and m v denotes the number of demonstrated trajectories passing through it. If these estimated node durations are combined with the state vectors means r v, the most probable robot movement can be denoted as a sequence {( r 1,T 1 ), ( r 2,T 2 ),...,( r n R,T nr )}, (1) where the number of nodes in the robot path is denoted b n R. Time stamps for each node s mean state vector are defined as { i =1 T i = t vi 1 +t vi 2 i>1. (11) In the last step DMPs are used to encode each dimension of the most probable robot movement. The initial position values for the DMP is set to the current robot position. The details on DMPs are omitted in this paper and the readers are referred to [1], [11]. Primar Part of the Secondar Part of the Fig. 2. A simple representation of a recognized human sequence, most probable human and robot path. The most probable sequence of nodes, representing recognized human sequence, is denoted in blue. It was found b traversing the primar with the currentl observed human movement. B continuing through the transition graph, the most probable subsequent sequence of human nodes can be inferred. We denote this as a human path. It can be seen in the primar part of the, marked in green. B transferring it to the secondar part oh the, the most probable robot path can be found. It is represented b a sequence of nodes corresponding to the nodes in the human path. A simple representation can be seen in the secondar part of the, denoted in green. 42

4 Angle BendedLine CShape DoubleBendedLine GShape JShape JShape2 Khamesh Leaf1 Leaf2 NShape PShape Trapezoid Worm Fig. 3. Used sets from the LASA database. Eample sets used in our evaluation, where blue circles mark common starting points for all trajectories in a single set. IV. EVALUATION The proposed approach was evaluated using the LASA database [26] consisting of two-dimensional point-to-point human handwriting motions. While one of the sets was used as a substitute for demonstrated human movements, another represented cooperative robot movements. The LASA trajectories were slightl modified, i.e., reversed. See Fig. 3 for LASA eample sets used in our evaluation. The proposed approach of snthesizing cooperative movements using search was evaluated through comparison to Interactive Primitives () proposed b Ben Amor et al. [21]. A. Interactive Primitives Interactive Primitives () are build on a DMP framework. The determine the distribution over DMP parameters and use it to infer further movement. The approach can be divided into three major parts: phase estimation, predictive DMP distribution, and agent correlation. The phase estimation is used to temporall align human and robot movements. A part of the Dnamic Time Warping approach is used to estimate the number of frames alread eecuted in the human observed reference movement and in turn estimate the current DMP phase. In the net step predictive DMP distributions are calculated. Each demonstration is encoded as a DMP and its parameters (weights and goal) are stored in a single parameter vector θ. Using multiple human demonstrations, distribution over parameters p(θ) can be determined. The distribution and a partiall observed reference human trajector Do h is used to obtain an updated parameter distribution p(θ Do h ), which is in turn used to predict the future human movement. The last step is achieved b etending predictive DMP distributions. Both human and robot demonstrated movements are used, but the conditioning is done on just human movements. The updated distribution over robot DMP parameters is then used to predict the cooperating robot movement. Further details on are omitted and the reader is referred to [21]. B. Comparison For the purpose of evaluation, sets from the LASA database were used as demonstrations. Used sets, each containing 7 trajectories, can be seen in Fig. 3. Ten pairs of sets were used, where one set represented human and the other robot trajectories. A leave-one-out-cross-validation (LOOCV) was emploed in order to evaluate both approaches. For each of 1 pairs, 7 human eample databases and 7 corresponding robot eample databases were constructed, each missing one human and one robot trajector. In each case the missing human trajector represented the observed trajector, while the corresponding missing robot trajector took part of the reference trajector, i.e., ground truth. For each of these cases, both approaches were emploed under 9 different conditions. s represented different percentage of the observed trajector used as input for both approaches. Each time both approaches snthesized the desired corresponding robot trajector as an output. It was compared to the reference trajector, i.e., the robot trajector left out in the database in the current LOOCV iteration. In order to compare the position part of the two trajectories, arc length parametrization was used. The similarities between the two trajectories was then measured b point-wise mean squared error. Two eample pairs of LASA sets can be seen on top of Fig. 4. Results for each eample pair are presented below each pair. For each condition the means and standard deviations of values over all LOOCV iterations are showed. The approach produces quite high if less then 15% of the observed trajector is used as input. When more then 15% of the observed trajector is used, the approaches produces similar values. We can also observe that approach produces higher errors when the human and robot sets are less similar (eample pair on the left side). Fig. 5 presents mean and standard deviation values for all 1 pairs of sets used in our evaluation. We can again observe high values when using approach and less then 15% of the observed trajector. The right side of the 43

5 Human movements - GShape 3 3 Robot movements - JShape Human movements - T rapezoid 3 15 Robot movements - Worm Mean Square Error Mean Square Error Fig. 4. Two eample pairs of sets used. The two figures in the top left show the LASA sets used as a human database and robot database, respectivel. The means and standard deviation of, showed in the bottom left figure are gained b using this pair of sets for evaluating both approaches. A different pair of LASA sets and corresponding results are depicted in the three figures on the right. figures shows in more detail b ecluding the first two condition values. When more then 25% of the observed trajector is used as input, both approaches produce similar results. In the last part of the evaluation, calculation times needed from both approaches were estimated. We need to note here that calculation times were gained b eecuting both approaches with a non-optimized Matlab code. Their values are intended to indicate how the approaches compare to each other and how the change w.r.t the conditional vector and do not represent optimum times. Means and standard deviations for calculation times over all used pairs of sets can be seen in Fig. 6. As the approach keeps the sliding window size constant, calculation times remain similar throughout the percentage of the observed trajector. When using calculation time rises with the size of the input. This is due to the DTW calculation used for phase estimation Fig. 5. over all used sets. The left figure shows means and standard deviations for over all reference trajectories in all used pairs of LASA sets. represents the percentage of observed human trajector. The right figure shows mean and standard deviation values in more detail b ecluding first two condition values. We can see that both approaches produce similar values once the percentage of the observed trajector is above 25 %. Below that percentage, outperforms. 44

6 Calc. Time Fig. 6. Needed calculation time over all used sets. The figure shows means and standard deviations for needed calculation time over all reference trajectories in all used pairs of LASA sets. represents the percentage of observed reference human trajector. When using, calculation time is highl dependent on the size of the input. As approach uses a constant sliding window size, the calculation times remain constant. *The calculation times are far from optimum times. The were gained b eecuting both approaches with a non-optimized Matlab code. Their values are intended to indicate how the approaches compare to each other and how the change w.r.t the conditional vector. V. CONCLUSIONS We presented an approach for snthesizing collaborative robot movements from a hierarchical database of demonstrated human-robot tasks. Evaluation was done through comparison to while using a LASA database of demonstrated human movements. LOOCV evaluation on different pairs of sets was performed. We observed a better performance from HDB in movement snthesis, while using the start of the reference human trajector as input. As the observation got longer and encompassed more of the reference trajector, both approaches performed equall good. While performed faster with small observations parts, the became slower as the input trajector got longer. On the other hand, the calculation time for remained approimatel the same. We should note, that widening the sliding window would greatl effect the needed calculation time while using. We should also note some theoretical differences between the two approaches. While our approach can onl snthesize robot movements from demonstrations, can snthesize trajectories within the range of demonstrations. On the other side, can onl handle a set of similar demonstrations, while can handle a database of diverse movements. While needs time snchronization, it is inherit in the proposed approach. As our approach is related to the work of Yamane et al. [22], [23] some main disparities need to be noted. The main difference with regards to building the primar database is the stop clustering criterion. Where the observe number of nodes in clusters, we also take into account the variabilit of the data in each cluster. With regards to recognition, the compare probable sequence of nodes to the ones found w.r.t. the sliding window in the previous step and penalize changes. With this the avoid sudden changes in the snthesized robot movement. But if the human wishes to change the reference movement during recognition, the approach will be slow to change. Instead of comparing probable sequences and penalizing changes, we ensure the reference change is smooth and continuous on the robot side b encoding the probable sequence with DMPs. We also infer future human movement b traversing the TG. Future work includes evaluating approach using a real human-robot collaborating scenario using demonstrations with higher dimensionalit. ACKNOWLEDGMENT The research leading to these results has received funding from the Horizon 22 ICT-FoF Innovation Action no , AUTOWARE (Wireless Autonomous, Reliable and Resilient ProductIon Operation ARchitecture for Cognitive Manufacturing). REFERENCES [1] S. Schaal, Is imitation learning the route to humanoid robots?, Trends Cogn. Sci., vol. 3, no. 6, pp , [2] C. Breazeal and B. Scassellati, Robots that imitate humans, Trends Cogn. Sci., vol. 6, no. 11, pp , 22. [3] R. Dillmann, Teaching and learning of robot tasks via observation of human performance, Robot. Auton. Sst., vol. 47, no. 2, pp , 24. [4] N. S. Pollard, J. K. Hodgins, M. J. Rile, and C. G. Atkeson, Adapting human motion for the control of a humanoid robot, in Proc. IEEE Int. Conf. Robotics and Automation (ICRA), (Washington, DC, USA), pp , 22. [5] A. Ude, C. G. Atkeson, and M. Rile, Programming full-bod movements for humanoid robots b observation, Robot. Auton. Sst., vol. 47, no. 2, pp , 24. [6] J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, M. Cook, and R. Moore, Real-time human pose recognition in parts from single depth images, Commun. ACM, vol. 56, no. 1, pp , 213. [7] T. B. Moeslund, A. Hilton, and V. Krüger, A surve of advances in vision-based human motion capture and analsis, Comput. Vis. Image Und., vol. 14, no. 2, pp , 26. [8] M. Hersch, F. Guenter, S. Calinon, and A. Billard, Dnamical sstem modulation for robot learning via kinesthetic demonstrations, IEEE Trans. Robot., vol. 24, no. 6, pp , 28. [9] L. Rozo, S. Calinon, D. Caldwell, P. Jiménez Schlegl, and C. Torras, Learning collaborative impedance-based robot behaviors, in In Proc. AAAI Conference on Artificial Intelligence, (Bellevue, WA, USA), pp , 213. [1] S. Schaal, P. Mohajerian, and A. Ijspeert, Dnamics sstems vs. optimal control a unifing view, Prog. Brain Res., vol. 165, pp , 27. [11] A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, Dnamical movement primitives: learning attractor models for motor behaviors, Neural Comput., vol. 25, no. 2, pp , 213. [12] A. Ude, A. Gams, T. Asfour, and J. Morimoto, Task-specific generalization of discrete and periodic dnamic movement primitives, IEEE Trans. Robot., vol. 26, no. 5, pp , 21. [13] D. Forte, A. Gams, J. Morimoto, and A. Ude, On-line motion snthesis and adaptation using a trajector database, Robot. Auton. Sst., vol. 6, no. 1, pp , 212. [14] H. Ben Amor, G. Neumann, S. Kamthe, O. Kroemer, and J. Peters, Interaction primitives for human-robot cooperation tasks, in IEEE Int. Conf. Robotics and Automation (ICRA), (Hong Kong, China), pp , 214. [15] T. Inamura, I. Toshima, H. Tanie, and Y. Nakamura, Embodied smbol emergence based on mimesis theor, Int. J. Robot. Res., vol. 23, no. 4-5, pp ,

7 [16] S. Calinon, F. D halluin, E. L. Sauser, D. G. Caldwell, and A. G. Billard, Learning and reproduction of gestures b imitation, IEEE Robot. Automat. Mag, vol. 17, no. 2, pp , 21. [17] S. M. Khansari-Zadeh and A. Billard, Learning stable nonlinear dnamical sstems with gaussian miture models, IEEE Trans. Robot., vol. 27, no. 5, pp , 211. [18] A. Paraschos, C. Daniel, J. R. Peters, and G. Neumann, Probabilistic movement primitives, in Advances in Neural Information Processing Sstems 26 (C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, eds.), pp , Curran Associates, Inc., 213. [19] D. Lee, C. Ott, and Y. Nakamura, Mimetic communication model with compliant phsical contact in humanhumanoid interaction, The International Journal of Robotics Research, vol. 29, no. 13, pp , 21. [2] H. B. Amor, D. Vogt, M. Ewerton, E. Berger, B. Jung, and J. Peters, Learning responsive robot behavior b imitation, in Intelligent Robots and Sstems (IROS), 213 IEEE/RSJ International Conference on, pp , IEEE, 213. [21] H. B. Amor, G. Neumann, S. Kamthe, O. Kroemer, and J. Peters, Interaction primitives for human-robot cooperation tasks, in Robotics and Automation (ICRA), 214 IEEE International Conference on, pp , IEEE, 214. [22] Y. Yamane, K.and Yamaguchi and Y. Nakamura, Human motion database with a binar tree and node transition graphs, Autonomous Robots, vol. 3, no. 1, 211. [23] K. Yamane, M. Revfi, and T. Asfour, Snthesizing object receiving motions of humanoid robots with human motion database, in In Proc. IEEE Int Conf on Robotics and Automation (ICRA), (Karlsruhe, German), pp , 213. [24] M. Deniša and A. Ude, Snthesis of new dnamic movement primitives through search in a hierarchical database of eample movements, Int J Adv Robot Sst., vol. 12, no. 137, 215. [25] M. Deniša, T. Petrič, T. Asfour, and A. Ude, Snthesizing compliant reaching movements b searching a database of eample trajectories, in In Proc. IEEE Int Conf on Humanoid Robots, (Atlanta, GA, USA), pp , 213. [26] S. Khansari-Zadeh, Benchmark Data. amarsi-project.eu/open-source, 212. [Online; accessed 12-Januar-217]. 46