Dynamic Movement Primitives for Human Robot interaction

Transcription

1 Workhop on Robot Motion Planning: Online, Reactive, and in Real-time 22 IEEE/RSJ International Conference on Intelligent Robot and Sytem, IROS 22 Vilamoura, Algarve, Portugal, October 7-2, 22 Dynamic Movement Primitive for Human Robot interaction Miguel Prada and Anthony Remazeille* Abtract A pecialization of the generic Dynamic Movement Primitive (DMP) framework i propoed in thi article to correctly addre a key activity for human robot collaboration that i object exchange. A a firt tep toward implementing thi challenging kill, thi paper focue on the arm motion to reach the initially unknown exchange ite. Two improvement related with thi application are propoed. Firt of all a better control of the tranition in between the two main component of the DMP repectively providing a kill hape-attractor and a goal-attractor i decribed, enabling to define when and how the tranition in between thee two component occur. Then an extenion to handle ituation where the goal poition varie along time i propoed, which improve the convergence of the trajectory toward a moving target (i.e. the human partner hand). Thee two improvement are validated by comparing the obtained behavior with human obervation realized through motion capture. I. INTRODUCTION The realization of robotic tak in non completely controlled environment require to provide the robotic ytem with a motion control cheme that adapt it behavior to the oberved ituation. Senor-baed approache uch a viual ervoing [3] define the control law a a cloed loop minimization of the error oberved in between the current and deired viual feature value. Depending on the framework ued, the robot motion can be optimal in the configuration pace or in the image feature pace. However, thee approache, in their baic verion, are trongly goal-driven and do not allow reproducing more complex kill in which the whole motion profile i a important a the convergence toward the goal. The learning of complex behavior can be addreed by programming by demontration approache, in which the robot imitate a tak demontrated either by a human operator oberved with a motion capture ytem, or by manually moving the robot itelf.statitical approache are frequently ued for the learning. In [], Hidden Markov Model are ued to recognize and reproduce nine different full body expreion by a imulated humanoid. Calinon et al. propoe in [2] to combine Gauian Mixture Model and Gauian Mixture Regreion to reproduce everal graping tak taught through kinethetic. The Dynamic Movement Primitive (DMP) method i another approach tudied in that field. Initially introduced by Ijpeert et al. [], the DMP approach relie on a non-linear dynamical ytem forced to *Thi work wa upported in part by the CogLaboration European project under contract FP7-ICT , and by the Fluent National project under contract DPI M. Prada and A. Remazeille are with the Aitive Technologie Department of the Health Diviion of Tecnalia Reearch and Innovation, Parque Tecnológico de San Sebatin, E-29 Donotia - San Sebatián, Spain follow a deired trajectory by a parametric forcing term. It i propoed in thi article to pecialize the baic DMP framework to the pecial cae of human-robot interaction during an object tranfer. Phyical human-robot interaction, and pecifically object exchange, i a key apect to get a fluid and efficient human robot collaboration. Several recent work are focuing on thi pecific ituation: [6] how that the human partner can reduce the complexity of thi tak by adapting to the robot behavior; [8] implement different velocity profile for the robot, and compare the reult with human-human exchange procedure; direct v. indirect (placing the object on a flat urface for the peron to grap) exchange procedure are compared in [4]; and [] focue motly on making the robot tranmit the intent of performing an exchange. In [3] the concrete exchange procedure i handled within an offline planning cheme. The A algorithm i ued to etimate the bet trajectory to exchange the object with the human partner, baed on a 3D cot map which combine three cot function focued on afety, viibility and arm convenience criteria. Once the optimal exchange path i obtained, the actual trajectory to follow i computed with the Soft Motion Trajectory planner, allowing active control of maximum jerk, acceleration and velocitie [2]. Neverthele, the obtained trajectory plan i not explicitly driven by the human obervation, and neither deigned to adapt to the partner behavior, which i omething inherent to the DMP approach propoed here. It i furthermore proved here that the initial tage of exchange location can be kipped by adapting accordingly the DMP framework. Thi paper i propoing a DMP pecialization for realizing human robot object exchange. A a firt tep, the focu i et on the definition of the control ytem to bring the robotic arm toward the exchange ite. Two improvement of the baic DMP framework are propoed, in relation with the exchange application. The firt one i related to a better control of the tranition between the feed-forward and feedback component of the DMP by introducing a cutom weighing function. The econd one addree the dynamic nature of the goal poition in exchange motion; a velocity baed feedback term i appended to the DMP ytem which improve convergence with the moving goal. Thi preent paper i organized a follow: next ection provide the needed background related to the DMP. Section III decribe the two extenion propoed, and the lat ection compare the reulting cheme behavior with real human-human exchange data recorded with motion capture equipment.

2 II. DYNAMIC MOVEMENT PRIMITIVES A. Original formulation The DMP framework learn a trajectory from jut one reference ample. It can then reproduce it and optionally adapt it to different configuration. Thi i achieved by uing a econd order linear dynamical ytem (i.e. a damped pring-like model) which i timulated with a non-linear forcing term. Let x(t) denote a one-dimenional trajectory tarting at x(t ) = x toward x(t f ) = g. In the original DMP framework the following ytem i introduced [9]: τ v = K(g x) Dv + (g x ) f () τẋ = v, (a) (b) with the forcing term f repreenting an arbitrary non-linear function a a um of weighted exponential bai function: and: f () = N i= ψ i()w i N i= ψ, (2) i() ψ i () = exp( h i ( c i ) 2 ). (3) The above dynamical ytem, named tranformation ytem by the author, i compoed of two driving component, aide of the global damping term Dv: K(g x) i an attractor toward the goal poition. (g x ) f () repreent the contribution of the non-linear forcing term caled by the g x factor. The variable on which the forcing term depend i a phae variable and it evolution i determined by the following decoupled linear ytem, called the canonical ytem: τṡ = α (4) Thi variable evolve exponentially from to. It i ued to remove the direct time dependency of the forcing term f (), and provide the complete ytem with a time calability by adjuting the parameter τ. The phae variable i alo ued to weigh the forcing term, enabling thi way to continuouly hift toward a purely goal-attracted ytem. When conidering multi-dimenional trajectorie, either the complete ytem above need to be replicated or, a propoed in [9], a common canonical ytem can be ued for all dimenion, with pecific tranformation ytem for each dimenion. B. Bio-inpired formulation In [7], Hoffmann highlight that thi formulation ha caling iue when the goal poition g i cloe to the trajectory tarting point x. Furthermore, thi model doe not adapt correctly to ituation where the goal parameter i et to the oppoite ide of the trajectory origin x with the repect to the original: the complete trajectory i then completely inverted. A lightly different bio-inpired model i thu propoed, baed on evidence obtained on in vivo tudie on frog. Thi modified DMP formulation i: τ v = K( f () + x x) + ( )K(g x) Dv (5a) τẋ = v (5b) Similarly, thi ytem i mainly compoed of two attractor field: The term K(g x) i an attractor toward the goal poition (from now on referred to a the goal-attractor). +x x) repreent an attractor toward f () + x (the hape-attractor). Each of thee attractor field ha it influence weighed according to the evolution of the phae variable: the hapeattractor, weighed by, i predominant in the beginning of the movement, when ; while the goal-attractor, weighed by ( ), i predominant in the end of the movement, a. Thi formulation bypae the iue ariing when the goal i cloe to the origin of the trajectory, and vatly improve the adaptation to new goal ince the hape-attractor doe not cale anymore with (g x ). Alo, the addition of the x component on the hape-attractor enable the ytem to behave properly when the initial tarting point i changed. Thee two propertie together make the ytem affine tranforminvariant when learning multi-dimenional trajectorie. The term K( f () the moving point C. Trajectory learning The learning procedure i the ame in both model. The firt tep i to give value to the parameter of the ytem: K and D involve the inherent dynamic of the econd order linear ytem, and determine it repone to online change in the goal parameter. τ i the time contant and hould be et to the duration of the ample trajectory τ = t f t. α determine the decay rate of the phae variable. A value α 4 will enure that.2 at t = τ. Once thee value are fixed, the next tep i to compute the deired value for the forcing term, by iolating it from (5a) (or (a) for the firt formulation), which reult in: f de () = K (τ v K(g x) + Dv + K(g x )) (6) and then inerting the value of the ample trajectory x = x(t), v = τẋ(t) and v = τẍ(t), by taking into account the nominal evolution of the phae variable = exp( α τ t). With thee deired value for the forcing term, the appropriate center and width of the bai exponential in (2) can be et, and the weight w i can be computed by fitting (2) to (6) by leat quare. D. Limitation with repect to the intended application Both the above formulation are quite enitive to variation in the goal from the very beginning, a illutrated on Fig., where a ample trajectory x(t) (black olid line) i learnt and reproduced with the goal changed from to.5 from the beginning. In the cae of the original formulation (red curve), the contribution of g in both the hape-attractor and goal-attractor (ee (a)) make both component cale when the goal i changed. In the cae of the bio-inpired formulation, a it can be een on (5a), the hape-attractor i not affected by the goal parameter. Neverthele, by tudying the evolution of the phae variable (Fig. 2)one can oberve

3 that more weight i given to the goal-attractor for t >.73τ (i.e. tarting at le than 2% of the trajectory duration). Thu, from thi early moment, any variation of the goal with repect to the reference one ha a trong effect which override the influence of the hape-attractor term. A previouly mentioned, the application we are conidering i the arm control during an object exchange with a human partner. The involvement of the human in the loop require the robotic ytem to deal with the exchange location uncertainty. It alo naturally contraint the robot motion to be human-friendly or fluent. One of the mean to improve the fluency of object exchange i to overlap the motion of the robot with the motion of the human partner, without waiting for the human to reach a table poition to tart moving. A olution to achieve thi i to launch the robot motion uing an etimation of the exchange ite, a propoed in [3]. Neverthele, thi initial gue would till need to be adjuted on-line to adjut the robot motion to the human behavior. To avoid thi initial etimation, we are propoing to et the DMP goal to the current poition of the hand of the human partner from the beginning of the movement generation. Thi enable to enure the convergence toward the exchange ite (which i currently aumed to be the human final hand location). Neverthele, from thi perpective, the fact that the DMP generator i too enitive to alteration in the goal parameter i conidered a a hortcoming, ince the initial goal fed to the ytem can be quite different to the poition reached by the non predictable human partner. In addition, the analyi of the human behavior ugget that reaching motion performed by human contain two ucceive component [5]: The onet of the movement i performed baed on imperfect target information and motly determined by an internal dynamical model and feed-forward control. The final part i dominated by viual feedback control, once the target poition information get more precie. Thi evidence upport the objective of initiating the movement with a dominantly feed-forward control policy, x(t) [m].5 ample Original DMP Bio inpired DMP Fig.. Senitivity of both the original (red curve) and the bio-inpired DMP (blue curve) generated motion with repect to a change in the goal. The black curve repreent the learn trajectory. (t).73τ τ Fig. 2. Evolution of the weight of the hape-attractor (in black) and the goal-attractor (in red) within the original and bio-inpired DMP model. and delaying the hift of weight toward the feedback component of the DMP tranformation ytem to later in the trajectory. Thi way the firt part of motion i mainly hapedriven, and le dependent on the goal variation, while the econd part take care of the convergence toward the goal. Next ection preent the propoed modification to the DMP method to achieve thi deired behavior. III. EXTENSION OF THE DMP MODEL A. Decoupled weighing function Two approache are conidered to modify the evolution of importance of each term driving the motion generation in the tranformation ytem: A change in the evolution of the phae variable can change the weight balance between the two component. Thi can be ued to delay the hift of importance from the hape-attractor toward the goal-attractor. A decoupling of the weight applied to each of the term in the tranformation ytem from the phae variable. Intead of weighing the attractor directly with the phae variable, an arbitrary function of the phae variable can be ued to compute the deired weight. The firt approach propoed require to find an appropriate ubtitute for the canonical ytem with the deired evolution, and in ome cae thi ytem might be difficult or even impoible to find without recurring to piecewie or untable ytem. Alo, changing the evolution of the phae variable by mean of altering the canonical ytem affect all the dimenion of the trajectory being reproduced by the DMP method. The econd approach i intereting in the ene that the canonical ytem can be kept in it original form. Furthermore, each of the tranformation ytem depending on the ame phae variable can ue a different weighing function if needed. Therefore it i decided to tick with thi econd approach which i conidered more veratile. The new ytem equation which ue the decoupling approach propoed are ( f w () and w g () are repectively

4 noted f w and w g for notational compactne): τ v = ( w g )( f w + x x) + w g K(g x) Dv τẋ = v τṡ = α, where f w () i now defined a: (7a) (7b) (7c) f w () = N i= ψ i()w i N i= ψ i(). (8) In comparion with (2) the phae variable i not included in the (8) anymore, ince it not longer required for the forcing term to fade away. Note that the gain K multiplying the hape-attractor in (5a) ha been dropped a well, ince it doe not have any effect at all on the ytem repone; thi i obviou by oberving how the deired value of the forcing term are computed with (6). It i propoed to ue a weighing function in the hape of a igmoid imilar to the Cumulative Ditribution Function (CDF) of the Normal ditribution. Thi function ha the advantage of relying on two parameter which eaily allow determining when the hift will occur (the mean µ of the Normal ditribution) and the duration of the hift (the tandard deviation σ of the ditribution). Fig. 3 how everal variation obtained by changing thee two parameter. The expreion for the function i, ubtituting the dependency on for dependency on time: w g (t) =.5 [ + er f ( t µ σ 2 )], (9) where er f tand for the Gau error function. Thi weighing function ha one problem, which become evident when the formula for the deired hape of the forcing term f w () i computed. To obtain the deired f w () one need to iolate it from (7a), reulting in: f w = ( w g ) (τ v w gk(g x) + Dv) x + x () (where the dependence on ha been dropped again for compactne). It i eay to ee that thi expreion tend to infinity a w g, thu cauing numerical iue. A imple product of the igmoid function with a linear term (e.g. tarting at.9 for t = and tending toward a t τ) olve thi problem, while till enuring that the hape-attractor influence i fading away when (i.e. t τ). Thi reult on the function hown in Fig. 4. By uing the Decoupled DMP formulation propoed in (7a), (7b) and (7c), the moment where the change of goal affect the output of the DMP algorithm can be adjuted at will. Fig. 5 illutrate the behavior of the new formulation propoed. The original trajectory learnt a well a the value of the goal et during execution are the ame a the one ued in Fig.. Three trajectorie are generated with different value of µ, howing how the ytem output i affected. In the three cae, the g parameter i et to it final value g =.5 from the beginning of the trajectory, but thi only affect the trajectory at the choen point in time. Notice that the rightmot cae, with µ =.7, witche to the goal-attractor too late for the trajectory to reach the goal at t = τ, although it will reach it hortly after, ince by that time the ytem i almot purely a table linear econd order ytem. B. Adpatation for dynamic goal A previouly mentioned, and a a firt implification, the DMP goal i et to the poition of the human partner hand. If [3] propoe to realize an off-line etimation of the bet exchange location, our approach preent the advantage of avoiding uch etimation, while maintaining a reactive proce o that the robot adapt to the human behavior and not the contrary. However, in ome cae, and even if the modification explained in the previou ection i in place, the fact of uing the human current hand poition a goal at each intant in the motion generation may introduce ome undeired ocillation in the reulting trajectory. The example on Fig. 6 how thi effect with a et of data from real human motion. In thi figure the black line how the original trajectory ued to learn the robot motion; the blue line how the oberved motion of the human partner, with whom the robot i performing the exchange operation; the red line repreent the generated trajectory (with the DMP modification wg(t) µ =.5τ σ =.τ µ =.5τ σ =.5τ µ =.5τ σ =.τ µ =.3τ σ =.5τ µ =.7τ σ =.5τ wg(t) Fig. 3..3τ.5τ.7τ τ Weighing function w g (t) with different et of parameter.3τ.5τ.7τ τ Fig. 4. Weighing function modified to avoid divide-by-zero numerical error uing the ame color code a in Fig. 3

5 x(t) [m].5. Original DMP Decoupled (µrel =.3) DMP Decoupled (µrel =.5) DMP Decoupled (µrel =.7) Fig. 5. Decoupled DMP with different value of µ and σ =.5. preented) a repone to the oberved movement. It can be een that, given that the partner poition i lagging with repect to the robot one when the hift of weight i done in favor of the goal-attractor, the robot motion revere for a certain time lape. Thi ocillation i not deired, and a gentle deceleration would be much more convenient. To alleviate thi iue, a modification of the model i propoed which improve the moothne of the convergence toward a moving goal. Thi modification conit in adding a velocity feedback term to the tranformation ytem, reulting in: τ v = ( w g )( f w + x x) + w g [K(g x) + K v ġ] Dv () Fig. 9 on the following ection how the repone trajectory generated to the ame oberved human motion, with the velocity feedback term in place. IV. EXPERIMENTAL VALIDATION To validate the propoed technique before implementing it onto a real robotic ytem, ome tet have been performed on real data involving two peron exchanging different object from different location, a hown in Fig. 7. Marker were intalled on the human bodie, mainly on the right arm of each partner (on the houlder, elbow and hand), although in the preent tudy only the hand marker are effectively.2 ued. Marker were tracked uing a Vicon motion capture ytem. The DMP verion preented in thi paper wa ued to learn the three Carteian dimenion of the right hand motion data from a elected equence. Then data from different equence have been ued a oberved human motion, and the reulting generated trajectorie have been compared to the recorded repone of the partner. The reulting behavior for one pecific data et i hown in Fig. 8, 9 and. In each of thee figure the black olid line repreent the ample trajectory ued for learning, the blue olid line repreent the data ued a oberved Human hand poition, the red olid line repreent the output of the propoed DMP method, the dotted blue line repreent the real recorded repone of the other Human partner to the movement in the olid blue line, and the olid green line how the repone of the bio-inpired DMP formulation under the ame condition. The meaured poition are in millimeter, and the reference ued for the data capture i located on the floor between the two uer, oriented a hown in Fig. 7, where the XYZ axi are colored in RGB order. Alo, for every motion dimenion being learnt the ame et of parameter ha been ued for the weighing function: µ =.7 and σ =.5. It can be een that the generated trajectorie adjut to the oberved partner trajectory without looing the inherent dynamic of the ample trajectory from which they were learnt. It i alo evident that the trajectory generated reemble much more cloely the real recorded repone of the human partner than the repone of the bio-inpired DMP method. Thi upport the idea that the previou verion of the DMP do actually require an initial etimation of the exchange location, ince uing the current hand poition of the partner a goal create ome unpredictable and undeired effect in the motion generated, epecially on Fig 8 and 9. A illutrated on thee example, The extended model we are propoing doe not require uch initial etimation to provide a atifactory behavior. V. CONCLUSIONS Thi article ha propoed an extenion of the DMP framework to correctly learn and reproduce the human arm approach during an object tranfer procedure. By changing the phae variable behavior, we obtain a better control of.. y(t) [m] Original Y DMP Decoupled Y (µrel =.7) Oberved partner Y Fig. 6. Ocillation with the DMP propoed in ection III-A. Fig. 7. Motion capture data aquired (left) and a picture of the capture eion (right).

6 Original X DMP Decoupled X (µrel =.7) Oberved partner X Repone to partner X Bio inpireddmp X.65.6 x(t) [m].2.. z(t) [m].55.5 Original Z DMP DecoupledZ (µrel =.7) Oberved partner Z Repone to partner Z Bio inpireddmp Z Fig. 8. Evolution of the generated trajectory in the X axi. Fig.. Evolution of the generated trajectory in the Z axi. y(t) [m] Fig. 9. Original Y DMP Decoupled Y (µrel =.7) Oberved partner Y Repone to partner Y Bio inpireddmp Y Evolution of the generated trajectory in the Y axi. the tranition in between the hape-attractor and the goalattractor, thu avoiding the need for an exchange location etimation. Furthermore, by adding in the tranformation ytem a compenation for the goal velocity, the model obtained improve it convergence toward moving target. It would be intereting to invetigate how thee improvement could benefit other application of the DMP framework. Thee experiment do not take yet into account the repone of the human partner to the robot motion; indeed, the behavior of the human might not be equivalent when interacting with a peron or with a robot. In order to complete the validation of our approach and to analyze the perception and reaction of the human when interacting with uch ytem, at the time of writing thi article, thi method i being implemented onto a real robotic etup. The equipment ued i a Kuka LWR robot, mounted onto a vertical tructure to reemble the configuration of a human houlder and arm; and a Kinect device to capture the motion of the human partner in front of the robot. Finally, one of the main iue that will need to be tackled regarding uch application i the triggering of the robotic motion tart to get a perfect timing with the human partner. The proper implementation of uch a triggering method will indeed highly influence the real time behavior of the preented technique. REFERENCES [] M. Cakmak, S. Srinivaa, M.K. Lee, S. Kieler, and J. Forlizzi. Uing Spatial and Temporal Contrat for Fluent Robot-Human Hand-over. In ACM/IEEE International Conference on Human-Robot Interaction, 2. [2] S. Calinon, F. Guenter, and A. Billard. On learning, repreenting, and generalizing a tak in a humanoid robot. IEEE tranaction on ytem, man, and cybernetic, 37(2): , April 27. [3] F. Chaumette and S. Hutchinon. Viual ervo control, part 2: Advanced approache. IEEE Robotic and Automation Magazine, 4():9 8, March 27. [4] Y.S. Choi, T. Chen, A. Jain, C. Anderon, J. Gla, and C. Kemp. Hand It Over or Set It Down: A Uer Study of Object Delivery with an Aitive Mobile Manipulator. In IEEE International Sympoium on Robot and Human Interactive Communication, 29. [5] M. Demurget and S. Grafton. Forward modeling allow feedback control for fat reaching movement. Trend in cognitive cience, 4():423 43, November 2. [6] A. Edinger and C. Kemp. Human-Robot Interaction for Cooperative Manipulation : Handing Object to One Another. In IEEE International Sympoium on Robot and Human Interactive Communication, 27. [7] H. Hoffmann, P. Pator, D.-H. Park, and S. Schaal. Biologicallyinpired dynamical ytem for movement generation: Automatic realtime goal adaptation and obtacle avoidance. IEEE International Conference on Robotic and Automation, page , May 29. [8] M. Huber, M. Rickert, A. Knoll, T. Brandt, and S. Glaauer. Human- Robot Interaction in Handing-Over Tak. In IEEE International Sympoium on Robot and Human Interactive Communication, page 7 2, 28. [9] A. Ijpeert, J. Nakanihi, H. Hoffmann, P. Pator, and S. Schaal. Learning Nonlinear Dynamical Sytem Model. Neural Computation, page 33, 2. [] A. Ijpeert, J. Nakanihi, T. Shibata, and S. Schaal. Nonlinear dynamical ytem for imitation with humanoid robot. IEEE-RAS International Conference on Humanoid Robot, 2. [] D. Lee and Y. Nakamura. Mimei cheme uing a monocular viion ytem on a humanoid robot. In IEEE International Conference on Robotic and Automation, page 4, April 27. [2] E. Sibot, L. Marin-Uria, R. Alami, and T. Simeon. A human aware mobile robot motion planner. IEEE Tranaction on Robotic, 23(5), 27. [3] E. Sibot, L. Marin-Uria, X. Broquere, D. Sidobre, and R. Alami. Syntheizing robot motion adapted to human preence. International Journal of Social Robotic, 2(3), 2.