Explaining and Exploiting Impedance Modulation in Motor Control

Transcription

1 Explaining and Exploiting Impedance Modulation in Motor Control Professor Sethu Vijayakumar FRSE Microsoft Research RAEng Chair in Robotics University of Edinburgh, UK Director, Edinburgh Centre for Robotics

2 Estimated State Efference Copy State PLAN Noise Controller Motor Command Biomechanical Plant Estimator Sensory Data Noise Sensory Apparatus

3 Control signals u Arm states x [q;q] Target Redundancy is a fundamental feature of the human motor system that arises from the fact that there are more degrees of freedom available o Task to -> control End Effector a movement Trajectory than (Min. are strictly Jerk, Min. necessary Energy to etc.) achieve o End the Effector task goal ->(Bernstein, Joint Angles 1967). (Inverse Kinematics) o Joint Angles -> Joint Torques (Inverse Dynamics) o Joint Torques -> Joint Stiffness (Variable Impedance) Redundancy at various levels:

4 Stiffness + Damping Impedance

5 This capability is crucial for safe, yet precise human robot interactions and wearable exoskeletons. HAL Exoskeleton, Cyberdyne Inc., Japan KUKA 7 DOF arm with Schunk 7 DOF Univ. of Edinburgh

6 OFC Open Loop OC Inv. dyn. model. TASK Trajectory planning Solve IK Controller (Feedback gains, constraints, ) PLANT - min. jerk, min time, TASK Optimise cost function (e.g. minimum energy) Task & constraints are intuitively encoded Optimal controller PLANT

7 Given: Start & end states, fixed-time horizon T and system dynamics x And assuming some cost function: v d f( x,u) dt F( x,u) dω T ( t, x) E h( x( T)) l(, x( ), π(, x( ))) d t Final Cost How the system reacts ( x) to forces (u) Running Cost Apply Statistical Optimization techniques to find optimal control commands Aim: find control law π that minimizes v π (0, x 0 ).

8 cost function (incl. target) L,x feedback controller x dynamics model OFC u u δu plant (robot) OFC law

9 Analytic Methods Linear Quadratic Regulator (LQR) Linear Quadratic Gaussian (LQG) Local Iterative Methods ilqg, ildp Dynamic Programming (DDP) Inference based methods AICO, PI^2,ψ-Learning

10 Konrad Rawlik, Marc Toussaint and Sethu Vijayakumar, On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference, Proc. Robotics: Science and Systems (R:SS 2012), Sydney, Australia (2012).

11 Explaining Human Impedance Modulation Exploiting Impedance Modulation in Robots Explosive Movement Tasks (e.g., throwing) Periodic Movement Tasks and Temporal Optimization (e.g. walking, brachiation) Djordje Mitrovic, Stefan Klanke, Rieko Osu, Mitsuo Kawato and Sethu Vijayakumar, A Computational Model of Limb Impedance Control based on Principles of Internal Model Uncertainty, PLoS ONE (2010).

12 Locally Weighted Projection Regression (LWPR) for dynamics learning (Vijayakumar et al., 2005). ~ f(q, q,u) q Φ(q, q,u) [q, q,u] dx f(x,u) dt F(x, u) dω dx ~ f(x,u) dt (x,u) dω S. Vijayakumar, A. D'Souza and S. Schaal, Online Learning in High Dimensions, Neural Computation, vol. 17 (2005)

13 OFC-LD uses LWPR learned dynamics for optimization (Mitrovic et al., 2010a) Key ingredient: Ability to learn both the dynamics and the associated uncertainty (Mitrovic et al., 2010b) Djordje Mitrovic, Stefan Klanke and Sethu Vijayakumar, Adaptive Optimal Feedback Control with Learned Internal Dynamics Models, From Motor Learning to Interaction Learning in Robots, SCI 264, pp , Springer-Verlag (2010).

14 Reproduces the trial-to-trial variability in the uncontrolled manifold, i.e., exhibits the minimum intervention principle that is characteristic of human motor control. KUKA LWR Simulink Model Minimum intervention principle

15 Can predict the ideal observer adaptation behaviour under complex force fields due to the ability to work with adaptive dynamics Constant Unidirectional Force Field Velocity-dependent Divergent Force Field Cost Function: Djordje Mitrovic, Stefan Klanke, Rieko Osu, Mitsuo Kawato and Sethu Vijayakumar, A Computational Model of Limb Impedance Control based on Principles of Internal Model Uncertainty, PLoS ONE, Vol. 5, No. 10 (2010).

16 See: Osu et.al., 2004; Gribble et al., 2003

17 Focus: Signal Dependent Noise (SDN) n ( u) isotonicu1 u2 isometric u1 u2, ξ ~ N(0, I2) m

18 Stochastic OFC-LD Deterministic OFC-LD Djordje Mitrovic, Stefan Klanke, Rieko Osu, Mitsuo Kawato and Sethu Vijayakumar, A Computational Model of Limb Impedance Control based on Principles of Internal Model Uncertainty, PLoS ONE (2010).

19 Explaining Human Impedance Modulation Exploiting Impedance Modulation in Robots Explosive Movement Tasks (e.g., throwing) Periodic Movement Tasks and Temporal Optimization (e.g. walking, brachiation) David Braun, Matthew Howard and Sethu Vijayakumar, Exploiting Variable Stiffness for Explosive Movement Tasks, Proc. Robotics: Science and Systems (R:SS), Los Angeles (2011)

20 Highly dynamic tasks, explosive movements David Braun, Matthew Howard and Sethu Vijayakumar, Exploiting Variable Stiffness for Explosive Movement Tasks, Proc. Robotics: Science and Systems (R:SS), Los Angeles (2011)

21 The two main ingredients: Compliant Actuators Torque/Stiffness Opt. VARIABLE JOINT STIFFNESS Model of the system dynamics: τ τ( q, u) x f( x, u) u MACCEPA: Van Ham et.al, 2007 K K( q, u) Control objective: T 1 J d w 2 F dt Optimal control solution: 0 2 min. DLR Hand Arm System: Grebenstein et.al., 2011 u( t, x) u * ( t) L ilqg: Li & Todorov 2007 DDP: Jacobson & Mayne 1970 * ( t)( x x * ( t)) David Braun, Matthew Howard and Sethu Vijayakumar, Exploiting Variable Stiffness for Explosive Movement Tasks, Proc. Robotics: Science and Systems (R:SS), Los Angeles (2011)

22 2-link ball throwing - MACCEPA stiffness modulation speed: 20 rad/s distance thrown: 5.2m

23 Benefits of Stiffness Modulation: Quantitative evidence of improved task performance (distance thrown) with temporal stiffness modulation as opposed to fixed (optimal) stiffness control David Braun, Matthew Howard and Sethu Vijayakumar, Exploiting Variable Stiffness for Explosive Movement Tasks, Proc. Robotics: Science and Systems (R:SS), Los Angeles (2011)

24 Exploiting Natural Dynamics: a) optimization suggests power amplification through pumping energy b) benefit of passive stiffness vs. active stiffness control David Braun, Matthew Howard and Sethu Vijayakumar, Exploiting Variable Stiffness for Explosive Movement Tasks, Proc. Robotics: Science and Systems (R:SS), Los Angeles (2011)

25 Behaviour Optimization: Simultaneous stiffness and torque optimization of a VIA actuator that reflects strategies used in human explosive movement tasks: a) performance-effort trade-off 1 T 2 J d w b) qualitatively similar stiffness pattern F dt 2 0 c) strategy change in task execution David Braun, Matthew Howard and Sethu Vijayakumar, Exploiting Variable Stiffness for Explosive Movement Tasks, Proc. Robotics: Science and Systems (R:SS), Los Angeles (2011)

26 Scalability to More Complex Hardware DLR HASY: State-of-the-art research platform for variable stiffness control. Restricted to a 2-dof system (shoulder and elbow rotation) Max motor side speed: 8 rad/s Max torque: 67Nm Stiffness range: Nm/rad Speed for stiffness change: 0.33 s/range

27 DLR - FSJ Schematic representation of the DLR-FSJ Motor-side positions: q Constraint: min T 2 θ,σ] [ ( σ) max 4 ( σ)

28 Dealing with Complex Constraints M, q 1)q 1 G1( q1) τ1( q1, 2) 11(q1)q1 C11(q1 q 2 q 2 2βq 2 κ q2 κ 2 u Incorporating the constraints: 1. Range constraints: 2. Rate/effort limitations: Φ( q1, q2) [ Φmin ( q2), Φmax ( q2)] u u,u ] Φ ( q 1, q ) [ min max 2 κ [ 0, κmax ]

29 DLR FSJ: optimisation with state constraints variable stiffness fixed stiffness Spring Length vs Stiffness Modulation

30 DLR FSJ: optimisation with state constraints variable stiffness fixed stiffness Spring Length and Stiffness Modulation (plotted against time)

31 Ball throwing with DLR HASy motor velocity limited to: 2rad/s, 3rad/s

32 Explaining Human Impedance Modulation Exploiting Impedance Modulation in Robots Explosive Movement Tasks (e.g., throwing) Periodic Movement Tasks and Temporal Optimization (e.g. walking, brachiation)

33 Explaining Human Impedance Modulation Exploiting Impedance Modulation in Robots Explosive Movement Tasks (e.g., throwing) Periodic Movement Tasks and Temporal Optimization (e.g. walking, brachiation) Jun Nakanishi, Konrad Rawlik and Sethu Vijayakumar, Stiffness and Temporal Optimization in Periodic Movements: An Optimal Control Approach, Proc. IEEE Intl Conf on Intelligent Robots and Systems (IROS 11), San Francisco (2011).

34 Representation what is a suitable representation of periodic movement (trajectories, goal)? Choice of cost function how to design a cost function for periodic movement? Exploitation of natural dynamics how to exploit resonance for energy efficient control? optimize frequency (temporal aspect) stiffness tuning

35 Cost Function for Periodic Movements Optimization criterion Terminal cost ensures periodicity of the trajectory Running cost tracking performance and control cost Jun Nakanishi, Konrad Rawlik and Sethu Vijayakumar, Stiffness and Temporal Optimization in Periodic Movements: An Optimal Control Approach, Proc. IEEE Intl Conf on Intelligent Robots and Systems (IROS 11), San Francisco (2011).

36 Another View of Cost Function Running cost: tracking performance and control cost Augmented plant dynamics with Fourier series based DMPs Reformulated running cost Find control and parameter such that plant dynamics (1) should behave like (2) and (3) while min. control cost Jun Nakanishi, Konrad Rawlik and Sethu Vijayakumar, Stiffness and Temporal Optimization in Periodic Movements: An Optimal Control Approach, Proc. IEEE Intl Conf on Intelligent Robots and Systems (IROS 11), San Francisco (2011).

37 Temporal Optimization How do we find the right temporal duration in which to optimize a movement? Solutions: Fix temporal parameters... not optimal Time stationary cost... cannot deal with sequential tasks, e.g. via points Chain first exit time controllers... Linear duration cost, not optimal Canonical Time Formulation 38

38 Canonical Time Formulation Dynamics: Cost: n.b. represent real time Introduce change of time

39 Canonical Time Formulation Dynamics: Cost: n.b. represent real time n.b. now represents canonical time Introduce change of time Konrad Rawlik, Marc Toussaint and Sethu Vijayakumar, An Approximate Inference Approach to Temporal Optimization in Optimal Control, Proc. Advances in Neural Information Processing Systems (NIPS '10), Vancouver, Canada (2010).

40 AICO-T algorithm Use approximate inference methods EM algorithm E-Step: solve OC problem with fixed β M-Step: optimise β with fixed controls Konrad Rawlik, Marc Toussaint and Sethu Vijayakumar, An Approximate Inference Approach to Temporal Optimization in Optimal Control, Proc. Advances in Neural Information Processing Systems (NIPS '10), Vancouver, Canada (2010). 41

41 Spatiotemporal Optimization 2 DoF arm, reaching task 2 DoF arm, via point task

42 Temporal Optimization in Brachiation Optimize the joint torque and movement duration Cost function Time-scaling : gripper position : canonical time Find optimal using ilqg and update in turn until convergence [Rawlik, Toussaint and Vijayakumar, 2010] Jun Nakanishi, Konrad Rawlik and Sethu Vijayakumar, Stiffness and Temporal Optimization in Periodic Movements: An Optimal Control Approach, Proc. IEEE Intl Conf on Intelligent Robots and Systems (IROS 11), San Francisco (2011).

43 Temporal Optimization of Swing Locomotion vary T=1.3~1.55 (sec) and compare required joint torque significant reduction of joint torque with Jun Nakanishi, Konrad Rawlik and Sethu Vijayakumar, Stiffness and Temporal Optimization in Periodic Movements: An Optimal Control Approach, Proc. IEEE Intl Conf on Intelligent Robots and Systems (IROS 11), San Francisco (2011).

44 Optimized Brachiating Manoeuvre Swing-up and locomotion Jun Nakanishi, Konrad Rawlik and Sethu Vijayakumar, Stiffness and Temporal Optimization in Periodic Movements: An Optimal Control Approach, Proc. IEEE Intl Conf on Intelligent Robots and Systems (IROS 11), San Francisco (2011).

45 Brachiating Hardware with Constraints 46

46 Variable Impedance Biped (BLUE: Bipedal UoE)

47 Cue integration under uncertain causal structure Motor disturbance affects arm position Sensory disturbances affect observations

48 Test whether force field exposure leads to sensory adaptation Experimental setup and design: Reaches in a single direction Lateral force applied to hand Forward velocity-dependent Target F x a y Cursor

49 Compare Pre vs Postadaptation alignment errors x-direction localization error Significant shifts following adaptation p < 0.05 (2-tailed T) for both modalities

50 Proves that sensory and motor adaptation are NOT independent systematic motor perturbation elicits sensory recalibration Evidence that brain resolves sensorimotor adaptation in a unified and principled manner We should revisit human motor adaptation results/paradigm with this new insight! Adrian Haith, Carl Jackson, Chris Miall and Sethu Vijayakumar, Unifying the Sensory and Motor Components of Sensorimotor Adaptation, Proc. Advances in Neural Information Processing Systems (NIPS ), Canada (2009).

51 Case Study 2: Sensory vs. Motor Noise Does visual perturbation provoke impedance control? Closing the control loop with EMG feedback Manipulating visual and proprioceptive feedback On-line impedance adaptation and data driven stiffness/visual displacement model

52 Dr. Matthew Howard Dr. David Braun Dr. Jun Nakanishi Konrad Rawlik Dr. Takeshi Mori Dr. Djordje Mitrovic Evelina Overling Alexander Enoch

53 My webpage and relevant publications: Our group webpage: