A dynamical systems account of the uncontrolled manifold and motor equivalence in human pointing movements

Transcription

1 A dynamical systems account of the uncontrolled manifold and motor equivalence in human pointing movements A theoretical study DOCTORAL THESIS submitted at the International Graduate School of Neuroscience (IGSN) by Valère Martin. Ruhr Universität Bochum Institut für Neuroinformatik Universität Strasse 15, Gebäude D DE-4485 Bochum

2

3 To my parents

4

5 ABSTRACT Human movements are executed with effectors that have more degrees of freedom (DOFs) than the minimum number of independent parameters that completely describes the task. The effector is redundant for the task and thus, there is an abundance of task equivalent effector configurations to achieve the task. The UnControlled Manifold (UCM) theory proposes a motor control strategy to generate movement that makes use of the manifold formed by the task equivalent configurations of the effector. The UCM motor control strategy does not enforce one unique trajectory in the effector space thus allowing to use multiple task equivalent configurations of the effector. The UCM theory is supported in experiments by a task specific structure of the joint variability. This UCM signature is characterized in the joint space by a higher variability in the task equivalent manifold than in the non task equivalent manifold. In this study, a theoretical account of the UCM motor control strategy in pointing movement is given based on dynamical systems theory. A model is made up of a redundant arm and a neuronal movement generator that specifies the movement to be executed and accounts for a task-oriented organization of movement generation. The task is specified in a limit cycle framework while the movement generator specifies the coordination of the DOFs according to the task specification. The movement generator and the effector are reciprocally compliant by mutual couplings. Neuronal and muscle noises generate movement variability. The model allows to investigate the conditions and the causes for a correct UCM signature. Our results show that the task-oriented organization of the movement generator accounts alone for the UCM signature independently of the mutual couplings or a similar task-oriented organization at the effector level. Motor equivalence, the use of task equivalent configurations of the effector in response to a mechanical perturbation, requires on the other hand mutual couplings between the effector and the movement generator. We conclude from this study that the task-oriented organization of movement generation at a neuronal level is a fundamental feature of movements that accounts for the UCM signature and motor equivalence.

6

7 Acknowledgements I would like to acknowledge great people that I met during these three years (and a little bit more...) that I spent a the Institute for Neuroinfomatics at Bochum. My special thanks go to Prof. Gregor Schöner, my main supervisor on this project, for his constant support. I am indebted to him for many things. I really enjoyed the long and fruitful discussions we had about this project and other scientific and non-scientific topics. I owe him to open my mind on many aspects of living sciences. His support was essential to accomplish this work. I wish also to thank the team at the University of Delaware, Prof. John P. Scholz for introducing me to the world of scientific experiments in motor control and for his pertinent questions as well as my colleagues Ya-weng and Darcy for their kindness and our scientific exchanges during my short trips in the US and electronically over the too large Atlantic ocean. I have also to thank the GK and IGSN students with whom I share my (few) leisure times during this PhD period. They shared the best and the worst of me. Anna, Arundhati, Britta, Evelina, Illah and Pauline thank you for your friendship. Susanne with whom I share the office earns a special thanks for tolerating my bad German during our breaks. Many thanks also to my parents and my brother for their constant and unrestricted support during this PhD and for their helps with the redaction of this thesis. I am also grateful for the time I spent at the Institute for Neuroinformatics. I appreciated in particular the collaborative atmosphere that enabled to access to an essential technical support. Furthermore, I would like to thank the International Graduate School of Neuroscience (IGSN) for their financial support. If in the rush to finish the redaction of this thesis, I failed to thank someone, I beg for apologies. Certainly that his/her contribution was valuable.

8

9 Contents INTRODUCTION Motor control and the degrees of freedom Redundancy and coordination Optimality approach to redundancy Coupling among the degrees of freedom Constraints for a motor control strategy Dynamical systems Dynamical systems and motor control The UnControlled Manifold (UCM) Equilibrium Point Hypothesis (EPH) Equilibrium point hypothesis and movements Equilibrium point hypothesis versus inverse dynamics Equilibrium point hypothesis and oscillators Scientific approach to motor control A modular approach to motor function Biological cybernetics Neural networks Motor equivalence and Selfmotion Goal METHODS Trajectory Trajectory specification Trajectory representation Effector and end-effector space Forward and inverse kinematics Null space Dynamical systems and the virtual task trajectory Elements of motor control Movement initiation and termination Posture and movement Movement specification and representation Joint muscle model Physics of the skeleton i

10 ii CONTENTS Noise model Experimental and simulation methods and data analysis Data collection The UnControlled Manifold (UCM) Selfmotion Motor equivalence Model implementation RESULTS Model functioning End-effector kinematics Variability End-effector and joint variability The UnControlled Manifold (UCM) for the hypothesis end-effector position UCM and backcoupling Wrong UCM Hypothesis for the end-effector position Effects of the neuronal noise level on the UCM signature The UnControlled Manifold for the hypothesis centre of mass Selfmotion Virtual UCM and virtual selfmotion Motor equivalence Rhythmic movements Optimal control DISCUSSION Movement generation in living systems Situating the model in a broader context Trajectories Discrete movements Rhythmic movements Muscle model and muscle impedance The UnControlled Manifold Motor equivalence Selfmotion The UCM and stochastic feedback optimal control Perspectives for the model Bibliography 161 Appendix A 177 Appendix B 181

11 INTRODUCTION 1.1 Motor control and the degrees of freedom Redundancy and coordination A fundamental ability of most animals is to move effortless within their environment. Although everyday movements do not require particular attention for healthy humans, the unconscious processes that generate appropriate and adapted movements are complex. These processes require complex actions and numerous interactions of various anatomical structures. In view of the involvement of numerous substructures in movement generation, appropriate concepts must be set to construct a framework that allows to investigate movements. In this study, movements are conceptualized at the level of the organism with respect to behavioral macrostructures, not necessarily related to anatomy, like the task that expresses the movement objectives, the effector that interacts with the external world, the end-effector that characterizes the execution of the task or the movement generator that specifies the movement at the neuronal level (sec. 2.1). This framework invites questions that form the core of this work. This first section introduces to the notions of degrees of freedom and redundancy. A movement is constrained by a task and by the mean by which the movement is realized. The task expresses the movement objectives. The independent dimensions or parameters of the task are the task variables that uniquely describe the task. These independent parameters span the task space. The mean by which the movement is achieved is the body which is composed of numerous articulations or joints. A movement arises from the motion of the joints that are linked together by the skeleton. These joint motions are organized to fullfil the movement objectives. That means that the joint motions are constrained in time and in space by the task. Thus, the joint motions must be specified such that the task variables evolve in time according to the movement objectives. The task realization is the product of the coherent motion of the joints in time and in space. Space is an inherent dimension to any movement. In addi- 11

12 12 Introduction tion to space, time is a dimension that characterizes the joint motions and the task variables. Thus, a structured time relationship between the joints in addition to a space relationship is required to generate a meaningful movement. This time-dependent structuring of the joint motions defines a coordination for the joint motions. The joints move independently from each other but at the same time the individual joint motion is constrained by the movement objectives. A neuronal supervising structure dictates the coordination of the joints so that the outcome of the joint motions fullfils the movement objectives in the task space. Coordination is the capability of the joints to move together to produce coherent and meaningful time-variant outcomes. Coordination is thus at the basis of distinct movements. The possibility to move each joint independently from the other joints means that the joints are so many Degrees Of Freedoms (DOFs) of the body. A full specification of coordination from a supervising structure determines the relative state changes in time of all the DOFs in relationship to a common outcome. That means that the DOFs state changes must be organized by a supervising structure. It does not necessarily imply that these relative state changes are reproducible for similar trials identified with a identical starting configuration but that the coherence of the outcome is preserved in any case. The outcome determines the coordination because the coherence or the meaningfullness of the joint motions is expressed in terms of the outcome, i.e the task. Thus, coordination can only make sense if there is a task associated with the movement. This clearly implies that coordination is constrained by the task. In this study, the task variables are restricted to the position of the end-effector in a Cartesian coordinate system. The task thus expresses the motion of the end-effector in space and in time. The movement objective is a particular motion of the end-effector in the task space. In pointing movements, the end-effector is typically the finger tip or the hand. In a drawing task, the end-effector is typically the tip of the pen. Thus, the arm is called the effector. The mathematical relationship between the task space and the joint space is called a mapping. The feature of the mapping between the joint space and the task space should be characterized within a biological context. If the task variables can be uniquely described with respect to the joint angles, the mapping is determined (sec ). Knowing the joint angles unequivocally set the task variable values and reciprocally. This special case turns out to be uncommon in living systems. Usually, there are many more joints than task variables. The number of DOFs of the effector is larger than the minimum number of independent parameters required to describe the task. Thus, multiple combinations of joint angles produce an identical position of the end-effector in the task space. Reciprocally, a given end-effector position can be realized with various combina-

13 1.1. MOTOR CONTROL AND THE DEGREES OF FREEDOM 13 tions of joint angles. The system is said to be redundant. A concrete example to illustrate our point is given with a multijoint arm in fig A. Again, the joints are assumed to be independent. The task set the end-effector position. By end-effector is meant in this context the hand tip. There are, in this example, 4 joints. So the effector is characterized by 4 DOFs. The hand position is uniquely given in a coordinate system with two independent dimensions. Thus, the end-effector position is characterized by 2 DOFs. The task is then said to be 2 DOFs redundant. Because this effector/end-effector mapping is redundant, a set of joint configurations yields the same end-effector position. In this example, the inverse mapping from the end-effector space to the joint space is said to be ill-defined because knowing the end-effector position does not permit to deduce the joint angles. It should be stressed that redundancy can only be defined relatively to the task because the definition of redundancy depends on the task space dimension. In that context, the task space should be understood as a space spans by various independent parameters that describe the task. This redundancy definition can be illustrated with a simple example. If a task is to make the sum x and y to equal a, the system (x,y) is then redundant. If the task is to make the sum of x and y equal to a and x is equal to b, the system is no longer redundant although the number of DOFs is unchanged. In this example redundancy is reduced by increasing the task space dimension. In the arm example in fig A, the task is to position the hand in any arbitrary orientation. It is thus assumed that the task can be achieved with any hand orientation but it could be that the hand orientation is uniquely specified by the hand position (Desmurget & Prablanc 1997). In that case, the task space is increased and the system redundancy reduced. The task can be further underdetermined by the movement objectives. In the same arm example (fig ), if the task is to move the hand to a given location starting from an initial end-effector position, not only is the final arm configuration ill-defined as in the static case (fig A) but an infinity of different end-effector trajectories fullfills the task. In addition to the joint redundancy is a task trajectory redundancy. The mapping from the task space to the joint space is further ill-defined during the movement. In reality, the endeffector trajectories are not random between movement repetitions but present invariant features. Measured features of the end-effector kinematics demonstrate that the end-effector trajectories although variable are strictly bounded (Morasso 1981; Boessenkool et al. 1998). The observed trajectory variability is smaller than the variability that would be observed if all the possible task-equivalent trajectories constrained by the task alone were used. The low variability of the trajectory for repetitions of the task does not indicate the origin and nature of the constraints on the trajectory. It has been suggested that the specification of the task by the CNS constrains coordination for instance

14 14 Introduction by minimizing the end-effector jerk during the movement (Flash & Hogan 1985). This constrained coordination generates invariant kinematic features and eliminates thus the task trajectory redundancy. This implies that the mean end-effector trajectory, assuming weak nonlinearities, is the internally selected trajectory among the set of possible end-effector trajectories that fullfill the task. Such principle of redundancy reduction can also be applied for the joint redundancy (Uno et al. 1989). A reduction of the task trajectory redundancy does not however necessarily imply that the CNS task specification alone constrains the executed trajectory. It has been suggested that the endeffector trajectory features come from the low-pass biomechanical property of the muscles (Barto et al. 1999). That means that the end-effector trajectory is constrained by the low-pass biomechanical property of the muscles independently of the CNS task specification. These observations emphasize that a movement is made up of two essential components. One component induces and generates the movement and the second component executes the movement. The degree of redundancy can also change for a specific task setting because of internal system constraints (fig B). At the limit of the workspace of the effector, a single arm configuration allows to position the hand. The arm is no more redundant. Singularity is a typical problem encountered in robotic manipulators. In particular configurations, the motion of the end-effector in particular directions requires infinite joint motions. The end-effector motion ability is locally decreased. It is mathematically expressed by a reduction of the rank of the manipulator Jacobian (Murray et al. 1994). In humans, the motion ability of the end-effector may be locally reduced because of the limited range of motion of the joints. These examples illustrate that the mechanical linkage between the joints constrains the movement of the effector and can lead locally to a loss of DOFs. From a control theory viewpoint, redundancy as encountered in living systems is an unusual challenge. Usually, redundancy is considered as a problem and avoided. However, redundancy may also present advantages. In robotics research, more attention to redundant and hyperredundant manipulators has been paid in recent years as advantages such as flexibility and adaptability have been recognized (Kreutz-Delgado et al. 1992; Zlajpah 1998; Chirikjian & Burdick 1994). These advantages can be understood by considering a nonredundant effector. By definition, a unique configuration in the joint space complies to the task. Any deviation of one joint from its expected position induces a task error. In a redundant system, a failure of one joint can be compensated for by the remaining joints such as to keep the task variable invariant. Moroever, a redundant system can easily accommodate new constraints in the form of secondary tasks while the main task is preserved. This ability allows to adapt to new environmental constraints. For instance, for

15 1.1. MOTOR CONTROL AND THE DEGREES OF FREEDOM 15 1) 2) x A B y Figure 1.1: 1) A pointing task in human is illustrated for a 4-DOF arm and a 2-DOF task. The system is thus 2 DOFs redundant. The joint axes of rotation are perpendicular to the page. The task consists of moving the end-effector from a starting position to a target position. When the task is repeated, intertrial variability is observed for the end-effector path but also for the joint path. Throughout the movement, multiple task-equivalent effector configurations lead to the same end-effector position. 2) A. The task is to place the hand at the location depicted by the square. As shown on the figure not a unique effector configuration fullfills the task. If the hand orientation is specified in addition, the task is still 1 DOF redundant. B. The target is located at the edge of the workspace. A single joint configuration satisfies the task requirement and thus, the same arm is no longer redundant for this particular task. This position is a singular position. a given end-effector position, the limb configuration can be changed to modulate the manipulability of the end-effector (Barreca & Guenther 21; Murray et al. 1994). The absolute effect of a perturbation on the endeffector for a fixed limb configuration is scaled according to the direction of the perturbation. A limb configuration can be chosen such as to better resist perturbations from specific directions (Hogan 1985b). In humans, not only the end-effector inertia but also the end-effector impedance resists perturbations (Mussa-Ivaldi et al. 1985). Unlike the inertia, the impedance is linked to the muscle properties. Impedance, like inertia, can be modulated to some extent for a fixed end-effector position (Gomi & Osu 1998; Osu et al. 24; Darainy et al. 24). Muscle redundancy grants this possibility. Although redundancy offers advantages, redundancy presents also specific difficulties. To control a redundant system, an adapted control strategy to use redundancy that should be characterized by flexibility must be applied (sec ).

16 16 Introduction Redundancy asset has been investigated in robotics. In motor control research, attempts to understand redundancy often overlook the advantages of redundant systems and focus on reducing the excessive DOFs (Gielen & van Bolhuis 1998). An alternative way of looking at redundancy recognizes an opportunity to exploit the abundance of solutions that redundancy offers. Redundancy reduction idea postulates the existence of constraints among the joints or proposes an augmented task space. For a complete redundancy reduction, joints are made dependent such that the effector space is no more redundant for the task (Medendorp et al. 2). The alternative solution for redundancy reduction consists of increasing the task space by constraining the task specifically with a performance index. For instance, muscle redundancy can be solved by postulating a minimum energy expenditure. If and only if the minimal energy solution is unique, the system is made non redundant. Redundancy reduction is not a priori compatible with the idea of flexibility if the constraints to reduce redundancy are not made context dependent. Often a combination of both task specifications and joint rules is implicitly assumed to eliminate redundancy. If and only if these added constraints are task-specific and can be modified during movement execution, flexibility may be somewhat preserved. Multiple task-dependent constraints are proposed that are either organized in a hierarchical fashion or integrated in a potential field whose minimum solution determines the desired trajectory (Rosenbaum et al. 24; Morasso & Sanguineti 1995). These solutions to solve the redundancy problem conserve a flexible framework in the sense that the constraints can be adapted to the movement context. The potential field approach is a general solution to navigate in a redundant space. In this example, the potential field is set on a neural postural map of effector configurations (Morasso & Sanguineti 1995). The field dynamics generates flexibility. The underlying assumptions are the ability of the brain to store a set of constraints, if they cannot be determined from perception alone, and the ability to retrieve them in the appropriate circumstances to form the potential field. How these processes may function is not explained by the authors (Morasso & Sanguineti 1995). These approaches distinguish themselves from most other approaches to the redundancy issue because they assume context dependent constraints Optimality approach to redundancy In motor control, the motivation for reductionist approaches to redundancy is to be found in evolution and learning. Evolution optimizes an organism to maximize its fitness. The process of learning further adapts the organism during its life to its environment. The organism thus improves towards an optimum. This principle motivates motor control model based on optimization principle (Todorov 24; Scott 24; Uno et al. 1989; Flash & Hogan

17 1.1. MOTOR CONTROL AND THE DEGREES OF FREEDOM ; Otha et al. 23; Winters & Woo 199). This evolutive argument is presented on an intuitive basis and has not been thoroughly explained. Moreover, flexibility is not accounted for in most of these optimality processes (for an exception (Todorov & Jordan 22; Todorov & Jordan 23)). Flexibility certainly distinguishes humans from robots. In an unknown changing environment, humans adapt pretty well while most robots fail. So flexibility appears to be an evolutive constraint as well. Adaptation moreover assumes interactions with the environment to drive the process that leads to optimality. Most reductionist approaches to redundancy express constraints that are independent from the environment. The principle of redundancy reduction is presented below. A movement is optimized such as to minimize a cost defined by a cost function which is usually the integral over the movement time of some instantaneous cost. When this minimum exists uniquely, the corresponding movement is also unique. There is no redundancy anymore. Popular optimal functions include a minimum jerk for the end-effector motion and a minimum torque change in the joint space (Flash & Hogan 1985; Uno et al. 1989). A minimum movement energy in joint space could not account for experimental data (Cruse et al. 1993). A minimum energy principle can still be claimed for muscle energy expenditure without being incompatible with the observed end-effector and joint trajectories. The minimum torque change model can be rejected based on the authors claim that the curvature of the hand path is an inherent effect of the optimality strategy. Alteration of the vision of the hand path induces the subjects to generate a straighter hand path (Wolpert et al. 1995). This minimum torque change model is also not compatible with the ability to generate straight paths in a rotating room environment (Dizio & Lackner 1995). The minimum jerk model generates straight hand path (Flash & Hogan 1985; Hogan 1984). In humans performing pointing movements, the mean hand paths are slightly curved in various parts of the workspace what the minimum jerk model does not account for (Morasso 1981). The minimum jerk model could however account for movement planning (sec. 1.4). A criticism against these models is the arbitrariness of the cost function because various cost functions lead to approximately the right movement kinematics. Local cost functions are postulated for the mapping between the task space and the joint space. Locally, the mapping is constrained such that the task is no more redundant (sec ). A local constraint can be, for instance, a potential function with a minimum at a preferred joint configuration (Guenther & Barreca 1997). These local constraints may or may not lead to an optimal trajectory in the sense of a cost function as discussed above. A popular solution for the inverse kinematic map of redundant arms is to minimize joint displacements (Penrose 1955). This mapping minimizes joint energy locally and globally and reduces the effort at the joint (Guenther & Barreca

18 18 Introduction 1997). Such mappings are equivalent to increasing the task space dimension. These constrained mappings offer an alternative interpretation to an arbitrary cost function. In comparison to robots, human movements are highly variable between repetitions of the same task in controlled experimental conditions (fig. 1.1). The optimality-based models examined so far capture only a mean behavior and often assume implicitly that variability emerges from the inability of the controller to servocontrol the prescribed trajectory. Actually, only the mean deterministic component of the trajectory is accounted for in these models. It is this movement variability and the lack of flexibility of these optimal models that motivate models based on the minimization of the variability. A minimum variance model has been proposed in which the variability of the movement end-point is the cost function (van Beers et al. 22). This cost, according to the authors, is more natural because it represents the task success directly. This model relies on the assumption that the motorneuron noise is signaldependent. It is however demonstrated experimentally that, in order to increase movement end-point accuracy, subjects increase the agonist/antagonist muscle activity at the joint which should thus potentially induce more variability with a signal dependent noise (Osu et al. 24; Gribble et al. 23). Thus, Osu & al. have proposed an extended minimum variance model that adds a cost for the motor command magnitude (in (Osu et al. 24)). Within a hierarchical framework, Loeb & al. have proposed a model which selects an optimal solution in the parameter space based on the expected type of perturbations and their effects on the end-point error (Loeb et al. 1999). Within the framework of stochastic feedback optimal control, a minimum intervention principle is proposed to account for movement variability (Todorov & Jordan 22; Todorov 1998). The cost is based on a general task penalty and a quadratic form of the muscle command signals. The task penalty includes a movement end-point error. This model is also based on signal-dependent noise which may be in conflict with the muscle activity precision trade-off as mentioned above (Osu et al. 24; Gribble et al. 23). In addition, the minimum intervention principle accounts for a specific structure of the variability in human movements (sec ). The merit of these models is to add a new essential dimension to biological movements: variability. Many approaches to biological movements fail to capture movement variability Coupling among the degrees of freedom An alternative reductionist approach to redundancy is to assume fixed restrictive rules that govern joint motions (Gielen & van Bolhuis 1998). The eye is a redundant system from a motor control viewpoint. Each eye can move left/right, up/down and rotates. Because the task is to position a point of interest on the fovea, 1 DOF is superfluous and the system is redundant. In

19 1.1. MOTOR CONTROL AND THE DEGREES OF FREEDOM , Dr. Donder observed the rotation of his own eye using an afterimage paradigm. He stared at a red cross and, because of habituation, he saw after a long fixation time a green cross. He then looked at a screen and observed that the cross rotated when the gaze was shifted. The orientation of the afterimage changed according to the gaze direction. The law that was postulated from these observations is known as Donder s law. This orientation constraint decreases the number of available DOFs such that the eye position is completely determined. Donder s law has been the subject of controversies, rejected and extended to a law that specifies the behavior of both eyes at the same time. A Donder s law is also claimed for the arm but violations of that law have been observed. The law holds only for some specific conditions. That leads to postulate a set of Donder s laws which may reflect a set of kinematic rules (Medendorp et al. 2). It is suggested that the various context-dependent laws form cuts along an iso-donder surface in the kinematic space. These imposed mechanical restrictions are called holonomic constraints. Similarly to Donder s law, the limb configuration for various workspace positions has been assumed to be determined before the movement is initiated (Rosenbaum et al. 24; Rosenbaum et al. 1991). The final configuration could be selected either based on the limb preferred frequency/amplitude (resonance), or on a weighted sum of stored postures or on a hierarchical series of constraints with successive eliminations (Rosenbaum et al. 24; Rosenbaum et al. 1991). The quest for a Donder s law for the arm has not lead to a concrete law that faithfully describes the arm configurations for various end-effector positions. Donder s law implies that the mechanical DOFs are coupled. Thus, the true DOFs of the eye form a non-redundant space for the task. Dynamic coupling/uncoupling processes have also been claimed to underlie the coordination of the joints during movement generation (Martin et al. 22). Coupling among the joints was found in a throwing task based on the joint variability. In this study, coupling among the joints is derived from joint covariance measurements (PCA analysis) and data shuffling (Monte Carlo method). Because the shuffled data generates more variability in the throw than the real data, a task dependent coupling among the joints is claimed. Contrary to Donder s law, this joint coupling is aimed specifically to preserve task accuracy. The authors do not however specify the modality of the coupling and do not relate explicitly this coupling to a redundancy suppression. This idea of coupling among the apparent DOFs to reduce the number of effective DOFs available is fundamentally different from coordination. A task specific relationship among the DOFs is at the heart of coordination as well but coordination is also required for non-redundant effectors. Coordination preserves the coherence of the task variables. Coordination does not aim to reduce the degree of redundancy but simply makes use of the DOFs available to achieve the task properly. Contrary to joint coupling, coordination does

20 2 Introduction not require specific interactions of the mechanical joints because coordination emerges at a neuronal level from a coordinative structure. Indeed, coordination is constrained by the task which is defined at the neuronal level. Coordination specifies an organization of the DOFs in time Constraints for a motor control strategy The previous sections have introduced to the redundancy that characterizes living systems. Redundancy is perceived in most cases as a problem to be solved by the CNS. The reductionist approaches that come from considering redundancy as a problem do not recognize the potential benefit of redundancy. Moreover, reductionist approaches fail to demonstrate a common reduction principle that accounts for the movement properties. In addition to redundancy, coordination of the DOFs of the effector is pointed out as a fundamental feature of movements. An account of movements that fails to explain coordination cannot reproduce biological movements (sec ). This study focuses on the coordination of redundant effectors. A motor control strategy, based on dynamical systems theory, to coordinate a redundant system and that does not rely on a reductionist method is implemented in a model (sec and sec. 2). 1.2 Dynamical systems Dynamical systems and motor control The dynamical systems approach to motor control focuses on the ability of the nervous system to maintain persistent patterns of behavior in complex changing environments. This ability is challenged by the incompleteness and multitude of sensory information available to the organism, the omnipresence of variability in natural environment, as well as the continuous nature of the sensory information and motor actions while behaviors are discrete. The intrapersonal and extrapersonal environment is characterized by continuous variations because of the omnipresence of variability in nature. The stochastic nature of the environment comes from the fluctuations of numerous strongly interacting subsystems. These fluctuations lead to uncertainty in percepts and motor acts. A selected behavior must be immune to these uncertainties. The stability of a behavioral pattern is thus a central requirement for meaningful and efficient behaviors to emerge. Stability is the central concept in dynamical systems theory. The variety of stable behaviors that emerges from the interaction of the organism and its extrapersonal and intrapersonal environment highlights the large flexibility that a living system exhibits. Flexibility is the ability to change of

21 1.2. DYNAMICAL SYSTEMS 21 behavioral pattern while stability is the ability to preserve behavioral petterns in a variable environment. Flexibility requires thus the release from stability of a behavior to initiate another stable behavior. Flexibility is the ability to decouple and couple motor action, perception and internal CNS states. Stability aims to maintain a selected behavior or, in other words, to stabilize a particular coupling. The interplay between flexibility and stability is the property that dynamical systems are particularly well suited to deal with. Dynamical systems models employ variables that directly define the behavior. These variables characterize the behavioral pattern and its quantitative as well as qualitative changes. For instance, in the study of interlimb coordination, the relative phase between the limbs allows for a faithfull description of the behavioral pattern transitions (Kelso 1984; Kelso et al. 1981). Another example of behavioral variable is, in goal-directed movements, the position and velocity of the end-effector that characterize the end-effector trajectory (Schöner 199). The stability property of the behavioral variables determines the system behavior and in particular its variability. The stability of a behavior is directly reflected in the variability of the variable that describes the behavior. The more stable is the behavior the less variable is the behavioral variable. A behavioral variable is characterized in particular by its fixed points along the behavioral dimension (Glendinning 1994; Drazin 1994). A fixed point corresponds to a state of a behavioral variable at which the rate of change of the variable is null. An attractor is a fixed point toward which the behavioral variable progresses in the presence of noise and evolves around because of the noise when no qualitative changes of the system happen. As long as fluctuations do not force the state of the variable outside the basin of attraction of the attractor, the behavioral variable evolves around the attractor. The attractor needs not to be fixed in time. The behavioral variable is however constantly attracted toward this moving attractor. Attractors determine the qualitative and quantitative features of a behavioral variable. Other fixed points that influence the pattern of behavior are repellors which, as the name indicates, are states from which the behavioral variable is repelled. The state of the behavioral variable is unstable under noise pressure at the repellor position. The state of the variable may reach temporarily a repellor but rapidly diverges from it because of the stochastic forces. The nature and the number of fixed points may change. These qualitative changes, called nonequilibrium phase transitions, express the diversity of the behavioral patterns. These transitions occur, in the order parameter space, at particular points called bifurcations (Schöner & Kelso 1988; Glendinning 1994; Drazin 1994). The order parameters are collective variables that describe the dynamics of the system. At a bifurcation, qualitative changes are induced in the system behavior. For instance, the transitions between the

22 22 Introduction various gaits in animal locomotion can be described by bifurcations. An order parameter is, for instance, in interlimb coordination the frequency of the rhythmic movement. The emergent patterns at a bifurcation are said to be self-organized because the order parameter that induces the changes does not contain information about the new pattern. The emerging behavioral pattern is only caused by specific interactions between the numerous subsystems that are involved in the behavior (Schöner & Kelso 1988; Schöner 199). Dynamical systems explain successfully interlimb coordination and in particular the relative phase transitions of the limbs (Kelso 1984; Schöner 199; Collins et al. 1998). The coordination of phase and antiphase rhythmic movements is accounted for in a framework that reproduces the spontaneous phase transitions between these two coordination patterns as the frequency of the movement is varied. The concept of behavioral information is introduced to describe adaptation of the stable behavioral patterns to various constraints from the extrapersonal and intrapersonal environments (Schöner 199). A behavioral information sets the behavioral pattern which is adapted to the environmental circumstances. Dynamical systems approach successfully explains rhythmic movements but is not restricted to these movements. The dynamical systems framework has been extended to account for discrete movements (Schöner 199). In a model that generates discrete and rhythmic movements, the temporal order of rhythmic movements as well as the temporal order of discrete movements come from limit cycle oscillators (sec ). Limit cycle oscillators are stable selfsustained nonlinear oscillators. A perturbation that drives the oscillator state away from the limit cycle is resisted such that the oscillator state returns back on the limit cycle. A perturbation that drives the oscillator forward or backward on the limit cycle is not resisted in isolated oscillators. The movement model in (Schöner 199) relies on limit cycle stability property to capture the collective behavior of the system DOFs at the level of the end-effector trajectory. Limit cycles have been proposed to be a general framework that accounts for the numerous DOFs and the collective outcome of these DOFs in generating movements (Kelso et al. 1981; Schöner & Kelso 1988). In this study, a dynamical systems framework is used for a model of discrete movement. The motor control strategy that underlines movement generation for a task redundant effector is based on the UnControlled Manifold (UCM) theory set within a dynamical systems framework (sec ). In addition, in our framework, an oscillator space is used to specify the movement task (sec ).

23 1.2. DYNAMICAL SYSTEMS The UnControlled Manifold (UCM) This section introduces the theoretical concept of the UnControlled Manifold (UCM). Experimental results support this theory for human movements. The current understanding of the UCM theory from these experimental results set, for our study, a framework for a motor control strategy. The UnControlled Manifold (UCM) theory proposes a motor control strategy for redundant systems that makes use of the abundance of solution inherent to redundancy (Schöner 1995). The UCM theory postulates that the taskredundant space of the effector is not homogeneous but is structured according to the task in a way that will be made clear below. The effector DOFs are organized in task equivalent and non task equivalent combinations. By definition of a redundant system, a set of combinations of the DOFs corresponds to the same task variable values (sec. 1.1). The structure of the effector space is set according to the task equivalent combinations of DOFs. This task structuring of the DOF space emphasizes an organization specific to the task. In the following, effector space and joint space are used as synonymous. The UCM theory is rooted in dynamical systems theory and the concept of stability (sec ). The task oriented organization of the joint space implies that the task variables must be stable. In terms of the joints, that means that the combinations of the joints that are not task equivalent must be stable while the stability of the task equivalent combinations is unimportant for the success of the task. A particular coordinate in the effector space is thus not an attractor point in the sense of dynamical systems theory. Rather fluctuations of the joint angles are tolerated so long these fluctuations do not affect the task variables. The structure of the joint variability reflects the task-oriented organization of the effector space and reveals a tolerant strategy for motor control. From a dynamical systems perspective, the effector thus resists perturbations in a task-equivalent manifold less than in the complementary, non task equivalent manifold. This motor control strategy does not enforce a single solution in the effector space but rather a set of solutions that is indifferent for the task success. This strategy makes use of the abundance of solutions offered by redundancy. So, a trajectory in the effector space is, unlike classical approaches to motor control, not a unique solution but a set of solutions or a manifold. The concept of an optimal joint trajectory is thus irrelevant in the context of a UCM motor control strategy. Let s consider the task of a human pointing to a target with a redundant arm (fig. 1.1). The task is successfully accomplished when the hand or the finger is in a specific position. A relevant task variable is consequently the hand position. The task possess less DOFs than the independent dimensions of the effector space assuming independent joints. The task itself does not

24 24 Introduction attribute a unique angle to all the joints because the system is redundant for the task. Thus, the successful task achievement is bounded to a specific subset of effector configurations. The hand position is not uniquely determined by one joint configuration but by a set of joint configurations. In other words, some effector configuration changes lead to hand position changes but other effector configuration changes do not lead to changes of the hand position. The more stable are the joint combinations that lead to a change of the hand position the less variable is the hand position about a specific configuration. The variability of the joint combinations that do not affect the hand position does not change the hand position by definition. Thus, the stability of the joints can be differentiated in the effector space between the stability of the joint combinations equivalent for the task and the stability of the joint combinations that are not equivalent for the task. The effector space stability is structured to stabilize the hand position. The hand position is preserved while the effector configuration is not specifically controlled. This structure allows to use various joint combinations to achieve the task. This is the UnControlled Manifold (UCM) theory. Experimentally, one can ask what are, in the sense of the UCM theory, the task variables for a particular movement. To test a specific experimental hypothesis, the joint variability in relationship to some task variables is examined (sec ). The variability can be decomposed into a component that lies in the UCM and a perpendicular component that affects the task variable. If the variability is structured such as to preserve the task variable, the hypothesis is accepted. The variability or UCM signature of a task variable is a higher variability in the subspace formed by the combinations of DOFs that do not affect the task variable (the Goal Equivalent Variability or GEV) than in the complementary subspace (the Non Goal Equivalent Variability or NGEV). If the stability is not structured the way predicted by the UCM theory, the hypothesis is rejected. This UCM signature is thus a signature of the UCM motor strategy applied to specific task variables. This UCM signature has been found experimentally for various tasks in various conditions (Scholz et al. 2; Scholz et al. 21; Reisman et al. 22; Tseng et al. 22; Scholz & Schöner 1999; Tseng et al. 23; Scholz et al. 22). The UCM theory is a motor control strategy for redundant systems whose signature is a specific structure of the joint variability. From an experimental viewpoint, the UCM theory is a method to identify task variables in various movements based on the UCM signature. The experimental procedure is explained in detail in the methods (sec ). Two task variables will repeatedly appear throughout this work. The task variable movement extent is a variable that represents the hand position along the straight line between the starting hand position and the target position in pointing movement. The task variable movement direction is the variable that represents the deviations

25 1.2. DYNAMICAL SYSTEMS 25 of the hand from the straight line between the starting hand position and the target position in pointing task. A pointing movement can thus be examined for the hypothesis movement extent and the hypothesis movement direction. The UCM signature reflects a task-oriented organization of the joint space. Thus, when the task is changed, the structure of that space should be adapted. This hypothesis of task-dependent changes of the joint space structure has been examined experimentally (Scholz et al. 2; Tseng et al. 22). Two tasks involving the same effector, the arm, are compared. The first task is a pointing task. It is shown that the finger position is a task variable at the end of the movement in the sense of the UCM theory. A pistol shooting task consists of shooting at a target as fast as possible starting from a fixed arm configuration. In this shooting experiment, the hand or gun position, is not a task variable in the sense of the UCM theory whereas the gun barrel direction is a task variable in the sense of the UCM theory at the end of the movement. Although both tasks involve arm movements, the applied motor control strategy to achieve the task is different. The UCM motor control strategy is thus task dependent. The task difficulty also affects the effector variability. A more difficult task can potentially induce more variability of the task variables. The way the organism deals with these more challenging conditions can be an indication of a particular motor control strategy. In a sit-to-stand task, a subject moves from a sitting position to a standing position. The sensory information available to the subject is manipulated (Scholz et al. 21; Scholz & Schöner 1999). For instance, the subject is deprived of vision. In the deprived condition, the variability of the task variable horizontal head position increases because horizon information is normally retrieved from the visual system. The subject can still rely on the less accurate vestibular system to detect the horizon. The variability increase of the task variable is however less than that could be expected from the joint variability alone as shown by the increased difference between GEV and NGEV. In the more challenging condition, there is relatively more variability in the uncontrolled manifold. This increase is not surprising in the sense that a UCM motor control strategy does not resist perturbations in the uncontrolled manifold while perturbations in the task manifold are resisted. The uncontrolled manifold is a manifold whose stability is lower than the task manifold. In fact, the name uncontrolled suggests that the variability within the uncontrolled manifold should accumulate as the movement unfolds in time like a diffusion process. Experimental data show however that this is not the case (Tseng et al. 22; Tseng et al. 23). The variability in the task equivalent space increases at the beginning of the movement but decreases toward the end of the movement for the hypothesis movement extent. For the hypothesis movement direction, the variability within the uncontrolled manifold shows the same profile in time as for the hypothesis movement ex-

26 26 Introduction Figure 1.2: In a shooting task experiment, the task variable is the pistol barrel orientation. On the left part of the figure are the pooled unsuccessful trials on the right is the pool of the successful trials for one participant. P3, p6 and p9 refer to time instant along the normalized movement time in percent of the trajectory (1% is the total trajectory). The dark bars are the variability within the uncontrolled manifold. The light bars are the variability in the non task equivalent space. When the subject misses the target the variability at p9 in both subspaces are similar (left) while when the subject successfully hits the target, the variability at p9 in the task equivalent space is higher compare to the variability in the non task equivalent space (right). tent. The UCM appears in fact not uncontrolled but less controlled, compared to the non task equivalent subspace. It is important to note that to compute the joint variability the trajectories for all the trials are aligned in time. This alignment procedure, a time normalization of the trials, may cancel joint variability related to movement timing variability. In theory, the time normalization cancels the accumulation of errors when the movement unfolds. So, the term uncontrolled is justified. The UCM theory is based on a differential stability in the joint space that preserves the task variable. It may be asked if this structure decreases the variability of the task variable compared to a structure that enforces a unique solution in the task space. There is one piece of experimental evidence, in particular, that points toward a reduction of the variability of the task variable. If the UCM strategy is aimed at reducing task variable variability, then a failure to implement this strategy should result in an increased variability of the task variable and maybe a failure to accomplish the task. In the shooting experiment, a subject failed to hit the target a number of times (Scholz et al. 2). When the successfull trials and respectively the missed trials were pooled together, the successfull trials were accompanied by the appropriate signature of a task variable while the missed trials showed a failure to implement the UCM strategy (fig. 1.2 reproduced from (Scholz et al. 2)). This result hints

27 1.3. EQUILIBRIUM POINT HYPOTHESIS (EPH) 27 to a potential role of the UCM motor control strategy to reduce task variable variability. One could also hypothezised that the UCM motor control strategy potentially channels variability into the task equivalent space, thus reducing the effect of experimental condition changes on the variability of the task variable. This is based on assuming that the increased joint variability induced by more challenging task conditions is redistributed such as to maintain to an acceptable level the task variable variability. There is, however, no evidence that the purpose of the UCM motor control strategy is to decrease task variable variability by channelling variability in the UCM. Another hypothesis for the benefit of the UCM motor control strategy is that suppressing unnecessary corrective actions at the muscle level reduces interaction torques (Scholz et al. 2). This suppressive mechanism may also reduce the total amount of joint torques during the movement. Yet another hypothesis is that the UCM signature reflects the internal organization of the brain processes without direct positive effects on the task performance. This hypothesis is related indirectly to the efficiency of living systems to evolve in variable environments. The interactions between the organism and its environment have a purpose which is dictated by the task. So, to be efficient, the organism must be able to stabilize and adapt its behavior in relationship to the environment. This can be achieved if the processes that underline the behavior are oriented to the task and not to the mean by which the task is achieved, i.e the effector. 1.3 Equilibrium Point Hypothesis (EPH) Equilibrium point hypothesis and movements An analysis of movement requires to consider two broad classes of variables. The first set is made up of kinematic and kinetic variables. These variables characterize the motor outputs or the motor realization and as such can be measured by an external observer. They depend on the internal and external environments. The second set is the control variables that are internal variables peculiar to the system. These variables set a motor state independently from the environment. The motor output emerges from the interactions between the internal variables and the environment (Feldman & Levin 1995; Ghafouri & Feldman 21). Muscles can be characterized by a static force-length relationship. Static means that the first time derivative of the length is null. This characteristic can be measured experimentally (Feldman 1965). Forces are applied to an almost intact muscle preparation of which one end is separated from the bone. The forces cause the muscle to stretch. The muscle length increases monotonically as the force increases. This relationship between applied forces

28 28 Introduction and muscle lengths forms a force-length characteristic. Descending neuronal pathways act on the motorneuron pool to activate muscles. This neuronal activity influences the muscle state and its force-length characteristic. To one constant level of activity of the descending neurons corresponds one unique force-length characteristic. The characteristic is shifted along the length axis as the neuronal activity is varied. The neuronal activity determines a set of more or less parallel force length characteristics. Within the Equilibrium Point Hypothesis (EPH) theory, the descending neuronal input is interpreted as an internal variable as defined above. The force-length characteristic of a single muscle can also be measured for a multimuscle driven joint in the form of a torque-angle relationship. A subject is prescribed a given starting position in the joint load space. Varying the load generates a torque-angle characteristic. To each characteristic corresponds a specific internal state determined in the experiment by the starting experimental condition. The internal state is assumed to be constant if the subject does not intentionally react to the load changes. This assumption relies on the Do not intervene paradigm in which the subject is asked not to react to the experimental condition changes. The reliability of the method depends critically on the subject ability to suppress any voluntary correction or any internal state change. The torque-angle characteristics and the force-length characteristics have similar features (Latash 1993). These characteristics for the muscles and for the joints are called invariant characteristics (IC) (Feldman 1965; Feldman 1966). Invariance refers to the invariant shape of the characteristics as the internal state is varied. The concept of a joint as an independent structure driven by an ensemble of muscles is an extension from a single muscle and will be used throughout this work in different contexts. In particular, joint muscle refers to an abstract muscular element made up of every muscle acting at a joint and that set this joint in motion. The concept of a joint as an independent DOF of an effector has been already used in the previous sections (sec. 1.1 and sec. 1.2). In the following, the EPH theory is presented for the muscles because this is more intuitive but this theory applies also to the joints as emphasized by the experiment reported above. These invariant characteristics have many implications. First, it should be noted that they are inherent property of muscles. Varying force changes the muscle length because a muscle is by no mean a rigid body. The second important aspect of the invariant characteristics is the shape invariance. A characteristic defines uniquely a set of positions in the force-length space of the muscle. The third aspect is the threshold property of the characteristic. The muscle stretches in reaction to applied forces only above a specific muscle length. One characteristic can thus be unequivocally defined by its own

29 1.3. EQUILIBRIUM POINT HYPOTHESIS (EPH) 29 threshold. To specify another unique force-length set in the force-length space, another invariant characteristic is selected by setting another threshold. With respect to one particular muscle, this is equivalent to varying the internal state or the descending neuronal activity. A muscle length is then uniquely defined by a two-dimensional vector made up of a threshold and an external force, (λ, F). This vector sets an equilibrium position and the related theory is the Equilibrium Point Hypothesis (EPH) theory. An equilibrium position is determined by two independent variables and consequently there are two ways to change the equilibrium position of a muscle. The equilibrium position is changed if the external force varies in which case the equilibrium position moves along the invariant characteristic toward a new resting length. This equilibrium position change is not the consequence of internal state changes. It is important to emphasize that although the kinetic variables are changed, as seen from an observer, the internal variables are not modified, as seen from the CNS. The motor output is thus not entirely determined by the internal state. This feature distinguishes equilibrium point models from position servocontrollers. The second way to change the equilibrium position is to change the threshold. This change is the consequence of an internal state change described by an internal variable. This leads to a shift of the invariant characteristic along the length axis. The EPH theory offers a framework to specify the muscle state. In addition, this framework offers a simple interpretation of internal variables in terms of muscle thresholds. One version of the EPH-based models, the λ model, is examined in more detail. From an external observer viewpoint, muscles generate position- and external force-dependent forces and also velocity-dependent force in the nonstatic case not yet considered (Feldman & Levin 1995). The CNS can modify the muscle responses to a fixed environment operating on the muscle threshold. This threshold is given the name λ. Some authors support the idea that this threshold is an abstract internal variable rather than being potentially measurable. This is the abstract version of the λ model (Latash 1993). Feldman and coworkers strive to give a physiological meaning to the threshold. According to their proposal, this threshold concept can be traced back to the α-motorneuron membrane threshold (Feldman & Levin 1995). Proprioceptive afferents induce muscle activity when a motorneuron threshold is exceeded. The threshold of the tonic stretch reflex is assumed to be explicitly set by an internal control variable, λ. The motorneuron threshold is, however, not only set by the descending neuronal command. Indirect inputs from various sensory modalities such as for instance the skin receptors, the tendon receptors or the joint capsules also act on the threshold level. Intermuscular interactions affect also indirectly the threshold of the motorneuron (Feldman & Levin 1995). The notion of the central variable λ becomes confused at this point because of the threshold dependence on the muscle interactions although the dedicated descending command is still a central variable. Our framework

30 3 Introduction is based on the abstract λ model because an account of accurate physiological mechanisms of muscles is beyond our objectives (sec ). In addition, the abstract version of the λ model gives a behavioral function to λ explicitly and is thus appropriate for a joint muscle model. The reflex properties of the joints lead to another interesting property of the limb. Within an EPH framework at the joint level and for a predetermined posture, muscles at the joint are activated if an external transient perturbation drives the joint away from its resting position. The muscle activity moves the articulation back toward its pre-perturbation equilibrium state. This principle is called the equifinality principle. Equifinality is a core concept in the EPH theory that relies in experiments on the assumption of a constant internal state. Violations of the equifinality principle are however claimed in certain conditions (Dizio & Lackner 1995; Hinder & Milner 23; Feldman et al. 1998). A force field is a vector field of force. A force field can be defined in the end-effector space or in the joint space. For each position in the end-effector space or in the joint space, there is a force vector that defines the force direction and the force strength that are exerted on the system at this particular position. It has been shown that the ensemble of the arm muscles establishes an elastic force field at the end-effector whose property is to resist transient mechanical perturbations that drive the limb away from its pre-perturbation position (Won & Hogan 1995; Hogan 1985b; Mussa-Ivaldi et al. 1985; Shadmehr et al. 1993). A similar force field has been attested for individual joints (Kelso & Holt 198). In terms of muscles, that means that a change in muscle length elicits a muscle force that tends to restore the pre-perturbation muscle length. The muscle state is characterized by an equilibrium position beyond the passive inherent force-length property of muscle. Thus, the forcelength characteristic is not simply an inherent property of the muscles but results from the ability of muscle to store elastic energy. The muscles behave essentially like a spring. Note that the muscle spring-like property has been attributed to the muscle alone or to the tonic stretch reflex of the muscle like in the λ model (Latash 1993; Feldman & Levin 1995; Hogan 1984; Hogan 1985b; Bizzi et al. 1984). This distinction is not made in this study. The spring-like property of the muscles has a drawback for the limb. In order to move from one position to another, the posture has to be reset otherwise the limb will always be attracted back toward the original position. This is the von Holst principle (see for instance (Ostry & Feldman 23) or an analogy in robotics in (Williamson 1999)). The EPH theory provides exactly a solution to this paradox because movement is the consequence of resetting posture (see below). Indeed, modulating the internal muscle state changes the force field structure in particular the force field equilibrium position. A pure force approach to movement generation does not necessarily solve this paradox of

31 1.3. EQUILIBRIUM POINT HYPOTHESIS (EPH) 31 the spring property of muscles. The elastic force field property of muscles in humans has also been attested by spinal cord and higher brain centre stimulations in the frog (Bizzi & Mussa- Ivaldi 1995; D Avella & Bizzi 1998). It is proposed that the motor primitives, a set of basic force fields, may be combined to produce movements. This experiment shows that the elastic property of muscle is logically not only a feature of human muscles. So far, the notion of a joint muscle is presented such that an internal variable set the invariant characteristic (IC) of the joint muscle. This internal variable forms a (1D) coordinate system in which the joint IC can be specified. Muscles at a joint are usually described in terms of an agonist and an antagonist muscle group with opposite actions. For both muscle groups, one internal variable defines, according to the EPH theory, an invariant characteristic for one muscle group. The net muscle torque at the joint is the sum of the torques of both muscle groups. To set a joint IC in this framework requires two central variables that form a coordinate system in which the joint IC is specified. It is also possible to describe these internal control variables in another coordinate frame, namely the r and c command. The r command specifies the desired joint IC. The c command or cocontraction is the difference between the antagonist and the agonist λ s divided by two (Latash 1993). It specifies the slope of the joint compliant characteristic. The cocontraction command modifies independently from the joint position the joint stiffness (sec. 1.4). This possibility to change the joint stiffness is of interest to deal with new environments (Burdet et al. 21; Hogan 199; Hogan 1985a). So far only posture has been considered but movements can also be accounted for within the EPH framework. Only movements in the absence of external forces are envisaged in this study. A movement consists of a set of transient joint postures. So, in order to generate a movement, a time series of joint ICs should be specified. Moving means thus to switch the internal state. Because the force field structure is changed by a modulation of the internal state, the effector must adopt a new state which corresponds to the equilibrium position of the newly structured force field. The effector evolves toward this new equilibrium position because of the elastic property of the muscles. The limb thus tracks the equilibrium position of the force field. The set of internally defined states during the movement forms a virtual trajectory (Feldman & Levin 1995; Latash 1993). The shape of the virtual trajectory has been discussed abundantly in the literature (Latash 1993; Ghafouri & Feldman 21; Latash 1994; Barto et al. 1999; Cesari et al. 21; Lussanet et al. 22; Gribble & Ostry 2) (sec. 2.1). The assumption that muscle activation can be expressed in terms of muscle lengths is an advantage of the λ model. This assumption simplifies the planning of the movement as long as the virtual

32 32 Introduction trajectory remains close to the real effector trajectory (Latash 1993). Thus, the joint muscle stiffness must be high enough (sec. 1.4). With a weak elastic field, the movement can still be achieved but the virtual trajectory in the joint space or in the muscle space is more complex and even outside the physiological range to account for the kinematic features of the movement (Barto et al. 1999). The EPH theory offers a general framework to account for movements. The λ model is one model within this framework but the EPH theory is not only restricted to this model. Different versions of the EPH model have been developed apart from the original version proposed by Feldman (Feldman & Levin 1995; Flanagan et al. 1993; Latash 1993). The original λ model and the abstract λ model have been discussed above. Two other EPH models deserve mentioning. One model stipulates a pulse-step like virtual trajectory (Barto et al. 1999). The virtual equilibrium point first overshoots the desired target which generates a strong acceleration to overcome muscle inertia. The pulse command starts the movement and may be interpreted as equivalent to the cocontraction command of the λ model. The step command set the final movement state and terminates the movement. The main difference with the λ model is that the cocontraction-like command initiates the movement (Barto et al. 1999). In the original λ model, the c command contributes to the stability of the movement rather than to the movement itself. This pulsestep approach is based on the assumption that EMG bursts reflect directly the neuronal descending command (Gottlieb 1993). Two models differ from the original EPH models in that the central variable λ is a two dimensional vector of the form, (λ, λ) (McIntyre & Bizzi 1993; Lussanet et al. 22; Shadmehr & Mussa-Ivaldi 1994). Bizzi & al. propose a model with a second velocity-based central variable that increases the velocity feedback gain in addition to the position-based central variable (McIntyre & Bizzi 1993). In the second model, the torques at the joint depend on an elastic force field but also on a viscous force field. (Lussanet et al. 22; Shadmehr & Mussa- Ivaldi 1994). The limb force field is not only dependent on the difference between the virtual and the real position but also on the first derivative of this difference. The assumption for more control variables is not at odds with Feldman s view. In his early work, Feldman postulates the existence of various central variables for rhythmic movements with the so called dynamic parameter or later a velocity dependent central variable (Feldman 1966; Feldman & Levin 1995). Movement generation is a consequence of the modification of the central variables independently from the environment. Within the reference frame interpretation of the λ model, movement is thus a shift of the internal reference frame (Feldman & Levin 1995). This intrapersonal frame of reference which defines a body referent configuration in the sense that it sets the muscle ICs

33 1.3. EQUILIBRIUM POINT HYPOTHESIS (EPH) 33 does not specify an orientation in the extrapersonal space. The intrapersonal frame must be calibrated on the extrapersonal space (Feldman & Levin 1995; Lackner & Dizio 2). This idea is closely related to the concepts of the body schema and the body image which are a representation of the body configurations that integrates various motor and sensory information (Holmes & Spence 24; Paillard 1999; Gallagher & Cole 1995) (sec ) Equilibrium point hypothesis versus inverse dynamics Equilibrium point hypothesis theory and inverse dynamics are often opposed as two radically different views of motor control (sec ). A simple consideration shows that both approaches are in fact intimately related in some aspects. The next equation is the classical λ model for a linear joint muscle model (sec. 1.4 and sec ) (Hogan 1984) I d2 θ = K (θ λ) (1.1) dt2 λ expresses the joint virtual position in the joint space according to the λ model. The left part of the equation is a simple model of a rotating rigid body with inertia I. The right part of the equation is a linear torque field, i.e the torque increases linearly as the difference between the desired virtual position and the real position increases. This equation can be rewritten in a slightly different form I d2 θ = K θ + φ (1.2) dt2 In that case, φ has the dimension of a torque. When the actuation function of the muscle is known, φ can be specified such that the system (θ) behaves according to the task. This is a kind of inverse dynamic approach (sec ) (Uno et al. 1989; Wolpert & Kawato 1998; Kawato 199). For simplicity the problem is presented for a single DOF system. The issue of inverse dynamic versus EPH model is however especially acute for multijoint movements. One of the advantages of the λ model is the simplicity of the virtual path. It can be true only if the arm dynamics in multijoint movements can be ignored because the joint elastic force field resists enough perturbing torques (K must be high enough in equ. 1.1). Without a strong elastic force field, an inverse dynamic calculation must be performed to account for the end-effector straight paths in multijoint movement according to the inverse dynamics perspective. It is believed in light of recent impedance measurements that the joint stiffness is too small to account for a simple virtual trajectory (Gomi & Kawato 1996). It should be emphasized that this reason alone cannot discard the EPH approach because it is still possible to account for movements in a

34 34 Introduction weak force field with a virtual trajectory non isomorphic to the effector trajectory (Ostry & Feldman 23; Barto et al. 1999; Latash 1993). Inverse dynamics based models often neglect the postural resetting issue which is required for the limb to adopt a new location without being attracted back to the previous equilibrium position (see for an exception (Shadmehr & Mussa- Ivaldi 1994)). In fact, the limb elastic property is most of the time ignored within control theory frameworks (Ostry & Feldman 23) Equilibrium point hypothesis and oscillators Rhythmic movements exhibit the properties of limit cycle oscillator (Kelso et al. 1981). One can ask what is the relationship between the dynamical systems approach and the EPH theory if any. Two aspects are briefly examined, the oscillators and the stability. The oscillator property of the limb is also attested by the EPH theory (Feldman & Levin 1995). Equation 1.1 is indeed an oscillator. The difference between both frameworks lies in the oscillator property. The dynamic systems approach rests on nonlinear selfsustained oscillators. The EPH theory supports linear not selfsustained oscillators. In the dynamical systems approach, the observed behavior of the limb is the result of a collective behavior of many elements that behave like mutually coupled limit cycles. The EPH theory assumes a preprogrammed virtual path that may also emerge from the collective behavior of numerous elements. In order to generate rhythmic movements in equation 1.1, λ should take the form of a limit cycle oscillator to entrain the limb oscillator. In our study, limit cycle oscillators are used to entrain the effector within an EPH framework. The EPH theory assumes a force field whose equilibrium position is unique. The system state moves toward the equilibrium position according to the gradient of the elastic potential field. The equilibrium position of the force field defines a state toward which the system evolves (sec ). This attractor property for single joint movement has been measured experimentally (Bizzi et al. 1982; Bizzi et al. 1984). In addition, two sets of experiments account for the equilibrium property of the end-effector. The first experiments show that the end-effector at rest behaves like a damped spring (Mussa-Ivaldi et al. 1985; Hogan 1985b; Hogan 199). If the end-effector deviates from its resting position because of a transient mechanical perturbation, the limb muscles generate a restoring force. An experiment proves formally the existence of a potential elastic field, i.e a curl-free force field (Hogan 1985b). The second evidence is the equilibrium property of the end-effector path during movement (Won & Hogan 1995). In this experiment, the end-effector is forced to follow a slightly curved path between the starting and the target position. During the movement, forces that tend inward the curvature are exerted on the endeffector. These forces cannot be accounted for by interaction torques for their amplitudes are too important. This equilibrium property dominates the arm

35 1.4. SCIENTIFIC APPROACH TO MOTOR CONTROL 35 dynamics and cannot be neglected as sometimes suggested (Won & Hogan 1995). This attractor behavior is fundamental to manipulate objects because the stability of a manipulator is not guaranteed in this task. An end-effector spring-like behavior is enough to guarantee such a stability (Hogan 1985a). It is also important to note that the spring-like behavior at the joint does not necessarily guarantee a stable behavior for the limb. A similar issue of stability underlies the force field primitives theory (D Avella & Bizzi 1998; Slotine & Lohmiller 21). 1.4 Scientific approach to motor control A modular approach to motor function Motor control is traditionally conceived of as a hierarchical system whose inputs are the information provided by the sensory systems and whose outputs are motor commands to the muscles. In this framework, the brain is the controller that computes the commands while the muscles are the motors that drive the effector. This conceptual view of motor control has its root in control theory and is known as biological cybernetics. This approach is supported by the apparent modular organization of the brain (Wiesendanger 199; Keele et al. 199; Dehaene et al. 1998; Loeb et al. 1999; Cisek et al. 1998; Hoff & Arbib 1993; Wolpert & Kawato 1998; Kawato 199). Based on physiological data, from electrophysiology recordings, brain imaging studies, clinical studies of lesions and anatomical data, the brain is broken up into various distinctive functional regions. This brain dissection logically fosters hierarchical and heterarchical functional concepts that are either explicitly but most often implicitly assumed. This modular functionality approach is expressed in engineering style models that essentially describe functions in terms of input-output systems connected through forward and feedback connections (Bhushan & Shadmehr 1999; Cisek et al. 1998; Mehta & Schaal 21). This framework implies the possibility to break down the complex system made up of the CNS and the body into weakly interacting functional units. The socalled internal models, including forward models and inverse dynamics models are the best up-to-date examples of this methodology (sec ). A conceptual motivation for a computational approach to brain function is probably to be found in the universal Turing machine idea. This brain view governs thinking in motor control although its foundation is rather intuitive instead of deeply founded. The goal in this paragraph is not to develop an alternative motor control theory but to point out that the appropriate language to talk about brain functions must be critically evaluated especially with respect to the implicit assumptions underlying each approach. The control theory approach assumes that the CNS can be separated from the body and furthermore that the brain and the body can be divided into specific func-

36 36 Introduction tional units. Theories that challenge this traditional brain understanding are to be found, for instance within the dynamical systems approach to motor function (Kelso et al. 1981; Schöner & Kelso 1988; Sternad 2), for instance the UnControlled Manifold theory (sec ) or in integrative theory that embodied action and perception in a single framework (Gallese 23; Hommel et al. 21). A few relevant examples that illustrate the implications of an appropriate language to talk about motor control are briefly presented below. The motor concept for the muscles is called into question with the viscoelastic properties of the muscles. In the traditional view, movements are parametrized naturally with position, velocity and acceleration. From a viscoelastic perspective, parameters are inertia, stiffness and viscosity. This viscoelastic property of muscle confers to the limb attributes that are beyond a passive controlled effector. A limb actively resists external mechanical perturbations (Bizzi et al. 1982; Hogan 1985b; Ostry & Feldman 23) (sec. 1.3 and sec ). The elastic property confers to the limb the capability to generate a restoring force in response to a transient perturbation. Elasticity is described in the linear domain by Hooke s law F = K (x x p ) (1.3) where F is the force, K is the stiffness, x the current position and x p is the resting position. The stiffness indicates how much does the system tolerate a position deviation for a specific force. This elastic property of the limb is characterized in numerous experiments (Bizzi et al. 1982; Kelso & Holt 198; Feldman 1965; Feldman 1966; Won & Hogan 1995; Mussa-Ivaldi et al. 1985) (sec. 1.3). Furthermore, muscles exhibit a velocity dependent resistance. A linear relationship for viscosity is F = µ ẋ (1.4) with F the force, µ the viscosity and x is the time derivative of the position. A muscle is thus a damped oscillator rather than a simple actuator that transforms an electric stimulation into a force. Muscle tension is dependent on muscle position, muscle velocity and muscle activity. The muscles are strongly nonlinear (Feldman 1965). Thus, the laws 1.3 and 1.4 are only valid locally (Winters & Woo 199; Latash 1993; Houk et al. 22). Muscle active state, in a simplified view the motorneuron activity, furthermore changes muscle attributes like the muscle impedance. One aspect of these attribute changes induced by the muscle active state is the concept of a virtual equilibrium position (Hogan 1984; Hogan 199). A change in muscle activity leads to a change of the virtual position toward which the muscle is driven. Thus, muscle activity specifies a virtual state.

37 1.4. SCIENTIFIC APPROACH TO MOTOR CONTROL 37 The notion of a mechanical impedance summarizes all motion-dependent effects of a muscle. The stiffness and viscosity are only the simplest components that make up the impedance and often higher terms are ignored (Hogan 199). Muscle stiffness and viscosity are modulated in a complex way by parameters like the workspace position or the force (Gomi & Osu 1998; Tsuji et al. 1995; Gomi & Kawato 1996; Shadmehr et al. 1993). The impedance is also typically modulated by the CNS during learning (Burdet et al. 21; Bhushan & Shadmehr 1999; Shadmehr & Mussa-Ivaldi 1994). The muscle properties originate from the muscle structure and its appended neuronal structures that form a complex system with multiple interactions (Cisek & Lephart 22). Thus, a muscle cannot be assimilated to a simple motor and its function cannot be reduced to generating force. Muscles grant some specific properties to the arm that a rigid manipulator driven by motors does not have. Traditionally, the CNS is seen as the controller that generates the adequate motor commands. This is the hierarchical top-down approach (Loeb et al. 1999; Goodman & Gottlieb 1995). Each hypothezised subsystem is analyzed individually experimentally and theoretically. An organism is in fact made up of numerous strongly mutually interacting elements. The elements constrain each other such that their behaviors depend on their mutual interactions. An isolated element demonstrates a behavior that differs from the behavior exhibited when the element is embedded in its environment. A stable and reproducible feature is the global outcome of the system that unambiguously describes the system function. To illustrate this idea, oscillators are a good example. The behavior of an ensemble of interacting limit cycles can explain stable coordination patterns in rhythmic movements (Kelso et al. 1981; Schöner & Kelso 1988; Schöner 1995). Indeed, limbs behave qualitatively like limit cycle oscillator. Coupling is also fundamental for the Central Pattern Generators (CPGs). CPGs are neuronal oscillators whose location is the spinal cord and which belong to the motor system. A CPG behavior is determined by regulated interactions among various neuronal elements. A CPG is not a stereotypical oscillator but rather generates qualitatively dissimilar patterns depending on its direct environment. These considerations constrain the framework to analyze movement. In particular, the assumptions that underline the analysis must be carefully examined. The use of a particular theoretical framework has also consequences on the meaning of the experimentally reported features of the living organisms. A concrete example is variability. Variability is omnipresent in biological processes. From a control theory viewpoint, the CNS controls the effector to generate the appropriate behavior. The CNS controls variables. Any variability reflects the failure to better control the variables. Within a dynamical systems framework, variability is inherent to the dynamics of the organism

38 38 Introduction embedded in its complex environment. Variability comes from the continuous nature of the world (Schöner 1995) (sec. 1.2). From a dynamical systems perspective, variability permits also to switch from one behavioral pattern to another Biological cybernetics Internal models Classical approaches to motor control require to issue the appropriate motor commands to the muscles during movement. These commands are issued either independently of the effector system in open loop, based on the concept of a motor program, or based on perceived and estimated system state in closed loop (Gottlieb 1993; Bhushan & Shadmehr 1999). In the closed loop approach, the feedback properties constrain the control process. The controller s knowledge of the system state is updated based on previous information, new sensory information and issued motor commands. This task may be expressed generally as u(t) = π(x(t), t, α) (1.5) where u(t) is the muscle command vector, x(t) is the vector of the estimated system state, t is time and α is a vector of parameters that are adjusted during learning (Mehta & Schaal 21). A direct controller is a system whose inputs are behavioral variables and whose outputs are motor commands. The system input-output function is learnt by reinforcement learning. Within this framework, no modules are explicitly assumed and the internal processes are hidden within the system. The concept is to mimic living system behaviors based on a neural network whose connectivity and complexity may also be learnt. The system performance is determined by the learning process. The internal network organization is largely unknown and may be significantly unrelated to the brain structure. The alternative to a direct controller is an indirect controller that is based on supervised learning. The indirect controller is made up of various information processing modules. In motor control, special attention is given to specific type of internal models, the forward models that reproduce the sensory consequences of a motor command and inverse models that find the desired motor command to achieve a specific task (Wolpert & Kawato 1998; Desmurget & Grafton 2; van Beers et al. 22; Mehta & Schaal 21; Bhushan & Shadmehr 1999). The advantage of a sensory forward model is the possibility to distinguish the self generated sensory changes from the externally induced sensory changes. The inverse model function is to account for the physics of the world. The best known example is an inverse model of the arm dynamics (Jordan & Rumelhart 1992). Forward models and

39 1.4. SCIENTIFIC APPROACH TO MOTOR CONTROL 39 inverse dynamics models are used in various control scheme that include multiple feedback loops (Bhushan & Shadmehr 1999; Mehta & Schaal 21; Jordan & Rumelhart 1992). These internal models typically form quantitative models of the internal and external world. The existence of internal models with a world-simulation function in the brain is debated (Ostry & Feldman 23). The ability to interact with a rapidly changing environment, for instance when catching a ball or anticipating an action like moving an empty or full bottle indicates that the CNS possesses some predictive capabilities to estimate and adapt to the environment (Mehta & Schaal 21; Gribble & Ostry 2). Another instance is the anticipative adaptive muscle activity that occurs at a stationary joint when another joint moves to compensate for interaction torques at the stationary joint (Gribble & Ostry 1999). In another experiment, subjects also adapt after a few trials to an environment in which Coriolis forces are induced although they are never given visual feedbacks on the success of the task (Dizio & Lackner 1995). These examples do not prove however that a simulation of the consequences of actions is internally performed (Gribble & Ostry 2). Alternatives to this class of internal models that perform quantitative world simulation exist. Instances of such alternatives are feedbacks made up of look-ahead units which are integrators limited to a small time window to compensate for the feedback delays, or adaptive learning procedure (Hoff & Arbib 1993; Gribble & Ostry 2). There is so far no evidence that the CNS must perform internal simulations relying on a quantitative world model in spite of the popularity of this approach (Ostry & Feldman 23). Optimality The optimality principle for motor control has already been introduced as a reductionist method for redundant systems (sec. 1.1). In this section, the optimality approach is examined briefly. Optimality has a long tradition in motor control (Todorov 24). The optimality principles in motor control are aimed to find out what are the optimal rules that govern movement generation. Optimality ideas are motivated by the invariant features of movements like the bell-shape end-effector velocity profile and the straight path of the end-effector in pointing movements. Optimality principles are also anchored in evolutionary thinking. The optimal solution is often a minimum or maximum of a cost function for the movement time. Many cost functions have been proposed like a minimum torque change, a minimum muscle force change, a minimum endeffector jerk, a minimum variability of the final position of the end-effector, a maximum joint position comfort, a minimum effort of the muscle neural input, a minimum intervention principle (Flash & Hogan 1985; Uno et al. 1989; Otha et al. 23; Osu et al. 24; Cruse et al. 1993; Lan & Crago 1994). The concept is to discover the constraints that lead to the observed

40 4 Introduction movement. The ability of a model to perform well in diverse conditions relies on the choice of the cost function. None of the cost functions proposed so far has reproduced all movement and sensorimotor properties. Optimality models do not usually address the issue of the implementation of movement generation within the CNS. These models presumably imply a perfect realization of the plan assuming a controller. The cost function determines the ideal trajectory and the controller tries to match at best the plan. Thus, these models are closely connected to the hierarchical modular approaches briefly presented in section Indeed, the control model aims to find the control architecture that leads to the optimal solution Neural networks Motor control approaches based on neural networks are introduced separately because of their intrinsic dual nature. Neural networks are an ideal framework to implement some modules of an indirect controller (Jordan & Rumelhart 1992; Jordan 1996) (sec ). Neural networks also exhibit features of dynamical systems, like attractor properties (Glasius et al. 1996; Glasius et al. 1995; Erlhagen & Schöner 22; Amari 1977; Deneve et al. 21; Pouget & Snyder 2). A dual approach with neural networks and dynamical systems consists of a neural network which is a learnt map of states overlaid with a dynamical system or a potential field (Morasso & Sanguineti 1995; Toussaint 24). An attractor of the dynamical system points to a unique state of the cortical map based on some task constraints or task objectives. The basic substrate of the nervous system motivates numerous and various motor control models based on artificial neural networks (Morasso & Sanguineti 1995; Bullock & Grossberg 1988; Bullock et al. 1993; Glasius et al. 1996; Glasius et al. 1995; Kawato 199; Wolpert & Kawato 1998; Deneve et al. 21; Pouget & Snyder 2; Jordan & Rumelhart 1992; Jordan 1996). Three types of models are particularly relevant in this study and are briefly reviewed below (Deneve et al. 21; Pouget & Snyder 2; Glasius et al. 1996; Glasius et al. 1995; Morasso & Sanguineti 1995). The model of Grossberg & al. focuses on coordinate transformations between various sensory spaces and the motor space (Bullock & Grossberg 1988; Bullock et al. 1993). A weighted go signal accounts for movement velocity. Movement is generated through successive integration of direction vectors betwen the current position and the target position. This model accounts for redundant effectors by learning mappings and includes perception. In the model of Morasso & al. and Glasius & al., the focus is on an internal representation of the body, a so-called body schema, on which movements are planned (Glasius et al. 1996; Glasius et al. 1995; Morasso & Sanguineti 1995). In Morasso & al., the neural network learns a forward mapping between joint

41 1.5. MOTOR EQUIVALENCE AND SELFMOTION 41 angles to sensory consequence (Morasso & Sanguineti 1995). The neural network forms a map of body postures. A potential field that integrates movement constraints is overlaid over the body schema. The gradient of the potential field drives the current arm configuration toward a new configuration. Another dynamics that accounts for the target position and the task space trajectory shapes the potential field in time. The model from Glasius & al. consists of a sensory neural map that contains information about the movement constraints in an activation field and a motor neural network whose single local peak shifts from the current position of the arm toward the target (Glasius et al. 1996; Glasius et al. 1995). Both neural networks are isomorphic to the workspace and made up of excitatory and inhibitory lateral connections. The sensory map is connected to the motor map through feedforward connections. The current arm configuration is fed back into the motor map and constrains the map dynamics. Movement speed emerges from the network dynamics. All these neural models do not deal with the dynamic aspect of movements. These models also assume cortical maps with an appropriate topologies for the effector. The neural model of motor control proposed by Pouget & al. integrates multiple perceptual and motor information in a single map (Deneve et al. 21; Pouget & Snyder 2). This neural map with attractor properties has inputs from multiple sensory sources and outputs to the motor system. All information are multiplexed into a common framework thus avoiding explicit coordinate transformations, like in (Bullock & Grossberg 1988; Bullock et al. 1993). 1.5 Motor equivalence and Selfmotion Motor equivalence refers to the ability to perform a task with various taskequivalent effector configurations. Redundancy is thus a necessary condition to observe motor equivalence and motor equivalence is a priori inherent to redundancy (sec. 1.1). The issue about motor equivalence is thus if humans make use of the abundance of solution that is offered by redundancy. Motor equivalence is closely related to the UCM control strategy in the sense that both focus on the use of task equivalent solution in the redundant space (sec ). In the literature, motor equivalence also describes the ability to realize the same motor outcome with different effectors. For instance, one usually writes with a pen clutched in the hand but one can also write using the foot or the mouth. This different definition of motor equivalence comes from the concept of a generalized motor program. This motor program idea is a movement planning independently from the effector (Gottlieb 1993). In their cortical imaging study, Kelso & al. show that details of the task, flexion or extension, are not reflected in the brain activity pattern whereas the task requirement,

42 42 Introduction syncopation versus synchronization, is observed in the brain activity (Kelso et al. 1998). This shows motor equivalence in the sense that task planning occurs independently from the effector motion. Motor equivalence is also claimed for the redundant space of muscles. Different load conditions produce the same movement kinematics although the muscle EMGs vary (Levin et al. 23). The different EMG profiles while the movement kinematics is invariant are interpreted in terms of task equivalent muscle groupings. These various muscle groupings are claimed as evidence for motor equivalence. Motor equivalence as the use of task-equivalent effector configurations was first shown for vowels pronouncing (Kelso et al. 1984; Abbs et al. 1984). This experiment demonstrates that the task objective is not absolute articulatory positions but only relative positions. In this case, the task variable is the acoustic sound. This same task can be achieved with different task-equivalent effector configurations when, for instance, the upper lip is mechanically perturbed. In a trunk arrest paradigm, hand motion is invariant in spite of the perturbation in comparison to the unperturbed condition (Adamovitch et al. 21). Interestingly, this adaptation occurs early in the movement, even faster than any intentionally induced corrective movements. In a walking versus standing pointing task, the respective amount of joint excursions is different between the two experimental conditions but at the same time the hand kinematics is preserved (Marteniuk et al. 2). Motor equivalence is also demonstrated in a model based on local end-effector/ joint mapping whose input is a direction vector in the task space from the actual position to the target position. Movement is also properly achieved if a joint is impaired because the system is redundant (Guenther & Barreca 1997). Motor equivalence is thus also a signature for flexibility. The main challenge posed by the claim for motor equivalence is the variability of the task. The task is never reproduced perfectly. So, motor equivalence by nature is a comparison between the task variability and the effector variability. The claim of motor equivalence is based on a higher variability in the joint space than in the task space but, because the task space and the effector space are different, the task variability and the effector variability cannot be compared formally. This is the dilemma that the UCM approach offers to solve because the comparison is achieved in a common space, the effector space. Motor equivalence is attested with the relative difference of effector configurations that lies in the task equivalent subspace and in the non task equivalent subspace. Only if this configuration difference lies mainly in the task equivalent space can motor equivalence be claimed (sec and sec ). The relationship to the UCM approach is thus obvious because the taskequivalent joint configurations form the uncontrolled manifold (Scholz et al.

43 1.6. GOAL 43 2). Motor equivalence goes, however, beyond the usual movement variability between the trials and makes use abundantly of the task equivalent subspace. The UCM motor control strategy implies motor equivalence because, by definition, this strategy permits configuration deviations that are task equivalent and resists perturbations that induce non task equivalent change of the effector configuration. This tolerance for perturbations that lie in the task equivalent space is reflected in motor equivalence as defined above. A difference between the UCM signature and the motor equivalence signature lies in the cause for the use of task equivalent solutions. In our definition, the UCM signature comes from internal small continuous perturbations, i.e internal noise while motor equivalence results from constraints from the external environment that induce a larger use of the task equivalent solutions. However, in the literature, motor equivalence is also sometimes used with respect to noise (Martin et al. 22). Selfmotion is a motion of the joints of the redundant effector that does not affect the end-effector position. Any combination of joint motions that maps onto a zero end-effector change is selfmotion. Selfmotion is not an explicit notion in motor control although it has been implied indirectly. This notion is however of considerable interest in robotics for redundant manipulators (Lewis & Maciejewski 1997). In motor control, Cruse & al. look for a pseudoinverse solution at the kinematic level and could not account for experimental data (Cruse et al. 1993). A pseudoinverse solution minimizes the quadratic joint velocity function and implies no selfmotion. By definition and as for motor equivalence, selfmotion can only occur in redundant systems. 1.6 Goal Our goal is to account for a UCM motor control strategy and joint coordination for a redundant effector. A model of pointing movement is developed within a dynamical systems framework and an EPH muscle framework. The strategy adopted for this model is an organization of movements at a behavioral level (sec. 1.1). A special focus is on the UCM motor control strategy. This strategy is to be validated with the appropriate UCM signature between model simulations and experimental data. The role of this motor control strategy in improving task performance is examined according to the hypotheses formulated in section In the same way and also with experimental validations to the extent that data are available, selfmotion and motor equivalence are examined. The role and causes for the UCM signature, selfmotion and motor equivalence are investigated in the model.

44 44 Introduction

45 METHODS This section presents a model of discrete movement in humans and the related model framework. It introduces furthermore the theories and experiments that constrain the model organization. Our goal is a model that accounts for movements and movement features within a framework constrained by experimental knowledge. In order to evaluate the model with respect to real human movements, an appropriate model of movement must be selected that can be set in a comprehensive explicit model framework and at the same time that can be tested experimentally. A pointing task with a 4-joint effector is chosen (fig. 1.1). The joints correspond respectively to the wrist, the elbow, the shoulder and the sternum. The effector motion is restricted to a 2D horizontal plane. The joints are considered independent and their unique rotation axes are perpendicular to the plan of motion. Each joint is one DOF. Thus, the task is 2 DOFs redundant (sec. 1.1). 2.1 Trajectory This section describes some concepts on the trajectory specification and the trajectory representation that underline movement execution. This guide sets in a context the model framework. It would be overly ambitious to provide an exhaustive description of the experiments that deal with trajectory specification and trajectory representation, so only relevant features are presented. This section takes up and expands a number of issues already raised in the Introduction (sec. 1.3 and sec. 1.4). It is thus addressed specifically to the reader interested in the nature of the trajectory while the Introduction provides the reader with the basic and sufficient assumptions made in the model. The nature of a trajectory must be clarified in order to specify a trajectory. A set of joint positions defines a body configuration. During a movement, the body adopts successive instantaneous configurations. The set of all these configurations forms a path in the joint space that describes all the transient positions adopted by the body during the movement. A path in the joint space is not sufficient to reproduce unequivocally a movement. The path must be given a temporal dimension. The rate of change in time of the angles of the joints must be specified. A path and its temporal order form a trajectory. 45

46 46 Methods Each movement is characterized by a trajectory. The neuronal structure that initiates and generates a movement is distinct from the effector that executes a movement. The observed trajectory emerges from the interactions between this movement generator and the effector. Thus, two essential interacting systems are required to produce a movement. The movement generator and the effector properties determine the trajectory features of the effector. The movement generator by definition represents the neuronal activity linked to movement generation Trajectory specification Trajectory features A trajectory specification is required to initiate a movement. A minimum requirement consists of specifying the final effector configuration at the end of the movement. Theoretically, it is sufficient to determine the starting configuration and the terminal configuration to generate a movement. Each joint would have to move from its current position to its terminal position driven by a servocontroller system or an elastic potential field. An elastic potential field forces the body into a movement state so long an equilibrium position for the effector is not reached (sec. 1.3). Without any joint coordination, the various joints move independently from each other from their respective starting positions to their respective final positions. The trajectory of each joint is determined by the gradient of the potential field and is easy to predict. The trajectory of any coordinate of the body will exhibit a complex time course that depends on the body configuration. These trajectories are complex because each joint moves independently from each other. This is clearly in contradiction with the invariant features of the hand trajectory observed in experiments. The path of the hand in discrete movements typically forms a slightly curved line segment whatever the position in the workspace and even when the movement speed is varied (Morasso 1981; Sergio & Scott 1998; Boessenkool et al. 1998; Adamovich et al. 1999; Flash 1987; Won & Hogan 1995). The hand path is more invariant in the workspace that would be expected if the effector DOFs were not coordinated (sec. 1.1). Moreover, joint excursions vary for various movement speeds while the hand path is roughly invariant (Thomas et al. 23). Uncoordinated configuration to configuration movements will not produce the invariant features observed for the end-effector. Joint trajectories must be specified that preserves the joint coordination. So accounting for body configurations alone in trajectory specification is not appropriate because a joint coordination implies to account for the trajectory of some body coordinates, i.e a trajectory in the task space. As noted in the introduction (sec. 1.1), joint coordination implies a time structure of joint motions according to the movement task.

47 2.1. TRAJECTORY 47 The coordination of the joints can be accounted for in various ways in the trajectory specification. Motivated by the bursts-like electromyograms, it has been proposed that movements are ballistic in nature (Gottlieb 1993; Agarwal et al. 1993; Barto et al. 1999). A sudden single shift of the appropriate motor signal brings forth the movement and the dynamic property of the limb permits to complete the intended movement smoothly. An appropriate timing between the muscle activity bursts can in principle generate coordinated movements. One experiment in particular casts doubts on this ballistic strategy. Bizzi & al. train monkeys to perform pointing movements (Bizzi et al. 1982; Bizzi et al. 1984). Some of these monkeys are unable to see their arms and have their arms surgically deafferented while other monkeys cannot see their arms but do not have their arms deafferented. Before movement onset, the monkey s hand is moved to the target position by the experimenter and released at various moments in time after movement onset. Instead of their arm remaining at the target as would be expected if movements are driven by the arm dynamics alone, the monkey s arm proceeds first toward the starting position and then reverses its direction toward the target. This experiment demonstrates the continuous nature of internal state changes during the movement (sec. 1.3). Thus, continuity characterizes the trajectory specification. The changes of the muscle states induced by the CNS during movement can be parametrized with a virtual trajectory (Hogan 1984; Feldman 1965) (sec. 1.3). The virtual trajectory set the muscle states whose changes cause the movement of the effector. As an extension from muscles to joints, the virtual trajectory can be parametrized in the joint space according to the Equilibrium Point Hypothesis (EPH) theory (Feldman 1965). Theoretical works within this EPH framework also support a graded virtual trajectory (Hatsopoulos 1994). Hatsopoulos has modeled a non-redundant arm with nonlinear overdamped spring-like synergetic muscles (Hatsopoulos 1994). He demonstrated that the model whose virtual trajectory is graded, performed better than the model whose virtual trajectory is step-like but the muscle response is graded. Hogan concluded similarly that smooth end-effector paths cannot be accounted for by the sluggish behavior of muscles (Hogan 1984). These results are in agreement with experiments analyzed within the EPH theory (Ghafouri & Feldman 21; Feldman & Levin 1995; Latash 1993; Ostry & Feldman 23). Although the virtual trajectory is graded, it reaches its terminal state earlier than the real arm (Ghafouri & Feldman 21). The lag of the real arm is not surprising because the virtual trajectory induces the movement of the effector. Within the EPH framework, the virtual trajectory is tracked by the real effector. The muscle properties correct for errors between the real arm configuration and the virtual configuration. The modality of these corrections depends on the muscle properties (sec. 1.3).

48 48 Methods The meaning of the trajectory specification differs according to the motor control strategy assumed. The EPH theory assumes a virtual trajectory that set the muscle states (sec. 1.3). Control-based approaches to motor control privilege fixed predetermined planned trajectories, implicitly assuming that movement planning and movement execution are separated. Trajectories are completely determined beforehand and executed the best possible way by acting on actuators (Flash & Hogan 1985; Uno et al. 1989; Otha et al. 23; Bullock & Grossberg 1988; Bullock et al. 1993; Loeb et al. 1999; Bhushan & Shadmehr 1999; Tee et al. 24). The planned trajectory is by consequence isomorphic to the real trajectory and the variability of the real trajectory reflects the imperfection of the controller. This approach is inspired by robotics in which trajectories are servocontrolled at the joint level. The execution process incorporates at various degrees error measurements. This is the sensorimotor approach. Ideas based on predictive control theory including Kalman filter and forward internal models have revived close loop system approach in movement execution (Grewal & Andrews 1993; Stengel 1994; Desmurget & Grafton 2; Mehta & Schaal 21; Wolpert & Kawato 1998; Jordan & Rumelhart 1992; Bhushan & Shadmehr 1999; Scott 24). These approaches stress precise execution based on internal models or statistical state estimation processes. Deviations between the planned and estimated trajectory are translated into error signals and corrected for. Without a task constrained by specific forces, the form of the planned trajectory differs from the form of the virtual trajectory when the virtual trajectory is not tracked perfectly by the muscles (sec. 1.3). Similar trajectories imply a high enough muscle stiffness in the EPH framework. The trajectory specification in our model is based on a continuous virtual trajectory. A virtual trajectory for the trajectory specification is motivated based on a few reasonable assumptions on the muscle property (Hogan 1984). This virtual trajectory, by definition, induces the movement and thus must account for joint coordination. Generating movements with a virtual trajectory implies an Equilibrium Point Hypothesis framework. It means that the gradient of an elastic potential field generated by the muscles is the force that set the joints in motion. This is only a description and it is not meant that the virtual trajectory must exist in this form (Hogan 1984) (sec. 1.3). The virtual trajectory is only one way to interpret the concept of a movement generator and muscle activity. The virtual trajectory may be isomorphic to the real trajectory but does not have to. Moreover, it is hypothesized that the virtual trajectory is compliant to the real trajectory in a way that will be specified later (sec ). This virtual trajectory is thus not equivalent to a planned trajectory because it does not have to be isomorphic to the executed trajectory and it can be modified during the movement. In agreement with recent experimental results, the joint stiffness must be low enough to account for the compliance of the virtual trajectory (Gomi & Osu 1998; Tsuji et al. 1995). This low

49 2.1. TRAJECTORY 49 stiffness implies that the virtual trajectory is not expected to be isomorphic to the real trajectory because the real arm cannot track the virtual trajectory closely. Movement time The temporal aspect of the trajectory is constrained by the coordination requirement but must also account for various movement speeds. Humans may intentionally vary movement time or can even synchronize movement with an external metronome. This leads to the assumption that movement time is not an explicit parameter but emerges from multiple processes including cognitive processes as described, for instance, by the concept of urgency (Gottlieb 1993). Movement time may also be determined by a trade-off between movement effort and the mechanics of the limb (resonance) (Rosenbaum et al. 1991). Thus, movement time could emerge from the neuronal network dynamics itself (Glasius et al. 1996) (sec ). In this study, it has been suggested that movement time emerges from the interactions of a motor map with real effector feedbacks and a sensory map. How time is represented in connectionist neural networks is still a debated issue. It is most often assumed that time is implicit rather than explicit. In addition, a link between movement time and task performance has been shown to exist in certain conditions. The trade-off between movement time and target size is expressed in the empirical Fitt s law that reflects the internal planning processes (Latash 1993). Its validity is challenged by a number of experiments that lead to postulate different versions of the law. In 3D movement, a modified Fitt s law is proposed to explain experimental data (Murata & Iwase 21). The selection of movement time is not addressed explicitly in this work and thus movement time is varied as an algorithmic variable Trajectory representation The practical implementation of a trajectory specification is linked to the issue of the trajectory representation. An attempt is made to define in a few sentences what is meant in this context by a trajectory representation. Representation refers to neural activity that is not the mere consequence of sensory information or motor outputs. Representation-related activity is built in a network shaped by sensory and motor activities. This activity can be changed as the result of intrapersonal processes alone and can be used as a reference along which other processes can be measured or compared to at a time scale larger than the time scale of the sensory dynamics. The representation persists (see also for representation in relationship to content, concept and motor action Gallese 23). The fact that movements are not simply a reaction to stimuli but are also structured in a continuous flow of sensory information suggests a representation for movements. This representation is also supported by force

50 5 Methods field experiments and the theoretical concepts of the body schema and the body image. In addition, works on sensorimotor calibration also points to a internal representation of the body (Lackner & Dizio 2). The body schema is an unconscious representation of the body which is used to explain the behavior of patients with sensorimotor deficits (Paillard 1999; Holmes & Spence 24; Gallagher & Cole 1995). The body schema is thought to be shaped by perception and action. This internal representation of the body forms a framework to which actions and perception are referenced. In a simplified view, the body schema is postulated to be a map of body postures in the brain. This map can also be extended to a framework to execute movements (Morasso & Sanguineti 1995). A force field is the gradient of an elastic potential field that characterizes the limb elastic property (sec. 1.3). The muscle generated force field accounts for the fast resistive responses of the limb to perturbations. Along the hand path during a movement, the hand resists mechanical perturbations that drive the hand away from its unperturbed path (Won & Hogan 1995). The inward forces that push the hand toward the unperturbed path form a resistive force field. The force field entity of the hand is the consequence of internal brain processes that set the effector muscle states. The consistency of the force field equilibrium point during movement and in the workspace implies that the hand force field must be reset according to the actual hand position (Ostry & Feldman 23; Bizzi & Mussa-Ivaldi 1995; Tsuji et al. 1995; Gomi & Osu 1998). Otherwise, the hand would be forced back to a fixed equilibrium position. This resetting process is certainly not adaptive but rather anticipative as claimed by the EPH theory (Feldman & Levin 1995; Hogan 1984). Indeed, the EPH theory postulates movement generation based on force field dynamics. Because the changes of the force field structure induce the movement, they must, by definition, occur in advance of the real movement. The coordinated collective activity of neurons, the internal variable virtual trajectory, generate muscle state changes to induce a desired movement and also to reset the limb position. Force field resettings have been measured in humans, monkeys and frogs (Won & Hogan 1995; Mussa-Ivaldi et al. 1985; Bizzi & Mussa-Ivaldi 1995; Bizzi et al. 1982) (sec. 1.3). The body schema and the force field of the end-effector indirectly support the existence of an internal body representation for movement generation but neither theory tells about the exact form of the representation and much less on the coordinate system of the representation. This topic is strongly debated. Multiple experimental data point to various representation models that a priori do not converge toward a common framework. Electrophysiological studies strive to find a brain representation of movement

51 2.1. TRAJECTORY 51 parameters (Christopher decharms & Zador 21). The basic idea is to find covariance between spike discharge rates and movement parameters measured in the extrapersonal space. This methodology makes implicit assumptions for movement generation that can differ, for instance, from the concept of internal variables (Feldman & Levin 1995). A brief overview of some electrophysiological studies sheds light on the multiplicity of the results. Three tendencies emerge : muscle-related or force related parameters, kinematics parameters and behavioral parameters. In the somatosensory cortex, the homunculus map, a dynamic cortical map of the body, is the best known example of neuronal correlates. The failure to find an equivalent map in the primary motor cortex and the premotor cortex has led to the belief that other complex features may be represented such as muscle tension or muscle groups (Kakei et al. 1999; Graziano et al. 22). Some experiments point to such a representation. Correlation between cortical activity and muscle EMGs are demonstrated (Feige et al. 2). A force representation is in the heart of the motor primitives theory. Stimulations of the frog spinal cord and higher centres have revealed a set of low-dimensional force fields or motor primitives characterized by a unique equilibrium point (D Avella & Bizzi 1998; Bizzi & Mussa-Ivaldi 1995). The authors postulate that combinations of these primitives generate force fields whose equilibrium position varies in the workspace. Generating a trajectory with time-dependent motor primitive combinations is straightforward providing some requirements to ensure the stability of the combinations (Slotine & Lohmiller 21). The motor primitives hypothesis states about a basic framework for movement generation made up of a basic force field alphabet. The relatively invariant features of the hand mean trajectory within the workspace in pointing movement are considered evidence for a task related planning reference frame (Morasso 1981; Boessenkool et al. 1998; Sergio & Scott 1998). A straight line in an extrapersonal reference frame corresponds to a curved path in the joint space and vice versa. Because the hand path is straight and invariant in the workspace while the joint path is curved and more variant within the workspace, it is postulated that movement planning is achieved in the task space. The planned movement is ultimately transformed in the effector space for execution. This assumption proposes a simple planning process for any pointing movement in the workspace, compatible with the integration of visual information in the planning process. In reality, the mean end-effector paths are slightly curved and these curvatures vary with the workspace position. Studies on prism adaptation or in virtual environment in which the hand path curvature is adaptively reduced after the perceived hand path curvature is experimentally increased in pointing movement support the idea of a kinematic planning of movement in the task space (Wolpert et al. 1995; Flangan & Rao 1995). These studies point to the fact that the hand path itself is of matter in movement execution but can-

52 52 Methods not exclude on that basis a joint space representation. One of these studies demonstrates furthermore that the hand path can be made straighter than during the normal conditions (Wolpert et al. 1995). The straight path of the hand is only a descriptive approximation of the real hand path. These results highlight flaws in the argumentation on invariant kinematic features to motivate a simple planning process. Although counterintuitive, effector space accounts have also been postulated (Torres & Zipser 22). In order for a relatively straight end-effector path and an effector space representation to coexist, a new metric for the effector space can be defined. This proposal states that motor representation is about defining metrics in the effector space that preserves the metrics in the task space, i.e a task-related representation. In other words, the effector space metric accounts for the coordination of the effector DOFs and allows to dissociate geometry and time. Thus, movement invariant features cannot account unambiguously for a particular coordinate system for movement planning. Electrophysiological recordings in the brain aim to discover single neuron correlates with kinematic movement parameters. For instance, neuron firing rate covaries with movement direction and instantaneous finger speed in the motor cortex of the monkey (Moran & Schwartz 1999). A study of brain activity demonstrates correlated cortical activity with the velocity of the effector (Kelso et al. 1998). This correlated activity furthermore reflects the spontaneous transitions of the effector between a syncopation mode to a synchronization mode. More studies show covariance between single neuron activity and kinematic or dynamic movement parameters like force, position, distance, target location, arm configuration, speed and direction. The synchronization of neurons or ensemble of neurons has been further shown to correlate with movement direction (Hatsopoulos et al. 1998). These covariance features are not strictly speaking evidence for a task space coordinate frame nor for an end-effector movement representation. The multiplicity of results suggests on the contrary that the representation of movement generation may be more complex and abstract than what a simple movement or force parameters space imply. Recent studies support a complex high level feature representation. A new stimulation protocol in the motor cortex has led to the entirely new concept of behavioral postural map for the primary motor cortex and the premotor area of the cortex (Graziano et al. 22). To each point in the cortex corresponds a behavioral relevant posture of the hand or of the arm which is independent of the starting configuration of the limb and of the starting position of the hand. The novelty in the method is the increased stimulation time. This finding is supported by the discovery of the canonical and especially the mirror neurons. A mirror neuron activity is correlated to various behavioral forms of the same task, for instance when a movement is executed by the monkey and when the monkey observed the same movement being executed by the

53 2.1. TRAJECTORY 53 experimenter or when the monkey executes an action and hears the sound of that action (Kohler et al. 22; Gallese 21). These findings are complemented by a study using electroencephalograms that shows motor activity that anticipates the action of an observed person (Kilner et al. 24). This study suggests a motor representation to interpret an observed action. The peculiarity of the mirror neurons is the apparent immunity of the self in one case and the modality perception in the second case. The authors conclude to an action or task representation independent of the context of the action. The mirror neuron characteristics have led to a number of new theoretical ideas that differ conceptually from the classical cortical maps. These characteristics suggest primarily a stronger connection between the usually considered separated field of action and perception. A straightforward action-perception bond can be postulated with a motor representation in the sensory space (Loeb et al. 1999). This approach however does not account for the task-oriented organization of the representation. The Theory of Event Coding (TEC) accounts for a task oriented link between action and perception (Hommel et al. 21). This theory postulates that information in the brain is coded as events whether the information pertains to a percept or a planned action. TEC is a common coding framework for action and perception which bears some similarity to the old body schema concept (Holmes & Spence 24; Paillard 1999; Gallagher & Cole 1995) (see above). Another theory inspired from the mirror neurons points to the non-symbolic representational content of real and situated biological agents emphasizing the interactions between the organism and its environment (Gallese 23). The common features of these proposals in addition to the mirror neurons motivation is the importance attributed to perception and action or in general behavior to build and shape the representation. The representation depends on the organism acting in its environment. A theoretical neural network study demonstrates the feasibility of a common abstract integrative representation for sensorimotor processes (Deneve et al. 21). A basis function neural network with noisy neurons and multidimensional attractors integrates various sensory modalities and motor functions into a basis function space without the need for explicit coordinate transformations (see for instance (Bullock & Grossberg 1988; Bullock et al. 1993)). These results may furthermore account for the idea of abstract, in the sense that they are not directly related to an external observable parameter, internal variables as proposed in the EPH theory (Feldman & Levin 1995). These views challenge more analytical approach to motor control in favour of a more integrative approach. These results also suggest that the representation of movement generation is part of a broader abstract framework. This framework is characterized by a task-oriented organization and is linked to motor action and perception. We have seen the constraints imposed on the trajectory specification. The

54 54 Methods trajectory specification is continuous in time, accounts for coordination and generates the movement. For the trajectory representation, we will adopt an integrative approach that in view of the latest experimental results is more appealing. Because this integrative framework is, by nature, complex, a pragmatic approach is required. Our model is restricted to a single effector that executes pointing movements, so the movement generator will be also restricted to this simple behavior for a single effector. Moreover, no explicit accounts of vision and most other sensory modalities are included in our model. So the movement generator does not have to account for these modalities too. This decoupling of perception from action is motivated by the fact that perceptually deprived subjects and patients can still execute movements. So perception is not a necessary condition for the movement generator to generate movement providing that the movement generator already exists, i.e. no learning account. Coordination and the integrative framework property emphasize that the organization of the movement generator is task-oriented. This task oriented organization of the movement generator is central to our framework. A coordinate system that is compatible with the EPH framework is postulated for a virtual trajectory in the joint space (sec. 1.3). It is not possible to provide a complete biomechanical account of the arm. The number of muscles and especially their mechanical properties like, for instance, the arm moments are not completely known. The muscles are thus organized around the joints into neuromuscular joint ensembles that generate joint motions. A virtual joint trajectory assumes implicitly that the joints are independent DOFs of the effector. Indeed, human subjects can move independently each joint. The virtual trajectory is furthermore not assumed to be a measurable variable but an abstract representation of the outcome of the activity of the neurons that make up the movement generator. The equilibrium trajectory is an abstract parametrization that captures the complex interactions between the muscles and the CNS (Hogan 1984). The issue of a task coordinate system is then irrelevant so far the model functioning does not depend on the coordinate system. 2.2 Effector and end-effector space Forward and inverse kinematics This section presents the concept of effector/end-effector mapping for articulated redundant manipulators. The relationship between the joint configuration (θ) and the position of the end-effector in the task space (x) of an articulated manipulator, called the forward kinematics, is a vector map x = f(θ) (2.6)

55 2.2. EFFECTOR AND END-EFFECTOR SPACE 55 This is a geometric model of the manipulator. The differential transformation from configuration displacements to end-effector displacements is dx = J(θ) dθ (2.7) The matrix J is called the Jacobian of the manipulator. The manipulator Jacobian depends on the joint configuration. For instance, assuming a 4 joint manipulator constrained to move in a plane implies that the end-effector has 2 DOFs, x R 2, the joint space has 4 DOFs, θ R 4 (fig 1.1) and the manipulator Jacobian is J = x 1 x 1 x 1 x 1 θ 1 θ 2 θ 3 θ 4 x 2 x 2 x 2 x 2 θ 2 θ 2 θ 3 θ 4 (2.8) From the differential map, a velocity map can be derived. This map transforms the joint velocity vector into the velocity vector of the end-effector ẋ = J(θ) θ (2.9) The dots in this equation designate the first time derivative of the variable. In robotics and in motor control, the inverse velocity mapping is most often of interest. It describes the mapping from the velocity vector in the task space to the effector space velocity vector θ = G(θ) 1 ẋ (2.1) If the equation is consistent and determined, the inverse of the Jacobian exists and is unique G 1 J 1 θ = J(θ) 1 ẋ (2.11) In human motor control application, the system is typically underdetermined like our example of the 4-DOF arm. That means that the inverse mapping gives an infinite set of solutions. These inverse matrices (G 1 ) are called generalized inverses (Rao & Rao 1998). The problem in many applications is thus to select one appropriate generalized inverse. A specific solution is selected based on problem-specific constraints. A widely used constraint is a minimal joint displacement ( min θt θ ). This system of equations can then be solved exactly, for instance with the method of the Lagrangian multipliers, and leads to the well known Moore-Penrose pseudoinverse J + (Penrose 1955) G 1 = J + = J T (J J T ) 1 (2.12) The Moore-Penrose pseudoinverse has been used in motor control models (Barreca & Guenther 21; Goodman & Gottlieb 1995; Guenther

56 56 Methods & Barreca 1997; Bullock & Grossberg 1988; Bullock et al. 1993; Morasso & Sanguineti 1995) and in robotics (Lewis & Maciejewski 1997; Zlajpah 1998; Murray et al. 1994). A slightly different solution, a weighted pseudoinverse, minimizes θ T A θ with A a positive definite matrix (Whitney 1969). Various contributions to the total cost are attributed to the joints. A particular case is when the cost for some joints is infinite. The original system can then be reduced to non redundant subsystems. The inverse mapping of one of these subsystems is then uniquely determined and the submatrix of the original manipulator Jacobian can be inverted. The remaining undefined vector components in the effector space corresponding to that submatrix solution are set to zero or set to fullfil a constraint (Chen & Walker 1993). This solution has been proposed in motor control in which case the submatrices form a basis for the mapping (Feldman & Levin 1995). Instead of selecting a matrix among the generalized inverses, the task space can be augmented to form a non-redundant system. A constraint is written to complement the task space and to form an augmented system whose matrix is then full rank (Benallegue et al. 23). The problem of that solution is that the task velocity is not invariant because the augmented null space is not spanned by a basis. A related strategy consists of increasing the Jacobian dimension. The augmented Jacobian solution makes the Jacobian nullity zero by increasing the Jacobian dimension with the transpose of vectors that span the null space of the Jacobian (E) (Murray et al. 1994; Chen & Walker 1993; Kreutz-Delgado et al. 1992) J e = ( J E T ) (2.13) E T is the transpose of the null space vectors. If the vectors that span the null space form a basis, the inverse solution is straightforward (Murray et al. 1994) J 1 e = ( J + E ) (2.14) J + is the Moore-Penrose pseudoinverse of the Jacobian. The fundamental feature is that the mapping is expressed in the velocity space or for end-effector displacements. This mapping feature constrains the specification of the virtual trajectory. This constraint has been also recognized in motor control research. A model of motor control proposes to plan movement in the task space with direction vectors from the actual end-effector position to the target position (Bullock & Grossberg 1988; Bullock et al. 1993; Guenther & Barreca 1997). Direction vectors are then mapped on the joint space. In our framework, a velocity mapping from the task space to the effector space is used (sec ).

57 2.2. EFFECTOR AND END-EFFECTOR SPACE Null space The null space of a matrix is the set of all vectors in the high dimension space that maps onto a zero vector in the low dimension space. These vectors define a subspace which can be spanned by one linearly independent vector set E forming a basis for that null space J E = (2.15) For the differential mapping, the null space corresponds locally to a subspace in the joint space. For the velocity mapping, the null space corresponds to a subspace in the joint velocity space. Any motion restricted to the null space of the manipulator Jacobian is called a selfmotion or internal motion because it leads to no displacement of the end-effector. Motion within the null space can be set to zero or can fullfil a constraint. A null space constraint takes the form of a negative gradient of a function to be minimized, for instance a function that measures the distance of the joint position and an absolute joint range limit (Guenther & Barreca 1997). The constraint is a secondary task which is realized within the limit allowed by the null space. With an augmented Jacobian system, a constraint in the null space can also be imposed Dynamical systems and the virtual task trajectory In the previous sections, two properties of discrete movements are emphasized. These movements are task-oriented and the effector DOFs are coordinated. Coordination implies to define for what objectives the movement is coordinated. The answer is that the movement is coordinated to preserve the task. So, the task must be specified explicitly to permit coordination. A movement includes a task, an effector and in most cases an end-effector. The effector is the body because the body interacts with the environment. Most movement involves the complete body but not every body part has to move during a movement. So ideally the movement generator should account for the full body. In our model, the body however is restricted to an illustrative example of a 4-DOF arm. The end-effector is directly related to the movement objectives. In pointing movement, the movement objective is to use the finger to point. The finger is thus the end-effector. The end-effector is the mean by which the task is fulfilled. In our model, the end-effector is the hand tip (the fingers are not accounted for). The task dictates the behavior of the end-effector. In our model, the task thus determines the movement of the hand. This movement is described with two parameters, space and time. In a breach of language and because the task trajectory comes from the movement generator, the task trajectory is also called a virtual task trajectory.

58 58 Methods The space component of the end-effector behavior consists of a path in the task space. The path is determined by the movement objective. In our pointing task example, any path that leads to the target position is potentially a path that fullfils the task. One path that leads to the successfull achievement of the movement objective specifies the space component of the task. A path specification for the end-effector fullfils the requirement of continuity discussed in section The second component of the end-effector behavior is time. Many but not all movements are timed. Movements that are timed are speaking, chewing, dancing or rhythmic movements. Movements that are not timed are locomotory displacements. Timed movements are movements whose temporal form is reproducible and stable in the face of perturbations (Schöner 23). The temporal order of discrete movements is reproducible as attested by the invariant velocity profile of the end-effector (Morasso 1981). Thus, pointing movements are timed. So, in addition to the path, a time order specifies the behavior of the end-effector. The behavior of the end-effector can be characterized by a trajectory. The organization of movement around the task implies that the end-effector trajectory must be specified. This task structure must moreover be stable in the sense of dynamical systems theory to guarantee the success of the task (sec ). Indeed, if any perturbation leads to a failure to accomplish the task, a task organization does not make sense. This stability is especially required because the neuronal networks from which the task virtual trajectory emerges fluctuates continuously (Arieli et al. 1996). Thus, the task virtual trajectory must be stabilized in order to achieve successfully the behavior. Neuronal activity induces movements. We describe this process abstractly with the movement generator (sec. 2.1). Neural networks can thus be naturally proposed to account for the movement generator (Morasso & Sanguineti 1995; Glasius et al. 1996; Glasius et al. 1995). Neural networks can form a map of effector configurations and have the attractor property to guarantee stability. However, as already noted, timing within neural networks is poorly characterized (sec ). Limit cycle oscillators offer an alternative framework that fullfils the requirements for a movement specification in the task space. Limit cycles are selfsustained oscillators and are stable against perturbations that drive the oscillator state away from the limit cycle. Limit cycles have a temporal order with a regular phase variation in time, i.e the oscillator frequency. The intrinsic dynamic of the oscillator sets the temporal order of discrete movements (Schöner 199). The space dimension of the movement trajectory is the oscillator variable state. Limit cycles are moreover a biologically relevant framework. Limb and interlimb rhythmic movements show the property of limit cycles (Kelso 1984; Kelso et al. 1981; Schöner

59 2.2. EFFECTOR AND END-EFFECTOR SPACE 59 & Kelso 1988). Limit cycles have already been proposed to account for discrete and rhythmic movements with success (Schöner 199). Finally, limit cycle properties have been shown for neural networks motivated by physiological considerations (Amari 1977). A threshold function appears repeatedly in the next equations. This threshold is the sigmoidal function. The sigmoidal function is parametrized in the following way f(x, x, a) = (1 + exp a (x x ) ) 1 (2.16) x specifies a threshold value and a is a gain. The sigmoidal function range lies between when the variable is much smaller than the threshold and 1 then the variable is much bigger than the threshold. The gain determines the switching range. At very high gain, the sigmoidal function can be approximated with a step function. The Amari oscillator is a nonlinear limit cycle inspired by the neuronal structure and properties of the brain (Amari 1977; Schöner 23). The equations of the Amari oscillator are τ u = u + h u + w uu f(u) w uz f(z) τ ż = z + h z + w zu f(u) (2.17) u the activation variable of the excitatory layer z the activation variable of the inhibitory layer h x is the resting level w xx are the mutual interactions between the inhibitory layer and the excitatory layer as well as the excitatory interactions within the excitatory layer f is a sigmoidal function (see above) τ is the relaxation time The first two terms of each equation describe two linear dynamical systems with a single stable fixed point. The coupling terms make the system of equations nonlinear. The inhibitory and the excitatory variables are then coupled. The coupling generates a limit cycle behavior of the system of equations for an appropriate choice of parameters. The inhibitory variable does not have any meaning in terms of the task specification but comes from the constraints set by the limit cycle framework. The main inconvenience of the Amari oscillator as a model for a task space trajectory is the difficulty to control parametrically

60 6 Methods the limit cycle time. For practical reason, a nonlinear Hopf oscillator is finally preferred (Drazin 1994; Glendinning 1994) with ( u ż ) ( ) α ω = ω α ( u z ) γ (u 2 + z 2) u the activation variable of the excitatory variable z the activation variable of the inhibitory variable ω specifies the movement time α specifies the stability of the limit cycle ( u z γ defines the oscillator amplitude (in combination with α) ) (2.18) In our 4-DOF effector model, the end-effector space or the task space has 2 DOFs. This implies that two limit cycles or at least two dynamical systems are needed. When one limit cycle and a linear dynamic are selected, the limit cycle describes the movement along the line segment between the actual hand position and the target position while the linear dynamics describes the task perpendicular to the movement direction. This approach presents the disadvantage that any target position change implies a change of the coordinate system for the task description. Therefore, two limit cycles are preferred. A target position change implies to parametrize the oscillator amplitudes without changing the coordinate system. The limit cycle amplitudes are parametrized independently. Thus, an appropriate parametrization allows to generate movements of any distance in any direction. The limit cycles frequency could also be parametrized independently. However, for simplicity an identical frequency is chosen for both oscillators. This decision implies that the movement generator produces virtual paths in the task space that are line segments. Two oscillators are a minimum requirement imposed by the task but more oscillators could also be selected. For our purpose however, the model does not need to contain more oscillators. The oscillators are not mutually coupled. In order to account for successive movements, a full limit cycle must describe one discrete movement. So, a movement is terminated the limit cycle state is exactly the same as at the movement beginning and the next movement can be executed. The alternative would be that a full movement corresponds to a half limit cycle of the oscillator. In that case, the limit cycle state at the end of the movement would not be equivalent to the limit cycle state at the beginning of the movement (Schöner 199). The full cycle hypothesis expresses more generally the idea that after a discrete movement the organism is in a state equivalent to the pre-movement state. There is indeed no tendency to return

61 2.3. ELEMENTS OF MOTOR CONTROL 61 to the pre-movement state. The idea of postural resetting within the EPH theory account for the same principle, in agreement with the von Holst principle of postural resetting (Ostry & Feldman 23; Feldman & Levin 1995) (sec. 1.3). The body adopts a new configuration that is, at the level of the internal state, equivalent to the pre-movement state. In our framework that means that the movement generator must account for the new body posture. The task variable is properly specified by two limit cycles. This is a requirement for our movement generator to be task-oriented but the virtual trajectory that effectively drives the muscles at the joint level still need to be specified in the effector space. We have seen that the mapping between the joint space and the end-effector space is appropriately described in the corresponding velocity spaces. Thus, the task description must also be achieved in the velocity spaces (sec ). The excitatory variables of the oscillators describe thus the task variable velocity rather than the task variable path. Because in discrete movements, by definition the movement always unfolds forward from the starting position to the target position, the velocity must always be positive. Thus, the oscillator variables are offset by the limit cycle amplitudes to describe the task velocity trajectory properly. The virtual task trajectory that we have defined does not assume any direct neuronal correlates and does not comply to all definitions of a virtual trajectory. This task virtual trajectory expresses the concept of a neuronal trajectory that specifies the movement in the task space. At rest, the postand pre-movement states of the movement generator account for posture. Indeed, the limb resists at rest perturbations as well (Mussa-Ivaldi et al. 1985; Hogan 1985b; Shadmehr et al. 1993). Thus, the body configuration during the resting phase must be accounted for in the model. This account is however minimal because this is not the focus of our work. We will be satisfied with a marginal stability at rest. For clarity purpose, an arbitrary Cartesian representation for the task is adopted with the origin of the coordinate system located at the first joint. Another coordinate system could have been chosen for the task space as well. 2.3 Elements of motor control We have defined the quality and property of the virtual task trajectory. The task-oriented organization of the movement generator is particularly emphasized with a stable virtual task trajectory. A limit cycle framework for the task representation is motivated by the task features (sec and sec ). Moreover, the mapping constraints from the task space to the effector space are accounted for with a virtual task velocity specification (sec ). An

62 62 Methods account of this mapping must still be provided. Within our framework, the temporal order of movements comes from the intrinsic dynamics of the oscillators. It also implies that at rest the oscillators must be inactive. An account for posture must then be provided. We will assume the system has two distinctive phases, a movement phase during which the movement is executed and a resting phase during which the body is motionless. A movement comes from an intention to move. This intention changes the resting state of the organism into a movement state. This intention can be expressed with the concept of behavioral information (Schöner 199). Behavorial information is internal or external inputs to the system that determine the relevant dynamic pattern in the actual context. For instance, the subject is asked to move at a go signal. This go signal switches the internal organism state from the resting phase to the movement phase. The go signal is a behavioral information Movement initiation and termination Movement initiation switches the organism state from a resting phase to a movement phase. Movement termination switches the organism state from a movement phase to a resting phase. Movement initiation is based on the dynamical concept of behavioral information (Schöner & Kelso 1988; Schöner 199). Movement termination is based on the internal state of the organism. The movement generator unfolds a virtual movement. When the unfolding comes to an end, the organism state switches to a resting phase again. This is in agreement with the fact that deafferented patients or deafferented monkeys can stop moving, even without vision (Bizzi et al. 1982). Perception is thus not essential to stop a movement. Because both resting and movement phases cannot coexist, they must inhibit each other. This inhibition must on the other hand be overcome when a phase change occurs. We propose to account for this inhibition with a competitive neuronal dynamics. A neuronal population is active during the movement phase and another population is active during the resting phase. These neuronal populations inhibit each other. A state change can occur only when either a neuronal behavioral information is activated or when the virtual movement comes to an end. These neuronal activities foster the activity of the appropriate neuronal population of the competitive dynamics and initiate a switch of the competitive dynamics. In our approach, complex cognitive and perceptual processes are reduced to an overly simplified dynamical systems (for a detailed account of movement preparation (Erlhagen & Schöner 22)). This schema provides the main advantage of being explicit. This neuronal competitive dynamics is based on equations proposed by Amari (Amari 1977)

63 2.3. ELEMENTS OF MOTOR CONTROL 63 with s r = β r ( s r + h δ f(s m, s m, a) + a I r ) s m = β m ( s m + h δ f(s r, s r, a) + I m ) (2.19) s m is the neural population that represents a movement internal state s r is the neural population that represents a resting internal state f is a sigmoidal function with parameters (x,,1) h is a resting level δ is a constant I m is an input that represents the intention to move; this a transient behavioral variable I r is an input that represents the end of the movement based on the virtual velocities a is a constant The parameter β s characterize the dynamical properties of the dynamical system. The first two terms in the bracket of both equations form simple linear dynamical systems. The sigmoids make the dynamics nonlinear and coupled. The sigmoids are at the essence of the competition between the two dynamical variables. I s are the inputs to the dynamical system. For the system to change state, the input of the nonactive state must be activated transiently, i.e the input takes a positive value. The nonactive state is then more competitive than the active state and inhibits the latter. When the system reaches a new stable state, the input can be nullified. An inactive state is modelled with a negative value. I r is activated when the virtual end-effector velocity is close enough to zero. I m is activated when an intention to move is formulated. The intention to move set I m to a positive value at some time instant during a simulation to initiate the movement. The end of the movement is set according to the rule if t < t start +.5 F I r = (2.2) f(2 exp (1 u+a 2),.9, 8) otherwise with u the virtual task velocity vector, t start the time of movement onset, F the oscillator frequency and A a vector of the square roots of the limit cycle amplitudes. Because I r is based on the virtual task velocity, the end of the

64 64 Methods movement must be disambiguated from the beginning. This is done with the upper part of equation 2.2. I r becomes active at the end of the movement. When the norm in the lower part of equation 2.2 is close to zero, the sigmoid becomes positive. When I r is positive the resting state becomes more active and inhibits the movement state (equ. 2.19). While this dynamic is relatively simple, a neuronal account of how the inputs are activated is not given. It should be stressed that this is not a trivial process when considered at the level of the organism embedded in its environment, especially in light of movement preparation (Erlhagen & Schöner 22) Posture and movement An account of movement unfolding is given within a framework of limit cycles and a competitive dynamical mechanism permits to switch from an internal resting state to an internal movement state. An account of posture during the resting state must still be provided. This account is in agreement with the definition of the movement generator in the task space. The resting state in the task space is accounted for with a linear dynamics with a unique attractor per dimension of the task space. The virtual dynamics for one dimension of the task space is then given ( u ż ) (( ) ( ) α ω u = S m γ (u ( )) 2 + z 2) u ω α z z ( ) u + A S r β o + ψ z o (2.21) The competitive dynamics variables activate their respective internal states and are normalized with S x = f(s x,, 1). The movement is initiated by a transient activation of I m which activates for approximately the duration of the movement time the movement state (S m = 1, equ. 2.18). During this period, the limit cycles define the virtual task trajectory. At the end of the movement, I r is activated and s m is inhibited. The limit cycle variables are stabilized at an initial state on the limit cycle with a linear dynamics, the second term in equation The limit cycle variable (u) stability set the end-effector stability of the virtual task space. β o is a positive constant that defines the dynamics of the oscillator variables at rest. ψ o is time-correlated noise (sec ) and A is the square root of the limit cycle amplitude (Drazin 1994; Glendinning 1994; Schöner & Santos 21).

65 2.3. ELEMENTS OF MOTOR CONTROL Movement specification and representation We have defined in the task space a virtual task trajectory. The movement generator must however induce joint motion in order to achieve the task. Indeed, the DOFs of the effector are the means by which the movement can be executed (sec. 1.1). A coordination of the joints is required in order to preserve the coherence of the task (sec. 1.1 and sec ). That means that a temporal order must be given to the joint path with respect to the task. A temporal order is set in the task space by the limit cycles. This temporal order in the task space must be preserved by the joint temporal order. In fact, the temporal order in the task space imposes a joint coordination. This section explains how the coordination of the joints comes from, given the virtual task trajectory. In section 2.2.1, the mapping between the effector space and the end-effector space is formulated by ẋ = J λ (2.22) J is the effector Jacobian for the task as defined previously. ẋ is a vector that describes the virtual task velocity. Each component of this vector corresponds to one dimension of the task space. λ is the virtual joint velocity in the joint space. The inverse mapping from the task space to the effector space defines the virtual joint velocity, λ. As for the virtual task trajectory, the virtual joint trajectory must be stabilized because the movement generator is embedded in a neuronal fluctuating environment lest the stability imposed on the task variable is useless. So we impose that the stability of the virtual task trajectory be preserved in the virtual effector space. This is required by our assumption of a task-oriented organization of the movement generator (sec ). We require a simple dynamics that stabilizes the virtual task velocity ẋ mo = (u + A) ẍ m = β v (J λ ẋ mo ) (2.23) with A the vector of the square root of the limit cycle amplitude components, u the task velocity vector in the limit cycle space and β v a positive constant. This equation (ẍ m ( λ)) means that the virtual velocity in the joint space (J λ) tracks the virtual task velocity (u) as defined in the oscillator space. In other words, the joint coordination is defined such that the virtual task velocity in the limit cycle space acts as a moving attractor. This dynamics stabilizes the task trajectory in the joint space as we desire. This tracking defines a virtual acceleration of the end-effector in the effector space. Because we want the virtual task trajectory to be tracked, the gain β v must be high. Moreover, we

66 66 Methods impose a similar stability during the resting phase for the end-effector. We call this tracking at rest a fine positioning of the end-effector. The fine positioning movement consists of slowly moving the end-effector by setting an equilibrium point solution at a coordinate in the task space. A simple linear dynamics fullfils the requirements for a fine positioning ẋ ro = S r β f (1 I r ) (p d p λ ) ẍ r = β v (J λ ẋ ro ) (2.24) β f is a positive constant, p d is the desired end-effector position in the arbitrary task coordinate system and p λ is the current virtual position of the end-effector computed from the arm geometric model (sec ). I r and S r have been defined previously. This fine positioning is only active at the beginning of the movement according to the experimental protocol. This experimental protocol imposes on the subject not to correct for errors at the end of the movement. So, at the end of the movement, there is no fine positioning with an equilibrium solution at the target position. This constraint is expressed by 1 I r. The stability constraint imposed on the task in the effector space leads us to write an acceleration variable for the task. The mapping between the task space and the effector space must be consequently adapted (equ. 2.22). The time derivative of the mapping accounts for acceleration ẍ = J λ + J λ (2.25) Because the arm model is redundant, a unique inverse solution does not exist. We propose to use an extended Jacobian to account for redundancy (sec ). The task Jacobian is augmented with a set of vectors that spans the null space of the task Jacobian (E). ṡ = E T λ (2.26) This mapping gives a selfmotion vector expressed in an arbitrary null space basis (ṡ) (sec ). That is any combination of virtual joint motion that does not lead to task variable changes. We must also transform this map in an acceleration space to be consistent with the task space s = ĖT λ + E T λ (2.27) Both acceleration mappings can be written in a single equation ( ) J E T λ = [ ẍ s ] [ J λ Ė T λ ] (2.28)

67 2.3. ELEMENTS OF MOTOR CONTROL 67 The matrix in front of the equation is full rank and can be inverted. This inversion allows to write the map from the task space to the effector space λ = ( J E T ) 1 ẍ J λ β s S r λ + ψ λ (2.29) s ĖT λ ψ λ is a vector of independent time-correlated noise (sec ). In addition, a damping term in the virtual joint space ( β s S r λ) completes the equation to guarantee stability at rest (explained below). So we obtain with this expression a mapping from the task space to the effector space that is task oriented because the task trajectory is stable against neuronal fluctuations. This equation expresses a task decoupling in the effector space in the sense of the task-oriented organization of this space. The task acceleration is from equ and equ ẍ = β v (J λ ẋ mo ẋ ro ) (2.3) β v is a positive constant. The dynamics of the selfmotion ( s) must still be defined. When s =, the null space acceleration essentially depends on the noise diffusion process. Instead, a damping is proposed to reduce the noise effect in the null space. We postulate that the null space contribution expresses the idea that the virtual joint trajectory is compliant to the real joint trajectory. In the null space, the virtual joint trajectory yields to deviations of the real joint trajectory from the virtual joint trajectory. It implies that the virtual joint trajectory is not specified completely before the movement starts, but rather that the virtual joint trajectory is updated while the movement unfolds. This postulate is expressed with s = β n1 E (λ θ d ) β n2 E ( λ θ d ) (2.31) β n1, β n2 are positive constants. θ is the vector of the effector joint angles and θ is the vector of the effector joint velocities (the index d stands for delay; see below). Equ is a coupling between the movement generator and the effector and is called the backcoupling. The negative signs in this equation expresses the yielding of the virtual trajectory. Equation 2.29 expresses with the extended space the idea that there is no unique perfect virtual trajectory but rather that the movement generator defines a set of task equivalent trajectories. With this selfmotion, the virtual joint trajectory in the null space does not necessarily come to rest quickly at the end of the movement. In order for the virtual trajectory to rest quickly, a damping term is written in the joint space (the last term in equ before the noise). β s is a positive constant. S r and λ are explained previously. This joint damping guarantees that the virtual joint trajectory rests quickly but because this is a velocity term, the position of the end-effector at rest is not stabilized. We will not consider in more details the

68 68 Methods issue of the end-effector position at the end of the movement. The Jacobian is computed analytically from the geometric model of the arm. The null space vectors are derived by applying the Graham-Schmidt theorem on the reduced form of the Jacobian. These vectors are then orthonormalized. The derivatives of the Jacobian and the null space are also computed analytically. Delays for the real joint positions and velocities (θ, θ) in the backcoupling are introduced to demonstrate that any conduction delays that come from the sensory system can be integrated in our framework. The real effector state is thus delayed in equ The sensory delays are not specific in the sense that the backcoupling is not attributed to a specific sensory modality but rather represents globally an information inflow from the world into the movement generator. The following second order equation is used for each dimension of the effector state τ 2 ẍ d + 2 τ ẋ d + x d = x (2.32) τ is the time constant and the index d stand for delays Joint muscle model We have defined a virtual trajectory in the joint space. This definition is in agreement with the EPH theory that extends the concept of a virtual trajectory from the muscles to the joints (sec. 1.3). The joints are set in motion by an ensemble of muscles. A detailed biomechanical model of the arm is beyond the objectives of this study. Such a model is very complex and current models of the biomechanics of the arm rest on many simplifications with respect to the number of muscles, the muscle properties and how muscle forces set articulations in motion (Winters & Woo 199; Houk et al. 22; Gribble et al. 1998). The complexity of this problem has been recognized earlier and efforts have been made to describe the action of muscles at the joint level in a joint muscle framework (Hogan 1984). We will adopt here the same strategy to figure out a model that accounts for the action of muscles at the joint level. The features of a joint muscle must be properly characterized within the EPH framework. A single independent joint can be described mathematically by I θ = T (θ, θ, {a}) (2.33) I is the inertia of the body segment set in motion by the joint, θ is the joint angle and T is the muscle-generated torque with {a} the set of muscle states

69 2.3. ELEMENTS OF MOTOR CONTROL 69 (Hogan 1984; Hogan 1985b). This very general joint description assumes that a joint is made up of a rotating mass with a fixed rotation axis. The general function T describes the mechanical properties of the muscles acting at the joint. At equilibrium and without external load, T and the joint velocity are null. Experimental evidence points to a mass-spring model for a joint (Kelso et al. 1981; Bizzi et al. 1984). A joint is thus equivalent to a spring. A transient perturbation that drives the joint away from its resting state is resisted and the joint regains its pre-perturbation state if the CNS internal state is not changed by the perturbation. Assuming that the dependencies of the joint muscle activity on θ, θ and a are uncoupled and linear, the right side of equation 2.33 can be written T ( θ, θ, {a}) = T ({a}) K θ B θ (2.34) with B the linear viscosity and K the linear stiffness of the joint muscle. The stiffness term expresses the dependence of the joint muscle property on the joint position and the viscosity term, a dependence on the joint velocity. Stiffness and viscosity are two features of the muscle impedance (Hogan 1985b; Hogan 199). A muscle impedance relates muscle forces to imposed muscle motion and thus includes all motion dependent effect of the muscles. The impedance defines here is the static impedance of the joint muscle, i.e the limb is perturbed at rest. The static impedance is quantitatively and probably also qualitatively different from the dynamic impedance. T ({a}) can be written equivalently at equilibrium T ({a}) = K θ θ ({a}) = T ({a})/k (2.35) The variable θ ({a}) is a virtual position for the joint. This variable represents the joint muscle state and is under direct control of the nervous system. Thus, equation 2.33 can be written again I θ + B θ = K (θ θ ) (2.36) This equation is a linear mass spring model for a joint (Hogan 1984). In order to set the joint in motion, the CNS modifies the joint muscle state. A change of the joint muscle state defines a new equilibrium position that the joint must adopt (without external torque). This description of a joint is a linear joint model equivalent to the λ theory description of a joint (Latash 1993) (sec. 1.3). In the EPH framework, θ is λ which is a central variable that the CNS use to set the joint muscle state and set the joint in motion. This model (equ. 2.36) is somewhat limited. The linear assumption is only relevant for small joint displacements. Moreover, the joint muscle decoupling of θ, θ and {a} is only an approximation of the real behavior of a joint muscle (Hogan 199).

70 7 Methods A joint muscle model that accounts for the muscle property is described in equ This model is compatible with an EPH framework. For our purpose, this joint muscle model must be extended for a limb with multiple joints. Moreover, a nonlinear account must be given because the linear model is only valid for small changes of the muscle state. This linear assumption may be violated during movement. While a joint behaves like a spring, it is a priori not obvious that the endeffector shows also a spring-like property. Static hand stiffness is measured by applying small mechanical perturbations (Tsuji et al. 1995; Mussa-Ivaldi et al. 1985; Gomi & Osu 1998). The hand stiffness is graphically and mathematically described as an ellipse with an orientation, a size and a ratio betwen the main ellipse axis. It has been demonstrated that indeed the stiffness matrix of the hand defines a potential field at the hand (Mussa-Ivaldi et al. 1985; Hogan 1985b; Shadmehr et al. 1993). The existence of an elastic potential function at the hand is sufficient to attest of the spring-like behavior of the hand. Thus, the limb in response to a transient perturbation stores energy that allows to restore the pre-perturbation state. A spring model to characterize the hand stiffness is however a crude approximation. The restoring forces at the hand are not oriented toward the equilibrium position. Indeed, the spring-like behavior has a directional property determines by the shape of the stiffness ellipse (Hogan 1985b; Shadmehr et al. 1993). Moreover, the further away from the initial position is the hand, the less stiff is the spring (Shadmehr et al. 1993). Hand viscosity has also been reported (Tsuji et al. 1995; Gomi & Osu 1998). The viscosity ellipse is roughly isomorphic to the stiffness ellipse. Thus, at rest, the hand impedance is well characterized. The hand impedance is a macro property of the limb that comes from the multiple muscles that compose the limb musculature. Because our model emphasizes the joints for movement generation, the hand impedance is less important than the joint impedance. It is however possible to relate the measured hand impedance and the joint impedance. The joint stiffness can be derived from the hand stiffness (Tsuji et al. 1995; Mussa- Ivaldi et al. 1985). This mapping between the task space and the joint space is X j = J T X J (2.37) with J the arm Jacobian. X is the stiffness but equation 2.37 is also valid for the viscosity and the inertia. This relationship between both spaces permits also to derive information on the muscle contributions to the hand stiffness. The symmetric components of the hand stiffness matrix can be for instance directly attributed to the biarticular muscles (Hogan 1985b). This impedance constrains our joint muscle model, in particular, the structure of the interjoint

71 2.3. ELEMENTS OF MOTOR CONTROL 71 muscle impedance. During the movement, the hand does not have to show a spring-like property like at rest. The hand along its path may not be indifferent to mechanical perturbations. Won & Hogan have shown formally that the hand path is indeed an equilibrium solution in the sense that mechanical perturbations that drive the hand away from the unperturbed path are resisted by an opposing force (Won & Hogan 1995). During the movement, the limb impedance varies (Gomi & Kawato 1996; Cesari et al. 21; Suzuki et al. 21). The nature of this variation is unfortunately not clearly known. It is especially difficult to separate the impedance changes that are controlled from the impedance changes that result from the muscle activity that set the arm in motion. Experiments suggest however that stiffness modulation is under active CNS control. In catching task, the hand is made compliant to minimize the effect of the impact (Lacquaniti et al. 1992). The mechanical impedance is crucial when interacting with objects in order to guarantee the stability of the coupled system (Hogan 1985b; Hogan 1985a). However, the possibility to vary stiffness may be limited depending on the arm configuration (Gomi & Osu 1998; Darainy et al. 24). The main characteristic of the dynamic impedance and the impedance in general is to vary in a complex fashion according to various tasks and task parameters. While in object interaction tasks impedance variation ensures the limb stability, impedance variation advantages in other tasks like pointing movements are less obvious. The invariance of the plane of the arm, measured as the angle between the horizontal and the arm plane, as movement speed increases shows that the speed-dependent dynamics forces are compensated for (Nishikawa et al. 1999). A speed invariance has also been found in another study of 2D motion (Boessenkool et al. 1998). An increased stiffness may account for this invariance. Another claimed advantage of increased stiffness is improvement of the task accuracy (Osu et al. 24; Gribble et al. 23). Stiffness modulation may be also a mechanism to resist better mechanical perturbations or to learn to evolve in new environments (Darainy et al. 24; Burdet et al. 21; Hogan 1985a; Takahashi et al. 21; Shadmehr & Mussa-Ivaldi 1994; Izawa et al. 24). However, an increased stiffness can also potentially induce instability of the limb. The modulation and the nature of control of viscosity have not been investigated so much as the stiffness and are thus not well known. From these considerations, it appears that for pointing movement task, the impedance changes during the movement are unconstrained. Thus, we will only assume a stiffness modulation with joint muscle torques motivated by the nonlinear invariant characteristics of the muscles (sec. 1.3). Interjoint

72 72 Methods impedance components are set according to the elastic potential field property of the limb. In practice, the impedance matrix must be symmetric (Hogan 1985b; Mussa-Ivaldi et al. 1985; Tsuji et al. 1995). Moreover, the relative size of the off diagonal and diagonal terms of the impedance matrix is set according to (Tsuji et al. 1995) (although this is a static impedance matrix for a non redundant arm!). Within the EPH framework, a third control variable describes the dependence of the motorneuron threshold on the velocity but will not be considered here (Feldman & Levin 1995). Actually the number of central variables that specifies the muscle state is not known like their dependences and their effects on the muscle impedance. We will assume basically only two central variables, λ that specifies the joint muscle virtual position and a cocontraction command that changes the joint muscle stiffness. The local stiffness within the λ model framework is the tangent to the invariant characteristic. Within this framework, the stiffness takes also a more complex twist. The stiffness defined as the slope on the invariant characteristic for a constant central variable characterizes the muscle properties. The stiffness at constant angle but for small changes of the central variables defines the sensitivity to the central variables of the muscle torque. This second definition may lead to complications when trying to measure the dynamic impedance of the limb as it can only be speculated that the central variable(s) is constant (Feldman & Levin 1995) (sec. 1.3). We postulate that the invariant characteristics are parallel curves for constant cocontraction commands. That means that the stiffness does not depend on the joint angle. The stiffness sensitivity to central variable changes and to joint angle changes is determined by the shape of the invariant characteristic. We will approximate the invariant characteristics for the agonist muscle group and the antagonist muscle group with a parametrized exponential function (Feldman 1965; Houk et al. 22; Gribble et al. 1998) (sec. 1.3). The cocontraction command that determines the overlapping activity of the agonist and antagonist muscle groups further defines the joint stiffness (sec. 1.3). Various speculations have been made for the function and time course of the cocontraction command (Lan & Crago 1994; Loeb et al. 1999; Flanagan et al. 1993). Because the nature of the cocontraction changes is not commonly agreed upon, we will make a simple unconstrained assumption of a constant cocontraction command during the movement. That means that the joint invariant characteristics are invariant during movement unfolding. The issue of the viscosity is more complicated than the stiffness because of the lack of relevant experimental data. Other model assumptions are examined. Within an EPH framework, it is proposed an overdamped muscle model with linear stiffness and linear viscosity (Hatsopoulos 1994; Flash 1987; Hogan

73 2.3. ELEMENTS OF MOTOR CONTROL ), a linear stiffness with a nonlinear viscosity (Barto et al. 1999) or a nonlinear stiffness and a nonlinear viscosity (Gribble et al. 1998; Loeb et al. 1999; Tee et al. 24). Nonlinear viscosities are modelled variously with atan( θ), a ( θ T θ + b).5, θ.2 or θ.5 (Gribble et al. 1998; Tee et al. 24; Loeb et al. 1999; Barto et al. 1999). All these models assume a higher viscosity as the joint velocity comes close to zero. The x.y models present the disadvantage of an infinite viscosity for zero velocity inducing a creeping effect at the end of the movement. The atan function does not present this disadvantage but saturates asymptotically. Within an EPH framework with a nonlinear model (see below), we explored qualitatively these various viscosity models with constant gain during the movement. Parametrizing the viscosity term is difficult to get the right movement kinematic features. This parametrization is moreover largely unconstrained as the viscosity is in general poorly characterized. We thus opt for a linear viscosity model. We have defined in the previous paragraphs a framework for a joint muscle model. In summary, we selected within a λ model framework, exponential functions to model the joint invariant characteristic with constant parameters during the movement and a linear viscosity also with constant parameters. The parametrization of the stiffness is inspired from static stiffness. We impose the stiffness at rest to be in a physiologically plausible range of values. Because the nature of the variation of the joint impedance is unclear, we chose constant parameters. We reduce the number of central variables to λ and the cocontraction command per joint muscle. So in theory, we have 8 DOFs to generate the movement of the effector. However, only the joint λ s are varied during the movement. For simplicity, the cocontraction commands for all the joints are assumed to be equal. Our joint muscle model is inspired from the muscle model proposed by Gribble & al. (Gribble et al. 1998). Some simplifications are however proposed to account for the extension from muscle model to joint muscle model. We interpret the λ within the framework of the movement generator as a global joint muscle activity rather than as a specific physiological variable. Forward conduction delays and calcium dynamics are suppressed because they infer a level of details that is inappropriate in our model of movement. In the original model, a nonlinear viscosity based on muscle lengthening rate and derived from complex muscle physiology is proposed while we use a linear viscosity. In the Gribble & al. model, the resultant joint muscle viscosity is rather complex considering that two muscles and two biarticular muscles act at each joint. No passive stiffness is accounted for because a precise biomechanical description of the arm is not part of our objectives. We account for biarticular muscles with interjoint impedance components. This assumption is not at odd with our assumption of independent joint muscles. The joint independence is meant from a viewpoint of movement generation and not at an effector level. Indeed, at the effector level the joints are not

74 74 Methods independent because they are linked to each other through the skeleton. The joints interact. The interjoint impedance imposes, in addition to the skeleton interactions, joint interactions whose origin is muscular. The torques at the joint (T m ) are given by T m = Z τ 1. τ n + ψ m (2.38) τ s are the torques produced by each joint muscle. T m is a vector of the joint torques that set the joints in motion. Z is the impedance matrix. Note that in our muscle model calling Z an impedance is abusive because Z does not have any units. Z specifies the joint relative and interactive properties. We thus assume that at rest the stiffness and the viscosity are isomorphic (Tsuji et al. 1995; Gomi & Osu 1998). ψ m is a vector of independent time correlated noise (sec ). We assume that the joint muscle torque is applied directly to rotate the joint. The joint muscle torque is given for each joint by ( ) τ i = K l (e [K nl (θ i λ p i )]+ 1) (e [K nl (θ i λ m i )] 1) + µ rl θ i (2.39) A joint agonist and antagonist muscle group account is given according to the λ model (sec. 1.3) (Latash 1993) λ p i = λ i cc the agonist muscle neuronal input λ m i = λ i + cc the antagonist muscle neuronal input Thus, each joint has two independent parameters, λ i and cc, the cocontraction command. The joint model parameters are K l muscle stiffness linear contribution K nl muscle stiffness gain (non linear contribution) µ rl muscle viscosity linear contribution θ i the effector joint angle This is the EPH joint muscle model. The parameters are listed in section Another joint muscle model is introduced for the purpose of comparison. For physiologically plausible stiffness values, the EPH model shows some limitations. The movement kinematic features are not similar to the experimental data we compare the model simulations to. In particular the end-effector velocity time course is at odds with experimental data at movement termination. Higher stiffness and viscosity values, beyond a plausible physiological range, reproduce some movement kinematic features pretty well but the hand paths

75 2.3. ELEMENTS OF MOTOR CONTROL 75 are too straight in comparison to the experimental data. We thus propose another joint muscle model that accounts for a low stiffness value. This second muscle model is based on a EPH framework that accounts for the virtual trajectory (λ, λ) (Lussanet et al. 22; McIntyre & Bizzi 1993). This model introduces a third control variable λ. This velocity variable is not another DOF in the set of central variables because it depends on λ. That means that the joint muscle torque depends on the central variable λ and its derivative λ. A similar model presented by Lussanet & al. includes linear stiffness and linear relative viscosity (Lussanet et al. 22). We will however propose a nonlinear model for both viscosity and stiffness. The elastic contribution is similar to equation We impose a viscosity contribution that is not infinite at rest and does not saturate but retains the general property of the muscle viscosity (see above). The arc sinus hyperbolic function is selected. This function is positive for positive values of the argument and negative for negative arguments. This active viscosity component thus has the potential to brake the movement when the argument is negative (sec. 3.1). Furthermore, the muscle model is complemented with a small constant passive viscosity that stabilizes the effector at rest. This muscle model is for each joint τ i = K l ( ) (e [K nl (θ i λ p i )]+ 1) (e [K nl (θ i λ m i )] 1) with + µ bl asinh( θ i λ i ) + µ rl θ i (2.4) µ bl muscle relative viscosity contribution µ rl muscle resting viscosity linear contribution This model will be referred to as the VEPH joint muscle model. The property of this model is shown in fig. 2.1 with the reference parameter setting (sec ) Physics of the skeleton The joint muscle torques (T m ), (equ and 2.4) set in motion the joints of the arm skeleton. We thus need an appropriate model of the dynamics of the arm skeleton. This dynamics cannot be predicted by extending the dynamics of a single joint to a 4-joint arm because the joints are linked through the skeleton. In other words, the joints are not independent. The dynamic equations for articulated manipulators can be derived with different formalisms. We will introduce the formalism that we use to derive the equation of the dynamics of the 4-DOF arm (left side of equ. 2.41).

76 76 Methods λ.4.2 τ i λ Figure 2.1: In the VEPH joint muscle model, the joint torque depends on the difference between the real position and the virtual position λ = (θ λ) as well as on the first derivative of that difference λ = ( θ λ). The plot does not account for the interjoint muscular relationship, i.e the off diagonal terms of the impedance matrix (Z) as well as the passive viscosity. The reference parameter setting of the VEPH muscle is used for the plot (sec ). The joint muscle torque is mainly caused by the active viscosity term λ instead of the stiffness contribution as in the EPH model. The arm skeleton is well approximated by a rigid manipulator with fixed-axis revolute joints. The dynamics of such an articulated manipulator is described mathematically by the general formulation M(θ) θ + H(θ, θ) = T m (2.41) T m vector of the joint torques (equ and 2.4) H(θ, θ) vector of Coriolis and Centrifugal forces M(θ) inertia matrix θ the joint angle vector of the manipulator Different formalisms are available to compute the dynamic equation for a particular manipulator with the Lagrangian equations. The Screw theory for rigid body motion is one convenient framework (Murray et al. 1994). This framework is convenient for our application because it rests on transformations that are based on a starting reference configuration of the arm. We will illustrate the basic idea by computing the geometric equations of the arm.

77 2.3. ELEMENTS OF MOTOR CONTROL 77 A rotation of a rigid body around a rotation axis which goes through the origin of the coordinate system at constant unit velocity is q(t) = ω q(t) = ˆω q(t) (2.42) q is a point on the rigid body and ω specifies the direction of the rotation axis. ˆω is the linear operator of the cross product. Solving for the differential equation q(t) = eˆωt q() (2.43) Thus, equation 2.43 implies that eˆωθ called the exponential map of ω. is a rotation matrix R(ω, θ) and it is To extend this same equation to any rigid body rotation (and motion) with arbitrary rotation axis, a new representation for vectors is introduced, the homogeneous representation. The vector is appended with a fourth row. A coordinate of the rigid body has for the fourth vector entry one and the velocity vector an entry zero. The effect of a rotation on a rigid body can then be described by [ ṗ ] [ ] ˆωi ω q = [ p 1 ] = ˆξ [ p 1 ] (2.44) q is a point on the axis of rotation of the rigid body from the origin of the reference coordinate system and p is a point on the rigid body. The cross product term is a linear velocity component in relationship to the arbitrary rotation axis. ξ is called a twist. The solution of the differential equations leads to the exponential map of the twist in the homogeneous representation p(t) = eˆξt p() (2.45) This transformation can be interpreted as a mapping from the initial configuration p() to the coordinates of p after the rigid body motion is applied. The twist can be extended to a Screw which gives his name to the theory. A screw parametrized a motion with an axis, a rotation around that axis and a translation along this axis. The screw will not be needed for a human arm model Equation 2.45 is a map from a reference configuration to a configuration after a rigid body motion is applied at one rotation axis. This map can be extended to a map for an end-effector whose motion depends on several joints. The map for the end-effector gives the position of the end-effector after rotations are applied to the joints of the articulated manipulator. For a multijoint manipulator, a twist is attributed to each joint. The specification of the configuration of the end-effector frame relative to the reference frame at time zero is an affine

78 78 Methods Ya end effector Yb Xa j1 j2 j3 j4 Xb Figure 2.2: The arm reference configuration is zero for all joint angles (joint space representation). The arm lies fully stretch on the abscissa in the task space. Two coordinate systems are attached to the arm: the absolute reference frame A at the sternum and the end-effector reference frame B at the hand. transformation made up of a rotation and a translation. In an homogeneous representation this operator is [ ] R p g ab = (2.46) 1 R is the rotation matrix between the two reference frames and p is the vector between the origin of the reference frame and the origin of the end-effector frame. The forward kinematic of the manipulator between a reference frame and an end-effector frame is simply the product of all the twists and g ab () and is called the product of exponentials formula (Murray et al. 1994). It is a simple geometric description of the manipulator g ab (t) = eˆξ 1 θ1 eˆξ 2 θ 2... eˆξ nθn g ab () (2.47) We will apply this mapping to our 4-DOF arm model. This is particularly straightforward because each joint is characterized by a rotation axis perpendicular to the plan of motion. Given the arm configuration in fig.2.2, the twist as a linear operator in the wedge form is given for the i th joint by equ [ ] ˆωi v ˆξ i = i (2.48) The wedge rotation axis vector is and the velocity component is ˆω i = v i = 1 1 (2.49) l i (2.5)

79 2.3. ELEMENTS OF MOTOR CONTROL 79 where l i is the segment length. It is straightforward to write the exponential map of the twist eˆξ i θ i. The reference configuration of the arm in fig. 2.2 is computed in the homogeneous representation. The starting configuration relationship between the absolute reference frame at the origin of the coordinate system and the body reference frame at the end-effector is [ ] R P g st () = (2.51) 1 The orientation of the coordinate frames are similar so R = I 3x3 and the vector between the origin of both frames of reference is L p = (2.52) L is the total length of the arm. We have derived a model of the arm geometry. Given any rotation at the rotation axes, the coordinate of the end-effector can be computed. This geometric model of the arm gives a taste for the Screw theory framework. The dynamic equations can be derived from the geometric model. The next step is to compute velocities. Unfortunately, the derivation of the velocities within the Screw theory framework is not straightforward. It is beyond the scope of this report to describe this theory in detail and the reader is referred to (Murray et al. 1994) for more details. The derivation of the arm dynamics follows using the Lagrangian equation. The result is a complex set of matrix multiplication equations. This method has been implemented in Mathematica 5. to compute the dynamics of our 4-DOF arm model. The dynamic equations for a 4-DOF manipulator are given in Appendix A Noise model We need to account for the variability of human movements. For that purpose, variability is introduced in the form of noise in our model. Appropriate noise models are required and thus the sources of variability must be identified. Variability must be considered at the level of our model framework. Our model is made up of three coupled but distinctive structures, a limit cycle framework, a neuronal movement generator and an effector. Both the movement generator and the limit cycle framework come from neuronal processes that involve potentially many neurons. Although we have not modelled the underlying neuronal circuits, both the oscillator framework and the movement generator are supported by neuronal networks in the spinal cord, the subcortical and the cortical structures. Noise at this level is thus neuronal network noise. The effector variability may come from the muscle variability. Within our joint muscle framework, the variability that matters is the variability of

80 8 Methods an ensemble of muscles functioning together at a joint. Evidence for neuronal noise is briefly examined. Electrophysiological recordings of individual neurons show variability of the spike trains for the same stimulus. This indicates that either the neurons are intrinsically variable or that the other neurons that are embedded in the same neuronal network induce variability in the measured neurons. There is indeed evidence that points to variable neuronal activities at the level of neuronal networks. Space correlation in the variability of individual neurons indicates that neurons do not vary independently from each other (Constantinidis & Goldman-Rakic 22). The deviation of the mean firing rate among inhibitory or excitatory neurons is correlated independently from task parameters. It is termed residual noise correlation whose correlation strength decays with cortex distance. The neuronal variability has the property to impinge on the signal-to-noise ratio (SNR) in a homogeneous averaging population coding by a factor which is inversely proportional to the correlation coefficient (Rieke et al. 1999). A decrease of the SNR comes from correlated noise in averaging network of neurons whereas at the same time this noise increases the inter neurons synchronization (Wang et al. 24). Neuronal imaging studies demonstrate also a variability of neuronal activity in the cortex for instance in in vivo brain optical imaging recordings. Arieli & al. investigated the origin of the response variability to a stimuli with optical imaging, local field potential and single cell recordings simultaneously (Arieli et al. 1996). They postulated that the ongoing activity is the source of variability. Measuring snapshots of brain activity before stimuli onset, they could predict the size of the stimuli response tens of milliseconds after the onset. This feature argues again against averaging of neuronal activity to extract the absolute value of the stimuli response. The second interesting feature is the prediction capability up to some time after the stimuli onset. It indicates the time correlated nature of the ongoing brain activity. It was further shown that single neuron activity is correlated to a brain state (shown as a optical image brain pattern), even in the absence of any stimuli (spontaneous brain state) (Tsodyks et al. 1999). These results demonstrate that, at the level of neuronal populations, the variable neuronal activity is time correlated. It can be modelled by adding time correlated noise to the virtual trajectory variables. Time correlated noise is expressed mathematically as the solution of τ ψ ψ = ψ + n ψ ζ (2.53) with ζ white Gaussian noise, τ ψ the correlation time, n ψ noise strength and ψ the correlated noise. The interested reader can consult (Haenggi & Jung 1995; van Kampen 24) for more information. At the muscle level, noise has been poorly studied or even ignored. At a

81 2.4. EXPERIMENTAL AND SIMULATION METHODS AND DATA ANALYSIS81 behavioral level, a trade-off has been formulated to describe limb-level variability changes. Schmidt s law relates movement time and movement distance to movement accuracy (Latash 1993). SD = a + b D T (2.54) SD is the standard deviation of the end-point at the target, D is the movement distance, T is the movement time and a, b are parameters. The shorter is the movement time the higher is the terminal variability. Contrary to Fitt s law, Schmidt s law express features of movement outcome rather than planning. This empirical law translates the assumption that muscle noise increases with muscle forces. Schmidt s law does not however identify explicitly the noise sources. This relationship is the basic motivation to use noise models that are multiplicative with the muscle neuronal inputs (Todorov & Jordan 22). This multiplicative assumption in relationship to the endpoint variability is, however, not corroborated by various studies (Osu et al. 24; van Beers et al. 24; Darling et al. 1988). Some studies have furthermore shown that accuracy increases as the muscle activity (measured with EMGs) is increased (Osu et al. 24; Gribble et al. 23). We do not therefore consider multiplicative noise. The respective effect on movement variability of neuronal and muscle noise has not been studied thoroughly. One combined experiment-model study has concluded to a dominant contribution of noise in the muscle motor command to the total movement variability (van Beers et al. 24). In light of these results, muscle noise is modelled as in equ but with a larger correlation time. Moreover, relative noise level between neuronal noise and muscle noise will be varied to test different contributions to the total movement variability. 2.4 Experimental and simulation methods and data analysis Data collection In order to evaluate the model, we compare our simulations to human movements. Data were collected in a pointing movement experiment, that was specifically designed for that purpose of comparison with the model. The experimental data have been collected by Prof. J.P. Scholz and colleagues at the University of Delaware (USA) for three participants. Each subject performed six movements divided into two groups in order to sample the workspace and arm configurations. Movements 1 to 3 have a similar target on the right hand side of the subject (fig. 3.4). Movements 1 and 2 have

82 82 Methods the same end-effector starting position but different effector starting configurations. Movements 3 to 6 are aimed at the same target on the left hand side of the subject (fig. 3.4). Movements 3 and 4 have the same end-effector starting position but different effector starting configuration. Movement speed was consistent and not varied in this experiment. The subjects were asked to move at a fast movement speed. Furthermore, the subjects were asked not to correct for a miss target at movement termination. Each movement was repeated about 35 trials but when the subject failed to comply with the experimental conditions, data were immediately discarded. Further data are eliminated when the velocity profile of the end-effector reveals corrective actions. At the end of the experiment, for each subject and each condition, is more than 2 valid trials (except for subject B in 3 conditions in which 18 respectively 19 and 14 valid trials are selected). The end-effector starting position, the target position and the joint starting configuration were controlled from one trial to the next. The collected data consist of the movement time, the joint positions and, the joint velocities computed from the joint positions for each trial and sampled at 1/12 seconds. The movement involved 4 joints, sternum, shoulder, elbow and wrist. Other joint movements were restricted by the experimental set-up. These data were specifically collected to constrain parameter setting in the model and to allow for a direct comparison between model performance and real human performance. These data are thus not intended for rigorous statistical analysis, as attested by the small number of subjects. The features that are extracted from these data have been analysed and published elsewhere on similar but different data sets (Scholz & Schöner 1999; Scholz et al. 2; Reisman et al. 22). The exact data collection methods and the materials can also be found in these publications The UnControlled Manifold (UCM) The UnControlled Manifold (UCM) theory is presented in section This section presents the methodology to compute the UCM signature in the experiments and in the model. The same method is applied for the experimental data and the model data. The UCM is computed from a set of discretized trajectories. In the model, the number of sample points for each movement is reduced for the purpose of an efficient computation. In experiments, the number of sample points is increased with the Matlab function spline to an interval of.1 s. This is done for the purpose of the time normalization. Two hypotheses in the sense of the UCM theory are considered, the hypothesis position of the end-effector and the hypothesis position of the centre of mass of the arm. The end-effector position is computed from the forward kinematic map (assuming the rigid body of fig. 2.2) (sec ). The task variable for

83 2.4. EXPERIMENTAL AND SIMULATION METHODS AND DATA ANALYSIS83 the arm centre of mass position is given by a weighted forward kinematic r = lcmi m i M (2.55) l cmi is the centre of mass position for the ith link and m i is the mass of the limb segment. M is the total mass of the arm. The task variable is postulated to be the end-effector position but usually a task variable for movement extent, along the straight line between the starting position and the target position, and a task variable for movement direction, perpendicular to movement extent are examined separately. The coordinate system attached to this decomposition is called the local coordinate system whose abscissa corresponds to the direction of movement extent. The motivation for this decomposition of the task variable is a timing issue. For movement direction, movement time is not important as the direction of movement varies little during the movement. The movement extent hypothesis depends on the other hand on movement time because the position of the end-effector is shifted along the abscissa of the local coordinate system during the movement. Thus, these two hypotheses differ in relationship to the time dependence. This decomposition is now the standard procedure in experimental UCM analysis to study pointing movements (Tseng et al. 23; Tseng & Scholz 24). The variability is computed around mean sampled values. In order to compute the mean, the trajectories must be matched from one trial to the next because the movement time varies significantly. Thus, trajectories have to be normalized in time. Observable events at the beginning and at the end of the movement are reliable indices to find equivalent trajectory states among the trials. Because no such discrete events are observed in pointing movement, the velocity invariant profile is chosen as a reference. The beginning of the movement is chosen to be 1% of the peak end-effector velocity. The end of the movement corresponds to 3% of the peak velocity. This normalization method comes from the experimental procedure for computing the UCM signature. The mean movement time between these two events is used to normalize the time between the trials. The same time normalization procedure is applied to the model data although the internal system time is precisely controlled. The mean value of each joint angle for each time instant is computed from all the valid trials for each subject. The variability from all the valid trials computed around the mean value for successive time instants reflects the evolution of the variability in time during the movement. This variability can be computed for the individual joints, the individual joint velocities, the end-effector position and the end-effector velocity with the same normalized time. The UCM variability structure is unravelled by decomposing the joint variability into a component within the UCM (Goal Equivalent Variability; GEV) and perpendicular to the UCM (Non Goal Equivalent Variability; NGEV). The

84 84 Methods UCM is a complex object that can be approximated locally by a linear subspace characterized by a basis. To compute GEV, the UCM can be approximated locally with the null space of the task variable Jacobian. This assumption is valid for sufficiently small configuration changes around the mean. For each normalized time sample, GEV is estimated around the mean joint configuration. The linearized forward differential kinematic map around the mean reference configuration for each time instant is (sec ) r ni r i = J(θ i ) (θ ni θ i ) (2.56) J is the task Jacobian computed for the mean configuration, θi is the mean configuration at the normalized time i and ri is the mean value of the task variable at time i. θ ni is the configuration at time i for the trial n and r ni is the task variable value at time i for the trial n. A basis for the null space is computed with the Matlab function null. At each time instant, the deviation of the joint configuration of the trial n from the mean can be divided into the component that affect the task variable and a perpendicular component that does not affect the task variable. The vector component of the joint variability for the time i and the trial n that lies in the UCM is Θ n = l [ɛ T il (θn i θ i )] ɛ il (2.57) with ɛ il the normalized vectors that form a basis for the null space at time i. Θ n is the vector of the joint variability that lies in the null space of the task Jacobian for the time i. The vector component of the joint variability that lies outside the UCM is the difference between the joint variability vector and the vector component of the joint variability that lies within the UCM Θ n = (θ θ ) Θ n (2.58) Θ n is the vector of the joint variability that lies in the subspace perpendicular to the linearized subspace of the UCM for the time i (the indices i are omitted for clarity purpose). For the time i, the vector decomposition is computed for all the trials for one subject. The norms of the vectors are computed and summed over the trials. The variability in both subspaces depends on the subspace dimension. The higher the dimension the larger is the variability. If the variability of the joints was independent for each joint and equidistributed, then the variability in each subspace should be the same providing that both subspaces have the same dimension. The variability is therefore normalized per DOF. The amount of variability per DOF for both subspaces is estimated with σ 2 i = R 1 N 1 N Θn 2 (2.59)

85 2.4. EXPERIMENTAL AND SIMULATION METHODS AND DATA ANALYSIS85 σ 2 i = K 1 N 1 N Θn 2 (2.6) R is the dimension of the null space of the task Jacobian, K is the dimension of the rank space of the task Jacobian and N is the number of trials. When σ 2 i is bigger than σ 2 i, the hypothesis is accepted for time i. If this is the opposite, the hypothesis is rejected. This method is also described in detail in (Scholz & Schöner 1999; Scholz et al. 2; Scholz et al. 21) Selfmotion The selfmotion concept is introduced in sec This section describes the methodology to compute selfmotion. This computation is based on the effector/end-effector velocity map (sec ). The computation of the selfmotion is similar to that of the UCM signature. The selfmotion space is the null space of the task Jacobian for the end-effector position. The projection of the joint velocity vector on the null space of the task Jacobian is the selfmotion. The difference between the joint velocity vector and the selfmotion vector component is the transport component that drives the end-effector toward the target. S n i = l [ɛ T il θ ni ] ɛ il (2.61) T n i = θ ni S n i (2.62) S n, T n are respectively the selfmotion for the n trial at time i and the transport motion for the trial n at time i. Both velocity components are summed over the number of trials and normalized with the number of trials but not with the number of DOFs which is anyway equal for both spaces in the 4-DOF planar arm model N is the number of trials Motor equivalence S i = N 1 S n (2.63) n T i = N 1 T n (2.64) The motor equivalence concept is introduced in section 1.5. This section describes the methodology to compute motor equivalence. This computation is based on the UCM computation procedure. Motor equivalence is a method n

86 86 Methods to evaluate the significance of arm configurational changes. analysis, motor equivalence is a comparative method. Like the UCM Motor equivalence is computed only in the model as no experimental data are available so far. The simulations are performed without noise. Two trials are compared to each other. The first trial is the unperturbed trial and the second trial is the perturbed trial. The unperturbed trial gives the reference movement time as well as the reference configuration from which the task Jacobian is computed. The computation is then identical to the UCM but for the number of trials (sec ). The motor equivalence components are not normalized by the number of trials and the number of DOFs per subspace which is anyway identical in our model of movement. This is not important as by definition only the relative contribution matter. The following perturbations are examined : The second joint is clamped at movement onset. θ2 and θ 2 are set to zero throughout the movement. The second joint is clamped for 5ms during the movement. A task space constant force F is applied to the end-effector during the movement for 5ms with a movement time of 4ms; P = J T F is then the perturbation in the joint space with J T the transpose of the task Jacobian Model implementation The model is implemented in Matlab version 13. The differential equations are solved with an Euler method with a time step of.2. Gaussian white noise is generated with the built-in function randn. Uncertainty of the end-effector position at the movement onset is modelled by an uniform distribution with the function randunifc (B. Huxley; Random Variable Generation Box from the Matlab File exchange at fileexchange) (see also below). To compute the variability, the UCM signature and the selfmotion, 1 trials are generated for each movement. The simulation procedure starts from an intertrial-fixed arbitrary parameter values for the joint state, so for each joint (θ i, θ i ) is ((θ 1 +1 o, -.27);(θ o,.4);(θ o, -.55);(θ o,.72)). The effector relaxes in the presence of noise toward the starting position of the end-effector. The effector configuration is not explicitly given at movement onset. This procedure allows to reproduce the experimental starting conditions. The arm is maintained in the resting state for a period of time of 1 s for a movement time of about 4ms, with fine positioning activated before the movement starts and in the presence of noise. An algorithmically

87 2.4. EXPERIMENTAL AND SIMULATION METHODS AND DATA ANALYSIS87 set variable, I m > for a transient arbitrary time of.2 s, starts the movement. The movement stops when the limit cycles reach again the starting state (I r ). No fine positioning is active at the end of the movement in agreement with the Do not correct for missed target paradigm in the experiments. A reference set of parameter values is called the reference parameter setting. Simulations are performed with these reference parameter values if no other parameter values are given. These parameter values are given in three tables; the general parameters (Tab. 2.2), the biometrics parameters (Tab. 2.1) and the muscle parameters (Tab. 2.3). To fit the model with the experimental data, the noise parameter values, the backcoupling constant values and the muscle parameter values are varied. In practice, the strategy followed for parameter setting is to first get right for all six movements: the UCM signature, the selfmotion consistent features such as the selfmotion increase at the beginning of the movement and the selfmotion decrease at the end of the movement and the joint and joint velocity variability in as far as common features can be observed among the subjects. The kinematic features are then also considered. A unique parameter setting is selected for all six movements under considerations instead of setting a set of parameter values for each movement independently. The number of variable parameters for the muscles is reduced to a minimum in agreement with experimental data and model details. A single symmetric impedance matrix (Z) is used instead of one matrix for the stiffness and one for the viscosity. It is assumed that the impedance matrix does not vary in time, rather than assuming another time-varying constraint for the impedance matrix. The nonlinearity feature of the joint muscle models (k nl ), the parametrization of the exponential function (equ and 2.4), is similar for all the joints. The biometrics parameters are taken from one subject (Tab. 2.1). The centre of mass and the inertia of the various arm segments are computed following (Hanavan 1964). The data for the first joint were not given and are estimated to be 1/4 of the upper torso biometrics data. Parameter name Symbol Value Units Body mass M 55 kg First segment length l m Second segment length l m Third segment length l m Fourth segment length l m Table 2.1: Biometric parameters. The noise levels are not strongly constrained by physiological evidence. The

88 88 Methods relative variability contribution to the total movement variability of the joint muscle noise and the neuronal noise is unknown. The joint velocity variability features constrain the muscle noise to the extent that the muscle low-pass property is captured in the muscle model and that the noise model is sufficiently good. This is only a weak constraint. The uniform variability distribution for the end-effector starting uncertainty, generated with the function randunifc, is parametrized with (-.8;.8; 1) to match roughly the experimental data (fig. 3.4). Parameter name Symbol Value Units Number of trials 1 Euler integration step T step.2 s Oscillator limit cycle time ω 16 Hz Oscillator stability α.5 s 1 Oscillator amplitude γ determined s m 2 Neuronal noise level (NN) n ψλ.8 NN correlation time τ λ.2 s Oscillator noise level (ON) n ψo.8 ON correlation time τ o.2 s Muscle noise level (MN) n ψm.5 MN correlation time τ m.2 s Switching dynamic constant βr, βm 4 s 1 Switching dynamic resting state h 1 Switching dynamic constant δ 2 Oscillator relaxation constant β o 15 s 1 Virtual relaxation constant β v 3 s 1 Stopping activity gain (I r ) a 15 Fine positioning constant β f 25 s 1 Backcoupling constant position β n1 1 s 2 Backcoupling constant velocity β n2 35 s 1 Virtual velocity damping constant β s 3 s 1 Delay constant τ.1 s Table 2.2: Model parameters The joint muscle parameters are set based on general knowledge of joint muscle features, with a specific focus on setting the stiffness values within a physiological range (fig. 2.3) (Tsuji et al. 1995). The impedance matrix parametrization is constrained by the elastic force field property of the hand and the movement kinematics. The balance between viscosity values and stiffness values are crucial to terminate the movement properly.

89 2.4. EXPERIMENTAL AND SIMULATION METHODS AND DATA ANALYSIS89 Parameter name Symbol Value Units Cocontraction cc pi/9 rad Impedance matrix Z (see above) 3 6 Linear stiffness (EPH) K l 1.2 kg m 2 s 2 Nonlinear stiffness gain (EPH) K nl 1 Linear passive visc. (EPH) µ rl.25 kg m 2 s 1 Linear stiffness (VEPH) K l.4 kg m 2 s 2 Nonlinear stiffness gain (VEPH) K nl 1 Linear active visc. (VEPH) µ bl.3 kg m 2 s 1 Linear passive visc. (VEPH) µ rl.3 kg m 2 s 1 Table 2.3: Muscle parameters. VEPH is the VEPH joint muscle model and EPH is the EPH joint muscle model. 3 Torque 2 m1 m2 1 s λ Figure 2.3: The EPH (m1) and the VEPH (m2) joint muscle stiffness functions are shown for an arbitrary λ and for the first joint, with λ in [rad] the difference between the virtual and the real joint position. The models are parametrized with the reference parameter values. Note the difference of the linear stiffness at the origin for the first muscle model (dash line; m1) and the second muscle model (dotted line; s). The VEPH muscle model static stiffness is in the physiological range. The EPH muscle model static stiffness is too large. The torques are in [Nm].

90 9 Methods

91 RESULTS 3.1 Model functioning Each simulation of a movement is composed of four different phases; a relaxation phase, a pre-movement resting phase, the movement itself and a postmovement resting phase (fig. 3.1). The relaxation phase implicates that the limb rests from an arbitrary, but constant for all the trials, state to the resting state. The relaxation phase conforms to the experimentalist setting the subject arm before the trial starts. Because the noise is active throughout this phase, variability of the arm starting configuration is generated between the trials. The pre-movement resting state is identified with a uniformally distributed random end-effector position around a mean end-effector starting position and zero joint velocities. The arm starting configuration is not specifically given. The pre- and post-movement resting phases are characterized by constant effector configurations. Unless stated, the noise is effective for all simulation phases. The pre-movement resting phase is the waiting time before movement onset. The post-movement resting phase corresponds to the experimental instruction to maintain the terminal configuration until the end of the trial is notified. In the model, the resting phase is symbolized by a corresponding internal state of the body (fig. 3.1; equ. 2.19). At movement onset, the internal resting state commutes to an internal movement state. Only one state can be active at a time. An active state is modelled by a positive value (fig. 3.1; equ. 2.19). The virtual end-effector velocity switches the internal movement state to an internal resting state at the end of the movement (sec. 2.3). Then, the arm comes to rest in a new final configuration. The torques generated at each joint depend on the difference between the real trajectory and the virtual trajectory. The real joint deviation from the virtual joint trajectory generates a joint force field that forces the joint in motion as long as no equilibrium position is reached by the effector (sec. 1.3). The virtual trajectory generates a shift in time of the equilibrium position of the effector force field. The real joints track the virtual trajectory but, because of the joint inertia, the real joint angles lag behind the virtual joint angles. In addition, interjoint interactions caused by the limb dynamics and the interjoint muscles induce deviations of the effector from the virtual trajectory. 91

92 92 Results Joint angle Virtual end effector velocity Movement Movement Real end effector velocity Time Time Figure 3.1: Three phases of the movement unfolding are shown; the premovement resting phase, the movement and the post-movement resting phase. The resting phases are identified by a neuronal state of a competitive neuronal dynamics (equ. 2.19) (dashed blue line; upper plot). The internal movement state is active during the movement (black solid line; upper plot). An intention to move switches the neuronal dynamics from a resting state to a movement state. The real arm moves delayed compared to the virtual arm (compare the time profiles of the end-effector velocity). The delays arise because of the arm and muscle properties. The movement is terminated when the neuronal system switches again to a resting state. The lower figure shows the time course of the real and the virtual joint angles (light line) numbered from 1 (the sternum) to 4 (the wrist) for movement 1. During the resting phase, virtual and real joint angles are constant and equal. The joint angles are given in [rad], the velocity amplitude in [m/s] and the time in [s]. Because the force field is not strong enough, the real joint trajectory deviates considerably from the virtual joint trajectory. Stiffer joint muscles reduce the discrepancies between the real trajectory and the virtual trajectory. The joint angles often evolve monotonically during the movement but not always (fig. 3.2). For instance, the first virtual joint position first decreases

93 3.1. MODEL FUNCTIONING 93 and then later in the movement increases in movements 1 to 3 (fig. 3.1). The real joint trajectory is determined by the effector properties and the virtual trajectory and thus does not necessarily follow closely the virtual trajectory (fig. 3.1). In addition, the virtual joint trajectory is not determined fully before movement onset. During the movement, the virtual joint trajectory is influenced by the deviations between the real and the virtual joint trajectory because the virtual joint trajectory is compliant to the real trajectory (fig. 3.3; sec ). This compliance is expressed in the null space of the virtual joint trajectory with the backcoupling. The small damping of the virtual 2 Joint angle Time Figure 3.2: The joint paths for the movement 6 are shown (real path solid line; virtual path dashed/dotted line). During this movement, the joint angles either increase or decrease such that between some joint pairs, the joint motions and the joint torque are in opposed directions. The neuronal dynamics that determines the internal resting and movement states is plotted in the background as in fig The time is in [s] and the joint angles in [rad]. joint velocity at rest prevents the real joint configuration to drift continuously, especially in the null space, at the end of the movement (equ. 2.29). This damping stabilizes the virtual joint velocity around zero. A damping or a fine positioning in the virtual task space does not provide stability in the null space at rest (equ. 2.24). This is so because of the task decoupling of the movement generator. The stability of the limb at rest and during the movement has not been examined formally. We are satisfied if the real joint configuration is seemingly constant during the resting phases. During the movement, the vector field in the null space of the movement generator which determines in particular the variability within this null space is set by the backcoupling (sec ). The strength of this vector field in the null space is shown in fig 3.3 during the movement. At the end of the movement,

94 94 Results the virtual acceleration in the null space tends toward zero in agreement with the definition of the resting phase. At rest, the joints are motionless and the virtual and the real joint configuration are identical. During the movement, the virtual trajectory is attracted in the null space toward an equilibrium point formed by the real joint trajectory. The larger is the difference between the virtual trajectory and the real trajectory the stronger is the vector field. A perturbation in the null space of the movement generator that drives the virtual effector state away from the real effector space is thus resisted. The difference between the virtual trajectory and the real trajectory that set the vector field strength depends, in addition to the arm dynamics and the interjoint effects, on the afferent delays (sec ). Co 2 Movement 1 Co Movement Time Time Figure 3.3: The magnitudes of the vector field for both dimensions of the null space are shown during the movement for movements 1 and 4 and without noise. The vectors that form a basis for the null space are arbitrary (see also fig. 3.31). The dashed dotted lines show the time when the virtual internal state commutes (see fig. 3.1). The time is not normalized. The vector field magnitude (co) is in [m/s 2 ] and the time in [s] 3.2 End-effector kinematics For all movements in the model and in experiments, the end-effector path is roughly straight as claimed in previous studies (fig. 3.4 and fig. 3.5) (Morasso 1981; Flanagan et al. 1993; Boessenkool et al. 1998). It is, however, undeniable that some paths display a curvature whose degree varies with the workspace position. Non redundant arm movements in a plane show also slightly curved end-effector paths whose curvature varies again in various regions of the workspace (Flanagan et al. 1993). Moreover, these paths are hooked at the end of the movement. These hooks can also be observed in the model for certain parameter values of the muscle impedance. The path of the end-effector is determined in the model by the muscle properties and the arm dynamics but only weakly by the arm configuration. In

95 3.2. END-EFFECTOR KINEMATICS 95 addition, one can expect that a different time course of the task virtual trajectory would lead to a different end-effector path (see also Latash 1993). The muscle properties directly affect the path of the end-effector. Indeed, stiffer muscles lead to straighter end-effector path in the model. In that case, the arm dynamics is compensated for by the muscle elastic property alone without any computation of the arm inverse dynamics. These straight end-effector paths, however, do not match the experimentally measured paths tending to be too straight. What the path curvature indicates about the motor control strategy in humans is unclear, but it has been shown that visual perception of the end-effector path influences the path curvature as well as learning to move in new environment (Flangan & Rao 1995; Wolpert et al. 1995; Guenther & Barreca 1997). These results indicate that the path curvature does not simply depend on the inability to compensate for the joint interactions. The path curvature can be understood as a side effect of the movement generation strategy rather than an objective of the movement strategy as often postulated. In the model, the end-effector path curvatures do not have a unique cause. A critical comparison between the experiments and the model reveals endeffector path differences that may have various causes (fig. 3.4, fig. 3.5, see the path of movement 6). The muscle impedance may be one of the major reason that leads to discrepancies between the experimental end-effector paths and the end-effector paths in the model. Muscle impedance varies during the movement and in various parts of the workspace (Gomi & Osu 1998; Gomi & Kawato 1996). Recall that the modelling strategy is to set a unique parameter set for all trajectories. At variance, each trajectory could have been specifically matched to an arbitrary particular parameter set to reproduce better the end-effector paths. Moreover, muscle impedance could have been varied during the movement. Unfortunately, the passive and active impedance changes are still poorly understood among other reasons because the effector impedance is practically difficult to measure (sec ). In the model, the path curvature depends moreover on the movement time in that slower movements tend to be straighter which reflects the dependence of the arm dynamics on the joint velocities (fig. 3.6; appendix A). This endeffector path change for different movement speeds is reduced when the joint muscles are very stiff or could be reduced if movement speed is accounted for in the virtual trajectory. In the model, movement time is not accounted for in the movement generator. In experiments, invariance of the end-effector path for various movement speeds is reported in some conditions (sec. 4.3). The arm configuration may not be a determinant factor for the curvature of the end-effector path in pointing movements, at least in the part of the workspace examined in the present study and for planar movements, but see (Desmur-

96 96 Results y end.4 end 6 start start start x Figure 3.4: The end-effector paths in horizontal planar movements executed with a 4-DOF effector are shown for various targets and for various starting positions and for the first subject (referred to as A in the next figures). The first joint coordinate is (,). Each path is given a number (1,3,4,6) which is used in the next figures to designate the individual paths. Each pointing movement starts from an experimentally controlled starting configuration. Note the variability of the paths but also the variability of the end-effector starting positions and of the end-effector endpoint positions. The subject is instructed not to correct for errors at the target. The paths 1 and 3 are clearly curved, the path 4 is rather straight whereas the path 6 shows again a curvature. The straight line in the plot links the mean end-effector starting position and the location of the target. Movements 4 and 6 clearly undershoot the target. x and y are given in [m]. get & Prablanc 1997). Experimental data and model data show no qualitative difference for the curvature of the end-effector path between various starting configuration conditions (movement 1 with the joint configuration (from the proximal to the distal joint and in [rad]) : (-.4733; ; ;.385) and movement 2 : (-.688;1.6348;2.2192;-.3239) respectively movement 4

97 3.2. END-EFFECTOR KINEMATICS 97 y Figure 3.5: The end-effector paths for movements executed with a 4-DOF effector are shown for various target locations and various starting positions similar to fig. 3.4, in the model. This figure should be compared to fig The path curvatures are qualitatively well reproduced except for movement 6. This figure shows moreover the virtual paths (magenta dashed lines). The virtual paths are straight but for some variability. The uncertainty of the end-effector starting positions is modelled with a uniformally distributed noise added to the mean starting positions. Note that the end-effector never undershoots the targets as in experiments because a pseudo-equilibrium position characterizes the virtual end-effector position when the post-movement resting phase is active, which is always at the target proximity at the end of the movement. With a fine positioning for the target, the end-effector reaches the target precisely (fig. 3.8; sec ). x and y are in [m]. x : (-.4462;1.5693;1.957;.2851) and movement 5 : (-.6595;1.6427;2.237;-.3175)) (not shown). With a model restricted to planar movements, these results do not allow to conclude on an invariance of the end-effector path for

98 98 Results various starting configurations of the effector. Moreover, the limited range of joint motions is not accounted for at the effector level or at the neuronal level in the model which does not allow to compare the model to experiments for the terminal arm configurations. Despite this model simplification, the endeffector paths are well reproduced for the various configurations tested. Differences between the experiments and the model are also noted at movement termination. The experimental instruction not to correct for target errors is probably not accounted for properly in the model (fig. 3.4, fig. 3.5, see the undershoot of the end-effector path for the movements 4 and 6). In the model, the movements are terminated without the instruction to remain at a precise end-effector position which would require to set an arbitrary virtual end-effector position at the end of the movement. In addition, possible distortions in target perception are not accounted for (Wolpert et al. 1995). This shows two important aspects of modelling arm movements. An account of the arm biomechanics is important but an appropriate account of the experimental paradigms must also be provided. The EPH joint muscle model produces similar trajectories, given the proper ratio between viscosity and stiffness, in comparison to the VEPH joint muscle model (sec ). However, an inappropriate muscle parameter setting for the EPH muscle model induces a second acceleration at the end of the movement to reach the target that comes from an initial undershoot of the target (not shown). This default can be alleviated if the stiffness and the viscosity values are increased beyond a physiological range. The end-effector paths are then really straight because the real trajectory sticks to the virtual trajectory. The end-effector paths are, in this case, too straight in comparison to the experimental data. The very stiff EPH joint muscle model (K l = 5; µ rl =.8) will be referred to in the next sections to underline the qualitative different behaviors of the model with this particular muscle model. The time profile of the end-effector velocity is bell-shaped and slightly asymmetric in the model and in experiments for all movements (fig. 3.7). No terminal velocity bumps are observed in the experiments or in the model. This may be related to the experimental instructions and, respectively, to the model movement termination. In an experiment with a non-redundant arm, the endeffector velocity profile is bell-shaped at the subject preferred movement speed but shows some bumps as movement speed increases (Flanagan et al. 1993). These terminal bumps may arise from the inability to balance accelerating torques and decelerating torques so that the movement first undershoots or overshoots the predefined target, as seen also in fig. 3.4 for movements 4 and 6. In the model, such terminal velocity bumps can occur for the EPH muscle model with low muscle impedance. In fig. 3.7 the time profile of the endeffector velocity is examined along the axes of the local coordinate system.

99 3.2. END-EFFECTOR KINEMATICS 99 y Velocity Time x Figure 3.6: Each movement (1 to 6) is executed at two different speeds without noise in the model. The path curvature increases as the movement time decreases (dashed line, high speed). The inset shows the end-effector velocity magnitude in time for the two movement times. The end-effector velocity is in [m/s], the time in [s] and x,y in [m]. The main axis of this coordinate system lies along the line segment between the mean starting end-effector position and the target position. The second axis is perpendicular to this main axis (fig. 1.1). The component of the endeffector velocity along this second axis reflects the path curvature. Indeed, end-effector deviations from the straight line between the starting end-effector position and the target position results from a velocity component perpendicular to the main axis of the local coordinate system. The model reproduces these curvatures nicely with the exception of movement 6 (fig. 3.7). Subjects are able to accommodate a change of the target position during an ongoing movement (see also Desmurget & Prablanc 1997). This ability motivates the use of one oscillator per DOF of the task space (sec ). This target update does not change qualitatively the velocity profile of the end-effector. When a new target position is defined after movement onset, the task dynamics in the model is adapted and a new end-effector trajectory emerges (fig. 3.8). The real end-effector follows the new virtual path until the new target is reached. Our oscillator framework allows to reach the new target smoothly.

100 1 Results Experiment Model Time Time Figure 3.7: The mean end-effector velocity amplitude in time is depicted for the subject A and for the model for various movements. The four profiles numbered on the right-hand side correspond to the four paths shown in fig The mean velocity is computed based on time normalized trajectories as explained in the methods. The velocity vectors are divided into a component along the straight line between the mean starting position and the mean target position (red solid line) and a component perpendicular to this straight line (magenta dashed line) in a conventional Cartesian coordinate system referred to as the local coordinate system. The perpendicular velocity component reflects the path curvature. Again, movement 6 differs the most from the experimental data. The time is in [s] and the end-effector velocity in [m/s].

101 3.3. VARIABILITY 11 y.5 end start Velocity Time x Figure 3.8: A new target (arrow) is given after movement onset (ca. 25% of the virtual movement time). The end-effector virtual path (fine dashed line) as well as the real end-effector path (solid line) move smoothly to the new target position. The velocity profile is not changed qualitatively (inset). A fine positioning at the end of the movement is introduced so that the endeffector reaches the target properly. The limit cycle stability is also increased to α = 3 for this simulation. x,y are in [m], the velocity in [m/s] and the time in [s]. 3.3 Variability End-effector and joint variability The end-effector variability during the movement is decomposed along the main axes of the local coordinate system (fig. 3.9 and fig. 3.1). The time is normalized as explained in the methods. The component of this variability projected along the main axis of the local coordinate system, i.e along the straight line between the mean starting position and the target position is characterized by a bell-shaped profile whose maximum corresponds in time roughly to the end-effector peak velocity occurrence. The perpendicular component of this variability is more or less constant throughout the movement. The end-effector variability profiles do not depend on the joint muscle model used. The time profile of the variability component along the main axis of the local coordinate system is determined mainly by the time normalization procedure. Without this time normalization, the variability of the end-effector increases more or less monotonically with the movement time like a diffusion process. The time normalization cancels effector position errors induced by timing difference between the trials. Thus, the time normalization procedure squeezes the end-effector paths between the start and the end of the movement and, by consequence, generates small errors at the beginning and at the end

102 12 Results of the movement but, more variability in the middle of the movement. This phenomenon is less apparent for the hypothesis movement direction because the movement direction does not depend strongly on movement time. The time profiles of the end-effector variability are similar for the model and the experiments (fig. 3.9 and fig. 3.1). The variability for each joint is evaluated during the movement with the same time normalization method (fig. 3.9 and fig. 3.1). Globally, the joint variability increases in time although not necessarily monotonically. A monotonic increase of the joint variability in time is observed in the model without time normalization (see above). Joint variability profiles share common features between the experiments and the model (for instance see the joint 3; fig. 3.9). These features may be dictated by the arm dynamics and the arm geometry. The similarity of the time profiles of the joint variability between the model and the experiments is remarkable as it is not obvious a priori that the model should reproduce the joint variability changes in time. The joint variability at the beginning of the movement is smaller in the model than in experiments (fig. 3.9 and fig. 3.1). In the model, this variability comes during the relaxation phase. Unlike the end-effector starting position, no variability is specifically added to the starting joint configuration between the trials. In experiments, one may expect some starting joint configuration variability although special care is taken to start each trial in the same configuration. This difference is also reflected in the UnControlled Manifold (UCM) signature (sec ). Variability of the joint configuration could also be accounted for in the model with different joint starting configurations at the relaxation phase onset between the trials. This would generate a strong pre-movement UCM. The next section explains the consequence of this premovement UCM in detail. The cocontraction command in the EPH muscle framework is claimed to reduce end-effector variability at the end of the movement (Gribble et al. 23) (sec. 1.3). In the model, the cocontraction command is varied from -.2 to 1.2 and the end-effector variability examined with and without time normalization (sec. 2.4). In any case, the cocontraction command does not affect the end-effector variability at the end of the movement. Note that target error is not accounted for in the computation of the end-effector variability at the end of the movement. Only the endpoint dispersion is considered. The variability of the joint velocity during the movement can also be estimated with the same time normalization method. Similarly to the joint variability, the joint velocity variability is well captured in the model (fig. 3.11). It should be noted that stiffer muscles contrain the muscle noise model much more than compliant muscles. With stiff EPH joint muscles, a high level of muscle noise cannot account for the right joint velocity variability while it can for compliant

103 3.3. VARIABILITY 13 Experiment Model.1.8 A B C Time Figure 3.9: The variability of the end-effector is decomposed in the local coordinate system into a component along the straight line between the mean starting position and the target position (dotted black line) and a perpendicular component (solid blue line). The time is normalized. The individual joint variability is computed following the same time normalization procedure (label 1 to 4). The results are shown for each subject (A,B,C) and the model for the movement 1. The arrow highlights the smaller joint variability at the beginning of the movement in the model. The time is in [s], the joint variability in [rad] and the end-effector variability in [m]. Experiment Model A B C Time Figure 3.1: The variability of the end-effector and the joints are shown for the three subjects (A,B,C) and the model for movement 4. Note the different ordinate scale for the subject B. See fig. 3.9 for the label and line convention. The arrow highlights the smaller joint variability at the beginning of the movement in the model. The time is in [s], the joint variability in [rad] and the end-effector variability in [m]. muscles (fig. 3.11). The compliant joint muscles act as a low-pass filter on the noise.

104 14 Results Experiment Reference High muscle noise dir ext Figure 3.11: The joint velocity variability is depicted for the movement 1 and for the subject A, the model with the reference parameter setting and the model with a high muscle noise level (n ψm =.6). In the model, the qualitative features of the variability are well reproduced (see for instance joint 2). In the right plot, the variability of the end-effector velocity in the local coordinate system is also shown. The time is normalized as explained in the method. The time is in [s], the joint velocity variability in [rad/s] and the end-effector velocity variability in [m/s]; ext stands for the hypothesis movement extent and dir for the hypothesis movement direction. 4 3 Time The UnControlled Manifold (UCM) for the hypothesis end-effector position The time profile of the joint variability decomposed according to the UCM method, in short the UCM signature, is similar for all movements (fig. 3.12, fig. 3.13, fig and fig. 3.15). The UCM signature for the hypothesis movement extent is characterized for NGEV by a bell-shaped profile whose maximum coincides roughly with the peak end-effector velocity. NGEV at the end and at the beginning of the movement is quantitatively similar. GEV is more variable among the subjects. The overall feature of this component of the joint variability is to increase as the movement unfolds. A less pronounced decrease or at least a stabilization at the end of the movement is observed. A curve cross-over between GEV and NGEV is sometimes observed but is not systematic for all movements and all subjects. The UCM signatures for the hypothesis movement direction can be grouped based on their features for the right side movements (1 to 3) and respectively for the left side movements (4 to 6). In the first group, NGEV is identified with an increase early in the movement followed by a fast decrease before movement termination (fig and fig. 3.13). This feature is systematically observed in the model for various parameter values (fig. 3.17). GEV varies a lot among the subjects but in general is similar to GEV for the hypothesis movement extent.

105 3.3. VARIABILITY 15 Experiment Model Time Figure 3.12: Joint variability decomposition for the movement 1 within the UCM (red continuous line) and perpendicular to the UCM (black dot line) for the hypothesis movement direction (1,3,5) and movement extent (2,4,6). Three different subjects are shown (subject A: 1,2; subject B: 3,4; subject C: 5,6). The abscissa represents the normalized time (in [s]). The ordinate is the standard deviation per DOF (in [rad]). On the right are the same figures for the hypothesis movement direction (upper plot) and movement extent (lower plot) computed from the model data (sec ). The second movement group is characterized by a flat NGEV profile for the complete movement whereas the GEV profile varies in time as for the other movements (fig and fig. 3.15). The model captures well the general features of the UCM signature for all movements. The reason for these features is thus probably to be found in the physical property of the effector. For both the EPH and VEPH muscle model, the UCM hypothesis and the UCM signature can be easily accounted for with multiple parameter settings. This success supports the general structure assumed for the movement generator, in particular, its task-oriented organization. This task decoupling is crucial to generate the right UCM signature. The contribution of each noise source to the overall effector variability can be examined in the model. The level and the correlation time of the noise sources can be varied to understand the formation of the UCM signature. In general, the UCM hypothesis (GEV higher than NGEV) and often the UCM signature (relative GEV/NGEV proportion) are very robust to all kind of noise changes but also to the presence or the absence of the backcoupling.

106 16 Results Experiment Model Time Figure 3.13: Joint variability decomposition per DOF for the movement 3 within the UCM and perpendicular to the UCM for the hypothesis movement direction (1,3,5) and movement extent (2,4,6). Three different subjects are shown (subject A: 1,2; subject B: 3,4; subject C: 5,6). On the right are the same figures computed from the model. See fig for the legends For the reference parameter setting in the model, the various noise sources contribute differently to the overall variability of the effector. Each noise source contribution can be tested in the model by setting its level to zero from movement onset. The noise source at the oscillator level is rather inefficient in perturbing the limit cycles. This noise will not be further examined. That does not mean that the virtual task trajectory in the joint space with respect to the task decoupling of the movement generator is not perturbed by the neuronal noise but only that the task specification in the limit cycle space is little influenced by the oscillator noise (equ. 2.23). The muscle noise also does not contribute much to the variability of the effector with the reference parameter values. Thus, suppressing the muscle noise does not change much the UCM signature. The muscle noise level is simply too weak in the reference parameter setting to affect the effector variability. Suppressing the neuronal noise does not change qualitatively the UCM signature. That changes however quantitatively the UCM signature. The starting position uncertainty of the end-effector contributes to increasing NGEVs at the beginning of the movement. Thus, with the reference parameter values, the observed variability of the effector during the movement comes mainly from the neuronal noise. We conclude that, within our framework, the UCM signature can emerge from the neuronal noise only, independently of the muscle noise.

107 3.3. VARIABILITY 17 Experiment Model Time Figure 3.14: Joint variability decomposition per DOF for the movement 4 within the UCM and perpendicular to the UCM for the hypothesis movement direction (1,3,5) and movement extent (2,4,6). Three different subjects are shown (subject A: 1,2; subject B: 3,4; subject C: 5,6). On the right are the same figures computed from the model. See fig for the legends. Various noise conditions can be examined in the model in order to understand the formation of the UCM signature. Each case is examined in comparison to the reference parameter values. In order to reproduce the effector variability at movement onset between the different noise conditions, noise sources are set to zero from movement onset and until the end of the simulation. The effect of the end-effector variability and the joint configuration variability at movement onset is examined separately. Various relevant cases are presented below. The effect on the UCM signature of the end-effector variability at movement onset is compared to the effect of the neuronal noise during the movement in the model. With end-effector starting uncertainty but no neuronal noise from movement onset, NGEVs are higher at the beginning of the movement than at the end of the movement for both UCM hypotheses. Thus, for the reference parameter values, the neuronal noise induces effector variability during the movement which is reflected in NGEVs at the end of the movement. Without variability of the starting end-effector position and with neuronal noise throughout the movement, NGEVs are higher at the end of the movement than at the beginning which confirms the statement above. GEVs are not strongly affected by these noise manipulations. So without neuronal noise, the UCM signature is unchanged but for smaller NGEVs at the end of the move-

108 18 Results Experiment Model Time Figure 3.15: Joint variability decomposition per DOF for the movement 6 within the UCM and perpendicular to the UCM for the hypothesis movement direction (1,3,5) and movement extent (2,4,6). Three different subjects are shown (subject A: 1,2; subject B: 3,4; subject C: 5,6). On the right are the same figures computed from the model. See fig for the legends. ment. The UCM hypothesis does not depend on the neuronal noise during the movement. That means that a right pre-movement UCM hypothesis is sufficient to generate a UCM signature during the movement independently from the neuronal noise during the movement. The experimental data show a more or less equal value for NGEVs at the end and at the beginning of the movement which points to an end-effector variability at movement onset (see also fig. 3.4) and neuronal noise during the movement. The effect of the variability of the end-effector position at movement onset on the UCM signature is examined while all other noise sources are set to zero from the beginning of the simulations in the model. When the only variability source is the starting end-effector position variability during the complete movement simulation, the UCM hypothesis is logically wrong at the beginning of the movement. The starting end-effector position uncertainty induces variability in the end-effector position but also variability in the joint configuration at movement onset. By chance, this variability is divided equivalently between the UCM and the task subspace perpendicular to the UCM. All variability components decrease at the end of the movement compared to the beginning of the movement because the target position and the effector during the movement are not affected by noise.

109 3.3. VARIABILITY 19 The effect of increasing the muscle noise level on the UCM signature is examined in the model. An increased muscle noise level generates a right UCM signature, qualitatively and quantitatively, even if the backcoupling is not present or if the neuronal noise level is suppressed from movement onset (fig and fig. 3.17). Thus, this does not support the idea that the neuronal noise is essential for the UCM signature but a strong neuronal noise is a sufficient condition to generate the UCM signature. This is also true for the muscle noise. So there is so far no constraints from the UCM signature to determine the relative contribution of the noise sources to the total movement variability. Both noise sources allow to generate the UCM signature, alone or in combinations, given the appropriate backcoupling constant. The effect of increasing the neuronal noise level further on the UCM signature is examined in the model. Increasing the neuronal noise level from.8 in the reference parameter setting to 2.5 increases both GEVs and NGEVs. The reason for this increase is that neuronal noise increases joint variability throughout the movement and that there is no end-effector fine positioning at the end of the movement with respect to the paradigm Don t correct for target error at the end of the movement (equ. 2.24). If there is such a fine positioning at the end of the movement, NGEVs decrease more at the end of the movement. With this higher neuronal noise level, the UCM hypothesis is still right. Thus, a high neuronal noise cannot destroy the UCM hypothesis. In general, the UCM hypothesis is robust to various muscle and neuronal noise levels. The UCM hypothesis does not depend on modelling assumptions. This is an essential aspect of the model. The following tests have been performed in the model. The UCM signature is robust to the muscle models, i.e. the EPH joint muscle model or the VEPH joint muscle model. A purely diagonal impedance matrix (equ. 2.38) does not change the UCM signature significantly. A decrease of the muscle noise correlation time or an increase of the neuronal noise correlation time does not change the UCM signature (τ m :.2 to.2; τ λ :.2 to.5). Movement time (tested for ±25% of the reference velocity) and the backcoupling delays (varied from to τ =.2) do not destroy the UCM signature. When the backcoupling delays are set to zero however, the effect of the backcoupling at the end of the movement is limited for the reference backcoupling constant (sec ). The effect of the time normalization on the UCM signature is examined in the model for the reference parameter setting. The time normalization is required in experiments to compute the mean trajectory between the trials. The subjects cannot control the movement time precisely enough and thus the movement time varies considerably among the trials. In the model, the movement time can be controlled precisely. It is possible to compute the mean

110 11 Results.1 Movement 1 Movement GEVdir GEVext GEVdir GEVext Time Figure 3.16: The muscle noise level is increased to.2 throughout the simulation and the starting end-effector uncertainty is set to zero. From movement onset, furthermore, the neuronal noise level and the backcoupling constant are set to zero. The UCM signature is shown for movements 1 and 6. The black dotted line is NGEV for the hypothesis movement extent and the blue dashed line is NGEV for the hypothesis movement direction. Compare this figure to fig and fig NGEVs at the beginning of the movement are reduced (arrows) because of the exact starting end-effector position but the UCM signature is preserved without the backcoupling and without neuronal noise. The normalized time is in [s] and the variability in [rad]. trajectory independently of the time normalization using the real movement time. The time normalization suppresses any movement timing differences between the trials. When perturbations act along with the joint movement, perturbations are not resisted. On the other hand, perturbations that act against the movement are resisted. The results show that indeed the real-time UCM profiles are consistently different from the usual time normalized UCM profiles. In particular, the bell-shape of NGEV for the hypothesis movement extent is destroyed, like for the end-effector variability along the extent coordinate in the local coordinate system. NGEV for the hypothesis movement extent increases slightly and monotonically throughout the movement. The UCM hypothesis is however still right and the backcoupling effect on the effector variability is similar to the time normalized case (see below). It should be pointed out that a model with very stiff joint muscles will not show this strong time distortion because the effector sticks to the the joint virtual trajectory UCM and backcoupling This section examines the effect of the backcoupling with respect to movement performance and to the various noise sources. One motivation to model the

111 3.3. VARIABILITY 111 Reference High muscle noise.1.1 GEVdir.8.8 GEVext GEVdir GEVext Time Figure 3.17: The muscle noise is increased to.6 throughout the simulation for movement 1 while the other parameters are not changed in the model. The UCM signature is preserved although both GEVs and NGEVs increase at the middle of the movement. Note the increase of GEVs at the movement beginning (arrow). The black dotted line is NGEV for the hypothesis movement extent and the blue dashed line is NGEV for the hypothesis movement direction. NGEVs are similar at the beginning and at the end of the movement (arrow). Compare to the experimental results in fig The normalized time is in [s] and the variability in [rad]. UCM motor control strategy is to understand what are the advantages of this strategy. An improvement of the task performance is hypothesized to be the main benefit of the UCM motor control strategy. The backcoupling is thought to be essential in this role (sec ). The backcoupling may for instance reduce end-effector variability. The backcoupling hypothesis can be tested by setting the backcoupling constant to zero from movement onset. Before testing the hypothesis of an improvement of task performance, the function of the backcoupling in relationship to the various noise sources should be clarified. The model reveals that the UCM hypothesis (GEV bigger than NGEV) does not depend on the backcoupling. The right UCM signature (GEV/NGEV relative proportion) is obtained by adjusting the backcoupling constant and the noise levels but the backcoupling is not strictly necessary to get the right UCM signature (fig. 3.16). Three different ways to generate the UCM signature in the model are distinguished and discussed below : high neuronal noise / backcoupling, high muscle noise / backcoupling and muscle and neuronal noise / without backcoupling. The term low and high noise level are used in a breach of language to designate the level of the noise parameter values. Again, every parameter change is compared to the reference parameter setting. The virtual null space designates the null space of the task Jacobian of the

112 112 Results movement generator (sec and sec ). Task decoupling refers to the structure of the movement generator that is divided into a task space, the limit cycle space and its corresponding null space. Movement 1 Movement GEV dir GEV ext GEV dir GEV ext With backcoupling Time Time GEV dir GEV ext GEV dir GEV ext Without backcoupling Time Time Figure 3.18: This figure shows the effect of the backcoupling on the UCM signature in the high neuronal noise condition (reference parameter values). The backcoupling constant is set to zero from movement onset and referred to as without coupling. The backcoupling pulls down GEVs at the end of the movement (arrows). NGEVs are hardly affected by the backcoupling. GEVs are not affected at the beginning of the movement because at rest the system is stable. NGEVs are the black dotted lines for the hypothesis movement extent and the blue dashed line for the hypothesis movement direction. The normalized time is in [s] and the variability per DOF in [rad]. The effect of the backcoupling and a high neuronal noise on the UCM signature is examined in the model. With a high neuronal noise for the complete simulation time, the backcoupling reduces GEV (fig. 3.18). This effect is also present although smaller when the muscle noise is in addition increased to.6. Without backcoupling, the only deterministic contribution in the virtual null space depends on the time derivative of the basis vectors that span this null space which practically appears to be small (sec ). The null space of the virtual joint trajectory is thus essentially under the influence of the neuronal noise without backcoupling. This leads to a pure uncontrolled manifold for the

113 3.3. VARIABILITY 113 virtual joint trajectory. The neuronal noise induces random motion in the null space of the virtual trajectory and the variability accumulates in time like a diffusion process. This process is reflected at the effector level, independently from the time normalization procedure to compute the UCM, because of the task decoupling of the movement generator. With a high neuronal noise condition, how the backcoupling reduces the variability in the virtual null space is examined in the model. The backcoupling imposes a dynamics in the virtual null space that reduces the variability in that subspace. The backcoupling sets a vector field in the virtual null space whose equilibrium position is the effector state. In the virtual null space, the virtual trajectory is attracted to the effector state. This is not a close tracking of the effector because the backcoupling constants are too low (fig. 3.3). Neuronal noise perturbations are resisted when the perturbations drive the virtual joint trajectory away from the real joint trajectory but only in the null space. Thus, the backcoupling reduces the effect of the neuronal noise in the virtual null space if the neuronal noise dominates the variability of the effector. In that case, GEVs are reduced. The task decoupling of the movement generator is reflected in the variability structure of the real effector although the real effector is not task decoupled. This underlines the importance of the virtual trajectory property in the generation of the UCM signature. It also demonstrates that a task decoupling does not have to exist at the effector level, as we postulated in our model to obtain a UCM signature at the effector level. The effect on the UCM signature of the backcoupling and a high muscle noise is examined in the model. The backcoupling introduces variability in the virtual null space whose origin is the muscle noise. This effect is negligible with the reference parameter setting but explains why the UCM signature can also be amplified by the backcoupling given a proportionally low neuronal noise and a high muscle noise (fig. 3.19). This effect can be related to the effector state that is more variable with a high muscle noise. Because the vector field in the virtual null space is shaped by the equilibrium position set by the effector state, the force field structure is highly variable amid the trials with a high muscle noise. So the virtual null space dynamics imposes the virtual trajectory to track a variable equilibrium point. The variability in the virtual null space increases. This effect is valid only if the muscle noise is high enough. The effect on the UCM signature of the noise levels without backcoupling is examined in the model. Above is stated that the backcoupling introduces variability in the virtual null space or reduces the variability in the null space of the task Jacobian. It is also possible to generate a correct UCM signature given an appropriate muscle and neuronal noise level without backcoupling. In

114 114 Results Movement 1 Movement GEV dir GEV ext GEV dir GEV ext With backcoupling Time Time GEV dir GEV ext GEV dir GEV ext Without backcoupling Time Time Figure 3.19: The effect of the backcoupling on GEVs can be reversed when the muscle noise dominates the neuronal noise. In that case, the backcoupling introduces noise in the null space of the joint virtual trajectory (arrows). The plots show the UCM signature for movements 1 and 4 with and without backcoupling from movement onset. The plan noise is n ψλ =.5 and the muscle noise level is n ψm =.8 throughtout the movement. NGEV direction is the blue dashed-dotted line, NGEV extent is the black dashed line. The normalized time is in [s] and the variability per DOF in [rad]. that case, the neuronal noise induces the proper variability in the virtual null space to generate the UCM signature. To obtain, a correct UCM signature in this condition a correct pre-movement UCM signature is required. Otherwise, the UCM signature may be correct only at the end of the movement. It is worth to note that this UCM signature is a particular case with respect to the situations examined above. In addition in this condition, the UCM signature depends to some extent on the muscle properties, in particular the muscle stiffness. The important aspect is that the UCM signature can be generated without backcoupling. In conclusion, one can formally dwell on two UCM signature mechanisms that can be combined in various ways. The first mechanism that generates a UCM signature is based on a backcoupling and a high neuronal noise while the second mechanism is based on high muscle noise and various backcoupling constants. Both mechanisms rely on the task-oriented organization of

115 3.3. VARIABILITY 115 the movement generator. Thus, the UCM signature does not depend on a specific relative contribution of the neuronal noise and the muscle noise. The most important aspect is the dissociation that can be made between the UCM theory and the backcoupling. The UCM signature does not depend strictly on the backcoupling With backcoupling.12.1 Without backcoupling ext dir ext.2 dir Figure 3.2: This figure shows the effect of the backcoupling on the joint (label 1 to 4) and end-effector variability in the local coordinate system (see also fig. 3.9, fig. 3.1) for movement 1 in the high neuronal noise condition (reference parameter values). The end-effector variability is hardly affected by the backcoupling. The backcoupling, on the other hand, reduces significantly the joint variability at the end of the movement (arrows). The normalized time is in [s], the variability of the end-effector in [m] and the variability of the joints in [rad]. 2 Time Now that we understand the relationship between the backcoupling and the various noise sources, the effect of the UCM motor control strategy on movement variability can be examined. Although the backcoupling reduces GEVs when the neuronal noise is high enough, this reduction is rather small in comparison to the effect such noise may have on the effector dynamics (fig. 3.2). This is the reason why the backcoupling does not decrease the end-effector variability in this condition. This statement depends upon the fact that the effector is inherently stable. Otherwise, the stability of the effector depends on the backcoupling and, thus the backcoupling reduces the end-effector variability. The form of the backcoupling is examined in the model. The backcoupling is made up of two terms, a position and a velocity dependent term. Both terms contribute significantly to shape the UCM signature. When one of these terms is set to zero, GEVs increase in the high neuronal noise condition toward the end of the movement (not shown). The exact form of the backcoupling does not matter to generate the correct UCM signature when the backcoupling

116 116 Results contributes to shape the effector variability. A backcoupling in the virtual null space like α E θ also reduces GEVs at the end of the movement in the high neuronal noise condition. A backcoupling in the joint space, and not only in the null space of the task Jacobian, of the form α (λ θ) induces also a correct UCM signature given the appropriate noise levels. This demonstrates that the UCM signature is robust to the form of the backcoupling. Another hypothesis related to the UCM control strategy and the backcoupling is the reduction of energy measured as the square root of the sum of the individual joint muscle torque square throughout the movement. With the reference parameter setting, this energy is increased between 3% (movements 1 and 2) and 1% (movements 4), without the backcoupling from movement onset and for the rest of the simulation including the post-movement resting phase. With this particular setting, this effect is robust for all trajectories. The effect is however really small so that it cannot be taken as an interpretation for a role for the backcoupling. Moreover, inconsistencies on torque reduction for different muscle models, especially for very stiff EPH muscles, have been noted in the model. The fact that the backcoupling can be clearly dissociated from the UCM signature in light of the above results does not allow to relate this potential positive effect to the UCM Wrong UCM Hypothesis for the end-effector position In the previous section, we show how robust the UCM signature and the UCM hypothesis are to various noise levels and various backcoupling constants. One may wonder if the UCM hypothesis can be undone for some conditions. The previous section emphasizes the role of the task-oriented organization of the movement generator in the UCM signature. Thus, if the task-oriented organization of the movement generator is lost, the UCM signature should be destroyed. Secondly, a pre-movement UCM signature was shown to be sufficient to generate a UCM signature during movement even when the movement unfolds free from noise. If this pre-movement UCM does not exist, the UCM signature may not emerge during the movement. Both hypotheses are tested in the model. This section examines the effect on the UCM signature of perfectly reproduced configurations of the effector at movement onset between the trials. The starting effector configuration and end-effector position can be made perfectly identical from one trial to the next (at numerical precision) in the model (fig. 3.21). During the movement, the noise sources are active. The UCM hypothesis is wrong at the beginning of the movement but is right at the end of the movement. So, a UCM structure emerges during the movement although the starting configurations are perfectly reproduced. This UCM signature is

117 3.3. VARIABILITY 117 to be directly related to the structure of the movement generator. The UCM signature reflects the task-oriented organization of movements. If there is no such organization then the UCM should be wrong. To test this hypothesis, a strong damping is set in the virtual null space ( 3 E λ) instead of the backcoupling such that the null space is very stable. In this condition and for the reference parameter setting without backcoupling, the UCM hypothesis is destroyed for the hypothesis movement extent throughtout the movement and at the beginning and at the end of the movement for the hypothesis movement direction (fig. 3.22). This indicates that the UCM signature depends on the task-oriented organization of the movement generator. In this condition with the damping in the virtual null space, there is no deterioration of the end-effector variability or of the energy compared to the reference situation with the backcoupling. Thus, the UCM signature cannot be related to an improvement of the task performance in this case either..7.6 GEVdir.5.4 GEVext Time Figure 3.21: The starting effector configurations are perfectly reproduced between the trials for the movement 6. The UCM signature is not right anymore at the beginning of the movement (arrow). Note however that the right UCM signature develops as the movement unfolds. NGEV for the hypothesis movement extent is the black dotted line and NGEV for the hypothesis movement direction is the blue dashed line. Compare this figure to fig The noise levels are set according to the reference parameter setting. The time is in [s] and the variability in [rad].

118 118 Results Effects of the neuronal noise level on the UCM signature The neuronal noise stands for the brain activities that are not directly related to the movement but impinge on the neurons involved in the movement. It is expected that this noise level will vary depending on the task context. Indeed, significantly different brain states may occur in different tasks (see also Arieli, Sterkin, Grinvald, & Aersten 1996). The effect of an increased neuronal noise on the effector variability with respect to the UCM signature and the end-effector variability is examined. The neuronal noise is increased throughout the simulation. The effect of an increased neuronal noise on the end-effector variability is examined in the model. The variability of the end-effector increases as the movement unfolds in the high neuronal noise condition compared to the low neuronal noise condition. The neuronal noise engenders end-effector variability during the movement (fig. 3.23, upper graphics). This is, of course, not surprising as in the reference configuration the neuronal noise dominates the variability of the effector..1 Movement 1 Movement NGEV ext.8.6 NGEV ext.4 GEVdir.4 GEVdir.2 GEvext GEVext Time Figure 3.22: When the virtual null space is stabilized ( 3 E λ), the UCM signature is wrong. This figure shows the UCM signature for movements 1 and 6. The dotted black line is NGEV for the hypothesis movement extent and the blue dashed line is NGEV for the hypothesis movement direction. The noise levels are set according to the reference parameter setting. The time is in [s] and the variability per DOF in [rad]. The effect of an increased neuronal noise on the UCM signature and in relationship to the end-effector variability is examined in the model. The effect of an increased neuronal noise on the UCM signature is not predicted easily and certainly not matched directly to the changes observed for the end-effector variability. The induced changes in the ratio between GEV and NGEV are not easily guessed. The UCM hypothesis is however in any case conserved.

119 3.3. VARIABILITY 119 The ratio between GEV and NGEV is not invariant between both noise conditions (fig. 3.23). In this example, GEVs increase strongly at the beginning of the movement for the high neuronal noise condition which is translated into a higher GEV/NGEV ratio. At the end of the movement, NGEVs increase too which produces the same GEV/NGEV ratio as in the low noise condition. This inhomogeneity of the effect of an increased neuronal noise level is not surprising regarding nonlinearities and the stability difference between the virtual task space and its virtual null space. One concludes from this example that a change in the ratio between GEV and NGEV can result from a global change, in this example of brain background activity, independently from an effect related specifically to the task organization, for instance a channelling of noise into the subspace of the joint space. Low neuronal noise High neuronal noise ext ext end effector Variability Time 3 3 GEV/NGEV 2 1 ext ext Time Figure 3.23: The effect of two different neuronal noise levels is examined for the end-effector variability (top) and the ratio of GEV/NGEV for both movement hypotheses (bottom). The neuronal noise strength, n ψλ, is.8 (reference parameter value, left) and 2.5 (right). The noise level increase enhances the variability of the end-effector at the end of the movement. The effect on the UCM signature, plotted as the ratio between GEV and NGEV (solid line, movement extent and dashed line movement direction), is not to be predicted easily. At the beginning of the movement, the increased noise level is reflected in increased GEVs (arrows). At the end of the movement, the ratios GEV/NGEV for both noise conditions are of the same order of magnitude although the end-effector variability is increased. The number of trials is 3. The time is in [s], the end-effector variability in [m] and the ratio in [ ].

120 12 Results The UnControlled Manifold for the hypothesis centre of mass. The centre of mass (CM) is claimed to be an important variable that the CNS specifically care of to maintain equilibrium (Scholz et al. 2; Scholz et al. 21; Reisman et al. 22). Although in this pointing task experiment the subjects are seated and the torso is fixed, the centre of mass is examined in the sense of the UCM. One illustrative example of a UCM signature for the CM hypothesis is given in fig for the movement 4. The UCM hypothesis for the CM can be wrong at the beginning of the movement in both the model and the experiments and for the various movements but the UCM hypothesis is always right at the end of the movement for both the model and the experiments and for all movements tested. In our model, the CM is not accounted for in the movement generator as a task variable. However, the CM hypothesis is not so different from the hypothesis end-effector position (sec ). Because the movement generator is not structured for the task variable CM, these results for the CM indicate that a correct UCM signature occurs because of the similarity of the hypothesis. In other words, a correct UCM hypothesis does not necessarily mean a corresponding organization for the movement generator. Thus, these results hint at the possibility to measure UCM signature for variables that are not specified by a UCM motor control strategy. Experiment Model GEV GEV Figure 3.24: The centre of mass of the limb is thought to be a task variable to maintain equilibrium. The UCM for the hypothesis centre of mass is examined for the movement 4 in the model and in experiments. The UCM hypothesis is in a first approximation correct for the entire movement. In the model, the centre of mass is not accounted for as a task variable in the movement generator. The time is in [s] and the variability per DOF in [rad]. Time

121 3.4. SELFMOTION A 4 B 4 C Experiment Model Time Figure 3.25: The projection of the vector of the joint velocity on the UCM for the hypothesis end-effector position is the selfmotion or internal motion of the effector (red continuous line). The complement of that motion accounts for the movement of the end-effector toward the target (black dotted line). Each graphic shows this joint velocity decomposition for the movement 1 for the three subjects (A,B,C) and the model. The time is in [s] and the velocity in [rad/s]. 3.4 Selfmotion Selfmotion is a combination of joint motions that does not change the endeffector position (sec. 1.5 and sec ). The results for selfmotion are not normalized per DOF and anyway the number of DOFs is equal for both subspaces defined by the UCM analysis method in our model. The selfmotion time profile during the movement varies among the subjects and for the various movements (fig. 3.25, fig. 3.26, fig and fig. 3.28). Qualitatively, selfmotion is never negligible in comparison to the motion component that moves the end-effector. Because at rest the arm is motionless, selfmotion increases at the beginning of the movement and decreases at the end of the movement. Selfmotion is qualitatively not dependent on the arm configuration or the movement speed. In both the model and the experiments, selfmotion is of the same order of magnitude for the two configuration sets tested (movements 1 and 2 respectively movements 3 and 4) (not shown). In the model and in experiments, selfmotion does not depend qualitatively on movement time in comparison to the variations observed between the subjects (not shown). In the model, the cause for selfmotion is investigated. We hypothezised that selfmotion may come from the neuronal noise, from a planned selfmotion or from decoordination. Neuronal noise can generate selfmotion if the noise level is very high. In that case, neuronal noise induces quantitative system changes because of the nonlinear nature of the model. Planned selfmotion means motion in the virtual null space of the task Jacobian that fullfills any task requirement for specific effector configurations. For instance, the joint configuration could be constrained to stay as close as possible to a preferred joint configura-

122 122 Results A B Experiment C Model Time Figure 3.26: The selfmotion and the transport velocity component are shown for the movement 3. See fig for the legends. The time is in [s] and the velocity in [rad/s] A 4 B 4 C Experiment Model Time Figure 3.27: The selfmotion and the transport velocity component are shown for the movement 4. See fig for the legends. The time is in [s] and the velocity in [rad/s] A 4 B 4 C Experiment Model Time Figure 3.28: The selfmotion and the transport velocity component are shown for the movement 6. See fig for the legends. The time is in [s] and the velocity in [rad/s]. tion. By decoordination is meant that the virtual trajectory does not account for muscle nonlinearities, limb dynamics and interjoint muscles. Thus, a component of the joint velocity that results from the deviation of the real effector trajectory from the virtual joint trajectory may be tangent to the uncontrolled manifold.

123 3.4. SELFMOTION Reference Diagonal impedance Without coupling EPH model Mvt 1 Time Mvt Time Figure 3.29: This figure shows the effect of the muscle impedance, the backcoupling and the muscle model on the selfmotion (red solid line) for movements 1 and 4. The joint velocity component that moves the end-effector is shown in black dotted line. The off diagonal terms of the impedance matrix (Z, equ. 2.38) are set to zero from movement onset to test their contributions to selfmotion. The backcoupling is also set to zero from movement onset to test its contribution to selfmotion. The VEPH muscle model driven by λ and λ contributes the most to the selfmotion (compare the Reference to the EPH model). Each contribution to selfmotion can be linked to decoordination between the virtual trajectory and the real trajectory. The time is in [s] and the velocity in [rad/s]. A neuronal noise cause for selfmotion is examined in the model. Neuronal noise cannot account for selfmotion because of the trade-off between the GEV/NGEV ratio during movement and the size of the selfmotion. The backcoupling pulls down the noise-generated selfmotion providing that neuronal noise be strong enough to contribute to selfmotion. Thus, the backcoupling suppresses noiseinduced selfmotion (sec ). Moreover, the noise level must be unreasonably strong to induce selfmotion. This cause for selfmotion is thus not compatible with the UCM signature. A virtual planned selfmotion to account for selfmotion measured in experiments is examined in the model. Selfmotion can be generated by a specific contribution in the virtual null space or if the time derivative of the null space vectors increases (sec ). None of these hypotheses could find a justification in a biological context although a planned selfmotion cannot be rejected formally on this consideration alone. The backcoupling in the null space of the virtual trajectory does not account alone for selfmotion. The backcoupling

124 124 Results contributes to some extent to the selfmotion but a strong damping in the virtual null space, for instance, does not suppress selfmotion which indicates that in the model the source of selfmotion is not only the backcoupling (see also fig. 3.29). A decoordination cause for selfmotion is examined in the model. Decoordination means that the virtual trajectory coordination is lost during the execution of the movement. There are various possible sources for decoordination. One source for decoordination is an inhomogeneity between the joint muscles. Joint relative muscle strengths and joint relative muscle functions, i.e. the exponential gains (k nl, equ and equ. 2.4) are varied. Joint muscle inhomogeneity increases selfmotion because this enhances the absolute difference between the real and the virtual trajectory. Interjoint muscles, the off diagonal terms of the impedance matrix (Z, equ. 2.38), contribute also to selfmotion. Both proposals rely on strongly inhomogeneous joint muscles. Small muscle anisotropies do not generate significant selfmotion (fig. 3.29/EPH model). Very stiff joint muscles allow for the effector to track closely the virtual trajectory and thus no decoordination is generated. The VEPH muscle model produces significant selfmotion independently of the backcoupling or of the off diagonal terms (fig. 3.29). This selfmotion is attributed to decoordination because of the compliant joint muscles. When the selfmotion is caused by the low muscle impedance, the backcoupling as well as the off diagonal terms of the impedance matrix still shape the selfmotion time profile (fig. 3.29). The selfmotion generated in the model is remarkable given that impedance is not expected to be constant throughout the movement. Interindividual selfmotion variability in experiments could be accounted for by variable backcoupling constants and variable limb impedance. This cause for the selfmotion is interesting because it explicitly assumes that the virtual joint trajectory is not a mirror copy of the real effector trajectory. In the model, the real effector trajectory does not follow the joint virtual trajectory because the muscle impedance is too low. If this decoordination is the result of a specific mechanism, like a muscle strategy or tradeoff for instance, or reflects more directly the inherent structure of neuroanatomical elements involved in movements is not clear. In the model, multiple causes induce selfmotion although all these causes are linked to the effector decoordination (fig. 3.29). The classical EPH model does not comply always nicely with selfmotion generation, but this model can generate selfmotion if the muscles are more heterogeneous. In view of selfmotion origin, one should emphasize the very different relationship in terms of causes and effects between the selfmotion and the UCM signature. Although both bear in principle similarities, for instance the virtual null space, their features are essentially distinct. One is not the consequence of the other.

125 3.5. VIRTUAL UCM AND VIRTUAL SELFMOTION Virtual UCM and virtual selfmotion The UCM signature and the selfmotion observed at the effector level can also be examined for the virtual trajectory in the model. The virtual UCM signature and the virtual selfmotion are expected to be similar to the real effector UCM signature and respectively the real effector selfmotion because of the task decoupling of the movement generator. These virtual features are briefly presented in this section for the purpose of completeness. A right virtual UCM hypothesis is observed (fig. 3.3). The task space is more stable thus it is less variable. The virtual null space is less stable so it is more variable. This virtual UCM signature is not simply a copy of the real UCM signature, but its general features are very similar. For instance, a high muscle noise increases the virtual GEV through the backcoupling. The backcoupling reduces the virtual null space variability as for the real null space noise in a high neuronal noise condition. Some UCM signature features, like the flat or non flat NGEV for the hypothesis movement direction, are also observed at the virtual level which indicates that these may be related to the effector geometry rather than the effector dynamics. A correct virtual UCM hypothesis is a direct consequence of the task-oriented organization of the movement generator. The virtual selfmotion is also not negligible in comparison to the transport motion component (fig. 3.31). The virtual selfmotion time course is different from the real effector selfmotion. The virtual selfmotion comes essentially from the backcoupling and only a small constant residual virtual selfmotion exists without backcoupling. On the contrary, the real selfmotion time course is only slightly changed by the backcoupling and certainly not suppressed. An increased muscle impedance (EPH joint muscle model) reduces the virtual selfmotion. As for the virtual UCM signature, the virtual selfmotion is not really a surprising feature given the task decoupling postulated at the level of the movement generator and the backcoupling. Again these virtual features are just for the purpose of understanding and are not meant to necessarily exist in the neuronal substrate. 3.6 Motor equivalence Motor equivalence is the ability to use different task-equivalent effector configurations in response to external perturbations of the limb. Motor equivalence is presented in detail in section and in section 1.5. So only a brief reminder is given here. In order to account for the unavoidable task variable variability, the motor equivalence signature is defined by the proportion of the

126 126 Results Movement 3 Movement GEV dir.5.4 GEV dir.3.2 GEV ext.3.2 GEV ext Time Figure 3.3: The UCM signature is also observed for the movement generator. The virtual UCM signature for movements 3 and 6 are shown. This virtual signature is the consequence of the task-oriented organization of the movement generator. Some typical features observed at the effector level already exist at this level, for instance the flat NGEV for the hypothesis movement direction for movement 6 (blue dashed line). NGEV extent is the black dotted line. The simulations are performed with the reference parameter values. The time is in [s] and the variability per DOF in [rad] Movement 3 Movement Time Figure 3.31: The backcoupling generates selfmotion (solid red line) in the virtual joint trajectory. This virtual selfmotion is not similar to the real effector selfmotion. For all movements, it dominates toward the end of the movement. Movement 6 selfmotion is the most important relative to the transport motion. The motion that moves the end-effector is the black dashed line. The simulations are performed with the reference parameter values. The time is in [s] and the velocity in [rad/s]. difference between two effector configurations that lies in the UCM and its complementary task subspace. If the UCM projection dominates the component projected on the task subspace, the abundance of DOFs is exploited on purpose in reference to the task variable.

127 3.6. MOTOR EQUIVALENCE 127 In order to examine motor equivalence in the model, three cases are regarded: clamping a joint during the movement, clamping a joint for a short while and applying a constant force to the end-effector for a short while (sec ). The first case may be considered somewhat as artificial whereas the two other cases are more realistic. The simulations with and without backcoupling from movement onset are examined below in the model. Motor equivalence is examined in the model with backcoupling. The vector of the effector configuration difference lies mainly in the UCM for all perturbations tested (fig. 3.32). This indicates that task equivalent configurations are made use of in response to external perturbations. The general effect is similar for all three perturbations tested. The configuration change induced by the perturbation affects also the non task equivalent component until the end of the movement. This means that the end-effector path is not restored, after the perturbation vanishes, toward the unperturbed end-effector path. The motor equivalence signature and the UCM signature seem a priori identical features of the movement. This is true so far, in the UCM signature case, the muscle noise dominates the variability of the effector. When the neuronal noise dominates the variability of the effector, motor equivalence and UCM signatures should not be compared. For instance, the backcoupling would have opposite effect on the UCM signature and on the motor equivalence signature (see below; sec ). Motor equivalence is examined in the model without backcoupling. It is expected that the backcoupling plays a significant role in updating the movement generator when a perturbation is applied to the effector. The backcoupling does indeed contribute to motor equivalence (fig. 3.32). The effect of the backcoupling is particularly obvious for the last two conditions (B, C in fig. 3.32). In the joint clamping case (A in fig. 3.32), the configuration change lies in both subspaces defined by the UCM analysis method because perturbations are not specifically task decoupled. The effect of the backcoupling with respect to motor equivalence is examined. The force field generated by the joint muscle drives back the effector toward the joint virtual trajectory after the perturbation vanishes. With the backcoupling, there is a trace left by the perturbation in the virtual null space of the task Jacobian. The joint virtual trajectory is changed by the perturbation. This leads to different effector configurations. In addition, the end-effector path is changed because of the effector dynamics. Without the backcoupling, the joint muscle force drives the effector toward the virtual trajectory too after the perturbation vanishes. But because the virtual trajectory is not affected by the perturbation, the real effector tends to catch up the unperturbed joint virtual trajectory exactly the same way as in the unperturbed trial but for the

128 128 Results effector interjoint interactions. This leads to effector configuration difference that lies in the UCM space and its task subspace. The main effect of the backcoupling is thus an update of the virtual trajectory on the real effector state. The backcoupling, except for the joint clamping during the complete movement, does not reduce task variable error (see the NGEV-like component in fig. 3.32). In the joint clamping case during the full movement, the trajectory difference without backcoupling is mainly caused by the inability of the limb to keep on the timing of the virtual trajectory, i.e the real limb is delayed. This delay arises because the clamped joint does not move and the virtual trajectory is not updated on the effector state. Thus, especially in this clamping experiment, the backcoupling updates the plan so that the trajectory is still properly executed. The backcoupling acts to improve the task execution for this specific perturbation. Such positive effect is however not to be observed for the transient perturbations. Motor equivalence relies on the task-oriented organization of the virtual trajectory and the backcoupling. If the perturbation is completely corrected for at the effector level by the muscles, there is no trace of the perturbation left in the movement generator. If the perturbation is not completely corrected for at the muscle level, the backcoupling updates the virtual trajectory on the real effector state. The muscle elastic property permits to resist perturbations and, the stiffer are the muscles, the stronger is the resistance. Thus, very stiff joint muscles that track closely the virtual trajectory do not permit motor equivalence. Because motor equivalence exists (Scholz et al. 24), the stiffness of the muscle cannot be high which support our assumption of a low stiffness for the joint muscle (see also the selfmotion 3.4). The motor equivalence analysis is a good tool to examine various movement conditions. For instance, the terminal effector configuration change that is induced by a different movement speed can be examined in the sense of the UCM theory to extract the motor equivalence signature. When this configuration difference lies more in the UCM space than in the perpendicular space, the task-equivalent configurations inherent to redundancy are made used of. This analysis is performed for two different movement speeds in the model (fig. 3.33). The motor equivalence signature clearly shows that task-equivalent configurations are not made used of between the two task conditions. This result illustrates the principle of the method only, because the model does not account for a movement speed dependence of the virtual trajectory as well as the joint limited range of motion (sec. 4.3).

129 3.6. MOTOR EQUIVALENCE With backcoupling.5 Without backcoupling A Time B Time C Time Figure 3.32: This figure shows the motor equivalence signature for three different perturbations in the model (sec ). Motor equivalence is the use of the abundance of solutions offered by a redundant effector. A) The second joint is clamped from movement onset B) a transient clamping is applied to the second joint and C) a transient constant force is applied to the end-effector. For all three perturbations the configuration changes are dominated by motor equivalence with the backcoupling (red solid line). A smaller configurational change can be associated with end-effector error (black dot line). The backcoupling generates or at least amplifies the motor equivalence effect. In the first case the UCM linear hypothesis may be violated because the configuration deviations are large. The time is in [s] and motor equivalence in [rad].

130 13 Results y Velocity end Time Motor equivalence start Time x Figure 3.33: Motor equivalence is computed for the movement 1 achieved at two different speeds (ω = 16 respectively 14) in the model. The insets show the velocity profiles (upper left) and motor equivalence (lower right, the motor equivalence component red solid line). The configuration difference lies mainly in the task space which indicates the absence of motor equivalence. This is true for all movements tested. The simulations are performed without noise. The time is in [s], the velocity in [m/s], x,y in [m] and motor equivalence in [rad]. 3.7 Rhythmic movements The motivation for a limit cycle framework to specify the movement task is the possibility to generate rhythmic movements within the same framework. In the model, rhythmic movements are made up of a forward and a backward movements of the end-effector. Because a discrete movement is specified by a full limit cycle, one limit cycle is required for the forward motion and another one is required for the backward motion. In the model, rhythmic movements are thus generated by switching alternatively between two sets of oscillators (fig. 3.34) (sec ). The properties of these rhythmic movements with respect to perturbations have not been examined. The purpose for generating rhythmic movements in the model is to demonstrate the feasibility principle. The oscillators specify the end-effector velocity and not the end-effector position as usually assumed which may induce different responses to perturbations. One notable feature of the simulated rhythmic movements is the drift of the end-effector away from the initial starting position and from the initial target position (fig. 3.34). Interestingly, this drift has also been observed in experiments (Brown et al. 23). Brown & al. demonstrate that the drift comes from an accumulation of error while the movement is repeated. This is exactly the cause of the drift in the model. The virtual trajectory is constant while the real trajectory deviates from the desired trajectory because of the

131 3.8. OPTIMAL CONTROL 131 y.6 y x x Figure 3.34: The end-effector paths for two rhythmic movements are depicted in this figure. The end-effector paths drift from the original starting and target positions. The inset shows another example of a rhythmic movement for a different path and with more oscillations. x,y are in [m]. arm dynamics. The authors speculation on multiple controllers is however not corroborated by our model. 3.8 Optimal control A feedback stochastic optimal control model has been proposed to account for the variability structure of pointing movements (Todorov & Jordan 22; Todorov & Jordan 23; Todorov 1998). In brief, the model is restricted to a linear system with a time invariant plant model. The arm model is made up of 3 linear joints and 5 muscles moving in a 1 dimensional task space (Appendix B). An effector state estimate is provided by a Kalman filter (Grewal & Andrews 1993). A Bolza type cost function sums penalties for the terminal position, the terminal velocity and the terminal force and a Lagrange cost accounts for a penalty for the control signal (Stengel 1994). Within this framework, a CNS model of the effector dynamics is required. Todorov & al. assume furthermore that multiplicative noise proportional to the control

132 132 Results x 1 3 UCM.8 Selfmotion NGEV GEV s Time Figure 3.35: Within the framework of feedback stochastic optimal control, the UCM signature and the selfmotion (s) are examined. Both signatures show time profiles at odds with experimental data (sec ). The time is in [s], variability in [m] and selfmotion in [m/s]. signal perturbs the execution of the movement in addition to a noise source in the state measurement. This model, as well as the additive noise version of this model, are implemented for the purpose of comparison. The UCM is computed as described in the methods for the hypothesis end-effector position. The parameters are given in (Todorov & Jordan 22) except for w v which is increased at.5 instead of.1 because it gives better results. The weights of the various costs and the noise levels are varied to evaluate the model. The velocity profile is bell-shape for the reference parameter setting but does not need to be so for all parameters. The path is slightly curved with a curvature reversal with respect to the straight line segment between the starting position and the target position. The UCM signature and selfmotion for this model are shown in fig The UCM signature is not qualitatively correct. Varying the noise levels and the cost function weights do not lead to the correct signature. The UCM hypothesis is also not robust to every parameter setting and can be wrong (GEV < NGEV ) for the entire movement. GEV always increases monotonically as the movement unfolds except for cases like fig With additive noise, GEV can be nonmonotonic and decrease toward the end of the movement but it was not possible to obtain an incorrect UCM hypothesis at the middle of the movement (NGEV > GEV ) as observed in experiments for the hypothesis movement extent or end-effector position (not shown) (see fig. 3.12, fig. 3.13, fig and fig. 3.15). Usually, the UCM hypothesis is right throughout the movement. Selfmotion has not been examined in detail, but the time profile for the reference parameter setting does not match qualitatively the selfmotion observed in experiments. This can be related to some extent at least to the coarse arm model. This model is further discussed in section 4.8.

133 DISCUSSION 4.1 Movement generation in living systems Motor control in living systems is viewed traditionally within the conceptual framework of interconnected hierarchical or heterachical modules. Each module fullfills a specific function in the motor control process. The various motor control scheme differ in the structure, function, connectivity and coordinate system of these modules. Many models based on this framework enforce a strict division between movement planning and movement execution (Bullock & Grossberg 1988; Bullock et al. 1993; Bhushan & Shadmehr 1999; Tee et al. 24; Morasso 1981; Kawato 1999; Desmurget & Grafton 2; Loeb et al. 1999; Goodman & Gottlieb 1995). The planning process generates a trajectory in the joint space while the execution process executes the planned trajectory. This is the dominant thinking in motor control. In addition to this assumption on the organization of movement planning and execution, most motor control approaches neglects features of biological movements that severely constrain the motor control strategy. These features are variability, redundancy and the biological framework. Within a controlled experimental environment, the movement unfolding is different from one trial to the next but the task is always achieved. The paradox lies in the high variability of the trajectory in the task space or in the joint space and the simultaneous successfull achievement of the task. Classical approaches to motor control assume implicitly that variability betrays the inability of the system to be immune to internal perturbations. The motor control process cannot correct perfectly for errors induced by the noisy intrapersonal environment. Fortunately for the experimenter, the mean behavior reveals the true intention of the controller. Redundancy is a fundamental and widespread feature of living systems. Unlike many robots for which the number of mechanical DOFs reflects exactly the task dimension, living systems have more DOFs than the strictly necessary number of DOFs required to achieve the task. There is for each task an abundance of task equivalent solutions. Redundancy 133

134 134 Discussion is found at various levels of the organism: task, mechanical DOFs, muscles, muscle fibres, neurons,... The traditional approach to redundancy focuses on reducing the number of available DOFs by imposing new constraints on the task or on the organization of the DOFs (sec. 1.1). The developmental context of living systems and the objectives of and constraints on this development influence the internal organization of living systems. Comparing robots and humans emphasizes that the two systems are really different. Differences in flexibility, robustness, reproducibility, adaptability are conspicuous. The respective frameworks that constrain the functioning and the development are fundamentally different between humans and robots. What functions well for robots is not necessarily valuable for living systems. The framework used to describe living systems deserves to be precisely evaluated to delimit clearly its explanation power in the context of biology. This study addresses specifically these issues. Variability is dealt with at a neuronal level and a muscle level; redundancy is considered for an arm whose number of independent mechanical DOFs exceeds the task dimension and the biological context constrains the model framework and, in particular, the motor control strategy. A brief summary of the structure of the model is given below. Our model, motivated by biological observations, is based on an explicit analytical description that makes its functioning tractable. It should be stressed that any excessive detail for the sake of completeness can significantly impair comprehensiveness. The goal is clearly to retain simplicity to highlight fundamental principles. Mechanical redundancy of the effector is considered. More independent joints than the minimum number of DOFs imposed by the task are available to achieve the task. Variability is accounted for at a neuronal level and a muscle level with noise. These noise sources generate joint variability and task variable variability. The mechanics of the skeleton is faithfully captured by an articulated rigid body model whose joint rotation axes are perpendicular to the 2D plane of motion. The outcome of the neuronal activity that causes the movement is assimilated in a virtual trajectory. This assumption hampers our understanding of how the neuronal processes generate the movement. This is however not our goal to gain such an understanding of the neuronal substrate. The ensemble of neurons whose activity sustains the movement is referred to as the (virtual) movement generator. Muscles are grouped into joint ensembles and muscle actuation is considered within the Equilibrium Point Hypothesis (EPH) framework. These assumptions lead to an analytical transparent joint muscle model. The integrity of movement generation and execution is accounted for with reciprocal couplings. These reciprocal couplings are, from the effector to the movement generator, the backcoupling and, from the movement generator to the effector, the effector state coupling (fig. 4.2). The virtual trajectory yields to deviations of the real trajectory from

135 4.2. SITUATING THE MODEL IN A BROADER CONTEXT 135 the virtual trajectory. Reciprocally, the real trajectory yields to deviations of the virtual trajectory from the real trajectory. No specific account of perception is given. However, the sense given to the backcoupling from the effector to the movement generator must still be explained within the context of the UnControlled Manifold (UCM) theory. 4.2 Situating the model in a broader context Our model is not based on a hierarchical schema. A hierarchical principle for motor control is motivated by the brain division into distinctive functional regions. While a modular functionality can be highlighted anatomically for controlled experimental environments, the brain regions may overlap functionally more in a natural complex environment. Indeed, various experiments indicate that many functional overlappings exist between the anatomical brain regions (Doya 2). Hierarchy in a classical sense refers to unidirectional connectivity between functional modules that underline a series of ordered steps in the process of generating a movement. This structure is clearly lost in our framework with the postulated reciprocal couplings between the movement generator and the effector. The reciprocal couplings set a mutual compliance between the movement generator and the effector. Mutual couplings are also recognized for action and perception in which action influences perception and perception influences actions (Creem-Regehr et al. 24). Our movement generator can be understood as a supplementary or intermediary level in the action perception cycle (fig. 4.1). The movement generator integrates constraints from the environment during the movement. The structure of the model implies various time scales in the integration of environmental constraints. The backcoupling and the fast muscle response due to the elastic property of the muscles integrate at two different time scales effector perturbations. While the muscles correct for deviations of the limb, the backcoupling updates the movement generator on the effector state. This integration of external environmental constraints is a fundamental feature of the movement generator. The movement generator is a neuronal image of the effector on which movements are internally organized (fig. 4.1). To execute a task meaningfully, the movement generator must be calibrated on the real effector state (Lackner & Dizio 2). In addition to the integration of the effector state in the virtual trajectory, the movement generator integrates the task specification which is, in our model, set within a limit cycle framework. Recent experimental findings support this highly interconnected and integrative view for motor control, for instance the mirror and canonical neurons, the behavioral map in the motor cortex, the deafferented patients behavior

136 136 Discussion Action Movement generator Perception Situated agent Figure 4.1: A concept of behavior is shown upon which our framework is built. The model does not include a complete description of this schema. Notably perception is missing. This framework enforces strong interactions on usually considered separated fields. This approach is different from a hierarchical control system. in relationship to body awareness and body knowledge or the sensorimotor calibration (Kohler et al. 22; Kelso et al. 21; Graziano et al. 22; Paillard 1999; Lackner & Dizio 2). These results challenge a strict separation between the motor functions and the perceptual system at a behavioral level but also at a neurophysiological level. On the contrary, these experimental observations point to an integrative system for perception and action (Hommel et al. 21; Deneve et al. 21). Our framework is inspired by this integrative view of behavioral functions (sec. 2.1). Our model does not necessarily imply multiple coordinate transformations like a hierarchical system (Bullock & Grossberg 1988; Bullock et al. 1993). For instance, coordinate transformations are postulated from the eye coordinate system to the head coordinate system and then to the body coordinate system. These successive remappings of various sensory modalities and motor functions toward a common coordinate system can be avoided in a multiplexed representation (Deneve et al. 21). This multiplexed representation advocates a direct integration of various sensory and motor sources into a common reference frame. In our framework, the task is specified, assuming sensory information about the external world, within a limit cycle framework. The task realization involves to set the effector state according to the task specification. This approach is in agreement with a multiplexed space whose inputs are the sensory systems and whose outputs are the muscle states. This multiplexed space is similar in many aspects to our limit cycle space. Our model is restricted to a single task and a unique effector which allows to simplify greatly

137 4.2. SITUATING THE MODEL IN A BROADER CONTEXT 137 the structure of the movement generator but at the same time does not allow to fully explore its potential. In our model, the movement generator generates a virtual joint trajectory. Moreover, reciprocal couplings cause mutual interactions between the real and the virtual trajectory. These couplings are not similar to the forward and feedback connections in control systems. The feedback connection from the motor system allows to correct adequately the muscle neuronal inputs for errors between the virtual trajectory and the real trajectory. The feedback forms a close loop system that servocontrols the plan. Instead in our framework, the backcoupling acts directly on the virtual trajectory and makes the virtual trajectory compliant to the real trajectory. The feedforward connection activates the muscles. In our model, the virtual trajectory sets the muscle state. The muscle activity emerges from the muscle properties. Our model does not assume a perfect joint trajectory that has to be executed the best possible way. The reciprocal couplings between the effector and the movement generator emphasize a principle of close reciprocal interactions that force the coupled system to track the coupling system. The main effect of the reciprocal couplings is entrainment and compliance between the effector and the movement generator. This compliance emphasizes an approach that differs from traditional controllers (see also Williamson 1999). In classical approaches to motor control, the focus is on the selection of a trajectory and the execution of the selected trajectory (Flash & Hogan 1985; Loeb et al. 1999; Bhushan & Shadmehr 1999; Tee et al. 24). Experimentally, the trajectory quantification relies on the mean trajectory and its variability. This description is of course purely experimental because any movement does not have to be repeated consecutively in everyday life situation. This leads to the concept of the perfect trajectory toward which each movement is aimed to by the controller. The real movement is thus controlled to track the planned ideal trajectory. The inability to comply perfectly with this goal trajectory leads to variability amid the trials (fig. 3.4). The motor control approaches based on this concept fail usually to explain the high variability of the trajectories in humans and the simultaneous task success. By comparison, a robot reproduces perfectly each movement with minimum error; humans do not. Let s formulate an alternative view on biological movements: for each movement exists a unique specific virtual trajectory. This is then the essence of the high variability observed. The UnControlled Manifold theory rests on this hypothesis. Instead of a single trajectory, the movement generator is structured by a task equivalent subset of virtual joint trajectories from which one is finally realized. In our model, the virtual joint trajectory is not identical for each movement but depends on the intra- and extrapersonal environment. A perturbation at the effector level is integrated into the virtual joint trajectory

138 138 Discussion and leads to a different virtual joint trajectory in comparison to an unperturbed trial. The movement generator is structured by a vector field in the joint space which is task specific and thus anisotropic. This vector field, the noise and the external environment define a virtual joint trajectory on a trial basis. In fact, this vector field expresses the movement objectives because it set the movement generator structure to achieve the task without respect for specific effector configurations during the movement. That means that the movement objectives constrain the generation of movement while a specific effector state which is only the mean by the movement is achieved is unimportant. The vector field does not enforce a specific trajectory in the joint space as postulated for instance in (Morasso & Sanguineti 1995). Indeed, this vector field is not isotropic for all directions in the joint space but is weaker along the tangent of the uncontrolled manifold. Thus, a unique trajectory for a specific task does not exist in our model. A movement generator structured by the task is assumed, but the effector is not structured according to the task. The movement generator, a neuronal image of the effector that unfolds a virtual trajectory, is task decoupled. A task decoupling designates a specific organization that divides the joint space into a task space and an independent complementary space for which the parametrization is independent of the task (sec. 4.5). In our model, the real effector is not task decoupled in that sense but the joints are mutually interacting independently from the task. The joint interaction at the effector level comes from the physical links between the joints and the interjoint muscles (Hogan 1985b; Mussa-Ivaldi et al. 1985). In various approaches to motor control that account for redundant effectors, this task decoupling is not assumed, neither at the virtual or planning level nor at the effector level (Bullock & Grossberg 1988; Bullock et al. 1993; Morasso & Sanguineti 1995). This task decoupling allows to make use of multiple joint trajectories to achieve the goal. It is fundamental to point that although the task decoupling exists only at the neuronal level, this virtual task decoupling is reflected at the effector level in the UCM signature (sec. 4.5). In our framework, the virtual trajectory in the task space is unique but for the neuronal noise perturbations. A motor control strategy expressed within the framework of stochastic feedback optimal control assumes a subset of trajectories in the joint space and in the task space (Todorov & Jordan 22; Todorov 1998) (sec. 4.8). Within this framework, the behavioral goal is expressed by an abstract cost function and a movement time. Unlike our model, the subset of task space trajectories is thus rather large. In the joint space, the trajectory specification is similar to our model because a subset of trajectories rather than one single trajectory is available to achieve the goal. This model is discussed in detail in section 4.8.

139 4.3. TRAJECTORIES 139 Our approach is not necessarily incompatible with a principle of error corrections of the virtual trajectory in the task space, similar to error correction mechanisms postulated in classical motor control models (Bullock et al. 1993). These mechanisms of error compensation are not considered explicitly in this work but it is important to note that they are not conceptually rejected by our motor control strategy. In our framework, these error corrections should only exist if these error compensations are meaningful to the task. For instance, the virtual task trajectory could be modified according to visually perceived errors. The functioning of these corrections is beyond the scope of this study for visual perception is not integrated within our framework. Intuitively, correction in our framework means to specify another subset of virtual joint trajectories. From another viewpoint, it means to restructure the vector field of the movement generator according to the task error. Our model does not postulate a reduction of the number of DOFs of the effector. The usual approach to redundancy in motor control is to organize the DOFs into a smaller number of independent units (sec. 1.1). Grouping the DOFs into units sums up to accept redundancy as a problem and implies to reduce the number of DOFs available. In robotics, the usual strategy to deal with redundant robots consists equally of reducing the number of DOFs. This solution is well known for 1 or 2-DOF redundant robots and has also been applied for hyperredundant robots (Lewis & Maciejewski 1997; Kreutz-Delgado et al. 1992; Xia & Wang 21; Chirikjian & Burdick 1994). This redundancy approach cannot conform to our framework which states exactly the opposite. Redundancy exists for the task realization and task equivalent solutions are made use of (sec. 4.5 and sec. 4.6). From our perspective, the only component of movement that should be precisely set is the behavioral goal and this is what humans are doing so well. 4.3 Trajectories Discrete movements The form of the virtual trajectory is strongly debated nowadays (Gomi & Kawato 1996; Latash 1993; Ghafouri & Feldman 21). In our model, we postulated a straight task path between the current end-effector position and the target position. The central issue is about the contribution to movement of the limb elastic property. The limb elastic property is demonstrated experimentally but its exact role in movement generation is not understood (Mussa-Ivaldi et al. 1985; Won & Hogan 1995; Shadmehr et al. 1993; Bizzi et al. 1984; Gomi & Kawato 1996). Two views are opposed: the inverse dynamics hypothesis and the EPH theory (sec. 1.3). The EPH theory relies on the limb elastic force field to generate movement. The virtual trajectory modulates the structure of the elastic force field in the joint space,

140 14 Discussion in particular the equilibrium point position of this field. Given low muscle impedance, a simple task virtual trajectory within the EPH framework cannot account for the interaction torques and thus leads to very curved end-effector paths (Gomi & Kawato 1996). This impedance consideration rejects a simple virtual trajectory but does not address the role of the elastic force field in movement generation. The alternative force control hypothesis assumes an inverse dynamics causal model (Wolpert & Kawato 1998; Kawato 1999; Jordan & Rumelhart 1992) (sec ). The limb and muscle forces have to be computed exactly which leads to a complex virtual trajectory (sec. 4.4) (Gomi & Kawato 1996). Our low stiffness VEPH muscle model does not support the hypothesis of a simple virtual trajectory like postulated in the EPH theory. Furthermore with the VEPH muscle model, the movement is not strongly constrained by the joint elastic field because the joint stiffness is too low. The effector motion is mainly dictated by the viscous force field property (equ. 2.4) (Lussanet et al. 22). This is certainly a distinctive feature of our model compared to the classical EPH models (Gribble et al. 1998; Mussa Ivaldi et al. 1988). Our model does not support an inverse dynamics approach or a mechanical decoupling approach either because our virtual trajectory does not account for interaction torques (Kawato 199; Khatib 1995). Despite this simplification, the end-effector paths are well accounted for in our model in comparison to the experimental data. In our model, the joint state is set by an elastic field and a viscosity field generated by the joint muscles (equ. 2.4). The elastic field property of the effector suggests the EPH theory as a basic principle for movement generation (Feldman 1965; Mussa Ivaldi et al. 1988). Although the elastic property of the limb is hardly refutable, its role during movement is not necessarily to set the effector in motion (Todorov 24). The elastic property of the limb may support a mechanism that stabilizes the limb against external perturbation during movement independently from movement generation (Won & Hogan 1995). This implies that the elastic field property must vary as the movement unfolds. Some experiments point to an elastic force field that cannot account for movement generation. Non-equifinality of movement end point following perturbations of the arm indicates that the muscle elastic property alone cannot compensate for all mechanical perturbations (Dizio & Lackner 1995; Hinder & Milner 23). Non-equifinality means that the perturbed limb does not reach the same final end position of the unperturbed limb. Equifinality is predicted by the EPH theory. Again these violations do not indicate that the elastic field does not exist but the elastic field may simply not be the source of movement generation. Our model does not use the limb elastic field to generate movement but the elastic field of the limb does not either stabilize the limb along its path during movement (sec. 3.6 and sec. 4.4). The causes for the end-effector path curvature are thought to be multiple.

141 4.3. TRAJECTORIES 141 In our model, the path curvature results from the inability of the effector to track the virtual trajectory closely. The rather straight path of the end-effector observed in experiments motivates task space planning approach or are claimed to reflect biomechanical constraints. For instance, it is claimed that the small curvature of the end-effector path represents an ideal trade off trajectory that minimizes joint torques (Uno et al. 1989). It is then surprising to observe that altered perception of the end-effector path provokes an adaptation of the end-effector path curvature (Flangan & Rao 1995; Wolpert et al. 1995). Subjects are also able to produce a straighter end-effector path when the visually perceived path curvature is artificially increased (Wolpert et al. 1995). Thus, visual perception distortion of the end-effector path could account for the end-effector path curvature. This assumption is not supported by a study that shows that the end-effector path curvature depends on the use of the right or left arm (Boessenkool et al. 1998). Learning is also evoked to explain path curvatures. The arm manipulability can bias the learning of the endeffector/effector kinematic map towards the direction of the bigger axis of the manipulability ellipse (Barreca & Guenther 21). The direction of the movement rotates toward the direction of the main axis of the manipulability. Within our framework, the virtual trajectory does not account for the interaction torques or the muscle properties and thus the real end-effector deviates from the virtual trajectory. The interjoint interactions caused by the muscles contribute to the path curvature as well as shown in a previous study (Flash 1987). The end-effector path curvature appears to be a trade off between multiple constraints that encompass the motor system, the perceptual system and possibly learning. Because learning and perception are not included in our model a full account of the end-effector path curvature is not possible. In our model, the virtual task trajectory is assumed to be invariant for various workspace positions and various movement times. The virtual task trajectory is only scaled appropriately to generate various movements. This assumption explains the variance of the real end-effector path for various movement times (fig. 3.6). Indeed, the interaction torques increase as the joint speeds increase. Speed invariant end-effector paths would indicate that the arm dynamics is accounted for, at least, partly in the virtual trajectory. Speed invariant endeffector paths are reported for discrete movements in 2D (Boessenkool et al. 1998; Nishikawa et al. 1999; Messier et al. 23). However, in a discrete movement experiment with a 1 DOFs arm-body effector moving in 3D, the end-effector path curvature varies as the movement speed varies (J. P. Scholz, pers. com.). No speed invariance for the end-effector path curvature is also reported for movements that involve the full body (Pozzo et al. 22). The discordance of these results indicates that speed invariance is not an objective of movement realization. However, these results suggest too that speed may be a parameter that constrains the virtual trajectory. The virtual trajectory then becomes speed dependent and speed is an explicit constraint in the move-

142 142 Discussion ment generator. Modulating limb impedance could also contribute, to some extent at least, to preserve path invariant features. A decoupling of trajectory planning and impedance modulation has been proposed to account for various environments (Mussa Ivaldi et al. 1988). Alltogether these pieces of evidence support a qualitative model of the world rather than an explicit inverse dynamics computation (Ostry & Feldman 23; Gribble & Ostry 2). The speed dependence of the virtual trajectory is not accounted for in our model which does not lead to a speed invariant end-effector path. Movement time should also acts upon the task equivalent subset of virtual joint trajectories. Indeed, joint excursions do not simply scale in time when movement speed is varied (Thomas et al. 23). For various movement speeds, the task equivalent subset of virtual joint trajectories is made used of differently. Thus, speed should also be a constraint on the virtual joint trajectory subset. Intuitively, this could be achieved with an end-effector/effector mapping that depends on movement speed. Our results suggest that the virtual trajectory needs not to be isomorphic to the real trajectory. The virtual trajectory is not closely tracked, in agreement with the EPH theory in the context of external forces (Ghafouri & Feldman 21) (sec. 1.3). If external forces were to be considered, these could only be compensated for by a difference between the virtual position and the real position to produce the same movement kinematics as without external forces. The discrepancy between the real and the virtual trajectory unfortunately does not guarantee the stability of the end-effector along the movement path (Won & Hogan 1995). In our model, and despite the VEPH joint muscle model, the real joint trajectory lags behind the virtual joint trajectory which does not guarantee the stability of the end-effector path after a perturbation because the equilibrium point is determined by the virtual trajectory (fig. 3.32). In the next paragraph, we examine how the stability of the end-effector path could be set during the movement. In the EPH framework, movement generation is parametrized by the joint positions. This implies an elastic force field to generate movement. The EPH model assumes that the only stable state of the effector corresponds to the virtual trajectory in the absence of external forces. Thus, the virtual trajectory that defines a moving equilibrium point of the force field is tracked during the movement by the real effector but low stiffness leads to an important lag between the real trajectory and the virtual trajectory. When a perturbation is applied to the real effector such as to decrease the difference between the real and the virtual path, the effector does not resist the perturbation. This is also true for the end-effector. This property is at odds with the measured stability of the end-effector path during movement (Won & Hogan 1995). Won & Hogan demonstrate that the real end-effector path acts like an equilibrium

143 4.4. MUSCLE MODEL AND MUSCLE IMPEDANCE 143 point during movement in the sense that the real limb resists transient perturbations. This stable path corresponds to a minimum of the elastic potential field of the limb. The corresponding virtual trajectory that generates the real effector equilibrium path is not isomorphic to the real trajectory (Hodgson & Hogan 2). Both definitions of the virtual trajectory are conceptually very different. The EPH virtual trajectory does not account for the stability property of the real end-effector path. And thus our model fails to capture the limb stability property during the movement (sec. 4.4). At rest, the limb resists external perturbations because the virtual and the real position are identical. A different parametrization of the virtual trajectory could account for the end-effector path stability during the movement. This parametrization should account for the mechanics of the effector Rhythmic movements Within our framework, discrete movements and rhythmic movements are closely related. Indeed, discrete movements are generated using limit cycles. A forward motion covers one complete limit cycle period. Our framework allows to generate rhythmic movements with alternating sets of oscillators, for respectively, the forward motion and the backward motion. In general, the relationship between rhythmic and discrete movements is not well known. Evidence from evolution points to an evolution of discrete movements from rhythmic movements. Cortical imaging shows, for instance, that for discrete movements several additional cortical areas are activated in addition to the activated areas for rhythmic movements (Schaal et al. 24). The authors claim that rhythmic movements cannot be part of a more general discrete movement system. So rhythmic movements are more general. Our oscillator framework is thus compatible with this idea. It has also been proposed that the extension from rhythmic and straight discrete movements to any movement can be accommodated within an oscillator framework (Sternad & Schaal 1999)(see also Williamson 1999). Because our movement generator is set within a limit cycle framework, the limit cycle property of movement may be potentially accounted for. This certainly should be investigated in future work. 4.4 Muscle model and muscle impedance Our approach is based on muscles organized around the mechanical joints. The joint muscle model captures an actuator at each joint. In addition, an interjoint muscle relationship is included in the impedance matrix and is assumed constant during the movement (sec ). With our VEPH joint muscle model, low stiffness values are compensated for by a velocity-based muscle activation. This model is comprehensive but have also some limitations. Muscle model and muscle impedance are briefly examined in this section to situate

144 144 Discussion our joint muscle model within a more global context. In our model, the muscle impedance has been simplified. Only the elastic and the viscous component of the muscle impedance, the stiffness and the viscosity, have been considered (Mussa-Ivaldi et al. 1985; Hogan 199; Kelso & Holt 198). In our model, the muscle elastic hypothesis is based on end-effector stiffness measurements for a supposed constant neuronal input to the muscle. The muscle force depends also on the neuronal input to the muscle by definition. Within the EPH framework, muscle force dependence on the variation of the neuronal input for constant muscle length is accounted for by the parallel invariant characteristics. Given that muscle length and muscle neuronal input can vary simultaneously, the force dependence on simultaneous changes of muscle length and neuronal inputs should be considered if these are not independent. A linear muscle model that fullfills this requirement is for instance the bi-linear muscle model (Hogan 199). Thus, our joint muscle model is restricted to the lower order feature of the impedance. In our model, the variation of the stiffness is not considered (Smith 1996; Hogan 1985b). Muscle stiffness and, by extension, joint stiffness depend on various parameters. Our nonlinear joint muscle accounts for the dependence of stiffness on muscle torques but otherwise in our model the muscle impedance is assumed constant (Feldman 1965; Houk et al. 22; Smith 1996). For instance, the joint torques do not depend on the absolute muscle length. This property makes sense when muscle attachment to the skeleton is considered. Moreover, the torque dependence on joint velocity is not modelled and the CNS modulation of the joint stiffness is not considered (Houk et al. 22; Gribble et al. 1998). In addition to the stiffness modulation, the skeleton admittance is assumed perfect. In other words, the joint muscle transmitts perfectly torque to the skeleton. The joint model is also an approximation of the action of an ensemble of muscles that acts at the joint. For instance, the moment of arm that determines the effectiveness of the muscle force to rotate the articulation varies with the muscle length (Houk et al. 22). Thus, some muscle properties are not captured in our joint muscle model. Providing these simplifications the mean end-effector path and the mean effector velocity are well reproduced (sec. 3.2). The time profiles of the variability of the joints and the end-effector are also remarkably reproduced in light of the complexity of the real limb (sec ). Our strategy to neglect some properties of the muscle does not impair qualitatively the movement features in the sense that these are well captured. One feature often underappreciated of a spring-like muscle is damping. A spring muscle without damping oscillates but damping resists movement. A strong muscle damping thus induces the effector to reach slowly its final state

145 4.4. MUSCLE MODEL AND MUSCLE IMPEDANCE 145 whereas a weak damping generates oscillations of the effector. A passive damping is sufficient to make the effector stable. In our VEPH joint muscle model, the damping takes the form of a complex nonlinear function. In addition, the neuronal input to the muscle have been parametrized with the virtual position but also with the virtual velocity. This parametrization of the muscle neuronal input is arbitrary and should not be related to neurophysiological elements. A simple consideration of our joint muscle model emphasizes this point. The VEPH muscle model can be approximated locally by a linear model (see also Hogan 1984) T = K (θ λ) µ ( θ λ) = K θ µ θ+λ(λ, λ) = K (θ Λ k ) µ θ (4.1) where Λ stands for the virtual torque, Λ k stands for an equivalent virtual trajectory and µ θ stands for the damping. The virtual torque time course is complex and not isomorphic to the real torque time course. Equation 4.1 emphasizes one important aspect of our VEPH joint muscle model. It differs only on the time course of the virtual trajectory compared to an EPH model that assumes a simple trajectory and high muscle stiffness. Our VEPH joint muscle model assumes a low stiffness in agreement with experimental data (Tsuji et al. 1995; Gomi & Osu 1998; Gomi & Kawato 1996). In the VEPH joint muscle model, the real arm velocity tracks with some delays the virtual velocity. So there is in addition to a weak position control also a weak velocity control. The term control is abusive because the stiffness and the viscosity are too low to track the virtual trajectory. In robotics, velocity control is proposed instead of position control for some specific tasks that do not require a precise timing (Li & Horowitz 1999). The analogy between both systems is however limited because the viscosity gain (µ) of the muscle is presumably too small. If the viscosity gain is larger, the virtual velocity could be tracked closely by the muscle. There is to our knowledge no experimental data to support a high gain for a velocity tracking. Our modelling strategy does not focus particularly on the end-effector impedance although, in various environments or for other tasks, the impedance of the endeffector is crucial (Lacquaniti et al. 1992; Hogan 1985b; Hogan 199). A fundamental property of the limb is the stability of the end-effector during movement and at rest (Won & Hogan 1995; Tsuji et al. 1995; Shadmehr et al. 1993; Gomi & Osu 1998; Hogan 1985b; Mussa-Ivaldi et al. 1985). This stability refers to external perturbations. Our model of end-effector stability is derived at rest from the joint and interjoint property for constant internal variables but the model is not focused on a specific end-effector impedance, because in pointing tasks, the features of the end-effector impedance are not so fundamental as for task involving object interactions.

146 146 Discussion We should clearly state that the mechanical stability of the end-effector is different from the notion of stability referring to the UCM theory. The stability property of the end-effector quoted above specifically emphasizes the stability in relationship to external perturbations (Won & Hogan 1995; Kelso & Holt 198). In our framework, the effector stability is studied in reference to internal perturbations at the muscle level and at the neuronal level. In our model, the stability of the joints in multijoint movements is analyzed with respect to the task, while in the studies referred to above, either the end-effector stability or the joint stability in single joint movement is considered for a given position. This is a fundamental difference. In particular, for movements without external perturbations, a neuronal coordinative structure that preserves the task variables induces the UCM signature at the effector level (sec. 4.5). From our model, it is shown that the UCM stability structure emerges from a task decoupling of the virtual trajectory while such decoupling does not have to exist at the effector level. If such a decoupling was also to be found at the mechanical level, it could reinforce the UCM signature. If a UCM structure at the muscle level exists and can be accommodated with the effector stability property is not known experimentally but would necessitate a complex muscle organization (Scholz et al. 2). In the model, the stability property of the effector and the end-effector after a transient perturbations is reflected in the motor equivalence signature (sec. 3.6 and sec. 4.6). Without backcoupling, the end-effector trajectory does not return to the unperturbed trajectory after the perturbation vanishes and the difference between the perturbed and the unperturbed trajectory lies mainly in the task space. With backcoupling, the end-effector trajectory does not return to the unperturbed trajectory after the perturbation vanishes either but the difference between the perturbed and the unperturbed trajectory lies mainly in the UCM. This clearly indicates that the path of the end-effector is not stable. This comes from the fact that the movement is induced, according to the EPH theory, by a (large) difference between the real and the virtual position (sec. 4.3). In our VEPH joint muscle model, another internal variable is introduced to parametrize the muscle input: λ (Feldman & Levin 1995). This internal variable affects the real trajectory. As equation 4.1 shows, this internal variable does not have to exist if the virtual trajectory adopts a complex time course. Moreover, this internal variable is not another independent DOF to set the muscle state because λ depends uniquevocally on λ. Unfortunately there is no experimental data to support a precise set of internal variables. In addition to λ and λ, a second independent control variable sets the muscle state, the cocontraction command. The cocontraction command is assumed to be constant throughout the movement in our model (sec ). The exact function of the cocontraction command is not fully understood. Increasing the cocontraction increases the slope of the invariant characteristic, i.e the

147 4.5. THE UNCONTROLLED MANIFOLD 147 muscle is stiffer. The cocontraction command sets the muscle impedance during the movement (Feldman 1966; Feldman & Levin 1995). Alternatively, the cocontraction command has been suggested to be directly related to the end-effector variability (Gribble et al. 23; Osu et al. 24). A higher cocontraction may reduce the variability of the end-effector at the target either the end-point scattering and end-point target error (Gribble et al. 23) or the end-point target error alone (Osu et al. 24). Both studies examine single joint movements. Cocontraction measurements are based on agonist and antagonist muscle EMGs. EMGs are directly related to muscle forces so when agonist and antagonist muscles are activated simultaneously the joint stiffness increases. Increasing joint stiffness in the model does not reduce end-point scattering errors. The target error is not tested because movement end-point termination is dictated by the Do not correct for target error paradigm. When a terminal movement end-point position is set in the model, this position acts like an equilibrium point and reduces the end-point variability in any case. Thus, increasing cocontraction per se does not affect variability in the model. This finding is supported by (Osu et al. 24). In this study, a larger variability in EMG activity, torques and joint positions but a reduced end-point target error and a constant end-point scattering error is found while cocontraction increases. The fact that the torque variability increases while at the same time the end-point scattering error is constant appears at odds. This paradox may point to a UCM-like muscle organization that compensates for muscle errors to preserve the task variable. Theoretically, increasing the end-effector stiffness toward the end of the movement can reduce the endpoint target error because the real trajectory deviates less from the virtual trajectory. Learning paradigms support also a cocontraction strategy to reduce variability in unknown environments (Burdet et al. 21; Shadmehr & Mussa-Ivaldi 1994). In this sense, increasing cocontraction decreases the effect of extrapersonal perturbations. 4.5 The UnControlled Manifold The UCM concept defines a theory for a motor control strategy and a method to unravel the structure of the effector variability. The UCM signature is computed according to the UCM method to discover a specific UCM motor control strategy (sec ). The model allows to investigate the relationship between the UCM motor control strategy and the UCM signature. In particular, the cause and the nature of the UCM signature are investigated. Furthermore, attempts are made to discover the advantages that such a motor control strategy confers. In the next paragraphs, a brief reminder of the UCM method, the UCM theory and the backcoupling function in the model are given before discussing the results.

148 148 Discussion The variability of the joints and the variability of the end-effector have been compared to unravel a movement strategy related to the task space or to the joint space. This comparison does not make sense because the variability is contrasted between two spaces whose dimensions and physical units are different. Only within a common space can a comparison be meaningful. The UnControlled Manifold (UCM) method allows to compare variability in the task-redundant space of the joints with respect to the task. The joint variability is decomposed into a component that lies in a task equivalent subspace and a component that lies in a non task equivalent subspace. Only the variability within the non task equivalent subspace matters because the variability in the task equivalent subspace does not induce task variable error by definition. If a UCM motor control strategy exists, an appropriate variability signature in the joint space should characterize the task variables examined. This UCM signature is a high variability within the UCM and a relatively low variability in the subspace perpendicular to the UCM. A UCM variability signature is inferred only for variables that the CNS specifically organized for the successful accomplishment of the task in a redundant system. The UCM signature for various tasks and task variables have been found experimentally in numerous tasks (Scholz & Schöner 1999; Scholz et al. 2; Scholz et al. 21; Reisman et al. 22; Tseng et al. 23; Tseng & Scholz 24; Scholz et al. 22). The UCM theory emphasizes a tolerant motor control strategy rather than a strategy that selects and enforces a unique planned trajectory in the redundant effector space. The UCM is fundamentally oriented to the task and the successful realization of the task. The cause for the UCM signature is postulated to be embedded in the neuronal network that generates movements, i.e the movement generator. The UCM signature reveals that the UCM motor control strategy enforces in the joint space any task equivalent trajectory. The task equivalent trajectories form a subset of trajectories, a manifold, from which any can be executed. This task equivalent trajectory subset is indifferent to the success of the task variable. This is the uncontrolled manifold (Schöner 1995). The trajectory subset is made use of because of the intrinsic variability of the organism and because it exists explicitly. The use and existence of the task equivalent space is reflected in the UCM signature. In the model, the virtual trajectory drives the joints of the effector by establishing a force field in the effector space. Reciprocally, the backcoupling from the effector to the movement generator drives the virtual trajectory toward the real trajectory providing that the virtual task variable is unaffected (fig. 4.2). The backcoupling sets a vector field in the null space of the movement generator. The strength of the vector field in the null space of the virtual trajectory is related to the difference between the real and the virtual trajectory. This difference between the real and the virtual trajectory depends on

149 4.5. THE UNCONTROLLED MANIFOLD 149 the effector dynamics, the interjoint muscles, the muscle impedance and the backcoupling delays. Our results in the model clearly point for the cause for the UCM signature to the organization of the movement generator. At this neuronal level, the UCM emerges. This organization of the movement generator is defined by the UCM motor control strategy. This motor control strategy emphasizes a task specific vector field in the neuronal image of the effector. This vector field is not isotropic for all directions in the virtual joint space but specifically preserves the task variables. This means that task equivalent solutions in the virtual joint space exist and are tolerated. Our simulations show that this specific task-oriented organization is necessary to generate the right UCM signature. When a unique trajectory is specified in the neuronal image of the effector, the UCM signature does not emerge. In addition, a specific organization that preserves the task variable, i.e a task decoupling does not have to exist at the effector level for the UCM signature to emerge. The structure of the taskoriented brain processes is reflected in the variability signature of the effector although the effector is not structured according to the task variables. The task decoupling of the movement generator is a sufficient condition to generate the UCM signature (sec ). The UCM signature does not depend on specific noise and backcoupling assumptions. Our results show that the UCM hypothesis is the consequence of movement generation and does not depend on movement execution or muscle constraints (Scholz & Schöner 1999). The UCM signature for the end-effector position hypothesis is qualitatively and quantitatively well reproduced in the model for all movements tested. The general time course of the UCM signature emerge naturally. So the time course of the UCM signature can be attributed to the effector dynamics and kinematics. The effect of the backcoupling on the UCM signature depends on the relative noise levels between the muscle noise and the neuronal noise. The backcoupling can either enhance the variability in the UCM or decrease the variability in the UCM. Moreover, the UCM signature does not depend strictly on the backcoupling. Thus, the UCM signature cannot unequivocally be associated with the backcoupling. The backcoupling however is necessary for the motor equivalence signature (sec. 4.6). The UCM signature is also robust to various noise levels (sec ). Thus, based on the UCM signature alone, it is not possible to find out which noise sources contribute the most to the total movement variability. In addition to the noise during the movement, our results point to the importance of the effector variability at movement onset to shape the UCM signature. There is in the literature, to our knowledge, no experimental evidence for the respective contribution of the noise sources to the global movement variability. It has been estimated that constant execution noise in addition to a small contribu-

150 15 Discussion Movement generator interactions effectorstate coupling backcoupling Effector Situated agents Figure 4.2: A conceptual representation of the model is shown. The virtual trajectory emerges from neuronal activity. This neuronal activity leads to muscle state changes that form a force field in the joint space. Reciprocally, the effector state is coupled back into the movement generator. Thus, the effector and the movement generator interact intimately. The movement generator is a neuronal image of the real effector. tion of multiplicative execution noise dominates the total movement variability (van Beers et al. 24). Execution noise refers to neuronal noise, more precisely to motorneuron noise. In our framework, it means, in a coarse analogy, that the muscle noise would be dominant. On the other hand, imaging studies attest of the high variability of the cortical neuronal activity (Arieli et al. 1996). This brain variability cannot however be interpreted directly with respect to the effector variability. These contradictory indications do not allow to conclude in favour of one or the other hypothesis. However, because the UCM signature is robust to all noise conditions, this lack of evidence does not call into questions our results. Our simulations with a perfectly reproducible effector configuration at movement onset indicate that the UCM hypothesis at the beginning of the movement is an artifact of the variability of the effector configuration at movement onset. These data in fact demonstrate that the UCM motor control strategy is already effective while the arm configuration is set before the movement starts. In addition, the UCM signature is shaped by the noise during the movement. The UCM motor control strategy does not provide an immunity for the task variables to internal noise during the movement. When the neuronal noise level is increased, the virtual joint variability is globally increased. The neuronal noise induced variability is however less resisted within the UCM than perpendicular to it. The task variable variability still increases. Thus, the