Haptic Discrimination of Fields and Surfaces

Transcription

1 Haptic Discrimination of Fields and Surfaces by Vikram S. Chib Submitted to the Department of Biomedical Engineering in partial fulfillment of the requirements for the degree of Masters of Science in Biomedical Engineering at NORTHWESTERN UNIVERSITY July 2003 Northwestern University 2003

2 Abstract The ability to discriminate an object s shape and mechanical properties from touch is one of the most fundamental somatosensory functions. When exploring mechanical properties of an object, such as stiffness, humans probe the surface and obtain information from the many sensory receptors in their upper limbs. This sensory information is critical for the guidance of actions. For example, when navigating a room in total darkness, one may run his or her hand along the surface of a wall while moving through the room to obtain information about movement direction and any obstacles in one s path. Experiments were performed to reveal how humans acquire information about the shape and mechanical properties of surfaces through touch and how this information affects the execution of trajectories over the surface. These experiments involved subjects executing trajectories while holding a planar manipulandum that renders virtual surfaces with variable shape and mechanical properties. Subjects were instructed to make reaching movements with the hand between points laying on the boundary of a curved virtual surface of varying stiffness (200, 400, 800, 1200, 1600, 2000 N/m). It was found that subjects trajectory adaptation was dependent on surface stiffness. When a surface exceeded a threshold stiffness, subjects adapted by learning to produce a smooth trajectory on the surface, while at lower stiffness they adapted by recovering their original kinematic pattern of movement in free space. This adaptation suggests the internal representation of two distinct categories through a continuum of force fields: force perturbations and object boundaries. In the first case, the interaction forces are 2

3 opposed and the trajectory is restored. In the second case, the trajectory is modified so as to reduce the interaction forces. 3

4 Acknowledgements First of all, I d like to thank my advisors, Sandro Mussa-Ivaldi and Kevin Lynch. Their continuous guidance and support was a calming influence throughout the course of this project. I d also like to thank the members of LIMS and the SMPP Robotics Lab. Special thanks go to Chris Mah for valuable discussions regarding bootstrap statistics. Many thanks also go to Santiago Acosta for his patience and insight during our lively discussions of my experiments. Most importantly, I d like to thank Jim Patton. Without his direction and encouragement these experiments would not have been possible. Last but not least, I d like to thank my family for always being there for me. 4

5 Contents Background Kinematics and Internal Representation of Movement A Sensory-Motor Adaptation Paradigm Learning of Object Mechanics Overview of Thesis Experimental Methods System Description Virtual Surfaces Experimental Protocol Analysis Area Reaching Deviation Jerk Interface Force Psychometric Fitting and Bootstrap Confidence Intervals Results and Discussion Low Stiffness Field Adaptation High Stiffness Field Adaptation A Possible Interaction Strategy The Effect of Interaction Speed on Surface Perception Conclusions Key Findings Potential Future Work Bibliography Appendix A: Learning Analysis Appendix B: Manipulandum Force Field Program Appendix C: Matlab Analysis Programs

6 Chapter 1 Background The ability to discriminate an object s mechanical properties from touch is one of the most fundamental somatosensory functions. When manually exploring the mechanical properties of an object, such as stiffness, viscosity, and hysteresis, humans probe the object and obtain information from the many sensory receptors in their upper limbs. This sensory information is implicitly incorporated into actions. For example, when navigating a room in total darkness, one may run one s hand along the surface of a wall while moving through the room, so as to obtain a general idea of the direction of movement and of the location of any obstacles along the path. While this is a relatively simple task for healthy individuals, a loss or reduction in the ability to interact with objects safely and effectively may take place as a consequence of a neurological disorder. This thesis presents experiments that were performed to reveal how healthy humans acquire information about the shape and mechanical properties of surfaces through touch and how this information affects the execution of hand movements over the surface. Experiments involved subjects executing hand movements while holding a device that renders surfaces with variable shape and mechanical properties. It was hypothesized that 6

7 the adaptation to object mechanics will vary depending on surface stiffness. Particularly, we expect that when a surface exceeds a stiffness threshold, subjects will classify the virtual surface as an object boundary and adapt by learning to produce a smooth trajectory on the surface. In contrast, at lower stiffness they will classify the surface as a force disturbance and will adapt by recovering their original kinematic pattern of movement in free space. Further, it was hypothesized that repeated movements of the hand over a curved surface would lead to an adaptive process and that this adaptation would reveal, through the change in movement trajectories, an autonomic and implicit learning of surface geometry. 1.1 Kinematics and Internal Representation of Movement Kinematics, motor control, and learning of arm movement have been studied extensively (Morasso 1981; Flash and Hogan 1985; Conditt et al. 1997; Matsuoka 1998). These analyses are generally constrained to planar movement of the arm in order to simplify the numerous redundancies inherent in the musculoskeletal system. Seminal experiments of motor control and kinematics of arm movement were performed by Morasso. His observations showed that when executing goal directed reaching arm movements humans have a tendency to move the hand in a straight path, with a symmetric, bell-shaped, velocity profile (Morasso 1981). This finding is intriguing since it implies that subjects may implement rather complex patterns of segmental coordination to preserve straight line movement of the hand. Hogan and Flash introduced an objective function that could account for the observed kinematics of 7

8 the arm in single joint and multiple joint movements (Flash and Hogan 1985). They proposed that the motor system strives to achieve maximally smooth endpoint movements. Smoothness of these point to point movements was quantified by the square of the magnitude of jerk (i.e. the first temporal derivative of acceleration). The tendency to generate smooth and straight movements of the hand suggests that humans have an internal representation of the geometric structure of the space in which they move, and the motor commands necessary to reach desired locations in this space. It is believed that these internal representations are achieved through the experience of moving through free space (Atkeson 1989). It is thought that this representation is defined in humans as either transformations from motor commands to the resulting behaviors, or as transformations from desired actions to the corresponding motor commands. Representations which map motor commands to desired behaviors are called forward models, while those which map desired actions to the corresponding motor commands are called inverse models. System level models of motor control can be designed from behavior observed during psychophysical experiments. During these experiments subjects are given detailed instruction regarding movement execution in a specific dynamic environment. In order to execute a movement the CNS must signal a motor command to the muscles. The inverse model is a proposed way to calculate this motor command from knowledge of a desired movement (Shadmehr and Mussa-Ivaldi 1994). This model requires full knowledge of limb kinematics, which results in a rather complex calculation. 8

9 Forward models have been proposed as way to compensate for feedback delays in the nervous system. While inverse models generate the feedforward component of a motor command, they do not incorporate sensory information which is subject to large time delays and still necessary for movement. For fast arm movements, these time delays range from ms (Hollerbach and Flash 1982). It has been observed that movement corrections are made before sensory information is available from afferents in the spinal cord. To compensate for the time delays of sensory feedback it has been proposed that a system is present that is able to predict future sensory feedback information from knowledge of efferent signals (Jordan and Rumelhart 1992). This is the crux of the forward model. 1.2 A Sensory-Motor Adaptation Paradigm Numerous studies have been performed to reveal the internal model of the controlled dynamics of movement (Shadmehr and Mussa-Ivaldi 1994; Gandolfo et al. 1996; Matsuoka 1998; Thoroughman and Shadmehr 2000). A common means of studying these models is the use of an instrumented manipulandum capable of applying deterministic perturbing forces to the arm movements of subjects. These motor adaptation studies often impart a deterministic field of velocity dependent forces to a subject s hand as he grasps the instrumented manipulandum. Using this experimental paradigm, it has been shown that when exposed to a velocity dependent field arm movements are first distorted and after repeated practice the subject adapts to 9

10 the perturbing field and the initial kinematics of arm movement are recovered (Shadmehr and Mussa-Ivaldi 1994). This adaptation is further manifested by suddenly removing the field after a training period in the field. In this sudden null-field condition after-effects of adaptation are visible as mirror images of the initial perturbation. This after-effect adaption displays the persistence of the altered kinematic map. It has been concluded that this adaptation is a result of the central nervous system s formation of an internal model that maps between the positions and velocities experienced during the training period and their corresponding forces (Conditt et al. 1997). Further, this adaptation is a form of implicit learning (Seger 1994), which takes place at a nearly subconscious level with minimal attention or awareness of what has been learned. Evidence of this implicit learning are seen in subject after-effects, which occur without subjects making a conscious effort to produce motor commands counteracting the force field. 1.3 Learning of Object Mechanics Information about the mechanical properties of an object is acquired through active haptic discrimination, or active touch. During haptic discrimination the sensory receptors used to discriminate mechanical properties of objects can be divided into two functional classes: kinesthetic (subcutaneous) and tactile (cutaneous) sensors (Johansson and Westling 1984). Kinesthetic information refers to geometric, kinetic, and force information about the limbs, such as position and velocity of joints, actuation forces, and joint interaction forces. These signals are mediated by sensory receptors in muscles, articular cartilage, and tendons. Tactile or cutaneous information refers to pressure and 10

11 indentation distributions over the skin. These signals are mediated by mechanoreceptors innervating the dermis and epidermis (Squire et al. 2002). During active haptic discrimination individuals acquire and process information from these sensory receptors in order to learn object features. A large part of the human sensory cortex is dedicated to the processing and consolidation of information acquired during haptic discrimination using the hand (Squire et al. 2002). A number of studies have been performed to understand the perception of shape through active touch. Kappers et al found that humans are capable of learning and distinguishing slight differences in the shape index and curvature of various surfaces (Kappers et al. 1994). Investigation of actively touched curved surfaces has shown that adaptation and after-effects are present following haptic exploration (Vogels et al. 1996). These haptic after-effects are manifested as flat surfaces being judged as convex following the touching of a concave surface, and flat surfaces being judged as concave following the touching of a convex surface. Haptic after-effects were found to increase with the time of contact with the curved surface, until effect saturation occurred, and decrease with the time lapse between the touching of the first surface and the next (Vogels et al. 2001). Psychophysical studies have also been performed to determine thresholds for stiffness discrimination. Using a contralateral limb matching procedure in which subjects adjusted the stiffness of a motor connected to one arm until it was perceived to be the same as that 11

12 connected to the other arm, Jones and Hunter determined that the sensitivity to stiffness discrimination is less than the sensitivities for force discrimination and displacement discrimination (Jones and Hunter 1990). This poor sensitivity to stiffness discrimination was much worse than would be expected by combining the sensitivities for force and displacement discrimination. Studies of manual discrimination of stiffness by Srinivasan and LaMotte found dissimilarity between the just noticeable differences for compliant objects with deformable surfaces and rigid surfaces where compliance is coming from a compliant attachment (Srinivasan and LaMotte 1995). They concluded that when a surface is deformable, the spatial pressure distribution within the contact region between the hand and the surface is dependent on object compliance, thus information from cutaneous mechanoreceptors is sufficient for discrimination of subtle differences in compliance. When the surface is rigid, kinesthetic information is necessary for discrimination, and the discriminability is much poorer than that for objects with deformable surfaces. 1.4 Overview of Thesis This thesis will present the results of experiments that were performed to understand how information about the shape and mechanical properties of surfaces is acquired through active touch and how the internal representation of surface mechanics affects the execution of hand movements in contact with a surface. The experiments presented here were inspired by motor learning and adaptation studies, described in Section 1.2, which use an instrumented planar manipulandum to impart systematic force fields during movement execution. 12

13 Constraining surfaces and objects that are encountered during daily activity can be thought of as position dependent force fields, exerting reaction forces that prevent or limit motions inside the surface boundary. With this in mind, elastic virtual surfaces of varying stiffness were programmed into the workspace of the manipulandum. During testing subjects were instructed to make goal directed movements in the presence of the virtual object/force field. In order to understand how mechanical properties and shape are learned, adaptation and learning were observed during the execution these movements. Chapter 2 will present details regarding the programming of virtual surfaces and the experimental protocol. Chapter 3 will present the results and analysis of these experiments in the context of learning and internal representation of surfaces and their mechanical properties. 13

14 Chapter 2 Experimental Methods 2.1 System Description All experiments were performed using a two degree of freedom planar manipulandum as seen in Figure 2.1. Subjects made goal-directed movements in the plane of the manipulandum while grasping its handle. The manipulandum was similar to those described in previous motor adaptation experiments (Shadmehr and Mussa-Ivaldi 1994; Mussa-Ivaldi and Bizzi 1999). It was instrumented with positional encoders which record angular position of the manipulandum links. The resolution of the encoders is 20 arcsec per rotation (Teledyne Gurley, model 25/045-NB17-TA-PPA-QAR1S). Position and velocity of the manipulandum handle were computed from these encoder signals. These signals were sampled at a frequency of 100 Hz. The manipulandum was also equipped with two torque motors (PMI Motor Technologies, model JR24M4M4CH) that were used to generate the appropriate virtual surfaces. Endpoint kinetics were acquired using a six degree of freedom load cell fixed to the handle of the robot. 14

15 2 cm 8 cm 10 cm y Figure 2.1 Manipulandum with Dimensions of Virtual Surface. During testing subjects grasped the handle of the manipulandum using their dominant hand. Subjects did not have a direct view of their hand while interacting with the manipulandum handle, as it was covered by a screen. Movements of the manipulandum handle were presented via a projection system. Using this system, a cursor was displayed on the screen covering the subjects arm. The movement of this cursor was registered to the movement of the manipulandum handle. In this way subjects were presented with visual feedback of their hand movement. 15

16 2.2 Virtual Surfaces Object Boundary O r R Figure 2.2 Schematic of Virtual Surface of Radius (R) Centered at the Origin (O). The force fields experienced by subjects occurred radially, and were defined by the following formula. F( r) = K( R r) + B r 0 r R r > R (2.1) This expression defines a circular, elastic, virtual surface centered at O of radius R. The interface force produced when contacting the virtual surface was proportional to the stiffness of the object K and the displacement into the virtual surface r. A component of damping B was incorporated into this formulation to alleviate instabilities encountered at higher stiffnesses. This is a technique commonly imparted when programming virtual surfaces (Colgate and Brown 1994). Values of the stiffness and damping coefficients can be seen in Table

17 Inertial compensation was added to the dynamics of the manipulandum so as to negate the effect of the manipulandum mass and inertia. Stiffness Coefficient Damping Coefficient K (N/m) B (N/m s) Table 2.1 Stiffness and Damping Coefficients. 2.3 Experimental Protocol During testing subjects were seated in a pneumatic chair and raised to a position such that the height of their shoulder was approximately that of the manipulandum. The forearm of the subject was constrained to the plane of the manipulandum using a planar harness. This arrangement restricted arm movement of the subject entirely to the plane of the manipulandum. The subject was oriented such that the center of rotation of his shoulder was in line with the axis of rotation of the manipulandum motors. Experiments consisted of subjects making goal directed reaching movements from a start target to a goal target. During a given trial a target was projected onto the subject s workspace and the subject was asked to make one continuous movement to place a cursor registered to the manipulandum handle within the target, while achieving a desired maximum velocity. The next target appeared after the subject held the cursor at the prior target for one second. Subjects were given feedback if they moved faster or slower than 17

18 the desired maximum velocity. The optimal speed was specified prior to each experiment. When subjects achieved a maximum velocity more than 5% faster than the desired maximum velocity the target turned green. If the target was reached more than 5% slower than the maximum desired velocity the target turned blue. When the target was reached within the desired velocity window the target was animated to explode and a loud squawking noise was presented as a reward. These feedback cues allowed subjects to match their movement speed to the optimal speed and to achieve a consistent speed for all trajectories. Prior to the introduction of force fields, subjects practiced making point to point movements under the required velocity constraints, in the absence of a virtual surface, for 60 movements. In order to assess the typical performance of the subject, undisturbed in free space, surfaces were not introduced during this baseline unperturbed phase. This phase of the experiment allowed subjects to familiarize themselves with the unperturbed dynamics of the manipulandum. Following the baseline unperturbed phase, virtual surfaces were presented to the subject. The dimensions of the virtual surface can be seen in Figure 2.1. A testing phase consisted of the subject moving between targets located on the boundary of the virtual surface. Subjects made 100 reaching movements between the presented start and goal positions. The first 50 movements of a testing phase served as a learning period for the subject to acquire a motor adaptation to the field. During the final 50 movements of the phase catch trials were introduced pseudorandomly for 12.5% of the movements. These catch trials, movements during which no force field was present, were introduced in order 18

19 to reveal any motor adaptations that may have occurred after learning of the virtual surface. After completion of the phase consisting of 100 movements while in contact with the virtual surface was completed a phase consisting of 50 movements in the presence of a null field was introduced. This phase was meant to allow for deadaption and unlearning of the motor adaptation encountered during the previous phase of movement. This protocol was repeated for all stiffness levels. The stiffness levels were presented in order of increasing magnitude. A schematic of the testing protocol can be seen in Figure Virtual Surface Stiffness (N/m) Trial Number Figure 2.3 Experimental Protocol. This protocol was repeated for two different targeted velocities. The first experiment involved subjects moving at a fast speed (0.4 m/s) while the second experiment involved subjects moving at a slower speed (0.2 m/s). The fast speed (0.4 m/s) was chosen to isolate subjects feed forward component of movement while the slower movement speed 19

20 was investigated to evaluate the effect of more sensory information on the discrimination of virtual surfaces. Nineteen naïve, normal, right-handed volunteers (age range years) participated in this study after giving informed consent in accordance with the standards of the Institutional Review Board of Northwestern University. The first experimental group (0.4 m/s) consisted of 9 subjects while the second experimental group (0.2 m/s) consisted of 10 subjects. 2.4 Analysis Several measures were used to quantify subjects response to virtual surfaces and their subsequent motor learning. These trajectory measures and other data analysis techniques are described in the following sections Area Reaching Deviation In order to evaluate a subjects deviation from a straight line path, due to the exposure of a virtual surface, the measure of area reaching deviation was used. This measure was defined as the area between the trial trajectory and a reference straight line path between the start and goal positions. Area reaching deviation is the spatial average of the lateral deviation away from the reference straight line trajectory. Trajectories to the right of the reference straight line trajectory were given positive weight, while those to the left were given a negative weight. y f A = x( y) dy (2.2) y i 20

21 2.4.2 Jerk Flash and Hogan proposed that voluntary arm movements are executed with the objective of minimizing the square of the magnitude of jerk (Flash and Hogan 1985). This cost function, defined by Equation (2.3) essentially states that during arm movement humans strive to maximize smoothness of their endpoint. C = 1 2 t f 0 3 d x 3 dt d y + 3 dt dt (2.3) This hypothesis is called the minimum jerk hypothesis. The minimum jerk model is based solely on the kinematics of movement ignoring the dynamics of the musculoskeletal system. The minimum jerk hypothesis has been shown to fit simple continuous movements quite well Interface Force To determine the extent to which a subject contacted the virtual surface the measure of interface force was used. Force measures were acquired from the 6 degree of freedom force sensor affixed to the handle of the manipulandum. These force measures were integrated over the duration of the movement to acquire a resulting force cost (Equation 2.4) for an entire reaching movement. C = t f F( r, t) dt ti t f t i (2.4) 21

22 2.4.4 Psychometric Fitting and Bootstrap Confidence Intervals A common means of quantifying a subject s performance of a psychophysical task is the fitting of a psychometric function (Wichmann and Hill 2001). The psychometric function relates an observer s performance of a psychophysical task to some physical aspect of stimulus. For these experiments the performance metric used was the sign of the area reaching deviation. A three alternative forced choice paradigm (3-AFC) was implemented for the catch trials performed at each stiffness level. Catch trials having a negative area reaching deviation were classified as discrimination of a field while those having a positive area reaching deviation were classified as discrimination of an object boundary or surface. Subjects results were compiled into a single group measure for each stiffness level. This measure was expressed as the proportion of positive surface responses (in forced-choice paradigms) at each stiffness level. To model the process underlying the experimental data it is common to assume the number of correct responses in a given block of testing to be the sum of random samples from a Bernoulli process with a probability of success π. A model must then provide a psychometric function ψ(x), which specifies the relationship between the underlying probability of a positive response p and the stimulus intensity x. A frequently used general form is: ψ ( x; α, β, γ, λ) = γ + (1 γ λ) F( x; α, β ) (2.5) The shape of the curve is determined by the parameters {α, β, λ}, and the choice of a two-parameter function F, which is typically a sigmoid function. For these experiments a cumulative Gaussian was used for F. The function F was chosen to have a range [0, 1]. From the defined range it follows the parameter γ gives the lower bound of x, which can 22

23 be interpreted as the base rate of performance in the absence of a signal. For these experiments the base rate of performance was defined as zero, since in the case of a null field one would find that subjects have a zero probability of detecting a surface. The upper bound of the curve, the performance level for an arbitrarily large stimulus level, is given as 1- λ. For these experiments lambda was chosen as zero since it is was found that as stiffness increased subjects had a greater propensity to discriminate a surface. Between the two bounds the shape of the curve is determined by α and β. These two variables determine the displacement along the abscissa and the slope of the psychometric function. A bootstrap 95% confidence interval was computed for the resulting psychometric function. The bootstrap method is a Monte Carlo resampling technique relying on a large number of simulated repetitions of the original experiment. It is especially well suited for analysis of psychophysical data because its accuracy does not rely on a large number of trials, as do methods derived from conventional statistical asymptotic theory (Wichmann and Hill 2001). Psychophysical tests, similar to those performed, are limited in that few measures are required. During Monte Carlo techniques a large number of synthetic data sets are generated by resampling the collected data. During Monte Carlo simulations one takes a small sample repeatedly and constructs a distribution from this resampling. The bootstrap confidence interval for these experiments was obtained by sampling with replacement from the subject population. A histogram was constructed from the resulting mean and a 95% confidence interval over stiffness levels was obtained from this distribution. 23

24 Chapter 3 Results and Discussion 3.1 Low Stiffness Field Adaptation A typical set of movement trajectories in the presence of various field stiffnesses and for different stages of the field exposure (early exposure, mid exposure, and late exposure) for movement speeds of 0.4 m/s can be seen in Figure 3.1. For low stiffness surfaces (200 N/m, 400 N/m) in the early exposure phase it can be seen that the effect of the virtual surface on the hand trajectory of the subject was quite significant and could be divided into two parts. During the first part the hand was driven off course by the surface and forced away from the surface boundary. At the end of this first part of movement the field had caused the hand to veer off direction from the target before making a second movement back towards the target. The effect of these two segments of the trajectory appeared as a hook that was oriented clockwise or counterclockwise depending on the direction of movement with respect to the surface. 24

25 Stiffness Level K (N/m) Initial Field Exposure 1-6 Mid Field Exposure Late Field Exposure Catch Trials Following Learning Figure 3. Trajectories from various stages of adaption for a typical subject. Green squares represent the start position, red circles represent the goal position. 25

26 This is a similar result as to that found for hand trajectories prior to the adaptation of velocity dependent force fields (Shadmehr and Mussa-Ivaldi 1994). In studies of these adaptations it has been hypothesized that the hook shaped movements are corrective movements that are generated to compensate for errors caused by the unexpected field (Shadmehr and Mussa-Ivaldi 1994). It has been hypothesized that these corrective movements are consistent with the operation of a controller that is attempting to move the endpoint along a desired trajectory bringing it to a specified target position. Since the hypothesized controller uses muscle viscoelastic properties to define an attractor region about the desired trajectory, the hand is eventually brought back to vicinity close to the target position. The hooks result from the interaction of the viscoelastic properties of the muscles and the force field that perturbs the system from its desired trajectory. In Figure 3.1 it can be seen that after adaptation subjects produced straight-line movements through the field. This result is further manifested by looking at the pre and post adaptation area reaching deviations. Figure 3.2 shows that the area reaching deviations for pre and post adaptation for both 200 and 400 N/m were significantly different (p<0.05) after learning had taken place. Further the area reaching deviations were reduced to nearly zero, indicating that subjects were producing straight line movements. The ability to produce straight line movements after force field adaptation is indicative of subjects formation of an internal model to compensate for the dynamics encountered during force field interaction. 26

27 Area Area Reaching Deviation Error m/s Area Reaching Deviation (m^2) Area Reaching Error (m^2) p =0.00 p = 0.03 p = 0.16 p 0.51 p =0.08 p =0.72 Pre Post Stiffness (N/m) Figure 3.2 Area Reaching Deviation Pre and Post Adaptation Averaged For All Subjects. After effects from the low stiffness field adaptation show more proof of the development of internal models similar to those found during velocity dependent force field adaptation studies (Shadmehr and Mussa-Ivaldi 1994). These after affects are characterized as being the mirror image of the initial field exposure, and thus mirror images of the dynamics forces necessary to produce straight-line motions through the field. This suggests that in the presence of low stiffness position dependent force fields subjects nervous systems adapt by creating an internal model that approximates the dynamics of the environment. This model is used to predict and compensate for the forces imposed by the environment. 27

28 3 x Area (m 2 ) % CI Standard Deviation Stiffness (N/m) Figure 3.3 Trajectories and Area Reaching Deviation Averaged for All Subjects Catch Trials 3.2 High Stiffness Field Adaptation Adaptation to force fields of stiffness higher than 400 N/m (i.e. K = 800, 1200, 1600, 2000 N/m) resulted in a different outcome than generally seen during force field adaptation studies. A typical set of trajectories for these stiffness levels can be seen in Figure 3.1. As in the case of the low stiffness adaptation, initial exposure to the field resulted in a two segment movement with the first portion corresponding to the hand being perturbed off course by the surface and forced away from the surface boundary. At the end of this 28

29 first part of movement the field had caused the hand to veer off direction before making a second movement back towards the target. The effect of these two segments of the trajectory appeared as a hook that was oriented clockwise or counterclockwise depending on the direction of movement with respect to the surface. Unlike the adaptation seen with low stiffness fields, the higher stiffnesses adaptation did not result in subjects recovering straight line movements. Even at moderate stiffness (800 N/m) when subjects were capable of overpowering the force field to produce straight line movements, during mid and late exposure subjects produced movements that followed the curvature profile of the virtual surface. At stiffnesses greater than 400 N/m, after adaptation, subjects data showed statistically significant decreases in interface force and jerk (Figures 3.4, 3.5). This indicates that subjects reduced their penetration into the surface, and their resulting compensatory movements after adaptation were smooth over the surface. 29

30 Interface Force m/s Interface Force (N s) p = 0.63 p = 0.37 p = 0.08 p = p = 0.67 p = 0.04 Pre Post Stiffness (N/m) Figure 3.4 Interface Force Pre and Post Adaptation Averaged For All Subjects. Jerk m/s Jerk (m/s^3) p = 0.02 p = p = 0.22 p = 0.35 p = 0.28 p = Stiffness (N/m) Pre Post Figure 3.5 Jerk Pre and Post Adaptation Averaged For All Subjects. 30

31 After effects from catch trials at these higher stiffness levels further show the difference in adaptation between low stiffness and high stiffness fields (Figures 3.1, 3.3). At lower stiffness levels one sees that the area reaching deviation for catch trials is negative, indicating a negative after effect, and thus an adaptation which compensates for the field dynamics and results in straight line movements. At higher stiffness levels this is not the case. Once a stiffness threshold is exceeded subjects area reaching deviation for aftereffects is positive, indicating after effects in the direction of the applied forces and following the profile of the virtual surfaces. The magnitude of this positive after effect increases with increasing stiffness. Below the stiffness threshold the magnitude of the negative after effect increases with decreasing stiffness. 3.2 The Threshold of Force Perturbation and Object Boundary Detection The resulting field adaptations showed that subjects response to object mechanics varied depending on surface stiffness. From the fitting of a psychometric function it can be seen in Figure 3.6 that when a surface exceeds a threshold stiffness of 1200 ± 205 N/m subjects have a greater propensity for adapting by learning to produce a smooth trajectory on the surface and thus have after effects of positive area, while at lower stiffness they have a greater likelihood of adapting by recovering their original kinematic pattern of movement in free space and thus have after effects of negative area. The gradual slope of the psychometric curve implies that rather than multiple neural processes being active at different stiffness levels, there may be a single neural process active through a continuum of force fields. 31

32 Probablity of Discriminating a Surface % CI Probablity Fit Stiffness (N/m) Figure 3.6 Psychometric Function for Perception of Discriminating an Object Boundary or Surface 3.3 A Possible Interaction Strategy While the visual appearances of trajectories at various stiffnesses can be seen to vary greatly after the threshold of object boundary detection is reached, a unifying feature amongst these trajectories is integral interface force. Through the continuum of virtual surface stiffness levels subjects produce similar integral interface forces after adaptation (Figure 3.4). A two factor ANOVA without replication did not find a significant difference between the six stiffness levels, for subjects, after adaptation had occurred. 32

33 It can be seen that at low stiffness (200 N/m) subjects produce a higher average interface force after adaptation as compared to interface forces observed before adaptation (Figure 3.4). Further at low stiffness levels (200 N/m, 400 N/m) significant differences were found between the area reaching deviation pre and post adaptation. Subjects reduced the area reaching deviation after adaptation at these low stiffness levels. This implies that at low stiffness subjects formulate an internal representation of the virtual surface, as they reduce their deviation from a straight-line path. This internal representation appears to be similar to those described in previous force field adaptation experiments in that ability to recover a straight-line movement after adaptation is indicative of an internal model that is able to completely compensate for the dynamics encountered during force field interaction. At the higher stiffness levels it can be seen that subjects abandoned the goal of reducing straight line deviations and instead strived to reduce the interface force. Statistically significant reductions in interface force and jerk were found after adaptation at the stiffness levels of 1200 N/m and 2000 N/m. This result implies that at high stiffnesses subjects form an internal representation of the virtual surface, and that this representation is based on the reduction of the interface force to a baseline value and the goal of producing a smooth movement over the surface. In essence subjects encountered the boundary and to some extent complied with the shape of the boundary while producing smooth movements over the surface. 33

34 These results suggest the internal representation of two distinct categories through a continuum of force fields: force disturbances (low stiffness fields) and object boundaries (high stiffness fields). With force disturbances interaction forces are resisted and free space trajectories are recovered. Interaction with object boundaries results in trajectory modification so as to reduce interface forces and produce a smooth movement over the surface. 3.4 The Effect of Interaction Speed on Surface Perception The previously presented datum were collected for trajectories executed with a targeted speed of 0.40 m/s. These high speed movements were meant to isolate the feed forward component of subjects movement and thus expose subjects generation of an internal model of the surface. The experiment was also repeated for a targeted speed of 0.20 m/s in order to ascertain the effect of more sensory information on the discrimination of virtual surfaces. 34

35 Stiffness Level K (N/m) Initial Field Exposure 1-6 Mid Field Exposure Late Field Exposure Catch Trials Following Learning Figure 3.7 Trajectories From Various Stages of Adaption For a Typical Subject moving at a targeted velocity of 0.2 m/s. Green Squares Represent the Start Position, Red Circles Represent the Goal Position. 35

36 A typical set of subject trajectories for the movement speed of 0.20 m/s is shown in Figure 3.7. It can be seen that at low stiffness subjects generally achieve a close to straight line movement after adaptation. Negative catch trials at these low stiffness levels reveal that subjects produce forces which counteract those experienced in the force field, a similar result as that seen in the trajectories observed for movement speeds of 0.40 m/s. At higher stiffnesses it can be seen that the subject is initially disturbed by the field, however after adaptation he produces trajectories which conform to the curvature of the surface. Learning measures do not show any systematic trends for trajectories executed at this speed (Figure 3.8, 3.9, 3.10). 36

37 Area Area Reaching Deviation Error m/s m/s Area Reaching Deviation (m^2) Area Reaching Error (m^2) p = 0.45 p = 0.14 p = 0.19 p = 0.47 p = 0.34 p = Stiffness (N/m) Pre Post Figure 3.8 Area Reaching Deviation Pre and Post Adaptation Averaged For All Subjects. Interface Force m/s p = 0.03 p = 0.24 p = 0.81 Interface Force (N s) p = 0.36 p = 0.03 p = 0.66 Pre Post Stiffness (N/m) Figure 3.9 Interface Force Pre and Post Adaptation Averaged For All Subjects. 37

38 Jerk m/s p = 0.01 p = 0.07 p = 0.27 Jerk (m/s^3) p = 0.01 p = 0.16 p = 0.33 Pre Post Stiffness (N/m) Figure 3.10 Jerk Pre and Post Adaptation Averaged For All Subjects. Catch trials at the slower speed show that at low stiffness subjects generally have a negative reaching deviation, indicating the classification of force disturbances (Figure 3.11). While at higher stiffness catch trials begin to have a positive reaching deviation, indicating a higher propensity for classification of object boundaries. It should be noted that object boundary classification does not exceed 65% for trajectories executed at 0.20 m/s (Figure 3.12), this is as opposed to the trajectories executed at 0.40 m/s which reach a object boundary classification of 85%. 38

39 3 x Area (m 2 ) % CI Standard Deviation Stiffness (N/m) Figure 3.11 Trajectories and Area Reaching Deviation Averaged for All Subjects Catch Trials This increased uncertainty of object boundary detection at lower interaction speed could be due to subjects acquisition of more sensory information. This sensory information may cause subjects to use more cognitive strategies to actively explore the surface during movement. Since subjects necessity to rely on the internal model of movement is reduced during these slower speed movements, they may actively think about the boundary they are interacting with and thus press against the surface throughout the movement to get a better understanding of the surface properties, which results in the negative after effects even at higher stiffnesses. 39

40 Probablity of Discriminating a Surface Stiffness (N/m) Figure 3.6 Psychometric Function for Perception of Discriminating an Object Boundary or Surface It is interesting to note that a two factor ANOVA without repeated measures reveals that subjects produce consistent interaction forces across stiffness levels after adaptation for these slower speed interactions. This result suggests that subjects associate the same cost of movement with interaction amongst surfaces. Subjects strive to interact with surfaces in such a way that allows them to produce a consistent interface force despite the surface stiffness. 40

41 Chapter 4 Conclusions 4.1 Key Findings It was found that subjects trajectory adaptation was dependent on surface stiffness. When a surface exceeded a threshold stiffness of 1200 ± 205 N/m, subjects adapted by learning to produce a smooth trajectory on the surface, while at lower stiffness they adapted by recovering their original kinematic pattern of movement in free space. This adaptation suggests the internal representation of two distinct categories through a continuum of force fields: force perturbations and object boundaries. In the first case, the interaction forces are opposed and the trajectory is restored. In the second case, the trajectory is modified so as to reduce the interaction forces and produce a smooth movement along the surface boundary. A unifying theme through the continuum of force fields is the integral interface force. Results show that subjects produce a consistent mean interface force after adaptation, regardless of the stiffness of the surface with which they are interacting. This indicates that a possible goal of interaction could be to 41

42 maintain a level of force contact with the surface to aid in the guidance of motion within or along the surface boundary. 4.2 Potential Future Work The results from this thesis provide a number of different avenues to be explored in the fields of human motor control and haptics. Using similar experimental methods it would be interesting to test the effect virtual surface curvature has on the perception of object boundaries. Specifically, does the stiffness threshold at which subjects perceive fields as boundaries shift as the level of curvature of the surface is increased or decreased? Potential experiments to study the effect of curvature on stiffness perception would involve varying the radius of curvature of the virtual surface and testing the same stiffness levels as those presented in this thesis. The effect of visual information on the perception of stiffness could also be tested using a similar experimental protocol. Subjects would be presented with a discordant visual stiffness and physical stiffness. In this case visual feedback would be given to the subject that did not match the physical displacement of the surface. The visual information would be made to appear visually stiffer or less stiff than the physical stiffness. The continuum of stiffness levels could be tested in order to determine if this discordant visual data may be used to shift the threshold of boundary detection. Results from these experiments may shed light on the incorporation of visual information into haptic perception of objects. 42

43 Another avenue that is worthy of exploration is the effect of the orientation of the virtual surface with respect to the subject. In the case of the experiments presented in this thesis the surface was oriented such that subjects would generally interact with the object boundary. It would be of great interest to study the interactions of the subject with virtual surfaces that aid movements. That is to say, surfaces which reduce the degrees of freedom of a task. By studying the effect of surface orientation insights could be gained into how learning of object boundaries may generalize to movements in different directions, and more generally how different field orientations may effect movements. These results could also be useful in the design of virtual surfaces to guide motions in virtual environments. 43

44 Bibliography Atkeson, C. (1989). "Learning arm kinematics and dynamics." Annual Review of Neuroscience 12: Colgate, J. and J. Brown (1994). Factors Affecting the Z-Width of a Haptic Display. IEEE International Conference on Robotics & Automation, San Diego, CA. Conditt, M., F. Gandolfo, et al. (1997). "The motor system does not learn the dynamics of the arm by rote memorization of past experience." Journal of Neurophysiology 78(1): Flash, T. and N. Hogan (1985). "The Coordination of Arm Movements: An Experimentally Confirmed Model." The Journal of Neuroscience 5(7): Gandolfo, F., F. Mussa-Ivaldi, et al. (1996). "Motor learning by field approximation." Proceedings of the National Academy of Science USA 93: Hollerbach, J. and T. Flash (1982). "Dynamic Ineractions Between Limb Segments Durring Planar Arm Movement." Biological Cybernetics 44: Johansson, R. and G. Westling (1984). "Roles of glabrous skin receptors and sensorimotor memory in autonomic control of precision grip when lifting rougher or more slippery objects." Experimental Brain Research 56: Jones, L. and I. Hunter (1990). "A perceptual analysis of stiffness." Experimental Brain Research 79: Jordan, M. and D. Rumelhart (1992). "Forward models: Supervised learning with distal teacher." Cognitive Science 16: Kappers, A., J. Koenderink, et al. (1994). "Haptic identification of curved surfaces." Perception & Psychophysics 56(1): Matsuoka, Y. (1998). Models of Generalization in Motor Control. Department of Electrical Engineering and Computer Science. Cambridge, Massachusetts Institute of Technolgoy: 215. Morasso, P. (1981). "Spatial Control of Arm Movements." Experimental Brain Research 42: Mussa-Ivaldi, F. and E. Bizzi (1999). "Motor learning through the combination of primitives." Philisophical Transactions of the Royal Society of London 355: Seger, C. (1994). "Implicit Learning." Psychological Bulletin 115(2):

45 Shadmehr, R. and F. Mussa-Ivaldi (1994). "Adaptive Representation of Dynamics During Learning of a Motor Task." Journal of Neuroscience 14: Squire, L., F. Bloom, et al. (2002). Fundamental Neuroscience, Academic Press. Srinivasan, M. and R. LaMotte (1995). "Tactual Discrimination of Softness." Journal of Neurophysiology 73: Thoroughman, K. and R. Shadmehr (2000). "Learning of action through adaptive combination of motor primitives." Nature 407: Vogels, I., A. Kappers, et al. (1996). "Haptic aftereffect of curved surfaces." Perception 25(1): Vogels, I., A. Kappers, et al. (2001). "Haptic after-effect of successively touched curved surfaces." Acta Psychologica 106(3): Wichmann, F. and N. Hill (2001). "The psychometric function: I. Fitting, sampling, and goodness of fit." Perception & Psychophysics 63: Wichmann, F. and N. Hill (2001). "The psychometric function: II. Bootstrap-based confidence intervals and sampling." Perception & Psychophysics 63: