Methodology for Translating Upper Extremity Motion to Haptic Interfaces

Transcription

1 Methodology for Translating Upper Extremity Motion to Haptic Interfaces Nalini Vishnoi and Cody Narber and Zoran Duric Department of Computer Science George Mason University Fairfax, VA {nvishnoi, cnarber, Naomi Lynn Gerber College of Human and Health Services George Mason University Fairfax, VA Abstract The goal of our research is to design a hapticbased system to assist people with disabilities in learning functional motor tasks. Specifically, we are studying the upper extremity while performing daily activities. We have explored fine motor activities, such as writing, and gross motor activities, such as feeding and grooming. We use electromagnetic sensors (MotionStar Wireless 2) to capture unencumbered movements performed by a normal individual. The captured movement is translated to the haptic coordinate system with the use of a body centered, intermediate frame for gross motor activities and with the use of a table-top centered frame for fine motor activities. We segment the movement into simple strokes and then smooth the strokes using low pass filtering, cubic spline fitting and polynomial curve fitting. Our system varies the haptic force as the function of performance of the trainee. We have used our method to program the Phantom Omni for writing tasks and the Phantom Premium 3./6 DoF for gross motor tasks. We demonstrate our methodology with several examples of fine and gross functional activities. Keywords-Phantom Omni, Phantom Premium, Flock of Birds I. INTRODUCTION Rehabilitation of people experiencing difficulties with upper extremity activities is an active and challenging area of research. The upper extremity poses two significant challenges to the current methods of data capture: 1. Unlike lower extremity motion, it is unconstrained; and 2. Patterns of movement are highly varied, not periodic or highly repetitive. Any task involving the upper extremity can be performed in a number of different ways. For example, different trajectories to bring a utensil to the mouth would have the same initial and target points, but the kinesiological and force trajectories might be different. These patterns also incorporate three dimensional motion throughout the trajectory in many, if not most, functional activities. Haptic devices are being used extensively in computer/video games, research and robotics. These devices provide a proprioceptive input in the form of force feedback through which a person can perceive movement and location of a limb in 3D space. One needs proprioception to maintain balance and perform movements. It also influences the knowledge of the direction to move while manipulating objects or ambulating. The haptics can record and report 3D spatial positions, velocity vectors as well as the forces applied in three dimensions: x, y and z at the gimbal. Variable forces can be applied on the stylus of the haptics to allow them to move in an active frame. These features make them highly useful for training and learning purposes. A lot of work has been done on whether these force feedback devices (haptics) can be used therapeutically to improve hand functions. In [1], [2], [3], [4], [5], [6] several different approaches for teaching handwriting have been proposed. Recently, work has been reported ([7], [8]) which justifies that haptic training improves learning of motor-skills. The major limitation of these approaches is: training trajectory is programmed by either using some primitives or using a trainer s trajectory, recorded earlier on the haptic ([9], [1]). In our experiments, we recorded pilot data from people performing a functional activity on the haptic and then performing the same activity in a free, unencumbered way. The plots of the trajectories comparing unencumbered movement to haptic generated movement are shown in Fig. 1, and they clearly demonstrate the difference in performance. We would require the subjects to learn the trainer s trajectory, as gathered from unencumbered data, in order to make the training process feel as natural as possible to the user. Figure 1: Haptic and FoB trajectories superimposed. We present a novel approach of haptic training using the free form movement of a normal person recorded by a motion tracker and then translating it to the haptic workspace. This approach works well with both fine and gross motor tasks. Using haptics for training is a three step process. The first requires the ability to simulate the human motion with the haptic. The second requires programming the force control to permit a smooth natural trajectory, with minimal jerking and transition points experienced by the user, and the third is to use them as intervention tools

2 to improve people s capability who suffer from disabilities. The goal of this research is to simulate the upper extremity movement in a fashion that seems natural to the user. It requires obtaining positional data in 3D. This objective has been accomplished by using a 3D motion tracker called Motionstar Wireless 2 Electromagnetic Sensor (EMS) system. It is commonly known as Flock of Birds (FoB). It reports 3D spatial location of the sensors. Next, the limb movement must be guided by sufficient force to accomplish the functional task. The force should be smooth, i.e. the derivatives (velocities and accelerations) should not change sharply. Objective two is accomplished by translating FoB data into the workspace of the haptic. We have used the Phantom Omni as the guiding interface for fine motor activities and the Phantom Premium for gross functional tasks. Our technique works both for gross motor functions which involve 6 degrees of freedom and for fine motor activities which involve 3 degrees of freedom. We have applied this method to laboratory experiments to assess the ability of the programs to accomplish the desired tasks (e.g. bringing the hand to the mouth or writing letters and shapes). The remainder of the paper is organized as follows. In Sec. II we describe our methodology of capturing free form movement and translating it to the haptic coordinates. In Sec. III we present examples of applying our methodology to both fine and gross motor activities. Finally, in Sec. IV we present conclusions and discuss some future work directions. II. TECHNICAL APPROACH In this section we describe how we captured unencumbered movement from the FoB and then transferred it to the haptic coordinate system. This transfer of movement is important because the free form movement performed with the FoB feels more natural and is considerably different when compared to the movement performed by the same person while holding the stylus of the haptic. The FoB system captures position and orientation of the EMS sensors with respect to the FoB transmitter. Typically, upper extremity movements are constrained by purpose and function unlike lower extremity in which anatomical and physiological constraints are very strong. It is usually possible to accomplish upper body tasks using varying combinations of joint movements. This movement is translated to a haptic device, which typically has fewer degrees of freedom than a human arm. Each haptic device has different constraints. For the Phantom Omni, it is necessary for the stylus to touch the paper while writing, whereas the Phantom Premium permits a greater number of degrees of freedom for the individual to reach the desired target, regardless of the trajectory used to get there. A. Transfer of movement In this section we describe the translation of movement from the FoB frame to the haptic coordinates. We first describe how we utilize an intermediate frame to avoid direct calibration between the FoB and the haptic frames. Our method assumes that we use a single EMS sensor x z O y P 3 P 2 z 1 y 1 Figure 2: FoB and intermediate coordinate frames. that is usually placed on the end effector manipulated by a person while performing functional activities. We first derive equations relating positions in two rectangular coordinate frames, one (Oxyz) fixed in 3D space, the other (Cx 1 y 1 z 1 ) fixed in the 3D reference frame and moving with it. The coordinates X 1, Y 1, Z 1 of any point P of the moving body with respect to the moving frame are constant with respect to time t, while the coordinates X, Y, Z of the same point P with respect to the fixed frame are functions of t. The location of the moving frame at any instant is given by, the position d c = (X c Y c Z c ) T of the origin C, and by the nine direction cosines of the axes of the moving frame with respect to the fixed frame. Let i, j, and k be the unit vectors in the directions of the Ox, Oy, and Oz axes, respectively; and let i 1, j 1, and k 1 be the unit vectors in the directions of the Cx 1, Cy 1, and Cz 1 axes, respectively. For a given position p of P in Cx 1 y 1 z 1 we have the position r p of P in Oxyz: r p i i 1 i j 1 i k 1 j i 1 j j 1 j k 1 k i 1 k j 1 k k 1 C x 1 X 1 Y 1 Z 1 P 1 + X c Y c Rp + d c (1) where R is the matrix of the direction cosines (the frames are taken as right-handed so that det R = 1). In our notation, frames fixed to the FoB and/or haptics are static. For translating the free form movement from the FoB to the haptic we need an intermediate coordinate system that represents a moving frame. While performing gross motor functional tasks, such as eating, brushing teeth or setting up the table, the body coordinate system is best represented as a moving frame. Our assumption is based on the fact: When an activity is repeated, the end effector/hand moves in the same manner with respect to the body. The relative motion of the activities stays same with respect to the body, regardless of the fixed coordinate space. These activities are translated to Phantom Premium because of its large workspace. Similarly, for fine motor activities such as writing, we calibrate the movement with respect to the plane of paper and translate the movement to Phantom Omni. Z c 2

3 In order to completely define the intermediate frame, we need to first find a plane in the moving reference frame with respect to the coordinate systems of the FoB and the haptic devices. We have found that for gross-motor activities it is convenient to use the sagittal plane, i.e. the body plane dividing the left and right sides of the body. To compute the sagittal frame we choose three points, the first point on the trajectory at the knees P 3, the point at the mouth P 1, and the point P 2 as the approximate middle of the trajectory (that is a point on the trajectory that is approximately equidistant from the ends of the trajectory). We place the origin of the moving frame at C = (P 1 + P 3 )/2 (see Fig. 2). The difference m x = P 3 P 1 gives a vector in x-direction. We obtain vector m y = P 2 C pointing approximately in the y-direction. Note that vectors m x and m y span the sagittal plane. A cross product of these two vectors m z = m x m y is orthogonal to the sagittal plane. Computing unit vectors of these three directions form the columns of a rotational matrix R f transforming the coordinates in the intermediate frame into the FoB coordinates. Similarly, we can make use of an equivalent triplet of non-collinear points with respect to the haptic frame and use this method to compute a rotation matrix R h transforming the coordinates from the intermediate frame to the coordinate system of the haptic. Coordinates of a point r p in the moving frame can be transformed to the static FoB frame using P = R f r p + d f, where d f is the displacement of the intermediate frame with respect to the FoB frame. Then, given a point P measured in the FoB frame, its coordinates p measured in the haptic frame would be given by: p = R h (R T f P d f ) + d h where d f and d h are the displacement vectors associated with the intermediate frame in the FoB and the haptic coordinate systems, respectively. Note that we have assumed here that the end-effector trajectories during functional movements would be exactly the same with respect to the body, i.e. the intermediate frame. Similarly, in the case of fine motor activities, we use the plane of the workspace/tabletop as a reference plane for calibration and obtain three non-collinear points P 1, P 2 and P 3 from the FoB and the haptic (Phantom Omni). B. Smoothing and Interpolation Raw data obtained from EMS suffers from noise. The noise is due to the interference of electromagnetic signals with the metal present in the experimental space. Smooth movement is necessary for computing velocities and accelerations. They give information about the strategy being used by a person to complete a functional task. This is important, since we require the haptic to train people in a way that is easy to learn and close to the natural movement. An example of raw data is shown in Fig. 3. It shows first 25 samples of the x-coordinate of a sensor trajectory, while making a symbol $. The original data is very noisy. We used median filtering to remove some high frequency noise present in the raw data. Median filtered data, shown in red, is superimposed on the original data. Once we obtained the movement trajectory, we used three x (mm) Original data Median filtered data Sample [n] Figure 3: Raw x-coordinate data obtained from wireless motion capture. Median filtered data (thick red line). approaches to smooth trajectories, described below. 1) Low Pass Filtering: We have used a critically damped low-pass filter for digital filtering [11]. The theory behind low pass filtering assumes that signal and noise can be divided into separate regions of low and high frequencies occupying different ends of the frequency spectrum. The low pass filter attenuates the high frequency component and allows the low frequency signal to pass through it. The overlapping region between the two has to pass through the cutoff frequency f c whose sharpness is determined by the order of the filter. The general form of a recursive filter in the time domain is: X 1 (nt ) = a X(nT ) + a 1 X(nT T ) + a 2 X(nT 2T ) +b 1 X 1 (nt T ) + b 2 X 1 (nt 2T ) where X 1 = filtered output coordinates. X = unfiltered coordinate data. X(nT ) = nth sample. X(nT T ) = (n-1)th sample. a,, b 2 = filter coefficients. The filter coefficients depend on the type and order of the filter used and always sum up to 1.. As long as the ratio f s /f c is the same, where f s is the frequency of the signal, the filter coefficients are the same. The equations used for computing these coefficients are given as: ω c = tan(πf s/f c ) C where C is the correction factor for the number of passes required. For a single pass, C = 1. We define, K 1 = 2ω c, K 2 = ω 2 c, K 3 = 2a /K 2. The filter coefficients are then computed using the following equations: a = K 2 (1 + K 1 + K 2 ) a 1 = 2a a 2 = a b 1 = 2a + K 3 b 2 = 1 2a K 3 3

4 or equivalently b 2 can also be written as, b 2 = 1 a a 1 a 2 b 1 One way filtering introduces a phase shift in the data. To keep the data in its original phase, we filter the data in both forward and backward directions. We have used several values of f s /f c. Fig. 4a shows the results using low pass filtering. f s /f c = 15 was used for the example shown here. function of the form [12]: S i (x) = y i + b i (x x i ) + c i (x x i ) 2 + d i (x x i ) 3 for x [x i, x i+1 ], i = 1...N 1 The unknowns in this equation are selected to satisfy the following constraints: 1. Interpolation conditions: S i (x i+1 ) = S i+1 (x i+1 ) = y i+1 i = 1,..., n 2 S N 1 (x N ) = y N x (mm) Low pass filtering Original data Low pass filtered data Velocity (m/s) Velocities Figure 4: (a) Low pass filtered x-coordinate data (thick red line) superimposed on the original data from Fig. 3. (b) Velocity in the x-direction obtained from low pass filtered data in Fig. 4a. Once we have smoothed the data we can calculate velocities and accelerations as the first and second derivatives. The traditional method for the calculation of velocities and accelerations is x/ t (determining the velocity in x direction), where x = x i+1 - x i, and t is the time between adjacent samples x i+1 and x i. The velocity calculated in this way does not represent the actual velocity at time t. Instead, it shows the velocity at mid point between the points x i and x i+1. This method can result in errors. Hence we have calculated velocities and accelerations [11] on the basis of 2 t rather than t. Thus the velocity at ith sample is: V xi = (x i+1 x i 1 ) 2 t This represents the velocity of ith sample half way between x i 1 and x i+1. Similarly, the acceleration in terms of displacement is given as: (2) A xi = (x i+1 2x i + x i 1 ) t 2 (3) The velocity obtained from low pass filtering is shown in Fig. 4b. The velocity is not smooth, and the second order derivatives, i.e. acceleration, is even more irregular. Low pass filtering is used mainly for gait analysis. The phases of gait are well defined and are known to be periodic. Hence, computation of cutoff frequencies becomes an easier task for gait analysis. Upper extremity movement, e.g. writing block letters, is rarely periodic and finding proper cutoff frequencies is very hard. 2) Cubic Spline Fitting: Our second approach to data smoothing is based on a cubic spline approximation of the original noisy data. A cubic spline is a piecewise-defined 2. Smoothness conditions: S i(x i+1 ) = S i+1(x i+1 ) i = 1,..., n 2 S i (x i+1 ) = S i+1(x i+1 ) i = 1,..., n 2 All of the above constraints give a total of 3N 5 constraints for 3N 3 coefficients. Remaining two conditions uniquely define the cubics, e.g.: 1. Clamped Spline: The values of S 1(x 1 ) and S N 1 (x N) are explicitly defined. x (mm) 2. Not-A-Knot: The third derivatives at x 2 and x N 1 are equal. S 1 (x 2 ) = S 2 (x 2 ); S N 2 (x N 1) = S N 1 (x N 1) Cubic spline fitting Original data Cubic spline fitted data Velocity (m/s) Velocities Figure 5: (a) Clamped cubic spline fitted x-coordinate data (thick red line) superimposed on the original data from Fig. 3. Control points are shown in blue diamonds. (b) Velocity in the x-direction obtained from fitted clamped cubic spline in Fig. 5a. To fit cubic splines we use polygonal approximation to choose control points from the median filtered data. The algorithm for polygonal approximation is given in Algorithm 1. The algorithm recursively finds a control point t 3 between two points t 1, t 2 if the distance between t 3 and the line t 1 t 2 is greater than the given tolerance d. This method finds the points at the sharp transitions of the curve. We have tried several ways of choosing the control points for spline fitting. The best results were obtained by using the mid-points of straight line segments, obtained by the polygonal approximation, as the control points. The slopes of the first and last polygonal segments were used to specify the first derivatives at the end points for clamped spline fitting. The results of polygonal approximation and 4

5 Algorithm 1 Polygonal Approximation (t 1, t 2, f(x 1...x N ), d) 1: Project all the points in the sequence [f(t 1 )...f(t 2 )] on the line f(t 1 )f(t 2 ). 2: Find the distance between the points and their respective projections on the line. 3: Find the index, t 3, of the point that has the maximum distance, d max. 4: if (d max > d) then Return Polygonal Approximation(t 1, t 3, f(x 1...x N ), d) Return Polygonal Approximation(t 3, t 2, f(x 1...x N ), d) 5: end if degrees of freedom, the VC-dimension is h = m. In the case of polynomial curve fitting h = k + 1. The model providing minimal prediction risk R est is then chosen. x (mm) Polynomial curve fitting Original data Polynomial fitted data Velocity (m/s) Velocities cubic spline fitting are shown in Fig. 5a. The tolerance used in this example is 2 mm. The first order and the second order derivatives of each piece of cubic spline is given as, S i(x) = b i + 2c i (x x i ) + 3d i (x x i ) 2 S i (x) = 2c i + 6d i (x x i ) Velocity computed using this approach is shown in Fig. 5b. 3) Polynomial Curve fitting: Our third and final approach for smoothing uses the polynomial curve fitting to the median filtered data. We fit k-th order polynomials to each coordinate of stroke (x(t) y(t) z(t)) T. For a given coordinate, say x, we estimate functional form of x(t) from measurements: x(it ) = a + a 1 (it ) a k (it ) k, i = 1...N (4) This equation can be rewritten as p k i ak x(it ) where p k i = (1 it (it )2 (it ) k ), a k = (a a 1 a 2 a k ) T and x(it ) is the value of x measured at time t = it. We usually have N k. We need to find a solution of P a k b = e, where b is an N-element vector with elements x(it ), P is an N k parameter matrix with rows p k i, and e is an N-element error vector. We seek the model a k that minimizes e = b P a k ; the solution satisfies the system P T P a k = P T b and corresponds to the linear least squares (LS) solution. We implemented an automatic computation of the order of the polynomial for given data points. We used the method described in [13] to find the order of the polynomial that minimizes the prediction risk. The estimate of the prediction risk takes the form: R est (a) = r(p, n) n N (X(iT ) p i a) 2, (5) i=1 where p k i and ak are already defined for a polynomial of order k, and r(p, n) is the penalization factor called the Vapnik s measure (vm): ( ) 1 r(p, n) = 1 p p ln p + ln n (6) 2n where p = h/n, h denotes the VC-dimension of a model and ( ) + =, for x <. For linear systems, with m + Figure 6: (a) Polynomial curve fitted x-coordinate data (thick red line) superimposed on the original data from Fig. 3. (b) Velocity in the x-direction obtained from fitted polynomial curve in Fig. 6a. Once the best fit order of the polynomial is estimated, the best fit curve is computed for the given trajectory. Subsequently velocities and accelerations of the model in Eq. 4 are computed as, ẋ(it ) = a 1 + 2a 2 (it ) + 3a 3 (it ) ka k (it ) k 1 ẍ(it ) = 2a 2 + 6a 3 (it )... + ka k (k 1)(iT ) k 2 The order of the curve can vary, depending on the amount of noise and shape of the original data. An example of approximating data with polynomial curve fitting is shown in Fig. 6a. The order of the polynomial curve fitted in this example is 5. Velocity obtained from fitting the polynomial curve of Fig. 6a is shown in Fig. 6b. Accelerations from the positional data requires computing its second order derivatives. Noise makes the data undesirable for higher order derivatives. Using polynomial fitting smoothes the data to a point at which higher order derivatives can be computed without difficulty. Velocity (m/s) Comparison of Velocities Figure 7: (a) Velocities obtained using low pass filtering (blue line), cubic spline fitting (red line) and polynomial curve fitting (green line). (b) Polynomial fitting superimposed on Original Trajectory in 3D. The above three approaches for smoothing original noisy data have different limitations. Low pass filtering requires the cutoff frequency f c to be known. It works best with the lower extremities, where most of the gait motions are periodic and can be approximated with harmonic curves. The computation of the cutoff frequency

6 for the upper extremities is not trivial since upper extremity movements are highly non-periodic in nature. The first order derivatives-velocities obtained from low pass filtering are not smooth. Cubic spline fitting is highly dependent on the control points used for approximating the trajectory. The lower the threshold (tolerance) used in the polygonal approximation, the more accurate the fitting with cubic splines. The low threshold, however, increases the computational time. The velocities obtained from cubic splines are smoother as compared to the low pass filtering. Polynomial curve fitting provides the best results for smoothing and velocities. The upper bound on the order for polynomial fitting has to be provided depending on the functional movement being studied. A comparison of velocities first order derivatives obtained from the three approaches is shown in Fig. 7a. Fitting the polynomial curve to the original data results in the smoothest first order derivative. A 3D plot of the trajectories fitted using the polynomial curve is shown in Fig. 7b. The data obtained from the FoB is recorded at a frequency of 86.1 Hz, whereas the haptic generates and reads data at 1 Hz frequency. Due to the difference in the operating frequencies of the haptic and the FoB, transferring of data from one device to the other requires up-sampling or down-sampling of data. Up-sampling is required if the movement is transferred from FoB to haptic, down-sampling is required in the second case. We have used cubic spline interpolation for up-sampling of the data. Making the frequencies equal ensures that the same time is being taken to complete the maneuver, hence the same velocity. C. Force Control The simulation of motor tasks has two necessary components: 1. Understanding the trajectories of movements to simulate the movement. 2. Determining forces and accelerations of the movement. In our experiments we have computed the guiding force based on the subject s deviation from the current position to the next estimated position. The force F is (Hooke s Law ([12]): F = κ dr where κ is the spring constant of the haptic. The larger the value of κ, the greater would be the force exerted to get to the desired point. dr denotes the distance from the current point to the next point in space. If the subject is going farther away from the desired trajectory, the haptic will increase the force in order to ensure that it guides the hand along the right trajectory. This type of control can be thought of as a spring, which is tied to the trajectory and keeps on pulling the stylus of the haptics towards the trajectory. As a person is being guided by the haptic repetitively to perform a functional task, the force can gradually be dialed down to allow him/her to drive the stylus of the haptic, based on his memory and learning. III. EXPERIMENTS AND DISCUSSION This section has been divided into two parts: fine motor tasks and gross motor tasks. Fine motor and gross motor activities differ primarily by the muscle-groups involved. Fine motor activity primarily requires prehensile patterns of movement, and is designed to manipulate objects. Gross motor activity requires larger muscles activity and is primarily responsible for hand positioning. Fine motor activities involve intrinsic as well as extrinsic muscles of the hand and wrist, whereas gross motor activities involve large muscles and joints that are located proximal to the wrist. We have used MathWorks MATLAB for processing the data and OpenHaptics toolkit from SensAble Technologies [14] for programming the haptics. A. Fine motor tasks We have used the Phantom Omni to guide fine motor tasks e.g. writing. It has a small workspace of mm, and it can apply upto 3.3 N of force on its gimbal. It reports the position, velocity and force applied in 3D coordinates. The reason for using the Omni is that it is more restrictive in terms of workspace and gives greater control needed to tune fine motor skills. It can apply high force in a smaller work area. We have used κ =.1 N/mm for all our experiments. The FoB is used to capture unencumbered 3D motions of a normal person s hand while performing functional activities, such as writing. The tracker reports the positions of the sensor in 3D. A sensor was placed on the endeffector while recording the functional activities. We have used the plane of paper as the intermediate frame between the FoB and the haptic for writing purposes. Three points on the plane of paper were used to calibrate the intermediate frame against the FoB and the haptic coordinate systems. The captured motion is then transformed to the haptic coordinate system so that it can be used to guide a subject s hand through a desired movement. The entire trajectory of movement is made up of a number of strokes. For example, writing a dollar sign consists of 3 strokes: an S, a lift, then a straight line. The trajectory needs to be broken down into these separate strokes to facilitate the understanding of motor learning. The subjects perceive the movement through the proprioceptive interface with the haptic. The guiding force is proportional to the current position and the next position in time. Fig. 8a. shows the translation of symbol $ to the Omni workspace, followed by polynomial fitting. Fig. 8b. displays the different strokes involved in the writing of symbol. The order of the polynomials fitted in x, y and z dimensions for the first stroke are [8 8 7], for second [7 7 3] and those for the third stroke are [4 4 4]. Another example of writing an infinity symbol is shown in Fig. 8c. The symbol is segmented into two strokes as shown in Fig. 8d. The two strokes are the two ellipsoids that make up an infinity symbol. The order of the polynomials fitted in x, y and z dimensions for the first 6

7 st 2nd 3rd 1st 2nd Figure 8: Polynomial curve fitted data superimposed over original trajectory. (b) Strokes involved in making a $ symbol. (c) Polynomial curve fitted data superimposed over original trajectory. (d) Strokes involved in making an symbol. stroke are [1 9 9] and those for the second stroke are [1 1 1]. The translated and smoothed movement trajectory is loaded into the haptic s training program. The program displays the desired trajectory, including lifts. As the user manipulates the haptic-pen, a cursor is drawn on screen reflecting the user s movement. Fig. 1 shows an example of the graphical user interface used. For fine motor tasks we have used the Omni to guide the hand of a person to trace different letters and shapes. Figure 1: Graphical User Interface. In freeform mode, users are in the tracking task as they hold down a button on the haptic device. As they continue to hold down the button their path is drawn in the same manner as is done in guidance mode. There is no force applied by the haptic in this mode. When the users release the button, their accuracy is computed as described above. Figure 9: Subject using Phantom Omni to draw the $ symbol. The translated movement from FoB is in green and as the subject is guided though the trajectory, the output is shown in red. The user can perform the trajectory matching tasks with either guidance or freeform settings. In guidance mode, the haptic applies a force, as previously described, to move the user to the desired position at a specific time. The training task begins by applying force to move the pen to the starting position; the task will not begin until the cursor is within bounds of the starting position. As the user moves the cursor during the tracking task, another path is drawn, marking where the cursor has been providing a visual cue of performance over time. Once all the points in the desired trajectory have been used for the force calculations, a performance measure is calculated using the Longest Common Subsequence. B. Gross motor tasks The Phantom Premium 3./6DoF has been used to guide gross motor tasks. It has a much larger workspace, mm, and the maximum force is 22 N. Because of its large workspace, it permits shoulder rotation and good hand positioning for gross motor functional tasks like feeding with great ease and high force along the 3 dimensions. The free form movement is captured from the FoB by placing the sensor on the end effector used for the activity. The body centered frame is used as the intermediate frame for gross motor tasks. Three points were recorded in the sagittal plane of the person performing the activity. These points calibrate the body frame with the FoB and the haptic coordinate systems. We have used κ =.1 N/mm for all our experiments. Fig. 11a shows the translation of free form data from the FoB to the haptic (an eight shape). Fig. 11b shows the different strokes involved in the maneuver. The order of the polynomials fitted in x, y and z dimensions for the first stroke are [1 1 1] and those for the second stroke are [1 1 1]. 7

8 1st 2nd st 2nd Figure 11: Polynomial curve fitted data superimposed over original trajectory. (b) Strokes involved in making a 8 symbol. (c) Polynomial curve fitted data superimposed over original trajectory. (d) Strokes involved in simulating eating activity Fig. 11c shows another example of translation of free form data from the FoB to the haptic. The task performed here is the simulation of an eating activity. It has been separated into two strokes as shown in Fig. 11d. The first stroke represents the movement of taking the end-effector close to the mouth and the second stroke represents the backward movement, coming to the start position again. The order of the polynomials fitted in x, y and z dimensions for the first stroke are [5 6 4] and those for the second stroke are [6 5 6]. A possible improvement in force control approach would be to divide the force into several components, each of which tends to control different aspects like deviation from the desired trajectory, deviation from the expected velocities and accelerations. IV. CONCLUSIONS The movement in the haptic space is not identical to the movment in real space (free form) because the hand is holding the end effector of haptic device. The ability to capture real space 3D data using FoB permits the direct translation of uncontrained movement to the haptic. The data from FoB were rough and described a noisy trajectory, which converted to haptic movements that were experienced as jerky with obvious transition points. The data initially obtained from the FoB produced highly irregular first and second order derivatives. We designed a unique approach to smooth the data, which includes low pass filtering, cubic splines fitting and polynomial curve fitting. Polynomial fitting smoothes the data to a point at which even the second order derivatives are regular. After the process of smoothing and interpolation, the haptics were able to guide a subject s hand through the desired trajectory. Our methodology simulated movement in a fashion that was experienced as natural, and from which subjects were able to identify programmed shapes. We plan to continue this approach towards force control using the first (velocities) and second order derivatives (accelerations) in order to simulate human movement more naturally and assess the value of force driven, proprioceptive input in motor learning. ACKNOWLEDGMENT This material is based upon work supported by the National Science Foundation under Grant No. CNS and Henry Jackson Foundation under Award No. HJF REFERENCES [1] K. Henmi and T. Yoshikawa, Virtual lesson and its application to virtual calligraphy system, in IEEE International Conference on Robotics and Automation, [2] Y. K. Kim and X. Yang, Hand-writing rehabilitation in the haptic virtual environment, in IEEE International Workshop on Haptic Audio Visual Environments and their Applications, 26. [3] S. Saga, N. Kawakami, and S. Tachi, Haptic teaching using opposite force presentation, in World Haptics Conference, 25. [4] J. Solis, C. A. Avizzano, and M. Bergamasco, Teaching to write japanese characters using a haptic interface, in 1th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 22. [5] C. Teo, E. Burdet, and H. Lim, A robotic teacher of chinese handwriting, in 1th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 22. [6] Y. Kim, Z. Duric, N. L. Gerber, A. R. Palsbo, and S. E. Palsbo, Teaching letter writing using a programmable haptic device interface for children with handwriting difficulties, in IEEE 3D User Interfaces, 29. [7] J. Bluteau, S. Coquillart, Y. Payan, and E. Gentaz, Haptic guidance improves the visuo-manual tracking of trajectories, PLoS ONE, vol. 3, no. 3, p. e1775, Mar 28. [8] X.-D. Yang, B. W.F., and P. Boulanger, Validating the performance of haptic motor skill training, in Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 28. [9] S. C. J. Lee, Effects of haptic guidance and disturbance on motor learning: Potential advantage of haptic disturbance, in IEEE Haptics Symposium, 21. [1] A. H. C. H. Park, J. W. Yoo, Transfer of skills between human operators through haptic training with robot coordination, in IEEE International Conference on Robotics and Automation, 21. [11] D. A. Winter, Biomechanics and Motor Control of Human Movement. Wiley, 24. [12] E. Kreyszig, Advanced Engineering Mathematics. Wiley, 24. [13] V. Cherkassky, Model complexity control and statistical learning theory, Natural Computing: An International Journal, Kluwer, vol. 1, no. 1, pp , 22. [14] Sensable technologies,