The majority of medical ultrasound imaging is carried

Transcription

1 1330 ieee transactions on ultrasonics, ferroelectrics, and frequency control, vol. 52, no. 8, august 2005 A System for Simultaneously Measuring Contact Force, Ultrasound, and Position Information for Use in Force-Based Correction of Freehand Scanning Michael R. Burcher, Member, IEEE, J. Alison Noble, Member, IEEE, Lianghao Han, and Mark Gooding Abstract During freehand ultrasound imaging, the sonographer places the ultrasound probe on the patient s skin. This paper describes a system that simultaneously records the position of the probe, the contact force between the probe and skin, and the ultrasound image. The system consists of an ultrasound machine, a probe, a force sensor, an optical localizer, and a host computer. Two new calibration methods are demonstrated: a temporal calibration to determine the time delay between force and position measurements, and a gravitational calibration to remove the effect of gravity on the recorded force. Measurements made with the system showed good agreement with those obtained from a standard materials testing machine. The system s uses include three-dimensional (3-D) ultrasound imaging, force-based deformation correction of ultrasound images, and indentation testing. I. Introduction The majority of medical ultrasound imaging is carried out using the freehand technique whereby the sonographer holds the probe in contact with the skin, adjusting its position and orientation so that the scan plane intersects the tissues of interest. During this process, the contact force between the probe and skin causes the underlying tissues to deform by an amount that depends on the applied force and the composition of the tissues. The contact force is often deliberately varied in order to improve image quality or induce movements in the tissue being imaged. The improvements in image quality are believed to be due to a reduction in the thickness of the aberrating subcutaneous fat layer [1], or the flattening of refracting structures, such as Cooper s ligaments in the breast [2]. The ability to image tissues during palpation is unique to real-time ultrasonography and can reveal useful diagnostic informa- Manuscript received July 31, 2004; accepted January 31, M.R.B. was funded by EPSRC grant GR/M54995/01. L.H. was funded by MRC grant G M. R. Burcher was with the Medical Vision Laboratory, Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, UK. He is now with Philips Research, Briarcliff Manor, NY ( michael.burcher@philips.com). J. A. Noble and M. Gooding are with the Medical Vision Laboratory, Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, UK. L. Han was with the Medical Vision Laboratory, Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, UK. He is now with the Engineering Department, Cambridge University, Cambridge, UK. tion, for example, the distensibility of an artery [3], [4] or the compressibility of a lesion in the breast [5]. However, in current clinical practice, the contact force that induces the deformation is not measured, and the interpretation of the observed deformation, especially after the imaging examination, is made without reference to the magnitude of the deforming force. One reason why ultrasound images are hard to reproduce at a later date is that the deformation caused by contact with the probe is not corrected. On a repeat scan, even if the probe is imaging the same tissues, the deformation almost certainly will be different because of changes in the contact force and/or the mechanical properties of the tissues. This can cause significant changes in the geometry of the tissue structures, which make combining a given freehand ultrasound image with other ultrasound images (compounding) or other imaging modalities much more challenging. For these tasks, it would be desirable to remove the image deformation caused by probe pressure. Generating an image of the tissues in an undeformed position (i.e., as if there were no contact force) would then be the first stage in combining the ultrasound image with other images. Additional compensation may be required to deal with other sources of deformation, such as variations in patient posture or breathing motion. The variation in tissue geometry caused by varying contactforce is also a problemin three-dimensional (3-D) freehand ultrasound, in which a 3-D volume is reconstructed from individual slices acquired at different probe positions [6], [7]. In order to remove the deformation induced by the contact force during freehand scanning, we have proposed a force-based correction method [8] [10]. The aim of this method is to produce repeatable ultrasound scans independent of the probe pressure applied during the acquisition. This is done by using contact force and probe position measurements to predict and correct the deformation induced by the contact with the ultrasound probe. Because the mechanical properties of tissue vary between individuals and over time, the prediction must be performed with a patient-specific elastic model. The parameters of the model also are determined using measurements of contact force and probe position, preferably acquired at the same time as the scan. Fig. 1 shows how the force-based deformation correction method can use force information to improve the registration between two ultrasound sweeps acquired /$20.00 c 2005 IEEE

2 burcher et al.: deformation correction and investigation of deformations 1331 Fig. 1. Sections through uncorrected and corrected volumetric images. Each is reconstructed from two overlapping freehand ultrasound sweeps with different applied contact forces and approximately 100 B-mode images per sweep. The section plane is perpendicular to the plane of the B-mode images and aligned with the axial direction of the ultrasound probe. (a) Uncorrected: the boundary between the two sweeps is clearly seen and there is a large discontinuity at the skin line. The lesion is blurred. (b) Corrected using force information and a surface model: the skin line is now continuous between the two sweeps and layered structures are better aligned across the boundary. The lesion is clearly seen. with different contact forces. In this case the positions of the ultrasound slices have been corrected by using a nonlinear surface model [8], [9] to predict the displacement of theskinsurfaceatthetimeofacquisition. The measurements required by the deformation correction method are made with a system that simultaneously records the position of the ultrasound probe, the contact force it applies to the skin, and B-mode ultrasound images of subcutaneous tissues. This paper describes the design, construction, and calibration of this novel acquisition system. This is called the MVL system because it was developed at the Medical Vision Laboratory, University of Oxford. In addition to its use with the force-based correction method, the MVL system has been used to quantitatively investigate deformation processes [11] [14]. The first part of this paper describes the principle of operation of the acquisition system, then details the hardware and software components of the prototype implementation. In Section IV, calibration procedures are described that determine previously unknown delays in the system and account for the effect of the probe weight on the force measurements. The challenges of spatial and temporal calibration for freehand 3-D ultrasound systems are well-known [15]. Spatial calibration must be carried out to determine the rigid body transformation between the coordinate system of the tracking device and the plane of the B-mode image. In addition, temporal calibration is required to match each image with the position measurement made at the same time. If these calibration steps are not carried out correctly, they will distort the reconstructed data and introduce errors that limit the overall accuracy of the system. A number of methods for spatial calibration have been proposed, and most are based on scanning objects of known geometry such as points [16], planes [15], or wires [17]. The various approaches are compared in [17], [18]. The issue of temporal calibration has been addressed by measuring latencies in response to a step input [19] or correlating the image and position data [18]. These calibration techniques have been used to create high-definition freehand ultrasound systems with 3-D point location accuracy of 0.5 mm [18]. In addition to these spatial and temporal calibrations, the MVL system detailed in this paper must undergo two extra calibration procedures: an additional temporal calibration and a gravitational calibration. The additional temporal calibration is required in order to determine the delay between the force measurements and the position data. Knowing this delay allows the correct force to be paired with a given image-position pair. An accurate determination of this delay is critical when investigating the deformation of visco-elastic materials that exhibit hysteresis. However, to the authors knowledge, this is a problem that has not been addressed previously in the literature. The gravitational calibration is required in order to eliminate the effect of the probe weight on the force measurements. The problem of gravitational calibration was addressed in [20] in which the force applied to laproscopic instruments was measured. In that case, a calibration was required to remove the effect of the instruments weight on the measurement. This was achieved by attaching a series of weights to the tip of the grasper, altering its orientation, and performing linear regression. The method proposed in this paper is quicker and simpler because it does not require additional weights to be attached to the probe. The last part of the paper describes a validation experiment in which measurements from the MVL system are compared to those acquired using a materials testing machine. II. Principle of Operation The novel aspect of the acquisition system is its ability to measure the force that the ultrasound probe applies to the tissue during the scan. This is done by measuring the force that the tissue applies to the probe because, from Newton s third law of motion, the two quantities are equal in magnitude and opposite in direction. The measurement is carried out using a six-axis force transducer which measures three forces and three torques. All six components of force and torque are recorded, although the torques are not currently used, either for calibration or during analysis. One side of the force transducer is attached to the ultrasound probe, and the other side is attached to an enclosing box (Fig. 2).

3 1332 ieee transactions on ultrasonics, ferroelectrics, and frequency control, vol. 52, no. 8, august 2005 Fig. 2. Schematic of force measurement apparatus. The force transducer is rigidly attached to the ultrasound probe on one side and the enclosing box on the other. The probe protrudes from the bottom of the box and makes contact with the skin. The box surrounds the probe, preventing other objects from touching the probe. Fig. 4. Schematic of system components. The host computer houses a frame grabber to grab images from the ultrasound machine, a controller card that interfaces to the force transducer, and a serial port that communicates with the localizer. During acquisition, the images, positions, and forces are sampled asynchronously and stored in RAM. Once acquisition is complete, the data is saved to the hard disk. Fig. 3. Free body diagram of the probe. The only forces acting on the probe are the contact force, F CONTACT,itsweight,W,andthe true, unbiased force measurement, F TRUE. The box prevents the ultrasound probe from being touched by other objects. If this is the case, then the only forces acting on the probe are the true, unbiased measured force F TRUE, the contact force, F CONTACT,andtheprobe weight, W (Fig. 3). If the probe is moved slowly during the acquisition, then its acceleration can be ignored and so for force equilibrium: Hence, F TRUE + F CONTACT + W = 0. (1) F CONTACT = W F TRUE. (2) Note: the bold typeface indicates a vector quantity. Because F TRUE is recorded in a coordinate system that is fixed relative to the probe, in order to calculate F CONTACT using (2), the probe s orientation must also be recorded. Furthermore, a gravitational calibration process (Section IV-B) is required in order to determine the magnitude and direction of W. III. Implementation The components of the acquisition system are shown in Fig. 4. The system has been used with two different ultrasound machines: a Sonos 5500 scanner with a L MHz linear array probe (Philips Medical Systems, Andover, MA); and an AuIdea 4 scanner with a LA MHz linear array probe (Esaote, Genoa, Italy). In each case the ultrasound B-mode images from the scanner s video output are grabbed using a Meteor II frame grabber (Matrox Imaging, Dorval, Quebec, Canada). The frame grabber is not ideal for several reasons: it introduces noise into the ultrasound images, interlacing artefacts can occur, and there is a variable latency between ultrasound image acquisition and frame grabbing (this is described in detail at the end of this section). It does, however, allow the acquisition system to be used with different ultrasound scanners and in a normal clinical environment. A better alternative would be to capture the digital images [18], [21] or radio frequency (RF) data directly, as we are now doing with the Analogic AN2300 ultrasound engine (Analogic, Peabody MA). This eliminates the noise and interlacing introduced by the analogue frame grabber and allows the time of ultrasound acquisition to be measured more accurately. The force transducer used in the prototype system is a Mini-40 (ATI Industrial Automation, Apex, North Carolina). This is connected by a highly flexible cable to a controller card in the host computer. Forces of ±20 N can be measured with a resolution of 1.25 mn and an absolute accuracy of ±0.2 N. The force transducer is mounted on the probe using a probe holder, which also serves to enclose the probe and prevent it from being touched except at the probe face. The probe holders used with the Philips probe is shown in Fig. 5. The force transducer is mounted on an aluminum plate that is rigidly attached to the ultrasound probe. The other flat surface of the force transducer then is attached to a Perspex enclosing box that surrounds the probe on four sides. The face of the probe protrudes from the bottom of the box, so that it can touch the object being imaged. The cable for the ultrasound probe is looped and clamped to

4 burcher et al.: deformation correction and investigation of deformations 1333 Fig. 5. Probe holder used with Philips probe for phantom experiments. One side of the force transducer is attached to the ultrasound probe, and the other side is attached to the inside of the transparent enclosing box. The Polaris tool is mounted at an angle on the outside of the box. the box, so that forces acting on the unlooped part of the cable are not recorded by the force transducer. A localizer is used to record the position and orientation of the ultrasound probe, and hence that of the scan plane. This allows a 3-D volumetric image to be reconstructed by placing the individual 2-D scans correctly relative to one another. The position information is also useful in dynamic studies in which the contact force is deliberately varied: by tracking a landmark within the tissue using image-processing software, its motion relative to a fixed coordinate system can be calculated [22]. In addition, the position information is needed for the gravitational calibration process (Section IV-B). The localizer used is a Polaris Hybrid optical tracker (Northern Digital Inc., Waterloo, Ontario, Canada), which consists of two calibrated cameras mounted in a camera unit, and a tool, which is rigidly fixed to the probe holder. The Polaris tracks the position of four LEDs in the tool, and so calculates the rigid body transform between the camera frame and the tool frame. Acquisition is controlled by a host computer that houses the frame grabber and force transducer controller card. The computer has dual 500 Mz Pentium III processors and 512 MB of RAM. Multithreaded software is used to control the hardware, capturing and preprocessing the data then storing it on the computer s hard disk. Separate threads capture the video, position, and force information at rates of 25, 60, and 500 Hz, respectively. The time of capture for each sample is recorded using the performance counter on the computer, with a resolution of 1.4 µs. All measurements are taken within 1.0 ms of the desired sampling instant, so the small amount of jitter this introduces is not a significant source of error. An alternative to sampling the video, position, and force data at different rates would be to trigger measurements of force and position each time an ultrasound image is acquired. However, this was not done because the high bandwidth force signal then would be undersampled and possibly aliased. Fig. 6. Signal latencies. An event, such as jerking the probe away from the object being imaged, causes a step change in each signal. Because the measuring devices that record the different signals have different latencies, the event is not recorded at the same time in each signal. As shown, the values of the differences between the latencies are T force-position,t force-video,andt video-position. In the acquisition stage, all the data are captured and stored in RAM. Once capture has stopped, the force and position data are resampled by a controller thread before being saved to the hard disk along with the images. The resampling is required for two reasons: to align the measurements in time, and to reduce the sample rate to that of the video signal. The need to align the measurements in time arises because each has a different latency the time between the measurement being taken and being available on the host computer. These latencies are shown schematically in Fig. 6. An event, such as a rapid movement of the probe away from the object being imaged, causes a step change in each signal, but the responses occur at different times. The origin of the latencies is different for each measurement: Force. The force transducer samples the analogue voltages across its strain gauges at a rate of 7800 Hz. The controller card within the computer then uses these values to calculate force and torque vectors, which are made available in registers on the card. The calculations performed on the controller card are carried out in hardware with a nearly constant but unknown latency. The force capture thread running on the host computer polls these registers every 2 ms and stores the values in RAM. Because the polling is not synchronized with the 7800 Hz strain gauge sampling, the sample may have spent up to 0.13 ms in the register beforehand. In addition, the commands to read the registers can take a variable time to execute (depending on processor loading), which can increase the latency by up to 0.4 ms. Position. Inside the localizer, the stereo images are processed to determine the position of the LEDs within the image, and the Polaris tool s location is

5 1334 ieee transactions on ultrasonics, ferroelectrics, and frequency control, vol. 52, no. 8, august 2005 then calculated from these positions. This hardware processing takes a fixed but unknown amount of time. The position information is communicated over a 115 kbaud serial interface to the host computer in 4.5 ms. A command to get the next measurement is then sent to the localizer, so that the communication of the measurement commences as soon as it is calculated. The latency is due to hardware processing and communications overhead, and it is nearly constant, with an observed variation of <0.2 ms. Video. The ultrasound data is acquired in an amount of time that depends on the depth of imaging, number of focal zones, and number of lines in the image. The ultrasound machine then performs signal and image processing operations, such as detection and scan conversion, before writing the ultrasound image to its output video buffer. Images then are grabbed by the frame grabber and transferred to RAM in the host computer. The output video buffer is not synchronized with the production of ultrasound images; therefore, a given image will spend a variable amount of time within the output buffer up to the duration of one ultrasound frame. For example, if the ultrasound machine is imaging at 55 frames per second, each image will spend 18.2 ms in the buffer. The amount of time that a given image spends in the buffer before being grabbed is unknown, so the latency of the video signal is variable, although for a given ultrasound acquisition protocol it does have a constant, well-defined average value. In the MVL system, these latencies are not known a priori because the measurements are performed by proprietary devices for which detailed timing information is not available. In order to realign the data, the absolute values of the latency are not required, only the differences between them: T force-position and T video-position in Fig. 6. For the MVL system, the average values of these are determinedinatemporalcalibration process (see Section IV- A). Alternatively, if the latency of each device were known, then the values of T force-position and T video-position could be calculated, and the temporal calibration process would not be necessary, although it still could be used to validate the calculated values. The latencies of the force and position measurements have a small variation (<0.6 ms), whereas the ultrasound image has a variation of at least 18.2 ms. This variable latency causes uncertainty in the time that a given ultrasound image is acquired, which is a source of error when aligning the ultrasound images with the force and position data. It will cause errors in position and force that are proportional to the speed of the probe and the rate of change of force, respectively. These errors are minimized, however, by aligning the images using the average latency values of T force-position and T video-position. For systems in which the timing data within the ultrasound machine can be accessed (such as the Analogic scanner), the time of ultrasound acquisition can be determined accurately, and so the latency does not vary. However, temporal calibration is still necessary because the latencies of the force and position signals are unknown. The sample rates of the position and force measurements are reduced in order to produce a single force and position reading for each image. The force data is filtered using a 12.5 Hz antialiasing filter (10th order, zero phase shift finite impulse response, designed using a Hamming window), then an interpolated value is calculated for the time of each video frame. The position signal is also time-shifted and resampled in a similar way. Because the original data has a low bandwidth and does not contain significant energy above 12.5 Hz, it does not need to be filtered before downsampling. Instead, the position at the video sample time is linearly interpolated from the two adjacent positions using quaternions [23]. The end result of the acquisition is a series of ultrasound images, each with a corresponding position and force measurement. IV. Calibration The MVL system must undergo three types of calibration: spatial, temporal, and gravitational. The need for spatial calibration and temporal calibration between images and positions is common to all freehand 3-D ultrasound systems. The need for temporal calibration between force and position and gravitational calibration is a unique requirement of our novel system. Therefore, this section describes algorithms for performing these new types of calibration. Spatial calibration is required to determine the rigid body transformation between the image plane and the coordinate system of the Polaris tool. The position of the probe relative to the probe holder changes slightly each time the holder is attached, so the calibration must be carried out whenever this occurs. The calibration is performed using a temperature-controlled water bath containing a ping-pong ball with three wires crossing at its center. Approximately 40 images showing the intersection of the wires are taken from different angles. The position of the ball within each image is identified automatically with a Hough transform and used to deduce the position of the wire intersection. The image measurements then are combined with the localizer readings to solve for the spatial calibration parameters [24]. A. Temporal Calibration There are unknown latencies in the ultrasound machine, localizer, frame grabber, and force transducer. These latencies vary from one sample to the next, mainly due to the lack of synchronization between the ultrasound scan and the output video buffer. The aim of the temporal calibration process is to use a large number of samples to determine the average delays between the ultrasound scan,

6 burcher et al.: deformation correction and investigation of deformations 1335 Fig. 7. Apparatus for force-position temporal calibration. The probe holder is attached to a wooden arm, which is free to pivot about A. The force transducer is mounted on the outside of the probe holder and records the force applied to it by the plastic ruler, which acts as a cantilevered spring. The localizer records the position of the Polaris tool. T force-position is calculated from the phase difference between the force and position measurements when the system oscillates. position measurement, and force measurement. These are illustrated as T force-position and T video-position in Fig. 6. It is assumed that the values of T force-position and T video-position are the same for all acquisitions carried out with a given configuration of hardware, software, and ultrasound imaging protocol. Two calibration procedures are required: one to determine T force-position and one to determine T video-position.t force-video is not calibrated for as it is hard to measure directly and its value can be calculated as T force-video =T force-position T video-position. 1. Force-Position Temporal Calibration: In order to calculate T force-position, force and position data were recorded simultaneously using the apparatus shown in Fig. 7. The probe holder used was the one shown in Fig. 5. This was modified by removing the ultrasound probe and force transducer from inside the enclosing box and rigidly attaching the underside of the box to a wooden arm that was free to pivot about A. The force transducer then was mounted outside the box, on its upper surface. The upper surface of the force transducer was attached by a metal wire to the end of a plastic ruler, which acted as a cantilevered spring. At rest, the weight of the arm and probe holder was balanced by the bending ruler. The probe holder was displaced from the equilibrium position, then released. If damping were negligible, then during system vibration the force and position would have varied in phase with one another. If in addition T force-position were zero, then the measurements of force and position also would have been in phase. Therefore, T force-position was determined by delaying one set of measurements relative to the other and finding the time shift which best aligned the two. In this configuration, the force acts along the axis of the force transducer, whereas in a typical imaging scenario, such as that depicted in Fig. 2, the shear force will dominate. This difference will not affect the temporal calibration, however, as the force transducer calculates all six force and torque measurements simultaneously. The probe holder was displaced from the equilibrium position, then released. Position and force data were recorded for 4 s during the subsequent vibration. The max- Fig. 8. Schematic representation of apparatus for force-position temporal calibration. (a) The system can be considered as a 1 DOF system with a spring and dashpot connected in parallel. The mass, m, is the equivalent mass of the force transducer body, probe holder, and arm. The mass of the force transducer top plate is much less than m. (b) Free body diagram of the top plate showing that the measured force is equal to the sum of the spring and dashpot forces. imum amplitude of the oscillation was 25 mm, which is small compared with the length of the arm (500 mm). Therefore, the motion of the probe holder can be approximated by a 1 degree-of-freedom (DOF) system: a mass suspended from a spring and dashpot connected in parallel (Fig. 8). If the damping dashpot were ignored, then the force recorded by the force transducer, F MEASURED,wouldequal ky, wherek is the spring constant and y is the displacement of the mass from equilibrium. Therefore, the force and position would be exactly in phase. The presence of damping in the system complicates matters by causing a small delay of position relative to force. The magnitude of this delay, T damping, is constant for a given system and can be calculated as shown in the Appendix. Therefore, the time shift that best aligns the force and position signals is T cal,where: T cal =T force-position +T damping. (3) The value of T cal can be found by shifting the force signal in time with respect to the position signal, and finding the shift that best aligns them. This was done using the iterative algorithm shown in Fig. 9 in which n is the iteration number. The first stage of the algorithm shifts the force signal in time by the current value of T cal, generating the shiftedforce signal. The algorithm requires an initial value for T cal (T cal (0) in Fig. 9) to be selected. Preliminary investigations indicated that T cal was approximately 0.04 s, so this value was used for T cal (0). The choice is not critical, however, because the algorithm converged to the same result for several values of T cal (0) between 0 and 0.1 s. In order to compare the force and position signals, the relationship between them must be determined. This is done in Stages 2 and 3 by plotting the shifted force against the position data and fitting a curve through the points. A third order polynomial curve is used to allow for the small nonlinearity of the spring. The polynomial curve is then used to map the position signal to the equivalent force (termed force-from-position). This is the force value that we expect at this position. T cal is then calculated by

7 1336 ieee transactions on ultrasonics, ferroelectrics, and frequency control, vol. 52, no. 8, august 2005 Fig. 10. Shifted force versus position for T cal =0.04 s, the initial value of T cal used by the algorithm. The force and position signals are not aligned in time, so the points are scattered about the approximating curve. Fig. 9. Temporal calibration algorithm to find T cal from simultaneously acquired force and position data. shifting this force-from-position signal relative to the original force signal and finding the shift that gives the minimum squared difference between the two. The minimum is found to the nearest 0.1 ms using a recursive algorithm. The whole process is repeated five times. Each time the force is shifted by the new value of T cal in Stage 1. In general, T cal converged to within 0.1 ms of its final value after two or three iterations. A simpler approach would have been to use cross correlation to determine the delay. In this case, the delay would correspond to the lag that gave the maximum cross correlation between the force and displacement signals. However, preliminary experiments with simulated signals indicated that the estimate of the delay would be biased toward zero if the data consists of an incomplete number of vibration cycles. This is the case for the data here, so the method outlined above was used instead. It does not suffer from this bias because the estimation of the delay between the force and displacement signals is decoupled from the estimation of the scaling and offset relating them. The experiment was repeated 52 times to verify that the delay being measured was constant over time. Three different lengths of ruler were used (7 cm, 9 cm, and 12 cm), forming systems with different parameters. The corresponding values of T damping were found using the method described in the Appendix. T force-position was then calculated using (3). Fig. 10 shows the shifted force signal plotted against position for a single experiment. The force has been shifted by 0.04 s, the initial value of T cal used by the algorithm. Because this is not the correct value of T cal,thepoints are scattered about the approximating curve. After five iterations, the algorithm converges to a value of T cal = s. When the force signal is shifted by this amount and plotted against position, the points lie much closer to the (new) approximating curve (Fig. 11). The effect of T cal in the time domain is illustrated in Figs. 12 and 13. The raw data (Fig. 12) shows the force leading the position. The two signals become aligned, however, when the force signal is delayed by T cal (Fig. 13). (The unit of force used in Figs figures is Counts, which is the force transducer s measurement unit. 800 Counts = 1 Newton.) The mean value of T force-position from 52 experiments was 31.5 ms, with a standard deviation of 0.61 ms and range of 3.3 ms. The values of T damping shown in the Appendix (Table I) are between 2.2 ms and 3 ms. Incorporating the effect of damping using (3) was worthwhile because otherwise the value of T force-position would have been biased by this amount. The results of the experiments show that T force-position can be found to a high level of accuracy. Therefore, error in its value will not be a significant source of error in the overall system.

8 burcher et al.: deformation correction and investigation of deformations 1337 Fig. 11. Shifted force versus position for T cal = s. This value of T cal gives the best alignment between the signals, and they now lie close to the approximating curve. Fig. 13. Shifted force and position versus time (T cal = s). The two signals are well aligned in time. Fig. 12. Force and position versus time (T cal =0.0 s). The phase difference between the two signals is clearly seen. Fig. 14. Apparatus for video-position temporal calibration. The slide restricts the motion of the probe, so that the wire in the calibration object always is seen on the ultrasound image. Image and position data are simultaneously recorded as the probe is translated horizontally. 2. Video-Position Temporal Calibration: As mentioned previously, although the latency of the position signal is fixed, the latency of the video signal is variable because it is not synchronized with the ultrasound acquisition. The aim of the calibration process is to use a large number of images to calculate the average value of the delay T video-position. This was done by acquiring a sequence of images and positions using the apparatus shown in Fig. 14. The slide restricted the probe to move along a line within the imaging plane and perpendicular to the probe s axial direction. The probe imaged a wire within the calibration object that appeared as a dot in the image. As the probe was moved from side to side along the slide, the dot moved back and forth across the image. The movement of the dot should be in phase with the position. However, a nonzero value of T video-position will delay the video signal relative to the position signal. Therefore, T video-position can be determined by delaying the video signal and finding the delay that best aligns the two sets of measurements. The delay was determined using an algorithm very similar to that used for the force-position temporal calibration (Fig. 9), except that the force signal was replaced by an image signal. The image signal was the x-coordinate of the dot center, which was manually identified in each image. The orientation of the image coordinate system ( Im x, Im y) was as shown in Fig. 14. Here, the leading superscript indicates the frame in which the quantity is measured. Because the image signal and position were linearly related,

9 1338 ieee transactions on ultrasonics, ferroelectrics, and frequency control, vol. 52, no. 8, august 2005 a straight line, rather than a polynomial, was used in step 3 to fit the shifted-image versus position data. Preliminary investigations indicated that T video-position was approximately 0.06 s and so this was used as the initial value. The choice is not critical, however, because the algorithm converged to the same result from several starting points between 0.1 and 0 s. The experiment was repeated 19 times. More experiments were not carried out because of the significant time taken to manually process the images in each data set. The algorithm failed to converge for one data set because it contained insufficient temporal features. All of the other 18 data sets were used in the subsequent analysis. The measurements of T video-position had a mean of s, a standard deviation of 6.6 ms, and a range of 22 ms. The performance is significantly better than is achievable with the schemes described in [19], [25], which only would determine the delay to within ±28 ms. The performance is comparable to that obtained using a similar, correlation-based method [18], which achieved a standard deviation of 4.9 ms using 40 temporal calibrations. B. Gravitational Calibration AsnotedinSectionII,inordertoaccuratelymeasure the contact force, it is necessary to compensate for the weight of the probe using (1). This requires the magnitude of the probe weight and its direction relative to F CONTACT to be known. The actual force measurement, F ACTUAL,differsfrom the true measurement by a bias that can be considered constant during a given experiment: Eq. (1) becomes: F ACTUAL = F TRUE + F BIAS. (4) F ACTUAL F BIAS + F CONTACT + W = 0. (5) This bias cannot be removed by zeroing the force transducer as the weight W is not known. The weight could be measured directly. However, this is not easy because a significant part of the weight is due to the cable of the ultrasound probe, and its contribution will depend on how the cable is secured to the enclosing box. However, both the weight and bias force can be determined using the calibration process described next. The localizer measures the position of the ultrasound probe relative to a world frame, which is aligned with the cameras of the Polaris localizer. The first stage of the calibration process is to determine the direction of gravity relative to the world frame. The probe is placed adjacent to a plumb line, and its position is recorded twice: once at the top of the line and once at its bottom. The difference between these two positions is normalized to give the unit vertical direction World V. During the calibration procedure, F CONTACT can be set to zero easily by ensuring that the probe is only touched by the force transducer. If the orientation of the probe is now changed, the direction of the weight changes relative to the probe s frame of reference. Because the force measurement F ACTUAL is made relative to the probe s frame of reference, this will change too. Because (5) is a vector equation, it is valid in any frame of reference, including that of the probe. Because F CONTACT = 0, (5) can be written as: Probe F ACTUAL Probe F BIAS + Probe W = 0. (6) The bias is constant in the force transducer s frame of reference, which is aligned with the probe s frame of reference. Therefore Probe F BIAS = constant. Probe W has unknown magnitude but known direction, and so: Probe W = w Probe V, (7) where w is the (unknown) magnitude of W and Probe V is a unit vector in the vertical direction mapped to the probe frame: Probe F BIAS w Probe V = Probe F ACTUAL. (8) Eq. (8) can be written as: Probe F BIAS,1 w Probe F BIAS,2 Probe F BIAS,3 Probe V 1 Probe V 2 = Probe V 3 Probe F ACTUAL,1 Probe F ACTUAL,2 Probe F ACTUAL,3. (9) Our aim is to discover Probe F BIAS and w. Thisisdone by collecting a large number of force readings with the probe in different, known orientations and solving (9) for the unknowns Probe F BIAS and w. In order to get a range of orientations, the probe is rotated smoothly about several horizontal axes by hand. Rotating the probe about an axis that does not pass through the center of mass will cause centripetal acceleration, invalidating the assumption that the probe is in equilibrium. Therefore, the axis of rotation should pass close to the center of mass and, because there may be some misalignment, the rotation also should be slow. For example, if the probe is rotated at ω =2rads 1 (one revolution every 3.1 s) about an axis that has a separation from the center of mass, r = 25 mm, the centripetal acceleration will be ω 2 r =0.1 ms 2. Therefore, the rotations are carried out with ω<2rads 1 and r<25 mm to ensure that the centripetal acceleration is negligible compared to the acceleration due to gravity. Because all the vectors are now in the probe s frame of reference, the probe superscript applies to all variables and will be omitted. The i th line of (9) can be written for each measurement recorded: F BIAS,i w V i = F ACTUAL,i (i =1 3). (10)

10 burcher et al.: deformation correction and investigation of deformations 1339 The normal equations for the three different values of i are coupled by the scalar value w. This can be accommodated by combining the equations as shown: N 0 0 V1 V1 0 0 (V1 ) 2 F BIAS,1 0 N 0 V2 F BIAS,2 0 V2 0 (V2 ) 2 F BIAS,3 = 0 0 N V3 w 0 0 V3 (V3 ) 2 FACTUAL,1 FACTUAL,1 V 1 FACTUAL,2 FACTUAL,2 V 2, (11) FACTUAL,3 FACTUAL,3 V 3 where N is the number of measurements, and sums are taken over all measurements. Eq. (11) has the form Ax = b, andsocanbesolved using the pseudo-inverse method: x =(A T A) 1 A T b. In order to solve for the calibration parameters, 2007 force-position pairs were recorded over a 36 s interval as the probe was slowly rotated. A 12 s validation data set with 718 pairs also was acquired with similar probe motions; and (11) was solved to obtain the following values: Probe F BIAS = N, w = 1.38 N These parameters were used in (4) to find F CONTACT for the validation set. Because there was no contact at the time of acquisition, the corrected output should be zero. The rms value of the corrected force magnitude was N, which is small compared to the rms value of the original force signal (7.04 N). This shows that the gravity calibration process can almost eliminate the effect of gravity on the contact force measurement. The range of angles used in the validation data set is much greater than that normally used during a typical acquisition. The error due to gravity during the acquisition, therefore, will be smaller than that shown here and is not a significant source of error in the system. In the current implementation, the gravity calibration process has to be carried out before each clinical acquisition. This is because the direction of gravity in the world coordinate system, World V, depends on the orientation of the Polaris camera unit, which is not the same for each acquisition. Also, the weight of the probe, W, will be affected by the length of the ultrasound probe cable tethered to the enclosing box. V. System Validation This section presents experiments that characterize the performance of the acquisition system, in order to evaluate its overall accuracy. The results of validation experiments are presented in which the output of the system was Fig. 15. Apparatus for validation experiments. A wooden model of the probe is used to compress a gelatine phantom. The force is recorded by both the load cell and the force transducer. The displacement is measured by the extensiometer in the load frame and the Polaris localizer. compared with that from a conventional materials testing machine (Denison Mayes Group, Leeds, UK), which was certified for static uniaxial materials testing to British Standard BS EN ISO The purposes of the validation experiments were: To verify that the force being recorded by the force transducer was the contact force. To verify that the Polaris localizer and spatially calibrated probe can accurately measure indentation for a deformable object. The apparatus used for the experiments is shown in Fig. 15. Because the ultrasound probe could not be removed from the hospital, a wooden model was made with identical dimensions. The force transducer was mounted on this in the usual way, as described in Section III, and its other side was attached to the probe holder box. The lid of the box was replaced by a thick, angled piece of Perspex, the top side of which could be screwed onto the load frame. The head of the load frame was lowered until the model probe was on top of the soft gelatine phantom. The load frame s controlling software then was used to move the probe down by a total of 5 mm in steps of 0.13 mm, then return it to the starting position, again in steps of 0.13 mm. At each position, the displacement and applied force were recorded by the load frame s extensiometer and load cell. Simultaneous recordings of the probe position and contact force were made with the MVL acquisition system. The plot of load frame displacement versus MVL displacement shows good agreement between the two sets of measurements. A straight line can be fitted to the data as shown in Fig. 16. If the measurements agreed perfectly, the gradient of this line would be exactly 1. The gradient was estimated as 1.019, indicating a small systematic difference. This might have been caused by bending in the load frame head, or using the extensiometer at a temperature other than that at which it was calibrated. In addition to this small systematic difference, there were small amounts

11 1340 ieee transactions on ultrasonics, ferroelectrics, and frequency control, vol. 52, no. 8, august 2005 of noise in both the Polaris and extensiometer readings, which contribute to a standard deviation of mm between the two measurements. However, the differences in the two readings were all less than 0.05 mm in magnitude and, therefore, are not significant in the current application. Fig. 17 shows load frame force plotted against MVL force. The slope of the graph is 1.014, indicating a small systematic difference. This could be due to calibration errors for either the load cell or the force transducer. The two measurements agree well, and the standard deviation of the error between them is N. The graph indicates that the force transducer in the MVL system records approximately the same values as the load cell in the load frame and, therefore, that it is a reasonable measurement of the contact force. VI. Conclusions Fig. 16. Load frame displacement versus MVL displacement. The two measurements show excellent agreement. This paper has presented the design and calibration of a system for simultaneously measuring force, ultrasound, and position information. The system has been used already in a clinical setting to collect data for force-based deformation correction and quantitative investigations of deformation. The results of these studies will be the subject of future publications. A number of other potential applications exist, for example, measuring the current contact force so that it can be indicated to the operator, or recording it so that a future scan can be made with a similar contact force. The system described here records B- mode ultrasound images. However, the calibration schemes described above could be applied to similar systems that record other forms of ultrasonic data, such as RF data or real-time 3-D volumes. Appendix A Mechanical Analysis for Force-Position Temporal Calibration In this Appendix, we analyze the mechanical system used for force-position temporal calibration. The system can be represented by the 1 DOF system shown in Fig. 8. If the top plate of the force transducer is assumed to be light, then the forces acting on it will balance: F MEASURED = ky + λẏ. (A1) If the system is held at a displacement y and the recorded force is F MEASURED, then, since ẏ =0,k is given by: Fig. 17. Load frame force versus MVL force. The strong agreement between the two sets of measurements validates the MVL system s force measurements. k = F MEASURED. (A2) y The force F MEASURED is applied to the mass of the arm and probe holder, m. Writing Newton s Second Law for m: F MEASURED = mÿ. (A3) Combining (A1) and (A2): mÿ + λẏ + ky =0. (A4) This is the standard equation for damped harmonic oscillation. The solution for the time-domain response y(t) is given in [26]: y = e at C 1 cos(qt), (A5) where C 1 is the initial amplitude of the motion, and: a = λ 2m, (A6) k q = m λ2 4m 2. (A7)

12 burcher et al.: deformation correction and investigation of deformations 1341 In general, (A5) will have a sine term as well as the cosine term shown. However, only the cosine term is considered here because the time origin can shift so that it coincides with a peak of y(t). The values of the parameters a and q can be determined by observing the system as it oscillates freely, and these can be used to calculate m and λ. The period of damped oscillation, T, givesq: q = 2π T. (A8) The rate at which the oscillation decays can be used to determine a. If the displacement peaks at y 1 in one cycle and y 2 in the next, then by evaluating (A5) at t =0and t = T : y 1 y 2 = e a 0 e a ( 2π q ) = ea ( 2π q ) a = q 2π ln Because the damping is small: ( y1 y 2 ). (A9) k m λ2 4m 2, hence: (A10) k q m m k q 2. (A11) Eq. (A9) and (A11) can be combined with (A6) to give a value for the dashpot coefficient, λ. We will now derive an expression for the lag between the displacement and the force caused by the damping. We still consider the case of damped free oscillation, so that the displacement is given by (A5). Differentiating (A5): ẏ = ae at C 1 cos(qt) e at C 1 q sin(qt). (A12) Substituting (A5) and (A12) into (A1): F MEASURED = k ( e at C 1 cos(qt) ) λ ( ae at C 1 cos(qt)+e at C 1 q sin(qt) ) = C 1 e at ((k λa)cos(qt) λq sin(qt)) (A13) = C 2 e at cos(qt + φ), where φ is the phase angle of the force with respect to the displacement: tan(φ) = λq k λa. (A14) This will cause the displacement to be delayed by T damping : T damping = φ 2π T =tan 1 ( ) λq 1 k λa q. (A15) By substituting in the previously calculated values for the system parameters, λ, k, a, andq, T damping can be calculated for each of the different ruler lengths used in the temporal calibration, as shown in Table I. TABLE I Values of T DAMPING for the Three Systems Used in Force-Position Temporal Calibration. Ruler length λ k a q T damping (mm) (N s m 1 ) (N m 1 ) (s 1 ) (rad s 1 ) (ms) Acknowledgment The authors would like to thank Ruth English and the staff at Churchill Hospital, Oxford, UK, for use of the Esaote ultrasound machine and probe. References [1] P. D. Freiburger, D. C. Sullivan, B. H. LeBlanc, S. W. Smith, and G. E. Trahey, Two dimensional ultrasonic beam distortion in the breast: In vivo measurements and effects, Ultrason. Imag., vol. 14, no. 4, pp , [2] M. Halliwell, Breast ultrasound offers valuable diagnostic tool, Diagnostic Imaging Europe, pp , Dec [3] S. Boudet, J. Gariepy, and S. Mansour, An integrated robotics and medical control device to quantify atheromatous plaques: Experiments on the arteries of a patient, in Proc. IEEE/RSJ Int. Conf. Intell. Robot Syst. Innovative Robotics Real World Appl., Sep. 11, 1997, pp [4] G. Bambi, T. Morganti, S. Ricci, E. Boni, F. Guidi, C. Palombo, and P. Tortoli, A novel ultrasound instrument for investigation of arterial mechanics, Ultrasonics, vol. 42, pp , Apr [5] K. A. Scanlan, Ultrasound of the breast, in Handbook of Breast Imaging. M. E. Peters, D. R. Voegeli, and K. A. Scanlan, Eds. New York: Churchill-Livingstone, 1989, pp [6] A. Gee, R. Prager, G. Treece, and L. Berman, Engineering a freehand 3D ultrasound system, Pattern Recognition Lett., vol. 24, pp , Feb [7] A. Fenster and D. B. Downey, 3-D ultrasound imaging: A review, IEEE Eng. Med. Biol. Mag., vol. 15, no. 6, pp , [8] M. R. Burcher, L. Han, and J. A. Noble, Deformation correction in ultrasound images using contact force measurements, in Proc. Workshop Math. Methods Biomed. Image Analysis, 2001, pp [9] M. Burcher, A force-based method for correcting deformation in ultrasound images of the breast, D.Phil. dissertation, Department of Engineering Science, University of Oxford, Oxford, UK, [10] M. R. Burcher and J. A. Noble, Method and apparatus for ultrasound examination, World Patent , Mar. 20, [11] L. Han, A. Noble, and M. Burcher, The elastic reconstruction of soft tissues, in Proc. IEEE Int. Symp. Biomed. Imaging, 2002, pp [12] L. Han, J. A. Noble, and M. Burcher, A novel ultrasound indentation system for measuring biomechanical properties of in vivo soft tissue, Ultrasound Med. Biol., vol. 29, no. 6, pp , [13] J. Li, J. A. Noble, L. Han, and M. Burcher, Inversion elasticity reconstruction of soft tissue using split-and-merge strategy from strain map of ultrasound image sequence, in Proc. IEEE Ultrason. Symp., 2003, vol. 2, pp [14] L. Han, M. Burcher, and J. A. Noble, Non-invasive measurement of biomechanical properties of in vivo soft tissues, in Proc. Med. Image Comput. Comput.-Assisted Intervention, 2002, vol. 1, pp