Prospective Motion Correction of X-ray Images for Coronary Interventions

Transcription

1 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 1 Prospective Motion Correction of X-ray Images for Coronary Interventions Guy Shechter, Barak Shechter, Jon R. Resar, Rafael Beyar Guy Shechter is with the Rappaport Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, 31096, Israel. ( gshec[at]tx.technion.ac.il). Barak Shechter is with the Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, MD 21201, USA. Jon R. Resar is with the Division of Cardiology, Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA. Rafael Beyar is with the Rappaport Faculty of Medicine, Technion - Israel Institute of Technology, Haifa, 31096, Israel. c 2004 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

2 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 2 Abstract A method for prospective motion correction of x-ray imaging of the heart is presented. A 3D+t coronary model is reconstructed from a biplane coronary angiogram obtained during free breathing. The deformation field is parameterized by cardiac and respiratory phase, which enables the estimation of the state of the arteries at any phase of the cardiac respiratory cycle. The motion of the 3D coronary model is projected onto the image planes and used to compute a dewarping function for motion correcting the images. The use of a 3D coronary model facilitates motion correction of images acquired with the x-ray system at arbitrary orientations. The performance of the algorithm was measured by tracking the motion of selected left coronary landmarks using a template matching cross correlation. In three patients, we motion corrected the same images used to construct their 3D+t coronary model. In this best case scenario, the algorithm reduced the motion of the landmarks by 84-85%, from mean RMS displacements of pixels to pixels. Prospective motion correction was tested in five patients by building the coronary model from one dataset, and correcting a second dataset. The patient s cardiac and respiratory phase are monitored and used to calculate the appropriate correction parameters. The results showed a 48-63% reduction in the motion of the landmarks, from a mean RMS displacement of pixels to pixels. Index Terms Motion compensation, X-ray angiography, Chest imaging I. INTRODUCTION X-ray imaging is routinely used to guide intravascular therapy. Percutaneous transluminal coronary angioplasty and stent placement are two common cardiac procedures that are performed under x-ray fluoroscopy. First, a contrast-enhanced angiogram is acquired to diagnose and locate stenoses in the coronary artery tree. Then, a catheter carrying a balloon, or stent, is advanced to the site of the lesion and deployed under image guidance. Accurate placement of the therapeutic catheter is complicated by several factors. Contrast agent toxicity limits the number of times that the arteries can be opacified, such that guidance is performed in a fluoroscopic imaging mode, when the arteries are not visible. The interventional cardiologist must mentally register the diagnostic angiograms that depict the location of the stenosis to the fluoroscopic images that only show the catheter. Cardiac and respiratory motion of the heart can introduce large changes in image position of a given stenosis, requiring the

3 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 3 clinician to continuously estimate the target position. Final placement of the catheter is verified with small puffs of contrast agent. In this paper, we present a method for motion correcting x-ray images of the heart. The algorithm relies on motion information derived from a patient specific 3D+t (three dimensional + time) model of the coronary arteries. Methods for generating these models from diagnostic coronary angiograms acquired on biplane and rotational single plane systems have been described previously [1], [2]. We then perform a cardiac respiratory parametric decomposition of the 3D motion field, which allows us to predict the correction parameters for images acquired at any arbitrary cardiac respiratory phase [3]. Eck et. al. described an image based method for synthesizing angiographic images at any cardiac and respiratory phase [4]. Their warping function is view-angle dependent, and must be recomputed if the imaging C-arm is reoriented. In contrast, our method uses a 3D deforming model of the coronary tree, which makes it possible to prospectively correct images acquired at any imaging angle. A. X-ray projection imaging II. METHODS X-ray projection imaging is a 3D to 2D imaging process that can be described by a 3x4 projection matrix P : P = ( n u)(sid) n 0 u IS 2 ( n 0 v)(sid) n v IS R pa R sa 0... SOD which is the product of a 3x3 matrix representing a perspective projection, and a 3x4 matrix describing the orientation of the imaging system relative to a world coordinate system. n u and n v are the image dimensions in pixels. The 3x3 matrix R pa R sa represents the orientation of the imaging C-arm, as defined by the primary and secondary angles (Fig. 1). IS is the intensifier size, SOD is the source-to-isocenter distance, and SID is the source-to-intensifier distance (Fig. 2). Figure 3 is a diagram of the imaging coordinate system. The imaging process is modeled as a perspective projection along the -z axis. In this mathematical description, the patient is oriented within a fixed camera coordinate system. The primary angle rotation, R pa, is a straightforward (1)

4 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 4 rotation about the +y axis, with a positive angle rotation corresponding to an LAO view: cos(pa) 0 sin(pa) R pa = (2) sin(p A) 0 cos(pa) If PA=0, then the secondary angle rotation R sa is simply a rotation about the +x axis, with a positive angle rotation corresponding to a cranial view: R sa = 0 cos(sa) sin(sa). (3) 0 sin(sa) cos(sa) In most cases, PA 0, and a new axis of rotation must be computed for the secondary rotation. The new axis u is obtained by rotating the x axis by -(PA) around the +y axis. Using the transpose of R PA as the negative rotation, v = R T pa [ 1 0 0] T (4) and u = (u x,u y,u z ) T = v/ v. The secondary angle rotation matrix is then computed using ( ) 0 u z u y R sa = uu T + cos(sa) uut +sin(sa) u z 0 u x. (5) u y u x 0 which represents a 3D rotation about an arbitrary axis u [5]. In homogeneous coordinates, multiplying the 3D point q = (q x,q y,q z, 1) T by the projection matrix P, generates the 2D point r = (r u,r v,r w ) T : r = Pq. (6) The image coordinates of this point are obtained by dividing r by r w. B. Imaging protocol Patients undergoing diagnostic left heart catheterization were recruited to participate in an IRB approved protocol. All images were acquired on a Siemens biplane cardiovascular angiography system at a rate of 30 frames per second. The clinician was not constrained in positioning

5 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 5 the C-arms. The technicians were requested to not move the patient table during the image acquisition. Images of a calibration grid and phantom were acquired with the imaging system in the same configurations (primary and secondary angles, source-intensifier distance) used to acquire patient data. The grid of metal beads (1 cm horizontal and vertical spacing) was used to correct for the geometric distortion introduced by the image intensifier [6]. The plastic rectangular phantom ( cm) containing 18 metal beads was reconstructed and compared to its known geometry in order to optimize the imaging parameters and validate the accuracy of the 3D reconstruction [7]. C. Cardiac and Respiratory Phase A personal computer with digital acquisition system recorded the ECG and an image acquire signal from the Siemens digital acquisition subsystem (Fig. 4a). The cardiac phase (χ) of each image was then calculated so that χ represents the percentage of the cardiac cycle, with the QRS peak at χ = 0. Beat-to-beat variations in heart rate were normalized by calculating the systolic and diastolic intervals using the method of [8]. The systolic interval was then linearly rescaled between [0, 0.42), which corresponds to the systolic interval of a 60 beat/minute heart rate. The diastolic interval was rescaled between [0.42, 1). Thus, each dataset had a cardiac phase 0 χ < 1. Respiratory phase (ρ) was measured from the movement of the diaphragm as seen in the x-ray images. A profile perpendicular to the diaphragm lung interface was manually positioned on a first image (Fig. 4b), and all the images in a dataset were sampled at this location. The displacement of the diaphragm-lung interface over time generates a one dimensional curve (Fig. 4c) that was manually segmented, with the maximum and minimum values corresponding to endexpiration (EE, ρ = 0) and end-inspiration (EI, ρ = ±1) respectively. Intermediate values of ρ were computed as a linear distance between the respiratory extrema. The sign of ρ depends on whether it is an inspiratory (-) or expiratory (+) maneuver. D. Patient Specific Coronary Motion Models A patient specific model of coronary artery motion was used to compute the motion correction parameters. The model was reconstructed from biplane coronary angiograms using a stereo-vision

6 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 6 reconstruction and automatic motion tracking method [1]. The angiogram was acquired with a hand-injected bolus of contrast lasting 4-8 seconds while the patient was freely breathing. First, a static 3D model of the coronary arteries, Υ 0, was reconstructed from one image biplane image pair at end diastole. Using a semi interactive segmentation tool, a user segmented the arterial centerlines in one pair of projection images. This process was constrained by a potential field generated using multi-scale filters which detect rectilinear structures such as vessels [9]. The epipolar constraint was used to define point to point correspondences between the arteries two projections, and a tree of 3D cubic B spline curves was constructed using a dynamic programming algorithm [10]. Υ 0 was represented within a patient-specific coordinate system. The x-axis corresponds to the patient s right-to-left axis, with +x directed to the patient s left. The +y axis points toward the patient s feet, with the +z axis directed toward the posterior. A 0 LAO/RAO, 0 cranial/caudal x-ray view corresponds to a projection along the z axis of the patient s coordinate system. The arterial model Υ 0 served as the initialization for an automatic motion tracking algorithm that recovered the motion of the arteries from the cine-angiogram images. The algorithm computes the transformations M : R 3 R 3 which keep the arteries projection onto the image planes consistent with the temporally changing images [1]. If Υ 0 was reconstructed from image pair t 0, then the transformation M : R 3 R 3 maps any 3D point q 0 Υ 0 to the point s 3D position consistent with image t 0 + t: q t = M(q;t), (7) which we also write as Υ t = M[Υ 0,t] to represent the deformation of the whole arterial tree. Since every image t represents a cardiac and respiratory phase (χ t,ρ t ), M(q;t) M(q;χ t,ρ t ) (8) The deformation field M was recovered from free breathing angiograms that provided a discrete sampling of different cardiac and respiratory phases. In order to perform motion correction of x-ray images, we needed to represent the deformation field for any cardiac and respiratory phase combination. A cardiac respiratory parametric model, M, was fit to the observed data M [3]. The model uses approximating B spline basis functions to separate the motion into a cardiac contraction motion and a respiratory motion: M(q;χ,ρ) M(q;χ,ρ) = M χ (q;χ) + M ρ (q;ρ) (9)

7 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 7 where M χ is the motion due to the cardiac contraction, and M ρ is the respiratory motion of the heart. Fundamentally, splines are smooth piecewise polynomial functions that can be used to interpolate data points. They provide the mechanism that allows us to estimate the deformation field at any cardiac-respiratory phase based on the data recovered from the angiograms. E. Motion correction of X-ray images The acquisition of x-ray images of the heart at cardiac phase χ and respiratory phase ρ is described by the equation r = P M(q;χ,ρ) (10) where P represents the geometric configuration of the x-ray imaging system (Eq. 1), and M is the transformation of points of a coronary tree from some reference state, q Υ 0. r is the 2D image position of the 3D coronary point q. M was reconstructed from one free-breathing coronary angiogram for each patient. The coronary tree Υ 0 was generated from images acquired at (χ 0,ρ 0 ). We expressed the image projection of the coronary tree at the reference time by the set of points {r 0 }: r 0 = P M(q;χ 0,ρ 0 ). (11) At some later time, at cardiac respiratory phase (χ i,ρ i ), the coronary arteries could be found at {r i } in the image planes: r i = P M(q;χ i,ρ i ). (12) The 2D projection of the 3D motion of a point q from its reference position to its position at cardiac respiratory phase (χ i,ρ i ) can be expressed as the vector r0 r i. To motion correct an x-ray image, we first formed the point set {q} by sampling the 3D coronary tree Υ 0 at discrete points separated by 1 mm along the length of the arteries. The set of 2D displacements, { r0 r i }, was then computed for the set {q}. We formulated a 2D image warping function d : R 2 R 2 as a 2-dimensional B-spline function: d(r) = n i 1 i=0 n j 1 j=0 b i (r x )b j (r y )u ij (13) where {u ij = (u x ij,u y ij ) R2 } are the n i n j control points, and {b} are cubic B-spline basis functions defined over a uniform knot vector. The knot vector range spanned the dimensions of the image and a 10% boundary (i.e. [-51, 563] for a 512 pixel image).

8 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 8 The B-spline control points {u} were computed so that Eq. (13) approximated the set of observed displacements { r0 r i } of the projected coronary tree points. The solution was recovered using a linear minimum mean-square estimator with a second derivative smoothing constraint [11], [12]. A bounding box was defined in the image plane around the projection of the coronary tree model. Outside this region, we added zero displacement vectors at regularly spaced intervals of 20 pixels in the horizontal and vertical directions. This was done so that the border regions of the image would remain unaffected by the warping function. After fitting the 2D warping function to the vector field for an image I χ,ρ, we generated a motion compensated image Ĩχ,ρ. The 2D warping function from Eq. (13) was evaluated at integer values corresponding to image pixel coordinates of Ĩχ,ρ i.e. (0, 0), (0, 1), (0, 2),...,(512, 512). The function returned the 2D vector offset corresponding to the pixel s position in I χ,ρ, which was then sampled using bilinear interpolation. F. Performance Evaluation The performance of the motion correction algorithm was measured by tracking landmarks on the coronary tree. A set of landmarks was manually selected in the reference image near (1) the left main (LM) left anterior descending (LAD) bifurcation; (2) the origin of the first obtuse marginal (OM) branch from the left circumflex artery (LCx); and (3) the origin of the first or second diagonal (D) branch off the LAD [13]. After identifying a landmark in the reference image, its position was automatically tracked in the rest of the dataset s images. A pixel template T was generated around the landmark. A cross correlation metric was computed between the k-th landmark s template, T k, and image I χ,ρ : c(x,y) = N/2 N/2 i= N/2 j= N/2 T k (i,j) I χ,ρ (x + i,y + j). (14) The position of the k-th landmark in the image was determined by finding the peak of the cross correlation function. Since the coronary arteries are deforming during a cardiac cycle, this rigid template matching procedure was ineffective for isolated cardiac phases at some landmarks. Mistracked points were identified visually and corrected by manually adjusting the segmentation. For each landmark, we computed a baseline measure of motion by computing the 2D distance between the landmark s position in the reference image, and its position in the other images in

9 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 9 the original angiogram dataset. These values represent the amount of motion observed due to the cardiac contraction and respiratory motion of the heart. The root mean square (RMS) of these values provides a scalar measure of the landmark s variation throughout the image frames. The position of each landmark was similarly tracked in the motion corrected images, and its 2D distance from the landmark s reference position was computed. A perfect motion correction would produce a zero displacement measure for all the images in the dataset. III. RESULTS In five patients, we acquired two coronary angiograms and used one to generate a 3D+t coronary model. The average length of the reconstructed tree was 24±7 cm (range, cm). For each patient, the cardiac respiratory parametric model was fit using an average of 98±12 biplane image pairs. This corresponds to 3.3±0.4 seconds of free breathing data based on a frame rate of 30 frames per second. A. Retrospective Motion Correction The motion correction algorithm was first applied to the same images that were used to create the 3D+t coronary motion model. The algorithm requires that the user specify two parameters of the image-plane warping function: (1) the control point density (n i n j ) and (2) the smoothing parameter. We tested various combinations of these parameters on left coronary angiograms from three patients. Figure 5 shows the displacement plots for the three left coronary landmarks in a biplane dataset. The dashed lines represent the motion of the coronary landmarks in the original images. A periodic motion spanning three to four cardiac cycles is seen. The solid lines show the 2D displacement of each landmark after applying motion correction with a control point density of 6 6 and a smoothing factor of At five of the six points that are shown, there was an order of magnitude reduction in the 2D motion after motion correction. The behavior of the LAD-Diagonal bifurcation in the LAO projection is an artifact of the template matching algorithm. In this patient, contrast material flowed back into the aorta during the bolus injection. When the aortic valve was closed during diastole, the accumulation of contrast material opacified the aortic root, which overlapped the bifurcation and caused the

10 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 10 template matching to fail. Notice that the peaks in the motion corrected trace are out of phase with the peaks (systolic displacement) of the original trace. Figure 6 shows the image results from another patient. Image frame 132 (middle column) is shown next to the reference image (image 81). The reference image was acquired at middiastole, near end-expiration, while frame 132 was acquired at end-diastole, near end-inspiration. The third column of the image shows frame 132 after motion correction (compare to reference image in first column). Table I shows the average 2D RMS displacement for three left coronary landmarks on two x-ray projections (N=6). With control point grid size ranging from 6 6 to and smoothing factors ranging from to 1.0, the average RMS displacements remained between 2-3 pixels. In contrast, without motion correction, the average RMS displacements were 12.8±1.8, 14.3±1.3, and 14.6±1.8 pixels for the three patients. With a control point grid size of 10 10, and a smoothing factor of 0.01, the mean RMS displacements were 2.1±1.7, 2.2±1.4, and 2.2±2.1 pixels. This represents an 84% to 85% reduction in motion in these three cases. B. Prospective Motion Correction Prospective motion correction was studied by motion correcting a second dataset for each patient. That is, the 3D+t cardiac respiratory parametric motion model was generated from one dataset. It was then used to correct another biplane acquisition, acquired with different projection angles, at a later time. A control point grid size of and a smoothing parameter of 0.01 were used. Analysis was performed on 100±10 images per dataset. For each patient, the mean RMS displacement of the landmarks in the original images and in the motion corrected images is plotted in Figure 7. Each bar represents the motion of three coronary landmarks in two projection views (N=6). The mean RMS displacement in the original images was between 11.5 and 13.6 pixels, whereas the motion corrected displacements were between 4.4 and 7.1 pixels. The motion of the landmarks was reduced by 48-63% in these five datasets. The difference between the original and motion corrected measurements was statistically significant for each patient (two-tailed t-test for paired samples, significance level α=0.05).

11 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 11 IV. DISCUSSION In the first set of experiments, we tested the motion correction algorithm in a retrospective mode in which the 3D+t coronary model was generated from the angiogram images that were then corrected. This allowed us to study the motion correction algorithm, without immediately addressing the quality of patient-specific motion models for prospective correction of de novo images. The results showed that the performance of the algorithm remained flat over over a wide range of control point grid sizes and B-spline smoothing factors. The second set of experiments tested the practical prospective use of this algorithm. In the clinic, the patient-specific motion model would be constructed from one angiogram at the start of a procedure and then be used to motion correct angiographic and fluoroscopic images acquired subsequently. Although we performed our motion correction off-line in this study, the methods can be used in an on-line implementation. The only requirements are that the cardiac and respiratory phase be measured in real-time, and that the geometry of the imaging system be registered to the geometry used in building the patient specific coronary model. In the current set of experiments, we requested that the patient table not be moved during the acquisition. This translational motion could also be easily tracked and incorporated into the correction algorithm. The orientation of the imaging system (primary angle, secondary angle) was held constant during an acquisition, but was moved between acquisitions. Newer rotational angiography systems are designed to spin around the patient during a bolus injection. This should not pose a problem for the motion correction algorithm, as long as the orientation parameters can be obtained in real time. However, the rotation will likely interfere with measuring the displacement of the diaphragm. The changing view angle will capture different portions of the diaphragm over time, changing the profile of the diaphragmatic silhouette. The use of an external respiratory signal, such as a pneumatic bellow placed around the chest wall, could be an alternative measure of respiratory phase. The mean RMS displacements obtained with the prospective correction ( pixels) were larger than were measured in the first set of experiments ( pixels). This observation raises the possibility that the cardiac respiratory motion model has limitations. Consider that the cardiac respiratory motion model is generated from 3-4 seconds worth of data that spans a few heartbeats and approximately one respiratory cycle. Additional data could be used to improve the

12 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 12 reliability of the model. However, a typical bolus of contrast agent is 10 cc administered over 4-8 seconds. The bolus interferes with normal blood flow in the coronaries which precludes longer injections. Additional unessential injections are not tolerated due to concerns over radiation exposure and renal toxicity from total contrast dose. The model requires two inputs: cardiac and respiratory phase. Two parameters may be insufficient considering the variability that exists physiologically. The state of the heart during a cardiac contraction is dependent on multiple factors including preload, afterload, and contractility, and may not be adequately indexed by the ECG signal used to calculate cardiac phase. Respiratory phase is measured from diaphragmatic displacement, whereas breathing is also a function of chest wall motion. Finally, the cardiac-respiratory parametric motion model assumes that the cardiac motion and the respiratory motion are independent. In reality, cardiac and respiratory function are interdependent [14], [15]. The quality of the motion correction also depends on the completeness and accuracy of the 3D+t coronary tree model. Arteries that are not included in the 3D coronary model cannot influence the dewarping function. Inspection of the warped images showed that artifacts sometimes developed in areas not supported by coronary arteries (Fig. 8). The artifacts were more pronounced when the warping function was constructed using a large number of control points and small smoothing factor. Solutions to this problem include using a more constrained deformation field, or by reconstructing a more complete coronary tree model. Another source of error could be a 3D-2D misregistration between the model coordinate system and the imaging projection space. This would happen if the table or the patient moves between acquisitions. For the experiments we presented in this paper the arterial tree was visible in both sets of angiograms, the one used for generating the model, and the one subsequently used to test the prospective motion correction algorithm. To reduce the effect of this misregistration, we performed a rigid body 3D-to-2D registration of the reconstructed static coronary tree model to a pair of images from the second acquisition having nearly the same cardiac and respiratory phase as the model. This solution could not be used when motion correcting fluoroscopic images of a guiding catheter, when the arteries are not opacified and therefore not visible. For the general case, we propose the use of external radio-opaque fiducial markers to maintain the registration between the arterial model coordinate system and the imaging projection space.

13 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 13 A. Comparison to Image Based 2D Methods Motion stabilization of angiograms has been studied previously [16], [17], [18], [19]. However, those techniques are image-based methods in which the motion of the arteries is analyzed in the 2D images and subsequently corrected. The paradigm we present in this paper uses information from patient specific 3D+t anatomic models to effect motion correction. The power of the proposed method is in its ability to correct images in which the vessels are not visible. During a coronary intervention, the cardiologist inserts a guidewire catheter through the coronary tree under x-ray guidance. This is done blindly, in the sense that the arteries are not filled with contrast agent, and are therefore not visible. Our model based method can be used to stabilize the physiologic motion of the catheter in fluoroscopic images to assist the cardiologist in reaching a predetermined target stenosis location. An additional advantage of our method over 2D image based methods is that it is view angle independent. The anatomic models are initially generated from angiograms acquired with the x-ray C-arms in some orientation. Since the anatomic models are 3D objects, they can be rotated and translated to match a re orientation of the x-ray C-arms. Thus, the same anatomic model can be used to motion correct images acquired from different projection angles (Section III-B). B. Computation Time The computational complexity of the prospective motion correction approach we describe was studied in two parts. 1) Preparation Phase: First, there is the preparation phase during which a patient specific coronary motion model is generated. In this paper we propose generating this model from coronary angiograms acquired at the beginning of a cardiac catheterization procedure. This process should ideally be completed within minutes, corresponding to the amount of time between the initial diagnostic angiographic portion of the catheterization, and the subsequent intervention to deliver therapy. We relied on a semi-automatic technique for constructing an initial 3D model of the coronary arteries, which required about 20 minutes of user interaction [10]. Tracking the 3D motion of a coronary tree model through a biplane x-ray acquisition took 16±22 minutes per frame for the five datasets we used. The actual speed is proportional to the number of arteries being tracked, the spatial resolution of the model and deformation field, and the optimization termination criteria.

14 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 14 Then, fitting the cardiac respiratory parametric model to the motion field added an additional 74±21 minutes of processing time for each dataset. Thus, building a patient specific coronary motion model from a 4 second acquisition acquired at 30 frames per second, took more than 30 hours (20 min + 16 min/frame ( 4 30 frames) + 74 min). The tracking and modeling algorithms are currently implemented in MATLAB and little effort was made to optimize their performance. Reimplementation in a compiled language with better memory management should significantly speed up the algorithms. In addition, a simple gradient descent technique with stringent termination criteria was used to optimize the cost functions for motion tracking [1]. Conjugate gradient methods with variable step size should be studied for improving the performance of the algorithm. Finally, the paradigm we propose for prospective motion correction does not require that the 3D+t anatomic models be generated with the method we used. For example, Blondel et al. described a method for automatic reconstruction of coronary models from a rotational x-ray system which would eliminate the need to manually reconstruct the initial coronary tree [20]. They also described a 4D motion tracking method that may converge faster due to additional constraints on the temporal smoothness of the recovered motion field [2]. 2) Online Motion Correction: The second part of our prospective motion correction technique involves (1) estimating the patient s physiologic state, (2) predicting the current arterial state using the precomputed motion model, (3) computing the image warping function that effects the motion correction, and (4) interpolating a new motion corrected image. These series of events have to be completed in real time for each acquired x-ray image so that it may be used on-line during an intervention. The motion correction algorithms from Section II-E were implemented in MATLAB and were run on a Linux system with an Intel Xeon TM 2.8 GHz processor. Some code fragments were implemented in C MEX functions for faster performance. The following computation times were measured in one of the patient data sets. Projecting the 3D coronary tree displacement into the 2D imaging plane (Eqs. [11] and [12]), and fitting the B-spline image warping function (Eq. [13]) was accomplished in 52.7±0.5 milliseconds. Evaluating the warping function for every pixel in a image was done in 429.5±1.9 milliseconds. Finally, generating the motion corrected image through bilinear interpolation of the original image was accomplished in 327.2±1.9 milliseconds.

15 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 15 Real time measurement of physiologic parameters was not addressed in our work. However, work with navigator echos for respiratory gated MR imaging has shown that it is possible to measure diaphragmatic displacement from a one-dimensional grayscale signal similar to that shown in Figure 4(b) in less than 22 milliseconds [21]. Taken together, the steps required for online motion correction currently take approximately one second to execute. When implemented using pipeline processing, this time can be regarded as the lag, or delay, introduced by the motion correction algorithm. The use of hardware processing, instead of the current software implementation, will reduce this computation time to tens or hundreds of milliseconds. A real-time processing delay of this order makes the motion correction algorithm clinically practical. V. CONCLUSION We developed a method for motion correcting x-ray images of the heart. Correction factors are computed using a patient specific 3D+t coronary motion model that is constructed from a biplane coronary angiogram acquired during free breathing. This 3D model allows the correction to be subsequently computed for any orientation of the imaging C-arm, and for images acquired at any cardiac and respiratory phase combination. Prospective motion correction reduced the in-plane motion of three landmarks on the left coronary tree by 48-63% in five patients. REFERENCES [1] G. Shechter, F. Devernay, E. Coste-Manière, A. Quyyumi, and E. McVeigh, Three dimensional motion tracking of coronary arteries in biplane cineangiograms, IEEE Trans. Med. Imaging, vol. 22, no. 4, pp , Apr [2] C. Blondel, G. Malandain, R. Vaillant, and N. Ayache, 4D deformation field of coronary arteries from monoplane rotational X ray angiography, in CARS Computer Assisted Radiology and Surgery. Proceedings of the 17th International Congress and Exhibition, June 2003, pp [3] G. Shechter, C. Ozturk, J. Resar, and E. McVeigh, Respiratory motion of the heart from free breathing coronary angiograms, IEEE Trans. Med. Imaging, vol. 23, no. 8, pp , Aug [4] K. Eck, I. Wachter, and J. Bredno, Synthesis of angiographic images using iterative approximation, in Proc. SPIE Med. Imaging, J. Fitzpatrick and M. Sonka, Eds., vol. 5370, 2004, pp [5] O. A. R. Board, M. Woo, J. Neider, T. Davis, and D. Shreiner, OpenGL Programming Guide, 3rd ed. Addison Wesley, 1999, pp [6] E. Gronenschild, The accuracy and reproducibility of a global method to correct for geometric image distortion in the x-ray imaging chain, Med. Phys., vol. 24, pp , [7] G. Shechter, F. Devernay, E. Coste-Manière, and E. McVeigh, Temporal tracking of 3D coronary arteries in projection angiograms, in Proc. SPIE Med. Imaging, M. Sonka and J. Fitzpatrick, Eds., vol. 4684, May 2002, pp

16 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 16 [8] A. Weissler, W. Harris, and C. Schoenfeld, Systolic time intervals in heart failure in man, Circulation, vol. 37, pp , [9] A. Frangi, W. Niessen, R. Hoogeveen, T. van Walsum, and M. Viergever, Model based quantitation of 3-D magnetic resonance angiographic images, IEEE Trans. Med. Imaging, vol. 18, no. 10, pp , Oct [10] F. Mourgues, F. Devernay, G. Malandain, and E. Coste-Manière, 3D+t modeling of coronary artery tree from standard non simultaneous angiograms, in Proc. MICCAI, vol. 2208, Oct. 2001, pp [11] J. Declerck, T. Denney, C. Ozturk, W. O Dell, and E. McVeigh, Left ventricular motion reconstruction from planar tagged MR images: a comparison, Phys. Med. Biol., vol. 45, pp , [12] T. Denney and J. Prince, Reconstruction of 3-D left ventricular motion from planar tagged cardiac MR images: An estimation theoretic approach, IEEE Trans. Med. Imaging, vol. 14, no. 5, pp , Dec [13] W. Austen, J. Edwards, R. Frye, G. Gensini, V. Gott, L. Griffith, D. McGoon, M. Murphy, and B. Roe, A reporting system on patients evaluated for coronary artery disease, Circ., vol. 51, no. 4 Suppl, pp. 5 40, Apr [14] W. Summer, S. Permutt, K. Sagawa, A. Shoukas, and B. Bromberger-Barnea, Effects of spontaneous respiration on canine left ventricular function, Circ. Res., vol. 45, pp , [15] S. Scharf, R. Brown, N. Saunders, and L. Green, Effects of normal and loaded spontaneous inspiration on cardiovascular function, J. Appl. Physiol., vol. 47, no. 3, pp , Sept [16] N. Eigler, M. Eckstein, K. Mahrer, and J. Whiting, Improving detection of coronary morphological features from digital angiograms. Effect of stenosis-stabilized display, Circ., vol. 89, no. 6, pp , June [17] S. Roehm, J. Smith, and R. Mangalik, Digital x-ray imaging system with automatic tracking, US Patent 5,293,574, Mar. 8, [18] M.-P. Dubuisson-Jolly, C. Liang, and A. Gupta, Optimal polyline tracking for artery motion compensation in coronary angiography, in Sixth International Conference on Computer Vision (ICCV), 1998, pp [19] E. Meijering, W. Niessen, and M. Viergever, Retrospective motion correction in digital subtraction angiography, IEEE Trans. Med. Imaging, vol. 19, no. 1, pp. 2 21, Jan [20] C. Blondel, R. Vaillant, F. Devernay, G. Malandain, and N. Ayache, Automatic trinocular 3D reconstruction of coronary artery centerlines from rotational X-ray angiography, in Computer Assisted Radiology and Surgery 2002 Proceedings. Paris: Springer Publishers, Heidelberg, June 2002, pp [21] Y. P. Du, M. Saranathan, and T. Foo, An accurate, robust, and computationally efficient navigator algorithm for measuring diaphragm positions, J. Cardiovasc. Magn. Reson., vol. 6, no. 2, pp , Apr

17 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 17 RAO LAO Cranial Caudal Primary Angle Secondary Angle Fig. 1. The primary angle (PA) and secondary angle (SA) define the geometric orientation of the imaging system with respect to the patient. Zero degree primary and secondary angles correspond to an anterior-posterior projection. The primary angle diagram is viewed from the patient s feet.

18 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 18 Image Intensifier IS Isocenter SOD SID X-ray source Fig. 2. Diagram of an x-ray imaging system. An x-ray source and image intensifier are arranged on a C-arm gantry. The isocenter is defined as the point of rotation of the imaging arm. The SOD, or source to object distance, is measured to the isocenter, where the imaged object is typically placed. SID is the source to image intensifier distance, and IS is the intensifier size. The figure is adapted from the Series 9800 Mobile C-Arm Operator s Guide (OEC Medical Systems Inc).

19 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 19 Y Z X Image Intensifier X-ray Source 90 LAO c Arai/Shechter Fig. 3. The coordinate system of the imaging C-arm. For the mathematical description of the imaging process, different projections of the patient are obtained by rotating the patient in the camera s fixed frame of reference. (Drawing courtesy of Yaniv Arai.)

20 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 20 (a) ECG X ray Trigger χ=0.5 χ=0 χ= Time (s) (b) (c) ρ = 0 End Expiration ρ=0 ρ =1 ρ = 1 End Inspiration ρ = 1 End Inspiration Image Number Seconds Fig. 4. (a) Cardiac phase (χ) was measured by digitizing an ECG in synchrony with the image acquisition. (b) Respiratory phase (ρ) was measured by tracking the displacement of the diaphragm along a profile in the angiogram images. (c) The displacement of the lung diaphragm interface is shown as a function of image number [3].

21 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) RAO, 0 Cranial 54 LAO, 2 Caudal 2D distance from first frame (pixels) Original Motion Corrected LM bifurcation 2D distance from first frame (pixels) Original Motion Corrected LM bifurcation Image Frame Number Image Frame Number 2D distance from first frame (pixels) Original Motion Corrected LCx OM bifurcation 2D distance from first frame (pixels) Original Motion Corrected LCx OM bifurcation Image Frame Number Image Frame Number 2D distance from first frame (pixels) Original Motion Corrected LAD Diag bifurcation 2D distance from first frame (pixels) Original Motion Corrected LAD Diag bifurcation Image Frame Number Image Frame Number Fig. 5. Two dimensional displacement of three left coronary tree landmarks in two projection angiograms. The dashed lines represent the displacement in the original images. The solid line is the displacement of the point after motion correcting the x-ray projection images. The behavior of the LAD-Diagonal bifurcation in the LAO projection is an artifact of the template matching algorithm used to track the landmarks. Contrast accumulation in the aortic root during diastole interfered with localizing the nearby bifurcation.

22 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 22 Fig. 6. Motion correction of an x-ray angiogram acquired during free breathing. The images from the middle column were corrected, with the images from the first column serving as the reference state. Compare the results in the third column, to the reference images from the first column.

23 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 23 TABLE I THE EFFECT OF CONTROL POINT DENSITY AND B-SPLINE SMOOTHING FACTOR ON THE QUALITY OF MOTION CORRECTION OF THE LEFT CORONARY TREE. THE 2D DISPLACEMENT OF A CORONARY LANDMARK WAS MEASURED FROM A REFERENCE IMAGE. A SCALAR MEASURE OF THE LANDMARK S MOTION IN TIME THROUGH ONE DATASET WAS COMPUTED AS THE RMS OF THE DISPLACEMENTS. THE REPORTED VALUES (IN PIXELS) ARE THE MEAN AND STANDARD DEVIATION OF THE RMS DISTANCES FOR THREE LANDMARKS IN TWO PROJECTIONS (N=6). Patient I: Without motion correction, the RMS displacement was 14.6±1.8 pixels. Number of Smoothing Factor Control Points ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± ± 1.7 Patient II: Without motion correction, the RMS displacement was 12.8±1.8 pixels. Number of Smoothing Factor Control Points ± ± ± ± ± ± ± ± ± ± ± ± 1.5 Patient III: Without motion correction, the RMS displacement was 14.3±1.3 pixels. Number of Smoothing Factor Control Points ± ± ± ± ± ± ± ± ± ± ± ± 1.6

24 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) RMS 2D Displacement Original Images Motion Corrected Images * P<0.004 * P< * P< * P< * P< I II III IV V Patient Number Fig. 7. Prospective motion correction of left coronary angiograms in five patients. The 3D+t coronary model was reconstructed from a first angiogram, and used to motion correct another angiogram acquired at a later time. Mean RMS 2D displacement was measured in the original images and in the motion corrected images, at three coronary landmarks in two projection views (N=6). In three of the five patients, the displacement in the motion corrected images was significantly less than in the original images (paired t-test, α = 0.05).

25 IEEE TRANSACTION ON MEDICAL IMAGING (PREPRINT) 25 Fig. 8. Artifacts can appear in the motion corrected images when using a dense 2D B-spline control point grid (n i = n j = 14), and a weak smoothing parameter (0.001).