Reconstruction of Time-Varying 3-D Left-Ventricular Shape From Multiview X-Ray Cineangiocardiograms

Transcription

1 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 7, JULY Reconstruction of Time-Varying 3-D Left-Ventricular Shape From Multiview X-Ray Cineangiocardiograms Masamitsu Moriyama, Yoshinobu Sato*, Hiroaki Naito, Masayuki Hanayama, Takashi Ueguchi, Toshinobu Harada, Fujiichi Yoshimoto, and Shinichi Tamura Abstract This paper reports on the clinical application of a system for recovering the time-varying three-dimensional (3-D) left-ventricular (LV) shape from multiview X-ray cineangiocardiograms. Considering that X-ray cineangiocardiography is still commonly employed in clinical cardiology and computational costs for 3-D recovery and visualization are rapidly decreasing, it is meaningful to develop a clinically applicable system for 3-D LV shape recovery from X-ray cineangiocardiograms. The system is based on a previously reported closed-surface method of shape recovery from two dimensional occluding contours with multiple views. To apply the method to real LV cineangiocardiograms, user-interactive systems were implemented for preprocessing, including detection of LV contours, calibration of the imaging geometry, and setting of the LV model coordinate system. The results for three real LV angiographic image sequences are presented, two with fixed multiple views (using supplementary angiography) and one with rotating views. 3-D reconstructions utilizing different numbers of views were compared and evaluated in terms of contours manually traced by an experienced radiologist. The performance of the preprocesses was also evaluated, and the effects of variations in user-specified parameters on the final 3-D reconstruction results were shown to be sufficiently small. These experimental results demonstrate the potential usefulness of combining multiple views for 3-D recovery from real LV cineangiocardiograms. Index Terms Arterial septal defect (ASD), B-spline, left ventricle, multiview, time-varying closed surface, X-ray cineangiocardiogram. Manuscript received July 6, 2001; revised March 26, This work was supported in part by the Ministry of Education, Culture, Sports, Science and Technology (Japan) under a Grant-in-Aid for Scientific Research. The Associate Editor responsible for coordinating the review of this paper and recommending its publication was A. Amini. Asterisk indicates corresponding author. M. Moriyama is with the Faculty of Management and Information Science, Osaka International University, Hirakata, Osaka, , Japan. *Y. Sato is with the Division of Interdisciplinary Image Analysis, Department of Medical Robotics and Image Sciences, Osaka University Graduate School of Medicine, 2-2-D11, Yamada-oka, Suita, Osaka , Japan ( yoshi@image.med.osaka-u.ac.jp). H. Naito is with the Department of Radiology, National Cardiovascular Center, Suita, Osaka , Japan. M. Hanayama and T. Ueguchi are with the Department of Radiology, Osaka University Hospital, Suita, Osaka , Japan. T. Harada is with the Department of Design and Information Sciences, Faculty of Systems Engineering, Wakayama University, Wakayama , Japan. F. Yoshimoto is with the Department of Computer and Communication Sciences, Faculty of Systems Engineering, Wakayama University, Wakayama , Japan. S. Tamura is with the Division of Interdisciplinary Image Analysis, Department of Medical Robotics and Image Sciences, Osaka University Graduate School of Medicine, Osaka , Japan. Publisher Item Identifier /TMI I. INTRODUCTION THREE-DIMENSIONAL (3-D) modeling of time-varying cardiac ventricular shapes is becoming a useful tool for the quantitative evaluation of cardiac function such as by means of ventricular volume measurement and regional wall motion analysis [1]. Magnetic resonance (MR) imaging and X-ray computed tomography (CT) are both suitable for this purpose since direct 3-D data-sets are obtainable from these modalities, and several recent studies have reported on the reconstruction of 3-D ventricular shapes and motion from CT and MR data [2] [5]. However, cardiac CT or MR examinations are currently optional at most clinical sites. In spite of its invasiveness, X-ray cineangiocardiography during cardiac catheterization is carried out more commonly [6], and some cardiologists and surgeons still regard it as one of the most reliable modalities for cardiac diagnosis. While LV volume measurement and regional wall motion analysis based on X-ray angiocardiography have been typically performed using images from one or two views [6] [8], images from additional views should also be utilized if they are available. In our hospital (Osaka University Hospital, Osaka, Japan), X-ray ventriculograms are taken from multiple views (four views or more) when physicians judge that images from additional views are necessary for diagnosis despite the increased invasiveness especially in the case of pediatric patients with cardiac malformations who may lack the patience necessary for long examinations entailed in gated MR imaging or CT. Considering that X-ray cineangiocardiography remains a common procedure in clinical cardiology and the computational costs of 3-D recovery and visualization are rapidly decreasing, developing a method of recovering left-ventricular (LV) shapes that fully utilizes the constraints obtained from multiview two dimensional (2-D) projections is a worthwhile goal. To reconstruct an accurate 3-D ventricular model from conventional X-ray cineangiocardiograms with one or two views, several methods have been reported that involve combining X-ray cineangiocardiograms with additional constraints including convex shape and constant X-ray absorption constraints [9], density profiles [10], variable entropy [11], and projections from other fixed views [12]. We previously reported a method for reconstructing the time-varying 3-D LV shape by using occluding contours extracted from X-ray cineangiocardiograms with rotating views as well as fixed multiple views, which we evaluated by conducting simulation and phantom experiments [13]. Following on from our earlier work, here we report on the implementation and clinical application of this method in regard to several features /02$ IEEE

2 774 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 7, JULY ) User-interactive systems for preprocessing are implemented including detection of LV contours, calibration of the imaging geometry, and setting of the LV model coordinate system in order to apply the basic method to real LV cineangiocardiograms. Additionally, the formulation is extended to incorporate calibration parameters for the imaging geometry and transformation for the LV model coordinate system. 2) The LV contour detection performance is evaluated. The effects of user-specified parameters on the final 3-D reconstruction results is also assessed for the purposes of calibration and model coordinate system determination. 3) The usefulness of multiview integration is demonstrated using real LV cineangiocardiograms with both rotating views and fixed multiple views. The paper is organized as follows: In Section II, we describe the formulation of the method and its extension to incorporate the calibration parameters of the imaging geometry and the transformation for the LV model coordinate system. In Section III, after evaluating the effects of user-specified parameters in the preprocessing stages, we present the results of three actual LV angiographic image sequences, two with fixed multiviews (using supplementary angiography) and one with rotating views. In Section IV, we discuss the work and suggest future problems. Finally, in Section V we summarize our conclusions. II. TIME-VARYING 3-D LV SHAPE RECONSTRUCTION A. X-Ray Cineangiocardiography Coordinate System Biplane X-ray cineangiocardiographic equipment [Fig. 1(a)] has two image intensifiers (I.I.) that are able to rotate on a C arm. The viewing directions of the two I.I.s are always orthogonal. When a patient s body is aligned in the polar direction of the spherical coordinate system, as shown in Fig. 1(b), the variations in the LAO RAO (left- and right-anterior-oblique view) and CRAN CAUD (cranial caudal) angles, respectively, correspond to the longitude and latitude variations. Here, represents the LAO angle if, and the RAO angle if. Similarly, represents the CRAN angle if, and the CAUD angle if. We assume an orthographic image projection model. 1 The viewing direction of an I.I. is given by The two orthogonal directions of the image axes are given by (1) (2) (3) B. Preprocessing Application of the reconstruction algorithm [13] to real LV cineangiocardiograms requires three preprocesses: 1) detection 1 The source to I.I. distance (SID) is usually about 100 cm, and the radius of the I.I is five in (12.5 cm), in which most of the left ventricle is imaged within a 4-in (10 cm) radius. The maximum angle between the ray direction and the direction orthogonal to the I.I. plane can thus be expected to be about arctan 10= Therefore, orthographic projection can reasonably approximate the X-ray projection process. Fig. 1. (a) X-ray cineangiocardiographic equipment, and (b) coordinate system of biplane X-ray cineangiocardiography. of LV contours in X-ray cineangiocardiograms; 2) calibration of the imaging geometry in a multiview image sequence; and 3) determination of a model coordinate system for the time-varying 3-D LV shape. Each preprocess is implemented with a graphical user interface (Fig. 2) for it to be interactively performed. In the first preprocess, the 2-D coordinates of LV contour point are obtained from all input image sequences, as shown in Fig. 2(a), where and is the number of contour points for all viewpoints and times. We use digital subtraction images generated by subtracting mask images without contrast material from live images with contrast material. Hence, we can assume that the input digital subtraction images are projections of contrast material inside the ventricular cavity while the effects of other tissues are removed. The first preprocess itself consists of three interactive steps [Fig. 4(a)]. First, we extract the zero-crossing points of the Laplacian-Gaussian filter ( pixels) with a large gradient magnitude within mask regions whose gray levels are between two specified threshold values [upper frames in Fig. 2(a)].

3 MORIYAMA et al.: RECONSTRUCTION OF TIME-VARYING 3-D LV SHAPE FROM MULTIVIEW X-RAY CINEANGIOCARDIOGRAMS 775 Fig. 2. Graphical user interface for three preprocesses. (a) Detection of an LV contour in an X-ray cineangiocardiogram. (b) Calibration of the imaging geometry in an image sequence with multiviews. (c) Determination of a model coordinate system for the time-varying 3-D LV shape. Second, we remove the aortic contour connected to the LV contour by specifying a straight line between the LV and aorta so as to select only the LV contour [lower frame in Fig. 2(a) and second step in Fig. 4(a)]. Third, if outliers such as the catheter and/or diaphragm exist, we remove them by specifying one or more rectangular region [third step in Fig. 4(a)]. In the second preprocess, the imaging geometry in a multiview sequence is calibrated so that images with different views can be consistently integrated into the cineangiocardiography coordinate system. An iron ball with a known diameter, (60 or 100 mm), is used for the calibration, which includes determining the scale parameter of each view and the registration between images obtained using different views. We use the diameter and center point estimated from the imaged iron ball for the scale determination and registration, respectively. In our biplane X-ray system (Siemens BICOR), the viewing directions of the two I.I.s and the SID are memorized and the system configuration is reproducible using these memorized parameters. The images of the iron ball are obtained as follows. After all examinations are completed, the biplane system is arranged in the same configurations as when the LV image sequences were acquired. It is then confirmed whether the iron ball is placed so that its projections in the two I.I.s approximately overlap the regions corresponding to the LV in the angiocardiograms taken during the examinations. Based on the above confirmation in two directions, the ball can be approximately positioned where the LV was so as to minimize the introduction of scale inaccuracy into the system. Let be the diameter of the ball estimated in the image obtained from the viewing direction, and be the center point in the image viewed from. The scale parameter,, is given by Assuming that the center of the iron ball defines the origin of the cineangiocardiography coordinate system [Fig. 1(b)], the optical ray corresponding to the image point can be represented as where is a scalar value and where and are given in (2) and (3), respectively, and and are determined by interactively specifying the rectangle inscribed in the imaged iron ball with the graphical interface [Fig. 2(b)]. For image sequences with fixed multiviews, the above preprocess is performed for all the viewing directions. For rotating views, it is performed for the initial and final viewing directions and the calibration parameters are interpolated for other views by assuming that the angular variation of rotational motion is linear. In the third preprocess, the model coordinate system for representing the time-varying 3-D LV shape is determined by setting the spherical coordinate system of the LV model as follows. The long axis of the LV is defined as the 3-D line passing through the LV apex and the upper wall near the aorta. The line (4) (5) (6)

4 776 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 7, JULY 2002 segment corresponding to the long axis of the LV shape is manually defined from the two systolic images taken with the orthogonal I.I.s according to the above definition [Fig. 2(c)]. The line segment in the 3-D space whose projections agree with the axes specified in the two images is then defined as the axis of the LV coordinate system. The axis represents the polar direction of the spherical coordinate system. The origin is defined as the mid-point of the above line segment in the 3-D space. The directions of the and the axes are defined as the two orthogonal directions along which the two images are taken. Let,, and be unit vectors representing the axes of the LV coordinate system, and be the 3-D position of its origin in the cineangiocardiography coordinate system. defines the rotation of the LV coordinate system in the cineangiocardiography coordinate system while defines the translation. C. Representation and Reconstruction of Time-Varying 3-D Shape The LV reconstruction algorithm previously reported [13] is extended so as to incorporate the calibration parameters, and (,, of the imaging geometry, and the transformation parameters, and, of the LV model coordinate system. The time-varying 3-D shape is represented by a uniform B-spline function. The spherical coordinate is used to parameterize the closed surface. The distance is defined as the distance from the origin along the direction corresponding to the latitude and longitude. In this representation, the closed surface to be reconstructed is limited to the star-shaped surface around the origin of the spherical coordinate system. We assume that the motion is periodic, with a period. Then, the time-varying 3-D closed surface is represented as follows: where,, and ; is a B-spline coefficient; is a nonperiodic and uniform cubic B-spline; and are uniform cubic B-splines that can be extended to the periodic function; and,, and are the numbers of knots in the latitude, longitude, and time directions, respectively. The time is associated with the surface control points as suffix and the latitude and longitude as suffixes and, respectively. The expression represents the distance at time along the direction from the origin of the LV coordinate system determined in the preprocessing stage. Thus, the 3-D position on the timevarying surface in the X-ray cineangiocardiography coordinate system is obtained by transformation of the spherical coordinate to a Cartesian one. That is, the 3-D position is given by where is rotation and is translation of the LV model coordinate system determined in the prepro- (7) (8) Fig. 3. Constraints relating 2-D coordinates of contour points on an image to the 3-D position X(u; v; t) and normal n(u; v; t) on the time-varying surface r(u; v; t). cessing stage. The surface normal at is given by where normalizes the magnitude of vector, which is given by. Let us assume that an image has a known viewpoint and time. It is necessary to derive the constraint that relates the 2-D coordinate of the contour point on the image to the 3-D position and normal on the time-varying surface (Fig. 3). If the image contour point for the viewing direction and time is the projection point of the occluding contour of the surface, the following two constraints must be satisfied for a coordinate on the surface by using the optical ray : (9) (10) (11) where and are given in (1) and (6), respectively, and takes a scalar value. Equation (10) indicates that the optical ray corresponding to the image contour point should pass the 3-D point. Equation (11) indicates that the point on the occluding contour should be tangential to the viewing direction. Note that we can obtain using (6) from the calibration parameters and (, ), the viewing directions (, ), and image coordinate. Based on the constraints (10) and (11), is estimated. It is difficult to directly derive satisfying these constraints, since the surface coordinate that corresponds to the optical ray determined by the given image coordinate is unknown. Hence, let the number of contour points for all viewpoints and times be ; then. In order to obtain an approximate solution, we use an iterative procedure and decompose the iterative step into two stages. The algorithm is as follows. [Initial setting] is defined as the initial shape. In our experiment, the initial shape used was spherical. We calculate and. For the iteration, is used as the counter and the initial value is set as 0. [First stage]

5 MORIYAMA et al.: RECONSTRUCTION OF TIME-VARYING 3-D LV SHAPE FROM MULTIVIEW X-RAY CINEANGIOCARDIOGRAMS 777 We determine the tentative correspondence between the surface coordinate and optical ray determined by the image coordinate at time.for,, and the optical ray corresponding to all image contour points, we determine (, ) and so that the following two constraints are satisfied: TABLE I PATIENT INFORMATION and (12) (13) where is the scalar coefficient. [Second stage] Based on ( ) and determined in the first stage, we estimate by solving the set of linear equations derived from the following relation: (14) In practice, we derive, minimizing the following expression by combining it with a smoothness constraint: (15) where the value of in is ( at the poles and 1 on the equator). The first and second terms of (15) represent the data and smoothness constraints, respectively, being the weight parameter for the smoothness constraint. In the second term, the first and second subterms realize smoothness along equi-longitude and equi-latitude lines, respectively. The second subterm is multiplied by because the length along equi-latitude lines is reduced as the poles are approached. The third subterm realizes smoothness along time. When all the B-spline coefficients,,have the same value, the smoothness constraint is minimum (zero). Thus, without the data constraint, [see (7)] is a constant function and [see (8)] represents a spherical surface with a constant radius with respect to time. [Convergence condition] If the error for the newly estimated is nearly the same as the previous error, the algorithm ends. If not, we set and go back to the first stage. III. RESULTS We applied the reconstruction system to actual LV angiographic image sequences of three patients, each suffering from an arterial septal defect (ASD), taken when they underwent cardiac catheterization at Osaka University Hospital (Table I). ASD is a congenital condition manifested by a hole in the wall between the left and right auricles. As a result, the LV is reduced in size and its shape is deformed. In the reconstruction algorithm, unless otherwise specified, the number of knots in the B-spline function of (7) was set as follows:,, and, i.e., a grid was used. The smoothness constraint weight in (15) was set as. A. Preprocessing Fig. 4(a) shows a typical original image and LV contours detected when the three steps of the first preprocess described in Section II-B were applied. To evaluate the accuracy of the LV contour detection, we used contours manually traced by an experienced cardiac radiologist as reference contours, that is, the gold standard. To quantitatively compare detected and manually traced contours, we calculated the fractions of true-positive ( and false-positive detections ( in each of the three steps and plotted them as curves (similar to receiver operating characteristic curves). The TPF, was taken as (16) where is the total number of manually traced contour points and is the number that were detected correctly (defined as points that coincided with the reference contour points or were within a neighborhood of 3 3 pixels). The FPF, was taken as (17) where is the total number of contour points detected and is the number of incorrectly detected points, that is,. When and, the detection is considered to be ideal in the sense that all the true contour points are detected and there are no false detections. Fig. 4(b) shows the and plots for the three steps in Fig. 4(a) for the three clinical cases. The and values are the averages for 40 images in Case 1, 30 images in Case 2, and 11 images in Case 3. Our reconstruction algorithm requires that the number of false-positive detections, that is, outliers, is sufficiently small. In all three cases, the decreased with each step to around

6 778 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 7, JULY 2002 (a) (b) Fig. 4. Evaluation of LV contour detection. (a) Original image and results of LV contour detection by the three steps in the first preprocess. (b) Relationship between true-positive and false-positive fractions. The first, second, and third steps are, respectively, represented by the symbols +, 2, and 3. Solid, broken, and dotted lines show Cases 1, 2, and 3, respectively. 0.1 or less while the was still around 0.6 after the three steps i.e., the number of false-positive detections was sufficiently small. 2 For the second and third preprocesses, that is, imaging geometry calibration and LV model coordinate system setting, the effects of user-specified parameters on reconstructed LV shapes were evaluated. For this purpose, we performed reconstruction experiments using ten sets of parameters specified by ten different operators. First, we examined the variations in the parameter values themselves. Fig. 5(a) shows the parameter variations among the ten operators for the three clinical cases referred to above. In each case, the center point and radius of the circle fitted to the iron ball were used as parameters to calibrate the imaging geometry, and the origin and polar direction as parameters to determine the LV model coordinate system. Second, we examined the variations in the LV shapes reconstructed using 2 We also carried out the calculation under the condition that the neighborhood for defining contour points as being correctly detected was increased to a size of pixels, and obtained almost the same results as those shown in Fig. 4(b). the ten sets of parameters specified by the different operators in order to assess the effects of parameter variations on LV shape reconstruction. The reconstructed shapes were evaluated using two measures: the difference of volumes (DoV) and the volume of differences (VoDs). To compare two reconstructed shapes, the volume of each shape is first measured. To obtain the DoV, we simply take the difference between the two volume values. In contrast, the VoD is not simply the numerical difference between the two volume values but the difference between two shapes expressed in terms of the volume [Fig. 5(b)]. Although the DoV is always zero (that is, the two volume values are the same) if the VoD is zero (that is, two shapes are the same), the VoD is not necessarily zero even if the DoV is. To evaluate the variations among the ten reconstructed shapes, we selected the shape reconstructed using the parameters that were nearest to the mean values as a reference shape, and estimated the DoV and VoD values between this reference shape and each of the nine remaining shapes. The DoV and VoD were obtained at each time phase, and were further divided by the volume of the reference

7 MORIYAMA et al.: RECONSTRUCTION OF TIME-VARYING 3-D LV SHAPE FROM MULTIVIEW X-RAY CINEANGIOCARDIOGRAMS 779 Fig. 5. Evaluation of imaging geometry calibration and determination of a model coordinate system. (a) Scattering of parameters set by ten operators. The symbols + and 2 show the center points and radii of circles fitted to iron balls used to calibrate the imaging geometry, and 3 and show the origins and angles between polar directions used to determine the model coordinate system. The iron balls fitted had a radius of 30 mm in Cases 1 and 2, and 50 mm in Case 3. (b) VoD. The VoD is not simply the numerical difference between two volume values but the volume of the hatched regions in the figure, that is, the differences between the two shapes. (c) Differences of volume (DoV) due to additional views and to operator variations. Solid lines show the DoV between shapes reconstructed from multiviews and two fixed views. Broken lines show the average and standard deviations of the DoV between the reference shape and shapes reconstructed from parameters set by ten different operators using multiviews. Left: Case 1; middle: Case 2; right: Case 3. ES, end-systolic, ED end-diastolic (see text). (d) VoD due to additional views and to operator variations. shape at each time phase for normalization. Using the above two measures, we compared the effects of using additional views with those of operator variations i.e., variations in the parameters specified by different operators during the preprocessing stage to verify that the effects of operator variations are small compared with the effects of using additional views. Fig. 5(c) shows the DoV comparison. Four, six, and rotating views were used for the reconstruction in clinical Cases 1, 2, and 3, respectively. The DoV due to operator variations was around 3% while using additional views it was 8% 10%. Fig. 5(d) shows the VoD comparison. The VoD due to operator variations was around 5% while using additional views it was 15% 20%. The effects of operator variations were thus confirmed to be relatively small compared with those of using additional views, that is, the reconstructed shapes were virtually independent of operator variations. B. Results of Reconstruction Using the reconstruction algorithm, computation on a Sun Ultra 1 (UltraSPARC MHz) required about 1.5 h for Case 1, 1.0 h for Case 2, and 3.0 h for Case 3. In this algorithm, the computation is concentrated in the first stage (described in Section II-C). We can, however, employ parallel computing in the first stage and thereby save computational time. 1) Case 1: After taking an ordinary image sequence with two fixed views, another LV angiographic image sequence with

8 780 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 7, JULY 2002 Fig. 6. Input image sequences and reconstructed shapes in Case 1. (a) (d) Part of LV digital subtraction angiographic image sequence with four fixed views. (e), (f) Colored displays of time-varying 3-D LV shapes during one cycle, respectively, reconstructed from two and four fixed views; in both cases, the shapes are viewed from the viewpoint (045, 029 ), which was used for the image subsequence shown in (c). A virtual aorta approximated to a cylinder is displayed simultaneously to show the viewpoint location. The green, yellow, and blue colors are texture-mapped onto the reconstructed shapes. Green and yellow, respectively, represent parts reconstructed by LV contours shown in (a) (b) and (c) (d). Blue represents parts reconstructed only by the smoothness constraint. (g) (i) Input image with superimposed extracted LV contour and colored displays of shapes reconstructed from two and four fixed views at the end-systolic phase [from (c), (e), and (f), respectively]. Fig. 7. Convergence for reconstruction from four fixed views in Case 1. two additional fixed views is sometimes taken, especially when the patient is an infant. This additional sequence, referred to as supplementary angiography, may be necessary due to difficulty in identifying the LV shape in the first image sequence. Fig. 6(a) (d) shows part of an LV angiographic image sequence with four fixed views. These are digital subtraction images after correction for I.I. distortion [14]. The four viewpoints were (0,0 ) and (90,0 ) in the ordinary angiography, and ( 45, 29 ) and (44, 16 ) in the supplementary angiography. The time interval was, where represents the period of a heartbeat; that is, each image sequence consisted of ten LV angiographic images. Fig. 6(e) shows the shapes reconstructed from the ordinary angiography image sequence with two fixed views [Fig. 6(a) and (b)], while Fig. 6(f) depicts those reconstructed from the ordinary and supplementary angiography image sequence with four fixed views [Fig. 6(a) (d)]. Both sets of reconstructed shapes are viewed from the viewpoint ( 45, 29 ). This viewpoint is the same as that for the image subsequence in Fig. 6(c), which was used to reconstruct the set of shapes from four fixed views but not that from two fixed views. The colors green, yellow, and blue texture-mapped onto the reconstructed shapes, respectively, represent the parts

9 MORIYAMA et al.: RECONSTRUCTION OF TIME-VARYING 3-D LV SHAPE FROM MULTIVIEW X-RAY CINEANGIOCARDIOGRAMS 781 reconstructed by LV contours extracted from the ordinary angiography [Fig. 6(a) and (b)], the parts reconstructed by LV contours extracted from the supplementary angiography [Fig. 6(c) and (d)], and the remaining parts reconstructed only by the smoothness constraint. It can be seen that in the set of shapes reconstructed from four fixed views, a greater proportion is accounted for by LV contours compared with the set reconstructed from two fixed views. Furthermore, the shapes in the two sets are themselves different. Fig. 6(g) shows the end-systolic phase image from the sequence in Fig. 6(c) with the extracted LV contour superimposed on it, while Fig. 6(h) and (i) shows the end-systolic phase reconstructed shapes, respectively, taken from the sets in Fig. 6(e) and (f). It is apparent that the shape reconstructed from four fixed views coincides with the extracted LV contours better than that reconstructed from two fixed views. Fig. 7 shows the convergence for the reconstruction from four fixed views for clinical Case 1. The convergence was investigated under different smoothness constraint weight parameters,. Four iterations were needed for the weight parameters to converge in the reconstruction from four fixed views, except in the case of the small weight parameter. This convergence tendency is similar to that for the reconstruction from two fixed views (data not shown), as well as to Cases 2 and 3 (see Fig. 10). Fig. 8 shows the effects of varying the number of knots in the B-splines on the reconstructed shapes in the three clinical cases. Using different numbers of B-spline knots i.e., 6 6 3, , and the VoD between the respective reconstructed shapes was ascertained. The plots in Fig. 8 shows the VoDs between the shape reconstructed using a grid and those reconstructed using the other grids. While the shape reconstructed using a grid differed significantly from that obtained with the grid, the difference between the shapes reconstructed using the and grids was not significant. 2) Case 2: Fig. 9(a) shows images with six fixed views in the X-ray cineangiocardiography coordinate system and LV digital subtraction angiographic images at the end-diastolic phase. The six viewpoints were (0,0) and (90,0) in the ordinary angiography, ( 45,21) and (43,17) in a first supplementary angiography, and ( 28,22) and (63,26)in a second supplementary angiography. The time interval was ; that is, each image subsequence consisted of five LV angiographic images. The reconstructions in Case 2 are summarized in Table II. Fig. 9(b) (d) shows shapes reconstructed from image sequences with two, four, and six fixed views (RECONSTRUCTIONS-1, -2, and -4 in Table II), respectively. The three reconstructed shapes are viewed from the viewpoint ( 28, 22 ), indicated by an orange arrow in Fig. 9(a). Fig. 9(e) and (f) shows LV digital subtraction angiographic images on which contours traced manually by a cardiac radiologist (yellow lines) and the contours of the reconstructed shapes (red lines) are superimposed (RECONSTRUCTION-1 [from two views] and RECONSTRUCTION-2 [from four views], respectively, in Table II). The viewpoint is again ( 28, 22 ) and the time-phase is end-diastolic. Note that the viewpoint ( 28, Fig. 8. VoD due to variations in the number of B-spline knots. The number of B-spline knots was varied as follows: , , and The plots show the VoD between the shape reconstructed using a grid and the shapes reconstructed using the other two grids. Solid lines show the VoD between the and grids. Broken lines show the VoD between the and grids. (a) Case 1. (b) Case 2. (c) Case ) was not used to reconstruct the two shapes but only to evaluate the method [ in Table II]. The contour in RECONSTRUCTION-2 coincides with the manually traced contour better than that in RECONSTRUCTION-1. For a more detailed and comprehensive analysis, we evaluated how the weight parameter of the smoothness constraint affected the reconstructed shape. Evaluating the bias 3 of the method is difficult in a reconstruction from a real LV angiographic image sequence, since the true time-varying 3-D LV shape cannot be obtained in vivo. Instead, we used the LV contour traced manually by an experienced cardiac radiologist as the gold standard. To measure the error, we used the mean value of the 2-D distance between LV contour points traced manually and those projected from the reconstructed shape. Fig. 10 shows variations in the reconstruction error in terms of the weight parameter of the smoothness constraint. Two shapes reconstructed from two and four fixed views are compared [RECONSTRUCTIONS-1 and -2 in Fig. 10(a) and RECONSTRUCTIONS-1 and -3 in Fig. 10(b)]. To evaluate the method objectively, we used LV contours that were not used to reconstruct the two shapes; that is, LV contours extracted from the second supplementary angiography [ in Table II] and the first supplementary angiography [ in Table II] in Fig. 10(a) and (b), respectively. In this way, we were able to evaluate the tradeoff between the accuracy and stability of the method. The method worked well over a wide range of 3 The accuracy consists of two measures, bias and precision. Bias is the discrepancy from the true value; precision is the amount of scattering of each measurement.

10 782 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 7, JULY 2002 Fig. 9. Input images and reconstructed shapes in Case 2. (a) Six viewpoints in the coordinate system of the X-ray cineangiocardiography and LV digital subtraction angiographic images at the end-diastolic phase. (b), (c), (d) Colored displays of time-varying 3-D LV shapes of RECONSTRUCTIONS-1, -2, and -4 at the end-diastolic phase; all three shapes are viewed from the viewpoint (028, 22 ), which is indicated by an orange arrow in (a). Green, yellow, and orange, respectively, represent the parts reconstructed by LV contours in the ordinary, first supplementary, and second supplementary angiographies. Blue represents the part reconstructed only by the smoothness constraint. (e), (f) LV digital subtraction angiographic images with superimposed contours traced manually by a cardiac radiologist (yellow lines) and contours of shapes reconstructed from two and four fixed views (red lines). The viewpoint is (028, 22 ), indicated by an orange arrow in (a); the time-phase is end-diastolic. TABLE II SUMMARY OF RECONSTRUCTIONS IN CASE 2 weight parameters ( ), but rapidly became unstable with weight parameters smaller than a critical value ( ) due to its nonlinear and underconstrained nature. The error in the reconstruction from four fixed views was smaller than that in the reconstruction from two fixed views in terms of weight parameters. When was in Fig. 10(a), the errors in the reconstructions from two and four fixed views were 4.9 mm (13.2%) and 3.1 mm (8.2%), respectively, where the percentage represents the ratio between the error and a 37.3-mm length of the reconstructed LV major axis at the end-systolic phase. 3) Case 3: After taking the ordinary angiography sequence, an LV image sequence was taken from two varying viewpoints. The time interval was. The two I.I.s were rotated on

11 MORIYAMA et al.: RECONSTRUCTION OF TIME-VARYING 3-D LV SHAPE FROM MULTIVIEW X-RAY CINEANGIOCARDIOGRAMS 783 Fig. 10. (a) (b) Variations in reconstruction error in terms of weight parameter of the smoothness constraint. The weight parameter of the smoothness constraint (0: w 5: ) was varied. Broken and dotted lines show errors for shapes reconstructed from two and four fixed views, respectively, when the number of B-spline knots was In order to evaluate the method objectively, the LV contours were different from those used to reconstruct the two shapes. (a) Comparison of RECONSTRUCTIONS-1 and 2. (b) Comparison of RECONSTRUCTIONS-1 and 3. the C arm fixed at. Images of the chamber filled with an angiographic agent were taken at and. Contours were extracted for a total of 84 LV angiographic images, covering about 6 heartbeats. Fig. 11(a) shows the two fixed viewpoints and the varying viewpoints used for the reconstructions in the X-ray cineangiocardiography coordinate system. The two fixed viewpoints in the ordinary angiography were (0,0) and (90,0); the varying viewpoints in the rotated angiography were from (92.1,0) to (61.2,0) and from (2.1,0)to( 28.2, 0 ). Fig. 11(b) and (c), respectively, shows the shapes reconstructed from the ordinary and rotated angiographies. The two reconstructed shapes are viewed from the viewpoint ( 35.7, 0 ) at the end-systolic phase. Fig. 11(d) shows the LV digital subtraction angiographic image, on which the manually traced contour is superimposed, also viewed from the viewpoint ( 35.7, 0 ) at the end-systolic phase. The viewpoint in Fig. 11(b) (d) is indicated by the red arrow in Fig. 11(a). As in Case 2, this viewpoint was not used for reconstruction but only to evaluate the method. The shape reconstructed from varying views coincides with the manually traced LV contour better than that reconstructed from two fixed views. To verify the effectiveness of reconstruction from varying views, we used the mean value of the 2-D distance between the manually traced contour points [Fig. 11(d)] and the contour points projected from the two reconstructed shapes [Fig. 11(b) and (c)] to measure the error. In Fig. 11(b), the mean value of the distance was 4.2 mm (5.8%) and the maximum distance was 9.5 mm (13.1%). In Fig. 11(c), the mean value of the distance was 1.8 mm (2.5%) and the maximum distance was 8.2 mm (11.3%). In each case, the percentage represents the ratio between the error and a 72.5-mm length of the reconstructed LV major axis at the end-systolic phase. IV. DISCUSSION 1) LV Contour Extraction: Although fully automated edge detection from X-ray angiocartiograms is still a research topic [15], [16], the problem of edge detection was greatly simplified by using digital subtraction images when fixed views were involved, as in Cases 1 and 2. Despite the fact that our method of LV contour extraction, described in Section II-B, is not very sophisticated, the user-interactions were completed within one minute, and only a few minutes were required to compute the edge operator in all the images (30 40 images) so as to complete the LV contour extraction in Cases 2 and 3. However, with varying views, as in Case 3, some spurious edges were caused by artifacts of the catheter and the diaphragm and these had to be removed manually [the third step in Fig. 4(a)]. In Case 3, it took about one hour to complete the LV contour extraction in all the images (84 images), including user-interactions and computation. In addition, the in Case 3 was smaller than in Cases 1 and 2 [Fig. 4(b)], the reason being that it is difficult to take the mask image sequence with viewpoints and time-phases that are the same as those of live image sequences. Most of the operations in Case 3 involved the removal of spurious edges. To reduce the number of operations and take account of unexpected detection of spurious edges, we need to extend the method to incorporate the concept of robust estimation [17]. 2) Comparison of Reconstruction Results Between Synthesized Images and Real LV Cineangiocardiograms: One purpose of this work was to evaluate the basic method reported in [13] using real LV cineangiocardiograms. The method s performance with actual clinical data was shown to be comparable to that with simulated data previously reported [13] in terms of its convergence characteristics, the tradeoff between accuracy and stability, and the effects of additional views. The results can be summarized as follows. The reconstruction algorithm converged in four iterations (Fig. 7) for suitable weight parameters. The method provided a good tradeoff between accuracy and stability (Fig. 10). The method worked well over a wide range of weight parameters ( and in the simulation [13] and clinical experiments, respectively). Shapes reconstructed from multiviews conformed better to the LV contours than those reconstructed from two fixed views in ordinary angiography (Fig. 6 and 9 11). 3) Validation With Clinical Data: One problem in evaluating the usefulness of a method with respect to real LV cineangiocardiograms is how to obtain the gold standard [1]. Here, we used as the gold standard LV contours traced manually by an experienced radiologist. These were employed not only to validate the LV edge detection but also for the final 3-D shape reconstruction in Case 2 where six views were available. Although the clinical validation was limited by the fact that an error could be measured only in 2-D projections, whereas it should be measured in the 3-D space, the clinical validation using manually traced contours gave similar results to those observed in our previous validation using simulated data [13], and weight parameter ranges that gave a good balance between stability and accuracy could be ascertained (Fig. 10). 4) Representation of Time-Varying 3-D LV Shape and Stability of Recovery: In the reconstructed LV shapes obtained in Cases 1, 2, and 3, the mean values of the 2-D distances between the LV contour points projected from the reconstructed shape

12 784 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 21, NO. 7, JULY 2002 Fig. 11. Input image and reconstructed shapes in Case 3. (a) Two fixed and varying viewpoints in the X-ray cineangiocardiography coordinate system used for the reconstruction. (b), (c) Colored displays of time-varying 3-D LV shapes reconstructed from two fixed views and varying views. The viewpoint is (035.7, 0 ) and the time-phase is end-systolic, which are the same as the viewpoint and time-phase of the image shown in (d). Green represents the parts reconstructed by LV contours and blue the parts reconstructed only by the smoothness constraint. (d) LV digital subtraction angiographic image with superimposed contour traced manually by a cardiac radiologist. The viewpoint is (035.7,0 ), which is shown by the red arrow in (a); the time-phase is end-systolic. and the LV contour points manually traced by a cardiac radiologist almost coincided. However, the maximum 2-D distance in Case 3 was greater than that in Cases 1 and 2. The maximum distance in Case 3 was 8.2 mm (11.3%), and as can be seen in the lower left region of Fig. 11(d), the undulation details of the contour were not reflected on the reconstructed shape. It is therefore difficult to evaluate the local contractility of the LV shape in this part of the image. To reflect detailed contour undulations, the reconstruction method should be extended to allow local refinement of the surface, such as by employing a hierarchical B-spline [18], or a nonuniform B-spline with adaptive knot arrangements [19], rather than a uniform B-spline. In this case, the weight of the smoothness constraint should be adaptively varied depending on the knot interval. However, the incorporation of adaptive surface modeling may introduce another tradeoff between accuracy and instability, while the currently used uniform B-spline has advantage of making the fitting problem simpler and more stable. Using different 3-D-shape models from B-splines would also give rise to the same problem. For stable and accurate recovery, strong data constraints and model constraints will eventually be needed. An increase in the number of views, as well as a priori models of the LV shape and motion, should be useful in attaining both accuracy and stability. 5) Tradeoff Between Accuracy and Patient Invasiveness: To reconstruct the LV shape accurately, images of the heart chamber filled with contrast agent taken from a wide range of viewpoints and over multiple time-phases are needed. However, the number of images that can be acquired is related to the permissible amounts of X-ray radiation and contrast agent. Thus, there has to be a tradeoff between the accuracy of the reconstruction and an acceptable level of patient invasiveness. In the rotated angiography in Case 3, the X-ray dose and amount of contrast agent injected were the same as in Case 1. However, it is desirable that the angular range of views acquired in Case 3 should be expanded in order to improve the accuracy of the reconstructed LV shapes. (The angular range is ideal when it is 180, but it was only 80 in Case 3.) To obtain a sufficient angular coverage using our biplane X-ray system (Siemens Bicore) it would be necessary to raise the quantity of contrast agent to 1.5 times the normal amount, which would increase the patient invasiveness. However, this constraint arises from a limitation in the rotational speed of our biplane system. If the I.I.s could rotate on the C arm at a higher speed, it would be possible to obtain a sufficient angular coverage within a shorter time. That is, the LV cavity would need to be filled with the contrast agent for a shorter time, thereby reducing the amount of contrast agent required. Future improvement in the rotational speed of biplane X-ray systems will facilitate improve angular coverage, which should provide good accuracy with less contrast agent.

13 MORIYAMA et al.: RECONSTRUCTION OF TIME-VARYING 3-D LV SHAPE FROM MULTIVIEW X-RAY CINEANGIOCARDIOGRAMS 785 V. CONCLUSION To acquire a model of the time-varying 3-D LV shape, we have proposed a reconstruction method that utilizes multiview X-ray cineangiocardiograms. For clinical use, we systematized the method by means of three preprocesses: detection of LV contours in X-ray cineangiocardiograms, calibration of the imaging geometry, and determination of a model coordinate system. The method was shown to be effective when applied to two actual LV angiographic image sequences with multiviews (ordinary and supplementary angiography) and one sequence with varying views. Future work will include the incorporation of an adaptive B-spline knot arrangement as well as a priori model of an LV in order to stably recover detailed contour undulations, and a mechanism for outlier rejection to deal with spurious LV edges, which simultaneously performs automated edge detection and LV shape recovery. Improvement in the rotational speed of biplane X-ray systems used for rotated cineangiocardiography is also desirable so that good accuracy can be attained with a minimal amount of contrast agent. ACKNOWLEDGMENT The authors would like to thank the members of the Heart- Artery Laboratory, First Department of Surgery, Osaka University Medical School for undertaking the clinical tests. REFERENCES [1] A. F. Frangi, W. J. Niessen, and M. A. Viergever, Three-dimensional modeling for functional analysis of cardiac images: A review, IEEE Trans. Med. Imag., vol. 20, pp. 2 25, Jan [2] P. Shi, A. J. Sinusas, R. T. Constable, E. Ritman, and J. S. Duncan, Point-tracked quantitative analysis of left-ventricular surface motion from 3-D image sequences, IEEE Trans. Med. Imaging, vol. 19, pp , Jan [3] M. Kachelriess, S. Ulzheimer, and W. A. Kalender, ECG-correlated imaging of the heart with subsecond multislice spiral CT, IEEE Trans. Med. Imag., vol. 19, pp , Sept [4] D. Suter and F. Chen, Left-ventricular motion reconstruction based on elastic vector splines, IEEE Trans. Med. Imag., vol. 19, pp , Apr [5] T. S. Denney, Jr., Estimation and detection of myocardial tags in MR image without user-defined myocardial contours, IEEE Trans. Med. Imag., vol. 18, pp , Apr [6] J. Beier, E. Wellnhofer, H. Oswald, and E. Fleck, Accuracy and precision of angiographic volumetry methods for left and right ventricle, Int. J. Cardiol., vol. 53, pp , Feb [7] H. J. Hermann and S. H. Bartle, Left ventricular volumes by angiocardiography: Comparison of methods and simplification of techniques, Cardiovasc. Res., vol. 2, pp , Oct [8] S. Eiho, S. Yamada, and M. Kuwahara, Three-dimensional display of left ventricle by biplane X-ray angiocardiograms and assessment of regional myocardial function, in Proc. Medinfo 1980, Tokyo, Japan, pp [9] H. Matsuo, A. Iwata, I. Horiba, and N. Suzumura, Dynamic left ventricular cavity shape reconstruction system using bi-directional projections (in Japanese), IEICEJ Trans. (D-II), vol. J74-D-II, pp , Jun [10] G. P. M. Prause and D. G. W. Onnasch, 3-D reconstruction of the ventricular dynamic shape from the density profiles of biplane angiocardiographic image sequences, in IEEE 1994 Proc. Computers in Cardiology, Los Alamitos, CA, 1994, pp [11] U. Raff, P. F. Vargas, and B. M. Groves, Automated determination of left ventricular volume curves from bi-plane digital angiography without explicit use of edge detection algorithms, Int. J. Card. Imag., vol. 12, pp , Mar [12] S. Eiho, M. Kuwahara, K. Shimura, M. Wada, M. Ohta, and T. Kozuka, Reconstruction of the left ventricle from X-ray cineangiocardiograms with a rotating arm, in Proc. Computers in Cardiology, 1984, pp [13] Y. Sato, M. Moriyama, M. Hanayama, H. Naito, and S. Tamura, Acquiring 3-D models of nonrigid moving objects from time and viewpoint varying image sequences: A Step toward left ventricle recovery, IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-19, pp , [14] Y. Kawada, N. Niki, H. Sato, and T. Kumazaki, 3-D reconstruction of vessels by high-speed X-ray rotational imaging (in Japanese), IEICEJ Trans. (D-II), vol. J76-D-II, pp , [15] M. Dubuisson-Jolly, C. Liang, and A. Gupta, Optimal polyline tracking for artery motion compensation in coronary angiography, in Proc. 6th ICCV, Bombay, India, Jan. 1998, pp [16] L. Sui, R. M. Haralick, and F. H. Sheehan, A knowledge based boundary delineation from left ventriculograms, in IAPR Workshop Machine Vision Application 2000, Nara, Japan, Nov. 2000, pp [17] E. J. Besl, J. B. Birsh, and L. T. Watson, Robust window operator, in Proc. ICCV 88, Tampa, FL, pp [18] D. Forsey and D. Wong, Multiresolution surface reconstruction for hierarchical B-splines, in Proc. Graphics Interface 98, Vancouver, BC, Canada, 1998, pp [19] F. Yoshimoto, M. Moriyama, and T. Harada, Automatic knot placement by a genetic algorithm for data fitting with a spline, in Proc. Shape Modeling Int. 99, Aizu, Japan, Mar. 1999, pp