Automated left and right heart ventricle segmentation and model construction in tagged MRI: targeting current and future pulse sequences

Transcription

1 Automated left and right heart ventricle segmentation and model construction in tagged MRI: targeting current and future pulse sequences Albert Montillo 1, Dimitris Metaxas 2, Leon Axel 3 1 Dept of Computer Science, University of Pennsylvania, Philadelphia, PA, USA 2 Dept of Biomedical Engineering, Rutgers University, New Brunswick, NJ, USA 3 Department of Radiology, New York University, NY, NY, USA Abstract Measurement of myocardial contractility is one important way to characterize the function of the heart, which may lead to improved patient diagnosis and more successful treatment of many forms of cardiovascular disease. Over the past decade, researchers have developed methods for this characterization that require: (1) tracking the motion of points within the heart walls, using a modality with which the motion of the myocardium can be estimated, such as tagged MRI (tmri), and (2) an accurate segmentation of the heart walls, particularly the epicardial and endocardial surfaces. Both the need to perform this image analysis in a clinical setting and the development of newer, high contrast, volumetric data acquired with steady state free precession and 3D acquisition has led to the growing need for an accurate, automated way to segment the boundary surfaces, replacing the largely manual methods currently in use. We propose a fully automated method which achieves this segmentation in current sparse, noisy tmri, yet is specifically applicable to the newer tmri. Our method identifies landmarks in blended 3D features and projects them along characteristic curves in the solution of a PDE to construct a biventricular volumetric mesh. Using the finite element method, we define biomechanical constraints which produce regularized dynamic segmentation results with excellent agreement to expert manual segmentation. The resulting segmentation and mesh can be used to (1) directly measure classical descriptors of heart function, and (2) facilitate the recovery of tag motion, and myocardial strain distribution. Keywords: Segmentation, deformable model, hexahedral mesh, finite element method, tagged MRI 1. Introduction Accurate, automated quantitative characterization of normal and abnormal myocardial tissue may lead to improved patient diagnosis and more successful treatment of many forms of cardiovascular disease. Since in most developed countries this disease kills more men and women than any other disease, a computational methodology providing such a capability would be of tremendous value to physiology researchers and clinicians alike. Of particular value would be greater understanding of how the contraction of the myocardial tissue, which forces blood throughout the body, deteriorates from prolonged ischemia and progresses to stunned, hibernating, or infarcted myocardial tissue. Measuring the contraction of the myocardium throughout the ventricular walls is an important way to quantitatively characterize the contractile function and health of the heart. Accurate measurement requires tracking points within the heart walls during systole; however, since the heart tissue is largely homogeneous, tracking requires an appropriate imaging technique. Tagged MRI (tmri) is one such technique which magnetically tags parallel sheets of tissue at end-diastole (Axel et al., 1989; Zerhouni et al., 1988). Tagging creates the necessary myocardial tissue heterogeneity; however, tracking the deformation requires the accurate segmentation of the epicardial and endocardial surfaces and the recovery of tag motion. A method which automatically segments the boundary surfaces of the heart in 4D tmri can facilitate a wide variety of tag motion recovery methods. Researchers (Declerck et al., 1998; Dougherty et al., 1999; Haber et al., 2000; Ozturk and McVeigh, 1999; Park et al., 1996; Park et al., 2003; Tustison and Amini, 2003;Young et al., 1999;) who develop such methods using tag sheet tracking, optical flow or its spectral domain analogy Corresponding author. address: montillo@alumni.upenn.edu (A. Montillo). 1

2 (e.g. HARP method (Osman and Prince, 1998)), are all beginning to rely upon segmented boundaries to improve the performance of motion reconstruction as segmenting the boundaries of the left ventricle (LV) facilitates tag motion recovery by restricting to the myocardium, the portion of the tags which are tracked. These researchers have used manual contouring methods; consequently, their techniques would be aided by an automated segmentation method such the one presented below. If the epicardial and endocardial boundaries can be segmented, interpolating the 3D motion throughout the myocardium with a FEM model also becomes possible, even in thin walled structures, including the right ventricle (RV), where there is a sparsity of tags (Haber et al., 2000). Other researchers (Declerck et al.,1998; Ozturk and McVeigh, 1999; Park et al., 1996) have shown that the endocardial and epicardial surfaces can be used in conjunction with tag sheet tracking to form a low dimensional description of myocardial motion. A method that segments the boundaries of the heart in tmri would also provide the boundary conditions necessary to study the distribution of myocardial stress (Hu et al., 2003). Recent research has shown that boundaries segmented from tmri can themselves be used to directly calculate the classical descriptors of heart function, including ventricular volume and ejection fraction, at the same accuracy of those obtained through non-tagged MRI (Dornier et al., 2002). An automated segmentation method could therefore benefit the patient, by providing these descriptors, without increasing time in the scanner, since an non-tagged scan would be obviated. Several reasons motivate our development of a fully automated method to segment the boundary surfaces. It has been noted (Amini and Prince, 2001) that a fully automated analysis method is essential for the clinical adoption of tmri, because only such a solution will provide the response time and objectivity necessary for routine clinical use. In addition, full automation is essential for analyzing the additional large data sets consisting of higher spatiotemporal resolution images produced with the latest MR pulse sequences, including 3D volumetric tmri acquisitions (Ryf et al., 2002). Full automation is also important for generating highly detailed segmentation in high-contrast images employing steady state free precession (SSFP) (Herzka et al., 2003; Zwanenburg et al., 2003) as the manual processing time increases substantially when tracing the increased visible anatomical detail. A system which automates the processing of tmri has been proposed (Chandrashekara et al., 2002); however this system analyzes the LV only and not the more challenging thinwalled RV. In addition, this system does not detail a method for segmenting the LV epicardial surface. Prior to the method we propose, there has not been a fully automated method to extract the bounding surfaces of the biventricular heart from tmri, due to these arduous processing challenges: (1) MRI artifacts & noise must be suppressed and the tag lines, which obscure the boundaries, removed, (2) boundary-delineating features must be extracted, (3) information must be integrated from features that can lie on irregularly arranged image planes, and (4) the number of subjects in most tmri databases, particularly those using the latest tmri pulse sequences, is typically inadequate to train an appropriate detail-level statistical prior-model for segmentation. Moreover, a segmentation method which also constructs a geometric mesh that parcellates the ventricles into regular volumetric elements would facilitate a host of cardiovascular studies. Researchers have demonstrated that such a mesh can be embedded in a finite element model to compute myocardial strain distribution (Park et al., 2003; Haber et al., 2000), stress distribution (Hu et al., 2003), and by fusing multiple image modalities, electrical activation (Sermesant et al., 2003). 2. Research objectives Our research objectives are: (1) to develop a fully automated method that segments the myocardium in tmri throughout the heart s contraction, (2) to simultaneously construct a 3D volumetric mesh with elements aligned to traditional physician-friendly coordinates and demonstrate its suitability for quantitative analysis by embedding it in a mechanical FEM model, and (3) to target our segmentation method to both existing sparse, low contrast tmri, as well as future dense, high contrast tmri. We hypothesize that we can develop an accurate method to dynamically segment the ventricles throughout systole by constructing a biventricular mesh directly in the patient s tmri data which roughly corresponds to the pose, size and shape of patient s ventricles at early systole. Furthermore, we hypothesize that 2

3 bestowing elastic mechanical properties on the mesh will provide proper model surface-separation and surfacesmoothness constraints. There are alternative model-based approaches for dynamic segmentation in non-tagged MRI including statistical active appearance models (Mitchell et al., 2001) and models with global shape priors (Park et al., 1996). However, we want to use our segmentation method in conjunction with tag tracking and our patient database is not representative of the variations in myocardial shape and systolic shape change to which we would like to ultimately apply a combined segmentation and tag tracking method. We would also like to create a method useful for bootstrapping other methods by creating the ventricular shape training sequences to form active appearance models or models with statistical motion priors. For example, our approach could be used to generate the segmented 3D blood pool and LV myocardium shapes required to train the statistical shape model proposed in (Frangi et al., 2002) for non-tagged MRI. By combining our method and Frangi s, a statistical shape model could be constructed for tagged-mri. We want to develop a method which will likely be suitable to segment the heart in images from newer tmri pulse sequences that are currently under development by MR physicists, including those based on SSFP, true 3D acquisition, and fast parallel acquisition. Images from these novel pulse sequences tend to vary in following aspects: 1. image tissue contrast 2. spatiotemporal resolution 3. image SNR 4. the level of visible structural detail To handle variations in image tissue contrast, we propose the use of ventricular blood as the anchoring feature for constructing a biventricular model. We suggest that use of this feature for pulse sequences, such as the SSFP-based ones, which have even brighter blood-pixel intensity and higher blood-myocardium contrast than the current tmri pulse sequences, will make this feature even more valuable, particularly for our morphological opening-based blood feature extraction method that we will describe. To handle variations in spatiotemporal resolution and image SNR, we propose a method that extracts highly detailed 3D features by averaging and interpolating data across (1) time, (2) space (tmri slices), and (3) tag-orientations through the use of a distancetransform feature representation. As the spatiotemporal resolution or SNR vary, adjusting the amount of interpolation or averaging should enable extraction of patient specific features. To handle variations in the level of visible structural detail, we propose a segmentation method that builds a 3D model directly from our blended 3D data features rather than requiring manual segmentation to build a statistical model (which would require significant delay since such manual training data is typically unavailable for pulse sequences under development). In the remainder of this paper we present our approach and resulting system. In section 3, we describe the salient elements (Fig. 1) of our methodology including: (1) image post-processing, (2) feature extraction, (3) mesh creation, (4) mechanical model construction, (5) force field computation, and (6) intermediate constraint segmentation. We present mesh creation and segmentation results in section 4 and finish with a discussion of the results in the final section. 3

4 Fig. 1. Relationship between the salient elements of our automated model-based tmri segmentation system. 3. Methodology 3.1 Image post-processing Using a 1.5T GE MR scanner with an ECG-gated tagged gradient echo pulse sequence, we acquired three corresponding image sets every 30ms during systole. Two of the sets contain parallel short axis (SA) images; one of these has horizontal tags and the other vertical tags. These images are roughly perpendicular to an axis through the center of the left ventricle (LV). The other set consists of long axis (LA) images acquired along planes having an angular separation of 20 degrees and whose intersection approximates the axis through the LV. We denote these image sets (Fig 2) as V,t,horz, V,t,vert, and V,t,LA where t indicates the time frame and the horz, vert or LA subscript indicates the tag orientation. To keep the time a patient must remain in the scanner to a minimum, we do not acquire additional untagged images. Fig. 2. The complete data set,,t V for a time t consists of the set of image montages: { V,t,horz, V,t,vert, V,t,LA} We apply three types of intensity inhomogeneity suppression (Montillo et al., 2003a) which: normalize spatial intensity inhomogeneity in each image from uneven receiver coil sensitivity (Fig 3a), normalize temporal intensity changes from tag fading and blood movement (Fig 3b), and normalize intensity differences between subjects from the inherent lack of an absolute intensity in MRI (Fig 3c). We denote these corrected, standardized volumes, V cs,t,horz, V cs,t,vert, V cs,t,la. 4

5 a b c Fig. 3. (a) Spatial intensity normalization (b) Temporal intensity normalization (c) Intersubject intensity normalization 3.2 Feature extraction The ventricles contain the largest contiguous pool of blood in the body, therefore we propose a method to locate the LV and RV ventricle blood cavities, denoted O LVB and O RVB, by finding regions corresponding to the LV and RV blood regions in our short axis images. We hypothesize that they can be combined from multiple time frames to form good features to delineate the endocardial surfaces of the ventricles. To find these features in each image, we apply a sequence of 3D grayscale morphological opening operations (Montillo et al., 2002) that isolates the blood in each image (Fig. 4), followed by a region selection operation that uses the temporal consistency and spatial similarity of the ventricular blood regions to distinguish them from features caused by noise or other blood-filled structures such as the aorta. The regions from each ventricle (Fig 4c) are placed into separate binary indicator images. We group the regions from the vertically tagged images into indicator image sets, O LVB,t,vert and O RVB,t,vert, and those from the horizontally tagged images into OLVB,t,horz and O RVB,t,horz. 5

6 a b c d e Fig 4. Selected short axis images (a) are processed with grayscale opening (b) and thresholded to blood regions which are pruned to ventricular blood (c) using shape similarity metrics. Grayscale closing and noise suppression provides the input (d) for detecting edge features (e) that facilitate the epicardial surface segmentation described in section Compute a 3D distance transform representing the shape of each blood cavity for each time frame Convert ventricular blood indicator image sets to 3D distance transforms (DTs) Each indicator image set, O { OLVB,t,horz, O RVB,t,horz,OLVB,t,vert, ORVB,t,vert}, consists of a set of parallel cross sections o1, o2,..., o m at axial locations z1, z2,..., z m. The spacing of these images makes the voxels effectively anisotropic with 1:1:8 side length ratio, so we form a new, roughly isotropic set of object indicator cross sections, o 1, o 2,..., o n at z 1, z 2,..., z n which we will denote O. We think of the task as a univariate interpolation problem. For each ( x, yz, i ) where i 1.. m we associate the Euclidean distance, d i, to the nearest point on the boundary of o i. We assign a negative distance if ( x, yz, i ) o i meaning ( x, yz, i ) is inside the object, and positive, if ( x, yz, i ) is in the complement of the object. We define a univariate function, f s, which computes a spline-based interpolation of the d1, d2,..., d m and we evaluate fs ( z j ) at j = 1.. n for the distances d j at ( x, yz, j ). We assemble these d j for each ( x, yz, j ) into D which is a 2.5D distance transform. It is not a full 3D distance transform because the 2D distance transforms have been simply extended in the orthogonal direction. We then compute a full 3D Euclidean distance transform, in two steps. First we consider ( x, yz, j ) to belong to O if fs( z j) 0. Then for every ( x, yz, j ) we compute the Euclidean distance, d i to the nearest point in the boundary of O and assign it to D ( x, yz, ). These processing steps to construct the 3D distance transform LVB,, t horz j D for the LV blood cavity at time t for the horizontally tagged data are summarized in Fig. 5. O LVB,, t horiz LVB object indicator cross sections D LVB,t,horiz set of 2D DTs for LVB D O LVB,, t horiz D,, LVB,, t horiz D DT interpolated and extrapolated for LVB Fig. 5. Converting a set of 2D ventricular blood indicator images into a 3D distance transform. 3D object image LVB t horiz 3D DT for LVB We follow similar steps to compute the distance transform, D RVB,, t horz, for the RV blood cavity. We find it helpful to also compute an estimate of O LVB which does not include the papillary muscles, which we denote O LVBNoPapillary. We estimate the cross sections of O LVBNoPapillary by computing the convex hull of O LVB,t,horz which we denote O LVBch,t,horz. Then we repeat the above processing to form distance transforms for the vertically tagged data. 6

7 Suppress residual noise and MR artifact by averaging DTs from multiple tag orientations and time frames We form the weighted average (Fig. 6. right-most column) of the 3D distance transforms from the multiple tag directions and time frames, using the following equations: t2 t2 LVB = αhorz αt LVB,, t horz + αvert αt LVB,, t vert t= t1 t= t1 D D D t2 t2 LVBNoPapillary = αhorz αt LVBch,, t horz + αvert αt LVBch,, t vert t= t1 t= t1 D D D t2 t2 D = α α D + α α D RVB horz t RVB,, t horz vert t RVB,, t vert t= t1 t= t1 The coefficients α horz and α vert are set to 0.5 to weight equally the information from the horizontally and vertically tagged images. We have roughly eight systolic frames and we typically set t 1 = 2, t 2 = 4, and α 2 = 0.5, α3 = α4 = 0.25 to preferentially select and weight the early systolic time frames (1) Exploiting time redundancy Phase n n+1,...,n+m-1 Phase n+m Initial shape estimate Features from vertically tagged data Features from horizontally tagged data Fig. 6. Exploiting redundancy in time and data enables a robust estimation of 3D shape ventricular shape. 2D features are combined to form 3D distance transform features representing the shape of each ventricle at each time. Distance transforms from acquired time and tag orientation data are combined to estimate the shape of the ventricles at early systole. 3.3 Mesh creation In this section we propose a systematic procedure for dividing the myocardial volume into elements suitable for the finite element method. Such elements must satisfy certain criteria. The elements must be convex; that is none of angles at the element vertices may exceed 180, and for maximum numerical stability the angles should be should be close to 90. The elements must also be non-overlapping and must span the volume enclosed by surfaces that bound the myocardium. Moreover, we would like a mesh that uses a small number of element types for which integration rules are readily available to make the system computationally tractable and manageable. We would also like elements aligned to a spherical or prolate-spheroidal coordinate system, because these systems have been widely adopted in the literature for referencing points in the myocardium. Such a coordinate system will enable us to display cut-away views of the model, showing what is occurring at different transmural depths or at different points from apex to the base. We would like to form a biventricular model, therefore we seek a meshing procedure with RV sewn to LV elements. For our meshing strategy we select two volumetric elements: the six-sided hexahedral element and five-sided wedge element. To 7

8 facilitate ventricular stitching, we define vertices along the curve of insertion of the RV into the LV. To do this we first find 3D surface estimates for the endocardial and epicardial surfaces Three-dimensional endocardial and epicardial surface construction We use the distance transforms to implicitly define surfaces heart model surfaces. The LV endocardial surface is defined as: SLVEndo = { x, y, z D LVB ( x, y, z) = 0}. This is a set of points approximated by the set of triangular faces extracted using the marching cubes algorithm. The LV epicardial surface is similarly defined as: S = { x, y, z D LVBNoPapillary ( x, y, z) = α LVThickness}. We estimate the thickness of the LV myocardium, α LVThickness, as the thickness of the septal wall measured by computing the trimmed mean distance between points on the surface of the LV blood cavity, SLVB = (( x, y, z) D LVB = 0) and points on the surface of the RV blood cavity, ( x y z D ) SRVB = (,, ) RVB = 0. We form an initial estimate of the RV myocardium freewall thickness as 1 α = α and note that the actual thickness of the freewall will be accurately determined during RVThickness 2 LVThickness model fitting. The endocardial (red) and epicardial (translucent red) surfaces of the LV are shown in Fig. 7d. To form an anatomically inspired insertion curve of the RV into the LV, we blend the DTs from the LVB and RVB. To begin, we form surfaces that smoothly blend the RV into the LV. We model the blending problem as the extraction of isoenergy surface in an energy field. We think of charged positive particles on the surface of S LVB and S RVB whose energy level exponentially drops exponentially with distance from the charges, while adding appreciably at positions between both surfaces effectively blending them (equation 2): D RVEpi DRVB α RVThickness DLVB 0.5( αlvthickness ) = exp + exp αrvthickness 1.5( αlvthickness ) D RVB DLVB 0.5( α LVThickness ) D RVEndo = exp + exp αrvthickness αlvthickness (2) These distance transforms implicitly define surfaces, a portion of which are the desired RV epicardial and endocardial surfaces. The RV epicardial surface is extracted from: SRVEpi = { x, y, z D RVEpi ( x, y, z) = 0}, while the RV endocardial surface comes from: SRVEndo = { x, y, z D RVEndo ( x, y, z) = 0}. Of the three surfaces rendered in Fig. 7a, the circular one on the left (translucent red) is the LV epicardial surface, S, while the two surfaces extending to the right are S RVEndo (translucent purple) and S RVEndo (yellow). They intersect the surface, S, forming anatomically realistic cusps at the insertion lines shown in blue in Fig. 7b. In an operation, analogous to constructive solid geometry, we take the biventricular myocardial model to be the union of the volume between the LV surfaces (Fig. 7d) and the volume between RV surfaces that is outside S, and we discard the portion of S RVEndo and S inside S. We compute the two intersection curves of the RV and LV (Fig. 7b). The first curve, C IntEpi, is formed by the intersection between the LV epicardial surface, S curve, C IntEndo surface, S RVEndo., and the RV epicardial surface, S RVEpi. The second, is formed by the intersection between the LV epicardial surface, S, and the RV endocardial 8

9 ω v a b u c d e f Fig 7. (a) Insertion curves form at intersection of blended RV-LV surfaces and LV epicardial surface (b) Insertion curves help define spherical coordinate system (c) Surface subdivision. Transmural projection through LV in (d) and RV in (e) form the points for biventricular mesh, (f) Coordinate projection We construct a biventricular mesh with elements arranged along spherical coordinate curves ( uvw,, ) shown in Fig. 7b. For the LV epicardial surface, the latitudinal u coordinate starts at the apex and increases towards the base (top) of the heart. The longitudinal coordinate, v, begins at the insertion curve and traverses counterclockwise around S. The radial coordinate, w, begins at S LVEndo and increases transmurally. To mesh RV elements to LV elements our process includes these steps: 1. Compute α numbiventdivisions, the number of divisions for each intersection curve. 2. Divide C IntEpi into α numbiventdivisions divisions. 3. Extract circumferential curves around the epicardial surfaces and septal wall. 4. Evenly distribute points along each circumferential curve, C circ, according to arc length. Table 1. lists the parameters controlling mesh resolution, along with their descriptions. Table 1. Mesh parameters. Parameter Description of aspect affected by parameter α Number of U divisions from apex to base numudivisions α Number of V divisions around the LV freewall numvdivisionslvfreewall α Number of V divisions around the LV septal wall numvdivisionsseptalwall α Number of V divisions around the RV freewall numvdivisionsrvfreewall α Number of W divisions for the LV numwdivisionslv α Number of W divisions for the RV numwdivisionsrv We estimate the number of U divisions to allocate to the upper portion of the mesh, based on its extent compared to the height of the entire biventricular volume, and evenly divide the intersection curves into the 9

10 appropriate number of equal, arc length segments. We denote the dividing points as start point, p i and label them with a 1 in Fig 7c. To compute circumferential curves on S and S RVEpi extending from these start points on the intersection curves, we associate with each point on the surface, S, a distance value equal to the axial height, z, of the point and thereby form a scalar function on the surface. Beginning at each start point, p i on the intersection curve, we traverse the surface by moving along the tangent to this scalar surface function until we arrive again at the intersection curve, C IntEpi, on the other side of the heart. This yields the set of circumferential curves along the LV freewall, { i, circlvfreewall} yields circumferential curves, { C i, circrvfreewall} curves,{ C i, circseptalwall} αnumvdivisionslvfreewall circumferential curves yields point sets, { p circlvfreewall, j}, { p circseptalwall, k} { circrvfreewall l} α C illustrated in magenta in Fig. 7c. A similar process, illustrated in red along the RV freewall, and circumferential, shown in magenta along the septal wall. Evenly distributing points along these numvdivisionsrvfreewall p, for the LV freewall, septal wall and RV freewall respectively. l= 1 j= 1 α numvdivisionsseptalwall To build a thick volumetric mesh, we project the points from these outer walls transmurally to the inner walls. We begin by defining two 3D labeled volumes which identify the isotropic voxels that constitute the LV and RV. For example, the LV labeled volume is a labeled 3D rectilinear lattice with label, L1, assigned to positions (voxels) that are inside S LVEndo (the LV blood), while label L2 is assigned to voxels outside S LVEndo but inside S (the LV myocardium), and L3 is assigned to voxels outside S (exterior to the LV). We assign numeric labels with L1<L2<L3 and think of the resulting labeled volume as a 3D intensity image, f. We project each point on the walls, through the appropriate LV or RV labeled volume, by defining the 3D gradient T v xyz,, = u xyz,,, v xyz,,, wxyz,, which minimizes the energy functional: ( ) vector flow field ( ) ( ) ( ) ( ) ( ) μ ( x y z x y z x y z ) 2 2 E v = u + u + u + v + v + v + w + w + w + f v f dxdydz (3) 2 2 The PDE, v t = μ v f ( v f ), minimizes this functional and is found through the calculus of variations. 1 n+ 1 n 2 We use the discrete temporal derivatives, ut = Δt ( ui, j ui, j) and spatial derivatives, u L3x3x3 (, i j) u(, i j) and a discrete approximation to the 3D Laplacian operator: L 3x3x3= ; ; (4) to provide speed and accuracy with a hardware-based convolution to iteratively solve the PDE until an * equilibrium solution, v, is reached. The characteristic curves of the solution are traversed to project each point from the outer walls and the resulting curves (thin blue lines in Fig. 7d,e) are divided according to their arc p and those in length from the outer to the inner wall. We denote the division boundary points in the LV as { } LV the RV as { } RV p. Our solution produces points which are well suited to form the vertices of a hexahedral mesh for numerically stable FEM analyses. The characteristic curves of a PDE solution do not cross over one another, which helps ensure that the volumetric elements that we form are convex. The projection curves also leave the originating surface at an angle nearly orthogonal to the surface and intersect the destination surface at an angle near 90 because the vector flow is defined from the gradient of the myocardium-identifier volume. This helps k = 1, and 10

11 to ensure that the interior angles formed by a line segment on an epicardial or endocardial surface and a projection curve are ~90. Researchers (Yezzi and Prince, 2002) have proposed a related method to project a grid through annular structures, such as the left ventricle without its apex; our method can also mesh nonannular structures, such as the biventricular myocardium, which contain parts that meet at the RV insertion and the apex Mesh creation Hexahedra are used from the base of the mesh (Fig. 7f) until the apex, where wedge elements are used. When compared with the commonly used mesh by tetrahedralization, our mesh facilitates the display of information for a physician accustomed to peel away views of the heart model depicting variation at different transmural depth from epicardial surface to endocardial surface. Also, the spherical coordinate system we use has historically been used to report quantitative results in cardiac physiology literature. 3.4 Mechanical model construction We embed material properties to approximate aspects of myocardial tissue, such as stiffness and compressibility. We use subject-independent, spatially invariant material properties for model-based segmentation. These properties regularize the fit of our model in the presence of residual image noise, and compensate for a missing boundary edge or region feature due to poor image quality or subject movement. (We note that were we to compute strain distribution as well, we would use anisotropic, spatially-variant, subject-specific, material properties.) A 3D isotropic, elastic material is characterized by two properties: (1) 1 Young s modulus, E > 0, and (2) the Poisson ratio, 1< ν 2. Young s modulus is the ratio of the stress to strain on the loading plane in the loading direction. Increasing Young s modulus makes the material stiffer; decreasing it makes the material more pliable. The Poisson ratio, ν, is the ratio of lateral strain to axial strain; it describes how much a material will expand (contract) in one direction while being compressed (stretched) in another direction. Increasing ν makes the material less compressible. A completely incompressible material, (i.e. ν=0.5), undergoes an equal expansion (contraction) in one direction for a given compression (stretching) in another. An ideally elastic material, (i.e. ν = 0.0 ), can be completely compressed (stretched) in a given direction without expansion (contraction) in another. We have found that E 0.2 works well for the image quality in our database. We have also found that ν 0.2 works well during the initial fitting of our model to the data, which typically requires a fair amount of expansion to fit the subject s anatomy. In the remaining time frames, we increase the Poisson ratio, ν 0.4, to better approximate the compressibility of myocardial tissue and to compensate for any missing epicardial or endocardial features at those time frames. Our derivation of the elemental stiffness matrix for a linear elastic material, K el, closely follows that given in (Cook et al., 1989) and the shape functions we use follow the presentation given in (Macneal, 1994). From the individual K el, we assemble a global stiffness matrix, K, and use it to compute the internal forces of the model: f int = Kq where q is the displacement of the model nodes. 3.5 Force field computation Enodcardial surface forces To extract the image forces to fit the endocardial surfaces, we let O be a member of the set of 2D object indicator images for left and right ventricular blood cavities (LVB and RVB): O { O, O, LVB,t,horz LVB,t,vert OLVB,t,LA, O RVB,t,horiz, ORVB,t,vert, O RVB,t,LA}. We compute the gradient, O, of the indicator images, and interpolate using the gradient vector flow (GVF) operation, forming six 2D vector fields of the form f (, ) {,,, t x y = flvb,t,horz f LVB,t,vert, flvb,t,la, f RVB,t,horiz, frvb,t,vert, f RVB,t,LA} (5) 11

12 where t indicates the time frame. The 2D vector fields, { RVB,t,horiz, RVB,t,vert, RVB,t,LA} f f f, are combined to form 3D forces acting on the RV endocardial surface nodes. The reliability of the information in the images depends on angle between the tags and the boundary surface of the myocardium. In the circled region of Fig 8a the boundary of the myocardium is obscured by the tags while in the corresponding orthogonally tagged image (Fig 8b), the boundaries are plainly visible. 1 Confidence 0.5 a b c Fig. 8. (a,b) Information reliability depends on tag-boundary angle. (c) Confidence as a function of the angle between tag tangent and boundary normal directions. We define our confidence, C H, in the features extracted from an image based on the angle, θ, between the tag tangent, ˆT, and the normal to the boundary, n ˆb, as: C H 1 2 ( θ) = ( 1+ cos2θ) = cos θ = ( Tˆ nˆ ) 2 2 We apply the image forces from f,, t to the model where its edges intersection the image planes. For example, in Fig. 9a, an edge, ab, from one of the hexahedral elements is shown intersected by a SA image plane at the point d. Two image forces influence d: (1) the force from the horizontally tagged image, g horz, and (2) the force g vert from the vertically tagged image. We compute the weighted sum g of g horz and g vert, using the affine combination: ( θ) ( 1 ( θ) ) b = H horz + H vert (7) g C g C g Z g Z ' Z a horz Z f g ' g a f h a ' XY d g vert f Z f c XY f c b d X X b X b Y Y Y a b c Fig. 9. 2D forces from SA images (a) and LA images (b) are interpolated and combined forces to form node forces (c). Forces from the long axis (LA) images also influence the model. For example, in Fig 9b edge ab is intersected XY Z at point c, where the local image force, f, has one component, f, parallel to the XY plane and another, f, parallel to the Z axis. We interpolate the forces from all image planes that act on model edges to form the forces acting on the edge s endpoints which are our model s nodes (e.g. denoted with a in Fig 9c). The interpolation involves weighting the forces based on the relative distance of the intersection point to the edge s endpoints and computing each node s 3D force by combining the intersection force from all edges connected to the node. (6) 12

13 3.5.2 Epicardial surface forces To extract image forces for the single epicardial surface, we let V be a member of the set of the corrected, standardized image volumes, V { Vcs,t,horz, Vcs,t,vert, Vcs,t,LA } (e.g. containing images like those in Fig. 3 bottom rows). For each V, we apply a grayscale morphological closing operation to suppress the tags, yielding images we denote as V, and we apply a median filter, to reduce artifacts from morphological structuring element shape, and denote the resulting image volume V,med. We then perform a scale-based noise suppression (Saha and Udupa, 2001) on each 2D image, and denote the resulting image volume V,medSbf (Fig 4d). We compute the gradient V,medSbf, whose magnitude is shown in Fig 4e and apply the GVF operation to form a 2D force field: { } f (, ),,, xy = f f, f t cs medsbf,t,horz cs medsbf,t,vert cs medsbf,t,la for each time frame, t. Forces at model intersection points are combined to form node forces acting on epicardial model surfaces using the weighting methods illustrated in Fig Intermediate constraint segmentation We think of myocardial segmentation as a model fitting process governed by a force balance equation in which internal and external forces compete to explain the features extracted from tmri. The motion of our model s nodes are governed by the Lagrangian equations of motion, Mq + Dq + Kq = f. We set the mass matrix, M, to zero and the dampening matrix, D, to the identity to simulate a model without inertia that comes to rest as soon as the forces equilibrate or vanish. In the resulting first order system, q + Kq= f, K is the stiffness matrix, f is the external image force, and Kq is the internal regularizing force. We solve this equation with the explicit Euler difference scheme: q( t+δ t) = q( t) +Δt( f Kq ( t) ) (9) and evolve our model until the forces equilibrate or vanish, as the term ( f Kq ( t) ) goes to zero. To track the heart s shape change throughout systole, we begin by fitting the model to the first frame, t n, in which the motion of the blood has washed away the tags. To fit the model to subsequent time frames (t> t n ) we propagate the previous frame solution and fit the model to the data in the new time frame. To fit the model to an earlier time frame, (e.g. t n-1 ) in which the blood is tagged, we propagate the solution from t n and fit the model but we do not apply forces to the endocardial walls because the insufficient blood-myocardium contrast yields unreliable blood regions features. Both the epicardial and endocardial bounding surfaces are recovered by the combination of (1) physiologically based mechanical constraints embedded into the model, (2) the equilibrium model fit at t n, and (3) reliable epicardial features at t n and t n-1. For all of the images in our database we have contours interactively-drawn by an expert delineating the epicardium and endocardium. Each heart is imaged with three orthogonal tag directions and we have three corresponding expert heart-contour datasets. We combine them into one expert surface to facilitate evaluation of our automated segmentation, using a process similar to the one described in section for converting 2D blood region features into 3D surfaces, only now we form 3D distance transforms from stacked expert contoured regions by weighted averaging. This produces one expert DT for each feature (i.e. the LV enodocardial surface, the RV endocardial surface, and the combined LV and RV epicardial surface) whose zero isosurface implicitly defines the expert surface for the feature. (8) 13

14 4. Results We have found the proposed biventricular mechanical model segments the myocardial boundaries with high accuracy throughout systole. Our segmented surfaces and the expert s surfaces are quite similar in shape and location throughout systole for the endocardial surfaces (Fig. 10) and for the epicardial surface (Fig. 11). These results are illustrative of the advantage of computerized segmentation strategies such as ours compared to interactive expert segmentation. Our method combines all data (Fig. 2) and tends to avoid the expert s spurious undulations (second row Fig. 10) that may be created when individual slices are traced and may not be aligned exactly, due to the serial breadth-hold MR acquisitions. Frame 2 Frame 4 Frame 6 Frame 8 Fig. 10. Endocardial surfaces throughout systole. First row: segmented endocardial surface for LV (red) and RV (magenta). Bottom row: the expert s surfaces. Frame 2 Frame 4 Frame 6 Frame 8 Fig. 11. Epicardial surface throughout systole. First row: epicardial surface (translucent orange). Bottom row: the expert s surface. The other subjects in our database, show similar agreement between the model s surfaces and the myocardial boundaries visible in the tmri. For example Fig. 12 and Fig. 13 show our results on two subjects, where the first row in each figure shows the coronal view of the fitted biventricular model throughout systole, while the second row shows the axial view. 14

15 Frame 2 Frame 4 Frame 6 a b c d e f Fig. 12. Subject 9n model fitting results. First row: coronal view. Second row: basal axial view. Frame 2 Frame 4 Frame 7 a b c d e f Fig. 13. Subject 8n model fitting results. First row: coronal view. Second row: mid-ventricular axial view. For a quantitative analysis, we compute the distance between points on each segmented model surface, A and points on the corresponding expert surface, B, by computing the distance dab (, ) = min a b for all points a on b B the segmented surface and dba (, ) for all points, b on the expert s surfaces. We map the errors onto the model s surfaces (Fig. 14) to facilitate evaluation of our segmentation. Surface points with a distance of 0 mm to the expert s surfaces are blue, while those within 4.5 mm to 5.5 mm are rendered yellow to red. Fig. 14 is typical of the accuracy level we achieve for all subjects; our surfaces tend to be mostly blue or cyan with errors restricted to a few locations. Further investigation of these locations reveals that either residual noise slightly corrupted 15

16 the feature extraction or our algorithm is actually more correct with respect to the tmri visible boundary than the expert s surfaces, since we integrate more information to compute a single model. Fig. 14. Distances between segmented and expert endocardial surfaces are texture mapped onto the segmented model (axial view). Our model-based segmentation with intermediate-strength, local mechanical priors, demonstrates excellent ability to deform its initial shape to the subject s specific, frame-specific anatomy. For example, Fig. 15 plots the 90 th and 50 th percentile cumulative distance errors of the boundary surface nodes during the model fitting of a frame. There are four curves in the figure for each percentile group, one for each of the three boundary surfaces and one for all surfaces combined. The fitting error decreases during fitting, reaching a limit of approximately 5mm at the 90 th percentile and 2mm at the 50 th percentile. Percentile distance error in mm n percentile error during fitting of frame 2 AllSurfaces90th AllSurfaces50th Epi90th Epi50th RVEndo90th RVEndo50th LVEndo90th LVEndo50th Fitting iteration Fig. 15. Plot of the 90 th and 50 th percentile error for all of the model s surfaces during the fitting of a frame. Combining the equilibrium errors from all systolic frames, we find that the proposed segmentation tracks heart shape faithfully as shown in Fig. 16. This figure plots the cumulative distance errors throughout systole. There are no long tails appearing at the 100 th percentile, which indicates that all points on our model remain proximal to the expert surfaces during systole. For the LV endocardial surface, all points track within 5.5 mm of the corresponding expert surface, and 90% of the points are within 3.4 mm throughout every frame of systole. For the RV endocardial surface, all points are within 7mm, 90% within 4.2mm, and for the epicardial surface, 100% of the points are within 8mm of the experts, with 90% within 5.0mm. Other subjects show nearly identical plots. 16

17 Percentage of surface points AllSurfaces Epi RVEndo LVEndo Distance error in mm Fig. 16. Cumulative distance error plot. Our segmentation results also demonstrate a significant improvement over those found when surface models are used for the segmentation which employ thin-plate type constraints (Montillo et al., 2003b). For comparison, this method produced an LV endocardial surface for which all points were within 9mm, 90% within 3.3mm, for the RV epicardial surface all points within 17mm, 90% within 5.3mm, and for the epicardial surface, all points were within 15mm, and 90% within 6.3mm. 5. Discussion Our model-based segmentation is fully automated. Since user interaction is no longer required, the most time-consuming part of tmri analysis, which interactively takes 4 to 17 hours per subject (depending on the boundary detail level visible in each image), is reduced to just 1.75 hours of CPU time on a 2.4GHz Pentium 4. Such time savings free the expert for other tasks and make large volume studies practical. Furthermore, we expect the time savings to be even more substantial for facilitating the processing of 3D acquisition tmri, in which an order of magnitude more data must be processed per subject. The suitability of our system for handling the range of myocardial variations that can occur in normal and diseased hearts depends on the particular anatomical or physiological condition. If images can be acquired with sufficient resolution to enable our system to find the blood in the heart, it should be able to construct a model around the blood cavities. Consequently our system should handle the following variations well: variations in size, location, orientation of the myocardium due to subject s age, sex, weight, diseases that enlarge the heart (e.g. left and right ventricular hypertrophy), and diseases resulting in the regional heterogeneity of myocardial contractility. Additionally, researchers (Wu et al., 2002) have acquired in vivo, tmri of a genetically modified mouse heart with resolution sufficient for our system. A direct model-from-data approach (Haber et al., 2000) constructs a biventricular heart model directly on the initial features in an early systolic time frame, while parameter-function approach (Park et al., 1996; Park et al., 2003) makes apriori assumptions about the allowable variations of biventricular myocardial shape. Our system presents an alternative approach for implementing segmentation constraints. We blend features from multiple time frames to construct an initial model, and leverage volumetric mechanical constraints to regularize the fit to all frames, which is more constrained than Haber (e.g. in interpreting the first frame) yet less constrained than the parameter function approaches of J. Park and K. Park; hence our method provides new, intermediate-level constraints. 17

18 There are several types of elements that can be used to construct a volumetric mesh, such as tetrahedral and hexahedral. An advantage of our method is that a mesh constructed with hexahedra aligned to the circumferential, transmural and longitudinal coordinates corresponds closely to those traditionally used in the cardiovascular disease literature, facilitate intuitive cut-away visualizations (examples in Fig. 17), and may be more informative for physicians. a b c d Fig. 17. Our mesh design intrinsically supports visualization of (a) biventricular axial cutaways, (b) transmural cutaway showing the LV subendocardium, (c) axial cutaway showing the RV myocardium, and (d) longitudinal cutaway showing sections of the RV. Our system targets several tmri pulse sequences under development. The use of an intermediate-level of constraints should be useful for processing future data such as SSFP tmri, which has increased visible structural detail. Through the use multi-frame feature blending, we have demonstrated that our system can handle existing sparse noisy data; however, by decreasing the amount of feature blending, our system should also handle future dense, high contrast, low-noise data such as the coming 3D tmri acquisition. While we perform image post processing and feature extraction taking into consideration all data in the temporal sequence, in the future, we could improve the motion of the model nodes somewhat by incorporating 4D motion priors as in (Montagnat and Delingette, 2004) to smooth their systolic trajectory. Embedding our system in a tag tracking system may also yield similar benefits. We plan to investigate the performance gains using combing these methods and from using our method on SSFP and 3D acquisition pulse sequences. 6. Conclusions We have developed a method for segmenting the ventricles directly in tmri. This method fills a gap in the research left by a growing number of researchers who have traditionally focused on tag motion analysis. Our method considerably reduces model-based analysis time and was developed with an eye towards new tmri pulse sequences with increased visible detail and spatiotemporal resolution, for which our method s time savings and intermediate-level constraints should be even more important. Our method automatically constructs a biventricular mesh and demonstrates its suitability for FEM-based analysis. The mesh, built on traditional spherical coordinates, may prove particularly useful for studying a wide variety of biophysical phenomena requiring finite element analysis. References: 1. Amini, A., Prince, J., Measurement of Cardiac Deformations from MRI: Physical and Mathematical Models. Kluwer Academic Publishers, Dordrecht. 2. Axel, L., Dougherty, L., Heart wall motion: improved method of spatial modulation of magnetization for MR imaging. Radiology, 172, Chandrashekara, R., Mohiaddin, R. H., Rueckert, D., Analysis of myocardial motion in tagged MR images using non-rigid image registration. In Proc. SPIE Medical Imaging: Image Processing Cook, R., Malkus, D., Plesha M., Concepts and applications of finite element analysis, John Wiley and Sons, New York, third edition. 18