Multi-Atlas Segmentation of the Cardiac MR Right Ventricle Yangming Ou, Jimit Doshi, Guray Erus, and Christos Davatzikos Section of Biomedical Image Analysis (SBIA) Department of Radiology, University of Pennsylvania Abstract. As an entry to the MICCAI 2012 Cardiac MR Right Ventricle Segmentation Challenge, this paper presents a multi-atlas-based automatic pipeline for segmenting the right ventricle in MR images. Multiatlas segmentation relies on two major components: image registration to propagate segmentation labels into target image that needs to be segmented, and label fusion to effectively combine those labels from multiple atlases into final segmentation. In the challenge dataset, we observe different imaging fields-of-view (FOVs), different structures around cardiac structures, and as such, registration and label fusion become quite difficult. We propose to drive both components by an attribute-based similarity metric and a mutual-saliency-based reliability metric. The fundamental idea is to improve registration and label fusion by looking for corresponding voxels that are similar (as measured by their Gabor attributes in the neighborhood), and more importantly, reliably similar (as measured by the mutual-saliency of their matching) between atlas and target images. 1 Introduction In MICCAI 2012 Cardiac MR Right Ventricle Segmentation Challenge, participants are provided with training MR images from 15 subjects, each having expert-defined segmentation of the right ventricle (RV) on the End Diastole (ED) and End Systole (ES) cardiac-phase images. The task is to segment right ventricle on ED and ES cardiac-phase images from 16 testing subjects. The major difficulty of the segmentation task lies in the difference of the RV structure / shape / size among different subjects. A further complication is that different cardiac images cover different fields-of-view. Traditionally, this RV segmentation can be approached by two major branches of methods: deformable model based methods (e.g., [1]), and registration-based methods (e.g., [2]). The former branch trains a shape / curve model of the RV, and lets the curve model evolve in new subjects until it converges to RV boundaries. It takes advantage of the fact that different RV will have similar shape information. But the difficulty is how to define a proper energy function to drive the curve evolution towards to right boundary, which is usually not completely clear. Our proposed method belongs to the second major approach, which is to use image registration for the segmentation.
2 Ou, Doshi, Erus, Davatzikos We present a multi-atlas-registration framework for the segmentation. The central idea is to transfer those expert-segmentations in training images (i.e., atlases) onto target image through image registration, and then fuse the transferred segmentations to derive an ultimate segmentation. Multi-atlas segmentation has gained increasing interest in recent years [3 7]. One premise in this approach is that it allows using a priori knowledge, as encoded in atlas segmentations, to infer segmentation in target image via atlas-to-target image registration. Another premise is that it allows different atlases to correct each other s errors in a process often known as label fusion. The fused segmentation has shown remarkable improvement over single-atlas-based segmentation in various brain, cardiac and prostate structures. Recently, multi-atlas approaches have been applied to cardiac structural segmentations and obtained very promising results [8]. Despite exciting research in recent years, both image registration and label fusion are not without challenges. In registration, a fundamental question is how to find reliable correspondences across images, especially when different subjects cardiac image are usually taken at different fields-of-view (FOVs), containing different structures around heart, and exhibiting largely variable shape/size of the hearts. As a result, the registration, affine or deformable, may often times result in significant errors. In this paper, we propose to improve both registration and label fusion by attribute-based similarity and mutual-saliency-based reliability metrics. The main idea is the following: When registering an atlas to target, we rely more on those regions, compared to other regions, that can establish reliable matching. When fusing labels, we assign higher confidence / weight to those atlases, compared to other atlases, that are more reliably similar to the target at each voxel. We applied our method on cardiac images from 20 subjects. We obtained average dice scores of 0.627 (±0.254) and 0.629 (±0.246) for the segmentation of the endocardial and epicardial volumes respectively. 2 Methods 2.1 Registration We use image registration from multiple atlases to the target image in order to segment the right ventricle. Due to large variations in imaging protocols, structures, anatomies and even pathology conditions among different subjects, registration from an atlas to the target is a very difficult task. A recently-developed non-rigid registration algorithm is used for warping atlas images to the target. This algorithm, termed DRAMMS registration [9], finds voxel correspondences by using a rich set of geometric texture features at each voxel, other than by image intensity alone. The high dimensional multi-scale and multi-orientation image features are used to make each imaging voxel more distinctive and therefore better identifiable during search for correspondence. Furthermore, when registering an atlas to the target image, this algorithm relies more on the regions
Multi-Atlas-Based Segmentation of the Cardiac MR Right Ventricle 3 that can establish a more reliable matching compared to other regions. Such an approach is particularly well suited to the registration of cardiac images, where the two images may have significant differences, or even missing correspondence (i.e., some structures present in one image but not the other). In DRAMMS, voxels are matched by their geometric context other than intensity, and the whole registration is mainly driven by regions/voxels that can reliably match across images. We describe each voxel x by the geometric context of this voxel, in a d-dimensional multi-scale and multi-orientation Gabor attribute vector A(x). This attribute descriptor renders each voxel more distinctive than intensity information alone [9]. Then the similarity between two voxels x and y from two images is defined as 1 sim(x, y) = 1 + 1 [0, 1] (1) d A(x) A(y) 2 A pair of voxels x, y in two images is said mutually-salient, if they are similar to each other and meanwhile less similar to any other voxels in the neighborhood. In this case, the matching between those two voxels are reliable, because no other voxel in the neighborhood of y can replace it with higher similarity. The similarity and mutual-saliency values are used to modulate registration. Specifically, DRAMMS seeks a non-rigid transformation T, based on free form deformation (FFD) model [10], that minimizes the mutual-saliency-weighted attribute differences over target image domain Ω R 3, With DRAMMS registration, segmentation labels from all atlases can be mapped to the same target image space. The next sub-section describes how to fuse those multiple segmentation labels into a single segmentation in the target image. 2.2 Label Fusion Let N atlases, indexed by n, be each registered to the same target image via a deformation T n. A voxel u in the target image space Ω will tentatively have N segmentation labels propagated from all those N atlases, denoted as {label(tn 1 To fuse them into a single segmentation label, we use a similarity and mutualsaliency weighted voting strategy. Specifically, we first calculate the probability of this voxel having each of all L segmentation labels {1, 2,..., L}, i.e., l 1, 2,..., L Pr(label(u) = l) = n (u))} N n=1. 1 sim(tn (u), u) ms(tn 1 (u), u) 1(label(Tn 1 (u)) = l) 1 sim(tn (u), u) ms(tn 1 (2) (u), u) n Then, we assign the most likely label l to this voxel u, i.e., label(u) = l s.t. l = arg max Pr(label(u) = l) (3) l In equation 2, if we have sim(, ) 1 and ms(, ) 1, then the proposed label fusion scheme becomes the classic majority voting algorithm.
4 Ou, Doshi, Erus, Davatzikos 3 Results Cardiac images of ED and ES phases from 15 subjects, on which endocardial and epicardial volumes were manually segmented by an expert radiologist, are used as the training set. A set of cardiac images from 20 subjects is provided for testing the method. The method is applied on each test image using all training samples as templates. From the final segmentation, contour points of the endocardial and epicardial regions have been extracted and submitted for the evaluation. The technical performance of the method is quantitatively assessed through overlap measure (Dice metric, DM) and a distance-based measurement (Hausdorff distance, HD). Tables 1 and 2 show the average DM and HD values obtained. Table 1. Average Dice Scores Phase All Endocardial Epicardial µ σ µ σ µ σ ED and ES 0.628 0.250 0.627 0.254 0.629 0.246 ED 0.663 0.233 0.659 0.237 0.667 0.230 ES 0.582 0.264 0.583 0.271 0.582 0.258 Table 2. Average Hausdorff distances Phase All Endocardial Epicardial µ σ µ σ µ σ ED and ES 17.409 8.433 16.850 8.136 17.941 8.690 ED 17.549 8.587 17.664 8.699 17.437 8.509 ES 17.227 8.245 15.750 7.208 18.579 8.912 The average Dice scores for each subject are given in figure 2. We observed that the registration-based approach could fairly detect the right ventricle for most of the subjects. One of the subjects had a very low Dice score. According to our visual evaluation, it seems that the subject has significantly smaller ventricles that have been incorrectly matched to the training templates (Figure??). 4 Discussion We used a multi-atlas registration based method for segmenting the right ventricle on cardiac MRI. A similarity- weighted label fusion strategy is then used for combining the warped labels. A voxelwise weighting is used for combining
Multi-Atlas-Based Segmentation of the Cardiac MR Right Ventricle 5 Fig. 1. Average dice scores per subject. Fig. 2. A sample case with low Dice score. Segmented endo and epicardiac regions overlaid on the ED image atlas labels. We obtained promising results from a purely registration based perspective. An inconvenience of this approach is that the final segmentation might not preserve the mostly regular boundaries of the ground-truth masks. Applying morphological operations combined with an intensity-based correction step could significantly improve the final segmentation. Alternatively, a template selection approach, e.g. selecting a few templates that are the most similar to the target image after registration, might be applied. Finally, one of our perspectives is to use the segmentations as an initialization to a curve evolution approach like level sets.
6 Ou, Doshi, Erus, Davatzikos References 1. Robert M. Lapp, Maria Lorenzo-Valdes and Daniel Rueckert, 3D/4D Cardiac Segmentation Using Active Appearance Models, Non-rigid Registration, and the Insight Toolkit, MICCAI, 419-426, (2004). 2. Zhuang, X., Rhode, K.S., Razavi, R.S., Hawkes, D.J., Ourselin, S. A Registration- Based Propagation Framework for Automatic Whole Heart Segmentation of Cardiac MRI. Medical Imaging, IEEE Transactions on, 1612 1625, (2010). 3. Heckemann RA, Hajnal JV, Aljabar P, Rueckert D, Hammers A., Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. Neuroimage. 15;33(1):115-26, (2006). 4. Artaechevarria X, Munoz-Barrutia A, Ortiz-de-Solorzano C., Combination strategies in multi-atlas image segmentation: application to brain MR data, IEEE Trans Med Imaging. 28(8):1266-77, (2009). 5. Sabuncu MR, Yeo BT, Van Leemput K, Fischl B, Golland P., A generative model for image segmentation based on label fusion. IEEE TMI 29(10):1714-29, (2010). 6. Warfield SK, Zou KH, Wells WM, Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation, IEEE Trans Med Imag 23(7), 903-21 (2004). 7. Asman AJ, and Landman BA, Robust Statistical Label Fusion Through Consensus Level, Labeler Accuracy, and Truth Estimation (COLLATE), IEEE Trans Med Imag, 30(10), 1779-1794 (2011). 8. Isgum, I., Staring, M., Rutten, A., Prokop, M., Viergever, M.A., van Ginneken, B. Multi-Atlas-Based Segmentation With Local Decision Fusion Application to Cardiac and Aortic Segmentation in CT Scans. 1000-1010, (2009). 9. Ou Y, Sotiras A, Paragios N, Davatzikos C, DRAMMS: Deformable registration via attribute matching and mutual-saliency weighting. MedIA 15(4):622-39, (2011). 10. Rueckert D, Sonoda LI, Hayes C, Hill DL, Leach MO, Hawkes DJ. Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans Med Imaging. 18(8):712-21. (1999).