Interactive 3D Heart Chamber Partitioning with a New Marker-Controlled Watershed Algorithm

Transcription

1 Interactive 3D Heart Chamber Partitioning with a New Marker-Controlled Watershed Algorithm Xinwei Xue School of Computing, University of Utah xwxue@cs.utah.edu Abstract. Watershed transform has been widely used in medical image segmentation. One fundamental problem with it is over-segmentation. There are mainly two approaches to deal with this problem: hierarchical segmentation and segmentation with markers. The markers, either automatically extracted or interactively generated, are mostly used in the homotopy modification of morphological gradients prior to the watershed segmentation. Most of the current techniques does not incorporate domain knowledge of the data. In this paper, we propose a two-step marker-controlled watershed segmentation algorithm with simple domain knowledge incorporated: (1) Modified image foresting transform (IFT) algorithm is used to produce the initial segmentation; (2) The marker-controlled watershed region merging process is incorporated with domain knowledge. A min-cut criterion for region merging is proposed. This approach is effectively applied to the interactive 3D heart chamber partitioning. 1 Introduction Image segmentation is a very important task in medical image analysis, which provide important information for medical diagnosis, surgical procedures, and realtime navigation. Various techniques has been developed for different medical subjects from different imaging modalities. Cardiac imaging has been such a subject, where heart chamber segmentation plays an essential role. Due to clinical demands, left ventricle(lv) segmentation has attracted lots of attention, and a vast literature exists for this subject, as shown in Suri s survey [1]. In our medical application, we are interested in the segmentation of each heart chambers, especially left and right atriums from 3D CT data. The result of the segmentation is used for heart registration and surgical navigation. There are several categories in segmentation techniques: point based techniques, region based techniques, and contour based techniques [2]. Point based techniques rely heavily on intensity values and is prone to noise. Contour-based segmentation techniques such as active contours or level set methods are iterative processes and computationally intensive. Simple image editing tools requires lots of interaction of user, and is not effective for our application: the left and right sides of the heart are usually connected in a way that is difficult to determine the connection by eyeballing. The fast competitive region growing feature provided by watershed algorithm suggests a promising approach for this task. We are thankful to Dr. Chenyang Xu and Dr. Yiyong Sun for helpful discussions. G. Bebis et al. (Eds.): ISVC 2005, LNCS 3804, pp , c Springer-Verlag Berlin Heidelberg 2005

2 Interactive 3D Heart Chamber Partitioning 93 Though fully automatic segmentation is desirable, it is not reliable in reality due to variations in acquired image quality. Here we propose a semi-automatic approach using watershed algorithm controlled by interactively placed markers. Once the markers are in place, our algorithm can automatically segment the heart into four chambers. Due to the effect of the contrast agent, the whole heart can be easily extracted by interactively selecting a intensity threshold. Then the proposed marker-controlled algorithm is applied on the binary heart volume. The main contribution of this paper lies in the following aspects: 1) We incorporate domain knowledge, i.e. the shape property of heart into the algorithm. The anatomy shows that the heart has neck-like shape (or concave) in the connections between left and right sides of the heart and between the chambers. 2) A min-cut criterion is proposed for the new marker controlled watershed segmentation. The min-cut concept originally comes from graph-cut algorithms, here we use it as a region merging criterion. Firstly, a modified image foresting transform(ift) is applied to produce primitive regions; then the regions, treated as a graph, and controlled by markers, are merged by this criterion. The segmentation cuts at the minimum-connection between neighbored regions. 3) Our algorithm has been successfully applied to heart chamber segmentation and is currently being used in imaging product. And it is effective in partitioning binary data where objects has neck-like concave shape. The rest of the paper is organized as follows: Section 2 describes the related work; Section 3 introduces our approach in detail; In Section 4, we discuss the results. And finally, conclusion is drawn in Section 5. 2 Related Work The watershed transform, generally defined in terms of SKIZ (skeleton by influence zone) [3] has been widely applied to image segmentation, especially for medical images (e.g. [4]), ever since introduced by Lantuejoul and Beucher [5]. Watershed segmentation can be applied to both gray scale images or binary images. For gray scale images, the watershed transform is applied to the morphological gradient of the images. For binary images, the transform is applied to the inverted distance transform. Basically, the watershed algorithm transforms the image into local minima-based catchment basins and watersheds. One problem of watershed segmentation is its severe over-segmentation. Two approaches exist to overcome this problem: hierarchical segmentation [6], and segmentation with markers [7, 8, 9]. In the former approach, the height of the watershed (the lowest saddle point separating two neighbored basins), also called dynamics, is used to control the region merging process. In the marker-controlled approach, the markers, either automatically extracted or interactively placed, are used in homotopy modification of the morphological gradient prior to watershed transform, reducing the number of local minima of the image, and thus the number of primitive regions. The markers are usually applied in the pixel level, which controls the flooding process. An alternative approach is to apply markers in a region level, controlling

3 94 X. Xue the merging process of the primitive regions resulted from the initial watershed transform. Haris [10] combined region adjacency graph (RAG) representation of image regions and a nearest neighbor graph in the hierarchical segmentation. Meyer [11] describes a unified framework for watershed segmentation with markers in both pixel and region level, and minimum spanning tree algorithm is used. Markers in the region level, gives flexibility for interactive segmentation. In terms of implementation, Vincent and Soille s algorithm based on immersion simulation [8] has been widely used in medical image processing. Roerdink and Meijster [12] gives a survey of existing watershed implementation algorithms. Recently, Lotufo and Falcao [13] developed an optimal watershed algorithm based on the Image Foresting Transform(IFT), where the watershed is seen as the solution of the shortest-path forest problem in the graph theory framework. Nguyen et al. [14] reformulates the watershed segmentation as energy minimization problem and boundary smoothness prior is incorporated. Our algorithm is different from the algorithms available from literature. Firstly, most of the existing watershed algorithms does not incorporate the domain knowledge of the image data. We incorporate simple yet effective domain knowledge; Secondly, we propose a min-cut criterion for region merging, which is different from measures like watershed height or its analogy [15], nearest neighbor, or minimum distance, etc. Thirdly, we apply the marker in region-level with a novel implementation. 3 The Proposed Approach 3.1 The 3D Heart Chamber Segmentation Work Flow The work flow for the heart chamber segmentation is shown in Figure 1. The input is 3D CT/MR gray-scale volume. The binary whole heart is first extracted from the gray-scale image. Since the heart is solid structure, we can apply a hole-removal algorithm on the binary image. Then the Euclidean distance transform is performed on the holes-filled binary image. Prior to applying watershed transformation, the distance map is filtered (by a median filter, for example) and inverted. The local minima of the inverted distance map is detected. The hole-removal and filtering process helps to reduce the number of minima in the image, which, in turn, reduces the number of primitive regions generated by the watershed algorithm. Finally the IFT watershed algorithm is applied, and our marker-controlled region merging algorithm is performed on the primitive regions. If four markers are placed in the four chambers area, then the final output is a labeled volume with labels ranging from 1-4. We can then extract each chamber by label. As shown in Figure 1, to the right of the dashed vertical line, all the operations are performed internally and need only to perform once, while the marker-controlled region merging part is an interactive process. The user can visually manipulate markers to generate desirable results. If the result is not satisfactory, the user can erase the old markers and place new ones. The markers can be placed by picking marker points from 2D slice views or from the 3D volume view. For each region of interest, the number of markers

4 Interactive 3D Heart Chamber Partitioning 95 Fig. 1. Heart Chamber Segmentation Work Flow to be placed are not necessarily limited to one. Instead, a set of markers can be used to represent one heart chamber, but with the same marker label. Thus we can get robust segmentation. 3.2 The Modified IFT Watershed Algorithm The flooding process in watershed segmentation starts from the original or modified local minima. The IFT algorithm [13] treats the image as a graph, each pixel/voxel as a node. The watershed transform is solved as a shortest-path forest problem. It finds for each node the shortest path connecting it to the nearest root node, which are the marker nodes or local minima nodes. Let C(u) denote the cost along the path from its nearest marker up to node u, W (u, v) (here, W (u, v) =InvDist(v)) denote the weight associated with the arc (u, v), L(p) is the input marker image(the minima node of inverted distance transform are initialized to different labels) and also the output of the watershed partitioning. flag(p) indicates whether a node has been processed or not. The modified IFT algorithm works as follows: The queue used in figure 2 is an ordered queue. More detailed information about this algorithm can be found in [13, 16]. We chose IFT watershed algorithm because it is specifically appropriate for segmenting binary images, where the Euclidean distance transform is used. There is great similarity between the Euclidean distance transform and the IFT flooding process. We made several important modifications to the original IFT algorithm to incorporate our shape prior. First, the markers nodes in the figure are the minima nodes of the inverted distance transform. The cost of marker nodes are initialized to its minima values. Second, we label the watershed pixels while labeling the primitive regions, as shown in the bold font in Figure 2. For each newly labeled pixel, if it has a neighbor labeled as belonging to another region, then it is relabeled as a watershed pixel. To make sure that the watershed is one thin layer of pixels, 6-connected(for 3D) neighborhood is used for labeling primitive

5 96 X. Xue Fig. 2. Modified IFT Algorithm Fig. 3. Region Merging with Markers regions, while 26-connected neighborhood is used for watersheds. This algorithm creates a list of primitive regions and a labeled image. 3.3 The Marker-Controlled Region Merging Algorithm After applying the IFT watershed algorithm, we get the primitive regions and watersheds. The regions are then merged according the min-cut criterion: the neighbored regions with largest physical connection is merged first, while the merging stops at the regions with minimum connections. Here the watershed size, i.e. the number of pixels in the watershed that separates two neighbored regions, is used as a measure of physical connection of the regions. Our new approach to the marker-controlled region merging algorithm lies in the following three steps: 1) Relabel watersheds. Relabel watershed pixels according to the two regions it separates. The watershed pixels that separates the same two regions, will be assigned to a unique label. A list of watershed elements is created. Each watershed element records the neighbored region label and watershed size. In the end, a watershed table is produced. 2) Sort the watershed list in the descending order of watershed size. The watersheds are processed in the descending order of their size. This is where our shape prior knowledge and min-cut criterion are incorporated: the shape of the heart indicates that connections between chambers is narrower than connections between regions residing in the same chamber, thus we can merge regions with larger connections first and safely cut at the regions with minimum connections. Also, it is very important to process the larger regions earlier. This helps

6 Interactive 3D Heart Chamber Partitioning 97 to avoid mistakenly splitting large neighbor regions that belongs to the same chamber. Usually the larger the neighbored regions, the larger the watersheds. Thus, we can be sure that the segmentation will not result in major errors. 3) Marker-controlled region merging and relabeling. In this step, the markers picked interactively are bound with the primitive regions where they fall on. The corresponding region is assigned a label associated with the marker. If the marker falls on a watershed, one of the neighbored regions is selected and assigned the marker label. A region equivalency table(of regions merged) is maintained during the merging process. Starting from the watershed element list, the neighbored regions that are separated by the largest watershed size are merged first. There are four cases depending on the fact that whether the two neighbored regions has been relabeled with a marker label, as shown in figure 3. If the two primitive regions were not assigned marker label previously, the watershed and the two regions are placed on the equivalency table. If only one of them has a marker label, the watershed, the other region and its equivalent regions are assigned to the same new marker label. If the two primitive region have different marker labels, they are not placed onto equivalent table, and the watershed is assigned to the marker label of one of the neighbored regions. The merged regions will be updated to the same marker label. The merging process terminates when each watershed element in the list has been processed. Thus the image is relabeled with the new marker labels. Each region of interest(associated with a marker) can then be selected by specifying a marker label. 4 Results and Analysis The proposed algorithm has been successfully applied to the 3D heart chamber partitioning task. Several 3D CT/MR heart datasets from different patients have been experimented. The radiologists in a major hospital has done clinical validation and show very positive results. We used a PC with 2.0 Hz CPU with 1GB Memory to measure the performance. The fairly optimized algorithm can segment the 256x256x256 and 512x512x512 volumes in less than 20 seconds and 2 minutes respectively. Felkel et al. [17] discusses optimization methods of the IFT algorithm to further improve performance. Figures from (a) to (h)in figure 4 shows the volume view of original CT, the extracted binary whole heart image, the left side, right side, and the four chambers segmented by our algorithm. Experiments have been conducted to compare the interactive segmentation using our algorithm with the hierarchical watershed segmentation application implemented in ITK (Insight Segmentation & Registration Toolkit) [18], and our approach generates better result. The comparison pictures are not shown here due to limited space. This algorithm can be applied to other images that have similar shape property, and it is especially good for binary image segmentation.

7 98 X. Xue p h _ c _ a p S W š Š N P S g } g p h ] L É g V ] K _ g _ c N ] K _ g g É g V ] ] h _ [ Y V _ c N ] ] h _ [ Y c É g V ] Ä g a ] h _ L S g N _ c N ] Ä g a ] h _ L S g Fig. 4. 3D Heart Chamber Partitioning. Whole heart (b) is first extracted from original data(a): our algorithm can also be used to segment the whole heart (b) from (a) by placing one set of markers in object, while another set on the background. Then left side (c) and right side (d) are extracted from (b) by placing two sets of markers. The four champers (e)-(h) are segmented by placing four sets of markers on (b).

8 Interactive 3D Heart Chamber Partitioning 99 5 Conclusions A new approach to the marker-controlled watershed segmentation algorithm incorporated with object shape property and min-cut region merging criterion has been proposed. It is simple, yet very effective. The two-step segmentation algorithm introduced in this paper has been successfully applied to the interactive 3D heart chamber partitioning problem. Extension of this algorithm to solve other partition tasks with similar neck-like shape is readily available. References 1. Suri, J.: Computer vision, pattern recognition and image processing in left ventricle segmentation: the last 50 years. Pattern Analysis and Applications 3 (2000) Pham, D., Xu, C., Prince, J.: A survey of current methods in medical image segmentation. Annual Review of Biomedical Engineering 2 (2000) Soille, P.: Morphological Image Analysis. Springer-Verlag (1999) 4. Riddell, C., Brigger, P., Carson, R., Bacharach, S.: The watershed algorithm: a method to segment noisy pet transmission images. IEEE Transactions on Nuclear Science 46 (1999) Lantuejoul, C., Beucher, S.: Use of watersheds in contour detection. In: Proc. Int l Workshop on Image Processing, Real-Time Edge and Motion Detection/Estimation, Rennes, France (1979) 6. Najman, L., Schmitt, M.: Geodesic saliency of watershed contours and hierarchical segmentation. IEEE Transactions on PAMI 18 (1996) Meyer, F., Beucher, S.: Morphological segmentation. Journal of Visual Communication and Image Understanding 1 (1990) Vincent, L., Soille, P.: Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Transactions on PAMI 13 (1991) Rivest, J., Beucher, S.: Marker-controlled segmentation: an application to electrical borehole imaging. Journal of Electronic Imaging 1 (1992) Haris, K., Efstratiadis, S., Maglaveras, N., Katsaggelos, A.: Hybrid image segmentation using watersheds and fast region merging. IEEE Transaction on Image Processing 7 (1998) Meyer, F.: An overview of morphological segmentation. International Journal of Pattern Recognition and Artificial Intelligence 15 (2001) J.B.T.M., R., Meijster, A.: The watershed transform: definitions, algorithms and parallelization strategies. Fundamenta Informaticae 41 (2001) Lotufo, R., Falcao, A.: The ordered queue and the optimality of the watershed approaches. In: Math. Morphology and its Applications to Image and Signal Processing. Volume 18. Kluwer (2000) Nguyen, H., Worring, M., van den Boomgaard, R.: Watersnakes: Energy-driven watershed segmentation. IEEE Trans. on PAMI 25 (2003) Mangan, A., Whitaker, R.: Partioning 3d surface meshes using watershed segmentation. IEEE Transactions on Visualization and Computer Graphics 5 (1999) 16. Falcao, A., Stolfi, J., Lotufo, R.: The image foresting transform: theory, algorithms, and applications. IEEE Transactions on PAMI 26 (2004) Felkel, P., Wegenkittl, R., Bruckschwaiger, M.: Implementation and complexity of the watershed-from-markers algorithm computed as a minimal cost forest. In: Proc. Eurographics. (2001) 18. Cates, J.: Itk application:segmentation editor. (2003)