Graph Cuts Based Left Atrium Segmentation Refinement and Right Middle Pulmonary Vein Extraction in C-Arm CT

Transcription

1 Graph Cuts Based Left Atrium Segmentation Refinement and Right Middle Pulmonary Vein Extraction in C-Arm CT Dong Yang a, Yefeng Zheng a and Matthias John b a Imaging and Computer Vision, Siemens Corporate Technology, Princeton, NJ, USA b Healthcare Sector, Siemens AG, Forchheim, Germany ABSTRACT Automatic segmentation of the left atrium (LA) with the left atrial appendage (LAA) and the pulmonary vein (PV) trunks is important for intra-operative guidance in radio-frequency catheter ablation to treat atrial fibrillation (AF). Recently, we proposed a model-based method 1, 2 for LA segmentation from the C-arm CT images using marginal space learning (MSL). 3 However, on some data, the mesh from the model-based segmentation cannot exactly fit the true boundary of the left atrium in the image since the method does not make full use of local voxel-wise intensity information. Furthermore, due to the large variations of the PV drainage pattern, extra right middle pulmonary veins are not included in the LA model. In this paper, a graph-based method is proposed by exploiting the graph cuts method to refine results from the model-based segmentation and extract right middle pulmonary veins. We first build regions of interest to constrain the segmentation. The region growing method is used to construct graphs within the regions of interest for the graph cuts optimization. The graph cuts optimization is then performed and newly segmented foreground voxels are assigned into different parts of the left atrium. For the extraction of right middle pulmonary veins, occasional false positive PVs are removed by examining multiple criteria. Experiments demonstrate that the proposed graph-based method is effective and efficient to improve the LA segmentation accuracy and extract right middle PVs. Keywords: Left Atrium, Automatic Segmentation, Graph Cuts, Mesh Refinement, Pulmonary Vein Detection 1. INTRODUCTION According to a report from American Heart Association, 4 about 15% of all strokes (which is the second most common cause of death worldwide) are caused by atrial fibrillation (AF), which is a disease especially related with the left atrium (LA). Automatic segmentation of the LA together with the left atrial appendage (LAA) and the pulmonary vein (PV) trunks is vital for intra-operative guidance in radio-frequency catheter ablation, which is a widely used minimally invasive surgery to treat AF. 5 Details of the PVs provide accurate pulmonary venous drainage pattern, which can help physicians to adapt a generic ablation strategy for a specific patient in the pre-operative assessment. Compared to computed tomography (CT) or magnetic resonance imaging (MRI) scans, C-arm CT has an advantage that it is scanned during the ablation procedure on the same C-arm system that is used to capture real-time fluoroscopy for visual guidance of the ablation. Overlay of the extracted heart anatomy onto fluoroscopy can be performed easily using the machine coordinate of the C-arm system. Normally, un-gated reconstruction is performed using the projection images from the whole cardiac cycle; therefore, the 3D images are contaminated by cardiac motion artifacts. In addition, the streak artifacts may occur with various catheters inserted into the heart during the acquisition time. All these artifacts increase the difficulty of the LA segmentation. The intensity thresholding based method would not work well in C-arm CT since it is difficult to find a good threshold for the complicated intensity distribution among all images. Although the multilevel banded graph cuts segmentation of left atrium works effectively in CT images, 6 it is hard to estimate the boundary of the LA in C-arm CT images because the intensity contrast of boundary is much weaker than that in CT images. Leakage often occurs in the final result when applying the method to the C-arm CT images. Furthermore, the method is not fully automatic and a user needs to specify the center of the LA. Further author information: yefeng.zheng@siemens.com. Send correspondence to Dong Yang, don.yang.mech@gmail.com, or Yefeng Zheng,

2 Figure 1. Part based left atrium (LA) mesh model. Left: Meshes for the separate LA parts. Middle: Final consolidated mesh model. Right: Overlay of the model onto fluoroscopic images to provide visual guidance during surgery. Note: Cyan for the LA chamber, dark red for the appendage, green for the left inferior pulmonary vein (PV), magenta for the left superior PV, orange for the right inferior PV, and blue for the right superior PV. Recently, we proposed a robust model-based LA segmentation method using marginal space learning (MSL) 3 to extract six major LA parts: LA, LAA, left inferior pulmonary vein (LIPV), left superior pulmonary vein (LSPV), right inferior pulmonary vein (RIPV) and right superior pulmonary vein (RSPV). 1 The segmented LA parts are then merged into a consolidated mesh 2 as shown in Fig. 1. Although the model-based method is robust, slight defects may occur on the final segmentation results of some cases as shown in Fig. 2. The model-based method does not make full use of local details of the voxel-wise intensity information so that it may cause slight deviation of the final mesh from the true boundary. Moreover, the extra PVs between two major right PVs are also required for cardiac diagnosis and treatment planning and these important anatomies are not included in the model-based segmentation (Fig. 3). The drainage patterns of right middle pulmonary veins vary across patients. 71% people only have two major right pulmonary vein with small side brunches on the left atrium chamber. In other cases, there are one to three right middle pulmonary veins attached with the chamber between two major pulmonary veins. The case that one may have a single right pulmonary vein (venous ostium) also exists in rare cases. Therefore it is not trivial to explicitly model the pattern of right middle pulmonary veins. In this paper, we propose an automatic graph-based method to refine segmentation of the LA and extract the right middle PVs in the C-arm CT volumes. The model-based method 1, 2 provides the initial segmentation, which is used to define region of interest (ROI) for refinement and the extraction of right middle PVs. The initial segmentation also provides positive (foreground) and negative (background) seeds to automatically initialize the graph cuts procedure. Graph cuts optimization is then performed, which gives the best balance of boundary and region properties among all segmentations satisfying the constraints. 7 Occasional false positive PVs are removed in a post-processing step. Finally, the newly extracted foreground voxels are assigned into different parts of the left atrium. The workflow of the proposed method is shown in Fig. 4. In summary, we make the following contributions. 1. We propose an automatic scheme to refine the left atrium segmentation from the C-arm CT images. 2. We develop an automatic method to extract the right middle pulmonary veins from the C-arm CT images. 3. We develop a pruning scheme for the extracted PVs based on multiple criteria. 4. The whole procedure is very efficient, which mostly takes 5 seconds in 90% of datasets including the previous modal-based segmentation step. The remainder of the paper is organized as following. In Section 2, we describe the procedure to refine the previous model-based segmentation with the graph-based optimization. The extraction of the right middle PVs is described in Section 3. Section 4 presents the quantitative evaluation results and a brief conclusion follows in Section 5.

3 Figure 2. Small defects (yellow arrows) in the results of the model-based segmentation. 1 Figure 3. The right middle pulmonary veins (yellow arrows) need to be extracted, but are not included in the model-based segmentation LEFT ATRIUM SEGMENTATION REFINEMENT Our previous model-based left atrium segmentation approach 1 works well on un-gated C-arm CT, where thin boundaries between the LA blood pool and surrounding tissues are often blurred due to the cardiac motion artifacts (which presents a big challenge compared to the highly contrasted gated CT/MRI). The method is based on a part based LA model (including the chamber, appendage, and four major PVs) and each part is a much simpler anatomical structure compared to the holistic one. 1 We adopt marginal space learning (MSL) to detect and segment each part. 3 MSL is an efficient method to estimate the position, orientation, and size of the object in a 3D volume. After automatic object pose estimation, a mean shape is aligned with the pose as an initial mesh. A machine learning based boundary detector is used to guide the boundary evolution. For each LA part, we have an MSL based pose detector and a learning based boundary detector. To avoid segmentation leakage, the shape prior is exploited in the model based approach to segment the LA parts. However, independent detection of each part is not optimal and its robustness needs further improvement (especially for the appendage and PVs). We enforce a statistical shape constraint during the estimation of pose parameters (position, orientation, and size) of different parts. The segmentation process is computationally efficient, taking about 1.5 s to process a volume with voxels.

4 Input Retrieving Mask M0 from the Model Based Segmentation Refinement Generating the ROI R1 for Refinement Generating a Graph G and Positive/Negatives Seeds inside R1 Converting the Updated M0 to Mesh Using the Marching Cubes Algorithm Combining the New Mask M1 after Graph Cuts into M0 Segmentation Using Graph Cuts Retrieving Two Major PVs from Previous Segmentation Generating the Region-of-Interest (ROI) R2 for Extraction of the Right Middle PVs Combining the New Mask M2 after Graph Cuts into M1 Segmentation Using Graph Cuts Generating a Graph G and Positive/Negatives Seeds inside R2 Converting the Updated M0 to Mesh Using the Marching Cubes Algorithm Extraction Removal of the False Positives Output Figure 4. Workflow of the graph-based optimization. In this section, we present a graph-based method for the refinement of the left atrium segmentation in C-arm CT images. In our approach, inputs are a multi-labeled mask and an LA mesh which have been generated from the previous model-based segmentation. The processed result is internally represented as masks, and the output mesh is generated for visualization purpose via the marching cubes algorithm. 8 Our technique includes two steps: determination of the Region-of-Interest (ROI) and segmentation by the graph cuts. In the following presentation, M 0 and M 1 denote the mask from the model-based segmentation and the mask after the refinement of the segmentation, respectively. 2.1 Determination of the Region-of-Interest The region-of-interest (ROI) R 1 is a prerequisite for the automatic refinement to narrow down the consideration scope. The quality of the C-arm images varies greatly due to different patients characteristics and settings of the C-arm CT scanners. Most data contain visible noise or defects. The use of an ROI reduces the disturbance from image noise and redundant voxel information in the following processing, and also reduces the computation time. The ROI R 1 is determined along the surface layer of the mask M 0 since deviation of the initial segmentation from the true boundary is normally less than 2 mm. Morphological erosion and dilation are exploited to select voxels to form R 1 as shown in Fig Segmentation by the Graph Cuts Within the ROI, we adopt graph cuts to relabel the voxels of R 1 into foreground and background. At first, we select the positive and negative seeds (voxels) to form the graph and provide necessary guidance for global optimization. The positive and negative seeds are chosen by the morphological dilation based on R 1. During the dilation, the inner and outer layers of R 1 expand inward and outward for depth of m voxels respectively. m Z + is an adjustable parameter based on the varied volume resolution. The voxels of inner layer are positive seeds as foreground and the voxels of outer layer are negative seeds as background. Then, the nodes of graph

5 Negative Seeds The Region to be determined Positive Seeds Figure 5. Left: The ROI for the refinement of the model-based LA segmentation. Right: Distribution of positive and negative seeds in a C-arm CT image. Figure 6. Comparison before and after refinement of left atrium segmentation refinement. are the combination of positive seeds, negative seeds and voxels of R 1. The graph G is constructed by a region growing process connecting the negative/positive seeds and voxels of R 1 together without duplication. The growing region is recorded by a point-list P L and an edge-list EL. The seed is chosen randomly from R 1 and set as the first element of P L. The region grows via adding new neighbor of nodes n 1 P L continuously into P L from the positive/negative seeds or the voxels of R 1. When a new node n i is added into P L, the edge e i,j connecting n i and n j is recorded into EL. P L and EL represent the whole graph G of R 1 when no more node can be added into P L. The weights ω in the graph is assigned as follows ω exp [ (I p I q ) 2 ] 2σ 2. (1) Here, I p,q are the intensities of neighboring voxels and σ is the standard deviation of intensity distribution of R 1. 7 The labels of the voxels of R 1 will be determined by the global optimization process of the graph cuts algorithm. 9 After graph cuts optimization, the voxels of R 1 are identified as foreground or background. The voxels belonging to foreground form masks M 1 of the left atrium. They are then labeled into different parts of the left atrium by examining the label of the closest voxel in M 0. Fig. 6 shows two examples of LA segmentation refinement results. Slight defects (yellow arrows) on both datasets are fixed after refinement. 3. RIGHT MIDDLE PULMONARY VEIN EXTRACTION This section describes a graph-based method for the extraction of right middle pulmonary veins attached to the left atrium chamber. The input is the generated mask from the last step M 1. Similarly, the region-of-interest R 2 is determined for the following computation and the graph cuts optimization with a different setting is adopted to detect right middle pulmonary veins. To reduce the leakage of the graph cuts, a pruning step is exploited to remove false positive PVs. The final meshes are generated from the output masks by the marching cubes algorithm. In the following presentation, M 2 denotes the mask after the extraction of the right middle PVs.

6 Figure 7. Left two columns: The ROI for the extraction of pulmonary veins. Right two columns: The forbidden regions around the distal ends of the RIPV and RSPV trunks for pruning of the extracted pulmonary veins. 3.1 Determination of the Region-of-Interest The ROI R 2 for the extraction of the right middle PVs is constructed for avoiding unnecessary interference from the irrelevant background of the image environment. R 2 is the space between two major PVs (namely, RIPV and RSPV) to be examined in the extraction. An octahedron with six prisms is constructed as the boundary of R 2 based on the model-based segmentation as shown in Fig. 7. The two major prisms are the center lines of two major pulmonary veins. And, the extra PVs with minimum length 20 mm to be extracted are guaranteed to be inside of R 2. Admittedly, segmentation of voxels outside R 2 might also benefit from a graph cuts based method. But limiting the segmentation to voxels inside R 2 reduces leakages and computation time. 3.2 Graph Cuts Segmentation For extraction of the right middle PVs, the voxels in M 1 are denoted as positive seeds. The negative seeds are voxels outside M 1 in R 2 with intensities lower than a fixed threshold ζ, which is set to the 85 th percentile of the intensities of all voxels inside R 2. Parameters ζ is very critical in determining the accuracy of the final segmentation M 2. With a small ζ (which results in a small number of negative seeds in graph G), we can detect the right middle PVs with dark intensities, but leakage may occur in M 2. With a large ζ, we can avoid leakage, but some dark right middle PVs would be missed. In this work, we choose a small ζ to achieve a high detection rate of the right middle PVs and leakage (false positive PVs) is removed in a post-processing step. Inside the ROI, we build the graph with the weighted links by region-growing as described in subsection 2.2. After graph cuts, voxels inside the R 2 are categorized as foreground or background. The newly detected foreground voxels are merged into M 1 to form mask M 2. The voxel in M 2 are labeled into different parts of the LA by examining the label of the closest voxels in M Removal of False Positive PVs The graph cuts optimization is set to be sensitive for the target and may introduce noise and leakage to the segmentation results. The leakage often happens inside the right pulmonary artery, which touches the RSPV. The boundary between right pulmonary artery and RSPV is very thin and due to the cardiac motion artifacts, the boundary can completely disappear as shown in Fig. 8. The model based method can avoid the leakage because of the strong prior shape constraints of the RSPV enforced in the method. 1 But, the graph-based optimization does not enforce such a strong shape constraint; therefore local artifacts may affect the segmentation result. A pruning procedure is necessary to remove the leakage (false positive PVs) and it is conducted based on multiple criteria. We first perform connected component analysis for the newly added foreground voxels. Isolated connected components not connected to any LA part in M 1 are eliminated as noise. The remaining major connected components MCR i=1,2,... are labeled as one of LA, RIPV and RSPV according to the label of the part they are connected to. After that, multiple criteria are adopted for determining if MCR i is a false positive. The first criterion is based on the location of MCR i because the right middle PVs should originate from areas close to the LA chamber, not from a distal part of major PVs. Two octahedrons are built as the forbidden

7 Figure 8. The leakage in the right middle pulmonary vein extraction results after graph cuts. Figure 9. Right middle pulmonary vein extraction results. First Row: Comparison before and after right middle pulmonary vein extraction; Second Row: Comparison of mesh before and after right middle pulmonary vein extraction. regions around distal end of the RIPV and RSPV trunks as shown in Fig. 7. The position of the distal end is determined from the model-based segmentation. M CRi is regarded as a false positive and removed if it has voxels inside the forbidden regions. The second criterion is the maximal distance Dmax from voxels of M CRi to the background voxels q Dmax (p) = min (px qx )2 + (py qy )2 + (pz qz )2. p M CRi, q M / 2 (2) Here, p is a voxel in the M CRi, and q is a voxel from outside of M2. The Euclidean distance map for M CRi is built for verifying the distances between voxels in M CRi and outer background. Since the right middle PV is tubular-shaped and slimmer than the RIPV or RSPV, therefore the M CRi is a false positive if its Dmax is larger than a threshold Dthreshold. For all the results presented in this work, Dthreshold is fixed to 4 mm. At last, principle component analysis (PCA) providing three main axes of the point set in 3D space is also adopted as a judging criterion. For a tubular-shaped PV, the length along the major axis would be much larger

8 Table 1. Quantitative evaluation using Dice s coefficient on 140 datasets (70 big X-ray panel data and 70 small X-ray panel data). Dice s coefficient d 1 (for the whole mask) Dice s coefficient d 2 (only in the ROI R 2 ) Data Type Original Segmentation After Segmentation Refinement and Right Middle PV Extraction Big Panel Small Panel Big Panel Small Panel than the lengths along the other two axes. The PCA eigenvalues λ i=0,1,2 (λ 0 λ 1 λ 2 > 0) of the 3D co-variance matrix represent the lengths along three orthogonal axes. Thus, MCR i is regarded a true PV theoretically if λ 0 λ 1 and λ 1 λ 2. In our experiment, MCR i is a true PV if λ 0 > 3λ 1 and λ 1 < 2λ 2. Fig. 9 shows a few examples of the extracted right middle PVs. 4. EXPERIMENTS We collected 687 C-arm CT datasets, scanned by Siemens Axiom Artis zee C-arm systems at 17 clinical sites from Europe and the USA. Among them, 253 datasets were scanned with large X-ray detector panels (30 40 cm 2 ) and reconstructed to volumes with an isotropic resolution of 0.30 mm. The other 434 datasets were scanned with small X-ray detectors (20 20 cm 2 ) with a volume resolution of 0.18 mm. Due to the limited field-of-view of small X-ray detectors, the reconstructed volumes contain artifacts, especially around the volume margin. To quantitatively evaluate the LA segmentation accuracy, the LA (together with the LAA and four major PVs) are annotated using the part-based LA mesh model. 1 The mesh model can be easily converted to a mask. The right middle PVs are much more difficult to annotate since they are thin and have variable shapes. On some datasets, they can be easily confused with side branches of a major PV if the side branches originate around the ostium of the major PV. Instead of using a mesh annotation, the right middle PVs are annotated at the voxel level. A paint brush tool is developed to paint all voxels inside the ROI of the right middle PVs as foreground or background. Therefore, not only the right middle PVs are painted; the low-bifurcated side branches are also painted as long as they are inside the ROI. The voxel-wise annotation is performed slice-by-slice. Since they are much more time consuming to annotate than the other LA parts, the right middle PVs are annotated on a subset of the data. We randomly selected 70 cases from each data category (the small and large X-ray detector data), resulting in a total of 140 annotated cases for the right middle PVs. The segmentation quality is evaluated using the Dice s coefficient d d = 2 X Y X + Y. (3) Here, X, Y are two voxel sets, one for the annotated mask and the other for the automatically segmented mask. Dice s coefficient is a real number in the range of [0, 1]. The higher the Dice s coefficient, the better the detection. Table 1 shows the quantitative evaluation results. Here, d 1 denotes Dice s coefficient for the whole mask, which measures the overall improvement after LA segmentation refinement and extracting right middle PVs. d 2 denotes Dice s coefficient computed only in the ROI R 2, which measures the accuracy in the right middle PV extraction. We can see the segmentation refinement and the extraction of right middle pulmonary veins have improved the performance, comparing to the original results. The proposed method is efficient. The average computation time of the whole procedure (including modelbased segmentation, 1, 2 LA segmentation refinement, and right middle PV extraction) is about 5.0 seconds on a computer with a 2.66 GHz CPU and 3 GB memory.

9 5. CONCLUSION In the paper, we proposed a graph-based method to refine the model-based segmentation and extract the right middle pulmonary veins of the left atrium automatically in the C-arm CT images. To improve the efficiency of the proposed method, the regions of interest are defined and graphs are constructed for voxels in the ROI. The graph cuts method is adopted benefiting from the known voxels in foreground and background of the image based on the initial model-based segmentation. The false positive right middle PVs are removed in a post-processing. At last, we assign the labels of different parts of the LA to voxels in the final mask M 2 to generate the multilabeled LA mask and mesh. The method is very efficient and improves the original segmentation. Therefore it is potentially helpful for medical diagnosis and to guide ablation procedures for treating atrial fibrillation. REFERENCES [1] Zheng, Y., Wang, T., John, M., Zhou, S. K., Boese, J., and Comaniciu, D., Multi-part left atrium modeling and segmentation in C-arm CT volumes for atrial fibrillation ablation, Proc. Int l Conf. Medical Image Computing and Computer Assisted Intervention, (2011). [2] Zheng, Y., John, M., Boese, J., and Comaniciu, D., Precise segmentation of the left atrium in C-arm CT volumes with applications to atrial fibrillation ablation, in [Proc. IEEE Int l Sym. Biomedical Imaging], (2012). [3] Zheng, Y., Barbu, A., Georgescu, B., Scheuering, M., and Comaniciu, D., Four-chamber heart modeling and automatic segmentation for 3D cardiac CT volumes using marginal space learning and steerable features, IEEE Trans. Medical Imaging 27(11), (2008). [4] Laupacis, A., Boysen, G., Connolly, S., Ezekowitz, M., Hart, B., James, K., Kistler, P., Kronmal, R., Petersen, P., and Singer, D., Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation: Analysis of pooled data from five randomized controlled trials, Advances of Internal Medicine 154(13), (1994). [5] Karim, R., Mohiaddin, R., and Rueckert, D., Left atrium segmentation for atrial fibrillation ablation, Proc. of SPIE Medical Imaging (2008). [6] Lombaert, H., Sun, Y., Grady, L., and Xu, C., A multilevel banded graph cuts method for fast image segmentation, Proc. Int l Conf. Computer Vision, (2005). [7] Boykov, Y. and Jolly, M.-P., Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images, Proc. Int l Conf. Computer Vision, (2001). [8] Lorensen, W. E. and Cline, H. E., Marching cubes: A high resolution 3D surface construction algorithm, in [Proc. SIGGRAPH], (1987). [9] Boykov, Y. and Kolmogorov, V., An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision, IEEE Trans. Pattern Anal. Machine Intell. 26(19), (2004).