3D colonic polyp segmentation using dynamic deformable surfaces

Transcription

1 3D colonic polyp segmentation using dynamic deormable suraces Jianhua Yao 1, Ronald M. Summers 1 1 Diagnostic Radiology Department, Clinical Center National Institute o Health ABSTRACT An improved 3D method or colonic polyp segmentation was developed based on previous D method. The method is based on combination o 3D nowledge-guided intensity adjustment, uzzy clustering, and dynamic deormable suraces. Ater intensity adjustment and uzzy clustering, a deormable surace is employed to locate the polyp boundaries. The surace is also dynamically maintained to preserve the resolution and topology. The deormable surace is operated on a sub-volume o the data set and driven by image orces, balloon orces and internal spline orces. Compared to previous D method, the improved method produces much smoother polyp boundaries, and 3D eatures can be derived rom the segmentation, such as 3D aspect ratio, curvatures, and polyp wall thicness etc. The computer segmentations were validated with manual segmentations. Preliminary results showed that the average volume overlap percentage among 5 polyp detections was 80.6%. Keywords: Colonic polyp segmentation, deormable surace, CT colonography 1. INTRODUCTION CT colonography (CTC) is a minimally invasive technique or the examination o the whole colon and screening o colon cancers [1]. Computer Aided Detection (CAD) o colonic polyps in CTC has been under investigation in the past ew years [-4]. Polyp segmentation is essential in CTC CAD since it extracts entire polyp regions rom the image, and thereore comprehensive volumetric and shape eatures o polyps can be derived, such as density distribution, volume, curvature, and relationship with surrounding tissues. We deine the segmentation o a polyp as the 3D boundary surace and interior voxels o the polyp. Colonic polyp segmentation is a complex tas or several reasons. First, polyp shapes are irregular. Second, polyp sizes vary greatly. Third, the surrounding regions are complex. Sophisticated methods utilizing both the shape and densitometry inormation are necessary or successul colonic polyp segmentation. Several methods had been proposed or colonic polyp segmentation [5-7]. Yoshida and Nappi et al. [5] developed a segmentation method by means o hysteresis thresholding. They irst extracted voxels with low curvedness values and high shape index, and then clustered them based on uzzy logic, and used the cluster as the polyp segmentation. Their method may have trouble in segmenting big masses where the shape o polyps are usually irregular and voxels in the polyp region may not have similar curvedness and shape index to be clustered together. Jerebo et al. [7] used Radon transormation and Canny edge operators to detect polyp boundaries. They irst applied Canny edge operators to locate potential polyp-lumen boundaries, and used Radon transormation to connect the boundaries and identiy round shape polyps. Their method may ail in cases o polyps with irregular shape or polyps on colon old. Our previous polyp segmentation method [6] was based on combination o D nowledge-guided intensity adjustment, uzzy c-means clustering, and active contour models. The segmentation was conducted slice by slice. Ater D intensity adjustment and uzzy clustering is perormed on one slice, a D active contour model was applied to obtain the polyp boundary. The D process was then propagated to neighboring slices to orm a 3D boundary o the polyp. Results showed that this technique was capable o handling most polyps in our database. However, since no smoothing and continuity constraints were imposed between slices, the polyp boundary is jittery in the CT axial direction. The segmentation may also ail to propagate to adjacent slices i the slice interval is too big. To address these problems and improve the results, we proposed a 3D polyp segmentation method in this paper. The ramewor o the new method is similar to the previous one. Major improvements lie in two aspects: 1) the nowledge-guided intensity adjustment is extended to 3D; ) the D active contour models are replaced with 3D dynamic deormable suraces. Several 3D spatial

2 CT Colonography Surace based ilter Preparation Polyp candidates Sub-volume extraction Enhancement Knowledge guided adjustment Fuzzy Clustering Dynamic deormable suraces Segmentation 3D polyp boundaries Figure 1. Flow chart o 3D colonic polyp segmentation constraints are also imposed. The remainder o this paper is organized as ollows. Section elaborates our method. Section 3 presents our results and provides some discussion.. METHODS The low chart o our method is illustrated in Figure 1. The method has three stages: preparation, enhancement, and segmentation. In the preparation stage, the colon surace is irst extracted rom CTC using region growing and isosurace techniques [, 3]. Surace vertices are then iltered using their curvature and local density distribution. The iltered vertices are then clustered based on connectivity, and the centroid o each cluster is used as a polyp candidate seed. There are usually a large number o polyp candidates, including true and alse detections. Multiple detections may occur on the same polyp. Based on our observation, polyp detection is a local operation, i.e. only local regions around the polyp need be examined to identiy it. Thereore, a 64 pixels * 64 pixels * 30 slices sub-volume centered at the candidate seed is extracted rom the image volume (the dimension is chosen based on the size o the largest mass in our database, but it is an adjustable parameter in the algorithm). Operations in stage and stage 3 are conducted on the subvolume to speed up the computational process and reduce artiacts rom unrelated tissues (e.g. liver, bones). In the enhancement stage, a 3D nowledge-guided intensity adjustment strategy based on spherical ray casting is perormed to enhance potential polyp regions. Then, a uzzy clustering is applied to compute membership values o lumen air, polyp tissue and non-polyp tissue class or each voxel in the sub-volume. In the segmentation stage, an initial surace is irst placed at the centroid o the potential polyp region. The deormable surace is then driven by image orces computed rom the membership map o the polyp tissue class, together with external balloon orces and intrinsic model orces. The topological structure and resolution o the surace is dynamically maintained during the deormation process. Details o stage 1, especially colon segmentation and surace ilters, were described in [3]. Details o stage and 3, including 3D nowledge-guided intensity adjustment, and dynamic deormable suraces, will be elaborated in ollowing subsections.

3 B non-polyp polyp A non-polyp polyp (a) (b) Figure. 3D Knowledge guided intensity adjustment (a) original image; (b) curvature map (red: convex, green: concave, and blue: lat); spherical shooting rays, red dots are hitting points; (d) enhanced image (d).1 3D nowledge-guided intensity adjustment We designed a 3D nowledge-guided intensity adjustment strategy to enhance potential polyp regions in the subvolume surround the polyp candidate. Two pieces o nowledge have been used in our strategy: 1) Colonic polyps abut lumen air ) Polyp-lumen boundaries tend to have convex curvatures To mae use o the nowledge, the iso-boundaries between lumen air and colon walls were irst located using iso-value o -700 HU. The curvature o the iso-boundary was also computed. The 3D digital image is convolved with Gaussian unction to be an ininitely dierentiable 3D unction (x, y, z). The mean curvature κ o an iso-boundary point is then computed using Equation (1). [8] x ( yy + zz ) y z yz + y ( xx + zz ) x z xz + z ( xx + yy ) x y xy κ = (1) / ( ) x y where x, y, z are irst order derivatives, xx, yy, zz, xy, yz, xz are second order derivatives. The boundaries are categorized into three classes according to their curvatures: convex, lat, and concave. A threshold value o curvature (C th ) is used or the categorization. Boundaries with curvature greater than C th are concave boundaries, those with curvature smaller than C th are convex boundaries, and the rest are lat boundaries. C th in our experiment was 0. mm -1. Figure b is the curvature map o the iso-boundary on one slice. To explore the relationship between a voxel v and its surrounding tissue-lumen boundary, a bunch o evenly spaced rays are shot rom v in dierent direction d. Unlie sampling the shooting angles in D plane, generating evenly spaced out-shooting rays in 3D space is not trivial. We adopted a spiral-point technique proposed in [9] to generate uniormly distributed points on a sphere and used the spherical coordinates (θ, φ), 0 θ π, 0 φ π, to compute the directions o out-shooting rays, ( 1) h = 1+ ( N 1) θ = arccos( h ) () 3.6 ϕ = ϕ 1 + mod π N(1 h ) d = (sinθ cosϕ,sinθ sinϕ,cosθ ) here is the ray index, 0 N, N is the total number o rays (N=50 in current method), d is the shooting direction o ray. Figure c illustrates the shooting rays in D cases. A shooting ray stops when it hits the iso-boundary or reaches the shooting distance, and a score is assigned according to its hitting situation. The score o a voxel is the summation o the scores o all its shooting rays. The scoring scheme is as ollows: z

4 (a) (b) (d) Figure 3. Fuzzy clustering results (a) Polyp tissue membership map; (b) non-polyp tissue membership map; lumen air membership map; (d) combined color map; (original image is Figure a) N score( v) = E( v, d, m) = 1 (3) 1, hit a convex boundary within distancem 0.5, hit a lat boundary within distancem E( v, d, m) = -1, hit a concaveboundary within distancem 1,nohit within distancem here E is the score o a shooting ray, N is the total number o ray directions, d is the shooting direction, and the shooting distance m is the maximum diameter o a polyp (30 mm in our method). Under this scoring scheme, a voxel will be awarded a high score i it is close to convex boundaries, and will be penalized i it is not. In Figure o the illustration in D, pixel A in the potential polyp region is given a high score since hal o its shooting rays hit convex boundaries; while pixel B in the non-polyp region is given a low score (a negative score in this case) since all its out-shooting rays either hit a concave boundary or hit no boundaries. The intensity o a pixel is then adjusted based on its score, Adjustment ( v) = score( v) r (4) here r is the adjustment rate (r=10 HU in current method). Voxels with positive adjustment are deined as enhanced regions, and the rest are non-enhanced regions. One improvement o the 3D method over our previous D method is that the shooting rays are in 3D directions so that 3D boundary nowledge can be taen into account. Figure shows the 3D nowledge-guided intensity adjustment process (only one D slice is shown). Figure a is the original image, where some polyp regions and some non-polyp regions have very similar intensity values. Figure b is the curvature map o the iso-boundary. Figure c is the illustration o shooting rays rom a pixel in a D case. Figure d is the enhanced image ater the intensity adjustment.. Fuzzy c-means clustering Fuzzy c-means (FCM) clustering method is employed to classiy pixels into several tissue categories. Details o the FCM technique can be ound in [6, 10]. We have deined three tissue classes in the polyp segmentation: lumen air, polyp tissue and non-polyp tissue. Three membership values are computed or each pixel. The initial estimate o the class centroid is derived rom prior nowledge about the CT attenuation in CT colonography. The initial centroid or lumen air class is a small value (e.g. 900 HU). The centroid o polyp tissue class is given as the average intensity o the enhanced region in a 30mm diameter spherical neighborhood o the seed location, and that o non-polyp tissue class is the average intensity o the non-enhanced region in the same neighborhood. The enhanced and non-enhanced regions were deined in the intensity adjustment process in Section.1. Figure 3 shows some results o the uzzy clustering. Figure 3a-3c shows the membership map or lumen air, polyp tissue and non-polyp tissue, respectively. Brighter color in the map indicates higher membership value. The original image can be ound in Figure a. Figure 3d is the combined color map, with polyp tissue in green channel, non-polyp tissue in red channel, and lumen air in blue channel. From the uzzy clustering, we can approximately identiy the polyp region. There is still a lot o noise, and the boundaries between polyp tissues and non-polyp tissues are blurry.

5 Initialization Deormation orce computation Surace deormation Surace maintenance Control parameter update N Finished? Y End Figure 4. Flow chart o deormable surace.3 Dynamic deormable suraces Deormable suraces have been widely used in medical image segmentation [11]. In the segmentation stage, a dynamic deormable surace is applied on the membership unction map to obtain the polyp boundary. Figure 4 shows the low chart o the dynamic deormable surace process. An initial surace is irst placed at the seed location and initial orce weights are set. Then an iterative process is started. During each iteration, deormation orces are computed or surace vertices, and the surace is deormed according to the orces. The surace is also dynamically maintained aterward to reserve the resolution and topology. Then the orce weights and other control parameters are adaptively updated. The iterative process is repeated until all orces reach balance (suraces remain unchanged between iterations) or maximum number o iteration is run. Details on each step will be explained in ollowing sections. The initial surace is a cube o two voxel size in each dimension (**) centered at the seed location. The size and location o the initial surace can also be reined by the centroid and bounding box o the enhanced potential polyp region computed in the enhancement stage. The deormable surace is driven by combination o internal orces, image orces and external orces. Internal orces intend to maintain the smoothness and continuity o the surace. They are computed rom the surace itsel, which can be written as F ( v) = α s( v) β ( s( v)) (5) int ernal v v v where s is the deormable surace, v is vertex on the surace, is the second order derivative operator. The derivatives are computed using Laplacian operator and inite dierence on the triangular surace mesh (mesh data structure will be introduced in section.4). The irst term maes the surace act lie an elastic membrane and eeps it rom breaing. The second term maes it act lie a rigid thin plate and eeps it rom collapsing. By adjusting the weights α and β, one can control the relative importance o the membrane term and the thin-plate term. We used α=1 and β=1 in current method. Image orces are the major orces to attract the surace to polyp boundaries. Since polyp boundaries have larger gradient than other regions, we used the gradient o the edge map o the membership unction as the image orces, i.e. ( G ( G ( ( )) ) σ σ F image ( v) = µ v (6)

6 (a) (b) (d) Figure 5. Image orces (a-c) Image orces on selected D slices; (d) image orces in 3D view where µ(v) is the membership map o polyp tissue class, is the gradient operator, G σ is the Gaussian operator. Gaussian operator is used to smooth the image and increase the capture range. We used σ=0.5 in our method. Since the initial surace is usually placed at the center o a polyp where image orces are not strong enough to pull the model, balloon orces are added to inlate the surace. The balloon orce at a vertex v can be computed as F v v c balloon ( v) = (7) v vc where v c is the centroid o current surace. The use o balloon orces also helps speed up the converging process. The deormation orce is the summation o weighted internal orces, image orces and balloon orces. Each component o the deormation orce is a vector in 3D space, F v) = w F ( v) + w F ( v) w F ( v) (8) ( int ernal int ernal image image + balloon balloon here F(v) is the deormation orce at vertex v, w internal, w image and w balloon are respective orce weights. The deormation orce is also normalized. The weights o orce components are adaptively updated during the iterative process. In the initial phase, balloon orces are dominant orces to inlate the surace. As the surace moves closer to the boundary, the impact o balloon orces diminishes gradually to prevent the surace rom being overblown. Meanwhile, image orces and internal orces are boosted to attract the surace to boundaries and eep the surace smooth. We designed a weight adapting scheme or this purpose, ( t) ( t 1) wballoon = wballoon 0.1 ( t) ( t 1) w = w (9) w int ernal ( t) image = w int ernal ( t 1) image (0) (0) (0) here t is the iteration number. Initially, w = 5, w, and w = 5 in current method. All weights are also balloon int ernal = enorced to be greater than 0. The location o vertex v is updated at each iteration using Equation (10), ( t) ( t 1) ( t 1) v = v + τ F( v ) (10) here τ is the deormation step size. The deormation process will stop when all orces reach a balance or the maximum number o iterations is perormed. Figure 5 illustrates image orces on selected D slices (5a-5c) and a 3D view o the image orces in the sub-volume (5d). The orce direction in the needle graph is displayed as direction rom red color to light blue color. image.4 Dynamic surace maintenance The deormable surace is represented by triangular meshes. The surace data structure stores a list o triangles and a list o vertices. Each vertex stores its 3D coordinate and pointers to triangles sharing this vertex. Each triangle stores pointers to its vertices and pointers to adjacent triangles sharing edges with it. The surace resolution is measured by the distance between adjacent vertices (or the length o edges). It is desirable to maintain the surace resolution in the deormable surace process. I the surace resolution is too high (vertices are too

7 v v 1 v' v 1 v v 1 v v (a) Edge collapsing Figure 6. Dynamic mesh maintenance operations (b) Edge splitting close), too many vertices need to be updated each iteration and the converging process will be very slow. On the other hand, i the resolution is too low (vertices are too sparse), there might not have enough vertices on surace to accurately describe the polyp shape. Furthermore, it is also desirable to have evenly distributed vertices on the surace, i.e. the resolution should be consistent everywhere on the surace. Uneven vertex distribution will cause uneven internal orces and balloon orces, which may result in incorrect surace deormation. We developed edge-based operations to dynamically maintain the surace resolution. The main idea is: 1) i one edge is too short (two adjacent vertices are too close), it will be collapsed into one vertex; ) i one edge is too long (two adjacent vertices are too ar apart), it will be split into two edges by inserting one vertex in the middle. To eep the valid mesh topology, several mesh update operations need to be perormed in the neighborhood o edge collapsing and edge splitting. Figure 6 illustrates the dynamic mesh maintenance operations. In Figure 6a, edge v 1 -v is collapsed into one vertex v and its neighborhood is re-triangulated. In Figure 6b, edge v 1 -v is split into two edges by inserting a vertex v, and two triangles incident to the edge are also split into our triangles. The edge length is maintained within 1 to 4 pixel size in current method..5 Deormable surace results and visualization Figure 7 shows the deormable surace process. The irst row demonstrates the initial state, intermediate state and inal state o the deormable surace. The second row shows one cross section o the surace and corresponding D slice o the image. From the igure, one can see that the surace is gradually converged to the polyp boundary. The segmentation result can be visualized and color painted in dierent ways to demonstrate polyp properties. Figure 8 shows three dierent visualizations o the segmentation result. Figure 8a is the usion o the surace and three orthogonal views o the image, which can help the user to examine the 3D relationship between the polyp and its surrounding region. Figure 8b is the boundary map. Polyps have two types o boundaries: outer boundary (the boundary abut to lumen) and inner boundary (the boundary abut to colon wall). The ability to distinguish inner boundaries and outer boundaries is important or computing eatures such as polyp wall thicness. The boundary map can help visualize the connection between the polyp and the colon wall (Red color indicates the outer boundary, and green color indicates the inner boundary). Figure 8c is the curvature map. The surace is painted according to local curvatures (red: convex, green: concave, and blue: lat). The curvature map can help visualize the terrain o the polyp. Several interesting eatures can also be derived rom these maps, such as the area ratio o inner boundaries and outer boundaries and the area ratio o convex regions and concave regions. 3. RESULTS and DISCUSSION The CT colonography data in our experiment were obtained rom 0 patients (each patient had a supine study and a prone study) [4]. CT scans were done on a GE HiSpeed scanner. The scanning parameters were 10 Vp, 50 mas (mean), ield o view to it was cm, 5 mm collimation, HQ mode, and 3 mm reconstruction interval. The data size was 51*51*N, where N is the number o transverse slices, which is around 300. Based on colonoscopic examination and CT colonography o the same patient, 5 polyps larger than 1cm were identiied. The segmentation process was run on true positive detections to obtain the polyp boundaries. The results in Figure 7 showed that the segmentation visually aligns with the polyp boundaries. In order to quantitatively validate the accuracy o the segmentation, we manually segmented all true positive detections and stored the manual segmentation in a database. The manual segmentation was careully perormed by a trained student and veriied by an experienced radiologist. The manual segmentation was used

8 (a) (b) (d) Figure 7. Deormable surace process (a) Initial state; (b) intermediate state; inal state; (d-) one cross section o the deormable surace (e) () (a) (b) Figure 8. Visualization o segmentation results (a) Orthogonal view; (b) boundary map; curvature map as the gold standard segmentation in our investigation. We validate the accuracy o the computer segmentation results by computing the volume overlap percentage between the manual segmentation and the computer segmentation. The volume overlap percentage is computed as cs I c overlap = c + c s p p 100% (11)

9 (a) (b) (d) Figure 9. Comparison o 3D method and D method (a-b) results o 3D method; (c-d) results o D method here c s is the computer segmentation, and c p is the manual segmentation, I is the set intersect operation, is the number o voxels in a segmentation. Table 1 lists the validation results among 5 polyp detections. The average volume overlap percentage was 80.6%, the standard deviation was 7.8%, the minimum percentage was 64.4%, and the maximum percentage was 90.5%. Table 1. Volume overlap percentage between computer segmentation and manual segmentation among 5 polyp detections Average Std dev Minimum Maximum 80.6% 7.8% 64.3% 90.5% The 3D polyp segmentation method presented in this paper is an improvement over previous D method. In previous D method, D contours were segmented on each slice and propagated to adjacent slices. No smooth constraints were imposed between contours. The major improvement includes 3D nowledge-guided intensity adjustment and 3D dynamic deormable surace. The result o 3D segmentation is a smooth surace instead o a stac o D contours. Figure 9 shows the comparison o the results rom the 3D method and previous D method. It shows both the 3D boundary surace and one D cross section o the surace. The surace rom the D method is constructed by stacing D contours. Although both D method and 3D method produce smooth contours on D slices, the 3D surace rom D method is jittery and the one rom 3D method is much smoother. Smooth suraces have great advantage in computing 3D spatial eatures such as curvatures. Feature extraction and classiier are two essential parts o Computer Aided Detection (CAD) system. Several advanced classiiers have been proposed in the past two decades, including Neural Networ, Support Vector Machine, etc. However, the eectiveness o a classiier depends heavily on the quality o eatures. I true detections and alse detections mingled together in the eature space, no classiier can perorm very well. The 3D polyp segmentation not only produces the boundary surace o the polyp, it also extracts all interior voxels o the polyp. The boundary surace can be used to compute 3D spatial properties o the polyp. And the interior voxels can be employed to analyze the density distribution. Furthermore, the connection points between the polyp and the colon wall (the polyp nec) can also be extracted. All these properties can be combined to characterize polyps and build a eature space that could separate true detections and alse detections. An ongoing investigation is under way to select useul combination o eatures rom the 3D segmentation and build a classiier or computer aided colonic polyp detection. ACKNOWLEDGEMENTS We than Amy Hara, MD, Mayo Clinic Scottsdale Department o Radiology and C. Daniel Johnson, MD, Mayo Clinic Rochester Department o Radiology or providing CT colonography data. We also than Meghan Miller or conducting the manual polyp segmentation.

10 REFERENCES 1. Pichardt, P.J., et al., Computed Tomographic Virtual Colonoscopy to Screen or Colorectal Neoplasia in Asymptomatic Adults. the New England Journal o Medicine, (3): p Summers, R.M., Challenges or Computer-Aided Diagnosis or CT Colonography. Abdominal Radiology, 00: p. in press. 3. Summers, R.M., et al., Colonic Polyps: Complementary Role o Computer-Aided Detection in CT Colonography. Radiology, 00. 5: p Summers, R.M., et al., Automated polyp detection at CT colonography: easibility assessment in a human population. Radiology, (1): p Yoshida, H., et al., Computerized detection o colonic polyps at CT colonography on the basis o volumetric eatures: pilot study. Radiology, 00. (): p Yao, J., et al. Automatic Segmentation o Colonic Polyps in CT Colonography Based on Knowledge-Guided Deormable Models. in SPIE Medical Imaging San Diego. 7. Jerebo, A., M. Franasze, and R. Summers. Radon transorm based polyp segmentation method or CT colonography computer aided diagnosis. in RSNA Thirion, J.-P. and A. Gourdon, Computing the Dierential Characteristics o Isointensity Suraces. Computer Vision and Image Understanding, (): p Sa, E.B. and A.B.J. Kuijlaars, Distributing Many Points on a Sphere. The Mathematical Intelligencer, (1): p Pham, D.L. and J. Prince, An adaptive uzzy c-means algorithm or image segmentation in the presence o intensity inhomogeneities. Pattern Recognition Letters, : p Xu, C., D.L. Pham, and J.L. Prince, Medical Image Segmentation Using Deormable Models, in Handboo o Medical Imaging, Volume. Medical Image Processing and Analysis, M. Sona and J.M. Fitzpatric, Editors. 000, SPIE. p