HepaTux A Semiautomatic Liver Segmentation System

Transcription

1 HepaTux A Semiautomatic Liver Segmentation System Andreas Beck and Volker Aurich Institut für Informatik, Heinrich-Heine-Universität Düsseldorf, D Düsseldorf becka-miccai@acs.uni-duesseldorf.de Abstract. While the MICCAI challenge strives for fully automatic liver segmentation, we present a fast semiautomatic method that is capable of segmenting livers with the help of an experienced human supervisor within 3 to 15 minutes. As fully automatic methods need to be checked for errors by a human observer anyway, we believe that for many practical applications the presented method will be useful. 1 Introduction HepaTux [1] is a framework for liver segmentation we developed using the EC- CET Toolkit [2] developed at our university. It features several methods to segment liver tissue and vessels within the liver. For the purposes of the challenge we implemented a new wizard called NLC NTX for the current development version of HepaTux. ECCET wizards are interactive web applications that guide the user through the steps of the segmentation process by both giving advice on how to proceed and by controlling the running ECCET program to trigger complex actions. ECCET is capable of loading a variety of file formats including ITK metaheader files as used in the challenge as well as DICOM image stacks and it supports loading DICOM data directly from PACS servers. This makes ECCET based applications easy to integrate into clinical workflow. 2 Method 2.1 Workflow After starting the main application and launching the Hepatux wizard, the user is prompted to load the liver dataset to process. A few options are offered for converting unusual datasets. Then three orthogonal planar views of the dataset are presented and the following steps are performed without any further preprocessing.

2 Step 1: Interactive filling The user clicks at any voxel he considers to be liver tissue and a three-dimensional fill algorithm will start to flood-fill the volume from there. The NLC criterion (described below in detail) serves as a stop condition. It forces the algorithm to stop wherever the distribution of the grey values in the neighbourhood differs significantly from that at the seedpoint. By a control parameter which is not very critical the stop condition can be adapted to the particular noise level of the liver tissue. Usually liver tissue and lesions are distinguished by the user and marked with different colors. For the purpose of the challenge lesions were treated as liver tissue. This filling step is repeated by choosing new seedpoints until the liver is segmented in its entirety. The choice of the seedpoints is intuitive and does not require a high level of accuracy since remaining small gaps (e.g. at large vessels and the border of lesions) will be filled by the postprocessing step. The three-dimensional nature of the algorithm allows to fill the liver quickly. In order to check the progress of filling it is dangerous to use only one planar view because misinterpretation of the local shape of the liver easily occurs. This effect can even be seen in some of the provided reference segmentations (see the rectangular artifacts on the first line of figures 1 and 2). Therefore HepaTux offers instant visualization of the filling state in all three planar views resulting in an intuitive and non-tedious procedure. The time needed for the whole filling step depends on the complexity of the situation. Datasets with very inhomogeneous texture of the liver tissue and irregular lesions like the testing example # 3 in the challenge require more time than those with quite homogeneous textures of liver and lesions. Step 2: Touching up the result in the 3D view During the whole segmentation process a 3D view of the current result is available to visualize progress. After the filling step this view is used to identify and correct remaining problematic parts. Typical problem areas are the apex of the heart, small pieces of tissue between the ribs or parts of the stomach touching the liver. Such errors are easily spotted in the 3D view and corrected using various intuitive tools (like the virtual knife for cutting off unwanted pieces) for 3D volume operations. If the 3D view reveals missing parts, one can easily go back to the fill step. For the purposes of the challenge, the vessels leading to the liver were also cut away using this tool to match the standards set by the training datasets. Note that we did not emulate the training datasets property of including internal vessels 1. Including vessels dependent on a specific view does not seem to be anatomically justified but rather inspired by the reference segmentation methodology. We have run postprocessing filters on our datasets to emulate this property, and improvements are minimal (about 0.1% higher Dice similarity or lower error). The influence on distance based scoring systems would probably be higher, though. 1 From the comments to the reference segmentation: In general, a vessel counts as internal if it is completely surrounded by liver tissue (in the transversal view).

3 Step 3: Postprocessing After the 3D touch up, a fully automated postprocessing step follows. Its purpose is the elimination of the gaps left after the filling step and the smoothing of the 3D shape of the surface of the segmented object. Both is achieved by creating a limited convex hull: The segmentation result is extended by all straight lines with endpoints on border voxels which lie within a fixed size bounding box. The whole postprocessing step takes about one minute to compute. 2.2 Employed algorithms The NLC criterion The breadth first search of the interactive filling step is controlled by a criterion which we call non-linear coupling (NLC). Its definition is similar to that of the non-linear Gaussian filters in [3][4][5] which are used for edge-preserving smoothing. Let f(q) be the grey value of the voxel q and σ and ζ be positive numbers. Then the non-linear coupling to a level L in a voxel p is defined as NLC (σ,ζ,l) f(p) = g σ ( q p ) g ζ ( f(q) L ) q Here g λ (t) = exp( t2 2λ ) is the non-normalized Gaussian function with standard 2 deviation λ > 0, and the sum extends to all voxels q in a rectangular neighbourhood of p with edge length of approximately 6σ. In the above-mentioned filling step the level L is chosen as the linear Gaussian mean of the grey values in the neigbourhood of the selected seedpoint. During the breadth first search the NLC value to this level L is calculated for each new voxel. Only if this value is sufficiently high the voxel is included in the segmented volume and in the search queue for further expansion. The parameter σ is fixed; for usual CT data σ = 2 is well-suited (assuming the distance of adjacent voxels is 1). The parameter ζ is adapted to the liver texture. A good value is the estimated standard deviation σ grey of the grey value distribution of the tissue to be filled. A tool is included to help getting an estimate of σ grey if desired. The choice of this parameter is not particularly critical, so keeping it at the default value will work for most datasets. When chosen too low the filling algorithm stops very soon and has to be restarted more often. To facilitate adding areas with irregular, not very homogeneous texture it is advantageous to start the filling step with a larger ζ assuring the filling of the immediate vicinitiy of the clicked seedpoint and then gradually lowering ζ while progressing in the breadth first search. A good strategy is to choose as initial value ζ 0 = 3σ grey and then to update the current value ζ n according to ζ n = 0.9 ζ n σ grey after each iteration over the breatdh first search queue (i.e. after each layer added). Despite the simplicity of the criterion it has turned out in many experiments to be a reliable condition to stop the filling at texture boundaries. The virtual knife Interaction with 3D objects for the purpose of sculpting is known to be difficult as only few operations are intuitive and efficient. One such operation we employ is a virtual knife. When clicking at a point of the object in the 3D view the object is cut along the vertical plane through the clicked point

4 in viewing direction. This is achieved by flood-filling within a small vicinity of the cut-plane that is defined by the clicked voxel, the camera location and the camera up -vector. At the clicked point a breadth first search is started that will terminate when it deviates too much from this cut-plane. The method provides intuitive behaviour much like cutting with a real knife: It will only go through the first object seen, leaving objects behind untouched. 3 Results 3.1 Performance on the training datasets File Time Vol. Over- Dice Sim. Difference number (in s) lap Error Coefficient Reference HepaTux % 95.5% 5.8% 3.8% % 95.4% 6.0% 3.6% % 94.8% 4.9% 6.0% % 97.3% 2.5% 3.1% % 96.9% 4.0% 2.3% % 96.4% 3.0% 4.5% % 96.8% 3.1% 3.5% % 96.3% 1.9% 5.7% % 96.8% 3.6% 2.9% % 97.2% 3.2% 2.6% % 97.1% 3.3% 2.7% % 97.2% 1.8% 3.9% % 94.2% 3.7% 8.7% % 96.4% 3.2% 4.3% % 97.0% 3.9% 2.2% % 97.0% 2.5% 3.8% % 96.1% 5.0% 3.0% % 96.2% 6.1% 1.8% % 97.0% 4.1% 2.2% % 95.3% 6.6% 3.2% Average: % 96.4% 3.9% 3.7% Worst case: 11.0% 94.2% 6.6% 8.7% Table 1. Performance of HepaTux on the challenge training datasets Table 1 shows the performance of the method on the training datasets made available for the challenge. In this and all other tables, the figures were derived as follows: If A and B are the voxel sets obtained by two segmentations then Volumetric Overlap is calculated as the ratio #(A B) #(A B) and Volumetric Overlap Error is the difference of that value to 1. 2#(A B) Dice Similarity Coefficient is the ratio #(A B)+#(A B). Difference is the ratio #(A\B) #(B\A) #(A B) or #(A B) respectively, given to provide an idea about the symmetry of the errors. If e.g. a volume is entirely contained in the other, its difference value is 0. Segmentation time varied between about 3 and 15 minutes, with an average of 7 minutes. As this already includes the time taken for error-checking and correction of results, we think this method is feasible for clinical use.

5 File Vol. Over- Dice Sim. Difference number lap Error Coefficient 1st 2nd 1 3.6% 98.2% 0.9% 2.8% 2 5.5% 97.2% 4.1% 1.7% 3 6.0% 96.9% 4.4% 2.0% 4 4.1% 97.9% 2.9% 1.3% 5 3.6% 98.2% 1.1% 2.6% 6 3.3% 98.3% 1.4% 2.0% 7 3.0% 98.5% 2.0% 1.1% 8 4.6% 97.7% 4.3% 0.5% 9 4.3% 97.8% 1.0% 3.5% % 97.6% 4.0% 0.9% % 97.6% 1.7% 3.1% % 98.2% 1.7% 1.9% % 97.5% 2.6% 2.5% % 97.4% 3.6% 1.7% % 98.1% 1.9% 1.9% % 96.7% 5.7% 1.0% % 97.3% 3.3% 2.3% % 97.3% 1.5% 4.1% % 97.8% 2.1% 2.3% % 96.3% 5.3% 2.4% Average: 4.6% 97.6% 2.8% 2.1% Worst case: 7.1% 96.3% 5.7% 4.1% Table 2. Intra-Observer variance (training datasets) Intra-/Inter-observer-variance Semiautomatic methods inherently depend on user input. However, for clinical practice it is important to get reliable results independent of the variations in inputs provided by the same user and even more importantly by other users. Therefore, the training measurements were repeated by the same observer (A. Beck) and a second observer (V. Aurich) to estimate the degree of intra- and inter-oberserver-variance. Results of these measurements are shown in Tables 2 and 3. File Vol. Over- Dice Sim. Difference number lap Error Coefficient 1st 2nd 1 5.9% 96.9% 1.9% 4.4% 2 7.5% 96.1% 3.6% 4.5% 3 7.2% 96.3% 4.9% 2.8% 4 4.8% 97.6% 4.1% 0.9% 6 5.0% 97.5% 2.2% 3.0% 7 5.8% 97.0% 5.5% 0.6% 8 4.0% 98.0% 3.0% 1.2% % 97.7% 1.1% 3.7% % 96.0% 2.6% 5.7% % 95.8% 6.3% 2.6% Average: 6.1% 96.9% 3.5% 2.9% Worst case: 8.1% 95.8% 6.3% 5.7% Table 3. Inter-Observer variance (training datasets)

6 3.2 Performance on the test datasets Internal tests The challenge test datasets were also segmented twice to acquire an intra-observer confidence measure. Segmentation time varied between about 4 and 15 minutes, with an average of 7.5 minutes. Table 4 gives the times taken to segment each liver and the intra-observer variance. File Time/s Vol. Over- Dice Sim. Difference number 1 2 lap Error Coefficient % 98,0% 2,7% 1,4% % 97,7% 2,6% 2,0% % 97,3% 2,1% 3,5% % 96,2% 2,2% 5,7% % 96,3% 2,8% 4,8% % 97,3% 1,7% 3,9% % 97,4% 0,7% 4,7% % 97,2% 2,1% 3,6% % 97,9% 2,2% 2,1% % 95,8% 2,1% 6,7% Average: % 97,1% 2,1% 3,8% Worst case: % 95,8% 2,8% 6,7% Table 4. Intra-Observer variance (test datasets) Challenge evaluation Both segmentation series were submitted for evaluation. The results are shown in tables 5 and 6 and figures 1 and 2. 4 Discussion 4.1 Strengths and Weaknesses As readily seen from the middle columns of figures 1 and 2, our method is less prone to creating rectangular-looking artifacts than manual segmentation using plane-based methods. In areas where delineation of the target tissue is not very pronounced (like in areas where the used plane is almost tangential to the target volume) or where exemption of parts would require extra work (like exempting the internal vessels), this can lead to very unnatural-looking results when viewed in another plane. However, the algorithm does not try to create smooth borders, except for a local smoothing step at the end. This yields visually less appealing results near borders where the grey values do not present a clear edge. This can be seen most readily in the bottom row, where a large, structured lesion impedes segmentation. As stated in the results, the authors were also the observers. We thus assume that the required time for an observer that uses the tool only occasionally might

7 Dataset Overlap Error Volume Diff. Avg. Dist. RMS Dist. Max. Dist. Total [%] Score [%] Score [mm] Score [mm] Score [mm] Score Score Average Table 5. Segmentation 1: Results of the comparison metrics and scores for all ten test cases. Fig. 1. Segmentation 1: From left to right, a sagittal, coronal and transversal slice from a relatively easy case (1, top), an average case (4, middle), and a relatively difficult case (3, bottom). The outline of the reference standard segmentation is in red, the outline of the segmentation of the method described in this paper is in blue. Slices are displayed with a window of 400 and a level of 70.

8 Dataset Overlap Error Volume Diff. Avg. Dist. RMS Dist. Max. Dist. Total [%] Score [%] Score [mm] Score [mm] Score [mm] Score Score Average Table 6. Segmentation 2: Results of the comparison metrics and scores for all ten test cases. Fig. 2. Segmentation 2: From left to right, a sagittal, coronal and transversal slice from a relatively easy case (1, top), an average case (4, middle), and a relatively difficult case (3, bottom). The outline of the reference standard segmentation is in red, the outline of the segmentation of the method described in this paper is in blue. Slices are displayed with a window of 400 and a level of 70.

9 be higher. However, there are no complex decisions to be made that require background knowledge of image processing in general or the system in particular. We thus do not expect that usage by a non-expert would cause large timing impacts. We would expect accuracy to increase when the system is used by an expert of the anatomy side. However, the testing results score our segmentation about equal to the student performing the second reference segmentation. We are currently planning to put this version of the software into clinical testing to see how it performs in the hands of radiologists that are not yet proficient with ECCET applications. Some preliminary tests with a radiologist that has been using other applications of the ECCET toolkit showed good acceptance and good segmentation results at acceptable time after a supervised first segmentation. 4.2 Overall Score The scoring system was set up to give a score of 75 for a second human observer. We are thus very pleased to see our system perform at that value for both submitted results on average and no total score going below 66. These results seem acceptable, especially as the segmentation was done by a computer scientist with only rudimentary anatomical knowledge. Due to our ignorance of the internal vessel rule, we were expecting higher deviations in the distance scores, especially the Maximum Distance Score. Looking at absolute values, we see those expected high deviations but they seem to be comparable to those arising from intra-observer differences used to calibrate the scoring system, which is surprising. Comparing the overlap errors we retrieved from our intra-observer testing with those provided by the challenge evaluation, we find the errors to be slightly (1.3% average, 3.2% maximum) higher. This matches our finding from the training datasets, that the method has an about 1.5% higher overlap error on the inter-observer case. References Aurich V., Weule J.: Non-Linear Gaussian Filters Performing Edge Preserving Diffusion. Proceedings 17. DAGM Symposium über Musterekennung, Springer , Aurich V., Mühlhaus E., Grundmann S.: Kantenerhaltende Glättung von Volumendaten bei sehr geringem Signal-Rausch-Verhältnis. Proceedings Bildverarbeitung fr die Medizin 1998, Springer. 5. G. Winkler, K. Hahn, A. Martin, K. Rodenacker, V. Aurich: Noise Reduction in Images: Some Recent Edge-Preserving Methods. Pattern Recognition and Image Analysis, Vol.9, No. 4, 1999,