Automatic Segmentation of the Lung Fields in Portable Chest Radiographs based on Bézier Interpolation of Salient Control Points

Transcription

1 IEEE International Workshop on Imaging Systems and Techniques IST 2008 Chania, Greece, September 10 12, 2008 Automatic Segmentation of the Lung Fields in Portable Chest Radiographs based on Bézier Interpolation of Salient Control Points Dimitris K. Iakovidis Dept. of Informatics and Computer Technology Technological Educational Institute of Lamia Lamia, Greece George Papamichalis 1 st Dept. of Thoracic Surgery Chest Hospital of Athens Sotiria Athens, Greece Abstract Portable chest radiographs are performed on the most critically-ill patients, who need highly skilled patient care and the best diagnostic tools. However, such radiographs are usually of low quality, mainly due to misaligned body positioning during their acquisition, and subject to a higher misinterpretation rate than the ones obtained with a non-portable x-ray device. This paper presents a pioneering computational approach that copes with the segmentation of the lung fields in portable chest radiographs. The proposed methodology involves detection of salient points on the anatomic structures around the lung fields by subsequent application of simple intensity and edge feature extraction techniques. The salient points detected are interpolated using Bézier curves intuitively approximating the boundaries of the lung fields. Unlike current methodologies, the proposed one does not exclude the overlapping region of the heart from the lung fields, where abnormalities can also be present. The results illustrate the robustness of the proposed methodology on a set of real portable radiographs of patients with bacterial pulmonary infections. A qualitative comparison with a state of the art approach based on graph cuts validates its effectiveness. Keywords-image segmentation; portable chest radiography; salient points, Bézier curves; lung; infections I. INTRODUCTION Radiography is a valuable tool to chest screening providing useful information for the prevention and diagnosis of various diseases [1]. Plain chest radiographs are usually obtained in a controlled setup where the patients are positioned in a standard way at the x-ray device. However, in the case of critically ill patients, this is not always feasible as they may be in pain or disabled. To cope with this problem, portable x-ray devices are commonly used to obtain the radiographs from various relative distances and angles to the patient who is immobilized in bed - not necessarily in the preferred position. Portable radiographs are usually of low quality mainly because of misaligned body positioning during their acquisition [2]. Consequently, they are more difficult to read and subject to a higher misinterpretation rate. A methodology that would automatically analyze such radiographs would be an asset to the medical community. A variety of image processing and analysis methodologies has been proposed for the segmentation of plain chest radiographs. Many of them have focused on the segmentation of the lung fields, whereas fewer have focused on the segmentation of the rib cage or other anatomic structures of the chest [3]. The importance of the segmentation of the lung fields becomes obvious considering that it is a prerequisite for almost any computational radiographic image analysis task. Lung field segmentation approaches that have been proposed include rulebased methodologies [5]-[10], machine learning algorithms [11]-[13], active shape models [4][14]-[17], and graph cuts[18]. The studies [5]-[10] indicate that the segmentation performance of the existing rule-based methodologies usually degrades if the quality of the radiograph is poor or the positioning of the patient deviates from the standard. Machine learning algorithms and active shape models can be less affected by a patient s positioning; however, they both require training, which makes them dependent on the subjective choice of representative training samples. The methodologies reported in [14],[16],[17] have been mainly evaluated on radiographs of normal or minimally distorted lungs. Robustness against the presence of abnormalities that interfere with the interior intensities of the lungs has been demonstrated in [4],[15] using active shape models. The graph cuts algorithm proposed in [18] is the most recent of the limited approaches coping with unsupervised segmentation of abnormal lung fields. It is based on morphological operations for obtaining an initial estimate of the region of the lung boundaries, and dilates the contour obtained by graph cuts within a potential contour band. The evaluation of this algorithm on a limited set of standard radiographs of patients infected with severe acute respiratory syndrome (SARS) showed that although it exhibits some resistance to boundary discontinuities and noise, the detected boundaries of the lung fields are affected by the edges of the ribs and result in a rough output contour. To the best of our knowledge, the problem of lung field segmentation in portable chest radiographs has not been previously addressed. In this paper we accept the challenge of coping with automatic segmentation of the lung fields in portable chest radiographs, and we propose a novel methodology that is resistant to patient positioning, to boundary discontinuities, and to the presence of abnormalities interfering with the interior intensity of the lungs. This work was supported in part by the European Commission s Seventh Framework Information Society Technologies (IST) Programme, Unit ICT for Health, project DEBUGIT (no ) /08/$ IEEE 82

2 The proposed methodology involves detection of salient points on the anatomic structures around the lung fields by subsequent application of simple intensity and edge feature extraction techniques. The salient points detected, are interpolated using Bézier curves [19] smoothly approximating the boundaries of the lung fields. The rest of this paper consists of three sections. Section II presents the proposed lung field segmentation approach. Its performance is evaluated in Section III, and a summary of conclusions derived from this study is provided in the last section. Internal control points P 1 P 2 Piecewise linear interpolation Bézier interpolation P 3 II. PROPOSED APPROACH A. Overview The proposed methodology assumes that: (1) the patient s body may not necessarily be aligned with the portable x-ray device, since the patient may be immobilized in bed; (2) the spine lies roughly somewhere in the middle of the radiograph, considering that the radiograph displays both the patient s lungs; (3) the spine and the bones of the ribcage are the densest structures visible in a chest radiograph [20], and consequently they are generally characterized by higher intensities than other anatomic structures. Based on these assumptions, points belonging to the densest structures bounding the lung fields are detected. As indicated in [10] the local maxima of the horizontal and vertical image profiles - i.e. signals produced by averaging image intensities across columns and rows, respectively - can be useful for the detection of such points. However, in the case of portable radiographs the local maxima corresponding to the structures bounding the lung fields are not located in particular positions as in the case of the stationary radiographs used in [10]. To cope with this problem, the proposed approach involves additional steps (Subsection II.C: Steps 2 and 3) for selecting local maxima of the horizontal profiles corresponding to the spine and to the outer left and right boundaries of the lung fields. Local maxima with intensities deviating from the average intensities of each boundary are discarded as outliers. Considering that the region of the spine is typically characterized by the highest intensities in a radiograph, the inner lung field boundaries will be more or less identifiable by their edges. These edges will be less prevalent in regions where the lung fields exhibit higher intensities such as the region of the heart, and regions with abnormalities appearing as bright shadows next to the inner lung field boundaries. The determination of these edges becomes even more difficult if the airing of the lungs is poor during the acquisition of the radiograph, which is usually the case in portable radiography as practiced in intensive care units. However, even if the inner lung field edges are prevalent they could easily get confused with the edges of the ribs within the lung fields. In order to minimize the likelihood of such confusions, the intensities of the anatomic structures lying in the region of the spine are set to zero via a novel selective thresholding algorithm. This algorithm results in an auxiliary image referred to as thresholded (Subsection II.C: Step 4) which is utilized in the subsequent steps of the proposed methodology. The select- P 0 Figure 1. Bézier vs piecewise linear interpolation of four points. ive thresholding algorithm identifies the intensities of the anatomic structures lying in the region of the spine by accumuaccumulating the most frequent intensities of samples acquired from the detected spine points. Using only the most frequent intensities from each sample, the possibility of discarding intensities belonging to the lung fields is minimized. Moreover, all the intensities that are higher than those belonging to the region of the spine are also set to zero, since they are unlikely to belong to the lung fields. It is worth noting that the resulting thresholded image will have zero intensities not only between the lung fields, but also in other regions of similar densities, including the diaphragms which define the lower boundaries of the lung fields. The inner lung field boundaries are then determined by the maximum edge points between the spine and the lung fields (Subsection II.C: Step 5). The edges are obtained via convolution with directional Sobel edge detectors [21]. This way, unlike current approaches [3]-[13],[15]-[18], the proposed one does not exclude the overlapping region of the heart from the lung fields, where abnormalities can also be present [1]. A similar approach is followed for the detection of the lower lung field boundaries (Subsection II.C: Step 6). In both cases the resulting points are plenty but include many outliers, most of which can be discarded by using a k nearest neighbor approach. The remaining salient points from the outer, the inner and the lower boundaries of each lung field are smoothly interpolated using Bézier curves. The subsequent curves are finally manipulated so that they form continuous closed curves segmenting the lung fields. B. Bézier interpolation Given a set of control points P 0, P 1,,P n, a Bézier curve is estimated according to the equation where t [0,1]. B n n n i i ( t) = Pi (1 t) t (1) i= 0 i Bézier curves are intuitively controlled by the detected salient points, in the sense that the curve is attracted by the internal control points, without necessarily passing through them (Fig. 1). This provides several advantages over other 83

3 interpolation methods that include insensitivity to outliers, and production of smoother curves. C. Algorithmic implementation The salient control points of a radiograph I of N M pixels, are detected and interpolated as follows (Fig. 2): Step 1. Detect local maxima of horizontal profiles. a) Acquire s h consecutive non-overlapping rectangular samples of h M, h<n pixels from the whole image. b) For each sample calculate its average horizontal profile. c) Detect the local maxima of each profile (white points in Fig.2b). Step 2. Detect the central, the left and the right columns (indicated in black in Fig. 2b) a) Acquire s v non-overlapping rectangular samples of N w, w<m pixels from the whole image. b) For each sample calculate the average intensity. c) The central column C is the one with the maximum average intensity within columns M/2-M/4 and M/2+M/4. d) The left column L is the one with the maximum average intensity within columns 0 and M/2-M/4. e) The right column R is the one with the maximum average intensity within columns M/2+M/4 and M-1. Step 3. Approximate the spine, and the outer left and right boundaries of the left and the right lung field respectively. a) For each of the s h samples - The points closest to C are considered as spine points. - The points closest to the left side of column L+(C- L)/2 (left white column in Fig. 2b) are considered to belong to the outer left boundary of the left lung field. - The points closest to the right side of column R+(R-C)/2 (right white column in Fig. 2b) are considered to belong to the outer right boundary of the right lung field. b) For each set of detected points discard outliers. c) Interpolate the remaining points of each set (Fig. 2c). Step 4. Threshold all the intensities of I corresponding to anatomic structures lying in the region of the spine. a) For each spine point acquire a square sample of x 2 pixels. b) Calculate the intensity histogram of each sample. c) Sum up the largest histogram components of all samples into a single histogram. d) Find the last non-zero component m of the resulting histogram. e) Set the intensities that correspond to non-zero histogram components, and the intensities that are larger than m, to zero, to obtain a thresholded image T(I) (Fig. 2d). (a) (c) (e) (g) (i) (j) Figure 2. Steps of the proposed methodology for the segmentation of a portable chest radiograph. Step 5. Detect edge points of the inner right and left boundaries of the left and the right lung field respectively. a) Convolve I with the vertical 3 3 Sobel operator having the negative signs on the left to obtain a map E vl (I) of vertical left image edges (Fig. 2e). b) Convolve I with the vertical 3 3 Sobel operator having the negative signs on the right to obtain a map E vr (I) of vertical right image edges (Fig. 2f). c) For each row start from the spine point and move left to find the point of E vl (I) with the maximum intensity for which T(I) has zero intensity. (d) (f) (h) 84

4 d) For each row start from the spine point and move right to find the point of E vr (I) with the maximum intensity for which T(I) has zero intensity. e) Discard points deviating from the average horizontal position of its k nearest neighbors as outliers. f) Interpolate the detected points (Fig 2g). Step 6. Detect edge points of the lower boundaries of the lung fields. a) Convolve I with the horizontal 3 3 Sobel operator to obtain a map E h (I) of horizontal image edges (Fig. 2h). b) For each column start from row M-1 and move upwards to find the point of E h (I) with the maximum intensity for which T(I) has zero intensity. c) Discard points deviating from the average vertical position of its k nearest neighbors as outliers. d) Interpolate the detected points (Fig 2i). Finally, the interpolating Bézier curves estimated in steps 3c, 5f and 6d, are cropped or extended so that they form continuous closed curves segmenting the lung fields (Fig.2j). This algorithm was developed in Java using a custom image processing and analysis framework providing crossplatform compatibility. III. RESULTS The effectiveness of the proposed methodology was evaluated on a set of twenty four anonymous radiographs obtained with a portable x-ray device from patients with pulmonary bacterial infections hospitalized in an intensive care unit of the Chest Hospital of Athens Sotiria. Such infections produce abnormalities most usually manifested as consolidations that are visible as bright shadows interfering with the interior intensities of the lungs [1],[2]. The segmentation performance was assessed in terms of accuracy, sensitivity and specificity, which are defined by the following equations: TP + TN accuracy = (2) TP + TN + FP + FN TP sensitivity = (3) TP + FN TN specificity = (4) TN + FP where TP is the number of true positive pixels, TN is the number of true negative pixels, FP is the number of false positive pixels, and FN is the number of false negative pixels. The ground truth region of the lung fields has been designated by an expert. All radiographs used in the experiments have been digitized at 8 bits and have been downscaled to fit a pixel bounding box. The parameters used in the experimental study presented in this paper include w=h, x=32, and k=5. Figure 3 illustrates the results obtained with the proposed methodology using different values of h. As indicated by the overlapping Accuracy Sensitivity Specificity 100% 95% 90% 85% 80% 75% 70% 100% 95% 90% 85% 80% 75% 70% 100% 95% 90% 85% 80% 75% 70% (a) h (pixels) h (pixels) h (pixels) (c) Figure 3. Results obtained with the proposed methodology for a range of values h, in terms of average (a) accuracy, sensitivity, and (c) specificity. The error bars depict the corresponding standard deviations. error bars, the average segmentation performance is not affected significantly by the selection of h. Best segmentations were obtained for h=9 and h=17. The sensitivity and the specificity in the former case reached 95.3% and 94.3%, and in the latter 95.0% and 93.6%. Indicative segmentations are illustrated in Fig. 2j and Fig. 4. In both cases the lung fields appear misaligned; whereas the boundaries of the right lung 85

5 (a) Figure 4 Portable radiograph of a patient with bacterial pulmonary infection. (a) Original. Segmented using the proposed methodology. it achieves smoother delineation of the lung field boundaries than the state of the art methodology based on graph cuts [18]; unlike current methodologies, it does not exclude the overlapping region of the heart from the lung fields, where abnormalities can also be present. Future objectives include enhancement of the proposed methodology with shape prior information, further experimentation with larger datasets and comparisons with other state of the art methodologies. The perspectives of this research extend to the development of a multimodal data mining system for adverse events detection, which will be capable of co-evaluating radiographic findings of patients with bacterial infections [22]. (a) Figure 5. Segmentation of radiographs obtained from patients infected with SARS, using the proposed methodology. The original images have been obtained from [18], where segmentation results with various methodologies are also provided. field in Fig. 4 are hardly distinguishable even by an expert. As noted in the introduction, current lung field segmentation methodologies do not cope explicitly with the segmentation of portable chest radiographs. However, the state of the art methodology presented in [18], which has been proposed for lung field segmentation in standard radiographs, could be considered as the most relevant methodology to compare with, since it is also unsupervised and it copes with infected lung fields. The results from the application of the proposed methodology on the images presented in [18] are illustrated in Fig. 5. The obtained segmentations are less noisy than the ones presented in [18], illustrate higher sensitivity, whereas the region of the heart overlapping the lung fields has not been excluded, enabling the detection of possible abnormalities even behind the heart. IV. CONCLUSIONS We presented a novel methodology for automatic segmentation of portable chest radiographs. It is capable of detecting salient points which are intuitively interpolated via Bézier curves. The following major conclusions can be derived from the experimental evaluation of the proposed methodology: it produces accurate segmentation of portable radiographs regardless of the patients positioning; it generally remains unaffected by the presence of abnormalities originating from bacterial pulmonary infections; REFERENCES [1] N.L. Müller, T. Franquet, K.S. Lee, C. Isabela, and S. Silva, Imaging of Pulmonary Infections, Lippincott Williams & Wilkins, 2006 [2] F.S. Bongard, and D.Y. Sue, Current Critical Care Diagnosis & Treatment, 2nd ed., McGraw-Hill/Appleton & Lange, 2002 [3] B.V. Ginneken, B.T.H. Romeny, and M.A. Viergever, Computer-Aided Diagnosis in Chest Radiography: A Survey, IEEE Trans. Medical Imaging, vol. 20, no. 12, pp , Dec [4] B.V. Ginneken, S. Katsuragawa, B.T.H. Romeny, K. Doi, and M.A. Viergever, Automatic Detection of Abnormalities in Chest Radiographs Using Local Texture Analysis, IEEE Trans. Medical Imaging, vol. 21, no. 2, pp , Feb [5] P.H. Meyers, H.C. Nice, C.M. Becker, N.J. Nettleton, J.W. Sweeney, and G.R. Meckstroht, Automated Computer Analysis of Radiolographic Images, Radiology, vol. 13, pp , [6] J. Duryea, and J. Boone, A Fully Automatic Algorithm for the Segmentation of Lung Fields in Digital Chest Radiographic Images, Medical Physics, vol. 22, no. 2, pp , [7] H. Luo, R. Gaborski, R. Acharya, Automatic Segmentation of Lung Regions in Chest Radiographs: A Model Guided Approach, in Proc. Int. Conf. Image Processing (ICIP 2000), vol. 2, 2000, pp [8] B.V. Ginneken, and B.T.H. Romeny, Automatic Segmentation of Lung Fields in Chest Radiographs, Medical Physics, vol. 27, no. 10, pp , [9] I. Iglesias, P.G. Tahoces, M. Souto, A. Martinez de Alegria, M.J. Lado, and J.J. Vidal, Lung Segmentation on Postero-anterior Digital Chest Radiographs Using Active Contours, Fred A. et al. (Eds.): SSPR&SPR 2004, Lecture Notes in Computer Science, vol. 3138, pp , [10] X.-W. Xu, and K. Doi, Image Feature Analysis for Computer Aided Diagnosis: Accurate Determination of the Ribcage Boundary in Chest Radiographs, Medical Physics, vol. 22, no. 5, pp , [11] G. Coppini, S. Diciotti, M. Falchini, N. Villari, and G. Valli, Neural Networks for Computer-Aided Diagnosis: Detection of Lung Nodules in Chest Radiograms, IEEE Trans. on Information Technology in Biomedicine, vol. 7, no. 4, pp , Apr [12] N. Vittitoe, R. Vargas-Voracek, and C. Floyd Jr, Markov Random Field Modeling in Posteroanterior Chest Radiograph Segmentation, Medical Physics, vol 26, no. 8, pp , [13] B.V. Ginneken, M.B. Stegmann, M. Loog, Segmentation of Anatomical Structures in Chest Radiographs using Supervised Methods: A Comparative Study on a Public Database, Medical Image Analysis, vol. 10, pp , [14] T. Cootes, C. Taylor, D. Cooper, and J. Graham, Active Shape Models - Their Training and Application, Computer Vision and Image Understanding, vol 61, no. 1, pp , [15] X. Xie, X. Li, S. Wan, and Y. Gong, Mining X-Ray Images of SARS Patients, in G.J. Williams and S.J. Simoff (Eds.) Data Mining, Lecture Notes in Artificial Intelligence, vol. 3755, pp , [16] J. Liang, T. McInerney, and D. Terzopoulos, United Snakes. Medical Image Analysis, vol. 10, pp ,

6 [17] T. Yu, J. Luo, N. Ahuja, Shape Regularized Active Contour using Iterative Global Search and Local Optimization, in Proc. Computer Vision and Pattern Recognition Conference (CVPR 2005), vol.2, pp , [18] S. Chen, L. Cao, J. Liu, X. Tang, Automatic Segmentation of Lung Fields from Radiographic Images of SARS Patients Using a New Graph Cuts Algorithm, in Proc. Int. Conf. Pattern Recognition (ICPR 2006), vol. 1, pp , [19] R.H. Bartels, J.C. Beatty, and B.A. Barsky, Bézier Curves. Ch. 10 in An Introduction to Splines for Use in Computer Graphics and Geometric Modelling. San Francisco, CA: Morgan Kaufmann, pp , [20] Novelline, R.A. (1997) Squires s Fundamentals of Radiology. Cambridge: Harvard University Press. [21] R.C. Gonzalez, and R.E. Woods, Digital Image Processing, 3rd ed., Prentice Hall, Upper Saddle River, NJ, [22] C. Lovis, D. Colaert, V.N. Stroetmann, DebugIT for Patient Safety - Improving the Treatment with Antibiotics through Multimedia Data Mining of Heterogeneous Clinical Data, Stud Health Technol. Inform., vol. 136, ,