Fast and effective characterization of 3D Region of Interest in medical image data

Transcription

1 Fast and effective characterization of 3D Region of Interest in medical image data Despina Kontos, Vasileios Megalooikonomou Department of Computer and Information Sciences, Temple University, 1805 N. Broad St., Philadelphia, PA 19122, USA ABSTRACT We propose a framework for detecting, characterizing and classifying spatial Regions of Interest (ROIs) in medical images, such as tumors and lesions in MRI or activation regions in fmri. A necessary step prior to classification is efficient extraction of discriminative features. For this purpose, we apply a characterization technique especially designed for spatial ROIs. The main idea of this technique is to extract a k-dimensional feature vector using concentric spheres in 3D (or circles in 2D) radiating out of the ROI s center of mass. These vectors form characterization signatures that can be used to represent the initial ROIs. We focus on classifying fmri ROIs obtained from a study that explores neuroanatomical correlates of semantic processing in Alzheimer s disease (AD). We detect a ROI highly associated with AD and apply the feature extraction technique with different experimental settings. We seek to distinguish control from patient samples. We study how classification can be performed using the extracted signatures as well as how different experimental parameters affect classification accuracy. The obtained classification accuracy ranged from 82% to 87% (based on the selected ROI) suggesting that the proposed classification framework can be potentially useful in supporting medical decision-making. Keywords: Regions of Interest (ROI), classification, feature extraction, characterization, diagnosis, medical decision making 1. INTRODUCTION Recent technological advances have made it possible to acquire medical information captured in the form of modern imaging modalities. These data can be utilized in order to assist diagnosis and help to further explore the nature and functionality of the human body. Large repositories are being created worldwide by several institutions for the purpose of storing and analyzing these data. Some of the most popular medical imaging modalities used in our times are MRI *, PET, and fmri that capture structural and/or functional/physiological information. Techniques initially introduced in statistics and electronic imaging are being extended in order to process these medical data and develop efficient medical informatics tools. A very interesting area in medical imaging that has drawn the attention of several researchers is brain imaging. The Human Brain Project 1 has identified as goals to trace the roots of diseases and detect relationships between human brain structures and brain functions (i.e., human brain mapping). Diseases such as Alzheimer s disease (AD) and Schizophrenia have been the focus of many studies 2-3 utilizing brain imaging data. As mentioned earlier, the techniques employed in these studies have been initially introduced in other scientific areas. Electronic imaging, statistics, pattern recognition and data mining are some of the disciplines that have provided methods suitable for medical data analysis. In several cases though, the nature of medical imaging data requires different treatment than other data encountered in image processing and data mining. For example, most of the traditional imaging techniques seek to interpret information expressed in the image by constructing features that refer to * Magnetic Resonance Imaging: shows soft-tissue structural information. Positron Emission Tomography: shows physiological activity. Functional-Magnetic Resonance Imaging: shows physiological activity.

2 (a) (b) (c) Fig. 1. Examples of ROIs in medical images: (a) lesion, (b) tumor, and (c) activation areas in fmri. the entire image content 4-5. However, analysis of medical images requires focusing on specific Regions of Interest (ROI s), such as tumors, lesions and brain activation regions, in order to analyze critical information 6. It is often necessary to focus in such ROIs in order to mine associations between either structures and functions or lesions and neurological deficits in the human brain 7. In this paper we propose a framework for detecting, characterizing and classifying ROIs in medical images. Since feature extraction is a very important step prior to classification, we employ a quantitative characterization technique initially introduced for spatial ROIs in the image processing community 8. This technique effectively extracts k- dimensional feature vectors using concentric spheres in 3D (or circles in 2D) radiating out of the ROI s center of mass. We extend this approach to become applicable on real fmri datasets. As a case study we use fmri contrast maps obtained from a study designed to explore neuroanatomical correlates in AD by examining the activation caused by semantic processing in brain images of control and patient samples 2. The 3D images we consider here consist of region data that can be defined as sets of (often connected) voxels (volume elements) that form structures or objects. These structures reflect activation/deactivation levels in the form of spatial patterns. We detect a ROI (activation pattern in a certain area) that discriminates best AD patients from controls 9. Using the features extracted by the characterization technique, we apply artificial neural networks (ANNs) as classifiers achieving an accuracy level between 82% and 87% when distinguishing control from patient samples. We present classification experiments and study how different experimental parameters affect the obtained classification accuracy (prediction/diagnosis). These results show that the proposed classification framework can be potentially useful in diagnosis assisting medical decision making. 2. BACKGROUND Not many approaches have been proposed in the literature for detecting, characterizing and classifying ROIs in medical images. Classification of medical images has been so far mostly based on information extracted from the entire image rather than ROIs. Distribution estimation has been employed in lesion-deficit studies for distinguishing between separate classes (e.g., ADHD vs non-adhd subjects after closed head injury) in medical image data 10. Recently though, some novel methods have been introduced that detect discriminative ROIs based on activation levels in fmri contrast maps. Classification is then performed using several techniques based on the information extracted from these regions. More specifically, Dynamic Recursive Partitioning (DRP), a technique that partitions the space adaptively guided by statistical tests 6, has been employed in past studies 9, 11 for detecting highly informative ROIs with respect to the development of AD. Classification was performed utilizing ANNs and measurements taken from the indicated ROIs, such as the mean activation level. A linearization of the 3D image space by the Hilbert space-filling curve has also been utilized in conjunction with advanced time series classification techniques and ANNs. This approach performs statistical processing in the linear domain while reducing the multiple comparison problem encountered by other popular techniques such as Statistical Parametric Mapping (SPM) 14. All these approaches seek to classify medical image data, such as fmri contrast maps, based on information extracted from the image content. Some of them proceed with further analysis and indicate specific highly informative ROIs with respect to class membership. In several cases though of medical diagnosis there is a need to focus more on

3 these discriminative ROIs and perform specialized analysis on them. This introduces the necessity to develop advanced classification techniques that are based on more detailed classification of certain delineated spatial ROIs. In general, when analyzing a spatial region, informative features are the ones that describe shape and volume content. Most of the attempts to characterize ROIs in the literature focus mostly in representing the shape. Popular techniques that have been used for this purpose range from simpler ones that use anatomical or mathematical landmarks, simpler shapes and polygonalization 15 to more advanced ones that extend to obtaining numerical vectors from various transformations of the boundary. Such techniques include the Fourier Transform 16, the Wavelet transform 17 and moments of inertia 18. Mathematical morphology has introduced operators utilized in the framework of medical image analysis as well 19. In this approach the shape is mapped to a point in the k-dimensional space. For this purpose, a small primitive shape interacts with the input image (via morphological operators like dilation, erosion, opening, and closing) to transform it and in the process, extract useful information about its geometrical and topological structure. 3. METHODOLOGY In this work we focus on classifying medical images while restricting our analysis in specific highly informative regions. ROIs in medical images can be homogeneous or non-homogeneous, depending on the format of the data. Hence, shape as well as internal volume properties are of particular interest, for example when detecting cancerous tissue or high activation levels. As a consequence, there is a need to utilize effective ROI characterization techniques which can extract features that reflect both shape and internal volume properties. For this purpose we propose a framework for in depth ROI analysis in medical images. Our goal is to detect, characterize and classify ROIs according to their discriminative properties. The main steps of the proposed approach are the following: I. Identify highly informative ROIs by utilizing contrasting classes such as controls vs. patients and appropriate medical image data mining techniques, such as DRP 6 or statistical processing after applying space filling curves 12. More traditional approaches can be used for this purpose, for example SPM 14 with voxel-based statistics and clustering techniques to resolve problems due to multiple comparisons as well as atlas-based techniques where whole brain structures instead of voxels are being considered reducing the complexity in the analysis. II. Proceed with focusing on the highly discriminative ROIs detected and extract informative features that form characterization signatures. This is a necessary and very important step prior to classification. Many characterization techniques exist that can be applied for this purpose. We propose using a quantitative feature extraction technique initially presented in 8 which was designed particularly for spatial region data. The main idea is to use concentric spheres in 3D (or circles in 2D) radiating out of the ROI s center of mass with a standard increment of radius. At each increment quantitative measurements are obtained that reflect fractional volumes of the intersection between the ROI and the structural element (sphere or circle). More specifically, the steps of the characterization process are the following: 1. Calculate the center of mass V of the region using a weighted contribution of each voxel of the region (based on each value) for the non-homogeneous ROIs. 2. Using V as center, construct a series of 1 k concentric spheres in 3D (or circles in 2D) with regular increments of their radius. 3. At each increment of radius measure features such as (a) the fraction of the sphere (or circle) occupied by the region, (b) the fraction of the region occupied by the sphere (circle), forming respectively feature vectors f s and f r of size k. The volume calculations are equivalent to the sum of voxels of the ROI or the sphere respectively. For the case of non-homogeneous ROIs, the volume is calculated by a weighted contribution of each voxel based on its value. The obtained vector maps the entire ROI to a specific point in the k-dimensional space. Figure 2 illustrates a snapshot of the characterization process for a ROI in 3D. This technique has been shown to be two orders of magnitude faster than mathematical morphology (namely the pattern spectrum ) although it achieves comparable to or even better characterization results 8. III. Based on this characterization technique, perform classification and similarity searches using geometrical distance measurements between the corresponding vectors, clustering of the characterization signatures or other classification models. We propose using ANNs for classifying the medical image data utilizing the features

4 Fig.2. Intersecting concentric circles with the ROI being characterized. extracted from the highly informative ROIs. More specifically, when dealing with a small number of features we propose using perceptron neural networks. The perceptron architecture is based only on one neuron and can provide robust classification in lower dimensions. When a large number of features is extracted, a hidden layer of 5 neurons can be implemented in the neural network to enhance prediction. For small datasets the Pocket algorithm 20 can be used for training to avoid overfitting. On the other hand, when dealing with large datasets the Levenberg- Marquardt optimization can be employed to reduce training epochs. In all cases the neural network has as inputs the features extracted by the characterization technique and one output to predict class (control vs. patients). More outputs can be added depending on the nature of the prediction required by the analysis. Other classifier models could be utilized as well, such as decision trees or k-nearest neighbor, depending on the nature of the application. We view the above steps as a unified framework proposed for an in depth analysis of ROIs in medical images. The goal is to extract both quantitative and qualitative information in a form that can be utilized for classification and, as an extension to that, for similarity searches and indexing in large medical data repositories. Finally, in the context of this work we apply the feature extraction technique to one selected ROI although the method can be extended to combine characterization signatures from more than one region. 4. EXPERIMENTS AND RESULTS 4.1. Data and preprocessing As mentioned earlier, in this work we seek to classify and characterize ROIs in medical image data. As a case study we analyze a dataset of 3D fmri activation contrast maps obtained from a study efficiently designed to explore neuroanatomical correlates of semantic processing in AD 2. The task was designed to probe semantic knowledge of categorical congruence between word pairs. Each subject was tested with the same timing and word set with a blocked design. The task consisted of an auditory presentation of word pairs (categories and possible exemplars) requiring a semantic decision (match-mismatch). The word pairs were presented in groups of four at 7.0 second intervals, with each 28.0 second block of decision followed by a 10.5 second period of rest. Scans were conducted at 1.5 Tesla using a single shot, gradient echo, echo planar functional scan sequence (TR = 3500 ms, TE = 40 ms, interleaved, FOV = 24 cm, slice thickness = 6 mm, NEX = 1, flip angle= 90) on a General Electric Signa scanner with a multi-axial local gradient head coil system (Medical Advances, Inc., Milwaukee, WI). Scans consisted of contiguous sagittal slices in a 64x64 matrix with in-plane resolution of 3.75mm 2 (total slice acquisitions per run = 1920 scans) with anatomical reference images in the same slice locations using at1-weighted spin-echo pulse sequence (TR = 450 ms; TE = 17 ms; interleaved; matrix = 256x192; NEX = 1; same FOV, slice thickness, and locations as the functional scans). All scans for each subject were acquired in the same session. Our study includes 9 control and 9 patient samples. Prior to the application of the proposed technique, we applied preprocessing to bring homologous regions into spatial coincidence through spatial normalization. The spatial normalization of the scans to a standard template brain using the anatomical reference images was carried out in SPM99, resulting in resampling of the data to 2mm 3 isotropic voxels. The resampled data were smoothed with a Gaussian filter (FWHM 15mm 3 ). Each subject's task-related activation was analyzed individually versus the subject s rest condition, resulting in individual contrast maps giving a measurement of fmri signal change at each voxel. To reduce the effect of noise and sensor fluctuations in the original functional data we applied the following steps. First, we removed the effect of the background noise by subtracting the signal value measured in representative background

5 Fig. 3. The ROI being characterized in consecutive 2D slices of the 3D volume, after being overlaid on the T1 canonical atlas. (a) (b) Fig. 4. A 2D slice of the 3D activation contrast map for a control (a) and a patient (b) sample. voxels from all the voxels of the 3D volume. Second, we masked the data using a binary mask extracted from the T1 anatomical atlas used as the template the data were spatially registered to. Only signal within the binary mask was included in the analysis. For the experiments presented here, we focused only on a specific region of the brain that has been shown to discriminate best AD patients from controls 9,11. In 9,11 the Dynamic Recursive Partitioning (DRP) algorithm 6 has been used to detect spatial fmri activation patterns (ROIs) indicative of AD. The ROI that we focus here is constructed by two neighboring sub-regions within the medial temporal lobe of the human brain. These sub-regions have p_values of and respectively when using t-test to determine the significance of their association with AD. Figure 3 shows this ROI in consecutive 2D slices of the 3D volume, after being overlaid on the T1 canonical atlas. Figure 4 illustrates a 2D slice of the 3D contrast map for a control and a patient sample Experimental evaluation and results In order to perform classification we first applied the proposed feature extraction technique to the selected ROI in each sample. We experimented with 0.1, 0.2 and 0.5 radius increments, extracting respectively 18, 40 and 80 dimensional feature vectors. Figure 5 illustrates the obtained characterization signatures for each of these cases and the feature vectors f r and f s. As we can observe, signatures of different classes tend to cluster and follow similar behavior, especially for the case of f s signatures. The curvature of the signatures conveys information for the activation patterns of the original data. For example, as Figure 5 (d) (e) (f) shows, patient samples exhibit positive activation in the specific ROI, whereas the control subjects have lower, mostly negative activation (deactivation) levels. This information is highly discriminative and the proposed feature extraction technique has the ability to reflect it in an interpretable form. Using these feature vectors as inputs to classification models we proceed with classification experiments, as proposed in the methodology section. All experiments were implemented using PRTools toolbox for MATLAB 21. More precisely, we used ANNs and implemented one-layer perceptron networks trained by the Pocket algorithm 20, to avoid overfitting due to a small training dataset. The leave-one-out approach was used to evaluate out-of-sample performance. The training set consisted of patients and controls with indices 1,2,3,,i-1,i+1, 9 and the method was tested on patient and control with an index i, where i=1,,9. Taking into consideration the stochastic nature of the Pocket algorithm, we repeated the process of training and testing the model in each of the leave-one-out loops for 30 times and averaged the percentage of the correct predictions to obtain the reported accuracy. Table 1 shows the results obtained for all the different experimental settings.

6 (a) (b) (c) (d) (e) (f) Fig. 5. Characterization signatures for the controls (blue) and the patient samples (red +). In (a), (b), (c) the f r signatures and (d), (e), (f) the f s signatures for radius increment (a), (d) 0.1, (b), (e) 0.2 and (c), (f) 0.5. Classification Accuracy Radius Increment (features) Control Patients Overall 0.5 (18 features) (40 features) (80 features) Table 1: Classification accuracies obtained for different experimental settings. As we can observe the classification reaches the accuracy range of 84%-87%, which is really encouraging taking under consideration that feature vectors extracted from only one ROI have been utilized. Also, it seems that the radius increment has a rather significant influence on the effectiveness of the feature extraction process; this is illustrated by the classification experiments. As the results demonstrate the dimensionality affects the classification accuracy not in a proportional way to the number of features selected. Best results are obtained with 80 features, while at the same time 40 features provide worst performance than the 18 features selected when using the largest radius increment. This suggests that a dimensionality reduction technique such as PCA or SVD could provide a smaller set of features with more discriminative power. These would construct more informative characterization signatures and further improve the classification accuracy. In general though, these results show that the proposed ROI classification framework has the ability to become very useful for classifying ROIs in medical image analysis.

7 5. DISCUSSION In the experiments presented above we validated the applicability of the proposed ROI analysis and classification framework. The DRP 6 was applied to our dataset in order to detect the highly informative ROIs. As mentioned though in the methodology section other techniques could be employed for this purpose. For example, in a technique that employs the linearization of the 3D space with space-filling curves and statistical processing in the 1D has been proposed. DRP though has the advantage of performing adaptive analysis of the image space reducing the number of tests performed (i.e., the multiple comparison problem) and unnecessary computational cost. The linearization approaches should be preferred when classification is performed by advanced time series techniques in the 1D space, such as in 13. One drawback in this case though is that space-filling curves do not preserve the spatial locality of the data with absolute accuracy and some information could be lost. Moreover, since the statistical processing is applied statically and not adaptively/dynamically there is a possibility that the ROIs are not delineated with high precision. Finally, in very good classification results were achieved using measurements obtained from all the indicated ROIs. In this paper, we achieve comparable classification constructing signatures from only one highly discriminative ROI. We believe that the proposed ROI analysis framework has the ability to achieve even better classification results when analyzing simultaneously more than one discriminative ROI and combine the signatures extracted from them. This is an extension that requires further research and experimentation. 6. CONCLUSIONS In this work we focus on classifying and characterizing ROIs in medical images. More specifically, we extend a feature extraction technique designed for spatial region data to be applied to real medical ROIs and further utilized the extracted feature vectors to perform classification. This technique has been shown to be two orders of magnitude faster than other approaches used for similar purposes. As a case study we analyzed a dataset obtained from a study designed to explore neuroanatomical correlates in semantic processing for AD. We included 9 control and 9 patient 3D fmri contrast maps and focused in a ROI that has been shown to be highly informative for AD. As classifiers we implemented one-layer perceptron ANNs trained by the Pocket algorithm. The classification accuracy reached the level of 84%-87%. These results show the ability of the proposed classification framework to effectively assist in medical decision making (diagnosis). Finally, we believe that the proposed approach can be further improved by combining the characterization signatures from more than one ROI and significantly increase the classification performance. ACKNOWLEDGEMENT The authors would like to thank A. Saykin for providing the fmri data set and clinical expertise and J. Ford for performing some of the preprocessing of this data set. This work was supported in part by the National Science Foundation (NSF) under grants IIS and IIS and by the National Institutes of Health (NIH) under grant R01-MH The funding parties specifically disclaim responsibility for any analyses, interpretations and conclusions. REFERENCES 1. S.H. Koslow and M.F. Huerta, NeuroInformatics: an Overview of the Human Brain project, Lawrence Erlbaum, Mahway, NJ, A.J. Saykin, L.A. Flashman, S.A. Frutiger, S.C. Johnson, A.C. Mamourian, C.H. Moritz, J.R. O'Jile, H.J. Riordan, R.B. Santulli, C.A. Smith and J.B. Weaver,: Neuroanatomic substrates of semantic memory impairment in Alzheimer's disease: Patterns of functional MRI activation, Journal of the International Neuropsychological Society, 5, pp , 1999.

8 3. J. Ford, L. Shen, F. Makedon, L. Flashman and A. Saykin, A Combined Structural-Functional Classification of Schizophrenia using Hippocampal Volume plus fmri Activation, in Proc. of EMBS-BMES 2002 Second Joint Meeting of the IEEE Engineering in Medicine and Biology Society and the Biomedical Engineering Society, Houston, Texas, October A. Pentland, R. W. Picard, and S. Sclaroff, "Photobook: tools for content-based manipulation of image databases," in Proc. of the SPIE Conference, Storage and Retrieval of Image and Video Databases II, San Jose, CA, M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, b. Dom, and M. Gorkani, "Query by image and video content: the QBIC system," IEEE Computer, pp , V. Megalooikonomou, D. Pokrajac, A. Lazarevic, Z. Obradovic, "Effective classification of 3-D image data using partitioning methods", in Proc. of the SPIE 14th Annual Symposium in Electronic Imaging: Conference on Visualization and Data Analysis, San Jose, CA, Jan V. Megalooikonomou, J.Ford, L.Shen, F.Makedon, and A.Saykin, "Data mining in brain imaging", Statistical Methods in Medical Research, 9, No. 4, pp , V. Megalooikonomou, H. Dutta and D. Kontos, "Fast and Effective Characterization of 3D Region Data", in Proc. of the IEEE International Conference on Image Processing (ICIP) 2002, pp , Rochester, NY, Sep V. Megalooikonomou, D. Kontos, D. Pokrajac, A. Lazarevic, Z. Obradovic, O. Boyko, A. Saykin, J. Ford and F. Makedon, Classification and Mining of Brain Image Data Using Adaptive Recursive Partitioning Methods: Application to Alzheimer Disease and Brain Activation Patterns, Human Brain Mapping Conf. (OHBM'03), New York, NY, 2003, also in NeuroImage, 19 (2) S48, June A. Lazarevic, D. Pokrajac, V. Megalooikonomou and Z. Obradovic, " Distinguishing Among 3-D Distributions for Brain Image Data Classification ", in Proc. of the 4th International Conference on Neural Networks and Expert Systems in Medicine and Healthcare, pp , Milos Island, Greece, June D.Kontos and V. Megalooikonomou, Computationally Intelligent Methods for Mining 3D Medical Images, in Proc. of SETN 2004 the 3 rd Hellenic Conference on Artificial Intelligence, Lecture Notes in Artificial Intelligence, Springer-Verlag, 2004 (to appear). 12. D. Kontos, V. Megalooikonomou, N. Ghubade and C. Faloutsos, "Detecting discriminative functional MRI activation patterns using space filling curves", in Proc. of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp , Cancun, Mexico, Sep Q. Wang, D. Kontos, G. Li and V. Megalooikonomou, Application of time series techniques to data mining and analysis of spatial patterns in 3D images in Proc. of The 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2004), Montreal, Canada, KJ. Friston, AP. Holmes, KJ. Worsley, JP. Poline, CD. Frith, RSJ. Frackowiak, Statistical parametric maps in functional imaging: a general linear approach, Human Brain Mapping, 2, pp , M. P. Deseilligny, G. Stamon and C. Y. Suen, Veinerization: a new shape description for flexible skeletonization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, pp , E. Pesoon and K. Fu, Shape discrimination using Fourier descriptors, IEEE Transactions on SMC, 7, pp , S. Mallat, A theory for multiresolution signal decomposition: the wavelet representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, pp , M. K. Hu, Visual pattern recognition by moment invariants, IRE Transactions on Information Theory, 8, pp , F. Korn, N. Sidiropoulos, C. Faloutsos, E. Siegel and Z. Protopapas, Fast and effective retrieval of medical tumor shapes, IEEE Transactions on Knowledge and Data Engineering, 10, pp , S.I. Gallant, Perceptron-Based Learning Algorithms, IEEE Transactions on Neural Networks, 1 (2), pp , R.P.W. Duin, PRTools Version 3.0, A Matlab Toolbox for Pattern Recognition,