Analysis of texture patterns in medical images with an application to breast imaging

Size: px

Start display at page:

Download "Analysis of texture patterns in medical images with an application to breast imaging"

Damian Richardson
6 years ago
Views:

1 Analysis of texture patterns in medical images with an application to breast imaging Vasileios Megalooikonomou *a, Jingjing Zhang a, Despina Kontos b, Predrag R. Bakic b a Data Engineering Laboratory (DEnLab), Department of Computer and Information Sciences, Temple University, 1805 N. Broad St., Philadelphia, PA, 19122, USA b Department of Radiology, Hospital of the University of Pennsylvania, 3400 Spruce St., Philadelphia, PA, 19104, USA ABSTRACT We propose a methodological framework for texture analysis in medical images that is based on Vector Quantization (VQ), a method traditionally used for image compression. In this framework, the codeword usage histogram is used as a texture descriptor of the image. This descriptor can be used effectively for similarity searches, clustering, classification and other retrieval operations. We present an application of this approach to the analysis of x-ray galactograms; we analyze the texture in retroareolar regions of interests (ROIs) in order to distinguish between patients with reported galactographic findings and normal subjects. We decompose these ROIs into equi-size blocks and use VQ to represent each block with the closest codeword from a codebook. Each image is represented as a vector of frequencies of codeword appearance. We perform k-nearest neighbor classification of the texture patterns employing the histogram model as a similarity measure. The classification accuracy reached up to 96% for certain experimental settings; these results demonstrate that the proposed approach can be effective in performing similarity analysis of texture patterns in breast imaging. The proposed texture analysis framework has a potential to assist the interpretation of clinical images in general and facilitate the investigation of relationships among structure, texture and function or pathology. Keywords: Texture descriptors, vector quantization, pattern analysis, classification, x-ray galactograms 1. INTRODUCTION Modern medical imaging modalities such as Magnetic Resonance Imaging (MRI), Computed Tomography (CT) and x- ray mammography have made available large collections of two-dimensional (2D) and three-dimensional (3D) images that capture the structure and function of different types of anatomical structures. Numerous texture patterns are met frequently in such medical images; the problem of analyzing these patterns is very challenging. Based on the advancements in multimedia technology over the last decade, content-based image retrieval has become one of the most vivid research areas in the field of computer vision. Considering medical image repositories, content-based retrieval and analysis systems have also been developed; computer science technologies for image querying, management and pattern analysis are critical for effectively performing these tasks. In this paper, we propose a methodological framework for texture analysis in medical images that is based on Vector Quantization (VQ). VQ has been traditionally used for image compression; the use of VQ for mammogram compression has been reported in the literature [1]. Our goal is to perform texture analysis in particular regions of interests (ROIs) in medical images in order to assist diagnosis. The proposed framework consists mainly of two steps: (i) apply the VQ keyblock encoding [2] on the ROIs to obtain compressed images, and (ii) employ the Histogram Model as a similarity measure for texture descriptors. * Phone: , Fax: , vasilis@temple.edu

2 The rest of the paper is organized as follows: in Section 2 we provide background information and discuss previous work. In Section 3 we introduce the framework of our methodology and the proposed similarity measure. In Section 4 we present the experimental evaluation of our approach after introducing the dataset we used. Finally, Section 5 concludes the paper by summarizing the main points of our contribution. 2. BACKGROUND Early research in image texture analysis focused on using low-level features such as color, texture, shape and others extracted from images as characteristics to describe the image content. Various content-based image retrieval systems, including QBIC (Query by Image Content) [8], Photobook [9] and others, have been built for general or specific image retrieval tasks. Unfortunately, the capabilities of these technologies are limited by the extreme difficulty of characterizing different images. Recently, new approaches that solve more precise and general image retrieval tasks have been introduced, such as the keyblock approach [2] which has demonstrated its effectiveness in the image similarity search field. The keyblock approach has recently been used to analyze the texture of medical images in [10]. In particular in that paper, the keyblock approach was applied to compress fmri data and generate texture descriptors for these images. When a query image is given, the similarity between the query image and each image in the dataset is computed and the most similar images are retrieved. In addition to employing the keyblock approach into texture analysis of fmri data, the authors proposed an improved extended version of it that retains comparable effectiveness and achieves higher efficiency. Texture analysis of medical images has attracted many investigators particularly for the purpose of developing computer-aided diagnosis (CAD) systems, which are becoming more and more prevalent due to their ability to increase the precision and accuracy of clinical interpretation by radiologists [11]. Searching medical databases for images similar to a given query image that corresponds to a case under current study and enabling access to those other clinical data and known diagnoses from similar cases is expected to have great impact in CAD systems, which are aimed to assist radiologists and physicians to provide diagnosis based on medical images [12]. However, efficient and accurate analysis of texture of medical images remains a challenging task. Considering in particular texture analysis in breast images, there is a need to assess properties of the underlying breast ductal network; the texture patterns observed in the image are the radiographic effect of the underlying anatomical arrangement. The analysis of such natural tree-like structures in biomedical images presents special challenges; the surrounding tissue may obscure branching patterns. Examples of such tree-like structures include the bronchial tree, the blood vessel network, the nervous system, and the breast ductal network. Properties such as topology, spatial distribution of branching, and tortuosity, have been analyzed in the literature and associated with altered function and/or pathology. For example, regional changes in vessel tortuosity have been used to identify early tumor development in the human brain [3]. Similarly, studies have shown that the morphology of the ductal network can provide valuable insight to the development of breast cancer and assist in diagnosing pathological breast tissue [4]. However, imaging techniques that clearly visualize tree-like structures may be impractical in terms of cost, safety, and comfort. Galactography, for example, can be performed to visualize the breast ductal network by injecting a contrast agent into the lactiferous ducts of the breast. Galactography can be useful for visualizing early symptoms of papilloma or ductal ectasia, which cause spontaneous nipple discharge, without an identifiable mammographic lesion. Nevertheless, such a procedure is not frequently performed and is considered to be complicated and painful. In previous work [5,6,7] we have investigated topological descriptors of tree-like structures representing the breast ductal network and developed methods for quantitative characterization and classification. Here, in order to overcome such obstacles in the analysis of tree-like structures, we study the problem of automatic extraction of descriptive features that correspond to discriminative characteristics and patterns of texture. In this paper, we present an application of the keyblock approach to texture analysis of x-ray galactographic images. We perform our analysis on breast images due to the particular challenging task of visualizing and characterizing the breast ductal network. We focus on texture analysis of particular regions of interests (ROIs) in galactograms having as goal to assist diagnosis.

3 3. METHODOLOGY The proposed methodological framework for texture analysis consists of the following steps: (i) apply the Vector Quantization (VQ) keyblock technique to generate texture descriptors, (ii) employ the Histogram Model to measure the similarities among different texture descriptors, and (iii) use the k-nearest neighbor algorithm (K-NN) to classify the images. 3.1 The keyblock approach The keyblock approach decomposes each image into equi-size blocks and uses vector quantization (VQ) to represent each block with the closest codeword from a codebook. VQ is widely used in image compression and encoding; it is a lossy compression method based on the principal of block encoding. Given a fixed block size, each image is first decomposed into a number of small blocks. Each small block contains features of the sub area of its corresponding image. Based on such small blocks from different images in the database, a codebook containing codewords (keyblocks) is generated. In order to generate the codebook, the Generalized Lloyd Algorithm (GLA), which produces a local optimal codebook based on two conditions (the Nearest Neighbor Condition (NNC) and the Centroid condition (CC)), is used. Starting with an initial codebook, the GLA repeats the Lloyd iteration until the average distortion becomes less than a given threshold. GLA is a clustering technique. It performs a number of iterations; each one recomputes the set of more appropriate partitions of the input vectors, and their centroids. The algorithm is shown in Table 1. It uses a training set T of M k-dimensional data vectors to generate a set C of N desired clusters. Table.1. The Generalized Lloyd Algorithm (GLA) 1. Begin with an initial codebook C Repeat (a) Given a codebook (set of clusters defined by their centroids) C m { y i ; i 1,..., N}, redistribute each training set vector into one of the clusters in C m by selecting the one whose centroid is the closest to the vector (i.e., NNC). (b) Recompute the centroids for each cluster just created, using the Centroid Definition, to obtain the new codebook C m+1 (i.e., CC). (c) If an empty cell was generated in the previous step, an alternative code vector assignment is made (instead of the centroid computation). (d) Compute the average distortion for C m+1. Until the distortion has only changed by a small enough amount since the last iteration. Every image in the database is then encoded using the codebook. Initially, each image is decomposed into blocks, then for each block, the closest entry in the codebook is located and the corresponding index is stored. In such a way, each image can be represented as a vector of frequencies of keyblock (codeword) appearance. Finally, when a query image q is submitted, it is decomposed into blocks and then encoded using the codebook of keyblocks. The similarity between the query q and each image t in the database is estimated. The top k most similar images retrieved by the system are those with the largest similarity values with the current query. Figure 1 shows a flowchart of the keyblock approach.

The idea of HM is to compute the distance between two images based on the codeword appearance frequency and then get a quantitative measure of similarity.

4 Fig.1. Flowchart illustrating the keyblock approach for image similarity and classification 3.2 Similarity measure In our framework, we employ the Histogram Model (HM) as the similarity measure of texture descriptors. The idea of HM is to compute the distance between two images based on the codeword appearance frequency and then get a quantitative measure of similarity. More specifically, for a query image Q, we compute the similarity Sim HM between image Q and each image X in the database, based on the Histogram Model, using the following formula: where, 1 Sim HM ( X, Q) 1 dis( X, Q) s f i, x f i, q dis( X, Q). 1 f f i 1 i, x i, q In the above formula, f i,x and f i,q refer to the appearance frequency of codeword C i in images X and Q respectively while s refers to the size of the codebook. 3.2 K-NN classification K-NN was first introduced in [13]. The goal of k-nn is to separate the data based on the assumed similarities into various classes. Thus, the classes can be differentiated from one another by searching for similarities between the data provided. In pattern recognition, the k-nearest neighbor algorithm (k-nn) is a method for classifying objects based on closest training examples in the feature space. The training phase of the algorithm consists of storing the feature vectors and class labels of the training samples. In the actual classification phase, the same features as before are computed for the test sample (whose class is not known). The distances from the new vector to all stored vectors are computed and the k closest samples are selected. The new point is predicted to belong to the most numerous class within the set. In our framework, the test sample is the query image whose class is not known. During the training phase, the whole dataset is used except for the test sample. The best choice of k depends on the data; generally, larger values of k reduce the effect of noise on the classification but make boundaries between classes less distinct. A good value for k can be selected by parameter optimization using, for example, cross-validation. The special case where the class is predicted to be the class of the closest training sample (i.e., when k = 1) is called the nearest neighbor algorithm.

4. EXPERIMENTS We applied the proposed approach to characterize and classify the texture of regions of interest (ROIs) manually extracted from x-ray galactograms [5].

pathology [14]. 4.1 Dataset The dataset we used is part of a study [5] where 23 galactographic views were retrospectively collected from 14 patients.

5 4. EXPERIMENTS We applied the proposed approach to characterize and classify the texture of regions of interest (ROIs) manually extracted from x-ray galactograms [5]. The ROIs were taken from the galactogram area behind the nipple; such ROI selection for texture analysis has been shown in the literature to be highly indicative of a woman s risk to develop pathology [14]. 4.1 Dataset The dataset we used is part of a study [5] where 23 galactographic views were retrospectively collected from 14 patients. This dataset contains images from 2 groups: 10 cases with no radiological findings (NF) and 13 cases with radiological findings (RF). The images were digitized with a spatial resolution of (100 micron) 2 /pixel using a Lumisys digitizer (Sunnyvale, CA). From every image, a pixel ROI was manually segmented in the region of the breast behind the nipple; this positioning of the ROI has been shown to have high discriminative power [14]. The analysis of texture patterns was performed based on the 23 ROIs. We classified the galactograms based on the classification of the corresponding textures. Figure 2 shows examples of different texture patterns in galactograms. Fig.2. Two different texture patterns obtained from x-ray galactograms. 4.2 Approach We experimentally studied the keyblock-based approach using the Histogram Model as a similarity measure. We performed k-nearest neighbor classification to discriminate among two classes: cases with no radiological findings (NF) and cases with radiological findings (RF). We employed the similarity measure to retrieve the k most similar images (i.e., neighbors) to every image in the dataset. The class was decided by the k-nearest neighbors and the accuracies were obtained by averaging the binary prediction result. After the codebook was generated, the system decomposed each ROI in the database into a certain number of blocks and mapped the block features using representative elements from the codebook to get the encoded vector. All images in the database were encoded and the texture descriptors were stored. Figure 3 illustrates a sample of part of the codebook, an original image and the corresponding image reconstructed based on the texture descriptors. K-nearest neighbor classification was performed on each of the 23 ROIs. Each time, we considered one ROI and computed its similarity to all other ROIs in the database using the Histogram Model. Based on the computed similarities, we retrieved the top k (a constant that the user may specify) most similar matches. Then, the prediction of the class label of the query ROI was voted by the k nearest neighbors. If the voted result matched the real group that the query belonged to, then we recorded the binary prediction indicator to be 1; if the result did not match, the binary prediction indicator was set to 0. The reported accuracy was the average of the 23 prediction indicators.

6 (a) (b) (c) Fig.3. (a) An original ROI of an x-ray mammography (256*256), (b) part of a codebook, (c) the reconstructed image using the codebook Intuitively, the size of the codebook, the size of the keyblocks and the value of k, all, influence accuracy. In our experiments, we explored the effect of varying the value of k on the classification performance, while varying codebook sizes as well as keyblock sizes. In addition, for comparison, we used different parameters for codebook generation, i.e., different keyblock sizes (e.g., 4 4 or 8 8 or 16 16) and codebook sizes (e.g., 256 or 512). Moreover, to obtain more stable results for both the keyblock and the extended approach, given the similarity measure, the codebook size, and the keyblock size along with a value for k, we repeated the experiments five times and reported the average performance (see next section). 4.3 Results We considered each of the ROIs as a query and retrieved the k-nearest neighbors, i.e., the k most similar ROIs with respect to texture based on the VQ encoding and the histogram model. We performed each experiment five times. Table 2 illustrates the average classification performance for several values of codebook size, block size, and k. Table.2. Classification accuracies obtained using k-nearest neighbor classification on the keyblockcompressed images (the Histogram Model was used as a similarity measure) cdbk size / block size k = 1 k = 2 k = 3 k = / / / / / /

7 There is a considerable variation in accuracy for different values of k. While the best choice for k depends on the data, we were able to achieve the highest classification accuracy for k = 2 in this study. We are in the process of further investigating this relationship to establish the statistical significance of these results. Despite the fluctuations in classification accuracy caused by the choice of k, these results demonstrate the effectiveness of the proposed methodology for texture analysis of breast images. The classification accuracy based on texture features reached up to 96% for certain experimental settings. These results complement our previous work on analyzing ductal topological properties [6]. We are in the process of further investigating how to combine texture and topological descriptors to obtain more robust performance. We believe that texture analysis tools can potentially assist the interpretation of clinical images and facilitate the investigation of relationships among structure, texture and function or pathology. 5. CONCLUSION The proposed texture descriptors can be used efficiently and effectively for similarity searches, clustering, classification and other operations in medical image databases. Developing such tools for characterizing and effectively classifying texture patterns in medical images would greatly assist the interpretation of clinical images and the investigation of the relationship between texture and branching patterns and the associations between morphology and function or pathology. Advancing our understanding of breast anatomy and physiology can greatly assist early cancer detection and cancer risk estimation, and may even improve computer simulations of breast tissue for the purpose of evaluating novel breast imaging modalities. 6. ACKNOWLEDGEMENTS This work was supported in part by the National Science Foundation under grant IIS Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding agency. REFERENCES [1] Mitra, S., Yang, S.Y., High Fidelity Adaptive Vector Quantization at Very Low Bit Rates for Progressive Transmission of Radiographic Images, Journal of Electronic Imaging, Vol. 8, pp.23-35, [2] Zhu, L., Rao, A., Zhang, A., Theory of Keyblock-Based Image Retrieval, ACM Transactions on Information Systems, Vol. 20 (2), pp , Apr [3] E. Bullitt, D. Zeng, G. Gerig, S. Aylward, S. Joshi, J. K. Smith, W. Lin, and M. G. Ewend, "Vessel Tortuosity and Brain Tumor Malignancy: A Blinded Study," Academic Radiology, vol. 12, pp , [4] B. Pereira and K. Mokbel, "Mammary ductoscopy: past, present, and future," Mammary ductoscopy: past, present, and future, vol. 10, pp , [5] Bakic, PR., Albert, M., Maidment, AD., Classification of galactograms with ramification matrices: preliminary results, Academic Radiology, Vol. 10 (2) pp , [6] Megalooikonomou, V., Kontos, D., Danglemaier, J., Javadi, A., Bakic, PR., Maidment, AD., A representation and classification scheme for tree-like structures in medical images: An application on branching pattern analysis of ductal trees in x-ray galactograms, Proceedings of the SPIE Conference on Medical Imaging, San Diego, CA, Feb

8 [7] Kontos, D., Megalooikonomou, V., Javadi, A., Bakic, PR., Maidment, AD., Classification of galactograms using fractal properties of the breast ductal network, Proceedings of the IEEE International Symposium on Biomedical Imaging, Arlington, VA, Apr [8] J. Ashley, M. Flickner, J. Hafner, D. Lee, W. Niblack, D. Petkovic, The query by image content (QBIC) system, ACM SIGMOD Record, Vol. 24, No. 2, pp. 475, November [9] A. Pentland, R.W. Picard, S. Sclaroff, Photobook: Content-based manipulation of image databases, International Journal of Computer Vision, Vol. 18, No. 3, pp , June [10] Zhang, J., Megalooikonomou, V., An Effective and Efficient Technique for Searching for Similar Brain Activation Patterns, Proceedings of The Fourth IEEE International Symposium on Biomedical Imaging, Washington DC, [11] Jiang Y, Nishikawa RM, Schmidt RA, Metz CE, Giger ML, Doi K. Improving breast cancer diagnosis with computer-aided diagnosis. Academic Radiology 1999; 6: [12] Tourassi GD, Vargas-Voracek R, Catarious DM, Floyd CE. Computer-assisted detection of mammographic masses: a template matching scheme based on mutual information. Medical Physics 2003; 30: [13] Fix, E., J. Hodges, J., Discriminatory Analysis: Nonparametric Discrimination: Consistency Properties, Air Technical Index, pp. 24, Feb [14] Li, H., Giger, ML., Huo, Z., et al. Computerized analysis of mammographic parenchymal patterns for assessing breast cancer risk: effect of ROI size and location, Medical Physics, Vol. 31, Issue 3, pp , 2004.

University, 1805 N.Broad St., Philadelphia PA 19122, USA. Philadelphia, Pennsylvania 19104, USA ABSTRACT

University, 1805 N.Broad St., Philadelphia PA 19122, USA. Philadelphia, Pennsylvania 19104, USA ABSTRACT A representation and classification scheme for tree-like structures in medical images: An application on branching pattern analysis of ductal trees in x-ray galactograms Vasileios Megalooikonomou 1, Despina