Tissues Image Retrieval System Based on Cooccuerrence, Run Length and Roughness Features

Transcription

1 Tissues Image Retrieval System Based on Cooccuerrence, Run Length and Roughness Features Loay Edwar George Department of Computer Science University of Baghdad Baghdad, Iraq Esraa Zeki Mohammed State Company of Internet Services Ministry of Communication Kirkuk, Iraq Abstract The research presented in this paper was aimed to improve the retrieval performance of an images retrieval system in medical applications based on texture features. In general, the work consists of two phases: () enrollment phase, which consist of feature extraction based on Co-occurrence matrix and run length matrix features combined with developed method to measure the roughness, (2) retrieving phase, which use the artificial neural network and similarity measurement. The conducted tests were carried on 600 medical images from four types of tissues (i.e., blood cells, breast tissues, GI tissues, liver tissues) and give very high precision and recall rates (00,98). Keywords Terms: -Coccurrence matrix, image retrieval, run length matrix,, texture analysis, medical diagnosis, neural network I. INTRODUCTION Medical image databases represent the key components in future diagnosis and preventive medicine. There is an increasing trend towards the digitization of medical imagery and the formation of adequate archives. More and more hospitals purchase picture archiving through communication systems (PACS). The medical imagery over worldwide is increasingly acquired, transferred and stored digitally []. Medical image content representation and retrieval is playing an increasing role in a wide spectrum of applications within the clinical process. Clinical decision support techniques such as case-based reasoning or evidence-based medicine can produce a strong need to retrieve images valuable for supporting certain diagnoses. Computer-aided diagnostics are on the rise and create a need for powerful data and meta-data management and retrieval. Besides diagnostics, teaching and research are expected to improve through the use of visual access methods in existing large repositories. The goals of medical information systems have often been defined to deliver the needed information at the right time, the right place and to the right persons for improving the quality and efficiency of care processes [2]. The field of content-based image retrieval (CBIR) focuses on the analysis of image content and the development of tools to represent the visual content in a way that can be efficiently searched and compared. CBIR techniques extract visual features and perform the indexing, querying and retrieval based on such features. General CBIR approaches are intended for browsing large archives of general (arbitrary) content. More recently, CBIR systems are starting to emerge in medical applications, and the medical domain is currently cited as one of the principle application domains for content-access technologies. Medical image retrieval tasks impose several distinct challenges from retrieval of natural or general imagery. While rapidly developing, the field of medical content retrieval is in its infancy and there are many challenges ahead []. Scientific interest in artificial neural networks mainly arises from their potential ability to perform interesting computational tasks. In principle, ANNs are mostly used for pattern matching capabilities. Their human-like characteristics are utilized to assist medical decision making. Neural networks are extremely useful, because they are not only capable of recognizing patterns with the aid of expert, but also of generalizing the information contained in the input data. ANN with ability to learn by example is very flexible and powerful in medical diagnosis. The objective of ANNs is to support doctors and not to replace them [3, 4]. Many studies have applied the concepts of texture feature extraction and neural network to analyze and retrieve cells, tissues, and other region of interest in biomedical images. Lehmann, et al [5], suggested an automatic categorization consist of more than 80 categories describing the imaging modality and direction. They examined the system as well as the body part and biological system images. Based on 623 reference images from hospital routine, 85.5% correctness is obtained combining global texture features with scaled images. With a frequency of 97.7%, the correct class is within the best ten matches, which is sufficient for medical CBIR applications. Andre, et al [6], investigated the use of content based image retrieval methods to classify microscopic images into two categories: neoplastic (pathological) and benign. First, they describe the method that maps an image into a visual feature signature which is a numerical vector invariant with respect to some particular classes of geometric and intensity transformations. Then they explain how these signatures are used to retrieve from a database the k-closest images to a new image. The classification is finally achieved through a procedure of votes weighted by a proximity criterion (weighted k-nearest neighbors). The accuracy rate was 80% on a database

2 consist of 54 patients colonoscopy images. Mehta, et al [7], proposed a method in which both color and texture features of the images are used to improve the retrieval results in terms of its accuracy. Color extraction and comparison are performed using Conventional Color Histograms (CCH) and the Quadratic Distance Metric (QDM). The texture extraction and comparison tasks are performed using the concept of Pyramid Structure Wavelet Transform Model (PSWTM) and the Euclidean distance measure. Suganya, et al [8], presented a hybrid approach called Support Vector Machine combined with relevance feedback for the retrieval of liver diseases from Ultrasound images. First the ultrasound images are registered in the database based on the modality. Then the features, derived from first order statistics, gray level Co-occurrence matrix and fractal geometry are obtained from the Pathology Bearing Regions (PBRs) among the normal and abnormal ultrasound images. The Correlation Based Feature Selection (CFS) algorithm selects the certain features for the specific diseases and also reduces dimensionality space for classification. Finally, they implemented hybrid approach for retrieval of specific diseases from the database. The accuracy rate of hybrid approach was 72.0% on a database consist of 50 images of liver diseases. II. CONCEPTS AND METHODS A. Co-occuerrence Matrix In a statistical texture analysis, texture features were computed on the basis of statistical distribution of pixel intensity at a given position relative to others in a matrix of pixels representing image. Depending on the number of taken pixels in each combination, there is first-order statistics, second-order statistics or higher-order statistics. Feature extraction based on Co-occurrence matrix is the second-order statistics that can be used to analyze image content as a texture. Figure () below presents an example about the formation of the Co-occurrence matrix of the gray image (4 levels) image at the distance d and the direction of 0 [9]. characteristics, which will reflect important information s about the nature and extent of existing local correlation between pixels (i.e., several values were tested in this work,2, and 3). After counting the frequency of each possible transition between pixels values, there is still one step to take before texture measures can be calculated. The measures require that each Co-occurrence matrix cell contain not a count, but rather a probability. It is defined by P(a,b d,θ) which expresses the probability of the couple pixels at θ direction and d interval. When θ and d is determined, P(a,bd,θ) is showed by P(a,b). Once the normalized Co-occurrence matrix has been created, various features can be computed from it. Haralick and his colleagues extracted 4 features from the Co-occurrence matrix, although in many applications only eight features are widely used, and in this work only these 8 features have been used, they are: Contrast, Energy, Norm Entropy, Homogeneity, Cluster Shade, Cluster Prominence, Inverse Difference Moment, and Maximum Probability. B. Run Length Matrix Run-length statistics capture the coarseness of a texture in specified directions. A run is defined as a string of consecutive pixels which have the same intensity along a specific linear orientation. Fine textures tend to contain more short runs with similar intensities, while coarse textures have more long runs with significantly different intensities. Run length is the number of adjacent pixels that have the same intensity in a particular direction. Run-length matrix is a two-dimensional matrix where each element is the number of elements j with the intensity i, in a given direction. For example, Figure (2.a) below shows a matrix of size 4x4 pixel image with 4 levels. Figure (2.b) the corresponding Run-length matrix in the direction of 0 [9]. Figure 2. (a) (b) An example of RLM formation Figure. An example of GLCM formation In this work in addition to the horizontal direction (0º), GLCM can also be formed for the direction of 45º, 90º and 35º. Co-occurrence matrix is a matrix of frequencies at which two pixels, separated by a certain vector, occur in the image. The contents of the GLCM matrix depend on the scan direction and the distance relationship between pixels. By varying the separation distance it allows to capture different texture In addition to the 0º direction, run length matrix can also be formed in the other directions, in this work run length matrix was calculated in 0º direction (horizontally) and 90º direction (vertically), because most of the medical images are concerned with complication tissues which doesn t show fine isotropic symmetries along specific direction. After determination the frequency of occurrence for each possible run, then the probability is calculated for each direction and for both directions at same time, which in turn is used to calculate the run length features values. From each run-length matrix p(i, j), many numerical texture measures can be computed. In this

3 work thirty features of run length statistics have been used (i.e., ten for horizontal, ten for vertical and ten for both directions), and these features are: Short Run Emphasis, Long Run Emphasis, Gray-Level Non uniformity, Run Length Non uniformity, Low Gray-Level Run Emphasis, High Gray-Level Run Emphasis, Short Run Low Gray-Level Emphasis, Short Run High Gray-Level Emphasis, Long Run Low Gray-Level Emphasis, and Long Run High Gray-Level Emphasis. C. Roughness Feature Roughness is a measure of the texture of a surface. It is quantified by the vertical deviations of a real surface from its ideal form. If these deviations are large, the surface is rough; if they are small the surface is smooth. Roughness is typically considered to be the indication for the degree of existing high frequency, short wavelength component of a measured surface. The surface of a rough texture presents a high number of asperities. In an image, roughness can be described as a set of quick spatial transitions with varying amplitude. From a frequential point of view, the image asperities in the spatial domain correspond to the presence of high frequencies. The implements roughness sub module have two stages, the first one aims to determine the deflections (i.e., residue) of pixels values of the original image, from the corresponding values which belong the image after applying smoothing. The smoothing task is performed using the average filter. The residue values will be considered as indicators for the existing local roughness in the image. The second stage is feature extraction, where a set of 5 statistical moments are determined for the determined histogram of residue component. ) Smoothing Image Using Average Filter In this stage, the average filter is applied on image. The average (mean) filter smooth the image data and it thus eliminates the noise. This filter performs spatial filtering on each individual pixel in an image in a square window surrounding each pixel, then the average filter computes the sum of all pixels in the filter window and then divides the sum by the number of pixels in the filter window, that is L / 2 L / 2 Im g ( x, y) Im g ( x + i, y + j) 2 L L / 2 L / 2 () Where L is the length of the square window. 2) Moments Determination In this stage, the central moments (moments about the mean) were used because they are more interesting than the moments about zero. The expression for the nth order moments about the mean is given by: Where, zi is a random variable indicating intensity, P(zi) is the histogram of the intensity levels in a region, L is the number of possible intensity levels and m is the mean (average) intensity. The first step towards extracting the feature vectors of the roughness attribute is the determination of histogram of residue which represents the difference between the original image and smoothed image. Since the using of histogram as it is as a feature vector; then the feature size with be high such that it will increase the computation cost of the next steps in the retrieval system. So to overcome this problem and reduce the size of the feature vector then instead of histogram elements values the moments (up to five orders) are calculated. Also, as smoothing component the mean, median and maximum were used. So, the size of roughness feature vector was taken 5, five features for each one. D. Neural Network Artificial neural network models have been studied for many years in the hope of achieving human-like performance in several fields such as image understanding. The networks are composed of many nonlinear computational elements operating in parallel and arranged in patterns reminiscent of biological neural networks. The network nodes, belong to adjacent layers, are connected, and their weights are typically adapted during the training phase to achieve high network performance [0]. In this research the multilayer feed forward artificial neural network had been trained to retrieve different kinds of tissues. For training purpose, the back-propagation algorithm was used. The architecture of the applied neural network consists of four layers: an input layer, two hidden layers and an output layer. The ANN input is 85 extracted texture features. E. Proposed System Content-Based Image Retrieval (CBIR) systems operate under the query-by-example (QBE) paradigm. As shown in figure (3) an example image is presented to the system and the user makes a query for images that are similar to the given example. Performance of CBIR systems is highly dependent on the properties of the example image. L ( n μn zi m) ρ( zi ) i 0 (2) L m i 0 z i * ρ( z ) i (3) Figure 3. The Proposed System Interface Firstly, a user uploads the query image through a user interface, and then its features will be extracted and assembled

4 in a feature vector, which is then compared with archived feature vectors those pre-extracted from the images stored in the image databases. Then a set of images that are very similar to the query image are retrieved and displayed. The operation of any typical image retrieval system passes through two main phases (enrollment and retrieving similar images). In the first phase the whole set of databases images are passed through the feature extraction module to extract the features vectors and then stored in feature vector databases. The collection of samples consists of 360 images stored in image database, and same number of feature vectors have been registered and saved in a dedicated database called feature vector database. In the second phase of system operation (i.e., retrieving phase), a set of test samples had been used to investigate the efficiency of the established retrieval system. In this phase, the tested image sample (i.e., 240) is passed through the feature extraction module to extract its feature vector and match this extracted feature vector with stored vectors in feature vector database. Finally, a list of similar images to a given sample is retrieved from image database. For purpose of performance evaluation some of the retrieval results for the conducted tests were used to determine the precision and recall rates of the proposed system. The block diagram for the proposed system is shown in Figure (4). Where, R is red, G is green, and B is blue color component, Gr is the gray. Also, the quantization level was taken (20) to quantize the intensity component, because fewer number of grey levels faster the computation when the statistics are applied. 2) Feature extraction and analysis: Eighty Five features were extracted, divided into three categories: forty features extracted from Co-occurrence matrix, eight features for each direction (horizontally, vertically, diagonal, inverse diagonal, all directions) and thirty features based on runlength matrix, ten features for each direction (horizontal, vertical and both direction) and fifteen features for roughness measure using five order moment (five for mean, five for median and five for maximum probability). 3) Training the neural network: In this stage, the multilayer feed forward artificial neural network had been trained, using back-propagation algorithm, a set of feature vectors are extracted from known tissues images (i.e., blood cells, breast tissues, GI issues, liver tissues), and saved in a feature vector database, then these vectors are used to train a feed forward neural network by adjusting its nodes weights and bias values using back-propagation algorithm. The collection of training samples consists of 360 tissues, and same number of feature vectors have been registered and saved in feature vector database. The computed weights and bias values of the trained network are also registered in the dedicated database called weight vector database. The block diagram for the training stage is shown in Figure (5). Figure 4. The Diagram of Proposed System In both system operation phases (i.e., enrollment and retrieval), the same image loading module is included and also the same feature extraction module is applied. So, the system is composed of the following main processes: ) Image loading: The type of image format used in this research is 24BMP format. Then the loaded RGB color image was transformed into gray images using the following equations: Gr R + G 3 + B (4) Figure 5. The diagram of training ANN In the training stage the network starts with a random set of weights and the training pattern is presented at the input layer. Then, the outputs of the network are evaluated and compared with the expected output vector, the error is calculated and the results are fed back from output layer to adjust weights. These steps are repeated for all the training set, and at each time the weights are adjusted. The training continues until the overall mean square error (MSE) between the desired and actual output becomes less than (or equal to) a predefined threshold value. When reaching this

5 error level then the network is considered well trained, and can be used to recognize the types on any unknown input tissue image. In the established system, the number of input nodes is set equal to the size of extracted texture features (i.e., 85). Two hidden layer were used, the number of hidden nodes were varied to find out the best smallest number of hidden nodes required to get best retrieving rate. Also, the best value of learning rate was investigated during the learning phase, taking into consideration that this parameter has significant effect on the training time and accuracy. The number of output nodes was taken 2, to represent the tissue class index in binary form. The sigmoid function was taken: out + e inp In general, to train the ANN many of the available data should be used, although it is not necessary to use them all. From the available training data a sufficient number of patterns are needed to be included in the training data set. The remaining data (i.e., 240) can be used to test the network to verify that the network can perform the desired mappings on the input vectors; which they have never been encountered during training. 4) Retrieving and displaying of similar images: For measuring the similarity between the query image and images in tissues database the neural network and similarity measurement based on Euclidian Distance were used. In this stage, the weights of the ANN are loaded from the database; these weights are pre-calculated by applying BPNN algorithm on the set of training patterns. Then, the extracted texture feature vector is passed forward through the ANN to get its class index. A feed-forward network can be viewed as a graphical representation of parametric function which takes a set of input values and maps them to a corresponding set of output values. The output nodes of ANN represent the image class index. In this work two output nodes were used, taking into consideration that the number of classes is four. So, the patterns of breast tissues, are mapped to class index (0, 0); patterns of GI tissues are mapped to class index (0, ); patterns of blood cells are mapped to class index (, 0); and patterns of liver tissues are mapped to class index (, ). For more accurate retrieving results after determining the class index of the query image by feed forward neural network, the similarity measurement were calculated between the query image class index and class index of images in the tissues database to retrieve most similar images to a given query image. The similarity measure is therefore: d ( Q, T ) N i 0 Q i III. RESULTS AND DISCUSSION The main stages of the established system are: feature extraction and retrieving using artificial neural network and similarity measurement. The feature extraction unit has two parameters, namely; Co-occurrence jump step and roughness T i (5) (6) window size. The parameters of this stage have considerable effects on the discriminating power of extracted feature vector. In the retrieval unit the artificial neural network has several parameters, namely; learning rate, number of hidden layer, and number of hidden nodes each one plays important role to achieve good precision rate. Two metrics for retrieval effectiveness were used; they are recall and precision. Recall signifies the relevant images in the database that are retrieved in response to a query. Precision is the proportion of the retrieved images that are relevant to the query. They defined as follows:- Retrieved related images Precision Total retrieved images Retrieved related images Recall Total related images * 00 % * 00 % A. The Effect of Co-occurrence Matrix Jump Step When using Co-occurrence matrix not only one jump step is adopted, several jump steps are tested to analyze each area. It is important to find the suitable value for it. In this work, the tested values are (, 2, and 3). The assignment of jump step value is very important to get more accurate texture analysis for the image. Table () shows the effect of using different jump steps on texture retrieval system precision and recall. TABLE. THE EFFECT OF CO-OCCURRENCE JUMP STEP ON SYSTEM PRECISION AND RECALL VALUE Jump step Precision Recall 00% 98% 2 00% 98% 3 95% 98% B. The Effect of Roughness Window Size In this set of tests the system precision and recall were determined for different window sizes to estimate the image surface roughness. Table (2) illustrates the effect of window size parameter on the precision and recall of the retrieval system based on textural features. TABLE2. THE EFFECT OF WINDOW SIZE ON SYSTEM PRECISION AND RECALL Window size Precision Recall 5 00% 98% 7 95% 98% 9 90% 98% C. The Effect of Number of Hidden Layers When using ANN, not only single architecture is adopted, several architectures were tested. In this paper, the effect of number of hidden layers had been investigated to define the effect of number of hidden layers on system efficiency and ANN learning time. When using single hidden layer, the best attained system precision was (85%), while for multi hidden (7) (8)

6 layers neural network the attained efficiency was increased. Also, the effect of number of hidden layers on learning time was tested, taking into consideration when adding new additional hidden layer additional time is required to train the neural network, see table (3) which illustrates the effect of this parameter on system precision and learning time. TABLE3. EFFECT OF NO. OF HIDDEN LAYERS ON SYSTEM PRECISION AND LEARNING TIME No. of layers Precision Learning time 85% 2 Minuet 2 00% 63 Minuet 3 00% 85 Minuet D. The Effect of Learning Rate on the Achieved Error Level This test was conducted to study the effect of learning rate on the achieved error level. Table (4) presents the results of the conducted test. During training stage of the system the maximum number of iterations is taken as stopping condition, the value of maximum iterations is set (500). TABLE4. THE EFFECT OF LEARNING RATE ON ACHIEVED ERROR LEVEL Learning Rate Achieved Error Level E. The Effect of Number of Hidden Nodes on Precision Rate and Learning Time Deciding the number of neurons in the hidden layers is a very important part of deciding your overall neural network architecture. In both layers the number of neurons in each of these hidden layers must be carefully considered. Table (5) shows the effect of the number of hidden nodes on precision rate and learning time of the ANN respectively. TABLE5. THE EFFECT OF NUMBER OF HIDDEN NODES ON SYSTEM PRECISION AND LEARNING TIME No. of first layer hidden nodes No. of second layer hidden nodes Precision 93% 97% 00% 00% 00% Learning Time (in Seconds) 40 Minuet 52 Minuet 63 Minuet 69 Minuet 76 Minuet IV. CONCLUSIONS The use of 85 features extracted from Co-occurrence, run length matrixes, and roughness measure can be utilized to describe the textural content of various tissues. A new idea based on taking advantage from using the histogram of residue between the original image and smoothed image to be used as indicator for roughness existence in an image. Using developed method in roughness feature extraction for textured images lead to more accurate results when combined with other traditional methods (i.e., Cooccurrence and run length) to overcome the weakness of these methods. The established texture image retrieval system gave better precision and recall rate (00, 98), when Co-occurrence jump taken, roughness window size taken 5, value of ANN hidden layers is set 2, the number of input nodes was 85, value of ANN first hidden nodes equal 65, value of ANN second hidden nodes equal 35, value of learning rate is set 0.2, the number of output nodes was 2, and the time required for training the neural network was 63 minutes. V. REFERENCES [] Greenspan, H., and Pinhas, A.T., Medical Image Categorization and Retrieval for PACS Using the GMM-KL Framework, IEEE Transactions on Information Technology in BioMedicine, vol., pp , [2] Muller, H., Michoux, N., Bandon, D., and Geissbuhler, A., A Review of Content Based Image Retrieval Systems in Medical Application Clinical Benefits and Future Directions, International Journal of Medical Informatics, vol. 73, no., pp. -23, [3] H. A. Abbass, An Evolutionary Artificial Neural Networks Approach for Breast Cancer Diagnosis, Artificial Intelligence in Medicine, Published by Scientific Literature Digital Library (CiteSeer.IST), vol. 25, no. 3, pp , [4] G. D. Magoulas, V. P. Plagianakos and M. N. Vrahatis, Improved Neural Network-based Interpretation of Colonoscopy Images Through On-line Learning and Evolution, Proceedings of European Symposium on Intelligent Technologies, Hybrid Systems and their Implementation on Smart Adaptive Systems, ISBN , pp , 200. [5] Lehmanna T., Gu lda M., Deselaersb T., Keysersb D., Schubertc H., Spitzera K.., Neyb H., and Wei B., Automatic Categorization of Medical Images for Content-Based Retrieval and Data Mining, Computerized Medical Imaging and Graphics, vol. 29, pp , [6] André B., Vercauteren T., Perchant A., Buchne, A. M., Wallace M. B., and Ayache, N., Endomicroscopic Image Retrieval and Classification Using Invariant Visual Features, Proceedings of the Sixth IEEE International Symposium on Biomedical Imaging (ISBI), pp , [7] Mehta R., Mishra N., Sharma S., Color - Texture Based Image Retrieval System, International Journal of Computer Applications, vol. 24, no. 5, pp , 20. [8] Suganya, R. and S. Rajaram, Content Based Image Retrieval of Ultrasound Liver Diseases Based on Hybrid Approach, American Journal of Applied Sciences, ISSN , vol. 6, pp , 202. [9] Mohanty, A., Beberta, S., and Lenka, S., Classifying Benign and Malignant Mass using GLCM and GLRLM based Texture Features from Mammogram, International Journal of Engineering Research and Applications (IJERA), ISSN: , vol., issue 3, pp , 200. [0] M. L. Antonie, O. R. Zaiane and A. Coman, "Application of data mining techniques for medical image classification", Proceedings Of second international workshop on multimedia data mining, pp , san Francisco, USA, 200