COMPARISON OF SOME CONTENT-BASED IMAGE RETRIEVAL SYSTEMS WITH ROCK TEXTURE IMAGES

Transcription

1 COMPARISON OF SOME CONTENT-BASED IMAGE RETRIEVAL SYSTEMS WITH ROCK TEXTURE IMAGES Leena Lepistö 1, Iivari Kunttu 1, Jorma Autio 2, and Ari Visa 1 1 Tampere University of Technology, Institute of Signal Processing P.O. Box 553, FIN Tampere, Finland 2 Saanio & Riekkola Consulting Engineers, Laulukuja 4 FIN Helsinki, Finland Texture is commonly used feature in most of the content-based image retrieval systems. This texture retrieval ability can be also applied to rock texture. The retrieval of the rock texture is a demanding task because of special character of rock. In this paper some existing contentbased image retrieval systems are tested with a sample set representing clearly different rock images. The recall ability of these systems is measured based on the retrieval experiments. Texture retrieval has also been made using some well-known classifying features, which are extracted from the textures. The extracted features are based on second order statistics (cooccurrence matrix) and texture directionality. These features proved to be powerful in the sample texture retrieval. 1. INTRODUCTION Texture analysis and classification is an usual task in pattern recognition. Textures are important for example in many quality control problems in timber, fiber, paper and printing industries. Another field of texture analysis is aerial photography and remote sensing [11]. Definition of textures is not simple, because they are not always regular and homogenous. Usually texture is defined as a structure, which is composed of many more or less ordered similar elements or patterns [14]. Textures can be characterised and classified in many different ways. Different methods has been developed for that purpose. Classification of rock textures is a very demanding task, because the rock texture is not homogenous. There are differences in colour, granularity and other texture properties within the same rock texture type. Also directionality is very strong in certain rock types. In many cases differences between textures are visible but it is difficult to observe them in automatic way. Recognition of rock has not been an object of active research in recent years although there have been some studies in this field. In the approach of Crida and de Jager [2] a rock recognition system was developed to measure rock size distribution and fragmentation. The results of practical experiments of the system were encouraging. Autio et al [1] developed an analysis and classification procedure for rock textures. Tools for classification were in that research co-occurrence matrix and Hough transform. In addition to colour and shape, texture is an important feature in image retrieval. In the retrieval systems texture similarity is used in distinguishing between the areas of images with similar colour [3],[12]. Texture retrieval in general has been researched a lot [6],[7],[8]. These studies use Wold features, Gabor Wavelets and filters in texture analysis. In most of these studies the considered textures have been homogenous and deterministic. Therefore, the methods which are presented in them cannot directly be applied to the rock image retrieval. During the last few years, development in the field of content-based image retrieval has been rapid. As a result of this research work, some prototype retrieval systems have been

2 Figure 1. Examples of seven different rock image classes. developed [13]. There has been introduced some commercial content-based image retrieval applications as well [4]. There has been published several papers about current content-based retrieval systems [3], [12]. These systems are suitable for retrieval of images from large databases. The results, which are usually published, concern retrieval of photographic images. On the other hand, there has been made only a few practical comparisons of the systems using texture images. In order to be able to manage large databases of rock images, it is necessary to have a automatic system for retrieval of them. In this work our main purpose is to clarify how well some existing content-based image retrieval systems [3], [12] perform queries using rock texture images. In the section two we make experiments to measure recall ability of the systems in case of these images. In that way we are indicating how suitable these systems are for classification of the rock textures. Based on results of the experiments, we select and extract some rock texture features in the section three. These features are used to classify different rock textures. Classification ability of the existing retrieval systems and our own features are discussed in the section four. 2. CONTENT-BASED IMAGE RETRIEVAL OF ROCK TEXTURES The growth of digital imaging has caused the need for efficient storage and retrieval of images. Therefore, content-based image retrieval has been the subject of intense research work. The goal of this research has been to develop a method, which finds from a large database a desired image based on the content of image. Mathematical measures of color, shape and texture are typically used features to characterize image content [3]. Features are usually automatically extracted from images and stored in the image database. Search efficiency is often improved by applying tree-structures in indexing [12]. The queries are made by matching appropriate features of prototype (query) image and stored images. There has been developed several methods and systems for content-based image retrieval. In [3] and [12] there are introduced current varieties of different retrieval methods and applications. The majority of the applications are prototype systems, which are the result of academic research, but there are some commercial systems as well. We selected three different kinds of systems for testing. The selection criteria were availability of the system, ability for handling large databases and different principles in retrieval. MUVIS [13] is a system for content-based image indexing and retrieval. It is a prototype system developed in

3 Percent of the same texture within nine best matches IMatch QBIC MUVIS Rock Texture class Figure 2. Query results of three selected systems. an academic research project. It combines color, texture and shape features of images in retrieval. In texture analysis MUVIS uses among other descriptors second order statistical measures. QBIC [4] is developed by IBM and it is the first and maybe the best-known commercially available system for image retrieval. Also QBIC offers retrieval by combinations of color, texture and shape, which are extracted from each image and stored to the database. Database indexing is made in QBIC using a R*-tree. In the texture retrieval QBIC uses mathematical representations of coarseness, contrast and directionality. An another commercial application is IMatch developed by Mario M. Westphal. The system is based on the databases, which contain the feature vectors. IMatch uses in image identification color and shape features, which can be combined using fuzzy logic. When we make a query, we get a set of database images, which are closest to the query image. The recall ability of a retrieval system can be measured by indicating, how many of the result set images are similar to the query image. In testing we used a testing database, which consisted of 168 rock texture samples. The size of each sample image was 500x500 pixels. The samples represented seven different rock textures (figure 1), and there were 24 samples from each texture class. The samples in the figure 1 indicate that in some classes (for example class 4) the samples are strongly non-homogenous. The queries were made for each system using prototype samples of each texture. The idea was to clarify, how many similar texture samples as the query image the system was able to find from the database. In this experiment, the recall was nine best matches of each query and the number of the correct matches among them was counted. In the figure 2 there are the query results of three selected systems for each texture. The average results of MUVIS, QBIC and IMatch were 76.3%, 61.6% and 52.3% respectively. The level of obtained results can be inspected using a chi square (χ 2 ) calculation [9], in which the difference between observed and expected frequencies f 0 and f e gives the value of chi square:

4 2 ( f f ) 0 c 2 χ = (1) f e The expected frequency was obtained using a hypergeometric distribution [10]. The calculation indicated that there is a significant difference between obtained results and expectation value (α>>0.01). The results indicate variety in the texture classification ability of the systems. Also different textures act in different ways in the comparison. The queries using IMatch were not very successful. Reason for that may be that the system has not been developed primarily for texture classification. IMatch made its best result when using fuzzy combination of shape and color features. The query results of QBIC are in most texture classes better. Especially in case of the textures, which have strong directionality QBIC has worked better than IMatch. Ability for rock texture classification is the best by MUVIS system, which uses measures calculated from the co-occurrence matrices in the texture retrieval. 3. FEATURE EXTRACTION Testing part of our work indicated that some of the existing content-based image retrieval systems are capable of classifying the rock textures. However, systems are not developed to work with special types of textures. They do not consider the special character of rock texture. Therefore additional research had to be done in the area of the texture features. The cooccurrence matrix proved to be powerful in texture classification of MUVIS. Therefore, the co-occurrence matrix was selected to the basis of our own feature extraction. In addition, the texture directionality was also considered as a distinguishing feature. In this section we are testing how well these features distinguish between the rock texture samples Textural features One of the second order statistical measures is the co-occurrence matrix [14], which is based on the spatial gray level dependence. The approach is commonly used in the texture analysis and classification. It is based on the estimation of the second order joint probability density functions g(i,j d,θ). Each of them is the probability of going from gray level i to gray level j, when the intersample spacing is d and the direction is Θ. The probabilities create the cooccurrence matrix M(i,j d,θ). It is possible to extract several texture features from the matrix, including contrast, entropy and energy. Contrast = i j 2 ( i j) M ( i, j d, Θ) Energy = M ( i, j d, Θ) Entropy = i j i j 2 M ( i, j d, Θ) log M ( i, j d, Θ) We calculated these measures for each texture. The results varied a little when using different values for d. In our experiments, we used value five for d. The results showed that these measures are able to distinguish between most of the texture samples. Figure 3 shows how contrast and entropy can distinguish between the samples. In addition to these textural features, we used also some statistical features to characterize the texture images. These features were mean gray level and standard deviation of the texture image. Contrast, entropy, energy, mean gray level and standard deviation of the texture image form a feature vector for each image. Image database is indexed using these feature vectors. In the retrieval the nearest feature vectors for each query image can be found using the Euclidean distance [5]. (2) (3) (4)

5 Entropy class class 2 class 3 class 4 7 class 5 class 6 class Contrast Figure 3. Contrast-entropy map of the texture samples in the testing database. Figure 4. Directional masks for eight directions Directionality Certain rock textures have strong directionality, which can be used as a distinguishing feature between the rock textures. The directionality can be measured using the Hough transform or a directional filtering technique [1]. The Hough transform [5] is a method for detecting directions in images, and it has proved to be powerful also in the rock texture characterization [1]. The idea of the directional filtering is close to the method used in the edge detection [5], in which masks (figure 4) are used to find the edges in certain direction in image. In our approach there are eight directions and a mask is made for each direction (figure 4). The texture is measured by a set of these directional masks, which glide across the texture image. In both methods the directions form a distribution that is called the directional histogram. Directional histograms can be used in the image database indexing and retrieval. The distance D between two histogram h 1 and h 2, can be calculated in the following way: 8 D = h1 ( i) h2 ( i) (5) i= Retrieval Experiments We tested the features presented in the sections 3.1 and 3.2 with the texture samples in the testing database. The testing principle was similar to the testing method in the section 2: A prototype image from each texture class was used as a query image. The distances between the query image and the images in the testing database were calculated. After that the performance of the retrieval was measured by counting the images of the same class as the

6 Percent of the same texture within nine best matches Textural features Directional measurement based on filtering masks Directional measurement based on hough transform Combination Rock Texture Class Figure 5. Results of retrieval experiments in each texture class. The average results of textural features, filtering masks, Hough transforms, and their combination were 65.1%, 68.3%, 63.5%, and 81.0% respectively. query image within nine best matches. We made the retrieval experiments in three ways, which were the textural features (presented in the section 3.1), directional histograms based on filtering technique and Hough transform. The average results of the queries using these methods were 65.1%, 68.3% and 63.5%, respectively. Because these methods measure only some particular features from the texture images, it is practical to combine the features in the retrieval. Therefore, we made also a query using a feature combination by adding the normalized distances together. In this case the average retrieval result was 81.0 %. The query results in each texture class are presented in figure 5. The computing time of the retrieval using all the prototypes was less than one second using Matlab on a PC with 804 MHz Pentium III CPU and 256 MB primary memory. 4. DISCUSSION Experiments using the existing content-based image retrieval systems in the section two indicated that the systems are not very suitable for classification of the rock textures, which is quite demanding task. Therefore, some additional research in the field of the feature selection and extraction has to be made. Our experiments showed that the statistical measures obtained from the co-occurrence matrix have good classification ability for the rock images used in this study. The proposed directionality measurement proved to be useful in classification. The experiments indicated that the directional histogram can be used as a classifying feature also for the textures, which do not have strong directionality. Because of the special character of the rock, the statistical measures are not always constant within the same texture class. This can be problematic when different textures must be distinguished. On the other hand, this ability can also be utilized when researching the variations within the same texture class. The performance of the directionality measurement is dependent on the resolution of the texture image, because the size of the mask should be

7 clearly larger than the grain size of the rock. If the image resolution is too high, the directionality cannot be measured properly. This work indicated that there is a great potential in the feature analysis in the field of rock texture classification. The texture features proposed in this work proved to be effective in classification of the rock texture samples. In case of the rock textures these methods can give even better results compared to the existing content-based image retrieval systems. In comparison to the best of the existing system, MUVIS (78.6%), our approach was better (81.0%). The difference is relatively small. However, in case of the demanding, nonhomogenous textures (e.g. class 4), our method was significantly better. Therefore, our approach can also be applied to the demanding rock textures. The features in our approach are simple and they are easy to calculate. They are very suitable for the image retrieval from large and distributed databases. Additional distinguishing texture features may improve the retrieval and they offer possibility for the further research at this area. 5. ACKNOWLEDGMENT We would like to thank Saanio & Riekkola Oy and Technology Development Centre of Finland (TEKES s grant 40397/01) for financial support. 6. REFERENCES 1. Autio J., Lukkarinen, S., Rantanen, L., and Visa, A. The Classification and Characterisation of Rock Using Texture Analysis by Co-occurrence Matrices and the Hough Transform, International Symposium Applications in Geology, Belgium, May , Crida, R., C., and de Jager, G. Rock Recognition Using Feature Classification, GOMSIG-94, Proceedings of the Symposium on 1994 IEEE South Africa, 1994, Eakins, J. P., and Graham, M. E.,Content-based Image Retrieval, A report to the JISC Technology Applications Programme, Institute of Image Data Research, University of Northumbria at Newcastle, January 1999, Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D. and Yanker, P. Query by Image and Video Content: The QBIC system, IEEE Computer, vol. 28, September 1995, Gonzales, R. C. and Woods, R. E. Digital Image Processing, Addison Wesley, United States of America, Levienaise-Obadia, B., Christmas, W. and Kittler, J. Defining quantisation strategies and a perceptual similarity measure for texture-based annotation and retrieval, Proceedings of 15 th International Conference on Pattern Recognition, vol. 3, 2000, Liu, F. and Picard, R. W. Periodicity, Directionality, and Randomness: Wold Features for Image Modeling and Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no 7, July 1996, Manjunath, B. S. and Ma, W. Y. Texture Features for Browsing and Retrieval of Image Data, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.18, no 8, August 1996, Manning, C. D. and Schutze, H. Foundations of Statistical Natural Language Processing, The MIT Press, London, Milton, J. S. and Arnold, J. C. Introduction to Probability and Statistics. Principles and Applications for Engineering and the Computing Sciences, Mc Graw-Hill, New York, 1995.

8 11. Rhrauer, H., Seidel, K. and Datcu, M. Multi-scale indices for content-based image retrieval, IEEE International, vol. 5, 1999, Smeulders, A. W. M., Worring, M., Santini, S. Gupta, A. and Jain, R. Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, December 2000, Trimeche, M., Cheich, F. A., Gabbouj, M. and Cramariuc, B. Content-Based Description of Images for Retrieval in Large Databases: MUVIS, X European Signal Processing Conference, Eusipco-2000, Finland, September 5-8, Visa, A. Texture Classification and Segmentation Based on Neutral Network Methods, Helsinki University of Technology, Finland, 1990.