Automatic Texture Segmentation for Texture-based Image Retrieval

Automatic Texture Segmentation for Texture-based Image Retrieval Ying Liu, Xiaofang Zhou School of ITEE, The University of Queensland, Queensland, 4072, Australia liuy@itee.uq.edu.au, zxf@itee.uq.edu.au Abstract Texture-segmentation is the crucial initial step for texture-based image retrieval. Texture is the main difficulty faced to a segmentation method. Many image segmentation algorithms either can t handle texture properly or can t obtain texture features directly during segmentation which can be used for retrieval purpose. This paper describes an automatic texture segmentation algorithm based on a set of features derived from wavelet domain, which are effective in texture description for retrieval purpose. Simulation results show that the proposed algorithm can efficiently capture the textured regions in arbitrary images, with the features of each region extracted as well. The features of each textured region can be directly used to index image database with applications as texture-based image retrieval. 1. Introduction Region-based image retrieval (RBIR) systems retrieve images on the basis of automatically-derived features such as color, texture and shape from interested regions in an image. Automatic image segmentation is a crucial initial step before performing RBIR. A variety of techniques have been proposed in the past, including stochastic model based approaches [1], curve evolution [2], energy diffusion[3], region growing[4], and graph partitioning[5]. Quantitative evaluation methods have also been suggested [6]. Among contents based features, texture is a fundamental feature which provides significant information for scene interpretation and image classification [7], hence texture-based image retrieval is an important topic. Textures can be defined as homogeneous patterns or spatial arrangements of pixels that regional intensity or color alone does not sufficiently describe [8]. Texture is the main difficulty faced to a segmentation method [9]. There are many existing techniques for the segmentation of images that contains only homogeneous color regions, such as direct clustering methods in color space [10]. But natural scenes are rich in both color and texture. Most texture segmentation algorithms [1][3] requires the estimation of texture model parameters, which is proved to be difficult [11]. To solve this problem, a region growing method based on image color space quantization, named JSEG, is presented in [12]. Experiments show that JSEG provides good segmentation results on a variety of images. In [11], a method based on local homogeneity analysis is presented. This method is similar to JSEG but with simpler criterion used. The above algorithms though can catch textured regions, the segmentation is not based on texture features. Instead, they make use of other information such as color and spatial arrangement to handle texture. For texturebased image retrieval purpose, we need texture features which can be used for image retrieval. For this, one way is to use the above mentioned segmentation algorithm, and extract texture feature of the segmented regions after segmentation. Alternatively, we can implement texture segmentation in such a way that the features used for segmentation can be directly used for retrieval purpose. The latter is simpler as feature extraction after segmentation is not necessary, and more efficient as segmentation using salient texture features could handle texture more properly. In this paper, we proposed an automatic texture segmentation method based on Discrete Wavelet Transform (DWT). This method can well catch textured regions of arbitrary images. In addition, feature of each segmented region are obtained during segmentation and can be used directly for the purpose of texture-based image retrieval. The remainder of this paper is organized as follows. Section 2 describes the related work. Section 3 introduces the proposed algorithm. Section 4 gives the experimental results. Finally, section 5 concludes this paper. 2. Related work

A wide range of natural images can be considered as a mosaic of different textured regions, each textured region of the images in the database can be characterized and added to a texture-based index of the database for retrieval purpose. To efficiently handle texture, salient texture features can be used in texture segmentation. Gabor filters are designed to respond to different spatial frequencies and have been applied to texture segmentation [13]. However, it is computationally expensive as a large combination of parameters are used. Wavelet has been proved to be a promising alternative of Gabor filters for texture segmentation purpose [14]. In [15], a quad-tree decomposition based on wavelet features is proposed. The approach is top-down in a way that it starts from the whole image, which is the root node of the decomposition tree. Using a quad-tree decomposition, the algorithm extracts texture feature from spatial blocks at a hierarchy of scales in each image. Finally, homogeneous blocks of texture are extracted which can be used in a database index. In the algorithm in [15], at each level of the tree, texture content of the parent block is compared with the children block, to judge if it is a texture. The problem is that before the algorithm stops, the parent block is not a texture, and the children blocks might not be texture either. As we know, description of non-textured images is more difficult than that of textured images. Hence, to properly measure the similarity between the parent block and the children blocks, more complicated feature extraction algorithm is required. In addition, this algorithm stops at low resolution textures, and can t obtain high-resolution textures which users might be interested in. For example, it can catch an arrangement of oranges which is a texture. But what if we are interested in the skin of an orange? To solve these problems, in this paper, we proposed a bottom-up texture segmentation algorithm, which starts from small homogeneous blocks and grow up gradually to find textured regions. Bottom-up texture segmentation can be pixel-wise or block-wise. Pixel-wise segmentation schemes evaluate the texture features in a neighbourhood surrounding each pixel of the image. The advantage of pixel-wise segmentation lies in the removal of blockyness at region boundaries. However, the computation load is heavier. As image retrieval system does not require exact boundary of the segmented regions [15], block-wised segmentation is often chosen since it is much faster [16]. The proposed block-wise texture segmentation algorithm is based on a set of features derived from wavelet domain. This set of features has been proved to be efficient for texture description. The algorithm can well capture textured regions of arbitrary images. Moreover, the features of each region can be directly used to index image database for retrieval purpose. 3. The proposed algorithm In the proposed system, an image is first partitioned into blocks of 4*4 pixels, then a 2-level wavelet transform using 4-tap Daubechies filter is applied to each block. This produces 7 sub-bands as shown in Figure 1. Features in wavelet domain have been shown to be effective in texture representation for retrieval purpose [17]. In our experiments, mean, and variance of each of the 7 sub-bands are extracted to form feature vector of 14 terms. Let S w and S h be the width and height of a certain subband in wavelet domain, mean (m) and variance (var) of the coefficients in this sub-band can be calculated as: ( S h 1) ( S w 1) m = [ C ( i, j )] /( S * S ) (1) where var C f = i = 0 ( i, j) ( j = 0 ( S h 1) ( S w 1) i = 0 f j = 0 w 2 [ C ( i, j) m ] (2) f is the transform coefficient. With features of each block available, we can classify all blocks into different classes. The block at the top-left corner belongs to the 1 st class, and its feature is the initial feature of this class. We then scan all blocks one by one. For block Bc, suppose the blocks previously scanned belong to m different classes, if Bc belongs to any of the m classes, we update the feature of this class (each term of the updated feature is simply calculated as the average of the corresponding term of all feature vectors in the class so far); if not, add a new class as the (m+1) th class, and feature of Bc is the initial feature of the new class. In order to determine which class Bc belongs to, we compute its Euclidean distance d i ( i = 1,2,..., m ) to each of the m classes obtained so far. Euclidean distance between two l-dimensional vectors X and Y is defined as: l 1 i = 0 2 d = ( x y ) (3) If the smallest distance i d j i h falls below the given threshold Thr, then Bc belongs to class j. If all distances are above Thr, Bc belongs to a new class. Note that heavier weight is assigned to var when we compute Euclidean distance of two feature vectors. Experiments show that a weights ratio of 9:1(for var and m) is proper. Finally we obtain N classes each containing Mi (i=1,,n) blocks, and we have the features of each classes obtained. Note that one class might contain more than one regions which are disconnected.

Results show that this texture segmentation algorithm is effective in capturing the textured regions of an image. Examples are given in Figure 2. There are some tiny isolated regions existing after segmentation. In most cases, these regions are meaningless. We have not implemented post-processing of these regions yet since these errors are often not significant. They can be merged with larger visually close regions if any, or ignored. Figure 1. 7 sub-bands in a 2-level DWT 4. Adjusting parameters 4.1. Thr Obviously, the higher the Thr, the smaller the number of classes we can get. If Thr is too low, one textured region we are interested in might be segmented into different parts. On the other hand, if Thr is too high, different textured regions might be classified into same part. Experiments show that Thr between 10 and 20 is proper for many images. 4.2 Eng_High 0 1 2 3 With segmentation above, we found that regions with higher variance are easy to be segmented into different parts, as in Figure 3(a). Increasing Thr can to certain degree relieve this problem. But if Thr is too high, some different regions might be classified into same class, as shown in Figure 3(b). Hence, we consider using various Thr instead of uniform Thr for different blocks. According to their texture property, regions of an image can be roughly classified as flat regions which contain little intensity changes, textured regions which are composed of repetitive patterns, and non-textured regions. High frequency coefficients in wavelet domain correspond to sharp intensity changes in the image, like 4 5 6 edges and boundaries. Sub-bands 4, 5, 6 in Figure 1 are the three high frequency sub-bands. We define a parameter Eng_High, which is the percentage of energy of coefficients in sub-bands 4,5,6, relative to the total coefficient energy across all 7 sub-bands. To study the relationship between Eng_High and the texture property of images, we randomly chose 100 textured images and 100 non-textured images of different sizes, and calculated their Eng_High values. Figure 4 shows the histograms of Eng_High for the two types of images. It is shown that the two histograms separate around the decision threshold 0.03. For flat images which contain very little high frequency information, their Eng_High values are very small. We found that a threshold around 0.005 is proper to separate flat images from textured and non-textured images. Figure 5 shows a few images with their Eng_High values given. We found that the above property is invariant to image sizes, and the decision threshold varies with the DWT decomposition level we chose. Previous work proved that larger threshold should be set for blocks with larger variance during the segmentation procedure [15]. Textured regions contain more intensity changes than non-textured regions and hence higher Thr should be used. In our algorithm, experimentally, different Thr are set to blocks with different Eng_High as below: if (Eng_High<0.005) Thr=Thr 0 ; //flat regions else if(eng_high<0.03) Thr=Thr 0 *1.7; //non-textured regions else Thr=Thr 0 *2.5; //textured regions Number of Images 20 16 12 8 4 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 textured Eng_High non-textured Figure 4. The histogram of Eng_High over 100 textured and 100 non-textured images With higher threshold for blocks with more high frequency information, textured regions which were segmented into different parts before can be recognized as

one region. In addition, the number of classes is reduced and hence computational load is less. Figure 3(c) shows an example. 5. Image retrieval based on texture content With the proposed texture segmentation algorithm, we can capture the textured regions of arbitrary images. In addition, texture features of the segmented regions are obtained. These features can be used directly to index the image database with applications as texture-based image retrieval. Our testing database is a collection of 250 arbitrary images with different sizes and contents. Texture segmentation is applied to each image in the database, and features of each region are stored. The feature of the query region is calculated using the same way as described in Section 2. The images containing region(s) similar to the query region are selected as retrieval results. Figure 6 shows a retrieval example with the top 8 selected images listed. 6. Conclusions This paper presents a block-wise texture segmentation algorithm, which automatically segment an image into textured regions. Meanwhile, texture features of each region are extracted which can be used as indexing of image databases. We expect the proposed algorithm to allow efficient image retrieval on the bases of texture content. In our algorithm, the feature of the segmented region is calculated as the average of the features of all the blocks it contains. To describe a region more efficiently, information of the entire region could be used. We will work on this later. In addition, we will extend the system to large real-world image database and apply index to facilitate fast retrieval in our future work. Acknowledgement This work is partly completed when the first author was attached to the School of Computing, National University of Singapore, under project R-252-000-015-112/303. 7. References [1] S.Belongie, C. Carson, H. Greenspan, and J. Malik, Colorand texture-based image segmentation using EM and its application to content-based image retrieval, Proc. of ICCV, 1998, pp675-682. [2] H.Feng, D.A. Castanon, and W.C. Karl, A curve evolution approach for image segmentation using adaptive flows, Proc. of ICCV, 2001. [3] W.Y.Ma and B.S. majunath, Edge flow: a framework of boundary detection and image segmentation, Proc. of CVPR, 1997, pp744-749. [4] R.Adams and L.Bischof, Seeded region growing, IEEE Trans. On Pattern Analysis and Machine Intelligence (PAMI), 16(6), 1994, pp641-647. [5] J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE Trans. On Pattern Analysis and Machine Intelligence (PAMI), 22(8), Aug. 2000, pp888-905. [6] M. Borsotti, P.Campadelli, and R. Schettini, Quantitative evaluation of color image segmentation results, Pattern Recognition letters, vol.19, no.8, 1998, pp741-748. [7] Yong Man Ro, Matching pursuit: contents featuring and image indexing, SPIE, Vol. 3527, pp89-92, 1998. [8] J. R. Smith, Integrated Spatial Feature Image Systems: Retrieval, Analysis and Compression, Ph.D. thesis, Graduate School of Arts and Sciences, Columbia University, 1997. [9] Yining Deng, B.S. Manjunath, Unsupervised segmentation of color-texture regions in images and video, Unsupervised segmentation of color-texture regions in images and video (PAMI), vol 23, No.8, Aug., 2001, pp800-810. [10] D. Comaniciu and P.Meer, Robust analysis of feature spaces:color image segmentation, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition,1997, pp750-755. [11] Feng Jing, Mingling Li, Hongjiang Zhang, Bo Zhang, Unsupervised image segmentation using local homogeneity analysis, Proc. IEEE Inter. Symposium on Circuits and Systems, 2003. [12] Yining Deng, B.S.Manjunath and Hyundoo Shin, Color Image Segmentation, Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, CVPR 99, Fort Collins, CO, vol.2, June 1999, pp446-451. [13] Dunn, D. and Higgins, W.E., Optimal Gabor filters for texture segmentation, IEEE Trans. on Image Processing, Vol4., No.7,1995, pp947-964. [14] Mausumi Acharyya and Malay K. Kundu, Adaptive basis selection for multi texture segmentation by M-band wavelet packet frames, Inter. Conf. on Image Processing, 2001. pp.622-625. [15] J.R.Smith and Shih-Fu Chang, Quad-tree segmentation for texture-based image query, Proc. of the 2 nd Annual ACM Multimedia Conference, San Francisco, Ca., Oct. 1994. [16] Jia Li, James Ze Wang, and Gio Wiederhold, Classification of textured and non-textured images using region segmentation, Inter. Conf. on Image Processing, Sep.2001, pp 754-757. [17] J.R.Smith, S.-F. Chang, Transform features for texture classification and discrimination in large image databases, Inter. Conference on Image Processing, vol.3, 1994, pp407-411.

Figure 2. Examples of texture segmentation (Left: Original images, Right: Segmentation results) Uniform, Thr=10, N=54 Uniform Thr=20, N=9 Various Thr (Thr 0 =10), N=6 (a) (b) (c) Figure 3. Segmentation results with uniform Thr and various Thr

Eng_High=0.260 Eng_High=0.025 Eng_High=0.001 Figure 5. Textured and non-textured images with Eng_High Query region query image Figure 6. Texture-based image retrieval