Color and Texture Feature For Content Based Image Retrieval

International Journal of Digital Content Technology and its Applications Color and Texture Feature For Content Based Image Retrieval 1 Jianhua Wu, 2 Zhaorong Wei, 3 Youli Chang 1, First Author.*2,3Corresponding Author Institute of Electronic and Information Engineering, Key Laboratory of Medical Image Computing (Northeastern University), Ministry of Education, wujianhua@mail.neu.edu.cn weizhaorong@yahoo.com.cn changyouli@yahoo.com.cn doi: 10.4156/jdcta.vol4.issue3.4 Abstract Content based image retrieval (CBIR) has been one of the most important research areas in computer science for the last decade. A retrieval method which combines color and texture feature is proposed in this paper. According to the characteristic of the image texture, we can represent the information of texture by Dual-Tree Complex Wavelet (DT-CWT) transform and rotated wavelet filter (RWF). We choose the color histogram in RGB and HSV color space as the color feature. The experiment results show that this method is more efficient than the traditional CBIR method based on the single visual feature and other methods combining color and texture. Keywords: Content based image retrieval(cbir), Dual-Tree Complex Wavelet Transform(DTVWT), Rotate Wavelet Filter(RWF) 1. Introduction With the development of the multimedia network technology and the increase of image data, the retrieval of image data based on pictorial queries is becoming an interesting and challenging problem. CBIR has become a hot spot of technical research. In a CBIR system, it extracts the visual features from images and uses them to index images, such as color feature, texture feature and shape feature. As long as the content of an image does not change, the extracted features are always consistent. Color feature is one of the most widely used features in low-level feature [1]. Compared with shape feature and texture feature, Color feature shows better stability and is more insensitive to the rotation and zoom of image. Color histogram [2] is widely used to represent color feature. In this paper, histogram-based search method is investigated in two different color spaces. In addition, in CBIR system, the texture also plays an important role in computer vision and pattern recognition, especially in describing the content of image. Texture feature currently used in the CBIR system are mainly derived from Gabor wavelets [3], the conventional discrete wavelet transform [4] (DWT) and discrete wavelet frames. In this paper, we use DT-CWT for decomposing an image into six bandpass subimages that are strongly oriented at six different angles and two lowpass subimges, and then calculate the means and standard deviations of these subimages and form the feature vector. The organization of the paper is as follows. In Section 2, a brief review of color histogram method is given. We compare the retrieval results in the two different color spaces, and choose the better color space. The texture feature extraction algorithm is explained in Section 3. Efficiency comparisons with other feature extraction methods are performed and results are listed in Section 4. Section 5 contains the discussion of the results. 2. Color feature A color histogram refers to the probability mass function of the image intensities. This is extended for color images to capture the joint probabilities of the intensities of the three color channels. More formally, the color histogram is defined by h NProb( Aa, Bb, C ) (1) A, B, C c 43

Color and Texture Feature For Content Based Image Retrieval Jianhua Wu, Zhaorong Wei, Youli Chang where A, B and C represent the three color channels (R,G,B or H,S,V) and N is the number of pixels in the image. Because the computer represents color image with up to 224 colors, this process requires substantial quantization of the color space. The histograms of color image are composed of 4D vectors. This makes the histograms of color image very difficult to visualize. In this paper, we adopt a non-uniform quantization method based on HSV color space, and compare the results in HSV space and RGB color space. In the RGB color space, we apply a color quantization method using 256 colors (8 levels for each channel). The histogram of the image is calculated by equation (1) and stored in a feature vector database. For the HSV color space, we map the original image into the HSV color space. Because the color resolution of human vision system is limited. In this paper, a color quantization is done using 72 colors (8 levels for H channel, 3 levels for S channel and 3 levels for V channel). 0, H [ 30,30] 1, H [30,90] 0, S [0,0.25] 2, H [90,150] 0, I [0,0.3] 1, S [0.25,0.45] H S V 1, I [0.3,0.8] (2) 3, H [150,210] 2, S [0.45,0.65] 4, H [210,270] 3, S [0.65,1] 2, I [0.8,1] 5, H [270,330] Then, We use the equation (2) to construct 1D feature vector. G HQ Q SQ V (3) s v v where Qs and Qv are the quantization levels of component S and V. In this paper, Q s 3, Qv 3 then G 9 H 3S V (4) We used the precision X recall graph and the L2 (Euclidean) distance to evaluate the performance of the histogram in RGB [12] and HSV color spaces. The results are showed in Figure1. The query images were selected from a image subset of the COREL database. Those figs showed that the effect of HSV+RGB was better than the others. So we take the integrated histogram as the color feature. 1 0.95 Precision X recall graph for African RGB HSV RGB+HSV 0.7 0.65 Precision X recall graph for Architectural RGB HSV RGB+HSV 0.9 0.6 0.85 0.55 Precision 0.8 Precision 0.5 0.75 0.45 0.7 0.4 0.65 0.35 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Recall (a) African 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 Recall (b) Buildings Figure 1 Precision X recall graph 3. Texture Feature Wavelet has been widely used in image processing application including compression, enhancement, reconstruction and image analysis. A wavelet transformation provides a multiscale decomposition of the image data. Manjunath and Ma have used texture feature derived from Gabor wavelet coefficients [3]. Do and Vetterli proposed wavelet based texture retrieval 44

International Journal of Digital Content Technology and its Applications using generalized Gaussion density [4]. Manjunath and Ma have done extensive experiments on a large set of texture images and shown that the retrieval result using the Gabor wavelet was better than using conventional wavelet [12]. Even though the Gabor wavelet based method gives better retrieval performance, but it has two fatal disadvantages: (1) Gabor function do not form an orthogonal basis set and hence the representation is not compact; (2) Computational time required for feature extraction is quite high, which limits the retrieval speed. 3.1 RWF To overcome the drawbacks of Gabor wavelet, Sastry proposed a modified Gabor function for content based image retrieval [5]. Ju Han also proposed a rotation-invariant and scaleinvariant Gabor feature [6]. The two methods both improve the retrieval speed at the cost of reducing the retrieval performance. 2D discrete wavelet transform (2D-DWT) is a useful tool in image analysis. Its computational complexity is lower than Gabor wavelet. But 2D-DWT only has four directions (0 0,90 0,45 0,135 0 ). This lack of directional selectivity limits the application of 2D-DWT in image retrieval. Kokare and Biswas have used rotated discrete wavelet filters that are obtained by rotating the standard 2D-DWT filters (RWF) [7]. Computational complexity is same as that of the standard 2D-DWT filters decomposition, if both are implemented in 2D frequency domain. The method for designing the 2D rotated wavelet filters is showed in [7]. Figure 2 Frequency domain partition resulting from the one level DWT decomposition Figure 3 Frequency domain partition resulting from the one level RWT decomposition ILL of DWT ILH of DWT ILL of RWF ILH of RWF IHL of DWT IHH of DWT IHL of RWF IHH of RWF (a) DWT (b) RWF Figure 4 Four decomposed subbands The frequency partition for one level 2D wavelet transform and rotate wavelet transform decomposition is illustrated in Figure 2 and Figure 3 respectively. An example of one level image decomposition using standard DWT and RWF is showed in Figure 4(a) and Figure 4(b) respectively. Texture characteristic oriented in 45 0 and 135 0 are clearly seen in the subband ILH of RWF and IHL of RWF respectively. 3.2 DT-CWT 45

Color and Texture Feature For Content Based Image Retrieval Jianhua Wu, Zhaorong Wei, Youli Chang The redundant complex wavelet transform designed by KingsBury [8][9][10] is called dual tree complex wavelet transform (DT-CWT). The main advantage of the complex wavelet is that it has improved properties in terms of shift sensitivity, directionality and phase information. The DT-CWT decomposes a signal in terms of a complex shifted and dilated mother wavelet. The DT-CWT is implemented by using separable transforms and combining subband signals appropriately [11]. Although it is non-separable yet, it inherits the computational efficiency of separable transform. Specifically, the 1D DT-CWT is implemented by using two filter banks in parallel operating on the same data as illustrated in Figure 5. A complex value wavelet (t) can be obtained as ( t) h ( t) j g ( t) (5) where (t h ) and g (t) are both real value wavelet. Thus far, the dual tree does not appear to be a complex transform at all. However, when the outputs from the two trees in Figure 5 are interpreted as the real and imaginary parts of complex coefficients, the transform effectively becomes complex. Figure 5 The 1D dual-tree complex wavelet transform Extension of the one dimensional DT-CWT to two dimensional is performed by separable filtering along rows and colus [11]. If colu and row filters both suppress negative requencies, only the first quadrant of the 2D signal spectrum is retained. Two adjacent quadrants of the spectrum are needed in order to fully represent a real 2D signal, so filtering with complex conjugates of the row filters is performed as well. This gives 4:1 redundancy in the transformed 2D signal. Since complex filters are able to separate all parts of the 2D frequency space, they provide true directional selectivity. Figure 6 shows impulse responses of these six wavelets associated with 2D complex wavelet transform. These six wavelet sub-bands of the 2D DT- 0 0 0 CWT are strongly oriented at { 15, 45, 75 }. 3.3 Texture image retrieval Figure 6 Impulse response of these six wavelets In this section, we choose the Brodatz texture image database which consists of 116 different texture images. Size of each image is 512*512. In this paper, each 512*512 image is divided into sixteen 128*128 non-overlapping sub-images. We select sixteen classes from the database randomly to generate the database D. Each image from the database D is decomposed five level using RWF, DWT and DT-CWT. The feature vector is formed by using energy and standard deviation of every sub-band. The Energy and Standard Deviation of wavelet sub-band are computed as follows, M N 1 Energy X ij (6) M N i1 j1 46

1 International Journal of Digital Content Technology and its Applications M N Std 1/ 2 [ ( X ) 2 ij ij ] M N (7) i1 j1 where M N is the size of wavelet sub-band, X ij is wavelet coefficient, and ij is the mean value of wavelet coefficient matrix. We repeat the above procedure to create the feature database for all the images and these feature vectors are stored in the feature database. Then we use the normalized Duclidean distance metric [3] as the similarity measure, which is given by NED( x, y) d ( x, y) (8) m n Where x y x y d( x, y) (9) ( ) ( ) where m and n i s the scale and the orientation. resulting from wavelet decomposition, and are the mean and the standard deviation of the magnitude of the wavelet decomposed sub-band that are used as the feature of the image, ( ) and ( ) are the standard deviations of the respective features over the entire database and are used to normalize the individual feature components. There are five different sets of feature : (1) RWF only (2) DWT only (3) DTCWT (4) DWT+RWF (5) RWF+DT-CWT. The retrieval results are showed in table1. Figure 7 shows retrieval example of texture D10 and D5 from the database D. From the Table 1, we know that the novel approach of RWF+DT- CWT is better than the other methods. 4. Combined Feature In this section, we combined the color feature and texture feature to retrieve image and compare the performance of these methods that we mentioned before.the database used in the experimentation consists of 10 different groups, and each group consists of 100 images from the Correl database. All these images in the database are natural images. The experiment environment is Matlab2009a. In this experiment, we choose top 50 images as the retrieval results and computer the average precision. The result shows in Table 2. We can know that the method we proposed is more effective. Figure 8 shows the retrieval results of some images. The query image is the first one. Table 1 An average retrieval accuracy of texture images(decomposition level is five) Image RWF Std DWT RWF+Std DWT DTCWT DTCWT+RWF samples D1 0.4332 0.4324 0.6238 0.6256 0.6890 D5 0.4379 0.5234 0.4375 0.5036 0.5642 D10 0.6860 0.6824 0.8125 0.8162 0.7588 D11 0.5135 0.6875 0.9374 0.9368 0.9842 D15 0.5123 0.3756 0.5136 0.6253 0.6876 D23 0.3762 0.2564 0.3245 0.2536 0.4628 D108 0.6876 0.5628 0.6254 0.5128 0.7524 47

Color and Texture Feature For Content Based Image Retrieval Jianhua Wu, Zhaorong Wei, Youli Chang Image samples RGB+HSV (a) Figure 7 Retrieved top 16 similar images for given query image Table 2 The average precision of (Top 50 images) RWF+RGB+H SV DWT+RGB+HS V RWF+DWT+R GB+HSV (b) DTCWT+RGB +HSV DTCWT+R WFRGB+H SV African 0.5730 0.5874 0.6058 0.5979 0.5932 0.6123 Beaches 0.2950 0.3130 0.3020 0.3150 0.3100 0.3230 Building 0.3360 0.3060 0.3280 0.3120 0.3220 0.3170 Buses 0.7210 0.7250 0.7310 0.7290 0.7460 0.7390 Dinosaurs 0.9820 0.9850 0.9850 0.9860 0.9860 0.9850 Elephants 0.3610 0.3810 0.4010 0.3960 0.3950 0.4040 Flowers 0.5130 0.5640 0.5750 0.5960 0.6210 0.6290 Horses 0.5070 0.5160 0.5340 0.5220 0.5000 0.5510 Mountains 0.3790 0.4110 0.4210 0.4310 0.4670 0.4720 Total 0.5186 0.5320 0.5425 0.5428 0.5489 0.5591 (a) African (b) Beaches (c) Buses (d) Dinosaurs Figure 8 Retrieval results 5. Conclusions In this paper, a novel approach called RWF+DT-CWT+color histogram in CBIR is presented. Simulation results demonstrated higher performance of the proposed method compared to the DWT, RWF and DWT+RWF in terms of average precision. The performance of the proposed method can be improved by applying the same low level features on region based image retrieval. We can know that when the image is simple, the performance is great, and when the image is complex, the average accuracy is worse. So our future work is focus on the average accuracy of complex image. 6. References [1] Th.Gevers (2001). Color Based Image Retriev-al. Springer Verlag GmbH. pp.886-917 [2] M.J. Swain, D.H. Ballard (1991). Color indexing. Int. J. Comput. Vis. 7 11-32. [3] B.S. Manjunath and W.Y. Ma (1996). Texture features for browsing and retrieval of image data, IEEE Trans. Pattern Anal. Mach. Intell, vol. 8, no. 8, pp. 837-842. [4] M. N. Do and M. Vetterli (2002). Wavelet-based texture retrieval using generalized Guassian density and Kullback-leibler distance. IEEE Trans.Image Process, vol. 11, no. 2, pp. 146-158. 48

International Journal of Digital Content Technology and its Applications [5] Challa S. Sastry *, M. Ravindranath (2007). A modified Gabor function for content based image retrieval, Pattern Recognition Letters. pp, 293-300. [6] Ju Han, Kai-Kuang Ma (2007). Rotation-invariant and scale-invariant Gabor features for texture image retrieval. Image and Vision Computing. Vol. 12. pp,1474-1481. [7] Manesh Kokare *, P.K. Biswas (2007). Texture image retrieval using rotated wavelet filters. Pattern Recognition Letters Vol.28. pp, 1240-1249 [8] N. G. Kingsbury (1999). Image processing with complex wavelets, Philos.Trans. R. Soc. London A Math. Phys. Sci, vol. 357, no. 3, pp. 2543-2560. [9] N. G. Kingsbury (1994). A Dual-Tree Complex Wavelet Transform with improved orthogonality and symmetry properties. Proc. IEEE Conf. On Image Processing, vol. II, pp, 1429. [10] N. G. Kingsbury (2001). Complex wavelets for shift invariant analysis and filtering of signals.journal of Applied and Computational Harmonic Analysis, vol. 10, no. 3, pp. 234-253. [11] N. Kingsbury (1998). The dual tree complex wavelet transform: A new efficient tool for image restoration and enhancement. Proc of EUSIPCO 98, pp. 319-322. [12] Subrahman yam Murala(2009). Color and Texture Features for Image Indexing and Retrieval. 2009 IEEE International Advance Computing Conference, pp. 1411-1416 49