COLOR HISTOGRAM AND DISCRETE COSINE TRANSFORM FOR COLOR IMAGE RETRIEVAL

Size: px

Start display at page:

Download "COLOR HISTOGRAM AND DISCRETE COSINE TRANSFORM FOR COLOR IMAGE RETRIEVAL"

Gwen Flowers
6 years ago
Views:

1 COLOR HISTOGRAM AND DISCRETE COSINE TRANSFORM FOR COLOR IMAGE RETRIEVAL 1 Te-Wei Chiang ( 蔣德威 ), 2 Tienwei Tsai ( 蔡殿偉 ), 3 Jeng-Ping Lin ( 林正平 ) 1 Dept. of Accounting Inforation Systes, Chilee Institute of Technology 2 Dept. of Inforation Manageent, Chilee Institute of Technology 3 Dept. of Inforation Networking Technology, Chilee Institute of Technology ABSTRACT This paper proposes a content-based iage retrieval (CBIR) ethod based on color histogras and the discrete cosine transfor (DCT). Color histogra characterizes an iage by its color distribution, but the drawback of a global histogra representation is that spatial inforation (such as object location, shape, and texture) is discarded. In our approach, each iage is first transfored into the YUV space for the purpose of extracting the features based on color tones easily. Then, the histogra for each coponent (i.e., luinance Y, blue chroinance U, and red chroinance V) of the iage is obtained, which is served as the color feature of the iage. To copensate the inherent shortcoing of the color histogras, the DCT is applied to extract the spatial features fro the Y coponent of iages. Furtherore, a screening schee based on fuzzy cognition is incorporated into our CBIR syste for the purpose of efficiently retrieving the desired iages. We have perfored experients on a database with 1000 iages and the results show the effectiveness of our approach. Keywords: Content-based iage retrieval, color histogra, discrete cosine transfor, fuzzy cognition 1. INTRODUCTION Digital contents are becoing an iportant ediu for inforation collection and exchange. Given the exploding arket on digital photo and video caera's, the fast growing aount of iage content further increases the need for iage retrieval systes. To retrieve iages according to their contents is known as content-based iage retrieval (CBIR) [3]. CBIR is a technology to search for siilar iages to a query based only on the iage pixel representation. However, the query based on pixel inforation is quite tie-consuing because it is necessary to devise a eans of describing the location of each pixel and its intensity. Therefore, how to choose a suitable color space and reduce the data to be coputed is a critical proble in iage retrieval. Soe of the systes eploy color histogras. The histogra easures are only dependent on suations of identical pixel values and do not incorporate orientation and position. In other words, the histogra is only statistical distribution of the colors and loses the local inforation of the iage. Thus the iages retrieved by using the color histogra only ay not be desired even though they share siilar color distribution. Therefore, we are otivated to develop an iage retrieval schee to retrieve iages according to not only the color features derived fro their color histogras but also the texture and shape features derived fro their transfor doain. There are soe existing transfor-based feature extraction techniques that can be applied in CBIR, such as discrete wavelet transfor (DWT) [5], discrete cosine transfor (DCT) [4], Walsh, Fourier, 2-D oent, and Karhunen-Loeve. In our approach, the discrete cosine transfor (DCT) is used to extract low-level features. In our approach, each iage is first transfored

2 fro the standard RGB color space into the YUV space for the purpose of extracting the features based on color tones easily. Then, two types of features are extracted fro the YUV color space: Color features: the histogra obtained fro each coponent (i.e., luinance Y, blue chroinance U, and red chroinance V) of the iage can be regarded as a color feature of the iage; Spatial features: the DCT coefficients obtained fro the Y coponent (or the luinance) of the iage can be used as the spatial features of the iage. In the iage database establishing phase, the features of each iage are stored; in the iage retrieving phase, the syste copares the features of the query iage with those of the iages in the database, and find out good atches. 2. FEATURE EXTRACTION Before the feature extraction process, the iages have to be converted to the desired color space Color conversion A color space is a odel for representing color in ters of intensity values. It specifies how color inforation is represented. There exist any odels [1] through which to define the valid colors in iage data. Basically, color odels can be distinguished as: (1) hardware-oriented odels, e.g. RGB (Red, Green, and Blue), CMYK (Cyan, Magenta, Yellow, and Black Ke, and (2) user-oriented odels, e.g. YUV (Luinance and Chroa channels), HSV (Hue, Saturation, and Value). Hardware-oriented odels are defined according to properties of optic devices used to reproduce colors; it is difficult for the user to deal with these odels. On the other hand, user-oriented odels are based on huan perception of colors; it is ore appropriate for the developent of our user-interactive CBIR syste. In our approach, the RGB iages are first transfored to the YUV color space for the purpose of extracting the features based on the color tones ore easily. The details of RGB and YUV color spaces are introduced as follows. 1) RGB color space: A gray-level digital iage can be defined to be a function of two variables, f(, where x and y are spatial coordinates, and the aplitude f at a given pair of coordinates is called the intensity of the iage at that point. Every digital iage is coposed of a finite nuber of eleents, called pixels, each with a particular location and a finite value. Siilarly, for a color iage, each pixel ( consists of three coponents: R(, G(, and B(, each of which corresponds to the intensity of the red, green, and blue color in the pixel, respectively. 2) YUV color space: Originally used for PAL (European "standard") analog video, YUV is based on the CIE Y priary, and also chroinance. The Y priary was specifically designed to follow the luinous efficiency function of huan eyes. Chroinance is the difference between a color and a reference white at the sae luinance. The following equations are used to convert fro RGB to YUV spaces: Y ( = 0.299R( G( B(, (1) U( = 0.492( B( Y( ), and (2) V( = 0.877( R( Y( ). (3) Basically, the Y, U, and V coponents of an iage can be regarded as the luinance, the blue chroinance, and the red chroinance, respectively Color histogra The color histogra for an iage is constructed by discretizing (or quantizing) the colors within the iage and counting the nuber of pixels of each color. More forally, it is defined as

3 h ( y, z) = N Prob( X = Y = y, Z ), (4) X, Y, Z = z where X, Y and Z represent the three color channels (R, G, B or Y, U, V) and N is the nuber of pixels in the iage. The color histogra can be regarded as a set of vectors. For gray-scale iages these are 2-D vectors. One diension gives the value of the gray-level and the other the count of pixels at the gray-level. As for color iages each color channel can be regarded as gray-scale iages. More generally, we can set the nuber of bins in the color histogras to obtain the feature vector of desirable size. Basically, the bin nuber used for color histogras corresponds to the fineness degree of the color feature. Intuitively, using larger bin nuber could be better, but it is not always the case. Actually, it depends on the characteristics of the iages Discrete cosine transfor To extract spatial features fro color iages, DCT can be applied to the Y coponent of the iages for this purpose. Basically, the Y coponent of an iage can be regarded as a gray-level iage and DCT are ainly used for gray-level iages. A gray-level iage is organized in pixels, where each pixel contains the inforation of light intensity in a gray scale code fro 0 (black) to 255 (white). The intensity can be considered as a function of two variables, f(, where x and y are spatial coordinates, and the aplitude f at a given pair of coordinates is called the intensity of the iage at that point. Developed by Ahed et al. [1], the DCT is a technique converting a signal into eleentary frequency coponents. It uses the orthogonal real basis vectors whose coponents are cosines. The DCT approach has an excellent energy copaction property and requires only real operations in the transforation process. The DCT for an M N gray-level iage represented by pixel values f(i, j) for i=0, 1,, M-1, j=0, 1,, N-1 can be defined as C( u, v) = α ( u) α( v) M 1N 1 i= 0 j= 0 f ( i, j) (2i + 1) uπ (2 j + 1) vπ cos( ) cos( ), (5) 2M 2N for u=0, 1,, M-1, v=0, 1,, N-1, where α ( u ) = 1/ M if u = 0 and 2 / M otherwise; α ( v ) = 1/ N if v = 0 and 2 / N otherwise. C(u, v) are the f(i, j) are the input pixels. For ost iages, uch of the signal energy lies at low frequencies; these appear in the upper left corner of the DCT coefficients. The lower right values represent higher frequencies, and are often sall - sall enough to be neglected with little visible distortion. In our approach, the S S low frequency DCT coefficients of the Y coponent of an iage are used to constitute a feature vector for the iage. The value of S relates to the resolution level of the extracted features; the larger the value of S, the higher the resolution level is. 3. DISTANCE MEASUREMENT In our approach, the distance between two vectors is calculated on the basis of the su of squared differences (SSD). Assue that q and x represent the th feature of the query iage Q and an iage X in the database, respectively; each feature ay coe fro either the color histogras or the DCT coefficients. Then, the distance between q and x can be defined as d( q, x K 1 ) = ( q i= 0 [ i] x [ i]) where i is the ith coefficient of the th feature and q = x = K. Since several features are used siultaneously, it is necessary to integrate siilarity scores resulting fro 2, (6)

4 of the distances of the three color features are apped into a reduced set of qualitative linguistic labels: Sae, Very Siilar, Not Very Siilar, and All. Such that users can choose one of the four options according to their requireents for each color feature via the GUI of the syste. Fig. 1 Illustration of the for the screening under various precision labels. each individual feature. In our case, the total distance can be derived fro the following equation: M D ( Q, X ) = w. d( q, x ). (7) = 1 Here, Q and X are the query iage and one of the iages in the iage database, respectively. d is the distance function defined as Eq. (6); w R is the weight of the th feature; M is the nuber of feature being considered. 4. SCREENING SCHEME To efficiently retrieve the desired iages, a screening schee based on fuzzy cognition is incorporated into our CBIR syste; users can adjust the precision requireents for each query via the GUI provided by the syste. In our approach, three color features (i.e. luinance, blue chroinance, and red chroinance) are used for screening. The goals of the screening schee are twofold: Speed up the iage retrieving process: The concept underlying screening is to eliinate the obviously unqualified candidates via a certain criterion. Confor to the users requireents: Users ay have specific expectations about the retrieved iages; for exaple, the color tone of the retrieved iages ust be very siilar to the query iage, and so on. Nuerical easureents To filter out the iages which are dissiilar to the query iage Q fro the aspect of the distance derived fro the th feature, d(q, x ), we devise distance threshold for the kth precision label as: θ ( k ) 2 k = µ, (8) where = 1(luinance), 2(blue chroinance), and 3(red chroinance); k = 1(Sae), 2(Very Siilar), 3(Siilar); µ is the ean of the d(q, x ), which is defined as follows: n= N 1 µ = d( q, x ), (9) N 1 where N is the nuber of iages in the iage database. Figure 1 illustrates the for the screening under various precision labels. 5. EXPERIMENTAL RESULTS We evaluated perforance on a test iage database, which was downloaded fro the WBIIS database [6]. It is a general-purpose database including 1,000 color iages. The iages are ostly photographic and have various contents, such as natural scenes, anials, insects, building, people, and so on. Figure 2 shows the GUI of our CBIR syste and the retrieved results using a butterfly as the query iage. The size of the query iage is The iages of the sae size in the iage database were regarded as the initial candidates. For the query iage, 407 of 1000 iages are served as the initial candidate iages. The retrieved results using the color histogras

5 of Y, U and V (Bin Nuber=5) are shown in Figure 3(a); the results are the top 10 in siilarity, where the ites are ranked in the ascending order of the distance to the query iage fro the left to the right. To further iprove the retrieved results, the spatial feature extracted by the DCT is involved as well. Figure 3(b) shows the retrieved results using the color histogras and 3 3 DCT coefficients; the two features are cobined with equal weights. It can be found that, considering both the color features (i.e., the color histogras of the Y, U, V coponents) and the spatial features (i.e., the DCT coefficients of the Y coponent), the retrieved results are better than those by using a single feature, where the nuber of butterfly-related iages being retrieved is iproved fro 5 to 7. Figure 3(c) shows the retrieved results incorporating the proposed screening schee; the for the Y, U, and V coponents are set to those related to Siilar, Very Siilar, and Very Siilar labels, respectively. After the screening process, not only the nuber of butterfly-related iages being retrieved is further iproved to 8, but also the nuber of candidate iages is reduced fro 407 to 13; in other words, both the effectiveness and efficiency are iproved. To copare the three color spaces and their coponents in a quantitative anner, three classes of query iages, referring to white owl (5 iages), pupkins (4 iages), and deer (9 iages), are served as the benchark queries. To assess the ground-truth relevance score to each iage for each benchark query, each target iage in the collection is assigned a relevance score as follows: 1 if it belonged to the sae class as the query iage, and 0 otherwise. The process was repeated for all the relevant iages, and the overall average retrieval effectiveness was coputed for each of the ethods and each of the query exaples. The overall average relevance score in top 10 was coputed by averaging the individual values in each top 10. The bin nuber used for each color histogra is 5. Then, the retrieval effectiveness for each color space and their coponents can be evaluated. Tables 1 to 3 show the retrieval results for each ethod using the three classes of benchark query iages. Table 4 shows the average results of the three classes of benchark query iages. The results also deonstrate the effectiveness of our approach. 6. CONCLUSIONS We propose a CBIR ethod benefited fro the robustness of color histogras and the energy copacting property of DCT. In our approach, each iage is first transfored into the YUV space for the purpose of extracting the features based on color tones ore easily. Then, the color and spatial features of the iages can be derived fro their color histogras and DCT coefficients. In our CBIR syste, users can retrieve the desired iages efficiently via the syste s interactive user interface and the proposed screening schee. Future works include the incorporation of the coarse classification schee and other kinds of transfor-based features to the syste. 7. REFERENCES [1] N. Ahed, T. Natarajan, and K. R. Rao, Discrete cosine transfor, IEEE Trans. on Coput., 23, pp , [2] A. D. Bibo, Visual Inforation Retrieval, San Francisco: Morgan Kaufann, [3] V. Gudivada and V. Raghavan, "Content-Based Iage Retrieval Systes," IEEE Coputers, vol. 28, no. 9, pp , [4] Y.-L. Huang and R.-F. Chang, "Texture Features for DCT-Coded Iage Retrieval and Classification," Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, pp , [5] K.-C. Liang and C. C. Kuo, "WaveGuide: A Joint Wavelet-Based Iage Representation and

6 Description Syste," IEEE Trans. on Iage Processing, vol. 8, no. 11, pp , [6] J. Z. Wang. Content Based Iage Search Deo Page. Available at project/isearch/wbiis.htl, Fig. 2 The GUI of our CBIR syste. (a) (b)

7 (c) Fig. 3 The retrieved results (using the query iage shown in Fig. 2) based on (a) YUV color histogra (Bin Nuber =5), (b) (Bin Nuber =5) and 3 3 DCT coefficients, and (c) (Bin Nuber =5), 3 3 distance thresholds. Table 1 (Bin Nuber =5) Qurey 1: White Owl (5 relevant iages) (Bin Nuber =5) and (Bin Nuber =5), 3 3 Precision Rate (299) 0 (299) (89) Table 2 Qurey 2: Pupkins (4 relevant iages) (Bin Nuber =5) (Bin Nuber =5) and (Bin Nuber =5), 3 3 Precision Rate (299) 0 (299) (20.5) Table 3 (Bin Nuber =5) Qurey 3: Deer (9 relevant iages) (Bin Nuber =5) and (Bin Nuber =5), 3 3 Precision Rate (299) 0 (299) (38) Table 4 (Bin Nuber =5) The Overall Average Results (Bin Nuber =5) and (Bin Nuber =5), 3 3 Precision Rate (299) 0 (299) (49.2)

Novel Image Representation and Description Technique using Density Histogram of Feature Points

Novel Image Representation and Description Technique using Density Histogram of Feature Points Novel Iage Representation and Description Technique using Density Histogra of Feature Points Keneilwe ZUVA Departent of Coputer Science, University of Botswana, P/Bag 00704 UB, Gaborone, Botswana and Tranos