(b) Side view (-u axis) of the CIELUV color space surrounded by the LUV cube. (a) Uniformly quantized RGB cube represented by lattice points.

Transcription

1 Appeared in FCV '99: 5th Korean-Japan Joint Workshop on Computer Vision, Jan , 1999, Taegu, Korea1 Image Indexing using Color Histogram in the CIELUV Color Space Du-Sik Park yz, Jong-Seung Park y?, and Joon Hee Han y y Dept. of Comp. Sci. & Eng., POSTECH San 31, Hyo-Ja-Dong, Pohang, , Korea z Signal Processing Lab., Samsung A.I.T. San 14-1 Nongseo-Ri, Kiheung-Eup, Yongin, , Korea? Human Computing Research Dept., ETRI 161 Kajong-Dong, Yusong-Gu, Taejon, , Korea fpds, joonhang@postech.ac.kr, parkjs@falcon.postech.ac.kr Abstract Image indexing is a process of image retrieval from an image or video database based their contents. Specially, Histogram-based color indexing has been widely studied and is now considered to be the standard measure for color image indexing. We describe a new method of color space quantization in the CIELUV color space, named the weighted LUV quantization. In this method, each bin in the LUV space has a dierent weighting factor that is applied to the histogram intersection. The weighted LUV quantization provides the advantage of perceptual uniformity of the CIELUV color space. An additional advantage is the consideration of the perceptual sensitivity to the more saturated color with a weighting factor for each bin. 1 Introduction Image indexing is a process of image retrieval from an image or video database based their contents. The indexing process must satisfy the automated extraction of features, ecient indexing and eective retrieval of images within those database. Color feature has proven to be ecient in discriminating between relevant and non-relevant images. Moreover, the histogram-based technique has been widely studied and is now considered to be the standard measure for color image indexing. This work was supported in part by Automation Research Center(ARC) designated by KOSEF. The key issue in the histogram-based technique is selection of an appropriate color space and quantization of the selected color space. We describe a new method of color space quantization in the CIELUV color space. In the proposed method, The LUV space, a cube surrounding all of the CIELUV color space dened by conversion of RGB color space, is subdivided into uniform sized bins along with each axis. Each bin in the LUV space has a dierent weighting factor according to the volume of the CIELUV color space included within it. We denote this quantization scheme as the weighted LUV quantization. The weighting factor for each bin is applied to the calculation of the histogram intersection to measure the similarity between a query image and database images. 2 Image Indexing Histogram-based indexing The advantage of using color histograms is their robustness with respect to geometric changes of projected objects. Histograms are invariant to translation and rotation about the viewing axis and only change slowly under a change of angle of view, change in scale, and occlusion. We describe a histogram matching algorithm that is essentially the same as those of Swain and Ballard [1], Stricker and Swain [2], and Funt and Finlayson [3]. The colors in an image are mapped into a discrete color space containing n colors. A histogram of im-

2 age I is a n-dimensional vector where each element represents the number of pixels of color j in image I. We assume that each element of the histogram has been scaled so that the histogram represents the image without regard to the image size. The element of the normalized histogram H(I) is dened as H j (I) = ~ Hj (I)= nx ~H j (I) ; (1) where ~ Hj (I) is the number of pixels of color j in image I. We denote d(h(i); H(I 0 )) as the distance between two histograms. Swain and Ballard [1] introduced a histogram matching method called Histogram Intersection. Given a pair of histograms, H(I) and H(I 0 ), of image I and image I 0, respectively, each containing n bins, they dened the histogram intersection to be P n ~H(I) \ H(I ~ 0 ) = min( Hj ~ (I); Hj ~ (I 0 )) P n ~ H j (I) : (2) The denominator normalizes the histogram intersection and makes the value of the histogram intersection between 0 and 1. The measure ~ H(I)\ ~ H(I 0 ) is equivalent to the use of the L 1 -norm, d L1 ( ~ H(I); ~ H(I 0 )) = nx jh j (I)? H j (I 0 )j ; (3) as in Swain and Ballard [1], Stricker and Swain [2], and Funt and Finlayson [3]. Also, the L 2 -norm, which is similar to the one induced by the L 1 -norm, can be used to measure histogram distance: d L2 ( ~ H(I); ~ H(I 0 )) = vu ux t n (H j (I)? H j (I 0 )) 2 : (4) For a given distance T, two histograms are said to be similar if their distance is less than or equal to T. For a given threshold T, an image in a database is retrieved if its histogram distance is similar to the histogram of the given query image. Quantization The histogram dimension (the number of histogram bins) n is determined by a color representation scheme and quantization level. Most color spaces represent a color as a 3D vector with real values (eg. RGB, CIE XYZ, HSV, CIELUV). We quantize the color space of three axes into k bins for the rst axis, l bins for the second axis, and m bins for the third axis. The histogram can be represented as an n-dimensional vector where n = k l m. In general high resolution schemes, the histogram of RGB color sapce with [255; 255; 255] range of three axes is represented as a dimensional vector, HSV color sapce with [360; 100; 100] range of three axes as a dimensional vector, LUV sapce with [100; 354; 262] range of three axes as a dimensional vector. These high resolution representations, however, are too large and unnecessary for image indexing. Because the retrieval performance is saturated when the number of bins is increased beyond some value [4]. Typical quantization is much coarser, for example 16 bins for hue, 8 bins for saturation, and 8 bis for value in 1024 total combinations. 3 Color Spaces The word \color" may be interpreted in several ways: a certain kind of light, its eect on the human eye, or, most important of all the result of this eect in the mind of the viewer. Color is the perceptual result of visible light, which lies within the range of approximately 380nm - 750nm of the spectrum wavelength, incident upon the retina. CIE XYZ color space The assumption that there are three types of cone receptors in the retina is widely accepted, so three components are necessary and sucient to describe a color. Accordingly the set of all perceivable colors is olny a three-dimensional space. The axes of a color space, called primary colors, can be chosen arbitrarily. A convenient set, universally used for color measurement, is the CIE 1931 (X,Y,Z)-system adopted by the Commission Internationale de l 0 Eclairage (CIE) [5]. Each distinct point in the CIE XYZ space corresponds to a unique color perception. In this space, pure color component in the absence of brightness, such as hue and chroma, can be represented with x and y chromaticity coordinates dened by x = X X+Y +Z ; y = Y X+Y +Z : (5)

3 RGB color space RGB color space is represented with red (R), green (G), and blue (B) primaries and is an additive system. An additive RGB system is specied by the chromaticities of its primaries and its white point. This system's extent (gamut) is given in the (x,y) chromaticity diagram [5] by a triangle whose vertices are the chromaticities of the primaries. RGB values in a particular set of primaries can be transformed to and from CIE XYZ by three-bythree matrix transform. To transform from NTSC RGB with its C white point [6] into CIE XYZ the following transform is used: 2 4 X Y Z 3 5 = 2 4 0:607 0:174 0:2000 0:299 0:587 0:114 0:000 0:066 1: R G B 3 5 : (6) Because white is normalized to unity, the middle row sums to unity. To recover the white point, we need to transform RGB = [1; 1; 1] to CIE XYZ, then to compute x and y. The RGB color space provides a simple and fast computation. However, it is neither the perceptually uniform space nor the intuitive space. HSV color space The hue(h), saturation(s), and value(v) system is based on a warped version of the RGB cube and is directly related to intuitive color notions of hue, saturation, and brightness. The conversion from the RGB color space to the HSV color space is done through the following equation [7]: max = Maximum(r; g; b) min = Minimum(r; g; b) delta = max? min h 0 = 8 >< >: (g?b) delta if r == max 2+(b?r) delta if g == max 4+(r?g) delta if b == max v = max s = max?min max h = h 0 60 (7) (8) ; (9) where (r; g; b) is a point in the normalized RGB color space, (h; s; v) the corresponding point in the HSV color space. For r; g; b 2 [01], the conversion gives h 2 [0 360] and s; v 2 [0 1]. The intuitiveness of the HSV color space is very useful because we can quantize each axis independently. Wan and Kuo [4] reported that the color quantization scheme based on the HSV color space performed much better than one based on the RGB color space. Because the HSV color space involves dierent computations around 60 degree segments of the hue circle, The visible discontinuities are occurred in the color space. The HSV color space linearly converted from the RGB color space is not the perceptually uniform space also. CIELUV color space Consider the distance from color C 1 = (X 1 ; Y 1 ; Z 1 ) to color C 1 +C, and the distance from color C 2 = (X 2 ; Y 2 ; Z 2 ) to color C 2 + C, where C = (X; Y; Z). Both distances are equal to C, yet in general they will not be perceived as being equal. This is because of the variation of the just noticeable dierence throughout the space [7]. Perceptual uniformity means that the same C at two dierent points in the color space makes the equal perceived color dierence. The CIE XYZ, RGB, and HSV color spaces do not exhibit perceptual uniformity. There are two perceptually uniform color spaces agreed as the standard in CIE. One of the two is the CIELUV color space. The three variables L, u, and v are dened by [5]: L = ( 116( Yn Y ) 1 3 Y? 16 if Yn > 0: :3 Y otherwise Yn (10) u = 13L (u 0? u 0 n) v = 13L (v 0? vn) 0 ; (11) where u 0 ; v 0 and u 0 n; v 0 n are calculated from, 4X u 0 = X+15Y +3Z ; v0 = u 0 n = 4Xn Xn+15Yn+3Zn ; v0 n = 9Y X+15Y +3Z 9Yn Xn+15Yn+3Zn : (12) The tristimulus values X n ; Y n and Z n are those of the illuminant, with Y n equal to 1. The total color dierence E uv between two colors in the CIELUV color space is calculated from: E uv = [(L ) 2 + (u ) 2 + (v ) 2 ] 1 2 ; (13) where L, u, and v are the dierence in L, u, and v, between two colors, respectively.

4 4 Optimal LUV Space Quantization In the uniform quantization, Each axis of the color space is uniformly subdivied into a prespeci- ed number of bins and each bin is of the same size. The advantage of the uniform splitting method is that it is straightforward and simple [4]. However, the disadvantage of the uniform quantization for the perceptually non-uniform color spaces, such as RGB and HSV, is that they don't take into account the perceptual similarity between the dierent bins. To overcome this perceptual similarity problem, We select the CIELUV color space, which is the same color space that Taycher [8] chose for uniform quantization. Figure 1 shows the RGB space and the CIELUV color space that is surrounded by the LUV space. Each bin in the LUV space includes a dierent volume of the CIELUV color space dened by a two step transformation of the RGB space using Eqs. 6, 10, 11. Therefore, each bin has a dierent probability that a pixel will fall into the bin also. Here, we introduce the uniform quantization method with a weighting factor in the LUV space. In this method, each bin in the LUV space has a dierent weighting factor according to the volume of the CIELUV color space included within it. We denote this method as weighted LUV quantization. The weighting factor for each bin is applied to the calculation of the histogram intersection in the form: h ~H(I) \ ~ H(I 0 )iw = P n W j min( ~ Hj (I); ~ Hj (I 0 )) P n H ~ ; j (I) (14) where W j is the weight for the bin j in the LUV space. We denote h the histogram intersection, with subscript w, ~H(I) \ H(I ~ 0 )i, as the weighted histogram intersection. w The weight for each bin is determined by the following procedures: 1) Transform R, G, and B values at the lattice points in uniformly subdivided RGB color space with prespecied number into L ; u ; and v values of the CIELUV color space. 2) Compute the probability as an estimated volume included in each bin: P i = NP P i n NP ; i = 1; 2; :::; n (15) i where NP i is the pixel number in bin i, n is the total number of a bin in the space and P i is the probability that a pixel will fall in bin i. 3) Compute the weight, W i = 1 Pi P n 1 Pj ; (16) where W i is the weight for bin i. In computing the weighting factor, there is an assumption that the probability P i is proportional to the volume of the CIELUV color space included in the bin. The weight of the bin, including the smaller volume within it, will be the larger, that is, the bin including high saturated colors will get the large weight. The weighted LUV quantization is designed to provide the advantage of perceptual uniformity in the histogram similarity comparison and to consider the perceptual sensitivity to the high saturated color. 5 Experimental Results Experiments were carried out to present the color indexing performance of the weighted LUV quantization. To compare the performance, the histogram intersection is calculated on the HSV space and the LUV space with the typical uniform quantization and the weighted quantization. Captured frames from \Star TV" and the movie \Deep Impact" were used as database images. We denote the Star TV images as the rst database set and the Deep Impact images as the second database set. The rst database set has 47 images and the second database set has 63 images. A query image was selected from a database set and the histogram of the image was compared with the histograms of the rest of the images in the database set. Figure 2 shows the result of color indexing for the rst database set applied quantization scheme for the each space. The rst two rows in Figure 2 show the images in the sequence ranked as the top ten similar images in the three quantization scheme, the uniform quantization of the HSV space, the typical uniform quantization of LUV space(luv), and the weighted LUV quantization(luvw). The left side image for each quantization scheme is the query image followed by nine matched images

5 (b) Side view (-u axis) of the CIELUV color space surrounded by the LUV cube. (a) Uniformly quantized RGB cube represented by lattice points. (c) Top view (+z axis) of the CIELUV color space surrouned by the LUV cube. Figure 1: The RGB cube and a perspective view of the CIELUV color space surrounded by the LUV cube. in order of similarity. The centered number below each image is the sequence number in the original scene and the italicized number is the normalized similarity to the query image. Q. Step Measure HSV LUVw LUV VAR AD(%) VAR AD(%) Adaptive VAR AD(%) Table 1: Comparison of the variance and the percentile of the dierence of the average similarity for the rst database set. Table 1 shows the variance (VAR) of the three relevant images (including query image and the two top ranked images) and the percentile of the dierence (AD(%)) between the average histogram similarity of the three relevant images and the average histogram similarity of the seven non-relevant images. Q. Step means the quantization step used and Adaptive quantization step has the step for the HSV space and the step for the LUV space. Figure 3 shows the result of color indexing for the second database set. The quantization step used is the Adaptive for HSV and LUV, the step for LUVw. Q. Step Measure HSV LUVw LUV VAR AD(%) Adaptive VAR AD(%) Table 2: Comparison of variance and the percentile of the dierence of the average similatiry for the second database set. Table 2 shows the variance (VAR) of the four relevant images and the percentile of the dierence (AD(%)) of the average histogram similarity between the four relevant images and the six nonrelevant images for the second database set. In all quantization step, the uniform LUV quantization scheme has the least variance for the relevant images and the weighted LUV quantization scheme has the maximum dierence of the average similarity between the relevant images and the nonrelevant images. The general trend that the increament of the quantization resolution makes the increament of the percentile of the dierence of the average similarity and the variance was founded from the results. For the image indexing, the large difference of the average similarity and the appropriately small variance is very useful characteristic for discriminating between the relevant images and the

6 Figure 2: Comparison of the uniform quantization on the HSV space and the LUV space for the rst database set. Figure 3: Comparison of the uniform quantization on the HSV space and the LUV space for the second database set.

7 non-relevant images. From the viewpoint of these discussion, the weighted LUV quantization scheme has the better performance than others. Even in the low resolution quantization step, 5 5 5, this proposed method presents the best performance in the discriminating characteristic with an appropriately small variance. [7] J. D. Foley, A. van Dam, S. K. Feiner, and J. F. Hughes, Computer Graphics: Priciples and Practice. Addison-Wesley Publishing, [8] L. Taycher, Image Feature Extraction Subsystem of the ImageRover WWW Image Search System. Mater thesis, Boston University, Conclusion To improve color indexing performance, the weighted LUV quantization scheme was proposed. For given database sets, Experimental results show that the weighted LUV quantization method even in the lower resolution quantization gives better performance than others. The proposed weighted LUV quantization has the complete three dimensional subdivision scheme. For speed up the processing, we can apply the pseudo three dimensional subdivision scheme in which three axes are quantized independently of each other. Another interesting issue is to nd the method assigning the weighting factor to special bins for indexing images having special color, such as esh tone. These are remained as further investigation issues. References [1] M. J. Swain and D. H. Ballard, \Color indexing," International Journal of Computer Vision, vol. 7, no. 1, pp. 11{32, [2] M. Stricker and M. Swain, \The capacity of color histogram indexing," in CVPR94, pp. 704{ 708, [3] B. V. Funt and G. D. Finlayson, \Color constant color indexing," PAMI, vol. 17, no. 5, pp. 522{ 529, May [4] X. Wan and C.-C. J. Kuo, \Color distribution analysis and quantization for image retrieval," in SPIE proceedings, vol. 2670, February, [5] G. Wyszecki and W. S. Stiles, Color Science, concept and Methods, Quantitative Data and Formulae. John Wiley & Sons, [6] W. N. Sproson, Colour science in television and display systems. Adam Hilger, 1983.