Colour Model for Outdoor Machine Vision for Tropical Regions and its Comparison with the CIE Model

Transcription

1 IOP Conference Series: Materials Science and Engineering Colour Model for Outdoor Machine Vision for Tropical Regions and its Comparison with the CIE Model To cite this article: Nasrolah Sahragard et al 2011 IOP Conf. Ser.: Mater. Sci. Eng Related content - Color constancy and the natural image Brian A Wandall - Embedded System Implementation on FPGA System With CLinux OS Ahmad Fairuz Muhd Amin, Ishak Aris, Raja Syamsul Azmir Raja Abdullah et al. - Mechanochemical Synthesis and Characterization of Calcium-doped Ceria Oxide Ion Conductor Ong Poh Shing, Tan Yen Ping and Taufiq Yap Yun Hin View the article online for updates and enhancements. This content was downloaded from IP address on 03/10/2018 at 01:45

2 Colour Model for Outdoor Machine Vision for Tropical Regions and its Comparison with the CIE Model Nasrolah Sahragard 1, Abdul Rahman B Ramli 1,Mohammad Hamiruce Marhaban 2, and Shattri B Mansor 3 1 Institute of Advanced Technology, Universiti Putra Malaysia Serdang, Selangor, Malaysia 2 Department of Electrical and Electronic Engineering, Faculty of Engineering, Universiti Putra Malaysia Serdang, Selangor, Malaysia 3 Department of Civil Engineering, Faculty of Engineering, Universiti Putra Malaysia Serdang, Selangor, Malaysia sahragard@yahoo.com Abstract Accurate modeling of daylight and surface reflectance are very useful for most outdoor machine vision applications specifically those which are based on color recognition. Existing daylight CIE model has drawbacks that limit its ability to predict the color of incident light. These limitations include lack of considering ambient light, effects of light reflected off the ground, and context specific information. Previously developed color model is only tested for a few geographical places in North America and its accountability is under question for other places in the world. Besides, existing surface reflectance models are not easily applied to outdoor images. A reflectance model with combined diffuse and specular reflection in normalized HSV color space could be used to predict color. In this paper, a new daylight color model showing the color of daylight for a broad range of sky conditions is developed which will suit weather conditions of tropical places such as Malaysia. A comparison of this daylight color model and daylight CIE model will be discussed. The colors of matte and specular surfaces have been estimated by use of the developed color model and surface reflection function in this paper. The results are shown to be highly reliable. 1. Introduction Color Model and Surface Reflectance Model play a major role in predicting the color of a surface since color is an important feature for color object recognition in machine vision. The CIE daylight model (Commission of International Illumination 1931) [1] has three drawbacks that make it unsuitable for use for outdoor machine vision namely the lack of taking into account the effect of ambient light mainly from the sky, the lack of considering the reflection off the ground, and the lack of covering any scene information. There are other daylight color models [1] and surface reflectance models [2] [3] [4] [5] that have not been applied to realistic outdoor machine imagery. A lot of computational work has been done for color recognition under varying illumination in the area of color constancy where the goal of which is to match object colors under varying illumination without knowing the composition of the incident light or surface reflectance. Although there have been a lot of advances in the area of color constancy by Simonds [6], Land E [7], Shafer [5], Maloney and Wandell [8], Yuille [9], Klinker et al. [10], Forsyth [11], D Zumra and Iverson [12], Novak and Shafer [13], Ohata and Hayashi [14], Published under licence by Ltd 1

3 Finlayson [15], Funt and Finlayson [16], Funt et al. [17], Finlayson and Hordley [18], but their usefulness for outdoor images has not been established. Surface reflectance Models as described by Dana [19], Nicodemus [20], and Horn [2] is modeled by a bidirectional reflection distribution function (BRDF) explaining how light from a given direction is reflected from a surface at a given orientation. Upon the composition of the incident light and the characteristics of the surface, different spectra of light may be reflected at different orientations, thereby making the BRDF very complex. Lambertian model explained by Horn [2] is the simplest and it is used to describe the reflectance of matte surfaces see Oren and Nayar [21], Wolff et al. [22]. It predicts that light incident on a surface is scattered equally in all directions such that the total amount of light reflected is a function of the angle of incidence. Cook and Torrance [23] and Klinker [10] use the composite of the specular and Lambertian components to model surfaces with specular component. For instance, Shafer [5] models surface reflectance as a linear combination of the diffuse and specular components, and determines the weights of each component from a measure of specularity. Shafer s dichromatic Reflectance Model shows that color variation in RGB lies within a parallelogram, the length and breadth of which are determined by the two reflectance components. Klinker [10] refines the Dichromatic model by showing that surface reflectance follows a dog legged ( -shaped) distribution in RGB. Sato and Ikeuchi [24] uses temporally images to model the surface components. All of these methods depend on the presence of pure specular reflection from a point-source light. Since daylight is composite extended light source and not a point source, none of the methods mentioned above could be applied to outdoor images. Buluswar [25] developed a color model for outdoor machine vision that along with his surface reflectance model called Normalized photometric Function which is an adapted form of dichromatic surface reflectance model could estimate the color of a surface. His model has been applied to parts of North America under good weather condition and its validity for other places under different environment where there are humidity, fog, haze, hail, etc remains to be seen. This model for eastern tropical countries like Malaysia does not work for two reasons i) the sun angles in the model does not cover the whole range of sun angles in Malaysia and ii) the model is not built under humid condition as is the case for tropical regions. Gasparini and Schettini [26] in 2004 came up with an algorithm for automatic color correction. It was designed to distinguish between true cast and predominant color in a completely unsupervised way so capable of discriminating between images requiring color correction and those in which the chromaticity must be reserved. This was much better than white patch algorithm but it is not clear what the algorithm will do when the cast detector classifies an image as unclassifiable. Marchant Tillett et al. [27] addresses the problem of color changes caused by variation on natural illumination both in intensity and spectral content. Their method was tested only on vegetable and soil, very application specific. Also, they used a transformation method that gives a scalar output F from a 3D vector input (CR, CG, CB). The center wavelength for the camera filters must be known in order to calculate F. This information is hard to get since manufacturers think of it as proprietary information. Because of the transformation some information is lost, which means some objects with different colors could transform to the same value of F. This is similar to objects with different reflectance spectra being seen as the same color ( metamerism ) through a color camera. Manduchi [28] in 2006 presented a color constancy algorithm for color classification with explicit illuminant estimation and compensation. He made a lot of modeling assumptions. For his algorithm to work, there is a need to hand label and collect extensive training data sets which makes it time consuming and impractical in real world scenarios. Inter-reflections are ignored in his work. Also in 2006 Ebner [29] reported a genetic algorithm for color constancy. It is assumed that each pixel has one processing element that will compute an estimate for the local illumination given the color of its input pixel and the data available from neighboring processing elements. He has used color Mondrians for his algorithm and it is not mentioned if it is applicable for outdoor machines. No context information such as weather conditions humidity, fog, cloud, etc were considered in the development of the algorithm. Although good performance result is reported but the timing required for such huge task 2

4 is unclear. In depth explanation of algorithms and techniques for outdoor machine vision is presented thoroughly by Sahragard [30]. Machine vision is mainly related to the techniques developed for color prediction and recognition, surface reflectance and daylight models. The related algorithms often make assumptions or need requirements that may not work properly for outdoor environment since outdoor conditions are usually uncontrollable. So a new daylight model is constructed in HSV color space based on the weather conditions in Malaysia. This new model along with surface reflectance function could be used to estimate the color of a surface. 2. CIE Daylight Model Sun angle, cloud cover, and other weather conditions are important factors that bring changes to daylight color. Judd et al. [1] in their work show the amount of variation in CIE daylight color model in RGB color space as parabola. The information has been transferred into HSV space to make it relevant to the intention of this paper. The quadratic equation for CIE model in HSV color space is S= H H (1) Where H Figure 1: CIE Model sun vs sky in HSV color space and color circle from 3

5 The linear approximation of the above CIE equation is S= 0.87H (2) Where 6.36 H and its schematic is shown in figure 2. Figure 2: Linear CIE Model sun vs sky in HSV color space In Figures 1 and 2 the points to the right of the red point which is the circle origin represent the sun color in HSV color space and those to the left show the color of sky based on CIE variations in HSV space. The circle radius is supposed to be one for s going from 0 to 1 radial and it is seen at brightness 1 and of course h that is from 0 to 360. The CIE model has three disadvantageous due to the way this model has been constructed see [1]. It lacks considering large portion of ambient light from the sky, ground reflection, and context information. To include these factors in the model of daylight for more realistic situation, a Color Model has been developed by Buluswar [25]. This model is used for predicting the color of surfaces for several places in North America under clear weather conditions and its applicability is not certain for other places in the world and under different weather conditions such as humidity, fog, haze, rain, etc. The aim here is to use such a model to estimate the color of surfaces in hot and humid environment like Malaysia. The previous model fails to be useful in sampling the color of the daylight color for two reasons. It fails to cover all variations in illumination angle and it is not constructed in a hot and humid place in order to convey this information and its effects. So a new model suitable for tropical places such as Malaysia is built and the color of daylight is shown as normalized HSV. 3. Developing Daylight Model This color Model is a table of daylight color in the direction of the sun and away from the sun and indexed by sun angle or sun elevation, percentage of cloud cover, sun visibility. Other entries are brightness, temperature, and humidity. The changes in sun angle is from 0 to 90 degrees and cloud cover is partitioned into four groups of 0-20, 21-50, 51-70, indicating the percentage of cloud in the sky, and lastly the sun visibility factor as of 1 for visible sun, 0.5 for the sun covered with thin cloud and 0 for the wholly covered sun. The average color of the sky is also measured and tabulated. Later by use of this 4

6 color model along with the surface reflectance function, the apparent color of a surface could be predicted. Data comes as HSV calculated from images of a board having a number of matte surfaces of various colors mounted on a tripod with angular markings of 15 along with a rotatable head mount with marking of 10. The surface in the middle of the board is matte Munsell white N/9 with maximum reflectivity of 98% and is used to sample the incident light. Pictures were taken during the four months from January until May. During data collection, angular marking on the tripod is changed to vary the viewing geometry in the azimuth; images were taken at every change in sun angle. The viewing geometry with respect to white surface was approximately fixed by keeping a constant distance about 1 to 1.5 meters between the camera and the surface, as well as a constant camera height one and half meters. Other table entries include average HSV over 20x20 pixels to reduce the noise, and brightness from the direction of the sun hsv sun, Br sun and away from the sun hsv away and Br away. Standard deviation for each of these measurements σ namely (σ h, σ s, σ br, σ h-away, σ s-away, σ br-away ) and number of samples # taken are also mentioned. Average sky color hsv sky measured with the white surface facing up the sky is tabulated. The developed color model is partially shown as table 1. 5

7 Table 1: Partial representation of Illumination color model showing hsv and brightness (Br) values for the incident light in the direction of the sun (hsvsun and Brsun) and away from the sun (hsvaway and Braway) for various conditions, along with the corresponding standard deviations and the number of samples (#). The different conditions are indexed by sun angle, cloud cover (cloud %) and the sun visibility factor (SV). Hsvsky in the last column shows the average color of the sky measured with the white surface facing up towards the sky. Sun Angle Cloud % Sun Visibility hsv sun σ h σ s Br sun σ br-sun T H # hsv away σ h-away Br away σ braway σ s-away hsv sky

8 4. Comparison of Developed Color Model and CIE Model As we see in Figure 3 below there is discrepancy between our data and CIE model because of two reasons. First is the Field Of View of 180 in this case of study which relates to ambient skylight for large portion of the sky whereas FOV for CIE model is only 1.5. The second is the effects of sunlight reflected off the ground. It is shown that the effect of ambient skylight accounts for about 79% of the shift while the ground s reflection of sunlight accounts for 15%. Figure 3: Our data vs Linear CIE Model where ambient skylight and ground reflection are causes of the shift. For CIE linear: H=(h max - h min ) x 360= S=(s max - s min )= For color model: H=(h max - h min ) x 360= S=(s max - s min ) = So hue for color model is ( / ) x 100=67% It means that observed data is covering only 67% of angles covered by CIE linear in HSV. Saturation for the developed color model is (0.3617/0.7199) x 100=50.24% so the observed data is covering only 50.24% of saturation covered by CIE linear in HSV. 5. Surface reflectance Model Surface reflectance models mainly Shafer s Dichromatic model [5],Nayar s hybrid reflectance model [3], and Phong s shading model [4] make three assumptions that are not suitable for outdoor images. Daylight is a composite illumination but they assume single source illumination. They require brightness something that is hard to model. They assume all specular surfaces always show specular effect that may not be true if the illuminant is an extended source. From the existing surface reflectance models, Normalized Photometric Function in HSV is derived. This function really plots the normalized HSV color distance against relative viewing angle which is the difference of incident and viewing angles. NPF ignores the effect of intensity, separating surface intensity from normalized color. This function is drawn for five different surfaces including matte paper, two traffic sign as specular surfaces, concrete slab, and asphalt. Figure 4 shows the Normalized Photometric Function for matte paper and stop sign surfaces used for experiments. They are resulted from three to five images to cover the whole relative viewing angles from -90 to

9 Figure 4: Normalized Photometric Function for matte paper and stop sign in HSV color space 6. Estimating Surface Color NPF function for each surface along with the color table developed above will work together to estimate the color of a surface. Down to the image pixel level, the color of each pixel is calculated using A(n) = C + [ I C ] * (3) Where A is the apparent HSV color vector of a surface at a relative viewing angle n, under Illumination color vector I and a Lambertian component vector C, at relative distance ρ between C and I. Lambertian color C can be calculated through the use of 3x3 color coefficient matrix. Color estimation is done in HSV in this paper. 7. Results and Discussion Tests have been done on 102 images of the matte paper, two traffic signs, concrete, and asphalt surfaces. The estimation accuracy is measured by the number of observed samples found to be within confidence interval of the estimate. Besides, the certainty of the estimation is measured by the area of the rectangle being created by the mapping of estimated pixel color to xy coordinates within the 2 8

10 confidence interval. Technically speaking the smaller the area of interval the tighter and more useful the constraint imposed by the confidence interval will be. For illustrative purposes, Table 2 and 3 in the following show the result of apparent color estimation of matte paper and stop sign. Likewise figures 5 and 6 show estimates of the apparent color of the matte paper and stop sign under different sets of conditions by the corresponding confidence interval. Also relevant tables and confidence interval plots are derived for other surfaces. In the cases of specular surfaces there may be Lambertian (L), Transitional (T), and specular (S) cases. Sun angle Illumination angle Table 2: Results of apparent color estimation for matte paper over 5 sets of conditions Viewing angle % cloud cover Sun visibility angle # samples % observation within CI Ave_h Ave_s Averror % % % % % Figure 5: Estimates of apparent color of the matte paper presented by the corresponding confidence interval. Table 3: Results of apparent color estimation for the stop sign over 7 sets of conditions sa i va %cc Sv Effect #samples %obs Ave_h Ave_s averror within L 4 100% L 4 100% L 2 100% L 4 100% L 3 100% L 2 100% L 2 100% L 2 100% S 4 100% S 3 100%

11 S 5 100% T 4 62% T 4 100% T 5 100% T 2 80% T 3 83% T 2 100% Figure 6: Estimates of the apparent color of the stop sign under seven sets of Lambertian conditions are represented by the corresponding confidence interval. The average error is measured as the average Euclidean distance between the observed and estimated values. This error for matte paper is with the CI area of square units with 100% observations within 2 confidence interval of estimates. Likewise, the average error for traffic stop sign is 0.064(L), (S), (T) with the CI area of (L), (S), 0.026(T) square units with 95.6% observations within 2 confidence interval of estimates. These results show that the daylight color table developed here indicates the true color of daylight in tropical regions. 8. Conclusion A new model for outdoor color vision suitable for tropical places such as Malaysia is built and the color of daylight is shown as normalized HSV. This color model along with a model for surface reflectance function would help us to predict the color of a surface. NPF is the normalized HSV distance between Illuminant and lambertian color versus relative viewing angle since the color will lie on the dichromatic line between Lambert and Illuminant color. To make this happen the albedo of the surface or the 3x3 color coefficient matrix must be found for the surface being under red, blue, and green light. Several outdoor machine vision applications such as obstacle detection, road-following, and landmark recognition can benefit greatly from accurate color based models of daylight and surface reflectance. At the moment the model is expected to work for flat surfaces. An extension of the work for the spherical or elliptical surfaces may apply to food industry for outdoor machine fruit pick up where the color of the fruit could be determined to check if it is ready for harvesting. With the same token the ripeness of the fruit could be measured for example in the case of oil palm fruit maturity a major industry in Malaysia and Indonesia. 10

12 Acknowledgements The authors wish to thank Mrs Rosiah Osman for office support and Institute of Advanced Technology for providing the tools needed for practical experiments involved. This work is supported by fund of Malaysia under Grant No References [1] Judd, D., MacAdam, D. and Wyszecki, G., "Spectral Distribution of Typical daylight as a Function of Correlated Color temperature",. Journal of the Optical Society of America (8): p [2] Horn.B, Robot Vision. MIT Press, Cambridge, MA, [3] Nayar, S.K., K. Ikeuchi, and T. Kanade, Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE Transactions on Robotics and Automation, (4): p [4] Phong, B.T., Illumination for Computer Generated Pictures. Communications of the ACM, (6): p [5] Shafer, S.A., Using Color to Seperate Reflection Components. Color Research Application, (4): p [6] Simonds, J., Application of characteristic vector analysis to photographic and opyical response data Journal of the Optical Society of America A: Optics and Image Science, and Vision, (8). [7] Land E, M.J., Lightness and Retinex Theory. Optical Society of America, (1): p [8] Maloney, L.T. and B.A. Wandell, Color Constancy: A Method for Recovering Surface Spectral Reflectance. Journal of the Optical Society of America A: Optics and Image Science, and Vision, (1): p [9] Yuille, A., A Method for Computing Spectral reflectance. Biological Cybernetics, Springer- Verlog, : p [10] Klinker, G., Shafer, S. and Kanade, T., "Color Image Analysis with an Intrinsic Reflection Model" Proceedings of the International Conference on Computer Vision, [11] Forsyth, D., A Novel Approach for Color Constancy International Journal of Computer Vision, : p [12] D'Zmura, M. and G. Iverson, Color Constancy: Basic theory of two satge linear recovery of spectral descriptions for lights and surfaces. Journal of the Optical Society of America A: Optics and Image Science, and Vision, : p [13] Novak, C.L. and S.A. Shafer, Method for estimating scene parameters from color histograms. Journal of the Optical Society of America A: Optics and Image Science, and Vision, (11): p [14] Ohta, Y. and Y. Hayashi, Recovery of Illuminant and surface Colors from Images Based on the CIE daylight. Proceedings of the Third european Conference on Computer Vision, [15] Finlayson, G.D. Color constancy in diagonal chromaticity space. in IEEE International Conference on Computer Vision Cambridge, MA, USA: IEEE. [16] Funt, B. and G. Finlayson, "The State of Computational Color Constancy". Proceedings of the First Pan-chromatic Conference, Inter-Society color Council, [17] Funt, B., K. Barnard, and L. Martin, "Is machine Color Constancy Good enough?" Proceedings of the Fifth European Conference on Computer Vision [18] Finlayson, G.D. and S.D. Hordley, Color constancy at a pixel. Journal of the Optical Society of America A: Optics and Image Science, and Vision, (2): p [19] Dana, K.J., et al., Reflectance and texture of real-world surfaces. ACM Transactions on Graphics, (1): p [20] Nicodemus, F., et al., "Geometrical Considerations and Nomenclature for Reflectance ". NBS Monograph 160, National Bureau of standards, [21] Oren, M. and S.K. Nayar, Generalization of the Lambertian model and implications for machine vision. International Journal of Computer Vision, (3): p [22] Wolff, L.B., S.K. Nayar, and M. Oren, Improved Diffuse Reflection Models for Computer Vision. International Journal of Computer Vision, (1): p

13 [23] Cook, R. and K. Torrance, "A Reflectance Model For Computer Graphics". Image Understanding, : p [24] Sato, Y. and K. Ikeuchi, "temporal-color Space Analysis of Reflection". Journal of the Optical Society of America, (11): p [25] Buluswar, S.D. and B.A. Draper, Color models for outdoor machine vision. Computer Vision and Image Understanding, (2): p [26] Gasparini, F. and R. Schettini, Color balancing of digital photos using simple image statistics. Pattern Recognition, (6): p [27] Marchant, J.A., N.D. Tillett, and C.M. Onyango, Dealing with color changes caused by natural illumination in outdoor machine vision. Cybernetics and Systems, (1): p [28] Manduchi, R., Learning outdoor color classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, (11): p [29] Ebner, M., Evolving color constancy. The Journal of the pattern recognition society, : p [30] Sahragard, N. and A.R.B. Ramli, A review on algorithms and techniques for outdoor machine vision. European Journal of Scientific Research, (1): p