Eye Localization Using Color Information. Amit Chilgunde

Transcription

1 Eye Localization Using Color Information Amit Chilgunde Department of Electrical and Computer Engineering National University of Singapore, Singapore ABSTRACT In this project, we propose localizing the eyes by using the color information of the human iris and sclera. Assuming that the upper-half of the face is available, we first filtered these face pixels based on their (R-G) and (G-B) color components. Secondly, the remaining pixels are classified into either eye or non-eye pixels based on their Cr and Cb color components. Using the proposed method, we were able to localize the eyes at an accuracy of 97.4%. 1. INTRODUCTION Localizing eyes of a person in an image has a number of uses including facilitating people s identity authentication using iris patterns, gaze detection and for robots with Artificial Intelligence capability. A lot of research has been carried out in this area and many methods have been proposed. We tried a few methods and finally used the method that gave the best results. In this project we tried a number of approaches to localize eyes like Bayesian networks and Kmeans clustering. In the approach using Kmeans we tried clustering on both gray and color images but could not get proper segmentation. The approach using Bayesian networks gave good results but this method was not very efficient. In this approach the first round of filtering involved using the R-G and G-B values and then their R, G and B values are quantized into a few levels and fed to the Bayesian Network. Later when we found out that the method using color information gives better results we decided to use that method. Input to this project is the upper-half section of a detected face. Based on color information of pixels on the upper-half of the detected face, we localize a pair of eyes by enclosing them with bounding boxes. 2. EYE LOCALIZATION USING COLOR INFORMATION Two layers of filtering are used to isolate eye pixels from non-eye ones. The first layer makes use of the R-G and G-B values of pixels as in Betke, The iris, sclera and non-eye regions are manually marked. Based on them, we generated some plots as well as some statistics of the R-G and G-B values. The graphs are shown in Figures 1 and 2 while the statistics are tabulated in Table 1. 1

2 iris/scl non-eye iris/scl gaussian non-eye gaussian Figure 1: Graph of frequency vs value (red-green) iris/scl gaussian non-eye gaussian iris/scl non-eye Figure 2: Graph of frequency vs value (green-blue). 2

3 Table 1: Statistics of` iris and sclera s colour components. Iris/Sclera Non-eye Mean Variance Mean Variance Red-Green Green-Blue Total Number of pixels used From the plots in Figures 1 and 2, threshold values of 8 and 19 are selected for R-G and G-B respectively for the first layer of filtering. After the first round of filtering some non-eye pixels are still wrongly considered as eye pixels. This is shown in Figure 3. (a) (b) Figure 3: A sample where the eye is not properly segmented. (a) Original image. (b) Binary image after using R-G and G-B for thresholding. White pixels indicate pixels which are eye pixels. The second layer of filtering involves the use of Cr and Cb. The distribution of Cr and Cb of pixels not filtered out at the first layer of filtering is shown in Figure 4. 3

4 Figure 4: Three-dimensional plots of Cr and Cb of remaining pixels. From the distribution, 3 linear equations are defined to identify 3 regions for iris, sclera and non-eye. The equations are: 1) Cr-Cb = 3, 2) Cr+0.154*Cb = -2.77, and 3) Cr + 4*Cb = RESULTS AND DISCUSSION The following table shows the results and accuracy of this method. Table 2 : Results of eye localization using color components. Number of eyes Percentage Accuracy Total number of eyes correctly localized

5 (a) (b) Figure 5: Localization of eyes using color components. (a) Bounding boxes around the eyes. (b) Binary image after iris/sclera identification. White region is the eye pixels. Generally, non-eye pixels from the hair on the side of the face or pixels in the regions where there is tremendous reflection from the ceiling lamps (the camera is placed at a tilted angle) tend to be wrongly classified as eye pixels. This is shown in Figure 5. (a) (b) Figure 6: (a) Lot of non-eye pixels are classified as eye pixels. All of them will be bounded in one box. (b) after applying the algorithm, only the eye is bounded by the box. In Figure 6(a), it can be seen that a lot of non-eye pixels are classified as eye pixels since they are connected to the eye pixels rendering an incorrect localization of the eye. To solve this problem, we made use of the fact that for any pixel to belong to the eye there should be either at least (height of box/2) pixels above it or below it or (width of box/2) pixels to the left of it or to the right of that pixel that belong to the eye. We can also change the division factor (2 in this case i.e. height/2 etc.) to get better results. The best results that we got were with a factor of 3. This gets rid of sufficient number noneye pixels such that the hair (or bright light) pixels are separated from the eye pixels 5

6 and thus the bounding box around the eye is much smaller and it bounds the eye more tightly as shown in Figure 6(b). 4. CONCLUSION In this project we tried a few methods for localizing eye like Bayesian Networks, Kmeans clustering, Color information etc. and finally we used the method using color information since it gives the best results. The proposed method for localizing the eyes involves two levels of filtering. In the first level, the pixels are filtered based on their Red-Green and Green-Blue values. During the second level, the remaining pixels are filtered based on their Cr and Cb values. Morphological processing on connected components is performed to finally obtain a bounding box around the eyes. REFERENCES M.Betke, W. J. Mullally and J. J. Magee, Active Detection of Eye Scleras in Real Time, IEEE CVPR Workshop on Human Modeling, Analysis and Synthesis, HMAS 2000, Hilton Head Island, SC, June