IRIS SEGMENTATION OF NON-IDEAL IMAGES

IRIS SEGMENTATION OF NON-IDEAL IMAGES William S. Weld St. Lawrence University Computer Science Department Canton, NY 13617 Xiaojun Qi, Ph.D Utah State University Computer Science Department Logan, UT 84322 ABSTRACT Iris segmentation plays a pivotal role in accurate iris recognition systems. Many iris segmentation methods rely on a controlled environment, wherein the subject may be required to submit themselves to have their eyes photographed in an ideal scenario. In unconstrained environments, however, iris segmentation becomes increasingly difficult to perform due to off-axis tilts, off-angles, and occlusion from reflections, eyelids, eyelashes, eye aids, and hair. This paper focuses specifically on non-ideal images, presents a novel method for locating the pupil, uses least-squares based ellipse fitting techniques to fit boundaries to the pupil and iris regions, describes geometric calibration techniques to compensate for distortion caused by off-angle acquisition, and demonstrates segmentation from within the ellipse boundaries. The proposed method has been tested on off-axis and off-angle images collected at the WVU eye center. Classified by hand, the ellipse fitting process yielded 89.7% accuracy on 195 images and the segmentation stage produced results at 88.2% accuracy on 195 images. Index Terms Segmentation, occlusion, least-squares ellipse fitting, geometric calibration 1. INTRODUCTION Iris recognition systems have been able to reliably automate the recognition of persons in controlled environments. Such systems rely on the knowledge that each person has a stable, unique iris pattern, and may thus be identified by it without error [3]. The acquisition of iris images is of great concern for these systems, as many of them rely on ideal images where the iris is fully visible and not occluded. Noisy iris segmentation relaxes the assumption that images of the subjects eyes are acquired by voluntary participation on the subjects behalf. Instead, it is understood that iris recognition systems need the capability to perform iris segmentation using images captured at a distance and on the move without the formal involvement of a subject. There are many techniques for segmenting the iris in an image. The most well-known approach to iris segmentation may be attributed to Daugman [3], who uses an integrodifferential operator to locate the relevant features in the eye. Also well-known for his work in iris segmentation is Wildes [4], who uses different types of Hough transforms to fit parametric curves to certain features in the eye. As the quality of the image degrades, however, both methods proposed by these men may fail to give accurate results. There has been much work done recently on segmenting noisy iris images, including some methods using various models of active contours like that of Ross and Shah [5]. Iris segmentation is a prerequisite for an accurate iris recognition system. In non-ideal images, the presence of occlusions interferes with the ability to effectively segment the iris and deteriorates the recognition accuracy. In this paper, a method is proposed which uses standard least-squares ellipse fitting techniques such as in [2]. In order to detect reflections, a binary threshold is applied to the image along with various morphological operations. This is part of an image pre-processing stage. After this, ellipse fitting is performed on the pupil. Geometric calibration is then performed to try and translate non-ideal cases involving off-axis and off-angle images back into frontal images, which are more ideal. The iris boundary is fitted iteratively after the calibration using the same techniques as the pupil ellipse fitting process. Segmentation is then performed using a Canny edge detector to locate eyelids, eyelashes, and iris boundary mistakes in the ellipse fitting stages. 2. NON-IDEAL IRIS ACQUISITION The WVU off-angle and off-axis data set was collected from about 100 people. For each eye of a person, two images were acquired from a front angle and two from an off-angle. The off-angles were preset to be 15 and 30, however, those angles are not specifically used in locating the iris due to varying gaze and head positions [1]. Each image is a grayscale image, which helps to reduce the processing time and complexity, as opposed to operating on an RGB image. There are two different scenarios within the off-angle images. In the first case, the camera and the eyes are at the same height, and thus the following scaling transformation relates

the front-angle image to its off-angle counterpart [ ] x = y [ cosθ 0 0 1 ] [ ] x y (1) boundaries in order to remove excess occlusions. Finally, section 9 depicts the experimental results and the conclusion and future work may be found in section 10. by compressing the horizontal direction [1]. In the second case, the camera and the eyes are not in the same horizontal plane, and thus the ellipse needs to be rotated with the angle being determined by the tilting of the camera. The nature of non-ideal images is centered on the notion that some part of the iris is occluded by either a single factor or possibly many. Common occlusions in non-ideal iris images include reflections, eyelids, eyelashes, eye aids, and any off-angles or off-axis tilts. Examples of non-ideal iris images from the data set can be seen in Fig. 1. Reflections, off-angle, eyelashes Reflections, off-angle Fig. 2. Overall process of proposed method. 4. IMAGE PRE-PROCESSING Reflections, off-angle, off axis Reflections, eyelashes Fig. 1. Examples non-ideal images from the WVU dataset. 3. DESCRIPTION OF PROPOSED METHOD The diagram in Fig. 2 shows the overall process of the proposed method. The goal of the image pre-processing stage is to prepare the image for pupil detection in a way that also reduces the time complexity of the system. A novel pupil detection algorithm, as demonstrated in section 4 is then implemented in order to locate the pixel coordinates so that we may begin the ellipse fitting process of the pupil as described in section 5. Once the ellipse parameters of the pupil have been returned, the image is geometrically calibrated to compensate for any distortion that may be present, as depicted in section 6. Then, the pupil ellipse parameters are scaled up by a calculated ratio to begin the iterative ellipse fitting process for the iris as described in section 7. Once we have the ellipse parameters for both the pupil and the iris, section 8 refers to the segmentation and noise removal that occur within the ellipse The image pre-processing stage is used to obtain the best pixel coordinates to be used for the ellipse fitting process of the pupil. The resulting pupil ellipse will be scaled up and used as the original ellipse during the iterative ellipse fitting process for the iris. The images in Fig. 3 show the steps in image pre-processing. First, the image is resized using bicubic interpolation, which averages pixel intensities in a pixels neighborhood to assign a new value to the given pixel, resulting in a smaller image with a smoother surface that is computationally less expensive to perform operations on. The reflections are then filled using the complement of the image and an appropriate threshold for isolating the pupil is chosen by dividing the mean intensity of the filled image by half. The holes in the image are then filled after the binary thresholding to produce more solid structures within the image. Structures that have an area less than 400 are removed so as not to interfere with locating the pupil, and any such structure would be too small to be considered the pupil. Then, a closing operation is performed using a disk structuring element of radius 8 to restore the elliptical or circular shape of the pupil that may have been compromised from removing reflections on the border between the pupil and the iris or within the pupil itself. The result is a binary image with several regions, including the pupil. Finally, the image seen in Fig. 3(f), is then submitted for ellipse fitting and geometric calibration.

Original image Image with reflections filled Binary image using threshold Closing Result points and the ellipse. However, it is sometimes the case that the pupil ellipse fitting returns a region that is not the pupil. In order to compensate for this error, the eccentricity of the fitted ellipse for each region (separate areas of white pixels in the binary images are considered distinct regions) is measured and only regions below 0.8 eccentricity may be considered the pupil. This threshold was chosen as an upper bound since the pupil region may not be ideal to fit an ellipse to. In experimentation, no returned pupil had an eccentricity level above 0.6. Ellipse fitting returns five parameters: the horizontal and vertical coordinates of the ellipse center (c x, c y ), the length of the major and major axes (r x, r y ), and the orientation of the ellipse φ. Once the ellipse parameters for the pupil have been returned, the two-step geometric calibration process may begin. 6. GEOMETRIC CALIBRATION (e) Isolated pupil (f) Pupil outline Fig. 3. Example of the pupil detection algorithm. Image is the binary image using the threshold obtained from, is the result after performing a closing operation using a disk structuring element, (e) is the isolated pupil, and (f) is the pupil outline, where the locations of the white pixels are used as the coordinates to begin the ellipse fitting process. 5. PUPIL ELLIPSE FITTING The pupil, which is near-circular, is characterized by dark pixel intensities similar to those of the eyelashes. Often times, reflections will appear within the pupil or on the border between the pupil and the iris. The pupil is detected by the image pre-processing stage, which takes care of the aforementioned issues. The result from the image pre-processing stage is depicted in Fig. 3 (f). Note, this is a binary image with the white pixels identifying the boundary of the pupil returned from pre-processing. These white pixels pixels are used to fit an ellipse that encompasses the pupil region and does not include any part of the iris structure. The fitting of the pupil ellipse is a least-squares based process and is approached by minimizing the sum of squared algebraic distances D A = N F (x i ) 2 (2) i=1 of the ellipse curve to the N data points [2]. The idea behind least-squares techniques is to find the set of parameters that minimize the distance measure in Eq. 2 between the N data Geometric calibration of non-ideal iris images attempts to compensate for geometric distortion caused by off-angle cameras. The idea here is to restore the shape of the pupil to make it as circular as possible. Although some pupils have been found to be non-circular [1], performing geometric calibration helps to translate the non-ideal case back into an ideal one. In the first step of geometric calibration, the image is rotated around the pupil center, (c x, c y ), by φ to restore the straight position of the ellipse. Second, we apply the inverse of the scaling transformation defined by Eq. (1) to restore the circular shape of the pupil. For instance, by applying the inverse of Eq. (1), an ellipse in f(x, y ) becomes a circle in f(x, y) whose vertical and horizontal directions are parallel to the long and short axes. In the scaling transformation, the parameter is given by cosθ = rx r y where r x, r y are the short and long axes of the ellipse respectively. When the image has been geometrically calibrated using the pupil ellipse, we then return to fit an ellipse to the iris using the pupil ellipse parameters scaled up by a calculated ratio. Once the pupil has been calibrated using its ellipse parameters, the ellipse fitting process iteratively fits an ellipse around the iris. Fig. 4 displays the results of geometric calibration on two images from the data set. 7. ITERATIVE IRIS ELLIPSE FITTING The iris ellipse fitting process uses the same techniques as the pupil ellipse fitting process as seen in section 5, except that it differs in the fact that it is iterative. The iris ellipse fitting process begins with ellipse parameters obtained from scaling the previously obtained pupil ellipse parameters up by a calculated ratio. The implemented system iteratively fits 99 ellipses to the image. For each ellipse, the squared algebraic distances are calculated using Eq. 2 and stored for comparison. After

Fig. 4. Examples of two non-ideal images and the result of their geometric calibration [1]. Image is off-angle, but that the same level, is off-angle and tilted, is the calibrated image of, and is the calibrated image of. Fig. 5. Examples where the ellipse fitting process successfully identified the pupil and iris boundaries. the iterative process has finshed, the ellipse that best approximated a curve to the iris boundary is chosen. That is to say, the ellipse that minimized the sum of the squared algebraic distances is selected to represent the boundary of the iris. Fig. 5 shows successful outcomes from the ellipse fitting process and Fig. 6 shows examples where the process obtained poor results. As you can see from Fig. 6, the ellipses include reflections, eyelids, and eyelashes within the boundaries. These obstructions are dealt with in the segmentation section. Further details about the ellipse fitting method can be read from [2]. 9. EXPERIMENTAL RESULTS 8. SEGMENTATION Since the ellipse fitting process is robust and includes eyelids, eyelashes, and reflections within the ellipse boundaries, the segmentation process is used to eliminate these remaining occlusions. Within the ellipse boundaries, a Canny edge detector is applied to the image. Ideally, the edge detector will locate any eyelids, reflections, or eyelashes interfering with the iris. The segmentation algorithm fills image regions contained between the detected edges and ellipse boundaries. Since the Canny edge detector is unspecific, it occasionally runs in to problems with very detailed iris patterns and will mistake parts of the patterns for either reflections or eyelashes, as shown in Fig. 8. Fig. 7 shows a sample of images that were successfully segmented using the Canny edge detector after ellipse fitting, whereas Fig. 8 displays errors made in the segmentation technique. In both figures, the yellow portions of the images represent pixels in the iris and may be used for recognition purposes. The proposed method has been tested on noisy iris images acquired from the WVU eye center. This dataset includes images captured at off-axis tilts and off-angles. Also present in these non-ideal images are occlusions from reflections, eyelids, and eyelashes. Each of these obstructions contribute to making iris segmentation more difficult. The contrived algorithm presented in this paper is heavily based on the work of [1]. Our implementations are similar through the end of the ellipse fitting process and diverge after that. Hence, only ellipse fitting results will be compared. Since it is difficult to compare the ellipse fitting results, they have been classified into three categories: Poor, Slightly Off, and Within Boundaries as shown in Fig. 6. Poor results are similar to those displayed in Fig. 6 and Fig. 6. Results that are considered Slightly Off are akin to the result displayed in Fig. 6, where you can see that the iris ellipse boundary includes a small portion of the sclera, which is the white region in the eye surrounding the iris. Results classified as Within Boundaries are analogous to that of Fig. 6, where the ellipse fitting process is slightly off and has left out a small part of the iris, but does not include extraneous information from the sclera. The Within Boundaries classification is the least meaningful in terms of error, since the fitted ellipse does not include the sclera, but disregards some useful information from the iris. Tables 1 and 2 display the results of the ellipse fitting processes from the work of [1] and the proposed method respectively on 72 images from the data set.

Fig. 6. Examples where the ellipse fitting process made mistakes. In, the eyelid and sclera are included in the ellipse, failed to identify the right boundary of the iris, most likely had trouble with the reflections, and stretches out to the left too far and includes some bits of the sclera. Fig. 8. Examples of poorly segmented images due to errors in the Canny edge detector. Images and don t include useful iris information located above the pupil, fails to eliminate the eyelashes, and excludes useful iris information and includes eyelashes. Table 1. Ellipse fitting results from the implementation of [1] on 72 images. Poor 5 6.9 Slightly Off 6 8.3 Within Boundaries 8 11.1 Total 19 26.4 Table 2. Ellipse fitting results from the implementation of the proposed method on 72 images. Poor 2 2.8 Slightly Off 4 5.6 Within Boundaries 2 2.8 8 11.1 Total Fig. 7. Examples of successfully segmented images. Please note that these images have been inspected and classified by hand. In addition to more accurate ellipse fitting results, the proposed method is computationally less expensive and more time efficient. On the 72 images, the implementation by [1] ran for 116 seconds through the ellipse fitting process, averaging 1.61 seconds per image. The implementation of the proposed method ran for 96 seconds until the end of the ellipse fitting process, averaging 1.33 seconds per image. There is a 0.28 second difference per image between the two methods, which may be a considerable difference de- pending on the size of the data set. The tables in 3 and 4 depict the ellipse fitting results and the segmentation results respectively on 195 images from the data set. 10. CONCLUSION AND FUTURE WORK An iris segmentation method for non-ideal images has been proposed, which uses standard least-squares based ellipse fitting in order to approximate the boundaries of the pupil and iris regions. In addition, the proposed method uses a Canny edge detector in order to eliminate errors from the robust ellipse fitting process. There are still very many improvements to be made in the

Table 3. Ellipse fitting results from the implementation of the proposed method on 195 images. Poor 4 2.9 Slightly Off 7 3.6 Within Boundaries 9 4.6 Total 20 10.3 Table 4. Segmentation results from the implementation of the proposed method on 195 images. Poor 6 3.1 Slightly Off 8 4.1 Within Boundaries 9 4.6 Total 23 11.8 implementation of the proposed method. In future work, using a linear Hough transform to fit a contour model to the eyelid boundary rather than a Canny edge detector may be more accurate with regards to eliminating eyelids and eyelashes within the fitted iris ellipse. In order to move from iris segmentation to recognition, it would be practical to diverge from the proposed method after the ellipse fitting process and perform elliptic unwrapping and normalization into the polar domain. From there, detecting occlusions would be simple with an edge detector or linear Hough transform and a resulting noise mask would be trivial to construct for matching purposes. The proposed method was implemented in Matlab and run on an Intel(R) Core(TM) i5 CPU 650 at 3.20GHz with 4.00GB of RAM. 11. REFERENCES [1] X. Li, Modeling intra-class variation for nonideal iris recognition, Advances in Biometrics, pp. 419-427, Springer 2005. [2] Fitzgibbon, A. W. and Pilu, M. and Fisher, R. B., Direct Least-Squares Fitting of Ellipses, IEEE Trans. on Pattern Anal. Mach. Intell., 21:476-480 1999. [3] J. Daugman, How Iris Recognition Works? IEEE Transactions on Circuits Syst. Video Tech., 14:21-30 2004. [4] R.P. Wildes, Iris Recognition: An Emerging Biometric Technology, Proceedings of the IEEE 85 (9) 1997 [5] Ross, A. and Shah, S., Segmenting Non-ideal Irises Using Geodesic Active Contours, Proceedings of Biometrics Symposium, September 2006