Face Recognition Under Varying Illumination Using Gradientfaces

Transcription

1 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 11, NOVEMBER [13] C. C. Liu, D. Q. Dai, and H. Yan, Local discriminant wavelet packet coordinates for face recognition, J. Mach. Learn. Res., vol. 8, pp , [14] M. Lades and J. C. Vorbruggen et al., Distortion invariant object recognitionin the dynamic link architecture, IEEE Trans. Comput., vol. 42, no. 3, pp , Mar [15] L. L. Shen and L. Bai, A review on gabor wavelets for face recognition, Pattern Anal. Appl., vol. 9, pp , [16] C. J. Liu and H. Wechsler, Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition, IEEE Trans. Image Process., vol. 11, no. 4, pp , Apr [17] L. L. Shen, L. Bai, and M. Fairhurst, Gabor wavelets and generalized discriminant analysis for face identification and verification, Image Vis. Comput., vol. 25, no. 5, pp , [18] B. C. Zhang, S. G. Shan, X. L. Chen, and W. Gao, Histogram of gabor phase patterns (hgpp): A novel object representation approach for face recognition, IEEE Trans. Image Process., vol. 16, no. 1, pp , Jan [19] S. Mallat, Wavelets for a vision, Proc. IEEE, vol. 84, no. 4, pp , Apr [20] I. W. Selesnick, R. G. Baraniuk, and N. G. Kingsburg, The dual-tree complex wavelet transform-a coherent framework for multiscale signal and image processing, IEEE Signal Process. Mag., vol. 22, no. 6, pp , Nov [21] G. Y. Zhang, S. Y. Peng, and H. M. Li, Combination of dual-tree complex wavelet and svm for face recognition, in Proc. Int. Conf. Machine Learning and Cybernetics, 2008, vol. 5, pp [22] Y. H. Sun and M. H. Du, Dt-cwt feature combined with onpp for face recognition, in Computational Intelligence and Security. Berlin/Heidelberg, Germany: Springer, 2007, vol. 4456, pp [23] Z. M. Huang, X. H. Zhang, and D. Yang, Face recognition based on dual-tree complex wavelet feature, J. Comput. Appl., vol. 27, no. 5, pp , [24] N. G. Kingsbury, Complex wavelets for shift invariant analysis and filtering of signals, Appl. Comput. Harmon. Anal., vol. 10, no. 3, pp , [25] H. Rabbani and M. Vafadust, Image/video denoising based on a mixture of laplace distributions with local parameters in multidimensional complex wavelet domain, Signal Process., vol. 88, pp , [26] E. H. S. Lo, M. R. Pickering, M. R. Frater, and J. F. Arnold, Image segmentation using invariant texture features from the double dyadic dual-tree complex wavelet transform, in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Apr. 2007, vol. 1, pp. I-609 I-612. [27] G. Y. Chen, T. D. Bui, and A. Krzyżak, Invariant pattern recognition using radon, dual-tree complex wavelet and fourier transforms, Pattern Recognit., vol. 42, no. 9, pp , Sep [28] T. Çelik, H. Özkaramanlı, and H. Demirel, Facial feature extraction using complex dual-tree wavelet transform, Comput. Vis. Image Understand., vol. 111, pp , [29] P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss, The feret evaluation methodology for face recognition algorithms, IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp , [30] T. Sim, S. Baker, and M. Bsat, The cmu pose, illumination, and expression database, IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 12, pp , Dec [31] [Online]. Available: [32] C. Nastar, B. Moghaddam, and A. Pentland, Flexible images: Matching and recognition using learned deformations, Comput. Vis. Image Understand., vol. 65, no. 2, pp , [33] J. P. Ye, Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems, J. Mach. Learn. Res., vol. 6, pp , [34] L. Yu and H. Liu, Efficient feature selection via analysis of relevance and redundancy, J. Mach. Learn. Res., vol. 5, pp , Face Recognition Under Varying Illumination Using Gradientfaces Taiping Zhang, Student Member, IEEE, Yuan Yan Tang, Fellow, IEEE, Bin Fang, Member, IEEE, Zhaowei Shang, and Xiaoyu Liu Abstract In this correspondence, we propose a novel method to extract illumination insensitive features for face recognition under varying lighting called the Gradientfaces. Theoretical analysis shows Gradientfaces is an illumination insensitive measure, and robust to different illumination, including uncontrolled, natural lighting. In addition, Gradientfaces is derived from the image gradient domain such that it can discover underlying inherent structure of face images since the gradient domain explicitly considers the relationships between neighboring pixel points. Therefore, Gradientfaces has more discriminating power than the illumination insensitive measure extracted from the pixel domain. Recognition rates of 99.83% achieved on PIE database of 68 subjects, 98.96% achieved on Yale B of ten subjects, and 95.61% achieved on Outdoor database of 132 subjects under uncontrolled natural lighting conditions show that Gradientfaces is an effective method for face recognition under varying illumination. Furthermore, the experimental results on Yale database validate that Gradientfaces is also insensitive to image noise and object artifacts (such as facial expressions). Index Terms Face recognition, gradientfaces, gradient domain, illumination insensitive measure. I. INTRODUCTION Within the past decade, major advances have occurred in face recognition. Many methods have been proposed for face recognition [1] [4]. However, the performance of most existing face recognition methods is highly sensitive to illumination variation. It will be seriously degraded if the training/testing faces under variable lighting. Thus, illumination variation is one of the most significant factor affecting the performance of face recognition and has received much attention in recent years. Many methods have been proposed to handle the illumination problem. In general, these methods can be divided into three main categories. The first approach uses image processing technique/model to normalize face images under different illumination conditions. For instance, histogram equalization (HE) [5], [6], logarithm transform [7] are widely used for illumination normalization. However, it is difficult for these image processing techniques to account for different lighting conditions. There have been models developed to remove lighting effects from images under illumination conditions. In [8], the authors compared images under variable illumination and proposed the complexity of ratio of two image as the similarity measures. The authors in [9] introduced the low curvature image simplifier (LCIS) using anisotropic diffusion for eliminating the notorious halo artifacts effect. However, the algorithm is computationally intensive and requires manual selection of no less than 8 different parameters. Manuscript received June 15, 2008; revised June 23, First published July 24, 2009; current version published October 16, This work was supported in part by the National Natural Science Foundations of China ( and ) and in part by the Natural Science Foundations of Chongqing CSTC (CSTC2007BA2003). The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Arun Abraham Ross. T. Zhang, Y. Y. Tang, B. Fang, and Z. Shang are with the Department of Computer Science, Chongqing University, Chongqing, China, ( tpzhang@cqu.edu.cn; yytang@cqu.edu.cn; fb@cqu.edu.cn, szw@cqu.edu.cn). X. Liu is with the Department of Mathematics and Physics Science, Chongqing University, Chongqing, China, ( xyliu@cqu.edu.cn). Digital Object Identifier /TIP /$ IEEE

2 2600 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 11, NOVEMBER 2009 The second approach, which handles the illumination problem by constructing 3-D face model. In [10] and [11], the authors showed that face images with fixed pose under varying illumination form a convex cone in the space of images, called illumination cone. It can be approximated well by low-dimensional linear subspace whose basis vectors are estimated from training data using the generative model. However, it requires multiple face images under different illumination conditions. The spherical harmonic model [12], [13] is applied to represent the low-dimensional subspace of different illumination face images. A segmented linear subspace model is presented in [14]; it generalizes the 3-D illumination subspace model by segmenting the image into regions with similar surface normal. However, these methods based on 3-D model either require the assumptions of light source or need many training samples, which are not practical for real applications. The third approach attempts to extract illumination invariant features or illumination insensitive measure, which is then used for face recognition. In general, a face image I(x; y) is regard as product I(x; y)= R(x; y)l(x; y), where R(x; y) is the reflectance and L(x; y) is the illuminance at each point (x; y) [15]. However, it is difficult to extract R from real images. Assuming L changes more slowly than R, the authors in [16] proposed to extract R by high-pass filtering on the logarithm of the image. Similarly, the authors in [17] proposed Retinex model to deal with illumination problem. In order to reduce halo artifacts, the authors in [18] extended the Retinex theory to multiscale retinex (MSR). Quotient image (QI) [19] is regarded as an illumination invariant and designed for dealing with illumination variation in face recognition. The authors in [20] and [21] proposed the self-quotient image (SQI) model to extend the QI theory, which parallels the former idea of Brajovic [22]. The illumination invariant is obtained by division over a smoothed version of the image itself. However, the weighted Gaussian filter they used has trouble keeping sharp edges in low frequency illumination fields, and the parameter selection is empirical and complicated. In order to overcome these limitations by utilizing the edge-preserving capability of the total variation (TV) model, the authors in [23] proposed to utilize the logarithm TV (LTV) model to factorize an image, and then obtain illumination invariant. However, the authors in [24] show that there are no illumination invariants, the illumination invariant obtained by the above-mentioned methods is at best insensitive to illumination. In this correspondence, we propose an illumination insensitive measure extraction method for face recognition under varying lighting called Gradientfaces. First, the proposed method transforms image into gradient domain, then Gradientfaces is extracted from the gradient domain for face recognition. Theoretical analysis shows that Gradientfaces is insensitive to illumination and robust to different illumination, including uncontrolled natural lighting conditions. We summarize the advantage of the proposed method as follows. 1. Gradientfaces is insensitive to illumination changes and can effectively deal with face recognition under different illumination. Compared with SQI and LTV methods, the comparison analysis show that Gradientfaces is more robust to different illumination, including uncontrolled (natural) lighting conditions. 2. Gradientfaces is extracted from the gradient domain; thus, it has the advantage of the gradient domain. Compared with pixel domain, the gradient domain considers the relationships between the neighboring pixel points such that the gradient domain is able to reveal underlying inherent structure of image data. 3. The proposed method is able to apply directly to any single face image neither does it require any prior information on 3-D face shape and many training samples. The rest of the correspondence is organized as follows. In Section II, Gradientfaces is described in detail. A variety of experimental results are presented in Section III. Finally, the conclusion is given in Section IV. II. METHODOLOGY In this section, we introduce how to extract illumination insensitive measure from face images under lighting conditions. We begin with the analysis of the reflectance model and gradient domain, then Gradientfaces is extracted from gradient domain. Finally, we compare Gradientfaces with SQI and LTV method, the comparison analysis shows that Gradientfaces is more robust to different illumination. A. Reflectance Model The reflectance model used in many cases can expressed as I(x; y)=r(x; y)l(x; y) (1) where I(x; y) is image pixel value, R(x; y) is the reflectance and L(x; y) is the illuminance at each point (x; y). Here, the nature of L(x; y) is determined by the lighting source, while R(x; y) is determined by the characteristics of the surface of object. Therefore, R(x; y) which can be regarded as illumination insensitive measure. Unfortunately, separating the reflectance R and the illuminance L from real images is an ill-posed problem. In order to attempt to solve the problem. A common assumption is that L varies very slowly while R can change abruptly. Let us analyze the common assumption. Considering two neighbor point (x; y) and (x+1x; y) of an image I(x; y) taken under lighting, since L(x; y) is determined by the lighting source, then it is reasonable that L(x; y) and L(x+1x; y) are approximately equal in general when 1x is small. Therefore, L is approximately smooth. This coincides with the common assumption (L varies very slowly), which means L is approximately smooth. In fact, the common assumption is a widely accepted assumption which has been used in [16] [18], [22], [25]. In [26, pp. 2], there is a detailed review of the reflectance model and the common assumption. It should be pointed out that the conclusion (L be approximately smooth) may be violated in shadow boundaries. However, shadow is an open problem in the field of image processing and pattern recognition. To address the issue of shadow boundaries, a Gaussian kernel function is introduced to the Gradientfaces in Section II-D. Furthermore, different from image processing, we ought to be more concerned with recognition performance rather than image itself in face recognition applications. B. Gradient Domain Versus Pixel Domain The gradient domain is very important to image processing. For example, edge detecting is important application of the gradient domain [27]. The gradient domain compositing approach is employed in constructing seamless composites using the gradient domain of images [28]. In [29], the authors present a general framework for performing constrained mesh deformation tasks with gradient domain techniques. In [30], the authors describe a unified framework for performing these operations on video in the gradient domain. Therefore, there may be more discriminating features in the gradient domain than the pixel domain. As all we know, the pixel points are not completely independent of each other, there are some relationships between neighboring pixel points. However, the conventional face recognition methods, such as PCA and LDA, which are implemented in pixel domain such that they ignore the underlying relationships between neighboring pixel points. While the gradient domain explicitly considers such relationships between neighboring pixel points such that it is able to reveal underlying inherent structure of image data. Therefore, the gradient domain has more discriminating power to discover key facial features than pixel domain. In order to take advantage of the benefits of the gradient domain, we propose a novel method to extract illumination insensitive measure from the gradient domain.

3 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 11, NOVEMBER C. Gradientfaces In order to extract illumination insensitive measure from gradient domain, we have the following theorem by studying the relationships between the components of gradient domain: Theorem 1: Given an arbitrary image I(x; y) taken illumination condition, the ratio of y-gradient of I(x; y) to x-gradient of I(x; y) is an illumination insensitive measure. Proof: Considering two neighboring points (x; y) and (x +1x; y), according to the illumination model (1), we have TABLE I GRADIENTFACES IMPLEMENTATION I(x; y) =R(x; y)l(x; y) (2) I(x +1x; y) =R(x +1x; y)l(x +1x; y): (3) Subtracting (2) from (3), we obtain I(x +1x; y) 0 I(x; y) = R(x +1x; y)l(x +1x; y) 0 R(x; y)l(x; y): Based on the above-mentioned common assumption, which means L is approximately smooth; thus, we have I(x +1x; y) 0 I(x; y) R(x +1x; y)l(x; y) 0 R(x; y)l(x; y) (R(x +1x; y) 0 R(x; y))l(x; y): (4) Taking the limitation of the above equality (4), we can y) L(x; y) Similarly, we have Dividing (5) by (6), we y) L(x; y) : (7) According to illumination model (1), R can be considered as an illumination insensitive measure. Thus, the ratio of y-gradient of I(x; y) (@I(x; y)=@y) to x-gradient of I(x; y) (@I(x; y)=@x) is also an illumination insensitive measure. In practical application, the ratio of y-gradient of image to x-gradient of image might be infinitude derived by zero value of x-gradient of image. Thus, it can not be directly used as the illumination insensitive measure. These considerations lead us to defining Gradientfaces as follows. Definition 1: I be an image under variable lighting conditions, then Gradientfaces (G) of image I can be defined as G = arctan I y-gradient Ix-gradient ; G 2 [0; 2) (8) where Ix-gradient and Iy-gradient are the gradient of image I in the x, y direction, respectively. D. The Implementation In order to extract Gradientfaces, firstly, we need to calculate the gradient of face image in the x, y direction, then Gradientfaces can be computed by the definition (8). There are many methods for calculating the gradient of image. However, the numerical calculation of deriva- Fig. 1. Original images and the corresponding Gradientfaces. (a) Ideal face image and its Gradientfaces. (b) Face image with severe illumination and its Gradientfaces. Generally, to lighting image: (in Gaussian kernel) = 0:1 to 0.5, lighting image with noise: = 0:6 to 1.0. tive (gradient) is typically ill-posed. To compute the gradient stably, we smoothen the image first with Gaussian kernel function. With a convolution-type smoothing, the numerical calculation of gradient is much more stable in calculation. The same operations are used in [27]. Furthermore, it should be pointed out that the main advantage for using Gaussian kernel is twofold: (a) Gradientfaces is more robust to image noise and, (b) it can reduce the effect of shadows. The implementation of Gradientfaces can be summarized in Table I. Fig. 1 shows the original images and the corresponding Gradientfaces. As can be seen, Gradientfaces can extract the important features of face, such as facial shapes and facial objects (e.g., eyes, noses, mouths, and eyebrows) even under extremely lighting, which are key features for face recognition. Therefore, Gradientfaces is an illumination insensitive measure. Furthermore, compared with original images, the relative position of key features represented by Gradientfaces is not changed. Therefore, Gradientfaces is insensitive to object artifacts (such as facial expressions). E. Comparison Analysis of Gradientfaces, SQI, and LTV In this section, we present a comparison analysis of Gradientfaces, SQI, and LTV methods. 1) Connections to SQI and LTV: SQI [20], [21] extract illumination insensitive measure by division over a smoothed version of the image itself, which can be expressed as Q = I^I where I is face image under varying illumination, ^I is the smoothed I, and Q is Self-Quotient Image, which is an illumination insensitive mea- (9)

4 2602 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 11, NOVEMBER 2009 Fig. 2. Illumination images (first column) and illumination insensitive measures extracted by MSR (second column), SQI (third column), LTV (fourth column), and Gradientfaces (fifth column). Fig. 3. Five images (from left to right) of one subject from subset 1 to 5, respectively (first row), and the corresponding Gradientfaces (second row). sure. The same idea is proposed in [22]. In [20] and [21], the authors used the weighted Gaussian filter to obtained ^I. However, the weighted Gaussian filter has trouble keeping sharp edges in low frequency illumination fields, in order to overcome this limitation, the authors in [23] proposed to use the TV 0 L 1 [31] model to obtain the smoothed I in image logarithm domain, which utilized the edge-preserving capability of total variation model. As can be seen, LTV and SQI are based on the same idea that the ratio of image and the smoothed version of the image itself is an illumination insensitive measure. Now, we analyze Gradientfaces, which is derived from illumination model (1) based on the common assumption. According to the common assumption, we draw an approximate conclusion that L is approximately smooth. Thus, the illumination model (1) Gradientfaces relied on the assumption is equivalent to SQI model (9). The assumption is L is viewed as the smoothed I. 2) Comparison Analysis: In order to obtain illumination insensitive measure R, most existing methods based on illumination model (1), including SQI, LTV, and MSR, try to find an estimation of L first, then obtain illumination insensitive measure R by solving R = I=^L, where ^L is an estimation of L. However, it is very difficult to accurately estimate L based on the common assumption, especially for images taken under different lighting. It is inevitable to produce error in the estimation of L, which may lead SQI and LTV methods to being unrobust to different lighting. However, we only utilize the smoothness of L in the extraction of Gradientfaces such that there is no need to estimate L. Therefore, Gradientfaces is more robust to different illumination than MSR, SQI, and LTV methods. Fig. 2 shows the comparison of illumination insensitive measures extracted by different methods from two face images with different lighting. As can be seen, there is much difference between the two illumination insensitive measures extracted from images with different lighting by SQI, LTV, and MSR methods, while there is only little difference between the two illumination insensitive measures extracted by Gradientfaces. The results provide an intuitionistic view that Gradientfaces is more robust to different lighting than SQI, LTV, and MSR methods. III. EXPERIMENTS In this section, we evaluate the proposed Gradientfaces method by several experiments on different databases, including an outdoor database taken under uncontrolled natural lighting conditions, and compared the proposed method with MSR [18], SQI [20], [21], and LTV [23] methods under the same experimental conditions. It is worth noting that a commercial digital image processing software PhotoFlair (TruView Imaging Company) is used to implement MSR algorithm with its default parameter settings, SQI algorithm is based in part on the publicly available Torch3Vision toolbox ( with parameters (K = 5, 9, 15). A. Recognition Protocol Instead of generating a feature vector in the pixel domain, Gradientfaces is extracted from the gradient domain, whose components are the argument of gradient. Therefore, in order to calculate the similarity between Gradientfaces, we define the distance (L1 distance) between two Gradientfaces vectors G1 and G2 as follows: n d(g1;g2) = i=1 min(jg 1i 0 g2ij; 2 0jg1i 0 g2ij) (10) where G1 = (g11;g12;...; g1n) and G2 = (g21;g22;...;g2n) are the Gradientfaces vectors, n is the dimensionality of the vector. It is worth noting that the smaller d(g1;g2) means the higher similarity. In this correspondence, we evaluate the performance of face recognition by template matching method, and apply the minimum-distance classifier for its simplicity, L1 metric defined in (10) is used as our distance measure. For the other competing method, the distance measure L1 metric is defined under the conventional sense. It should be pointed out that template matching method rather than other learning methods, such as PCA and LDA, is used to evaluate the performance of different methods, this is because the learning methods (PCA, LDA, etc.) have some capability for handling illumination while template matching method does not have. Therefore, the results achieved by template matching method can directly show the capability for handling illumination of different methods. B. Results on Yale B Database The Yale face database B [11] contains images of ten individuals with nine poses and 64 illuminations per pose. The frontal face images of all subjects, each with 64 different illuminations are used for evaluation. All images are cropped and resized to They are divided into five subsets based on the angle of the light source directions. The five subsets are: subset 1 (0 to 12 ), subset 2 (13 to 25 ), subset 3 (26 to 50 ), subset 4 (51 to 77 ), subset 5 (above 78 ). As a result, there are total 640 images: 70, 120, 120, 140, and 190 images in subset 1 to 5, respectively. Fig. 3 shows five images for one subject from different subsets and the corresponding Gradientfaces. In order to compare with the results from important literatures, the first experiment is devised to use the ideal images (subset 1) as reference images, the other images from subset 2 to 5 as query images. Comparison results with other methods designed for dealing with face recognition under varying lighting are shown in Table II. The results of MSR, SQI, and LTV methods are obtained under same experimental conditions and the other results (Linear subspace [11], Cones-attached [11], Cones-cast [11], and Harmonic images [32]) are directly from other literatures. As can be seen, Gradientfaces outperforms most existing method except the Cones-cast method. However, as aforementioned, the Cones-cast method needs complicated 3-D model; thus, it cannot be applied in practical applications. Moreover, the recognition results on subset 5 with extremely lighting are not given.

5 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 11, NOVEMBER TABLE II PERFORMANCE COMPARISON ON YALE B DATABASE TABLE IV PERFORMANCE COMPARISON [THE MEAN AND STANDARD DEVIATION (IN PARENTHESES)] ON PIE DATABASE TABLE III PERFORMANCE COMPARISON [THE MEAN AND STANDARD DEVIATION (IN PARENTHESES)] ON YALE B DATABASE Fig. 4. (a) Sample images for a single subject in PIE database. (b) Corresponding Gradientfaces. The second experiment was designed to use one image per subject choose from subset 1 as reference image sets, the other images from subset 1 to 5 as query image sets, in turn.table III shows that the recognition results of different methods on different query sets. As can be seen, Gradientfaces outperforms MSR, SQI, and LTV methods on all subsets. The recognition accuracy of MSR and SQI drops significantly when query image come from subsets (subsets 4 5) with the extremely lighting conditions. Gradientfaces and LTV methods achieve very high recognition rates on all subsets. C. Results on PIE Database The PIE face database [33] contains 68 subjects with face images as a whole. The face images were captured by 13 synchronized cameras and 21 flashes, under variations in pose, illumination and expression. Our work here concerns about illumination variations rather than pose and expression. So illumination subset (C27) was used in our experiment. There are 21 distinct sources of lights used to illuminate the face. In addition, these 21 sources are sampled both with the background room lights on and the background room lights off. We use 68 subjects with 1428 face images, each with 21 different illuminations for testing. All images are cropped and resized to Fig. 4 shows the 21 different lighting images of one subject in PIE database and the corresponding Gradientfaces. One experiment is conducted on this database. We use one image per subject as the reference images, the other images as query images, in turn. The average recognition results are shown in Table IV. As can be seen, Gradientfaces significantly outperforms MSR, SQI, and LTV methods, which show that each query image can be matched in almost all cases even if the reference images or query images come from extremely bad illumination conditions. In order to further analyze recognition rates versus different lighting conditions, the recognition accuracy under each reference set is shown in Fig. 5. As can be seen, Gradientfaces achieves closely to 100% recognition rates on all reference sets, while the other competing methods including SQI and LTV can reach 95% recognition rates when only relative ideal images are selected as reference image, and MSR obtain only about 85% recognition rates in this case. However, the recognition rates of MSR, SQI, and LTV seriously degrade when the reference images are taken under lighting conditions. It is evident that Gradientfaces is very robust to reference image and query image with different illumination, while the other competing methods including MSR, SQI, and LTV are very sensitive to reference image under different lighting conditions. D. Results on Outdoor Database The images in Yale B and PIE face database are taken under controlled lighting conditions. In order to evaluate the performance of Gradientfaces under uncontrolled illuminations, an outdoor database is used to evaluate the performance of Gradientfaces under uncontrolled natural lighting conditions. The database contains 660 images of 132 individuals, each with five images taken under outdoor lighting conditions, a part of images come from Caltech [34] and FRGC Version 2 [35] face databases. All the images are manually aligned (two eyes are aligned at the same position), cropped and resized to Fig. 6 shows all images of one subject in this database. In this experiment, we choose one image per subject as reference images, the other images as query image, in turn. The average results of gradientfaces and the other competing methods are illustrated in Table V. As shown in the results, Gradientfaces achieves more than 95% recognition rate on average, while MSR, SQI, and LTV methods only reach 59.08%, 64.27%, and 72.46% recognition rate, respectively. Therefore, Gradientfaces is also robust to face recognition under uncontrolled natural lighting conditions, while the other competing methods, including, MSR, SQI, and LTV, are very sensitive to uncontrolled natural lighting. E. Results on Yale Database The Yale face database is taken from the Yale Center for Computational Vision and Control. It contains 165 grayscale images of 15 individuals. Original image size is pixels. All images are manually cropped to include internal structures (such as the brow, eyes, nose, and mouth), and resized to pixels. Fig. 7 shows cropped six images of one subject and corresponding Gradientfaces. As can be seen, the main challenge on this database is facial expressions (normal, happy, sad, sleepy, surprised, and wink) and occlusion (with/without glasses). We perform two experiments on this database. The first experiment is designed to test Gradientfaces is insensitive to different facial expressions and occlusion (with/without glasses). The second one is designed to test Gradientfaces is robust to noise. The face images with noise used in second experiment are simulated by using the MATLAB function imnoise with different parameter. The sample images and comparison of results are illustrated in Fig. 7.

6 2604 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 11, NOVEMBER 2009 Fig. 5. Recognition rates versus reference image sets. The Horizontal axis shows all 21 images per subject in PIE database, which belong to different reference sets, respectively. F. Efficiency Fig. 6. Images under natural lighting condition (first row) and the corresponding gradientfaces (second row). TABLE V PERFORMANCE COMPARISON (%) [THE MEAN AND STANDARD DEVIATION (IN PARENTHESES)] ON OUTDOOR DATABASE In two experiments, recognition rates were determined by the leaving-one-out strategy, and template matching (TM) method is still applied to evaluate the performance of face recognition. The recognition results are shown in Table VI. As can be seen, template matching method on the Gradientfaces achieves 93.94% and 91.51% recognition rate, respectively. Template matching method on original images only reaches 80.61% and 69.09% recognition rate, respectively. Recognition rate on the images computed by MSR is 80% and 44.85%, respectively. Recognition rate on the images computed by SQI is 80.91% and 53.33%, respectively. Recognition rate on the images computed by LTV is 87.27% and 70.91%, respectively. Therefore, Gradientfaces is very robust to noise, facial expressions, and occlusion. In this subsection, we study the efficiency of the proposed Gradientface on PIE database by measuring the CPU time, and compare it with the other competing algorithms. The results are summarized in Table VII. As can be seen, the computational time of Gradientface is less than the others. Therefore, Gradientface is a computationally efficient method. Various experiments have been systematically performed. Let us summarize the advantages of Gradientfaces as follows. Gradientfaces is very robust to different lighting. It only relies on the smoothness of L in model (1) that is a widely accepted assumption. However, the other competing methods, such as SQI, LTV, and MSR methods, have to estimate the value of L from images with lighting. It is inevitable that there will produce error in the estimation of L such that these methods are not robust to different lighting. In addition, compared with original image, the relative position of key feature represented by Gradientfaces is not changed (see Fig. 1). Therefore, Gradientfaces is insensitive to object artifacts (such as facial expressions) in comparison with original image. This conclusion is also supported by the experimental results on Yale database. Gradientfaces can effectively reduce the effect of shadows and noise. The main purpose of using Gaussian smoothing kernel to gradient transformation is to calculate the gradient of image stably. However, another important advantage for using Gaussian kernel is it can reduce the effects of noise and shadows. The high recognition rates on PIE, Yale B, and Yale databases have confirmed that Gradientfaces can reduce the effect of shadows and noise (The images in PIE database and subset 5 of Yale B database have many shadows (see Fig. 3), and the image used in second experiment on Yale database is corrupted by noise).

7 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 11, NOVEMBER then an illumination insensitive measure (Graidnetfaces) is extracted for recognizing. The work relies on the widely accepted assumption that L in model (1) might be approximately smooth. The high recognition rates achieved by Gradientfaces on all databases have justified this assumption, and show that Gradientfaces is effective method for illumination problem in face recognition, and robust to different lighting and noise. ACKNOWLEDGMENT The authors would like to thank the associate editor and the anonymous reviewers for helpful comments that greatly improved the correspondence. Fig. 7. Results of different methods on face images with noise in Yale database. The images are (from top to bottom): original images, original images with noise, error (difference) images between original images and noise images, error images of MSR processing on original images and noise images, error images of SQI processing on original images and noise images, error images of LTV processing on original images and noise images, and error images of the computed Gradientfaces. As can be seen, the difference of the computed Gradientfaces is very small, which validates Gradientfaces is robust to noise, facial expressions, and occlusion. TABLE VI PERFORMANCE COMPARISON ON YALE DATABASE TABLE VII EFFICIENCY COMPARISON ON PIE DATABASE Gradientfaces is able to apply directly to any single face image neither does it require any prior information on 3-D face shape and many training samples. In addition, Gradientfaces has low computational cost such that it can be applied to practical applications. IV. CONCLUSION In this correspondence, we propose a Graidnetfaces method as an image preprocessing technique for face recognition under varying lighting. This method transforms image into the gradient domain first, REFERENCES [1] H. Shim, J. Luo, and T. Chen, A subspace model-based approach to face relighting under unknown lighting and poses, IEEE Trans. Image Process., vol. 17, pp , [2] T. Zhang, B. Fang, Y. Y. Tang, G. He, and J. Wen, Topology preserving non-negative matrix factorization for face recognition, IEEE Trans. Image Process., vol. 17, pp , [3] J. Zou, Q. Ji, and G. Nagy, A comparative study of local matching approach for face recognition, IEEE Trans. Image Process., vol. 16, no. 10, pp , Oct [4] Z. Liu and C. Liu, A hybrid color and frequency features method for face recognition, IEEE Trans. Image Process., vol. 17, pp , [5] S. M. Pizer and E. P. Amburn, Adaptive histogram equalization and its variations, Comput. Vis. Graph., Image Process., vol. 39, no. 3, pp , [6] S. Shan, W. Gao, B. Cao, and D. Zhao, Illumination normalization for robust face recognition against varying lighting conditions, in Proc. IEEE Workshop on AMFG, 2003, pp [7] M. Savvides and V. Kumar, Illumination normalization using logarithm transforms for face authentication, in Proc. IAPR AVBPA, 2003, pp [8] D. W. Jacobs, P. N. Belhumeur, and R. Basri, Comparing images under variable illumination, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1998, pp [9] J. Tumblin and G. Turk, LCIS: A boundary hierarchy for detail-preserving contrast reduction, in ACM SIGGraph, 1999, pp [10] A. S. Georghiades, P. N. Belhumeur, and D. J. Kriegman, From few to many: Generative models for recognition under variable pose and illumination, in Proc. 4th IEEE Int. Conf. Automatic Face and Gesture Recognition, 2000, pp [11] A. S. Georghiades, P. N. Belhumeur, and D. J. Kriegman, From few to many: Illumination cone models for face recognition under variable lighting and pose, IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp , Jun [12] R. Basri and D. Jacobs, Lambertian Reflectance and Linear Subspaces, NEC Research Inst. Tech. Rep R, 2000, Tech. Rep. [13] R. Basri and D. W. Jacobs, Lambertian reflectance and linear subspaces, IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 2, pp , Feb [14] A. U. Batur and M. H. Hayes, Linear subspaces for illumination robust face recognition, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001, pp [15] B. K. P. Horn, Robot Vision. Cambridge, MA: MIT Press, [16] T. G. Stockham, Jr., Image processing in the context of a visual model, Proc. IEEE, vol. 60, pp , [17] E. H. Land and J. J. McCann, Lightness and retinex theory, J. Opt. Soc. Amer., vol. 61, pp. 1 11, [18] D. J. Jobson, Z. Rahman, and G. A. Woodell, A multi-scale retinex for bridging the gap between color images and the human observation of scenes, IEEE Trans. Image Process., vol. 6, no. 7, pp , Jul [19] A. Shashua and T. Riklin-Raviv, The quotient image: Class-based re-rendering and recognition with varying illuminations, IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 2, pp , Feb [20] H. Wang, S. Z. Li, and Y. Wang, Generalized quotient image, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004, pp

8 2606 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 18, NO. 11, NOVEMBER 2009 [21] H. Wang, S. Z. Li, and Y. Wang, Face recognition under varying lighting conditions using self quotient image, in Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, 2004, pp [22] R. Gross and V. Brajovic, An image preprocessing algorithm for illumination invariant face recognition, in Proc. Int. Conf. Audio and Video Based Biometric Person Authentication, 2003, pp [23] T. Chen, W. Yin, X. Zhou, D. Comaniciu, and T. S. Huang, Total variation models for variable lighting face recognition, IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 9, pp , Sep [24] H. F. Chen, P. N. Belhumeur, and D. W. Jacobs, In search of illumination invariants, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000, pp [25] T. Zhang, B. Fang, Y. Yuan, Y. Y. Tang, Z. Shang, D. Li, and F. Lang, Multiscale facial structure representation for face recognition under varying illumination, Pattern Recognit., vol. 42, no. 2, pp , [26] R. Fattal, D. Lischinski, and M. Werman, Gradient domain high dynamic range compression, in Proc. 29th Annu. Conf. Computer Graphics and Interactive Techniques, 2002, pp [27] J. Canny, A computational approach to edge detection, IEEE Trans. Pattern Anal. Mach. Intell., vol. 8, no. 6, pp , Jun [28] A. Agarwala, Efficient gradient-domain compositing using quadtrees, ACM Trans. Graph., vol. 26, no. 3, pp , [29] J. Huang, X. Shi, X. Liu, K. Zhou, L.-Y. Wei, S.-H. Teng, H. Bao, B. Guo, and H.-Y. Shum, Subspace gradient domain mesh deformation, ACM Trans. Graph., vol. 25, no. 3, pp , Mar [30] S. Fleishman, D. Cohen-Or, I. Drori, T. Leyvand, and H. Yeshurun, Video Operations in the Gradient Domain, Tech. Rep., Tel-Aviv Univ., Tel-Aviv, Israel, [31] D. Strong and T. Chan, Edge-preserving and scale-dependent properties of total variation regularization, Inv. Probl., vol. 19, no. 6, pp , [32] L. Zhang and D. Samaras, Face recognition under variable lighting using harmonic image exemplars, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003, pp [33] T. Sim, S. Baker, and M. Bsat, The CMU pose, illumination, and expression (PIE) database, in Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, 2002, pp [34] M. Weber, The Caltech Frontal Face Database, California Inst. Technol. [Online]. Available: html [35] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek, Overview of the face recognition grand challenge, presented at the IEEE Conf. Computer Vision and Pattern Recognition 2005.