Preprocessing Techniques for Face Recognition using R-LDA under Difficult Lighting Conditions and Occlusions

Transcription

1 Preprocessing Techniques for Face Recognition using R-LDA under Difficult Lighting Conditions and Occlusions Ashraf S. Huwedi University of Benghazi, Faculty of Information Technology(IT) Benghazi - Libya ash_huwedi@yahoo.com Abstract In recent years face recognition has received substantial attention from researchers in biometrics, pattern recognition, and computer vision communities. At least two reasons account for this trend: the first is the wide range of commercial and law enforcement applications, and the second is the availability of feasible technologies in recent years of research. Many existing methods in face recognition area perform well under certain conditions, but still facing challenging with illumination changes and occlusions. This paper attempts to deal with the above challenges by combining robust illumination normalization techniques with powerful feature extraction method. Keywords: Face Recognition, Wavelet Analysis, Waveletbased illumination normalization, Wavelet-based image Denoising, Gradient face, Linear Discriminate Analysis (LDA). F I. INTRODUCTION ACE recognition (FR) has been an active research topic in the fields of computer vision and biometrics over the past several decades [1][2][3] due to its numerous potential applications, such as human-computer interfaces, access control, security, surveillance, e-commerce, entertainment and so on. However, Illumination variation is one of the most significant factors limiting the performance of face recognition. Since several images of the same person appear to be dramatically different under different illumination, the performance of most existing face recognition methods is highly sensitive to illumination variation, and will be severely degraded if the training/testing face is exposed to severe lighting variations. To overcome the problem caused by illumination variation on face, various approaches have been introduced, such as preprocessing and illumination normalization techniques, illumination invariant feature extraction techniques, and 3D face modeling techniques. Many methods have been proposed to handle the illumination problem. In general, these methods can be divided into two main categories: A. Illumination Invariant/Insensitive Feature Extraction Techniques These techniques attempt to extract facial features robust to illumination variations. Early researches employed simple descriptors such as logarithmic transformation, edge map, and image gradient to reduce the impact of illumination variation. These methods are built on primitive image processing theories and they are easy to implement with a relatively limited performance. However, the recently proposed Gradient face method [6] developed from the image gradient presents a very good performance. Waveletbased facial structure representation of the face image method was presented in [5]. In Retinex approaches the luminance is estimated by the smoothed image. The image can then be divided by the luminance to obtain the reflectance, which is an invariant feature to illumination. The sum of several Gaussian functions with different scales is applied to smooth the image in the multi-scale retinex approach[13]. Logarithm transform is employed to compress the dynamic range in [13]. Moreover, the Weber Face [15] and the Local Binary Pattern (LBP) [14] are all effectual descriptors robust to illumination changing. B. Illumination Pre-processing Techniques These techniques attempt to normalize the variation in order to make the images appear stable under different lighting conditions. This is accomplished either by image transformation or by synthesizing a new image from the given image in some normalized form, then the normalized images are used for recognition. For instance, Histogram Equalization (HE), Gamma correction, logarithm transform, etc. are widely used for illumination normalization [16]. In [7],Tan and Triggs proposed a method which combines the strengths of illumination pre-processing, invariant feature extraction, distance transform based matching, kernel-based feature extraction and multiple feature fusion. The illumination normalization process is the same as that described in [8]. Local Ternary Patterns (LTP),a generalization of the Local Binary Pattern (LBP) local texture descriptor, is then extracted. Finally, the robustness is further improved by adding kernel PCA feature extraction and incorporating rich local appearance cues from two complementary sources Gabor wavelets and LBP. Within our work, we propose to use three robust preprocessing methods in order to compare the effectives and weakness of each one. each pre-processing consists of two stages namely, Difference of Gaussian (DoG) /Gradient Face, Difference of Gaussian (DoG) /Contrast Equalization

2 and Normalization/wavelet de-noising, and then extract the features using powerful statistical method namely Regularized Linear Discriminant Analysis (R-LDA). Finally, the extracted feature vectors are classified by calculating the cosine distance between them. II. ILLUMINATION NORMALIZATION Real-world input data always contains some amount of noise and certain preprocessing is needed to reduce its effect. The term noise is to be understood broadly: anything that hinders a pattern recognition system in fulfilling its commission may be regarded as noise no matter how inherent this "noise" is in the nature of the data. Some desirable properties of the data may also be enhanced with preprocessing before the data is fed further in the recognition system. within this section we describes three illumination normalization methods. These preprocessing methods run before feature extraction that incorporates a series of stages designed to count the effects of illumination variations, local shadowing and highlights while preserving the essential elements of visual appearance. A. DoG / Contrast Equalization Difference of Gaussian (DoG) Filtering is a convenient way to obtain the resulting band pass behavior. Fine spatial detail is critically important for recognition so the inner (smaller) Gaussian is typically quite narrow (σ 1 pixel), while the outer one might have σ 2 of 2 4 pixels or more, depending on the spatial frequency at which low frequency information becomes misleading rather than informative. We find that σ 1 =1 and σ 2 =2.30 gives best results as previous stage before Contrast Equalization. Contrast Equalization is the second step of this preprocessing. It globally rescales the image intensities to standardize a robust measure of overall contrast or intensity variation. It is important to use a robust estimator because the signal typically still contains a small admixture of extreme values produced by highlights, garbage at the image borders and small dark regions such as nostrils. One could, e.g., use the median of the absolute value of the signal for this, but here we have preferred a simple and rapid approximation based on a two stage process:,,,,,,, Here, a is a strongly compressive exponent that reduces the influence of large values, τ is a threshold used to truncate large values after the first phase of normalization, and the mean is over the whole image. By default use a= 0.3and τ= 1.5. The resulting image is now well scaled but it can still contain extreme values. To reduce their influence on subsequent stages of processing, we apply a final nonlinear mapping to compress over-large values. The exact functional form is not critical. Here we use the hyperbolic tangent, tanh, /, thus limiting I to the range,. Figure 1 shows the two steps of this preprocessing method and the images before and after preprocessing process. Figure 1. DoG + Contrast Equalization B. DoG / Gradient Face DoG as mentioned above, is a convenient way to obtain the resulting band pass behavior. The DoG can be used as first stage before Gradient Face preprocessing. For this we find best results 1 as inner filter to reduce aliasing and noise, and 4 as outer filter to suppresses low frequency lighting variation without suppressing too much class information. Gradient face is an illumination insensitive measure derived from the image gradient domain such that it can discover underlying inherent structure of face images since the gradient domain explicitly considers the relationships between neighboring pixel points. Therefore, Gradient face has more discriminating power than the illumination insensitive measure extracted from the pixel domain. In order to extract illumination insensitive measure from gradient domain, we must refer to the Reflectance Model which can be expressed as the following:,,, Where I(x,y) is image pixel value, R(x,y) is the reflectance and L(x,y) is the luminance at each point (x,y). Here, the nature of L(x,y) is determined by the lighting source, while R(x,y) is determined by the characteristics of the surface of object. Therefore, R(x,y) which can be regarded as illumination insensitive measure. We have the following theorem by studying the relationships between the components of gradient domain: Theorem 1: Given an arbitrary image I(x,y) taken illumination condition, the ratio of y-gradient of,, / to x-gradient of,, / is an illumination insensitive measure. In practical application, the ratio of y-gradient of image to x-gradient of image might be infinitude derived by zero value of x-gradient of image. Thus, it cannot be directly

3 used as the illumination insensitive measure. These considerations lead us to defining Gradient faces as follows: Definition 1: I be an image under variable lighting conditions, then Gradient faces (G) of image I can be defined as:, 0,2 where and are the gradient of image I in the x,y direction respectively. To compute the gradient stably, we suggest to smoothen the image first with Gaussian kernel function. With a convolution-type smoothing, the numerical calculation of gradient is much more stable in calculation. Furthermore, we pointed out that the main advantage for using Gaussian kernel is twofold: (a) Gradient face is more robust to image noise and, (b) it can reduce the effect of shadows. The Gradient faces method is summarized in algorithm 1. Figure 2 shows the two steps of this preprocessing method and the images before and after preprocessing process. Algorithm 1. Demonstration of the Gradient Faces method. Figure 2. DoG + Gradient Face C. Wavelet Norm/Wavelet De-noising In Wavelet-based illumination normalization, the waveletbased image analysis decomposes an image into approximate coefficients and detail coefficients. Contrast enhancement can be done by histogram equalization of the approximation coefficients, meanwhile edge enhancement can be achieved by multiplying the detail coefficients with a scalar (>1),In our experiments we find the scalar 1.2 gives best results. A normalized image is obtained from the modified coefficients by inverse wavelet transform. In Wavelet-based image de-noising, the wavelet-based image de-noising approach is exploited to obtain an illumination invariant representation of the facial image. The technique starts with the modified imaging model of the retinex theory given by the following equation:,,, Where, log,,, log, and, log,. In fact, There are two reason for transforming image into the logarithm domain. The first one, In order to apply the wavelet-based approach to extract illumination invariant R. the other reason is that the logarithm of the luminance is a crude approximation to the perceived brightness, hence logarithm transform can partly reduce the effect of lighting. Based on the "common assumption", L varies slowly while R can change abruptly, L may be regarded as low frequency part of signal I and R is high frequency part of signal I. Under the assumption that the key facial features (R) are high frequency phenomena equivalent to noise in the image de-noising model, we propose to estimate the luminance, by the wavelet de-noising model and then to extract the illumination invariant reflectance, in accordance with the following equation:, log, log, Let us denote the wavelet coefficient of the input image, log, as,,, where W stands for the 2D Discrete Wavelet Transformed (DWT) operator; and, similarly, let,, denotes the matrix of wavelet coefficients of the luminance,. The 2D DWT decompose the image into approximation coefficients, LL (low-low) and detailed coefficients; LH (low-high), HL (high-low) and HH (high-high). The HH sub band gives the details of the image; the HL sub band gives the horizontal features while the LH sub band represents the vertical structures. The LL sub band is low resolution residuals consisting of low frequency components. The estimate of the luminance in the wavelet domain Y(x,y) is then obtained by modifying the detail coefficients of X(x,y) using the so-called soft thresholding technique and keeping the approximation coefficients unaltered. Here, the soft threshold procedure for each location (x,y) is defined as:,,,,,,, 0,, where X s (x,y) denotes one of the three sub-bands generated by the detail DWT coefficients (either the lowhigh (LH), the high-low (HL) or the high-high (HH) subbands, i.e.,, ), Y s (x,y) stands for the corresponding soft threshold sub-band and T represent a predefined threshold. It is clear that for an efficient rendering of the facial images, an appropriate threshold has

4 to be defined. We propose to compute the threshold T as follows:, where the standard deviations σ and σ x are robustly estimated from:,,,,0 In the above expressions mad denotes the mean absolute deviation and var denotes the variance. Note that the noise variance σ 2 is estimated from the HH sub-band, while the signal standard deviation σ x is computed based on the estimate of the variance of the processed sub-band X s (x,y) for,,. Algorithm 2. Demonstration of the wavelet-based de-noising approach Figure 3. Wavelet De-noising + HE For an optimal implementation of the presented denoising procedure, we suggest using a value of λ somewhere in the range from 0.01 to Once, all three detail coefficient sub-bands have been threshold, they are combined with the unaltered approximate coefficient subband to form the de-noised wavelet coefficient matrix Y(x,y). The estimate of the luminance in the spatial domain is ultimately obtained by applying the inverse DWT to the wavelet coefficients in Y(x,y), and can be used to compute the illumination invariant reflectance. The wavelet-based image de-noising approach is summarized in algorithm 2. In figure 3, the result of two steps preprocessing method using Wavelet De-noising and Histogram Equalization is shown. The input and output images of this preprocessing method is showing as well. III. FEATURE EXTRACTION Feature extraction is the task of reducing the high dimensional training data to a set of features to investigate properties (morphological, geometric etc.) of the data [4], [11]. Features are used by recognition approaches to differentiate between faces of different identities. During the feature extraction process the dimensionality of data is reduced. This is almost always necessary, due to the technical limits in memory and computation time. A good feature extraction scheme should maintain and enhance those features of the input data which make distinct pattern classes separate from each other. At the same time, the system should be immune to variations produced both by the humans using it and the technical devices used in the data acquisition stage. Linear discriminant analysis (LDA) is powerful method used for feature extraction and data reduction. It has been widely used in most of pattern recognition applications. However, a critical issue using LDA, particularly in face recognition area, is the Small Sample Size (SSS) Problem. This problem is encountered in practice since there are often a large number of pixels available, but the total number of training samples is less than the dimension of the feature space. To overcome this limitation many LDA-based methods have been proposed [9], [10]. within this paper, we have used LDA-based method namely regularized Linear discriminant analysis (R-LDA) to deal with small sample size scenarios. The basic idea of R- LDA is to find a linear transformation such as the feature clusters are most separable after the transformation. This is done as follow: given a face image matrix A of size mxn, we construct a vector representation by concatenating all the columns of A to form a column vector x of dimension mxn. Given a set of training vectors x i ; i = 1, 2,..., M for all persons,,,, TABLE 1 RECOGNITION RESULTS OF OUR METHOD COMPARED WITH ANOTHER METHODS THOSE HANDLE THE ILLUMINATION VARIATION PROBLEM Feature Illumination Preprocessing Methods Extractor SQI LDCT Correlation Eigenfaces LBP LGBP Fisherfaces LEC R-LDA Feature Extractors: LGBP Local Gabor Binary Pattern LEC Local Ensemble Classifier Illumination Preprocessing Methods: SQI: Self-Quotient Image LDCT: Logarithmic Discrete Cosine Transform TT: Tan & Triggs [7] TT DOG/CE DOG/GF Norm/WD

5 where X represent all the face images. The matrix X is composed of C classes; in each class we have m i individuals, Then, we express the between-class scatter matrix S B as:. with:,,,,,, where µ i is the mean of the i th class and µ is the mean of all classes. they can be calculated as follows: Next, we determine the d eigenvectors of, denoted as,,, where d<=c-1. after that, we calculate the first d most significant eigenvectors U d of S B and their corresponding Eigen values (Λ B) by:, Λ B U T S B U Let Λ, we find the eigenvectors of denoted as,, sorted in increasing eigen value order, S w is the within-class scatter matrix which is denoted as: then, we choose the first D eigenvectors of P ( D d). Let P D and Λ w are the obtained eigenvectors matrix and their corresponding diagonal eigenvalues matrix, respectively. In R-LDA the projection vector W are computed as follows: Λ where I is the (DxD) identity matrix, and η is a regularization parameter in the range 01. In general, we choose η=1. Finally, we project the training vectors as follows: Given a test face image Test, first we transform it on a column vector T, then use Eq. (1) to get the feature vector, then we calculate the similarity between vectors by using Cosine Distance. IV. EXPERIMENTAL RESULTS A. Database Since this proposed face recognition system mainly deals with illumination problem, we used Extended Yale-B face database. The Extended Yale-B face database provides an excellent testing platform for extreme variation in illumination [17]. It consists of still images, in frontal pose, for 38 subjects, each having 64 images captured under different illumination conditions. The total number of images in the database is 2432 images, These images are divided into five subsets according to the angle between the light source direction and the central camera axis 12,25,50,77,90 as shown in Figure 4. The images are resample to a fixed size of 128 x 128 for the purpose. B. Experiment 1: Recognition without occlusion In this experiment, the ideal images (subset1) were used as reference images to form the gallery set and all the remaining images were used for testing. Table 1 shows the results compared with another existing methods that handle illumination variation problem[12]. As shown in table 1, Experimental results show that, a good recognition rate is achieved by R-LDA using these 3 methods of preprocessing. Compared them with other methods, our proposed methods got a highest recognition rate. C. Experiment 2: Recognition with small sample size Within this experiment, we have tested each method based on small size of the training set (one and two images), the results demonstrated in table 2 show that the system work well even with small sample size scenario. TABLE 2 RECOGNITION RESULTS OF FACE BASED ON SMALL SAMPLE SIZE IN THE TRAINING STAGE No. of Training Set DoG/ CE DoG/ GradientFaces Norm/Wavelet Denoising D. Experiment 3: Recognition with random block occlusion In this experiment, we test the robustness of our proposed methods to random block occlusion. Subset 1 and subset 2 of the Extended Yale B database are used for training process, while subset 3 is used for testing stage. Each testing sample will be inserted an unrelated image as a block occlusion, and the blocking ratio is changing from 10% to 50%. Table3, shows the results compared with another existing methods that handle occlusion problem. As shown in table 3, We can see that when the block occlusion ratio is low, all methods can achieve good recognition accuracy; when the block occlusion ratio increases, the accuracy of our proposed method scan still have good results up to more than 90% even with 50% block occlusion. E. Experiment 4: Recognition with important facial occlusion In this experiment, the occlusions cover important facial parts like eyes, nose and mouth. The key facial parts are blackened out (likening them to occlusion).the tests have been done on more challenging subset namely, subset 5, whereas the subset 1 was selected as training images. The occlusion ratio was 23.4% of the image, Figure 5 shows some of the sample face images with occlusion. The recognition results of the face are tabulated in table 4.

6 TABLE 3 RECOGNITION RESULTS OF FACE USING VARIOUS RANDOM OCCLUSION RATIOS, COMPARED WITH DIFFERENT APPROACHES Occlusion Ratio 10% 20% 30% 40% 50% SRC [18] 100% 99.8% 98.5% 90.3% 65.3% CRC_RLS [21] 99.8% 93.6% 82.6% 70.0% 52.3% GSRC [19] 100% 100% 99.8% 96.5% 87.4% RSC [20] 100% 100% 99.8% 96.9% 83.9% DoG+CE/R-LDA 98.9% 98.9% 97.6% 94.7% 93.2% DoG+GF/R-LDA 98.2% 97.6% 97.4% 94.3% 90.3% HE/WD/R-LDA 100% 99.8% 99.6% 98.5% 98.2% V. CONCLUSION In this paper, we have presented three robust illumination preprocessing methods Difference of Gaussian (DoG) +Contrast Equalization(CE), DoG+ GradientFace(GF), and Histogram Equalization + Wavelet-based image denoising (HE+WD). These methods effectively eliminate unwanted illumination effect and enhance the local features of facial images, which play a vital role in recognition. The presented methods are quite different from each other; the first method (DoG+CE) is very simple method which is used in order to enhance the contrast and the suppression of frequencies that cause noise (low frequencies) and changes in lighting (high frequencies), this method is applied directly on the image in pixel domain and does not seek to obtain an illumination insensitive measure. The second method (DoG+GF) was focused on finding illumination insensitive measure from gradient domain, the gradient domain explicitly considers the relationships between neighboring pixel points such that it is able to reveal underlying inherent structure of image data unlike the methods which are implemented in pixel domain ignore the underlying relationships between neighboring pixel points. this method was combined with DoG filter in order to reduce the effects of noise and lighting variations. Using and Gradient Face after DoG improves the recognition rate about 2%. The third method (HE+WD) exploited the wavelet denoising model to obtain an illumination invariant representation of the facial image in the wavelet domain. Applying histogram equalization prior to the de-noising model would result in improving the image's contrast and TABLE 4 RECOGNITION RESULTS OF FACE USING OUR PROPOSED METHODS ON THE EXTENDED YALE B DATABASE WITH KEY FACIAL LANDMARK OCCLUDED 23.4% OF THE IMAGE Without Occlusion With Occlusion DoG+CE /RLDA 99.6% 92.3% DoG+GF/RLDA 99.3% 96.1% Norm/WD /RLDA 99.6% 96.9% redistributing the pixel intensities equally. this procedure adds to images variability needed to successfully discriminate between different subjects. Using this method the recognition rate is improved especially in occlusion case. Both second and third methods depending on imaging model to obtain the illumination invariant for face recognition. However, In any face recognition system, the preprocessing methods is not enough to get significant results (high recognition rate). Therefore, it's important to choose robust and effective mechanism to extract distinctive features and classified it well. the combination of R-LDA and cosine distance to extract and classify features provides very promising results. R-LDA extracts features for each class which are more related to the entire class, since it minimizes the variance characteristic within class scatter (minimizing the distance within class scatter) and increases the variance between class scatter (maximizing the distance between class scatter), even if there are random occlusions in the testing images, R-LDA provided sufficient measurements are extracted. Experiments were performed based on the Extended Yale B facial databases. Experimental results show that very high recognition rates were achieved by R-LDA and Cosine distance recognition with the proposed preprocessing methods. we have demonstrated the effectiveness of the proposed methods compared with another methods that handle illumination variation and occlusions. The proposed algorithms in all experiments was conducted under MATLAB programming environment on a PC with Intel R Core (i5) 2.1 GHz CPU and 2.99 GB RAM. In the mean time, we have evaluated the training time of each proposed algorithm, the training time is also a critical factor in many applications. we have used one image per person in the database. thus, the total number of training images in this evaluation is 38 images. We find out that, the spent time to process and extract features in the training stage for both DoG+CE and DoG+GF is around 0.3 second. while the spent time to process and extract features in the training stage for HE+WD is more than 0.5 second. as observed, the DoG+CE and DoG+GF are more appropriate than HE+WD for real-time systems. REFERENCES [1] W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld, Face recognition: A literature survey,acm Computing Survey, 35(4), 2003, pp [2] R. Jafri and H. R. Abrania, A survey of face recognition techniques, Journal of Information Processing Systems, 5(2):41-68, [3] Y. Adini, Y. Moses and S. Ullman, Face Recognition: the problem of compensating for changes in illumination direction, IEEE Trans. Pattern Analysis and Machine Intelligence, 19(7): ,1997. [4] A. Shashua and T. Riklin-Raviv, The quotient image: Class-based rerendering and recognition with varying illuminations, IEEE Trans. Pattern Analysis and Machine Intelligence, 23(2): ,2001. [5] T. Zhang, B. Fang, Y. Yuan, Y. Y. Tang, Z. Shang, D. Li and F. Lang, Multiscale facial structure representation for face recognition under varying illumination, Pattern Recognition, 42(2): , [6] T. P. Zhang, Y. Y. Tang, B. Fang, Z. W. Shang and X. Y. Liu, Face recognition under varying illumination using gradientfaces, IEEE Trans. Image Process, 18(11): ,2009.

7 [7] X. Tan and B. Triggs, Enhanced local texture feature sets for face recognition under difficult lighting conditions, IEEE Trans. Image Process, 19(6): , [8] C.N Fan, F.Y Zhang, Homomorphic filtering based illumination normalization method for face recognition, Pattern Recognition Letters, 32(10): , [9] L. F. Chen, H. Y. M. Liao, M. T. Ko, J. C. Lin and G. J. Yu, A new LDA-based face recognition system which can solve the small sample size problem, Pattern Recognition, vol. 33(10): , [10] J. Lu, K. N. Plataniotis and A. N. Venetsanopoulos, Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition, Pattern Recognition Letters, 26, pp , [11] A. Globerson, N. Tishby, I. Guyon, and A. Elisseeff, Sufficient dimensionality reduction, Machine Learning Research, [12] H. Han, S. Shan, X. Chen and W. Gao, A comparative study on illumination preprocessing in face recognition, Pattern Recognition, 46: , [13] D. J. Jobson, Z. Rahman, and G. A. Woodell, A multi-scale retinex for bridging the gap between color images and the human observation of scenes, IEEE Trans. Image Process., vol. 6, no. 7, pp , Jul [14] T. Ahonen, A. Hadid, M. Pietikainen, Face description with local binary patterns, Application to face recognition. IEEE TPAMI 28(12) [15] B. Wang, W. Li, W. Yang, Q. Liao, Illumination Normalization Based on Weber s Law with Application to Face Recognition, IEEE Signal Processing Letter 18, , [16] S. Shan, W. Gao, B. Cao and D. Zhao Illumination normalization for robust face recognition against varying lighting conditions, Proc. IEEE Int. Workshop Anal. Model. Faces Gestures, , [17] K. Lee, J. Ho, and D. Kriegman, Acquiring linear subspaces for face recognition under variable lighting, IEEE Trans. Pattern Analysis & Machine Intelligence, vol. 27, no. 5, pp , [18] J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, Robust face recognition via sparse representation, IEEE Trans. Pattern Analysis and Machine Intelligence, 31(2): , [19] M. Yang and L. Zhang, Gabor Feature based Sparse Representation for Face Recognition with Gabor Occlusion Dictionary, Proc. European Conf. Computer Vision, [20] M. Yang, L. Zhang, J. Yang and D. Zhang, Robust sparse coding for face recognition, In CVPR, [21] L. Zhang, M. Yang and X. Feng, Sparse representation or collaborative representation: Which helps face recognition?, In ICCV, Figure 4. Image sample of each subset Figure 5. Image sample of key facial parts occluded