*School of Electronics Engineering, Kyungpook National University, Daegu, Korea

Transcription

1 FACE RECOGNITION USING MULTI-LAG DIRECTIONAL LOCAL CORRELATIONS Nam Chu/ Kim, Ying Ai Ju, Hyun Joo So, and Mi Hye Kim * *School of Electronics Engineering, Kyungpook National University, Daegu, Korea nckim@knu.ac.kr ABSTRACT This paper presents a face recognition method using a set of efficient local texture features, called multi-lag directional local correlations (MDLCs). They measure the intensity similarity between a local region and each of the counterparts which are multi-lag directional vectors distant from it, which is a sort of local correlation coefficient that is well normalized and bounded. Each of the MDLC images extracted from a facial image is then low pass filtered in the global 2D DCT (discrete cosine transform) domain, which reduces not only feature dimension but also noisy disturbance obstructing elaborate face recognition. The DCT coefficients retained from low pass filtering are all fused into a ID feature vector for an input of a stabilized whitened cosine (SWC) distance classifier. The performance of the MDLC features is compared with those of Gabor wavelet, LBP (local binary pattern), gradientfaces, the fusion of BDIP (block difference of inverse probabilities) and BVLCs (block variation of local correlation coefficients). Experimental results for six facial databases (DBs) with a single training image per person and with multiple training images show the MDLC features yield almost the best performance robust to variations of expression, lighting, and aging among the discussed features. Index Terms- Face recognition, local texture feature, multi-lag directional local correlation 1. INTRODUCTION Face recognition is to identify from a given facial image one of persons whose facial images are enrolled in a stored DB. In capturing a facial image of a subject for verification, different 3D (three dimensional) motion of his or her head leads to variations of pose and different motion of the facial muscles to variations of expression. Including these variations, different ages and illumination conditions may yield some-what unlike facial images compared with a stored facial image of the same one. Therefore, they may be main factors of degrading the performance of face recognition. Over the last decade, a lot of research efforts have contributed to developing face recognition methods more robust to variations of expression, lighting, aging, and pose [1]-[7]. The main part of typical face recognition systems consists of feature extraction and classification. In feature extraction, shape and/or texture features are extracted. This paper concentrates on extraction of local texture features. We may have a question about what texture feature is indeed most essential to face recognition. Is the answer one of conventional texture features, one of their combinations, or a new texture feature to be developed? Up to now various local texture features have been employed in face recognition. Some of them are listed as Gabor wavelet [8], [9], LBP (local binary pattern) [10]-[12], local ternary pattern (LTP) [4], gradientfaces [5], and the fusion of BDIP (block difference of inverse probabilities) and BVLCs (block variation of local correlation coefficients) [13]. It has been reported in the related studies that all of them may produce better performances than raw image itself. Most of them also showed good results in texture classification, image retrieval, and facial expression. It however is not easy to find texture features that operate well all in various abnormal or complex situations. We thus need to develop texture features robust to diverse circumstances. In this paper, we propose a face recognition method using a set of efficient local texture features, called multi-lag directional local correlations (MDLCs). The set of MDLC features is an extended version of local correlations used in the computation of BVLCs. The extracted MDLCs are then transformed by 2D global DCT and low pass filtered in the DCT domain. The function of 2D global DCT filtering is to remove noisy variations negative to recognition as well as to reduce feature dimension. The filtered DCT coefficients finally enter a whitened cosine (WC) distance classifier, which is known to yield excellent recognition performance [14]-[16]. The performance of the proposed method is evaluated for three DBs of FERET-UP, FERET-ID, and Yale with single training, and three DBs of Yale B, Weizmann, and ORL with multiple training. 2. PROPOSED FACE RECOGNITION The overall block diagram of the proposed face recognition system is given in Fig. 1. A facial image first enters MDLC /11/$ IEEE

2 Image Classification result Fig. 1. Overall block diagram of the proposed face recognition svstem. extraction, which produces MDLC images. A DCT-domain low pass filtering then yields the filtered DCT coefficients from the MDLC images. Finally, the fused vector of filtered DCT coefficients goes through a stabilized WC (SWC) distance classifier, which provides the classification result. *'(a) o o o oop o OO flbo 0 )/0 0 0,' = 2 r 3 d o b o (b) Fig. 2. Example of a set of unit direction vectors and configuration of multi-lag directional points. (a) Eight unit direction vectors and (b) configuration of multi-lag directional points with K=3 and L=8. o 2.1. MDLC Features Consider an image {Jp}PEP, where Ip denotes the intensity at a pixel p of the image and P the set of all the pixels. Let Rp be a local region centered at p. Then we can consider the following conditional mean and variance, which are defined as (a) f.jp = E[ I q I 'liq E Rp] (J' = E[(J q - f.jp) 2 l'liq E Rp]. (1) (2) We use the two local statistics to define a directional local correlation (DLC) along a vector rd at p as E[ (Iq+rd - f.jp+rd )(Iq - f.jp) I 'Ii q E Rp] (J' p+ rd(j' p (3) where d denotes a unit direction vector and r the Manhattan distance between pixels q and q + rd. Fig. 2 (a) shows an example of a set of unit direction vectors. The DLC measures the intensity similarity between the local region centered at p and that centered at the pixel rd distant from p. It is a sort of correlation coefficient whose absolution is less than or equal to unity. It is usual in texture classification that since homogeneous textures of the same class often contain some local disturbance yet even though they have the similar global statistics, the negative local disturbance should be further removed. In [17]-[19], for this purpose, a local statistics of DLCs, called BVLC, is computed for texture classification and image retrieval, which is written as (b) Fig. 3. MDLC images of four lags, four directions, and 3x3 local region for an image of a gallery set in FERET. (a) Raw image, (b) MDLC images. Different rows and columns mean different lags and directions, respectively. than their local or global statistics to perceive local differences in facial shapes. Considering this point, we suggest the MDLCs as useful features for face recognition, which are expressed as p yr = m ax[pr d ] _ mi n[prd ] ded p ded p (4) where D denotes the set of four or eight direction vectors. We see that the BVLC stands for the maximum deviation of DLCs. Finally, the global mean and standard deviation of BVLCs over all the pixels determine the classification performance. On the other hand, in face recognition, the directional local behavior of facial features should be more respected where L means the number of lags and K the number of unit direction vectors. Fig. 2 (b) shows the configuration of multi-lag directional points in case of L=3 and K=8. Note that we get KL MDLC images from a raw image so that they have a dimension KL times higher than that of its raw image. Fig. 3 shows the MDLC images for an image of a galley set or training set in FERET DB. Since a 3 x 3 window is often

3 chosen as a local region, the MDLC images are shown to be somewhat noisy. We can select a larger window so that they may become less noisy but lose finer local characteristics DCT-Domain Low Pass Filtering In face recognition area, 2D global DCT has already been utilized for the reduction of feature dimension [20] or 2D local DCT for the extraction of texture features [21]. However, we adopt a DCT-domain low pass filtering to reduce the feature dimension as well as noisy variations. Consider the DCT coefficients for the MDLC images given in (5), where cr d (u,v) represents a DCT coefficient of frequency (u, v), especially c rd (I,I) a DC coefficient, and F the set of all DCT frequencies. Each of the low pass filtered coefficients is expressed as subtracted by the mean vector of all training feature vectors, 11'11 the vector norm, and W = wk 1I2 the whitening transform matrix. The two matrices w and A are an eigenvector matrix and a diagonal eigenvalue matrix for the covariance matrix of all training feature vectors, respectively. That is, they are expressed as W = [<p], "', <Pn] and A = diag[a], "', An], where <Pi is the eigenvector corresponding to the ith largest eigenvalue Ai for i = 1, " ', n. In measuring the WC distance for real feature vectors, W may bring about two problems: the singularity due to eigenvectors related to noisy eigenvalues and the deterioration due to eigenvectors which are related to facial variations negative to face recognition. In order to solve the problems, we introduce into (9) the fifth assumption that the problematic eigenvalues should be discarded. As a result, we obtain the SWC distance which is given as dswc(f, g ) (W fl(w g ) IIW fllll w g ll (10) Ard ( ) {c rd (u, v), 1:::; u,v:::; N, u +v < (1 +a)n c u,v = 0, otherwise (7) Wm = [ A /2 ] =[ Jx; j, Jx; 2 ' "', k m 1 (11) where the zonal shape is determined considering the energy distribution of DCT coefficients. The shape becomes triangular as a approaches 21 N but rectangular as it goes to one. An intermediate shape between the two extreme ones is chosen in our experiments because of its better results. The coefficients retained from DCT filtering contribute to forming the ID feature vector as an input for a SWC distance classifier, which is formulated as f = d d 2d 2d 2-1 d 2-1 d [f 1, "', f K, f 1,"', f K,"', f 1,"', f K] (8) where f is an input of an SWC distance classifier and f 2i d j the result of one-dimensionally stacking the retained DCT coefficients of the ith lag and the jth direction SWC Distance Classification The combination of whitened principal component analysis (WPCA) and cosine distance chosen here among diverse classification schemes is also called WC distance classifier in [14], which showed that the WC distance can be derived from the Bayes decision rule with three ordinary assumptions and the fourth additional one that the whitened feature vectors are normalized to unit norm. The WC distance is formulated as where f and g denote two feature vectors of dimension n (9) (12) where Am is a m x m (m::s n) matrix, 0 stands for a (n - m) x m zero matrix, and the m remaining eigenvalues are all significant and in a descending order. The eigenvalue discarding technique is not new but has been commonly applied in face recognition area. 3. EXPERIMENTAL RESULTS As test DBs for performance evaluation, we select six facial DBs: FERET-UP, FERET-ID, Yale, Yale B, Weizmann, and ORL. The first three have a single training image for each person and the last three multiple training images. The overall description on each DB is given below: 1) FERET-UP: It originates from the FERET September 1996 test set which is a subset of FERET DB [22]. In the subset, there are a gallery set for training, which contains 1196 images of 1196 persons, and four probe sets, Fb, Fc, Dupl, and Dup2, for expression, lighting, aging, and long aging variant tests, respectively. All images are normalized by affine transformation and cropped to images of 128 x 128 with the inter-eye distance of 70 pixels and the eye centers (29, 34) and (99,34) [15]. As a result, each image becomes a closeup view to four central facial components of eyebrows, eyes, noses, and mouths. 2) FERET-ID: We make another DB where all image boundaries tightly enclose even peripheral facial components (hair, ears, and jaw) needed for personal identification. It is obtained using the same affine

4 90. expression (Fb) 75.lightiog.(Fc) aging (Dup I) ---e- [ong aging (Dup2) 70L-c-- ---: - ==;=:=::==:'J Fig. 4. Performance of the proposed method according to the number of lags. Fig. 5. Performance of the proposed method according to the size of a local region. transformation and cropping of FERET-UP except the inter-eye distance of 30 pixels and the cropped size of 112 x 92 with the eye centers (31, 55) and (61,55). 3) Yale: It contains 165 images for 15 persons, each of which has 11 images of different expressions [23]. We select a normal expression image for each person as training images and only five expressions (happy, sad, sleepy, surprised, and winking) for each person as test images. The facial parts of images are cropped and resized to images of 112 x 92 pixels. 4) Yale B: It consists of 450 images of 10 persons with frontal pose and normal expression, which are chosen from the original Yale B as in [16]. They are separated into four subsets according to the angle between the light source direction and the camera axis. The first two subsets are used for training and all subsets for lighting variant test. The facial parts of images are cropped and resized to images of 112 x 92 pixels. 5) Weizmann: It includes 1170 images of 26 persons which are selected from the original Weizmann as in [16]. They are divided into four sets: a training set with 130 images of 26 persons and three test sets for expression, lighting, and lighting plus expression tests, respectively. All images are resized to images of 112 x 92 pixels without cropping. 6) ORL: It is also called AT&T DB and contains 400 images of 40 persons, each of which is 112 x 92. The images were taken at different times, varying the lighting, and facial expressions [24]. We select oddnumbered images for training and even-numbered images for testing. The performance here means the top recognition rate given The performance here means the top recognition rate given by the best selection among various ones for the number m of remaining eigenvalues in (12). Fig. 4 shows the performance of the proposed method according to the number of lags L. In the experiment, the MDLCs are extracted with a 3 x 3 window in half of 8 directions for the probe sets in FERET-UP. The variances in (3) are limited to 0.1 to avoid the unstability. The DCT filtering parameter is set to be a= 116. The case of L=4 is shown to give the best result for all probe sets, which tells us that the further large lags are not helpful for face recognition. Fig. 5 shows the performance of the proposed method according to the size of a local region IRpl. The region size stands for the number of pixels in the local region. In case of IRpl = 1, we set mean and variance to be zero and one, respectively, since we cannot find such statistics. The experiment is identical to one in Fig. 4, except fixing L = 4 and varying IRpl. We see that a local region of IRpl =9 or a 3 x 3 window is the best and so local regions of larger sizes may lose the valuable local information on face recognition. For performance comparison, we implement not only the proposed methods using MDLC features but also the methods using raw image, Gabor wavelet [15], LBP [25], gradientfaces [5], the fusion of BDIP and BVLCs (hereafter denoted as BDIP-BVLCs) [13]. Fig. 6 shows the performance of various face recognition methods for FERET-UP. In this figure, the Gaussian filtering parameter of gradientfaces is set 0-= 0.5 and the parameters of MDLC features the same ones as in Fig. 4 except fixing L=4. Table 1 contains the dimensions of various feature vectors entering SWC distance classifier. The dimension of MDLC features, depending on the DCT filtering parameter, is 1,248, 2,144, and 3,952 for N=12, 15, and 20, respectively, because 78, 134, and 247 DCT coefficients are retained for each of 16 MDLC images. In Fig. 6, gradientfaces outperform raw image except for lighting test and BDIP-BVLCs also yield the gains above 7%, but both of them do not reach the rate above 90% for expression and lighting tests and the rate above 75% for aging and long aging tests. LBP and Gabor wavelet features produce excellent rates higher than 95% for expression and lighting tests but give deteriorated ones less than 86% for aging and long aging tests. However, MDLC features show the best results for all tests and a good rate greater than 90% even for long aging test, while they have the least dimension among the implemented features. Fig. 7 shows the performance of various face recognition methods for FERET-ID. All parameters are the same as in Fig. 6, except setting N= 15, 20, 30, and 30 for Fb, F c, Dup 1, and Dup 2, respectively. Compared to the results of Fig. 6, most of the features yield the improved

5 Table 1. Dimensions of various feature vectors entering SWC distance classifier for FERET-UP. Feature Dimension Raw image 16,384 Gradientfaces 16,384 BDIP-BVLCs 49,152 Gabor 10,240 LBP 15,104 N= 12 1,248 MDLCs N=15 2,144 N=20 3,952 rate[%) I 00?_ n?? I expression lighting aging long aging o Raw image o Gradientface BDIP-BVLCs [] Gabor OLBP.MDLC(K 4) Fig. 6. Performance of various face recognition methods for FERET-UP. performance for expression and lighting tests, but the severe degradation for aging and long aging tests. From these results, we can see that the information on peripheral facial components is helpful for the former tests but operates negatively for the latter tests. Gradientfaces are shown to yield the excellent rate above 96% for expression test but not so good one for the other tests. Especially MDLC features show the rates above 99% for the former tests and the rates above 80% even for the latter tests, which means that MDLCs are the best and least sensitive to expression, lighting, aging, and long aging. Fig. 8 shows the performance of various face recognition methods for Yale, Subset 4 in Yale B, Subset 3 in Weizmann, and ORL. Note that all DBs except Yale have multiple training images for each person. All parameters are the same as in Fig. 4, except fixing K=3 and N=15, 45, 15, and 10 for Yale, Yale B, Weizmann, and ORL, respectively. For Yale DB, raw image attains the perfect rate and so do Gabor and MDLCs. However, gradientfaces, BDIP-BVLCs, and LBP are inferior even to raw image for this DB. For lighting test (Subset 4) in Yale B, gradientfaces give the perfect rate, followed by BDIP-BVLCs and MDLCs above 98%, but Gabor wavelet and LBP features are inferior even to raw image feature. For expression plus lighting test (Subset 3) in Weizmann, MDLC, Gabor wavelet, LBP, and BDIP-BVLCs provide the rates above 99% and the first two further accomplish the perfect rate. For ORL DB, Gabor wavelet shows excellent performance above 99% and MDLCs reach the perfect rate. On the whole, MDLC features are shown in Fig. 8 to yield the best for all DBs except for Subset 4 in Yale B. Besides, we observe that all the features except raw image show the rate of % for Subsets 1 and 2 in Weizmann and Subsets 1 3 in Yale B. For these subsets, the raw image feature also appears relatively inferior to the others, yielding 99.23%, %, and 99.17% for Subsets 1 and 2 in Weizmann and Subset 3 in Yale B, respectively, and the perfect rate for Subsets 1 and 2 in Yale B. 4. CONCLUSION We have proposed a face recognition method using MDLC rate[%) :: , 8 ' Raw image :::: I ----r II o Gradientface LUuaKU -W ua expression lighting aging long aging BDIP-BVLCs -- [] Gabor olbp.mdlc(k 4) Fig. 7. Performance of various face recognition methods for FERET-ID. rate[%) Wcizmanll n : Yale B ORL Yale [] Raw image [] Grodicntfacc -BDIP-BVLCs [] Gabor o LBP -MDLC(K-3) Fig. 8. Performance of various face recognition methods for Yale, Yale B, Weizmann, and ORL. features. The MDLCs, which are a sort of local correlations, are more proper for face recognition than BVLCs since they hold directional facial information in multi-lags. The global DCT filtering was shown to be very effective in reducing the local variations negative to recognition, especially in DBs of single training. The MDLCs attained the best performances for all DBs except for Subset 4 in Yale B, which means that the MDLCs are least sensitive to expression, lighting, and aging among the tested features. Their robustness to lighting may be because they are well locally normalized and bounded. The reason of insensitiveness to expression and aging may come from the point that they utilize not the raw local information itself but

6 the directional local correlations instead. 5. REFERENCES [1] W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, "Face recognition: A literature survey, " ACM Computing Surveys, vol. 35, no. 4, pp , Dec [2] C. K. Hsieh, S. H. Lai, and Y. C. Chen, "An optical flow-based approach to robust face recognition under expression variations, " IEEE Trans. Image Process., vol. 19, no. l, pp , Jan [3] I. A. Kakadiaris, G. Passalis, G. Toderici, M. N. Murtuza, Y. Lu, N. Karampatziakis, and T. Theoharis, "Three-dimensional face recognition in the presence of facial expressions: An Annotated deformable model Approach, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 4, pp , Apr [4] X. Tan and B. Triggs, "Enhanced local texture feature sets for face recognition under difficult lighting conditions, " IEEE Trans. Image Process., vol. 19, no. 19, pp , Jun [5] T. Zhang, Y. Y. Tang, B. Fang, Z. Shang, and X. Liu, "Face recognition under varying illumination using gradientfaces, " IEEE Trans. Image Process., vol. 18, no. 11, pp , Nov [6] U. Park, Y. Tong, and A. K. Jain, "Age-invariant face recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 5, pp , May [7] V. Blanz and T. Vetter, "Face recognition based on fitting a 3D morphable model, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 9, pp. 1-12, Sep [8] C. Liu and H. Wechsler, "Independent component analysis of Gabor features for face recognition, " IEEE Trans. Neural Network, vol. 14, no. 4, pp , Jul [9] C. Liu, "Gabor-based kernel PCA with fractional power polynomial models for face recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 5, pp , May [10] X. Li, W. Hu, Z. Zhang, and H. Wang, "Heat kernel based local binary pattern for face representation, " IEEE Trans. Signal Process., vol. 17, no. 3, pp , Mar [11] T. Ahonen, A. Hadid, and M. Pietika, "Face description with local binary patterns: Application to face recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 12, pp , Dec [12] L. Wolf, T. Hassner, and Y. Taigman, "Descriptor Based Methods in the Wild, " in Proc. European Conference on Computer Vision, pp , Marseille, France, Oct [13]Y. Ju, H. J. So, N. C. Kim, and M. H. Kim, "Face recognition using local statistics of gradient and correlations, " in Proc. European Signal Processing Conf. 2010, Aalborg, Denmark, Aug [14] c. Liu, "The Bayes decision rule induced similarity measures, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, Jun [15]W. Deng, J. Hu, J. Guo, W. Cai, and D. Feng, "Robust, accurate and efficient face recognition from a single training image: A uniform pursuit approach, " Pattern Recognit., vol. 43, no. 5, pp , May [16]P. C. Hsieh and P. C. Tung, "A novel hybrid approach based on sub-pattern technique and whitened PCA for face recognition, " Pattern Recognit., vol. 42, no. 5, pp , May [17] Y. D. Chun, N. C. Kim, and I. H. Jang, "Content-based image retrieval using multiresolution color and texture features, " IEEE Trans. Multimedia, vol. 10, no. 6, pp , Oct [18]Y. D. Chun, S. Y. Seo, and N. C. Kim, "Image retrieval using BDIP and BVLC moments, " IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 9, pp , Sep [19]H. J. So, M. H. Kim, and N. C. Kim, "Texture classification using wavelet-domain BDIP and BVLC features, " in Proc. European Signal Processing Conf. 2009, pp , Glasgow, Scotland, Aug [20] Z. M. Hafed ad M. D. Levine, "Face recognition using the discrete cosine transform, " Int'!. J. Computer Vision, vol. 43, no. 3, pp , Jan 200l. [21]A. Amine, S. Ghouzali, and M. Rziza, and D. Aboutajdine, "Investigation of feature dimension reduction based DCT/SVM for face recognition, " in Proc. IEEE Symposium, Computers and Communications, pp , Marrakech, Morocco, Jul [22] P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss, "The FERET evaluation methodology for face recognition algorithms, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp , Oct [23] Yale face database < faces/ya lefaces.html>. [24] ORL face database < dtg! attarchive/facedatabase.html>. [25] X. Tan and B. Triggs, "Fusing Gabor and LBP Feature Sets for Kernel-based Face Recognition, " LNCS, vol , pp , 2007.