A Novel Scheme for Face Recognition and Authentication Against Pose, Illumination and Expression Changes *

Transcription

1 JOURNAL OF INFORMAION SCIENCE AND ENGINEERING 27, (2011) Short Paper A Novel Scheme for Face Recognition and Authentication Against Pose, Illumination and Expression Changes * YING-NONG CHEN 1, CHIN-CHUAN HAN 2, CHENG-ZU WANG 3 AND KUO-CHIN FAN 1 1 Department of Computer Science and Information Engineering National Central University aoyuan, 320 aiwan 2 Department of Computer Science and Information Engineering National United University Miaoli, 360 aiwan 3 Department of Computer Science National aipei University of Education aipei, 106 aiwan cchan@nuu.edu.tw In this paper, a novel scheme for face recognition or authentication is proposed against pose, illumination, and expression (PIE) variation using modular face features. A sub-image in low-frequency sub-band is extracted by a wavelet transform (W) to reduce the image dimensionality. It is partitioned into four parts for representing the local features and reducing the PIE effects, and the small image in a coarse scale is generated via the W without losing the global face features. Five modular feature spaces were constructed. he most discriminative common vectors in each feature space were found, and a nearest feature space-based (NFS-based) distance was calculated for classification. Finally, a weighted summation is performed to fuse the five distances. Experiments were conducted to show that the proposed scheme is superior to other methods in terms of recognition and authentication rates. Keywords: discrete wavelet transform, face recognition, face authentication, linear discriminant analysis, discriminative common vectors 1. INRODUCION Face recognition and authentication are two important and active research issues in computer vision applications such as military, secure control and crime prevention systems. Because of its affordability, a built-in camera is mounted on many mobile devices. Face images grabbed from cameras have become the most user-friendly biometric features in personal identification systems. Using face-based verification, the important data stored in mobile devices can be protected. However, the working environments for mobile users are quite varied. People aim the capturing device at their face with various poses and expressions and, furthermore, human faces are not evenly lit. Pose, illumina- Received January 19, 2009; revised April 21 & June 23, 2009; accepted August 5, Communicated by yng-luh Liu. * he work was supported by National Science Council of aiwan under Grant No. NSC E

2 370 YING-NONG CHEN, CHIN-CHUAN HAN, CHENG-ZU WANG AND KUO-CHIN FAN tion, and expression (PIE) usually create most critical problems in face recognition. hree stages, face representation, discriminative feature analysis, and data classification, comprise the identification process. In the first stage, effective features are extracted representing face images of one person with various expressions, illuminations, and poses. In stage 2, the best discriminative bases for dimensionality reduction are found by minimizing the scatter of data points. Metric distances between templates and an input sample are calculated for data classification in stage 3. Principal component (PC) is the most popular representation in an eigenspace [1]. he most representative and global features of face images are extracted by principal component analysis (PCA). Its main goal is to find the best representation of the original images. Sufficient information could be kept in low dimensional feature spaces. In addition, high dimensional face images are reduced into low dimensional feature vectors. Yang et al. [2] extended the 1D-PCA to 2D-PCA. Moreover, face images can be represented in vector forms using the kernel PCA [3], Gabor-based PCA [4], wavelet transform (W) [5], and discrete cosine transform (DC) [6] methods. All of them represent a whole face appearance as a feature vector, globally. Recently, a new face representation method is proposed based on the Laplacian matrix transform [7, 8]. He et al. [7] proposed a manifold learning method called Laplacianface which can preserve the local topology and information for recognition. Furthermore, Yan et al. [8] and Hu [9] used the priori class information in the discriminant analysis for recognition improvement. In their approaches, a single feature vector is globally represented by the whole face image. hey preserve the local structure in the eigenspaces. In contrast, a descriptor extracted from local facial regions is another representation using local features. In [10], facial components are first represented in modular eigenspaces. Moreover, facial landmarks are extracted for describing the facial components using the Gabor filter. Ahonen et al. [11] describe the appearance of local facial regions based on local binary pattern texture features. en facial components are detected and combined into a feature vector by ignoring the cheeks [12]. According to their conclusions, the component-based face recognition system outperforms the global approach. However, two critical problems should be solved in their approach: accurately locating the facial components (i.e. region of interested, ROI) and effectively fusing the features of ROIs. Linear discriminant analysis (LDA) is another popular technique for discriminant feature analysis. Different from the PCA-based representation, the class information is considered in the analysis process. he most discriminant bases are found to project the original feature vectors into the discriminant spaces for identification. Some variants of LDA, such as F-LDA [13], D-LDA [5], K-DDA [14], FD-LDA [15], and discriminative common vectors (DCV) [16], are proposed for increasing the discriminant powers. All these approaches find the optimal subspaces by optimizing a criterion, and a sample vector is projected into the subspaces using the found eigenvectors with nonzero eigenvalues. However, the computational complexity of these statistical approaches strongly depends on the dimensionality of the original data and the number of training samples. herefore, image transformation techniques, such as discrete cosine transform (DC) [6] or discrete wavelet transform (DW) [5], reduce the dimensionality and extract the intrinsic features for face recognition. he novel scheme as shown in Fig. 1 was designed for enhancing the recognition and authentication performances against PIE changes. In the first stage, high dimensional

3 A NOVEL SCHEME FOR FACE RECOGNIION AND AUHENICAION 371 Fig. 1. he proposed recognition/authentication scheme. face images were transformed into small images in a low dimension using DW. According to the consequences in [5], the sub-images of sub-band LL had higher discriminant power. In order to reduce the PIE effects, local features with uniform illumination were extracted. he transformed image was easily partitioned into four half-a-face parts: a left face (L), a right face (R), a top face (), and a bottom face (B). hey represented the local features and formed the multiple modal feature spaces. In addition, the sub-image of sub-band LL was transformed again to obtain a full face (F) in a smaller scale to keep the global features of the whole face. he energy-based features of these five sub-images were extracted to enhance the discriminant powers. In stage 2, discriminative common vectors (DCVs) of each modal feature space were obtained from the training samples. A nearest matching strategy was adopted to output the five modal classification results in stage 3. hree possible combinations, point-to-point, point-to-line, and point-to-space, were designed to increase the performance. Multiple classifiers were thus generated and fused by a weighted summation. Similarly, a new test image was transformed, partitioned, and projected to obtain five scores. hey were fused to determine the final result. 2. HREE-SAGE FACE-BASED SCHEME In this study, three stages, face representation, discriminative feature analysis, and data classification, comprised the identification process. In addition, multiple classification results were fused to obtain the final results. 2.1 Face Representation he performance of face recognition highly depends on PIE variation. Many approaches have been proposed for reducing the effects of the changes, such as the Gabor filter function, the derivatives of distribution, the logarithmic transformation, the W, the discrete cosine transformation on an edge map, and so on. According to the consequencein [5], low frequency components extracted by W are less sensitive to varying image sand containing the higher discriminating power. herefore, the first step in our scheme is

4 372 YING-NONG CHEN, CHIN-CHUAN HAN, CHENG-ZU WANG AND KUO-CHIN FAN (a) (b) (c) (d) (e) (f) Fig. 2. An example of five modular face images; (a) he original image, and (b) to (f) five parts L, R,, B, and F. to extract the low-frequency features of face images via W for reducing PIE changes. Basically, lighting on a local face region is assumed to be in a linear change or uniform. As shown in Fig. 2, our simple approach is to partition the face images into four parts, i.e. parts L, R,, and B. An assumption is made in this study: he light comes from one direction, and the illumination on each half face is uniform or linearly changed. Without losing the global face features, a new small face image F is generated in a coarse scale via the W. he advantage of our face representation is that: Local features are represented by four sub-images for reducing the PIE effects, and the global face features are preserved in a coarse and whole face image. Given a sample x, it is partitioned and represented as five feature vectors f i (x), i = L, R,, B, F. hese form the multiple modular feature spaces Φ i. 2.2 Discriminative Feature Analysis Face images are represented in five sub-images. he most salient and invariant features should be analyzed and extracted for face recognition. In the last two decades, eigenspace-based transformation in a feature space has been the most popular linear projection approach. his popular projecting transformation, called Fisher linear discriminant (FLD), or LDA, satisfies Fisher s criterion as follows: J FLD W S ( ) arg max B W Wopt =. (1) W W S W W Here, symbols S B and S W denote the between-class and the within-class scatter matrices as defined below, respectively. K M k k W = m μk m μk k= 1m= 1 K B = ( k )( k ). k = 1 S ( x )( x ), and (2) S M μ μ μ μ (3) For simplification, the training set is assumed to be composed of K classes and each class contains M samples. he total mean μ and the class mean μ k are the average of all training samples and the samples x m k in the kth class. In Eq. (1), S3 problem is encountered due to scatter S W being singular. Singular value decomposition (SVD) algorithm is performed to find the inverse scatter S W 1 when matrix S W is singular. Many approaches were proposed and surveyed in [16]. Cevikalp et al. generalized the projection into the

5 A NOVEL SCHEME FOR FACE RECOGNIION AND AUHENICAION 373 null space and found the discriminative common vectors (DCVs). Unlike the PCA-based or LDA-based approaches, DCVs were operated in a null space of the original feature space. he DCV-based method tries to find a transformation matrix which is subject to the following equation: FLD opt B W SWW = 0 W SWW = 0 J ( W ) = arg max W S W = arg max W S W, (4) where S = S W + S B is a total scatter matrix. In addition, the common vectors form a subspace in which the scatter between classes equals the total scatter. he scatter of the training samples is only generated from the between-class scatter, and the within-class scatter nearly equals zero. 2.3 Data Classification In the recognition process, MK training samples of K persons were collected. Due to the S3 problem, the DCVs were found, and the transformed prototypes were generated for matching. he most popular classification rule is the nearest neighbor matching of the prototypes using the Euclidean-based distance. he nearest-feature-space (NFS) [5] strategy was adopted in this study. A new testing sample was transformed by the DCVs. hree kinds of distance-based computation in the transformed feature space were designed as follows: (1) the nearest point-to-point (i.e. the nearest neighbor, 1NN), (2) the nearest point-to-line, and (3) the nearest point-to-space. he last two approaches generated the pseudo prototypes of a class when the training samples were few. In this study, the smallest distance of these three kind distances d(x) = d(f i (x), Φ i ) was chosen, and the ID of the testing sample was thus determined. In the authentication process, M training samples of an individual X in the enrollment stage were considered. Similarly, M randomly selected samples from the other persons were collected to be the negative samples. his is considered to be a two-class classification problem. Personal DCV-based feature space Φ(x) for individual X was thus constructed. If a sample x belongs to class X, the distance value should be smaller than a threshold value t X. Otherwise, it is not a member of class X, i.e., a forged sample. In this study, five modular feature spaces Φ i were constructed and five distances were calculated. Given a testing sample x, five distances d L (x), d R (x), d (x), d B (x), and d F (x) were calculated in the multiple modular feature spaces, and then fused by a weighted summation as follows: Vi d( x, X) = wd i ( fi( x), Φi), wi =, and (5) S i i { L, R,, B, F} σ Bi Vi =, and S = V L + V R +V +V B +V F. (6) σw Here, weight w i is a value that is in proportion to the magnitude of V i in each feature space Φ i. he discriminability V i for each feature space is defined as σb i /σw i which is the same in [17]. wo variances σb and σw are the between-class and the within-class variances. hey were all computed from the training samples.

6 374 YING-NONG CHEN, CHIN-CHUAN HAN, CHENG-ZU WANG AND KUO-CHIN FAN 3. EXPERIMENAL RESULS Some experiments were conducted to show the effectiveness of the proposed algorithm. Several benchmarks of face images, including ORL, Yale, CMU, and IIS were collected for evaluating the recognition and authentication performance. hese data sets are briefly described as follows: he CMU face data set comprised of 68 persons with PIE variations. In this study, 170 images per individual were selected for the experiments. Fifteen images were randomly selected for training, and the others were used for testing. he face-only images were cropped from the original ones to eliminate the influence of hair and background. he Yale data set is another popular set for face recognition. A total of 165 images of 15 individuals were grabbed in a format of 256 gray scales. Lighting, expression, and wearing spectacles were considered for the evaluation. Four and 7 images were respectively chosen for the training and testing samples. Next, the ORL data set was consisted of 400 images, 40 individuals and 10 images per person, with a frontal view and a neutral expression. he images in this data set were grabbed under well-controlled conditions. he images of a class (i.e., person) were divided into 2 parts: 5 images for training, and 5 for testing. Finally, the IIS data set was a set with a large number of image samples. 128 persons were selected, 6 images per person for training, and the other 24 images were used for testing. he experimental parameters, e.g., image size, the training and testing samples for each data set were summarized in able 1. Moreover, all of them were aligned with the eyes and mouths. able 1. he rates for various recognition schemes for four benchmarks (%). Rates Data Sets CMU YALE ORL IIS Image size raining samples esting samples R1 Fig R2 Fig. 2 W R3 Fig. 3 (a) R4 Fig. 3 (a) W R5 Fig. 3 (b) R6 Fig. 3 (b) W R7 Fig. 3 (c) R8 Fig. 3 (c) W R9 Fig. 3 (d) R10 Fig. 3 (d) W he experimental results were divided into two groups: one for recognition and the other for authentication. he rates of the proposed scheme are listed in able 1. In addition to the proposed scheme, eight alternative schemes were implemented and compared. In general, images were first reduced by the W for dimension reduction as shown in Fig. 3. On the other hand, the W process was skipped in evaluating the feature preserving power of W. Four possible combinations were compared. First, the DCV-based trans-

7 A NOVEL SCHEME FOR FACE RECOGNIION AND AUHENICAION 375 (a) (b) (c) (d) Fig. 3. Eight alternative recognition/authentication schemes. formation was replaced with the conventional LDA transformation in Fig. 3 (a). Second, the NFS-based rule was replaced by the nearest neighbor (1NN) in each classifier (Fig. 3 (b)). he first two schemes were classified into the multiple-classifier-based approach. hird, images were only transformed by the DCVs and classified by the NFS rule as shown in Fig. 3 (c). hese schemes are similar to the approach of Cevikalp et al. [16]. Lastly, the original LDA-based schemes were implemented as shown in Fig. 3 (d). hese combinations were repeated again by skipping the W process, e.g. row R4 means the scheme in Fig. 3 (a) skips the W process. he recognition rates for the proposed approach and the alternative schemes are shown in able 1. Each recognition rate was obtained by running the process for 10 times. he training samples were randomly selected from the data sets and the other samples were used for testing as listed in able 1. Images were transformed by W for dimension reduction as shown in rows R1, R3, R5, R7 and R9. W preserved the discriminant power and reduced the noises in a high dimension because the recognition performances were almost similar, e.g. the data pairs: (R1, R2), (R3, R4), (R5, R6), (R7, R8), and (R9, R10). he DCV-based scheme had more discriminating power than the LDA approach when the S3 problem occurred as shown in the compared results listed in the row pairs (R1, R3) and (R2, R4) expect for data sets CMU and IIS, (R7, R8) and (R7, R9) for all

8 376 YING-NONG CHEN, CHIN-CHUAN HAN, CHENG-ZU WANG AND KUO-CHIN FAN able 2. he rates of modular-based and vector-based approaches for CMU and Yale benchmarks (%). Data Sets CMU YALE LPPFace [7] MFA [8] ONPDA [9] OLPPFace [18] Fig Fig. 2 W five data sets. Features were projected to and matched on the null spaces. he inverse of within-class scatter matrix was calculated by an SVD-based approach when the S3 problem occurs. However, the recognition performance was almost the same because the S3 problem did not occur, e.g. (R1, R3) and (R2, R4) for CMU and IIS data sets. In order to show the performance of DCVs on data sets CMU and IIS, a new process was run by setting the image size as 64 by 64. he recognition rates on data set CMU for the schemes in rows R2 and R4 are 85.16% and 82.34%, respectively. Similar, the rates on set IIS are 86.30% and 82.23%. More than 3% improvement is achieved by using the DCVs if the S3 problem occurs. From the results of data pairs (R3, R7), (R4, R8), (R3, R9), and (R4, R10), the performance of the multi-classifier fusion was better than that of a single classifier. Since the pseudo prototypes were generated when the training samples were few, the NFS rules enhanced the recognition performance as shown in data pairs (R1, R5), and (R2, R6). he proposed modular-based approach is compared with the vector-based approaches, LPPFace [7], MFA [8], ONPDA [9], and OLPPface [18] on CMU and Yale benchmarks. he recognition rates are listed at able 2. Images of size 32 by 32 have been reduced to the feature vectors of size 256 by PCA process for avoiding the S3 problem. With the similar conditions, the performance of Fig. 2 W is almost the same with MFA and ONPDA on CMU benchmarks. he rate of row Fig. 2 is a bit of less than these two methods, because the dimensional size is smaller than the other ones. From the other recognition results, the modular-based method outperforms the vector-based approaches. Next, the performance of authentication process was analyzed as shown in Fig. 4. In the enrollment stage, M randomly selected samples for a specified individual X and M samples selected from the other persons were collected as listed in able 3. In this case, the S3 problem occurs in each data set. In the testing stage, all the samples, except for the able 3. he parameters for various authentication schemes for four benchmarks (%). Data sets CMU YALE ORL IIS Image size Enrollment for a person PS NS esting for a person PS NS otal testing samples PS NS

9 A NOVEL SCHEME FOR FACE RECOGNIION AND AUHENICAION 377 (a) (b) (c) Fig. 4. he ROC curves of various authentication schemes for four face benchmarks.

10 378 YING-NONG CHEN, CHIN-CHUAN HAN, CHENG-ZU WANG AND KUO-CHIN FAN (d) Fig. 4. (Cont d) he ROC curves of various authentication schemes for four face benchmarks. training ones, were used for evaluation. wo rates, a false acceptance rate (FAR) and a false rejection rate(frr), were calculated by running the process for 10 times. hese two values were contradictory to each other with various threshold values. he ROC curves were obtained for various data sets as shown in Fig. 4. From this analysis, the proposed scheme outperformed the others. 4. CONCLUSION In this study, a novel scheme integrated the W, DCV transform, modular features, NFS rules, and weighted fusions for face identification. he advantages of the proposed scheme are manifold. First, both local and global features are extracted to reduce the PIE variations. Modular feature spaces are constructed for designing the multiple classifiers. he performance of a multiple classifier-based scheme is better than that of a single classifier. For each classifier, the DCVs and the NFS rules are adopted for solving the S3 problem and increasing the matching prototypes. Finally, the weighted summation fuses the multiple classifiers by automatically calculating the weighting values. REFERENCES 1. M. urk and A. Pentland, Eigenfaces for recognition, Journal of Cognitive Neuroscience, Vol. 3, 1991, pp J. Yang, D. Zhang, A. Frangi, and J. Yang, wo-dimensional PCA: A new approach to appearance-based face representation and recognition, IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 26, 2005, pp J. Yang, A. F. Frangi, J. Y. Yang, D. Zhang, and Z. Jin, KPCA plus LDA: A complete kernel Fisher discriminant framework for feature extraction and recognition,

11 A NOVEL SCHEME FOR FACE RECOGNIION AND AUHENICAION 379 IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 27, 2005, pp C. Liu, Gabor-based kernel PCA with fractional power polynomial models for face recognition, IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 26, 2004, pp J.. Chien and C. C. Wu, Discriminant waveletfaces and nearest feature classifiers for face recognition, IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 24, 2002, pp M. J. Er, W. Chen, and S. Wu, High-speed face recognition based on discrete cosine transform and RBF neural networks, IEEE ransactions on Neural Networks, Vol. 16, 2005, pp X. He, S. Yan, Y. Hu, P. Niyogi, and H. J. Zhang, Face recognition using Laplacianfaces, IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 27, 2005, pp S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin, Graph embedding and extensions: A general framework for dimensionality reduction, IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 29, 2007, pp H. Hu, Orthogonal neighborhood preserving discriminant analysis for face recognition, Pattern Recognition, Vol. 41, 2008, pp A. Pentland, B. Moghaddam, and. Starner, View-based and modular eigenspaces for face recognition, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1994, pp Ahonen, A. Hadid, and M. Pietikainen, Face descriptor with local binary patterns: Application to face recognition, IEEE ransactions on Pattern Analysis and Machine Intelligence, 2006, pp B. Heisele, P. Ho, J. Wu, and. Poggio, Face recognition: component-based versus global approaches, Computer Vision and Image Understanding, 2003, pp R. Lotlikar and R. Kothari, Fractional-step dimensionality reduction, IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 22, 2000, pp K. Etemad and R. Chellappa, Discriminant analysis for recognition of human face recognition, Journal of Optical Society America, Vol. 14, 1997, pp J. Fortuna and D. Capson, Improved support vector classification using PCA and ICA feature space modification, Pattern Recognition, Vol. 37, 2004, pp H. Cevikalp, M. Wilkes, and A. Barkana, Discriminative common vectors for face recognition, IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 27, 2005, pp K. Kim, H. Kim, W. Hwang, and J. Kittler, Independent component analysis in a local facial residue space for face recognition, Pattern Recognition, Vol. 37, 2004, pp D. Cai, X. He, J. Han, and H. Zhang, Orthogonal laplacianfaces for face recognition, IEEE ransactions on Pattern Analysis and Machine Intelligence, Vol. 15, 2006, pp Ying-Nong Chen ( ) was born in aipei, aiwan, in He received the B.S. and M.S. degrees in Information Management and Informatics from the Nan Hua

12 380 YING-NONG CHEN, CHIN-CHUAN HAN, CHENG-ZU WANG AND KUO-CHIN FAN University, aiwan and the Fo Guang University, aiwan, in 2000 and 2003, respectively. He is currently pursuing the Ph.D. degree in Computer Science and Information Engineering at the National Central University, aiwan. His research interests include pattern recognition, and computer vision. Chin-Chuan Han ( ) received the B.S. degree in Computer Engineering from National Chiao ung University in 1989, and M.S. and Ph.D. degrees in Computer Science and Electronic Engineering from National Central University in 1991 and 1994, respectively. From 1995 to 1998, he was a Postdoctoral Fellow in the Institute of Information Science, Academia Sinica, aipei, aiwan. From 2000 to 2004, he worked with the Department of Computer Science and Information Engineering, Chunghua University, aiwan. In 2004, he joined the Department of Computer Science and Information Engineering, National United University, aiwan, where he became a Professor in Prof. Han is a member of IEEE, SPIE, and IPPR in aiwan. His research interests are in the areas of face recognition, biometrics authentication, video surveillance, and pattern recognition. Cheng-zu Wang ( ) is currently an Associate Professor in the Department of Computer Science at National aipei University of Education, aiwan. He received his M.S. and Ph.D. degrees in the Center for Advanced Computer Studies from the University of Louisiana in 1991 and 1994, respectively. His current interests include image processing, hybrid soft computing models, and software engineering. Kuo-Chin Fan ( ) was born in Hsinchu, aiwan, R.O.C., on June 21, He received the B.S. degree in Electrical Engineering from the National sing Hua University, Hsinchu, in 1981, and the M.S. and Ph.D. degrees from the University of Florida (UF), Gainesville, in 1985 and 1989, respectively. In 1983, he joined the Electronic Research and Service Organization (ERSO), aiwan, as a Computer Engineer. From 1984 to 1989, he was a Research Assistant with the Center for Information Research, UF. In 1989, he joined the Institute of Computer Science and Information Engineering, National Central University, Chungli, aiwan, where he became a Professor in From 1994 to 1997, he was Chairman of the Department. Currently, he is the Director of the Computer Center. His current research interest includes image analysis, optical character recognition, and document analysis. Prof. Fan is a member of SPIE.