APPEARANCE-BASED methods have been widely used in face

Transcription

1 IEEE RANSACIONS ON PAERN ANALYSIS AND MACHINE INELLIGENCE, VOL. 35, NO. 1, JANUARY Discriminative Multimanifold Analysis for Face Recognition from a Single raining Sample per Person Jiwen Lu, Member, IEEE, Yap-Peng an, Senior Member, IEEE, and Gang Wang, Member, IEEE Abstract Conventional appearance-based face recognition methods usually assume that there are multiple samples per person (MSPP) available for discriminative feature extraction during the training phase. In many practical face recognition applications such as law enhancement, e-passport, and ID card identification, this assumption, however, may not hold as there is only a single sample per person (SSPP) enrolled or recorded in these systems. Many popular face recognition methods fail to work well in this scenario because there are not enough samples for discriminant learning. o address this problem, we propose in this paper a novel discriminative multimanifold analysis (DMMA) method by learning discriminative features from image patches. First, we partition each enrolled face image into several nonoverlapping patches to form an image set for each sample per person. hen, we formulate the SSPP face recognition as a manifold-manifold matching problem and learn multiple DMMA feature spaces to maximize the manifold margins of different persons. Finally, we present a reconstruction-based manifold-manifold distance to identify the unlabeled subjects. Experimental results on three widely used face databases are presented to demonstrate the efficacy of the proposed approach. Index erms Face recognition, manifold learning, subspace learning, single training sample per person Ç 1 INRODUCION APPEARANCE-BASED methods have been widely used in face recognition, and a large number of such algorithms have been proposed in recent years [2], [3], [4], [5], [9], [14], [15], [16], [23], [24], [25], [ 41], [46], [49], [51], [53], [55]. he common goal of these methods is to learn, in a supervised, semisupervised, or unsupervised manner, a compact, lowdimensional feature subspace for face representation, in such a way that the intrinsic characteristics of the original face samples are well preserved. Representative and popular algorithms include principal component analysis (PCA) [41], linear discriminant analysis (LDA) [2], locality preserving projections (LPPs) [15], marginal fisher analysis (MFA) [49], and their weighted, kernelized, and tensorized variants [23], [24], [49], [50], [53]. Despite having different assumptions, these methods can be unified into a general graph embedding (GE) framework [3], [4], [49], [51] with different constraints. he performance of the appearance-based methods in face recognition, however, is heavily influenced by the number of training samples per person [17]. Specifically, if the number of training samples per person is much smaller than the feature dimension of face samples, it is generally inaccurate to estimate the within-class variance of LDA and to exploit. J. Lu is with the Advanced Digital Sciences Center, 1 Fusionopolis Way, #08-10, Connexis North ower, Singapore jiwen.lu@adsc.com.sg.. Y.-P. an and G. Wang are with the School of Electrical and Electronic Engineering, Nanyang echnological University, Block S1, Nanyang Avenue, Singapore {eyptan, wanggang}@ntu.edu.sg. Manuscript received 24 Aug. 2011; revised 20 Dec. 2011; accepted 28 Feb. 2012; published online 13 Mar Recommended for acceptance by C. Bregler. For information on obtaining reprints of this article, please send to: tpami@computer.org, and reference IEEECS Log Number PAMI Digital Object Identifier no /PAMI discriminant and geometrical information of most existing manifold learning algorithms [5], [9], [ 16], [49], [55] for face recognition. In many practical face recognition applications, such as law enhancement, e-passport, and ID card identification, there is usually only a single sample per person (SSPP) recorded in these systems because it is normally difficult to collect additional samples under these scenarios. herefore, many existing appearance-based methods such as LDA and its variants [3], [4], [16], [53] cannot be directly applied for feature extraction due to the lack of samples to compute the within-class scatter. Given a (frontal) face image, there are usually two eyes, one nose, and one mouth in the image. Intuitively speaking, the similarity between two noses of different persons is usually higher than that between the nose and the mouth of the same person. Hence, when a face image is partitioned into several local patches, it is more likely that these patches reside in a manifold and each patch corresponds to a point in the manifold. Moreover, there is high overlapping between these manifolds and the distances between patches at the same location of different images (persons) are usually smaller than those at different locations of the same image (person). Hence, a key challenge to the SSPP face recognition problem is to extract discriminative features such that the points in the same manifold become closer and those in different manifolds are far apart. o achieve this goal, we propose in this paper a novel discriminative multimanifold analysis (DMMA) method to learn the local discriminant features to maximize the manifold margins of different persons so that more discriminant information can be exploited for recognition. In the recognition phase, we propose a reconstruction-based manifold-manifold distance to identify the unlabeled subjects. Experimental results on three widely used face databases are presented to demonstrate the efficacy of the proposed approach /13/$31.00 ß 2013 IEEE Published by the IEEE Computer Society

2 40 IEEE RANSACIONS ON PAERN ANALYSIS AND MACHINE INELLIGENCE, VOL. 35, NO. 1, JANUARY Related Work o address the SSPP problem of face recognition, there have been some attempts in the literature which can be classified into four main categories [39]: local feature representations, generic learning, virtual sample generation, and image partitioning. For the first category, an effective and robust feature descriptor is applied to represent each face image as a feature vector, and different people are expected to be separated as much as possible in the feature space. State-ofthe-art local feature methods include local binary patterns (LBPs) [1], local Gabor binary patterns (LGBPs) [60], histogram of Gabor phase patterns (HGGPs) [57], patterns of oriented edge magnitudes (POEMs) [42], local derivative patterns (LDPs) [56], and Gabor volume-based local binary patterns (GV-LBPs) [22]. For the second category, an additional generic training set with multiple samples per person (MSPP) is applied to extract discriminative features which are subsequently used to identify the people each enrolled with a single sample. For example, Kim and Kittler [20] employed a generic training set to learn discriminative features to address the SSPP problem in pose-invariant face recognition. Wang and ang [47] proposed a generic learning framework and presented several appearance-based discriminant feature extraction methods under this framework. While discriminative information can be exploited, one common assumption among these methods is that the interclass and intraclass variations of the MSPP generic training set and the SSPP gallery set are similar, which may not hold in many real-world applications because it is not easy to collect a generic MMSP training set which represents all the gallery sets well. o address this issue, Su et al. [37] proposed an adaptive generic learning method to infer the discriminative information of the SSPP gallery set by using a prediction model learned from the generic training set. Si et al. [36] proposed a transfer subspace learning approach to transfer the discriminative model learned from the MSPP training set to the SSPP gallery set for face recognition. Even if there is some improvement on addressing the SSPP face recognition, the performance of these methods depends heavily on the generic training set, which is still very difficult to obtain in practical applications. For the third category, some additional training samples for each person are virtually generated such that discriminative subspace learning can be used for feature extraction. For example, Zhang et al. [58] and Gao et al. [10] presented two singular value decomposition (SVD)-based perturbation algorithms to obtain multiple images for each person and then applied the conventional LDA for feature extraction. While these methods can alleviate the SSPP problem to a certain extent, one common shortcoming among these methods is that there is high correlation among the virtually generalized samples as they cannot be considered as independent samples for feature extraction; this may result in much redundancy in the learned discriminative feature subspace, as stated in [28]. For the fourth category, each face image is first partitioned into several local patches and then some discriminant learning techniques can be applied for feature extraction. An early such attempt [29] divided each face image into six elliptical parts and learned a local probabilistic model for Fig. 1. Manifold-manifold matching for SSPP face recognition. Each image is partitioned into many local patches and represented by a manifold, and the SSPP face recognition becomes a manifold-manifold matching problem, where M 1, M 2 ;...; M N are the manifolds of the training images and M t is the manifold of the testing image. recognition. an et al. [38] extended this work by proposing an alternative way of representing each face subspace with self-organizing maps (SOMs). Chen et al. [6] employed LDA and Kanan et al. [19] presented a weighted pseudo-zernike moment to extract discriminative features of each local patch, respectively. However, these methods ignore the geometrical information of the local patches in the feature extraction procedure. Specially, when a face image is partitioned into several local patches, they represent different parts (semantics) of the original face image, such as nose, mouth, and eyes, and may not be modeled accurately by a simple distribution. It is more likely that these patches reside in a manifold and each patch corresponds to a point in the manifold because the underlying structure of these patches could be controlled by a few factors which can be modeled by a low-dimensional manifold [34], [40]. Motivated by this observation, we consider the local patches of each person as a manifold, and the SSPP face recognition can be formulated as a manifoldmanifold matching problem, as illustrated in Fig Contributions of his Work In this paper, we propose a novel DMMA method by learning discriminative features from image patches. First, we partition each enrolled face image into several nonoverlapping patches to form an image set for each sample per person. hen, we formulate the SSPP face recognition as a manifoldmanifold matching problem and learn multiple DMMA feature spaces to maximize the manifold margins of different persons. Finally, we propose a reconstruction-based manifold-manifold distance to identify the unlabeled subjects. Experimental results are presented to demonstrate the efficacy of the proposed approach. his is an extended version of our conference paper [26]. We make the following new contributions to the conference version:. We have compared our DMMA method with the state-of-the-art local feature representation methods

3 LU E AL.: DISCRIMINAIVE MULIMANIFOLD ANALYSIS FOR FACE RECOGNIION FROM A SINGLE RAINING SAMPLE PER PERSON 41 Fig. 2. Visualization of five manifolds corresponding to five people from the FERE database. [1], [35], [42], [56], [57], [60] for face recognition from a single training sample per person.. We have analyzed different parameter settings of our proposed DMMA method, including the image patch size, nearest intermanifold neighbor size, and nearest intramanifold neighbor size.. We have investigated the performance of our proposed DMMA method when the face images are automatically detected by a face detector.. We have included the application scenario when the testing samples are nonfrontal faces. he rest of the paper is organized as follows: Section 2 details the proposed method. Section 3 presents the experimental results. Section 4 discusses some other possible computer vision applications of our proposed method. Section 5 concludes the paper. 2 PROPOSED APPROACH Let X ¼½x 1 ;x 2 ;...;x N Š be the training set, where x j is the training image of the jth person with a size of m n, 1 j N, N is the number of persons in the training set. We first divide each image x j into t nonoverlapping local patches with an equal size of a b, where t ¼ mn ab. Let M i ¼ ½x i1 ;x i2 ;...;x it Š be the image patch set of the ith person, represented by a manifold M i. 2.1 Motivation o explore the geometrical information of the local patches, we randomly selected five subjects from the FERE database [32] to visualize the manifold structure of these local patches. Fig. 2a shows five well-aligned face images; each is partitioned into 3, nonoverlapping patches. he distribution of all the image patches of the five subjects is shown in Fig. 2b in a 3D space for ease of presentation. We can see that there is high overlapping among the manifolds corresponding to different people. We can also see that there is a large variation on the appearance of intrasubject patches. As we mentioned above, the similarity between the same semantic patches of different subjects is usually higher than Fig. 3. Illustration of the proposed DMMA method. (a) hree pairs of patches of two subjects in the original high-dimensional space. (b) Desired distribution in the low-dimensional feature space. that of different semantic patches of the same subject. his is illustrated in Fig. 3a, where the similarity of two eyes from two different subjects is usually higher than that of an eye and a cheek from the same person. Hence, in the original image patch space, different semantic patches of the same subject are separated, while the same semantic patches of different subjects are clustered, which is the main challenge in our task. We aim to learn some mappings to project the patches in the same manifold to be close and those in different manifolds to be far apart, respectively. Fig. 3b shows the desired result of the proposed method, where local patches of different subjects are well separated while those of the same subject become closer after the mappings. As a result, the manifold margin in the low-dimensional feature space is much larger and more discriminative information can be exploited for recognition. Most existing discriminative manifold learning algorithms assume that the samples from different classes define a single manifold in the feature space and seek a common feature space for all samples so that the ratio of between-class locality variance to within-class locality variance is maximized [5], [9], [16], [49], [55]. However, such an assumption is arguably the most suitable because it is still unknown that whether a single manifold could model the data well and guarantee the best recognition accuracy. Differently from these methods, we model the local patches of each subject as a manifold so that they can be better separated when the feature dimensions are selected to be different in the lowdimensional feature spaces. here are two main reasons for us to model each person as a manifold:. Person-specific feature representation and extraction methods usually gain better recognition accuracy than generic face recognition methods, as shown in [13], [27], and [54]. hat is because person-specific methods utilize the information which is especially effective for a given person. Hence, the geometrical information of these patches is person-specific, and we model the local patches from each person as a single manifold to extract person-specific features for recognition by using multimanifold analysis.. Existing manifold learning-based face recognition methods model all face samples as a single manifold,

4 42 IEEE RANSACIONS ON PAERN ANALYSIS AND MACHINE INELLIGENCE, VOL. 35, NO. 1, JANUARY 2013 Fig. 4. (a) here are three intramanifold neighbors (x ir1, x ir2, and x ir3 ) and six intermanifold neighbors (x 0 ir1, x0 ir2, x0 ir3, x0 ir4, x0 ir5, and x0 ir6 ) for a local patch x ir. he points with the same color and shape denote local patches from the same subject and those with different colors and shapes denote local patches from different subjects, respectively. (b) he intramanifold graph connects intramanifold neighbors. (c) he intermanifold graph connects intermanifold neighbors. (d) After DMMA, the manifold margin between the ith person and others subjects is maximized. and the feature dimension of all face samples is chosen the same for recognition. he optimal feature dimension for each specific person may be different due to the intrinsic differences of different persons. Hence, we model each person as a manifold such that we can optimize the feature dimension for each specific person. 2.2 DMMA Let M¼½M 1 ; M 2 ;...; M N Š be the training set of N people and M i ¼½x i1 ;x i2 ;...;x it Š be the manifold of the ith person, x ir 2 R d, 1 i N, 1 r t. he aim of DMMA is to seek N feature matrices W 1 ;W 2 ;...;W N, W i 2 R dd i, i ¼ 1; 2;...; N, to project the training set into N lowdimensional feature spaces such that the manifold margins are maximized, where d and d i denote the feature dimension of each original image patch and the lowdimensional feature learned by W i, respectively. o the best of our knowledge, a formal definition of manifold margin has not been defined. In the literature, there have been a few attempts to compute the distance between two manifolds. For example, Wang et al. [45] proposed a maximal linear patch (MLP) clustering method to partition each manifold into several clusters, modeled each cluster with a subspace, and then computed the nearest subspace distance as the manifold distance. o address the unbalanced cluster issue in the MLP method, Wang and Chen [44] further proposed a hierarchical divisive clustering method to better approximate the manifold by several subspaces, and then learned a discriminant function for feature extraction. While these methods can calculate the manifold distance approximately, they assumed that all data are sampled from one single manifold and only one projection is derived for feature extraction. In our proposed approach, we aim to seek multiple projection matrices to uncover the geometrical information of these manifolds and better characterize the manifold margin in the low-dimensional feature space. Given a sample x ir, which is the rth patch of the ith manifold, there are usually two kinds of neighbor in these manifolds: intramanifold neighbors N intra and intermanifold neighbors N inter. From the viewpoint of classification, we aim to minimize the intramanifold variance and maximize the intermanifold separability in the low-dimensional feature spaces simultaneously so that the manifold margin can be maximized for feature extraction. o achieve this goal, we formulate the proposed DMMA as the following optimization problem: max JðW 1 ;W 2 ;...;W N Þ W 1 ;W 2 ;;W N ¼ J 1 ðw 1 ;W 2 ;...;W N Þ J 2 ðw 1 ;W 2 ;...;W N Þ! X t X k1 W i x ir Wi x0 irp k2 A irp ¼ XN i¼1 XN i¼1 X t X k2 r¼1 q¼1 kw i x ir W i x irqk 2 B irq!; where x 0 irp represents the pth k 1 -nearest intermanifold neighbors and x irq denotes the qth k 2 -nearest intramanifold neighbors of x ir, A irp and B irq are two affinity matrices to characterize the similarity between x ir and x 0 irp as well as that between x ir and x irq, respectively. While many graph construction methods have been proposed in the machine learning and computer vision community recently [18], we apply the conventional k-nearest-neighbor method for its high effectiveness and efficiency to calculate the affinity matrices A and B as follows: A irp ¼ exp ð kx ir x 0 irp k2 = 2 ; if x 0 irp 2 Nk 1 inter ðx irþ ð2þ 0; otherwise; B irq ¼ exp ð kx ir x irq k 2 = 2 Þ; if x irq 2 N k 2 intra ðx irþ ð3þ 0; otherwise; where N k 1 inter ðx irþ and N k 2 intra ðx irþ denote the k 1 -intermanifold neighbors and k 2 -intramanifold neighbors of x ir, respectively; k 1, k 2, and are three empirically set parameters. he objective function of J 1 in (1) is to ensure that if x ir and x 0 irp are close and from different subjects, then their lowdimensional representations are separated as far as possible. On the other hand, J 2 in (1) ensures that if x ir and x irq are close and from the same subject, then their low-dimensional representations are close as well. Fig. 4 presents an example to show how these intramanifold and intermanifold neighbors are used to maximize the manifold margins and separate different manifolds, where k 1 and k 2 were set to 6 and 3, respectively. o our knowledge, there is no closed-form solution for the optimization problem defined in (1) as there are N projection matrices to be obtained simultaneously. We solve this problem in an iterative manner as inspired by the recent advances in high order tensor decomposition [23], [49]. he basic idea is to first initialize W 1 ;W 2 ;...;W i 1 ;W iþ1 ;...;W N with a valid initial solution, and then solve W i sequentially. ð1þ

5 LU E AL.: DISCRIMINAIVE MULIMANIFOLD ANALYSIS FOR FACE RECOGNIION FROM A SINGLE RAINING SAMPLE PER PERSON 43 Fig. 5. he proposed DMMA algorithm. Given W 1 ;W 2 ;...;W i 1 ;W iþ1 ;...;W N, (1) can be rewritten as where max JðW i Þ W i ¼ðJ 1 ðw i ÞþF 1 Þ ðj 2 ðw i ÞþF 2 Þ! ¼ Xt X k1 W i x ir Wi x0 2 irp A irp þ F 1 F 1 ¼ F 2 ¼ XN j¼1;j6¼i XN j¼1;j6¼i Xt X k2 r¼1 q¼1 X t X k1 X t X k 2 r¼1 q¼1 W i x ir Wi x 2 irq B irq þ F 2!;! W j x jr Wj x0 2 jrp A jrp and! W j x jr Wj x 2 jrq B jrq are two constant matrices which can be ignored as they do not affect the optimization of W i. J 1 ðw i Þ can be written in the following form: J 1 ðw i Þ ¼ Xt ¼ Xt ¼ Xt ¼ tr X k1 X k 1 X k1 W i kw i x ir W i x0 irp k2 A irp tr Wi x ir Wi irp x0 W i x ir Wi x0 irp Airp tr W i ¼ tr W i H 1W i ; ðxir x 0 irp xir xirp 0 Airp Wi "! X t X k1 xir x 0 irp xir x 0 irp Airp #W i ð4þ ð5þ where H 1 ¼ 4 Xt X k1 ðx ir x 0 irp Þðx ir x 0 irp Þ A irp : Similarly, we can simplify J 2 ðw i Þ as follows: J 2 ðw i Þ where ¼ Xt ¼ tr X k2 r¼1 q¼1 W i W i x ir Wi x 2 irq B irq " #W i! X t X k2 r¼1 q¼1 ¼ tr Wi H 2W i ; H 2 ¼ 4 Xt X k2 r¼1 q¼1 ðx ir x irq Þðx ir x irq Þ B irq ðx ir x irq Þðx ir x irq Þ B irq : Having derived H 1 and H 2, we can obtain the bases of W i by solving the following eigenvalue equation: ðh 1 H 2 Þw ¼ w: Let fw 1 ;w 2 ;...;w di g be the eigenvectors corresponding to the d i largest eigenvalues f j jj ¼ 1; 2;...;d i g ordered in such a way that 1 2 di. hen, W ¼½w 1 ;w 2 ;...; w di Š is the projection matrix of W i. We can iteratively and sequentially solve (4) to determine the N projection matrices W 1, W 2 ;...;W N. he proposed DMMA algorithm is summarized in Fig. 5. Now, we discuss how to determine the feature dimension d i for the ith projection matrix W i. Most existing manifold learning algorithms [3], [4], [5], [9], [15], [16], [49], [55] empirically select the optimal feature dimension because there is only one projection matrix to be sought for feature extraction. In our case, there are N feature projection matrices corresponding to N different manifolds. ð6þ ð7þ ð8þ ð9þ

6 44 IEEE RANSACIONS ON PAERN ANALYSIS AND MACHINE INELLIGENCE, VOL. 35, NO. 1, JANUARY 2013 Fig. 6. Distribution of low-dimensional manifolds learned by our DMMA method. Hence, it is nontrivial to determine the optimal feature dimensions because there are Q N i¼1 d i candidates to be searched, for which it is time-communing, if not impossible, to find the best one. We propose a new feature dimension determination method by analyzing the eigenvalues of ðh 1 H 2 Þ. Since ðh 1 H 2 Þ is not positive semidefinite, the eigenvalues of ðh 1 H 2 Þ may be positive, zero, and negative. Let 1 2 di > 0 diþ1 d and W i be ordered according to their corresponding eigenvalues. We can select the first d i eigenvectors to maximize (4). his is because when the samples are projected to one specific eigenvector w j corresponding to an eigenvalue j, (4) can be written as Jðw j Þ¼J 1 ðw j Þ J 2 ðw j Þ ¼ w j ðh 1 H 2 Þw j ¼ w j jw j ¼ j : ð10þ If j > 0, this means that the intermanifold distance is larger than the intramanifold distance along the direction of w j and the samples can likely be correctly classified. According to this criterion, we can automatically determine the feature dimension d i of each projection matrix W i. Having obtained W i, we can use it to represent the local patches of the ith person. Fig. 6 illustrates the lowdimensional manifolds of the five face subjects in Fig. 2a learned by our DMMA method. he values of the parameters k 1, k 2, and were set to 15, 5, and 100, respectively. We can see that there is a clear separation for the manifolds corresponding to different persons. Hence, more discriminative information can be exploited for recognition. 2.3 Recognition Given a test face sample, we first partition it into t nonoverlapping local patches and model it as a manifold M ¼½x 1 ;x 2 ;...;x t Š. We then assign a label c to M as follows: c ¼ arg min dðm ; M i Þ; i ¼ 1; 2;...;N: ð11þ i where dðm ; M i Þ is the manifold distance between M and M i. Fig. 7. Illustration of our manifold-manifold distance method. For the jth point y j in the testing manifold M, we first find its k-nearest neighbors G k ðy j Þ in Y i, where k is set to 3 in this example. hen, we use the points in G k ðy j Þ to reconstruct y j, the reconstruction error is adopted as the distance between the point y j to the manifold Y i. Finally, the average point-manifold distance is adopted as the manifold-manifold distance. Differently from existing manifold-manifold distance methods [44], [45], which calculate the nearest subspacesubspace distance to approximate the manifold-manifold distance, we propose here a novel reconstruction-based method to compute the manifold distance by using all, rather than selected, samples in the two manifolds. he main reason is that there is a high correlation among the samples used in existing manifold-manifold distance methods, and a subspace can approximate their variations in each manifold well. However, the samples used in our case are local patches of an image in the same manifold and we will lose some patch information if not all the patches are used to calculate the subspace distance. Let Y i ¼ Wi M i ¼½y i1 ;y i2 ;...;y it Š and Y ¼ Wi M ¼ ½y 1 ;y 2 ;...;y t Š be the low-dimensional representations of manifold M i and M, mapped by the projection matrix W i. he manifold distance between them is defined as follows: dðm ; M i Þ¼ 1 t X t j¼1 dy j ;G k ðy j Þ ; ð12þ where G k ðy j Þ denotes the k-nearest neighbors of y j in Y i, and dðy j ;G k ðy j ÞÞ can be easily obtained by solving the following constrained optimization problem, similarly to the locally linear embedding method discussed in [34], and described below as dy j ;G k ðy j Þ ¼ P min k s¼1 c s¼1 y j Xk s¼1 c s G s k ðy 2 jþ ; ð13þ where c s is the reconstruction coefficient of the neighbor G s k ðy jþ to y j. Fig. 7 illustrates the basic idea of computing the distance of two manifolds.

7 LU E AL.: DISCRIMINAIVE MULIMANIFOLD ANALYSIS FOR FACE RECOGNIION FROM A SINGLE RAINING SAMPLE PER PERSON 45 ABLE 1 Detailed Information of All Eight Subsets of the AR Database Used in Our Experiments 3 EXPERIMENS We have evaluated the proposed DMMA method by conducting a number of SSPP face recognition experiments on three widely used face databases. he following describes the details of the experiments and results. 3.1 Data Sets hree publicly available face databases, namely, AR [30], FERE [31], [32], and FG-NE [21], are used for experimental evaluation. he AR face database contains over 4,000 color face images of 126 people (70 men and 56 women), including frontal views of faces with different facial expressions, lighting conditions, and occlusions (sunglasses and scarves). here are 26 different images per person, taken in two sessions (separated by two weeks), and each session contains color images. In our experiments, eight subsets of 800 images (100 images per subject) from 100 different subjects (50 men and 50 women) were used, which were taken from two different sessions and with different expressions. able 1 provides the detailed information of each subset. Fig. 8a shows eight sample images of one subject from the Subset A to H of the AR database. In our experiments, we used Subset A for training and the remaining seven subsets for testing. he FERE database consists of 13,539 facial images corresponding to 1,565 subjects, who are diverse across ethnicity, gender, and age. In our experiments, we used two subsets (FERE-1 and FERE-2) from the FERE database, and compared our method with the state-of-the-art methods which can also address the SSPP face recognition problem. Similar to [7], [38], [48], [58], FERE-1 is a subset of the FERE database including 400 gray-level frontal face images comprising 200 different people, with the size of here are 71 females and 129 males, each of whom has two images (Fa and Fb), with different races, genders, ages, expressions, illuminations, and scales, etc. For this database, we used Fa images for training and Fb images for testing. FERE-2 follows the standard FERE evaluation protocol, and there are five sets (Fa, Fb, Fc, Dup I, and Dup II) in FERE-2. Fa, containing 1,196 frontal images of 1,196 subjects, is used as Gallery, while Fb (1,195 images of expression variations), Fc (194 images taking under different illumination conditions), Dup I (722 images taken later in time), and Dup II (234 images, a subset of Dup I) are the Probe sets. Dup II consists of images that were taken at least one year after the corresponding Gallery images. Fig. 8b shows some sample images of one subject from the FERE database, where the first row are Fa images and the second row are Fb images. Fig. 8. Sample face images from the (a) AR, (b) FERE, and (c) FG- NE databases. he FG-NE database consists of 1,002 face images with large variations in lighting, pose, and expression. here are 82 subjects (around 12 images/subjects) in total, with ages ranging from 0 to 69 years old. In our experiments, we constructed four subsets (Subsets A-D) from the FG-NE database. Each subset contains 82 subjects and there are two images (S 1 and S 2 ) for each subject. he age gaps between S 1 and S 2 for Subsets A to D are ð0; 3Š, ð3; 6Š, ð6; 9Š, and ð9; 12Š, respectively. Fig. 8c shows some sample images from the FG-NE database extracted by using the active appearance model features, where the first row are S 1 images and the second row are S 2 images. In our experiments, we used S 1 for training and S 2 for testing. For all images in the above three datasets, the facial part of each image was manually cropped, aligned, and resized into pixels according to eyes positions. 3.2 Results and Analysis Comparisons with State-of-the-Art Learning-Based Algorithms We have compared our method with 11 learning-based algorithms which can be used to address the SSPP face recognition problem, including PCA [41], ðpcþ 2 A (projectedcombined principal component analysis) [48], EðPCÞ 2 A (enhanced projected-combined principal component analysis) [7], 2DPCA (2D PCA) [52], (2D) 2 PCA (two-directional 2D PCA) [59], SOM [38], LPP [15], SVD-LDA (singular value decomposition-based linear discriminant analysis) [58], Block PCA [12], Block LDA [6], and uniform pursuit (UP) [8]. he nearest neighbor classifier with the euclidean distance was adopted for classification. We implemented these methods ourselves and tuned the parameters for each method for a fair comparison. For ðpcþ 2 A, the weighting parameter was set as For EðPCÞ 2 A, two parameters and were used to weight the projected-combined principal component and they were empirically set to be 0.25 and 0.5, respectively. For SVD-LDA, the first three singular values and the corresponding singular vectors were applied to construct a virtual sample to calculate the within-class scatter. For the LPP and UP methods, the number of nearest neighbors was selected as 4 and t was set to be 100 to calculate the similarity matrix. For all block-based methods, such as Block PCA, Block LDA, and our proposed

8 46 IEEE RANSACIONS ON PAERN ANALYSIS AND MACHINE INELLIGENCE, VOL. 35, NO. 1, JANUARY 2013 ABLE 2 Rank-1 Recognition Accuracy (Percent) of Different Methods on Different Subsets of the AR and FERE-1 Databases ABLE 4 Rank-1 Recognition Accuracy (Percent) of Different Methods on Different Subsets of the FERE-2 Database According to the Standard FERE Evaluation Protocol DMMA method, the size of each block was empirically set as For our DMMA method, the values of the parameters k 1, k 2, k, and were empirically tuned to be 15, 5, 4, and 100, respectively, and the feature dimensions of these manifolds varied from 15 to 25. able 2 tabulates the rank-1 recognition rate of different methods on the AR and FERE-1 databases. he best recognition accuracy of each method was recorded for a fair comparison. As shown in this table, the proposed DMMA consistently outperforms the 11 compared methods on both the AR and FERE-1 databases. It achieved a gain in accuracy of 1, 4, 5, 2, 8, 11, and 3 percent on Subsets B-H of the AR database, and 2 percent on the FERE-1 database compared with the other best compared method, respectively. We have made three observations from the results listed in able 2.. PCA and Block PCA obtain similar performance on both the AR and FERE-1 databases, which indicates that there is no significant difference on the performance of either holistic-based or patch-based feature representation for unsupervised learning on the SSPP face recognition.. SVD-LDA obtains the worst performance on the AR database even though it is a supervised method. he reason is that the virtually generated samples by SVD- LDA are highly related to the original sample as the new sample is just an approximation by discarding ABLE 3 Rank-1 and Rank-10 Recognition Accuracies (Percent) of Different Methods on Different Subsets of the FG-NE Database some smaller singular values of the original image. Hence, the within-class variance cannot be accurately estimated in this scenario. Another reason is that LDA usually obtains better recognition when the number of samples per class is large. However, when the number of sample per class is small, the recognition performance of LDA is poor and even worse than PCA, as also shown in [29].. DMMA obtains the best recognition performance on all the experiments, which implies that both discriminant and geometrical information of local patches are important for recognition. able 3 shows the rank-1 and rank-10 recognition accuracy obtained by different methods on the four subsets of the FG-NE database. We can observe from this table that the recognition performance of all the methods is generally lower than those obtained in the AR and FERE-1 databases, which implies that aging could affect the performance of the SSPP face recognition more than other factors such as lighting and expression variations. However, we can still see that our proposed DMMA consistently outperforms other methods in terms of both rank-1 and rank-10 recognition accuracies. his is because, compared with holistic-based methods such as PCA, 2DPCA, LPP, and UP, the proposed DMMA extracts features from local patches and the appearance differences caused by aging are mitigated in the local patches and better age-invariant feature can be preserved. Compared to other block-based methods such as Block PCA and Block LDA, DMMA can exploit both discriminant and locality information to improve the recognition performance Comparisons with State-of-the-Art Local Feature Methods o further validate the effectiveness of DMMA, we compare it with the state-of-the-art local feature methods which can also address the SSSP face recognition problem. We follow the same gallery and probe sets as the standard FERE evaluation protocol, where Fa is used as the gallery set and the other four as the probe sets. en state-of-the-art local feature methods were compared, including LBP [1], LGBP [60], HGGP [57], POEM [42], LDP [56], G-LDP (Gabor-based LDP) [56], RSA (robust and scale approach) [35], GV-LBP [35], and the methods proposed in [61] and [43], respectively. able 4 shows the rank-1 recognition accuracy obtained by different methods on the FERE-2 dataset.

9 LU E AL.: DISCRIMINAIVE MULIMANIFOLD ANALYSIS FOR FACE RECOGNIION FROM A SINGLE RAINING SAMPLE PER PERSON 47 ABLE 5 Rank-1 Recognition Accuracy (Percent) of Different Methods on Different Subsets of the FERE-2 Database As shown in able 4, our DMMA method can achieve comparable results on the FERE evaluation protocol with the state-of-the-art local feature face recognition methods. Specifically, DMMA outperforms most of the compared local feature methods across aging effect (Dup I and Dup II), and achieves comparable results across expression (Fb) and lighting (Fc) variations. here are two reasons to explain why our method is comparable to these state-of-the-art methods even if only raw data of local patches is used in our method: 1) Our method is explicitly discriminative while most state-of-the-art local methods are intrinsically generative even they can extract some discriminative information for feature representation; 2) our method extracts features in a person-specific manner while others extract features in a generic way Comparisons with Existing Single-Manifold Learning Methods We also compare our DMMA method with three existing supervised dimensionality reduction methods, where a single manifold is modeled for the local patches of all training samples. hree supervised manifold learning algorithms, including supervised locally linear embedding (SLLE) [33], supervised Isomap (S-Isomap) [11], and MFA [51], were selected for comparison. We evaluate these methods on the FERE-2 database, with the same experimental settings in Section For SLLE, the parameter was set to 1. For S-Isomap, two parameters and were used to compute the pairwise distance of two samples. In our implementation, was empirically set to 0.5 and was set as the average euclidean distance between all pairs of data points. For MFA, the numbers of intraclass and interclass neighbors were set to 5 and 15, respectively. able 5 tabulates the experimental results on the FERE-2 database. We can observe from this table that our DMMA consistently outperforms the other compared supervised dimensionality reduction methods for SSSP face recognition, which further indicates that modeling each person as a manifold is better because it is more personspecific and classification-oriented. o better compare our proposed method with existing manifold learning algorithms, we conduct face recognition experiments on the AR database with different numbers of enrolled face images for manifold learning. Specifically, we use images from the Subset A-D for training and the Subset E for testing. We randomly select k images per subject from the Subset A-D to construct the training set and calculated the average recognition rate (ARA) as the recognition accuracy of these 20 runs as the final recognition accuracy. Fig. 9 shows the ARA of different manifold learning algorithms versus different number of training images per person. We can observe from this figure that our proposed method consistently outperforms the other compared supervised manifold Fig. 9. ARA of different manifold learning methods versus different number of training samples per person on the AR database. learning methods in all experiments. Moreover, the gains of our method over other manifold learning algorithms decrease as the number of training samples per person increases. hat is because, with the increase in the number of training samples per person, the total number of local patches in the training set increases and more samples can be obtained to model the manifold. Hence, it is more reasonable to model them in a single manifold and the advantage of our proposed method over single-manifold-based methods decreases Computational Complexity We now briefly analyze the computational complexity of the DMMA method, which involves iterations, and in each iteration, we solve a generalized eigenvalue equation. Let N be the number of face samples in the training set, each image patch is d 2 ¼ a b, and t is the number of local patches of each image. Assume k ¼ minfn t; d 2 g, the computational complexity of our proposed the DMMA method is OðNk 3 Þ. We also list the computational time of the proposed DMMA method and compare it with five other subspace learning methods, including PCA, LPP, UP, Block PCA, and Block LDA. Our hardware configuration comprises a 2.4-GHz CPU and a 6 GB RAM. able 6 shows the time spent on the training and the testing (recognition) phases by these subspace methods, where the Matlab software, AR database, and the nearest neighbor classifier were used. From able 6, we can see that the computational complexity of the proposed DMMA method for training is generally larger than other subspace learning methods. In practical applications, however, training is usually an offline process ABLE 6 CPU imes (in Second) Used by Different Methods on the AR Database

10 48 IEEE RANSACIONS ON PAERN ANALYSIS AND MACHINE INELLIGENCE, VOL. 35, NO. 1, JANUARY 2013 Fig. 10. Parameter analysis of DMMA. Rank-1 recognition accuracy versus (a) varying image patch sizes a b, (b) varying nearest intermanifold neighbor size k 1, and (c) varying nearest intramanifold neighbor size k 2 on the FERE-1 database. Fig. 11. (a) Values of the objective function in (1). (b) Recognition performance of DMMA versus different number of iterations. and only recognition needs to be performed in real time. hus, the recognition time is usually more of our concern than the training time. As shown in able 6, the recognition time of the proposed algorithm is less than 0.5 seconds even though it is more time-consuming than other methods. Hence, the computational complexity will not limit the real world applications of our proposed method Parameter Analysis We investigate the effect of the parameters of our proposed DMMA method: image patch size (a b), nearest intermanifold neighbor size k 1, and nearest intramanifold neighbor size k 2. Since each parameter could affect the recognition accuracy, we fix the other two parameters as those used in the previous experiments, and test the remaining one. Fig. 10 shows the influence of these parameters on our method, where Figs. 10a, 10b, and 10c plot the rank-1 recognition accuracy on the FERE-1 database versus different values of the image size, nearest intramanifold neighbor size, and nearest intermanifold neighbor size, respectively. We can observe that DMMA demonstrates a stable recognition performance versus varying patch sizes. Hence, it is easy to select appropriate image patch size and nearest intramanifold/intermanifold neighbor sizes for DMMA to obtain good performance Convergence Analysis Since DMMA is an iterative algorithm, we evaluate its performance with different number of iterations. Figs. 11a and 11b show the objective function of (1) and the rank-1 recognition accuracy of DMMA versus different numbers of iterations on the FERE database, respectively. We can see from this figure that our proposed DMMA can converge to a local optimal peak in several iterations Fully Automatic Face Recognition In the above experiments, the facial part of each image was manually cropped, aligned, and resized into pixels according to the eyes positions. In many practical applications, it is not realistic to manually localize the eye positions in each face image. Hence, we examine the performance of our proposed method when the face images are automatically detected by a face detector. We manually aligned and cropped each face image in the training set and automatically detected, cropped, and resized each image in the testing set into by using the well-known Viola-Jones face detector. We performed such fully automatic face recognition on the FERE database. Fig. 12 shows the recognition Fig. 12. Rank-1 recognition accuracy comparisons when manual and automatic face detection and alignment were applied, respectively. Methods 1-12 correspond to the PCA, (PC) 2 A, E(PC) 2 A, 2DPCA, (2D) 2 PCA, SOM, LPP, SVD-LDA, Block PCA, Block LDA, UP, and DMMA, respectively.

11 LU E AL.: DISCRIMINAIVE MULIMANIFOLD ANALYSIS FOR FACE RECOGNIION FROM A SINGLE RAINING SAMPLE PER PERSON 49 ABLE 7 Rank-1 Recognition Accuracies (Percent) of Different Methods on the Multiview Subset of the FERE Database percent on the bb-bi subsets, respectively. Moreover, patch-based methods such as Block-PCA, Block-LDA, and the proposed DMMA perform better than the other holistic appearance-based subspace methods, and the larger the pose difference between the training and testing images, the bigger the gain that can be obtained. he reason is patch-based methods can better alleviate the image appearance difference caused by pose such that better performance can be attained. performance of our proposed DMMA and the other 11 compared methods. We make the following two observations from the results shown in Fig. 12:. Automatically detecting face regions affects the recognition accuracy of all the 12 methods. hat is because there are usually some spatial misalignments in the automatically detected face images, which leads to a drop on the recognition performance.. Patch-based recognition methods such as Block- PCA, Block-LDA, and our proposed DMMA are more robust than other appearance-based methods such as PCA, LPP, and UP. he reason is patchbased methods can alleviate the misalignment effects of the automatically detected face images when partitioning each face into smaller patches. Moreover, our proposed DMMA demonstrates the smallest drop on the recognition accuracy. hat is possibly because we modeled all the patches of each image as a manifold and such manifold can better discover the nonlinear relationship of the image patches than Block-PCA and Block-LDA Face Recognition across Pose Differences All the above experiments assume the face images in both the training and testing phases are frontal. In many real applications, we can easily control the training samples to be frontal because they are usually collected offline. However, in the testing phase, the testing samples are usually collected online and could be nonfrontal. o investigate the performance of the proposed DMMA method in such scenarios, we applied another subset of the FERE database [31], [32] which consists of 1,800 images from 200 subjects (each with nine images labeled as ba, bb, bc, bd, be, bf, bg, bh, and bi) to perform SSPP face recognition across pose differences. In our experiments, we used each ba image for training and the remaining eight nonfrontal images for testing. Similarly, the facial part of each image was also manually cropped, aligned, and resized into pixels according to the eyes positions, and the nearest neighbor classifier with euclidean distance was applied for recognition. able 7 tabulates the rank-1 recognition rate of different methods on the multiview FERE subset. As shown in this table, the proposed DMMA consistently outperforms the 11 compared methods with the lowest gains in accuracy of 6.5, 5.0, 4.0, 3.0, 2.0, 3.5, 5.0, and DISCUSSIONS While our proposed DMMA method is specifically applied to the SSSP face recognition in this paper, it can also be applied to other computer vision applications. For example, when we perform object recognition from image sets where the images in the gallery and probe sets are collected from consecutive video sequences or unordered photo albums, such recognition tasks can also be formulated as a manifoldmanifold matching problem. Each image set can be described as a nonlinear manifold and each sample in the set can be considered as a point in the manifold. Since object images could be captured from different viewpoints, lighting conditions, and have nonrigid deformations, there may be some overlapping of these manifolds. Hence, we can consider adopting our DMMA method to learn multiple discriminative projection matrices to map object images into low-dimensional feature spaces such that the manifold margins are maximized and more discriminative information can be exploited for recognition. Another example includes video-based face recognition under uncontrolled conditions, where human faces are captured from different poses, illumination, expressions, and backgrounds in the videos. We can also model each face video as a manifold and each frame of the video can be seen as a point in the manifold. hen, video-based face recognition can also be formulated a manifold-manifold matching problem. he last example is video search. Given a query or example video, we can first extract the feature for each frame of the video and represent the video as a manifold in the high-dimensional feature space. hen, video search can be cast as measuring the similarity (distance) of two manifolds represented by a query video and an example video. o exploit more discriminative information, we can use our DMMA method to learn multiple projection matrices to map these manifolds into low-dimensional feature spaces such that the manifold margin of interclass videos are maximized, which is more useful for the search task. 5 CONCLUSION AND FUURE WORK We have proposed in this paper a novel discriminative multimanifold analysis method to address the SSPP problem in face recognition. We partition each enrolled image into several nonoverlapping patches, and construct an image set for each sample per person, and then learn multiple feature spaces to maximize the manifold margins of different persons. Experimental results on three widely used face databases are presented to demonstrate the efficacy of the proposed approach.

12 50 IEEE RANSACIONS ON PAERN ANALYSIS AND MACHINE INELLIGENCE, VOL. 35, NO. 1, JANUARY 2013 For future work, we are interested in designing more effective manifold-manifold distance computing methods to improve the recognition performance. Moreover, how to combine local patch information and global holistic information of face images to further improve the SSSP face recognition accuracy appears to be another interesting direction of future work. REFERENCES [1]. Ahonen, A. Hadid, and M. Pietikäinen, Face Recognition with Local Binary Patterns, Proc. European Conf. Computer Vision, pp , [2] P.N. Belhumeur, J. Hespanha, and D.J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp , July [3] D. Cai, X. He, and J. Han, Spectral Regression for Efficient Regularized Subspace Learning, Proc. IEEE 11th Int l Conf. Computer Vision, pp. 1-8, [4] D. Cai, X. He, Y. Hu, J. Han, and. Huang, Learning a Spatially Smooth Subspace for Face Recognition, Proc. IEEE Int l Conf. Computer Vision and Pattern Recognition, pp. 1-7, [5] H. Chen, H. Chang, and. Liu, Local Discriminant Embedding and Its Variants, Proc. IEEE Int l Conf. Computer Vision and Pattern Recognition, pp , [6] S. Chen, J. Liu, and Z. Zhou, Making FLDA Applicable to Face Recognition with One Sample per Person, Pattern Recognition, vol. 37, no. 7, pp , [7] S. Chen, D. Zhang, and Z. Zhou, Enhanced ðpcþ 2 A for Face Recognition with One raining Image per Person, Pattern Recognition Letters, vol. 25, no. 10, pp , [8] W. Deng, J. Hu, J. Guo, W. Cai, and D. Feng, Robust, Accurate and Efficient Face Recognition from a Single raining Image: A Uniform Pursuit Approach, Pattern Recognition, vol. 43, no. 5, pp , [9] Y. Fu, S. Yan, and.s. Huang, Classification and Feature Extraction by Simplexization, IEEE rans. Information Forensics and Security, vol. 3, no. 1, pp , Mar [10] Q. Gao, L. Zhang, and D. Zhang, Face Recognition Using FLDA with Single raining Image per Person, Applied Math. and Computation, vol. 205, no. 2, pp , [11] X. Geng, D. Zhan, and Z. Zhou, Supervised Nonlinear Dimensionality Reduction for Visualization and Classification, IEEE rans. Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 35, no. 6, pp , Dec [12] R. Gottumukkal and V. Asari, An Improved Face Recognition echnique Based on Modular PCA Approach, Pattern Recognition Letters, vol. 25, no. 4, pp , [13] R. Gross, I. Matthews, and S. Baker, Generic vs. Person Specific Active Appearance Models, Image and Vision Computing, vol. 23, no. 12, pp , [14] X. He, D. Cai, S. Yan, and H. Zhang, Neighborhood Preserving Embedding, Proc. IEEE Int l Conf. Computer Vision, pp , [15] X. He, S. Yan, Y. Hu, P. Niyogi, and H.J. Zhang, Face Recognition Using Laplacianfaces, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp , Mar [16] H. Hu, Orthogonal Neighborhood Preserving Discriminant Analysis for Face Recognition, Pattern Recognition, vol. 41, no. 6, pp , [17] A. Jain and B. Chandrasekaran, Dimensionality and Sample Size Considerations in Pattern Recognition Practice, Handbook of Statistics, vol. 2, pp , [18]. Jebara, J. Wang, and S. Chang, Graph Construction and B-Matching for Semi-Supervised Learning, Proc. Int l Conf. Machine Learning, pp , [19] H. Kanan, K. Faez, and Y. Gao, Face Recognition Using Adaptively Weighted Patch PZM Array from a Single Exemplar Image per Person, Pattern Recognition, vol. 41, no. 12, pp , [20]. Kim and J. Kittler, Locally Linear Discriminant Analysis for Multimodally Distributed Classes for Face Recognition with a Single Model Image, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp , Mar [21] A. Lanitis, Evaluating the Performance of Face-Aging Algorithms, Proc. IEEE Int l Conf. Automatic Face and Gesture Recognition, pp. 1-6, [22] Z. Lei, S. Liao, M. Pietikainen, and S. Li, Face Recognition by Exploring Information Jointly in Space, Scale and Orientation, IEEE rans. Image Processing, vol. 20, no. 1, pp , Jan [23] H. Lu, K. Plataniotis, and A. Venetsanopoulos, MPCA: Multilinear Principal Component Analysis of ensor Objects, IEEE rans. Neural Networks, vol. 19, no. 1, pp , Jan [24] J. Lu and Y. an, A Doubly Weighted Approach for Appearance- Based Subspace Learning Methods, IEEE rans. Information Forensics and Security, vol. 5, no. 1, pp , Mar [25] J. Lu and Y.-P. an, Regularized Locality Preserving Projections and Its Extensions for Face Recognition, IEEE rans. Systems, Man, and Cybernetics: Part B, Cybernetics, vol. 40, no. 2, pp , June [26] J. Lu, Y.-P. an, G. Wang, and X. Zhou, Discriminative Multi- Manifold Analysis for Face Recognition from a Single raining Sample per Person, Proc. IEEE Int l Conf. Computer Vision, pp , [27] J. Luo, Y. Ma, E. akikawa, S. Lao, M. Kawade, and B. Lu, Person-Specific Sift Features for Face Recognition, Proc. IEEE Int l Conf. Acoustics, Speech, and Signal Processing, pp , [28] A. Martínez, Recognizing Imprecisely Localized, Partially Occluded, and Expression Variant Faces from a Single Sample per Class, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp , June [29] A. Martínez and A. Kak, PCA versus LDA, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp , Feb [30] A.M. Martinez and R. Benavente, he AR Face Database, technical report, CVC, [31] P. Phillips, H. Moon, S. Rizvi, and P. Rauss, he FERE Evaluation Methodology for Face-Recognition Algorithms, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp , Oct [32] P. Phillips, H. Wechsler, J. Huang, and P. Rauss, he FERE Database and Evaluation Procedure for Face-Recognition Algorithms, Image and Vision Computing, vol. 16, no. 5, pp , [33] D. Ridder, O. Kouropteva, O. Okun, M. Pietikäinen, and R. Duin, Supervised Locally Linear Embedding, Proc. Int l Conf. Artificial Neural Networks and Neural Information Processing, pp , [34] S. Roweis and L. Saul, Nonlinear Dimensionality Reduction by Locally Linear Embedding, Science, vol. 290, no. 5500, pp , [35] W. Schwartz, H. Guo, and L. Davis, A Robust and Scalable Approach to Face Identification, Proc. European Conf. Computer Vision, pp , [36] S. Si, D. ao, and B. Geng, Bregman Divergence Based Regularization for ransfer Subspace Learning, IEEE rans. Knowledge and Data Eng., vol. 22, no. 7, pp , July [37] Y. Su, S. Shan, X. Chen, and W. Gao, Adaptive Generic Learning for Face Recognition from a Single Sample per Person, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp , [38] X. an, S. Chen, Z. Zhou, and F. Zhang, Recognizing Partially Occluded, Expression Variant Faces from Single raining Image per Person with SOM and Soft K-NN Ensemble, IEEE rans. Neural Networks, vol. 16, no. 4, pp , July [39] X. an, S. Chen, Z. Zhou, and F. Zhang, Face Recognition from a Single Image per Person: A Survey, Pattern Recognition, vol. 39, no. 9, pp , [40] J.B. enenbaum, V. Silva, and J. Langford, A Global Geometric Framework for Nonlinear Dimensionality Reduction, Science, vol. 290, no. 5500, pp , [41] M. urk and A. Pentland, Eigenfaces for Recognition, J. Cognitive Neuroscience, vol. 3, no. 1, pp , [42] N. Vu and A. Caplier, Face Recognition with Patterns of Oriented Edge Magnitudes, Proc. European Conf. Computer Vision, pp , [43] S. Vu et al., Mining Patterns of Orientations and Magnitudes for Face Recognition, Proc. Int l Joint Conf. Biometrics, pp. 1-8, [44] R. Wang and X. Chen, Manifold Discriminant Analysis, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2009.

13 LU E AL.: DISCRIMINAIVE MULIMANIFOLD ANALYSIS FOR FACE RECOGNIION FROM A SINGLE RAINING SAMPLE PER PERSON 51 [45] R. Wang, S. Shan, X. Chen, and W. Gao, Manifold-Manifold Distance with Application to Face Recognition Based on Image Set, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, [46] X. Wang and X. ang, A Unified Framework for Subspace Face Recognition, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp , Sept [47] X. Wang and X. ang, Random Sampling for Subspace Face Recognition, Int l J. Computer Vision, vol. 70, no. 1, pp , [48] J. Wu and Z. Zhou, Face Recognition with One raining Image per Person, Pattern Recognition Letters, vol. 23, no. 14, pp , [49] S. Yan, J. Liu, X. ang, and. Huang, A Parameter-Free Framework for General Supervised Subspace Learning, IEEE rans. Information Forensics and Security, vol. 2, no. 1, pp , Mar [50] S. Yan, D. Xu, Q. Yang, L. Zhang, X. ang, and H. Zhang, Multilinear Discriminant Analysis for Face Recognition, IEEE rans. Image Processing, vol. 16, no. 1, pp , Jan [51] S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin, Graph Embedding and Extensions: A General Framework for Dimensionality Reduction, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp , Jan [52] J. Yang, D. Zhang, A. Frangi, and J. Yang, wo-dimensional PCA: A New Approach to Appearance-Based Face Representation and Recognition, IEEE rans. Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp , Jan [53] M.H. Yang, Kernel Eigenfaces vs. Kernel Fisherfaces: Face Recognition Using Kernel Methods, Proc. IEEE Int l Conf. Face and Gesture Recognition, pp , [54] B. Yao, H. Ai, and S. Lao, Person-Specific Face Recognition in Unconstrained Environments: A Combination of Offline and Online Learning, Proc. IEEE Int l Conf. Automatic Face and Gesture Recognition, pp. 1-8, [55] W. Yu, X. eng, and C. Liu, Face Recognition Using Discriminant Locality Preserving Projections, Image and Vision Computing, vol. 24, no. 3, pp , [56] B. Zhang, Y. Gao, S. Zhao, and J. Liu, Local Derivative Pattern versus Local Binary Pattern: Face Recognition with High-Order Local Pattern Descriptor, IEEE rans. Image Processing, vol. 19, no. 2, pp , Feb [57] B. Zhang, S. Shan, X. Chen, and W. Gao, Histogram of Gabor Phase Patterns (HGPP): A Novel Object Representation Approach for Face Recognition, IEEE rans. Image Processing, vol. 16, no. 1, pp , Jan [58] D. Zhang, S. Chen, and Z. Zhou, A New Face Recognition Method Based on SVD Perturbation for Single Example Image per Person, Applied Math. and Computation, vol. 163, no. 2, pp , [59] D. Zhang and Z. Zhou, ð2dþ 2 PCA: wo-directional wo- Dimensional PCA for Efficient Face Representation and Recognition, Neurocomputing, vol. 69, nos. 1-3, pp , [60] W. Zhang, S. Shan, W. Gao, X. Chen, and H. Zhang, Local Gabor Binary Pattern Histogram Sequence (LGBPHS): A Novel Non- Statistical Model for Face Representation and Recognition, Proc. IEEE Int l Conf. Computer Vision, pp , [61] J. Zou, Q. Ji, and G. Nagy, A Comparative Study of Local Matching Approach for Face Recognition, IEEE rans. Image Processing, vol. 16, no. 10, pp , Oct Jiwen Lu received the BEng degree in mechanical engineering, the MEng degree in electrical engineering from Xi an University of echnology, China, and the PhD degree in electrical engineering from Nanyang echnological University, Singapore, respectively. Currently, he is working as a research fellow at the Advanced Digital Sciences Center, Singapore. His research interests include computer vision, pattern recognition, machine learning, multimedia, and biometrics. He has authored/coauthored more than 60 scientific papers in peer-reviewed journals and conferences, including some top venues such as the IEEE ransactions on Pattern Analysis and Machine Intelligence, IEEE ransactions on Image Prpcoessing, IEEE ransactions on Systems, Man, and Cybernetics-Part B, IEEE ransactions on Systems, Man, and Cybernetics-Part C, IFS, ICCV, CVPR, and ACM MM. He received the first-prize National Scholarship and the National Outstanding Student awarded by the Ministry of Education of China in 2002 and 2003 and the Best Student Paper Award from PREMIA of Singapore in 2012, respectively. He is a member of the IEEE. Yap-Peng an received the BS degree from National aiwan University, aipei, aiwan, in 1993, and the MA and PhD degrees from Princeton University, New Jersey, in 1995 and 1997, respectively, all in electrical engineering. From 1997 to 1999, he was with Intel Corporation, Chandler, Arizona, and Sharp Laboratories of America, Camas, Washington. In November 1999, he joined Nanyang echnological University of Singapore, where he is currently an associate professor and associate chair (Academic) of the School of Electrical and Electronic Engineering. His research interests include image and video processing, content-based multimedia analysis, computer vision, and pattern recognition. He is the principal inventor or co-inventor on 15 US patents in the areas of image and video processing. He was the recipient of an IBM Graduate Fellowship from the IBM.J. Watson Research Center, Yorktown Heights, New York, from 1995 to He currently serves as the secretary of the Visual Signal Processing and Communications echnical Committee of the IEEE Circuits and Systems Society, a member of the Multimedia Signal Processing echnical Committee of the IEEE Signal Processing Society, and a voting member of the ICME Steering Committee. He is an editorial board member of the EURASIP Journal on Advances in Signal Processing and EURASIP Journal on Image and Video Processing, and an associate editor of the Journal of Signal Processing Systems. He was also a guest editor for special issues of several journals, including the IEEE ransactions on Multimedia. He cochaired the 2010 IEEE International Conference on Multimedia and Expo (ICME 2010). He is a senior member of the IEEE. Gang Wang received the BS degree from Harbin Institute of echnology in electrical engineering in 2005 and the PhD degree from the Department of Electrical and Computer Engineering, University of Illinois at Urbana- Champaign (UIUC) in He is working as an assistant professor in the School of Electrical and Electronic Engineering, Nanyang echnological University, and as a research scientist at the Advanced Digital Sciences Center, Singapore. His research interests include computer vision and machine learning. He received the Harriett & Robert Perry Fellowship ( ) and CS/AI award (2009) at UIUC. He is a member of the IEEE.. For more information on this or any other computing topic, please visit our Digital Library at