Gabors Help Supervised Manifold Learning

Transcription

1 Gabors Help Supervised Manifold Learning Junping Zhang 1,, Chao Shen 1, Jufu Feng 2 1 Shanghai Key Laboratory of Intelligent Information Processing Department of Computer Science and Engineering, Fudan University, Shanghai , China {jpzhang, }@fudan.edu.cn 2 Center for Information Science, National Key Laboratory for Machine Perception School of Electronics Engineering and Computer Science Peking University, Beijing , China fjf@cis.pku.edu.cn Abstract. As one of nonlinear dimensionality reduction approaches, manifold learning algorithms can discover intrinsic low-dimensional manifold or subspace embedded in the high-dimensional Euclidean space. However, the discriminant ability of the low-dimensional subspaces obtained by the algorithms is often lower than those obtained by the conventional dimensionality reduction approaches. Furthermore, the original feature vectors may include redundancy such as high-order correlation which cannot be removed by manifold learning algorithm. In this paper, we attempt to address the two problems through classifying face images. First, Gabor wavelets are employed to remove intrinsic redundancies within images and obtain a set of over-completed feature vectors. Then the previous proposed supervised manifold learning algorithm (ULLELDA) is applied to project Gabor-based data and out-ofthe-samples into a common low-dimensional subspace. Experiments in two FERET face databases indicate that Gabors indeed help supervised manifold learning to remarkably improve the discriminant ability of lowdimensional subspaces. 1 Introduction Recently, manifold learning has attracted the attention of researchers both in machine learning and computer vision. Many impressive algorithms, such as the Locally Linear Embedding (LLE) algorithm [1] and the Isometric mapping (ISOMAP) algorithm [2], were proposed to recover a low-dimensional manifold embedded in a high-dimensional Euclidean space. However, it is difficult to obtain a more effective discriminant subspace (i.e., the discriminant subspace has the ability to improve classification accuracy when data are projected into the subspace) through manifold learning than through conventional dimensionality reduction approaches. One possible reason is that high-order information of feature vectors hasn t been extracted as feature vectors. The other reason is that many manifold learning algorithms lay stress on recovering a global coordinate system but put less attention on enhancing the discriminant ability of the coordinate system.

2 In this paper, we focus our research on face images which are empirically thought of as a low-dimensional manifold embedded in the observation space [3, 4]. First, Gabor wavelets are applied to calculate high-order statistical information of face images and obtain a group of over-completed feature basis sets. Then, a supervised manifold learning approach, unified locally linear embedding and linear discriminant analysis (ULLELDA) is adopted for projecting data and out-of-the-samples into a discriminant subspace. For simplicity, the two combinations of Gabor wavelets and ULLELDA are abbreviated to GF+ULLELDA and GT+ULLELDA, respectively. Once data are projected into the discriminant subspace, the face images are recognized by some classifiers. Experiments in two face databases show that when several classifiers are applied in the lowdimensional subspaces through several dimensionality reduction approaches, the combination of Gabors with ULLELDA obtains the highest discriminant ability than conventional principal component analysis and ULLELDA. The remainder of this paper is organized as follows. In Section 2 we propose the combination of Gabors and supervised manifold learning approach. We report experiments on two face databases in Section 3. We end up with a discussion in Section 4. 2 The Proposed combination approaches of Gabor and the ULLELDA Algorithm Some high-order and local statistics in the images are important but may be neglected for supervised manifold learning if we regard each pixel as one dimension. One way to address the issue is to map images into frequency domain where unnoticeable information in the spatial domain will become clear (e.g., periodic noise). It is our motivation to use Gabor wavelets to improve the discriminant ability of supervised manifold learning. For better understanding of the algorithm, we divide the introduction of the proposed algorithms into two subsections. 2.1 Gabor Feature Representation Gabor wavelets have been widely adopted in the field of face recognition and image segmentation in recent years. They are, in theory, particularly aggressive at capturing the features of the local structures corresponding to spatial frequency, spatial localization, and orientation selectively [5, 6]. Consequently, it is reasonably believed that they are able to extract perfect features from which we can get a more effective feature function. The Gabor function is basically a Gaussian modulated by a complex sinusoid. An example of Gabors is that the kernel of the 2D Gabor wavelets commonly used in face recognition (henceforth GF), has the following expression at the spatial position x = (x 1, x 2 ) [5]: ψ µ,ν (x) = k µ,ν 2 σ 2 exp ( k µ,ν 2 x 2 2σ 2 )[exp (ik µ,ν x) exp ( σ2 )] (1) 2

3 where µ and ν define the orientation and scale of the Gabor filter, which is a product of a Gaussian envelope and a complex oscillation with wave vector κ u,v. k µ,ν = k ν e iφmu, k ν = k max /f ν, φ mu = πµ/8, k max is the maximum frequency and f is the spacing factor between wavelets in the frequency domain. And we choose the following parameters: σ = 2π, k max = π/2, and f = 2 [7]. In most cases one would use Gabor wavelets of five different scales, ν {0,..., 4} and eight orientations, µ {0,..., 7}. Figure 1(a) show the real part of the GF kernels at five scales and eight orientations. Real parts of Gabor wavelet Real Parts of Gabor wavelet in the Frequency Domain scale scale orientation (a) orientation (b) Fig. 1. GF wavelets. (a) The real part of the GF kernels at five scales and eight orientations. (b) GF wavelets representation of the first image in the Figure 2(a) in the Frequency domain The Gabor wavelet representation of an image z is the convolution of the image with a family of Gabor kernels as defined by (1). Let I(x, y) be an intensity image, the convolution of image I and a Gabor kernel ψ µ,ν is defined as follows: O µ,ν (z) = I(z) ψ µ,ν (z) (2) where z = (x, y), denotes the convolution operator, and O µ,ν (z) is the convolution result corresponding to the Gabor kernel at orientation µ and scale ν. Therefore, the set S = {O µ,ν (z) : µ {0,...4}, ν {0,..., 7}} forms the Gabor wavelets representation of the image I(x, y). We can compute O µ,ν (z) using FFT and inverse FFT: O µ,ν (z) = I 1 {I{I(z)}I{ψ µ,ν (z)}} (3) where I and I 1 denote the FFT and inverse FFT respectively. Considering computational cost, we first down-sample each O µ,ν (z) to reduce the size of the image, and then concatenate all row vectors of each image into a single vector. Finally, when the number of scales is 5 and the number of

4 orientations is 8, the Gabor wavelet representation of an image is: χ ρ = (O ρ 0,0, Oρ 0,1,, Oρ i,j,, Oρ 4,7 ) (4) where the subscript i, j of O denotes scale and orientation, respectively, and the superscript ρ = 2 k denotes the degree of downsampling (k: steps of halving the image). An example of Gabor representation of an image is illustrated as in Figure 1(b). It is not difficult to see that image in Frequency domain is sparse and localization. Applying different complex oscillation or modulation function in Gabor filter, some other forms of Gabor wavelets are used in feature extraction, texture analysis [8], and stereo disparity estimation. In this paper, Gabor wavelet for texture analysis (henceforth GT) is also employed to extract high-order information [8]. 2.2 The ULLELDA Algorithm After high-order information is extracted by the Gabor wavelets, an immediate problem to be solved is that dimensions need to be reduced to overcome the curse of dimensionality and recover the intrinsic low-dimensional manifold. Furthermore, to improve classification, the subspace learned from images should have the ability to enlarge between-cluster distance and decrease withincluster distance. Therefore, among the many supervised manifold learning approaches, the previous proposed ULLELDA algorithm is adopted for projecting high-dimensional data into a corresponding low-dimensional discriminant subspace [9]. The basic framework of the ULLELDA algorithm can be summarized as followings: 1. Let the neighborhood factor be K and reduced dimension be d. Training samples are first projected into a d-dimensional subspace through the LLE algorithm. The first step of the LLE algorithm is to approximate manifold around sample x i χ ρ with a linear hyperplane passing through its neighbors {x i1,..., x ik } χ ρ. And the weights w ij is calculated through minimizing the cost ψ(w ) = n K x i w ij x ij 2 x i, x ij R N (5) i=1 j=1 by constrained linear fitting. Where n denotes the number of training samples. j w ij = 1 and w ij = 0 if x ij does not belong to the neighborhood set of x i. 2. Assuming that the weight relationship of sample and its neighborhood samples is invariant when data are mapping into a low-dimensional subspace. Therefore, define n Φ(Y ) = y i wijy ij 2 (6) i=1 j

5 Where W = arg min W ψ(w ). Subjecting to constraints i y i = 0 and i y iyi T = I, the optimal solution of Y in Formula (6) is the smallest or bottom d + 1 eigenvectors of matrix (I W ) T (I W ) except that the zero eigenvalue needs to be discarded. 3. The d-dimensional data are further mapped into a d -dimensional discriminant subspace through the LDA algorithm (linear discriminant analysis). Suppose that the occurrence of each class is equally probable, intra-class scatter matrix is defined as: L n i S w = (y j m i )(y j m i ) T (7) i=1 j=1 for n i samples from class i with class-specific mean m i, i = 1, 2,..., L. For the overall mean m of all the classes, meanwhile, the inter-class scatter matrix is defined as L S b = (m i m)(m i m) T (8) i=1 To maximize the inter-class distances while minimizing the intra-class distances of face manifolds, the column vectors of discriminant matrix D are calculated by the eigenvectors of Sw 1 S b associated with the largest eigenvalues. Then the matrix D projects vectors in the low-dimensional face subspace into the common discriminant subspace which can be formulated as follows: Z = DY Z R d, Y R d, D R d d 4. To project out-of-the-sample into the discriminant subspace, we assume that the neighbor weights among the sample and its neighbor samples from training set are the same as those in the discriminant subspace. Therefore, the weights of and the low-dimensional positions of out-of-the-samples are calculated as follows: K φ(w ) = x i w i jx i j x i j R N (10) z i = j=1 (9) K w ijz i j z i j R d (11) i=1 Where x i denote an out-of-the-sample. Once all the test images or out-of-the-samples are projected into the discriminant subspace, a classifier can be employed to identify unlabelled samples. 3 Experiments To verify whether the proposed combination approach of Gabors and the UL- LELDA algorithm can project high-dimensional data into a more effective discriminant subspace, we manually selected two collections of face images from

6 the FERET databases [10] to construct two face databases. They both include 43 person, each with 10 images of the face, varying intrinsic features such as illumination and expression. The difference of the two databases is that the first database is mainly composed of frontal images while the second pose variation is introduced. In the following text, we use FERET1 and FERET2 to call these two databases respectively. All the initial images are pixels with 8-gray levels. The intensity of each pixel is regarded as one dimension. Thus, pixels are equal to dimensions. In this paper, the images in FERET1 are first cropped to pixels for removing the influences of background and hairstyle, meanwhile, face images are roughly and manually aligned. Moreover, the sizes of some face images which are less than pixels are zoomed into the pixels with bicubic interpolation technique. To the FERET2, we do the same preprocessor except that ellipse mask is used and images are zoomed into instead of cropping the images to The cropped face images of two face databases are illustrated in Figure 2 (Note: The figure can be seen clearer in the PDF file). Finally, histogram equalization is applied to correct the illumination and normalization is carried out on the whole databases. (a) Cropped Face images in FERET1 (b) Ellipse Masked Face Images in FERET2 Fig. 2. Examples of Face images 3.1 The Parameter Setting of The GF+ULLELDA Algorithm To perform the GF+ULLELDA algorithm, several parameters need to be fixed. First, for the GF wavelet, we choose 5 scales (0, 1, 2, 3, 4) and 8 orientations (0, 1, 2,, 7). Thus, a vector of dimension is obtained for each image. After the GF feature vector of each image is obtained, the GF feature vectors are downsampled (where ρ = 8) to get a matrix of for each scale and orientation, so, the whole feature dimensions are equal to For the

7 GT wavelet, the parameter setting is similar. Considering the limitation of the length, we omit the introduction of the GT wavelet in this paper. Second, the two reduced dimensions d and d of the ULLELDA algorithm are fixed. d is set to be 120. For the second mapping (LDA-based reduction), the reduced discriminant dimension L is generally no more than C 1 (where C denotes the number of face clusters). Otherwise eigenvalues and eigenvectors will have complex values. Actually, we remain the real number of complex values when the second reduced dimension is higher than C 1. In addition, the neighbor factor K of the LLE algorithm is set to be Other Dimensionality Reduction Approaches For comparing the performance of the proposed dimensionality reduction techniques, several combined dimensionality reduction approaches are employed to project the complex face database into different discriminant subspaces. The approaches include ULLELDA [9], PCALDA [11,12], GF+PCALDA, GT+ULLELDA and GT+PCALDA. It is noticeable that the formations of two wavelets (GF and GT) are remarkably different by employing different modulation functions. 3.3 Classification Algorithms Finally, three classifiers (mean algorithm, 1-nearest neighbor algorithm and pairwise SVM-linear algorithm [13]) are adopted for evaluating which dimensionality reduction technique is better for classifying face databases. In the mean algorithm, the prototype of each person is represented by an average face of the same person, and the cluster of the unknown sample corresponds to the minimum Euclidean distance from the sample to the prototypes. Furthermore, the face database is randomly divided into a training set and test set without overlapping. For each person, 5 images are selected to be training samples and the remaining ones are test samples. The experimental results are the average of 20 repetitions. From the table 1 it is clear that when only the ULLELDA algorithm is employed, the discriminant ability is the worst one in the mentioned dimensionality reduction approaches for the two face databases. However, when combining the Gabor wavelets and supervised manifold learning, the discriminant ability of corresponding subspace is remarkable improved. For example, to FERET1, the average error rate using GF+ULLELDA with mean algorithm is about 55.49% of the GF+PCALDA, 45.80% of the GT+PCALDA, 58.64% of the GF+PCALDA, 28.63% of the PCALDA, 25.49% of the ULLELDA. Furthermore, the combination of GF+ULLELDA and several classifiers achieves the lowest error rates in the table 1. We also investigate the influence of training samples each class. The results can be seen in Figure 3(a) and Figure 3(b). For FERET1, the performance of the GF+ULLELDA algorithm is the best one. For FERET2, when the number of training samples each class is less than 6, the performance of the GF+ULLELDA is the best. And when the number is greater than 6, the performance of the

8 Table 1. The error rates (%) and standard deviation (%) of the combination of dimensionality reduction and several classifiers in FERET1 and FERET2. K = 50, L = 50 Mean 1-NN SVM-Linear FERET1 PCALDA ± ± ± 2.56 ULLELDA ± ± ± 2.69 GF+PCALDA ± ± ± 3.19 GF+ULLELDA 5.56 ± ± ± 2.08 GT+PCALDA ± ± ± 2.97 GT+ULLELDA 6.41± ± ± 2.25 FERET2 PCALDA ± ± ± 2.64 ULLELDA ± ± ± 3.47 GF+PCALDA ± ± ± 3.68 GF+ULLELDA ± ± ± 3.31 GT+PCALDA ± ± ± 4.12 GT+ULLELDA 21.77± ± ± 3.75 GF+ULLELDA is comparable to other dimensionality reduction approaches. By analyzing the table and Figures, it is obvious that Gabor wavelets, both GF and GT, do help supervised manifold learning achieve a remarkable improvement on recognition accuracy. Finally, some parameters influence the performance of the GF+ULLELDA algorithm such as the reduced discriminant dimension L and the neighbor factor K. The results on the parameters are illustrated in Figure 4(a) and Figure 4(b). From Figure 4(a) it can be seen that as L decreases, the GF+ULLELDA has a steeper ramp in the error rate curve than other approaches. We guess that some valuable discriminant information may be included in small eigenvectors of the discriminant subspace based on the GF+ULLELDA. From the Figure 4(b) we observe that when the reduced discriminant dimensions are close to 10, the influence of the neighbor factor on the error rate is negligible. However, when reduced discriminant dimensions are close to the number of classes, for example, 50 in the face database, the influence is remarkable. For instance, when the neighbor factor K is equal to 50, the error rate is 5.56%. When the neighbor factor K is equal to ten, however, the error rate is 12.98%. Therefore, how to select a suitable neighbor factor is worthy of further research.

9 classifier:mean algorithm, K=50, Reduced Dimension=50 ULLELDA PCALDA GF+ULLELDA GF+PCALDA GT+ULLELDA GT+PCALDA classifier:mean algorithm, K=50, Reduced Dimension=50 ULLELDA PCALDA GF+ULLELDA GF+PCALDA GT+ULLELDA GT+PCALDA error rate 0.2 error rate train sample number (a) FERET train sample number (b) FERET2 Fig. 3. Recognition with several dimensionality reduction algorithms 4 Conclusion In this paper, we propose a combination approach of Gabor wavelet and supervised manifold learning. Two specific Gabor wavelets, Gabor for face (GF) [7] and Gabor for texture (GT) [8] are employed to extract high-order and nonorthogonal feature vectors of face images, and the ULLELDA algorithm is adopted to project these vectors into a discriminant subspace. Compared with other dimensionality reduction techniques, the proposed combination techniques can help classifiers to obtain higher recognition performance in face database. Meanwhile, the discriminant ability based on supervised manifold learning is remarkably improved through the introduction of two Gabor wavelets. Several problems deserve further study. First, since the main difference between GF and GT is selecting different complex oscillations in kernel function, how to choose a proper form of Gabor wavelets in certain application? Second, we observe that when using ULLELDA without GF wavelet, PCALDA can obtain better accuracy than ULLELDA in the two face databases. Therefore, how to extract features of face images seems to be a key to supervised manifold learning. Maybe using Ada-Boosted Gabor Features can help us to further refine supervised manifold learning algorithm. Finally, the neighbor factor influences the generation of low-dimensional discriminant subspace. The way to automatically select the factor will be studied in the future. Acknowledgement This research is partially sponsored by NSFC under contract No Part of the research in this paper uses the Gray Level FERET database of facial images collected under the FERET program.

10 error rate classifier:mean algorithm, K=50, train sample=5 ULLELDA PCALDA GF+ULLELDA GF+PCALDA GT+ULLELDA GT+PCALDA error rate Reduced Dimension Reduced Dimension L Neighbor Factor K 50 (a) The influence of the reduced dimension (b) The influence of the neighbor factor K. L. The neighbor factor K is 50. The number The number of training samples is five per of training samples is five per person person Fig. 4. Experiments on the influence of some parameters to FERET1 References 1. Roweis, S. T., Lawrance, K. S.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science, 290, (2000) Tenenbaum, J. B., de Silva, Langford, J. C.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science, 290, (2000) Lu, H. M. Fainman, Y., Hecht-Nieslen, R.: Image Manifolds. in Proceedings of SPIE, 3307, (1998) Nayar, S. K., Nene, S. A., Murase, H.: Subspace Methods for Robot Vision. Technical Report CUCS-06-95, Columbia University, New York, (1995) 5. Daugman, J. G.: Uncertainly relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filtres. J. Opt. Soc. Amer. A., Vol. 2. (1985) Daugman, J. G.: Complete Discrete 2-D Gabor Transforms by Neural Networks for Image Analysis and Compression. IEEE Trans. on Acoustics, Speech, and Signal Processing, Vol. 36, No. 7. (1988) Liu, C., Wechsler, H.: A Gabor Feature Classifier for Face Recognition. ICCV01, (2001) Manjunath, B. S., Ma, W. Y.: Texture features for browsing and retrieval of image data. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 18, No. 8. (1996) Zhang, J., Shen, H., Zhou, Z. H.: Unified Locally Linear Embedding and Linear Discriminant Analysis Algorithm (ULLELDA) for Face Recognition. Advances in Biometric Personal Authentication. Lecture Notes in Computer Science, Vol Springer-Verlag, (2004) Phillips, P. J., Moon, H., Rizvi, S. A., Rauss, P. J.: The FERET Evaluation Methodology for Face Recognition Algorithms. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 22. (2000) Turk, M., Pentland, A.: Eigenfaces for Recognition. Journal of Cognitive Neuroscience, Vol. 3, No.1. (1991) Swets, D. L., Weng, J.: Using Discriminant Eigenfeatures for Image Retrieval. IEEE Trans. on Pattern Analysis and Machine Intelligences, Vol. 18, No. 8. (1996) Cawley, G. C.: MATLAB Support Vector Machine Toolbox (v0.50β) [ University of East Anglia, School of Information Systems, Norwich, Norfolk, U.K. NR4 7TJ, (2000) 14. Yang, P., Shan, S., Gao, W., Li, S. Z., Zhang D.: Face Recognition Using Ada-Boosted Gabor Features. FGR 04, (2004)