Neurocomputing. Local Kernel Feature Analysis (LKFA) for object recognition. Baochang Zhang a,n, Yongsheng Gao b, Hong Zheng a. abstract.

Transcription

1 Neurocomputing 74 (2011) Contents lists available at ScienceDirect Neurocomputing journal homepage: Local Kernel Feature Analysis (LKFA) for object recognition Baochang Zhang a,n, Yongsheng Gao b, Hong Zheng a a National Key Laboratory of Science and Technology on Integrated Control Technology, School of Automation Science and Electrical Engineering, Beihang University, Beijing, China b School of Engineering, Griffith University, Australia article info Article history: Received 16 December 2009 Received in revised form 20 April 2010 Accepted 15 September 2010 Communicated by D. Tao Available online 16 October 2010 Keywords: Local Kernel Biometric Face Palmprint abstract This paper proposes a new Local Kernel Feature Analysis (LKFA) method for object recognition. LKFA captures the nonlinear local relationship in an image via kernel functions. Different from traditional kernel methods for object recognition, the proposed method does not need to reserve the training samples. LKFA is designed to etract the eigenvalue features from the Hermite matri of a local feature representation, which we have theoretically proven its robustness to noise and perturbations. Eperiment results on palmprint and face recognitions demonstrated the effectiveness of the proposed LKFA that significantly improved the performance of the local feature based object recognition method. & 2010 Elsevier B.V. All rights reserved. 1. Introduction Feature representation is one of the key steps in a pattern recognition system. Recently, local features have drawn much attention in the appearance based methods, showing a promising way toward high performance in the real application system. Compared to its global representation counterpart, local representation can be more robust to changes of illumination, view points, occlusion, etc. Several successful vision systems that use local features have been implemented, demonstrating their effectiveness for real applications [1 7]. Many methods have been proposed to etract local features for object recognition, which focus on teture features such as edge, point, line. Moravec [1] developed one of the first signal based interest point detectors using auto-correlation function. It measures the greyvalue differences between a window and windows shifted in several directions. Harris corner detector [2], built upon the idea of [1], is a popular interest point detector due to its strong invariance to rotation, scale, illumination variation, and image noise. Scale-invariant feature transform (SIFT) operator [4] is a well-known technology to etract interest points from a given image based on the local appearance of the object. The SIFT features are invariant to image scale and rotation, which has been widely used in the field of image processing and pattern recognition. Canny [3] considered the mathematical problem of deriving an optimal smoothing filter given the criteria of detection, localization, and minimizing multiple responses to a single edge. n Corresponding author. address: bczhang@buaa.edu.cn (B. Zhang). The distribution analysis of local pattern features is another way in representing objects for recognition purposes. For eample, Local Binary Pattern histogram features were used to model the distribution of the micro-pattern representing edge, point, etc., which has been widely used in teture analysis and object recognition [7 9]. These kinds of features, including Harris corner, LBP, and SIFT, only pay attention to the relationship between a given point and its neighborhoods in the original space. However, no work is reported on the investigation into the relationship between two local features of a single image in the high-dimensional space, which can actually reveal the nonlinear structure information of the input object. The statistic learning based methods such as Principal Component Analysis (PCA) [10,11], Fisher Linear Analysis (FLA) [12,13], Kernel Principle Component Analysis (KPCA) [15], and Kernel Fisher Analysis (KFA) [14] are popular ways to etract global features. When using these methods to etract local features, the researchers often apply them on local regions [14,19] instead of on the global image. Different from linear methods such as PCA and Fisher analysis, KPCA and KFA is more effective on nonlinear feature etraction. However, a common limitation of the traditional KPCA and KFA for feature etraction is that they have to reserve the training samples or part of the training dataset [14,15], which cause the storage space problem for comple pattern recognition systems. Similarly, SVM classifiers [16] also need to save support vectors of the training database. Traditionally, a kernel-based method uses kernel functions to calculate the inner product of different input image samples in a high-dimensionality space. Different from the eisting works, this paper proposes a new Local Kernel Feature Analysis (LKFA) method for local feature etraction, which calculates the local kernel similarity from a single input image without the /$ - see front matter & 2010 Elsevier B.V. All rights reserved. doi: /j.neucom

2 576 B. Zhang et al. / Neurocomputing 74 (2011) training samples saved. LKFA is proposed to capture the nonlinear relationship among local neighborhoods by using the kernel function. It is proved in this paper that the proposed LKFA method is robust to noise. Moreover, the LKFA can be further combined with Fisher Linear Analysis for better performance in applications with multiple training eamples per class. The rest part of the paper is organized as follows. Section 2 briefly reviews the related work. In Section 3, we describe the details of the proposed method and theoretical analysis. Section 4 presents etensive eperiments on both palmprint and face recognition. Conclusions are drawn in Section Related work Kernel function is successfully used on SVM, KPCA, and KFA which are nonlinear etensions to the linear methods. Ecept for directly using the classic kernel functions such as Gaussian, polynomial, and RBF kernels, many researchers focus on designing new types of kernels for improved performances in various applications. For eample, inspired by similarity measure, histogram intersection (HI) [17] and Gaussian weighted chi-square kernel (GWchi) [18] are designed as new kernel functions for vision application. Wallraven et. al. [19] trained a local kernel from multiple sample images in a given database, showing that local kernel is a promising way toward high performance in real applications, though it has to save part of the training samples. Derived from a general definition of teture in a local neighborhood, Local Binary Pattern (LBP) [7] is defined as a grey-scale invariant teture measure and is regarded as a useful tool to model teture images. LBP later has shown ecellent performance in many comparative studies, in terms of both speed and discrimination performance [7 9]. The original LBP operator labels the piels of an image by thresholding the 3 3 neighborhood of each piel with the value of the central piel and concatenating the results to form a number. The thresholding function f() for LBP can be formally represented as ( f ðiðz i Þ,IðZ 0 ÞÞ ¼ 0, if IðZ iþ IðZ 0 Þrthreshold ð1þ 1, if IðZ i Þ IðZ 0 Þ4threshold where Z i, i¼1, y,8 is an 8-neighborhood point around Z 0 as shown. A LBP can also be considered as the concatenation of the binary gradient directions, and is called a micro-pattern. The histograms of these micro-patterns contain information of the distribution of the edges, spots, and other local features in an image. In this paper, the proposed Local Kernel Feature Analysis (LKFA) uses LBP as a representative local feature for object recognition and its performance is benchmarked with a standard LBP. 3. Local Kernel Feature Analysis In kernel-based methods, the input data is first projected into an implicit feature space F represented by a function F(). F is not necessarily eplicit when computing the inner product of two vectors in F, as which can be calculated by using a kernel function: kð1,2þ¼ðfð1þufð2þþ ð2þ where F(1) and F(2) are two vectors in F corresponding to 1 and 2 in the original space, and k(,) is the kernel function. An input image I, divided into m sub-regions, can be represented as an ensemble of local features (L i (I), i¼1, y,m). We further investigate nonlinear relationship among these features etracted from difference sub-regions of an image. This actually etracts the intrinsic topology information contained in the teture of an object image. We eploit the kernel product to calculate the nonlinear structure relationship among local features. The Local Kernel Feature (LKF) is a symmetric matri, element of which is defined as LKFði,jÞ¼kðL i,l j Þ, i,j ¼ 1,...,m ð3þ where L i in this paper is the histogram feature. When 1,2 in Eq. (2) is any pair of L i, L j, the kernel function can be redefined using the chi-square distance as: kð1,2þ¼k GWchi ð1,2þ ð4þ K GWchi ð1,2þ¼epð r* GWchi ð1,2þþ where S GWchi ð1,2þ¼ PB ð1 i 2 i Þ 2 =1 i þ2 i is the Chi-square i ¼ 1 statistic, and r is a constant. K GW chi (1,2) had been proved to be positively definite in [17,18]. Eq. (4) is equivalent to performing inner product of two local features in the high-dimensional space. It is obvious that LKF is a semi positive-definite matri, which is also a Hermite matri as can be seen from its definition in Eq. (3). Compared to KPCA and KFA, we know that LKFA merely etracts features from itself; therefore, avoiding the storage problem as no training sample needs to be saved. The LKF matri is a generic nonlinear feature; however, it is symmetric which contains redundant information. To obtain a compact and robust feature representation from LKF, we calculate its eigenvalue vector, as the final etracted feature which is proven below to be robust to a small degree of noise such as the Gaussian noise: Let A be a LKF matri defined in the above section, E be a symmetric noise matri, and A+E be the LKF matri corrupted with noise. It should be noted that E is symmetric because the noise added on a sub-region feature is used to calculate LKF resulting in symmetric matri as shown in Eq. (3). Therefore, A, E, and A+E are all Hermite matrices. Their eigenvalue vectors of A, E, and A+E are l 1 Zl 2 Z Zl n, e 1 Ze 2 Z Ze n, and m 1 Zm 2 Z Zm n respectively. For the Hermite matri H, we can have the Rayleigh quotient as R H ðþ¼ T H ð6þ T Then we can define the eigenvalue l k as follows: l k ¼ ma min R AðÞ ð7þ C k A C k, a 0 where c k can be any k-dimensional subspace. Thus we have m k ¼ ma min R T A A þ EðÞ¼ma min C k A C k, a 0 C k A C k, a 0 T þ T E T ð8þ l k ¼ ma C k min A C k, a 0 T A T e T E k ¼ ma min C k A C k, a 0 T As the eigenvalue is in the decreasing order, we have e T E 1 Zma min C k A C k, a 0 T Ze n From Eqs. (8), (9), and 11, we have m k rl k þe 1 We also know that A ¼ðAþEÞ E where A+E and E are both Hermite matrices, we can get l k ¼ m k e k Therefore, we have m k Zl k e n ð5þ ð9þ ð10þ ð11þ ð12þ ð13þ ð14þ ð15þ

3 B. Zhang et al. / Neurocomputing 74 (2011) From Eqs. (12) (15), we can know that l k e n rm k rl k þe 1 ð16þ classification procedure, we use the same cosine similarity rule as shown in Eq. 18. The trace of E T E is defined as trðe T EÞ¼ Xn e 2 i i ¼ 1 ð17þ where eotr(e T E), and tr(e T E) is in general small; therefore, the eigenvalue is robust to noise. We choose a sample (see Fig. 1a) from the FRGC database to show how LKFA can eliminate the noise. The image shown in Fig. 1c is obtained by adding the Gaussian noise with and 20/ ( ) as mean and variance on the original image. In LKFA, the first eigenvalue feature is etracted and rescaled to a 256 greyscale image (see Fig. 1c) for visualization purpose. The signal to noise ratio (SNR) is for Fig. 1a and b, but for Fig. 1c and d, which show that LKFA is effective on eliminating the noise. We randomly select 100 samples from the FRGC database, SNRs for the case as in Fig. 1a and b, Fig. 1c and d are and , respectively, which show that the proposed method has a better robustness to noise. In the classification procedure, assume v 1 and v 2, as the eigenvalue vectors generated from two LKF matrices of images P 1 and P 2,the cosine similarity rule is used to calculate the similarity as dðp 1,P 2 Þ¼ v1 Uv 2 :v 1 :U:v 2 : ð18þ For a comple object recognition problem such as face recognition, it is always beneficial to further eploit the subspace method, i.e. Fisher Linear Analysis, to etract the discriminant features from LKFA. If W T is the transformation matri calculated from the FLA method, we can get the etracted features as W T v 1,W T v, where v 1 and v 2 are calculated by the proposed LKFA method. In the 4. Eperiments In the eperiment, we eploit Gabor wavelet to enhance the performance of object recognition. The Gabor wavelets (kernels, filters) are defined as follows [13,20,21]: c u,v ðzþ¼ :k u,v: 2 where z ¼ s 2 e ð :ku,v: 2 2 :z: =2s 2 Þ!, k y u,v ¼ k! j ¼ k jy h i e iku,vz e s2 =2 k! vcosf u, k k v sinf v ¼ p=2 v=2, u ð19þ f u ¼ u p 8, v ¼ 0,...,v ma 1, u ¼ 0,...,u ma 1, v is the frequency, u is the orientation, with v ma ¼ 5, u ma ¼ 8, and s ¼ 2p for a given piel in the feature space, around which we have 8 neighbors. For eample, the 8 neighbors of Z 0 are Z 1 Z 8 as shown in Fig. 2(a). Therefore, the size of the LKFA matri at the position Z 0 is 9 9. In the eperiment, the final chosen number of the eigenvalues is 9 1¼8. However, for the marginal position like Z 1, Z 3, Z 5, Z 7, the size of LKFA matri is 4 4, and then the size of the final feature is 4 1¼3. The size of the LKFA matri for other marginal piels as Z 2, Z 4, Z 6, Z 8 is 6 6, and then the final etracted feature size is 5. It should be noted that LKFA is calculated from Gabor magnitude Eperiment on Polyu Palmprint database We first do eperiment on the Polyu Palmprint database [22], which is captured under different illumination conditions with deformation. The database has 600 images from 100 people, with 6 images per person. The gallery database contains 100 images with 1 image per person, and the remaining 500 images are used as the probe database. In this eperiment, the images in the database are normalized into the size of We first test the proposed approach on grey-level images, while the baseline algorithm is the nearest neighborhood classifier with the cosine similarity as the distance measure, which achieve 75.8% recognition rate. The proposed method is also based on the original image with each piel grey-value as the local feature as shown in Fig. 3. In this Z 1 Z 2 Z 3 Z 8 Z 0 Z 4 Fig. 1. illumination of the LKFA visualization: (a) is the original image, (b) is the image with Gaussian noise, (c) is the visualization of the first eigenvalue of LKF on image (a), (d) is the visualization of the first eigenvalue of LKF on image (b). Z 7 Z 6 Z 5 Fig. 3. Samples of Polyu Palmprint database. Fig. 2. The illustration of local kernel feature: (a) is local neighbor for different kind situation, (b) is the division method for the sized region, on which 16 sub-regions with size of 4 6 are further achieved.

4 578 B. Zhang et al. / Neurocomputing 74 (2011) situation, the dimension of the LKF matri for each piel is 9 9, from which we just calculate a 1D eigenvalue vector as the final feature. We achieve 77.7% and 81% recognition rates with polynomial kernel (parameter is 2) and Gaussian kernel, respectively. The performance of LKFA with Gaussian kernel is the best, which shows that the nonlinear Local Kernel Feature is more effective for Palmprint recognition. We perform the eperiment based on the Gabor magnitude LBP feature, the proposed LKFA method using LBP as the local feature achieves a better performance than LBP when the number of histogram bins is changed from 8 to 256 as shown in Fig. 3. LKFA with Gaussian kernel achieves the best result, which shows that the nonlinear relationship among local regions is useful for object recognition. LBP performs better when using a large number of histogram bins. It is interesting to note that even using a smaller number of histogram bins (e.g., 8 bins), LKFA still outperformed LBP with much larger number of histogram bins (e.g., 256 bins) Comparisons based on the FRGC Version 1 To verify the performance of the proposed method on face recognition, we conducted eperiments on the well-known FRGC database using Eperiment #4 protocol. Some samples are given as in Fig. 5. As shown in [23], the Eperiment #4 is designed for indoor controlled still images versus uncontrolled still images, which is the most challenging FRGC eperiment. In the FRGC Version 1, the training set contains 366 images, the target (gallery) set contains 943 controlled images, and the query (probe) set has 943 uncontrolled images. For Eperiment #4 of the FRGC Version 2, the training set contains 12,776 images, the target set includes 16,028 controlled images, and the query set has 8014 uncontrolled images [23]. In both eperiments, face images are normalized and cropped to the size of images, which are divided into sized sub-regions. Then we construct an ensemble of 8 6 classifiers based on the sum rule, and each one of which is applied on one sub-region with the size of To further reserve more spatial information, we divide each sub-region (classifier) into smaller regions with the size of 4 6, as shown in Fig. 2(b). In each region, we etract LBP histogram feature from the Gabor magnitude for LKFA, and the bin number is set as 8. The eperiment is first conducted on Eperiment #4 of the FRGC Version 1 by comparing LBP+Fisher, LBP+KFA, and LKFA+Fisher. In this eperiment, the mean sample for each class is calculated in the target set. It should be noted that the LBP+KFA method directly uses the LBP feature with the GWChi kernel. As shown in Fig. 4, the LKFA+Fisher method achieved a much better result than LBP+Fisher which confirms that the proposed method is effective toward high performance in the real application, as the FRGC database is designed for both controlled and uncontrolled conditions. Compared to the Efficient Kernel Fisher method [24], the proposed method achieved a higher performance as shown in Table Comparisons based on the FRGC Version 2 In this section, Eperiment #4 on the FRGC Version 2 is used to evaluate the performance of the face recognition system for face verification. Besides the LBP+Fisher method, we also compared our results with reported results of some state-of-the-art methods. From Table 2, it can be seen that the proposed method achieved a better performance than the original LBP based method, because LKFA can use the histogram similarity measure and capture the nonlinear structure of the input object. The LBP+Fisher method cannot effectively use the histogram measure, which may cause the decrease of the recognition performance [25]. The proposed method also achieves a much better result than some state-ofthe-art methods, which shows that LKFA is also an effective way to enhance the face recognition performance. Table 1 Recognition rates of LBP, LBP+KFA, LBP+Fisher, and LKFA+Fisher on FRGC Version 1. LBP Result in [24] LBP+KFA LBP+FLA LKFA+FLA Recognition rates Recognition Rates Histogram Bin Number Fig. 4. Comparative results on the Palmprint database. LBP LKFA Table 2 Performance comparison with some state-of-art results in the FRGC Version 2. Methods Baseline [23] 0.12 Result in [15] 0.76 Result in [26] Result in [24] LBP+FLA 0.78 LKFA+FLA 0.83 Recognition rates with FAR¼0.1% Fig. 5. Samples of FRGC database.

5 B. Zhang et al. / Neurocomputing 74 (2011) Conclusions and future work This paper proposes a new method, named Local Kernel Feature Analysis, for object recognition. Different from the traditional kernel-based method using kernel functions to calculate the inner product of different input samples in the high-dimensionality space, this paper eploits it for the feature etraction from a single input image to capture the relationship among local neighborhood. We also prove in theory that the eigenvalue vector feature is stable, which is etracted as the final feature for object representation. Comparative eperiments on both Palmprint and face recognition show that the proposed method achieves a better performance than the original LBP-based method. The kernel function affects the final performance of object recognition, we will focus on this topic to design new kernel for a better performance. The future work will also focus on the application of the proposed method on other object recognition. Acknowledgement The work was supported by the Natural Science Foundation of China (NSFC) under Contract No. No , and No This work is also supported by a grant from the Ph.D. Programs Foundation of Ministry of Education of China (No ) and the Fundamental Research Funds for the Central Universities. References [1] H.p. Moravec, Towards automatic visual obstacle avoidance, International Joint Conferences on Artificial Intelligence (1977) 584. [2] C. Harris, M. Stephens, A combined corner and edge detector, Avery Vision Conference (1988) [3] J. Canny, A computational approach to edge detection, IEEE Transactions Pattern Analysis and Machine Intelligence 8 (6) (1986) [4] D. Lowe, Object recognition from local scale invariant feature, International Conference on Computer Vision (1999) [5] W. Bian, D. Tao, Manifold regularization for SIR with rate root-n convergence, Advances in Neural Information Processing Systems (2009) 1 8. [6] S. Si, D. Tao, B. Geng, Bregman divergence based regularization for transfer subspace learning, IEEE Transactions on Knowledge and Data Engineering (2009). [7] T. Ojala, M. Pietikäinen, T. Mäenpää, Multiresolution gray-scale and rotation invariant teture classification with local binary patterns, IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (7) (2002) [8] M. Heikkilä, M. Pietikäinen, A teture-based method for modelling the background and detecting moving objects, IEEE Transactions on Pattern Analysis and Machine Intelligence 28 (4) (2006) [9] T. Ahonen, A. Hadid, M. Pietikäinen, Face description with local binary patterns: application to face recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 28 (12) (2006) [10] P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs. Fisherfaces: recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence 19 (7) (1997) [11] J. Li, D. Tao, Simple eponential family PCA, in: Proccedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), Journal of Machine Learning Research (JMLR) Workshop and Conference Proceedings, vol. 9, 2010, pp [12] D. Tao, X. Li, X. Wu, S.J. Maybank, Geometric mean for subspace selection, IEEE Transactionson PatternAnalysis andmachine Intelligence 31 (2) (2009) [13] C. Liu, H. Wechsler, Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition, IEEE Transaction on Image Processing 11 (4) (2002) [14] Hanqing Lu, Songde Ma Lu, Improving kernel Fisher discriminant analysis for face recognition, IEEE Transactions Circuits System Video Technology 14 (1) (2004) [15] C. Liu, Capitalize on dimensionality increasing techniques for improving face recognition grand challenge performance, IEEE Transactions on Pattern Analysis and Machine Intelligence 28 (5) (2006) [16] M. Pontil, A. Verri, Support vector machine for 3D object recognition, IEEE Transactions onpattern Analysis and MachineIntelligence 20 (6) (1996) [17] A. Barla1, F. Odone, A. Verri, Histogram intersection kernel for image classification, International Conference on Image Processing (2003) [18] S. Belongie, C. Fowlkes, F.N. Chung, J Malik, Spectral partitioning with indefinite kernels using the nystorm etensions, Europe Conference on Computer Vision (2002) [19] C. Wallraven, B. Caputo, Arnulf B.A. Graf, International Conference on Computer Vision (2003) [20] B. Zhang, S. Shan, X. Chen, W. Gao, Histogram of Gabor phase patterns: a novel object representation for face recognition, IEEE Transactions on Image Processing 16 (1) (2007) [21] Y. Su, S. Shan, X. Chen, W. Gao, Hierarchical ensemble of Gabor fisher classifier for face recognition, Proceedings of International Conference on Automatic Face and Gesture Recognition (2006) [22] D. Zhang, A. Wai-Kin Kong, J. You, M. Wong, Online palmprint identification, IEEE Transactions on Pattern Analysis and Machine Intelligence 25 (9) (2003) [23] P.J. Philips, P.J. Flynn, T. Scruggs, K.W. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, W. Worek, Overview of the face recognition grand challenges, International Conference on Computer Vision (2005) [24] W. Hwang, G. Park, J. Lee, Multiple face model of hybrid Fourier feature for large face image set, IEEE Conference on Computer Vision and Pattern Recognition (2006) [25] J. Zhao, H. Wang, H. Ren, S.C. Kee, LBP discriminant analysis for face verification, IEEE Conference on Computer Vision and Pattern Recognition (2005) [26] B. Zhang, Y. Qiao, Face recognition based on gradient Gabor feature and efficient kernel fisher analysis, Neural Computing & Applications (2010). Baochang Zhang received the B.S., M.S., and Ph.D. degrees in computer science from Harbin Institute of Technology, China, in 1999, 2001 and 2006, respectively. From 2006 to 2008, he was a research fellow with The Chinese University of Hong Kong and Griffith University, Australia. Currently, he is a lecturer with Beihang University, China. His research interests include pattern recognition, machine learning, face recognition, and wavelets. Yongsheng Gao received the B.Sc. and M.Sc. degrees in Electronic Engineering from Zhejiang University, China, in 1985 and 1988, respectively, and the Ph.D. degree in Computer Engineering from Nanyang Technological University, Singapore. Currently, he is an Associate Professor with the Griffith School of Engineering, Griffith University, Australia. He is also with National ICT Australia, Queensland Research Laboratory leading the Biosecurity group. His research interests include face recognition, biometrics, biosecurity, image retrieval, computer vision, and pattern recognition. He is a senior member of the IEEE. Hong Zheng received her Ph.D. degree in Harbin Institute of Technology, China, Currently, she is Professor with Beihang University. Her research interests include Hardware Design, Embedded System, pattern recognition, computer vision.