Decision Level Fusion of Face and Palmprint Images for User Identification

XI Biennial Conference of the International Biometric Society (Indian Region) on Computational Statistics and Bio-Sciences, March 8-9, 2012 83 Decision Level Fusion of Face and Palmprint Images for User Identification Savitri B. Patil and Sanjeevkumar M. Hatture Abstract--- Automatic person identification is an important task in our day-to-day life. The traditional method of establishing a person s identity include knowledge based like password or token based like ID cards, but representation of these identity can easily be lost, stolen or shared. Therefore they are not sufficient for identity verification. Biometric offers natural and reliable solution to the problem of identity determination by recognizing individuals by using certain physiological or behavioural traits associated with the persons. However a single biometric characteristic sometimes fails to be accurate enough for the identification. Thus the multimodal biometric system is used to overcome the limitations of unimodal biometric system. The proposed system is designed for authenticating a user based on two traits i.e. face and palmprint. Integrating the palmprint and face features increases robustness of the person authentication. The final decision is made by fusion at decision level architecture in which features vectors are created independently for query images and are then compared to the enrollment template, which are stored during database preparation. The main goal of this paper is to achieve good matching score. Keywords--- Biometrics, Multimodal, Face, Palmprint, Wavelets, Decision Level Fusion, Probabilistic Neural Network. I I. INTRODUCTION DENTITY management system is challenging task in providing authorised user with secure and easy access to information and services across a wide verity of applications. A reliable identity management is a critical component in several applications that provide services to only legitimately enrolled users. Some of applications include physical access control to secure facility, access to computer networks, performing remote financial transactions etc. The primary task in an identity management system is determination of individual s identity. This action may necessary for many reasons but in most applications, primary intention is to prevent imposters from accessing protected resources. The traditional method of establishing a person s identity include knowledge based like password or token based like ID cards, but these representations of the identity can easily be lost stolen or shared. Therefore they are not sufficient for identity verification. Biometric offers natural and reliable solution to the problem of identity determination by recognizing individuals by using certain physiological or behavioural traits associated with the persons. Biometrics is the science of identifying or verifying the identity of person based on physiological or behavioural characteristics. Physiological traits are related to physiology of the body and mainly include fingerprint, face, ear, iris, retina, hand, palm geometry. Behavioural traits related to the person such as signature, voice etc. However, a single biometric characteristic sometimes fails to be accurate enough for the identification because it may suffer a variety of problems such as noisy data, non-university, within-class variation, between-class similarity and spoof attacks [1]. Multibiometric systems can significantly improve the recognition performance in addition to improving population coverage, deterring spoof attacks, increasing the degrees of freedom, and reducing the failure-to-enroll rate. The key to successful multibiometric system is in an effective fusion scheme, which is necessary to combine the information presented by multiple domain experts. The goal of fusion is to determine the best set of experts in a given problem domain and devise an appropriate function that can optimally combine the decisions rendered by the individual experts. A decision made by a biometric system is either a genuine individual type of decision or an impostor type of decision [5]. The genuine distribution and the impostor distribution, which are used to establish the following two error rates. 1. False Acceptance Rate (FAR), which is defined as the probability of an impostor being accepted as a genuine individual. It is measured as the fraction of impostor score exceeding the predefined threshold. 2. False Rejection Rate (FRR), which is defined as the probability of a genuine individual being rejected as an impostor. It is measured as the fraction of genuine score below the predefined threshold. A FAR of zero means that no impostor is accepted as a genuine individual. Sometimes, another term, genuine acceptance rate (GAR) or correct identification rate (CIR), is used to measure the accuracy of a biometric system. In this paper, we propose a multimodal biometric identification method using face and palmprint based on fusion at decision Savitri B. Patil, Department of Computer Science and Engineering, Basaveshwar Engineering College, Bagalkot, VTU, Belgaum, Karnataka, India. E-mail: savithri3010@yahoo.co.in Sanjeevkumar M. Hatture, Department of Computer Science and Engineering, Basaveshwar Engineering College, Bagalkot, VTU, Belgaum, Karnataka, India. E-mail: smhatture@yahoo.com

XI Biennial Conference of the International Biometric Society (Indian Region) on Computational Statistics and Bio-Sciences, March 8-9, 2012 84 level. The main goal of proposed multimodal biometric system is to achieve high recognition performance and further the security of authentication. The rest of this paper is organized as follows: Related work in section II. The proposed approach in section III. The experiments results are reported to evaluate the performance of our proposed approach in Section IV. Finally, conclusions are given in Section V. II. RELATED WORK In 2005, a bimodal approach was described by Slobodan Ribaric, Ivan Fratric and Kristina Kis for palmprint recognition based on the principal lines, face recognition with eigenfaces, fusion of the unimodal results at the matching-score level [6]. The equal error rate of 0.74% and the minimum total error rate of 1.72% respectively were achieved with proposed approach. In the same year, Ajay Kumar, David Zhang was proposed a new method of personal authentication based on eigenface, PCA based face and palmprint identification system with a fusion method at the matching score level [2]. At a FAR of 1.13%, the multimodal systems obtained FRRs of 0.95%. In the year 2008, a Multimodal Biometrics Fusion Using Correlation Filter Bank was developed by Yan and Yu-Jin Zhang based on Class-dependence Feature Analysis (CFA) method [8]. The recognition rate of 100% was achieved using proposed method. In the same year, a feature fusion of palmprint and face was proposed by Yucheng Wang, Dongmei Sun, based on Kernel Fisher Discriminant Analysis (KFDA) which is Kernel Principal Component Analysis (KPCA) plus Linear Discriminant Analysis (LDA) [10]. The recognition rate of 99.9% was achieved using proposed method. In the same year, A Multi-modal Authentication Method was proposed by Yinghua Lu Yao Fu, Jinsong Li, Xiaolu Li, Jun Kong for a Gabor transform and Independent Component Analysis (ICA) based face and palmprint [9]. The recognition rate of 98.8% was achieved using the proposed approach. In 2009, a contactless biometric system was designed by Audrey Poinsot and Fan Yang, Michel Paindavoine, for a PCA based face and Gabor filter was used as palmprint feature extraction. High recognition performance was obtained by respecting embedded system context, with palmprint only and with fusion of palmprint and face: recognition rates of respectively 96.39% and 98.85% were achieved using only 2 samples per modality [3]. In the same year, a hierarchical multi scale Local Binary pattern (LBP) algorithm for face and palmprint recognition was proposed by Zhenhua Guo1, Lei Zhang, David Zhang and Xuanqin Mou. The recognition accuracy of 99.6% was achieved using the proposed approach [11]. III. THE PROPOSED APPROACH The flowchart of proposed multimodal biometric identification system is shown in Figure 1. The proposed system is divided into three parts: face identification, palmprint identification and multimodality identification using face and palmrpint based on fusion at decision level. In our work, the biometric images come from the AR face database and the PolyUpalmprint database. The face features namely width of head, forehead, jaw, chin and height of face and total darkness in the face as well as distance between head to forehead and discrete wavelet transformation is employed for feature extraction of palmprint. After face and palmprint identification, the two scores are combined to a score for finally identification using probabilistic neural network. Figure 1: The Flowchart of Proposed Approach A. Face Feature Extraction In this proposed system face features namely width of head, forehead, jaw, chin and height of face and total darkness in the face as well as distance between head to forehead are extracted. A separate database is created to store the extracted features. From observation it is found that the Head Top as approximately at row position 20, for finding this width we have to count nonwhite pixels (i.e.<250), forehead is approximately at row position 50, for finding this width we have to count gray pixels (i.e. >80), face is approximately at row position 100, for finding this width we have to count non-white pixels (i.e. <250), jaw is

XI Biennial Conference of the International Biometric Society (Indian Region) on Computational Statistics and Bio-Sciences, March 8-9, 2012 85 approximately at row position 130, for finding this width we have to count non-white pixels (i.e. <250), chin is approximately at row position 150, for finding this width we have to count non-white pixels (i.e. <250). Middle column considers to find face height (i.e. 62), for finding this height we have to count Gray pixels (i.e. >=70 &<250).To find darkness in face we calculate the black shade pixel (i.e. <=30). The features of face are shown in Figure 2. Face Height Head(Row 20) Fore Head (Row 50) Width Face Width Jaw Throat at chinlevel Figure 2: Face Features B. Palmprint Feature Extraction Palmprint is rich in texture information. In this paper, 2 dimensional discrete wavelet transformation is used for extracting the features of palmprint. DWT (Discrete Wavelet Transform) decompose the image in to four sub images when two level of decomposition is used. One of these sub-images is smoothed version of the original image corresponding to the low pass information and the other three ones are high pass information that represents the horizontal, vertical and diagonal edges of the image respectively. When two images are similar, their difference would be existed in high frequency information. A DWT with N decomposition level has 3N+1 frequency bands with 3N high frequency bands. Wavelet analysis is a time-frequency localization method whose window size is fixed but shape can optionally change. Here we use Daubechies wavelet and do the wavelet decomposition. After that we only choose low-frequency sub-band part as feature for reducing dimension of original images. After wavelet transformation the image becomes 1/4 of the original image which is used to extract the features. C. Fusion The proposed biometric authentication system is based on decision level fusion model where the integration is performed on the individual matching scores to generate a composite decision score. These decision scores are used to classify the user into genuine or imposter class. The acquired grey-level images from the palmprint and face are presented to the system. Each of the acquired images are used to extract the features. Then the database is created to store the features. The stored features from database are presented to a trained Probabilistic neural network classifier. Then a reliable decision score is generated from the output of the trained neural network which is used to assign genuine or imposter class label to the user. Two distinct experiments, each for palmprint and face, were performed to observe the performance of proposed fusion strategy using test user identity. D. Recognition In this paper, Probabilistic Neural Network (PNN) [7, 12] is used to identify the weighted feature. A Probabilistic Neural Network is predominantly Classifier. It maps the any input into number of classifiers. The PNN have gained interest because they offer a way to interpret the network s structure in the form of a probability density function and their performance is often superior to other state-of-art classifier. In addition, most training methods for PNNs are easy to use. IV. EXPERIMENTAL RESULTS In this paper two databases are employed to test our proposed approach in our experiments, and each contains face images and palm images. The face images are taken from the AR database [13]. The palmprint images are taken from partial Hong Kong Polytechnic University [14]. Database 1: The AR database contains over 4000 facial images. In this paper 1000 face images of 50 users randomly selected for testing our proposed multimodal biometric identification approach. Among the 20 images of each user, the first twelve images of which are used as the training samples and the others as the testing samples. The size of all the original face images is 125X165 with 256 gray levels. Figure 3 shows sample images of users from AR database. Figure 3: Samples of users from AR Database Database 2: The Poly_UPalmprint Database contains 7752 gray scale images. In this paper 1000 palmprint images from 50

XI Biennial Conference of the International Biometric Society (Indian Region) on Computational Statistics and Bio-Sciences, March 8-9, 2012 86 users are randomly selected for testing our proposed multimodal biometric identification approach. Among the 20 images of each user, the first twelve images of which are used as the training samples and the others as the testing samples. The size of all the original palmprint images is 384X284 pixels. Figure 4 shows sample palmprint images of users. Figure 4: Sample Palmprint Images of Users from Poly_Udatabase In this, we experimentally evaluate the effectiveness and feasibility of our proposed multimodality approach. First the images are acquired and presented to the system to extract the features. Then the extracted features are stored in the database. The stored features from database are then presented to a trained Probabilistic neural network classifier. Thus a reliable decision score is generated from the output of the trained neural network which is used to assign genuine or imposter class label to the user. The comparison of both unimodal systems (face and palmprint modality) and the multimodality system is shown in TABLE 1. From the results it is clear that the identification accuracy based on the palmprint outperforms the face identification whichever distance criterion is adopted. It can also be seen that the fusion of face and palmprint improves the accuracy and the CIR can reach 99%. Table 1: Recognition Rates (%) under Different Modality Users Face CIR (%) Palmprint CIR (%) Face+Palmprint CIR(%) 10 98.7 100 100 20 97.5 98.7 100 30 94.5 98.3 100 40 92.8 98.1 99 50 92.2 98.0 99 100 80 60 40 20 Face Palmprint Face+Palmprint 0 10 20 30 40 50 Figure 5: Recognition Rates (%) under Different Modality We can see from the graph of Figure 5 that, the recognition rate of fusion is better than palmprint and face recognition whereas face recognition rate is the worst. Because of the expression, illumination conditions, and occlusions face recognition has some difficulties. So we can conclude the multimodal method is better in recognition that it can not only obtain better recognition rate but also improve system security. V. CONCLUSION Automatic person identification is an important task in our day-to-day life. The traditional method of establishing a person s identity include knowledge based like password or token based like ID cards, but representation of these identity can easily be

XI Biennial Conference of the International Biometric Society (Indian Region) on Computational Statistics and Bio-Sciences, March 8-9, 2012 87 lost, stolen or shared. Therefore they are not sufficient for identity verification. Biometric offers natural and reliable solution to the problem of identity determination by recognizing individuals by using certain physiological or behavioural traits associated with the persons. But the unimodal biometric system fails in case of biometric data for particular trait. Thus multimodal biometric system is developed using the two traits (face &palmprint). The objective of this work is to integrate face and palmprint features at decision level and achieve higher performance that may not be possible with single biometric system alone. Features of face and palmprint are extracted. PNN (Probabilistic neural network) is used as classifier to classify the fused features. Experimental results show that the palmprint modality is a very efficient biometric which allows us to realize high recognition performance using discrete wavelet transformation. Face recognition is less efficient in the same conditions because face image can change widely depending on external elements such as light, noise, facial expressions, hairdo, posture, and so on. In future this approach can be extended to other modalities to achieve high recognition performance and further the security of authentication. The use of other methods such as textural properties, Independent component analysis, Gabor transform may result in further improvement of the system accuracy. REFERENCES [1] A.K. Jain, A. Ross, Multibiometric systems, Commun. ACM 47 (1), 2004, pp. 34-40. [2] A. Kumar and D. Zhang, 2005 Integrating palmprint with face for user authentication, Proc. Multi Modal User Authentication Workshop, pp. 107-112, Santa Barbara, CA, USA. [3] Audrey Poinsot and Fan Yang, Michel Paindavoine, 2009, Small Sample Biometric Recognition Based on Palmprint and Face Fusion, Fourth International Multi-Conference on Computing in the Global Information Technology, pp 118-122. [4] G.S. Gill and J. S. Sohal, Battlefield Decision Making: A Neural Network Approach Journal of Theoretical and Applied Information Technology 2005-2008 JATIT. All rights reserved. [5] R. Frischholz and U. Dieckmann, BiolD: A multimodal biometric identification system, Computer, vol.33, no.2, pp.64-68, Feb, 2000. [6] Slobodan Ribaric, Ivan Fratric and Kristina Kis, 2005, A Biometric Verification System Based on the Fusion of Palmprint and Face Features, In Proc. 4th Int l Symposium on Image and Signal Processing and Analysis, ISPA 2005, pp. 12 17. [7] S.N. Sivandm, S. Sumathi, S.N.Deepa, Introduction to Neural Networks Using Matlab 6.0, Tata mcgraw-hill Publishing Company Limited, Copyright 2006. [8] Yan Yan and Yu-Jin Zhang, 2008, Multimodal Biometrics Fusion Using Correlation Filter Bank, International Conference on Pattern Recognition, pp. 1 4. [9] Yinghua Lu Yao Fu, Jinsong Li, Xiaolu Li, Jun Kong, 2008, A Multi-modal Authentication Method Based on Human Face and Palmprint, Second International Conference on Future Generation Communication and Networking. [10] Yucheng Wang, Dongmei Sun, 2008, Feature Fusion of Palmprint and Face Based on KFDA, 9th International Conference on Signal Processing, pp. 2092 2095. [11] Zhenhua Guo1, Lei Zhang, David Zhang and XuanqinMou, 2009, Hierarchical MultiscaleLBP for face and palmprint recognition, 17 th IEEE International Conference on Image Processing (ICIP), pp. 4521 4524 [12] Neural Network Matlab Toolbox 2008 [13] The AR Face Database, http://rvl1.ecn.purdue.edu/~aleix/aleix_face_db.html [14] The PolyUPalmprint Database, http://www.comp.polyu.edu.hk/~biometrics.