Face Recognition using Several Levels of Features Fusion

Transcription

1 Face Recognition using Several Levels of Features Fusion Elizabeth García-Rios, Gualberto Aguilar-Torres, Enrique Escamilla-Hernandez, Omar Jacobo-Sanchez 2, ariko Nakano-iyatake, Hector Perez-eana echanical and Electrical Engineering School National Polytechnic Institute Av. Santa Ana #000 Col. San Francisco Culhuacán, C.P EXICO, D.F. 2 Autonomous University of Hidalgo State, CITIS. Carretera Pachuca-Tulancingo Km. 4.5, CP , ineral de la Reforma, Hgo. egarciar009@alumno.ipn.mx; hperezm@ipn.mx Abstract: - This paper presents a face recognition system using several fusion levels of face features obtained from stereo images. These levels are; sensor level fusion, feature level fusion and decision level fusion, using for each level of fusion the Eigenfaces and Gabor Filter image face feature extraction and a backpropagation neural network as classifier.. At each level of fusion it is possible to evaluate the performance of face recognition marking the highest identification and verification rates. Evaluation results show that the feature level fusion that consists of two images with different angles of face provides the best results, because with this we get more coefficients or information to build a new template of the face. This allows achieving a higher the recognition than that obtained when only one picture with a single angle is used. Key-Words: - Face recognition; Levels of Fusion; Eigenfaces; Gabor Filter; Stereo Image; Neural Network. Introduction The face recognition system is an authentication and identification people method that focuses on specific face features. ost face recognition systems uses a two-dimensional face map. One of the major difficulties that appear in these systems stems the fact that the face is a three dimensional object and almost all cases the face recognition systems are based on two dimensions images, which means that they could be cheated by placing a photo of another person in front of a camera. To avoid this problem the proposed system uses stereo face images that allow doing the fusion of this information, improving in such way the precision of a face recognition system. The proposed face recognition system receives stereo images (left and right images) which are subjected to several different processing methods, implementing three levels of fusion, which are: sensor level fusion, feature level fusion and decision level fusion. The usage of these fusion levels is with both components of a stereo image of an individual form, combining them to form a 3- dimensions model of the face. The experimental results show that a multibiometric system reduces some of limitations present in most unimodal biometric system by consolidating the evidence presented by multiple biometric sources. Analyzing several fusion levels for parameter evaluation, the identification and verification system is be able to significantly improve the recognition rate using the feature level fusion. Despite all evaluates fusion methods improves the recognitions rates, the arithmetic mean of two features vectors obtained from the face image provides the highest match score. This fusion of evidence, enhance the matching accuracy of a recognition system. Thus, working with a multibiometric system increases the system accuracy and reduces the probability of spoofing identity. The rest of the paper is organized as follows: Section 2 provides a description of the face recognition system, which is applied on different forms using several fusion levels such as; sensor level fusion, feature level fusion and decision level fusion, which are described in the section 2., that also defines some other terms used in the system as features extraction methods and learning algorithms for input data. The results of each fusion level are show in Section 3 and finally the conclusions are show in Section 4. 2 Problem Formulation A biometric system is essentially a pattern recognition system that acquires biometric data from the person to be analyzed, extracts a relevant feature set from the data, compares this feature set with the feature set stored in the database, and executes an ISBN:

2 action based on the result of such comparison []. Those systems carry out the person identification or identity verification of the people under analysis based on their physical features (face, fingerprint, iris, voice, etc.) or behaviour (signature, dynamic typing, way of walking, etc.). Among them the facial recognition systems are computer-based security systems that are able to automatically detect and identify human faces. As first step for face recognition system, the system carries out the acquisition of stereo images, by using a camera with two lenses. Next the face image will be segmented to avoid useless information that might affect the operation of the feature extraction module, allowing it to obtain the relevant information of face image. Then, this information is compared with the computed information stored in the database in order to find the best match to take the decision regarding the identified person. The Fusion information is a key part in any multibiometric system and according to the different modules of proposed the face recognition system; in which fusion can be done at distinct levels such as: sensor level, feature level and decision level. These levels can be broadly classified as: fusion prior to matching and fusion after matching [2, 3]. 2. Fusion Levels. 2.. Sensor fusion level (Sensor-LF). Sensor-fusion level refers to the consolidation of raw data obtained using multiple sensors or multiple snapshots of a biometric feature using a single sensor. For the proposed face recognition system, the sensor level fusion is considered as the fusion of left and right images of the face under analysis such that together form the stereo image, as shown in figure. Figure Block diagram of sensor-lf. The input images are feed to a preprocessing stage before being introduced to the system, which find the middle point in each image and cutting only half of face where the more significant information is found, considering that the half left part of the face image is complementary to half right part of such image. Figure 2 Fusion image Feature fusion level (Feature-LF). Feature fusion level is consolidates the evidence presented by two biometric characteristic sets of the same individual. Whether two feature sets are originated from the same feature extraction algorithm using different images face of the same person, the feature fusion level can be used for the template update. The template in the database can then be updated based on the evidence presented by the current feature set in order to reflect permanent changes in a person's biometric. Gabor function and Eigenfaces, are methods widely used to extract the most relevant information from a facial image, allowing to handle only the main parameters that make up each face. Then once the feature vector of each face is obtained, this is introduced to the matching module using a backpropagation Neural Network, which decides if the identified person corresponds or not to the input image. This procedure which is often used when a single biometric information source available, must be modified in order that the recognition system must be able handle two sources. Thus one suitable approach is to consider the above process twice, since it has to process two separate images with different angles for each face. But within this level of fusion, one process for each image is carried out, adding a pre-processing before feeding them into the matching module, as shown in figure 3. The pre-processing is divided into two tasks: the first one estimate the arithmetic mean of feature vectors of both images and the second concatenates the ISBN:

3 feature vectors, giving at the end a single vector to characterize the face image, this new template will vector was used as the template for evaluating the system as shown in figure 5. be introduced to the matching module. Figure 3 Block diagram of feature-lf. The first method used for the features fusion, is the calculation of the arithmetic mean between the eigenvectors, corresponding to the stereo image of the face. This operation is performed with each of the values contained in the vector, consecutively, taking the first element of the features vector of the left part of the image face together with the first element of the vector of the right part of the image face and calculating the arithmetic mean between both values. This operation is repeated until all elements of both feature vectors have been processed. Thus a new features vector is obtained, which is used as the new template new to evaluate the system as shown in figure 4. Figure 5 Concatenation of characteristic vectors 2..3 Decision fusion level (Decision-LF). This fusion is operated in the decision stage. Here every unimodal system takes an acceptation or rejection decision about a person asking for access to the global system. These binary answers are processed by a supervisor, which has the results of all individual systems and takes the final decision. The simplest method for combining the decisions outputs of the different matchers the use of the OR operation, which means the identification is consider positive if any the output at least of individual system is positive; or the AND operation in which the identification decision is positive only if the decision of all individual systems is positive. The logical operation used in this system is the "AND" operation. Thus the system output is a "match" only when all the biometric matchers agree that the input sample matches with the stored template. In this case the features of each image face are, separately, extracted performing the recognition process, since the face image is captured until the decision matching module or classifier for each side of the face images is taken. Figure 4 Arithmetic mean features vectors. The feature vectors obtained for each face image have the same length size, 70 values. A second method for this fusion level can be applied which consists in the concatenation the eigenvectors corresponding to the integration of the stereo face image, (left and right parts, respectively). This means that it is the union of all the values contained in both vectors, yielding a resultant vector with twice length, i.e. 40 values, where the resulting Figure 6 Block diagram of decision-lf. ISBN:

4 Next the fusion data in the decision module is done, comparing the results of each image satisfying the rule AND, where only if both results are considered as a person recognized he/she is finally is accepted by the system otherwise it will be rejected as shown in figure Eigenfaces. The Principal Components Analysis (PCA) is a widely used method for data dimensionality reduction, which has also been used in the computer vision area such as face and object recognition, resulting in the so called Eigenfaces method [4]. In the PCA, the recognition is performed by projecting a new image into the subspace spanned by the eigenfaces (face space) and then classifying the face by comparing its position in the face space with the positions of known individuals. Each individual, therefore, would be characterized by a small set of feature weights needed to describe and reconstruct them. That is an extremely compact representation when compared with the images themselves [5]. The eigenfaces method involves the following operations: Firstly Assume the width and height of the image be equal to n and m pixels respectively, such that the the size of the transformed vector of this image is d=n*m. Next, given pre-processed face images as training data, we covert these images into corresponding column vectors I={I n (x,y), n=,2,,}. Subsequently the average each training face is estimated as follows I n n And using it the mean-adjusted image is defined as () I n, n,2,.., (2) Let C denote the covariance given by C n T AA T (3) where. Thus the principal components are then the eigenvectors of C, which is a dxd matrix, that contains d eigenvectors V, V 2,,V d and d eigenvalues ʎ,ʎ 2,,ʎ d. However, it is time-consuming to determine d eigenvalues and eigenvectors. Therefore, it is necessary to reduce the computational complexity. Thus according to SVD (Singular Value Decomposition), AA T and A T A have the same eigenvalue ʎ d. As result, instead of directly computing eigenvectors u i of matrix AA T, eigenvectors v i of matrix A T A is computed. Eigenvectors u i of matrix AA T can be defined by u i Avi (4) i Gabor filters. Gabor filters are bandpass filters which are used in image processing for feature extraction. The impulse response of these filters is created by multiplying a Gaussian envelope function with a complex oscillation. By extending these functions to two dimensions it is possible to create filters which are selective for orientation [8]. Under certain conditions the phase response of Gabor filters is approximately linear. The Gabor function has the following general form: Gx ( x, y) 2 x y 2 2 x y exp x y exp( j2 u x) o (5) where u o denotes the radial frequency of the Gabor function. The space constants and define the Gaussian envelope along the X - and Y-axes. 2.3 Neural network Back propagation. The back-propagation neural network is a generalization of the least squares algorithm. This algorithm is a multilayer network which performs the task of updating its weights minimizing the mean square error. The back-propagation network works in a supervised form, where the set value of weights is updated using on the generated error. This technique is widely used because it allows an optimization which defines the gradient of the error and minimizes it with respect to the parameters of the neural network [6]. The learning algorithm using back-propagation consists of: Start with any synaptic weights (usually random). Enter an input layer randomly chosen from the input data to be used for training. Let the network generates an output data ISBN:

5 vector (forward propagation). Compare the output generated by the network with the desired output. The obtained difference between the generated and desired output (called error) is used to adjust the synaptic weights of the neurons of the output layer. The error is propagated back (backpropagation), to the previous layer of neurons, and is used to adjust the synaptic weights in this layer. Continue backward propagating the error and adjusting the weights until it reaches the input layer. This process is repeated with different training data. 3 Problem Solution The testing the fusion level methods was carried out using stereo face images, which were captured with a digital camera containing two lenses. Figure 7 show a set of the images. from 60 different people, that by each person we took 5 photographs and considering that for every capture of image is a pair of images extracted, it has a total of 800 images of faces. In the matching module was used a Backpropagation Neural Network trained using 200 images, taking the rest of the images for the tests of face identification and verification tasks. Thus, evaluating the behavior of the recognition system in each of the levels of fusion were obtained different percentages of identification and verification, by making a selection of the level of fusion with the best face recognition performance. Among the tests that were performed the identification was done in two ways: a test with the Eigenfaces algorithm and the second test were applied Gabor Filter. TABLE Percentages of identification in the face recognition system Level fusion Eigenfaces Gabor function Sensor level 96.22% 95.00% Feature level 96.66% 97.88% Arithmetic mean Feature level 9.77% 94.66% Concatenation Decision level 9.55% 93.77% Only right part 94.77% 94.33% Only left part 94.33% 93.77% TABLA 3 Percentages of verification in the face recognition system using a backpropagation ANN and eigenfaces as feature extraction method. Level fusion FAR FRR Verification Sensor level 0.% 0.0% 99.88% Feature level. Arithmetic mean 0.% 0.0% 99.88% Fusion level Concatenation % Decision level 0.0% 0.0% 99.89% Figure 7 Samples of the face images data based used for evaluating the proposed algorithm. The proposed system was evaluated in a controlled environment, since all images were taken with a uniform background, with approximately the same distance between the face and camera, the same type of lighting, without rotation and with little variation of gestures. The images taken were 4 Conclusions This paper presents an evaluation of the different fusion levels using stereo face images. The purpose of using different levels of fusion is make a multibiometric system making it more robust since these fusion levels evaluate more information from more than one characteristic of the face and provide a better identification of the person. The three levels ISBN:

6 of fusion that were treated, in general, all show good identification performance, however, the behavior of each is different. The method with best performance is feature level fusion by calculating the arithmetic mean of the characteristic vectors of the face, because at joining the main features of the face is shown that the greater the number of obtained features the system is more likely to identify a person, while using less features the system is more susceptible to several forms of cheating. References: [] Jain, Anil K., Ross, Arun A., Nandakumar, Karthik, Introduction Biometrics, Springer Science+Business edia, LLC 20, USA. [2] Arun A. Ross, Karthik Nandakumar, and Anil K. Jain, Handbook ultibiometrics, Springer Science+Business edia, USA. LLC 2006 [3] P. Buyssens,. Revenu, Fusion Levels of Visible and Infrared odalities for Face Recognition, Theory Applications and Systems (BTAS), pp. -6, 200. [4] Smith L. A Tutorial on Principal Analysis, [5]. Turk, A. Pentland, Eigenfaces for Recongnition, Journal of Cognitive Neuroscience, pp. 7-86, 99. [6] P. Ponce Cruz, Inteligencia Artificial con aplicaciones a la ingeniería, er edición, Alfaomega Grupo Editor, exico, Julio 200. [7] Pizer,.S., Adaptive Histogram Equalization and its Variations, Computer Vision, Graphics and Image processing, 39, pp ,987. [8] J.G., Daugman, Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters, J. Optical Society of America A, vol. 2, no. 7, pp , July 985. ISBN: