Correlation filters for facial recognition login access control

Size: px

Start display at page:

Download "Correlation filters for facial recognition login access control"

Bernard Hall
5 years ago
Views:

1 Correlation filters for facial recognition login access control Daniel E. Riedel, Wanquan Liu and Ronny Tjahyadi Department of Computing, Curtin University of Technology, GPO Box U1987 Perth, Western Australia 6845 Abstract. Correlation filtering has recently been reintroduced to the facial recognition domain with promising recognition rates being seen over a variety of different facial databases. In this paper we utilise the minimum average correlation energy (MACE) and unconstrained minimum average correlation energy (UMACE) filters in conjunction with two correlation plane performance measures, max peak value and peak-tosidelobe ratio, to determine the effectiveness of this approach in relation to facial recognition login access control. A new technique for determining performance measure thresholds with the individual filters derived from the AMP and Biodm facial databases, was successfully developed producing high recall rates (87-94%) with 100% precision. A comparison of the precision and recall statistics obtained from the two different correlation plane measures, further demonstrated that max peak value is the better performance measure for use with MACE and UMACE filters for facial recognition login access control. 1 Introduction Facial recognition is a popular biometric approach to login access control due to the uniqueness of the human face, ease of collecting facial images and good acceptability as a result of the techniques unobtrusiveness [1]. Typically, most current operating systems use password verification for the establishment of a users identity with the requested service being access to the system. However, these systems are vulnerable to password cracking, password sniffing and acquisition of passwords through simplistic means. Facial recognition attempts to minimise these inherent vulnerabilities by controlling the port of entry into a system via utilising ones own facial signature. In comparison to other applications of facial recognition, login access control requires the system to exhibit and maintain 100% precision for all users. This is necessary to prevent unauthorised users from being incorrectly accessed to a system and secondly, to prevent authorised persons from being incorrectly identified as other valid users. The technique of correlation filtering was initially applied in pattern recognition tasks, such as automatic target recognition, through the use of matched spatial filters (MSF) generated from a single training image [2]. Unfortunately, this technique demonstrated distortion variance, poor generalisation and poor

2 localisation properties due to the use of a single training image and the broad correlation peak generated. To address these short-comings, the synthetic discriminant function (SDF) was introduced that linearly combined a set of training images into the one filter which further allowed one to constrain the filter output at the origin of the correlation plane [3]. By incorporating training images that best represent the expected distortions of an object, the SDF filter provided some degree of distortion invariance, yet like MSFs they did exhibit broad correlation peaks, making localisation difficult. To maximise peak sharpness for better object localisation and detection, the minimum average correlation energy (MACE) [3] and unconstrained MACE (UMACE) filters [4] were developed. Prior research has demonstrated that MACE and UMACE correlation filters are effective at facial recognition, producing identification rates of % and verification rates of % with the AMP and PIE facial databases [5 7, 10]. Furthermore, MACE and UMACE filters have been shown to exhibit limited built-in tolerance to illumination variation [6], which is attractive for multi-environment computing login access control. Facial recognition with correlation filters is performed by correlating a lexicographically ordered test image (transformed into the frequency domain via a Discrete Fast Fourier transform) with a synthesized, lexicographically ordered filter also in the frequency domain. The output correlation is subjected to an Inverse Fast Fourier transform and reordered into the dimensions of the original training image, prior to being phase shifted to the center of the frequency square. The resulting correlation plane is then quantified using performance measures, typically the peak-to-sidelobe (PSR) ratio. This paper investigates the use of an alternate correlation plane quantification metric (max peak value) to determine its effect on precision and recall rates in comparison to the standard PSR measure. In addition, an empirical method for deriving filter classification thresholds, is also introduced to avoid current ad hoc methods of threshold selection. Threshold selection is an important issue in login access control in order to maximise recall, whilst maintaining 100% precision. 1.1 Minimum average correlation energy filters MACE filters function to increase the peak sharpness by minimising the average correlation energy over a set of training images, whilst constraining the correlation peak height at the origin to a user-defined value. This in turn produces sharp peaks at the origin of the correlation plane, whilst producing values close to zero over the rest of the plane. The optimal solution to the MACE filter H is found using Lagrange multipliers in the frequency domain and is given by Equation (1). H = D 1 X(X + D 1 X) 1 u (1) D is a diagonal matrix of size d d, (d is the number of pixels in the image) containing the average correlation energies of the training images across its diagonals. X is a matrix of size N d where N is the number of training images

3 and + is the complex conjugate. The columns of the matrix X represent the Discrete Fourier coefficients for a particular training image X n. The column vector u of size N contains the correlation peak constraint values for a series of training images. These values are normally set to 1.0 for images of the same class [3, 7]. The UMACE filter like the MACE filter minimizes the average correlation energy over a set of training images, but does so without constraint (u), thereby maximising the peak height at the origin of the correlation plane. The UMACE filter expression H is given by Equation (2). H = D 1 X (2) D is a diagonal matrix containing the average correlation energy of the training images. X is a column vector of size d, containing the average Fourier coefficients of the training images [4]. Computationally, UMACE filters are more attractive than their MACE counterparts as they require only the inversion of a single diagonal matrix. Furthermore, UMACE filters have been shown to perform similar to MACE filters in facial recognition [6]. 1.2 Correlation plane performance measures Correlation plane performance measures are used to quantify the correlation plane generated by the correlation of a filter and a test image. In the context of facial recognition, the correlation plane is gauged with a performance metric in conjunction with a threshold, in order to classify a test image as belonging to a true or false class. Previous research with correlation filters and facial recognition has used the peak-to-sidelobe ratio (PSR) [5 7, 10] as a benchmark for correlation plane quantification. PSR can be calculated as per equation (3). P SR = peak mean σ peak is the maximum peak value in the correlation plane, mean is the average of the sidelobe region surrounding the peak (20 20 pixels for a pixel image, with a 5 5 excluded zone around the peak) and σ is the standard deviation of the sidelobe region values. More details on this technique can be found in [7] and [8]. In this study, we introduce an alternate simplistic measure, using just the maximum peak value, for measuring correlation planes in the context of facial recognition login access control. The maximum peak value is taken as the maximum correlation peak value over a correlation plane and is equivalent to the peak parameter in Equation 3. This performance metric measures only the strength of a match without taking into account sidelobe regions and is therefore more susceptible to misclassifications with correlations that generate large sidelobes. However, in relation to true class matches, it is believed that the max peak value metric should yield an equal or higher percentage of true class matches in comparison to PSR, as good correlations produce strong correlation peaks irrespective of the size of the sidelobe regions. (3)

4 1.3 Facial databases Experimentation was carried out using the Advanced Multimedia Processing (AMP) facial expression [7] and the Biodm facial databases 1. As much of the correlation filtering research in facial recognition has utilised the AMP facial database (13 subjects, 75 images per subject), it serves as a good experimental benchmark. Comparative studies were also carried out with the Biodm facial database. Images of this database were captured in the Department of Computing, Curtin University of Technology, with a real-time Linux biometric login system, incorporating facial recognition with a Logitec Quickcam Express Webcam. Histogram equalization was performed on all captured images, which were obtained over several sessions. The Biodm database comprises 11 subjects with 14 images each and of size pixels in PGM greyscale format. 1.4 Threshold derivation To determine the effectiveness of the MACE and UMACE filters for facial recognition login access control, individual classification thresholds were required for each filter synthesized. Initially, a global threshold was utilised across the filters of the same facial database but poor recognition performance was observed with both filter types. Consequently, individual upper and lower thresholds were derived using the training set images of the database for each filter, according to Fig. 1. The purpose of this approach was to find a threshold range in which to better select a classification threshold. With the individual upper and lower thresholds per filter, ratios of the difference between the upper and lower thresholds were used globally to determine an optimal pseudo-global threshold θ global to be set for all filters according to Fig. 2. As the θ global value uses the training images of a facial database to determine the maximum TP with zero FP, it is suitable for login access control, which requires a system to obtain 100% precision with a high recall rate for good usability. 2 Testing Methodology Prior to experimentation, the AMP and Biodm facial database images were segmented into two data sets: the training set and the testing set. MACE and UMACE filters were synthesized for the individuals comprising the uniformly distributed training set images, whilst the testing set was used to measure the precision and recall statistics of the approach over a range of pseudo-global thresholds. The testing set also contained 2 persons who were not part of the training set and thus should not be recognised. This particular approach of using testing images of persons not in the training set allows for a more accurate interpretation of the recognition results, by taking into account the ability of the system to reject external persons. 1 riedelde/biodm.html

5 1. Filter H i is synthesized using training images X i (of size m), where X i X, X={X 1X 2...X n} and n is the number of individuals comprising the training set. 2. True class correlations α i are performed in the frequency domain with filter H i and the training images X i. The corresponding correlation planes are quantified (β i) using an appropriate correlation plane measure and the m values in β i are then sorted in increasing order. The θ upper threshold is then set to the minimum of the of the β i values. ( is the complex conjugate) FOR j 1 TO m DO α j i H i Xj i β j i CORRELAT ION MEASURE(αj i ) ω 1 SORT (β i) θ upper β ω i 3. False class correlations γ i are performed with the filter H i and the training images Y i, where Y i X and Y i X i. The corresponding correlation planes are quantified (δ i) using the same correlation plane measure as in Step (2) and the θ lower threshold is set to the maximum of the δ i values. FOR j 1 TO m (n 1) DO γ j i H i Y j i δ j i CORRELAT ION MEASURE(γj i ) θ lower MAX(δ i) 4. IF θ lower θ upper AND ω m THEN ω = ω + 1 θ upper = β ω i REPEAT Step (4) ELSE OUTPUT θ lower and θ upper STOP Fig. 1. Upper (θ upper) and Lower (θ lower ) threshold derivation algorithm. To quantify the recognition performance of the filters with the testing sets, precision and recall statistics were applied to the performance values. Precision allows one to measure the ability of the technique to correctly classify, whilst recall measures the completeness of a techniques classification i.e. the proportion of the true class test cases that were identified [9]. 3 Experimental Results The training set for the AMP database consisted of the first 11 of 13 individuals comprising the database, with 5 training images per person (images 0,14,29,44 and 59). The corresponding testing set consisted of 40 uniformly selected images not present in the training set. Training sets for the Biodm facial database utilised the first 9 of the 11 subjects in the database, with 7 images selected per subject (images 1,3,5,7,9,11,13). The corresponding testing sets contained the remaining 7 images in the database for each of the 11 subjects (images 2,4,6,8,10,12,14). Upper and lower thresholds were derived for MACE and UMACE filters with the AMP and Biodm facial databases, using the max peak value and PSR performance measures. Figures 3-5 show the thresholds obtained from the AMP facial database using the algorithm in Fig. 1. One should note that the max peak value metric in Fig. 3. produced constant upper threshold values for the

6 1. The total number of false positives (FP) and true positives (TP) for all filters m and training images m n are determined, using thresholds lying between the θ lower and θ upper of the m filters. NUM INCREMENT S 10 CurrentIncrement 0 T P 0 F P 0 FOR i 1 TO NUM INCREMENT S DO FOR j 1 TO m n DO p ((j-1) mod m)+1 q ((j-1) div m)+1 FOR k 1 TO n DO φ k j H k Xp q λ k j CORRELAT ION MEASURE(φk j ) MaxV alue MAX(λ j ) r F IND(MaxV alue, λ j) CurrentT hreshold θ r upper -(θr upper -θr lower ) CurrentIncrement NUM INCREMENTS IF MaxV alue>currentt hreshold THEN IF r = q THEN T P T P +1 ELSE F P F P +1 IF F P > 0 THEN IF i > 1 THEN CurrentIncrement CurrentIncrement-1 OUTPUT CurrentIncrement NUM INCREMENTS STOP ELSE CurrentIncrement CurrentIncrement+1 2. The optimal threshold ratio between the θ lower and θ upper obtained in Step (1), occurs where the TP is maximum and the FP is zero. This value is set as the θ global for the facial database and can used for filter s as follows: OptimalT hreshold θupper s -(θs upper -θr lower ) θ global Matches are observed when the correlation measure value exceeds the OptimalT hreshold for a filter s. Fig. 2. Pseudo-Global threshold (θ global ) derivation algorithm. MACE filter with both databases, which can be explained by the MACE filter constraining the peak height at the origin of the correlation plane to a userdefined value [3]. Furthermore, MACE filters with the max peak value metric in Fig. 3. demonstrated good separation between upper and lower thresholds. The observed good separation may allow the future use of a global threshold for all filters of a particular facial database. UMACE filter training set correlations in Fig. 4. and Fig. 5. were observed to consistently produce variable upper thresholds in comparison to that of MACE filters for both performance measures. As predicted, the lower threshold values generated from Fig. 1. were also highly variable, as shown in Fig. 4. and Fig. 5. The inconsistent nature of the generated upper and lower thresholds for UMACE filters demonstrates an unsuitability of global thresholds for this filter type. One should note that the highly variable upper thresholds observed, are likely the

7 Fig. 3. MACE upper and lower max peak value thresholds for the AMP facial database. Fig. 4. UMACE upper and lower max peak value thresholds for the AMP facial database. result of the UMACE filter maximizing the peak height at the origin without constraint [6]. Following θ global threshold derivation, the recognition performance of the MACE and UMACE filters was ascertained using correlations with the corresponding testing sets. The correlation planes were quantified using the respective performance metric and the precision and recall statistics were in turn calculated (Table. 1.). The precision and recall rates produced with the optimal thresholds in Table 1. show that max peak value performance measure produced larger recall rates with the MACE filter. This result was found to be consistent with both filter types and facial databases. One may attribute this observation to the max peak value metric being better at true class detection than PSR and yet innately good at rejecting false class images. The lower recall rates obtained with the PSR measure could also indicate that the sidelobe regions are of less significance to facial recognition correlation plane quantification. As shown in Table 1, the recognition performance of the MACE and UMACE filters also appears to be data dependent as the UMACE filters performed better with the AMP database, yet worse with the Biodm database.

8 Fig. 5. MACE/UMACE upper and lower PSR thresholds for the AMP facial database. Table 1. Recall rates for optimal (100% precision) thresholds. Facial Database Filter Type Performance Metric Recall (%) AMP Biodm MACE UMACE MACE UMACE Max peak size 92.7% PSR 42.9% Max peak size 94.1% PSR 80.2% Max peak size 87.3% PSR 42.9% Max peak size 66.7% PSR 39.9% 4 Conclusion This paper has provided a performance overview of two correlation plane performance measures with MACE and UMACE correlation filters as well as a mechanism with which to derive a threshold selection range for correlation filters. In this study, it was found that the max peak value performance measure outperformed the standard PSR measure in relation to login access control constraints. In addition, the threshold derivation procedure was seen to be suitable for ascertaining optimal precision filter thresholds, in turn generating high recall rates. We are currently endeavouring to verify these findings with larger facial datbases, such as the PIE and BANCA databases. References 1. Jain, A., Hong, L., Pankanti S.: Biometric identification, Communications of the ACM 43(2) (2000) Vanderlugt A.B.: Signal detection by complex matched spatial filtersing, IEEE Trans. Inf. Theory IT-10 (1964) Mahalanobis, A., Vijaya Kumar, B.V.K., Casasent, D.: Minimum average correlation energy filters, Applied Optics 26(17) (1987)

9 4. Mahalanobis, A., Vijaya Kumar, B.V.K., Song, S., Sims, S.R.F., Epperson, J.F.: Unconstrained correlation filters, Applied Optics 33(17) (1994) Savvides, M., Venkataramani, K., Vijaya Kumar, B.V.K.: Incremental updating of advanced correlation filters for biometric authentication systems, Proc. Of the IEEE International Conference on Multimedia and Expo 3 (2003) Savvides, M., Vijaya Kumar, B.V.K.: Efficient design of advanced correlation filters for robust distortion-tolerant face recognition, Proc. Of the IEEE Conference on Advanced Video and Signal Based Surveillance (2003) Savvides, M., Vijaya Kumar, B.V.K., Khosla, P.: Face verification using correlation filters, Proc. Of the Third IEEE Automatic Identification Advanced Technologies (2002) Vijaya Kumar, B.V.K., Hassebrook, L.: Performance measures for correlation filters, Applied Optics 29(20) (1990) Vijay, V., Bollmann, B., Jung, G.S.: A critical investigation of recall and precision as measures of retrieval system performance, ACM Trans. Info. Sys. 7(3) (2003) Vijaya Kumar, B.V.K., Savvides, M., Venkataramani, K., Xie, C.: Spatial frequency domain image processing for biometric recognition, Proc. Of the IEEE International Conference on Image Processing 1 (2002) I-53 I-56

Spatial Frequency Domain Methods for Face and Iris Recognition

Spatial Frequency Domain Methods for Face and Iris Recognition Dept. of Electrical and Computer Engineering Carnegie Mellon University Pittsburgh, PA 15213 e-mail: Kumar@ece.cmu.edu Tel.: (412) 268-3026