Support Vector Machines for Face Authentication

Transcription

1 Suort Vector Machines for Face Authentication K Jonsson 1 2, J Kittler 1,YPLi 1 and J Matas CVSSP, University of Surrey Guildford, Surrey GU2 5XH, United Kingdom 2 CMP, Czech Technical University Prague, Czech Reublic Abstract The aer studies Suort Vector Machines (SVMs) in the context of face authentication. Our study suorts the hyothesis that the SVM aroach is able to extract the relevant discriminatory information from the training data. We believe this is the main reason for its suerior erformance over benchmark methods. When the reresentation sace already catures and emhasises the discriminatory information content as in the case of Fisherfaces, SVMs loose their sueriority. SVMs can also coe with illumination changes, rovided these are adequately reresented in the training data. However, on data which has been sanitised by feature extraction (Fisherfaces) and/or normalisation, SVMs can get over-trained, resulting in the loss of the ability to generalise. SVMs involve many arameters and can emloy different kernels. This makes the otimisation sace rather extensive, without the guarantee that it has been fully exlored to find the best solution. 1 Introduction Verification of erson identity based on biometric information is imortant for many security alications. Examles include access control to buildings, surveillance and intrusion detection. Furthermore, there are many emerging fields that would benefit from develoments in erson verification technology such as advanced human-comuter interfaces and tele-services including tele-shoing and tele-banking. Comaring verification to recognition there are several asects which differ. First, a client an authorised user of a ersonal identification system is assumed to be co-oerative and makes an identity claim. Comutationally this means that it is not necessary to consult the comlete set of database images (denoted model images below) in order to verify a claim. An incoming image (referred to as a robe image) is thus comared to a small number of model images of the erson whose identity is claimed and not, as in the recognition scenario, with every image (or some descritor of an image) in a otentially large database. Second, an automatic authentication system must oerate in near-real time to be accetable to users. Finally, in recognition exeriments, only images of eole from the training database are resented to the system, whereas the case of an imoster (most likely a reviously unseen erson) is of utmost imortance for authentication. 543 BMVC 1999 doi: /c.13.54

2 The field of face recognition is well established and a large number of algorithms have been roosed in the literature. Poular aroaches include the ones based on deformable temlates [10], dynamic link matching [4], and Eigenfaces [9]. These techniques vary in comlexity and erformance and the choice of algorithm is tyically deendent on the secific alication. The verification roblem, on the other hand, is less exlored. Recent examles include [3] in which a robust form of correlation is alied to face authentication. The aim of the aer is to evaluate the effectiveness of the SVM aroach to face authentication. An earlier study of SVMs in face verification has been reorted by Phillis [8]. An SVM verification system design was comared with a standard Princial Comonent Analysis (PCA) face authentication method and the former was found to be significantly better. In this aroach the SVM was trained to distinguish between the oulations of within-client and between-client difference images resectively, as originally roosed by Moghaddam [7]. This method gives client non-secific suort vectors. In our aroach we adot a client-secific solution which requires learning clientsecific suort vectors. However, this is not the main distinguishing feature of our work: it only reflects our focus on authentication as oosed to recognition which is of concern in [8]. Our rimary motivation for carrying out a similar study was to establish why the erformance of the SVM aroach is suerior. We want to investigate the inherent otential of SVMs to extract the relevant discriminatory information from the training data irresective of reresentation and rerocessing. In order to achieve this objective we have designed exeriments in which faces are reresented in both Princial Comonent (PC) and Linear Discriminant (LD) subsaces. The latter basis (Fisherfaces) is used as an examle of a face reresentation with focus on discriminatory feature extraction while the former achieves simly data comression. We also study the effect of image hotometric normalisation on the erformance of the SVM method. A number of criteria have been considered as a basis for the SVM aroach evaluation, using other baseline techniques as a benchmark. We have included as benchmark verification methods not only the classical PC variants with the L 2 norm and correlation coefficient resectively, but also the LD sace with the same two decision schemes. As criteria for evaluating SVMs in relation to the benchmark methods we have concentrated on the following: comutational comlexity (training mode and routine verification mode), robustness (sensitivity to inut data conditioning), ability to extract discriminatory information, the merits of nonlinear boundaries. The last three criteria are exressed quantitatively in terms of the false client rejection and imostor accetance rates. The findings of our study strongly suort the hyothesis that the SVM aroach is owerful in the sense of being able to extract the relevant discriminatory information from the training data. This is the main reason for the large difference between the observed erformance of the classical Eigenface classification methods and SVMs (factor of almost 3). When the reresentation sace already catures and emhasises the discriminatory information content as in the case of LD bases, SVMs cease to be suerior to the simle Euclidean distance or correlation decision rules. SVMs also show a suerior caability to coe with illumination changes, rovided these are adequately reresented in the training data. However, on data which has been sanitised by feature extraction (Fisherfaces) and/or normalisation, SVMs can get overtrained, resulting in the loss of the ability to generalise. SVMs involve many arameters and can emloy different kernels. This makes the otimisation sace rather extensive, 544

3 without the guarantee that it has been fully exlored to find the best solution. The aer is organised as follows. In the next section we introduce the two face reresentation saces used in our study, namely Eigenfaces and Fisherfaces. In Section 2 we overview the SVM aroach to face identity verification and summarise the benchmark classification methods. Section 3 introduces the face database used in exerimentation and describes the exeriments carried out, their objectives and the results obtained. The results are discussed in Section 4 and conclusions are drawn in Section 5. 2 Face Authentication Any authentication rocess involves two basic comutational stages. In the first stage a suitable reresentation is derived with the multile objective of making the subsequent, decision-making stage, comutationally feasible, immune to environmental changes during the biometric data acquisition, and effective by roviding it only with information which is ertinent to the authentication task. The urose of the second stage is to accet or reject the identity claim corresonding to a robe biometric measurement. This is basically a two-class attern recognition roblem. In the following subsections we introduce the methods adoted for the design of each of these two stages in the context of the face authentication study ursued in this aer. 2.1 Reresentation of Faces The first ste in the face reresentation rocess involves image re-rocessing in order to establish corresondence between face images to be comared. Once an image is registered, it can further be normalised hotometrically. In our study we set out to investigate the resilience of different decision making methods to varying illumination and thus this ste was alied only in a subset of exeriments. In the final ste of rocessing, the image is rojected into a coordinate system which facilitates the decision making rocess comutationally and ossibly emhasises the imortant attributes for face verification. Geometric Normalisation. As the focus of the aer is on the decision making asects of face authentication we have tried to eliminate the deendency of our exeriments on rocesses which may lack robustness. For this reason we have erformed face registration semi-automatically. The rocedure is based on manually localised eye ositions. Four arameters comuted from the eye coordinates (rotation, scaling and translation in the horisontal and vertical directions) are used to cro the face art from the original image and scale it to any desired resolution. Photometric Normalisation. When alied, the hotometric normalisation consisted of removing the mean of the geometrically normalised image and scaling the ixel values by their standard deviation, estimated over the whole croed image. Image Projection. Suose that we have c clients and M training face images x i i = 1 ::: M, x i 2 R D each belonging to one of the client classes fc1 C2 ::: C c g.then we can define the following second-order statistics: 545

4 Between-class scatter matrix: cx S B = 1 ( k ; )( k ; ) T (1) c k=1 Within-class scatter matrix: S W = 1 M cx X (x i ; k )(x i ; k ) k=1 ijx i2ck T (2) Total scatter matrix: S T = S W + S B (3) where is the grand mean and k is the mean of class C k. The aim of the Princial Comonent Analysis is to identify the subsace of the image sace sanned by the training face image data and to decorrelate the ixel values. This can be achieved by finding the eigenvectors W ca of matrix S T associated with nonzero eigenvalues by solving S T W ca ; W ca =0 (4) These eigenvectors are referred to as Eigenfaces. The classical reresentation of a face image is obtained by rojecting it to the coordinate system defined by the Eigenfaces. The rojection of face images into the Princial Comonent (Eigenface) subsace achieves information comression, decorrelation and dimensionality reduction to facilitate decision making. If one is also interested in identifying imortant attributes (features) for face authentication, one can adot a feature extraction maing. A oular technique is to find the Fisher linear discriminants (Fisherfaces) by solving S B W lda ; S W W lda =0 (5) The rojection of a face image into the system of Fisherfaces associated with nonzero eigenvalues will yield a reresentation which will emhasise the discriminatory content of the image. The solution of the generalised eigenvalue roblem in Equation 5 is known, but due to the high dimensionality many standard methods fail and the choice of a stable numerical algorithm is non-trivial [5]. Figure 1 shows the first few PC and LD basis images. In Section 3, we erform exeriments with different number of basis vectors taken from either the Eigenface or Fisherface systems. In the following, for the sake of notational simlicity, we shall not distinguish between the two different basis systems, nor shall we exlicitly denote the dimensionality of the reresentation sace. The actual reresentation used will be clear from the exeriment descrition. Thus in general, in each exeriment we shall work with some transformation matrix W. A samle face image y will then be reresented by a rojection x obtained as x = W T y. Similarly, the client model k will be rojected into a vector! k in the aroriate reresentation sace. 2.2 Classification Suort Vector Machines. The main decision making tool investigated in this aer is the Suort Vector Machine. Below we give a brief resentation of the basic theory. The 546

5 (a) Fisherfaces, unnormalised data (b) Eigenfaces, unnormalised data (c) Eigenfaces, normalised data Figure 1: Basis vectors: Subsace comuted using (a) LDA (unnormalised data) and (b c) PCA (unnormalised and normalised data). In all three cases, the first six basis vectors are shown. reader is referred to [1] for a more comrehensive introduction. SVMs are based on the rincile of structural risk minimisation. The aim is to minimise the uer bound on the exected (or actual) risk defined as 1 Z 1 R() = jz ; f (x )jdp (x z) (6) 2 where is a set of arameters defining the trained machine, z a class label associated with a training samle x, f (x ) a function roviding a maing from training samles to class labels, and P (x z) the unknown robability distribution associating a class label with each training samle. Let l denote the number of training samles and choose some such that 0 1. Then, with robability 1 ;, the following bound on the exected risk holds: r h(log(2l=h) +1); log(=4) R() R em () + (7) l where R em () is the emirical risk as measured on the training set and h is the so called Vanik Chervonenkis (VC) dimension. The second term on the right hand side is called the VC confidence. There are two strategies for minimising the uer bound. The first one is to kee the VC confidence fixed and to minimise the emirical risk and the second one to fix the emirical risk (to a small value) and minimise the VC confidence. The latter aroach is the basis for SVMs and below we will briefly outline this rocedure. First consider the linear searable case. We are looking for the otimal hyerlane in the set of hyerlanes searating the given training samles. This hyerlane minimises the VC confidence and rovides the best generalisation caabilities. Giving a geometric 1 The notation is similar to the one in [1]. 547

6 interretation, the otimal hyerlane maximises the sum of the distances to the closest ositive and negative training samles. This sum is called the margin of the searating hyerlane. It can be shown that the otimal hyerlane w x+b =0(where w is normal to the hyerlane) is obtained by minimising kwk 2 subject to a set of constraints. This is a quadratic otimisation roblem. These concets can be extended to the non-searable and non-linear case. The searability roblem is solved by adding a term to the exression subject to minimisation. This term is the sum of the deviations of the non-searable training samles from the boundary of the margin. This sum is weighted using a arameter controlling the cost of misclassification. The second roblem is how to handle non-linear decision boundaries. This is solved by maing the training samles to a high-dimensional feature sace using kernel functions. In this sace the decision boundary is linear and the techniques outlined above can be directly alied. The kernel functions used in the exeriments reorted in Section 3 are linear, olynomial and radial basis functions (RBFs) defined as K(x i x j ) = x i x j (8) K(x i x j ) = (x i x j +1) d (9) K(x i x j ) = e ;kxi;xjk2 (10) where x i and x j denote two samles. The user-controlled arameters are the degree d in the case of the olynomial and the -value in the case of the RBF kernel. In addition to the SVMs with different kernels we have also imlemented the following standard classification rules as baselines for exerimental comarison. Euclidean Distance. The most commonly used decision rule is based on the Euclidean distance between the samle rojection x and the rojection of the k-th client mean! k, i.e. q d E (x! k )= (x ;! k ) T (x ;! k ) (11) The claimed client identity is acceted if d E (x! k ) is below a threshold Ek. Otherwise it is rejected. Normalised Correlation. Alternatively, the decision can be based on the correlation score d C (x! k )= jxt! k j (12) jxjj! k j In the case of the correlation measure the claimed identity is acceted if d C (x! k ) exceeds a re-secified threshold Ck. Client-Secific Thresholding. The client-secific threshold k can be determined from the receiver oerating characteristic (ROC) comuted on an indeendent evaluation set. The rocedure amounts to generating ROC curves arametrised by secific ercentiles of the imoster distance distributions. Each ercentile defines client-secific offsets. The ROC curves are roduced by measuring the false rejection and false accetance rates for different distance increments measured from these offsets. The ROC curve yielding the minimum equal error rate and the actual increment giving this error jointly define the client secific thresholds. 548

7 3 Exerimental Results BMVC99 The exeriments summarised below were all erformed on frontal-face images from the extended M2VTS multi-modal database [6]. This ublicly available database contains face images and seech recordings of 295 ersons. The subjects were recorded in four searate sessions uniformly distributed over a eriod of 5 months, and within each session a number of shots were taken including both frontal-view and rotation sequences. In the frontal-view sequences the subjects read a secific text (roviding synchronised image and seech data), and in the rotation sequences the head was moved vertically and horizontally (roviding information useful for 3D surface modelling of the head). The exeriments were conducted according to the Laussane evaluation rotocol [6]. This rotocol rovides a unified framework within which the erformance of vision- and seech-based erson authentication systems running on the extended M2VTS database can be measured. The rotocol secifies a artitioning of the database into three disjoint sets: a training set (200 clients), an evaluation set (200 clients and 25 imostors) and a test set (200 clients and 70 imostors). The training set is used to build client models, the evaluation set to get distributions of client and imostor scores used to establish verification thresholds, and the test set to obtain a reliable estimate of the verification rate on indeendent data. 3.1 Results on Face Authentication Exeriments were erformed for the two different reresentations with and without face normalisation, giving four results for each authentication method. The results are summarised in Table 1. Let us first of all look at the baseline methods. One can see that the erformance of both the Euclidean distance and the correlation classifiers imroves monotonically with the data quality (PCs without normalisation, PCs with normalisation, LD without normalisation, LD with normalisation). The Euclidean distance is articularly sensitive to the deviations from the imlicit model underlying the aroach, i.e. client clusters being very comact and roughly sherical. The correlation coefficient can coe better with deviations from the shericity. However, once the data is of that form as in the case of the LD bases with normalised data, the inherent flexibility of this classification method results in a slightly worse erformance than that achieved by the Euclidean distance classifier. The verification erformance as a function of subsace dimensionality for the PCA subsaces using SVMs as classification scheme is shown in Table 2. For both subsaces (unnormalised and normalised), the total error rate dros when the number of coefficients is increased. However, when a certain oint is reached (about 200 coefficients ) the erformance saturates and there is no further imrovement of the verification rates. This series of exeriments shows that the SVMs are robust to changes in the quality of the reresentation and erform well on both under-reresented data (low number of coefficients) and when noise is resent (high number of coefficients). A somewhat surrising result of the exeriments is that the SVMs tend to erform better for lower data quality. In fact Table 1 shows that the erformance is almost without excetion inversely related to data quality. The best results have been obtained for PCs without face normalisation and the worst results for LDs with face normalisation when the authentication roblem becomes relatively easy. This suggests that SVMs can be over- 549

8 SSP CLM NOR KRN PAR Evaluation set Test set EER TE FR FA TE PCA EUD N/A N/A NOC N/A N/A SVM LIN N/A ? EUD NOC SVM POL RBF N/A N/A N/A N/A LIN N/A POL RBF RBF SSP CLM NOR KRN PAR Evaluation set Test set EER TE FR FA TE LDA EUD N/A N/A NOC N/A N/A SVM LIN N/A EUD NOC SVM BMVC99 POL RBF N/A N/A N/A N/A LIN N/A POL RBF Table 1: Verification erformance on the extended M2VTS database: False rejection (FR), false accetance (FA) and total error rate (TE) as functions of subsace (SSP, uer table PCA, lower table LDA), classification method (CLM), hotometric normalisation (NOR) and kernel (KRN, if alicable). The classification methods are Euclidean distance (EUD), normalised correlation (NOC) and suort vector machines (SVM). The kernels are linear (LIN), olynomial (POL) and radial basis functions (RBF). The kernel arameters (PAR) are indicated when alicable (degree for olynomial and for RBF). The row marked with a? was obtained using a different subsace (see text). SSP NoC ERG Evaluation set Test set EER TE FR FA TE UN NO Table 2: Verification erformance as function of PCA subsace dimensionality: False rejection (FR), false accetance (FA) and total error rate (TE) as functions of subsace (SSP) and number of rojection coefficients (NoC). The two subsaces were obtained from unnormalised (UN) and normalised (NO) data. The corresonding ercentage of the energy (ERG) in the subsace is also indicated. All results were obtained using RBF kernel ( =0:01). 550

9 trained on relatively easy data and cannot generalise to new data so readily. This finding has been confirmed by another exeriment where the PC basis system was comuted for normalised face data but without removing the global face mean (row marked with?). In this case the first Eigenface is comletely determined by this global mean face and the successive Eigenfaces are influenced accordingly. For this system of PCs the test set erformance of 5.75% is comarable to the best achieved with the Euclidean distance decision rule (5.68%). The following conclusions can be drawn from the study: The SVM aroach is able to extract the relevant discriminatory information from the data fully automatically. It can also coe with illumination changes. The major role in this characteristic is layed by the SVMs ability to learn non-linear decision boundaries. On data which has been sanitised by feature extraction (Fisherfaces) and/or normalisation, SVMs can get over-trained, resulting in the loss of the ability to generalise. SVMs involve many arameters and can emloy different kernels. This makes the otimisation sace rather extensive, without the guarantee that it has been fully exlored to find the best solution. An SVM takes about 5 seconds to train er client (on a Sun Ultra Enterrise 450). This is about an order of magnitude longer than determining client-secific thresholds for the Euclidean and correlation coefficient classifiers. However, from the ractical oint of view the difference is insignificant. 4 Discussion It is interesting to note that the total error rates achieved on the evaluation set by all the SVM methods are very similar. The better results on the test set yielded by some of the techniques suggest that the test set data is easier than the evaluation set data. The differences in the test set erformance are erhas indicative of the different generalisation caabilities of the resective methods. While it is true that the SVMs are designed not to over-train, if the reresentation sace used is excessively tuned to the training data already, the SVM cannot mitigate such an inherent roblem. This could exlain the better test set erformance on a less owerful reresentation afforded in terms of the PC sace. Note also the difference in the information used by the various decision making schemes which can be gleaned from Figure 2. In this figure we show the original of a robe image, its PC reconstruction in Figure 2b, its LD reconstruction in Figure 2c and finally its SVM reconstruction in Figure 2d. The latter two reconstructions have been roduced in an analogical way to the PC reconstruction aroach. The classical Euclidean distance and correlation methods use the standard PCA reconstruction of the robe image shown in Figure 2b. The SVM using the PC coefficients work with a similar source of information but some regions in the image are weighted more heavily. Thus, SVMs seem to be caable of erforming client-deendent feature extraction. The Fisherface reconstruction uses the global mean image as a starting oint. The bright areas indicate the increased weighting alied to some ixels in the image. This weighting is client deendent and is a function of the robe image rojection into the Fisher sace. 551

10 (a) (b) (c) (d) Figure 2: Reconstruction from subsace reresentations: (a) original image, reconstruction using (b) PCA (191 coefficients), (c) LDA (199 coefficients) and (d) SVM (37 coefficients) subsaces. All subsaces were obtained from unnormalised data. 5 Conclusions The aer studied SVMs in the context of face authentication. Our study roved the hyothesis that the SVM aroach is able to extract the relevant discriminatory information from the training data. This is the main reason for the large difference between the observed erformance of the classical Eigenface classification methods used as a benchmark and SVMs (factor of almost 3). When the reresentation sace already catures and emhasises the discriminatory information content as in the case of LD bases, SVMs cease to be suerior to the simle Euclidean distance or correlation decision rules. SVMs also show a suerior caability to coe with illumination changes, rovided these are adequately reresented in the training data. However, on data which has been sanitised by feature extraction (Fisherfaces) and/or normalisation, SVMs can get overtrained, resulting in the loss of the ability to generalise. SVMs involve many arameters and can emloy different kernels. This makes the otimisation sace rather extensive, without the guarantee that it has been fully exlored to find the best solution. Acknowledgements The research reorted in this aer was carried out within the framework of the Euroean Union ACTS roject M2VTS and ESPRIT RETINA. We would also like to acknowledge Thorsten Joachims at the University of Dortmund for making the SVM Light rogram [2] available to the research community. References [1] C. J. C. Burges. A tutorial on suort vector machines for attern recognition. Data Mining and Knowledge Discovery, 2(2): , [2] T. Joachims. Making large-scale SVM learning ractical. In B. Schölkof, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods: Suort Vector Learning, ages 41 56, Cambridge, USA, MIT Press. 552

11 [3] K. Jonsson, J. Matas, and J. Kittler. Learning salient features for real-time face verification. In S. Akunuri and C. Kullman, editors, Audio- and Video-Based Biometric Person Authentication, Washington D. C., March 1999, ages 60 65, [4] M. Lades, J. C. Vorbrüggen, J. Buhmann, J. Lange, C. v.d. Malsburg, R. P. Würtz, and W. Konen. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Comuters, 42(3): , Mar [5] Y P Li, J Kittler, and J Matas. Effective imlementation of linear discriminant analysis for face recognition and verification. In A. Leonardis and F. Solina, editors, Comuter Analysis of Images and Patterns, Ljubljana, Slovenia, Set, to aear. [6] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre. XM2VTSDB: The extended M2VTS database. In S. Akunuri and C. Kullman, editors, Audio- and Video- Based Biometric Person Authentication, Washington D. C., March 1999, ages 72 77, [7] B. Moghaddam, W. Wahid, and A. Pentland. Beyond eigenfaces: Probabilistic matching for face recognition. In Automatic Face and Gesture Recognition, Nara, Jaan, Aril 1998, ages 30 35, [8] P. J. Phillis. Suort vector machines alied to face recognition. In M. S. Kearns, S. A. Solla, and D. A. Cohn, editors, Advances in Neural Information Processing Systems 11, [9] M. A. Turk and A. P. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71 86, [10] A. L. Yuille. Deformable temlates for face recognition. Journal of Cognitive Neuroscience, 3(1):59 70,