Fusion of multiple clues for photo-attack detection in face recognition systems

Transcription

1 Fusion of multiple clues for photo-attack detection in face recognition systems Roberto Tronci, Daniele Muntoni, Gianluca Fadda, Maurizio Pili, Nicola Sirena, Gabriele Murgia, Marco Ristori Sardegna Ricerche, Ambient Intelligence Lab. Loc. Piscinamanna, 91 Pula (CA), Italy Fabio Roli Deparment of Electrical and Electronic Engineering, Univ. of Cagliari Piazza d Armi, 9123 Cagliari, Italy roli@diee.unica.it Abstract We faced the problem of detecting 2-D face spoofing attacks performed by placing a printed photo of a real user in front of the camera. For this type of attack it is not possible to relay just on the face movements as a clue of vitality because the attacker can easily simulate such a case, and also because real users often show a low vitality during the authentication session. In this paper, we perform both video and static analysis in order to employ complementary information about motion, texture and liveness and consequently to obtain a more robust classification. 1. Introduction Biometrics had been introduced to improve the reliability and the security for access control by an automatic assessment of an individual s identity. Improvement is granted by the fact that while traditional identification systems depend on a passwords to remember or a key to possess, a biometric is always with the individual and it can t be lost or forgotten [2]. As biometric systems spread over and increase their performance, growing interest should be given to possible direct attacks, where potential intruders may gain access to the system by directly interacting with the system input device, like a normal user would. Such attempts are commonly referred as spoofing attacks [9]. A lot of work exists in the literature about fingerprint spoofing (probably because it is the most used biometric) and since the early works[6], many methods for liveness verification have been developed to recognize fake fingers or even real fingers taken away from people or from death bodies. On the other hand, although face recognition systems had captured increasing attention in the past decades, the possibility to deceive a face recognition system is not widely studied. Despite of lower relative performance with respect to other biometrics, like fingerprint and iris recognition, face recognition proves to be a good basis for complete user friendly solutions [12], that makes it a perfect solution especially for low risk applications. However, its technology is intrinsically prone to spoof attacks. The main reason is that a biometric sample is a face represented in a digital image, which is intrinsically highly reproducible by several means like printed photos and electronic portable devices capable of showing images and videos (laptops, UMPCs and even cellular phones have nowadays wide and very good quality screens). In general, there are three possible ways to generate a face spoof attack [8]: generating a photograph of a valid user reproducing a video of a valid user presenting a 3D reproduction of the face of a valid user A few articles mention the video spoof attacks, but solutions are demanded to future researches [8]. To the best of our knowledge, there are no papers that regard 3D face reproductions without hardware assistance. Instead, 3D information is used only as a clue to detect photo attacks (e.g. the Lambertian surface in [1]). But in the next future this will be the next challenge in face spoofing also because nowadays some technologies exist to produce high quality wearable masks /11/$ IEEE

2 Obviously, these types of attacks can be easily detected with the help of specific hardware sensors (IR sensors, stereoscopic cameras), or with the use of multimodality (parallel use of different biometrics or serial approaches), or exploiting the interaction with the user (like challenge response). Besides, a face recognition system can be built with very low cost hardware, and it is particularly suitable for low risk application, therefore in these cases addition of specific hardware or interaction to ensure reliability is not necessarily an affordable solution. This implies that a simple photo spoofing attack can represent a security problem for a face recognition system. In fact, most of papers in the literature refer to the problem as a task of photo attack detection as it represents a cheap and effective way to perform an attack. Commonly cited papers refer to the problem of photo attack detection in two major complementary directions [8]: static analysis, based on the fundamental idea that during the manufacturing process of a photo attack a certain loss of information occurs and also peculiar noise is introduced. video analysis, that tries to detect, as humans do, facial physiological clues like blinks, mouth movements and changes in facial expression. Many static analysis approaches start from the observation that real sessions and reproductions may differ in high frequency details because of the loss of information [4]. Video analysis, conversely, rely on the information extracted from the sequences of images. In [8][1] blinks are used in order to assess vitality. Thus, face photo attacks can be detected through many state of the art algorithms that recognize physiological clues of vitality, such as facial expression variation, mouth movement, head rotation, eye change. However, also this kind of verification is not a definitive proof of vitality due to the possibility to easily reproduce videos of the real users. This implies that static and video analysis must be combined to ensure high reliability in detecting face spoofing attacks. In this work, we regard the problem of 2-D face spoofing attacks detection as a combination of several clues that result from both static and dynamic analysis of the scenes. Static analysis, that allows to detect photo attacks, is therefore reinforced with a video analysis based on motion and liveness clues. The paper is organized as follows: in section 2 we describe our approach with full details about static and video analysis and the fusion algorithm that we propose; section 3 describes experimental results on a photo attack dataset; finally in section 4 we draw our conclusions. 2. Fusion of multiple clues for face spoofing detection Most of the approaches proposed in literature are specifically designed to detect single aspects of vitality and nonvitality (physiological movements, printed photo details, manufacturing details detection), and in fact, with these regards, state of the art algorithms already prove excellent results. However, the reported performance are strictly linked to the specific kind of attack they refer to. For example, there are really good vitality detectors based on image movements that attackers can spoof by waving randomly a photo or by bending it [3] in order to generate non-rigid deformations. So datasets that don t provide this kind of behavior by attackers risk to result in algorithms with a little robustness against such a kind of attacker s adaptations. Besides, the datasets used in the experimental setups risk to be biased and therefore inevitable lack of universality in the datasets doesn t grant performance for real attacks. Our method is therefore a multi-clue algorithm that aims to avoid specificity and lack of universality. The major concern of this approach is that typical vitality features are often strictly correlated. Consequently, the contribute of fusion in a standard multiple classifier system framework is expected to be normally small in terms of performance, due to the fact that classic fusion schemes rely on the statistical independence between classifiers. However the contribution in terms of robustness can be highly significant. Therefore our aim is not only the search for good classification performance but also the search for the robustness in case an attacker manages to spoof the single methods. These facts lead to the rationale behind our method, that is, a fusion of static and video analysis is a strict requirement for spoof detection, provided that no other hardware detection system is available. The method consists of three steps: static feature extraction, video based feature extraction and score level fusion. We first collect a series of possible clues that we separated into two different categories: static features and video features. Then for each category we provide a set of selected scores that we fuse with the Dynamic Combination (DSC) methodology [11] Static Analysis The aim of static analysis is to discover some peculiarities related to the visual structure of the input samples used in the verification process. The basic idea is that differences exists in the visual information between data captured from a real scene, and from a photo or video attack. Basically, these differences can be captured directly from a single image or frame by frame if we are acquiring a video. This approach has the big advantage that it needs only

3 few images/frames in order to have an estimation of whether the input belongs to a real person or to a fake. Instead, in the case of vitality detection several seconds of video are needed to perform the analysis. As mentioned in the introduction, some papers exploit the loss of information at high frequency [4] to detect an attack. In this paper, instead, we propose to use different visual features that are usually used in the content based image retrieval field. Doing so, we are able to detect differences at different visual levels. The visual features that we propose to use are: Color and Edge Directivity Descriptor, Fuzzy Color and Texture Histogram, MPEG-7 Descriptors (like Scalable Color and Edge Histogram), Gabor Texture, Tamura Texture, RGB and HSV Histograms, and JPEG Histogram. These features take into account different visual aspects of an image. In this way we can represent each image with a set of different feature spaces. After obtaining the above mentioned representations, we can train (one for each feature space) a classifier. Thus, for each frame of a video and for each feature space we can obtain a similarity scores i,f with the class real user or face spoof attack through the classifier. Thus, for each frame we have N scores, where N is the number of visual features used. This allows to use Multiple Classifier System also at frame level. Moreover, if the input data is a video composed by M frames, through this schema we obtain for each videon M similarity scores. To obtain a single similarity score for the video we propose to use the Dynamic Combination (DSC) methodology [11]. s sa = (1 α) min{s i,f }+α max{s i,f } (1) This combination rule is able to perform a dynamic combination at score level, that results in the dynamic selection of the best scores and weights to be combined for each video. In [11] different methods to compute dynamically the weights α are presented. In this paper we propose a novel method to compute the weights for the DSC by reviewing the majority vote rule: for each videoαwill be the percentage of frames, among all the visual feature spaces, that are classified as a real user Video Analysis Video analysis, in our approach, is performed as combination of simple measures of movement using standard state of the art methods. Simplicity is the criterion that allows the chosen algorithms to be suitable for real-time or near realtime applications, Owing to our goal of making an online. liveness detection software. The first clue we look for in the video is the classic vitality clue, represented by blinks [8]. Blinks have been exploited for liveness detection in face recognition since the early works in literature. A wide set of publications show blink detection methods that provide excellent results. Blink detector used in this work is the one used in [7]. The original algorithm was modified in order to get a more robust localization through a set of heuristics and in order to work in a multi-threaded application. We take into account the possibility that the algorithm, as well as other state of the art blink detectors, may show a certain number of detection errors. In fact, a number of false detections was observed especially in hand photo attacks, where movement of the photo and light reflections can easily lead detector to some mistakes. It is worth noting that even if blinks are a proof of vitality, a total absence of blinks is not sufficient to infer that the video provided is an attack. On the other hand, this is perfectly acceptable, as it is a recurrent unconscious behavior for the cooperative users to keep steady during authentication. Conversely, for the same reasons, we also have to consider the case of failures to detect real blinks. Therefore a single blink will not be considered as a proof and increasing number of detections must correspond to an increasing confidence and strengthen the evidence of vitality. A normalized score is computed for each video as a function of the number of blinks. The values grow with the number of blinks and tend asymptotically to 1. Besides, for ordinary numbers of blinks, the function should output non-saturated scores in order to ensure the highest sensibility. The above mentioned considerations were taken into account with the introduction of a translation term and a scale factor α in the logistic function. Therefore, the score is computed as: s bl (n b ) = ( 2 1+e α n b ) 1 wheren b is the total number of blinks detected in the video. We then perform a second analysis on videos, based on the simple measure of movement in order to provide some evidence when very little movement is observed in the video. Of course this is a strong proof of non-vitality, but it occurs only when a photo attack is performed with a fixed printed photo. However, this measure tends to strengthen the algorithms when the attack is performed with a very high quality photo. Movement is measured on the foreground mask extracted from a classic motion detection system [5]. The movement scores m is computed through the average movement across the image (the quantity of movement assigned to each pixel): avg m = n mov n tot wheren mov is the number of moving pixels detected by the

4 motion detection and n tot is the total number of pixel in each image. Finally the score s m is computed through a normalization function, once again performed with the logistic function Fusion Fusion was performed at score level using a weighted sum: photo detection provided excellent results, therefore it was given a higher weight in combination; movement measures provided some contribute only for fixed photo attacks therefore the score is used only in case of very little movement. By taking into account the above considerations, we used the following score combination scheme: { s α ssa +(1 α) s = bl, ifs m is high α 1 s sa +α 2 s bl +α 3 s m, ifs m is low (2) where s sa is the score obtained by the static analysis proposed in Section 2.1, while s bl and s m are respectively the scores derived from the blink and motion detection described in Section Experiments For our experiments we used the Print-Attack Replay Database developed for the IJCB 211 Competition on counter measures to 2D facial spoofing attacks from the Idiap Research Institute 2. This dataset consists of 2 video clips of printed-photo attack attempts to 5 clients, under different lighting conditions. It also contains 2 realaccess attempt videos from the same clients. The data is split into 3 sub-groups: training (3% of the data), development (3% of the data), and test (4% of the data). Moreover, the clients that appear on the one of the dataset do not appear in any other set. The dataset were generated by either having a (real) client trying to access a laptop through a built-in webcam or by displaying a printed photo of the same client for at least 9 seconds (videos are acquired with a frame rate of 25 Hz). The photo attack was generated by using high-resolution photos from each client taken under the same conditions as in their authentication sessions. The attacks can be divided into two subsets. The first subset is composed of videos generated using a stand to hold the client printed photo ( fixed ). For the second set, the attacker holds the picture with their own hands. Moreover, all the data (real and attack) is divided in two categories: controlled, office light was turned on, blinds are down, background is homogenous; adverse, blinds up, more complex background, office lights are out. Following the competition instructions, we employed the training set to train our anti-spoof classifiers, the develop- 2 ment set for estimating thresholds and weights to be used in the final fusion scheme applied to the test set. For the static analysis we used the visual features cited in Section 2.1. And for the computation of the similarity scores we used different Support Vector Machines with a radial basis function for their kernels. The results of this analysis are plotted in Figure 1 in terms of score distributions of real access and spoofing attacks. From the figure is clear that both for the development and the test dataset the analysis proposed in Section 2.1 achieve a perfect separation Static Analysis: development dataset Static Analysis: test dataset Figure 1. distributions of the similarity scores obtained only from static analysis for spoofing attacks (in red) and real access (in blue). In Figure 2 we can clearly see that the vitality detection performed through blinks is able to distinguish between real accesses and spoofing attacks only if the number of blinks is high. If the number is low we can t infer that the proposed pattern is an attack due to some false detection rate and the fact that some real access shows no blinks. These

5 2 Blink Detection: development dataset 6 Total Movement distribution: developement dataset Number of Blinks 3 Blink Detection: test dataset Figure 3. Distribution of scores for movement analysis of the videos: the fixed attacks (in red) can highly distinguished from the hand attacks (in green) and the real access (in blue) Number of Blinks Figure 2. Distributions of number of blinks detected in the videos for spoofing attacks (in red) and real access (in blue). experimental results confirm what we claimed in 2.2: in the case of low vitality scores we can t claim that the access tentative is a fake/spoof. The motion detection score distributions depicted in Figure 3 allow to detect, for the development set, when a fixed hand attack is performed. These experimental results confirm the fusion schema proposed in Equation 2: when the motion is above a certain threshold we can t infer if the input video is from a real access or a spoofing attack. Finally, in Figure 4 we show the results of the formula proposed in section 2.3 in terms of score distributions. It can be clearly seen in the figures that a perfect separation of the score distributions can be achieved for both development set and test set (where the weights are estimated on the development set). By comparing Figure 1 and 4 it is clear that for this specific dataset the static analysis allows to obtain a bet- ter classification than the motion analysis. Nevertheless, we believe that a spoofing detection system must take into account also vitality and video analysis (even if in a small portion). 4. Conclusions In this work, we addressed the problem of photo detection within the context of automatic face verification systems. We proposed a linear fusion combination between static and video analysis. The video analysis was used to detect vitality and motion clues through state of the art algorithms. While a novel static analysis was used to find clues frame by frame. The proposed static analysis exploits different visual features and SVM classifiers through a dynamic score combination methodology and provides excellent results in terms of performance. The results achieved only with the static analysis point out that actually photo spoofing attacks can be easily detected by using this kind of analysis. Moreover, the proposed static analysis is able to work also by using only a single frame. This is a clear advantage with respect of other methods that also look for visual degradation clues, but they need to process a portion of video. Obviously, a biometric spoofing detector can t discard vitality information. Thus the proposed combination rule proposed in this paper allows to take into account different clues from the static and the video analysis by giving a better confidence on the classification results. As a part of future work, by considering the speed in the evolution of spoofing attacks and technologies, we will investigate other forms of 2-D and 3-D attacks for face spoofing.

6 Static and Video Analysis combination: development dataset Static and Video Analysis combination: test dataset Figure 4. distributions of the similarity scores obtained from the fusion schema proposed for spoofing attacks (in red) and real access (in blue). Acknowledgements [3] K. Kollreider, H. Fronthaler, and J. Bigun. Verifying liveness by multiple experts in face biometrics. In Computer Vision and Pattern Recognition Workshops, 28. CVPRW 8. IEEE Computer Society Conference on, pages 1 6, june 28. [4] J. Li, Y. Wang, T. Tan, and A. K. Jain. Live face detection based on the analysis of fourier spectra. In In Biometric Technology for Human Identification, pages , 24. [5] L. Li, W. Huang, I. Y. H. Gu, and Q. Tian. Foreground object detection from videos containing complex background. In In MULTIMEDIA 3: Proceedings of the eleventh ACM international conference on Multimedia, pages 2 1. ACM Press, 23. [6] T. Matsumoto, H. Matsumoto, K. Yamada, and S. Hoshino. Impact of Artificial Gummy Fingers on Fingerprint Systems. volume Society of Photo-optical Instrumentation Engineers, International Society for Optical Engineering, 22. [7] G. Pan, L. Sun, and Z. Wu. Eyeblink-based anti-spoofing in face recognition from a generic webcamera, 27. [8] G. Pan, Z. Wu, and L. Sun. Liveness detection for face recognition. In Recent Advances in Face Recognition, pages I-Tech, 28. [9] S. A. C. Schuckers, S. A. C. Schuckers, P. D, and P. D. Spoofing and anti-spoofing measures. Information Security Technical Report, 7:56 62, 22. [1] X. Tan, Y. Li, J. Liu, and L. Jiang. Face liveness detection from a single image with sparse low rank bilinear discriminative model. In Proceedings of the 11th European conference on Computer vision: Part VI, ECCV 1, pages , Berlin, Heidelberg, 21. Springer-Verlag. [11] R. Tronci, G. Giacinto, and F. Roli. Dynamic score combination: A supervised and unsupervised score combination method. In P. Perner, editor, Machine Learning and Data Mining in Pattern Recognition, volume 5632 of Lecture Notes in Computer Science, pages Springer Berlin / Heidelberg, 29. [12] W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld. Face recognition: A literature survey. ACM Computing Surveys, 35: , 23. This work was partially supported by the TABULA RASA project, 7th Framework Research Programme of the European Union (EU), grant agreement number: , and by the PRIN 28 project Biometric Guards - Electronic guards for protection and security of biometric systems funded by the Italian Ministry of University and Scientific Research (MIUR). References [1] M. Chau and M. Betke. Real time eye tracking and blink detection with usb cameras. Technical report, 25. [2] A. K. Jain, A. Ross, and S. Prabhakar. An introduction to biometric recognition. IEEE Trans. on Circuits and Systems for Video Technology, 14:4 2, 24.