Face Anti-spoofing based on Deep Stack Generalization Networks

Transcription

1 Xi Nig,, Weiju Li,, Meili Wei, Liju Su, ad Xiaoli Dog, Istitute of Semicoductors, Chiese Academy of Scieces, 00083, Beijig, Chia Cogitive Computig Techology Wave Joit Lab, 00083, Beijig, Chia Keywords: Abstract: Face Ati-spoofig, Covolutioal Neural Networks, Stacked Geeralized Approach, Itra-class Variatios. Thaks for the recet developmet of Covolutioal Neural Networks (CNNs), the performace of face ati-spoofig methods has bee improved by extractig more distiguishig features betwee geuie ad fake faces tha the had-crafted texture features. As kow, the way of fraud is diverse, thus the fake class has large itra-class variatios, so traiig as a biary classificatio problem is hard to lear the distiguishig features. I this work, our cotributio is a ovel model fusio approach for face atispoofig, which ca reduce the itra-class variatios. Accordig to the type of fraud, we firstly trai differet models for face ati-spoofig problem by CNN, thus the itra-class variatios of fake class has reduced durig traiig each model. Distiguishig features ca be leared more easily. The the stacked geeralized method is used for combiig the lower models to achieve better predictive accuracy. For perfectig the geeralized accuracy, the stacked geeralized approach chages the weight of each model's predictio, so that the model after fusio ca predict precisely whether the face image is fake or geuie. Meawhile, the experimetal results idicate our method ca obtai excellet results compared to the stateof-the-art methods. INTRODUCTION Face recogitio has achieved great success durig the past decades, ad has bee widely applied i access cotrol system, logi system ad so o. However, a high security requiremet for face autheticatio is urget, because oly a photo, video replay, or 3D-mask of valid user ca easily spoof a face recogitio system to access secure iformatio illegally. With the popularity of electroic devices, people ca easily get photos ad videos belogig to others through the etwork, which caused a lot of face recogitio system for security risks. Therefore, ati-spoofig problem for face biometric system has gaied great attetio to the scholars ad compaies. The fragility of face recogitio systems to face spoof attacks has motivated a umber of studies o face ati-spoofig, such as LBP (Maatta et al., 0), HOG (Komulaie et al., 03), LBP-TOP (Pereira, 0), Image quality assessmet (We et al., 05), CNN (Yag et al., 04), etc. However, published studies are limited i their scope, because these methods are more to trai a commo model to prevet attack from photo, video or 3D-mask. Because of the diversity of fraud, a geeral model caot be leared i such complexities, may ot work whe facig specific fraud. Besides, these methods regard face atispoofig as a biary classificatio problem, that is to say the all kids of fake face is oe class, ad the real face is aother class. As we all kow, the fake face ca be a photo, a video replay, or a 3D-mask. Due to differet ways of fraud, the fake class has large itra-class variatios. These existig classifiers i idetifyig sample work idividually, as is well kow, whe makig critical decisios, wise people ofte take ito accout the opiios from several experts rather tha oly oe. I this paper, we do ot address 3D-mask attacks, which are more costly to lauch, we focus o photo ad replayed video attacks. We lear a CNN model for each type of fraud, the the stacked geeralizatio method will be use to itegrate the two models, which will decide whether the iput face image is real or ot. Stacked geeralizatio is a geeral method of usig a highlevel model to combie lower-level models to achieve greater predictive accuracy. The remaider of the paper is orgaized as follows: Sectio briefly reviews the relevat work o face ati-spoofig. Sectio 3 presets our 37 Nig, X., Li, W., Wei, M., Su, L. ad Dog, X.. DOI: 0.50/ I Proceedigs of the 7th Iteratioal Coferece o Patter Recogitio Applicatios ad Methods (ICPRAM 08), pages ISBN: Copyright 08 by SCITEPRESS Sciece ad Techology Publicatios, Lda. All rights reserved

2 ICPRAM 08-7th Iteratioal Coferece o Patter Recogitio Applicatios ad Methods approach. The experimetal setup ad results are discussed i Sectio 4. Fially, i Sectio 5 we summarize this work highlightig its mai cotributios. RELATED WORK Because of the diversity of spoofig attacks, existig traditioal face ati-spoofig approaches ca be maily categorized ito four categories: (i) motio based methods, (ii) texture based methods, (iii) method based o image quality aalysis, ad (iv) methods based o other cues. The motio based methods was desiged primarily to couter prited photo attacks. Eye blikig (Pa et al., 007) or lip movemet (Kollreider et al., 007) are used for face ati-spoofig. Give that motio is a relative feature across video frames, these methods are expected to have better geeralizatio ability tha the texture based methods. However, motio based methods eed a relatively log time to accumulate stable vitality features for face spoof detectio. The texture based methods iclude LBP (Maatta et al., 0), HOG (Komulaie et al., 03), etc. Pereira et al. (0) used LBP-TOP to extract spatial ad time domai features from three orthogoal plaes. Ulike motio based methods, texture based methods eed oly a sigle image to detect a spoof. However, the geeralizatio ability of may texture based methods has bee foud to be poor. A recet work (Galbally et al., 04) proposed a biometric liveess detectio method for iris, figerprit ad face images usig 5 image quality measures, icludig full-referece measures ad 4 oreferece measures. Differet from traditioal methods, CNNs ca extract distiguishig ed-to-ed features directly from raw data, ad has bee proved efficiet i may other visio fields. Yag et al., (04) extract features by CNN, the feedig them to a SVM classifier. Xu et al., (06) proposed LSTM-CNN architecture to lear the temporal structure from videos. Those works cosider the face ati-spoofig as a biary classificatio problem, all real face is oe class, ad the other is all kids of fake face. Because of the variety of fake face, photo attacks ad video attacks will be differet o the texture, reflect illumiatio, resolutio, etc., thus the large itravariace will icrease the difficulty of classificatio. Each model is heterogeeous ad has strog classificatio ability, therefore, the itegratio model with stacked geeralizatio method will make full use of the advatages of differet models, complemet each other, thus will achieve better predictive accuracy. 3 PROPOSED METHOD For reducig the itra-class variatios, we trai differet models accordig to the type of fraud face. Each model ca lear the deep ad distiguishig feature for classifyig fake or real. The stacked geeralizatio method takes full advatage of each model's predictio ad chage the weights of each predictio. The a wiser decisio would be made for maximizig geeralize accuracy. With the stacked geeralized method, the traiig difficulty of atispoofig problem is decreased tha traiig a geeral model. Besides, the model ca coverge more easily. 3. Stacked Geeralizatio Stacked geeralizatio (Wolpert, 99; Tig ad Witte, 997) is a geeral method of usig a highlevel model to combie lower-level models to achieve greater predictive accuracy. It's a scheme for miimizig the geeralizatio error rate of oe or more classifiers, ad works by reducig the biases of the classifiers with respect to a provided learig set. Whe fusig the multiple classifiers, stacked geeralizatio exploited a strategy more sophisticated for combiig the idividual classifiers. Stacked geeralizatio tries to lear which classifiers are reliable oes, ad use a higherlevel learig algorithm, the so-called "metaclassifier", to discover the best way of how to combie the outputs of the base classifiers. As show i Figure, there are two kids of classifiers: several base classifiers (leavel-0 classifiers) ad oe meta-classifier (level- classifier). The output class probabilities geerated by level-0 models are used to form level- data. The a multivariable liear regressio model (MLR) is adapted for classificatio tasks for level- classifier. 3.. Level-0 Geeralizers As show before (Krizhevsky et al., 0; Simoya ad Zisserma, 04; He et al., 06), deeper ad better pre-traiig etworks lead to better performace. ResNet (He et al., 06) wo the st place o the ILSVRC 05 classificatio task. The depth of represetatios is of cetral importace for may visual recogitio tasks, especially i face 38

3 ati-spoofig. I the task of face ati-spoofig, the itra-class variatios are large, ad are maily caused by the appearace of differet people, the differet ways of fraud, ad the differet resolutios of faces images captured by differet camera, such as the photo attacks ca be prited o differet types of paper, the video attacks ca be played o differet electroic equipmet ad so o. Therefore, the deeper etworks ca extract more useful ad distiguishig features. I this paper, we use a ResNet50 (He et al., 06) model as the level-0 geeralizers. Give two traiig data sets Q y, x,,..., N ad P y, x,,..., N,where y is the class value ad x represets the attribute values of the ss istace, Q defies the set of real face ad video attack face, ad P defies the set of real face ad photo attack face. The specific type of fraud is icluded i each dataset, causig the small itraclass variatios ad large iter-class variatios. So the traiig difficulty ca be decreased. The radomly split the data sets ito two parts Q, P ad Q, P. Defie P y, x,,..., N M Q y, x,,..., M P y, x,,..., N M Q y, x,,..., N M, ad, to be the traiig ad validatio sets. The Q ad P datasets are used for traiig level-0 data, ad the Q ad P datasets are used for formig the level- data. Give ResNet50 called level-0 geeralizers, traiig o the data i the traiig set Q ad P to iduce a model, for k,..., K, which are called level-0 models. The level- data assembled from the outputs of the K models is ' Q y, z,..., z I,..., zk,..., zki,..., zk,..., zki,,..., M, ' P y, z,..., z,..., z,..., z,..., z,..., z,,..., M, where zki Pki x 0 models, ad I k ki K KI is the predictio from the level- Pki x deote the probability of the ith output class, ad i the ResNet50, Pki x is the class probability of the Softmax layer's output, ad I Pki x. i Figure : The illustratio of stacked geeralizatio. 3.. Level- Geeralizers After obtaiig the level- data, we use MLR (Breima, 996) to derive from this data a model i this work. MLR which Breima (Breima, 996) used i regressio settigs, is a adaptatio of a least-square liear regressio algorithm. If the classificatio problem is with real-valued attributes, it ca be trasformed ito a multi-respose regressio problem. I the face ati-spoofig task, they are two classes, they ca be coverted ito two separate regressio problems, where the problem for class has istaces with resposes equal to whe they have class ad 0 otherwise. The liear regressio for class is simply: K k I i ki ki LR z P z () We solve this problem to choose the liear regressio coefficiets to miimize: ki y ki Pki z () j ' y, z Q k i where y is equal to whe the istace's label is correspodig with the class,s ad 0 otherwise. These regressio coefficiets ca be the weight of each level-0 model's predictio probability. I the test phase, after the fusio, the probability of each sample predictig fake or real is readjusted, so that accordig to the probability, the sample is classifyig correctly. I the experimetal stage, we compare the accuracy with the AND rule. As show i Table, our method is better tha the AND rule with a large margi. Accordig the AND rule, if both the two model's predictio is real, the the fial predictio is real, otherwise is fake. We believe the 39

4 ICPRAM 08-7th Iteratioal Coferece o Patter Recogitio Applicatios ad Methods reaso is that our method lears which model is more reliable ad which ot. Because each level-0 model is heterogeeous, for each specific sample, the classificatio accuracy rates of these models are quite differet. Therefore, the weights ca adjust the predictios so that the more wisdom decisio ca be give. Whe classifyig a ew istace x, compute LR x for the two classes, ad assig the istace to class real which has the greatest value. That is to LR x LR x, the we believe the say, if real fake sample is a real face, otherwise fake face. Figure : Examples of real access ad spoofig attempts i the CASIA-FASD database. The first row is low-quality, the secod row is middle-quality ad the last row is highquality. The first colum is geuie, ad the others form left to right are warp-photo, cut-photo ad video fraud. Figure 4: Examples of real access ad spoofig attempts i our database. The five rows are captured by differet cameras, from top to dow are Phoe Low Resolutio frot camera, Pad frot camera, Phoe Normal Resolutio frot camera, USB Low Resolutio ad USB High Resolutio. The four colums are differet type of fraud, from left to right are warp A4 photo fraud, cut eye A4 photo, warp copper photo ad video replay fraud. 4 EXPERIMENTS Our proposed method is successful o the face atispoofig data sets. We evaluated it i this sectio o two differet datasets, icludig CASIA (Zhag et al., 0), ad Replay-attack (Chigovska et al., 0). 4. Implemetatio Details Figure 3: Examples of real access ad spoofig attempts i the REPLAY-ATTACK database. The four rows from top to dow are real video, prit fraud video, mobile fraud video ad high-defiitio fraud video. Ad the first colum is adverse eviromet ad the secod colum is cotrolled. I this work, durig the traiig, we first separate the video attacks ad photo attacks ito differet sets. For each video, we selected oe frame every three frames, thus formig the traiig, validatio ad test sample sets. For each frame, the face ca be detected by usig Viola-Joes algorithm. To provide precise face locatios, we implemet the face aligmet algorithm proposed i Su et al., (04) 30

5 after a commo Viola-Joes face detectio. After obtaiig the ladmarks, their boudig box is regarded as the fial face locatio. Accordig to Yag et al. (04), beyod the covetioal face regio, the backgrouds are helpful for the classificatio as well. Therefore, we elarge the origial oes with re-scalig ratio.8. Fially, all iput images are resized to 4*4. For the CNN, we use Caffe toolbox ad adopt a commoly used structure ResNet50, which was ever used i He et al. (06). I the traiig of the ResNet50 for video attack face ad photo attack face, the learig rate is 0.000; decay rate is 0.00; ad the mometum is 0.9. Before fed ito the ResNet50, the data are first cetralized by the mea of traiig data. 4. Datasets 4.. CASIA-FASD Database The database collects 600 short videos from 50 cliets (Zhag et al., 0). For each subject, there differet quality videos are captured by differet resolutio camera. For each camera, oe real video ad correspodig three differet kids of attacks were recorded. The three kids of attacks are warped photo attack, cut photo attack ad electroic scree attack. Therefore, each subject has sequeces (3 geuie ad 9 fake oes). Three differet imagig quality coditios were recorded usig a imagig device of () High-quality, () Middle-quality, ad (3) Low-quality. Example frames from geuie ad fake videos are show i Figure. For evaluatio, the total set of videos is divided ito two ooverlappig subsets for traiig ad testig. 4.. REPLAY-ATTACK Database This database cotais 00 short videos from 50 subjects (Chigovska et al., 0), icludig both real accesses ad face spoofig attacks. Each perso i the database was recorded the videos i two illumiatio coditios: cotrolled ad adverse. A high resolutio pictures ad videos were take for each subject uder the same coditio. There are three attacks: prit attacks, mobile attacks ad highdefiitio attacks. Ad the videos were divided ito had based attacks ad fixed based attacks. Example frames from geuie ad fake videos are show i Figure 3.For evaluatio, the total set of videos is divided ito three o-overlappig subsets for traiig, developmet ad testig Our Database The database cotais 500 short videos from 00 subjects, icludig both real accesses ad face spoofig attacks. For each subject, there differet quality videos are captured by differet resolutio camera. For each camera, oe real video ad correspodig three differet kids of attacks were recorded. The three kids of attacks are warped photo attack, cut photo attack ad electroic scree attack. Example frames from geuie ad fake videos are show i Figure 4. Therefore, each subject has 5 sequeces ( geuie ad 4 fake oes). For evaluatio, the total set of videos is divided ito two o-overlappig subsets for traiig. 4.3 Experimetal Results The performace of the proposed stack geeralized based face ati-spoofig approach was evaluated o the three databases. All these results are give i Table. For a performace compariso, the results of the state-of-the-art coutermeasures ad the baselie algorithms i databases to face spoofig attacks are listed i Table. O the CASIA-FASD database, best performace i previous work was achieved by the LBPs form three orthogoal plaes (LBP-TOP) method, explorig the spatial ad temporal LBP distributios simultaeously. Our method achieved a EER of 3.4%, which is better tha the LBP-TOP method. O the REPLAY- ATTACK, our method achieved a EER of 0.3% ad HTER of 0.63% respectively, both of which are superior to the others. Besides, the performace of our method was compared with the AND rule method o the REPLAY-ATTACK, CASIA-FASD ad our database. The results are listed i Table.From the results, the proposed stacked geeralized method achieved a huge performace improvemet i liveess detectio compared with the AND rule method. O the REPLAY-ATTACK, CASIA-FASD ad our database, our method achieved a huge performace improvemet i face ati-spoofig problem. These results illustrate the effectiveess of the proposed stacked geeralized face liveess detectio approach. 3

6 ICPRAM 08-7th Iteratioal Coferece o Patter Recogitio Applicatios ad Methods Table : Compariso betwee the proposed coutermeasure ad state-of-the-art methods based o the REPLAY-ATTACK, CASIA-FASD database. Approach IQA based (Galbally ad Marcel, 04) Motio (Pereira et al., 03 ) LBP + SVM (Yag et al., 03) LBP-TOP + SVM (Pereira et al., 0) SBIQF+NN (Feg et al., 06 ) YCbCr + HSV + LBP (Boulkeafet et al., 05) LSTM (Xu et al., 06) Replayattackdev(EER)% Replayattacktest(HTER)% CASIA- FASD-test- EER (%) Our Method Table : Compariso betwee the proposed coutermeasure ad the AND rule method based o the REPLAY-ATTACK, CASIA-FASD database. Approach Replay-attackdev(EER)% Replay-attacktest(HTER)% CASIA-FASDtest(EER)% AND Rule Our Method CONCLUSIONS With the rapid developmet of face ati-spoofig techiques, the threats of spoofig attacks will also icrease i the diversity. It's hard to lear a model to detect all types of fraud. Hece, the itegratio of several coutermeasures is a promisig approach. The proposed method is a way of combiatio. Ad our method ca be easily combied with other algorithms, as log as these algorithms are helpful for liveess detectio. I future work, other advace eural etworks will be ivestigated to improve face ati-spoofig performace, such as the log shortterm memory (LSTM) etwork, which may be more effective i learig face liveess features. ACKNOWLEDGEMENTS This work was supported by the Natioal Nature Sciece Foudatio of Chia (No ), ad the Chia Scholarship Coucil (Grat No ). REFERENCES Maatta J, Hadid A, Pietikaie M., 0. Face spoofig detectio from sigle images usig micro-texture aalysis[c]// Iteratioal Joit Coferece o Biometrics. IEEE. Komulaie J, Hadid A, Pietikaie M., 03. Cotext based face ati-spoofig[c]// IEEE Sixth Iteratioal Coferece o Biometrics: Theory, Applicatios ad Systems. IEEE. Pereira T D F, Ajos A, Martio J M D, et al., 0. LBP TOP Based Coutermeasure agaist Face Spoofig Attacks [J]. We D, Ha H, Jai A K., 05. Face Spoof Detectio with Image Distortio Aalysis [J]. IEEE Trasactios o Iformatio Foresics & Security. Yag J, Lei Z, Li S Z., 04. Lear Covolutioal Neural Network for Face Ati-Spoofig [J]. Computer Sciece. Pa G, Su L, Wu Z, et al., 007. Eyeblik-based Ati- Spoofig i Face Recogitio from a Geeric Webcamera[C]// IEEE, Iteratioal Coferece o Computer Visio. Kollreider K, Frothaler H, Faraj M I, et al., 007. Real- Time Face Detectio ad Motio Aalysis with Applicatio i Liveess Assessmet [J]. Iformatio Forssssesics & Security IEEE Trasactios o. Galbally J, Marcel S, Fierrez J., 04. Image Quality Assessmet for Fake Biometric Detectio: Applicatio to Iris, Figerprit, ad Face Recogitio [J]. IEEE Trasactios o Image Processig a Publicatio of the IEEE Sigal Processig Society. Xu Z, Li S, Deg W., 06 Learig temporal features usig LSTM-CNN architecture for face atispoofig[c]// Patter Recogitio. IEEE. Wolpert D H., 99. Origial Cotributio: Stacked geeralizatio [J]. Neural Netw. Tig K M, Witte I H., 997. Stacked geeralizatio: whe does it work? [C]// Fifteeth Iteratioal Joit Coferece o Artificial Itelligece. Morga Kaufma Publishers Ic. Krizhevsky A, Sutskever I, Hito G E., 0. ImageNet classificatio with deep covolutioal eural etworks[c]// Iteratioal Coferece o Neural Iformatio Processig Systems. Curra Associates Ic. Simoya K, Zisserma A., 04 Very Deep Covolutioal Networks for Large-Scale Image Recogitio [J]. Computer Sciece. He K, Zhag X, Re S, et al., 06. Deep Residual Learig for Image Recogitio[C]// Computer Visio ad Patter Recogitio. IEEE. Breima L., 996. Stacked regressios [J]. Machie Learig. 3

7 Chigovska I, Ajos A, Marcel S., 0. O the effectiveess of local biary patters i face atispoofig[c]// Biometrics Special Iterest Group. IEEE. Zhag Z, Ya J, Liu S, et al., 0. A face at spoofig database with diverse attacks[c]// Iapr Iteratioal Coferece o Biometrics. Galbally J, Marcel S., 04. Face Ati-spoofig Based o Geeral Image Quality Assessmet[C]// Iteratioal Coferece o Patter Recogitio. IEEE Computer Society. Pereira T D F, Ajos A, Martio J M D, et al., 03. Ca face ati-spoofig coutermeasures work i a real world sceario? [C]// Iteratioal Coferece o Biometrics. IEEE. Yag J, Lei Z, Liao S, et al., 03. Face liveess detectio with compoet depedet descriptor[c]// Iteratioal Coferece o Biometrics. IEEE. Pereira T D F, Ajos A, Martio J M D, et al., 0. LBP TOP, Based Coutermeasure agaist Face Spoofig Attacks[C]// Iteratioal Coferece o Computer Visio. Feg L, Po L M, Li Y, et al., 06. Itegratio of image quality ad motio cues for face ati-spoofig [J]. Joural of Visual Commuicatio & Image Represetatio. Boulkeafet, Zielabidie, Jukka Komulaie, ad Abdeour Hadid., 05 "Face ati-spoofig based o color texture aalysis." Image Processig (ICIP), 05 IEEE Iteratioal Coferece o. IEEE. Xu Z, Li S, Deg W., 06. Learig temporal features usig LSTM-CNN architecture for face atispoofig[c]// Patter Recogitio. IEEE. Su J, We F, Wei Y, et al., 04. Face aligmet by Explicit Shape Regressio [J]. Iteratioal Joural of Computer Visio. 33