Disguised Face Identification Based Gabor Feature and SVM Classifier

Transcription

1 Disguised Face Identification Based Gabor Feature and SVM Classifier KYEKYUNG KIM, SANGSEUNG KANG, YUN KOO CHUNG and SOOYOUNG CHI Department of Intelligent Cognitive Technology Electronics and Telecommunications Research Institute 18 Gaeongno, Yuseong-gu, Daeeon, KOREA {kyekyung, kss, ykchung, Abstract: - This paper proposes the identification method of disguised faces wearing a mask or a sunglass using the SVM (Support Vector Machine) classifier and Gabor features, which are derived from eyes, nose and mouth of a face. The face features are extracted from the feature points of a face graph on a face rectangle image, which is detected by Adaboost face detection algorithm. The face graph is matched on a face image by a LMM (land mark model) algorithm using PSO (Particle Swarm Optimization) that represents whole feature points of a face image whether a part of face is hidden or not. SVM classifier distinguishes between normal face and disguised face wearing a mask or a sunglass. SVM is designed for classifying normal face from disguised face using some features on the mouth area of the face graph. The experimental result of the proposed method shows the classification performance of 96% on face image database, which have 150 normal faces, 100 disguised faces and 460 training sample faces. Key-Words: - Disguised face recognition, Gabor features, Support Vector Machine, Adaboost face detection 1 Introduction Face recognition [1-4] is an efficient and a competitive biometric technology because the face recognition does not need recognition subect s cooperation in contrast to finger print recognition and iris recognition. Face recognition technology has used to control gate access by personal identification and verification. This kind of gate access control method has great advantage for preventing criminals and tracking suspicious human using the gate access records with face images and access time in the gate. Especially, face recognition technology also can be used in identification of disguised people, wearing mask or sunglass, who tries to access a restricted area, a banking machine or a computer account. The study on access control using face recognition is mainly focused on face detection capability of normal face images, while disguised face images wearing mask or sunglass could not be detected using normal face detection algorithms. If face could not be detected, it is considered that the face does not exist in the image or the face is assumed by disguised with mask or sunglass. The disguised face is hard to detect and identify due to the lake of face features using normal technologies. The face detection algorithms [6] are made by skin color detection, PCA (Principal Component Analysis), neural network, AAM (Active Appearance Model), ASM (Active Shape Model) and Adaboost technologies. The face identification algorithms are implemented by PCA, EBGM (Elastic Bunch Graph Model), LBP (Local Binary Pattern) and other face graph matching technologies such as AAM, ASM and LMM. Because disguised face images are normally hidden a part of face region, Adaboost face detection algorithm has used that is well known for detection of occluded faces. Also face graph matching technology using LMM is good for extracting feature points of occluded faces. As a face feature, Gabor features are great for identification of face due to similar model of human being s eye model. In this paper, the identification algorithm of disguised faces wearing mask is presented using Adaboost for face detection, LMM [9] for face graph generation to extract Gabor features of faces and SVM for classification of disguised faces. Adaboost algorithm is applied to detect a face region even for the occluded face images. LMM algorithm is used for obtaining a face graph matched on face image. Gabor features are calculated from the points of face graph, which is derived by LMM optimization algorithm. Two classes of face images between normal faces and disguised faces are classified using SVM [5] classifier with Gabor features of face image. The idea of identification for disguised faces is that we derive the answer about whether the area of mouth and nose on face image is occluded by mask or not by using SVM classifier ISBN:

2 with feature values of points on the mouth and nose area of a face graph derived by LMM using PSO. The proposed method can be used in real-time application for identification of disguised people because the algorithms using in this paper are very fast. In this paper, overview of the algorithm is explained in section, face detection, face graph extraction and disguised face identification by using SVM are introduced in section 3 and experimental results are introduced in section 4. Overview of Disguised face Identification The algorithm flow of disguised face identification according to the face detection includes the face graph extraction, the Gabor feature extraction and the disguised face classification. The algorithm is consisted of three parts such as the face detection part using Adaboost, the Gabor feature extraction part using the face graph extracted by LMM, and the disguised face classification part using SVM as shown in Fig. 1. Fig. 1 An algorithm flow of disguised face identification. In the part of face detection, Adaboost algorithm is trained for positive samples and negative samples. This face detection result is used for SVM training procedures and SVM classification procedures are composed of face graph matching, Gabor feature extraction, disguised face classification. In the part of Gabor feature extraction, the initial location of a face graph on face image is derived using the rectangle area of Adaboost face detection result. LMM optimization algorithm is used to match a face graph on a face image. A standard face graph is composed of average location of feature points and average Gabor et values of each feature points. The disguised face is influenced decisively by features, which are derived from the mouth area in the LMM face graph matching result. The Gabor feature values of disguised face are calculated from the feature points including mouth area in the face image. In the part of SVM classification, the Gabor features are used for classification whether wear a mask or not. In the training step of SVM classifier, two classes of face images are used that is categorized into normal face images which does not wear mask and disguised face images which wear mask. The Gabor features of the points on mouth area are extracted from two classes of face images. The SVM is trained for classification of these two different classes of face images. The best classification hyper-plan of SVM is obtained using these two different classes of mouth features in face images. Un-known face image can be identified whether normal face or masked disguised face after implementation of disguised face identification algorithm that includes processes such as face detection using Adaboost algorithm, Gabor feature extraction on face graph, SVM classification. 3. Disguised face Identification algorithm 3.1 Face detection by MCT-Adaboost Modified MCT (Modified Census Transform) algorithm [8] is implemented based on Adaboost training procedures. In the Adaboost training step, face images wearing mask or sunglass are used for positive samples. After Adaboost training procedures, face detection algorithm is implemented for detecting face rectangle whether wearing mask or not. The face detector analyzes image patches W of size pixel. The first stage has the lowest complexity but it is able to reect more than 99% of all windows as background while retaining almost all of the face locations. Let H (Γ) be the classifier of the -th stage, which classifies the current analysis window, represented by the modified census features Γ, H ( ) x ' xw h ( ( x)) where x denotes the location within the analysis window and W W is the set of pixel-locations with an associated pixel-classifier h(x). The pixelclassifier h(x) also called elementary classifier consists of a lookup table of length 511 which is the number of the possible kernel indices of the Modified Census Transform. The lookup table holds a weight for each kernel index. A response from an (1) ISBN:

3 elementary classifier is the weight addressed by the kernel index. As the features are integer indices of the active structure kernel they are lying in the range [0..511] and are directly feed into the lookup table. Basically Adaboost searches several candidates of face rectangles in a face image that resizes face detection scales for adopting different size of faces. The average size of face rectangles are decided as a final face rectangle area of an image as shown in Fig..This face rectangle information is used for the initial location of graph matching algorithm. Positive samples Negative samples Normal face Disguised face Training samples Cascade Adaboost training Adaboost Face Detector Fig. Adaboost face detection steps. 3. Face model graph generation A face model graph [9] is made by using average point locations and average Gabor et values of each point for 10 training face graph on images where marked 44 feature points manually as shown in Fig. 3. The feature points are composed of 44 points such as13 points on face edge, 8 points on mouth area, 9 points on nose area and 14 points on eyes area as shown in Fig. 3. Fig. 3 A manually marked face graphs on face image. Gabor feature [9] vector J i ( x 0 ) is defined as follows with respect to an intensity value I (x) at pixel location x ( x, y). J ( x 0 ) I(x) ( x x d i 0 ) x k ( x) k x (exp( ))(exp( ik x) exp( )) () In order to reflect the various faces, 10 training sample face images are selected from the FERET face image with respect to gender, race and age Gabor feature vectors for a single face graph are derived with 40 Gabor ets for each 44 points. Finally the face model graph is made by average feature locations and average Gabor ets for each point. 3.3 Face graph matching and Gabor feature extraction In order to extract optimal face graph by matching face model graph on a face image, the initial location of face graph is very important. The face rectangle information of Adaboost face detection result is used for initial face graph location. Face model graph is resized and located on a face image for fitting a face graph of the face using the face rectangle information. The four points of face model graph are matched on the face rectangle edge as shown in Fig. 4 (a). The face graph is modified repeatedly by searching best graph matching result between face model graph and face graph using PSO algorithm. The face graph can be modified using model parameters such as graph size scaling, down area scaling, up area scaling and distance scaling of two eyes with respect to the center point which is defined at the middle point between two eyes. After calculating the similarity between a face model graph and a face graph with respect to Gabor et, the face graph size and location are modified using PSO algorithm to find a best matching result between two graphs. FMG FG The face graph similarity S a ( J, J ) between FMG FG face model graph J and face graph J is calculated with et value a at feature point as follows. S (J a FMG,J FG 40 ' a a / a 1 ' ) a (3) 1 1 Even some feature points of face are occluded by mask or sunglass, face graph can be extracted feature points as shown in Fig. 4(b). ISBN:

4 Uy Dy Lx Rx (a) Initial location of face graph (b) Face graph of partly occluded face Fig. 4 Examples of face graph extraction. 3.4 SVM identification of disguised faces SVM is a classifier model which is able to classify feature vectors using optimal classification hyper-plane derived from high dimensional feature space in which feature vectors are mapping. SVM is a statistical learning model which can classify two classes optimally with respect to optimal classification hyper-plane. Because the complicated feature vectors such as normal faces and disguised faces could not be separated using linear discriminate plane, in order to classify lineally, the feature vector should be mapped on high dimensional space using Kernel function. The feature points on mouth and nose area between the normal faces and masked faces have different statistical characteristics as shown in Fig. 4. SVM is trained using training face samples with masked faces and non-masked normal faces. A 1,000 dimensional feature vector is derived from 5 points on mouth and nose area with respect to 40 Gabor ets per point using face graph on face image. SVM can classify two classes according to a 1,000 dimensional feature vector of each face graph on face image. 4 Experiment MCT-Adaboost face detection algorithm is implemented to locate a face rectangle. MCT features of s single face image are extracted from 484 pixel positions in x size face image. Each feature can be defined at center pixel of a 3x3 area in face image with 511 structural elements. MCT feature vector is very short than Viola-Jones Haar features with 180,000 dimensional feature vectors. The masked face images are also included in training procedures in order to detect disguised face images. Face graph matching algorithm is implemented using 10 selected training face image samples of color FERET face database with respected to manually marked 44 feature points. The face model graph is derived from each face graph of training face images. A disguised face database is created with respect to 339 normal face images and 11 masked face images to evaluate the proposed algorithm. 1,000 dimensional SVM feature vectors are derived from 5 points of face graph for each 460 face images. SVM classifier is modeled to classify two classes between normal faces and masked face images using 1,000 dimensional feature vectors for 460 faces. The trained samples of 460 faces are perfectly classified between normal faces and masked face images. Test database is also composed of other 50 face images with 150 normal face images and 100 masked face images. Among 50 face images, 10 face images could not be detected on face detection step. So we have tested 96% of test samples on masked or not. The simulation results show that 100% of 40 face images are classified into normal face or masked face classes perfectly. Therefore, the disguised faces are classified successfully if face is detected successfully. 5 Conclusions In this paper, the disguised face identification algorithm is presented using Adaboost for face detection, LMM for face graph generation to extract Gabor features of faces and SVM for identification of masked faces. Several face databases are used for experiments such as face rectangle samples for face detection, 10 color FERET samples for face graph extraction, 460 face samples with masked and not-masked faces in SVM training and 50 face samples for disguised face identification. Experimental results show that 96% of test face samples are classified successfully whether masked faces or not. The performance is mainly related on ISBN:

5 face detection algorithm. While face is successfully detected, face image samples could be successfully identified whether masked face or not. Therefore, face graph extracting method for obtaining SVM feature vectors is the excellent method for identifying disguised faces from normal face images. Acknowledgement This work was supported by the IT R&D program of MSIP & KEIT [ ], Development of the 5-senses convergence sports simulator based on multi-axis motion platform. References: [1] M. Turk, and A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp , March [] L. Wiskott, J. M. Fellous, N. Kruger and C. Malsburg, Face Recognition by Elastic Graph Matching, IEEE Trans. on Pattern Analysis & Machine Intelligence, Vol. 19, No. 7, pp , [3] P. S. Penev and J. J. Atick, Local Feature Analysis: A General Ststistical Theory for obect Representation, Computation in Neural Systems, Vol. 7, No. 3, pp , [4] K. Etermad and R. Chellappa, Discriminant Analysis for Recognition of Human Face Images, Journal of Optical Society of America, Vol.14, pp , [5] C. J. C. Burges, A tutorial on support vector machines for pattern recognition, Data Mining & Knowledge Discovery, Vol.. No.. pp , [6] M. Yang, D. Kriegman, and N. Ahua, Detecting Faces in Images: A survey," IEEE Trans. PAMI, Vol. 4, No. 1, pp , 00. [7] R. Viola and M. Jones, Robust Real-Time Face Detection, Int'l Journal of Computer Vision, Vol. 57, No., pp , 004. [8] B. Froba and A. Ernst, Face Detection with the Modified Census Transform, Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 1-6, 004. [9] R. Ramadan and R. Abdel-kader, Face Recognition Using Particle Swarm Optimization-Based Selected Features, Int'l Journal of Signal Processing, Image Processing and Pattern Recognition, Vol., No., pp.51-65, June 009. ISBN: