A Near-Infrared Face Detection and Recognition System Using ASM and PCA+LDA

Transcription

1 2728 JOURNAL OF NETWORKS VOL. 9 NO. 10 OCTOBER 2014 A Near-Infrared Face Detection and Recognition System Using ASM and PCA+LDA Peiyi Shen 1 Liang Zhang 1 Juan Song 1 Hu Xu 1 Lianie Qin 1 Wei Wei 2 Wenzheng Zhang 3 Bin Leng 3 and Mengqi Zeng 3 1. National school of Software Xidian University Xi an P. R. China 2. School of Computer Science and Engineering Xian University of Technology PR China 3. Science and Tech. On Com. Security Lab Chengdu P. R. China Abstract Near-infrared (NIR) images have a lot of advantages which can make up the shortages of visible light (VL) images. A novel NIR detection and recognition system is presented in this paper. First a NIR imaging system is developed to provide good illumination conditions for subsequent face detection and recognition. Then Active Shape Model (ASM) method is applied to detect the features and the low-dimensional Gabor features are extracted for recognition by combining Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). The recognition rate of the system is up to 90% in a few milliseconds. Finally by using such real-time NIR face recognition system comparative results are provided from that we can seen that this system can work robustly and effectively. Index Terms NIR; ASM; Gabor; PCA; LDA I. INTRODUCTION Most current face recognition systems are based on face images captured under visible light (VL) condition which cannot provide accurate recognition due to changes of environmental illumination [1]. As the Near-infrared (NIR) images have a lot of advantages such as strong anti-interference and independence to VL source great attentions have been paid to face recognition for NIR images. Accurate face position is important to feature extraction for face recognition. During past 20 years scholars have carried out a lot of research. Active Shape Model (ASM) proposed by Cootes [2] has been a popular obect detection method. It can restrict the adustment of parameters according to the training data thereby limiting the shape change in a reasonable range. Today the methods of feature extraction and description in face recognition are divided into two categories [3]: geometrical characteristics based and statistical characteristics based. Recent years most of the proposed methods are based on statistical characteristics for example Principal Components Analysis (PCA) Linear Discriminant Analysis (LDA) Support Vector Machine (SVM) and Artificial Neural Network (ANN) and etc. Among them PCA and LDA methods based on subspace are widely used. However PCA is less sensitive to the classification information of different training samples and LDA is fully considerate of the classification information but the calculation process of LDA is too complex to ensure the accuracy. To achieve illumination invariant face recognition we present a novel solution using NIR imaging techniques in this paper. The solution consists of NIR imaging hardware and NIR-based face detection and recognition algorithms. The remainder of this paper is organized as follows. Section 2 introduces the presented imaging hardware system for producing face images of a good illumination condition. Section 3 describes the method of face detection and recognition. Experiments and results are provided and analyzed in Section 4 prior to discussion and conclusion in Section 5. II. NIR IMAGING SYSTEM The NIR imaging system is developed to overcome the problem arising from uncontrolled environmental lights so as to produce face images of a good illumination condition for face detection and recognition. The key problems to capturing NIR images with good illumination condition are 1) the design of camera module and 2) the arrangement of the LED lamps. To solve the first problem we choose LED lamps which can emit 940nm NIR lights as active radiation source which are strong enough to produce a clear frontal-lighted face image without causing disturbance to the human eyes. As shown in Figure 1(b) we place a optical filter on the camera to cut off the visible environmental lights while allowing most of the 940nm NIR light to pass. D Cameras A: the lamp panel B: the touch screen C: IC/ID card reader P C B (1) (2) NIR camera module Color camera module spacer Baffle plate Lamp panel Optical filter Lens shade Figure 1. The structure of the NIR imaging system. In the figure (1) the front view of the system. In the figure (2) the side view of the system doi: /nw

2 JOURNAL OF NETWORKS VOL. 9 NO. 10 OCTOBER For the purpose of providing homogeneous light on the face it is important to carefully design the layout of the LEDs. After simulation experiments the solution of the second problem is to choose the number of LEDs as 24 to arrange LEDs around the cameras as shown in Figure 2(a) and to fix each LED lamp with an angle ϕ as shown in Figure 2(b). With the design above there exists an image area with homogeneous illumination about 60-80cm distance from the camera. (1) Figure 3 shows example images of one person taken by the color camera and the NIR camera with different direction of environmental illumination. It can be seen that the VL images are intensely influenced by the lighting condition which is adverse to the further face recognition; while the NIR images are robust to the changes of illumination with the NIR LEDs lighting in the front and the filter to cut off most of the visible lights. In this way the NIR images produced by the equipment can provide a good foundation of face recognition. III. DESCRIPTION OF ALGORITHMS A. ASM Method The primary task of developing a real-time face detection and recognition system is to find a suitable face detection algorithm. Active Shape Model (ASM) an image searching method based on statistical model includes four steps [4]: labeling and aligning the training set building the global shape model and the local grey model and searching the profile of the obect iteratively. The advantage of ASM method is that it can restrict the adustment of parameters according to the training data thereby limiting the shape change in a reasonable range. The detailed procedures are described in Figure 4. (2) Figure 2. The architecture of the NIR lighting equipment. In the figure (1) 24 LEDs which can emit 940nm NIT are placed in the way that the NIR light can be proected onto a person s face equally. In the figure (2) each LED lamp has a fixed angle ϕ After analysis of the problems we design the hardware structure of the NIR lighting system including: the NIR lighting equipment cameras image sensor (CMOS OV7725) and CPU (OMAP3530). The NIR lighting equipment can provide a stable infrared illumination condition. There are two cameras to collect both the VL and NIR images which can be transmitted to the CPU through an image sensor and processed using algorithms presented in Section 3. The structure of the system is shown in Figure 1. The NIR lighting equipment can provide a maximum frame number of 30 per second and produce images. (1) The color images Training set Testing image Labeling and aligning the training set Finding the approximate face location Building the global shape model Initializing the shape model Building the local grey model N Searching the best fit for each feature point Figure 4. The process of ASM method Y Converge? Updating the model parameters The best face profile The shape model of 2-Dimensional contains 68 feature points to represent the Point Distribution Model (PDM) of human face. After labeling the training samples manually we align all the samples through similarity transformation and then get the average shape. One disadvantage of ASM is that it is more dependent on the initial point set so the initial position of the model is very important. It is usual to get an approximate face location through some other image processing methods. A Haar-feature cascade classifier [5] provided by OpenCV is used to find a rough face location in this paper. As shown in Figure 5 the initial shape is obtained after similarity transformation from the rough face location. In the meantime the person should always stand ust before the image acquisition equipment. (2) The NIR images Figure 3. The VL images and their corresponding NIR images are taken by the present NIR imaging system. The influence of the environmental lighting on the VL images is obvious oppositely the NIR images can minimum the effect of the bad lighting condition Figure 5. The model's initialization. Left: the red box shows the rough face location. Right: the initial shape

3 2730 JOURNAL OF NETWORKS VOL. 9 NO. 10 OCTOBER 2014 Another disadvantage of ASM is that it is easy to fall into local extremum during the searching process. To avoid this we do some improvements for searching method. For each feature point we select the candidate points along its normal direction. Through calculating the Mahalanobis Distance between the candidate points and feature points we can find the best fit to the feature points. The Mahalanobis Distance can be described as [6]: Training Recognizing Face Database Testing image Gabor transform Gabor transform and PCA PCA and LDA Proect on subspace Extracting Feature and reducing dimensionality Classify by the nearest-neighbour rule Pattern recognition Figure 7. The flow chart of face recognition The result f i y i y s y i y (1) T 1 ( ) ( ( ) ) ( ( ) ) where y and s are the local grey model of the feature point and y(i) is the normalized grey differential vector of the candidate point i calculated by the same way to the grey model. This distance reflects the similarity of the sample point to the feature point. Figure 6 shows an iterative process for ASM to search the best fit to the feature points. After face detecting using ASM method we can get an accurate face location as the red box shown in Figure 6(f) for the further use of face recognition. (1) (2) (3) (4) (5) (6) Figure 6. An Iterative Process of improved ASM: 1) initial location 2) after 2 iterations 3) after 4 iterations 4) after 8 iterations 5) after 16 iterations 6) the detecting result of ASM method B. PCA+LDA Method Based on Gabor Feature Face recognition is a process to compare the target face image with the given face in database whose purpose is to reach an identification conclusion. The core of this process is extracting suitable feature and choosing proper matching strategy. On the foundation of the research of VL image based face recognition and with the consideration of the calculation capability of the presented embedded system we put forward a new recognition method suitable to NIR image based face recognition. The first step is to extract the Gabor feature of the face image and the following is to identify the human face with the combinative method of PCA and LDA. The process of face recognition is shown in Figure 7. The NIR images in face database which is under constructing are all collected through the NIR lighting equipment developed by our lab. At the current stage the database includes 80 persons and each one has at least 80 representative pictures for example different expressions face rotating and wearing eyeglasses etc. C. Gabor Feature Extraction Gabor feature extraction is a method based on Gabor Filter which is widely applied in computer vision and image processing fields because of the high resolution in time and frequency domain. The kernel function of Gabor Filter is defined as follows [7]: ( k 2 z /2 ) izk ( /2) k e [ e e ] (2) 2 where and define the orientation and scale of the Gabor kernels z ( x y) denotes the norm operator and the definition of the wave vector k is k cos k. Where k k sin = k max / f and /8. k max is the maximum frequency and f is the interval factor between kernel functions in frequency domain. We choose 5 different scales {01234} and 8 different directions μ { } and let σ=2π k max = /2 f = 2. Substitute the value of these parameters into the equation and then we can get a set of kernel functions () z corresponding to different direction and scale factors. Given a face image I( x y ) the Gabor features are extracted by convolving I( x y ) with () z as follows [8]: O ( z) I( z) ( z) (3) O () z is the high-dimensional Gabor features of face image I( x y ) after Gabor filtering. D. Face Recognition Using PCA+LDA Because the calculation cost of the high-dimensional data is adverse to the performance of the real-time system it is necessary to reduce the dimensionality of the Gabor features obtained above simultaneously retaining most of the original feature information. Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) are two common dimensionality reduction methods. By the analysis of the advantages and disadvantages of PCA and LDA [9] we can conclude that PCA method is less sensitive to the classification information of different training samples while LDA is fully considerate of the classification information but its calculation process is too complex to

4 JOURNAL OF NETWORKS VOL. 9 NO. 10 OCTOBER ensure the accuracy. Therefore a mixed method combining PCA and LDA is adopted in this paper. The steps of PCA+LDA [10] method are as follows: 1) Reduce the dimensionality of original data by PCA to achieve a low-dimensional feature space of PCA; 2) Calculate the within-class scatter matrix and the between-class scatter matrix of LDA in PCA subspace; 3) Calculate the best classification subspace of LDA and fuse the subspace of LDA and PCA; 4) Proect training samples and testing images separately onto the fusion subspace get the recognition features and accomplish face recognition using the Nearest Neighbor (NN) Rule. The NN Rule is based on Cosine Similarity which is defined as [11]: T ( ) x y cos xy (4) x y where vector x and y represent the two classes remaining to be compared. The smaller ( xy ) cos is the more similar the two classes are. Figure 9. Face detection results of ASM: the red blocks show face rectangle and the green gridding shows feature points on the profile of the whole face 2) Face recognition--algorithm evaluation on self-built database In experiment 2 we set up the subset of 50 persons from the whole face database we have build with the training number of each person increasing from 1 to 7 images and the training face images normalized into size. The testing set has 2500 images in total. No images in the testing set are included in the training set. Based on these a comparison among PCA LDA and the presented PCA+LDA is done. (1) The original images (2) The matching faces in database (3) The matching scores Figure 8. The recognition result: 1) the green frames show the detected faces by ASM 2) the best matching faces of the upper images 3) the matching scores between upper and lower images in the same column Figure 8 shows the recognition results using the PCA+LDA method. It can be seen from the matching scores that the NN Rule based on Cosine Similarity can separate among the different classes well. The recognition rate of the algorithm can reach up to 90% with different indoor light conditions and training faces of size. IV. EXPERIMENTS AND RESULTS Based on the presented hardware and algorithms a real-time NIR based face recognition system is built. In the following we evaluate the performance of the system with three experiments. 1) Face detection--asm method As shown in Figure 9 the presented ASM could adapt to various situations such as rotating shaping different expressions and wearing glasses. The average detecting time in a image for ASM is 65ms. Then it can be concluded that ASM is a suitable face detection algorithm for NIR images. The recognition rate The numbler of training samples PCA PCA+LDA Figure 10. Performance with different number of training samples. As shown in Figure 10 no matter how many the training samples are the recognition rate of PCA+LDA is higher than that of PCA or LDA. Along with the increase of the training samples the recognition rate of all the algorithms is on the rise. When the training number of each person increases to 6 the recognition rate of presented PCA+LDA reaches the highest recognition rate in this experiment. 3) Face recognition--algorithm evaluation on PolyU Near-Infrared face database (PolyU-NIRFD) In order to evaluate the performance of the face recognition algorithms on a much bigger subset than experiment 2 we do experiment 3 using the PolyU Near- Infrared face database (PolyU-NIRFD) [12] which was constructed by the Hong Kong Polytechnic University. The database contains images from 350 subects each contributing about 100 samples with variations of pose expression focus scale and time etc. Figure 11 shows sample images of one person in the database [13-17]. LDA

5 2732 JOURNAL OF NETWORKS VOL. 9 NO. 10 OCTOBER 2014 Figure 11. Parts of NIR face images of a subect in PolyU-NIRFD In this experiment we extract 200 subects from PolyU-NIRFD with 6 training samples of each subect. The testing set contains 80 images of each person in a total of As shown in Figure 12 the recognition rate of the PCA+LDA method in experiment 3 is 89% which proves that the presented algorithm works well on other NIR face database. conditions of VL eliminating influence of VL and NIR light. It can be seen that unfavorable lighting is obvious under VL condition while the changes of environmental illumination is almost eliminated under NIR light condition [18-25]. The training set has 30 persons in which each person has 6 images of size for training. We set up the real-time system in the lab and do experiments under three different cases respectively. During the experiment people in the training set stand before the image acquisition equipment in order. The experimental result is shown in Table 1. We can see that the presented algorithms under NIR condition can well satisfy the requests of real-time system. TABLE I. ANALYSIS ON EFFECTS OF DIFFERENT ILLUMINATION CONDITIONS Situations Detection timerecognition time recognition rate Case A Case B Case C 60ms 73ms 65ms 78ms 90ms 75ms 78% 87% 90% The recognition rate PCA LDA PCA+LDA Figure 12. Performance of the algorithms on PolyU-NIRFD 4) The performance of the real-time system-- comparison among different illumination conditions V. DISCUSSION AND CONCLUSION In this paper a real-time NIR face recognition system using ASM for detection and PCA+LDA for recognition has been proposed. The novel NIR lighting system is developed to solve the problem of uncontrolled environmental lighting. An accurate face location is obtained by the ASM method. By combining PCA and LDA suitable Gabor features are extracted thus the calculation cost is reduced and the face recognition rate is increased. Experimental results show the robustness and effectiveness of the presented system. The proposed system still exist some shortages. Specular reflection on eyeglasses under NIR condition is a limitation for face recognition. The algorithms presented are practical but not optimal. Our further work is to solve these problems. (1) Case A: training images captured by color camera (2) training images captured by color camera and eliminating the influence of VL (3) training images captured by NIR camera Figure 13. Training samples under different conditions: images in training set are all captured by the presented hardware device and furthermore we conduct homomorphic filtering and histogram equalization on images in Case B Figure 13 shows parts of training images in three cases to compare the performance of the system under ACKNOWLEDGMENT This work is partially supported by the Open Proects Program of National Laboratory of Pattern Recognition. The Proect is partially supported by Natural Science Basic Research Plan in Shaanxi Province of China (Program No. 2010JM8005). This program is also supported by Scientific Research Program Funded by Shaanxi Provincial Education Department (Program No. 2013JK1139) and Supported by China Postdoctoral Science Foundation (No. 2013M542370) and supported by the Specialized Research Fund for the Doctoral Program of Higher Education of China (Grant No ).And this proect is also supported by NSFC Grant (No No No No No No No ) by 863 Program (2013AA014601) by shannxi Scientific research plan (2013K06-09). REFERENCES [1] Stan Z. Li Ru Feng Chu Illumination Invariant Face Recognition Using Near-Infrared Images IEEE

6 JOURNAL OF NETWORKS VOL. 9 NO. 10 OCTOBER transaction on pattern analysis and machine intelligence vol. 29 no. 4 pp [2] T. F. Cootes C. J. Taylor Active Shape Models--Their Training and Application Computer Vision and Image Understanding vol. 61 no. 1 pp [3] Qing Shan Liu Han Qing Lu A Survey: Subspace Analysis for Face Recognition Acta Automatica Sinica vol. 29 no. 6 pp [4] Jie Zhu Zhen Min Tang ASM and Color Gabor Features for Facial Feature Extraction Computer Science vol. 37 no. 4 pp [5] Chao Yan Yuan Qing Wang Infrared face detection based on real Adaboost algorithm and Cascade structure Laser and Infrared vol. 39 no. 11 pp [6] Yan Peng Sun Rong Fu Face feature points detection based on refined ASM Computer Engineering and Applications vol. 44 no. 10 pp [7] Chen g Jun Liu Harry Wechsler Gabor Feature Based Classification Using the Enhanced Fisher Linear Discriminant Model for Face Recognition IEEE Transactions on Image Processing vol. 11 no. 4 pp [8] Dong Li Xu Dong Xie Gabor Boost Linear Discriminant Analysis for Face Recognition ICME pp [9] Aleix M. Martinez Avinash C. Kak PCA versus LDA IEEE transaction on pattern analysis and machine intelligence vol. 23 no. 2 pp [10] Shun Gang Hua Yu Zhou Thermal Infrared Face Image Recognition Based on PCA and LDA Pattern Recognition and Artificial Intelligence vol. 21 no. 2 pp [11] Rui Guo Shu Ling Zhang Study of Feature Extraction and Similarity Match Method on Face Recognition Computer Engineering vol. 32 no. 11 pp [12] Baochang Zhang Lei Zhang Directional Binary Code with Application to PolyU Near-Infrared Face Database Pattern Recognition Letters vol. 31 no. 14 pp [13] Wei Wei Bin Zhou Features Detection Based on a Variational Model in Sensornets International Journal of Digital Content Technology and its Applications ISSN: DOI: /dcta. vol4. issue7. 11 Volume 4 Number 7 Octber 2010 On page(s): [14] Wei Wei Hui Yang etc Queuing Schedule for Location Based on Wireless Ad-hoc Networks with D-Cover Algorithm International Journal of Digital Content Technology and its Applications ISSN: DOI: /dcta. vol5. issue (1) January 2011 on page(s): [15] Jiang Jinlin Wei Wei Many-facet Rasch Model s Application in the Evaluation of Test Validity International Journal of Digital Content Technology and its Applications ISSN: [16] Wei Wei Liang Jun The Integration of p-laplace Model Certifiable Protocols with ID-based Group Key in WSNs JDCTA [17] FENG Lei Wei Wei "Research of PSO/Genetic Algorithms and Development of its Hybrid Algorithm" JDCTA 2012 (EI Journal). [18] Yuni Wang Hai-Chao Han Jack Y. Yang Merry L Lindsey Yufang Jin. A conceptual cellular interaction model of left ventricular remodelling post-mi: dynamic network with exit-entry competition strategy. BMC Systems Biology (Suppl 1): 1-10 [19] Yuni Wang Tianyi Yang Yonggang Ma Ganesh V Halade Jianqiu Zhang Merry L Lindsey Yu-Fang Jin. Mathematical modeling and stability analysis of macrophage activation in left ventricular remodeling postmyocardial infarction. BMC Genomics : 1-8. [20] Tianyi Yang Ying A Chiao Yuni Wang Andrew Voorhees Hai-Chao Han Merry L Lindsey Yu-Fang Jin. Mathematical modeling of left ventricular dimensional changes in mice during aging. BMC Systems Biology : [21] Yuni Wang Philip Chen Yufang Jin Traectory planning for an unmanned ground vehicle group using augmented particle swarm optimization in a dynamic environment in: IEEE International Conference on Systems Man and Cybernetics San Antonio TX USA Oct 2009 San Antonio pp [22] Yuni Wang Yufang Jin Yonggang Ma Halade V Halade Merry L Lindsey Mathematical modeling of macrophage activation in left ventricular remodeling post-myocardial infarction in: Genomic Signal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop on: San Antonio TX USA 4-6 Dec 2011 pp [23] Nguyen Nguyen. Xiaolin Zhang Yuni Wang Hai-Chao Han Yufang Jin Galen Schmidt Richard A Lange Robert J Chilton Merry L Lindsey Targeting myocardial infarction-specific protein interaction network using computational analyses in: Genomic Signal Processing and Statistics (GENSIPS) 2011 IEEE International Workshop on: San Antonio TX USA 4-6 Dec 2011 pp [24] Wenting Zha Junyong Zhai Shuming Fei Yuni Wang. Finite-time stabilization for a class of stochastic nonlinear systems via output feedback. ISA transactions 2014 (In Press) [25] Yi Wang Cheng Gong Baoku Su Yuni Wang. Delaydependent robust stability of uncertain TS fuzzy systems with time-varying delay. International Journal of Innovative Computing Information and Control (9):