International Journal of Advanced Biotechnology and Research (IJBR) ISSN 0976-2612, Online ISSN 2278 599X, Vol-7, Special Issue-Number5-July, 2016, pp1283-1291 http://www.bipublication.com Research Article Vahid Ghanbari 1 and Hamid Sadeghi 1 1 Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran. ABSTRACT Face recognition is one of the active fields in computer vision. The most important aspect of face recognition system is the facial feature description. The extracted features should be robust against pose and illumination and expression variations. Local binary patterns are one of the Well-known texture descriptors which can be utilized for facial feature extraction. In this paper, an improvement method for this descriptor is proposed which is based on decorrelation of pixel information to reduce the lost information imposed by the descriptor. The effectiveness of our algorithm is proved by extensive experiments performed on three databases. The results show that beside an improvement in overall recognition rate, the robustness of descriptor is increased against face variations, although it has low computational complexity. Key words:- Face recognition, extraction, Local Binary Patterns (LBP), Decorrelation, Markov process INTRODUCTION Face recognition is an established field of artificial intelligence with the wide variety of applications. Numerous commercial face recognition systems confirm vast progress in this field. Despite this progress, face recognition is still an active research topic in computer vision. The reason is that face recognition algorithms mostly perform well under constrained conditions, but have poor performance under unconstrained situations like pose and illumination and expression variations. So current researches are mostly focused on increasing the robustness of face recognition algorithms against these mentioned factors. The most important part of face recognition system is facial feature extraction. If weak features are extracted, even the best classifiers will not reach high recognition rate. On the other hand if redundant features are extracted, the performance and generalization of the algorithm will decrease. Facial features should have the following characteristics: 1) extracted features of the same person under different conditions should have low variations, 2) extracted features from different people should have large difference, 3) feature extraction process should have low complexity and the feature vector length must be as small as possible to occupy lower space in memory and enable the system for fast classification. These characteristics empowers the system for fast and accurate recognition. Unfortunately feature extraction algorithms cannot achieve all of these goals at the same time. Methods which extract robust features have mostly high computational complexity and large feature vector, hence perform slowly. On the other hand methods which perform fast, do not have acceptable accuracy. The goal of this paper is to introduce and develop a feature extraction method to
improve the mentioned characteristics. Local binary pattern (LBP) is one of the feature extraction methods introduced in [13] and is used for different tasks such as texture description and has good performance, but has low recognition rate in face recognition. In this paper, a method for decorrelating the information of extracted features by LBP is introduced which improves its recognition rate, as well as increasing its robustness against face appearance variations. This method which is called Decorrelated Local Binary Pattern (DLBP) has low computational complexity and its performance is tested on various datasets. The rest of the paper is structured as follows: Section II briefly discusses the related works. Section III details the DLBP feature extraction method. Section IV explains experimental results for face recognition task and compares the proposed method with other methods. Finally section V concludes the paper. Related work There are numerous feature extraction methods in face recognition field. Subspace-based methods such as eigenfaces [17] and Fisherfaces [2] and methods based on local appearance features [1, 9] are some of these methods. In [7] global and local approaches are compared and it is shown that local approaches outperform global methods. Also methods based on graph matching such as EBGM [19] have been proposed, but their application is limited because of inherent high computational complexity. Based on the goal of this paper, to find an efficient and accurate feature extraction method, local feature extraction methods are reviewed in the following. One of the most successful methods is Local Binary Pattern (LBP) [1]. This approach was introduced in [13] for the first time and had good results in texture description. Later several variants of LBP have been presented and successfully applied to different applications, such as Three-Patch LBP (TPLBP), Four-Patch LBP (FPLBP) [20], Local Ternary Patterns (LTP) [16], and so on. But all of these methods have higher computational complexity and larger feature vector compared to original LBP operator. Some of the other methods which have successful results in capturing edge and local shape information are Scale-Invariant Transform (SIFT) and Histogram of Oriented Gradient (HOG) [4], although these methods have lower recognition rates compared to LBP operator [11]. In [18] methods based on edge extraction and texture description are combined and one of the most powerful appearance features are extracted which is called Patterns of Oriented Edge Magnitude (POEM). This feature extraction procedure has high recognition rate and large feature vector. 3. Decorrelated Local Binary Pattern (DLBP) Original LBP operator was introduced in [13] as a powerful texture description method. This operator generates a binary number for each pixel according to the labels of eight neighbor pixels. The labels are formed by thresholding the neighbor pixel values by the center pixel value. Afterwards these labels are concatenated circularly to form an eight bit number. This process is shown in Figure (1). Fig (1): Original LBP operator After labelling pixels by LBP operator, the histogram of labels is defined using Equation (1). H i I( fi ( x, y) i), i 1,..., n 1 x, y (1) Where n is the number of generated labels by LBP and the function I is defined in Equation (2). 1 A is True I(A) 0 A is False (2) Vahid Ghanbari and Hamid Sadeghi 1284
A limitation of the original LBP operator was small neighbor size and the lack of control on large images. Therefore an extension of this operator proposed in [14] which defines a neighborhood of radius R pixels operating on P pixels. This operator is shown by LBP P,R and generate 2 P distinct numbers according to 2P different pattern of pixels. Figure (2) shows how neighbor pixels are chosen for three different radiuses. Fig (2): LBP operator for different radiuses and neighbor numbers [1]. To describe the improvement procedure on LBP operator, firstly the original operator is partitioned into two concatenated stages. In the first stage which is called Comparison, the center pixel is subtracted from neighbor pixels. In the second stage which is called Quantization, the results of the previous stage are compared with zero and the numbers which are bigger than zero are assigned value 1 and the numbers which are less than zero are assigned value 0. Finally by concatenating these quantized numbers circularly, the label of center pixel is formed. This process is shown in Figure (3). In quantization step the data is compressed in order to assign a label to the center pixel, so part of the image information is lost in this step. This quantization is performed on neighbor pixels, so their information is highly correlated and the correlation increases as the radius decreases. If it is possible to disjoint the pixels information before quantization step, it is expected that less information is lost and the descriptive power of the operator will increase. To this end it is possible to use whitening method. In whitening method, the covariance matrix of all feature vectors is calculated. Then using eigenvector decomposition, the eigenvalues and eigenvectors of the covariance matrix are obtained [3] and the final feature vector is calculated by projecting the original feature vector on these eigenvectors. In LBP operator if a data vector is defined for each pixel after the comparison step in location (x,y) by V (x,y) and the eigenvectors matrix is denoted by Q, the final whitened data vectors are calculated using Equation (3). T W Q V (3) ( x, y) ( x, y) These whitened data vectors W (x,y), have decorrelated information and if the quantization step is performed on these data vectors the lost information is decreased. However in order to calculate the covariance matrix of data vectors, for each image all the data vectors must be calculated to obtain their covariance matrix. This process need much computational time and will slow down the resulting operator. To resolve this problem the covariance matrix can be estimated using Markov process. Fig (3): The LBP operator decomposed into two subsections. As stated in [12], an image can be considered as a first-order Markov process, so by using this theorem the covariance matrix of data vectors V (x,y) can be estimated. The covariance matrix of a patch of image of size z z which is denoted by C can be calculated using Equation (4). Vahid Ghanbari and Hamid Sadeghi 1285
C 1,1 1,2 2 1,z 2,1 2,2 2 2,z 2 2 2 2 z,1 z,2 z,z (4) Where i, j is the covariance between pixels in locations x i and x j. If the image is considered as a first-order Markov process, these covariance values can be modeled by a Correlation Coefficient and pixels distances as in Equation (5). i j i, j x x (5) As neighbor pixels are highly correlated, the value of is 0.9 in this work. As a conclusion, for a specified set of parameters of LBP operator the values of i, j can be calculated and the C matrix will be obtained. Performing eigenvector decomposition on C, the Q matrix can be calculated. As a result of this procedure there will be no need to calculate the covariance matrix for each input image and the speed of the new operator will be almost the same as the original LBP operator. The procedure of extracting Decorrelated Local Binary Pattern is summarized in figure (4). After applying the DLBP operator on every pixel of the input image, a histogram of generated labels is calculated as the feature vector of the whole image. In order to increase the descriptive power of the operator, the input image can be segmented into smaller sections and concatenate the histograms of all the sections. This process is shown in Figure (5). An example of applying the original LBP operator and the DLBP operator and extracting the final histogram on face images are shown in Figure (6) and (7), respectively. Fig (5): Applying texture descriptor on image segments in order to increase the recognition rate In the next section, experimental results of comparing the proposed method with the stateof-the-art methods are presented. Fig (4): Procedure of extracting Decorrelated Local Binary Pattern Vahid Ghanbari and Hamid Sadeghi 1286
Fig (6): The result of original LBP operator. Left: input image. Middle: resulting image after applying LBP operator. Right: histogram of the generated labels by considering the whole image as one segment using Equation (1) Fig (7): The result of DLBP operator. Left: input image. Middle: resulting image after applying DLBP operator. Right: histogram of the generated labels by considering the whole image as one segment using Equation (1) 4. Experiments To evaluate the proposed method compared with other methods, decorrelated LBP along with basic LBP [1] and combined POEM method [18] are tested in this section. For this reason, recognition rate, feature extraction time, and feature length of these methods are compared. Moreover, to evaluate the robustness of the algorithms against such variation as pose, expression, and illumination, several datasets are utilized in the experiment. In this section, at first, the datasets are described. Then, the experimental results on these dataset are compared with the mentioned methods. 4.1 Datasets Asian dataset: Asian Face Image DataBase PF01 [5] contains 107 Asian subjects (56 males and 51 females). Each subject has 17 images: 1 frontal image, 4 images with varied illumination, 8 images for pose variations, and 4 images with facial expression. The dataset has different variations make it useful for evaluation of face recognition algorithms under these conditions. For this purpose, the frontal image of each subject is used as gallery and the other ones are used as probe images. Some examples of these images are shown in Figure (8). Vahid Ghanbari and Hamid Sadeghi 1287
Fig (8): Example images of Asian dataset JAFFE dataset: JAFFE (The Japanese Female Facial Expression) dataset contains 213 images of 10 Japanese females with different facial expressions. Each subject has 3 or 4 images for six basic facial expressions as well as neutral one. The image resolution is 256 256 pixels in frontal view. The dataset is proposed for evaluation of facial expression recognition algorithms [10]; however, it can be used in face recognition algorithm. Some samples of the images are shown in Figure (9). Yale-B dataset: this dataset contains 2432 images of 38 subjects captured under 64 illumination conditions. All images were manually aligned and cropped in 192 168 pixels [6, 8]. Therefore, the dataset can be used to evaluate different feature extraction methods against illumination variations. Figure (10) shows some sample images from Yale-B dataset. 4.2 Experimental setup Due to different image sizes in the described datasets, the images should be normalized to a fixed size. Consequently, all images are aligned and resized into 180 160 pixels based on two eye locations. Then, in each dataset, one frontal image from each subject is used as gallery image and the other ones (with different variations) are used as probes images. For feature extraction, each image is divided into several local regions and LBP [1], POEM [18], and the proposed decorrelated LBP operators are applied to each region. The histogram of the patterns of each operator in the regions is calculated. The calculated histograms are concatenated in the final feature vector. Each facial region is 30 40 pixels in the experiments. 2 To compare two feature vector, Chi-square ( ) [1], which is a suitable histogram based distance method, is employed in this paper as Equation (6). 2 2 xi y (6) i X, Y x y i i i Where X and Y are two feature vectors and i shows the ith bin of histogram. Chi-square distance of each probe image to all gallery images are calculated, and the nearest neighbor gallery image is used as the match result. 4.3 The results on different datasets As mentioned previously, Asian dataset has various illumination, pose, and expression condition. The results of three algorithms are summarized in Table (1). The experiments were performed on an Intel Core i5-2.6 GHz with 6GB RAM computer on Matlab. Fig (10): Example images of Yale-B dataset Vahid Ghanbari and Hamid Sadeghi 1288
Fig (9): Example images of JAFFE dataset Table (1) the results of different methods on Asian dataset method length Recognition rate (%) extraction time (ms) 6.9 90 9 LBP [1] 6144 94.12 POEM [18] 5664 91.65 DLBP 6144 94.41 As can be seen from Table (1), DLBP operator has the highest recognition rate compared with the other ones on Asian dataset. Moreover, the feature extraction time of DLBP is not significantly increased compared with LBP. JAFFE dataset is used to investigate the mentioned method against facial expression. The results of three algorithms on JAFFE dataset are summarized in Table (2). Table (2) the results of different methods on JAFFE dataset method length Recognition rate (%) extraction time (ms) 7 92 9 LBP [1] 6144 96.55 POEM [18] 5664 93.59 DLBP 6144 97.04 As can be seen from Table (2), the proposed method has the highest robustness against facial expressions. POEM has the most computation time and the lowest recognition rate in this experiment. As mentioned previously, Yale-B dataset is used to evaluate three methods against illumination variations. The results of three algorithms on Yale-B dataset are summarized in Table (3). As can be seen from Table (3), POEM has the highest computational time compared with the other methods. DLBP has the highest recognition rate and an acceptable feature extraction time on this dataset. Table (3) the results of different methods on Yale-B dataset method LBP [1] POEM [18] DLBP length 6144 5664 6144 Recognition rate (%) 88.72 92.06 95.36 extraction time (ms) 8 94 10 5. CONCLUSION In this paper, a method is proposed to increase the accuracy and robustness of face recognition algorithm against pose, illumination and expression variations. This method is obtained by an alteration in the LBP operator which reduces the lost information in the quantization step of LBP by removing the correlation between adjacent pixels. Experimental results on various Vahid Ghanbari and Hamid Sadeghi 1289
datasets prove this claim that the proposed method increases recognition rate in addition to increased robustness against pose, illuminations and expression and has low computational complexity compared to state-of-the-art algorithms. For future work, decorrelation process can be deployed for other feature extraction methods to increase their accuracy and robustness. 6. REFERENCES 1. Ahonen, T., Hadid, A., & Pietikäinen, M. (2004). Face recognition with local binary patterns. In Computer vision-eccv 2004 (pp. 469-481). Springer Berlin Heidelberg. 2. Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 19(7), 711-720. 3. Bishop, C. M. (2006). Pattern Recognition. Machine Learning. 4. Déniz, O., Bueno, G., Salido, J., & De la Torre, F. (2011). Face recognition using histograms of oriented gradients. Pattern Recognition Letters, 32(12), 1598-1603. 5. Dong, H., & Gu, N. (2001). Asian face image database PF01. In Technical Report. Pohang University of Science and Technology. 6. Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 23(6), 643-660. 7. Heisele, B., Ho, P., Wu, J., & Poggio, T. (2003). Face recognition: component-based versus global approaches. Computer vision and image understanding, 91(1), 6-21. 8. Lee, K. C., Ho, J., & Kriegman, D. J. (2005). Acquiring linear subspaces for face recognition under variable lighting. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27(5), 684-698. 9. Liu, C., & Wechsler, H. (2002). Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. Image processing, IEEE Transactions on, 11(4), 467-476. 10. Lyons, M. J., Budynek, J., & Akamatsu, S. (1999). Automatic classification of single facial images. IEEE Transactions on Pattern Analysis & Machine Intelligence, (12), 1357-1362. 11. Meyers, E., & Wolf, L. (2008). Using biologically inspired features for face processing. International Journal of Computer Vision, 76(1), 93-104. 12. Murphy, K. P. (2012). Machine learning: a probabilistic perspective. MIT press. 13. Ojala, T., Pietikäinen, M., & Harwood, D. (1996). A comparative study of texture measures with classification based on featured distributions. Pattern recognition, 29(1), 51-59. 14. Ojala, T., Pietikäinen, M., & Mäenpää, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(7), 971-987. 15. Shu, C., Ding, X., & Fang, C. (2011). Histogram of the oriented gradient for face recognition. Tsinghua Science & Technology, 16(2), 216-224. 16. Tan, X., & Triggs, B. (2010). Enhanced local texture feature sets for face recognition under difficult lighting conditions. Image Processing, IEEE Transactions on, 19(6), 1635-1650. 17. Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of cognitive neuroscience, 3(1), 71-86. 18. Vu, N. S., & Caplier, A. (2012). Enhanced patterns of oriented edge magnitudes for face recognition and image matching. Image Vahid Ghanbari and Hamid Sadeghi 1290
Processing, IEEE Transactions on, 21(3), 1352-1365. 19. Wiskott, L., Fellous, J. M., Kuiger, N., & Von Der Malsburg, C. (1997). Face recognition by elastic bunch graph matching. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 19(7), 775-779. 20. Wolf, L., Hassner, T., & Taigman, Y. (2008). Descriptor based methods in the wild. In Workshop on faces in'real-life'images: Detection, alignment, and recognition. Vahid Ghanbari and Hamid Sadeghi 1291