Wavelet Based Statistical Adapted Local Binary Patterns for Recognizing Avatar Faces

Transcription

1 Wavelet Based Statistical Adated Local Binary atterns for Recognizing Avatar Faces Abdallah A. Mohamed 1, 2 and Roman V. Yamolskiy 1 1 Comuter Engineering and Comuter Science, University of Louisville, Louisville, KY USA 2 Deartment of Mathematics, Menoufia University, Shebin El-Koom, Menoufia Egyt {aamoha04, rvyam01}@louisville.edu Abstract. In this aer, we roose a novel face recognition techniue based on discrete wavelet transform and Local Binary attern (LB) with adated threshold to recognize avatar faces in different virtual worlds. The original LB oerator mainly thresholds ixels in a secific redetermined window based on the gray value of the central ixel of that window. As a result the LB oerator becomes more sensitive to noise esecially in near-uniform or flat area regions of an image. To deal with this roblem we roose a new definition to the original LB oerator not based only on the value of the central ixel of a certain window, but based on all ixels values in that window. Exeriments conducted on two virtual world avatar face image datasets show that our techniue erforms better than original LB, wavelet LB, adative LB and wavelet adative LB in terms of accuracy. Keywords: Avatar, face recognition, LB, SALB, wavelet transform. 1 Introduction Face recognition is one of the biometric traits that received a great attention from many researchers during the ast few decades because of its otential alications in a variety of civil and government-regulated domains [1]. Face recognition however is not only concerned with recognizing human faces, but also with recognizing faces of non-biological entities or avatars. To address the need for a decentralized, affordable, automatic, fast, secure, reliable, and accurate means of identity authentication for avatars, the concet of Artimetrics has emerged [2, 3]. Artimetrics is a new area of study concerned with visual and behavioral recognition and identity verification of intelligent software agents, domestic and industrial robots, virtual world avatars and other non-biological entities [2, 3]. Extracting discriminant information from a facial image is one of the key comonents for any face recognition system [1]. There are many different algorithms roosed to extract features, such as rincial Comonent Analysis (CA) [4], Linear Discriminant Analysis (LDA) [5] and Local Binary attern (LB) [6]. LB was alied to face recognition for the first time by Ahonen et al. [7]. But the original LB method worked as a local descritor to cature only local information [8]. It thresholds all ixels in the neighborhood based on the gray value of the central

2 ixel. As a result the original LB becomes more sensitive to noise secially in near uniform or flat areas [9]. One solution to this roblem is to threshold all the ixels automatically based on the available data. Most of the work done so far on face recognition has focused on humans. Research on recognizing virtual world's avatars has not received due attention. Some methods develoed LB further for either recognizing human faces or avatar faces. For examle, Yang et al. [10] alied LB for face recognition with Hamming distance constraint. Chen et al. [11] used Statistical LB for face recognition. Mohamed et al. [2] alied hierarchical multi-scale LB with wavelet transform to recognize avatar faces. Mohamed et al. [12] alied wavelet transform as well but with adated local binary atterns and direction statistical features to recognize avatar faces. In this aer, we roose a novel LB face recognition techniue to recognize avatar faces from different virtual worlds. In this aroach, we combine the idea of discrete wavelet transform with new definition of the original LB oerator, Statistical Adated Local Binary attern (SALB) oerator. The efficiency of our roosed method is demonstrated by exeriments on two different avatar datasets from Second Life and Entroia Universe virtual worlds. The rest of this aer is organized as follows; Section 2 briefly rovides an introduction to wavelet decomosition. In Section 3, an overview of the LB is resented. Section 4 resents SALB oerator. Wavelet Statistical Adated Local Binary attern (WSALB) oerator is descrited in section 5. Section 6 reorts exerimental results which followed by conclusions in Section 7. 2 Discrete Wavelet Transform Wavelet Transform (WT) or Discrete Wavelet Transform (DWT) is a oular tool for analyzing images in a variety of signal and image rocessing alications including multi-resolution analysis, comuter vision and grahics. It rovides multi-resolution reresentation of the image which can analyze image variation at different scales. (a) (b) Fig. 1. (a) Wavelet coefficient structure [14] (b) Original image, One and two levels wavelet decomosition for an image

3 WT was alied in image decomosition for many reasons [13]: it reduces the comutational comlexity of the system, decomoses images into sub-images corresonding to different freuency ranges and this can lead to reduction in the comutational overhead of the system and allows obtaining the local information in different domains (sace and freuency). Thus it suorts both satial and freuency characteristics of an image at the same time. WT decomoses facial images into aroximate, horizontal, vertical and diagonal coefficients. Aroximate coefficient of one level is reeatedly decomosed into the four coefficients of the next level of decomosition [13]. Decomosing an image with the first level of WT rovides four sub-bands LL 1, HL 1, LH 1 and HH 1 as in Fig Local Binary attern LB oerator, roosed by Ojala et al. [6], is a very simle and efficient local descritor for describing textures. It labels the ixels of an image by thresholding the ixels in a certain neighborhood of each ixel with its center value, multilied by owers of two and then added together to form the new value (label) for the center ixel [15]. The outut value of the LB oerator for a block of 3x3 ixels can be defined as follows [15]: LB 7 2 S( g g ), R c 0 where g c corresonds to the gray value of the central ixel, g ( = 0,1,2,..,7) are the gray values of its surrounding 8 ixels and S(g - g c ) can be defined as follows: (1) S( g g c 1, ) 0, g g c otherwise (2) Later new versions of LB oerator have emerged as an extension to the original one. They used neighborhoods of different sizes to be able to deal with large scale structures that may be the reresentative features of some tyes of textures [8, 16]. (Fig. 2 gives examles of different LB oerators where is the neighborhood size and R is its radius). (=8, R=1) (= 8, R=1.5) (=12, R=1.5) Fig. 2. Three different LB oerators [7, 8]

4 4 Statistical Adative Local Binary attern (SALB) Using the central ixel value of the neighborhood of any ixel as a threshold has a negative effect on how the LB method can deal with noise esecially in near uniform or flat areas. To obtain good results, we roose a novel LB oerator, SALB, which overcomes the high sensitivity of LB to noise. The SALB s threshold is not fixed but is comuted automatically from the available image data using simle statistics. To minimize the effect of noise, we roose the following euation by which differences between the central ixel value of any local atch and its surrounding ixels values can be reduced. The reduction resulting from comuting the weight for each ixel in a local atch by using the following euation and then that weight can be used to redefine a new value for each ixel. J ( g ( i, c 1 w g ( i, ) where g c corresonds to the central ixel, g to the surrounding ixels, w is the weight for any ixel and 1 w 1 By deriving both sides of euation 3 with resect to the weight for any ixel and assign the result to zero we get: then: J w 2g ( i, ( g ( i, gc( i, w g( i, wg 1 c 2 1 ( i, w g ( i, ) 0 From euation 4 we can obtain the value of w using the following euation: w g ( i, c 1 g ( i, w g ( i, where is any ixel we currently comute its weigh where is any ixel other than in the surrounding. For more exlanation about how we can comute w for different ixels in the neighborhood we have to follow the following stes: 1- Initialization w = 1/ for every = 1,2,, 2- Use the udated euation 5 Reeat For = 1 : Udate w with the new value and each time when we comute the new value of w for any ixel we use this value to comute the w for the next ixels. end After alying these stes we would have the weights for all ixels in the neighborhood and thus we can define the SALB oerator as following: (3) (4) (5)

5 SALB 1 0 s( g * w k * std )2 And so the obtained binary code can be seen as: 1, if g * w k * std s( g * w k * std ) (7) 0, otherwise where std is the standard deviation of all ixels values in an image atch and k is a scaling factor such that 0 < k 1. For more exlanation about how this new LB oerator works see Fig. 3 below shows the result of alying the roosed techniue in a local image area with k = 0.3. (6) LB SALB Noise: LB Noise: SALB Fig. 3. Comarison of LB and SALB oerators. First row: original encodings. Second row: encodings with noise. The bold underlined red ixels are changed with noise. 5 Wavelet Statistical Adative Local Binary attern (WSALB) The roosed algorithm has three stes: rerocessing, feature extraction and recognition or classification: 5.1 rerocessing Facial Images We followed some of rerocessing oerations to imrove the efficiency of extracting the face features. First, the inut images are manually croed to ure face images by removing the background which is not useful in recognition. Second, these ure face images have to be normalized and then decomosed using the first level of wavelet decomosition to obtain the ure facial exression images. Detailed images resulting from alying wavelet decomosition contain changes which reresent the difference of face images. So considering only the aroximation images will enhance the common features of the same class of images and at the same time the differences will be reduced. For this reason, we decomosed only the aroximation images resulted from the first level of wavelet decomosition to obtain the second level of decomosition. So, our exeriments were concerned only with the aroximation

6 images resulted from the second level of wavelet decomosition and which we used in training and testing to evaluate the erformance of the roosed algorithm. 5.2 Extracting Facial Features using SALB The choice of the threshold is very imortant to obtain good results in any face recognition system. One of the best ways for choosing the roer threshold is by automatically comuting this threshold using the local statistics of the available images ixels. So, we have used the SALB oerator, defined by euation 6, to rovide the new value for each ixel of the aroximation facial image obtained from the second level of decomosition. In order to comute the SALB binary code for each ixel we have to follow: - Comute the local weight for each ixel in an image atch by using euation 5 and then multily this weight by its ixel to roduce a new ixel value. - Comute standard deviation of all old ixels values in the image atch including the central ixel value. - Choose a scaling factor k which has a value between 0 and 1 and multily k with the comuted standard deviation to obtain new threshold. (there is no fixed value for k but we have to try different values till we obtain a roer one since its value affects the obtained binary code and then affects also the accuracy rate, during our exeriments we used k = 0.3). Use our threshold with the new ixels values to comute the SALB binary code for each ixel (see Fig. 3 as an examle of how we can use the SALB oerator). 5.3 Dissimilarity Measure The last stage of our roosed techniue is to classify each facial image to its subject. To this end we have used chi-suare distance to comute the dissimilarity between each inut image and the training model as in the following euation [8]: N 2 ( X n Yn ) D LB ( X, Y) (8) n1 X n Yn where X is the testing image, Y is the training model and N is the number of bins. 6 Exerimental Results and Analysis In this section, we verify the erformance of the roosed algorithm on two different tyes of avatar datasets: the first tye is the Second Life (SL) dataset and the second is the Entroia Universe (ENT) dataset (Fig. 5 gives an examle of a subject from each dataset). The roosed method is comared with other single scale techniues such as, the original LB, wavelet LB (WLB), adative LB (ALB) and Wavelet ALB.

7 (a) (b) Fig. 5. Samles of one subject of facial images from: a) Second Life dataset b) Entroia dataset 6.1 Exerimental Setu To evaluate our roosed techniue, we have used two facial image datasets. The first dataset was acuired from a large collection of SL virtual world avatar face dataset [17]. This dataset contains 847 gray scale images with size 1280 x 1024 each to reresent 121 different avatars. Each avatar subject has 7 different images for the same avatar with different frontal ose angle (front, far left, mid left, far right, mid right, to and bottom) and facial exression. The second dataset was collected from ENT virtual world [18]. ENT dataset contains 545 gray scale images with size 407 x 549 ixels. These images were organized in 109 subjects (avatars). Each subject has different 5 images for the same avatar with different frontal angle and facial details (with and without a mask). In order to comare the erformance of our techniue against other techniues we added two tyes of noise to each image in both datasets and then obtain the recognition rate after alying each techniue. These two tyes of noise are noise Gaussian noise and Salt & eer noise. We used the default arameters for each one of the two tyes of noise. The facial art of each image in SL and ENT datasets was manually croed from the original images based on the location of the two eyes, mouth and the nose. The new size of each facial image in SL dataset is 260 x 260 ixels while in ENT dataset each facial image was resized to 180 x 180 ixels. After alying the second level of wavelet decomosition the resolution of each facial image was reduced to 65 x 65 ixels and to 45 x 45 ixels for SL dataset and ENT dataset resectively. Each of the two datasets was slit in two indeendent sets. One set is used for training and the second set for testing. For training we used four image from each subject in the SL dataset and three images from each subject in the ENT dataset. All training images were randomly chosen while the rest were used for testing. 6.2 Exerimental Results To ensure the efficiency of using our new definition of the LB oerator with noisy facial images, we comared WSALB with the original LB, ALB, WLB and WALB with several exeriments. First we got the erformance of WSALB, LB, ALB, WLB and WALB with different LB oerators alied on Gaussian noisy

8 facial images from the two datasets (SL and ENT) and then on Salt & eer noisy facial images. We alied all techniues with R = 1, 2, 3 and = 8, 16, 24. It is evident in figures 6 and 7 that changing the LB oerator affects the recognition rate for all techniues. Also the recognition rate for the same techniue and LB oerator changes from one dataset to another. In case of Gaussian noise, the erformance of the WSALB techniue is better than the erformance of other techniues with most of all LB oerators and with both SL and ENT datasets. In the SL dataset, the WSALB recognition rate is greater than that of its closest cometitor, WALB, by about 3% in average. The highest recognition rate obtained (when the LB oerator is (16, 3)) is 92.37%. In the ENT dataset, the WSALB recognition rate is greater than that of its closest cometitor, WALB, by more than 4% in average. The highest recognition rate obtained (when the LB oerator is (16, 2)) is 81.63%. (a) (b) Fig. 6. Recognition rate for the SL dataset and ENT dataset with Gaussian noise: a) SL dataset b) ENT dataset (a) (b) Fig. 7. Recognition rate for the SL dataset and ENT dataset with Salt & eer noise: a) SL dataset b) ENT dataset There was no great difference in erformance of the WSALB techniue on the Gaussian and the Salt & eer noisy images. That is, erformance of the WSALB is better if comared to most other techniues. The closest cometitor to The WSALB techniue on the Salt & eer noisy images is the ALB techniue (in case of the SL dataset). The increase in the recognition rate is 3% in average and the highest recognition rate is 94.77% obtained when the LB oerator is either (16, 3) or (24, 3). In case of the ENT dataset, the closest cometitor to the WSALB techniue is the WALB with difference of about 2% in average to the side of the WSALB

9 techniue. The highest recognition rate is 81.63% when the LB oerator is either (16, 2) or (8, 3) (see table 1 which shows the average recognition rate for all techniues where SL G stands for SL dataset with Gaussian noise and SL S stands for SL dataset with Salt & eer noise and the same for ENT G and ENT S ). Table 1. Average recognition rate for different algorithms Dataset Techniues LB ALB WLB WALB WSALB SL G 51.14% 62.56% 58.59% 70.73% 73.92% SL S 71.44% 89.29% 72.69% 8920% 92.29% ENT G 53.57% 60.70% 61.67% 70.28% 74.77% ENT S 66.68% 75.76% 69.17% 76.97% 78.81% 7 Conclusion In this aer, a novel LB face recognition aroach (WSALB) is roosed based on discrete wavelet transform and a new definition to the original LB oerator (SALB). This roosed aroach treats some of the LB limitations, that is, noise sensitivity. The effectiveness of this method is demonstrated on recognizing faces from two different virtual worlds, Second Life and Entroia Universe. Our roosed techniue imroved the recognition rate for the SL dataset by about 3% for both Gaussian and Salt & eer noise when comared to LB, ALB, WLB and WALB and different LB oerators. Also, our techniue imroved the recognition rate for the ENT dataset by about 4% and 2% for Gaussian and Salt & eer noise resectively. We intend to use our roosed techniue to build a comlete recognition system for avatars in the future. References 1. Guo, Z., Zhang, L., Zhang, D., Mou, X.: Hierarchical Multiscale LB for Face and almrint Recognition. In: International Conference on Image rocessing. Hong Kong (2010) Mohamed, A.A., D'Souza, D., Baili, N., Yamolskiy, R.V.: Avatar Face Recognition using Wavelet Transform and Hierarchical Multi-scale LB. In: 10th IEEE International Conference on Machine Learning and Alications. Honolulu, Hawaii (2011) Gavrilova, M.L., Yamolskiy, R.V. : Alying Biometric rinciles to Avatar Recognition. In: International Conference on Cyberworlds. Singaore (2010) Turk, M., entland, A.: Face Recognition using Eigenfaces. In: IEEE Conference on Comuter Vision and attern Recognition. Maui, HI (1991) Lu, J., lataniotis, K.N., Venetsanooulos, A.N.: Face Recognition Using LDA Based Algorithms. IEEE Transactions on Neural Networks 14(1), (2003). 6. Ojala, T., ietikainen, M., Harwood, D.: A comarative study of texture measures with classification based on featured distributions. attern Recognition 29(1), (1996). 7. Ojala, T., ietikainen, M., Maenaa, T.: Multiresolution Gray-Scale and Rotation Invriant Texture Classification with Local Binary atterns. IEEE Transactions on attern

10 Analysis and Machine Intelligence 24(7), (2002). 8. Ahonen, T., Hadid, A., ietikainen, M.: Face Descrition with Local Binary atterns: Alication to Face Recognition. IEEE Transactions on attern Analysis and Machine Intelligence 28(12), (2006). 9. Akhloufi, M.A., Bendada, A.: Locally Adative Texture Features for Multisectral Face Recognition. In: IEEE International conference on Systems Man and Cybernetics. Istanbul, Turkey (2010) Yang, H., Wang, Y.D.: A LB-based Face Recognition Method with Hamming Distance Constraint. In: 4th International Conference on Image and Grahics. Sichuan, China (2007) Chen, L., Wang, Y.H., Wang, Y.D., Huang, D.: Face Recognition with Statistical Local Binary atterns. In: 8th International Conference on Machine Learning and Cybernetics. Baoding, China (2009) Mohamed, A.A., Gavrilova, M.L., Yamolskiy, R.V.: Artificial Face Recognition using Wavelet Adative LB with Directional Statistical Features. In: 12th International Conference on Cyberworlds. Darmstadt, Germany (2012). 13. Mazloom, M., Ayat, S.: Combinational Method for Face Recognition: Wavelet, CA and ANN. In: International Conference on Digital Image Comuting: Techniues and Alications. Canberra (2008) Garcia, C., Zikos, G., Tziritas, G.: A Wavelet-based Framework for Face Recognition. In 5th Euroean Conference on Comuter Vision. Freiburg, Allemagne (1998) Meng, J., Gao, Y., Wang, X., Lin, T., Zhang, J.: Face Recognition based on Local Binary atterns with Threshold. In IEEE International Conference on Granular Comuting. San Jose, CA (2010) Wang, W., Chang, F., Zhao, J., Chen, Z.: Automatic facial exression recognition using local binary attern. In: 8th World Congress on Intelligent Control and Automation. Jinan, China (2010) Second Life. Available from: Entroia Universe. Available from: