Facial Expression Recognition based on Affine Moment Invariants

Transcription

1 IJCSI International Journal o Computer Science Issues, Vol. 9, Issue 6, No, November 0 ISSN (Online): Facial Expression Recognition based on Aine Moment Invariants Renuka Londhe and Vrushsen Pawar College o Computer Science and IT, Swami Ramanand Teerth Marathwada University, Nanded, Maharashtra, India School o Computational Studies, Swami Ramanand Teerth Marathwada University, Nanded, Maharashtra, India Abstract Facial Expression Recognition is rapidly becoming area o interest in computer science and human computer interaction because the most expressive way o displaying the emotions by human is through the acial expressions. In this paper, recognition o acial expression is studied with the help o several properties associated with the ace itsel. As acial expression changes, the curvatures on the ace and properties o the objects such as, eyebrows, nose, lips and mouth area changes. We have used Aine Moment Invariants to compute these changes and computed results (changes) are recorded as eature vectors. We have introduced a method or acial expression recognition using Aine Moment Invariants as eatures. We have used Artiicial Neural Network as a classiication tool and we developed associated scheme. The Generalized Feed-orward Neural Network recognizes six universal expressions i.e. anger, disgust, ear, happy, sad, and surprise as well as seventh one neutral. The Neural Network trained and tested by using Scaled Conjugate Gradient Backpropogation Algorithm. As a result we got 93.8% classiication rate. Keywords: Artiicial Neural Network, Facial Expressions, Aine Moment Invariants, Human computer Interaction.. Introduction The computer-based recognition o acial expression has received a lot o attention in recent years. The analysis o acial expression or behavior would be beneicial or dierent ields such as lawyers, the police, and security agents, who are interested in issues concerning dishonesty and attitude. The eventual goal in research area is the realization o intelligent and transparent communication between human being and machines. Some researchers used only our expressions or analysis using computer and some used six expressions. We have used Japanese Female Facial Expression (JAFFE) database with seven expressions or analysis through the computer. Several acial expression recognition methods proposed in literature. For example Facial Action Coding System (FACS) [, ] Facial Point Tracking [3] and Moment Invariant. Ekman and Friesen [] developed the Facial Action Coding System (FACS) or describing expressions, such as anger, disgust, ear, happy, neutral, sad, and surprise. The Maja Pantic [4] did machine analysis o acial expressions. James J. Lien and Takeo Kanade have developed Automatic acial expression recognition system based on FACS Action Units in 998 [5]. They have used aine transormation or image normalization and acial eature point tracking method or eature extraction as well as Hidden Marko Model (HMM) was used or classiication. L. Ma., K. Khorasani developed Constructive Feed-orward Neural Network or acial expression recognition [6]. First they generated the dierence image rom Neutral and expression image, -D DCT coeicients o dierence images considered as input to the constructive neural network. Praseeda Lekshmi and Dr. M. Sasikumar proposed a Neural Network Based Facial Expression Analysis using Gabor Wavelets. They used Gabor Wavelets as a eature extraction method and neural network as a classiication technique [7]. Guoying Zhao and Matti have developed local binary patterns or dynamic texture recognition which was urther applied or acial expression recognition [8]. Y. Zhu. L. C. DE. Silva used moment invariants as a eature extraction rom acial expressions and Hidden Marko Model or classiication [0]. But they classiied only our types o expressions i.e. anger, disgust, happy and surprise. Using Hu moment invariants acial expression recognition is done with 9.4% recognition accuracy []. In this research work we have tried to classiy acial expressions into seven categories with the help o Aine Moment Invariants as eature extraction technique and artiicial neural network as classiication. This paper is organized as ollows. Section introduces Database and image normalization. Section 3 describes Feature extraction technique. Section 4 describes neural network process like training algorithm. Section 5 depicts the experimental results and it discusses the conusion matrix and accuracy levels o classiication. Finally, conclusions are drawn in section 6. Copyright (c) 0 International Journal o Computer Science Issues. All Rights Reserved.

2 IJCSI International Journal o Computer Science Issues, Vol. 9, Issue 6, No, November 0 ISSN (Online): Data Collection and Normalization This method is trained and tested on the JAFFE (Japanese Female Facial Expression) Database []. The database contains 3 images o seven acial expressions (6 basic acial expressions + neutral) such as anger, disgust, ear, happy, neutral, sad and surprise posed by 0 Japanese emale models. For our experiment, we selected 0 images rom the database or the expression recognition. There are images come rom one subject with seven expressions i.e. three images o one expression. Some sample images rom database are in Fig.. Happiness Sadness Anger Fig. 3 Image ater applying Sobel edge detection operator 3. Feature Extraction Surprise Disgust Fear Fig. Sample images rom database It is observed that acial eatures contributing to acial expressions mainly lie in some regions, such as eye area and mouth area and nose area. Facial classiication needs inormation that is more useul. These regions contain such type o inormation. The important eatures o expressions relected through the eyes and mouth. Thereore, we extract average ace o size 5 x 06 rom original image. Again, the average images divided in to three blocks to extract the more eatures such as upper, middle and lower as shown in Fig. Moment invariants are invariant under shiting, scaling and rotation. They are widely used in pattern recognition because o their discrimination power and robustness. We divide ace into three areas or eature extraction, which are upper, middle and lower areas o ace. Images are converted into binary images or extracting the boundary values using Sobel edge detection operator. Aine Moments: In order to understand how to utilize moment invariant method, let b be a binary digital image with size X x Y and (, ) b x y is the gray level value or the pixel at row x and column y. The two-dimensional moments o order ( p+ q) o (, ) b x y which is variant to the scale, translation and rotation is deined as X Y = x= y= p q m x y ( x, y) The central moments o order ( p q) deined as Where b + o X Y q = c c b x= y= ( x x ) ( y y ) ( x, y) () (, ) b x y is () xc And yc are the centers o mass o the object deined as Fig. Average ace divided into three blocks We have decided to extract the eatures rom edges and boundaries o images. For extracting the edges we have used Sobel edge detection operator. Fig. 3 shows the image ater applying Sobel edge detection operator. x m and m 0 0 c = yc m m = (3) The moment invariant under scale is deined as η = ( ) ' ' γ (4) Copyright (c) 0 International Journal o Computer Science Issues. All Rights Reserved.

3 IJCSI International Journal o Computer Science Issues, Vol. 9, Issue 6, No, November 0 ISSN (Online): p+ q γ = + (5) ' and = (6) Where ( p q ) α + + Normalized un-scaled central moment is then given by ν = ( ) γ (7) The Aine moment invariants are derived to be invariants to translation, rotation, scaling o shapes and under general D Aine transormation. The six Aine moment invariants used are deined as ollows: I = ( ) (8) I = ( (9) ) I 3 = ( 0( 0 03 ) ( ) + ( )) 0 30 I = ( (0) ) () I = ( ) () I 6 = ( ) (3) The algorithm o Aine Moment is implemented as:. Calculating the value o xcandyc based on Equation (3).. Computing each value o,,,,,,,,,,, and using Equation (). 3. Calculating the value o Aine Moments I I 6 according to equation (8) to (3). Two sets o six eatures using Aine Moment unctions, are computed in this research, one set derived rom the area o average ace and the other rom the boundary o average ace. Thereore, eatures are generated rom one section; total 36 eatures are generated rom one image. 4. Classiications For classiication purpose, we have used the two layered eed orward neural network in which learning assumes the availability o a labeled set o training data made up o N input and output. T = {( X, )} N i d i i= (4) X Where, i is input vector or the i th example, N is the sample size. A two layered eed orward neural network with sigmoid activation unction is desired with 36 input neurons, 30 hidden neurons and 7 output neurons or classiication. Created neural network is trained and tested using Scaled Conjugate Gradient Backpropogation Algorithm. Scaled Conjugate Gradient Method: SCG is a second order conjugate gradient algorithm that helps minimizing goal unction o several variables. SCG algorithm was proposed by Moller [3] in 993. Minimization is a local iterative process in which an approximation to the unction, in a neighborhood o the current point in the weight space is minimized. The Scaled Conjugate Gradient (SCG) algorithm denotes the quadratic approximation to the error E in the neighborhood o a point w by: ' T T '' Eqw( y) = E( w) + E ( w) y+ y E ( w) y (5) SCG belongs to the class o Conjugate Gradient Methods. SCG use a step size scaling mechanism and avoids a time consuming line-search per iteration, which makes the algorithm aster than other methods and second order algorithms. 5. Experimental Results In this work, Aine Moment Invariants are used to extract the eatures. In the irst step average ace is extracted then the average ace is divided into three parts such as upper, middle and lower. We have extracted the edges or boundaries o average images using the Sobel edge detection operator. Using the Aine moment invariants six eatures are extracted rom each part o the image as well as its edges or boundaries. We compute the 36 eature Copyright (c) 0 International Journal o Computer Science Issues. All Rights Reserved.

4 IJCSI International Journal o Computer Science Issues, Vol. 9, Issue 6, No, November 0 ISSN (Online): values rom each average ace. With the help o these eatures we classiy 0 images o 0 subjects o seven acial expressions into seven classes such as anger, disgust, ear, happy, neutral, sad, and surprise. By using Feed orward neural network classiication is done. Scaled Conjugate Gradient Backpropogation algorithm is used to train and test the network. The implementation o neural network and the training method were done with the Neural Networks Toolbox o MATLAB The conusion matrix gives the accuracy o the classiication problem. 0 images o 0 subjects are classiied into seven categories with 93.8% accuracy. The diagonal elements in the conusion matrix show the classiied groups. For the input we gave 30 images o every class. Resultant Conusion matrix (Fig. 4) illustrates that, out o 30 samples 30 samples are all in anger class. Out o 30 samples o disgust class 9 alls in disgust and remaining one is in ear class. Similarly classiication is done with all classes as shown in conusion matrix. Thereore total average accuracy o classiication is 93.8% and misclassiication or error rate is 6.%. 6. Conclusions Fig. 4 Conusion Matrix In this paper, an eicient acial expression recognition methodology is introduced. Here the acial expression recognition is based on Aine Moment Invariants. Feature vectors are computed by using unctions o Aine Moment Invariants. These eatures are classiied using the two layers eed orward neural network. For training this neural network Scaled Conjugate Gradient Backpropogation Algorithm is used. The six universal expressions i.e. anger, disgust, ear, happy, sad, and surprise as well as seventh one neutral, are recognized with as high as 93.8% accuracy with the combination o 0 subject s images. Reerences [] Ioanna-Ourania and George A. Tsihrintzis, An improved Neural Network based Face Detection and Facial Expression classiication system, IEEE international conerence on Systems Man and Cybernetics 4. [] Ying-li Tian, Takeo Kanade, and Jaery F. Cohn, Recognizing Action Units or Facial Expression Analysis, IEEE transaction on PAMI, Vol. 3 No. Feb. [3] Paul Ekman, Basic Emotions, University o Caliornia, Francisco, USA. [4] Maja Pantic and L.L.M. Rothkrantz, Automatic analysis o acial expressions: The state o the art, IEEE Trans. PAMI Vol. no. 0. [5] James J. Lien and Takeo Kanade, Automated Facial Expression Recognition Based on FACS Action Units, IEEE published in Proceeding o FG 98 in Nara Japan. [6] L. Ma and K. Khorasani, Facial Expression Recognition Using Constructive Feed orward Neural Networks, IEEE TRANSACTION ON SYSTEM, MAN AND CYBERNETICS, VOL. 34 NO. 3 JUNE 4. [7] Praseeda Lekshmi V Dr. M. Sasikumar, A Neural Network Based Facial Expression Analysis using Gabor Wavelets, Word Academy o Science, Engineering and Technology. [8] Guoying Zhao and Matti Pietikainen, Dynamic Texture Recognition using Local Binary Patterns with an Application to Facial Expressions, IEEE transaction on PAMI 7. [9] Renuka R. Londhe, Dr. V. P. Pawar, Analysis o Facial Expression using LBP and Artiicial Neural Network, International Journal o Computer Applications ( ) volume-44 April 0. [0] Y. Zhu, L. C. DE. Silva, C. C. Co, Using Moment Invariant and HMM or Facial Expression Recognition, Pattern Recognition Letters Elsevier. [] Renuka R. Londhe, Dr. V. P. Pawar, Facial Expression Recognition based on Hu Moment Invariants, Journal o Computing, Volume 4, Issue 8, August 0. [] Japanese Female Facial Expression Database, [3] Martin F. Moller, A Scaled Conjugate Gradient Algorithm or Fast Supervised Learning, Neural Networks, 993. Renuka R. Londhe received the M.C.S. degree rom SRTM University, Nanded in the year 4. She received the M. Phil. Degree in Computer Science rom Y.C.M.O. University, Nashik in the year 9. She is currently working as lecturer in the College o Computer Science and inormation Technology, Latur, Maharashtra. She is leading to PhD degree in S. R. T. M. University, Nanded. Vrushsen P. Pawar received MS, Ph.D. (Computer) Degree rom Dept. CS & IT, Dr. B. A M. University & PDF rom ES, University o Cambridge, UK also received MCA (SMU), MBA (VMU) degrees respectively. He has received prestigious ellowship rom Copyright (c) 0 International Journal o Computer Science Issues. All Rights Reserved.

5 IJCSI International Journal o Computer Science Issues, Vol. 9, Issue 6, No, November 0 ISSN (Online): DST, UGRF (UGC), Sakal oundation, ES London, ABC (USA) etc. He has published and more research papers in reputed national international Journals & conerences. He has recognized Ph. D. Guide rom University o Pune, SRTM University & Sighaniya University (India). He is senior IEEE member and other reputed society member. He is currently working as an Associate Proessor at School o computational studies o SRTM University, Nanded. Copyright (c) 0 International Journal o Computer Science Issues. All Rights Reserved.