Recognizing Facial Expressions Based on Gabor Filter Selection

Transcription

1 Recognzng Facal Expessons Based on Gabo Flte Selecton Zang Zhang, Xaomn Mu, Le Gao School of Infomaton Engneeng Zhengzhou Unvest Zhengzhou, Chna Abstact Recognton of human emotonal state s an mpotant component fo effcent human-compute nteacton. In ths pape a method of Gabo flte selecton fo facal expesson ecognton s nvestgated. We fst pepocess facal mages based on affne tansfom to nomalze the faces. hen the usng of a sepaablt judgment s poposed to evaluate the sepaablt of dffeent Gabo fltes, and onl use those fltes that can bette sepaate dffeent expessons. In the ecognton pocess a PCA and FLDA multclassfe scheme s used. he expement esult shows that the ntoducng of Gabo flte selecton can not onl educe the dmenson of featue space but also educe the computaton complext sgnfcantl, whle etanng hgh ecognton ate of above 93%. Kewods-Facal expesson ecognton; Gabo flte selecton; FLDA I. INRODUCION Facal expesson ecognton plas an mpotant ole n effectve human-compute nteacton, and can be appled to vaous stuatons such as patent montong and human behavo ntepetaton. heefoe t attacted a lot of eseach nteests dung ecent eas. Based on the wok b Dawn [1], who poposed the exstng of basc pototpc facal expessons, most eseaches classf 6 basc facal expessons: happness, sadness, supse, ange, dsgust, fea, o 7 expessons that nclude neutal state. hee have been vaous knds of methods to extact expesson featues fom facal mages. Among these methods, Gabo featues have been used extensvel afte Lons et al. [2] fst poposed that the Gabo epesentaton shows a sgnfcant degee of pschologcal plausblt. Rose [3] used Gabo wavelet to extact featues fom facal mages, and then used PCA+LDA fo dmenson educton and classfcaton, fnall got an 86% ecognton accuac on JAFFE database. Zhang et al. [4] adopted Gabo wavelet coeffcents and geometc postons to constuct the featue vecto fo each mage and appled two-lae pecepton to dstngush seven dffeent facal expessons. he ecognton ate s 90.1%. Shh et al. [5] compaed the ecognton accuac of dffeent ecognton sstems on JAFFE database, and found that the hghest ecognton ate acheved usng Gabo featues s 92.00%. Although Gabo featues do not eque pecse egstaton of mages, Gabo fltes suffe fom the dsadvantage of hgh dmensonal featue space. hs esults fom the fact that fo each sngle flte n the Gabo flte bank, the output s equal n sze wth the ognal mage, and the featues extacted have geat edundanc. heefoe, dmenson educton s needed, whch was often acheved b pncple component analss (PCA). Howeve, just usng PCA s ndeed lmted, because t causes exta computaton and does not educe the tme of featue extacton. In ths pape, we popose a method fo educng the numbe of Gabo fltes, based on sepaablt analss, and buld a ecognton sstem usng a 2-class FLDA classfe scheme to evaluate the pefomance of flte selecton. he est of ths pape s oganzed as follows: Secton II pesents the pepocessng method of mages. Secton III ntoduces the Gabo flte selecton method and featue extacton pocess. Desgn of classfe s dscussed n Secton IV. Secton V gves the expement esults and conclusons ae dawn n Secton VI. II. IMAGE PREPROCESSING It s known that most of the facal expesson nfomaton s concentated aound aeas such as the ees and the mouth. Includng elevant aeas lke ha o the backgound mght poduce wthn-class dssmlat and cause ncoect decsons, so t would be bette to mnmze an othe possble vaatons except fo the pattens we ae classfng. Hee we ntoduce a method smla wth [6], based on affne tansfom, to extact and nomalze the face egon: Fstl, manuall locate 3 ke ponts (cente of ees and cente of mouth) on the facal mage. hen use an affne tansfom (whch s a knd of coodnate tansfom that conssts of tanslaton, scalng and otaton) to locate the 3 ponts at fxed postons n the output mage, so that faces n all mages ae appoxmatel of the same sze and n the same place, as shown n Fg. 1. Secondl, we use the geometc face model [7] llustated n Fg. 1(b) to cop out the face ectangle, the sze of whch s pxels n ou case. We then cop the left and ght lateal pats of faces to onl consde the ntenal aea. At last we pefom hstogam equalzaton to elmnate the vaatons n llumnaton and skn colos.

2 (a) (b) Fgue 1. Examples of mage pepocessng. appoxmatel tangent wth each othe [8], as shown n Fg. 3. Let f h denotes the cente fequenc of the fst scale. Let K be the numbe of oentatons and S be the numbe of scales. hen the desgn stateg esults n the followng fomulas: a1 fh fh u, v tan. (4) a 1 2ln 2 2 K 2ln 2 hs pape uses a 2, fh 0.4, S 4, and K 6. hese paamete values have been poved to geneate good epesentaton of mage textue featue [8]. he fltes fequenc esponses ae shown n Fg. 3. Fgue 2. Example mages afte pepocessng. Fg. 2 shows some example mages afte pepocessng. he ognal mages ae fom JAFFE [2] (Japanese Female Facal Expesson) database. III. GABOR FILER SELECION AND FEAURE EXRACION A. Gabo functons and wavelets A complex-valued two dmensonal Gabo functon can be gven as follows [tutoal on Gabo fltes]: x g( x, ) exp j2 f 2 2 0x, (1) 2 x 2x whee x xcos sn, (2) and xsn cos (3) epesent a otaton. specfes the oentaton of the paallel stpes of the fltes. x and ae the standad devaton of the Gaussan facto and detemne the sze of ts eceptve feld. he half-peak magntude (3dB) suppot egon of the Gabo flte s an ellpse centeed at u f cos and v f sn n the spatal fequenc doman, wth ts long and shot axes equal to 2 2ln2 2 and 2 2ln2 2, espectvel. x he use of scale facto a m m, so that x ' x a, m ' a and f0' f0 a m, togethe wth the oentaton paamete, geneate a bank of Gabo fltes called Gabo wavelets. Howeve, these fltes ae not othogonal, esultng n the edundanc of nfomaton. As the some edundanc comes fom the ovelappng of dffeent fltes n the spatal fequenc doman, t s easonable to consde mnmzng the aea of ovelap, b placng the 3dB pofles of the flte esponses so that these ellpses ae B. Featue Extacton at Sample Ponts Suppose the ga scaled mage ead fo featue extacton s expessed b ( x, ). Gven a Gabo flte g ( x, ) whch s of m th scale and n th oentaton, the Gabo tansfom of the mage s then defned as: (5) W ( x, ) ( x, ) g ( xx, ) dxd hs convoluton pocess poduces an equal szed output mage fo each sngle Gabo flte. Even f we use onl one flte, we get Gabo coeffcents n total. he featue dmenson s alead too hgh fo futhe pocessng. Consdeng that passng though the low-pass Gabo flte blus the mage, we ma not use all the coeffcents fo futhe classfcaton. Instead, ths pape samples the flteed mages at gven ponts (10 ows, 8 colus n ths case, wth lateal ponts dscaded, esultng n 62 ponts), whose locatons n the ognal mage s llustated n Fg. 4. hen combne these sampled coeffcents nto a ow vecto x. In consdeaton of convenence, we denote the whole pocedue b: Fgue 3. Half-peak (3dB) fequenc esponses of the Gabo flte bank x FeatueExtact, g. (6) C. Selectng Gabo Fltes based on Sepaablt Analss hough a set of fltes has been buld, whose fequenc esponses ae appoxmatel tangent to each othe, the stll have some edundanc, as long as the ae not othogonal.

3 featue fom all fltes s geneall between the maxmum and the mnmum value of the featues fom sngle flte. We wll sot the fltes based on the judgment value and use dffeent numbe of fltes that have the best sepaablt. Expement esults wll show that the dscad of man fltes wth lowe J value can hadl nfluence the fnal ecognton esult. Fgue 4. Locatons of sample ponts Besdes, usng so man fltes (24 n ths case) eques lage amount of computaton esouce and geneates hgh dmensonal featue space. Howeve, f we want to select a subset of fltes fom the ognal flte bank, whch pncple should we follow? hs pape poposes the usng of a sepaablt judgment to evaluate the sepaablt of a gven flte, and then select the appopate ones [9]. Suppose the sample mage set s I, whch contans N mages, and thee ae totall c classes, whch ae denoted as w1, w2, wc. Fo each sngle flte g ( x, ) n the flte bank we deved fome, appl the followng steps to compute ts judgment functon: Use the pocedue defned n (7) to get the featue vecto set X [ x1, x2,, x ]: N xk FeatueExtact k, g, k 1, 2,, N. (7) hen the featue set X has the same class label wth the ognal mage set I. Let m ( 1, 2, c) denote the mean value of the featue vectos n class w, n epesents the numbe of sample mages n class w, and m denotes the mean value of featue vectos n all dffeent classes. hen the wthn-class scatte matx and between-class scatte matx s defned as: c w 1 xw S x m x m, (8) c S n m m m m (9) b 1 We evaluate the sepaablt of the featue set X usng the followng judgment functon [9]: t( Sb ) J( X), (10) t( Sw) whee t() epesents the tace of matx. hs cteon s dffeent wth the Fshe cteon n that the latte one uses the detemnant of the between-class and wthn-class scatte matx, whch ae often close to sngula and make t dffcult to compute. hough ths cteon s not dectl elated to the fnal ecognton accuac, t does ende some nfomaton about the sepaablt of the featue extacted. Expement esults wll show that the sepaablt value of IV. DESIGN OF CLASSIFIER Fo a c class poblem, wth a t dmensonal featue space, thee must be at least t c tanng samples to guaantee the feasblt of lnea dscmnant analss (LDA) [10]. Howeve, afte Gabo flte selecton, the featue dmenson t s stll hghe than the total numbe of tanng samples. [11] and [12] popose the use of an ntemedate space,.e. befoe cang out LDA, use PCA fst to poject the featue space nto an ntemedate space of N c dmensons, whee N s the numbe of tanng samples. Hee we adopt ths method, usng PCA to educe the featue dmenson to N c : k Wpcaxk, k 1,2,, N. (11) hen a 2-class FLDA multclassfe scheme s buld fo the ecognton pocess [13]. FLDA assumes the dscmnant functon to be a lnea functon of the featue data. In ths case the data s the featue vecto obtaned n (12). hus fo each of the 6 facal expessons, constuct a 2-class dscmnant functon: g( ) W w0, 1,2, c, (12) whee W s a vecto, and w 0 s a constant. Fshe s LDA fnds the vecto W that best sepaates the two classes, b maxmzng the ato of the detemnants of between-class and wthn-class scatte matx. hs pape bulds 6 such 2-class classfes, each coespondng to one of the 6 basc expessons (HA, SA, SU, AN, DI, FE). In the tanng pocess, fo each classfe, we label all the tanng samples that do not belong to the coespondng expesson as one class. he output of each classfe s the pobablt of belongng to the coespondng expesson. Fo example, n the HA classfe, all the samples of happness ae labeled as happness, whle all othe tanng samples ae labeled as non-happness. If the output s geate than 50%, the sample ma be classfed as happness, and othewse non-happness. he output of each classfe s used to detemne the fnal classfcaton esult. he decson ule hee s that, f the output s geate than 50% n one o moe than one classfes, we classf the sample to the class that has the geatest output value. Othewse, f all the classfes outputs ae below 50%, label the sample as neutal. he classfcaton scheme s shown n Fg. 5.

4 Fgue 5. Mult-classfe scheme V. EXPERIMEN RESULS AND ANALYSIS he database used hee to examne ou ecognton sstem s the JAFFE [2] database. It contans 10 female expessos, each delveng 7 facal expessons (happness, sadness, supse, ange, dsgust, fea, as well as neutal). hee ae 213 ga scaled mages n total. hs pape uses the leave-one-out stateg [5] to test the ecognton ate. A. Sepaablt of Fltes Afte pefomng Gabo flte selecton based on all the mages n JAFFE, the sepaablt judgment value of each sngle flte n the flte bank s shown n able 1. Fom able 1 we can see that featues fom scale 2 and scale 3 have bette oveall sepaablt than scale 0 and scale 1, and the sepaablt value deceased as scale deceases. It shows that fltes of lowe fequenc can gve moe sepaable nfomaton about facal expessons. Besdes, esult also shows that wthn each scale, fltes of decton 2 and 3 geneall have lage sepaablt value, showng that fltes at the oentaton of appoxmatel 60~120 degees ae moe sutable n facal expesson ecognton. B. Recognton Rate We sot the 24 fltes n tems of the sepaablt judgment value. o evaluate the pefomance of dffeent numbe of fltes, the fst k fltes that have the lagest sepaablt value ae used fo featue extacton, whee the numbe k anges fom 4 to 24. he mnmum numbe 4 s set to make sue the featue dmenson s lage than the numbe of tanng samples, to enable the use of PCA n the classfcaton method. he ecognton ates and computaton tmes of dffeent numbe k ae gven n able 2 and llustated n Fg. 6. he tme data s collected n the envonment of Intel(R) Coe(M) 2 Duo CPU 6570, 3GB RAM, wth Wndows 7 and MALAB 7.4 softwae nstalled. he tme value s account fo the tanng pocess and the test of one sample mage. Results show that the use of all 24 fltes geneates the hghest ecognton ate of 95.77%, and when flte numbe deceases, the ecognton ate also deceases. hs ndcates that all the fltes ae helpful fo classfcaton. When usng all the fltes, the tme of computaton s seconds, whch mght be too long fo a pactcal sstem. When the numbe of fltes deceases, the tme of computaton gets shote, appoxmatel n an exponental wa. Howeve, the ange of decease of ecognton ate s small. When flte numbe s moe than 8, the ecognton ates ae all above 90%. And when the numbe of fltes goes up to 9, the ecognton ate ses to 93.90%, whch s not a lage lose fom the hghest 95.77%, whle the tme of computaton s educed to 1.88 seconds, 20 tmes less than the hghest seconds. he ecognton ate 93.90% s alead hghe than man exstng sstems [3,4,6]. hs esult demonstates that the dscad of fltes wth lowe sepaablt makes ve lmted nfluence on the fnal ecognton ate, whle sgnfcantl educes the tme of computaton and makes the ecognton sstem moe effcent. On the othe hand, the hgh ecognton ate of ou sstem ma esult fom the accuac of the pepocessng of mages. Because n the nomalzed facal mages, ees and mouths ae appoxmatel at the same locaton, whch mnmzes the elevant vaatons except fo the expesson dffeences we ae ecognzng. VI. CONCLUSION hs pape poposes a facal expesson ecognton sstem based on Gabo flte selecton. he goal of selectng Gabo fltes s to educe the computatonal complext, whle etanng good ecognton ates. Fst a moe pecse face nomalzaton method s ntoduced to the Gabo-based sstem. hen flte selecton s pefomed based on sepaablt analss. hs pape poposes the use of a sepaablt judgment to evaluate each sngle flte n the bank of Gabo fltes, and chose those fltes wth bette sepaablt. At last a PCA and FLDA based multclassfe scheme s used fo ecognton. ABLE I. SEPARABILIY JUDGMEN VALUE OF FILERS Oentaton Scale Scale Scale Scale Fgue 6. Expement esults wth dffeent k values

5 ABLE II. EXPERIMEN RESUL Numbe of Fltes Recognton Rate me Numbe of Fltes Recognton Rate me Expement esults show that ou poposed method can sgnfcantl educe the tme of computaton (fom seconds to 1.88 seconds), whle the ecognton ate onl deceases fom 95.77% to 94.00%, whch s stll hghe than most exstng sstems. Although b no means conclusve, ths stud sheds some ve nteestng lghts and leads to new dectons n facal expesson ecognton. Howeve, the good pefomance of ou sstem ma be patl due to the pecse pepocessng of facal mages, based on affne tansfom. he postons used to ca out affne tansfom s located manuall, thus ma not be sutable fo an automatc sstem. In the futue a moe accuate automatc method to nomalze faces wll make geat mpovement on the ecognton sstem. REFERENCES [1] C. Dawn, he Expesson of Emotons n Man and Anmals, John Mua, epnted b Unv. of Chcago Pess, [2] M. Lons, S. Akamatsu, M. Kamach, and J. Goba, Codng facal expessons wth Gabo wavelets, Poceedngs of the hd IEEE Confeence on Face and Gestue Recognton, Naa, Japan, Apl 1998, pp [3] N. Rose, Facal expesson classfcaton usng Gabo and log-gabo fltes, Poceedngs of the 7th ntenatonal confeence on automatc face and gestue ecognton (FGR'06), [4] Z. Zhang, M. Lons, M. Schuste, and S. Akamatsu, Compason between geomet-based and Gabo-wavelets-based facal expesson ecognton usngmult-lae pecepton, Poceedngs of the hd IEEE Confeence on Face and Gestue Recognton, Naa, Japan, Apl 1998, pp [5] Fank Y. Shh, Chao-Fa Chuang and Patck S. P. Wang, Pefomance compasons of facal expesson ecognton n JAFFE database, Intenatonal Jounal of Patten Recognton and Atfcal Intellgence, Vol. 22, No. 3 (2008) [6] S. Dubusson, F. Davone and M. Masson, A soluton fo facal expesson epesentaton and ecognton, Sgnal Pocessng: Image Communcaton, 17 (2002) [7] Fank Y. Shh and Chao-Fa Chuang, Automatc extacton of head and face boundaes and facal featues, Infomaton Scence, 158 (2004) [8] B. S. Manjunath and W. Y. Ma, extue featues fo bowsng and eteval of mage data, IEEE ans. Patten Anal. Machne Intell., vol. 18, no. 8, pp , Aug [9] Xao-Yuan Jng, Yuan-Yan ang and Davd Zhang, A Foue-LDA appoach fo mage ecognton, Patten Recognton, 38 (2005) [10] R. O. Duda, P. E. Hat and D. G. Stok, Patte Classfcaton, 2nd Edton, John Wle & Sons, [11] P. N. Belhumeu, J. P. Hespanha, and D. J. Kegman, Egenface vs. Fsheface: ecognton usng class specfc lnea pojecton, IEEE ans. Patten Anal. Machne Intell., vol. 19, no. 7, pp , Jul [12] A. M. Matnez and A. C. Kak, PCA vesus LDA, IEEE ans. Patten Anal. Machne Intell., vol. 23, no. 2, pp , Feb [13] Yongjn Wang and Lng Guan, Recognzng human emotonal state fom audovsual sgnals, IEEE tans. Multmeda, vol. 10, no. 5, pp , Aug