Local Ridge Regression for Face Recognition

Transcription

1 Local Rge Regresson for Face Recognton Hu Xue 1,2 Yulan Zhu 1 Songcan Chen *1,2 1 Department of Computer Scence & Engneerng, Nanjng Unversty of Aeronautcs & Astronautcs, , Nanjng, P.R. Chna 2 State Key Lab. for Novel Software echnology, Nanjng Unversty, P.R. Chna Abstract: Rge regresson (RR) for classfcaton s a regularze least square metho to moel the lnear epenency between covarate varables an labels. By applyng approprate technques to encoe the multvarate labels n face recognton as the vertces of the regular smplex whch can separate ponts wth hghest egree of symmetry, RR maps the face mages nto a face subspace where the mages from each nvual wll locate near ther nvual targets. However, as a holstc metho, RR operates rectly on a whole face regon represente as a vector an thus can not effectvely recognze the faces wth llumnaton varatons an partal occlusons. In ths paper, we present a novel algorthm, terme as Local Rge Regresson (LRR). Dfferent from RR, LRR emphaszes on each local face regon matchng rather than the whole. As a result, LRR can not only enhance the robustness to the local varatons by utlzng the spatal an geometrcal nformaton of facal components, but also avo the mensonalty reucton n the holstc RR as a preprocessng. Furthermore, an effcent cross-valaton algorthm s aopte to select the regularzaton parameters n each local regon. Experments on two stanar face atabases emonstrate that the propose algorthm sgnfcantly outperforms RR an the two popular lnear face recognton technques (Egenface an Fsherface). Although we concentrate on rge regresson n ths paper, followng the propose lne of the research, many current mult-category classfers can also be apple n face recognton through combnng the characterstcs of face mages an may be obtan better recognton accuraces. * Corresponng author: el: Ext ; Fax: ; E-mal: s.chen@nuaa.eu.cn (S. Chen), xuehu@nuaa.eu.cn (H. Xue) an lany_1999@nuaa.eu.cn (Y. Zhu)

2 Keywors: Rge regresson (RR); Local nformaton; Local rge regresson (LRR); Face recognton 1. Introucton In the past few ecaes, face recognton has become a hot ssue of research n computer vson communty. Among varous evelope technques n ths fel, appearance-base metho s one of the most wely use technques whch usually represents a face mage as a hgh mensonal vector of pxels[1]. o overcome the ffculty ncurre by hgh menson, a lot of subspace methos have been propose where Egenface[2] an Fsherface[3] are two of the most popular algorthms. Egenface s an unsupervse metho whch utlzes the ea of Prncple Component Analyss (PCA) to project the orgnal hgh mensonal ata onto a low mensonal subspace that can maxmally preserve orgnal mage nformaton[4]. Fsherface s a supervse algorthm whch combnes PCA an Lnear Dscrmnant Analyss (LDA) to extract the most scrmnant features that can maxmally separate the mages of fferent classes n the resultant face subspace. Although Fsherface further ntrouces the class nformaton compare to Egenface, t s affecte heavly by the relatve postons of the labele tranng mages ue to the weakness of LDA. An et al.[1] have ncate that n the mult-category face recognton problems, whle LDA tres to maxmze the between-class stances an mnmze the wthn-class stances smultaneously, the parwse stances can be sgnfcantly unbalance an ths may result n ba performance for classes wth small parwse between-class stances n the reuce subspace. Recently, a new generalze Rge Regresson (RR) metho[1] has been propose to solve the latent problem of Fsherface. Motvate by the fact that the m vertces of a regular m-smplex s the most balance an symmetrc separate ponts n the (m-1)-mensonal space, the metho frst encoes the targets for m stnct nvuals as the m vertces an then apples the rge regresson to map the tranng face mages nto the (m-1)-mensonal subspace so as to the mages from each

3 nvual wll locate near ther nvual targets[1]. Recognton s performe by mappng the new face mages nto the subspace an comparng ts stance to all the targets. However, although RR has yele much better recognton performance than Egenface an Fsherface expermentally, lke the two methos, RR s also a holstc technque whch operates rectly on a whole face regon an neglects the local nformaton. As a result, RR s senstve to the local varatons n face mages, such as llumnaton varatons an partal occlusons. For usng as much local nformaton hen n face mages as possble to relax the nfluence of local varaton for recognton, recently varous local regon matchng technques have been evelope[4-10]. he general ea of local regon matchng technques s to frst locate several facal features (components), an then classfy the faces by comparng an combnng the corresponng local statstcs[11]. Hesele et al.[12] further ncate that comparng the component (local) system an the global systems, the former outperforms the latter n recognton rate larger than 60%[11]. Consequently, n ths paper, we also apply local regon matchng technques nto RR an present a novel algorthm, terme as Local Rge Regresson (LRR). LRR frst parttons an orgnally whole mage nto L equally sze local regons n non-overlappng or overlappng ways, an then collects all those local regons sharng the same orgnal feature components respectvely from the tranng set to compose L corresponng local regon tranng sets. Rge regresson s performe on each of such L local regon sets to rectly tran fferent L classfers by the fferent face features. he new unlabele face mage s entfe by also parttonng nto L local regons n the same way as the tranng phase an classfyng each local regon by the corresponng classfer. he fnal recognton result s obtane by assemblng the total L results from the L classfers an votng. In ths way, not only s the spatal an geometrcal nformaton n a face mage preserve n each local regon, but also the nfluence of local varatons s restrane n several local regons by the classfers votng so as to greatly mprove the recognton accuracy. here are three major contrbutons of the propose LRR. Frst, LRR s more robust to the local varatons than the holstc RR. Secon, LRR can smultaneously

4 tran a set of classfers corresponng to fferent local regons an thus t s qute sutable for parallel computaton to greatly mprove the computatonal effcency of the holstc RR, especally n the large-sze mage cases. hr, the partton of local regons n LRR s nepenent on the menson of the face mage. Consequently, t can avo the mensonalty reucton n the holstc RR as a preprocessng. Furthermore, t can also escape the latent mensonal curse when the menson of the mages s qute large. he rest of the paper s organze as follows. In secton 2, we brefly revew the holstc RR metho. Secton 3 presents the propose LRR. Secton 4 proves expermental results on two face atabases to llustrate the superorty of LRR. Some conclusons are rawn n Secton Rge Regresson (RR) Suppose there are m nvuals for recognton. RR algorthm usually has three parts: labelng tranng mages, learnng classfer an recognton. Frstly, RR chooses the regular smplex vertces as the nvual targets an uses these targets as the multvarate labels of the tranng mages[1]. Let m 1 R ( 1, 2,, 1 2 = m ) are the vertces of one regular m-smplex an = [,,, ]. RR constructs as follows[1]: m 1. Let 1 = [1, 0,, 0] an 1, = 1/( m 1), for = 2,, m. 2. For 1 k m 2, k 2 k+ 1, k+ 1 1 = 1, k = k+ 1, k+ 1 k+ 1, j =, j = k+ 2,, m (1) m k 1 = k, 1 0, k + 1 < + m 1 hen, RR treats the face recognton as a rge regresson problem to locate the mages from each nvual as near ther nvual targets as possble. As a result, the task of learnng classfer n RR s to fn a matrx W that can moel the lnear

5 epenency between the mage x an the label Y, where Y = j f x belongs to the jth nvual, = 1,, n, j = 1,, m. Meanwhle, RR also penalzes the norm of W to reuce the varance of the estmate as the regularzaton term. herefore, the objectve functon (2) of RR s mnmzng n = J ( W) = Y W x + λ W (2) where λ s the regularzaton parameter to balance the bas an varance of the estmate. akng ervatve of (2) wth respect to the W an equalng t to zero, we have the matrx W as 1 ( W= XX +λi) XY (3) where X= [ x1,, xn] an Y= [ Y1,, Yn ]. Fnally, let x be a new mage. RR compares the stances between Wx an the nvual target an entfes x as that wth mnmal stance. 3. Local Rge Regresson (LRR) In ths secton, we propose a novel algorthm to solve the senstvty of local varatons n RR. Relatvely to the holstc RR, here we abuse the termnology,.e. local, to name the propose algorthm as Local Rge Regresson (LRR). Followng the lne of the research n the local regon matchng methos, LRR also nvolves three steps: local regon partton, classfer tranng an classfcaton. It s noteworthy that LRR avos the general menson reucton preprocessng n the classfer tranng phase ue to the partton, an thus t s much smpler Local Regon Partton Generally, there are two fferent technques to mplement the partton, that s, local components an local regons. Local components are areas occupe by the facal components, such as eyes, noses an mouths, an centere nepenently at the component centers; Local regons are local wnows centere at esgnate

6 coornates of a common coornate system[11]. Zou et al.[11] have verfe that comparson of corresponng local regons s better than comparng corresponng facal components. So, n ths paper, we aopt the smplest rectangular regons to partton mages, whch not only are convenently use but also can better preserve the spatal an geometrcal nformaton n the orgnal mages[4-6]. Suppose that there are n W1 W2 mages belongng to m nvuals n the tranng set, an these nvuals possess n 1, n 2,, nm face mages respectvely. Each mage s frst ve nto L equally sze local regons n a non-overlappng way whch are further concatenate nto corresponng column vectors wth mensonalty of W / 1 W L 2. hen we collect these vectors at the same poston of all face mages to form a specfc local regon tranng set, n ths way, L separate local regon sets are forme[5]. hs process s llustrate n Fg. 1. Non-overlappng partton sometmes may ve the relaton between each sub-mage an lea to totally neglect the relaton between local regons. Consequently, we also attempt the overlappng partton way whch can connect the ajacent local regons an combne the fferent nformaton n each regons. he process s llustrate n Fg. 2. In Secton 4, we wll verfy the conjecture that comparson of overlappng regons s better than comparng non-overlappng regons. Local Regon Set 1 Local Regon Set 2 Local Regon Set L Fg.1. he constructon of local regon face mage sets (Images are from the Extene Yale face atabase B[13])

7 Fg.2. he constructon of overlappng local regon face mage set (he mage s also from [13]) 3.2. Classfer ranng After parttonng L local regons, we can apply the rge regresson n each local regon set to fn the corresponng matrx W, 1,, = L. 1 W = [ X ( X ) +λi] X Y (4) where X = [ x 1, x 2,, x ] s the pxel matrx of the th local regon set of the n tranng mages, an Y= [ Y1,, Yn ] s the label matrx as n the orgnal RR algorthm. λ s the regularzaton parameter. Due to the partton nepenent on the menson of the mages, the LRR algorthm avos the menson reucton preprocessng n the orgnal RR an rectly learns the local classfers. Especally, the regularzaton parameter λ s a crucal hyper-parameter n the LRR whch controls the goo generalzaton performance of the trane classfer. Hence, here we wll scuss the opton of λ n etal. A popular way to estmate λ s cross-valaton. In k-fol cross-valaton, the tranng ataset s ranomly splt nto k sjont subsets. A classfer s trane for k tmes on stochastc k -1 subsets an a subset s left out as the valaton set to be use for estmatng the generalzaton error at the same tme[14]. Fnally, the classfer corresponng to the parameter wth the lowest average estmate rsk s chosen. However, the orgnal mplementaton of k-fol cross-valaton trans a prector for each splt of the ata an thus has much expensvely computatonal complexty f k s large[1]. An et al.[1] further evelope an effcent technque for general k-fol cross-valaton of the generalze RR wth

8 multvarate labels. So, here we also aopt ths technque to estmate λ n each local regon set. Generally, for a new mage x, we frst partton t nto L local regons as n the tranng mages. hen the corresponng precte label n each local regon s Y ( x) = ( W ) x = YX ( ) [ X( X) +λi] x 1 = Y X X +λi X x 1 [( ) ] ( ) ( A ) ( X ) x (5) where x enotes the pxel vector of the th local regon n the new mage, an 1 A = [( X ) X +λi] Y (6) In k-fol cross-valaton, we splt the ata nto k approxmately equally sze subsets { x,} nl 1, l = 1,, k. As n [1], we also splt the label matrx Y an A l = nto k sub-matrces as follows: where Y() l Yl,1, Yl,2,, Yl, nl = (1), (2),, ( k ) Y Y Y Y, A = A (1), A (2),, A ( k ) (7) =. Wthout loss of generalty, we leave the lth subset ase as the valaton set. hen the corresponng precte labels Y can be rectly compute as follows: () l cv Y = Y B A, l = 1,, k (8) () l 1 cv () l ll () l where B B B B B B [( X ) X + λi] B1k B2k Bkk k k 1 (9) n n j an B R, for, j = 1,, k. For more etals, the reaers can refer to [1]. j We entfy each local regon x cv of the valaton mages by comparng the stances from the precte labels Y () l to the nvual targets, = 1,, m. cv hen we sum up all recognton errors n the k-fol cross-valaton n the

9 corresponng local regon set an choose the optmal parameter λ wth the mnmal error for each local classfer. In summary, the proceure of classfer tranng n each local regon set can be formally state as follows: Input: he local regon set X = [ x 1, x 2,, x n ], = 1,, L Output: λ an W. 1. Label the multvarate label Y of x as the regular smplex vertces, j = 1,, m; j 2. Choose the regularzaton parameter λ : 2.1. Compute 2.2. Compute 1 X X + λi ; [( ) ] A an B ll from (6) an (9) respectvely; 2.3. Compute the precte label Y from (8), l = 1,, k ; () l cv 2.4. Ientfy the valaton mages x cv by comparng the stances from the precte labels () Y l cv to the nvual targets j, j = 1,, m; 2.5. Sum up all recognton errors n the k-fol cross-valaton an choose the optmal parameter λ wth the mnmal error; 3. Compute W from (4) Classfcaton For an unknown face mage x, we wll classfy t by classfers votng. In ths way, the nfluence of local varatons, such as llumnaton varatons an partal occlusons, wll be restrcte n the several local regons so as to greatly mprove the recognton robustness to the varatons. As escrbe n Secton 3.2, we frst partton x nto L local regons. hen n each local regon, the mage s entty s etermne by comparng the stances from the precte label prouce by the corresponng local classfer to the nvual targets s. Snce one classfcaton result for the unknown mage s generate nepenently n each local regon, there wll be total L results from L local regons. Let the probablty of the mage x belongng to the cth class s[6]

10 P c 1 L qc L = 1 = (10) where q c 1, = 0, f the th sub mage belongs to the cth class otherwse hen the fnal classfcaton result s Ientty( x ) = argmax( P ) (11) 1 c m c 4. Experments 4.1. Face Image Databases We carry out our experments on two face mage atabases: the AR face atabase[15] an the Extene Yale face atabase B[13]. he AR atabase contans 100 nvuals wth fferent facal expressons, llumnaton contons an occlusons. Each nvual has 26 face mages taken n two sessons. he frst sesson has 13 face mages name from 01 to 13, nclung neutral expresson (01), fferent facal expresson (02-04), fferent lghtng (05-07), occlusons wth sunglasses (08-10) an a scarf (11-13) uner fferent lghtng. he secon sesson exactly uplcates the frst sesson two weeks later[11]. For psychophyscal experments have ncate that eye s most mportant for recognton[16, 17], here we omt the pctures occlue by sunglasses. We use pctures n the frst sesson from each nvual as gallery. An our experments are conucte on four probe sets: AR11-13 ( 11, 12 an 13 pctures, occlusons wth a scarf n Sesson 1), AR15-17 ( 15, 16 an 17 pctures, fferent expressons n Sesson 2), AR18-20 ( 18, 19 an 20 pctures, fferent lghtng contons n Sesson 2), AR24-26 ( 24, 25 an 26 pctures, occlusons wth a scarf n Sesson 2). he 2000 mages are all croppe nto the same sze of pxels. he extene Yale face atabase B contans 38 nvuals an aroun 64 near frontal mages uner fferent llumnatons per nvual. All mage ata are manually algne, croppe an then resze to pxels just as n [1]. A ranom subset wth l (l=5, 10, 20, 30) mages per nvual s taken wth labels to form the

11 tranng set, an the rest of the atabase s the testng set[1]. For each gven l, the experments s repeate over 50 ranom splts by usng the matlab ata fles n [13], an the average results are reporte Evaluaton of Classfcaton Performance We compare the propose LRR wth the most popular face recognton methos: Egenface, Fsherface an the orgnally holstc RR on the two face atabases. In LRR, we attempt the fferent partton szes accorng to the fferent szes of the mages n the atabases by cross-valaton an the selecte expermental results are reporte. In the AR atabase, the szes of local regons are 11 8, an 33 24; An n the extene Yale atabase B, the szes are esgnate to 4 4, 8 8 an respectvely. We also attempt the non-overlappng an overlappng partton ways. In the overlappng way, the ajacent local regons overlap each other almost 50%. he corresponng classfcaton results are lste n able 1 an 2 respectvely, where Egenface, Fsherface an the holstc RR all nvolve the menson reucton an the optmal results are reporte. As shown n able 1, LRR s sgnfcantly superor to the other three holstc methos n all the probe sets n the AR atabase, bascally wthn all combnatons of the sze of local regons an non-overlappng or overlappng way. Especally n the AR11-13, AR18-20 an AR24-26 corresponng to fferent lght contons an occlusons wth a scarf, LRR shows the surprsngly hgh robustness to these local varatons an the optmal recognton error rates are less than 50% of those n the other algorthms. he smlar concluson can also be rawn n the extene Yale atabase B n able 2. he face mages n the atabase mostly have pose an llumnaton varatons. LRR also shows the best classfcaton performance corresponng to all the fferent tranng sets. Especally, when the numbers of the tranng mages are smaller, such as n 5ran an 10ran, LRR acheves much better recognton accuraces. Furthermore, t s noteworthy that, the optons of the approprate sze of local regons an overlappng way are stll open problems. However, here we can capture

12 some emprcal observatons about the optons n LRR. Obvously, from ables 1 an 2, n the two atabases, the optmal classfcaton performances of LRR are accomplshe both n the mle partton sze an n overlappng way, whch even excee the other combnatons n LRR over 50%. hese exactly accors wth our conjecture n Secton 3.1, that s, the mle-sze an overlappng partton way can connect more spatal an geometrcal nformaton n ajacent local regons. able 1. Classfcaton performance (error rate %) comparson on the AR face atabase AR11-13 AR15-17 AR18-20 AR24-26 Egenface Fsherface RR LRR 11 8, non-overlappng LRR 11 8, overlappng LRR 22 16, non-overlappng LRR 22 16, overlappng LRR 33 24, non-overlappng LRR 33 24, overlappng able 2. Classfcaton performance (error rate %) comparson on the extene Yale face atabase B 5 ran 10 ran 20 ran 30 ran Egenface Fsherface RR LRR 4 4, non-overlappng LRR 4 4, overlappng LRR 8 8, non-overlappng LRR 8 8, overlappng LRR 16 16, non-overlappng LRR 16 16, overlappng Conclusons In ths paper, we have propose a new face recognton technque LRR base on the nsghts of the orgnally holstc RR. o overcome the senstvty of RR to local varatons, LRR aopts the popular local regon matchng technques. As a result, LRR not only enhances the robustness to the varatons, but also effectvely avos the

13 latent mensonal curse when the menson of the mages s very large. Expermental results emonstrate the surprsngly goo classfcaton performance of LRR. It s worth to note that, although we concentrate on the mprovement of RR n the whole paper, the propose lne of the research about LRR s general. hrough combnng the spatal an geometrcal nformaton of facal components n local way, many current mult-category classfers can also be apple n face recognton an may obtan better recognton performance, whch eserves our future researches. Furthermore, tensor subspace moels are one of the moern research rectons n the face recognton. Many researches have showe that representng the mages as tensors of arbtrary orer can further mprove the performance of algorthms n most cases[18-24]. Consequently, how to generalze LRR to tensor learnng s another nterestng topc for future stuy. Acknowlegment hs work was supporte by Natonal Natural Scence Founatons of Chna ( an ) an Natural Scence Founatons of Jangsu Provnce (BK an BK2008xxx). We also thank Prof. Xaoyang an for benefcal scusson. References [1] S. An, W. Lu, an S. Venkatesh. Face recognton usng kernel rge regresson. CVPR, [2] M. urk an A. Pentlan. Egenfaces for recognton. J. Cogntve Neurosc,3(1): 71-86, [3] P. Belhumeur, J. Hespanha, an D. Kregman. Egenfances vs. fsherfaces: recognton usng class specfc lnear projecton. IEEE rans. Pattern Analyss an Machne Intellgence,19(7): , [4] S. Chen an Y. Zhu. Subpattern-base prncple component analyss. Patter Recognton,37(5): , [5] K. an an S. Chen. Aaptvely weghte sub-pattern PCA for face recognton. Neurocomputng,64: , [6] Q. Hong, S. Chen, an X. N. Sub-pattern canoncal correlaton analyss wth applcaton n face recognton. ACA AUOMAICA SINICA,34(1): 21-30, 2008 (In Chnese). [7] A. Pentlan, B. Moghaam, an. Starner. Vew-base an moular egenspaces for face recognton. CVPR: 84-91, [8] A.M. Martnez. Recognzng mprecsely localze, partcally occlue an expresson varant faces

14 from a sngle sample per class. IEEE rans. Pattern Analyss an Machne Intellgence,24(6): , [9] R. Sngh, M. Vatsa, an A. Noore. Face recognton wth sguse an sngle gallery mages. Image an Vson Computng, [10] X. Geng an Z. Zhou. Image regon selecton an ensemble for face recognton. JCS, [11] J. Zou, Q. J, an G. Nagy. A comparatve stuy of local matchng approach for face recognton. IEEE rans. on Image Processng,16(10): , [12] B. Hesele, P. Ho, J. Wu, an. Poggo. Face recognton: component-base versus global approaches. Comput. Vs. Image Unerstan.,91(1): 6-12, [13] he Extene Yale Face Database B. [14] R.O. Dua, P.E. Hart, an D.G. Stork. Pattern Classfcaton. Wley, [15] A.M. Martnez an R. Benavente. he AR Face Database. CVC echncal Report #24, June, [16] D.I. Perrett, P.A.J. Smth, D.D. Potter, A.J. Mstln, A.S. Hea, A.D. Mlner, an M.A. Jeeves. Vsual cells n the temporal cortex senstve to face vew an gaze recton. Proc. of the Royal Socety of Lonon. Seres B: Bologcal Scences,223: , [17] E. Bart an S. Ullman. Class-base feature matchng across unrestrcte transformatons. IEEE rans. on Pattern Analyss an Machne Intellgence,30(9): , [18] H. Zhou, Y. Yuan, an A.H. Saka. Applcaton of semantc features n face recognton. Patter Recognton,41(10): , [19] D. Xu, S. Yan, D. ao, S. Ln, an H. Zhang. Margnal Fsher analyss an ts varants for human gat recognton an content-base mage retreval. IEEE rans. on Image Processng,16(11): , [20] D. ao, X. L, X. Wu, an S.J. Maybank. General tensor scrmnant analyss an Gabor features for gat recognton. IEEE rans. on Pattern Analyss an Machne Intellgence,29(10): , [21] D. ao, X. L, X. Wu, W. Hu, an S.J. Maybank. Supervse tensor learnng. Knowlege an Informaton Systems,13(1): 1-42, [22]. Zhang, D. ao, an J. Yang. Dscrmnatve localty algnment. he 10th European Conference on Computer Vson (ECCV), [23] D. ao, J. Sun, J. Shen, X. Wu, X. L, S.J. Maybank, an C. Faloutsos. Bayesan tensor analyss. IEEE Internatonal Jont Conference on Neural Networks (IJCNN), [24]. Zhang, K. Huang, X. L, J. Yang, an D. ao. Dscrmnatve orthogonal neghborhoo preservng projectons for classfcaton. IEEE rans. on Systems, Man, an Cybernetcs, Part: B, 2008.