Tone-Aware Sparse Representation for Face Recognition

Transcription

1 Tone-Aware Sparse Representaton for Face Recognton Lngfeng Wang, Huayu Wu and Chunhong Pan Abstract It s stll a very challengng task to recognze a face n a real world scenaro, snce the face may be corrupted by many unknown factors. Among them, llumnaton varaton s an mportant one, whch wll be manly dscussed n ths paper. Frst, the llumnaton varatons caused by shadow or overexposure are regarded as a multplcatve scalng mage over the orgnal face mage. The purpose of ntroducng scalng vector (or scalng mage) s to enhance the pxels n shadow regons, whle depress the pxels n overexposure regons. Then, based on the scalng vector, we propose a novel tone-aware sparse representaton (TASR) model. Fnally, a EMlke algorthm s proposed to solve the proposed TASR model. Extensve experments on the benchmark face databases show that our method s more robust aganst llumnaton varatons. I. INTRODUCTION Face recognton [14], [26] plays an mportant role n many computer vson applcatons, such as access control and securty survellance. Nevertheless, t s stll a challengng problem, snce face mages may be corrupted by many unknown factors, e.g., llumnaton, occluson, expresson, and agng. In the past decades, a number of researchers have been attracted to tackle these dffcultes. A. Prevous Work The face recognton approaches can be manly grouped nto two categores,.e., the subspace (manfold learnng) based and sparse representaton based methods. The tradtonal subspace (manfold learnng) based face recognton methods [19], [2], [22], [8], [3] are based on the assumpton that hgh dmensonal face mages should be le on lower dmensonal subspace or sub-manfold. So far, many subspace learnng (manfold learnng) methods have been successfully appled n face recognton. For example, the classcal Egen-face [19] and Fsherface [2] methods consder all face mage globally, and utlze prncpal component analyss or fsher lnear dscrmnant analyss to learn subspaces. To ntroduce local structures of all face mages, manfold learnng based methods, such as localty-preservng projecton [8] and local dscrmnant embeddng [3], have been proposed. Recently, the spare representaton (or sparse codng) based face recognton methods [20], [25], [23], [24], [7] have been pad much attentons. To the best of our knowledge, the earlest sparse representaton based face recognton has Ths work was supported by Natonal Natural Scence Foundaton of Chna (NSFC No , NSFC No , NSFC No and NSFC No ). L. Wang, H. Wu and C. Pan are wth the Department of NLPR, Insttute of Automaton, Chnese Academy of Scences, Bejng, Chna {lfwang,hywu,chpan}@nlpr.a.ac.cn Fg. 1. The overexposure and shadow examples obtaned from benchmark face databases. been proposed n [20]. In ths method, all tranng mages are frst collaborated together. Then, for each testng mage, t s sparsely represented over all tranng mages. Fnally, the classfcaton result s obtaned by fndng the class whch yelds least representaton error. Improved performances have been reported n [20], as compared wth the classcal subspace or manfold learnng based methods. To mprove robustness n case of occlusons, robust sparse representaton algorthms have been proposed n [24], [7] by ntroducng a weght for each pxel. The occluson pxels are assgned lower weghts, whle other pxels are assgned hgher weghts. The weghts and the codng vector are optmzed alternately. To mprove the raw ntensty feature used n the orgnal sparse representaton, [25], [10], [12], adopt LBP [16] feature, whle [23] uses Gabor [9] feature. In [21], both of features are used. B. Motvaton and the Proposed Method The orgnal sparse representaton based face recognton method can perform well when tranng and testng mages are all acqured n the well llumnaton condtons. Unfortunately, both mages may be corrupted by overexposure or shadow (see Fg. 1 for detal). In such cases, the tone of the mage s changed. To address ths problem, for each pxel on the testng mage, we ntroduce a scalng value. Based on ths, the test mage s transformed to a new tone-free verson, n whch the shadow regon can be enhanced whle the overexposure regon can be depressed. Then, by ntroducng ths tone-free mage nto the sparse representaton framework, we propose a novel tone-aware sparse representaton (TASR) model. The detals and advantages of our proposed TASR are lsted as follows: The proposed TASR model ncorporates tone varatons nto the sparse representaton. Hence, compared wth the classcal sparse representaton based face recognton, our TASR based face recognton s less senstve to llumnaton varatons on face mages. Extensve exper-

2 Fg. 2. An vsual descrpton of our tone-aware sparse representaton (TASR) model. The mages wth blue rectangles are known, whle the mages wth red rectangles are unknown. In our TASR model, the tone-free mage ȳ s sparsely represented by the nput tranng mages X. That s, our TASR need to estmate both the sparse codng vector α and the scalng vector s (scalng mage S). ments on the benchmark face databases also ndcate that our method provdes better adaptablty. An mage guded Laplacan regularzaton (IGLR) s presented to make the scalng mage locally smooth, whch s the matrx form of scalng vector. The man dea behnd IGLR s to transfer the local smoothness from gudance mage to a scalng mage by usng locally lnear regresson. The gudance mage s denoted as a medan face mage, n whch each pxel value s the medan flter result of the correspondng pxel values n the tranng mages. The expectaton maxmzaton lke (EM-lke) optmzaton algorthm s used to mnmze the TASR model. Based on ths algorthm, both the scalng vector and the sparse codng vector can be obtaned alternatvely. The remander of ths paper s organzed as follows. In Secton II, we brefly revew the orgnal sparse representaton based face recognton method. In Secton III, we descrbe our tone-aware sparse representaton (TASR) model n detal. Some expermental results are provded n Secton IV. The conclusve remarks are gven n Secton V. II. OVERVIEW OF ORIGINAL SPARSE REPRESENTATION Denote the tranng mages by X = [X 1, X 2,, X C ] R d n, where X s the subset of tranng mages of class, and C s the number of classes. Note that n s the number of tranng mages, and d = h w s feature length (or the number of pxels n each mage), where h and w are the heght and wdth of the mage, respectvely. For a gven testng mage y R d 1, the orgnal sparse representaton n [20] ams to obtan a codng vector α R n 1 by mnmzng followng equaton, gven by ˆα = mn α y Xα λ α 1, (1) where λ s a weghtng constant, whch balances the reconstructon error and the sparseness of codng vector. After obtanng the codng vector ˆα, the recognton s determned by fndng the least reconstructon error of all classes: IDENTITY(y) = arg mn y X ˆα, (2) where ˆα s the coeffcent vector assocated wth -th class, and ˆα = [ˆα 1 ; ˆα 2 ; ; ˆα C ]. III. TONE-AWARE SPARSE REPRESENTATION (TASR) To ntroduce tone, the testng mage y s transformed by multplyng a scalng vector s R d 1, gven by ȳ = y s, (3) where s the Hadamard product operator, and ȳ R d 1 s the transformed tone-free testng mage. The scalng vector s s used to depress or enhance face mage y, adaptvely. That s, for each pxel n the testng mage, f t s corrupted by shadow, the correspondng scalng value should be large (enhanced), whle f t s overexposured, the correspondng scalng value should be small (depressed). Our tone-aware sparse representaton model (TASR) makes the tone-free mage y s be sparsely represented by the tranng mages X, gven by ˆα, ŝ = mn α,s y s Xα λ α 1 + γreg(s), (4) where Reg(s) s the mage guded Laplacan regularzaton (IGLR) term to make the scalng vector be locally smooth, and parameter γ s a weghtng constant. Fg. 2 shows the flow chart of the proposed TASR n detal. It can be seen that the TASR need to nfer both the scalng vector (or the scalng mage) and the codng vector. In the followng, we frst descrbe the IGLR. Then, we present a expectaton maxmzaton lke (EM-lke) optmzaton algorthm to mnmze the proposed TASR model. A. Image Guded Laplacan Regularzaton (IGLR) To facltate the descrpton of Laplacan regularzaton, the scalng vector s s reshaped nto a h w matrx, called the scalng mage, whch s denoted as S R h w. In practce, the scalng mage S could be smooth locally, snce the shadow or overexposure regons on the testng face mage are pece-wse smooth. To ensure the smoothness of the scalng mage, we propose a new IGLR, n whch

3 the scalng mage S s locally lnear regressed by a known gudance mage G. Now, we consder a local regon (3 3 local wndow for example) Ω centered at -th pxel, namely Ω = { 0, 1, 2,..., 8 } (5) where 0 = s the ndex of -th pxel, and 1, 2,..., 8 are the ndexes of pxels arrangng from top-left to bottom-rght. In ths local regon, the scalng mage values are lnearly regressed by the gudance mage values, gven by S j = w G j + b, j = 0, 1, 2,..., 8, (6) where w and t are the regresson coeffcents n ths local regon Ω. To determne two regresson coeffcents w and b, we defne the followng cost functon: 8 E({S j } 8 ( j=0, w, t ) = Sj w G j b ) 2 + εw 2, (7) j=0 where ε s a small postve constant. The total cost functon s formulated by summng all local costs,.e., n 8 ( (Sj E(S, W, B) = w G j t ) ) 2 + εw 2, =1 j=0 (8) where W and B are the matrx formulatons of all regresson coeffcents. As proved n [13], mnmzng the cost functon E(S, W, B) wth respect to W and B equals to the Laplacan regularzaton s T Ls, that s, J(S) = mn W,B E(S, W, B) = st Ls. (9) Matrx L s a n n Laplacan matrx, whose (j, k)-th entry s ( δ jk 1 ( )) ε (G j µ )(G k µ ). σ2 { {j,k} Ω } Here, δ jk s the Kronecker delta functon, whle µ and σ are the mean and varance of the gudance mage G at the local regon Ω, respectvely. In ths paper, the proposed IGLR s defned as Reg(s) = s T Ls. (10) The IGLR s proposed to transfer the local smoothness from the gudance mage to the scalng mage. Hence, t s mportant to choose an approprate gudance mage G. In ths paper, the gudance mage G s set as the medan face mage, whch s obtaned by followng two steps. The -th entry of gudance vector g s frst calculated by g = med(x. ), (11) where X. the -th row of X, and med(.) s the medan flter operaton. Then, the gudance mage G s obtaned by reshapng the gudance vector g nto h w sze. Fg. 3 gves an example of a gudance mage obtaned from the AR face database. From ths fgure, t can be seen that the gudance mage s smooth, thus, makes the scalng mage also be smooth. Fg. 3. The gudance mage (or medan face mage) obtaned from the AR face database. The sub-fgure (b) gves a 3D vew result. B. Optmzaton of the Proposed TASR Model By substtutng the IGLR of Eqn. (10) nto Eqn. (4), we can rewrte the objectve functon TASR model as ˆα, ŝ = mn α,s y s Xα λ α 1 + γs T Ls + ξ s 1 2 2, (12) where ξ s weghtng constant and 1 s a n 1 vector wth all entres are 1. The term s s used to ensure scale values be close to 1. From Eqn. (12), t can be seen that t s very hard to obtan the codng vector α and the scalng vector s smultaneously. To tackle ths dffculty, a EM-lke method s utlzed by optmzng these two vectors alternately. Step 1: Fxng Scalng Vector s, Calculatng Codng Vector α. For a fxed scalng vector, the objectve functon of the TASR model s reformulated as ˆα = mn α ȳ Xα λ α 1, (13) where ȳ = y s s known. From Eqn. (13), we see that ths model s the same wth the orgnal sparse representaton model. Thereby, t can be solved by the tradtonal l 1 mnmzng methods, such as LASSO [18], FIST [1], LARS [5], and Homotopy [4]. In ths paper, Homotopy s utlzed to solve Eqn. (13), because t s less senstve to the choce of sparse regularzaton parameter λ. Please refer to [4] for more nformaton about Homotopy algorthm. Step 2: Fxng Codng Vector α, Calculatng Scalng Vector b. Keepng the codng vector, the scalng vector s obtaned by mnmzng the followng functon, gven by ŝ = mn s y s ŷ γs T Ls + ξ s 1 2 2, (14) where ŷ = Xα s known. To determne the scalng vector s, we frst reformulate Eqn. (14) as ŝ = mn s Ys ŷ γs T Ls + ξ s 1 2 2, (15) where Y s a n n dagonal matrx obtaned from vector y, namely, Y = dag(y). Therefore, the scalng vector can be obtaned by settng the dervatve of Eqn. (15) wth respect to s s equal to zero, gven by ŝ = (Y T Y + γl + ξi) 1 (Y T ŷ + ξ1), (16) where I s a dentty matrx. In ths method, the ntal scalng vector s assgned to the scalng vector obtaned n the prevous tme. Eqn. (16) s calculated by flterng method proposed n [6].

4 Fg. 4. Vsual comparson example of the AR database. (a), the testng mage. (b), the tone-free testng mage obtaned by our method. (c), seven tranng mages of the 35-th class. (d), seven tranng mages of the 47-th class. (e), the codng vector obtaned by the orgnal SR. (f), the codng vector obtaned by our TASR. The left-bottom mages of sub-fgures (e,f) are the testng mage and tone-free testng mage, respectvely. The testng mage belongs to the 35-th class. The orgnal SR based method msclassfes t as the 47-th class. Detals are gven n the text. C. Summarzaton of TASR Based Face Recognton As mentoned above, the codng vector α and the scalng vector s are obtaned alternately. Then, the recognton s determned by fndng the least reconstructon error of all classes: IDENTITY(y) = arg mn y ŝ X ˆα, (17) where ˆα s the coeffcent vector assocated wth -th class and ˆα = [ˆα 1 ; ˆα 2 ; ; ˆα C ]. Our TASR based face recognton algorthm s summarzed n Algorthm 1. IV. EXPERIMENTAL RESULTS Extensve experments on three publc face databases,.e., AR, Extended Yale B, and CMU-PIE, are performed to evaluate the effectveness of the proposed TASR based face recognton algorthm. We compare our algorthm wth the orgnal sparse representaton algorthm [20], snce t s most related to ours. For convenence, we denote sparse representaton method by SR for short. For our and the orgnal sparse representaton algorthms, the major parameter λ s set to a value of A. Results on AR wthout Occluson Database We frst test our TASR based face recognton algorthm on the AR wthout occluson database [15]. Same to [20], a subset of mages wth only llumnaton and expresson varatons are selected. For each object, seven mages that acqured n the frst sesson are selected for tranng, whle other seven mages that obtaned n the second sesson are selected for testng. All the mages, ncludng tranng and testng mages, are cropped and reszed to Algorthm 1: TASR Based Face Recognton Data: The tranng mage X, and testng mage y. Result: Identty result c. 1 Intalzng scalng vector s = 1, where 1 s a n 1 vector wth all entres are 1; 2 /* MAIN PROCEDURES OF TASR */ 3 for t 1 to maxiter do 4 Calculatng codng vector α t based on the Homotopy optmzaton of Eqn. (13); 5 Calculatng scalng vector s t based on Eqn. (16); 6 Calculatng reconstructon error e t = y s t Xα t ; 7 f e t e t 1 then 8 s t = s t 1 ; 9 α t = α t 1 ; break; end end /* MAIN PART OF FACE RECOGNITION */ c = arg mn y ŝ X ˆα ; The comparatve results are shown n Table I. From ths table, we see that the recognton rate of our method s hgher than that of the orgnal SR. In Fg. 4, we further present a vsual example to compare our TASR wth the orgnal SR. From ths fgure, we see that two persons are very smlar, snce both of them wear glasses. The testng mage belongs to the 35-th class. The orgnal SR msclassfes t as the 47-th class. It could be

5 TABLE I COMPARISONS ON THE AR WITHOUT OCCLUSION DATABASE Algorthm Recognton Rate SR TASR manly caused by the strong reflecton on the glass corner (see the red crcle on Fg. 4 (a)). Note that, the mages of the 47-th class also contan a sample wth strong reflecton on the glass corner (see the red crcle on Fg. 4 (d)). From Fg. 4 (b), we can see that the strong reflecton on the testng mage s weaken by our method, whch makes other regons, such as the mouth regon, become more salent. Fg. 4 (e) and (f) also verfy ths observaton. These two fgures show the codng vectors obtaned by the orgnal SR and our TASR, respectvely. As shown n sub-fgure (e), the codng value of the 324-th mage, whch belongs to 47-th class, s very hgh. However, as shown n Fg. 4 (f), the codng value of the 324 mage becomes lower, whle the codng value of the 240 mage, whch belongs to 35-th class, becomes larger, compared wth the orgnal codng vector of Fg. 4 (e). Although, the codng value of the 240-th mage s stll smaller than the codng value of the 324-th mage, t s suffcent to recognze the testng mage correctly wth the help of other mages belongng to the 35-th class. belongng to the 86-th class, whch further mprove the correct rate of our TASR method. B. Results on Extended YelaB Database The extended Yale B database [11] s consst of 16, 128 face mages of 38 subjects under 9 poses and 64 laboratorycontrolled llumnaton condtons. In ths experment, we select 2, 414 frontal mages. The resoluton of each mage s cropped and reszed nto Fg. 6 gves some examples. From ths fgure, we see that some face mages are greatly corrupted by the shadow. Fg. 6. Image samples from the extended Yale B database. Some face mages are corrupted by shadows. Table II presents the comparatve results on dfferent tranng samples. For each person, we randomly select 5, 10, 15, 20, 25 mages for tranng, and randomly select 300 mages from the rest mages for testng. For each fxed tranng and testng mages, we perform 10 tasks recognton for each algorthm. From table II, we see that the recognton rates of our method are all hgher than those of the SR. Fg. 7. Some face mages, whch are msclassfed by the orgnal SR method. The proposed TASR classfes these face mages correctly. Fg. 5. Another vsual comparson on the AR database. (a), the testng mage. (b), the tone-free testng mage obtaned by our method. (c), seven tranng mages of the 86-th class. (d), seven tranng mages of the 68- th class. The testng mage belongs to the 86-th class. The orgnal sparse representaton based method msclassfes t as the 68-th class. Fg. 5 llustrates another vsual comparson example. The orgnal SR based method msclassfes t as the 68-th class. The man reason s that both the llumnaton condton and the mouth expresson on the orgnal testng mage are both smlar to those on the ffth mage of the 68- th class (see the red rectangle). However, for the mages belongng to the 86-th class, they are dfferent from the testng mage ether n llumnaton condton or n the mouth expresson. Fortunately, by removng the overexposure, the tone-free testng mage could be more close to the the mages Fg. 7 presents some face mages, whch are msclassfed by the orgnal SR method, whle are classfed correctly by our TASR method. From ths fgure, we see that some of face mages are severely corrupted by shadows. For some face mages, t s even hard for human beng to recognze them. The success on these mages ndcates that our TASR model s more robust to shadow than the orgnal SR model. C. Results on CMU-PIE Database In the thrd experment, we compare our TASR wth the orgnal SR on the CMU-PIE database [17]. Ths face database has 41, 368 mages of 68 subjects under 13 dfferent poses, 43 dfferent llumnaton condtons, and 4 dfferent expressons. We select the frontal face mages n the experment. All mages are cropped and reszed to Table III presents the comparsons on dfferent tranng samples. The arrangement of the tranng and testng mages s the same wth the second experment. For each fxed

6 TABLE II COMPARISONS ON THE EXTENDED YALE B DATABASE ON DIFFERENCE TRAINING SAMPLES FOR EACH OBJECT. Algorthm Recognton Rate (Tranng Images=5) Recognton Rate (=10) Recognton Rate (=15) SR ± ± ± TASR ± ± ± Algorthm Recognton Rate (=20) Recognton Rate (=25) SR ± ± TASR ± ± TABLE III COMPARISONS ON THE CMU-PIE DATABASE ON DIFFERENCE TRAINING SAMPLES FOR EACH OBJECT. Algorthm Recognton Rate (Tranng Images=5) Recognton Rate (=10) Recognton Rate (=15) SR ± ± ± TASR ± ± ± tranng and testng mage, we perform 20 recognton tasks for each algorthm. From ths table, we see that our algorthm provdes the hghest recognton rate, especally when tranng mages become large. Furthermore, the varances of our method are lower than those of the orgnal SR, whch ndcates that our method s more stable under dfferent choces of tranng mages. V. CONCLUSION In ths paper, we propose a novel tone-aware sparse representaton model for face recognton. The man contrbuton of ths work s that we ncorporate tone varances nto sparse representaton model, makng t more effectve under unwell llumnaton varances caused by shadow, overexposure or others. Moreover, as a general sparse representaton model, the proposed TASR can be appled to other classfcatonbased applcatons, such as gender recognton and character recognton. REFERENCES [1] A. Beck and M. Teboulle. A fast teratve shrnkage-thresholdng algorthm for lnear nverse problems. SIAM Journal on Imagng Scences, 2(1): , [2] P. N. Belhumeur, J. P. Hespanha, and D. J. Kregman. Egenfaces vs. fsherfaces: Recognton usng class specfc lnear projecton. IEEE Transactons on Pattern Analyss and Machne Intellgence, 19(7): , [3] H. Chen, H. Chang, and T. Lu. Local dscrmnant embeddng and ts varants. In IEEE Conference on Computer Vson and Pattern Recognton, pages , [4] D. L. Donoho and Y. Tsag. Fast soluton of l 1 -norm mnmzaton problems when the soluton may be sparse. IEEE Transactons on Informaton Theory, 54(11): , [5] B. Efron, T. Haste, I. Johnstone, and R. Tbshran. Least angle regresson. Annals of Statstcs, 32: , [6] K. He, J. Sun, and X. Tang. Guded mage flterng. In European Conference on Computer Vson, pages 1 14, [7] R. He, W.-S. Zheng, and B.-G. Hu. Maxmum correntropy crteron for robust face recognton. IEEE Transactons on Pattern Analyss and Machne Intellgence, 33(8): , [8] X. He, S. Yan, Y. Hu, P. Nyog, and H. J. Zhang. Face Recognton Usng Laplacanfaces. IEEE Transactons on Pattern Analyss and Machne Intellgence, 27(3): , [9] J. P. Jones and L. A. Palmer. An evaluaton of the two-dmensonal Gabor flter model of smple receptve felds n cat strate cortex. Journal of neurophysology, 58(6): , [10] C. Kang, S. Lao, S. Xang, and C. Pan. kernel sparse representaton wth local patterns for face recognton. In IEEE Internatonal Conference on Image Processng, pages , [11] K.-C. Lee, J. Ho, and D. J. Kregman. Acqurng lnear subspaces for face recognton under varable lghtng. IEEE Transactons on Pattern Analyss and Machne Intellgence, 27(5): , [12] Z. Le, T. Ahonen, M. Petkänen, and S. Z. L. Local frequency descrptor for low-resoluton face recognton. In IEEE Internatonal Conference on Automatc Face and Gesture Recognton, pages , [13] A. Levn, D. Lschnsk, and Y. Wess. A closed-form soluton to natural mage mattng. IEEE Transactons on Pattern Analyss and Machne Intellgence, 30(2): , [14] S. Z. L and A. K. Jan, edtors. Handbook of Face Recognton, 2nd Edton. Sprnger, [15] A. Martnez and R. Benavente. The ar face database. Techncal Report 24, CVC, June [16] T. Ojala, M. Petkanen, and T. Maenpaa. Multresoluton gray-scale and rotaton nvarant texture classfcaton wth local bnary patterns. IEEE Transacton on Pattern Analyss and Machne Intellgence, 24(7): , [17] T. Sm, S. Baker, and M. Bsat. The cmu pose, llumnaton, and expresson (pe) database. In IEEE Internatonal Conference on Automatc Face and Gesture Recognton, pages 53 58, [18] R. Tbshran. Regresson shrnkage and selecton va the lasso. Journal of the Royal Statstcal Socety, 58(1): , [19] M. Turk and A. Pentland. Egenfaces for Recognton. Journal of Cogntve Neuroscence, 3(1):71 86, [20] J. Wrght, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma. Robust face recognton va sparse representaton. IEEE Transactons on Pattern Analyss and Machne Intellgence, 31(2): , [21] S. Xe, S. Shan, X. Chen, and J. Chen. Fusng local patterns of gabor magntude and phase for face recognton. IEEE Transactons on Image Processng, 19(5): , [22] J. Yang and J. Yang. Why can lda be performed n pca transformed space. Pattern Recognton, 36: , [23] M. Yang and L. Zhang. Gabor feature based sparse representaton for face recognton wth gabor occluson dctonary. In European Conference on Computer Vson, pages , [24] M. Yang, L. Zhang, J. Yang, and D. Zhang. Robust sparse codng for face recognton. In IEEE Conference on Computer Vson and Pattern Recognton, pages , [25] X. Yuan and S. Yan. Vsual classfcaton wth mult-task jont sparse representaton. In IEEE Computer Socety Conference on Computer Vson and Pattern Recognton, pages , [26] W. Zhao, R. Chellappa, A. Rosenfeld, and P. J. Phllps. Face Recognton: A Lterature Survey. ACM Computng Surveys, 35(4): , 2003.