Robust Low-Rank Regularized Regression for Face Recognition with Occlusion

Transcription

1 Robust Low-Rank Regularzed Regresson for ace Recognton wth Occluson Janjun Qan, Jan Yang, anlong Zhang and Zhouchen Ln School of Computer Scence and ngneerng, Nanjng Unversty of Scence and echnology Key Lab. of Machne Percepton (MO), Pekng Unversty Abstract Recently, regresson analyss based classfcaton methods are popular for robust face recognton. hese methods use a pel-based error model, whch assumes that errors of pels are ndependent. hs assumpton does not hold n the case of contguous occluson, where the errors are spatally correlated. Observng that occluson n a face mage generally leads to a low-rank error mage, we propose a low-rank regularzed regresson model and use the alternatng drecton method of multplers (ADMM) to solve t. We thus ntroduce a novel robust low-rank regularzed regresson (RLR 3 ) method for face recognton wth occluson. Compared wth the estng structured sparse error codng models, whch perform error detecton and error support separately, our method ntegrates error detecton and error support nto one regresson model. perments on benchmark face databases demonstrate the effectveness and robustness of our method, whch outperforms state-of-the-art methods.. Introducton Automatc face recognton has been a hot topc n the area of computer vson and pattern recognton due to the ncreasng need for real-world applcatons []. Recently regresson analyss becomes a popular tool for face recognton. Naseem et al. presented a lnear regresson classfer (LRC) for face classfcaton [3]. Wrght et al. proposed a sparse representaton based classfcaton (SRC) method to dentfy human faces wth varyng llumnaton changes, occluson and real dsguse []. A test sample mage s coded as a sparse lnear combnaton of the tranng mages, and then the classfcaton s made by dentfyng whch class yelds the least reconstructon resdual. Although SRC performs well n face recognton, t lacks theoretcal justfcaton. Yang et al. gave an nsght nto SRC and sought reasonable supports for ts effectveness [3]. hey thought that the L -optmzer has two propertes, sparseness and closeness. Sparseness determnes a small number of nonzero representaton coeffcents and closeness makes the nonzero representaton coeffcents concentrate on the tranng samples havng the same class label as the test sample. However, the L 0 -optmzer can only acheve sparseness. Yang et al. constructed a Gabor occluson dctonary to mprove the performance and effcency of SRC [4]. Yang and Zhang proposed a robust sparse codng (RSC) model for face recognton [6]. RSC s robust to varous knds of outlers (e.g., occluson and facal epresson). Based on the mamum correntropy crteron, He et al. [7, 8] presented robust sparse representaton for face recognton. Recently, some researchers have begun to queston the role of sparseness n face recognton [9, 0]. In [], Zhang et al. analyzed the work rule of SRC and beleved that t s the collaboratve representaton that mproves the classfcaton performance, rather than the L -norm sparseness. hey further ntroduced the collaboratve representaton based classfcaton (CRC) wth the non-sparse L -norm to regularze the representaton coeffcents. CRC can acheve smlar results as SRC and sgnfcantly speed up the algorthm. he regresson methods mentoned above all use the pel-based error model [6], whch assumes that errors of pels are ndependent. hs assumpton does not hold n the case of contguous occluson, where errors are spatally correlated [5]. In addton, characterzng the representaton error pel by pel ndvdually neglects the whole structure of the error mage. o address these problems, Zhou et al. ncorporated the Markov Random eld model nto the sparse representaton framework for spatal contnuty of the occluson [5]. L et al. eplored the ntrnsc structure of contguous occluson and proposed a structured sparse error codng (SSC) model []. hese two works share the same two-step teraton strategy: () Detectng errors va sparse representaton or codng, and () stmatng error supports (.e. determnng the real occluded part) usng graph cuts. he dfference s that SSC uses more elaborate technques, such as the teratvely reweghted sparse codng n the error detecton step and a morphologcal graph model n the error support step for achevng better performance. However, SSC does not numercally converge to the desred soluton; t needs an addtonal qualty assessment model to choose the desred soluton from the teraton sequence. 43

2 Low-rank Orgnal Image Image wth Occluson Some recent works pont out that the vsual data has lowrank structure [7]. Most of the etng methods am to fnd a low-rank appromaton for matr completon. However, the rank mnmzaton problem s NP hard n general. In [4], azel et al. appled the nuclear norm heurstc to solve the rank mnmzaton problem, where the nuclear norm of a matr s the sum of ts sngular values. Based on these results, robust prncple component analyss (RPCA) s proposed to decompose an mage nto two parts: data matr (low-rank) and the nose (sparse) [5, 6]. Zhang et al. ntroduced a matr completon algorthm based on the runcated Nuclear Norm Regularzaton for estmatng mss values [7]. Ma et al. ntegrated rank mnmzaton nto sparse representaton for dctonary learnng and appled the model for face recognton [8]. Chen et al. presented a novel low-rank matr appromaton algorthm wth structural ncoherence for robust face recognton [9]. hs paper focuses on face recognton wth occluson. We observe that contguous occluson n a face mage generally leads to a low-rank representaton error mage (.e. error mage or nose mage), as shown n g.. Motvated by ths observaton, we add a rank functon (whch s replaced by a nuclear norm for easy optmzaton) of the representaton error mage nto a regresson model. he model can be solved va the alternatng drecton method of multplers (ADMM) []. he proposed method has the followng merts: () Compared wth state-of-the-art regresson methods such as SRC, RSC and CSR, whch characterze the representaton error ndvdually and neglect the whole structure of the error mage, our model vews the error mage as a whole and takes full use of ts low-rank structure. () Compared wth SSC [] and Zhou s method [5], whch perform the error detecton step and the error support step teratvely but cannot guarantee the convergence of the whole algorthm, our method ntegrates error detecton and error support nto one regresson model, and ts ADMM algorthm theoretcally converges well.. Low-Rank Regularzed Regresson In ths secton, we present the low-rank regularzed regresson model to code the mage and use the alternatng drecton method of multplers [] to solve the model... Low-Rank Regularzed Regresson Suppose that we are gven a dataset of n matrces,, d l A An R d l and a matr Y R. Let us represent Y lnearly by takng the followng form: Y ( ), () where ( ) A A nan,,, n s the representaton coeffcent vector. s the nose (representaton error). Generally, the can be determned by solvng the followng optmzaton problem (lnear regresson): where mn ( ) Y, () s the robenus norm of a matr. o avod over fttng, we often solve the followng regularzed model (rdge regresson) nstead: mn ( ) Y. (3) he above optmzaton problem can be solved n a closed form. he rank of a matr s a good tool to descrbe the structural characterstcs of an error mage. he error (representaton error) mage, as shown n g., s typcally low rank as opposed to the full rank orgnal mage. However, the estng lnear regresson models do not make use of ths knd of structural nformaton. o address ths problem, we ntroduce the low-rank regularzaton to the rdge regresson model. Specfcally, the optmzaton problem s formulated as follows: mn ( ) Y rank(( ) Y), (4) where s the balance factor and s the regularzed parameter. he optmzaton problem of q. (4) s etremely dffcult to solve due to the dscrete nature of the rank functon. ortunately, as suggested by the matr completon method [4, 0, ], the rank mnmzaton problem can be replaced by the nuclear norm mnmzaton problem: mn ( ) Y ( ) Y. (5) In the followng secton, we wll develop the optmzaton algorthm to solve q. (5) by usng the alternatng drecton method of multplers []. Resdual Image gure : he mage wth block occluson s lnearly represented by s dfferent mages and the resdual (nose) mage. 43

3 .. Optmzaton va ADMM In ths secton, we show how the alternatng drecton method of multplers (ADMM) can be adopted to solve q. (5) effcently. or more detals of ADMM, we refer readers to []. o deal wth our problem, the model n q. (5) s rewrtten as: mn,, (6) st.. ( ) Y. he augmented Lagrange functon s gven by: (,, ) L Z + where 0 r Z ( ) Y ( ) Y, s a penalty parameter, Z s the Lagrange multpler, and r( ) s the trace operator. ADMM conssts of the followng teratons: k arg mn ( ) (8) (7) L k arg mn L ( ) (9) k k k k Z Z (( ) Y ) (0) Updatng Denote H [Vec( A),,Vec( An)], g Vec( + Y Z ), where Vec( ) convert matr nto a vector, then the objectve functon L ( ) n q. (8) s equvalent to L ( ) r Z ( ) ( ) Y () ( ) ( Y Z) r ( Y) Z ZZ. he problem of q. (8) can be reformulated as: k arg mn H g. () q. () s actually a rdge regresson model. So we can obtan the soluton of q. () by: ( H H I) H g (3) k Updatng he objectve functon ( ) n q. (9) can be rewrtten by L L ( ) r Z ( ) Y (4) Y Z r Z Y Y r (( ) ) ( ) r ( ) const Algorthm Solvng LR 3 va ADMM Input: A set of matrces A,,A and a n p q matr Y R, the model parameter, the termnaton condton parameter. 0 0 Intalze, Z, whle do ( ) Y Z const where const and const are constant terms, whch are ndependent of the varable. he optmzaton problem q. (9) can be reformulated as k arg mn ( ) Y Z (5) As suggested by [3], the above optmzaton problem s solved by k U S V [ ], where ( U, S, V ) = svd ( ) Y Z. he sngular value shrnkage operator (6) [ S] s defned as [ S ] dag {ma(0, S j j )} j n. (7), Stoppng crteron As suggested n [], the stoppng crteron of the algorthm s: the prmal resdual must be small: k k ( ) Y, and the dfference between successve teratons should also be small: k k k k ma,, where s a gven k k ( ) Y or k k k k ma, ( H H I) H g, k k arg mn Y tolerance. In summary, the pseudo code of our method to solve q. (6) s shown n Algorthm. Algorthm can be nterpreted as usng the two-step teraton strategy for robust face recognton as those used n [5, ]. he step of updatng s actually an error detecton ( ) Y Z, k k k k Z Z Y (( ) ). end whle Output: Optmal representaton coeffcent. 3433

4 step for determnng the representaton coeffcents and representaton errors, and the step of updatng s actually an error support step for determnng the real occluded part. So we can say that LR 3 provdes a unfed framework to ntegrate error detecton and error support detecton nto one smple model. 3. Classfcaton based on Robust Low-Rank Regularzed Regresson We notce that SSC [] adopts a robust sparse representaton model,.e. teratvely reweghted sparse codng n the error detecton step, but our low-rank regularzed regresson (LR 3 ) only uses a smple rdge regresson model for updatng. o further mprove the robustness of the proposed method, n ths secton we borrow the dea of robust regresson to our model and ntroduce a robust low-rank regularzed regresson model (RLR 3 ): mn W(( ) Y) W (( ) Y). (8) where W s a weght matr of the representaton error, and denotes the Hadamard product of two matrces. he robust low-rank regularzed regresson model can be solved by usng the teratvely reweghted process [6]. ach teraton step s to solve a low-rank regularzed regresson problem. Specfcally, gven a test sample Y, we compute the representaton coeffcent va Algorthm and the representaton error of Y n order to ntalze the weght. he representaton error s ntalzed as Y Y n, where Y n s the ntal estmaton of the mages from the gallery set. In ths study, we smply set Y n as the mean mage of all samples n the codng dctonary snce we do not know whch class the test mage Y belongs to. Wth the ntalzed Y n, our method can estmate the weght matr W teratvely. W, s the weght assgned to each pel of the test mage. he weght functon [6] s: W ep( (, j) ), j, j, (9) ep( ( ) ) where and are postve scalars. he convergence s acheved when the dfference between the weghts n successve teratons satsfes the followng condton: ( t) ( t) ( t) W W / W. (0) Based on the optmzaton soluton va the teratve process, we obtan a weghted dctonary B [ B Bn], where B W D,,, n and D s the codng dctonary whch s composed of the tranng samples. he test sample Y s reconstructed as Yˆ jb, j, where j ( ) ( ) s the functon that selects the ndces of the coeffcents assocated wth the -th class. he correspondng reconstructon error of the -th class s defned as: r ( y) W ( Y Yˆ ). () he decson rule s: f r ( y) mn r ( y ), then y s assgned to Class l. able he recognton rates (%) of each classfer for face recognton on the AR database wth dsguse occluson. Methods Sunglasses Scarves CRC LR SRC [] CSR [8] SSC RSC [6] RLR perments In ths secton, we compare the proposed methods LR 3 and RLR 3 wth CRC, SRC, CSR, SSC and RSC. In our eperments, there are fve parameters of the proposed RLR 3. he parameters and n q. (9) follows the suggeston n [6]. he default value for penalty parameter s. Both the balance factor and the regularzed parameter are ntroduced n the followng eperments. 4.. ace Recognton wth Real Dsguse In our eperments, we only use a subset of AR face mage database [4]. he subset contans 00 ndvduals, 50 males and 50 females. All the ndvduals have two sesson mages and each sesson contans 3 mages. he face porton of each mage s manually cropped and then normalzed to 4 30 pels. he frst eperment chooses the frst four mages (wth varous facal epressons) from sesson and sesson of each ndvdual to form the tranng set. he total tranng mages s 800. here are two test sets: the mages wth sunglasses and the mages wth scarves. ach set contans 00 mages (one mage per sesson of each ndvdual wth neutral epresson). he balance factor s 0 and 0 - for the test mages wth sunglasses and scarves, respectvely. he regularzed parameter s able lsts the recognton rates of CRC, SRC, CSR, SSC, RSC, LR 3 and RLR 3. rom able, we can see that RLR 3 acheves the best performance among all the methods. LR 3 also gves better results than CRC. Both RSC and CSR obtan the same results as RLR 3 when the test mages are wth sunglasses. However, the results of SRC and CSR are l 4434

5 sgnfcantly lower than those of RLR 3 when the test mages are wth scarves. In the second eperment, four neutral mages wth dfferent llumnaton from the frst sesson of each ndvdual are used for tranng. he dsguse mages wth varous llumnaton and glasses or scarves per ndvdual n sesson and sesson for testng. he balance factor s 0 - and the regularzaton parameter s he recognton rates of each method are lsted n able. rom able, we can see that RLR 3 sgnfcantly outperforms CRC, LR 3, SRC, CSR, SSC and RSC on dfferent test subsets. SRC and CSR perform well on mages wth sunglasses and poorly on mages wth scarves. SSC gves smlar results as RSC n dfferent cases. Compared to RSC, 3.0%,.0%, 4.0% and 4.6% mprovement are acheved by RLR 3 on four dfferent testng sets. (a) able he recognton rates (%) of each classfer for face recognton on the AR database wth dsguse occluson. Methods Sunglasses Scarves Sesson Sesson Sesson Sesson CRC LR SRC [] CSR [8] SSC RSC [6] RLR ace Recognton wth Random Block Occluson he etended Yale B face mage database [5] contans 38 human subjects under 9 poses and 64 llumnaton condtons. All frontal-face mages marked wth P00 are used n our eperment, and each s reszed to pels. In the frst eperment, we use the same eperment settng as n [] to test the robustness of RLR 3. Subsets and of tended Yale B are used for tranng and subset 3 wth the unrelated block mages s used for testng. Both and are set to 0. g. (a) plots recognton rates of CRC, SRC, CSR, SSC, RSC, LR 3 and RLR 3 under dfferent levels of occlusons (from 0% to 50%). Wth the ncrement of the level of occluson, RLR 3 begns to sgnfcantly outperform the other methods. When the occluson percentage s 50%, the recognton rate of RLR 3 s 0.4%,.6%, 36.9% and 9% hgher than RSC, SSC, CSR and SRC, respectvely. he second eperment settng s smlar to that of the frst eperment. he only dfference s that subset 3 wth nose block mages s used for testng. s 0. and the regularzaton parameter s set to0. he recognton rates of each method versus the varous levels of occluson (from (b) (c) gure : he recognton rates (%) of CRC, SRC, CSR, SSC, RSC, LR 3 and RLR 3 wth the occluson percentage beng from 0% to 50%. (a) the test mages are wth unrelated block mages; (b) the test mages are wth nose block mages; (c) the test mages are wth mture nose. 5435

6 0% to 50%) are shown n g. (b). rom g. (b), we observe that the proposed RLR 3 sgnfcantly outperforms CRC, SRC, CSR, SSC and RSC. he performances of SRC and CSR are not good n ths case. SSC gves good performance when the occluson level s hgher. However, SSC cannot perform well when the occluson level s lower. RSC acheves comparable results when the occluson percentage s lower than 40%. However, the recognton rate of RLR 3 s 6.% hgher than that of RSC when the occluson percentage s 50%. In the thrd eperment, subsets and of tended Yale B are used for tranng and subset 3 wth the mture nose (pel corrupton and block occluson) s used for testng. s and the regularzaton parameter s set to0. he recognton rates of each method wth dfferent level of pel corrupton (and occluson) are shown n g. (c). Although the performance of each method degrades wth the ncrement of the mture nose level, RLR 3 stll acheves the best results among all the methods. he recognton rates of SSC are poor when facng wth the mture noses (pel corrupton and mage occluson). A probable reason s that SSC manly addresses the contguous occluson problem. 5. Conclusons In ths paper, we present a novel low-rank regularzed regresson model and apply the alternatng drecton method of multplers to solve t. he robust low-rank regularzed regresson based classfcaton (RLR 3 ) method s ntroduced for face recognton. RLR 3 takes advantage of the structural characterstcs of nose and provdes a unfed framework for ntegratng error detecton and error support nto one regresson model. tensve eperments demonstrate that the proposed RLR 3 s robust to corruptons: real dsguse and random block occluson, and yelds better performances as compared to state-of-the-art methods. References [] W. Zhao, R. Chellappa, P. J. Phllps et al., ace recognton: A lterature survey, ACM Computng Surveys, vol. 35, no. 4, pp , Dec, 003. [] J. Wrght, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma. Robust face recognton va sparse representaton. I. PAMI, 3():0 7, 009. [3] J. Yang, L. Zhang, Y. Xu, and J.Y. Yang, Beyond sparsty: the role of L-optmzer n pattern classfcaton, Pattern Recognton, 45 (0) [4] M. Yang and L. Zhang. Gabor feature based sparse representaton for face recognton wth Gabor occluson dctonary. In CCV, 00. [5] Z. Zhou, A. Wagner, H. Mobah, J. Wrght, and Y. Ma. ace recognton wth contguous occluson usng Markov random felds. In ICCV, 009. [6] M. Yang, L. Zhang, J. Yang, and D. Zhang, Robust sparse codng for face recognton, In CVPR, 0. [7] R. He, W.S. Zheng, B.G. Hu, and X.W. Kong, A regularzed correntropy framework for robust pattern recognton, Neural Computaton, vol. 3, pp , 0. [8] R. He, W.S. Zheng, and B.G. Hu, Mamum correntropy crteron for robust face recognton, I. PAMI, vol. 33, no. 8, pp , 0. [9] R. Rgamont, M. Brown, and V. Lepett. Are sparse representatons really relevant for mage classfcaton? In CVPR, 0. [0] Q. Sh, A. rksson, A. Hengel, and C. Shen. Is face recognton really a compressve sensng problem? In CVPR, 0. [] L. Zhang, M. Yang, and X. C. eng. Sparse representaton or collaboratve representaton whch helps face recognton? In ICCV, 0. [] X.-X. L, D.-Q. Da, X.-. Zhang, and C.-X. Ren, Structured sparse error codng for face recognton wth occluson, I. IP, 03, (5), [3] I. Naseem, R. ogner, and M. Bennamoun. Lnear regresson for face recognton. I. PAMI, 3():06-, 00. [4] M. azel, H. Hnd, and S. Boyd, A rank mnmzaton heurstc wth applcaton to mnmum order system appromaton, n Proceedngs of the Amercan Control Conference, vol. 6, 00, pp [5]. Candès, X.D. L, Y. Ma, and J. Wrght. Robust prncpal component analyss? Journal of the ACM, 58(3): Artcle, 0. [6] J. Wrght, A. Ganesh, S. Rao, Y. Peng, and Y. Ma. Robust prncpal component analyss: act recovery of corrupted low-rank matrces va conve optmzaton, In NIPS, 009. [7] D. Zhang, Y. Hu, J. P. Ye, X. L. L, and X.. He. Matr completon by truncated nuclear norm regularzaton. In CVPR, 0. [8] L. Ma, C. Wang, B. Xao, and W. Zhou. Sparse representaton for face recognton based on dscrmnatve low-rank dctonary learnng. In CVPR, 0. [9] C.-. Chen, C.-P. We, and Y,-C, rank Wang. Low-rank matr recovery wth structural ncoherence for robust face recognton. In CVPR, 0. [0]. J. Candès and Y. Plan. Matr completon wth nose. Proceedngs of the I, 98(6):95 936, 009. []. J. Candès and B. Recht, act matr completon va conve optmzaton, Commun. ACM 55(6): -9 (0). [] S. Boyd, N. Parkh,. Chu, B. Peleato, and J. cksten. Dstrbuted optmzaton and statstcal learnng va the alternatng drecton method of multplers. oundatons and rends n Machne Learnng, 3():-, 0. [3] J.. Ca,.J. Candès, Z. Shen, A sngular value thresholdng algorthm for matr completon, SIAM Journal on Optmzaton, 00. [4] A. Martnez and R. benavente. he AR face database. echncal Report 4, CVC, 998. [5] K.C. Lee and J. Ho, and D. Dregman, Acqurng lnear subspaces for face recognton under varable lghtng, I. PAMI, 005, 7(5), pp