Neurocomputing 101 (2013) Contents lists available at SciVerse ScienceDirect. Neurocomputing

Transcription

1 Neurocomputng (23) 4 5 Contents lsts avalable at ScVerse ScenceDrect Neurocomputng journal homepage: Localty constraned representaton based classfcaton wth spatal pyramd patches Fumn Shen n,, Zhenmn Tang, Jngsong Xu School of Computer Scence and Technology, Nanjng Unversty of Scence and Technology, Nanjng 294, PR Chna artcle nfo Artcle hstory: Receved 26 Aprl 22 Receved n revsed form 2 August 22 Accepted 2 August 22 Communcated by L. Shao Avalable onlne 23 September 22 Keywords: Face recognton Lnear representaton Localty constrant abstract In ths work, we propose a lnear representaton based face recognton (FR) method ncorporatng localty nformaton from both spatal features and tranng samples. Instead of holstc face mages, the proposed method s conducted on the spatal pyramd local patches, whch are aggregated by a Bayesan based fuson method. The localty constrant on the representaton coeffcents leads to an approxmately sparse representaton, whch effectvely explores the dscrmnatve nature of spatal local features. Dfferent from the sparse representaton based classfcaton (SRC) exposng an -norm constrant on the coeffcents, the proposed localty constraned representaton based classfcaton (LCRC) s formulated wth a computatonally effcent 2 -norm. The proposed method s robust to two crucal problems n face recognton: occluson and lack of tranng data. A smple localty based concentraton ndex (LCI) s defned to measure the relablty of each local patch, by whch not only the heavly corrupted patches but also the less dscrmnant ones are rejected. Due to the use of both local patches and the localty constrant, less tranng data are requred by the proposed method. Based on the localty constraned representaton, we present three algorthms whch outperform the state-ofthe-art on the AR and Extended Yale B datasets for both the occluson and sngle sample per person (SSPP) problems. Crown Copyrght & 22 Publshed by Elsever B.V. All rghts reserved.. Introducton Lnear representaton based face recognton methods attract a lot of nterests recently due to ts effcacy and smplcty. These methods are based on the assumpton that a hgh-dmensonal probe face mage les on a low-dmensonal subspace spanned by the tranng samples of the same subject []. The decson s made by mnmzng the resduals of reconstructng the probe face by a lnear combnaton n terms of tranng samples wth a set of coeffcents. In practce, however, these methods do not perform well enough when tranng samples of each class are not suffcent to model varous potental facal varatons, e.g., changes of expresson, llumnaton, occluson, etc. Recently sparse representaton based classfer (SRC) [2] has obtaned a breakthrough success on face recognton. To address the problem, t takes samples from not one but all subjects to formulate a overcomplete dctonary. Then a sparse representaton s obtaned by a -mnmzaton problem. However, Sh et al. [3] argue that the sparsty assumpton s not supported by the data and the 2 approach s more robust and effcent. Smlarly, t s argued n [4] n Correspondng author. E-mal addresses: fumn.shen@gmal.com (F. Shen), tzm.cs@mal.njust.edu.cn (Z. Tang), xjsxujngsong@gmal.com (J. Xu). Hs contrbuton was made when vstng The Unversty of Adelade. that t s the collaboratve representaton but not the -norm sparsty constrant that n fact boosts the face recognton performance. The proposed collaboratve representaton based classfcaton (CRC) wth regularzed least square n [4] acheves comparatve recognton results wth SRC. Dfferent from SRC, both these two methods have analytcal solutons due to the use of 2 -norm, whch makes them much more effcent. One problem of these methods s that they treat all samples belongng to dfferent subjects equally, and a too redundant dctonary makes these 2 methods [3,4] less dscrmnant especally when usng relatvely less complex features, e.g., local features. Local features are more robust than holstc ones for face recognton on nosy data. Varous local feature descrptors such as hstograms of Local Bnary Patterns [5], Gabor wavelets[6] have been suggested to mprove the robustness of FR systems. Another popular way to extract local features s the modular approach, whch frst partton a whole face mage nto several blocks and then features are extracted and processed ndependently based on these local regons. Usng ths technque, recognton accuraces are largely mproved on data wth occlusons [2,7]. However, these methods fal to explore the localty nformaton of the local features among the tranng samples, and there are no effectve ways to aggregate the results for ndvdual blocks. As an extenson of the bag-of-features model, spatal pyramd matchng (SPM) [8] has made a remarkable success on mage /$ - see front matter Crown Copyrght & 22 Publshed by Elsever B.V. All rghts reserved.

2 F. Shen et al. / Neurocomputng (23) classfcaton. SPM parttons an mage nto ncreasngly fne subregons where hstograms of local features are computed. Inspred by SPM [8], n ths paper we subdvde each mage nto local patches at dfferent spatal pyramd levels. Then the proposed method s conducted on these patches, by whch both the holstc (correspondng to the frst level) and local features wth ncreasngly fne resolutons can be taken nto classfcaton. A Bayesan based fuson method s then proposed to aggregate the ntermedate results wth respect to these patches. The Bayesan method s based on the assumpton that patches wthn a face are ndependent to each other, for smplcty. In ths work, we explore the dscrmnatve nature of localty constraned representaton (LCR) of local patches for dentfyng faces. For local patches, the resdual gap between dfferent subjects obtaned by the aforementoned 2 based methods s small. when face mages suffer from severe dstorton, the test mage s possbly far from some tranng samples (even from the same class). The localty constrant encourages the coeffcents wth respect to nearby samples and smultaneously penalzes the coeffcents correspondng to dstant ones, whch forces the representaton dscrmnant (see examples n Fgs. and 3). Unlke SRC computed by the -mnmzaton, the proposed LCR based classfcaton (LCRC) s formulated wth a weghted rdge regresson problem. It s well known that the conventonal 2 -mnmzaton usually result n dense solutons. However, we show that, wth the localty constrant, the 2 -norm can also lead to a sparse representaton. In [9], the authors argue that localty s more essental than sparsty snce sparsty dose not necessarly lead to localty but localty always ncurs sparsty. Observng that, a classfer based on the sparsty of the coeffcents (denoted as LCRC-Spr) s presented. The dscrmnant nature of the localty constrant s valdated by the hgh accuracy of LCRC-Spr, whch s very close to (sometmes even better) the correspondng resdual based LCRC. Takng advantage of the localty constrant, large representaton coeffcents are concentrated on a small number of entres, whch are expected to manly fall n the same class. Based on that we also represent a class based algorthm C-LCRC, whch computes the representaton coeffcents from one class each tme. Wth a smaller tranng data matrx, C-LCRC s more effcent. The method descrbed n ths paper effectvely addresses two crucal problems n face recognton: Occluson. The presence of contguous occluson s one of the most challengng problems n the context of robust face recognton. Human may easly recognze a famlar person wearng sunglasses or scarves; however, t s a hard job for a computer to automatcally make a correct dentfcaton on an obstructed facal mage. For lnear representaton based methods, outlers ncurred by occluson may dramatcally bas the regresson model and results n a bad representaton. The spatal pyramd partton and Bayesan fuson method proposed n ths paper can sgnfcantly gnore the nfluence caused by occluson. In addton, a localty based concentraton ndex (LCI) s defned to measure the relablty of local patches, by whch not only non-face patches but also the less dscrmnant ones (generc to many subject) are rejected. Lack of tranng samples. In some real face recognton applcatons, very few or even only sngle sample per person (SSPP) s avalable. The LR based methods (e.g., LRC, SRC) usng holstc facal features become unstable n ths stuaton snce they do not have enough samples to represent the ncomng test mage, whch make the resdual large even for the correct subject. The fact much less nherent facal varatons exst n a local patch together wth the localty constrant make t possble that much less samples are necessary for our method to cover these varatons. Moreover, the proposed Bayesan fuson method can effectvely preserve most of the dscrmnant nformaton. Ths s verfed by our experments n Secton 5. The remander of the paper s organzed as follows: In Secton 2, a bref dscusson of related lnear representaton based methods s gven. The proposed method LCRC s descrbed n Secton 3 and another two related algorthms are developed n Secton 4. In Secton 5 the proposed three algorthms and several other methods are evaluated on the AR and Extended Yale B databases. Fnally the concluson and dscusson are offered n Secton Related works In face recognton communty, lnear representaton based methods have been wdely used due to ther effectveness and smplcty. These LR methods are based on the assumpton that any probe mage les on a low-dmensonal subspace [,], and the subspace s spanned by samples from the same subject []. The smlar dea was prevously used n nearest lnear combnatons (NLC) [] and nearest feature lne (NFL) [2]. Suppose we have a data matrx AAR mn contanng the gallery face mages from all the C classes wth each mage n a column vector, then a probe mage yar m belongng to the th subject can be approxmately represented as a lnear combnaton of samples from the same subject: y ¼ A a, ¼,2,...,C, ðþ where a AR N s the coeffcent vector and N s the number of tranng samples of the th class. After seekng a lnear representaton of the test mage y wth respect to each class, the nearest subspace (NS) methods [,3,4] assgn y as the class wth the smallest resdual: denttyðyþ¼arg mn r, ¼,2,...,C, ð2þ where r ¼ :y A a : 2 s the resdual wth respect to class and J J 2 denotes 2 -norm. Based on ths assumpton many varants have been suggested [2 4,7,4]. To obtan the coeffcent vector, NLC [] drectly solve the followng least squares problem: mn:y A a a : 2, ¼,2,...,C: ð3þ In NLC [], the coeffcent vector s obtaned by a pseudo-nverse matrx. Smlarly n [7], the author also formulated face recognton as the above lnear regresson problem, hence termed as lnear regresson classfcaton (LRC), whch has a closed-form soluton ^a ¼ðA T A Þ A T y. Recently sparse representaton classfcaton (SRC) presented n [2] acheves the state-of-the-art performances for face recognton. It ncorporates the compressve sensng technque nto the LR method. Unlke other popular classfcaton methods n face recognton, all mages n the tranng set (not from only one class each tme) are used to represent the query mage. The SRC problem wrtes ^a ¼ arg mnjaj s:t: y ¼ Aa, ð4þ a where J J denotes -norm and aar n. Durng the classfcaton phase the resdual wth respect to subject s defned as r ¼ :y Ad ðaþ: 2, ð5þ where d ðaþar n s a new vector whose only nonzero entres are the entres n a wth respect to class. The -norm s utlzed to force the representaton coeffcents sparse, whch means only a small number of samples truly partcpate n the representaton. To deal wth small dense nose, model (4) s then modfed by

3 6 F. Shen et al. / Neurocomputng (23) 4 5 solvng the followng problem: mnjaj s:t: :y Aa: a 2 re, ð6þ where E4 s the error tolerance. Ths method s shown to be very robust to random pxel corrupton [2]. It s well known that wth an approprate parameter lz we can rewrte (6) to an equvalent unconstraned form: mn:y Aa: 2 a 2 þljaj, ð7þ whch s known as the lasso problem or bass pursut denosng problem n sgnal processng communty. Several effcent algorthms for the lasso are avalable, and [5] provde a comparatve study. In the flowng sectons, we refer to (7) nstead of (6) as the SRC problem. More recently a smlar method wth SRC s suggested n [3], where the dfference s that the regularzaton s gnored. Due to the use of only 2 -norm, ths method can effcently deal wth hgh dmensonal data (more than ). Another method proposed n [4] called collaboratve representaton based classfcaton (CRC) smply replace the constrant n SRC (7) wth the 2 constrant, and obtans comparatve performances usng egenfaces [6]. It wrtes mn:y Aa: 2 a 2 þl:a:2 2, ð8þ whch can also be effcently solved by an analytcal soluton a ¼ðA T AþlIÞ A T y, where IAR nn s the dentty matrx. Besdes the fact that all these three methods utlze an effcent 2 -norm nstead of the -norm, they acheve comparatve (or even better) performances wth SRC. Moreover, [3] argues that the sparsty assumpton for SRC s not supported by the data and [4] argues that t s the collaboratve representaton but not the -norm sparsty constrant that n fact mprove the face recognton performance. Yang et al. [7] gve a reasonable support for the effectveness of SRC; however, argue that t s closeness but not sparsty acheved by the -optmzer that guarantees the effectveness of SRC. In ths work, we wll re-examne the role of and 2 n classfcaton. Although these methods mentoned above can effectvely handle small nose, they stll do not perform well on datasets wth severe contguous occlusons. Wth mages parttoned nto several blocks [7,2], both LRC and SRC acqured much better results. However, both the fuson method dstance-based evdence fuson (DEF) n [7] and the votng strategy n [2] lose much dscrmnant nformaton. In ths paper, we show that a better fuson method can sgnfcantly mprove the performance. Recently several methods consder face recognton wth nosy data as robust regresson problems, where the squared resduals are replaced wth a robust functon (such as Huber loss functon n [8]). In [9,2] the robust maxmum correntropy crteron (a specal case of M-estmator), whch can effectvely deal wth non-gaussan nose, was developed n the regularzed sparse representaton framework. Both two-stage sparse representaton (TSR) [2] and robust sparse codng (RSC) [22] teratvely learn a robust metrc to suppress the nfluence caused by outlers. Good results have been obtaned by both these two methods on several datasets. In ths paper we focus on the lnear method and the robust verson can be easly extended. There are several other prevous methods related n ths category, such as the nearest feature lne (NFL) method proposed by L and Lu [2]. A bref revew of these lnear representaton methods s gven n [9]. The proposed method for face recognton s manly based on the Bayesan fuson of spatal pyramd features. In addton to SPM [8], pyramd methods have been wdely used n computer vson problems. For example, a pyramd feature descrptor called PHOG was developed n [23] based on SPM and the hstogram of gradent orentaton (HOG) [24]. Recently another pyramd feature descrptor PCOG was proposed n [25], whch conssts of a correlogram of orentaton gradents over sub-regons at dfferent resoluton levels. PCOG was appled n [25,26] for the human moton classfcaton and acton segmentaton problems. Our method s largely nspred by the followng two works. A codng scheme called localty-constraned lnear codng (LLC) was recently proposed n [27] and acheved mpressve performances wth a lnear SVM classfer for mage classfcaton. LLC s a fast mplementaton of the local coordnate codng (LCC) [9] whch approxmates the hgh dmensonal nonlnear functon by a global lnear functon wth respect to a local coordnate codng scheme. The man dfference between our method and these two methods s that they are formulated for the mage codng or nonlnear functon learnng problem whle our method s to explore the dscrmnatve nature of localty for local patches for face recognton. 3. Localty constraned representaton for spatal pyramd patches 3.. Localty constraned lnear representaton Recall that n the SRC formulaton (7), the same weght parameter l s used for all regresson coeffcents. However, ths constrant dose not necessarly hold n practce. 2 Intutvely the coeffcents correspondng to less relevant predctors should be penalzed, whereas the most relevant predctors should be well kept n the regresson model. Penalzng the coeffcents by dssmlarty between the probe mage and the tranng samples provdes a meanngful way, whch wrtes mn:y Aa: 2 a 2 þljwaj, ð9þ where W s a dagonal matrx wth ts th dagonal entry w the dstance dðy,x Þ: Jy x dðy,x Þ¼ J 2, ¼,...,n: ðþ max ðjy x J 2 Þ Here x s the th column of A representng a tranng sample. The wdely used heat kernel functon [3,32,27] expðjy x J 2 =sþ, where s4 s the kernel sze, can also be used as the dstance. However, one may need to choose the parameters l and s carefully such that only the most reconstructve neghbours of the test mage can be well kept. We fnd n practce () works well and wth that the algorthm s not senstve to l, whch s always set as n our experments. It s clear that wth the weght parameter W, coeffcents wth respect to tranng samples far away from the test mage are penalzed heavly and encouraged to be zeros, and those coeffcents wth respect to samples close to the test mage wll be well kept. Ths s reasonable because, ntutvely n a subspace spanned by tranng samples, the query data pont s more possbly reconstructed by ts neghbours. Ths s the key dea of locally lnear embeddng (LLE) [33] whch assumes each data pont and ts neghbours le on or close to a locally lnear patch of an underlyng manfold. Another varable selecton method adaptve lasso [29] assgn the weght parameter w n (9) as =9 ^a 9 y, where y4 and a s the th element of the least square (LS) soluton ^a ls ¼ðA T AÞ A T y. Wth an approprate selecton of y and l, the adaptve lasso has the 2 Actually ths constrant may make the lasso estmators largely based [28] (toward zeros). In general the lasso shrnkage s not consstent [29,3]. More theoretcal analyss can be found n [28 3].

4 F. Shen et al. / Neurocomputng (23) oracle property [28] under some assumptons [29], whch ensure the estmaton s consstent and unbased. However, the LS soluton can be easly nfluenced by nose or outlers. For face recognton, LS may lead to extremely dstorted estmaton especally there exst occlusons or corruptons n face mages. Lke SRC (7), however, wth the use of -norm problem (9) s not very effcent n practce for face recognton systems. Several prevous works [3,4] argue that -norm s not necessary for classfcaton, therefore n ths paper we relax (9) wth 2 -norm n the constrant as follows: mn:y Aa: 2 a 2 þljwaj2 2 : ðþ Formulaton () s a weghted rdge regresson problem whch can be effcently solved wth a closed-form soluton: a ¼ðA T AþlW T WÞ A T y: ð2þ CR due to the localty regularzaton. Here sparse means most tranng samples are assocated wth nearly zero (not exactly zero) coeffcents and only a small number of samples (of subject n the example) are wth large coeffcents (the largest coeffcent for CR and LCR s about.22 and.4, respectvely). Fg. b shows the resduals wth respect to dfferent subjects for varous representaton methods. Consstent wth the coeffcents shown n Fg. a, all these three methods have the smallest resdual for subject. Followng [2], the ratos of the two smallest resduals are compared to show the dscrmnant ablty of these representaton methods. We can see that the rato between the two smallest resduals (correspondng to two subjects) obtaned by LCR (6.8) are much hgher than that by CR (2.94) and even hgher than that by SR (3.94), whch means n ths case LCR s more dscrmnant than the other two. Ths observaton confrms the argument n Secton that localty provdes useful nformaton for recognton An Bayesan nterpretaton Followng an dea from [34], we gve the above problem a Bayesan nterpretaton. Wth a Laplacan pror assumpton on the parameter a,.e., f ða Þ¼ð=2gÞexpð 9a 9=gÞ, where g ¼ =lw,(9) s actually equvalent to a maxmum a posteror (MAP) estmaton of a lnear model wth Gaussan dstrbuted nose error. Smlarly f we relax the Laplacan pror assumpton on the parameter to a Gaussan one, we arrve at the weghted rdge regresson problem (). Wth a smaller scale parameter g (correspondng to a larger dstance w n ()), the Gaussan densty functon put more mass near zero, whch makes the tranng samples far away form the probe mage more possbly assocated wth zero or near zero coeffcents. Therefore the weghted rdge regresson problem (9) s expected to produce an approxmately sparse estmaton whch wll be shown n Fg Comparson wth other lnear representaton methods Next let us compare the localty constraned representaton (LCR) and other two related methods: sparse representaton (SR) [2] and collaboratve representaton (CR) [4] through an example. Fg. shows a face mage from the frst subject of the AR dataset and ts representaton coeffcents n terms of all tranng samples (see descrpton n Secton 5.3) computed by dfferent methods. It s clearly seen that although both LCR and CR have a weaker sparse representaton than SRC, all these three methods correctly concentrate the largest coeffcents on the correct subject. Noteworthy s the coeffcents obtaned by LCR are sparser than that by 3.2. Spatal pyramd local patches and Bayesan based fuson Face recognton becomes far more challengng n the presence of occlusons, whch may dramatcally bas the estmatons through tradtonal technques, such as least squares (LS). The nverse effect of occluson can be sgnfcantly allevated by utlzng the spatal nformaton of face mages. In ths work, the proposed method n the prevous secton s conducted on the spatal pyramd local patches. SPM [8] parttons an mage nto ncreasngly fne sub-regons where hstograms of local features are computed. Smlarly we subdvde each mage nto 2 2 non-overlappng blocks at dfferent levels, ¼,,...,L. Then totally T ¼ P L ¼ 4 patches are generated from each mage. We refer to the correspondng tranng data and test sample of the jth patch from level as A ðj Þ and y ðj Þ. Fg. 2 llustrates a three-level pyramd for parttonng a face mage. Wth the precomputed localty weght matrx W ðj Þ, each patch s processed ndependently, ^a ðj Þ ¼ mn:y ðj Þ A ðj Þ a: 2 a 2 þljwðj Þ aj 2 2 : ð3þ For matchng problems SPM aggregates all levels by weghtng each level wth =2 L, whch s nversely proportonal to ts level wdth. Through that a smaller weght s assocated wth a larger subregon whch nvolves ncreasng dssmlar features [8]. For face recognton, we aggregate these levels n another way. All the patches are frst downsampled nto the same sze and then aggregated wth ther correspondng reconstructon errors (resduals). Ths s more straghtforward because t s resduals that.4 LCR CR SR LCR CR SR.5 rato: rato: rato: Fg.. Representaton of a downsampled 2 2 mage from subject n the AR dataset by LCR, CR, and SR. (a) Coeffcents and (b) resduals wth respect to tranng samples n dfferent subjects. The rato of the two smallest resduals s shown on the top of each chart of (b).

5 8 F. Shen et al. / Neurocomputng (23) 4 5 level level level 2 Algorthm. Localty constraned representaton based classfcaton (LCRC). make the fnal classfcaton decson. Intutvely we want to penalze those patches wth larger resduals, whle patches producng smaller resduals whch s more mportant for classfcaton should be well kept. A heurstc method to measure the score of patch t s s ðtþ ðyþ¼expð br ðtþ ðyþþ, ¼,...,C, ð4þ where b4 s the scale parameter and r ðtþ ðyþ s the resdual produced by the tth patch wth respect to subject, r ðtþ ðyþ¼:y ðtþ A ðtþ d ð ^a ðtþ Þ: 2 : ð5þ Smlar as (4), Heetal.[9] has prevously used the Gaussan kernel functon to form a correntropy-based sparse model, whch s showntobeveryrobustndealng wth non-gaussan nose and large outlers. At the classfcaton phase, we then sum the scores of all the patches and get the dentty of y as X T denttyðyþ¼arg max s ðtþ ðyþ: ð6þ t ¼ We now suggest another Bayesan based fuson method. Suppose we have T patches for each test mage: y ¼fM,M 2,...,M T g.the maxmum a posteror (MAP) estmaton of the class label c of y s as follows: ^c ¼ arg max Pðc 9M,M 2,...,M T Þ, ¼,...,C: ð7þ Wth an unform pror for all classes, the above equaton wrtes ^c ¼ arg max PðM,M 2,...,M T 9c Þ: ð8þ For smplcty, we assume that the patches wthn a face are ndependent to each other as n [35,36], then ^c ¼ arg max Y T t ¼ PðM t 9c Þ: ð9þ Here PðM t 9c Þ represents the lkelhood of the patch M t whch s from class. It s natural that we can model the lkelhood by the correspondng resdual as n (4), PðM t 9c Þ¼expð br ðtþ Þ, so we obtan ^c ¼ arg max whch s block: block: 2 5 block: 6 2 Fg. 2. Illustraton of mage partton n a three-level spatal pyramd. Totally 2 patches are obtaned: from level, 4 from level, and 6 from level 2. Each patch s assgned a block label from to 2 by row major order. The example mage s from the frst subject n the AR dataset. Y T t ¼ ^c ¼ arg max exp expð br ðtþ Þ, ð2þ b XT t ¼ r ðtþ! : ð2þ Ths s smlar to (4) and actually n practce we fnd these two fuson methods acheve very smlar performances. We descrbe the procedure of the proposed method n Algorthm. : Input: Partton each mage nto T local patches at dfferent spatal pyramd levels as descrbed n Secton 3, and we get the tranng data matrces A ðtþ AR mn, t ¼,...,T and test patch vectors y ðtþ AR m, t ¼,...,T. Set the regularzaton parameter l4 and the scale parameter b4. 2: Normalze y and columns n A ðtþ to be 2 -norm unt vectors. 3: For each patch, compute the dagonal localty matrx W ðtþ wth ts entres: w ðtþ ¼ Jy ðtþ x ðtþ J 2 =Q, ¼,...,n. Here Q normalze W ðtþ to have maxmum entry value. 4: For each patch, compute the representaton coeffcents: ^a ðtþ ¼ðA ðtþt A ðtþ þlw ðtþt W ðtþ Þ A ðtþt y ðtþ, and the resduals correspondng to dfferent classes r ðtþ by Eq. (5). (For datasets wth heavy occlusons, frst dscard those unrelable patches usng the valdaton method descrbed n Secton 3.4.) 5: Output: dentfy y by (6) or (2). It s not surprsng that by parttonng mages nto patches at dfferent pyramd levels one can obtan a more robust estmaton, snce occluson exstng n one patch wll not affect estmatons on other patches. For an ncomng test mage wth occluson (see Fg. 2 for example), we get several patches (4 patches n the example) wthout or very small occluson. Wth these clean patches, one can obtan a more accurate estmaton than that just usng the whole mage. In addton, by ths partton scheme and fuson method both holstc nformaton (from level ) and ncreasngly local nformaton (from sub-regons at level to L) are taken nto account for classfcaton. To elmnate the mpact of sunglasses and scarves occluson, the modular approach s used n [2,7], whch smply parttons the mage nto blocks and then aggregate results of these ndvdual blocks by majorty votng or the competng method dstance-based evdence fuson (DEF). However, both these two strateges only use nformaton from part of these blocks for classfcaton. For the votng method [2], all blocks whch lead to dssmlar class labels wth the majorty one n the classfcaton phase are dscarded. If the mage s heavly corrupted, t s lkely that the clean patches are dscarded, whch makes votng unstable. Moreover ths method get the fnal decson based on the ntermedate decsons (class labels) nstead of the more nformatve resduals. For the DEF method [7], whch actually use only one block wth the smallest resdual and useful nformaton from all other blocks s lost. Indeed t s necessary to reject the heavly corrupted patches and n the meanwhle to effectvely fuse nformaton from the remanng ones. In Secton 3.4 we descrbe an effectve way to automatcally reject the nvald patches due to the localty nformaton Sparsty nduced by localty for local patches and ts dscrmnant nature Let us frst see an example demonstratng the effectveness of LCR for local patches. Fg. 3 shows a comparson of LCR, CR, and SR for both a clean patch and a corrupted one from pyramd level 2 of a 2 2 dmensonal mage as n Fg. 2 (see settngs n Secton 5.2). For the clean patch, Fg. 3a shows that both LCR and SR have a sparse 3 representaton and concentrate the large coeffcents on a few entres. In contrast, CR has a dense representaton where 3 We regard a lnear representaton sparse f ts large coeffcents are concentrated on only a small fracton of entres and all other coeffcents are zeros or nearly zeros.

6 F. Shen et al. / Neurocomputng (23) LCR CR SR LCR CR SR rato: rato: rato: LCR.2 LCR.5.5 rato:.6 CR SR CR SR rato: rato: Fg. 3. Comparson of dfferent representaton methods: LCR, CR and SR for local patches. Coeffcents for (a) patch 6 and (c) patch 2 of the same mage shown n Fg. 2. The resduals of (b) patch 6 and (d) patch 2 wth respect to tranng samples of dfferent subjects, and the ratos of the two smallest resduals obtaned by dfferent methods are shown on the top. varaton of the coeffcents wth respect to dfferent subjects s small, and the largest coeffcent s only.27 whch s far smaller than that of SR (.34) and LCR (.44). Ths observaton demonstrates that besdes the -norm constrant, the 2 -norm constraned by localty can also lead to sparsty. And localty results n sparse representatons n a more natural way: test mages are more lkely to be represented by t neghbours. Furthermore, localzaton of the representaton coeffcents s also helpful n classfcaton. From Fg. 3b we can see that LCR obtans a larger rato of the two smallest resduals (5.34) than that by SR (2.72) and CR(.6). Consstent wth the example shown n Fg., ths result further shows the dscrmnatve nature of localty. Although CR correctly assocates ts smallest resdual wth the test subject (subject ), the gap between resduals correspondng to dfferent subjects s very small. Usng only the 2 -norm regularzaton wthout localty constrant CR dose not perform as well as LCR. On the heavly corrupted patch, all these methods fal. The dense coeffcents (Fg. 3c) provde lttle nformaton for classfcaton, whch s valdated by the correspondng resduals shown n Fg. 3d. Apparently the heavly corrupted patch s not relable for classfcaton because t may be relatvely closer to an unrelated subject (see the resdual for subject 38 n Fg. 3d). We next descrbe an effectve way to measure the relablty of a patch through the localty nformaton Valdaton based on localty concentraton ndex In [2], Wrght et al. present a sparse representaton based valdaton method, whch rejects an nvald mage f the proposed sparsty concentraton ndex (SCI) of ts coeffcent vector a s below a threshold. Ths method s based on the argument that a vald test mage should have a sparse representaton whose nonzero entres concentrate mostly on one subject [2]. Smlarly, we assume that a vald patch should be close to some samples belongng to the same class and far away from those n other classes. A local patch far away from (or not close to) patches of any subject s expected to be less helpful for classfcaton and should be dscarded. Ths usually happens when a local patch s heavly corrupted. Wth the precomputed localty vector w ¼ dagðwþ of a test mage, the followng localty concentraton ndex (LCI) s defned to measure the relablty of an mage (patch): Defnton (localty concentraton ndex (LCI)). The LCI of a localty vector war n s defned as LCIðwÞ¼ C mn Jd ðwþj JwJ A½,Þ: ð22þ If LCIðwÞ¼, the test mage s evenly far away from (or close to) all classes, and f LCIðwÞ s nearly, the test mage wll be very close to mages from at least one subject. 4 An mage or patch s 4 Note that n the second stuaton, the local patch s possbly very close to more than one subject and n that case ths patch seems not dscrmnant between these subjects. However, ths patch s expected to be effectvely represented by the tranng patches from these nearby subjects and ths patch s also taken nto classfcaton.

7 F. Shen et al. / Neurocomputng (23) Fg. 4. LCI values for the example mage wth sunglasses occluson n Fg. 2. All the heavly corrupted blocks ( 3) are assocated wth low LCI values whle patches wthout corruptons (4 9 and 7 2) are wth hgh LCI values. The clean patches 5 and 6 wth low LCIs are regarded less dscrmnant. regarded nvald and rejected f LCIðwÞo ¼ t, where t4 s the threshold. ð23þ Recall the example n Fg. 2, all the tranng data and the face mage wth sunglasses dsguse are frst reszed to a resoluton of 2 2 and then subdvded nto local patches n a 3-level spatal pyramd. The localty vector w of each patch aganst the correspondng tranng patches of all classes s computed. Fg. 4 shows the LCI values of these totally 2 localty vectors. We can clearly see that the patches wth heavly corruptons are all assocated wth low LCI values. Before classfcaton rejectng the unrelable patches wth low LCI values wll be helpful to robust face recognton. In Secton 5.2 mproved accuraces are obtaned usng ths method on the AR dataset wth sunglasses and scarf occlusons. Note that LCI values for some patches wthout corruptons (patches 5 and 6 n the example) are also possbly low, and that s because the localty varaton of ths patch between some subjects s relatvely small, whch means these patches are less dscrmnant even they are clean. 4. Classfcaton based on localty constraned representaton Besdes the algorthm shown n Secton 3.2, n ths secton we propose another two algorthms based on the localty constraned representaton. 4.. Sparsty decson rule based classfcaton As shown n prevous sectons, localzaton of the representaton coeffcents provdes useful nformaton for classfcaton. In ths secton we desgn a classfer drectly based on the localty nduced sparse coeffcents. Ideally the coeffcent vector a of a probe mage y should concentrate a small number of ts largest entres on the tranng samples from the same subject. Based on that, we assgn y to the class wth the largest coeffcent vector n terms of 2 -norm: denttyðyþ¼maxjd ð ^aþj 2, ¼,...,C: ð24þ For the patch based algorthm, we can easly modfy Algorthm by substtutng steps 5 and 6 wth the output, X T denttyðyþ¼max Jd ð ^a ðtþ ÞJ 2, ¼,...,C: ð25þ t ¼ We refer to ths algorthm as LCRC-Spr n the followng sectons. In Secton 5, we wll show that the classfer perform close to (and n some cases even slghtly better than) the resdual based one. As the example llustrated n Secton 3, ths agan shows the dscrmnatve nature of localty n face recognton Classfcaton usng data from homo-class Most of the methods we dscussed n the former sectons are based on collaboratve representaton,.e., takng tranng samples from all subjects to reconstruct the probe mage. Ths method s helpful especally when tranng data sze of each class s small, whch takes advantage of the fact that face mages from dfferent subject share smlartes [4]. A comparatve study about the relatonshps of collaboratve (sparse) representaton based classfcaton wth the class based nearest neghbour (NN) and nearest subspace (NS) s gven n the supplementary materal of [2]. Dfferent from these methods and the method descrbed n Secton 3, we propose another algorthm based on homo-class classfcaton. As mentoned above, LCRC need much less tranng samples due to the use of local patches. Moreover, the localty constrant effectvely concentrate the large representaton coeffcents of a vald test mage (patch) on ts neghbours, whch are expected to manly fall n the same class (see the example shown n Fg. 3). We wll see that drectly reconstructng the test mage by samples from only one class each tme does not sgnfcantly affect the performance of LCRC n most cases. Gven tranng data of each class A and a test mage y, the class based localty constraned representaton based classfcaton (C-LCRC) wrtes ^a ¼ arg mn:y A a a : 2 2 þljw a J 2 2, ¼,...,n, ð26þ where W s the localty matrx based on class. Then we get the resdual wth respect to class, r ðyþ¼:y A ^a J 2, ¼,...,n: ð27þ Smlar as (4), the patch based algorthm can be easly obtaned by modfyng (27) as s ðyþ¼ XT t ¼ expð b:y ðtþ A ðtþ ^a ðtþ J 2 Þ, ¼,...,n, ð28þ where s ðyþ s the aggregated score of y wth respect to class. Here A ðtþ and y ðtþ are the parttoned tranng and test data of patch t, and a ðtþ s the correspondng estmated coeffcents. After gettng scores correspondng to all classes, y s then assgned to the class wth the hghest score. In Secton 5, we wll show that C-LCRC also acheves hgh accuraces on databases n varous condtons. In addton, by solvng a set of small-sze problems nstead of a large problem, the proposed C-LCRC becomes even more effcent than LCRC.

8 F. Shen et al. / Neurocomputng (23) 4 5 Computaton tme (s) SRC Hom CRC LCRC C LCRC LCRC Spr LRC Fg. 5 shows a comparson of the runnng tme varous methods need to recognze a face from 38 subjects enrolled n the Extended Yale B database wth ncreasng dmensons: 25, 5,, y, 64. It s clearly seen that the class based algorthm s much faster especally on hgh dmensonal data. Note that SRC s mplemented usng the effcent Homotopy method [37], denoted as SRC-Hom n Fg Expermental results 2 3 Dmenson Fg. 5. Comparson of computaton tme (n seconds) of varous methods to recognze one face mage. Note that SRC s mplemented usng the effcent Homotopy methods [37], denoted as SRC-Hom. In ths secton, we evaluate the proposed three algorthms: LCRC, LCRC-Spr, C-LCRC on publc benchmark databases for face recognton. We wll frst demonstrate the robustness of our methods to contguous occluson: both artfcal and natural. And then we wll show the effcacy of the proposed method wth nsuffcent tranng data or SSPP. Several state-of-the-art methods are also performed for comparson. As well as the methods dscussed n the former secton, we wll also compare our methods wth extended SRC [38], whch s proposed most recently for FR problem wth nsuffcent tranng samples. Apart from the orgnal tranng samples, the method constructs an extra ntra-class varant dctonary whch also partcpates n the sparse representaton. Ths method obtans superor results than the orgnal SRC on several datasets (see detals n [38]). The code for TSR s obtaned from the authors [2]. We mplement SRC and extended SRC usng Homotopy 5 [37,39] due to ts accuracy and effcency [5] for the lasso problem (7). We set l ¼ : for these two algorthms. Due to the settngs n [2,7], the modular methods for SRC and LRC are carred out wth mages parttoned nto 4 2 blocks, and the votng and DEF algorthms are used to combned results for these blocks, respectvely. For CRC, we set l ¼ : n=7 accordng to the authors [4]. We set l ¼ for our methods n all experments unless otherwse specfed. We set b as for the proposed LCRC 6 and LCRC-Spr and for the class based C-LCRC. In all our experments, we set three spatal pyramd levels,.e., L¼ Face recognton wth random block occluson In ths secton, we evaluate our methods on face recognton problem wth artfcal contguous occluson. Followng [2], a square monkey face s placed on each test mage at a random locaton whch s unknown to the algorthms. Two occluded example samples are shown n Fg. 6. Specally we use the Extended Yale B database [4], whch conssts of 244 frontal face mages from 38 subjects under varous lghtng condtons. The mages are cropped and normalzed to pxels [4]. Half of the mages were randomly selected for tranng (.e., about 32 mages per subject), and the remanng half are for testng. In our experment, the smple downsampled mages are used for features, wth resoluton 4 4. In order to evaluate the performance of varous methods on ths data each was run on fve sets of mages wth randomly placed occlusons from 2% to 5%. Experments are also carred out on orgnal data wthout occluson for baselne. Recognton rates of dfferent methods are reported n Table. We can clearly see that the proposed three methods obtan the best results n all stuatons. It s noteworthy that wth occlusons below % LCRC gets % recognton rate. When occluson ncreases to 5% accuracy rate of LCRC s stll above 88%, whle the best result of all other methods s 73.% obtaned by SRCvotng. Both the two non-modular methods CRC and TSR do not perform well wth large occlusons. LCRC-Spr and C-LCRC perform very close to LCRC except that when occluson ncreases to 5%, the former two classfers obtans around 87% accuraces whch s lower than that of the thrd one by about %. The close performances of LCRC and LCRC-Spr show that a good representaton s more mportant than the decson rule. For far comparson, we also carry out CRC, TSR and SRC wth spatal pyramd features (CRC-SP, TSR-SP and SRC-SP n Table ). We can see that wth the spatal pyramd features the accuraces of these three methods are sgnfcantly mproved. TSR-SP and SRC-SP also outperform the other two modular methods LRC-DEF and SRC-votng by large gaps especally when wth large occlusons. However, they are stll nferor to LCRC and ts two extensons, whch s manly because the localty nformaton ndeed boosts the FR performance as mentoned above. We also evaluate the performance of LCRC wth -norm regularzaton as n (9). We set l ¼ : for LCRC-. As can be seen, LCRC- obtans a very close performance on ths dataset and the -norm regularzaton does not necessarly mprove the accuracy for LCRC as stated before. Ths s consstent wth the arguments n [3,4] Face recognton wth dsguse The AR database [4] conssts of over 4 facal mages from 26 subjects (7 men and 56 women). For each subject 26 facal mages were taken n two separate sessons. The mages exhbt a 5 The code s obtaned va mark/. 6 In the classfcaton phase, (6) and (2) yeld a very smlar result for LCRC and the reported results are based on (2). Fg. 6. Two face mages occluded by monkey faces n Extended Yale B database wth 2% and 4% occlusons, respectvely.

9 2 F. Shen et al. / Neurocomputng (23) 4 5 Table Classfcaton accuracy (%) on the Extended Yale B database usng 4 4 downsampled pxels. Monkey faces wth varous szes are randomly placed on the test faces. Every result set: accuracy rate (standard devaton) s calculated based on 5 runs. Approach Occluson rate % % 2% 3% 4% 5% CRC TSR SRC-votng LRC-DEF CRC-SP TSR-SP SRC-SP LCRC LCRC LCRC-Spr C-LCRC Fg. 7. Images from two subjects n the AR database wearng sunglasses (left) and two wearng scarves (rght). Table 2 Classfcaton accuracy (%) on the AR database wth sunglasses and scarves occluson usng 4 4 downsampled pxels. For far comparson, TSR s performed wth our spatal pyramd features. Approach SRC-votng LRC-DEF TSR-SP LCRC LCRC-Spr C-LCRC Sunglasses Scarf Table 3 Comparson of dfferent methods conducted on both whole mages and our spatal pyramd local patches on the AR database usng 4 4 downsampled pxels. Approach Usng whole mages Usng pyramd patches SRC CRC LCRC SRC CRC LCRC Sunglasses Scarf number of varatons ncludng varous facal expressons (neutral, smle, anger, and scream), llumnatons (left lght on, rght lght on and all sde lghts on) and occluson by sunglasses and scarves. Of the 26 subjects avalable have been randomly selected for testng (5 males and 5 females) and the mages are cropped to 65 2 pxels. Eght mages of each subject wth varous facal expressons but wthout occlusons were selected for tranng. Testng was carred out on two mages of each subject wearng sunglasses and two wearng scarves. All the mages are reszed to a resoluton of 4 4, and we smply use the raw pxels for nput features. Two test example mages of subjects wearng sunglasses and two wearng scarves from the AR dataset are shown n Fg. 7. For LCRC and LCRC-Spr, we reject patches va (23) and t s chosen as.5. Recognton rates of varous methods are summarzed n Table 2. C-LCRC obtans 99.5% and 98% recognton rates for datasets wth sunglasses and scarf dsguses, respectvely, whch beats all the other state-of-the-art methods compared n ths experment. Both LCRC and LCRC-Spr acheve almost the same results as C-LCRC. Note that no extra spatal pror knowledge s known to the proposed approach. SRC-votng and LRC-DEF do not perform as well as our methods, whch show the votng and DEF decson fuson method s not robust enough for heavly corrupted data as descrbed n Secton 3.2. Wth our pyramd features, performance of the robust method TSR s even mproved, wth very hgh accuraces 99% and 97% for the sunglasses and scarves cases, respectvely. To farly compare these methods, SRC [2], CRC [4] and LCRC 7 are also evaluated usng both whole mages and our spatal pyramd features. Indvdual results of SRC and CRC on local patches are aggregated by votng as n [2,4]. The comparatve results are shown n Table 3. From Table 3, we can see that LCRC acheves the best recognton rates for both cases. Specfcally, LCRC obtans a 66% accuracy whch outperforms CRC and SRC by 8%. Wth the spatal pyramd features, accuraces of all methods are largely mproved. However, accuraces of LCRC are stll hgher (by 2.5% and 3.5%) than the best results of the other two methods: 97% and 94% accuraces whch are both acheved by SRC. The superor results of LCRC ndeed show the dscrmnant ablty of localty nformaton for face recognton, especally on the occluded data. Take the sunglasses case for example, Fg. 8 shows the mpact of valdaton usng LCI on local patches for LCRC (left) and LCRC- Spr (rght) where each mage s reszed to 4 pxels. When the valdaton threshold t ¼, no patch s rejected. As can be seen that for both these two algorthms accuraces are mproved wth t between about.4 and.6. Accuracy for LCRC ncreases to % 7 When wthout pyramd features, l s set as. for LCRC.

10 F. Shen et al. / Neurocomputng (23) Recognton rate Recognton rate Valdaton threshold τ Valdaton threshold τ Fg. 8. Recognton rates on the AR dataset wth sunglasses occluson of LCRC (left) and LCRC-Spr (rght) aganst dfferent valdaton threshold t SRC Votng LRC DEF LRC CRC Extended SRC LCRC LCRC Spr C LCRC 6 when t ¼ :52. When t s larger than.7, recognton rates drop dramatcally, whch s because occluded patches as well as too many useful ones are rejected. As mentoned n Secton 3.2, dscardng most blocks makes the DEF and votng methods unstable Recognton from nsuffcent tranng samples 5 Fg. 9. Classfcaton accuracy (%) on the AR database wth nsuffcent tranng data. The x-axs shows the number of tranng samples per subject. In ths secton we test our methods wth nsuffcent tranng data usng the AR database. We only use the data wthout occlusons (4 samples of each subject) n ths test. For each subjects, the frst seven mages from sesson are used for tranng and the frst seven mages from sesson 2 are for testng. All the mages are reszed to a resoluton of 2 2, and the raw pxels are used for nput features. Smlar as the settng n [38],we reduce the number of tranng samples for each subject from 7 to 2 one by one. In ths test all mages are wth only llumnaton and expresson changes, therefore we take all patches nto classfcaton wthout patch valdaton. Extended SRC s also carred out for comparson, whch constructs the ntra-class varant dctonary by subtractng the centrod of each class form all mages from the same class [38]. Fg. 9 shows the comparatve recognton results of varous methods. The proposed approach LCRC and LCRC-Spr acheve very smlar performances, both of whch outperform all other methods n all stuatons. In partcular, when all the seven tranng samples per subject are avalable, LCRC acheves 98% accuracy whch s hgher than that of SRC-votng, LRC-DEF by 7.9% and 3.3%, respectvely. Wth only two tranng samples per class, LCRC stll acheves more than 8% recognton rate whle Table 4 Classfcaton accuracy (%) on the AR database usng only one tranng sample per subject wth downsampled pxels. Extended SRC [38] s performed wth ntra-class varant dctonary constructed by both subtractng the centrod (ExSRC) and natural mage of each class (ExSRC2), respectvely. Approach CRC ExSRC ExSRC2 SRCvotng LRC- DEF accuraces of all other methods are below 64%. Not surprsngly, the class based algorthm C-LCRC does not perform as well as the other two LCRC algorthms on ths dataset, and the accuracy gap becomes larger as the tranng data sze decreases. However, we can see that C-LCRC stll outperforms other methods compared n ths experment. On ths dataset, extended SRC 8 does not perform as well as the modular SRC, however much better than LRC-DEF. We also compare LRC-DEF wth LRC and CRC whch use data wthout partton, and the DEF algorthm performs even worse than the orgnal LRC (n some cases) and CRC on ths dataset. It s not surprsng because there are no occluson on ths dataset and all blocks should partcpate n the fnal classfcaton, whle the DEF selects only one block (correspondng to the smallest resdual) Recognton from sngle sample per person LCRC LCRC- Spr C- LCRC We next test the robustness of our method on the SSPP problem usng the AR dataset. For each subject, the frst mage wth natural expresson and llumnaton from sesson s used for tranng, and the rest 2 mages wth expresson and llumnaton changes and sunglasses and scarves dsguses from sesson are for testng. We resze all the mages to three dfferent resolutons 2 2, 32 32, 4 4. For LCRC and LCRC-Spr, we set the same valdaton parameter t.5 as n Secton 5.2. For C-LCRC, to elmnate the nverse effect of occluson n the dataset, a large b (4) s chosen. For extended SRC, the frst 2 subjects n sesson 2 (26 mages) are used to construct the ntra-class varant dctonary by two ways [38]: () subtractng the centrod of each class form all mages from the same class, and (2) subtractng the natural mage form other mages from the same class. Table 4 shows the recognton rates of varous methods wth dfferent feature dmensons. Accordng to the table, LCRC-Spr and LCRC perform the best n all dmensonal feature spaces. 8 A smlar settng as n [38] s used for extended SRC except 2 2 nstead of 27 2 downsampled mages are used. The results reported by the authors [38] are: about 93% accuracy wth seven mages per class avalable and 78% accuracy wth two mages per class.

11 4 F. Shen et al. / Neurocomputng (23) 4 5 Specfcally, LCRC-Spr acheves 89.6% recognton rate wth 4 dmensonal raw pxel features whch outperforms SRC-votng, LRC-DEF by than 2.8% and 24.4%, respectvely. Ths shows that LCRC copes well wth the sngle tranng sample FR problems and also confrms that LCRC needs much less tranng samples than the other methods. Due to the use of ntra-class varant dctonary, extended SRC performs much better than CRC and even the modular approaches SRC-votng and LRC-DEF on ths sngle tranng sample problem, however stll worse than the proposed LCRC and LCRC-Spr. Both these two LCRC algorthms obtan above 9% accuracy n hgher dmensonal feature spaces, whle the best accuracy 88.4% of all other methods s obtaned by extended SRC wth feature dmenson 6. Ths result s mpressve, snce only one tranng sample s avalable for each subject and the test data ncorporates both expresson, llumnaton changes and severe facal dsguses. Note that compared to other methods, extended SRC requres extra tranng samples to construct the bases dctonary on the one tranng sample problem, whch possbly cannot be satsfed n real-world applcatons. Note that for the classed based algorthms LRC-DEF and C-LCRC, wthout collaboraton of data from other classes the sngle tranng sample problem becomes even more challengng, snce the probe mage s actually represented by only one mage each tme. However, C-LCRC stll acheves much hgher accuraces than other collaboratve representaton based methods (CRC and the SRC modular approach), whch s due to the effectveness of the spatal pyramd partton and fuson method. 6. Conclusons and future work In ths work we propose a new face recognton method ncorporatng localty on both representaton samples and spatal features. The localty constrant enforces the representaton sparse, whch effectvely concentrates the large representaton coeffcents on a small number of tranng samples, whle other ones are nearly zeros. The spatal pyramd local patches nstead of holstc features are used to sgnfcantly boost the classfcaton performances. Due to both, the proposed method s very robust for two crtcal problems n face recognton: occluson and lack of tranng data. Based on the localty constraned representaton, we proposed three algorthms. The frst two: LCRC and LCRC-Spr take tranng samples from all classes nto representng the probe mage, whle the thrd one C-LCRC s homo-class based. All these three algorthms outperform the state-of-the-art on the publc Yale B and AR databases wth heavy occlusons. Our methods also cope well wth the SSPP problem, and obtans 92.3% accuracy on the AR dataset n the presence of varyng llumnatons, expressons and facal dsguses. In our method, each test patch s represented by ts correspondng tranng patches at the same locaton and patches n a face are consdered ndependently. Ths may not work well on datasets wthout well algned, for example, wth large pose varatons. How to extend the proposed methods by modellng the dependency between patches appears to be nterestng n the future work. References [] R. Basr, D. Jacobs, Lambertan reflectance and lnear subspaces, IEEE Trans. Pattern Anal. Mach. Intell. 25 (2) (23) [2] J. Wrght, A.Y. Yang, A. Ganesh, S.S. Sastry, Y. Ma, Robust face recognton va sparse representaton, IEEE Trans. Pattern Anal. Mach. Intell. 3 (29) [3] Q. Sh, A. Erksson, A. van den Hengel, C. Shen, Is face recognton really a compressve sensng problem?, n: Proceedngs of the IEEE Conference on Computer Vson and Pattern Recognton (CVPR), 2, pp [4] L. Zhang, M. Yang, X. Feng, Sparse representaton or collaboratve representaton: Whch helps face recognton?, n: Proceedngs of the IEEE Internatonal Conference on Computer Vson (ICCV), 2, pp [5] T. Ahonen, A. Hadd, M. Petkanen, Face descrpton wth local bnary patterns: applcaton to face recognton, IEEE Trans. Pattern Anal. Mach. Intell. 28 (2) (26) [6] M. Lades, J. Vorbruggen, J. Buhmann, J. Lange, C. von der Malsburg, R. Wurtz, W. Konen, Dstorton nvarant object recognton n the dynamc lnk archtecture, IEEE Trans. Comput. 42 (3) (993) 3 3. [7] I. Naseem, R. Togner, M. Bennamoun, Lnear regresson for face recognton, IEEE Trans. Pattern Anal. Mach. Intell. 32 () (2) [8] S. Lazebnk, C. Schmd, J. Ponce, Beyond bags of features: spatal pyramd matchng for recognzng natural scene categores, n: Proceedngs of the IEEE Conference on Computer Vson and Pattern Recognton (CVPR), vol. 2, 26, pp [9] K. Yu, T. Zhang, Y. Gong, Nonlnear learnng usng local coordnate codng, n: Advances n Neural Informaton Processng Systems 22 (NIPS), 29. [] P.N. Belhumeur, J.a.P. Hespanha, D.J. Kregman, Egenfaces vs. fsherfaces: recognton usng class specfc lnear projecton, IEEE Trans. Pattern Anal. Mach. Intell. 9 (997) [] S. L, Face recognton based on nearest lnear combnatons, n: Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR), 998, pp [2] S. L, J. Lu, Face recognton usng the nearest feature lne method, IEEE Trans. Neural Networks (2) (999) [3] J. Ho, M.-H. Yang, J. Lm, K.-C. Lee, D. Kregman, Clusterng appearances of objects under varyng llumnaton condtons, n: Proceedngs of the IEEE Computer Socety Conference on Computer Vson and Pattern Recognton (CVPR), vol., 23, pp. I- I-8. [4] K.-C. Lee, J. Ho, D. Kregman, Acqurng lnear subspaces for face recognton under varable lghtng, IEEE Trans. Pattern Anal. Mach. Intell. 27 (5) (25) [5] A. Yang, S. Sastry, A. Ganesh, Y. Ma, Fast l-mnmzaton algorthms and an applcaton n robust face recognton: a revew, n: Proceedngs of the IEEE Conference on Image Processng (ICIP), 2, pp [6] M. Turk, A. Pentland, Egenfaces for recognton, J. Cogntve Neuroscence 3 () (99) [7] J. Yang, L. Zhang, Y. Xu, J.yu Yang, Beyond sparsty: the role of l-optmzer n pattern classfcaton, Pattern Recognton 45 (3) (22) 4 8. [8] I. Naseem, R. Togner, M. Bennamoun, Robust regresson for face recognton, Pattern Recognton 45 () (22) 4 8. [9] R. He, W.-S. Zheng, B.-G. Hu, Maxmum correntropy crteron for robust face recognton, IEEE Trans. Pattern Anal. Mach. Intell. 33 (8) (2) [2] R. He, W.-S. Zheng, B.-G. Hu, X.-W. Kong, A regularzed correntropy framework for robust pattern recognton, Neural Comput. 23 (8) (2) [2] R. He, B.-G. Hu, W.-S. Zheng, Y. Guo, Two-stage sparse representaton for robust recognton on large-scale database, n: AAAI, 2, pp.. [22] M. Yang, L. Zhang, J. Yang, D. Zhang, Robust sparse codng for face recognton, n: Proceedngs of the IEEE Conference on Computer Vson and Pattern Recognton (CVPR), 2, pp [23] A. Bosch, A. Zsserman, X. Munoz, Representng shape wth a spatal pyramd kernel, n: Proceedngs of the 6th ACM Internatonal Conference on Image and Vdeo Retreval, 27. [24] N. Dalal, B. Trggs, Hstograms of orented gradents for human detecton, n: Proceedngs of the IEEE Conference on Computer Vson and Pattern Recognton (CVPR), 25, pp [25] L. Shao, L. J, A descrptor combnng mh and pcog for human moton classfcaton, n: Proceedngs of the ACM Internatonal Conference on Image and Vdeo Retreval (CIVR), 2, pp [26] L. Shao, L. J, Y. Lu, J. Zhang, Human acton segmentaton and recognton va moton and shape analyss, Pattern Recognton Lett. 33 (4) (22) [27] J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, Y. Gong, Localty-constraned lnear codng for mage classfcaton, n: Proceedngs of the IEEE Conference on Computer Vson and Pattern Recognton (CVPR), 2, pp [28] J. Fan, R. L, Varable selecton va nonconcave penalzed lkelhood and ts oracle propertes, J. Am. Stat. Assoc. 96 (456) (2) [29] H. Zou, The adaptve lasso and ts oracle propertes, J. Am. Stat. Assoc. (476) (26) [3] T. Haste, R. Tbshran, J. Fredman, The Elements of Statstcal Learnng: Data mnng, Inference and Predcton, second ed., Sprnger, 29. [3] M. Belkn, P. Nyog, Laplacan egenmaps and spectral technques for embeddng and clusterng, n: Proceedngs of Advances n Neural Informaton Processng Systems 4 (NIPS), MIT Press, 2, pp [32] X. He, S. Yan, Y. Hu, P. Nyog, H.-J. Zhang, Face recognton usng laplacanfaces, IEEE Trans. Pattern Anal. Mach. Intell. 27 (3) (25) [33] S.T. Rowes, L.K. Saul, Nonlnear dmensonalty reducton by locally lnear embeddng, Scence 29 (2) [34] R. Tbshran, Regresson shrnkage and selecton va the lasso, J. R. Stat. Soc. Ser. B 58 (996) [35] T. Kanade, A. Yamada, Mult-subregon based probablstc approach toward pose-nvarant face recognton, n: Proceedngs of the IEEE Internatonal Symposum on Computatonal Intellgence n Robotcs and Automaton (CIRA), 23, pp

12 F. Shen et al. / Neurocomputng (23) [36] A.B. Ashraf, S. Lucey, T. Chen, Learnng patch correspondences for mproved vewpont nvarant face recognton, n: Proceedngs of the IEEE Internatonal Conference on Computer Vson and Pattern Recognton (CVPR), 28. [37] D. Donoho, Y. Tsag, Fast soluton of l-norm mnmzaton problems when the soluton may be sparse, IEEE Trans. Inf. Theory 54 () (28) [38] W. Deng, J. Hu, J. Guo, Extended SRC: undersampled face recognton va ntra-class varant dctonary, IEEE Trans. Pattern Anal. Mach. Intell. (99) (22). [39] B.T.M. Osborne, B. Presnell, A new approach to varable selecton n least squares problems, IMA J. Numer. Anal. 2 (3) (2) [4] A.S. Georghades, P.N. Belhumeur, D.J. Kregman, From few to many: llumnaton cone models for face recognton under varable lghtng and pose, IEEE Trans. Pattern Anal. Mach. Intell. 23 (6) (2) [4] A. Martnez, R. Benavente, The AR face database, CVC, Techncal Report, 998. Fumn Shen receved hs bachelor degree n Appled Mathematcs from Shandong Unversty, Chna. Currently he s a Ph.D. student n School of Computer Scence, Nanjng Unversty of Scence and Technology, Chna. Hs major research nterests nclude computer vson and machne learnng, ncludng face recognton, mage analyss, hashng methods, and robust statstcs wth ts applcatons n computer vson. Zhenmn Tang receved hs Ph.D. degree from Nanjng Unversty of Scence and Technology, Nanjng, Chna. He now s a professor and also the head of School of Computer Scence, Nanjng Unversty of Scence and Technology. Hs major research areas nclude ntellgent system, pattern recognton, mage processng, Embedded system. He has publshed over 8 papers. He s also the leader of several key programs of Natonal Nature Scence Foundaton of Chna. Jngsong Xu s a Ph.D. student at Pattern Recognton and Intellgent System, Nanjng Unversty of Scence and Technology. He receved the B.Sc. degree from the same unversty n 27. Currently, he s vstng Unversty of Technology, Sydney. Hs research nterests nclude computer vson and machne learnng.