Nearest. Points between Image Sets. Meng Yang. Electrical Engineering/IBBT, K.U. Leuven, Belgium

Transcription

1 Face Recognton based on Regularzed Ponts between Image Sets Nearest Meng Yang, Pengfe Zhu, Luc Van Gool,3, Le Zhang Department of Informatonn echnology and Electrcal Engneerng, EH Zurch, Swtzerland Department of Computng, he Hong Kong Polytechnc Unversty, Hong Kong, Chna 3 Department of Electrcal Engneerng/IBB, K.U. Leuven, Belgum {yangme@ee.ethz.ch} Abstract In ths paper, a novel regularzed nearest ponts (RNP) method s proposed for mage sets based face recognton. By modelng an mage set as a regularzed affne hull (RAH), two regularzed nearest ponts (RNP), one on each mage set s RAH, are automatcally determned by an effcent teratve solver. he between-set dstance of RNP s then defned by consderng both the dstance between the RNPs and the structure of mage sets. Compared wth the recently developed sparse approxmated nearest ponts (SANP) method, RNP has a more concse formulaton, less varables and lower tme complexty. Extensve experments on benchmar databases (e.g., Honda/UCSD, CMU Mobo and Youube databases) clearly show that our proposed RNP consstently outperforms state-of-the-art methods n both accuracy and effcency. approaches, whch manly fall nto two categores [][9]: parametrc model based methods and non-parametrc model- approaches [][4-5] free methods. Parametrc model based frstly represent each mage set by some parametrc dstrbuton wth the parameters estmated from the set data tself, and then calculate the between-set dstance by measurng the smlarty between two dstrbutons (e.g., n terms of Kullbac-Lebler dvergence). However, the parametrc methods need to solve the dffcult parameter estmaton problem and heavly requre that the gallery and probe sets should have strong statstcal correlatons, whch may not be true n practce. Keywords: regularzed nearest ponts; regularzed affne hull; mage set; face recognton I. INRODUCION he recognton of objects of nterest (e.g., human faces) s one of the most mportant problems n the communtes of computer vson and pattern recognton. he tradtonal face recognton s usually formulated as a problem of dentfyng a human face from a sngle probe mage, although the gallery set per subject could be a sngle mage or multple mages. However, t s a bg challenge to correctly dentfy a person from only a sngle face mage n less-controlled/uncontrolled envronments snce the facal appearance changes dramatcally due to varous varatons n pose, llumnaton, expresson, dsguse, etc. Wth the developments of vdeo cameras and large-capacty-storage meda, t becomes very convenent to collect gallery and probe mage sets from vdeo sequences or photo album for a nown subject. he probe/gallery set for each subject ncorporates more wthn-class appearance varaton (e.g., the mage sets shown n the bottom of Fg. ), mang the mage sets based face recognton be able to acheve more satsfactory performance n practcal applcatons. Over the past decade, there has been growng nterest n face recognton by sets of mages. One specal case of mage- [][0- set-based recognton s vdeo-based face recognton 3], where the mages are collected from consecutve vdeo sequences. In ths paper, we focus on a more general mage-sets no temporal based recognton problem, where there nformaton exsted/provded n the mage set (e.g., unordered photo album mages). o solve ths mage-set based recognton problem researchers have proposed numerous Fg.. Regularzed Nearest Ponts (RNPs) of two mage sets. Gven two mage sets (from Honda/UCSD dataset []), the RNPs are the two ponts, one on each set s regularzed affne hull, between whch the dstance s the smallest. In order to avod the drawbacs of parametrc methods, non-parametrc model-free methods were proposed by representng an mage set as a lnear/affne subspace [3][9] [6-8], mxture of subspaces [9-3], or nonlnear manfolds [4][7][3-33]. Usually the nonlnear-manfold methods express an mage set as a combnaton of local lnear subspaces [4][33]. Based on the representaton of mage sets, the between-set dstance could be defned as the dstance between two exemplars (e.g., the sample means) from these two mage sets. Cevalp et al. [3] characterzed each mage set by an affne/convex hull spanned by ts samples, and selected two ponts (one pont n one hull) wth the closest approach as the exemplars. Another type of between-set dstance for non- the structure of the parametrc methods s by comparng non-

2 parametrc model. For nstance, canoncal correlaton analyss [9], whch analyzes the prncpal angles and canoncal correlatons between lnear subspaces, s wdely used n [4][9][6][7][8][30][3]. Besdes, Wang et al. [6] represented each mage set wth ts natural second-order statstcal covarance matrx, and formulated the mage-set based classfcaton as classfyng ponts lyng on a Remannan manfold. Very recently, Hu et al. [] proposed an nterestng mageset-based face recognton method, namely sparse approxmated nearest ponts (SANP). By modelng each mage set as an affne hull, Hu et al. selected two ponts (one pont n each mage set) wth the closest dstance as the sparse approxmated nearest ponts (SANP), where SANPs are requred to be sparsely represented by the orgnal samples. he fnal between-set dstance s the dstance between the SANPs multpled by the dmenson of the affne hull. SANP acheves state-of-the-art performance compared to prevous methods. However, SANP does not model the mage set well although t utlzes both affne hull representaton and sparse regularzaton n a brute-force way. he complex model (e.g., three representaton terms and four unnown varables) maes SANP somewhat confusng, and the sparse constrant and many unnown varables also ncrease the dffculty and complexty to solve SANP. hs paper presents an effcent and effectve regularzed nearest ponts (RNP) method for mage-set based face recognton. We wll show that the complex formulaton and the sparse constrant on the representaton coeffcents n SANP are not necessary. By modelng an mage set as a regularzed affne hull (RAH), two regularzed nearest ponts (RNP), one on each RAH, are automatcally computed, as shown n Fg.. hen the between-set dstance s defned as the modulated dstance between RNPs by the structure of mage sets. Compared to SANP, RNP models the mage set better and has a concse formulaton wth less number of parameters and unnown varables. An effcent algorthm s proposed to solve the proposed RNP wth very low tme complexty. Our experments on benchmar mage set databases clearly show that RNP leads to hgher recognton accuracy than the prevous methods, ncludng SANP. And more mportantly, the proposed RNP has a very fast speed, e.g., t s over 0 tmes faster than SANP on the CMU Mobo database [5]. he rest of ths paper s organzed as follows. Secton II brefly revews the SANP method n []. Secton III presents the proposed RNP. Secton IV conducts experments and Secton V concludes the paper. II. SPARSE APPROXIMAED NEARES POINS (SANP) Based on the wor n [3] where each mage set s modeled as an affne/convex hull, recently Hu et al. [] proposed the sparse approxmated nearest ponts (SANP) to combne the affne hull representaton [3] and sparse representaton [5]. SANP has two objectves. One s that the affne-hull regularzed dstance between two pont sets should be small by mnmzng ( ) ( ) F ν ν = μ Uν μ U ν (), j j j j where µ s the sample mean of the th class data matrx X, the columns of U are the orthonormal bases obtaned from the sngular value decomposton (SVD) of the centered data matrx of class. It can be seen that ths part s smlar to the affne hull method n [3]. After mnmzng Eq. (), µ U ν and µ j U j ν j are called the nearest ponts between the th and j th classes, where ν and ν j are the codng coeffcents. he other objectve of SANP s that each of the two nearest ponts should be sparsely represented by the orgnal data matrx,.e., ( U ), = X F Gν λ μ ν λ, ( ), = j j j j j F Q λ λ ν β β μ U ν X β β, where λ and λ are the parameters to tune the effect of sparse constrant. he fnal model of SANP s ( ˆ ν ˆ ˆ ˆ, ν j,, β ) = mn ( Fν, (,, ) ν γ G Q λ j ν ν j β λ β ) ν, ν j,, β () and the fnal classfcaton of a testng mage set s conducted to fnd whch class has the mnmal between-set dstance, whch s defned as D( c, cj) = ( d d ) j Fˆ, ˆ γ j ( Gˆ, ˆ Q ˆ, ˆ ) ν ν ν ν j β (3) where d and d j are the dmenson of the affne hulls (.e., U and U j ) of th class (.e, c ) and j th class (.e., c j ), respectvely. For X, there s another parameter, ϕ, as a threshold of preservng energy (e.g., ϕ=85%) n determnng U and d. Although SANP has acheved very nterestng results on mage sets based face recognton, there are several ssues needng to be further consdered. a. he brute-force way to combne the affne hull representaton and sparse representaton maes the model of SANP rather complex (e.g., three representaton terms n Eqs. () and (3), four parameters and four unnown varables), whch ncrease the dffculty and complexty of solvng SANP. b. he l -norm sparse regularzaton on the representaton coeffcents and β maes the solvng of SANP tmeconsumng, although the fast solver of Accelerated Proxmal Gradent (APG) method was adopted n SANP. III. REGULARIZED NEARES POINS In ths secton, we frst present the model of the proposed regularzed nearest ponts (RNP). hen we descrbe the solvng algorthm and classfcaton of RNP. Fnally the tme complexty of the proposed RNP s dscussed. F

3 A. Model of RNP Denote X = x,, x,,, x, n as the data matrx of th class, and x, s the th mage feature vector (e.g., raw pxel value) of the th class. Based on these sample data, n [] and [3] the mage set was approxmated as an affne/convex and affne hull, respectvely. In ths paper, we propose a novel regularzed affne hull (RAH) to model an mage set: RAH = { x = X =, σ } (4) where R for =,,, n, and s the -norm of the representaton coeffcents. In order to gve an ntutve llustraton of RAH, we plot the soluton space (.e., the constrant) of RAH wth n =3 and p= n Fg.. One could see that the soluton space of RAH s not a hyperplane but a regularzed partal regon wth the pont of { = n for =,,, n } as ts center. Compared to the conventonal affne hull to model an mage set, RAH can avod contanng the meanngless ponts whch are too far from the sample mean. Fg.. An example of the soluton space of RAH. he blue parallelogram represents the affne plane (.e., the soluton space of affne hull), whle the green ellpse represents the soluton space of the -norm (p=) RAH. For a gallery mage set and a probe mage set, our basc dea s to fnd two nearest ponts, each pont n the RAH of an mage set, as the regularzed nearest ponts (RNP). Let X be the th class data matrx n the gallery set and Y =,,, y y y n y be the probe mage set where y s the th mage of Y. hen we fnd the RNPs of X and Y by the followng mnmzaton: mn β, X Yβ s.t. =, β =, σ, β lp where the -norm terms (e.g., and β ) could mae the representaton more stable by suppressng unnecessary samples contrbuton to the representaton, and the affne hull constrant lp σ (4) ) could (e.g., =, β = avod the trval soluton (.e., =β=0). Usng the Lagrangann formulaton, the problem of RNP could be rewrtten as mn β, ( Y λ λ X β β where λ and λ are the two Lagrangan multplers. Accordng to the number of samples n the mage set, the proposed RNP can be dvded nto two specal cases: regularzed nearest subspace classfer [7] and nearest neghbor. In the frst case, one of X and Y has only one sample. ang the probe mage set as an example (e.g., y for the probe mage set), we could get β= and the proposed RNP degenerates to mn X y λ ) s.t. l = p (6) ( whch s actually the regularzed nearest subspace classfer [7][8][34] wth an addtonal affne constrant. In the second case, each mage set wll have only one sample (e.g., x for th gallery set or y for the probe set). We have =β= and the proposed RNP degenerates to mn ( x y ) (7) whch s the model of nearest neghbor classfer. B. Algorthm of RNP he proposed RNP model has varous nstantatons by applyng dfferent norms to the representaton coeffcents. More specfcally, when p=0 or, RNP s regularzed by l 0 /l - norm sparse constrant; when p=, l -norm regularzaton s appled to the representaton coeffcents. Some other constrants (e.g., non-negatve constrant) could also be addtvely mposed on the representaton coeffcents. In ths paper, we prefer to focus on a specal nstantaton of RNP wth p= snce hgh recognton accuracy and fast speed could be both acheved. Snce =, β = are two lnear equatons, by relaxng them as, β t s easy to ntegrate them wth the frst term of Eq. (5). hus Eq. (5) wth p= could be rewrtten as β, ( λ where z=[0;;], Y = Y ; 0 ;, ; ; X = X 0, and the two column vectors, 0 and, have approprate szes based on the context. In fact, the l -norm regularzed model of Eq. (8) s the Rdge Regresson, whch s also a shrnage method as the l -norm regularzed sparse codng (.e., Lasso) [0]. Eq. (8) has a closed-form soluton, whch, however, s not the fastest solver snce a calculaton of matrx nverse s needed for each par of X and Y. lp ) s.t. =, β = ) (5) mn zx Yβ λ β (8)

4 For the face recognton problem based on mage sets, Eq. (8) could be solved very effcently by alternatvely calculatng and β. When s fxed, β could be solved by where ( ) by ( ) β = P z X (9) P = Y Y λ I Y. When β s fxed, we compute where ( ) P = X X λ I X. ( ) = P z Y β (0) he algorthm of RNP wth p= s summarzed n Algorthm. Here we ntalze 0 =/n, where n s the number of samples n the th class. It s easy to see that the cost functon of Eq. (8) s lower bounded ( 0) and jontly convex to the varables and β. Because n each step of Algorthm the cost functon value wll decrease, the proposed Algorthm wll converge to the global optmal soluton. Algorthm : Algorthm of Regularzed Nearest Ponts (RNP) wth p= Input: Projecton matrces P and P, data matrces z, and an ntalzaton of 0. Whle not converged do Compute the representaton coeffcents: β = P z X ; ( ) ( ) t t t = P z Y βt ; End whle Output: representaton coeffcents ˆ and ˆ β X and Y, a column vector C. Classfer of RNP Wth the solved coeffcents ˆ and ˆβ, the between-set dstance of RNP s computed as follows ( ) = X Y X ˆ Y ˆ β () e where X (.e., nuclear norm of X) s the sum of the sngular * values: = ( ) * σ X X, and X ˆ ˆ Yβ represents the Eucldean dstance between the two regularzed nearest ponts. he term X * Y * n Eq. () ams to remove the dsturbance unrelated to the class nformaton. For example, a wrong class whch has much more samples than the correct class wll have a lower value of X ˆ ˆ Y β. erm X * s the convex relaxaton of the ran of matrx X, whch could reflect the representaton ablty of mage set X (n our paper each column vector of X s normalzed to have unt l -norm energy). he proposed e consders both the dstance of RNPs and the structure of mage sets, and t could well reflect the class nformaton of X and Y. he term of d d j n Eq. (3) of SANP also consders the structure of each mage set, however, d d j s senstve to the threshold ϕ (.e., energy preservng percent). he dentty of the probe mage set Y s decded by ( ) = { e } dentty Y arg mn () D. Complexty analyss In ths secton, we compare the tme complexty of the proposed RNP and the state-of-the-art sparse approxmated nearest ponts (SANP) []. Some emprcal analyss of sparse codng s frstly presented snce SANP nvolves the step of sparse representaton. Some fast l -norm mnmzaton solvers have been recently revewed n []. However, t s nown that sparse codng wth an m n -szed dctonary has a computatonal complexty of O(m n ε ), where ε.[][3], m s the dmensonalty of sgnal feature, and n s the number of dctonary atoms. he sparse codng step of SANP has emprcal complexty O(m (n n y ) ε ) for computng the between-class dstance of X and Y, where n and n y are the numbers of samples belongng to th gallery class and the probe mage set, respectvely. Besdes, SANP needs addtonal calculatons of SVD (e.g., U and U y ) and varables (e.g., ν, and ν y ), where U y and ν y are assocated to Y. Consderng U for the gallery mage set could be offlne computed, the total tme complexty of SANP for classfyng the probe mage set Y s about O svd Σ O(m (n n y ) ε ). Here the summarzaton Σ (.) means all the between-set dstance of Y and X, =,,, should be calculated, and O svd denotes the tme complexty of Y s SVD. Let s analyze the complexty of the proposed RNP. For the query mage set Y, all the projecton matrces of P and X * for all gallery sets could be computed offlne. he computng of P nvolves a matrx nverse, whose tme complexty s roughly equal to the calculaton of SVD of Y n SANP. hus the step to calculate P n RNP has a complexty of O svd. he next step of RNP,.e., the onlne teraton codng for X has a tme complexty of O(lm(n n y )), where l s the teraton number. Usually a small value of l (e.g., l=5) could already get a good soluton. In classfcaton, Y * could be fast computed due to t only nvolves the sngular values. herefore, the total complexty of RNP for a probe mage set has a complexty of O svd Σ O(lm(n n y )) wth l=5 n ths paper. he overall tme complexty of RNP and SANP are lsted n able. Because ε. and the teraton number of RNP s much less than the feature dmenson (e.g., l=5<<m=900 n Youube), RNP has much lower tme complexty than SANP. ABLE. me complextes of RNP and SANP for classfy one probe mage set. Method Step Step SANP O svd for SVD Σ O(m (n n y) ε ) for sparse codng RNP O svd for P Σ O(lm(n n y)) for teratve codng

5 IV. EXPERIMENAL RESULS We perform experments on benchmar mage-set face databases to demonstrate the effectveness of RNP. We frst dscuss the expermental setup n Secton A. In Secton B, we evaluate RNP on three benchmar datasets, followed by the runnng tme comparson n Secton C. In ths paper, the parameters of RNP s fxed as λ =e-3, and λ =e- for all the experments. A. Expermenal setup hree benchmar mage set databases, ncludng Honda/UCSD [], CMU Mobo [5], and Youube Celebrtes [6] datasets, are used to evaluate the proposed RNP. All the face mages n the three datasets were detected by usng the Vola and Jone s face detector [4]. For Honda/UCSD and Youube datasets, after hstogram equalzaton the face mages are reszed to 0 0 and 30 30, respectvely; and the raw pxel values of each mage were drectly used as the feature n the data matrx. For CMU Mobo dataset, the hstogram of LBP feature [8] was extracted as the facal feature. For each dataset, three nds of experments are conducted wth the frame number 50, 00 and 00, respectvely. It should be noted that all mages are used for classfcaton f the number of frames n a set s fewer than the gven frame number. he proposed RNP s compared wth several state-of-theart and representatve mage set classfcaton methods, among whch the Dscrmnant Canoncal Correlatons (DCC) [9] and Mutual Subspace Method (MSM) [8] are lnear subapce based methods; Manfold-Manfold Dstance (MMD) [4] and Manfold Dscrmnant Analyss (MDA) [33] are nonlnear manfold based methods; and Affne Hull based Image Set Dstance (AHISD) [3], Convex Hull based Image Set Dstance (CHISD) [3], and Sparse Approxmated Nearest Pont (SANP) [] are affne subspace based methods. All the competng methods are mplemented by usng the source codes provded by the authors, wth the parameters tuned for ther best results accordng to the recommendatons n the orgnaapers. For AHISD, CHISA and SANP, we used ther lnear versons snce we ddn t consder the ernel verson of RNP n ths paper. In Honda/UCSD and CMU Mobo datasets, there s a sngle tranng mage set for each class. hus followng the setng of [9], each sngle tranng mage set for DCC was randomly dvded nto two subsets to construct the wthnclass sets. B. Expermental results and analyss Honda/UCSD Dataset he Honda/UCSD dataset contans 59 vdeo sequences of 0 dfferent subjects []. For each subject, dfferent poses and expressons appear across dfferent sequences, as shown n the face mages n Fgure. As the expermental settng of [][], we use 0 sequences for tranng, wth the remanng sequences for testng. he recognton results usng dfferent number of tranng frames are reported n able. We can clearly see that the proposed RNP acheves the best performance n all cases, especally when the frame number s 00 all the testng sets are correctly recognzed. he lnear RNP outperforms SANP and even has the same performance to the ernel verson of SANP []. When there are enough mage samples n each mage set, good performance could be acheved by all the methods, except MSM, whch usually gets the worst result. When the number of mage samples s not hgh (e.g., 50), the nonlnear manfold based methods (e.g., MMD) could not get a hgh recognton rate. However, the performance of the affne subspace based methods (e.g., AHISD, SANP) s stll good. ABLE. Recognton rates on the Honda/UCSD Dataset Methods/Set Length 50 Frames 00 Frames 00 Frames DCC 76.9% 84.6% 94.87% MMD 69.3% 87.8% 94.87% MDA 8.05% 94.87% 97.44% AHISD 87.8% 84.6% 89.74% CHISD 8.05% 84.6% 9.3% MSM 74.36% 79.49% 76.9% SANP 84.6% 9.3% 94.87% RNP 87.8% 94.87% 00% CMU Mobo Dataset he CMU Mobo (Moton Boday) dataset [5] contans 96 sequences of 4 subjects walng on a treadmll. For each subject, there are 4 vdeo sequences (wth sgnfcant pose varaton) collected n four walng patterns, respectvely. As [], the employed sample features are the unform LBP hstograms usng crcular (8, ) neghborhoods extracted from the 8 8 squares of the gray-scale mages. One mage set per subject s randomly selected as the tranng data, wth the remanng mage sets as the testng data. ABLE 3. Recognton rates on the CMU Mobo Dataset Methods/Set Length 50 Frames 00 Frames 00 Frames DCC 8.%±.7% 85.5%±.8% 9.6%±.5% MMD 90.%±.3% 94.6±.9% 96.4%±0.7% MDA 86.%±.9% 93.%±.8% 95.8%±.3% AHISD 9.6%±.8% 94.%±.0% 9.9%±.6% CHISD 9.%±3.% 93.8%±.5% 97.4%±.9% MSM 84.3%±.6% 86.6%±.% 89.9%±.4% SANP 9.8%±3.% 94.7%±.7% 97.3%±.3% RNP 9.9%±.5% 94.7%±.% 97.4%±.5% en experments are conducted, wth the average recognton rates and the standard devatons are summarzed n able 3. In all cases, RNP has the hghest dentfcaton rates. Although SANP and CHISD have close recognton accuracy to RNP, we wll see that the runnng tme of RNP s much less than that of SANP and CHISD n the followng Secton of runnng tme comparson. When there are 50 frames, DCC, MSM and MDA have recognton rates lower than 90%, whch

6 may result from the fact that extracton of dscrmnatve nformaton and manfold analyss depend on enough samples per mage set. Compared to AHISD, the advantage of RNP s sgnfcant, whch valdates that the regularzaton of RAH ndeed brngs benefts to the fnal classfcaton. Youube Celebrtes Dataset he Youube Celebrtes dataset [6] s a large-scale vdeo dataset. hs dataset s more challengng than the prevous two datasets snce the mages are mostly low resoluton and have large pose/expresson varaton, moton blur, etc, as shown n Fg. 3. In ths part, the vdeo sequences of the frst 9 celebrtes are used to do the experments. For each subject, three vdeo sequences are randomly selected as the tranng data, wth the other three randomly selected sequences as the testng data. We conduct 5 experments by repeatng the random selecton of tranng/testng data. he expermental results, ncludng the average recognton rate and the standard devaton, are summarzed n able 4. Smlar conclusons to those on the prevous two datasets could be made. RNP has better performance than all the competng methods. Compared to the second best method, SANP, over % mprovement s acheved when the frame number s 50 and 00. In ths challengng test, MSM has the worst result, wth average dentfcaton rates less than 70%. It s also nterestng to see that AHISD s recognton rate fluctuates wth the ncrease of the frame number, smlar to what have found n the prevous two datasets. runnng tme, whch s one the most mportant concerns n practcal applcatons. We do face recognton on CMU Mobo dataset [5] wth the same expermental settng as that n Secton B. he programmng envronment s Matlab verson 00a. he destop used s of 7.8 GHz CPU and wth 4GB RAM. In order to mae the runnng tme comparson farer, we also lst the offlne tranng tme of some methods. Apart from these dscrmnant methods (e.g., DCC, MDA) whch need a tranng phase, the constructon of local lnear subspace n MMD, the SVD of tranng sets n SANP, and the projecton matrx learnng of the tranng sets n RNP are also regarded as the offlne tranng. he offlne tranng tme and onlne testng tme for classfyng one mage set wth frame number as 00 s lsted n able 5. RNP has very short offlne tranng tme snce only several matrx nverse computatons are needed. he onlne testng tme s more mportant for a classfer. From able 5, we can see that the runnng tme (.e., for classfyng a testng mage set) of RNP s much less than all the other methods. Compared to SANP, the speedup of RNP s over 0 tmes. RNP s about 5 tmes faster than the second fastest method, MDA, wth havng much hgher recognton accuracy. In order to more comprehensvely evaluate the runnng tme, n Fg. 4 we plot all the methods testng tme versus dfferent frame numbers. It can be seen that the proposed RNP s consstently faster than all the competng methods. he runnng tme of all the methods wll ncrease as the frame number except some specal cases (e.g., DCC and MDA when the frame number s 00). Especally, AHISD s runnng tme wll dramatcally rse as the frame number ncreases. 0 Fg.3.Some examples of the Youube dataset. ABLE 4. Recognton rates on the Youube Dataset Methods/Set Length 50 Fames 00 Frames 00 Frames DCC 68.7%±3.% 73.8%±4.7% 76.%±.5% MMD 69.0%±3.5% 7.0%±4.6% 76.3%±4.3% MDA 63.9%±3.9% 74.%±5.9% 74.5%±5.0% AHISD 73.3%±5.4% 7.6%±7.6% 66.9%±4.8% CHISD 7.4%±5.5% 73.6%±5.% 75.%±5.% MSM 66.%±4.6% 66.0%±8.6% 65.3%±6.5% SANP 73.3%±3.9% 74.9%±5.9% 78.3%±4.% RNP 74.9%±5.4% 76.%±5.5% 78.9%±6.4% C. Runng tme comparson From Secton B, we can see that RNP acheves hgher recognton rates than all the competng methods, ncludng the recently developed SANP []. Next let s compare ther Computaton tme DCC MMD MDA AHISD CHISD MSM SANP RNP Frame number Fg. 4. he testng tme for one mage set versus the frame number for all the competng methods on the CMU Mobo dataset. ABLE 5. Computaton tme (seconds) of dfferent methods on the CMU Mobo dataset wth 00 frames for tranng and testng (classfcaton of one mage set). #: offlne tranng tme; #: onlne testng tme. DCC MMD MDA AHISD. CHISD MSM SANP RNP # N/A N/A N/A #

7 V. CONCLUSION In ths paper, we proposed a regularzed nearest ponts (RNP) method for robust and fast face recognton based on mage sets. We developed a novel regularzed affne hull (RAH) to represent an mage set, and defned the between-set dstance as the dstance between RNPs wth consderaton of the structure of mage set. An effcent algorthm was also developed to mplement RNP for mage set based face recognton. We evaluated the proposed RNP on several benchmar mage set databases. he extensve expermental results clearly demonstrated that RNP could acheve hgher dentfcaton accuracy than the state-of-the-art methods (e.g., sparse approxmated nearest ponts) but wth much faster speed, mang mage sets based face recognton more applcable n practcal applcatons. In ths paper, we only dscussed RNP wth l -norm regularzaton. Nevertheless, RNP s a general classfcaton scheme, and dfferent regularzatons (e.g., sparse, non-negatve) and the ernel trcs (e.g., Gaussan ernel) could be employed for dfferent applcatons. ACKNOWEDGEMEN hs wor was supported by the Hong Kong Polytechnc Unversty nternal grant G-YK5. REFERENCES [] K.-C. Lee, J. Ho, M.-H. Yang and D. Kregman, Vdeo-base face recognton usng probablstc appearance manfolds, n Proc. CVPR, 003. [] Y. Q. Hu, A. S. Man and R. Owens, Face recognton usng sparse approxmated nearest ponts between mage sets, IEEE PAMI 34(0), , 0. [3] H. Cevalp and B. rggs, Face recognton based on mage sets, n Proc. CVPR 00. [4] R. Wang, S. Shan, X. Chen and W. Gao, Manfold-manfold dstance wth applcaton to face recognton based on mage set, n Proc. CVPR, 008. [5] J. Wrght, A. Y. Yang, A. Ganesh, S. S. Sastry and Y. Ma, Robust face recognton va sparse representaton, IEEE PAMI 3(): 7, 009. [6] R. Wang, H. Guo, L. S. Davs, Q. Da, Covarance Dscrmnatve Learnng: A Natural and Effcent Approach to Image Set Classfcaton, n Proc. CVPR 0. [7] L. Zhang, M. Yang, X. Feng, Y. Ma and D. Zhang, Collaboratve Representaton based Classfcaton for Face Recognton, arxv: [8] L. Zhang, M. Yang, and X. Feng, Sparse Representaton or Collaboratve Representaton: Whch Helps Face Recognton? n Proc. ICCV, 0. [9] H. Hotellng, Relatons between tow sets of varates, Bometra, 8(3-4): 3-377, 936. [0]. Haste, R. bshran, and J. Frdrman, he Elements of Statstcal Learnng, nd ed., Sprnger, 009. [] A. Yang, A. Ganesh, Z. H. Zhou, S. Sastry, and Y. Ma, Fast L- Mnmzaton Algorthms for Robust Face Recognton, (preprnt) [] S. J. Km, K. Koh, M. Lustg, S. Boyd, and D. Gornevsy, A nterorpont method for large-scale l-regularzed least squares, IEEE Journal on Selected opcs n Sgnal Processng, (4):606 67, 007. [3] Y. Nesterov, A. Nemrovs, Interor-pont polynomal algorthms n convex programmng, SIAM Phladelpha, PA, 994. [4] P. Vola and M. J. Jones, Robust real-tme face detecton, Internatonal Journal of Computer Vson, 57(): 37-54, 004. [5] R. Gross and J. Sh, he CMU Moton of Body (MoBo) Database. echncal Report CMU-RI-R-0-8, Robust nsttute, 00. [6] M. Km, S. Kumar, V. Pavlovc and H. Rowley, Face tracng and recognton wth vsual constrants n real-world vdeos, n Proc. CVPR, 008. [7]. Wang and P. Sh, Kernel grassmannan dstances and dscrmnant analyss for face recognton from mage sets, PRL, 30(3): 6-65, 009. [8]. Ahonen, A. Hadd and M. Petanen, Face descrpton wth local bnary patterns: Applcaton to face recognton, IEEE PAMI, 8(): , 006. [9].-K. Km, O. Arandjelovc and R. Cpolla, Dscrmnatve learnng and recognton of mage set classes usng canoncal correlatons, IEEE PAMI, 9(6): , 007. [0] W. Lu, Z. L, and X. ang, Spato-temporal Embeddng for Statstcal Face Recognton from Vdeo, n Proc. ECCV, 006. [] X. Lu and. Chen, Vdeo-Based Face Recognton Usng Adaptve Hdden Marov Models, n Proc. CVPR, 003. [] J. Stallamp, H. K. Eenel, R. Stefelhagen, Vdeo-based Face Recognton on Real-World Data, n Proc. ICCV 007. [3] S. Zhou and R. Chellappa, Probablstc Human Recognton from Vdeo, n Proc. ECCV, 00. [4] O. Arandjelovc, G. Shahnarovch, J. Fsher, R. Cpolla and. Darrel, Face recognton wth mage sets usng manfold densty dvergence, n Proc. CVPR, 005. [5] G.Shahnnarvovch, J. W. Fsher and. Darrel, Face recognton from long-term observaton, n Proc. ECCV, 00. [6] K.-C. Lee, J. Yamaguch, he ernel orthogonal mutual subspace method and ts applcaton to 3D object recognton, n Proc. ACCV, 007. [7] M. Nshyama, O. Yamaguch and K. Fuu, Face recognton wth the multple constraned constraned mutual subspace method, n Proc. AVBBPA, 005. [8] O. Yamaguch, K. Fuu and K.-. Maeda, Face recognton usng temporal mage sequence, n Proc. FG, 998. [9] M. Nshyama, M. Yuasa,. Shbata,. Waasug,. Kawahara and O. Yamaguch, Recognzng faces of movng people by herarchcal mage-set matchng, n Proc. CVPR, 007. [30].-K. Km, J. Kttler and R. Cpollar, Incremental learnng of locally orthogonal subspaces for set-based object recognton, n Proc. BMVC, 006. [3] W. Fan and D.-Yeung, locally lnear models on face appearance manfolds wth applcaton to dual-subspace based classfcaton, n Proc. CVPR, 006. [3] A. W. Ftzgbbon and A. Zsserman, Jont manfold dstance: a new aproach to appearance based clusterng, n Proc. CVPR, 003. [33] R. Wang and X. Chen, Manfold dscrmnant analyss, n Proc. CVPR, 009. [34] I. Naseem, R. ogner, and M. Bennamoun, Lnear regresson for face recognton, IEEE PAMI 3(): 06-, 00.