Fast High Dimensional Vector Multiplication Face Recognition

Transcription

1 Fast Hgh Dmensonal Vector Multplcaton Face Recognton Oren Barkan Tel Avv Unversty Jonathan Well Tel Avv Unversty Lor Wolf Tel Avv Unversty Haga Aronowtz IBM Research Abstract Ths paper advances descrptor-based face recognton by suggestng a novel usage of descrptors to form an over-complete representaton, and by proposng a new metrc learnng ppelne wthn the same/not-same framework. Frst, the Over-Complete Local Bnary Patterns (OCLBP) face representaton scheme s ntroduced as a mult-scale modfed verson of the Local Bnary Patterns (LBP) scheme. Second, we propose an effcent matrx-vector multplcaton-based recognton system. The system s based on Lnear Dscrmnant Analyss (LDA) coupled wth Wthn Class Covarance Normalzaton (WCCN). Ths s further extended to the unsupervsed case by proposng an unsupervsed varant of WCCN. Lastly, we ntroduce Dffuson Maps (DM) for non-lnear dmensonalty reducton as an alternatve to the Whtened Prncpal Component Analyss (WPCA) method whch s often used n face recognton. We evaluate the proposed framework on the LFW face recognton dataset under the restrcted, unrestrcted and unsupervsed protocols. In all three cases we acheve very compettve results.. Introducton The Labeled Faces n the Wld (LFW) face recognton benchmark [] s currently the most actve research benchmark of ts knd. It s bult around a smple bnary decson task: gven two face mages, s the same person beng photographed n both? The comprehensve results tables show a large varety of methods whch can be roughly dvded nto two categores: par comparson methods and sgnature based methods. In the par comparson methods [, 3, 4], the decson s based on a process of comparng the two mages part by part, oftentmes nvolvng an teratve local matchng process. In the sgnature based methods [5, 6, 7, 8, 9], each face mage s represented by a sngle descrptor vector and s then dscarded. To compare two face mages, ther sgnatures are compared usng predefned metrc functons, whch are sometmes learned based on the tranng data. The par comparson methods allow for flexblty n representaton, based on the actual mage par to be compared. On the other hand, the sgnature based methods are often much more effcent. Furthermore, there s a practcal value n sgnature based methods n whch the sgnature s compact. Such systems can store and retreve face mages usng lmted resources. In ths paper, we propose an effcent sgnature based method, n whch the storage footprnt of each sgnature s on the order of a hundred floatng pont numbers. Ths compares to storage footprnts of one to three orders of magntude larger n prevous work. Our method ncludes multple contrbutons. Frst, as detaled n Secton, we propose to use over-complete representatons of the nput mage. Ths s shown to sgnfcantly contrbute to the overall performance. However, ths added accuracy s hdden untl dmensonalty reducton s performed. In Secton 3, we propose the use of the WCCN [0] metrc learnng technque for face recognton. In Secton 4, we propose a general scheme for generatng labeled data from an unlabeled data. In Secton 5, we descrbe n detal our proposed recognton system, whch s applcable for both supervsed and unsupervsed learnng by utlzng the scheme descrbed n Secton 4. Ths results n an extenson of the WCCN metrc learnng to the unsupervsed case. In Secton 6, the Dffuson Maps technque (DM) [] s ntroduced as a non-lnear dmensonalty reducton method for face recognton. We nvestgate t as an alternatve to WPCA [6] and show that t can mprove performance over the baselne when beng fused wth WPCA. In Secton 7, we evaluate the proposed system on the LFW dataset under the restrcted, unrestrcted and unsupervsed protocols and report state of the art results on these benchmarks. Fnally, n Secton 8, we conclude and dscuss future work.. Overvew of the recognton ppelne A unfed ppelne s used n order to solve the unsupervsed case and the two supervsed scenaros of the LFW benchmark: the restrcted and the unrestrcted protocols. Frst, a representaton s constructed from the face mages. Ths ether uses exstng methods, such as LBP [9], TPLBP [] and SIFT [7], or methods whch are ntroduced to the felds n ths paper, such as the OCLBP

2 and the use of the Scatterng transform [3]. Second, a dmensonalty reducton step takes place. Ths s ether WPCA or the Dffuson Maps for the unsupervsed case, or PCA-LDA or DM-LDA for the two supervsed settngs. Thrd, WCCN s appled. For the supervsed settngs, the orgnal WCCN method [0] s appled. For the unsupervsed case, our unsupervsed WCCN varant s appled. As a last step, cosne smlartes based on multple representatons and mage features are combned together usng unform weghtng.. Over-complete representatons Over-complete representatons have been found to be useful for mprovng the robustness of classfcaton systems by usng rcher descrptors [4, 5]. In ths work, we ntroduce two new adaptatons of descrptors for the doman of face recognton. Both of them share the property of over-complete representaton. In the expermental results secton, we show that the mprovement n the accuracy of usng over-complete representatons remans hdden untl some dmensonalty reducton s nvolved. However, ts contrbuton to the fnal score s sgnfcant... Over-complete local bnary patterns LBP [6] s one of the most successful features for texture classfcaton. Specfcally, a modfed 'unform' verson [9] of the orgnal LBP was found to be useful for the task of face recognton. Several attempts to extend or modfy the LBP have been made n [, 7]. However, most of them resulted n new varants of LBP whch do not necessarly outperform the orgnal one. The standard LBP operator for face recognton s u denoted as LBP where u stands for unform patterns, p, r p defnes the number of ponts that are unformly sampled over a crcle wth a radus r. Ths computaton s done block-wse and the results from all blocks are concatenated to form a fnal descrptor. For an overvew of the LBP operator for face recognton we refer the reader to [9]. In ths work we keep the orgnal form of the LBP as t s, but suggest an over-complete representaton bult on top of t. The proposed Over-Complete LBP (OCLBP) dffers from the orgnal LBP n two major propertes. Frst, t s computed wth overlappng blocks, smlar to [8]. The amount of vertcal- and horzontal-overlap s controlled by the two parameters v, h [0,) wth h= v= 0 degeneratng to non-overlappng blocks. The second dfference s n the vared block and radus szes. We repeat the LBP computaton for dfferent szes of block and radus, smlar to the mult-scale varant n [9]. We name the resultng representaton as OCLBP. More formally, gven an nput mage and a set of confguratons S = {( a,,,,, )} k b v h p r =, we dvde the mage to blocks n a sze of a b wth vertcal overlap of v, horzontal overlap of h and compute a LBP descrptor usng the u operator LBP. We repeat ths computaton for all p, r confguratons n S and concatenate the descrptors to a sngle vector whch s the resultng OCLBP descrptor. Snce the computatons of the dfferent confguratons are ndependent, the OCLBP descrptor can be easly paralleled. We show n Secton 7 that the OCLBP descrptor acheves the same performance as the standard LBP when they are used n ther orgnal dmenson. However, after applyng dmensonalty reducton, a sgnfcant gan n accuracy s acheved by the more elaborate scheme... Scatterng transform for face recognton The Scatterng Transform was ntroduced by Mallat n [3]. Ths work has been extended to varous computer vson tasks n [0, ]. As an mage representaton, a scatterng convoluton network was proposed n [0]. Ths representaton leads to an extremely hgh dmensonal descrptor that s nvarant for small local deformatons n the mage. For texture classfcaton, a Scatterng wavelet network managed to acheve state of the art results []. The output of the frst layer of a scatterng network can be consdered as a SIFT-lke descrptor whle the second layer adds further complementary nvarant nformaton whch mproves dscrmnaton qualty. The thrd layer, however, was found to have a neglgble contrbuton for classfcaton accuracy whle ncreasng the computatonal cost sgnfcantly. In ths work, we nvestgate the contrbuton of the Scatterng descrptor to our face recognton framework. In a smlar manner to the OCLBP, we fnd that the Scatterng descrptor s much more effectve when combned wth dmensonalty reducton. We refer the reader to [3] for a detaled descrpton of the Scatterng transform. 3. Wthn class covarance normalzaton Wthn Class Covarance Normalzaton (WCCN) has been used mostly n the speaker recognton communty and was frst ntroduced n [0]. The wthn class covarance matrx W s computed as follows: C n j j T W = ( x µ )( x µ ), C n = j= Where C s the number of dfferent classes, n s the j number of nstances belongng to class, x s the jth nstance of class and µ s the mean of class. In a sense, WCCN s smlar to the famly of methods

3 that down-regulate the contrbuton of the drectons n the vector space that account for much of the wthn class covarance. Ths s often done by projectng the data onto the subspace spanned by the egenvectors correspondng to the smallest egenvalues ofw. In WCCN, ths effect s performed n a softer way wthout performng explct dmensonalty reducton: nstead of dscardng the drectons that correspond to the top egenvalues, WCCN reduces the effect of the wthn class drectons by employng a normalzaton transform T = W /. Whle to the best of our knowledge t was prevously unused n face recognton, we show a clear mprovement n performance over the state of the art by usng the WCCN method when appled n the LDA subspace. In ths work, we also ntroduce an unsupervsed verson of WCCN, whch s shown to be useful n case we lack the necessary labeled data. In Secton 7, we evaluate our proposed method and show that t s an mprovement over the baselne algorthms. Furthermore, we show that although the unsupervsed WCCN algorthm does not make use of any label nformaton, t s compettve wth the orgnal supervsed WCCN n several scenaros. 4. Unsupervsed labelng A common and challengng problem n machne learnng s the benefcal utlzaton of successful supervsed algorthms n the absence of labeled data. In ths secton, we propose a smple unsupervsed algorthm for generatng valuable labels for the par matchng problem. Before descrbng the algorthm, we enumerate our two assumptons. Frst, we assume that we are equpped wth an unsupervsed algorthm that s able to acheve some classfcaton accuracy we consder ths algorthm as the baselne algorthm. We focus our dscusson on Algorthm ( A, B, T, t, t ) l r Inputs: A - a traned model of the baselne unsupervsed algorthm, B - supervsed algorthm, T - tranng set. tl - a threshold on the left tal, tr - a threshold on the rght tal. Output: C - a new traned model.. Compute the par-wse score matrx S usng A and T.. Assgn a label of to all pars wth a score above t r. 3. Assgn a label of to all pars wth a score below t l. 4. Assgn a label of 0 to all the other pars. 5. Tran a new model C usng the assgned labels and B 6. Return C algorthms that produce a classfcaton score and not just bnary labels. The second assumpton s on the shape of the dstrbuton of the classfcaton scores. We assume that the score dstrbuton s approxmately un-modal and has two tals. If our baselne algorthm manages to acheve a reasonable accuracy on the tranng set, we would expect to fnd many fewer classfcaton mstakes on the tals, rather than n the area around the mean score. In the case of the "same/not-same" classfcaton, we would expect the majorty of the scores n one tal to belong to pars that are matched and the majorty of the scores on the other tal to belong to pars that are msmatched. Ths behavor leads to the formaton of two (hopefully) separated sets: one conssts mostly of "same" pars and the other conssts mostly of "not-same" pars. The sze of each cluster s determned by the number of pars we pck from the correspondng tal. Ths number s a parameter that defnes a tradeoff between the number of desred labels and the confdence that we have n ths labelng. Therefore, we propose Algorthm. Note that except for postve and negatve labels there are also 'unknown' labels. In case we are equpped wth an algorthm (B) that s desgned to handle unlabeled samples (.e., a sem-supervsed algorthm), we provde t wth ths nformaton. Otherwse, we provde B exclusvely wth the postve and negatve sets of examples. The optmal values of the parameters t l and t r are related to the accuracy of the baselne model A, the shape of the score dstrbuton, and the number of labels that we want to generate. For example, f we are provded wth a baselne model whch acheves poor accuracy, we should expect poor labelng as well. In case the emprcal dstrbuton s symmetrc we can choose t l = t r, otherwse we mght consder the sze of the tals for each tal separately. Snce the generated labels are used to tran a new supervsed model we can apply Algorthm teratvely. Another possble extenson s to use a set of supervsed algorthms nstead of a sngle one and to determne the fnal labelng accordng to a votng scheme. 5. Fast supervsed and unsupervsed vector multplcaton recognton system We now descrbe n detal our proposed recognton system, whch we call VMRS for Vector Multplcaton Recognton System. Gven two samples, we need to decde whether they belong to the same class or not. Frst, each sample s projected to a low dmensonal subspace by WPCA. Then, we perform an addtonal supervsed dmensonalty reducton by applyng LDA. Fnally, we perform WCCN to the resultant feature vectors n the low dmensonal LDA-subspace and produce a score by applyng cosne smlarty. Therefore, the ppelne can be reduced to two matrx-vector multplcatons followed by cosne smlarty. We formally denote P, L and W as the 3

4 WPCA projecton matrx, LDA projecton matrx and Wthn Class Covarance (WCC) matrx, respectvely. Thus, gven two vectors,,, representng two face mages, the fnal score s defned as: T ( Mx) ( My) s( x, y, M ) = Mx My / Where, M = W LP. The fnal decson s made accordng to a prescrbed threshold that can be set to an Equal Error Rate (EER) pont, Verfcaton Rate (VR) pont, or alternatvely, can be learned by a SVM []. 5.. Unsupervsed ppelne The ppelne descrbed above s supervsed and requres labeled data. However, n many real-world scenaros we lack labels. In such cases we can apply Algorthm (Secton 4) n order to generate artfcal labels for the tranng set. Specfcally, we use the WPCA model as a baselne A and generate new labels accordng to the dstrbuton of the scores of pars n the tranng set. We then use these labels to estmate the wthn class covarance matrx (note that we do not apply LDA n ths case, snce t s unsupervsed). Snce WCCN computaton s based on pars from the same class, we only choose scores from one of the two tals (the 'same' tal). Then we treat each par n the 'same' group as a sngle class and merge classes that share the same samples,.e., we utlze strongly connected components n the connectvty graph nduced by the smlar pars. In our experments, we selected the parameter so that the pars wth dstances n the bottom 5% of the dstances of all possble pars wll consttute the same pars. Ths value was determned once, when performng a lmted nvestgaton of Vew of the LFW benchmark (ntended for parameter fttng) and remaned fxed. In Secton 7, we show that ths approach mproves over the baselne WPCA system. As already mentoned n Secton 4, one can terate between generatng new labels, usng them for tranng a new supervsed model, and generatng new scores. However, we dd not fnd that performng multple teratons mproves performance. Hence, Algorthm s employed only once. Wth the ntroducton of ths unsupervsed varant of WCCN, the proposed system s sutable for both the supervsed and the unsupervsed scenaros. It s mportant to clarfy that our proposed system, excludng the feature extracton phase, s extremely effcent n the sense of computatonal complexty. The most demandng computaton whch takes place durng the test phase s the lnear transformaton M on the par of orgnal feature vectors x, y. Ths has a great advantage over "lazy" learnng approaches such as [] whch make an explct use of the tranng set durng the test phase. The complexty of the tranng phase s domnated by the complexty of the computaton of the egen-problems that are encountered n WPCA and LDA and the computaton of the matrx square root ofw. 6. Dffuson Maps Many of the state of the art face recognton systems ncorporate a dmensonalty reducton component. The am of dmensonalty reducton s twofold. Frst, learnng n hgh dmensonal vector spaces s computatonally demandng. Second, n some cases and especally when the hgh dmensonalty stems from over-complete representatons, there s a large amount of redundancy n the data. Dmensonalty reducton technques attempt to solve both of these problems by explorng meanngful connectons between the data ponts and dscover the geometry that best represents that data. Most of the work done so far n face recognton appled lnear dmensonalty reducton. One of the problems wth lnear dmensonalty reducton s the mplct assumpton that the geometrc structure of data ponts s well captured by a lnear subspace. It has been shown [3] that real world sgnals, n most cases, have non-lnear structures and resde over a manfold. We propose to use a non-lnear dmensonalty reducton technque called Dffuson Maps (DM). We ntroduce a whtened varant of the conventonal DM framework and show how to deal wth the out-of-sample extenson problem, whch occurs n the test phase. In Secton 7, we show that by ncorporatng the DM framework nto the proposed recognton system of Secton 5, we acheve results whch are on a par wth the state of the art. Fnally, we show that by combnng DM and WPCA we are able to get an addtonal mprovement n accuracy. We wll brefly descrbe the man steps of DM (for a fully rgorous mathematcal dervaton we refer the reader to []). In the DM framework, we are provded wth a tranng set and affnty kernel k(, ). A commonly used kernel s the Gaussan kernel: c( x, x j ) k( x, x j ) = exp σ Where c(, ) s a metrc and σ s a parameter whch determnes the sze of the neghborhood over whch we trust our local smlarty measure. Usng the affnty kernel, we compute a par-wse affnty matrx K. Then, we convert K to a transton Markov matrx P by normalzng each row n K by ts sum: =, where D s a dagonal matrx normalzng the rows of K. Therefore, t P s a matrx, n whch the entry P s the ț j 4

5 Tables -3: Classfcaton accuracy (± standard error) of varous combnatons of classfers and descrptors n the unsupervsed, restrcted and unrestrcted settngs, respectvely. See text for detals regardng the classfers and descrptors. Table LBP OCLBP TPLBP SIFT SCATTERING Unsupervsed SQRT SQRT SQRT SQRT SQRT RAW 7.48 ± ± ± ± ± ± ± ± ± ± 0.63 WPCA ± ± ± ± ± ± ± ± ± ± 0.48 DM ± ± ± ± ± ± ± ± ± ± 0.5 WPCA+WCCN 78.8 ± ± ± ± ± ± ± ± ± ± 0.55 DM+WCCN ± ± ± ± ± ± ± ± ± ± 0.56 Table LBP OCLBP TPLBP SIFT SCATTERING Restrcted SQRT SQRT SQRT SQRT PCALDA ± ± ± ± ± ± ± ± ± ± 0.69 DMLDA 8.53 ± ± ± ± ± ± ± ± ± ± 0.58 WPCA 8.03 ± ± ± ± ± ± ± ± ± ± 0.38 DM 8.9 ± ± ± ± ± ± ± ± ± ± 0.33 Table 3 LBP OCLBP TPLBP SIFT SCATTERING Unrestrcted SQRT SQRT SQRT SQRT SQRT PCALDA ± ± ± ± ± ± ± ± ± ± 0.70 DMLDA 83.3 ± ± ± ± ± ± ± ± ± ± 0.73 WPCA 8.9 ± ± ± ± ± ± ± ± ± ± 0.65 DM 8. ± ± ± ± ± ± ± ± ± ± 0.6 probablty of transton from node x to node x j n t steps. A dffuson dstance after t steps s defned by: n t t t j =, k j, k k= D ( x, x ) ( P P ). Snce the dffuson dstance computaton requres the evaluaton of the dstances over the entre tranng set, t results n an extremely complex operaton. Fortunately, the same dstance can be computed n a much smpler way: By spectral decomposton of P, we get a complete set of egenvalues = λ0 λ... λn and left and rght egenvectors satsfyng: Pψ = λϕ. We then defne a mappng H :{ } n t x V accordng to: T t t Ht ( x ) = λψ,..., λψ l l, where ψ k ndcates the -th element of the k -th egenvector of P and l s the dmenson of the dffuson space V. It has been shown [] that for l= m the followng equaton holds: t t j t j H ( x ) H ( x ) = D ( x, x ). Ths result justfes the use of squared Eucldean dstance n the dffuson space. In practce, one should pck l< m accordng to the decay of ( λ ) n =. Ths decay s related to the complexty of the ntrnsc dmensonalty of the data and the choce of the parameterσ. 6.. Unform scalng Inspred by WPCA, we propose to wegh all coordnates n the dffuson space unformly. We do that by smply omttng the egenvalues when computng the embeddng. Therefore, we change the mappng H to [ ] H ( x ) = ψ,..., ψ. l T Whle orgnally nspred by the relaton between PCA and = WPCA, ths modfcaton results n a sgnfcant mprovement when applyng t to DM framework. We hypothesze that ths mprovement occurs because DM, as an unsupervsed algorthm, holds lttle nformaton n ts egenvalues regardng the actual dscrmnaton capablty. Confoundng factors, such as llumnaton, can be assocated wth some of the leadng egenvectors. 6.. Out of sample extenson Snce the doman of H s defned only on the tranng set, we cannot compute the embeddng for a new test sample. A trval soluton would be to re-compute the spectral decomposton on the whole tranng data and the new test sample from scratch. However, ths soluton s extremely costly n the sense of computaton tme. Thus, we propose a smpler soluton: Our approach assumes that the tranng data s suffcently dverse n order to capture most of the varablty of the face space. In ths case, we would expect the embeddng of a new test sample to be approxmated well by a lnear combnaton of embeddngs of the tranng samples n the low dmensonal dffuson space. A natural choce s to set the coeffcent for each tranng sample as the probablty of movng from t to the new test sample. Thus, for a new test sample x n + we compute the transton probabltes P n +, j, j n and defne ts embeddng to be n n H ( xn+ ) = Pn +, jψ j,..., Pn +, jψ lj. As a result we j= j= get an extended mappng :{ } n + H x = V, whch ncludes x n + as well. Our proposed extenson s qute smlar to the Nystrom method [4] that has been used n spectral graph theory. T 5

6 The man dfference n our formulaton s that we gnore the egenvalues due to the modfcaton descrbed above. 7. Expermental setup and results We evaluate the methods descrbed above on the LFW dataset []. As s customary, we test the effect of the varous contrbutons on the 0 folds of vew of the LFW dataset. There are three benchmarks that are commonly used, and we provde very compettve results on all three. The most popular supervsed benchmark s the "magerestrcted tranng''. Ths s a challengng benchmark whch conssts of 6,000 pars, half of whch are same pars. The pars are dvded nto 0 equally szed sets. The benchmark experment s repeated 0 tmes, where n each repetton, one set s used for testng and nne others are used for tranng. The task of the tested method s to predct whch of the testng pars are matched, usng only the tranng data (n all three benchmarks, the decson s done one par at a tme, wthout usng nformaton from the other testng pars). The second supervsed benchmark, constructed on top of the LFW dataset, s the "unrestrcted'' benchmark. In ths benchmark, the persons denttes wthn the nne tranng splts are known, and the systems are allowed to use ths nformaton. For example, n ths benchmark, the orgnal WCCN method can be used drectly snce the tranng set s dvded nto denttybased classes. Last, the unsupervsed benchmark uses the same tranng set. Here, however, all the tranng mages are gven as one large set of mages wthout any parng or label nformaton. The evaluaton task remans the same as before dstngush between matchng ("same'') and nonmatchng ("not-same'') pars of face mages. 7.. Front-end Our system makes no use of tranng data outsde of the LFW dataset, except for the mplct use of outsde tranng data through traned facal feature detectors that are used to algn the mages, snce we use the algned LFW-a [] set of mages. The algned mages were cropped to pxels as suggested n [6]. In contrast to other leadng contrbutons [5, 5, 6], we dd not apply any further type of preprocessng that utlzes pose estmators or 3D modelng. 7.. Descrptors and parameters We evaluate 5 dfferent descrptors: LBP, Three Patch LBP (TPLBP), OCLBP, SIFT and the Scatterng descrptor. For LBP we used the same parameters that were used n [6] whle for TPLBP we used the parameters reported n []. We used the SIFT descrptors computed by [7]. For the OCLBP descrptors, we used Vew n order to determne the followng set of confguratons (see Secton 3. for a detaled descrpton of the OCLBP parameters): S = {(0,0,,,8,),(4,4,,,8,),(8,8,,,8,3)} Note that n all three scales, the horzontal and vertcal overlap parameters are both set to half. For the Scatterng descrptor we used the Scatterng Toolbox release from [7]. We set t to use the Gabor wavelet and the values suggested n [7]: a scatterng order of, maxmum scale of 3 and 6 dfferent orentatons. The orgnal descrptor dmensons are 7080, 40887, 96, 3456 and 9650 for the LBP, OCLBP, TPLBP, SIFT and Scatterng, respectvely System parameters We used Vew of the dataset to determne the parameters of the system. The WPCA dmenson s set to 500, the DM dmenson s also set to 500 and the Gaussan kernel parameter s fxed atσ = 4. In the unrestrcted and restrcted benchmarks, we used LDA dmensons of 00, 00, 00, 30 and 70 for the LBP, OCLBP, TPLBP, SIFT and Scatterng descrptors, respectvely. As already mentoned n Secton 6, we chose the threshold n the unsupervsed WCCN algorthm such that the number of generated 'same' labels s 5% of all pars Results We evaluate the proposed system for each feature and ts square root verson under the restrcted, unrestrcted and unsupervsed protocols. The expermental results are presented n Tables -6, and depct the mean classfcaton accuracy û and standard error of the mean SE. The unsupervsed results for the ndvdual face descrptors are depcted n Table. The table shows the progresson from the baselne "raw" descrptors, before any learnng was appled, through the use of dmensonalty reducton (WPCA or DM) to the results of applyng unsupervsed WCCN (Secton 6.) on the dmensonalty reduced descrptors. As can be seen, the suggested ppelne mproves the recognton qualty of all descrptors sgnfcantly, n both the dmensonalty reducton step and n the unsupervsed WCCN step. No clear advantage to ether WPCA or DM s observed. The results obtaned by combnng the facal descrptors together (excludng the orgnal LBP descrptor) are reported n Table 4. Ths combnaton, here and throughout all fuson results n ths paper, s done by a smple summaton of the smlarty scores usng unform weghts. The table also shows, for comparson, the results of solely employng OCLBP and the best results obtaned by prevous works. Whle our face descrpton method s consderably smpler than I-LPQ* [8], whch s currently 6

7 the state of the art n ths category, t outperforms t, even wth the usage of a sngle descrptor. Results n the supervsed-restrcted benchmark are reported n Table for the ndvdual features and n Table 5 for the combned features. In Table, we present four possbltes whch dffer by the dmensonalty reducton algorthm used: PCA followed by LDA (PCALDA), DM followed by LDA (DMLDA), WPCA or DM. WCCN, s then appled n all four cases. As a usual trend, t seems that employng LDA n between the unsupervsed dmensonalty reducton (PCA or DM) and the WCCN method mproves results. It s mportant to clarfy that both LDA and WCCN were appled n a restrcted manner by usng only pars nformaton,.e. no explct nformaton about the denttes was used and each par formed a mn-class of ts own. Table 5 presents the combned results of all the descrptors, excludng the orgnal LBP descrptor (due to the use of OCLBP). The combned method ("DM+WPCA fuson") ncludes the four descrptors (wth and wthout square root) and both PCA+LDA+WCCN and DM+LDA+WCCN (a total of 6 scores). It s evdent that combnng the DM based method together wth the PCA based method mproves performance over usng PCA or DM separately. In comparson to prevous methods, our method outperforms the state of the art by a large margn. The only excepton s the "Tom-vs-Pete" [5] method whch uses an external labeled dataset, whch s much bgger than the LFW dataset, and employs a much more sophstcated face algnment method. Our system consderably outperforms the accuracy of 90.57% obtaned by [3] n the case of a smlarty-based algnment as used by LFW-a, n spte of the fact that our method does not use the added external data. The results for the supervsed-unrestrcted benchmark are depcted n Tables 3 and 6. The classcal form of WCCN [0] apples drectly to ths setup. Two systems outperform ours n ths category. The frst s CMD+SLBP (algned) whch s a commercal system [9]. The second [30] has a few dstngushng characterstcs, whch can be further utlzed to mprove our results. Frst, a dfferent algnment method was used. Second, features were extracted on facal landmarks. Fnally, ther proposed algorthm operated n a much hgher-dmensonal feature space, whch requres more computatonal resources. In all three experments, OCLBP acheves a very compettve accuracy as a sngle feature. For example, as can be seen n Table 5, n the restrcted case t acheves an accuracy whch s much better than the current best reported accuracy obtaned by [6]. The Scatterng transform based descrpton (Secton 3.), however, does not seem to mprove over descrptors of lower dmensonalty by a sgnfcant margn. Nevertheless, t plays a crucal role n ncreasng performance n fuson. System Accuracy I-LPQ*, algned [8] 86.0 ± 0.46 OCLBP ± 0.30 WPCA fuson ± 0.36 DM fuson ± 0.4 DM+WPCA fuson ± 0.37 Table 4: Comparson of classfcaton accuracy (± standard error) for varous systems operatng n the unsupervsed settng. System Accuracy LBP + CSML, algned [6] ± 0.5 CSML + SVM, algned [6] ± 0.37 Hgh-Throughput BIF, algned [4] 88.3 ± 0.58 Assocate-Predct [3] ± 0.56 Tom-vs-Pete + Attrbute [5] ±.8 OCLBP ± 0.69 PCA fuson 90.6 ± 0.56 DM fuson 90.6 ± 0.55 DM+PCA fuson 9.0 ± 0.59 Table 5: Comparson of classfcaton accuracy (± standard error) for varous systems operatng n the restrcted settng. System Accuracy LBP PLDA, algned [6] ± 0.55 combned PLDA [6] ± 0.5 face.com r0b [5] 9.30 ± 0.30 CMD + SLBP, algned [9] 9.58 ±.36 combned Jont Bayesan [3] ±.48 hgh-dm LBP [30] 93.8 ±.07 OCLBP ± 0.60 DM fuson 9.56 ± 0.45 PCA fuson 9.56 ± 0.54 DM+PCA fuson 9.05 ± 0.45 Table 6: Comparson of classfcaton accuracy (± standard error) for varous systems operatng n the unrestrcted settng. One can also notce that the unsupervsed WCCN n some of the cases acheves an accuracy whch s not far away from the accuracy obtaned by the orgnal supervsed WCCN. For example, for the OCLBP descrptor, WPCA + supervsed WCCN acheves an accuracy of 87.% for the restrcted case whle the WPCA + unsupervsed WCCN ppelne acheves an accuracy of 86.7 %. 8. Conclusons We propose an effectve method that seems to be unque n that t addresses all three benchmarks n a unfed manner. In all three cases, very compettve results are acheved. The method s heavly based on dmensonalty reducton algorthms, both supervsed and unsupervsed, n order to utlze hgh dmensonalty representatons. Necessary adjustments are performed n order to adapt methods such as WCCN and DM to the requrements of face dentfcaton and of the varous benchmark protocols. From a hstorcal perspectve, our method s "reactonary". The emergence of the new face recognton benchmarks has led to the abandonment of the classcal algebrac methods such as Egenfaces and Fsherfaces. However, both PCA and LDA play mportant roles n our ppelne, even though these methods are not appled 7

8 drectly to mage ntenstes. WCCN, whch s a major contrbutng component to our ppelne, was borrowed and adapted from the speaker recognton doman. However, t s closely related to other algebrac dmensonalty reducton methods. In contrast to recent contrbutons such as CSML [6] or the Ensemble Metrc Learnng method [9] that are nfluenced by modern trends n metrc learnng, our method demonstrates that classcal face recognton methods can stll be relevant to contemporary research. References [] G. B. Huang, M. Ramesh, T. Berg and E. Learned-Mller, "Labeled faces n the wld: A database for studyng face recognton n unconstraned envronments," Unversty of Massachusetts, Amherst, 007. [] Z. Cao, Q. Yn, X. Tang and J. Sun, "Face Recognton wth Learnng-based Descrptor," n Computer Vson and Pattern Recognton (CVPR), 00. [3] Q. Yn, X. Tang and J. Sun, "An Assocate-Predct Model for Face Recognton," n Computer Vson and Pattern Recognton (CVPR), 0. [4] E. Nowak and F. Jure, "Learnng vsual smlarty measures for comparng never seen objects," n Computer Vson and Pattern Recognton (CVPR), 007. [5] T. Berg and P. N. Belhumeur, "Tom-vs-Pete Classfers and Identty-Preservng Algnment for Face Verfcaton," n Brtsh Machne Vson Conference (BMVC), 0. [6] H. V. Nguyen and L. Ba, "Cosne Smlarty Metrc Learnng for Face Verfcaton," n Asan Conference on Computer Vson (ACCV), 00. [7] M. Gullaumn, J. Verbeek and C. Schmd, "Is that you? Metrc Learnng Approaches for Face Identfcaton," n Internatonal Conference on Computer Vson (ICCV), 009. [8] N. Kumar, A. C. Berg, P. N. Belhumeur and S. K. Nayar, "Attrbute and Smle Classfers for Face Verfcaton," n Internatonal Conference on Computer Vson (ICCV), 009. [9] T. Ahonen, A. Hadd and M. Petkanen, "Face descrpton wth local bnary patterns: Applcaton to face recognton," IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 8, p , 006. [0] A. Hatch and A. Stolcke, "Generalzed lnear kernels for oneversus-all classfcaton: applcaton to speaker recognton," n ICASSP, 006. [] R. Cofman and S. Lafon, "Dffuson maps," Appled and computatonal harmonc analyss, vol., no., pp. 5-30, 006. [] L. Wolf, T. Hassner and Y. Tagman, "Descrptor based methods n the wld," n Faces n Real-Lfe Images Workshop n European Conference on Computer Vson (ECCV), 008. [3] S. Mallat, "Group nvarant scatterng," [Onlne]. Avalable: arxv.org/abs/0.86. [4] N. Pnto and D. Cox, "Beyond Smple Features: A Large-Scale Feature Search Approach to Unconstraned Face Recognton," n Internatonal Conference on Automatc Face and Gesture Recognton (FG), 0. [5] N. Pnto, J. J. DCarlo and D. D. Cox, "How far can you get wth a modern face recognton test set usng only smple features?," n Computer Vson and Pattern Recognton (CVPR), 009. [6] T. Ojala, M. Petkanen and D. Harwood, "A comparatve study of texture measures wth classfcaton based on feature dstrbutons," Pattern Recognton, vol. 9, no., pp. 5-59, 996. [7] M. Hekklä, M. Petkänen and C. Schmd, "Descrpton of nterest regons wth center-symmetrc local bnary patterns," n Computer Vson, Graphcs and Image Processng, 006. [8] X.-M. Ren, X.-F. Wang and Y. Zhao, "An Effcent Mult-scale Overlapped Block LBP Approach for Leaf Image Recognton," Lecture Notes n Computer Scence, vol. 7390, pp , 0. [9] S. Lao, X. Zhu, Z. Le, L. Zhang and S. Z. L, "Learnng Multscale Block Local Bnary Patterns for Face Recognton," n Advances n Bometrcs, 007. [0] J. Bruna and S. Mallat, "Invarant scatterng convoluton networks," 0. [Onlne]. Avalable: [] L. Sfre and S. Mallat, "Combned scatterng for rotaton nvarant texture analyss," n European Symposum on Artfcal Neural Networks, 0. [] Y. Tagman, L. Wolf and T. Hassner, "Multple One-Shots for Utlzng Class Label Informaton," n Brtsh Machne Vson Conference (BMVC), 009. [3] Z. N. Karam and W. M. Campbell, "Graph-embeddng for speaker recognton," n Interspeech, 00. [4] C. Fowlkes, S. Belonge, F. Chung and J. Malk, "Spectral Groupng Usng the Nystrom Method," IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 6, no., pp. 4-5, 004. [5] Y. Tagman and L. Wolf, "Leveragng bllons of faces to overcome performance barrers n unconstraned face recognton," 0. [Onlne]. Avalable: [6] P. L, Y. Fu, U. Mohammed, J. H. Elder and S. J.D.Prnce, "Probablstc Models for Inference About Identty," IEEE Transactons on Pattern Analyss and Machne Intellgence, vol. 34, no., pp , 0. [7] "Image scatterng toolbox v3," [Onlne]. Avalable: [8] S. u. Hussan, T. Napoléon and F. Jure, "Face Recognton Usng Local Quantzed Patterns," n Brtsh Machne Vson Conference (BMVC), 0. [9] C. Huang, S. Zhu and K. Yu, "Large Scale Strongly Supervsed Ensemble Metrc Learnng, wth Applcatons to Face Verfcaton and Retreval," NEC, 0. [30] D. Chen, X. Cao, F. Wen and J. Sun, "Blessng of Dmensonalty: Hgh-dmensonal Feature and Its Effcent Compresson for Face Verfcaton," n Computer Vson and Pattern Recognton (CVPR), 03. [3] D. Chen, X. Cao, L. Wang, F. Wen and J. Sun, "Bayesan Face Revsted: A Jont Formulaton," n European Conference on Computer Vson (ECCV), 0. 8