Adaptive Class Preserving Representation for Image Classification

Transcription

1 Adaptve Class Preservng Representaton for Image Classfaton Jan-Xun M,, Qankun Fu,, Wesheng L, Chongqng Key Laboratory of Computatonal Intellgene, Chongqng Unversty of Posts and eleommunatons, Chongqng, Chna College of Computer Sene and ehnology, Chongqng Unversty of Posts and eleommunatons, Chongqng, Chna Abstrat In lnear representaton-based mage lassfaton, an unlabeled sample s represented by the entre tranng set. o obtan a stable and dsrmnatve soluton, regularzaton on the vetor of representaton oeffents s neessary. For eample, the representaton n sparse representaton-based lassfaton (SRC) uses L norm penalty as regularzaton, whh s equal to lasso. However, lasso overemphaszes the role of sparseness whle gnorng the nherent struture among samples belongng to a same lass. Many reent developed representaton lassfatons have adopted lasso-type regressons to mprove the performane. In ths paper, we propose the adaptve lass preservng representaton for lassfaton (ACPRC). Our method s related to group lasso based lassfaton but dfferent n two key ponts: When tranng samples n a lass are unorrelated, ACPRC turns nto SRC; when samples n a lass are hghly orrelated, t obtans smlar result as group lasso. he superorty of ACPRC over other state-of-the-art regularzaton tehnques nludng lasso, group lasso, sparse group lasso, et. are evaluated by etensve eperments.. Introduton SRC s a wdely used method n many omputer vson applatons nludng fae reognton and other mage lassfaton problems []. In SRC, a test sample s ollaboratvely represented by a dtonary onsstng of all the tranng samples, whh s formulated as y = A () where y s the test sample, A s the dtonary matr n whh eah olumn represents a sample, and s the orrespondng representaton vetor for y. A model fttng proedure s used to produe the soluton of. Ordnary least squares (OLS) s a smple way to estmate and t s well known that to norporate regularzaton on nto OLS results n more stable and nterpretable soluton. he lnear regresson model used n SRC s bult under the prnple of parsmony whh s also mportant n human perepton. Representaton of y by a small subset of entre tranng samples s a prate of parsmony []. In SRC, the lasso regresson s a promsng approah to selet a subset of tranng samples by mposng a L norm penalty on whh produes a sparse vetor [3]. hs onstraned least squares model s gven by: = arg mn y A + λ () where s L norm (vetor norm), s L norm, and λ balanes the fdelty term wth the regularzaton term. However, there s a lmtaton to lasso regresson. If there are some samples whose orrelaton are hgh, lasso prefers to selet only one sample from them and gnores others [4]. Suh phenomenon leads to nonsstent seleton of tranng samples [5]. Elast net (EN) regresson provdes a fleble model to address problem above [4]. Elast net rteron s defned as the followng optmzaton problem: = arg mn y A + λ( α) + λα. (3) he omple regularzaton funton ( α) + α s a onve ombnaton of the lasso and rdge penalty where α 0,. Besdes the sparsty of, the soluton of EN [ ] regresson has the effet of groupng that hghly orreted samples share smlar regresson oeffents. Many other mprovement studes of lasso regresson also tend to address the orrelaton problem. In [6], sample de-orrelaton s onsdered n the lasso regresson. A ovarate-orrelated lasso s proposed to selet ovarates most strongly orrelatng wth response varable n [7]. Another reent study [8] also onsders orrelaton among tranng samples whh proposes adaptve model balanng the L norm wth L norm guded by tranng set tself. here s an obvous drawbak for the mentoned methods above n lassfaton. hey do not norporate the lass 747

2 nformaton nto the representaton proedure, that s, the regresson s not aware of any ntra-lass struture. For eample, n SRC, the soluton of problem () s omputed frst and the followed deson s to determne whh lass has the smallest representatve resdual: Label ( y) = argmn ya ( =,, ), (4) where s the number of lasses, A denotes the samples from the th lass and s a vetor onsstng of oeffents assoated wth A (the entre tranng set A= A, A,, A and the representng vetor [ ] =,,, ). Only at ths moment, the label of tranng set s used. Besdes ths drawbak, let us arefully look at the parsmony prnple for lassfaton. A sparse means only a few tranng samples are nvolved n representng a test sample. hese tranng samples an be dstrbuted aross dfferent lasses or ome from only one or two lasses. Surely the seond ase s favorable for lassfaton. Hene t ould be more dsrmnatve to onstran the number of lasses but not the number of samples. In some ases, a good representaton of a test sample ould be a dense one by samples from the orret lass. So, strong sample-wse sparse onstrant on an result n a poor soluton. he group lasso (GL) regresson, proposed n [9], produes a representaton vetor of group-wse sparsty. GL predefnes sample groups before regresson and s to let groups ompete wth eah other durng the regresson, whh means all the members of a partular group are ether used or not used. In [0] lass spef sparse representaton lassfaton s proposed based on the group lasso regresson. In ths method tranng samples from a ertan lass are defned as a group, and a group sparse soluton of an be obtaned by solvng the followng regularzed optmzaton problem: = arg mn + λ = y A (5) Further mprovement of group lasso an be found n [] where the sparse group lasso (SGL) regresson s proposed: = arg mn + λ( α) + λα = y A. (6) SGL an be also seen as a varant of elast net. SGLR not only onsders the group-wse sparseness but also the sample-wse sparseness. A manually seleted value of α determnes the rato of these two knds of sparseness. Note that all lasses are equally onstraned despte of possble dfferent ntra-lass sample struture. In ths paper, we propose an adaptve lass preservng sparse representaton (ACPR) method and apply t to perform mage lassfaton. Our proposed method takes three fators nto aount. Frst, the lass nformaton should be preserved durng regresson; Seond, the sparseness of the representaton vetor s onstraned; hrd, the etent of the sample-wse sparseness of a lass s adaptve to ntra-lass orrelaton struture. Although prevous studes of lasso-type regresson onsder lass nformaton, there s no study, to our knowledge, whh eplots the ntra-lass orrelaton to buld an adaptve model. In ACPR, lasses reeve separate onstrant,.e., the onstrant on vetor s determned by orrelaton struture of the th lass. We fnd that our regularzer on eah lass an adaptvely balane the L norm wth L norm. hus, regresson wth ACPR an obtan a fne representaton result that all three mportant fators are mplemented. In the followng, we frst brefly revew and summarze the related lasso-type regresson methods. hen our regresson model s presented. Net, we gve an effetve teratve algorthm to solve the optmzaton problem. Eperments on dfferent datasets are preformed to evaluate the method.. Related lasso-type regresson tehnques hat a test mage s ollaboratvely represented by tranng set s a key phase n representaton based mage lassfaton algorthms. hey assume the test sample an be represented well as a lnear ombnaton of the tranng samples belongng to ts orret lass. Under ths lnear regresson framework, regularzaton on representaton vetor plays an mportant role n obtanng a orret soluton when data set s large. A general onstraned regresson model employs the followng optmzaton form: = arg mn y A +λ g( ) (7) where g( ) s a penalty to regularze. Varatons and noses are hghly lkely to be seen n a test mage. Wthout an effetve regularzer, the performane of regresson ould be degraded serously. Many lasso-type regularzers reeve nreasng attenton. hey tend to ompel the soluton of to be sparse. here are at least two merts to do ths. Frst sparse representaton has dsrmnatve nature to perform lassfaton. Seond noses on test mage are more lkely to be deteted. When g( )=, (7) s lasso regresson problem whh s very popular to perform sparse representaton. Lasso manly onsders sample-wse sparsty that only a very small subset of samples represent a test sample. If we defne samples from a same lass as a group, group sparse representaton onstrans the number of lass used n the regresson whh results n group-wse sparsty. When eah lass onssts of only one or two samples, group sparse representaton obtans a smlar result as lasso. Aordng to the defnton of group lasso = that g( )=, the ntra-lass representaton 748

3 Methods sub-vetor s onstraned by L norm whh produes a dense soluton of. By usng nave ombnaton of both sample-wse sparsty and group-wse sparsty together g( )=( α) + α, SGL balanes the two knds = of sparseness by a pre-assgned parameter. he soluton of EN regresson shows groupng effet, however, a group of samples whh are hghly orrelated an be aross lasses. Nevertheless, EN has ts mert over lasso that orrelaton among samples s onsdered. Here, we summarze the mentoned lasso-type regressons from four aspets n able. 3. Model of ACPR In ths seton, we onsder a regresson model whh preserves lass nformaton and has self-adjustable onstrant aordng to the spef ntra-lass nherent struture. For a ertan lass, we prefer L norm to L norm onstrant when samples of the lass are hghly orrelated. On the ontrary, L norm based onstran s a better hoe when samples are unorrelated. 3.. Formulaton he formulaton of the proposed regresson model s epressed as follows: mn Φ ( ) st.. y = A (8) = Samplewse sparsty Groupwse sparsty Property Correla ton Adaptve ness Lasso EN GL SGL ACPR able. A summary of four aspets of the fve related regresson methods. Correlaton stands for that the orre-laton among samples are onsdered. Adaptveness means the model has the apablty of self-adjustment for regularzaton aordng to the struture of data set. where Φ ( ) = ADag( ), the regularzer denotes the nulear norm of a matr whh s the sum of the sngular values of the matr, and Dag( ) transforms a vetor nto a dagonal matr. Suppose there are n samples n the th lass and eah sample s preproessed by normalzaton of zero mean and unt vetor length. he representaton vetor for the th lass s a olumn vetor n = [,,,, n ] R. Here we frstly dsuss the property of Φ( ). Assume that samples belongng to the th lass are dstnt from eah other, whh means samples are orthogonal AA = I. hen the regularzer s deomposed nto ( ) Φ ( ) = r Dag( ) A A Dag( ) =. (9) hus, for the th lass, the regularzer Φ( ) turns nto L norm penalty. hs s reasonable beause when there s no ntra-lass struture L norm s a better hoe to prevent overfttng. In the speal ase that every lass does not have ntra-lass struture, (8) s equvalent to lasso regresson. If we assume that the samples n the th lass are hghly orrelated that AA ( s a olumn vetor of sze n and every element s equal to one). hen we have Φ ( )= =. (0) So, f a lass has very smlar samples, ts orrespondng regularzer appromates L norm. Under the rumstanes that every lass has hghly orrelated samples, (8) turns nto GL regresson. In general, samples from a lass are nether too ndependent nor totally dental. he regularzer balanes the L norm wth L norm adaptvely aordng to the ntra-lass struture of a lass: Φ( ). () Let us onsder a speal ase that one sample, say the frst one, s mstakenly grouped nto a dfferent lass and t s ndependent to all other samples. Here we defne two blok matres: 0, = and Dag( )= 0 A Dag( / /,/) A 0 AA 0 () where 0 s a olumn vetor of sze n - and n whh every element s equal to zero, A / s sample matr wthout the frst ndependent sample, and,/ s the representaton vetor for A /. hen we an see the regularzer has followng property: Φ( ), 0 0, 0 = r 0 Dag(,/ ) 0 A Dag( / /,/) A 0, 0 =r Dag(,/ ) / / Dag(,/) 0 A A,,/ = + Φ ( ). (3) 749

4 So the ndependent sample s separated from others whh s onstraned by L norm alone. Furthermore, t s not hard to prove that f there are some subsets n a lass whh are ndependent of eah other, the regularzer s transformed nto subgroups: p Φ( )= Φ ( ) + +Φ( ) (4) where p denotes the number of subgroups n the lass. It ndates that, ompared wth lasso and group lasso, the proposed regularzer an eplot the ntra-lass struture further. hat s, although sample group s determned by the lass label, adaptaton of (8) s strong enough to reflet the ntra-lass struture. In real-world applatons, a test mage s more or less ontamnated by varatons and noses. So, we solve the followng optmzaton problem nstead of (8) 3.. Optmzaton arg mn y A + λ Φ ( ). (5) An teratve optmzaton algorthm s proposed to solve (8), whh s summarzed n Algorthm. In eah teraton of updatng, we deompose (5) nto sub-problems: = +λ arg mn yˆ A A Dag( ), =,,.(6) whh s known as trae lasso problem []. In ths way, Algorthm updates one lass at a tme. Algorthm : Proedure of Adaptve Class Preservng Sparse Representaton A= A A, the query mage y, : Input: tranng set [ ] parameter λ > 0, ε > 0, ntal representaton (0) (0) (0) vetor = ( ) ( ), t =. : Generate a random permutaton vetor m R flled wth ntegers from to the number of lass. new () t 3: Defne a temporary varable =. For = to new () t 4: Compute yˆ = y A + A. m() m() m() 5: Update only () t () by solvng (6) wth Algorthm. m he nput nludes y ˆ () and A m(). new (+) t 6: Let =. m() m() m End; 7: Let (+) t new =. (+) t () t 8: If ε, go to step 9, otherwse, let t = t+ and go to step ; 9: Output: he optmal representaton vetor. In optmzaton problem (8), the objetve funton onssts of a fdelty term and a regularzaton term. It s well known that OLS s onve and n [3] nulear norm s proved to be onve as well. So, lnear ombnaton of them s also a onve funton. And then we let L( )= L(,, ) denote the objetve funton of (8). We break nto small sub-vetors and update eah sub-vetor one by one at random order to avod prorty. he orrespondng sub-vetor of s replaed by new sub-vetor mmedately. So, f the sub-problem (6) s solved effetvely, the orgnal problem wll onverge to the unque global optmum soluton. o solve (6), we frst onvert t to the followng equvalent problem: J, mn yˆ A + λ J st.. J= ADag( ). (7) It an be solved by mnmzng the orrespondng augmented Lagrange multpler formulaton: ˆ = + λ + L(, J ) y A J r( Y ( J A Dag( ))) µ (8) + JADag( ) F where Y s a Lagrange multpler matr and µ s a penalty parameter. We solve (8), whh now s an unonstraned problem, wth alternatng dreton method (ADM) [4, 5]. J and are updated respetvely by fng the other one, and then Y s updated. Frst, we f to solve λ J=argmn J + J( A Dag( ) Y ). (9) J µ µ Sngular Value hresholdng operator an provde a lose form soluton to (9)[4]. hen we f J, and s obtaned by solvng ˆ = + arg mn y A r( Y ( J A Dag( ))) µ + JADag( ). F It also has a losed form soluton: = P(A yˆ + dag( A ( Y+ µ J))) P= (A A + µ Dag(dag( A A ))) F (0) () where dag( ) takes the dagonal elements of a matr and stores them n a vetor. he multpler s updated by gradent desent method: Y = Y+ µ ( JA Dag( )). () he proedure of solvng the sub-problem s summarzed n Algorthm. Sne the whole proedure of solvng optmzaton (6) s derved by the theory of ADM, the onvety of ths sub-problem guarantees ts global onvergene. After omputng optmal, we make the 7430

5 deson of label by (4) whh s wdely used n representaton based lassfatons. Algorthm : Solvng sub-problem (7) : Input: tranng set of the th lass A, the regressand y ˆ, parameter λ. : Intalze:, Y 0 0, J, µ > 0, ε > 0, k = 0. (0) k + 3: F the others and ompute J by solvng (9). ( k + ) 4: F the others and ompute wth (). 5: Update the multpler 6: If ( k + ) Y aordng to () k+ k k+ k J J ε, ε k + k + ε, and J A Dag( ), go to step 7, otherwse, let k = k+ and go to step 3; 7: Output: he optmal vetor me omplety Lke onventonal trae lasso, teratve optmzaton s used n solvng ACPR model. In one round of updatng, the 3 tme omplety of trae lasso s Ο ( n ), where n s the number of the entre tranng samples. he sub-problem of ACPR s also a trae lasso problem whose tme omplety 3 s Ο ( p ) (here we smply assume that eah lass has p tranng samples). And for eah update of the entre, the 3 tme omplety of proposed ACPR s Ο ( p ). Sne n= p, ( p 3 ) ( 3 p 3 ) = ( n 3 ) Ο Ο Ο. 4. Sparsty vs orrelaton For mage lassfaton, many papers argue that the omputaton of a representaton vetor by ollaboratvely representng a test sample s better than by representng t separately [3, 6]. he reason s that onstraned lnear regresson ompels samples to ompete wth eah other, whh makes the deson more relable. Sne t s nevtable that there are varatons and noses on test samples, to onstran the sparsty of the representaton vetor prevents t from overfttng. Some studes suggest usng lasso regresson wth L norm an perform robust lassfaton. In lasso regresson, ompetton s among entre tranng samples. However, lasso gnores the orrelaton of the tranng set. If some samples are orrelated, t s very lkely that only one of them represents a test sample. In [0, 7], t s proved that ompetton among lasses may obtan better performane. hese papers use GL regresson to represent a test sample where samples belongng to a same lass form a group. It s ommon that mages from a same lass may be orrelated to some etent. For a spef lass, the representaton sub-vetor s onstraned by L norm whh allows orrelated samples Fgure. Illustraton of the representaton vetors by lasso regresson, rdge regresson, group lasso regresson, and ACPR. he ORL fae data set onsstng of 40 subjets s used and eah subjet has 0 frontal fae mages. An artfal test sample s reated by ombnng 5 mages (from the frst lass) and some Gaussan noses together. from a lass to represent the test sample together and makes the representaton adequate. However, f there are plenty of tranng samples n a lass, the rsk of overfttng wll nrease when the ntra-lass struture s omple. SGL regresson and EN both onsder the orrelaton and sparsty, but a manually hosen parameter s requred to balane these two fators. An obvous drawbak of prevous lasso-type regressons s that t s mpossble to assgn dfferent onstrant for eah spef lass. In general, the ntra-lass struture an vary a lot n terms of orrelaton. he proposed ACPR onstrans eah spef lass dfferently by balanng L norm wth L norm. ACPR also eplots the potental rh ntra-lass struture of a spef lass. If there are mutually ndependent sub-groups, ACPR regroups them adaptvely so as to provde a more-refned representaton. We provde a vsualzaton of the representaton vetors omputed by dfferent regresson approahes n Fgure. Lasso regresson uses the least tranng samples to represent the test sample, whh apparently gnores the orrelaton. Rdge regresson whh uses a L norm as ts regularzer, produes a dense vetor where many orrelated samples respondng together. ACPR performs smlarly to GL regresson. However, the ntra-lass sparsty of ACPR s stronger than GL regresson and weaker than lasso regresson. It ndates that ACPR not only takes nto aount ntra-lass sample-wse sparsty but also group-wse sparsty n an adaptve way. For EN regresson and SGL regresson, f two parameters n them are arefully hosen ase by ase, they may obtan an appromate vetor as ACPR. But t s almost mpossble to mplement n prate. 743

6 Method t = t = 5 t = 8 t = 0 Lasso Rdge GL SGL EN ACPRC able. he reognton auray (%) orrespondng to t samples from eah lass on the G database. Method t = t = 3 t = 4 t = 5 t = 6 Lasso Rdge GL SGL EN ACPRC able 3. he mean auray (%) orrespondng to t samples from eah lass on the ORL database. 5. Eperments In ths seton, we evaluate the effetveness of the proposed ACPRC by omparng wth fve state-of-the-art regressons nludng Rdge, Lasso, GL, SGL, and EN. Some well-known mage datasets are used: two ommon fae databases (G and ORL), two large fae databases (LFW and FERE), MNIS handwrtten dgts database, and COIL0 objet reognton database. Some samples are shown n Fgure where ntra-lass varatons an be seen. All used mages are normalzed to have zero mean and unt L norm. 5.. Georga eh (G) database he G database, whh an be downloaded from onssts of 5 mages per subjet from 50 ndvduals taken n dfferent tmes. It haraterzes some varatons suh as faal epressons, luttered bakgrounds, and lghtng ondtons. Images were proessed to an order of 5 5. For eah subjet, we seleted t ( =, 5, 8, 0) mages to form the tranng set and the rest for testng. able shows the reognton rates and standard devatons of all the algorthms over 0 random splts. akng nto aount suh a omple stuaton, all the used methods faed a huge hallenge. he performanes of all algorthms, espeally rdge regresson, are not satsfatory beause of the lak of tranng samples and varous noses n the ase of t =. he reognton auraes rse along wth nrease of tranng samples for the most methods. ACPRC has advantage of performane n all ases, whh obvously ndate that our method has shown better Fgure. Some samples from FERE, LFW, G, ORL, COIL0, and MNIS databases (from top to bottom). tolerane for llumnaton and pose varatons. Espeally, ACPRC s the only one whose auray eeeds 80% when t = ORL database he ORL fae database s omposed of 40 dstnt subjets wth 0 mages per subjet sampled at dfferent tmes wth sorts of varatons suh as faal epressons, varyng llumnatons and faal detals (glasses or not). We seleted the frst, 3, 4, 5, and 6 mages from eah subjet for tranng and the remanng for testng. Images were reszed to 5. he reognton auray rates are reported n the able 3. We an fgure out that the auray rates of all the algorthms nrease along wth addng more tranng samples. GL and SGL perform better than Lasso, whh s dfferent from the results on G database. he ause may be that G has lower ntra-lass orrelaton, whh weakens role of group struture. Although methods wth groupng effet performed better, ACPRC outperformed the seond best around % on average LFW database he LFW s a large-sale database omposed of more than 3000 mages of unonstraned faes wth varatons of pose, llumnaton, epresson, msalgnment and oluson, and so on. In the frst test sheme (LFW), a subset of 43 subjets wth no less than mages per subjet was hosen. We randomly seleted 0 mages as tranng dataset and the 743

7 Method LFW LFW6 Lasso 7.± ±.44 Rdge 7.79 ± ±.0 GL 7.95 ± ±.05 SGL ± ±.68 EN 70.6 ± ±.86 ACPRC ± ±.57 able 4. he mean auray (%) wth standard devatons orrespondng to two test shemes respetvely on the LFW database. Method t = t = 3 t = 4 t = 5 Lasso Rdge GL SGL EN ACPRC able 5. he mean auray (%) orrespondng to t samples from eah lass on the FERE database. remanng mages as testng dataset. In the seond subset (LFW6), whh onssts of 85 subjets wth no less than 6 mages per subjet, we randomly used 5 mages as tranng dataset and the remanng mages as testng dataset. Images were proessed to an order of 5. he reognton auray rates of eah method were reported n able 4. We an easly fgure out that the advantage of our proposed ACPRC by the ompettve results. Espeally, n the subset of LFW6, only our method aheves the auray of 80% up. he method of EN just reahes 70.6% and 74.00% on the two subsets, and the reason s that EN does not onsder the group-wse sparsty FERE database In ths eperment, a subset wth 400 mages from 00 subjets s seleted from the FERE database, whh s arguably one of the largest publly avalable database. Eah subjet onssts of 7 faal mages wth manly epresson varatons and posture hanges. We used the frst, 3, 4, and 5 mages n eah ndvdual for tranng and the remanng mages for testng. Images were proessed to the resoluton of 5. he eperment results are lsted n able 5. It an be seen from able 5 that our method stll obtans the best results n all ases, although all the method perform not better when the number of mages per subjet s 3. More spefally, the reognton auray of our method reahes 70.50% when the number of mages per subjet s 5. he mprovement over other algorthms are respetvely 6.50%, 8.50%, 3.75%, 4.5%, and 9.5%. Method MNIS COIL0 Lasso ± ± 0.55 Rdge 8.36 ± ±.08 GL 84.6 ± ± 0.93 SGL ± ± 0.5 EN 85.0 ± ± 0.76 ACPRC ± ± 0.60 able 6. he mean auray (%) wth standard devatons on the MNIS and COIL0 databases MNIS and COIL0 databases he MNIS database of handwrtten has been wdely adopted n the feld of pattern reognton. It ontans 0 mages lasses, orrespondng to 0 handwrtten dgts from 0 9 and eah lass has more than 5000 mages. We seleted the frst 50 samples of eah subjet and randomly hose a half as tranng set and the rest as test set. he COIL0 database ontans 0 objets and eah objet has 7 mages taken at pose nterval of 5 degrees. We randomly seleted a half of eah objet as tranng dataset and the other half as test set respetvely. For these two databases, mages were reszed to 5 5. We reported the means and standard devatons of the auray rates n able 6. On the both databases, the state-of-the-art regularzaton methods worked smlar eept Rdge. ACPRC outperformed the seond best over % on the MNIS database and around % on the COIL0 database respetvely Dsusson on eperments A full summary of our etensve epermental results s llustrated n Fgure 3. Wthout mposng sparsty, performane of Rdge regresson s not omparable wth others n all ases, whh onfrms the fat that sparsty s essental to aheve robust reognton. On the ORL database, lasso regresson performs only a lttle better than Rdge regresson beause there are small varatons wthn lass. GL aheves the seond best result on G database. he reason ould be that the hange wthn eah lass of G database s larger and group-wse sparsty benefts the representaton. Group-wse sparsty of GL shows nstablty wth the larger standard devaton than ACPRC on the LFW database beause of the dfferent ntra-lass orrelaton for eah group. However, our proposed ACPRC onsders both the group-wse sparsty and the orrelaton of data whh s self-adjustable to spef wth-n lass struture. For the roles of group-wse sparsty and sample-wse sparsty, one s not neessarly better than another. Espeally, our method performs the perfet ompettve on G database beause of the superorty of adaptve lass preservng regularzaton term whh an balane the L regularzaton 7433

8 Fgure 3. Reognton auray rates on dfferent databases. (a) G database wth, 5, 8, and 0 mages of eah subjet for tranng. (b) ORL database wth, 3, 4, 5, and 6 mages of eah subjet for tranng. () FERE database wth, 3, 4, and 5 mages of eah subjet for tranng. (d) LFW, MNIS and COIL0 database. and group sparse regularzaton aordng to the orrelaton. None of these algorthms nludng our ACPRC performs perfet on the MNIS database, although ACPRC obtans the hghest auray. he man reason may le n the fat that the number of mages for eperment s not enough and the handwrtten dgt data do not ft the subspae struture well. ACPRC get the best auraes 99.03% on the COIL0 database, whh ndate that the adaptve lass preservng sparse regularzaton term also helps handle general pattern reognton problems. 6. Conluson In ths paper, we propose adaptve lass preservng sparse representaton for mage lassfaton. In the proedure of ollaboratvely representng a test sample, ACPR penaltes the representaton vetor of eah lass dfferently. Comparng wth prevous lasso-type regressons, ACPR balanes lasso regresson wth group lasso regresson adaptvely. o solve optmzaton problem of ACPR, we deompose the model nto sub-problems and solve t by ADM. he superorty of ACPR based mage lassfaton s valdated by etensve eperments. Aknowledgments: hs work was supported partally by the Natonal Nature Sene Foundaton of Chna under Grant Nos , U405, and And partally ths researh s funded by the Natonal Key Researh and Development Program of Chna under Grant Nos. 06YFB Referenes [] J. Wrght, M. Y, J. Maral, G. Sapro,. S. Huang and Y. Shuheng. Sparse Representaton for Computer Vson and Pattern Reognton. Proeedngs of the IEEE, 98(6) :03-044, 00. [] M. H. Hansen and B. Yu. Model Seleton and the Prnple of Mnmum Desrpton Length. Journal of the Ameran Statstal Assoaton, 96(454) : , 00. [3] J. Wrght, A. Y. Yang, A. Ganesh, S. S. Sastry and Y. Ma. Robust Fae Reognton va Sparse Representaton. IEEE ransatons on Pattern Analyss and Mahne Intellgene, 3() :0-7, 009. [4] H. Zou and. Haste. Regularzaton and varable seleton va the elast net. Journal of he Royal Statstal Soety Seres B-statstal Methodology, 67() :30-30, 005. [5] H. Zou. he adaptve lasso and ts orale propertes. Journal of the Ameran Statstal Assoaton, 0(476) 0. [6] S.-B. Chen, C. Dng, B. Luo and Y. Xe. Unorrelated Lasso. AAAI Conferene on Artfal Intellgene

9 [7] B. Jang, C. Dng and B. Luo. Covarate-Correlated Lasso for Feature Seleton. Mahne Learnng and Knowledge Dsovery n Databases: European Conferene. 04. [8] J. Wang, C. Lu, M. Wang, P. L, S. Yan and X. Hu. Robust Fae Reognton va Adaptve Sparse Representaton. IEEE ransatons on Systems, Man, and Cybernets, 44() : , 04. [9] M. Yuan and Y. Ln. Model seleton and estmaton n regresson wth grouped varables. Journal of he Royal Statstal Soety Seres B-statstal Methodology, 68() :49-67, 006. [0] S. Huang, Y. Yang, D. Yang, L. Huangfu and X. Zhang. Class spef sparse representaton for lassfaton. Sgnal Proessng, 6(38-4, 05. [] N. Smon, J. Fredman,. Haste and R. bshran. A Sparse-Group Lasso. Journal of Computatonal and Graphal Statsts, () :3-45, 03. [] E. Grave, G. R. Oboznsk and F. R. Bah. rae lasso: a trae norm regularzaton for orrelated desgns. Advanes n Neural Informaton Proessng Systems [3] B. Reht, M. Fazel and P. A. Parrlo. Guaranteed Mnmum-Rank Solutons of Lnear Matr Equatons va Nulear Norm Mnmzaton. Sam Revew, 5(3) :47-50, 00. [4] J. Ca, E. J. Candes and Z. Shen. A Sngular Value hresholdng Algorthm for Matr Completon. Sam Journal on Optmzaton, 0(4) :956-98, 00. [5] R. Lu, Z. Ln and Z. Su. Lnearzed alternatng dreton method wth parallel splttng and adaptve penalty for separable onve programs n mahne learnng. Asan Conferene on Mahne Learnng [6] L. Zhang, M. Yang and X. Feng. Sparse Representaton or Collaboratve Representaton: Whh Helps Fae Reognton? IEEE Internatonal Conferene on Computer Vson. 0. [7] N. Rao, R. Nowak, C. Co and. Rogers. Classfaton Wth the Sparse Group Lasso. IEEE ransatons on Sgnal Proessng, 64() : ,