2014 International Joint Conference on Neural Networks (IJCNN) July 6-11, 2014, Beijing, China Robust Bilinear Matrix Recovery by Tensor Low-Rank Repr

Transcription

1 4 Iteratioal Joit Coferece o eural etwors (IJC) July 6-, 4, Beiig, Chia Robust Biliear Matrix Recovery by Tesor Low-Ra Represetatio Zhao Zhag Shuicheg Ya Migbo Zhao Fa-Zhag Li Abstract For low-ra recovery ad error correctio, Low- Ra Represetatio () row-recostructs give data matrix X by seeig a low-ra represetatio, while Iductive Robust Pricipal Compoet Aalysis (I) aims to calculate a lowra proectio to colum-recostruct X. But either colum or row iformatio of X is lost by ad I. I additio, the matrix X itself is chose as the dictioary by, but (grossly) corrupted etries may greatly depress its performace. To solve these issues, we propose a simultaeous low-ra represetatio ad dictioary learig framewor termed Tesor (T) for robust biliear recovery. T recostructs give matrix X alog both row ad colum directios by computig a pair of low-ra matrices alterately from a uclear orm miimizatio problem for costructig a low-ra tesor subspace. As a result, T i the optimizatios ca be regarded as ehaced I with oises removed by low-ra represetatio, ad ca also be cosidered as ehaced with a clea iformative dictioary usig a low-ra proectio. The compariso with other criteria shows that T exhibits certai advatages, for istace strog geeralizatio power ad robustess ehacemet to the missig values. Simulatios verified the validity of T for recovery. Keywords Low-ra represetatio; tesor represetatio; biliear recovery; dictioary learig; error correctio I. ITRODUCTIO igh-dimesioal observatios ca be ecoutered i various emergig real applicatios attributed to the rapid developmet of sciece ad techology, such as face recogitio [3], [], ad gee selectio, leadig to the research topics of recoverig low-dimesioal structures from high-dimesioal data. ote that plety of real data (such as images, videos ad documets) ca ofte be characterized by low-ra subspaces [][][3][6], thus ivestigatig the low-ra structures of high-dimesioal data have attracted explosively icreasig attetio i the last years. Represetative wors dedicated to this topic cosist of [-], [5], [8], [6-7], [9-39], etc. Z. Zhag is ow with the School of Computer Sciece ad Techology, Soochow Uiversity, Suzhou 56, P. R. Chia; also with the Provicial Key Laboratory for Computer Iformatio Processig Techology, Soochow Uiversity ( cszzhag@gmail.com) S. C. Ya is with the Departmet of Electrical ad Computer Egieerig, atioal Uiversity of Sigapore, Sigapore ( eleyas@us.edu.sg) M. B. Zhao is ow with the Departmet of Electroic Egieerig, City Uiversity of og Kog, Tat Chee Aveue, Kowloo, og Kog SAR ( mzhao4@cityu.edu.h) F. Z. Li is ow with the School of Computer Sciece ad Techology, Soochow Uiversity, Suzhou 56, P. R. Chia; also with the Provicial Key Laboratory for Computer Iformatio Processig Techology, Soochow Uiversity ( lfzh@suda.edu.c) Oe most represetative low-ra recovery criterio is called Robust Pricipal Compoet Aalysis () [3], [8], [9], [6]. For a give observed data matrix X x, x,, x corrupted by certai sparse errors E, is able to recover X ( X X E ) from the followig uclear orm problem: Mi Y E, Sub X Y E, () Y, E where is the uclear orm of a matrix, i.e., the sum of the sigular values of a matrix, is, l -orm ( ) or l -orm ( ) for characterizig the error term E, ad is a positive, parameter. ote that uder much broader coditios, as log as the give observatios are corrupted by sufficietly sparse errors, ca exactly recover X from the above covex optimizatio problem [3]. The miimizer Y (with respect to the variable Y ) correspods to the pricipal compoets of X ad is also the low-ra recovery to X. It is also worth otig that ca well address the gross corruptios with large magitude [3] if oly a fractio of etries are corrupted [6]. But is a trasductive algorithm, so it caot embed ew data [6]. Besides, the formulatio of implicitly assumes that the uderlyig data structures lie i or ear a sigle lowra subspace. But most real-world data are described usig a uio of multiple subspaces [], [], so the recovery of may be iaccurate i practice. To address these problems, a effective extesio to, referred to as Iductive Robust Pricipal Compoet Aalysis (I) [6] was proposed to improve recetly via calculatig a low-ra proectio U u, u,, u to remove the possible corruptios ad recover the origial data. Give a observed data matrix X, I calculates the uderlyig low-ra proectio U ad the pricipal compoets Y y, y,, y from the followig covex uclear orm problem: Mi U E, Sub X Y E, Y UX. () U, E After the miimizer U is achieved, the origial data ca be recovered through U X (or X E ). Based o the leart lowra proectio U, the give data ca be mapped oto the uderlyig subspaces ad the possible corruptios ca also be efficietly removed [6]. But ote that I performs error correctio alog colum directio of give matrix. As a result, row iformatio of data is lost i the I formulatio. To well cope with mixed data with corrupted observatios, aother effective low-ra criterio, Low-Ra Represetatio () [], [], was also proposed for subspace recovery ad segmetatio. For subspace recovery, sees a low-ra represetatio V v, v,, v amog all cadidates that represet all data vectors as the liear combiatio of bases i /4/$3. 4 IEEE

2 a give dictioary D. By settig X itself as the dictioary (i.e., D=X), the covex criterio of is give as Mi V E, Sub X DV E, D X. (3) V, E After calculatig the optimal solutio V, E, the origial data is recovered as X E (or XV ). Differet from I, recovers or segmets give data alog row directio, but i cotrast colum iformatio of the give data matrix is lost i the problem. ote that ca also be used to hadle the issue of error correctio, but is also a trasductive method as, so it also caot hadle ew data efficietly. As a result, both ad will be iappropriate for the practical applicatios requirig fast olie computatio [6]. It is also oted that, for robust subspace recovery, requires that sufficiet oiseless data is available i the dictioary (i.e., oly a part of D is corrupted). But ote that most real data are cotamiated by various errors such as outliers, corruptios ad oise, so settig X itself as the dictioary directly may be ivalid ad may depress the robustess performace of greatly for subspace recovery ad segmetatio [], [3]. To ehace the robustess of the low-ra recovery to oises, (grossly) corruptios or missig values i data, we propose to icorporate the cocept of tesor represetatio ito the lowra represetatio ad preset a biliear recovery criterio called Tesor Low-Ra Represetatio (T) for ehaced subspace represetatio ad error correctio. Compared with, I ad, the proposed T exhibits certai attractive advatages. First, to ehace the robustess agaist oise or corruptios, ad to well hadle the data with missig values, T aims to recostructs give data alog both row ad colum directios at the same time by embeddig the data oto a low-ra tesor subspace U V spaed applyig two low-ra matrices U ad V calculated alterately from a uclear orm miimizatio problem. Secod, T exhibits a strog geeralizatio capability. ote that the formulatio of T seamlessly itegrates the low-ra represetatio ad dictioary learig ito a uified framewor, that is, it ca perform simultaeous subspace recovery, error correctio ad dictioary learig. Specifically, whe learig a low-ra proectio to costruct a clea iformative dictioary, T is cosidered as a ehaced versio of I based o the oises ad corruptios removed data. Similarly, whe learig the low-ra represetatio for subspace recovery, T is regarded as a ehaced versio of learig with the traied iformative dictioary. Thus, the subspace recovery performace ad the robustess agaist oise, corruptios ad missig values ca be greatly boosted by our T algorithm, compared with other related criteria. The paper is outlied as follows. Sectio II briefly reviews other related wors. Sectio III proposes the T algorithm mathematically. Subsequetly, we i Sectio IV describe the simulatio settigs ad evaluate our algorithm. Fially, the paper is cocluded i Sectio V. II. RELATED WORK Recet years have witessed a lot of efforts ad icreasig iterests o the low-ra recovery i the literature. The most related uclear orm miimizatio based recovery criteria to ours are, I, ad Latet (Lat) [], []. The priciples of I ad are illustrated i the top left ad top right of Figure, respectively. Obviously, I ad perform recovery ad error correctio alog either colum or row directio of the data matrix, thus row or colum iformatio of the data matrix is lost by them. u, u, u, x, x, x, v, v, v, u, u, u, x, x, x, v, v, v, u u u x x x v v v,,,,,,,,, u, u, u, x, x, x, x, x, x, v, v, v, u, u, u, x, x, x, x, x, x, v, v, v, u u u x x x x x x v v v,,,,,,,,,,,, Figure : The priciples of I (top left), (top right), T (top) ad Lat (bottom). Aother related recovery criterio to T is Lat that recostructs the data matrix X from two directios as well. To well deal with the issue of isufficiet samplig ad improve the robustess to oise, Lat costructs the dictioary by usig both observed ad uobserved hidde data, ad solves the followig covex problem: Mi V, Sub X DV, D X, X, (4) V where X is the hidde data, ad the cocateatio (alog colum) of X ad X is applied as the dictioary D. Fially, the optimizatio problem of Lat is formulated as Mi U V E, Sub X UX XV E (5) U, V, E whe corrupted data is icluded. Lat resolves the problem of isufficiet samplig ad is show to be more robust tha [], but ote that observed ad hidde data are sampled from the same collectio of low-ra subspaces []. Thus, Lat may suffer from the same problem as, sice oe still caot esure there are sufficiet oiseless data available i D X, X. The priciple of Lat is illustrated i the bottom of Figure. Ituitively, Lat is a combiatio of ad I. After the solutio U, V, E is calculated, Lat decomposes X ito a low-ra XV, a low-ra U X, ad a sparse part E fittig oise. Although colum ad row iformatio of the data matrix X are reflected i the fial recostructive procedure of Lat, i.e., U X XV, ote that the low-ra matrices U ad V ca be alterately calculated from the followig two equivalet covex problems at each iteratio: Mi V E, Sub X XV E, E UX E, 6(a) V, EU U, EV U U U Mi U E, Sub X UX E, E XV E. 6(b) V V V ote that whe solvig V for low-ra represetatio at each iteratio, E is fixed ad the optimizatio i Eq.6 (a) is U equivalet to the formulatio by settig the matrix X as

3 the dictioary if E is cosidered as the error matrix E i U. Similarly, whe solvig U for low-ra proectio, E V is fixed ad the problem i Eq.6 (b) is equivalet to I for seeig pricipal compoets of X if E V is regarded as the error matrix E i I. It is also worth otig that the so-called hidde effects that X brig to the problem is still uclear. To boost the robustess to oise ad well hadle data with missig values, this paper itroduces a ew mechaism to recover the origial data from two directios, as illustrated i the top of Figure. Based o this strategy, the proposed T ca exhibit certai properties over existig criteria, e.g., strog geeralizatio power ad robustess ehacemet. III. BILIEAR LOW-RAK RECOVERY FRAMEWORK We address the biliear matrix recovery problem by learig two low-ra factors U ad V at the same time to recover the give matrix X from row ad colum directios i the form of tesor represetatio (i.e. UXV ) of X. I tesor scearios, X ca be regarded as the secod-order tesor i tesor space d [], [8]. Deote by U u, u,, u d d d ad V v, v,, v d d to represet data X, the the tesor product U V is a subspace of ad the dd proectio of X oto the subspace U V is UXV. This paper cosiders d ad d. We will also elaborate the proposed biliear recovery criterio ca perform simultaeous dictioary learig ad low-ra represetatio. A. Problem Formulatio For a give data matrix X corrupted by certai sparse errors or missig values E, we propose to recover the origial data X ( X X E ) alog both row ad colum directios simultaeously. Specifically, we aim at calculatig a low-ra represetatio matrix V ad a low-ra proectio matrix U such that X DV, D UX from the followig ra miimizatio problem: Mi ra U ra V X Y, Sub Y DV, D UX, (7) U, V where is the uclear orm of a matrix, X Y idetifies the sparse errors, (either, l -orm or l -orm ) is for, characterizig the errors E, is a positive parameter, ad a iformative dictioary D UX is defied by computig a lowra proectio U to proect give poits oto the uderlyig subspaces ad the etries will be updated at each iteratio. Specifically, E E is desiged for hadlig radom i, i, corruptios, ad E E ca model the samplespecific corruptios ad outliers well. If l -orm is imposed, i i,, o E, we ca rewrite the above problem as U, V, E. (8) Mi ra U ra V E, Sub X Y E, Y DV, D UX, As a commo practice i ra miimizatio problems [], [], [3], [4], we ca replace the ra fuctio with the uclear orm. The the above problem further becomes Mi U V E, Sub X DV E, D UX, (9) U, V, E, from which the optimal solutio U, V, E ca be achieved. Therefore, the origial data ca be recostructed or recovered as U XV (or X E ). ote that both U ad V are required to recover X from two directios, but it is difficult to compute U ad V simultaeously. I this paper, we solve U ad V alterately ad idepedetly, that is, other variables are fixed whe optimizig U or V at each iteratio. More specifically, oe ca calculate U ad V alterately from the followig two equivalet covex problems to Eq.9: Mi U E, Sub X U E, XV, (a) U, E, Mi V E, Sub X DV E, D UX, (b) V, E, where XV deotes the errors corrected ad oise removed data by the low-ra represetatio V, ad D UX is a clea iformative dictioary defied by proectig give data oto the uderlyig subspaces usig the low-ra proectio U as I. I other words, the proposed T framewor ca perform simultaeous subspace recovery, error correctio ad dictioary learig. Although the recovery problem i Eq.6 ca also lear a low-ra proectio, our approach is differet from theirs i two mai aspects. First, the formulatio of Eq.6 is directly built o the criterio by ivolvig the lowra proectio ad low-ra represetatio to estimate for low-ra subspace recovery ad proectio, while our T criterio is based o a ew biliear model that aims to recover give data matrix from two directios (i.e. row ad colum) by calculatig a pair of low-ra matrix factors to well hadle the cases corrupted by oises ad missig values i additio to recoverig low-ra subspaces. That is, the proectio i Eq.6 is updated directly at each iteratio, but we istead calculate a low-ra proectio U to update the dictioary i T by proectig the give data ito the uderlyig subspaces at each iteratio. Secod, T is more geeral tha Eq.6, sice our T formulatio is cosidered as a ehaced versio of i the optimizatios. ote that the above two covex problems ca be solved by usig various methods, e.g., Augmeted Lagrage Multiplier (ALM) [8], [9]. Whe calculatig the low-ra coefficiets U to colum-recostruct, we firstly set U to be a idetity matrix, that is, the dictioary is iitialized with the give matrix X. After obtaiig V from Eq. (b), we ca update the low-ra proectio matrix U from Eq. (a) to columrecostruct. With U ad V obtaied alterately, the sparse error matrix E ca be obtaied from Eq.9. ote that, i the alteratig optimizatios, the covex problem i Eq.(a) ca be cosidered as the ehaced I usig errors corrected ad oise removed data matrix by V. Recall that I aims at learig a low-ra proectio to remove the possible corruptios i give data efficietly through proectig give data oto the uderlyig subspaces [6]. It is worth otig that the data structures represeted by XV will be easier to be proected oto the uderlyig subspaces tha X, because the process of optimizig XV has already corrected the errors i data ad segmeted poits ito their respective subspaces by the low-ra represetatio V. Similarly, the problem i Eq. (b) ca be cosidered as ehaced criterio with a clea iformative dictioary D leart from Eq. (a). Therefore, T has the potetial to improve the robustess agaist the

4 oises ad missig values, compared with ad I. Based o the relatioships amog, I ad Lat described i Sectio, T ca also be cosidered as the ehaced versio of Lat. B. Optimizatio for Recoverig Low-Ra Matrices I this paper, we use the iexact ALM method [9] to solve our T problem due to efficiecy. We first covert the problem i Eq.9 to the followig equivalet oe: Mi J F E, Sub X UXV E, U J, V F, () J, F, U, V, E, from which the variables ca be obtaied. ote that if l -orm is imposed o the sparse error term E, that is to solve E from Mi U V E with respect to X DV E, D UX, the U, V, E term E ca be obtaied from E arg mi / / E E E E / Y that ca be solved usig the shriage F operator [], [9]. Motivated by [],[6], we istead compute U ad V by covertig Eq.9 to a simpler problem. With similar argumet, the solutio U to Eq.9 ca be factorized ito U U R T with R obtaied by orthogoalizig the colums of XV. Similarly, the solutio V to Eq.9 ca be factorized ito V Q V, where Q is obtaied through orthogoalizig T T the colums of X U. As a result, the problem i Eq.9 ca be equivalet to the followig formulatio by replacig V ad U with Q V ad U R T respectively: Mi J F E, J, F, U, V, E, () Sub X DV E, U J, V F, D U where R T XQ. Based o such strategy, the computatioal burde ca be greatly reduced, especially for low-dimesioal large-scale problems, that is, is larger ad is relatively smaller, because the above problem is solved with a complexity of O 3. ote that whe orthogoalizig R ad Q, U ad V are defied as the idetity matrices. The covergece properties of the iexact ALM are well studied if the umber of blocs is at most two ad it is able to geerally perform well i practice [], [4]. Sice there are more tha two blocs i our T, which is similar to I, ad Lat, ad the obective fuctio is o-covex, it will be hard to prove that the solutio of our T coverges to the global optimal solutio, which is actually a commo situatio whe hadlig o-covex problems [9], [3], [7]. So, the theoretical study for the covergece aalysis of our T method eeds to be explored i future. The optimizatios of T alterately solve the five blocs at each iteratio, which are easily solvable. This paper observes that. is a good choice for our T. Uder this settig, we experimetally observe that T ca usually coverge with the iteratio umber withi the rage of 3~5. IV. SIMULATIO RESULTS AD AALYSIS We evaluate the effectiveess of our T algorithm, alog with illustratig the compariso results. Oe bechmar face dataset ad oe sythetic dataset are ivolved i this study. The real face database is ORL database [7], ad the sythetic dataset is a Swiss roll dataset which follows a Swiss-roll distributio. As a commo practice, the images of ORL are resized to 3 3 pixels for computatioal cosideratio. A. Baselies ad Simulatio Settigs I this paper, we maily examie T for low-ra recovery ad error correctio. The performace is compared with five most related methods, that is,, Lat,, I, ad Sparse Represetatio (SR) [3], [4], [], []. For fair compariso, l, -orm or l -orm is regularized o the sparse error term E of the formulatios of SR,, I,, Lat ad T for each simulatio. (a) SR has similar appearace ad applicatios as, e.g., recovery, recostructio ad oise removal. For recovery, SR computes a sparse represetatio S by solvig the followig l -orm miimizatio based problem [3], []: Mi S E, Sub X DS E, D X, Diag S, (3) S, E where SR eforces Diag S to avoid the trivial solutio S I, ad the give matrix X is usually set as the dictioary for learig the sparse represetatios. After the miimizer S s, s,, s (with respect to the variable S) to the above problem is achieved, the origial data ca be similarly recostructed as XS (or X E ), which is aalogous to the recovery of, where each colum vector s i represets the coefficiets for recostructig the data poit x ad each etry i s represets the cotributio of x for recostructig x. i i, (b) Parameter Settigs. For the problems of, I,, Lat ad SR, there is a commo parameter that depeds o the actual oise level of datasets to estimate []. Accordig to [], a relatively large should be used whe the icluded errors are slight ad otherwise oe should tue to be relatively small. Besides, LPP ad PE eed to estimate the eighborhood size. I this study, the parameters of each criterio are carefully chose ad the best results over tued parameters are reported for comparig the performace. (c) Evaluatio Metrics. For recovery ad error correctio, the result of each criterio is evaluated by the recostructio error X X / X, where X is the low-ra recovery cl co cl cl F F to the give data matrix which is ot corrupted ad X is the co recovered result over differet percetages of corruptios. We perform all simulatios o a PC with Itel (R) Core (TM) i5 CPU 3. Gz 3.9 Gz 4G. B. Face Image De-oisig via Error Correctio We evaluate the image de-oisig capability of the proposed T for hadlig face images uder differet levels of pixel corruptios. The recovery performace of our T is maily compared with those of, I, ad Lat. The images are selected from the ORL database that cosists of variatio i facial expressio (smilig/o smilig), facial details (glasses/o glasses) ad poses. I total, the database has 4 persos ad cosists of images per perso. Three faces are selected for the experimets ad a data matrix of size 3 96 is created. ote that the gray values of the face images are ormalized to [, ] for this simulatio. To ivestigate the robustess of various low-ra recovery criteria to corruptios,

5 two settigs (i.e., faces are corrupted by radom corruptios ad sample-specific corruptios) are tested respectively. Recovery agaist radom pixel corruptios. We firstly test the case that face images are corrupted by radom pixel corruptios. I this simulatio, we corrupt a percetage of radomly selected pixels from the face images by replacig the gray values with iverted values, i.e., each gray value g is replaced by -g. The corrupted pixels are radomly selected from the test faces ad the locatios are uow to each method. I this study, we vary the percetage of corrupted pixels from percet to 9 percet, ad accordigly icrease the values of for each method. We apply exp as a quatitative evaluatio criterio of the recovery performace of each algorithm, i.e., the closer X ad X are, the bigger cl co the value of exp is. Figure shows the results of T ad its four competitors as a fuctio of the level of pixel corruptios. Figures (b), (c), (d) ad (e) maily evaluate T for recoverig the faces with (or ) percetage of corrupted pixels. Figures (f) ad (g) quatitatively evaluate each l -orm or l, -orm based method for error correctio. The results are averaged over 5 radom pixel selectios. From the results, we fid that our proposed T wors better tha other criteria i correctig corruptios, ad the performace of T degrades slower tha the others with the icreasig corruptio percets. It is also observed that l - orm based criteria are more appropriate choice for recoverig, the radom corruptios tha l -orm based criteria. Recovery agaist sample-specific corruptios. We corrupt a percetage of radomly chose sample-specific corruptios, i.e., colums of the data matrix. I this study, we add Gaussia oise with zero mea ad. variace to the colums. The corrupted colums are radomly chose ad the locatios are also uow to the users. We also vary the percetage of corrupted colums from percet to 9 percet, ad icrease the values of accordigly for each algorithm. Figure 3 illustrates the result of each method as a fuctio of the level of colum corruptios. Figures 3 (b), (c), (d) ad (e) examie our preseted T for recoverig the face images with (or 5) percetage of colum corruptios. Illustratios show that our T is able to effectively detect the corruptios ad correct them. Figures 3 (f) ad (g) illustrate the quatitative evaluatio of error correctio for l, -orm or l -orm based criterio. We average the results over 5 radom colum selectios. Similar fidigs ca be foud here, that is, T ca outperform other methods i idetifyig ad correctig, the corruptios i most cases. It is also observed that the l - orm based error terms are able to well model the samplespecific corruptios tha l -orm based criteria. (a) Recostructio Accuracy (f) I Lat T (b) (d) (c) Recostructio Accuracy (g) I Lat T Percetage of Corrupted Pixels (e) Percetage of Corrupted Pixels Figure : Recovery uder radom corruptios. (a) Origial face images from ORL; (b) percet of pixels are corrupted; (c) percet of pixels are corrupted; (d) Recovered faces by l -orm based T; (e) Estimated sparse errors; (f) Recostructio accuracies across corrupted percets for each l, -orm based method; (g) Recostructio accuracies across corrupted percets for each l -orm based criterio. (a) Recostructio Accuracy (f) I Lat T (b) (d) (c) Recostructio Accuracy (g) I Lat T Percetage of Corrupted Colums (e) Percetage of Corrupted Colums Figure 3: Recostructio uder sample-specific corruptios. (a) Origial faces; (b) percet of colums are corrupted; (c) 5 percet of, colums are corrupted; (d) Recostructed faces by the l -orm based T; (e) Estimated sparse errors; (f) Recostructio accuracies across corrupted percets for each l, -orm based method; (g) Recostructio accuracies across corrupted percets for each l -orm based criterio.

6 C. Error Correctio o Sythetic Data I this simulatio, we address aother experimet to evaluate the proposed T method for error correctio i oisy case usig the sythetic Swiss roll dataset. The sampled dataset X cotais colum vectors totally ad each colum vector correspodig to a sample poit i a three-dimesioal space. The subspace recovery result of our proposed T method is compared with those of I,, SR ad Lat. We illustrate the recovery result of each approach i Figure 4. I our experimets, we add Gaussia oise (with zero mea ad variace equalig to two) to the x-coordiates of percet of the data vectors (deoted by square symbol) ad also add the oise with the same desity to the y-coordiates of aother percet of vectors (deoted by circle symbol), respectively. The oised Swiss roll dataset is illustrated i Figure 4(a). Observig from the recovered data i Figure 4, we see clearly that our T algorithm outperform other compared criteria, which is because of its capability of embeddig the give data matrix X ito a low-ra tesor represetatio subspace oto which the icluded errors ca be automatically ad effectively corrected by the proposed T criterio from both row ad colum directios at the same time. oised "Swiss roll" (a) (b) (c) (d) (e) - (f) Figure 4: Corrected result of each algorithm o Swiss roll : (a) oised dataset; (b) Recovery result ( U X ) of I; (c) Recovery result ( XV ) of ; (d) Recovery result ( XS ) of SR; (e) Recovery result ( U X XV ) of Lat; (f) Recovery result ( U XV ) of our T. V. COCLUDIG REMARKS I this paper, we have proposed a biliear subspace recovery framewor called T for effective error correctio ad data repairig. A attractive property of our T criterio is its formulatio seamlessly itegrates low-ra represetatio ad dictioary learig ito a uified framewor, which provides us with a ew mechaism for recoverig low-ra subspaces ad simultaeously learig a iformative dictioary from a uclear orm problem. Specifically, T proceeds low-ra recovery through ehacig the robustess agaist oise ad missig values by simultaeously cosiderig colum ad row iformatio of the give data matrix, thus the shortcomigs of ad I ca be effectively overcome. I additio, we mathematically elaborate that our T ca be regarded as ehaced versios of other existig criteria. Image recovery ad visualizatio simulatios have verified the effectiveess of our T i represetig real images ad boostig the robustess agaist to ad corruptios. But there are certai future wor that still requires to be explored. First, the parameter for the error term cotrols the performace of virtually all low-ra recovery criteria, icludig our proposed T. I this study, we experimetally observe that T is capable of worig well uder a wider rage of the parameter cofiguratios, but it still remais uclear how to optimally determie the values of theoretically for data cotamiated by various errors. Secod, to date it is still challegig to strictly prove the covergece of iexact ALM based recovery criteria, icludig I,, Lat ad our proposed T, which iclude more tha two blocs to optimize [] [4]. I this paper, we observe from the experimetal results that T ca coverge with satisfactory iteratio umbers, but the theoretical covergece proofs for the preseted T are still worth ivestigatig i future. VI. ACKOWLEDGEMETS This preset wor is partially supported by the Maor Program of atioal atural Sciece Foudatio of Chia (Grat o. 6333), ad the Sigapore atioal Research Foudatio uder its Iteratioal Research Sigapore Fudig

7 Iitiative ad admiistered by the IDM Programme Office. REFERECES [] G.C Liu, Z.C Li, S.C Ya, J. Su, Y. Yu, ad Y. Ma, Robust Recovery of Subspace Structures by Low-Ra Represetatio, IEEE Tras. Patter Aalysis ad Machie Itelligece, vol.35, o., pp.7-84, 3. [] G.C. Liu, Z.C. Li, ad Y. Yu, Robust Subspace Segmetatio by Low-Ra Represetatio, I: Proceedigs of the Iteratioal Coferece o Machie Learig (ICML),. [3] E. Cades, X.D. Li, Y. Ma, ad J. Wright, Robust Pricipal Compoet Aalysis? Joural of the ACM, vol.58, o.3, pp.-37,. [4] E. Cades, ad P. Yaiv, Matrix completio with oise, I: Proceedigs of the IEEE, vol.97, o.6, pp., , 9. [5] E. Cadµes, ad R. Beami, Exact matrix completio via covex optimizatio, Foudatios of Computatioal Mathematics, vol.9, o.6, pp.77-77, 9. [6] B.K. Bao, G.C. Liu, C.S. Xu, ad S.C. Ya, Iductive Robust Pricipal Compoet Aalysis, IEEE Tras. o Image Processig (TIP), vol., o.8, pp ,. [7] Z. Zhag, S. C. Ya, ad M. B. Zhao, Robust Image Represetatio ad Decompositio by Laplacia Regularized Latet Low-Ra Represetatio, I: Proceedigs of the IEEE Iteratioal Joit Coferece o eural etwors (IJC), Dallas, Texas, 3. [8]Z.C. Li, A. Gaesh, J. Wright, L.Q. Wu, M.M. Che, ad Y Ma, Fast Covex Optimizatio Algorithms for Exact Recovery of a Corrupted Low-Ra Matrix, UIUC Techical Report UILU-EG- 9-4, 9. [9] Z. Li, M. Che, L. Wu, ad Y. Ma, The augmeted Lagrage multiplier method for exact recovery of corrupted low-ra matrices, Techical report, UILU-EG-9-5, 9. [] Z. Zhag, C. L. Liu, ad M. B. Zhao, adwritig Represetatio ad Recogitio through a Sparse Proectio ad Low-Ra Recovery Framewor, I: Proceedigs of the IEEE Iteratioal Joit Coferece o eural etwors (IJC), Dallas, Texas, 3. [] G.C. Liu, S.C. Ya, Latet Low-Ra Represetatio for subspace segmetatio ad feature extractio, I: Proceedigs of the Iteratioal Coferece o Computer Visio (ICCV),. [] Z. Zhag, T. W. S. Chow, Tesor Locally Liear Discrimiative Aalysis, IEEE Sigal Processig Letter, 8(): ,. [3] J. Wright, A. Yag, S. Sastry, ad Y. Ma, Robust face recogitio via sparse represetatio, IEEE Tras. O Patter Aalysis ad Machie Itelligece, vol.3, o., pp.-7, 9. [4] Z. Zhag, M.B. Zhao, ad T. W. S. Chow, Biary- ad Multi-Class Group Sparse Caoical Correlatio Aalysis for Feature Extractio ad Classificatio, IEEE Tras. Kowledge ad Data Egieerig, vol.5, o., pp.9-5, 3. [5] A. Gaesh, J. Wright, X.D. Li, E. Cades, ad Y. Ma, Dese Error Correctio for Low-Ra Matrices via Pricipal Compoet Pursuit, I: Proceedigs of the Iteratioal Symposium o Iformatio Theory,. [6]. Xu, C. Caramais, ad S. Saghavi, Robust PCA via outlier pursuit, I: Advaces i eural Iformatio Processig Systems (IPS),. [7] F. Samaria, ad A. arter, Parameterisatio of a Stochastic Model for uma Face Idetificatio, I: Proceedigs of the d IEEE Worshop o Applicatios of Computer Visio, Sarasota FL, 994. [8] Z. Li, R. Liu, ad Z. Su, Liearized alteratig directio method with adaptive pealty for low-ra represetatio, I: Advaces i eural Iformatio Processig Systems (IPS), pp.6-6,. [9] A. Del Bue, J. Xavier, L. Agapito, M. Paladii, "Biliear Modelig via Augmeted Lagrage Multipliers (BALM)," IEEE Tras. o Patter Aalysis ad Machie Itelligece, vol. 34, o. 8, pp ,. [] Z. Zhag, S.C. Ya, ad M.B. Zhao, Pairwise Sparsity Preservig Embeddig for Usupervised Subspace Learig ad Classificatio, IEEE Tras. o Image Processig, vol., iss., pp , 3. [] L. S. Qiao, S. C. Che, X. Y. Ta, Sparsity Preservig Proectios with Applicatios to Face Recogitio, Patter Recogitio, vol.43, o., pp.33-34,. [] A. M. Martiez, A. C. Ka, PCA versus LDA, IEEE Tras. o Patter Aalysis ad Machie Itelligece, vol.3, o., pp.8-33,. [3] S. M. Wei, ad Z. C. Li, Aalysis ad Improvemet of Low Ra Represetatio for Subspace segmetatio, CoRR abs/7.56,. [4] Y. Zhag, Recet advaces i alteratig directio methods: Practice ad theory, Tutorial,. [5] A. Bec, ad M. Teboulle, A fast iterative shriage-thresholdig algorithm for liear iverse problems, SIAM Joural o Imagig Scieces, vol., pp. 83-, 9. [6] Y. G. Peg, A. Gaesh, J. Wright, ad Y. Ma, RASL: Robust aligmet by sparse ad low-ra decompositio for liearly correlated images, I: Proceedigs of IEEE Coferece o Computer Visio ad Patter Recogitio (CVPR),. [7] P. Favaro, R. Vidal, ad A. Ravichadra, A Closed Form Solutio to Robust Subspace Estimatio ad Clusterig, I: Proceedigs of the IEEE Coferece o Computer Visio ad Patter Recogitio (CVPR), pp. 8 87,. [8] X. F. e, D. Cai, ad P. iyogi, Tesor Subspace Aalysis, I: Advaces i eural Iformatio Processig Systems 8, Vacouver, Caada, 5. [9] Y. i, J. Su, X. Yua, S. Ya, L. Cheog, Robust Low-Ra Subspace Segmetatio with Semidefiite Guaratees, I: Proceedigs of the IEEE Iteratioal Coferece o Data Miig, pp.79-88,. [3] R. S. Liu, Z. C. Li, F. De la Torre, ad Z.X. Su, Fixed-Ra Represetatio for Usupervised Visual Learig, CoRR abs/3.,. [3] L. S. Zhuag,. Y. Gao, Z. C. Li, Y. Ma, X. Zhag, ad.. Yu, o-egative low ra ad sparse graph for semi-supervised learig, I: Proceedigs of the IEEE Coferece o Computer Visio ad Patter Recogitio (CVPR), pp ,. [3] G. C. Liu, ad S. C. Ya, Active subspace: toward scalable lowra learig, eural Computatio, vol.4, o., pp ,. [33] Y. Mu, J. Dog, X. Yua, ad S. C. Ya, Accelerated Low-Ra Visual Recoveryby Radom Proectio, I: Proceedigs of the IEEE Coferece o Computer Visio ad Patter Recogitio (CVPR), pp.69-66,. [34] T. Zhou, ad D. Tao, GoDec: Radomized Low-Ra & Sparse Matrix Decompositio i oisy Case, I: Proceedigs of the Iteratioal Coferece o Machie Learig, pp.33-4,. [35] C. Che, B. e, ad X. Yua, Matrix completio via a alteratig directio method, IMA Joural of umerical Aalysis, vol.3, o., pp.7-45,. [36] Y. Y. Liu, L. C. Jiao, F.. Shag, F. Yi, ad F. Liu, A efficiet matrix bi-factorizatio alterative optimizatio method for low-ra matrix recovery ad completio, eural etwors, vol.48, pp.8-8, 3. [37] Y. Y. Liu, L. C. Jiao, ad F.. Shag, A fast tri-factorizatio method for low-ra matrix recovery ad completio, Patter Recogitio, vol.46, o., pp.63-73, 3. [38] Y. Y. Liu, L. C. Jiao, ad F.. Shag, A efficiet matrix factorizatio based low-ra represetatio for subspace clusterig Patter Recogitio, vol.46, o., pp.84-9, 3. [39] X. Re, ad Z. C. Li, Liearized Alteratig Directio Method with Adaptive Pealty ad Warm Starts for Fast Solvig Trasform Ivariat Low-Ra Textures, Iteratioal Joural of Computer Visio, vol.4, o., pp.-4, 3.