New l 1 -Norm Relaxations and Optimizations for Graph Clustering

Transcription

1 Proceedngs of the Thrteth AAAI Conference on Artfcal Intellgence (AAAI-6) New l -Norm Relaxatons and Optmzatons for Graph Clusterng Fepng Ne, Hua Wang, Cheng Deng 3, Xnbo Gao 3, Xuelong L 4, Heng Huang Department of Computer Scence and Engneerng, Unversty of Texas at Arlngton, USA Department of Electrcal Engneerng and Computer Scence, Colorado School of Mnes, USA 3 School of Electronc Engneerng, Xdan Unversty, X an, Chna 4 X an Insttute of Optcs and Precson Mechancs, Chnese Academy of Scences, Chna fepngne@gmal.com, huawang@mnes.edu, {chdeng,xbgao}@mal.xdan.edu.cn, xuelong l@opt.ac.cn, heng@uta.edu Abstract In recent data mnng research, the graph clusterng methods, such as normalzed cut and rato cut, have been well studed and appled to solve many unsupervsed learnng applcatons. The orgnal graph clusterng methods are NP-hard problems. Tradtonal approaches used spectral relaxaton to solve the graph clusterng problems. The man dsadvantage of these approaches s that the obtaned spectral solutons could severely devate from the true soluton. To solve ths problem, n ths paper, we propose a new relaxaton mechansm for graph clusterng methods. Instead of mnmzng the squared dstances of clusterng results, we use the l -norm dstance. More mportant, consderng the normalzed consstency, we also use the l - norm for the normalzed terms n the new graph clusterng relaxatons. Due to the sparse result from the l -norm mnmzaton, the solutons of our new relaxed graph clusterng methods get dscrete values wth many zeros, whch are close to the deal solutons. Our new objectves are dffcult to be optmzed, because the mnmzaton problem nvolves the rato of nonsmooth terms. The exstng sparse learnng optmzaton algorthms cannot be appled to solve ths problem. In ths paper, we propose a new optmzaton algorthm to solve ths dffcult non-smooth rato mnmzaton problem. The extensve experments have been performed on three two-way clusterng and eght mult-way clusterng benchmark data sets. All emprcal results show that our new relaxaton methods consstently enhance the normalzed cut and rato cut clusterng results. Introducton Clusterng s an mportant task n computer vson and machne learnng research wth many applcatons, such as mage segmentaton (Sh and Malk ), mage categorzaton (Grauman and Darrell 6), scene analyss (Koppal and Narasmhan 6), moton modelng (P.Ochs and T.Brox ), and medcal mage analyss (Brun, Park, and To whom all correspondence should be addressed. Ths work was partally supported by the followng grants: NSF-IIS 7965, NSF-IIS 3675, NSF-IIS 3445, NSF-DBI 35668, NSF-IIS 4359, NIH R AG4937. Copyrght c 6, Assocaton for the Advancement of Artfcal Intellgence ( All rghts reserved. Shenton 4). In the past decades, many clusterng algorthms have been proposed. Among these approaches, the use of manfold nformaton n graph clusterng has shown the state-of-the-art clusterng performance. The graph based clusterng methods model the data as a weghted undrected graph based on the par-wse smlartes. Clusterng s then accomplshed by fndng the best cuts of the graph that optmze the predefned cost functons. Two types of graph clusterng methods, normalzed cut (Sh and Malk ) and rato cut (Cheng and We 99; Hagen and Kahng 99), are popularly used to solve the clusterng problems due to ther good clusterng performance. Solvng the graph clusterng problem s a dffcult task (NP-hard problems). The man dffculty of the graph clusterng problem comes from the constrans on the soluton. It s hard to solve the graph clusterng problems exactly. However, the approxmaton solutons are possble wth spectral relaxatons. The optmzaton usually leads to the computaton of the top egenvectors of certan graph affnty matrces, and the clusterng result can be derved from the obtaned egen-space. However, the tradtonal spectral relaxatons lead the non-optmal clusterng results. The spectral solutons don t drectly provde the clusterng results and the thresholdng post-processng has to be appled, such that the results often severely devate from the true soluton. More recently, tght relaxatons of balanced graph clusterng methods were proposed (Bühler and Hen 9; Luo et al. ; Hen and Setzer ), and gradent based method was used to solve the problem, whch s tme consumng and slow to converge n practce. In order to solve the above challengng ssues, n ths paper, we revst the normalzed cut and rato cut methods, and propose new relaxatons for these methods to acheve the dscrete and sparse clusterng results whch are close to the deal solutons. Instead of mnmzng the projected squared clusterng ndctors dstance, we mnmze the l dstance. Meanwhle, our new relaxatons also use the l -norm for the normalzaton terms. Due to the l -norm mnmzaton, most elements of each clusterng ndctor are enforced to be zero and hence the clusterng results are close the deal solutons. However, our new relatons ntroduce a dffcult optmzaton problem whch optmzes the rato of two nonsmooth terms. The standard optmzaton methods for sparse learnng, such as Proxmal Gradent, Iteratve Shrnkage- 96

2 Thresholdng, Gradent Projecton, Homotopy, and Augmented Lagrange Multpler methods, cannot be utlzed to solve such an l -norm rato mnmzaton problem. We propose a new optmzaton algorthm to solve ths dffcult problem wth theoretcally proved convergence, and our algorthm usually converges wthn teratons. The extensve clusterng experments are performed on three two-way clusterng data sets and eght mult-way clusterng data sets to evaluate our new relaxed normalzed cut and rato cut methods. All emprcal results demonstrate our new relaxatons consstently acheve better clusterng results than the tradtonal relaxatons. Graph Clusterng Revst Gven a graph G =(V,E) and the assocated weght matrx W, we partton t nto two dsjont sets A and B, A B = V, A B =, Two types of graph clusterng methods, normalzed cut (Sh and Malk ) and rato cut (Cheng and We 99; Hagen and Kahng 99), are usually appled to measure the qualty of the partton. The man task s to mnmze the defned graph cut to obtan a satsfed partton. Normalzed Cut and Relaxaton The normalzed cut (Sh and Malk ) s defned as cut(a, B) cut(a, B) Ncut = + assoc(a, V ) assoc(b,v ), () where cut(a, B) = A,j B W j and assoc(a, V ) = A,j V W j Denote a vector y R n as follows y =[,...,, r,...r] T. () }{{}}{{} n n Denote d = A D, d = B D, (Sh and Malk ) proved that when r = d d, the normalzed cut defned n Eq. () can be wrtten as Ncut = W j (y y j ),j D y = yt Ly y T Dy, (3) where L = D W s the Laplacan matrx, D s the dagonal matrx wth the -th dagonal element as D = j W j. Prevous paper (Sh and Malk ) provded proof, but here we provde a much more concse proof as follows. Let c = A,j B W j, then we have W j (y y j ),j D y = ( r) c d + r. (4) d On the other hand, accordng to Eq. (), we have Ncut = c + c. (5) d d Combnng the above equatons, we have: ( r) c d + r = c + c r + r d d d d + r = d + d d d d (d + rd ) = r = d d, whch completes the proof. In order to mnmze the normalzed cut to obtan a satsfed partton, we need to solve the followng problem: mn y=[,...,, d d,..., d d ] T W j (y y j ),j D y Due to the constrant on y, the problem s NP-hard. In order to solve ths problem, usually we need to relax the constrant. The constrant n Eq. (6) ndcates that T Dy =, thus the problem can be relaxed by usng the constrant T Dy = to replace the constrant n Eq. (6). The relaxed problem s as follows: mn T Dy= W j (y y j ),j D y The optmal soluton to the relaxed problem s the egenvector of D L correspondng to the second smallest egenvalue. However, ths relaxaton makes the soluton y devate from the constrant n Eq. (6) so much. The egenvector of D L usually take on contnuous values whle the real soluton of y should only take on two dscrete values. As suggested n (Sh and Malk ), One can take or the medan value as the splttng pont or one can search for the splttng pont such that the resultng partton has the best normalzed cut value. Rato Cut and Relaxaton The rato cut (Cheng and We 99; Hagen and Kahng 99) s defned as Rcut = cut(a, B) A + (6) (7) cut(a, B), (8) B where A denotes the number of ponts n A. Smlarly, t can be easly proved that when r = n n n Eq. (), the rato cut defned n Eq. (8) can be wrtten as W j (y y j ),j Rcut = = yt Ly y T y. (9) y In order to mnmze the normalzed cut to obtan a satsfed partton, we solve the followng problem W j (y y j ) mn y=[,...,, n n,..., n n ] T,j y () Due to the constrant on y, t was also proved that ths problem s NP-hard. The constrant n Eq. () ndcates that T y =, thus the problem can be relaxed by usng the constrant T y =to replace the constrant n Eq. (). The relaxed problem s as follows: mn T y= W j (y y j ),j y () 963

3 The optmal soluton to the relaxed problem s the egenvector of L correspondng to the second smallest egenvalue. The relaxaton also makes the soluton y devate from the constrant n Eq. (), and the fnal partton can be obtaned by the same strateges as n the case of normalzed cut. New Graph Clusterng Relaxatons and Optmzaton Algorthms As dscussed n the above secton, the tradtonal graph clusterng relaxatons make the soluton y devate from the deal soluton. In ths secton, we wll propose the new relaxatons for normalzed cut and rato cut, to whch the solutons are dscrete and close to the deal ones. We wll also provde new optmzaton algorthms to solve the proposed problems. New Relaxaton of Normalzed Cut Frst, we have the followng theorem for normalzed cut: Theorem Denote y = [,...,, d d,..., d d ] T, then W j y y j,j D y = Ncut Proof: As before, denote c = A,j B W j, then we have W j y y j d,j ( + d = )c = (d + d )c D y d d d = ( c + c )= d d Ncut, whch completes the proof. Based on Theorem, the problem (6) s equvalent to the followng problem wth the same constrant but dfferent objectve functon: W j y y j,j mn () y=[,...,, d d,..., d d ] D T y Accordngly, we can relax the problem as the followng one: W j y y j,j mn (3) T Dy= D y Note that problem (3) mnmzes a l -norm, whch usually results n sparse soluton (Ne et al. b). That s to say, y y j =for many (, j)-pars, whch ndcates the soluton y wll take on dscrete values. Therefore, the soluton to the relaxed problem (3) s close to the deal soluton. New Relaxaton of Rato Cut Smlarly, we have the followng theorem for rato cut: Theorem Denote y = [,...,, n n,..., n n ] T, then W j y y j,j y = Rcut Proof: As the above proof, denote c = A,j B W j, then we have: W j y y j n,j ( + n = )c = (n + n )c y n + n n n n n = ( c + c )= n n Rcut, whch completes the proof. Based on Theorem, the problem () s equvalent to the followng problem wth the same constrant but dfferent objectve functon: W j y y j,j mn (4) y=[,...,, n n,..., n n ] T y Accordngly, we can relax the problem as the followng one: W j y y j,j mn (5) T y= y Smlarly, the relaxed problem (5) wll result n sparse soluton,.e., y y j =for many (, j)-pars. Therefore, the soluton to the relaxed problem (5) s a good approxmaton to the deal soluton. Relaton to Cheeger cut In spectral graph theory (Chung 997), the Cheeger cut s defned as cut(a, B) Ccut = (6) mn{ A, B } As ponted by (Chung 997; Hen and Buhler ), the optmal Cheeger cut s the same as the value obtaned by optmal thresholdng the optmal soluton to the followng problem: mn y,medan(y)= W j y y j,j. (7) y Comparng Eq. (7) and Eq. (4), t s nterestng to see that the optmal Cheeger cut and the optmal rato cut can be obtaned wth the same objectve functon but under dfferent constrants. Note that the feasble soluton y to problem (7) can be contnuous values accordng to the constrant n Eq. (7), thus one can reasonably conjecture that the value obtaned by optmal thresholdng of the optmal soluton to problem (5) s close to the optmal rato cut n Eq. (4). Algorthms to Solve New Relaxaton Problems Our new relaxed graph clusterng methods ntroduce a dffcult optmzaton problem,.e. mnmze the rato of nonsmooth terms. The standard optmzaton methods for sparse learnng, such as Proxmal Gradent, Iteratve Shrnkage- Thresholdng, Gradent Projecton, Homotopy, and Augmented Lagrange Multpler methods, cannot be utlzed to 964

4 solve such l -norm rato mnmzaton problem. In ths secton, we wll propose a new optmzaton algorthm to solve ths challengng optmzaton problem. We frst ntroduce the soluton to a general problem, and then provde the solutons to problems n Eqs. (3) and (5), respectvely. A General Framework Before solvng the new relaxatons of graph clusterng methods, we solve the followng general problem frst: f (x) mn x C g (x). (8) Motvated by (Ne et al. 9; ; a; Ne, Yuan, and Huang 4), we gve an algorthm to solve ths problem, whch s very easy to mplement. The detaled algorthm s descrbed n Algorthm. In the followng, we wll prove that the algorthm wll monotoncally decrease the objectve value of problem (8) untl converges. Algorthm Algorthm to solve the general problem (8). Intalze x C whle not converge do. Calculate the objectve value λ = f (x) g (x). For each, calculate s = f and b (x) = sgn(g (x)). Update x by arg mn s f (x) λ b g (x) x C end whle Theorem 3 The procedure of Algorthm wll monotoncally decrease the objectve value of problem (8) untl converges. Proof: Denote the updated x by x. Accordng to step, s f ( x) λ b g ( x) s f (x) λ b g (x) Notce the defntons of s and b n step, we have f ( x) f (x) λ sgn(g (x))g ( x) f (x) λ g (x) It can be checked that the followng two nequaltes hold: ( f ( x) f ( x) ) f (x) (9) f (x) (sgn(g (x))g ( x) g ( x) ) () Addng the above three nequaltes n Eqs. (9-), we have f ( x) λ g ( x), whch ndcates f ( x) f (x) g ( x) λ = () g (x) Therefore, the algorthm wll monotoncally decrease the objectve value untl converges. Algorthm Algorthm to solve the problem (3). Intalze y such that T Dy = whle not converge do. Calculate λ = W j y y j,j D y ; the matrx S, where the (, j)-th element s S j = y y j ; and the vector b, where the -th element s b = sgn(d y ). Update y by y =arg mn T Dy= y T ˆLy λb T y, where ˆL = ˆD Ŵ, Ŵ = W S and ˆD s a dagonal matrx wth the -th element as ˆD = j Ŵj end whle Solutons to Problem (3) and Problem (5) We can use the algorthm framework n Algorthm to solve the proposed problem (3) and (5). The detaled algorthm to solve the problem (3) s descrbed n Algorthm. The algorthm to solve the problem (5) s smlar, we omt the detaled algorthm here durng to space lmtatons. In Step of the Algorthm, we need to solve the problem mn y T ˆLy λb T y. Solvng ths problem seems tme T Dy= consumng because of the constrant n the problem. Fortunately, the problem s equvalent to the followng problem mn y T ˆLy λb T y + ηy T D T Dy wth a large enough y η. Ths problem has a closed form soluton y = λ(ˆl + ηd T D) b and can be effcently solved by usng Woodbury matrx dentty and solvng a very sparse system of lnear equatons. Extenson to Mult-Way Parttonng The Algorthm parttons the graph nto two parts, we can recursvely run the algorthms to obtan the desred number of parttons. Specfcally, when the graph s dvded nto k parts, the k +part can be obtaned by runnng the algorthms on the k parts ndvdually, and select the one that the defned cut s mnmal when ths part s dvded nto parts. Another method to perform the mult-way parttonng s as follows. After we obtan k vectors by the algorthms, the k +vector y s obtaned by runnng the algorthms wth an addtonal constrant that the vector y s orthogonal to the pervous k vectors. Recursvely run the algorthms, we can obtan the desred number of vectors, and then run K-means clusterng on the vectors to obtan the fnal parttonng of the graph as n (Ne et al. b). Expermental Results In ths secton, we expermentally evaluate the two proposed graph clusterng methods n both two-way and mult-way clusterng tasks. We abbrevate the proposed new relaxaton of the normalzed cut as NR-NC, and abbrevate the proposed new relaxaton of the rato cut as NR-RC. To evaluate the clusterng results, we adopt the two wdely used standard metrcs: clusterng accuracy and normalzed mutual nformaton (NMI) (Ca et al. 8). 965

5 Table : Performance and objectve value comparson of the proposed methods aganst ther tradtonal counterparts. Rato Cut NR-RC Normalzed Cut NR-NC Data Acc NMI Acc NMI Acc NMI Acc NMI Hepatts onosphere breast cancer Logarthmc objectve value Rato Cut NR RC (our method) 3 4 Number of teratons (a) Objectve value vs. teraton. Average precson Number of teratons (b) Clusterng accuracy of NR-RC vs. teraton. Logarthmc objectve value Normalzed Cut NR NC (our method) 3 4 Number of teratons (c) Objectve value vs. teraton. Average precson Number of teratons (d) Clusterng accuracy NR-NC vs. teraton. Fgure : Convergence analyss of -way clusterng on hepatts data set. Logarthmc objectve value Rato Cut NR RC (our method) 3 4 Number of teratons (a) Objectve value vs. teraton. Average precson Number of teratons (b) Clusterng accuracy of NR-RC vs. teraton. Logarthmc objectve value Normalzed Cut NR NC (our method) 3 4 Number of teratons (c) Objectve value vs. teraton. Average precson Number of teratons (d) Clusterng accuracy NR-NC vs. teraton. Fgure : Convergence analyss of -way clusterng on onosphere data set. Logarthmc objectve value Rato Cut NR RC (our method) 3 4 Number of teratons (a) Objectve value vs. teraton. Average precson Number of teratons (b) Clusterng accuracy of NR-RC vs. teraton. Logarthmc objectve value Normalzed Cut NR NC (our method) 3 4 Number of teratons (c) Objectve value vs. teraton. Average precson Number of teratons (d) Clusterng accuracy NR-NC vs. teraton. Fgure 3: Convergence analyss of -way clusterng on breast cancer data set. Two-Way Clusterng Usng NR-RC and NR-NC Methods We frst evaluate the two proposed methods n two-way clusterng, and compare them aganst ther respectve tradtonal counterparts. Three benchmark data sets from UCI machne learnng repostory are used n our experments, ncludng hepatts database wth 55 nstances and attrbutes, onosphere database wth 35 nstances and 34 attrbutes, breast cancer database wth 86 nstances and 9 attrbutes. All these three data sets have only classes, therefore we can perform two-way clusterng on them. We construct nearestneghbor graph for each data set followng (Gu and Zhou 9). The clusterng results by the compared results are shown n Table, from whch we can see that the proposed new re- laxaton graph clusterng methods consstently outperforms ther tradtonal counterparts, sometmes very sgnfcantly. These results clearly demonstrate the advantage of the proposed methods n terms of clusterng performance. Because our methods employ teratve algorthms, we nvestgate the convergence propertes of our algorthms wth some detals. Gven the output vertex rankng from each teraton of the algorthms, we compute the objectve value by Eq. (8) for the NR-RC method and by Eq. () for the NR- NC method, whch are plotted n Fgure (a) and Fgure (c) for hepatts data, Fgure (a) and Fgure (c) for onosphere data, Fgure 3(a) and Fgure 3(c) for breast cancer data, respectvely. The clusterng accuracy wth respect each teraton of the two proposed methods are also plotted n Fgure (b) and Fgure (d) for hepatts data, Fgure (b) and Fgure (d) for onosphere data, Fgure 3(b) and Fgure 3(d) 966

6 Table : Clusterng accuracy (%) comparson of mult-way clusterng on the eght data sets. DATA SET KM PCA+KM LDA-KM RC NR-RC NC NR-NC DERMATOL ECOLI COIL BINALPHA UMIST AR YALEB PIE Table 3: NMI (%) comparson of mult-way clusterng on the eght data sets. DATA SET KM PCA+KM LDA-KM RC NR-RC NC NR-NC DERMATOL ECOLI COIL BINALPHA UMIST AR YALEB PIE for breast cancer data, respectvely. From these fgures we can see that our algorthms converge very fast wth typcally no more than teratons, whch concretely confrm ther computatonal effcency. Moreover, as shown n Fgure 3(a) and Fgure 3(c), n contrast to the objectve values of the tradtonal graph clusterng methods, the objectve values at convergence of our new relaxed graph clusterng methods are much smaller, whch provde another evdence to support the correctness of both our objectves and algorthms. Mult-Way Clusterng Usng NR-RC and NR-NC Methods Now we evaluate the proposed methods n mult-way clusterng. In our experments, we mplement our methods usng the second strategy ntroduced n Secton. Eght benchmark data sets are used n the experments, ncludng two UCI data sets, dermatology and ecol, one object data set, COIL- (Nene, Nayar, and Murase 996), one dgt and character data sets, Bnalpha, and four face data sets, Umst (Graham and Allnson 998), AR (Martnez and Benavente 998), YaleB (Georghades, Belhumeur, and Kregman ), and PIE (Sm and Baker 3). Besde comparng our methods to ther tradtonal counterparts, we also compare to K-means (denoted by Km), PCA+K-means (denoted by PCA+Km), LDA-Km (Dng and L 7) methods. Agan, we construct nearest-neghbor graph for each data set and set the neghborhood sze for graph constructon as (Gu and Zhou 9). The dmenson of PCA+K-means s searched from fve canddates rangng from to the dmenson of data. The results of all clusterng algorthms depend on the ntalzaton. To reduce statstcal varety, we ndependently repeat all clusterng algorthms for 5 tmes wth random ntalzatons, and then we report the results correspondng to the best objectve values. The clusterng performance measured by clusterng accuracy and NMI are reported n Table and Table 3, from whch we can see that the proposed methods stll perform the best among all compared methods. In addton, our methods are always better ther respectve tradtonal counterparts. These advantages valdate the effectveness of the proposed methods and justfy our motvatons. Conclusons In ths paper, we proposed new relaxatons for normalzed cut and rato cut methods. The l -norm dstances are utlzed n the relaxed graph clusterng formulatons. Such l -norm based relaxatons can naturally get the dscrete and sparse clusterng solutons (wth many zeros) whch are close to the optmal ones. Moreover, we proposed a new optmzaton algorthm to address the mnmzaton problem of a rato of non-smooth terms whch cannot be solved by other standard sparse learnng optmzaton algorthms. The valdatons were performed on both two-way and mult-way clusterng problems. On all eleven benchmark data sets, our new relaxed normalzed cut and rato cut methods consstently outperform the tradtonal ones. 967

7 References Brun, A.; Park, H.-J.; and Shenton, M. E. 4. Clusterng fber traces usng normalzed cuts. Medcal Image Computng and Computer-Asssted Interventon (MICCAI) Bühler, T., and Hen, M. 9. Spectral clusterng based on the graph p-laplacan. In Proceedngs of the 6th Annual Internatonal Conference on Machne Learnng, Ca, D.; He, X.; Wu, X.; and Han, J. 8. Non-negatve matrx factorzaton on manfold. In ICDM. Cheng, C.-K., and We, Y.-C. A. 99. An mproved twoway parttonng algorthm wth stable performance. IEEE Transactons on Computer-Aded Desgn of Integrated Crcuts and Systems ():5 5. Chung, F. R. K Spectral Graph Theory. CBMS Regonal Conference Seres n Mathematcs, No. 9, Amercan Mathematcal Socety. Dng, C. H. Q., and L, T. 7. Adaptve dmenson reducton usng dscrmnant analyss and -means clusterng. In ICML, Georghades, A.; Belhumeur, P.; and Kregman, D.. From few to many: Illumnaton cone models for face recognton under varable lghtng and pose. IEEE Transactons on PAMI 3(6): Graham, D. B., and Allnson, N. M Characterzng vrtual egensgnatures for general purpose face recognton. n face recognton: From theory to applcatons. NATO ASI Seres F, Computer and Systems Scences 63: Grauman, K., and Darrell, T. 6. Unsupervsed learnng of categores from sets of partally matchng mage features. In CVPR (), 9 5. Gu, Q., and Zhou, J. 9. Co-clusterng on manfolds. In SIGKDD. Hagen, L. W., and Kahng, A. B. 99. New spectral methods for rato cut parttonng and clusterng. IEEE Transactons on Computer-Aded Desgn of Integrated Crcuts and Systems (9): Hen, M., and Buhler, T.. An nverse power method for nonlnear egenproblems wth applcatons n -spectral clusterng and sparse PCA. In NIPS. Hen, M., and Setzer, S.. Beyond spectral clusterng - tght relaxatons of balanced graph cuts. In NIPS, Koppal, S., and Narasmhan, S. 6. Clusterng Appearance for Scene Analyss. CVPR. Luo, D.; Huang, H.; Dng, C. H. Q.; and Ne, F.. On the egenvectors of p-laplacan. Machne Learnng 8():37 5. Martnez, A. M., and Benavente, R The ar face database. In CVC Techncal Report. Nene, S. A.; Nayar, S. K.; and Murase, H Columba object mage lbrary (COIL-), Techncal Report CUCS Ne, F.; Xang, S.; Ja, Y.; and Zhang, C. 9. Semsupervsed orthogonal dscrmnant analyss va label propagaton. Pattern Recognton 4(): Ne, F.; Huang, H.; Ca, X.; and Dng, C.. Effcent and robust feature selecton va jont l, -norms mnmzaton. In NIPS. Ne, F.; Huang, H.; Dng, C.; Luo, D.; and Wang, H. a. Robust prncpal component analyss wth non-greedy l- norm maxmzaton. In IJCAI Proceedngs-Internatonal Jont Conference on Artfcal Intellgence, volume, 433. Ne, F.; Wang, H.; Huang, H.; and Dng, C. b. Unsupervsed and sem-supervsed learnng va l-norm graph. In IEEE Internatonal Conference on Computer Vson (ICCV), Ne, F.; Yuan, J.; and Huang, H. 4. Optmal mean robust prncpal component analyss. In Proceedngs of the 3st Internatonal Conference on Machne Learnng (ICML), 6 7. P.Ochs, and T.Brox.. Hgher order moton models and spectral clusterng. In IEEE Internatonal Conference on Computer Vson and Pattern Recognton (CVPR). Sh, J., and Malk, J.. Normalzed cuts and mage segmentaton. IEEE Transactons on PAMI (8): Sm, T., and Baker, S. 3. The cmu pose, llumnaton, and expresson database. IEEE Transactons on PAMI 5():