Low Rank Representation Theories, Algorithms, Applications. 林宙辰 北京大学 May 3, 2012
|
|
- Sharlene Parks
- 5 years ago
- Views:
Transcription
1 Low Rank Representation Theories, Algorithms, Applications 林宙辰 北京大学 May 3, 2012
2 Outline Low Rank Representation Some Theoretical Analysis Solution by LADM Applications Generalizations Conclusions
3 Sparse Representation Sparse Representation m in jjxjj 0 ; s:t: y = A x: (1) Sparse Subspace Clustering m in jjz i jj 0 ; s:t: x i = X^iz i ; 8i: (2) where X^i = [x 1 ; ; x i 1 ; x i+1 ; ; x n ]. m in jjz jj 0 ; s:t: X = X Z ; diag(z ) = 0: (3) m in jjz jj 1 ; s:t: X = X Z ; diag(z ) = 0: E. Elhamifar and R. Vidal. Sparse Subspace Clustering. CVPR2009. (4)
4 Sparse Representation Construct a graph W = (jz j + j(z ) T j)=2 Normalized cut on the graph E. Elhamifar and R. Vidal. Sparse Subspace Clustering. CVPR2009.
5 Sparse Representation Theorem. Assum e the data is clean and is drawn from independent subspaces, then Z is block diagonal. dim ( P i S i ) = P i dim (S i ): E. Elhamifar and R. Vidal. Sparse Subspace Clustering. CVPR2009.
6 Drawback of SSC Sensitive to noise: no cross validation among coefficients m in jjz jj 1 ; s:t: X = X Z ; diag(z ) = 0: (4) m in jjz i jj 1 ; s:t: x i = X z i ; (z i ) i = 0: (5) E. Elhamifar and R. Vidal. Sparse Subspace Clustering. CVPR2009.
7 Hints from 2D Sparsity Rank is a good measure of 2D sparsity Real data usually lie on low-dim manifolds > 1B dim low-dim subspaces low rank data matrices Low rank high correlation among rows/columns
8 Low Rank Representation m in jjz jj 1 ; s:t: X = X Z ; diag(z ) = 0: (4) m in jjz jj ; s:t: X = X Z : no additional constraint! (6) jjz jj = P j ¾ j (Z ), nuclear norm, a convex surrogate of rank. Liu et al. Robust Subspace Segmentation by Low-Rank Representation, ICML 2010.
9 Low Rank Representation T heorem. A ssum e the data is clean and is draw n from indep endent subspaces, then there exists Z w hich is blo ck diagonal, and the rank of each blo ck equals the dim ension of the corresp onding subspace. Liu et al. Robust Subspace Segmentation by Low-Rank Representation, ICML 2010.
10 Low Rank Representation When there is noise and outliers m in jjz jj + jje jj 2;1 ; s:t: X = X Z + E : (7) where jje jj 2;1 = P i jje :;i jj 2. Liu et al. Robust Subspace Segmentation by Low-Rank Representation, ICML 2010.
11 Low Rank Representation Connection to Robust PCA m in ka k + jje jj 1 ; A ;E s:t: X = A + E ; (8) The clean data is low rank w.r.t. the dictionary I. m in jjz jj + jje jj 2;1 ; s:t: X = X Z + E : (7) The clean data is low rank w.r.t. the dictionary X. Candes, Li, Ma, and Wright, Robust Principal Component Analysis? Journal of the ACM, 2010.
12 Low Rank Representation Generalization m in jjz jj + jje jj 2;1 ; s:t: X = A Z + E : (9) The clean data is low rank w.r.t. the dictionary A. Liu et al. Robust Subspace Segmentation by Low-Rank Representation, ICML 2010.
13 More Theoretical Analysis Closed form solution at noiseless case m in kz k ; Z s:t: X = X Z ; has a unique closed-form optim al solution: Z = V r V T r skinny SV D of X. Shape Interaction Matrix, w here U r r V T r when X is sampled from independent subspaces, Z* is block diagonal, each block corresponding to a subspace is th e min kzk = rank(x): X=XZ S. Wei and Z. Lin. Analysis and Improvement of Low Rank Representation for Subspace segmentation, arxiv: , Liu et al. Robust Recovery of Subspace Structures by Low-Rank Representation, TPAMI 2012.
14 More Theoretical Analysis Closed form solution at general case m in Z kz k ; s:t: X = A Z ; has a unique closed-form optim al solution: Z = A y X. Liu et al., Robust Recovery of Subspace Structures by Low-Rank Representation, TPAMI 2012.
15 Follow-up Work More problems with closed form solutions min Z Speeding up optimization "kzk kxz Xk2 F min A;Z "kzk + 2 kaz Ak2 F kd Ak2 F min "kzk + 1 A;Z 2 ka Xk2 F ; s:t: A = AZ: P. Favaro, R. Vidal and A. Ravichandran, A Closed Form Solution to Robust Subspace Estimation and Clustering, CVPR 2011.
16 Exact Recoverability of LRR T heorem : L et = 3=(7kX k p n). T hen there exists > 0 such that w hen < =, LR R can exactly recover the row space and the colum n supp ort of (Z 0 ; E 0 ): U (U ) T = V 0 V T 0 ; I = I 0 ; w here = ji 0 j=n is the fraction of outliers, U is the colum n space of Z, V T 0 is the row space of Z 0, and I and I 0 is the colum n supp orts of E and E 0, resp ectively. Liu et al., Exact Subspace Segmentation and Outlier Detection by Low-Rank Representation, AISTATS Liu et al., Robust Recovery of Subspace Structures by Low-Rank Representation, TPAMI 2012.
17 Linearized Alternating Direction Model Problem ADM Difficulties min x;y Method (LADM) f(x) + g(y); s:t: A(x) + B(y) = c; where x, y and c could be either vectors or matrices, f and g are convex functions, and A and B are linear mappings. L A (x; y; ) = f(x) + g(y) + h ; A(x) + B(y) yi + 2 ka(x) + B(y) ck2 ; x k+1 = arg min x y k+1 = arg min y L A (x; y k ; k); L A (x k+1 ; y; k); k+1 = k + [A(x k+1 ) + B(y k+1 ) c]: Lin et al., Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation, NIPS 2011.
18 Linearized Alternating Direction Method (LADM) x k+1 = arg min f(x) + x 2 ka(x) + B(y k) c + k= k 2 ; y k+1 = arg min g(y) + y 2 ka(x k+1) + B(y) c + k= k 2 Linearize the quadratic term x k+1 = arg min x = arg min x y k+1 = arg min y Adaptive Penalty f(x) + ha ( k) + A (A(x k ) + B(y k ) c); x x k i + A 2 kx x k k 2 f(x) + A 2 kx x k + A ( k + (A(x k ) + B(y k ) c))=( A)k 2 ; g(y) + B 2 ky y k + B ( k + (A(x k+1 ) + B(y k ) c))=( B)k 2 :
19 Linearized Alternating Direction Method (LADM) Theorem: If f kg is non-decreasing and upper bounded, A > kak 2, and B > kbk 2, then the sequence f(x k ; y k ; k)g generated by LADMAP converges to a KKT point of the model problem. Lin et al., Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation, NIPS 2011.
20 Applying LADM to LRR LRR Further Acceleration Technique O(n 3 ) O(rn 2 ) min Z;E kzk + kek 2;1 ; s:t: X = XZ + E: Lin et al., Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation, NIPS 2011.
21 Experiments Table 1: Comparison among APG, ADM, LADM, standard LADMAP and accelerated LADMAP (denoted as LADMAP(A)) on the synthetic data. For each quadruple (s, p, d, ~r), the LRR problem, with ¹ = 0:1, was solved for the same data using di erent algorithms. We present typical running time (in 10 3 seconds), iteration number, relative error (%) of output solution ( ^E; ^Z) and the clustering accuracy (%) of tested algorithms, respectively. Size (s, p, d, ~r) Method Time Iter. (10, 20,200, 5) (15, 20,300, 5) (20, 25, 500, 5) (30, 30, 900, 5) k^z Z 0 k kz 0 k k ^E E 0 k ke 0 k Acc. APG ADM LADM LADMAP LADMAP(A) APG ADM LADM LADMAP LADMAP(A) APG ADM LADM LADMAP LADMAP(A) APG ADM LADM N.A. N.A. N.A. N.A. N.A. LADMAP LADMAP(A)
22 Applications of LRR Image segmentation min Z;E kzk + kek 2;1 ; s:t: X = XZ + E: B. Cheng et al. Multi-task Low-rank Affinity Pursuit for Image Segmentation, ICCV 2011.
23 Saliency detection Applications of LRR m in kz k + ke k 2;1 ; Z ;E s:t: X = X Z + E : Lang et al. Saliency Detection by Multitask Sparsity Pursuit. IEEE TIP 2012.
24 Generalizations of LRR LRR with clean data min D ;Z;E kz k + ke k 2;1 ; s:t: D = D Z; X = D + E : m in D ;Z ;E kz k + ke k 2 F ; s:t: D = D Z ; X = D + E : S. Wei and Z. Lin. Analysis and Improvement of Low Rank Representation for Subspace segmentation, arxiv: , P. Favaro, R. Vidal and A. Ravichandran, A Closed Form Solution to Robust Subspace Estimation and Clustering, CVPR 2011.
25 Latent LRR Generalizations of LRR To address insufficient sample problem (X = [X ; X H ]Z O ;H =) X = X Z + LX ) m in Z ;L ;E kz k + klk + jje jj 2;1 ; s:t: X = X Z + LX + E : Liu and Yan. Latent Low-Rank Representation for Subspace Segmentation and Feature Extraction, ICCV 2011.
26 Generalizations of LRR Fixed Rank Representation (FRR) To address insufficient sample problem Subspace clustering: using m in Z ; ~ Z ;E Feature extraction: by m in Z ; ~ Z ;E ~Z kz ~ Z k 2 F + jje jj 2;1 ; s:t: X = X Z + E ; rank( ~ Z ) <= r: y = ~ Z x kz ~ Z k 2 F + jje jj 2;1 ; s:t: X = Z X + E ; rank( ~ Z ) <= r: Liu et al., Fixed-Rank Representation for Unsupervised Visual Learning, CVPR 2012.
27 Generalizations of LRR Semi-supervised learning min Z;E kz k + kz k 1 + ke k 2;1 ; s:t: X = AZ + E ; Z 0: propogate labels on the graph with weights (Z + (Z ) T )=2. L. Zhuang et al. Non-Negative Low Rank and Sparse Graph for Semi-Supervised Learning, CVPR 2012.
28 Kernel LRR Generalizations of LRR To address nonlinear multi-manifold segmentation m in Z ká(x ) Á(X )Z k 2 F + jjz jj : min Z X (z T i K (X ; X )z i 2K (x i ; X )z i + K (x i ; x i )) + jjz jj : i Kernel trick: há(x); Á(y)i = K (x; y) Wang et al., Structural Similarity and Distance in Learning, Annual Allerton Conf. Communication, Control and Computing 2011.
29 Thanks! m in kz k + ke k 2;1 ; Z ;E s:t: X = X Z + E : LRR = SSC + RPCA
Robust Principal Component Analysis (RPCA)
Robust Principal Component Analysis (RPCA) & Matrix decomposition: into low-rank and sparse components Zhenfang Hu 2010.4.1 reference [1] Chandrasekharan, V., Sanghavi, S., Parillo, P., Wilsky, A.: Ranksparsity
More informationSubspace Clustering. Weiwei Feng. December 11, 2015
Subspace Clustering Weiwei Feng December 11, 2015 Abstract Data structure analysis is an important basis of machine learning and data science, which is now widely used in computational visualization problems,
More informationNULL SPACE CLUSTERING WITH APPLICATIONS TO MOTION SEGMENTATION AND FACE CLUSTERING
NULL SPACE CLUSTERING WITH APPLICATIONS TO MOTION SEGMENTATION AND FACE CLUSTERING Pan Ji, Yiran Zhong, Hongdong Li, Mathieu Salzmann, Australian National University, Canberra NICTA, Canberra {pan.ji,hongdong.li}@anu.edu.au,mathieu.salzmann@nicta.com.au
More informationLatent Low-Rank Representation
Latent Low-Rank Representation Guangcan Liu and Shuicheng Yan Abstract As mentioned at the end of previous chapter, a key aspect of LRR is about the configuration of its dictionary matrix. Usually, the
More informationFixed-Rank Representation for Unsupervised Visual Learning
Fixed-Rank Representation for Unsupervised Visual Learning Risheng Liu, Zhouchen Lin, Fernando De la Torre and Zhixun Su School of Mathematical Sciences, Dalian University of Technology Key Lab. of Machine
More informationIN pattern analysis and signal processing, an underlying
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 35, NO. 1, JANUARY 2013 171 Robust Recovery of Subspace Structures by Low-Rank Representation Guangcan Liu, Member, IEEE, Zhouchen Lin,
More informationarxiv: v1 [cs.cv] 18 Jan 2015
Correlation Adaptive Subspace Segmentation by Trace Lasso Canyi Lu, Jiashi Feng, Zhouchen Lin 2,, Shuicheng Yan Department of Electrical and Computer Engineering, National University of Singapore 2 Key
More informationA REVIEW ON LOW-RANK MODELS IN DATA ANALYSIS. Zhouchen Lin. (Communicated by XXX)
Manuscript submitted to AIMS Journals Volume X, Number 0X, XX 200X doi:10.3934/xx.xx.xx.xx pp. X XX A REVIEW ON LOW-RANK MODELS IN DATA ANALYSIS Zhouchen Lin Key Lab. of Machine Perception (MOE), School
More informationSubspace Clustering by Block Diagonal Representation
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Subspace Clustering by Block Diagonal Representation Canyi Lu, Student Member, IEEE, Jiashi Feng, Zhouchen Lin, Senior Member, IEEE, Tao Mei,
More informationCOMPLETION OF STRUCTURALLY-INCOMPLETE MATRICES WITH REWEIGHTED LOW-RANK AND SPARSITY PRIORS. Jingyu Yang, Xuemeng Yang, Xinchen Ye
COMPLETION OF STRUCTURALLY-INCOMPLETE MATRICES WITH REWEIGHTED LOW-RANK AND SPARSITY PRIORS Jingyu Yang, Xuemeng Yang, Xinchen Ye School of Electronic Information Engineering, Tianjin University Building
More informationA Geometric Analysis of Subspace Clustering with Outliers
A Geometric Analysis of Subspace Clustering with Outliers Mahdi Soltanolkotabi and Emmanuel Candés Stanford University Fundamental Tool in Data Mining : PCA Fundamental Tool in Data Mining : PCA Subspace
More informationSubspace Clustering by Block Diagonal Representation
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Subspace Clustering by Block Diagonal Representation Canyi Lu, Student Member, IEEE, Jiashi Feng, Zhouchen Lin, Senior Member, IEEE, Tao Mei,
More informationDetecting Burnscar from Hyperspectral Imagery via Sparse Representation with Low-Rank Interference
Detecting Burnscar from Hyperspectral Imagery via Sparse Representation with Low-Rank Interference Minh Dao 1, Xiang Xiang 1, Bulent Ayhan 2, Chiman Kwan 2, Trac D. Tran 1 Johns Hopkins Univeristy, 3400
More informationRobust Subspace Segmentation with Block-diagonal Prior
Robust Subspace Segmentation with Block-diagonal Prior Jiashi Feng, Zhouchen Lin 2, Huan Xu 3, Shuicheng Yan Department of ECE, National University of Singapore, Singapore 2 Key Lab. of Machine Perception,
More informationGeneralized Transitive Distance with Minimum Spanning Random Forest
Generalized Transitive Distance with Minimum Spanning Random Forest Author: Zhiding Yu and B. V. K. Vijaya Kumar, et al. Dept of Electrical and Computer Engineering Carnegie Mellon University 1 Clustering
More informationGraph Connectivity in Sparse Subspace Clustering
www.nicta.com.au From imagination to impact Graph Connectivity in Sparse Subspace Clustering Behrooz Nasihatkon 24 March 2011 Outline 1 Subspace Clustering 2 Sparse Subspace Clustering 3 Graph Connectivity
More informationSubspace Clustering for Sequential Data
Subspace Clustering for Sequential Data Stephen Tierney and Junbin Gao School of Computing and Mathematics Charles Sturt University Bathurst, NSW 2795, Australia {stierney, jbgao}@csu.edu.au Yi Guo Division
More informationLearning Low-rank Transformations: Algorithms and Applications. Qiang Qiu Guillermo Sapiro
Learning Low-rank Transformations: Algorithms and Applications Qiang Qiu Guillermo Sapiro Motivation Outline Low-rank transform - algorithms and theories Applications Subspace clustering Classification
More informationSupplementary Material : Partial Sum Minimization of Singular Values in RPCA for Low-Level Vision
Supplementary Material : Partial Sum Minimization of Singular Values in RPCA for Low-Level Vision Due to space limitation in the main paper, we present additional experimental results in this supplementary
More informationRobust Subspace Segmentation by Low-Rank Representation
Guangcan Liu roth@sjtu.edu.cn Zhouchen Lin zhoulin@microsoft.com Yong Yu yyu@apex.sjtu.edu.cn Shanghai Jiao Tong University, NO. 8, Dongchuan Road, Min Hang District, Shanghai, China, 224 Microsoft Research
More informationSparse Subspace Clustering for Incomplete Images
Sparse Subspace Clustering for Incomplete Images Xiao Wen 1, Linbo Qiao 2, Shiqian Ma 1,WeiLiu 3, and Hong Cheng 1 1 Department of SEEM, CUHK, Hong Kong. {wenx, sqma, hcheng}@se.cuhk.edu.hk 2 College of
More informationRobust Subspace Clustering via Half-Quadratic Minimization
2013 IEEE International Conference on Computer Vision Robust Subspace Clustering via Half-Quadratic Minimization Yingya Zhang, Zhenan Sun, Ran He, and Tieniu Tan Center for Research on Intelligent Perception
More informationOutlier Pursuit: Robust PCA and Collaborative Filtering
Outlier Pursuit: Robust PCA and Collaborative Filtering Huan Xu Dept. of Mechanical Engineering & Dept. of Mathematics National University of Singapore Joint w/ Constantine Caramanis, Yudong Chen, Sujay
More informationarxiv: v1 [cs.cv] 27 Apr 2014
Robust and Efficient Subspace Segmentation via Least Squares Regression Can-Yi Lu, Hai Min, Zhong-Qiu Zhao, Lin Zhu, De-Shuang Huang, and Shuicheng Yan arxiv:1404.6736v1 [cs.cv] 27 Apr 2014 Department
More informationNon-Negative Low Rank and Sparse Graph for Semi-Supervised Learning
Non-Negative Low Rank and Sparse Graph for Semi-Supervised Learning Liansheng Zhuang 1, Haoyuan Gao 1, Zhouchen Lin 2,3, Yi Ma 2, Xin Zhang 4, Nenghai Yu 1 1 MOE-Microsoft Key Lab., University of Science
More informationOnline Low-Rank Representation Learning for Joint Multi-subspace Recovery and Clustering
ACCEPTED BY IEEE TRANSACTIONS ON IMAGE PROCESSING 1 Online Low-Rank Representation Learning for Joint Multi-subspace Recovery and Clustering Bo Li, Risheng Liu, Junjie Cao, Jie Zhang, Yu-Kun Lai, Xiuping
More informationOn Order-Constrained Transitive Distance
On Order-Constrained Transitive Distance Author: Zhiding Yu and B. V. K. Vijaya Kumar, et al. Dept of Electrical and Computer Engineering Carnegie Mellon University 1 Clustering Problem Important Issues:
More informationHIGH-DIMENSIONAL data are ubiquitous in many areas of
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 35, NO. 11, NOVEMBER 2013 2765 Sparse Subspace Clustering: Algorithm, Theory, and Applications Ehsan Elhamifar, Student Member, IEEE,
More informationLearning a Manifold as an Atlas Supplementary Material
Learning a Manifold as an Atlas Supplementary Material Nikolaos Pitelis Chris Russell School of EECS, Queen Mary, University of London [nikolaos.pitelis,chrisr,lourdes]@eecs.qmul.ac.uk Lourdes Agapito
More informationLatent Space Sparse Subspace Clustering
Latent Space Sparse Subspace Clustering Vishal M. Patel, Hien Van Nguyen Center for Automation Research, UMIACS UMD, College Park, MD 20742 {pvishalm,hien}@umiacs.umd.edu René Vidal Center for Imaging
More informationComplex Non-Rigid Motion 3D Reconstruction by Union of Subspaces
Complex Non-Rigid Motion 3D Reconstruction by Union of Subspaces Yingying Zhu University of Queensland Dong Huang Carnegie Mellon University zhuyingying2@gmail.com dghuang@andrew. Fernando De La Torre
More informationHIGH-dimensional data are commonly observed in various
1 Simplex Representation for Subspace Clustering Jun Xu 1, Student Member, IEEE, Deyu Meng 2, Member, IEEE, Lei Zhang 1, Fellow, IEEE 1 Department of Computing, The Hong Kong Polytechnic University, Hong
More informationLOW-RANK representation (LRR) [30], [33], as a promising
504 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 38, NO. 3, MARCH 2016 Laplacian Regularized Low-Rank Representation and Its Applications Ming Yin, Junbin Gao, and Zhouchen Lin,
More informationOracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering
Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering Chong You Chun-Guang Li Daniel P. Robinson René Vidal Center for Imaging Science, Johns Hopkins University SICE, Beijing University
More informationELEG Compressive Sensing and Sparse Signal Representations
ELEG 867 - Compressive Sensing and Sparse Signal Representations Introduction to Matrix Completion and Robust PCA Gonzalo Garateguy Depart. of Electrical and Computer Engineering University of Delaware
More informationA new Graph constructor for Semi-supervised Discriminant Analysis via Group Sparsity
2011 Sixth International Conference on Image and Graphics A new Graph constructor for Semi-supervised Discriminant Analysis via Group Sparsity Haoyuan Gao, Liansheng Zhuang, Nenghai Yu MOE-MS Key Laboratory
More informationLarge-Scale Face Manifold Learning
Large-Scale Face Manifold Learning Sanjiv Kumar Google Research New York, NY * Joint work with A. Talwalkar, H. Rowley and M. Mohri 1 Face Manifold Learning 50 x 50 pixel faces R 2500 50 x 50 pixel random
More informationPrincipal Coordinate Clustering
Principal Coordinate Clustering Ali Sekmen, Akram Aldroubi, Ahmet Bugra Koku, Keaton Hamm Department of Computer Science, Tennessee State University Department of Mathematics, Vanderbilt University Department
More informationUnsupervised and Semi-Supervised Learning vial 1 -Norm Graph
Unsupervised and Semi-Supervised Learning vial -Norm Graph Feiping Nie, Hua Wang, Heng Huang, Chris Ding Department of Computer Science and Engineering University of Texas, Arlington, TX 769, USA {feipingnie,huawangcs}@gmail.com,
More informationIN some applications, we need to decompose a given matrix
1 Recovery of Sparse and Low Rank Components of Matrices Using Iterative Method with Adaptive Thresholding Nematollah Zarmehi, Student Member, IEEE and Farokh Marvasti, Senior Member, IEEE arxiv:1703.03722v2
More informationADAPTIVE LOW RANK AND SPARSE DECOMPOSITION OF VIDEO USING COMPRESSIVE SENSING
ADAPTIVE LOW RANK AND SPARSE DECOMPOSITION OF VIDEO USING COMPRESSIVE SENSING Fei Yang 1 Hong Jiang 2 Zuowei Shen 3 Wei Deng 4 Dimitris Metaxas 1 1 Rutgers University 2 Bell Labs 3 National University
More informationConnections between the Lasso and Support Vector Machines
Connections between the Lasso and Support Vector Machines Martin Jaggi Ecole Polytechnique 2013 / 07 / 08 ROKS 13 - International Workshop on Advances in Regularization, Optimization, Kernel Methods and
More informationAnalyzing Stochastic Gradient Descent for Some Non- Convex Problems
Analyzing Stochastic Gradient Descent for Some Non- Convex Problems Christopher De Sa Soon at Cornell University cdesa@stanford.edu stanford.edu/~cdesa Kunle Olukotun Christopher Ré Stanford University
More informationPose Locality Constrained Representation for 3D Human Pose Reconstruction
Pose Locality Constrained Representation for 3D Human Pose Reconstruction Xiaochuan Fan, Kang Zheng, Youjie Zhou, Song Wang Department of Computer Science & Engineering, University of South Carolina {fan23,zheng37,zhou42}@email.sc.edu,
More informationAlternating Direction Method of Multipliers
Alternating Direction Method of Multipliers CS 584: Big Data Analytics Material adapted from Stephen Boyd (https://web.stanford.edu/~boyd/papers/pdf/admm_slides.pdf) & Ryan Tibshirani (http://stat.cmu.edu/~ryantibs/convexopt/lectures/21-dual-meth.pdf)
More informationData-Dependent Sparsity for Subspace Clustering
Data-Dependent Sparsity for Subspace Clustering Bo Xin Microsoft Research, Beijing Yizhou Wang Peking University Wen Gao Peking University David Wipf Microsoft Research, Beijing Abstract Subspace clustering
More informationConstructing a Non-Negative Low Rank and Sparse Graph with Data-Adaptive Features
1 Constructing a Non-Negative Low Rank and Sparse Graph with Data-Adaptive Features Liansheng Zhuang, Shenghua Gao, Jinhui Tang, Jingjing Wang, Zhouchen Lin, Senior Member, and Yi Ma, IEEE Fellow, arxiv:1409.0964v1
More informationLatent Space Sparse and Low-rank Subspace Clustering
IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, VOL. X, NO. X, MONTH 2XX 1 Latent Space Sparse and Low-rank Subspace Clustering Vishal M. Patel, Member, IEEE, Hien V. Nguyen, Student Member, IEEE,
More informationELEG Compressive Sensing and Sparse Signal Representations
ELEG 867 - Compressive Sensing and Sparse Signal Representations Gonzalo R. Arce Depart. of Electrical and Computer Engineering University of Delaware Fall 211 Compressive Sensing G. Arce Fall, 211 1 /
More informationFAST PRINCIPAL COMPONENT PURSUIT VIA ALTERNATING MINIMIZATION
FAST PRICIPAL COMPOET PURSUIT VIA ALTERATIG MIIMIZATIO Paul Rodríguez Department of Electrical Engineering Pontificia Universidad Católica del Perú Lima, Peru Brendt Wohlberg T-5 Applied Mathematics and
More informationarxiv: v1 [cs.cv] 8 Sep 2017
Deep Subspace Clustering Networks Pan Ji 1, Tong Zhang 2, Hongdong Li 2, Mathieu Salzmann 3, Ian Reid 1 1 University of Adelaide, 2 Australian National University, 3 EPFL arxiv:1709.02508v1 [cs.cv] 8 Sep
More informationarxiv: v1 [cs.cv] 9 Sep 2013
Learning Transformations for Clustering and Classification arxiv:139.274v1 [cs.cv] 9 Sep 213 Qiang Qiu Department of Electrical and Computer Engineering Duke University Durham, NC 2778, USA Guillermo Sapiro
More informationCompressive Sensing. Part IV: Beyond Sparsity. Mark A. Davenport. Stanford University Department of Statistics
Compressive Sensing Part IV: Beyond Sparsity Mark A. Davenport Stanford University Department of Statistics Beyond Sparsity Not all signal models fit neatly into the sparse setting The concept of dimension
More informationReal-time Background Subtraction via L1 Norm Tensor Decomposition
Real-time Background Subtraction via L1 Norm Tensor Decomposition Taehyeon Kim and Yoonsik Choe Yonsei University, Seoul, Korea E-mail: pyomu@yonsei.ac.kr Tel/Fax: +82-10-2702-7671 Yonsei University, Seoul,
More informationDescriptorEnsemble: An Unsupervised Approach to Image Matching and Alignment with Multiple Descriptors
DescriptorEnsemble: An Unsupervised Approach to Image Matching and Alignment with Multiple Descriptors 林彥宇副研究員 Yen-Yu Lin, Associate Research Fellow 中央研究院資訊科技創新研究中心 Research Center for IT Innovation, Academia
More informationProduct Grassmann Manifold Representation and Its LRR Models
Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) Product Grassmann Manifold Representation and Its LRR Models Boyue Wang 1, Yongli Hu 1, Junbin Gao 2, Yanfeng Sun 1 and
More informationGeneralized Principal Component Analysis via Lossy Coding and Compression Yi Ma
Generalized Principal Component Analysis via Lossy Coding and Compression Yi Ma Image Formation & Processing Group, Beckman Decision & Control Group, Coordinated Science Lab. Electrical & Computer Engineering
More informationarxiv: v1 [cs.lg] 3 Jan 2018
CLUSTERING OF DATA WITH MISSING ENTRIES Sunrita Poddar, Mathews Jacob Department of Electrical and Computer Engineering, University of Iowa, IA, USA arxiv:1801.01455v1 [cs.lg] 3 Jan 018 ABSTRACT The analysis
More informationIntroduction to Topics in Machine Learning
Introduction to Topics in Machine Learning Namrata Vaswani Department of Electrical and Computer Engineering Iowa State University Namrata Vaswani 1/ 27 Compressed Sensing / Sparse Recovery: Given y :=
More informationarxiv: v2 [cs.cv] 7 Oct 2016
Shape Interaction Matrix Revisited and Robustified: Efficient Subspace Clustering with Corrupted and Incomplete Data arxiv:159.2649v2 [cs.cv] 7 Oct 216 Pan Ji 1, Mathieu Salzmann 2,3, and Hongdong Li 1,4
More informationFault detection with principal component pursuit method
Journal of Physics: Conference Series PAPER OPEN ACCESS Fault detection with principal component pursuit method Recent citations - Robust Principal Component Pursuit for Fault Detection in a Blast Furnace
More informationRobust Face Recognition via Sparse Representation Authors: John Wright, Allen Y. Yang, Arvind Ganesh, S. Shankar Sastry, and Yi Ma
Robust Face Recognition via Sparse Representation Authors: John Wright, Allen Y. Yang, Arvind Ganesh, S. Shankar Sastry, and Yi Ma Presented by Hu Han Jan. 30 2014 For CSE 902 by Prof. Anil K. Jain: Selected
More informationLearning Robust Low-Rank Representations
Learning Robust Low-Rank Representations Pablo Sprechmann Dept. of Elec. and Comp. Eng. Duke University Alexander M. Bronstein School of Elec. Eng. Tel Aviv University Guillermo Sapiro Dept. of Elec. and
More informationOptic Cup Segmentation for Glaucoma Detection Using Low-Rank Superpixel Representation
Optic Cup Segmentation for Glaucoma Detection Using Low-Rank Superpixel Representation Yanwu Xu 1, Lixin Duan 1, Stephen Lin 2, Xiangyu Chen 1, Damon Wing Kee Wong 1, Tien Yin Wong 3, and Jiang Liu 1 1
More informationGeneralized Principal Component Analysis CVPR 2007
Generalized Principal Component Analysis Tutorial @ CVPR 2007 Yi Ma ECE Department University of Illinois Urbana Champaign René Vidal Center for Imaging Science Institute for Computational Medicine Johns
More informationGraph Autoencoder-Based Unsupervised Feature Selection
Graph Autoencoder-Based Unsupervised Feature Selection Siwei Feng Department of Electrical and Computer Engineering University of Massachusetts Amherst Amherst, MA, 01003 siwei@umass.edu Marco F. Duarte
More informationRobust Subspace Segmentation by Simultaneously Learning Data Representations and Their Affinity Matrix
Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 015) Robust Subspace Segmentation by Simultaneously Learning Data Representations and Their Affinity Matrix
More informationDistributed Low-rank Subspace Segmentation
2013 IEEE International Conference on Computer Vision Distributed Low-rank Subspace Segmentation Ameet Talwalkar a Lester Mackey b Yadong Mu c Shih-Fu Chang c Michael I. Jordan a a University of California,
More informationLow-Rank Matrix Recovery III: Fast Algorithms and Scalable Applications Zhouchen Lin
Low-Rank Matrix Recovery III: Fast Algorithms and Scalable Applications Zhouchen Lin Visual Computing Group Microsoft Research Asia Aug. 11, 2011 Why do we need new algorithms? min kak + kek 1 subj A +
More informationFace Recognition via Sparse Representation
Face Recognition via Sparse Representation John Wright, Allen Y. Yang, Arvind, S. Shankar Sastry and Yi Ma IEEE Trans. PAMI, March 2008 Research About Face Face Detection Face Alignment Face Recognition
More informationRobust l p -norm Singular Value Decomposition
Robust l p -norm Singular Value Decomposition Kha Gia Quach 1, Khoa Luu 2, Chi Nhan Duong 1, Tien D. Bui 1 1 Concordia University, Computer Science and Software Engineering, Montréal, Québec, Canada 2
More informationarxiv: v1 [cs.cv] 13 Apr 2017
Collaborative Low-Rank Subspace Clustering Stephen Tierney a,, Yi Guo c, Junbin Gao b arxiv:1704.03966v1 [cs.cv] 13 Apr 2017 a School of Computing and Mathematics, Charles Sturt University, Bathurst, NSW
More informationRandom Walk Distributed Dual Averaging Method For Decentralized Consensus Optimization
Random Walk Distributed Dual Averaging Method For Decentralized Consensus Optimization Cun Mu, Asim Kadav, Erik Kruus, Donald Goldfarb, Martin Renqiang Min Machine Learning Group, NEC Laboratories America
More informationIMA Preprint Series # 2281
DICTIONARY LEARNING AND SPARSE CODING FOR UNSUPERVISED CLUSTERING By Pablo Sprechmann and Guillermo Sapiro IMA Preprint Series # 2281 ( September 2009 ) INSTITUTE FOR MATHEMATICS AND ITS APPLICATIONS UNIVERSITY
More informationLearning to Match. Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li
Learning to Match Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li 1. Introduction The main tasks in many applications can be formalized as matching between heterogeneous objects, including search, recommendation,
More informationShape Interaction Matrix Revisited and Robustified: Efficient Subspace Clustering with Corrupted and Incomplete Data
Shape Interaction Matrix Revisited and Robustified: Efficient Subspace Clustering with Corrupted and Incomplete Data Pan Ji 1, Mathieu Salzmann 2,3, and Hongdong Li 1,4 1 Australian National University,
More informationLearning based face hallucination techniques: A survey
Vol. 3 (2014-15) pp. 37-45. : A survey Premitha Premnath K Department of Computer Science & Engineering Vidya Academy of Science & Technology Thrissur - 680501, Kerala, India (email: premithakpnath@gmail.com)
More informationFINDING a subset of a large number of models or data
Dissimilarity-based Sparse Subset Selection Ehsan Elhamifar, Member, IEEE, Guillermo Sapiro, Fellow, IEEE, and S. Shankar Sastry, Fellow, IEEE arxiv:7.v [cs.lg] 5 Jul Abstract Finding an informative subset
More informationGlobally and Locally Consistent Unsupervised Projection
Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence Globally and Locally Consistent Unsupervised Projection Hua Wang, Feiping Nie, Heng Huang Department of Electrical Engineering
More informationFINDING a subset of a large number of models or data points,
Dissimilarity-based Sparse Subset Selection Ehsan Elhamifar, Member, IEEE, Guillermo Sapiro, Fellow, IEEE, and S Shankar Sastry, Fellow, IEEE The problem of finding data representatives has been well-studied
More informationBatch image alignment via subspace recovery based on alternative sparsity pursuit
Computational Visual Media DOI 10.1007/s41095-017-0080-x Vol. 3, No. 3, September 2017, 295 304 Research Article Batch image alignment via subspace recovery based on alternative sparsity pursuit Xianhui
More informationThe Pre-Image Problem in Kernel Methods
The Pre-Image Problem in Kernel Methods James Kwok Ivor Tsang Department of Computer Science Hong Kong University of Science and Technology Hong Kong The Pre-Image Problem in Kernel Methods ICML-2003 1
More informationDevelopment in Object Detection. Junyuan Lin May 4th
Development in Object Detection Junyuan Lin May 4th Line of Research [1] N. Dalal and B. Triggs. Histograms of oriented gradients for human detection, CVPR 2005. HOG Feature template [2] P. Felzenszwalb,
More informationCorrentropy Induced L2 Graph for Robust Subspace Clustering
2013 IEEE International Conference on Computer Vision Correntropy Induced Graph for Robust Subspace Clustering Canyi Lu 1, Jinhui Tang 2, Min Lin 1, Liang Lin 3, Shuicheng Yan 1, houchen Lin 4, 1 Department
More informationSignal Reconstruction from Sparse Representations: An Introdu. Sensing
Signal Reconstruction from Sparse Representations: An Introduction to Compressed Sensing December 18, 2009 Digital Data Acquisition Suppose we want to acquire some real world signal digitally. Applications
More information60 2 Convex sets. {x a T x b} {x ã T x b}
60 2 Convex sets Exercises Definition of convexity 21 Let C R n be a convex set, with x 1,, x k C, and let θ 1,, θ k R satisfy θ i 0, θ 1 + + θ k = 1 Show that θ 1x 1 + + θ k x k C (The definition of convexity
More informationUnsupervised Outlier Detection and Semi-Supervised Learning
Unsupervised Outlier Detection and Semi-Supervised Learning Adam Vinueza Department of Computer Science University of Colorado Boulder, Colorado 832 vinueza@colorado.edu Gregory Z. Grudic Department of
More informationarxiv: v1 [cs.cv] 12 Apr 2017
Provable Self-Representation Based Outlier Detection in a Union of Subspaces Chong You, Daniel P. Robinson, René Vidal Johns Hopkins University, Baltimore, MD, 21218, USA arxiv:1704.03925v1 [cs.cv] 12
More informationBilevel Sparse Coding
Adobe Research 345 Park Ave, San Jose, CA Mar 15, 2013 Outline 1 2 The learning model The learning algorithm 3 4 Sparse Modeling Many types of sensory data, e.g., images and audio, are in high-dimensional
More informationHashing with Graphs. Sanjiv Kumar (Google), and Shih Fu Chang (Columbia) June, 2011
Hashing with Graphs Wei Liu (Columbia Columbia), Jun Wang (IBM IBM), Sanjiv Kumar (Google), and Shih Fu Chang (Columbia) June, 2011 Overview Graph Hashing Outline Anchor Graph Hashing Experiments Conclusions
More informationSupport vector machine (II): non-linear SVM. LING 572 Fei Xia
Support vector machine (II): non-linear SVM LING 572 Fei Xia 1 Linear SVM Maximizing the margin Soft margin Nonlinear SVM Kernel trick A case study Outline Handling multi-class problems 2 Non-linear SVM
More informationBSIK-SVD: A DICTIONARY-LEARNING ALGORITHM FOR BLOCK-SPARSE REPRESENTATIONS. Yongqin Zhang, Jiaying Liu, Mading Li, Zongming Guo
2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) BSIK-SVD: A DICTIONARY-LEARNING ALGORITHM FOR BLOCK-SPARSE REPRESENTATIONS Yongqin Zhang, Jiaying Liu, Mading Li, Zongming
More informationLecture 17 Sparse Convex Optimization
Lecture 17 Sparse Convex Optimization Compressed sensing A short introduction to Compressed Sensing An imaging perspective 10 Mega Pixels Scene Image compression Picture Why do we compress images? Introduction
More informationA fast algorithm for sparse reconstruction based on shrinkage, subspace optimization and continuation [Wen,Yin,Goldfarb,Zhang 2009]
A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization and continuation [Wen,Yin,Goldfarb,Zhang 2009] Yongjia Song University of Wisconsin-Madison April 22, 2010 Yongjia Song
More informationAn R Package flare for High Dimensional Linear Regression and Precision Matrix Estimation
An R Package flare for High Dimensional Linear Regression and Precision Matrix Estimation Xingguo Li Tuo Zhao Xiaoming Yuan Han Liu Abstract This paper describes an R package named flare, which implements
More informationMultiple-View Object Recognition in Band-Limited Distributed Camera Networks
in Band-Limited Distributed Camera Networks Allen Y. Yang, Subhransu Maji, Mario Christoudas, Kirak Hong, Posu Yan Trevor Darrell, Jitendra Malik, and Shankar Sastry Fusion, 2009 Classical Object Recognition
More informationOutline Introduction Problem Formulation Proposed Solution Applications Conclusion. Compressed Sensing. David L Donoho Presented by: Nitesh Shroff
Compressed Sensing David L Donoho Presented by: Nitesh Shroff University of Maryland Outline 1 Introduction Compressed Sensing 2 Problem Formulation Sparse Signal Problem Statement 3 Proposed Solution
More informationA (somewhat) Unified Approach to Semisupervised and Unsupervised Learning
A (somewhat) Unified Approach to Semisupervised and Unsupervised Learning Ben Recht Center for the Mathematics of Information Caltech April 11, 2007 Joint work with Ali Rahimi (Intel Research) Overview
More informationData-Driven Geometry Processing Map Synchronization I. Qixing Huang Nov. 28 th 2018
Data-Driven Geometry Processing Map Synchronization I Qixing Huang Nov. 28 th 2018 Shape matching Affine Applications Shape reconstruction Transfer information Aggregate information Protein docking Pair-wise
More informationRobust and Scalable Column/Row Sampling from Corrupted Big Data
Robust and Scalable Column/Row Sampling from Corrupted Big Data Mostafa Rahmani University of Central Florida mostafa@knights.ucf.edu George Atia University of Central Florida George.Atia@ucf.edu Abstract
More information