CALCULATING RANKS, NULL SPACES AND PSEUDOINVERSE SOLUTIONS FOR SPARSE MATRICES USING SPQR

Transcription

1 CALCULATING RANKS, NULL SPACES AND PSEUDOINVERSE SOLUTIONS FOR SPARSE MATRICES USING SPQR Leslie Foster Department of Mathematics, San Jose State University October 28, 2009, SIAM LA 09 DEPARTMENT OF MATHEMATICS, SAN JOSE STATE UNIVERSITY CALCULATING RANKS, NULL SPACES AND PSEUDOINVER

2 OUTLINE Goal: Reliable algorithm for calculating ranks, null spaces and pseudoinverse solutions for large sparse matrices in the presence of errors I. Numerical rank, Numerical Null Space,... II. Tools: SPQR, subspace iteration III. Algorithm IV. Numerical Experiments V. Conclusions

3 NUMERICAL RANK, NUMERICAL NULL SPACE For m n matrix A and a tolerance tol Numerical rank: r = no. of singular values > tol Numerical Null Space Basis: n (n r) matrix with orthonormal columns with AX <= tol Pseudoinverse solution: min norm solution to b Âx,  has rank r and  = A

4 TOOLS: SPQR SPQR (Tim Davis, 2009): Fast Multifrontal Sparse QR Factorization Estimates numerical rank approximately; often an accurate estimate, but not always Can return Q as sparse Householder transforms

5 TOOLS: SUBSPACE ITERATION Algorithm SSI: apply to k k triangular matrix T, returns approximate singular values <= tol, and one singular value > tol plus singular vectors block power method applied to T 1 T T increase block size until estimate of σ r of T > tol References: Chan, Vogel, Gotsman, Toledo

6 ALGORITM SPQR_NULL SPQR_NULL: returns accurate numerical rank, orthogonal null space basis, and pseudoinverse solution accurate numerical rank means that when the estimated numerical rank is not correct, a flag is returned warning the user

7 ALGORITHM SPQR_NULL, CONT D Input: m n matrix A and tolerance tol Use SPQR twice to construct approximate complete orthogonal decomposition ( ) T 0 A = Q 1 P 2 Q2 T P1 T + E, 0 0 with Q 1, Q 2, P 1, P 2 orthogonal. Know E F. Use SSI to find basis X for numerical null space of T. T usually has a small, often 0, nullity.

8 ALGORITHM SPQR_NULL, CONT D The null ( space basis ) of A is X 0 X = P 1 Q 2. Save in factored form. 0 I Use singular value estimates and E F to confirm that the rank is correct, or return warning if not possible to confirm. Use factorization and, when T is numerical singular, deflation to calculate pseudoinverse solution.

9 MATRIX TEST SET SJSU Singular Matrix Database at singular/matrices/ 767 numerically singular matrices (with tolerance max(m, n) eps(normest(a)) ) Matrices come from real world applications or have characteristic features of real world problems Most matrices from UF Sparse Matrix Collection (Davis)

10 PROPERTIES OF TEST SET MATRICES:

11 PROPERTIES OF TEST SET MATRICES (CONT D):

12 ACCURACY, CORRECT RANKS:

13 ACCURACY, CORRECT RANKS OR WARNING:

14 ACCURACY OF RANK CALCULATION: warning flag had no false positives

15 ACCURACY OF NULL SPACE BASES: QR based method overall as good as SVD

16 RUN TIMES: SPQR_NULL VERSUS MATLAB S SVD:

17 RUN TIMES: SPQR_NULL VERSUS MATLAB S SVDS: For a run over 372 matrices with smaller dimensions MATLAB s SVDS required 8 hours SPQR_NULL required 1.5 minutes SVDS error flag reported an error for 89% of the matrices MATLAB s SVDS not useful for finding null spaces bases of matrices

18 RUN TIMES: SPQR_NULL VERSUS SPQR: SPQR_NULL is slower than SPQR

19 MEMORY USE FOR NULL SPACE BASES Overall SPQR_NULL null space bases requires less memory than dense bases

20 CONCLUSIONS / FURTHER WORK: SPQR_NULL reliably determines rank, null space basis and psuedoinverse solutions Much faster than MATLAB s SVD More reliable than SPQR and MATLAB s SVDS More features than SPQR Further work: Related algorithms Applications

21 RUN TIMES: SPQR_NULL VERSUS SPQR + SSI maximum number total time total time numerical of (hours) for (hours) for nullity matrices SPQR_NULL SPQR + SSI not available TABLE: Total times (sum of the run times for the matrices in the indicated subsets of our test set) for SPQR_NULL and SPQR + SSI