2.7 Numerical Linear Algebra Software

Transcription

1 2.7 Numerical Linear Algebra Software In this section we will discuss three software packages for linear algebra operations: (i) (ii) (iii) Matlab, Basic Linear Algebra Subroutines (BLAS) and LAPACK. There are a number of other software packages (see and many of these are being continually updated by the numerical analysis community. Matlab. Matlab features a nice front end with convenient input/output and state-of-the-art algorithms. Some Matlab functions include the following. Linear algebra: norm norm of a vector/matrix cond condition number of a matrix lu chol inv LU factorization Cholesky factorization matrix inverse Eigenvalues hess Hessenberg form qr orthogonal-triangular decomposition schur Schur decomposition svd eig singular value decomposition eigenvalues and eigenvectors

2 2 Solves : rref reduced row echelon form A\d if A is nxn, returns the solution to Ax = d if A is mxn, m > n, returns least squares solution to Ax = d. Graphics: plot y versus x mesh contour slice 3D plots contours of 3D plots allows you to draw planes through a 3D Programming: looping conditional statements Programming constructs in Matlab are slow because they are interpreted. For example, calculating the product of two matrices using Matlab s matrix multiplication operator is much faster than calculating the same product using code with three loops. Matlab is often used for experimentation because it is easier and faster to write code for Matlab than languages like Fortran or C. Matlab is also commonly used for graphics. The command load filename where filename is the name of an output file from some program in table form (i.e. columns separated by whitespace, rows separated by newlines), creates a matrix named filename with the data from the input file. You can then graph the data using one of the graph functions. Basic Linear Algebra Subroutines (BLAS) 1, 2, or 3. See for more information. BLAS 1 functions where the number of operations = O(n) constant times a vector

3 3 dot product a constant times a vector plus a vector The BLAS command for this is SAXPY(list), where list is a list of parameters. BLAS 2 functions where the number of operations = O(n 2 ) Matrix times a vector Ax...SGEMV(list) Triangular solve Ux = b BLAS 3 functions where the number of operations = O(n 3 ) Matrix times a matrix AB...SGEMM(list) These subroutines are usually optimized for the memory hierarchy of a particular computer. LAPACK. Some commands: SGETRF(list) Gaussian elimination triangular factorization. Factors PA=LU. O(2/3 n 3 ) SGETRS(list) triangular solve. Solves Ux = d. O(n 2 ) SGECON(list) condition number The following calculations were done on the Cray T916 at the North Carolina Supercomputing Center. All calculations were for a matrix-vector product where the matrix was 500x500 for 2(500) 2 = floating point operations. In the ji Fortran code the inner loop was vectorized or not vectorized by using the Cray directives cdir$ vector and cdir$ novector just before the loop 30. There are two ways to compute a matrix-vector product. The traditional way is to compute the dotproduct of the rows in the matrix with the vector. The second method is to view the product

4 4 as a linear combination of the columns of the matrix where the coefficients are the components of the vector. These methods are also referred to as the ij and ji methods, respectively. This is because of the order of the loops as illustrated below. Dotproduct Method for Ax = d (ij version). for i =1,m d(i) = 0 for j = 1,n d(i) = d(i) + a(i,j)*x(j) Linear Combination of Columns Method for Ax = d (ji version). d(1:m) = 0 for j = 1,n for i = 1,m d(i) = d(i) + a(i,j)*x(j) Cray T916 Code (mvji.f) program matvec dimension a(500,500),x(500),prod(500) print*, 'input n' read(5,*) n do 10 i = 1,n prod(i) = 0.0 x(i) = sqrt(1./i) do 10 j = 1,n a(i,j) = 1.0/j 10 continue tbegin = second() do 20 j = 1,n cdir$ vector do 30 i = 1,n prod(i) = prod(i) + a(i,j)*x(j) 30 continue 20 continue tend = second() print*, tend - tbegin stop end

5 5 The Cray compiler cf77 was used in the five calculations in the next table. The first two calculations indicate a speedup of over 20 for the ji method. The next two calculations illustrate that the ij method is slower than the ji method. This is because Fortran stores numbers of a two dimensional array by columns. Since the ij method gets rows of the array, the input into the vector pipe will be in stride equal to n. The last calculation used the -Zv option. Here the code was rewritten. The loops 20 and 30 were recognized to be a matrix-vector product and an optimized BLAS2 (basic linear algebra subroutine with order n 2 operations, see or ) subroutine was used. This gave an additional speedup over 2 for an overall speedup equal to about 46. The floating point operations per second for this last computations was ( )/ ( ) or about 1,600 megaflops. Table: Matrix-vector Computation Times Method Time x 10-4 sec. ji (vec. on) 6.59 ji (vec. off) 146. ij (vec. on) ij (vec. off) 153. SGEMVX via -Zv 3.12 Why is SGEMVX faster? It takes advantage of the memory hierarchy of the Cray. It divides A and x into smaller blocks that fit in the Cray's register memory. This allows the program to access the data much faster. Also the blocks are ordered so as to avoid repeated moving the same data in and out of the cache memory.