Project Report Bernardo A. Gonzalez Torres beaugonz@ucsc.edu Abstract The final term project consist of two parts: a Fortran implementation of a linear algebra solver and a Python implementation of a run setup, a run scheduler, and a data visualizer. In order to solve the matrix equation Ax = b () where A R n n and x, b R n, the Fortran implementation applies Gaussian elimination with or without partial pivoting to decompose matrix A into a lower L and upper U triangular matrices such that LU. Substituting this in equation (): Ax = LUx = b () Let y R n be a vector such that Ux = y, then from equation (): LUx = Ly = b (3) Solving successively the matrix equations Ly = b and then Ux = y is straightforward because both are triangular matrices. Solution x obtained after this process is a valid solution for the matrix equation Ax = b. Algorithms. Gaussian elimination without partial pivoting Following the reading material provided by the instructor [], the algorithm applied to compute the LU decomposition without pivoting is shown in Algorithm.. Gaussian elimination with partial pivoting Unlike the method without pivoting, Gaussian elimination with partial pivoting consecutively applies row permutation to matrix A in order to avoid possible a kk diagonal entries of matrix A being equal to zero. Gaussian elimination with partial pivoting solves the matrix equation Ax = b decomposing matrix A into a lower L and upper U triangular matrices such that P LU, where P is a row permutation matrix. Multiplying equation () by matrix P and substituting P LU: PAx = Pb (4) PAx = LUx = Pb (5)
Algorithms Algorithm LU factorization by Gaussian elimination without pivoting : for k = to n do : [loop over column] 3: if a kk = then 4: stop 5: [stop if pivot (or divisor) is zero] 6: end if 7: for i = k + to n do 8: l ik = a ik /a kk 9: [compute multipliers for each column] : end for : for j = k + to n do : for i = k + to n do 3: a ij = a ij l ik a kj 4: [transformation to remaining submatrix] 5: end for 6: end for 7: end for Similarly to the method without pivoting, let y R n be a vector such that Ux = y, then from equation (5): LUx = Ly = Pb (6) To calculate a solution x for the matrix equation Ax = b, we successively solve the matrix equations Ly = Pb and then Ux = y. Following [], the algorithm applied to compute the LU decomposition with (partial) pivoting is shown in Algorithm. Matrix I is the identity matrix. Algorithm LU factorization by Gaussian elimination with (partial) pivoting : U = A, L = I : for k = to n do 3: Select i k to maximize u ik 4: if i k then 5: u k,k:n u i,k:n 6: b k b i 7: l k,:k l i,:k 8: [interchange rows if needed] 9: end if : for j = k + to n do : l jk = u jk /u kk : [compute multipliers for each column] 3: u j,k:n = u j,k:n l jk u k,k:n 4: [transformation to remaining submatrix] 5: end for 6: end for
3 Fortran Implementation 3 3 Fortran Implementation The Fortran implementation imports the values of the entries of matrix A and vector b from input files A i.dat and b i.dat respectively, for i N, and after reading those from files, writes A and b onto screen so that users can check if the inputs are correct. Files A i.dat and b i.dat are initialized by the Python implementation. Then, the implementation applies Gaussian elimination with or without partial pivoting to decompose matrix A into a lower L and upper U triangular matrices such that LU, Ly = b and Ux = y. Once that matrices L and U had been computed, the implementation solves successively the matrix equations Ly = b and then Ux = y to obtain solution x. This solution is written to screen and to a file called x i.dat, i N. All Fortran files are in the directory project/linalg. The implementation has one main driver routine called linear solve which calls others necessary subroutines and functions. The structure of the code is the following: linear solve.f9 : main driver routine. setup module.f9 : module that reads in runtime parameters from setup.init file. read data.f9 : module that reads in matrix A and vector b from files. write to screen.f9 : module that writes matrix A and vector b to screen for sanity check. LU decomp.f9 : module that implements Gaussian elimination with or without partial pivoting (Algorithms and ). forward solve.f9 : module that solves Ly = b. backward solve.f9 : module that solves Ux = y. write data.f9 : module that outputs the solution vector x onto screen as well as to a file called x i.dat, for each i. 4 Python Implementation The Python implementation initialize three different inputs of a pair A and b and write them to files A i.dat and b i.dat respectively, for each i. Then, it compiles the Fortran code in the directory project/linalg from within the Python directory project/pyrun. After compiling the Fortran code, the Python implementation runs three cases for different matrices A and vectors b. The matrices A and vectors b are the following: (7)
5 User s manual 4 4 3 3 4 3 3 4 3 3 4 3 3 3 4 7 7 3 7 4 (8) 8 (9) Once that solution x had been computed, the Python implementation check the correctness of the Fortran solutions computing x using numpy library. The implementation compares Fortran and Python solutions and outputs Pass or Fail depending on the results. A threshold value is used to quantify the error between the two solutions. This threshold value could be modified in the Python code. In both cases (Fail or Pass) the Fortran and Python solutions are written to screen. If the result is Fail, the error is written to screen too. Finally, the implementation produces three plots, one for each matrix A, and another three plots of vectors x and b. The pyrun LinAlg.py file is in the directory project/pyrun. following: The structure of the code is the make make() : function that compiles the Fortran code. matrix init(input number) : function that writes the A i.dat and b i.dat files, for each i, and the setup.init file. The argument input number is the number of the matrix A and vector b (, or 3). run linearsolver() : function that runs the Fortran executable file. check solution(input number, threshold) : function that compares the solution of the Fortran code with one computed with numpy library. The argument input number is the number of the solution vector x (, or 3) and the argument threshold is the value of the threshold used to compare the solutions. plot data(input number) : function that plots the matrix A and vector b. The argument input number is the number of the matrix A and vector b (, or 3). 5 User s manual To run the code, you just need to type in the command line: >> python pyrun LinAlg.py while you are in the directory project/pyrun. The only modifiable value in the code is the variable threshold, and it s value is used to compare the solutions computed by Python and Fortran. Once the execution begins, you will be asked if the input in screen is correct. If you answer is no, the message SORRY, INPUT MATRIX A AND/OR VECTOR b ARE INCORRECT will prompt in screen and the execution will halt with no results, as shown in figure.
6 Results 5 Fig. : Incorrect input In a similar way, you will be asked if you want to use the version with or without pivoting. If you answer anything different to zero or one, the message INVALID PIVOTING METHOD will prompt in screen and the execution will halt with no results, as shown in figure. Fig. : Incorrect input for pivoting method A successful execution is shown in figure 3. 6 Results The computed vectors x are the following: 4 3 3 4 3 3 4 3 3 4, x =, x = 3 () ()
7 Conclusion 6 Fig. 3: Successful execution 3 3 3 4 7 7 3 7 4 8, x = 3 3 () Both the Gaussian elimination with and without pivoting gets the same results, as well as the Python solutions calculated using numpy. In figures 4 and 5 are shown the plots of the matrix A and vectors x and b shown in (). Due to the lack of space, the rest of the plots of the matrices A and vectors x and b are not shown in this report, but can be easily found in the directory project/linalg. Fig. 4: Matrix A 7 Conclusion Gaussian elimination without pivoting was straightforward to implement using the reading material provided by the Professor. This method worked well in the test matrices provided for the project. However, this method is known to fail when a diagonal element of the matrix results zero (a kk = ).
7 Conclusion 7 Fig. 5: Vectors x and b On the other hand, the computation of the matrices L and U using Gaussian elimination with partial pivoting is more robust due to the row permutations. This method was not easy to implement using the material provided, but the algorithm and the explanation in [] were very useful for it s implementation. Although another methods like complete pivoting exists, one advantage of partial pivoting is that the complexity does not increases due to the fact that it only search for pivots in the same row, which is a O(n) search. References [] Lee, Dongwook. AMS 9, Fall 6. Final Project Type A Numerical Linear Algebra: Gaussian Elimination with Pivoting for Solving Linear Systems https://users.soe.ucsc. edu/~dongwook/wp-content/uploads/6/ams9/lecturenote/_build/html/_downloads/ gaussian_elimination_pivoting.pdf [] Lecture. Pivoting http://www4.ncsu.edu/~kksivara/ma55/handouts/lu-pivot.pdf