Iterative Methods for Linear Systems

Transcription

1 Iterative Methods for Linear Systems 1 the method of Jacobi derivation of the formulas cost and convergence of the algorithm a Julia function 2 Gauss-Seidel Relaxation an iterative method for solving linear systems the algorithm MCS 471 Lecture 5(b) Numerical Analysis Jan Verschelde, 27 June 2018 Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

2 Iterative Methods for Linear Systems 1 the method of Jacobi derivation of the formulas cost and convergence of the algorithm a Julia function 2 Gauss-Seidel Relaxation an iterative method for solving linear systems the algorithm Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

3 a fixed point formula We want to solve Ax = b for A R n n, b R n, for very large n. Consider A = L + D + U, where L = [l i,j ], l i,j = a i,j, i > j, l i,j = 0, i j. L is lower triangular. D = [d i,j ], d i,i = a i,i 0, d i,j = 0, i j. D is diagonal. U = [u i,j ], u i,j = a i,j, i < j, u i,j = 0, i j. U is upper triangular. Then we rewrite Ax = b as Ax = b (L + D + U)x = b Dx = b Lx Ux Dx = Dx + b Lx Ux Dx Dx = Dx + b Ax x = x + D 1 (b Ax). The fixed point formula x = x + D 1 (b Ax) is well defined if a i,i 0. Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

4 the Jacobi iterative method The fixed point formula x = x + D 1 (b Ax) leads to ( x (k+1) = x (k) + D 1 b Ax (k)), k = 0, 1,... } {{ } x Writing the formula as an algorithm: Input: A, b, x (0), ɛ, N. Output: x (k), k is the number of iterations done. for k from 1 to N do x := D 1 (b Ax (k) ) x (k+1) := x (k) + x exit when ( x ɛ) end for. Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

5 Iterative Methods for Linear Systems 1 the method of Jacobi derivation of the formulas cost and convergence of the algorithm a Julia function 2 Gauss-Seidel Relaxation an iterative method for solving linear systems the algorithm Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

6 cost and convergence Counting the number of operations in for k from 1 to N do x := D 1 (b Ax (k) ); x (k+1) := x (k) + x; exit when ( x ɛ); end for. we have a cost of O(Nn 2 ), O(n 2 ) for Ax (k), if A is dense. Theorem The Jacobi method converges for strictly row-wise or column-wise diagonally dominant matrices, i.e.: if a i,i > j i a i,j or a i,i > j i a j,i, i = 1, 2,..., n. Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

7 Iterative Methods for Linear Systems 1 the method of Jacobi derivation of the formulas cost and convergence of the algorithm a Julia function 2 Gauss-Seidel Relaxation an iterative method for solving linear systems the algorithm Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

8 a Julia function function jacobi(mat::array{float64},rhs::array{float64}, sol::array{float64}, maxit::int=100,tol::float64=1.0e-8) # # Runs the method of Jacobi on the linear system with coefficient matrix # in mat, with right hand side vector in rhs, and a start solution in sol. # Running stops if the maximum number of iterations in maxit is reached, # or if the norm of the correction is less than the given tolerance. # nbrows, nbcols = size(mat) result = deepcopy(sol) numit = 0 while numit < maxit numit = numit + 1 deltax = rhs - mat*result for i=1:nbrows deltax[i] = deltax[i]/mat[i,i] end result = result + deltax nrm = norm(deltax) println(nrm) if norm(deltax) <= tol return [result, numit] end end return [result, numit] end Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

9 a test system For the dimension n, we consider the diagonally dominant system: n x 1 2n 1 n x = 2n n + 1 x n 2n The exact solution is x: for i = 1, 2,..., n, x i = 1. Exercise 1: Start the Jacobi iteration method at x (0) = 0, with parameters: ɛ = 10 4 and N = 2n 2. How many steps does the method of Jacobi take to converge? Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

10 Iterative Methods for Linear Systems 1 the method of Jacobi derivation of the formulas cost and convergence of the algorithm a Julia function 2 Gauss-Seidel Relaxation an iterative method for solving linear systems the algorithm Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

11 fixed point formula for Gauss-Seidel relaxation The fixed point formula for Ax = b where A = L + D + U, L is strict lower triangular, L = [a i,j ], i > j, 0 otherwise D is diagonal, D = [a i,j ], i = j, 0 otherwise U is strict upper triangular, U = [a i,j ], i < j, 0 otherwise Ax = b (L + D + U)x = b (L + D)x + Ux = b (L + D)x = b Ux Observe that L + D is lower triangular. We apply forward substitution in each step. Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

12 the formulas for Gauss-Seidel relaxation We want to solve Ax = b for A R n n, b R n, for very large n. Writing the method of Jacobi componentwise: x (k+1) i := x (k) i + 1 n b i a i,j x (k) a j, i,i j=1 we observe that we can already use x (k+1) j for j < i. This leads to the following formulas x (k+1) i := x (k) i + 1 i 1 b i a i,j x (k+1) a j i,i j=1 i = 1, 2,..., n n a i,j x (k), i = 1, 2,..., n. j=i j Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

13 Iterative Methods for Linear Systems 1 the method of Jacobi derivation of the formulas cost and convergence of the algorithm a Julia function 2 Gauss-Seidel Relaxation an iterative method for solving linear systems the algorithm Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

14 the Gauss-Seidel method Writing the formulas as an algorithm: Input: A, b, x (0), ɛ, N. Output: x (k), k is the number of iterations done. for k from 1 to N do for i from 1 to n do x i := b i for j from 1 to i 1 do x i := x i a i,j x (k+1) j for j from i to n do x i := x i a i,j x (k) j x (k+1) := x (k) + x exit when ( x ɛ) Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

15 convergence We have the same condition on convergence as the method of Jacobi: Theorem The Gauss-Seidel method converges for strictly row-wise or column-wise diagonally dominant matrices, i.e.: if a i,i > j i a i,j or a i,i > j i a j,i, i = 1, 2,..., n. The method of Gauss-Seidel converges faster than the method of Jacobi. Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16

16 some more exercises Exercise 2: Write a script to run the method of Gauss-Seidel. Exercise 3: Consider again the test system as in exercise 1. Solve exercise 1 with the method of Gauss-Seidel. Compare the convergence of the method of Gauss-Seidel with the method of Jacobi. Numerical Analysis (MCS 471) Iterative Methods for Linear Systems L-5(b) 27 June / 16