Stopping Criteria for Iterative Solution to Linear Systems of Equations

Transcription

1 Stopping Criteria for Iterative Solution to Linear Systems of Equations Gerald Recktenwald Portland State University Mechanical Engineering Department

2 Iterative Methods: High-level view Solve Ax = b by an iterative method that obtains a sequence of approximations to the solution x k k = 0, 1, 2,... such that (b Ax k ) 0 as k. The residual at iteration k is r k = b Ax k k = 0, 1, 2,... (1) If an iterative methods converges, then r k 0 as k. ME 448/548: Stopping criteria for iterative solvers page 1

3 Convergence analysis Let x designate the exact solution. x is obtained only after an infinite number of iterations of an iterative method. Substitute Ax for b into Equation (1) r k = Ax Ax k = A(x x k ) = x x k = A 1 r k. x k x is the error at iteration k ME 448/548: Stopping criteria for iterative solvers page 2

4 Convergence analysis The magnitude of the error at the k th iteration is x x k = A 1 r k. From the definition of matrix norms, (see, e.g., [1, p. 42]) A 1 r k A 1 r k so x x k A 1 r k. Dividing through by the scalar, x, gives x x k x A 1 rk x. (2) ME 448/548: Stopping criteria for iterative solvers page 3

5 Convergence analysis We need a replacement for 1 x on the right hand side of Equation (2). Consider b = Ax A x = 1 x A b (3) Substituting the right hand side of Equation (3) into Equation (2) gives x x k x A 1 r k A b = A 1 A rk b (4) ME 448/548: Stopping criteria for iterative solvers page 4

6 Convergence analysis Introduce the condition number of A into Equation (4) to get κ(a) A A 1 x x k x κ(a) is large if A is ill-conditioned. Therefore κ(a) rk b (5) r k / b is an estimate of the relative error at iteration k For large κ(a) r k / b is not a reliable indicator of convergence ME 448/548: Stopping criteria for iterative solvers page 5

7 Implementation The preceding analysis suggests that iterations should continue until r k b < ɛ (6) where ɛ is a small value, but much larger than machine precision. Typically > ɛ > , but the value of ɛ will depend on the problem being solved and the cost of the iterations. ME 448/548: Stopping criteria for iterative solvers page 6

8 Implementation A =... % A and b are defined by the problem to be solved b =... x =... % initial guess tol =... % Convergence tolerance, e.g. tol = 5.0e-4 normb = norm(b); % compute b only once itermax =... % Limit on the number of iterations while i<itermax r = b - A*x; % Compute residual with previous x normr = norm(r) % Save for printing if normr/normb < tol, break; end % Test before solving; exit loop if true x = itersolver(a,b,x); % Update the solution i = i + 1; fprintf( %4d %12.3e\n,i,normr); end ME 448/548: Stopping criteria for iterative solvers page 7

9 Implementation in CFD Codes In CFD codes a typical stopping criterion is r k r 0 < ɛ (7) where r 0 = b Ax 0 is the residual based on the initial guess, x 0. If x 0 = 0, i.e., if the initial guess is a vector of zeros, then r 0 = b. However, for any other x 0, the value of r 0 may be large and unrelated to the true solution. If r 0 is artificially large then the iterations may be stopped prematurely. ME 448/548: Stopping criteria for iterative solvers page 8

10 Non-linear Systems of Equations Now consider where A = A(x) and b = b(x). Ax = b, (8) Solution methods for nonlinear systems are generalizations of root-finding methods for scalar equations f(x) = 0 Rearrange Equation (8) where f(x) is a vector valued function of x. f(x) = Ax b = 0, (9) Note that r = b Ax = f(x). (10) ME 448/548: Stopping criteria for iterative solvers page 9

11 Non-linear Systems of Equations An iterative method to solve Equation (8) can be written as x k+1 = x k + x k, k = 1, 2,..., (11) x k is obtained from the linearized coefficient matrix and right-hand-side vector A k = A(x k ), b k = b(x k ). Before the update to x k is computed, the vector f k = f(x k ) = r k, or f k = A k x k b k, (12) will not be zero unless x k is the solution to the nonlinear problem. ME 448/548: Stopping criteria for iterative solvers page 10

12 Non-linear Systems of Equations Two very important observations: 1. The non-linear system must use iterations to approach the solution. There is no direct method to solve Ax = b when A = A(x) and/or b = b(x). 2. The update step, x k+1 requires work equivalent to solving Ax = b. Each step of the iterative solver requires solution of a linear(ized) system of equations. ME 448/548: Stopping criteria for iterative solvers page 11

13 Inner and Outer Iterations Outer Iterations: Each step of the updating the non-linear system of equations A k x k+1 = b k is called an outer iteration. The SIMPLE algorithm is the recipe for the outer iterations when the segregated solver is used. Inner Iterations: Each step of the iterative solver applied to the frozen coefficients A k and b k for finding x k+1 is called an inner iteration. ME 448/548: Stopping criteria for iterative solvers page 12

14 Inner and Outer Iterations We only really care about the residual of the outer iterations. Don t spend too much effort on the inner iterations because A k and b k need to be updated. Don t spend too little effort on the inner iterations because we want x k+1 moves the non-linear solution in the right direction. ME 448/548: Stopping criteria for iterative solvers page 13

15 Summary Direct methods for solving Ax = b become prohibitive for very large numbers of unknowns. Iterative methods evolved in the 1980 s and 1990 s to become both robust and efficient for very large systems. Non-linear systems require iterations anyway. A k and b k are updated for outer iterations Ill-conditioned systems cause trouble Uncertainty in the solution Poor convergence of iterative methods References [1] G. Golub and J. M. Ortega. Scientific Computing: An Introduction with Parallel Computing. Academic Press, Inc., Boston, ME 448/548: Stopping criteria for iterative solvers page 14