An Investigation into Iterative Methods for Solving Elliptic PDE s Andrew M Brown Computer Science/Maths Session (2000/2001)

Transcription

1 An Investigation into Iterative Methods for Solving Elliptic PDE s Andrew M Brown Computer Science/Maths Session (000/001)

2 Summary The objectives of this project were as follows: 1) Investigate iterative methods for solving elliptic PDE s ) Implement with Matlab and assess their effectiveness 3) Investigate Multigrid methods to improve the performance of the basic iterative methods 4) Modify components of the Multigrid methods and compare the results 5) Apply the method to a non-linear problem I have selected a 1-dimensional model elliptic problem to study, and developed different methods for solving this problem. Their performance was evaluated with respect to a variety of the methods parameters. The minimum requirements of objectives 1-3 were all met during the course of the project. After the minimum requirements were completed I decided to focus on objective 4 rather than moving on to objective 5.

3 Acknowledgements I would like to thank Mark Walkley for his valuable input and support during this investigation.

4 Contents 1.0 Introduction Project Outline 1 1. Project Management 1.0 Background Research.1 Partial Differential Equations.. Elliptic Equations.3 Model Problem Numerical Methods for the Model Problem Finite Difference Method 4 3. Direct Method Iterative Method Jacobi Method Weighted Jacobi Method Gauss-Seidel Method Error Equation and Residual Two-Grid Correction Scheme Restriction and Interpolation Multigrid Method V-Cycle Scheme Good Initial Guess Implementation Direct Method Jacobi Method Weighted Jacobi Method Gauss-Seidel Method Restriction and Interpolation Calculating the Residual Multi-Structure Two-Grid Correction Scheme. 1

5 4.9 V-Cycle Scheme Initial Guess Work Units Results Jacobi Method Two-Grid Correction Scheme V-Cycle Scheme Initial Guess 5.5 Gauss-Seidel Method Iterative Methods, Evaluation Iterative Methods, Conclusions and Recommendations 8 References 9 Appendix A.. 30 Appendix.. 31

6 1.0 Introduction 1.1 Project Outline Investigate iterative methods for solving partial differential equations ( PDE s). Firstly basic iterative methods such as Jacobi and Gauss- Seidel are explored. The weaknesses of these methods should be apparent and we will see why there is a need for a more refined method such as Multigrid. The Multigrid method can then be implemented and examined to see how performance is improved in comparison to the straightforward methods investigated previously. The program Matlab can be used to implement these methods so that their effectiveness can be easily analysed and demonstrated using graphical techniques. 1. Project Management The project went to schedule as outlined in the mid-term report. All milestones were met by the dates specified. The only revision to the plan was not to investigate a non-linear problem, instead to spend extra time on modifying the Multigrid method further.

7 .0 - Background Research.1 Partial Differential Equations Partial differential equations ( PDE s) are very important in mathematics and engineering because the equations can model many real-life situations. Often these equations are too complicated to be solved using exact methods so we have to find numerical solutions. For linear equations in two dimensions there is a simple classification in terms of the general equation (Smith, 1978), ² u ² u a + b x² x y ² u u u + c + d + e + y² x y fu + g = 0 (where a, b, c, d, e, f and g may be functions of the independent variables x and y) as shown in the table below (Smith, 1978): Condition Type Example b² < ac Elliptic Laplace s Equation b² > ac Hyperbolic Wave Equation b² = 0 Parabolic Diffusion The remainder of this project will concentrate on elliptic PDE problems i.e. when b² < ac.. Elliptic Equations Elliptic equations are generally associated with equilibrium or steady-state problems. Two well known elliptic equations (in one-dimension) are: Poisson s equation: Laplace s equation: d u + g = 0 dx d u dx = 0 where we have dependent variable u(x), independent variable x and a function g = f (x). Poisson s equation can model incompressible viscous fluid, theories of electricity and magnetism. Laplace s equation can model the steady flow of heat or electricity, the irrotational flow of incompressible fluid, and potential problems in electricity and magnetism (Smith, 1978).

8 .3 Model Problem We are aiming to find numerical methods that can be applied to problems that cannot be handled analytically. However, if we are going to test the performance of the different methods we develop we need a model problem that can be solved both analytically and numerically. A simple problem is Poisson s equation in one dimension, with exact solution: u j = sin jkπ n which will be our model problem. The parameter k affects how oscillatory the solution is. When k is small the solution is smooth and when k is large it is more oscillatory. The graph below shows the exact solution of the problem when k = 1,3 and 6: u k=1 k= x k=

9 3.0 Numerical Methods for the Model Problem 3.1 Finite Difference Method The finite difference method (FDM) is a method for numerically solving PDE s. As we have already seen in section the model PDE involves an unknown function u(x) defined for all x in the domain. We can then use the FDM to determine an approximation for u(x). We can see from the graph how we would divide the curve (function of x) into two sectors with width h. u(x) C A B u(x - h) u(x) u(x + h) x h x x + h x When a function u and its derivatives are single-valued, finite and continuous functions of x, then by Taylor s theorem (Smith, 1978), u(x + h) = u(x) + hu (x) + ½h²u (x) + 1/6h 3 u (x) +. (Equation 3.1.1) and also: u(x - h) = u(x) - hu (x) + ½h²u (x) - 1/6h 3 u (x) +. (Equation 3.1.) We can then add the expansions to get u(x + h) + u(x - h) = u(x) + h²u (x) + O(h 4 ) (Equation 3.1.3) where O(h 4 ) represents all the higher order terms. We now assume that this is very small in comparison to the lower powers of h, therefore it follows that, d u u (x) = dx 1 h {u(x + h) u(x) + u(x h)} (Equation 3.1.4) This technique can be extended to approximate derivatives of different order and in higher dimensions (Smith, 1978).

10 We consider the one dimensional Poisson s equation, d u dx = f(x) defined for x [0,1]. The range of x is split into n+1 equally spaced points. The second derivative can be approximated at point i using equation to give u j-1 u j + u j+1 = h². f j (Equation 3.1.5) where we define f j = f(x = x j = jh) and j is the point number. We are assuming Dirichlet boundary conditions for which u 1 and u n+1 are known. If the boundary conditions are in the form u(x) = u 1 at x= x 1 and u(x) = u n+1 at x = x n + 1. Then since we know u 1 and u n+1 we can write equation as (Briggs, 1987), -u + u 3 = h². f u 1, at point 1 u n-1 u n = h². f n u n+1, at point n+1 We can see that when the equations are written like this, that all the unknowns are on the left hand side and the known on the right hand side. This allows us to put equation in matrix form, which can be seen below. This is in the form of a simple matrix equation, Au = f - 1 u h². f u u 3 h². f 3 = 1-1 u i h². f i 1-1 u n-1 h². f n u n h². f n u n + 1 in which A is tridiagonal. This system of equations must be solved to give our numerical solution for u(x). 3. Direct Method This method is called the direct method because it calculates the numerical solution of the difference equations in one step. When equation is put into the form of the matrix equation (as described at the end of the last section) it can be solved using several methods (Van Loan, 1997), however we will be using a numerical function in the program Matlab to solve equations of this kind quickly and efficiently. 3.3 Iterative Method

11 An iterative method is an alternative numerical technique used to solve a linear system of equations. An initial guess is used to approximate the solution. The idea behind an iterative method is that we use the initial guess in the first iteration and then get an improved solution. As we iterate further the approximation should ideally converge to the exact solution. Sections investigate three common iterative methods. 3.4 Jacobi Method Also known as simultaneous displacement the Jacobi method is formed by solving the j th equation of for the j th unknown u j. This results in an iteration scheme which may be written in component form as (Briggs, 1987), v (1) = v (0) + (h /)*r, where r j = (v j-1 v j + v j-1 )/(h²) ) + f j j n Before the first iteration we need v (0), which is known as an initial guess. This can be a vector with any numbers in it that we choose, the null vector is often used. v (1) can then be calculated and that effectively becomes v (0) for the next iteration. The iterations carry on until v (1) has converged to the solution. 3.5 Weighted Jacobi Method The weighted Jacobi method is similar to the Jacobi method except a damping factor is included. A damping factor means that the new approximation is found by adding a proportion of the residual to the old solution. It can be shown that the optimum value is /3 (Briggs, 000), when we work on more than one grid level. The Jacobi method (3.4) has a ratio of one. The weighted Jacobi method has the following iteration scheme which may be written in component form as: v (1) = v (0) + weight*(h /)*r, where r j = (v j-1 v j + v j-1 )/(h²) ) + f j j n 3.6 Gauss-Seidel Method The Gauss-Seidel method is very similar to the Jacobi method apart from the equations of Jacobi are used one at a time in sequence and elements that are in vector v are overwritten with the new elements as soon as they have been calculated. This results in a method which may be expressed in component form as (Briggs, 000), 1 v j ( v j 1 + v j+ 1 + h² f j ), j n

12 where the arrow notation stands for replacement or overwriting. In the Jacobi method updates can be done all at once but in the Gauss- Seidel method each component of the new iterate depends on components previously calculated. 3.7 Error Equation and Residual (Briggs, 000) If we take the exact equation Au = f, and define the residual, r, of the approximate solution v, r = f Av and also define the error, e = u v we can deduce: r = Au Av = A(u v) i.e Ae = r This equation can now be approximately solved using a relaxation method, e.g. Jacobi or Gauss- Seidel method. Before this equation can be solved we first need to calculate the residual. The residual is calculated using the following formula in component form: r j = (v j-1 v j + v j-1 )/(h²) ) + f j j n 3.8 Two-Grid Correction Scheme The problem with simpler iterative methods like the Jacobi method is that they tend to damp out high frequency components of error fastest but are less effective at damping low frequency errors. This reason has led people to develop methods based on the following heuristic (Drakos, 1993): 1) Perform some steps of a basic method in order to smooth out the error ) Restrict the current state of the problem to a subset of the grid points, the so-called coarse grid, and solve the resulting problem. 3) Interpolate the coarse grid solution back to the original grid, and perform a number of steps of the basic method again. The terms restrict and interpolate represent a transfer of information between grids, and are defined in section 3.9. Step 1 is called pre-smoothing and step 3 is called post-smoothing. The technique above is known as the two-grid correction scheme as only two grids are used; a fine grid Ω h and a coarse grid Ω h. If we start on a fine grid we can relax a few times with an initial guess v (0). We can then calculate the fine grid residual r h = f h A h v h and restrict it to Ω h. The equation A h e h = r h can be solved on Ω h to give us an

13 estimate of the error. We can then interpolate the coarse-grid error to Ω h and update the fine grid approximation by v h v h + e h. A number of relaxations can be done with initial guess v h to give us an improved solution. The same procedure can then be repeated until the necessary convergence is met. 3.9 Restriction and Interpolation (Briggs, 000) Restriction transforms points on Ω h to Ω h. Restriction will be denoted by h h h I h v = v, where we will be using the most obvious type of restriction, injection, where h h v j v j =, 0 j (n-1)/. An example of restriction can be seen below: Ω h to Ω h Interpolation transforms points on Ω h to Ω h. There are different ways of interpolation but we will use a technique called linear interpolation that is quite effective. The linear interpolation operator will be denoted h h I hv = h v, where interpolation can be seen below: v h h j = v j, and 1 h h ( h v j+ 1 v j + v j+ 1) =, 0 j n/ 1. An example of linear Ω h to Ω h 3.10 Multigrid Method V-Cycle Scheme The V-Cycle is the first true Multigrid method we have discussed. If we go back to section 3.8 and apply the method recursively to step it becomes a Multigrid method. We start at a fine grid and work down in stages to the coarsest grid. At each stage we calculate the residual and interpolate the residual to next grid down, calculate the error and then repeat the process until we reach the coarsest grid. We can then work our way back up from the coarsest grid correcting the solution and relaxing until we get back to the fine grid we started with. One important element that should be considered is that we will need an initial guess when solving the residual equations when working down to the coarsest grid. As we have no information about v, which represents the error, at these points we use an initial guess of 0. A recursive definition of the V-Cycle Scheme (Briggs, 000) is described below:

14 Vcycle(v h,f h ): 1. Relax n1 times on A h u h = f h with a given initial guess v h.. If Ω h = coarsest grid, then go to step 4. Else F h h h h I ( f A v h ), h v h 0, v h Vcycle h (v h,f h ). 3. Correct v h v h h h + I v. h 4. Relax n times on A h u h = f h with initial guess v h. The V-Cycle is just one class of (Briggs,1987). Multigrid scheme; another cycle scheme widely used is the W-cycle 3.11 Good Initial Guess A good way of improving a relaxation scheme is to have a good initial guess (define the initial guess as v0). A way of obtaining v0 is to start on a coarse grid and solve the equation A u = f using an iterative method such as the Jacobi method with a small number of iterations. We can then interpolate the solution v to a finer grid and solve using the same iterative method. We can repeat this process until we have an approximate solution on the required fine grid.

15 4.0 Implementation 4.1 Direct Method The direct method was implemented as a function. The function can be seen in the appendix section C.1. In section 3. we showed how it was possible to use the finite difference method to arrange the equations of the model problem so that all unknowns are on the left hand side and the known on the right hand side. This then gave us an equation in the form A u = f. The matrix, A, can be constructed within the function, but the right hand side (vector f) is passed into the function as a parameter. Vector u is calculated using an operator which solves equation of the form Au = f. The function is called by: direct(f), this will return u, which is correct to machine precision. 4. Jacobi Method The Jacobi method was implemented as a function. The function can be seen in the appendix section C.. In section 3.4 we deduced the iterative formula: v (1) = v (0) + (h /)*r, where r j = (v j-1 v j + v j-1 )/(h²) ) + f j j n Three parameters are passed into the function: i) v0, which is an initial guess at the solution u. ii) nt, the number of Jacobi iterations we wish to do. iii) f, the right hand side of the equation Au = f. The function is called by: jac(v0, nt, f), this will return approximation v. 4.3 Weighted Jacobi Method The implementation of the weighted Jacobi method is very trivial since we already have a function for the Jacobi method. We just need to multiply the residual by a damping factor, which has been named omega. Omega is presently set to what has been proved as the optimal value in Briggs, of /3. The updated Jacobi function can be seen in the appendix in section C.3. The parameters are the same as for the Jacobi method.

16 4.4 Gauss-Seidel Method The Gauss-Seidel method was implemented as a function. The function can be seen in the appendix section C.4. In section 3.6 we deduced the iterative formula: 1 v j ( v j 1 + v j+ 1 + h² f j ) j n Three parameters are passed into the function: i) v0, which is an initial guess at the solution u. ii) nt, the number of Gauss-Seidel iterations we wish to do. iii) f, the right hand side of the equation Au = f. The function is called by: gauss(v0, nt, f), this will return approximation v. 4.5 Restriction and Interpolation Both the restriction and interpolation operators have been implemented as functions; the code can be seen in the appendix section s C.5 and C.6. The restriction function is called coarse, and the interpolation function is called fine. Each function takes parameter v and transfers the points on that grid to the specified coarser or finer grid. Restriction function is called by: course(v), which will restrict v to a courser grid. Interpolation function is called by: fine(v), which will interpolate v to a finer grid. 4.6 Calculating the Residual The residual is calculated in a function that has input variables, appendix section C.7. The residual, r is returned by the function; r = res(v, f) v and f. The function can be seen in the 4.7 Multi-Structure Before we can implement the two-grid correction scheme we need to design an efficient way of storing the Multigrid information. As we will be moving between grids we need to store v, f, r at each grid level. We can use a structure that has been named multi_struc (see appendix section C.8) to hold the information. This function needs the parameter ns, which stands for the number of levels we require. i.e. if n = 64 at the top level then ns = 6

17 Level 6 64 intervals Level 5 3 intervals Level 4 16 intervals Level 3 8 intervals Level 4 intervals Level 1 intervals The function will calculate the number of points required at each level and create v, f, and r at each level, initialising each parameter to the correct sized zero vector. E.g. ms(k).v will access the vector v on grid k. If we wanted to create a structure for when n = 64 and store the initial guess, we would type the following code: ms = multi_struc(6); ms(6).v = v0, where v0 is the initial guess. 4.8 Two-Grid Correction Scheme The two-grid correction scheme is the first program that will collectively use the functions that have been implemented in this section so far. The code can be seen in the appendix, section C.9. Below is a brief description of the two-grid method in terms of the functions we have created: 1. Create a structure using the function multi_struc. Although we will only be using two levels storage space (the finest grid and the level below) all levels down to the coarsest grid of two intervals will be created, as we will be using this same function for the Multigrid method.. Perform some relaxations on the finest grid using the jac function. Usually 1, or 3 iterations are used and we will explore what the optimum value for this is in chapter 5). 3. Calculate the residual of the finest grid using the res function 4. Restrict the residual to the course grid using the course function. 5. Perform some relaxations using the jac function to solve A h e h = r h on the coarser grid to give us an estimate of the error. 6. Interpolate the error to the finer grid using the fine function. Then add the error to the current solution get an improved solution. 7. Finally, perform further relaxation sweeps using the jac function. We can repeat steps 6 so that we are effectively doing more than one cycle of the scheme. In the code I have included an error counter, which calculates the error in the approximation after each cycle. We can then graph this array to show how the approximation improves/converges after each cycle.

18 4.9 V-Cycle Scheme The V-Cycle is constructed similarly to that described in section 4.8. The program can be seen in the appendix in section C.10. The main difference is that steps 4 (from the scheme in 4.8) are repeated until the coarsest grid has been reached. The coarsest grid is not necessarily the bottom level grid; there is a variable in the program named cg that determines the smallest level. Steps 5-6 are then repeated until we reach the top-level grid. We can then perform some final relaxations using the Jac function. Like the twogrid method we can repeat the cycle described above in order for the approximation to converge to the exact solution. There is also an error counter that stores the error in the approximation after each cycle. It is then possible to plot the error so that we have a graphical representation of how it decreases after each cycle Initial Guess The method described in section 3.11 describes how we find a solution to the equation A u = f on a coarse grid and then interpolate the solution to a finer grid. We can repeat this process on the current fine grid until we have a solution on the required grid size. Like the two-grid scheme and the multi-grid method we will need to create a structure that can hold the values of v and f at each grid level. The structure is named make_struc (see appendix section C.11), which calculates n at the coarsest level and initialises v and f with the zero vectors of length n. The process is repeated for the next level up and keeps repeating until we have initialised the finest grid. We now can use a function named initial_guess (see appendix section C.1) that uses the structure to formulate an initial guess. The function requires a structure (s), the level of the finest grid ( ns), the number of iterations to be performed at each level (nt), the solution parameter k, for initiating f on each grid (k) and the level of the coarsest grid ( icg). The function will return a vector v0, which is the initial guess. An example of how we would create a structure and calculate an initial guess is as follows: s = make_struc(ns) v0 = initial_guess(s, ns, nt, k, icg)

19 4.11 Work Units We have implemented four iterative methods for solving the equation Au = f: 1. Jacobi Method. Gauss-Seidel Method 3. Two-Grid Correction Scheme 4. V-Cycle Multigrid Method A good way to compare how well the methods work is to measure the amount of work the computer has to do to get a good approximation to the exact solution for each of the methods. We can define: 1 work unit = 1 iteration of a relaxation method on the finest grid restriction from the finest grid calculating the residual on the finest grid ½ work unit = 1 iteration of a relaxation method on the grid below the finest grid interpolation from the grid below the finest grid restriction from the grid below the finest grid calculating the residual on the grid below the finest grid. We can use the same concept to devise work units for functions on lower grid levels than the ones described above. The function named multi_struc is now updated to include two more variables, work and error. We can calculate the error and how much work has been done after each function call and store the amounts in the work and error arrays. When the program is complete we can plot work versus error to show how the error decreases as the more work units we do. This allows us to compare the different methods by how much work they have to done to obtain a good approximation. We can also find optimum values for variables such as the coarsest grid (cg) and number of relaxations (nt) in the Multigrid method. When the initial guess is included in a program we need to make sure we calculate the amount of work done to get the initial guess. This is done in a little loop that calculates the work done on the finest level, work done on the next grid down,., work done on the coarsest grid. The error in the approximation is also calculated and these two values are the first values to be stored in the work and error arrays. I have written three programs that all calculate an initial guess and plot the error against the work done for each of the following three methods: Jacobi Method, Two-Grid Correction Scheme and the V-Cycle. These can be seen in the appendix section C.13, C14 and C15 respectively.

20 5.0 Results We are now ready to analyse the methods we have implemented. All the methods are set up to solve the model problem. Within each program there are lots of variables that need to be considered in this section. We will try and find optimum values for the variables that affect the performance for this particular problem. However we need to remember that these values may not be the optimum values when solving another problem. It is sensible to fix some of the variables when comparing the methods for continuity purposes. A sensible number of intervals on the finest grid is 64, as this is not to large that computational time will be extreme and it is not to small that the problem is easy to solve. We also need to compare how the methods can solve both smooth and oscillatory curves. The variable k effects the period so for each method we will set k to three values: 1 (representing a smooth solution), 3 and 6 (representing an oscillatory solution). An important point we should consider now is the error in the approximation. The model problem, d u dx = f has the exact solution u(x). Our methods are derived from the Finite Difference Method that has exact solution u, satisfying Au = f with approximation u j, j = 1,.., n The FDM has an error of O(h ) from the exact solution. i.e. u(x j ) u j = 0(h ) In section 3. we derived the Direct Method that solves the FDM to machine precision. So we can measure the error by taking a norm of the approximate solution of FDM method (e.g. Jacobi, Multigrid, etc) minus the exact solution of the FDM method (Direct Method). In a series of experiments we will evaluate the methods we have implemented in section 4.

21 5.1 Jacobi Method Experiment 5.1A: How does the Jacobi method behave? Fig K = 3 K = 1 Variables set to: K = 6 n = 64 nt = 00 k = 1,3 and 6 Inference: We can see that when the curve is very smooth i.e. k = 1, the Jacobi method s convergence is very poor indeed. When k = 3 the approximation is converging slowly and when k = 6 (the solution is relatively oscillatory), the approximation is converging more rapidly. The problem with the Jacobi method is that it tends to damp out high frequency components of error fastest but is less effective at damping low frequency errors. 5. Two-Grid Correction Scheme Experiment 5.A: How does the two-grid correction scheme behave compared to the Jacobi method? Fig 5..1 (k = 1) Fig 5.. (k = 3) Jacobi Jacobi Two- Grid Two- Grid

22 Fig 5..3 (k = 6) Variables Set to: n = 64 ns = 6 Jacobi Iterations = 80 nt = 3 cycles = 40 Inference: The two-grid correction scheme converges at a faster speed than the Jacobi method in all three cases. The approximation converges to machine precision for the more oscillatory case after only 0 work units. When k=1 the approximation is not very good, after 80 work units the norm of the error is approximately 0.5 which is not very accurate at all. There is significant improvement when k=3 as the norm of the error is correct to the exact solution to the nearest one thousandth where as with the Jacobi method it was only correct to within one tenth. Experiment 5.B: How does the two-grid correction scheme behave when we use the weighted function? Key Two-Grid with Jacobi Two-Grid with weighted Jacobi Fig 5..4 Jacobi Variables set to: n = 64 nt = 80 k = 1,3 and 6 Inference: It is clear that the weighted Jacobi method is not as effective as the Jacobi method when we are working on two grids.

23 Experiment 5.C: How does the two-grid correction scheme behave when we change the number of relaxations from 3? Fig 5..5 (k = 1) Fig 5..7 (k = 6) Variables Set to: n = 64 ns = 6 nt = 1,, 3, 4, 5 and 6 cycles = 70, 51, 40, 33, 8 and 4 Key n t = 1 n t = n t = 3 n t = 4 n t = 5 n t = 6 Inference: For all three values of k either 3 or 5 iterations is the optimum number of relaxations. We have used three relaxations in the other two-grid scheme experiments so far, however when k = 1 and 3 five relaxations produces the best results. When k = 6 three relaxations is the best. 5.3 V-Cycle Scheme Experiment 5.3A: How does the V-Cycle scheme behave compared to the Correction Scheme? Jacobi method and Two-grid Key to Diagrams Jacobi Method V-Cycle Scheme

24 Fig (k = 1)