Suppose we want to find approximate values for the solution of the differential equation

Transcription

1 A course in differential equations Chapter 2. Numerical methods of solution We shall discuss the simple numerical method suggested in the last chapter, and explain several more sophisticated ones. We shall also deal with the problem of estimating error in the calculations. We make no attempt to produce the best routines possible, but just to present techniques which are reasonably simple to understand as well as reasonably effective in most circumstances. 1. Euler s method Suppose we want to find approximate values for the solution of the differential equation y 0 = f (t; y) with initial condition y(t 0 )=y 0 : The differential equation tells us that the instantaneous rate of change of y at time t can be calculated in terms of y and t alone. Now if t is any small interval of time we know that as a first order approximation we can write y(t +t) : =y(t)+ty 0 (t) in the sense that the difference between the right and left sides, for a fixed value of t, is essentially of order (t) 2. If y is a solution of the differential equation, this tells us that If t 1 = t 0 +twe know therefore that y(t +t) : =y(t)+tf(t; y(t)) : y(t 1 ) : = y 1 = y(t 0 )+tf(t 0 ;y(t 0 )) = y 0 +tf(t 0 ;y 0 ): We can apply this reasoning once again. If t 2 = t 1 +t=t 0 +2twe obtain a further estimate y(t 2 ) : = y 2 = y 1 +tf(t 1 ;y 1 ): Etc. If t n+1 = t n +t=t 0 +(n+ 1)t we get an estimate y n+1 for y(t n+1 ) from the estimate y n for y(t n ): y n+1 = y n +tf(t n ;y n ): This technique for finding approximate values for the solution of a first order differential equation is the simplest of several similar ones. Each of them proceeds in this stepwise fashion, obtaining an estimate for y(t +t)from that for y(t). This one is called Euler s method. It has some great virtues: It is simple to carry out. Doing it by hand is not impossible (although tedious and hence error prone). It can be easily implemented on a computer, for example with a spread sheet. The step from one estimate to the next is intuitive. 2. Euler s method and slope fields Euler s method has a simple interpretation in terms of slope fields. In the figure below we are looking at the differential equation & initial condition y 0 = y t; y(0) = 0:5 :

2 Numerical methods of solution 2 This is a linear equation and has the explicit solution y =(t+1) 0:5e t : The differential equation tells us how to draw the slope field and the initial condition tells us where the graph starts. y =1 y=0 At the starting point we know that the slope of the graph is equal to f (0; 0:5) = 0:5. This means that the tangent line to the graph at the starting point is the line y (t) =0:5+0:5t. This tangent line and the graph of y(t) will lie very close to each other for small values of t, and we can then use y as an estimate for y(t) for those values of t. After a while, however, it will cease to be a useful approximation. How can we get a better one? We pick some interval t and at t =twe modify the tangent line. We must modify it by using only information available to us in the calculation so far, and in view of this it seems reasonable to use as the new approximation the straight line through (t; y (t)) whose slope agrees with the slope field at that point. y =1 y=0 We then make new breaks in the approximation after every interval of size t. As we take smaller and smaller values of t we get better approximations to the graph of the solution. 3. The error in Euler s method

3 Numerical methods of solution 3 We expect that the error in the approximation we get from Euler s method decreases as we let the step size t get smaller. We can get a good idea of how the error depends on the choice of t by plotting in one picture the approximations we get for different values of t. Here we let t =1, 1=2, 1=4, etc. again with the differential equation y 0 = y t; y(0) = 0:5 : y =1 y=0 It looks very much as though we have the error as we halve the step size. We can verify this in more detail by comparing estimates for y(1) with the true value which we know to be (t +1) 0:5e t at t =1which is equal to 2 0:5 e = 0: N estimated y(1) true y(1) error 2 0: : : : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : : : This example leads to a guess which turns out to be true: The error in Euler s method for estimating y(t) with a given differential equation and initial condition of y at a fixed value of t is proportional to the step size chosen. The effect of this is to make Euler s method extremely inefficient for serious calculation. It requires an enormous amount of calculation to achieve reasonable accuracy. Very roughly speaking, the amount of work it takes to achieve a certain level of accuracy is proportional to the accuracy you want. Thus if it takes N steps to get 1 digit of accuracy (answer within 1=10), it will take 10 N to get an extra valid digit (within 1=100), 100 N to get 3 valid digits, :::, and N to get 6. The reason for the way error and step size interact for Euler s method is not hard to understand at least informally. Euler s method uses the estimate y(t +t) : =y(t)+ty 0 (t)+ terms of order (t) 2 :

4 Numerical methods of solution 4 That means that the error in making each step is essentially of order (t) 2. But for a fixed interval across which we want to calculate, the number of steps necessary is proportional to the inverse of t, which makes it plausible that the overall error is of order (1=t)(t) 2 =t. 4. Illustrating errors graphically We can use this to estimate the error in Euler s method even when we don t know the exact answer! For example, suppose we look at the equation y 0 = xy +1; y(0) = 1 This is a linear equation, but if we were to use the usual formula for the solution we would see it produces an integral we cannot find explicitly. Here is the estimate we get for y(1) with various values of N: N y(1) 4 2: : : : : : : : : And here is a graph of the pairs (;y(1)). 3:5 3:0 y =2:5! Figure 4.1. A plot of y(1) versus step size. What we can see from this plot is that the points (;y(1)) are to a good approximation on a straight line. This is in complete agreement with the principle I stated above, that the error is proportional to. But we can also see that to some extent we can predict what the end of the line is. This is because we can estimate from each successive pair what the slope of the purported straight line is it ought to be about y N y N 1 N N 1 for each N. Of course this is only an approximation, so we don t expect any of these estimates to be exact. Here is what we get in the runs above:

5 Numerical methods of solution 5 N N =1=N y N (1) y N (1) y N 1 (1) estimated slope 4 0: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : The rough proportionality certainly shows up here, and the last column tells us roughly what the constant of proportion is. One run of Euler s method gives you no idea of the error involved. Two runs with values y 1 and y 2 for step sizes 1 and 2 give you the estimate y1 y for the error with step size. 5. How errors propagate The exact way in which errors propagate as we carry out Euler s method can be rather complicated. But a fairly simple argument can lead to some idea of the worst things can go, at least under mild conditions on the equation. Suppose y(t) is the exact solution to y 0 = f (t; y); y(t 0 )=y 0 : and let y n be the estimate for y(t n ) obtained by Euler s method, where t n = t 0 + nt. Let " n = y(t n ) the error in the method, the difference between the true and the estimated methods. Then y n be " n+1 = y(t n+1 ) y n+1 = y(t n+1 ) [y n +tf(t n ;y n )] =[y(t n ) y n ]+[y(t n+1 ) y(t n ) t f (t n ;y(t n ))]+t[f(t n ;y(t n )) f (t n ;y n )] = " n +[y(t n+1 ) y(t n ) t f (t n ;y(t n ))]+t[f(t n ;y(t n )) f (t n ;y n )] : If we assume that jy 00 j is bounded in the range we are looking at, say by A, then we have The quantity will be bounded by if in the range we are looking at and then we have jy(t n+1 ) y(t n ) t f (t n ;y(t n ))j A(t) 2 : jf (t n ;y(t n )) f (t n ;y n )j B jy(t n ) y n j = B" " n+1 " n + B t" n +A(t) 2 = " n (1 + B t)+a(t) 2 :

6 Numerical methods of solution 6 this leads to " 1 A(t) 2 " 2 (1 + B t)a(t) 2 + A(t) 2 " 3 (1 + B t) 2 A(t) 2 +(1+Bt)A(t) 2 + A(t) 2 ::: " n A(t) 2 [(1 + B t) n 1 + +(1+Bt) 2 +(1+Bt)+1] =A(t) 2 (1 + B t) n (1 + B t) 1 1 = A B t [(1 + B t)n 1] = A B t [ebnt 1] = A B t [eb(t n t 0 ) 1] if t n = t 0 + nt, since (1 + x) e x for all x 0. In other words, we can see that Under very mild restrictions, in Euler s method the error for a fixed t is at worst proportional to t and the constant of proportionality grows at worst exponentially with t. This is really a worst case scenario under the restrictions necessary. Most of the time the constant will behave much better. But for equations whose solutions grow exponentially the estimate will be reasonably close to what happens. The restrictions are mild, but there are cases where they don t hold. For the equation y 0 = y 2 the conditions are not satisfied for the solutions, which all go off to infinity in bounded time. 6. A better method Euler s method is quite inefficient. Because the error is essentially proportional to the step size, if we want to halve our error we must double the number of steps taken across the same interval. So if N steps give us one significant figure, then 10 N give us 2 figures, 100 N give us 3, N give us 6, and 10 n 1 N give us n figures. Not too good, because even a fast computer will think a bit about doing several billion operations. The method can be improved dramatically by a simple modification. Look again at what goes on in one step of Euler s method. Say we are at step n. We know t n and y n and want to calculate t n+1, y n+1. In Euler s method we assuming that the slope across the interval [t n ;t n +t]=[t n ;t n+1 ] is what it is at t n. Of course it is not true, but it is very roughly OK.

7 Numerical methods of solution 7 y =1 y=0 After we have taken one step of Euler s method, we can tell to what extent the assumption on the slope was false, by comparing the slope of the segment coming from the left at (t n+1 ;y n+1 ) with the value f (T n+1 ;y n+1 ) that it ought now to be. The new method will use this idea to get a better estimate for the slope across the interval [t n ;t n+1 ]. One step of the new method goes like this: Start at t n with the estimate y n. Do one step of Euler s method to get an approximate value y = y n +F(t n ;y n ) for y n+1. Calculate what the slope should now be at the point (t n+1 ;y ). It should be f (x n+1 ;y ) We now make a modified guess as to what the slope across [t n ;t n+1 ] should have been by taking the average of these values f (t n ;y n )and f (t n+1 ;y ). In other words In one step of the new method we calculate the sequence This is in contrast with one step of Euler s: s 0 = F (x n ;y n ) y =y n +s 0 x n+1 = x n + s 1 =F(x n+1 ;y ) y n+1 = y n +h s0 +s 1 2 s 0 = F (x n ;y n ) y n+1 = y n +s 0 x n+1 = x n + In the new method, we are doing essentially twice as much work in one step, but it should be a somewhat more accurate. It is, and dramatically so. Let s see, for example, how the new method, which is called the improved Euler s method, does with the simple problem y 0 = y t; y(0) = 0:5 we looked at before. Here is how one run of the new method goes in covering the interval [0; 1] in 4 steps: i

8 Numerical methods of solution 8 t y s 0 y s 1 0:00 0: : : : :25 0: : : : :50 0: : : : :75 0: : : : :00 0: And here is how the estimates for y(1) compare for different step sizes: N estimated y(1) true y(1) error 2 0: : : : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : : : Halving the step size here cuts down the error by a factor of 4. This suggests: The error in the improved Euler s method is proportional to 2. The new method, which is called either the improved Euler s method or the Runge-Kutta method of order 2, is much more efficient than Euler s. If it takes N intervals to get 2 decimal accuracy, it will take 10 N to get 4 decimals, 100 N to get 6 (as opposed to N in Euler s method). Very roughly, to get accuracy of " it takes 1= p " steps. 7. Relations with numerical calculation of integrals If y is a solution of y 0 (t) =f(t; y(t)) then for any interval [t n ;t n+1 ] with t n+1 = t n +twe can integrate to get y(t n+1 ) y(t n )= Z tn+1 tn f (t; y(t)) dt : This does not amount to a formula for y(t n+1 ) in terms of y(t n ) because the right hand side involves the unknown function y(t) throughout the interval [t n ;t n+1 ]. In order to get from it an estimate for y(t n+1 ) we must use some sort of approximation for the integral. In Euler s method we set Z tn+1 tn f (t; y(t)) dt : = f (t n ;y(t n )) t : We then replace y(t n ) by its approximate value y n to see how one step of Euler s method goes. In the special case where f (t; y) =f(t)does not depend on y, the solution of the differential equation & initial condition y 0 = f (t); y(t 0 )=y 0 is given by a definite integral y(t n )=y 0 + Z tn t 0 f(t)dt

9 Numerical methods of solution 9 and Euler s method is equivalent to the rectangle rule for estimating the integral numerically. Recall that the rectangle rule approximates the graph of f (t) by a sequence of horizontal lines, each one agreeing with the value of f (t) at its left end. In this special case it is simple to see geometrically why the overall error is roughly proportional to the step size. In the improved Euler s method we approximate the integral Z tn+1 by the trapezoid rule, which uses the estimate t tn f (t; y(t)) dt 2 [f (t n;y(t n )) + f (t n+1 ;y(t n+1 ))] for it. We don t know the exact values of y(t n ) or y(t n+1 ), but we approximate them by the values y n and y n +tf(t n ;y n ). In the special case f (t; y) =f(t)the improved Euler s method reduces exactly to the trapezoid rule. Again, it is easy to visualize why the error over a fixed interval is proportional to the square of the step size. 8. Runge-Kutta Neither of the two methods discussed so far is used much in practice. The simplest method which is in fact practical is one called Runge-Kutta method of order 4. It involves a more complicated sequence of calculations in each step. Let be the step size. We calculate in succession:

10 Numerical methods of solution 10 y = y n +h i s 2 x = x +h i 2 s = f (x ;y ) y = y n +h i s 2 s = f (x ;y ) x n+1 = x + y = y n +s s = f (x n+1 ;y ) y n+1 = y n +h s +2s +2s + s 6 The purpose of using here is that the numbers like y etc. are not of interest beyond the immediate calculation they are involved in. You wouldn t want to do this by hand, but again it is not too bad for a spread sheet. Here is a table of the calculations for y 0 = y t; y(0) = 0:5 with 4 steps. t y s y s y s y s 0:00 0: : : : : : : : :25 0: : : : : : : : :50 0: : : : : : : : :75 0: : : : : : : : :00 0: And here is how the estimates for y(1) compare for different step sizes. The method is so accurate that we must use nearly all the digits of accuracy possible: N estimated y(1) true y(1) error 2 0: : : : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : ::: 0: : : : Here halving the step size reduces the error by a factor of 16. This suggests: The error in RK4 is proportional to 4. i

11 Numerical methods of solution 11 Euler s method and the improved Euler s method arise from using the rectangle rule and the trapezoid rule for estimating the integral Z tn+1 tn f (t; y(t)) dt : RK4 uses a variant of Simpson s Rule for calculating integrals and reduces to it in the special case when f (t; y) doesn t depend on y. 9. Remarks on how to do the calculations When implementing these by hand, which you might have to do every now and then, you should set the calculations out neatly in tables. Here is what it might look like for the improved Euler s method applied to the differential equation y 0 =2xy 1 with step size =0:1. etc. x y s =f(x; y) y = y +s s = f (x +;y ) 0: : : : : : : : : : : : : : : Of course this is redundant and tedious work, and ideally suited to a computer. You can transfer the idea of tables to a spread sheet without much trouble. And here is a simple Pascal program to implement Euler s method for the same differential equation. program euler; var x, y, xf, h: real; i, N : integer; function f(x, y:real): real; begin f := 2*x*y - 1; end; function next_y(x, y, h: real):real; begin next_y := y + h*f(x, y); end; begin x := 0.0; xf := 1.0; y := 1.0; N := 10; h := (xf - x)/n; for i := 1 to N do begin writeln(x:8:6,, y:8:6); y := next_y(x, y, h); x:=x+h; end; writeln(x:8:6,, y:8:6);

12 Numerical methods of solution 12 end. This is very convenient, because to change the differential equation involves very little change in the program (but it does involve recompiling). 10. How to estimate the error When you set out to approximate the solution of a differential equation y 0 = f (x; y) by a numerical method, you are given initial conditions y(x 0 )=y 0 and a range of values [x 0 ;x f ] (f for final ) over which you want to approximate the solution. The particular numerical method you are going to use is usually specified in advance, and your major choice is then to choose the step size the increment by which x changes in each step of the method. The basic idea is that if you choose a small step size your approximation will be closer to the true solution, but at the cost of a larger number of steps. What is the exact relationship between step size and accuracy? We have discussed this already to some extent, but now we shall look at the question again. Some explicit data can give you a feel for how things go. Here is the outcome of several runs of Euler s method for solving the equation over the interval [0; 1]. y 0 =2xy 1; y(0) = 1 Number of steps Step size Estimated value of y(1) This table suggests that if we keep making the step size smaller, then the approximations to y(1) will converge to something, but does not give a good idea, for example, of how many steps we would need to take if we wanted an answer correct to 4 decimals. To get this we need some idea of how rapidly the estimates are converging. The way to do this most easily is to add a column to record the difference between an estimate and the previous one. Number of steps Step size Estimated value of y(1) Difference from previous estimate

13 Numerical methods of solution 13 The pattern in the last column is simple. Eventually, the difference gets cut in half at each line. This means that at least when h is small, the difference is roughly proportional to the step size. Thus we can extrapolate entries in the last column to be 0:000674; 0:000337; 0:000169; 0:000084; :::: But the total error should be the sum of all these differences, which can be written as 0: (1=2+1=4+1=8+1=16 + :::)=0:001348! It is not in fact necessary to look at a whole sequence of runs. We can summarize the discussion more simply: if we make two runs of Euler s method, the error in the second of the two estimates for y(x f ) will be (roughly) the size of the difference between the two estimates. The reason the error can be estimated in this way is because the error in Euler s method is proportional to the step size. Suppose we have made two runs of the method, one with step size h and another with step size h=2. Then because of the proportionality, and the meaning of error, we know that true answer : = estimate h + Ch true answer : = estimate h=2 + C(h=2) for some constant C. Here estimate h means the estimate corresponding to step size h. If we treat the approximate equalities as equalities, we get two equations in the two unknowns true answer and C. We can solve them to get true answer : =2estimate h=2 estimate h = estimate h=2 +(estimate h=2 estimate h ) error in estimate h=2 : = estimate h=2 estimate h In other words, one run of Euler s method will tell us virtually nothing about how accurate the run is, but two runs will allow us to get an idea of the error and an improved estimate for the value of y(x f ) as well. The same is true for any of the other methods, since we know that in each case the error is roughly proportional to some power of h for improved Euler s it is h 2 and for RK4 it is h 4. In general, if the error is proportional to h k and we make two runs of step size h and h=2 we get and subtract getting true answer : = estimate h + Ch k true answer : = estimate h=2 + C(h=2) k 2 k true answer : =2 k estimate h=2 + Ch k and finally: (2 k 1) true answer : =2 k estimate h=2 estimate h true answer = : 2k estimate h=2 estimate h 2 k 1 = estimate h=2 + estimate h=2 estimate h 2 k 1 If you do one run with step size h and another of size h=2 with a method of order k then error in estimate h=2 : = estimate h=2 estimate h 2 k 1 true answer : = estimate h=2 + error in estimate h=2

14 Numerical methods of solution 14 The earlier formula is the special case of this with k = Step size choice When we use one of the numerical methods to approximate the solution of the differential equation, we usually have ahead of time some idea of how accurate we want the approximation to be to within 6 or 7 decimals of accuracy, say. The question is, how can we choose the step size in order to obtain this accuracy? The discussion above suggests the following procedure: Make two runs of the method over the given range, with step size h and then with step size h=2. We get two estimates for y(x f ), estimate h and estimate h=2. If the method has error proportional to h k (i.e. if the error is of order k) then we know that the error in estimate h is approximately error in estimate h : = estimate h estimate h=2 1 2 k : If we multiply the step size by we multiply the error by k, so if we want an error of about : Solve this equation to get the new step size h : h k = h error in estimate h h = h error in estimate h 1=k The procedure looks tedious, but plausible: we choose some initial step size h which we judge (arbitrarily) to be reasonable. We do two runs at step sizes h and h=2 to see how accuracy depends on step size (to get the constant of proportionality). We then calculate the step size h needed to get the accuracy we want, and do a third run at the new size. Example. Here is a typical question: In solving the differential equation y 0 = xy +1; y(0) = 1 by RK4, with step sizes 0:0625 and 0:03125, the estimates 3: and 3: for y(1) were calculated. (a) Make an estimate of the error involved in these calculated values. (b) What step size should be chosen to obtain 16-decimal accuracy? One thing to notice is that we don t need to know at all what the differential equation is, or even the exact method used. The only important thing to know is the order of the method used, which is k =4. We set y 1 =3: y 2 =3: The formula for the error in the second value, say, is simple. We estimate it to be y 2 y 1 2 k 1 = 0: = 0: To figure out what step size to use to get a given accuracy, we must know how the error will depend on step size. It is c 4

15 Numerical methods of solution 15 for some constant c, where is the step size. In the second run the step size is 0:03125, so we estimate 0: c = (0:03125) 4 =0: To get the step size we want we solve 10 c 4 =0: = =4 ; = = : :3017 =0: Summary of the numerical methods introduced so far We have seen three different methods of solving differential equations by numerical approximation. In choosing among them there is a trade-off between simplicity and efficiency. Euler s method is relatively simple to understand and to program, for example, but almost hopelessly inefficient. The third and final method, the order 4 method of Runge-Kutta, is very efficient but rather difficult to understand and even to program. The improved Euler s method lies somewhere in between these two on both grounds. Each one produces a sequence of approximations to the solution over a range of x-values, and proceeds from an approximation of y at one value of x to an approximation at the next in one more or less complicated step. In each method one step goes from x n to x n+1 = x n +. The methods differ in how the step from y n to y n+1 is performed. More efficient single steps come about at the cost of higher complexity for one step. But for a given desired accuracy, the overall savings in time is good. Method The step from (x n ;y n )to (x n+1 ;y n+1 ) Overall error Euler s s = f (x n ;y n ) proportional to y n+1 = y n +s x n+1 = x n + improved Euler s s = f (x n ;y n ) proportional to 2 (RK of order 2) y = y n +s x n+1 = x n + s = f (x n+1 ;y ) y n+1 = y n +h s +s 2 y = y n +h i 2i Runge-Kutta of order 4 s = f (x n ;y n ) proportional to 4 s h x = x + 2 s = f (x ;y ) y = y n +h 2 s = f (x ;y ) x n+1 = x + i s i y = y n +s s = f (x n+1 ;y ) y n+1 = y n +h s +2s +2s + s 6 i

16 Numerical methods of solution 16 In practice, of course, you cannot expect to do several steps of any of these methods except by computer. You should, however, understand how to one step of each of them; how to estimate errors in using them; how to use error estimates to choose efficient step sizes. Numerical methods are often in practice the only way to solve a differential equation. They can be of amazing accuracy, and in fact the methods we have described here are not too different from those used to send space vehicles on incredibly long voyages. But often, too, in practice you want some overall idea about how solutions of a differential equation behave, in addition to precise numerical values. There is no automatic way of obtaining this. In all these methods, one of the main problems is to choose the right step size. The basic principle in doing this is that: Two runs of any method will let you estimate the error in the calculation, as well as decide what step size to use in order to get the accuracy you require. 13. The problem of stability Although we shall not show how to solve it, we want to point out here a problem which, although somewhat technical, does cause some trouble in practice. First consider the equation & initial conditions y 0 = c y=; y(0) = 0 : This is linear with constant coefficients and has the exact solution y = c ce t= = c (1 e t= ) : Assume that > 0. Then this is the sum of two terms. One is the constant c and the other is exponentially decaying. The rate of decay the time it takes to reduce any fixed factor, for example is proportional to. The quantity is called the relaxation time of the equation, and when it is short the function y reaches equilibrium or relaxes quickly. Solve this by Euler s method with step size. We get t 0 =0 y 0 =0 t n+1 = t n + c y n+1 = y n +(c y n = ) = c +y n (1 = ) The problem is that we can get some rather strange behaviour if is not chosen carefully. Here is the graph we get with not quite small enough.

17 Numerical methods of solution 17 Figure The behaviour of Euler s method for a stiff equation with threshold step size. You can see what happens all the solutions move very rapidly towards the equlibrium, and this means that their slopes are very steep not too far from the equilibrium solution. You have to take very small steps in Euler s method to take this into account. Here is what happens when is just a bit smaller. The process does converge, but very slowly. Figure The behaviour of Euler s method for a stiff equation with step size barely below threshold. We can explain more precisely what we see. Let =1 = for convenience. Then y 0 =0 y 1 =c y 2 =c(1 + ) y 3 = c ( ) y n = c ( n 1 ) Now as n!1this is supposed to converge to c, just like the exact solution. This can t possibly happen unless jj < 1 or j1 =j < 1 Since and are both positive, j1 =j < 1 means that 1 = > 1; < 2 If this condition does not hold then 1 = < 1 and the numbers y n will just oscillate back and forth with larger and larger swings. In particular, when the relaxation time is relatively small we must choose the step size small also if Euler s method is to make any sense. This is a paradoxical situation. When is small then the true solution will be just a constant for all large values of t, the simplest possible sort of behaviour. But we must still make very small steps in order to have Euler s method produce reasonable numbers. This means that we will be working very hard to produce a lot of apparently simple data. Unfortunately, this sort of thing is fairly common in real-life situations whenever there are components of a physical system whose change takes place on scales widely different. Here the two components undergo no change and rapid decay. Such systems are said to be stiff. The idea is that in trying to apply Euler s method to the system we have to make small steps in order to keep the rapidly changing part from causing our calculation to vary wildly, even though it is only the slowest changing parts we can actually measure. How can we do better?

18 Numerical methods of solution 18 This is a complicated topic. There is a simple way to take care of the difficulty in the case of Euler s method which does suggest how to proceed in general. We perform something called the backward Euler s method. The forward Euler s method (the one we are familiar with) assumes the slope across a time segment to be what it is at the beginning: y n+1 = y n +f(t n ;y n ) The backward one sets y n+1 = y n +f(t n+1 ;y n+1 ) Of course this doesn t give an immediate expression for y n+1 since it occurs on both sides. The equation has to be solved for y n+1. In the example above we get y n+1 = y n + (1 y n+1 = ) y n+1 = y n + 1+= which will always converge as long since and are positive. Note that when = is small so the new rule and the old one are consistent. 1 1+= : =1 = Each of the other methods we have seen has backwards versions too. 14. Rounding errors Computers can only deal with numbers of limited accuracy, normally about 16 digits of precision. They cannot calculate even the product and sums of numbers exactly, much less calculate the exact values of functions like sin. Errors in computations due to this limited accuracy are called rounding errors. Rounding errors are not usually a problem in the methods we have seen, except in the circumstance where Euler s method is used to obtain extremely high accuracy, in which case it will require enough steps for serious error to build up. But then Euler s method should never be used for high accuracy anyway. There is one circumstance, however, in which there will be a problem. Consider this differential equation: y 0 = cos t y; y(0) = y 0 The general solution is y = (sin t cos t)=2+(y 0 +1=2)e ct Suppose c>0. Then whenever y 0 6= 1=2 the solution will grow exponentially. In the exceptional case y 0 =1=2 it will simply oscillate. Here, however, is the graph of a good numerical aproximation matching initial condition y(0) = 1=2:

19 Numerical methods of solution 19 Figure How rounding errors affect the calculations for a stiff equation. Rounding errors are inevitably made at some point in almost any computer calculation, and as soon as that happens the estimated y(x) will make an almost certainly irreversible move from the graph of the unique bounded solution to one of the nearby ones with exponential growth. From that point on it must grow unbounded, no matter how well it has looked so far. This is not a serious problem in the following sense: in the real world, we can never measure initial conditions exactly, so that we can never be sure to hit the one bounded solution exactly. Nor can nature. Since the bounded solution is unstable in the sense that nearby solution eventually diverge considerably from it, we should not expect to meet with it in physical problems. Unstable solutions do not realistically occur in nature. Rounding errors simulate the approximate nature of our measurements. Another way to say this: Because of rounding errors, the best we can hope to do with machine computations is to find the exact answer to a problem near to the one that is originally posed.

20 Appendix: Euler s method in C 20 Appendix. Euler s method in C Below is a complete program in C to implement Euler s method for the equation y 0 = y. It includes routines to scan the command line to read values of x 0, x f, y 0, N. It also prints out the run data as the run proceeds. In order to change this program to handle another differential equation, only the function f (x; y) has to be changed, and the program recompiled. #include <stdio.h> #include <math.h> /* global variables */ double xinit, xf, yinit; int N; /* function f(x, y) */ double f(x, y) double x, y; { } return(y); /* options */ void usage() { } fprintf(stderr, "usage: exit(1); euler -xi <xinit> -xf <xf> -yi <yinit> -N <N>\n"); #define OPTNO 4 enum {Optxinit, Optxf, Optyinit, OptN}; char opt_string[][16] = {"-xi", "-xf", "-yi", "-N"}; int get_opt(s) char *s; { } int i; for (i=0;i<optno;i++) { if (strcmp(s, opt_string[i]) == 0) return(i); } fprintf(stderr, "Invalid option [%s]\n", s); usage(); double atof();

21 Appendix: Euler s method in C 21 int read_command_line(argc, argv) int argc; char **argv; { int opt; char *a; argv++; argc--; } while (argc > 0) { opt = get_opt(a = *argv++); argc--; switch(opt) { case Optxinit: a = *argv++; argc--; xinit = atof(a); break; case Optxf: a = *argv++; argc--; xf = atof(a); break; case Optyinit: a = *argv++; argc--; yinit = atof(a); break; case OptN: a = *argv++; argc--; N = atof(a); break; default: usage(); } } /* */ main(argc, argv) int argc; char **argv; { int i; double x, y, h; if (argc > 1) {

22 Appendix: Euler s method in C 22 } read_command_line(argc, argv); } else { usage(); } x = xinit; y = yinit; h = (xf - xinit)/((double) N); for (i=0;i<n;i++) { printf("%8.6lf %8.6lf\n", x, y); y=y+h*f(x, y); x=x+h; } printf("%8.6lf %8.6lf\n", x, y); exit(0);