COURSE: NUMERICAL ANALYSIS. LESSON: Methods for Solving Non-Linear Equations

COURSE: NUMERICAL ANALYSIS LESSON: Methods for Solving Non-Linear Equations Lesson Developer: RAJNI ARORA COLLEGE/DEPARTMENT: Department of Mathematics, University of Delhi Page No. 1

Contents 1. LEARNING OUTCOMES... 3 2. PRELIMINARIES... 3 3. INTRODUCTION... 4 3.1. DIRECT METHOD... 4 3.2. ITERATIVE METHOD... 4 Definition 3.3. CONVERGENCE... 5 Definition 3.4. RATE OF CONVERGENCE... 5 Definition 3.5. ORDER OF CONVERGENCE... 6 4. SOLUTIONS OF NON LINEAR EQUATIONS... 6 4.1. FIXED POINT ITERATION METHOD... 6 4.1.1. Geometrical Interpretation... 7 Problem 4.1.2.... 7 4.1.3. ORDER OF COVERGENCE OF FIXED POINT ITERATION METHOD... 9 4.2. NEWTON RAPHSON METHOD... 10 4.2.1. GEOMETRICAL INTERPRETATION... 10 4.2.2. DERIVATION... 11 PROBLEM 4.2.3.... 11 PROBLEM 4.2.4.... 12 4.2.5. ORDER OF CONVERGENCE OF NEWTON RAPHSON METHOD... 13 4.3. SECANT METHOD... 14 4.3.1. GEOMETRICAL INTERPRETATION... 15 4.3.2. DERIVATION... 15 PROBLEM 4.3.3.... 16 4.3.4. ORDER OF COVERGENCE OF SECANT METHOD... 16 4.4. REGULA FALSI METHOD... 18 4.4.1. GEOMETRICAL INTERPRETATION... 19 Problem 4.4.2.... 19 4.4.3. ORDER OF COVERGENCE OF REGULA FALSI METHOD... 20 4.5. BISECTION METHOD... 22 4.5.1 GEOMETRICAL INTERPRETATION... 22 Problem 4.5.2.... 22 5. SUMMARY... 25 6. EXERCISES... 26 7. REFERENCES... 27 Institute of Lifelong Learning, University of Delhi Page No. 2

1. LEARNING OUTCOMES We believe that after going through this chapter, the reader will become familiar with various methods for approximating roots of linear as well as non linear equations and will be able to analyze which method is better, in terms of computation effort as well as order of convergence. 2. PRELIMINARIES Definition 2.1. Linear Equation An equation in the form is called a linear equation where unknowns coefficients. For example,,. have power 1 and have constant Definition 2.2. Non Linear Equation An equation in the form is called a non linear equation if it is not linear. For example,. Definition 2.3. Solutions/Roots of an Equation Consider the equation, where is a real valued function. A number is called a solution/root of this equation, if. Theorem 2.4. Taylor s Theorem with Lagrange s Formula for Remainder Let has continuous derivatives on for some, let. Then there exists a point between and such that where. Theorem 2.5. (A special case of Taylor s theorem: Mean Value Theorem) If is continuous on and if exists on, then for and in, there exists a point between and such that Theorem 2.6. (A special case of Mean value theorem: Rolle s Theorem) If is continuous on and if exists on, and if, then for some in. Theorem 2.7. If be a sequence of real numbers such that Institute of Lifelong Learning, University of Delhi Page No. 3

Theorem 2.8. (Intermediate value Theorem) If is a continuous function on some interval and be a number such that, then a number exists such that. Theorem 2.9. (Corollary to Intermediate value Theorem) If is a continuous function on some interval and, then the equation has atleast one real root or an odd number of real roots in the interval. 3. INTRODUCTION From an early age, we are dealing with equations of various types, for example, linear equations of the type, whose root is given by the expression (provided ), quadratic equations of the type, whose root is given by the formula (provided ). However, there are no direct formulae for finding roots of general third or higher order polynomial equations. Neither are there any direct formulae for finding roots of non linear equations of the type. Our aim in this chapter is to introduce and discuss methods for finding roots of equations of this kind. There are two types of methods for finding roots of equations, one of which is Direct Method, with which we are familiar and other one is Iterative Method, which is our main focus in this chapter. 3.1. DIRECT METHOD These methods give the exact value of the roots in a finite number of steps. These give all the roots at the same time. For example, in linear and quadratic equations we are able to find roots directly. 3.2. ITERATIVE METHOD These methods are based on the idea of successive approximation, i.e. given a recursion formula, we start with one or more initial approximations of the root and obtain a sequence of approximations or iterates, say The method gives one root at a time. which in the limit converges to the root. For example, to solve quadratic equation ( ), although we have a direct method for finding roots, we may obtain the following three iterative methods: (i) Then, the recursion formula can be written as Institute of Lifelong Learning, University of Delhi Page No. 4

(ii) This formula is called iterative method. Then, the iterative method becomes (iii) Then, the iterative method becomes Using initial approximation, we can calculate the value of from any of the above iterative methods. Again, using and same iterative method, we calculate. We continue following these steps till tolerance). becomes less than permissible error (error Now, the question arises, which of these iterative methods is the best suitable method? For this, we discuss the concept of convergence and in later sections obtain conditions to ensure convergence of such methods. Definition 3.3. CONVERGENCE A sequence of iterates converges to the root of equation, if In case where it is not possible to find root, we attempt to find an approximate root such that either or where are two consecutive iterates and is the prescribed error tolerance. The convergence of the sequence of iterates to the root of the given equation depends on the rearrangement and the choice of the initial approximation. Definition 3.4. RATE OF CONVERGENCE Let be a sequence of iterates that converges to. If there exists a sequence that converges to zero and a positive constant, independent of, such that Institute of Lifelong Learning, University of Delhi Page No. 5

for all sufficiently large values of, then is said to converge to with the rate of convergence O, called big-o of. If converges to with rate of convergence O, we express it as +O(.The big-o term provides a reference for how quickly error approaches zero. For example, a sequence with rate of convergence O converges more slowly than a sequence with rate of convergence O( which in turn converges more slowly than a sequence with rate of convergence O( ). For example, consider the sequences of iterates and both converging to 1. Observe that and In both the cases whereas the rates of convergence are O and O respectively. The sequence converges to zero faster than and hence the sequence approaches to 1 faster than. Definition 3.5. ORDER OF CONVERGENCE We say that an iterative method has order of convergence, if is the largest such positive real number for which where = denotes the error in iterate and is called asymptotic error constant and usually depends on derivatives of at. We also say that the method is of order if the above condition is satisfied. Order of convergence examines the relationship between successive errors which measures the effectiveness with which each iteration reduces the approximation error. 4. SOLUTIONS OF NON LINEAR EQUATIONS 4.1. FIXED POINT ITERATION METHOD Consider the equation. As discussed in section 3, it can be rewritten in the form It has been shown in section 3 that this representation is not unique. However, the choice of function is based on the fulfilment of the condition, where are iterates otherwise the sequence of iterates will not converge to the root. Then the iterative method is given by Institute of Lifelong Learning, University of Delhi Page No. 6

4.1.1. Geometrical Interpretation We first convert the problem of finding root of to solving. can be written as coupled equation. We try to find the point of intersection of line and curve. Starting with initial iterate, where the distance AB between is large. Using the iterative method, and with the fulfilment of condition we obtain next iterate and observe that this distance has now reduced. This distance further gets reduced on each successive iteration and moving along a spiral path, we approach towards the point of intersection and hence the root. In some cases, we may observe that the successive iterates approach the root along staircase path. A B Note: If initial approximation is not given, then we take two numbers and such that Then the root lies between and. Now, take if is nearer to zero compared to or take if is nearer to zero compared to. In this iterative method, one needs to be a little careful while choosing initial approximation. We check and not, since and are entirely different curves and we aim at finding root of. Problem 4.1.2. Find positive root of method. using the fixed point iteration Solution. Let Institute of Lifelong Learning, University of Delhi Page No. 7

So, we may choose either 1 or 2 as initial approximation. Let. Now, to apply fixed point iteration method, we first convert in the form. The possible representations could be 1., This is not suitable since is not defined. 2., This form also is not suitable since is not defined. 3., Let s choose. (Observe that and are entirely different functions) Then, by using fixed point iteration method, For Now, Condition is satisfied. So we proceed to next step Then, and =0.3912<1 Continuing this way, we get and =0.3924 and =0.3929 Institute of Lifelong Learning, University of Delhi Page No. 8

and =0.3931 and =0.3931 and =0.3931 and =0.3931 Hence, the required root is 1.6953. 4.1.3. ORDER OF COVERGENCE OF FIXED POINT ITERATION METHOD We show that the fixed point iteration method has linear order of convergence. Let the iterative method be defined as Suppose that the sequence of iterates converges to the value so that, in the limiting case, (4.1.3.1) gives Subtracting (4.1.3.2) from (4.1.3.1), we get Now if error at ( +1) th and th stage is defined as Then, the relation (4.1.3.3) gives Consider. Expanding about the point with the help of Taylor series expansion Now relation (4.1.3.4) gives us Neglecting the terms of order, and terms of higher order, the relation (4.1.3.6) provides us, =0,1,2,..., =0,1,2,..., =0,1,2,... Then the sequence of iterates converge to the root if which holds if (by using Theorem 2.7) and hence the order of convergence of the method is one with asymptotic error constant. Institute of Lifelong Learning, University of Delhi Page No. 9

Algorithm GIVEN, INITIAL APPROXIMATION AND PERMISSIBLE ERROR GIVEN =NUMBER OF ITERATIONS Define FOR =0 TO COMPUTE IF and < SOLUTION= ELSE STOP END IF STATEMENT STOP ITERATIONS END FOR LOOP 4.2. NEWTON RAPHSON METHOD Consider the equation, where is a real valued function, continuous together with its first and second derivatives within a certain interval. Let Then the Newton Raphson method is given by the following iterative formula be the initial iterate., provided. 4.2.1. GEOMETRICAL INTERPRETATION The idea is to approximate roots by using tangents. Let initial approximation be Consider the graph of. Draw tangent line at the point (. The point where tangent line meets the -axis is the second iterate point (. Now, draw tangent at the and repeat this process to obtain next iterate and hence by following this process the sequence of iterates is obatined. Institute of Lifelong Learning, University of Delhi Page No. 10

( ( ( 0 4.2.2. DERIVATION Derivation of the method is based on the steps discussed in its geometrical representation. Let be an approximate root/initial iterate of the equation and let tangent line drawn at ( to the curve meets -axis at. Then the equation of segment of tangent line joining ( and ( is given by The above equation is simplified to obtain Now taking as next iterate and assuming tangent line to the curve at ( meets - axis at. Equation of segment of tangent line joining ( and ( gives next iterate The generalisation of this process is given by the following formula The condition is necessary or otherwise tangent line will become parallel to - axis and then no roots can be found using Newton Raphson method. PROBLEM 4.2.3. Perform four iterations of the Newton Raphson method to find the root of the non linear equation. SOLUTION. Institute of Lifelong Learning, University of Delhi Page No. 11

Let Choose. Now, The required root is (approx) PROBLEM 4.2.4. Find an iterative formula to compute the reciprocal of a natural number. SOLUTION. Let Let Using Newton Raphson formula Institute of Lifelong Learning, University of Delhi Page No. 12

So, the required iterative formula is 4.2.5. ORDER OF CONVERGENCE OF NEWTON RAPHSON METHOD The Newton Raphson method for solving is given by Let be the exact root of, such that Let the errors at th and ( +1) th stage be defined as, Then,, Substituting the values of and in (4.2.5.1), we get Expanding and about by Taylor series and using (4.2.5.2), we get Substituting the values of and in (4.2.5.3) Institute of Lifelong Learning, University of Delhi Page No. 13

Neglecting the terms of and higher order terms, we obtain Now, is a constant. So, let. Then, we have Hence, the order of convergence of the Newton Raphson Method is 2. Algorithm GIVEN, INITIAL APPROXIMATION AND PERMISSIBLE GIVEN =NUMBER OF ITERATIONS FIND FOR =0 TO COMPUTE IF < SOLUTION= STOP ITERATIONS END IF STATEMENT END FOR LOOP 4.3. SECANT METHOD Consider the equation, where is a real valued function. If and are two approximations to the root of such that, then the next approximation to the root is given by the secant method: This method is also called chord method. It is a two point iteration method. Institute of Lifelong Learning, University of Delhi Page No. 14

4.3.1. GEOMETRICAL INTERPRETATION ( ( ( 0 Unlike use of tangents in the case of Newton Raphson method, we will make use of secants in this method. Given two initial approximations and, draw secant passing through the points ( and (. The point where this secant meets the -axis is the next iterate. Now, again draw secant passing through the points ( and (, the point where it meets the -axis gives the next iterate. Hence by repeating this process the sequence of iterates converging to the root is obtained. 4.3.2. DERIVATION The secant method can be derived very easily by following the ideas illustrated in derivation of Newton Raphson method. Let, be initial approximations to the equation. Then the equation of secant passing trough ( and ( is given by Let, this secant meets the -axis at, so point satisfies this equation. Then we have By eliminating, we obtain Similarly, we can obtain next iterate Generalising this process to obtain the following formula Institute of Lifelong Learning, University of Delhi Page No. 15

PROBLEM 4.3.3. Determine the root of the equation method. by using secant SOLUTION. Let Taking the initial approximations as and. Using the secant method, next approximation can be obtained as and Next, and Similarly, Therefore, the required root is 0.5177(approx). 4.3.4. ORDER OF COVERGENCE OF SECANT METHOD The secant method for solving the equation is given by Let be the exact root of, such that Let the errors at th th and ( +1) th stage be defined as Institute of Lifelong Learning, University of Delhi Page No. 16

, Then,, Substituting the values of and in (4.3.4.1), we get Expanding and about by Taylor series, we have Using these relations, we obtain and Substituting the values in (4.3.4.3), and using, we obtain Neglecting the higher order terms and taking, we get Now, we determine such that Replacing by, Institute of Lifelong Learning, University of Delhi Page No. 17

Substituting these values in (4.3.4.4) Comparing the like powers of on both sides, we get which gives, positive root. Hence,. Since, order of convergence can never be negative, we take (approx) is the order of convergence of secant method. Algorithm GIVEN, INITIAL APPROXIMATIONS, AND PERMISSIBLE ERROR GIVEN =NUMBER OF ITERATIONS FOR =1 TO COMPUTE IF < SOLUTION= STOP ITERATIONS END IF STATEMENT END FOR LOOP 4.4. REGULA FALSI METHOD Regula Falsi method is also known as False position method or Method of False position. Consider the equation, where is a real valued function. If and are two approximations to the root of such that, then the next approximation to the root is given by the regula falsi method: Regula falsi method becomes secant method if the condition is dropped. The advantage of this method over secant method is that it is based on intermediate value theorem which guarantees the existence of root whereas secant method does not guarantee. Institute of Lifelong Learning, University of Delhi Page No. 18

4.4.1. GEOMETRICAL INTERPRETATION ( 0 ( ( Given two initial approximations and, such that, draw chord joining the points ( and (. The point where this chord meets the -axis is the next iterate. Now observe that in the particular figure chosen. So, we draw a chord joining the points ( and (, the point where it meets the - axis gives the next iterate.had it been, we would have drawn chord joining ( and (. Hence in this method, we not only use chords or secants but also make use of intermediate value theorem at every step. Note: Reader can now compare and analyze that the derivation of regula falsi method is almost same as the derivation of secant method. Problem 4.4.2. Solve for the root lying between 2 and 4 by regula falsi method. Solution. Let and, such that So, by applying regula falsi method, and Observe that, so root lies between and. Now, Institute of Lifelong Learning, University of Delhi Page No. 19

and Now,, so root lies between and. Proceeding this way, we obtain So, approximated root is 2.943. 4.4.3. ORDER OF COVERGENCE OF REGULA FALSI METHOD We consider a particular case of regula falsi method: which can be rewritten as Let be the exact root of, such that Let the errors at th and ( +1) th stage be defined as,, similarly, Then, Substituting the values of, and in (4.4.3.1), we get, Expanding and about by Taylor series, we have Institute of Lifelong Learning, University of Delhi Page No. 20

Substituting the values in (4.4.3.3), and using, we obtain Neglecting the higher order terms, we get Taking Therefore, the method has linear order of convergence. Algorithm GIVEN, INITIAL APPROXIMATIONS, AND PERMISSIBLE ERROR GIVEN =NUMBER OF ITERATIONS FOR =1 TO COMPUTE IF < UPDATE and ELSE UPDATE and END IF STATEMENT IF < SOLUTION= STOP ITERATIONS END IF STATEMENT END FOR LOOP Institute of Lifelong Learning, University of Delhi Page No. 21

4.5. BISECTION METHOD This method is based on the repeated application of the intermediate value theorem. The Bisection method involves the following steps: 1) Let interval which contains the root of be given 2) Bisect at the point 3) Calculate and If take, else if take. So, now interval also contains the root. 4) Bisect at the point and by repeating step 3 obtain interval. We continue this process and obtain a nested sequence of intervals containing root:..., such that after repeating the process times, we either find a root or find the interval of length which contains the root. 5) Take mid-point of this last interval as the derived approximation to the root. 4.5.1 GEOMETRICAL INTERPRETATION We give geometrical interpretation of bisection method with the help of an illustration. Problem 4.5.2. Solve with the help of bisection method. Solution. Let Hence, root lies between 2 and 3. (Using this information, we draw the following intuitive figure) =2 =3 Step 1. Take. Step 2. and Institute of Lifelong Learning, University of Delhi Page No. 22

Step 3. and. So root lies between 2.5 and 3. Take. =2 =2.5 =3 Step 4. Again, let s repeat steps 2 and 3. and So, root lies between 2.5 and 2.75. Take =2 =2.5 =2.75 =3 Again and So, root lies between 2.625 and 2.75. Take =2 =2.625 =3 So, proceeding in this way, the interval containing root is shrinking and we are gradually approaching towards the root. Institute of Lifelong Learning, University of Delhi Page No. 23

. So,. So,. So,. So,. So,. So,. So,. So, We stop iterations here, since and are identical upto third decimal place. The root lies in interval and it can be chosen as 2.645. Result 4.5.3. If a root of lies in the interval, then the minimum number of iterations required for the convergence when the permissible error is is Proof: If the root of lies in the interval, then after n bisections the length of the interval which contains the root is. Therefore, the minimum number of iterations with determined from the relation, as the permissible error may be Taking log on both sides, Institute of Lifelong Learning, University of Delhi Page No. 24

Algorithm GIVEN, INITIAL INTERVAL AND PERMISSIBLE ERROR GIVEN =NUMBER OF ITERATIONS FOR =0 TO COMPUTE IF < UPDATE and ELSE UPDATE and END IF STATEMENT IF < SOLUTION= STOP ITERATIONS END IF STATEMENT END FOR LOOP SOLUTION= END IF END FOR 5. SUMMARY In this chapter, we discussed five methods for solving non linear equations register their order of convergence in the following table, we Method Order of convergence Fixed point iteration method 1 Newton Raphson method 2 Secant method 1.61 Regula falsi method 1 Bisection method 1 We have also provided geometrical interpretations and algorithms for each of the above methods. Clearly, order of convergence of Newton Raphson method is the highest which makes it the most desirable method but it requires derivative of the function, hence adding an additional step to its algorithm. Moreover, faster convergence in Newton Raphson Institute of Lifelong Learning, University of Delhi Page No. 25

method is possible if has large value otherwise computation of the root either becomes slow or impossible. If vanishes at some point, it becomes impossible to find next iterate. Next, we discussed secant method and regula falsi method which are based on secants and hence no derivatives are required. Order of convergence of secant method is higher than regula falsi method, so secant method seems like more promising method. However, in certain cases where permissible error is chosen such that the difference between two successive iterations is small, then the secant may become parallel to -axis and thus, no further iterations can be done, whereas, regula falsi method is based on intermediate value theorem which guarantees the existence of solution. Bisection method is based on repeated application of intermediate value theorem which without any additional condition guarantees the existence of a root. 6. EXERCISES 1. Compute and also determine the corresponding rate of convergence. 2. The errors of successive iterates obtained while approximating using two different methods are registered in the following table. Estimate the order of convergence of each method. Error Method I Method II 0.40 10-1 0.37 10-3 0.91 10-3 0.12 10-14 0.48 10-6 0.15 10-59 3. Find a positive root of and by using each of the following methods: Newton Raphson method, Bisection method, Secant method, Regula falsi method, Fixed point iteration method. (Ans.1.856,0.567) 4. The equation has one root near which is to be computed by the iteration method is an integer. (a) Determine which value of will give the fastest convergence. (b) Using the value of, perform three iterations and calculate. (Ans. ; 4.223) 5. By applying Newton Raphson method to the function. Prove that If, where, show that Institute of Lifelong Learning, University of Delhi Page No. 26

7. REFERENCES [1] S Salleh, A Y Zomaya and S A bakar, Computing for Numerical Methods using Visual C++, Wiley Interscience, 2007. [2] K E Atkinson, An Introduction to Numerical Analysis, John Wiley & Sons, Second edition. [3] M K Jain, S R K Iyengar and R K Jain, Numerical Methods (Problems and Solutions), New Age International Publishers, Revised second edition. [4] M K Jain, S R K Jain, R K Jain, Numerical Methods for Scientific and Engineering Computation, New Age International Publishers, Fifth Edition, 2007. [5] B Bradie, A Friendly Introduction to Numerical Analysis, Pearson education, India, 2007. Institute of Lifelong Learning, University of Delhi Page No. 27