Chapter - 2 Complexity of Algorithms for Iterative Solution of Non-Linear Equations

Transcription

1 Chapter - Compleity of Algorithms for Iterative Solution of Non-Linear Equations

2 Compleity of Algorithms for Iterative CHAPTER - Compleity of Algorithms for Iterative Solution of Non-Linear Equations.1 Introduction Computational compleity can be defined as a function of the size of input resources required for computation Michael (8. The running time of an algorithm or data structure operation typically depends on number of factors lie input size, hardware and software used Thomas (11. We can study its running time on various inputs and recording the actual time spent in each eecution. Such measurement can be taen in an accurate manner by using system calls built in language for which algorithm is written. In general, we are interested in determining the dependency of running time on size of input. In order to determine this, we can perform several eperiments on many different test inputs of various sizes. Very often engineering and mathematics, a non-linear equation are solved using iterative methods. The Bisection Method by Mathew (199 is among the few iterative methods which guarantee convergences. Here we will present the algorithm for Bisection method and then find the compleity for it. In this chapter algorithms for Bisection, Secant, Regula-Falsi, and Newton Rapson and Adaptive Bisection method for find the roots of non-linear equation is studied for their compleity. Using RAM (Random Access Machine model the compleity of all algorithms are calculated. The compleity can represent in the form of Big-Oh notation in term of time function. All algorithms have the same compleity O (n, so that less number of iteration algorithm will eecute faster. Adaptive Bisection method is used to find the roots in less number of iteration so its time compleity will better and eecuter faster..1.1 RAM Machine Model Definition An approach of simply counting primitive operations given rise to a computational model called Random Access Machine (RAM Goodrich (8 defines a set of high level primitive operations that are largely independent from

3 Compleity of Algorithms for Iterative... programming language used and can be defined also in the pseudo-code. Primitive operation include following: 1. Assigning the value to variable.. Calling a method 3. Performing an arithmetic operation 4. Comparing two numbers 5. Indeing into array. 6. Following an object reference 7. Returning from a method. Specifically, a primitive operation corresponds to low-level instruction with an eecution time that depends on the hardware and software environment but is nevertheless constant. Instead of trying to determine the specific eecution time of each primitive operation are eecuted, and use this number t as high level estimate of the running time of algorithm. We assume the CPU in RAM model can perform any primitive operation in constant number of steps, which do not depend on the size of the input. Thus an accurate bound on the number of primitive operations an algorithm performs corresponds directly to the running time of that algorithm in the RAM model..1. Counting Primitive Operation We now show to count the number of primitive operations eecuted by an algorithm, used for all non linear iterative methods. We do this analysis by focusing on each step of the algorithm and counting primitive operation that it taes, taing into consideration that some operation are repeated, because they are enclosed in the body of a loop..1.3 Asymptotic Notation We have clearly gone into detail for evaluating the running time of such simple algorithm. Such an approach would clearly prove cumbersome if we had to perform it for more complicated algorithms. In general, each step is pseudo-code description and each statement in a high level language implementation corresponds to small number of primitive operations that does not depend on input size. Thus we can perform a simplified analysis that estimates the number of primitive operation

4 Compleity of Algorithms for Iterative... 1 eecuted up to a constant factor, by counting the steps of pseudo-code or the statements of high level language eecuted. Fortunately, there is a notation that allow us to characterize the main factor affecting an algorithm s running time without going into all detail of eactly how many primitive operations are performed for each constant time of instructions.1.4 The Big-Oh Notation Let f ( n and g( n be functions mapping non-negative integers to real numbers. We say that f ( n is O ( g( n if there is a real constant c > and an integer constant n >= 1 such that f ( n <= c( g( n for every integer n >= n. This definition is referred to as big-oh notation. Alternatively, we can also say is order g( n defined by Goodrich (8. f ( n. Bisection Method Mathews (199 shows that the Bisection Method is among few iterative methods which guarantee convergence which is in linear rate. If a function f ( a continues function between a and b and f ( a and f ( b are opposite sign such that f(a f(b<, then there eist at least one zero r for f on ( a, b. If [ a, b ] is used is as the initial interval, then the bisection algorithm generates a sequence intervals on which the root lies. [ a, b ] of Convergences are reached when [ b a ] less than some tolerance, say ε is...1 Algorithm for Bisection Method a Let the root of with a prescribed tolerance say epsilon. Given that and b such that f (a f (b <. The value c is used to store the middle point of the interval. 1. Read a, b. Read Epsilon f ( = 3. Repeat step 4 to 5 while ( ( a b a <Epsilon and ( f ( c

5 Compleity of Algorithms for Iterative Set c = ( a + b 5. If ( f (a f (c < then Set b Elses Set a = c End if 6. Write Approimate root is, 7. End = c c.. Counting Primitive Operation 1. Reading the value of variable a, b contributes two unit of count.. Reading the value of epsilons contributes one unit of count. ( a, b 3. Before entering the body of loop condition < epsilon is verified. a This action corresponds to three (one for subtraction, one for division and one for comparison primitive operation and performed n times. 4. At the beginning of loop c is calculated. This action correspond to eecuting three primitive operation. 5. In the body of the loop, condition f (a f (b < is verified. This action corresponds to eecuting four unit of operation (one for multiplication and one for verification the condition. 6. As per result of verification one assignment statement will eecute requires one operation. 7. The body of loop is eecuted n times. Hence, at each iteration of loop, eleven primitive operations is performed 11n times. 8. Printing the value of requires one operation. To summarize, the number of primitive operations t (n eecuted by algorithm is at least. C t( n = ( n + 1 t( n = 11n + 4

6 Compleity of Algorithms for Iterative The Big-Oh Notation By the big-oh notation, we need to find a real constant and an integer constant n >= 1 such that 11n + 4 = cn for every integer n >= n. It is easily see that a possible choice is c=15 and n=1. The big- Oh notation allows us to say that a function of n is less than or equal to another function (by the equality <= in the definition, up to constant factor infinitely (by the statement definition. in the The number of primitive operations eecuted by algorithm for Bisection method is at most 11n + 4. We may therefore apply the big-oh notation with and n = and conclude the running time of algorithm in O( n. 1 c < n >= n = O( n So the compleity of algorithm used for Bisection method is O( n. In fact, any polynomial a n +a -1 n -1 +a will always be O( n. #include<math.h> #include<stdlib.h> //#define f( for which root is required #include<conio.h> void main ( int i; float, 1, e,, s; clrscr (; printf ("\n Enter first point of interval"; /* reading initial values*/ pcanf ("%f", &; printf ("Enter the second point of interval"; scanf ("%f", &1; printf ("Enter the prescribed tolerance"; /* Reading tolerance*/ scanf ("%f", &e; i=; if (f(*f(1>

7 Compleity of Algorithms for Iterative... 4 printf ("Starting values are unsuitable"; getch (; eit (1; while (fabs ((1-/>e =(+1/; /* middle value*/ i++; /* Iteration counter*/ if (f(*f(> =; else 1=; /* End of While loop*/ printf ("\Solution convergence to root"; printf ("No of iteration: %d",i; printf ("\n Root is %f, function value is%f",,f(; getch (;..4 Result The following table shows results of few functions and their number of iterations to find the root of equations: Table.1: Bisection Method Function Initial Intervals No of Iteration n 11n+4 for c=15 f ( = e 3 f 3 ( = 1 f ( = ( + 3 f ( = cos( e f ( = e 1 f 3 ( = + [1,] 11 +4< 15 [-,] 11 +4<= 15 [-4,-1] <=3 115 [,1] <=17 15 [-1,] <=19 15 [-,-] <=19 15

8 Compleity of Algorithms for Iterative Regula-Falsi method A way to avoid such pathology is to ensure that the root is braceted between the two starting values and remains between the successive pairs. When this is done, the method is nown as linear interpolation or method of false position. This technique is similar to bisection ecept the net iterate is taen at the intersection of a line between the pair of -values and the -ais is rather than at the midpoint..3.1 Algorithm for Regular False method 1. Read,,Epsilon. Repeat 3. Set f ( 1 ( 1 = 1 ( f ( f ( 4. If f ( is opposite sign to f ( then Set Else Set 5. End if 1 = 1 = 6. Until absolute f ( <tolerance value 7. Print Approimation root is,.3. Counting primitive operations 1 1. Reading the value of variable, contributes two unit of count.. Reading the value of epsilons contributes one unit of count. 3. As per result of verification one is calculated using 9 operations 4. The condition f f < contributes 3 operation. ( ( 5. One operation for assignment statement after the condition. 6. In the body of the loop, condition fabs( > is verified. This action corresponds to eecuting two unit of operation (one for for verification the condition. and one 7. The body of loop is eecuted n times. Hence, at each iteration of loop, [9+3+1+] 15 primitive operations is performed 15n times. 8. Printing the value of requires one operation. 1 f (

9 Compleity of Algorithms for Iterative... 6 To summarize, the number of primitive operations t (n eecuted by algorithm is at least. t(n = +1+15n + 1 = 14n The Big-Oh Notation By the big-oh notation, we need to find a real constant and an integer constant n > such that for every integer n >= n. It is easily see that a possible choice is c=19 and n = 1. The big- Oh notation allows us to say that a function of n is less than or equal to another function (by the equality <= in the definition, up to constant factor infinitely (by the statement n >= n c > in the definition. The number of primitive operations eecuted by algorithm for Regula-Falsi method is at most 15n+4. We may therefore apply the big-oh notation with c=19 and n = and conclude the running time of algorithm in O( n. 1 So the compleity of algorithm used for Regula-Falsi method is O( n. #include<stdio.h> #include<conio.h> #include<math.h> #include<stdlib.h> //#define f( for which to find the roots void main( float,1,,e; int i=; printf("enter Initial roots, and tolerance factor"; scanf("%f %f %f",&,&1,&e; if((f(*f(1>. printf("initial roots are unsuitable"; eit(1;

10 Compleity of Algorithms for Iterative... 7 getch(; While(while(fabs(f(>e i++; =1-f(1*(-1/(f(-f(1; if((f(*f(< 1=; else =; printf(" root is %f and no of iteration=%d and f(=%f",,i,f(; getch(; Enter Initial roots, and tolerance factor -.1 Root a and no of iteration= and f( =.3.4 Result The following table shows results of few functions and their number of iterations to find the root of equations: Table-.: Regula Falsi Method Function Initial Intervals No of Iteration n 15n + 4 for C=19 f ( = e 3 f = 3 ( 1 f = + ( ( 3 f ( = cos( e f ( = e 1 f = + 3 ( [1,] <19 18 [-,] < 18 [-4,-1] <3 18 [,1] <13 18 [-1,] <38 18 [-,] <19 18

11 Compleity of Algorithms for Iterative The Secant method The Secant method begins by finding two points on curve of f (, hopefully near to the root we see. If f ( were truly linear, the straight line would intersect at the -ais at root. But f ( will never be eactly linear because we would never use a root finding method on a linear function. That means the intersection of the line with -ais is not at =r but that it should be close to it. From the obvious similar tingle we can write ( 1 ( 1 =. f ( f ( f ( 1 1 And from this solve from = 1 ( 1 f ( f ( 1 Because f ( is not eactly linear is not equal to r but it should closer than either of the two points we begin with. If we repeat this we have: ( ( n 1 n n+ 1 = n f ( n. f ( f ( ( n 1 Because each newly computed value should be nearer to the root, we can do it easily after second iterate has been computed, by always using the last two computed points. But after the first point there aren t two last computed points. So we mae sure to start with 1 closer to the root than by testing and swapping if first functional value is smaller. n f ( and f ( Algorithm for Secant method 1. Read 1 e,n // that are near to the root to determine a root of f ( o,. If f ( < f ( 1 then swap 1 //interchange with 3. Repeat step 4 to 6 until f ( < c tolerance o, 1 4. Set 5. Set f ( 1 ( 1 = 1 ( f ( f ( = 1 1

12 Compleity of Algorithms for Iterative Set = 1 7. Print root is 8. End.4. Counting primitive operations 1. Reading the value of variable,, e, n contributes four unit of count.. Step requires four units of count. 3. In the body of loop is calculated, it requires nine operations To compute and contributes two operations. 5. The condition fabs( f ( < e contributes 3 operation. And hence the body of loop will eecute n times, so (9++3n=14n 6. Printing the value of requires one operation. To summarize, the number of primitive operations t (n eecuted by algorithm is at least. t(n = n + 1 = 14n The Big-Oh Notation By the big-oh notation, we need to find a real constant and an integer constant n >= 1 such that 14n + 9 <=c n for every integer n >= n. It is easily see that a possible choice is c=3 and n =. The big- Oh notation allows us to say that a function of n is less than or equal to another function (by the equality <= in the definition, up to constant factor infinitely (by the statement definition. 1 in the The number of primitive operations eecuted by algorithm for Secant method is at most 14n+9. We may therefore apply the big-oh notation, with c=3 and n = and conclude the running time of algorithm in O( n. 1 c > n >= n So the compleity of algorithm used for Secant method is O( n.

13 Compleity of Algorithms for Iterative... 3 // Define function f(=**-*-5 // df(=3*.*- #include<stdlib.h> #include<stdio.h> #include<conio.h> #include<math.h> //#define f( for which roots are to be calculated //#define df( for above eample void main( float,1,e,ep,delta; int ma_iter,i; printf("enter Initial roots, maimum no of iteration and tolerance factor"; scanf("%f %d %f",&,&ma_iter,&e; printf("enter delta"; scanf("%f",δ for(i=1;i<ma_iter;++i if(fabs(df(<delta printf("\n slope is too small"; getch(; eit(1; 1=-(f(/df(; ep=fabs((1-/1; =1; if((ep<=e printf("solution is convergence\n"; printf("no of iteration=%d",i; printf("\n Root of the given equation is=%8.3f\n",1;

14 Compleity of Algorithms for Iterative getch(; eit(1; getch(; Run Enter Initial roots, maimum no of iteration and tolerance factor 1.1 Enter delta.1 Solution is conversance No of iteration=7 Root of the given equation is= Result The following table shows results of few functions and their number of iterations to find the root of equations: Table-.3: Secant Method Function Initial Intervals No of Iteration n 14n+9<cn for c=3 f ( = e 3 f 3 ( = 1 f ( = ( + 3 f ( = cos( e f ( = e 1 f 3 ( = + [1,] <3 9 [-,] < 15 [-4,-1] < 15 [,1] < 15 [-1,] < 15 [-,] < 15

15 Compleity of Algorithms for Iterative Newton Rapson Method This method is based on a linear approimation of the function but does not so using a tangent to the curve. Starting from a single initial value that is not too far from a root we move along the tangent to its intersection with -ais, and tae that the net approimation. This is continued until either the successive -values are sufficiently close or the value of the function is sufficiently near zero. The general terms = n+ 1 n f ( n f '( n For n=, 1,, Algorithm for Newton Rapson method To determine a root of 1. Read, e. Compute 3. If ( f ( and ( f '( then 4. Repeat step 5 and 6 5. Set 6. Set = 1 = f ( f '( f ( = f (, f '( 7. Until (Absolute ( < e 1, given reasonably close to the root 8. Print Approimation value of root is, 1 9. End if.5. Counting primitive operations 1. Reading the value of variable e contributes two unit of count.. To compute f (, f '( and compare with contributes four operations To compute contributes one unit to count.,

16 Compleity of Algorithms for Iterative To compute contributes five operations. 5. To chec the condition Absolute ( < e contributes three operations The loop is eecuted n times, so (1+5+3 operations are eecuted 9n times. 7. Printing the value of requires one operation. 1 To summarize, the number of primitive operations t (n eecuted by algorithm is at least. t( n = n + 1 t( n = 9n The Big-Oh Notation By the big-oh notation, we need to find a real constant and an integer constant n >= 1 such that 9n + 7 <= cn for every integer n >= n. It is easily see that a possible choice is c=16 and n =. The big- Oh notation allows us to say that a function of n is less than or equal to another function (by the equality <= in the definition, up to constant factor infinitely (by the statement definition. 1 in the The number of primitive operations eecuted by algorithm for Secant method is at most 9n+7. We may therefore apply the big-oh notation with c=16 and n = and conclude the running time of algorithm in O( n. 1 c > n >= n. So the compleity of algorithm used for Newton Rapson method is O( n. #include<stdlib.h> #include<stdio.h> #include<conio.h> #include<math.h> #define f( for which to find roots #define df( (for above eample void main( float,1,e,ep,delta; int ma_iter,i;

17 Compleity of Algorithms for Iterative printf("enter Initial roots, maimum no of iteration and tolerance factor"; scanf("%f %d %f",&,&ma_iter,&e; printf("enter delta"; scanf("%f",δ for(i=1;i<ma_iter;++i if(fabs(df(<delta printf("\n slope is too small"; getch(; eit(1; 1=-(f(/df(; ep=fabs((1-/1; =1; if((ep<=e printf("solution is convergence\n"; printf("no of iteration=%d",i; printf("\n Root of the given equation is=%8.3f\n",1; getch(; eit(1; getch(; Run Enter Initial roots, maimum no of iteration and tolerance factor 1.1 Enter delta.1 Solution is conversance No of iteration=7 Root of the given equation is=.94545

18 Compleity of Algorithms for Iterative Result Table-.4: Newton Rapson Method Function Initial Intervals No of Iteration n 9n+7<cn f ( = e 3 f = 3 ( 1 f = + ( ( 3 f ( = cos( e f ( = e 1 f = + 3 ( [1,] <16 4 [-,] <16 18 [-4,-1] <16 5 [,1] <16 18 [-1,] <16 1 [-,] <16 5ss.6 Adaptive Weighted Bisection Method (AWBM Let f be continuous and twice differentiable over the interval [ a, b] and f (a f (b < such that there eist a number r [a, b] where f ( r = and f "( r if we define the sequence C ; K =,1,... as Dauhoo M.Z (3 C = b f ( c < ζ (i opt C = b + W ( a b Where 1 If [ a, b] < < 1 W opt 1 W = Otherwise Where both f '( and "( are of the same sign for all a, b. f [ ] (ii opt C = a + W ( b a Where W f ( b = ( a b if < W < 1 opt 1 W = Otherwise

19 Compleity of Algorithms for Iterative Where both f '( and "( are of the same sign for all a, b Then for a given ε> there eist a natural number N such that for K > N, s f ( c < <ε f [ ].6.1 Algorithm for Adaptive Weighted Bisection Method 1. Read a, b such that f a f ( < ( b. If both f '( b and f "( b are the same sign f ( b Compute W = f '( b for =, 1, ( a b Choose opt W such that If W > and W < 1 then W Else Calculate Else opt f ( a Compute W = f '( a for =, 1, ( b a If W > and W < 1 then W Else opt Calculate c = a + W ( b a End if 3. If f ( c tolerance then r = c algorithm terminates. 4. If f a f ( c < then Else 5. Eit opt 1 W =. opt 1 W =. ( opt opt c = b + W ( a b Go to step [ a, b ] [ c, b ] Go to step. opt < [ a, b ] [ a, b ] = = = W = W

20 Compleity of Algorithms for Iterative Counting Primitive Operation 1. Reading the value of variable a, b contributes two unit of count. Reading the value of epsilons contributes one unit of count. 3. At the beginning of loop sign of f '( b and f ''( b compared. This action corresponds to eecuting three primitive operation. 4. In the loop is calculated using assignment it taes one operation. opt 5. Is compared with 1 and then chooses requires three operations. W 6. C is compared with e needs one operation and assigning C to r requires W one operation, so two more operation is required 7. At end f ( a and f ( c is calculated and compared with need one operation and then two more assignment are required for a and b, total five operation is required. 8. The loop eecuted n+1 times. Hence each iteration of loop, 1 primitive operations is performed. To summarize, the number of primitive operations t (n eecuted by algorithm is at least. 3+1(n+1 1n+15 W.6.3 The Big-Oh Notation By the big-oh notation, we need to find a real constant c> and an integer constant n >=1 such that 1n+15 <= for every integer n >= n. It is easily see that a possible choice is c=7 and n =1. The big Oh notation allows us to say that a function of n is less than or equal to another function (by the equality <= in the definition, up to constant factor infinitely (by the statement definition. in the The number of primitive operations eecuted by algorithm for adaptive Bisection method is at most 1n+15. We may therefore apply the big Oh definition with c=7 and n = 1 cn and conclude the running time of algorithm is O (n. So the compleity of algorithm used for Adaptive Bisection method is O (n. n >= n

21 Compleity of Algorithms for Iterative Program in C for Adaptive Bisection method for f( = 3 + #include<stdio.h> #include<math.h> #include<stdlib.h> //#define f( (for which to find root //#define f1( for above eample //#define f11(for above eample #include<conio.h> void main( int i,; float r, a, b, c, e, s, w, wopt; clrscr (; printf ("\nenter first point of interval"; scanf ("%f", &a; printf ("Enter the second point of interval"; scanf ("%f", &b; printf ("Enter the prescribed tolerance"; scanf ("%f", &e; s=f (a*f (b; if(s> printf ("Starting values are unsuitable"; getch (; eit (1; for (=;;++ if(((f1(b<&&(f11(b< ((f1(b> && (f11(b> w=-1*(f (b/((a-b*f1(b; if((w> && (w<1 wopt=w; else Wopt=1/.;

22 Compleity of Algorithms for Iterative c=b+wopt*(a-b; else w=-1*f(a/((b-a*f1(a; if ((w> && (w<1 wopt=w; else wopt=1./.; c=a+wopt*(b-a; if (fabs(f(c<.1 r=c; printf ("Root is %f",r; printf ("No of iteration is %d",+1; getch (; eit (1; if (f (a*f (c < b=c; else a=c; Run: Input: Enter first point of interval:- Enter second point of interval:-1 Enter prescribed tolerance:-1 Output Root is:-1.5 Number of iteration: 5.

23 Compleity of Algorithms for Iterative Result Function f ( = e 3 f = 3 ( 1 f = + ( ( 3 f ( = cos( e f ( = e 1 f = + 3 ( Table -.5: Results of some various functions Initial Intervals Bisection O(n Adaptive Bisection O(n No of Iteration n Secant Method O(n False Method O(n Newton Rapson method O(n [1,] [-,] [-4,-1] [,1] [-1,] [-,] Conclusion: In order to compare the adaptive Bisection method with Bisection method, Secant method, Regula False method and Newton Rapson method a variety of functions are used with same criteria i.e The time compleity of all algorithms are O( n where n is number of iteration. The table.5 shows that for function f = + ( ( 3 the number of iteration in Bisection methods are 3 while in adaptive Bisection method is 5. Since time compleity of both algorithms are O( n, so adaptive bisection algorithm will eecute faster to compare Bisection method and it will tae less time. Similarly the same result shows for function f ( = e 1.The following graphs shows the value of root for f ( = ( + 3 and f ( = e 1.