Using Templates to Introduce Time Efficiency Analysis in an Algorithms Course Irena Pevac Department of Computer Science Central Connecticut State University, New Britain, CT, USA Abstract: We propose introducing algorithm time efficiency analysis by utilizing a set of template examples that illustrate the following seven basic time performance complexity categories: log n, n, n log n, n 2, n 3, a n, and n!. The complexity category of some function f(n) is the set of all functions that have the same asymptotic growth as f(n), which is denoted Ө(f(n)). The template examples are simple, their time efficiency analysis is not mathematically intensive, and they correspond to various algorithms. Both iterative and recursive algorithm templates are discussed. In addition, we provide template modifications that do not affect asymptotic growth. Modified templates remain in the same complexity category. Finally, we list several problems for which algorithm time performance belongs to a given complexity category. We argue that the approach of using templates and their transformations not only complements the introduction to time efficiency analysis, but also can improve understanding of this traditionally difficult topic in an Algorithms course. Its advantage is that it sheds some light as to which part or steps of the algorithm affect its run time the most. The template approach also provides flexibility for reaching the appropriate level of in depth discussion suitable for particular learner's background. Keywords: Algorithms, Time Performance Analysis, Teaching 1 Introduction Algorithm time efficiency is an important topic in an algorithms course. This paper discusses the mathematical approach to analyze time efficiency which is independent of the implementation and machine performance details and is used in algorithm course textbooks [1] to [8], [10], and [11]. Most textbooks introduce formal analysis with O, o, Ө, ω, and Ω, and illustrate growth rates functions: log n, n, n log n, n 2, n 3, a n, and n! by displaying their values for different values of n, and by displaying pictures of plotted functions together to provide comparison. The introduction is then typically followed by detailed time performance analysis of several algorithms. The students in an algorithms course often experience difficulty understanding formal analysis of mathematically involved advanced algorithms from various domains and design types. We believe that: 1) learning templates for each of the seven basic complexity categories; 2) learning which modifications of templates produce results that still remain in the same complexity category; and 3) practicing time efficiency analysis of simple algorithms for each complexity category should be done before starting with mathematically intensive analysis of algorithms that are difficult to grasp. This also smooths the transition into studying how to group algorithms by their design type and their domain. We strongly believe that this approach will improve learning the traditionally difficult topic of time efficiency analysis. 2 Templates for the seven basic complexity categories In the next subsections we discuss several templates used in many simple problems to analyze algorithms that belong to the following seven basic complexity categories: log n, n, n log n, n 2, n 3, a n, and n!. Both iterative and recursive algorithm templates are discussed.
Java based pseudo-code is used to write templates. Next, for each template, we list possible modifications that will not change its asymptotic time performance. We omit the illustration of each type of modification with concrete values and also omit proof that it does not change asymptotic growth. In the classroom, an instructor can provide the proof for one or two complexity categories and students can be divided into groups to prove the rest. At the end, for each of the seven complexity categories, we list several examples of problems that can be implemented by using one of the templates to produce an algorithm that belongs to that particular complexity category. More algorithm examples for various complexity categories can be found in [9]. In the classroom students could be asked to provide additional examples for each category. Again, work could be distributed among several student groups. 2.1 Templates for logarithmic run time Template 1 for (i=1; i<=n; i=i*2) Variations for template 1 include one or more of the following: 1) control variable i can be initialized to start at any nonnegative constant instead of starting at 1; 2) performing one basic operation in the loop can be replaced with performing any constant number of basic operations; and 3) control variable i can be multiplied by any other positive constant c in place of 2. Template 2 for (i=n; i>=1; i=i/2) Variations for template 2 include one or more of the following: 1) condition to stop looping can have control variable i greater than or equal to a nonnegative constant; 2) performing one basic operation in the loop can be replaced with performing any constant number of basic operations; and 3) control variable i can be divided by any positive constant c instead of 2. Template 3 LogRec(int n) if (n==1) { LogRec(n/2) } Variations for template 3 include one or more of variable n has one or more of given initial constant values; 2) performing one basic operation can be replaced with performing any constant number of basic operations either in the base case or in the recursive step part; and finally, 3) a recursive call can be made with variable n reduced by any positive constant factor c instead of 2. Obviously, it must be provided that base case is reached. The following simple algorithms with logarithmic time performance can be used to illustrate the above template examples: 1) determine the number of digits in binary representation of a given decimal number n; 2) determine the number of digits of a given integer number n; 3) convert base ten (decimal) number n into base 2 (or base 3, or base 8, or any other constant); 4) calculate a raised to the power of n based on formula a n = 1 if n=1, a n = a n/2 *a n/2 if n>1, and n is even, and a n = a * a n/2 *a n/2, otherwise; 5) print recursively letter "A" log b n times; 6) for given integer number n return the string of digits representing number n in base 3; 7) binary search for worst case (unsuccessful search); and 8) display the digits of a given positive integer n in reverse order.
2.2 Templates for linear run time Template 4 values; 2) performing one basic operation can be replaced with performing a constant number of basic operations in the base case or in the recursive step part; and finally, 3) recursive call can be made with variable n decreased by any positive constant c instead 1, provided that base case is reached.. Variations for template 4 include one or more of the following: 1) control variable i can be initialized to start at any nonnegative constant; 2) performing one basic operation in the loop can be replaced with performing any constant number of basic operations; and 3) control variable i can be increased by any positive constant c instead of 1. Template 5 for (i=n; i>=1; i=i-1) Variations for template 5 include one or more of the following: 1) the initial value for variable i can be n + constant (positive or negative) instead of n; 2) condition to stop looping can have control variable i greater than or eaqual to any positive constant; 3) performing one basic operation in the loop can be replaced with performing any constant number of basic operations; and 4) control variable i can be decreased by any positive constant c instead of 1. Template 6 LinearRec(int n) if n=1 { LinearRec(n-1) } Variations for template 6 include one or more of variable n has one or more of given constant Template 7 LinearRec(int n) if n>1 { LinearRec(n/2) LinearRec(n/2) } Variations for template 7 include one or more of variable n has one or more given constant values; 2) performing one basic operation can be replaced with performing any constant number of basic operations in the base case or in the recursive step part; 3) the order of performing one or more basic operations and making two recursive calls to sub-problems can be any permutation of those three steps; 4) several basic operations can be done before, in between, or after the two recursive calls; and finally, 5) recursive calls can be made with variable n reduced by another positive constant factor c instead of 2. Template 8 LinearRec(int n) if n>1 { LinearRec(n/3) LinearRec(n/3) LinearRec(n/3) } Variations for template 7 include one or more of variable n has one or more of given constant values; 2) performing one basic operation can be replaced with performing any constant number of basic operations in the base case or in the recursive step part before, after, or in
between any two of the three recursive calls to sub-problems; 3) a constant number k of recursive calls can be made instead of three calls, where each recursive sub-problem has variable n reduced by factor k. The following simple algorithms with linear time performance can be used to illustrate the above discussed template examples 4-7: 1) determine the sum of first n numbers; 2) determine the sum of first n squares; 3) determine the sum of first n cubes; 4) determine the sum of array components; 5) determine the harmonic sum; 6) determine the largest array component; 7) determine the index of the largest array component; 8) perform a linear search for a given key in the array with n components; 9) calculate a raised to the power of n based on the formula a n = 1 if n=1, and a n = a n/2 *a n/2 if n>1 and nis even, and a n = a * a n/2 *a n/2 otherwise, where two identical recursive calls to subproblems of size n/2 are made in the code; and 10) print letter "A" n times. 2.3 Templates for n - log n run time Template 9 for (j=1; j<=n; j=j*2) Variations for template 9 include one or more of the following: 1) control variable i of the outer loop can be initialized to start at any nonnegative constant instead of starting at 1; 2) performing one basic operation in the inner loop can be replaced with performing any constant number of basic operations; 3) control variable i can be increased by any positive constant instead of being increased by 1; 4) control variable j can be initialized to any positive constant; and 5) control variable j can be multiplied by another positive constant c instead of being multiplied by 2. Finally, since the two loops are not correlated they can be interchanged without affecting the time performance. Template 10 N_Log_N_Rec(int n) { N_Log_N_Rec( n/2 ) N_Log_N_Rec( n/2 ) Perform n basic operations } Variations for template 10 that do not affect asymptotic time performance include one or more of the following: base case might be one of more values for n that are constant. For such base case values any constant number of basic operations can be performed. During the recursive step the basic operation can be performed an+b times where a and b are constants. Those basic operations can be performed before, after or in between the two recursive calls of size n/2. Template 11 N_Log_N_Rec(int n) { N_Log_N_Rec( n/4 ) N_Log_N_Rec( n/4 ) N_Log_N_Rec( n/4 ) N_Log_N_Rec( n/4 ) Perform n basic operations } Variations for template 11 that do not affect asymptotic time performance include one or more of the following: 1) base case might be one of more values for n that are constant. For such base case values any constant number of basic operations can be performed; 2) during the recursive step the basic operation can be performed an+b times where a and b are constants. Those basic operations can be performed before, after or in between any two of the four recursive calls of size n/4. The following algorithms with n - log n time performance can be used to illustrate the above
discussed template examples 9-11: 1) draw letter "A" n log n times; 2) merge sort; 3) heap sort; and 4) draw the following squares using recursive algorithm. Base case is when side length is less than or equal to a given constant. No basic operation is performed in that case. At the highest level of recursion, one rectangle centered at x,y with side length equal to n is drawn. In addition, four recursive calls are made with center coordinates shifted 2n in each of the following four directions: 1) up, left, 2) up, right, 3) down, left, and 4) down, right. Each of the four recursive calls has size n reduced by factor 4. 2.4 Templates for quadratic run time Template 12 for (j=1; j<=n; j=j+1) Template 13 for (j=i; j<=n; j=j+1) Template 14 for (j=1; j<=i; j=j+1) All variations described for a single loop for linear time are applicable for each of the two nested loops in template examples 12-14. Template 15 QuadraticRec(int n) if (n>0) { do n basic operations QuadraticRec( n-1) } Variations to template 15 that preserve quadratic run time include: 1) base case and/or recursive step where several constant number of basic operations is performed; 2) a recursive call to a sub-problem with size decreased by constant value instead of n-1; 3) the order of recursive call to a sub-problem, and performing one, or performing some constant number of basic operations, can be interchanged. Template 16 QuadraticRec(int n) { QuadraticRec( n/2 ) QuadraticRec( n/2 ) QuadraticRec( n/2 ) QuadraticRec( n/2 ) Do constant number of basic operations } The following simple algorithms with quadratic time performance can be used to illustrate the above discussed template examples 12-16: 1) add all components in a given n x n matrix; 2) determine average of all components in a two dimensional array of size n x n; 3) determine the largest of all components in a two dimensional array of size n x n; 4) selection sort; 5) insertion sort; 6) bubble sort; 7) draw given letter "A" n squared times; 8) draw the squares using the following recursive algorithm: base case is when side length is less than or equal to a given constant. No basic operation is performed for the base case. At highest level of recursion, one rectangle centered at x,y with side length equal to n is drawn. In addition, four recursive calls are made with the center coordinates shifted 2n in each of the following four directions: 1) up, left, 2) up, right, 3) down, left, and 4) down, right. Each of the four recursive calls has size n reduced by factor 2; 9) draw the squares using the following recursive algorithm. Base case is when side length is less than or equal to a given constant. One basic operation is performed in that case. At the highest level of recursion, one rectangle is drawn centered at x,y with side length equal to a given constant. In addition, four recursive calls are made with center coordinates shifted 2n in each of the following
four directions: 1) up, left, 2) up, right, 3) down, left, and 4) down, right. Each of the four recursive calls has size n reduced by factor 2. } 2.5 Templates for cubic run time Template 17 for (j=1; j<=n; j=j+1) for (k=1; k<=n; k=k+1) Template 18 for (j=1; j<=i; j=j+1) for (k=1; k<=n; k=k+1) Variations are analogous to the variations specified for iterative quadratic templates. Template 19 CubicRec(int n) { do n squared basic operations CubicRec(n-1)} Variations to template 20 include base case for n equal to a given constant or n less than given constant and performing up to n*n basic operations. In recursive step, performing one basic operation can be replaced by performing a linear or quadratic number of basic operations. The following simple algorithms with cubic time performance can be used to illustrate the above discussed template examples 17-19: 1) add all components in a given n x n x n matrix; 2) determine the average of all components in a three dimmensional array of size n x n x n; 3) determine the largest of all components in a three dimmensional array of size n x n x n; 4) determine the lengths of the shortest paths between all pairs of vertices in a graph represented with adacency matrix (use Floyd algorithm); 5) print letter the "A" n^3 times; 6) determine the product of two n x n matrices using a brute force algorithm; 7) determine if a given three sets of cardinality n are disjoint; 8) apply Gaussian elimination to transform n x n matrix into upper triangular form; and 9) use Floyd algorithm for shortest paths between all pairs of vertices in a graph represented with adjacency matrix. 2.6 Templates for exponential run time Variations are analogous to the variations for the LinerRec example. Template 20 CubRec(int n) { Template 21 IterativeExponential(int n) limit=1 for (i=1; i<=n; i++) { for(j=1; j<= limit; j++) limit = limit * 2} Variations include: 1) starting loop variables from given constants instead of 1; or 2) ending the outer loop at n const; 3) we can replace the statement limit= limit*2 with the statement
limit= limit*k where k is a constant that is greater than 2. The run time will be a faster growing class but it will still be exponential k^n. for recursive algorithms only. Such learning tool could be used as a supplemental course material for Algorithms courses using several textbooks. Template 22 ExponentialRec(int n) if (n=0) { ExponentialRec( n-1 ) ExponentialRec( n-1 )} Variations include base case at n equal to or less than a given constant and performing a constant number of basic operations at base case or anywhere during the recursive step. 2.7 Templates for n factorial run time Template 23 FactorialRec(int n) if (n=1) for (int i=1; i<=n; i++) FactorialRec(n-1) The following algorithms with n factorial time performance can be used to illustrate the template example 23: 1) print all permutations for given n; 2) print all paths for traveling salesman (exsaustive search brute force algorithm); and 3) print letter "A" n factorial times. 3 Future work The above described approach could be used to design a learning tool similar to ESRATEA (Pevac, 2010) intended for learning and practicing time performance analysis of all types of algorithms. Currently, ESRATEA is designed References [1] Aho, A. V., Ullman, J. D. (1995). Foundations of Computer Science (2nd ed.). Computer Science Press. [2] Baase, S., Van Gelder, A. (2000). Computer Algorithms (3rd ed.). Addison Wesley Longman. [3] Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Clifford, S. (2007). Introduction to Algorithms (2nd ed.). MIT Press; McGraw- Hill Book Company. [4] Goodrich M., Tamassia R. (2002). Algorithm Design: Foundations, Analysis, and Internet Examples John Wiley & Sons. [5] Johnsonbaugh, R., Schaefer, M. Algorithms (2004). Pearson. [6] Levitin, A. (2007). The Design & Analysis of Algorithms (2nd ed.). Pearson- Addison Wesley. [7] McConnell, J. J. (2008). Analysis of Algorithms (2nd ed.). Jones and Bartlett Publishers. [8] Neapolitan, R., Naimipour K. (2004). Foundations of Algorithms Using Java Pseudocode Jones and Bartlett. [9] Pevac, I. ESRATEA Educational Software for Recursive Algorithm Time Efficiency Analysis WORLDCOMP'10 Proc. of the 2010 Int. Conf. on Frontiers in Education: Computer Science & Computer Engineering, (Ed. Arabnia, Clincy), Las Vegas, NV, SCREA Press, 2010, pages 367-373. [10] Sedgewick, R., Wayne, K. (2011) Algorithms (4th ed.). Addison Wesley. [11] Shaffer C. (1998) A Practical Introduction to Data Structures and Algorithm Analysis Prentice Hall.