15. Recursion and Search Algorithms

Transcription

1 15. Recursion and Search Algorithms Recursion is a powerful mathematical and programming technique for certain kinds of problems. In programming, it manifests itself as a function that calls itself! Recursion is appropriate for several kinds of problems: Those whose nature is defined in a recursive way. Some that use a divide and conquer strategy. This section of the course will introduce you to recursion, show you how to program it, and illustrate when it is NOT appropriate to use. You will also get a brief introduction to search algorithms, which we will use to illustrate the difference between an iterative and a recursive implementation. Readings: Ch. 11 of [Savitch2001]. 15. Recursion and Search Algorithms Introduction To Recursion Iterative vs. Recursive Examples Infinite Recursion Indirect Recursion Efficiency and Tail Recursion (Advanced) Wasteful Recursion Identifying Tail Recursion (Advanced) Converting to Tail Recursion (Advanced) Search Algorithms Binary Search Appendix A (Advanced) Why Organize Data Properly? B+ Trees Copyright 1998 by R. Tront 15-1 Copyright 1998 by R. Tront 15-2

2 15.1 Introduction To Recursion Sometimes it is hard to write a mathematical function in an purely analytical way by using just an expression. Some functions, some which are even widely used, unfortunately cannot be expressed using a proper mathematical expression. One that comes to mind immediately is the factorial function, which many of you will see in your math courses over the next few years. The factorial function isn t even written like a function. i.e. not like sine(angle). Instead, for reasons I do not know the history of, N factorial is written as N!. The factorial function is defined like this: N! = N x (N-1) x (N-2) x... x 1 So: 2! = 2 x 1 = 2 3! = 3 x 2 x 1 = 6 4! = 4 x 3 x 2 x 1 = 24 et cetera. The definition shown above is an iterative one in nature, rather than a closed mathematical expression. Copyright 1998 by R. Tront Iterative vs. Recursive Examples Here are two simple iterative functions in Java to perform this calculation. One uses a for loop that counts down, and the second uses one that counts up. static long factorialdown(int n){ //Note: does not work correctly for //n==0. int result = n; for(int i = n-1; i >= 1; i--){ result = result * i; //Note that this loop doesn t even //run once unless n>1. return result; // static long factorialup(int n){ int product = 1; for(int i=2; i<=n; i++){ product = product * i; return product; The above factorial function definition, and the two implementations in Java, use an interative loop. Copyright 1998 by R. Tront 15-4

3 Some mathematicians prefer to define the factorial function recursively. They would write: factorial(n) = 1 if n=0 n x factorial(n-1) when n>0 We see then that factorial(0) should be 1. But our former definition was not this clear -- which is why mathematicians prefer this latter one. And our function factorialdown(0) has a bug that will give the result 0 rather than 1 -- which is why programmers prefer recursive functions. You can fixed factorialdown( ) with an if statement, or just use factorialup( ) for the correct evaluation of this function. Modern languages that provide both: automatic local variables in functions and call by value usually allow a function to call itself. Though the first example below will not use local variables, they are important in that they give each invocation/call of a particular function (including calls from itself) its own new local variables to work with. The same is true of call by value parameters (recall that the formal parameters inside a function are very similar in behavior to automatic local variables). Here is the factorial function written in a recursive style in Java: long factorialrecurs(int n){ if (n==0) return 1; else return(n * factorialrecurs(n-1)); Notice how simple and intuitive it is. The interesting thing about recursive implementations of algorithms is that they almost always exactly resemble the recursive mathematical definition. For that reason you are less likely to make a mistake writing such a Java implementation. Copyright 1998 by R. Tront 15-5 Copyright 1998 by R. Tront 15-6

4 15.2 Infinite Recursion Your computer will compute forever and eventually crash if you, by mistake, allow infinite recursion. All recursive algorithms must have a stopping condition. All recursive programs must have an if statement that decides whether to stop or recurse further. If you leave this statement out, or if you fail to program it precisely, you risk infinite recursion. This is no worse than an infinite loop (your program and perhaps your computer locks up, because it is so busy recursing or runs out of RAM memory needed for recursing). Like loop ending conditions, you must pay close attention to each recursion ending condition Indirect Recursion Sometimes, you might write several functions that among themselves are indirectly recursive. E.g. function1( ) calls function2( ), which in turn calls function1( ). This is fine, as long as the recursion eventually stops. Copyright 1998 by R. Tront 15-7 Copyright 1998 by R. Tront 15-8

5 15.4 Efficiency and Tail Recursion (Advanced) Recursion has a very small penalty in performance and memory. This is because recursive algorithms make many more function calls than an iterative algorithm. A function call causes a very small delay and very small mount of memory (as low as 4 bytes extra) to be used in a program. If recursive algorithm is tail recursive (to be discussed shortly), you can eliminate the slight inefficiency of recursion by re-writing it as an iterative function. Sometimes, good compilers will even notice tail recursion and generate iterative code in the.class file for your otherwise recursive function. Since it is extremely hard to write some kinds of algorithms without recursion (e.g. some artificial intelligence programs that need backtracking ), we use recursion when it makes sense to do so. Therefore, recursion should used when: the algorithm is difficult or error prone to write nonrecursively. the algorithm is not very wasteful in its recursion (i.e. doesn t call itself several times with the same arguments). you will not need to recurse very deep (no more than say 100 deep?). Copyright 1998 by R. Tront 15-9 or you are using a special computer language that has no loops and iteration, and instead relies on recursion! Yes, there are languages without iteration loops! In fact, they are some of the most powerful artificial intelligence and so called functional languages. They are pretty amazing because they don t really have variables either. They work entirely on functions and function return values! This goes to show you that computing science is a lot more than just hack out code in C++ or Java. Copyright 1998 by R. Tront 15-10

6 Wasteful Recursion The Fibonacci number series is: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,... Each number is the sum of the previous two. The recursive mathematical definition of this series is: fib(0) = 0 fib(1) = 1 fib(n) = fib(n-2) + fib(n-1) if n>1 To program this is easy: overhead of the function call mechanism; it is just a lousy algorithm. Very Good Exercise: Write an efficient version for fib(). Hint: try it iteratively, computing each element of the series linearly. You will need to store the last two elements of the series for use in the next iteration of the loop. static long fib(int n){ if (n=0) return 0; else if (n=1) return 1; else return (fib(n-2) + fib(n-1)); Unfortunately this is a very inefficient algorithm. If you try computing fib(6), in the process you will find that fib( ) will be called three times with an argument of 3, and five times with an argument of 2. Why would you want to compute fib(2) five times? This is an example of a recursively wasteful algorithm. This waste has nothing to do with the administrative Copyright 1998 by R. Tront Copyright 1998 by R. Tront 15-12

7 Identifying Tail Recursion (Advanced) A tail recursive algorithm is one in which the recursive call is the last thing done in the function. It doesn t really matter whether the recursive call is physically at the top or bottom of the function, as long as for the recursive if case(s), it is the last thing done before the function return. I.e. you have to add some variables after the recursive call but before returning from/ending that call to the function. The Fibonacci function is NOT tail recursive because the last thing done before returning is the multiplication with the returned value from the further recursive call. long factorialrecurs(long n){ if (n==0) return 1; else return(n * factorialrecurs(n-1)); And the recursive Fibonacci function is not tail recursive. Below is another algorithm that is definitely not tail recursive. In addition to literally doing a complete extra statement after the recursion, it also nicely illustrates the use of local variables. It basically reads in a string of n characters, one at a time, then prints Copyright 1998 by R. Tront them out in reverse. It actually reads the characters as it is going down into the depths of the recursion, and prints them out after returning from each recursion. Note that it has no return value; it outputs its results directly to cout! void printinreverse(int n){ char thechar; thechar = SavitchIn. ReadLineNonwhiteChar( ); if (n!=1){ printinreverse(n-1); System.out.print(theChar); else { System.out.print(theChar); It may be a little hard initially for you to envision how this function works. If you call this with n=3, then think of there being 3 separate copies of this function, each with their own input parameter and each with their own local variable thechar. The first copy of the function reads into its local variable the first character, and then calls the second one with a reduced count (2) of how many more to read. Copyright 1998 by R. Tront 15-14

8 The second one reads in the second character into its local variable, and then calls the third telling it to read in 1 more. The third does that, outputs the third character, and returns to the second. The next statement after the call within the second causes the second to print out its local variable (which contains the second character). The second then returns to the first, which is waiting to print out the first character which it has in its local variable. This is a silly little program but illustrates how some very interesting kinds of reversal and backtracking algorithms can be written. Additionally, it illustrates that not all recursive algorithms are simple to understand (though once you get the hang of it, you don t really need to worry about how this actually works. You just begin to trust recursion). The above function illustrates an algorithm that is definitely non-tail recursive (i.e. one that is hard to simply convert to iterative), and also one which has local variables. Each invocation of the function has a new different local variable (even though the local variable in each copy has the same name). Fortunately, many algorithms are by their nature tail recursive or can be turned into tail recursive algorithms. If a program is in tail recursive form, some compilers which are particularly smart will turn the tail recursion Copyright 1998 by R. Tront into the iterative form when they generate the machine code for the function! This totally removes even the small inefficiency of recursion. This is wonderful as it allows you to write a function in a nice clear recursive form, yet you suffer none of the overhead of recursion at run time! Copyright 1998 by R. Tront 15-16

9 Converting to Tail Recursion (Advanced) Here is another silly little program that is not tail recursive, but with a tiny change can be made tail recursive. It calculates the sum of A and B. It does this by adding 1 to A B times. short sum(short a, short b){ if (b==0) return a; else return (sum(a,b-1) + 1); Basically, this passes a down the recursion and then successively adds 1 to it on the way back out of the recursion. The b parameter only controls how deep the recursion goes. Unfortunately, the last thing done before returning from the recursive case is an addition, not the actual recursive function call. However, some non-tail recursive algorithms can be re-written into tail recursive form: In this version of the program, one is added to the first parameter a, on each recurse on the way down, and that same final result is passed back by every single function return value in the back out process. Most artificial intelligence and so called functional languages automatically convert tail recursion into iteration (even though they don t allow the programmer to use iteration). And some imperative programming language compilers (the normal ones like C++ and Java that you are used to) optimize away tail recursion. I have heard that Java will soon do this. Note that the overhead of recursion is not much of a burden unless: you are recursing very deep (>100 times), or each invocation requires a LOT of local variable space for each recurse. short sum(short a, short b){ if (b==0) return a; else return sum(a+1,b-1); Copyright 1998 by R. Tront Copyright 1998 by R. Tront 15-18

10 15.5 Search Algorithms It is often required to search large amounts of data or databases for a desired piece of data. Think of trying to find the record of a university student in the university s database of all present and past students. In order to make this very fast, it is common to (somehow) organize the data in sorted order. This course has not covered sorting in much detail, but you will study it in Cmpt 201 (including recursive sorting techniques!). However, back to topic of searching. If you have a very large array or file of student records, and you want to find out the phone number of the one whose number is , there are better ways than using a linear search. Remember, though, that if the data is not sorted then a linear search of the array, file, or linked list is your only option. Very Good Exercise: Can you write a linear search using recursion? (e.g. if the record you are examining is not the one you want, move to the next record, then call yourself recursively) Binary Search If you have an array of say 30,000 students, to find one using a linear search will require, on average, searching 15,000 students. Sometimes you will be lucky and find the one you want in the first dozen. Sometimes you will be unlucky and have to search to the very last one to find the one you want. But on average, you will need to search about half the records to find the one you want. Would you be surprised if I told you there is another way to do this that would only require searching log 2 (30,000) = 15. This is a factor of 1/1000 of the time needed to find the student s record! This provides a FANTASTIC increase in performance. To do this requires two criteria: 1) that the data be alreadly sorted according some search key (e.g. the student number). 2) that the binary search algorithm is used. The binary search algorithm basically keeps dividing the problem in half (that s why it is called binary). After 15 iterations or recursions, it can find one particular student among 30,000. Copyright 1998 by R. Tront Copyright 1998 by R. Tront 15-20

11 Note: Do not assume that student numbers are given out in sequence; there are big gaps from one year to another. Let s assume the following record class: class Student{ long studnum; //the search key. StudentData data; Let s define our function signature: public static long binarysearch( long id, Student[] thearray, long firstindex, long lastindex); Next, let s illustrate how an application programmer would use this function: class SearchDemo{ static Student thestudents = new Student[30000]; public static void main( String[] args){ //assume students are read in //here, then sorted lowest first //according to student number. long id; System.out.print( Enter student number: ); id = SavitchIn.readLineLong(); index = binarysearch( id, thestudents, 0, thestudents.length-1); if(index == -1) System.out.println( Not found! ); else System.out.println( thestudents[index].data); //end main. //end class. Copyright 1998 by R. Tront Copyright 1998 by R. Tront 15-22

12 Now lets look at how we can do the search in 1/1000th of the time that it would take if otherwise using linear search: Copyright 1998 by R. Tront public static long binarysearch(long id, Student[] array, long first, long last){ //Four different returns in this //function. if(first > last) //Then have partitioned too far. return -1;//indicating not found! //Now an important local var. int mid = (first + last)/2; if (array[mid].studnum == id) //then we have found the one. return mid; else if(id < array[mid].studnum) //then desired instance has an //index less than mid. return binarysearch(id, array, first, mid-1); else //the desired instance must have //an index greater than mid. return binarysearch(id, array, mid+1, last); Copyright 1998 by R. Tront 15-24

13 Note that we pass both the upper and lower limit of the array. We can t just pass the array size, because the second and subsequent recursions are working on only the bottom or top half of the array. The third invocation is working on either the 1st, 2nd, 3rd, or 4th quarter of the array. And so it proceeds narrowing its search by half on each recursion. If the student you are looking for is in the lower half of the array, this means you saved yourself linearly searching the whole upper half of the array, etc. This is why the number of recursions is proportional to log 2 (MAX_STUD). After no more than 15 (or less if you are lucky) recursions, you will have found the desired student. The index of the found student is passed back via the return value of every function call all the way back up from the recursive depths. Advanced Question: Is this binary search algorithm tail recursive? Exercise: Since it is, try programming binary search using a non-recursive algorithm. It is not hard at all, but having seen the recursive algorithm, you might have to take a serious moment to change your way of thinking. There are a lot of sorted files, and also ordered linked tree structures that are amenable to recursive searching. Copyright 1998 by R. Tront Most of these require so few recursions, even if they contain millions of records, that worrying about whether your brand of compiler will actually optimize away the tail recursion is not really that important because searching even a million records would require only 20 recursions. I would like to give you a peak into some of the advanced data structures that you will cover in Cmpt 201, 307, and 354, look at the following Appendix. It contains the most widely used database lookup structure called a B+ tree. Note that we may not have covered linked records yet, but basically an object can refer to another object which can refer to another object, thus allowing us to build very interesting linked structures in RAM memory or on disk, as is shown in the appendix. Tree structures are particularly amenable to being searched also using recursive algorithms (though it not binary search which is data that is layed out in RAM array or file in actual sorted order). Note: B+ trees are useful because the data does not have to be layed out in RAM or file in sorted order. And of course requiring that the data be layed out in sorted order has two drawbacks: First, you have to go to the trouble of sorting the data, and sorting is quite computationally burdensome. Copyright 1998 by R. Tront 15-26

14 Second, if you have to add a new record in its correct position within a sorted array or file, then you have to move every record in latter half of the array to the right by one element, which is also burdensome performance-wise Appendix A (Advanced) This subsection's notes are taken from Cmpt 275 and 370. Every computing student must learn this material to graduate, Cmpt 212 students must read it, and you may need it on assignments, but I will not test you on it. For Cmpt 101, it is optional but very interesting reading. Copyright 1998 by R. Tront Copyright 1998 by R. Tront 15-28

15 Why Organize Data Properly? Information systems, by definition retain persistent data. Persistent data, usually, but not always, mean data retained overnight, between program executions, or over a power shutdown. Integers in RAM memory are also `retained' for shorter times, but the problems of informations systems are usually that of: storing huge amounts of inter-related data, for long periods of time, and being able to rapidly create outputs which have been composed from the retained data, or to be able to easily and rapidly modify particular data records! Interestingly, learning how to organize persistent data gives us wonderfully clear insights as to how to also organize even our non-persistent (RAM) objects! Hard disk drives have access times typically in the order of seconds (16 ms). Unfortunately, this is still a million times slower (i.e. 6 orders of magnitude) than the instruction time of a 66 MIPS processor! To obtain fast (relative to the processor) random access times requires very special techniques to be used in data bases. Invariably, this requires that retained records in a file all be the same length. That way, to read the 1000th record, you can simply seek the read head of the Copyright 1998 by R. Tront disk drive a distance (999 x Record_Size) from the beginning of the file, and thus avoid reading all the records earlier in the file. It also means that when a record is deleted, then another one added, the added one can fit in the same length space as the one deleted. (If the added record were shorter, some space would be wasted. If it was longer, it wouldn't fit and would have to be placed elsewhere and the entire space from the deletion would be wasted until a smaller record needed to be added.) But not all data records appear to be of fixed length. You will learn in your 2nd and 3rd year course how to reorganize it in a way that it can be stored in several files, each of which at least, use fixed length records. Properly analyzing the data structure for such an application is done by constructing a data model of the information that needs to be retained, and modifying that model until you get: 1) fixed length records. 2) no wasted space due to empty or duplicated data. 3) fast random and sequential access. 4) fast insert and delete operations (not possible with simple sorted records - why?) Copyright 1998 by R. Tront 15-30

16 B+ Trees This sub-sub-section gives a wonderful introduction to basic database indexed trees and links. Generally, characteristics 1) and 2) above are very important tasks of the Analyst. The latter characteristics are achieved by writing or purchasing database software that uses the Indexed Sequential Access Method (ISAM) which is based on B+ tree index structures on disk. This special kind of tree is very short in height, thus requiring very few disk accesses to traverse down to any randomly-chosen leaf. The tree is always balanced so that no branches are very deep. Additionally, the leaves are accessible via a linear, doubly-linked (disk) list. The use of a tree makes random searches, inserts, and deletes as fast as is possible on disk. The sequential links make access of the `next' and `previous' items in the sorted structure even faster. Disk access speed is important for two reasons: 1) Disks are orders of magnitude slower than RAM. We absolutely must minimize the number of disk accesses needed. 2) Information systems store huge amount of persistent information. 3) The solid state RAM memory in computers is too small and too valuable to store the large amounts of data needed for an information system. RAM is instead used for running programs and frequently used variables. In addition, RAM is volatile; all data in it is lost when the computer is turned off or the power or computer fails. One mistake that Cmpt 275 students often make is, when needing to modify a list of entities in a disk file, they wrongly read the whole data file into RAM and built it into a linked RAM list! You should presume the data files for most information systems (and the application you will be developing in 275) are too big and thus cannot totally fit in RAM. This is slow and silly. But Cmpt 201 graduates often think lists and trees only exist in RAM. This is not necessarily their fault as most 201 instructors do not insist on giving an assignment that allows students to build a tree (or something) on disk. And often students from Cmpt 101 do not understand random access file I/O, where data in the middle of a file can be written without disturbing/ invalidating the data further down the file. The pointers in a file tree, unlike in a RAM tree, are no longer RAM addresses. They are either record numbers or byte numbers measured from the beginning of the file. To find/insert/delete a particular single record, all that is needed is to call a seek function which positions Copyright 1998 by R. Tront Copyright 1998 by R. Tront 15-32

17 the disk head at the correct position in the file. The trick is knowing where to find that record. Here are 3 advantages of B+ trees: 1) B+ trees are great as they have an tree part which allows fast random access (see next diagram). Not only that, the nodes in the tree part are often a full disk sector in size (512 bytes) and thus provide a high order tree (one which many arrows coming out of each node). This makes the tree height short, to reduce the number of disk accesses. Here is a diagram of a B+ tree. J C G M T Bill Celine Harry Kathy Peter Valerie Copyright 1998 by R. Tront Copyright 1998 by R. Tront 15-34

18 2) In B+ trees (not to be confused with B trees or binary trees), the leaves of the tree are special nodes that are doubly-linked to additionally provide rapid sequential access to the `next' or `previous' record. 3) Third, the data is not in the leaves or nodes of the tree. Typically the data is in a separate file from the index nodes. The tree and sequential index nodes are normally in a special file called an `index' file. The leaf nodes simply contain the record number of the data records in a data file (as well as the forward and backward links). The advantage of this is that you can have a single data file of persons as shown above, and MULTIPLE index files/trees pointing into it (2 in the above case). One index file is for fast random access by, say, name. And the second index tree is for fast random access by, say, driver's licence number! Actually, one index file is for general access by name, fast random access (of order log N), and very fast sequential access (of order log 1). This is the fundamental disk data structure underlying almost every database management system! Copyright 1998 by R. Tront 15-35