Math 4242 Fibonacci, Memoization lecture 13 Today we talk about a problem that sometimes occurs when using recursion, and a common way to fix it. Recall the Fibonacci sequence (F 0, F 1, F 2,...) that occurs naturally in many places in nature: F 0 = 0 and F 1 = 1, and for n > 1, F n = F n 1 + F n 2. Because the sequence is defined recursively, it is not surprising we can write a recursive function that computes the sequence: def fib(n): Computes Fibonacci sequence recursively track total function calls using a global variable global fibcallcounter fibcallcounter +=1 Check for the base cases fib(0) and fib(1) first so the recursion eventually ends, otherwhise use recursion if n==1: return 1 elif n==2: return 1 else: val = fib(n-1) + fib(n-2) print "Determined (with help) that fib(",n,") = ",val return val fib(6) see how calls to fib function grows as input n grows for i in range(1,1): print i, fib(i) print "took ", fibcallcounter, "calls to fib function" 1
It is instructive to consider a table for n = 10, 20, 30,... to get an idea of the algorithmic complexity ( cost ) of this algorithm. One sees that when n gets around 50 this function slows down quite noticably and then seems to stop working. As you will see on the assignment, using this algorithm to compute the F n requires almost 2 n calls to the fib function (in total counting calls at lower levels of the recursion hierarchy). As a rule of thumb Modern computers can do 2 64 things in a day where each thing is a very simple operation like one of the state transitions we talked about so long ago. So we could never use this algorithm to compute F 100, for example, as that would require 2 100 64 = 2 36 = 68719476736 days (188 million years). THE POINT?: Algorithms where the running time is an EXPONENTIAL function of the input can only deal with small inputs. Why is the program taking so long? Thinking about the tree of recursive function calls made by this algorithm is helpful. See the diagram of function calls. Notice how visualizing the recursion highlights the fact that the computer is solving the same subproblem over and over again. In a situation like this there are techniques that can help. It is only useful when the computer is spending most of its time solving the same subproblems over and over again. This can happen somewhat in recursive functions but also in other areas. There are many ways to solve this problem once we recognize it. The simplest is to REMEMBER the results of our previous work! For the Fibonnacci sequence, we can remember results very easily: we ll just create a global array to store the various Fibonacci numbers once we ve computed them. See attached code. This process of remembering is sometimes called Memo-ization (from memo/table ). Also called table lookup. The memo-ized version of the recursive Fibonnacci function has complexity O(n). Again visualizing the recursive call tree makes this clear the tree is still about n levels deep but it doesn t keep branching out like the first time. This keeps the total number of calls to the fib function a small constant times n. Interestingly, most people, if asked to compute the n th Fibonacci number (say n=10) will compute them starting at the smallest and working up, so computing F 0, F 1, F 2, F 3, and so on. In this case they will implicitly be remembering and using their previous work. The algorithmic version of this results in an O(n) algorithm also. So recursion is a beautiful idea and allows for slick little functions to be written (less code) but sometimes important details are hiding underneath. Recursion is essentially the mathematical induction technique. Many problems that can be solved by induction have a nice recursive method associated with it. You may know of another method for finding the n th Fibonnacci number: In particular the algorithm utilizing the closed-form expression formula for the nth Fibonacci number, namely F n = φn (1 φ) n 5, where φ = 1+ 5 2. (1) An algorithm implementing this function requires approximately 4 log 2 (n) multiplications and a small number (independent of n) of other arithmetic operations, and so is a O(log 2 (n)) algorithm and thus is much faster than the above algorithms. However this algorithm is very specific to the Fibonacci numbers and has a lot of knowledge about the numbers built-in to the formula, so not surprisingly it is the fastest. 2
3
4
Example of a recursive function (to compute fibonacci numbers) that uses the technique of memoization MemoFib = [0]*1000 MemoFib[1]=1 MemoFib[2]=1 we ll remember up to 1000 numbers def fibm(n): global fibcallcounter fibcallcounter +=1 needed to change a variable defined globally/out if MemoFib[n]!= 0 : we ve seen this before... return remembered value return MemoFib[n] MemoFib[n] = fibm(n-1) + fibm(n-2) return MemoFib[n] n=300 print( "Fibonnacci number ", n, " is ", fibm(n) ) print( "fibm took ",fibcallcounter, "calls." ) 5