Tail Recursion 1 Tail Recursion In place of loops, in a functional language one employs recursive definitions of functions. It is often easy to write such definitions, given a problem statement. Unfortunately, in many cases, the naive recursive solution is grossly inefficient. We discuss here a heuristic, that can be used in many cases, to transform programs to more efficient programs. We use our evaluation model as a basis for estimating the costs of running a program on some arguments. The time complexity is the number of steps. Each configuration in the computation is represented by an expression; we take the size of the expressions as the space cost. Space complexity: Here is a function for computing the factorial. ;; a recursive program for factorial (define fact (lambda (m) ;; m is non-negative (if (= m 0) 1 (* m (fact (- m 1)))))) Here is an outline of the computation from (fact 6). (fact 6) + (* 6 (fact (- 6 1))) + (* 6 (* 5 (fact (- 5 1))) + (* 6 (* 5 (* 4 (fact (- 4 1))))) + (* 6 (* 5 (* 4 (* 3 (* 2 (* 1 1))))))) + (* 6 (* 5 24)) + 720 Here, fact is invoked seven times, with arguments 6, 5, 4, 3, 2, 1, 0. In the expression (* 6 (fact (- 6 1))), the second argument of the * operation, namely (fact (- 6 1)) needs to be evaluated. While it is being evaluated, the (* 6 part of the expression is pending waiting for the second argument of the multiplication to become available. As the evaluation unwinds, more partial expressions become pending. These form a belly, that is clearly seen above. The space complexity is linear. Here is another program for the factorial: ;; an "iterative" program for factorial (define fact1 (lambda (m) ;; m is non-negative (letrec ( (fact-helper (lambda (acc counter) ;; Assertion: acc is factorial(counter-1) (if (> counter m) acc
Tail Recursion 2 ;in (fact-helper 1 1)))) (fact-helper (* counter acc) (+ counter 1)))))) The helper here is essentially a loop, that computes the values of the factorial, starting from 1. Here is an outline for the computation of (fact1 6) (arrows omitted), and look, the belly is gone. This program takes constant space. (fact1 6) (fact-helper 1 1) (fact-helper 1 2) (fact-helper 2 3) (fact-helper 6 4) (fact-helper 24 5) (fact-helper 120 6) (fact-helper 720 7) 720 Here, as in the first version, the recursion is linear. However, here the recursive call occurs at the end of the evaluation of the body, so no expressions remain pending. Hence, it takes only constant space. Time complexity: The evaluation of an expression typically involves many function activations. Let us briefly consider what is an activation. A function activation is the sequence of steps that takes place when a function application is evaluated. It starts from an expression of the form ((lambda pars body) args), where args is a sequence of values. If it ends, then this expression has been reduced to a single value; this is the return value of the application. Note, given an expression of the form (e 0...e n ), the steps spent to evaluate e 0 to a lambda expression, and to evaluate the argument expressions to values are not part of the activation. In the computation of (fact 6) above, there are seven activations. The first expression of the first one is ((lambda (m) (if...)) 6). The belly occurs since each activation but the last takes up space for a partial expression representing pending computation, while nested activations are evaluated. It turns out that the time complexity of a computation is proportional to the number of activations in it: Claim: The number of steps performed in the evaluation of an expression, excluding those performed inside function activations that occur in it, is linearly bounded by size of the expression. Since the body of a given function has a fixed size, the above implies that its run-time on arguments is proportional to the number of activations in that run (details omitted).
Tail Recursion 3 Now, let us look at an exmaple for time complexity. Consider the following program for computing the Fibonacci numbers: (define fib (lambda (m) ;; m is a non-negative integer (cond ((= m 0) 1) ((= m 1) 1) (else (+ ( fib (- m 1)) ( fib (- m 2)))) ))) In a computation of (fib n), for a large n, there are activations of the (value of fib as follows: one with argument n; one with argument n-1; two with argument n-2 (one invoked by the computation of (fib n), the other by the computation of (fib (- n 1))); 1+2 = 3 activations with argument n-3; (fib k) activations with argument n-k. The last claim is shown by induction. The total number of steps is therefore proportional to the sum of the first n Fibonacci numbers exponential in n! 1 Note that the tree of all activations has depth n. But each internal node has two children, hence the size of the tree, which is the number of activations, is exponential. Here is another program for the Fibonacci numbers. (define fast-fib (lambda (m);; m is a non-negative integer (letrec ( (fib-helper (lambda (acc1 acc2 counter) ;; Assertion: acc1 is Fibonacci(counter - 1) ;; acc2 is Fibonacci(counter) (if (>= counter m) acc2 (fib-helper acc2 (+ acc1 acc2) (+ counter 1))))) ) ;in (fib-helper 1 1 1)))) Here are steps in a computation of this program: (fast-fib 6) (fib-helper 1 1 1) (fib-helper 1 2 2) (fib-helper 2 3 3) (fib-helper 3 5 4) 1 See the Blue Book, p. 37 38 for details.
Tail Recursion 4 (fib-helper 5 8 5) (fib-helper 8 13 6) 13 There is a small constant number of steps between each of these expressions, that is the steps performed in an activation of fib-helper. Here also, note the similarity of the computations of fib-helper to those performed by a loop. Tail recursion: In general, evaluation of a function body may take several different paths for different argument values, since it may contain conditionals, and these may lead to different branches in different computations. We say that a subexpression of the body, that is a function application, is in a tail position, if whenever it is reached in some computation, it is then the full expression of the current state of the computation, so its evaluation is all that needs to be done to finish the computation. For example, in (if (f 1) (g 2) (h 3)) (f 1) is not in a tail position, since after its value is computed, either (g 2) or (h 3) still needs to be evaluated. Thus, we need to keep the original if expression and our position in it when we start the activation of (f 1). But, after it is evaluated, say to #t, then we can keep only the subexpression (g 2), whose value is the value of the whole expression. Thus, the applications (g 2) and (h 3) are in tail positions. A function (or a program a collection of functions) is called tail recursive if all recursive calls in it are in tail positions. It can be seen that both optimized programs above, for the factorial and for the Fibonacci numbers, are tail recursive. Tail-recursive programs mimic closely the loops found in programs written in an imperative language such as C. Compare the code and the executions of the following C program with the code and executions of fact1. int fact(int m){ int acc = 1; int counter = 1; while(counter <= m){acc = acc * counter; counter++} return acc; } This is a general observation: tail-recursion induces the same flow as a loop. Further, any loop can be simulated by a tail recursive program. Although functional programs do not use assignment, the repeated calls to the same function have the effect of using the parameters with different values, and this
Tail Recursion 5 mimics the repeated assignments to variables in a loop. In a sense, loops are just syntactic sugar for (tail) recursion. (But, they do make life much sweeter). The heuristic of introducing extra accumulators for transforming general recursion into tail recursion works in many cases. But, do not be misled to believe that all programs can be optimized (easily) by it. A final comment: We have assumed that cost estimates based on our model reflect in a reasonable manner the actual costs to run our programs on a Scheme system. Is this assumption justified? We do not go into details, but the answer is yes. In particular, the Scheme specification requires implementations to recognize function applications in tail recursive positions, and optimize their execution as follows: For a tail-recursive application in activation A1, rather than starting a new activation A2 but still keeping A1 on the run-time stack, it replaces A1 on the stack by A2. Such implementations are called proper tail recursive. This gives us the required guarantee, that the optimizations produced above indeed reduce the space/time complexity as claimed.