CDM Other Models. Klaus Sutner Carnegie Mellon Universality. Fall 2017

Transcription

1 CDM Other Models Klaus Sutner Carnegie Mellon Universality Fall 2017

2 1 General Recursion Herbrand-Gödel Computability µ-recursive Functions Church-Turing Thesis

3 Where Are We? 3 We have encountered two clones of intuitively computable functions: the primitive recursive functions and the register machine computable functions (following Turing, these arguably capture the notion of computability perfectly). By nesting recursions deeply, one can produce primitive recursive functions that are nowhere near computable in any practical sense of the word (see below). So the questions is: is every computable function primitive recursive? And, if not, how far off are we? How much do we have to add to p.r. to get all computable functions?

4 Are We There Yet? 4 The key idea behind primitive recursive functions is to formalize a certain type of recursion (or, if you prefer, inductive definition). As we have seen, even some extension of this basic idea like course-of-value recursion remains within this framework. Here is another example: simultaneous recursion. To simplify matters, let us only consider two simultaneously defined functions as in f i (0, y) = g i (y) f i (x +, y) = h i (x, f 1 (x, y), f 2 (x, y), y) where i = 1, 2.

5 Lame Example 5 f 1 (0) = 1 f 1 (x + ) = f 2 (x) f 2 (0) = 0 f 2 (x + ) = f 1 (x) Then f 1 is the characteristic function of the even numbers. We ll see better examples later when we talk about context free grammars. Lemma Simultaneous recursion is admissible for primitive recursive functions.

6 Ackermann s Function (1928) 6 And yet, there is more to recursion that this. Define the Ackermann function A : N N N by the following double recursion (this version is actually due to Rózsa Péter and simpler than Ackermann s original function): A(0, y) = y + A(x +, 0) = A(x, 1) A(x +, y + ) = A(x, A(x +, y)) On the surface, this looks more complicated than primitive recursion. Of course, it is not at all clear that Ackermann could not somehow be turned into a p.r. function.

7 Ackermann is Computable 7 Here is a bit of C code that implements the Ackermann function (assuming that we have infinite precision integers). int acker(int x, int y) { return( x? (acker( x-1, y? acker( x, y-1 ): 1 )): y+1 ); } All the work of organizing the nested recursion is handled by the compiler and the execution stack. So this provides overwhelming evidence that the Ackermann function is intuitively computable.

8 Memoizing 8 We could memoize the values that are computed during a call to A(a, b). It is not hard to see that whenever A(x, y) is in the hash table, so is A(x, z) for all z < y (except for x = 0). Think of the table as having x rows. But note that the computation of A(a, b) will require entries A(a 1, z) for z > b. So each row in the table gets longer and longer as x gets smaller. For example, the computation of A(3, 3) = 61 requires A(2, 29) and A(1, 59). Note that the hash table provides a proof that A(3, 3) = 61.

9 Family Perspective 9 It is useful to think of Ackermann s function as a family of unary functions (A x ) x 0 where A x (y) = A(x, y) ( level x of the Ackermann hierarchy ). The definition then looks like so: A 0 (y) = y + A x +(0) = A x (1) A x +(y + ) = A x (A x +(y)) From this it follows easily by induction that Lemma Each of the functions A x is primitive recursive (and hence total).

10 The Bottom Hierarchy 10 A(0, y) = y + A(1, y) = y ++ A(2, y) = 2y + 3 A(3, y) = 2 y+3 3 A(4, y) = The first 4 levels of the Ackermann hierarchy are easy to understand, though A 4 starts causing problems: the stack of 2 s in the exponentiation has height y + 3. This usually called super-exponentiation or tetration and often written y x or x y.

11 The Mystery of A(6, 6) 11 Alas, if we continue just a few more levels, darkness befalls. A(5, y) super-super exponentiation A(6, y) an unspeakable horror A(7, y) speechless For level 5, one can get some vague understanding of iterated super-exponentiation, but things start to get murky. At level 6, we iterate over the already nebulous level 5 function, and things really start to fall apart. At level 7, Wittgenstein comes to mind: Wovon man nicht sprechen kann, darüber muss man schweigen.

12 Ackermann and Union/Find 12 One might think that the only purpose of the Ackermann function is to refute the claim that computable is the same as p.r. Surprisingly, the function pops up in the analysis of the Union/Find algorithm (with ranking and path compression). The running time of Union/Find differs from linear only by a minuscule amount, which is something like the inverse of the Ackermann function. But in general anything beyond level 3.5 of the Ackermann hierarchy is irrelevant for practical computation. Exercise Read an algorithms text that analyzes the run time of the Union/Find method.

13 Ackermann vs. PR 13 Theorem The Ackermann function dominates every primitive recursive function f in the sense that there is a k such that Hence A is not primitive recursive. f(x) < A(k, max x). Sketch of proof. One can argue by induction on the buildup of f. The atomic functions are easy to deal with. The interesting part is to show that the property is preserved during an application of composition and of primitive recursion. Alas, the details are rather tedious.

14 Insane Growth 14 Informally, the Ackermann is not primitive recursive because it grows too fast. We will give another example of mind-numbing growth. To this end, it is easier to use a slight variant of the Ackermann function. B 1 (x) = 2x B k +(x) = B x k (1) So B 1 is doubling, B 2 exponentiation, B 3 super-exponentiation and so on. In general, B k is closely related to A k+1.

15 Subsequence Order 15 Recall the subsequence ordering on words where u = u 1... u n precedes v = v 1 v 2... v m if there exists a strictly increasing sequence 1 i 1 < i 2 <... i n m of positions such that u = v i1 v i2... v in. In symbols: u v. In other words, we can erase some letters in v to get u. Subsequence order is not total unless the alphabet has size 1. Note that subsequence order is independent of any underlying order of the alphabet (unlike, say, lexicographic or length-lex order).

16 Warmup: Antichains 16 An antichain in a partial order is a sequence x 0, x 1,..., x n,... of elements such that i < j implies that x i and x j are incomparable. Example Consider the powerset of [n] = {1, 2,..., n} with the standard subset ordering. How does one construct a long antichain? For example, x 0 = {1} is a bad idea, and x 0 = [n] is even worse. What is the right way to get a long antichain?

17 Higman s Lemma 17 Theorem (Higman s 1952) Every antichain in the subsequence order is finite. Proof. Here is the Nash-Williams proof (1963): assume there is an antichain. For each n, let x n be the length-lex minimal word such that x 0, x 1,..., x n starts an antichain, producing a sequence x = (x i ). For any sequence of words y = (y i ) and a letter a define a 1 y to be the sequence consisting of all words in y starting with letter a, and with a removed. Since the alphabet is finite, there exists a letter a such that x = a 1 x is an antichain. But then x 0 < x 0, contradiction.

18 Friedman s Self-Avoiding Words 18 For a finite or infinite word x write x[i] for the block x i, x i+1,..., x 2i. We will always assume that i x /2 when x is finite. Bizarre Definition: A word is self-avoiding if for i < j the block x[i] is not a subsequence of x[j]. The following is an easy consequence of Higman s theorem. Theorem Every self-avoiding word is finite.

19 How Long? 19 Write Σ k for an alphabet of size k. By the last theorem and König s lemma, the set S k of all finite self-avoiding words over Σ k must itself be finite. But then we can define the following function: α(k) = max ( x x S k ) So α(k) is the length of the longest self-avoiding word over Σ k. Note that α is intuitively computable: we can build the tree of all self-avoiding words by brute-force.

20 How Big? 20 Trivially, α(1) = 3. A little work shows that α(2) = 11, as witnessed by abbbaaaaaaa. But α(3) > B 7198 (158386), an incomprehensibly large number. Smelling salts, anyone? It is truly surprising that a function with as simple a definition as α should exhibit this kind of growth.

21 PR versus Computable 21 Functions like A, B or α are all intuitively computable, but fail to be primitive recursive. So how much do we have to add to primitive recursion to capture these functions? Proposition (Unbounded Search) There is a primitive recursive relation R such that A(a, b) = (min ( z R(a, b, z) ) ) 0 Think of z as a pair c, t where t encodes a data structure that represents the whole computation of the Ackermann function on input a and b, something like a huge memoization table. c is the final result of this computation. Here is a much better description.

22 Ackermann and Lists 22 A computation of A(2, 1) might start like A(2, 1) = A(1, A(2, 0)) = A(1, A(1, 1)) = A(1, A(0, A(1, 0))) =... Note that the A s and parens are just syntactic sugar, a better description would be 2, 1 1, 2, 0 1, 1, 1 1, 0, 1, 0 1, 0, 0, 1 1, 0, 2 1, 3 0, 1, 2 0, 0, 1, 1 0, 0, 0, 1, 0 0, 0, 0, 0, 1 0, 0, 0, 2 0, 0, 3 0, 4 5 We can recover the original expression by applying A like a right-associative binary operator to these argument lists. Clearly, we can model these steps by a function defined on sequences of naturals or, via coding, just naturals.

23 List Operation 23 (..., 0, y) = (..., y + ) (..., x +, 0) = (..., x, 1) (..., x +, y + ) = (..., x, x +, y) Since A is total, we know that for any a and b there is some time t such that t (a, b) = (c) Clearly this condition is primitive recursive.

24 A(3, 4) 24

25 The Diagonal 25 The diagonal function A(x, x) fails to be primitive recursive. But Claim The predicate A(x, x) = z is primitive recursive. Proof. We have A(x, y) > y. But then (L) L i + i 1.

26 Evaluation 26 Here is another obstacle to capturing computability: recall the evaluation operator for our PR language: eval(τ, x) = value of τ on input x It is clear that eval is intuitively computable (take a compilers course). If fact, it is not hard to implement in eval in any modern programming language. Question: Could eval be primitive recursive? A useless answer would be to say no, the types don t match.

27 Indices 27 The first argument of eval is a term τ in our PR language, so our first step will be to replace τ by an index τ N. The index τ will be constructed in a way that makes sure that all the operations we need on indices are clearly primitive recursive. The argument vector x N n will also be replaced by its sequence number x 1,..., x n. Hence we will be able to interpret eval as a function of type N N N and this function might potentially be primitive recursive.

28 Coding PR 28 C n 0, n Pi n 1, n, i S 2, 1 Prec[h, g] 3, n, ĥ, ĝ Comp[h, g 1,..., g n ] 4, m, ĥ, ĝ 1,..., ĝ n Thus for any index e, the first component (e) 0 indicates the type of function, and (e) 1 indicates the arity. There is nothing sacred about this way of coding PR terms, there are many other, equally natural ways.

29 Diagonalization 29 Now suppose eval is p.r., and define the following function f(x) := eval(x, x) + 1 Then f is also p.r. and must have an index e. But then and we have a contradiction. f(e) = eval(e, e) + 1 = f(e) + 1 So eval is another example of an intuitively computable function that fails to be primitive recursive. This example may be less sexy than the Ackermann function, but it appears in similar form in other contexts.

30 Avoiding Diagonalization 30 More precisely, given a good formal definition of computability (one that matches our intuitive ideas), we would still expect to have indices for the functions in this class. We also would expect to have something like an operation eval : N N N that works like an interpreter for functions in this class. Most importantly, we want eval to be itself computable. In light of the last result, this may sound like mission impossible, but we will see that things work out fine as long as we are willing to deal with partial functions.

31 General Recursion 2 Herbrand-Gödel Computability µ-recursive Functions Church-Turing Thesis

32 Models of Computation 32 As we already mentioned, there are several other ways to define computability that, on the face of it, use rather different approaches. It is interesting to see how some of these methods compare to what we already have. Note that it could happen in principle that each definition produces a new and separate kind of computability. As it turns out, this is not the case: all roads lead to Cairo. Computability is very, very robust and does not depend on little incidentals.

33 Equations and Computability 33 Here is another way to tackle the computability question that stays close to standard mathematical practice and uses equations to define arithmetic functions. We will allow the use of constant 0 and the successor function x x + 1. Recall that we often write x + rather than x + 1. Consider the following equation E: f(x) = x ++ Intuitively, E defines the operation plus two. For example f(2) = 2 ++ = 3 + = 4 This may seem utterly braindead, but hold on.

34 Numerals 34 The last chain of equations assumes basic arithmetic, but this is really backwards: we want our equations to define arithmetic, not the other way around. To fix this problem, we will use numerals denoting natural numbers: 0 1 = = = In general, we have precisely one numeral n for every natural number n. Writing 2 instead of 0 ++ is just syntactic sugar, nothing new is happening here. The previous example really should be written f(2) = f(0 ++ ) = = = 2 ++ = 3 + = 4

35 Example: Addition 35 Time for a slightly more ambitious example. As Dedekind pointed out in the late 19th century, we can define addition by the following system of equations E: f(x, 0) = x f(x, y + ) = f(x, y) + We can check that E really defines addition: f(3, 2) = f(3, 1 + ) = f(3, 1) + = f(3, 0 + ) + = f(3, 0) ++ = 3 ++ = 4 + = 5

36 Equational Reasoning 36 If we wanted to be strictly formal about this, we would need to spell out a few rules on how to manipulate our equations. You already know these rules from high school: Substitution: We can replace a free variable everywhere in an equation by a numeral. Replacement: A term f(a 1,..., a k ) can be replaced by a numeral b if we have already proven f(a 1,..., a k ) = b.

37 3 + 2 = 5 37 (1) f(3, 0) = 3 subst of E 1 (2) f(3, 1) = f(3, 0) + subst. E 2 (3) f(3, 2) = f(3, 1) + subst. E 2 (4) f(3, 1) = 3 + = 4 repl. (1), (2) (5) f(3, 2) = 4 + = 5 repl. (4), (3) Tedious and blindingly boring for a human, but completely mechanical and easy to implement in any language with good support for pattern matching.

38 Herbrand-Gödel Computability 38 This idea extends naturally to arbitrary systems of equations E: we can manipulate the equations to derive assertions about concrete numerals. With a little more effort we could define a notion of derivability (or provability), usually written E f(a 1,..., a k ) = b Definition A partial function f : N k N is Herbrand-Gödel computable if there is a finite system of equations E that has an k-ary function symbol f such that E f(a 1,..., a k ) = b f(a 1,..., a k ) b for all a i, b N.

39 Comments 39 Note that we assume f only to be a partial function: for some inputs a it may not be possible to derive f(a) = b for any b. This is a bit of a nuisance, but remember the comment at the end of the section on evaluation: divergence is your friend. Other, traditional names in the literature: partial recursive functions or general recursive functions. This is rather misleading, since recursion (or inductive definition) is not really the basic design principle here. Science advances one funeral at a time. Max Planck

40 Warning: Identity 40 Also note the mysterious notation α β This means that either both α and β are defined (the computations involved all terminate) and have the same value, or both α and β are undefined (some computation diverges). By contrast, we write α = β if convergence is not an issue. For example, for primitive recursive functions there is no problem. In computability theory, identity is more complicated than you might think.

41 Good News 41 We have not given a careful and detailed description of what we mean by a system of equations, but it should be rather clear that any primitive recursive function can be defined by such a system. The basic functions successor, zero and projection are all defined in terms of equations. Composition and primitive recursion are also defined in terms of equations (rename and combine). Exercise Verify the last two claims by writing down the appropriate systems of equations.

42 More Good News 42 Recall our definition of the Ackermann function: A(0, y) = y + A(x +, 0) = A(x, 1) A(x +, y + ) = A(x, A(x +, y)) It follows immediately that A is Herbrand-Gödel computable. Alas, that it is far from clear that the evaluation function from above is similarly Herbrand-Gödel computable (it is, but it is a pain to prove this directly, we ll take a different route).

43 Bad News 43 Not every system of equations defines a function. Here are two counterexamples (we ll drop the underscores from now on). f(0, 0) = 0 f(x +, y) = f(x, y + ) g(x, 0) = x + g(x, y + ) = g(x +, y) g(x +, y + ) = g(x, g(x, y)) Exercise Figure out what s wrong with these systems of equations.

44 Closing the Gap 44 Systems of equations are naturally very appealing to any math person, but less so from a computer science perspective: we need to filter out the right systems of equations. Unfortunately, as it turns out, there is no algorithm to do this: we can not tell whether a system of equations defines an arithmetic function. It is therefore rather tempting to try to expand primitive recursive functions in a different way, by adding some other construction method that closes the gap to full computability.

45 General Recursion Herbrand-Gödel Computability 3 µ-recursive Functions Church-Turing Thesis

46 (General) Recursive Functions 46 Definition The clone of µ-recursive functions is defined like the primitive recursive functions, but with one additional operation: unbounded search. f(x) = min ( z g(z, x) = 0 ) Here g is required to be total. Thus, f(x) = 3 means g(3, x) = 0, but g(2, x), g(1, x), g(0, x) > 0.

47 More Convergence 47 Note f is intuitively computable whenever g is: we just conduct a potentially unbounded search for the least z such that g(z, x) = 0. As a result, even though we insist that g is total, f will in general just be a partial function: for some x there may well be no suitable witness z. Warning: one cannot allow for g itself to be partial; there are examples that show that this would violate intuitive computability.

48 Terminology 48 This last class of intuitively computable functions is due to Kleene. The notion µ-recursive function comes from the fact that in the older literature one often finds the more compact and rather cryptic notation f = µ g rather than a reference to min. So now we have primitive recursive functions, register machine computable functions, Herbrand-Gödel computable (partial recursive) functions, and µ-recursive functions. We know that p.r. is weaker than the other two classes, but what is the relationship between them?

49 Comparisons 49 Register machine computable is at least as strong as all the others: think about how a register machine could simulate the other models (using our coding machinery). Also, primitive recursive functions are strictly weaker. So the real questions are these: What is the relationship between register machine computable, µ-recursive and Herbrand-Gödel computable?

50 HG implies Mu 50 Suppose we have a finite system E of equations that defines a (partial) function f. To show that f(x) y we need to construct a derivation that uses only the given equations, plus substitution and replacement. With modest effort one can show that this whole machinery is primitive recursive in the following sense: there is a p.r. relation D such that D(t, x, y) t is a derivation of f(x) = y from E But then we can simply search for the least such t and extract the corresponding y.

51 Mu implies HG 51 We need to show how to express the min operator in terms of equations. Here is a trick due to Kleene (1952). Assume f(x) = µz(g(z, x) = 0). Introduce three new function symbols α, β and G with equations α(x, y + ) = x β(x, 0) = x G(0, x) = g(0, x) G(y +, x) = α(g(y +, x), G(y, x)) f(x) = β(z, G(z, x)) Ponder deeply.

52 RM versus Mu 52 With a bit of effort one can convince oneself that µ-recursive functions can be computed by a register machine: the unbounded search is no problem, we can have a loop that increments a register until a witness is found. In the opposite direction, each step in a computation of a register machine is primitive recursive, and by adding one unbounded search we can express computations of a register machine in terms of a µ-recursive function. So, they are all equivalent, and there are simple (primitive recursive) translations between the different models.

53 General Recursion Herbrand-Gödel Computability µ-recursive Functions 4 Church-Turing Thesis

54 The Key Ideas 54 We can specify a model M of computation by defining a space C of possible configurations (snapshots), a one-step relation, an input and output convention, a coding convention (if needed). The details vary greatly, but we always have the same pattern. Major Warning: Minute details about input/output/coding conventions become really important in low complexity classes; higher up they are mostly interchangeable.

55 Whole Computation 55 1 Given a one-step relation C M C, multiple steps and whole computations are defined in the obvious way: C M 0 C : C = C C M t C : C t 1 C M C C 1 M C C M C : t C M t C A computation (or a run) of M is a sequence of configurations C 0, C 1, C 2,... where C i M C i+1.

56 Input/Output 56 One needs a way to provide input from some set X, a map inp : X C as well as an output map to some set Y : outp: C Y Both maps are very simple, essentially just a bit of re-formatting. A minor technical issue: the output map may only be defined on some of the configurations (the halting configurations).

57 Models of Computation 57 C C C N k N The number-theoretic scenario: input and output are natural numbers.

58 Models of Computation, II 58 C C C Σ Σ The string scenario: input and output are words over some alphabet.

59 More History 59 In 1930, A. Church introduced yet another way of defining computability, based on his λ-calculus, the general theory of functional composition. There are lots of places in our curriculum where you can hear about the λ-calculus, so we will skip all details. In fact, the λ-calculus is hugely important for the semantics of programming languages, but plays much less of a role in classical computability theory. Suffice it to say that Church s student S. C. Kleene showed that many arithmetic functions are λ-definable.

60 Empirical Observation 60 For all the models mentioned so far, one can simulate model M in any other model M. In fact, the cross-model simulation maps are all very simple (say, primitive recursive). Of course, the length of a computation may differ a bit in different models, a major issue in complexity theory.

61 Church s Proposals 61 Church s Thesis I (1934): Effectively calculable iff λ-definable. Church s Thesis II (1935): Effectively calculable iff general recursive. Gödel rejected these early proposals by Church.

62 Gödel s Position 62 But then in 1936, A. Turing introduced his theory of computability based on Turing machines. Gödel responded most enthusiastically to Turing s work: This concept,... is equivalent to the concept of a computable function of integers... The most satisfactory way, in my opinion, is that of reducing the concept of finite procedure to that of a machine with a finite number of parts, as has been done by the British mathematician Turing.

63 Gödel Later 63 But I was completely convinced only by Turing s paper. See A. Turing [1936] and the almost simultaneous paper by E.L. Post [1936]. As for previous equivalent definitions of computability, which, however, are much less suitable for our purpose, see A. Church [1936]. Gödel explains his position in He always gave full credit to Turing, never to Church or himself.

64 Church s Response to Turing 64 The first has the advantage of making the identification with effectiveness in the ordinary (not explicitly defined) sense evident immediately, i.e., without the necessity of proving preliminary theorems. Church, in his 1937 review of Turing [1936].

65 Church-Turing Thesis 65 Church was the first to become convinced that All (reasonable) models of computation are equivalent, and correspond exactly to our intuitive notion of computability. So far (Fall 2017), there are no interesting objections to this idea. So from now on we will often simply talk about computable functions and reference specific models only when necessary.

66 The Cult of Hypercomputation 66 A lot of people would agree that the Church-Turing thesis also has a physical interpretation: All (reasonable) models of computation correspond exactly to physically realizable computability. This is much more problematic, since our actual universe appears to be finite in many ways we would need to ignore these constraints. Alas, we have nothing resembling an axiomatization of physics, so there is the logical possibility that some currently unknown physical phenomenon would allow one to break through the Turing limit. Possible, but not very likely. Forcryingoutloud, we can t even factor numbers.