Notes on Turing s Theorem and Computability Walter Neumann About 60 years ago there was a revolution in mathematics and philosophy. First Gödel and then Turing showed that there are impossible problems in mathematics. Gödel showed that there are true theorems that can t be proved (his famous Incompleteness Theorem ). Turing showed, before the first electronic computer had even been built, that there is no systematic way of deciding if any given computer calculation will stop with an answer or go into an endless loop (i.e., crash ). One consequence of these results is the fact that there are easily stated problems with impossibly difficult answers. This fact is fundamental to the existence of public key cryptosystems like the one discussed in class. These notes are a very brief introduction to the theorems of Turing and Gödel. If you want to find out more, not just about Turing and Gödel, but also about quantum mechanics and lots more, an excellent reference is the 1989 best-seller: The Emperor s New Mind by Roger Penrose, published by Oxford University Press. Computable sequences. We shall study infinite sequences of numbers like: sequence 1: 1, 1, 1, 1, 1, 1,... sequence 2: 1, 2, 3, 4, 5, 6,... sequence 3: 0, 1, 0, 1, 0, 10,... sequence 4: 3, 14, 159, 2653, 58979, 323846,... A sequence may have an obvious pattern, like the first three of these examples, or a less obvious pattern, like sequence 4. The issue that Turing studied in the 1930 s is: We shall say a sequence When is such a sequence computable? a(1), a(2), a(3),... is computable if a competent programmer can write a program in his or her favourite computer language which does the following: 1. It asks you to input a number n. 2. It then computes a(n) and prints it out. One of Turing s conclusions was that there exist sequences that are not computable in this sense. But as we shall see, he actually proved rather more.
2 Undergraduate Math. Society Before we discuss this, it is important to stress that the concept of computability does not depend on what computer language or computer we choose to work with, so long as the language is not too primitive. We shall not need to worry much about exactly what the computer looks like, or what language it uses. You can think of it as being like any of the computers one sees around nowadays. However, we do need to assume it has one idealized feature that your favourite real computer does not yet have: it should be able to work with numbers of any size, and therefore, in particular, it should have unlimited memory space. Turing went to some trouble to design an idealized computer and referred all his arguments to this. In this way he was confident that he had a rigorous mathematical interpretation of what computable meant. At the time, electronic computers had not yet been invented, so ideas related to them were very unfamiliar. Nowadays we are much more comfortable with them. In particular, you may be familiar with such software as SoftWindows, which allows you to run Windows software on Macintosh or Sun computers, and thus make your Mac or Sun act like a PC. Since one computer can simulate another computer the first computer can do any computation that the second can do. This is why the concept of computability robust: it doesn t depend on the specifics of the computer and computer language you imagine doing the computations. A baby Turing theorem. We shall first prove a baby version of Turing s theorem. Theorem. Non-computable sequences exist. In fact, there is a non-computable sequence b(1), b(2), b(3),... which eventually grows faster than any computable sequence in the following sense. If a(1), a(2), a(3),... is a computable sequence, then it is smaller than the b-sequence from some point on, that is, there is some number N such that b(n) > a(n) for n N. Proof. The proof depends on the basic fact that we can list all computable sequences in some order. Since we need infinitely many names to list them all, we shall call them a 1, a 2, and so on. Thus the first computable sequence is and the second is a 1 (1), a 1 (2), a 1 (3),..., a 2 (1), a 2 (2), a 2 (3),..., and so on. How do we list them? Well, we know that a computable sequence is computed by some computer program P. When stored in computer memory, this computer program is just a string of (binary) digits. We can think of this string of digits as the digits for some (large) number. Thus in computer memory the program is really indistinguishable from a number. Not every number stored in computer memory will be a proper computer program; in fact, very few of them will just as very few strings of letters of the alphabet spell out grammatical sentences. We do not care about this. All we need is that every program is a number in computer memory.
Turing s Theorem and Computability 3 Since each computable sequence is determined by a program and hence by a number in this way, we can just list computable sequences in order of the numbers that determine them. Thus a 1 (1), a 1 (2), a 1 (3),... would be the sequence determined by the simplest program (in the sense of smallest number). This program would quite likely be the one that ignores your input and just immediately prints 0 so this sequence is probably 0, 0, 0,.... But it is not important whether this is really what it is. The order of listing computable sequences does of course depend on what computer hardware and software we imagine we are using, but this does not matter to our arguments. So here is our list of computable sequences: a 1 (1), a 1 (2), a 1 (3), a 1 (4), a 1 (5), a 1 (6),... a 2 (1), a 2 (2), a 2 (3), a 2 (4), a 2 (5), a 2 (6),... a 3 (1), a 3 (2), a 3 (3), a 3 (4), a 3 (5), a 3 (6),... a 4 (1), a 4 (2), a 4 (3), a 4 (4), a 4 (5), a 4 (6),... a 5 (1), a 5 (2), a 5 (3), a 5 (4), a 5 (5), a 5 (6),... a 6 (1), a 6 (2), a 6 (3), a 6 (4), a 6 (5), a 6 (6),............... We now construct our new sequence b as follows. b(1) = a 1 (1) + 1 b(2) = max(a 1 (2), a 2 (2)) + 1 b(3) = max(a 1 (3), a 2 (3), a 3 (3)) + 1 and so on. That is, the n-th term of the sequence b is 1 more than the largest of the n-th terms of the sequences a 1, a 2,..., a n. Now if a is any computable sequence then we want to show that the sequence b is eventually larger than sequence a. Sequence a is one of the sequences in our list of computable sequences. It might be a 37 for example. But b is constructed so that from the 37-th term on, all its terms are at least 1 larger than each of a 1 (n), a 2 (n),..., a 37 (n),..., a n (n). Thus b(n) > a 37 (n) when n 37. This completes the proof. Is our sequence b computable? Before we discuss the above question, a note about terminology. Rather than speaking of the sequence b, mathematicians usually speak of the function b. This function is the rule that generates the sequence b(1), b(2),.... This is no more than a linguistic difference, reflecting the two different views we have of b, a rule for generating a list of numbers or the list of numbers itself. It is not an important difference, but from now on we will stick to mathematical terminology and call b a function. At this point it looks as if we are faced with a paradox. In the previous section we constructed a non-computable function b. But on the other hand, it looks as if we gave a perfectly good procedure to compute it. Is it computable or is it not? The answer must be that it is not computable, since we carefully constructed it not to be. Thus the procedure must have a non-computable aspect in it somewhere. Where is it? The answer depends on something that is familiar to any computer programmer that programs don t always stop and give you an answer for example, they can get trapped in endless loops.
4 Undergraduate Math. Society To illustrate this, suppose we have a simple computer that has an input/output store and working memory and operates as follows. We load our program into working memory, put a number n in the input/output store and press the start key. The answer is whatever is in the input/output store once the program stops. Consider the following program: Program A 1. Replace n n + 1 2. If n = 100 stop, otherwise goto step 1. If the input is less than 100 this eventually stops with 100 in the input/output store. But if the input is 100 or more this program just keeps running for ever. Sometimes it can be hard to decide if a program will stop. For example: Program B 1. If n is a power of 2 then stop, 2. If n is even then replace n by 3n/2+1, while if n is odd replace n by (n 1)/2, 3. Goto step 1. For example, the input n = 14 results in this program computing the numbers 22, 34, 52, 79, 39, 19,9, 4 and then stopping. Thus if the function that this program computes is called f then f(14) = 4. You can check that the first few values of f(n) for n = 1, 2,..., 14 are 1, 2, 1, 4, 2, 16, 1, 16,4, 16,2, 4, 16, 4. If we gave this program a large number and it did not stop for several hours, how would we decide if the program will continue working for ever or if it will eventually stop with an answer? Is there some general way of deciding this, whatever the program? If you think very carefully about this, you will realise that the answer must be no, because if it were yes then the function b that we constructed above would indeed be computable, but we made sure that it is not. To compute b we need to decide which computer programs really define sequences, which means being able to decide by computation which programs always stop, no matter what the input. If we could do this, then we could actually compute the list of all computable sequences and hence compute b. This will become clearer if we make it more precise, which we now do. Turing s halting theorem We shall work with a computer like the one we described above. Remember that a program is just stored as a string of digits in the computer s working memory. Any string of digits can be stored there, and it may make sense as a program or not. If it does not make sense, then when we push the start button the computer will probably stop immediately with some cryptic message like illegal op code in location H0000A0. Since it won t have changed the number in the input/output store, we can think of such a nonsense program as being a program that computes the trivial counting function f(n) = n. A string of digits in working memory is just a number. Interpreting numbers to be programs in this way, every number represents a program, but most programs are rather boring in that they compute this function f(n) = n. We shall give the program corresponding to the number j the name P j.
Turing s Theorem and Computability 5 Instead of trying to list computable sequences, we list the output of every program as follows: p 1 (1), p 1 (2), p 1 (3), p 1 (4), p 1 (5), p 1 (6),... p 2 (1), p 2 (2), p 2 (3), p 2 (4), p 2 (5), p 2 (6),... p 3 (1), p 3 (2), p 3 (3), p 3 (4), p 3 (5), p 3 (6),... p 4 (1), p 4 (2), p 4 (3), p 4 (4), p 4 (5), p 4 (6),... p 5 (1), p 5 (2), p 5 (3), p 5 (4), p 5 (5), p 5 (6),... p 6 (1), p 6 (2), p 6 (3), p 6 (4), p 6 (5), p 6 (6),............... Here p j (n) means the output of program P j when started with n in the input/output buffer. Thus the first few rows of this table almost certainly belong to nonsense programs that never change the input and are thus 1, 2, 3, 4, 5, 6,.... We need some symbol to represent the situation that program P j never stops if started with n in the input/output buffer. In this case we write p j (n) =. Thus if the number 27 happens to represent the instruction goto step 1 then p 27 (n) = for any n since program P 27 will just keep repeating its first instruction and never stop. Thus the 27-th row of the table would be,,,,,,,.... We can now make very precise the statement that we can not decide whether a program stops. We shall give two versions. Define: { 0 if program Pj fails to stop for some input H 1 (j) := 1 if program P j stops for every input. H 2 (j, n) := { 0 if program Pj fails to stop for input n 1 if program P j stops for input n. Turing s Halting Theorem. The functions H 1 and H 2 are non-computable functions. The argument given briefly in the previous section is really an argument that shows that H 1 is non-computable. For if it were, then we could use it to discard programs that do not define sequences and thus construct by computation enough of our list of computable sequences to compute any desired value b(n) of the function b. However, it is maybe not really so surprising that H 1 is non-computable, since to decide that the j-th computer program P j stops whatever its input appears to involve deciding infinitely many things all at once 1). But it is more surprising that the function H 2 is non-computable. That is, there is no program which, given both j and n as input, will infallibly compute H 2 (j, n) and thus tell us whether program number j will stop if run with input n. 1) Of course we often do this for example, when we prove a theorem by induction we are proving infinitely many cases of some statement all at once.
6 Undergraduate Math. Society To prove this, let us assume instead that there is such a program. Let us call this program D (for decision ). For each j we now write a new computer program: Program Q j 1. Look at the number n in the input/output store and run program D with input j and n to compute H 2 (j, n). 2. If H 2 (j, n) = 0 then put 0 in the input/output buffer and stop, otherwise 3. Run program P j. Notice that this program always stops, since it only gets to step 3 once it knows that step 3 will stop. It stops with value 0 in the input/output buffer if program P j does not stop with input n and otherwise it moves on to step 3 which eventually stops with output p j (n). We list the output of the programs Q j in a table: q 1 (1), q 1 (2), q 1 (3), q 1 (4), q 1 (5), q 1 (6),... q 2 (1), q 2 (2), q 2 (3), q 2 (4), q 2 (5), q 2 (6),... q 3 (1), q 3 (2), q 3 (3), q 3 (4), q 3 (5), q 3 (6),... q 4 (1), q 4 (2), q 4 (3), q 4 (4), q 4 (5), q 4 (6),... q 5 (1), q 5 (2), q 5 (3), q 5 (4), q 5 (5), q 5 (6),... q 6 (1), q 6 (2), q 6 (3), q 6 (4), q 6 (5), q 6 (6),............... The only difference between this table and the previous one is that every time a p j (n) equals it is replaced by 0. Now every computable sequence occurs as a row of this table, since every computable sequence occurred already in the table of p j (n) and is unchanged in this table. In fact, each computable sequence will probably occur many times in this table, but we do not care about this. The whole table is computable, since we have given an explicit way of computing each entry: run program Q j with input n. We now use this table as our list of computable sequences to create a version of the function b that we constructed earlier. Thus now b(n) = max(q n (1), q n (2),..., q n (n)) + 1. Now we really do have a contradiction: our function b is definitely computable since we have carefully constructed it to make sure it can be computed at every stage. But it also eventually grows faster than any row of the table and therefore eventually grows faster than any computable function. So it can t be computable. This contradiction shows that the assumption on which this argument was based that H 2 is a computable function must have been wrong after all. We have proved Turing s theorem. Gödel s Theorem. One of Gödel s famous theorems says loosely that in any formal mathematical system sufficiently strong to do elementary arithmetic there exist theorems that are true but cannot be proved. Turing s theorem implies a version of this. We first note that if every true theorem were provable then we could give a formal algorithm to actually find a proof. We assume a formal mathematical system in which we can write down proofs and check, step by
Turing s Theorem and Computability 7 step, that they are indeed logically correct that each step follows from the preceding ones. Since proofs are just strings of words, we can list all possible strings of words in dictionary order and then read through thme one by one and check each one whether it is logically correct and whether the final sentence is the theorem we are interested in. Since we are assuming our theorem has a proof, and this proof must be somewhere on our list of all possible strings of words, we must eventually get to it and find it. But now we have a way of computing H 2 (j, n): we read our proofs as above one by one until we find one that either proves the theorem H 2 (j, n) = 0 or one that proves the theorem H 2 (j, n) = 1. Since one of these two must be true, this procedure eventually finds which one it is. Thus if Gödel s theorem were false then the function H 2 would, after all, be computable, but we know it is not. Gödel in fact proved much more than what we have just proved. Our proof does not exhibit an explicit theorem that is not provable, it just shows it must exist. Gödel actually gives an explicit theorem with this property. This is also only one of two famous theorems he proved, the other of which says roughly that it is impossible to prove that the formal mathematical system in question is consistent, i.e., cannot lead to any internal contradictions.