The Undecidable and the Unprovable

Transcription

1 The Undecidable and the Unprovable Jeff Sanford Russell Spring 2017 The main purpose of these notes is to help you understand some beautiful and important facts about the limits of computation and logic. There are problems that cannot be solved with any general systematic method and these include the problem of deciding which systematic methods will eventually succeed. There are facts that can be formally stated, but cannot be formally proved and these include important facts like the fact that our theory of mathematics (and in particular our theory of formal logic and consistency!) is consistent. These results are philosophically deep, and they give us tools for doing other deep philosophical work. These tools help us use clear language and careful reasoning to talk about clear language and careful reasoning. The results also give us an understanding of this project s pitfalls. Before we get started, here s a puzzle: a certain barber shaves just those people who do not shave themselves. Does the barber shave himself? If so, then he shaves someone who does shave himself, contradicting the assumption. If not, then he fails to shave someone who does not shave himself, again contradicting the assumption. So the premise of the puzzle is impossible: nobody shaves just those people who do not shave themselves. Here s another closely related puzzle, called the Liar Paradox. Consider the sentence This sentence is not true. Call this sentence L. Since apparently what L says is just that L is not true, it seems that (1) L is true iff L is not true Is L true? If so, then 1 says that L is not true, which is a contradiction. Alternatively, if L is not true, then 1 says that L is true, so again we have a contradiction. Since we have a contradiction either way, we have a paradox. The main idea we will use to show that there are undecidable questions and unprovable statements comes from a variant of the Liar Paradox, called Grelling s paradox. Unlike the original Liar, this variant does not depend on a self-referential sentence. Adjectives, like short, interesting, or simple, are words that can be truly applied to some things but not others. (Let s ignore the problems of vagueness for now, and pretend that all of these adjectives are perfectly precise.) 1

2 Words are themselves among the things that adjectives can apply to: for instance, red is a short word, so the adjective short applies to red. Furthermore, short is a short word, so short applies to itself; long is not a long word, so long does not apply to itself. Some adjectives self-apply, and others don t. Now consider the set of all adjectives which do not self-apply. Is there any adjective which has this set as its extension? Suppose there were such an adjective: in particular, suppose that the adjective non-self-applying applies just to those adjectives which do not self-apply. For all adjectives a: (2) non-self-applying applies to a iff a does not apply to a Does non-self-applying self-apply? If so, then it applies to some adjective which self-applies namely non-self-applying itself contradicting the assumption 2. If not, then it fails to apply to some adjective which does not self-apply again, non-self-applying itself again contradicting the assumption 2. Like the barber puzzle, principle 2 is paradoxical. But what we ll see is that this self-applying idea doesn t just raise puzzles. The same idea plays a central role in some great discoveries. 1 The Undecidable For some questions, there is a systematic procedure you can follow that will eventually bring you to an answer. For instance, which numbers are prime? This question can be answered, for any particular number n, by checking each number k less than n, one by one, for whether n is divisible by k. This is an algorithm for answering the question. A question like this, which can be systematically answered somehow or other, is called effectively decidable. We can also think about questions which have different sorts of answers. For instance, the question What is the remainder when m is divided by n? has a number as its answer. Many of us learned the long-division algorithm in elementary school, which provides a systematic way of answering any question of this form. A family of questions like this, the answers to which can be arrived at systematically, is called effectively computable. We can describe these families of questions as functions, whose values are answers to the question. The remainder function takes a pair of numbers (m, n) to the number which is the remainder when m is divided by n. Similarly, the question Which numbers are prime? can be represented by the function that takes each number n to either True or False. 2

3 The main thing we ll be working up to in this chapter is a central result about undecidable questions. 1.1 Programs An algorithm is a general systematic recipe for answering a question. (This is also called an effective procedure.) For example, given a string like "ABC", what is the string of the same symbols in reverse order? For this example, the answer is "CBA". How can we work out the answer in general, for an arbitrary sequence? One approach is to follow these steps. 1. Set the result to "", the empty string. 2. Go through the symbols in the string one by one, from left to right. For each symbol x, append x to the beginning of the current result. We can describe algorithms using formal languages: these are called programming languages, and a formal description written in such a language is called a program. Programs are everywhere now. There are millions of lines of programming code that make your phone work, and about a hundred million for a new car. Hundreds of different programming languages have been developed for different purposes: Javascript, C++, Python, Lisp, Haskell, and so on. Here is an example of a program written in Python that describes (or implements ) the counting algorithm we just described: def reverse(s): result = "" for symbol in s: result = symbol + result return result The most important thing about these basic programs we ll describe is that they only do things that can be worked out mechanically and systematically. given enough time and space to write things down. So if we can write a program that answers a certain question, this shows that the question is effectively computable. Here we ll explore a very simple subset of the Python language. We ll call it Py. Because it s a subset of real Python, that means you can enter our programs into any Python interpreter and they should run (for example, this one: it/languages/python3). This is a useful way to check your work. 3

4 Py-programs work with strings. A string is a sequence of symbols. Each symbol can be a letter, a number, a punctuation mark, or whitespace (including indentation or linebreaks). Symbols can also be fancier things like technical symbols or emoji. Strings can be used to represent all sorts of things: English sentences, or technical formulas, or numbers, or programs. These notes are represented by one long string of symbols. When we want to talk about particular strings, we use quotation marks: for example, "ABC" is the three-symbol string that starts with the letter A, followed by the letter B, followed by the letter C. Each Py-program takes a string as input, does some work, and spits out another string as its output. The reverse program above starts with the line def reverse(s): Here def is short for define, and it indicates that we re beginning a new program. We specify what the name of the program is (reverse), and what the name of its input is (s). The last line of the program is return result This tells us what the program s output is. A program can represent different strings using variables. In this program, s, symbol, and result are three different variables. At any stage in computation, each variable can stand for some string, which is called its value. As we go through the steps of the program, the value of a variable can change. For example, consider this line: result = symbol + result Here the equals sign means to change the value of the variable result. The result of this step is to make the value of result longer: whatever string result happens to represent, this statement says to add the string represented by symbol onto the front of that. (In general, we can stick two strings together using the plus sign.) This is called a let statement. In general, a statement of the form x = a means to change the value of x to be whatever string is described on the right-hand side of the equals sign. In general, a statement is an instruction, which says to do something. A statement doesn t describe how things are; rather, it gives an instruction for how to change the way things are. 4

5 You can think of the program as a list of instructions for someone who has a sheet of paper that lists all of the variables and their values for example: s result symbol "ABC" "A" "B" The person follows the instructions one by one. When they reach a let statement, they erase one of the values in the right-hand column and write in some new value. For instance, when they see the instruction result = symbol + result they will change the table to look like this: s result symbol "ABC" "BA" "B" When they reach the end of the instructions, they ll tell you what is written in the result row of the table. We can chain together statements by putting them in lines one after another, like this: firstvalue = "" secondvalue = "A" secondvalue = secondvalue + secondvalue This means to do what each statement says in order. In this example, first they set the variable firstvalue to the empty string (a string with zero symbols). Second, they set the variable secondvalue to the string "A". Finally, they change the value of secondvalue so it s "AA" instead. We can also chain together statements in more complex ways. 5

6 1.2 Branching One important thing to do is branching. We can write programs that can go in two different alternative directions, depending on whether two strings are the same. if a == b: flag = "True" else: flag = "False" What it means is to first check whether a and b denote the same string. (Note the double equals sign this is because the single equals sign was already taken for let statements. Here we definitely don t want to change the value of a to be the same as b that would defeat the purpose!) Ifa and b do have the same value, then we do the statements in the first block in this case, we set flag to True. If a and b denote different strings, then instead we do the statements after the else in this case, we set flag to "False". Here s another example: if s == "": result = "Error: the string is empty!" else: result = s[0] If s is not empty, then we set the value of result to its first symbol. (In general, we can pick out specific symbols from a string using this square-bracket notation. In Py, we start counting the symbols from 0 rather than 1.) Otherwise, we set result to be an error message. Example 1. This program returns "True" for an empty sequence, and "False" for a non-empty sequence. def empty(seq): if seq == "": result = "True" else: result = "False" return result 6

7 Exercise 2. Show the following functions are computable writing programs using if and else. 1. Are s and t both equal to "True"? 2. Are either of s or t equal to "True"? 1.3 Looping Another common pattern in programs is to go through each of the elements of a string one by one, and do something with each one. This is called a for loop. For example, this program decides whether every symbol in a string is "A". def allas(s): result = "True" for symbol in s: if not (symbol == "A"): result = "False" return result The for loop goes through the elements of the string s one by one, and stores each symbol in the variable symbol. Example 3. This program takes a string and repeats each symbol an extra time. For instance, it takes "ABC" to "AABBCC". def repeatsymbols(s): result = "" for x in s: result = result + x + x return result The second basic kind of statement is a loop. This lets us write programs that do the same steps over and over again as long as the world is a certain way. For any terms a and b, and any block of statements block, we can build this kind of statement: while a == b: block 7

8 This means to repeatedly do what block says as long as the values of a and b are the same. One important difference between for loops and while loops is that for loops are guaranteed to eventually stop, when the program gets to the end of the string. In contrast, a while loop might go on running forever: there is no guarantee that the values of a and b will ever be differennt, so the condition a == b might hold forever. Definition 4. A program A halts for input s iff running A with the input s eventually returns some output. In other words, a program halts for a certain input iff it does not end up in an infinite loop. Exercise 5. Write a program loop that never halts for any input. Exercise 6. Write a program flip that halts for the input False, and does not halt for the input True. Definition 7. (a) A set of strings X is Py-decidable iff there is some Py-program that, when given any string s as its input, eventually returns "True" as its output if s is in X, and eventually returns "False" as its output if s is not in X. (b) A function f is Py-computable iff there is some Py-program that, when given any input s, returns f(s) as its output. 1.4 The Church-Turing Thesis If we want to show that a question is decidable, we can write a program to answer it. But how would we show that a question is undecidable? To do this, we wouldn t just need to show that no program in our little language Py answers the question we d need to show that no program in any reasonable programming language can answer it. If a question is undecidable, then there isn t any systematic algorithm for solving it at all. Alonzo Church and Alan Turing both hypothesized that there are universal programming languages: languages which are expressive enough to describe every systematic algorithm. In fact, they didn t just hypothesize that such languages exist: they proposed some specific candidates. (In Church s case, these consisted of a small family 8

9 of operations on functions of natural numbers. In Turing s case, the language consisted of Turing Machines hypothetical devices for reading and printing on a long piece of tape.) These proposals amounted to giving a formal analysis of the intuitive concept of a decidable question. You might doubt whether such an analysis could succeed. (Surely any analysis like this would have counterexamples!) But in fact, we have very strong evidence that Church and Turing s proposal is true. The key philosophical claim is called the Church-Turing Thesis. The first bit of evidence for it is packed right into its name: for Church s and Turing s apparently different theses apparently different analyses of the concept of a decidable question turned out to be equivalent. That is, any question which is decidable using a Turing Machine is also decidable using Church s functions, and vice versa. Today we have hundreds more examples formal languages like C++ or Python or Haskell and so on: these also turn out to be equivalent to Turing and Church s languages. This means that we get a little bit more inductive evidence for the truth of the Church-Turing Thesis every time a programmer takes a precisely described algorithm and implements it in their favorite programming language. The Church- Turing Thesis is thus a hypothesis which is extraordinarily well-confirmed by the practice of modern programming. Even so, it s worth remembering that it is a philosophical thesis an extraordinarily successful philosphical thesis, but not a theorem. We can prove lots of theorems about various kinds of formal languages. But the Church-Turing Thesis is about the relationship between these formal languages and the intuitive notion of a decidable question. In particular, our little language Py is equivalent to each of these other programming languages: a set is Py-decidable if and only if it is decidable using a Turing Machine, if and only if it is decidable using Church s functions, if and only if it can be decided by a program in C++ or any other standard programming language. So if any of these languages is a universal programming language, so is Py. So according to the Church-Turing Thesis, whatever can be done in any systematic way by any algorithm at all can also be done using humble Py. The Church-Turing Thesis. (a) A set of strings is effectively decidable if and only if it is Py-decidable. (b) A function is effectively computable if and only if it is Py-computable. In what follows, we will freely appeal to the Church-Turing Thesis (though it s generally a good idea to be clear about when exactly we re relying on it). This is extremely 9

10 useful in two ways. First, this lets us deduce the existence of programs, even without formally writing them out. In order to show that a question is decidable, it s enough to informally give some reasonably careful description of a systematic procedure for answering it. But transforming an informal algorithm into a formal program can still be pretty tricky. (That s what professional programmers are for.) Given the Church-Turing Thesis, we can deduce the existence of a program from the existence of an algorithm, even when we re not sure exactly how to write that program. We ll do this in what follows: rather than writing out fully detailed programs in our little language, we can just outline how a program ought to work, and posit that some program does in fact work that way, appealing to the Church-Turing Thesis. Second, this lets us prove results about undecidability. We can mathematically prove that every Py-program has certain properties. From this we can deduce properties that all Py-decidable sets have in common. Then any set that doesn t have these properties must be undecidable. 1.5 The Universal Program We ve been writing programs by typing sequences of symbols. That is, we are representing programs themselves using strings. That means that programs themselves can be inputs and outputs of programs. We can write programs to answer questions about programs. Programs that manipulate programs might sound recherché, but it s actually very common and practical. When we write a program in Python and then want to run it, what we do is type a sequence of symbols that represent that program, and we give this string as an argument to a Python interpreter which is some other program. Someody wrote that program, too, in some programming language. In fact, the interpreter might be written in Python itself! Even our little language Py can do this. We can write a Py-interpreter in Py. This is a program that takes as its input the string representation for another Pyprogram A, together with an input value, and returns the resulting value that results from running A with this input value. At least, it does this if A has any output value. It could be that A crashes or goes into an infinite loop. In that case, the interpreter will also just crash or run forever. Theorem 8. The Py-interpretation function is the function that takes each Pyprogram A and string s to the output of running A with input s, if A halts for the input 10

11 s, and is undefined otherwise. The Py-interpretation function is Py-computable. Proof Sketch. We won t write out a full program for this, but we will informally describe an algorithm for doing this. By the Church-Turing Thesis, this algorithm can be implemented by some program. The first part of this project is called parsing. We need to take a string representing a program, and split it up into its meaningful parts. We can write a bunch of small programs to handle basic parsing tasks. Here s our to-do list. (We won t actually do all of this: the goal here is just to make it apparent that the interpretation function is computable, not to actually write a complete parser and interpreter. But if you have the time and interest, it s fun to work out some of these details in front of a computer.) 1. (a) Write a program that takes a program as input and returns its first statement. (b) Write a program that takes a program as input and returns all of it except its first statement. (Remember that a statement can be a complete while loop. One tricky part here is figuring out where the loop ends. In Python, this depends on the indentation.) 2. Write a program that takes a statement as input and identifies what kind of statement it is a let-assignment, a for loop, an if statement, or a while loop and returns "let" "for", "if", or "while" accordingly. 3. Write two programs that take a let -statement x = s and return (a) the variable on the left of the equals sign, (b) the term on the right of the equals sign. 4. Write three programs that take a while -statement while a == b: and return (a) the first term a, (b) the second term b, and (c) the inner block. Similarly for for, and if statements. 5. Write a program that takes a term and identifies what kind of term it is: whether it is of the form head(a), tail(a), a + b, the empty string "", or represents a single character like "A", or a variable. In the first three cases, we should also write programs that return the terms a and b. So far we have just been handling the syntax of programs. To make them work, we ll also need to come up with a way of representing assignment functions with strings. An assignment function is a table that represents the values for each variable at some stage along the way. (We only need to worry about assignment functions that are defined for finitely many values which is a good thing, because there is no way to represent arbitrary infinite assignment functions using finite strings. There 11

12 are too many of them.) There are many ways to do this, but here s one. Say we have the assignment function that assigns the variable x the value "hello" and assigns the variable result the empty string "". We can represent this with this string: { x : "hello", result : "" } We ll also need to write a couple basic programs for manipulating assignment functions. 6. Write a program that takes an assignment and the name of a variable, and which returns the value that the assignment has for that variable (if there is one). For instance, given the assignment above and "x", this should return the string "hello". 7. Write a program that takes an assignment a, the name of a variable v, and another string s, and which returns the assignment that results from changing the value of v to s in a. Finally, we can build our interpreter by putting all these components together. There will be two parts: a term-interpreter, and a block-of-statements-interpreter. The term-interpreter takes an assignment and a term and returns the string that the term denotes (with respect to that assignment). We start by figuring out which form the term has. If it s a variable, then we look up the value of the variable in our assignment (using function 6 above). If, for example, the term is head(a) for some other term a, then we apply our term-interpretation method to a, and then get its first symbol using our actual head function. Things go similarly for the other kinds of term. To interpret a block of statements, first we look at the first statement in the program. If it s a let-assignment, x = a, then first we use our term-interpreter to evaluate the term a, and we use our update an assignment program (program 7 in the to-do list) to set the value of x to whatever value a denotes. If it s a while-statement, of the form while a == b: block, then we will interpret it using a while loop. Here it is in pseudo-code : done = "False" while done == "False": avalue = interpret-term(a) bvalue = interpret-term(b) 12

13 if avalue == bvalue: g = the result of interpreting block with respect to assignment g else: done = "True" And that about finishes it up. We haven t written a whole program, but we have described an algorithm for interpreting programs, a bit roughly. (For a really rigorous proof, we would need to fill in more of the details.) By the Church-Turing thesis, there is some program that implements this algorithm. So the interpretation function is computable. 1.6 An Undecidable Problem Recall that a program halts iff it has some well-defined value. A program that halts is one that neither crashes with an error nor goes into an infinite loop. Here is perfectly sensible question: which programs halt? For any particular program A, either A has some final value, or it doesn t. The halting problem is the perfectly precise question of which programs return some value, and which don t. This is a practically important question. It would be extremely useful to have a program-checking program: a program that determines whether your program is going to crash, or whether it will go into a never-ending while loop, or not. Unfortunately, there is no such program. The question of which programs halt, while a perfectly precise question with a correct answer, is effectively undecidable. There is no systematic method for determining, in general, which programs are going to eventually return a value. This is what we ll show in this section. For every string, there s an expression in Py that labels it, using quotation marks. For example, the string ABC is labeled by the string "ABC" which adds quotation marks around the original string. Let s call this Py-expression 13

14 the canonical label for a string (in Py). 1 If s is a string, let s use the notation s for the canonical label for s. We can use canonical labels as a way of plugging data into programs. If s is a string and A(x) is a program, then A s is a program that, before running A(x), begins with a let assignment x = s which has the result of setting the value of x to s (since s is a term that denotes s). So the result of running the program A s is the same as the result of running A(x) with the input s. In particular, we can use canonical labels for the string-representations of programs in order to write programs that do basic manipulations of the syntax of other programs. Each program A has a canonical label A, which is a term in our programming language that stands for the program A. Essentially, it just puts the whole program in quotation marks. Lemma 9. The function that takes each string to its canonical label is computable. We ll call a program that does this label. So for any string s, the result of running label s is the string s. (The basic idea is that we just need to add quotation marks to s. But there are a few tricky details to handle when s is a string that contains quotation marks itself.) Lemma 10. The substitution function that takes a program A(x) and a term t to the program A(t) is computable. Proof. def subst(prog, term): result = "x = " + term + "\n" + prog return result 1 There are some slightly tricky details about how to give canonical labels for strings that involve quotation marks themselves. For example, what is the canonical label for the string consisting of a single quotation mark? It isn t """, because if we allowed quotation marks to appear within quotation marks like this we d end up with lots of annoying ambiguities. One solution is to use '"', with single quotes around the double-quote mark. Then we can label a string like A"BC"" as "A" + '"' + "BC" + '"' + '"' But let s not worry too much about these technicalities. 14

15 We can put these two things together. Exercise 11. There is a program apply(x, y) such that, given any programs A(x) and B as input, returns the program A B that plugs the program B in as input to A(x). Proof. We get apply(x, y) by sticking together subst and label: in particular, apply(x, y) is the same as subst(x, label(y)). We now have a way of applying programs to programs. But remember our paradox of self-applying adjectives? We can do exactly the same trick with programs. Turing s Theorem. The set of programs that halt is undecidable. Proof. We will suppose that the set of programs that halt is decidable and derive a contradiction from this assumption. Suppose that there is some program halt(x) such that, given any program A as input, halt(x) returns "True" if A halts, and halt(x) returns "False" if A does not halt. Remember that we defined a function flip that halts for the input "False", but not for the input "True". So we can put flip and halt together to get a program flip(halt(x)). Call this combined program opposite(x). So if you give opposite(x) the program A as input, then opposite(x) halts if and only if A does not halt. Now, remember the paradoxical adjective non-self-applying? Since we have a way of applying programs to other programs, we can use the same trick here. In place of the adjective non-self-applying, we ll consider the program opposite(apply(x, x)). Call this program A. The question that got us into trouble earlier was Does non-self-applying self-apply? We can ask a similar question: does the program A halt when it is given itself as input? That is, does the program A A halt? What it means for A A to halt is that opposite(apply( A, A )) halts. But we know that this is true if and only if the program given by apply( A, A ) does not halt. And we know that this is true if and only if A A does not halt. So A A halts if and only if A A does not halt. If it halts, it doesn t; and if it doesn t, it does. This is a contradiction! 15

16 This whole line of reasoning got started when we assumed that there was a program halt(x). So the conclusion we have to draw is that really there is no such program: the question of which programs halt is undecidable. 1.7 Semi-Decidable and Effectively Enumerable Sets It might be a little confusing how it could be that the interpretation function for programs is computable, and yet the question of whether a program halts is undecidable. We can clarify the relationship between these two facts by introducing another notion, which is less demanding than decidability, but still goes a long way toward it. A semi-decidable set is a set X that can be decided in one direction. What that means is that there is an algorithm such that, for any given d, if d is in X, then the algorithm will eventually tell you so and the algorithm won t ever tell you something is in it which isn t. But if d is not in X, then there is no guarantee that the algorithm will tell you anything. It will tell you the good things are good, and it won t say any bad things are good, but the bad things might just end up sending your program into an infinite loop instead. Definition 12. A set X is semi-decidable iff there is a program that returns "True" for each input that is in X, and does not return "True" for anything that is not in X. (But for inputs that are not in X, the program isn t guaranteed to return anything at all.) Theorem 13. The set of programs that halt is semi-decidable. Thus there is a set which is semi-decidable, but not decidable. Proof. For any program A, we can use the Universal Program to interpret it. If A halts, then the Universal Program will also eventually halt. Once we are done, we simply return "True". If the program crashes or enters an infinite loop, we will never reach this last step, and so in this case we will not return any value. Semi-decidability is closely linked to another idea. Some sets can be systematically listed. The idea is that we can write a program that spits out each element of X one by one. Definition 14. A set X is effectively enumerable iff there is some program that when run repeatedly eventually returns every value that is in X, and no values that are not in X. 16

17 Listing a set is easier than deciding which things are in it. If a set is decidable, then one way to list its elements is to go through every string one by one in alphabetical order, and for each string check whether it s in your set. If it is, then spit it out, and if it isn t, then don t spit anything out, and go on to the next string. But just because you can list a set doesn t guarantee that you can determine whether any particular thing is in it. You might try to through the list looking for the thing you want but just because you haven t found it yet doesn t guarantee you never will find it. Not every effectively enumerable set is decidable, but every effectively enumerable set is semi-decidable. In fact, the converse is also true: every semi-decidable set is effectively enumerable. But this takes significantly more work to prove. Theorem 15. For any set X, X is effectively enumerable iff X is semi-decidable. Proof Sketch. Here s the basic idea. Say we have a program that returns "True" if and only if its input is in X. Then we can go through strings one by one in alphabetical order. The obvious thing to try is to check each string, and print it out if we get "True". The problem with this approach is that the semi-decision program might go into an infinite loop. The first time this happens, the whole program will stop working. So we need to make sure we never allow the semi-decision program to go on forever. Here s how we can do this. We can run a modified program, which replaces each while loop with a for loop that only runs n times, for some number n, and returns "Fail if the loop-ending condition still hasn t been met at that point. Call this the n-bounded variant of a program. If a program halts, then each of its while loops only goes through finitely many steps, which means there is some number n such that its n-bounded variant program succeeds. So here s what we can do. We can go through the pairs (s, n) of a string and a number, one by one. For each pair, we ll feed the input s to the n-bounded variant of the semi-decision program. If we get "True", then we print out s. If we get "False" or "Fail" then we don t print out s (yet) and we go on to the next pair. Because we are using bounded programs, the computation we do for each pair can only take finitely many steps. So we ll eventually check every pair, and so eventually every string that the semi-decision program returns "True" for will get printed out. 17

18 2 The Unprovable So far we ve been discussing the limits of our ability to answer questions using programs. In this section we ll be considering limits on our ability to describe the world. It turns out that any feasible way of trying to make true statements about the world and deducing their consequences is going to end up leaving out some facts. In short, there are true statements that cannot be proved. 2.1 Statements and Proofs So far we have been talking about a programming language, where the statements are instructions for how to carry out a task. Now we ll turn out attention to descriptive language. Instead of imperative sentences (like Make it so ) we ll be using declarative sentences (like It is so ). In this kind of language, the point of statements is to say something about the way the world is. For instance, we might make statements like Snow is white, or = 4, or The halting problem is undecidable. These aren t instructions for how to change the world; they are claims that can be true or false. One distinctive feature of declarative sentences is that they have negations. For every true sentence, like Snow is white, there is a corresponding false sentence, like Snow is not white. Similarly, for every false sentence, like equals 5, there is a corresponding true sentence, like is not equal to 5. If A is a sentence, we ll represent the negation of A as not-a. (Sometimes people use the notation A or A or A instead.) Here are two important logical facts, for any sentence A. (1) A and not-a cannot both be true. A theory of the world that says both A and not-a is logically inconsistent: it conflicts with itself, and so it cannot be correct. (2) Either A is true or not-a is true. A theory of the world that does not say A, and does not say not-a is logically incomplete: it leaves out some of the facts. What we ll be showing, a bit roughly, is that any reasonable theory is either inconsistent or incomplete. But one of the main tricky details is to say exactly what we mean by a reasonable theory. In particular, we ll be looking at theories that consist of everything that can be deduced from some basic principles. We won t say what the basic principles are just that there have to be some. The basic principles are called axioms, and the way that sentences are deduced from axioms is called a proof. A proof looks like this: 18

19 1. Snow is white. 2. If snow is white then grass is green. 3. So grass is green. 4. If grass is green, then all ravens are black. 5. So all ravens are black. A proof consists of a list of statements, where for each statement A in the list, either A is an axiom (one of our basic premises), or else A logically follows in a simple way from some of the earlier statements. The way in which a sentence can follow from earlier statements is called an inference rule. To keep it simple, we ll look at proofs that only have one kind of inference rule, which has this form: 1. If A then B. 2. A 3. So B. This rule is called modus ponens. Definition 16. Suppose that X is a set of sentences the axioms. A proof from X is a sequence of sentences such that each sentence is either in X, or else follows from two earlier sentences by modus ponens. A sentence A is provable from X if and only if there is some proof from X whose last sentence is A. There can be many different axioms even infinitely many. But in order for a theory to be understandable and useful, the set of axioms should be decidable. There should be some general systematic method for determining which statements are axioms and which are not. If we didn t have this, then there would be no way in principle to tell whether or not an arbitrary sequence of sentences was supposed to count as a proof. Definition 17. (a) An axiomatizable theory is a set of sentences T such that, for some decidable set of axioms X, T contains all and only the sentences that are provable from X. If A is one of the sentences in T, then we say that T proves A, or that A is provable-in-t, or for short A is provable. (b) A theory is inconsistent iff for some sentence A, T proves A and T also proves not-a. Otherwise T is consistent. 19

20 (c) A theory is incomplete iff for some sentence A, T does not prove A, and T does not prove not-a. Otherwise T is complete. Remember, the main result we are going to show that any reasonable way of making statements about the world is either inconsistent or incomplete. The first part of what we mean by reasonable is that it is axiomatizable. Here is a fact about axiomatizable theories that will take us part of the way to the main result. Lemma 18. If T is a consistent and complete axiomatizable theory, then T is decidable. Proof. Suppose that X is some decidable set of axioms. For any sequence of statements, we can systematically check whether or not it counts as a proof from those axioms: we just have to go through and check each line one by one. For each line, first we decide whether it is an axiom (which we are assuming is possible: the set of axioms is decidable). If it is not an axiom, we check whether it follows from some previous lines by modus ponens. If any line fails both tests, then we don t have a proof; if every line passes one of the two tests, we do have a proof. Say we have a sentence A, and we want to decide whether A is provable in T. We can go through the sequences of statements one by one say in alphabetical order. For each sequence, if it is a proof whose last line is A, return "True" this sentence is provable in T. If it is a proof whose last line is not-a, return "False" this sentence is not provable in T, because T is consistent. Since T is complete, eventually either we will find a proof of A or we will find a proof of not-a, so this method is guaranteed to eventually return "True" if A is provable, or "False" if A is not provable. 2.2 Representing Language We can make statements about many things in the world. One of the things we can talk about is language itself. We can talk about symbols and how they can be put together to make sentences, proofs, and programs. We ve been doing this all along. A description (also known as a formula) is a sentence with a hole in it. We represent the hole with the letter x. For example: x is a prime number, or Jeff is taller than x and x is taller than Maggie. The idea is that you can plug a name for something into the hole in order to get a sentence that describes that thing. For instance, we can plug the name two into the first description to get the sentence 20

21 Two is a prime number. We can plug the name Bea into the second description to get the sentence Jeff is taller than Bea and Bea is taller than Maggie. Suppose we have a set of things D, and we have picked out a name for each one of them. Let s use the notation d for the name for d. Then if A(x) is some description, we ll use the notation A d for the sentence you get by plugging in the name for d, to get a sentence that describes d. (This is the same notation we used for plugging inputs into programs. That s on purpose.) In particular, as I said, we will be looking at theories that can describe language. For instance, in English we can say things like this: Snow is white is a sentence. Obama is a name of a president. The result of plugging the name Obama into the description x is a president is the sentence Obama is a president. In order to talk about bits of language, we use quotation marks. If we put a string of symbols between quotation marks, the result is a name for that string of symbols. We ll be looking at theories that have the resources to talk about strings of symbols. So we ll suppose that for each string of symbols s, there is a certain name s. This might use quotation marks, but it also might work some other way. (Again, this is very similar to what we did with programs.) We are interested in theories that are powerful enough to say some things about strings of symbols and how they can be put together. But first, we need to say what we mean for a theory to say something about strings. The first part, which we have already discussed, is that we need names for strings. The second part, though, is that we need some descriptions that we can plug those names into. For example, we might want a theory to be able to say which strings represent programs. In that case we should have some description x is a program. For this description to represent the programs in a theory T, T should be able to prove s is a program for any string s which is a program, and T should prove s is not a program for any string s which is not a program. Here s the general idea: Definition 19. Suppose that X is some set of strings and T is a theory. We say that T represents X iff there is some description A(x) such that, for every string s which is in X, T proves A s, and for every string s which is not in X, T proves not-a s. 21

22 (Representability is similar to decidability. The key difference is that representability has to do with what statements you can prove in a descriptive language, while decidability has to do with what questions you can answer with a programming language.) One simple thing we can describe is how programs work. For example, if we have axioms that describe how each basic programming statement works, we could use this to prove that a certain program has a certain result. The basic operations of programs are pretty simple, so if a theory isn t even strong enough to describe them, then it really can t describe very much. Remember again what our main goal is: to prove any reasonable way of making statements about the world is either inconsistent or incomplete. As we noted, the first part of what we mean by reasonable is that it is an axiomatizable theory. The second part of what we mean by reasonable is that the theory is strong enough to describe programs. Definition 20. A theory is T sufficiently strong iff T represents every decidable set of strings. You might think that being sufficiently strong is a big hurdle to clear. But in fact it isn t very difficult at all. One of the things that Gödel and others showed is that even very simple theories whose axioms are just a few basic principles of logic and arithmetic can count as sufficiently strong. (But we won t go into the details of this.) Certainly any theory that we would treat as a serious attempt to describe our world will include that much. One important thing to notice is that if T is sufficiently strong, then it can represent some basic operations for putting together strings. In particular, suppose you have a description A(x) and a string s. Then one of the things you can do is replace each x in the description with the name for s, in order to get the sentence A s. This is a simple, mechanical thing to do, so it can be done by a program. That means that if T is a sufficiently strong theory, then there is some description x is the result of applying y to z, such that for any formula A(x) and string s, T proves A s is the result of applying A(x) to s. 2.3 An Unprovable Fact Now we have everything we need to prove the main fact about unprovability. 22

23 Gödel s First Incompleteness Theorem. No sufficiently strong axiomatizable theory is both consistent and complete. Proof. Let T be a sufficiently strong axiomatizable theory, and suppose that T is complete. We ll show that T is inconsistent. By Lemma 18, we know that T is decidable. Since T is sufficiently strong, this means that T represents the set of sentences that are provable in T. There is some description x is provable such that for any sentence A, if T proves A, then T proves A is provable, and if T does not prove A, then T proves A is not provable. We can now use the Liar paradox trick again: in place of the adjectives that do not self-apply, this time we ll look at the descriptions that cannot be proved to apply to themselves. Since T is sufficiently strong, we know T contains a description x is the result of applying y to z, as we discussed above. We can now put this together with the description x is provable to form this more complicated description: the result of applying x to x is not provable Call this description A(x). What we know is that for any description B(x), if the result of applying B(x) to B(x) is not provable, then T proves A B(x). Likewise, if the result of applying B(x) to B(x) is provable, then T proves not-a B(x). In short, since B B(x) is the result of applying B(x) to B(x), we can summarize this: (3) If T does not prove B B(x), then T proves A B(x) If T proves B B(x), then T proves not-a B(x) For the Liar paradox, we considered applying non-self-applying to itself. We can do the same trick here: we ll now let G be the sentence you get from plugging A(x) into itself. That is, G = A A(x) This is called the Gödel sentence. Since G is the result of applying A(x) to A(x), we can apply property 3, by specifically plugging in A(x) for B(x): (4) If T does not prove G, then T proves G If T proves G, then T proves not-g But (similar to the reasoning in the Liar paradox), this shows that T is inconsistent. Suppose first that T does not prove G. By the first part of 4, it follows that T proves G. But this is a contradiction; so T must prove G after all. But now, if T proves G, 23

24 by the second part of 4 it follows that T also proves not-g. So T proves both G and not-g. So T is inconsistent. What we have shown is that if T is axiomatizable, sufficiently strong, and complete, then T is inconsistent. We can put that the other way around: if T is axiomatizable, sufficiently strong, and consistent, then T is incomplete. Gödel s Theorem tells us something about any attempt to describe the world. If our theory is tractable, in the sense that it is possible in principle to determine what its basic principles say, and it includes some basic logic and arithmetic, and it does not contradict itself, then our theory is guaranteed to leave out some of the facts. In this sense, it is impossible to give a theory of everything. 2.4 Consistency is Unprovable What we ve shown is that for any sufficiently strong, consistent axiomatizable theory, there is some true statement that it cannot prove. But Gödel showed something more: he gave a specific example of an unprovable statement which is of particularly deep importance. Any sufficiently strong axiomatizable theory has the resources to talk about what is provable in that very theory. So one of the things it can talk about is whether it can prove any contradictions. That is, if T is a sufficiently strong axiomatizable theory, then it includes a sentence that says T is consistent that is, one which says no contradiction is provable in T. The second important thing Gödel showed is that if T really is consistent, then this true statement is unprovable. No reasonable theory can prove its own consistency. This is called Gödel s Second Incompleteness Theorem. The basic idea of the proof is that in a reasonable theory T, the reasoning behind Gödel s First Incompleteness Theorem can be formalized. The steps we went through to justify First Incompleteness Theorem can also be carried out in an official proof in T. We won t go through this in detail, but we ll look at the main ideas. The first step is to come up with a description x is provable. (This doesn t work quite the same way as in the First Incompleteness Theorem, because this time we are not assuming that T is consistent and complete. But the idea is similar. We won t worry too much here about the exact details about how this description works.) Note that T is inconsistent if and only if T proves, for example, that 0 1 and 0 = 1. So let T is consistent be a shorthand for 0 1 and 0 = 1 is not provable. What we are going to show is that if T proves this sentence, then T is inconsistent. The second step is to use the idea of the First Incompleteness Theorem to come up with a Gödel sentence. In particular, we can define a sentence G in the same way as 24