General guidelines: You may work with other people, as long as you write up your solution in your own words and understand everything you turn in. Make sure to justify your answers they should be clear and concise, with no irrelevant information. Please put your name and net id on your homework! 1. Select a topic from the list of topics on the course website beyond finite automata. You may select any topic after the heading context-free languages. Write a half page explanation of your topic that should be understandable to any student in the class. Then create two problems on the topic and work out solutions. These might be used by someone teaching the course as examples. Then create eight more homework exercises that would help a student understand the topic. To illustrate what is intended the following are three write ups by students. Diagonalization If we want to compare the cardinalities of two finite sets, we can simply count the elements in each set and compare the number of elements. For infinite sets, such as the set of integers or the set of real numbers, we can compare the cardinalities of two sets by finding a one-to-one correspondence between the elements of the sets. A set is countably infinite if its elements can be put in one-to-one correspondence with integers. For example, we can create a one-to-one correspondence between the set of all even numbers and the set of all integers. Therefore, the set of all even numbers is countably infinite. Even numbers: 0 2 4 6 8 Integers: 0 1 2 3 4 If a set is countable, then we can list the elements of the set or, equivalently, we can put the elements of the set in a table. We can show that certain sets are not countably infinite by diagonalization. For example, diagonalization can be used to show that the set of all real numbers between 0 and 1 is not countably infinite. To do this, try to put all the elements in the following table:
0. 1 0 1 0 1 0 0. 0 1 0 1 0 1 0. 1 1 1 1 1 1 0. 0 0 1 0 0 1 0. 1 1 0 1 1 0 0. 0 0 0 0 0 1 By selecting all the elements along the diagonal beginning with the digit in the tenths place, namely 111011, we have selected an element that has not already been listed in the table. This shows that all the real numbers cannot be listed in a table. Thus, the set of real numbers is not countably infinite. There exist a countably infinite number of computer programs, or names, but there are too many sets, or functions to be counted. Thus, there exist functions that cannot be computed by a computer program. Exercises: 1. Write a short essay about diagonalization that would explain to a high school student the concepts of countably infinite, not countably infinite, and diagonalization. It should talk about how to compare the cardinality of two sets and it should give a convincing argument that there exists a non countably infinite set. 2. Prove that the set of all rational numbers is countably infinite. 3. Prove the following: (a) The set of strings {0,1}* is countably infinite. (b) The set of all subsets of {0,1}* is not countably infinite. 4. Compare the cardinalities of the following pairs of sets: (a) The set of all real numbers between 0 and 1 and the set of all integers.
(b) The set of all finite length strings over the alphabet {0,1} and the set of all finite length strings over the alphabet {0,1,2,3}. (c) The set of all odd numbers between 0 and 100 inclusive and the set of all negative numbers greater than -75. 5. Consider the set of all finite length strings over the alphabet {0,1} and the set of all finite length strings over the alphabet {0,1,2,3}. Give a one-to-one mapping between the elements of the two sets. 6. Compare the cardinalities of the set of all real numbers and the set of all perfect squares. 7. What is the cardinality of the set of all square integer matrices? The size of the matrix can be arbitrarily large as well as the value of any entry. Pumping Lemma Consider a deterministic finite automaton that accepts some language L. If L defines an infinite number of strings, such as 0*1*, there must be some string in the language that has more symbols than the number of states in the machine, because the machine has a finite number of states. If the string is processed by the machine, at least one of the states must be visited more than once due to the pigeonhole principle. Thus, there must be a loop in the DFA beginning and ending at that state. Let us denote by X the substring which sends the DFA through the loop. Clearly, the machine should then accept a string with as many contiguous copies of X as you want. So, for instance, if the machine accepted 0X1, then it should accept 0XX1, 0XXX1, or even 0XXXXXX1, because the machine will just loop over and over until it is time to continue on. Essentially the pumping lemma says is that if you have a regular language and a machine that accepts it, then for the strings in the language whose lengths exceed the number of states in the machine there must be some substring that can be looped as many times you want and still be an accepted string. This theorem can be used to prove that a language is not regular. The proof proceeds by contradiction. Assume that the language is regular and that we have some DFA which accepts it. If we can find a string in that language which, after duplicating any substring gives something not accepted by the DFA (and thus not in the language), then clearly the pumping lemma is not satisfied, and thus the string cannot have been regular to begin with. Example 1: L = {0 i 1 i i 1}
Assume that this language is regular, and that some DFA with n states accepts it. Consider the string 0 n 1 n. Clearly this string has 2n symbols. Because this exceeds the number of states in the machine, one state must be visited at least more than once somewhere in the first n symbols. In this case, that means the loop in the DFA must be some length substring consisting of all 0 s. Write the string as some string xyz, where x is the part of the string before the loop, y is the piece between the repeated state, and z is the piece of the string after the loop. It should be clear that if there are n states in the DFA, xy n. The pumping lemma states that for any string in the language, xy j z is also in the language, for any j 0. However, no matter how you cut 0 n 1 n up into xyz with xy n, y must necessarily be made up entirely of 0 s. Thus, if we considered the string xy 2 z we would get something where there are clearly more 0 s than 1 s (because all of the 1 s are in z), and thus is not a part of the language. This is a contradiction. Because the pumping lemma doesn t hold for all the strings in our language, the language is not regular. Example 2: L = {ww w (0+1)*} Assume that this language is regular, and that some DFA with n states accepts it. Consider the string 0 n 1 n 0 n 1 n. Clearly this string has 4n symbols. Because this exceeds the number of states in the machine, one state must be visited at least more than once somewhere in the first n symbols. In this case, that means the loop in the DFA must be some length substring consisting of all 0 s. Write the string as some string xyz, where x is the part of the string before the loop, y is the piece between the repeated state, and z is the piece of the string after the loop. It should be clear that if there are n states in the DFA, xy n. The pumping lemma states that for any string in the language, xy j z is also in the language, for any j 0. However, no matter how you cut 0 n 1 n 0 n 1 n up into xyz with xy n, y must necessarily be made up entirely of 0 s. Thus, if we considered the string xy 2 z we would get something where there are clearly more 0 s in the first block than in the second (because the second block of zeros are all in z), and thus is not a part of the language. This is a contradiction. Because the pumping lemma doesn t hold for all the strings in our language, the language is not regular. Exercises: Prove the following languages to be non-regular:
1.) {0 n n is a power of 2} 2.) The set of strings of the form w1 n, where w is a string of 0 s and 1 s of length n. 3.) The set of strings of balanced parentheses. These are the strings of characters ( and ) that can appear in a well-formed arithmetic expression. 4.) {0 p p is a prime number} 5.) {a n b n c n n 0} 6.) {a n b m c n d m n,m 0} 7.) {a i b j c k 0 i j k} Mathematical Induction Mathematical induction is a proof technique used to show that a statement involving a number n is true for any n k, where n and k are natural numbers. This proof is done by proving that 1) the statement is true for n = k (the base case), and that 2) if the statement is true for n, it is also true for n+1 (the inductive step). Example 1. Let s use dominos as an example to demonstrate this technique. In order to prove that all dominos will fall over, we must show two things: that the first one is knocked over (our base case), and that each domino will knock over its neighbor (our inductive step). Obviously, if we have shown these two facts true, than we have also shown that all the dominos will fall over. Example 2. Now let s move to a more formal example. Let s prove that = (that is, that the sum of natural numbers from 1 to n is ) for n 0. Induction Hypothesis (S n ): = Base Case (S 0 ): We must show that our induction hypothesis is true for n=0: = = 0.The sum of all natural numbers from 0 to 0 is indeed 0. Base Case is true. Inductive Step (S n => S n+1 ): We must now show that if S n is true, this implies that S n+1 is true as well. Let s assume that S n is true, that is, that =. Now let s show that this implies that =. = (The sum of all natural numbers from 0 to n+1 equals the sum all natural numbers from 0 to n and n+1.) = (By the induction hypothesis, we can replace with.) = = = (Using algebraic transformations, we get the expression we wanted.) We have thus shown that = implies that =. Conclusion: By induction, we have shown that our induction hypothesis, =, is true for all n 0. Induction is really useful in computer science because of its strong link to recursion. Its techniques can be used on recursively defined structures such as trees.
Once again, to prove a statement S n true for all n k using induction, we must 1) prove our base case, S k, and 2) prove that S n S n+1. Homework problems. #1 Prove that = for all natural numbers. #2 Prove that = for all natural numbers. #3 Prove that for all natural numbers. #4 Prove that for all natural numbers. #5 Let be the number of nodes and be the number of levels in a binary tree. Prove that. #6 The Towers of Hanoi is a puzzle involving three rods and a number of disks all of which are a different size and fit onto the rods. Initially, all the disks are on the first rod in order of increasing size from top to bottom. The point of the game is to move all the disks from one rod to another, without breaking the following two rules: 1) only one disk may be moved at a time; 2) a larger disk may never be on top of a smaller disk. Prove by induction that this may be done for any number of disks.