CSCI 270: Introduction to Algorithms and Theory of Computing Fall 2017 Prof: Leonard Adleman Scribe: Joseph Bebel
|
|
- Laurence Beasley
- 5 years ago
- Views:
Transcription
1 CSCI 270: Introduction to Algorithms and Theory of Computing Fall 2017 Prof: Leonard Adleman Scribe: Joseph Bebel We will now discuss computer programs, a concrete manifestation of what we ve been calling algorithms. Computer programs are written in specific programming languages. Here, for simplicity we will consider a simplified version of BASIC that we call SBASIC. It will have the usual arithmetic operations and usual decision logic that most programming languages have (if/then/else, for loops, goto, etc). One major simplification we will make is to assume that the input and output of an SBASIC program are natural numbers, and that input can only occur in the first line of a program and output can only occur on the last line. We say that an SBASIC program halts on a given input, if when executed on that input, the program reaches this last line and produces some output. Note that some SBASIC programs may halt on some inputs and not halt on other inputs. One important concept is that each SBASIC program has an index. Let us put the SBASIC programs in lexicographical order based on their source code. Then there is a 1st SBASIC program, a 2nd SBASIC program, and so on (ad infinitum). Since there are an infinite number of SBASIC programs, each SBASIC program has a unique natural number as its index, and each natural number corresponds to a unique SBASIC program. You can also think of the index as the natural number (binary string) corresponding to that SBASIC program s source code. We can ask, for a set of natural numbers, if there exists an SBASIC program that decides if a number is in that set or not: The Decision Problem for S N INPUT: n, a natural number OUTPUT: 1 if n S 0 if n S For example, consider the set E = {0, 2, 4, 6, 8, 10,...} of even numbers. The decision problem for E is then: on input an even number, output 1, otherwise output 0. DEF: 1. A subset S N is decidable iff there exists an SBASIC program that solves the decision problem for S DEF: 2. A subset S N is undecidable iff there does not exist an SBASIC program that solves the decision problem for S Note that the set E is decidable; the SBASIC program just needs to look at one bit of the input to decide if the input is even or odd. 1
2 We can now pose the following question: given an SBASIC program and an input, does it halt on that input? We can more precisely ask this question about SBASIC programs that are given their own index as input. It turns out that there is no SBASIC program that can solve that decision problem: Theorem 1. Undecidability of the Halting Problem K = {i B i on input i halts } is undecidable. Proof. Assume there exists an SBASIC program P that solves the decision problem for K. execute P on input G if P outputs 1: A: GOTO A else if P outputs 0: PRINT 0 Let c be the index of B c. Is c K? if c K, then since P solves the decision problem for K, P outputs 1 on input c. Therefore, B c on input c goes into an infinite loop. Therefore, c K, which is a contradiction if c K, then since P solves the decision problem for K, P outputs 0 on input c. Therefore, B c on input c prints 0. Therefore, c K, which is a contradiction Since we reach a contradiction in both cases, our original assumption must be false, and there does not exist an SBASIC program that solves the decision problem for K, and K is undecidable. The Decision Problem for K INPUT: n, a natural number OUTPUT: 1 if n K 0 if n K 2
3 So have we proven that the decision problem for K is not solvable? Not quite; we ve shown that there does not exist an SBASIC program that solves the decision problem for K, but maybe you can solve it in some other programming language like C++. We make the claim that, actually, every program in another programming language can be translated or compiled into an SBASIC program with equivalent behavior. This concept has been explored over the years and has resulted in something we might call Church s Thesis : Church s Thesis: If an algorithm has an input/output behavior then there is an SBASIC program with the same input/output behavior. Note that Church s Thesis is philosophy, not science or mathematics. It says that every machine that you can conceivably build in the real world, whatever its input/output is, can be simulated by some equivalent SBASIC program. This statement of Church s Thesis is more frequently stated with Turing machines rather than SBASIC programs. But Turing machines are horrible to program, and in principle, it doesn t matter; SBASIC programs (or C++ programs, or Java programs, etc) are all equivalent in this regard. But we cannot prove philosophy, so we have to accept that whatever we prove about SBASIC programs will apply to all computers and computer programs which might exist in the future. So the tool (the computer) which (presumably) we are dedicating our lives to studying and using is fundamentally flawed; in fact, most problems are undecidable, and most problems 3
4 which are decidable are not solvable in polynomial time. It s the fundamental limits of the tool. Another point to make about the Undecidability of the Halting Problem; it depends greatly on self-reference or what is sometimes called diagonalization. It is like the sentence This sentence is false ; it cannot be true, because then it would imply its own falsehood, and it cannot be false, because then it would be true. While such logical contradictions are more mathematical toys than anything else, the basic concept of self-reference is critical to the Undecidability of the Halting Problem. Let s consider another mind experiment. Can we build a robot that can assemble exact copies of itself? Let s say that we build such a robot and leave it in a big warehouse full of its parts. Assuming you build a sufficiently sophisticated robot, maybe it can assemble the parts into a copy of itself. But then it remains the question of how the new robot gets its software. Somehow, it has to be copied from the old robot. Maybe this is possible to do by ensuring that the software is in some readable form on the disk of the robot. But actually, that is not necessary. Kleene Recursion Theorem: put simply, every program can compute its own index. That is, every program can have a line of code which performs the operation j = MY INDEX. So a program, mid-way through its execution, can somehow (we don t say how) obtain a reference or a copy of itself. Why is this important? Consider the following set and the decision problem for that set: S = {i B i on input 0 outputs 1 } At first glance, it may not be clear whether this problem is decidable or undecidable. On one hand, it seems like just as difficult a problem as the decision problem for K. On the other hand, it is not immediately clear how to obtain self-reference; for the decision problem for K, we only talk about the behavior of each SBASIC program on input its own index, so in the counterexample we just assume the input is the index. However, using Kleene s recursion theorem, we can show this set is also undecidable: Theorem 2. S is undecidable. Proof. Assume there exists an SBASIC program P that solves the decision problem for S. N MY INDEX execute P on input N if P outputs 1: A: GOTO A else if P outputs 0: PRINT 1 Let c be the index of B c. Is c S? 4
5 if c S, then since P solves the decision problem for S, P outputs 1 on input c. Therefore, B c on input 0 goes into an infinite loop. Therefore, c S, which is a contradiction if c S, then since P solves the decision problem for S, P outputs 0 on input c. Therefore, B c on input 0 prints 0. Therefore, c S, which is a contradiction Since we reach a contradiction in both cases, our original assumption must be false, and there does not exist an SBASIC program that solves the decision problem for S, and S is undecidable. T = {i ( n)[b i on input n halts ]} Theorem 3. T is undecidable. Proof. Assume there exists an SBASIC program P that solves the decision problem for T. N MY INDEX execute P on input N if P outputs 1: A: GOTO A else if P outputs 0: PRINT 0 Let c be the index of B c. Is c T? if c T, then since P solves the decision problem for T, P outputs 1 on input c. Therefore, B c on all inputs goes into an infinite loop. Therefore, c T, which is a contradiction if c T, then since P solves the decision problem for T, P outputs 0 on input c. Therefore, B c on all inputs prints 0. Therefore, c T, which is a contradiction Since we reach a contradiction in both cases, our original assumption must be false, and there does not exist an SBASIC program that solves the decision problem for T, and T is undecidable. 5
6 So there are decision problems that cannot be solved by SBASIC programs. Can we still say something (computationally) about these undecidable sets, or are computers totally useless here? We will introduce the concept of recursively enumerable/listable; to do so, we relax the definition of SBASIC program to allow output at arbitrary lines (and to continue executing after producing output) DEF: 3. A set S N is recursively enumerable, or listable iff there exists an SBASIC program P that (given no input) prints a sequence of numbers such that: 1. P never prints a number not in S 2. for all elements n S, P will eventually output n (if given enough time) Note that if the set S has infinitely many elements, then a program that lists S must run forever. Every decidable set S is also listable: just loop over all numbers n, run the decision problem for S solver to decide if n is in the set, and if so output n; otherwise go to the next number. But is every listable set also decidable? Theorem 4. K is listable. Proof. The following program lists K: for i 1 to infinity: for j 1 to i: execute B j on input j for i steps if B j halts within i steps, output j. otherwise, continue to next j We need to show that the output follows the definition of listable. Assume a number n is output by this program: then that must have meant that B n was run on input n for some number of steps, and B n halted. Therefore n K and the program correctly output n. Does it output every element of K? Assume n K. Then it must be that B n halts on input n after t steps for some number t. Since i, j loop over all numbers to infinitely, there must be an iteration of the loop where i = n and j = t, in which case B n will be run on input n for t steps, halt, and n is output. Therefore, K is listable. Are there any unlistable sets? Theorem 5. For all X N, if X is undecidable, then either X is not listable or X is not listable. Proof. Assume P lists X and P lists X. Consider the following program: 6
7 for i 1 to infinity: run P until it outputs i numbers. If the i th number is G, then output 1. run P until it outputs i numbers. If the i th number is G, then output 0. This program decides X. Note that every number n must be in either X or X. Therefore, either P or P will eventually output n. We run both listers until one of them outputs n, then we have decided whether n X. If X is undecidable, then it must be the case that either X or X is not listable. Theorem 6. K is not listable. Is it possible for both a set and its complement to be unlistable? Yes: Theorem 7. T is not listable. Proof. Assume there exists an SBASIC program P that lists T. N MY INDEX execute P until it outputs G numbers if the G th output of P is N: A: GOTO A else: PRINT 0 Let c be the index of B c. Is c T? if c T, then since P lists T, P eventually outputs c. Therefore, for some input G, B c will compute the G th output of P, find it is c, and go into an infinite loop. Therefore, c T, which is a contradiction if c T, then since P lists T, on all possible inputs G, the G th output of P will never be c. Therefore, B c on all inputs prints 0. Therefore, c T, which is a contradiction Since we reach a contradiction in both cases, our original assumption must be false, and there does not exist an SBASIC program that lists T, and T is unlistable. Theorem 8. T is not listable. Proof. Assume there exists an SBASIC program P that lists T. N MY INDEX for i 1 to infinity: execute P until it outputs i numbers 7
8 if the i th output of P is N: GOTO A else next i A: PRINT 0 Let c be the index of B c. Is c T? if c T, then since P lists T, P eventually outputs c. Therefore, for some value of i, B c will compute the i th output of P, find it is c, and goto the PRINT 0 line. Therefore, c T, which is a contradiction if c T, then since P lists T, for all values of i, the i th output of P will never be c. Therefore, B c will never exit the loop, always incrementing i, and thus never halt. Therefore, c T, which is a contradiction Since we reach a contradiction in both cases, our original assumption must be false, and there does not exist an SBASIC program that lists T, and T is unlistable. Therefore T and T are both undecidable and both unlistable. Is unlistable the worst thing that a set can be? Shortest Program: Consider the string Would you think that this string was generated randomly (e.g. by flipping a coin and writing 0 if you get heads and 1 if you get tails)? Of course it is possible to get this sequence by coin flips, but it s not likely. If our concern is getting roughly an equal number of 0 s and 1 s, then what about the string ? Is this likely to come from coin flips? Again, it is possible to get this string randomly, but there is something about it that suggests it was not random. It can sometimes be difficult to tell. For example, the string This might begin to look more random, but in fact it is the digits 0-8, each written in binary, concatenated together. One problem with trying to call these strings random or not random is that they are all equally likely to occur as any other 22 bit string. That is: you get each possible string with probability 2 22 including and (an actual string generated randomly). So why does one string seem more random than the other? Kolmogorov gave a computational answer to this question. He observed that non-random strings have short programs that output them. For example, consider a 1-million bit long string of only 0 s. There is a program of length approximately 1 million bits that outputs that string: simply hard-code the string into the program, and output it. But there is a much shorter program, which simply has a loop that counts from 1 to 1 million and outputs 0 each iteration. This program is much shorter than the hard-coding program, yet produces the same output. 8
9 Similar short programs can output the other non-random strings we ve discussed. However, for a string generated randomly, with high probability it does not have a shorter program than the one that outputs a hard-coded string. DEF: 4. For all numbers n, an SBASIC program P is a shortest program for n iff: 1. On input 0, P outputs n 2. There does not exist another program P that on input 0 outputs n, and the length of P is less than the length of P. The length of a program P is the number of ASCII characters in its source code. Sh = {i ( n)[b i is a shortest program for n]}. DEF: 5. For all X N, X is immune iff 1. X is infinite 2. X has no infinite subset that is listable. Theorem 9. Sh is immune. Proof. Assume there exists an SBASIC program P that lists some infinite subset of Sh. N MY INDEX for i 1 to infinity execute P until it produces its i th output, k if B k is an SBASIC program longer than B N : execute B k on input 0 PRINT whatever B k outputs else next i What is the output of B c on input 0? It will compute its own index first, then run P until it outputs a shortest program that is longer than B c. Note that this must occur if P lists an infinite subset of Sh, since there are only finitely many programs shorter than or equal in length to B c (assuming SBASIC programs are written in ASCII). So after outputting that finite number of elements of Sh, P must eventually output the index of an SBASIC program B k longer than B c. When B c now executes that other SBASIC program, B c will produce the same output as B k. However, B c is shorter than B k, contradicting the claim that B k was a shortest program and that k Sh. Therefore, P does not list an infinite subset of Sh, and Sh is immune. 9
10 The study of shortest programs has implications for data compression. Consider a PNG image, which is a lossless compression of a bitmap. Some images are very compressible; they are very large images with very short representations. The compressed form can be considered to be a short program that outputs that image. Yet we know that a random bitmap image is incompressible; there does not exist a representation that is much shorter than the raw data. 10
Recursively Enumerable Languages, Turing Machines, and Decidability
Recursively Enumerable Languages, Turing Machines, and Decidability 1 Problem Reduction: Basic Concepts and Analogies The concept of problem reduction is simple at a high level. You simply take an algorithm
More informationMaterial from Recitation 1
Material from Recitation 1 Darcey Riley Frank Ferraro January 18, 2011 1 Introduction In CSC 280 we will be formalizing computation, i.e. we will be creating precise mathematical models for describing
More informationWe show that the composite function h, h(x) = g(f(x)) is a reduction h: A m C.
219 Lemma J For all languages A, B, C the following hold i. A m A, (reflexive) ii. if A m B and B m C, then A m C, (transitive) iii. if A m B and B is Turing-recognizable, then so is A, and iv. if A m
More informationNotes on Turing s Theorem and Computability
Notes on Turing s Theorem and Computability Walter Neumann About 60 years ago there was a revolution in mathematics and philosophy. First Gödel and then Turing showed that there are impossible problems
More information6. Advanced Topics in Computability
227 6. Advanced Topics in Computability The Church-Turing thesis gives a universally acceptable definition of algorithm Another fundamental concept in computer science is information No equally comprehensive
More informationCourse notes for Data Compression - 2 Kolmogorov complexity Fall 2005
Course notes for Data Compression - 2 Kolmogorov complexity Fall 2005 Peter Bro Miltersen September 29, 2005 Version 2.0 1 Kolmogorov Complexity In this section, we present the concept of Kolmogorov Complexity
More informationLecture 5: The Halting Problem. Michael Beeson
Lecture 5: The Halting Problem Michael Beeson Historical situation in 1930 The diagonal method appears to offer a way to extend just about any definition of computable. It appeared in the 1920s that it
More informationToday, we will dispel the notion that Java is a magical language that allows us to solve any problem we want if we re smart enough.
CSE 311: Foundations of Computing Lecture 25: Limits of Computation Showing a Language L is not regular 1. Suppose for contradiction that some DFA M accepts L. 2. Consider an INFINITE set of half strings
More informationTheory of Programming Languages COMP360
Theory of Programming Languages COMP360 Sometimes it is the people no one imagines anything of, who do the things that no one can imagine Alan Turing What can be computed? Before people even built computers,
More informationp x i 1 i n x, y, z = 2 x 3 y 5 z
3 Pairing and encoding functions Our aim in this part of the course is to show that register machines can compute everything that can be computed, and to show that there are things that can t be computed.
More information14.1 Encoding for different models of computation
Lecture 14 Decidable languages In the previous lecture we discussed some examples of encoding schemes, through which various objects can be represented by strings over a given alphabet. We will begin this
More informationLimitations of Algorithmic Solvability In this Chapter we investigate the power of algorithms to solve problems Some can be solved algorithmically and
Computer Language Theory Chapter 4: Decidability 1 Limitations of Algorithmic Solvability In this Chapter we investigate the power of algorithms to solve problems Some can be solved algorithmically and
More informationDiagonalization. The cardinality of a finite set is easy to grasp: {1,3,4} = 3. But what about infinite sets?
Diagonalization Cardinalities The cardinality of a finite set is easy to grasp: {1,3,4} = 3. But what about infinite sets? We say that a set S has at least as great cardinality as set T, written S T, if
More informationCS 275 Automata and Formal Language Theory. First Problem of URMs. (a) Definition of the Turing Machine. III.3 (a) Definition of the Turing Machine
CS 275 Automata and Formal Language Theory Course Notes Part III: Limits of Computation Chapt. III.3: Turing Machines Anton Setzer http://www.cs.swan.ac.uk/ csetzer/lectures/ automataformallanguage/13/index.html
More informationComputer Sciences Department
1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER 3 D E C I D A B I L I T Y 4 Objectives 5 Objectives investigate the power of algorithms to solve problems.
More informationTuring Machine Languages
Turing Machine Languages Based on Chapters 23-24-25 of (Cohen 1997) Introduction A language L over alphabet is called recursively enumerable (r.e.) if there is a Turing Machine T that accepts every word
More informationUniversity of Nevada, Las Vegas Computer Science 456/656 Fall 2016
University of Nevada, Las Vegas Computer Science 456/656 Fall 2016 The entire examination is 925 points. The real final will be much shorter. Name: No books, notes, scratch paper, or calculators. Use pen
More information2.8 Universal Turing Machines and the Halting Problem
2.8 Universal Turing Machines and the Halting Problem Through the hierarchy of Slide 74 we indirectly get a sense that Turing Machines are at least as computationally powerful as any other known model
More informationUNIT 14C The Limits of Computing: Non computable Functions. Problem Classifications
UNIT 14C The Limits of Computing: Non computable Functions 1 Problem Classifications Tractable Problems Problems that have reasonable, polynomialtime solutions Intractable Problems Problems that may have
More informationMergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015
CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: April 1, 2015 1 Introduction Today, we will introduce a fundamental algorithm design paradigm, Divide-And-Conquer,
More informationintroduction to Programming in C Department of Computer Science and Engineering Lecture No. #40 Recursion Linear Recursion
introduction to Programming in C Department of Computer Science and Engineering Lecture No. #40 Recursion Linear Recursion Today s video will talk about an important concept in computer science which is
More informationThe Undecidable and the Unprovable
The Undecidable and the Unprovable Jeff Sanford Russell Spring 2017 The main purpose of these notes is to help you understand some beautiful and important facts about the limits of computation and logic.
More informationLecture T4: Computability
Puzzle ("Post s Correspondence Problem") Lecture T4: Computability Given a set of cards: N card types (can use as many of each type as possible) Each card has a top string and bottom string Example : N
More informationCSE 120. Computer Science Principles
Adam Blank Lecture 17 Winter 2017 CSE 120 Computer Science Principles CSE 120: Computer Science Principles Proofs & Computation e w h e q 0 q 1 q 2 q 3 h,e w,e w,h w,h q garbage w,h,e CSE = Abstraction
More informationOutline. Language Hierarchy
Outline Language Hierarchy Definition of Turing Machine TM Variants and Equivalence Decidability Reducibility Language Hierarchy Regular: finite memory CFG/PDA: infinite memory but in stack space TM: infinite
More informationWhat computers just cannot do. COS 116: 2/28/2008 Sanjeev Arora
What computers just cannot do. COS 116: 2/28/2008 Sanjeev Arora Administrivia In class midterm in midterms week; Thurs Mar 13 (closed book;? No lab in midterms week; review session instead. What computers
More informationIn this section we will study problems for which we can prove that there is no algorithm solving them.
8 Uncomputability In this section we will study problems for which we can prove that there is no algorithm solving them. 8.1 What is an algorithm? The notion of algorithm is usually defined as Turing machines
More informationTAFL 1 (ECS-403) Unit- V. 5.1 Turing Machine. 5.2 TM as computer of Integer Function
TAFL 1 (ECS-403) Unit- V 5.1 Turing Machine 5.2 TM as computer of Integer Function 5.2.1 Simulating Turing Machine by Computer 5.2.2 Simulating Computer by Turing Machine 5.3 Universal Turing Machine 5.4
More informationWeek - 01 Lecture - 03 Euclid's Algorithm for gcd. Let us continue with our running example of gcd to explore more issues involved with program.
Programming, Data Structures and Algorithms in Python Prof. Madhavan Mukund Department of Computer Science and Engineering Indian Institute of Technology, Madras Week - 01 Lecture - 03 Euclid's Algorithm
More informationThe Big Picture. Chapter 3
The Big Picture Chapter 3 Examining Computational Problems We want to examine a given computational problem and see how difficult it is. Then we need to compare problems Problems appear different We want
More informationTheory of Computation, Homework 3 Sample Solution
Theory of Computation, Homework 3 Sample Solution 3.8 b.) The following machine M will do: M = "On input string : 1. Scan the tape and mark the first 1 which has not been marked. If no unmarked 1 is found,
More informationTime Complexity of an Algorithm
CPSC 211 Data Structures & Implementations (c) Texas A&M University [ 335] Time Complexity of an Algorithm Time complexity of an algorithm: the function T (n) that describes the (worst-case) running time
More informationSystem Correctness. EEC 421/521: Software Engineering. System Correctness. The Problem at Hand. A system is correct when it meets its requirements
System Correctness EEC 421/521: Software Engineering A Whirlwind Intro to Software Model Checking A system is correct when it meets its requirements a design without requirements cannot be right or wrong,
More informationChapter 9 Graph Algorithms
Introduction graph theory useful in practice represent many real-life problems can be if not careful with data structures Chapter 9 Graph s 2 Definitions Definitions an undirected graph is a finite set
More information05. Turing Machines and Spacetime. I. Turing Machines and Classical Computability.
05. Turing Machines and Spacetime. I. Turing Machines and Classical Computability. 1. Turing Machines A Turing machine (TM) consists of (Turing 1936): Alan Turing 1. An unbounded tape. Divided into squares,
More informationMergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri
CS161, Lecture 2 MergeSort, Recurrences, Asymptotic Analysis Scribe: Michael P. Kim Date: September 28, 2016 Edited by Ofir Geri 1 Introduction Today, we will introduce a fundamental algorithm design paradigm,
More informationTopology and Topological Spaces
Topology and Topological Spaces Mathematical spaces such as vector spaces, normed vector spaces (Banach spaces), and metric spaces are generalizations of ideas that are familiar in R or in R n. For example,
More informationDenotational semantics
1 Denotational semantics 2 What we're doing today We're looking at how to reason about the effect of a program by mapping it into mathematical objects Specifically, answering the question which function
More informationChapter 9 Graph Algorithms
Chapter 9 Graph Algorithms 2 Introduction graph theory useful in practice represent many real-life problems can be if not careful with data structures 3 Definitions an undirected graph G = (V, E) is a
More informationVariants of Turing Machines
November 4, 2013 Robustness Robustness Robustness of a mathematical object (such as proof, definition, algorithm, method, etc.) is measured by its invariance to certain changes Robustness Robustness of
More information1 Algorithms, Inputs, Outputs
1 Algorithms, Inputs, Outputs In the last class we discussed Universal Turing machines and the evolution in thought from circuitry being the important thing to the table of state transitions, or computer
More informationEnglish for Computer Science
English for Computer Science Mohammad Farshi Department of Computer Science, Yazd University 1388-1389 (CS Dept. Yazd U.) Yazd Univ. English4CS 1388-1389 1 / 15 Azmoone 1389(CS) Azmoone 1389(CS) (CS Dept.
More informationLambda Calculus and Computation
6.037 Structure and Interpretation of Computer Programs Chelsea Voss csvoss@mit.edu Massachusetts Institute of Technology With material from Mike Phillips and Nelson Elhage February 1, 2018 Limits to Computation
More information1. [5 points each] True or False. If the question is currently open, write O or Open.
University of Nevada, Las Vegas Computer Science 456/656 Spring 2018 Practice for the Final on May 9, 2018 The entire examination is 775 points. The real final will be much shorter. Name: No books, notes,
More informationComputability, Cantor s diagonalization, Russell s Paradox, Gödel s s Incompleteness, Turing Halting Problem.
Computability, Cantor s diagonalization, Russell s Paradox, Gödel s s Incompleteness, Turing Halting Problem. Advanced Algorithms By Me Dr. Mustafa Sakalli March, 06, 2012. Incompleteness. Lecture notes
More informationTyping Control. Chapter Conditionals
Chapter 26 Typing Control 26.1 Conditionals Let s expand our language with a conditional construct. We can use if0 like before, but for generality it s going to be more convenient to have a proper conditional
More informationComputability Summary
Computability Summary Recursive Languages The following are all equivalent: A language B is recursive iff B = L(M) for some total TM M. A language B is (Turing) computable iff some total TM M computes
More informationThe Turing Machine. Unsolvable Problems. Undecidability. The Church-Turing Thesis (1936) Decision Problem. Decision Problems
The Turing Machine Unsolvable Problems Motivating idea Build a theoretical a human computer Likened to a human with a paper and pencil that can solve problems in an algorithmic way The theoretical machine
More informationComputational Complexity and Implications for Security DRAFT Notes on Infeasible Computation for MA/CS 109 Leo Reyzin with the help of Nick Benes
Computational Complexity and Implications for Security DRAFT Notes on Infeasible Computation for MA/CS 109 Leo Reyzin with the help of Nick Benes The Study of Computational Complexity Let s summarize what
More informationEnumerations and Turing Machines
Enumerations and Turing Machines Mridul Aanjaneya Stanford University August 07, 2012 Mridul Aanjaneya Automata Theory 1/ 35 Finite Sets Intuitively, a finite set is a set for which there is a particular
More information11/22/2016. Chapter 9 Graph Algorithms. Introduction. Definitions. Definitions. Definitions. Definitions
Introduction Chapter 9 Graph Algorithms graph theory useful in practice represent many real-life problems can be slow if not careful with data structures 2 Definitions an undirected graph G = (V, E) is
More informationPresented By : Abhinav Aggarwal CSI-IDD, V th yr Indian Institute of Technology Roorkee. Joint work with: Prof. Padam Kumar
Presented By : Abhinav Aggarwal CSI-IDD, V th yr Indian Institute of Technology Roorkee Joint work with: Prof. Padam Kumar A Seminar Presentation on Recursiveness, Computability and The Halting Problem
More informationChapter 9 Graph Algorithms
Chapter 9 Graph Algorithms 2 Introduction graph theory useful in practice represent many real-life problems can be slow if not careful with data structures 3 Definitions an undirected graph G = (V, E)
More informationTheory of Computations Spring 2016 Practice Final Exam Solutions
1 of 8 Theory of Computations Spring 2016 Practice Final Exam Solutions Name: Directions: Answer the questions as well as you can. Partial credit will be given, so show your work where appropriate. Try
More informationLecture Notes on Contracts
Lecture Notes on Contracts 15-122: Principles of Imperative Computation Frank Pfenning Lecture 2 August 30, 2012 1 Introduction For an overview the course goals and the mechanics and schedule of the course,
More informationPrim Rec, Decidable, Undecidable, and Beyond. William Gasarch-U of MD
Primitive Recursive An attempt to pin down the set of functions that are computable. A function f is Primitive Recursive 1) f (x 1,..., x n ) = 0 OR f (x 1,..., x n ) = x i + c, c N. 2) If g 1 (x 1,...,
More informationLanguages and Strings. Chapter 2
Languages and Strings Chapter 2 Let's Look at Some Problems int alpha, beta; alpha = 3; beta = (2 + 5) / 10; (1) Lexical analysis: Scan the program and break it up into variable names, numbers, etc. (2)
More informationComputable Sets. KR Chowdhary Professor & Head.
Computable Sets KR Chowdhary Professor & Head Email: kr.chowdhary@acm.org Department of Computer Science and Engineering MBM Engineering College, Jodhpur March 19, 2013 kr chowdhary Comp-Set 1/ 1 Computable
More informationIntroduction to Automata Theory. BİL405 - Automata Theory and Formal Languages 1
Introduction to Automata Theory BİL405 - Automata Theory and Formal Languages 1 Automata, Computability and Complexity Automata, Computability and Complexity are linked by the question: What are the fundamental
More informationProof Techniques Alphabets, Strings, and Languages. Foundations of Computer Science Theory
Proof Techniques Alphabets, Strings, and Languages Foundations of Computer Science Theory Proof By Case Enumeration Sometimes the most straightforward way to prove that a property holds for all elements
More informationCantor s Diagonal Argument for Different Levels of Infinity
JANUARY 2015 1 Cantor s Diagonal Argument for Different Levels of Infinity Michael J. Neely University of Southern California http://www-bcf.usc.edu/ mjneely Abstract These notes develop the classic Cantor
More informationLecture 1 Contracts. 1 A Mysterious Program : Principles of Imperative Computation (Spring 2018) Frank Pfenning
Lecture 1 Contracts 15-122: Principles of Imperative Computation (Spring 2018) Frank Pfenning In these notes we review contracts, which we use to collectively denote function contracts, loop invariants,
More informationLecture 1 Contracts : Principles of Imperative Computation (Fall 2018) Frank Pfenning
Lecture 1 Contracts 15-122: Principles of Imperative Computation (Fall 2018) Frank Pfenning In these notes we review contracts, which we use to collectively denote function contracts, loop invariants,
More informationPrograms with infinite loops: from primitive recursive predicates to the arithmetic hierarchy
Programs with infinite loops: from primitive recursive predicates to the arithmetic hierarchy ((quite) preliminary) Armando B. Matos September 11, 2014 Abstract Infinite time Turing machines have been
More informationTHE HALTING PROBLEM. Joshua Eckroth Chautauqua Nov
THE HALTING PROBLEM Joshua Eckroth Chautauqua Nov 10 2015 The year is 1928 Sliced bread is invented. Calvin Coolidge is President. David Hilbert challenged mathematicians to solve the Entscheidungsproblem:
More informationPolynomial SAT-Solver Algorithm Explanation
1 Polynomial SAT-Solver Algorithm Explanation by Matthias Mueller (a.k.a. Louis Coder) louis@louis-coder.com Explanation Version 1.0 - December 1, 2013 Abstract This document describes an algorithm that
More informationComplexity Theory. Compiled By : Hari Prasad Pokhrel Page 1 of 20. ioenotes.edu.np
Chapter 1: Introduction Introduction Purpose of the Theory of Computation: Develop formal mathematical models of computation that reflect real-world computers. Nowadays, the Theory of Computation can be
More informationCS21 Decidability and Tractability
CS21 Decidability and Tractability Lecture 9 January 26, 2018 Outline Turing Machines and variants multitape TMs nondeterministic TMs Church-Turing Thesis decidable, RE, co-re languages Deciding and Recognizing
More informationCS61A Lecture 38. Robert Huang UC Berkeley April 17, 2013
CS61A Lecture 38 Robert Huang UC Berkeley April 17, 2013 Announcements HW12 due Wednesday Scheme project, contest out Review: Program Generator A computer program is just a sequence of bits It is possible
More informationCS 3512, Spring Instructor: Doug Dunham. Textbook: James L. Hein, Discrete Structures, Logic, and Computability, 3rd Ed. Jones and Barlett, 2010
CS 3512, Spring 2011 Instructor: Doug Dunham Textbook: James L. Hein, Discrete Structures, Logic, and Computability, 3rd Ed. Jones and Barlett, 2010 Prerequisites: Calc I, CS2511 Rough course outline:
More informationCOMPUTABILITY THEORY AND RECURSIVELY ENUMERABLE SETS
COMPUTABILITY THEORY AND RECURSIVELY ENUMERABLE SETS JOSHUA LENERS Abstract. An algorithm is function from ω to ω defined by a finite set of instructions to transform a given input x to the desired output
More informationAdmin. How's the project coming? After these slides, read chapter 13 in your book. Quizzes will return
Recursion CS 1 Admin How's the project coming? After these slides, read chapter 13 in your book Yes that is out of order, but we can read it stand alone Quizzes will return Tuesday Nov 29 th see calendar
More informationGreedy Algorithms 1 {K(S) K(S) C} For large values of d, brute force search is not feasible because there are 2 d {1,..., d}.
Greedy Algorithms 1 Simple Knapsack Problem Greedy Algorithms form an important class of algorithmic techniques. We illustrate the idea by applying it to a simplified version of the Knapsack Problem. Informally,
More informationPropositional Logic Formal Syntax and Semantics. Computability and Logic
Propositional Logic Formal Syntax and Semantics Computability and Logic Syntax and Semantics Syntax: The study of how expressions are structured (think: grammar) Semantics: The study of the relationship
More informationAlgorithms and Data Structures
Algorithms and Data Structures Spring 2019 Alexis Maciel Department of Computer Science Clarkson University Copyright c 2019 Alexis Maciel ii Contents 1 Analysis of Algorithms 1 1.1 Introduction.................................
More informationCS 531: Notes II. January 31, 2014
CS 531: Notes II January 31, 2014 1 Acceptable and Decidable Languages Let P be a program and x be a string. What happens when we run P on on input x. There are there possibilities. i) The program runs
More informationVertex Cover Approximations
CS124 Lecture 20 Heuristics can be useful in practice, but sometimes we would like to have guarantees. Approximation algorithms give guarantees. It is worth keeping in mind that sometimes approximation
More informationCS221: Algorithms and Data Structures. Asymptotic Analysis. Alan J. Hu (Borrowing slides from Steve Wolfman)
CS221: Algorithms and Data Structures Asymptotic Analysis Alan J. Hu (Borrowing slides from Steve Wolfman) 1 Learning Goals By the end of this unit, you will be able to Define which program operations
More informationReading 8 : Recursion
CS/Math 40: Introduction to Discrete Mathematics Fall 015 Instructors: Beck Hasti, Gautam Prakriya Reading 8 : Recursion 8.1 Recursion Recursion in computer science and mathematics refers to the idea of
More informationCardinality of Sets MAT231. Fall Transition to Higher Mathematics. MAT231 (Transition to Higher Math) Cardinality of Sets Fall / 15
Cardinality of Sets MAT Transition to Higher Mathematics Fall 0 MAT (Transition to Higher Math) Cardinality of Sets Fall 0 / Outline Sets with Equal Cardinality Countable and Uncountable Sets MAT (Transition
More informationMidterm Exam 2B Answer key
Midterm Exam 2B Answer key 15110 Principles of Computing Fall 2015 April 6, 2015 Name: Andrew ID: Lab section: Instructions Answer each question neatly in the space provided. There are 6 questions totaling
More informationLog-Space. A log-space Turing Machine is comprised of two tapes: the input tape of size n which is cannot be written on, and the work tape of size.
CSE 431 Theory of Computation Scribes: Michelle Park and Julianne Brodhacker Lecture 18 May 29 Review of Log-Space Turing Machines: Log-Space A log-space Turing Machine is comprised of two tapes: the input
More informationMATH Iris Loeb.
MATH 134 http://www.math.canterbury.ac.nz/math134/09/su1/c Iris Loeb I.Loeb@math.canterbury.ac.nz Office Hours: Thur 10.00-11.00, Room 703 (MSCS Building) The Limits of Formal Logic We now turn our attention
More informationPractice Problems for the Final
ECE-250 Algorithms and Data Structures (Winter 2012) Practice Problems for the Final Disclaimer: Please do keep in mind that this problem set does not reflect the exact topics or the fractions of each
More informationLecture 1. 1 Notation
Lecture 1 (The material on mathematical logic is covered in the textbook starting with Chapter 5; however, for the first few lectures, I will be providing some required background topics and will not be
More informationAssignment: 7. Due: Language level: Allowed recursion:
Assignment: 7 Due: Language level: Allowed recursion: CS 135 Winter 2018 Graham, Nijjar Tuesday, March 13th, 2018 9:00pm Beginning Student with List Abbreviations Pure Structural and Structural Recursion
More informationγ(ɛ) (a, b) (a, d) (d, a) (a, b) (c, d) (d, d) (e, e) (e, a) (e, e) (a) Draw a picture of G.
MAD 3105 Spring 2006 Solutions for Review for Test 2 1. Define a graph G with V (G) = {a, b, c, d, e}, E(G) = {r, s, t, u, v, w, x, y, z} and γ, the function defining the edges, is given by the table ɛ
More informationReducibilities relations with applications to symbolic dynamics
Reducibilities relations with applications to symbolic dynamics Part I: Computability E. Jeandel LORIA (Nancy, France) E. Jeandel, CASD, Part I: Computability 1/1 Introduction What is computability? Why
More informationFormal Methods of Software Design, Eric Hehner, segment 24 page 1 out of 5
Formal Methods of Software Design, Eric Hehner, segment 24 page 1 out of 5 [talking head] This lecture we study theory design and implementation. Programmers have two roles to play here. In one role, they
More informationSolutions to In Class Problems Week 5, Wed.
Massachusetts Institute of Technology 6.042J/18.062J, Fall 05: Mathematics for Computer Science October 5 Prof. Albert R. Meyer and Prof. Ronitt Rubinfeld revised October 5, 2005, 1119 minutes Solutions
More informationDistributed Algorithms 6.046J, Spring, Nancy Lynch
Distributed Algorithms 6.046J, Spring, 205 Nancy Lynch What are Distributed Algorithms? Algorithms that run on networked processors, or on multiprocessors that share memory. They solve many kinds of problems:
More information/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Approximation algorithms Date: 11/27/18
601.433/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Approximation algorithms Date: 11/27/18 22.1 Introduction We spent the last two lectures proving that for certain problems, we can
More informationIntroduction to the Analysis of Algorithms. Algorithm
Introduction to the Analysis of Algorithms Based on the notes from David Fernandez-Baca Bryn Mawr College CS206 Intro to Data Structures Algorithm An algorithm is a strategy (well-defined computational
More informationCISC 4090 Theory of Computation
CISC 4090 Theory of Computation Turing machines Professor Daniel Leeds dleeds@fordham.edu JMH 332 Alan Turing (1912-1954) Father of Theoretical Computer Science Key figure in Artificial Intelligence Codebreaker
More information(Refer Slide Time: 0:19)
Theory of Computation. Professor somenath Biswas. Department of Computer Science & Engineering. Indian Institute of Technology, Kanpur. Lecture-15. Decision Problems for Regular Languages. (Refer Slide
More informationOperational semantics questions and answers
Operational semantics questions and answers COMP 105 30 January 2019 Contents Functions vs syntactic forms............ 1 Environments and their notation......... 1 Function environments...............
More informationCSE 417 Network Flows (pt 4) Min Cost Flows
CSE 417 Network Flows (pt 4) Min Cost Flows Reminders > HW6 is due Monday Review of last three lectures > Defined the maximum flow problem find the feasible flow of maximum value flow is feasible if it
More informationELEMENTARY NUMBER THEORY AND METHODS OF PROOF
CHAPTER 4 ELEMENTARY NUMBER THEORY AND METHODS OF PROOF Copyright Cengage Learning. All rights reserved. SECTION 4.2 Direct Proof and Counterexample II: Rational Numbers Copyright Cengage Learning. All
More informationEat (we provide) link. Eater. Goal: Eater(Self) == Self()
15-251: Great Theoretical Ideas Guru: Yinmeng Zhang Assignment 12 Due: December 6, 2005 1 Reading Comprehension (0 points) Read the accompanying handout containing Ken Thompson s Turing Award Lecture,
More informationCOT 3100 Spring 2010 Midterm 2
COT 3100 Spring 2010 Midterm 2 For the first two questions #1 and #2, do ONLY ONE of them. If you do both, only question #1 will be graded. 1. (20 pts) Give iterative and recursive algorithms for finding
More information