Spring 2015 Unit 2: Models of computation
What is an algorithm? So far... An inductively defined function Limitation Doesn t capture mutation of data
Imperative models of computation Computation = sequence of reads from and writes to memory Represented in a finite way Commonly asked questions: How much memory can a program access? How do you define a program s runtime? How do you prove correctness?
Two imperative models of computation Turing machines Structured programs
Two imperative models of computation Turing machines Structured programs Defining a language Syntax of programs Semantics of programs
Model (1): Turing machines q a 0... a i... a n
Model (1): Turing machines q a 0... a i... a n Initially: Input on the leftmost n cells of the tape. Head on leftmost cell. Transitions: Change state of head, write a symbol in the current cell, and optionally, move the head left or right. Acceptance: Accept input, reject input, or loop forever. Programs with string inputs and boolean outputs.
Detour: Notation For sets A 1,...,A k, A 1 A k represents the Cartesian product of the sets: A 1 A k = {(a 1,...,a k ):a i 2 A i } If all the A i s are the same set A, then we denote the product by A k. For any set A, A represents the set of finite words over symbols drawn from A. As a degenerate case, A always includes the empty word.
Turing machines q a 0... a i... a n Syntax: Finite set Q of control states; unique initial state q 0 ;accept state q acc,rejectstateq rej (we have q acc 6= q rej ) Finite set of input symbols; special blank symbol t (t /2 ) Finite set of tape symbols, with t2 and ; Transition function : Q! Q {LEFT, RIGHT, STAY } [See readings provided through Owlspace.]
Turing machine: State diagram start q 1 0!t, R x! R q 2 t!r q 5 0! x, R t!l 0! L x! L q 3 x! R t!r x! R t!r 0! x, R 0! R q rej q acc q 4 t!r x! R
Turing machines: Semantics Configuration: C =(q, w 1, a, w 2 ), where q 2 Q: control state w 1 2 : tape fragment to the left of the head a 2 : current symbol read by the head w 2 2 : tape fragment to the right of the head is the empty word. Initial configuration: C 0 =(q 0,, a, w) aw is a word over ; it stheinput. Accepting configuration: C =(q acc, w 1, a, w 2 ) Rejecting configuration: C =(q rej, w 1, a, w 2 )
Turing machines: Semantics Execution: Sequence of configurations C 0 C 1 C 2...C n,where C i+1 is obtained from C i by applying the transition function Input accepted the first time an accepting configuration C =(q acc, w 1, a, w 2 )isreached.inthiscase,thewordw 1 aw 2 is the output. Input rejected the first time a rejecting configuration C =(q rej, w 1, a, w 2 )isreached.inthiscase,thewordw 1 aw 2 is the output. Language: Set of input words accepted by the Turing machine
Turing machine: Example 1 start 1! 0, R q 0! 1, S t!1, S q acc
Questions What is the tape configuration, at the point of acceptance, on input 1?... on input 011?... on input 111?
Turing machine: Example 1 Computes Succ(x) = x + 1 Assuming the input x is written in reverse binary notation. q 1 1 1 1 0 1 t
Turing machine: Example 2 q 5 0! L x! L start q 1 0!t, R x! R q 2 t!r 0! x, R t!l q 3 x! R t!r x! R t!r 0! x, R 0! R q rej q acc q 4 t!r x! R
Questions What does the machine do on input 0?... on input 00?... on input 000?... on input 000000? Notation: 0 n stands for a word with n 0 s.
Turing machine: Example 2 Accepts the set {0 2n : n 0} Dynamics: 1 Sweep from left to right, crossing o every other 0. 2 If in stage 1 the tape contained a single 0, accept. 3 If in stage 1 the tape contained more than a single 0 and the number of 0 s was odd, reject. 4 Return the head to the left-hand end of the tape. 5 Go to stage 1.
Turing machines: complexity A Turing machine may not halt on every input. Thus, semi-algorithm rather than algorithm
Turing machines: complexity A Turing machine may not halt on every input. Thus, semi-algorithm rather than algorithm Runtime: number of steps as a function of the size n of input Assumes binary encoding of input Just reading the input takes n time units!
Variants of Turing machines Many variations: One-way tape Multiple tracks on the tape Multiple tapes Multiple heads Subroutines But... All of these models are equivalent to the version we studied A machine in any of these models can be compiled to an equivalent machine in any other
Example: multitape Turing machine Multitape Turing machines: Turing machines with k tapes Basically the same as ordinary (single-tape) Turing machines, except the transition function is of the form : Q k! Q k {LEFT, RIGHT } k.
Equivalence with single-tape Turing machines Show that multitape Turing machines are equivalent to single-tape Turing machines. In other words: For every multitape Turing machine MT,there is a (single-tape) Turing machine ST that accepts the same set of words. For every (single-tape) Turing machine ST, there is a multitape Turing machine MT that accepts the same set of words.
Equivalence with single-tape Turing machines Show that multitape Turing machines are equivalent to single-tape Turing machines. In other words: For every multitape Turing machine MT,there is a (single-tape) Turing machine ST that accepts the same set of words. For every (single-tape) Turing machine ST, there is a multitape Turing machine MT that accepts the same set of words. This is obvious, as every single-tape machine is also a multitape machine!
Simulation argument Consider any multitape Turing machine MT. Construct a single-tape Turing machine ST so that every execution of MT can be simulated by an execution of ST.
Simulation argument Consider any multitape Turing machine MT. Construct a single-tape Turing machine ST so that every execution of MT can be simulated by an execution of ST. Arrange the contents of k tapes on a single tape, sequentially and separated by a special delimiter. See details in readings.
Nondeterminism So far, one only move allowed at each point in execution (i.e., for fixed control state + current symbol) Nondeterminism: multiple moves permitted at a given point For example, (q, a) ={(q 1, a, LEFT ), (q 2, b, RIGHT )} Input accepted if some execution leads to q acc Many possible outputs start q 0 0! 1, R q acc 0! 0, R q rej
A nondeterministic algorithm 1 method Search(a: array<int>,value: int, 2 low: int, high: int) returns (index: int) 3 { 4 var index := Guess(); 5 if (a[index] == value) 6 return true; 7 else 8 return false; 9 } Can be encoded by a nondeterministic Turing machine Nondeterminism = power to guess and check Can t be implemented directly on a computer
Nondeterminism vs. determinism Any nondeterministic Turing machine can be compiled to a deterministic one:
Nondeterminism vs. determinism Any nondeterministic Turing machine can be compiled to a deterministic one: Explore the tree of possible computations Breadth-first rather than depth-first
Nondeterminism vs. determinism Any nondeterministic Turing machine can be compiled to a deterministic one: Explore the tree of possible computations Breadth-first rather than depth-first If Nondeterministic TM accepts along some branch, then Deterministic TM also accepts If Nondeterministic TM doesn t accept along any branch, then Depeterministic TM loops forever
Nondeterminism vs. determinism Any nondeterministic Turing machine can be compiled to a deterministic one: Explore the tree of possible computations Breadth-first rather than depth-first If Nondeterministic TM accepts along some branch, then Deterministic TM also accepts If Nondeterministic TM doesn t accept along any branch, then Depeterministic TM loops forever See more details in readings.
Nondeterminism vs. determinism But what about runtime? If a problem has an e cient (polynomial time) nondeterministic algorithm, does it necessarily have an e cient deterministic algorithm? If the answer is yes, all cryptography can be broken, e AI possible,... Biggest open question in theoretical computer science cient
Turing machines Pros: Mathematically clean As powerful as every known model of computation Cons: Abstracts away many of the details of modern machines No high-level data type See Turing machine simulator at http://morphett.info/turing/turing.html
Model (2): Structured imperative programs A simple but high-level programming language: 1 i := 0; 2 sum := 0; 3 while (i > n) { 4 i := i + 1; 5 sum := sum + i; 6 }
Model (2): Structured programs Language syntax (can be augmented with other constructs): Assume a set of variables Var, a set of constants Const, a set of arithmetic operators Func, a set of relational operators Pred e ::= f(e 1, e 2 ) x n where x 2 Var, n 2 Const, f 2 Func b ::= R(e 1, e 2 ) where R 2 Pred S ::= x := e S 1 ; S 2 skip Example: 1 i := 0; 2 sum := 0; 3 while (i > n) { 4 i := i + 1; 5 sum := sum + i; 6 } if b then S 1 else S 2 while b do S 1
Semantics What do expressions/statements mean?
Semantics What do expressions/statements mean? Give rules that define... What expressions evaluate to on given inputs How the program s state changes when a statement is executed.
Structured programs Pros: Cons: Resembles modern high-level languages Allows for complex data types, can be extended with additional language features Easy to compose Not as simple as Turing machines
The Church-Turing thesis You can convert a Turing machine into an equivalent structured program, and vice versa What does equivalent mean?
The Church-Turing thesis You can convert a Turing machine into an equivalent structured program, and vice versa What does equivalent mean? Theorem holds even if you add recursion; datatypes such as records, lists, and trees; and higher-order functions (lambdas)
The Church-Turing thesis You can convert a Turing machine into an equivalent structured program, and vice versa What does equivalent mean? Theorem holds even if you add recursion; datatypes such as records, lists, and trees; and higher-order functions (lambdas) In fact, just lambdas are enough to encode Turing machines Church-Turing thesis: Turing machines capture the notion of algorithmic computation
Other models of computation RAM machines PRAMs: Parallel RAMs Probabilistic Turing machines String rewrite systems Recursive functions Lambda-calculus...
Moral of the story An algorithm is a mathematical object Model of computation: Language for expressing algorithms Practical angles: Proving properties of algorithms and unsolvability of problems