CS152: Programming Languages. Lecture 7 Lambda Calculus. Dan Grossman Spring 2011

CS152: Programming Languages Lecture 7 Lambda Calculus Dan Grossman Spring 2011

Where we are Done: Syntax, semantics, and equivalence For a language with little more than loops and global variables Now: Didn t IMP leave some things out? In particular: scope, functions, and data structures (Not to mention threads, I/O, exceptions, strings,...) Time for a new model... Dan Grossman CS152 Spring 2011, Lecture 7 2

Data + Code Higher-order functions work well for scope and data structures Scope: not all memory available to all code let x = 1 let add3 y = let z = 2 in x + y + z let seven = add3 4 Data: Function closures store data. Example: Association list let empty = (fun k -> raise Empty) let cons k v lst = (fun k -> if k =k then v else lst k ) let lookup k lst = lst k (Later: Objects do both too) Dan Grossman CS152 Spring 2011, Lecture 7 3

Adding data structures Extending IMP with data structures isn t too hard: e ::= c x e + e e e (e, e) e.1 e.2 v ::= c (v, v) H ::= H, x v H ; e c all old rules plus: H ; e 1 v 1 H ; e 2 v 2 H ; (e 1, e 2 ) (v 1, v 2 ) H ; e (v 1, v 2 ) H ; e.1 v 1 H ; e (v 1, v 2 ) H ; e.2 v 2 Notice: We allow pairs of values, not just pairs of integers We now have stuck programs (e.g., c.1) What would C++ do? Scheme? ML? Java? Perl? Division also causes stuckness Dan Grossman CS152 Spring 2011, Lecture 7 4

What about functions But adding functions (or objects) does not work well: e ::=... fun x -> s v ::=... fun x -> s s ::=... e(e) H ; e c H ; s H ; s Additions: H ; fun x -> s fun x -> s H ; e 1 fun x -> s H ; e 2 v H ; e 1 (e 2 ) H ; x := v; s Does this match the semantics we want for function calls? Dan Grossman CS152 Spring 2011, Lecture 7 5

What about functions But adding functions (or objects) does not work well: e ::=... fun x -> s v ::=... fun x -> s s ::=... e(e) H ; fun x -> s fun x -> s H ; e 1 fun x -> s H ; e 2 v H ; e 1 (e 2 ) H ; x := v; s NO: Consider x := 1; (fun x -> y := x)(2); ans := x. Scope matters; variable name doesn t. That is: Local variables should be local Choice of local-variable names should have only local ramifications Dan Grossman CS152 Spring 2011, Lecture 7 6

Another try H ; e 1 fun x -> s H ; e 2 v y fresh H ; e 1 (e 2 ) H ; y := x; x := v; s; x := y Dan Grossman CS152 Spring 2011, Lecture 7 7

Another try H ; e 1 fun x -> s H ; e 2 v y fresh H ; e 1 (e 2 ) H ; y := x; x := v; s; x := y fresh isn t very IMP-like but okay (think malloc) Dan Grossman CS152 Spring 2011, Lecture 7 7

Another try H ; e 1 fun x -> s H ; e 2 v y fresh H ; e 1 (e 2 ) H ; y := x; x := v; s; x := y fresh isn t very IMP-like but okay (think malloc) not a good match to how functions are implemented Dan Grossman CS152 Spring 2011, Lecture 7 7

Another try H ; e 1 fun x -> s H ; e 2 v y fresh H ; e 1 (e 2 ) H ; y := x; x := v; s; x := y fresh isn t very IMP-like but okay (think malloc) not a good match to how functions are implemented yuck: the way we want to think about something as fundamental as a call? Dan Grossman CS152 Spring 2011, Lecture 7 7

Another try H ; e 1 fun x -> s H ; e 2 v y fresh H ; e 1 (e 2 ) H ; y := x; x := v; s; x := y fresh isn t very IMP-like but okay (think malloc) not a good match to how functions are implemented yuck: the way we want to think about something as fundamental as a call? NO: wrong model for most functional and OO languages (Even wrong for C if s calls another function that accesses the global variable x) Dan Grossman CS152 Spring 2011, Lecture 7 7

The wrong model H ; e 1 fun x -> s H ; e 2 v y fresh H ; e 1 (e 2 ) H ; y := x; x := v; s; x := y f 1 := (fun x -> f 2 := (fun z -> ans := x + z)); f 1 (2); x := 3; f 2 (4) Should set ans to 6: f 1 (2) should assign to f 2 a function that adds 2 to its argument and stores result in ans Actually sets ans to 7: f 2 (2) assigns to f 2 a function that adds the current value of x to its argument Dan Grossman CS152 Spring 2011, Lecture 7 8

Punch line Cannot properly model local scope via a global heap of integers. Functions are not syntactic sugar for assignments to globals So let s build a new model that focuses on this essential concept (can add back IMP features later) Or just borrow a model from Alonzo Church And drop mutation, conditionals, integers (!), and loops (!) Dan Grossman CS152 Spring 2011, Lecture 7 9

The Lambda Calculus The Lambda Calculus: e ::= λx. e x e e v ::= λx. e You apply a function by substituting the argument for the bound variable (There is an equivalent environment definition not unlike heap-copying; see future homework) Dan Grossman CS152 Spring 2011, Lecture 7 10

Example Substitutions e ::= λx. e x e e v ::= λx. e Substitution is the key operation we were missing: (λx. x)(λy. y) (λy. y) (λx. λy. y x)(λz. z) (λy. y λz. z) (λx. x x)(λx. x x) (λx. x x)(λx. x x) After substitution, the bound variable is gone, so its name was irrelevant. (Good!) Dan Grossman CS152 Spring 2011, Lecture 7 11

A Programming Language Given substitution (e 1 [e 2 /x] = e 3 ), we can give a semantics: e e e[v/x] = e (λx. e) v e e 1 e 1 e 1 e 2 e 1 e 2 e 2 e 2 v e 2 v e 2 A small-step, call-by-value (CBV), left-to-right semantics Terminates when the whole program is some λx. e But (also) gets stuck when there s a free variable at top-level Won t cheat like we did with H(x) in IMP because scope is what we re interested in This is the heart of functional languages like Caml But real implementations don t substitute; they do something equivalent Dan Grossman CS152 Spring 2011, Lecture 7 12

Roadmap Motivation for a new model (done) CBV lambda calculus using substitution (done) Notes on concrete syntax Simple Lambda encodings (it s Turing complete!) Other reduction strategies Defining substitution Dan Grossman CS152 Spring 2011, Lecture 7 13

Concrete-Syntax Notes We (and Caml) resolve concrete-syntax ambiguities as follows: 1. λx. e 1 e 2 is (λx. e 1 e 2 ), not (λx. e 1 ) e 2 2. e 1 e 2 e 3 is (e 1 e 2 ) e 3, not e 1 (e 2 e 3 ) Convince yourself application is not associative More generally: 1. Function bodies extend to an unmatched right parenthesis Example: (λx. y(λz. z)w)q 2. Application associates to the left Example: e 1 e 2 e 3 e 4 is (((e 1 e 2 ) e 3 ) e 4 ) Like in IMP, assume we really have ASTs (with non-leaves labeled λ or application ) Rules may seem strange at first, but it s the most convenient concrete syntax Based on 70 years experience Dan Grossman CS152 Spring 2011, Lecture 7 14

Lambda Encodings Fairly crazy: we left out constants, conditionals, primitives, and data structures In fact, we re Turing complete and can encode whatever we need (just like assembly language can) Motivation for encodings: Fun and mind-expanding Shows we aren t oversimplifying the model ( numbers are syntactic sugar ) Can show languages are too expressive (e.g., unlimited C++ template instantiation) Encodings are also just (re)definition via translation Dan Grossman CS152 Spring 2011, Lecture 7 15

Encoding booleans The Boolean ADT There are two booleans and one conditional expression. The conditional takes 3 arguments (e.g., via currying). If the first is one boolean it evaluates to the second. If it s the other boolean it evaluates to the third. Dan Grossman CS152 Spring 2011, Lecture 7 16

Encoding booleans The Boolean ADT There are two booleans and one conditional expression. The conditional takes 3 arguments (e.g., via currying). If the first is one boolean it evaluates to the second. If it s the other boolean it evaluates to the third. Any set of three expressions meeting this specification is a proper encoding of booleans. Dan Grossman CS152 Spring 2011, Lecture 7 16

Encoding booleans The Boolean ADT There are two booleans and one conditional expression. The conditional takes 3 arguments (e.g., via currying). If the first is one boolean it evaluates to the second. If it s the other boolean it evaluates to the third. Any set of three expressions meeting this specification is a proper encoding of booleans. Here is one of an infinite number of encodings: true false if λx. λy. x λx. λy. y λb. λt. λf. b t f Example: if true v 1 v 2 v 1 Dan Grossman CS152 Spring 2011, Lecture 7 16

Evaluation Order Matters Careful: With CBV we need to thunk... diverges, but if true (λx. x) ((λx. x x)(λx. x x)) }{{} an infinite loop doesn t. if true (λx. x) (λz. ((λx. x x)(λx. x x)) z)) }{{} a value that when called diverges Dan Grossman CS152 Spring 2011, Lecture 7 17

Encoding Pairs The pair ADT : There is 1 constructor (taking 2 arguments) and 2 selectors 1st selector returns the 1st arg passed to the constructor 2nd selector returns the 2nd arg passed to the constructor Dan Grossman CS152 Spring 2011, Lecture 7 18

Encoding Pairs The pair ADT : There is 1 constructor (taking 2 arguments) and 2 selectors 1st selector returns the 1st arg passed to the constructor 2nd selector returns the 2nd arg passed to the constructor Example: mkpair λx. λy. λz. z x y fst λp. p(λx. λy. x) snd λp. p(λx. λy. y) snd ( fst ( mkpair ( mkpair v 1 v 2 ) v 3 )) v 2 Dan Grossman CS152 Spring 2011, Lecture 7 18

Reusing Lambdas Is it weird that the encodings of Booleans and pairs both used λx. λy. x and λx. λy. y for different purposes? Is it weird that the same bit-pattern in binary code can represent an int, a float, an instruction, or a pointer? Von Neumann: Bits can represent (all) code and data Church (?): Lambdas can represent (all) code and data Beware the Turing tarpit Dan Grossman CS152 Spring 2011, Lecture 7 19

Encoding Lists Rather than start from scratch, notice that booleans and pairs are enough to encode lists: Empty list is mkpair false false Non-empty list is λh. λt. mkpair true ( mkpair h t) Is-empty is... Head is... Tail is... Note: Not too far from how lists are implemented Taking tail ( tail empty ) will produce some lambda Just like, without page-protection hardware, null->tail->tail would produce some bit-pattern Dan Grossman CS152 Spring 2011, Lecture 7 20

Encoding Recursion Some programs diverge, but can we write useful loops? Yes! Dan Grossman CS152 Spring 2011, Lecture 7 21

Encoding Recursion Some programs diverge, but can we write useful loops? Yes! Write a function that takes an f and calls it in place of recursion Example (in enriched language): λf. λx. if (x = 0) then 1 else (x f(x 1)) Dan Grossman CS152 Spring 2011, Lecture 7 21

Encoding Recursion Some programs diverge, but can we write useful loops? Yes! Write a function that takes an f and calls it in place of recursion Example (in enriched language): λf. λx. if (x = 0) then 1 else (x f(x 1)) Then apply fix to it to get a recursive function: fix λf. λx. if (x = 0) then 1 else (x f(x 1)) Dan Grossman CS152 Spring 2011, Lecture 7 21

Encoding Recursion Some programs diverge, but can we write useful loops? Yes! Write a function that takes an f and calls it in place of recursion Example (in enriched language): λf. λx. if (x = 0) then 1 else (x f(x 1)) Then apply fix to it to get a recursive function: fix λf. λx. if (x = 0) then 1 else (x f(x 1)) fix λf. e reduces to something roughly equivalent to e[( fix λf. e)/f], which is unrolling the recursion once (and further unrollings will happen as necessary). Dan Grossman CS152 Spring 2011, Lecture 7 21

Encoding Recursion Some programs diverge, but can we write useful loops? Yes! Write a function that takes an f and calls it in place of recursion Example (in enriched language): λf. λx. if (x = 0) then 1 else (x f(x 1)) Then apply fix to it to get a recursive function: fix λf. λx. if (x = 0) then 1 else (x f(x 1)) fix λf. e reduces to something roughly equivalent to e[( fix λf. e)/f], which is unrolling the recursion once (and further unrollings will happen as necessary). The details, especially for CBV, are icky; the point is it s possible and you define fix only once Not on exam: fix λg. (λx. g (λy. x x y))(λx. g (λy. x x y)) Dan Grossman CS152 Spring 2011, Lecture 7 21

Encoding Arithmetic Over Natural Numbers How about arithmetic? Focus on non-negative numbers, addition, is-zero, etc. Dan Grossman CS152 Spring 2011, Lecture 7 22

Encoding Arithmetic Over Natural Numbers How about arithmetic? Focus on non-negative numbers, addition, is-zero, etc. How I would do it based on what we have so far: Lists of booleans for binary numbers Zero can be the empty list Use fix to implement adders, etc. Like in hardware except fixed-width avoids recursion Dan Grossman CS152 Spring 2011, Lecture 7 22

Encoding Arithmetic Over Natural Numbers How about arithmetic? Focus on non-negative numbers, addition, is-zero, etc. How I would do it based on what we have so far: Lists of booleans for binary numbers Zero can be the empty list Use fix to implement adders, etc. Like in hardware except fixed-width avoids recursion Or just use list length for a unary encoding Addition is list append Dan Grossman CS152 Spring 2011, Lecture 7 22

Encoding Arithmetic Over Natural Numbers How about arithmetic? Focus on non-negative numbers, addition, is-zero, etc. How I would do it based on what we have so far: Lists of booleans for binary numbers Zero can be the empty list Use fix to implement adders, etc. Like in hardware except fixed-width avoids recursion Or just use list length for a unary encoding Addition is list append But instead everybody always teaching Church numerals. Why? Tradition? Some sense of professional obligation? Better reason: You don t need fix: Basic arithmetic is often encodable in languages where all programs terminate In any case, we ll show some basics just for fun Dan Grossman CS152 Spring 2011, Lecture 7 22

Church Numerals 0 λs. λz. z 1 λs. λz. s z 2 λs. λz. s (s z) 3 λs. λz. s (s (s z))... Numbers encoded with two-argument functions The number i composes the first argument i times, starting with the second argument z stands for zero and s for successor (think unary) The trick is implementing arithmetic by cleverly passing the right arguments for s and z Dan Grossman CS152 Spring 2011, Lecture 7 23

Church Numerals 0 λs. λz. z 1 λs. λz. s z 2 λs. λz. s (s z) 3 λs. λz. s (s (s z)) successor λn. λs. λz. s (n s z) successor: take a number and return a number that (when called) applies s one more time Dan Grossman CS152 Spring 2011, Lecture 7 24

Church Numerals 0 λs. λz. z 1 λs. λz. s z 2 λs. λz. s (s z) 3 λs. λz. s (s (s z)) successor λn. λs. λz. s (n s z) plus λn. λm. λs. λz. n s (m s z) plus: take two numbers and return a number that uses one number as the zero argument for the other Dan Grossman CS152 Spring 2011, Lecture 7 25

Church Numerals 0 λs. λz. z 1 λs. λz. s z 2 λs. λz. s (s z) 3 λs. λz. s (s (s z)) successor λn. λs. λz. s (n s z) plus λn. λm. λs. λz. n s (m s z) times λn. λm. m ( plus n) zero times: take two numbers m and n and pass to m a function that adds n to its argument (so this will happen m times) and zero (where to start the m iterations of addition) Dan Grossman CS152 Spring 2011, Lecture 7 26

Church Numerals 0 λs. λz. z 1 λs. λz. s z 2 λs. λz. s (s z) 3 λs. λz. s (s (s z)) successor λn. λs. λz. s (n s z) plus λn. λm. λs. λz. n s (m s z) times λn. λm. m ( plus n) zero iszero λn. (λx. false ) true iszero: an easy one, see how the two arguments will lead to the correct answer Dan Grossman CS152 Spring 2011, Lecture 7 27

Church Numerals 0 λs. λz. z 1 λs. λz. s z 2 λs. λz. s (s z) 3 λs. λz. s (s (s z)) successor λn. λs. λz. s (n s z) plus λn. λm. λs. λz. n s (m s z) times λn. λm. m ( plus n) zero iszero λn. (λx. false ) true predecessor minus isequal (with 0 sticky) the hard one; see Wikipedia similar to times with pred instead of plus subtract and test for zero Dan Grossman CS152 Spring 2011, Lecture 7 28

Roadmap Motivation for a new model (done) CBV lambda calculus using substitution (done) Notes on concrete syntax (done) Simple Lambda encodings (it s Turing complete!) (done) Other reduction strategies Defining substitution Then start type systems Later take a break from types to consider first-class continuations and related topics Dan Grossman CS152 Spring 2011, Lecture 7 29