COMP80 Programming Languages Slides Courtesy of Prof. Sam Guyer Lambda Calculus Formal system with three parts Notation for functions Proof system for equations Calculation rules called reduction Idea: What is the simplest possible programming language that is still capable of expressing any program? Tufts University Computer Science 2 History Alonzo Church, 1903-1995 Created -calculus in the 30 s Study of functions Definition, application Question: what things are computable? -calculus is equivalent to Turing Machines Important part of CS history and current theory Influenced design of Lisp, ML, Haskell, Big ideas Lambda calculus Simplest possible programming language Almost nothing is built in Provides a model for computability Question: what is it possible to compute? Examples: Easy: what is nth fibonacci number? Hard: what are the prime factors of a number? Undecidable: do two BNFs produce the same language? Turns out: Decidable (computable) functions are those that can be expressed using lambda calculus Tufts University Computer Science 3 Tufts University Computer Science 4 Expressions and Functions Definitions Expressions x + y x + 2*y + z -calculus is a formal notation for defining functions expressions in this notation are called -expressions every -expression denotes a function Functions x. (x + y) z. (x + 2*y + z) Application (x. (x + y)) 3 (z. (x + 2*y + z)) 5 3 + y x + 2*y + 5 A -expression consists of 3 kinds of terms: Variables: x, y, z, we use V for arbitrary variables Abstractions: V.E Where V is some variable and E is another -term Applications: E 1 E 2 Where E 1 and E 2 are -terms Tufts University Computer Science 5 Tufts University Computer Science 6 1
Formal Syntax Formal Syntax in BNF Term ::= Variable Variable. Term Term Term ( Term ) Variable ::= x y z Or, more compactly: E ::= V V.E E E (E) V ::= x y z Application is juxtaposition (no operator) Application has higher precedence than Application is left-associative Tufts University Computer Science 7 Highly ambiguous Again: E ::= V V.E E E (E) V ::= x y z Application is juxtaposition (no operator) Application has higher precedence than Application is left-associative Examples x. x y z x.x y.y f. f(y.y) g(z) f(g)(h x) x. (x(y))(z) x. (x(y.y)) f. f(y.y) (g(z)) f(g)(h(x)) Tufts University Computer Science 8 Abstractions V.E is an abstraction V is a bound variable over the body E An abstraction represents a function V is the head and E is the body. For example the abstraction: I = x.x is the identity function Applications E 1 E 2 is an application Apply the function E 1 to the argument E 2 -calculus only has single-argument functions For example, given the identity function, I = x.x some applications using it (x.x)1 1 (x.x)a a (x.x)(y.y) (y.y) Tufts University Computer Science 9 Tufts University Computer Science 10 Reduction Basic computation rule is -reduction (x. e 1 ) e 2 [e 2 /x]e 1 Reduction Apply reduction to any subexpression Repeat until no more reductions possible Result is called normal form Confluence Church-Rosser theorem Final result (if there is one) is uniquely determined Two functions compute the same thing iff they reduce to the same normal form Tufts University Computer Science 11 Free and Bound Variables Bound variable is placeholder Variable x is bound in x.(x+y) Function x.(x+y) is same function as z.(z+y) Compare x+y dx = z+y dz x P(x) = z P(z) Name of free (=unbound) variable does matter Variable y is free in x.(x+y) Function x.(x+y) is not same as x.(x+z) Occurrences y is free and bound in x. ((y. y+2) x) + y Tufts University Computer Science 12 2
Rename Bound Variables Function application (f. x. f (f x)) (y. y+x) Substitute blindly x. [(y. y+x) ((y. y+x) x)] = x. x+x+x Rename bound variables (f. z. f (f z)) (y. y+x) = z. [(y. y+x) ((y. y+x) z))] = z. z+x+x Easy rule: always rename variables to be distinct Substitution [N/x]E means replace x with N in E Variables [N/x]x [N/x]y Application N y [N/x](M 1 M 2 ) ([N/x]M 1 ) ([N/x]M 2 ) Abstraction [N/x] x.m x.m [N/x] y.m y.[n/x]m where y not free in N Tufts University Computer Science 13 Tufts University Computer Science 14 Conversion/reduction rules -conversion x.e y.[y/x]e where y not free in E Rename bound variables to avoid naming conflicts -reduction (x.m)n [N/x]M Function application defined by substitution -conversion v.ev E where v is not free in E Remove redundant abstractions Proof: (x. f x) y = f y therefore x. f x = f Higher-Order Functions Given function f, return function f f f. x. f (f x) How does this work? (f. x. f (f x)) (y. y+1) = x. (y. y+1) ((y. y+1) x) = x. (y. y+1) (x+1) = x. (x+1)+1 Tufts University Computer Science 15 Tufts University Computer Science 16 Programming -calculus -calculus is Turing complete What does that mean? Equivalent to any other complete programming language How to make Booleans and conditional functions numerals and arithmetic functions data structures, such as ordered pairs, lists, etc. recursion For convenience x.y.m is written xy.m cuts down on number of s Church Booleans We define Booleans and logical operators in the -calculus as functions: True = t. f. t False = t. f. f And = x. y. x y (t.f. f) = x y. x y False Or = x.y. x (t.f. t) y = x y. x True y Neg = x. x (u.v. v) (a.b. a) = x. x False True Example: NEG True = (x.x(uv.v)(ab.a))(tf.t) (t.f.t)(uv.v)(ab.a) (f.(uv.v)) (ab.a) (uv.v) = False Tufts University Computer Science 17 Tufts University Computer Science 18 3
Church Booleans: If statements Given True = tf.t False = tf.f Define a conditional test function if C then X else Y If C X Y If = cxy.cxy If True X Y True X Y X If False X Y False X Y Y Note If = c.x.(y.(cx)y) c.(x.cx) c.c = I by -conversion Booleans are if functions Church Numerals The natural numbers may be defined using zero and the successor function: 0, 1=succ(0), 2=succ(succ(0)),, etc. In the -calculus, we only have functions Define the natural numbers as functions: 0 = fx.x What else has this definition? 1 = fx.f(x) 2 = fx.f(f(x)) 3 = fx.f(f(f(x)))) n = fx.f n (x) Idea: we represent a number as a lambda expression that applies some function (doesn t matter what function) that number of times. Tufts University Computer Science 19 Tufts University Computer Science 20 Successor function So how do we write a successor function? S = n.fx.f (n f x) Let s test it on zero = gy.y S 0 = (n.fx.f (n f x)) (gy.y) fx.f((gy.y) f x) fx.f((y.y) x) fx.f(x) 1 beta reduction eta conversion beta reduction Note that yx.y(x) = fx.f(z) by -conversion bound variables names are dummy variables More Arithmetic: PLUS := m n f x. n f (m f x) Interpretation: m, n are functions that apply f n times To add them together, start with m, and use the function n to apply f n more times Lists: CONS := f. s. b. b f s CAR := p. p TRUE CDR := p. p FALSE NIL := x.true Tufts University Computer Science 21 Tufts University Computer Science 22 Recursive Functions How to represent recursive functions? let Fact(n) = if (n=0) then 1 else n*fact(n-1) in Fact(5) Write this as -calculus expression P = (Fact. Fact(5)) (n. If (Eq n 0) 1 (Mult n (Fact (Sub n 1)))) assuming definitions for Eq, Mult, Sub What s the problem? Fact is a free variable in the second part of P Cannot use a function before it is defined Tufts University Computer Science 23 Recursive Functions Back up & focus on the recursive definition Fact = n. if (n=0) then 1 else n * Fact(n-1) Abstract Fact to make non-recursive generator G( f ) = n. if (n=0) then 1 else n * f(n-1) performs 1 step of factorial computation, then calls f G = f. n. if (n=0) then 1 else n * f(n-1) How do we compute more steps? Two steps: G(G(f)) Three steps: G(G(G(f))) Actually, we want f = G Intuitively: call G recursively as many times as it takes Tufts University Computer Science 24 4
Fixed point function Called a fixpoint Keep calling G recursively until stops When will that happen? G( 0 ) = n. if (n=0) then 1 else n * G(n-1) How do we make this work in -calculus? Idea: a function that computes the fixpoint of other functions Called a combinator Example: Y combinator Y = f. (x. f(x x)) (x. f(x x)) Fixed point function Y combinator in -calculus: Y = f. (x. f(x x)) (x. f(x x)) Has special property that Y( G ) = G( Y(G) ) = G ( G ( Y (G) ) ) = G ( G ( G ( G (... )))) Uses self-application in (x x) Y = f. (x. f (x x)) (y. f(y y)) = f. f ( (y. f(y y)) (y. f(y y)) ) = f. f ( (y. f(y y)) (y. f(y y)) ) = f. f ( f ( (y. f(y y)) (y. f(y y)) ) We will see self-application again when we talk about objects Tufts University Computer Science 25 Tufts University Computer Science 26 Reduction Order Reduction strategy An way of defining which reduction to perform Normal order: always perform left-most -reduction Applicative order: reduce arguments before application Confluence All reduction strategies that return a normal form return the same normal form but some strategies may not terminate, while others do Cost Different reduction strategies can involve different number of reduction steps What is a functional language? Functional languages have Functions as first-class values A data type is first-class if it can be used anywhere: passed to and returned from functions, assigned to variables May also have imperative constructs Examples: Lisp, Scheme, ML, Erlang Pure functional languages Have no implicit side effects or other imperative features Example: Miranda, Haskell Tufts University Computer Science 27 Tufts University Computer Science 28 Functional languages Two big ideas: Functions as data A program can create and manipulate functions on the fly Higher-order functions Twice and compose Currying: customize a function for one input No side-effects Given the same inputs, a function always returns the same result A function has no effects other than producing the return value Variables don t change value there s no store operation We can reason mathematically about functional programs Goal: Translate a program into a mathematic expression Get rid of syntactic details Give a meaning that is completely unambiguous Often: generate a functional program (e.g. in lambda calculus) How is this useful? Systematic conversion into formal notation We can reason about the correctness In practice Most programming languages have informal semantics Example: A break statement terminates the nearest enclosing loop or switch statement Tufts University Computer Science 29 Tufts University Computer Science 30 5
Example Syntax B ::= 0 1 N ::= B N B E ::= N E + E value function : E number [[ 0 ]] = 0 [[ 1 ]] = 1 [[ N B ]] = 2 * [[ N ]] + [[ B ]] [[ E 1 + E 2 ]] = [[ E 1 ]] + [[ E 2 ]] Key ideas: Gives a meaning to the syntax Eliminates syntactic details Example: 001 = 01 = 1 Tufts University Computer Science 31 Denotational semantics Idea: Translate a program into a function More mathematical, more precise Program s meaning: Translate each construct into a function from inputs to outputs Compose functions for a program into a large function that computes the whole thing Roughly like compilation Translation to assembly code => for execution We will compile down to lambda calculus => for reasoning Tufts University Computer Science 32 Translation into functions Translating a program into a function How do we do that? Translate each primitive construct into a function Define rules to assemble these functions What kinds of constructs? Assignment: x = y + 5; Conditional: if (c < 10) z = 20; Loops: while (z!= 0) z--; How to represent these things as functions? Example: what does assignment do? Updates a location in memory (whose name is x ) Tufts University Computer Science 33 Statements as functions Statement: x = 5; Meaning: Update the memory location called x with value 5 How do I represent x=5 as a function? Mapping from one program state to another What is a program state? Typically: a mapping from variables to values Example: s = { (x=0), (y=10), (z=999) } Key: Each function produces a new state never modify a state Now we re functional Tufts University Computer Science 34 Example Statement x = 5; : [[ x=5 ]]: state state Start state: s = { (x=0), (y=10), (z=999) } Next state: this is a function t = [[ x=5 ]] (s) = { (x=5), (y=10), (z=999) } Voila: no store operation, no side-effects Basic principle of denotational semantics Compositionality The meaning of a compound program must be defined from the meanings of its parts (not the syntax of its parts). Syntax guides translation to functions Tells us how to compose the functions of the parts Example { P; Q; } composition of two functions, state state Q : state state and P : state state Complete function: Q ( P ( start state ) ) Tufts University Computer Science 35 Tufts University Computer Science 36 6
of Imperative Programs Syntax P ::= x := E P; P if E then P else P while E do P : Program (State State) State = Variable Value = v. (give me the value of v) of Assignment Assignment [[ x := E ]] : state state change state with x updated to new value [[ x := E ]]s = s where s = v. IF ( v=x ) ( [[ E ]]s )( s(v) ) s : variable value state = (variable value) s is identical to s except for value of variable x Tufts University Computer Science 37 Tufts University Computer Science 38 Expressions More straightforward [[ E ]]s : [[ E + E ]]s =? = [[ E ]]s + [[ E ]]s [[ E / E ]]s =? = [[ E ]]s / [[ E ]]s [[ (E) ]]s =? = [[ E ]]s [[ v ]]s =? = s(v) of Sequences Sequence [[ P 1 ; P 2 ]] : state state Perform changes from P 1, then perform changes in P 2 [[ P 1 ; P 2 ]] s = [[ P 2 ]] ( [[ P 1 ]] s ) [[ P 1 ; P 2 ]] = s.[[ P 2 ]] ( [[ P 1 ]] s ) [[ P 1 ; P 2 ]] = [[ P 2 ]] [[ P 1 ]] Where is function composition F G = s.f(g(s)) Tufts University Computer Science 39 Tufts University Computer Science 40 of Conditional Conditional [[ if B then P else Q ]] : state state test B in input state, then perform either P or Q on state [[ if B then P else Q ]] s = = IF ( [[ B ]]s ) ( [[ P ]]s ) ( [[ Q ]]s ) Simplification: assume B does not have side effects What would we have to do if B had side effects? Tufts University Computer Science 41 of Iteration Iteration [[ while B do P ]] : state state test B in input state, if true do P, then while B do P Two parts: [[ B ]] the evaluation of expression B [[ P ]] the translation of the body P What do iterations look like? One: if [[ B ]]s then [[ P ]]s else s Two: if [[ B ]]s then let s = [[ P ]]s in (if [[ B ]]s then [[ P ]]s else s ) else s Tufts University Computer Science 42 7
of Iteration Generalize: [[ while B do P ]] = the function f such that f(s) = IF ( [[ B ]]s ) ( f( [[ P ]]s ) ) ( s ) What does this look like? I want X, such that X(P) = f(x(p)) = f(f(x(p))) Yep, it s defined by the Y combinator This is what we expect, right? Iteration (e.g., while) becomes recursion Recursion is defined as a fixpoint on the body of the loop How does this help me? Problem: Real languages have many constructs They can be combined in wacky ways Need a clear understanding of what they mean Examples: x = i++ + ++i; a = b = c = 0; while (*p++ = *q++) ; Tufts University Computer Science 43 Tufts University Computer Science 44 Example Statement: x = 5; y = f(10) + x; What if f modifies x? int f(int a) { x = a; return a; } What about: y = x + f(10); How can I explain this? [[ f ]]s produces a new state s [[ f(10) + x ]] = let s = [[ t = f(10) ]]s in [[ t + x ]]s Tufts University Computer Science 45 8