Where we are SEMANTIC ANALYSIS. ! Program is lexically well- formed: ! Program is syntac,cally well- formed:

Transcription

1 SEMANTIC ANALYSIS Where we are! Program is lexically well- formed: o Iden:fiers have valid names. o Strings are properly terminated.! Program is syntac,cally well- formed: o Class declara:ons have the correct structure.! Expressions are syntac:cally valid.! Does this mean that the program is legal? 1

2 2

3 Seman:c Analysis! Seman:c Analysis Ensure that the program has a well- defined meaning. o Verify proper:es of the program that aren't caught during the earlier phases: o Variables are declared before they're used. o Expressions have the right types. o Arrays can only be instan:ated with NewArray. o Classes don't inherit from nonexistent base classes! Once we finish seman:c analysis, we know that the user's input program is legal. Challenge! Reject the largest number of incorrect programs.! Accept the largest number of correct programs. 3

4 Validity vs Correctness 4

5 Seman:c Analysis! Gather useful informa:on about program for later phases: o Determine what variables are meant by each iden:fier. o Build an internal representa:on of inheritance hierarchies. o Count how many variables are in scope at each point. 5

6 ! Using CFGs: Limits of CFG o How would you prevent duplicate class defini:ons? o How would you differen:ate variables of one type from variables of another type? o How would you ensure classes implement all interface methods?! For most programming languages, these are provably impossible. o Use the pumping lemma for context- free languages CFGs do not suffice! Let y: String abc in y + 3! Let y: Int in x + 3! An example property that is not context free. 6

7 Implemen:ng Seman:c Analysis! A6ribute Grammars o Approach suggested in the Compilers book.! Recursive AST Walk o Construct the AST, then use recursion to explore the tree. SCOPE, AGAIN 7

8 ! The same name in a program may refer to fundamentally different things:! This is perfectly legal Java code: 8

9 Symbol Table! A symbol table is a mapping from a name to the thing that name refers to.! As we run our seman:c analysis, con:nuously update the symbol table with informa:on about what is in scope.! Ques:ons: o What does this look like in prac:ce? o What opera:ons need to be defined on it? o How do we implement it? 9

10 10

11 11

12 12

13 Symbol Table! Typically implemented as a stack of maps. o Each map corresponds to a par:cular scope.! Stack allows for easy enter and exit opera:ons.! Symbol table opera:ons are! Push scope: Enter a new scope. Pop scope: Leave a scope, discarding all declara:ons in it.! Insert symbol: Add a new entry to the current scope.! Lookup symbol: Find what a name corresponds to. Using Symbol Table! To process a por:on of the program that creates a scope (block statements, func:on calls, classes, etc.)! Enter a new scope.! Add all variable declara:ons to the symbol table. Process the body of the block/func:on/class.! Exit the scope. 13

14 14

15 Spagheh Stack! Treat the symbol table as a linked structure of scopes.! Each scope stores a pointer to its parents, but not vice- versa.! From any point in the program, symbol table appears to be a stack.! This is called a spaghej stack. 15

16 16

17 TYPES What is a type! What is a type? The no:on varies from language to language! Consensus A set of values A set of opera:ons on those values! Classes are one instan:a:on of the modern no:on of type 17

18 Types! Certain opera:ons are legal for values of each type o It doesn t make sense to add a func:on pointer and an integer in C o It does make sense to add two integers! But both have the same assembly language implementa:on! Type Systems! A language s type system specifies which opera:ons are valid for which types! The goal of type checking is to ensure that opera:ons are used with the correct types! Enforces intended interpreta:on of values, because nothing else will! 18

19 Type Checking! Sta:cally typed: All or almost all checking of types is done as part of compila:on (C, Java, C#)! Dynamically typed: Almost all checking of types is done as part of program execu:on (LISP)! Untyped: No type checking (machine code) The type war! Sta:c typing proponents say: o Sta:c checking catches many programming errors at compile :me o Avoids overhead of run:me typechecks! Dynamic typing proponents say: o Sta:c type systems are restric:ve o Rapid prototyping difficult within a sta:c type system 19

20 Type Checking vs Type Inference! Type Checking is the process of verifying fully typed programs! Type Inference is the process of filling in missing type informa:on! The two are different, but the terms are onen used interchangeably Sta:cs! Aner parsing, we have an AST.! Cri:cal issue: o not all opera:ons are defined on all values. o e.g., (3 / 0), sub("foo",5), 42(x)! Op:ons 1. don't worry about it (C, C++, etc.) 2. force all opera:ons to be total (Scheme) e.g., (3 / 0) - > 0, 42(x) - > try to rule out ill- formed expressions (ML) 20

21 Type Soundness! Construct a model of the source language o i.e., interpreter o This tells us where opera:ons are par:al. o And par:ality is different for different languages (e.g., "foo" + "bar" may be meaningful in some languages, but not others.)! Construct a func:on TC: AST - > bool o when true, should ensure interpre:ng the AST does not result in an undefined opera:on.! Prove that TC is correct. 21

22 22

23 23

24 Simple Language: type tipe = Int_t Fn_t of tipe*tipe Pair_t of tipe*tipe type exp = Var of var Int of int Plus_i of exp*exp Lambda of var * tipe * exp App of exp*exp Pair of exp * exp Fst of exp Snd of exp Interpreter: let rec interp (env:var->value)(e:exp) = match e with Var x -> env x Int I -> Int_v i Plus_i(e1,e2) -> (match interp env e1, interp env e2 of Int i, Int j -> Int_v(i+j) _,_ => error()) Lambda(x,t,e) => Closure_v{env=env,code=(x,e)} App(e1,e2) => (match interp env e1, interp env e2 of Closure_v{env=cenv,code=(x,e)},v -> interp (extend cenv x v) e _,_ -> error() 24

25 Type Checker: let rec tc (env:var->tipe) (e:exp) = match e with Var x -> env x Int _ -> Int_t Plus_i(e1,e2) -> (match tc env e1, tc env e with Int_t, Int_t -> Int_t _,_ => error()) Lambda(x,t,e) -> Fn_t(t,tc (extend env x t) e) App(e1,e2) -> (match (tc env e1, tc env e2) with Arrow_t(t1,t2), t -> if (t1!= t) then error() else t2 _,_ -> error()) Growing the language type tipe =... Bool_t type exp =... True False If of exp*exp*exp let rec interp env e =... True -> True_v False -> False_v If(e1,e2,e3) -> (match interp env e1 with True_v -> interp env e2 False_v -> interp env e3 _ => error()) 25

26 Type- Checking let rec tc (env:var->tipe) (e:exp) = match e with... True -> Bool_t False -> Bool_t If(e1,e2,e3) -> (let (t1,t2,t3) = (tc env e1,tc env e2,tc env e3) in match t1 with Bool_t -> if (t2!= t3) then error() else t2 _ => error()) Refining Types We can easily add new types that dis:nguish different subsets of values. type tipe =... True_t False_t Bool_t Pos_t Neg_t Zero_t Int_t Any_t 26

27 Modified Type- Checker let rec tc (env:var->tipe) (e:exp) =... True -> True_t False -> False_t If(e1,e2,e3) -> (match tc env e1 with True_t -> tc env e2 False_t -> tc env e3 Bool_t -> (let (t2,t3) = (tc env e2, tc env e3) in lub t2 t3) _ => error()) Least Upper Bound let lub t1 t2 = match t1, t2 with True_t (Bool_t False_t) -> Bool_t False_t,(Bool_t True_t) -> Bool_t Zero_t, (Neg_t Pos_t Int_t) -> Int_t Neg_t, (Zero_t Pos_t Int_t) -> Int_t Pos_t, (Zero_t Pos_t Int_t) -> Int_t _,_ -> if (t1 = t2) then t1 else Any_t 27

28 Refining Integers into Zero,Neg,Pos let rec tc (env:var->tipe) (e:exp) =... Int 0 -> Zero_t Int I -> if i < 0 then Neg_t else Pos_t Plus(e1,e2) -> (match tc env e1, tc env e2 with Zero_t,t2 -> t2 t1,zero_t -> t1 Pos_t,Pos_t -> Pos_t Neg_t,Neg_t -> Neg_t (Neg_t Pos_t Int_t),Int_t -> Int_t Int_t,(Neg_t Pos_t) -> Int_t _,_ -> error()) Subtyping as Subsets! If we think of types as sets of values, then a subtype corresponds to a subset and lub corresponds to union.! e.g., Pos_t <= Int_t since every posi:ve integer is also an integer.! For condi:onals, we want to find the least type of the types the two branches might return. o Adding "Any_t" ensures there is a type. o (Not always a good thing to have ) o Need NonPos_t, NonNeg_t and NonZero_t to get least upper bounds. 28

29 Extending Subtyping:! What about pairs? o (T 1 * T 2 ) <= (U 1 * U 2 ) when T 1 <= U 1 and T 2 <= U 2 o But only when immutable! o Why?! What about func:ons? o (T 1 - > T 2 ) <= (U 1 - > U 2 ) when U 1 <= T 1 and T 2 <= U 2. o Why? Problems with Mutability: let f(p :ref(pos_t)) = let q :ref(int_t) = p in q := 0; 42 div (!p) 29

30 Another Way to See it:! Any shared, mutable data structure can be thought of as an immutable record of pairs of methods (with hidden state): o p : ref(pos_t) => o p : { get: unit -> Pos_t, set: Pos_t -> unit }! When is ref(t) <= ref(u)? When: o unit->t <= unit->u or T <= U and o T->unit <= U->unit or U <= T. o Thus, only when T = U! N- Tuples and Simple Records:! (T 1 * * T n * T n+1 ) <= (U 1 * * U n ) when T i <= U i. o Why?! Non- Permutable Records: {l 1 :T 1,,l n :T n,l n+1 :T n+1 } <= {l 1 :U 1,, l n :U n } when T i <= U i. o Assumes {x:int,y:int}!= {y:int,x:int} o That is, the posi:on of a label is independent of the rest of the labels in the type. o In SML (or for Java interfaces) this is not the case. 30

31 SML- Style Records:! Compiler sorts by label.! So if you write {y:int,z:int,x:int}, the compiler immediately rewrites it to {x:int,y:int,z:int}.! So you need to know all of the labels to determine their posi:ons.! Consider: {y:int,z:int,x:int} <= {y:int,z:int} but {y,z,x} == {x,y,z} <= {y,z} If you want both:! If you want permutability & dropping, you need to either copy or use a dic:onary: x y z p = {x=42,y=55,z=66}:{x:int,y:int,z:int} y z q : {y:int,z:int} = p 31

32 Type Inference let rec tc (env:(var*tipe) list) (e:exp) = match e with Var x -> lookup env x Lambda(x,e) -> (let t = guess() in Fn_t(t,tc (extend env x t) e)) App(e1,e2) -> (match tc env e1, tc env e2 with Fn_t(t1,t2), t -> if t1!= t then error() else t2 _,_ => error()) Extend Types with Guesses: type tipe = Int_t Fn_t of tipe*tipe Guess of (tipe option ref) fun guess() = Guess(ref None) 32

33 Must Handle Guesses Lambda(x,e) -> let t = guess() in Fn_t(t,tc (extend env x t) e) App(e1,e2) -> (match tc env e1, tc env e2 with Fn_t(t1,t2), t -> if t1!= t then error() else t2 Guess (r as ref None), t -> let t2 = guess() in r := Some(Fn_t(t,t2)); t2 Guess (ref Some (Fn_t(t1,t2))), t -> if t1!= t then error() else t2 Cleaner: let rec tc (env: (var*tipe) list) (e:exp) = match e with Var x -> lookup env x Lambda(x,e) -> let t = guess() in Fn_t(t,tc (extend env x t) e) App(e1,e2) -> let (t1,t2) = (tc env e1, tc env e2) in let t = guess() in if unify t1 (Fn_t(t2,t)) then t else error() 33

34 Where: let rec unify (t1:tipe) (t2:tipe):bool = if (t1 = t2) then true else match t1,t2 with Guess(ref(Some t1')), _ -> unify t1 t2 Guess(r as (ref None)), t2 -> (r := t2; true) _, Guess(_) -> unify t2 t1 Int_t, Int_t -> true Fn_t(t1a,t1b), Fn_t(t2a,t2b)) -> unify t1a t2a && unify t1b t2b Subtlety! Consider: fun x => x x! We guess g1 for x o We see App(x,x) o recursive calls say we have t1=g1 and t2=g1. o We guess g2 for the result. o And unify(g1,fn_t(g1,g2)) o So we set g1 := Some(Fn_t(g1,g2)) o What happens if we print the type? 34

35 Fixes:! Do an "occurs" check in unify: let rec unify (t1:tipe) )t2:tipe):bool = if (t1 = t2) then true else case (t1,t2) of (Guess(r as ref None),_) => if occurs r t2 then error() else (r := Some t2; true)! Alterna:vely, be careful not to loop anywhere. o In par:cular, when comparing t1 = t2, we must code up a graph equality, not a tree equality. Polymorphism:! Consider: fun x => x! We guess g1 for x o We see x. o So g1 is the result. o We return Fn_t(g1,g1) o g1 is unconstrained. o We could constraint it to Int_t or Fn_t(Int_t,Int_t) or any type. o In fact, we could re- use this code at any type. 35

36 type exp = ML Expressions: Var of var Int of int Lambda of var * exp App of exp*exp Let of var * exp * exp Naïve ML Type Inference: let rec tc (env: (var*tipe) list) (e:exp) = match e with Var x -> lookup env x Lambda(x,e) -> let t = guess() in Fn_t(t,tc (extend env x t) e) end App(e1,e2) -> let (t1,t2) = (tc env e1, tc env e2) in let t = guess() in if unify t1 (Fn_t(t2,t)) then t else error() Let(x,e1,e2) => (tc env e1; tc env (substitute(e1,x,e2)) 36

37 Example: let id = fn x => x in (id 3, id "fred") end ====> ((fun x => x) 3, (fun x => x) "fred") Be~er Approach (DM): type tvar = string type tipe = Int_t Fn_t of tipe*tipe Guess of (tipe option ref) Var_t of tvar type tipe_scheme = Forall of (tvar list * tipe) 37

38 ML Type Inference let rec tc (env:(var*tipe_scheme) list) (e:exp) = match e with Var x -> instantiate(lookup env x) Int _ -> Int_t Lambda(x,e) -> let g = guess() in Fn_t(g,tc (extend env x (Forall([],t)) e) App(e1,e2) -> let (t1,t2,t) = (tc env e1,tc env e2,guess()) in if unify(t1,fn_t(t2,t)) then t else error() Let(x,e1,e2) -> let s = generalize(env,tc env e1) in tc (extend env x s) e2 end Instan:a:on let instantiate(s:tipe_scheme):tipe = let val Forall(vs,t) = s val vs_and_ts : (var*tipe) list = map (fn a => (a,guess()) vs in end substitute(vs_and_ts,t) 38

39 Generaliza:on: let generalize(e:env,t:tipe):tipe_scheme = let t_gs = guesses_of_tipe t in let env_list_gs = map (fun (x,s) -> guesses_of s) e in let env_gs = foldl union empty env_list_gs let diff = minus t_gs env_gs in let gs_vs = map (fun g -> (g,freshvar())) diff in let tc = subst_guess(gs_vs,t) in Forall(map snd gs_vs, tc) end Explana:on:! Each let- bound value is generalized. o e.g., g- >g generalizes to Forall a.a - > a.! Each use of a let- bound variable is instan:ated with fresh guesses: o e.g., if f:forall a.a- >a, then in f e, then the type we assign to f is g- >g for some fresh guess g.! But we can't generalize guesses that might later become constrained. o Sufficient to filter out guesses that occur elsewhere in the environment. o e.g., if the expression has type g1- >g2 and y:g1, then we might later use y in a context that demands it has type int, such as y+1. 39

40 Effects:! The algorithm given is equivalent to subs:tu:ng the let- bound expression.! But in ML, we evaluate CBV, not CBN! let id = (print "Hello"; fn x => x) in (id 42, id "fred )!= ((print "Hello";fn x=>x) 42, (print "Hello";fn x=>x) "fred") Problem: let r = ref (fn x=>x) in (* r : Forall 'a.ref('a->'a) *) r := (fn x => x+1); (* r:ref(int->int) *) (!r)("fred") (* r:ref(string->string) *) 40

41 "Value Restric:on"! When is let x=e1 in e2 equivalent to subst(e1,x,e2)?! If e1 has no side effects. o reads/writes/alloca:on of refs/arrays. o input, output. o non- termina:on.! So only generalize when e1 is a value. o or something easy to prove equivalent to a value. Real Algorithm: let rec tc (env:var->tipe_scheme) (e:exp) = match e with... Let(x,e1,e2) -> let s = if may_have_effects e1 then Forall([],tc env e1) else generalize(env,tc env e1) in tc (extend env x s) e2 end 41

42 Checking Effects: let rec may_have_effects e = match e with Int _ -> false Var _ -> false Lambda _ -> false Pair(e1,e2) -> may_have_effects e1 may_have_effects e2 App _ -> true 42