EDAN65: Compilers, Lecture 08 Reference A<ribute Grammars AG mechanisms, SemanBc analysis. Görel Hedin Revised:

Transcription

1 EDAN65: Compilers, Lecture 08 Reference A<ribute Grammars AG mechanisms, SemanBc analysis Görel Hedin Revised:

2 This lecture Regular expressions Context- free grammar A<ribute grammar Lexical analyzer (scanner) SyntacBc analyzer (parser) Seman&c analyzer source code (text) tokens AST (Abstract syntax tree) Visitors StaBc aspects A.ribute grammars A<ributed AST runtime system activation records objects stack garbage collection heap Intermediate code generator Interpreter OpBmizer intermediate code Virtual machine code and data intermediate code Target code generator target code machine 2

3 A<ribute mechanisms Synthesized the equabon is in the same node as the a<ribute Inherited the equabon is in an ancestor Broadcas&ng the equabon holds for a complete subtree Reference the a<ribute can be a reference to an AST node. Parameterized the a<ribute can have parameters NTA the a<ribute is a "nonterminal" (a fresh node or subtree) Collec&on the a<ribute is defined by a set of contribubons, instead of by an equabon. Circular the a<ribute may depend on itself (solved using fixed- point iterabon) 3

4 Example computabons on an AST... Name analysis: find the declarabon of an idenbfier Mul Type analysis: compute the type of an expression IdExpr Div IdExpr IdExpr Expression evaluabon: compute the value of a constant expression Code generabon: compute an intermediate code representabon of the program Unparsing: compute a text representabon of the program 4

5 SemanBc analysis class A { int f; int m1(int x) { return x * f; class B extends A { int m2() { System.out.println(m1(3)); Name analysis: bind each use of an idenbfier to its declarabon Type analysis: compute the type of each expression Error checking: find compile- Bme errors like missing declarabons, wrong type, etc. 5

6 Reference and parameterized a<ributes Name analysis 6

7 Reference a<ribute grammars (RAGs) Can build graphs on top of the AST. Trivial example: A B myc C myb JastAdd specificabon: A ::= B C; B; C; aspect Graph { inh C B.myC(); inh B C.myB(); eq A.getB().myC() = getc(); eq A.getC().myB() = getb(); 7

8 Parameterized a<ributes A ::= B*; B; Draw some isbefore a-ributes and their values! A An a<ribute can have parameters. List inh boolean B.isBefore(int i); eq A.getB(int index).isbefore(int i) = index < i; B B B 8

9 Parameterized a<ributes A ::= B*; B; A An a<ribute can have parameters. inh boolean B.isBefore(int i); eq A.getB(int index).isbefore(int i) = index < i; List B B isbefore(0) = false isbefore(1) = false isbefore(2) = true isbefore(3) = true isbefore(...) =... B There is one a<ribute instance for each possible parameter combinabon. PotenBally infinitely many a<ribute instances. Only the accessed ones will be evaluated. 9

10 Name analysis class A { int f; int m1(int x) { return x * f; class B extends A { int m2() { System.out.println(m1(3)); Name analysis: bind each use of an idenbfier to its declarabon Name binding: a reference from a use to its declarabon Scope of a declara&on: the parts of the program where it is visible. Name binding rules: also known as scope rules or visibility rules. Typically, there are rules for blocks, nesbng, inheritance name collisions, shadowing declarabon order (insignificant or declare- before- use?) visibility restricbons (private, public,...) qualified access (a.b) overloading, namespaces... 10

11 Name binding: Blocks class A { int f; int m1(int x) { int a = 4; int f = a + 5; return x * f + b; class B extends A { int m2() { System.out.println(m1(3)); Block: a syntacbc unit containing declarabons and statements. Can be nested. Called block strucure or lexical nes<ng. DeclaraBons in inner blocks shadow declarabons in outer blocks DeclaraBon ordering can be insignificant, or declare- before- use... 11

12 Name binding: Inheritance class A { int x; class AA { int y; class B extends A { int y, z; class BB extends AA { int v; void m() { int w = x + y + z + v;... Inheritance and block nesbng can be combined. In what block order should we look for the declarabon of x? m BB AA B A globally Which declarabon of y is y bound to? AA.y since inheritance binds Bghter than lexical nesbng. 12

13 Name binding: Qualified access class A { int m() {... ; class B extends A { void m() { A a = new B(); System.out.println(a.m()); Qualified access (dot notabon) The binding of m depends on the type of a

14 Recall: RepresenBng name bindings in an AST { int a; a = 3; VarDecl Block Assign IntType IdDecl IdUse IntExp IdDecl for declared names IdUse for used names ID="a" ID="a" decl INT="3" An a<ribute decl represents the name binding. 14

15 Recall: CompuBng name bindings imperabvely { int a; a = 3; VarDecl Block Assign IntType IdDecl ID="a" Use a symbol table data structure: For each block, a map from visible names to declarabons. Use a stack of maps to handle nested blocks. IdUse ID="a" decl IntExp INT="3" Algorithm: Traverse the AST push/pop symbol table when entering/leaving a block add/lookup idenbfiers when encountering IdDecls/IdUses 15

16 Problems with the imperabve approach Need to write an algorithm that computes things in the right order. What if we have more complex name binding rules? Need a more elaborate symbol table. The algorithm may get complex. What if we extend the language? Need to change the algorithm. SoluBon: A<ribute grammars a declarabve approach 16

17 Name analysis using RAGs Think declara&vely!! What a<ributes would I like the nodes to have? Synthesized defined in the node Inherited defined in a parent Program Block eq child.lookup(string) =... VarDecl Assign IntType IdDecl IdUse IntExp ID="a" ID="a" INT="3" syn decl eq decl = lookup(id) inh lookup(string) 17

18 The Lookup pa.ern for name analysis in RAGs decl the name binding lookup(string) finds the declarabon locallookup(string) looks locally eq child.lookup(string) delegates to locallookup and lookup a<ributes, according to scope rules. Program Block eq child.lookup(string) = NoDecl eq child.lookup(string) =... syn locallookup(string) inh lookup(string) Decl Block eq child.lookup(string) =... syn locallookup(string) inh lookup(string) Decl Use ID="a" syn decl inh lookup(string) General pa<ern for name analysis. Can handle block structure, inheritance, qualified access,... 18

19 Example implementabon in JastAdd Abstract grammar: Program ::= Block; Block : Stmt ::= Decl* Stmt*; Decl ::= Type IdDecl; IdDecl ::= <ID:String>; Type; abstract Stmt; Assign : Stmt ::= To:IdUse From:IdUse; IdUse ::= <ID:String>; A<ributes and equabons: syn IdDecl IdUse.decl() = lookup(getid()); inh IdDecl IdUse.lookup(String s); eq Block.getStmt().lookup(String s) { IdDecl d = locallookup(s); if (d!= null) return d; return lookup(s); Decl Program Block Block syn locallookup(string) inh lookup(string) syn locallookup(string) inh lookup(string) syn IdDecl Block.localLookup(String s) { for (Decl d: getdecls()) { if (d.getiddecl().getid().equals(s)) return d.getiddecl(); return null; inh IdDecl Block.lookup(String s); Decl Use ID="a" eq Program.getBlock().lookup(String s) { return null; syn decl inh lookup(string) 19

20 Demand evaluabon A<ributes are not evaluated unbl demanded. Simple recursive caching algorithm: Program If not cached find the equabon compute its right- hand side cache the value fi Return the cached value List Block locallookup("a") locallookup("b") lookup("b") List Decl Assign Example program demanding a<ributes: Program p =... Assign a = p.getblock().getstmt(0); System.out.println(a.getTo().decl()); System.out.println(a.getFrom().decl()); Type IdDecl IdUse IdUse ID="a" ID="a" ID="b" decl 1 lookup("a") 2 decl lookup("b")

21 More efficient implementabon of locallookup syn IdDecl Block.localLookup(String s) { for (Decl d: getdecls()) { if (d.getiddecl().getid().equals(s)) return d.getiddecl(); return unknowndecl(); What happens if there are 1000 elements in the declarabon list? What is the complexity? Linear search for each declarabon gives quadrabc Bme complexity: O(n 2 ) More efficient: Use a local hashmap which is built on the first access. Aler that each access is done in constant Bme. ResulBng complexity: O(n) syn IdDecl Block.localLookup(String s) { IdDecl result = localmap().get(s); return (result!=null)?result:unknowndecl(); syn Map<String,IdDecl> Block.localMap() { Map<String,IdDecl> map = new HashMap<String,IdDecl>(); for (Decl d: getdecls()) { IdDecl id = d.getiddecl(); map.put(id.getid(), id); return map; 21

22 Nonterminal a<ributes The Null Pa<ern 22

23 Nonterminal a<ributes (NTAs) A ::= B; B; C ::= D; D; A nonterminal a<ribute is both an a<ribute and a nonterminal (AST node). Draw the n a-ribute and its value! A B The value must consist of fresh nodes. (If you reuse exisbng nodes, the a<ributes may get inconsistent values.) syn nta C B.n() = new C(new D()); Use for reifying informabon (making implicit informabon explicit). For example, primibve types, predefined funcbons,... 23

24 Nonterminal a<ributes (NTAs) A ::= B; B; C ::= D; D; A nonterminal a<ribute is both an a<ribute and a nonterminal (AST node). A B n The value must consist of fresh nodes. (If you reuse exisbng nodes, the a<ributes may get inconsistent values.) syn nta C B.n() = new C(new D()); Use for reifying informabon (making implicit informabon explicit). For example, primibve types, predefined funcbons,... C D Warning! If you reuse exis<ng nodes for NTAs, the AST will be inconsistent. JastAdd does not check this. 24

25 Nonterminal a<ributes (NTAs) A ::= B; B; C ::= D; D; Draw the x, y, and z a-ributes and their values! A syn nta C B.n() = new C(new D()); B n An NTA may itself have a<ributes. inh int C.x(); eq B.n().x() = 5; syn int B.y() = n().z() * 3; syn int C.z() = x() + 2; C D 25

26 Nonterminal a<ributes (NTAs) A ::= B; B; C ::= D; D; A syn nta C B.n() = new C(new D()); y=21 B n An NTA may itself have a<ributes. inh int C.x(); eq B.n().x() = 5; syn int B.y() = n().z() * 3; syn int C.z() = x() + 2; C D x=5 z=7 26

27 The Null object pa.ern Use a real object instead of null. Give the object suitable behavior. The code becomes simpler. In RAGs: use null objects for missing declarabons, unknown types, etc. But how can we implement unknowndecl()? syn IdDecl IdUse.decl() = lookup(getid()); inh IdDecl IdUse.lookup(String s); eq Block.getStmt().lookup(String s) { IdDecl d = locallookup(s); if (!d.isunknown()) return d; return lookup(s); syn IdDecl Block.localLookup(String s) { for (Decl d: getdecls()) { if (d.getiddecl().getid().equals(s)) return d.getiddecl(); return unknowndecl(); inh IdDecl Block.lookup(String s); eq Program.getBlock().lookup(String s) { return unknowndecl(); See h<p://en.wikipedia.org/wiki/null_object_pa<ern 27

28 Non- terminal a<ributes (NTAs) An NTA is both a node and an a.ribute. The right- hand side of its defining equabon must be a fresh object (not part of any AST). A eq child.a =... Useful for reifying implicit constructs (make them explicit in the AST), like: Missing declarabons Unknown types PrimiBve types and funcbons C B n N inh a an NTA syn nta N B.n() = new N(); An NTA can have a<ributes. The owning node (or its ancestors) must define the inherited a<ributes of the NTA. 28

29 Non- terminal a<ributes (NTAs) An NTA is both a node and an a.ribute. The right- hand side of its defining equabon must be a fresh object (not part of any AST). Add an UnknownDecl object to the AST using a non- terminal a<ribute Extend the abstract grammar: UnknownDecl : IdDecl; List Program unknowndecl Block UnknownDecl List Add the NTA: syn nta UnknownDecl Program.unknownDecl() = new UnknownDecl("<Unknown>"); Implement the special behavior: syn boolean IdDecl.isUnknown() = false; eq UnknownDecl.isUnknown() = true; Type Decl IdDecl IdUse Assign IdUse ID="a" ID="a" ID="b" But how can we make the UnknownDecl object known throughout the AST? 29

30 The Root A.ribute pa<ern Intent: Make an a<ribute in the root visible throughout the AST. Solu&on: Add an equabon in the root, propagabng the value to the children. Expose the a<ribute by declaring it as inherited where it is needed. Or declare it in ASTNode. Then it will be visible in all nodes. B D Root A C syn a eq child.a = a E inh a 30

31 ImplemenBng unknowndecl Program Apply the Root A<ribute pa<ern: unknowndecl eq Program.getBlock().unknownDecl() = unknowndecl(); unknowndecl Block UnknownDecl inh UnknownDecl Block.unknownDecl(); List List Decl Assign Type IdDecl IdUse IdUse ID="a" ID="a" ID="b" decl 31

32 Type analysis 32

33 Type analysis Type analysis: compute the type of each expression Add a type a<ribute to Expr syn Type Expr.type(); Implement it for IdUses eq IdUse.type() = decl().type(); Define the type a<ribute for IdDecls inh Type IdDecl.type(); eq Decl.getIdDecl().type() = gettype(); eq Program.unknownDecl().type() = unknowntype(); List Decl Program unknowndecl unknowntype Block UnknownDecl List Assign type... Define unknowntype as an NTA... Type IdDecl IdUse IdUse ID="a" ID="a" ID="b" type decl decl type type 33

34 Type checking Type checking: Check if types are used correctly First a<empt: syn boolean Assign.compatibleTypes() = getto().type().equals(getfrom().type()); Second a<empt: syn boolean Assign.compatibleTypes() = getfrom().type().isunknowntype() getto().type().isunknowntype() getto().type().equals(getfrom().type()); List Decl Program unknowndecl unknowntype Block UnknownDecl List Assign type... Type IdDecl IdUse IdUse ID="a" ID="a" ID="b" type decl decl type type Problem with first a<empt: Missing declarabon errors will give type checking errors as well. Would be nicer to view unknowntype as compabble with all other types. 34

35 CollecBon a<ributes Error checking 35

36 CollecBon a<ributes A collecbon a<ribute is defined by contribubons, instead of by a single equabon. A B Use for values combined from many small parts spread out over the tree. collection s C D Example uses: - collect compile- Bme errors in a program - collect what uses are bound to a specific declarabon - count the number of if- statements in a method E E contributes... When a collecbon a<ribute is accessed, the a<ribute evaluator will automabcally traverse the AST and find the contribubons. 36

37 CollecBon a<ributes An collec&on a<ribute has a composite value. Contribu&on rules can declare elements that should contribute to the composite value. collec<on a-ribute t B A C The a<ribute evaluator will automabcally traverse the AST starbng from a given root, and add the contribubons, using a method m which must be commuta&ve. Declare the collecbon: D E contribu<on x theb coll T B.t() [new T()] with m root B; Declare a contribubon (suppose E has an a<ribute x): E contributes x when condition to B.t() for theb(); Propagate theb (Root A<ribute pa<ern): inh B E.theB(); eq B.getChild().theB() = this; 37

38 CollecBon a<ributes, example 1... IdDecl ::= <ID:String>; IdUse ::= <ID:String>; Program ClassDecl ClassDecl IdDecl IdDecl IdUse IdUse ID="b" ID="a" ID="a" decl ID="a" decl 38

39 CollecBon a<ributes, example 1... IdDecl ::= <ID>; IdUse ::= <ID>; Program ClassDecl ClassDecl A "uses" a-ribute contains the collec<on of IdUses referring to the IdDecl IdDecl ID="b" uses= IdDecl ID="a" uses={, IdUse ID="a" decl IdUse ID="a" decl coll Set<IdUse> IdDecl.uses() [new Set<IdUse>()] with add; IdUse contributes this to IdDecl.uses() for decl(); 39

40 JastAdd Java grammar ClassDecl... MethodDecl... ConstructorDecl... CollecBon a<ributes, example 2 Program ClassDecl ClassDecl MethodDecl ConstructorDecl MethodDecl MethodDecl Private Public Public Public ContribuBons can be condibonal. Example: NPM metric- Number of Public Methods and constructors in a class 40

41 JastAdd Java grammar ClassDecl... MethodDecl... ConstructorDecl... CollecBon a<ributes, example 2 Program publicmethods=1 ClassDecl publicmethods=2 ClassDecl MethodDecl ConstructorDecl MethodDecl MethodDecl Private Public Public Public ContribuBons can be condibonal. Example: NPM metric- Number of Public Methods and constructors in a class coll Counter ClassDecl.publicMethods () [new Counter()] with add; MethodDecl contributes 1 when ispublic() to ClassDecl.publicMethods() for hosttype(); ConstructorDecl contributes 1 when ispublic() to ClassDecl.publicMethods() for hosttype(); 41

42 Error checking Error checking: collect all errors Program errors unknowndecl unknowntype We would like an a<ribute errors in the root, containing all error messages. Block UnknownDecl type... We would like an easy way to "contribute" different kinds of errors from different nodes in the AST. List Decl List Assign Type IdDecl IdUse IdUse ID="a" ID="a" ID="b" type decl decl type type contribute error 42

43 Example: CollecBng errors Error checking: collect all errors Program Declare the errors collecbon: errors unknowndecl unknowntype coll Set<String> Program.errors() [new HashSet<String>()] with add root Program; Block UnknownDecl type... Propagate a reference to the Program root: (Root A<ribute pa<ern): List List inh Program ASTNode.theProgram(); eq Program.getChild().theProgram() = this; Decl Assign Type IdDecl IdUse IdUse Contribute an error IdUse contributes "Undeclared variable" when decl().isunknown() to Program.errors() for theprogram(); ID="a" ID="a" ID="b" type decl decl type type contribute error 43

44 Circular a<ributes 44

45 Circular a<ributes syn Set<State> State.succ() =...; StateMachine State 1 State 2 State 3 State 4 reachable={2,3 reachable={2,3 reachable={2,3 reachable= What states are reachable from state k? MathemaBcal definibon: ImplementaBon using a circular a<ribute syn Set<State> State.reachable() circular [new HashSet<State>()] { HashSet<State> result = new HashSet<State>(); for (State s : succ()) { result.add(s); result.addall(s.reachable()); return result; A circular a<ribute may depend (transibvely) on itself. 45

46 Circular a<ributes - terminabon Does the computabon terminate? Yes! The values (sets of states) can be arranged in a latce. The latce is of finite height (the number of states is finite). The equabons are monotonic: they use set union. Warning! JastAdd does not check this property. If you use non- monotonic equabons or values that can grow unbounded, you might get nonterminabon. ImplementaBon using a circular a<ribute syn Set<State> State.reachable() circular [new HashSet<State>()] { HashSet<State> result = new HashSet<State>(); for (State s : succ()) { result.add(s); result.addall(s.reachable()); return result; A circular a<ribute may depend (transibvely) on itself. 46

47 Useful latce types Set laoce Start with the empty set. Use the UNION operator. Make sure there is a finite set of possible values in a set. Boolean laoces The latce is of finite height: only two possible elements TRUE FALSE FALSE TRUE Start with FALSE Use the OR operator Start with TRUE Use the AND operator 47

48 Circular a<ributes beware of externally visible side effects! It is ok to use local side effects: syn Set<State> State.reachable() circular [new HashSet<State>()] { HashSet<State> result = new HashSet<State>(); for (State s : succ()) { result.add(s); result.addall(s.reachable()); return result; Only the local object is changed. There are no externally visible side effects. This is fine! Warning! If you by mistake change the value of an a<ribute, e.g.... s.reachable().add(...)... JastAdd does not detect this error, and inconsistent a<ribubon may result. 48

49 There are many fixed- point problems in compilers and program analysis tools Cyclic class hierarchy: find out if a class inherits from itself Definite assignment: find out if every variable is guaranteed to have been assigned a value before it is used. Call graph analysis: for example, find methods that are never called (dead code) Data flow analysis: for example, find variables that are never used (dead code) Nullable, FIRST, and FOLLOW (if your "compiler" is actually a parser generator)... 49

50 Summary quesbons: reference a<ributes, name analysis What is a reference a<ribute grammar? What is a reference a<ribute? What is a parameterized a<ribute? What is name analysis? What is a name binding? What does scope mean? Give examples of some typical name binding rules. What does declare- before- use mean? What is qualified access? How does the Lookup pa<ern work? 50

51 Summary quesbons: NTAs, type checking What is a nonterminal a<ribute (NTA)? What is the Null Object pa<ern? How does the Root A<ribute pa<ern work? Why is it useful to implement missing declarabons and unknown types as AST nodes? What is type analysis and type checking? How can unnecessary error propagabon be avoided? 51

52 Summary quesbons: CollecBon a<ributes, Error checking, Circular a<ributes What is a collecbon a<ribute? How can a collecbon of error message be implemented? What is a circular a<ribute? How is a circular a<ribute evaluated? How can you know if the evaluabon of a circular a<ribute will terminate? Give examples of properbes that can be computed using circular a<ributes. 52