Parsing Techniques C41/C413 Introduction to Compilers Tim Teitelbaum Lecture 11: yntax-directed Definitions February 14, 005 LL parsing Computes a Leftmost derivation Determines the derivation top-down LL parsing table indicates which production to use for expanding the leftmost non-terminal LR parsing Computes a Rightmost derivation Determines the derivation bottom-up Uses a set of LR states and a stack of symbols LR parsing table indicates, for each state, what action to perform (shift/reduce) and what state to go to next Use these techniques to construct an AT C 41/413 pring 007 Introduction to Compilers 1 C 41/413 pring 007 Introduction to Compilers AT Review Derivation = sequence of applied productions E + 1 + 1 + E 1 + Parse tree = graph representation of a derivation Doesn t capture the order of applying the productions Abstract yntax Tree (AT) discards unnecessary information from the parse tree Parse Tree E + 1 E + E 3 AT 1 + + 3 AT Data tructures abstract class Expr { class extends Expr { Expr left, right; (Expr L, Expr R) { left = L; right = R; class Num extends Expr { int value; Num (int v) { value = v; Num 1 Num Num 3 C 41/413 pring 007 Introduction to Compilers 3 C 41/413 pring 007 Introduction to Compilers 4 AT Construction LL/LR parsing techniques implicitly walk the parse tree during parsing LL parsing: Parse tree is implicitly represented by the sequence of applied derivation steps (preorder) LR parsing: Parse tree is implicitly represented by the sequence of applied reductions (endorder) The AT is implicitly defined by the parse tree We want to explicitly construct the AT during parsing: add code in the parser to explicitly build the AT LL AT Construction LL parsing: extend procedures for nonterminals Example: E' ' ε + E num ( ) vo parse_() { switch (token) { case num: case ( : parse_e(); parse_ (); return; default: throw new ParseError(); Expr parse_() { switch (token) { case num: case ( : Expr left = parse_e(); Expr right = parse_ (); if (right == null) return left; else return new (left, right); default: throw new ParseError(); C 41/413 pring 007 Introduction to Compilers 5 C 41/413 pring 007 Introduction to Compilers 6 1
LR AT Construction LR parsing We also need to add code for explicit AT construction AT construction mechanism for LR Parsing tore parts of the tree on the stack For each symbol X on stack, also store the AT subtree rooted at X on stack Whenever the parser performs a reduce operation for a production A β, create an AT node for A from AT fragments on stack for constituents of β Example stack LR AT Construction, ctd. + E Before reduction E+ Num() Num(3) Num(1) E+ E num ( ) Num(1) After reduction E+ Num() Num(3) C 41/413 pring 007 Introduction to Compilers 7 C 41/413 pring 007 Introduction to Compilers 8 Issues Unstructured code: mixed parsing code with AT construction code Automatic parser generators The generated parser needs to contain AT construction code How to construct a customized AT data structure using an automatic parser generator? May want to perform other actions concurrently with the parsing phase E.g., semantic checks This can reduce the number of compiler passes yntax-directed Definition olution: syntax-directed definition Extends each grammar production with an associated semantic action (code): E+ { action The parser generator adds these actions into the generated parser Each action is executed when the corresponding production is reduced C 41/413 pring 007 Introduction to Compilers 9 C 41/413 pring 007 Introduction to Compilers 10 emantic Actions Actions = code in a programming language ame language as the automatically generated parser Examples: Yacc = actions written in C CUP = actions written in Java The actions can access the parser stack! Parser generators extend the stack of states (corresponding to RH symbols) symbols with entries for user-defined structures (e.g., parse trees) The action code should be able to refer to the states (corresponding to the RH grammar symbols in the production Need a naming scheme C 41/413 pring 007 Introduction to Compilers 11 Naming cheme Need names for grammar symbols to use in the semantic action code Need to refer to multiple occurrences of the same nonterminal symbol E E 1 + E Distinguish the nonterminal on the LH E 0 E + E C 41/413 pring 007 Introduction to Compilers 1
Naming cheme: CUP CUP: Name RH nonterminal occurrences using distinct, user-defined labels: expr ::= expr:e1 PLU expr:e Use keyword REULT for LH nonterminal CUP Example: expr ::= expr:e1 PLU expr:e {: REULT = e1 + e; : Naming cheme: yacc Yacc: Uses keywords: $1 refers to the first RH symbol, $ refers to the second RH symbol, etc. Keyword $$ refers to the LH nonterminal YaccExample: expr ::= expr PLU expr { $$ = $1 + $3; C 41/413 pring 007 Introduction to Compilers 13 C 41/413 pring 007 Introduction to Compilers 14 Building the AT Use semantic actions to build the AT AT is built bottom-up during parsing non terminal Expr expr; User-defined type for semantic objects on the stack Nonterminal name expr ::= NUM:i {: REULT = new Num(i.val); : expr ::= expr:e1 PLU expr:e {: REULT = new (e1,e); : expr ::= expr:e1 MULT expr:e {: REULT = new Mul(e1,e); : expr ::= LPAR expr:e RPAR {: REULT = e; : Example E num (E) E+E E*E Parser stack stores value of each symbol (1+)*3 (1 +)*3 (E Num(1) +)*3 REULT=new Num(1) (E+ )*3 (E+E Num() )*3 REULT=new Num() (E (, ) )*3 REULT=new (e1,e) (E) *3 E *3 REULT=e C 41/413 pring 007 Introduction to Compilers 15 C 41/413 pring 007 Introduction to Compilers 16 AT Design Keep the AT abstract Do not introduce a tree node for every node in parse tree (not very abstract) E + ( ) E E + 3 1 E? 1 3 um Expr um LPar um RPar Expr Expr um 3 1 Expr C 41/413 pring 007 Introduction to Compilers 17 AT Design Do not use one single class AT_node E.g., need information for if, while, +, *, ID, NUM class AT_node { int node_type; AT_node[ ] children; tring name; int value; etc Problem: must have fields for every different kind of node with attributes Not extensible, Java type checking no help C 41/413 pring 007 Introduction to Compilers 18 3
Use Class Hierarchy Can use subclassing to solve problem Use an abstract class for each interesting set of non-terminals in grammar (e.g. expressions) E E+E E*E -E (E) abstract class Expr { class extends Expr { Expr left, right; class Mult extends Expr { Expr left, right; // or: class BinExpr extends Expr { Oper o; Expr l, r; class Minus extends Expr { Expr e; Another Example E ::= num (E) E+E ::= E ; if (E) if (E) else = E ; ; abstract class Expr { class Num extends Expr { Num(int value) class extends Expr { (Expr e1, Expr e) class Id extends Expr { Id(tring name) abstract class tmt { class If extends tmt { If(Expr c, tmt s1, tmt s) class Empty extends tmt { Empty() class Assign extends tmt { Assign(tring, Expr e) C 41/413 pring 007 Introduction to Compilers 19 C 41/413 pring 007 Introduction to Compilers 0 Other yntax-directed Definitions Can use syntax-directed definitions to perform semantic checks during parsing E.g., type-checking Benefit = efficiency One single compiler pass for multiple tasks Disadvantage = unstructured code Mixes parsing and semantic checking phases Perform checks while AT is changing Limited to one pass in bottom-up order C 41/413 pring 007 Introduction to Compilers 1 Type Declaration Example D T D D 1, { Type(, T.type); D.type = T.type; { Type(, D 1.type); D.type = D 1.type; T int { T.type = inttype; T float { T.type = floattype; C 41/413 pring 007 Introduction to Compilers Propagation of Values Propagate type attributes while building the AT int a, b T.type D.type D T inttype int D.type D, Type(,D.type) Type(,T.type) D T L Another Example { D.type = T.type; L.type = T.type; T int { T.type = inttype; T float { T.type = floattype; L { Type(,???); L L 1, { Type(, L 1.type);??? C 41/413 pring 007 Introduction to Compilers 3 C 41/413 pring 007 Introduction to Compilers 4 4
Propagation of Values Propagate values both bottom-up and top-down int a, b T.type T inttype int LR parsing: AT is built bottom-up! D L.type L.type L L Type(,L.Type), Type(,L.type) tructured Approach eparate AT construction from semantic checking phase Traverse the AT and perform semantic checks (or other actions) only after the tree has been built and its structure is stable This approach is more flexible and less error-prone It is better when efficiency is not a critical issue C 41/413 pring 007 Introduction to Compilers 5 C 41/413 pring 007 Introduction to Compilers 6 Where We Are Visitor Methodology for AT Traversal ource code (character stream) Token stream Abstract syntax tree (AT) if (b == 0) a = b; if ( b == 0 ) a = b ; if == b 0 = a b Lexical Analysis yntax Analysis (Parsing) emantic Analysis Visitor pattern: useful OO programming pattern that separates data structure definition (e.g., the AT) from code that traverses the structure (e.g., the name resolution code and the type checking code). Define a Visitor interface for all traversals of the AT Extend each AT class with a method that accepts any Visitor Code each traversal as a separate class that implements the Visitor interface C 41/413 pring 007 Introduction to Compilers 7 C 41/413 pring 007 Introduction to Compilers 8 AT Data tructure abstract class Expr Expr { class extends Expr Expr {...... Expr Expre1, e e class Num Numextends Expr Expr {...... int intvalue class Id Idextends Expr Expr {...... tring name interface Visitor { vo visit( e); e); vo visit(num e); e); vo visit(id e); e); Visitor Interface C 41/413 pring 007 Introduction to Compilers 9 C 41/413 pring 007 Introduction to Compilers 30 5
Accept methods Visitor Methods abstract class classexpr {{ abstract public publicvo voaccept(visitor v); v); class class extends Expr Expr {{ public publicvo voaccept(visitor v) v) {{ v.visit(this); class classnum Numextends Expr Expr {{ public publicvo voaccept(visitor v) v) {{ v.visit(this); class classid Idextends Expr Expr {{ public publicvo voaccept(visitor v) v) { v.visit(this); The declared type of this is the subclass it which it occurs. Overload resolution of v.visit(this); invokes appropriate visit function in the Visitor. For each kind of traversal, implement the Visitor interface, e.g., class PostfixOutputVisitor implements Visitor { vo visit( e) { e.e1.accept(this); e.e.accept(this); ystem.out.print( + ); vo visit(num e) { ystem.out.print(value); vo visit(id e) { ystem.out.print(); Dispatch to e.accept in the visit methods eliminates case analysis on AT subclasses To traverse expression e: PostfixOutputVisitor v = new PostfixOutputVisitor(); e.accept(v); C 41/413 pring 007 Introduction to Compilers 31 C 41/413 pring 007 Introduction to Compilers 3 Inherited and ynthesized Information o far, OK for traversal and action w/o communication of values But we need a way to pass information Down the AT (inherited) Up the AT (synthesized) To pass information down the AT add parameter to visit functions To pass information up the AT add return value to visit functions Visitor Interface () interface Visitor { Object visit( e, e, Object inh); Object visit(num e, e, Object inh); inh); Object visit(id e, e, Object inh); inh); C 41/413 pring 007 Introduction to Compilers 33 C 41/413 pring 007 Introduction to Compilers 34 Accept methods () abstract class classexpr {{ abstract public publicobject Objectaccept(Visitor v, v, Object Object inh); inh); class class extends Expr Expr {{ public publicobject Objectaccept(Visitor v, v, Object Object inh) inh) {{ return returnv.visit(this, inh); inh); class classnum Numextends Expr Expr {{ public publicobject Objectaccept(Visitor v, v, Object Object inh) inh) {{ return returnv.visit(this, inh); inh); class classid Idextends Expr Expr {{ public publicobject Objectaccept(Visitor v, v, Object Object inh) inh) {{ return returnv.visit(this, inh); inh); C 41/413 pring 007 Introduction to Compilers 35 Visitor Methods () For kind of traversal, implement the Visitor interface, e.g., class EvaluationVisitor implements Visitor { Object visit( e, Object inh) { int left = (int) e.e1.accept(this, inh); int right = (int) e.e.accept(this, inh); return left+right; Object visit(num e, Object inh) { return value; Object visit(id e, Object inh) { return Lookup(, (ymboltable)inh); To traverse expression e: EvaluationVisitor v = new EvaluationVisitor (); e.accept(v, EmptyTable()); C 41/413 pring 007 Introduction to Compilers 36 6
ummary yntax-directed definitions attach semantic actions to grammar productions Easy to construct the AT using syntax-directed definitions Can use syntax-directed definitions to perform semantic checks eparate AT construction from semantic checks or other actions that traverse the AT C 41/413 pring 007 Introduction to Compilers 37 7