Agenda. Previously. Tentative syllabus. Fall Compiler Principles Lecture 5: Parsing part 4 12/2/2015. Roman Manevich Ben-Gurion University

Transcription

1 Fall Compiler Principles ecture 5: Parsing part 4 Tentative syllabus Front End Intermediate epresentation Optimizations Code Generation Scanning Operational Semantics Dataflow Analysis egister Allocation Top-down Parsing () owering oop Optimizations Instruction Selection Bottom-up Parsing () oman Manevich Ben-Gurion University 2 Previously (0) parsing unning the parser Constructing transition diagram Constructing parser table Detecting conflicts S(0) Eliminating SHIFT-EDUCE via FOOW sets Agenda (1) A(1) Automatic parser generation Handling ambiguities 3 4 1

2 Going beyond S(0) Some common language constructs introduce conflicts even for S (0) S S (1) S (2) S (3) (4) (5) q0 S S S S q4 S q3 q1 q5 S S S S q9 S S q8 5 q7 6 shift/reduce conflict Inputs requiring shift/reduce (0) S S (1) S (2) S (3) (4) (5) S vs. FOOW() contains S S cannot resolve conflict S S (0) S S (1) S (2) S (3) (4) (5) S For the input the rightmost derivation S > S > > > requires reducing in For the input S > S > > > > requires shifting S 7 8 2

3 (1) grammars (1) item In S: a reduce item N α is applicable only when the lookahead is in FOOW(N) But FOOW(N) merges lookahead for all alternatives for N Insensitive to the context of a given production Input Already matched To be matched (1) keeps lookahead with each item Idea: a more refined notion of FOOW computed per item 9 N α β, t Hypothesis about αβ being a possible handle, so far we ve matched α, expecting to see β and after reducing N we expect to see the token t 10 (1) items Computing Closure for (1) (1) item is a pair (0) item ookahead token Meaning We matched the part left of the dot, looking to match the part on the right of the dot, followed by the lookahead token Example The production yields the following (1) items For every [A α Bβ, c] in S for every production B δ and every token b in the grammar such that b FIST(βc) Add [B δ, b] to S (1) items (0) S S (1) S (2) S (3) (4) (5) (0) items [ ] [ ] [, ] [, ] [, ] [, $] [, ] [, ] [, ] [, $]

4 q0 (S S, $) (S, $) (S, $) (, ) (, ) (, $ ) (, $ ) (, $ ) q4 (, ) (, ) (, ) (, ) (, $) (, $) (, $) (, $) S q3 (S, $) q1 (S S, $) (S, $) (, $) q5 (, $) (, ) q7 (, ) (, $) q9 (S, $) (S, $) (, $) (, $) (, $) q8 (, ) (, $) q11 (, $) q12 (, $) q13 q10 (, $) (, $) (, $) (, $) (, $) 13 Back to the conflict (S, $) (, $) Is there a conflict now? (S, $) (, $) (, $) (, $) 14 A(1) (1) tables have huge number of entries Often don t need such refined observation (and cost) Idea: find states with the same (0) component and merge their lookaheads component as long as there are no conflicts A(1) not as powerful as (1) in theory but works quite well in practice Merging may not introduce new shift-reduce conflicts, only reduce-reduce, which is unlikely in practice 15 q0 (S S, $) (S, $) (S, $) (, ) (, ) (, $ ) (, $ ) (, $ ) q4 (, ) (, ) (, ) (, ) (, $) (, $) (, $) (, $) S q3 (S, $) q1 (S S, $) (S, $) (, $) q5 (, $) (, ) q7 (, ) (, $) q9 (S, $) (S, $) (, $) (, $) (, $) q8 (, ) (, $) q11 (, $) q12 (, $) q13 q10 (, $) (, $) (, $) (, $) (, $) 16 4

5 q0 (S S, $) (S, $) (S, $) (, ) (, ) (, $ ) (, $ ) (, $ ) q4 (, ) (, ) (, ) (, ) (, $) (, $) (, $) (, $) S q3 (S, $) q1 (S S, $) (S, $) (, $) q5 (, $) (, ) q7 (, ) (, $) q9 (S, $) (S, $) (, $) (, $) (, $) q8 (, ) (, $) q11 (, $) q12 (, $) q13 q10 (, $) (, $) (, $) (, $) (, $) q0 (S S, $) (S, $) (S, $) (, ) (, ) (, $ ) (, $ ) (, $ ) q4 (, ) (, ) (, ) (, ) (, $) (, $) (, $) (, $) S q3 (S, $) q1 (S S, $) (S, $) (, $) q5 (, $) (, ) q7 (, ) (, $) q9 (S, $) (S, $) (, $) (, $) (, $) q8 (, ) (, $) q12 (, $) q10 (, $) (, $) (, $) (, $) eft/ight- recursion Example: non-(1) grammar At home: create a simple grammar with leftrecursion and one with right-recursion Construct corresponding (0) parser Any conflicts? un on simple input and observe behavior Attempt to generalize observation for long inputs S Y b c, $ S Y b c, $ Y a, b Z a, b a (1) S Y b c $ (2) S Z b d $ (3) Y a (4) Z a Y a, b Z a, b reduce-reduce conflict on lookahead b

6 Automated parser generation (via CUP) exer spec ANG.lex Parser spec ANG.cup High-level structure JFlex CUP.java exer.java.java Parser.java sym.java javac javac text exical analyzer tokens (Token.java) Parser 21 AST 22 Expression calculator Syntax analysis with CUP - / - ( ) number CUP parser generator Generates an A(1) Parser Input: spec file Output: a syntax analyzer Can dump automaton and table tokens Goals of ession calculator parser: Is 2345 a val ession? What is the meaning (value) of this ession? Parser spec CUP.java javac Parser 23 AST 24 6

7 CUP spec file Package and import specifications User code components Symbol (terminal and non-terminal) lists Terminals go to sym.java Types of AST nodes Precedence declarations The grammar Semantic actions to construct AST Parsing ambiguous grammars Expression Calculator 1 st Attempt Ambiguities terminal Integer NUMBE terminal PUS, MINUS, MUT, DIV terminal PAEN, PAEN non terminal Integer :: PUS MINUS MUT DIV MINUS PAEN PAEN NUMBE Symbol type explained later a b c 27 a b c 28 7

8 Ambiguities as conflicts for (1) a b c a b c terminal Integer NUMBE terminal PUS,MINUS,MUT,DIV terminal PAEN, PAEN terminal UMINUS non terminal Integer 29 precedence left PUS, MINUS precedence left DIV, MUT precedence left UMINUS Expression Calculator 2 nd Attempt :: PUS MINUS MUT DIV MINUS %prec UMINUS PAEN PAEN NUMBE Increasing precedence Contextual precedence 30 Parsing ambiguous grammars using precedence declarations Each terminal assigned with precedence By default all terminals have lowest precedence User can assign his own precedence CUP assigns each production a precedence Precedence of rightmost terminal in production or user-specified contextual precedence On shift/reduce conflict resolve ambiguity by comparing precedence of terminal and production and deces whether to shift or reduce In case of equal precedences left/right help resolve conflicts left means reduce right means shift More information on precedence declarations in CUP s manual esolving ambiguity (associativity) precedence left PUS a b c

9 esolving ambiguity (op. precedence) precedence left PUS precedence left MUT esolving ambiguity (contextual) precedence left MUT MINUS %prec UMINUS - - a b a b a b c - a b esolving ambiguity More CUP directives terminal Integer NUMBE terminal PUS,MINUS,MUT,DIV terminal PAEN, PAEN terminal UMINUS precedence left PUS, MINUS precedence left DIV, MUT precedence left UMINUS :: PUS MINUS MUT DIV MINUS %prec UMINUS PAEN PAEN NUMBE UMINUS never returned by scanner (used only to define precedence) ule has precedence of UMINUS precedence nonassoc NEQ Non-associative operators: < >! etc. 1<2<3 entified as an error (semantic error?) start non-terminal Specifies start non-terminal other than first non-terminal Can change to test parts of grammar Getting internal representation Command line options: -dump_grammar -dump_states -dump_tables -dump

10 Scanner integration import java_cup.runtime. %% %cup Generated from token %eofval{ declarations in.cup file return new Symbol(sym.EOF) %eofval} NUMBE[0-9] %% <YYINITIA> { return new Symbol(sym.PUS) } <YYINITIA> - { return new Symbol(sym.MINUS) } <YYINITIA> { return new Symbol(sym.MUT) } <YYINITIA> / { return new Symbol(sym.DIV) } <YYINITIA> ( { return new Symbol(sym.PAEN) } <YYINITIA> ) { return new Symbol(sym.PAEN) } <YYINITIA>{NUMBE} { return new Symbol(sym.NUMBE, new Integer(yytext())) } <YYINITIA>\n { } <YYINITIA>. { } Parser gets terminals from the scanner 37 ecap Package and import specifications and user code components Symbol (terminal and non-terminal) lists Define building-blocks of the grammar Precedence declarations May help resolve conflicts The grammar May introduce conflicts that have to be resolved 38 Assigning meaning Abstract syntax tree construction :: PUS MINUS MUT DIV MINUS %prec UMINUS PAEN PAEN NUMBE So far, only valation Add Java code implementing semantic actions

11 non terminal Integer Assigning meaning :: :e1 PUS :e2 {: ESUT new Integer(e1.intValue() e2.intvalue()) :} :e1 MINUS :e2 {: ESUT new Integer(e1.intValue() - e2.intvalue()) :} :e1 MUT :e2 {: ESUT new Integer(e1.intValue() e2.intvalue()) :} :e1 DIV :e2 {: ESUT new Integer(e1.intValue() / e2.intvalue()) :} MINUS :e1 {: ESUT new Integer(0 - e1.intvalue() :} %prec UMINUS PAEN :e1 PAEN {: ESUT e1 :} NUMBE:n {: ESUT n :} Symbol labels used to name variables ESUT names the left-hand se symbol 41 Abstract Syntax Trees More useful representation of syntax tree ess clutter Actual level of detail depends on your design Basis for semantic analysis ater annotated with various information Type information Computed values Technically a class hierarchy of abstract syntax tree nodes 42 Parse tree vs. AST AST hierarchy example int_const plus minus times dive 1 ( 2 ) ( 3 )

12 AST construction AST construction AST Nodes constructed during parsing Stored in push-down stack Bottom-up parser Grammar rules annotated with actions for AST construction When node is constructed all children available (already constructed) Node (ESUT) pushed on stack 1 (2) (3) (2) (3) () (3) (3) () :: :e1 PUS :e2 {: ESUT new plus(e1,e2) :} PAEN :e PAEN {: ESUT e :} INT_CONST:i {: ESUT new int_const(, i) :} plus e1 e2 plus e1 e2 1 ( 2 ) ( 3 ) int_const val 1 int_const val 2 int_const val Example of lists terminal Integer NUMBE terminal PUS,MINUS,MUT,DIV,PAEN,PAEN,SEMI terminal UMINUS non terminal Integer non terminal _list, _part precedence left PUS, MINUS precedence left DIV, MUT precedence left UMINUS Executed when e is shifted _list :: _list _part _part _part :: :e {: System.out.println(" " e) :} SEMI :: PUS MINUS MUT DIV MINUS %prec UMINUS PAEN PAEN NUMBE 47 Next lecture: Operational Semantics 12