# Fall Compiler Principles Lecture 5: Parsing part 4. Roman Manevich Ben-Gurion University

Size: px
Start display at page:

Download "Fall Compiler Principles Lecture 5: Parsing part 4. Roman Manevich Ben-Gurion University"

Transcription

1 Fall Compiler Principles Lecture 5: Parsing part 4 Roman Manevich Ben-Gurion University

2 Tentative syllabus Front End Intermediate Representation Optimizations Code Generation Scanning Lowering Local Optimizations Register Allocation Top-down Parsing (LL) Dataflow Analysis Instruction Selection Bottom-up Parsing (LR) Loop Optimizations Attribute Grammars mid-term exam 2

3 Previously LR(0) parsing Running the parser Constructing transition diagram Constructing parser table Detecting conflicts SLR(0) Eliminating SHIFT-REDUCE via FOLLOW sets 3

4 SLR parsing 4

5 SLR parsing A handle should not be reduced to a nonterminal N if the lookahead is a token that cannot follow N A reduce item N α is applicable only when the lookahead is in FOLLOW(N) If b is not in FOLLOW(N) we just proved there is no terminating derivation S =>* βnb and thus it is safe to remove the reduce item from the conflicted state Differs from LR(0) only on the ACTION table Now a row in the parsing table may contain both shift actions and reduce actions and we need to consult the current token to decide which one to take 5

6 SLR action table Lookahead token from the input State id + ( ) [ ] \$ state action 0 shift shift q0 shift 1 shift shift q1 shift 2 accept q2 3 shift shift q3 shift 4 E E+T E E+T E E+T q4 E E+T 5 T id T id r5, s6 T id vs. q5 T id 6 E T E T E T q6 E T 7 shift shift q7 shift 8 shift shift q8 shift 9 T (E) T (E) T (E) q9 T E as before T id T id[e] SLR use 1 token look-ahead [ is not in FOLLOW(T) LR(0) no look-ahead 6

7 Agenda LR(1) LALR(1) Dealing with ambiguity CUP parser generator 7

8 Going beyond SLR(0) Some common language constructs introduce conflicts even for SLR (0) S S (1) S L = R (2) S R (3) L * R (4) L id (5) R L 8

9 q0 S S S L = R S R L * R L id R L R S L q3 q1 q2 S R S S S L = R R L = q9 S L = R q6 R * * q4 id id q5 L id id S L = R R L L * R L id L * R R L L * R L id * L q8 L R L R L * R q7 9

10 shift/reduce conflict (0) S S (1) S L = R (2) S R (3) L * R (4) L id (5) R L q2 S L = R R L = q6 S L = R vs. R L FOLLOW(R) contains = S L = R * R = R SLR cannot resolve conflict S L = R R L L * R L id 10

11 Inputs requiring shift/reduce (0) S S (1) S L = R (2) S R (3) L * R (4) L id (5) R L q2 S L = R R L = q6 S L = R R L L * R L id For the input id the rightmost derivation S => S => R => L => id requires reducing in q2 For the input id = id S => S => L = R => L = L => L = id => id = id requires shifting 11

12 LR(1) grammars In SLR: a reduce item N α is applicable only when the lookahead is in FOLLOW(N) But FOLLOW(N) merges lookahead for all alternatives for N Insensitive to the context of a given production LR(1) keeps lookahead with each LR item Idea: a more refined notion of FOLLOW computed per item 12

13 LR(1) item Already matched To be matched Input N α β, t Hypothesis about αβ being a possible handle, so far we ve matched α, expecting to see β and after reducing N we expect to see the token t 13

14 LR(1) items LR(1) item is a pair LR(0) item Lookahead token Meaning We matched the part left of the dot, looking to match the part on the right of the dot, followed by the lookahead token Example The production L id yields the following LR(1) items LR(1) items (0) S S (1) S L = R (2) S R (3) L * R (4) L id (5) R L LR(0) items [L id] [L id ] [L id, *] [L id, =] [L id, id] [L id, \$] [L id, *] [L id, =] [L id, id] [L id, \$] 14

15 Computing Closure for LR(1) For every [A α Bβ, c] in S for every production B δ and every token b in the grammar such that b FIRST(βc) Add [B δ, b] to S 15

16 * q0 (S S, \$) (S L = R, \$) (S R, \$) (L * R, = ) (L id, = ) (R L, \$ ) (L id, \$ ) (L * R, \$ ) q4 * (L * R, =) (R L, =) (L * R, =) (L id, =) (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) id R id q5 R L S q3 (S R, \$) q1 (S S, \$) q2 (S L = R, \$) (R L, \$) (L id, \$) (L id, =) q7 L (L * R, =) (L * R, \$) q6 q9 (S L = R, \$) = R (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) q8 (R L, =) (R L, \$) L * q11 (L id, \$) id q12 (R L, \$) q13 q10 (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) R (L * R, \$) id 16

17 Back to the conflict q2 (S L = R, \$) (R L, \$) = q6 (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) Is there a conflict now? 17

18 LALR(1) LR(1) tables have huge number of entries Often don t need such refined observation (and cost) Idea: find states with the same LR(0) component and merge their lookaheads component as long as there are no conflicts LALR(1) not as powerful as LR(1) in theory but works quite well in practice Merging may not introduce new shift-reduce conflicts, only reduce-reduce, which is unlikely in practice 18

19 * q0 (S S, \$) (S L = R, \$) (S R, \$) (L * R, = ) (L id, = ) (R L, \$ ) (L id, \$ ) (L * R, \$ ) q4 * (L * R, =) (R L, =) (L * R, =) (L id, =) (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) id R id q5 R L S q3 (S R, \$) q1 (S S, \$) q2 (S L = R, \$) (R L, \$) (L id, \$) (L id, =) q7 L (L * R, =) (L * R, \$) q6 q9 (S L = R, \$) = R (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) q8 (R L, =) (R L, \$) L * q11 (L id, \$) id q12 (R L, \$) q13 q10 (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) R (L * R, \$) id 19

20 * q0 (S S, \$) (S L = R, \$) (S R, \$) (L * R, = ) (L id, = ) (R L, \$ ) (L id, \$ ) (L * R, \$ ) q4 * (L * R, =) (R L, =) (L * R, =) (L id, =) (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) id R id q5 R L S q3 (S R, \$) q1 (S S, \$) q2 (S L = R, \$) (R L, \$) (L id, \$) (L id, =) q7 L (L * R, =) (L * R, \$) q6 q9 (S L = R, \$) = R (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) q8 (R L, =) (R L, \$) L * q11 (L id, \$) id q12 (R L, \$) q13 q10 (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) R (L * R, \$) id 20

21 * q0 (S S, \$) (S L = R, \$) (S R, \$) (L * R, = ) (L id, = ) (R L, \$ ) (L id, \$ ) (L * R, \$ ) q4 * (L * R, =) (R L, =) (L * R, =) (L id, =) (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) id R id q5 R L S q3 (S R, \$) q1 (S S, \$) q2 (S L = R, \$) (R L, \$) (L id, \$) (L id, =) q7 L id (L * R, =) (L * R, \$) q6 q9 (S L = R, \$) = R (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) q8 (R L, =) (R L, \$) L * q12 (R L, \$) q10 (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) R id 21

22 Left/Right- recursion At home: create a simple grammar with leftrecursion and one with right-recursion Construct corresponding LR(0) parser Any conflicts? Run on simple input and observe behavior Attempt to generalize observation for long inputs 22

23 Automated Parser Generation (via CUP) 23

24 High-level structure text Lexer spec JFlex.java javac Lexical analyzer ADTL.lex Lexer.java tokens (Token.java) Parser spec CUP.java javac Parser ADTL.cup sym.java Parser.java AST 24

25 Expression calculator expr expr + expr expr - expr expr * expr expr / expr - expr ( expr ) number Goals of expression calculator parser: Is a valid expression? What is the meaning (value) of this expression? 25

26 Syntax analysis with CUP CUP parser generator Generates an LALR(1) Parser Input: spec file Output: a syntax analyzer tokens Parser spec CUP.java javac Parser AST 26

27 CUP spec file Package and import specifications User code components Symbol (terminal and non-terminal) lists Terminals go to sym.java Types of AST nodes Precedence declarations The grammar Semantic actions to construct AST 27

28 Expression Calculator 1 st Attempt terminal Integer NUMBER; terminal PLUS, MINUS, MULT, DIV; terminal LPAREN, RPAREN; non terminal Integer expr; Symbol type explained later expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr LPAREN expr RPAREN NUMBER ; 28

29 Ambiguities + * * a b c + a + b * c + a b c + + a b c a + b + c + a b c 29

30 Ambiguities as conflicts for LR(1) + * + * a b c a + b * c + a b c + + a b c a + b + c + a b c 30

31 terminal Integer NUMBER; terminal PLUS,MINUS,MULT,DIV; terminal LPAREN, RPAREN; terminal UMINUS; non terminal Integer expr; Expression Calculator 2 nd Attempt precedence left PLUS, MINUS; precedence left DIV, MULT; precedence left UMINUS; Increasing precedence expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr %prec UMINUS LPAREN expr RPAREN NUMBER ; Contextual precedence 31

32 Parsing ambiguous grammars using precedence declarations Each terminal assigned with precedence By default all terminals have lowest precedence User can assign his own precedence CUP assigns each production a precedence Precedence of rightmost terminal in production or user-specified contextual precedence On shift/reduce conflict resolve ambiguity by comparing precedence of terminal and production and decides whether to shift or reduce In case of equal precedences left/right help resolve conflicts left means reduce right means shift More information on precedence declarations in CUP s manual 32

33 Resolving ambiguity precedence left PLUS a b c + a b c a + b + c 33

34 Resolving ambiguity precedence left PLUS precedence left MULT * + + a b c * a b c a * b + c 34

35 Resolving ambiguity precedence left MULT MINUS expr %prec UMINUS * - - a b a * b - a * b 35

36 Resolving ambiguity terminal Integer NUMBER; terminal PLUS,MINUS,MULT,DIV; terminal LPAREN, RPAREN; terminal UMINUS; UMINUS never returned by scanner (used only to define precedence) precedence left PLUS, MINUS; precedence left DIV, MULT; precedence left UMINUS; expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr %prec UMINUS LPAREN expr RPAREN NUMBER ; Rule has precedence of UMINUS 36

37 More CUP directives precedence nonassoc NEQ Non-associative operators: < > ==!= etc. 1<2<3 identified as an error (semantic error?) start non-terminal Specifies start non-terminal other than first non-terminal Can change to test parts of grammar Getting internal representation Command line options: -dump_grammar -dump_states -dump_tables -dump 37

38 Scanner integration import java_cup.runtime.*; %% %cup Generated from token %eofval{ declarations in.cup file return new Symbol(sym.EOF); %eofval} NUMBER=[0-9]+ %% <YYINITIAL> + { return new Symbol(sym.PLUS); } <YYINITIAL> - { return new Symbol(sym.MINUS); } <YYINITIAL> * { return new Symbol(sym.MULT); } <YYINITIAL> / { return new Symbol(sym.DIV); } <YYINITIAL> ( { return new Symbol(sym.LPAREN); } <YYINITIAL> ) { return new Symbol(sym.RPAREN); } <YYINITIAL>{NUMBER} { return new Symbol(sym.NUMBER, new Integer(yytext())); } <YYINITIAL>\n { } <YYINITIAL>. { } Parser gets terminals from the scanner 38

39 Recap Package and import specifications and user code components Symbol (terminal and non-terminal) lists Define building-blocks of the grammar Precedence declarations May help resolve conflicts The grammar May introduce conflicts that have to be resolved 39

40 Assigning meaning expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr %prec UMINUS LPAREN expr RPAREN NUMBER ; So far, only validation Add Java code implementing semantic actions 40

41 Assigning meaning expr ::= expr:e1 PLUS expr:e2 {: RESULT = new Integer(e1.intValue() + e2.intvalue()); :} expr:e1 MINUS expr:e2 {: RESULT = new Integer(e1.intValue() - e2.intvalue()); :} expr:e1 MULT expr:e2 {: RESULT = new Integer(e1.intValue() * e2.intvalue()); :} expr:e1 DIV expr:e2 {: RESULT = new Integer(e1.intValue() / e2.intvalue()); :} MINUS expr:e1 {: RESULT = new Integer(0 - e1.intvalue(); :} %prec UMINUS LPAREN expr:e1 RPAREN {: RESULT = e1; :} NUMBER:n {: RESULT = n; :} ; Symbol labels used to name variables RESULT names the left-hand side symbol 41

42 Building an AST More useful representation of syntax tree Less clutter Actual level of detail depends on your design Basis for semantic analysis Later annotated with various information Type information Computed values Technically a class hierarchy of abstract syntax tree nodes 42

43 Parse tree vs. AST expr + expr + expr expr expr 1 + ( 2 ) + ( 3 )

44 AST hierarchy example expr int_const plus minus times divide 44

45 AST construction AST Nodes constructed during parsing Stored in push-down stack Bottom-up parser Grammar rules annotated with actions for AST construction When node is constructed all children available (already constructed) Node (RESULT) pushed on stack Top-down parser More complicated 45

46 AST construction 1 + (2) + (3) expr + (2) + (3) expr + (expr) + (3) expr + (3) expr + (expr) expr expr expr ::= expr:e1 PLUS expr:e2 {: RESULT = new plus(e1,e2); :} LPAREN expr:e RPAREN {: RESULT = e; :} INT_CONST:i {: RESULT = new int_const(, i); :} plus e1 e2 expr plus e1 e2 expr expr expr 1 + ( 2 ) + ( 3 ) int_const val = 1 int_const val = 2 int_const val = 3 46

47 Example of lists terminal Integer NUMBER; terminal PLUS,MINUS,MULT,DIV,LPAREN,RPAREN,SEMI; terminal UMINUS; non terminal Integer expr; non terminal expr_list, expr_part; precedence left PLUS, MINUS; precedence left DIV, MULT; precedence left UMINUS; expr_list ::= expr_list expr_part expr_part ; expr_part ::= expr:e {: System.out.println("= " + e); :} SEMI ; expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr %prec UMINUS LPAREN expr RPAREN NUMBER ; 47

48 Next lecture: Intermediate Representation

### Fall Compiler Principles Lecture 4: Parsing part 3. Roman Manevich Ben-Gurion University of the Negev

Fall 2016-2017 Compiler Principles Lecture 4: Parsing part 3 Roman Manevich Ben-Gurion University of the Negev Tentative syllabus Front End Intermediate Representation Optimizations Code Generation Scanning

### Agenda. Previously. Tentative syllabus. Fall Compiler Principles Lecture 5: Parsing part 4 12/2/2015. Roman Manevich Ben-Gurion University

Fall 2015-2016 Compiler Principles ecture 5: Parsing part 4 Tentative syllabus Front End Intermediate epresentation Optimizations Code Generation Scanning Operational Semantics Dataflow Analysis egister

### Compilers. Bottom-up Parsing. (original slides by Sam

Compilers Bottom-up Parsing Yannis Smaragdakis U Athens Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Bottom-Up Parsing More general than top-down parsing And just as efficient Builds

### Last Time. What do we want? When do we want it? An AST. Now!

Java CUP 1 Last Time What do we want? An AST When do we want it? Now! 2 This Time A little review of ASTs The philosophy and use of a Parser Generator 3 Translating Lists CFG IdList -> id IdList comma

### EDAN65: Compilers, Lecture 06 A LR parsing. Görel Hedin Revised:

EDAN65: Compilers, Lecture 06 A LR parsing Görel Hedin Revised: 2017-09-11 This lecture Regular expressions Context-free grammar Attribute grammar Lexical analyzer (scanner) Syntactic analyzer (parser)

### Compilers and Language Processing Tools

Compilers and Language Processing Tools Summer Term 2011 Prof. Dr. Arnd Poetzsch-Heffter Software Technology Group TU Kaiserslautern c Prof. Dr. Arnd Poetzsch-Heffter 1 Parser Generators c Prof. Dr. Arnd

### COMP 181. Prelude. Prelude. Summary of parsing. A Hierarchy of Grammar Classes. More power? Syntax-directed translation. Analysis

Prelude COMP 8 October, 9 What is triskaidekaphobia? Fear of the number s? No aisle in airplanes, no th floor in buildings Fear of Friday the th? Paraskevidedekatriaphobia or friggatriskaidekaphobia Why

### Abstract Syntax. Mooly Sagiv. html://www.cs.tau.ac.il/~msagiv/courses/wcc06.html

Abstract Syntax Mooly Sagiv html://www.cs.tau.ac.il/~msagiv/courses/wcc06.html Outline The general idea Cup Motivating example Interpreter for arithmetic expressions The need for abstract syntax Abstract

### Context-free grammars

Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,

### Simple LR (SLR) LR(0) Drawbacks LR(1) SLR Parse. LR(1) Start State and Reduce. LR(1) Items 10/3/2012

LR(0) Drawbacks Consider the unambiguous augmented grammar: 0.) S E \$ 1.) E T + E 2.) E T 3.) T x If we build the LR(0) DFA table, we find that there is a shift-reduce conflict. This arises because the

### Using JFlex. "Linux Gazette...making Linux just a little more fun!" by Christopher Lopes, student at Eastern Washington University April 26, 1999

"Linux Gazette...making Linux just a little more fun!" by Christopher Lopes, student at Eastern Washington University April 26, 1999 This is the third part of a series begun in the April 1999 issue of

### Let us construct the LR(1) items for the grammar given below to construct the LALR parsing table.

MODULE 18 LALR parsing After understanding the most powerful CALR parser, in this module we will learn to construct the LALR parser. The CALR parser has a large set of items and hence the LALR parser is

### CUP. Lecture 18 CUP User s Manual (online) Robb T. Koether. Hampden-Sydney College. Fri, Feb 27, 2015

CUP Lecture 18 CUP User s Manual (online) Robb T. Koether Hampden-Sydney College Fri, Feb 27, 2015 Robb T. Koether (Hampden-Sydney College) CUP Fri, Feb 27, 2015 1 / 31 1 The CUP Parser Generator 2 The

### Compiler Construction: Parsing

Compiler Construction: Parsing Mandar Mitra Indian Statistical Institute M. Mitra (ISI) Parsing 1 / 33 Context-free grammars. Reference: Section 4.2 Formal way of specifying rules about the structure/syntax

### Conflicts in LR Parsing and More LR Parsing Types

Conflicts in LR Parsing and More LR Parsing Types Lecture 10 Dr. Sean Peisert ECS 142 Spring 2009 1 Status Project 2 Due Friday, Apr. 24, 11:55pm The usual lecture time is being replaced by a discussion

### Parsing. Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1)

TD parsing - LL(1) Parsing First and Follow sets Parse table construction BU Parsing Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1) Problems with SLR Aho, Sethi, Ullman, Compilers

### Compilation 2013 Parser Generators, Conflict Management, and ML-Yacc

Compilation 2013 Parser Generators, Conflict Management, and ML-Yacc Erik Ernst Aarhus University Parser generators, ML-Yacc LR parsers are tedious to write, but can be generated, e.g., by ML-Yacc Input:

### MIT Parse Table Construction. Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology

MIT 6.035 Parse Table Construction Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Parse Tables (Review) ACTION Goto State ( ) \$ X s0 shift to s2 error error goto s1

### For example, we might have productions such as:

1. Error Recovery This is an extract from http://www.cs.pr inceton.edu/ appel/moder n/java/cup/manual.html#errors A impor tant aspect of building parsers with CUP is support for syntactic error recovery.

### Lecture Bottom-Up Parsing

Lecture 14+15 Bottom-Up Parsing CS 241: Foundations of Sequential Programs Winter 2018 Troy Vasiga et al University of Waterloo 1 Example CFG 1. S S 2. S AyB 3. A ab 4. A cd 5. B z 6. B wz 2 Stacks in

### Lecture 8: Deterministic Bottom-Up Parsing

Lecture 8: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Fri Feb 12 13:02:57 2010 CS164: Lecture #8 1 Avoiding nondeterministic choice: LR We ve been looking at general

### In One Slide. Outline. LR Parsing. Table Construction

LR Parsing Table Construction #1 In One Slide An LR(1) parsing table can be constructed automatically from a CFG. An LR(1) item is a pair made up of a production and a lookahead token; it represents a

### Lecture 7: Deterministic Bottom-Up Parsing

Lecture 7: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Tue Sep 20 12:50:42 2011 CS164: Lecture #7 1 Avoiding nondeterministic choice: LR We ve been looking at general

### JavaCUP. There are also many parser generators written in Java

JavaCUP JavaCUP (Construct Useful Parser) is a parser generator Produce a parser written in java, itself is also written in Java; There are many parser generators. YACC (Yet Another Compiler-Compiler)

### LR Parsing - The Items

LR Parsing - The Items Lecture 10 Sections 4.5, 4.7 Robb T. Koether Hampden-Sydney College Fri, Feb 13, 2015 Robb T. Koether (Hampden-Sydney College) LR Parsing - The Items Fri, Feb 13, 2015 1 / 31 1 LR

### Bottom-Up Parsing. Lecture 11-12

Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 9/22/06 Prof. Hilfinger CS164 Lecture 11 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient

### Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery. Last modified: Mon Feb 23 10:05: CS164: Lecture #14 1

Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery Last modified: Mon Feb 23 10:05:56 2015 CS164: Lecture #14 1 Shift/Reduce Conflicts If a DFA state contains both [X: α aβ, b] and [Y: γ, a],

### RYERSON POLYTECHNIC UNIVERSITY DEPARTMENT OF MATH, PHYSICS, AND COMPUTER SCIENCE CPS 710 FINAL EXAM FALL 96 INSTRUCTIONS

RYERSON POLYTECHNIC UNIVERSITY DEPARTMENT OF MATH, PHYSICS, AND COMPUTER SCIENCE CPS 710 FINAL EXAM FALL 96 STUDENT ID: INSTRUCTIONS Please write your student ID on this page. Do not write it or your name

### A clarification on terminology: Recognizer: accepts or rejects strings in a language. Parser: recognizes and generates parse trees (imminent topic)

A clarification on terminology: Recognizer: accepts or rejects strings in a language Parser: recognizes and generates parse trees (imminent topic) Assignment 3: building a recognizer for the Lake expression

### Bottom-Up Parsing. Lecture 11-12

Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 2/20/08 Prof. Hilfinger CS164 Lecture 11 1 Administrivia Test I during class on 10 March. 2/20/08 Prof. Hilfinger CS164 Lecture 11

### JavaCC Parser. The Compilation Task. Automated? JavaCC Parser

JavaCC Parser The Compilation Task Input character stream Lexer stream Parser Abstract Syntax Tree Analyser Annotated AST Code Generator Code CC&P 2003 1 CC&P 2003 2 Automated? JavaCC Parser The initial

### LR Parsing LALR Parser Generators

LR Parsing LALR Parser Generators Outline Review of bottom-up parsing Computing the parsing DFA Using parser generators 2 Bottom-up Parsing (Review) A bottom-up parser rewrites the input string to the

### Monday, September 13, Parsers

Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions

### Syntax-Directed Translation. Lecture 14

Syntax-Directed Translation Lecture 14 (adapted from slides by R. Bodik) 9/27/2006 Prof. Hilfinger, Lecture 14 1 Motivation: parser as a translator syntax-directed translation stream of tokens parser ASTs,

### LR Parsing LALR Parser Generators

Outline LR Parsing LALR Parser Generators Review of bottom-up parsing Computing the parsing DFA Using parser generators 2 Bottom-up Parsing (Review) A bottom-up parser rewrites the input string to the

### Wednesday, September 9, 15. Parsers

Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Parsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:

What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### EDAN65: Compilers, Lecture 04 Grammar transformations: Eliminating ambiguities, adapting to LL parsing. Görel Hedin Revised:

EDAN65: Compilers, Lecture 04 Grammar transformations: Eliminating ambiguities, adapting to LL parsing Görel Hedin Revised: 2017-09-04 This lecture Regular expressions Context-free grammar Attribute grammar

### How do LL(1) Parsers Build Syntax Trees?

How do LL(1) Parsers Build Syntax Trees? So far our LL(1) parser has acted like a recognizer. It verifies that input token are syntactically correct, but it produces no output. Building complete (concrete)

### CS606- compiler instruction Solved MCQS From Midterm Papers

CS606- compiler instruction Solved MCQS From Midterm Papers March 06,2014 MC100401285 Moaaz.pk@gmail.com Mc100401285@gmail.com PSMD01 Final Term MCQ s and Quizzes CS606- compiler instruction If X is a

### CSE 401 Midterm Exam Sample Solution 11/4/11

Question 1. (12 points, 2 each) The front end of a compiler consists of three parts: scanner, parser, and (static) semantics. Collectively these need to analyze the input program and decide if it is correctly

### Parsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468

Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure

### Wednesday, August 31, Parsers

Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2017-2018 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman

### Topic 3: Syntax Analysis I

Topic 3: Syntax Analysis I Compiler Design Prof. Hanjun Kim CoreLab (Compiler Research Lab) POSTECH 1 Back-End Front-End The Front End Source Program Lexical Analysis Syntax Analysis Semantic Analysis

### SLR parsers. LR(0) items

SLR parsers LR(0) items As we have seen, in order to make shift-reduce parsing practical, we need a reasonable way to identify viable prefixes (and so, possible handles). Up to now, it has not been clear

### CSE P 501 Compilers. LR Parsing Hal Perkins Spring UW CSE P 501 Spring 2018 D-1

CSE P 501 Compilers LR Parsing Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts UW CSE P 501 Spring 2018

### CS 11 Ocaml track: lecture 6

CS 11 Ocaml track: lecture 6 n Today: n Writing a computer language n Parser generators n lexers (ocamllex) n parsers (ocamlyacc) n Abstract syntax trees Problem (1) n We want to implement a computer language

### shift-reduce parsing

Parsing #2 Bottom-up Parsing Rightmost derivations; use of rules from right to left Uses a stack to push symbols the concatenation of the stack symbols with the rest of the input forms a valid bottom-up

### Lexical and Syntax Analysis. Bottom-Up Parsing

Lexical and Syntax Analysis Bottom-Up Parsing Parsing There are two ways to construct derivation of a grammar. Top-Down: begin with start symbol; repeatedly replace an instance of a production s LHS with

### Topic 5: Syntax Analysis III

Topic 5: Syntax Analysis III Compiler Design Prof. Hanjun Kim CoreLab (Compiler Research Lab) POSTECH 1 Back-End Front-End The Front End Source Program Lexical Analysis Syntax Analysis Semantic Analysis

### COMP-421 Compiler Design. Presented by Dr Ioanna Dionysiou

COMP-421 Compiler Design Presented by Dr Ioanna Dionysiou Administrative! Any questions about the syllabus?! Course Material available at www.cs.unic.ac.cy/ioanna! Next time reading assignment [ALSU07]

### Review of CFGs and Parsing II Bottom-up Parsers. Lecture 5. Review slides 1

Review of CFGs and Parsing II Bottom-up Parsers Lecture 5 1 Outline Parser Overview op-down Parsers (Covered largely through labs) Bottom-up Parsers 2 he Functionality of the Parser Input: sequence of

### LR Parsing. Table Construction

#1 LR Parsing Table Construction #2 Outline Review of bottom-up parsing Computing the parsing DFA Closures, LR(1) Items, States Transitions Using parser generators Handling Conflicts #3 In One Slide An

### Section A. A grammar that produces more than one parse tree for some sentences is said to be ambiguous.

Section A 1. What do you meant by parser and its types? A parser for grammar G is a program that takes as input a string w and produces as output either a parse tree for w, if w is a sentence of G, or

### Formal Languages and Compilers Lecture VII Part 4: Syntactic A

Formal Languages and Compilers Lecture VII Part 4: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

Downloaded from http://himadri.cmsdu.org Page 1 LR Parsing We first understand Context Free Grammars. Consider the input string: x+2*y When scanned by a scanner, it produces the following stream of tokens:

### Example CFG. Lectures 16 & 17 Bottom-Up Parsing. LL(1) Predictor Table Review. Stacks in LR Parsing 1. Sʹ " S. 2. S " AyB. 3. A " ab. 4.

Example CFG Lectures 16 & 17 Bottom-Up Parsing CS 241: Foundations of Sequential Programs Fall 2016 1. Sʹ " S 2. S " AyB 3. A " ab 4. A " cd Matt Crane University of Waterloo 5. B " z 6. B " wz 2 LL(1)

### LR(0) Parsing Summary. LR(0) Parsing Table. LR(0) Limitations. A Non-LR(0) Grammar. LR(0) Parsing Table CS412/CS413

LR(0) Parsing ummary C412/C41 Introduction to Compilers Tim Teitelbaum Lecture 10: LR Parsing February 12, 2007 LR(0) item = a production with a dot in RH LR(0) state = set of LR(0) items valid for viable

### Lexical Analysis. Textbook:Modern Compiler Design Chapter 2.1

Lexical Analysis Textbook:Modern Compiler Design Chapter 2.1 A motivating example Create a program that counts the number of lines in a given input text file Solution (Flex) int num_lines = 0; %% \n ++num_lines;.

### Syntax Analysis. Amitabha Sanyal. (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay

Syntax Analysis (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay September 2007 College of Engineering, Pune Syntax Analysis: 2/124 Syntax

### Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

### LR Parsing. Leftmost and Rightmost Derivations. Compiler Design CSE 504. Derivations for id + id: T id = id+id. 1 Shift-Reduce Parsing.

LR Parsing Compiler Design CSE 504 1 Shift-Reduce Parsing 2 LR Parsers 3 SLR and LR(1) Parsers Last modifled: Fri Mar 06 2015 at 13:50:06 EST Version: 1.7 16:58:46 2016/01/29 Compiled at 12:57 on 2016/02/26

### S Y N T A X A N A L Y S I S LR

LR parsing There are three commonly used algorithms to build tables for an LR parser: 1. SLR(1) = LR(0) plus use of FOLLOW set to select between actions smallest class of grammars smallest tables (number

### CS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F).

CS 2210 Sample Midterm 1. Determine if each of the following claims is true (T) or false (F). F A language consists of a set of strings, its grammar structure, and a set of operations. (Note: a language

### Syntax-Directed Translation

Syntax-Directed Translation ALSU Textbook Chapter 5.1 5.4, 4.8, 4.9 Tsan-sheng Hsu tshsu@iis.sinica.edu.tw http://www.iis.sinica.edu.tw/~tshsu 1 What is syntax-directed translation? Definition: The compilation

### CSE 401 Compilers. LR Parsing Hal Perkins Autumn /10/ Hal Perkins & UW CSE D-1

CSE 401 Compilers LR Parsing Hal Perkins Autumn 2011 10/10/2011 2002-11 Hal Perkins & UW CSE D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts 10/10/2011

### Bottom-up parsing. Bottom-Up Parsing. Recall. Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form

Bottom-up parsing Bottom-up parsing Recall Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form If α V t,thenα is called a sentence in L(G) Otherwise it is just

### Fall Compiler Principles Lecture 3: Parsing part 2. Roman Manevich Ben-Gurion University

Fall 2014-2015 Compiler Principles Lecture 3: Parsing part 2 Roman Manevich Ben-Gurion University Tentative syllabus Front End Intermediate Representation Optimizations Code Generation Scanning Lowering

### COMPILER (CSE 4120) (Lecture 6: Parsing 4 Bottom-up Parsing )

COMPILR (CS 4120) (Lecture 6: Parsing 4 Bottom-up Parsing ) Sungwon Jung Mobile Computing & Data ngineering Lab Dept. of Computer Science and ngineering Sogang University Seoul, Korea Tel: +82-2-705-8930

### CSE 401 Midterm Exam 11/5/10

Name There are 5 questions worth a total of 100 points. Please budget your time so you get to all of the questions. Keep your answers brief and to the point. The exam is closed books, closed notes, closed

### 10/4/18. Lexical and Syntactic Analysis. Lexical and Syntax Analysis. Tokenizing Source. Scanner. Reasons to Separate Lexical and Syntactic Analysis

Lexical and Syntactic Analysis Lexical and Syntax Analysis In Text: Chapter 4 Two steps to discover the syntactic structure of a program Lexical analysis (Scanner): to read the input characters and output

### LR Parsing Techniques

LR Parsing Techniques Introduction Bottom-Up Parsing LR Parsing as Handle Pruning Shift-Reduce Parser LR(k) Parsing Model Parsing Table Construction: SLR, LR, LALR 1 Bottom-UP Parsing A bottom-up parser

### Principle of Compilers Lecture IV Part 4: Syntactic Analysis. Alessandro Artale

Free University of Bolzano Principles of Compilers Lecture IV Part 4, 2003/2004 AArtale (1) Principle of Compilers Lecture IV Part 4: Syntactic Analysis Alessandro Artale Faculty of Computer Science Free

SYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic

### PART 3 - SYNTAX ANALYSIS. F. Wotawa TU Graz) Compiler Construction Summer term / 309

PART 3 - SYNTAX ANALYSIS F. Wotawa (IST @ TU Graz) Compiler Construction Summer term 2016 64 / 309 Goals Definition of the syntax of a programming language using context free grammars Methods for parsing

### Formal Languages and Compilers Lecture VII Part 3: Syntactic A

Formal Languages and Compilers Lecture VII Part 3: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

### CSE302: Compiler Design

CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University March 20, 2007 Outline Recap LR(0)

### CS 4120 Introduction to Compilers

CS 4120 Introduction to Compilers Andrew Myers Cornell University Lecture 6: Bottom-Up Parsing 9/9/09 Bottom-up parsing A more powerful parsing technology LR grammars -- more expressive than LL can handle

### Compiler Construction

Compiler Construction Thomas Noll Software Modeling and Verification Group RWTH Aachen University https://moves.rwth-aachen.de/teaching/ss-17/cc/ Recap: LR(1) Parsing Outline of Lecture 11 Recap: LR(1)

### CSE 401/M501 18au Midterm Exam 11/2/18. Name ID #

Name ID # There are 7 questions worth a total of 100 points. Please budget your time so you get to all of the questions. Keep your answers brief and to the point. The exam is closed books, closed notes,

### Concepts Introduced in Chapter 4

Concepts Introduced in Chapter 4 Grammars Context-Free Grammars Derivations and Parse Trees Ambiguity, Precedence, and Associativity Top Down Parsing Recursive Descent, LL Bottom Up Parsing SLR, LR, LALR

### Principles of Programming Languages

Principles of Programming Languages h"p://www.di.unipi.it/~andrea/dida2ca/plp- 14/ Prof. Andrea Corradini Department of Computer Science, Pisa Lesson 8! Bo;om- Up Parsing Shi?- Reduce LR(0) automata and

### Table-driven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and non-terminals.

Bottom-up Parsing: Table-driven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and non-terminals. Basic operation is to shift terminals from the input to the

### Defining syntax using CFGs

Defining syntax using CFGs Roadmap Last time Defined context-free grammar This time CFGs for specifying a language s syntax Language membership List grammars Resolving ambiguity CFG Review G = (N,Σ,P,S)

### Compiler Construction

Compiler Construction Thomas Noll Software Modeling and Verification Group RWTH Aachen University https://moves.rwth-aachen.de/teaching/ss-17/cc/ Recap: LR(1) Parsing LR(1) Items and Sets Observation:

### LR Parsers. Aditi Raste, CCOEW

LR Parsers Aditi Raste, CCOEW 1 LR Parsers Most powerful shift-reduce parsers and yet efficient. LR(k) parsing L : left to right scanning of input R : constructing rightmost derivation in reverse k : number

### Syntactic Analysis. Syntactic analysis, or parsing, is the second phase of compilation: The token file is converted to an abstract syntax tree.

Syntactic Analysis Syntactic analysis, or parsing, is the second phase of compilation: The token file is converted to an abstract syntax tree. Compiler Passes Analysis of input program (front-end) character

### 3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino

3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the

### 10/5/17. Lexical and Syntactic Analysis. Lexical and Syntax Analysis. Tokenizing Source. Scanner. Reasons to Separate Lexical and Syntax Analysis

Lexical and Syntactic Analysis Lexical and Syntax Analysis In Text: Chapter 4 Two steps to discover the syntactic structure of a program Lexical analysis (Scanner): to read the input characters and output

### CS453 : JavaCUP and error recovery. CS453 Shift-reduce Parsing 1

CS453 : JavaCUP and error recovery CS453 Shift-reduce Parsing 1 Shift-reduce parsing in an LR parser LR(k) parser Left-to-right parse Right-most derivation K-token look ahead LR parsing algorithm using

### FROWN An LALR(k) Parser Generator

FROWN An LALR(k) Parser Generator RALF HINZE Institute of Information and Computing Sciences Utrecht University Email: ralf@cs.uu.nl Homepage: http://www.cs.uu.nl/~ralf/ September, 2001 (Pick the slides

### Compiler Passes. Syntactic Analysis. Context-free Grammars. Syntactic Analysis / Parsing. EBNF Syntax of initial MiniJava.

Syntactic Analysis Syntactic analysis, or parsing, is the second phase of compilation: The token file is converted to an abstract syntax tree. Compiler Passes Analysis of input program (front-end) character

### Context-Free Grammar. Concepts Introduced in Chapter 2. Parse Trees. Example Grammar and Derivation

Concepts Introduced in Chapter 2 A more detailed overview of the compilation process. Parsing Scanning Semantic Analysis Syntax-Directed Translation Intermediate Code Generation Context-Free Grammar A

### 3. Parsing. Oscar Nierstrasz

3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/

### CS143 Handout 20 Summer 2011 July 15 th, 2011 CS143 Practice Midterm and Solution

CS143 Handout 20 Summer 2011 July 15 th, 2011 CS143 Practice Midterm and Solution Exam Facts Format Wednesday, July 20 th from 11:00 a.m. 1:00 p.m. in Gates B01 The exam is designed to take roughly 90

### Parsing Wrapup. Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing. This lecture LR(1) parsing

Parsing Wrapup Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing LR(1) items Computing closure Computing goto LR(1) canonical collection This lecture LR(1) parsing Building ACTION

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2016-2017 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman