# Fall Compiler Principles Lecture 4: Parsing part 3. Roman Manevich Ben-Gurion University of the Negev

Size: px
Start display at page:

Download "Fall Compiler Principles Lecture 4: Parsing part 3. Roman Manevich Ben-Gurion University of the Negev"

Transcription

1 Fall Compiler Principles Lecture 4: Parsing part 3 Roman Manevich Ben-Gurion University of the Negev

2 Tentative syllabus Front End Intermediate Representation Optimizations Code Generation Scanning Operational Semantics Dataflow Analysis Register Allocation Top-down Parsing (LL) Lowering Loop Optimizations Energy Optimization Bottom-up Parsing (LR) Instruction Selection mid-term exam 2

3 Previously LR(0) parsing Running the parser Constructing transition diagram Constructing parser table Detecting conflicts SLR(0) Eliminating conflicts via FOLLOW sets 3

4 Agenda LR(1) LALR(1) Automatic LR parser generation Handling ambiguities 4

5 Going beyond SLR(0) Some common language constructs introduce conflicts even for SLR (0) S S (1) S L = R (2) S R (3) L * R (4) L id (5) R L 5

6 q0 S S S L = R S R L * R L id R L R S L q3 q1 q2 S R S S S L = R R L = q9 S L = R q6 R * * q4 id id q5 L id id S L = R R L L * R L id L * R R L L * R L id * L q8 L R L R L * R q7 6

7 shift/reduce conflict (0) S S (1) S L = R (2) S R (3) L * R (4) L id (5) R L q2 S L = R R L = q6 S L = R vs. R L FOLLOW(R) contains = S L = R * R = R SLR cannot resolve conflict S L = R R L L * R L id 7

8 Inputs requiring shift/reduce (0) S S (1) S L = R (2) S R (3) L * R (4) L id q2 (5) R L S L = R R L For the input id the rightmost derivation S S R L id requires reducing in q2 For the input id = id S S L = R L = L L = id id = id requires shifting = q6 S L = R R L L * R L id 8

9 LR(1) grammars In SLR: a reduce item N α is applicable only when the lookahead is in FOLLOW(N) But for a given context (state) are all tokens in FOLLOW(N) indeed possible? Not always We can compute a context-sensitive (i.e., specific to a given state) subset of FOLLOW(N) and use it to remove even more conflicts LR(1) keeps lookahead with each LR item Idea: a more refined notion of FOLLOW computed per item 9

10 LR(1) item Already matched To be matched Input N α β, t Hypothesis about αβ being a possible handle: so far we ve matched α, expecting to see β and after reducing N we expect to see the token t 10

11 LR(1) items LR(1) item is a pair LR(0) item Lookahead token Meaning We matched the part left of the dot, looking to match the part on the right of the dot, followed by the lookahead token Example The production L id yields the following LR(1) items LR(1) items (0) S S (1) S L = R (2) S R (3) L * R (4) L id (5) R L LR(0) items [L id] [L id ] [L id, *] [L id, =] [L id, id] [L id, \$] [L id, *] [L id, =] [L id, id] [L id, \$] 11

12 Computing Closure for LR(1) For every [A α Bβ, c] in S for every production B δ and every token b in the grammar such that b FIRST(βc) Add [B δ, b] to S 12

13 * q0 (S S, \$) (S L = R, \$) (S R, \$) (L * R, = ) (L id, = ) (R L, \$ ) (L id, \$ ) (L * R, \$ ) q4 * (L * R, =) (R L, =) (L * R, =) (L id, =) (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) id R id q5 R L S q3 (S R, \$) q1 (S S, \$) q2 (S L = R, \$) (R L, \$) (L id, \$) (L id, =) q7 L (L * R, =) (L * R, \$) q6 q9 (S L = R, \$) = R (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) q8 (R L, =) (R L, \$) L * q11 (L id, \$) id q12 (R L, \$) q13 q10 (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) R (L * R, \$) id 13

14 Back to the conflict q2 (S L = R, \$) (R L, \$) = q6 (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) Is there a conflict now? 14

15 LALR(1) LR(1) tables have huge number of entries Often don t need such refined observation (and cost) Idea: find states with the same LR(0) component and merge their lookaheads component as long as there are no conflicts LALR(1) not as powerful as LR(1) in theory but works quite well in practice Merging may not introduce new shift-reduce conflicts, only reduce-reduce, which is unlikely in practice 15

16 * q0 (S S, \$) (S L = R, \$) (S R, \$) (L * R, = ) (L id, = ) (R L, \$ ) (L id, \$ ) (L * R, \$ ) q4 * (L * R, =) (R L, =) (L * R, =) (L id, =) (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) id R id q5 R L S q3 (S R, \$) q1 (S S, \$) q2 (S L = R, \$) (R L, \$) (L id, \$) (L id, =) q7 L (L * R, =) (L * R, \$) q6 q9 (S L = R, \$) = R (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) q8 (R L, =) (R L, \$) L * q11 (L id, \$) id q12 (R L, \$) q13 q10 (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) R (L * R, \$) id 16

17 * q0 (S S, \$) (S L = R, \$) (S R, \$) (L * R, = ) (L id, = ) (R L, \$ ) (L id, \$ ) (L * R, \$ ) q4 * (L * R, =) (R L, =) (L * R, =) (L id, =) (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) id R id q5 R L S q3 (S R, \$) q1 (S S, \$) q2 (S L = R, \$) (R L, \$) (L id, \$) (L id, =) q7 L (L * R, =) (L * R, \$) q6 q9 (S L = R, \$) = R (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) q8 (R L, =) (R L, \$) L * q11 (L id, \$) id q12 (R L, \$) q13 q10 (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) R (L * R, \$) id 17

18 * q0 (S S, \$) (S L = R, \$) (S R, \$) (L * R, = ) (L id, = ) (R L, \$ ) (L id, \$ ) (L * R, \$ ) q4 * (L * R, =) (R L, =) (L * R, =) (L id, =) (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) id R id q5 R L S q3 (S R, \$) q1 (S S, \$) q2 (S L = R, \$) (R L, \$) (L id, \$) (L id, =) q7 L id (L * R, =) (L * R, \$) q6 q9 (S L = R, \$) = R (S L = R, \$) (R L, \$) (L * R, \$) (L id, \$) q8 (R L, =) (R L, \$) L * q10 (L * R, \$) (R L, \$) (L * R, \$) (L id, \$) R id 18

19 Left/Right- recursion At home: create a simple grammar with left-recursion and one with right-recursion Construct corresponding LR(0) parser Any conflicts? Run on simple input and observe behavior Attempt to generalize observation for long inputs 19

20 Example: non-lr(1) grammar (1) S Y b c \$ (2) S Z b d \$ (3) Y a (4) Z a S Y b c, \$ S Y b c, \$ Y a, b Z a, b a Y a, b Z a, b reduce-reduce conflict on lookahead b 20

21 Automated parser generation (via CUP) 21

22 High-level structure text Lexer spec JFlex.java javac Lexical analyzer LANG.lex Lexer.java tokens (Token.java) Parser spec CUP.java javac Parser LANG.cup Parser.java sym.java AST 22

23 Expression calculator expr expr + expr expr - expr expr * expr expr / expr - expr ( expr ) number Goals of expression calculator parser: Is a valid expression? What is the meaning (value) of this expression? 23

24 Syntax analysis with CUP CUP parser generator Generates an LALR(1) Parser Input: spec file Output: a syntax analyzer Can dump automaton and table tokens Parser spec CUP.java javac Parser AST 24

25 CUP spec file Package and import specifications User code components Symbol (terminal and non-terminal) lists Terminals go to sym.java Types of AST nodes Precedence declarations The grammar Semantic actions to construct AST 25

26 Parsing ambiguous grammars 26

27 Expression Calculator 1 st Attempt terminal Integer NUMBER; terminal PLUS, MINUS, MULT, DIV; terminal LPAREN, RPAREN; non terminal Integer expr; Symbol type explained later expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr LPAREN expr RPAREN NUMBER ; 27

28 Ambiguities + * * a b c + a + b * c + a b c + + a b c a + b + c + a b c 28

29 Ambiguities as conflicts for LR(1) + * + * a b c a + b * c + a b c + + a b c a + b + c + a b c 29

30 terminal Integer NUMBER; terminal PLUS,MINUS,MULT,DIV; terminal LPAREN, RPAREN; terminal UMINUS; non terminal Integer expr; Expression Calculator 2 nd Attempt precedence left PLUS, MINUS; precedence left DIV, MULT; precedence left UMINUS; Increasing precedence expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr %prec UMINUS LPAREN expr RPAREN NUMBER ; Contextual precedence 30

31 Parsing ambiguous grammars using precedence declarations Each terminal assigned with precedence By default all terminals have lowest precedence User can assign his own precedence CUP assigns each production a precedence Precedence of rightmost terminal in production or user-specified contextual precedence On shift/reduce conflict resolve ambiguity by comparing precedence of terminal and production and decides whether to shift or reduce In case of equal precedences left/right help resolve conflicts left means reduce right means shift More information on precedence declarations in CUP s manual 31

32 Resolving ambiguity (associativity) precedence left PLUS a b c + a b c a + b + c 32

33 Resolving ambiguity (op. precedence) precedence left PLUS precedence left MULT + * * a b c + a b c a + b * c 33

34 Resolving ambiguity (contextual) precedence left MULT MINUS expr %prec UMINUS * - - a b a * b - a * b 34

35 Resolving ambiguity terminal Integer NUMBER; terminal PLUS,MINUS,MULT,DIV; terminal LPAREN, RPAREN; terminal UMINUS; UMINUS never returned by scanner (used only to define precedence) precedence left PLUS, MINUS; precedence left DIV, MULT; precedence left UMINUS; expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr %prec UMINUS LPAREN expr RPAREN NUMBER ; Rule has precedence of UMINUS 35

36 More CUP directives precedence nonassoc NEQ Non-associative operators: < > ==!= etc. 1<2<3 identified as an error (semantic error?) start non-terminal Specifies start non-terminal other than first non-terminal Can change to test parts of grammar Getting internal representation Command line options: -dump_grammar -dump_states -dump_tables -dump 36

37 Scanner integration import java_cup.runtime.*; %% %cup Generated from token %eofval{ declarations in.cup file return new Symbol(sym.EOF); %eofval} NUMBER=[0-9]+ %% <YYINITIAL> + { return new Symbol(sym.PLUS); } <YYINITIAL> - { return new Symbol(sym.MINUS); } <YYINITIAL> * { return new Symbol(sym.MULT); } <YYINITIAL> / { return new Symbol(sym.DIV); } <YYINITIAL> ( { return new Symbol(sym.LPAREN); } <YYINITIAL> ) { return new Symbol(sym.RPAREN); } <YYINITIAL>{NUMBER} { return new Symbol(sym.NUMBER, new Integer(yytext())); } <YYINITIAL>\n { } <YYINITIAL>. { } Parser gets terminals from the scanner 37

38 Recap Package and import specifications and user code components Symbol (terminal and non-terminal) lists Define building-blocks of the grammar Precedence declarations May help resolve conflicts The grammar May introduce conflicts that have to be resolved 38

39 Abstract syntax tree construction 39

40 Assigning meaning expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr %prec UMINUS LPAREN expr RPAREN NUMBER ; So far, only validation Add Java code implementing semantic actions 40

41 non terminal Integer expr; Assigning meaning expr ::= expr:e1 PLUS expr:e2 {: RESULT = new Integer(e1.intValue() + e2.intvalue()); :} expr:e1 MINUS expr:e2 {: RESULT = new Integer(e1.intValue() - e2.intvalue()); :} expr:e1 MULT expr:e2 {: RESULT = new Integer(e1.intValue() * e2.intvalue()); :} expr:e1 DIV expr:e2 {: RESULT = new Integer(e1.intValue() / e2.intvalue()); :} MINUS expr:e1 {: RESULT = new Integer(0 - e1.intvalue(); :} %prec UMINUS LPAREN expr:e1 RPAREN {: RESULT = e1; :} NUMBER:n {: RESULT = n; :} ; Symbol labels used to name variables RESULT names the left-hand side symbol 41

42 Abstract Syntax Trees More useful representation of syntax tree Less clutter Actual level of detail depends on your design Basis for semantic analysis Later annotated with various information Type information Computed values Technically a class hierarchy of abstract syntax tree nodes 42

43 Parse tree vs. AST expr + expr + expr expr expr 1 + ( 2 ) + ( 3 )

44 AST hierarchy example expr int_const plus minus times divide 44

45 AST construction AST Nodes constructed during parsing Stored in push-down stack Bottom-up parser Grammar rules annotated with actions for AST construction When node is constructed all children available (already constructed) Node (RESULT) pushed on stack 45

46 AST construction 1 + (2) + (3) expr + (2) + (3) expr + (expr) + (3) expr + (3) expr + (expr) expr expr expr ::= expr:e1 PLUS expr:e2 {: RESULT = new plus(e1,e2); :} LPAREN expr:e RPAREN {: RESULT = e; :} INT_CONST:i {: RESULT = new int_const(, i); :} plus e1 e2 expr plus e1 e2 expr expr expr 1 + ( 2 ) + ( 3 ) int_const val = 1 int_const val = 2 int_const val = 3 46

47 Example of lists terminal Integer NUMBER; terminal PLUS,MINUS,MULT,DIV,LPAREN,RPAREN,SEMI; terminal UMINUS; non terminal Integer expr; non terminal expr_list, expr_part; precedence left PLUS, MINUS; precedence left DIV, MULT; precedence left UMINUS; Executed when e is shifted expr_list ::= expr_list expr_part expr_part ; expr_part ::= expr:e {: System.out.println("= " + e); :} SEMI ; expr ::= expr PLUS expr expr MINUS expr expr MULT expr expr DIV expr MINUS expr %prec UMINUS LPAREN expr RPAREN NUMBER ; 47

48 Next lecture: IR and Operational Semantics

### Fall Compiler Principles Lecture 5: Parsing part 4. Roman Manevich Ben-Gurion University

Fall 2014-2015 Compiler Principles Lecture 5: Parsing part 4 Roman Manevich Ben-Gurion University Tentative syllabus Front End Intermediate Representation Optimizations Code Generation Scanning Lowering

### Agenda. Previously. Tentative syllabus. Fall Compiler Principles Lecture 5: Parsing part 4 12/2/2015. Roman Manevich Ben-Gurion University

Fall 2015-2016 Compiler Principles ecture 5: Parsing part 4 Tentative syllabus Front End Intermediate epresentation Optimizations Code Generation Scanning Operational Semantics Dataflow Analysis egister

### Compilers. Bottom-up Parsing. (original slides by Sam

Compilers Bottom-up Parsing Yannis Smaragdakis U Athens Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Bottom-Up Parsing More general than top-down parsing And just as efficient Builds

### COMP 181. Prelude. Prelude. Summary of parsing. A Hierarchy of Grammar Classes. More power? Syntax-directed translation. Analysis

Prelude COMP 8 October, 9 What is triskaidekaphobia? Fear of the number s? No aisle in airplanes, no th floor in buildings Fear of Friday the th? Paraskevidedekatriaphobia or friggatriskaidekaphobia Why

### Compilers and Language Processing Tools

Compilers and Language Processing Tools Summer Term 2011 Prof. Dr. Arnd Poetzsch-Heffter Software Technology Group TU Kaiserslautern c Prof. Dr. Arnd Poetzsch-Heffter 1 Parser Generators c Prof. Dr. Arnd

### Last Time. What do we want? When do we want it? An AST. Now!

Java CUP 1 Last Time What do we want? An AST When do we want it? Now! 2 This Time A little review of ASTs The philosophy and use of a Parser Generator 3 Translating Lists CFG IdList -> id IdList comma

### Context-free grammars

Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,

### Abstract Syntax. Mooly Sagiv. html://www.cs.tau.ac.il/~msagiv/courses/wcc06.html

Abstract Syntax Mooly Sagiv html://www.cs.tau.ac.il/~msagiv/courses/wcc06.html Outline The general idea Cup Motivating example Interpreter for arithmetic expressions The need for abstract syntax Abstract

### EDAN65: Compilers, Lecture 06 A LR parsing. Görel Hedin Revised:

EDAN65: Compilers, Lecture 06 A LR parsing Görel Hedin Revised: 2017-09-11 This lecture Regular expressions Context-free grammar Attribute grammar Lexical analyzer (scanner) Syntactic analyzer (parser)

### Compiler Construction: Parsing

Compiler Construction: Parsing Mandar Mitra Indian Statistical Institute M. Mitra (ISI) Parsing 1 / 33 Context-free grammars. Reference: Section 4.2 Formal way of specifying rules about the structure/syntax

### Simple LR (SLR) LR(0) Drawbacks LR(1) SLR Parse. LR(1) Start State and Reduce. LR(1) Items 10/3/2012

LR(0) Drawbacks Consider the unambiguous augmented grammar: 0.) S E \$ 1.) E T + E 2.) E T 3.) T x If we build the LR(0) DFA table, we find that there is a shift-reduce conflict. This arises because the

### Using JFlex. "Linux Gazette...making Linux just a little more fun!" by Christopher Lopes, student at Eastern Washington University April 26, 1999

"Linux Gazette...making Linux just a little more fun!" by Christopher Lopes, student at Eastern Washington University April 26, 1999 This is the third part of a series begun in the April 1999 issue of

### In One Slide. Outline. LR Parsing. Table Construction

LR Parsing Table Construction #1 In One Slide An LR(1) parsing table can be constructed automatically from a CFG. An LR(1) item is a pair made up of a production and a lookahead token; it represents a

### Conflicts in LR Parsing and More LR Parsing Types

Conflicts in LR Parsing and More LR Parsing Types Lecture 10 Dr. Sean Peisert ECS 142 Spring 2009 1 Status Project 2 Due Friday, Apr. 24, 11:55pm The usual lecture time is being replaced by a discussion

### Compilation 2013 Parser Generators, Conflict Management, and ML-Yacc

Compilation 2013 Parser Generators, Conflict Management, and ML-Yacc Erik Ernst Aarhus University Parser generators, ML-Yacc LR parsers are tedious to write, but can be generated, e.g., by ML-Yacc Input:

### CUP. Lecture 18 CUP User s Manual (online) Robb T. Koether. Hampden-Sydney College. Fri, Feb 27, 2015

CUP Lecture 18 CUP User s Manual (online) Robb T. Koether Hampden-Sydney College Fri, Feb 27, 2015 Robb T. Koether (Hampden-Sydney College) CUP Fri, Feb 27, 2015 1 / 31 1 The CUP Parser Generator 2 The

### JavaCUP. There are also many parser generators written in Java

JavaCUP JavaCUP (Construct Useful Parser) is a parser generator Produce a parser written in java, itself is also written in Java; There are many parser generators. YACC (Yet Another Compiler-Compiler)

### For example, we might have productions such as:

1. Error Recovery This is an extract from http://www.cs.pr inceton.edu/ appel/moder n/java/cup/manual.html#errors A impor tant aspect of building parsers with CUP is support for syntactic error recovery.

### Bottom-Up Parsing. Lecture 11-12

Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 9/22/06 Prof. Hilfinger CS164 Lecture 11 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient

### Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery. Last modified: Mon Feb 23 10:05: CS164: Lecture #14 1

Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery Last modified: Mon Feb 23 10:05:56 2015 CS164: Lecture #14 1 Shift/Reduce Conflicts If a DFA state contains both [X: α aβ, b] and [Y: γ, a],

### MIT Parse Table Construction. Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology

MIT 6.035 Parse Table Construction Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Parse Tables (Review) ACTION Goto State ( ) \$ X s0 shift to s2 error error goto s1

### Lecture 8: Deterministic Bottom-Up Parsing

Lecture 8: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Fri Feb 12 13:02:57 2010 CS164: Lecture #8 1 Avoiding nondeterministic choice: LR We ve been looking at general

### Let us construct the LR(1) items for the grammar given below to construct the LALR parsing table.

MODULE 18 LALR parsing After understanding the most powerful CALR parser, in this module we will learn to construct the LALR parser. The CALR parser has a large set of items and hence the LALR parser is

### A clarification on terminology: Recognizer: accepts or rejects strings in a language. Parser: recognizes and generates parse trees (imminent topic)

A clarification on terminology: Recognizer: accepts or rejects strings in a language Parser: recognizes and generates parse trees (imminent topic) Assignment 3: building a recognizer for the Lake expression

### LR Parsing - The Items

LR Parsing - The Items Lecture 10 Sections 4.5, 4.7 Robb T. Koether Hampden-Sydney College Fri, Feb 13, 2015 Robb T. Koether (Hampden-Sydney College) LR Parsing - The Items Fri, Feb 13, 2015 1 / 31 1 LR

### Lecture 7: Deterministic Bottom-Up Parsing

Lecture 7: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Tue Sep 20 12:50:42 2011 CS164: Lecture #7 1 Avoiding nondeterministic choice: LR We ve been looking at general

### Bottom-Up Parsing. Lecture 11-12

Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 2/20/08 Prof. Hilfinger CS164 Lecture 11 1 Administrivia Test I during class on 10 March. 2/20/08 Prof. Hilfinger CS164 Lecture 11

### Lecture Bottom-Up Parsing

Lecture 14+15 Bottom-Up Parsing CS 241: Foundations of Sequential Programs Winter 2018 Troy Vasiga et al University of Waterloo 1 Example CFG 1. S S 2. S AyB 3. A ab 4. A cd 5. B z 6. B wz 2 Stacks in

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2017-2018 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman

### LR Parsing LALR Parser Generators

LR Parsing LALR Parser Generators Outline Review of bottom-up parsing Computing the parsing DFA Using parser generators 2 Bottom-up Parsing (Review) A bottom-up parser rewrites the input string to the

### Lexical and Syntax Analysis. Bottom-Up Parsing

Lexical and Syntax Analysis Bottom-Up Parsing Parsing There are two ways to construct derivation of a grammar. Top-Down: begin with start symbol; repeatedly replace an instance of a production s LHS with

### Parsing. Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1)

TD parsing - LL(1) Parsing First and Follow sets Parse table construction BU Parsing Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1) Problems with SLR Aho, Sethi, Ullman, Compilers

### LR Parsing LALR Parser Generators

Outline LR Parsing LALR Parser Generators Review of bottom-up parsing Computing the parsing DFA Using parser generators 2 Bottom-up Parsing (Review) A bottom-up parser rewrites the input string to the

### Topic 5: Syntax Analysis III

Topic 5: Syntax Analysis III Compiler Design Prof. Hanjun Kim CoreLab (Compiler Research Lab) POSTECH 1 Back-End Front-End The Front End Source Program Lexical Analysis Syntax Analysis Semantic Analysis

### LR Parsing. Table Construction

#1 LR Parsing Table Construction #2 Outline Review of bottom-up parsing Computing the parsing DFA Closures, LR(1) Items, States Transitions Using parser generators Handling Conflicts #3 In One Slide An

### CSE 401 Midterm Exam Sample Solution 11/4/11

Question 1. (12 points, 2 each) The front end of a compiler consists of three parts: scanner, parser, and (static) semantics. Collectively these need to analyze the input program and decide if it is correctly

### Formal Languages and Compilers Lecture VII Part 3: Syntactic A

Formal Languages and Compilers Lecture VII Part 3: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

### CS606- compiler instruction Solved MCQS From Midterm Papers

CS606- compiler instruction Solved MCQS From Midterm Papers March 06,2014 MC100401285 Moaaz.pk@gmail.com Mc100401285@gmail.com PSMD01 Final Term MCQ s and Quizzes CS606- compiler instruction If X is a

### Topic 3: Syntax Analysis I

Topic 3: Syntax Analysis I Compiler Design Prof. Hanjun Kim CoreLab (Compiler Research Lab) POSTECH 1 Back-End Front-End The Front End Source Program Lexical Analysis Syntax Analysis Semantic Analysis

### SLR parsers. LR(0) items

SLR parsers LR(0) items As we have seen, in order to make shift-reduce parsing practical, we need a reasonable way to identify viable prefixes (and so, possible handles). Up to now, it has not been clear

### shift-reduce parsing

Parsing #2 Bottom-up Parsing Rightmost derivations; use of rules from right to left Uses a stack to push symbols the concatenation of the stack symbols with the rest of the input forms a valid bottom-up

### 10/4/18. Lexical and Syntactic Analysis. Lexical and Syntax Analysis. Tokenizing Source. Scanner. Reasons to Separate Lexical and Syntactic Analysis

Lexical and Syntactic Analysis Lexical and Syntax Analysis In Text: Chapter 4 Two steps to discover the syntactic structure of a program Lexical analysis (Scanner): to read the input characters and output

### Lexical Analysis. Textbook:Modern Compiler Design Chapter 2.1

Lexical Analysis Textbook:Modern Compiler Design Chapter 2.1 A motivating example Create a program that counts the number of lines in a given input text file Solution (Flex) int num_lines = 0; %% \n ++num_lines;.

### S Y N T A X A N A L Y S I S LR

LR parsing There are three commonly used algorithms to build tables for an LR parser: 1. SLR(1) = LR(0) plus use of FOLLOW set to select between actions smallest class of grammars smallest tables (number

### Syntax-Directed Translation. Lecture 14

Syntax-Directed Translation Lecture 14 (adapted from slides by R. Bodik) 9/27/2006 Prof. Hilfinger, Lecture 14 1 Motivation: parser as a translator syntax-directed translation stream of tokens parser ASTs,

### Monday, September 13, Parsers

Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions

### Formal Languages and Compilers Lecture VII Part 4: Syntactic A

Formal Languages and Compilers Lecture VII Part 4: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

### How do LL(1) Parsers Build Syntax Trees?

How do LL(1) Parsers Build Syntax Trees? So far our LL(1) parser has acted like a recognizer. It verifies that input token are syntactically correct, but it produces no output. Building complete (concrete)

### CS 11 Ocaml track: lecture 6

CS 11 Ocaml track: lecture 6 n Today: n Writing a computer language n Parser generators n lexers (ocamllex) n parsers (ocamlyacc) n Abstract syntax trees Problem (1) n We want to implement a computer language

### Wednesday, September 9, 15. Parsers

Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Parsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:

What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Syntax Analysis. Amitabha Sanyal. (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay

Syntax Analysis (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay September 2007 College of Engineering, Pune Syntax Analysis: 2/124 Syntax

### PART 3 - SYNTAX ANALYSIS. F. Wotawa TU Graz) Compiler Construction Summer term / 309

PART 3 - SYNTAX ANALYSIS F. Wotawa (IST @ TU Graz) Compiler Construction Summer term 2016 64 / 309 Goals Definition of the syntax of a programming language using context free grammars Methods for parsing

### Fall Compiler Principles Lecture 3: Parsing part 2. Roman Manevich Ben-Gurion University

Fall 2014-2015 Compiler Principles Lecture 3: Parsing part 2 Roman Manevich Ben-Gurion University Tentative syllabus Front End Intermediate Representation Optimizations Code Generation Scanning Lowering

### CSE302: Compiler Design

CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University March 20, 2007 Outline Recap LR(0)

### RYERSON POLYTECHNIC UNIVERSITY DEPARTMENT OF MATH, PHYSICS, AND COMPUTER SCIENCE CPS 710 FINAL EXAM FALL 96 INSTRUCTIONS

RYERSON POLYTECHNIC UNIVERSITY DEPARTMENT OF MATH, PHYSICS, AND COMPUTER SCIENCE CPS 710 FINAL EXAM FALL 96 STUDENT ID: INSTRUCTIONS Please write your student ID on this page. Do not write it or your name

### 10/5/17. Lexical and Syntactic Analysis. Lexical and Syntax Analysis. Tokenizing Source. Scanner. Reasons to Separate Lexical and Syntax Analysis

Lexical and Syntactic Analysis Lexical and Syntax Analysis In Text: Chapter 4 Two steps to discover the syntactic structure of a program Lexical analysis (Scanner): to read the input characters and output

### Parsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468

Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure

### Principle of Compilers Lecture IV Part 4: Syntactic Analysis. Alessandro Artale

Free University of Bolzano Principles of Compilers Lecture IV Part 4, 2003/2004 AArtale (1) Principle of Compilers Lecture IV Part 4: Syntactic Analysis Alessandro Artale Faculty of Computer Science Free

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2016-2017 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman

### Wednesday, August 31, Parsers

Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically

### FROWN An LALR(k) Parser Generator

FROWN An LALR(k) Parser Generator RALF HINZE Institute of Information and Computing Sciences Utrecht University Email: ralf@cs.uu.nl Homepage: http://www.cs.uu.nl/~ralf/ September, 2001 (Pick the slides

### Programming Languages (CS 550) Lecture 4 Summary Scanner and Parser Generators. Jeremy R. Johnson

Programming Languages (CS 550) Lecture 4 Summary Scanner and Parser Generators Jeremy R. Johnson 1 Theme We have now seen how to describe syntax using regular expressions and grammars and how to create

### CSE P 501 Compilers. LR Parsing Hal Perkins Spring UW CSE P 501 Spring 2018 D-1

CSE P 501 Compilers LR Parsing Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts UW CSE P 501 Spring 2018

### Compiler Construction

Compiler Construction Thomas Noll Software Modeling and Verification Group RWTH Aachen University https://moves.rwth-aachen.de/teaching/ss-17/cc/ Recap: LR(1) Parsing Outline of Lecture 11 Recap: LR(1)

### Properties of Regular Expressions and Finite Automata

Properties of Regular Expressions and Finite Automata Some token patterns can t be defined as regular expressions or finite automata. Consider the set of balanced brackets of the form [[[ ]]]. This set

### CSE 401 Compilers. LR Parsing Hal Perkins Autumn /10/ Hal Perkins & UW CSE D-1

CSE 401 Compilers LR Parsing Hal Perkins Autumn 2011 10/10/2011 2002-11 Hal Perkins & UW CSE D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts 10/10/2011

### Fall Compiler Principles Lecture 6: Intermediate Representation. Roman Manevich Ben-Gurion University of the Negev

Fall 2015-2016 Compiler Principles Lecture 6: Intermediate Representation Roman Manevich Ben-Gurion University of the Negev Tentative syllabus Front End Intermediate Representation Optimizations Code Generation

### Review: Shift-Reduce Parsing. Bottom-up parsing uses two actions: Bottom-Up Parsing II. Shift ABC xyz ABCx yz. Lecture 8. Reduce Cbxy ijk CbA ijk

Review: Shift-Reduce Parsing Bottom-up parsing uses two actions: Bottom-Up Parsing II Lecture 8 Shift ABC xyz ABCx yz Reduce Cbxy ijk CbA ijk Prof. Aiken CS 13 Lecture 8 1 Prof. Aiken CS 13 Lecture 8 2

### CSE302: Compiler Design

CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University February 01, 2007 Outline Recap

### Compiler Construction

Compiler Construction Thomas Noll Software Modeling and Verification Group RWTH Aachen University https://moves.rwth-aachen.de/teaching/ss-17/cc/ Recap: LR(1) Parsing LR(1) Items and Sets Observation:

### Review of CFGs and Parsing II Bottom-up Parsers. Lecture 5. Review slides 1

Review of CFGs and Parsing II Bottom-up Parsers Lecture 5 1 Outline Parser Overview op-down Parsers (Covered largely through labs) Bottom-up Parsers 2 he Functionality of the Parser Input: sequence of

### LR Parsers. Aditi Raste, CCOEW

LR Parsers Aditi Raste, CCOEW 1 LR Parsers Most powerful shift-reduce parsers and yet efficient. LR(k) parsing L : left to right scanning of input R : constructing rightmost derivation in reverse k : number

### Last time. What are compilers? Phases of a compiler. Scanner. Parser. Semantic Routines. Optimizer. Code Generation. Sunday, August 29, 2010

Last time Source code Scanner Tokens Parser What are compilers? Phases of a compiler Syntax tree Semantic Routines IR Optimizer IR Code Generation Executable Extra: Front-end vs. Back-end Scanner + Parser

### Lexical Analysis. Chapter 1, Section Chapter 3, Section 3.1, 3.3, 3.4, 3.5 JFlex Manual

Lexical Analysis Chapter 1, Section 1.2.1 Chapter 3, Section 3.1, 3.3, 3.4, 3.5 JFlex Manual Inside the Compiler: Front End Lexical analyzer (aka scanner) Converts ASCII or Unicode to a stream of tokens

### COMP-421 Compiler Design. Presented by Dr Ioanna Dionysiou

COMP-421 Compiler Design Presented by Dr Ioanna Dionysiou Administrative! Any questions about the syllabus?! Course Material available at www.cs.unic.ac.cy/ioanna! Next time reading assignment [ALSU07]

### Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

### Lexical Analysis. Textbook:Modern Compiler Design Chapter 2.1.

Lexical Analysis Textbook:Modern Compiler Design Chapter 2.1 http://www.cs.tau.ac.il/~msagiv/courses/wcc11-12.html 1 A motivating example Create a program that counts the number of lines in a given input

### Lexical and Syntax Analysis

Lexical and Syntax Analysis In Text: Chapter 4 N. Meng, F. Poursardar Lexical and Syntactic Analysis Two steps to discover the syntactic structure of a program Lexical analysis (Scanner): to read the input

### Example CFG. Lectures 16 & 17 Bottom-Up Parsing. LL(1) Predictor Table Review. Stacks in LR Parsing 1. Sʹ " S. 2. S " AyB. 3. A " ab. 4.

Example CFG Lectures 16 & 17 Bottom-Up Parsing CS 241: Foundations of Sequential Programs Fall 2016 1. Sʹ " S 2. S " AyB 3. A " ab 4. A " cd Matt Crane University of Waterloo 5. B " z 6. B " wz 2 LL(1)

### CSE 401 Midterm Exam 11/5/10

Name There are 5 questions worth a total of 100 points. Please budget your time so you get to all of the questions. Keep your answers brief and to the point. The exam is closed books, closed notes, closed

### Fall Compiler Principles Lecture 5: Intermediate Representation. Roman Manevich Ben-Gurion University of the Negev

Fall 2016-2017 Compiler Principles Lecture 5: Intermediate Representation Roman Manevich Ben-Gurion University of the Negev Tentative syllabus Front End Intermediate Representation Optimizations Code Generation

### Context-Free Grammars

Context-Free Grammars Lecture 7 http://webwitch.dreamhost.com/grammar.girl/ Outline Scanner vs. parser Why regular expressions are not enough Grammars (context-free grammars) grammar rules derivations

### A left-sentential form is a sentential form that occurs in the leftmost derivation of some sentence.

Bottom-up parsing Recall For a grammar G, with start symbol S, any string α such that S α is a sentential form If α V t, then α is a sentence in L(G) A left-sentential form is a sentential form that occurs

### Syntax and Parsing COMS W4115. Prof. Stephen A. Edwards Fall 2003 Columbia University Department of Computer Science

Syntax and Parsing COMS W4115 Prof. Stephen A. Edwards Fall 2003 Columbia University Department of Computer Science Lexical Analysis (Scanning) Lexical Analysis (Scanning) Goal is to translate a stream

SYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic

### 3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino

3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the

### Syntax-Directed Translation

Syntax-Directed Translation ALSU Textbook Chapter 5.1 5.4, 4.8, 4.9 Tsan-sheng Hsu tshsu@iis.sinica.edu.tw http://www.iis.sinica.edu.tw/~tshsu 1 What is syntax-directed translation? Definition: The compilation

### Context-Free Grammar. Concepts Introduced in Chapter 2. Parse Trees. Example Grammar and Derivation

Concepts Introduced in Chapter 2 A more detailed overview of the compilation process. Parsing Scanning Semantic Analysis Syntax-Directed Translation Intermediate Code Generation Context-Free Grammar A

### CS143 Handout 20 Summer 2011 July 15 th, 2011 CS143 Practice Midterm and Solution

CS143 Handout 20 Summer 2011 July 15 th, 2011 CS143 Practice Midterm and Solution Exam Facts Format Wednesday, July 20 th from 11:00 a.m. 1:00 p.m. in Gates B01 The exam is designed to take roughly 90

### Concepts Introduced in Chapter 4

Concepts Introduced in Chapter 4 Grammars Context-Free Grammars Derivations and Parse Trees Ambiguity, Precedence, and Associativity Top Down Parsing Recursive Descent, LL Bottom Up Parsing SLR, LR, LALR

### CS 4120 Introduction to Compilers

CS 4120 Introduction to Compilers Andrew Myers Cornell University Lecture 6: Bottom-Up Parsing 9/9/09 Bottom-up parsing A more powerful parsing technology LR grammars -- more expressive than LL can handle

### Bottom-up parsing. Bottom-Up Parsing. Recall. Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form

Bottom-up parsing Bottom-up parsing Recall Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form If α V t,thenα is called a sentence in L(G) Otherwise it is just

### A simple syntax-directed

Syntax-directed is a grammaroriented compiling technique Programming languages: Syntax: what its programs look like? Semantic: what its programs mean? 1 A simple syntax-directed Lexical Syntax Character

### Lecture Notes on Bottom-Up LR Parsing

Lecture Notes on Bottom-Up LR Parsing 15-411: Compiler Design Frank Pfenning Lecture 9 September 23, 2009 1 Introduction In this lecture we discuss a second parsing algorithm that traverses the input string

### CS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F).

CS 2210 Sample Midterm 1. Determine if each of the following claims is true (T) or false (F). F A language consists of a set of strings, its grammar structure, and a set of operations. (Note: a language

### Compiler Design 1. Bottom-UP Parsing. Goutam Biswas. Lect 6

Compiler Design 1 Bottom-UP Parsing Compiler Design 2 The Process The parse tree is built starting from the leaf nodes labeled by the terminals (tokens). The parser tries to discover appropriate reductions,

### CS 4240: Compilers and Interpreters Project Phase 1: Scanner and Parser Due Date: October 4 th 2015 (11:59 pm) (via T-square)

CS 4240: Compilers and Interpreters Project Phase 1: Scanner and Parser Due Date: October 4 th 2015 (11:59 pm) (via T-square) Introduction This semester, through a project split into 3 phases, we are going

### Bottom-Up Parsing II (Different types of Shift-Reduce Conflicts) Lecture 10. Prof. Aiken (Modified by Professor Vijay Ganesh.

Bottom-Up Parsing II Different types of Shift-Reduce Conflicts) Lecture 10 Ganesh. Lecture 10) 1 Review: Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient Doesn

### 3. Parsing. Oscar Nierstrasz

3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/