# Lecture Notes on Bottom-Up LR Parsing

Size: px
Start display at page:

Transcription

1 Lecture Notes on Bottom-Up LR Parsing : Compiler Design Frank Pfenning Lecture 9 1 Introduction In this lecture we discuss a second parsing algorithm that traverses the input string from left to right. The parsing algorithm LL(1) from the last lecture makes a decision on which grammar production to use based on the first character of the input string. Now we consider LR(1), which can postpone the decision at first by pushing input characters onto a stack and then deciding on the production later, taking into account both the first input character and the stack. There are several LL parser generator tools and several LR parser generator tools. Most hand-written parsers are recursive descent parsers, which follow the LL(1) principles. Alternative presentations of the material in this lecture can be found in a paper by Shieber et al. [SSP95]. The textbook [App98, Chapter 3] covers parsing. For more detailed background on parsing, also see [WM95]. 2 LR(1) Parsing One difficulty with LL(1) parsing is that it is often difficult or impossible to rewrite a grammar so that 1 token look-ahead during a left-to-right traversal becomes unambiguous. To illustrate this, we return to the earlier exam- With edits by André Platzer

2 L9.2 Bottom-Up LR Parsing ple of simple arithmetic expressions. [plus] E E + E [times] E E * E [ident] E id [number] E num [parens] E ( E ) If we see a simple expression such as * 5 (which becomes the token stream num + num * num), we cannot predict when we see the + symbol which production to use because of the inherent ambiguity of the grammar. We can rewrite the grammar, at significant expense of readability, or we could just specify that multiplication has higher precedence than addition, + < *. Obviously, the latter is more convenient, but how can we make it work? The idea is to put off the decision on which productions to use and just shift the input symbols onto a stack until we can finally make the decision! We write γ w parse input w under stack γ where, as generally in predictive parsing, the rules are interpreted as transitions from the conclusion to the premises. The parsing attempt succeeds if we can consume all of w and and produce the start symbol S on the left-hand side. That is, the deduction representing a successful parse of terminal string w 0 has the form S ɛ R 1. ɛ w 0 Parsing is defined by the following rules: S ɛ R 1 γ a w γ a w R 2(= shift) [r]x β γ X w γ β w R 3 (r)(= reduce(r))

3 Bottom-Up LR Parsing L9.3 We resume the example above, parsing num + num * num. After one step (reading this bottom-up) num + num * num? ɛ num + num * num shift we already have to make a decision: should we shift + or should we reduce num using rule [number]. In this case the action to reduce is forced, because we will never get another chance to see this num as an E. E + num * num? num + num * num reduce(number) ɛ num + num * num shift At this point we need to shift +; no other action is possible. We take a few steps and arrive at E + E * num E +num * num reduce(number) E + num * num shift E + num * num shift num + num * num reduce(number) ɛ num + num * num shift At this point, we have a real conflict. We can either reduce, viewing E + E as a subexpression, or shift and later consider E * E as a subexpression. Since the * has higher precedence than +, we need to shift. E ɛ R 1 E + E ɛ reduce(plus) E + E * E ɛ reduce(times) E + E * num ɛ reduce(number) E + E * num shift E + E * num shift E +num * num reduce(number) E + num * num shift E + num * num shift num + num * num reduce(number) ɛ num + num * num shift Since E was the start symbol in this example, this concludes the deduction. If we now read the lines from the top to the bottom, ignoring the separator,

4 L9.4 Bottom-Up LR Parsing we see that it represents a rightmost derivation of the input string. So we have parsed analyzing the string from left to right, constructing a rightmost derivation. This type of parsing algorithms is called LR-parsing, where the L stands for reading the input from left-to-right and the R stands for producing the rightmost derivation. The decisions above are based on the suffix (i.e., top) of the stack on the left-hand side and the first token lookahead on the right-hand side. Here, the suffix of the stack on the left-hand side must be a prefix substring of a grammar production. If not, it would be impossible to complete it in such a way that a future grammar production can be applied in a reduction step: the parse attempt is doomed to failure. 3 LR(1) Parsing Tables We could now define again a slightly different version of follow(γ, a), where γ is a prefix substring of the grammar or a non-terminal, and then specialize the rules. An alternative, often used to describe parser generators, is to construct a parsing table. For an LR(1) grammar, this table contains an entry for every prefix substring of the grammar and every token seen on the input. An entry describes whether to shift, reduce (and by which rule), or to signal an error. If the action is ambiguous, the given grammar is not LR(1), and either an error message is issued, or some default rule comes into effect that choses between the option. We now construct the parsing table, assuming + < *, that is, multiplication binds more tightly than addition. Moreover, we specify that both addition and multiplication are left associative so that, for example, should be parsed as (3 + 4) + 5. We have removed id since it behaves identically to num. [plus] E E + E [times] E E * E [number] E num [parens] E ( E ) As before, we assume that a special end-of-file token \$ has been added to the end of the input string. When the parsing goal has the form γ β a w where β is a prefix substring of the grammar, we look up β in the left-most column and a in the top row to find the action to take. The non-terminal ɛ E in the last line is a special case in that E must be the only thing on the

5 Bottom-Up LR Parsing L9.5 stack. In that case we can accept if the next token is \$ because we know that \$ can only be the last token of the input string. γ β a w + * num ( ) \$ E + E reduce(plus) shift error error reduce(plus) reduce(plus) (+ left assoc.) (+ < *) E * E reduce(times) reduce(times) error error reduce(times) reduce(times) (+ < *) (* left assoc.) num reduce(number) reduce(number) error error reduce(number) reduce(number) ( E ) reduce(parens) reduce(parens) error error reduce(parens) reduce(parens) E + error error shift shift error error E * error error shift shift error error ( E shift shift error error shift error ( error error shift shift error error ɛ E shift shift error error error accept(e) We can see that the bare grammar has four shift/reduce conflicts, while all other actions (including errors) are uniquely determined. These conficts arise when E + E or E * E is on the stack and either + or * is the first character in the remaining input string. It is called a shift/reduce conflict, because either a shift action or a reduce action could lead to a valid parse. Here, we have decided to resolve the conflicts by giving a precedence to the operators and declaring both of them to be left-associative. It is also possible to have reduce/reduce conflicts, if more than one reduction could be applied in a given situation, but it does not happen in this grammar. A simple illustration where reduce/reduce conflicts can possibly show is when you have add a test statement with natural equation notation in your language. S A; S T? A id = E T id = E In this grammar, id=e can be reduced to an assignment (A) or to a test statement (T). Both reductions are possible. Here, it s easy to distinguish between both cases based on taking the follow sets into account. A can only be followed by ; and test can only be followed by?, which makes it unambiguous at the time of reduction here with a 1 token lookahead. Parser generators will generally issue an error or warning when they detect a shift/reduce or reduce/reduce conflict. For many parser generators, the default behavior of a shift/reduce conflict is to shift, and for a

6 L9.6 Bottom-Up LR Parsing reduce/reduce conflict to apply the textually first production in the grammar. Particularly the latter is almost never what is desired, so we strongly recommend rewriting the grammar to eliminate any conflicts in an LR(1) parser. One interesting special case is the situation in a language where the elseclause of a conditional is optional. For example, one might write (among other productions) Now a statement S IS S E E ID IS if E then S IS if E then S else S if (b) then if (c) then x else y is ambiguous because it could be read as if (b) then (if (c) then x) else y or if (b) then (if (c) then x else y) In fact, LR parsers will report a shift/reduce conflict for this grammar. In a shift/reduce parser, typically the default action for a shift/reduce conflict is to shift. This means that the above grammar in a tool such as ML-Lex will parse the ambiguous statement into the second form, that is, the else is match with the most recent unmatched if. This is consistent with language such as C (or the language used in this course), so we can tolerate the above shift/reduce conflict, if you wish, instead of rewriting the grammar to make it unambiguous. Yet, this is a dangerous business, because we depend on the mercy of the LR parser generator to hopefully choose the right conflict resolutions for us. state 7 contains 1 shift/reduce conflict.... state 7 SI -> if E then S. (rule 4) IS -> if E then S. else S (rule 5) else shift, and go to state 8 else [reduce using rule 4 (IS)] \$default reduce using rule 4 (IS)

7 Bottom-Up LR Parsing L9.7 How else can we fix the conflict? The easiest and best fix is to change the language and require some form of an end if token. This solution will also make the subtlety transparent to the user, because there can no longer be surprises in how the ambiguity is resolved. The problem is that this changes the language. The other fix is to disambiguate the grammar by factorization. S IS S E E ID IS if E then S IS if E then IT E else S IT E if E then IT E else IT E IT E E The effect of this factorization is that the then part of an if-then-else statement can no longer be a single if-then statement. It has to be a full if-thenelse or not an if-statement at all. 4 Implementations of LR Parser Generators LR parser generators are quite tricky to implement. A full coverage of how they work would need several lectures. The principle behind actual implementations of LR parser generators is to define a nondeterministic (pushdown) automaton called characteristic automaton that follows all possible parse positions. This characteristic automaton is then turned into an actual implementation by converting it into a deterministic (pushdown) automaton using the subset construction. This automaton will follow all possible analysis situations until a reduce becomes evident. There are several similarities between lexer generators (previous lectures) and LR parser generators. In fact, lexers can actually be generated using a special case of LR grammars called LALR(0), where we leave out the stack so that every shifted character has to be reduced immediately. Full LR(1) parsers also have too many states. That is why subclasses of LR(1) are often used for the parsing situations, with a set of states restricted to those of LR(0). Quiz 1. What happens if we remove the ɛ from the last entry in the LR parser table? Aren t ɛ s irrelevant and can always be removed?

8 L9.8 Bottom-Up LR Parsing 2. What makes x*y; difficult to parse in C and C0? Should this be handled in the lexing? How can this be handled in the parser? How can this be handled in later stages after parsing? 3. Give a very simple example of a grammar with a shift/reduce conflict. 4. Give an example of a grammar with a shift/reduce conflict that occurs in programming language parsing and is not easily resolved using associativity/precedence of arithmetic operators. 5. Give a very simple example of a grammar with a reduce/reduce conflict. 6. Give an example of a grammar with a reduce/reduce conflict that occurs in programming language parsing and is not easily resolved. 7. Suppose you want to add a short hand notation for asserts into your programming language. Should a programming language have a test statement of the form?e; or of the form E?; and which one is better for which purpose? 8. In the reduce rule, we have used a number of symbols on the top of the stack and the lookahead to decide what to do. But isn t a stack something where we can only read one symbol off of the top? Does it make a difference in expressive power if we allow decisions to depend on 1 or on 10 symbols on the top of the stack? Does it make a difference in expressive power if we allow 1 or arbitrarily many symbols from the top of the stack for the decision? References [App98] Andrew W. Appel. Modern Compiler Implementation in ML. Cambridge University Press, Cambridge, England, [SSP95] Stuart M. Shieber, Yves Schabes, and Fernando C. N. Pereira. Principles and implementation of deductive parsing. J. Log. Program., 24(1&2):3 36, [WM95] Reinhard Wilhelm and Dieter Maurer. Compiler Design. Addison- Wesley, 1995.

### Lecture Notes on Bottom-Up LR Parsing

Lecture Notes on Bottom-Up LR Parsing 15-411: Compiler Design Frank Pfenning Lecture 9 September 23, 2009 1 Introduction In this lecture we discuss a second parsing algorithm that traverses the input string

### Lecture Notes on Shift-Reduce Parsing

Lecture Notes on Shift-Reduce Parsing 15-411: Compiler Design Frank Pfenning, Rob Simmons, André Platzer Lecture 8 September 24, 2015 1 Introduction In this lecture we discuss shift-reduce parsing, which

### Lecture Notes on Top-Down Predictive LL Parsing

Lecture Notes on Top-Down Predictive LL Parsing 15-411: Compiler Design Frank Pfenning Lecture 8 1 Introduction In this lecture we discuss a parsing algorithm that traverses the input string from l eft

### Bottom-Up Parsing. Lecture 11-12

Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 2/20/08 Prof. Hilfinger CS164 Lecture 11 1 Administrivia Test I during class on 10 March. 2/20/08 Prof. Hilfinger CS164 Lecture 11

### Lecture 8: Deterministic Bottom-Up Parsing

Lecture 8: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Fri Feb 12 13:02:57 2010 CS164: Lecture #8 1 Avoiding nondeterministic choice: LR We ve been looking at general

### Lecture 7: Deterministic Bottom-Up Parsing

Lecture 7: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Tue Sep 20 12:50:42 2011 CS164: Lecture #7 1 Avoiding nondeterministic choice: LR We ve been looking at general

### Bottom-Up Parsing. Lecture 11-12

Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 9/22/06 Prof. Hilfinger CS164 Lecture 11 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient

### In One Slide. Outline. LR Parsing. Table Construction

LR Parsing Table Construction #1 In One Slide An LR(1) parsing table can be constructed automatically from a CFG. An LR(1) item is a pair made up of a production and a lookahead token; it represents a

### LR Parsing LALR Parser Generators

LR Parsing LALR Parser Generators Outline Review of bottom-up parsing Computing the parsing DFA Using parser generators 2 Bottom-up Parsing (Review) A bottom-up parser rewrites the input string to the

### Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery. Last modified: Mon Feb 23 10:05: CS164: Lecture #14 1

Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery Last modified: Mon Feb 23 10:05:56 2015 CS164: Lecture #14 1 Shift/Reduce Conflicts If a DFA state contains both [X: α aβ, b] and [Y: γ, a],

### LR Parsing LALR Parser Generators

Outline LR Parsing LALR Parser Generators Review of bottom-up parsing Computing the parsing DFA Using parser generators 2 Bottom-up Parsing (Review) A bottom-up parser rewrites the input string to the

### Parsing Wrapup. Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing. This lecture LR(1) parsing

Parsing Wrapup Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing LR(1) items Computing closure Computing goto LR(1) canonical collection This lecture LR(1) parsing Building ACTION

### S Y N T A X A N A L Y S I S LR

LR parsing There are three commonly used algorithms to build tables for an LR parser: 1. SLR(1) = LR(0) plus use of FOLLOW set to select between actions smallest class of grammars smallest tables (number

### Principle of Compilers Lecture IV Part 4: Syntactic Analysis. Alessandro Artale

Free University of Bolzano Principles of Compilers Lecture IV Part 4, 2003/2004 AArtale (1) Principle of Compilers Lecture IV Part 4: Syntactic Analysis Alessandro Artale Faculty of Computer Science Free

### Conflicts in LR Parsing and More LR Parsing Types

Conflicts in LR Parsing and More LR Parsing Types Lecture 10 Dr. Sean Peisert ECS 142 Spring 2009 1 Status Project 2 Due Friday, Apr. 24, 11:55pm The usual lecture time is being replaced by a discussion

### Lexical and Syntax Analysis. Bottom-Up Parsing

Lexical and Syntax Analysis Bottom-Up Parsing Parsing There are two ways to construct derivation of a grammar. Top-Down: begin with start symbol; repeatedly replace an instance of a production s LHS with

### Compilers. Bottom-up Parsing. (original slides by Sam

Compilers Bottom-up Parsing Yannis Smaragdakis U Athens Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Bottom-Up Parsing More general than top-down parsing And just as efficient Builds

### LR Parsing, Part 2. Constructing Parse Tables. An NFA Recognizing Viable Prefixes. Computing the Closure. GOTO Function and DFA States

TDDD16 Compilers and Interpreters TDDB44 Compiler Construction LR Parsing, Part 2 Constructing Parse Tables Parse table construction Grammar conflict handling Categories of LR Grammars and Parsers An NFA

Downloaded from http://himadri.cmsdu.org Page 1 LR Parsing We first understand Context Free Grammars. Consider the input string: x+2*y When scanned by a scanner, it produces the following stream of tokens:

### Lecture Bottom-Up Parsing

Lecture 14+15 Bottom-Up Parsing CS 241: Foundations of Sequential Programs Winter 2018 Troy Vasiga et al University of Waterloo 1 Example CFG 1. S S 2. S AyB 3. A ab 4. A cd 5. B z 6. B wz 2 Stacks in

### LR Parsing. Table Construction

#1 LR Parsing Table Construction #2 Outline Review of bottom-up parsing Computing the parsing DFA Closures, LR(1) Items, States Transitions Using parser generators Handling Conflicts #3 In One Slide An

### MIT Parse Table Construction. Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology

MIT 6.035 Parse Table Construction Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Parse Tables (Review) ACTION Goto State ( ) \$ X s0 shift to s2 error error goto s1

### shift-reduce parsing

Parsing #2 Bottom-up Parsing Rightmost derivations; use of rules from right to left Uses a stack to push symbols the concatenation of the stack symbols with the rest of the input forms a valid bottom-up

### Let us construct the LR(1) items for the grammar given below to construct the LALR parsing table.

MODULE 18 LALR parsing After understanding the most powerful CALR parser, in this module we will learn to construct the LALR parser. The CALR parser has a large set of items and hence the LALR parser is

### LR Parsing - The Items

LR Parsing - The Items Lecture 10 Sections 4.5, 4.7 Robb T. Koether Hampden-Sydney College Fri, Feb 13, 2015 Robb T. Koether (Hampden-Sydney College) LR Parsing - The Items Fri, Feb 13, 2015 1 / 31 1 LR

### Context-free grammars

Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,

### Parsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468

Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure

### Compilation 2013 Parser Generators, Conflict Management, and ML-Yacc

Compilation 2013 Parser Generators, Conflict Management, and ML-Yacc Erik Ernst Aarhus University Parser generators, ML-Yacc LR parsers are tedious to write, but can be generated, e.g., by ML-Yacc Input:

### Formal Languages and Compilers Lecture VII Part 4: Syntactic A

Formal Languages and Compilers Lecture VII Part 4: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

### Wednesday, September 9, 15. Parsers

Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Parsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:

What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### CS 4120 Introduction to Compilers

CS 4120 Introduction to Compilers Andrew Myers Cornell University Lecture 6: Bottom-Up Parsing 9/9/09 Bottom-up parsing A more powerful parsing technology LR grammars -- more expressive than LL can handle

### CSE P 501 Compilers. LR Parsing Hal Perkins Spring UW CSE P 501 Spring 2018 D-1

CSE P 501 Compilers LR Parsing Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts UW CSE P 501 Spring 2018

### Wednesday, August 31, Parsers

Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically

### Monday, September 13, Parsers

Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions

### More Bottom-Up Parsing

More Bottom-Up Parsing Lecture 7 Dr. Sean Peisert ECS 142 Spring 2009 1 Status Project 1 Back By Wednesday (ish) savior lexer in ~cs142/s09/bin Project 2 Due Friday, Apr. 24, 11:55pm My office hours 3pm

### Compiler Construction: Parsing

Compiler Construction: Parsing Mandar Mitra Indian Statistical Institute M. Mitra (ISI) Parsing 1 / 33 Context-free grammars. Reference: Section 4.2 Formal way of specifying rules about the structure/syntax

### Example CFG. Lectures 16 & 17 Bottom-Up Parsing. LL(1) Predictor Table Review. Stacks in LR Parsing 1. Sʹ " S. 2. S " AyB. 3. A " ab. 4.

Example CFG Lectures 16 & 17 Bottom-Up Parsing CS 241: Foundations of Sequential Programs Fall 2016 1. Sʹ " S 2. S " AyB 3. A " ab 4. A " cd Matt Crane University of Waterloo 5. B " z 6. B " wz 2 LL(1)

### CS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F).

CS 2210 Sample Midterm 1. Determine if each of the following claims is true (T) or false (F). F A language consists of a set of strings, its grammar structure, and a set of operations. (Note: a language

### Review main idea syntax-directed evaluation and translation. Recall syntax-directed interpretation in recursive descent parsers

Plan for Today Review main idea syntax-directed evaluation and translation Recall syntax-directed interpretation in recursive descent parsers Syntax-directed evaluation and translation in shift-reduce

### Compilers and computer architecture From strings to ASTs (2): context free grammars

1 / 1 Compilers and computer architecture From strings to ASTs (2): context free grammars Martin Berger October 2018 Recall the function of compilers 2 / 1 3 / 1 Recall we are discussing parsing Source

### Bottom-Up Parsing II. Lecture 8

Bottom-Up Parsing II Lecture 8 1 Review: Shift-Reduce Parsing Bottom-up parsing uses two actions: Shift ABC xyz ABCx yz Reduce Cbxy ijk CbA ijk 2 Recall: he Stack Left string can be implemented by a stack

### 3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino

3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the

### Formal Languages and Compilers Lecture VII Part 3: Syntactic A

Formal Languages and Compilers Lecture VII Part 3: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

### Bottom-up parsing. Bottom-Up Parsing. Recall. Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form

Bottom-up parsing Bottom-up parsing Recall Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form If α V t,thenα is called a sentence in L(G) Otherwise it is just

### Introduction to Parsing. Lecture 5

Introduction to Parsing Lecture 5 1 Outline Regular languages revisited Parser overview Context-free grammars (CFG s) Derivations Ambiguity 2 Languages and Automata Formal languages are very important

### CSE 401 Compilers. LR Parsing Hal Perkins Autumn /10/ Hal Perkins & UW CSE D-1

CSE 401 Compilers LR Parsing Hal Perkins Autumn 2011 10/10/2011 2002-11 Hal Perkins & UW CSE D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts 10/10/2011

### PART 3 - SYNTAX ANALYSIS. F. Wotawa TU Graz) Compiler Construction Summer term / 309

PART 3 - SYNTAX ANALYSIS F. Wotawa (IST @ TU Graz) Compiler Construction Summer term 2016 64 / 309 Goals Definition of the syntax of a programming language using context free grammars Methods for parsing

### Chapter 3: Lexing and Parsing

Chapter 3: Lexing and Parsing Aarne Ranta Slides for the book Implementing Programming Languages. An Introduction to Compilers and Interpreters, College Publications, 2012. Lexing and Parsing* Deeper understanding

### Parsing. Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1)

TD parsing - LL(1) Parsing First and Follow sets Parse table construction BU Parsing Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1) Problems with SLR Aho, Sethi, Ullman, Compilers

### CS415 Compilers. Syntax Analysis. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University

CS415 Compilers Syntax Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Limits of Regular Languages Advantages of Regular Expressions

### UNIT III & IV. Bottom up parsing

UNIT III & IV Bottom up parsing 5.0 Introduction Given a grammar and a sentence belonging to that grammar, if we have to show that the given sentence belongs to the given grammar, there are two methods.

### Section A. A grammar that produces more than one parse tree for some sentences is said to be ambiguous.

Section A 1. What do you meant by parser and its types? A parser for grammar G is a program that takes as input a string w and produces as output either a parse tree for w, if w is a sentence of G, or

### Midterm I (Solutions) CS164, Spring 2002

Midterm I (Solutions) CS164, Spring 2002 February 28, 2002 Please read all instructions (including these) carefully. There are 9 pages in this exam and 5 questions, each with multiple parts. Some questions

### Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

### Syntax-Directed Translation

Syntax-Directed Translation ALSU Textbook Chapter 5.1 5.4, 4.8, 4.9 Tsan-sheng Hsu tshsu@iis.sinica.edu.tw http://www.iis.sinica.edu.tw/~tshsu 1 What is syntax-directed translation? Definition: The compilation

### Simple LR (SLR) LR(0) Drawbacks LR(1) SLR Parse. LR(1) Start State and Reduce. LR(1) Items 10/3/2012

LR(0) Drawbacks Consider the unambiguous augmented grammar: 0.) S E \$ 1.) E T + E 2.) E T 3.) T x If we build the LR(0) DFA table, we find that there is a shift-reduce conflict. This arises because the

### Compiler Construction 2016/2017 Syntax Analysis

Compiler Construction 2016/2017 Syntax Analysis Peter Thiemann November 2, 2016 Outline 1 Syntax Analysis Recursive top-down parsing Nonrecursive top-down parsing Bottom-up parsing Syntax Analysis tokens

### EDAN65: Compilers, Lecture 06 A LR parsing. Görel Hedin Revised:

EDAN65: Compilers, Lecture 06 A LR parsing Görel Hedin Revised: 2017-09-11 This lecture Regular expressions Context-free grammar Attribute grammar Lexical analyzer (scanner) Syntactic analyzer (parser)

### Formal Languages and Compilers Lecture V: Parse Trees and Ambiguous Gr

Formal Languages and Compilers Lecture V: Parse Trees and Ambiguous Grammars Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

### Introduction to Parsing. Lecture 5

Introduction to Parsing Lecture 5 1 Outline Regular languages revisited Parser overview Context-free grammars (CFG s) Derivations Ambiguity 2 Languages and Automata Formal languages are very important

### 3. Parsing. Oscar Nierstrasz

3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/

### UNIVERSITY OF CALIFORNIA

UNIVERSITY OF CALIFORNIA Department of Electrical Engineering and Computer Sciences Computer Science Division CS164 Fall 1997 P. N. Hilfinger CS 164: Midterm Name: Please do not discuss the contents of

### Lexical and Syntax Analysis. Top-Down Parsing

Lexical and Syntax Analysis Top-Down Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax

### MODULE 14 SLR PARSER LR(0) ITEMS

MODULE 14 SLR PARSER LR(0) ITEMS In this module we shall discuss one of the LR type parser namely SLR parser. The various steps involved in the SLR parser will be discussed with a focus on the construction

### LR Parsing Techniques

LR Parsing Techniques Introduction Bottom-Up Parsing LR Parsing as Handle Pruning Shift-Reduce Parser LR(k) Parsing Model Parsing Table Construction: SLR, LR, LALR 1 Bottom-UP Parsing A bottom-up parser

### CS143 Handout 20 Summer 2011 July 15 th, 2011 CS143 Practice Midterm and Solution

CS143 Handout 20 Summer 2011 July 15 th, 2011 CS143 Practice Midterm and Solution Exam Facts Format Wednesday, July 20 th from 11:00 a.m. 1:00 p.m. in Gates B01 The exam is designed to take roughly 90

### Principles of Programming Languages

Principles of Programming Languages h"p://www.di.unipi.it/~andrea/dida2ca/plp- 14/ Prof. Andrea Corradini Department of Computer Science, Pisa Lesson 8! Bo;om- Up Parsing Shi?- Reduce LR(0) automata and

### LR Parsing - Conflicts

LR Parsing - Conflicts Lecture 15 Sections 4.5, 4.6 Robb T. Koether Hampden-Sydney College Fri, Feb 20, 2015 Robb T. Koether (Hampden-Sydney College) LR Parsing - Conflicts Fri, Feb 20, 2015 1 / 15 1 Shift/Reduce

### Review: Shift-Reduce Parsing. Bottom-up parsing uses two actions: Bottom-Up Parsing II. Shift ABC xyz ABCx yz. Lecture 8. Reduce Cbxy ijk CbA ijk

Review: Shift-Reduce Parsing Bottom-up parsing uses two actions: Bottom-Up Parsing II Lecture 8 Shift ABC xyz ABCx yz Reduce Cbxy ijk CbA ijk Prof. Aiken CS 13 Lecture 8 1 Prof. Aiken CS 13 Lecture 8 2

### Bottom-Up Parsing II (Different types of Shift-Reduce Conflicts) Lecture 10. Prof. Aiken (Modified by Professor Vijay Ganesh.

Bottom-Up Parsing II Different types of Shift-Reduce Conflicts) Lecture 10 Ganesh. Lecture 10) 1 Review: Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient Doesn

### How do LL(1) Parsers Build Syntax Trees?

How do LL(1) Parsers Build Syntax Trees? So far our LL(1) parser has acted like a recognizer. It verifies that input token are syntactically correct, but it produces no output. Building complete (concrete)

### Introduction to Parsing

Introduction to Parsing The Front End Source code Scanner tokens Parser IR Errors Parser Checks the stream of words and their parts of speech (produced by the scanner) for grammatical correctness Determines

### Syntax Analysis: Context-free Grammars, Pushdown Automata and Parsing Part - 4. Y.N. Srikant

Syntax Analysis: Context-free Grammars, Pushdown Automata and Part - 4 Department of Computer Science and Automation Indian Institute of Science Bangalore 560 012 NPTEL Course on Principles of Compiler

### Syntax Analysis, V Bottom-up Parsing & The Magic of Handles Comp 412

Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, V Bottom-up Parsing & The Magic of Handles Comp 412 COMP 412 FALL 2018 source code IR Front End Optimizer Back End IR target

### Automatic generation of LL(1) parsers

Automatic generation of LL(1) parsers Informatics 2A: Lecture 12 John Longley School of Informatics University of Edinburgh jrl@staffmail.ed.ac.uk 14 October, 2011 1 / 12 Generating parse tables We ve

### JavaCC Parser. The Compilation Task. Automated? JavaCC Parser

JavaCC Parser The Compilation Task Input character stream Lexer stream Parser Abstract Syntax Tree Analyser Annotated AST Code Generator Code CC&P 2003 1 CC&P 2003 2 Automated? JavaCC Parser The initial

### COMPILER (CSE 4120) (Lecture 6: Parsing 4 Bottom-up Parsing )

COMPILR (CS 4120) (Lecture 6: Parsing 4 Bottom-up Parsing ) Sungwon Jung Mobile Computing & Data ngineering Lab Dept. of Computer Science and ngineering Sogang University Seoul, Korea Tel: +82-2-705-8930

### Parsing Part II. (Ambiguity, Top-down parsing, Left-recursion Removal)

Parsing Part II (Ambiguity, Top-down parsing, Left-recursion Removal) Ambiguous Grammars Definitions If a grammar has more than one leftmost derivation for a single sentential form, the grammar is ambiguous

### Compiler Construction Using

Compiler Construction Using Java, JavaCC, and Yacc ANTHONY J. DOS REIS Stale University ofnew York at New Pallz IEEE computer society WILEY A JOHN WILEY & SONS, INC., PUBLICATION Preface xv Chapter 1 Strings,

### CS606- compiler instruction Solved MCQS From Midterm Papers

CS606- compiler instruction Solved MCQS From Midterm Papers March 06,2014 MC100401285 Moaaz.pk@gmail.com Mc100401285@gmail.com PSMD01 Final Term MCQ s and Quizzes CS606- compiler instruction If X is a

### Configuration Sets for CSX- Lite. Parser Action Table

Configuration Sets for CSX- Lite State s 6 s 7 Cofiguration Set Prog { Stmts } Eof Stmts Stmt Stmts State s s Cofiguration Set Prog { Stmts } Eof Prog { Stmts } Eof Stmts Stmt Stmts Stmts λ Stmt if ( Expr

### Lecture Notes on Intermediate Representation

Lecture Notes on Intermediate Representation 15-411: Compiler Design Frank Pfenning Lecture 10 September 26, 2013 1 Introduction In this lecture we discuss the middle end of the compiler. After the source

### Table-driven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and non-terminals.

Bottom-up Parsing: Table-driven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and non-terminals. Basic operation is to shift terminals from the input to the

### 4. Lexical and Syntax Analysis

4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal

### COMP 181. Prelude. Prelude. Summary of parsing. A Hierarchy of Grammar Classes. More power? Syntax-directed translation. Analysis

Prelude COMP 8 October, 9 What is triskaidekaphobia? Fear of the number s? No aisle in airplanes, no th floor in buildings Fear of Friday the th? Paraskevidedekatriaphobia or friggatriskaidekaphobia Why

### LL(1) predictive parsing

LL(1) predictive parsing Informatics 2A: Lecture 11 Mary Cryan School of Informatics University of Edinburgh mcryan@staffmail.ed.ac.uk 10 October 2018 1 / 15 Recap of Lecture 10 A pushdown automaton (PDA)

### Torben./Egidius Mogensen. Introduction. to Compiler Design. ^ Springer

Torben./Egidius Mogensen Introduction to Compiler Design ^ Springer Contents 1 Lexical Analysis 1 1.1 Regular Expressions 2 1.1.1 Shorthands 4 1.1.2 Examples 5 1.2 Nondeterministic Finite Automata 6 1.3

### 4. Lexical and Syntax Analysis

4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal

### COMP-421 Compiler Design. Presented by Dr Ioanna Dionysiou

COMP-421 Compiler Design Presented by Dr Ioanna Dionysiou Administrative! Any questions about the syllabus?! Course Material available at www.cs.unic.ac.cy/ioanna! Next time reading assignment [ALSU07]

### Recursive Descent Parsers

Recursive Descent Parsers Lecture 7 Robb T. Koether Hampden-Sydney College Wed, Jan 28, 2015 Robb T. Koether (Hampden-Sydney College) Recursive Descent Parsers Wed, Jan 28, 2015 1 / 18 1 Parsing 2 LL Parsers

### Review of CFGs and Parsing II Bottom-up Parsers. Lecture 5. Review slides 1

Review of CFGs and Parsing II Bottom-up Parsers Lecture 5 1 Outline Parser Overview op-down Parsers (Covered largely through labs) Bottom-up Parsers 2 he Functionality of the Parser Input: sequence of

### LR Parsing. Leftmost and Rightmost Derivations. Compiler Design CSE 504. Derivations for id + id: T id = id+id. 1 Shift-Reduce Parsing.

LR Parsing Compiler Design CSE 504 1 Shift-Reduce Parsing 2 LR Parsers 3 SLR and LR(1) Parsers Last modifled: Fri Mar 06 2015 at 13:50:06 EST Version: 1.7 16:58:46 2016/01/29 Compiled at 12:57 on 2016/02/26

### Syntax. 2.1 Terminology

Syntax 2 Once you ve learned to program in one language, learning a similar programming language isn t all that hard. But, understanding just how to write in the new language takes looking at examples

### EDAN65: Compilers, Lecture 04 Grammar transformations: Eliminating ambiguities, adapting to LL parsing. Görel Hedin Revised:

EDAN65: Compilers, Lecture 04 Grammar transformations: Eliminating ambiguities, adapting to LL parsing Görel Hedin Revised: 2017-09-04 This lecture Regular expressions Context-free grammar Attribute grammar

### LALR stands for look ahead left right. It is a technique for deciding when reductions have to be made in shift/reduce parsing. Often, it can make the

LALR parsing 1 LALR stands for look ahead left right. It is a technique for deciding when reductions have to be made in shift/reduce parsing. Often, it can make the decisions without using a look ahead.

### Syntax Analysis Check syntax and construct abstract syntax tree

Syntax Analysis Check syntax and construct abstract syntax tree if == = ; b 0 a b Error reporting and recovery Model using context free grammars Recognize using Push down automata/table Driven Parsers