# Lecture Notes on Top-Down Predictive LL Parsing

Size: px
Start display at page:

Transcription

1 Lecture Notes on Top-Down Predictive LL Parsing : Compiler Design Frank Pfenning Lecture 8 1 Introduction In this lecture we discuss a parsing algorithm that traverses the input string from l eft to right giving a l eft-most derivation and makes a decision on which grammar production to use based on the first character/token of the input string. This parsing algorithm is called LL(1). If that were ambiguous, the grammar would have to be rewritten to fall into this class, which is not always possible. Most hand-written parsers are recursive descent parsers, which follow the LL(1) principles. Alternative presentations of the material in this lecture can be found in a paper by Shieber et al. [SSP95]. The textbook [App98, Chapter 3] covers parsing. For more detailed background on parsing, also see [WM95]. 2 LL(1) Parsing We have seen in the previous section, that the general idea of recursive descent parsing without restrictions forces us to non-deterministically choose between several productions which might be applied and potentially backtrack if parsing gets stuck after a choice, or even loop (if the grammar is left-recursive). Backtracking is not only potentially very inefficient, but it makes it difficult to produce good error messages in case the string is not grammatically well-formed. Say we try three different ways to parse a given input and all fail. How could we say which of these is the source With edits by André Platzer

2 L8.2 Top-Down Predictive LL Parsing of the error? How could we report an error location? This is compounded because nested choices multiply the number of possibilities. We therefore have to look for ways to disambiguate the choices, making parsing more efficient and syntax errors easier to locate. One way is to require of the grammar that at each potential choice point we can look at the next input token and based on that token decide which production to take. This is called 1 token lookahead, and grammars that satisfy this restriction are called LL(1) grammars. Here, the first L stands for Left-to-right reading of the input; the second L stands for Leftmost parse (which a recursive descent parser generates) and 1 stands for 1 token lookahead. We can also define LL(2), LL(3), etc. But for higher k, LL(k) parsers become fairly inefficient and are used less frequently. The LL parser generators (or SLL parser generators which are a simplification of LL) themselves, however, are much more efficient than the LR parser generators. Since we are restricting ourselves to parsing by a left-to-right traversal of the input string, we will consider only tails, or suffixes of the input strings, and also of the strings in the grammar, when we restrict our inference rules. Those suffixes capture what still has to be parsed or worked on. For short, we will say γ is a suffix substring of the grammar, or w is a suffix substring of the input string w 0. For example, in the grammar [emp] S [pars] S [ S ] [dup] S S S the only suffix substrings are ɛ, [ S ], S ], ], S, and S S, but not the prefix [ S. Firsts & Nulls The main idea behind LL(1)-parsing is that we restrict our attention to grammars where we can use a 1 token lookahead to disambiguate which grammar production to use. That is, by peeking at the next input token, we want to know which grammar production helps us. The first thing we want to know for this is what the tokens are that could be generated from a sequence β of terminals and non-terminals at all. So imagine β describes a shape of how far we have come with parsing and what we expect for the rest of the input. It could be something like S]. We begin by defining two kinds of predicates (later we will have occasion to add a third), where β is either a non-terminal or suffix substring of the grammar. The predicate first(β, a) captures if token a can be the first token occurring in a word that matches the expression β. What we also need to know is if

4 L8.4 Top-Down Predictive LL Parsing N 1 expresses that ɛ can produce ɛ surprise. That Xβ can produce ɛ if both X and β can (N 2 ). And that X can produce ɛ if one of its productions has a right-hand side γ hat can (N 3 ). We can run these rules to saturation because there are only O( G ) possible strings in the first argument to both of these predicates, and at most the number of possible terminal symbols in the grammar, O( Σ ), in the second argument. Naive counting the number of prefix firings (see [GM02]) gives a complexity bound of O( G Ξ Σ ) where Ξ is the number of non-terminals in the grammar. Since usually the number of symbols is a small constant, this is roughly equivalent to O( G ) and so is reasonably efficient. Moreover, it only happens once, before any parsing takes place. Constructing LL(1) Parsing Rules Next, we modify the rules for recursive descent parsing from the last lecture to take these restrictions into account. The first two compare rules stay the same. The third generate rule, ɛ : ɛ L 1 [r]x β w : β γ w : X γ w : γ a w : a γ L 2 L 3 (r) is split into two, each of which has an additional precondition on when to use it: [r]x β first(β, a) a w : β γ a w : X γ L 3 [r]x β null(β) w : β γ w : X γ L 3? We would like to say that a grammar is LL(1) if the additional preconditions in these last two rules make all choices unambiguous when an arbitrary non-terminal X is matched against a string starting with an arbitrary terminal a. Unfortunately, this does not quite work yet in the presence nonterminals that can rewrite to ɛ, because the second rule above does not even look at the input string. This nondeterministic rule cannot possibly ensure the appropriate parse without looking at the input. To disambiguate, we need to know what tokens could follow X. Only if the first token of w could follow X would we want to use a production for X that would make it vanish.

5 Top-Down Predictive LL Parsing L8.5 Follows We thus need to know which tokens could follow X in order to know if it would make sense to skip over it by using one of its productions that produces ɛ. For that, we define one additional predicate, again on suffix strings in the grammar and non-terminals. follow(β, a) Token a can follow string β in a valid string See Figure 1 for an illustration of where first and follow come from in a parse tree. The set of all a for which first(x, a) characterizes what tokens X itself can start with. The set of all a for which follow(x, a) characterizes what tokens can follow X. nonterminals: X tokens: first(x, ) follow(x, ) Figure 1: Illustration of first and follow sets of non-terminal X in a parse tree in which X occurs We seed this follows relation with the rules Xγ suffix first(γ, a) follow(x, a) W 1 Here, Xγ suffix means that the string Xγ appears as a suffix substring on the right-hand side of some production, because then we would want to know if we should choose the productions for X that ultimately produce ɛ, because the lookahead token comes from γ. Whatever can come first in γ can follow X if Xγ occurs in the right-hand sides of some production. We then propagate this information applying the following rules from premises to conclusion until saturation is reached. follow(b γ, a) follow(γ, a) W 2 follow(x γ, a) follow(γ, a) W 3 follow(x γ, a) null(γ) follow(x, a) W 4 [r]x γ follow(x, a) follow(γ, a) W 5

7 Top-Down Predictive LL Parsing L There is a first/follow conflict if we cannot always decide between L 3 and L 3. This happens if there is a token a with first(β, a) for a production X β that would trigger L 3, but that token also satisfies follow(x, a) with a nonterminal X that is nullable and trigger L 3. Example We now use a very simple grammar to illustrate these rules. We have transformed it in the way indicated above, by assuming a special token \$ to indicate the end of the input string. [start] S S \$ [emp] S ɛ [pars] S [ S ] This grammar generates all strings starting with an arbitrary number of opening parentheses followed by the same number of closing parentheses and an end-of-string marker, i.e., the language [ n ] n \$. We have: null(ɛ) N 1 null(s ) N 3 first([ S ], [) F 1 first(], ]) F 1 first(s ], ]) F 3 first(s, [) F 4 [pars] first(s ], [) F 2 first(\$, \$) F 1 first(s \$, \$) F 3 first(s \$, [) F 2 first(s, \$) F 4 [start] first(s, [) F 4 [start] follow(s, \$) W 1 follow(s, ]) W 1 follow([ S ], \$) W 5 follow([ S ], ]) W 5 follow(s ], \$) W 3 follow(s ], ]) W 3 follow(], \$) W 4 follow(], ]) W 4

8 L8.8 Top-Down Predictive LL Parsing 3 Parser Generation Parser generation is now a very simple process. Once we have computed the null, first, and follow predicates by saturation from a given grammar, we specialize the inference rules L 3 (r) and L 3 (r) by matching the first two and three premises against grammar productions and saturated null, first, and follow database. This is essentially partial evaluation on inference rules. In this case, this leads to the following specialized rules (repeating once again the two initial rules). ɛ : ɛ L 1 w : γ a w : a γ L 2 [ w : S \$ γ [ w : S γ L 3 (start) \$ w : S \$ γ \$ w : S γ L 3 (start) [ w : [ S ] γ [ w : S γ L 3 (pars) ] w : γ ] w : S γ L 3 (emp) \$ w : γ \$ w : S γ L 3 (emp) Recall that these rules are applied from the bottom-up, starting with the goal w 0 \$ : S, where w 0 is the input string. It is easy to observe by pattern matching that each of these rules are mutually exclusive: if one of the rules applies, none of the other rules apply. Moreover, each rule except for L 1 (which accepts) has exactly one premise, so the input string is traversed linearly from left-to-right, without backtracking. When none of the rules applies, then the input string is not in the language defined by the grammar. This proves that our simple language [ n ] n is LL(1). Besides efficiency, an effect of this LL approach to parser generation is that it supports good error messages in the case of failure. For example, if we see the parsing goal ( w : ) γ we can report: Found ( while expecting ) along with a report of the error location. Similarly for other cases that match none of the conclusions of the rules. 4 Grammar Transformations This predictive parsing or LL(1) parsing works quite well. But now suppose we have a grammar X Y b X Y c

9 Top-Down Predictive LL Parsing L8.9 for a nonterminal Y that may produce token streams of unbounded length. This grammar is left-recursive, which is a problem, because we cannot know which production to use in predictive parsing without unbounded token lookahead. Yet we can left-factorize the grammar in order to capture the common prefix of the two productions and only distinguish their suffix in a new common nonterminal X. This gives X Y X X b X c This left-factorization is also useful to remove a First/First conflict from grammar S if E then S else S S if E then S turning it into a form where common prefixes have been factored out S if E then S S S else S S ɛ More generally, consider the grammar X αγ X αδ where we have difficulties deciding between the two productions based on token lookahead, because both start with α. We can left-factorize it into the following grammar where common prefixes have been factored out and the distinction between γ and δ happens at a new nonterminal X X αx X γ X δ The same phenomenon can also happen indirectly, though, even if the grammar rules do not literally start with the same expression Y. Suppose that we have a left-recursive grammar (snippet) E E + T E T T...

10 L8.10 Top-Down Predictive LL Parsing We cannot choose which grammar rule to use because both E productions can start with terms (T ), which may be arbitrarily long. We can change its left recursion into a right recursion by pulling commonalities up. Then we start with what the productions of the nonterminal have in common and postpone the distinction to a new subsequent nonterminal E : E T E E +T E E ɛ This process eliminates left recursion using right recursion. These changes of the grammar have also changed the structure of the parse trees, which needs to be taken into account, e.g., through postprocessing transformations. Quiz 1. Is there a language that CYK can parse but LL cannot parse? Give example or explain why none exists. 2. Is there a language that LL can parse but CYK cannot parse? Give example or explain why none exists. 3. Is there a language that recursive descent can parse but LL cannot parse? Give example or explain why none exists. 4. Is there a language that LL can parse but recursive descent cannot parse? Give example or explain why none exists. 5. Why do we limit the token lookahead to 1? Does it make a big difference if we would instead limit it to 5? 6. Should we allow token lookahead? 7. Suppose you only have an LL parser generator. Can you still write a compiler for any programming language? 8. Which features of programming languages are difficult or inconvenient for LL parsers? 9. Can follow(x, ) be computed from the grammar productions for X alone? Can first(x, ) be computed from only X productions?

11 Top-Down Predictive LL Parsing L Why should parser combinator libraries worry about whether a grammar is LL? Isn t that irrelevant? 11. What is very likely wrong with a compiler when you see a grammar production S ::= E = > E ; 12. Your combinator parser is beautiful, but for some input files it just produces a stack overflow without giving you any other information. What is wrong? References [App98] Andrew W. Appel. Modern Compiler Implementation in ML. Cambridge University Press, Cambridge, England, [GM02] Harald Ganzinger and David A. McAllester. Logical algorithms. In P.Stuckey, editor, Proceedings of the 18th International Conference on Logic Programming, pages , Copenhagen, Denmark, July Springer-Verlag LNCS [SSP95] Stuart M. Shieber, Yves Schabes, and Fernando C. N. Pereira. Principles and implementation of deductive parsing. J. Log. Program., 24(1&2):3 36, [WM95] Reinhard Wilhelm and Dieter Maurer. Compiler Design. Addison- Wesley, 1995.

### Lecture Notes on Shift-Reduce Parsing

Lecture Notes on Shift-Reduce Parsing 15-411: Compiler Design Frank Pfenning, Rob Simmons, André Platzer Lecture 8 September 24, 2015 1 Introduction In this lecture we discuss shift-reduce parsing, which

### Lecture Notes on Bottom-Up LR Parsing

Lecture Notes on Bottom-Up LR Parsing 15-411: Compiler Design Frank Pfenning Lecture 9 1 Introduction In this lecture we discuss a second parsing algorithm that traverses the input string from left to

### Lecture Notes on Bottom-Up LR Parsing

Lecture Notes on Bottom-Up LR Parsing 15-411: Compiler Design Frank Pfenning Lecture 9 September 23, 2009 1 Introduction In this lecture we discuss a second parsing algorithm that traverses the input string

### Types of parsing. CMSC 430 Lecture 4, Page 1

Types of parsing Top-down parsers start at the root of derivation tree and fill in picks a production and tries to match the input may require backtracking some grammars are backtrack-free (predictive)

### CS1622. Today. A Recursive Descent Parser. Preliminaries. Lecture 9 Parsing (4)

CS1622 Lecture 9 Parsing (4) CS 1622 Lecture 9 1 Today Example of a recursive descent parser Predictive & LL(1) parsers Building parse tables CS 1622 Lecture 9 2 A Recursive Descent Parser. Preliminaries

### Top-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7

Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess

### Top-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7

Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess

### Abstract Syntax Trees & Top-Down Parsing

Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree

### Abstract Syntax Trees & Top-Down Parsing

Abstract Syntax Trees & Top-Down Parsing Review of Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree

### Abstract Syntax Trees & Top-Down Parsing

Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree

### Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

### Note that for recursive descent to work, if A ::= B1 B2 is a grammar rule we need First k (B1) disjoint from First k (B2).

LL(k) Grammars We need a bunch of terminology. For any terminal string a we write First k (a) is the prefix of a of length k (or all of a if its length is less than k) For any string g of terminal and

### CS502: Compilers & Programming Systems

CS502: Compilers & Programming Systems Top-down Parsing Zhiyuan Li Department of Computer Science Purdue University, USA There exist two well-known schemes to construct deterministic top-down parsers:

### Top down vs. bottom up parsing

Parsing A grammar describes the strings that are syntactically legal A recogniser simply accepts or rejects strings A generator produces sentences in the language described by the grammar A parser constructs

### Wednesday, September 9, 15. Parsers

Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Parsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:

What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### 3. Parsing. Oscar Nierstrasz

3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/

### Building a Parser III. CS164 3:30-5:00 TT 10 Evans. Prof. Bodik CS 164 Lecture 6 1

Building a Parser III CS164 3:30-5:00 TT 10 Evans 1 Overview Finish recursive descent parser when it breaks down and how to fix it eliminating left recursion reordering productions Predictive parsers (aka

### Automatic generation of LL(1) parsers

Automatic generation of LL(1) parsers Informatics 2A: Lecture 12 John Longley School of Informatics University of Edinburgh jrl@staffmail.ed.ac.uk 14 October, 2011 1 / 12 Generating parse tables We ve

### CA Compiler Construction

CA4003 - Compiler Construction David Sinclair A top-down parser starts with the root of the parse tree, labelled with the goal symbol of the grammar, and repeats the following steps until the fringe of

### Syntactic Analysis. Top-Down Parsing

Syntactic Analysis Top-Down Parsing Copyright 2017, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make

### Lexical and Syntax Analysis. Top-Down Parsing

Lexical and Syntax Analysis Top-Down Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax

### Parsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468

Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure

### LL(k) Parsing. Predictive Parsers. LL(k) Parser Structure. Sample Parse Table. LL(1) Parsing Algorithm. Push RHS in Reverse Order 10/17/2012

Predictive Parsers LL(k) Parsing Can we avoid backtracking? es, if for a given input symbol and given nonterminal, we can choose the alternative appropriately. his is possible if the first terminal of

### 8 Parsing. Parsing. Top Down Parsing Methods. Parsing complexity. Top down vs. bottom up parsing. Top down vs. bottom up parsing

8 Parsing Parsing A grammar describes syntactically legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string

### Administrativia. WA1 due on Thu PA2 in a week. Building a Parser III. Slides on the web site. CS164 3:30-5:00 TT 10 Evans.

Administrativia Building a Parser III CS164 3:30-5:00 10 vans WA1 due on hu PA2 in a week Slides on the web site I do my best to have slides ready and posted by the end of the preceding logical day yesterday,

### Lexical and Syntax Analysis

Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Easy for humans to write and understand String of characters

### Monday, September 13, Parsers

Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions

### Parsing III. (Top-down parsing: recursive descent & LL(1) )

Parsing III (Top-down parsing: recursive descent & LL(1) ) Roadmap (Where are we?) Previously We set out to study parsing Specifying syntax Context-free grammars Ambiguity Top-down parsers Algorithm &

### CSCI312 Principles of Programming Languages

Copyright 2006 The McGraw-Hill Companies, Inc. CSCI312 Principles of Programming Languages! LL Parsing!! Xu Liu Derived from Keith Cooper s COMP 412 at Rice University Recap Copyright 2006 The McGraw-Hill

### 1 Introduction. 2 Recursive descent parsing. Predicative parsing. Computer Language Implementation Lecture Note 3 February 4, 2004

CMSC 51086 Winter 2004 Computer Language Implementation Lecture Note 3 February 4, 2004 Predicative parsing 1 Introduction This note continues the discussion of parsing based on context free languages.

### Syntax Analysis. COMP 524: Programming Language Concepts Björn B. Brandenburg. The University of North Carolina at Chapel Hill

Syntax Analysis Björn B. Brandenburg The University of North Carolina at Chapel Hill Based on slides and notes by S. Olivier, A. Block, N. Fisher, F. Hernandez-Campos, and D. Stotts. The Big Picture Character

### Syntax Analysis. The Big Picture. The Big Picture. COMP 524: Programming Languages Srinivas Krishnan January 25, 2011

Syntax Analysis COMP 524: Programming Languages Srinivas Krishnan January 25, 2011 Based in part on slides and notes by Bjoern Brandenburg, S. Olivier and A. Block. 1 The Big Picture Character Stream Token

### Ambiguity, Precedence, Associativity & Top-Down Parsing. Lecture 9-10

Ambiguity, Precedence, Associativity & Top-Down Parsing Lecture 9-10 (From slides by G. Necula & R. Bodik) 9/18/06 Prof. Hilfinger CS164 Lecture 9 1 Administrivia Please let me know if there are continued

### Lecture Notes on Liveness Analysis

Lecture Notes on Liveness Analysis 15-411: Compiler Design Frank Pfenning André Platzer Lecture 4 1 Introduction We will see different kinds of program analyses in the course, most of them for the purpose

### Wednesday, August 31, Parsers

Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically

### LL(1) predictive parsing

LL(1) predictive parsing Informatics 2A: Lecture 11 John Longley School of Informatics University of Edinburgh jrl@staffmail.ed.ac.uk 13 October, 2011 1 / 12 1 LL(1) grammars and parse tables 2 3 2 / 12

SYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic

### Ambiguity. Grammar E E + E E * E ( E ) int. The string int * int + int has two parse trees. * int

Administrivia Ambiguity, Precedence, Associativity & op-down Parsing eam assignments this evening for all those not listed as having one. HW#3 is now available, due next uesday morning (Monday is a holiday).

### LL parsing Nullable, FIRST, and FOLLOW

EDAN65: Compilers LL parsing Nullable, FIRST, and FOLLOW Görel Hedin Revised: 2014-09- 22 Regular expressions Context- free grammar ATribute grammar Lexical analyzer (scanner) SyntacKc analyzer (parser)

### Compilers. Predictive Parsing. Alex Aiken

Compilers Like recursive-descent but parser can predict which production to use By looking at the next fewtokens No backtracking Predictive parsers accept LL(k) grammars L means left-to-right scan of input

### Compilation Lecture 3: Syntax Analysis: Top-Down parsing. Noam Rinetzky

Compilation 0368-3133 Lecture 3: Syntax Analysis: Top-Down parsing Noam Rinetzky 1 Recursive descent parsing Define a function for every nonterminal Every function work as follows Find applicable production

### CS Parsing 1

CS414-20034-03 Parsing 1 03-0: Parsing Once we have broken an input file into a sequence of tokens, the next step is to determine if that sequence of tokens forms a syntactically correct program parsing

### LL(1) predictive parsing

LL(1) predictive parsing Informatics 2A: Lecture 11 Mary Cryan School of Informatics University of Edinburgh mcryan@staffmail.ed.ac.uk 10 October 2018 1 / 15 Recap of Lecture 10 A pushdown automaton (PDA)

### Syntax Analysis. Amitabha Sanyal. (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay

Syntax Analysis (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay September 2007 College of Engineering, Pune Syntax Analysis: 2/124 Syntax

### Table-driven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and non-terminals.

Bottom-up Parsing: Table-driven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and non-terminals. Basic operation is to shift terminals from the input to the

### Compiler Design 1. Top-Down Parsing. Goutam Biswas. Lect 5

Compiler Design 1 Top-Down Parsing Compiler Design 2 Non-terminal as a Function In a top-down parser a non-terminal may be viewed as a generator of a substring of the input. We may view a non-terminal

### EDAN65: Compilers, Lecture 04 Grammar transformations: Eliminating ambiguities, adapting to LL parsing. Görel Hedin Revised:

EDAN65: Compilers, Lecture 04 Grammar transformations: Eliminating ambiguities, adapting to LL parsing Görel Hedin Revised: 2017-09-04 This lecture Regular expressions Context-free grammar Attribute grammar

### CS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F).

CS 2210 Sample Midterm 1. Determine if each of the following claims is true (T) or false (F). F A language consists of a set of strings, its grammar structure, and a set of operations. (Note: a language

### Syntax Analysis, III Comp 412

COMP 412 FALL 2017 Syntax Analysis, III Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp

### Bottom-Up Parsing II. Lecture 8

Bottom-Up Parsing II Lecture 8 1 Review: Shift-Reduce Parsing Bottom-up parsing uses two actions: Shift ABC xyz ABCx yz Reduce Cbxy ijk CbA ijk 2 Recall: he Stack Left string can be implemented by a stack

### Extra Credit Question

Top-Down Parsing #1 Extra Credit Question Given this grammar G: E E+T E T T T * int T int T (E) Is the string int * (int + int) in L(G)? Give a derivation or prove that it is not. #2 Revenge of Theory

### EDA180: Compiler Construc6on. Top- down parsing. Görel Hedin Revised: a

EDA180: Compiler Construc6on Top- down parsing Görel Hedin Revised: 2013-01- 30a Compiler phases and program representa6ons source code Lexical analysis (scanning) Intermediate code genera6on tokens intermediate

### Part III : Parsing. From Regular to Context-Free Grammars. Deriving a Parser from a Context-Free Grammar. Scanners and Parsers.

Part III : Parsing From Regular to Context-Free Grammars Deriving a Parser from a Context-Free Grammar Scanners and Parsers A Parser for EBNF Left-Parsable Grammars Martin Odersky, LAMP/DI 1 From Regular

### Parsing. Lecture 11: Parsing. Recursive Descent Parser. Arithmetic grammar. - drops irrelevant details from parse tree

Parsing Lecture 11: Parsing CSC 131 Fall, 2014 Kim Bruce Build parse tree from an expression Interested in abstract syntax tree - drops irrelevant details from parse tree Arithmetic grammar ::=

### Parsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones

Parsing III (Top-down parsing: recursive descent & LL(1) ) (Bottom-up parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,

### Computer Science 160 Translation of Programming Languages

Computer Science 160 Translation of Programming Languages Instructor: Christopher Kruegel Top-Down Parsing Parsing Techniques Top-down parsers (LL(1), recursive descent parsers) Start at the root of the

### 3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino

3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the

### Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 1. Top-Down Parsing. Lect 5. Goutam Biswas

Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 1 Top-Down Parsing Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 2 Non-terminal as a Function In a top-down parser a non-terminal may be viewed

### Section A. A grammar that produces more than one parse tree for some sentences is said to be ambiguous.

Section A 1. What do you meant by parser and its types? A parser for grammar G is a program that takes as input a string w and produces as output either a parse tree for w, if w is a sentence of G, or

### Syntax Analysis, III Comp 412

Updated algorithm for removal of indirect left recursion to match EaC3e (3/2018) COMP 412 FALL 2018 Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, III Comp 412 source code

### Parsing. Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice.

Parsing Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice. Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students

### In this simple example, it is quite clear that there are exactly two strings that match the above grammar, namely: abc and abcc

JavaCC: LOOKAHEAD MiniTutorial 1. WHAT IS LOOKAHEAD The job of a parser is to read an input stream and determine whether or not the input stream conforms to the grammar. This determination in its most

### Parsing Part II (Top-down parsing, left-recursion removal)

Parsing Part II (Top-down parsing, left-recursion removal) Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit

### Topic 3: Syntax Analysis I

Topic 3: Syntax Analysis I Compiler Design Prof. Hanjun Kim CoreLab (Compiler Research Lab) POSTECH 1 Back-End Front-End The Front End Source Program Lexical Analysis Syntax Analysis Semantic Analysis

### Lecture 8: Deterministic Bottom-Up Parsing

Lecture 8: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Fri Feb 12 13:02:57 2010 CS164: Lecture #8 1 Avoiding nondeterministic choice: LR We ve been looking at general

### CS2210: Compiler Construction Syntax Analysis Syntax Analysis

Comparison with Lexical Analysis The second phase of compilation Phase Input Output Lexer string of characters string of tokens Parser string of tokens Parse tree/ast What Parse Tree? CS2210: Compiler

### Parser Generation. Bottom-Up Parsing. Constructing LR Parser. LR Parsing. Construct parse tree bottom-up --- from leaves to the root

Parser Generation Main Problem: given a grammar G, how to build a top-down parser or a bottom-up parser for it? parser : a program that, given a sentence, reconstructs a derivation for that sentence ----

### Fall Compiler Principles Lecture 3: Parsing part 2. Roman Manevich Ben-Gurion University

Fall 2014-2015 Compiler Principles Lecture 3: Parsing part 2 Roman Manevich Ben-Gurion University Tentative syllabus Front End Intermediate Representation Optimizations Code Generation Scanning Lowering

### 4 (c) parsing. Parsing. Top down vs. bo5om up parsing

4 (c) parsing Parsing A grammar describes syntac2cally legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string

### Compilers. Yannis Smaragdakis, U. Athens (original slides by Sam

Compilers Parsing Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Next step text chars Lexical analyzer tokens Parser IR Errors Parsing: Organize tokens into sentences Do tokens conform

### Lecture 7: Deterministic Bottom-Up Parsing

Lecture 7: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Tue Sep 20 12:50:42 2011 CS164: Lecture #7 1 Avoiding nondeterministic choice: LR We ve been looking at general

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2017-2018 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview Tokens and regular expressions Syntax and context-free grammars Grammar derivations More about parse trees Top-down and bottom-up

### Syntax Analysis. Martin Sulzmann. Martin Sulzmann Syntax Analysis 1 / 38

Syntax Analysis Martin Sulzmann Martin Sulzmann Syntax Analysis 1 / 38 Syntax Analysis Objective Recognize individual tokens as sentences of a language (beyond regular languages). Example 1 (OK) Program

### LR Parsing, Part 2. Constructing Parse Tables. An NFA Recognizing Viable Prefixes. Computing the Closure. GOTO Function and DFA States

TDDD16 Compilers and Interpreters TDDB44 Compiler Construction LR Parsing, Part 2 Constructing Parse Tables Parse table construction Grammar conflict handling Categories of LR Grammars and Parsers An NFA

### Outline. 1 Introduction. 2 Context-free Grammars and Languages. 3 Top-down Deterministic Parsing. 4 Bottom-up Deterministic Parsing

Parsing 1 / 90 Outline 1 Introduction 2 Context-free Grammars and Languages 3 Top-down Deterministic Parsing 4 Bottom-up Deterministic Parsing 5 Parser Generation Using JavaCC 2 / 90 Introduction Once

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview n Tokens and regular expressions n Syntax and context-free grammars n Grammar derivations n More about parse trees n Top-down and

### Building Compilers with Phoenix

Building Compilers with Phoenix Syntax-Directed Translation Structure of a Compiler Character Stream Intermediate Representation Lexical Analyzer Machine-Independent Optimizer token stream Intermediate

### Parsing Expression Grammars and Packrat Parsing. Aaron Moss

Parsing Expression Grammars and Packrat Parsing Aaron Moss References > B. Ford Packrat Parsing: Simple, Powerful, Lazy, Linear Time ICFP (2002) > Parsing Expression Grammars: A Recognition- Based Syntactic

### CS 321 Programming Languages and Compilers. VI. Parsing

CS 321 Programming Languages and Compilers VI. Parsing Parsing Calculate grammatical structure of program, like diagramming sentences, where: Tokens = words Programs = sentences For further information,

### Table-Driven Parsing

Table-Driven Parsing It is possible to build a non-recursive predictive parser by maintaining a stack explicitly, rather than implicitly via recursive calls [1] The non-recursive parser looks up the production

### CSE431 Translation of Computer Languages

CSE431 Translation of Computer Languages Top Down Parsers Doug Shook Top Down Parsers Two forms: Recursive Descent Table Also known as LL(k) parsers: Read tokens from Left to right Produces a Leftmost

### Parsing II Top-down parsing. Comp 412

COMP 412 FALL 2018 Parsing II Top-down parsing Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled

### CS 4120 Introduction to Compilers

CS 4120 Introduction to Compilers Andrew Myers Cornell University Lecture 6: Bottom-Up Parsing 9/9/09 Bottom-up parsing A more powerful parsing technology LR grammars -- more expressive than LL can handle

### Parsing Techniques. CS152. Chris Pollett. Sep. 24, 2008.

Parsing Techniques. CS152. Chris Pollett. Sep. 24, 2008. Outline. Top-down versus Bottom-up Parsing. Recursive Descent Parsing. Left Recursion Removal. Left Factoring. Predictive Parsing. Introduction.

### 15 212: Principles of Programming. Some Notes on Continuations

15 212: Principles of Programming Some Notes on Continuations Michael Erdmann Spring 2011 These notes provide a brief introduction to continuations as a programming technique. Continuations have a rich

### CS 314 Principles of Programming Languages

CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution

### JavaCC Parser. The Compilation Task. Automated? JavaCC Parser

JavaCC Parser The Compilation Task Input character stream Lexer stream Parser Abstract Syntax Tree Analyser Annotated AST Code Generator Code CC&P 2003 1 CC&P 2003 2 Automated? JavaCC Parser The initial

### Context-free grammars

Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,

### Compilers and computer architecture From strings to ASTs (2): context free grammars

1 / 1 Compilers and computer architecture From strings to ASTs (2): context free grammars Martin Berger October 2018 Recall the function of compilers 2 / 1 3 / 1 Recall we are discussing parsing Source

### Syntax Analysis/Parsing. Context-free grammars (CFG s) Context-free grammars vs. Regular Expressions. BNF description of PL/0 syntax

Susan Eggers 1 CSE 401 Syntax Analysis/Parsing Context-free grammars (CFG s) Purpose: determine if tokens have the right form for the language (right syntactic structure) stream of tokens abstract syntax

### CS 164 Programming Languages and Compilers Handout 9. Midterm I Solution

Midterm I Solution Please read all instructions (including these) carefully. There are 5 questions on the exam, some with multiple parts. You have 1 hour and 20 minutes to work on the exam. The exam is

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2016-2017 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman

### MIT Parse Table Construction. Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology

MIT 6.035 Parse Table Construction Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Parse Tables (Review) ACTION Goto State ( ) \$ X s0 shift to s2 error error goto s1

### Compiler Construction 2016/2017 Syntax Analysis

Compiler Construction 2016/2017 Syntax Analysis Peter Thiemann November 2, 2016 Outline 1 Syntax Analysis Recursive top-down parsing Nonrecursive top-down parsing Bottom-up parsing Syntax Analysis tokens

### Parsing Algorithms. Parsing: continued. Top Down Parsing. Predictive Parser. David Notkin Autumn 2008

Parsing: continued David Notkin Autumn 2008 Parsing Algorithms Earley s algorithm (1970) works for all CFGs O(N 3 ) worst case performance O(N 2 ) for unambiguous grammars Based on dynamic programming,

### Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery. Last modified: Mon Feb 23 10:05: CS164: Lecture #14 1

Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery Last modified: Mon Feb 23 10:05:56 2015 CS164: Lecture #14 1 Shift/Reduce Conflicts If a DFA state contains both [X: α aβ, b] and [Y: γ, a],