# Prelude COMP 181 Tufts University Computer Science Last time Grammar issues Key structure meaning Tufts University Computer Science

Size: px
Start display at page:

Download "Prelude COMP 181 Tufts University Computer Science Last time Grammar issues Key structure meaning Tufts University Computer Science"

Transcription

1 Prelude COMP Lecture Topdown Parsing September, 00 What is the Tufts mascot? Jumbo the elephant Why? P. T. Barnum was an original trustee of Tufts : donated \$0,000 for a natural museum on campus Barnum Museum, later Barnum Hall Jumbo : famous circus elephant : Jumbo died, was stuffed, donated to Tufts 9: Fire destroyed Barnum Hall, Jumbo Tufts University Computer Science Last time Finished scanning Produces a stream of tokens Removes things we don t care about, like white space and comments Contetfree grammars Formal description of language synta Deriving strings using CFG Depicting derivation as a parse tree Grammar issues Often: more than one way to derive a string Why is this a problem? Parsing: is string a member of L(G)? We want more than a yes or no answer Key: Represent the derivation as a parse tree We want the structure of the parse tree to capture the meaning of the sentence Tufts University Computer Science Tufts University Computer Science Grammar issues Parse tree: * y Often: more than one way to derive a string Why is this a problem? Parsing: is string a member of L(G)? We want more than a yes or no answer op Key: number Represent the derivation as a parse tree We want the structure of the parse op tree to capture + the meaning of the sentence * / Rule Rightmost derivation Sentential form op op <id,y> * <id,y> op * <id,y> op <num,> * <id,y> <num,> * <id,y> <id,> <num,> * <id,y> Parse tree op op * y Tufts University Computer Science Tufts University Computer Science

2 Abstract synta tree Left vs right derivations Parse tree contains etra junk Eliminate inediate nodes Move operators up to parent nodes Result: abstract synta tree op op * y * y Two derivations of * y Rule Sentential form op <id, > op <id,> <id,> op <id,> <num,> op <id,> <num,> * <id,> <num,> * <id,y> Rule Sentential form op op <id,y> * <id,y> op * <id,y> op <num,> * <id,y> <num,> * <id,y> <id,> <num,> * <id,y> Leftmost derivation Rightmost derivation Tufts University Computer Science Tufts University Computer Science Derivations One captures meaning, the other doesn t With precedence Last time: ways to force the right tree shape Add productions to represent precedence * y Leftmost derivation * y Rightmost derivation op number op + * / + * or / or or or number Tufts University Computer Science 9 Tufts University Computer Science 0 With precedence Parsing op op * y * * What is parsing? Discovering the derivation of a string If one eists Harder than generating strings Not surprisingly Two major approaches Topdown parsing Bottomup parsing y Don t work on all contetfree grammars Properties of grammar deine parseability Our goal: make parsing efficient We may be able to transform a grammar Tufts University Computer Science Tufts University Computer Science

3 Two approaches Topdown parsers LL(), recursive descent Start at the root of the parse tree and grow toward leaves Pick a production & try to match the input Bad pick may need to backtrack Bottomup parsers LR(), operator precedence Start at the leaves and grow toward root As input is consumed, encode possible parse trees in an internal state (similar to our NFA DFA conversion) Bottomup parsers handle a large class of grammars Grammars and parsers LL() parsers Lefttoright input Leftmost derivation symbol of lookahead LR() parsers Lefttoright input Rightmost derivation symbol of lookahead Also: LL(k), LR(k), SLR, LALR, Grammars that this can handle are called LL() grammars Grammars that this can handle are called LR() grammars Tufts University Computer Science Tufts University Computer Science Topdown parsing Start with the root of the parse tree Root of the tree: node labeled with the start symbol Algorithm: Repeat until the fringe of the parse tree matches input string At a node A, select a production for A Add a child node for each symbol on rhs If a inal symbol is added that doesn t match, backtrack Find the net node to be epanded (a noninal) Done when: Leaves of parse tree match input string (success) All productions ehausted in backtracking (failure) Tufts University Computer Science Eample Epression grammar + * or / or or or number Input string * y (with precedence) Tufts University Computer Science Eample Current position in the input stream Backtracking Rule Sentential form Input string * y + * y + * y or + * y <id> + * y <id,> + * y + Rule Sentential form Input string * y + * y + * y or + * y <id> + * y? <id,> + * y Undo all these productions Problem: Can t match net inal We guessed wrong at step Rollback productions Choose a different production for Continue Tufts University Computer Science Tufts University Computer Science

4 Retrying Successful parse Rule Sentential form Input string * y * y * y or * y <id> * y <id,> * y <id,> or * y <id,> <num> * y Problem: More input to read Another cause of backtracking Rule Sentential form Input string * y * y * y or * y <id> * y <id,> * y <id,> * * y <id,> * * y <id,> <num> * * y <id,> <num,> * * y <id,> <num,> * <id> * y All inals match we re done * y Tufts University Computer Science 9 Tufts University Computer Science 0 Other possible parses Rule Sentential form Input string * y + * y + + * y * y * y Problem: ination Wrong choice leads to infinite epansion (More importantly: without consuming any input!) May not be as obvious as this Our grammar is left recursive Tufts University Computer Science Left recursion Formally, A grammar is left recursive if a noninal A such that A * A α (for some set of symbols α) What does * mean? A B B A y Bad news: Topdown parsers cannot handle left recursion Good news: We can systematically eliminate left recursion Tufts University Computer Science Notation Eliminating left recursion Noninals Capital letter: A, B, C Terminals Lowercase, underline:, y, z Some mi of inals and noninals Greek letters: α, β, γ Eample: A B + A B α α = + Consider this grammar: Rewrite as foo foo α β foo β bar bar α bar New noninal Language is β followed by zero or more α This production gives you one β These two productions give you zero or more α Tufts University Computer Science Tufts University Computer Science

5 Back to essions Eliminating left recursion Two cases of left recursion: + Transform as follows: + * or / or or or * or / or Resulting grammar All right recursive Retain original language and associativity Not as intuitive to read Topdown parser Will always inate May still backtrack There s a lovely algorithm to do this automatically, which we will skip or * or / or or number Tufts University Computer Science Tufts University Computer Science Topdown parsers Problem: Leftrecursion Solution: Technique to remove it What about backtracking? Current algorithm is brute force Problem: how to choose the right production? Idea: use the net input token (duh) How? Look at our rightrecursive grammar Tufts University Computer Science Rightrecursive grammar or * or / or or number Two productions with no choice at all All other productions are uniquely identified by a inal symbol at the start of RHS We can choose the right production by looking at the net input symbol This is called lookahead BUT, this can be tricky Tufts University Computer Science Lookahead Goal: avoid backtracking Look at future input symbols Use etra contet to make right choice How much lookahead is needed? In general, an arbitrary amount is needed for the full class of contetfree grammars Use fancydancy algorithm CYK algorithm, O(n ) Fortunately, Many CFGs can be parsed with limited lookahead Covers most programming languages not C++ or Perl Topdown parsing Goal: Given productions A α β, the parser should be able to choose between α and β Trying to match A How can the net input token help us decide? Solution: FIRST sets (almost a solution) Informally: FIRST(α) is the set of tokens that could appear as the first symbol in a string derived from α Def: in FIRST(α) iff α * γ Tufts University Computer Science 9 Tufts University Computer Science 0

6 Topdown parsing Building FIRST sets We ll look at this algorithm later The LL() property Given A α and A β, we would like: FIRST(α) FIRST(β) = Parser can make right choice by looking at one lookahead token..almost.. Topdown parsing What about ε productions? Complicates the definition of LL() Consider A α and A β and α may be empty In this case there is no symbol to identify α Eample: What is FIRST()? = { ε } A B y C What lookahead symbol tells us we are matching production? Tufts University Computer Science Tufts University Computer Science Topdown parsing If A was empty What will the net symbol be? Must be one of the symbols that immediately follow an A Solution Build a FOLLOW set for each production with ε Etra condition for LL: FIRST(β) must be disjoint from FIRST(α) and FOLLOW(Α) FOLLOW sets Eample: FIRST() = { } FIRST() = { y } FIRST() = { ε } A B y C E A z What can follow A? Look at the contet of all uses of A FOLLOW(A) = { z } Now we can uniquely identify each production: If we are trying to match an A and the net token is z, then we matched production Tufts University Computer Science Tufts University Computer Science More on FIRST and FOLLOW Notice: FIRST and FOLLOW may be sets FIRST may contain ε in addition to other symbols Eample: FIRST() = {, y, ε } FOLLOW(A) = { z, w } Question: When would we care about FOLLOW(A)? Answer: if FIRST(C) contains ε A B C B y E A z F A w Tufts University Computer Science LL() property Including ε productions FOLLOW(A) = the set of inal symbols that can immediately follow A Def: FIRST+(A α) as FIRST(α) U FOLLOW(A), if ε FIRST(α) FIRST(α), otherwise Def: a grammar is LL() iff A α and A β and FIRST+(A α) FIRST+(A β) = Tufts University Computer Science

7 LL() property Question Can there be two rules A αand A βin a LL() grammar such that ε FIRST(α) and ε FIRST(β)? Answer Yes, as long as they have different FOLLOW sets Parsing LL() grammar Given an LL() grammar Code: simple, fast routine to recognize each production Given A β β β, with FIRST + (β i ) FIRST + (β j ) = /* find rule for A */ if (current token FIRST+(β )) select A β else if (current token FIRST+(β )) select A β else if (current token FIRST+(β )) for all i!= j select A β else report an error and return false Tufts University Computer Science Tufts University Computer Science Predictive parsing Recursive descent Predictive parsing The parser can predict the correct epansion Using lookahead and FIRST and FOLLOW sets Two kinds of predictive parsers Recursive descent Often handwritten Tabledriven Generate tables from First and Follow sets 9 0 goal + or * or / or or number ( ) This produces a parser with si mutually recursive routines: Goal Epr Epr Term Term Factor Each recognizes one NT or T The descent refers to the direction in which the parse tree is built. Tufts University Computer Science 9 Tufts University Computer Science 0 Eample code Goal symbol: Eample code Match main() /* Match goal > */ tok = nettoken(); if (() && tok == EOF) then proceed to net step; else return false; Toplevel ession () /* Match > */ if (() && ()); else return false; () /* Match > + */ /* Match > */ if (tok == + or tok == ) tok = nettoken(); if (()) then return (); else return false; /* Match > empty */ Check FIRST and FOLLOW sets to distinguish Tufts University Computer Science Tufts University Computer Science

8 Eample code or() /* Match or > ( ) */ if (tok == ( ) tok = nettoken(); if (() && tok == ) ) else synta error: epecting ) return false /* Match or > num */ if (tok is a num) return true /* Match or > id */ if (tok is an id) Topdown parsing So far: Gives us a yes or no answer We want to build the parse tree How? Add actions to matching routines Create a node for each production How do we assemble the tree? Tufts University Computer Science Tufts University Computer Science Building a parse tree Notice: Recursive calls match the shape of the tree Idea: use a stack Each routine: main or Pops off the children it needs Creates its own node Pushes that node back on the stack Building a parse tree With stack operations () /* Match > */ if (() && ()) _node = pop(); _node = pop(); _node = new Node(_node, _node) push(_node); else return false; Tufts University Computer Science Tufts University Computer Science Net time Finish topdown parsing Tabledriven parsers Building FIRST and FOLLOW sets Start bottomup parsing Tufts University Computer Science

### COMP 181. Prelude. Next step. Parsing. Study of parsing. Specifying syntax with a grammar

COMP Lecture Parsing September, 00 Prelude What is the common name of the fruit Synsepalum dulcificum? Miracle fruit from West Africa What is special about miracle fruit? Contains a protein called miraculin

### Compilers. Yannis Smaragdakis, U. Athens (original slides by Sam

Compilers Parsing Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Next step text chars Lexical analyzer tokens Parser IR Errors Parsing: Organize tokens into sentences Do tokens conform

### Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

### Parsing Part II (Top-down parsing, left-recursion removal)

Parsing Part II (Top-down parsing, left-recursion removal) Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit

### Types of parsing. CMSC 430 Lecture 4, Page 1

Types of parsing Top-down parsers start at the root of derivation tree and fill in picks a production and tries to match the input may require backtracking some grammars are backtrack-free (predictive)

### CSCI312 Principles of Programming Languages

Copyright 2006 The McGraw-Hill Companies, Inc. CSCI312 Principles of Programming Languages! LL Parsing!! Xu Liu Derived from Keith Cooper s COMP 412 at Rice University Recap Copyright 2006 The McGraw-Hill

### CS 406/534 Compiler Construction Parsing Part I

CS 406/534 Compiler Construction Parsing Part I Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof. Ken Kennedy and Dr.

### 3. Parsing. Oscar Nierstrasz

3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/

### Parsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones

Parsing III (Top-down parsing: recursive descent & LL(1) ) (Bottom-up parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,

### Syntactic Analysis. Top-Down Parsing

Syntactic Analysis Top-Down Parsing Copyright 2017, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make

### Parsing. Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice.

Parsing Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice. Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students

### Parsing III. (Top-down parsing: recursive descent & LL(1) )

Parsing III (Top-down parsing: recursive descent & LL(1) ) Roadmap (Where are we?) Previously We set out to study parsing Specifying syntax Context-free grammars Ambiguity Top-down parsers Algorithm &

### Top-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7

Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess

### Introduction to Parsing. Comp 412

COMP 412 FALL 2010 Introduction to Parsing Comp 412 Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit permission to make

### Syntax Analysis, III Comp 412

COMP 412 FALL 2017 Syntax Analysis, III Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp

### Top down vs. bottom up parsing

Parsing A grammar describes the strings that are syntactically legal A recogniser simply accepts or rejects strings A generator produces sentences in the language described by the grammar A parser constructs

### CS1622. Today. A Recursive Descent Parser. Preliminaries. Lecture 9 Parsing (4)

CS1622 Lecture 9 Parsing (4) CS 1622 Lecture 9 1 Today Example of a recursive descent parser Predictive & LL(1) parsers Building parse tables CS 1622 Lecture 9 2 A Recursive Descent Parser. Preliminaries

### Parsing II Top-down parsing. Comp 412

COMP 412 FALL 2018 Parsing II Top-down parsing Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled

### Top-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7

Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess

### CS 314 Principles of Programming Languages

CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution

### Syntax Analysis, III Comp 412

Updated algorithm for removal of indirect left recursion to match EaC3e (3/2018) COMP 412 FALL 2018 Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, III Comp 412 source code

### CS415 Compilers. Syntax Analysis. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University

CS415 Compilers Syntax Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Limits of Regular Languages Advantages of Regular Expressions

### Wednesday, September 9, 15. Parsers

Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Parsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:

What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Monday, September 13, Parsers

Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions

### Syntax Analysis. Martin Sulzmann. Martin Sulzmann Syntax Analysis 1 / 38

Syntax Analysis Martin Sulzmann Martin Sulzmann Syntax Analysis 1 / 38 Syntax Analysis Objective Recognize individual tokens as sentences of a language (beyond regular languages). Example 1 (OK) Program

### Wednesday, August 31, Parsers

Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically

### Parsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468

Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure

### Computer Science 160 Translation of Programming Languages

Computer Science 160 Translation of Programming Languages Instructor: Christopher Kruegel Top-Down Parsing Parsing Techniques Top-down parsers (LL(1), recursive descent parsers) Start at the root of the

### Parsing Part II. (Ambiguity, Top-down parsing, Left-recursion Removal)

Parsing Part II (Ambiguity, Top-down parsing, Left-recursion Removal) Ambiguous Grammars Definitions If a grammar has more than one leftmost derivation for a single sentential form, the grammar is ambiguous

### Chapter 4: LR Parsing

Chapter 4: LR Parsing 110 Some definitions Recall For a grammar G, with start symbol S, any string α such that S called a sentential form α is If α Vt, then α is called a sentence in L G Otherwise it is

### Syntax Analysis, V Bottom-up Parsing & The Magic of Handles Comp 412

Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, V Bottom-up Parsing & The Magic of Handles Comp 412 COMP 412 FALL 2018 source code IR Front End Optimizer Back End IR target

### 8 Parsing. Parsing. Top Down Parsing Methods. Parsing complexity. Top down vs. bottom up parsing. Top down vs. bottom up parsing

8 Parsing Parsing A grammar describes syntactically legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string

### Chapter 3. Parsing #1

Chapter 3 Parsing #1 Parser source file get next character scanner get token parser AST token A parser recognizes sequences of tokens according to some grammar and generates Abstract Syntax Trees (ASTs)

### CA Compiler Construction

CA4003 - Compiler Construction David Sinclair A top-down parser starts with the root of the parse tree, labelled with the goal symbol of the grammar, and repeats the following steps until the fringe of

### Syntax Analysis. The Big Picture. The Big Picture. COMP 524: Programming Languages Srinivas Krishnan January 25, 2011

Syntax Analysis COMP 524: Programming Languages Srinivas Krishnan January 25, 2011 Based in part on slides and notes by Bjoern Brandenburg, S. Olivier and A. Block. 1 The Big Picture Character Stream Token

### CS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F).

CS 2210 Sample Midterm 1. Determine if each of the following claims is true (T) or false (F). F A language consists of a set of strings, its grammar structure, and a set of operations. (Note: a language

### Lexical and Syntax Analysis. Top-Down Parsing

Lexical and Syntax Analysis Top-Down Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax

### Bottom-up parsing. Bottom-Up Parsing. Recall. Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form

Bottom-up parsing Bottom-up parsing Recall Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form If α V t,thenα is called a sentence in L(G) Otherwise it is just

### LL(k) Parsing. Predictive Parsers. LL(k) Parser Structure. Sample Parse Table. LL(1) Parsing Algorithm. Push RHS in Reverse Order 10/17/2012

Predictive Parsers LL(k) Parsing Can we avoid backtracking? es, if for a given input symbol and given nonterminal, we can choose the alternative appropriately. his is possible if the first terminal of

### Ambiguity, Precedence, Associativity & Top-Down Parsing. Lecture 9-10

Ambiguity, Precedence, Associativity & Top-Down Parsing Lecture 9-10 (From slides by G. Necula & R. Bodik) 9/18/06 Prof. Hilfinger CS164 Lecture 9 1 Administrivia Please let me know if there are continued

### Review of CFGs and Parsing II Bottom-up Parsers. Lecture 5. Review slides 1

Review of CFGs and Parsing II Bottom-up Parsers Lecture 5 1 Outline Parser Overview op-down Parsers (Covered largely through labs) Bottom-up Parsers 2 he Functionality of the Parser Input: sequence of

### 1 Introduction. 2 Recursive descent parsing. Predicative parsing. Computer Language Implementation Lecture Note 3 February 4, 2004

CMSC 51086 Winter 2004 Computer Language Implementation Lecture Note 3 February 4, 2004 Predicative parsing 1 Introduction This note continues the discussion of parsing based on context free languages.

### Lexical and Syntax Analysis

Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Easy for humans to write and understand String of characters

### Administrativia. WA1 due on Thu PA2 in a week. Building a Parser III. Slides on the web site. CS164 3:30-5:00 TT 10 Evans.

Administrativia Building a Parser III CS164 3:30-5:00 10 vans WA1 due on hu PA2 in a week Slides on the web site I do my best to have slides ready and posted by the end of the preceding logical day yesterday,

### MIT Parse Table Construction. Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology

MIT 6.035 Parse Table Construction Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Parse Tables (Review) ACTION Goto State ( ) \$ X s0 shift to s2 error error goto s1

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview n Tokens and regular expressions n Syntax and context-free grammars n Grammar derivations n More about parse trees n Top-down and

SYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic

### CS2210: Compiler Construction Syntax Analysis Syntax Analysis

Comparison with Lexical Analysis The second phase of compilation Phase Input Output Lexer string of characters string of tokens Parser string of tokens Parse tree/ast What Parse Tree? CS2210: Compiler

### Bottom Up Parsing. Shift and Reduce. Sentential Form. Handle. Parse Tree. Bottom Up Parsing 9/26/2012. Also known as Shift-Reduce parsing

Also known as Shift-Reduce parsing More powerful than top down Don t need left factored grammars Can handle left recursion Attempt to construct parse tree from an input string eginning at leaves and working

### Formal Languages and Compilers Lecture VII Part 3: Syntactic A

Formal Languages and Compilers Lecture VII Part 3: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

### Abstract Syntax Trees & Top-Down Parsing

Abstract Syntax Trees & Top-Down Parsing Review of Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree

### Abstract Syntax Trees & Top-Down Parsing

Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree

### Abstract Syntax Trees & Top-Down Parsing

Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree

### Syntactic Analysis. Top-Down Parsing. Parsing Techniques. Top-Down Parsing. Remember the Expression Grammar? Example. Example

Syntactic Analysis op-down Parsing Parsing echniques op-down Parsers (LL(1), recursive descent) Start at the root of the parse tree and grow toward leaves Pick a production & try to match the input Bad

### Table-Driven Parsing

Table-Driven Parsing It is possible to build a non-recursive predictive parser by maintaining a stack explicitly, rather than implicitly via recursive calls [1] The non-recursive parser looks up the production

### Parser. Larissa von Witte. 11. Januar Institut für Softwaretechnik und Programmiersprachen. L. v. Witte 11. Januar /23

Parser Larissa von Witte Institut für oftwaretechnik und Programmiersprachen 11. Januar 2016 L. v. Witte 11. Januar 2016 1/23 Contents Introduction Taxonomy Recursive Descent Parser hift Reduce Parser

### CS 406/534 Compiler Construction Parsing Part II LL(1) and LR(1) Parsing

CS 406/534 Compiler Construction Parsing Part II LL(1) and LR(1) Parsing Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof.

### Building a Parser III. CS164 3:30-5:00 TT 10 Evans. Prof. Bodik CS 164 Lecture 6 1

Building a Parser III CS164 3:30-5:00 TT 10 Evans 1 Overview Finish recursive descent parser when it breaks down and how to fix it eliminating left recursion reordering productions Predictive parsers (aka

### Compiler Construction 2016/2017 Syntax Analysis

Compiler Construction 2016/2017 Syntax Analysis Peter Thiemann November 2, 2016 Outline 1 Syntax Analysis Recursive top-down parsing Nonrecursive top-down parsing Bottom-up parsing Syntax Analysis tokens

### Example CFG. Lectures 16 & 17 Bottom-Up Parsing. LL(1) Predictor Table Review. Stacks in LR Parsing 1. Sʹ " S. 2. S " AyB. 3. A " ab. 4.

Example CFG Lectures 16 & 17 Bottom-Up Parsing CS 241: Foundations of Sequential Programs Fall 2016 1. Sʹ " S 2. S " AyB 3. A " ab 4. A " cd Matt Crane University of Waterloo 5. B " z 6. B " wz 2 LL(1)

### Front End. Hwansoo Han

Front nd Hwansoo Han Traditional Two-pass Compiler Source code Front nd IR Back nd Machine code rrors High level functions Recognize legal program, generate correct code (OS & linker can accept) Manage

### Context-free grammars

Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,

### LECTURE 7. Lex and Intro to Parsing

LECTURE 7 Lex and Intro to Parsing LEX Last lecture, we learned a little bit about how we can take our regular expressions (which specify our valid tokens) and create real programs that can recognize them.

### Parsing. Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1)

TD parsing - LL(1) Parsing First and Follow sets Parse table construction BU Parsing Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1) Problems with SLR Aho, Sethi, Ullman, Compilers

### Concepts Introduced in Chapter 4

Concepts Introduced in Chapter 4 Grammars Context-Free Grammars Derivations and Parse Trees Ambiguity, Precedence, and Associativity Top Down Parsing Recursive Descent, LL Bottom Up Parsing SLR, LR, LALR

### A programming language requires two major definitions A simple one pass compiler

A programming language requires two major definitions A simple one pass compiler [Syntax: what the language looks like A context-free grammar written in BNF (Backus-Naur Form) usually suffices. [Semantics:

### Compiler Construction: Parsing

Compiler Construction: Parsing Mandar Mitra Indian Statistical Institute M. Mitra (ISI) Parsing 1 / 33 Context-free grammars. Reference: Section 4.2 Formal way of specifying rules about the structure/syntax

### Compilers. Predictive Parsing. Alex Aiken

Compilers Like recursive-descent but parser can predict which production to use By looking at the next fewtokens No backtracking Predictive parsers accept LL(k) grammars L means left-to-right scan of input

### Parsing Wrapup. Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing. This lecture LR(1) parsing

Parsing Wrapup Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing LR(1) items Computing closure Computing goto LR(1) canonical collection This lecture LR(1) parsing Building ACTION

### Introduction to Parsing

Introduction to Parsing The Front End Source code Scanner tokens Parser IR Errors Parser Checks the stream of words and their parts of speech (produced by the scanner) for grammatical correctness Determines

### A left-sentential form is a sentential form that occurs in the leftmost derivation of some sentence.

Bottom-up parsing Recall For a grammar G, with start symbol S, any string α such that S α is a sentential form If α V t, then α is a sentence in L(G) A left-sentential form is a sentential form that occurs

### Ambiguity. Grammar E E + E E * E ( E ) int. The string int * int + int has two parse trees. * int

Administrivia Ambiguity, Precedence, Associativity & op-down Parsing eam assignments this evening for all those not listed as having one. HW#3 is now available, due next uesday morning (Monday is a holiday).

### Compilation Lecture 3: Syntax Analysis: Top-Down parsing. Noam Rinetzky

Compilation 0368-3133 Lecture 3: Syntax Analysis: Top-Down parsing Noam Rinetzky 1 Recursive descent parsing Define a function for every nonterminal Every function work as follows Find applicable production

### Syntax Analysis. COMP 524: Programming Language Concepts Björn B. Brandenburg. The University of North Carolina at Chapel Hill

Syntax Analysis Björn B. Brandenburg The University of North Carolina at Chapel Hill Based on slides and notes by S. Olivier, A. Block, N. Fisher, F. Hernandez-Campos, and D. Stotts. The Big Picture Character

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview Tokens and regular expressions Syntax and context-free grammars Grammar derivations More about parse trees Top-down and bottom-up

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2017-2018 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Context Free Grammars and Parsing 1 Recall: Architecture of Compilers, Interpreters Source Parser Static Analyzer Intermediate Representation Front End Back

### CSE P 501 Compilers. LR Parsing Hal Perkins Spring UW CSE P 501 Spring 2018 D-1

CSE P 501 Compilers LR Parsing Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts UW CSE P 501 Spring 2018

### Parsing #1. Leonidas Fegaras. CSE 5317/4305 L3: Parsing #1 1

Parsing #1 Leonidas Fegaras CSE 5317/4305 L3: Parsing #1 1 Parser source file get next character scanner get token parser AST token A parser recognizes sequences of tokens according to some grammar and

### Bottom-up Parser. Jungsik Choi

Formal Languages and Compiler (CSE322) Bottom-up Parser Jungsik Choi chjs@khu.ac.kr * Some slides taken from SKKU SWE3010 (Prof. Hwansoo Han) and TAMU CSCE434-500 (Prof. Lawrence Rauchwerger) Bottom-up

### 4 (c) parsing. Parsing. Top down vs. bo5om up parsing

4 (c) parsing Parsing A grammar describes syntac2cally legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string

### Introduction to Bottom-Up Parsing

Introduction to Bottom-Up Parsing Lecture 11 CS 536 Spring 2001 1 Outline he strategy: shift-reduce parsing Ambiguity and precedence declarations Next lecture: bottom-up parsing algorithms CS 536 Spring

### Bottom-Up Parsing. Lecture 11-12

Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 2/20/08 Prof. Hilfinger CS164 Lecture 11 1 Administrivia Test I during class on 10 March. 2/20/08 Prof. Hilfinger CS164 Lecture 11

### Lecture Bottom-Up Parsing

Lecture 14+15 Bottom-Up Parsing CS 241: Foundations of Sequential Programs Winter 2018 Troy Vasiga et al University of Waterloo 1 Example CFG 1. S S 2. S AyB 3. A ab 4. A cd 5. B z 6. B wz 2 Stacks in

### More on Syntax. Agenda for the Day. Administrative Stuff. More on Syntax In-Class Exercise Using parse trees

More on Syntax Judy Stafford Comp 80 Meeting February, 00 Agenda for the Day Administrative Stuff Moodle Classlist at without waiting list More on Syntax InClass Exercise Using parse trees Last time Syntax

### LL Parsing: A piece of cake after LR

LL Parsing: A piece of cake after LR Lecture 11 Dr. Sean Peisert ECS 142 Spring 2009 1 LL Parsing Still specified using a CFG Still reads left-to-right (Lx) Now is leftmost derivation (xl) rather than

### Parsing II Top-down parsing. Comp 412

COMP 412 FALL 2017 Parsing II Top-down parsing Comp 412 source code IR Front End OpMmizer Back End IR target code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled

### 4. Lexical and Syntax Analysis

4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal

### Fall Compiler Principles Lecture 3: Parsing part 2. Roman Manevich Ben-Gurion University

Fall 2014-2015 Compiler Principles Lecture 3: Parsing part 2 Roman Manevich Ben-Gurion University Tentative syllabus Front End Intermediate Representation Optimizations Code Generation Scanning Lowering

### Section A. A grammar that produces more than one parse tree for some sentences is said to be ambiguous.

Section A 1. What do you meant by parser and its types? A parser for grammar G is a program that takes as input a string w and produces as output either a parse tree for w, if w is a sentence of G, or

### Bottom-Up Parsing. Lecture 11-12

Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 9/22/06 Prof. Hilfinger CS164 Lecture 11 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient

### 4. Lexical and Syntax Analysis

4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal

### Outline. The strategy: shift-reduce parsing. Introduction to Bottom-Up Parsing. A key concept: handles

Outline Introduction to Bottom-Up Parsing Lecture Notes by Profs. Alex Aiken and George Necula (UCB) he strategy: -reduce parsing A key concept: handles Ambiguity and precedence declarations CS780(Prasad)

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2016-2017 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman

### Where We Are. CMSC 330: Organization of Programming Languages. This Lecture. Programming Languages. Motivation for Grammars

CMSC 330: Organization of Programming Languages Context Free Grammars Where We Are Programming languages Ruby OCaml Implementing programming languages Scanner Uses regular expressions Finite automata Parser

### Parsing. Lecture 11: Parsing. Recursive Descent Parser. Arithmetic grammar. - drops irrelevant details from parse tree

Parsing Lecture 11: Parsing CSC 131 Fall, 2014 Kim Bruce Build parse tree from an expression Interested in abstract syntax tree - drops irrelevant details from parse tree Arithmetic grammar ::=

### Syntax Analysis: Context-free Grammars, Pushdown Automata and Parsing Part - 4. Y.N. Srikant

Syntax Analysis: Context-free Grammars, Pushdown Automata and Part - 4 Department of Computer Science and Automation Indian Institute of Science Bangalore 560 012 NPTEL Course on Principles of Compiler