# Computer Science 160 Translation of Programming Languages

Size: px
Start display at page:

Transcription

1 Computer Science 160 Translation of Programming Languages Instructor: Christopher Kruegel

2 Top-Down Parsing

3 Parsing Techniques Top-down parsers (LL(1), recursive descent parsers) Start at the root of the parse tree from the start symbol and grow toward leaves (similar to a derivation) Pick a production and try to match the input Bad pick may need to backtrack Some grammars are backtrack-free (predictive parsing) Bottom-up parsers (LR(1), shift-reduce parsers) Start at the leaves and grow toward root We can think of the process as reducing the input string to the start symbol At each reduction step, a particular substring matching the right-side of a production is replaced by the symbol on the left-side of the production Bottom-up parsers handle a large class of grammars

4 Top-down Parsing Algorithm Construct the root node of the parse tree, label it with the start symbol, and set the current-node to root node Repeat until all the input is consumed (i.e., until the frontier of the parse tree matches the input string) 1 If the label of the current node is a non-terminal node A, select a production with A on its lhs and, for each symbol on its rhs, construct the appropriate child 2 If the current node is a terminal symbol: If it matches the input string, consume it (advance the input pointer) If it does not match the input string, backtrack 3 Set the current node to the next node in the frontier of the parse tree If there is no node left in the frontier of the parse tree and input is not consumed, then backtrack The key is picking the right production in step 1 That choice should be guided by the input string

5 Example Using version with correct precedence and associativity 1 S Expr 2 Expr Expr + Term 3 Expr - Term 4 Term 5 Term Term * Factor 6 Term / Factor 7 Factor 8 Factor num 9 id And the input: x 2 * y

6 Example Let s try x 2 * y : S Rule Sentential Form Input - S x 2 * y 1 Expr x 2 * y 2 Expr + Term x 2 * y 4 Term + Term x 2 * y 7 Factor + Term x 2 * y 9 <id,x> + Term x 2 * y <id,x> + Term x 2 * y Expr Term Fact. Expr + Term <id,x>

7 Example Let s try x 2 * y : S Rule Sentential Form Input - S x 2 * y 1 Expr x 2 * y 2 Expr + Term x 2 * y 4 Term + Term x 2 * y 7 Factor + Term x 2 * y 9 <id,x> + Term x 2 * y <id,x> + Term x 2 * y Expr Term Fact. Expr + Term <id,x> The parser must backtrack to here Note that doesn t match +

8 Example Continuing with x 2 * y : S Rule Sentential Form Input - S x 2 * y 1 Expr x 2 * y 3 Expr Term x 2 * y 4 Term Term x 2 * y 7 Factor Term x 2 * y 9 <id,x> Term x 2 * y - <id,x> Term x 2 * y - <id,x> Term x 2 * y Expr Term Fact. Expr Term <id,x>

9 Example Continuing with x 2 * y : S Rule Sentential Form Input - S x 2 * y 1 Expr x 2 * y 3 Expr Term x 2 * y 4 Term Term x 2 * y 7 Factor Term x 2 * y 9 <id,x> Term x 2 * y - <id,x> Term x 2 * y - <id,x> Term x 2 * y Expr Term Fact. Expr Term <id,x> This time and matched We can advance past to look at 2 Now we need to extend Term, the last NT in the fringe of the parse tree

10 Example Trying to match the 2 in x 2 * y : S Rule Sentential Form Input <id,x> Term x 2 * y 7 <id,x> Factor x 2 * y 9 <id,x> <num,2> x 2 * y <id,x> <num,2> x 2 * y Expr Term Expr Term Fact. Where are we? num matches 2 Fact. <id,x> We have more input, but no NTs left to expand The expansion terminated too soon Need to backtrack <num,2>

11 Example Trying again with 2 in x 2 * y : S Rule Sentential Form Input - <id,x> Term x 2 * y 5 <id,x> Term * Factor x 2 * y 7 <id,x> Factor * Factor x 2 * y 8 <id,x> <num,2> * Factor x 2 * y - <id,x> <num,2> * Factor x 2 * y - <id,x> <num,2> * Factor x 2 * y 9 <id,x> <num,2> * <id,y> x 2 * y - <id,x> <num,2> * <id,y> x 2 * y Expr Term Fact. Expr Term Fact. Term * Fact. <id,y> <id,x> <num,2> This time, we matched and consumed all the input Success!

12 Another possible parse Other choices for expansion are possible Rule Sentential Form Input S x - 2 * y 1 Expr x - 2 * y 2 Expr + Term x - 2 * y 2 Expr + Term +Term x - 2 * y 2 Expr + Term + Term +Term x - 2 * y 2 Expr +Term + Term + +Term x - 2 * y consuming no input! This does not terminate Wrong choice of expansion leads to non-termination, the parser will not backtrack since it does not get to a point where it can backtrack Non-termination is a bad property for a parser to have Parser must make the right choice

13 Left Recursion Top-down parsers cannot handle left-recursive grammars Formally, A grammar is left recursive if there exists a non-terminal A such that there exists a derivation A + Aα, for some string α (NT T ) + Our expression grammar is left-recursive This can lead to non-termination in a top-down parser For a top-down parser, any recursion must be right recursion We would like to convert the left recursion to right recursion (without changing the language that is defined by the grammar)

14 Eliminating Immediate Left Recursion To remove left recursion, we can transform the grammar Consider a grammar fragment of the form A A α β where α or β are strings of terminal and non-terminal symbols and neither α nor β start with A A We can rewrite this as A β R R α R ε where R is a new non-terminal This accepts the same language, but uses only right recursion β A β α A R α A α R α R ε

15 Eliminating Immediate Left Recursion The expression grammar contains two cases of left recursion Expr Expr + Term Expr Term Term Term Term * Factor Term / Factor Factor Applying the transformation yields Expr Term Exprʹ Exprʹ + Term Exprʹ - Term Exprʹ ε Term Factor Termʹ Termʹ * Factor Termʹ / Factor Termʹ ε These fragments use only right recursion

16 Eliminating Immediate Left Recursion Substituting back into the grammar yields 1 S Expr 2 Expr Term Exprʹ 3 Exprʹ + Term Exprʹ 4 - Term Exprʹ 5 ε 6 Term Factor Termʹ 7 Termʹ * Factor Termʹ 8 / Factor Termʹ 9 ε 10 Factor num 11 id This grammar is correct, if somewhat non-intuitive. A top-down parser will terminate using it.

17 Left-Recursive and Right-Recursive Grammar 1 S Expr 2 Expr Expr + Term 3 Expr Term 4 Term 5 Term Term * Factor 6 Term / Factor 7 Factor 8 Factor num 9 id 1 S Expr 2 Expr Term Exprʹ 3 Exprʹ + Term Exprʹ 4 Term Exprʹ 5 ε 6 Term Factor Termʹ 7 Termʹ * Factor Termʹ 8 / Factor Termʹ 9 ε 10 Factor num 11 id

18 Preserves Precedence S S E E E T T E T T * F F T T E F F <id,x> <num,2> <id,y> <id,x> ε F <num,2> T ε * F T <id,y> ε

19 Eliminating Left Recursion The previous transformation eliminates immediate left recursion What about more general, indirect left recursion? The general algorithm (Algorithm 4.1 in the Textbook): Arrange the NTs into some order A 1, A 2,, A n for i 1 to n for j 1 to i-1 replace each production A i A j γ with A i δ 1 γ δ 2 γ δ k γ, where A j δ 1 δ 2 δ k are all the current productions for A j eliminate any immediate left recursion on A i using the direct transformation This assumes that the initial grammar has no cycles (A i + A i ), and no epsilon productions (A i ε )

20 Eliminating Left Recursion How does this algorithm work? 1. Impose arbitrary order on the non-terminals 2. Outer loop cycles through NT in order 3. Inner loop ensures that a production expanding A i has no non-terminal A j in its rhs, for j < i 4. Last step in outer loop converts any direct recursion on A i to right recursion using the transformation showed earlier 5. New non-terminals are added at the end of the order and have no left recursion At the start of the i th outer loop iteration For all k < i, no production that expands A k contains a non-terminal A s in its rhs, for s < k

21 Picking the Right Production If it picks the wrong production, a top-down parser may backtrack Alternative is to look ahead in input & use context to pick correctly How much look-ahead is needed? In general, an arbitrarily large amount Use the Cocke-Younger, Kasami, or Earley s algorithm Complexity is O( x 3 ) where x is the input string Fortunately, Large subclasses of context free grammars can be parsed efficiently with limited look-ahead Linear complexity, O( x ) where x is the input string Most programming language constructs fall in those subclasses Among the interesting subclasses are LL(1) and LR(1) grammars

22 Left-Recursive and Right-Recursive Grammar 1 S Expr 2 Expr Expr + Term 3 Expr Term 4 Term 5 Term Term * Factor 6 Term / Factor 7 Factor 8 Factor num 9 id 1 S Expr 2 Expr Term Exprʹ 3 Exprʹ + Term Exprʹ 4 Term Exprʹ 5 ε 6 Term Factor Termʹ 7 Termʹ * Factor Termʹ 8 / Factor Termʹ 9 ε 10 Factor num 11 id Why is this better? It is no longer left recursive so we eventually need to eat a token before we can continue expanding our parse tree. We can use this token to help us figure out which rule to apply. Enter Predictive Parsing

23 Predictive Parsing Basic idea Given A α β, the parser should be able to choose between α & β based on peeking at the next token in the stream FIRST sets For a string of grammar symbols α, define FIRST(α) as the set of tokens that appear as the first symbol in some string that derives from α That is, x FIRST(α) iff α * x γ, for some γ ( * means a bunch of (0 or more) productions applied in series) The LL(1) Property If A α and A β both appear in the grammar, we would like FIRST(α) FIRST(β) = This would allow the parser to make a correct choice with a look-ahead of exactly one symbol! (Pursuing this idea leads to LL(1) parser generators...)

24 Recursive Descent Parsing Recursive-descent parsing A top-down parsing method The term descent refers to the direction in which the parse tree is traversed (or built). Use a set of mutually recursive procedures (one procedure for each non-terminal symbol) Start the parsing process by calling the procedure that corresponds to the start symbol Each production becomes one clause in procedure We consider a special type of recursive-descent parsing called predictive parsing Use a look-ahead symbol to decide which production to use

25 Recursive Descent Parsing 1 S if E then S else S 2 begin S L 3 print E 4 L end 5 ; S L 6 E num = num void S() { switch(lookahead) { case IF: match(if); E(); match(then); S(); match(else); S(); break; case BEGIN: match(begin); S(); L(); break; case PRINT: match(print); E(); break; default: error(); } } void E() { match(num); match(eq); match(num); } void match(int token) { if (lookahead==token) lookahead=getnexttoken(); else error(); } void L() { switch(lookahead) { case END: match(end); break; case SEMI: match(semi); S(); L(); break; default: error(); } } void main() { lookahead=getnexttoken(); S(); match(eof); }

26 Execution For Input: if 2=2 then print 5=5 else print 1=1 main: call S(); S 1 : find the production for (S, IF) : S if E then S else S S 1 : match(if); S 1 : call E(); E 1 : find the production for (E, NUM): E num = num E 1 : match(num); match(eq); match(num); E 1 : return from E 1 to S 1 S 1 : match(then); S 1 :call S(); S 2 : find the production for (S, PRINT): S print E S 2 : match(print); S 2 : call E(); E 2 : find the production for (E, NUM): E num = num E 2 : match(num); match(eq); match(num); E 2 : return from E 2 to S 2 S 2 : return from S 2 to S 1 S 1 : match(else); S 1 : call S(); S 3 : find the production for (S, PRINT): S print E S 3 : match(print); S 3 : call E(); E 3 : find the production for (E, NUM): E num = num E 3 : match(num); match(eq); match(num); E 3 : return from E 2 to S 3 S 3 : return from S 3 to S 1 S 1 : return from S 1 to main main: match(eof); return success;

27 Left Factoring What if the grammar does not have the LL(1) property? We already learned one transformation: Removing left-recursion There is another transformation called left-factoring Left-Factoring Algorithm: A NT, find the longest prefix α that occurs in two or more right-hand sides of A if α ε then replace all of the A productions, A αβ 1 αβ 2 αβ n γ 1 γ 2 γ k, with A α Z γ 1 γ 2 γ k Z β 1 β 2 β n where Z is a new element of NT Repeat until no common prefixes remain

28 Left Factoring A graphical explanation for the left-factoring A αβ 1 αβ 2 αβ n A αβ 1 αβ 2 becomes αβ 3 A α Z Z β 1 β 2 β n A αz β 1 β 2 β 3

29 Left Factoring - Example Consider the following fragment of the expression grammar 1 Factor Id 2 Id [ ExprList ] 3 Id ( ExprList ) FIRST(rhs 1 ) = { Id } FIRST(rhs 2 ) = { Id } FIRST(rhs 3 ) = { Id } After left factoring, it becomes 1 Factor Id Arguments 2 Arguments [ ExprList ] 3 ( ExprList ) 4 ε This grammar accepts the same language, and it has the the LL(1) property FIRST(rhs 1 ) = { Id } FIRST(rhs 2 ) = { [ } FIRST(rhs 3 ) = { ( } FIRST(rhs 4 ) =? (Intuitively, we can think of the FOLLOW of Arguments as the first of rhs 4 ) FOLLOW(Arguments)=FOLLOW(Factor) = { \$ } They are all distinct Grammar has the LL(1) property

### Syntactic Analysis. Top-Down Parsing

Syntactic Analysis Top-Down Parsing Copyright 2017, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make

### CS 406/534 Compiler Construction Parsing Part I

CS 406/534 Compiler Construction Parsing Part I Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof. Ken Kennedy and Dr.

### Parsing Part II (Top-down parsing, left-recursion removal)

Parsing Part II (Top-down parsing, left-recursion removal) Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit

### Types of parsing. CMSC 430 Lecture 4, Page 1

Types of parsing Top-down parsers start at the root of derivation tree and fill in picks a production and tries to match the input may require backtracking some grammars are backtrack-free (predictive)

### Parsing III. (Top-down parsing: recursive descent & LL(1) )

Parsing III (Top-down parsing: recursive descent & LL(1) ) Roadmap (Where are we?) Previously We set out to study parsing Specifying syntax Context-free grammars Ambiguity Top-down parsers Algorithm &

### Syntax Analysis, III Comp 412

COMP 412 FALL 2017 Syntax Analysis, III Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp

### Parsing Part II. (Ambiguity, Top-down parsing, Left-recursion Removal)

Parsing Part II (Ambiguity, Top-down parsing, Left-recursion Removal) Ambiguous Grammars Definitions If a grammar has more than one leftmost derivation for a single sentential form, the grammar is ambiguous

### CSCI312 Principles of Programming Languages

Copyright 2006 The McGraw-Hill Companies, Inc. CSCI312 Principles of Programming Languages! LL Parsing!! Xu Liu Derived from Keith Cooper s COMP 412 at Rice University Recap Copyright 2006 The McGraw-Hill

### Syntax Analysis, III Comp 412

Updated algorithm for removal of indirect left recursion to match EaC3e (3/2018) COMP 412 FALL 2018 Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, III Comp 412 source code

### Parsing. Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice.

Parsing Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice. Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students

### Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

### Syntactic Analysis. Top-Down Parsing. Parsing Techniques. Top-Down Parsing. Remember the Expression Grammar? Example. Example

Syntactic Analysis op-down Parsing Parsing echniques op-down Parsers (LL(1), recursive descent) Start at the root of the parse tree and grow toward leaves Pick a production & try to match the input Bad

### Introduction to Parsing. Comp 412

COMP 412 FALL 2010 Introduction to Parsing Comp 412 Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit permission to make

### Parsing II Top-down parsing. Comp 412

COMP 412 FALL 2018 Parsing II Top-down parsing Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled

### Parsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones

Parsing III (Top-down parsing: recursive descent & LL(1) ) (Bottom-up parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,

### 3. Parsing. Oscar Nierstrasz

3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/

### CA Compiler Construction

CA4003 - Compiler Construction David Sinclair A top-down parser starts with the root of the parse tree, labelled with the goal symbol of the grammar, and repeats the following steps until the fringe of

### Compilers. Yannis Smaragdakis, U. Athens (original slides by Sam

Compilers Parsing Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Next step text chars Lexical analyzer tokens Parser IR Errors Parsing: Organize tokens into sentences Do tokens conform

### Syntax Analysis, V Bottom-up Parsing & The Magic of Handles Comp 412

Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, V Bottom-up Parsing & The Magic of Handles Comp 412 COMP 412 FALL 2018 source code IR Front End Optimizer Back End IR target

### CS415 Compilers. Syntax Analysis. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University

CS415 Compilers Syntax Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Limits of Regular Languages Advantages of Regular Expressions

### Prelude COMP 181 Tufts University Computer Science Last time Grammar issues Key structure meaning Tufts University Computer Science

Prelude COMP Lecture Topdown Parsing September, 00 What is the Tufts mascot? Jumbo the elephant Why? P. T. Barnum was an original trustee of Tufts : donated \$0,000 for a natural museum on campus Barnum

### Parsing II Top-down parsing. Comp 412

COMP 412 FALL 2017 Parsing II Top-down parsing Comp 412 source code IR Front End OpMmizer Back End IR target code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled

### CS 314 Principles of Programming Languages

CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution

### Top-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7

Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess

### Chapter 4: LR Parsing

Chapter 4: LR Parsing 110 Some definitions Recall For a grammar G, with start symbol S, any string α such that S called a sentential form α is If α Vt, then α is called a sentence in L G Otherwise it is

### Top-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7

Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess

### Context-free grammars

Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,

### CS1622. Today. A Recursive Descent Parser. Preliminaries. Lecture 9 Parsing (4)

CS1622 Lecture 9 Parsing (4) CS 1622 Lecture 9 1 Today Example of a recursive descent parser Predictive & LL(1) parsers Building parse tables CS 1622 Lecture 9 2 A Recursive Descent Parser. Preliminaries

### Academic Formalities. CS Modern Compilers: Theory and Practise. Images of the day. What, When and Why of Compilers

Academic Formalities CS6013 - Modern Compilers: Theory and Practise Introduction V. Krishna Nandivada IIT Madras Written assignment = 5 marks. Programming assignments = 40 marks. Midterm = 25 marks, Final

### 4. Lexical and Syntax Analysis

4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal

### COMP 181. Prelude. Next step. Parsing. Study of parsing. Specifying syntax with a grammar

COMP Lecture Parsing September, 00 Prelude What is the common name of the fruit Synsepalum dulcificum? Miracle fruit from West Africa What is special about miracle fruit? Contains a protein called miraculin

### 4. Lexical and Syntax Analysis

4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal

### Compiler Construction: Parsing

Compiler Construction: Parsing Mandar Mitra Indian Statistical Institute M. Mitra (ISI) Parsing 1 / 33 Context-free grammars. Reference: Section 4.2 Formal way of specifying rules about the structure/syntax

### A programming language requires two major definitions A simple one pass compiler

A programming language requires two major definitions A simple one pass compiler [Syntax: what the language looks like A context-free grammar written in BNF (Backus-Naur Form) usually suffices. [Semantics:

### Compilers. Bottom-up Parsing. (original slides by Sam

Compilers Bottom-up Parsing Yannis Smaragdakis U Athens Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Bottom-Up Parsing More general than top-down parsing And just as efficient Builds

### The Parsing Problem (cont d) Recursive-Descent Parsing. Recursive-Descent Parsing (cont d) ICOM 4036 Programming Languages. The Complexity of Parsing

ICOM 4036 Programming Languages Lexical and Syntax Analysis Lexical Analysis The Parsing Problem Recursive-Descent Parsing Bottom-Up Parsing This lecture covers review questions 14-27 This lecture covers

### Top down vs. bottom up parsing

Parsing A grammar describes the strings that are syntactically legal A recogniser simply accepts or rejects strings A generator produces sentences in the language described by the grammar A parser constructs

### Bottom-up Parser. Jungsik Choi

Formal Languages and Compiler (CSE322) Bottom-up Parser Jungsik Choi chjs@khu.ac.kr * Some slides taken from SKKU SWE3010 (Prof. Hwansoo Han) and TAMU CSCE434-500 (Prof. Lawrence Rauchwerger) Bottom-up

### Sometimes an ambiguous grammar can be rewritten to eliminate the ambiguity.

Eliminating Ambiguity Sometimes an ambiguous grammar can be rewritten to eliminate the ambiguity. Example: consider the following grammar stat if expr then stat if expr then stat else stat other One can

### Compilerconstructie. najaar Rudy van Vliet kamer 140 Snellius, tel rvvliet(at)liacs(dot)nl. college 3, vrijdag 22 september 2017

Compilerconstructie najaar 2017 http://www.liacs.leidenuniv.nl/~vlietrvan1/coco/ Rudy van Vliet kamer 140 Snellius, tel. 071-527 2876 rvvliet(at)liacs(dot)nl college 3, vrijdag 22 september 2017 + werkcollege

### Syntax Analysis. Martin Sulzmann. Martin Sulzmann Syntax Analysis 1 / 38

Syntax Analysis Martin Sulzmann Martin Sulzmann Syntax Analysis 1 / 38 Syntax Analysis Objective Recognize individual tokens as sentences of a language (beyond regular languages). Example 1 (OK) Program

### Chapter 4: Syntax Analyzer

Chapter 4: Syntax Analyzer Chapter 4: Syntax Analysis 1 The role of the Parser The parser obtains a string of tokens from the lexical analyzer, and verifies that the string can be generated by the grammar

### Concepts Introduced in Chapter 4

Concepts Introduced in Chapter 4 Grammars Context-Free Grammars Derivations and Parse Trees Ambiguity, Precedence, and Associativity Top Down Parsing Recursive Descent, LL Bottom Up Parsing SLR, LR, LALR

SYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic

### Compiler Construction 2016/2017 Syntax Analysis

Compiler Construction 2016/2017 Syntax Analysis Peter Thiemann November 2, 2016 Outline 1 Syntax Analysis Recursive top-down parsing Nonrecursive top-down parsing Bottom-up parsing Syntax Analysis tokens

### Section A. A grammar that produces more than one parse tree for some sentences is said to be ambiguous.

Section A 1. What do you meant by parser and its types? A parser for grammar G is a program that takes as input a string w and produces as output either a parse tree for w, if w is a sentence of G, or

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Parsing CMSC 330 - Spring 2017 1 Recall: Front End Scanner and Parser Front End Token Source Scanner Parser Stream AST Scanner / lexer / tokenizer converts

### Introduction to Parsing

Introduction to Parsing The Front End Source code Scanner tokens Parser IR Errors Parser Checks the stream of words and their parts of speech (produced by the scanner) for grammatical correctness Determines

### CSE 130 Programming Language Principles & Paradigms Lecture # 5. Chapter 4 Lexical and Syntax Analysis

Chapter 4 Lexical and Syntax Analysis Introduction - Language implementation systems must analyze source code, regardless of the specific implementation approach - Nearly all syntax analysis is based on

### Parser Generation. Bottom-Up Parsing. Constructing LR Parser. LR Parsing. Construct parse tree bottom-up --- from leaves to the root

Parser Generation Main Problem: given a grammar G, how to build a top-down parser or a bottom-up parser for it? parser : a program that, given a sentence, reconstructs a derivation for that sentence ----

### CS502: Compilers & Programming Systems

CS502: Compilers & Programming Systems Top-down Parsing Zhiyuan Li Department of Computer Science Purdue University, USA There exist two well-known schemes to construct deterministic top-down parsers:

### Syntax Analysis. COMP 524: Programming Language Concepts Björn B. Brandenburg. The University of North Carolina at Chapel Hill

Syntax Analysis Björn B. Brandenburg The University of North Carolina at Chapel Hill Based on slides and notes by S. Olivier, A. Block, N. Fisher, F. Hernandez-Campos, and D. Stotts. The Big Picture Character

### Administrativia. PA2 assigned today. WA1 assigned today. Building a Parser II. CS164 3:30-5:00 TT 10 Evans. First midterm. Grammars.

Administrativia Building a Parser II CS164 3:30-5:00 TT 10 Evans PA2 assigned today due in 12 days WA1 assigned today due in a week it s a practice for the exam First midterm Oct 5 will contain some project-inspired

### MIT Top-Down Parsing. Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology

MIT 6.035 Top-Down Parsing Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Orientation Language specification Lexical structure regular expressions Syntactic structure

### Lexical and Syntax Analysis. Top-Down Parsing

Lexical and Syntax Analysis Top-Down Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax

### Syntax Analysis. The Big Picture. The Big Picture. COMP 524: Programming Languages Srinivas Krishnan January 25, 2011

Syntax Analysis COMP 524: Programming Languages Srinivas Krishnan January 25, 2011 Based in part on slides and notes by Bjoern Brandenburg, S. Olivier and A. Block. 1 The Big Picture Character Stream Token

### CSE302: Compiler Design

CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University February 20, 2007 Outline Recap

### CS 406/534 Compiler Construction Parsing Part II LL(1) and LR(1) Parsing

CS 406/534 Compiler Construction Parsing Part II LL(1) and LR(1) Parsing Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof.

### SLR parsers. LR(0) items

SLR parsers LR(0) items As we have seen, in order to make shift-reduce parsing practical, we need a reasonable way to identify viable prefixes (and so, possible handles). Up to now, it has not been clear

### PART 3 - SYNTAX ANALYSIS. F. Wotawa TU Graz) Compiler Construction Summer term / 309

PART 3 - SYNTAX ANALYSIS F. Wotawa (IST @ TU Graz) Compiler Construction Summer term 2016 64 / 309 Goals Definition of the syntax of a programming language using context free grammars Methods for parsing

### 3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino

3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the

### Ambiguity, Precedence, Associativity & Top-Down Parsing. Lecture 9-10

Ambiguity, Precedence, Associativity & Top-Down Parsing Lecture 9-10 (From slides by G. Necula & R. Bodik) 9/18/06 Prof. Hilfinger CS164 Lecture 9 1 Administrivia Please let me know if there are continued

### CS2210: Compiler Construction Syntax Analysis Syntax Analysis

Comparison with Lexical Analysis The second phase of compilation Phase Input Output Lexer string of characters string of tokens Parser string of tokens Parse tree/ast What Parse Tree? CS2210: Compiler

### Syntax Analysis Part I

Syntax Analysis Part I Chapter 4: Context-Free Grammars Slides adapted from : Robert van Engelen, Florida State University Position of a Parser in the Compiler Model Source Program Lexical Analyzer Token,

### Administrativia. WA1 due on Thu PA2 in a week. Building a Parser III. Slides on the web site. CS164 3:30-5:00 TT 10 Evans.

Administrativia Building a Parser III CS164 3:30-5:00 10 vans WA1 due on hu PA2 in a week Slides on the web site I do my best to have slides ready and posted by the end of the preceding logical day yesterday,

### CS 230 Programming Languages

CS 230 Programming Languages 10 / 16 / 2013 Instructor: Michael Eckmann Today s Topics Questions/comments? Top Down / Recursive Descent Parsers Top Down Parsers We have a left sentential form xa Expand

### Topic 3: Syntax Analysis I

Topic 3: Syntax Analysis I Compiler Design Prof. Hanjun Kim CoreLab (Compiler Research Lab) POSTECH 1 Back-End Front-End The Front End Source Program Lexical Analysis Syntax Analysis Semantic Analysis

### A left-sentential form is a sentential form that occurs in the leftmost derivation of some sentence.

Bottom-up parsing Recall For a grammar G, with start symbol S, any string α such that S α is a sentential form If α V t, then α is a sentence in L(G) A left-sentential form is a sentential form that occurs

### Syntax Analysis: Context-free Grammars, Pushdown Automata and Parsing Part - 4. Y.N. Srikant

Syntax Analysis: Context-free Grammars, Pushdown Automata and Part - 4 Department of Computer Science and Automation Indian Institute of Science Bangalore 560 012 NPTEL Course on Principles of Compiler

### Lexical and Syntax Analysis. Bottom-Up Parsing

Lexical and Syntax Analysis Bottom-Up Parsing Parsing There are two ways to construct derivation of a grammar. Top-Down: begin with start symbol; repeatedly replace an instance of a production s LHS with

### 컴파일러입문 제 6 장 구문분석

컴파일러입문 제 6 장 구문분석 목차 6.1 구문분석방법 6.2 구문분석기의출력 6.3 Top-down 방법 6.4 Bottom-up 방법 Syntax Analysis Page 2 구문분석방법 Text p.220 How to check whether an input string is a sentence of a grammar and how to construct

### The role of the parser

The role of the parser source code tokens scanner parser IR errors Parser performs context-free syntax analysis guides context-sensitive analysis constructs an intermediate representation produces meaningful

### Chapter 3. Parsing #1

Chapter 3 Parsing #1 Parser source file get next character scanner get token parser AST token A parser recognizes sequences of tokens according to some grammar and generates Abstract Syntax Trees (ASTs)

### 8 Parsing. Parsing. Top Down Parsing Methods. Parsing complexity. Top down vs. bottom up parsing. Top down vs. bottom up parsing

8 Parsing Parsing A grammar describes syntactically legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string

### Parsing #1. Leonidas Fegaras. CSE 5317/4305 L3: Parsing #1 1

Parsing #1 Leonidas Fegaras CSE 5317/4305 L3: Parsing #1 1 Parser source file get next character scanner get token parser AST token A parser recognizes sequences of tokens according to some grammar and

### Compiler Design Concepts. Syntax Analysis

Compiler Design Concepts Syntax Analysis Introduction First task is to break up the text into meaningful words called tokens. newval=oldval+12 id = id + num Token Stream Lexical Analysis Source Code (High

### Bottom-up parsing. Bottom-Up Parsing. Recall. Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form

Bottom-up parsing Bottom-up parsing Recall Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form If α V t,thenα is called a sentence in L(G) Otherwise it is just

### Lexical and Syntax Analysis

Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Easy for humans to write and understand String of characters

### Fall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev

Fall 2017-2018 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman

### Wednesday, August 31, Parsers

Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Context Free Grammars and Parsing 1 Recall: Architecture of Compilers, Interpreters Source Parser Static Analyzer Intermediate Representation Front End Back

### A parser is some system capable of constructing the derivation of any sentence in some language L(G) based on a grammar G.

Top Down Parsing 1 Parsers Formal Definitions of a Parser A parser is some system capable of constructing the derivation of any sentence in some language L(G) based on a grammar G. which talks about a

### Context-Free Languages and Parse Trees

Context-Free Languages and Parse Trees Mridul Aanjaneya Stanford University July 12, 2012 Mridul Aanjaneya Automata Theory 1/ 41 Context-Free Grammars A context-free grammar is a notation for describing

### LECTURE 7. Lex and Intro to Parsing

LECTURE 7 Lex and Intro to Parsing LEX Last lecture, we learned a little bit about how we can take our regular expressions (which specify our valid tokens) and create real programs that can recognize them.

### Wednesday, September 9, 15. Parsers

Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Parsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:

What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Outline CS412/413. Administrivia. Review. Grammars. Left vs. Right Recursion. More tips forll(1) grammars Bottom-up parsing LR(0) parser construction

C12/1 Introduction to Compilers and Translators pring 00 Outline More tips forll1) grammars Bottom-up parsing LR0) parser construction Lecture 5: Bottom-up parsing Lecture 5 C 12/1 pring '00 Andrew Myers

### Chapter 4. Lexical and Syntax Analysis. Topics. Compilation. Language Implementation. Issues in Lexical and Syntax Analysis.

Topics Chapter 4 Lexical and Syntax Analysis Introduction Lexical Analysis Syntax Analysis Recursive -Descent Parsing Bottom-Up parsing 2 Language Implementation Compilation There are three possible approaches

### Monday, September 13, Parsers

Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions

### Parsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468

Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure

### Parsing Wrapup. Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing. This lecture LR(1) parsing

Parsing Wrapup Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing LR(1) items Computing closure Computing goto LR(1) canonical collection This lecture LR(1) parsing Building ACTION

### Table-Driven Top-Down Parsers

Table-Driven Top-Down Parsers Recursive descent parsers have many attractive features. They are actual pieces of code that can be read by programmers and extended. This makes it fairly easy to understand

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview n Tokens and regular expressions n Syntax and context-free grammars n Grammar derivations n More about parse trees n Top-down and

### CS 321 Programming Languages and Compilers. VI. Parsing

CS 321 Programming Languages and Compilers VI. Parsing Parsing Calculate grammatical structure of program, like diagramming sentences, where: Tokens = words Programs = sentences For further information,

### Formal Languages and Compilers Lecture VII Part 3: Syntactic A

Formal Languages and Compilers Lecture VII Part 3: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/

### Syntax Analysis. Prof. James L. Frankel Harvard University. Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved.

Syntax Analysis Prof. James L. Frankel Harvard University Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved. Context-Free Grammar (CFG) terminals non-terminals start