Syntax Analysis. Martin Sulzmann. Martin Sulzmann Syntax Analysis 1 / 38
|
|
- Jody Spencer
- 5 years ago
- Views:
Transcription
1 Syntax Analysis Martin Sulzmann Martin Sulzmann Syntax Analysis 1 / 38
2 Syntax Analysis Objective Recognize individual tokens as sentences of a language (beyond regular languages). Example 1 (OK) Program text x := x + 2 yields token stream (ident x ) assign (ident x ) plus (const 2) Example 2 (Not OK) Program text x x := + 2 yields token stream (ident x ) (ident x ) assign plus (const 2) We will discuss later if the OK example shall be semantically valid. Martin Sulzmann Syntax Analysis 2 / 38
3 Approach: Parsing with CFG Context-Free Grammar (CFG) Language described in terms of a context-free grammar: Command ident assign Expression... Expression const ident Expression plus Expression ident, const, plus being tokens where attributes are omitted. Note: CFG uses uppercase for non-terminal and lowercase for terminal symbols. Our parsing tool OCamlyacc just uses it the other way around! Parsing yields Abstract Syntax Tree (AST) Check if stream of tokens is valid and produce AST. Martin Sulzmann Syntax Analysis 3 / 38
4 Parsing with CFG Example Token Stream (ident x ) assign (ident x ) plus (const 2) AST assign ident plus x ident plus x 2 Martin Sulzmann Syntax Analysis 4 / 38
5 Types of Parsing Top-Down Recursive descent From root to leaves ANTRL style Bottom-Up From leaves to root yacc style Others Parser combinators Derivative-based parser... Martin Sulzmann Syntax Analysis 5 / 38
6 Error Handling Types of Errors lexical misspelled keyword whle cond... syntactic unbalanced parentheses x:= x+1) semantic incompatible types x:= x + True logical infinite loop while True... Provide useful feedback to user. Mostly ignored here. Martin Sulzmann Syntax Analysis 6 / 38
7 Context Free Grammar (CFG) Definition A CFG G is a 4-tuple (V t,v n,s,p) where 1 V t is the set of terminal symbols (commonly the set of tokens). 2 V n is the set of non-terminal symbols. 3 S is a distinguished non-terminal symbol call the start symbol. 4 P is a finite set of productions (aka rewrite rules) of the form: non-terminal (non-terminal terminal). Note: All sets are finite. Left-hand side (lhs) of production consists of a single non-terminal symbol. Right-hand side (rhs) consists of arbitrary combinations of terminal and non-terminal symbol described by a regular expression. Martin Sulzmann Syntax Analysis 7 / 38
8 CFG Example and Notation Example E E +T T T T F F F (E) ident Notation Vocabulary V = V t V n a,b,c,... V t A,B,C,... V n α,β,γ,... V u,v,w,... V t Martin Sulzmann Syntax Analysis 8 / 38
9 CFG as a Rewrite System Derivations by Rewriting If A γ P then αaβ αγβ is a one-step derivation using A γ. We assume that denotes a derivation of 0 steps. In a leftmost derivation (denoted L ) we replace the leftmost non-terminal in each step. In a rightmost derivation (denoted R ) we replace the rightmost non-terminal in each step. If S β then β is a sentential form of G. If S w and w V t then w is a sentence of G. L(G) = {w S w,w V t } denotes the language described by G. Martin Sulzmann Syntax Analysis 9 / 38
10 Example Derivation CFG E E +T T T T F F F (E) ident Derivation E ident +(ident) E E +T T +T F +T ident +T ident +F ident +(E) ident +(T) ident +(F) ident +(ident) Martin Sulzmann Syntax Analysis 10 / 38
11 Example Derivation (2) CFG E E +E E E (E) Derivation E E E +E 3+E 3+E E 3+7 E Things to think about Multiple derivations for the same input word? How to represent derivations? Martin Sulzmann Syntax Analysis 11 / 38
12 Parse Trees Purpose Compact representation of a derivation E w. Most often in terms of a tree-like structure (but other formats like bit-encodings are also possible). Different from AST! Parse tree represents concrete input word. AST is an abstract representation of input. Martin Sulzmann Syntax Analysis 12 / 38
13 Parse Tree Example CFG + Derivation E E +E E E (E) E E +E 3+E 3+E E 3+7 E Parse Tree E E + E 3 E E 7 2 Martin Sulzmann Syntax Analysis 13 / 38
14 AST versus Parse Tree Parse tree E ( E ) E + E 1 2 AST A ident ident Parse tree: Maintain all source (concrete) syntax info. AST: Abstract representation. Martin Sulzmann Syntax Analysis 14 / 38
15 Ambiguity Definition Multiple parse trees for the same grammar and input word. Example E E +E E E (E) E E E E +E E 3+E E 3+7 E Another parse tree: E E E E + E Martin Sulzmann Syntax Analysis 15 / 38
16 Ambiguity Example Ambiguous Grammar E E +E E E (E) ident Equivalent Non-Ambiguous Grammar E E +T T E E +E T T F F E E E F (E) ident E (E) ident Fact Some context-free languages are inherently ambiguous. Martin Sulzmann Syntax Analysis 16 / 38
17 Ambiguity Example (2) Exercise Consider Stmt if Expr then Stmt if Expr then Stmt else Stmt other Ambiguous? Does an equivalent unambiguous grammar exist? Martin Sulzmann Syntax Analysis 17 / 38
18 Parsing Approaches: Top-Down versus Bottom-Up Example E E +E E E (E) Top-Down ( ANTLR ) Build left-most derivation (strictly reduce left-most non-terminal symbols). E E +E 3+E 3+E E 3+7 E Guess which rule to apply from left to right. Bottom-Down ( yacc ) Build right-most derivation. E E +E E +E E E +E 2 E Seek for matches of right-hand side, replace by left-hand side. Martin Sulzmann Syntax Analysis 18 / 38
19 Top-Down Parsing Example E E +T T T T F F F (E) ident Approach Build left-most derivation starting from start symbol, use target string to choose productions. Challenge: Left Recursion E E +T E +T +T... Several possible right-hand sides. Some lead to dead ends. Backtracking? Too inefficient. Martin Sulzmann Syntax Analysis 19 / 38
20 Removal of Left Recursion Example Consider E E +T T T T F F F (E) ident Remove left recursion, e.g. E TE E +TE ǫ Martin Sulzmann Syntax Analysis 20 / 38
21 Removal Left Recursion (2) Direct Left Recursion Removal Left recursion can be eliminated as follows: Grammar rules A Aα 1... Aα n β 1... β m are replaced by the equivalent rules A β 1 A... β m A A α 1 A... α n A ǫ Indirect Left Recursion Removal Consider A BC B Aβ Left recursion involves several rules. Can be eliminated (reduced to direct case) via unfolding. Martin Sulzmann Syntax Analysis 21 / 38
22 Left Factoring Issue Remove common prefix of right hand sides. Left Factoring Left factoring of A αβ 1... αβ n γ 1... γ m (where α ǫ is the longest common prefix) yields Example A αa γ 1... γ m A β 1... β n S iets ietses a E b transformed to S ietss a S ǫ es E b Martin Sulzmann Syntax Analysis 22 / 38
23 Top-Down Parsing: Short Summary Predictive Top-Down Parsing Eliminate left recursion. Apply left factoring. Deterministic. No backtracking. Lookahead? How far to lookahead in the input string to (deterministically) apply a rule? Observe right-hand sides! Martin Sulzmann Syntax Analysis 23 / 38
24 First (matching symbols) Intuition Which terminals can start a string matching the non-terminal? Definition First(α) = {a V t {ǫ} α aβ} Inductive Definition {x} if α = xα, x V t First(α) = First(x) if α = xα, ǫ First(x), x V n First(x) First(α ) if α = xα, ǫ First(x), x V n Sufficient to consider First(X) where X is a non-terminal. Martin Sulzmann Syntax Analysis 24 / 38
25 First Algorithm Algorithm Build First(X) as follows: 1 If X is a terminal then First(X) = {X}. 2 If X ǫ then add ǫ to First(X). 3 If X Y 1...Y k : 1 Put First(Y 1 ) {ǫ} in First(X). 2 i.1 < i <= k, if ǫ First(Y 1 )... First(Y i 1 ) (i.e. Y 1...Y i 1 ǫ) then put First(Y i ) {ǫ} in First(X). 3 If ǫ First(Y 1 )... First(Y k ) then add ǫ to First(X). Repeat until no more additions can be made. Martin Sulzmann Syntax Analysis 25 / 38
26 First Example Example E TE E +TE E ǫ T FT T FT T ǫ F ident F (E) First(E) = {ident,(} First(E ) = {+,ǫ} First(T) = {ident,(} First(T ) = {,ǫ} First(F) = {ident,(} Martin Sulzmann Syntax Analysis 26 / 38
27 Follow Issue Which terminals can start a string matching after the nonterminal? Consider ǫ right hand sides. Definition Follow(A) = {w First(β) S waβ} Algorithm Build Follow(A) as follows: 1 Add $ to Follow(S) where S is the start symbol. 2 If A αbβ: 1 Put First(β) {ǫ} in Follow(B) 2 If β = ǫ (i.e. A αb) or ǫ First(β) (i.e. β ǫ) then put Follow(A) in Follow(B). Repeat until no more additions can be made ($ = EOF token). Martin Sulzmann Syntax Analysis 27 / 38
28 First and Follow Example Example S BAa A ǫ BcA B ǫ b First Follow S a,b,c $ A ǫ,b,c a B ǫ,b a,b,c Martin Sulzmann Syntax Analysis 28 / 38
29 LL(1) Grammar definition A grammar G is LL(1) iff for each set of productions A α 1... α n : 1 First(α 1 ),...,First(α n ) are all pairwise disjoint. 2 If α i ǫ then First(α j ) Follow(A) = forall 1 <= j <= n,i j. If G is ǫ-free (there are no ǫ productions), first condition is sufficient. Facts 1 No left-recursive grammar is LL(1). 2 No ambiguous grammar is LL(1). 3 Some languages have no LL(1) grammar. Martin Sulzmann Syntax Analysis 29 / 38
30 LL(1) Grammar (cont d) Example S as a is not LL(1) because First(aS) = First(a) = {a}. However, the following equivalent grammar is LL(1). S as S as ǫ Lemma A grammar is LL(1) iff for each two productions A α and A β we have that Look(A α) Look(A β) = where Look(A α) = { First(α)\{ǫ} if ǫ Follow(α) (First(α)\{ǫ}) Follow(A) otherwise Martin Sulzmann Syntax Analysis 30 / 38
31 Recursive Descent Parsing Example E TE E +TE E ǫ T FT T FT T ǫ F ident F (E) First(E) = {ident,(} First(E ) = {+,ǫ} First(T) = {ident,(} First(T ) = {,ǫ} First(F) = {ident,(} Recursive Descent Parser Grammar is LL(1). Introduce a function for each non-terminal symbol. Martin Sulzmann Syntax Analysis 31 / 38
32 Recursive Descent Parser Parser Ignore the parse tree, so strictly speaking we only build a matcher. e() { t() ; e (); if token!= EOF then HALT; } e () { if token == PLUS then get_next_token(); t(); e (); } t() { f(); t (); } t () { if token == TIMES then get_next_token(); f(); t (); } f () { if token == IDENT then get_next_token(); else if token == OPENPAR then get_next_token(); e(); if token==closepar then get_next_token(); else HALT; } Martin Sulzmann Syntax Analysis 32 / 38
33 Table Driven Parsing Fact A program making use of recursive function can be simulated by a non-recursive program (by making use of a stack). Table Driven Approach Input Stack elements are elements in our voc Input consists of terminal symbols Table encodes LL(1) actions Stack LL(1) parser Parsing table M Martin Sulzmann Syntax Analysis 33 / 38
34 Parse Table Construction Algorithm Input: Grammar G Output: Parsing table M Method: 1 For each production A α: 1 For each a First(α) add A α to M[A,a]. 2 If ǫ First(α): 1 For each b Follow(A) add A α to M[A,b]. 2 If $ Follow(A), add A α to M[A,$]. 2 Set each undefined entry of M to error (or leave empty). If there are multiple entries then grammar is not LL(1). Martin Sulzmann Syntax Analysis 34 / 38
35 Table Driven Parsing Algorithm Push $; Push start symbol while Top of stack is not $ { X:= top of stack; a:= input symbol if X is a terminal then if X == a then pop X; advance input else error else if M[X,a] == X->Y1... Yk then pop X; push Yk,..., Y1 else error } Martin Sulzmann Syntax Analysis 35 / 38
36 Example First and Follow Sets E TE First(E) = {ident,(} Follow(E) = {$,)} E +TE First(E ) = {+,ǫ} Follow(E ) = {$,)} E ǫ T FT First(T) = {ident,(} Follow(T) = {$,+,)} T FT First(T ) = {,ǫ} Follow(T ) = {$,+,)} T ǫ F ident First(F) = {ident,(} Follow(F) = {$,+,,)} F (E) Martin Sulzmann Syntax Analysis 36 / 38
37 Parse Table + Execution ident + ( ) $ E E TE E TE E E +TE E ǫ E ǫ T T FT T FT T T ǫ T FT T ǫ T ǫ F F ident F (E) Stack Input Rule Stack Input Rule $E id + (id) E TE $E T )E id) E TE $E T id + (id) T FT $E T )E T id) T FT $E T F id + (id) F id $E T )E T F id) F id $E T id id + (id) $E T )E T id id) $E T +(id) T ǫ $E T )E T ) T ǫ $E +(id) E +TE $E T )E ) E ǫ $E T+ +(id) $E T ) ) $E T (id) T FT $E T $ T ǫ $E T F (id) F (E) $E $ E ǫ $E T )E( (id) $ $ Martin Sulzmann Syntax Analysis 37 / 38
38 Top-Down Parsing Summary Predictive parsing (removal left recursion, left factoring). LL(1) grammars. Recursive descent parsing. Table driven parsing. How to construct parse tree? (later) Error recovery important (but no time!), please consult text book. ANTRL state-of-the art LL-style parser for Java. Martin Sulzmann Syntax Analysis 38 / 38
CA Compiler Construction
CA4003 - Compiler Construction David Sinclair A top-down parser starts with the root of the parse tree, labelled with the goal symbol of the grammar, and repeats the following steps until the fringe of
More informationParsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing
Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides
More information3. Parsing. Oscar Nierstrasz
3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/
More informationTable-Driven Parsing
Table-Driven Parsing It is possible to build a non-recursive predictive parser by maintaining a stack explicitly, rather than implicitly via recursive calls [1] The non-recursive parser looks up the production
More informationTypes of parsing. CMSC 430 Lecture 4, Page 1
Types of parsing Top-down parsers start at the root of derivation tree and fill in picks a production and tries to match the input may require backtracking some grammars are backtrack-free (predictive)
More informationSyntactic Analysis. Top-Down Parsing
Syntactic Analysis Top-Down Parsing Copyright 2017, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make
More informationCompilerconstructie. najaar Rudy van Vliet kamer 140 Snellius, tel rvvliet(at)liacs(dot)nl. college 3, vrijdag 22 september 2017
Compilerconstructie najaar 2017 http://www.liacs.leidenuniv.nl/~vlietrvan1/coco/ Rudy van Vliet kamer 140 Snellius, tel. 071-527 2876 rvvliet(at)liacs(dot)nl college 3, vrijdag 22 september 2017 + werkcollege
More informationCS2210: Compiler Construction Syntax Analysis Syntax Analysis
Comparison with Lexical Analysis The second phase of compilation Phase Input Output Lexer string of characters string of tokens Parser string of tokens Parse tree/ast What Parse Tree? CS2210: Compiler
More informationConcepts Introduced in Chapter 4
Concepts Introduced in Chapter 4 Grammars Context-Free Grammars Derivations and Parse Trees Ambiguity, Precedence, and Associativity Top Down Parsing Recursive Descent, LL Bottom Up Parsing SLR, LR, LALR
More informationSYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic
More informationTop down vs. bottom up parsing
Parsing A grammar describes the strings that are syntactically legal A recogniser simply accepts or rejects strings A generator produces sentences in the language described by the grammar A parser constructs
More informationCS1622. Today. A Recursive Descent Parser. Preliminaries. Lecture 9 Parsing (4)
CS1622 Lecture 9 Parsing (4) CS 1622 Lecture 9 1 Today Example of a recursive descent parser Predictive & LL(1) parsers Building parse tables CS 1622 Lecture 9 2 A Recursive Descent Parser. Preliminaries
More informationCompilers. Yannis Smaragdakis, U. Athens (original slides by Sam
Compilers Parsing Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Next step text chars Lexical analyzer tokens Parser IR Errors Parsing: Organize tokens into sentences Do tokens conform
More informationDEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
KATHMANDU UNIVERSITY SCHOOL OF ENGINEERING DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING REPORT ON NON-RECURSIVE PREDICTIVE PARSER Fourth Year First Semester Compiler Design Project Final Report submitted
More informationCS 314 Principles of Programming Languages
CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution
More information8 Parsing. Parsing. Top Down Parsing Methods. Parsing complexity. Top down vs. bottom up parsing. Top down vs. bottom up parsing
8 Parsing Parsing A grammar describes syntactically legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string
More informationSyntax Analysis. Prof. James L. Frankel Harvard University. Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved.
Syntax Analysis Prof. James L. Frankel Harvard University Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved. Context-Free Grammar (CFG) terminals non-terminals start
More information3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino
3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the
More informationSection A. A grammar that produces more than one parse tree for some sentences is said to be ambiguous.
Section A 1. What do you meant by parser and its types? A parser for grammar G is a program that takes as input a string w and produces as output either a parse tree for w, if w is a sentence of G, or
More informationAbstract Syntax Trees & Top-Down Parsing
Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationAbstract Syntax Trees & Top-Down Parsing
Abstract Syntax Trees & Top-Down Parsing Review of Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationAbstract Syntax Trees & Top-Down Parsing
Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationTop-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7
Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess
More informationIntroduction to parsers
Syntax Analysis Introduction to parsers Context-free grammars Push-down automata Top-down parsing LL grammars and parsers Bottom-up parsing LR grammars and parsers Bison/Yacc - parser generators Error
More informationLL(k) Parsing. Predictive Parsers. LL(k) Parser Structure. Sample Parse Table. LL(1) Parsing Algorithm. Push RHS in Reverse Order 10/17/2012
Predictive Parsers LL(k) Parsing Can we avoid backtracking? es, if for a given input symbol and given nonterminal, we can choose the alternative appropriately. his is possible if the first terminal of
More informationCS 321 Programming Languages and Compilers. VI. Parsing
CS 321 Programming Languages and Compilers VI. Parsing Parsing Calculate grammatical structure of program, like diagramming sentences, where: Tokens = words Programs = sentences For further information,
More informationCompilers: CS31003 Computer Sc & Engg: IIT Kharagpur 1. Top-Down Parsing. Lect 5. Goutam Biswas
Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 1 Top-Down Parsing Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 2 Non-terminal as a Function In a top-down parser a non-terminal may be viewed
More informationCS502: Compilers & Programming Systems
CS502: Compilers & Programming Systems Top-down Parsing Zhiyuan Li Department of Computer Science Purdue University, USA There exist two well-known schemes to construct deterministic top-down parsers:
More informationCompiler Design 1. Top-Down Parsing. Goutam Biswas. Lect 5
Compiler Design 1 Top-Down Parsing Compiler Design 2 Non-terminal as a Function In a top-down parser a non-terminal may be viewed as a generator of a substring of the input. We may view a non-terminal
More informationWednesday, September 9, 15. Parsers
Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda
More informationParsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:
What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda
More informationParsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468
Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure
More informationParsing Part II (Top-down parsing, left-recursion removal)
Parsing Part II (Top-down parsing, left-recursion removal) Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit
More informationTop-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7
Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess
More informationParser Generation. Bottom-Up Parsing. Constructing LR Parser. LR Parsing. Construct parse tree bottom-up --- from leaves to the root
Parser Generation Main Problem: given a grammar G, how to build a top-down parser or a bottom-up parser for it? parser : a program that, given a sentence, reconstructs a derivation for that sentence ----
More informationCSCI312 Principles of Programming Languages
Copyright 2006 The McGraw-Hill Companies, Inc. CSCI312 Principles of Programming Languages! LL Parsing!! Xu Liu Derived from Keith Cooper s COMP 412 at Rice University Recap Copyright 2006 The McGraw-Hill
More informationPART 3 - SYNTAX ANALYSIS. F. Wotawa TU Graz) Compiler Construction Summer term / 309
PART 3 - SYNTAX ANALYSIS F. Wotawa (IST @ TU Graz) Compiler Construction Summer term 2016 64 / 309 Goals Definition of the syntax of a programming language using context free grammars Methods for parsing
More informationA programming language requires two major definitions A simple one pass compiler
A programming language requires two major definitions A simple one pass compiler [Syntax: what the language looks like A context-free grammar written in BNF (Backus-Naur Form) usually suffices. [Semantics:
More informationMonday, September 13, Parsers
Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions
More informationBottom up parsing. The sentential forms happen to be a right most derivation in the reverse order. S a A B e a A d e. a A d e a A B e S.
Bottom up parsing Construct a parse tree for an input string beginning at leaves and going towards root OR Reduce a string w of input to start symbol of grammar Consider a grammar S aabe A Abc b B d And
More informationChapter 4. Lexical and Syntax Analysis. Topics. Compilation. Language Implementation. Issues in Lexical and Syntax Analysis.
Topics Chapter 4 Lexical and Syntax Analysis Introduction Lexical Analysis Syntax Analysis Recursive -Descent Parsing Bottom-Up parsing 2 Language Implementation Compilation There are three possible approaches
More informationNote that for recursive descent to work, if A ::= B1 B2 is a grammar rule we need First k (B1) disjoint from First k (B2).
LL(k) Grammars We need a bunch of terminology. For any terminal string a we write First k (a) is the prefix of a of length k (or all of a if its length is less than k) For any string g of terminal and
More informationContext-free grammars
Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,
More informationCS 406/534 Compiler Construction Parsing Part I
CS 406/534 Compiler Construction Parsing Part I Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof. Ken Kennedy and Dr.
More informationWednesday, August 31, Parsers
Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically
More informationCompiler Construction: Parsing
Compiler Construction: Parsing Mandar Mitra Indian Statistical Institute M. Mitra (ISI) Parsing 1 / 33 Context-free grammars. Reference: Section 4.2 Formal way of specifying rules about the structure/syntax
More informationPart 3. Syntax analysis. Syntax analysis 96
Part 3 Syntax analysis Syntax analysis 96 Outline 1. Introduction 2. Context-free grammar 3. Top-down parsing 4. Bottom-up parsing 5. Conclusion and some practical considerations Syntax analysis 97 Structure
More informationCMSC 330: Organization of Programming Languages
CMSC 330: Organization of Programming Languages Context Free Grammars and Parsing 1 Recall: Architecture of Compilers, Interpreters Source Parser Static Analyzer Intermediate Representation Front End Back
More informationTopdown parsing with backtracking
Top down parsing Types of parsers: Top down: repeatedly rewrite the start symbol; find a left-most derivation of the input string; easy to implement; not all context-free grammars are suitable. Bottom
More informationCOMP 181. Prelude. Next step. Parsing. Study of parsing. Specifying syntax with a grammar
COMP Lecture Parsing September, 00 Prelude What is the common name of the fruit Synsepalum dulcificum? Miracle fruit from West Africa What is special about miracle fruit? Contains a protein called miraculin
More informationSyntax Analysis Check syntax and construct abstract syntax tree
Syntax Analysis Check syntax and construct abstract syntax tree if == = ; b 0 a b Error reporting and recovery Model using context free grammars Recognize using Push down automata/table Driven Parsers
More informationParsing III. (Top-down parsing: recursive descent & LL(1) )
Parsing III (Top-down parsing: recursive descent & LL(1) ) Roadmap (Where are we?) Previously We set out to study parsing Specifying syntax Context-free grammars Ambiguity Top-down parsers Algorithm &
More informationPrelude COMP 181 Tufts University Computer Science Last time Grammar issues Key structure meaning Tufts University Computer Science
Prelude COMP Lecture Topdown Parsing September, 00 What is the Tufts mascot? Jumbo the elephant Why? P. T. Barnum was an original trustee of Tufts : donated $0,000 for a natural museum on campus Barnum
More informationIntroduction to Parsing. Comp 412
COMP 412 FALL 2010 Introduction to Parsing Comp 412 Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit permission to make
More information4. Lexical and Syntax Analysis
4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal
More informationAcknowledgements. The slides for this lecture are a modified versions of the offering by Prof. Sanjeev K Aggarwal
Acknowledgements The slides for this lecture are a modified versions of the offering by Prof. Sanjeev K Aggarwal Syntax Analysis Check syntax and construct abstract syntax tree if == = ; b 0 a b Error
More informationVIVA QUESTIONS WITH ANSWERS
VIVA QUESTIONS WITH ANSWERS 1. What is a compiler? A compiler is a program that reads a program written in one language the source language and translates it into an equivalent program in another language-the
More informationCompilers. Predictive Parsing. Alex Aiken
Compilers Like recursive-descent but parser can predict which production to use By looking at the next fewtokens No backtracking Predictive parsers accept LL(k) grammars L means left-to-right scan of input
More informationSome Basic Definitions. Some Basic Definitions. Some Basic Definitions. Language Processing Systems. Syntax Analysis (Parsing) Prof.
Language Processing Systems Prof. Mohamed Hamada Software ngineering Lab. he University of Aizu Japan Syntax Analysis (Parsing) Some Basic Definitions Some Basic Definitions syntax: the way in which words
More informationLexical and Syntax Analysis. Top-Down Parsing
Lexical and Syntax Analysis Top-Down Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax
More informationParsing. Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice.
Parsing Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice. Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students
More information4. Lexical and Syntax Analysis
4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal
More informationWWW.STUDENTSFOCUS.COM UNIT -3 SYNTAX ANALYSIS 3.1 ROLE OF THE PARSER Parser obtains a string of tokens from the lexical analyzer and verifies that it can be generated by the language for the source program.
More informationChapter 3. Parsing #1
Chapter 3 Parsing #1 Parser source file get next character scanner get token parser AST token A parser recognizes sequences of tokens according to some grammar and generates Abstract Syntax Trees (ASTs)
More informationSyntax Analysis. Amitabha Sanyal. (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay
Syntax Analysis (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay September 2007 College of Engineering, Pune Syntax Analysis: 2/124 Syntax
More informationSyntactic Analysis. Chapter 4. Compiler Construction Syntactic Analysis 1
Syntactic Analysis Chapter 4 Compiler Construction Syntactic Analysis 1 Context-free Grammars The syntax of programming language constructs can be described by context-free grammars (CFGs) Relatively simple
More informationParsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones
Parsing III (Top-down parsing: recursive descent & LL(1) ) (Bottom-up parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,
More informationFall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev
Fall 2017-2018 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman
More informationChapter 4. Lexical and Syntax Analysis
Chapter 4 Lexical and Syntax Analysis Chapter 4 Topics Introduction Lexical Analysis The Parsing Problem Recursive-Descent Parsing Bottom-Up Parsing Copyright 2012 Addison-Wesley. All rights reserved.
More informationSyntax Analysis Part I
Syntax Analysis Part I Chapter 4: Context-Free Grammars Slides adapted from : Robert van Engelen, Florida State University Position of a Parser in the Compiler Model Source Program Lexical Analyzer Token,
More informationParsing. source code. while (k<=n) {sum = sum+k; k=k+1;}
Compiler Construction Grammars Parsing source code scanner tokens regular expressions lexical analysis Lennart Andersson parser context free grammar Revision 2012 01 23 2012 parse tree AST builder (implicit)
More informationIntroduction to Syntax Analysis
Compiler Design 1 Introduction to Syntax Analysis Compiler Design 2 Syntax Analysis The syntactic or the structural correctness of a program is checked during the syntax analysis phase of compilation.
More informationSometimes an ambiguous grammar can be rewritten to eliminate the ambiguity.
Eliminating Ambiguity Sometimes an ambiguous grammar can be rewritten to eliminate the ambiguity. Example: consider the following grammar stat if expr then stat if expr then stat else stat other One can
More informationCompiler Construction 2016/2017 Syntax Analysis
Compiler Construction 2016/2017 Syntax Analysis Peter Thiemann November 2, 2016 Outline 1 Syntax Analysis Recursive top-down parsing Nonrecursive top-down parsing Bottom-up parsing Syntax Analysis tokens
More informationBuilding Compilers with Phoenix
Building Compilers with Phoenix Syntax-Directed Translation Structure of a Compiler Character Stream Intermediate Representation Lexical Analyzer Machine-Independent Optimizer token stream Intermediate
More informationCSE302: Compiler Design
CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University February 20, 2007 Outline Recap
More informationBottom-up parsing. Bottom-Up Parsing. Recall. Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form
Bottom-up parsing Bottom-up parsing Recall Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form If α V t,thenα is called a sentence in L(G) Otherwise it is just
More informationSyntax Analysis. COMP 524: Programming Language Concepts Björn B. Brandenburg. The University of North Carolina at Chapel Hill
Syntax Analysis Björn B. Brandenburg The University of North Carolina at Chapel Hill Based on slides and notes by S. Olivier, A. Block, N. Fisher, F. Hernandez-Campos, and D. Stotts. The Big Picture Character
More information1 Introduction. 2 Recursive descent parsing. Predicative parsing. Computer Language Implementation Lecture Note 3 February 4, 2004
CMSC 51086 Winter 2004 Computer Language Implementation Lecture Note 3 February 4, 2004 Predicative parsing 1 Introduction This note continues the discussion of parsing based on context free languages.
More informationDerivations vs Parses. Example. Parse Tree. Ambiguity. Different Parse Trees. Context Free Grammars 9/18/2012
Derivations vs Parses Grammar is used to derive string or construct parser Context ree Grammars A derivation is a sequence of applications of rules Starting from the start symbol S......... (sentence)
More information4 (c) parsing. Parsing. Top down vs. bo5om up parsing
4 (c) parsing Parsing A grammar describes syntac2cally legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string
More informationThe analysis part breaks up the source program into constituent pieces and creates an intermediate representation of the source program.
COMPILER DESIGN 1. What is a compiler? A compiler is a program that reads a program written in one language the source language and translates it into an equivalent program in another language-the target
More informationBuilding a Parser III. CS164 3:30-5:00 TT 10 Evans. Prof. Bodik CS 164 Lecture 6 1
Building a Parser III CS164 3:30-5:00 TT 10 Evans 1 Overview Finish recursive descent parser when it breaks down and how to fix it eliminating left recursion reordering productions Predictive parsers (aka
More informationAdministrativia. WA1 due on Thu PA2 in a week. Building a Parser III. Slides on the web site. CS164 3:30-5:00 TT 10 Evans.
Administrativia Building a Parser III CS164 3:30-5:00 10 vans WA1 due on hu PA2 in a week Slides on the web site I do my best to have slides ready and posted by the end of the preceding logical day yesterday,
More informationTable-Driven Top-Down Parsers
Table-Driven Top-Down Parsers Recursive descent parsers have many attractive features. They are actual pieces of code that can be read by programmers and extended. This makes it fairly easy to understand
More informationCSE 130 Programming Language Principles & Paradigms Lecture # 5. Chapter 4 Lexical and Syntax Analysis
Chapter 4 Lexical and Syntax Analysis Introduction - Language implementation systems must analyze source code, regardless of the specific implementation approach - Nearly all syntax analysis is based on
More information20. Compilers. source language program. target language program. translator
20. Compilers When a programmer writes down a solution of her problems, she writes a program on a special programming language. These programming languages are very different from the proper languages
More informationICOM 4036 Spring 2004
Language Specification and Translation ICOM 4036 Spring 2004 Lecture 3 Copyright 2004 Pearson Addison-Wesley. All rights reserved. 3-1 Language Specification and Translation Topics Structure of a Compiler
More informationIntroduction to Syntax Analysis. The Second Phase of Front-End
Compiler Design IIIT Kalyani, WB 1 Introduction to Syntax Analysis The Second Phase of Front-End Compiler Design IIIT Kalyani, WB 2 Syntax Analysis The syntactic or the structural correctness of a program
More informationAmbiguity, Precedence, Associativity & Top-Down Parsing. Lecture 9-10
Ambiguity, Precedence, Associativity & Top-Down Parsing Lecture 9-10 (From slides by G. Necula & R. Bodik) 9/18/06 Prof. Hilfinger CS164 Lecture 9 1 Administrivia Please let me know if there are continued
More informationParsing II Top-down parsing. Comp 412
COMP 412 FALL 2018 Parsing II Top-down parsing Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled
More informationCOMP3131/9102: Programming Languages and Compilers
COMP3131/9102: Programming Languages and Compilers Jingling Xue School of Computer Science and Engineering The University of New South Wales Sydney, NSW 2052, Australia http://www.cse.unsw.edu.au/~cs3131
More informationSyntax Analysis. The Big Picture. The Big Picture. COMP 524: Programming Languages Srinivas Krishnan January 25, 2011
Syntax Analysis COMP 524: Programming Languages Srinivas Krishnan January 25, 2011 Based in part on slides and notes by Bjoern Brandenburg, S. Olivier and A. Block. 1 The Big Picture Character Stream Token
More informationLexical and Syntax Analysis
Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Easy for humans to write and understand String of characters
More informationCMSC 330: Organization of Programming Languages. Architecture of Compilers, Interpreters
: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Scanner Parser Static Analyzer Intermediate Representation Front End Back End Compiler / Interpreter
More informationCompilation Lecture 3: Syntax Analysis: Top-Down parsing. Noam Rinetzky
Compilation 0368-3133 Lecture 3: Syntax Analysis: Top-Down parsing Noam Rinetzky 1 Recursive descent parsing Define a function for every nonterminal Every function work as follows Find applicable production
More informationThe Parsing Problem (cont d) Recursive-Descent Parsing. Recursive-Descent Parsing (cont d) ICOM 4036 Programming Languages. The Complexity of Parsing
ICOM 4036 Programming Languages Lexical and Syntax Analysis Lexical Analysis The Parsing Problem Recursive-Descent Parsing Bottom-Up Parsing This lecture covers review questions 14-27 This lecture covers
More informationCS308 Compiler Principles Syntax Analyzer Li Jiang
CS308 Syntax Analyzer Li Jiang Department of Computer Science and Engineering Shanghai Jiao Tong University Syntax Analyzer Syntax Analyzer creates the syntactic structure of the given source program.
More informationMIT Parse Table Construction. Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology
MIT 6.035 Parse Table Construction Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Parse Tables (Review) ACTION Goto State ( ) $ X s0 shift to s2 error error goto s1
More informationArchitecture of Compilers, Interpreters. CMSC 330: Organization of Programming Languages. Front End Scanner and Parser. Implementing the Front End
Architecture of Compilers, Interpreters : Organization of Programming Languages ource Analyzer Optimizer Code Generator Context Free Grammars Intermediate Representation Front End Back End Compiler / Interpreter
More information