CS 321 Programming Languages and Compilers. VI. Parsing
|
|
- Julius Holt
- 5 years ago
- Views:
Transcription
1 CS 321 Programming Languages and Compilers VI. Parsing
2 Parsing Calculate grammatical structure of program, like diagramming sentences, where: Tokens = words Programs = sentences For further information, read: Aho, Sethi, Ullman, Compilers: Principles, Techniques, and Tools (a.k.a, the Dragon Book ) 2
3 Outline of coverage Context-free grammars Parsing Tabular Parsing Methods One pass» Top-down Yacc» Bottom-up 3
4 What parser does: Extracts grammatical structure of program function-def name arguments stmt-list main stmt expression expression variable cout operator << expression string hello, world\n 4
5 Context-free languages Grammatical structure defined by context-free grammar. statement fi labeled-statement expression-statement compound-statement labeled-statement fi ident: statement case constant-expression : statement compound-statement fi { declaration-list statement-list } Context-free = only one non-terminal in left-part. terminal non-terminal 5
6 Parse trees Parse tree = tree labeled with grammar symbols, such that: If node is labeled A, and its children are labeled x 1...x n, then there is a production A fi x 1...x n Parse tree from A = root labeled with A Complete parse tree = all leaves labeled with tokens 6
7 Parse trees and sentences Frontier of tree = labels on leaves (in left-to-right order) Frontier of tree from S is a sentential form. Frontier of a complete tree from S is a sentence. L L ; E E Frontier a 7
8 Example G: L fi L ; E E E fi a b Syntax trees from start symbol (L): L E a L E a L ; E L E a L L ; ; E b E b Sentential forms: a a;e a;b;b 8
9 Derivations Alternate definition of sentence: Given a, b in V*, say a b is a derivation step if a = a Aa and b = a ga, where A fi g is a production g is a sentential form iff there exists a derivation (sequence of derivation steps) S... g ( alternatively, we say that S * g ) Two definitions are equivalent, but note that there are many derivations corresponding to each parse tree. 9
10 Another example H: L fi E ; L E E fi a b L E a E L ; L E E b L ; E b L ; L E a a 10
11 Ambiguity For some purposes, it is important to know whether a sentence can have more than one parse tree. A grammar is ambiguous if there is a sentence with more than one parse tree. Example: E fi E+E E*E id E E E + E E * E id E * E E + E id id id id id 11
12 Ambiguity Ambiguity is a function of the grammar rather than the language. Certain unambiguous grammars may have equivalent ambiguous ones. 12
13 Grammar Transformations Grammars can be transformed without affecting the language generated. Three transformations are discussed next: Eliminating Ambiguity Eliminating Left Recursion (i.e.productions of the form AfiA ) Left Factoring 13
14 Grammar Transformation 1. Eliminating Ambiguity Sometimes an ambiguous grammar can be rewritten to eliminate ambiguity. For example, expressions involving additions and products can be written as follows: E fi E+T T T fi T*id id The language generated by this grammar is the same as that generated by the grammar on tranparency 11. Both generate id(+id *id)* However, this grammar is not ambiguous. 14
15 Grammar Transformation 1. Eliminating Ambiguity (Cont.) One advantage of this grammar is that it represents the precedence between operators. In the parsing tree, products appear nested within additions E E + T T id T id * id 15
16 Grammar Transformation 1. Eliminating Ambiguity (Cont.) The most famous example of ambiguity in a programming language is the dangling else. Consider S fi if bthen Selse S if b then S a 16
17 Grammar Transformation 1. Eliminating Ambiguity (Cont.) When there are two nested ifs and only one else.. S if β then S else S if βthen S α S α if β then S if β then S else S α α 17
18 Grammar Transformation 1. Eliminating Ambiguity (Cont.) In most languages (including C++ and Java), each else is assumed to belong to the nearest if that is not already matched by an else. This association is expressed in the following (unambiguous) grammar: S fi Matched Unmatched Matched fi if b then Matched else Matched a Unmatched fi if b then S if b then Matched else Unmatched 18
19 Grammar Transformation 1. Eliminating Ambiguity (Cont.) Ambiguity is a function of the grammar It is undecidable whether a context free grammar is ambiguous. The proof is done by reduction to Post s correspondence problem. Although there is no general algorithm, it is possible to isolate certain constructs in productions which lead to ambiguous grammars. 19
20 Grammar Transformation 1. Eliminating Ambiguity (Cont.) For example, a grammar containg the production AfiAA would be ambiguous, because the substring aaa has two parses. A A A A A A A A α α A A α α This ambiguity disappears if we use the productions AfiAB B and Bfi or the productions AfiBA B and Bfi. α α 20
21 Grammar Transformation 1. Eliminating Ambiguity (Cont.) Other three examples of ambiguous productions are: AfiAaA AfiaA Ab and AfiaA aaba A language generated by an ambiguous Context Free Grammar is inherently ambiguous if it has no unambiguous Context Free Grammar. (This can be proven formally) An example of such a language is L={a i b j c m i=j or j=m} which can be generated by the grammar: SfiAB DC AfiaA e BfibBc e CficC e DfiaDb e 21
22 Grammar Transformations 2. Elimination of Left Recursion A grammar is left recursive if it has a nonterminal A and a derivation A + Aa for some string a. Top-down parsing methods (to be discussed shortly) cannot handle left-recursive grammars, so a transformation to eliminate left recursion is needed. Immediate left recursion (productions of the form AfiA ) can be easily eliminated. We group the A-productions as AfiA 1 A 2 A m b 1 b 2 b n where no b i begins with A. Then we replace the A- productions by Afi b 1 A b 2 A b n A A fi 1 A 2 A m A e 22
23 Grammar Transformations 2. Elimination of Left Recursion (Cont.) The previous transformation, however, does not eliminate left recursion involving two or more steps. For example, consider the grammar SfiAa b AfiAc Sd e S is left-recursive because S Aa Sda, but it is not immediately left recursive. 23
24 Grammar Transformations 2. Elimination of Left Recursion (Cont.) Algorithm. Eliminate left recursion Arrange nonterminals in some order A 1, A 2,,, A n for i =1 to n { } for j =1 to i -1 { replace each production of the form A i A j γ by the production A i δ 1 γ δ 2 γ δ n γ where A j δ 1 δ 2 δ n are all the current A j -productions } eliminate the immediate left recursion among the A i -productions 24
25 Grammar Transformations 2. Elimination of Left Recursion (Cont.) To show that the previous algorithm actually works all we need notice is that iteration i only changes productions with A i on the left-hand side. And m > i in all productions of the form A i fia m. This can be easily shown by induction. It is clearly true for i=1. If it is true for all i<k, then when the outer loop is executed for i=k, the inner loop will remove all productions A i fia m with m < i. Finally, with the elimination of self recursion, m in the A i fia m productions is forced to be > i. So, at the end of the algorithm, all derivations of the form A + i A m a will have m > i and therefore left recursion would not be possible. 25
26 Grammar Transformations 3. Left Factoring Left factoring helps transform a grammar for predictive parsing For example, if we have the two productions S fi if bthen Selse S if bthen S on seeing the input token if, we cannot immediately tell which production to choose to expand S. In general, if we have Afi b 1 b 2 and the input begins with a, we do not know (without looking further) which production to use to expand A. 26
27 Grammar Transformations 3. Left Factoring(Cont.) However, we may defer the decision by expanding A to aa. Then after seeing the input derived from a, we may expand A to b 1 or to b 2. That is, left-factored, the original productions become Afi A A fi b 1 b 2 27
28 Non-Context-Free Language Constructs Examples of non-context-free languages are: L 1 ={wcw w is of the form (a b)*} L 2 ={a n b m c n d m n 1 and m 1 } L 3 ={a n b n c n n 0 } Languages similar to these that are context free L 1 ={wcw R w is of the form (a b)*} (w R stands for w reversed) This language is generated by the grammar» Sfi asa bsb c L 2 ={a n b m c m d n n 1 and m 1 } This language is generated by the grammar» Sfi asd aad» Afi bac bc 28
29 Non-Context-Free Language Constructs (Cont.) L 2 ={a n b n c m d m n 1 and m 1 } This language is generated by the grammar» Sfi AB» Afi aab ab» Bfi cbd cd L 3 ={a n b n n 1} This language is generated by the grammar» Sfi asb ab This language is not definable by any regular expression 29
30 Non-Context-Free Language Constructs (Cont.) Suppose we could construct a DFSM D accepting L 3. D must have a finite number of states, say k. Consider the sequence of states s 0, s 1, s 2,, s k entered by D having read e, a, aa,, a k. Since D only has k states, two of the states in the sequence have to be equal. Say, s i s j (i j). From s i, a sequence of i bs leads to an accepting (final) state. Therefore, the same sequence of i bs will also lead to an accepting state from s j. Therefore D would accept a j b i which means that the language accepted by D is not identical to L 3. A contradiction. 30
31 Parsing The parsing problem is: Given string of tokens w, find a parse tree whose frontier is w. (Equivalently, find a derivation from w.) A parser for a grammar G reads a list of tokens and finds a parse tree if they form a sentence (or reports an error otherwise) Two classes of algorithms for parsing: Top-down Bottom-up 31
32 Parser generators A parser generator is a program that reads a grammar and produces a parser. The best known parser generator is yacc. Both produce bottom-up parsers. Most parser generators - including yacc - do not work for every cfg; they accept a restricted class of cfg s that can be parsed efficiently using the method employed by that parser generator. 32
33 Top-down parsing Starting from parse tree containing just S, build tree down toward input. Expand left-most nonterminal. Algorithm: (next slide) 33
34 Top-down parsing (cont.) Let input = a 1 a 2...a n current sentential form (csf) = S loop { suppose csf = t 1...t k Aa if t 1...t k a 1...a k, it s an error based on a k+1..., choose production A fib csf becomes t 1...t k ba } 34
35 Top-down parsing example Grammar: H: L fi E ; L E E fi a b Input: a;b Parse tree Sentential form Input E E a L L a;b L E;L a;b ; L L a;l a;b ; L 35
36 Top-down parsing example (cont.) Parse tree Sentential form Input L E ; L a E L E ; L a E b a;e a;b a;b a;b 36
37 LL(1) parsing Efficient form of top-down parsing. Use only first symbol of remaining input (a k+1 ) to choose next production. That is, employ a function M:S Nfi P in choose production step of algorithm. When this works, grammar is (usually) called LL(1). (More precise definition to follow.) 37
38 LL(1) examples Example 1: H: L fi E ; L E E fi a b Given input a;b, so next symbol is a. Which production to use? Can t tell. \ H not LL(1). 38
39 LL(1) examples Example 2: Exp fi Term Exp Exp fi $ + Exp Term fi id (Use $ for end-of-input symbol.) Grammar is LL(1): Exp and Term have only one production; Exp has two productions but only one is applicable at any time. 39
40 Nonrecursive predictive parsing It is possible to build a nonrecursive predictive parser by maintaining as stack explicitly, rather tan implicitly via recursive calls. The key problem during predictive parsing is that of determining the production to be applied for a non-terminal. 40
41 Nonrecursive predictive parsing Algorithm. Nonrecursive predictive parsing Set ip to point to the first symbol of w$. repeat Let X be the top of the stack symbol and a the symbol pointed to by ip if X is a terminal or $ then if X == a then else else error() // X is a nonterminal pop X from the stack and advance ip if M[X,a] == X Y 1 Y 2 Y k then pop X from the stack push Y k Y k-1,, Y 1 onto the stack with Y 1 on top (push nothing if Y 1 Y 2 Y k is ε ) until X == $ output the production X Y 1 Y 2 Y k else error() 41
42 LL(1) grammars No left recursion. A fi Aa : If this production is chosen, parse makes no progress. No common prefixes. A fi ab ag Can fix by left factoring : A fi aa A fi b g 42
43 LL(1) grammars (cont.) No ambiguity. Precise definition requires that production to choose be unique ( choose function M very hard to calculate otherwise). 43
44 Top-down Parsing Input tokens: <t0,t1,,t-i,...> L E0 E-n Start symbol and root of parse tree Input tokens: <t-i,...> From left to right, grow the parse tree downwards L E0 E-n... 44
45 Checking LL(1)-ness For any sequence of grammar symbols a, define set FIRST(a) S to be those tokens a such that a ab for some b. (Notation: write a *ab.) 45
46 Checking LL(1)-ness Define: Grammar G = (N,, P, S) is LL(1) if whenever there are two left-most derivations (in which the leftmost non-terminal is always expanded first ) S =>* waa => wba =>* wx S =>* waa => wga =>* wy Such that FIRST(x) = FIRST(y), it follows that b =g. In other words, given 1. A string waa in V* and 2. The first terminal symbol to be derived from Aa, say t There is at most one production that can be applied to A to yield a derivation of any terminal string beginning with wt. FIRST sets can often be calculated by inspection. 46
47 FIRST Sets Exp Term Exp Exp $ + Exp Term id (Use $ for end-of-input symbol.) FIRST(Term Exp ) = {id} FIRST($) = {$}, FIRST(+ Exp) = {+} implies FIRST($) FIRST(+ Exp) = {} FIRST(id) = {id} grammar is LL(1) 47
48 FIRST Sets H: L E ; L E E a b FIRST(E ; L) = {a,b} = FIRST(E) FIRST(E ; L) FIRST(E) {} H not LL(1). 48
49 How to compute FIRST Sets of Vocabulary Symbols Algorithm. Compute FIRST(X) for all grammar symbols X forall X V do FIRST(X)={} forall X Σ (X is a terminal) do FIRST(X)={X} forall productions X ε do FIRST(X) = FIRST(X) U {ε} repeat forall productions X Y 1 Y 2 Y k do forall i [1,k] do FIRST(X) = FIRST(X) U (FIRST(Y i ) - {ε}) if ε v FIRST(Y i ) then continue outer loop FIRST(X) = FIRST(X) U {ε} until no more terminals or ε are added to any FIRST set 49
50 How to compute FIRST Sets of Strings of Symbols FIRST(X 1 X 2 X n ) is the union of FIRST(X 1 ) and all FIRST(X i ) such that ε FIRST(X k ) for k=1,2,..,i-1 FIRST(X 1 X 2 X n ) contains ε iff ε FIRST(X k ) for k=1,2,..,n. 50
51 FIRST Sets do not Suffice Given the productions Afi T x Afi T y Tfi w Tfi e Tfi w should be applied when the next input token is w. Tfi e should be applied whenever the next terminal (the one pointed to by ip) is either x or y 51
52 FOLLOW Sets For any nonterminal X, define set FOLLOW(X) S to be those tokens a such that S *axab for some a and b. 52
53 How to compute the FOLLOW Set Algorithm. Compute FOLLOW(X) for all nonterminals X FOLLOW(S) ={$} forall productions A αbβ do FOLLOW(B)=Follow(B) U (FIRST(β) - {ε}) repeat forall productions A αb or A αbβ with ε FIRST(β) do FOLLOW(B) = FOLLOW(B) U FOLLOW(A) until all FOLLOW sets remain the same 53
54 Construction of a predictive parsing table Algorithm. Construction of a predictive parsing table M[:,:] = {} forall productions A α do forall a FIRST(α) do M[A,a] = M[A,a] U {A α } if ε FIRST(α) then forall b FOLLOW(A) do M[A,b] = M[A,b] U {A α } Make all empty entries of M be error 54
55 Another Definition of LL(1) Define: Grammar G is LL(1) if for every A N with productions A fi a 1... a n, FIRST(a i FOLLOW(A)) FIRST(a j FOLLOW(A) ) = for all i, j; 55
Syntax Analysis. Martin Sulzmann. Martin Sulzmann Syntax Analysis 1 / 38
Syntax Analysis Martin Sulzmann Martin Sulzmann Syntax Analysis 1 / 38 Syntax Analysis Objective Recognize individual tokens as sentences of a language (beyond regular languages). Example 1 (OK) Program
More informationTypes of parsing. CMSC 430 Lecture 4, Page 1
Types of parsing Top-down parsers start at the root of derivation tree and fill in picks a production and tries to match the input may require backtracking some grammars are backtrack-free (predictive)
More informationNote that for recursive descent to work, if A ::= B1 B2 is a grammar rule we need First k (B1) disjoint from First k (B2).
LL(k) Grammars We need a bunch of terminology. For any terminal string a we write First k (a) is the prefix of a of length k (or all of a if its length is less than k) For any string g of terminal and
More informationParsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing
Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides
More informationTable-Driven Parsing
Table-Driven Parsing It is possible to build a non-recursive predictive parser by maintaining a stack explicitly, rather than implicitly via recursive calls [1] The non-recursive parser looks up the production
More informationCS2210: Compiler Construction Syntax Analysis Syntax Analysis
Comparison with Lexical Analysis The second phase of compilation Phase Input Output Lexer string of characters string of tokens Parser string of tokens Parse tree/ast What Parse Tree? CS2210: Compiler
More informationCompilerconstructie. najaar Rudy van Vliet kamer 140 Snellius, tel rvvliet(at)liacs(dot)nl. college 3, vrijdag 22 september 2017
Compilerconstructie najaar 2017 http://www.liacs.leidenuniv.nl/~vlietrvan1/coco/ Rudy van Vliet kamer 140 Snellius, tel. 071-527 2876 rvvliet(at)liacs(dot)nl college 3, vrijdag 22 september 2017 + werkcollege
More information3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino
3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the
More informationCA Compiler Construction
CA4003 - Compiler Construction David Sinclair A top-down parser starts with the root of the parse tree, labelled with the goal symbol of the grammar, and repeats the following steps until the fringe of
More information3. Parsing. Oscar Nierstrasz
3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/
More informationSYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic
More informationAmbiguity, Precedence, Associativity & Top-Down Parsing. Lecture 9-10
Ambiguity, Precedence, Associativity & Top-Down Parsing Lecture 9-10 (From slides by G. Necula & R. Bodik) 9/18/06 Prof. Hilfinger CS164 Lecture 9 1 Administrivia Please let me know if there are continued
More informationSyntactic Analysis. Top-Down Parsing
Syntactic Analysis Top-Down Parsing Copyright 2017, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make
More informationTop-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7
Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess
More informationTop down vs. bottom up parsing
Parsing A grammar describes the strings that are syntactically legal A recogniser simply accepts or rejects strings A generator produces sentences in the language described by the grammar A parser constructs
More informationTop-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7
Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess
More informationAmbiguity. Grammar E E + E E * E ( E ) int. The string int * int + int has two parse trees. * int
Administrivia Ambiguity, Precedence, Associativity & op-down Parsing eam assignments this evening for all those not listed as having one. HW#3 is now available, due next uesday morning (Monday is a holiday).
More informationMonday, September 13, Parsers
Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions
More informationCompiler Design 1. Top-Down Parsing. Goutam Biswas. Lect 5
Compiler Design 1 Top-Down Parsing Compiler Design 2 Non-terminal as a Function In a top-down parser a non-terminal may be viewed as a generator of a substring of the input. We may view a non-terminal
More informationDEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
KATHMANDU UNIVERSITY SCHOOL OF ENGINEERING DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING REPORT ON NON-RECURSIVE PREDICTIVE PARSER Fourth Year First Semester Compiler Design Project Final Report submitted
More informationCS1622. Today. A Recursive Descent Parser. Preliminaries. Lecture 9 Parsing (4)
CS1622 Lecture 9 Parsing (4) CS 1622 Lecture 9 1 Today Example of a recursive descent parser Predictive & LL(1) parsers Building parse tables CS 1622 Lecture 9 2 A Recursive Descent Parser. Preliminaries
More informationIntroduction to Syntax Analysis. The Second Phase of Front-End
Compiler Design IIIT Kalyani, WB 1 Introduction to Syntax Analysis The Second Phase of Front-End Compiler Design IIIT Kalyani, WB 2 Syntax Analysis The syntactic or the structural correctness of a program
More informationCS502: Compilers & Programming Systems
CS502: Compilers & Programming Systems Top-down Parsing Zhiyuan Li Department of Computer Science Purdue University, USA There exist two well-known schemes to construct deterministic top-down parsers:
More informationParsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468
Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure
More informationIntroduction to Syntax Analysis
Compiler Design 1 Introduction to Syntax Analysis Compiler Design 2 Syntax Analysis The syntactic or the structural correctness of a program is checked during the syntax analysis phase of compilation.
More informationCompilers: CS31003 Computer Sc & Engg: IIT Kharagpur 1. Top-Down Parsing. Lect 5. Goutam Biswas
Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 1 Top-Down Parsing Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 2 Non-terminal as a Function In a top-down parser a non-terminal may be viewed
More informationWednesday, August 31, Parsers
Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically
More informationWednesday, September 9, 15. Parsers
Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda
More informationParsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:
What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda
More informationCS 314 Principles of Programming Languages
CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution
More informationCompilers. Yannis Smaragdakis, U. Athens (original slides by Sam
Compilers Parsing Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Next step text chars Lexical analyzer tokens Parser IR Errors Parsing: Organize tokens into sentences Do tokens conform
More informationCS308 Compiler Principles Syntax Analyzer Li Jiang
CS308 Syntax Analyzer Li Jiang Department of Computer Science and Engineering Shanghai Jiao Tong University Syntax Analyzer Syntax Analyzer creates the syntactic structure of the given source program.
More informationSection A. A grammar that produces more than one parse tree for some sentences is said to be ambiguous.
Section A 1. What do you meant by parser and its types? A parser for grammar G is a program that takes as input a string w and produces as output either a parse tree for w, if w is a sentence of G, or
More informationPart 3. Syntax analysis. Syntax analysis 96
Part 3 Syntax analysis Syntax analysis 96 Outline 1. Introduction 2. Context-free grammar 3. Top-down parsing 4. Bottom-up parsing 5. Conclusion and some practical considerations Syntax analysis 97 Structure
More informationConcepts Introduced in Chapter 4
Concepts Introduced in Chapter 4 Grammars Context-Free Grammars Derivations and Parse Trees Ambiguity, Precedence, and Associativity Top Down Parsing Recursive Descent, LL Bottom Up Parsing SLR, LR, LALR
More informationFormal Languages and Compilers Lecture V: Parse Trees and Ambiguous Gr
Formal Languages and Compilers Lecture V: Parse Trees and Ambiguous Grammars Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/
More informationParsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones
Parsing III (Top-down parsing: recursive descent & LL(1) ) (Bottom-up parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,
More informationBottom up parsing. The sentential forms happen to be a right most derivation in the reverse order. S a A B e a A d e. a A d e a A B e S.
Bottom up parsing Construct a parse tree for an input string beginning at leaves and going towards root OR Reduce a string w of input to start symbol of grammar Consider a grammar S aabe A Abc b B d And
More informationVIVA QUESTIONS WITH ANSWERS
VIVA QUESTIONS WITH ANSWERS 1. What is a compiler? A compiler is a program that reads a program written in one language the source language and translates it into an equivalent program in another language-the
More informationLexical and Syntax Analysis. Top-Down Parsing
Lexical and Syntax Analysis Top-Down Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax
More informationParser Generation. Bottom-Up Parsing. Constructing LR Parser. LR Parsing. Construct parse tree bottom-up --- from leaves to the root
Parser Generation Main Problem: given a grammar G, how to build a top-down parser or a bottom-up parser for it? parser : a program that, given a sentence, reconstructs a derivation for that sentence ----
More informationAdministrativia. WA1 due on Thu PA2 in a week. Building a Parser III. Slides on the web site. CS164 3:30-5:00 TT 10 Evans.
Administrativia Building a Parser III CS164 3:30-5:00 10 vans WA1 due on hu PA2 in a week Slides on the web site I do my best to have slides ready and posted by the end of the preceding logical day yesterday,
More informationAbstract Syntax Trees & Top-Down Parsing
Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationPART 3 - SYNTAX ANALYSIS. F. Wotawa TU Graz) Compiler Construction Summer term / 309
PART 3 - SYNTAX ANALYSIS F. Wotawa (IST @ TU Graz) Compiler Construction Summer term 2016 64 / 309 Goals Definition of the syntax of a programming language using context free grammars Methods for parsing
More informationSyntax Analysis Check syntax and construct abstract syntax tree
Syntax Analysis Check syntax and construct abstract syntax tree if == = ; b 0 a b Error reporting and recovery Model using context free grammars Recognize using Push down automata/table Driven Parsers
More informationLexical and Syntax Analysis
Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Easy for humans to write and understand String of characters
More informationThe analysis part breaks up the source program into constituent pieces and creates an intermediate representation of the source program.
COMPILER DESIGN 1. What is a compiler? A compiler is a program that reads a program written in one language the source language and translates it into an equivalent program in another language-the target
More informationContext-Free Grammars
Context-Free Grammars 1 Informal Comments A context-free grammar is a notation for describing languages. It is more powerful than finite automata or RE s, but still cannot define all possible languages.
More informationSyntax Analysis. Prof. James L. Frankel Harvard University. Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved.
Syntax Analysis Prof. James L. Frankel Harvard University Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved. Context-Free Grammar (CFG) terminals non-terminals start
More informationAbstract Syntax Trees & Top-Down Parsing
Abstract Syntax Trees & Top-Down Parsing Review of Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationAbstract Syntax Trees & Top-Down Parsing
Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationCompilers Course Lecture 4: Context Free Grammars
Compilers Course Lecture 4: Context Free Grammars Example: attempt to define simple arithmetic expressions using named regular expressions: num = [0-9]+ sum = expr "+" expr expr = "(" sum ")" num Appears
More informationCSCI312 Principles of Programming Languages
Copyright 2006 The McGraw-Hill Companies, Inc. CSCI312 Principles of Programming Languages! LL Parsing!! Xu Liu Derived from Keith Cooper s COMP 412 at Rice University Recap Copyright 2006 The McGraw-Hill
More informationCS415 Compilers. Syntax Analysis. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University
CS415 Compilers Syntax Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Limits of Regular Languages Advantages of Regular Expressions
More informationCMSC 330: Organization of Programming Languages
CMSC 330: Organization of Programming Languages Context Free Grammars and Parsing 1 Recall: Architecture of Compilers, Interpreters Source Parser Static Analyzer Intermediate Representation Front End Back
More informationParsing Part II. (Ambiguity, Top-down parsing, Left-recursion Removal)
Parsing Part II (Ambiguity, Top-down parsing, Left-recursion Removal) Ambiguous Grammars Definitions If a grammar has more than one leftmost derivation for a single sentential form, the grammar is ambiguous
More informationLanguage Processing note 12 CS
CS2 Language Processing note 12 Automatic generation of parsers In this note we describe how one may automatically generate a parse table from a given grammar (assuming the grammar is LL(1)). This and
More informationParsing Part II (Top-down parsing, left-recursion removal)
Parsing Part II (Top-down parsing, left-recursion removal) Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit
More informationBuilding a Parser III. CS164 3:30-5:00 TT 10 Evans. Prof. Bodik CS 164 Lecture 6 1
Building a Parser III CS164 3:30-5:00 TT 10 Evans 1 Overview Finish recursive descent parser when it breaks down and how to fix it eliminating left recursion reordering productions Predictive parsers (aka
More informationΕΠΛ323 - Θεωρία και Πρακτική Μεταγλωττιστών
ΕΠΛ323 - Θεωρία και Πρακτική Μεταγλωττιστών Lecture 5b Syntax Analysis Elias Athanasopoulos eliasathan@cs.ucy.ac.cy Regular Expressions vs Context-Free Grammars Grammar for the regular expression (a b)*abb
More informationLR Parsing Techniques
LR Parsing Techniques Introduction Bottom-Up Parsing LR Parsing as Handle Pruning Shift-Reduce Parser LR(k) Parsing Model Parsing Table Construction: SLR, LR, LALR 1 Bottom-UP Parsing A bottom-up parser
More informationArchitecture of Compilers, Interpreters. CMSC 330: Organization of Programming Languages. Front End Scanner and Parser. Implementing the Front End
Architecture of Compilers, Interpreters : Organization of Programming Languages ource Analyzer Optimizer Code Generator Context Free Grammars Intermediate Representation Front End Back End Compiler / Interpreter
More informationAutomatic generation of LL(1) parsers
Automatic generation of LL(1) parsers Informatics 2A: Lecture 12 John Longley School of Informatics University of Edinburgh jrl@staffmail.ed.ac.uk 14 October, 2011 1 / 12 Generating parse tables We ve
More informationSyntax Analysis Part I
Syntax Analysis Part I Chapter 4: Context-Free Grammars Slides adapted from : Robert van Engelen, Florida State University Position of a Parser in the Compiler Model Source Program Lexical Analyzer Token,
More information1 Introduction. 2 Recursive descent parsing. Predicative parsing. Computer Language Implementation Lecture Note 3 February 4, 2004
CMSC 51086 Winter 2004 Computer Language Implementation Lecture Note 3 February 4, 2004 Predicative parsing 1 Introduction This note continues the discussion of parsing based on context free languages.
More informationCS 406/534 Compiler Construction Parsing Part I
CS 406/534 Compiler Construction Parsing Part I Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof. Ken Kennedy and Dr.
More informationA programming language requires two major definitions A simple one pass compiler
A programming language requires two major definitions A simple one pass compiler [Syntax: what the language looks like A context-free grammar written in BNF (Backus-Naur Form) usually suffices. [Semantics:
More informationCMSC 330: Organization of Programming Languages. Context-Free Grammars Ambiguity
CMSC 330: Organization of Programming Languages Context-Free Grammars Ambiguity Review Why should we study CFGs? What are the four parts of a CFG? How do we tell if a string is accepted by a CFG? What
More informationTable-driven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and non-terminals.
Bottom-up Parsing: Table-driven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and non-terminals. Basic operation is to shift terminals from the input to the
More informationWhere We Are. CMSC 330: Organization of Programming Languages. This Lecture. Programming Languages. Motivation for Grammars
CMSC 330: Organization of Programming Languages Context Free Grammars Where We Are Programming languages Ruby OCaml Implementing programming languages Scanner Uses regular expressions Finite automata Parser
More informationBottom-Up Parsing. Lecture 11-12
Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 9/22/06 Prof. Hilfinger CS164 Lecture 11 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient
More informationPrinciples of Programming Languages COMP251: Syntax and Grammars
Principles of Programming Languages COMP251: Syntax and Grammars Prof. Dekai Wu Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China Fall 2006
More informationCS 536 Midterm Exam Spring 2013
CS 536 Midterm Exam Spring 2013 ID: Exam Instructions: Write your student ID (not your name) in the space provided at the top of each page of the exam. Write all your answers on the exam itself. Feel free
More informationCompilers. Predictive Parsing. Alex Aiken
Compilers Like recursive-descent but parser can predict which production to use By looking at the next fewtokens No backtracking Predictive parsers accept LL(k) grammars L means left-to-right scan of input
More informationContext-Free Languages & Grammars (CFLs & CFGs) Reading: Chapter 5
Context-Free Languages & Grammars (CFLs & CFGs) Reading: Chapter 5 1 Not all languages are regular So what happens to the languages which are not regular? Can we still come up with a language recognizer?
More informationCompiler Design Concepts. Syntax Analysis
Compiler Design Concepts Syntax Analysis Introduction First task is to break up the text into meaningful words called tokens. newval=oldval+12 id = id + num Token Stream Lexical Analysis Source Code (High
More informationParsing. Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1)
TD parsing - LL(1) Parsing First and Follow sets Parse table construction BU Parsing Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1) Problems with SLR Aho, Sethi, Ullman, Compilers
More informationCMSC 330: Organization of Programming Languages. Architecture of Compilers, Interpreters
: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Scanner Parser Static Analyzer Intermediate Representation Front End Back End Compiler / Interpreter
More informationParsing III. (Top-down parsing: recursive descent & LL(1) )
Parsing III (Top-down parsing: recursive descent & LL(1) ) Roadmap (Where are we?) Previously We set out to study parsing Specifying syntax Context-free grammars Ambiguity Top-down parsers Algorithm &
More information8 Parsing. Parsing. Top Down Parsing Methods. Parsing complexity. Top down vs. bottom up parsing. Top down vs. bottom up parsing
8 Parsing Parsing A grammar describes syntactically legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string
More informationCS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F).
CS 2210 Sample Midterm 1. Determine if each of the following claims is true (T) or false (F). F A language consists of a set of strings, its grammar structure, and a set of operations. (Note: a language
More informationLL(1) predictive parsing
LL(1) predictive parsing Informatics 2A: Lecture 11 John Longley School of Informatics University of Edinburgh jrl@staffmail.ed.ac.uk 13 October, 2011 1 / 12 1 LL(1) grammars and parse tables 2 3 2 / 12
More informationPESIT Bangalore South Campus Hosur road, 1km before Electronic City, Bengaluru -100 Department of Computer Science and Engineering
TEST 1 Date : 24 02 2015 Marks : 50 Subject & Code : Compiler Design ( 10CS63) Class : VI CSE A & B Name of faculty : Mrs. Shanthala P.T/ Mrs. Swati Gambhire Time : 8:30 10:00 AM SOLUTION MANUAL 1. a.
More informationCSE P 501 Compilers. LR Parsing Hal Perkins Spring UW CSE P 501 Spring 2018 D-1
CSE P 501 Compilers LR Parsing Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts UW CSE P 501 Spring 2018
More informationLL(k) Parsing. Predictive Parsers. LL(k) Parser Structure. Sample Parse Table. LL(1) Parsing Algorithm. Push RHS in Reverse Order 10/17/2012
Predictive Parsers LL(k) Parsing Can we avoid backtracking? es, if for a given input symbol and given nonterminal, we can choose the alternative appropriately. his is possible if the first terminal of
More informationIntroduction to parsers
Syntax Analysis Introduction to parsers Context-free grammars Push-down automata Top-down parsing LL grammars and parsers Bottom-up parsing LR grammars and parsers Bison/Yacc - parser generators Error
More informationLecture 7: Deterministic Bottom-Up Parsing
Lecture 7: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Tue Sep 20 12:50:42 2011 CS164: Lecture #7 1 Avoiding nondeterministic choice: LR We ve been looking at general
More informationSyntax Analysis. Amitabha Sanyal. (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay
Syntax Analysis (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay September 2007 College of Engineering, Pune Syntax Analysis: 2/124 Syntax
More informationCMSC 330: Organization of Programming Languages
CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler
More informationCMSC 330: Organization of Programming Languages
CMSC 330: Organization of Programming Languages Parsing CMSC 330 - Spring 2017 1 Recall: Front End Scanner and Parser Front End Token Source Scanner Parser Stream AST Scanner / lexer / tokenizer converts
More informationFall Compiler Principles Lecture 2: LL parsing. Roman Manevich Ben-Gurion University of the Negev
Fall 2017-2018 Compiler Principles Lecture 2: LL parsing Roman Manevich Ben-Gurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman
More informationCompilation Lecture 3: Syntax Analysis: Top-Down parsing. Noam Rinetzky
Compilation 0368-3133 Lecture 3: Syntax Analysis: Top-Down parsing Noam Rinetzky 1 Recursive descent parsing Define a function for every nonterminal Every function work as follows Find applicable production
More informationLL(1) predictive parsing
LL(1) predictive parsing Informatics 2A: Lecture 11 Mary Cryan School of Informatics University of Edinburgh mcryan@staffmail.ed.ac.uk 10 October 2018 1 / 15 Recap of Lecture 10 A pushdown automaton (PDA)
More informationLecture 8: Deterministic Bottom-Up Parsing
Lecture 8: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Fri Feb 12 13:02:57 2010 CS164: Lecture #8 1 Avoiding nondeterministic choice: LR We ve been looking at general
More informationChapter 4: LR Parsing
Chapter 4: LR Parsing 110 Some definitions Recall For a grammar G, with start symbol S, any string α such that S called a sentential form α is If α Vt, then α is called a sentence in L G Otherwise it is
More informationContext-Free Grammars and Languages (2015/11)
Chapter 5 Context-Free Grammars and Languages (2015/11) Adriatic Sea shore at Opatija, Croatia Outline 5.0 Introduction 5.1 Context-Free Grammars (CFG s) 5.2 Parse Trees 5.3 Applications of CFG s 5.4 Ambiguity
More informationCMSC 330: Organization of Programming Languages. Context Free Grammars
CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler
More informationMore Bottom-Up Parsing
More Bottom-Up Parsing Lecture 7 Dr. Sean Peisert ECS 142 Spring 2009 1 Status Project 1 Back By Wednesday (ish) savior lexer in ~cs142/s09/bin Project 2 Due Friday, Apr. 24, 11:55pm My office hours 3pm
More informationIntroduction to Parsing. Comp 412
COMP 412 FALL 2010 Introduction to Parsing Comp 412 Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit permission to make
More informationContext-free grammars
Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,
More information