CS 321 Programming Languages and Compilers. VI. Parsing


 Julius Holt
 2 years ago
 Views:
Transcription
1 CS 321 Programming Languages and Compilers VI. Parsing
2 Parsing Calculate grammatical structure of program, like diagramming sentences, where: Tokens = words Programs = sentences For further information, read: Aho, Sethi, Ullman, Compilers: Principles, Techniques, and Tools (a.k.a, the Dragon Book ) 2
3 Outline of coverage Contextfree grammars Parsing Tabular Parsing Methods One pass» Topdown Yacc» Bottomup 3
4 What parser does: Extracts grammatical structure of program functiondef name arguments stmtlist main stmt expression expression variable cout operator << expression string hello, world\n 4
5 Contextfree languages Grammatical structure defined by contextfree grammar. statement fi labeledstatement expressionstatement compoundstatement labeledstatement fi ident: statement case constantexpression : statement compoundstatement fi { declarationlist statementlist } Contextfree = only one nonterminal in leftpart. terminal nonterminal 5
6 Parse trees Parse tree = tree labeled with grammar symbols, such that: If node is labeled A, and its children are labeled x 1...x n, then there is a production A fi x 1...x n Parse tree from A = root labeled with A Complete parse tree = all leaves labeled with tokens 6
7 Parse trees and sentences Frontier of tree = labels on leaves (in lefttoright order) Frontier of tree from S is a sentential form. Frontier of a complete tree from S is a sentence. L L ; E E Frontier a 7
8 Example G: L fi L ; E E E fi a b Syntax trees from start symbol (L): L E a L E a L ; E L E a L L ; ; E b E b Sentential forms: a a;e a;b;b 8
9 Derivations Alternate definition of sentence: Given a, b in V*, say a b is a derivation step if a = a Aa and b = a ga, where A fi g is a production g is a sentential form iff there exists a derivation (sequence of derivation steps) S... g ( alternatively, we say that S * g ) Two definitions are equivalent, but note that there are many derivations corresponding to each parse tree. 9
10 Another example H: L fi E ; L E E fi a b L E a E L ; L E E b L ; E b L ; L E a a 10
11 Ambiguity For some purposes, it is important to know whether a sentence can have more than one parse tree. A grammar is ambiguous if there is a sentence with more than one parse tree. Example: E fi E+E E*E id E E E + E E * E id E * E E + E id id id id id 11
12 Ambiguity Ambiguity is a function of the grammar rather than the language. Certain unambiguous grammars may have equivalent ambiguous ones. 12
13 Grammar Transformations Grammars can be transformed without affecting the language generated. Three transformations are discussed next: Eliminating Ambiguity Eliminating Left Recursion (i.e.productions of the form AfiA ) Left Factoring 13
14 Grammar Transformation 1. Eliminating Ambiguity Sometimes an ambiguous grammar can be rewritten to eliminate ambiguity. For example, expressions involving additions and products can be written as follows: E fi E+T T T fi T*id id The language generated by this grammar is the same as that generated by the grammar on tranparency 11. Both generate id(+id *id)* However, this grammar is not ambiguous. 14
15 Grammar Transformation 1. Eliminating Ambiguity (Cont.) One advantage of this grammar is that it represents the precedence between operators. In the parsing tree, products appear nested within additions E E + T T id T id * id 15
16 Grammar Transformation 1. Eliminating Ambiguity (Cont.) The most famous example of ambiguity in a programming language is the dangling else. Consider S fi if bthen Selse S if b then S a 16
17 Grammar Transformation 1. Eliminating Ambiguity (Cont.) When there are two nested ifs and only one else.. S if β then S else S if βthen S α S α if β then S if β then S else S α α 17
18 Grammar Transformation 1. Eliminating Ambiguity (Cont.) In most languages (including C++ and Java), each else is assumed to belong to the nearest if that is not already matched by an else. This association is expressed in the following (unambiguous) grammar: S fi Matched Unmatched Matched fi if b then Matched else Matched a Unmatched fi if b then S if b then Matched else Unmatched 18
19 Grammar Transformation 1. Eliminating Ambiguity (Cont.) Ambiguity is a function of the grammar It is undecidable whether a context free grammar is ambiguous. The proof is done by reduction to Post s correspondence problem. Although there is no general algorithm, it is possible to isolate certain constructs in productions which lead to ambiguous grammars. 19
20 Grammar Transformation 1. Eliminating Ambiguity (Cont.) For example, a grammar containg the production AfiAA would be ambiguous, because the substring aaa has two parses. A A A A A A A A α α A A α α This ambiguity disappears if we use the productions AfiAB B and Bfi or the productions AfiBA B and Bfi. α α 20
21 Grammar Transformation 1. Eliminating Ambiguity (Cont.) Other three examples of ambiguous productions are: AfiAaA AfiaA Ab and AfiaA aaba A language generated by an ambiguous Context Free Grammar is inherently ambiguous if it has no unambiguous Context Free Grammar. (This can be proven formally) An example of such a language is L={a i b j c m i=j or j=m} which can be generated by the grammar: SfiAB DC AfiaA e BfibBc e CficC e DfiaDb e 21
22 Grammar Transformations 2. Elimination of Left Recursion A grammar is left recursive if it has a nonterminal A and a derivation A + Aa for some string a. Topdown parsing methods (to be discussed shortly) cannot handle leftrecursive grammars, so a transformation to eliminate left recursion is needed. Immediate left recursion (productions of the form AfiA ) can be easily eliminated. We group the Aproductions as AfiA 1 A 2 A m b 1 b 2 b n where no b i begins with A. Then we replace the A productions by Afi b 1 A b 2 A b n A A fi 1 A 2 A m A e 22
23 Grammar Transformations 2. Elimination of Left Recursion (Cont.) The previous transformation, however, does not eliminate left recursion involving two or more steps. For example, consider the grammar SfiAa b AfiAc Sd e S is leftrecursive because S Aa Sda, but it is not immediately left recursive. 23
24 Grammar Transformations 2. Elimination of Left Recursion (Cont.) Algorithm. Eliminate left recursion Arrange nonterminals in some order A 1, A 2,,, A n for i =1 to n { } for j =1 to i 1 { replace each production of the form A i A j γ by the production A i δ 1 γ δ 2 γ δ n γ where A j δ 1 δ 2 δ n are all the current A j productions } eliminate the immediate left recursion among the A i productions 24
25 Grammar Transformations 2. Elimination of Left Recursion (Cont.) To show that the previous algorithm actually works all we need notice is that iteration i only changes productions with A i on the lefthand side. And m > i in all productions of the form A i fia m. This can be easily shown by induction. It is clearly true for i=1. If it is true for all i<k, then when the outer loop is executed for i=k, the inner loop will remove all productions A i fia m with m < i. Finally, with the elimination of self recursion, m in the A i fia m productions is forced to be > i. So, at the end of the algorithm, all derivations of the form A + i A m a will have m > i and therefore left recursion would not be possible. 25
26 Grammar Transformations 3. Left Factoring Left factoring helps transform a grammar for predictive parsing For example, if we have the two productions S fi if bthen Selse S if bthen S on seeing the input token if, we cannot immediately tell which production to choose to expand S. In general, if we have Afi b 1 b 2 and the input begins with a, we do not know (without looking further) which production to use to expand A. 26
27 Grammar Transformations 3. Left Factoring(Cont.) However, we may defer the decision by expanding A to aa. Then after seeing the input derived from a, we may expand A to b 1 or to b 2. That is, leftfactored, the original productions become Afi A A fi b 1 b 2 27
28 NonContextFree Language Constructs Examples of noncontextfree languages are: L 1 ={wcw w is of the form (a b)*} L 2 ={a n b m c n d m n 1 and m 1 } L 3 ={a n b n c n n 0 } Languages similar to these that are context free L 1 ={wcw R w is of the form (a b)*} (w R stands for w reversed) This language is generated by the grammar» Sfi asa bsb c L 2 ={a n b m c m d n n 1 and m 1 } This language is generated by the grammar» Sfi asd aad» Afi bac bc 28
29 NonContextFree Language Constructs (Cont.) L 2 ={a n b n c m d m n 1 and m 1 } This language is generated by the grammar» Sfi AB» Afi aab ab» Bfi cbd cd L 3 ={a n b n n 1} This language is generated by the grammar» Sfi asb ab This language is not definable by any regular expression 29
30 NonContextFree Language Constructs (Cont.) Suppose we could construct a DFSM D accepting L 3. D must have a finite number of states, say k. Consider the sequence of states s 0, s 1, s 2,, s k entered by D having read e, a, aa,, a k. Since D only has k states, two of the states in the sequence have to be equal. Say, s i s j (i j). From s i, a sequence of i bs leads to an accepting (final) state. Therefore, the same sequence of i bs will also lead to an accepting state from s j. Therefore D would accept a j b i which means that the language accepted by D is not identical to L 3. A contradiction. 30
31 Parsing The parsing problem is: Given string of tokens w, find a parse tree whose frontier is w. (Equivalently, find a derivation from w.) A parser for a grammar G reads a list of tokens and finds a parse tree if they form a sentence (or reports an error otherwise) Two classes of algorithms for parsing: Topdown Bottomup 31
32 Parser generators A parser generator is a program that reads a grammar and produces a parser. The best known parser generator is yacc. Both produce bottomup parsers. Most parser generators  including yacc  do not work for every cfg; they accept a restricted class of cfg s that can be parsed efficiently using the method employed by that parser generator. 32
33 Topdown parsing Starting from parse tree containing just S, build tree down toward input. Expand leftmost nonterminal. Algorithm: (next slide) 33
34 Topdown parsing (cont.) Let input = a 1 a 2...a n current sentential form (csf) = S loop { suppose csf = t 1...t k Aa if t 1...t k a 1...a k, it s an error based on a k+1..., choose production A fib csf becomes t 1...t k ba } 34
35 Topdown parsing example Grammar: H: L fi E ; L E E fi a b Input: a;b Parse tree Sentential form Input E E a L L a;b L E;L a;b ; L L a;l a;b ; L 35
36 Topdown parsing example (cont.) Parse tree Sentential form Input L E ; L a E L E ; L a E b a;e a;b a;b a;b 36
37 LL(1) parsing Efficient form of topdown parsing. Use only first symbol of remaining input (a k+1 ) to choose next production. That is, employ a function M:S Nfi P in choose production step of algorithm. When this works, grammar is (usually) called LL(1). (More precise definition to follow.) 37
38 LL(1) examples Example 1: H: L fi E ; L E E fi a b Given input a;b, so next symbol is a. Which production to use? Can t tell. \ H not LL(1). 38
39 LL(1) examples Example 2: Exp fi Term Exp Exp fi $ + Exp Term fi id (Use $ for endofinput symbol.) Grammar is LL(1): Exp and Term have only one production; Exp has two productions but only one is applicable at any time. 39
40 Nonrecursive predictive parsing It is possible to build a nonrecursive predictive parser by maintaining as stack explicitly, rather tan implicitly via recursive calls. The key problem during predictive parsing is that of determining the production to be applied for a nonterminal. 40
41 Nonrecursive predictive parsing Algorithm. Nonrecursive predictive parsing Set ip to point to the first symbol of w$. repeat Let X be the top of the stack symbol and a the symbol pointed to by ip if X is a terminal or $ then if X == a then else else error() // X is a nonterminal pop X from the stack and advance ip if M[X,a] == X Y 1 Y 2 Y k then pop X from the stack push Y k Y k1,, Y 1 onto the stack with Y 1 on top (push nothing if Y 1 Y 2 Y k is ε ) until X == $ output the production X Y 1 Y 2 Y k else error() 41
42 LL(1) grammars No left recursion. A fi Aa : If this production is chosen, parse makes no progress. No common prefixes. A fi ab ag Can fix by left factoring : A fi aa A fi b g 42
43 LL(1) grammars (cont.) No ambiguity. Precise definition requires that production to choose be unique ( choose function M very hard to calculate otherwise). 43
44 Topdown Parsing Input tokens: <t0,t1,,ti,...> L E0 En Start symbol and root of parse tree Input tokens: <ti,...> From left to right, grow the parse tree downwards L E0 En... 44
45 Checking LL(1)ness For any sequence of grammar symbols a, define set FIRST(a) S to be those tokens a such that a ab for some b. (Notation: write a *ab.) 45
46 Checking LL(1)ness Define: Grammar G = (N,, P, S) is LL(1) if whenever there are two leftmost derivations (in which the leftmost nonterminal is always expanded first ) S =>* waa => wba =>* wx S =>* waa => wga =>* wy Such that FIRST(x) = FIRST(y), it follows that b =g. In other words, given 1. A string waa in V* and 2. The first terminal symbol to be derived from Aa, say t There is at most one production that can be applied to A to yield a derivation of any terminal string beginning with wt. FIRST sets can often be calculated by inspection. 46
47 FIRST Sets Exp Term Exp Exp $ + Exp Term id (Use $ for endofinput symbol.) FIRST(Term Exp ) = {id} FIRST($) = {$}, FIRST(+ Exp) = {+} implies FIRST($) FIRST(+ Exp) = {} FIRST(id) = {id} grammar is LL(1) 47
48 FIRST Sets H: L E ; L E E a b FIRST(E ; L) = {a,b} = FIRST(E) FIRST(E ; L) FIRST(E) {} H not LL(1). 48
49 How to compute FIRST Sets of Vocabulary Symbols Algorithm. Compute FIRST(X) for all grammar symbols X forall X V do FIRST(X)={} forall X Σ (X is a terminal) do FIRST(X)={X} forall productions X ε do FIRST(X) = FIRST(X) U {ε} repeat forall productions X Y 1 Y 2 Y k do forall i [1,k] do FIRST(X) = FIRST(X) U (FIRST(Y i )  {ε}) if ε v FIRST(Y i ) then continue outer loop FIRST(X) = FIRST(X) U {ε} until no more terminals or ε are added to any FIRST set 49
50 How to compute FIRST Sets of Strings of Symbols FIRST(X 1 X 2 X n ) is the union of FIRST(X 1 ) and all FIRST(X i ) such that ε FIRST(X k ) for k=1,2,..,i1 FIRST(X 1 X 2 X n ) contains ε iff ε FIRST(X k ) for k=1,2,..,n. 50
51 FIRST Sets do not Suffice Given the productions Afi T x Afi T y Tfi w Tfi e Tfi w should be applied when the next input token is w. Tfi e should be applied whenever the next terminal (the one pointed to by ip) is either x or y 51
52 FOLLOW Sets For any nonterminal X, define set FOLLOW(X) S to be those tokens a such that S *axab for some a and b. 52
53 How to compute the FOLLOW Set Algorithm. Compute FOLLOW(X) for all nonterminals X FOLLOW(S) ={$} forall productions A αbβ do FOLLOW(B)=Follow(B) U (FIRST(β)  {ε}) repeat forall productions A αb or A αbβ with ε FIRST(β) do FOLLOW(B) = FOLLOW(B) U FOLLOW(A) until all FOLLOW sets remain the same 53
54 Construction of a predictive parsing table Algorithm. Construction of a predictive parsing table M[:,:] = {} forall productions A α do forall a FIRST(α) do M[A,a] = M[A,a] U {A α } if ε FIRST(α) then forall b FOLLOW(A) do M[A,b] = M[A,b] U {A α } Make all empty entries of M be error 54
55 Another Definition of LL(1) Define: Grammar G is LL(1) if for every A N with productions A fi a 1... a n, FIRST(a i FOLLOW(A)) FIRST(a j FOLLOW(A) ) = for all i, j; 55
Syntax Analysis. Martin Sulzmann. Martin Sulzmann Syntax Analysis 1 / 38
Syntax Analysis Martin Sulzmann Martin Sulzmann Syntax Analysis 1 / 38 Syntax Analysis Objective Recognize individual tokens as sentences of a language (beyond regular languages). Example 1 (OK) Program
More informationTypes of parsing. CMSC 430 Lecture 4, Page 1
Types of parsing Topdown parsers start at the root of derivation tree and fill in picks a production and tries to match the input may require backtracking some grammars are backtrackfree (predictive)
More informationNote that for recursive descent to work, if A ::= B1 B2 is a grammar rule we need First k (B1) disjoint from First k (B2).
LL(k) Grammars We need a bunch of terminology. For any terminal string a we write First k (a) is the prefix of a of length k (or all of a if its length is less than k) For any string g of terminal and
More informationParsing. Roadmap. > Contextfree grammars > Derivations and precedence > Topdown parsing > Leftrecursion > Lookahead > Tabledriven parsing
Roadmap > Contextfree grammars > Derivations and precedence > Topdown parsing > Leftrecursion > Lookahead > Tabledriven parsing The role of the parser > performs contextfree syntax analysis > guides
More informationTableDriven Parsing
TableDriven Parsing It is possible to build a nonrecursive predictive parser by maintaining a stack explicitly, rather than implicitly via recursive calls [1] The nonrecursive parser looks up the production
More informationCS2210: Compiler Construction Syntax Analysis Syntax Analysis
Comparison with Lexical Analysis The second phase of compilation Phase Input Output Lexer string of characters string of tokens Parser string of tokens Parse tree/ast What Parse Tree? CS2210: Compiler
More informationCompilerconstructie. najaar Rudy van Vliet kamer 140 Snellius, tel rvvliet(at)liacs(dot)nl. college 3, vrijdag 22 september 2017
Compilerconstructie najaar 2017 http://www.liacs.leidenuniv.nl/~vlietrvan1/coco/ Rudy van Vliet kamer 140 Snellius, tel. 071527 2876 rvvliet(at)liacs(dot)nl college 3, vrijdag 22 september 2017 + werkcollege
More information3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino
3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the
More informationCA Compiler Construction
CA4003  Compiler Construction David Sinclair A topdown parser starts with the root of the parse tree, labelled with the goal symbol of the grammar, and repeats the following steps until the fringe of
More information3. Parsing. Oscar Nierstrasz
3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/
More informationSYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic
More informationAmbiguity, Precedence, Associativity & TopDown Parsing. Lecture 910
Ambiguity, Precedence, Associativity & TopDown Parsing Lecture 910 (From slides by G. Necula & R. Bodik) 9/18/06 Prof. Hilfinger CS164 Lecture 9 1 Administrivia Please let me know if there are continued
More informationSyntactic Analysis. TopDown Parsing
Syntactic Analysis TopDown Parsing Copyright 2017, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make
More informationTopDown Parsing and Intro to BottomUp Parsing. Lecture 7
TopDown Parsing and Intro to BottomUp Parsing Lecture 7 1 Predictive Parsers Like recursivedescent but parser can predict which production to use Predictive parsers are never wrong Always able to guess
More informationTop down vs. bottom up parsing
Parsing A grammar describes the strings that are syntactically legal A recogniser simply accepts or rejects strings A generator produces sentences in the language described by the grammar A parser constructs
More informationTopDown Parsing and Intro to BottomUp Parsing. Lecture 7
TopDown Parsing and Intro to BottomUp Parsing Lecture 7 1 Predictive Parsers Like recursivedescent but parser can predict which production to use Predictive parsers are never wrong Always able to guess
More informationAmbiguity. Grammar E E + E E * E ( E ) int. The string int * int + int has two parse trees. * int
Administrivia Ambiguity, Precedence, Associativity & opdown Parsing eam assignments this evening for all those not listed as having one. HW#3 is now available, due next uesday morning (Monday is a holiday).
More informationMonday, September 13, Parsers
Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of nonterminals S is the start symbol P is the set of productions
More informationCompiler Design 1. TopDown Parsing. Goutam Biswas. Lect 5
Compiler Design 1 TopDown Parsing Compiler Design 2 Nonterminal as a Function In a topdown parser a nonterminal may be viewed as a generator of a substring of the input. We may view a nonterminal
More informationDEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
KATHMANDU UNIVERSITY SCHOOL OF ENGINEERING DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING REPORT ON NONRECURSIVE PREDICTIVE PARSER Fourth Year First Semester Compiler Design Project Final Report submitted
More informationCS1622. Today. A Recursive Descent Parser. Preliminaries. Lecture 9 Parsing (4)
CS1622 Lecture 9 Parsing (4) CS 1622 Lecture 9 1 Today Example of a recursive descent parser Predictive & LL(1) parsers Building parse tables CS 1622 Lecture 9 2 A Recursive Descent Parser. Preliminaries
More informationIntroduction to Syntax Analysis. The Second Phase of FrontEnd
Compiler Design IIIT Kalyani, WB 1 Introduction to Syntax Analysis The Second Phase of FrontEnd Compiler Design IIIT Kalyani, WB 2 Syntax Analysis The syntactic or the structural correctness of a program
More informationCS502: Compilers & Programming Systems
CS502: Compilers & Programming Systems Topdown Parsing Zhiyuan Li Department of Computer Science Purdue University, USA There exist two wellknown schemes to construct deterministic topdown parsers:
More informationParsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468
Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure
More informationIntroduction to Syntax Analysis
Compiler Design 1 Introduction to Syntax Analysis Compiler Design 2 Syntax Analysis The syntactic or the structural correctness of a program is checked during the syntax analysis phase of compilation.
More informationCompilers: CS31003 Computer Sc & Engg: IIT Kharagpur 1. TopDown Parsing. Lect 5. Goutam Biswas
Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 1 TopDown Parsing Compilers: CS31003 Computer Sc & Engg: IIT Kharagpur 2 Nonterminal as a Function In a topdown parser a nonterminal may be viewed
More informationWednesday, August 31, Parsers
Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically
More informationWednesday, September 9, 15. Parsers
Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda
More informationParsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:
What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda
More informationCS 314 Principles of Programming Languages
CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution
More informationCompilers. Yannis Smaragdakis, U. Athens (original slides by Sam
Compilers Parsing Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Next step text chars Lexical analyzer tokens Parser IR Errors Parsing: Organize tokens into sentences Do tokens conform
More informationCS308 Compiler Principles Syntax Analyzer Li Jiang
CS308 Syntax Analyzer Li Jiang Department of Computer Science and Engineering Shanghai Jiao Tong University Syntax Analyzer Syntax Analyzer creates the syntactic structure of the given source program.
More informationSection A. A grammar that produces more than one parse tree for some sentences is said to be ambiguous.
Section A 1. What do you meant by parser and its types? A parser for grammar G is a program that takes as input a string w and produces as output either a parse tree for w, if w is a sentence of G, or
More informationPart 3. Syntax analysis. Syntax analysis 96
Part 3 Syntax analysis Syntax analysis 96 Outline 1. Introduction 2. Contextfree grammar 3. Topdown parsing 4. Bottomup parsing 5. Conclusion and some practical considerations Syntax analysis 97 Structure
More informationConcepts Introduced in Chapter 4
Concepts Introduced in Chapter 4 Grammars ContextFree Grammars Derivations and Parse Trees Ambiguity, Precedence, and Associativity Top Down Parsing Recursive Descent, LL Bottom Up Parsing SLR, LR, LALR
More informationFormal Languages and Compilers Lecture V: Parse Trees and Ambiguous Gr
Formal Languages and Compilers Lecture V: Parse Trees and Ambiguous Grammars Free University of BozenBolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/
More informationParsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones
Parsing III (Topdown parsing: recursive descent & LL(1) ) (Bottomup parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,
More informationBottom up parsing. The sentential forms happen to be a right most derivation in the reverse order. S a A B e a A d e. a A d e a A B e S.
Bottom up parsing Construct a parse tree for an input string beginning at leaves and going towards root OR Reduce a string w of input to start symbol of grammar Consider a grammar S aabe A Abc b B d And
More informationVIVA QUESTIONS WITH ANSWERS
VIVA QUESTIONS WITH ANSWERS 1. What is a compiler? A compiler is a program that reads a program written in one language the source language and translates it into an equivalent program in another languagethe
More informationLexical and Syntax Analysis. TopDown Parsing
Lexical and Syntax Analysis TopDown Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax
More informationParser Generation. BottomUp Parsing. Constructing LR Parser. LR Parsing. Construct parse tree bottomup  from leaves to the root
Parser Generation Main Problem: given a grammar G, how to build a topdown parser or a bottomup parser for it? parser : a program that, given a sentence, reconstructs a derivation for that sentence 
More informationAdministrativia. WA1 due on Thu PA2 in a week. Building a Parser III. Slides on the web site. CS164 3:305:00 TT 10 Evans.
Administrativia Building a Parser III CS164 3:305:00 10 vans WA1 due on hu PA2 in a week Slides on the web site I do my best to have slides ready and posted by the end of the preceding logical day yesterday,
More informationAbstract Syntax Trees & TopDown Parsing
Review of Parsing Abstract Syntax Trees & TopDown Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationPART 3  SYNTAX ANALYSIS. F. Wotawa TU Graz) Compiler Construction Summer term / 309
PART 3  SYNTAX ANALYSIS F. Wotawa (IST @ TU Graz) Compiler Construction Summer term 2016 64 / 309 Goals Definition of the syntax of a programming language using context free grammars Methods for parsing
More informationSyntax Analysis Check syntax and construct abstract syntax tree
Syntax Analysis Check syntax and construct abstract syntax tree if == = ; b 0 a b Error reporting and recovery Model using context free grammars Recognize using Push down automata/table Driven Parsers
More informationLexical and Syntax Analysis
Lexical and Syntax Analysis (of Programming Languages) TopDown Parsing Lexical and Syntax Analysis (of Programming Languages) TopDown Parsing Easy for humans to write and understand String of characters
More informationThe analysis part breaks up the source program into constituent pieces and creates an intermediate representation of the source program.
COMPILER DESIGN 1. What is a compiler? A compiler is a program that reads a program written in one language the source language and translates it into an equivalent program in another languagethe target
More informationContextFree Grammars
ContextFree Grammars 1 Informal Comments A contextfree grammar is a notation for describing languages. It is more powerful than finite automata or RE s, but still cannot define all possible languages.
More informationSyntax Analysis. Prof. James L. Frankel Harvard University. Version of 6:43 PM 6Feb2018 Copyright 2018, 2015 James L. Frankel. All rights reserved.
Syntax Analysis Prof. James L. Frankel Harvard University Version of 6:43 PM 6Feb2018 Copyright 2018, 2015 James L. Frankel. All rights reserved. ContextFree Grammar (CFG) terminals nonterminals start
More informationAbstract Syntax Trees & TopDown Parsing
Abstract Syntax Trees & TopDown Parsing Review of Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationAbstract Syntax Trees & TopDown Parsing
Review of Parsing Abstract Syntax Trees & TopDown Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationCompilers Course Lecture 4: Context Free Grammars
Compilers Course Lecture 4: Context Free Grammars Example: attempt to define simple arithmetic expressions using named regular expressions: num = [09]+ sum = expr "+" expr expr = "(" sum ")" num Appears
More informationCSCI312 Principles of Programming Languages
Copyright 2006 The McGrawHill Companies, Inc. CSCI312 Principles of Programming Languages! LL Parsing!! Xu Liu Derived from Keith Cooper s COMP 412 at Rice University Recap Copyright 2006 The McGrawHill
More informationCS415 Compilers. Syntax Analysis. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University
CS415 Compilers Syntax Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Limits of Regular Languages Advantages of Regular Expressions
More informationCMSC 330: Organization of Programming Languages
CMSC 330: Organization of Programming Languages Context Free Grammars and Parsing 1 Recall: Architecture of Compilers, Interpreters Source Parser Static Analyzer Intermediate Representation Front End Back
More informationParsing Part II. (Ambiguity, Topdown parsing, Leftrecursion Removal)
Parsing Part II (Ambiguity, Topdown parsing, Leftrecursion Removal) Ambiguous Grammars Definitions If a grammar has more than one leftmost derivation for a single sentential form, the grammar is ambiguous
More informationLanguage Processing note 12 CS
CS2 Language Processing note 12 Automatic generation of parsers In this note we describe how one may automatically generate a parse table from a given grammar (assuming the grammar is LL(1)). This and
More informationParsing Part II (Topdown parsing, leftrecursion removal)
Parsing Part II (Topdown parsing, leftrecursion removal) Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit
More informationBuilding a Parser III. CS164 3:305:00 TT 10 Evans. Prof. Bodik CS 164 Lecture 6 1
Building a Parser III CS164 3:305:00 TT 10 Evans 1 Overview Finish recursive descent parser when it breaks down and how to fix it eliminating left recursion reordering productions Predictive parsers (aka
More informationΕΠΛ323  Θεωρία και Πρακτική Μεταγλωττιστών
ΕΠΛ323  Θεωρία και Πρακτική Μεταγλωττιστών Lecture 5b Syntax Analysis Elias Athanasopoulos eliasathan@cs.ucy.ac.cy Regular Expressions vs ContextFree Grammars Grammar for the regular expression (a b)*abb
More informationLR Parsing Techniques
LR Parsing Techniques Introduction BottomUp Parsing LR Parsing as Handle Pruning ShiftReduce Parser LR(k) Parsing Model Parsing Table Construction: SLR, LR, LALR 1 BottomUP Parsing A bottomup parser
More informationArchitecture of Compilers, Interpreters. CMSC 330: Organization of Programming Languages. Front End Scanner and Parser. Implementing the Front End
Architecture of Compilers, Interpreters : Organization of Programming Languages ource Analyzer Optimizer Code Generator Context Free Grammars Intermediate Representation Front End Back End Compiler / Interpreter
More informationAutomatic generation of LL(1) parsers
Automatic generation of LL(1) parsers Informatics 2A: Lecture 12 John Longley School of Informatics University of Edinburgh jrl@staffmail.ed.ac.uk 14 October, 2011 1 / 12 Generating parse tables We ve
More informationSyntax Analysis Part I
Syntax Analysis Part I Chapter 4: ContextFree Grammars Slides adapted from : Robert van Engelen, Florida State University Position of a Parser in the Compiler Model Source Program Lexical Analyzer Token,
More information1 Introduction. 2 Recursive descent parsing. Predicative parsing. Computer Language Implementation Lecture Note 3 February 4, 2004
CMSC 51086 Winter 2004 Computer Language Implementation Lecture Note 3 February 4, 2004 Predicative parsing 1 Introduction This note continues the discussion of parsing based on context free languages.
More informationCS 406/534 Compiler Construction Parsing Part I
CS 406/534 Compiler Construction Parsing Part I Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof. Ken Kennedy and Dr.
More informationA programming language requires two major definitions A simple one pass compiler
A programming language requires two major definitions A simple one pass compiler [Syntax: what the language looks like A contextfree grammar written in BNF (BackusNaur Form) usually suffices. [Semantics:
More informationCMSC 330: Organization of Programming Languages. ContextFree Grammars Ambiguity
CMSC 330: Organization of Programming Languages ContextFree Grammars Ambiguity Review Why should we study CFGs? What are the four parts of a CFG? How do we tell if a string is accepted by a CFG? What
More informationTabledriven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and nonterminals.
Bottomup Parsing: Tabledriven using an explicit stack (no recursion!). Stack can be viewed as containing both terminals and nonterminals. Basic operation is to shift terminals from the input to the
More informationWhere We Are. CMSC 330: Organization of Programming Languages. This Lecture. Programming Languages. Motivation for Grammars
CMSC 330: Organization of Programming Languages Context Free Grammars Where We Are Programming languages Ruby OCaml Implementing programming languages Scanner Uses regular expressions Finite automata Parser
More informationBottomUp Parsing. Lecture 1112
BottomUp Parsing Lecture 1112 (From slides by G. Necula & R. Bodik) 9/22/06 Prof. Hilfinger CS164 Lecture 11 1 BottomUp Parsing Bottomup parsing is more general than topdown parsing And just as efficient
More informationPrinciples of Programming Languages COMP251: Syntax and Grammars
Principles of Programming Languages COMP251: Syntax and Grammars Prof. Dekai Wu Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China Fall 2006
More informationCS 536 Midterm Exam Spring 2013
CS 536 Midterm Exam Spring 2013 ID: Exam Instructions: Write your student ID (not your name) in the space provided at the top of each page of the exam. Write all your answers on the exam itself. Feel free
More informationCompilers. Predictive Parsing. Alex Aiken
Compilers Like recursivedescent but parser can predict which production to use By looking at the next fewtokens No backtracking Predictive parsers accept LL(k) grammars L means lefttoright scan of input
More informationContextFree Languages & Grammars (CFLs & CFGs) Reading: Chapter 5
ContextFree Languages & Grammars (CFLs & CFGs) Reading: Chapter 5 1 Not all languages are regular So what happens to the languages which are not regular? Can we still come up with a language recognizer?
More informationCompiler Design Concepts. Syntax Analysis
Compiler Design Concepts Syntax Analysis Introduction First task is to break up the text into meaningful words called tokens. newval=oldval+12 id = id + num Token Stream Lexical Analysis Source Code (High
More informationParsing. Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1)
TD parsing  LL(1) Parsing First and Follow sets Parse table construction BU Parsing Handle, viable prefix, items, closures, goto s LR(k): SLR(1), LR(1), LALR(1) Problems with SLR Aho, Sethi, Ullman, Compilers
More informationCMSC 330: Organization of Programming Languages. Architecture of Compilers, Interpreters
: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Scanner Parser Static Analyzer Intermediate Representation Front End Back End Compiler / Interpreter
More informationParsing III. (Topdown parsing: recursive descent & LL(1) )
Parsing III (Topdown parsing: recursive descent & LL(1) ) Roadmap (Where are we?) Previously We set out to study parsing Specifying syntax Contextfree grammars Ambiguity Topdown parsers Algorithm &
More information8 Parsing. Parsing. Top Down Parsing Methods. Parsing complexity. Top down vs. bottom up parsing. Top down vs. bottom up parsing
8 Parsing Parsing A grammar describes syntactically legal strings in a language A recogniser simply accepts or rejects strings A generator produces strings A parser constructs a parse tree for a string
More informationCS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F).
CS 2210 Sample Midterm 1. Determine if each of the following claims is true (T) or false (F). F A language consists of a set of strings, its grammar structure, and a set of operations. (Note: a language
More informationLL(1) predictive parsing
LL(1) predictive parsing Informatics 2A: Lecture 11 John Longley School of Informatics University of Edinburgh jrl@staffmail.ed.ac.uk 13 October, 2011 1 / 12 1 LL(1) grammars and parse tables 2 3 2 / 12
More informationPESIT Bangalore South Campus Hosur road, 1km before Electronic City, Bengaluru 100 Department of Computer Science and Engineering
TEST 1 Date : 24 02 2015 Marks : 50 Subject & Code : Compiler Design ( 10CS63) Class : VI CSE A & B Name of faculty : Mrs. Shanthala P.T/ Mrs. Swati Gambhire Time : 8:30 10:00 AM SOLUTION MANUAL 1. a.
More informationCSE P 501 Compilers. LR Parsing Hal Perkins Spring UW CSE P 501 Spring 2018 D1
CSE P 501 Compilers LR Parsing Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 D1 Agenda LR Parsing Tabledriven Parsers Parser States ShiftReduce and ReduceReduce conflicts UW CSE P 501 Spring 2018
More informationLL(k) Parsing. Predictive Parsers. LL(k) Parser Structure. Sample Parse Table. LL(1) Parsing Algorithm. Push RHS in Reverse Order 10/17/2012
Predictive Parsers LL(k) Parsing Can we avoid backtracking? es, if for a given input symbol and given nonterminal, we can choose the alternative appropriately. his is possible if the first terminal of
More informationIntroduction to parsers
Syntax Analysis Introduction to parsers Contextfree grammars Pushdown automata Topdown parsing LL grammars and parsers Bottomup parsing LR grammars and parsers Bison/Yacc  parser generators Error
More informationLecture 7: Deterministic BottomUp Parsing
Lecture 7: Deterministic BottomUp Parsing (From slides by G. Necula & R. Bodik) Last modified: Tue Sep 20 12:50:42 2011 CS164: Lecture #7 1 Avoiding nondeterministic choice: LR We ve been looking at general
More informationSyntax Analysis. Amitabha Sanyal. (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay
Syntax Analysis (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay September 2007 College of Engineering, Pune Syntax Analysis: 2/124 Syntax
More informationCMSC 330: Organization of Programming Languages
CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler
More informationCMSC 330: Organization of Programming Languages
CMSC 330: Organization of Programming Languages Parsing CMSC 330  Spring 2017 1 Recall: Front End Scanner and Parser Front End Token Source Scanner Parser Stream AST Scanner / lexer / tokenizer converts
More informationFall Compiler Principles Lecture 2: LL parsing. Roman Manevich BenGurion University of the Negev
Fall 20172018 Compiler Principles Lecture 2: LL parsing Roman Manevich BenGurion University of the Negev 1 Books Compilers Principles, Techniques, and Tools Alfred V. Aho, Ravi Sethi, Jeffrey D. Ullman
More informationCompilation Lecture 3: Syntax Analysis: TopDown parsing. Noam Rinetzky
Compilation 03683133 Lecture 3: Syntax Analysis: TopDown parsing Noam Rinetzky 1 Recursive descent parsing Define a function for every nonterminal Every function work as follows Find applicable production
More informationLL(1) predictive parsing
LL(1) predictive parsing Informatics 2A: Lecture 11 Mary Cryan School of Informatics University of Edinburgh mcryan@staffmail.ed.ac.uk 10 October 2018 1 / 15 Recap of Lecture 10 A pushdown automaton (PDA)
More informationLecture 8: Deterministic BottomUp Parsing
Lecture 8: Deterministic BottomUp Parsing (From slides by G. Necula & R. Bodik) Last modified: Fri Feb 12 13:02:57 2010 CS164: Lecture #8 1 Avoiding nondeterministic choice: LR We ve been looking at general
More informationChapter 4: LR Parsing
Chapter 4: LR Parsing 110 Some definitions Recall For a grammar G, with start symbol S, any string α such that S called a sentential form α is If α Vt, then α is called a sentence in L G Otherwise it is
More informationContextFree Grammars and Languages (2015/11)
Chapter 5 ContextFree Grammars and Languages (2015/11) Adriatic Sea shore at Opatija, Croatia Outline 5.0 Introduction 5.1 ContextFree Grammars (CFG s) 5.2 Parse Trees 5.3 Applications of CFG s 5.4 Ambiguity
More informationCMSC 330: Organization of Programming Languages. Context Free Grammars
CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler
More informationMore BottomUp Parsing
More BottomUp Parsing Lecture 7 Dr. Sean Peisert ECS 142 Spring 2009 1 Status Project 1 Back By Wednesday (ish) savior lexer in ~cs142/s09/bin Project 2 Due Friday, Apr. 24, 11:55pm My office hours 3pm
More informationIntroduction to Parsing. Comp 412
COMP 412 FALL 2010 Introduction to Parsing Comp 412 Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit permission to make
More informationContextfree grammars
Contextfree grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals  tokens nonterminals  represent higherlevel structures of a program start symbol,
More information