Building a Parser Part III
|
|
- Cory Skinner
- 5 years ago
- Views:
Transcription
1 COMP 506 Rice University Spring 2018 Building a Parser Part III With Practical Application To Lab One source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp 506 at Rice University have explicit permission to make copies of these materials for their personal use. Faculty from other educational institutions may use these materials for nonprofit educational purposes, provided this copyright notice is preserved Section numbers refer to EaC2e.
2 Where Are We? Lab 1 is due in 17 days. You need to get started. You may feel that you need more knowledge. Last Class Ambiguous grammars, precedence, encoding meaning into grammar Basics of a bison input file Introduction to semantic actions in bison Today s Class How to get started with flex and bison Jump in and start coding Understanding what bison is trying to tell you 1 Handling syntax errors in bison 1 You are justified in thinking that it does a poor job of trying to tell you what is wrong COMP with your 506, grammar. Spring 2018 It takes a lot of context to understand the diagnostic information. 2
3 The Grammar for Demo In the lab 1 materials, you are given a grammar for Demo The grammar is not an LR(1) grammar. You must make it into one. You can use bison to identify the problems with the grammar You will find that we have already fixed some of the problems (e.g., precedence) You need to build flex and bison input files for Demo There are more details to learn about tokens, types, line numbers, interfaces, You will want to build some support routines (e.g., yyerror()) You will need to build a driver to call your parser You will need to test your parser on code we provide and your own examples You will want to build better error handling than bison, by default, provides The error handling facilities are somewhat cryptic & hard to use You will need to experiment with use & placement of the error token COMP 506, Spring
4 From Last Lecture Ambiguity: The Classic Example The if-then-else construct The straightforward grammar for the if-then-else construct is ambiguous Allows for both an if-then and an if-then-else Allows one if-then or if-then-else to control another 0 Stmt if Expr then Stmt 1 if Expr then Stmt else Stmt 2 other statements The problem lies in matching else clauses with then clauses The matching should be (1) consistent, (2) unambiguous, & (3) obvious This control-flow construct appears in nearly all programming languages COMP 506, Spring
5 IF-THEN-ELSE: A Final Word This ambiguity is a bit more subtle than it looks Stmts Stmts Stmt Stmt Stmt Reference = Expr IF ( Expr ) THEN Stmt IF ( Expr ) THEN Stmt ELSE Stmt Where Reference and Expr are non-terminals defined elsewhere We know how to fix this ambiguity, using the withelse transformation What happens if we add a Stmt that contains Stmt? Stmt WHILE ( Expr ) Stmt Since Stmt can expand to an if-then-else, you get an ambiguity if (a > b) then while (c > d) if (e < f) then Stmt 1 else Stmt 2 Only allow a WithElse, unless it is enclosed in some bracket construct, such as { } or begin and end. COMP 506, Spring
6 IF-THEN-ELSE: A Final Word This ambiguity is a bit more subtle than it looks Stmts Stmts Stmt Stmt Stmt Reference = Expr IF ( Expr ) THEN Stmt IF ( Expr ) THEN Stmt ELSE Stmt Where Reference and Expr are non-terminals defined elsewhere We know how to fix this ambiguity, using the withelse transformation Demo has both for and while loops The loop constructs have C-style brackets around the statement list. The brackets were added precisely to allow an if-then inside a loop. Yet another insight into why programming languages look as they do? COMP 506, Spring
7 Using bison on the if-then-else grammar When you invoke bison, it produces multiple files diana% ls ITE.y diana% bison -vd ITE.y ITE.y: conflicts: 1 shift/reduce diana% ls -1 ITE.output ITE.tab.c ITE.tab.h ITE.y diana% ITE.tab.c contains the parser itself ITE.tab.h is the set of token definitions for inclusion in the flex scanner ITE.output describes all of the states of the parser %token <int> if %token <int> then %token <int> else %token <int> Expr %token <int> assignment %type <int> Start %type <int> Stmt %% Start : Stmt ; Stmt : if Expr then Stmt if Expr then Stmt else Stmt assignment ; %% File ITE.y bison version of if-then-else Declared each terminal a token with stack type int; each nonterminal has stack type int, too. COMP 506, Spring
8 Using bison on the if-then-else grammar When you invoke bison, it produces multiple files diana% ls ITE.y diana% bison -vd ITE.y ITE.y: conflicts: 1 shift/reduce diana% ls -1 ITE.output ITE.tab.c ITE.tab.h ITE.y diana% What about this error message? The answer takes some context. ITE.tab.c contains the parser itself ITE.tab.h is the set of token definitions for inclusion in the flex scanner (-d option) ITE.output describes all of the states of the parser (-v option) %token <int> if %token <int> then %token <int> else %token <int> Expr %token <int> assignment %type <int> Start %type <int> Stmt %% Start : Stmt ; Stmt : if Expr then Stmt if Expr then Stmt else Stmt assignment ; %% File ITE.y bison version of if-then-else COMP 506, Spring
9 Using bison About that error message diana% bison -vd ITE.y ITE.y: conflicts: 1 shift/reduce diana The key to LR(1) parsing is that the parser can decide from the combination of the state, the stack, and the next word, whether to shift, reduce, accept, or throw an error The error message means that, in one parser configuration, the parser generator is unable to decide whether to shift or to reduce a shift/reduce conflict. Reduce/reduce conflicts are also possible We need to eliminate the ambiguities that create these conflicts We need to know where to look %token <int> if %token <int> then %token <int> else %token <int> Expr %token <int> assignment %type <int> Start %type <int> Stmt %% Start : Stmt ; Stmt : if Expr then Stmt if Expr then Stmt else Stmt assignment ; %% File ITE.y bison version of if-then-else COMP 506, Spring
10 Using bison When you invoke bison, it produces multiple files The ITE.output file is the key (-v option) Shows all the parser configurations Each configuration forms a state Any errors are called out by state The ITE.output file has two parts Top of the file is a summary Error messages, if any A listing of the grammar A directory of where symbols appear Bottom is a list of all the states and parser configurations State 8 conflicts: 1 shift/reduce Grammar 0 $accept: Start $end 1 Start: Stmt 2 Stmt: if Expr then Stmt 3 if Expr then Stmt else Stmt 4 assignment Terminals, with rules where they appear $end (0) 0 error (256) if (258) 2 3 then (259) 2 3 else (260) 3 Expr (261) 2 3 assignment (262) 4 Nonterminals, with rules where they appear $accept (8) on left: 0 Start (9) on left: 1, on right: 0 Stmt (10) on left: 2 3 4, on right: messages grammar directory COMP 506, Spring
11 Using bison Each state has a four-part structure Each state is labeled with a number state 0 0 $accept:. Start $end if shift, and go to state 1 assignment shift, and go to state 2 Start go to state 3 Stmt go to state 4 list of states & configurations state 1 2 Stmt: if. Expr then Stmt 3 if. Expr then Stmt else Stmt Expr shift, and go to state 5 state 2 4 Stmt: assignment. $default reduce using rule 4 (Stmt) state 3 0 $accept: Start. $end $end shift, and go to state 6 COMP 506, Spring
12 Using bison Each state has a four-part structure Each state is labeled with a number The state of the parse as a collection of LR(1) items Productions with placeholders Parser state, or configuration, is a collection of LR(1) items (often large) The listing shows the useful LR(1) items state 0 0 $accept:. Start $end if shift, and go to state 1 assignment shift, and go to state 2 Start go to state 3 Stmt go to state 4 state 1 2 Stmt: if. Expr then Stmt 3 if. Expr then Stmt else Stmt Expr shift, and go to state 5 state 2 4 Stmt: assignment. list of states & configurations $default reduce using rule 4 (Stmt) state 3 0 $accept: Start. $end $end shift, and go to state 6 See in EaC2e COMP 506, Spring
13 LR(1) Items The intermediate representation of the LR(1) table construction algorithm An LR(1) item represents a valid configuration of an LR(1) parser An LR(1) item is a pair [P, d], where P is a production A b with a at some position in the RHS d is a single symbol lookahead The in an item indicates the position of the top of the stack (symbol word or EOF) [A bg,a] means that the input seen so far is consistent with the use of A bg immediately after the symbol on top of the stack. We call an item like this a possibility. [A b g,a] means that the input sees so far is consistent with the use of A bg at this point in the parse, and that the parser has already recognized b (that is, b is on top of the stack). We call an item like this a partially complete item. [A bg,a] means that the parser has seen bg, and that a lookahead symbol of a is consistent with reducing to A. This item is complete. See in EaC2e COMP 506, Spring
14 Using bison Each state has a four-part structure Label that gives parser state number The state of the parse as a collection of LR(1) items Productions with placeholders The listing shows the useful LR(1) items A set of parser ACTIONs <word,action> pairs Actions are shift, reduce, or accept state 0 0 $accept:. Start $end if shift, and go to state 1 assignment shift, and go to state 2 Start go to state 3 Stmt go to state 4 state 1 2 Stmt: if. Expr then Stmt 3 if. Expr then Stmt else Stmt Expr shift, and go to state 5 state 2 4 Stmt: assignment. list of states & configurations $default reduce using rule 4 (Stmt) state 3 0 $accept: Start. $end $end shift, and go to state 6 COMP 506, Spring
15 Using bison Each state has a four-part structure Label that gives parser state number The state of the parse as a collection of LR(1) items Productions with placeholders The listing shows the useful LR(1) items A set of parser ACTIONs <word,action> pairs Actions are shift, reduce, or accept A set of GOTO transitions (may be empty) <NT, action> pairs Used on reduction to find new state The actions and transitions form the ACTION and GOTOtables, respectively state 0 0 $accept:. Start $end if shift, and go to state 1 assignment shift, and go to state 2 Start go to state 3 Stmt go to state 4 state 1 2 Stmt: if. Expr then Stmt 3 if. Expr then Stmt else Stmt Expr shift, and go to state 5 state 2 4 Stmt: assignment. $default reduce using rule 4 (Stmt) state 3 0 $accept: Start. $end list of states & configurations $end shift, and go to state 6 COMP 506, Spring
16 Using bison Each state has a three-part structure Label that gives parser state number The state of the parse as a collection of LR(1) items The LR(1) items encode what the parser has seen & what the possibilities are A set of parser ACTIONs Actions are taken based on the next word in the input stream: shift, reduce, or accept. No action an error A set of GOTO transitions GOTOs map a non-terminal (the lhs of a reduction) and a state into the parser s next state They thread together the derivation state 0 0 $accept:. Start $end if shift, and go to state 1 assignment shift, and go to state 2 Start go to state 3 Stmt go to state 4 state 1 2 Stmt: if. Expr then Stmt 3 if. Expr then Stmt else Stmt Expr shift, and go to state 5 state 2 4 Stmt: assignment. $default reduce using rule 4 (Stmt) state 3 0 $accept: Start. $end list of states & configurations $end shift, and go to state 6 COMP 506, Spring
17 What about those GOTO actions? The GOTO transitions thread together the derivation (what?) Look closely at reduce action Removes RHS from the stack Saves the revealed state in s Pushes LHS and some new state: GOTO[s,LHS] The revealed state encodes where the parser was when it began looking for an LHS reduce action says we found the LHS we needed GOTO transition moves parser ahead by the LHS symbol stack.push( INVALID ); stack.push(s 0 ); // initial state word NextWord(); loop forever { s stack.top(); if ( ACTION[s,word] == reduce A b ) then { stack.popnum( 2* b ); // pop RHS off stack s stack.top(); stack.push( A ); // push LHS, A stack.push( GOTO[s,A] ); // push next state } else if ( ACTION[s,word] == shift s i ) then { stack.push(word); stack.push( s i ); word NextWord(); } else if ( ACTION[s,word] == accept & word == EOF) then break; else throw a syntax error; } report success; COMP 506, Spring
18 Using bison About that error message diana% bison -vd ITE.y ITE.y: conflicts: 1 shift/reduce diana The parser has two conflicting options: shift the else onto the stack reduce the if-then binding the else to a surrounding if-then and it cannot choose between them To reach state 8, the parser must have already seen one or more if-then s Either option can lead to a derivation The grammar allows either one State 8 conflicts: 1 shift/reduce lots of stuff... state 8 2 Stmt: if Expr then Stmt. 3 if Expr then Stmt. else Stmt else shift, and go to state 9 else [reduce using rule 2 (Stmt)] $default reduce using rule 2 (Stmt) The way to read this state is that the parser has two legal actions on an else, because of an ambiguity in productions 2 and 3. The placeholders in the LR(1) items show how the productions line up for the ambiguity. And, we know how to fix it The if-then-else grammar is used as an COMP example 506, in Spring & of EaC2e 18
19 Using bison What about a reduce/reduce conflict? A reduce/reduce conflict indicates that we have two identical rhs s The third clause in our definition of ambiguity from last lecture Again, this situation is an ambiguity in the grammar Typically, this situation arises from overloading some symbol Typically, this situation is a deliberate decision by the language designer Example 0 Reference id 1 ArrayRef 2 FuncCall 3 ArrayRef id ( ExprList ) 4 FuncCall id ( ExprList ) Fortran, PL/I, & other languages used the same syntax for array subscript lists and function call parameter lists. COMP 506, Spring
20 Using bison To fix this kind of ambiguity, use just one production for the syntax The parser generator is saying that ArrayRef & FuncCall are syntactically identical The parser generator is correct. The compiler needs some other way to distinguish between these two cases. Typically, the type information on id Use one production Decide the correct use later The difference between ArrayRef and FuncCall is not syntax, but meaning Parsers lookat syntax We will revisit this decide later strategy in a couple of lectures. 0 Reference id 1 ArrayRef 2 FuncCall 3 ArrayRef id ( ExprList ) 4 FuncCall id ( ExprList ) 0 Reference id 1 id ( ExprList ) COMP 506, Spring
21 Automatic Generation of Scanners and Parsers Three time frames At design time, the compiler writer writes specifications for the microsyntax (spelling) and input program when the the syntax (grammar) What about errors in the compiler runs? At build time, (at compile the tools time?) convert specifications to How do we handle those code and compile that code in a bison-built parser? to produce the actual compiler At compile time, the user invokes the compiler to translate an application into an executable form specifications as CFGs specifications as REs stream of characters Parser Generator Scanner Generator Scanner Parser Semantic Elaboration Compiler-build time Compile time IR annotations Front End COMP 506, Spring
22 Error handling in bison First things first: bison s default error messages are terrible Enable more verbose (more precise? more explanatory?) error messages Define the preprocessor symbol YYERROR_VERBOSE Insert #define YYERROR_VERBOSE in code section at the top of the file Note that the code section is offset with brackets %{ and %} Top of the file DEMOgram.y in the lab1_ref parser %{ /* Copyright 2016, Keith D. Cooper & Linda Torczon * * Written at Rice University, Houston, Texas as part * of the instructional materials for COMP 506. */ #define YYERROR_VERBOSE #include <stdio.h> #include "demo.h" int yylineno; char *yytext; %} COMP 506, Spring
23 Error Handling in bison What does bison do with a syntax error? Action[state,lookahead] is set to invalid The parser pops <symbol,state> pairs off the stack until either It finds a state where the token error is valid, or It empties the parse stack and halts error is valid if Action[state,error] is set to something other than invalid If it finds a state where error is valid it goes back to shifting & reducing It generates an error message, to yyerror() It remains in error mode until it does three shifts or the code calls yyerrok() The compiler writer has the option of restarting with the current token (the default) or discarding the current token (call yyclearin() ) To recover from an error and keep parsing, there must be entries for error The compiler writer should add productions containing error tokens Adding error productions is subtle and takes practice error is a pseudo-token; it is treated like a token, but you need not declare it. COMP 506, Spring
24 Error Handling in bison How does the Action table entry for <state,error> get set? The compiler writer adds rules that contain the (pseudo-)token error These rules create shifts and reduces, just like other tokens A production with an error token allows the compiler writer to specify situations in which she believes that an error is likely, and A specific action that the parser should take if an error is detected The key to good error handling in a bison parser is use of error tokens The mechanism is not at all intuitive. It takes experimentation and practice. Lab 1 will give you that experience. COMP 506, Spring
25 Error handling in bison To catch a bad statement in demo, you might add Stmt Reference = Expr ; { Stmts } error { yyclearin; yyerrok;} To catch a missing right bracket, you might add Stmt { Stmts error /* no yyclearin; */ COMP 506, Spring
26 Error handling in bison Not all errors can be caught with error tokens To catch an empty statement or an empty statement list, you might add explicit productions describing the errors Stmt Reference = Expr ; ; { yyerror( empty statement, ; ); yyclearin; /* throw away ; */ } { Stmts } { } { yyerror( empty statement list ); } error { yyclearin; yyerrok;} It takes some practice and experimentation. That is why we provided you with the error input files. COMP 506, Spring
Syntax Analysis, VI Examples from LR Parsing. Comp 412
Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, VI Examples from LR Parsing Comp 412 COMP 412 FALL 2018 source code IR IR target Front End Optimizer Back End code Copyright
More informationComputing Inside The Parser Syntax-Directed Translation. Comp 412 COMP 412 FALL Chapter 4 in EaC2e. source code. IR IR target.
COMP 412 FALL 2017 Computing Inside The Parser Syntax-Directed Translation Comp 412 source code IR IR target Front End Optimizer Back End code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights
More informationComputing Inside The Parser Syntax-Directed Translation. Comp 412
COMP 412 FALL 2018 Computing Inside The Parser Syntax-Directed Translation Comp 412 source code IR IR target Front End Optimizer Back End code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights
More informationParsing II Top-down parsing. Comp 412
COMP 412 FALL 2018 Parsing II Top-down parsing Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled
More informationIntroduction to Parsing. Comp 412
COMP 412 FALL 2010 Introduction to Parsing Comp 412 Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit permission to make
More informationComputing Inside The Parser Syntax-Directed Translation, II. Comp 412 COMP 412 FALL Chapter 4 in EaC2e. source code. IR IR target.
COMP 412 FALL 20167 Computing Inside The Parser Syntax-Directed Translation, II Comp 412 source code IR IR target Front End Optimizer Back End code Copyright 2017, Keith D. Cooper & Linda Torczon, all
More informationSyntax Analysis, VII One more LR(1) example, plus some more stuff. Comp 412 COMP 412 FALL Chapter 3 in EaC2e. target code.
COMP 412 FALL 2017 Syntax Analysis, VII One more LR(1) example, plus some more stuff Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2017, Keith D. Cooper & Linda Torczon,
More informationParsing. Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice.
Parsing Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice. Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved. Students
More informationSyntax Analysis, V Bottom-up Parsing & The Magic of Handles Comp 412
Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, V Bottom-up Parsing & The Magic of Handles Comp 412 COMP 412 FALL 2018 source code IR Front End Optimizer Back End IR target
More informationParsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones
Parsing III (Top-down parsing: recursive descent & LL(1) ) (Bottom-up parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,
More informationComputing Inside The Parser Syntax-Directed Translation, II. Comp 412
COMP 412 FALL 2018 Computing Inside The Parser Syntax-Directed Translation, II Comp 412 source code IR IR target Front End Optimizer Back End code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights
More informationSyntax Analysis, III Comp 412
COMP 412 FALL 2017 Syntax Analysis, III Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp
More informationSyntax Analysis, III Comp 412
Updated algorithm for removal of indirect left recursion to match EaC3e (3/2018) COMP 412 FALL 2018 Midterm Exam: Thursday October 18, 7PM Herzstein Amphitheater Syntax Analysis, III Comp 412 source code
More informationCS415 Compilers. Syntax Analysis. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University
CS415 Compilers Syntax Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Limits of Regular Languages Advantages of Regular Expressions
More informationCS 406/534 Compiler Construction Parsing Part I
CS 406/534 Compiler Construction Parsing Part I Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof. Ken Kennedy and Dr.
More informationCS415 Compilers. LR Parsing & Error Recovery
CS415 Compilers LR Parsing & Error Recovery These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Review: LR(k) items The LR(1) table construction
More informationIntermediate Representations
COMP 506 Rice University Spring 2018 Intermediate Representations source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students
More informationEDAN65: Compilers, Lecture 06 A LR parsing. Görel Hedin Revised:
EDAN65: Compilers, Lecture 06 A LR parsing Görel Hedin Revised: 2017-09-11 This lecture Regular expressions Context-free grammar Attribute grammar Lexical analyzer (scanner) Syntactic analyzer (parser)
More informationLecture 14: Parser Conflicts, Using Ambiguity, Error Recovery. Last modified: Mon Feb 23 10:05: CS164: Lecture #14 1
Lecture 14: Parser Conflicts, Using Ambiguity, Error Recovery Last modified: Mon Feb 23 10:05:56 2015 CS164: Lecture #14 1 Shift/Reduce Conflicts If a DFA state contains both [X: α aβ, b] and [Y: γ, a],
More informationUNIT III & IV. Bottom up parsing
UNIT III & IV Bottom up parsing 5.0 Introduction Given a grammar and a sentence belonging to that grammar, if we have to show that the given sentence belongs to the given grammar, there are two methods.
More informationS Y N T A X A N A L Y S I S LR
LR parsing There are three commonly used algorithms to build tables for an LR parser: 1. SLR(1) = LR(0) plus use of FOLLOW set to select between actions smallest class of grammars smallest tables (number
More informationBottom-Up Parsing. Lecture 11-12
Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 2/20/08 Prof. Hilfinger CS164 Lecture 11 1 Administrivia Test I during class on 10 March. 2/20/08 Prof. Hilfinger CS164 Lecture 11
More informationA Bison Manual. You build a text file of the production (format in the next section); traditionally this file ends in.y, although bison doesn t care.
A Bison Manual 1 Overview Bison (and its predecessor yacc) is a tool that take a file of the productions for a context-free grammar and converts them into the tables for an LALR(1) parser. Bison produces
More informationIntroduction to Parsing
Introduction to Parsing The Front End Source code Scanner tokens Parser IR Errors Parser Checks the stream of words and their parts of speech (produced by the scanner) for grammatical correctness Determines
More informationWednesday, September 9, 15. Parsers
Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda
More informationParsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:
What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda
More informationImplementing Control Flow Constructs Comp 412
COMP 412 FALL 2018 Implementing Control Flow Constructs Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students
More informationCSCI312 Principles of Programming Languages
Copyright 2006 The McGraw-Hill Companies, Inc. CSCI312 Principles of Programming Languages! LL Parsing!! Xu Liu Derived from Keith Cooper s COMP 412 at Rice University Recap Copyright 2006 The McGraw-Hill
More informationParsing II Top-down parsing. Comp 412
COMP 412 FALL 2017 Parsing II Top-down parsing Comp 412 source code IR Front End OpMmizer Back End IR target code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled
More informationAction Table for CSX-Lite. LALR Parser Driver. Example of LALR(1) Parsing. GoTo Table for CSX-Lite
LALR r Driver Action Table for CSX-Lite Given the GoTo and parser action tables, a Shift/Reduce (LALR) parser is fairly simple: { S 5 9 5 9 void LALRDriver(){ Push(S ); } R S R R R R5 if S S R S R5 while(true){
More informationIntermediate Representations
Most of the material in this lecture comes from Chapter 5 of EaC2 Intermediate Representations Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP
More informationParsers. Xiaokang Qiu Purdue University. August 31, 2018 ECE 468
Parsers Xiaokang Qiu Purdue University ECE 468 August 31, 2018 What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure
More informationLecture 8: Deterministic Bottom-Up Parsing
Lecture 8: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Fri Feb 12 13:02:57 2010 CS164: Lecture #8 1 Avoiding nondeterministic choice: LR We ve been looking at general
More informationReview main idea syntax-directed evaluation and translation. Recall syntax-directed interpretation in recursive descent parsers
Plan for Today Review main idea syntax-directed evaluation and translation Recall syntax-directed interpretation in recursive descent parsers Syntax-directed evaluation and translation in shift-reduce
More informationParsing Wrapup. Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing. This lecture LR(1) parsing
Parsing Wrapup Roadmap (Where are we?) Last lecture Shift-reduce parser LR(1) parsing LR(1) items Computing closure Computing goto LR(1) canonical collection This lecture LR(1) parsing Building ACTION
More informationPrinciple of Compilers Lecture IV Part 4: Syntactic Analysis. Alessandro Artale
Free University of Bolzano Principles of Compilers Lecture IV Part 4, 2003/2004 AArtale (1) Principle of Compilers Lecture IV Part 4: Syntactic Analysis Alessandro Artale Faculty of Computer Science Free
More informationSyntax-Directed Translation
Syntax-Directed Translation ALSU Textbook Chapter 5.1 5.4, 4.8, 4.9 Tsan-sheng Hsu tshsu@iis.sinica.edu.tw http://www.iis.sinica.edu.tw/~tshsu 1 What is syntax-directed translation? Definition: The compilation
More information3.5 Practical Issues PRACTICAL ISSUES Error Recovery
3.5 Practical Issues 141 3.5 PRACTICAL ISSUES Even with automatic parser generators, the compiler writer must manage several issues to produce a robust, efficient parser for a real programming language.
More informationLecture 7: Deterministic Bottom-Up Parsing
Lecture 7: Deterministic Bottom-Up Parsing (From slides by G. Necula & R. Bodik) Last modified: Tue Sep 20 12:50:42 2011 CS164: Lecture #7 1 Avoiding nondeterministic choice: LR We ve been looking at general
More informationWednesday, August 31, Parsers
Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically
More informationContext-sensitive Analysis Part IV Ad-hoc syntax-directed translation, Symbol Tables, andtypes
Context-sensitive Analysis Part IV Ad-hoc syntax-directed translation, Symbol Tables, andtypes Quiz Name two differences between attribute grammars and ad-hoc syntax directed translation techniques? Example:
More informationBottom-up Parser. Jungsik Choi
Formal Languages and Compiler (CSE322) Bottom-up Parser Jungsik Choi chjs@khu.ac.kr * Some slides taken from SKKU SWE3010 (Prof. Hwansoo Han) and TAMU CSCE434-500 (Prof. Lawrence Rauchwerger) Bottom-up
More informationParsing III. (Top-down parsing: recursive descent & LL(1) )
Parsing III (Top-down parsing: recursive descent & LL(1) ) Roadmap (Where are we?) Previously We set out to study parsing Specifying syntax Context-free grammars Ambiguity Top-down parsers Algorithm &
More informationRYERSON POLYTECHNIC UNIVERSITY DEPARTMENT OF MATH, PHYSICS, AND COMPUTER SCIENCE CPS 710 FINAL EXAM FALL 96 INSTRUCTIONS
RYERSON POLYTECHNIC UNIVERSITY DEPARTMENT OF MATH, PHYSICS, AND COMPUTER SCIENCE CPS 710 FINAL EXAM FALL 96 STUDENT ID: INSTRUCTIONS Please write your student ID on this page. Do not write it or your name
More informationCSE P 501 Compilers. LR Parsing Hal Perkins Spring UW CSE P 501 Spring 2018 D-1
CSE P 501 Compilers LR Parsing Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 D-1 Agenda LR Parsing Table-driven Parsers Parser States Shift-Reduce and Reduce-Reduce conflicts UW CSE P 501 Spring 2018
More informationParsing Part II (Top-down parsing, left-recursion removal)
Parsing Part II (Top-down parsing, left-recursion removal) Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit
More informationBottom-Up Parsing. Lecture 11-12
Bottom-Up Parsing Lecture 11-12 (From slides by G. Necula & R. Bodik) 9/22/06 Prof. Hilfinger CS164 Lecture 11 1 Bottom-Up Parsing Bottom-up parsing is more general than topdown parsing And just as efficient
More informationEECS 6083 Intro to Parsing Context Free Grammars
EECS 6083 Intro to Parsing Context Free Grammars Based on slides from text web site: Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. 1 Parsing sequence of tokens parser
More informationLet us construct the LR(1) items for the grammar given below to construct the LALR parsing table.
MODULE 18 LALR parsing After understanding the most powerful CALR parser, in this module we will learn to construct the LALR parser. The CALR parser has a large set of items and hence the LALR parser is
More informationCOMP-421 Compiler Design. Presented by Dr Ioanna Dionysiou
COMP-421 Compiler Design Presented by Dr Ioanna Dionysiou Administrative! Any questions about the syllabus?! Course Material available at www.cs.unic.ac.cy/ioanna! Next time reading assignment [ALSU07]
More informationCS453 : JavaCUP and error recovery. CS453 Shift-reduce Parsing 1
CS453 : JavaCUP and error recovery CS453 Shift-reduce Parsing 1 Shift-reduce parsing in an LR parser LR(k) parser Left-to-right parse Right-most derivation K-token look ahead LR parsing algorithm using
More informationMonday, September 13, Parsers
Parsers Agenda Terminology LL(1) Parsers Overview of LR Parsing Terminology Grammar G = (Vt, Vn, S, P) Vt is the set of terminals Vn is the set of non-terminals S is the start symbol P is the set of productions
More informationContext-Free Grammar. Concepts Introduced in Chapter 2. Parse Trees. Example Grammar and Derivation
Concepts Introduced in Chapter 2 A more detailed overview of the compilation process. Parsing Scanning Semantic Analysis Syntax-Directed Translation Intermediate Code Generation Context-Free Grammar A
More informationConfiguration Sets for CSX- Lite. Parser Action Table
Configuration Sets for CSX- Lite State s 6 s 7 Cofiguration Set Prog { Stmts } Eof Stmts Stmt Stmts State s s Cofiguration Set Prog { Stmts } Eof Prog { Stmts } Eof Stmts Stmt Stmts Stmts λ Stmt if ( Expr
More informationSyntax Analysis Part IV
Syntax Analysis Part IV Chapter 4: Bison Slides adapted from : Robert van Engelen, Florida State University Yacc and Bison Yacc (Yet Another Compiler Compiler) Generates LALR(1) parsers Bison Improved
More informationCompilation 2013 Parser Generators, Conflict Management, and ML-Yacc
Compilation 2013 Parser Generators, Conflict Management, and ML-Yacc Erik Ernst Aarhus University Parser generators, ML-Yacc LR parsers are tedious to write, but can be generated, e.g., by ML-Yacc Input:
More informationUsing an LALR(1) Parser Generator
Using an LALR(1) Parser Generator Yacc is an LALR(1) parser generator Developed by S.C. Johnson and others at AT&T Bell Labs Yacc is an acronym for Yet another compiler compiler Yacc generates an integrated
More informationCS 4120 Introduction to Compilers
CS 4120 Introduction to Compilers Andrew Myers Cornell University Lecture 6: Bottom-Up Parsing 9/9/09 Bottom-up parsing A more powerful parsing technology LR grammars -- more expressive than LL can handle
More informationMore Bottom-Up Parsing
More Bottom-Up Parsing Lecture 7 Dr. Sean Peisert ECS 142 Spring 2009 1 Status Project 1 Back By Wednesday (ish) savior lexer in ~cs142/s09/bin Project 2 Due Friday, Apr. 24, 11:55pm My office hours 3pm
More informationLR Parsing LALR Parser Generators
LR Parsing LALR Parser Generators Outline Review of bottom-up parsing Computing the parsing DFA Using parser generators 2 Bottom-up Parsing (Review) A bottom-up parser rewrites the input string to the
More informationGenerating Code for Assignment Statements back to work. Comp 412 COMP 412 FALL Chapters 4, 6 & 7 in EaC2e. source code. IR IR target.
COMP 412 FALL 2017 Generating Code for Assignment Statements back to work Comp 412 source code IR IR target Front End Optimizer Back End code Copyright 2017, Keith D. Cooper & Linda Torczon, all rights
More informationCS 406/534 Compiler Construction Putting It All Together
CS 406/534 Compiler Construction Putting It All Together Prof. Li Xu Dept. of Computer Science UMass Lowell Fall 2004 Part of the course lecture notes are based on Prof. Keith Cooper, Prof. Ken Kennedy
More informationParsing Part II. (Ambiguity, Top-down parsing, Left-recursion Removal)
Parsing Part II (Ambiguity, Top-down parsing, Left-recursion Removal) Ambiguous Grammars Definitions If a grammar has more than one leftmost derivation for a single sentential form, the grammar is ambiguous
More informationProgramming Language Syntax and Analysis
Programming Language Syntax and Analysis 2017 Kwangman Ko (http://compiler.sangji.ac.kr, kkman@sangji.ac.kr) Dept. of Computer Engineering, Sangji University Introduction Syntax the form or structure of
More informationCSCI Compiler Design
Syntactic Analysis Automatic Parser Generators: The UNIX YACC Tool Portions of this lecture were adapted from Prof. Pedro Reis Santos s notes for the 2006 Compilers class lectured at IST/UTL in Lisbon,
More informationLexical Analysis - An Introduction. Lecture 4 Spring 2005 Department of Computer Science University of Alabama Joel Jones
Lexical Analysis - An Introduction Lecture 4 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved.
More informationSyntactic Analysis. Top-Down Parsing
Syntactic Analysis Top-Down Parsing Copyright 2017, Pedro C. Diniz, all rights reserved. Students enrolled in Compilers class at University of Southern California (USC) have explicit permission to make
More informationDEMO A Language for Practice Implementation Comp 506, Spring 2018
DEMO A Language for Practice Implementation Comp 506, Spring 2018 1 Purpose This document describes the Demo programming language. Demo was invented for instructional purposes; it has no real use aside
More informationHandling Assignment Comp 412
COMP 412 FALL 2018 Handling Assignment Comp 412 source code IR IR target Front End Optimizer Back End code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled in Comp
More informationLab 2. Lexing and Parsing with Flex and Bison - 2 labs
Lab 2 Lexing and Parsing with Flex and Bison - 2 labs Objective Understand the software architecture of flex/bison. Be able to write simple grammars in bison. Be able to correct grammar issues in bison.
More informationCompilers. Bottom-up Parsing. (original slides by Sam
Compilers Bottom-up Parsing Yannis Smaragdakis U Athens Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Bottom-Up Parsing More general than top-down parsing And just as efficient Builds
More informationLALR stands for look ahead left right. It is a technique for deciding when reductions have to be made in shift/reduce parsing. Often, it can make the
LALR parsing 1 LALR stands for look ahead left right. It is a technique for deciding when reductions have to be made in shift/reduce parsing. Often, it can make the decisions without using a look ahead.
More informationContext-free grammars
Context-free grammars Section 4.2 Formal way of specifying rules about the structure/syntax of a program terminals - tokens non-terminals - represent higher-level structures of a program start symbol,
More informationIn One Slide. Outline. LR Parsing. Table Construction
LR Parsing Table Construction #1 In One Slide An LR(1) parsing table can be constructed automatically from a CFG. An LR(1) item is a pair made up of a production and a lookahead token; it represents a
More informationConflicts in LR Parsing and More LR Parsing Types
Conflicts in LR Parsing and More LR Parsing Types Lecture 10 Dr. Sean Peisert ECS 142 Spring 2009 1 Status Project 2 Due Friday, Apr. 24, 11:55pm The usual lecture time is being replaced by a discussion
More informationLesson 10. CDT301 Compiler Theory, Spring 2011 Teacher: Linus Källberg
Lesson 10 CDT301 Compiler Theory, Spring 2011 Teacher: Linus Källberg Outline Flex Bison Abstract syntax trees 2 FLEX 3 Flex Tool for automatic generation of scanners Open-source version of Lex Takes regular
More informationSimple LR (SLR) LR(0) Drawbacks LR(1) SLR Parse. LR(1) Start State and Reduce. LR(1) Items 10/3/2012
LR(0) Drawbacks Consider the unambiguous augmented grammar: 0.) S E $ 1.) E T + E 2.) E T 3.) T x If we build the LR(0) DFA table, we find that there is a shift-reduce conflict. This arises because the
More informationLexical Analysis. Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice.
Lexical Analysis Note by Baris Aktemur: Our slides are adapted from Cooper and Torczon s slides that they prepared for COMP 412 at Rice. Copyright 2010, Keith D. Cooper & Linda Torczon, all rights reserved.
More informationLR Parsing. Leftmost and Rightmost Derivations. Compiler Design CSE 504. Derivations for id + id: T id = id+id. 1 Shift-Reduce Parsing.
LR Parsing Compiler Design CSE 504 1 Shift-Reduce Parsing 2 LR Parsers 3 SLR and LR(1) Parsers Last modifled: Fri Mar 06 2015 at 13:50:06 EST Version: 1.7 16:58:46 2016/01/29 Compiled at 12:57 on 2016/02/26
More informationCOMP 181. Prelude. Prelude. Summary of parsing. A Hierarchy of Grammar Classes. More power? Syntax-directed translation. Analysis
Prelude COMP 8 October, 9 What is triskaidekaphobia? Fear of the number s? No aisle in airplanes, no th floor in buildings Fear of Friday the th? Paraskevidedekatriaphobia or friggatriskaidekaphobia Why
More informationDownloaded from Page 1. LR Parsing
Downloaded from http://himadri.cmsdu.org Page 1 LR Parsing We first understand Context Free Grammars. Consider the input string: x+2*y When scanned by a scanner, it produces the following stream of tokens:
More informationLR Parsing LALR Parser Generators
Outline LR Parsing LALR Parser Generators Review of bottom-up parsing Computing the parsing DFA Using parser generators 2 Bottom-up Parsing (Review) A bottom-up parser rewrites the input string to the
More informationChapter 4. Lexical and Syntax Analysis
Chapter 4 Lexical and Syntax Analysis Chapter 4 Topics Introduction Lexical Analysis The Parsing Problem Recursive-Descent Parsing Bottom-Up Parsing Copyright 2012 Addison-Wesley. All rights reserved.
More informationCOMPILER CONSTRUCTION LAB 2 THE SYMBOL TABLE. Tutorial 2 LABS. PHASES OF A COMPILER Source Program. Lab 2 Symbol table
COMPILER CONSTRUCTION Lab 2 Symbol table LABS Lab 3 LR parsing and abstract syntax tree construction using ''bison' Lab 4 Semantic analysis (type checking) PHASES OF A COMPILER Source Program Lab 2 Symtab
More informationFormal Languages and Compilers Lecture VII Part 4: Syntactic A
Formal Languages and Compilers Lecture VII Part 4: Syntactic Analysis Free University of Bozen-Bolzano Faculty of Computer Science POS Building, Room: 2.03 artale@inf.unibz.it http://www.inf.unibz.it/
More informationThe Software Stack: From Assembly Language to Machine Code
COMP 506 Rice University Spring 2018 The Software Stack: From Assembly Language to Machine Code source code IR Front End Optimizer Back End IR target code Somewhere Out Here Copyright 2018, Keith D. Cooper
More informationBottom-up parsing. Bottom-Up Parsing. Recall. Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form
Bottom-up parsing Bottom-up parsing Recall Goal: For a grammar G, withstartsymbols, any string α such that S α is called a sentential form If α V t,thenα is called a sentence in L(G) Otherwise it is just
More informationBottom-Up Parsing II. Lecture 8
Bottom-Up Parsing II Lecture 8 1 Review: Shift-Reduce Parsing Bottom-up parsing uses two actions: Shift ABC xyz ABCx yz Reduce Cbxy ijk CbA ijk 2 Recall: he Stack Left string can be implemented by a stack
More informationPart 5 Program Analysis Principles and Techniques
1 Part 5 Program Analysis Principles and Techniques Front end 2 source code scanner tokens parser il errors Responsibilities: Recognize legal programs Report errors Produce il Preliminary storage map Shape
More informationLR Parsing of CFG's with Restrictions
LR Parsing of CFG's with Restrictions B. J. McKenzie University of Canterbury, Christchurch, New Zealand Postal Address: Dr. B.J.McKenzie, Phone: +64 3 642-349 Department of Computer Science, Fax: +64
More informationICOM 4036 Spring 2004
Language Specification and Translation ICOM 4036 Spring 2004 Lecture 3 Copyright 2004 Pearson Addison-Wesley. All rights reserved. 3-1 Language Specification and Translation Topics Structure of a Compiler
More informationComp 411 Principles of Programming Languages Lecture 3 Parsing. Corky Cartwright January 11, 2019
Comp 411 Principles of Programming Languages Lecture 3 Parsing Corky Cartwright January 11, 2019 Top Down Parsing What is a context-free grammar (CFG)? A recursive definition of a set of strings; it is
More informationGrammars. CS434 Lecture 15 Spring 2005 Department of Computer Science University of Alabama Joel Jones
Grammars CS434 Lecture 5 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. Students enrolled
More informationAlternatives for semantic processing
Semantic Processing Copyright c 2000 by Antony L. Hosking. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies
More informationCS415 Compilers. Lexical Analysis
CS415 Compilers Lexical Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Lecture 7 1 Announcements First project and second homework
More informationCSE P 501 Compilers. Parsing & Context-Free Grammars Hal Perkins Spring UW CSE P 501 Spring 2018 C-1
CSE P 501 Compilers Parsing & Context-Free Grammars Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 C-1 Administrivia Project partner signup: please find a partner and fill out the signup form by noon
More informationCSE 582 Autumn 2002 Exam 11/26/02
Name There are 8 questions worth a total of 100 points. Please budget your time so you get to all of the questions. Keep your answers brief and to the point. You may refer to the following reference materials:
More informationflex is not a bad tool to use for doing modest text transformations and for programs that collect statistics on input.
flex is not a bad tool to use for doing modest text transformations and for programs that collect statistics on input. More often than not, though, you ll want to use flex to generate a scanner that divides
More informationLR Parsing Techniques
LR Parsing Techniques Introduction Bottom-Up Parsing LR Parsing as Handle Pruning Shift-Reduce Parser LR(k) Parsing Model Parsing Table Construction: SLR, LR, LALR 1 Bottom-UP Parsing A bottom-up parser
More informationCSCE 531, Spring 2015 Final Exam Answer Key
CSCE 531, Spring 2015 Final Exam Answer Key 1. (40 points total) Consider the following grammar with start symbol S : S S S asb S T T T a T cs T ɛ (a) (10 points) Find FIRST(S), FIRST(T ), FOLLOW(S), and
More information