Lecture 12: Parser-Generating Tools
|
|
- Osborne Booker
- 6 years ago
- Views:
Transcription
1 Lecture 12: Parser-Generating Tools Dr Kieran T. Herley Department of Computer Science University College Cork KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
2 Summary Overview here KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
3 Parser-Generating Tools Parser-Generating Tools Yacc Tool to automate production of parsers Input: CFG (suitably formatted and annotated) Output: Parser code Yet Another Compiler Compiler Default parser generator tool in earlier years Produces LALR parser C-based KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
4 Javacc Java-based parser-generating tool Input:.jj file containing suitably formatted grammer etc. Output:.java file containing LL(1) parser Note: javacc (and related tools) installed in WGB labs. KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
5 Using Javacc Usage javacc Simple.jj generates Simple.java (plus possibly others) KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
6 Using Javacc Usage javacc Simple.jj javac Simple.java generates Simple.java (plus possibly others) generates Simple.class (plus others) KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
7 Using Javacc Usage javacc Simple.jj generates Simple.java (plus possibly others) javac Simple.java generates Simple.class (plus others) java Simple runs parser in the usual way Notes (Syntax) errors can occur at javacc or javac stage KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
8 Typical Javacc Code PARSER BEGIN(ExprParser) / Java main method that calls grammars start symbol / PARSER END(ExprParser) / Token descriptions here / / Grammar here /... Built around context-free grammar expressed in eccentric javacc syntax (details later) KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
9 Typical Javacc Code [ExprParser0.jj] PARSER BEGIN(ExprParser) / Java main method that calls Expression () / PARSER END(ExprParser) / Token descriptions here / / <Expression> > <SimpleExpression> / void Expression () : { { SimpleExpression()... / <SeqOfTerms> > <Term> {<Addop> <Term> / void SeqOfTerms() : { { Term() ( Addop() Term() )... KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
10 Java Code Generated Generated code forms recursive descent parser with one method per grammar production.... static final public void SeqOfTerms() throws ParseException { Term(); label 1 : while (true) { switch (( jj ntk == 1)?jj ntk():jj ntk) { case PLUS: case MINUS: ; break; default : jj la1 [0] = jj gen ; break label 1 ; Addop(); Term(); Note: Term()... Addop()... Term()... calling pattern Hideous code not intended for human consumption KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
11 Capabilities of Javacc Javacc very flexible Can augment basic parser with actions to accomplish tasks in syntax-aware way Can be used to build trees, generate code etc KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
12 Simple Balanced Parentheses Example Javacc s Grammar Syntax I MB EOF MB { [ MB ] Javacc s grammar notation void Input() :{ { MatchedBraces() ( \n \r ) EOF void MatchedBraces() :{ { { [ MatchedBraces() ] Terminals Java-function notation, e.g. MatchedBraces() Nonterminals quoted strings, e.g. { NB [...] denotes... optional (also (...)? NB < EOF > special end-offile symbol KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
13 Simple Balanced Parentheses Example Javacc s Grammar Syntax cont d MB { [ MB ] Each grammar production modelled by pseudo-java method: void MatchedBraces() :{ { / RHS goes here / KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
14 Simple Balanced Parentheses Example Javacc s Grammar Syntax cont d MB { [ MB ] Each grammar production modelled by pseudo-java method: void MatchedBraces() :{ { / RHS goes here / RHS symbols listed inside {: <MB> -> { [ <MB> ] v v v v void MatchedBraces():{{ "{" [ MatchedBraces() ] "" method call syntax for nonterminals (MatchedBraces()) terminals quoted ("{") KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
15 Simple Balanced Parentheses Example KHSimple1.jj PARSER BEGIN(KHSimple1) public class KHSimple1 { public static void main(string args []) throws ParseException { KHSimple1 parser = new KHSimple1(System.in); parser.input (); PARSER END(KHSimple1) / Grammar as before / Method main drives parse: stipulates input taken from System.in; calls method Input() to get ball rolling KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
16 Simple Balanced Parentheses Example Tidying Up SKIP : { \t \n \r... TOKEN : { <LBRACE: { > <RBRACE: > Include SKIP clause (before grammar) to specify whitespace characters to be ignored Can name token specifications (regular expressions) and use in grammar e.g. int MatchedBraces() : { { <LBRACE> [MatchedBraces() ] <RBRACE> NB confusing use of angle brackets for terminals here KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
17 Simple Balanced Parentheses Example Actions Can annotate CFG with actions (Java snippets) akin to jflex actions KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
18 Simple Balanced Parentheses Example Actions Can annotate CFG with actions (Java snippets) akin to jflex actions void Input() : { int count; { count=matchedbraces() <EOF> { System.out. println ( The levels of nesting is + count); / Action / int MatchedBraces() : { int nested count=0; { <LBRACE> [ nested count=matchedbraces() ] <RBRACE> { return ++nested count; / Action / Collectively actions compute number of nested { and outputs result KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
19 Simple Balanced Parentheses Example Actions cont d / 1 / int MatchedBraces() : / 2 / { int nested count=0; / 3 / { <LBRACE> [ nested count=matchedbraces() ] <RBRACE> / 4 / { return ++nested count; 1 Changing return type to int, generates int-returning method MatchedBraces KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
20 Simple Balanced Parentheses Example Actions cont d / 1 / int MatchedBraces() : / 2 / { int nested count=0; / 3 / { <LBRACE> [ nested count=matchedbraces() ] <RBRACE> / 4 / { return ++nested count; 1 Changing return type to int, generates int-returning method MatchedBraces 2 Adding statement inside : { bit smuggles declaration/initialization into generated code KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
21 Simple Balanced Parentheses Example Actions cont d / 1 / int MatchedBraces() : / 2 / { int nested count=0; / 3 / { <LBRACE> [ nested count=matchedbraces() ] <RBRACE> / 4 / { return ++nested count; 1 Changing return type to int, generates int-returning method MatchedBraces 2 Adding statement inside : { bit smuggles declaration/initialization into generated code 3 Assignment nested_count= assignment, captures value returned by recursive call KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
22 Simple Balanced Parentheses Example Actions cont d / 1 / int MatchedBraces() : / 2 / { int nested count=0; / 3 / { <LBRACE> [ nested count=matchedbraces() ] <RBRACE> / 4 / { return ++nested count; 1 Changing return type to int, generates int-returning method MatchedBraces 2 Adding statement inside : { bit smuggles declaration/initialization into generated code 3 Assignment nested_count= assignment, captures value returned by recursive call 4 Action { return... smuggled into generated code; returned brace count when executed at run time KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
23 Simple Balanced Parentheses Example Actions cont d int MatchedBraces() : { int nested count=0; { <LBRACE> [ nested count=matchedbraces() ] <RBRACE> { return ++nested count; / action / Execution of the action triggered after parse of <LBRACE>...<RBRACE> has been completed KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
24 Simple Balanced Parentheses Example Illustration void Input() : { int count; { count=matchedbraces() <EOF> { System.out. println ( The levels of nesting is + count); int MatchedBraces() : { int nested count=0; { <LBRACE> [ nested count=matchedbraces() ] <RBRACE> { return ++nested count; KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
25 Expression Grammar Simple Expression Grammar Here is our grammar expression simple-expression simple-expression seq-of-terms seq-of-terms term seq-of-terms addop term term seq-of-factors seq-of-factors factor seq-of-factors mulop factor factor INTLIT ( simple-expression ) addop + - mulop * / But Javacc dislikes left recursion so must recast to circumvent this... KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
26 Expression Grammar Modified Grammar Using EBNF to re-express repetition we get expression simple-expression simple-expression seq-of-terms seq-of-terms term { addop term term seq-of-factors seq-of-factors factor { mulop factor factor INTLIT ( simple-expression ) addop + - mulop * / Recall {... indicates zero or more reps of... KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
27 Expression Grammar Overall Structure of ExprParser0.jj File PARSER BEGIN(ExprParser)... PARSER END(ExprParser) / Tokens and whitespace definitions / / Grammar productions / Note PARSER BEGIN specifies name of generated Java class; need not match name of.jj file. KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
28 Expression Grammar Token Definitions SKIP : / WHITE SPACE / { \t \n \r \f TOKEN : { < PLUS: + > < MINUS: > < TIMES: > < DIV: / > < LPAREN: ( > < RPAREN: ) > Javacc supports full regular expression specifications for token definition Also provides for non-tokens SKIP: stuff to be ignored COMMENTS: not shown here TOKEN : / LITERALS / { < NUM: [ 1 9 ] ([ 0 9 ]) > KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
29 Expression Grammar Grammar void Expression () : { { SimpleExpression()<EOF> void SimpleExpression() : { { SeqOfTerms() void SeqOfTerms() : { { Term() ( Addop() Term() ) / ETC ETC ETC / (...) denotes zero or more repetitions (...)+ for one or more reps Note use of named tokens void Mulop() : { { <TIMES> <DIV> KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
30 Expression Grammar Main Method PARSER BEGIN(ExprParser) public class ExprParser { / Main entry point. / public static void main(string args []) throws ParseException { ExprParser parser = new ExprParser(System.in); parser. Expression (); PARSER END(ExprParser) Can specify more sophisticated main to take input from file for example. KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
31 Expression Grammar Extending Expression Parser Modify to compute value of expression Modifications: Change method/nonterminal return types to int Add actions to productions to compute value of corresponding subexpression, / <term> > <seq of factors> / int Term() : { int value ; { value = SeqOfFactors() { return value ; Action code gets integrated into Java code generated KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
32 Expression Grammar More Actions / <seq of factors> > <factor> {<mulop> <factor> / int SeqOfFactors() : { int value ; int temp; char op; { value = Factor() ( op = Mulop() temp = Factor() { if (op == *) value = temp; else value /= temp; ) { System.out. println ( <seqoffactors> > <factor> {<mulop> <factor> ); return value ; Note intermingling of RHS of production KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
33 Expression Grammar Illustration Return actions get executed in postorder fashion Node corresponding to each nonterminal returns value or subexpression to its parent KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
34 Expression Grammar Acknowledgements Examples KHSimple[123].jj are adapted from the Simple[123].jj examples bundled with the official javacc download. KH (31/10/17) Lecture 12: Parser-Generating Tools / 27
JavaCC: SimpleExamples
JavaCC: SimpleExamples This directory contains five examples to get you started using JavaCC. Each example is contained in a single grammar file and is listed below: (1) Simple1.jj, (2) Simple2.jj, (3)
More informationLecture 8: Context Free Grammars
Lecture 8: Context Free s Dr Kieran T. Herley Department of Computer Science University College Cork 2017-2018 KH (12/10/17) Lecture 8: Context Free s 2017-2018 1 / 1 Specifying Non-Regular Languages Recall
More informationSimple Lexical Analyzer
Lecture 7: Simple Lexical Analyzer Dr Kieran T. Herley Department of Computer Science University College Cork 2017-2018 KH (03/10/17) Lecture 7: Simple Lexical Analyzer 2017-2018 1 / 1 Summary Use of jflex
More informationCIT Lecture 5 Context-Free Grammars and Parsing 4/2/2003 1
CIT3136 - Lecture 5 Context-Free Grammars and Parsing 4/2/2003 1 Definition of a Context-free Grammar: An alphabet or set of basic symbols (like regular expressions, only now the symbols are whole tokens,
More informationPart III : Parsing. From Regular to Context-Free Grammars. Deriving a Parser from a Context-Free Grammar. Scanners and Parsers.
Part III : Parsing From Regular to Context-Free Grammars Deriving a Parser from a Context-Free Grammar Scanners and Parsers A Parser for EBNF Left-Parsable Grammars Martin Odersky, LAMP/DI 1 From Regular
More informationDr. D.M. Akbar Hussain
Syntax Analysis Parsing Syntax Or Structure Given By Determines Grammar Rules Context Free Grammar 1 Context Free Grammars (CFG) Provides the syntactic structure: A grammar is quadruple (V T, V N, S, R)
More informationLecture 15-16: Intermediate Code-Generation
Lecture 15-16: Intermediate Code-Generation Dr Kieran T. Herley Department of Computer Science University College Cork 2017-2018 KH (16/11/17) Lecture 15-16: Intermediate Code-Generation 2017-2018 1 /
More informationReview main idea syntax-directed evaluation and translation. Recall syntax-directed interpretation in recursive descent parsers
Plan for Today Review main idea syntax-directed evaluation and translation Recall syntax-directed interpretation in recursive descent parsers Syntax-directed evaluation and translation in shift-reduce
More informationProgramming Languages. Dr. Philip Cannata 1
Programming Languages Dr. Philip Cannata 0 High Level Languages This Course Java (Object Oriented) Jython in Java Relation ASP RDF (Horn Clause Deduction, Semantic Web) Dr. Philip Cannata Dr. Philip Cannata
More informationCIT 3136 Lecture 7. Top-Down Parsing
CIT 3136 Lecture 7 Top-Down Parsing Chapter 4: Top-down Parsing A top-down parsing algorithm parses an input string of tokens by tracing out the steps in a leftmost derivation. Such an algorithm is called
More informationLeft to right design 1
Left to right design 1 Left to right design The left to right design method suggests that the structure of the program should closely follow the structure of the input. The method is effective when the
More informationJavaCC Parser. The Compilation Task. Automated? JavaCC Parser
JavaCC Parser The Compilation Task Input character stream Lexer stream Parser Abstract Syntax Tree Analyser Annotated AST Code Generator Code CC&P 2003 1 CC&P 2003 2 Automated? JavaCC Parser The initial
More informationLexical and Syntax Analysis
Lexical and Syntax Analysis In Text: Chapter 4 N. Meng, F. Poursardar Lexical and Syntactic Analysis Two steps to discover the syntactic structure of a program Lexical analysis (Scanner): to read the input
More informationA Simple Syntax-Directed Translator
Chapter 2 A Simple Syntax-Directed Translator 1-1 Introduction The analysis phase of a compiler breaks up a source program into constituent pieces and produces an internal representation for it, called
More informationCOMP3131/9102: Programming Languages and Compilers
COMP3131/9102: Programming Languages and Compilers Jingling Xue School of Computer Science and Engineering The University of New South Wales Sydney, NSW 2052, Australia http://www.cse.unsw.edu.au/~cs3131
More informationChapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part2 3.3 Parse Trees and Abstract Syntax Trees
Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part2 3.3 Parse Trees and Abstract Syntax Trees 3.3.1 Parse trees 1. Derivation V.S. Structure Derivations do not uniquely represent the structure of the strings
More informationProgramming Languages. Dr. Philip Cannata 1
Programming Languages Dr. Philip Cannata 0 High Level Languages This Course Jython in Java Java (Object Oriented) ACL (Propositional Induction) Relation Algorithmic Information Theory (Information Compression
More information3. Context-free grammars & parsing
3. Context-free grammars & parsing The parsing process sequences of tokens parse tree or syntax tree a / [ / index / ]/= / 4 / + / 2 The parsing process sequences of tokens parse tree or syntax tree a
More information10/5/17. Lexical and Syntactic Analysis. Lexical and Syntax Analysis. Tokenizing Source. Scanner. Reasons to Separate Lexical and Syntax Analysis
Lexical and Syntactic Analysis Lexical and Syntax Analysis In Text: Chapter 4 Two steps to discover the syntactic structure of a program Lexical analysis (Scanner): to read the input characters and output
More information10/4/18. Lexical and Syntactic Analysis. Lexical and Syntax Analysis. Tokenizing Source. Scanner. Reasons to Separate Lexical and Syntactic Analysis
Lexical and Syntactic Analysis Lexical and Syntax Analysis In Text: Chapter 4 Two steps to discover the syntactic structure of a program Lexical analysis (Scanner): to read the input characters and output
More informationLecture 4: Stack Applications CS2504/CS4092 Algorithms and Linear Data Structures. Parentheses and Mathematical Expressions
Lecture 4: Applications CS2504/CS4092 Algorithms and Linear Data Structures Dr Kieran T. Herley Department of Computer Science University College Cork Summary. Postfix notation for arithmetic expressions.
More informationParser Combinators 11/3/2003 IPT, ICS 1
Parser Combinators 11/3/2003 IPT, ICS 1 Parser combinator library Similar to those from Grammars & Parsing But more efficient, self-analysing error recovery 11/3/2003 IPT, ICS 2 Basic combinators Similar
More informationRecursive Descent Parsers
Recursive Descent Parsers Lecture 7 Robb T. Koether Hampden-Sydney College Wed, Jan 28, 2015 Robb T. Koether (Hampden-Sydney College) Recursive Descent Parsers Wed, Jan 28, 2015 1 / 18 1 Parsing 2 LL Parsers
More informationCS 11 Ocaml track: lecture 6
CS 11 Ocaml track: lecture 6 n Today: n Writing a computer language n Parser generators n lexers (ocamllex) n parsers (ocamlyacc) n Abstract syntax trees Problem (1) n We want to implement a computer language
More informationEDAN65: Compilers, Lecture 06 A LR parsing. Görel Hedin Revised:
EDAN65: Compilers, Lecture 06 A LR parsing Görel Hedin Revised: 2017-09-11 This lecture Regular expressions Context-free grammar Attribute grammar Lexical analyzer (scanner) Syntactic analyzer (parser)
More informationCUP. Lecture 18 CUP User s Manual (online) Robb T. Koether. Hampden-Sydney College. Fri, Feb 27, 2015
CUP Lecture 18 CUP User s Manual (online) Robb T. Koether Hampden-Sydney College Fri, Feb 27, 2015 Robb T. Koether (Hampden-Sydney College) CUP Fri, Feb 27, 2015 1 / 31 1 The CUP Parser Generator 2 The
More information1 Lexical Considerations
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.035, Spring 2013 Handout Decaf Language Thursday, Feb 7 The project for the course is to write a compiler
More informationLast time. What are compilers? Phases of a compiler. Scanner. Parser. Semantic Routines. Optimizer. Code Generation. Sunday, August 29, 2010
Last time Source code Scanner Tokens Parser What are compilers? Phases of a compiler Syntax tree Semantic Routines IR Optimizer IR Code Generation Executable Extra: Front-end vs. Back-end Scanner + Parser
More informationLexical Analysis and jflex
Lecture 6: Lexical Analysis and jflex Dr Kieran T. Herley Department of Computer Science University College Cork 2017-2018 KH (03/10/17) Lecture 6: Lexical Analysis and jflex 2017-2018 1 / 1 Summary Lexical
More informationProgramming Languages (CS 550) Lecture 4 Summary Scanner and Parser Generators. Jeremy R. Johnson
Programming Languages (CS 550) Lecture 4 Summary Scanner and Parser Generators Jeremy R. Johnson 1 Theme We have now seen how to describe syntax using regular expressions and grammars and how to create
More informationCMSC 330: Organization of Programming Languages. Context Free Grammars
CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler
More informationQuestion Points Score
CS 453 Introduction to Compilers Midterm Examination Spring 2009 March 12, 2009 75 minutes (maximum) Closed Book You may use one side of one sheet (8.5x11) of paper with any notes you like. This exam has
More informationChapter 3. Parsing #1
Chapter 3 Parsing #1 Parser source file get next character scanner get token parser AST token A parser recognizes sequences of tokens according to some grammar and generates Abstract Syntax Trees (ASTs)
More informationUsing an LALR(1) Parser Generator
Using an LALR(1) Parser Generator Yacc is an LALR(1) parser generator Developed by S.C. Johnson and others at AT&T Bell Labs Yacc is an acronym for Yet another compiler compiler Yacc generates an integrated
More informationParser Tools: lex and yacc-style Parsing
Parser Tools: lex and yacc-style Parsing Version 6.11.0.6 Scott Owens January 6, 2018 This documentation assumes familiarity with lex and yacc style lexer and parser generators. 1 Contents 1 Lexers 3 1.1
More informationSimple LR (SLR) LR(0) Drawbacks LR(1) SLR Parse. LR(1) Start State and Reduce. LR(1) Items 10/3/2012
LR(0) Drawbacks Consider the unambiguous augmented grammar: 0.) S E $ 1.) E T + E 2.) E T 3.) T x If we build the LR(0) DFA table, we find that there is a shift-reduce conflict. This arises because the
More informationName SOLUTIONS EID NOTICE: CHEATING ON THE MIDTERM WILL RESULT IN AN F FOR THE COURSE.
CS 345 Fall TTh 2012 Midterm Exam B Name SOLUTIONS EID NOTICE: CHEATING ON THE MIDTERM WILL RESULT IN AN F FOR THE COURSE. 1. [5 Points] Give two of the following three definitions. [2 points extra credit
More informationSyntax-Directed Translation. Lecture 14
Syntax-Directed Translation Lecture 14 (adapted from slides by R. Bodik) 9/27/2006 Prof. Hilfinger, Lecture 14 1 Motivation: parser as a translator syntax-directed translation stream of tokens parser ASTs,
More informationCompilers. Compiler Construction Tutorial The Front-end
Compilers Compiler Construction Tutorial The Front-end Salahaddin University College of Engineering Software Engineering Department 2011-2012 Amanj Sherwany http://www.amanj.me/wiki/doku.php?id=teaching:su:compilers
More informationCOP 3402 Systems Software Syntax Analysis (Parser)
COP 3402 Systems Software Syntax Analysis (Parser) Syntax Analysis 1 Outline 1. Definition of Parsing 2. Context Free Grammars 3. Ambiguous/Unambiguous Grammars Syntax Analysis 2 Lexical and Syntax Analysis
More informationThe Parsing Problem (cont d) Recursive-Descent Parsing. Recursive-Descent Parsing (cont d) ICOM 4036 Programming Languages. The Complexity of Parsing
ICOM 4036 Programming Languages Lexical and Syntax Analysis Lexical Analysis The Parsing Problem Recursive-Descent Parsing Bottom-Up Parsing This lecture covers review questions 14-27 This lecture covers
More informationCS 536 Midterm Exam Spring 2013
CS 536 Midterm Exam Spring 2013 ID: Exam Instructions: Write your student ID (not your name) in the space provided at the top of each page of the exam. Write all your answers on the exam itself. Feel free
More informationCS 4240: Compilers and Interpreters Project Phase 1: Scanner and Parser Due Date: October 4 th 2015 (11:59 pm) (via T-square)
CS 4240: Compilers and Interpreters Project Phase 1: Scanner and Parser Due Date: October 4 th 2015 (11:59 pm) (via T-square) Introduction This semester, through a project split into 3 phases, we are going
More informationName EID. (calc (parse '{+ {with {x {+ 5 5}} {with {y {- x 3}} {+ y y} } } z } ) )
CS 345 Spring 2010 Midterm Exam Name EID 1. [4 Points] Circle the binding instances in the following expression: (calc (parse '+ with x + 5 5 with y - x 3 + y y z ) ) 2. [7 Points] Using the following
More informationASTs, Objective CAML, and Ocamlyacc
ASTs, Objective CAML, and Ocamlyacc Stephen A. Edwards Columbia University Fall 2012 Parsing and Syntax Trees Parsing decides if the program is part of the language. Not that useful: we want more than
More informationCOP4020 Programming Languages. Syntax Prof. Robert van Engelen
COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview Tokens and regular expressions Syntax and context-free grammars Grammar derivations More about parse trees Top-down and bottom-up
More informationCS 315 Programming Languages Syntax. Parser. (Alternatively hand-built) (Alternatively hand-built)
Programming languages must be precise Remember instructions This is unlike natural languages CS 315 Programming Languages Syntax Precision is required for syntax think of this as the format of the language
More informationProperties of Regular Expressions and Finite Automata
Properties of Regular Expressions and Finite Automata Some token patterns can t be defined as regular expressions or finite automata. Consider the set of balanced brackets of the form [[[ ]]]. This set
More informationMP 3 A Lexer for MiniJava
MP 3 A Lexer for MiniJava CS 421 Spring 2012 Revision 1.0 Assigned Wednesday, February 1, 2012 Due Tuesday, February 7, at 09:30 Extension 48 hours (penalty 20% of total points possible) Total points 43
More informationOutline. 1 Scanning Tokens. 2 Regular Expresssions. 3 Finite State Automata
Outline 1 2 Regular Expresssions Lexical Analysis 3 Finite State Automata 4 Non-deterministic (NFA) Versus Deterministic Finite State Automata (DFA) 5 Regular Expresssions to NFA 6 NFA to DFA 7 8 JavaCC:
More informationA clarification on terminology: Recognizer: accepts or rejects strings in a language. Parser: recognizes and generates parse trees (imminent topic)
A clarification on terminology: Recognizer: accepts or rejects strings in a language Parser: recognizes and generates parse trees (imminent topic) Assignment 3: building a recognizer for the Lake expression
More informationCourse Overview. Introduction (Chapter 1) Compiler Frontend: Today. Compiler Backend:
Course Overview Introduction (Chapter 1) Compiler Frontend: Today Lexical Analysis & Parsing (Chapter 2,3,4) Semantic Analysis (Chapter 5) Activation Records (Chapter 6) Translation to Intermediate Code
More informationEDA180: Compiler Construc6on. Top- down parsing. Görel Hedin Revised: a
EDA180: Compiler Construc6on Top- down parsing Görel Hedin Revised: 2013-01- 30a Compiler phases and program representa6ons source code Lexical analysis (scanning) Intermediate code genera6on tokens intermediate
More informationLexical Considerations
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.035, Fall 2005 Handout 6 Decaf Language Wednesday, September 7 The project for the course is to write a
More informationTML Language Reference Manual
TML Language Reference Manual Jiabin Hu (jh3240) Akash Sharma (as4122) Shuai Sun (ss4088) Yan Zou (yz2437) Columbia University October 31, 2011 1 Contents 1 Introduction 4 2 Lexical Conventions 4 2.1 Character
More informationBuilding lexical and syntactic analyzers. Chapter 3. Syntactic sugar causes cancer of the semicolon. A. Perlis. Chomsky Hierarchy
Building lexical and syntactic analyzers Chapter 3 Syntactic sugar causes cancer of the semicolon. A. Perlis Chomsky Hierarchy Four classes of grammars, from simplest to most complex: Regular grammar What
More informationMP 3 A Lexer for MiniJava
MP 3 A Lexer for MiniJava CS 421 Spring 2010 Revision 1.0 Assigned Tuesday, February 2, 2010 Due Monday, February 8, at 10:00pm Extension 48 hours (20% penalty) Total points 50 (+5 extra credit) 1 Change
More informationCompilers CS S-01 Compiler Basics & Lexical Analysis
Compilers CS414-2005S-01 Compiler Basics & Lexical Analysis David Galles Department of Computer Science University of San Francisco 01-0: Syllabus Office Hours Course Text Prerequisites Test Dates & Testing
More informationParser Tools: lex and yacc-style Parsing
Parser Tools: lex and yacc-style Parsing Version 5.0 Scott Owens June 6, 2010 This documentation assumes familiarity with lex and yacc style lexer and parser generators. 1 Contents 1 Lexers 3 1.1 Creating
More informationCOP4020 Programming Languages. Syntax Prof. Robert van Engelen
COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview n Tokens and regular expressions n Syntax and context-free grammars n Grammar derivations n More about parse trees n Top-down and
More informationDefining syntax using CFGs
Defining syntax using CFGs Roadmap Last time Defined context-free grammar This time CFGs for specifying a language s syntax Language membership List grammars Resolving ambiguity CFG Review G = (N,Σ,P,S)
More informationCS664 Compiler Theory and Design LIU 1 of 16 ANTLR. Christopher League* 17 February Figure 1: ANTLR plugin installer
CS664 Compiler Theory and Design LIU 1 of 16 ANTLR Christopher League* 17 February 2016 ANTLR is a parser generator. There are other similar tools, such as yacc, flex, bison, etc. We ll be using ANTLR
More informationCSE450 Translation of Programming Languages. Lecture 4: Syntax Analysis
CSE450 Translation of Programming Languages Lecture 4: Syntax Analysis http://xkcd.com/859 Structure of a Today! Compiler Source Language Lexical Analyzer Syntax Analyzer Semantic Analyzer Int. Code Generator
More informationIt parses an input string of tokens by tracing out the steps in a leftmost derivation.
It parses an input string of tokens by tracing out CS 4203 Compiler Theory the steps in a leftmost derivation. CHAPTER 4: TOP-DOWN PARSING Part1 And the implied traversal of the parse tree is a preorder
More informationIntroduction to Parsing. Lecture 8
Introduction to Parsing Lecture 8 Adapted from slides by G. Necula Outline Limitations of regular languages Parser overview Context-free grammars (CFG s) Derivations Languages and Automata Formal languages
More informationA simple syntax-directed
Syntax-directed is a grammaroriented compiling technique Programming languages: Syntax: what its programs look like? Semantic: what its programs mean? 1 A simple syntax-directed Lexical Syntax Character
More informationCompilers CS S-01 Compiler Basics & Lexical Analysis
Compilers CS414-2017S-01 Compiler Basics & Lexical Analysis David Galles Department of Computer Science University of San Francisco 01-0: Syllabus Office Hours Course Text Prerequisites Test Dates & Testing
More informationSyntax Intro and Overview. Syntax
Syntax Intro and Overview CS331 Syntax Syntax defines what is grammatically valid in a programming language Set of grammatical rules E.g. in English, a sentence cannot begin with a period Must be formal
More informationCSE 413 Final Exam. June 7, 2011
CSE 413 Final Exam June 7, 2011 Name The exam is closed book, except that you may have a single page of hand-written notes for reference plus the page of notes you had for the midterm (although you are
More informationLexical Considerations
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.035, Spring 2010 Handout Decaf Language Tuesday, Feb 2 The project for the course is to write a compiler
More informationChapter 3. Describing Syntax and Semantics ISBN
Chapter 3 Describing Syntax and Semantics ISBN 0-321-49362-1 Chapter 3 Topics Introduction The General Problem of Describing Syntax Formal Methods of Describing Syntax Copyright 2009 Addison-Wesley. All
More informationCSE 413 Final Exam Spring 2011 Sample Solution. Strings of alternating 0 s and 1 s that begin and end with the same character, either 0 or 1.
Question 1. (10 points) Regular expressions I. Describe the set of strings generated by each of the following regular expressions. For full credit, give a description of the sets like all sets of strings
More informationEECS 6083 Intro to Parsing Context Free Grammars
EECS 6083 Intro to Parsing Context Free Grammars Based on slides from text web site: Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. 1 Parsing sequence of tokens parser
More informationChapter 4. Abstract Syntax
Chapter 4 Abstract Syntax Outline compiler must do more than recognize whether a sentence belongs to the language of a grammar it must do something useful with that sentence. The semantic actions of a
More informationCOP4020 Programming Assignment 2 Spring 2011
COP4020 Programming Assignment 2 Spring 2011 Consider our familiar augmented LL(1) grammar for an expression language (see Syntax lecture notes on the LL(1) expression grammar): ->
More informationParsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones
Parsing III (Top-down parsing: recursive descent & LL(1) ) (Bottom-up parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,
More informationPart II : Lexical Analysis
Part II : Lexical Analysis Regular Languages Translation from regular languages to program code A grammar for JO Context-free Grammar of JO Assignment 1 Martin Odersky, LAMP/DI 1 Regular Languages Definition
More informationSML-SYNTAX-LANGUAGE INTERPRETER IN JAVA. Jiahao Yuan Supervisor: Dr. Vijay Gehlot
SML-SYNTAX-LANGUAGE INTERPRETER IN JAVA Jiahao Yuan Supervisor: Dr. Vijay Gehlot Target Design SML-Like-Syntax Build Parser in ANTLR Abstract Syntax Tree Representation ANLTER Integration In Interpreter
More informationDefining Program Syntax. Chapter Two Modern Programming Languages, 2nd ed. 1
Defining Program Syntax Chapter Two Modern Programming Languages, 2nd ed. 1 Syntax And Semantics Programming language syntax: how programs look, their form and structure Syntax is defined using a kind
More informationContext-Free Grammar. Concepts Introduced in Chapter 2. Parse Trees. Example Grammar and Derivation
Concepts Introduced in Chapter 2 A more detailed overview of the compilation process. Parsing Scanning Semantic Analysis Syntax-Directed Translation Intermediate Code Generation Context-Free Grammar A
More informationCS S-01 Compiler Basics & Lexical Analysis 1
CS414-2017S-01 Compiler Basics & Lexical Analysis 1 01-0: Syllabus Office Hours Course Text Prerequisites Test Dates & Testing Policies Projects Teams of up to 2 Grading Policies Questions? 01-1: Notes
More informationLECTURE 3. Compiler Phases
LECTURE 3 Compiler Phases COMPILER PHASES Compilation of a program proceeds through a fixed series of phases. Each phase uses an (intermediate) form of the program produced by an earlier phase. Subsequent
More informationLECTURE 7. Lex and Intro to Parsing
LECTURE 7 Lex and Intro to Parsing LEX Last lecture, we learned a little bit about how we can take our regular expressions (which specify our valid tokens) and create real programs that can recognize them.
More informationOutline. Limitations of regular languages. Introduction to Parsing. Parser overview. Context-free grammars (CFG s)
Outline Limitations of regular languages Introduction to Parsing Parser overview Lecture 8 Adapted from slides by G. Necula Context-free grammars (CFG s) Derivations Languages and Automata Formal languages
More informationParsing #1. Leonidas Fegaras. CSE 5317/4305 L3: Parsing #1 1
Parsing #1 Leonidas Fegaras CSE 5317/4305 L3: Parsing #1 1 Parser source file get next character scanner get token parser AST token A parser recognizes sequences of tokens according to some grammar and
More informationLL(k) Compiler Construction. Choice points in EBNF grammar. Left recursive grammar
LL(k) Compiler Construction More LL parsing Abstract syntax trees Lennart Andersson Revision 2012 01 31 2012 Related names top-down the parse tree is constructed top-down recursive descent if it is implemented
More informationLexical Analysis. Textbook:Modern Compiler Design Chapter 2.1
Lexical Analysis Textbook:Modern Compiler Design Chapter 2.1 A motivating example Create a program that counts the number of lines in a given input text file Solution (Flex) int num_lines = 0; %% \n ++num_lines;.
More informationCOP4020 Programming Assignment 2 - Fall 2016
COP4020 Programming Assignment 2 - Fall 2016 To goal of this project is to implement in C or C++ (your choice) an interpreter that evaluates arithmetic expressions with variables in local scopes. The local
More informationCS 230 Programming Languages
CS 230 Programming Languages 09 / 16 / 2013 Instructor: Michael Eckmann Today s Topics Questions/comments? Continue Syntax & Semantics Mini-pascal Attribute Grammars More Perl A more complex grammar Let's
More informationLexical Analysis 1 / 52
Lexical Analysis 1 / 52 Outline 1 Scanning Tokens 2 Regular Expresssions 3 Finite State Automata 4 Non-deterministic (NFA) Versus Deterministic Finite State Automata (DFA) 5 Regular Expresssions to NFA
More informationCS Parsing 1
CS414-20034-03 Parsing 1 03-0: Parsing Once we have broken an input file into a sequence of tokens, the next step is to determine if that sequence of tokens forms a syntactically correct program parsing
More informationContext-Free Grammar (CFG)
Context-Free Grammar (CFG) context-free grammar looks like this bunch of rules: ain idea: + 1 (),, are non-terminal symbols aka variables. When you see them, you apply rules to expand. One of them is designated
More informationCSC 4181 Compiler Construction. Parsing. Outline. Introduction
CC 4181 Compiler Construction Parsing 1 Outline Top-down v.s. Bottom-up Top-down parsing Recursive-descent parsing LL1) parsing LL1) parsing algorithm First and follow sets Constructing LL1) parsing table
More informationSyntax. Syntax. We will study three levels of syntax Lexical Defines the rules for tokens: literals, identifiers, etc.
Syntax Syntax Syntax defines what is grammatically valid in a programming language Set of grammatical rules E.g. in English, a sentence cannot begin with a period Must be formal and exact or there will
More informationParser. Larissa von Witte. 11. Januar Institut für Softwaretechnik und Programmiersprachen. L. v. Witte 11. Januar /23
Parser Larissa von Witte Institut für oftwaretechnik und Programmiersprachen 11. Januar 2016 L. v. Witte 11. Januar 2016 1/23 Contents Introduction Taxonomy Recursive Descent Parser hift Reduce Parser
More informationAbstract Syntax Trees & Top-Down Parsing
Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree
More informationCSEP 501 Compilers. Languages, Automata, Regular Expressions & Scanners Hal Perkins Winter /8/ Hal Perkins & UW CSE B-1
CSEP 501 Compilers Languages, Automata, Regular Expressions & Scanners Hal Perkins Winter 2008 1/8/2008 2002-08 Hal Perkins & UW CSE B-1 Agenda Basic concepts of formal grammars (review) Regular expressions
More informationIntroduction to Compiler Design
Introduction to Compiler Design Lecture 1 Chapters 1 and 2 Robb T. Koether Hampden-Sydney College Wed, Jan 14, 2015 Robb T. Koether (Hampden-Sydney College) Introduction to Compiler Design Wed, Jan 14,
More informationCS323 Lecture - Specifying Syntax and Semantics Last revised 1/16/09
CS323 Lecture - Specifying Syntax and Semantics Last revised 1/16/09 Objectives: 1. To review previously-studied methods for formal specification of programming language syntax, and introduce additional
More informationCOMPILER (CSE 4120) (Lecture 6: Parsing 4 Bottom-up Parsing )
COMPILR (CS 4120) (Lecture 6: Parsing 4 Bottom-up Parsing ) Sungwon Jung Mobile Computing & Data ngineering Lab Dept. of Computer Science and ngineering Sogang University Seoul, Korea Tel: +82-2-705-8930
More information