# CIT Lecture 5 Context-Free Grammars and Parsing 4/2/2003 1

Size: px
Start display at page:

Transcription

1 CIT Lecture 5 Context-Free Grammars and Parsing 4/2/2003 1

2 Definition of a Context-free Grammar: An alphabet or set of basic symbols (like regular expressions, only now the symbols are whole tokens, not chars), including ε. (Terminals) A set of names for structures (like statement, expression, definition). (Non-terminals) A set of grammar rules expressing the structure of each name. (Productions) A start symbol (the name of the most general structure compilation_unit in C). 4/2/2003 2

3 Basic Example: Simple integer arithmetic expressions 2 non-terminals exp exp op exp ( exp ) number op + - * 6 terminals In what way does such a CFG differ from a regular expression? digit = number = digit digit* Recursion! Recursive rules 6 productions (3 on each line) Base rule 4/2/2003 3

4 CFGs are designed to represent recursive (i.e. nested) structures But consequences are huge: The structure of a matched string is no longer given by just a sequence of symbols (lexeme), but by a tree (parse tree) Recognizers are no longer finite, but may have arbitrary data size, and must have some notion of stack. 4/2/2003 4

5 Recognition Process is much more complex: Algorithms can use stacks in many different ways. Nondeterminism is much harder to eliminate. Even the number of states can vary with the algorithm (only 2 states necessary if stack is used for state structure. 4/2/2003 5

6 Major Consequence: Many parsing algorithms, not just one Top down Recursive descent (hand choice) Predictive table-driven, LL (outdated) Bottom up LR and its cousin LALR (machinegenerated choice [Yacc/Bison]) Operator-precedence (outdated) 4/2/2003 6

7 Structural Issues First! Express matching of a string [ (34-3)*42 ] by a derivation: (1) exp expopexp [exp expopexp] (2) exp op number [exp number] (3) exp * number [op * ] (4) ( exp ) *number [exp ( exp )] (5) ( exp op exp ) *number [exp expopexp] (6) (exp op number) *number [exp number ] (7) (exp - number) *number [op - ] (8) (number - number)*number [exp number ] 4/2/2003 7

8 Abstract the structure of a derivation to a parse tree: 1 exp 4 exp 3 op 2 exp ( 5 exp ) * number 8 exp 7 op 6 exp number - number 4/2/2003 8

9 Derivations can vary, even when the parse tree doesn t: A leftmost derivation (Slide 8 was a rightmost): (1) exp exp op exp [exp exp op exp] (2) (exp) op exp [exp ( exp )] (3) (exp op exp) op exp [exp exp op exp] (4) (number op exp) op exp [exp number] (5) (number - exp) op exp [op -] (6) (number - number) op exp [exp number] (7) (number - number) * exp [op *] (8) (number - number) * number [exp number] 4/2/2003 9

10 A leftmost derivation corresponds to a (top-down) preorder traversal of the parse tree. A rightmost derivation corresponds to a (bottom-up) postorder traversal, but in reverse. Top-down parsers construct leftmost derivations. (LL = Left-to-right traversal of input, constructing a Leftmost derivation) Bottom-up parsers construct rightmost derivations in reverse order. (LR = Left-to-right traversal of input, constructing a Rightmost derivation) 4/2/

11 But what if the parse tree does vary?[ exp op exp op exp ] Correct one exp exp exp op exp exp op exp exp op exp * number number - exp op exp number - number number * number The grammar is ambiguous, but why should we care? Semantics! 4/2/

12 Principle of Syntax-directed Semantics The parse tree will be used as the basic model; semantic content will be attached to the tree; thus the tree should reflect the structure of the eventual semantics (semanticsbased syntax would be a better term) 4/2/

13 Sources of Ambiguity: Associativity and precedence of operators Sequencing Extent of a substructure (dangling else) Obscure recursion (unusual) exp exp exp 4/2/

14 Dealing with ambiguity Disambiguating rules Change the grammar (but not the language!) Can all ambiguity be removed? Backtracking can handle it, but the expense is great 4/2/

15 Example: integer arithmetic exp exp addop term term addop + - term term mulop factor factor mulop * factor ( exp ) number Precedence cascade 4/2/

16 Repetition and Recursion Left recursion: A A x y yxx: A A x A x y Right recursion: A x A y xxy: A x A x A 4/2/ y

17 Repetition & Recursion, cont. Sometimes we care which way recursion goes: operator associativity Sometimes we don t: statement and expression sequences Parsing always has to pick a way! The tree may remove this information (see next slide) 4/2/

18 Abstract Syntax Trees Express the essential structure of the parse tree only Leave out parens, cascades, and don t-care repetitive associativity Corresponds to actual internal tree structure produced by parser Use sibling lists for don t care repetition: s1 --- s2 --- s3 4/2/

19 Previous Example [ (34-3)*42 3)*42 ] * /2/

20 Data Structure typedef enum {Plus,Minus,Times} OpKind; typedef enum {OpK,ConstK} ExpKind; typedef struct streenode { ExpKind kind; OpKind op; struct streenode *lchild,*rchild; int val; } STreeNode; typedef STreeNode *SyntaxTree; 4/2/

21 Or (using a union): typedef enum {Plus,Minus,Times} OpKind; typedef enum {OpK,ConstK} ExpKind; typedef struct streenode { ExpKind kind; struct streenode *lchild,*rchild; union { OpKind op; int val; } attribute; } STreeNode; typedef STreeNode *SyntaxTree; 4/2/

22 Sequence Examples stmt-seq stmt ; stmt-seq stmt one or more stmts separated by a ; stmt-seq stmt ; stmt-seq ε zero or more stmts terminated by a ; stmt-seq stmt-seq ; stmt stmt one or more stmts separated by a ; stmt-seq stmt-seq ; stmt ε zero or more stmts preceded by a ; 4/2/

23 Obscure Ambiguity Example Incorrect attempt to add unary minus: exp exp addop term term - exp addop + - term term mulop factor factor mulop * factor ( exp ) number 4/2/

24 Ambiguity Example, continued Better: (only one at beg. of an exp) exp exp addop term term - term Or maybe: (many at beg. of term) term - term term1 term1 term1 mulop factor factor Or maybe: (many anywhere) factor ( exp ) number - factor 4/2/

25 Dangling else ambiguity statement if-stmt other if-stmt if ( exp ) statement if ( exp )statement else statement exp 0 1 The following string has two parse trees: if(0) if(1) other else other 4/2/

26 Parse trees for dangling else: statement statement Correct one if-stmt if-stmt if ( ) else exp statement statement if ( ) exp statement 0 other if-stmt 0 if-stmt if ( ) exp statement if ( ) else exp statement statement 1 other 1 other other 4/2/

27 Disambiguating Rule: An else part should always be associated with the nearest if-statement that does not yet have an associated else-part. (Most-closely nested rule: easy to state, but hard to put into the grammar itself.) Note that a bracketing keyword can remove the ambiguity: Bracketing keyword if-stmt if ( exp ) stmt end if ( exp )stmt else stmt end 4/2/

28 Extra Notation: So far: Backus-Naur Form (BNF) Metasymbols are ε Extended BNF (EBNF): New metasymbols [ ] and { } ε largely eliminated by these Parens? Maybe yes, maybe no: exp exp (+ -) term term exp exp + term exp - term term 4/2/

29 EBNF Metasymbols: Brackets [ ] mean optional (like? in regular expressions): exp term exp term becomes: exp term [ exp ] if-stmt if ( exp ) stmt if ( exp )stmt else stmt becomes: if-stmt if ( exp ) stmt [ else stmt ] Braces { } mean repetition (like * in regexps - see next slide) 4/2/

30 Braces in EBNF Replace only left-recursive repetition: exp exp + term term becomes: exp term { + term } Left associativity still implied Watch out for choices: exp exp + term exp - term term is not the same as exp term { + term } term { - term } 4/2/

31 Simple Expressions in EBNF exp term { addop term } addop + - term factor { mulop factor } mulop * factor ( exp ) number 4/2/

32 Final Notational Option: Syntax Diagrams (from EBNF): exp > term < addop < > ( > exp > ) factor > > number 4/2/

### Dr. D.M. Akbar Hussain

Syntax Analysis Parsing Syntax Or Structure Given By Determines Grammar Rules Context Free Grammar 1 Context Free Grammars (CFG) Provides the syntactic structure: A grammar is quadruple (V T, V N, S, R)

### Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part2 3.3 Parse Trees and Abstract Syntax Trees

Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part2 3.3 Parse Trees and Abstract Syntax Trees 3.3.1 Parse trees 1. Derivation V.S. Structure Derivations do not uniquely represent the structure of the strings

### 3. Context-free grammars & parsing

3. Context-free grammars & parsing The parsing process sequences of tokens parse tree or syntax tree a / [ / index / ]/= / 4 / + / 2 The parsing process sequences of tokens parse tree or syntax tree a

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview Tokens and regular expressions Syntax and context-free grammars Grammar derivations More about parse trees Top-down and bottom-up

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview n Tokens and regular expressions n Syntax and context-free grammars n Grammar derivations n More about parse trees n Top-down and

### EECS 6083 Intro to Parsing Context Free Grammars

EECS 6083 Intro to Parsing Context Free Grammars Based on slides from text web site: Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. 1 Parsing sequence of tokens parser

### COP 3402 Systems Software Top Down Parsing (Recursive Descent)

COP 3402 Systems Software Top Down Parsing (Recursive Descent) Top Down Parsing 1 Outline 1. Top down parsing and LL(k) parsing 2. Recursive descent parsing 3. Example of recursive descent parsing of arithmetic

### CSE 3302 Programming Languages Lecture 2: Syntax

CSE 3302 Programming Languages Lecture 2: Syntax (based on slides by Chengkai Li) Leonidas Fegaras University of Texas at Arlington CSE 3302 L2 Spring 2011 1 How do we define a PL? Specifying a PL: Syntax:

### CPS 506 Comparative Programming Languages. Syntax Specification

CPS 506 Comparative Programming Languages Syntax Specification Compiling Process Steps Program Lexical Analysis Convert characters into a stream of tokens Lexical Analysis Syntactic Analysis Send tokens

### Defining syntax using CFGs

Defining syntax using CFGs Roadmap Last time Defined context-free grammar This time CFGs for specifying a language s syntax Language membership List grammars Resolving ambiguity CFG Review G = (N,Σ,P,S)

### Theoretical Part. Chapter one:- - What are the Phases of compiler? Answer:

Theoretical Part Chapter one:- - What are the Phases of compiler? Six phases Scanner Parser Semantic Analyzer Source code optimizer Code generator Target Code Optimizer Three auxiliary components Literal

### It parses an input string of tokens by tracing out the steps in a leftmost derivation.

It parses an input string of tokens by tracing out CS 4203 Compiler Theory the steps in a leftmost derivation. CHAPTER 4: TOP-DOWN PARSING Part1 And the implied traversal of the parse tree is a preorder

### Syntax. In Text: Chapter 3

Syntax In Text: Chapter 3 1 Outline Syntax: Recognizer vs. generator BNF EBNF Chapter 3: Syntax and Semantics 2 Basic Definitions Syntax the form or structure of the expressions, statements, and program

### Syntax Analysis. COMP 524: Programming Language Concepts Björn B. Brandenburg. The University of North Carolina at Chapel Hill

Syntax Analysis Björn B. Brandenburg The University of North Carolina at Chapel Hill Based on slides and notes by S. Olivier, A. Block, N. Fisher, F. Hernandez-Campos, and D. Stotts. The Big Picture Character

### Syntax Analysis. The Big Picture. The Big Picture. COMP 524: Programming Languages Srinivas Krishnan January 25, 2011

Syntax Analysis COMP 524: Programming Languages Srinivas Krishnan January 25, 2011 Based in part on slides and notes by Bjoern Brandenburg, S. Olivier and A. Block. 1 The Big Picture Character Stream Token

### Compiler Principle and Technology. Prof. Dongming LU April 15th, 2019

Compiler Principle and Technology Prof. Dongming LU April 15th, 2019 PART TWO 6. Semantic Analysis Contents Part One 6.1 Attributes and Attribute Grammars Part Two 6.2 Algorithms for Attribute Computation

### Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part 1

Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part 1 1. Introduction Parsing is the task of Syntax Analysis Determining the syntax, or structure, of a program. The syntax is defined by the grammar rules

### Chapter 3. Describing Syntax and Semantics ISBN

Chapter 3 Describing Syntax and Semantics ISBN 0-321-49362-1 Chapter 3 Topics Introduction The General Problem of Describing Syntax Formal Methods of Describing Syntax Copyright 2009 Addison-Wesley. All

### flex is not a bad tool to use for doing modest text transformations and for programs that collect statistics on input.

flex is not a bad tool to use for doing modest text transformations and for programs that collect statistics on input. More often than not, though, you ll want to use flex to generate a scanner that divides

### Lexical and Syntax Analysis. Top-Down Parsing

Lexical and Syntax Analysis Top-Down Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax

### CS415 Compilers. Syntax Analysis. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University

CS415 Compilers Syntax Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Limits of Regular Languages Advantages of Regular Expressions

### A programming language requires two major definitions A simple one pass compiler

A programming language requires two major definitions A simple one pass compiler [Syntax: what the language looks like A context-free grammar written in BNF (Backus-Naur Form) usually suffices. [Semantics:

### CIT 3136 Lecture 7. Top-Down Parsing

CIT 3136 Lecture 7 Top-Down Parsing Chapter 4: Top-down Parsing A top-down parsing algorithm parses an input string of tokens by tracing out the steps in a leftmost derivation. Such an algorithm is called

### Parsing: Derivations, Ambiguity, Precedence, Associativity. Lecture 8. Professor Alex Aiken Lecture #5 (Modified by Professor Vijay Ganesh)

Parsing: Derivations, Ambiguity, Precedence, Associativity Lecture 8 (Modified by Professor Vijay Ganesh) 1 Topics covered so far Regular languages and Finite automaton Parser overview Context-free grammars

### Introduction to Parsing

Introduction to Parsing The Front End Source code Scanner tokens Parser IR Errors Parser Checks the stream of words and their parts of speech (produced by the scanner) for grammatical correctness Determines

### MIT Specifying Languages with Regular Expressions and Context-Free Grammars

MIT 6.035 Specifying Languages with Regular essions and Context-Free Grammars Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Language Definition Problem How to precisely

### Habanero Extreme Scale Software Research Project

Habanero Extreme Scale Software Research Project Comp215: Grammars Zoran Budimlić (Rice University) Grammar, which knows how to control even kings - Moliere So you know everything about regular expressions

### Building Compilers with Phoenix

Building Compilers with Phoenix Syntax-Directed Translation Structure of a Compiler Character Stream Intermediate Representation Lexical Analyzer Machine-Independent Optimizer token stream Intermediate

### Principles of Programming Languages COMP251: Syntax and Grammars

Principles of Programming Languages COMP251: Syntax and Grammars Prof. Dekai Wu Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China Fall 2006

### Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

### A Simple Syntax-Directed Translator

Chapter 2 A Simple Syntax-Directed Translator 1-1 Introduction The analysis phase of a compiler breaks up a source program into constituent pieces and produces an internal representation for it, called

### CS 315 Programming Languages Syntax. Parser. (Alternatively hand-built) (Alternatively hand-built)

Programming languages must be precise Remember instructions This is unlike natural languages CS 315 Programming Languages Syntax Precision is required for syntax think of this as the format of the language

### A simple syntax-directed

Syntax-directed is a grammaroriented compiling technique Programming languages: Syntax: what its programs look like? Semantic: what its programs mean? 1 A simple syntax-directed Lexical Syntax Character

### Lexical and Syntax Analysis

Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Easy for humans to write and understand String of characters

### MIT Specifying Languages with Regular Expressions and Context-Free Grammars. Martin Rinard Massachusetts Institute of Technology

MIT 6.035 Specifying Languages with Regular essions and Context-Free Grammars Martin Rinard Massachusetts Institute of Technology Language Definition Problem How to precisely define language Layered structure

### CMPS Programming Languages. Dr. Chengwei Lei CEECS California State University, Bakersfield

CMPS 3500 Programming Languages Dr. Chengwei Lei CEECS California State University, Bakersfield Chapter 3 Describing Syntax and Semantics Chapter 3 Topics Introduction The General Problem of Describing

### Syntax. A. Bellaachia Page: 1

Syntax 1. Objectives & Definitions... 2 2. Definitions... 3 3. Lexical Rules... 4 4. BNF: Formal Syntactic rules... 6 5. Syntax Diagrams... 9 6. EBNF: Extended BNF... 10 7. Example:... 11 8. BNF Statement

### Parsing. source code. while (k<=n) {sum = sum+k; k=k+1;}

Compiler Construction Grammars Parsing source code scanner tokens regular expressions lexical analysis Lennart Andersson parser context free grammar Revision 2012 01 23 2012 parse tree AST builder (implicit)

### Describing Syntax and Semantics

Describing Syntax and Semantics Introduction Syntax: the form or structure of the expressions, statements, and program units Semantics: the meaning of the expressions, statements, and program units Syntax

### Recursive Descent Parsers

Recursive Descent Parsers Lecture 7 Robb T. Koether Hampden-Sydney College Wed, Jan 28, 2015 Robb T. Koether (Hampden-Sydney College) Recursive Descent Parsers Wed, Jan 28, 2015 1 / 18 1 Parsing 2 LL Parsers

### CSE P 501 Compilers. Parsing & Context-Free Grammars Hal Perkins Spring UW CSE P 501 Spring 2018 C-1

CSE P 501 Compilers Parsing & Context-Free Grammars Hal Perkins Spring 2018 UW CSE P 501 Spring 2018 C-1 Administrivia Project partner signup: please find a partner and fill out the signup form by noon

### Lecture 10 Parsing 10.1

10.1 The next two lectures cover parsing. To parse a sentence in a formal language is to break it down into its syntactic components. Parsing is one of the most basic functions every compiler carries out,

### Architecture of Compilers, Interpreters. CMSC 330: Organization of Programming Languages. Front End Scanner and Parser. Implementing the Front End

Architecture of Compilers, Interpreters : Organization of Programming Languages ource Analyzer Optimizer Code Generator Context Free Grammars Intermediate Representation Front End Back End Compiler / Interpreter

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Context Free Grammars and Parsing 1 Recall: Architecture of Compilers, Interpreters Source Parser Static Analyzer Intermediate Representation Front End Back

### Programming Language Syntax and Analysis

Programming Language Syntax and Analysis 2017 Kwangman Ko (http://compiler.sangji.ac.kr, kkman@sangji.ac.kr) Dept. of Computer Engineering, Sangji University Introduction Syntax the form or structure of

### Syntax. Syntax. We will study three levels of syntax Lexical Defines the rules for tokens: literals, identifiers, etc.

Syntax Syntax Syntax defines what is grammatically valid in a programming language Set of grammatical rules E.g. in English, a sentence cannot begin with a period Must be formal and exact or there will

### Part 5 Program Analysis Principles and Techniques

1 Part 5 Program Analysis Principles and Techniques Front end 2 source code scanner tokens parser il errors Responsibilities: Recognize legal programs Report errors Produce il Preliminary storage map Shape

### Syntax Analysis/Parsing. Context-free grammars (CFG s) Context-free grammars vs. Regular Expressions. BNF description of PL/0 syntax

Susan Eggers 1 CSE 401 Syntax Analysis/Parsing Context-free grammars (CFG s) Purpose: determine if tokens have the right form for the language (right syntactic structure) stream of tokens abstract syntax

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### EDAN65: Compilers, Lecture 04 Grammar transformations: Eliminating ambiguities, adapting to LL parsing. Görel Hedin Revised:

EDAN65: Compilers, Lecture 04 Grammar transformations: Eliminating ambiguities, adapting to LL parsing Görel Hedin Revised: 2017-09-04 This lecture Regular expressions Context-free grammar Attribute grammar

### Context-free grammars (CFG s)

Syntax Analysis/Parsing Purpose: determine if tokens have the right form for the language (right syntactic structure) stream of tokens abstract syntax tree (AST) AST: captures hierarchical structure of

### programming languages need to be precise a regular expression is one of the following: tokens are the building blocks of programs

Chapter 2 :: Programming Language Syntax Programming Language Pragmatics Michael L. Scott Introduction programming languages need to be precise natural languages less so both form (syntax) and meaning

### CS 314 Principles of Programming Languages

CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution

### CSE 130 Programming Language Principles & Paradigms Lecture # 5. Chapter 4 Lexical and Syntax Analysis

Chapter 4 Lexical and Syntax Analysis Introduction - Language implementation systems must analyze source code, regardless of the specific implementation approach - Nearly all syntax analysis is based on

### Lecture 12: Parser-Generating Tools

Lecture 12: Parser-Generating Tools Dr Kieran T. Herley Department of Computer Science University College Cork 2017-2018 KH (31/10/17) Lecture 12: Parser-Generating Tools 2017-2018 1 / 27 Summary Overview

### COP 3402 Systems Software Syntax Analysis (Parser)

COP 3402 Systems Software Syntax Analysis (Parser) Syntax Analysis 1 Outline 1. Definition of Parsing 2. Context Free Grammars 3. Ambiguous/Unambiguous Grammars Syntax Analysis 2 Lexical and Syntax Analysis

### Lecture 4: Syntax Specification

The University of North Carolina at Chapel Hill Spring 2002 Lecture 4: Syntax Specification Jan 16 1 Phases of Compilation 2 1 Syntax Analysis Syntax: Webster s definition: 1 a : the way in which linguistic

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### Chapter 4. Syntax - the form or structure of the expressions, statements, and program units

Syntax - the form or structure of the expressions, statements, and program units Semantics - the meaning of the expressions, statements, and program units Who must use language definitions? 1. Other language

### CMSC 330: Organization of Programming Languages. Architecture of Compilers, Interpreters

: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Scanner Parser Static Analyzer Intermediate Representation Front End Back End Compiler / Interpreter

### Compilers. Yannis Smaragdakis, U. Athens (original slides by Sam

Compilers Parsing Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Next step text chars Lexical analyzer tokens Parser IR Errors Parsing: Organize tokens into sentences Do tokens conform

### Parsing II Top-down parsing. Comp 412

COMP 412 FALL 2018 Parsing II Top-down parsing Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled

### Introduction to Parsing. Lecture 5

Introduction to Parsing Lecture 5 1 Outline Regular languages revisited Parser overview Context-free grammars (CFG s) Derivations Ambiguity 2 Languages and Automata Formal languages are very important

### EDAN65: Compilers, Lecture 06 A LR parsing. Görel Hedin Revised:

EDAN65: Compilers, Lecture 06 A LR parsing Görel Hedin Revised: 2017-09-11 This lecture Regular expressions Context-free grammar Attribute grammar Lexical analyzer (scanner) Syntactic analyzer (parser)

### Introduction to Parsing. Lecture 5

Introduction to Parsing Lecture 5 1 Outline Regular languages revisited Parser overview Context-free grammars (CFG s) Derivations Ambiguity 2 Languages and Automata Formal languages are very important

### CMSC 330: Organization of Programming Languages. Context Free Grammars

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### ECE251 Midterm practice questions, Fall 2010

ECE251 Midterm practice questions, Fall 2010 Patrick Lam October 20, 2010 Bootstrapping In particular, say you have a compiler from C to Pascal which runs on x86, and you want to write a self-hosting Java

### 3. Parsing. Oscar Nierstrasz

3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/

### Building a Parser II. CS164 3:30-5:00 TT 10 Evans. Prof. Bodik CS 164 Lecture 6 1

Building a Parser II CS164 3:30-5:00 TT 10 Evans 1 Grammars Programming language constructs have recursive structure. which is why our hand-written parser had this structure, too An expression is either:

SYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic

### CSE 311 Lecture 21: Context-Free Grammars. Emina Torlak and Kevin Zatloukal

CSE 311 Lecture 21: Context-Free Grammars Emina Torlak and Kevin Zatloukal 1 Topics Regular expressions A brief review of Lecture 20. Context-free grammars Syntax, semantics, and examples. 2 Regular expressions

### CMSC 330: Organization of Programming Languages. Context Free Grammars

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### Lexical and Syntax Analysis (2)

Lexical and Syntax Analysis (2) In Text: Chapter 4 N. Meng, F. Poursardar Motivating Example Consider the grammar S -> cad A -> ab a Input string: w = cad How to build a parse tree top-down? 2 Recursive-Descent

### Introduction to Parsing. Lecture 8

Introduction to Parsing Lecture 8 Adapted from slides by G. Necula Outline Limitations of regular languages Parser overview Context-free grammars (CFG s) Derivations Languages and Automata Formal languages

### F453 Module 7: Programming Techniques. 7.2: Methods for defining syntax

7.2: Methods for defining syntax 2 What this module is about In this module we discuss: explain how functions, procedures and their related variables may be used to develop a program in a structured way,

### Homework & Announcements

Homework & nnouncements New schedule on line. Reading: Chapter 18 Homework: Exercises at end Due: 11/1 Copyright c 2002 2017 UMaine School of Computing and Information S 1 / 25 COS 140: Foundations of

### Formal Languages and Grammars. Chapter 2: Sections 2.1 and 2.2

Formal Languages and Grammars Chapter 2: Sections 2.1 and 2.2 Formal Languages Basis for the design and implementation of programming languages Alphabet: finite set Σ of symbols String: finite sequence

### Introduction to Parsing. Lecture 5. Professor Alex Aiken Lecture #5 (Modified by Professor Vijay Ganesh)

Introduction to Parsing Lecture 5 (Modified by Professor Vijay Ganesh) 1 Outline Regular languages revisited Parser overview Context-free grammars (CFG s) Derivations Ambiguity 2 Languages and Automata

### Outline. Regular languages revisited. Introduction to Parsing. Parser overview. Context-free grammars (CFG s) Lecture 5. Derivations.

Outline Regular languages revisited Introduction to Parsing Lecture 5 Parser overview Context-free grammars (CFG s) Derivations Prof. Aiken CS 143 Lecture 5 1 Ambiguity Prof. Aiken CS 143 Lecture 5 2 Languages

### Parsing Part II. (Ambiguity, Top-down parsing, Left-recursion Removal)

Parsing Part II (Ambiguity, Top-down parsing, Left-recursion Removal) Ambiguous Grammars Definitions If a grammar has more than one leftmost derivation for a single sentential form, the grammar is ambiguous

### Projects for Compilers

Projects for Compilers 1. Project One: Lexical Analysis (Required) (1) Directions Implement a transition-diagram-based lexical analysis for the programming language TINY. (2) Outputs Source code (implemented

### Abstract Syntax Trees & Top-Down Parsing

Review of Parsing Abstract Syntax Trees & Top-Down Parsing Given a language L(G), a parser consumes a sequence of tokens s and produces a parse tree Issues: How do we recognize that s L(G)? A parse tree

### Wednesday, September 9, 15. Parsers

Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Parsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:

What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### CSE P 501 Compilers. Parsing & Context-Free Grammars Hal Perkins Winter /15/ Hal Perkins & UW CSE C-1

CSE P 501 Compilers Parsing & Context-Free Grammars Hal Perkins Winter 2008 1/15/2008 2002-08 Hal Perkins & UW CSE C-1 Agenda for Today Parsing overview Context free grammars Ambiguous grammars Reading:

### Syntax-Directed Translation. Lecture 14

Syntax-Directed Translation Lecture 14 (adapted from slides by R. Bodik) 9/27/2006 Prof. Hilfinger, Lecture 14 1 Motivation: parser as a translator syntax-directed translation stream of tokens parser ASTs,

### CSC 4181 Compiler Construction. Parsing. Outline. Introduction

CC 4181 Compiler Construction Parsing 1 Outline Top-down v.s. Bottom-up Top-down parsing Recursive-descent parsing LL1) parsing LL1) parsing algorithm First and follow sets Constructing LL1) parsing table

### CSE450 Translation of Programming Languages. Lecture 4: Syntax Analysis

CSE450 Translation of Programming Languages Lecture 4: Syntax Analysis http://xkcd.com/859 Structure of a Today! Compiler Source Language Lexical Analyzer Syntax Analyzer Semantic Analyzer Int. Code Generator

### Administrativia. PA2 assigned today. WA1 assigned today. Building a Parser II. CS164 3:30-5:00 TT 10 Evans. First midterm. Grammars.

Administrativia Building a Parser II CS164 3:30-5:00 TT 10 Evans PA2 assigned today due in 12 days WA1 assigned today due in a week it s a practice for the exam First midterm Oct 5 will contain some project-inspired

### ( ) i 0. Outline. Regular languages revisited. Introduction to Parsing. Parser overview. Context-free grammars (CFG s) Lecture 5.

Outline Regular languages revisited Introduction to Parsing Lecture 5 Parser overview Context-free grammars (CFG s) Derivations Prof. Aiken CS 143 Lecture 5 1 Ambiguity Prof. Aiken CS 143 Lecture 5 2 Languages

### Chapter 2 :: Programming Language Syntax

Chapter 2 :: Programming Language Syntax Michael L. Scott kkman@sangji.ac.kr, 2015 1 Regular Expressions A regular expression is one of the following: A character The empty string, denoted by Two regular

### Parsing III. CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones

Parsing III (Top-down parsing: recursive descent & LL(1) ) (Bottom-up parsing) CS434 Lecture 8 Spring 2005 Department of Computer Science University of Alabama Joel Jones Copyright 2003, Keith D. Cooper,

### Chapter 3. Parsing #1

Chapter 3 Parsing #1 Parser source file get next character scanner get token parser AST token A parser recognizes sequences of tokens according to some grammar and generates Abstract Syntax Trees (ASTs)

### Compilers - Chapter 2: An introduction to syntax analysis (and a complete toy compiler)

Compilers - Chapter 2: An introduction to syntax analysis (and a complete toy compiler) Lecturers: Paul Kelly (phjk@doc.ic.ac.uk) Office: room 304, William Penney Building Naranker Dulay (nd@doc.ic.ac.uk)

### Where We Are. CMSC 330: Organization of Programming Languages. This Lecture. Programming Languages. Motivation for Grammars

CMSC 330: Organization of Programming Languages Context Free Grammars Where We Are Programming languages Ruby OCaml Implementing programming languages Scanner Uses regular expressions Finite automata Parser