# Context-Free Languages and Parse Trees

Size: px
Start display at page:

Transcription

1 Context-Free Languages and Parse Trees Mridul Aanjaneya Stanford University July 12, 2012 Mridul Aanjaneya Automata Theory 1/ 41

2 Context-Free Grammars A context-free grammar is a notation for describing languages. It is more powerful than finite automata or regular expressions. It still cannot define all possible languages. Useful for nested structures, e.g., parentheses in programming languages. Mridul Aanjaneya Automata Theory 2/ 41

3 Context-Free Grammars Basic idea is to use variables to stand for sets of strings (i.e., languages). These variable are defined recursively, in terms of one another. Recursive rules (productions) involve only concatenation. Alternative rules for a variable allow union. Mridul Aanjaneya Automata Theory 3/ 41

4 Example: CFG for {0 n 1 n n 1} Productions: S 01 S 0S1 Basis: 01 is in the language. Induction: If w is in the language, then so is 0w1. Mridul Aanjaneya Automata Theory 4/ 41

5 CFG Formalism Terminals: symbols of the alphabet of the language being defined. Variables (nonterminals): a finite set of other symbols, each of which represents a language. Start symbol: the variable whose language is the one being defined. Mridul Aanjaneya Automata Theory 5/ 41

6 Productions A production has the form variable string of variables and terminals Convention A, B, C,... are variables. a, b, c,... are terminals...., X, Y, Z are either terminals or variables...., w, x, y, z are strings of terminals only. α, β, γ,... are strings of terminals and/or variables. Mridul Aanjaneya Automata Theory 6/ 41

7 Example: Formal CFG Here is a formal CFG for {0 n 1 n n 1}. Terminals = {0,1}. Variables = {S}. Start symbol = S. Productions = S 01 S 0S1 Mridul Aanjaneya Automata Theory 7/ 41

8 Derivations - Intuition We derive strings in the language of a CFG by starting with the start symbol, and repeatedly replacing some variable A by the right side of one of its productions. That is, the productions for A are those that have A on the left side of the. Mridul Aanjaneya Automata Theory 8/ 41

9 Derivations - Formalism We say αaβ αγβ if A γ is a production. Example: S 01; S 0S1. S 0S1 00S Mridul Aanjaneya Automata Theory 9/ 41

10 Iterated Derivation means zero or more derivation steps. Basis: α α for any string α. Induction: if α β and β γ, then α γ. Mridul Aanjaneya Automata Theory 10/ 41

11 Example: Iterated Derivation S 01; S 0S1. S 0S1 00S So S S; S 0S1; S 00S11; S Mridul Aanjaneya Automata Theory 11/ 41

12 Sentential Forms Any string of variables and/or terminals derived from the start symbol is called a sentential form. Formally, α is a sentential form iff S α. Mridul Aanjaneya Automata Theory 12/ 41

13 Language of a Grammar If G is a CFG, then L(G), the language of G is {w S w}. Note: w must be a terminal string, S is the start symbol. Example: G has productions S ε and S 0S1. Note: ε is a valid right hand side. L(G) = {0 n 1 n n 0}. Mridul Aanjaneya Automata Theory 13/ 41

14 Context-Free Languages A language that is defined by some CFG is called a context-free language. There are CFL s that are not regular languages, such as the example just given. But not all languages are CFL s. Intuition: CFL s can count two things, not three. Mridul Aanjaneya Automata Theory 14/ 41

15 BNF Notation Grammars for programming languages are often written in Backus-Naur Form (BNF). Variables are words in <... >, e.g., <statement>. Terminals are often multicharacter strings indicated by boldface or underline, e.g., while or WHILE. Mridul Aanjaneya Automata Theory 15/ 41

16 BNF Notation Symbol ::= is often used for. Symbol is used for or. A shorthand for a list of productions with the same left side. Example: S 0S1 01 is a shorthand for S 0S1 and S 01. Mridul Aanjaneya Automata Theory 16/ 41

17 BNF Notation: Kleene Closure Symbol... is used for one or more. Example: <digit> ::= <unsigned integer> ::= <digit>... Note: that s not exactly the for RE s. Translation: Replace α... with a new variable A and productions A Aα α. Grammar for unsigned integers can be replaced by: U UD D D Mridul Aanjaneya Automata Theory 17/ 41

18 BNF Notation: Optional Elements Surround one or more symbols by [...] to make them optional. Example: <statement> ::= if <condition> then <statement> [;else <statement>] Translation: replace [α] by a new variable A with productions A ε. Grammar for if-then-else can be replaced by: S ictsa A ;es ε Mridul Aanjaneya Automata Theory 18/ 41

19 BNF Notation: Grouping Use {...} to surround a sequence of symbols that need to be treated as a unit. Typically, they are followed by a... for one or more. Example: <statement list> ::= <statement> [{;<statement>}...] Mridul Aanjaneya Automata Theory 19/ 41

20 BNF Notation: Grouping (Translation) You may, if you wish, create a new variable A for {α}. One production for A: A α. Use A in place of {α}. Mridul Aanjaneya Automata Theory 20/ 41

21 Example: Grouping L S[{;S}...] Replace L S[A...]; A ;S A stands for {;S}. Then by L SB; B A... ε; A ;S B stands for [A...] (zero or more A s). Finally by L SB; B C ε; C AC A; A ;S. C stands for A... Mridul Aanjaneya Automata Theory 21/ 41

22 Leftmost and Rightmost Derivations Derivations allow us to replace any of the variables in a string. Leads to many different derivations of the same string. By forcing the leftmost variable (or alternatively, the rightmost variable) to be replaced, we avoid these distinctions without a difference. Mridul Aanjaneya Automata Theory 22/ 41

23 Leftmost Derivations Say waα lm wβα if w is a string of terminals only and A β is a production. Also, α lm β if α becomes β by a sequence of zero or more lm steps. Mridul Aanjaneya Automata Theory 23/ 41

24 Example: Leftmost Derivations Balanced parantheses grammar: S SS (S) () S lm SS lm (S)S lm (())S lm (())() Thus, S lm (())() S SS S() (S)() (())() is a derivation, but not a leftmost derivation. Mridul Aanjaneya Automata Theory 24/ 41

25 Rightmost Derivations Say αaw rm αβw if w is a string of terminals only and A β is a production. Also, α rm β if α becomes β by a sequence of zero or more rm steps. Mridul Aanjaneya Automata Theory 25/ 41

26 Example: Rightmost Derivations Balanced parantheses grammar: S SS (S) () S rm SS rm S() rm (S)() rm (())() Thus, S rm (())() S SS SSS S()S ()()S ()()() is neither a rightmost derivation nor a leftmost derivation. Mridul Aanjaneya Automata Theory 26/ 41

27 Parse Trees Parse trees are trees labeled by symbols of a particular CFG. Leaves: labeled by a terminal or ε. Interior nodes: labeled by a variable. Children are labeled by the right side of a production for the parent. Root: must be labeled by the start symbol. Mridul Aanjaneya Automata Theory 27/ 41

28 Example: Parse Tree S SS (S) () S S S ( S ) ( ) ( ) Mridul Aanjaneya Automata Theory 28/ 41

29 Yield of a Parse Tree The concatenation of the labels of the leaves in left-to-right order is called the yield of the parse tree. Example: Yield of the given parse tree is (())(). S S S ( S ) ( ) ( ) Mridul Aanjaneya Automata Theory 29/ 41

30 Parse Trees, Leftmost and Rightmost Derivations For every parse tree, there is a unique leftmost and a unique rightmost derivation. We ll prove: 1 If there is a parse tree with root labeled A and yield w, then A lm w. 2 If A lm w, then there is a parse tree with root A and yield w. Mridul Aanjaneya Automata Theory 30/ 41

31 Part 1: Basis Induction on the height (length of the longest path from the root) of the tree. Basis: height 1. Tree looks like a A... a 1 n A a 1... a n must be a production. Thus, A lm a 1... a n. Mridul Aanjaneya Automata Theory 31/ 41

32 Part 1: Induction Assume (1) for trees of height < h, and let this tree have height h: By IH, X i lm w i. Note: If X i is a terminal, then X i = w i. X A... X 1 n w 1 w n Thus, A lm X 1...X n lm w 1X 2...X n lm w 1w 2 X 3...X n lm... lm w 1...w n Mridul Aanjaneya Automata Theory 32/ 41

33 Part 2 Given a leftmost derivation of a terminal string, we need to prove the existence of a parse tree. The proof is an induction on the length of the derivation. Mridul Aanjaneya Automata Theory 33/ 41

34 Part 2: Basis If A lm a 1a 2...a n by a one-step derivation, then there must be a parse tree a A... a 1 n Mridul Aanjaneya Automata Theory 34/ 41

35 Part 2: Induction Assume (2) for derivations of fewer than k > 1 steps, and let A lm w be a k-step derivation. First step is A lm X 1...X n. Key point: w can be divided such that the first portion is derived from X 1, the next is derived from X 2, and so on. If X i is a terminal, then w i = X i. Mridul Aanjaneya Automata Theory 35/ 41

36 Part 2: Induction That is, X i lm w i for all i such that X i is a variable. And the derivation takes fewer than k steps. By the IH, if X i is a variable, then there is a parse tree with root X i and yield w i. Thus, there is a parse tree X A... X 1 n w 1 w n Mridul Aanjaneya Automata Theory 36/ 41

37 Parse Trees and Rightmost Derivations The ideas are essentially the mirror image of the proof of the leftmost derivations. Left to the imagination! Mridul Aanjaneya Automata Theory 37/ 41

38 Parse Trees and Any Derivation The proof that you can obtain a parse tree from a leftmost derivation doesn t really depend on leftmost. First step still has to be A lm X 1...X n. And w can still be divided such that the first portion is derived from X 1, the next is derived from X 2, and so on. Mridul Aanjaneya Automata Theory 38/ 41

39 Ambiguous Grammars A CFG is ambiguous is there is a string in the language that is the yield of two or more parse trees. Example: S SS (S) () Two parse trees for ()()()! Mridul Aanjaneya Automata Theory 39/ 41

40 Ambiguity, Leftmost and Rightmost Derivations If there are two different parse trees, they must produce two different leftmost derivations by the construction given in the proof. Conversely, two different leftmost derivations produce different parse trees by the other part of the proof. Likewise for rightmost derivations. Mridul Aanjaneya Automata Theory 40/ 41

41 Ambiguity, Leftmost and Rightmost Derivations Thus, equivalent definitions for ambiguous grammar are: 1 There is a string in the language that has two different leftmost derivations. 2 There is a string in the language that has two different rightmost derivations. Mridul Aanjaneya Automata Theory 41/ 41

### Ambiguous Grammars and Compactification

Ambiguous Grammars and Compactification Mridul Aanjaneya Stanford University July 17, 2012 Mridul Aanjaneya Automata Theory 1/ 44 Midterm Review Mathematical Induction and Pigeonhole Principle Finite Automata

### Context-Free Grammars

Context-Free Grammars 1 Informal Comments A context-free grammar is a notation for describing languages. It is more powerful than finite automata or RE s, but still cannot define all possible languages.

### MA513: Formal Languages and Automata Theory Topic: Context-free Grammars (CFG) Lecture Number 18 Date: September 12, 2011

MA53: Formal Languages and Automata Theory Topic: Context-free Grammars (CFG) Lecture Number 8 Date: September 2, 20 xercise: Define a context-free grammar that represents (a simplification of) expressions

### Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018

Finite Automata Theory and Formal Languages TMV027/DIT321 LP4 2018 Lecture 11 Ana Bove April 26th 2018 Recap: Regular Languages Decision properties of RL: Is it empty? Does it contain this word? Contains

### Syntax Analysis. Prof. James L. Frankel Harvard University. Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved.

Syntax Analysis Prof. James L. Frankel Harvard University Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved. Context-Free Grammar (CFG) terminals non-terminals start

### Syntax Analysis Part I

Syntax Analysis Part I Chapter 4: Context-Free Grammars Slides adapted from : Robert van Engelen, Florida State University Position of a Parser in the Compiler Model Source Program Lexical Analyzer Token,

### Where We Are. CMSC 330: Organization of Programming Languages. This Lecture. Programming Languages. Motivation for Grammars

CMSC 330: Organization of Programming Languages Context Free Grammars Where We Are Programming languages Ruby OCaml Implementing programming languages Scanner Uses regular expressions Finite automata Parser

### Parsing. source code. while (k<=n) {sum = sum+k; k=k+1;}

Compiler Construction Grammars Parsing source code scanner tokens regular expressions lexical analysis Lennart Andersson parser context free grammar Revision 2012 01 23 2012 parse tree AST builder (implicit)

### Defining syntax using CFGs

Defining syntax using CFGs Roadmap Last time Defined context-free grammar This time CFGs for specifying a language s syntax Language membership List grammars Resolving ambiguity CFG Review G = (N,Σ,P,S)

### Introduction to Syntax Analysis. The Second Phase of Front-End

Compiler Design IIIT Kalyani, WB 1 Introduction to Syntax Analysis The Second Phase of Front-End Compiler Design IIIT Kalyani, WB 2 Syntax Analysis The syntactic or the structural correctness of a program

### Derivations of a CFG. MACM 300 Formal Languages and Automata. Context-free Grammars. Derivations and parse trees

Derivations of a CFG MACM 300 Formal Languages and Automata Anoop Sarkar http://www.cs.sfu.ca/~anoop strings grow on trees strings grow on Noun strings grow Object strings Verb Object Noun Verb Object

### Context-Free Languages & Grammars (CFLs & CFGs) Reading: Chapter 5

Context-Free Languages & Grammars (CFLs & CFGs) Reading: Chapter 5 1 Not all languages are regular So what happens to the languages which are not regular? Can we still come up with a language recognizer?

### CMSC 330: Organization of Programming Languages. Architecture of Compilers, Interpreters

: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Scanner Parser Static Analyzer Intermediate Representation Front End Back End Compiler / Interpreter

### Introduction to Parsing. Lecture 8

Introduction to Parsing Lecture 8 Adapted from slides by G. Necula Outline Limitations of regular languages Parser overview Context-free grammars (CFG s) Derivations Languages and Automata Formal languages

### Optimizing Finite Automata

Optimizing Finite Automata We can improve the DFA created by MakeDeterministic. Sometimes a DFA will have more states than necessary. For every DFA there is a unique smallest equivalent DFA (fewest states

### Context-Free Grammars and Languages (2015/11)

Chapter 5 Context-Free Grammars and Languages (2015/11) Adriatic Sea shore at Opatija, Croatia Outline 5.0 Introduction 5.1 Context-Free Grammars (CFG s) 5.2 Parse Trees 5.3 Applications of CFG s 5.4 Ambiguity

### Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

### COP 3402 Systems Software Syntax Analysis (Parser)

COP 3402 Systems Software Syntax Analysis (Parser) Syntax Analysis 1 Outline 1. Definition of Parsing 2. Context Free Grammars 3. Ambiguous/Unambiguous Grammars Syntax Analysis 2 Lexical and Syntax Analysis

### Dr. D.M. Akbar Hussain

Syntax Analysis Parsing Syntax Or Structure Given By Determines Grammar Rules Context Free Grammar 1 Context Free Grammars (CFG) Provides the syntactic structure: A grammar is quadruple (V T, V N, S, R)

### 3. Syntax Analysis. Andrea Polini. Formal Languages and Compilers Master in Computer Science University of Camerino

3. Syntax Analysis Andrea Polini Formal Languages and Compilers Master in Computer Science University of Camerino (Formal Languages and Compilers) 3. Syntax Analysis CS@UNICAM 1 / 54 Syntax Analysis: the

### Homework. Context Free Languages. Before We Start. Announcements. Plan for today. Languages. Any questions? Recall. 1st half. 2nd half.

Homework Context Free Languages Homework #2 returned Homework #3 due today Homework #4 Pg 133 -- Exercise 1 (use structural induction) Pg 133 -- Exercise 3 Pg 134 -- Exercise 8b,c,d Pg 135 -- Exercise

### Fall Compiler Principles Context-free Grammars Refresher. Roman Manevich Ben-Gurion University of the Negev

Fall 2016-2017 Compiler Principles Context-free Grammars Refresher Roman Manevich Ben-Gurion University of the Negev 1 xample grammar S S ; S S id := S print (L) id num + L L L, shorthand for Statement

### programming languages need to be precise a regular expression is one of the following: tokens are the building blocks of programs

Chapter 2 :: Programming Language Syntax Programming Language Pragmatics Michael L. Scott Introduction programming languages need to be precise natural languages less so both form (syntax) and meaning

### CS415 Compilers. Syntax Analysis. These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University

CS415 Compilers Syntax Analysis These slides are based on slides copyrighted by Keith Cooper, Ken Kennedy & Linda Torczon at Rice University Limits of Regular Languages Advantages of Regular Expressions

### Introduction to Syntax Analysis

Compiler Design 1 Introduction to Syntax Analysis Compiler Design 2 Syntax Analysis The syntactic or the structural correctness of a program is checked during the syntax analysis phase of compilation.

### Parsing Part II. (Ambiguity, Top-down parsing, Left-recursion Removal)

Parsing Part II (Ambiguity, Top-down parsing, Left-recursion Removal) Ambiguous Grammars Definitions If a grammar has more than one leftmost derivation for a single sentential form, the grammar is ambiguous

### Enumerations and Turing Machines

Enumerations and Turing Machines Mridul Aanjaneya Stanford University August 07, 2012 Mridul Aanjaneya Automata Theory 1/ 35 Finite Sets Intuitively, a finite set is a set for which there is a particular

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview Tokens and regular expressions Syntax and context-free grammars Grammar derivations More about parse trees Top-down and bottom-up

### COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview n Tokens and regular expressions n Syntax and context-free grammars n Grammar derivations n More about parse trees n Top-down and

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Context Free Grammars and Parsing 1 Recall: Architecture of Compilers, Interpreters Source Parser Static Analyzer Intermediate Representation Front End Back

### 3. Parsing. Oscar Nierstrasz

3. Parsing Oscar Nierstrasz Thanks to Jens Palsberg and Tony Hosking for their kind permission to reuse and adapt the CS132 and CS502 lecture notes. http://www.cs.ucla.edu/~palsberg/ http://www.cs.purdue.edu/homes/hosking/

### Syntax Analysis Check syntax and construct abstract syntax tree

Syntax Analysis Check syntax and construct abstract syntax tree if == = ; b 0 a b Error reporting and recovery Model using context free grammars Recognize using Push down automata/table Driven Parsers

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### CMSC 330: Organization of Programming Languages. Context Free Grammars

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### 3. Context-free grammars & parsing

3. Context-free grammars & parsing The parsing process sequences of tokens parse tree or syntax tree a / [ / index / ]/= / 4 / + / 2 The parsing process sequences of tokens parse tree or syntax tree a

### Lexical and Syntax Analysis

Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Lexical and Syntax Analysis (of Programming Languages) Top-Down Parsing Easy for humans to write and understand String of characters

### Languages and Compilers

Principles of Software Engineering and Operational Systems Languages and Compilers SDAGE: Level I 2012-13 3. Formal Languages, Grammars and Automata Dr Valery Adzhiev vadzhiev@bournemouth.ac.uk Office:

### CMSC 330: Organization of Programming Languages. Context-Free Grammars Ambiguity

CMSC 330: Organization of Programming Languages Context-Free Grammars Ambiguity Review Why should we study CFGs? What are the four parts of a CFG? How do we tell if a string is accepted by a CFG? What

### Introduction to Parsing. Lecture 5

Introduction to Parsing Lecture 5 1 Outline Regular languages revisited Parser overview Context-free grammars (CFG s) Derivations Ambiguity 2 Languages and Automata Formal languages are very important

### CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### Lexical and Syntax Analysis. Top-Down Parsing

Lexical and Syntax Analysis Top-Down Parsing Easy for humans to write and understand String of characters Lexemes identified String of tokens Easy for programs to transform Data structure Syntax A syntax

### Compilers Course Lecture 4: Context Free Grammars

Compilers Course Lecture 4: Context Free Grammars Example: attempt to define simple arithmetic expressions using named regular expressions: num = [0-9]+ sum = expr "+" expr expr = "(" sum ")" num Appears

### 4. Lexical and Syntax Analysis

4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal

### Outline. Limitations of regular languages. Introduction to Parsing. Parser overview. Context-free grammars (CFG s)

Outline Limitations of regular languages Introduction to Parsing Parser overview Lecture 8 Adapted from slides by G. Necula Context-free grammars (CFG s) Derivations Languages and Automata Formal languages

### CMSC 330: Organization of Programming Languages. Context Free Grammars

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### Lecture 4: Syntax Specification

The University of North Carolina at Chapel Hill Spring 2002 Lecture 4: Syntax Specification Jan 16 1 Phases of Compilation 2 1 Syntax Analysis Syntax: Webster s definition: 1 a : the way in which linguistic

### Part 5 Program Analysis Principles and Techniques

1 Part 5 Program Analysis Principles and Techniques Front end 2 source code scanner tokens parser il errors Responsibilities: Recognize legal programs Report errors Produce il Preliminary storage map Shape

### CS525 Winter 2012 \ Class Assignment #2 Preparation

1 CS525 Winter 2012 \ Class Assignment #2 Preparation Ariel Stolerman 2.26) Let be a CFG in Chomsky Normal Form. Following is a proof that for any ( ) of length exactly steps are required for any derivation

### Introduction to Parsing. Lecture 5

Introduction to Parsing Lecture 5 1 Outline Regular languages revisited Parser overview Context-free grammars (CFG s) Derivations Ambiguity 2 Languages and Automata Formal languages are very important

### 4. Lexical and Syntax Analysis

4. Lexical and Syntax Analysis 4.1 Introduction Language implementation systems must analyze source code, regardless of the specific implementation approach Nearly all syntax analysis is based on a formal

### Lecture 8: Context Free Grammars

Lecture 8: Context Free s Dr Kieran T. Herley Department of Computer Science University College Cork 2017-2018 KH (12/10/17) Lecture 8: Context Free s 2017-2018 1 / 1 Specifying Non-Regular Languages Recall

### Building Compilers with Phoenix

Building Compilers with Phoenix Syntax-Directed Translation Structure of a Compiler Character Stream Intermediate Representation Lexical Analyzer Machine-Independent Optimizer token stream Intermediate

### Introduction to Lexing and Parsing

Introduction to Lexing and Parsing ECE 351: Compilers Jon Eyolfson University of Waterloo June 18, 2012 1 Riddle Me This, Riddle Me That What is a compiler? 1 Riddle Me This, Riddle Me That What is a compiler?

### VIVA QUESTIONS WITH ANSWERS

VIVA QUESTIONS WITH ANSWERS 1. What is a compiler? A compiler is a program that reads a program written in one language the source language and translates it into an equivalent program in another language-the

### Describing Syntax and Semantics

Describing Syntax and Semantics Introduction Syntax: the form or structure of the expressions, statements, and program units Semantics: the meaning of the expressions, statements, and program units Syntax

### CSE 311 Lecture 21: Context-Free Grammars. Emina Torlak and Kevin Zatloukal

CSE 311 Lecture 21: Context-Free Grammars Emina Torlak and Kevin Zatloukal 1 Topics Regular expressions A brief review of Lecture 20. Context-free grammars Syntax, semantics, and examples. 2 Regular expressions

### Architecture of Compilers, Interpreters. CMSC 330: Organization of Programming Languages. Front End Scanner and Parser. Implementing the Front End

Architecture of Compilers, Interpreters : Organization of Programming Languages ource Analyzer Optimizer Code Generator Context Free Grammars Intermediate Representation Front End Back End Compiler / Interpreter

### Syntax Analysis: Context-free Grammars, Pushdown Automata and Parsing Part - 4. Y.N. Srikant

Syntax Analysis: Context-free Grammars, Pushdown Automata and Part - 4 Department of Computer Science and Automation Indian Institute of Science Bangalore 560 012 NPTEL Course on Principles of Compiler

### Formal Languages. Formal Languages

Regular expressions Formal Languages Finite state automata Deterministic Non-deterministic Review of BNF Introduction to Grammars Regular grammars Formal Languages, CS34 Fall2 BGRyder Formal Languages

### Syntax Analysis. Amitabha Sanyal. (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay

Syntax Analysis (www.cse.iitb.ac.in/ as) Department of Computer Science and Engineering, Indian Institute of Technology, Bombay September 2007 College of Engineering, Pune Syntax Analysis: 2/124 Syntax

### Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part 1

Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part 1 1. Introduction Parsing is the task of Syntax Analysis Determining the syntax, or structure, of a program. The syntax is defined by the grammar rules

### Principles of Programming Languages COMP251: Syntax and Grammars

Principles of Programming Languages COMP251: Syntax and Grammars Prof. Dekai Wu Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China Fall 2006

### Syntax Analysis. Martin Sulzmann. Martin Sulzmann Syntax Analysis 1 / 38

Syntax Analysis Martin Sulzmann Martin Sulzmann Syntax Analysis 1 / 38 Syntax Analysis Objective Recognize individual tokens as sentences of a language (beyond regular languages). Example 1 (OK) Program

### Defining Languages GMU

Defining Languages CS463 @ GMU How do we discuss languages? We might focus on these qualities: readability: how well does a language explicitly and clearly describe its purpose? writability: how expressive

### CMPS Programming Languages. Dr. Chengwei Lei CEECS California State University, Bakersfield

CMPS 3500 Programming Languages Dr. Chengwei Lei CEECS California State University, Bakersfield Chapter 3 Describing Syntax and Semantics Chapter 3 Topics Introduction The General Problem of Describing

### Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part2 3.3 Parse Trees and Abstract Syntax Trees

Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part2 3.3 Parse Trees and Abstract Syntax Trees 3.3.1 Parse trees 1. Derivation V.S. Structure Derivations do not uniquely represent the structure of the strings

### Normal Forms for CFG s. Eliminating Useless Variables Removing Epsilon Removing Unit Productions Chomsky Normal Form

Normal Forms for CFG s Eliminating Useless Variables Removing Epsilon Removing Unit Productions Chomsky Normal Form 1 Variables That Derive Nothing Consider: S -> AB, A -> aa a, B -> AB Although A derives

### Announcements. Written Assignment 1 out, due Friday, July 6th at 5PM.

Syntax Analysis Announcements Written Assignment 1 out, due Friday, July 6th at 5PM. xplore the theoretical aspects of scanning. See the limits of maximal-munch scanning. Class mailing list: There is an

### Compiler Design Concepts. Syntax Analysis

Compiler Design Concepts Syntax Analysis Introduction First task is to break up the text into meaningful words called tokens. newval=oldval+12 id = id + num Token Stream Lexical Analysis Source Code (High

### CSE302: Compiler Design

CSE302: Compiler Design Instructor: Dr. Liang Cheng Department of Computer Science and Engineering P.C. Rossin College of Engineering & Applied Science Lehigh University February 20, 2007 Outline Recap

### EECS 6083 Intro to Parsing Context Free Grammars

EECS 6083 Intro to Parsing Context Free Grammars Based on slides from text web site: Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. 1 Parsing sequence of tokens parser

### Properties of Regular Expressions and Finite Automata

Properties of Regular Expressions and Finite Automata Some token patterns can t be defined as regular expressions or finite automata. Consider the set of balanced brackets of the form [[[ ]]]. This set

### 15 212: Principles of Programming. Some Notes on Grammars and Parsing

15 212: Principles of Programming Some Notes on Grammars and Parsing Michael Erdmann Spring 2011 1 Introduction These notes are intended as a rough and ready guide to grammars and parsing. The theoretical

### The analysis part breaks up the source program into constituent pieces and creates an intermediate representation of the source program.

COMPILER DESIGN 1. What is a compiler? A compiler is a program that reads a program written in one language the source language and translates it into an equivalent program in another language-the target

### Introduction to Parsing. Lecture 5. Professor Alex Aiken Lecture #5 (Modified by Professor Vijay Ganesh)

Introduction to Parsing Lecture 5 (Modified by Professor Vijay Ganesh) 1 Outline Regular languages revisited Parser overview Context-free grammars (CFG s) Derivations Ambiguity 2 Languages and Automata

### MIT Specifying Languages with Regular Expressions and Context-Free Grammars

MIT 6.035 Specifying Languages with Regular essions and Context-Free Grammars Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Language Definition Problem How to precisely

### CS 314 Principles of Programming Languages

CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution

SYNTAX ANALYSIS 1. Define parser. Hierarchical analysis is one in which the tokens are grouped hierarchically into nested collections with collective meaning. Also termed as Parsing. 2. Mention the basic

VALLIAMMAI ENGNIEERING COLLEGE SRM Nagar, Kattankulathur 603203. DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING Year & Semester : III Year, V Semester Section : CSE - 1 & 2 Subject Code : CS6503 Subject

### A programming language requires two major definitions A simple one pass compiler

A programming language requires two major definitions A simple one pass compiler [Syntax: what the language looks like A context-free grammar written in BNF (Backus-Naur Form) usually suffices. [Semantics:

### Chapter 4: LR Parsing

Chapter 4: LR Parsing 110 Some definitions Recall For a grammar G, with start symbol S, any string α such that S called a sentential form α is If α Vt, then α is called a sentence in L G Otherwise it is

### Types of parsing. CMSC 430 Lecture 4, Page 1

Types of parsing Top-down parsers start at the root of derivation tree and fill in picks a production and tries to match the input may require backtracking some grammars are backtrack-free (predictive)

### MIT Specifying Languages with Regular Expressions and Context-Free Grammars. Martin Rinard Massachusetts Institute of Technology

MIT 6.035 Specifying Languages with Regular essions and Context-Free Grammars Martin Rinard Massachusetts Institute of Technology Language Definition Problem How to precisely define language Layered structure

### CSE P 501 Compilers. Parsing & Context-Free Grammars Hal Perkins Winter /15/ Hal Perkins & UW CSE C-1

CSE P 501 Compilers Parsing & Context-Free Grammars Hal Perkins Winter 2008 1/15/2008 2002-08 Hal Perkins & UW CSE C-1 Agenda for Today Parsing overview Context free grammars Ambiguous grammars Reading:

### Top-Down Parsing and Intro to Bottom-Up Parsing. Lecture 7

Top-Down Parsing and Intro to Bottom-Up Parsing Lecture 7 1 Predictive Parsers Like recursive-descent but parser can predict which production to use Predictive parsers are never wrong Always able to guess

### Syntax. In Text: Chapter 3

Syntax In Text: Chapter 3 1 Outline Syntax: Recognizer vs. generator BNF EBNF Chapter 3: Syntax and Semantics 2 Basic Definitions Syntax the form or structure of the expressions, statements, and program

### CHAPTER 10 GRAPHS AND TREES. Alessandro Artale UniBZ - artale/

CHAPTER 10 GRAPHS AND TREES Alessandro Artale UniBZ - http://www.inf.unibz.it/ artale/ SECTION 10.5 Trees Copyright Cengage Learning. All rights reserved. Trees In mathematics, a tree is a connected graph

### Wednesday, September 9, 15. Parsers

Parsers What is a parser A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Parsers. What is a parser. Languages. Agenda. Terminology. Languages. A parser has two jobs:

What is a parser Parsers A parser has two jobs: 1) Determine whether a string (program) is valid (think: grammatically correct) 2) Determine the structure of a program (think: diagramming a sentence) Agenda

### Compilers and computer architecture From strings to ASTs (2): context free grammars

1 / 1 Compilers and computer architecture From strings to ASTs (2): context free grammars Martin Berger October 2018 Recall the function of compilers 2 / 1 3 / 1 Recall we are discussing parsing Source

### CSE450 Translation of Programming Languages. Lecture 4: Syntax Analysis

CSE450 Translation of Programming Languages Lecture 4: Syntax Analysis http://xkcd.com/859 Structure of a Today! Compiler Source Language Lexical Analyzer Syntax Analyzer Semantic Analyzer Int. Code Generator

### CMSC 330: Organization of Programming Languages. Context Free Grammars

CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

### Compilers. Yannis Smaragdakis, U. Athens (original slides by Sam

Compilers Parsing Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Next step text chars Lexical analyzer tokens Parser IR Errors Parsing: Organize tokens into sentences Do tokens conform

### ICOM 4036 Spring 2004

Language Specification and Translation ICOM 4036 Spring 2004 Lecture 3 Copyright 2004 Pearson Addison-Wesley. All rights reserved. 3-1 Language Specification and Translation Topics Structure of a Compiler

### Compiler Construction 2016/2017 Syntax Analysis

Compiler Construction 2016/2017 Syntax Analysis Peter Thiemann November 2, 2016 Outline 1 Syntax Analysis Recursive top-down parsing Nonrecursive top-down parsing Bottom-up parsing Syntax Analysis tokens

### 2.2 Syntax Definition

42 CHAPTER 2. A SIMPLE SYNTAX-DIRECTED TRANSLATOR sequence of "three-address" instructions; a more complete example appears in Fig. 2.2. This form of intermediate code takes its name from instructions

### Chapter 4. Lexical and Syntax Analysis

Chapter 4 Lexical and Syntax Analysis Chapter 4 Topics Introduction Lexical Analysis The Parsing Problem Recursive-Descent Parsing Bottom-Up Parsing Copyright 2012 Addison-Wesley. All rights reserved.

### Outline. Regular languages revisited. Introduction to Parsing. Parser overview. Context-free grammars (CFG s) Lecture 5. Derivations.

Outline Regular languages revisited Introduction to Parsing Lecture 5 Parser overview Context-free grammars (CFG s) Derivations Prof. Aiken CS 143 Lecture 5 1 Ambiguity Prof. Aiken CS 143 Lecture 5 2 Languages