Context Free Grammars and Recursive Descent Parsing

Size: px
Start display at page:

Download "Context Free Grammars and Recursive Descent Parsing"

Transcription

1 Context Free Grammars and Recursive Descent Parsing Tim Dawborn January, 2018

2 cfg Parsing Recursive Descent Parsing Calculator 2 Outline 1 Context-Free Grammars (cfg) 2 Parsing 3 Recursive Descent Parsing 4 Calculator

3 cfg Parsing Recursive Descent Parsing Calculator 3 Regular Expressions Regular expressions are a useful tool for pattern matching as you ll no doubt recall. To warm up, write a regular expression to match strings containing any number of a s, followed by two b s, followed by one or more c s

4 cfg Parsing Recursive Descent Parsing Calculator 3 Regular Expressions Regular expressions are a useful tool for pattern matching as you ll no doubt recall. To warm up, write a regular expression to match strings containing any number of a s, followed by two b s, followed by one or more c s /a*bbc+/

5 cfg Parsing Recursive Descent Parsing Calculator 3 Regular Expressions Regular expressions are a useful tool for pattern matching as you ll no doubt recall. To warm up, write a regular expression to match strings containing any number of a s, followed by two b s, followed by one or more c s /a*bbc+/ Now write a regular expression to match any number of a s followed by the same number of b s

6 cfg Parsing Recursive Descent Parsing Calculator 3 Regular Expressions Regular expressions are a useful tool for pattern matching as you ll no doubt recall. To warm up, write a regular expression to match strings containing any number of a s, followed by two b s, followed by one or more c s /a*bbc+/ Now write a regular expression to match any number of a s followed by the same number of b s Cannot be done!

7 cfg Parsing Recursive Descent Parsing Calculator 4 Regular Languages There are limitations on what types of languages regular expressions are able to match Regular expressions only able to match regular languages Context Free Grammars (cfgs) have more expressive power than regular expressions recursively enumerable context-sensitive context-free regular

8 cfg Parsing Recursive Descent Parsing Calculator 5 Grammar Definitions An alternative way to express the constraints on the valid strings of a language is to use a grammar A grammar consists of four things: Terminals (T ) Non-terminals (V ) Production rules (R : V (T V ) ) A start symbol (one of the non-terminals) (S V )

9 cfg Parsing Recursive Descent Parsing Calculator 6 Grammars An example grammar for micro-english : 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is"

10 cfg Parsing Recursive Descent Parsing Calculator 6 Grammars An example grammar for micro-english : 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" Terminals are represented as "string literals" Non-terminals are delimited with <angle-brackets> <start-symbol> is where we begin, conventionally at the top left The ::= is the can be re-written as symbol Alternative rewrite rules can be separated by vertical bars or put on separate lines

11 cfg Parsing Recursive Descent Parsing Calculator 6 Grammars An example grammar for micro-english : 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" Terminals T = {., I, a, the, me, cat, mat, rat, like, see, is} Non-terminals, built V = {<sentence>, <subject>, <object>, <noun>} Production rules R = the above five rules Start symbol S = <sentence>

12 cfg Parsing Recursive Descent Parsing Calculator 7 Grammars 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" The grammar rules define a set of rewrites <sentence> <subject> <verb> <object>.

13 cfg Parsing Recursive Descent Parsing Calculator 7 Grammars 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" The grammar rules define a set of rewrites <sentence> <subject> <verb> <object>. I <verb> <object>.

14 cfg Parsing Recursive Descent Parsing Calculator 7 Grammars 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" The grammar rules define a set of rewrites <sentence> <subject> <verb> <object>. I <verb> <object>. I see <object>.

15 cfg Parsing Recursive Descent Parsing Calculator 7 Grammars 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" The grammar rules define a set of rewrites <sentence> <subject> <verb> <object>. I <verb> <object>. I see <object>. I see the <noun>.

16 cfg Parsing Recursive Descent Parsing Calculator 7 Grammars 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" The grammar rules define a set of rewrites <sentence> <subject> <verb> <object>. I <verb> <object>. I see <object>. I see the <noun>. I see the cat.

17 cfg Parsing Recursive Descent Parsing Calculator 8 Grammars 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" Other sentences in micro-english: I like the cat. I see a rat. The cat like the mat. unfortunately not good English The mat is the rat. syntactically valid string

18 cfg Parsing Recursive Descent Parsing Calculator 9 Grammars 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" How many strings are in the language of micro-english?

19 cfg Parsing Recursive Descent Parsing Calculator 9 Grammars 1 <sentence> ::= <subject> <verb> <object> "." 2 <subject> ::= "I" "a" <noun> "the" <noun> 3 <object> ::= "me" "a" <noun> "the" <noun> 4 <noun> ::= "cat" "mat" "rat" 5 <verb> ::= "like" "see" "is" How many strings are in the language of micro-english? <verb> = 3 <noun> = 3 <object> = = 7 <subject> = = 7 <sentence> = = 147

20 cfg Parsing Recursive Descent Parsing Calculator 10 Your turn Write a grammar rule for a 0x prefixed hexadecimal number.

21 cfg Parsing Recursive Descent Parsing Calculator 10 Your turn Write a grammar rule for a 0x prefixed hexadecimal number. 1 <hex> ::= "0" "x" ( "0"... "9" "A"... "F" )+ Note the parentheses (, ) for grouping, and the regex-like + for one or more of. (Admit it, you know you want to write it as a regular expression: 0x[0-9A-F]+)

22 cfg Parsing Recursive Descent Parsing Calculator 10 Your turn Write a grammar rule for a 0x prefixed hexadecimal number. 1 <hex> ::= "0" "x" ( "0"... "9" "A"... "F" )+ Note the parentheses (, ) for grouping, and the regex-like + for one or more of. (Admit it, you know you want to write it as a regular expression: 0x[0-9A-F]+) Next, write a grammar rule for an integer (positive or negative).

23 cfg Parsing Recursive Descent Parsing Calculator 10 Your turn Write a grammar rule for a 0x prefixed hexadecimal number. 1 <hex> ::= "0" "x" ( "0"... "9" "A"... "F" )+ Note the parentheses (, ) for grouping, and the regex-like + for one or more of. (Admit it, you know you want to write it as a regular expression: 0x[0-9A-F]+) Next, write a grammar rule for an integer (positive or negative). 1 <integer> ::= "-"? ( "0" "1"... "9" )+

24 cfg Parsing Recursive Descent Parsing Calculator 11 Your turn Write a grammar to accept expressions of integers, which could contain zero or more additions (e.g. 23, , ). You will need two grammar rules.

25 cfg Parsing Recursive Descent Parsing Calculator 11 Your turn Write a grammar to accept expressions of integers, which could contain zero or more additions (e.g. 23, , ). You will need two grammar rules. 1 <expr> ::= <integer> ( "+" <expr> )? 2 <integer> ::= "-"? ( "0" "1"... "9" )+ Note how the first rule is recursive: that means you can create any sequence of <integer> + <integer> + <integer>...

26 cfg Parsing Recursive Descent Parsing Calculator 11 Your turn Write a grammar to accept expressions of integers, which could contain zero or more additions (e.g. 23, , ). You will need two grammar rules. 1 <expr> ::= <integer> ( "+" <expr> )? 2 <integer> ::= "-"? ( "0" "1"... "9" )+ Note how the first rule is recursive: that means you can create any sequence of <integer> + <integer> + <integer>... Write a grammar to accept any number of a s followed by the same number of b s. Again you ll need two rules.

27 cfg Parsing Recursive Descent Parsing Calculator 11 Your turn Write a grammar to accept expressions of integers, which could contain zero or more additions (e.g. 23, , ). You will need two grammar rules. 1 <expr> ::= <integer> ( "+" <expr> )? 2 <integer> ::= "-"? ( "0" "1"... "9" )+ Note how the first rule is recursive: that means you can create any sequence of <integer> + <integer> + <integer>... Write a grammar to accept any number of a s followed by the same number of b s. Again you ll need two rules. 1 <string> ::= "a" <string> "b" 2 <string> ::= ε

28 cfg Parsing Recursive Descent Parsing Calculator 12 Regular Expression Languauge The language of regular expressions is governed by a grammar You know that /a*b bbc/ is valid and /a(*/ is invalid Imagine our own regular expression language only has support for OR, bracketing, and the Kleene star. How might we write this grammar?

29 cfg Parsing Recursive Descent Parsing Calculator 12 Regular Expression Languauge The language of regular expressions is governed by a grammar You know that /a*b bbc/ is valid and /a(*/ is invalid Imagine our own regular expression language only has support for OR, bracketing, and the Kleene star. How might we write this grammar? Not your turn! My turn!

30 cfg Parsing Recursive Descent Parsing Calculator 12 Regular Expression Languauge The language of regular expressions is governed by a grammar You know that /a*b bbc/ is valid and /a(*/ is invalid Imagine our own regular expression language only has support for OR, bracketing, and the Kleene star. How might we write this grammar? Not your turn! My turn! 1 <re> ::= <simple-re> ( " " <re> )? 2 <simple-re> ::= <basic-re>+ 3 <basic-re> ::= <elem-re> "*"? 4 <elem-re> ::= "(" <re> ")" 5 <elem-re> ::= "\" ( "*" "(" ")" " " "\" ) 6 <elem-re> ::= ( "*" "(" ")" " " "\" )

31 cfg Parsing Recursive Descent Parsing Calculator 13 Parsing We saw before how you can use the grammar rules to generate strings of the language of the grammar <sentence> <subject> <verb> <object>.

32 cfg Parsing Recursive Descent Parsing Calculator 13 Parsing We saw before how you can use the grammar rules to generate strings of the language of the grammar <sentence> <subject> <verb> <object>. I <verb> <object>.

33 cfg Parsing Recursive Descent Parsing Calculator 13 Parsing We saw before how you can use the grammar rules to generate strings of the language of the grammar <sentence> <subject> <verb> <object>. I <verb> <object>. I see <object>.

34 cfg Parsing Recursive Descent Parsing Calculator 13 Parsing We saw before how you can use the grammar rules to generate strings of the language of the grammar <sentence> <subject> <verb> <object>. I <verb> <object>. I see <object>. I see the <noun>.

35 cfg Parsing Recursive Descent Parsing Calculator 13 Parsing We saw before how you can use the grammar rules to generate strings of the language of the grammar <sentence> <subject> <verb> <object>. I <verb> <object>. I see <object>. I see the <noun>. I see the cat.

36 cfg Parsing Recursive Descent Parsing Calculator 14 Parsing Parsing is the opposite process: Given a string, 1 Check that the string is in the language of the grammar 2 Construct the derivation tree (parse tree) for the string <sentence> <verb> <object> <subject> <noun> I see the cat.

37 cfg Parsing Recursive Descent Parsing Calculator 15 Parsing in our re implementation We need to be able to convert input strings of regular expressions into nfas i.e. we need to parse the language of regular expressions /a(bc d*)*e/

38 cfg Parsing Recursive Descent Parsing Calculator 16 /a(bc d*)*e/ re simple basic-re basic-re basic-re elem-re elem-re * elem-re a ( re ) e re re simple-re simple-re basic-re basic-re basic-re elem-re elem-re elem-re * b c d

39 cfg Parsing Recursive Descent Parsing Calculator 17 Parsing a calculator language Here is a grammar for a basic calculator language 1 <e1> ::= <e2> ( "+" <e1> )? 2 <e2> ::= <e3> ( "*" <e2> )? 3 <e3> ::= "-"? ( "0" "1"... "9" )+ 4 <e3> ::= "(" <e1> ")" Come up with a string in the language of this grammar Draw the parse tree for this string: 4 * (3 + 2)

40 cfg Parsing Recursive Descent Parsing Calculator 18 Parse tree for 4*(3+2) <e1> <e2> <e3> "4" "*" <e2> <e3> "(" <e1> ")" <e2> <e3> "3" "+" <e1> <e2> <e3> "2"

41 cfg Parsing Recursive Descent Parsing Calculator 19 Evaluating 4*(3+2) <e1> <e2> <e3> "4" "*" <e2> <e3> "(" <e1> ")" <e2> <e3> "3" "+" <e1> <e2> <e3> "2"

42 cfg Parsing Recursive Descent Parsing Calculator 19 Evaluating 4*(3+2) <e1> <e2> <e3> "4" "*" <e2> <e3> "(" <e1> ")" <e2> "+" <e1> <e1>: 3+2=5 <e3> <e2> "3" <e3> "2"

43 cfg Parsing Recursive Descent Parsing Calculator 19 Evaluating 4*(3+2) <e1> <e2> <e2>: 4*5=20 <e3> "*" <e2> "4" <e3> "(" <e1> ")" <e2> "+" <e1> <e1>: 3+2=5 <e3> <e2> "3" <e3> "2"

44 cfg Parsing Recursive Descent Parsing Calculator 20 Parsing in our re implementation We need to be able to convert input strings of regular expressions into nfas i.e. we need to parse the language of regular expressions /a(bc d*)*e/

45 cfg Parsing Recursive Descent Parsing Calculator 21 Introduction There is a standard parsing technique for context free grammars using recursion: Recursive Descent Parsing The idea is to come up with one function/method for each non-terminal in the grammar Here we will learn how to construct a recursive descent parser for any cfg

46 cfg Parsing Recursive Descent Parsing Calculator 22 Balanced Language Grammar We want to write a parser which accepts strings of the grammar: 1 <s> ::= "a" <s> "b" 2 <s> ::= "e" What we re aiming for: 1 >>> Parser('aeb').parse() 2 True 3 >>> Parser('aaebb').parse() 4 True 5 >>> Parser('aaebbb').parse() 6 False Note: this is called an undecorated parser because it doesn t do any more than check syntactic correctness of the input.

47 cfg Parsing Recursive Descent Parsing Calculator 23 Step 1: Basic parsing framework 1 class Parser: 2 def init (self, tokens): 3 self._tokens = tokens 4 self._length = len(tokens) 5 self._upto = def end(self): 8 return self._upto == self._length 9 10 def peek(self): 11 return None if self.end() else self._tokens[self._upto] def next(self): 14 if not self.end(): 15 self._upto += 1

48 cfg Parsing Recursive Descent Parsing Calculator 24 Step 2: Helper method for each non-terminal 17 def _parse_s(self): 18 if self.peek() == 'a': # <s> ::= "a" <s> "b" 19 self.next() # move to the next input token 20 ret = self._parse_s() # recursively parse the <s> 21 if not ret: 22 return False 23 if self.peek()!= 'b': # assert the next input token 24 return False 25 self.next() # move to the next input token 26 elif self.peek() == 'e': # <s> ::= "e" 27 self.next() 28 else: # unknown input! 29 return False 30 return True

49 cfg Parsing Recursive Descent Parsing Calculator 25 Step 3: Wrap the start-symbol parsing helper method 32 def parse(self): 33 return self._parse_s() and self.end()

50 cfg Parsing Recursive Descent Parsing Calculator 26 Step 4: Test it 1 >>> Parser('ab').parse() 2 False 3 >>> Parser('cd').parse() 4 False 5 >>> Parser('e').parse() 6 True 7 >>> Parser('aeb').parse() 8 True 9 >>> Parser('aaebb').parse() 10 True 11 >>> Parser('aaebbb').parse() 12 False 13 >>> Parser('aaccaaebbddbb').parse() 14 False

51 cfg Parsing Recursive Descent Parsing Calculator 27 Calculator Let s build a calculator evaluator! (This is decorated.) What we re aiming for: 1 >>> Parser(['3','+','2']).parse().eval() >>> Parser(['3','*','2']).parse().eval() >>> Parser(['3','*','2','+','4']).parse().eval() >>> Parser(['3','+','2','*','4']).parse().eval() >>> Parser(['(','5','*','2',')','*','3']).parse().eval() >>> Parser(['5','*','2','*','4']).parse().eval() 12 40

52 cfg Parsing Recursive Descent Parsing Calculator 28 Calculator Two steps: Parse the input into something which can be evaluated Evaluate the returned object Build an object tree while parsing! Here s the grammar we will use in this example: 1 <e1> ::= <e2> ( "+" <e1> )? 2 <e2> ::= <e3> ( "*" <e2> )? 3 <e3> ::= "-"? ( "0" "1"... "9" )+ 4 <e3> ::= "(" <e1> ")"

53 cfg Parsing Recursive Descent Parsing Calculator 28 Calculator Two steps: Parse the input into something which can be evaluated Evaluate the returned object Build an object tree while parsing! Here s the grammar we will use in this example: 1 <e1> ::= <e2> ( "+" <e1> )? 2 <e2> ::= <e3> ( "*" <e2> )? 3 <e3> ::= "-"? ( "0" "1"... "9" )+ 4 <e3> ::= "(" <e1> ")" 1,2: "+" will be evaluated after "*"

54 cfg Parsing Recursive Descent Parsing Calculator 28 Calculator Two steps: Parse the input into something which can be evaluated Evaluate the returned object Build an object tree while parsing! Here s the grammar we will use in this example: 1 <e1> ::= <e2> ( "+" <e1> )? 2 <e2> ::= <e3> ( "*" <e2> )? 3 <e3> ::= "-"? ( "0" "1"... "9" )+ 4 <e3> ::= "(" <e1> ")" 3,4: <e3> is either an integer or a compound expression

55 cfg Parsing Recursive Descent Parsing Calculator 29 Expression Tree: Abstract Base Class Composite design expression tree We need an abstract base class 54 class Node: 55 def init (self, left, right): 56 self.left = left 57 self.right = right def eval(self): 60 raise NotImplementedError()

56 cfg Parsing Recursive Descent Parsing Calculator 30 Expression Tree: Concrete Subclasses We need concrete subclasses for each type of node 62 class AddNode(Node): 63 def eval(self): 64 return self.left.eval() + self.right.eval() class MultNode(Node): 67 def eval(self): 68 return self.left.eval() * self.right.eval() class LiteralNode(Node): 71 def init (self, value): 72 super(). init (None, None) 73 self.value = value def eval(self): 76 return self.value

57 cfg Parsing Recursive Descent Parsing Calculator 31 Step 1: Basic parsing framework 3 class Parser: 4 RE_NUMBER = re.compile(r'-?[0-9]+') 5 6 def init (self, tokens): 7 self._tokens = tokens 8 self._length = len(tokens) 9 self._upto = def end(self): 12 return self._upto == self._length def peek(self): 15 return None if self.end() else self._tokens[self._upto] def next(self): 18 if not self.end(): 19 self._upto += 1

58 cfg Parsing Recursive Descent Parsing Calculator 32 Step 2: Helper method for each non-terminal (<e3>) <e3> ::= "-"? ( "0" "1"... "9" )+ <e3> ::= "(" <e1> ")" 37 def _parse_e3(self): 38 node = None 39 if self.peek() == '(': 40 self.next() 41 node = self._parse_e1() 42 if self.peek()!= ')': 43 raise Exception('Closing parenthesis not found!') 44 self.next() 45 elif Parser.RE_NUMBER.match(self.peek()): 46 node = LiteralNode(int(self.peek())) 47 self.next() 48 return node

59 cfg Parsing Recursive Descent Parsing Calculator 33 Step 2: Helper method for each non-terminal (<e2>) <e2> ::= <e3> ( "*" <e2> )? 29 def _parse_e2(self): 30 node = self._parse_e3() 31 if self.peek() == '*': 32 self.next() 33 node2 = self._parse_e2() 34 node = MultNode(node, node2) 35 return node

60 cfg Parsing Recursive Descent Parsing Calculator 34 Step 2: Helper method for each non-terminal (<e1>) <e1> ::= <e2> ( "+" <e1> )? 21 def _parse_e1(self): 22 node = self._parse_e2() 23 if self.peek() == '+': 24 self.next() 25 node2 = self._parse_e1() 26 node = AddNode(node, node2) 27 return node

61 cfg Parsing Recursive Descent Parsing Calculator 35 Step 3: Wrap the start-symbol parsing helper method 50 def parse(self): 51 node = self._parse_e1() 52 if not self.end(): 53 raise Exception('Extra content found at end of input!') 54 return node

62 cfg Parsing Recursive Descent Parsing Calculator 36 Step 4: Test it 1 >>> Parser(['3','+','2']).parse().eval() >>> Parser(['3','*','2']).parse().eval() >>> Parser(['3','*','2','+','4']).parse().eval() >>> Parser(['3','+','2','*','4']).parse().eval() >>> Parser(['(','5','*','2',')','*','3']).parse().eval() >>> Parser(['5','*','2','*','4']).parse().eval() >>> Parser(['5','+','2','+','4']).parse().eval() 14 11

Syntax and Grammars 1 / 21

Syntax and Grammars 1 / 21 Syntax and Grammars 1 / 21 Outline What is a language? Abstract syntax and grammars Abstract syntax vs. concrete syntax Encoding grammars as Haskell data types What is a language? 2 / 21 What is a language?

More information

Grammars & Parsing. Lecture 12 CS 2112 Fall 2018

Grammars & Parsing. Lecture 12 CS 2112 Fall 2018 Grammars & Parsing Lecture 12 CS 2112 Fall 2018 Motivation The cat ate the rat. The cat ate the rat slowly. The small cat ate the big rat slowly. The small cat ate the big rat on the mat slowly. The small

More information

Specifying Syntax. An English Grammar. Components of a Grammar. Language Specification. Types of Grammars. 1. Terminal symbols or terminals, Σ

Specifying Syntax. An English Grammar. Components of a Grammar. Language Specification. Types of Grammars. 1. Terminal symbols or terminals, Σ Specifying Syntax Language Specification Components of a Grammar 1. Terminal symbols or terminals, Σ Syntax Form of phrases Physical arrangement of symbols 2. Nonterminal symbols or syntactic categories,

More information

Parsing. Parsing. Bottom Up Parsing. Bottom Up Parsing. Bottom Up Parsing. Bottom Up Parsing

Parsing. Parsing. Bottom Up Parsing. Bottom Up Parsing. Bottom Up Parsing. Bottom Up Parsing Parsing Determine if an input string is a sentence of G. G is a context free grammar (later). Assumed to be unambiguous. Recognition of the string plus determination of phrase structure. We constantly

More information

Principles of Programming Languages COMP251: Syntax and Grammars

Principles of Programming Languages COMP251: Syntax and Grammars Principles of Programming Languages COMP251: Syntax and Grammars Prof. Dekai Wu Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China Fall 2006

More information

Non-deterministic Finite Automata (NFA)

Non-deterministic Finite Automata (NFA) Non-deterministic Finite Automata (NFA) CAN have transitions on the same input to different states Can include a ε or λ transition (i.e. move to new state without reading input) Often easier to design

More information

COL728 Minor1 Exam Compiler Design Sem II, Answer all 5 questions Max. Marks: 20

COL728 Minor1 Exam Compiler Design Sem II, Answer all 5 questions Max. Marks: 20 COL728 Minor1 Exam Compiler Design Sem II, 2016-17 Answer all 5 questions Max. Marks: 20 1. Short questions a. Show that every regular language is also a context-free language [2] We know that every regular

More information

If you are going to form a group for A2, please do it before tomorrow (Friday) noon GRAMMARS & PARSING. Lecture 8 CS2110 Spring 2014

If you are going to form a group for A2, please do it before tomorrow (Friday) noon GRAMMARS & PARSING. Lecture 8 CS2110 Spring 2014 1 If you are going to form a group for A2, please do it before tomorrow (Friday) noon GRAMMARS & PARSING Lecture 8 CS2110 Spring 2014 Pointers. DO visit the java spec website 2 Parse trees: Text page 592

More information

Public-Service Announcement

Public-Service Announcement Public-Service Announcement Hi CS 61A students! Did you know that the university has a farm? The UC Gill Tract Community Farm is located just two miles from campus and is a community space, open to all.

More information

Compiler Design Concepts. Syntax Analysis

Compiler Design Concepts. Syntax Analysis Compiler Design Concepts Syntax Analysis Introduction First task is to break up the text into meaningful words called tokens. newval=oldval+12 id = id + num Token Stream Lexical Analysis Source Code (High

More information

Today. Assignments. Lecture Notes CPSC 326 (Spring 2019) Quiz 2. Lexer design. Syntax Analysis: Context-Free Grammars. HW2 (out, due Tues)

Today. Assignments. Lecture Notes CPSC 326 (Spring 2019) Quiz 2. Lexer design. Syntax Analysis: Context-Free Grammars. HW2 (out, due Tues) Today Quiz 2 Lexer design Syntax Analysis: Context-Free Grammars Assignments HW2 (out, due Tues) S. Bowers 1 of 15 Implementing a Lexer for MyPL (HW 2) Similar in spirit to HW 1 We ll create three classes:

More information

Introduction to Parsing. Lecture 8

Introduction to Parsing. Lecture 8 Introduction to Parsing Lecture 8 Adapted from slides by G. Necula Outline Limitations of regular languages Parser overview Context-free grammars (CFG s) Derivations Languages and Automata Formal languages

More information

Syntax Analysis. Prof. James L. Frankel Harvard University. Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved.

Syntax Analysis. Prof. James L. Frankel Harvard University. Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved. Syntax Analysis Prof. James L. Frankel Harvard University Version of 6:43 PM 6-Feb-2018 Copyright 2018, 2015 James L. Frankel. All rights reserved. Context-Free Grammar (CFG) terminals non-terminals start

More information

CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages CMSC 330: Organization of Programming Languages Context Free Grammars and Parsing 1 Recall: Architecture of Compilers, Interpreters Source Parser Static Analyzer Intermediate Representation Front End Back

More information

Lexical Analysis. COMP 524, Spring 2014 Bryan Ward

Lexical Analysis. COMP 524, Spring 2014 Bryan Ward Lexical Analysis COMP 524, Spring 2014 Bryan Ward Based in part on slides and notes by J. Erickson, S. Krishnan, B. Brandenburg, S. Olivier, A. Block and others The Big Picture Character Stream Scanner

More information

([1-9] 1[0-2]):[0-5][0-9](AM PM)? What does the above match? Matches clock time, may or may not be told if it is AM or PM.

([1-9] 1[0-2]):[0-5][0-9](AM PM)? What does the above match? Matches clock time, may or may not be told if it is AM or PM. What is the corresponding regex? [2-9]: ([1-9] 1[0-2]):[0-5][0-9](AM PM)? What does the above match? Matches clock time, may or may not be told if it is AM or PM. CS 230 - Spring 2018 4-1 More CFG Notation

More information

Principles of Programming Languages COMP251: Syntax and Grammars

Principles of Programming Languages COMP251: Syntax and Grammars Principles of Programming Languages COMP251: Syntax and Grammars Prof. Dekai Wu Department of Computer Science and Engineering The Hong Kong University of Science and Technology Hong Kong, China Fall 2007

More information

ECE251 Midterm practice questions, Fall 2010

ECE251 Midterm practice questions, Fall 2010 ECE251 Midterm practice questions, Fall 2010 Patrick Lam October 20, 2010 Bootstrapping In particular, say you have a compiler from C to Pascal which runs on x86, and you want to write a self-hosting Java

More information

COP 3402 Systems Software Top Down Parsing (Recursive Descent)

COP 3402 Systems Software Top Down Parsing (Recursive Descent) COP 3402 Systems Software Top Down Parsing (Recursive Descent) Top Down Parsing 1 Outline 1. Top down parsing and LL(k) parsing 2. Recursive descent parsing 3. Example of recursive descent parsing of arithmetic

More information

Languages and Compilers

Languages and Compilers Principles of Software Engineering and Operational Systems Languages and Compilers SDAGE: Level I 2012-13 3. Formal Languages, Grammars and Automata Dr Valery Adzhiev vadzhiev@bournemouth.ac.uk Office:

More information

Briefly describe the purpose of the lexical and syntax analysis phases in a compiler.

Briefly describe the purpose of the lexical and syntax analysis phases in a compiler. Name: Midterm Exam PID: This is a closed-book exam; you may not use any tools besides a pen. You have 75 minutes to answer all questions. There are a total of 75 points available. Please write legibly;

More information

Course Overview. Introduction (Chapter 1) Compiler Frontend: Today. Compiler Backend:

Course Overview. Introduction (Chapter 1) Compiler Frontend: Today. Compiler Backend: Course Overview Introduction (Chapter 1) Compiler Frontend: Today Lexical Analysis & Parsing (Chapter 2,3,4) Semantic Analysis (Chapter 5) Activation Records (Chapter 6) Translation to Intermediate Code

More information

Syntax. Syntax. We will study three levels of syntax Lexical Defines the rules for tokens: literals, identifiers, etc.

Syntax. Syntax. We will study three levels of syntax Lexical Defines the rules for tokens: literals, identifiers, etc. Syntax Syntax Syntax defines what is grammatically valid in a programming language Set of grammatical rules E.g. in English, a sentence cannot begin with a period Must be formal and exact or there will

More information

Syntax-Directed Translation. Lecture 14

Syntax-Directed Translation. Lecture 14 Syntax-Directed Translation Lecture 14 (adapted from slides by R. Bodik) 9/27/2006 Prof. Hilfinger, Lecture 14 1 Motivation: parser as a translator syntax-directed translation stream of tokens parser ASTs,

More information

Regexs with DFA and Parse Trees. CS230 Tutorial 11

Regexs with DFA and Parse Trees. CS230 Tutorial 11 Regexs with DFA and Parse Trees CS230 Tutorial 11 Regular Expressions (Regex) This way of representing regular languages using metacharacters. Here are some of the most important ones to know: -- OR example:

More information

Binary Tree Application Expression Tree. Revised based on textbook author s notes.

Binary Tree Application Expression Tree. Revised based on textbook author s notes. Binary Tree Application Expression Tree Revised based on textbook author s notes. Expression Trees A binary tree in which the operators are stored in the interior nodes and the operands are sored in the

More information

ADTS, GRAMMARS, PARSING, TREE TRAVERSALS

ADTS, GRAMMARS, PARSING, TREE TRAVERSALS 1 Pointers to material ADS, GRAMMARS, PARSING, R RAVRSALS Lecture 13 CS110 all 016 Parse trees: text, section 3.36 Definition of Java Language, sometimes useful: docs.oracle.com/javase/specs/jls/se8/html/index.html

More information

ADTS, GRAMMARS, PARSING, TREE TRAVERSALS

ADTS, GRAMMARS, PARSING, TREE TRAVERSALS 3//15 1 AD: Abstract Data ype 2 Just like a type: Bunch of values together with operations on them. Used often in discussing data structures Important: he definition says ntthing about the implementation,

More information

CSE431 Translation of Computer Languages

CSE431 Translation of Computer Languages CSE431 Translation of Computer Languages Top Down Parsers Doug Shook Top Down Parsers Two forms: Recursive Descent Table Also known as LL(k) parsers: Read tokens from Left to right Produces a Leftmost

More information

Outline. Limitations of regular languages. Introduction to Parsing. Parser overview. Context-free grammars (CFG s)

Outline. Limitations of regular languages. Introduction to Parsing. Parser overview. Context-free grammars (CFG s) Outline Limitations of regular languages Introduction to Parsing Parser overview Lecture 8 Adapted from slides by G. Necula Context-free grammars (CFG s) Derivations Languages and Automata Formal languages

More information

ADTS, GRAMMARS, PARSING, TREE TRAVERSALS. Regrades 10/6/15. Prelim 1. Prelim 1. Expression trees. Pointers to material

ADTS, GRAMMARS, PARSING, TREE TRAVERSALS. Regrades 10/6/15. Prelim 1. Prelim 1. Expression trees. Pointers to material 1 Prelim 1 2 Max: 99 Mean: 71.2 Median: 73 Std Dev: 1.6 ADS, GRAMMARS, PARSING, R RAVRSALS Lecture 12 CS2110 Spring 2015 3 Prelim 1 Score Grade % 90-99 A 82-89 A-/A 26% 70-82 B/B 62-69 B-/B 50% 50-59 C-/C

More information

GRAMMARS & PARSING. Lecture 7 CS2110 Fall 2013

GRAMMARS & PARSING. Lecture 7 CS2110 Fall 2013 1 GRAMMARS & PARSING Lecture 7 CS2110 Fall 2013 Pointers to the textbook 2 Parse trees: Text page 592 (23.34), Figure 23-31 Definition of Java Language, sometimes useful: http://docs.oracle.com/javase/specs/jls/se7/html/index.html

More information

CMPT 755 Compilers. Anoop Sarkar.

CMPT 755 Compilers. Anoop Sarkar. CMPT 755 Compilers Anoop Sarkar http://www.cs.sfu.ca/~anoop Parsing source program Lexical Analyzer token next() Parser parse tree Later Stages Lexical Errors Syntax Errors Context-free Grammars Set of

More information

ADTS, GRAMMARS, PARSING, TREE TRAVERSALS

ADTS, GRAMMARS, PARSING, TREE TRAVERSALS 1 Prelim 1 2 Where: Kennedy Auditorium When: A-Lib: 5:30-7 Lie-Z: 7:30-9 (unless we explicitly notified you otherwise) ADS, GRAMMARS, PARSING, R RAVRSALS Lecture 13 CS2110 Spring 2016 Pointers to material

More information

COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages. Syntax Prof. Robert van Engelen COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview n Tokens and regular expressions n Syntax and context-free grammars n Grammar derivations n More about parse trees n Top-down and

More information

Defining Program Syntax. Chapter Two Modern Programming Languages, 2nd ed. 1

Defining Program Syntax. Chapter Two Modern Programming Languages, 2nd ed. 1 Defining Program Syntax Chapter Two Modern Programming Languages, 2nd ed. 1 Syntax And Semantics Programming language syntax: how programs look, their form and structure Syntax is defined using a kind

More information

First Midterm Exam CS164, Fall 2007 Oct 2, 2007

First Midterm Exam CS164, Fall 2007 Oct 2, 2007 P a g e 1 First Midterm Exam CS164, Fall 2007 Oct 2, 2007 Please read all instructions (including these) carefully. Write your name, login, and SID. No electronic devices are allowed, including cell phones

More information

announcements CSE 311: Foundations of Computing review: regular expressions review: languages---sets of strings

announcements CSE 311: Foundations of Computing review: regular expressions review: languages---sets of strings CSE 311: Foundations of Computing Fall 2013 Lecture 19: Regular expressions & context-free grammars announcements Reading assignments 7 th Edition, pp. 878-880 and pp. 851-855 6 th Edition, pp. 817-819

More information

COP4020 Programming Languages. Syntax Prof. Robert van Engelen

COP4020 Programming Languages. Syntax Prof. Robert van Engelen COP4020 Programming Languages Syntax Prof. Robert van Engelen Overview Tokens and regular expressions Syntax and context-free grammars Grammar derivations More about parse trees Top-down and bottom-up

More information

CS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F).

CS 2210 Sample Midterm. 1. Determine if each of the following claims is true (T) or false (F). CS 2210 Sample Midterm 1. Determine if each of the following claims is true (T) or false (F). F A language consists of a set of strings, its grammar structure, and a set of operations. (Note: a language

More information

CS 314 Principles of Programming Languages

CS 314 Principles of Programming Languages CS 314 Principles of Programming Languages Lecture 5: Syntax Analysis (Parsing) Zheng (Eddy) Zhang Rutgers University January 31, 2018 Class Information Homework 1 is being graded now. The sample solution

More information

COP 3402 Systems Software Syntax Analysis (Parser)

COP 3402 Systems Software Syntax Analysis (Parser) COP 3402 Systems Software Syntax Analysis (Parser) Syntax Analysis 1 Outline 1. Definition of Parsing 2. Context Free Grammars 3. Ambiguous/Unambiguous Grammars Syntax Analysis 2 Lexical and Syntax Analysis

More information

Syntactic Analysis. The Big Picture Again. Grammar. ICS312 Machine-Level and Systems Programming

Syntactic Analysis. The Big Picture Again. Grammar. ICS312 Machine-Level and Systems Programming The Big Picture Again Syntactic Analysis source code Scanner Parser Opt1 Opt2... Optn Instruction Selection Register Allocation Instruction Scheduling machine code ICS312 Machine-Level and Systems Programming

More information

CS 4240: Compilers and Interpreters Project Phase 1: Scanner and Parser Due Date: October 4 th 2015 (11:59 pm) (via T-square)

CS 4240: Compilers and Interpreters Project Phase 1: Scanner and Parser Due Date: October 4 th 2015 (11:59 pm) (via T-square) CS 4240: Compilers and Interpreters Project Phase 1: Scanner and Parser Due Date: October 4 th 2015 (11:59 pm) (via T-square) Introduction This semester, through a project split into 3 phases, we are going

More information

Introduction; Parsing LL Grammars

Introduction; Parsing LL Grammars Introduction; Parsing LL Grammars CS 440: Programming Languages and Translators Due Fri Feb 2, 11:59 pm 1/29 pp.1, 2; 2/7 all updates incorporated, solved Instructions You can work together in groups of

More information

Properties of Regular Expressions and Finite Automata

Properties of Regular Expressions and Finite Automata Properties of Regular Expressions and Finite Automata Some token patterns can t be defined as regular expressions or finite automata. Consider the set of balanced brackets of the form [[[ ]]]. This set

More information

CPS 506 Comparative Programming Languages. Syntax Specification

CPS 506 Comparative Programming Languages. Syntax Specification CPS 506 Comparative Programming Languages Syntax Specification Compiling Process Steps Program Lexical Analysis Convert characters into a stream of tokens Lexical Analysis Syntactic Analysis Send tokens

More information

CMSC 330: Organization of Programming Languages. Architecture of Compilers, Interpreters

CMSC 330: Organization of Programming Languages. Architecture of Compilers, Interpreters : Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Scanner Parser Static Analyzer Intermediate Representation Front End Back End Compiler / Interpreter

More information

Announcements. Application of Recursion. Motivation. A Grammar. A Recursive Grammar. Grammars and Parsing

Announcements. Application of Recursion. Motivation. A Grammar. A Recursive Grammar. Grammars and Parsing Grammars and Lecture 5 CS211 Fall 2005 CMS is being cleaned If you did not turn in A1 and haven t already contacted David Schwartz, you need to contact him now Java Reminder Any significant class should

More information

Homework & Announcements

Homework & Announcements Homework & nnouncements New schedule on line. Reading: Chapter 18 Homework: Exercises at end Due: 11/1 Copyright c 2002 2017 UMaine School of Computing and Information S 1 / 25 COS 140: Foundations of

More information

CSE 130 Programming Language Principles & Paradigms Lecture # 5. Chapter 4 Lexical and Syntax Analysis

CSE 130 Programming Language Principles & Paradigms Lecture # 5. Chapter 4 Lexical and Syntax Analysis Chapter 4 Lexical and Syntax Analysis Introduction - Language implementation systems must analyze source code, regardless of the specific implementation approach - Nearly all syntax analysis is based on

More information

CSE450 Translation of Programming Languages. Lecture 4: Syntax Analysis

CSE450 Translation of Programming Languages. Lecture 4: Syntax Analysis CSE450 Translation of Programming Languages Lecture 4: Syntax Analysis http://xkcd.com/859 Structure of a Today! Compiler Source Language Lexical Analyzer Syntax Analyzer Semantic Analyzer Int. Code Generator

More information

Related Course Objec6ves

Related Course Objec6ves Syntax 9/18/17 1 Related Course Objec6ves Develop grammars and parsers of programming languages 9/18/17 2 Syntax And Seman6cs Programming language syntax: how programs look, their form and structure Syntax

More information

Types of parsing. CMSC 430 Lecture 4, Page 1

Types of parsing. CMSC 430 Lecture 4, Page 1 Types of parsing Top-down parsers start at the root of derivation tree and fill in picks a production and tries to match the input may require backtracking some grammars are backtrack-free (predictive)

More information

CS 315 Programming Languages Syntax. Parser. (Alternatively hand-built) (Alternatively hand-built)

CS 315 Programming Languages Syntax. Parser. (Alternatively hand-built) (Alternatively hand-built) Programming languages must be precise Remember instructions This is unlike natural languages CS 315 Programming Languages Syntax Precision is required for syntax think of this as the format of the language

More information

EECS 6083 Intro to Parsing Context Free Grammars

EECS 6083 Intro to Parsing Context Free Grammars EECS 6083 Intro to Parsing Context Free Grammars Based on slides from text web site: Copyright 2003, Keith D. Cooper, Ken Kennedy & Linda Torczon, all rights reserved. 1 Parsing sequence of tokens parser

More information

CMSC 201 Fall 2016 Lab 09 Advanced Debugging

CMSC 201 Fall 2016 Lab 09 Advanced Debugging CMSC 201 Fall 2016 Lab 09 Advanced Debugging Assignment: Lab 09 Advanced Debugging Due Date: During discussion Value: 10 points Part 1: Introduction to Errors Throughout this semester, we have been working

More information

Compiler Construction

Compiler Construction Compiler Construction Collection of exercises Version February 7, 26 Abbreviations NFA. Non-deterministic finite automaton DFA. Deterministic finite automaton Lexical analysis. Construct deterministic

More information

ITEC2620 Introduction to Data Structures

ITEC2620 Introduction to Data Structures ITEC2620 Introduction to Data Structures Lecture 10a Grammars II Review Three main problems Derive a sentence from a grammar Develop a grammar for a language Given a grammar, determine if an input is valid

More information

Finite State Automata are Limited. Let us use (context-free) grammars!

Finite State Automata are Limited. Let us use (context-free) grammars! Finite State Automata are Limited Let us use (context-free) grammars! Context Free Grammar for a n b n S ::= - a grammar rule S ::= a S b - another grammar rule Example of a derivation S => asb => a asb

More information

Question Bank. 10CS63:Compiler Design

Question Bank. 10CS63:Compiler Design Question Bank 10CS63:Compiler Design 1.Determine whether the following regular expressions define the same language? (ab)* and a*b* 2.List the properties of an operator grammar 3. Is macro processing a

More information

CSC148 Intro. to Computer Science

CSC148 Intro. to Computer Science CSC148 Intro. to Computer Science Lecture 4: Container implementation, Unit Test, Balanced Parentheses, Intro to Linked Lists Amir H. Chinaei, Summer 2016 Office Hours: R 10-12 BA4222 ahchinaei@cs.toronto.edu

More information

Optimizing Finite Automata

Optimizing Finite Automata Optimizing Finite Automata We can improve the DFA created by MakeDeterministic. Sometimes a DFA will have more states than necessary. For every DFA there is a unique smallest equivalent DFA (fewest states

More information

CSE 3302 Programming Languages Lecture 2: Syntax

CSE 3302 Programming Languages Lecture 2: Syntax CSE 3302 Programming Languages Lecture 2: Syntax (based on slides by Chengkai Li) Leonidas Fegaras University of Texas at Arlington CSE 3302 L2 Spring 2011 1 How do we define a PL? Specifying a PL: Syntax:

More information

Test I Solutions MASSACHUSETTS INSTITUTE OF TECHNOLOGY Spring Department of Electrical Engineering and Computer Science

Test I Solutions MASSACHUSETTS INSTITUTE OF TECHNOLOGY Spring Department of Electrical Engineering and Computer Science Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.035 Spring 2013 Test I Solutions Mean 83 Median 87 Std. dev 13.8203 14 12 10 8 6 4 2 0 0 10 20 30 40 50

More information

CMSC 330: Organization of Programming Languages. Context Free Grammars

CMSC 330: Organization of Programming Languages. Context Free Grammars CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

More information

Syntax and Semantics

Syntax and Semantics Syntax and Semantics Syntax - The form or structure of the expressions, statements, and program units Semantics - The meaning of the expressions, statements, and program units Syntax Example: simple C

More information

Architecture of Compilers, Interpreters. CMSC 330: Organization of Programming Languages. Front End Scanner and Parser. Implementing the Front End

Architecture of Compilers, Interpreters. CMSC 330: Organization of Programming Languages. Front End Scanner and Parser. Implementing the Front End Architecture of Compilers, Interpreters : Organization of Programming Languages ource Analyzer Optimizer Code Generator Context Free Grammars Intermediate Representation Front End Back End Compiler / Interpreter

More information

Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing

Parsing. Roadmap. > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing Roadmap > Context-free grammars > Derivations and precedence > Top-down parsing > Left-recursion > Look-ahead > Table-driven parsing The role of the parser > performs context-free syntax analysis > guides

More information

UVa ID: NAME (print): CS 4501 LDI Midterm 1

UVa ID: NAME (print): CS 4501 LDI Midterm 1 CS 4501 LDI Midterm 1 Write your name and UVa ID on the exam. Pledge the exam before turning it in. There are nine (9) pages in this exam (including this one) and six (6) questions, each with multiple

More information

Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part 1

Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part 1 Chapter 3: CONTEXT-FREE GRAMMARS AND PARSING Part 1 1. Introduction Parsing is the task of Syntax Analysis Determining the syntax, or structure, of a program. The syntax is defined by the grammar rules

More information

Syntax Analysis. Chapter 4

Syntax Analysis. Chapter 4 Syntax Analysis Chapter 4 Check (Important) http://www.engineersgarage.com/contributio n/difference-between-compiler-andinterpreter Introduction covers the major parsing methods that are typically used

More information

Where We Are. CMSC 330: Organization of Programming Languages. This Lecture. Programming Languages. Motivation for Grammars

Where We Are. CMSC 330: Organization of Programming Languages. This Lecture. Programming Languages. Motivation for Grammars CMSC 330: Organization of Programming Languages Context Free Grammars Where We Are Programming languages Ruby OCaml Implementing programming languages Scanner Uses regular expressions Finite automata Parser

More information

Part 5 Program Analysis Principles and Techniques

Part 5 Program Analysis Principles and Techniques 1 Part 5 Program Analysis Principles and Techniques Front end 2 source code scanner tokens parser il errors Responsibilities: Recognize legal programs Report errors Produce il Preliminary storage map Shape

More information

MATVEC: MATRIX-VECTOR COMPUTATION LANGUAGE REFERENCE MANUAL. John C. Murphy jcm2105 Programming Languages and Translators Professor Stephen Edwards

MATVEC: MATRIX-VECTOR COMPUTATION LANGUAGE REFERENCE MANUAL. John C. Murphy jcm2105 Programming Languages and Translators Professor Stephen Edwards MATVEC: MATRIX-VECTOR COMPUTATION LANGUAGE REFERENCE MANUAL John C. Murphy jcm2105 Programming Languages and Translators Professor Stephen Edwards Language Reference Manual Introduction The purpose of

More information

Parsing Techniques. CS152. Chris Pollett. Sep. 24, 2008.

Parsing Techniques. CS152. Chris Pollett. Sep. 24, 2008. Parsing Techniques. CS152. Chris Pollett. Sep. 24, 2008. Outline. Top-down versus Bottom-up Parsing. Recursive Descent Parsing. Left Recursion Removal. Left Factoring. Predictive Parsing. Introduction.

More information

CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

More information

MIT Specifying Languages with Regular Expressions and Context-Free Grammars

MIT Specifying Languages with Regular Expressions and Context-Free Grammars MIT 6.035 Specifying Languages with Regular essions and Context-Free Grammars Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology Language Definition Problem How to precisely

More information

Wednesday, August 31, Parsers

Wednesday, August 31, Parsers Parsers How do we combine tokens? Combine tokens ( words in a language) to form programs ( sentences in a language) Not all combinations of tokens are correct programs (not all sentences are grammatically

More information

Parsing Combinators: Introduction & Tutorial

Parsing Combinators: Introduction & Tutorial Parsing Combinators: Introduction & Tutorial Mayer Goldberg October 21, 2017 Contents 1 Synopsis 1 2 Backus-Naur Form (BNF) 2 3 Parsing Combinators 3 4 Simple constructors 4 5 The parser stack 6 6 Recursive

More information

CSCI312 Principles of Programming Languages

CSCI312 Principles of Programming Languages Copyright 2006 The McGraw-Hill Companies, Inc. CSCI312 Principles of Programming Languages! LL Parsing!! Xu Liu Derived from Keith Cooper s COMP 412 at Rice University Recap Copyright 2006 The McGraw-Hill

More information

A programming language requires two major definitions A simple one pass compiler

A programming language requires two major definitions A simple one pass compiler A programming language requires two major definitions A simple one pass compiler [Syntax: what the language looks like A context-free grammar written in BNF (Backus-Naur Form) usually suffices. [Semantics:

More information

Parsing III. (Top-down parsing: recursive descent & LL(1) )

Parsing III. (Top-down parsing: recursive descent & LL(1) ) Parsing III (Top-down parsing: recursive descent & LL(1) ) Roadmap (Where are we?) Previously We set out to study parsing Specifying syntax Context-free grammars Ambiguity Top-down parsers Algorithm &

More information

Chapter 4. Lexical and Syntax Analysis

Chapter 4. Lexical and Syntax Analysis Chapter 4 Lexical and Syntax Analysis Chapter 4 Topics Introduction Lexical Analysis The Parsing Problem Recursive-Descent Parsing Bottom-Up Parsing Copyright 2012 Addison-Wesley. All rights reserved.

More information

CSE 413 Final Exam. June 7, 2011

CSE 413 Final Exam. June 7, 2011 CSE 413 Final Exam June 7, 2011 Name The exam is closed book, except that you may have a single page of hand-written notes for reference plus the page of notes you had for the midterm (although you are

More information

Topic 3: Syntax Analysis I

Topic 3: Syntax Analysis I Topic 3: Syntax Analysis I Compiler Design Prof. Hanjun Kim CoreLab (Compiler Research Lab) POSTECH 1 Back-End Front-End The Front End Source Program Lexical Analysis Syntax Analysis Semantic Analysis

More information

SYED AMMAL ENGINEERING COLLEGE (An ISO 9001:2008 Certified Institution) Dr. E.M. Abdullah Campus, Ramanathapuram

SYED AMMAL ENGINEERING COLLEGE (An ISO 9001:2008 Certified Institution) Dr. E.M. Abdullah Campus, Ramanathapuram CS6660 COMPILER DESIGN Question Bank UNIT I-INTRODUCTION TO COMPILERS 1. Define compiler. 2. Differentiate compiler and interpreter. 3. What is a language processing system? 4. List four software tools

More information

Building a Parser III. CS164 3:30-5:00 TT 10 Evans. Prof. Bodik CS 164 Lecture 6 1

Building a Parser III. CS164 3:30-5:00 TT 10 Evans. Prof. Bodik CS 164 Lecture 6 1 Building a Parser III CS164 3:30-5:00 TT 10 Evans 1 Overview Finish recursive descent parser when it breaks down and how to fix it eliminating left recursion reordering productions Predictive parsers (aka

More information

Stacks. Revised based on textbook author s notes.

Stacks. Revised based on textbook author s notes. Stacks Revised based on textbook author s notes. Stacks A restricted access container that stores a linear collection. Very common for solving problems in computer science. Provides a last-in first-out

More information

LL(k) Parsing. Predictive Parsers. LL(k) Parser Structure. Sample Parse Table. LL(1) Parsing Algorithm. Push RHS in Reverse Order 10/17/2012

LL(k) Parsing. Predictive Parsers. LL(k) Parser Structure. Sample Parse Table. LL(1) Parsing Algorithm. Push RHS in Reverse Order 10/17/2012 Predictive Parsers LL(k) Parsing Can we avoid backtracking? es, if for a given input symbol and given nonterminal, we can choose the alternative appropriately. his is possible if the first terminal of

More information

Regular Expressions. Agenda for Today. Grammar for a Tiny Language. Programming Language Specifications

Regular Expressions. Agenda for Today. Grammar for a Tiny Language. Programming Language Specifications Agenda for Today Regular Expressions CSE 413, Autumn 2005 Programming Languages Basic concepts of formal grammars Regular expressions Lexical specification of programming languages Using finite automata

More information

Midterm I - Solution CS164, Spring 2014

Midterm I - Solution CS164, Spring 2014 164sp14 Midterm 1 - Solution Midterm I - Solution CS164, Spring 2014 March 3, 2014 Please read all instructions (including these) carefully. This is a closed-book exam. You are allowed a one-page handwritten

More information

ΕΠΛ323 - Θεωρία και Πρακτική Μεταγλωττιστών

ΕΠΛ323 - Θεωρία και Πρακτική Μεταγλωττιστών ΕΠΛ323 - Θεωρία και Πρακτική Μεταγλωττιστών Lecture 5a Syntax Analysis lias Athanasopoulos eliasathan@cs.ucy.ac.cy Syntax Analysis Συντακτική Ανάλυση Context-free Grammars (CFGs) Derivations Parse trees

More information

CMSC 330: Organization of Programming Languages

CMSC 330: Organization of Programming Languages CMSC 330: Organization of Programming Languages Context Free Grammars 1 Architecture of Compilers, Interpreters Source Analyzer Optimizer Code Generator Abstract Syntax Tree Front End Back End Compiler

More information

CS1622. Today. A Recursive Descent Parser. Preliminaries. Lecture 9 Parsing (4)

CS1622. Today. A Recursive Descent Parser. Preliminaries. Lecture 9 Parsing (4) CS1622 Lecture 9 Parsing (4) CS 1622 Lecture 9 1 Today Example of a recursive descent parser Predictive & LL(1) parsers Building parse tables CS 1622 Lecture 9 2 A Recursive Descent Parser. Preliminaries

More information

Course Project 2 Regular Expressions

Course Project 2 Regular Expressions Course Project 2 Regular Expressions CSE 30151 Spring 2017 Version of February 16, 2017 In this project, you ll write a regular expression matcher similar to grep, called mere (for match and echo using

More information

CS153: Compilers Lecture 4: Recursive Parsing

CS153: Compilers Lecture 4: Recursive Parsing CS153: Compilers Lecture 4: Recursive Parsing Stephen Chong https://www.seas.harvard.edu/courses/cs153 Announcements Proj 1 out Due Thursday Sept 20 11:59 Start soon if you haven t! Proj 2 released today

More information

The Parsing Problem (cont d) Recursive-Descent Parsing. Recursive-Descent Parsing (cont d) ICOM 4036 Programming Languages. The Complexity of Parsing

The Parsing Problem (cont d) Recursive-Descent Parsing. Recursive-Descent Parsing (cont d) ICOM 4036 Programming Languages. The Complexity of Parsing ICOM 4036 Programming Languages Lexical and Syntax Analysis Lexical Analysis The Parsing Problem Recursive-Descent Parsing Bottom-Up Parsing This lecture covers review questions 14-27 This lecture covers

More information

CSCE 314 Programming Languages. Functional Parsers

CSCE 314 Programming Languages. Functional Parsers CSCE 314 Programming Languages Functional Parsers Dr. Hyunyoung Lee 1 What is a Parser? A parser is a program that takes a text (set of tokens) and determines its syntactic structure. String or [Token]

More information

Context-Free Grammar (CFG)

Context-Free Grammar (CFG) Context-Free Grammar (CFG) context-free grammar looks like this bunch of rules: ain idea: + 1 (),, are non-terminal symbols aka variables. When you see them, you apply rules to expand. One of them is designated

More information