LR(0) Parsers. CSCI 3130 Formal Languages and Automata Theory. Siu On CHAN. Fall Chinese University of Hong Kong 1/31

Size: px

Start display at page:

Download "LR(0) Parsers. CSCI 3130 Formal Languages and Automata Theory. Siu On CHAN. Fall Chinese University of Hong Kong 1/31"

Randell O’Connor’
5 years ago
Views:

1 LR(0) Parsers CSCI 3130 Formal Languages and utomata Theory Siu On CHN Chinese University of Hong Kong Fall /31

2 Parsing computer programs if (n == 0) { return x; } First phase of javac compiler: lexical analysis if ( ID == INT_LIT ) { return ID ; } The alphabet of Java CFG consists of tokens like Σ = {if, return, (, ), {, }, ;, ==, ID, INT_LIT,... } 2/31

3 Parsing computer programs Statement if ParExpression Statement ( Expression ) Expression ExpressionRest Block { BlockStatements } Primary Identifier ID Infixop == Expression Primary BlockStatement Statement Literal INT_LIT return Expression Primary ; if (n == 0) { return x; } Identifier ID Parse tree of a Java statement 3/31

4 CFG of the java programming language Identifier: IdentifierChars but not a Keyword or BooleanLiteral or NullLiteral Literal: IntegerLiteral FloatingPointLiteral BooleanLiteral CharacterLiteral StringLiteral NullLiteral Expression: LambdaExpression ssignmentexpression ssignmentoperator: (one of) = *= /= %= += -= <<= >>= >>>= &= ^= = from http: //java.sun.com/docs/books/jls/second_edition/html/syntax.doc.html# /31

5 Parsing Java programs class Point2d { /* The X and Y coordinates of the point--instance variables */ private double x; private double y; private boolean debug; // trick to help with debugging public Point2d (double px, double py) { // Constructor x = px; y = py; debug = false; } // turn off debugging public Point2d () { // Default constructor this (0.0, 0.0); // Invokes 2 parameter Point2D constructor } // Note that a this() invocation must be the BEGINNING of // statement body of constructor public Point2d (Point2d pt) { x = pt.getx(); y = pt.gety(); }... } // nother consructor Simple Java program: about 1000 tokens 5/31

6 Parsing algorithms How long would it take to parse this program? try all parse trees CYK algorithm years hours Can we parse faster? CYK is the fastest known general-purpose parsing algorithm for CFGs Luckily, some CFGs can be rewritten to allow for a faster parsing algorithm! 6/31

7 Hierarchy of context-free grammars context-free grammars LR( ) grammars LR(1) grammars LR(0) grammars Java, Python, etc have LR(1) grammars We will describe LR(0) parsing algorithm grammar is LR(0) if LR(0) parser works correctly for it 7/31

8 LR(0) parser: overview S S (S) () input: ()() 1 ()() 2 ( )() 3 () () 4 () 5 S () 6 S( ) 7 S() 8 S 9 S S 8/31

9 LR(0) parser: overview S S (S) () input: ()() Features of LR(0) parser: Greedily reduce the recently completed rule into a variable Unique choice of reduction based on what has been read so far 3 () () 4 () 5 S () 9/31

10 LR(0) parsing using a PD To speed up parsing, keep track of partially completed rules in a PD P In fact, the PD will be a simple modification of an NF N The NF accepts if a rule B β has just been completed and the PD will reduce β to B 2 ( )() 3 () () 4 () 5 S () : NF N accepts 10/31

11 NF acceptance condition S S (S) () rule B β has just been completed if Case 1 input/buffer so far is exactly β Examples: 3 () () and 4 () Case 2 Or buffer so far is αβ and there is another rule C αbγ Example: 7 S() This case can be chained 11/31

12 Designing NF for Case 1 S S (S) () Design an NF N to accept the right hand side of some rule B β 12/31

13 Designing NF for Case 1 S S (S) () Design an NF N to accept the right hand side of some rule B β q 0 ε ε ε ε S S S S S S S S S (S) ( ( S) S (S ) ) (S) () ( ) () 12/31

14 Designing NF for Cases 1 & 2 S S (S) () Design an NF N to accept αβ for some rules C αbγ, B β and for longer chains 13/31

15 Designing NF for Cases 1 & 2 S S (S) () Design an NF N to accept αβ for some rules C αbγ, B β and for longer chains For every rule C αbγ, B β, add C α Bγ ε B β q 0 ε ε ε ε S S S S S S S S S ll blue are ε-transitions (S) ( ( S) S (S ) ) (S) () ( ) () 13/31

16 Summary of the NF For every rule B β, add ε B β q 0 For every rule B αxβ (X may be terminal or variable), add X B α Xβ B αx β Every completed rule B β is accepting B β For every rule C αbγ, B β, add ε C α Bγ B β The NF N will accept whenever a rule has just been completed 14/31

17 Equivalent DF D for the NF N Dead state (empty set) not shown for clarity S S S (S) S S S (S) () S S () S () ( ( ) ( ( S) ( ) S S S (S) () S ( (S ) S S (S) () ) (S) Observation: every accepting state contains only one rule: a completed rule B β, and such rules appear only in accepting states 15/31

18 LR(0) grammars grammar G is LR(0) if its corresponding D G satisfies: Every accepting state contains only one rule: a completed rule of the form B β and completed rules appear only in accepting states Shift state: no completed rule S S Reduce state: has (unique) completed rule (S) (S) () 16/31

19 Simulating DF D Our parser P simulates state transitions in DF D (() ) ( ) fter reducing () to, what is the new state? Solution: keep track of previous states in a stack go back to the correct state by looking at the stack 17/31

20 Let s label D s states q 1 q 2 q 3 S S S S S S S S (S) (S) () () q 6 ( ( q 5 (S ) S ( S) S S q 4 ( ) (S) S S S () ( S q 8 ) (S) q 7 () () (S) 18/31

21 LR(0) parser: a PD P simulating DF D P s stack contains labels of D s states to remember progress of partially completed rules t D s non-accepting state q i 1. P simulates D s transition upon reading terminal or variable X 2. P pushes current state label q i onto its stack t D s accepting state with completed rule B X 1... X k 1. P pops k labels q k,..., q 1 from its stack 2. constructs part of the parse tree B X 1 X 2... X k 3. P goes to state q 1 (last label popped earlier), pretend next input symbol is B 19/31

22 Example state stack 1 ()() q 1 $ 2 ( )() q 5 $1 3 () () q 8 $15 3 () q 1 $ 4 () q 4 $1 4 S () q 1 $ state stack 5 S () q 2 $1 6 S ( ) q 5 $12 20/31

23 Example state stack 7 S () q 8 $125 7 S q 2 $1 8 S q 3 $12 state stack 8 S q 1 $ S 9 S q 2 $1 S parser s output is the parse tree 21/31

24 nother LR(0) grammar L = {w#w R w {a, b} } C ac a bc b # NF N : a a C ac a C a C a C ac a C ac a ε ε ε q ε # 0 C # C # ε ε ε ε ε b C bc b C b C b C bc b C bc b b 22/31

25 nother LR(0) grammar C ac a bc b # a 1 C ac a # 2 C bc b C # C # b # a # 3 4 C a C a C ac a C bc b C # b a C b C b C ac a C bc b C # C C 5 6 C ac a C ac a a b 7 8 C ac a C bc b b input: ba#ab stack state action $ 1 S $1 4 S $14 3 S $143 2 R $143 5 S $ R $14 6 S $146 8 R 23/31

26 Deterministic PDs PD for LR(0) parsing is deterministic Some CFLs require non-deterministic PDs, such as L = {ww R w {a, b} } What goes wrong when we do LR(0) parsing on L? 24/31

27 Example 2 L = {ww R w {a, b} } C ac a bc b ε q 0 ε NF N : a a C ac a C a C a C ac a C ac a ε ε ε C ε ε ε ε ε b C bc b C b C b C bc b C bc b b 25/31

28 Example 2 C ac a C bc b C a b a C a C a C ac a C bc b C b a C b C b C ac a C bc b C b C ac a bc b ε shift-reduce conflicts C C ac a a C ac a C C ac a b C bc b 26/31

29 Parser generator C aca C bcb # C # C # a b # # C ac a C bc b C # parser generator a C a Ca C aca C bcb b C b Cb C aca C bcb b CFG G error C # C a C # C if G is not LR(0) C ac a a C ac a b C aca C bcb PD for parsing G Motivation: Fast parsing for programming languages 27/31

30 LR(1) Grammar: few words 28/31

31 LR(0) grammar revisited LR(1) grammars LR(0) grammars LR(0) parser: Left-to-right read, Rightmost derivation, 0 lookahead symbol S S (S) () Derivation S S S() () ()() Reduction (derivation in reverse) ()() () S() S S LR(0) parser looks for rightmost derivation Rightmost derivation = Leftmost reduction 29/31

32 Parsing computer programs if (n == 0) { return x; } Statement if ParExpression ( Expression. Statement else Statement. ) 30/31

33 Parsing computer programs if (n == 0) { return x; } else { return x + 1; } Statement if ParExpression ( Expression. Statement else Statement. ) CFGs of most programming languages are not LR(0) LR(0) parser cannot tell apart if then from if then else 30/31

34 LR(1) grammar LR(1) grammars resolve such conflicts by one symbol lookahead States in NF N LR(0): LR(1): α β [ α β, a] States in DF D LR(0): LR(1): no shift-reduce conflicts some shift-reduce conflicts allowed no reduce-reduce conflicts some reduce-reduce conflicts allowed as long as can be resolved with lookahead symbol a We won t cover LR(1) parser in this class; take CSCI 3180 for details 31/31

Lexical and Syntax Analysis. Bottom-Up Parsing

Lexical and Syntax Analysis. Bottom-Up Parsing Lexical and Syntax Analysis Bottom-Up Parsing Parsing There are two ways to construct derivation of a grammar. Top-Down: begin with start symbol; repeatedly replace an instance of a production s LHS with