LR(0) Parsers CSCI 3130 Formal Languages and utomata Theory Siu On CHN Chinese University of Hong Kong Fall 2015 1/31
Parsing computer programs if (n == 0) { return x; } First phase of javac compiler: lexical analysis if ( ID == INT_LIT ) { return ID ; } The alphabet of Java CFG consists of tokens like Σ = {if, return, (, ), {, }, ;, ==, ID, INT_LIT,... } 2/31
Parsing computer programs Statement if ParExpression Statement ( Expression ) Expression ExpressionRest Block { BlockStatements } Primary Identifier ID Infixop == Expression Primary BlockStatement Statement Literal INT_LIT return Expression Primary ; if (n == 0) { return x; } Identifier ID Parse tree of a Java statement 3/31
CFG of the java programming language Identifier: IdentifierChars but not a Keyword or BooleanLiteral or NullLiteral Literal: IntegerLiteral FloatingPointLiteral BooleanLiteral CharacterLiteral StringLiteral NullLiteral Expression: LambdaExpression ssignmentexpression ssignmentoperator: (one of) = *= /= %= += -= <<= >>= >>>= &= ^= = from http: //java.sun.com/docs/books/jls/second_edition/html/syntax.doc.html#52996 4/31
Parsing Java programs class Point2d { /* The X and Y coordinates of the point--instance variables */ private double x; private double y; private boolean debug; // trick to help with debugging public Point2d (double px, double py) { // Constructor x = px; y = py; debug = false; } // turn off debugging public Point2d () { // Default constructor this (0.0, 0.0); // Invokes 2 parameter Point2D constructor } // Note that a this() invocation must be the BEGINNING of // statement body of constructor public Point2d (Point2d pt) { x = pt.getx(); y = pt.gety(); }... } // nother consructor Simple Java program: about 1000 tokens 5/31
Parsing algorithms How long would it take to parse this program? try all parse trees CYK algorithm 10 80 years hours Can we parse faster? CYK is the fastest known general-purpose parsing algorithm for CFGs Luckily, some CFGs can be rewritten to allow for a faster parsing algorithm! 6/31
Hierarchy of context-free grammars context-free grammars LR( ) grammars LR(1) grammars LR(0) grammars Java, Python, etc have LR(1) grammars We will describe LR(0) parsing algorithm grammar is LR(0) if LR(0) parser works correctly for it 7/31
LR(0) parser: overview S S (S) () input: ()() 1 ()() 2 ( )() 3 () () 4 () 5 S () 6 S( ) 7 S() 8 S 9 S S 8/31
LR(0) parser: overview S S (S) () input: ()() Features of LR(0) parser: Greedily reduce the recently completed rule into a variable Unique choice of reduction based on what has been read so far 3 () () 4 () 5 S () 9/31
LR(0) parsing using a PD To speed up parsing, keep track of partially completed rules in a PD P In fact, the PD will be a simple modification of an NF N The NF accepts if a rule B β has just been completed and the PD will reduce β to B 2 ( )() 3 () () 4 () 5 S () : NF N accepts 10/31
NF acceptance condition S S (S) () rule B β has just been completed if Case 1 input/buffer so far is exactly β Examples: 3 () () and 4 () Case 2 Or buffer so far is αβ and there is another rule C αbγ Example: 7 S() This case can be chained 11/31
Designing NF for Case 1 S S (S) () Design an NF N to accept the right hand side of some rule B β 12/31
Designing NF for Case 1 S S (S) () Design an NF N to accept the right hand side of some rule B β q 0 ε ε ε ε S S S S S S S S S (S) ( ( S) S (S ) ) (S) () ( ) () 12/31
Designing NF for Cases 1 & 2 S S (S) () Design an NF N to accept αβ for some rules C αbγ, B β and for longer chains 13/31
Designing NF for Cases 1 & 2 S S (S) () Design an NF N to accept αβ for some rules C αbγ, B β and for longer chains For every rule C αbγ, B β, add C α Bγ ε B β q 0 ε ε ε ε S S S S S S S S S ll blue are ε-transitions (S) ( ( S) S (S ) ) (S) () ( ) () 13/31
Summary of the NF For every rule B β, add ε B β q 0 For every rule B αxβ (X may be terminal or variable), add X B α Xβ B αx β Every completed rule B β is accepting B β For every rule C αbγ, B β, add ε C α Bγ B β The NF N will accept whenever a rule has just been completed 14/31
Equivalent DF D for the NF N Dead state (empty set) not shown for clarity S S S (S) S S S (S) () S S () S () ( ( ) ( ( S) ( ) S S S (S) () S ( (S ) S S (S) () ) (S) Observation: every accepting state contains only one rule: a completed rule B β, and such rules appear only in accepting states 15/31
LR(0) grammars grammar G is LR(0) if its corresponding D G satisfies: Every accepting state contains only one rule: a completed rule of the form B β and completed rules appear only in accepting states Shift state: no completed rule S S Reduce state: has (unique) completed rule (S) (S) () 16/31
Simulating DF D Our parser P simulates state transitions in DF D (() ) ( ) fter reducing () to, what is the new state? Solution: keep track of previous states in a stack go back to the correct state by looking at the stack 17/31
Let s label D s states q 1 q 2 q 3 S S S S S S S S (S) (S) () () q 6 ( ( q 5 (S ) S ( S) S S q 4 ( ) (S) S S S () ( S q 8 ) (S) q 7 () () (S) 18/31
LR(0) parser: a PD P simulating DF D P s stack contains labels of D s states to remember progress of partially completed rules t D s non-accepting state q i 1. P simulates D s transition upon reading terminal or variable X 2. P pushes current state label q i onto its stack t D s accepting state with completed rule B X 1... X k 1. P pops k labels q k,..., q 1 from its stack 2. constructs part of the parse tree B X 1 X 2... X k 3. P goes to state q 1 (last label popped earlier), pretend next input symbol is B 19/31
Example state stack 1 ()() q 1 $ 2 ( )() q 5 $1 3 () () q 8 $15 3 () q 1 $ 4 () q 4 $1 4 S () q 1 $ state stack 5 S () q 2 $1 6 S ( ) q 5 $12 20/31
Example state stack 7 S () q 8 $125 7 S q 2 $1 8 S q 3 $12 state stack 8 S q 1 $ S 9 S q 2 $1 S parser s output is the parse tree 21/31
nother LR(0) grammar L = {w#w R w {a, b} } C ac a bc b # NF N : a a C ac a C a C a C ac a C ac a ε ε ε q ε # 0 C # C # ε ε ε ε ε b C bc b C b C b C bc b C bc b b 22/31
nother LR(0) grammar C ac a bc b # a 1 C ac a # 2 C bc b C # C # b # a # 3 4 C a C a C ac a C bc b C # b a C b C b C ac a C bc b C # C C 5 6 C ac a C ac a a b 7 8 C ac a C bc b b input: ba#ab stack state action $ 1 S $1 4 S $14 3 S $143 2 R $143 5 S $1435 7 R $14 6 S $146 8 R 23/31
Deterministic PDs PD for LR(0) parsing is deterministic Some CFLs require non-deterministic PDs, such as L = {ww R w {a, b} } What goes wrong when we do LR(0) parsing on L? 24/31
Example 2 L = {ww R w {a, b} } C ac a bc b ε q 0 ε NF N : a a C ac a C a C a C ac a C ac a ε ε ε C ε ε ε ε ε b C bc b C b C b C bc b C bc b b 25/31
Example 2 C ac a C bc b C a b a C a C a C ac a C bc b C b a C b C b C ac a C bc b C b C ac a bc b ε shift-reduce conflicts C C ac a a C ac a C C ac a b C bc b 26/31
Parser generator C aca C bcb # C # C # a b # # C ac a C bc b C # parser generator a C a Ca C aca C bcb b C b Cb C aca C bcb b CFG G error C # C a C # C if G is not LR(0) C ac a a C ac a b C aca C bcb PD for parsing G Motivation: Fast parsing for programming languages 27/31
LR(1) Grammar: few words 28/31
LR(0) grammar revisited LR(1) grammars LR(0) grammars LR(0) parser: Left-to-right read, Rightmost derivation, 0 lookahead symbol S S (S) () Derivation S S S() () ()() Reduction (derivation in reverse) ()() () S() S S LR(0) parser looks for rightmost derivation Rightmost derivation = Leftmost reduction 29/31
Parsing computer programs if (n == 0) { return x; } Statement if ParExpression ( Expression. Statement else Statement. ) 30/31
Parsing computer programs if (n == 0) { return x; } else { return x + 1; } Statement if ParExpression ( Expression. Statement else Statement. ) CFGs of most programming languages are not LR(0) LR(0) parser cannot tell apart if then from if then else 30/31
LR(1) grammar LR(1) grammars resolve such conflicts by one symbol lookahead States in NF N LR(0): LR(1): α β [ α β, a] States in DF D LR(0): LR(1): no shift-reduce conflicts some shift-reduce conflicts allowed no reduce-reduce conflicts some reduce-reduce conflicts allowed as long as can be resolved with lookahead symbol a We won t cover LR(1) parser in this class; take CSCI 3180 for details 31/31