LR Parsing, Part 2. Constructing Parse Tables. Need to Automatically Construct LR Parse Tables: Action and GOTO Table

Transcription

1 TDDD55 Compilers nd Interpreters TDDB44 Compiler Construction LR Prsing, Prt 2 Constructing Prse Tles Prse tle construction Grmmr conflict hndling Ctegories of LR Grmmrs nd Prsers Peter Fritzson, Christoph Kessler, IDA, Linköpings universitet, Need to Automticlly Construct LR Prse Tles: Action nd GOTO Tle Construct prse tles from the grmmr s follows: First uild GOTOgrph (n NFA) to recognize vile prefixes Mke it deterministic (DFA) Then fill in Action nd Goto tles Exmple Grmmr G 1. <L L, E 2. E 3. E 4. ACTION tle: stte --, 0 X X A 2 * * 2 X X R1 R1 * * 4 R3 R3 * * 5 R4 R4 * * 6 GOTO R2 R2 tle: * * stte L E * * 2 * 3 3 * * 4 * * 5 * * 6 * * 2 TDDD55/TDDB44 Compiler Construction, 2011 Clsses of LR Prsers/Grmmrs LR(0) Too wek (no lookhed) LR(1) imple LR, 1 token lookhed LALR(1) Most common, 1 token lookhed LR(1) 1 token lookhed ig tles LR(k) k tokens lookhed Even igger tles Differences etween LR prsers: Tle size vries widely. Errors not discovered s quickly y some vrints. Different limittions in the lnguge definitions, grmmrs. An NFA Recognizing Vile Prefixes A.k.. the chrcteristic finite utomton for grmmr G ttes: LR(0) items (= context-free items) of extended Grmmr (definition, see next pge) Input strem: The grmmr symols on the stck trt stte: [. ] Finl stte: [.] Trnsitions: 118 move dot cross symol if symol found next on stck: A.B to A B. A. to A. -trnsitions to LR(0)-items for nonterminl productions from items where the dot precedes tht nonterminl: A.B to B. 3 TDDD55/TDDB44 Compiler Construction, TDDD55/TDDB44 Compiler Construction, 2011 Hndle, Vile Prefix Consider rightmost derivtion * rm Xu rm u for context-free grmmr G. is clled hndle of the right sententil form u, ssocited with the rule X rm Ech prefix of is clled vile prefix of G. Exmple: Grmmr G with productions { -> c } Right sententil forms: e.g. c, c,,,.. For c: Hndle: c Vile prefixes:, c For c: Hndle: c,, c For : Hndle:,,, For : Hndle:,,,, 5 TDDD55/TDDB44 Compiler Construction, 2011 Right Derivtion nd Vile Prefixes Input:,, Right derivtion (hndles re underlined, nd lue) <list> rm <list>, <element> rm <list>, rm <list>, <element>, rm <list>,, rm <element>,, rm,, ome Vile prefixes of the sententil form: <list>,, re { ε; <list> ; <list>, ; <list>, ; <list>,, ; <list>,, } 6 TDDD55/TDDB44 Compiler Construction,

2 Definition of LR(0) Item An LR(0) item of rule P is rule with dot somewhere in the right side. Exmple: All LR(0) items of the production 1. <list> <list>, <element> re <list> <list>, <element> <list> <list>, <element> <list> <list>, <element> <list> <list>, <element> Intuitively n item is interpreted s how much of the rule we hve found nd how much remins. Items re put together in sets which ecome the LR nlyser s stte. Informl Construction of GOTO-Grph (NFA/DFA) We wnt to construct DFA which recognises ll vile prefixes of G(<Y>): GOTO-grph (A GOTO-grph is not the sme s GOTO-tle ut corresponds to n ACTION + GOTO-tle tle. The grph discovers vile prefixes.) Augmented Grmmr G(<sys>) 0. <Y> <list> 1. <list> <list>, <element> 2. <element> 3. <element> 4. Exmple. Find vile prefixes in rightmost derivtion elow, used for informl construction of goto grph <list> rm <list>, <element> rm <list>, rm <list>, <element>, rm <list>,, rm <element>,, rm,, 7 TDDD55/TDDB44 Compiler Construction, TDDD55/TDDB44 Compiler Construction, 2011 Informl Construction of GOTO-Grph (NFA/DFA) We wnt to construct DFA which recognises ll vile prefixes of G(<Y>): strt GOTO-grph (A GOTO-grph is not the sme s GOTO-tle ut corresponds to n ACTION + GOTO-tle tle. The grph discovers vile prefixes.) Augmented Grmmr G(<sys>) 0. <Y> <list> 1. <list> <list>, <element> 2. <element> 3. <element> 4. <list>, <element> <Y> <list> <list> <list>, <element> <element> 6 4 <element> <list> <element> 5 <element> Exmple righmost derivtion <list> rm <list>, <element> rm <list>, rm <list>, <element>, rm <list>,, rm <element>,, rm,, Constructing ets of LR(0) Items et I 0 <Y> <list> -- Kernel (Bsis) <list> <list>, <element> <list> <element> <element> <element> Additionl Closure (of kernel items) et I <Y> <list> -- Kernel (Bsis) 1 <list> <list>, <element> (empty closure s precedes terminls -- nd, ) Additionl Closure et I 2 <list> <list>, <element> Kernel (Bsis) <element> Additionl <element> Closure et I 3, etc. Augmented Grmmr G(<sys>) 0. <Y> <list> 1. <list> <list, <element> 2. <element> 3. <element> 4. 9 TDDD55/TDDB44 Compiler Construction, TDDD55/TDDB44 Compiler Construction, 2011 GOTO Grph with ttes s ets of LR(0) Items Bsed on the cnonicl collection of LR(0) items drw the GOTO grph. strt The GOTO grph discovers those prefixes of right sententil forms which hve (t most) one hndle furthest to the right in the prefix. Exmple Grmmr 1. L L, E 2. L E 3. E 4. E I0 L L L, E L E E E I I 1 2 L, L, L L, E L L, E E E E I6 L E I4 E I 5 E I 3 Fill in Action Tle from GOTO Grph 1. If there is n item <A> α β I i nd GOTOgrph(I i, ) = I j i I i j Fill in shift j for row i nd column for symol. 3. If we hve <Y> <> -- ccept the symol -- I j 4. Otherwise error. 2. If there is complete item (i.e., ends in dot ): <A> α I i Fill in reduce x where x is the production numer for x: <A> α tte numer I i : stte i (line i, itemset i) ACTION tle: i shift j Filled in Action tle stte --, 0 X X A 2 * * 2 X X R1 R1 * * 4 R3 R3 * * 5 R4 R4 * * 6 R2 R2 * * Nonterminls 11 TDDD55/TDDB44 Compiler Construction, TDDD55/TDDB44 Compiler Construction,

3 Tle Differences LR(0), LR(1), LALR(1) In which column(s) should reduce x e written? LR(0) fills in for ll input. LR(1) fills in for ll input in FOLLOW(<A>). LALR(1) fills in for ll those tht cn follow certin instnce of <A>, see lter Filling in the GOTO Tle <A> α I i If the GOTOgrph(I i, <A>) = I j fill in GOTOtle[ i, <A>] = j Filled in GOTO tle: stte L E * * 2 * 3 3 * * 4 * * 5 * * 6 * * tte numer i I i <A> GOTO tle: i j j I j <A> Exmple Grmmr 1. L L, E 2. L E 3. E 4. E Nonterminls 13 TDDD55/TDDB44 Compiler Construction, TDDD55/TDDB44 Compiler Construction, 2011 Computing the LR(0) Item Closure (Detiled Algorithm) For set I of LR(0) items compute Closure(I) (union of Kernel nd Closure): 1. Closure(I) := I (strt with the kernel) 2. If [A.B ] in Closure(I) nd production B then dd [B. ] to Closure(I) (if not lredy there) 3. Repet tep 2 until no more items cn e dded to Closure(I). Remrks: For s=[a B ], Closure(s) contins ll NFA sttes rechle from s vi -trnsitions, i.e., strting from which ny sustring derivle from B could e recognized. A.k.. -closure(s). Then pply the well-known suset construction to trnsform Closure-NFA -> DFA. DFA sttes will e sets unioning closures of NFA sttes Representing ets of Items Implementtion in Prser Genertor Any item [A ] cn e represented y 2 integers: production numer 118c position of the dot within the RH of tht production The resulting sets often contin closure items (where the dot is t the eginning of the RH). Cn esily e reconstructed (on demnd) from other ( kernel ) items Kernel items: strt stte [.], plus ll items where the dot is not t the left end. tore only kernel items explicitly, to sve spce 15 TDDD55/TDDB44 Compiler Construction, TDDD55/TDDB44 Compiler Construction, 2011 GOTOgrph Function nd DFA ttes Detiled lgorithm Given: et I of items, grmmr symol X GOTOgr( I, X ) := U [A.X ] in I Closure ( { [A X. ] } ) To ecome the stte trnsitions in the DFA Now do the suset construction to otin the DFA sttes: C := Closure( { [.] } ) // C: et of sets of NFA sttes repet for ech set of items I of C: for ech grmmr symol X if (GOTOgr(I,X) is not empty nd not in C) dd GOTOgr(I,X) to C until no new sttes re dded to C on round. 17 TDDD55/TDDB44 Compiler Construction, 2011 Resulting DFA All sttes correspond to some vile prefix Finl sttes: contin t lest one item with dot to the right recognized some hndle reduce my (must) follow Other sttes: hndle recognition incomplete -> shift will follow The DFA is lso clled the GOTO grph (not the sme s the LR GOTO Tle!!). 120 This utomton is deterministic s FA (i.e., selecting trnsitions considering only input symol consumption) ut cn still e nondeterministic s pushdown utomton (e.g., in stte I 1 ove: to reduce or not to reduce?) 18 TDDD55/TDDB44 Compiler Construction,

4 From DFA to prser tles: ACTION Detiled Algorithm, ummry 1. For ech DFA trnsition I i I j reding terminl in (thus, I i contins items of kind [X. ]) ACTION tle: enter j (shift, new stte I j ) in ACTION[ i, ] stte --, 2. For ech DFA finl stte I i (contining complete item [X.]) enter R x (reduce, x = prod. rule numer for X ) in ACTION[ i, t ] LR(0) prser: for ll t in (ll entries in row i) 0 X X A 2 * * 2 X X R1 R1 * * 4 R3 R3 * * 5 R4 R4 * * 6 R2 R2 * * LR(1) prser: for ll t in LA LR (i,[x.]) = FOLLOW 1 (X) LALR(1) prser: for ll t in LA LALR (i,[x.]) (see lter) Collision with n lredy existing or R entry? Conflict!! 3. For ech DFA stte contining [. --] enter A in ACTION[ i, -- ] (ccept). NB - Conflict? (s in 2.) 19 TDDD55/TDDB44 Compiler Construction, 2011 From DFA to prser tles: GOTO Tle ummry 1. For ech DFA trnsition I i I j reding nonterminl A (i.e., I i contins n item [X.A ]) enter GOTO[ i, A ] = j GOTO tle: stte L E * * 2 * 3 3 * * 4 * * 5 * * 6 * * 20 TDDD55/TDDB44 Compiler Construction, 2011 TDDD55 Compilers nd Interpreters TDDB44 Compiler Construction Conflicts nd Ctegories of LR Grmmrs nd Prsers Conflict Exmples in LR Grmmrs hift Reduce conflict: E id + E (shift +) id (reduce id) Reduce Reduce conflict: E id Pcll id (reduce id) (reduce id) (hift Accept conflict) L (ccept) L L, E (shift,) Peter Fritzson, Christoph Kessler, IDA, Linköpings universitet, TDDD55/TDDB44 Compiler Construction, 2011 Conflicts in LR Grmmrs Oserve conflicts in DFA (GOTO grph) kernels or t the ltest when filling the ACTION tle. hift-reduce conflict A DFA ccepting stte hs n outgoing trnsition, i.e. contins items [X.] nd [Y.Z ] for some Z in NU Reduce-Reduce conflict A DFA ccepting stte cn reduce for multiple nonterminls, i.e. contins t lest 2 items [X.] nd [Y.], X!= Y (hift/reduce-accept conflict) A DFA ccepting stte contining [. --] contins nother item [X.] or [X. ] Only for LR(0) grmmrs there re no conflicts. 23 TDDD55/TDDB44 Compiler Construction, 2011 Hndling Conflicts in LR Grmmrs (Overview): Use lookhed if lucky, the LR(0) sttes + few fixed lookhed sets re sufficient to eliminte ll conflicts in the LR(0)-DFA LR(1), LALR(1) otherwise, use LR(1) items [X., ] ( is look-hed) to uild new, lrger NFA/DFA expensive (mny items/sttes very lrge tles) if still conflicts, my try gin with k>1 even lrger tles Rewrite the grmmr (fctoring / expnsion) nd retry If nothing helps, re-design your lnguge syntx ome grmmrs re not LR(k) for ny constnt k nd cnnot e mde LR(k) y rewriting either 24 TDDD55/TDDB44 Compiler Construction,

5 Look-Ahed (LA) ets For LR(0) item [X ] in DFA-stte I i, define lookhed set LA( I i, [X ] ) ( suset of ) For LR(1), LALR(1) etc., the LA sets only differ for reduce items: For LR(1): LA,[X = {in : => * LR ( I i.] ) X } = FOLLOW 1 (X) for ll I i with [X.] in I i depends on nonterminl X only, not on stte I i For LALR(1): LA LALR ( I i, [X.] ) = { in : => * Xw nd the LR(0)-DFA strted in I 0 reches I i fter reding } usully suset of FOLLOW 1 ( X ), i.e. of LR LA set depends on stte I i 25 TDDD55/TDDB44 Compiler Construction, 2011 Mde it simple: Is my grmmr LR(1)? Construct the (LR(0)-item) chrcteristic NFA nd its equivlent DFA (= GOTO grph) s ove. Consider ll conflicts in the DFA sttes: hift-reduce: [X.] [Y. ] Consider ll pirs of conflicting items [X.], [Y. ]: If in FOLLOW 1 (X) for ny of these not LR(1). Reduce-Reduce: Consider ll pirs of conflicting items [X.], [Y.]: If FOLLOW 1 (X) intersects with FOLLOW 1 (Y) not LR(1). (hift-accept: similr to hift-reduce) [X.] [Y.] 26 TDDD55/TDDB44 Compiler Construction, 2011 Exmple: L-Vlues in C Lnguge L-vlues on left hnd side of ssignment. Prt of C grmmr: L = R 3. R 4. L *RR 5. id 6. R L Avoids tht R (for R-vlues) ppers s LH of ssignments But *R = is ok. This grmmr is LALR(1) ut not LR(1): 27 TDDD55/TDDB44 Compiler Construction, 2011 Exmple (cont.) LR(0) prser hs shift-reduce conflict in kernel of stte I 2 : I 0 = { [.], [.L=R], [.R], [L.*R], [L.id], R.L] } I 1 = { [ ->.] } I 2 = { [->L.=R], [R->L.] } I 3 = { [->R.] } I 4 = {[L->*.R], [R->.L], [L->.*R], [L->.id] } I 5 = { [L->id.] } I 6 = { [->L=.R], [R->.L], [L->.*R], L->.id] } I 7 = { [L->*R.] } I 8 = { [R->L.] } hift = or reduce to R? I 9 = { [->L=R.] } FOLLOW 1 (R) = {, = } LR(1) still shift-reduce conflict in I 2 s = does not dismigute 28 TDDD55/TDDB44 Compiler Construction, 2011 Exmple (cont.) I 0 = { [ ->.], [->.L=R], [->.R], [L->.*R], [L->.id], R->.L] } I 1 = { [ ->.] } I 2 = { [->L.=R], [R->L.] } I 3 = { [->R.] } I 4 = { [L->*.R], [R->.L], [L->.*R], [L->.id] } I 5 = {[L->id.] } I 6 = { [->L=.R], [R->.L], [L->.*R], L->.id] } I 7 = { [L->*R.] } I 8 = { [R->L.] } I 9 = { [->L=R.] } LA LALR ( I 2, [R->L.] ) = { } LALR(1) prser is conflict-free s computtion pth I 0 I 2 does not relly llow = following R. = cn only occur fter R if *R ws encountered efore. 29 TDDD55/TDDB44 Compiler Construction, 2011 LALR(1) Prser Construction Method 1: (simple ut not prcticl) 1. Construct the LR(1) items (see lter). (If there is lredy conflict, stop.) 2. Look for sets of LR(1) items tht hve the sme kernel, nd merge them. 3. Construct the ACTION tle s for LR(1). If conflict is detected, the grmmr is not LALR(1). 4. Construct the GOTOgrph function: For ech merged J = I 1 U I 2 U U I r, the kernels of GOTOgr(I 1,X),, GOTOgr(I r,x) re identicl ecuse the kernels of I 1,,I r re identicl. et GOTOgr( J, X ) := U { I: I hs the sme kernel s GOTOgr(I 1,X) } Method 2: (prcticl, used) (detils see textook) 1. trt from LR(0) items nd construct kernels of DFA sttes I 0, I 1, 2. Compute lookhed sets y propgtion long the GOTOgr(I j,x) edges (fixed point itertion). 30 TDDD55/TDDB44 Compiler Construction,

6 olve Conflicts y Rewriting the Grmmr Eliminte Reduce-Reduce Conflict: Fctoring ( A ) ( B ) [A ident. ] A chr integer ident [B ident. ] B flot doule ident fctor ident ( A ) ( B ) ( C ) A chr integer B flot doule C ident Eliminte hift-reduce Conflict: (one token lookhed: ( ) Inline-Expnsion ( A ) OptY ( B ) [. ( A ) ] ( A ) ( B ) OptY Y [. OptY ( B) ] Y ( B ) [OptY.Y ] expnd Y [OptY. ] OptY Y A A [OptY. ] B B [Y ] 31 TDDD55/TDDB44 Compiler Construction, 2011 LR(k) Grmmr - Forml Definition Let G e the ugmented grmmr for G (i.e., extended y new strt symol nd production rule -- ) G is clled LR(k) grmmr if rm =>* Xw rm => w nd rm=>* Yx rm=> y nd w[1:k] = y[1:k] imply tht = nd X = Y nd x = y = w. p.116 i.e., considering t most k symols fter the hndle, in ech rightmost derivtion the hndle cn e loclized nd the production to e pplied cn e determined. Remrk: w, x, y in * in (N U )* X, Y in N 32 TDDD55/TDDB44 Compiler Construction, 2011 ome grmmrs re not LR(k) for ny fixed k Exmple: B c B B descries lnguge { 2N+1 c : N >= 0 } This grmmr is not LR(k) for ny fixed k. No miguous grmmr is LR(k) for ny fixed k if E then if E then else other sttements is miguous the following sttement hs 2 prse trees: if E1 then if E2 then 1 else 2 Proof: As k is fixed (constnt), consider for ny n > k: =>* n B n c => n n c =>* n+1 B n+1 c => n+1 n+1 c By the LR(k) definition, The hndle cnnot e = n = w = n c loclized with only limited lookhed size k = n+1 = y = n+1 c Although w[1:k] = y[1:k], we hve!= grmmr is not LR(k). 33 TDDD55/TDDB44 Compiler Construction, 2011 if E then E1 if E then else E2 1 2 if E then else E1 if E then 2 E TDDD55/TDDB44 Compiler Construction, 2011 (cont.) Consider sitution (configurtion of shift-reduce prser) -- if E then else -- Not cler whether to shift else (following production 2, i.e. if E then is not hndle), or reduce hndle if E then to (following production 1) Any fixed-size lookhed (else nd eyond) does not help! uggestion: Rewrite grmmr to mke it unmiguous Rewriting the grmmr Mtched Open Mtched if E then Mtched else Mtched other sttements Open if E then if E then Mtched else Open is no longer miguous if E then E1 Open Mtched if E then Mt- else Mt- Impossile now to derive ny sententil form contining n Open nonterminl from Mtched 35 TDDD55/TDDB44 Compiler Construction, 2011 E TDDD55/TDDB44 Compiler Construction,

7 ome grmmrs re not LR(k) for ny fixed k Grmmr with productions is unmiguous ut not LR(k) for ny fixed k. (Why?) An equivlent LR grmmr for the sme lnguge is 37 TDDD55/TDDB44 Compiler Construction, 2011 LR(1) Items nd LR(k) Items LR(k) prser: Construction similr to LR(0) / LR(1) prser, ut pln for distinguishing etween sttes for k>0 tokens lookhed lredy from the eginning ttes in the LR(0) GOTO grph my e split up LR(1) items: [A-> >, ] for ll productions A-> nd ll in Cn e comined for lookhed symols with equl ehvior: [ A->, ] or [ A->, L ] for suset L of Generlized to k>1: [ A->, 1 2 k ] Interprettion of [ A->, ] in stte: If not, ignore second component (s in LR(0)) If =, i.e. [ A->., ], reduce only if next input symol =. 38 TDDD55/TDDB44 Compiler Construction, 2011 LR(1) Prser NFA strt stte is [ ->., ] Modify computtion of Closure(I), GOTO(I,X) nd the suset computtion for LR(1) items Detils see [AU86, p.232] or [ALU06, p.261] Cn hve mny more sttes thn LR(0) prser Which my help to resolve some conflicts Interesting to know For ech LR(k) grmmr with some constnt k>1 there exists n equivlent* grmmr G tht is LR(1). For ny LL(k) grmmr there exists n equivlent LR(k) grmmr (ut not vice vers!) e.g., lnguge { n n : n>0 } U { n c n : n > 0 } hs LR(0) grmmr ut no LL(k) grmmr for ny constnt k. ome grmmrs re LR(0) ut not LL(k) for ny k e.g., A A A (left recursion, could e rewritten) * Two grmmrs re equivlent if they descrie the sme lnguge. 39 TDDD55/TDDB44 Compiler Construction, TDDD55/TDDB44 Compiler Construction,