Eliminating left recursion grammar transformation. The transformed expression grammar

Transcription

1 Eliminting left recursion grmmr trnsformtion Originl! rnsformed! 0 0! 0 α β α α α α α α α α β he two grmmrs generte the sme lnguge, but the one on the right genertes the rst, nd then string of s, using rule tht is right recursive insted of left recursive opyright c1994 Ron K ytron ll rights reserved 37 SIGPLN 94 OMPILER ONSRUION UORIL he trnsformed expression grmmr E! E 0 E 0! +E 0 E 0! -E 0! 0 0! 0 0! / 0! E) Which rule to choose? nd wht bout? E E E *+* + ) opyright c1994 Ron K ytron ll rights reserved 38 SIGPLN 94 OMPILER ONSRUION UORIL

2 irst sets irst) = 8 >< >: irst 1 ::: L ) = f g if 2 S!! i )2P irst! i ) if 2 V f g if = [ 8,1 k=1 2irst k )) irst )! B EGH H B! b! c E! e! E G! g H! h! irst!) H f h; g G f g g f c; g B f b g E f e; g f c; e; g f b; e; c; g; h; g B f b g EGH f e; c; g g opyright c1994 Ron K ytron ll rights reserved 39 SIGPLN 94 OMPILER ONSRUION UORIL ollow sets 1 Initilly set ollown ) = ;; 8 N 2 V 2 Given production! B, set ollowb) = ollowb) [ irst),fg) 3 Given production! B, where 2 irst), set ollowb) = ollowb) [ ollow)! B EGH H B! b! c E! e! E G! g H! h N ollown) f g B irst) [ ollow) = f c g irstg) = f g g ollow) [ irste) [ ollow ) = f e; g g E irst ) [ irstg) = f c; e; g g G irsth) [ ollow) = f h g H ollow) = f g opyright c1994 Ron K ytron ll rights reserved 40 SIGPLN 94 OMPILER ONSRUION UORIL

3 Recursive descent prser genertion Procedure N on ermn if Lookhed) 2 irst! 1 ); where N!! 1 ) 2 P then /* Use! 1 to generte clls to Expect) nd other nonterminls */ if Lookhed) 2 ollown ) nd N! ) 2 P then return ) /* error */ opyright c1994 Ron K ytron ll rights reserved 41 SIGPLN 94 OMPILER ONSRUION UORIL Recursive descent Exmple S!! c! Bd BQ B! bb d Q! q irst ollow S f ; b; d; c; g f g f ; b; d; g f c; g B f b; d g f c; d; q g f c; g f d; g Q f q g f c; g opyright c1994 Ron K ytron ll rights reserved 42 SIGPLN 94 OMPILER ONSRUION UORIL

4 he generted procedures S!! c! Bd BQ B! bb d Q! q irst ollow S f ; b; d; c; g f g f ; b; d; g f c; g B f b; d g f c; d; q g f c; g f d; g Q f q g f c; g Procedure S) if Lookhed) 2 f ; b; d; c; g then cll ) cll ) cll Expect) /* error */ Procedure ) if Lookhed) 2 f c g then cll Expectc) if Lookhed) 62 f d; g then /* error */ opyright c1994 Ron K ytron ll rights reserved 43 SIGPLN 94 OMPILER ONSRUION UORIL he generted procedures cont d) S!! c! Bd BQ B! bb d Q! q irst ollow S f ; b; d; c; g f g f ; b; d; g f c; g B f b; d g f c; d; q g f c; g f d; g Q f q g f c; g Procedure ) if Lookhed) 2 f g then cll Expect) cll B) cll ) cll Expectd) if Lookhed) 2 f b; d g then cll B) cll Q) if Lookhed) 2 f c; g then return ) /* error */ opyright c1994 Ron K ytron ll rights reserved 44 SIGPLN 94 OMPILER ONSRUION UORIL

5 he generted procedures cont d) S!! c! Bd BQ B! bb d Q! q irst ollow S f ; b; d; c; g f g f ; b; d; g f c; g B f b; d g f c; d; q g f c; g f d; g Q f q g f c; g Procedure B) if Lookhed) 2 f b g then cll Expectb) cll B) if Lookhed) 2 f d g then cll Expectd) /* error */ Procedure Q) if Lookhed) 2 f q g then cll Expectq) /* error */ opyright c1994 Ron K ytron ll rights reserved 45 SIGPLN 94 OMPILER ONSRUION UORIL Recursive descent expression grmmr E! E 0 E 0! +E 0 E 0! -E 0! 0 0! 0 0! / 0! E) irst ollow E f ;g f ); g E 0 f +;,g f ); g f ;g f +;,; ); g 0 f; / g f +;,; ); g f ;g f; /; +;,; ); g Procedure E 0 if Lookhed+) then cll Expect+) cll cll E 0 if Lookhed,) then cll Expect,) cll cll E 0 if Lookhed; ) ) then return ) cll Error) opyright c1994 Ron K ytron ll rights reserved 46 SIGPLN 94 OMPILER ONSRUION UORIL

6 Mintining lookhed Procedure min) Ltok GetN ext oken) cll S) unction Lookhed) : token return Ltok) Procedure Expecttok) if Ltok = tok) then Ltok GetN ext oken) /* error */ lookhed of k tokens is mintined by ppropritely buffering the input echniclly, k lookhed is equivlent in power to single token of lookhed he proof is constructive: ech permuttion of k symbols is encoded s single token he Expecttok) procedure rst compres the incoming token ginst tok, nd then dvnces input into the lookhed buffer opyright c1994 Ron K ytron ll rights reserved 47 SIGPLN 94 OMPILER ONSRUION UORIL Recursive descent correctness nd properties When is our recursive descent prser construction successful? If the grmmr involves ny left-recursion, then our construction method will crete prser contining n innite loop So, we require tht the grmmr be free of leftrecursion he grmmr trnsformtion technique covered erlier cn help eliminte leftrecursion lso, we require tht the prser operte deterministiclly: ctions tken t ech step mke progress towrd completion, so tht bcktrcking is not necessry hus, given set of rules for nonterminl N N!! 1 we require 1 2 If =! ; 1 n, then we lso require [ i \ i! n irst! i ) = f g ollown ) \ irst! i )) = f g opyright c1994 Ron K ytron ll rights reserved 48 SIGPLN 94 OMPILER ONSRUION UORIL

7 Recursive descent nd leftmost derivtions Let s exmine how our recursive descent prser recognizes the string bbddc b S B d b B B d he procedure ctivtions trce leftmost derivtion of the string We cll this style of prsing LL,becuse it uses Leftmost scn of the input nd produces Left-most derivtion c 1 S! 2! c 3 4! Bd 5 BQ 6 7 B! bb 8 d 9 Q! q S ) ) B d ) b B d ) b b B d ) bb d d ) bbd d ) bbdd c In fct, the record of the prse is simply the order in which the grmmr rules re pplied: opyright c1994 Ron K ytron ll rights reserved 49 SIGPLN 94 OMPILER ONSRUION UORIL Error repir Good progrmming lnguges re designed with reltively lrge distnce between syntcticlly correct progrms, to increse the likelihood tht conceptul mistkes re cught s syntctic errors Error repir usully occurs t two levels: Locl: repirs mistkes with little globl import, such s missing semicolons nd undeclred vribles Scope: repirs the progrm text so tht scopes re correct Errors of this kind include unblnced prentheses nd begin/ blocks Repir ctions cn be divided into insertions nd deletions ypiclly the compiler will use some lookhed nd bcktrcking in ttempting to mke progress in the prse here is gret vrition mong compilers, though some lnguges PL/) crry trdition of good error repir Gols of error repir include: 1 No input should cuse the compiler to collpse 2 Illegl constructs re flgged 3 requently occurring errors re repired grcefully 4 Miniml stuttering or cscding of errors LL-style prsing ls itself well to error repir, since the compiler uses the grmmr s rules to predict wht should occur next in the input opyright c1994 Ron K ytron ll rights reserved 50 SIGPLN 94 OMPILER ONSRUION UORIL

8 ugmenting recursive descent prsers for error recovery Recursive nd LL prsers re often clled predictive, becuse they operte by predicting the next step in derivtion Suppose the prser is operting in procedure for some nonterminl If n error occurs, it seems resonble to recover by skipping to symbol tht could follow, nd then return E! E 0 E 0! +E 0 E 0! -E 0! 0 0! 0 0! / 0! E) irst ollow E f ;g f ); g E 0 f +;,g f ); g f ;g f +;,; ); g 0 f; / g f +;,; ); g f ;g f; /; +;,; ); g Procedure E 0 StopSet) if Lookhed+) then cll Expect+) cll f +;,g[stopset) cll E 0 StopSet) if Lookhed; ) ) then return ) cll ErrorRecoverStopSet) opyright c1994 Ron K ytron ll rights reserved 51 SIGPLN 94 OMPILER ONSRUION UORIL ble-driven LLk) prsing Our recursive descent prser contined procedure for ech nonterminl he genertion of these procedures could be utomted through the construction nd testing of irst nd ollow sets for ny grmmr free of left recursion nother eqully utomtble pproch is to use simple prsing engine tht is driven by tbles constructed by similr nlysis of the grmmr Grmmr Grmmr nlyzer LLk) Prsing Engine Input ble Stck he prsing engine begins by pushing the strt symbol S onto the stck subsequent ction is one of the following: Ech Mtch: pirs n input symbol n on top-of-stck pply: replces the nonterminl N with!, wheren!!) 2 P opyright c1994 Ron K ytron ll rights reserved 52 SIGPLN 94 OMPILER ONSRUION UORIL

9 Mtch If the top-of-stck contins the terminl symbol, then the prsing engine must nd n s the next input symbol; the stck is popped, nd the input is dvnced Before fter If mtch simultneously empties the stck nd exhusts the input strem, then the string is ccepted by the prser If mtch is ttempted, but the symbols disgree, then n error is declred opyright c1994 Ron K ytron ll rights reserved 53 SIGPLN 94 OMPILER ONSRUION UORIL pply If the top-of-stck contins nonterminl N, then the prsing engine must choose the pproprite rule for N, syn! he stck is popped of symbol N, nd the symbols,, nd re pushed onto the stck, such tht is the new top-of-stck N Before α β γ fter Since mtch is lwys required when terminl is exposed on top-of-stck, the only informtion tht must be coded in our tble is the rule tht should be pplied when nonterminl ppers on top-of-stck s with our recursive descent prser, this decision cn be bsed on k symbols of lookhed into the input strem opyright c1994 Ron K ytron ll rights reserved 54 SIGPLN 94 OMPILER ONSRUION UORIL

10 onstructing the tble 1 S! 2! c 3 4! Bd 5 BQ 6 7 B! bb 8 d 9 Q! q irst ollow S f ; b; d; c; g f g f ; b; d; g f c; g B f b; d g f c; d; q g f c; g f d; g Q f q g f c; g Lookhed Nonerm b c d q S B 7 8 Q 9 he nonblnk entries in the bove tble indicte the number of the rule tht should be pplied, given nonterminl on top-of-stck nd n input symbol s lookhed opyright c1994 Ron K ytron ll rights reserved 55 SIGPLN 94 OMPILER ONSRUION UORIL Using the tble 1 S! 2! c 3 4! Bd 5 BQ 6 7 B! bb 8 d 9 Q! q Lookhed Nonerm b c d q S B 7 8 Q 9 Below is shown the stck ctivity in prsing the input string bbddc S 1 B d 4 b B d 7 b B d 7 d d 8 d 3 c 2 Input string b b d d c opyright c1994 Ron K ytron ll rights reserved 56 SIGPLN 94 OMPILER ONSRUION UORIL

11 Bottom-up prsing b S B d b B B d bottom-up prse is essentilly rightmost derivtion, run in reverse Insted of replcing nonterminl by string, we recognize the string s reducing to the nonterminl c 1 S! 2! c 3 4! Bd 5 BQ 6 7 B! bb 8 d 9 Q! q S ) ) c ) B d c ) B dc ) b B dc ) b b B dc ) bb d dc he prsing engine issues the following instructions: shift: symbol is moved from input to top-of-stck reduce r: the stck is modied by pplying rule r opyright c1994 Ron K ytron ll rights reserved 57 SIGPLN 94 OMPILER ONSRUION UORIL Shift Before fter Like the top-down prser, the bottom-up prser checks for errors on shift he prse tble we shll construct indictes when shift is error-free ctully, insted of pushing symbol onto the stck, we push stte, which indexes the prse tble nd represents the current possibilities of the prse opyright c1994 Ron K ytron ll rights reserved 58 SIGPLN 94 OMPILER ONSRUION UORIL

12 α β γ Before N fter If the rule pplied is N!!, where! hs m symbols, then m symbols re popped off the stck, nd symbol representing N is pushed It s importnt to remember tht cnonicl prse cn perform reductions only t the top-of-stck opyright c1994 Ron K ytron ll rights reserved 59 SIGPLN 94 OMPILER ONSRUION UORIL Rightmost derivtion in reverse slow motion Stck Input ctivity bbddc Shift bbddc Shift b bddc Shift bb ddc Shift bbd dc B! d bbb dc B! bb bb dc B! bb B dc! B dc Shift Bd c! Bd c Shift c! c Shift S! S ccept his is LR-style prsing: scn from the left tht produces rightmost derivtion We could hve tried to pply! t ny point during the prse, but most would not hve mde progress towrd n ccept Where prse tble construction is successful, the tble directs the prse towrds n ccept if one is possible opyright c1994 Ron K ytron ll rights reserved 60 SIGPLN 94 OMPILER ONSRUION UORIL

13 LR tble construction Ech stte of the prser represents prsing possibilities fter processing given prex of the input string o construct the cnonicl LR0) set of sttes: 1 Ech stte begins with kernel tht represents progress through certin rules of the grmmr: 3) X! y z W! xzy! By he dot ) shows the progress through the rule chieved by moving into this stte 2 When is next to nonterminl, we must dd into this stte the closure by expnding ll rules of the nonterminl: 3)! bcd! z We then lbel ech component of the stte with n ction, indicting trnsfer to some other stte, reduction by rule, or ccept: 3) X! y z W! xzy! By! bcd! z which my crete new stte: 17) X! yz! z! bcd! z Stte 17 by rule 5 Stte 2 Stte 17 by rule 10 Stte 1 Stte 2 Stte 18 opyright c1994 Ron K ytron ll rights reserved 61 SIGPLN 94 OMPILER ONSRUION UORIL ble construction 1) S!! Bd BQ B! bb d 2) S!! c Stte 2 Stte 3 Stte 4 by rule 6 Stte 7 Stte 8 by rule 3 3)! Bd B! bb d 4)! B Q Q! q Stte 9 Stte 10 Stte 11 5) B! b B B! bb d Stte 12 6) B! d by rule 8 7) S! Stte 13 8)! c by rule 2 9)! B d! c 10)! BQ 11) Q! q 12) B! bb 13) S! Stte 14 Stte 8 by rule 3 by rule 5 by rule 9 by rule 7 ^ 14)! B d Stte 15 15)! Bd by rule 4 opyright c1994 Ron K ytron ll rights reserved 62 SIGPLN 94 OMPILER ONSRUION UORIL

14 onflict resolution 1 S! 2! c 3 4! Bd 5 BQ 6 7 B! bb 8 d 9 Q! q irst ollow S f ; b; d; c; g f g f ; b; d; g f c; g B f b; d g f c; d; q g f c; g f d; g Q f q g f c; g Within stte, how do we resolve whether to shift or reduce when either ction seems pproprite? 1) S!! Bd BQ B! bb d Stte 2 Stte 3 Stte 4 by rule 6 Exmining the ollow informtion shows tht only those input symbols in f c; g cn follow n In stte 1) we therefore by rule 6 only when c or ppers next in the input Since these symbols re disoint from the input symbols tht cuse shifts into other sttes f ; b; d g), we cn resolve the pprent conflict In generl, stte might hve n pprent shift/reduce or reduce/reduce conflict he more expensive tble construction methods generlly provide better conflict resolution opyright c1994 Ron K ytron ll rights reserved 63 SIGPLN 94 OMPILER ONSRUION UORIL ble for our exmple Stte b c d q B Q 1 Stte 3 by rule 6 by rule 6 Stte 2 Stte 4 2 Stte 8 by rule 3 by rule 3 Stte 7 3 Stte 9 4 Stte 11 5 Stte 12 6 by rule 8 7 Stte 13 8 by rule 2 9 Stte 8 by rule 3 by rule 3 Stte by rule 5 11 by rule 9 12 by rule 7 13 ^ ^ 14 Stte by rule 4 Stte 10 by rule 8 by rule 2 by rule 5 by rule 9 by rule 7 by rule 4 opyright c1994 Ron K ytron ll rights reserved 64 SIGPLN 94 OMPILER ONSRUION UORIL

15 Using the tble Stte b c d q B Q ^ ^ b 5 bbddc b bbddc bddc 1 3 b b 5 b b 5 b 5 d b 5 b b 5 b 5 12 B 1 3 b b 5 12 B b d d B B bb B B bb B bddc ddc dc Bdc dc Bdc dc 1 3 Bdc opyright c1994 Ron K ytron ll rights reserved 65 SIGPLN 94 OMPILER ONSRUION UORIL Using the tble cont d) Stte b c d q B Q ^ ^ B B 9 14 B Bdc dc dc 1 3 B B 9 14 d c d Bd c c dc c c c ^ ^ opyright c1994 Ron K ytron ll rights reserved 66 SIGPLN 94 OMPILER ONSRUION UORIL

16 Set of items construction for our expression grmmr 1 S! E 2 E! E+ 3 4! 5 6! E) 7 1) S! E E! E+!! E) Stte 2 Stte 2 Stte 3 Stte 3 Stte 4 2) S! E E! E + Stte 7 Stte 8 irst ollow S f ;g f g E f ;g f +; ); g f ;g f; +; ); g f ;g f; +; ); g 3) E!! by rule 3 Stte 9 he bove shift/reduce conflict is resolved by noting tht 62 ollowe) 4)! by rule 5 5)! E) E! E+!! E) Stte 10 Stte 10 Stte 3 Stte 3 Stte 4 6)! by rule 7 opyright c1994 Ron K ytron ll rights reserved 67 SIGPLN 94 OMPILER ONSRUION UORIL Set of items construction for our expression grmmr 1 S! E 2 E! E+ 3 4! 5 6! E) 7 7) S! E ^ 8) E! E+!! E) Stte 11 Stte 11 Stte 4 irst ollow S f ;g f g E f ;g f +; ); g f ;g f; +; ); g f ;g f; +; ); g 10) E! E +! E ) 11) E! E+! Stte 8 Stte 13 by rule 2 Stte 9 he bove shift/reduce conflict is resolved by noting tht 62 ollowe) 12)! by rule 4 9)!! E) Stte 12 13)! E) by rule 6 opyright c1994 Ron K ytron ll rights reserved 68 SIGPLN 94 OMPILER ONSRUION UORIL

17 he resulting prse tble Stte + ) E 1 Stte 2 Stte 3 2 Stte 8 Stte 7 3 by rule 3 Stte 9 by rule 3 by rule 3 4 by rule 5 5 Stte 10 Stte 3 6 by rule 7 7 ^ ^ 8 Stte Stte 8 Stte by rule 2 Stte 9 by rule 2 by rule 2 12 by rule 4 13 by rule 6 Stte 4 by rule 5 Stte 4 by rule 7 Stte 4 Stte 12 by rule 4 by rule 6 opyright c1994 Ron K ytron ll rights reserved 69 SIGPLN 94 OMPILER ONSRUION UORIL Using the tble Stte + ) E ^ ^ ) + +) + +) E Ẹ + + +) + +) + +) +) +) +) +) +) opyright c1994 Ron K ytron ll rights reserved 70 SIGPLN 94 OMPILER ONSRUION UORIL

18 Using the tble Stte + ) E ^ ^ ) +) +) 1 2 E E E E E E E E E 10 5 E E E E E + +) +) +) +) ) ) ) ) opyright c1994 Ron K ytron ll rights reserved 71 SIGPLN 94 OMPILER ONSRUION UORIL Using the tble Stte + ) E ^ ^ E E 10 5 E 10 ) E+ Ẹ ) ) ) 1 E 2 1 E 2 E 7 5 E E ) 13 E E) E+ E E ^^ + * + ) opyright c1994 Ron K ytron ll rights reserved 72 SIGPLN 94 OMPILER ONSRUION UORIL