10/12/17. Motivating Example. Lexical and Syntax Analysis (2) Recursive-Descent Parsing. Recursive-Descent Parsing. Recursive-Descent Parsing

Transcription

1 Motivting Exmple Lexicl nd yntx Anlysis (2) In Text: Chpter 4 Consider the grmmr -> cad A -> b Input string: w = cd How to build prse tree top-down? 2 Initilly crete tree contining single node (the strt symbol) Apply the -rule to see whether the first token mtches If mtches, expnd the tree Apply the A-rule to the leftmost nonterminl A ince the first token mtches both lterntives (A1 nd A2), rndomly pick one (e.g., A1) to pply c A d c A d b c ince the third token d does not mtch b, report A d filure nd go bck to A to try nother lterntive Rollbck to the stte before pplying A1 rule, nd then pply the lterntive rule The third token mtches, so prsing is successfully done c A b d 3 4 Algorithm uppose we hve scnner which genertes the next token s needed. Given string, the prsing process strts with the strt symbol rule: 1. if there is only one RH then 2. for ech terminl in the RH 3. compre it with the next input token 4. if they mtch, then continue 5. else report n error 6. for ech nonterminl in the RH 7. cll its corresponding subprogrm nd try mtch 8. else // there is more thn one RH 9. choose the RH bsed on the next input token (the lookhed) 10. for ech chosen RH 11. try mtch with 2-7 mentioned bove 12. if no mtch is found, then report n error 5 There is subprogrm for ech nonterminl in the grmmr, which cn prse sentences tht cn be generted by tht nonterminl EBNF is idelly suited for being the bsis for recursive-descent prser, becuse EBNF minimizes the number of nonterminls 6 1

2 An Exmple A grmmr for simple expressions: <expr> <term> {(+ -) <term> <term> <fctor> {(* /) <fctor> <fctor> id int_constnt ( <expr> ) /* Function expr prses strings in the lnguge generted by the rule: <expr> <term> {(+ -) <term> */ void expr() { printf( Enter <expr>\n ); /* Prse the first term */ term(); /* As long s the next token is + or -, cll lex to get the next token nd prse the next term */ 7 while (nexttoken == ADD_OP nexttoken == UB_OP){ term(); printf( Exit <expr>\n ); 8 This prticulr routine does not detect errors Convention: Every prsing routine leves the next token in nexttoken An Exmple (cont d) /* term Prses strings in the lnguge generted by the rule: <term> -> <fctor> {(* /) <fctor>) */ void term() { printf( Enter <term>\n ); /* Prse the first fctor */ fctor(); /* As long s the next token is * or /, next token nd prse the next fctor */ while (nexttoken == MULT_OP nexttoken == DIV_OP) { fctor(); printf( Exit <term>\n ); /* End of function term */ 9 10 /* Function fctor prses strings in the lnguge generted by the rule: <fctor> -> id int_constnt (<expr>) */ void fctor() { printf( Enter <fctor>\n ); /* Determine which RH */ if (nexttoken) == ID_CODE nexttoken == INT_CODE) /* For the RH id, just cll lex */ /* If the RH is (<expr>) cll lex to pss over the left prenthesis, cll expr, nd check for the right prenthesis */ else if (nexttoken == LP_CODE) { expr(); if (nexttoken == RP_CODE) else error(); /* End of else if (nexttoken ==... */ else error(); /* Neither RH mtches */ #define INT_LIT 10 #define IDENT 11 #define AIGN_OP 20 #define ADD_OP 21 #define UB_OP 22 #define MULT_OP 23 #define DIV_OP 24 #define LEFT_PAREN 25 #define RIGHT_PAREN 26 Token codes printf( Exit <fctor>\n );

3 (continued) Trce of the lexicl nd syntx nlyzers on (sum+47)/totl Next token is: 25 Next lexeme is ( Enter <expr> Enter <term> Enter <fctor> Next token is: 11 Next lexeme is sum Enter <expr> Enter <term> Enter <fctor> Next token is: 21 Next lexeme is + Exit <fctor> Next token is: -1 Next lexeme is EOF Key points bout recursive-descent prsing Recursive-descent prsing my require bcktrcking LL(1) does not llow bcktrcking By only looking t the next input token, we cn lwys precisely decide which rule to pply By crefully designing grmmr, i.e., LL(1) grmmr, we cn void bcktrcking Two Obstcles to LL(1)-ness Left recursion E.g., id_list -> id_list_prefix ; id_list_prefix -> id_list_prefix, id id When the next token is id, which rule should we pply? Common prefixes E.g., A -> b When the next token is, which rule should we pply? Common prefixes Pirwise Disjointness Unble to decide which RH should use by simply checking one token of lookhed Pirwise Disjointness Test For ech nonterminl A with more thn one RH, for ech pir of rules, the possible first chrcters of the strings (FIRT set) should be disjoint If A -> α 1 α 2, then FIRT(α 1 ) FIRT(α 2 ) = φ LL(1) Grmmr Grmmr which cn be processed with LL(1) prser Non-LL grmmr cn be converted to LL(1) grmmr vi: Left-recursion elimintion Left fctoring by extrcting common prefixes Left-Recursion Elimintion Replce left-recursion with rightrecursion id_list -> id_list_prefix ; id_list_prefix -> id_list_prefix, id id => id_list -> id id_list_til id_list_til -> ;, id id_list_til

4 Left Fctoring Extrct the common prefixes, nd introduce new nonterminls s needed A -> b => A -> B B -> b ε Bottom-up Prsing The prsing problem is finding the correct RH in right-sententil form to reduce to get the previous rightsententil form in the derivtion Non-LL Lnguges imply eliminting left recursion nd common prefixes is not gurnteed to mke LL(1) An exmple in Pscl: stmt -> if condition then_cluse else_cluse other_stmt then_cluse -> then stmt else_cluse -> else stmt ε How to prse if C1 then if C2 then 1 else 2? Non-LL Lnguges Define dismbiguting rule, use it together with mbiguous grmmr to prse top-down E.g., in the cse of conflict between two possible productions, the one to use is the one tht occurs first, textully in the grmmr to pir the else with the nerest then Dismbiguting rule cn be lso defined for bottom-up prsing Tble-Driven Prsing It is possible to build non-recursive predictive prser by mintining stck explicitly, rther thn implicitly vi recursive clls The non-recursive prser looks up the production to be pplied in prsing tble. The tble cn be constructed directly from LL(1) grmmrs 23 Tble-Driven Prsing An input buffer Contins the input string The string cn be followed by $, n end mrker to indicte the end of the string A stck Contins symbols with $ on the bottom, with the strt symbol initilly on the top A prsing tble (2-dimensionl rry M[A, ]) An output strem (production rules pplied for derivtion) 24 4

5 Input: string w, prsing tble M for grmmr G Output: if w is in L(G), leftmost derivtion of w; otherwise, n error indiction Method: set ip to point to the first symbol of w$ repet let X be the top stck symbol nd the symbol pointed to by ip; if X is terminl or $, then if X = then pop X from the stck nd dvnce ip else error() else /* X is non-terminl */ if M[X, ] = X->Y 1 Y 2 Y k, then pop X from the stck push Y k,, Y 2, Y 1 on to the stck output the production X->Y 1 Y 2 Y k end else error() 25 until X = $ An Exmple Input tring: id + id * id Input prsing tble for the following grmmr E -> TE E -> +TE ε T -> FT T -> *FT ε F -> (E) id 26 LL Prsing tck Input Output $E id + id * id$ $E T id + id * id$ E -> TE $E T F id + id * id$ T -> FT $E T id id + id * id$ F -> id $E T + id * id$ $ $ E -> ε 27 5