Unifying LL and LR syntax analysis of extended free grammars Luca Breveglieri Stefano Crespi Reghizzi Angelo Morzenti Politecnico di Milano 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 1 / 23
Outline motivations and target classical syntax analysis unification methodology hints on the constructions methodology deployment 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 2 / 23
Motivations PoliMi has a course on Formal Languages & Compilers for the master program course contents: regular and free languages grammars and automata classical syntax analysis (LL, LR and Earley) hints on semantic analysis practical compiler design (byflexand Bison) 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 3 / 23
Motivations wish to compact and simplify the LL and LR syntax analysis methodologies possibly also Early now being investigated these two methods are usually explained to the master students sequentially (e.g. first LL and then LR) independently (almost no notion share) and each one from beginning to end this appears to be the case also elsewhere 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 4 / 23
Motivations and to include grammars with (production) rules in the Extended Backus-Naur Form (EBNF grammars) as well that is, grammars with regular expressions on both the terminal and non-terminal alphabets in their right part extended rules stem naturally from the syntax diagrams of languages 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 5 / 23
Motivations example: standard Dyck gram.: S a S b S S ε and in the EBN Form: S ( a S b ) from a handbook of the C language: B S A B ( C ε) (, B ( C ε) ) S A C would look ugly in non-extended form 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 6 / 23
Objective derive the LL and LR syntax analysis methodologies in a unified framework starting directly from the EBNF grammar sharing notions and constructions (as much as possible) saving time and voice to explain (teacher) saving mind effort to learn (student) possibly include Earley and similar 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 7 / 23
Classical Syntax Analysis from the grammar to the syntax analyser a syntax analyser: is a Pushdown Automaton (PA) recognises the grammar language in addition builds the string derivation (or the string syntax tree) and is deterministic two classical methodologies: LL and LR 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 8 / 23
Classical LL Analysis (top-down or recursive descent method) constructs the leftmost derivation Syntax Analyser (SA) works intuitively SA is enough powerful for complier design yet does not capture all of determinism SA can be simply implemented by hand (by means of recursive syntactic procedures) commonly applied to EBNF grammars but with rules rather twisted to teach 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 9 / 23
Example LL S a S b look-ahead: a S ε look-ahead: b, eos S ( a S b ) look-ahead: a, b, eos procedure S if char = a then read char call S read char if char!= b error else if char = b, eos then null else error end if end procedure procedure S if char = a then while char = a do... end while else if char = b, eos then... else error end if end procedure 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 10 / 23
Classical LR Analysis (bottom-up or shift-reduction method) constructs the rightmost derivation Syntax Analyser (SA) is sophisticated SA is well suited for complier design and captures all of determinism SA need be implemented automatically there are attempts to apply to EBNF grammars (by Earley, Heilbrunner, Beatty and possibly others) but are unhappy: hard work and ugly SA these methods are unseen in textbooks 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 11 / 23
S asbs - S e - a Example LR pilot graph a S a S b S S ε S a SbS - S asbs b S e b S b S asb S - S asbs - S e - S S asbs - a a a S a SbS S asbs S e b b b S b S asb S b S asbs b S e b S S asbs b 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 12 / 23
References for LL and LR On the definition of ELR(k) and ELL(k) grammars, S. Heilbrunner, Acta Informatica, pp. 169-176, vol. 11, 1979 On the relationship between the LL(1) and LR(1) grammars, J. C. Beatty, Jou. of the ACM, pp. 1007-1022, vol. 29, n. 4, 1982 Tests for the LR- LL- and LC-Regular Conditions, S. Heilbrunner, Jou. of Comp. and Sys. Sci., pp. 1-13, vol. 27, n. 1, 1983 Formal Languages and Compilation, S. Crespi Reghizzi, Springer, 2009 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 13 / 23
Unification of LL and LR represent EBNF grammar rules as DFSA s (over both terminal and nonterminal alphabets) grammar becomes a network of recursive finite state automata basically is a PA start directly from the so-called pilot (or driver) graph of the automaton network (pilot is the heart of the classical LR analysis) both the LL and LR conditions and syntax analysers can be derived from the pilot 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 14 / 23
Grammar Rules to Finite Automata ε, a, ( ), (a), ((a)), (aa), (a) ( ), 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 15 / 23
Ext. P i l o t G r a p h 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 16 / 23
Unified LR Analysis use the classical LR condition: the EBNF grammar is LR if and only if in every macro-state of the extended pilot graph there are not any shift-reduction or reduction-reduction conflicts condition easy to be checked on the pilot the construction of the bottom-up syntax analyser is similar to the classical one 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 17 / 23
Unified LL Analysis use the so-called Beatty LL condition the EBNF grammar is LL if and only if it is LR (use the classical LR condition) and every macro-state of the ext. pilot graph has a base that is void or contains at most one state (or one marked rule in the classical version) condition easy to be checked on the pilot LL look-ahead sets are visibile in the pilot the construction of the top-down or recursive descent syntax analyser is sufficiently easy 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 18 / 23
Example LR yet not LL 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 19 / 23
Pro s of the Unification save teaching time, notion dup.s and tricky rules constructing the extended pilot graph is similar to the classical costruction for non-ext. gram.s in the unified scenario this will be the only rather ingenious step for the student costructing the SA s (top-down or bottom-up) is automatic (here the SA costruction rules are skipped for brevity) the extended pilot costruction looks like similar to the Berry-Sethi algorithm for obtaining a DFSA directly from a regular expression 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 20 / 23
and Con s there may be a little more complexity in deriving the Syntax Analyser (SA) from the pilot graph LR: the shift-reduction SA must in some cases use a stack enhanced with pointers (when reducing an ext. rule) LL: the SA implementation by recursive procedures may still be done yet is somehow less immediate but the conceptual steps (pilot graph costruction verification of the LL and LR conditions grammar modification if necessary for achieving determinism) ARE VERY DIRECT AND WELL INTEGRATED 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 21 / 23
Extension to Earley the Earley algorithm is used for analysing non-det. grammars (or even ambiguous) applied by Earley himself to EBNF gram.s but with a few errors and a hard notation might be made more easy to teach and handy to apply by representing the EBNF grammar as a network of recursive automata currently under investigation 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 22 / 23
Deployment Plan unification theory developed enough for the LL and LR methodologies (draft report almost finished) under investigation for the Earley algorithm teaching material (slides) still to be designed and written in the next two months (should be mostly based on examples) plan to test the unified syntax analysis methodology in the 2011-12 course Formal Languages & Compilers (Oct. 2011 Jan. 2012) if test successful, upgrade the next edition of the Springer textbook Formal Languages & Compilers (and suppress the classical version of syntax analysis) 5-7 settembre 2011 - PRIN Unifying LL and LR analysis pp. 23 / 23