Fall 2016-2017 Compiler Principles Context-free Grammars Refresher Roman Manevich Ben-Gurion University of the Negev 1
xample grammar S S ; S S id := S print (L) id num + L L L, shorthand for Statement shorthand for xpression shorthand for List (of expressions) 2
CFG terminology S S ; S S id := S print (L) id num + L L L, Symbols: Terminals (tokens): ; := ( ) id num print Non-terminals: S L Start non-terminal: S Convention: the non-terminal appearing in the first derivation rule Grammar productions (rules) N α 3
More definitions Sentential form: a sequence of symbols, terminals (tokens) and non-terminals Sentence: a sequence of terminals (tokens) Derivation step: given a sentential form αnβ and rule N µ a step is the transition αnβ αµβ Derivation sequence: a sequence of derivation steps 1 k such that i i+1 is the result of applying one production and k is a sentence 4
Language of a CFG A word ω is in L(G) (valid program) if there exists a corresponding derivation sequence Start the start symbol Repeatedly replace one of the non-terminals by a right-hand side of a production Stop when the sentence contains only terminals ω is in L(G) if S * ω Leftmost derivation Rightmost derivation 5
Leftmost derivation a := 56 ; b := 7 + 3 1 2 3 4 5 6 7 8 S S ; S S id := S print (L) id num + L L L, S => S ; S => id := ; S => id := num ; S => id := num ; id := => id := num ; id := + => id := num ; id := num + => id := num ; id := num + num id := num ; id := num + num 6
Rightmost derivation a := 56 ; b := 7 + 3 1 2 3 4 5 6 7 8 S S ; S S id := S print (L) id num + L L L, S => S ; S => S ; id := => S ; id := + => S ; id := + num => S ; id := num + num => id := ; id := num + num => id := num ; id := num + num id := num ; id := num + num 7
Canonical derivations Leftmost/rightmost derivations may not be unique but they allow describing a derivation by the sequence of production rules taken (since non-terminal is already known) Leftmost derivation example: 1, 2, 5, 2, 6, 5, 5 Rightmost derivation example: 1, 2, 6, 5, 5, 2, 5 8
Parse trees Tree nodes are symbols, children ordered left-to-right ach internal node is non-terminal and its children correspond to one of its productions N µ 1 µ k N µ 1 µ k Root is start non-terminal Leaves are tokens Yield of parse tree: left-to-right walk over leaves 9
Parse tree exercise S S ; S S id := S print (L) id Draw parse tree for expression num + L L L, id := num ; id := num + num 10
Parse tree exercise S S ; S S id := S print (L) id num + L L L, quivalently add parentheses labeled by non-terminal names S S Order-independent representation id := num ; id := num + num ( S ( S a := ( 56) ) S ; ( S b := ( ( 7) + ( 3) ) ) S ) S S 11
Capabilities and limitations of CFGs CFGs naturally express Hierarchical structure A program is a list of classes, A Class is a list of definition Alternatives A definition is either a field definition or a method definition Beginning-end type of constraints Balanced parentheses S (S)S ε Cannot express p. 173 Correlations between unbounded strings (identifiers) For example: variables are declared before use: ω S ω Handled by semantic analysis (attribute grammars) 12
Badly-formed grammars By Oren neu dag (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons 13
Badly-formed grammars A non-terminal N is reachable if S * αnβ A non-terminal N is generating if N * ω A grammar G is badly-formed if it either contains unreachable nonterminals or non-generating non-terminals G 1 = S x N y G 2 = S x N N a N b N xercise: algorithm to test whether a grammar is badly-formed Theorem: for every grammar G there exists an equivalent wellformed grammar G ( that is, L(G)=L(G ) ) Proof: exercise From now on, we will only handle well-formed grammars 14
Ambiguity in Context-free grammars 15
Sometimes there are two parse trees Arithmetic expressions: 1 + 2 + 3 id num 1 + (2 + 3) (1 + 2) + 3 + * ( ) num(1) + num(2) + num(3) num(1) + num(2) + num(3) Leftmost derivation + num + num + + num + num + num + num + num Rightmost derivation + + num + + num + num + num num + num + num 16
Is ambiguity a problem for compilers? Arithmetic expressions: 1 + 2 + 3 id num 1 + (2 + 3) (1 + 2) + 3 + Depends on semantics * ( ) num(1) + num(2) + num(3) = 6 num(1) + num(2) + num(3) = 6 Leftmost derivation + num + num + + num + num + num + num + num Rightmost derivation + + num + + num + num + num num + num + num 17
Problematic ambiguity example Arithmetic expressions: 1 + 2 * 3 id num 1 + (2 * 3) This is what we (1 + 2) * 3 + usually want: * has * precedence over + ( ) num(1) + num(2) * num(3) = 7 num(1) + num(2) * num(3) = 9 Leftmost derivation + num + num + * num + num * num + num * num Rightmost derivation * * num + * num + num * num num + num * num 18
Ambiguous grammars A grammar is ambiguous if there exists a word for which there are Two different leftmost derivations Two different rightmost derivations Two different parse trees Property of grammars, not languages 19
Facts about ambiguous grammars Some languages are inherently ambiguous no unambiguous grammars exist [Parikh 1961] Checking whether an arbitrary grammar is ambiguous is undecidable [Hopcroft, Motwani, Ullman, 2001] 20