Context Free Languages and Pushdown Automata COMP2600 Formal Methods for Software Engineering Ranald Clouston Australian National University Semester 2, 2013 COMP 2600 Context Free Languages and Pushdown Automata 1
Parsing The process of parsing a program is partly about confirming that a given program is well-formed syntax. But it is also about representing the structure of the program so that it can be executed semantics. For this purpose, the trail of sentential forms created en route to generating a given sentence is just as important as the question of whether the sentence can be generated or not. COMP 2600 Context Free Languages and Pushdown Automata 2
The Semantics of Parses Take the code if e1 then if e2 then s1 else s2 where e1, e2 are boolean expressions and s1, s2 are subprograms. Does this mean if e1 then ( if e2 then s1 else s2 ) or if e1 then ( if e2 else s1 ) else s2 We d better have an unambiguous way to tell which is right, or we cannot know what the program will do at runtime! COMP 2600 Context Free Languages and Pushdown Automata 3
Ambiguity Recall that we can present CFG derivations as parse trees. Until now this was mere pretty presentation; now it will become important. A context-free grammar G is unambiguous iff every string can be derived by at most one parse tree. G is ambiguous iff there exists any word w L(G) derivable by more than one parse tree. COMP 2600 Context Free Languages and Pushdown Automata 4
Example: If-Then and If-Then-Else Consider the CFG S if bexp then S if bexp then S else S prog where bexp and prog stand for boolean expressions and (if-statement free) programs respectively, defined elsewhere. The string if e1 then if e2 then s1 else s2 then has two parse trees: if e1 then if e2 then S S S if e1 then S S else S if e2 then S s1 s2 s1 else S s2 COMP 2600 Context Free Languages and Pushdown Automata 5
Example: If-Then and If-Then-Else That grammar was ambiguous. But here s a grammar accepting the exact same language that is unambiguous: S if bexp then S T T if bexp then T else S prog There is now only one parse for if e1 then if e2 then s1 else s2. This is given on the next slide: COMP 2600 Context Free Languages and Pushdown Automata 6
Example: If-Then and If-Then-Else if e1 then if e2 then S S T T else You cannot parse this string as if e1 then ( if e2 else s1 ) else s2. s1 S T s2 COMP 2600 Context Free Languages and Pushdown Automata 7
Reflecting on This Example We have seen that the same language can be presented with ambiguous and unambiguous grammars. Ambiguity is in general a property of grammars, not of languages. From the point of view of semantics and parsing, it is often desirable to turn an ambiguous grammar into an equivalent unambiguous grammar. This generally involves choices that are not driven by the language itself. E.g. there exists another grammar for the language of the previous slides that is unambiguous and allows the parse if e1 then ( if e2 else s1 ) else s2 but not if e1 then ( if e2 then s1 else s2 ). COMP 2600 Context Free Languages and Pushdown Automata 8
What Ambiguity Isn t You might wonder if our grammar is still ambiguous, given the production T if bexp then T else S After a derivation that uses this, we have a choice of whether we expand the T or the S first. Is this an ambiguity? No. A context-free grammar gets its name because non-terminals can be expanded without regard for the context they appear in. In other words, it doesn t matter if you expand T or S first. From the perspective of the parse tree these expansions are happening in parallel. This is a reason why a parse tree is a better formalism for presenting a CFG derivation than listing each step. COMP 2600 Context Free Languages and Pushdown Automata 9
Inherently Ambiguous Languages In fact not all context-free languages can be given unambiguous grammars some are inherently ambiguous. Consider the language L = {a i b j c k i = j or j = k} How do we know that this is context-free? First, notice that L = {a i b i c k } {a i b j c j } We then combine CFGs for each side of this union (a standard trick): S T W T UV U aub ε V cv ε W XY X ax ε Y by c ε COMP 2600 Context Free Languages and Pushdown Automata 10
Inherently Ambiguous Languages The problem with this CFG is that the union we used has a non-empty intersection, where the a s, b s, and c s all have equal number. The sentences in this intersection are a source of ambiguity: a U U S T V b c V X a X S W Y b Y ε ε ε ε c In fact (not proved here!) there is no alternative choice of grammar for this language that avoids this ambiguity. COMP 2600 Context Free Languages and Pushdown Automata 11
The Bad News We would like to have an algorithm that turns grammars into equivalent unambiguous grammars where possible. However this is an uncomputable problem. Worse determining whether a grammar is ambiguous or not in the first place is also uncomputable! Uncomputable problems are everyone in computer science. many more next week in particular. You will see The best response is not despair; rather we should see what tricks and techniques might help us out at least some of the time. COMP 2600 Context Free Languages and Pushdown Automata 12
Example: Subtraction Consider the grammar S S S int where int could be any integer and the symbol is intended to be executed as subtraction. This grammar is ambiguous, and this matters: S S S 1 5 S 5 3 3 1 The left tree evaluates 5 3 1 to 1; the left evaluates the same string to 3! COMP 2600 Context Free Languages and Pushdown Automata 13
Technique 1: Associativity We can remove the ambiguity of a binary infix operator by making it associate to the left or right. S S int int Now 5 3 1 can only be read as (5 3) 1 - this is left associativity. For right associativity we would use the production S int S instead. Idea: Force one side of our operator to a lower level, making sure we avoid loops in our grammar that allow the original non-terminal to be recovered. Here we force the right hand side of the minus sign to the lowest possible level a terminal symbol. COMP 2600 Context Free Languages and Pushdown Automata 14
Example: Multiplication and Addition S S S S + S int where is to be executed as multiplication and + as addition. Again this is obviously ambiguous 1 + 2 3 could evaluate to 7 or 9. (Note that 1 + 2 + 3 is also a source of ambiguity as it can be produced by different parse trees, even though it is not ambiguous in the interpretation we have in mind.) If all we care about is resolving ambiguity we can use the same trick as the last slide, making both and + left (or right) associative. But this is not the behaviour we expect from these operations: we expect to have higher precedence than +. COMP 2600 Context Free Languages and Pushdown Automata 15
Technique 2: Precedence S S + T T T T int int Given a string 1+2 3, or 2 3+1, we have no choice but to expand to S+T first, so that (thinking bottom-up) the + will be last command to be executed. Suppose we tried to derive 1 + 2 3 by first doing S T T 3. We are then stuck because we cannot send T to 1 + 2! As with associativity this trick works by forcing ourselves down to a lower level to generate parts of our sentences. Here we have three levels: S, then T, then the integers as non-terminals. COMP 2600 Context Free Languages and Pushdown Automata 16
Example: Basic Arithmetic S S + T S T T T T U T /U U U (S) int Note that we have brackets available to give a clearly labelled way to loop back to the top if we want to break the usual rules of arithmetic to get the execution (1 + 2) 3 then we must indicate this with explicit brackets. (Note also that the previous slides languages were actually regular and so could have been generated by right-linear grammars. The language above is truly context-free because of the need to keep track of bracket balancing to an arbitrary depth.) COMP 2600 Context Free Languages and Pushdown Automata 17
Example: Balanced Brackets The following grammar generates a language where each left bracket is exactly matched by a closing bracket: S ε (S) SS This is ambiguous in a rather stupid way: to generate () we could use the derivation S (S) (), which seems sensible. But we could also start with S SS, then use the left S to derive () as above while sending the right to ε immediately. Or we could have done the opposite. In fact there are infinitely many parse trees for any string in the language! Edit: There is another source of ambiguity here ()()() has two parses even without such abuse of ε. We fix this on the next slide with left associativity. COMP 2600 Context Free Languages and Pushdown Automata 18
Technique 3: Controlling ε Instead, we ensure there is only one way to access the empty string: S ε T T TU U U () (T ) The string ε can be derived directly from S, but everything else must go through T. This is another technique that relies on creating a hierarchy of levels within our grammar. ε as a source of ambiguity can be easy to miss, so keep your eyes out for it! COMP 2600 Context Free Languages and Pushdown Automata 19
Hierarchy of Automata We have seen that there is a hierarchy of grammars. A language may be generated by grammars at a certain level, but not by less powerful grammars. We have seen that regular (right-linear) grammars are exactly as expressive as finite state automata. Are there notions of automata exactly as expressive as context-free grammars? Context-sensitive grammars? Unrestricted grammars? Yes - there is a hierarchy of automata. Just as we needed a general definition of grammar to define their hierarchy, we need a general view of automata... COMP 2600 Context Free Languages and Pushdown Automata 20
General Structure of Automata a0 a1 a2....... an read head input tape Finite State Control Auxiliary Memory The input tape is a sequence of tokens. Each time a symbol is processed the read head advances. The finite state control (FSC) can be in any one of a finite number of states. The auxiliary memory is usually a linear organisation (e.g. a stack or array). COMP 2600 Context Free Languages and Pushdown Automata 21
General Automata ctd Each action of the machine may change the FSC state, change the auxiliary memory, or advance to the next input symbol, consuming the current one. The action of the machine depends on the current input symbol, the current FSC state, and the current memory. The machine starts in some particular start state (q 0 ), with the read head at the first input symbol, and the memory containing only the start symbol (Z). A machine accepts an input string as a sentence of the language if it reaches a final state (in F) with the input exhausted (we sometimes also require that the memory be emptied). COMP 2600 Context Free Languages and Pushdown Automata 22
Automata and Grammars The kind of auxiliary memory in a machine determines the class of languages that the machine can recognise: Language Class regular context-free context-sensitive unrestricted Memory none stack tape (bounded by input length) unbounded tape We have already looked at Finite State Automata (automata without memory) and their relation to regular languages. We now consider Push-Down Automata (i.e. automata with stack memory) and their relation to context-free grammars and languages. COMP 2600 Context Free Languages and Pushdown Automata 23
Push-down Automata PDA a0 a1 a2....... an read head input tape Finite State Control zk z2 z1 stack memory COMP 2600 Context Free Languages and Pushdown Automata 24
PDAs ctd Each action of the machine may involve change to the FSC state, pushing or popping the stack, and advancing to the next input symbol. The action of the machine may depend on the current FSC state, the current input symbol, and the current top-of-stack symbol. The machine accepts an input string if it reaches a final state, with the input exhausted, and the stack empty. ASIDE: Other (equivalent) definitions of PDAs exist, where an input string is accepted if the PDA has an empty stack with the input exhausted an input string is accepted if the PDA is in a final state with the input exhausted COMP 2600 Context Free Languages and Pushdown Automata 25
Example {a n b n n 1} Recall that this language cannot be recognised by a FSA (because there can only be a finite number of states), but can be generated by a context-free grammar. It can also be recognised by a PDA. Ad hoc design: phase 1: (state q 1 ) push a s from the input onto the stack phase 2: (state q 2 ) pop a s from the stack, if there is a b on input finalise: if the stack is empty and the input is exhausted in the final state (q 3 ), accept the string. COMP 2600 Context Free Languages and Pushdown Automata 26
Deterministic PDA Definition A deterministic PDA has the form (Q,q 0,F, Σ, Γ,Z, δ), where Q is the set of states, q 0 Q is the initial state and F Q are the final states; Σ is the alphabet, or set of input symbols; Γ is the set of stack symbols, and Z Γ is the initial stack symbol; δ is a (partial) transition function δ : Q (Σ {ε}) Γ Q Γ δ : (state, input token or ε, top-of-stack) (new state, stack string) such that for all q Q and s Γ, if δ(q,ε,s) is defined then δ(q,a,s) is undefined for all a Σ. This condition is vital to maintain determinism. COMP 2600 Context Free Languages and Pushdown Automata 27
Notation Given the transition function δ : Q (Σ {ε}) Γ Q Γ δ : (state, input token or ε, top-of-stack) (new state, stack string) we write δ(q,a,s) = q /σ to show how this function operates. Then the string σ in the result is the string with which we replace the top-ofstack symbol s. (Doing this makes it simple to specify pushes and pops in a uniform way.) We will usually indicate final states by underlining them, e.g. q. COMP 2600 Context Free Languages and Pushdown Automata 28
Two types of PDA transition when δ contains (q 1,x,s) q 2 /σ, state input stack the PDA can go from q 1 next symbol is x s on top-of-stack to q 2 x is consumed σ replaces s on stack when δ contains (q 1,ε,s) q 2 /σ, state input stack the PDA can go from q 1 any input, or none s on top-of-stack to q 2 input is ignored σ replaces s on stack Such transitions do not look at or consume an input symbol. COMP 2600 Context Free Languages and Pushdown Automata 29
Example ctd Recall that the stack starts off containing only the initial symbol Z. PDA to recognise a n b n (start state q 0, final state(s) underlined): δ(q 0,a,Z) = q 1 /az push first a δ(q 1,a,a) = q 1 /aa push further a s δ(q 1,b,a) = q 2 /ε start popping a s δ(q 2,b,a) = q 2 / ε pop further a s δ(q 2,ε,Z) = q 3 /ε accept Note that δ is a partial function, undefined for many arguments. COMP 2600 Context Free Languages and Pushdown Automata 30
Example ctd PDA Trace PDA configurations can be written as a triple (state, remaining input, stack) with the top of stack to the left. (q 0,aaabbb,Z) (q 1,aabbb,aZ) (push first a) (q 1,abbb,aaZ) (q 1,bbb,aaaZ) (q 2,bb,aaZ) (q 2,b,aZ) (q 2,ε,Z) (q 3,ε,ε) (push further a s) (push further a s) (start popping a s) (pop further a s) (pop further a s) (accept) The machine halts in the final state with input exhausted and an empty stack, so the string is accepted. COMP 2600 Context Free Languages and Pushdown Automata 31
Example ctd Rejection The string aaba should be rejected by this PDA: (q 0,aaba,Z) (q 1,aba,aZ) (q 1,ba,aaZ) (q 2,a,aZ)??? No transition applies, and the PDA is stuck without reaching a final state. Rejection happens when the transition function is undefined for the current configuration ( that is, state, input symbol and top of stack symbol ). COMP 2600 Context Free Languages and Pushdown Automata 32
Example: Palindromes with Centre Mark Consider the language {wcw R w {a,b} w R is w reversed} This is context-free, and we can design a deterministic PDA to accept it: Push a s and b s onto the stack as we seem them; When we see c, change state; Now try to match the tokens we are reading with the tokens on top of the stack, popping as we go; If the top of the stack is the empty stack symbol Z, pop it and enter the final state via an ε-transition. Hopefully our input has been used up too! Full formal details are left as an exercise. COMP 2600 Context Free Languages and Pushdown Automata 33
Non-Deterministic PDAs For deterministic PDAs transitions are a (partial) function: δ : Q (Σ {ε}) Γ Q Γ δ : (state, input token or ε, top-of-stack) (new state, stack string) with a side-condition about ε-transitions. For non-deterministic PDAs transitions are a relation δ Q (Σ {ε}) Γ Q Γ with no side-condition. There may be configurations where there are multiple choices of transition. COMP 2600 Context Free Languages and Pushdown Automata 34
Non-Deterministic PDAs ctd. Recall for finite state automata, non-determinism can be more convenient, but does not give extra power (thanks to the subset construction). This is not the case for PDAs - non-deterministic PDAs can recognise languages that deterministic PDAs cannot. It turns out that non-deterministic PDAs are the more important from the perspective of Chomsky s hierarchy. COMP 2600 Context Free Languages and Pushdown Automata 35
Example: Even-Length Palindromes The language of even length palindromes {ww R w {a,b} w R is w reversed} is context-free but cannot be recognised by a deterministic PDA, because without a centre mark it cannot know whether it is in the first half of a sentence (so should be pushing everything into memory) or the second half (so it should be matching input and stack, and popping). But a non-deterministic PDA can recognise this language via the transition δ(q, ε, x) = r/x where x {a,b,z}, q is the push state, and r the match and pop state. In other words, we continually guess which job we should be doing! COMP 2600 Context Free Languages and Pushdown Automata 36
Grammars and PDAs Theorem The class of languages recognised by non-deterministic PDA s is exactly the class of context-free languages. We will only justify this result in one direction: for any CFG, there is a corresponding PDA. This is the most interesting direction since it is the basis of automatically deriving parsers from grammars. (The other direction is a bit complicated). COMP 2600 Context Free Languages and Pushdown Automata 37
From CFG to PDA The PDA has three states: q 0 (initial), q 1 (processing), and q 2 (final). alphabet will be that of the CFG, and its stack symbols will be all the CFG s terminals and non-terminals. 1. Initialise the process by pushing the start symbol S onto the stack, and enter state q 1 : δ(q 0,ε,Z) q 1 /SZ Its 2. If a non-terminal is on top of stack, replace it with the right hand side of a production. For all productions A α: δ(q 1,ε,A) q 1 /α continued... COMP 2600 Context Free Languages and Pushdown Automata 38
From CFG to PDA, ctd 3. For all terminal symbols t, pop the stack if it matches the input: δ(q 1,t,t) q 1 /ε 4. For termination, add the transition, with final state q 2 : δ(q 1,ε,Z) q 2 /ε In general we get a non-deterministic PDA since there may be several productions for each non-terminal. COMP 2600 Context Free Languages and Pushdown Automata 39
Example Derive a PDA for a CFG S S + T T T T U U U (S) int 1. Initialise: δ(q 0,ε,Z) q 1 /SZ 2. Expand non-terminals: δ(q 1,ε,S) q 1 /S + T δ(q 1,ε,T ) q 1 /U δ(q 1,ε,S) q 1 /T δ(q 1,ε,U) q 1 /(S) δ(q 1,ε,T ) q 1 /T U δ(q 1,ε,U) q 1 /int COMP 2600 Context Free Languages and Pushdown Automata 40
CFG to PDA ctd 3. Match and pop terminals: δ(q 1,+,+) q 1 /ε δ(q 1,, ) q 1 /ε δ(q 1,int,int) q 1 /ε δ(q 1,(,() q 1 /ε δ(q 1,),)) q 1 /ε 4. Terminate: δ(q 1,ε,Z) q 2 /ε COMP 2600 Context Free Languages and Pushdown Automata 41
Example Trace (q 0, int int, Z) (q 1, int int, SZ) (q 1, int int, T Z) (q 1, int int, T UZ) (q 1, int int, U UZ) (q 1, int int, int UZ) (q 1, int, UZ) (q 1, int, UZ) (q 1, int, intz) (q 1, ε, Z) (q 2, ε, ε) accept COMP 2600 Context Free Languages and Pushdown Automata 42
A Context-Sensitive Language In case you re curious, a brief look at context-sensitive languages. The following language is not context-free: {a n b n c n n 1} Intuitively, we can imagine a CFG generating either the ab pairs or the bc pairs, but this language requires us to keep the generation process in step, in two different points in the sentential forms. A context-sensitive grammar is on the next slide. Each production has a righthand side of equal or longer length than their left. (If we wanted to include the empty word also, we need a special exemption to this requirement.) COMP 2600 Context Free Languages and Pushdown Automata 43
A Context-Sensitive Language ctd. On the left we have a context-sensitive grammar, and on the right an example derivation: S abc S asbc S asbc aabcbc CB BC aabcbc ab ab aabbcc bb bb aabbcc bc bc aabbcc cc cc aabbcc The automata that recognise CSGs have a tape memory, of length bounded by a linear function of the length of the input. COMP 2600 Context Free Languages and Pushdown Automata 44