Delimited Continuations, Applicative Functors and Natural Language Semantics

Transcription

1 Delimited Continuations, Applicative Functors and Natural Language Semantics Björn Bringert Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg November 17, 2008 Abstract We present a continuation-based approach to writing compositional semantics for type-theoretic grammars which accounts for quantifier scope ambiguity and scope islands. 1 Introduction Montague [1973] uses compositional rules that map English expressions to an intentional logic which combines higher-order logic with λ-calculus. There is a large body of work that extends this treatment to cover additional phenomena, such as scope ambiguities. These efforts, such as Cooper storage [Cooper, 1983], are often expressed as extensions to the core λ-calculus. We wish to stay within a (polymorphically) typed λ-calculus. Continuations have been proposed as an approach to deal with a range of semantic phenomena [Barker, 2002]. We show that it is possible to handle scope ambiguity and scope islands in a polymorphically typed λ-calculus. The language that we use has some syntactic extensions to λ-calculus, but these have straightforward translations to the core calculus. We use first-order logic as our target language, rather than the higher-order logic used by Montague [1973]. This lets us use our semantics for practical applications with existing automated first-order reasoning tools. In Section 2 we present a fragment of English described using a Grammatical Framework (GF) [Ranta, 2004a] grammar. Section 3 constitutes the bulk of this paper. It presents a sequence of progressively refined semantics for fragments of English. We start by introducing the most basic machinery in Section 3.1, which handles a small language fragment with only proper names and transitive and intransitive verbs. In this fragment, all constructs have straightforward interpretations. In Section 3.2, we add determiners, along with simple nouns, relational nouns and relative clauses. The problem of handling quantifiers in noun phrases is dealt with using Montague s type raising trick. In Section 3.3, we tackle quantifier scope ambiguity, by lifting our entire interpretation to a dynamic logic, in this case an applicative functor for continuation computations. By using a non-deterministic evaluation order, we can generate all possible quantifier scopings. In Section 3.4 we add a construct for delimited continuations, 1

2 and use it to handle scope islands. Section 4 describes how this approach can be implemented efficiently in the Haskell programming language. Section 5 gives a brief overview of some related work. 2 Syntax Grammatical Framework (GF) [Ranta, 2004a] is a type-theoretical grammar formalism. In order to introduce GF, and to give us an example syntax to work with in the following section, we present an example grammar for a fragment of English. The abstract syntax shown in Figure 1 defines categories (cat) and functions (fun). An example of an abstract syntax tree in this grammar is PredVP (DetCN Every (UseN Woman)) (UseV Walk). The concrete syntax shown in Figure 2 defines the linearization type (lincat) of each abstract syntax category, and the linearization (lin) of each abstract syntax function. In this concrete syntax, the abstract syntax tree above is linearized to: {s = every woman walks } In this simple example, no inflection, agreement or other complex morphosyntactic features are needed to implement the English concrete syntax. However, GF does allow more sophisticated linearization rules, with records, finite functions and algebraic data types, which can be used to implement more complex grammars without changing the abstract syntax. GF can be used to create multilingual grammars by associating multiple concrete syntaxes with a single abstract syntax. 3 Semantics Since we define the syntax with a GF grammar, we can write our interpretation functions over GF abstract syntax trees, which abstract away from details such as word order and agreement. We interpret each abstract syntax term as a term in a combination of λ-calculus and first-order logic with equality. We use the customary connectives and quantifiers, n-ary predicates, equality, inequality. When we show formulas which are the result of semantic interpretation, the will sometimes be simplified according to the rules of first-order logic. 3.1 Fragment 1: Basics Our first language fragment only contains sentences made up of proper names, and transitive or intransitive verbs. This lets use handle sentences such as John walks and John loves Mary, which are assigned the first-order logic formulas walk(john) and love(john, M ary), respectively. For each category C in the abstract syntax, there is a function ic : C C, where C is the interpretation type of C. We first provide a straightforward semantics for the lexical items. Proper names are plain constants: 2

3 abstract Toy = { } cat S; RS; VP; NP; Det; CN; cat V; V2; N; N2; PN; fun PredVP : NP VP S; fun RelVP : VP RS; fun UseV : V VP; fun ComplV2 : V2 NP VP; fun DetCN : Det CN NP; fun Everyone : NP; fun Someone : NP; fun UsePN : PN NP; fun Every : Det; fun A : Det; fun UseN : N CN; fun ComplN2 : N2 NP CN; fun RelCN : CN RS CN; fun Walk : V; fun Love, Eat : V2; fun Man, Woman, Burger : N; fun Owner : N2; fun John, Bill, Mary : PN; Figure 1: Toy.gf: Abstract syntax for a small language fragment. 3

4 concrete ToyEng of Toy = { } lincat S, RS, VP, NP, Det, CN = {s : Str}; lincat V, V2, N, N2, PN = {s : Str}; lin PredVP np vp = {s = np.s + vp.s }; lin RelVP vp = {s = who + vp.s }; lin UseV v = {s = v.s }; lin ComplV2 v2 np = {s = v2.s + np.s }; lin DetCN det cn = {s = det.s + cn.s }; lin Everyone = {s = everyone }; lin Someone = {s = someone }; lin UsePN pn = {s = pn.s }; lin Every = {s = every }; lin A = {s = a }; lin UseN n = {s = n.s }; lin ComplN2 n2 np = {s = n2.s + np.s }; lin RelCN cn rs = {s = cn.s + rs.s }; lin Walk = {s = walks }; lin Love = {s = loves }; lin Eat = {s = eats }; lin Man = {s = man }; lin Woman = {s = woman }; lin Burger = {s = burger }; lin Owner = {s = owner + of }; lin John = {s = John }; lin Bill = {s = Bill }; lin Mary = {s = Mary }; Figure 2: ToyEng.gf: Concrete syntax for a small fragment of English. 4

5 ipn : PN Ind ipn John = John ipn Mary = Mary ipn Bill = Bill Intransitive verbs are one-place predicates: iv : V (Ind Prop) iv Walk = λx. walk(x) Transitive verbs are two-place predicates. We take the subject as the first function argument, and the object as the second, which is the opposite to what Montague does. In addition to being more intuitive (in the opinion of the author), this order lets us use function composition in the complementation rules below. iv2 : V2 (Ind Ind Prop) iv2 Eat = λx y. eat(x, y) iv2 Love = λx y. love(x, y) Now for the semantics of the syntactic constructs. Noun phrases, which are only proper names at this point, are interpreted as individuals: inp : NP Ind inp (UsePN pn) = ipn pn Like intransitive verbs, verb phrases are interpreted as one-place predicates. ivp : VP (Ind Prop) For verb phrases formed from just an intransitive verb, nothing needs to be done: ivp (UseV v) = iv v In the case of transitive verb complementation, the interpretation of the object is given as the second argument to the two-place predicate which is the interpretation of the transitive verb (note that in our notation, function application binds harder than λ-abstraction): ivp (ComplV2 v2 np) = λy. (iv2 v2 ) y (inp np) Sentences are interpreted as propositions, and verb phrase predication is simply the application of the one-place verb phrase predicate to the noun phrase individual. is : S Prop is (PredVP np vp) = (ivp vp) (inp np) 5

6 3.2 Fragment 2: Adding determiners When we add determiners to our language fragment, we will need some way to handle quantifiers, as we would for example like the sentence everyone walks to have the interpretation x. walk(x). Our previous type of NP interpretations, Ind, is insufficient since we need to be able to introduce the universal quantifier on the top-level of the formula. Montague [1973] solved this problem by changing the type of NP interpretations to (Ind Prop) Prop, and we will do the same. We first add interpretations for the remaining lexical categories. Simple nouns are one-place predicates: in : N (Ind Prop) in Man = λx. man(x) in Woman = λx. woman(x) in Burger = λx. burger(x) Relational nouns are two-place predicates: in2 : N2 (Ind Ind Prop) in2 Owner = λx y. owner(x, y) Noun phrases now have the higher type needed to include quantified noun phrases: inp : NP ((Ind Prop) Prop) inp Everyone = λv. x. v x inp Someone = λv. x. v x Since we have changed the interpretation type of noun phrases, we also need to change the rule for UsePN, which lifts proper names to noun phrases: inp (UsePN pn) = λv. v (ipn pn) We can now handle noun phrases consisting of a determiner and a common noun, as in every man walks, which we would like to interpret as x. man (x) walk (x). inp (DetCN det cn) = (idet det) (icn cn) That requires determiners to be interpreted as two-place predicates over oneplace predicates. idet : Det (Ind Prop) (Ind Prop) Prop idet Every = λu v. x. u x v x idet A = λu v. x. u x v x We also have relative sentences, which are interpreted as one-place predicates. E.g. who walks, RelVP (UseV Walk) is interpreted as λx. walk (x). irs : RS (Ind Prop) irs (RelVP vp) = ivp vp Common nouns are one-place predicates. 6

7 icn : CN (Ind Prop) For simple nouns, there is nothing left to do: icn (UseN n) = in n Relational noun complementation is handled by letting the second argument of the relational noun be filled by the complement (this is done in a roundabout way because of the Montague trick). icn (ComplN2 n2 np) = λx. (inp np) ((in2 n2 ) x) or, equivalently, using the function composition operator (function application binds harder than composition): icn (ComplN2 n2 np) = inp np in2 n2 A relative sentence that modifies a common noun is interpreted as the conjunction of the propositions resulting from the two constituents. icn (RelCN cn rs) = λx. (icn cn) x (irs rs) x Since we have changed the type of noun phrase interpretations, we also need to change the rules which make use of noun phrases. In the case of NP VP sentences, all we need to do is to change the order of application. We apply the noun phrase interpretation ((Ind Prop) Prop) to the verb phrase interpretation (Ind Prop) to get the final Prop). is : S Prop is (PredVP np vp) = (inp np) (ivp vp) We also need to change the rule for transitive verb complementation since it uses NP: ivp (ComplV2 v2 np) = λx. (inp np) ((iv2 v2 ) x) This could also be written using, as with ComplN2 above. 3.3 Fragment 3: Quantifier scope ambiguity Consider a sentence such as every man loves a woman. previous section would interpret this as x. man(x) y. woman(y) love(x, y) However, the sentence also has the alternative meaning y. woman(y) x. man(x) love (x, y) The rules in the that is, that there is some woman whom every man loves. To be able to generate both these readings, we need to allow the quantifiers from a nested noun phrase to escape to the top level of the formula. We also need a mechanism for allowing interpretation to be non-deterministic. 7

8 A number of approaches have been proposed to handle quantifier scope ambiguity, such as Cooper storage [Cooper, 1983], and its improved version, Keller storage [Keller, 1988]. While it is possible to implement Keller storage in a typed λ-calculus, the result is not very elegant. Cooper storage seems difficult to implement in a typed way, because of the unsoundness that Keller pointed out. Instead of storage, we can use continuations to deal with quantifier scope. Barker [2002] shows that continuations can be used to handle a range of semantic phenomena, and that quantifier scope ambiguity can be handled by using a nondeterministic evaluation order in the continuized semantics. A continuation monad can be used to hide the plumbing details of continuation passing style in natural language semantics [Shan, 2001]. As Shan notes, one advantage of using a monadic style is that the monad in question can easily be replaced with a more elaborate one when we want to account for additional details, without having to change all the interpretation rules. However, it is in general not possible to make a continuation monad with non-deterministic evaluation order, since the bind operation of a monad ( in [Shan, 2001], >= in Haskell) requires left-to-right evaluation. But, fortunately for us, monads can be generalized to applicative functors [McBride and Paterson, 2008], whose combining operator ( ) is order-agnostic. Thus, instead of a continuation monad for natural language semantics, we propose a continuation applicative functor with non-deterministic evaluation order. In this section, we show that an applicative functor is sufficient for the needs of our semantics, and that it can handle quantifier scope ambiguities Interpretation Functor In the interest of readability, we will use set notation (e.g. {x, y, z}) below. This can be straightforwardly replaced by the Church encoding of lists to obtain a pure λ-calculus implementation. A non-deterministic continuized computation is a set of functions that return a value given its continuation (context). We will use Cont o a to mean a nondeterministic computation that returns a value of type o, and that gives access to a continuation that accepts an argument of type a. type Cont o a = {(a o) o} The raise function lifts a pure value into the functor. raise : a Cont o a raise a = {λc. c a} The operator performs lifted function application with both evaluation orders. In case the reader is concerned with the exponential behavior of this implementation, we refer to section 4.1. ( ) : Cont o (a b) Cont o a Cont o b xs ys = {λc. x (λf. y (λa. c (f a))) x xs, y ys} {λc. y (λa. x (λf. c (f a))) x xs, y ys} The raise (also called pure) and functions make this an applicative functor [McBride and Paterson, 2008]. Applicative functors generalize monads [Wadler, 8

9 1992, Moggi, 1989]. Shan [2001] shows how a continuation monad can be used to handle quantification. However, that treatment does not take quantifier scope ambiguity into account. We also define an evaluation operator ε (symbol borrowed from Shan [2004]) which runs a computation and retrieves all possible results. ε : Cont o o {o} ε x = {y (λf. f ) y x} We finally add a control operator ξ that gives access to the current continuation. Its implementation turns out to be very simple: ξ : ((a o) o) Cont o a ξ f = {f } In the interest of readability, instead of ξ (λκ. e), we will write ξκ. e Semantics In this example, the outer return type is always Prop, so we use I a as a shorthand for Cont Prop a. type I a = Cont Prop a We first lift the parts of our semantics that do not introduce any quantifiers to use our new continuation functor. Function application is replaced with lifted function application ( ), and raise is used to lift the combination functions where necessary. is : S I Prop is (PredVP np vp) = inp np ivp vp ivp : VP I (Ind Prop) ivp (UseV v) = iv v As we noted above, the interpretations of transitive verb and relation noun complementation can be written using function composition ( ). We now need to use lifted application of this operator. Note that application of a pure (nonlifted) function f to a lifted argument x is written as raise f x, and in the case of a two-argument function, raise f x y. As in Haskell, we use ( ) as a shorthand for (λf. λg. f g). ivp (ComplV2 v2 np) = raise ( ) inp np iv2 v2 Common noun interpretation is also the straightforward lifting of the interpretation in the previous section: icn : CN I (Ind Prop) icn (UseN n) = in n icn (ComplN2 n2 np) = raise ( ) inp np in2 n2 The interpretation of relational sentence modification looks a little hairy, but it is really just the lifted version of λx. (icn cn) x (irs rs) x. 9

10 icn (RelCN cn rs) = raise (λcn rs x. cn x rs x) icn cn irs rs Nothing needs to be done to the relative sentence interpretation, as it just returns the already lifted result of interpreting a verb phrase. irs : RS I (Ind Prop) irs (RelVP vp) = ivp vp Noun phrases and determiners introduce quantifiers. Since we want to move quantifiers to the top-level, these interpretations use the shift operator (ξ) to obtain the current continuation and wrap the quantifier around it. UsePN and DetCN are simply lifted. inp : NP I ((Ind Prop) Prop) inp Everyone = ξκ. x. κ (λv. v x) inp Someone = ξκ. x. κ (λv. v x) inp (UsePN pn) = raise (λx v. v x) ipn pn inp (DetCN det cn) = idet det icn cn idet : Det I ((Ind Prop) (Ind Prop) Prop) idet Every = ξκ. x. κ (λu v. u x v x) idet A = ξκ. x. κ (λu v. u x v x) Finally, we need to lift the lexicon to use the interpretation functor. This is very straightforward: we just use raise to lift each interpretation. Simple nouns: in : N I (Ind Prop) in Man = raise (λx. man(x)) in Woman = raise (λx. woman(x)) in Burger = raise (λx. burger(x)) Relational nouns: in2 : N2 I (Ind Ind Prop) in2 Owner = raise (λx y. owner(x, y)) Proper names: ipn : PN I Ind ipn John = raise John ipn Mary = raise Mary ipn Bill = raise Bill Intransitive verbs: iv : V I (Ind Prop) iv Walk = raise (λx. walk(x)) Transitive verbs: iv2 : V2 I (Ind Ind Prop) iv2 Eat = raise (λx y. eat(x, y)) iv2 Love = raise (λx y. love(x, y)) 10

11 3.4 Fragment 4: Scope islands A sentence such as a man who loves every woman eats a burger may at first appear to have 6 readings since the 3 quantifiers can be ordered in 3! = 6 ways. However, we may not consider all of those readings to be sensible. 1. x.man(x) y.woman(y) z.burger(z) love(x, y) eat(x, z) 2. * x.man(x) z.burger(z) y.woman(y) love(x, y) eat(x, z) 3. * y.woman(y) x.man(x) z.burger(z) love(x, y) eat(x, z) 4. * y.woman(y) z.burger(z) x.man(x) love(x, y) eat(x, z) 5. * z.burger(z) y.woman(y) x.man(x) love(x, y) eat(x, z) 6. z.burger(z) x.man(x) y.woman(y) love(x, y) eat(x, z) Readings 2 and 4 are not generated by our interpretation, in accordance with what Barker [2001] calls the syntactic constituent integrity scoping constraint. Semantic theory has it that relative clauses are scope islands, that is, no quantifier in the relative clause may take scope outside the relative clause. This disallows readings 3, 4, and 5. But 3 and 5 are returned by our interpretation in Fragment 3. It appears that we need a way to implement scope islands. Shan [2004] notes that delimited continuations are useful for modeling several natural language phenomena, and notes that Barker s [2002] treatment of scope islands implicitly uses reset ([ ]). Barker notes that this can only be done for categories whose interpretation type is t (Prop in our treatment). However, as we show below, we can do it for any type τ 1... τ n Prop (though we have only seen a need for Ind Prop so far). The [ ] function for creates a delimited continuation computation: [ ] : Cont o o Cont p o [x] = {λc. c y y ε x} We overload [ ] to also work on single argument functions. We use overloading here to simplify the notation. To stay within polymorphically typed λ-calculus, we could simply use two different names instead. [ ] : Cont o (a o) Cont p (a o) [x] = {λc. c y y ε x} This second version of [ ] uses a version of ε for single-argument functions: ε : Cont o (a o) {a o} ε x = {λe. y (λf. f e) y x} The rule for relative sentences now uses [ ] to make the relative sentence a scope island. irs : RS I (Ind Prop) irs (RelVP vp) = [ivp vp] With this addition, only readings 1 and 6 above are returned. 11

12 4 Haskell implementation The semantics is implemented as a Haskell [Peyton Jones, 2003] program. In fact, the rules that we have shown above are thinly camouflaged Haskell code. A Haskell program can be automatically extracted from the source code for this paper. 4.1 Efficiency The naive implementation above always uses both left-to-right and right-toleft evaluation whenever is used. This means that the number of formulas produced is roughly exponential in the number of nodes in the abstract syntax tree. However, since only a few of the interpretation rules make use of ξ, there are many duplicates. This blow-up can be avoided in most cases by changing the implementation of the continuation functor, as shown below. Now, the number of formulas produced is exponential in the number of uses of ξ. For some sentences, this may still be a significant number, but these are now syntactically different readings. Some of them may still be logically equivalent, for example if they only differ in the order of adjacent universal quantifiers A more efficient functor This is a representation of a non-deterministic continuation monad, with an added constructor (Pure) for computations which do not look at the continuation. data Cont o a = Pure {a} Cont {(a o) o} Getting a function that looks at the continuation from a Cont is easy: runcont : Cont o a {(a o) o} runcont (Pure xs) = {λc. c x x xs} runcont (Cont r) = r We define the applicative functor operators for our new functor. instance Applicative (Cont o) where The pure function of course uses the Pure constructor. pure a = Pure {a} We have a special case for Pure f Pure x that keeps the result pure, so that further uses of knows that it doesn t need both evaluation orders. If both of the sub-computations pure, the result is too. Pure fs Pure xs = Pure {f x f fs, x xs} If neither sub-computations is pure, the result is the cartesian product of the results of the sub-computations, with either left-to-right or right-to-left evaluation. Cont xs Cont ys = Cont $ concat {{λc. x (λf. y (λa. c (f a))), 12

13 λc. y (λa. x (λf. c (f a)))} x xs, y ys} If exacly one sub-computation is pure, then we only need one of the evaluation orders, since the result is the same regardless of the order. xs ys = Cont {λc. x (λf. y (λa. c (f a))) x runcont xs, y runcont ys} The ε operator is very similar to the naive version: ε : Cont o o {o} ε x = {y (λf. f ) y runcont x} ε : Cont o (a o) {a o} ε x = {λe. y (λf. f e) y runcont x} eval : Cont o (a b o) {a b o} eval x = {λe d. y (λf. f e d) y runcont x} The ξ operator is straightforward: ξ : ((a o) o) Cont o a ξ f = Cont {f } And so is the [ ] function for creating delimited continuation computations: [ ] : Cont o o Cont p o [x] = Pure (ε x) [ ] : Cont o (a o) Cont p (a o) [x] = Pure (ε x) reset : Cont o (a b o) Cont p (a b o) reset x = Pure (eval x) 5 Related Work Ranta [2004b] has a compositional semantics in dependent type theory. We share the idea of writing interpretation rules on the abstract syntax of a type theoretic grammar (GF in both cases). However, Ranta uses a dependently typed λ-calculus where we use a more pedestrian polymorphic λ-calculus. Also, Ranta handles quantifier scope ambiguity by making the grammar ambiguous while keeping the interpretation rules straightforward. Compared to Barker [2002], we have a more elegant way of writing interpretation rules, since the applicative functor formulation and control operators take care of the continuation plumbing. Also, we implement non-deterministic evaluation order in a single place, the definition of, rather than having to add multiple interpretations for each syntactic construct. 13

14 The idea of using a applicative functor for hiding the continuation plumbing is very similar to Shan s [2001] use of monads for the same purpose, but the restrictions of a monad does not allow the non-deterministic evaluation order that we need. In contrast to Blackburn and Bos [2005], we are using a an explicit abstract syntax, which makes it easier to change the surface language or the semantics independently. In particular, this makes it possible for us to construct a multilingual semantics. References Chris Barker. Integrity: A Syntactic Constraint on Quantificational Scoping. In Karine Megerdoomian and Leora A. Bar-El, editors, WC- CFL 20: Proceedings of the 20th West Coast Conference on Formal Linguistics, Somerville, MA, February Cascadilla Press. URL Chris Barker. Continuations and the nature of quantification. Natural Language Semantics, 10(3): , doi: /A: URL Patrick Blackburn and Johan Bos. Representation and Inference for Natural Language: A First Course in Computational Semantics. Center for the Study of Language and Information, April ISBN R. Cooper. Quantification and Syntactic Theory, volume 21 of Studies in Linguistics and Philosophy. Springer, 1 edition, April ISBN W. R. Keller. Nested Cooper storage: The proper treatment of quantification in ordinary noun phrases. In U. Reyle and C. Rohrer, editors, Natural Language Parsing and Linguistic Theories, pages Reidel, Dordrecht, Conor McBride and Ross Paterson. Applicative Programming with Effects. Journal of Functional Programming, 18(1):1 13, January doi: /S URL Eugenio Moggi. Computational lambda-calculus and monads. In Proceedings of the Fourth Annual Symposium on Logic in Computer Science (LICS 89), Pacific Grove, CA, USA, pages 14 23, Washington, DC, USA, June IEEE Computer Society Press. doi: /LICS URL Richard Montague. The Proper Treatment of Quantification in Ordinary English. pages , Simon Peyton Jones. The Haskell 98 Language. Journal of Functional Programming, 13(1):1 146, Aarne Ranta. Grammatical Framework: A Type-Theoretical Grammar Formalism. Journal of Functional Programming, 14(2): , March 2004a. ISSN doi: /S URL 14

15 Aarne Ranta. Computational Semantics in Type Theory. Mathematics and Social Sciences, 165:31 57, 2004b. URL Chung-Chieh Shan. Monads for natural language semantics. In Kristina Striegnitz, editor, Proceedings of the sixth ESSLLI Student Session, 13th European Summer School in Logic, Language and Information, Helsinki, Finland, pages , August URL Chung-Chieh Shan. Delimited continuations in natural language. In Hayo Thielecke, editor, Proceedings of the Fourth ACM SIGPLAN Continuations Workshop (CW 04), Venice, Italy, Birmingham B15 2TT, United Kingdom, January School of Computer Science, University of Birmingham. URL Philip Wadler. Comprehending Monads. Mathematical Structures in Computer Science, 2(4): , URL 15