On Learning Discontinuous Dependencies from Positive Data

Transcription

1 Formal Grammar 2004 p. 1 On Learning Discontinuous Dependencies from Positive Data Denis Béchet (1), Alexander Dikovsky (2), Annie Foret (3) and Erwan Moreau (2) (1) LIPN, University Paris 13 (2) LINA, University of Nantes (3) IRISA, University of Rennes

2 Formal Grammar 2004 p. 2 Gold s Model : identification in the limit Algorithm: Input : a finite set of sentences (positive examples) Output : a grammar in the class that generates the sentences ; the algorithm is required to converge Formally: G : class of grammars Σ : alphabet φ : function from finite subsets of Σ to G such that G G, e i i N with L(G) = e i i N : G G with L(G ) = L(G) n 0 N: n > n 0 φ({e 1,..., e n }) = G G where L(G) denotes the language associated to G

3 Conclusion Formal Grammar 2004 p. 3 PLAN Introduction Background [Dikovsky 2004] Categories, dependencies, trees and nets Categorial dependency grammars (CDGs) Languages of (untyped) nets/depencency trees/strings Learning k-valued/rigid CDGs Learning from untyped nets: an algorithm Learning from untyped nets: finite elasticity Learning from strings: finite elasticity (CDGs without optional or iterative types) Non-Learnability for CDGs with optional or iterative types

4 Formal Grammar 2004 p. 4 Categories, nodes, dependencies and nets Categories and nodes:.. D = det D. S R C S.# T. R I S = subj person Smith must R = attr rel. C = n copul D # T W T. I T = pre T O obj the to refer W = prep wh I = inf obj An untyped net: l l l. l d a l d l l a l person to whom you must refer is D D S R # T W W S S # T R I T I S S C C the local dependency distant dependency anchored dependency main conclusion Smith

5 Formal Grammar 2004 p. 5 CDG: definitions (1) Σ = alphabet for words in a natural language C = set of elementary categories : det, subj, Start(C) and End(C) = starting and ending atoms : C : start or end of 1 local dependency C : start of 0, 1 or more local dependencies (iterative) C + : start of 1 or more local dependencies (repetitive) C? : start 0 or 1 local dependency (optional) C : start (on the left) of 1 distant dependency C : end (from the right) of 1 distant dependency # C : start of 1 anchored dependency or end of 1 anchored dependency and of 1 distant dependency C, C and # C for the reverse direction

6 CDG: definitions (2) Cat(C) = categories = List(Start(C)) End(C) List(Start(C)) written L 1 \ \L i \C/R j / /R 1 = provability relation based on simplification rules (local, distant and anchored dependencies): Γ 1 C[C\α]Γ 2 Γ 1 αγ 2 (L) Γ 1 C[C \α]γ 2 Γ 1 [C \α]γ 2 (I) Γ 1 C[C + \α]γ 2 Γ 1 [C \α]γ 2 (R) Γ 1 C[C? \α]γ 2 Γ 1 αγ 2 (O) Γ 1 [C? \α]γ 2 Γ 1 αγ 2 (Ω) Γ 1 #(α)[#(α)\β]γ 2 Γ 1 αβγ 2 (A) Γ 1 ( C)Γ (1) 2 [( C)\α]Γ 3 Γ 1 Γ 2 αγ 3 (D) (1) Γ 2 without C, C and # C Formal Grammar 2004 p. 6

7 Formal Grammar 2004 p. 7 CDG: definitions (3) A CDG is a finite relation G between Σ and Cat(C) G generates a string c 1... c n Σ + iff A 1,..., A n Cat(C) such that: G : c i A i (1 i n) and A 1 A n S The language of G, L(G) = the set of strings generated by G G k-valued iff at most k types per symbol (rigid if k = 1)

8 Formal Grammar 2004 p. 8 Example of a CDG (1) G 0 : a # C \# B # B\# B b B\D c D\A d 1 # B d 2 # B \ C \S/D d 3 D L(G 0 ) = {d 1 a n d 2 b n d 3 c n n > 0} : a non-cf language

9 Formal Grammar 2004 p. 9 Example of a CDG (2) A proof of d 1 a 3 d 2 b 3 d 3 c 3 L(G 0 ), in which α = #( B), α 1 = ( B), β = #( C) and β 1 = ( C): d a a a d 2 b b b c c c d 3 β [β\α] [α\α] [α\α] [α\β 1 \S/D][α 1 \D/A] [α 1 \D/A] [α 1 \D/A] D [D\A] [D\A][D\A] ( B) A ( B) D ( B) A [β 1 \S/D] D A D S

10 Formal Grammar 2004 p. 10 Net, untyped net, tree or string Net: l l l. l d l a l person to whom you must refer is l D D S R # T W W S S # T R I T I S S C C the Untyped net: the l l l. l d l a l person to whom you must refer is l C Smith Smith Tree: the person to whom you must refer is C Smith String: the person to whom you must refer is Smith

11 Conclusion Formal Grammar 2004 p. 11 PLAN Introduction Background [Dikovsky 2004] Categories, dependencies, trees and nets Categorial dependency grammars (CDGs) Languages of (untyped) nets/depencency trees/strings Learning k-valued/rigid CDGs Learning from untyped nets: an algorithm Learning from untyped nets: finite elasticity Learning from strings: finite elasticity (CDGs without optional or iterative types) Non-Learnability for CDGs with optional or iterative types

12 Formal Grammar 2004 p. 12 Learning from untyped nets (Rigid grammars) Starting with the following positive examples the l l l l..... fish swims John swims fast

13 Formal Grammar 2004 p. 12 Learning from untyped nets (Rigid grammars) Starting with the following positive examples we assign distinct variables to slots l l l l X 1 X 2. X 3 X 4 X. 5. X 6 X 7 X 4 X. 5. X 6 X 8 the fish swims John swims fast

14 Formal Grammar 2004 p. 12 Learning from untyped nets (Rigid grammars) Starting with the following positive examples we assign distinct variables to slots l l l l X 1 X 2. X 3 X 4 X. 5. X 6 X 7 X 4 X. 5. X 6 X 8 the fish swims John swims fast and obtain the resulting grammar after unification (and link counting) General form Unification Rigid grammar the X 1 X 1 fish X 2 \X 3 X 2 = X 1 X 1 \X 3 swims X 4 \X 5 /X 6 X 4 = X 3 ; X 5 = S X 3 \S/X 6? John X 7 X 7 = X 4 (= X 3 ) X 3 fast X 8 X 8 = X 6 X 6

15 Formal Grammar 2004 p. 13 Finite elasticity and learnability Infinite elasticity: (e i ) i N an infinite sequence of sentences (L i ) i N an infinite { sequence of languages i N : e i L i such that {e 0,..., e i 1 } L i Properties [Wright 1989] unlearnability infinite elasticity finite elasticity learnability

16 Formal Grammar 2004 p. 14 Rigid languages of untyped nets Theorem: 1. Finite elasticity (in fact bounded finite thickness) 2. The class is learnable 3. The given algorithm learns the class Proof schemes: (1) There is a finite number of reduced languages in the class that correspond to a given list of untyped nets (2) Consequence of finite elasticity (3) The returned grammar corresponds to the minimal language compatible with the input untyped nets

17 Formal Grammar 2004 p. 15 Proof of finite elasticity (1) [Shinohara 1990]: finite thickness finite elasticity Definition. G 1 G 2 iff x Σ, C Cat(C), G 1 : x C G 2 : x C Definition and lemma. The mapping L UNet from link grammars to untyped net languages is monotonic: if G 1 G 2 then L UNet (G 1 ) L UNet (G 2 ) Definition. A grammar G is reduced with respect to a set X of untyped nets iff X L UNet (G) and for each grammar G G, G G X L UNet (G )

18 Formal Grammar 2004 p. 16 Proof of finite elasticity (2) Lemma. For each finite set X L UNet (G), there is a finite set of rigid untyped net languages that correspond to the grammars that are reduced from X Definition. Monotonicity and the previous property define a system that has bounded finite thickness Theorem [Shinohara 1990]. A formal system that has bounded finite thickness has finite elasticity Corollary. Rigid untyped net languages have finite elasticity

19 Formal Grammar 2004 p. 17 k-valued languages of strings (1) Theorem [Kanasawa 1998]: The images, through a finite-valued relation of a class of languages that has finite elasticity, forms a class that has finite elasicity Consequences: 1. For each k, the class of k-valued CDG (without optional and iterative category) languages of strings has finite elasticity 2. For each k, this class (of CDGs) is learnable from strings

20 Formal Grammar 2004 p. 18 k-valued languages of strings (2) Lemma: 1. For each k, the classes of k-valued CDG without optional and/or iterative category languages of strings accept a limit point 2. For each k, these classes (of CDGs) are not learnable from strings

21 Formal Grammar 2004 p. 19 Conclusion Rigid CDGs are learnable from untyped nets There exists a simple learning algorithm for this class Theoretical learnability result (finite elasticity) for k-valued CDGs without optional and iterative category learned from strings Rigid (or k-valued) CDGs with optional and/or iterative category are not learnable from strings Class Learnable from strings Finite elasticity on strings Finite elasticity on structures Finite-valued relation A no no yes no A? no no yes no A + yes yes yes yes