Ling 571: Deep Processing for Natural Language Processing

Transcription

1 Ling 571: Deep Processing for Natural Language Processing Julie Medero February 4, 2013

2 Today s Plan Assignment Check-in Project 1 Wrap-up CKY Implementations HW2 FAQs: evalb Features & Unification

3 Project 1 What was surprising? Any lingering questions? What affected run time the most?

4 CKY Implementations If yours isn t working, talk to us now! Need to modify for HW2 What is good time/score? Time depends on implementation Score should be fixed. Compare to other teams and/or output from built-in NLTK parser

5 evalb In dropbox Configurable implementation of PARSEVAL algorithm Some tree manipulation before scoring

6 evalb Parameter File DEBUG 0 MAX_ERROR 10 CUTOFF_LEN 40 LABELED 1 DELETE_LABEL TOP DELETE_LABEL, DELETE_LABEL : DELETE_LABEL `` DELETE_LABEL '' DELETE_LABEL. EQ_LABEL ADVP PRT

7 evalb Example (TOP (INTJ_UH Thanks) (PUNC.)) Remove TOP and PUNC (INTJ_UH Thanks) INTJ_UH is a POS tag No structure to predict

8 evalb Output Bracketing Recall Bracketing Precision Tagging Accuracy Number of Skip Sentences What do state of the art parsers do?

9 Today s Plan Assignment Check-in Project 1 Wrap-up CKY Implementations HW2 FAQs evalb Features & Unification

10 Constraints in Grammar S NP VP They run, He runs *They runs *He run *He disappeared the flight Problems: Agreement (number), subcategorization

11 Enforcing Constraints Add categories, rules S-> NPsg3p VPsg3p S-> NPpl3p VPpl3p VP-> Vtrans NP VP -> Vintrans, VP->Vditrans NP NP Explosive! Loses key generalizations

12 Features Person: 1st, 2nd, 3rd I, we; you; he, she, they am, are, is Number: sg, (I am), pl (We are) Case: nom (I, he), acc (me, him) Gender: masc, fem, neut Animacy: +/1

13 Why features? Need compact, general constraints S -> NP VP iff NP and VP agree Decompose into elementary features Agreement, subcat: consistency requirements on those features (E.g. number, person, gender) Augment CF rules with feature constraints Develop mechanism to enforce consistency Elegant, compact, rich representation

14 Features Fundamentally, Attribute-Value pairs NUMBER PERSON PL 3rd Values can be feature structures too CAT NP AGR NUMBER PERSON PL 3rd

15 Feature Path List of features in structure to value <CAT> = NP <AGR,NUMBER> = PL <AGR,PERSON> = 3rd CAT NP AGR NUMBER PERSON PL 3rd

16 Feature Representation Attribute-Value Matrix (AVM) CAT AGR NUMBER PERSON NP SG 3rd Directed Acyclic Graph (DAG)

17 Reentrant Feature Structures Features share same structure as value Not just equal values Shared substructure Feature paths lead to same nodes Changes to one will affect both

18 Reentrant AVM CAT S HEAD AGREEMENT [1] NUMBER PL PERSON 3rd SUBJECT AGREEMENT [1]

19 Reentrant DAG

20 Unification Two key roles: Merge compatible feature structures Reject incompatible feature structures

21 Unification When can 2 structures unify? Feature structures are identical Result is same structure Both have values but differ in missing or underspecified values Incorporate constraints of both

22 Subsumption Relation between feature structures Less specific f.s. subsumes more specific f.s. F subsumes G iff For every feature x in F, F(x) subsumes G(x) For all paths p and q in F s.t. F(p)=F(q), G(p)=G(q)

23 Subsumption Examples A: B: C: NUMBER PERSON PERSON NUMBER SG 3rd 3rd SG A subsumes C; B subsumes C; B,A don t subsume Partial order on feature states.

24 Unification Examples Identical [Number SG] U [Number SG]=[Number SG] Underspecified [Number SG] U [Number []] = [Number SG] Different specification [Number SG] U [Person 3] = Mismatched PERSON NUMBER 3rd SG [Number SG] U [Number PL] = Fails!

25 More Unification Examples AGREEMENT [1] SUBJECT AGREEMENT [1] U SUBJECT AGREEMENT PERSON 3rd = NUMBER SG AGREEMENT [1] SUBJECT AGREEMENT [1] PERSON 3rd NUMBER SG

26 Features in CFGs: Agreement Goal: Support agreement of NP/VP, Det Nominal Approach: Augment CFG rules with features Employ head features Each phrase: VP, NP has head Head: child that provides features to phrase Associates grammatical role with word VP V; NP Nom, etc

27 Agreement with Heads and Features VP Verb NP <VP HEAD> = <Verb HEAD> NP Det Nominal <NP HEAD> = <Nominal HEAD> <Det HEAD AGREEMENT> = <Nominal HEAD AGREEMENT> Nominal Noun <Nominal HEAD> = <Noun HEAD> Noun flights <Noun HEAD AGREEMENT NUMBER> = PL Verb serves <Verb HEAD AGREEMENT NUMBER> = SG <Verb HEAD AGREEMENT PERSON> = 3

28 Feature Applications Subcategorization: Verb-Argument constraints Number, type, characteristics of args (e.g. animate) Also adjectives, nouns Long distance dependencies E.g. filler-gap relations in wh-questions

29 Implementing Unification Data Structure: Extension of the DAG representation F.S. has a content field and a pointer field If pointer null, content field has the f.s. If pointer is non-null, it points to actual f.s.

30 DAG for Unification

31 Unification Algorithm Operates on pairs of feature structures Order independent, destructive If fs1 is null, point to fs2 If fs2 is null, point to fs1 If both are identical, point fs1 to fs2, return fs2 Subsequent updates will update both If non-identical atomic values, fail!

32 Unification Algorithm If non-identical, complex structures Recursively traverse all features of fs2 If feature in fs2 is missing in fs1 Add to fs1 with value null If all unify, point fs2 to fs1 and return fs1

33

34 Example AGREEMENT [1] NUMBER SG SUBJECT AGREEMENT [1] U SUBJECT AGREEMENT PERSON 3 [ AGREEMENT [1]] U [AGREEMENT [PERSON 3]] [NUMBER SG] U [PERSON 3] [NUMBER SG] U [PERSON 3] [PERSON NULL]

35

36

37

38 Parsing with Features One strategy: Parse as usual Test completed parses for unification constraints Pro: Simple, requires little modification Cons: Wasted effort Builds many partial parses that can t unify Instead: Integrate unification in parse construction

39 Parsing, Unification, & Earley Augment Earley parser for unification Modify representations: Augment CFG rules with constraints Use constraints to create f.s. as DAG Add DAG to state representation E.g., S -> NP VP, [0,0],[],DAG

40 Integrating Unification Main change: Completer Before: Advanced in rules where next constituent matched a just-completed constituent Now, unifies DAG from completed constituent with the part of the feature structure in rules advanced If fails, no new entry in chart Second change: Only add state if NOT subsumed by states in chart

41

42

43 Unification Parsing Abstracts over categories S NP VP => X0 X1 X2; <X0 cat> = S; <X1 cat>=np; <X2 cat>=vp Conjunction: X0 X1 and X2; <X1 cat> =<X2 cat>; <X0 cat>=<x1 cat>

44 Unification Parsing Issue: Completer depends on categories Solution: Completer looks for DAGs which unify with the just-completed state s DAG

45 Types & Inheritance Types and inheritance Issue: generalization across feature structures E.g. many variants of agreement More or less specific: 3rd vs sg vs 3rdsg

46 Types & Inheritance Approach: Type hierarchy Simple atomic types match literally Multiple inheritance hierarchy Unification of subtypes is most general type that is more specific than two input types Complex types encode legal features, etc

47 Conclusion Features allow encoding of constraints Enables compact representation of rules Supports natural generalizations Unification ensures compatibility of features Integrates easily with existing parsing mech. Many unification-based grammatical theories

48 Next Time HW2 Check-in Designing & Implementing feature-based grammars Semantic Features Leads us into semantics!