Ling 571: Deep Processing for Natural Language Processing

Transcription

1 Ling 571: Deep Processing for Natural Language Processing Julie Medero January 14, 2013

2 Today s Plan Motivation for Parsing Parsing as Search Search algorithms Search Strategies Issues

3 Two Goal of Parsing Analyze input strings to assign proper structures For input A, grammar G: Assign zero or more parse tree(s) T: Cover all and only the elements of A Root of T is S (the start symbol of G) Do not necessarily pick one (or correct) analysis

4 Two Goal of Parsing Recognition Subtask of parsing For input A, grammar G Is A in the language defined by G?

5 Questions for Parsing Is this sentence in the language? FSAs accept the regular languages defined by automaton Parsers accept language defined by CFG What is the syntactic structure of this sentence? Syntactic parse provides framework for semantic analysis What is the subject? Useful for e.g. question answering

6 Parsing as Search Search through possible parse trees Want one (or more) that derive input Formally, search problems are defined by: Start state S, Goal state G, Successor Function: Transitions between states, Path cost function

7 One Model of Parsing as Start State: Start Symbol from grammar Goal test: Search Does parse tree cover all and only input? Successor function: Expand a non-terminal using production in grammar where nonterminal is LHS of grammar Path cost: We ll ignore here

8 One Model of Parsing as Search Node: Partial solution to search problem: Partial parse Search start node: Initial State Input string Start symbol of CFG Goal node: Full parse tree: covering all and only input, rooted at S

9 Search Algorithms Depth first Keep expanding non-terminal until reach words If no more expansions, back up Breadth first Consider all parses with one non-terminal expanded Then all with two expanded, etc. Other alternatives if have associated path costs

10 Parse Search Strategies Two constraints: Must start with the start symbol Must cover exactly the input string Correspond to main parsing search strategies Top-down search (Goal-directed search) Bottom-up search (Data-driven search)

11 Parse Search Strategies Breadth-First Depth-First Top-Down Bottom-Up

12 Today s Plan Motivation for Parsing Parsing as Search Search algorithms Search Strategies Issues

13 Breadth-First Search Uninformed search strategy Visiting all neighboring (sister) nodes first Then go deeper into the tree

14 Breadth-First Example A B C D E F G H I J K L M

15 Depth-First Search Uninformed search strategy Going down one branch all the way When needed, backtrack.

16 Depth-First Example A B C D E F G H I J K L M

17 Today s Plan Motivation for Parsing Parsing as Search Search algorithms Search Strategies Issues

18 A Toy Grammar

19 Top-Down Search Begin with productions with S on LHS E.g., S NP VP Successively expand non-terminals E.g., NP Det Nominal; VP V NP Terminate when all leaves are terminals Book that flight

20

21 Pros and Cons of Top- Down Search Pros: Doesn t explore trees not rooted at S Doesn t explore invalid subtrees Cons: Produces trees that may not match input May not terminate with recursive rules May rederive subtrees during search

22 Bottom-Up Search Find all trees that span the input Start with input string: Book that flight. Use all productions with current subtree(s) on RHS E.g., N Book; V Book Stop when spanned by S (or no more rules apply)

23 Depth-First Breadth-First

24 Pros and Cons of Bottom-Up Search Pros: Only explore trees that match input Fewer problems with recursive rules Useful for incremental/ fragment parsing Cons: Explore subtrees that will not fit full sentences

25 Today s Plan Motivation for Parsing Parsing as Search Search algorithms Search Strategies Issues

26 Parsing Challenges Ambiguity Recursion Repeated substructure

27 Parsing Ambiguity Lexical ambiguity Book/N; Book/V Structural ambiguity: Attachment ambiguity Constituent can attach in multiple places I shot an elephant in my pyjamas. Coordination ambiguity Different constituents can be conjoined Old men and women

28 Attachment Ambiguity

29 Disambiguation Local ambiguity: Ambiguity in subtree, resolved globally Global ambiguity: Multiple complete alternative parses Need strategy to select correct one Alternatively, keep all

30 Resolving Global Ambiguity Exploit other information Statistical Some prepositional structs more likely to attach high/low Some phrases more likely, e.g., (old (men and women)) Semantic Pragmatic (e.g., elephants and pyjamas)

31 Garden Path Sentences Misleading Partial Analyses The old man the boat The player kicked the ball kicked the ball The cotton clothing is made of grows in Mississippi

32 Recursion Direct Recursion (e.g., S S CONJ S) water under the bridge, Bill ran and Jane jogged Indirect Recursion... on a thimble in a box on a stool beside a table near a sofa... NP DT Nom Nom Nom PP PP Prep NP Can yield infinite searches e.g., Top-down search with S S conj S

33 Repeated Work Avoid repeatedly parsing substructures Good subtrees in globally bad parses Overall, bad parses will fail Reconstruction subtrees on other branches Can t avoid with static backtracking Store shared substructure for efficiency Typically with dynamic programming

34

35 Dynamic Programming Want to avoid repeated work Global parse made of parse subtrees Record parses of subtrees Dynamic programming Tabulate solutions to subproblems Avoid repeated work

36 Parsing w/dynamic Programming Makes parsing algorithms (relatively) efficient Polynomial time in input length Typically cubic (n 3 ) or less Several different implementations Cocke-Kasami-Younger (CKY) algorithm Earley algorithm Chart parsing

37 Chomsky Normal Form (CNF) CKY parsing requires grammars in CNF All productions of the form: A B C, or A a Most of our grammars are not of this form E.g., S -> Wh-NP Aux NP VP Need a general conversion procedue

38 CNF Conversion Hybrid rules: INF-VP to VP Unit productions: A B (e.g. Nom N) Long productions: A B C D

39 Hybrid Rule Conversion Replace all terminals with dummy nonterminals Problem Rule: INF-VP to VP New Rules: INF-VP TO VP TO to

40 Unit Productions Conversion Rewrite RHS with RHS of all derivable nonunit productions Old Rules: NP Nom Nom Det N New Rule: NP Det N

41 Long Productions Conversion Introduce new non-terminals and spread over rules Old Rule: S Aux NP VP New Rules: S X1 VP X1 Aux NP

42 CNF Conversion Summary For all non-conforming rules, Convert terminals to dummy nonterminals Convert unit productions Binarize all resulting rules

43 Result of CNF Conversion

44 Next Time CKY Parsing Algorithm Homework 1 Wrap-Up Project 1 Introduction