Semantics as a Foreign Language. Gabriel Stanovsky and Ido Dagan EMNLP 2018
|
|
|
- Lauren Hunt
- 5 years ago
- Views:
Transcription
1 Semantics as a Foreign Language Gabriel Stanovsky and Ido Dagan EMNLP 2018
2 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015)
3 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015) a. DM (derived from MRS) (Copestake et al., 1999, Flickinger, 2000)
4 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015) a. DM (derived from MRS) b. Prague Semantic Dependencies (PSD) (Hajic et al., 2012)
5 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015) a. DM (derived from MRS) b. Prague Semantic Dependencies (PSD) c. Predicate Argument Structures (PAS) (Miyao et al., 2014)
6 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015) a. DM (derived from MRS) b. Prague Semantic Dependencies (PSD) c. Predicate Argument Structures (PAS) Aim to capture semantic predicate-argument relations
7 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015) a. DM (derived from MRS) b. Prague Semantic Dependencies (PSD) c. Predicate Argument Structures (PAS) Aim to capture semantic predicate-argument relations Represented in a graph structure
8 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015) a. DM (derived from MRS) b. Prague Semantic Dependencies (PSD) c. Predicate Argument Structures (PAS) Aim to capture semantic predicate-argument relations Represented in a graph structure a. Nodes: single words from the sentence
9 Semantic Dependency Parsing (SDP) A collection of three semantic formalisms (Oepen et al., 2014;2015) a. DM (derived from MRS) b. Prague Semantic Dependencies (PSD) c. Predicate Argument Structures (PAS) Aim to capture semantic predicate-argument relations Represented in a graph structure a. Nodes: single words from the sentence b. Labeled edges: semantic relations, according to the paradigm
10 Outline
11 Outline SDP as Machine Translation Different formalisms as foreign languages Motivation: downstream tasks, inter-task analysis, extendable framework
12 Outline SDP as Machine Translation Different formalisms as foreign languages Motivation: downstream tasks, inter-task analysis, extendable framework Previous work explored the relation between MT and semantics (Wong and Mooney, 2007), (Vinyals et al., 2015), (Flanigan et al., 2016)
13 Outline SDP as Machine Translation Model Different formalisms as foreign languages Motivation: downstream tasks, inter-task analysis, extendable framework Previous work explored the relation between MT and semantics (Wong and Mooney, 2007), (Vinyals et al., 2015), (Flanigan et al., 2016) Seq2Seq Directed graph linearization
14 Outline SDP as Machine Translation Model Results Different formalisms as foreign languages Motivation: downstream tasks, inter-task analysis, extendable framework Previous work explored the relation between MT and semantics (Wong and Mooney, 2007), (Vinyals et al., 2015), (Flanigan et al., 2016) Seq2Seq Directed graph linearization Raw text to SDP (near state-of-the-art) Novel inter-task analysis
15 Outline SDP as Machine Translation Different formalisms as foreign languages Motivation: downstream tasks, inter-task analysis, extendable framework Previous work explored the relation between MT and semantics (Wong and Mooney, 2007), (Vinyals et al., 2015), (Flanigan et al., 2016) Model Seq2Seq Linearization Results Raw text -> SDP (near state-of-the-art) Novel inter-task analysis
16 Semantic Dependencies as MT Source Raw sentence Target
17 Semantic Dependencies as MT Source Syntax Raw sentence Grammar as a foreign language Target
18 Semantic Dependencies as MT Source Raw sentence Grammar as a foreign language This work Syntax SDP Target
19 Semantic Dependencies as MT Standard MTL: 3 tasks Raw sentence PSD DM PAS
20 Semantic Dependencies as MT Standard MTL: 3 tasks Raw sentence PSD DM PAS Inter-task translation (9 tasks)
21 Outline SDP as Machine Translation Motivation: downstream tasks Different formalisms as foreign languages Model Seq2Seq Linearization Results Raw text -> SDP (near state-of-the-art) Novel inter-task analysis
22 Our Model Ⅰ: Raw -> SDP x Seq2Seq translation model: Bi-LSTM encoder-decoder with attention Linear DM <from: RAW> <to: DM> the cat sat on the mat
23 Our Model Ⅰ: Raw -> SDP x Seq2Seq translation model: Bi-LSTM encoder-decoder with attention Linear DM <from: RAW> <to: DM> the cat sat on the mat Special from and to symbols
24 Our Model Ⅱ: SDP y -> SDP x Seq2Seq translation model: Bi-LSTM encoder-decoder with attention Linear DM <from: PSD> <to: DM> Special from and to symbols Linear PSD
25 Our Model Seq2seq prediction requires a 1:1 linearization function Linear SDP x <from: SDPy > <to: SDP x > Linear SDP y
26 Linearization: Background Previous work used bracketed tree linearization (ROOT (NP (NNP Bob )NNP )NP (VP messaged (NP Alice )NP )VP )ROOT (Vinyals et al., 2015; Konstas et al., 2017; Buys and Blunsom, 2017)
27 Linearization: Background Previous work used bracketed tree linearization (ROOT (NP (NNP Bob )NNP )NP (VP messaged (NP Alice )NP )VP )ROOT (Vinyals et al., 2015; Konstas et al., 2017; Buys and Blunsom, 2017)
28 Linearization: Background Previous work used bracketed tree linearization (ROOT (NP (NNP Bob )NNP )NP (VP messaged (NP Alice )NP )VP )ROOT (Vinyals et al., 2015; Konstas et al., 2017; Buys and Blunsom, 2017)
29 Linearization: Background Previous work used bracketed tree linearization (ROOT (NP (NNP Bob )NNP )NP (VP messaged (NP Alice )NP )VP )ROOT (Vinyals et al., 2015; Konstas et al., 2017; Buys and Blunsom, 2017)
30 Linearization: Background Previous work used bracketed tree linearization (ROOT (NP (NNP John )NNP )NP (VP messaged (NP Alice )NP )VP )ROOT (Vinyals et al., 2015; Konstas et al., 2017; Buys and Blunsom, 2017)
31 Linearization: Background Previous work used bracketed tree linearization (ROOT (NP (NNP John )NNP )NP (VP messaged (NP Alice )NP )VP )ROOT (Vinyals et al., 2015; Konstas et al., 2017; Buys and Blunsom, 2017)
32 Linearization: Background Previous work used bracketed tree linearization (ROOT (NP (NNP John )NNP )NP (VP messaged (NP Alice )NP )VP )ROOT (Vinyals et al., 2015; Konstas et al., 2017; Buys and Blunsom, 2017) Depth-first representation doesn t directly apply to SDP graphs Non-connected components Re-entrencies
33 SDP Linearization (Connectivity) Problem: No single root from which to start linearization
34 SDP Linearization (Connectivity) Problem: No single root from which to start linearization
35 SDP Linearization (Connectivity) Problem: No single root from which to start linearization Solution: Artificial SHIFT edges between non-connected adjacent words All nodes are now reachable from the first word
36 SDP Linearization (Re-entrancies) Re-entrancies require a 1:1 node representation
37 SDP Linearization (Re-entrencies) Re-entrancies require a 1:1 node representation
38 SDP Linearization (Re-entrencies) Re-entrancies require a 1:1 node representation (relative index / surface form)
39 SDP Linearization (Re-entrencies) Re-entrancies require a 1:1 node representation (relative index / surface form) 0/couch-potato
40 SDP Linearization (Re-entrencies) Re-entrancies require a 1:1 node representation (relative index / surface form) 0/couch-potato compound +1/jocks
41 SDP Linearization (Re-entrencies) Re-entrancies require a 1:1 node representation (relative index / surface form) 0/couch-potato compound +1/jocks shift +1/watching
42 SDP Linearization (Re-entrencies) Re-entrancies require a 1:1 node representation (relative index / surface form) 0/couch-potato compound +1/jocks shift +1/watching ARG1-1/jocks
43 Outline SDP as Machine Translation Motivation: downstream tasks Different formalisms as foreign languages Model Linearization Dual Encoder-Single decoder Seq2Seq Results Raw text -> SDP (near state-of-the-art) Novel inter-task analysis
44 Experimental Setup Train samples per task: 35,657 sentences (Oepen et al., 2015) 9 translation tasks
45 Experimental Setup Train samples per task: 35,657 sentences (Oepen et al., 2015) 9 translation tasks Total training samples: 320,913 source-target pairs
46 Experimental Setup Train samples per task: 35,657 sentences (Oepen et al., 2015) 9 translation tasks Total training samples: 320,913 source-target pairs Trained in batches between the 9 different tasks
47 Evaluations:RAW SDP(x) Labeled F1 score
48 Evaluations:RAW SDP(x) Labeled F1 score
49 Evaluations:RAW SDP(x) Labeled F1 score
50 Evaluations:RAW SDP(x) Labeled F1 score
51 Evaluations:SDP(a) SDP(b) Labeled F1 score
52 Evaluations:SDP(a) SDP(b) Labeled F1 score Translating between representations is easier than parsing from raw text
53 Evaluations:SDP(a) SDP(b) Labeled F1 score Translating between representations is easier than parsing from raw text Easy to convert between PAS and DM
54 Evaluations:SDP(a) SDP(b) Labeled F1 score Translating between representations is easier than parsing from raw text Easy to convert between PAS and DM PSD is a good input, but relatively hard output
55 Conclusions
56 Conclusions Effective graph linearization for SDP Near state-of-the-art results
57 Conclusions Effective graph linearization for SDP Near state-of-the-art results Inter-task analysis Enabled by the generic seq2seq framework
58 Conclusions Effective graph linearization for SDP Near state-of-the-art results Inter-task analysis Enabled by the generic seq2seq framework Future work Apply linearizations in downstream tasks (NMT) Add more representations (AMR, UD)
59 Conclusions Effective graph linearization for SDP Near state-of-the-art results Inter-task analysis Enabled by the generic seq2seq framework Future work Apply linearizations in downstream tasks (NMT) Add more representations (AMR, UD) Thanks for listening!
60
61
62
63 BACKUP SLIDES
64 Evaluations: Node ordering Smaller-first ordering consistently does better across all representations
65 Semantic Formalisms Many formalisms try to represent the meaning of a sentence MRS, AMR, PSD, SDP, etc
66 Semantic Dependencies as MT Syntactic parsing as MT ( Grammar as a foreign language, [Vinyals et. al, 2014]) Jane had a cat We aim to do the same for SDP The different formalisms as foreign languages
67 Semantic Dependencies as MT Raw text PSD DM PAS
68 Our Model Seq2Seq translation model: Bi-LSTM encoder-decoder with attention Two shared encoders From raw to SDP graphs Between SDP graphs One global decoder for all samples Add <from:x> <to:y> tags to input as preprocessing Where X, Y in {RAW, PSD, PAS, DM} Different than Google s NMT, which didn t have <from:x> tags No code-switching is allowed
69 Motivation Linearization is an easy way to plug-in predicted structures in NNs MT Target side syntax (Aharoni and Goldberg, 2017; Wang et al., 2018) Allows Inter-task analysis
70 Motivation Linearization is an easy way to plug-in predicted structures in NNs MT Target side syntax (Aharoni and Goldberg, 2017; Wang et al., 2018)
71 Motivation Linearization is an easy way to plug-in predicted structures in NNs MT Target side syntax (Aharoni and Goldberg, 2017; Wang et al., 2018) Allows Inter-task analysis Easily extendable framework
72 SDP Linearization (node ordering)
73 SDP Linearization (node ordering)
74 SDP Linearization (node ordering)
75 SDP Linearization (node ordering) Neighbor orderings: a. Random - (play, for, jocks, now) b. Closest-first c. Sentence-order d. Smaller-first
76 SDP Linearization Neighbor orderings: a. Random b. Closest-first - (now, for, play, jocks) c. Sentence-order d. Smaller-first
77 SDP Linearization Neighbor orderings: a. Random b. Closest-first c. Sentence-order - (jocks, now, for, play) d. Smaller-first
78 SDP Linearization Neighbor orderings: a. Random b. Closest-first c. Sentence-order d. Smaller-first - (now, play, for, jocks)
79 Evaluations: Node ordering Smaller-first ordering consistently does better across all representations Labeled F1 score
Backpropagating through Structured Argmax using a SPIGOT
Backpropagating through Structured Argmax using a SPIGOT Hao Peng, Sam Thomson, Noah A. Smith @ACL July 17, 2018 Overview arg max Parser Downstream task Loss L Overview arg max Parser Downstream task Head
Forest-based Neural Machine Translation. Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Tiejun Zhao, EiichiroSumita
Forest-based Neural Machine Translation Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Tiejun Zhao, EiichiroSumita Motivation Key point: Syntactic Information To use or not to use? string-to-string model
Transition-based Parsing with Neural Nets
CS11-747 Neural Networks for NLP Transition-based Parsing with Neural Nets Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two Types of Linguistic Structure Dependency: focus on relations between
Context Encoding LSTM CS224N Course Project
Context Encoding LSTM CS224N Course Project Abhinav Rastogi arastogi@stanford.edu Supervised by - Samuel R. Bowman December 7, 2015 Abstract This project uses ideas from greedy transition based parsing
Unsupervised Semantic Parsing
Unsupervised Semantic Parsing Hoifung Poon Dept. Computer Science & Eng. University of Washington (Joint work with Pedro Domingos) 1 Outline Motivation Unsupervised semantic parsing Learning and inference
Assignment 4 CSE 517: Natural Language Processing
Assignment 4 CSE 517: Natural Language Processing University of Washington Winter 2016 Due: March 2, 2016, 1:30 pm 1 HMMs and PCFGs Here s the definition of a PCFG given in class on 2/17: A finite set
Ling/CSE 472: Introduction to Computational Linguistics. 5/21/12 Unification, parsing with unification Meaning representation
Ling/CSE 472: Introduction to Computational Linguistics 5/21/12 Unification, parsing with unification Meaning representation Overview Unification Unification algorithm Parsing with unification Representing
Large-Scale Syntactic Processing: Parsing the Web. JHU 2009 Summer Research Workshop
Large-Scale Syntactic Processing: JHU 2009 Summer Research Workshop Intro CCG parser Tasks 2 The Team Stephen Clark (Cambridge, UK) Ann Copestake (Cambridge, UK) James Curran (Sydney, Australia) Byung-Gyu
Online Graph Planarisation for Synchronous Parsing of Semantic and Syntactic Dependencies
Online Graph Planarisation for Synchronous Parsing of Semantic and Syntactic Dependencies Ivan Titov University of Illinois at Urbana-Champaign James Henderson, Paola Merlo, Gabriele Musillo University
Ling 571: Deep Processing for Natural Language Processing
Ling 571: Deep Processing for Natural Language Processing Julie Medero January 14, 2013 Today s Plan Motivation for Parsing Parsing as Search Search algorithms Search Strategies Issues Two Goal of Parsing
Topics in Parsing: Context and Markovization; Dependency Parsing. COMP-599 Oct 17, 2016
Topics in Parsing: Context and Markovization; Dependency Parsing COMP-599 Oct 17, 2016 Outline Review Incorporating context Markovization Learning the context Dependency parsing Eisner s algorithm 2 Review
Riga: from FrameNet to Semantic Frames with C6.0 Rules
Riga: from FrameNet to Semantic Frames with C6.0 Rules Guntis Barzdins, Peteris Paikens, Didzis Gosko University of Latvia, IMCS Rainis Blvd. 29, Riga, LV-1459, Latvia {guntis.barzdins,peteris.paikens,didzis.gosko}@lumii.lv
Learning Latent Linguistic Structure to Optimize End Tasks. David A. Smith with Jason Naradowsky and Xiaoye Tiger Wu
Learning Latent Linguistic Structure to Optimize End Tasks David A. Smith with Jason Naradowsky and Xiaoye Tiger Wu 12 October 2012 Learning Latent Linguistic Structure to Optimize End Tasks David A. Smith
Solving Natural Language Math Problems
Solving Natural Language Math Problems Takuya Matsuzaki (Nagoya University) Noriko H. Arai (National Institute of Informatics) Solving NL Math why? It is the first and the last goal of symbolic approach
Let s get parsing! Each component processes the Doc object, then passes it on. doc.is_parsed attribute checks whether a Doc object has been parsed
Let s get parsing! SpaCy default model includes tagger, parser and entity recognizer nlp = spacy.load('en ) tells spacy to use "en" with ["tagger", "parser", "ner"] Each component processes the Doc object,
Semantic Dependency Graph Parsing Using Tree Approximations
Semantic Dependency Graph Parsing Using Tree Approximations Željko Agić Alexander Koller Stephan Oepen Center for Language Technology, University of Copenhagen Department of Linguistics, University of
CS224n: Natural Language Processing with Deep Learning 1 Lecture Notes: Part IV Dependency Parsing 2 Winter 2019
CS224n: Natural Language Processing with Deep Learning 1 Lecture Notes: Part IV Dependency Parsing 2 Winter 2019 1 Course Instructors: Christopher Manning, Richard Socher 2 Authors: Lisa Wang, Juhi Naik,
Semantic Dependency Graph Parsing Using Tree Approximations
Semantic Dependency Graph Parsing Using Tree Approximations Željko Agić Alexander Koller Stephan Oepen Center for Language Technology, University of Copenhagen Department of Linguistics, University of
Dependency Parsing 2 CMSC 723 / LING 723 / INST 725. Marine Carpuat. Fig credits: Joakim Nivre, Dan Jurafsky & James Martin
Dependency Parsing 2 CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing
The MetaGrammar Compiler: An NLP Application with a Multi-paradigm Architecture
The MetaGrammar Compiler: An NLP Application with a Multi-paradigm Architecture Denys Duchier Joseph Le Roux Yannick Parmentier LORIA Campus cientifique, BP 239, F-54 506 Vandœuvre-lès-Nancy, France MOZ
A General-Purpose Rule Extractor for SCFG-Based Machine Translation
A General-Purpose Rule Extractor for SCFG-Based Machine Translation Greg Hanneman, Michelle Burroughs, and Alon Lavie Language Technologies Institute Carnegie Mellon University Fifth Workshop on Syntax
A Joint Model of Language and Perception for Grounded Attribute Learning
A Joint Model of Language and Perception for Grounded Attribute Learning Cynthia Matuszek*, Nicholas FitzGerald*, Liefeng Bo, Luke Zettlemoyer, Dieter Fox" Learning Attributes 2 u Physically grounded systems
Simple and Efficient Learning with Automatic Operation Batching
Simple and Efficient Learning with Automatic Operation Batching Graham Neubig joint work w/ Yoav Goldberg and Chris Dyer in http://dynet.io/autobatch/ https://github.com/neubig/howtocode-2017 Neural Networks
Dependency grammar and dependency parsing
Dependency grammar and dependency parsing Syntactic analysis (5LN455) 2014-12-10 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Mid-course evaluation Mostly positive
UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS INFR08008 INFORMATICS 2A: PROCESSING FORMAL AND NATURAL LANGUAGES
UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS INFR08008 INFORMATICS 2A: PROCESSING FORMAL AND NATURAL LANGUAGES Saturday 10 th December 2016 09:30 to 11:30 INSTRUCTIONS
Dependency grammar and dependency parsing
Dependency grammar and dependency parsing Syntactic analysis (5LN455) 2015-12-09 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Activities - dependency parsing
Sequence Labeling: The Problem
Sequence Labeling: The Problem Given a sequence (in NLP, words), assign appropriate labels to each word. For example, POS tagging: DT NN VBD IN DT NN. The cat sat on the mat. 36 part-of-speech tags used
Incremental Integer Linear Programming for Non-projective Dependency Parsing
Incremental Integer Linear Programming for Non-projective Dependency Parsing Sebastian Riedel James Clarke ICCS, University of Edinburgh 22. July 2006 EMNLP 2006 S. Riedel, J. Clarke (ICCS, Edinburgh)
Context-Free Grammars. Carl Pollard Ohio State University. Linguistics 680 Formal Foundations Tuesday, November 10, 2009
Context-Free Grammars Carl Pollard Ohio State University Linguistics 680 Formal Foundations Tuesday, November 10, 2009 These slides are available at: http://www.ling.osu.edu/ scott/680 1 (1) Context-Free
Topic 1: Introduction
Recommended Exercises and Readings Topic 1: Introduction From Haskell: The craft of functional programming (3 rd Ed.) Readings: Chapter 1 Chapter 2 1 2 What is a Programming Paradigm? Programming Paradigm:
Seq2SQL: Generating Structured Queries from Natural Language Using Reinforcement Learning
Seq2SQL: Generating Structured Queries from Natural Language Using Reinforcement Learning V. Zhong, C. Xiong, R. Socher Salesforce Research arxiv: 1709.00103 Reviewed by : Bill Zhang University of Virginia
Advanced Search Algorithms
CS11-747 Neural Networks for NLP Advanced Search Algorithms Daniel Clothiaux https://phontron.com/class/nn4nlp2017/ Why search? So far, decoding has mostly been greedy Chose the most likely output from
Approximate Large Margin Methods for Structured Prediction
: Approximate Large Margin Methods for Structured Prediction Hal Daumé III and Daniel Marcu Information Sciences Institute University of Southern California {hdaume,marcu}@isi.edu Slide 1 Structured Prediction
Programs that write themselves: Program synthesis for the masses. Yu Feng UT Austin
Programs that write themselves: Program synthesis for the masses Yu Feng UT Austin New platforms impose demand for programming 2 New platforms impose demand for programming 2 New platforms impose demand
Comp 411 Principles of Programming Languages Lecture 3 Parsing. Corky Cartwright January 11, 2019
Comp 411 Principles of Programming Languages Lecture 3 Parsing Corky Cartwright January 11, 2019 Top Down Parsing What is a context-free grammar (CFG)? A recursive definition of a set of strings; it is
SEMINAR: RECENT ADVANCES IN PARSING TECHNOLOGY. Parser Evaluation Approaches
SEMINAR: RECENT ADVANCES IN PARSING TECHNOLOGY Parser Evaluation Approaches NATURE OF PARSER EVALUATION Return accurate syntactic structure of sentence. Which representation? Robustness of parsing. Quick
Homework 2: Parsing and Machine Learning
Homework 2: Parsing and Machine Learning COMS W4705_001: Natural Language Processing Prof. Kathleen McKeown, Fall 2017 Due: Saturday, October 14th, 2017, 2:00 PM This assignment will consist of tasks in
Fully Delexicalized Contexts for Syntax-Based Word Embeddings
Fully Delexicalized Contexts for Syntax-Based Word Embeddings Jenna Kanerva¹, Sampo Pyysalo² and Filip Ginter¹ ¹Dept of IT - University of Turku, Finland ²Lang. Tech. Lab - University of Cambridge turkunlp.github.io
Dependency Parsing CMSC 723 / LING 723 / INST 725. Marine Carpuat. Fig credits: Joakim Nivre, Dan Jurafsky & James Martin
Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing
CS 236 Language and Computation
CS 236 Language and Computation Course Notes Chapt. 1: Introduction Anton Setzer (Based on a book draft by J. V. Tucker and K. Stephenson) Dept. of Computer Science, Swansea University http://www.cs.swan.ac.uk/
Homework & NLTK. CS 181: Natural Language Processing Lecture 9: Context Free Grammars. Motivation. Formal Def of CFG. Uses of CFG.
C 181: Natural Language Processing Lecture 9: Context Free Grammars Kim Bruce Pomona College pring 2008 Homework & NLTK Review MLE, Laplace, and Good-Turing in smoothing.py Disclaimer: lide contents borrowed
Computational Linguistics: Syntax-Semantics
Computational Linguistics: Syntax-Semantics Raffaella Bernardi University of Trento Contents 1 The Three Tasks Revised................................... 3 2 Lambda terms and CFG...................................
NLP in practice, an example: Semantic Role Labeling
NLP in practice, an example: Semantic Role Labeling Anders Björkelund Lund University, Dept. of Computer Science anders.bjorkelund@cs.lth.se October 15, 2010 Anders Björkelund NLP in practice, an example:
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision Anonymized for review Abstract Extending the success of deep neural networks to high level tasks like natural language
The Latest Progress of KB-QA. Ming Zhou Microsoft Research Asia NLPCC 2018 Workshop Hohhot, 2018
The Latest Progress of KB-QA Ming Zhou Microsoft Research Asia NLPCC 2018 Workshop Hohhot, 2018 Query CommunityQA KBQA TableQA PassageQA VQA Edited Response Question Generation Knowledge Table Doc Image
CSE 311 Lecture 21: Context-Free Grammars. Emina Torlak and Kevin Zatloukal
CSE 311 Lecture 21: Context-Free Grammars Emina Torlak and Kevin Zatloukal 1 Topics Regular expressions A brief review of Lecture 20. Context-free grammars Syntax, semantics, and examples. 2 Regular expressions
Syntax and Grammars 1 / 21
Syntax and Grammars 1 / 21 Outline What is a language? Abstract syntax and grammars Abstract syntax vs. concrete syntax Encoding grammars as Haskell data types What is a language? 2 / 21 What is a language?
COP 3402 Systems Software Top Down Parsing (Recursive Descent)
COP 3402 Systems Software Top Down Parsing (Recursive Descent) Top Down Parsing 1 Outline 1. Top down parsing and LL(k) parsing 2. Recursive descent parsing 3. Example of recursive descent parsing of arithmetic
Context-Free Grammars
Department of Linguistics Ohio State University Syntax 2 (Linguistics 602.02) January 3, 2012 (CFGs) A CFG is an ordered quadruple T, N, D, P where a. T is a finite set called the terminals; b. N is a
Compilers and Interpreters
Overview Roadmap Language Translators: Interpreters & Compilers Context of a compiler Phases of a compiler Compiler Construction tools Terminology How related to other CS Goals of a good compiler 1 Compilers
Computer Science 9608 (Notes) Chapter: 4.1 Computational thinking and problem-solving
UWhat is Computational Thinking? Computational thinking (CT) involves a set of skills and techniques that software engineers use to write programs that underlie the computer applications you use such as
Ling 571: Deep Processing for Natural Language Processing
Ling 571: Deep Processing for Natural Language Processing Julie Medero February 4, 2013 Today s Plan Assignment Check-in Project 1 Wrap-up CKY Implementations HW2 FAQs: evalb Features & Unification Project
The Earley Parser
The 6.863 Earley Parser 1 Introduction The 6.863 Earley parser will work with any well defined context free grammar, including recursive ones and those containing empty categories. It can use either no
Question Answering System for Yioop
Question Answering System for Yioop Advisor Dr. Chris Pollett Committee Members Dr. Thomas Austin Dr. Robert Chun By Niravkumar Patel Problem Statement Question Answering System Yioop Proposed System Triplet
Algorithms for Nearest Neighbors
Algorithms for Nearest Neighbors State-of-the-Art Yury Lifshits Steklov Institute of Mathematics at St.Petersburg Yandex Tech Seminar, April 2007 1 / 28 Outline 1 Problem Statement Applications Data Models
Structured Attention Networks
Structured Attention Networks Yoon Kim Carl Denton Luong Hoang Alexander M. Rush HarvardNLP ICLR, 2017 Presenter: Chao Jiang ICLR, 2017 Presenter: Chao Jiang 1 / Outline 1 Deep Neutral Networks for Text
Shallow Parsing Swapnil Chaudhari 11305R011 Ankur Aher Raj Dabre 11305R001
Shallow Parsing Swapnil Chaudhari 11305R011 Ankur Aher - 113059006 Raj Dabre 11305R001 Purpose of the Seminar To emphasize on the need for Shallow Parsing. To impart basic information about techniques
CIS192 Python Programming
CIS192 Python Programming HTTP Requests and HTML Parsing Robert Rand University of Pennsylvania March 30, 2016 Robert Rand (University of Pennsylvania) CIS 192 March 30, 2016 1 / 19 Outline 1 HTTP Requests
Dependency grammar and dependency parsing
Dependency grammar and dependency parsing Syntactic analysis (5LN455) 2016-12-05 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Activities - dependency parsing
Parsing. Parsing. Bottom Up Parsing. Bottom Up Parsing. Bottom Up Parsing. Bottom Up Parsing
Parsing Determine if an input string is a sentence of G. G is a context free grammar (later). Assumed to be unambiguous. Recognition of the string plus determination of phrase structure. We constantly
The CKY algorithm part 1: Recognition
The CKY algorithm part 1: Recognition Syntactic analysis (5LN455) 2014-11-17 Sara Stymne Department of Linguistics and Philology Mostly based on slides from Marco Kuhlmann Recap: Parsing Parsing The automatic
Asynchronous Parallel Learning for Neural Networks and Structured Models with Dense Features
Asynchronous Parallel Learning for Neural Networks and Structured Models with Dense Features Xu SUN ( 孙栩 ) Peking University xusun@pku.edu.cn Motivation Neural networks -> Good Performance CNN, RNN, LSTM
Natural Language Processing
Natural Language Processing Classification III Dan Klein UC Berkeley 1 Classification 2 Linear Models: Perceptron The perceptron algorithm Iteratively processes the training set, reacting to training errors
A Soft-Computing Approach to Knowledge Flow Synthesis and Optimization
A Soft-Computing Approach to Knowledge Flow Synthesis and Optimization Tomáš Řehořek Pavel Kordík Computational Intelligence Group (CIG), Faculty of Information Technology (FIT), Czech Technical University
Transition-Based Dependency Parsing with Stack Long Short-Term Memory
Transition-Based Dependency Parsing with Stack Long Short-Term Memory Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, Noah A. Smith Association for Computational Linguistics (ACL), 2015 Presented
A Primer on Graph Processing with Bolinas
A Primer on Graph Processing with Bolinas J. Andreas, D. Bauer, K. M. Hermann, B. Jones, K. Knight, and D. Chiang August 20, 2013 1 Introduction This is a tutorial introduction to the Bolinas graph processing
Stack- propaga+on: Improved Representa+on Learning for Syntax
Stack- propaga+on: Improved Representa+on Learning for Syntax Yuan Zhang, David Weiss MIT, Google 1 Transi+on- based Neural Network Parser p(action configuration) So1max Hidden Embedding words labels POS
Parsing with Dynamic Programming
CS11-747 Neural Networks for NLP Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two Types of Linguistic Structure Dependency: focus on relations between words
Dependency and (R)MRS
Dependency and (R)MRS Ann Copestake aac@cl.cam.ac.uk December 9, 2008 1 Introduction Note: for current purposes, this document lacks a proper introduction, in that it assumes readers know about MRS and
What s in a name? Rebecca Dridan -
Rebecca Dridan The Data Text taken from Linux blogs Automatic markup normalisation and sentence segmentation, manually corrected Parsed with off-the-shelf English Resource Grammar 452 manually annotated
Propositional Logic Formal Syntax and Semantics. Computability and Logic
Propositional Logic Formal Syntax and Semantics Computability and Logic Syntax and Semantics Syntax: The study of how expressions are structured (think: grammar) Semantics: The study of the relationship
Homework & Announcements
Homework & nnouncements New schedule on line. Reading: Chapter 18 Homework: Exercises at end Due: 11/1 Copyright c 2002 2017 UMaine School of Computing and Information S 1 / 25 COS 140: Foundations of
(Multinomial) Logistic Regression + Feature Engineering
-6 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University (Multinomial) Logistic Regression + Feature Engineering Matt Gormley Lecture 9 Feb.
2 Ambiguity in Analyses of Idiomatic Phrases
Representing and Accessing [Textual] Digital Information (COMS/INFO 630), Spring 2006 Lecture 22: TAG Adjunction Trees and Feature Based TAGs 4/20/06 Lecturer: Lillian Lee Scribes: Nicolas Hamatake (nh39),
Encoding RNNs, 48 End of sentence (EOS) token, 207 Exploding gradient, 131 Exponential function, 42 Exponential Linear Unit (ELU), 44
A Activation potential, 40 Annotated corpus add padding, 162 check versions, 158 create checkpoints, 164, 166 create input, 160 create train and validation datasets, 163 dropout, 163 DRUG-AE.rel file,
About Boxoft All to AMR Converter
Note: This product is distributed on a try-before-you-buy basis. All features described in this documentation are enabled. The registered version does not have time limit About Boxoft All to AMR Converter
Human Robot Interaction
Human Robot Interaction Emanuele Bastianelli, Daniele Nardi bastianelli@dis.uniroma1.it Department of Computer, Control, and Management Engineering Sapienza University of Rome, Italy Introduction Robots
Grammar Rules in Prolog!!
Grammar Rules in Prolog GR-1 Backus-Naur Form (BNF) BNF is a common grammar used to define programming languages» Developed in the late 1950 s Because grammars are used to describe a language they are
Mining Web Data. Lijun Zhang
Mining Web Data Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Web Crawling and Resource Discovery Search Engine Indexing and Query Processing Ranking Algorithms Recommender Systems
Structured Prediction Basics
CS11-747 Neural Networks for NLP Structured Prediction Basics Graham Neubig Site https://phontron.com/class/nn4nlp2017/ A Prediction Problem I hate this movie I love this movie very good good neutral bad
Parsing II Top-down parsing. Comp 412
COMP 412 FALL 2018 Parsing II Top-down parsing Comp 412 source code IR Front End Optimizer Back End IR target code Copyright 2018, Keith D. Cooper & Linda Torczon, all rights reserved. Students enrolled
Proseminar on Semantic Theory Fall 2013 Ling 720 An Algebraic Perspective on the Syntax of First Order Logic (Without Quantification) 1
An Algebraic Perspective on the Syntax of First Order Logic (Without Quantification) 1 1. Statement of the Problem, Outline of the Solution to Come (1) The Key Problem There is much to recommend an algebraic
A Deep Relevance Matching Model for Ad-hoc Retrieval
A Deep Relevance Matching Model for Ad-hoc Retrieval Jiafeng Guo 1, Yixing Fan 1, Qingyao Ai 2, W. Bruce Croft 2 1 CAS Key Lab of Web Data Science and Technology, Institute of Computing Technology, Chinese
Machine Learning in GATE
Machine Learning in GATE Angus Roberts, Horacio Saggion, Genevieve Gorrell Recap Previous two days looked at knowledge engineered IE This session looks at machine learned IE Supervised learning Effort
CSE P 501 Exam 8/5/04
Name There are 7 questions worth a total of 65 points. Please budget your time so you get to all of the questions. Keep your answers brief and to the point. You may refer to the following references: Course
Natural Language Processing (CSE 517): Compositional Semantics
Natural Language Processing (CSE 517): Compositional Semantics Noah Smith c 2018 University of Washington nasmith@cs.washington.edu May 18, 2018 1 / 54 Bridging the Gap between Language and the World In
Ortolang Tools : MarsaTag
Ortolang Tools : MarsaTag Stéphane Rauzy, Philippe Blache, Grégoire de Montcheuil SECOND VARIAMU WORKSHOP LPL, Aix-en-Provence August 20th & 21st, 2014 ORTOLANG received a State aid under the «Investissements
CMPS Programming Languages. Dr. Chengwei Lei CEECS California State University, Bakersfield
CMPS 3500 Programming Languages Dr. Chengwei Lei CEECS California State University, Bakersfield Chapter 3 Describing Syntax and Semantics Chapter 3 Topics Introduction The General Problem of Describing
CMU-UKA Syntax Augmented Machine Translation
Outline CMU-UKA Syntax Augmented Machine Translation Ashish Venugopal, Andreas Zollmann, Stephan Vogel, Alex Waibel InterACT, LTI, Carnegie Mellon University Pittsburgh, PA Outline Outline 1 2 3 4 Issues
ROCHESTER INSTITUTE OF TECHNOLOGY. SQL Tool Kit
ROCHESTER INSTITUTE OF TECHNOLOGY SQL Tool Kit Submitted by: Chandni Pakalapati Advisor: Dr.Carlos Rivero 1 Table of Contents 1. Abstract... 3 2. Introduction... 4 3. System Overview... 5 4. Implementation...
LSTM for Language Translation and Image Captioning. Tel Aviv University Deep Learning Seminar Oran Gafni & Noa Yedidia
1 LSTM for Language Translation and Image Captioning Tel Aviv University Deep Learning Seminar Oran Gafni & Noa Yedidia 2 Part I LSTM for Language Translation Motivation Background (RNNs, LSTMs) Model
Talen en Compilers. Jurriaan Hage , period 2. November 13, Department of Information and Computing Sciences Utrecht University
Talen en Compilers 2017-2018, period 2 Jurriaan Hage Department of Information and Computing Sciences Utrecht University November 13, 2017 1. Introduction 1-1 This lecture Introduction Course overview
Syntax-Directed Translation. Lecture 14
Syntax-Directed Translation Lecture 14 (adapted from slides by R. Bodik) 9/27/2006 Prof. Hilfinger, Lecture 14 1 Motivation: parser as a translator syntax-directed translation stream of tokens parser ASTs,
Parsing partially bracketed input
Parsing partially bracketed input Martijn Wieling, Mark-Jan Nederhof and Gertjan van Noord Humanities Computing, University of Groningen Abstract A method is proposed to convert a Context Free Grammar
Outline. NLP Applications: An Example. Pointwise Mutual Information Information Retrieval (PMI-IR) Search engine inefficiencies
A Search Engine for Natural Language Applications AND Relational Web Search: A Preview Michael J. Cafarella (joint work with Michele Banko, Doug Downey, Oren Etzioni, Stephen Soderland) CSE454 University
Chapter 3. Describing Syntax and Semantics
Chapter 3 Describing Syntax and Semantics Chapter 3 Topics Introduction The General Problem of Describing Syntax Formal Methods of Describing Syntax Attribute Grammars Describing the Meanings of Programs:
Ranking-Based Suggestion Algorithms for Semantic Web Service Composition
Ranking-Based Suggestion Algorithms for Semantic Web Service Composition Rui Wang, Sumedha Ganjoo, John A. Miller and Eileen T. Kraemer Presented by: John A. Miller July 5, 2010 Outline Introduction &
CS 230 Programming Languages
CS 230 Programming Languages 09 / 16 / 2013 Instructor: Michael Eckmann Today s Topics Questions/comments? Continue Syntax & Semantics Mini-pascal Attribute Grammars More Perl A more complex grammar Let's
Approximately Uniform Random Sampling in Sensor Networks
Approximately Uniform Random Sampling in Sensor Networks Boulat A. Bash, John W. Byers and Jeffrey Considine Motivation Data aggregation Approximations to COUNT, SUM, AVG, MEDIAN Existing work does not
Intro to XML. Borrowed, with author s permission, from:
Intro to XML Borrowed, with author s permission, from: http://business.unr.edu/faculty/ekedahl/is389/topic3a ndroidintroduction/is389androidbasics.aspx Part 1: XML Basics Why XML Here? You need to understand