Machine Translation and Discriminative Models
|
|
|
- Christal Booth
- 5 years ago
- Views:
Transcription
1 Machine Translation and Discriminative Models Tree-to-tree transfer and Discriminative learning Martin Popel ÚFAL (Institute of Formal and Applied Linguistics) Charles University in Prague March 23rd 2015, Seminar of Formal Linguistics, Prague
2 Outline 1 Intro TectoMT schema Isomorphic transfer Moses 2 Quiz 3 MT as labeling 4 TectoMT over years 2008 baseline transfer 2009 HMTM 2010 MaxEnt 2012 TectoMoses 2012 Gibbs 2013 Easy-first 2013 Interpol 2014 VowpalWabbit
3 TectoMT: analysis, transfer, synthesis rule based & statistical blocks ANALYSIS TRANSFER SYNTHESIS tectogramatical layer t-layer fill formems grammatemes use fill morphological categories query build t-tree dictionary HMTM impose agreement mark edges to contract add functional words analytical layer a-layer analytical functions generate parser (McDonald's MST) wordforms morphological layer m-layer tagger (Morce) concatenate lemmatization source language (English) tokenization (Czech) target language w-layer segmentation
4 SYNTHESIS TectoMT: isomorphic transfer (1-1 node mapping) Source tree (Czech) TRANSFER Target tree (English) P(optimal_tree) = P E(strojový machine) P T(machine translation) P E (překlad translation) P T (translation be) P E (snadný easy) P T (easy be) P E (být be) P T (be ROOT) ROOT ROOT P E (být have) = být P E (být be) = 0.8 be have ANALYSIS překlad snadný P E (překlad arcade) = 0.7 P E(překlad translation) = translation arcade easy simple P T (machine translation) = strojový Source sentence: Strojový překlad by měl být snadný. P E(strojový machine) = 0.4 machine P E(strojový engine) = 0.5 engine Target sentence: Machine translation should be easy. P E (source target) emission probabilities translation model P T (dependent governing) transition probabilities target-language tree model
5 Phrase-based Statistical Machine Translation currently most popular approach (Moses toolkit) no linguistic analysis needed (just tokenization) translates each phrase independently (except for LM) many segmentations to phrases considered, only one used
6 Quiz: English-Czech translation Is it possible to translate prime as vláda? In which context?
7 Quiz: English-Czech translation Is it possible to translate prime as vláda? In which context? prime minister předseda vlády
8 Quiz: English-Czech translation Is it possible to translate prime as vláda? In which context? prime minister předseda vlády How to formalize such translation rule? lemma=prime formeme=adj:attr parent_lemma=minister lemma=vláda formeme=n:2 parent_lemma=předseda
9 Quiz: English-Czech translation Is it possible to translate prime as vláda? In which context? prime minister předseda vlády How to formalize such translation rule? lemma=prime formeme=adj:attr parent_lemma=minister lemma=vláda formeme=n:2 parent_lemma=předseda This is still isomorphic transfer, unlike prime minister premiér.
10 Quiz: English-Czech translation Is it possible to translate find as přijít? In which context?
11 Quiz: English-Czech translation Is it possible to translate find as přijít? In which context? (on t-layer, find find_out)
12 Quiz: English-Czech translation Is it possible to translate find as přijít? In which context? (on t-layer, find find_out) Agatha found that book interesting. Agátě přišla ta kniha zajímavá.
13 Quiz: English-Czech translation Is it possible to translate find as přijít? In which context? (on t-layer, find find_out) Agatha found that book interesting. Agátě přišla ta kniha zajímavá. How to formalize such translation rule?
14 Quiz: English-Czech translation Is it possible to translate find as přijít? In which context? (on t-layer, find find_out) Agatha[n:subj] found that book[n:obj] interesting[adj:compl]. Agátě[n:3] přišla ta kniha[n:1] zajímavá[adj:1]. How to formalize such translation rule? child1_formeme=n:subj lemma=find child2_formeme=n:obj child3_formeme=adj:compl (adj:compl means predicative adjective) child1_formeme=n:3 lemma=přijít child2_formeme=n:1 child3_formeme=adj:1
15 Representation of t-layer lemma and formeme as two attributes find v:fin Agatha n:subj book n:obj interesting adj:compl this adj:attr
16 Representation of t-layer lemma and formeme as interleaved sub-nodes v:fin find n:subj Agatha n:obj book adj:attr this adj:compl interesting
17 Representation of t-layer lemma and formeme as interleaved sub-nodes v:fin n:subj Agatha find n:obj book adj:attr adj:compl interesting this grammatemes: translated in postprocessing (current approach) as subnodes (leaves, children of lemmas) encoded within lemma, but only if grammateme changed
18 Handling non-isomorphic transfer preprocessing or postprocessing within transfer (current approach) natively in the main transfer algorithm convert training data to isomorphic trees [not tried yet] n-1 alignment: add special [delete_node] label to the target side 1-n alignment: encode added nodes (L+F) into the main lemma encode topology change: as_child, as_sibling, as_parent
19 TectoMT transfer over years year BLEUdiff method 2008 initial baseline HMTM (TreeViterbi, TreeLM) HMTM + MaxEnt TectoMoses 2012 NA Gibbs sampling treelets Easy-first treelets Interpol treelets VowpalWabbit
20 TectoMT transfer over years year BLEUdiff method 2008 initial baseline HMTM (TreeViterbi, TreeLM) HMTM + MaxEnt TectoMoses 2012 NA Gibbs sampling treelets Easy-first treelets Interpol treelets VowpalWabbit +0.9 other improvements in
21 TectoMT transfer over years year BLEUdiff method 2008 initial baseline HMTM (TreeViterbi, TreeLM) HMTM + MaxEnt TectoMoses 2012 NA Gibbs sampling treelets Easy-first treelets Interpol treelets VowpalWabbit +0.9 other improvements in QTLeap en cs in two months
22 2008: baseline TectoMT transfer static translation model P(target source) = #(source,target) #(source) first translate formemes, then lemmas use only the top variant WMT 2009 en cs results BLEU human score Moses (CUNI) Google Moses (UEdin) Eurotran XP PC Translator TectoMT
23 SYNTHESIS 2009: Hidden Markov Tree Model (HMTM) still using static translation models, but also TreeLM (target lemma-formeme and parent-child compatibility) best labeling is found via HMTM (Tree Viterbi) Source tree (Czech) ROOT TRANSFER P(optimal_tree) = P E(strojový machine) P T(machine translation) P E (překlad translation) P T (translation be) P E (snadný easy) P T (easy be) P E (být be) P T (be ROOT) Target tree (English) ROOT P E (být have) = být P E (být be) = 0.8 be have ANALYSIS překlad snadný P E (překlad arcade) = 0.7 P E(překlad translation) = translation arcade easy simple P T (machine translation) = strojový Source sentence: Strojový překlad by měl být snadný. P E(strojový machine) = 0.4 machine P E(strojový engine) = 0.5 engine Target sentence: Machine translation should be easy. P E (source target) emission probabilities translation model P T (dependent governing) transition probabilities target-language tree model
24 2010: Maximum Entropy translation model still using HMTM (and generative TreeLM), but the static model P(lemma src_lemma) interpolated with context-sensitive discriminative (MaxEnt) model P(lemma src_lemma, other features) he / n:subj union / n:with+x agree / v:fin tense=past, voice=active negation=0, sempos=v cut / v:inf, has_left_child=0, sempos=v, has_right_child=1, tag=vb, position=right, named_entity=0 overtime / n:obj chop, saw, trim, shorten, lumber, hew, lower, delete, crop abolish, cancel,... all / adj:attr TRANSFER ANALYSIS SYNTHESIS He agreed with the unions to cut all overtime. Dohodl se s odbory na zrušení všech přesčasů.
25 2012: TectoMoses substitute transfer (MaxEnt+HMTM) in TectoMT with phrase-based decoding (Moses) of linearized t-trees 2 factors: lemma and formeme, but joint (L+F) n-gram LM better project dependencies rule based and use& TectoMT statistical synthesis blocks ANALYSIS TRANSFER SYNTHESIS tectogramatical layer t-layer fill formems grammatemes use fill morphological categories query build t-tree dictionary HMTM impose agreement mark edges to contract add functional words analytical layer a-layer analytical functions generate parser (McDonald's MST) wordforms morphological layer m-layer tagger (Morce) concatenate lemmatization source language (English) tokenization (Czech) target language w-layer segmentation
26 : Gibbs sampling and CRP segmentations example of treelet-based transfer, P(trg treelet src treelet) Bayesian approach use Chinese Restaurant Process on parallel treebank to learn optimal segmentation to treelet pairs optimal translations use Gibbs sampling both in training and decoding Problems Pitman Yor instead of Chinese Restaurant (heavy tail) Slice sampling or annealing instead of Gibbs (local maxima) We want cut+grass=sekat+tráva and cut+taxes=snížit+daně, but CRP/PYP prefers more reusable segments grass=tráva, taxes=daně (and cut=sekat, cut=snížit). CRP/PYP knows nothing about translation (cf. Chung et al. 2013: Sampling tree fragments from forests).
27 2013: Easy-first treelet-based transfer easy-first decoding of overlapping treelets guided learning of both how to translate each treelet in which order Handles non-isomorphic transfer natively (not tried). How to integrate TreeLM (for partial translations)?
28 2013: Interpolation treelet-based transfer (Interpol) similar to easy-first decoding one feature (weight): source treelet size many other features (lexicalized, PoS tags,...) combinations and quantizations of features all matching rules applied, their scores interpolated does not handle non-isomorphism no guided learning Example score(zajímavá) = P MaxEnt (zajímavá interesting) + w L s(interesting zajímavá) + w Lf s(interesting adj:compl zajímavá adj:1) + w Lf s(interesting adj:compl zajímavá adj:attr) + w Lfl s(interesting adj:compl find zajímavá adj:1 přijít) + w Lfl s(interesting adj:compl find zajímavá adj:attr najít) + w Lflf s(interesting adj:compl find v:fin zajímavá adj:1 přijít v:fin)
29 How is Interpol (2013) different from MaxEnt (2010)?
30 How is Interpol (2013) different from MaxEnt (2010)? The Logit, or There and Back Again
31 How is Interpol (2013) different from MaxEnt (2010)? The Logit, or There and Back Again Interpol can be trained with MaxEnt (multinomial logit) HMTM can be applied on top of Interpol Interpol is equivalent to the 2010 approach with additional features Summary features feature value precomputed on whole training data, relative frequency, or rather (#(src,trg) - 1)/#(src) dense feature (unlike sparse lexicalized higher-order features) Moses also uses summary features (TM-forward, TM-backward, LM)
32 2014: VowpalWabbit-based transfer VW is an ultra-fast and modular machine learning toolkit optimized SGD (AdaGrad, dense+sparse features,...) cost-sensitive one-against-all reduction to binary classification logistic loss enables probabilistic interpretation (for HMTM) all lemmas in one model, fixed memory requirements label-dependent features (features shared for more lemmas)
33 Thank you
Machine Learning for Deep-syntactic MT
Machine Learning for Deep-syntactic MT Martin Popel ÚFAL (Institute of Formal and Applied Linguistics) Charles University in Prague September 11, 2015 Seminar on the 35th Anniversary of the Cooperation
Machine Translation Zoo
Machine Translation Zoo Tree-to-tree transfer and Discriminative learning Martin Popel ÚFAL (Institute of Formal and Applied Linguistics) Charles University in Prague May 5th 2013, Seminar of Formal Linguistics,
TectoMT: Machine Translation System
: Machine Translation System Martin Popel ÚFAL (Institute of Formal and Applied Linguistics) Charles University in Prague Universität des Saarlandes November 4 th 2010, Saarbrücken, Germany Outline NLP
Treex: Modular NLP Framework
: Modular NLP Framework Martin Popel ÚFAL (Institute of Formal and Applied Linguistics) Charles University in Prague September 2015, Prague, MT Marathon Outline Motivation, vs. architecture internals Future
TectoMT: Modular NLP Framework
: Modular NLP Framework Martin Popel, Zdeněk Žabokrtský ÚFAL, Charles University in Prague IceTAL, 7th International Conference on Natural Language Processing August 17, 2010, Reykjavik Outline Motivation
INF5820/INF9820 LANGUAGE TECHNOLOGICAL APPLICATIONS. Jan Tore Lønning, Lecture 8, 12 Oct
1 INF5820/INF9820 LANGUAGE TECHNOLOGICAL APPLICATIONS Jan Tore Lønning, Lecture 8, 12 Oct. 2016 jtl@ifi.uio.no Today 2 Preparing bitext Parameter tuning Reranking Some linguistic issues STMT so far 3 We
A CASE STUDY: Structure learning for Part-of-Speech Tagging. Danilo Croce WMR 2011/2012
A CAS STUDY: Structure learning for Part-of-Speech Tagging Danilo Croce WM 2011/2012 27 gennaio 2012 TASK definition One of the tasks of VALITA 2009 VALITA is an initiative devoted to the evaluation of
Transition-Based Dependency Parsing with Stack Long Short-Term Memory
Transition-Based Dependency Parsing with Stack Long Short-Term Memory Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, Noah A. Smith Association for Computational Linguistics (ACL), 2015 Presented
Introduction to Data-Driven Dependency Parsing
Introduction to Data-Driven Dependency Parsing Introductory Course, ESSLLI 2007 Ryan McDonald 1 Joakim Nivre 2 1 Google Inc., New York, USA E-mail: ryanmcd@google.com 2 Uppsala University and Växjö University,
An Unsupervised Model for Joint Phrase Alignment and Extraction
An Unsupervised Model for Joint Phrase Alignment and Extraction Graham Neubig 1,2, Taro Watanabe 2, Eiichiro Sumita 2, Shinsuke Mori 1, Tatsuya Kawahara 1 1 Graduate School of Informatics, Kyoto University
Hidden Markov Models. Natural Language Processing: Jordan Boyd-Graber. University of Colorado Boulder LECTURE 20. Adapted from material by Ray Mooney
Hidden Markov Models Natural Language Processing: Jordan Boyd-Graber University of Colorado Boulder LECTURE 20 Adapted from material by Ray Mooney Natural Language Processing: Jordan Boyd-Graber Boulder
Let s get parsing! Each component processes the Doc object, then passes it on. doc.is_parsed attribute checks whether a Doc object has been parsed
Let s get parsing! SpaCy default model includes tagger, parser and entity recognizer nlp = spacy.load('en ) tells spacy to use "en" with ["tagger", "parser", "ner"] Each component processes the Doc object,
Shallow Parsing Swapnil Chaudhari 11305R011 Ankur Aher Raj Dabre 11305R001
Shallow Parsing Swapnil Chaudhari 11305R011 Ankur Aher - 113059006 Raj Dabre 11305R001 Purpose of the Seminar To emphasize on the need for Shallow Parsing. To impart basic information about techniques
Implementing a Variety of Linguistic Annotations
Implementing a Variety of Linguistic Annotations through a Common Web-Service Interface Adam Funk, Ian Roberts, Wim Peters University of Sheffield 18 May 2010 Adam Funk, Ian Roberts, Wim Peters Implementing
CS224n: Natural Language Processing with Deep Learning 1 Lecture Notes: Part IV Dependency Parsing 2 Winter 2019
CS224n: Natural Language Processing with Deep Learning 1 Lecture Notes: Part IV Dependency Parsing 2 Winter 2019 1 Course Instructors: Christopher Manning, Richard Socher 2 Authors: Lisa Wang, Juhi Naik,
Discriminative Training with Perceptron Algorithm for POS Tagging Task
Discriminative Training with Perceptron Algorithm for POS Tagging Task Mahsa Yarmohammadi Center for Spoken Language Understanding Oregon Health & Science University Portland, Oregon yarmoham@ohsu.edu
Graph-Based Parsing. Miguel Ballesteros. Algorithms for NLP Course. 7-11
Graph-Based Parsing Miguel Ballesteros Algorithms for NLP Course. 7-11 By using some Joakim Nivre's materials from Uppsala University and Jason Eisner's material from Johns Hopkins University. Outline
THE CUED NIST 2009 ARABIC-ENGLISH SMT SYSTEM
THE CUED NIST 2009 ARABIC-ENGLISH SMT SYSTEM Adrià de Gispert, Gonzalo Iglesias, Graeme Blackwood, Jamie Brunning, Bill Byrne NIST Open MT 2009 Evaluation Workshop Ottawa, The CUED SMT system Lattice-based
Depfix: Automatic post-editing of phrase-based machine translation outputs
Rudolf Rosa rosa@ufal.mff.cuni.cz Depfix: Automatic post-editing of phrase-based machine translation outputs Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied
NLP in practice, an example: Semantic Role Labeling
NLP in practice, an example: Semantic Role Labeling Anders Björkelund Lund University, Dept. of Computer Science anders.bjorkelund@cs.lth.se October 15, 2010 Anders Björkelund NLP in practice, an example:
Parsing with Dynamic Programming
CS11-747 Neural Networks for NLP Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two Types of Linguistic Structure Dependency: focus on relations between words
Agenda for today. Homework questions, issues? Non-projective dependencies Spanning tree algorithm for non-projective parsing
Agenda for today Homework questions, issues? Non-projective dependencies Spanning tree algorithm for non-projective parsing 1 Projective vs non-projective dependencies If we extract dependencies from trees,
Marcello Federico MMT Srl / FBK Trento, Italy
Marcello Federico MMT Srl / FBK Trento, Italy Proceedings for AMTA 2018 Workshop: Translation Quality Estimation and Automatic Post-Editing Boston, March 21, 2018 Page 207 Symbiotic Human and Machine Translation
Statistical Machine Translation Part IV Log-Linear Models
Statistical Machine Translation art IV Log-Linear Models Alexander Fraser Institute for Natural Language rocessing University of Stuttgart 2011.11.25 Seminar: Statistical MT Where we have been We have
Hidden Markov Models. Gabriela Tavares and Juri Minxha Mentor: Taehwan Kim CS159 04/25/2017
Hidden Markov Models Gabriela Tavares and Juri Minxha Mentor: Taehwan Kim CS159 04/25/2017 1 Outline 1. 2. 3. 4. Brief review of HMMs Hidden Markov Support Vector Machines Large Margin Hidden Markov Models
Conditional Random Fields and beyond D A N I E L K H A S H A B I C S U I U C,
Conditional Random Fields and beyond D A N I E L K H A S H A B I C S 5 4 6 U I U C, 2 0 1 3 Outline Modeling Inference Training Applications Outline Modeling Problem definition Discriminative vs. Generative
RLAT Rapid Language Adaptation Toolkit
RLAT Rapid Language Adaptation Toolkit Tim Schlippe May 15, 2012 RLAT Rapid Language Adaptation Toolkit - 2 RLAT Rapid Language Adaptation Toolkit RLAT Rapid Language Adaptation Toolkit - 3 Outline Introduction
Natural Language Processing CS 6320 Lecture 6 Neural Language Models. Instructor: Sanda Harabagiu
Natural Language Processing CS 6320 Lecture 6 Neural Language Models Instructor: Sanda Harabagiu In this lecture We shall cover: Deep Neural Models for Natural Language Processing Introduce Feed Forward
Introducing XAIRA. Lou Burnard Tony Dodd. An XML aware tool for corpus indexing and searching. Research Technology Services, OUCS
Introducing XAIRA An XML aware tool for corpus indexing and searching Lou Burnard Tony Dodd Research Technology Services, OUCS What is XAIRA? XML Aware Indexing and Retrieval Architecture Developed from
Dependency Parsing CMSC 723 / LING 723 / INST 725. Marine Carpuat. Fig credits: Joakim Nivre, Dan Jurafsky & James Martin
Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing
Incremental Integer Linear Programming for Non-projective Dependency Parsing
Incremental Integer Linear Programming for Non-projective Dependency Parsing Sebastian Riedel James Clarke ICCS, University of Edinburgh 22. July 2006 EMNLP 2006 S. Riedel, J. Clarke (ICCS, Edinburgh)
Final Project Discussion. Adam Meyers Montclair State University
Final Project Discussion Adam Meyers Montclair State University Summary Project Timeline Project Format Details/Examples for Different Project Types Linguistic Resource Projects: Annotation, Lexicons,...
Hierarchical Phrase-Based Translation with WFSTs. Weighted Finite State Transducers
Hierarchical Phrase-Based Translation with Weighted Finite State Transducers Gonzalo Iglesias 1 Adrià de Gispert 2 Eduardo R. Banga 1 William Byrne 2 1 Department of Signal Processing and Communications
Structured Perceptron. Ye Qiu, Xinghui Lu, Yue Lu, Ruofei Shen
Structured Perceptron Ye Qiu, Xinghui Lu, Yue Lu, Ruofei Shen 1 Outline 1. 2. 3. 4. Brief review of perceptron Structured Perceptron Discriminative Training Methods for Hidden Markov Models: Theory and
Modeling Sequence Data
Modeling Sequence Data CS4780/5780 Machine Learning Fall 2011 Thorsten Joachims Cornell University Reading: Manning/Schuetze, Sections 9.1-9.3 (except 9.3.1) Leeds Online HMM Tutorial (except Forward and
Better translations with user collaboration - Integrated MT at Microsoft
Better s with user collaboration - Integrated MT at Microsoft Chris Wendt Microsoft Research One Microsoft Way Redmond, WA 98052 christw@microsoft.com Abstract This paper outlines the methodologies Microsoft
Features for Syntax-based Moses: Project Report
Features for Syntax-based Moses: Project Report Eleftherios Avramidis, Arefeh Kazemi, Tom Vanallemeersch, Phil Williams, Rico Sennrich, Maria Nadejde, Matthias Huck 13 September 2014 Features for Syntax-based
Feature Extraction and Loss training using CRFs: A Project Report
Feature Extraction and Loss training using CRFs: A Project Report Ankan Saha Department of computer Science University of Chicago March 11, 2008 Abstract POS tagging has been a very important problem in
Information Retrieval and Organisation
Information Retrieval and Organisation Dell Zhang Birkbeck, University of London 2016/17 IR Chapter 02 The Term Vocabulary and Postings Lists Constructing Inverted Indexes The major steps in constructing
NLP Chain. Giuseppe Castellucci Web Mining & Retrieval a.a. 2013/2014
NLP Chain Giuseppe Castellucci castellucci@ing.uniroma2.it Web Mining & Retrieval a.a. 2013/2014 Outline NLP chains RevNLT Exercise NLP chain Automatic analysis of texts At different levels Token Morphological
Charles University at CLEF 2007 CL-SR Track
Charles University at CLEF 2007 CL-SR Track Pavel Češka and Pavel Pecina Institute of Formal and Applied Linguistics Charles University, 118 00 Praha 1, Czech Republic {ceska,pecina}@ufal.mff.cuni.cz Abstract
A Space-Efficient Phrase Table Implementation Using Minimal Perfect Hash Functions
A Space-Efficient Phrase Table Implementation Using Minimal Perfect Hash Functions Marcin Junczys-Dowmunt Faculty of Mathematics and Computer Science Adam Mickiewicz University ul. Umultowska 87, 61-614
Natural Language Processing
Natural Language Processing Language Models Language models are distributions over sentences N gram models are built from local conditional probabilities Language Modeling II Dan Klein UC Berkeley, The
Support Vector Machine Learning for Interdependent and Structured Output Spaces
Support Vector Machine Learning for Interdependent and Structured Output Spaces I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun, ICML, 2004. And also I. Tsochantaridis, T. Joachims, T. Hofmann,
Transition-based Parsing with Neural Nets
CS11-747 Neural Networks for NLP Transition-based Parsing with Neural Nets Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two Types of Linguistic Structure Dependency: focus on relations between
Forest-Based Search Algorithms
Forest-Based Search Algorithms for Parsing and Machine Translation Liang Huang University of Pennsylvania Google Research, March 14th, 2008 Search in NLP is not trivial! I saw her duck. Aravind Joshi 2
Statistical parsing. Fei Xia Feb 27, 2009 CSE 590A
Statistical parsing Fei Xia Feb 27, 2009 CSE 590A Statistical parsing History-based models (1995-2000) Recent development (2000-present): Supervised learning: reranking and label splitting Semi-supervised
Discriminative Training for Phrase-Based Machine Translation
Discriminative Training for Phrase-Based Machine Translation Abhishek Arun 19 April 2007 Overview 1 Evolution from generative to discriminative models Discriminative training Model Learning schemes Featured
A Bambara Tonalization System for Word Sense Disambiguation Using Differential Coding, Segmentation and Edit Operation Filtering
A Bambara Tonalization System for Word Sense Disambiguation Using Differential Coding, Segmentation and Edit Operation Filtering Luigi (Y.-C.) Liu Damien Nouvel ER-TIM, INALCO, 2 rue de Lille, Paris, France
Experimenting in MT: Moses Toolkit and Eman
Experimenting in MT: Moses Toolkit and Eman Aleš Tamchyna, Ondřej Bojar Institute of Formal and Applied Linguistics Faculty of Mathematics and Physics Charles University, Prague Tue Sept 8, 2015 1 / 30
Improving the quality of a customized SMT system using shared training data. August 28, 2009
Improving the quality of a customized SMT system using shared training data Chris.Wendt@microsoft.com Will.Lewis@microsoft.com August 28, 2009 1 Overview Engine and Customization Basics Experiment Objective
Apache UIMA and Mayo ctakes
Apache and Mayo and how it is used in the clinical domain March 16, 2012 Apache and Mayo Outline 1 Apache and Mayo Outline 1 2 Introducing Pipeline Modules Apache and Mayo What is? (You - eee - muh) Unstructured
Annotation Procedure in Building the Prague Czech-English Dependency Treebank
Annotation Procedure in Building the Prague Czech-English Dependency Treebank Marie Mikulová and Jan Štěpánek Institute of Formal and Applied Linguistics, Charles University in Prague Abstract. In this
Data for linguistics ALEXIS DIMITRIADIS. Contents First Last Prev Next Back Close Quit
Data for linguistics ALEXIS DIMITRIADIS Text, corpora, and data in the wild 1. Where does language data come from? The usual: Introspection, questionnaires, etc. Corpora, suited to the domain of study:
Start downloading our playground for SMT: wget
Before we start... Start downloading Moses: wget http://ufal.mff.cuni.cz/~tamchyna/mosesgiza.64bit.tar.gz Start downloading our playground for SMT: wget http://ufal.mff.cuni.cz/eman/download/playground.tar
Assignment 4 CSE 517: Natural Language Processing
Assignment 4 CSE 517: Natural Language Processing University of Washington Winter 2016 Due: March 2, 2016, 1:30 pm 1 HMMs and PCFGs Here s the definition of a PCFG given in class on 2/17: A finite set
Motivation: Shortcomings of Hidden Markov Model. Ko, Youngjoong. Solution: Maximum Entropy Markov Model (MEMM)
Motivation: Shortcomings of Hidden Markov Model Maximum Entropy Markov Models and Conditional Random Fields Ko, Youngjoong Dept. of Computer Engineering, Dong-A University Intelligent System Laboratory,
The HDU Discriminative SMT System for Constrained Data PatentMT at NTCIR10
The HDU Discriminative SMT System for Constrained Data PatentMT at NTCIR10 Patrick Simianer, Gesa Stupperich, Laura Jehl, Katharina Wäschle, Artem Sokolov, Stefan Riezler Institute for Computational Linguistics,
Annual Public Report - Project Year 2 November 2012
November 2012 Grant Agreement number: 247762 Project acronym: FAUST Project title: Feedback Analysis for User Adaptive Statistical Translation Funding Scheme: FP7-ICT-2009-4 STREP Period covered: from
Challenge. Case Study. The fabric of space and time has collapsed. What s the big deal? Miami University of Ohio
Case Study Use Case: Recruiting Segment: Recruiting Products: Rosette Challenge CareerBuilder, the global leader in human capital solutions, operates the largest job board in the U.S. and has an extensive
UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS INFR08008 INFORMATICS 2A: PROCESSING FORMAL AND NATURAL LANGUAGES
UNIVERSITY OF EDINBURGH COLLEGE OF SCIENCE AND ENGINEERING SCHOOL OF INFORMATICS INFR08008 INFORMATICS 2A: PROCESSING FORMAL AND NATURAL LANGUAGES Saturday 10 th December 2016 09:30 to 11:30 INSTRUCTIONS
An efficient and user-friendly tool for machine translation quality estimation
An efficient and user-friendly tool for machine translation quality estimation Kashif Shah, Marco Turchi, Lucia Specia Department of Computer Science, University of Sheffield, UK {kashif.shah,l.specia}@sheffield.ac.uk
A General-Purpose Rule Extractor for SCFG-Based Machine Translation
A General-Purpose Rule Extractor for SCFG-Based Machine Translation Greg Hanneman, Michelle Burroughs, and Alon Lavie Language Technologies Institute Carnegie Mellon University Fifth Workshop on Syntax
K-best Parsing Algorithms
K-best Parsing Algorithms Liang Huang University of Pennsylvania joint work with David Chiang (USC Information Sciences Institute) k-best Parsing Liang Huang (Penn) k-best parsing 2 k-best Parsing I saw
Dynamic Feature Selection for Dependency Parsing
Dynamic Feature Selection for Dependency Parsing He He, Hal Daumé III and Jason Eisner EMNLP 2013, Seattle Structured Prediction in NLP Part-of-Speech Tagging Parsing N N V Det N Fruit flies like a banana
Learning Latent Linguistic Structure to Optimize End Tasks. David A. Smith with Jason Naradowsky and Xiaoye Tiger Wu
Learning Latent Linguistic Structure to Optimize End Tasks David A. Smith with Jason Naradowsky and Xiaoye Tiger Wu 12 October 2012 Learning Latent Linguistic Structure to Optimize End Tasks David A. Smith
Digital Libraries: Language Technologies
Digital Libraries: Language Technologies RAFFAELLA BERNARDI UNIVERSITÀ DEGLI STUDI DI TRENTO P.ZZA VENEZIA, ROOM: 2.05, E-MAIL: BERNARDI@DISI.UNITN.IT Contents 1 Recall: Inverted Index..........................................
An undirected graph is a tree if and only of there is a unique simple path between any 2 of its vertices.
Trees Trees form the most widely used subclasses of graphs. In CS, we make extensive use of trees. Trees are useful in organizing and relating data in databases, file systems and other applications. Formal
MEMMs (Log-Linear Tagging Models)
Chapter 8 MEMMs (Log-Linear Tagging Models) 8.1 Introduction In this chapter we return to the problem of tagging. We previously described hidden Markov models (HMMs) for tagging problems. This chapter
Diagnostic evaluation of MT with DELiC4MT
Diagnostic evaluation of MT with DELiC4MT Final report MT Marathon 2012 Edinburgh, 5th September 2012 Walid Aransa, Luong Ngoc Quang, Antonio Toral Overview Automatic Diagonistic Evaluation on Linguistic
Detection and Extraction of Events from s
Detection and Extraction of Events from Emails Shashank Senapaty Department of Computer Science Stanford University, Stanford CA senapaty@cs.stanford.edu December 12, 2008 Abstract I build a system to
JOINT INTENT DETECTION AND SLOT FILLING USING CONVOLUTIONAL NEURAL NETWORKS. Puyang Xu, Ruhi Sarikaya. Microsoft Corporation
JOINT INTENT DETECTION AND SLOT FILLING USING CONVOLUTIONAL NEURAL NETWORKS Puyang Xu, Ruhi Sarikaya Microsoft Corporation ABSTRACT We describe a joint model for intent detection and slot filling based
CS395T Project 2: Shift-Reduce Parsing
CS395T Project 2: Shift-Reduce Parsing Due date: Tuesday, October 17 at 9:30am In this project you ll implement a shift-reduce parser. First you ll implement a greedy model, then you ll extend that model
iems Interactive Experiment Management System Final Report
iems Interactive Experiment Management System Final Report Pēteris Ņikiforovs Introduction Interactive Experiment Management System (Interactive EMS or iems) is an experiment management system with a graphical
CS54701: Information Retrieval
CS54701: Information Retrieval Basic Concepts 19 January 2016 Prof. Chris Clifton 1 Text Representation: Process of Indexing Remove Stopword, Stemming, Phrase Extraction etc Document Parser Extract useful
Automated Extraction of Event Details from Text Snippets
Automated Extraction of Event Details from Text Snippets Kavi Goel, Pei-Chin Wang December 16, 2005 1 Introduction We receive emails about events all the time. A message will typically include the title
Syntactic N-grams as Machine Learning. Features for Natural Language Processing. Marvin Gülzow. Basics. Approach. Results.
s Table of Contents s 1 s 2 3 4 5 6 TL;DR s Introduce n-grams Use them for authorship attribution Compare machine learning approaches s J48 (decision tree) SVM + S work well Section 1 s s Definition n-gram:
Constraints for corpora development and validation 1
Constraints for corpora development and validation 1 Kiril Simov, Alexander Simov, Milen Kouylekov BulTreeBank project http://www.bultreebank.org Linguistic Modelling Laboratory - CLPPI, Bulgarian Academy
Grammar Knowledge Transfer for Building RMRSs over Dependency Parses in Bulgarian
Grammar Knowledge Transfer for Building RMRSs over Dependency Parses in Bulgarian Kiril Simov and Petya Osenova Linguistic Modelling Department, IICT, Bulgarian Academy of Sciences DELPH-IN, Sofia, 2012
Refactored Moses. Hieu Hoang MT Marathon Prague August 2013
Refactored Moses Hieu Hoang MT Marathon Prague August 2013 What did you Refactor? Decoder Feature Func?on Framework moses.ini format Simplify class structure Deleted func?onality NOT decoding algorithms
KenLM: Faster and Smaller Language Model Queries
KenLM: Faster and Smaller Language Model Queries Kenneth heafield@cs.cmu.edu Carnegie Mellon July 30, 2011 kheafield.com/code/kenlm What KenLM Does Answer language model queries using less time and memory.
Stack- propaga+on: Improved Representa+on Learning for Syntax
Stack- propaga+on: Improved Representa+on Learning for Syntax Yuan Zhang, David Weiss MIT, Google 1 Transi+on- based Neural Network Parser p(action configuration) So1max Hidden Embedding words labels POS
Can Active Memory Replace Attention?
Google Brain NIPS 2016 Presenter: Chao Jiang NIPS 2016 Presenter: Chao Jiang 1 / Outline 1 Introduction 2 Active Memory 3 Step by Step to Neural GPU 4 Another two steps: 1. the Markovian Neural GPU 5 Another
NTT SMT System for IWSLT Katsuhito Sudoh, Taro Watanabe, Jun Suzuki, Hajime Tsukada, and Hideki Isozaki NTT Communication Science Labs.
NTT SMT System for IWSLT 2008 Katsuhito Sudoh, Taro Watanabe, Jun Suzuki, Hajime Tsukada, and Hideki Isozaki NTT Communication Science Labs., Japan Overview 2-stage translation system k-best translation
Natural Language Processing Pipelines to Annotate BioC Collections with an Application to the NCBI Disease Corpus
Natural Language Processing Pipelines to Annotate BioC Collections with an Application to the NCBI Disease Corpus Donald C. Comeau *, Haibin Liu, Rezarta Islamaj Doğan and W. John Wilbur National Center
Sparse Non-negative Matrix Language Modeling
Sparse Non-negative Matrix Language Modeling Joris Pelemans Noam Shazeer Ciprian Chelba joris@pelemans.be noam@google.com ciprianchelba@google.com 1 Outline Motivation Sparse Non-negative Matrix Language
Bayesian Networks Inference
Bayesian Networks Inference Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University November 5 th, 2007 2005-2007 Carlos Guestrin 1 General probabilistic inference Flu Allergy Query: Sinus
Lexicographic Semirings for Exact Automata Encoding of Sequence Models
Lexicographic Semirings for Exact Automata Encoding of Sequence Models Brian Roark, Richard Sproat, and Izhak Shafran {roark,rws,zak}@cslu.ogi.edu Abstract In this paper we introduce a novel use of the
Medical-domain Machine Translation in KConnect
Medical-domain Machine Translation in KConnect Pavel Pecina Charles University, Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics Czech Republic Apr 4th, 2017 QT21 workshop,
Recent Developments in the Czech National Corpus
Recent Developments in the Czech National Corpus Michal Křen Charles University in Prague 3 rd Workshop on the Challenges in the Management of Large Corpora Lancaster 20 July 2015 Introduction of the project
Predictive Analytics using Teradata Aster Scoring SDK
Predictive Analytics using Teradata Aster Scoring SDK Faraz Ahmad Software Engineer, Teradata #TDPARTNERS16 GEORGIA WORLD CONGRESS CENTER At Teradata, we believe. Analytics and data unleash the potential
Type-based MCMC for Sampling Tree Fragments from Forests
Type-based MCMC for Sampling Tree Fragments from Forests Xiaochang Peng and Daniel Gildea Department of Computer Science University of Rochester Rochester, NY 14627 Abstract This paper applies type-based
Conditional Random Fields. Mike Brodie CS 778
Conditional Random Fields Mike Brodie CS 778 Motivation Part-Of-Speech Tagger 2 Motivation object 3 Motivation I object! 4 Motivation object Do you see that object? 5 Motivation Part-Of-Speech Tagger -
Homework 2: Parsing and Machine Learning
Homework 2: Parsing and Machine Learning COMS W4705_001: Natural Language Processing Prof. Kathleen McKeown, Fall 2017 Due: Saturday, October 14th, 2017, 2:00 PM This assignment will consist of tasks in
EXPERIMENTAL SETUP: MOSES
EXPERIMENTAL SETUP: MOSES Moses is a statistical machine translation system that allows us to automatically train translation models for any language pair. All we need is a collection of translated texts
Forest-based Neural Machine Translation. Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Tiejun Zhao, EiichiroSumita
Forest-based Neural Machine Translation Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Tiejun Zhao, EiichiroSumita Motivation Key point: Syntactic Information To use or not to use? string-to-string model
A Quick Guide to MaltParser Optimization
A Quick Guide to MaltParser Optimization Joakim Nivre Johan Hall 1 Introduction MaltParser is a system for data-driven dependency parsing, which can be used to induce a parsing model from treebank data
A Framework for Processing Complex Document-centric XML with Overlapping Structures Ionut E. Iacob and Alex Dekhtyar
A Framework for Processing Complex Document-centric XML with Overlapping Structures Ionut E. Iacob and Alex Dekhtyar ABSTRACT Management of multihierarchical XML encodings has attracted attention of a
Easy-First POS Tagging and Dependency Parsing with Beam Search
Easy-First POS Tagging and Dependency Parsing with Beam Search Ji Ma JingboZhu Tong Xiao Nan Yang Natrual Language Processing Lab., Northeastern University, Shenyang, China MOE-MS Key Lab of MCC, University
Recurrent Neural Networks. Nand Kishore, Audrey Huang, Rohan Batra
Recurrent Neural Networks Nand Kishore, Audrey Huang, Rohan Batra Roadmap Issues Motivation 1 Application 1: Sequence Level Training 2 Basic Structure 3 4 Variations 5 Application 3: Image Classification