Incremental Integer Linear Programming for Non-projective Dependency Parsing
|
|
|
- Melvyn Hines
- 5 years ago
- Views:
Transcription
1 Incremental Integer Linear Programming for Non-projective Dependency Parsing Sebastian Riedel James Clarke ICCS, University of Edinburgh 22. July 2006 EMNLP 2006 S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
2 Motivation Labelled Dependency Parsing Labelled Dependency Parsing Find labelled head-child relations between tokens. S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
3 Motivation Labelled Dependency Parsing Labelled Dependency Parsing Find labelled head-child relations between tokens. Example S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
4 Motivation Non-projective Dependency Parsing Non-projective Dependency Parsing Dependencies are allowed to cross S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
5 Motivation Non-projective Dependency Parsing Non-projective Dependency Parsing Dependencies are allowed to cross Example S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
6 Motivation Non-projective Dependency Parsing Non-projective Dependency Parsing Dependencies are allowed to cross Example Methods Nivre et al. (2004) Yamada and Matsumoto (2003) McDonald et al. (2005) S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
7 Motivation McDonald et al. (2005) McDonald et al. (2005) State-of-the-art non-projective dependency parser. Based on nding the maximum spanning tree. Attachment decisions made independently. S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
8 Motivation McDonald et al. (2005) McDonald et al. (2005) State-of-the-art non-projective dependency parser. Based on nding the maximum spanning tree. Attachment decisions made independently. Example Mistake on Alpino Corpus S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
9 Motivation McDonald et al. (2005) McDonald et al. (2005) State-of-the-art non-projective dependency parser. Based on nding the maximum spanning tree. Attachment decisions made independently. Example Mistake on Alpino Corpus McDonald and Pereira, 2006 Second order scores Approximate search S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
10 Motivation More General Chu-Liu-Edmonds, CYK, Viterbi More local models Optimality guaranteed Polynomial runtime guaranteed S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
11 Motivation More General Chu-Liu-Edmonds, CYK, Viterbi More local models Optimality guaranteed Polynomial runtime guaranteed Beam Search, Sampling More global models Optimality not guaranteed Polynomial runtime guaranteed S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
12 Motivation More General Chu-Liu-Edmonds, CYK, Viterbi More local models Optimality guaranteed Polynomial runtime guaranteed Beam Search, Sampling More global models Optimality not guaranteed Polynomial runtime guaranteed Incremental ILP More global models Optimality guaranteed Polynomial runtime not guaranteed S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
13 Motivation Overview 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
14 Maximum Spanning Tree Problem Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
15 Maximum Spanning Tree Problem Maximum Spanning Tree Problem Example Graph with Scores MST Objective Find the tree with the maximum sum of scores S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
16 Maximum Spanning Tree Problem More Formal Score s(x, y) = s(i, j, l) = w f(i, j, l) (i,j,l) y (i,j,l) y for graph y S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
17 Maximum Spanning Tree Problem More Formal Score s(x, y) = s(i, j, l) = w f(i, j, l) (i,j,l) y (i,j,l) y for graph y Constraint (Exactly One Head) Exactly one head for each non-root token; no head for root S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
18 Maximum Spanning Tree Problem More Formal Score s(x, y) = s(i, j, l) = w f(i, j, l) (i,j,l) y (i,j,l) y for graph y Constraint (Exactly One Head) Exactly one head for each non-root token; no head for root Constraint (No Cycles) The dependency graph contains no cycles S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
19 Maximum Spanning Tree Problem More Formal Score s(x, y) = s(i, j, l) = w f(i, j, l) (i,j,l) y (i,j,l) y for graph y Constraint (Exactly One Head) Exactly one head for each non-root token; no head for root Constraint (No Cycles) The dependency graph contains no cycles MST Objective Maximise score under the two above constraints S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
20 Linguistic Constraints Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
21 Linguistic Constraints Linguistic Constraints Constraint Coordination arguments must be compatible Violated in S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
22 Linguistic Constraints Linguistic Constraints Constraint There must not be more than one subject for each verb Violated in S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
23 Linguistic Constraints Linguistic Constraints Constraint For each and coordination there is exactly one argument to the right and one more arguments to the left Violated in S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
24 Decoding Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
25 Decoding Decoding with Integer Linear Programming(ILP) Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
26 Decoding Decoding with Integer Linear Programming(ILP) Decoding Objective Maximise: s(x, y) = (i,j,l) y s(i, j, l) given dependency parsing constraints linguistic constraints S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
27 Decoding Decoding with Integer Linear Programming(ILP) Decoding Objective Maximise: s(x, y) = (i,j,l) y s(i, j, l) given dependency parsing constraints linguistic constraints Methods Use the Chu-Liu-Edmonds algorithm (McDonald et al., 2005) Use some approximate search (McDonald and Pereira, 2006) Use Integer Linear Programming (Roth and Yih, 2005) S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
28 Decoding Decoding with Integer Linear Programming(ILP) Decoding Objective Maximise: s(x, y) = (i,j,l) y s(i, j, l) given dependency parsing constraints linguistic constraints Methods Use the Chu-Liu-Edmonds algorithm (McDonald et al., 2005) Use some approximate search (McDonald and Pereira, 2006) Use Integer Linear Programming (Roth and Yih, 2005) S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
29 Decoding Decoding with Integer Linear Programming(ILP) Decoding Objective Maximise: s(x, y) = (i,j,l) y s(i, j, l) given dependency parsing constraints linguistic constraints Methods Use the Chu-Liu-Edmonds algorithm (McDonald et al., 2005) Use some approximate search (McDonald and Pereira, 2006) Use Integer Linear Programming (Roth and Yih, 2005) S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
30 Decoding Decoding with Integer Linear Programming(ILP) Integer Linear Programming (ILP) Decision Variables x 1, x 2, x 3 S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
31 Decoding Decoding with Integer Linear Programming(ILP) Integer Linear Programming (ILP) Decision Variables x 1, x 2, x 3 Objective Function 1.5x 1 + 2x 2 x 3 S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
32 Decoding Decoding with Integer Linear Programming(ILP) Integer Linear Programming (ILP) Decision Variables x 1, x 2, x 3 Objective Function 1.5x 1 + 2x 2 x 3 Linear Constraints x 1 + x 2 < 2 x 1 x 3 > 1 S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
33 Decoding Decoding with Integer Linear Programming(ILP) Integer Linear Programming (ILP) Decision Variables Linear Constraints x 1, x 2, x 3 x 1 + x 2 < 2 x 1 x 3 > 1 Objective Function 1.5x 1 + 2x 2 x 3 Integer Constraints x 1 {0, 1} S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
34 Decoding Decoding with Integer Linear Programming(ILP) Integer Linear Programming (ILP) Decision Variables Linear Constraints x 1, x 2, x 3 x 1 + x 2 < 2 x 1 x 3 > 1 Objective Function 1.5x 1 + 2x 2 x 3 Integer Constraints x 1 {0, 1} ILP Objective Maximise objective function under constraints. S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
35 Decoding Decoding with Integer Linear Programming(ILP) Integer Linear Programming (ILP) Decision Variables Linear Constraints x 1, x 2, x 3 x 1 + x 2 < 2 x 1 x 3 > 1 Objective Function 1.5x 1 + 2x 2 x 3 Integer Constraints x 1 {0, 1} ILP Objective Maximise objective function under constraints. Taskar 2004 Every Markov Network can be mapped to an polynomial-size ILP. S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
36 Decoding Decoding with Integer Linear Programming(ILP) Dependency Parsing with ILP Decision Variables { 1 if there is a dependency from i to j with label l e i,j,l = 0 otherwise for each token i, j and label l Objective Function s(i, j, l) e i,j,l i,j,l S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
37 Decoding Decoding with Integer Linear Programming(ILP) Dependency Parsing with ILP (2) Auxiliary Variables for each token i, j { 1 if there is a dependency from i to j d i,j = 0 otherwise S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
38 Decoding Decoding with Integer Linear Programming(ILP) Dependency Parsing with ILP (2) Auxiliary Variables for each token i, j { 1 if there is a dependency from i to j d i,j = 0 otherwise Only One Head d i,j = 1 for all j > 0. i S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
39 Decoding Decoding with Integer Linear Programming(ILP) Dependency Parsing with ILP (2) Auxiliary Variables for each token i, j { 1 if there is a dependency from i to j d i,j = 0 otherwise Only One Head d i,j = 1 for all j > 0. i No Cycles (i,j) G s d i,j s for possible subsets all sets s of tokens S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
40 Decoding Decoding with Integer Linear Programming(ILP) Dependency Parsing with ILP (2) Auxiliary Variables for each token i, j { 1 if there is a dependency from i to j d i,j = 0 otherwise Only One Head d i,j = 1 for all j > 0. i No Cycles (i,j) G s d i,j s for possible subsets all sets s of tokens S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
41 Decoding Decoding with Integer Linear Programming(ILP) Dependency Parsing with ILP (2) Auxiliary Variables for each token i, j { 1 if there is a dependency from i to j d i,j = 0 otherwise Only One Head d i,j = 1 for all j > 0. i No Cycles (i,j) G s d i,j s for possible subsets all sets s of tokens Germann et al. (2001) Same cycle problem in ILP formulation for MT S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
42 Decoding Incremental ILP Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
43 Decoding Incremental ILP Incremental Integer Linear Programming Setup base (e.g. exactly one head) constraints incremental (e.g. no cycles) constraints S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
44 Decoding Incremental ILP Incremental Integer Linear Programming Setup base (e.g. exactly one head) constraints incremental (e.g. no cycles) constraints Algorithm (see Warme (2002)) Set up ILP I with objective function and base constraints repeat Solve I Find violated incremental constraints Add constraints to I until No more constraints violated S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
45 Decoding Incremental ILP Incremental Integer Linear Programming Setup base (e.g. exactly one head) constraints incremental (e.g. no cycles) constraints Algorithm (see Warme (2002)) Set up ILP I with objective function and base constraints repeat Solve I Find violated incremental constraints Add constraints to I until No more constraints violated Tromble and Eisner (2006) Replace ILP with nite-state automata S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
46 Decoding Parsing Example Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
47 Decoding Parsing Example Example Sentence S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
48 Decoding Parsing Example First Solution S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
49 Decoding Parsing Example Add Violated Constraints Add Constraint d what,and + d and,get + d get,what < 3 S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
50 Decoding Parsing Example Next Solution S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
51 Decoding Parsing Example Add Violated Constraints Add Constraint d and,at + d and,get < 2 S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
52 Decoding Parsing Example Done S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
53 Training Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
54 Training Training Online Learning 1 single-best MIRA 2 Chu-Liu-Edmonds for parsing (McDonald et al. 2005) 3 No constraints S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
55 Training Training Online Learning 1 single-best MIRA 2 Chu-Liu-Edmonds for parsing (McDonald et al. 2005) 3 No constraints Roth and Yih (2005) Training without constraints can actually help S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
56 Experiments Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
57 Experiments Experiments Questions How accurate in comparison to McDonald et. al (2005)? How fast/slow? S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
58 Experiments Experiments Questions How accurate in comparison to McDonald et. al (2005)? How fast/slow? Data Dutch alpino corpus from the CoNLL shared task 2006 about Sentences, non-projective, 5% of edges crossing Split into development set and crossvalidation set Tuned feature and constraint set on dev set S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
59 Experiments Accuracy In Comparison with McDonald et. al (2005) Crossvalidation Labelled Unlabelled Complete Complete(U) McDonald % 88.9% 27.7% 42.2% Incremental ILP 85.1% 89.4% 29.7% 43.8% Statistical signicant (p < for Sign test and Dan Bikel's parse eval script) S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
60 Experiments Accuracy In Comparison with McDonald et. al (2005) Crossvalidation Labelled Unlabelled Complete Complete(U) McDonald % 88.9% 27.7% 42.2% Incremental ILP 85.1% 89.4% 29.7% 43.8% Statistical signicant (p < for Sign test and Dan Bikel's parse eval script) With Others Wins on CoNLL test set but not signicantly better than McDonald et al. (2006) Similar to performance of Malouf and van Noord (2004) (84.4%, smaller training set, evaluates control relations) S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
61 Experiments Runtime Evaluation Exact Inference reasonable fast (0.5s for sentences with length between tokens) signicantly slower than McDonald et al. (2005) (3ms!) 150 times slower when parsing the full corpus (50min vs 20s) without feature extraction 6 times slower with feature extraction (add 10 minutes) 2 times slower with nearly loss less approximation method (see paper) S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
62 Conclusion Outline 1 Maximum Spanning Tree Problem 2 Linguistic Constraints 3 Decoding Decoding with Integer Linear Programming(ILP) Incremental ILP Parsing Example 4 Training 5 Experiments 6 Conclusion S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
63 Conclusion Conclusion Accuracy Signicantly better than McDonald et. al (2005) headroom left - rule engineering S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
64 Conclusion Conclusion Accuracy Signicantly better than McDonald et. al (2005) headroom left - rule engineering Runtime Signicantly slower than McDonald et. al (2005) Feature extraction dominates runtime almost loss-less approximation available S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
65 Conclusion Conclusion Accuracy Signicantly better than McDonald et. al (2005) headroom left - rule engineering Runtime Signicantly slower than McDonald et. al (2005) Feature extraction dominates runtime almost loss-less approximation available In General Allows global models Guarantees optimality No polynomial runtime guarantee Good scores - fast processing S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
66 Conclusion Future Work Parsing 2nd order features Evaluate on more languages Joint POS tagging and parsing Joint constituent and dependency parsing General Other applications(collective IE, MT?) Generalise to features/potentials (as opposed to constraints) Theoretical runtime estimates S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
67 Conclusion Thank you S. Riedel, J. Clarke (ICCS, Edinburgh) EMNLP / 38
Incremental Integer Linear Programming for Non-projective Dependency Parsing
Incremental Integer Linear Programming for Non-projective Dependency Parsing Sebastian Riedel and James Clarke School of Informatics, University of Edinburgh 2 Bucclecuch Place, Edinburgh EH8 9LW, UK s.r.riedel@sms.ed.ac.uk,
Graph-Based Parsing. Miguel Ballesteros. Algorithms for NLP Course. 7-11
Graph-Based Parsing Miguel Ballesteros Algorithms for NLP Course. 7-11 By using some Joakim Nivre's materials from Uppsala University and Jason Eisner's material from Johns Hopkins University. Outline
Tekniker för storskalig parsning: Dependensparsning 2
Tekniker för storskalig parsning: Dependensparsning 2 Joakim Nivre Uppsala Universitet Institutionen för lingvistik och filologi joakim.nivre@lingfil.uu.se Dependensparsning 2 1(45) Data-Driven Dependency
Agenda for today. Homework questions, issues? Non-projective dependencies Spanning tree algorithm for non-projective parsing
Agenda for today Homework questions, issues? Non-projective dependencies Spanning tree algorithm for non-projective parsing 1 Projective vs non-projective dependencies If we extract dependencies from trees,
Dependency Parsing 2 CMSC 723 / LING 723 / INST 725. Marine Carpuat. Fig credits: Joakim Nivre, Dan Jurafsky & James Martin
Dependency Parsing 2 CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing
Parsing with Dynamic Programming
CS11-747 Neural Networks for NLP Parsing with Dynamic Programming Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two Types of Linguistic Structure Dependency: focus on relations between words
Online Learning of Approximate Dependency Parsing Algorithms
Online Learning of Approximate Dependency Parsing Algorithms Ryan McDonald Fernando Pereira Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104 {ryantm,pereira}@cis.upenn.edu
Online Graph Planarisation for Synchronous Parsing of Semantic and Syntactic Dependencies
Online Graph Planarisation for Synchronous Parsing of Semantic and Syntactic Dependencies Ivan Titov University of Illinois at Urbana-Champaign James Henderson, Paola Merlo, Gabriele Musillo University
Dependency Parsing CMSC 723 / LING 723 / INST 725. Marine Carpuat. Fig credits: Joakim Nivre, Dan Jurafsky & James Martin
Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing
Fast(er) Exact Decoding and Global Training for Transition-Based Dependency Parsing via a Minimal Feature Set
Fast(er) Exact Decoding and Global Training for Transition-Based Dependency Parsing via a Minimal Feature Set Tianze Shi* Liang Huang Lillian Lee* * Cornell University Oregon State University O n 3 O n
A Study of Global Inference Algorithms in Multi-Document Summarization
A Study of Global Inference Algorithms in Multi-Document Summarization Ryan McDonald Google Research 76 Ninth Avenue, New York, NY 10011 ryanmcd@google.com Abstract. In this work we study the theoretical
Spanning Tree Methods for Discriminative Training of Dependency Parsers
University of Pennsylvania ScholarlyCommons Technical Reports (CIS) Department of Computer & Information Science January 2006 Spanning Tree Methods for Discriminative Training of Dependency Parsers Ryan
Concise Integer Linear Programming Formulations for Dependency Parsing
Concise Integer Linear Programming Formulations for Dependency Parsing André F. T. Martins Noah A. Smith Eric P. Xing School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA Instituto
Dependency grammar and dependency parsing
Dependency grammar and dependency parsing Syntactic analysis (5LN455) 2015-12-09 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Activities - dependency parsing
Dynamic Feature Selection for Dependency Parsing
Dynamic Feature Selection for Dependency Parsing He He, Hal Daumé III and Jason Eisner EMNLP 2013, Seattle Structured Prediction in NLP Part-of-Speech Tagging Parsing N N V Det N Fruit flies like a banana
Introduction to Data-Driven Dependency Parsing
Introduction to Data-Driven Dependency Parsing Introductory Course, ESSLLI 2007 Ryan McDonald 1 Joakim Nivre 2 1 Google Inc., New York, USA E-mail: ryanmcd@google.com 2 Uppsala University and Växjö University,
CS395T Project 2: Shift-Reduce Parsing
CS395T Project 2: Shift-Reduce Parsing Due date: Tuesday, October 17 at 9:30am In this project you ll implement a shift-reduce parser. First you ll implement a greedy model, then you ll extend that model
A Quick Guide to MaltParser Optimization
A Quick Guide to MaltParser Optimization Joakim Nivre Johan Hall 1 Introduction MaltParser is a system for data-driven dependency parsing, which can be used to induce a parsing model from treebank data
Non-projective Dependency Parsing using Spanning Tree Algorithms
Non-projective Dependency Parsing using Spanning Tree Algorithms Ryan McDonald Fernando Pereira Department of Computer and Information Science University of Pennsylvania {ryantm,pereira}@cis.upenn.edu
Projective Dependency Parsing with Perceptron
Projective Dependency Parsing with Perceptron Xavier Carreras, Mihai Surdeanu, and Lluís Màrquez Technical University of Catalonia {carreras,surdeanu,lluism}@lsi.upc.edu 8th June 2006 Outline Introduction
Parse, Price and Cut Delayed Column and Row Generation for Graph Based Parsers
Parse, Price and Cut Delayed Column and Row Generation for Graph Based Parsers Sebastian Riedel David Smith Andrew McCallum Department of Computer Science University of Massachusetts, Amherst {riedel,dasmith,mccallum}@cs.umass.edu
Collins and Eisner s algorithms
Collins and Eisner s algorithms Syntactic analysis (5LN455) 2015-12-14 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Recap: Dependency trees dobj subj det pmod
Easy-First POS Tagging and Dependency Parsing with Beam Search
Easy-First POS Tagging and Dependency Parsing with Beam Search Ji Ma JingboZhu Tong Xiao Nan Yang Natrual Language Processing Lab., Northeastern University, Shenyang, China MOE-MS Key Lab of MCC, University
Dependency grammar and dependency parsing
Dependency grammar and dependency parsing Syntactic analysis (5LN455) 2014-12-10 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Mid-course evaluation Mostly positive
Dependency grammar and dependency parsing
Dependency grammar and dependency parsing Syntactic analysis (5LN455) 2016-12-05 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Activities - dependency parsing
Generalized Higher-Order Dependency Parsing with Cube Pruning
Generalized Higher-Order Dependency Parsing with Cube Pruning Hao Zhang Ryan McDonald Google, Inc. {haozhang,ryanmcd}@google.com Abstract State-of-the-art graph-based parsers use features over higher-order
An Empirical Study of Semi-supervised Structured Conditional Models for Dependency Parsing
An Empirical Study of Semi-supervised Structured Conditional Models for Dependency Parsing Jun Suzuki, Hideki Isozaki NTT CS Lab., NTT Corp. Kyoto, 619-0237, Japan jun@cslab.kecl.ntt.co.jp isozaki@cslab.kecl.ntt.co.jp
Hybrid Combination of Constituency and Dependency Trees into an Ensemble Dependency Parser
Hybrid Combination of Constituency and Dependency Trees into an Ensemble Dependency Parser Nathan David Green and Zdeněk Žabokrtský Charles University in Prague Institute of Formal and Applied Linguistics
Klein & Manning, NIPS 2002
Agenda for today Factoring complex model into product of simpler models Klein & Manning factored model: dependencies and constituents Dual decomposition for higher-order dependency parsing Refresh memory
Flexible Text Segmentation with Structured Multilabel Classification
Flexible Text Segmentation with Structured Multilabel Classification Ryan McDonald Koby Crammer Fernando Pereira Department of Computer and Information Science University of Pennsylvania Philadelphia,
Utilizing Dependency Language Models for Graph-based Dependency Parsing Models
Utilizing Dependency Language Models for Graph-based Dependency Parsing Models Wenliang Chen, Min Zhang, and Haizhou Li Human Language Technology, Institute for Infocomm Research, Singapore {wechen, mzhang,
Backpropagating through Structured Argmax using a SPIGOT
Backpropagating through Structured Argmax using a SPIGOT Hao Peng, Sam Thomson, Noah A. Smith @ACL July 17, 2018 Overview arg max Parser Downstream task Loss L Overview arg max Parser Downstream task Head
Density-Driven Cross-Lingual Transfer of Dependency Parsers
Density-Driven Cross-Lingual Transfer of Dependency Parsers Mohammad Sadegh Rasooli Michael Collins rasooli@cs.columbia.edu Presented by Owen Rambow EMNLP 2015 Motivation Availability of treebanks Accurate
Parsing partially bracketed input
Parsing partially bracketed input Martijn Wieling, Mark-Jan Nederhof and Gertjan van Noord Humanities Computing, University of Groningen Abstract A method is proposed to convert a Context Free Grammar
Transition-Based Dependency Parsing with Stack Long Short-Term Memory
Transition-Based Dependency Parsing with Stack Long Short-Term Memory Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, Noah A. Smith Association for Computational Linguistics (ACL), 2015 Presented
Advanced Search Algorithms
CS11-747 Neural Networks for NLP Advanced Search Algorithms Daniel Clothiaux https://phontron.com/class/nn4nlp2017/ Why search? So far, decoding has mostly been greedy Chose the most likely output from
Dependency Parsing. Ganesh Bhosale Neelamadhav G Nilesh Bhosale Pranav Jawale under the guidance of
Dependency Parsing Ganesh Bhosale - 09305034 Neelamadhav G. - 09305045 Nilesh Bhosale - 09305070 Pranav Jawale - 09307606 under the guidance of Prof. Pushpak Bhattacharyya Department of Computer Science
Structured Prediction Models via the Matrix-Tree Theorem
Structured Prediction Models via the Matrix-Tree Theorem Terry Koo, Amir Globerson, Xavier Carreras and Michael Collins MIT CSAIL, Cambridge, MA 02139, USA {maestro,gamir,carreras,mcollins}@csail.mit.edu
Natural Language Dependency Parsing. SPFLODD October 13, 2011
Natural Language Dependency Parsing SPFLODD October 13, 2011 Quick Review from Tuesday What are CFGs? PCFGs? Weighted CFGs? StaKsKcal parsing using the Penn Treebank What it looks like How to build a simple
Refresher on Dependency Syntax and the Nivre Algorithm
Refresher on Dependency yntax and Nivre Algorithm Richard Johansson 1 Introduction This document gives more details about some important topics that re discussed very quickly during lecture: dependency
Undirected Dependency Parsing
Computational Intelligence, Volume 59, Number 000, 2010 Undirected Dependency Parsing CARLOS GÓMEZ-RODRÍGUEZ cgomezr@udc.es Depto. de Computación, Universidade da Coruña. Facultad de Informática, Campus
Topics in Parsing: Context and Markovization; Dependency Parsing. COMP-599 Oct 17, 2016
Topics in Parsing: Context and Markovization; Dependency Parsing COMP-599 Oct 17, 2016 Outline Review Incorporating context Markovization Learning the context Dependency parsing Eisner s algorithm 2 Review
Guiding Semi-Supervision with Constraint-Driven Learning
Guiding Semi-Supervision with Constraint-Driven Learning Ming-Wei Chang 1 Lev Ratinov 2 Dan Roth 3 1 Department of Computer Science University of Illinois at Urbana-Champaign Paper presentation by: Drew
A New Perceptron Algorithm for Sequence Labeling with Non-local Features
A New Perceptron Algorithm for Sequence Labeling with Non-local Features Jun ichi Kazama and Kentaro Torisawa Japan Advanced Institute of Science and Technology (JAIST) Asahidai 1-1, Nomi, Ishikawa, 923-1292
Transition-Based Dependency Parsing with MaltParser
Transition-Based Dependency Parsing with MaltParser Joakim Nivre Uppsala University and Växjö University Transition-Based Dependency Parsing 1(13) Introduction Outline Goals of the workshop Transition-based
Transition-based Dependency Parsing with Rich Non-local Features
Transition-based Dependency Parsing with Rich Non-local Features Yue Zhang University of Cambridge Computer Laboratory yue.zhang@cl.cam.ac.uk Joakim Nivre Uppsala University Department of Linguistics and
Turning on the Turbo: Fast Third-Order Non- Projective Turbo Parsers
Carnegie Mellon University Research Showcase @ CMU Language Technologies Institute School of Computer Science 8-2013 Turning on the Turbo: Fast Third-Order Non- Projective Turbo Parsers Andre F.T. Martins
Learning with Probabilistic Features for Improved Pipeline Models
Learning with Probabilistic Features for Improved Pipeline Models Razvan C. Bunescu School of EECS Ohio University Athens, OH 45701 bunescu@ohio.edu Abstract We present a novel learning framework for pipeline
Bidirectional LTAG Dependency Parsing
Bidirectional LTAG Dependency Parsing Libin Shen BBN Technologies Aravind K. Joshi University of Pennsylvania We propose a novel algorithm for bi-directional parsing with linear computational complexity,
Optimal Incremental Parsing via Best-First Dynamic Programming
Optimal Incremental Parsing via Best-First Dynamic Programming Kai Zhao 1 James Cross 1 1 Graduate Center City University of New York 365 Fifth Avenue, New York, NY 10016 {kzhao,jcross}@gc.cuny.edu Liang
Large-Scale Syntactic Processing: Parsing the Web. JHU 2009 Summer Research Workshop
Large-Scale Syntactic Processing: JHU 2009 Summer Research Workshop Intro CCG parser Tasks 2 The Team Stephen Clark (Cambridge, UK) Ann Copestake (Cambridge, UK) James Curran (Sydney, Australia) Byung-Gyu
Non-Projective Dependency Parsing in Expected Linear Time
Non-Projective Dependency Parsing in Expected Linear Time Joakim Nivre Uppsala University, Department of Linguistics and Philology, SE-75126 Uppsala Växjö University, School of Mathematics and Systems
Discriminative Training with Perceptron Algorithm for POS Tagging Task
Discriminative Training with Perceptron Algorithm for POS Tagging Task Mahsa Yarmohammadi Center for Spoken Language Understanding Oregon Health & Science University Portland, Oregon yarmoham@ohsu.edu
Transition-based dependency parsing
Transition-based dependency parsing Syntactic analysis (5LN455) 2014-12-18 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Overview Arc-factored dependency parsing
Dependency Parsing. Johan Aulin D03 Department of Computer Science Lund University, Sweden
Dependency Parsing Johan Aulin D03 Department of Computer Science Lund University, Sweden d03jau@student.lth.se Carl-Ola Boketoft ID03 Department of Computing Science Umeå University, Sweden calle_boketoft@hotmail.com
Dependency Parsing. Allan Jie. February 20, Slides: Allan Jie Dependency Parsing February 20, / 16
Dependency Parsing Allan Jie February 20, 2016 Slides: http://www.statnlp.org/dp.html Allan Jie Dependency Parsing February 20, 2016 1 / 16 Table of Contents 1 Dependency Labeled/Unlabeled Dependency Projective/Non-projective
Forest-Based Search Algorithms
Forest-Based Search Algorithms for Parsing and Machine Translation Liang Huang University of Pennsylvania Google Research, March 14th, 2008 Search in NLP is not trivial! I saw her duck. Aravind Joshi 2
Assignment 4 CSE 517: Natural Language Processing
Assignment 4 CSE 517: Natural Language Processing University of Washington Winter 2016 Due: March 2, 2016, 1:30 pm 1 HMMs and PCFGs Here s the definition of a PCFG given in class on 2/17: A finite set
Support Vector Machine Learning for Interdependent and Structured Output Spaces
Support Vector Machine Learning for Interdependent and Structured Output Spaces I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun, ICML, 2004. And also I. Tsochantaridis, T. Joachims, T. Hofmann,
COMP 261 ALGORITHMS and DATA STRUCTURES
T E W H A R E W Ā N A N G A O T E Ū P O K O O T E I K A A M Ā U I VUW V I C T O R I A UNIVERSITY OF WELLINGTON Student ID:....................... EXAMINATIONS 2012 TRIMESTER 2 *** WITH SOLUTIONS *** COMP
JOINT INTENT DETECTION AND SLOT FILLING USING CONVOLUTIONAL NEURAL NETWORKS. Puyang Xu, Ruhi Sarikaya. Microsoft Corporation
JOINT INTENT DETECTION AND SLOT FILLING USING CONVOLUTIONAL NEURAL NETWORKS Puyang Xu, Ruhi Sarikaya Microsoft Corporation ABSTRACT We describe a joint model for intent detection and slot filling based
Easy-First Chinese POS Tagging and Dependency Parsing
Easy-First Chinese POS Tagging and Dependency Parsing ABSTRACT Ji Ma, Tong Xiao, Jingbo Zhu, Feiliang Ren Natural Language Processing Laboratory Northeastern University, China majineu@outlook.com, zhujingbo@mail.neu.edu.cn,
Transition-based Parsing with Neural Nets
CS11-747 Neural Networks for NLP Transition-based Parsing with Neural Nets Graham Neubig Site https://phontron.com/class/nn4nlp2017/ Two Types of Linguistic Structure Dependency: focus on relations between
Dependency Parsing by Belief Propagation
Dependency Parsing by Belief Propagation David A. Smith and Jason Eisner Dept. of Computer Science, Johns Hopkins University Baltimore, MD 21218, USA {dasmith,eisner}@jhu.edu Abstract We formulate dependency
CS224n: Natural Language Processing with Deep Learning 1 Lecture Notes: Part IV Dependency Parsing 2 Winter 2019
CS224n: Natural Language Processing with Deep Learning 1 Lecture Notes: Part IV Dependency Parsing 2 Winter 2019 1 Course Instructors: Christopher Manning, Richard Socher 2 Authors: Lisa Wang, Juhi Naik,
Homework 2: Parsing and Machine Learning
Homework 2: Parsing and Machine Learning COMS W4705_001: Natural Language Processing Prof. Kathleen McKeown, Fall 2017 Due: Saturday, October 14th, 2017, 2:00 PM This assignment will consist of tasks in
Transition-Based Parsing of the Chinese Treebank using a Global Discriminative Model
Transition-Based Parsing of the Chinese Treebank using a Global Discriminative Model Yue Zhang Oxford University Computing Laboratory yue.zhang@comlab.ox.ac.uk Stephen Clark Cambridge University Computer
A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing
A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing Alexander M. Rush 1,2 Michael Collins 2 SRUSH@CSAIL.MIT.EDU MCOLLINS@CS.COLUMBIA.EDU 1 Computer Science
Let s get parsing! Each component processes the Doc object, then passes it on. doc.is_parsed attribute checks whether a Doc object has been parsed
Let s get parsing! SpaCy default model includes tagger, parser and entity recognizer nlp = spacy.load('en ) tells spacy to use "en" with ["tagger", "parser", "ner"] Each component processes the Doc object,
A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing
A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing The MIT Faculty has made this article openly available. Please share how this access benefits you.
Dependency Parsing domain adaptation using transductive SVM
Dependency Parsing domain adaptation using transductive SVM Antonio Valerio Miceli-Barone University of Pisa, Italy / Largo B. Pontecorvo, 3, Pisa, Italy miceli@di.unipi.it Giuseppe Attardi University
Learning Latent Linguistic Structure to Optimize End Tasks. David A. Smith with Jason Naradowsky and Xiaoye Tiger Wu
Learning Latent Linguistic Structure to Optimize End Tasks David A. Smith with Jason Naradowsky and Xiaoye Tiger Wu 12 October 2012 Learning Latent Linguistic Structure to Optimize End Tasks David A. Smith
Natural Language Processing. SoSe Question Answering
Natural Language Processing SoSe 2017 Question Answering Dr. Mariana Neves July 5th, 2017 Motivation Find small segments of text which answer users questions (http://start.csail.mit.edu/) 2 3 Motivation
A Transition-Based Dependency Parser Using a Dynamic Parsing Strategy
A Transition-Based Dependency Parser Using a Dynamic Parsing Strategy Francesco Sartorio Department of Information Engineering University of Padua, Italy sartorio@dei.unipd.it Giorgio Satta Department
INF5820/INF9820 LANGUAGE TECHNOLOGICAL APPLICATIONS. Jan Tore Lønning, Lecture 8, 12 Oct
1 INF5820/INF9820 LANGUAGE TECHNOLOGICAL APPLICATIONS Jan Tore Lønning, Lecture 8, 12 Oct. 2016 jtl@ifi.uio.no Today 2 Preparing bitext Parameter tuning Reranking Some linguistic issues STMT so far 3 We
Reducing Over-generation Errors for Automatic Keyphrase Extraction using Integer Linear Programming
Reducing Over-generation Errors for Automatic Keyphrase Extraction using Integer Linear Programming Florian Boudin LINA - UMR CNRS 6241, Université de Nantes, France Keyphrase 2015 1 / 22 Errors made by
Comparing Dropout Nets to Sum-Product Networks for Predicting Molecular Activity
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
Shallow Parsing Swapnil Chaudhari 11305R011 Ankur Aher Raj Dabre 11305R001
Shallow Parsing Swapnil Chaudhari 11305R011 Ankur Aher - 113059006 Raj Dabre 11305R001 Purpose of the Seminar To emphasize on the need for Shallow Parsing. To impart basic information about techniques
K-best Parsing Algorithms
K-best Parsing Algorithms Liang Huang University of Pennsylvania joint work with David Chiang (USC Information Sciences Institute) k-best Parsing Liang Huang (Penn) k-best parsing 2 k-best Parsing I saw
Dynamic Programming for Higher Order Parsing of Gap-Minding Trees
Dynamic Programming for Higher Order Parsing of Gap-Minding Trees Emily Pitler, Sampath Kannan, Mitchell Marcus Computer and Information Science University of Pennsylvania Philadelphia, PA 19104 epitler,kannan,mitch@seas.upenn.edu
Statistical parsing. Fei Xia Feb 27, 2009 CSE 590A
Statistical parsing Fei Xia Feb 27, 2009 CSE 590A Statistical parsing History-based models (1995-2000) Recent development (2000-present): Supervised learning: reranking and label splitting Semi-supervised
Approximate Large Margin Methods for Structured Prediction
: Approximate Large Margin Methods for Structured Prediction Hal Daumé III and Daniel Marcu Information Sciences Institute University of Southern California {hdaume,marcu}@isi.edu Slide 1 Structured Prediction
Noisy Or-based model for Relation Extraction using Distant Supervision
Noisy Or-based model for Relation Extraction using Distant Supervision Ajay Nagesh 1,2,3 1 IITB-Monash Research Academy ajaynagesh@cse.iitb.ac.in Gholamreza Haffari 2 Faculty of IT, Monash University gholamreza.haffari@monash.edu
Complex Prediction Problems
Problems A novel approach to multiple Structured Output Prediction Max-Planck Institute ECML HLIE08 Information Extraction Extract structured information from unstructured data Typical subtasks Named Entity
A Class of Submodular Functions for Document Summarization
A Class of Submodular Functions for Document Summarization Hui Lin, Jeff Bilmes University of Washington, Seattle Dept. of Electrical Engineering June 20, 2011 Lin and Bilmes Submodular Summarization June
Computationally Efficient M-Estimation of Log-Linear Structure Models
Computationally Efficient M-Estimation of Log-Linear Structure Models Noah Smith, Doug Vail, and John Lafferty School of Computer Science Carnegie Mellon University {nasmith,dvail2,lafferty}@cs.cmu.edu
NLP in practice, an example: Semantic Role Labeling
NLP in practice, an example: Semantic Role Labeling Anders Björkelund Lund University, Dept. of Computer Science anders.bjorkelund@cs.lth.se October 15, 2010 Anders Björkelund NLP in practice, an example:
Basic Parsing with Context-Free Grammars. Some slides adapted from Karl Stratos and from Chris Manning
Basic Parsing with Context-Free Grammars Some slides adapted from Karl Stratos and from Chris Manning 1 Announcements HW 2 out Midterm on 10/19 (see website). Sample ques>ons will be provided. Sign up
On maximum spanning DAG algorithms for semantic DAG parsing
On maximum spanning DAG algorithms for semantic DAG parsing Natalie Schluter Department of Computer Science School of Technology, Malmö University Malmö, Sweden natalie.schluter@mah.se Abstract Consideration
Integer Linear Programming
Integer Linear Programming Micha Elsner April 5, 2017 2 Integer linear programming A framework for inference: Reading: Clarke and Lapata 2008 Global Inference for Sentence Compression An Integer Linear
STRUCTURES AND STRATEGIES FOR STATE SPACE SEARCH
Slide 3.1 3 STRUCTURES AND STRATEGIES FOR STATE SPACE SEARCH 3.0 Introduction 3.1 Graph Theory 3.2 Strategies for State Space Search 3.3 Using the State Space to Represent Reasoning with the Predicate
arxiv: v4 [cs.cl] 7 Jun 2016
![arxiv: v4 [cs.cl] 7 Jun 2016](/thumbs/74/71062823.jpg "arxiv: v4 [cs.cl] 7 Jun 2016")
Edge-Linear First-Order Dependency Parsing with Undirected Minimum Spanning Tree Inference Effi Levi 1 Roi Reichart 2 Ari Rappoport 1 1 Institute of Computer Science, The Hebrew Univeristy 2 Faculty of
Motivation: Shortcomings of Hidden Markov Model. Ko, Youngjoong. Solution: Maximum Entropy Markov Model (MEMM)
Motivation: Shortcomings of Hidden Markov Model Maximum Entropy Markov Models and Conditional Random Fields Ko, Youngjoong Dept. of Computer Engineering, Dong-A University Intelligent System Laboratory,
Exam Marco Kuhlmann. This exam consists of three parts:
TDDE09, 729A27 Natural Language Processing (2017) Exam 2017-03-13 Marco Kuhlmann This exam consists of three parts: 1. Part A consists of 5 items, each worth 3 points. These items test your understanding
V Advanced Data Structures
V Advanced Data Structures B-Trees Fibonacci Heaps 18 B-Trees B-trees are similar to RBTs, but they are better at minimizing disk I/O operations Many database systems use B-trees, or variants of them,
SOLVING LARGE CARPOOLING PROBLEMS USING GRAPH THEORETIC TOOLS
July, 2014 1 SOLVING LARGE CARPOOLING PROBLEMS USING GRAPH THEORETIC TOOLS Irith Ben-Arroyo Hartman Datasim project - (joint work with Abed Abu dbai, Elad Cohen, Daniel Keren) University of Haifa, Israel
Online Service for Polish Dependency Parsing and Results Visualisation
Online Service for Polish Dependency Parsing and Results Visualisation Alina Wróblewska and Piotr Sikora Institute of Computer Science, Polish Academy of Sciences, Warsaw, Poland alina@ipipan.waw.pl,piotr.sikora@student.uw.edu.pl
Comparisons of Sequence Labeling Algorithms and Extensions
Nam Nguyen Yunsong Guo Department of Computer Science, Cornell University, Ithaca, NY 14853, USA NHNGUYEN@CS.CORNELL.EDU GUOYS@CS.CORNELL.EDU Abstract In this paper, we survey the current state-ofart models
Principles of Programming Languages. Lecture Outline
Principles of Programming Languages CS 492 Lecture 1 Based on Notes by William Albritton 1 Lecture Outline Reasons for studying concepts of programming languages Programming domains Language evaluation
Efficient Parsing for Head-Split Dependency Trees
Efficient Parsing for Head-Split Dependency Trees Giorgio Satta Dept. of Information Engineering University of Padua, Italy satta@dei.unipd.it Marco Kuhlmann Dept. of Linguistics and Philology Uppsala