Corpus Linguistics. Seminar Resources for Computational Linguists SS Magdalena Wolska & Michaela Regneri
|
|
- Iris Evans
- 6 years ago
- Views:
Transcription
1 Seminar Resources for Computational Linguists SS 2007 Magdalena Wolska & Michaela Regneri
2 Armchair Linguists vs. Corpus Linguists Competence Performance 2
3 Motivation (for ) 3
4 Outline Corpora Annotation Data Analysis The Web as Corpus 4
5 Outline Corpora Annotation Data Analysis The Web as Corpus 5
6 Corpus - definition in principle: every collection of text (desired or necessary) properties of corpora for linguistic processing: representativeness finite size (mostly) machine-readability standard reference 6
7 Corpus - properties language mode (speech vs. text) languages and alignment: mono-/bilingual, comparable/parallel balance: homo-/heterogeneous, balanced/unbalanced annotation: plain/annotated, annotation type and depth text types (newspapers, novels, phone calls...) text domains (business, finance, love stories...) date / time span (of texts used) size 7
8 Outline Corpora Annotation Data Analysis The Web as Corpus 8
9 Annotation - principles linguistic information in a corpus maxims of annotation (Leech 1993): removable and extractable annotation guidelines available to end user awareness of fallibility (but potential usefulness) scheme should be based on widely-agreed principles which are theory-neutral 9
10 Annotation - Data Format Often variants of XML: stand-off: The dog barks. <sentence> <phrase type= NP > <word ind= 1 pos= det /> <word ind= 2 pos= N /> </phrase> <phrase type= VP > <word ind= 3 pos= VI /> </phrase> </sentence> inline: <sentence> <phrase type= NP > <word pos= det >the</word> <word pos= N >dog</word> </phrase> <phrase type = VP > <word pos= VI >barks</word> </phrase> </sentence> 10
11 Annotation - examples: Treebanks (syntax) 11
12 Annotation - examples: semantic roles (SALSA) 12
13 Annotation - examples: discourse structure 13
14 Annotation - Tools Graphical UIs, similar to output, for drawing annotations Example: RSTTool 14
15 Outline Corpora Annotation Data Analysis The Web as Corpus 15
16 Data Analysis Word counts (word frequency, token per type ) concordance: same word in different contexts La Streisand sounded just like the student activist she played in the film T s pilot's wings, he was judged top student. After his weapons training, he w with Der Bettelstudent (The Beggar Student) and Gasparone in the fairly rece S.LOWRY: THE MAN AND HIS ART: As a student and long-time resident of Salford Antony Fleat, a second-year law student at Oxford Brookes University; and t oung life. This second-year student at Robert Gordon's university in Ab erdeen, having matriculated as a student at Robert Gordon's university. In M, 78, from Harrow, an anthropology student at the University of the Third Ag he had enough of London as a law student at University College and the Colle n-grams: count the frequencies of word combinations of n words 3868 vergehen Jahr 1184 kommen Jahr 2385 neu Land 1181 jung Mann 2378 letzt Jahr 1107 groß Teil 2296 nah Jahr 997 lang Zeit 1398 erst Mal 986 nah Woche 16
17 Data Analysis - Information Access pattern matching with query languages like CQP: Query: [lemma="dog"] [pos!="\$.*"]* [pos="nn"] within s; Examples: dog for her daughter dogs on the street dogs and their leashes dog with a cruel owner 17
18 Outline Corpora Annotation Data Analysis The Web as Corpus 18
19 The web as corpus the web is a collection of text, thus it is a corpus the largest available corpus: more than 7.2*10 11 words (10 times bigger than the English Gigaword Corpus [Liu and Curran 2006]) nearly all kinds of text and lots of languages present not preprocessed, lots of ungrammatical (and linguistically useless) text how to access it? 19
20 The web as corpus Document counts are shown to correlate directly with real frequencies (Keller 2003), so search engines can help - but... lots of repetitions of the same text (not representative) very limited query precision (no upper/lower case, no punctuation...) only estimated counts, often hart to reproduce exactly how to access Google? :) (Google API, Scripts) Alexa: buy (parts of) web, and process it on their machines 20
21 The web as corpus - examples Extracting and filtering web documents to create linguistically annotated corpora (Kilgarriff 2006) gather documents for different topics (balance!) exclude documents which cannot be preprocessed with available tools (here taggers and lemmatizers) exclude documents which seem irrelevant for a corpus (too short or too long, word lists,...) do this for several languages and make the corpora available 21
22 The web as corpus - examples Directly using web counts (instead of corpus counts), e.g. VerbOcean (Chklovski & Pantel 2004, see ) gather verb pairs which are semantically related but the relation is unknown --> DIRT (Lin and Pantel 2001) example pair: love -- marry pick a semantic relation (e.g. happens-before ) and design typical patterns for this relation (e.g. to X and then Y ) instantiate the patterns ( to love and then marry ) and count Google hits (here: 6) estimate whether or not the number of hits indicates a significant correlation, then assign the relation (or not) 22
23 References Thanks to Sabine Schulte im Walde & Magdalena Wolska for some slides Literature: McEnery & Wilson (1996):. Edinburgh University Press. (See Chklovski & Pantel (2004): VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations. In Proceedings of EMNLP-04. Keller (2003): Using the Web to Obtain Frequencies for Unseen Bigrams. Computational Linguistics, , Nr. 3, Baroni and Kilgarriff (2006): Large linguistically-processed Web Corpora for multiple languages. In Proceedings of EACL Leech (1993): Corpus annotation schemes. Literary and Linguistic Computing 8(4): Lin and Pantel (2001): DIRT Discovery of Inference Rules from Text. In Proceedings of KDD-01. Liu and Curran (2006): Web Text Corpus for Natural Language Processing. In Proceedings of EACL
24 References Some Corpora: Brown: LOB: BNC: (online search: TIGER: Penn Treebank: Penn Discourse Treebank: Prague Dependency Treebank: - Michaela Regneri 24
HG2052 Language, Technology and the Internet. The Web as Corpus
HG2052 Language, Technology and the Internet The Web as Corpus Francis Bond Division of Linguistics and Multilingual Studies http://www3.ntu.edu.sg/home/fcbond/ bond@ieee.org Lecture 7 Location: S3.2-B3-06
More informationFinal Project Discussion. Adam Meyers Montclair State University
Final Project Discussion Adam Meyers Montclair State University Summary Project Timeline Project Format Details/Examples for Different Project Types Linguistic Resource Projects: Annotation, Lexicons,...
More informationIntroducing XAIRA. Lou Burnard Tony Dodd. An XML aware tool for corpus indexing and searching. Research Technology Services, OUCS
Introducing XAIRA An XML aware tool for corpus indexing and searching Lou Burnard Tony Dodd Research Technology Services, OUCS What is XAIRA? XML Aware Indexing and Retrieval Architecture Developed from
More informationCorpus Linguistics: corpus annotation
Corpus Linguistics: corpus annotation Karën Fort karen.fort@inist.fr November 30, 2010 Introduction Methodology Annotation Issues Annotation Formats From Formats to Schemes Sources Most of this course
More informationNgram Search Engine with Patterns Combining Token, POS, Chunk and NE Information
Ngram Search Engine with Patterns Combining Token, POS, Chunk and NE Information Satoshi Sekine Computer Science Department New York University sekine@cs.nyu.edu Kapil Dalwani Computer Science Department
More informationHidden Markov Models. Natural Language Processing: Jordan Boyd-Graber. University of Colorado Boulder LECTURE 20. Adapted from material by Ray Mooney
Hidden Markov Models Natural Language Processing: Jordan Boyd-Graber University of Colorado Boulder LECTURE 20 Adapted from material by Ray Mooney Natural Language Processing: Jordan Boyd-Graber Boulder
More informationBackground and Context for CLASP. Nancy Ide, Vassar College
Background and Context for CLASP Nancy Ide, Vassar College The Situation Standards efforts have been on-going for over 20 years Interest and activity mainly in Europe in 90 s and early 2000 s Text Encoding
More informationTreex: Modular NLP Framework
: Modular NLP Framework Martin Popel ÚFAL (Institute of Formal and Applied Linguistics) Charles University in Prague September 2015, Prague, MT Marathon Outline Motivation, vs. architecture internals Future
More informationLast Words. Googleology is Bad Science. Adam Kilgarriff Lexical Computing Ltd. and University of Sussex
Last Words Googleology is Bad Science Adam Kilgarriff Lexical Computing Ltd. and University of Sussex The World Wide Web is enormous, free, immediately available, and largely linguistic. As we discover,
More informationUIMA-based Annotation Type System for a Text Mining Architecture
UIMA-based Annotation Type System for a Text Mining Architecture Udo Hahn, Ekaterina Buyko, Katrin Tomanek, Scott Piao, Yoshimasa Tsuruoka, John McNaught, Sophia Ananiadou Jena University Language and
More informationCS473: Course Review CS-473. Luo Si Department of Computer Science Purdue University
CS473: CS-473 Course Review Luo Si Department of Computer Science Purdue University Basic Concepts of IR: Outline Basic Concepts of Information Retrieval: Task definition of Ad-hoc IR Terminologies and
More informationstructure of the presentation Frame Semantics knowledge-representation in larger-scale structures the concept of frame
structure of the presentation Frame Semantics semantic characterisation of situations or states of affairs 1. introduction (partially taken from a presentation of Markus Egg): i. what is a frame supposed
More informationTools for Annotating and Searching Corpora Practical Session 1: Annotating
Tools for Annotating and Searching Corpora Practical Session 1: Annotating Stefanie Dipper Institute of Linguistics Ruhr-University Bochum Corpus Linguistics Fest (CLiF) June 6-10, 2016 Indiana University,
More informationNatural Language Processing Pipelines to Annotate BioC Collections with an Application to the NCBI Disease Corpus
Natural Language Processing Pipelines to Annotate BioC Collections with an Application to the NCBI Disease Corpus Donald C. Comeau *, Haibin Liu, Rezarta Islamaj Doğan and W. John Wilbur National Center
More informationXML Support for Annotated Language Resources
XML Support for Annotated Language Resources Nancy Ide Department of Computer Science Vassar College Poughkeepsie, New York USA ide@cs.vassar.edu Laurent Romary Equipe Langue et Dialogue LORIA/CNRS Vandoeuvre-lès-Nancy,
More informationHow to.. What is the point of it?
Program's name: Linguistic Toolbox 3.0 α-version Short name: LIT Authors: ViatcheslavYatsko, Mikhail Starikov Platform: Windows System requirements: 1 GB free disk space, 512 RAM,.Net Farmework Supported
More informationImporting MASC into the ANNIS linguistic database: A case study of mapping GrAF
Importing MASC into the ANNIS linguistic database: A case study of mapping GrAF Arne Neumann 1 Nancy Ide 2 Manfred Stede 1 1 EB Cognitive Science and SFB 632 University of Potsdam 2 Department of Computer
More informationOrtolang Tools : MarsaTag
Ortolang Tools : MarsaTag Stéphane Rauzy, Philippe Blache, Grégoire de Montcheuil SECOND VARIAMU WORKSHOP LPL, Aix-en-Provence August 20th & 21st, 2014 ORTOLANG received a State aid under the «Investissements
More informationNLP in practice, an example: Semantic Role Labeling
NLP in practice, an example: Semantic Role Labeling Anders Björkelund Lund University, Dept. of Computer Science anders.bjorkelund@cs.lth.se October 15, 2010 Anders Björkelund NLP in practice, an example:
More informationCSC 5930/9010: Text Mining GATE Developer Overview
1 CSC 5930/9010: Text Mining GATE Developer Overview Dr. Paula Matuszek Paula.Matuszek@villanova.edu Paula.Matuszek@gmail.com (610) 647-9789 GATE Components 2 We will deal primarily with GATE Developer:
More informationINF FALL NATURAL LANGUAGE PROCESSING. Jan Tore Lønning, Lecture 4, 10.9
1 INF5830 2015 FALL NATURAL LANGUAGE PROCESSING Jan Tore Lønning, Lecture 4, 10.9 2 Working with texts From bits to meaningful units Today: 3 Reading in texts Character encodings and Unicode Word tokenization
More informationLing/CSE 472: Introduction to Computational Linguistics. 5/4/17 Parsing
Ling/CSE 472: Introduction to Computational Linguistics 5/4/17 Parsing Reminders Revised project plan due tomorrow Assignment 4 is available Overview Syntax v. parsing Earley CKY (briefly) Chart parsing
More informationImplementing a Variety of Linguistic Annotations
Implementing a Variety of Linguistic Annotations through a Common Web-Service Interface Adam Funk, Ian Roberts, Wim Peters University of Sheffield 18 May 2010 Adam Funk, Ian Roberts, Wim Peters Implementing
More informationLarge-Scale Syntactic Processing: Parsing the Web. JHU 2009 Summer Research Workshop
Large-Scale Syntactic Processing: JHU 2009 Summer Research Workshop Intro CCG parser Tasks 2 The Team Stephen Clark (Cambridge, UK) Ann Copestake (Cambridge, UK) James Curran (Sydney, Australia) Byung-Gyu
More informationLet s get parsing! Each component processes the Doc object, then passes it on. doc.is_parsed attribute checks whether a Doc object has been parsed
Let s get parsing! SpaCy default model includes tagger, parser and entity recognizer nlp = spacy.load('en ) tells spacy to use "en" with ["tagger", "parser", "ner"] Each component processes the Doc object,
More informationQuery Difficulty Prediction for Contextual Image Retrieval
Query Difficulty Prediction for Contextual Image Retrieval Xing Xing 1, Yi Zhang 1, and Mei Han 2 1 School of Engineering, UC Santa Cruz, Santa Cruz, CA 95064 2 Google Inc., Mountain View, CA 94043 Abstract.
More informationA Multilingual Social Media Linguistic Corpus
A Multilingual Social Media Linguistic Corpus Luis Rei 1,2 Dunja Mladenić 1,2 Simon Krek 1 1 Artificial Intelligence Laboratory Jožef Stefan Institute 2 Jožef Stefan International Postgraduate School 4th
More informationSupporting a Locale of One: Global Content Delivery for the Individual
Prof Vinny Wade Supporting a Locale of One: Global Content Delivery for the Individual Trinity College, Dublin University Deputy Director, CNGL Outline Challenge: Changing Face of Content Dynamic Multillingual
More informationSemantics Isn t Easy Thoughts on the Way Forward
Semantics Isn t Easy Thoughts on the Way Forward NANCY IDE, VASSAR COLLEGE REBECCA PASSONNEAU, COLUMBIA UNIVERSITY COLLIN BAKER, ICSI/UC BERKELEY CHRISTIANE FELLBAUM, PRINCETON UNIVERSITY New York University
More informationAnnotating Spatio-Temporal Information in Documents
Annotating Spatio-Temporal Information in Documents Jannik Strötgen University of Heidelberg Institute of Computer Science Database Systems Research Group http://dbs.ifi.uni-heidelberg.de stroetgen@uni-hd.de
More informationData for linguistics ALEXIS DIMITRIADIS. Contents First Last Prev Next Back Close Quit
Data for linguistics ALEXIS DIMITRIADIS Text, corpora, and data in the wild 1. Where does language data come from? The usual: Introspection, questionnaires, etc. Corpora, suited to the domain of study:
More informationRecursive Deep Models for Semantic Compositionality Over a Sentiment Treebank text
Philosophische Fakultät Seminar für Sprachwissenschaft Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank text 06 July 2017, Patricia Fischer & Neele Witte Overview Sentiment
More informationContext-Free Grammars
Department of Linguistics Ohio State University Syntax 2 (Linguistics 602.02) January 3, 2012 (CFGs) A CFG is an ordered quadruple T, N, D, P where a. T is a finite set called the terminals; b. N is a
More informationA Short Introduction to CATMA
A Short Introduction to CATMA Outline: I. Getting Started II. Analyzing Texts - Search Queries in CATMA III. Annotating Texts (collaboratively) with CATMA IV. Further Search Queries: Analyze Your Annotations
More informationPrivacy and Security in Online Social Networks Department of Computer Science and Engineering Indian Institute of Technology, Madras
Privacy and Security in Online Social Networks Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture - 25 Tutorial 5: Analyzing text using Python NLTK Hi everyone,
More informationParmenides. Semi-automatic. Ontology. construction and maintenance. Ontology. Document convertor/basic processing. Linguistic. Background knowledge
Discover hidden information from your texts! Information overload is a well known issue in the knowledge industry. At the same time most of this information becomes available in natural language which
More informationContents. List of Figures. List of Tables. Acknowledgements
Contents List of Figures List of Tables Acknowledgements xiii xv xvii 1 Introduction 1 1.1 Linguistic Data Analysis 3 1.1.1 What's data? 3 1.1.2 Forms of data 3 1.1.3 Collecting and analysing data 7 1.2
More informationANC2Go: A Web Application for Customized Corpus Creation
ANC2Go: A Web Application for Customized Corpus Creation Nancy Ide, Keith Suderman, Brian Simms Department of Computer Science, Vassar College Poughkeepsie, New York 12604 USA {ide, suderman, brsimms}@cs.vassar.edu
More informationSemantic and Multimodal Annotation. CLARA University of Copenhagen August 2011 Susan Windisch Brown
Semantic and Multimodal Annotation CLARA University of Copenhagen 15-26 August 2011 Susan Windisch Brown 2 Program: Monday Big picture Coffee break Lexical ambiguity and word sense annotation Lunch break
More informationTectoMT: Modular NLP Framework
: Modular NLP Framework Martin Popel, Zdeněk Žabokrtský ÚFAL, Charles University in Prague IceTAL, 7th International Conference on Natural Language Processing August 17, 2010, Reykjavik Outline Motivation
More informationCS 224N Assignment 2 Writeup
CS 224N Assignment 2 Writeup Angela Gong agong@stanford.edu Dept. of Computer Science Allen Nie anie@stanford.edu Symbolic Systems Program 1 Introduction 1.1 PCFG A probabilistic context-free grammar (PCFG)
More informationText Analytics Introduction (Part 1)
Text Analytics Introduction (Part 1) Maha Althobaiti, Udo Kruschwitz, Massimo Poesio School of Computer Science and Electronic Engineering University of Essex udo@essex.ac.uk 23 September 2015 Text Analytics
More informationDependency grammar and dependency parsing
Dependency grammar and dependency parsing Syntactic analysis (5LN455) 2014-12-10 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Mid-course evaluation Mostly positive
More informationIncremental Integer Linear Programming for Non-projective Dependency Parsing
Incremental Integer Linear Programming for Non-projective Dependency Parsing Sebastian Riedel James Clarke ICCS, University of Edinburgh 22. July 2006 EMNLP 2006 S. Riedel, J. Clarke (ICCS, Edinburgh)
More informationIterative Learning of Relation Patterns for Market Analysis with UIMA
UIMA Workshop, GLDV, Tübingen, 09.04.2007 Iterative Learning of Relation Patterns for Market Analysis with UIMA Sebastian Blohm, Jürgen Umbrich, Philipp Cimiano, York Sure Universität Karlsruhe (TH), Institut
More informationA BNC-like corpus of American English
The American National Corpus Everything You Always Wanted To Know... And Weren t Afraid To Ask Nancy Ide Department of Computer Science Vassar College What is the? A BNC-like corpus of American English
More informationNatural Language Processing SoSe Question Answering. (based on the slides of Dr. Saeedeh Momtazi)
Natural Language Processing SoSe 2015 Question Answering Dr. Mariana Neves July 6th, 2015 (based on the slides of Dr. Saeedeh Momtazi) Outline 2 Introduction History QA Architecture Outline 3 Introduction
More informationContext-Free Grammars. Carl Pollard Ohio State University. Linguistics 680 Formal Foundations Tuesday, November 10, 2009
Context-Free Grammars Carl Pollard Ohio State University Linguistics 680 Formal Foundations Tuesday, November 10, 2009 These slides are available at: http://www.ling.osu.edu/ scott/680 1 (1) Context-Free
More informationUsing Search-Logs to Improve Query Tagging
Using Search-Logs to Improve Query Tagging Kuzman Ganchev Keith Hall Ryan McDonald Slav Petrov Google, Inc. {kuzman kbhall ryanmcd slav}@google.com Abstract Syntactic analysis of search queries is important
More informationQuestion Answering Using XML-Tagged Documents
Question Answering Using XML-Tagged Documents Ken Litkowski ken@clres.com http://www.clres.com http://www.clres.com/trec11/index.html XML QA System P Full text processing of TREC top 20 documents Sentence
More informationSAPIENT Automation project
Dr Maria Liakata Leverhulme Trust Early Career fellow Department of Computer Science, Aberystwyth University Visitor at EBI, Cambridge mal@aber.ac.uk 25 May 2010, London Motivation SAPIENT Automation Project
More informationStatistical parsing. Fei Xia Feb 27, 2009 CSE 590A
Statistical parsing Fei Xia Feb 27, 2009 CSE 590A Statistical parsing History-based models (1995-2000) Recent development (2000-present): Supervised learning: reranking and label splitting Semi-supervised
More informationIntro to XML. Borrowed, with author s permission, from:
Intro to XML Borrowed, with author s permission, from: http://business.unr.edu/faculty/ekedahl/is389/topic3a ndroidintroduction/is389androidbasics.aspx Part 1: XML Basics Why XML Here? You need to understand
More informationLecture 14: Annotation
Lecture 14: Annotation Nathan Schneider (with material from Henry Thompson, Alex Lascarides) ENLP 23 October 2016 1/14 Annotation Why gold 6= perfect Quality Control 2/14 Factors in Annotation Suppose
More informationShrey Patel B.E. Computer Engineering, Gujarat Technological University, Ahmedabad, Gujarat, India
International Journal of Scientific Research in Computer Science, Engineering and Information Technology 2018 IJSRCSEIT Volume 3 Issue 3 ISSN : 2456-3307 Some Issues in Application of NLP to Intelligent
More informationAgenda. Information Extraction using Markov Models. I. The IE task. Augmented Google search IE types. 1. The IE problem and its motivation
Agenda Information Extraction using Markov Models Martin Labský labsky@vse.cz Dept. of Knowledge Engineering VŠE raha 1. he IE problem and its motivation symbolic vs. probabilistic, L2 [6] 1 2 I. he IE
More informationExam III March 17, 2010
CIS 4930 NLP Print Your Name Exam III March 17, 2010 Total Score Your work is to be done individually. The exam is worth 106 points (six points of extra credit are available throughout the exam) and it
More informationLIDER Survey. Overview. Number of participants: 24. Participant profile (organisation type, industry sector) Relevant use-cases
LIDER Survey Overview Participant profile (organisation type, industry sector) Relevant use-cases Discovering and extracting information Understanding opinion Content and data (Data Management) Monitoring
More informationDepartment of Electronic Engineering FINAL YEAR PROJECT REPORT
Department of Electronic Engineering FINAL YEAR PROJECT REPORT BEngCE-2007/08-HCS-HCS-03-BECE Natural Language Understanding for Query in Web Search 1 Student Name: Sit Wing Sum Student ID: Supervisor:
More informationOn-line glossary compilation
On-line glossary compilation 1 Introduction Alexander Kotov (akotov2) Hoa Nguyen (hnguyen4) Hanna Zhong (hzhong) Zhenyu Yang (zyang2) Nowadays, the development of the Internet has created massive amounts
More informationApache UIMA and Mayo ctakes
Apache and Mayo and how it is used in the clinical domain March 16, 2012 Apache and Mayo Outline 1 Apache and Mayo Outline 1 2 Introducing Pipeline Modules Apache and Mayo What is? (You - eee - muh) Unstructured
More informationAUTOMATIC VISUAL CONCEPT DETECTION IN VIDEOS
AUTOMATIC VISUAL CONCEPT DETECTION IN VIDEOS Nilam B. Lonkar 1, Dinesh B. Hanchate 2 Student of Computer Engineering, Pune University VPKBIET, Baramati, India Computer Engineering, Pune University VPKBIET,
More informationEditorial Style. An Overview of Hofstra Law s Editorial Style and Best Practices for Writing for the Web. Office of Communications July 30, 2013
Editorial Style An Overview of Hofstra Law s Editorial Style and Best Practices for Writing for the Web Office of Communications July 30, 2013 What Is Editorial Style? Editorial style refers to: Spelling
More informationHistorical Text Mining:
Historical Text Mining Historical Text Mining, and Historical Text Mining: Challenges and Opportunities Dr. Robert Sanderson Dept. of Computer Science University of Liverpool azaroth@liv.ac.uk http://www.csc.liv.ac.uk/~azaroth/
More informationAdvanced Topics in Information Retrieval Natural Language Processing for IR & IR Evaluation. ATIR April 28, 2016
Advanced Topics in Information Retrieval Natural Language Processing for IR & IR Evaluation Vinay Setty vsetty@mpi-inf.mpg.de Jannik Strötgen jannik.stroetgen@mpi-inf.mpg.de ATIR April 28, 2016 Organizational
More informationInformation Extraction Techniques in Terrorism Surveillance
Information Extraction Techniques in Terrorism Surveillance Roman Tekhov Abstract. The article gives a brief overview of what information extraction is and how it might be used for the purposes of counter-terrorism
More informationDependency grammar and dependency parsing
Dependency grammar and dependency parsing Syntactic analysis (5LN455) 2015-12-09 Sara Stymne Department of Linguistics and Philology Based on slides from Marco Kuhlmann Activities - dependency parsing
More informationBetter Evaluation for Grammatical Error Correction
Better Evaluation for Grammatical Error Correction Daniel Dahlmeier 1 and Hwee Tou Ng 1,2 1 NUS Graduate School for Integrative Sciences and Engineering 2 Department of Computer Science, National University
More informationFully Delexicalized Contexts for Syntax-Based Word Embeddings
Fully Delexicalized Contexts for Syntax-Based Word Embeddings Jenna Kanerva¹, Sampo Pyysalo² and Filip Ginter¹ ¹Dept of IT - University of Turku, Finland ²Lang. Tech. Lab - University of Cambridge turkunlp.github.io
More informationHeading-Based Sectional Hierarchy Identification for HTML Documents
Heading-Based Sectional Hierarchy Identification for HTML Documents 1 Dept. of Computer Engineering, Boğaziçi University, Bebek, İstanbul, 34342, Turkey F. Canan Pembe 1,2 and Tunga Güngör 1 2 Dept. of
More informationNatural Language Processing SoSe Question Answering. (based on the slides of Dr. Saeedeh Momtazi) )
Natural Language Processing SoSe 2014 Question Answering Dr. Mariana Neves June 25th, 2014 (based on the slides of Dr. Saeedeh Momtazi) ) Outline 2 Introduction History QA Architecture Natural Language
More informationTransition-Based Dependency Parsing with Stack Long Short-Term Memory
Transition-Based Dependency Parsing with Stack Long Short-Term Memory Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, Noah A. Smith Association for Computational Linguistics (ACL), 2015 Presented
More informationLING203: Corpus. March 9, 2009
LING203: Corpus March 9, 2009 Corpus A collection of machine readable texts SJSU LLD have many corpora http://linguistics.sjsu.edu/bin/view/public/chltcorpora Each corpus has a link to a description page
More informationA Method for Semi-Automatic Ontology Acquisition from a Corporate Intranet
A Method for Semi-Automatic Ontology Acquisition from a Corporate Intranet Joerg-Uwe Kietz, Alexander Maedche, Raphael Volz Swisslife Information Systems Research Lab, Zuerich, Switzerland fkietz, volzg@swisslife.ch
More informationDependency Parsing. Ganesh Bhosale Neelamadhav G Nilesh Bhosale Pranav Jawale under the guidance of
Dependency Parsing Ganesh Bhosale - 09305034 Neelamadhav G. - 09305045 Nilesh Bhosale - 09305070 Pranav Jawale - 09307606 under the guidance of Prof. Pushpak Bhattacharyya Department of Computer Science
More informationTopics in Parsing: Context and Markovization; Dependency Parsing. COMP-599 Oct 17, 2016
Topics in Parsing: Context and Markovization; Dependency Parsing COMP-599 Oct 17, 2016 Outline Review Incorporating context Markovization Learning the context Dependency parsing Eisner s algorithm 2 Review
More informationSustainability of Text-Technological Resources
Sustainability of Text-Technological Resources Maik Stührenberg, Michael Beißwenger, Kai-Uwe Kühnberger, Harald Lüngen, Alexander Mehler, Dieter Metzing, Uwe Mönnich Research Group Text-Technological Overview
More informationMaking Sense Out of the Web
Making Sense Out of the Web Rada Mihalcea University of North Texas Department of Computer Science rada@cs.unt.edu Abstract. In the past few years, we have witnessed a tremendous growth of the World Wide
More informationProbabilistic Parsing of Mathematics
Probabilistic Parsing of Mathematics Jiří Vyskočil Josef Urban Czech Technical University in Prague, Czech Republic Cezary Kaliszyk University of Innsbruck, Austria Outline Why and why not current formal
More informationAgenda for today. Homework questions, issues? Non-projective dependencies Spanning tree algorithm for non-projective parsing
Agenda for today Homework questions, issues? Non-projective dependencies Spanning tree algorithm for non-projective parsing 1 Projective vs non-projective dependencies If we extract dependencies from trees,
More informationQuestion Answering Systems
Question Answering Systems An Introduction Potsdam, Germany, 14 July 2011 Saeedeh Momtazi Information Systems Group Outline 2 1 Introduction Outline 2 1 Introduction 2 History Outline 2 1 Introduction
More informationA Quick Guide to MaltParser Optimization
A Quick Guide to MaltParser Optimization Joakim Nivre Johan Hall 1 Introduction MaltParser is a system for data-driven dependency parsing, which can be used to induce a parsing model from treebank data
More informationStandards for Language Resources
Standards for Language Resources Nancy IDE Department of Computer Science Vassar College Poughkeepsie, New York 12604-0520 USA ide@cs.vassar.edu Laurent ROMARY Equipe Langue et Dialogue LORIA/INRIA 54506
More informationUsing the Web as a Corpus. in Natural Language Processing
Using the Web as a Corpus in Natural Language Processing Malvina Nissim Laboratory for Applied Ontology ISTC-CNR, Roma nissim@loa-cnr.it Johan Bos Dipartimento di Informatica Università La Sapienza, Roma
More informationAn Extension of the TIGER Query Language for Treebanks with Frame Semantics Annotation
An Extension of the TIGER Query Language for Treebanks with Frame Semantics Annotation Thesis Presentation for M.Sc. LST Torsten Marek Saarland University February 9th, 2009
More informationThe American National Corpus First Release
The American National Corpus First Release Nancy Ide and Keith Suderman Department of Computer Science, Vassar College, Poughkeepsie, NY 12604-0520 USA ide@cs.vassar.edu, suderman@cs.vassar.edu Abstract
More informationGRADES LANGUAGE! Live, Grades Correlated to the Indiana English/ Language Arts Academic Standards. August 2017
GRADES 5 10 LANGUAGE! Live, Grades 5 10 August 2017 Correlated to the Indiana English/ Language Arts Academic LANGUAGE! Live correlated to, Grades 5-10 Grade 5 RF.1: READING: Foundations: Develop, build,
More informationMeaning Banking and Beyond
Meaning Banking and Beyond Valerio Basile Wimmics, Inria November 18, 2015 Semantics is a well-kept secret in texts, accessible only to humans. Anonymous I BEG TO DIFFER Surface Meaning Step by step analysis
More informationUsability Test Report: Bento results interface 1
Usability Test Report: Bento results interface 1 Summary Emily Daly and Ian Sloat conducted usability testing on the functionality of the Bento results interface. The test was conducted at the temporary
More informationManaging a Multilingual Treebank Project
Managing a Multilingual Treebank Project Milan Souček Timo Järvinen Adam LaMontagne Lionbridge Finland {milan.soucek,timo.jarvinen,adam.lamontagne}@lionbridge.com Abstract This paper describes the work
More informationSTRUCTURES AND STRATEGIES FOR STATE SPACE SEARCH
Slide 3.1 3 STRUCTURES AND STRATEGIES FOR STATE SPACE SEARCH 3.0 Introduction 3.1 Graph Theory 3.2 Strategies for State Space Search 3.3 Using the State Space to Represent Reasoning with the Predicate
More informationSheffield University and the TREC 2004 Genomics Track: Query Expansion Using Synonymous Terms
Sheffield University and the TREC 2004 Genomics Track: Query Expansion Using Synonymous Terms Yikun Guo, Henk Harkema, Rob Gaizauskas University of Sheffield, UK {guo, harkema, gaizauskas}@dcs.shef.ac.uk
More informationParallel Concordancing and Translation. Michael Barlow
[Translating and the Computer 26, November 2004 [London: Aslib, 2004] Parallel Concordancing and Translation Michael Barlow Dept. of Applied Language Studies and Linguistics University of Auckland Auckland,
More informationAssignment 4 CSE 517: Natural Language Processing
Assignment 4 CSE 517: Natural Language Processing University of Washington Winter 2016 Due: March 2, 2016, 1:30 pm 1 HMMs and PCFGs Here s the definition of a PCFG given in class on 2/17: A finite set
More informationContext-Free Grammars
Context-Free Grammars Carl Pollard yntax 2 (Linguistics 602.02) January 3, 2012 Context-Free Grammars (CFGs) A CFG is an ordered quadruple T, N, D, P where a. T is a finite set called the terminals; b.
More informationBD003: Introduction to NLP Part 2 Information Extraction
BD003: Introduction to NLP Part 2 Information Extraction The University of Sheffield, 1995-2017 This work is licenced under the Creative Commons Attribution-NonCommercial-ShareAlike Licence. Contents This
More informationIntroduction to Text Mining. Hongning Wang
Introduction to Text Mining Hongning Wang CS@UVa Who Am I? Hongning Wang Assistant professor in CS@UVa since August 2014 Research areas Information retrieval Data mining Machine learning CS@UVa CS6501:
More informationKnowledge Engineering with Semantic Web Technologies
This file is licensed under the Creative Commons Attribution-NonCommercial 3.0 (CC BY-NC 3.0) Knowledge Engineering with Semantic Web Technologies Lecture 5: Ontological Engineering 5.3 Ontology Learning
More informationLearning to Match. Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li
Learning to Match Jun Xu, Zhengdong Lu, Tianqi Chen, Hang Li 1. Introduction The main tasks in many applications can be formalized as matching between heterogeneous objects, including search, recommendation,
More informationParts of Speech, Named Entity Recognizer
Parts of Speech, Named Entity Recognizer Artificial Intelligence @ Allegheny College Janyl Jumadinova November 8, 2018 Janyl Jumadinova Parts of Speech, Named Entity Recognizer November 8, 2018 1 / 25
More information