Information Retrieval

Transcription

1 Information Retrieval Overview and Introduction All slides unless specifically mentioned are copyright Anton Leuski & Donald Metzler 1

2 Administrativa What is Information Retrieval (IR)? Issues in IR Dimensions of IR Course goals 2

3 What? CSCI 599. Special Topics. Applications of Natural Language Processing: Information Retrieval Two other Applications of Natural Language Processing (NLP) courses Machine Translation Information Extraction Related courses CSCI 544. Natural Language Processing CSCI 562. Empirical Methods in Natural Language Processing CSCI 572. Information Retrieval and Web Search Engines CSCI 599. Data Mining and Statistical Inference CSCI 599. Social Media Analysis 3

4 Who? Anton Leuski Institute for Creative Technologies Donald Metzler Information Sciences Institute 4

5 Where? Here: GFS 118 Web: nld/ir-class/ schedule lecture notes homework assignments discussions 5

6 When? Every Tuesday and Thursday, 3:30-4:50 PM. Office hours: after each lecture See the schedule on the web site for more details 6

7 Grading 3 programming/homework assignments: 30% Midterm exam: 20% Final exam: 20% Final project: 25% Discussion participation: 5% 7

8 Assignments Homework tasks might include modifying the "ranking function" or "indexer" of an open source information retrieval toolkit (Lucene) for some search task writing code to cluster documents based on their similarity writing code to automatically evaluate the quality of search results developing a system to automatically summarize a stream of Twitter messages Framework: Lucene Java-based, open source search engine Final project we would announce a number of topics to choose from at the middle of the semester you could create your own project, but the topic has to be approved by us 8

9 Reading Books W. B. Croft, D. Metzler, and T. Strohman. Search Engines: Information Retrieval in Practice S. Buettcher, C. L. A. Clarke, G. V. Cormack. Information Retrieval: Implementing and Evaluating Search Engines C. D. Manning, P. Raghavan and H. Schütze. Introduction to Information Retrieval C. J. van Rijsbergen. Information Retrieval I. H. Witten, A. Moffat, T. C. Bell. Managing Gigabytes A. Moffat, J. Zobel, D. Hawking. Recommended Reading for IR Research Students Papers TBA 9

10 Administrativa What is Information Retrieval (IR)? Issues in IR Dimensions of IR Course goals 10

11 Example: Web Search Document (web page) retrieval in response to a query Quite effective (at some things) Highly visible (mostly) Commercially successful (some of them) Is that it?.. 11

12 Information Retrieval Information retrieval is a field concerned with the structure, analysis, organization, storage, searching, and retrieval of information. (Salton, 1968) To solve the information overload problem IR is interdisciplinary computer sciences mathematics information science information architecture cognitive psychology linguistics statistics 12

13 History Since the beginning of the written word people tried to organize information 3rd BC: Library of Alexandria 1689: Vincentius Placcius invented a note-taking machine. 1880s-1890s: Herman Hollerith invents the recording of data on a machine readable medium 1920s-1930s: Emanuel Goldberg submits patents for his "Statistical Machine a document search engine that used photoelectric cells and pattern recognition to search the metadata on rolls of microfilmed documents. 13

14 History 1945: MEMory EXtender by Vannevar Bush hypothetical electro-mechanical system Proto-hypertext lateral browsing of microfilms following links between individual frames associative trails Features extending storing consulting the records Missing features search metadata 14

15 History 1950: The term information retrieval appears to have been coined by Calvin Mooers. 1950s-1960s: First automated IR systems. SMART MEDLARS MeSH 1970s: First online IR systems. MEDLINE. Lockheed's Dialog. Hypertext. 1978: 1st SIGIR conference 1989: WWW proposals 1992: 1st TREC conference late 1990s: First Web search engines 15

16 Knowledge Navigator An IR system mockup by Apple Computers from 1988 A device that can access a large networked database of hypertext information, and use software agents to assist searching for information v=qrh8eimu_20 16

17 17

18 IR is not databases Data structured Databases unstructured IR Fields well-defined semantics (SSN, age,...) Queries well-formed (relational algebra, SQL) Matching exact (results are always correct ) SELECT * FROM Accounts WHERE balance > 50,000 ORDER BY name; no well-defined semantics (text fields) free text, some fuzzy operators imprecise bank scandals in California 18

19 Question: Is grep an IR system? 19

20 Example: Text Matching How do you measure aboutness? Exact matching of words is not enough Many different ways to write the same thing in a natural language like English e.g., does a news story containing the text bank director in LA steals funds match the query bank scandals in California? Some stories will be better matches than others 20

21 Search Process Information need query text objects representation representation indexed objects indexed objects comparison evaluation/ feedback retrieved objects 21

22 Administrativa What is Information Retrieval (IR)? Issues in IR Dimensions of IR Course goals 22

23 Issues in IR relevance Information need query text objects Information need and user interaction Relevance Representation representation indexed objects representation indexed objects Comparison Evaluation comparison evaluation/feedback retrieved objects 23

24 Users and Information Needs Search is user centered Keyword queries are often poor descriptions of actual information needs Interaction and context are important for understanding user intent Query refinement techniques such as query expansion, query suggestion, relevance feedback improve ranking 24

25 Relevance What is it? Simple (and simplistic) definition: A relevant document contains the information that a person was looking for when they submitted a query to the search engine Many factors influence a personʼs decision about what is relevant: e.g., task, context, novelty, style Topical relevance (same topic) vs. user relevance (everything else) Retrieval models define a view of relevance Relevance Ranking algorithms used in search engines are based on retrieval models Most models describe statistical properties of text rather than linguistic i.e. counting simple text features such as words instead of parsing and analyzing the sentences Statistical approach to text processing started with Luhn in the 50s Linguistic features can be part of a statistical model 25

26 Representation Most successful approaches are statistical directly, or an effort to capture and use word probabilities Why not natural language understanding? computer understands documents and query and matches them state of the art is brittle in unrestricted domains can be highly successful in predictable settings, though information extraction on terrorism/takeovers (MUC) medical or legal settings with restricted vocabulary Could use manually assigned headings e.g., Library of Congress headings, Dewey Decimal headings expensive and human agreement is not good hard to predict what headings are interesting Statistical and not lexical count words lexical information plays secondary role 26

27 Example: Bag of Words Ignoring the word order Popular and effective Similar vocabulary similar content Consider reordering words in a headline Random: beating takes points falling another Dow 355 Alphabetical: 355 another beating Dow falling points Interesting : Dow points beating falling 355 another Original: Dow takes another beating, falling 355 points 27

28 What is this about? 16 said$ 14 McDonalds 12 fat$ 11 fries 8 new" 6 company french nutrition 5 food oil percent reduce taste Tuesday 4 amount change health Henstenburg make obesity 3 acids consumer fatty polyunsaturated US 2 amounts artery Beemer cholesterol clogging director down eat estimates expert fast formula impact initiative moderate plans restaurant saturated trans win 1... added addition adults advocate affect afternoon age Americans Asia battling beef bet brand Britt Brook Browns calorie center chain chemically crispy customers cut vegetable weapon weeks Wendys Wootan worldwide years York Copyright James Allan 28

29 The start of the original McDonald's slims down spuds Fast-food chain to reduce certain types of fat in its french fries with new cooking oil. NEW YORK (CNN/Money) - McDonald's Corp. is cutting the amount of "bad" fat in its french fries nearly in half, the fast-food chain said Tuesday as it moves to make all its fried menu items healthier. But does that mean the popular shoestring fries won't taste the same? The company says no. "It's a win-win for our customers because they are getting the same great french-fry taste along with an even healthier nutrition profile," said Mike Roberts, president of McDonald's USA. But others are not so sure. McDonald's will not specifically discuss the kind of oil it plans to use, but at least one nutrition expert says playing with the formula could mean a different taste. Shares of Oak Brook, Ill.-based McDonald's (MCD: down $0.54 to $23.22, Research, Estimates) were lower Tuesday afternoon. It was unclear Tuesday whether competitors Burger King and Wendy's International (WEN: down $0.80 to $34.91, Research, Estimates) would follow suit. Neither company could immediately be reached for comment Copyright James Allan 29

30 The Point? Basis of most IR is a very simple approach find words in documents compare them to words in a query this approach is very effective! Other types of features are often used phrases link structure named entities (people, locations, organizations) special features (chemical names, product names) Focus is on improving accuracy, speed and on extending ideas elsewhere 30

31 Comparison Retrieval model provide a mathematical framework for defining the matching process includes explanation of assumptions basis of many ranking algorithms can be implicit Some models that we will cover boolean vector space inference networks language models relevance models 31

32 Evaluation Experimental procedures and measures for comparing system output with user expectations Originated in Cranfield experiments in the 60s IR evaluation methods now used in many fields Typically use test collection of documents, queries, and relevance judgments Most commonly used are TREC collections Recall and precision are two examples of effectiveness measures 32

33 IR is not Search Engines A search engine is the practical application of information retrieval techniques to large scale text collections Information Retrieval Information needs User interaction Relevance Effective ranking Representation How to represent things Comparison How to match things Evaluation Testing and measuring Search Engines Performance Efficient search and indexing Incorporating new data Coverage and freshness Scalability Growing with data and users Adaptability Tuning for applications Specific problems e.g. Spam 33

34 Administrativa What is Information Retrieval (IR)? Issues in IR Dimensions of IR Course goals 34

35 Dimensions of IR IR is not just for the Web IR is not just search 3 dimensions: data application/domain task 35

36 Data Text Multiple languages accessing Chinese collection using English Scanned Text (handwritten or typed) either word images or OCRed text with errors Images features? Video features? Speech (audio) ASR output (with errors) Music features? 36

37 Application Web Enterprise like web, but smaller, more focused, more controlled Desktop smaller scale; different file formats; very user-centered Forums shorter than web; threads; typos; Social/twitter short; threads; typos; P2P distributed aspects 37

38 Application (continued) Literature the original domain; cross-references; citations Legal specific language; well-defined guidelines; Medical similar to legal; unusual vocabulary; Personal Information Management (PIM) contacts and schedules 38

39 Tasks Search collection is static, queries are dynamic Filtering & Routing think newswire; query is static, documents are dynamic Detection & Tracking newswire again; new topic discovery and tracking Classification & Clustering grouping similar documents together for analysis Summarization locating most important pieces Question answering factual information; Collaborative recommender systems; think Amazon reviews. multi-agent search 39

40 Dimensions of IR Content Applications Tasks Text Web Search Multiple languages Enterprise Filtering & Routing Scanned Text (handwritten or typed) Desktop Detection & tracking Images Forum Classification Video P2P Question answering Speech (audio) Literature Summarization Music Legal Collaborative PIM 40

41 Assignment Watch the Knowledge Navigator video Think about how would you build such a system what are the tasks the system performs? what are the challenges? Write down what IR dimensions that we mentioned are covered in the video? what dimensions are covered that we did not mention? 41

42 Administrativa What is Information Retrieval (IR)? Issues in IR Dimensions of IR Course goals 42

43 Course Goals Understand what IR is Analyze core issues... and how they vary under different conditions... Consider core solutions... and how they can be applied under different conditions... Acquire some practical skills how to apply that knowledge 43

44 Schedule Core IR search engines architecture text processing indexes retrieval models evaluation user modeling Topics in IR filtering multimedia: image & audio cross-lingual web search & advertising distributed & p2p question answering social semi-structured 44

45 Summary IR is a large interdisciplinary field with a long history IR deals with many different data types, applications, and tasks At the core of IR is the match or comparison operation 45