Informa(on Retrieval. Informa(on Retrieval

Size: px

Start display at page:

Download "Informa(on Retrieval. Informa(on Retrieval"

Phebe Fowler
5 years ago
Views:

1 Informa(on Retrieval CISC489/ , Lecture #1 Monday, Feb. 9 Ben CartereFe Informa(on Retrieval 1

2 Informa(on Retrieval Domains, Applica(ons, and Tasks Web search Ver(cal search Enterprise search Media search Ques(on answering Recommender systems Adver(sing Personal item search Passage retrieval Filtering Summariza(on Clustering Topic detec(on Cross language Federated search Metasearch Social search Novel item retrieval 2

3 What is IR? Gerard Salton, 1968: Informa(on retrieval is a field concerned with the structure, analysis, organiza(on, storage, searching, and retrieval of informa(on. This class is about computa(onal methods for the structure, analysis, organiza(on, storage, searching, and retrieval of informa(on. And primarily about text documents. What is a Document? Examples: web pages, , books, news stories, scholarly papers, text messages, Word, Powerpoint, PDF, forum pos(ngs, patents, IM sessions, etc. Common proper(es: Significant text content. Some structure (e.g., (tle, author, date for papers; subject, sender, des(na(on for ). 3

4 Examples of Documents <DOC> <DOC> <DOC> <DOCNO>WSJ </DOCNO> <DOCNO>FR </DOCNO> <DOCNO>ZF </DOCNO> <DD> = </DD> <DOC> <DOCID>fr f2.A1067</DOCID> <DOCID> OV: &M; </DOCID> <AN> <DOCNO>DOE </DOCNO> </AN> <TEXT> <HL> Politics & Policy: <TEXT> <ITAG tagnum=69> <JOURNAL>PC Magazine Dec v9 n21 FDA Focuses Interpretation <ITAG tagnum=41>[docket * Full Text of the relative GI No. COPYRIGHT toxicities 89N-0432]</ITAG> Ziff-Davis Publishing Co. of On Eli Lilly cytotoxic drugs depends on the endpoint In Drug <DOC> Inquiry Histological <ITAG assays tagnum=56>par </JOURNAL> of the dynamics Pharmaceutical, of mitotic Inc.; and --- <DOCNO>AP </DOCNO> necrotic to cells Withdraw in murine Approval <TITLE>Generic crypts of revealed Three 3D Abbreviated Drafting. few New (Software Drug Review) (one Probe of <FILEID>AP-NR the Industry apparently Shifts Applications; radical 1612EST</FILEID> differences Opportunity of three evaluations between for a individual Hearing</ITAG> of low-cost 3D CAD programs in Generic Drugs Illegal Low-cost Activities CAD: modeling by Manufacturers for the masses. To Possible <FIRST>u Brand-Name w drugs AM-GenericDrugs and between drugs and 0740</FIRST> radiation. Manufacturing <SECOND>AM-Generic Problems microcolony <T2>SUMMARY: Drugs,740</SECOND> assay of clonogenic </T2>The (evaluation) Food cells and reveals Drug Administration --- <HEAD>FDA differences Chief Says (FDA) Agency between proposes </TITLE> Descrip-on: Needs drugs to More in withdraw the Power ability approval to of cells of abbreviated By Bill Punish Richards Cheating maintain and Bruce on new crypt Drug Ingersoll drug Tests</HEAD> integrity applications <AUTHOR>Haase, or to (ANDA's) Bruce.&M; regenerate , crypt-like , Staff Reporters <BYLINE>By To be relevant a document must iden(fy a specific generic drug company structures of DEBORAH The Wall MESCE</BYLINE> Street held Journal by </AUTHOR> Par Pharmaceutical, </HL> Inc., One Ram <DD> 08/24/89 <BYLINE>Associated </DD> </TEXT> Ridge Press Rd., Writer</BYLINE> Spring <SUMMARY>Generic Valley, NY Software (Par). Inc s The $349 Generic 3D being inves(gated by the FDA or Congress. It also must iden(fy the drug, i.e., <SO> WALL <DATELINE>WASHINGTON STREET JOURNAL </DOC> (J) grounds </SO> (AP) </DATELINE> the Drafting proposed is withdrawal a low-cost the generic drug for Zantac. are (1) that the <CO> LLY <TEXT> PRX </CO> applications contain </SUMMARY> untrue statements of material <DESCRIPT> <IN> DRUG The MANUFACTURERS Food and Drug (DRG) fact, Administration </IN> and (2) that, chief based told Congress new information <GV> FOOD on AND Friday DRUG the ADMINISTRATION agency evaluated needs (FDA) more together Company: </GV> authority with the Generic to punish evidence Software available Inc. (Products).&O; when <TEXT> generic drug companies the applications Product: that cheat on were safety approved, Generic tests there 3-D Drafting is a lack 1.1 of (CAD Food and and Drug misrepresent Administration data substantial investigators to win product evidence Software).&O; are approvals. that the drugs will have the looking into possible brand-name effects drug they manufacturing purport Topic: or are Computer-Aided represented Design to have problems </TEXT> at an Indianapolis under plant the owned conditions by Eli of Lilly use prescribed, recommended, & Co. </DOC> or suggested in </DESCRIPT> their labeling. </ITAG> <TEXT> </TEXT> </TEXT> </TEXT> </DOC> </DOC> </DOC> Documents vs. Database Records Database records are typically made up of well defined fields (or a<ributes). e.g. company names, addresses, account numbers, drug names, patent numbers, inves(ga(on file numbers. Easy to compare fields with well defined seman(cs to queries in order to find matches. Our query has no fields and our documents have lifle structure. 4

5 IR vs. Databases Informa-on Retrieval Data: Semi structured. Heterogeneous. Noisy. Unstructured or semistructured queries. Natural language seman(cs. Infrequent off line index changes. Databases Data: Structured. Homogeneous. Clean. Structured queries. Well defined field seman(cs. Frequent on line index changes. Generic Drugs Illegal Ac(vi(es by Manufacturers <DOC> <DOCNO>FR </DOCNO> <DOC> <DOC> <DOCID>fr f2.A1067</DOCID> <DOCNO>WSJ </DOCNO> <DOCNO>ZF </DOCNO> <DD> = </DD> <DOC> <TEXT> <DOCID> OV: &M; </DOCID> <AN> <DOCNO>DOE </DOCNO> <ITAG tagnum=69> </AN> <HL> Politics & Policy: <TEXT> <ITAG tagnum=41>[docket <JOURNAL>PC No. Magazine 89N-0432]</ITAG> Dec v9 n21 FDA Focuses Interpretation of the relative * Full Text GI toxicities COPYRIGHT Ziff-Davis of Publishing On Eli Lilly cytotoxic <ITAG drugs tagnum=56>par depends 1990.&M; on the Pharmaceutical, endpoint chosen. Inc.; In Drug <DOC> Inquiry Histological Withdraw assays of Approval the </JOURNAL> dynamics of Three of Abbreviated mitotic and New --- <DOCNO>AP </DOCNO> necrotic Applications; cells in murine Opportunity <TITLE>Generic crypts revealed for 3D a Hearing</ITAG> few Drafting. (Software Review) Probe of <FILEID>AP-NR the Industry apparently Shifts radical 1612EST</FILEID> differences of three between evaluations individual of low-cost 3D CAD programs To Possible <FIRST>uw Brand-Name drugs AM-GenericDrugs and <T2>SUMMARY: between drugs </T2>The Low-cost and 0740</FIRST> radiation. Food CAD: and modeling The Drug Administration for the masses. Manufacturing <SECOND>AM-Generic Problems microcolony (FDA) Drugs,740</SECOND> assay proposes of clonogenic (evaluation) to withdraw cells approval reveals of major --- <HEAD>FDA differences Chief Says new Agency between drug applications Needs drugs </TITLE> More in the Power (ANDA's) ability to , of cells , to By Bill Punish Richards Cheating maintain and Bruce on crypt Drug Ingersoll Tests</HEAD> integrity held by <AUTHOR>Haase, Par or Pharmaceutical, to regenerate Bruce.&M; crypt-like Inc., One Staff Reporters <BYLINE>By structures of DEBORAH Ridge The Wall MESCE</BYLINE> Rd., Spring Street Journal </AUTHOR> Valley, NY (Par). The </HL> <DD> 08/24/89 <BYLINE>Associated </DD> </TEXT> grounds Press Writer</BYLINE> for the <SUMMARY>Generic proposed withdrawal Software (1) Inc s that $349 the Generic 3D <SO> WALL <DATELINE>WASHINGTON STREET JOURNAL </DOC> applications (J) </SO> (AP) </DATELINE> contain Drafting untrue is a statements low-cost of material <CO> LLY <TEXT> fact, and (2) that, PRX </CO> </SUMMARY> based on new information <IN> DRUG The MANUFACTURERS Food and Drug evaluated (DRG) Administration together </IN> <DESCRIPT> chief with told the Congress evidence available when <GV> FOOD on AND Friday DRUG the ADMINISTRATION agency the needs applications (FDA) more </GV> authority Company: were approved, to Generic punish there Software is a Inc. lack (Products).&O; of <TEXT> generic drug companies substantial that cheat evidence Product: on safety that Generic the tests drugs 3-D will Drafting have 1.1 the (CAD Food and and Drug misrepresent Administration data effects investigators to win they product purport Software).&O; are approvals. or represented to have looking into under the conditions of use prescribed, recommended, possible brand-name drug manufacturing Topic: Computer-Aided Design problems </TEXT> or suggested in their labeling. at an Indianapolis plant owned by Eli Lilly & Co. </DOC> </ITAG> </DESCRIPT> <TEXT> </TEXT> </DOC> </TEXT> </TEXT> </DOC> </DOC> 5

6 Comparing Text Determining whether a document matches a query is a fundamental problem of IR. Exact match is not enough: Many different ways to state the same informa(on Documents may be relevant even when lacking some of the query terms. Documents may be nonrelevant even if they contain all the query terms. Relevance What does it mean for a document to be relevant? Simple defini(on: A relevant document contains informa(on that a person was looking for when they submifed a query to the search engine. Many factors influence a person s decision about what is relevant: e.g., task, context, novelty, style. Topical relevance (same topic) vs. user relevance (everything else). How can we build an engine that retrieves relevant documents? 6

7 Retrieval Retrieval models define a view of relevance. Ranking algorithms used in search engines are based on retrieval models. Most models describe sta(s(cal proper(es of text rather than linguis(c proper(es. i.e. coun(ng simple text features such as words. Sta(s(cal approach started with Luhn in the 50s. Linguis(c features can be part of a sta(s(cal model. Evalua(on How do we know whether the engine is doing a good job of finding relevant documents? Evalua(on is experimental procedures and measures for comparing system output with user expecta(ons. IR evalua(on methods now used in many fields. Recall and precision are examples of effec(veness measures. 7

8 Not Just Documents New applica(ons increasingly involve new media. e.g. video, photos, music, speech Like text, content is difficult to describe and compare. text may be used to represent them (e.g. tags). IR approaches to search and evalua(on are appropriate. Dimensions of IR Content Applica-ons Tasks Text Web search Ad hoc search Images Ver(cal search Filtering Video Enterprise search Classifica(on Scanned docs Desktop search Ques(on answering Audio Forum search Music P2P search Literature search 8

9 Ad hoc search: IR Tasks Find relevant documents for an arbitrary text query. Filtering: Iden(fy relevant user profiles for a new document. Classifica(on: Iden(fy relevant labels for documents. Ques(on answering: Give a specific answer to a ques(on. IR and Search Engines A search engine is the prac(cal applica(on of informa(on retrieval techniques to large scale text collec(ons. Relevance, retrieval, evalua(on are issues. So are users and informa(on needs, performance, coverage, upda(ng, scalability, adaptability, and ability to handle specific problems (like spam). 9

Components of a Search Engine Universe of things to organize and search Filter/crawler/domain/ Corpus query Query parser Parser/tokenizer User Interface f(q,d) Indexer Retrieval func(on Server(s)

10 Components of a Search Engine Universe of things to organize and search Filter/crawler/domain/ Corpus query Query parser Parser/tokenizer User Interface f(q,d) Indexer Retrieval func(on Server(s) Displayed results Retrieved results Building a Search Engine Text processing and indexing. Parsing; tokenizing; stopping and stemming; inverted indexes; scalability; index updates. Query processing and ranking. Query languages; index look up; retrieval models; features; relevance feedback; user interac(on. Evalua(on. Effec(veness at performing task; querying speed; user sa(sfac(on. 10

11 Course Overview This course is about informa(on retrieval in prac(ce: the applica(on of IR to search engine design and implementa(on. Course project: Design and implement a small search engine capable of indexing and searching Wikipedia pages. Evaluate its performance over provided queries. Add something interes(ng to it. Course Structure First half: Fundamentals of indexing, retrieval, and evalua(on. By the midterm we will have covered all aspects of designing a basic search engine. Second half: Addi(onal topics in search engine func(onality. Fielded search, user interac(on, clustering, linkgraph features, crawling, etc. 11

12 Textbook Search Engines: Informa(on Retrieval in Prac(ce by W. Bruce Crop, Donald Metzler, and Trevor Strohman. Unfortunately not yet published. I have PDFs of chapters. Also check supplemental texts on the course web page. Course Project Design and implement a small search engine to index and search Wikipedia pages. Semester long project in three phases: I. Indexing. II. Searching and evalua(ng. III. Addi(onal features. By midterm we will have covered everything needed to complete the first two phases. 12

13 Course Project: Phases For phases I and II, you will produce: A wrifen report of your design decisions and implementa(on details, including problems you encountered and how you resolved them. Code. Milestone worksheet responses. Timeline: Phase I: about 1.5 months. Phase II: about 1 month. Phase III: about 1 month. Course Project: Milestones Each phase has milestones to make sure you are not running into trouble. Worksheets with ques(ons you can answer using your code. Milestones and worksheets will be available in advance so you may work ahead if desired (we recommend it). If you are having trouble, we will be able to help you early. 13

14 Course Project: Phase III Phase III involves adding extra features to your engine. Anything we cover in the second half of the course, or anything in the book but not covered, or something else. You will write a 2 4 page proposal explaining how you would add the feature to your current code base. At the end of the semester you will give a short presenta(on on your engine. Course Project: Implementa(on This is a programming project! You may use any programming language the professor and/or TA understand. We highly recommend C, C++, or Java. You will have accounts on my lab cluster. No disk quotas; 16Gb RAM per node; 8 cores per node. Do not use for file sharing or other illicit ac(vity! 14

15 Course Project: Data I have obtained all English language Wikipedia pages. The top 10% with highest PageRank are provided for the project. 489 students must index and search 20% of those (2% of English Wikipedia). 689 students must index and search 100% of those (10% of English Wikipedia). Extra credit: index and search even more. Project Grading Project is 60% of total grade. Each phase is 20%. Phases I and II break down as follows: WriFen report (2 4 pages): 5% Code: 10% Turning in worksheets: 5% Phase III: Proposal (2 4 pages): 10% for 689, 15% for 489. Code: 5% for 689, 0% for 489. Final presenta(on: 5%. 15

16 Homeworks and Exams In addi(on to the project, there will be 5 homeworks and 2 exams (midterm and final). Each homework is 4% of total grade. Each exam is 10% of total grade. Exams will cover implementa(on details of project. Books and Resources Informa(on Retrieval, Keith van Rijsbergen. hfp:// Introduc(on to Informa(on Retrieval, Manning et al. hfp://www csli.stanford.edu/~hinrich/ informa(on retrieval book.html Check the course web page open! hfp:// 16

CS6200 Informa.on Retrieval. David Smith College of Computer and Informa.on Science Northeastern University

CS6200 Informa.on Retrieval. David Smith College of Computer and Informa.on Science Northeastern University CS6200 Informa.on Retrieval David Smith College of Computer and Informa.on Science Northeastern University Course Goals To help you to understand search engines, evaluate and compare them, and