Machine Learning Approaches to Information Retrieval

Transcription

1 Machine Learning Approaches to Information Retrieval Hang Li Microsoft Research Asia Joint Work with Jun Xu Yunbo Cao Guoping Hu

2 Talk Outline Introuction to Information Retrieval Search by Type Factoi Search New Learning Algorithm for Ranking Information Desk Summary

3 Introuction to Information Retrieval

4 What Is Information Retrieval? search I want to access information X uery information

5 Why Important?

6 Current Approach Information Retrieval = Ranking

7 General Moel for Ranking n ~ ~ ~ n n f f f uery or uestion ocuments information relevance scores for ranking D Q f

8 Relevance: Matching between Query an Document f

9 Probabilistic Moel n ~ ~ ~ n n r P r P r P uery or uestion ocuments relevant scores for ranking } { r r R D Q R P

10 Okapi or BM5 Robertson an Walker 994 ocuments ranking function n uery or uestion w + + tf w b k + tf w l k b avgl

11 Language Moe Ponte an Croft 998; Lafferty an Zhai 00 nl n n n n l l w w w w w w w w w = = = l w w w = ~ ~ ~ n n P P P ocument = bag of wors relevance scores for ranking

12 Classification Moel n uery or uestion ocuments relevance scores for ranking } { r r R D Q R P Logistic Regression or SVM ~ ~ ~ n n r P r P r P

13 Orinal Regression Moel Herbrich et al. 000; Joachims 00 n ~ ~ ~ n n f f f uery or uestion ocuments relevance scores for ranking } { k R D Q f R = Ranking SVM

14 General Moel for Ranking n n n f g f g f g uery or uestion ocuments scores for ranking D g D Q f

15 Page Rank Brin an Page 998 P i = α n + α P j M L j j i

16 Challenges Proximity Synonym an polysemi Quality an freshness of ocument webpage Spamming Information granularity Search nee unerstaning Evaluation Personalization Training ata collection relevance feeback click-through

17 Search by Document Type

18 Search by Document Type I want to fin ocuments relevant to X in type Y uery ocuments

19 Probabilistic Moel n uery with type ocuments ranking function } { } { t t T r r R D Q T R P D T P D Q R P D Q T R P

20 Example Manuals Search Query create an external link Document External Links External links are to Web sites external to your Web site. You can create external links by using buttons highlighting text or creating hot spots on images. For example creating a linkage to the Far Eastern Economic Review [FEER] one woul go to the FEER through the Internet. You might want to right click on the FEER logo or recent issue cover an save the image to your computer [must be save as a graphical interface file - GIF]. To make either the button or the text or the picture or all three interactive you highlight what you want to connect an then you click the hyperlink tool or the Eit-Hyperlink. Press the Worl Wie Web tab in the Create Hyperlink iaolgue box [see box below]. Whatever aress you have in the running Net browser will show up in the box or you can browse. In this case the FEER aress is there one only nee to press OK an the external hyperlink is create. All three examples to the right will link you to the FEER site. Document Creating an External Link To create an anchor that is a link to another ocument: Select to select by click an rag or by keyboar the text for the link you are creating. Click the Link button first case or select the entry "Create or change link" in the Links menu secon case. In the first case the cursor changes from an arrow to a han to let you click the target ocument. If the target ocument is isplaye in another Amaya winow click anywhere within that winow to create the link. If the target ocument is not isplaye in another Amaya winow press the F or Delete key or click a part of the ocument which cannot be a vali target. A ialog prompts you for the location of the target ocument. Type the URI of the target ocument an then Confirm to create the link. In the secon case a ialog prompts you for the location of the target ocument. If the target ocument is isplaye in another Amaya winow an you want to select it by click click the Click button then click anywhere within that winow to create the link. If the target ocument is not isplaye in another Amaya winow type the URI of the target ocument an then Confirm to create the link.

21 Manuals Search -- Experiment MS intranet ata 50 ueries from log of Microsoft Web. answers per uery from 5000 ocuments Evaluation Measure Results RR i = Rank i MRR Q i = = Q RR i Metho Type Only Relevance Only Combine Moel our approach MRR

22 Factoi Search

23 People Search Cao et al. 005 I want to fin people relate to X uery people

24 Probabilistic Moel n uery with type ocuments ranking function D Q E P P e P P e P e m e e entities

25 Example People Search Query Who knows about igital ink Document Jian Wang Ph.D. Senior Researcher & Research Manager Multimoal User Interface Group Microsoft Research Asia Dr. Jian Wang is Research Manager of the Multimoal User Interface Group at Microsoft Research Asia MSR Asia. Dr. Jian Wang's research specializations are ink an pen computing usability multimoal user interface virtual reality an human cognition. The Multimoal UI Group's current research projects inclue: avance igital ink parser igital ink annotation an representation of igital ink for Tablet PC. The group previously invente an inline input an correction user interface for Asian languages calle Moeless Input User Interface which allows Chinese users of Office XP to smoothly enter English an Chinese text without constantly switching between input language moes. Answer Jian Wang

26 People Search -- Experiment MSR Corpus 3 ueries searching in 309 ocuments 80 Caniates Evaluation Measure #{ persons which appear in both Top - 5 ranke Top 5 Precision = 5 caniates an groun truth} Results Metho Profile-base Moel Our Moel Top-5 Precision %

27 Time Search I want to fin time relate to X uery ates

28 Probabilistic Moel passages t t entities e e t n e m ranking function uery with type P E Q T e ~ f t = λ f t i i i

29 Time Search -- Example Query: Nixon visit China Document: Nixon's most significant achievement in foreign affairs may have been the establishment of irect relations with the People's Republic of China after a -year estrangement. Following a series of low-level iplomatic contacts in 970 an the lifting of U.S. trae an travel restrictions the following year the Chinese inicate that they woul welcome high-level iscussions an Nixon sent his national security aviser Henry Kissinger to China for secret talks. The thaw in relations became apparent with the ping-pong iplomacy conucte by American an Chinese table-tennis teams in reciprocal visits in Nixon's visit to China in February March 97 the first by an American presient while in office conclue with the Shanghai Communiué in which the Unite States formally recognize the one-china principle that there is only one China an that Taiwan is a part of China. Answer February March 97

30 Time Search -- Experiment MS intranet ata 00 ueries from log of containing when scheule ay an time 8.73 answers per uery from 0000 ocuments Evaluation Measure Results RR i = Rank i MRR Q i RR = = Q i Metho Best Baseline Learning MRR %

31 New Learning Algorithm for Ranking

32 Ranking Learning Given: S = { x y } X Ywhere Y = { r r } m i i i= Η= { h: X Y} hypothesis space L: Y Y R loss function Question: base on S fin h * = arg min E L h x y ERM: SRM: m * choose herm = argmin L h xi yi i= m * choose hsrm = argmin L h xi yi + λq h i=

33 Viewing Ranking as Classification Formulizing ranking problem as classification of example pairs Measuring the loss of h by inversions y y yˆ ˆ y Ly ˆ ˆ ˆ ˆ y y y = y y y y 0 otherwise

34 Viewing Ranking as Classification cont Given S = { x y } = m m i= j= m i i i fin h ERM that minimizes Ly y hx hx i j i j Euivalently fin a f h that minimizes x x z S' i i i L f x x z 0 h i i i + yi yj where S ' = xi x j z = : xi yi x j y j S yi y j

35 Ranking Moel Moel: instances are ranke by Ux

36 Simulation Experiment Low penalty Error rate of pairs NDCG curve High penalty High penalty Our approach Ranking SVM NDCG Our approach Ranking SVM r-r0 r-r r-r0 All N

37 Information Desk Li et al. 005

38

39 Features -- what is

40 Features who is Bill Gates Go What is Who is Where is homepage of Who knows about What is Who is Where is homepage of Who knows about Bill Gates US-Executive-Chairman + 45 XXXXXXX XXXXXX CHRMN & CHIEF SFTWR ARCHITECT Documents of Bill Gates8 My avice to stuents: Eucation counts Evento NET Reviewers Seattle 7/8 Novembro Anotacoes Dago.oc A Vision for Life Long Learning Year 00 Library/Vision for Eucation_Microsoft_Final.oc Bill Gates answers most freuently aske uestions. >>more Top 0 terms appearing in ocuments of Bill Gates Term Term Term Term Term Term Term Term Term Term

41 Features where is homepage of

42 Features - who knows about

43 Summary

44 Summary Information retrieval = helping people access information Currently search = ranking Matching between uery an ocument Our work Search by Document Type Factoi Search New learning algorithm for ranking Many issues to stuy

45 References Brin S. an Page T.998 The Anatomy of a Large-Scale Hypertextual Web Search Engine. In : Proceeings of the seventh international conference on Worl Wie Web. Hang Li Yunbo Cao Jun Xu Yunhua Hu Shenjie Li an Dmitriy Meyerzon A New Approach to Intranet Search Base on Information Extraction. Proc. of ACM-CIKM'05 inustry track Herbrich R. Graepel T. & Obermayer K Large Margin Rank Bounaries for Orinal Regression.. Avances in Large Margin Classifiers pp Joachims T. 00 Optimizing Search Engines Using Clickthrough Data Proceeings of the ACM Conference on Knowlege Discovery an Data Mining. Lafferty J. an Zhai C. 00. Document Language Moels Query Moels an Risk Minimization for Information Retrieval. SIGIR Ponte J. M. an Croft W. B A language moeling approach to information retrieval. In Proceeings of ACM-SIGIR pp Robertson S. E. an Walker S. 994 Okapi at TREC 3. In Proceeings of TREC Yunbo Cao Jingjing Liu Shenghua Bao an Hang Li 005 Microsoft Research Asia MSRA at Enterprise Track of TREC 005: Expert Search. In Proceeings of TREC.