Sec. 8.7 RESULTS PRESENTATION

Transcription

1 Sec. 8.7 RESULTS PRESENTATION 1

2 Sec. 8.7 Result Summaries Having ranked the documents matching a query, we wish to present a results list Most commonly, a list of the document titles plus a short summary, aka 10 blue links 2

3 Sec. 8.7 Summaries The title is often automatically extracted from document metadata. What about the summaries? This description is crucial. User can identify good/relevant hits based on description. Two basic kinds: Static Dynamic A static summary of a document is always the same, regardless of the query that hit the document A dynamic summary is a query-dependent attempt to explain why the document was retrieved for the query at hand 3

4 Sec. 8.7 Static Summaries In typical systems, the static summary is a subset of the document Simplest heuristic: the first 50 (or so this can be varied) words of the document Summary cached at indexing time More sophisticated: extract from each document a set of key sentences Simple NLP heuristics to score each sentence Summary is made up of top-scoring sentences. Most sophisticated: NLP used to synthesize a summary Seldom used in IR; cf. text summarization work 4

5 Sec. 8.7 Dynamic summaries Present one or more windows within the document that contain several of the query terms KWIC snippets: Keyword in Context presentation 5

6 Sec. 8.7 Techniques for Dynamic Summaries Find small windows in doc that contain query terms Requires fast window lookup in a document cache Score each window with regard to the query Use various features such as window width, position in document, etc. Combine features through a scoring. Challenges in evaluation: judging summaries Easier to do pair-wise comparisons rather than binary relevance assessments 6

7 Quicklinks For a navigational query such as united airlines user s need likely satisfied on Quick links provide navigational cues on that home page 7

8 8

9 Alternative results presentations? An active area of HCI research An alternative: / copies the idea of Apple s Cover Flow for search results (searchme recently went out of business) Here is what happened : 9

10 More alternatives to lists of hits Nexplore, Search cube,

11 Resources for this topic Chapter 3 Carbonell and Goldstein The use of MMR, diversity-based re-ranking for reordering documents and producing summaries. SIGIR

12 Improving Results Via Query Expansion

13 This Topic Improving results For high recall. E.g., searching for aircraft doesn t match with plane; nor thermodynamic with heat Options for improving results Global methods Query expansion Thesauri Automatic thesaurus generation Local methods Relevance feedback Pseudo relevance feedback We do not see clustering here

14 Relevance Feedback Sec Relevance feedback: user feedback on relevance of documents in the initial set of results User issues a (short, simple) query The user marks some results as relevant or non-relevant. The system computes a better representation of the information need based on the feedback. Relevance feedback can go through one or more iterations. Idea: it may be difficult to formulate a good query when you don t know the collection well, so iterate.

15 Sec. 9.1 Relevance feedback We will use ad hoc retrieval to refer to regular retrieval without relevance feedback. We now look at examples of relevance feedback that highlight different aspects.

16 Similar pages

17 Another form of feedback from Google, 2011

18 Sec Relevance Feedback: Example Image search engine

19 Results for Initial Query Sec

20 Relevance Feedback Sec

21 Sec Results after Relevance Feedback

22 Ad hoc results for query canine source: Fernando Diaz

23 Ad hoc results for query canine source: Fernando Diaz

24 User feedback: Select what is relevant source: Fernando Diaz

25 Results after relevance feedback source: Fernando Diaz

26 Sec Initial query/results Initial query: New space satellite applications , 08/13/91, NASA Hasn t Scrapped Imaging Spectrometer , 07/09/91, NASA Scratches Environment Gear From Satellite Plan , 04/04/90, Science Panel Backs NASA Satellite Plan, But Urges Launches of Smaller Probes , 09/09/91, A NASA Satellite Project Accomplishes Incredible Feat: Staying Within Budget , 07/24/90, Scientist Who Exposed Global Warming Proposes Satellites for Climate Research , 08/22/90, Report Provides Support for the Critics Of Using Big Satellites to Study Climate , 04/13/87, Arianespace Receives Satellite Launch Pact From Telesat Canada , 12/02/87, Telecommunications Tale of Two Companies User then marks relevant documents with +.

27 Sec Expanded query after relevance feedback new satellite nasa launch instrument bundespost rocket broadcast oil space application eos aster arianespace ss scientist earth measure

28 Sec Results for expanded query , 07/09/91, NASA Scratches Environment Gear From Satellite Plan , 08/13/91, NASA Hasn t Scrapped Imaging Spectrometer , 08/07/89, When the Pentagon Launches a Secret Satellite, Space Sleuths Do Some Spy Work of Their Own , 07/31/89, NASA Uses Warm Superconductors For Fast Circuit , 12/02/87, Telecommunications Tale of Two Companies , 07/09/91, Soviets May Adapt Parts of SS-20 Missile For Commercial Use , 07/12/88, Gaping Gap: Pentagon Lags in Race To Match the Soviets In Rocket Launchers , 06/14/90, Rescue of Satellite By Space Agency To Cost $90 Million

29 Sec Query Expansion In relevance feedback, users give additional input (relevant/non-relevant) on documents, which is used to reweight terms in the documents In query expansion, users give additional input (good/bad search term) on words or phrases

30 Query assist (auto-complete) Would you expect such a feature to increase the query volume at a search engine?

31 Sec How do we augment the user query? Manual thesaurus E.g. MedLine: physician, syn: doc, doctor, MD, medico Can be query rather than just synonyms Global Analysis: (static; of all documents in collection) Automatically derived thesaurus (co-occurrence statistics) Refinements based on query log mining Common on the web Local Analysis: (dynamic) Analysis of documents in result set

32 Improving Search Results Local Methods Global Method User Relevance Feedback Pseudo Relevance Feedback Query Expansion Thesauri Documents Analysis Manual Generation Automatic Generation Local Analysis Global Analysis

33 Example of manual thesaurus Sec

34 Sec Thesaurus-based query expansion For each term, t, in a query, expand the query with synonyms and related words of t from the thesaurus Dalhousie Dalhousie university May weight added terms less than original query terms. Generally increases recall Widely used in many science/engineering fields May significantly decrease precision, particularly with ambiguous terms. interest rate interest rate fascinate evaluate Jaguar panther in the wild There is a high cost of manually producing a thesaurus And for updating it for scientific changes

35 Sec Automatic Thesaurus Generation Attempt to generate a thesaurus automatically by analyzing the collection of documents Fundamental notion: similarity between two words Definition 1: Two words are similar if they co-occur with similar words. Definition 2: Two words are similar if they occur in a given grammatical relation with the same words. You can harvest, peel, eat, prepare, etc. apples and pears, so apples and pears must be similar. Co-occurrence based is more robust, grammatical relations are more accurate. Why?

36 Automatic Thesaurus Generation Example Sec

37 Automatic Thesaurus Generation Discussion Sec Quality of associations is usually a problem. Term ambiguity may introduce irrelevant statistically correlated terms. Apple computer Apple red fruit computer Problems: False positives: Words deemed similar that are not False negatives: Words deemed dissimilar that are similar Since terms are highly correlated anyway, expansion may not retrieve many additional documents.???

38 Indirect relevance feedback On the web, DirectHit introduced a form of indirect relevance feedback. DirectHit ranked documents higher that users look at more often. Clicked on links are assumed likely to be relevant Assuming the displayed summaries are good, etc. Globally: Not necessarily user or query specific. This is the general area of clickstream mining Today handled as part of machine-learned ranking

39 Resources Chapter 9 of the course book.