Recap: Relevance Feedback. CS276A Text Information Retrieval, Mining, and Exploitation. Pseudo Feedback. Pseudo-Feedback: Performance.

Transcription

1 CS276A Tet Information Retrieval, Mining, and Eploitation Recap: Relevance Feedback Rocchio Algorithm: Lecture 9 5 Nov 2002 Typical weights: alpha = 8, beta = 64, gamma = 64 Tradeoff alpha vs beta/gamma: If we have a lot of judged documents, we want a higher beta/gamma. But we usually don t 2 Pseudo Feedback Pseudo-Feedback: Performance initial query apply relevance feedback retrieve documents label top k docs relevant documents top k documents 3 4 Today s topics The User in Information Access User Interfaces Browsing Visualization Find starting point Formulate/ Reformulate Query Send to system Receive results Information need Eplore results 5 User no Done? Stop yes 6

2 The User in Information Access Information Access in Contet Find starting point Formulate/ Reformulate Query Send to system Information Access Analyze Receive results Synthesize Information need User Focus of most IR! Eplore results no Done? Stop ye s 7 High-Level Goal User no Done? Stop yes 8 The User in Information Access Starting points Find starting point Information need User Formulate/ Reformulate Query Send to system Receive results Eplore results no Done? Stop yes 9 Source selection Highwire press Leis-neis Google! Overviews Directories/hierarchies Visual maps Clustering 10 Highwire Press Hierarchical browsing Source Selection Level 0 Level 1 Level

3 Visual Browsing: Themescape Browsing Scatter/Gather Starting point Scatter/gather allows the user to find a set of documents of interest through browsing. Take the collection and scatter it into n clusters. Pick the clusters of interest and merge them. Iterate Answer Credit: William Arms, Cornell Scatter/Gather Scatter/gather 17 18

4 How to Label Clusters Visual Browsing: Hyperbolic Tree Show titles of typical documents Titles are easy to scan Authors create them for quick scanning! But you can only show a few titles which may not fully represent cluster Show words/phrases prominent in cluster More likely to fully represent cluster Use distinguishing words/phrases But harder to scan Visual Browsing: Hyperbolic Tree Study of Kohonen Feature Maps H. Chen, A. Houston, R. Sewell, and B. Schatz, JASIS 49(7) Comparison: Kohonen Map and Yahoo Task: Window shop for interesting home page Repeat with other interface Results: Starting with map could repeat in Yahoo (8/11) Starting with Yahoo unable to repeat in map (2/14) 21 Credit: Marti 22Hearst Study (cont.) Participants liked: Correspondence of region size to # documents Overview (but also wanted zoom) Ease of jumping from one topic to another Multiple routes to topics Use of category and subcategory labels Study (cont.) Participants wanted: hierarchical organization other ordering of concepts (alphabetical) integration of browsing and search corresponce of color to meaning more meaningful labels labels at same level of abstraction fit more labels in the given space combined keyword and category search multiple category assignment (sports+entertain) Credit: Marti 23Hearst Credit: Marti 24Hearst

5 Browsing Searching vs. Browsing Effectiveness depends on Starting point Ease of orientation (are similar docs close etc, intuitive organization) How adaptive system is Compare to physical browsing (library, grocery store) 25 Information need dependent Open-ended (find an interesting quote on the virtues of friendship) -> browsing Specific (directions to Pacific Bell Park) -> searching User dependent Some users prefer searching, others browsing (confirmed in many studies: some hate to type) You don t need to know vocabulary for browsing. System dependent (some web sites don t support search) Searching and browsing are often interleaved. 26 Searchers vs. Browsers Eercise 1/3 of users do not search at all 1/3 rarely search (or urls only) Only 1/3 understand the concept of search (ISP data from 2000) 27 Observe your own information seeking behavior WWW University library Grocery store Are you a searcher or a browser? How do you reformulate your query? Read bad hits, then minus terms Read good hits, then plus terms Try a completely different query 28 The User in Information Access Query Specification Find starting point Information need User Formulate/ Reformulate Query Send to system Receive results Eplore results no Done? Stop yes 29 Recall: Relevance feedback Query epansion Spelling correction Query-log mining based Interaction styles for query specification Queries on the Web Parametric search Term browsing 30

6 Query Specification: Interaction Styles Shneiderman 97 Command Language Form Fillin Menu Selection Direct Manipulation Natural Language Eample: How do each apply to Boolean Queries Command-Based Query Specification command attribute value connector find pa shneiderman and tw user# What are the attribute names? What are the command names? What are allowable values? Credit: Marti 31Hearst Credit: Marti 32Hearst Form-Based Query Specification (Altavista) Form-Based Query Specification (Melvyl) Credit: Marti 33Hearst Credit: Marti 34Hearst Form-based Query Specification (Infoseek) Credit: Marti 35Hearst Credit: Marti 36Hearst

7 Menu-based Query Specification (Young & Shneiderman 93) Query Specification/Reformulation A good user interface makes it easy for the user to reformulate the query Challenge: one user interface is not ideal for all types of information needs Credit: Marti 37Hearst 38 Types of Information Needs Queries on the Web Most Frequent on 2002/10/26 Need answer to question (who won the game?) Re-find a particular document Find a good recipe for tonight s dinner Authoritative summary of information (HIV review) Eploration of new area (browse sites about Baja) Queries on the Web (2000) Intranet Queries (Aug 2000) 3351 bearfacts 3349 telebears 1909 etension 1874 schedule+of+classes 1780 bearlink 1737 bear+facts 1468 decal 1443 infobears 1227 calendar 989 career+center 974 campus+map 920 academic+calendar 840 map 773 bookstore 741 class+pass 738 housing 721 tele-bears 716 directory 667 schedule 627 recipes 602 transcripts 582 tuition 577 seti 563 registrar 550 info+bears 543 class+schedule 470 financial+aid 41 Source: Ray 42Larson

8 Intranet Queries Query Specification: Feast or Famine Summary of sample data from 3 weeks of UCB queries 13.2% Telebears/BearFacts/InfoBears/BearLink (12297) 6.7% Schedule of classes or final eams (6222) 5.4% Summer Session (5041) 3.2% Etension (2932) 3.1% Academic Calendar (2846) 2.4% Directories (2202) 1.7% Career Center (1588) 1.7% Housing (1583) 1.5% Map (1393) Average query length over last 4 months: 1.8 words This suggests what is difficult to find from the home page Famine Feast Specifying a well targeted query is hard. Bigger problem for Boolean. Source: Ray 43Larson 44 Parametric search Each document has, in addition to tet, some meta-data e.g., Language = French Format = pdf Subject = Physics etc. Date = Feb 2000 A parametric search interface allows the user to combine a full-tet query with selections on these parameters e.g., language, date range, etc. 45 Parametric search eample! " #$ %& '( )!* )+ %,' - ' $%,.(! ) /-( '( (0 0 ) " 46 Parametric search eample 12)'(--( 3 " Interfaces for term browsing 47 48

9 The User in Information Access Find starting point Information need Formulate/ Reformulate Query Send to system Receive results Eplore results User Done? no Stop yes Eplore Results Determine: Do these results answer my question? Summarization More generally: provide contet Hypertet navigation: Can I find the answer by following a link? Browsing and clustering (again) Browse to eplore results 51 Eplore Results: Contet We can t present complete documents in the result set too much information. Present information about each doc Must be concise (so we can show many docs) Must be informative Typical information about each document Summary Contet of query words Meta data: date, author, language, file name/url Contet of document in collection Information about structure of document 52 Contet in Collection: Cha-Cha Category Labels Advantages: Interpretable Capture summary information Describe multiple facets of content Domain dependent, and so descriptive Disadvantages Do not scale well (for organizing documents) Domain dependent, so costly to acquire May mis-match users interests 53 Credit: Marti 54Hearst

10 Evaluate Results Contet in Hierarchy: Cat-a-Cone Eplore Results: Summarization Query-dependent summarization KWIC (keyword in contet) lines (a la google) Query-independent summarization Summary written by author (if available) Eploit genre (news stories) Sentence etraction Natural language generation Evaluate Results Structure of document: SeeSoft Personalization User Query Outride Personalized Search System Interests Demographics Query Augmentation Intranet Search Result Processing Click Stream Search History Result Set Web Search Application Usage 57 $- (.! " $#! %& ' )( * +), + 58 How Long to Get an Answer? O u t r i d e 38.9 G o o g l e 75.4 Y a h o o! 81 E c i t e 83.5 A O L / :<;=?>$@ A 2?B C =?D$7EC>F2GCDIH 2 J =?DK L M N O P Q R S T U V W X Y Z [ Z X \ ] ^ _ ` a^ b c M b \ d W Z e f g g g

11 A E/AGF5HCIC@CD J A K L :9-M. N9O -QP,4 RS-UT VXW5V3RQY86 -CT -QPCN5+ 23Z Search Engine User Actions Difference (%) Outride 11.2 Google Yahoo! AOL Ecite Average Table 1. User actions study results. Eperienced Users Novice Users Overall Engine Epert Rank Novice Rank Average Rank % Time Time Difference Outride 32.8 (1) 45.1 (1) 38.9 (1) 0% AOL 92.3 (5) 87.0 (4) 89.6 (5) 130.2% Ecite 75.7 (3) 91.3 (5) 83.5 (4) 114.5% Google 72.5 (2) 78.4 (3) 75.4 (2) 93.7% Yahoo! 85.1 (4) 76.9 (2) 81.0 (3) 107.9% Table 2. Overall timing results (in seconds, with placement in parenthesis). 61 ) ' ( &% "# $!! Novice *,+ -/ :9-/ Eperts Others Outride Performance of Interactive Retrieval Boolean Queries: Interface Issues 63 [ Boolean logic is difficult for the average user. [ Much research was done on interfaces facilitating the creation of boolean queries by non-eperts. [ Much of this research was made obsolete by the web. [ Current view is that non-epert users are best served with non-boolean or simple +/- boolean (pioneered by altavista). [ But boolean queries are the standard for certain groups of epert users (eg, lawyers). 64 User Interfaces: Other Issues [ Technical HCI issues \ How to use screen real estate \ One monolithic window or many? \ Undo operator \ Give access to history \ Alternative interfaces for novel/epert users [ Disabilities Take-Away [ Don t ignore the user in information retrieval. [ Finding matching documents for a query is only part of information access and knowledge work. [ In addition to core information retrieval, information access interfaces need to support \ Finding starting points \ Formulation/reformulation of queries \ Eploring/evaluating results 65 66

12 Eercise Current information retrieval user interfaces are designed for typical computer screens. How would you design a user interface for a wallsize screen? Resources MIR Ch Donna Harman, Overview of the fourth tet retrieval conference (TR EC 4), National Institute of Standards and Technology. Cutting, Karger, Pedersen, Tukey. Scatter/Gather. ACM SIGIR. Hearst, Cat-a-cone, an interactive interface for specifying searches and viewing retrieving results in a large category hierarchy, ACM SIGIR