68A8 Multimedia DataBases Information Retrieval - Exercises

Transcription

1 68A8 Multimedia DataBases Information Retrieval - Exercises Marco Gori May 31, 2004 Quiz examples for MidTerm (some with partial solution) 1. About inner product similarity When using the Boolean model, the similarity based on the inner product reduces to: (a) Finding the longest document (vector) aligned withe the assigned query; (b) Finding the longest document (vector) with the highest angle with respect to the query; (c) Counting the number of terms which are in common between the query and the document; 2. About the Inverse Document Frequency Given the classic definition of inverse document frequency (IDF) is IDF > 21 possible for a document collection of 4 million documents? (a) Yes; (b) No, regardless of the document frequency of the term. (c) No answer is possible with the given data; 3. Focus crawling The focus crawling process consists of: (a) An appropriate focussed recording of the indexes onto distributed servers; (b) A focussed extraction of information of a given category from the Web; (c) Neither of the previous answers. 4. Caching on search engines The pages retrieved by a search engines which are reachable by the discovered hyperlink might be different with respect to the corresponding cached version? (a) False. The crawling process guarantees data coherence; (b) False. Every change of web pages are automatically notified to the search engine by the publish and subscribe mechanism; 1

2 (c) True. The last download of the page might be done before the page has been changed; 5. Anchor text Can Google (used in traditional text mode) retrieve an image contained in a page which does not contain keywords used in the query? (a) No, images can only be retrieved provided that they are in a page containing keywords used in the query; (b) Yes, because of the anchor-text concept; (c) No, it is impossible. Images cannot be retrieved by search engines; 6. The PageRank damping parameter The damping parameter d in Google s PageRank has the following meaning: (a) It is the probability of following the links in the random walk model; (b) It is the probability of jumping abandoning the links in the random walk model; (c) It is the probability that the random walk ends in a sink; 7. Boolean Model - by prof. Futrelle Consider the four terms, in order: park, mountain, trails, difficult. Assume that the query, in disjunctive normal form, DNF, is the following, where is the logical disjunction operator: Q = (1,0,1,0) (0,1,1,0) (1,1,1,0) Write an English language description of this, which could be a straightforward translation from DNF Answer Search for a document containing park and trails, but not mountain or difficult. Or, search for a document containing trails and mountain but not difficult. Or, search for a document containing park and mountain and trails, but not difficult. Another way of saying this, is that no document should contain difficult. All should contain trails. All should contain park or mountain or both. This latter description is not the DNF form but easier to understand. Now consider the result of applying the query to the following two (tiny) documents. Which of the two are retrieved, if either? Explain briefly how you arrived at your conclusions. (a) Document 1: Loon park contains a lovely lake and is near Mystery mountain. It s not difficult to get to from the city. (b) Document 2: The Mystery mountain area has many easy trails, but no difficult ones. 2

3 Answer: Neither will be retrieved, because they both contain difficult. Oddly, the second one contains difficult in a negated form. But essentially no retrieval systems can t take negation in English into account. The intent of the query was probably to find a park or mountain without difficult trails. But finding just what you want is not easy! Experimenting with google shows that even when +difficulty is included, phrases such as Difficulty level: Easy appear. Not easy! 8. Vector space model Assume you index the terms Mars, landed and rover in the following document: Document: After a successful landing on Mars, the Mars rover Opportunity landed on a Mars plain in Meridiani section of Mars. The ship landed at an excellent landing spot. Assume that the number of documents in the total collection of 64 that contain Mars is 16, landed, 4 and rover, 8. Using these, compute the three weight vector components for the document. Ignore the stop words: the, a, an, of, on, in and at. Answer: The highest frequency word is Mars, with 4 occurrences. The absolute frequencies of the others are landed (2) and rover (1). This gives tf-idf factors of: (a) For Mars, f = 1 and idf = log(64/16) = 2 so w = tf idf = 2 for Mars. (b) For landed, f = 1/2 and idf = log(64/4) = 4 so w = tf idf = 2 for landed. (c) For rover, f = 1/4 and idf = log(64/8) = 3 so w = tf idf = 3/4 for rover. Note that it is just a coincidence that a keyword, Mars, has the highest absolute frequency in the document. 9. Evaluation: Recall and Precision An IR system returns 3 relevant documents, and 2 irrelevant documents. There are a total of 8 relevant documents in the collection. What is the precision of the system on this search, and what is its recall? Answer: The precision is given by p = 3/5 The recall is given by r = 3/8 10. Index size estimation Give arguments to conclude whether it is more reasonable that the inverted index takes about 10% the size of a document collection or about 300% 11. Stemming Mark these statements true/false 3

4 (a) In a Boolean retrieval system, stemming never lowers precision. (Answer: F) (b) Stemming increases the size of the lexicon. (Answer: F) (c) Stemming should be invoked at indexing time but not while doing a query (Answer F) 12. Cosine similarity and document-query normalized difference Given a document and a query in the vector space, consider the similarity defined as sim(q,d). = ˆd ˆq where ˆd = d d ˆq = q q Prove that sorting in ascendent order the documents according to the defined similarity corresponds with the cosine similarity. 4

5 information retrieval in hyperlink environments This exercise provides an in-depth experimentation of information retrieval in a hyperlink environment. The following points need to be addressed: 1. Construct a document collection composed of four pages and give them a hyperlink structure; 2. Define the set of keywords and give documents the corresponding representation, after having defined the stop word elimination and the stemming processes; 3. When neglecting the hyperlink structure of the collection, find a query q 1 such that the inner product similarity and the cosine similarity yields the same result. Likewise, find a query q 2 such that the results of the two similarity measures are different: 4. Let ρ(d,q) be the cosine similarity and denote by x(d) the PageRank value of d according to the constructed hyperlink structure. Let use define the following integrated notion of similarity which takes into account both the link analysis and the cosine query-document similarity as follows: φ(d,q). = ρ(d,q) x(d) (1) Find the new ranking corresponding to queries q 1 and q 2 and determine the effect of the damping parameter on the final ordering. 5

6 Figure 1: Hyperlink environment of the document collection Solution Sketch only! 1. Document collection: definition and hyperlink structure d 1 : The new generation of operation systems currently being developed in Apple 2 and Microsoft 4 will be based on information retrieval; d 2 : Apple has not officially announced to the market the new operating system based on information retrieval, yet. However, there is evidence to claim that they are operating with a project similar to Microsoft 4 s Longhorn. d 3 : Microsoft 4 and Apple 2 are currently leaders in operating systems; d 4 : Microsoft has officially announced that Longhorn will appear on the market on the year They are trying to fire Apple 2 which is working on a similar project. The hyperlink structure is represented by the graph of Fig Document representation Let us define the following set of keywords: t 1 : operating t 2 : system t 3 : Microsoft t 4 : Apple t 5 : information t 6 : retrieval t 7 : similar t 8 : project t 9 : Longhorn t 10 : market The documents can be represented by the following matrix D = 3. queries with different similarities

7 Consider q 1 : Longhorn; we see straightforwardly that the two similarities yield the same ordering. Now take q 2 : information retrieval. This time the results are different. 4. φ(d, q): integrated rank Calculate φ(d,q). = ρ(d,q) x(d) and sort accordingly... 7