Solutions for Homework 2

Transcription

1 Solutios for Homework 2 IIR Book: Exercise.2 (0.5 ) Cosider these documets: Doc breakthrough drug for schizophreia Doc 2 ew schizophreia drug Doc 3 ew approach for treatmet of schizophreia Doc 4 ew hopes for schizophreia patiets a. Draw the term documet icidece matrix for this documet collectio. b. Draw the iverted idex represetatio for this collectio, as i Figure.3 (page 6). a. Term documet icidece matrix Doc Doc2 Doc3 Doc4 approach breakthrough drug 0 0 for 0 hopes ew 0 of patiets schizophreia treatmet b. iverted idex represetatio for this collectio (chage the order betwee hopes ad for ) approach 3 breakthrough drug 2 for 3 4 hopes 4 ew of 3 patiets 4

2 schizophreia treatmet 3 Exercise.3 (0.5 ) For the documet collectio show i Exercise.2, what are the retured results for these queries: a. schizophreia AND drug b. for AND NOT(drug OR approach) a. Doc, Doc 2 b. Doc 4 Exercise 4. ( ) Apply MapReduce to the problem of coutig how ofte each term occurs i a set of files. Specify map ad reduce operatios for this task. Write dow a example alog the lies of Figure 4.6. (should follow the example i Figure 4.6). Method : Schema: map: iput > list(k, v) reduce: (k, list(v)) > output Istatiatio of the schema for term coutig map: a set of files > list(term, ) reduce: <(term, ), (term2, ), (term3, ) > > list(term, total cout) Example for term coutig map: d: I hear, I forget. d2: I see, I remember. > <I, > <hear, ><I, ><forget > <I, ><see, ><I, ><remember > reduce: <I,(,,,)><hear, ><forget, ><see, ><remember, > > <I, 4><hear, ><forget, ><see, ><remember, > Method 2: Schema: map: iput > list(k, v) reduce: (k, list(v)) > output Istatiatio of the schema for term coutig map: a set of files > list(term, cout i oe file) reduce: <(term, cout), (term2, cout2), (term3, cout3) > > list(term, total cout) Example for term coutig map: d: I hear, I forget. d2: I see, I remember. > <I, 2> <hear, ><forget > <I, 2><see, ><remember >

3 reduce: <I,(2, 2)><hear, ><forget, ><see, ><remember, > > <I, 4><hear, ><forget, ><see, ><remember, > Exercise 6.0 (0.5 ) Cosider the table of term frequecies for 3 documets deoted Doc, Doc2, Doc3 i Figure 6.9. Compute the tf-idf weights for the terms car, auto, isurace, best, for eachdocumet, usig the idf values from Figure 6.8. Solutio (raw term frequecy weightig) Doc Doc2 Doc3 car auto isurace best Solutio 2(log term frequecy weightig) Doc Doc2 Doc3 car auto isurace best Exercise 6.7 ( ) With term weights as computed i Exercise 6.5, rak the three documets by computed score for the query car isurace, for each of the followig cases of term

4 weightig i the query:. The weight of a term is if preset i the query, 0 otherwise. 2. Euclidea ormalized idf. Solutio : Normalize the raw tf-idf weights computed i Ex. 6.0, we get Doc Doc2 Doc3 car auto isurace best The query is q = [, 0,, 0] score(q, doc)= , score(q, doc2) = , score(q, doc3) =.305 Rakig: doc3, doc, doc2 2. The query is q = [.62, 0,.6, 0] score(q, doc) =.4807, score(q, doc2) =.74, score(q, doc3) = Rakig: doc3, doc, doc2 Solutio 2: Normalize the log tf-idf weights computed i Ex. 6.0, we get Doc Doc2 Doc3 car auto isurace best The query is q = [, 0,, 0] score(q, doc)= , score(q, doc2) = , score(q, doc3) =.239 Rakig: doc3, doc2, doc 2. The query is q = [.62, 0,.6, 0] score(q, doc) =.048, score(q, doc2) =.535, score(q, doc3) =.9846 Rakig: doc3, doc2, doc Chapter 5 of MMDS Textbook: 5.. (0.5 ) The trasitio matrix for the graph is: /3 /2 0 /3 0 /2 /3 /2 /2 By equatio method (Mλ λ, we get the result λ,, By iteratio method, we get the followig list*:

5 , , 0.348, , (0.5 ) The iteratio process is : v βmvβe/ 4/5 2/5 0 /5 4/5 0 2/5 v/5 4/5 2/5 2/5 /5 If we solve this equatio directly, we get v,, By iteratio method, we get the followig iteratio list*: , , 0.325, , ( ) The matrix A for the graph is: β β β Easy to see that all odes i the clique have the same PageRak value, so we suppose vector v to be [x, x,, x, y]t, where x is PageRak of ode i the clique, ad y represets the additioal ode outside the clique. The we get the equatio x xy Solve the equatio, we get x β y β β Ad the correspodig vector will be give (0.5 ) There will be oly oe ode, the head ode with a self directio, ad PageRak for this ode is. PageRak for all the remaiig odes will be /2.

6 5.2.2 ( ) a) A 3 B, C, D B 2 A, D C E D 2 B, C b) a 3 a, b, c b 2 a, c c 2 b, c ( ) The four ode graph is divided ito four 2 by 2 blocks (M, M2, M2, M22). M: A 3 B B 2 A M2: D 2 B M2: A 3 C, D B 2 D M22: D 2 C 5.3. ( ) The trasitio matrix of Figure 5.5 is: 0 /2 0 /3 0 0 /2 /3 0 0 /2 /3 /2 0 0 Suppose β is 0.8 a) 0 2/5 4/5 0 /5 v 4/ /5 0 v 4/ /5 0 4/5 2/

7 If we solve this equatio directly, we get v,,, By iteratio method, we get the followig list is: b) ,,,,, /5 v 4/5 0 4/5 0 4/5 2/5 4/5 0 /0 0 2/5 0 v 0 2/5 / If we solve this equatio directly, we get v,,, By iteratio method, we get the followig list is: ,,,,, ( ) The lik matrix for Figure 5. is: 0 L Iitialize a ad h to be all s. give two equatios hla al h By iteratio method, we get the followig list is: h: a: