Queries. Inf 2B: Ranking Queries on the WWW. Suppose we have an Inverted Index for a set of webpages. Disclaimer. Kyriakos Kalorkoti

Transcription

1 Qeries Inf B: Ranking Qeries on the WWW Kyriakos Kalorkoti School of Informatics Uniersity of Edinbrgh Sppose e hae an Inerted Index for a set of ebpages. Disclaimer I Not really the scenario of Lectre. I Indexing for the eb is massie-scale: many distribted netorks orking in parallel. We search ith a term t. Index has many hits for t (say 6, for this t). Ho shold e rank them? A real search Ranking Qeries Inerted Index (probably) stores the freqency of the term t in each docment d (e.g., in preios lectre, or index contains f d,t ales). Idea Rank ansers to qeries in order of freqency of t in the arios ebpages. Problem Some great ebsites ill not een contain the term t. For example, there are not many occrrences of the term Uniersity of Edinbrgh" on Ne Idea Use strctre of eb to rank qeries.

2 Ranking Qeries sing eb strctre PageRank TM Principle: Link from one ebpage to another confers athority on the target ebpage. This is the concept behind: I The Hb-Athority model of Kleinberg. I PageRank TM ranking system of Google TM. In early 9s, hile PhD stdents at Stanford, Sergey Brin and Larry Page inented PageRank TM (and fonded Google TM ). Webgraph for a particlar qery: I ertices V =[N] here [N] ={,,...,N} corresponding to pages; I links are the directed edges of the graph, so E [N] [N]. Let G =(V, E). Recall: Definition Let denote some page [N] in the ebgraph. I In() is the set of in-edges to. The in-degree in() is in() = In(). I Ot() is the set of ot-edges from. The ot-degree ot() is ot() = Ot(). PageRank TM Principle of PageRank TM old se in-degree to measre ranking directly. Bt: I Want pages of high rank to confer more athority on the pages they link to. I A page ith fe links shold transfer more of its athority to its linked pages than one ith many links. Assmptions: (for basic PageRank TM ) I No dead-end" pages. I Eery page can hop to eery other page ia links. I Aperiodic. Let R() denote the rank of for any ebpage [N]. For eery ebpage in or collection, the folloing eqality shold hold: R() = X R()/ot() In() Rank of is the total amont of Rank gien from the incoming links to.

3 PageRank TM in matrix form PageRank TM in matrix form (R, R,...,R N ) = (R, R,...,R N ) p p... p N p p... p N p N p N... p NN A Shorthand ersion: R T = R T P, () here P =[p ],[N] and R is the ector of ranks for [N]. Eqialent to asking for here p = /ot(), if Ot();, otherise. R = P T R, () Looks like condition for R to be an eigenector of P T ith eigenale =. PageRank TM Example Qestions and Ansers I Ho do e kno that is an eigenale of the matrix P T? Anser: P T is a stochastic matrix (each colmn adds to ), so has eigenale. I If is an eigenale of P T, is it garanteed to be a simple eigenale? I i.e., any to ectors that satisfy P T R = R are the same p to a non-ero constant mltiple (linearly dependent). Anser: Under or assmptions, there is jst one linearly independent eigenector for. Example ebgraph retrned by a rare qery in ancient times.

4 Example Example Satisfies all the nice conditions for Basic PageRank TM model (no dead-end pages, can moe from any ertex x to any other ertex y, aperiodic). (R, R, R, R ) = (R, R, R, R ) Example (contined) Example (contined) (R, R, R, R )=(R, R, R, R ) an read-off" R = R /, and propagate this into matrix: (R, R, R, R )=(R, R, R, R ) No remoe R (keeping R = R / to side): B (R, R, R ) = (R, R, R

5 Example (contined) Alternatie (Eqialent) Approach) Expand ector-matrix prodct: B (R, R, R ) = (R, R, R A () (R, R R, R ) = (R, R, R Middle eqation reads R R = /(R R ), so R = R. Final eqation says R = /(R + R ), so R = R too. Soltion: R = R = R, R = R /. R = R + R + R R = R + R + R R = R R = R + R. I Sbtract the second eqation from the first: R R = R R I It follos that R = R. I Sbstitting into the forth eqation: R = R. I This method is probably preferable for sch small examples. Example (contined) General PageMark TM model I Remoe all or assmptions (dead-end pages, connectiity). I cannot be assmed to be. I Need to tinker the model. See Lectre Notes. Soltions are R = R = R, R = R /, i.e., (R, R, R, R )=c(,, /, ) here c is a constant. Not the same as conting in-degree (for this example).

6 Frther Reading Nothing in [GT] or [LRS]. Papers on the eb: I An Anatomy of a Large-Scale Hypertextal Web Search Engine, by Sergey Brin and Larence Page, 998. Online at: backrb/google.html I The PageRank itation Ranking: Bringing Order to the Web, by Page, Brin, Motani and Winograd, 998. Aailable online from: I Athoritatie Sorces in a Hyperlinked Enironment, by Jon Kleinberg. Aailable Online from Jon Kleinberg s ebpage: