Page rank computation HPC course project a.y Compute efficient and scalable Pagerank
|
|
- Coral Conley
- 6 years ago
- Views:
Transcription
1 Page rank computation HPC course project a.y Compute efficient and scalable Pagerank 1
2 PageRank PageRank is a link analysis algorithm, named after Brin & Page [1], and used by the Google Internet search engine, which assigns a numerical weighting to each element of a hyperlinked set of documents, such as the World Wide Web, with the purpose of "measuring" its relative importance within the set [wikipedia] [1] Brin, S. and Page, L. (1998) The Anatomy of a Large-Scale Hypertextual Web Search Engine. In: Seventh International World-Wide Web Conference (WWW 1998), April 14-18, 1998, Brisbane, Australia 2
3 PageRank: the intuitive idea PageRank relies on the democratic nature of the Web by using its vast link structure as an indicator of an individual page's value or quality. PageRank interprets a hyperlink from page x to page y as a vote, by page x, for page y However, PageRank looks at more than the sheer number of votes; it also analyzes the page that casts the vote. Votes casted by important pages weigh more heavily and help to make other pages more "important" This is exactly the idea of rank prestige in social network. 3
4 More specifically A hyperlink from a page to another page is an implicit conveyance of authority to the target page. The more in-links that a page i receives, the more prestige the page i has. Pages that point to page i also have their own prestige scores. A page of a higher prestige pointing to i is more important than a page of a lower prestige pointing to i In other words, a page is important if it is pointed to by other important pages 4
5 PageRank algorithm According to rank prestige, the importance of page i (i s PageRank score) is the sum of the PageRank scores of all pages that point to i Since a page may point to many other pages, its prestige score should be shared. The Web as a directed graph G = (V, E) The PageRank score of the page i (denoted by P(i)) is defined by: P(i) = ( j,i) E P( j) O j O j is the number of out-link of j 5
6 Matrix notation Let n = V be the total number of pages We have a system of n linear equations with n unknowns. We can use a matrix to represent them. Let P be a n-dimensional column vector of PageRank values, i.e., P = (P(1), P(2),, P(n)) T Let A be the adjacency matrix of our graph with A ij = # % $ &% 1 O i if (i, j) E 0 otherwise We can write the n equations P(i) = P = A T P (PageRank) ( j,i) E P( j) O j with 6
7 Solve the PageRank equation P = A T P This is the characteristic equation of the eigensystem, where the solution to P is an eigenvector with the corresponding eigenvalue of 1 It turns out that if some conditions are satisfied, 1 is the largest eigenvalue and the PageRank vector P is the principal eigenvector. A well known mathematical technique called power iteration can be used to find P Problem: the above Equation does not quite suffice because the Web graph does not meet the conditions. 7
8 Using Markov chain To introduce these conditions and the enhanced equation, let us derive the same above Equation based on the Markov chain. In the Markov chain, each Web page or node in the Web graph is regarded as a state. A hyperlink is a transition, which leads from one state to another state with a probability. This framework models Web surfing as a stochastic process. Random walk It models a Web surfer randomly surfing the Web as state transition. 8
9 Random surfing Recall we use O i to denote the number of out-links of a node i Each transition probability is 1/O i if we assume the Web surfer will click the hyperlinks in the page i uniformly at random. the back button on the browser is not used the surfer does not type in an URL 9
10 Transition probability matrix Let A be the state transition probability matrix: " $ $ $ A = $. $ $ $ # A 11 A A 1n % ' A 21 A A 2n '... ' '... '... ' ' A n1 A n 2... A nn & A ij represents the transition probability that the surfer in state i (page i) will move to state j (page j). Can A be the adjacency matrix previously discussed? 10
11 Let us start Given an initial probability distribution vector that a surfer is at each state (or page) p 0 = (p 0 (1), p 0 (2),, p 0 (n)) T (a column vector) an n n transition probability matrix A we have n p 0 (i) =1 i=1 n A ij =1 (1) j=1 If the matrix A satisfies Equation (1), we say that A is the stochastic matrix of a Markov chain 11
12 Back to the Markov chain In a Markov chain, a question of common interest is: What is the probability that, after m steps/transitions (with m ), a random process/walker reaches a state j independently of the initial state of the walk We determine the probability that the random surfer arrives at the state/page j after 1 step (1 transition) by using the following reasoning: p 1 ( j) = n i=1 where A ij (1) is the probability of going from i to j after 1 step. At beginning p 0 (i) = 1/N A ij (1) p 0 (i) for all i 12
13 State transition We can write this in matricial form: P 1 = A T P 0 In general, the probability distribution after k steps/transitions is: P k = A T P k -1 13
14 Stationary probability distribution By the Ergodic Theorem of Markov chain a finite Markov chain defined by the stochastic matrix A has a unique stationary probability distribution if A is irreducible and aperiodic The stationary probability distribution means that after a series of transitions p k will converge to a steady-state probability vector π, i.e., lim P = π k k 14
15 PageRank again When we reach the steady-state, we have P k = P k+1 =π, and thus π =A T π π is the principal eigenvector (the one with the maximum magnitude) of A T with eigenvalue of 1 In PageRank, π is used as the PageRank vector P: P = A T P 15
16 Is P = π justified? Using the stationary probability distribution π as the PageRank vector is reasonable and quite intuitive because it reflects the long-run probabilities that a random surfer will visit the pages. a page has a high prestige if the probability of visiting it is high 16
17 Back to the Web graph Now let us come back to the real Web context and see whether the above conditions are satisfied, i.e., whether A is a stochastic matrix and whether it is irreducible and aperiodic. None of them is satisfied. Hence, we need to extend the ideal-case to produce the actual PageRank model. 17
18 A is a not stochastic matrix A is the transition matrix of the Web graph A ij = # % $ &% It does not satisfy equation: 1 O i if (i, j) E 0 otherwise n j=1 A ij =1 because many Web pages have no out-links (dangling pages) This is reflected in transition matrix A by some rows of 0 s 18
19 An example Web hyperlink graph " % $ ' $ ' $ ' A = $ ' $ ' $ ' $ ' # & 19
20 Fix the problem: two possible ways 1. Remove pages with no out-links during the PageRank computation these pages do not affect the ranking of any other page directly 2. Add a complete set of outgoing links from each such page i to all the pages on the Web. Let us use the 2 nd method: " % $ ' $ ' $ ' A = $ ' $ ' $ ' $ ' # & 20
21 A is a not irreducible Irreducible means that the Web graph G is strongly connected Definition: A directed graph G = (V, E) is strongly connected if and only if, for each pair of nodes u, v V, there is a directed path from u to v. A general Web graph represented by A is not irreducible because for some pair of nodes u and v, there is no path from u to v In our example, there is no directed path from nodes 3 to 4 21
22 A is a not aperiodic A state i in a Markov chain being periodic means that there exists a directed cycle (from i to i) that a random walker traverses multiple times Definition: A state i is periodic (with period k > 1) if k is the smallest number such that all paths leading from state i back to state i have a length that is a multiple of k A Markov chain is aperiodic if all states are aperiodic. 22
23 An example: periodic This a periodic Markov chain with k = 3 If we begin from state 1, to come back to state 1 the only path is for some number of times, say h Thus any return to state 1 will take k h = 3h transitions. 23
24 Deal with irreducible and aperiodic matrices It is easy to deal with the above two problems with a single strategy. Add a link from each page to every page and give each link a small transition probability controlled by a parameter d Obviously, the augmented transition matrix becomes irreducible and aperiodic it becomes irreducible because it is strongly connected it become aperiodic because we now have paths of all the possible lengths from state i back to state i 24
25 Improved PageRank After this augmentation, at a page, the random surfer has two options With probability d, 0<d<1, she randomly chooses an outlink to follow With probability 1-d, she stops clicking and jumps to a random page The following equation models the improved model: P = ((1 d) E n + dat )P n is important, since the matrix has to be stochastic where E is a n n square matrix of all 1 s 25
26 Follow our example d = 0.9 Transposed matrix (1 d) E n + dat = The matrix made stochastic, which is still: periodic (see state 3) reducible (no path from 3 to 4) " % $ ' $ ' $ ' A = $ ' $ ' $ ' $ ' # & # & % ( % ( % ( % ( % ( % ( % ( $ ' 26
27 The final PageRank algorithm (1-d)E/n + da T is a stochastic matrix (transposed). It is also irreducible and aperiodic Note that E = e e T where e is a column vector of 1 s e T P = 1 since P is the stationary probability vector π If we scale this equation: P = ((1 d) E n + dat )P = (1 d) 1 n e et P + da T P = by multiplying both sides by n, we have: e T P = n and thus: = (1 d) 1 n e + dat P P = (1 d)e + da T P 27
28 The final PageRank algorithm (cont ) Given: P = (1 d ) e + da PageRank for each page i is: n j=1 that is equivalent to the formula given in the PageRank paper [BP98] The parameter d is called the damping factor which can be set to between 0 and 1. d = 0.85 was used in the PageRank paper T P P(i) = (1 d) + d A ji P( j) P(i) = (1 d) + d [BP98] Sergey Brin and Lawrence Page. The Anatomy of a Large-Scale Hypertextual Web Search Engine. WWW Int.l Conf., A ji = # % $ & % 1 O j ( j,i) E if ( j,i) E 0 otherwise P( j) O j 28
29 Compute PageRank Use the power iteration method Initialization 0 n Norm 1 less than
30 Again PageRank Without scaling the equation (by multiplying by n), we have e T P = 1 (i.e., the sum of all PageRanks is one), and thus: P(i) = 1 d n + d P( j) O ( j,i) E j Important pages are cited/pointed by other important ones In the example, the most important is ID=1 P(ID=1) = P(ID=1) distributes is rank among all its 5 outgoing links ID= 2, 3, 4, 5, = *
31 Again PageRank Without scaling the equation (by multiplying by n), we have e T P = 1 (i.e., the sum of all PageRanks is one), and thus: P(i) = 1 d n + d P( j) O ( j,i) E j The stationary probability P(ID=1) is obtained by: (1-d)/n + d ( )= (0.15)/ ( )=
32 1 st Assignment Write a sequential code (C or C++) that implements Pagerank Compile the code with O3 option, and measure the execution times (command time) for some inputs Input graphs: Test example: P[2] = P[1] = P[0] =
33 Hand in (1 st assignment) Create a tar/zip file with: your solution source code and the Makefile; a readme file a brief report (PDF) Groups of max 2 people Send me an (orlando@unive.it) with the composition of each group How to present the assignment Register in moodle.unive.it (High Performance Computing [CM0227]) and submit the assignment by Nov. 25 th 33
34 2 nd assignment Given the original incidence matrix A[][], if we know which are the dangling nodes, we can avoid filling zero-rows with values 1/n A T /n * * p k = p k Dangling nodes A T /n * * * p k = p k+1 34
35 2 nd assignment A T * p k + 1/n * * * p k = p k+1 A T * p k + X... i2danglings X i2danglings p k [i] n p k [i] n = p k+1 35
36 2 nd assignment Avoid transposing matrix A[][]! Still traverse A[][] in row major order for (i=0; i<n; i++) for (j=0; j<n; j++) p_new[j] = p_new[j] + a[i][j] * p[i]; Store matrix A[][] in sparse compressed form Compressed sparse row (CSR or CRS) 36
37 2 Assignment Compressed sparse row (CSR or CRS) Used for traversing matrix in row major order val col_ind row_ptr Start row 0 Position where the n-th row should Start row 1 start. Note that the matrix is sparse, and thus the row could be completely zero. Start row 2 In this case row_ptr[n] = row_prt[n+1] Start row 3 Start row 4 Start row 5 Start row 6 (1 more position) 37
38 2 Assignment Store big data like A[][] on a file Once we map the file to a memory region, we access it via pointers, just as you would access ordinary variables and objects You can mmap specific section/partition of the file, and share the files between more threads #include <stdio.h>! #include <sys/mman.h>! #include <sys/stat.h>! #include <fcntl.h>! #include <unistd.h>! #include <stdlib.h>!! int main() {! int i;! float val;! float *mmap_region;!! FILE *fstream;! int fd;! 38
39 2 Assignment! /* create the file */! fstream = fopen("./mmapped_file", "w+");! for (i=0; i<10; i++) {! val = i ;!! /* write a stream of binary floats */! fwrite(&val, sizeof(float), 1, fstream);! }! fclose(fstream);! /* map a file to the pages starting at a given address for a given length */! fd = open("./mmapped_file", O_RDONLY);! mmap_region = (float *) mmap(0, 10*sizeof(float), PROT_READ,!!!! MAP_SHARED, fd, 0);! if (mmap_region == MAP_FAILED) {!! close(fd);!! printf("error mmapping the file");!! exit(1);! }! close(fd);! Starting offset address in the file 39
40 2 Assignment!!!! }! /* Print the data mmapped */! for (i=0; i<10; i++)! printf("%f ", mmap_region[i]);! printf("\n");! /* free the mmapped memory */!!if (munmap(mmap_region, 10*sizeof(float)) == -1) {!! printf("error un-mmapping the file");! exit(1);!!}! 40
41 Hand in (2 nd assignment) Compile the code with O3 option, and measure the execution times (command time) for some (large) inputs Time as a function of number of nodes/edges Some example of graphs are available here: Create a tar/zip file with: your solution source code and the Makefile; a readme file a brief report (PDF) How to present the assignment Register in moodle.unive.it (High Performance Computing [CM0227]) and submit the assignment Dec. 9 th 41
42 3rd assignment The goal of this assignment is to parallelize the optimized code of the 2 assignment You can use shared or message passing (also hybrid) parallelization Measure speedup and efficiency as a function of processors/cores exploited (for a couple data sets) Point out the effects of the Amdahl law, concerning the serial sections that remain serial e.g., the input output phases if you are not able to parallelize Measure how the execution time changes when we increase the problem size, without changing the number of processors/cores employed This requires to consider subsets of nodes and edges of a given input graph 42
43 3rd assignment The issues to solve concern decision such as the right decomposition, the right granularity, and a strategy (static/dynamic) of task assignment I would only to point out that, if we don t transpose matrix A, and decomposes the problem over the input, we have: A * p k = p k+1 A * p k = p k+1 + reduce p k+1 A * p k = p k+1 43
44 Hand in (3 rd assignment) Compile the code with O3 option, measure the execution time, by also profiling the code with specific routines as MPI_Wtime(), or gettimeofday() if you don t use MPI) search examples of usage of gettimeofday() with a search engine Create a tar/zip file with: your solution source code and the Makefile; a readme file a brief report (PDF) How to present the assignment Register in moodle.unive.it (High Performance Computing [CM0227]) and submit the assignment Jan. 13 th 44
Link Analysis and Web Search
Link Analysis and Web Search Moreno Marzolla Dip. di Informatica Scienza e Ingegneria (DISI) Università di Bologna http://www.moreno.marzolla.name/ based on material by prof. Bing Liu http://www.cs.uic.edu/~liub/webminingbook.html
More informationLink Analysis from Bing Liu. Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data, Springer and other material.
Link Analysis from Bing Liu. Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data, Springer and other material. 1 Contents Introduction Network properties Social network analysis Co-citation
More information1 Starting around 1996, researchers began to work on. 2 In Feb, 1997, Yanhong Li (Scotch Plains, NJ) filed a
!"#$ %#& ' Introduction ' Social network analysis ' Co-citation and bibliographic coupling ' PageRank ' HIS ' Summary ()*+,-/*,) Early search engines mainly compare content similarity of the query and
More informationWeb search before Google. (Taken from Page et al. (1999), The PageRank Citation Ranking: Bringing Order to the Web.)
' Sta306b May 11, 2012 $ PageRank: 1 Web search before Google (Taken from Page et al. (1999), The PageRank Citation Ranking: Bringing Order to the Web.) & % Sta306b May 11, 2012 PageRank: 2 Web search
More informationInformation Retrieval. Lecture 11 - Link analysis
Information Retrieval Lecture 11 - Link analysis Seminar für Sprachwissenschaft International Studies in Computational Linguistics Wintersemester 2007 1/ 35 Introduction Link analysis: using hyperlinks
More informationCOMP Page Rank
COMP 4601 Page Rank 1 Motivation Remember, we were interested in giving back the most relevant documents to a user. Importance is measured by reference as well as content. Think of this like academic paper
More informationLecture #3: PageRank Algorithm The Mathematics of Google Search
Lecture #3: PageRank Algorithm The Mathematics of Google Search We live in a computer era. Internet is part of our everyday lives and information is only a click away. Just open your favorite search engine,
More informationLecture 8: Linkage algorithms and web search
Lecture 8: Linkage algorithms and web search Information Retrieval Computer Science Tripos Part II Ronan Cummins 1 Natural Language and Information Processing (NLIP) Group ronan.cummins@cl.cam.ac.uk 2017
More informationWeb consists of web pages and hyperlinks between pages. A page receiving many links from other pages may be a hint of the authority of the page
Link Analysis Links Web consists of web pages and hyperlinks between pages A page receiving many links from other pages may be a hint of the authority of the page Links are also popular in some other information
More informationInformation Networks: PageRank
Information Networks: PageRank Web Science (VU) (706.716) Elisabeth Lex ISDS, TU Graz June 18, 2018 Elisabeth Lex (ISDS, TU Graz) Links June 18, 2018 1 / 38 Repetition Information Networks Shape of the
More informationAgenda. Math Google PageRank algorithm. 2 Developing a formula for ranking web pages. 3 Interpretation. 4 Computing the score of each page
Agenda Math 104 1 Google PageRank algorithm 2 Developing a formula for ranking web pages 3 Interpretation 4 Computing the score of each page Google: background Mid nineties: many search engines often times
More informationInformation Retrieval Lecture 4: Web Search. Challenges of Web Search 2. Natural Language and Information Processing (NLIP) Group
Information Retrieval Lecture 4: Web Search Computer Science Tripos Part II Simone Teufel Natural Language and Information Processing (NLIP) Group sht25@cl.cam.ac.uk (Lecture Notes after Stephen Clark)
More informationCS6200 Information Retreival. The WebGraph. July 13, 2015
CS6200 Information Retreival The WebGraph The WebGraph July 13, 2015 1 Web Graph: pages and links The WebGraph describes the directed links between pages of the World Wide Web. A directed edge connects
More informationLarge-Scale Networks. PageRank. Dr Vincent Gramoli Lecturer School of Information Technologies
Large-Scale Networks PageRank Dr Vincent Gramoli Lecturer School of Information Technologies Introduction Last week we talked about: - Hubs whose scores depend on the authority of the nodes they point
More informationLink Structure Analysis
Link Structure Analysis Kira Radinsky All of the following slides are courtesy of Ronny Lempel (Yahoo!) Link Analysis In the Lecture HITS: topic-specific algorithm Assigns each page two scores a hub score
More informationROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015
ROBERTO BATTITI, MAURO BRUNATO. The LION Way: Machine Learning plus Intelligent Optimization. LIONlab, University of Trento, Italy, Apr 2015 http://intelligentoptimization.org/lionbook Roberto Battiti
More informationPageRank and related algorithms
PageRank and related algorithms PageRank and HITS Jacob Kogan Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland 21250 kogan@umbc.edu May 15, 2006 Basic
More informationLink Analysis. Hongning Wang
Link Analysis Hongning Wang CS@UVa Structured v.s. unstructured data Our claim before IR v.s. DB = unstructured data v.s. structured data As a result, we have assumed Document = a sequence of words Query
More informationEinführung in Web und Data Science Community Analysis. Prof. Dr. Ralf Möller Universität zu Lübeck Institut für Informationssysteme
Einführung in Web und Data Science Community Analysis Prof. Dr. Ralf Möller Universität zu Lübeck Institut für Informationssysteme Today s lecture Anchor text Link analysis for ranking Pagerank and variants
More informationPAGE RANK ON MAP- REDUCE PARADIGM
PAGE RANK ON MAP- REDUCE PARADIGM Group 24 Nagaraju Y Thulasi Ram Naidu P Dhanush Chalasani Agenda Page Rank - introduction An example Page Rank in Map-reduce framework Dataset Description Work flow Modules.
More informationIntroduction to Information Retrieval
Introduction to Information Retrieval http://informationretrieval.org IIR 21: Link Analysis Hinrich Schütze Center for Information and Language Processing, University of Munich 2014-06-18 1/80 Overview
More informationPagerank Scoring. Imagine a browser doing a random walk on web pages:
Ranking Sec. 21.2 Pagerank Scoring Imagine a browser doing a random walk on web pages: Start at a random page At each step, go out of the current page along one of the links on that page, equiprobably
More informationMathematical Analysis of Google PageRank
INRIA Sophia Antipolis, France Ranking Answers to User Query Ranking Answers to User Query How a search engine should sort the retrieved answers? Possible solutions: (a) use the frequency of the searched
More informationProximity Prestige using Incremental Iteration in Page Rank Algorithm
Indian Journal of Science and Technology, Vol 9(48), DOI: 10.17485/ijst/2016/v9i48/107962, December 2016 ISSN (Print) : 0974-6846 ISSN (Online) : 0974-5645 Proximity Prestige using Incremental Iteration
More informationPageRank Algorithm Abstract: Keywords: I. Introduction II. Text Ranking Vs. Page Ranking
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 19, Issue 1, Ver. III (Jan.-Feb. 2017), PP 01-07 www.iosrjournals.org PageRank Algorithm Albi Dode 1, Silvester
More informationBrief (non-technical) history
Web Data Management Part 2 Advanced Topics in Database Management (INFSCI 2711) Textbooks: Database System Concepts - 2010 Introduction to Information Retrieval - 2008 Vladimir Zadorozhny, DINS, SCI, University
More information.. Spring 2009 CSC 466: Knowledge Discovery from Data Alexander Dekhtyar..
.. Spring 2009 CSC 466: Knowledge Discovery from Data Alexander Dekhtyar.. Link Analysis in Graphs: PageRank Link Analysis Graphs Recall definitions from Discrete math and graph theory. Graph. A graph
More informationLecture 27: Learning from relational data
Lecture 27: Learning from relational data STATS 202: Data mining and analysis December 2, 2017 1 / 12 Announcements Kaggle deadline is this Thursday (Dec 7) at 4pm. If you haven t already, make a submission
More informationCOMP 4601 Hubs and Authorities
COMP 4601 Hubs and Authorities 1 Motivation PageRank gives a way to compute the value of a page given its position and connectivity w.r.t. the rest of the Web. Is it the only algorithm: No! It s just one
More informationMathematical Methods and Computational Algorithms for Complex Networks. Benard Abola
Mathematical Methods and Computational Algorithms for Complex Networks Benard Abola Division of Applied Mathematics, Mälardalen University Department of Mathematics, Makerere University Second Network
More informationA Modified Algorithm to Handle Dangling Pages using Hypothetical Node
A Modified Algorithm to Handle Dangling Pages using Hypothetical Node Shipra Srivastava Student Department of Computer Science & Engineering Thapar University, Patiala, 147001 (India) Rinkle Rani Aggrawal
More informationA brief history of Google
the math behind Sat 25 March 2006 A brief history of Google 1995-7 The Stanford days (aka Backrub(!?)) 1998 Yahoo! wouldn't buy (but they might invest...) 1999 Finally out of beta! Sergey Brin Larry Page
More informationCS224W: Social and Information Network Analysis Jure Leskovec, Stanford University
CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University http://cs224w.stanford.edu How to organize the Web? First try: Human curated Web directories Yahoo, DMOZ, LookSmart Second
More informationCentralities (4) By: Ralucca Gera, NPS. Excellence Through Knowledge
Centralities (4) By: Ralucca Gera, NPS Excellence Through Knowledge Some slide from last week that we didn t talk about in class: 2 PageRank algorithm Eigenvector centrality: i s Rank score is the sum
More informationCollaborative filtering based on a random walk model on a graph
Collaborative filtering based on a random walk model on a graph Marco Saerens, Francois Fouss, Alain Pirotte, Luh Yen, Pierre Dupont (UCL) Jean-Michel Renders (Xerox Research Europe) Some recent methods:
More informationCENTRALITIES. Carlo PICCARDI. DEIB - Department of Electronics, Information and Bioengineering Politecnico di Milano, Italy
CENTRALITIES Carlo PICCARDI DEIB - Department of Electronics, Information and Bioengineering Politecnico di Milano, Italy email carlo.piccardi@polimi.it http://home.deib.polimi.it/piccardi Carlo Piccardi
More informationAdaptive methods for the computation of PageRank
Linear Algebra and its Applications 386 (24) 51 65 www.elsevier.com/locate/laa Adaptive methods for the computation of PageRank Sepandar Kamvar a,, Taher Haveliwala b,genegolub a a Scientific omputing
More informationPage Rank Algorithm. May 12, Abstract
Page Rank Algorithm Catherine Benincasa, Adena Calden, Emily Hanlon, Matthew Kindzerske, Kody Law, Eddery Lam, John Rhoades, Ishani Roy, Michael Satz, Eric Valentine and Nathaniel Whitaker Department of
More informationINFO 4300 / CS4300 Information Retrieval. slides adapted from Hinrich Schütze s, linked from
INFO 4300 / CS4300 Information Retrieval slides adapted from Hinrich Schütze s, linked from http://informationretrieval.org/ IR 16: Other Link Analysis Paul Ginsparg Cornell University, Ithaca, NY 27 Oct
More informationCS224W: Social and Information Network Analysis Jure Leskovec, Stanford University
CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University http://cs224w.stanford.edu How to organize the Web? First try: Human curated Web directories Yahoo, DMOZ, LookSmart Second
More informationSocial Network Analysis
Social Network Analysis Giri Iyengar Cornell University gi43@cornell.edu March 14, 2018 Giri Iyengar (Cornell Tech) Social Network Analysis March 14, 2018 1 / 24 Overview 1 Social Networks 2 HITS 3 Page
More informationAdvanced Computer Architecture: A Google Search Engine
Advanced Computer Architecture: A Google Search Engine Jeremy Bradley Room 372. Office hour - Thursdays at 3pm. Email: jb@doc.ic.ac.uk Course notes: http://www.doc.ic.ac.uk/ jb/ Department of Computing,
More informationRecent Researches on Web Page Ranking
Recent Researches on Web Page Pradipta Biswas School of Information Technology Indian Institute of Technology Kharagpur, India Importance of Web Page Internet Surfers generally do not bother to go through
More informationFast Iterative Solvers for Markov Chains, with Application to Google's PageRank. Hans De Sterck
Fast Iterative Solvers for Markov Chains, with Application to Google's PageRank Hans De Sterck Department of Applied Mathematics University of Waterloo, Ontario, Canada joint work with Steve McCormick,
More informationCS224W: Social and Information Network Analysis Jure Leskovec, Stanford University
CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University http://cs224w.stanford.edu How to organize the Web? First try: Human curated Web directories Yahoo, DMOZ, LookSmart Second
More informationGraphs / Networks. CSE 6242/ CX 4242 Feb 18, Centrality measures, algorithms, interactive applications. Duen Horng (Polo) Chau Georgia Tech
CSE 6242/ CX 4242 Feb 18, 2014 Graphs / Networks Centrality measures, algorithms, interactive applications Duen Horng (Polo) Chau Georgia Tech Partly based on materials by Professors Guy Lebanon, Jeffrey
More informationAn Improved Computation of the PageRank Algorithm 1
An Improved Computation of the PageRank Algorithm Sung Jin Kim, Sang Ho Lee School of Computing, Soongsil University, Korea ace@nowuri.net, shlee@computing.ssu.ac.kr http://orion.soongsil.ac.kr/ Abstract.
More informationA Reordering for the PageRank problem
A Reordering for the PageRank problem Amy N. Langville and Carl D. Meyer March 24 Abstract We describe a reordering particularly suited to the PageRank problem, which reduces the computation of the PageRank
More informationINTRODUCTION TO DATA SCIENCE. Link Analysis (MMDS5)
INTRODUCTION TO DATA SCIENCE Link Analysis (MMDS5) Introduction Motivation: accurate web search Spammers: want you to land on their pages Google s PageRank and variants TrustRank Hubs and Authorities (HITS)
More informationCS249: SPECIAL TOPICS MINING INFORMATION/SOCIAL NETWORKS
CS249: SPECIAL TOPICS MINING INFORMATION/SOCIAL NETWORKS Overview of Networks Instructor: Yizhou Sun yzsun@cs.ucla.edu January 10, 2017 Overview of Information Network Analysis Network Representation Network
More information1 Random Walks on Graphs
Lecture 7 Com S 633: Randomness in Computation Scribe: Ankit Agrawal In the last lecture, we looked at random walks on line and used them to devise randomized algorithms for 2-SAT and 3-SAT For 2-SAT we
More informationThe PageRank Computation in Google, Randomized Algorithms and Consensus of Multi-Agent Systems
The PageRank Computation in Google, Randomized Algorithms and Consensus of Multi-Agent Systems Roberto Tempo IEIIT-CNR Politecnico di Torino tempo@polito.it This talk The objective of this talk is to discuss
More informationThe PageRank Computation in Google, Randomized Algorithms and Consensus of Multi-Agent Systems
The PageRank Computation in Google, Randomized Algorithms and Consensus of Multi-Agent Systems Roberto Tempo IEIIT-CNR Politecnico di Torino tempo@polito.it This talk The objective of this talk is to discuss
More informationCOMP5331: Knowledge Discovery and Data Mining
COMP5331: Knowledge Discovery and Data Mining Acknowledgement: Slides modified based on the slides provided by Lawrence Page, Sergey Brin, Rajeev Motwani and Terry Winograd, Jon M. Kleinberg 1 1 PageRank
More informationInformation Retrieval and Web Search Engines
Information Retrieval and Web Search Engines Lecture 12: Link Analysis January 28 th, 2016 Wolf-Tilo Balke and Younes Ghammad Institut für Informationssysteme Technische Universität Braunschweig An Overview
More informationHow Google Finds Your Needle in the Web's
of the content. In fact, Google feels that the value of its service is largely in its ability to provide unbiased results to search queries; Google claims, "the heart of our software is PageRank." As we'll
More information16 - Networks and PageRank
- Networks and PageRank ST 9 - Fall 0 Contents Network Intro. Required R Packages................................ Example: Zachary s karate club network.................... Basic Network Concepts. Basic
More informationPart 1: Link Analysis & Page Rank
Chapter 8: Graph Data Part 1: Link Analysis & Page Rank Based on Leskovec, Rajaraman, Ullman 214: Mining of Massive Datasets 1 Graph Data: Social Networks [Source: 4-degrees of separation, Backstrom-Boldi-Rosa-Ugander-Vigna,
More informationOn Finding Power Method in Spreading Activation Search
On Finding Power Method in Spreading Activation Search Ján Suchal Slovak University of Technology Faculty of Informatics and Information Technologies Institute of Informatics and Software Engineering Ilkovičova
More informationSocial Networks 2015 Lecture 10: The structure of the web and link analysis
04198250 Social Networks 2015 Lecture 10: The structure of the web and link analysis The structure of the web Information networks Nodes: pieces of information Links: different relations between information
More informationCS-C Data Science Chapter 9: Searching for relevant pages on the Web: Random walks on the Web. Jaakko Hollmén, Department of Computer Science
CS-C3160 - Data Science Chapter 9: Searching for relevant pages on the Web: Random walks on the Web Jaakko Hollmén, Department of Computer Science 30.10.2017-18.12.2017 1 Contents of this chapter Story
More informationMotivation. Motivation
COMS11 Motivation PageRank Department of Computer Science, University of Bristol Bristol, UK 1 November 1 The World-Wide Web was invented by Tim Berners-Lee circa 1991. By the late 199s, the amount of
More informationHypercubes. (Chapter Nine)
Hypercubes (Chapter Nine) Mesh Shortcomings: Due to its simplicity and regular structure, the mesh is attractive, both theoretically and practically. A problem with the mesh is that movement of data is
More informationLecture 8: Linkage algorithms and web search
Lecture 8: Linkage algorithms and web search Information Retrieval Computer Science Tripos Part II Simone Teufel Natural Language and Information Processing (NLIP) Group Simone.Teufel@cl.cam.ac.uk Lent
More informationIntroduction to Information Retrieval (Manning, Raghavan, Schutze) Chapter 21 Link analysis
Introduction to Information Retrieval (Manning, Raghavan, Schutze) Chapter 21 Link analysis Content Anchor text Link analysis for ranking Pagerank and variants HITS The Web as a Directed Graph Page A Anchor
More informationTODAY S LECTURE HYPERTEXT AND
LINK ANALYSIS TODAY S LECTURE HYPERTEXT AND LINKS We look beyond the content of documents We begin to look at the hyperlinks between them Address questions like Do the links represent a conferral of authority
More informationCS535 Big Data Fall 2017 Colorado State University 9/5/2017. Week 3 - A. FAQs. This material is built based on,
S535 ig ata Fall 217 olorado State University 9/5/217 Week 3-9/5/217 S535 ig ata - Fall 217 Week 3--1 S535 IG T FQs Programming ssignment 1 We will discuss link analysis in week3 Installation/configuration
More informationLecture Notes to Big Data Management and Analytics Winter Term 2017/2018 Node Importance and Neighborhoods
Lecture Notes to Big Data Management and Analytics Winter Term 2017/2018 Node Importance and Neighborhoods Matthias Schubert, Matthias Renz, Felix Borutta, Evgeniy Faerman, Christian Frey, Klaus Arthur
More informationAlgorithms, Games, and Networks February 21, Lecture 12
Algorithms, Games, and Networks February, 03 Lecturer: Ariel Procaccia Lecture Scribe: Sercan Yıldız Overview In this lecture, we introduce the axiomatic approach to social choice theory. In particular,
More informationLecture 9: I: Web Retrieval II: Webology. Johan Bollen Old Dominion University Department of Computer Science
Lecture 9: I: Web Retrieval II: Webology Johan Bollen Old Dominion University Department of Computer Science jbollen@cs.odu.edu http://www.cs.odu.edu/ jbollen April 10, 2003 Page 1 WWW retrieval Two approaches
More informationHow to organize the Web?
How to organize the Web? First try: Human curated Web directories Yahoo, DMOZ, LookSmart Second try: Web Search Information Retrieval attempts to find relevant docs in a small and trusted set Newspaper
More informationLink Analysis. CSE 454 Advanced Internet Systems University of Washington. 1/26/12 16:36 1 Copyright D.S.Weld
Link Analysis CSE 454 Advanced Internet Systems University of Washington 1/26/12 16:36 1 Ranking Search Results TF / IDF or BM25 Tag Information Title, headers Font Size / Capitalization Anchor Text on
More informationCS60092: Informa0on Retrieval
Introduc)on to CS60092: Informa0on Retrieval Sourangshu Bha1acharya Today s lecture hypertext and links We look beyond the content of documents We begin to look at the hyperlinks between them Address ques)ons
More informationBig Data Analytics CSCI 4030
High dim. data Graph data Infinite data Machine learning Apps Locality sensitive hashing PageRank, SimRank Filtering data streams SVM Recommen der systems Clustering Community Detection Web advertising
More informationGraph Algorithms. Revised based on the slides by Ruoming Kent State
Graph Algorithms Adapted from UMD Jimmy Lin s slides, which is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States. See http://creativecommons.org/licenses/by-nc-sa/3.0/us/
More informationCOMPARATIVE ANALYSIS OF POWER METHOD AND GAUSS-SEIDEL METHOD IN PAGERANK COMPUTATION
International Journal of Computer Engineering and Applications, Volume IX, Issue VIII, Sep. 15 www.ijcea.com ISSN 2321-3469 COMPARATIVE ANALYSIS OF POWER METHOD AND GAUSS-SEIDEL METHOD IN PAGERANK COMPUTATION
More informationWeek 10: DTMC Applications Randomized Routing. Network Performance 10-1
Week 10: DTMC Applications Randomized Routing Network Performance 10-1 Random Walk: Probabilistic Routing Random neighbor selection e.g. in ad-hoc/sensor network due to: Scalability: no routing table (e.g.
More informationBruno Martins. 1 st Semester 2012/2013
Link Analysis Departamento de Engenharia Informática Instituto Superior Técnico 1 st Semester 2012/2013 Slides baseados nos slides oficiais do livro Mining the Web c Soumen Chakrabarti. Outline 1 2 3 4
More informationLink analysis. Query-independent ordering. Query processing. Spamming simple popularity
Today s topic CS347 Link-based ranking in web search engines Lecture 6 April 25, 2001 Prabhakar Raghavan Web idiosyncrasies Distributed authorship Millions of people creating pages with their own style,
More informationA Survey of Google's PageRank
http://pr.efactory.de/ A Survey of Google's PageRank Within the past few years, Google has become the far most utilized search engine worldwide. A decisive factor therefore was, besides high performance
More informationApplication of PageRank Algorithm on Sorting Problem Su weijun1, a
International Conference on Mechanics, Materials and Structural Engineering (ICMMSE ) Application of PageRank Algorithm on Sorting Problem Su weijun, a Department of mathematics, Gansu normal university
More informationUsing Spam Farm to Boost PageRank p. 1/2
Using Spam Farm to Boost PageRank Ye Du Joint Work with: Yaoyun Shi and Xin Zhao University of Michigan, Ann Arbor Using Spam Farm to Boost PageRank p. 1/2 Roadmap Introduction: Link Spam and PageRank
More informationMining Web Data. Lijun Zhang
Mining Web Data Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj Outline Introduction Web Crawling and Resource Discovery Search Engine Indexing and Query Processing Ranking Algorithms Recommender Systems
More informationCLOUD COMPUTING PROJECT. By: - Manish Motwani - Devendra Singh Parmar - Ashish Sharma
CLOUD COMPUTING PROJECT By: - Manish Motwani - Devendra Singh Parmar - Ashish Sharma Instructor: Prof. Reddy Raja Mentor: Ms M.Padmini To Implement PageRank Algorithm using Map-Reduce for Wikipedia and
More informationCS6322: Information Retrieval Sanda Harabagiu. Lecture 10: Link analysis
Sanda Harabagiu Lecture 10: Link analysis Today s lecture Link analysis for ranking Pagerank and variants HITS Sec. 21.1 The Web as a Directed Graph Page A Anchor hyperlink Page B Assumption 1: A hyperlink
More informationBig Data Analytics CSCI 4030
High dim. data Graph data Infinite data Machine learning Apps Locality sensitive hashing PageRank, SimRank Filtering data streams SVM Recommen der systems Clustering Community Detection Web advertising
More informationCS 6604: Data Mining Large Networks and Time-Series
CS 6604: Data Mining Large Networks and Time-Series Soumya Vundekode Lecture #12: Centrality Metrics Prof. B Aditya Prakash Agenda Link Analysis and Web Search Searching the Web: The Problem of Ranking
More informationCSCE 5160 Parallel Processing. CSCE 5160 Parallel Processing
HW #9 10., 10.3, 10.7 Due April 17 { } Review Completing Graph Algorithms Maximal Independent Set Johnson s shortest path algorithm using adjacency lists Q= V; for all v in Q l[v] = infinity; l[s] = 0;
More informationLink Analysis SEEM5680. Taken from Introduction to Information Retrieval by C. Manning, P. Raghavan, and H. Schutze, Cambridge University Press.
Link Analysis SEEM5680 Taken from Introduction to Information Retrieval by C. Manning, P. Raghavan, and H. Schutze, Cambridge University Press. 1 The Web as a Directed Graph Page A Anchor hyperlink Page
More informationSearching the Web [Arasu 01]
Searching the Web [Arasu 01] Most user simply browse the web Google, Yahoo, Lycos, Ask Others do more specialized searches web search engines submit queries by specifying lists of keywords receive web
More informationUniversity of Maryland. Tuesday, March 2, 2010
Data-Intensive Information Processing Applications Session #5 Graph Algorithms Jimmy Lin University of Maryland Tuesday, March 2, 2010 This work is licensed under a Creative Commons Attribution-Noncommercial-Share
More informationRanking on Data Manifolds
Ranking on Data Manifolds Dengyong Zhou, Jason Weston, Arthur Gretton, Olivier Bousquet, and Bernhard Schölkopf Max Planck Institute for Biological Cybernetics, 72076 Tuebingen, Germany {firstname.secondname
More informationGraphs / Networks CSE 6242/ CX Centrality measures, algorithms, interactive applications. Duen Horng (Polo) Chau Georgia Tech
CSE 6242/ CX 4242 Graphs / Networks Centrality measures, algorithms, interactive applications Duen Horng (Polo) Chau Georgia Tech Partly based on materials by Professors Guy Lebanon, Jeffrey Heer, John
More informationPV211: Introduction to Information Retrieval https://www.fi.muni.cz/~sojka/pv211
PV211: Introduction to Information Retrieval https://www.fi.muni.cz/~sojka/pv211 IIR 21: Link analysis Handout version Petr Sojka, Hinrich Schütze et al. Faculty of Informatics, Masaryk University, Brno
More informationMAE 298, Lecture 9 April 30, Web search and decentralized search on small-worlds
MAE 298, Lecture 9 April 30, 2007 Web search and decentralized search on small-worlds Search for information Assume some resource of interest is stored at the vertices of a network: Web pages Files in
More informationRoadmap. Roadmap. Ranking Web Pages. PageRank. Roadmap. Random Walks in Ranking Query Results in Semistructured Databases
Roadmap Random Walks in Ranking Query in Vagelis Hristidis Roadmap Ranking Web Pages Rank according to Relevance of page to query Quality of page Roadmap PageRank Stanford project Lawrence Page, Sergey
More informationGraph Data Processing with MapReduce
Distributed data processing on the Cloud Lecture 5 Graph Data Processing with MapReduce Satish Srirama Some material adapted from slides by Jimmy Lin, 2015 (licensed under Creation Commons Attribution
More informationECEN : Microprocessor System Design Department of Electrical and Computer Engineering Texas A&M University. Homework #2 Solutions
ECEN 449 749: Microprocessor System Design Department of Electrical and Computer Engineering Texas A&M University Homework #2 Solutions Upload your homework solution to ecampus as a single file. Your homework
More informationEE/CSCI 451 Midterm 1
EE/CSCI 451 Midterm 1 Spring 2018 Instructor: Xuehai Qian Friday: 02/26/2018 Problem # Topic Points Score 1 Definitions 20 2 Memory System Performance 10 3 Cache Performance 10 4 Shared Memory Programming
More informationWeb Structure Mining using Link Analysis Algorithms
Web Structure Mining using Link Analysis Algorithms Ronak Jain Aditya Chavan Sindhu Nair Assistant Professor Abstract- The World Wide Web is a huge repository of data which includes audio, text and video.
More information