MA/CS-109: Graph Random Walks. Azer Bestavros

Transcription

1 /CS-09: Graph Random Walks zer estavros

2 Wandering around in a graph We saw that given a map graph, one can determine paths that satisy a speciic objective inimize cost or going rom to Shortest Path Visit every node once Traveling Salesman Cross every edge once returning to starting point Eulerian circuit 4/4/00 /CS-09 zer estavros Recall our use o graphs as abstractions o real-world settings speciically maps in order to answer speciic questions or somebody potentially using that map or example, inding the shortest path between two points or inding a path that satisies speciic properties e.g., visiting all nodes while crossing each edge exactly once.

3 Wandering around in a graph We saw that given a map graph, one can determine paths that satisy a speciic objective inimize cost or going rom to shortest path Cross every edge once returning to starting point Eulerian circuit What i we have no objective? tourist roaming around the city web surer aimlessly clicking rom a web page to another 4/4/00 /CS-09 zer estavros 3 Now consider a situation where the walker using the map has no speciic objective. He/she is just wandering around rom node to node. 3

4 Wandering around in DC 4/4/00 /CS-09 zer estavros 4 To illustrate this, consider a tourist moving around the subway system in DC where the tourists simply selects trips between nodes randomly i.e., upon arrival to a station, the tourist selects the next leg randomly rom among all possible outgoing trips rom that station. The above animation illustrates an example o what the tourist would be doing. From the animation one can see that the tourist ends up going in and out some stations much more requently than others. One can explain this intuitively by noting that the stations in the center o town are much better connected that the chances o wandering in them is higher i one moves between stations at random. So, here is a question we may wand to answer: What is the relative requency with which a random walker ends up in one station versus another? s we have done in this class, we want to answer this question not qualitatively using words like more or much more, but using some metric that we can be precise about. 4

5 Random walks on a graph. Start in some random node. t random, select one o the outgoing edges o the current node 3. Walk over that edge to a new node 4. Go to What properties would emerge as a result o doing the above or a very very very long time? 4/4/00 /CS-09 zer estavros 5 First, let s be speciic about the model at hand. random walker starts at some random node in the graph, selects an outgoing edge rom that node, traverses that edge and then repeats this process or a very very long time. 5

6 speciic question What are the relative requencies or probabilities o visiting the various nodes in the graph. What is the likelihood o inding the wandering tourist at a given subway station? What is the likelihood that a wandering tourist will stumble upon the useum o Science? What is the likelihood that a web surer will end up visiting Google? How about the U web site? 4/4/00 /CS-09 zer estavros 6 nd here is the speciic question we want answered: What are the relative requencies o visiting the various nodes in the graph? or stations in the DC etro, or milestones in oston, or pages on the web, 6

7 Shuttling between U campuses lways U edical Campus U ain Campus lways 50% chance to be on either campus 4/4/00 /CS-09 zer estavros 7 Let s start with a very simple scenario. Consider a U student who shuttles between the U Charles River Campus and the U edical Campus using the U shuttle. For simplicity, consider consecutive time periods e.g., one hour and assume that the students alternates between the two campuses every hour. Now, i we ask the question what is the relative requency o the student being on the main campus versus the medical campus, the answer will be clearly 50% on each. This is obvious; we don t need math to igure this out! 7

8 Let s make it more interesting lways Tails U edical Campus U ain Campus Heads Imagine many random walkers a population Question: t some random point in time, what is the percentage o the population we should expect on each campus? 4/4/00 /CS-09 zer estavros 8 How about a slightly more complicated scenario. Instead o one walker student, consider many walkers e.g., 000. Each one o the student does the ollowing: I in some hour, the student is on the main campus, he/she will lip a coin and i the result is heads, the student will hop on the shuttle and go to the medical campus or the next hour, otherwise i the coin toss is tails the student will just stay put. From the medical campus, the student always hops on the shuttle and goes back to the main campus. Now, let s pose the question: t some random point in time in the ar uture, what is the percentage o the population o,000 we should expect on each campus? What would be our answer now? Let s try to igure this out. 8

9 simple example continued Problways= ProbTails= U edical Campus U ain Campus ProbHeads= Denote by t v the population o vertex v in iteration t. Let us write down some relationships 4/4/00 /CS-09 zer estavros 9 Let s denote by v the number o walkers at time t at node v. t 9

10 simple example continued = 0 = 3 =... = t... = t 4/4/00 /CS-09 zer estavros 0 To simply things, let s denote the medical campus with the symbol and the main campus with symbol. ssume that the population at time 0 on the main campus is 0. Since at time, everybody who was in ends up leaving and hal o those who were in at time 0 end up in at time, we can write the relationship: = 0 Now with a similar logic, we can see that this will be the case or all time periods,, 3, all the way to a very very large number let s call it. Doing so, gives us the relationships shown on the slide, which relate the population on campus in terms o the population on campus in the previous time interval. dding up all the terms on both sides o the equations, we get 0

11 /CS-09 zer estavros 4/4/00 simple example continued =... 3 [ ] =... 3 [ ] [ ] [ ] = =... 3 Using simple algebraic manipulation by adding a quantity and subtracting it rom the right hand side, rearranging terms, and then dividing both sides by, we get =... 3 [ ] =... 3

12 simple example continued 3... = 0... [ ] 3... = 0... [ ] 3... = [... ] [ ] 0 X = /4/00 /CS-09 zer estavros We note that the let-hand-side o the equation is the sum o the population in in all time intervals rom to divided by, which is precisely the average size o the population in over time, which we denote by. Similarly, we can substitute on the right hand side or the value o. This leaves us with the second term on the right hand side. Since is very very large compared to the dierence between two numbers that are close to each other -- the population o at time 0 and at time, we can simply ignore that term as it tends to zero.

13 simple example continued = = Proportion at Proportion at 4/4/00 /CS-09 zer estavros 3 Now, dividing both sides by the total population, which is, which gives us the proportion o the population at each node, which we denote bt Pr and Pr which is also the probability o a walker being in versus, respectively. Thus, we can write the relationship Pr = Pr. 3

14 simple example continued Pr = Pr Pr Pr = 4/4/00 Pr = 3 Pr = 3 /CS-09 zer estavros 4 nother relationship we can write is that the sum o the two proportions, i.e., Pr Pr, must be equal to. So, we have two equations in two unknowns. y substituting or Pr rom the irst equation into the second, we get one equation in one unknown, which gives us the value o Pr = /3. This means that Pr which is hal o Pr is /3. That s it, we igured it out. We would expect one third o the population in the medical campus and two thirds o the population in the main campus. 4

15 How about this one? Pr = 0.Pr Pr Pr = Pr = 9% Pr = 9% 4/4/00 /CS-09 zer estavros 5 We can also consider a scenario where the walkers students choose to stay on the main campus 9 out o every 0 times and choose to shuttle to the medical campus out o every0 times. Following the exact same process, we conclude that 0.9 i.e., 9% o the population will be on the main campus with the rest i.e., 9% on the medical campus. 5

16 How about this one? Why is it interesting to randomly walk web graphs? 4/4/00 /CS-09 zer estavros 6 Well, these were toy examples! How about a very very large graph the web graph? 6

17 Random walks on web graphs Web pages are nodes o the graph; a link rom a page to another is an edge I web users sur randomly, then they are eectively doing a random walk on the web graph Computing the relative requency with which a page is visited would make or a good measure o the page s popularity 4/4/00 /CS-09 zer estavros 7 The web graph is made up o nodes the web pages and directed edges the links rom one page to another. I web users sur the web randomly, going rom one page to the next by picking one o the links rom that page at random, then they are eectively doing a random walk on the web graph! Now we can ask the question: What percentage o web surers would end up in one page versus another? This quantity could be interpreted as the popularity o a web page. This is a very valuable thing to know since it could be used by advertisers to decide the worth o putting an ad on one page e.g., cnn.com versus another e.g., msnbc.com. 7

18 Web graph example C C Which page is more important? 4/4/00 /CS-09 zer estavros 8 Just to illustrate this, consider a web o simply three pages, with the speciic link structure shown on the let. From that structure, we can iner the web graph on the right where we labeled the edges with the probability o traversing that edge. For example, since out o page there are two outgoing links, and since random web surers graph walkers would equally likely pick one o these, we label them both with a probability o. Now we ask the question: What is the relative popularity o pages,, and C? Which one o these pages is more popular than the others? 8

19 Web graph example Pr Pr Pr C = Pr = Pr Pr C Pr C = Pr C Pr = Pr = Pr C = 0. 4/4/00 /CS-09 zer estavros 9 y ollowing the same process as beore, we can write a set o equations. Since we have three unknowns Pr, Pr, and PrC we must have a set o 3 equations. The irst equation is always the easiest one to write that the sum o the relative popularity is. Now, by looking at one o the states say we can say that at any time interval, the population in will be composed o everybody who was in hal those in C in the previous time interval. This gives us a second equation. Doing the same or another state say C, we get the third equation. With 3 equations in 3 unknown, we can ind the relative popularity o,, and C namely, Pr = 0.444, Pr=0.333, and PrC=0.. So, page is the most popular it is the one in which we would expect that most random web surers will be. 9

20 Is suring really a random walk? O course not People are not robot clicking randomly on links People are not synchronized in their clicks People oten just type the URL they want Links on a page may change over time any web pages are dynamic ut despite all that Random walks on a web graph tell us something about how the structure o the web inluences surers. That s what models are or! 4/4/00 /CS-09 zer estavros 0 ut, is a random walk the best way to model suring behavior? Clearly, people are not robot clicking randomly on links; they do not go rom one page to the other every time interval e.g., every minute; they are not synchronized in their actions; they may visit a web page by typing its URL in the browser, or by searching or it and then clicking on the search results. oreover, the web structure itsel is not a static graph since links change as web pages change when you add a link to your Facebook page, you are eectively changing the structure o the web. These are all valid points, but despite all that, the random walk model is a very good abstraction that allows us to come up with precise answers. Notice that trying to capture all the nuances o how people sur the web, will result in a very complicated object that would be rather impossible to solve or to comprehend in the irst place. This is precisely why we use abstractions and models it is to simpliy the real world so that we may be able to answer questions! 0

21 Random walks on web graphs ut, the web graph has ~ 00 billion pages! We are not about to write 00 billion equations and solve them?! We can simulate the random walk and measure the requency o visits to the various pages. This process is at the heart o Google s PageRank algorithm and ~ $400 stock value. 4/4/00 /CS-09 zer estavros So ar, we igured out how to convert a real-world object maps or web to a graph and then using random walks solve analytically or the popularity o various nodes in the graph stations or web pages. ut the graphs corresponding to real-word objects are HUGE. The web has hundreds o billions o pages! We are not about to write 00 billion equations and expect to solve them algebraically!!! The way we deal with this is to simulate the random walkers! TW, this technique is at the heart o the Google search technology a technology that propelled one o the most successul companies in hostory!

22 Random walk simulation. Initialize: ssume that some arbitrary population at each node. Compute new population: Use edge probabilities to calculate the population at the next time interval C 3. Check or change: I population changed then go to 4/4/00 /CS-09 zer estavros So, what do we mean by simulating random walkers? It s simple, just start with some arbitrary population at each node. Now, each time step, calculate the new population at each node, and keep doing this until things stabilize or until we get tired. This would give us the relative popularity o the various nodes!

23 Random walk simulation C 6.7 C 5.0 C t=0 t= t= C.9 C.9 C t=3 t=4 t=5 4/4/00 /CS-09 zer estavros 3 Here is an example. We started with 00 people, whom we divided equally among the three nodes web pages at time t=0. Now we do the simulation or a number o steps and we notice how the population seems to stabilize with attracting around 44 people, attracting 33, and C attracting. 3

24 Random walk simulation Iteration Pr Pr PrC C 4/4/00 /CS-09 zer estavros 4 Indeed, the spreadsheet shown is exactly the resulting evolution o the population and as we can see, by the th time step, the population stabilizes and we have our answer. 4

25 Random walk simulation Iteration Pr Pr PrC C 4/4/00 /CS-09 zer estavros 5 Rather than reporting the number o people we expect to see at a node, it is customary to report the raction o the population at each node. This is nothing but the population at the node divided by the total population a number between 0 and, which we can think o as the probability that a single person will end up in a node. Indeed, we would get the same answer i we start with any initial population! 5

26 Larry PageRank PageRank relies on the uniquely democratic nature o the web by using its vast link structure as an indicator o an individual page's value. In essence, Google interprets a link rom page to page as a vote, by page, or page. ut, Google looks at more than the sheer volume o votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important". C Source: Google 4/4/00 /CS-09 zer estavros 6 Using random walks to estimate the popularity o various web pages based on the structure o the web i.e., how the various pages are linked to one another was the value proposition or Google search. y sorting the various pages returned rom a keyword search based on their relative popularity, users ound the search results to be much more useul you really don t want to scroll down to the 0 th page o search results to ind what you are looking or. Indeed, Google makes an interesting observation: The structure o the web relects votes rom one page to another. I a page points to another, then this constitutes a vote by the reerring page or the reerred-to page. I you post a link on your Facebook page, you are eectively saying that this link is interesting. Google search uses a slightly modiied version o random walks by recognizing that the popularity o a web page has two sources the irst is due to random walking as we have done so ar and the second is due to people going to the web page directly e.g., by typing a URL directly, instead o clicking on a link rom another page. The resulting algorithm is PageRank, which is due to Larry Page a co-ounder o Google. PageRank attributes a weight to the two sources o web page popualrity and computes a score based on the joint contribution o these two sources. While we will not cover how PageRank does this, it suices to say that it is very very similar to random walks. So, or all practical purposes, you can simply think o PageRank as the equivalent o evaluating popularity using random walks. 6

27 shton Kutcher vs CNN It is not only how many ollowers you have, but how important these ollowers are! Important implications or evaluating scholarship! 4/4/00 /CS-09 zer estavros 7 n interesting point to note is that the popularity o a web page according to a random walk evaluation or PageRank evaluation is not necessarily dependent on the number o other pages pointing to it, but in act it is dependent on the popularity o those pages pointing to it as well. This is why the Twitter debate about who is more popular shton Kutcher or CNN based on the number o ollowers o each does not make sense!!! ccording to random walks and Google, not all web pages are created equal and this a vote rom an important web page should count or more than a vote rom an insigniicant web page. This concept has signiicant implications or evaluating scholarship do we simply count the number o citations to a paper, or do we take into account the importance/popularity o the paper making the citation? 7

28 PageRank Simulation 4/4/00 /CS-09 zer estavros 8 Demo o PageRank simulations Simulator available through oodle. 8

29 Google ombs I we know how an algorithm works, we can manipulate its output ake a web page show up on the irst line o the irst page o a search! big business and even a political instrument! Try this seach: miserable ailure Food or thought: PageRank depends on the web s link structure to do a good job ut the web s link structure is inluenced by how PageRank works Is Google undermining the assumption that a link = a vote?! 4/4/00 /CS-09 zer estavros 9 One last point to make. The act that we know how Google sorts web pages means that we can manipulate it! This is precisely what people did in the early days o web search technologies and what has become known as Google ombing or Google ombs. People who make Google bombs do so by creating a bunch o web non-sense web pages that link to a page that they want to artiicially raise its popularity. The most amous o Google ombs is the miserable ailure bomb! Just ing miserable ailure to see who comes up very high on the list o returned web pages hint: it is a US president Finally, we make the ollowing philosophical observation: PageRank depends on the web s link structure to do a good job. ut the web s link structure is inluenced by how PageRank works -- e.g., people are likely to point to pages that Google ranks highly because they use Google to ind these pages. This means that Google is introducing bias in the web structure! This raises the ollowing intriguing question: Is Google undermining the assumption that a link = a vote?!! Is Google undermining its own business proposition?! Is a sotware program PageRank inluencing the evolution o our society, our public opinion, etc.? This is scary in many ways it suggests that algorithms are acting like DN ; they aect how we evolve as a society! 9