Centralities (4) By: Ralucca Gera, NPS. Excellence Through Knowledge

Transcription

1 Centralities (4) By: Ralucca Gera, NPS Excellence Through Knowledge

2 Some slide from last week that we didn t talk about in class: 2

3 PageRank algorithm Eigenvector centrality: i s Rank score is the sum of the Rank scores of all pages j that point to i :, Then Katz centrality adds the teleportation by adding a small weight edge to each node (using a weight of ):, BUT, since a page j may point to many other pages, its prestige score should be shared among these pages (For example NPS pointing to many sites), 3

4 Matrix notation (1) Let be a n-dimensional column vector of PageRank values, ie, ) T Let A be the adjacency matrix of our digraph with entries Then the PageRank centrality of node is given by: deg or 1 Where is the damping factor, generally set for = 85 (more on the next page) 4

5 So the PageRank centrality of node is given by: β where α is the damping factor (generally α = 85) Recall from eigenvector centrality: Matrix notation (2) A x = x or x = A x Small values (close to ): the contribution given by paths longer than one hop is small, so centrality scores are mainly influenced by in-degrees Large values (close to ): allows long paths to be devalued smoothly, and centrality scores influenced by the topology of G Recommendation: choose α,, where the centrality diverges at α = The default is usually 85 5

6 Overview What makes a vertex central in a network? How do you describe it mathematically? When is it appropriate to use it? How can we capture it? Lots of one-hop connections to high centrality vertices A weighted degree centrality based on the weight of the neighbors For example when the people you are connected to matter Eigenvector centrality α Where A is the in degree matrix Lots of one-hop connections to high out-degree vertices A weighted degree centrality based on the out degree of the neighbors Directed graphs that are not strongly connected Katz α + β Where β is some small weight for each node As above but distribute the weight that a node has to the nodes it points to deg As above but distributing the wealth of a node to the ones it points to Page Rank: α or deg α β PR: most known and influential algorithms for computing the relevance of web pages

7 An example as just described: Problem vertex (no outgoing links) Recall that the problem with vertices with indegree = was solved by using β in-degree matrix each row shows the in degree A each column shows the out degree α deg or α β Is the formula above well defined? β If not, how could we fix the formula or the matrix?

8 How can we fix the problem? 1 Remove those pages with no out-links during the PageRank computation as these pages do not affect the ranking of any other page directly (these pages will get outgoing links in the future) 2 Add a complete set of outgoing links from each such page i to all the pages on the Web The second choice is used in PR since matrix may get updated in-degree matrix each column shows the out degree A each row shows the in degree 8

9 How can we fix the out degree =? 1 1 α β A /2 in-degree matrix 1/2 1/1-1 D 1/3 1/6 Inverse of the out-degree matrix 1/2 9

10 1 PR centrality formula is well defined By multiplying them we obtain the matrix that captures: 1 The in and out degree per vertex 2 Divides the centrality of each vertex by its degree The contribution of node 5 is insignificant, and the formula is now well defined in-degree matrix out-degree matrix / AD α β

11 Transition probability matrix This modified matrix is called the state transition probability matrix Denote its entries by p ij : -1 AD p p p n1 p p p n2 p p p 1n 2n nn p ij represents the transition probability that the surfer in state i (page i) will move to state j (page j) Here is an example: 11

12 A small Internet consisting of just 4 websites Source: 12

13 A small Internet consisting of just 4 websites p ij represents the transition probability that the surfer on page j will move to page i: -1 AD p ij 1/ 3 1/ 3 1/ / 2 1 1/ Source: 13

14 A small Internet consisting of just 4 websites Random surfer: each page has equal probability ¼ to be chosen as a starting point -1 AD p ij 1/ 3 1/ 3 1/ / 2 1 1/ The probability that page i will be visited after k steps (ie the random surfer ending up at page i ) is equal to entry of A k x Simplification for this example: No β was involved since id i >, for all i Source: 14

15 Overview Updated! What makes a vertex central in a network? How do you describe it mathematically? When is it appropriate to use it? How can we capture it? Lots of one-hop connections to high centrality vertices A weighted degree centrality based on the weight of the neighbors For example when the people you are connected to matter Eigenvector centrality є Lots of one-hop connections to high out-degree vertices A weighted degree centrality based on the out degree of the neighbors Directed graphs that are not strongly connected Katz є + β Where β is some initial weight As above but distribute the weight that a node has to the nodes it points to deg As above but distributing the wealth of a node to the ones it points to Page Rank: α deg Where outdeg j = max{1, out degree of node j} β

16 Newman s book gives: Some comments where α is called the damping factor which can be set to between and 1(or the largest eigenvalue of A) And the formula in the original PageRank is: where d is the damping factor (d = 85 as default) Gephi: the default value for is the probability = 85 and Epsilon is the criteria for eigenvector convergence based on the power method

17 Final Points on PageRank Fighting spam A page is important if the pages pointing to it are important Since it is not easy for Web page owner to add in-links into his/her page from other important pages, it is thus not easy to influence PageRank PageRank is a global measure and is query independent The values of the PageRank algorithm of all the pages are computed and saved off-line rather than at the query time => fast Criticism: There are companies that can increase your pagerank by adding it to a cluster and increasing its indegree It cannot not distinguish between pages that are authoritative in general and pages that are authoritative on the query topic But it works based on the keyword search 17

18 Betweenness Centrality Some pages are adapted from Dan Ryan, Mills College

19 Different types of centralities: Betweenness Centrality Closeness Centrality Eigenvector Centrality Source: Discovering Sets of Key Players in Social Networks Daniel Ortiz-Arroyo Springer 21/ Degree Centrality 19

20 Betweenness Centrality Intuition: how many pairs of individuals would have to go through you in order to reach one another in the minimum number of hops? Interactions between two individuals depend on the other individuals in the set of nodes The nodes in the middle have some control over the paths in the graph Useful for flow, such as information or data packages

21 Assumptions When there is more than one geodesic, all geodesics are equally likely to be used Flow takes the shortest path (we ll look at alternatives) Every pair of nodes in G exchanges a message with equal probability per unit time Question: How many messages, on average, will have passed through each vertex en route to their destination? A node s betweenness is given by all pairs of nodes, including the node in question 21

22 Meaning of betweenness centrality Vertices with high betweenness centrality have influence in the network by virtue of their control over information passing between others They get to see the messages as they pass through They could get paid for passing the message along Thus they get a lot of power: their removal would disrupt communication How would you capture it in a mathematical formula? 22

23 Formula for betweenness centrality, where is the number of s-t geodesics that i belongs to (default: i could equal s or t, but in other versions it cannot and that s where you see values) in an undirected graph, an s-t geodesic is the same as a t-s geodesics, so the edge gets counted twice) It is applicable to directed networks as well 23

24 Bounds for disconnected graphs Let G be a disconnected graph: What is the minimum value of betweenness centrality a vertex can have in disconnected graphs? an isolated vertex: What is the maximum value of betweenness centrality a vertex can have in disconnected graphs? center of a star with center: Let at with center node at Then there are pairs of nodes, from which we take away the paths from to since is not on them 24

25 Bounds for connected graphs Let G be connected: What is the minimum value of betweenness centrality a vertex can have in connected graphs? A leaf x would have it: (11121 where we have 1 paths from x to each vertex And 1 more paths from each vertex to x Finally one path from x to x What is the maximum value of betweenness centrality a vertex can have in disconnected graphs? center of a star in the largest component: 1 which is the number of pairs of nodes minus the paths from a leaf to itself 25

26 A refined formula where, is the number of s-t geodesics that i belongs to is the number of s-t geodesics Convention: if = and =, then (in an undirected graph, an s-t geodesic is the same as a t-s geodesics, so it gets counted twice) 26

27 In class activity: betweenness of A? Fraction of shortest paths that include vertex A,, 1 shortest path of Number of paths 4 goes through A A B C D E F G A B C D E 1 shortest path of F 4 goes through A - 1 = 75 G 1 shortest path - of 4 goes through A

28 A normalized refined formula where ) / is the number of s-t geodesics that i belongs to is the number of s-t geodesics Convention: = and =, then 28

29 Another normalized formula where ) / is the number of s-t geodesics that i belongs to is the number of s-t geodesics Convention: = and =, then 29

30 Betweenness Centrality Used generally for Information flow Typically distributed over a wide range Betweenness only uses geodesic paths Information can also flow on longer paths Sometimes we hear it through the grapevine While betweenness focuses just on the geodesic, flow betweenness centrality focuses on how information might flow through many different paths

31 Flow betweenness centrality BUT, is the maximum flow transmitted from s to t through all possible paths that i belongs to is the maximum flow transmitted from s to t through all possible paths Convention: = and =, then (in an undirected graph, an s-t geodesic is the same as a t-s geodesics, so it gets counted twice) 31

32 Random walk betweenness centrality, BUT is the number of times a random walk from s to t passes through i, averaged over many repetitions of a walk Note that A good measure for traffic that doesn t have a particular destination 32

33 Other extensions of centralities How would you extend the centralities you have seen? What else would you introduce that would capture the centrality of a vertex? Would you use it for edges? This is a good time to share your thoughts Subgraph/subset centrality? How central are you to that particular subgraph? How central is the subgraph to the network? If so, would you repeat the centralities seen before for that subgraph? 33

34 Overview Local measure: degree Relative to rest of network: closeness, betweenness, eigenvector, Katz, PageRank How evenly is centrality distributed among nodes? hubs and authorities You ve learned the traditional centralities Based on your understanding of the methodologies that create them, decide which one is appropriate to use for your application 34

35 Let s practice in Gephi And if there is time, in Python (code on line, same code as before) 35