The Mathematical Description of Networks

Size: px

Start display at page:

Download "The Mathematical Description of Networks"

Arnold Randolph Walters
6 years ago
Views:

1 Feza Gürsey Institute - Imperial College International Summer School and Research Worshop on Complexity, Istanbul 5 th -10 th September 2011 Tim Evans The Mathematical Description of Networs Page 1

2 Feza Gürsey Institute - Imperial College International Summer School and Research Worshop on Complexity, Istanbul 5 th -10 th September 2011 Tim Evans The Mathematical Description of Networs 1. Notation 2. Random Graphs 3. Scale-Free Networs 4. Random Wals Page 2

3 Notation I will focus on Simple Graphs with multiple edges allowed (no values or directions on edges, no values for vertices) N = number of vertices in graph E = number of edges in graph N=6 E=8 Page 3

4 Notation - Adjacency Matrix The Adjacency Matrix A ij is 1 if vertices i and j are attached 0 if vertices i and j are not attached V6 vertices V1 V2 V3 V4 V5 V6 V V V V V V V5 V3 V1 V4 V2 Page 4

5 Notation degree of a vertex Number of edges connected to a vertex is called the degree of a vertex = degree of a vertex <> = average degree = (2E / N) Degree Distribution n() = number of vertices with degree p() = n()/n = normalised distribution = probability a vertex chosen at random (uniformly) has degree Degree =2 Page 5

6 Notation degree distribution Degree Distribution is n() = number of vertices with degree The normalised degree distribution is p() = n()/n = probability a vertex chosen at random (uniformly) has degree =2 Page 6 =2

7 How to excite a Mathematician give them the simplest model RANDOM GRAPHS Page 7

8 Classical Random Graphs [Solomonoff-Rapoport 51, Erdős-Réyni 59] For every pair of distinct vertices add a single edge with probability (1-p) p = <>/(N-1), otherwise with probability (1-p) no edge is added p Page 8

9 Classical Random Graph Gives Binomial Degree Distribution p( ) W p 1 W p with W = N(N-1)/2 number of possible edges and <>=(N-1)p Page 9

10 Classical Random Graph which is an approximate Normal Distribution exp p( )! with <>=(N-1)p Exponential cutoff so no hubs e.g. N=10 6, <>=4.0, typically has <17 Page 10

11 Example of Classical Random Graph N=200 <>~4.0 < 11 In figure vertex size Diffuse, no tight cores Page 11

12 Generalised Random Graphs The Molloy-Reed Construction [1995,1998] i. Fix N vertices ii. Attach stubs to each vertex, where is drawn from given distribution p() iii. Connect pairs of stubs chosen at random Page 12

13 No Vertex-Vertex Correlations Generalised Random Graphs have given p() but otherwise completely random in particular - Properties of all vertices are the same For any given source vertex, the properties of neighbouring vertices independent of properties of the source vertex Page 13

14 Random Wals on Random Graphs The degree distribution of a neighbour is not simply p() You are more liely to arrive at a high degree vertex than a low degree one p( ) n p( n i n ) n Degree of neighbour n independent of degree of starting point i i Page 14

15 A random friend is more popular than you n n p( ) n i 2 (Number of friends neighbour has) - (Number of your friends) n 2 0 Give a random friend that life saving vaccine (if social networs are random and uncorrelated) Page 15

16 Length of Random Wals on Random Graphs Suppose we follow a random wal where we never go bac along the edge we just arrived on, then for infinite graphs (N ) Wals always end if n < 2 No GCC 1 2 OUT ( n -1) Wals never end if n > 2 GCC n IN (GCC= Giant Connected Component) Page 16

17 Length of Random Wals on Random Graphs PROVIDED there are no loops. True for sparse random graphs in limit of infinite size (N ) Wals always end if n < 2 No GCC 1 2 OUT ( n -1) Wals never end if n > 2 GCC n IN (GCC= Giant Connected Component) Page 17

18 GCC (Giant Connected Component) transition GCC= Giant Connected Component, where a finite fraction of vertices in infinite graph are connected GCC exists if z>1 where 2 OUT z n n ( n -1) IN = Fractional measure of how much more popular your friends are Page 18

19 Other properties of General Random Graphs All global properties depend on same z 1 e.g. GCC size, component distribution, average path lengths 2 Page 19

20 Average Path Length in MR Random Graph For any random graph has an average shortest length which scales as ln( N ) d c ln( z)

21 Six Degrees of Separation [John Guare 1990] I read somewhere that everybody on this planet is separated by only six other people. Six degrees of separation.

22 Small World A Small World networ is one where the average shortest distance is <d> ~ O(ln(N)) All random graphs are small world In fact most complex networs are small world c.f. a regular lattice in d-dimensions where the distance scales as <d> ~ O(N 1/d )

23 Watts and Strogatz s Small World Model (1998) Start with lattice, pic random edge and rewire move it to two lin two new vertices chosen at random. 1 dim Lattice Small World Random N=20, E=40, = Number of rewirings

24 Clustering and Length Scale in WS networ Average distance drops very quicly, Loss of local lattice structure much slower Cluster coef N=100, =4, 1-Dim lattice start, 100 runs Distance Lattice Classical Random Graph Small World

25 Ensembles of Graphs Mathematically we do not consider a single instance of a random graph but an ensemble of random graphs Page 25

26 Ensembles of Graphs e.g. The probability of creating a particular simple graph with E edges and Ē empty edges is P( G) p 1 E p E Classical Random Graphs Page 26

27 Ensemble Averages Averages of quantities are strictly over both a) different graphs and b) over some element of a graph e.g. vertices G P( G ) 1 N i i V ( G) Page 27

28 Exponential Random Graphs (p* models) General ensemble of graphs, those with highest probability obey any given constraints f G P( G) f ( G) P( G) 1 Z e H ( G) H(G) chosen so that graphs with preferred properties are most liely Page 28

29 Example Graph Hamiltonians H(G) H( G) be Classical random graph with p=2e/(n(n-1)) H ( G) vv b v v Random Graph with given degree distribution. In both cases Lagrange multipliers b, b v fixed by specifying desired values of < E > and < v > Page 29

30 Summary of Random Graphs Calculations wor because lac of correlations between vertices few loops for large sparse graphs, graphs are basically trees Accessible analytically so can suggest typical behaviour even if very wea e.g. diameter vs N These can be reasonable approximations for many theoretical models Probably not for real world so then use these as a null model.

31 How to excite a physicist give them a power law SCALE FREE MODELS Page 31

32 log p() Distribution p() log(p()) Long Tails in Real Data Period 2 (excluding Haldane), degree distribution Imperial Library Loans (Hoo, Sooman, Warren, TSE) log() Imperial Medical School (URL, 4 wees, < 3rd Jan 2005) N=14805 <>=22.95 slope= E E-01 1.E-02 1.E-03 1.E-04 1.E-05 1.E-06 Imperial Medical School Web Site (Hoo, TSE) p Po 1.E Degree distribution, ebay Crawl (max 1000) Logged user in degree 1.E-08 1.E-09 Equiv. Binomial Degree (Sooman, Warren, TSE) Page 32 log gallery future-i com [Barabasi, Albert, 1999] All log() vs. log(p()) except text log(ran) vs. log(freq.) Text frequency in TSE Contemporary Physics review

33 Growth with Preferential Attachment [Yule 1925, 1944; Simon 1955; Price 1965,1976; Barabasi,Albert 1999 ] 1. Add new vertex attached to one end of m=½<> new edges 2. Attach other ends to existing vertices chosen with by picing random end of an existing edge chosen randomly, so probability is P() = / (2E) Preferential Attachment Rich get Richer Page 33 2/(2E) P() 2/(2E) 5/(2E) 4/(2E) Result: Scale-Free n() ~ -g g=3

34 Growth with Preferential Attachment [Yule 1925, 1944; Simon 1955; Price 1965,1976; Barabasi,Albert 1999 ] P() = / (2E) Preferential Attachment Rich get Richer Result: Scale-Free Networ n() ~ -g g=3 P() 2/(2E) 5/(2E) 4/(2E) 2/(2E) Page 34

35 N=200, <>~4.0, vertex size Classical Random Scale-Free = Power-Law p()~ 1/ 3 Diffuse, small degree vertices max =O(ln(N)) Page 35 Tight core of large hubs max =O(N 1/2 )

36 Master Equation Approach Let n(,t) represent the average number of vertices at time t. (I should really use <n(,t)>) Again average means we loo at an ensemble of such networs. The master equation the equation for evolution of the degree distribution averaged over different instances of networ in the ensemble n(,t) to n(,t+1) Page 36

37 Master Equation Processes n(,t) changes in one of three ways:- Increases as we add an edge to existing vertex of degree (-1). Decreases as we add an edge to existing vertex of degree. Number of vertices of degree =m= ½<> always increase by 1 as add new vertex. (-1) (-1) new vertex Page 37

38 Mean Field Degree Distribution Master Equation n(, t 1) n(, t) n(, m 1, t) m P( n(, t) m P( ) new vertex P() = Probability of attaching to a vertex of degree 1) in simplest preferential attachment models (-1) (-1) Page 38

39 The Mean Field Approach is an Approximation Distribution n i () different in each instance i Page 39 Ensembles over many instances i at one time t ni ( ) n ( ) i b b n i ( ) b 1 n ( ) Normalisation of probabilities not usually same for different i If P() is a function of degree then normalisation of this probability is different in each instance of a networ in the ensemble at a single time t. i b

40 Ensemble Invariants n(, t 1) n(, t) n(, m 1, t) P( n(, t) P( ) 1) Adding one vertex and m= ½<> edges at each time means that the number of edges E(t) = mt + E(0) number of vertices N(t) = t + N(0) are the same for all instances of networ in the ensemble at any one time t. Page 40

41 The Mean Field Approach Can Be Exact Distribution n i () different in each instance i ni ( ) n ( ) i b b n i ( ) b 1 n ( ) i b Ensembles over many instances i at one time t Normalisation of probabilities the same for different i if b=0 or 1 YES only if Page 41 n i ( ) b n i ( ) b b =0 or b=1

42 Exact Solution of Master Equation Possible if P( ) p p 2E p r 1 N, Note probability so Lowest degree is Thus 0 p p Preferential Attachment 0 P( ) 1 min Random Attachment 1 min m 21 &! p p p 2 r 1 Page 42

43 Page 43 Loo for asymptotic solutions Find for > m = ½<> Exact Solution of Master Equation m p p m p p N N p p r p r p P P 2) (1 1 1) ( 2) (1 ) (. 1 1) (. 1) ( ) ( ) ( ) ( ), ( p t N t n

44 Exact Solution of Master Equation Hence ) 1 ( ) ( ) ( b a a A p Large limit:- A p ) ( lim, 2, p p r p b p p a p p where &

45 Scale-Free Growing Model comments Illustrates use of master equations and their approximations statistical physics experience Exact solutions for ensemble average asymptotic value of degree distribution p() if Interpretation of parameters p p >1 allowed Finite Size effects? real networs are mesoscopic Fluctuations in ensemble? P( ) (1 [TSE, Saramäi 2004] Networ not essential =frequency of previous choices Growth not essential networ rewiring g ~ 1.0 p 1 N [Moran model, see TSE,Plato, 2008] r ) 2E p r, Page 45

46 Scale-Free in the Real World Attachment probability used was 1 P( ) p p 2E p r N, BUT if lim P() a for any a 1 then a power law degree distribution is not produced! Page 46

47 Preferential Attachment for Real Networs [Saramäi, Kasi 2004; TSE, Saramäi 2004] 1. Add a new vertex with ½<> new edges 2. Attach to existing vertices, found by executing a random wal on the networ of L steps Probability of arriving at a vertex number of ways of arriving at vertex =, the degree Preferential Attachment g=3 Page 47 Start Wal Here (Can also mix in random attachment with probability p r )

48 Preferential Attachment for Real Networs Probability of arriving at a vertex number of ways of arriving at vertex =, the degree Preferential Attachment =3 Can also mix in random attachment with probability p r Start Wal Here Page 48

Naturalness of the Random Wal algorithm Gives preferential attachment from any networ and hence a scale-free networ Uses only LOCAL information at each vertex Simon/Barabasi-Albert models use global

49 Naturalness of the Random Wal algorithm Gives preferential attachment from any networ and hence a scale-free networ Uses only LOCAL information at each vertex Simon/Barabasi-Albert models use global information in their normalisation Uses structure of Networ to produce the networs a self-organising mechanism e.g. informal requests for wor on the film actor s social networ e.g. finding lins to other web pages when writing a new one Page 49

50 Is the Wal Algorithm Robust? I varied: Length of wals <> Starting point of wals Length distribution of wals.. L=0 Pure Random Attachment exponential <d>~5 graph L=7 L=1 YES - Good Power Laws but NOT Universal values - 10% or 20% variation Page 50

51 Finite Size Effects Networs are mesoscopic systems In practice a networ of N ~ 1 million is still not large since many quatitities scale with the logarithm of system size e.g. Diameter scales as log(n) ~ 6. Page 51

52 Finite Size Effects for pure preferential attachment Log 10 (F s ) ( ) ( )., 1 / 2 ( 2) p p F S 1 p ( ) 3 N 2( 1)( 2) Scaling Function F S ( x) 1 Pure Pref.Attach. if x 1 Page runs to get enough data near 1

53 Mean Field Exact Finite Size Scaling Function F s (pure pref.attach.) F Can calculate the finite size effects in the mean field approximation to find s ( x) Page 53 erfc( x) exp( x 2 ) (TSE+Saramäi, 2005; 2x m 2 n3 generalisation of Krapivsy and Redner, 2002) 8 n! Asymptotic analytic solution and two numerical solutions to mean field equations n 1 (1 m) x H ( x) 2m=<> m1, n n3 Hermite Polynomials

54 What can physicsts and mathematicians do well? RANDOM WALKS Page 54

55 Properties of irreducible non-negative matrices (1) Will phrase this in terms of Adjacency Matrix A ij for a networ A ij = A ji = 1 for edges in simple graphs A ij is the weight of edge from j to i for weighted networ A ij A ji (symmetric matrix) if directed networ Page 55

56 Definition of irreducible non-negative matrices (1) In terms of an Adjacency Matrix A ij for a networ A non-negative matrix is A ij 0 Irreducible if there is a path from each vertex to every other vertex n A i, j n 0 s. t. ( ) ij 0 Page 56

57 Properties of irreducible non-negative matrices (2) Largest eigenvalue (l 1 ) is real and positive Largest eigenvalue is bounded by largest and smallest sums of each row and each column Eigenvector of largest eigenvalue has only positive entries Entries in all other eigenvectors differ in sign Page 57

58 Random Wal Transition Matrix The transition matrix for a simple unbiased random wal on a networ is T where the probability of moving from vertex j to vertex i is i T ij j A with strength j A ij i ij j 0.25 i Probability of following an edge from j to any vertex i is 0.25 j Page 58

59 Random Wal Transition Matrix (2) Another useful form is T A. D D ij ij j i Probability of following an edge from j to any vertex i is 0.25 j Page 59

60 Transition matrix properties (1) Adjacency matrices of networs are nonnegative (almost always) Irreducible if networ fully connected (or add some wea lins to mae it so) Transition matrix is also non-negative and irreducible i T ij i A ij j j j 1 Must go somewhere j Page 60

61 Transition matrix properties (2) Transition matrix columns always sum to 1 i T ij i A ij j j j 1 Must go somewhere j Transition matrix has unique largest eigenvalue equal to 1=l 1 Page 61

62 Transition matrix properties (3) Eigenvector of largest eigenvalue of transition matrix, v 1, of undirected networ is just i. ij T v 1 i j Aij j v1 i j A j i.e. Flow in = Flow Out is equilibrium reached if flow along each edge is equal to the weight of the edge j Page 62

63 Transition matrix properties (4) Flow in = Flow Out is equilibrium reached if flow along each edge is equal to the weight of the edge 1 For simple graph, ONE waler passes along each edge in each direction at each time step walers arrive and leave each vertex *** NOT solution if number of in- and out-edges different Page 63

64 Random wal as linear algebra Let w i (t) be the number of random walers at vertex i at time t (or the probability of finding one waler at i ) then the evolution is simply w( t 1) T. w( t ) w ( t 1) T w ( t) i ij j j Page 64

65 Random wal as linear algebra Decompose w i (t) in terms of eigenvectors v n as w ( t 0) then the evolution is simply c n v n n w( t) n c l v n n t n Page 65

66 Equilibrium Equilibrium reached is eigenvector with largest eigenvalue as w( t ) v 1 l l n 1 1 n 1 So for simple networs we have w( t ) i i Page 66

67 PageRan Google rans web pages using v 1 Follows lins between web pages lie a random waler Google maes money because the web is a directed graph so largest eigenvector, v 1, is not trivial Page 67

68 PageRan for Mathematicians Using MacTutor bibliography of over 200 mathematicians finds [Clare, TSE, Hopins, 2010] Page 68

69 Centrality Measures The closer a vertex is to the centre of a networ, the higher its Centrality Measures:- Degree PageRan Betweenness Simple Betweenness = number of shortest paths passing through each vertex etc. Page 69

70 Simple Betweenness 1. Calculate the shortest paths between all pairs of vertices. 2. Betweenness = number of shortest paths passing through each vertex Example BUT ONLY defined for simple graphs Page 70

71 Electric Current Betweenness [Newman 2005] I Treat undirected networ as resistors, with V R Ohm s Law Conductivity of resistor edge weights = A ij =A ij Voltage at vertex i = V i External current flowing into vertex i = I i I i I j A ( V V ) ij i j j I i i j Page 71

72 Betweenness and Currents (2) j A ij ( V i V D AV I j ) where D ij = i ij a diagonal matrix using degree I i 1 V D A I Wait till later to see why inverse is OK Page 72

73 Betweenness and Currents (3) where T =AD -1 is the random waler transition matrix Page 73 I T D I AD D I A D V

74 Betweenness and Currents (4) Define the net flow of current through a vertex to be F i so F i 1 2 then using j A ij ( V 1 D 1 1 V i T I V j ) we find that i F i j F i 1 T ((1 T) 1 I) T ((1 T) 1 ij j ji 2 j I ) i Page 74

75 Betweenness and Currents (5) So net flow of current F i through vertex i is F where i ij 1 2 j, ij is the current flowing from j to i due to external current put in at T ij I (1 T) 1 i j ji I ij j I i Page 75

76 Betweenness and Currents (6) However in terms of random walers (1 pt) 1 pt n So ij counts the number of random walers starting at, arriving at j after n steps, followed by a move to i T T n ij ij j n n if pl 1 1 Page 76

77 Betweenness and Currents (7) The total current put into the circuit must match the current taen out I 0 i i In terms of the transition matrix, this means this vector I i does not contain the equilibrium eigenvector with eigenvalue 1 Thus ij T n ij T n is well defined j if p<1 or if acts on I. Page 77

78 Betweenness and Currents (8) Suppose we put one unit of current in at source vertex s we tae one unit of current out at target vertex t I ( st) i 1 is it jis j 1 t i s jit Page 78

79 Betweenness and Currents (9) The net flow in terms of positive (from s) and negative (from t) random walers is F ( st) i neg 1 2 j ijs ijt jis jit pos t i j s Page 79

80 Betweenness and Currents (10) Newman suggests a centrality measure of summing over all possible source and sin currents with ( st) F i 1 2 F F i i j ijs ijt s, t ( st) and ij is the number of random walers starting from passing from j to i after n steps jis jit Page 80

81 Betweenness and Currents Summary Uses negative random walers Random waler picture wors for directed graphs Does not use equilibrium eigenvector unlie PageRan, Modularity for community detection,... Can introduce distance scale d= p/(1-p) Walers move on with probability p, stop with probability (1-p). Replace TpT and (1-T) -1 (1-p)/(1-pT) Can introduce biased random wals e.g. T ij A i z j ij i T, z j i A [Lambiotte et al. 2011] ij Page 81

82 Betweenness and Currents Summary Newman suggests a centrality measure of summing over all possible source and sin currents with ( st) F i 1 2 F F i i j ijs ijt s, t ( st) and ij is the number of random walers starting from passing from j to i after n steps jis jit Page 82

83 THANKS Page 83

84 Bibliography: Free General Networ Reviews Evans, T.S. Complex Networs, Contemporary Physics, 2004, 45, [arxiv.org/cond-mat/ ] Easley, D. & Kleinberg, J. Networs, Crowds, and Marets: Reasoning About a Highly Connected World Cambridge University Press, 2010 [ ] van Steen, M. Graph Theory and Complex Networs Maarten van Steen, 2010 [ ] Hanneman, R. A. & Riddle, M. Introduction to social networ methods 2005 Riverside, CA: University of California, Riverside [ ] Brandes, U. & Erlebach, T. (ed.) Networ Analysis: Methodological Foundations 2005 [ Page 84

85 Bibliography in more detail Evans, T.S. Complex Networs, Contemporary Physics, 2004, 45, [arxiv.org/condmat/ ] M.Newman, Networs: An Introduction, OUP, 2010 A.Froncza, P. Froncza and J.A. Holyst, How to calculate the main characteristics of random uncorrelated networs in Science of Complex Networs: From Biology to the Internet and WWW; CNET 2004, (ed.s Mendes,J.F.F. et al.) 776 (2005) 52 [cond-mat/ ] M. Molloy and B. Reed, A critical point for random graphs with a given degree sequence, Random Structures and Algorithms 6 (1995) M. Molloy and B. Reed, The size of the giant component of a random graph with a given degree sequence, Combin. Probab. Comput. 7 (1998) Watts, D. J. & Strogatz, S. H., Collective dynamics of 'small-world' networs, Nature, 1998, 393, Zachary, W. Information-Flow Model For Conflict And Fission In Small-Groups Journal Of Anthropological Research, 1977, 33, Evans, T.S. & Lambiotte, R., Line Graphs, Lin Partitions and Overlapping Communities, Phys.Rev.E, 2009, 80, [arxiv: ] Evans, T.S. Clique Graphs and Overlapping Communities, J.Stat.Mech, 2010, P12037 [arxiv.org: ] Blondel, V. D.; Guillaume, J.-L.; Lambiotte, R. & Lefebvre, E., Fast unfolding of community hierarchies in large networs, J STAT MECH-THEORY E, 2008, P10008 [arxiv.org: ] Also try arxiv.org and search for words networs with school or review in whole entry Page 85

86 Bibliography in more detail (2) Watts, D. J. & Strogatz, S. H. Collective dynamics of 'small-world' networs Nature, 1998, 393, T.S.Evans, J.P.Saramäi Scale Free Networs from Self-Organisation Phys.Rev.E 72 (2005) 1 [cond-mat/ ] Krapivsy, P. L. & Redner, S. Finiteness and Fluctuations in Growing Networs J. Phys. A, 2002, 35, 9517 M.Newman, A Measure of Betweenness centrality based on random wals Social networs 27 (2005) Lambiotte, R.; Sinatra, R.; Delvenne, J.-C.; Evans, T.; Barahona, M. & Latora, V., Flow graphs: interweaving dynamics and structure, Phys. Rev. E 84, (2011) T.S.Evans & J.Saramäi, Scale-free networs from self-organization Phys.Rev.E. 72 (2005) [arxiv:cond-mat/ ] T.S.Evans & A.D.K.Plato, Exact Solution for the Time Evolution of Networ Rewiring Models Phys. Rev. E 75 (2007) [arxiv:cond-mat/ ] Page 86

87 Bibliography: General Networ Reviews Newman, M. Networs: An Introduction, OUP, 2010 Page 87

88 Conclusions Considerable input possible from from a mathematical approach to networs Google Tim Evans Networs to find my web pages on networs Page 88

89 Average Path Length in MR Random Graph Let p ij (x) be the probability that a random wal (never returning along last step) starting at vertex i passes through vertex j at least once after x steps Number of different wals of length x from i to j, if no loops, is W(i,x) = i ( n -1) x-1 [Froncza et al, 2005] Final vertex j Initial vertex i i n n n

90 Average Path Length in MR Random Graph (2) Probability of not arriving at j on any one step = 1- ( j /2E) Probability that a random wal does not arrive at j after x steps is 1 ), ( 2 exp 2 1 ) ( x j i x i W j ij z E E x p 1 1 : 2 z n

91 Average Path Length in MR Random Graph (3) Probability that waler first arrives after x steps is p ij (x-1) - p ij (x) Average path length from i to j is d ij x1 x p ( x 1) p ( x) ij ij x0 Average path length <d> is (after some wor) d ln( N) ln( z) ln( z) ln( ) p ij 1 2 ( x) [Froncza et al,2005]

92 Average Path Length in MR Random Graph (4) For any random graph has an average shortest length which scales as ln( N ) d c ln( z)

The Mathematical Description of Networks

Modelling Complex Systems University of Manchester, 21 st 23 rd June 2010 Tim Evans Theoretical Physics The Mathematical Description of Networs Page 1 Notation I will focus on Simple Graphs with multiple