Feza Gürsey Institute - Imperial College International Summer School and Research Worshop on Complexity, Istanbul 5 th -10 th September 2011 Tim Evans The Mathematical Description of Networs Page 1
Feza Gürsey Institute - Imperial College International Summer School and Research Worshop on Complexity, Istanbul 5 th -10 th September 2011 Tim Evans The Mathematical Description of Networs 1. Notation 2. Random Graphs 3. Scale-Free Networs 4. Random Wals Page 2
Notation I will focus on Simple Graphs with multiple edges allowed (no values or directions on edges, no values for vertices) N = number of vertices in graph E = number of edges in graph N=6 E=8 Page 3
Notation - Adjacency Matrix The Adjacency Matrix A ij is 1 if vertices i and j are attached 0 if vertices i and j are not attached V6 vertices V1 V2 V3 V4 V5 V6 V1 0 1 1 0 0 0 V2 1 0 1 1 0 0 V3 1 1 0 1 1 0 V4 0 1 1 0 1 1 V5 0 0 1 1 0 0 V6 0 0 0 1 0 0 V5 V3 V1 V4 V2 Page 4
Notation degree of a vertex Number of edges connected to a vertex is called the degree of a vertex = degree of a vertex <> = average degree = (2E / N) Degree Distribution n() = number of vertices with degree p() = n()/n = normalised distribution = probability a vertex chosen at random (uniformly) has degree Degree =2 Page 5
Notation degree distribution Degree Distribution is n() = number of vertices with degree The normalised degree distribution is p() = n()/n = probability a vertex chosen at random (uniformly) has degree =2 Page 6 =2
How to excite a Mathematician give them the simplest model RANDOM GRAPHS Page 7
Classical Random Graphs [Solomonoff-Rapoport 51, Erdős-Réyni 59] For every pair of distinct vertices add a single edge with probability (1-p) p = <>/(N-1), otherwise with probability (1-p) no edge is added p Page 8
Classical Random Graph Gives Binomial Degree Distribution p( ) W p 1 W p with W = N(N-1)/2 number of possible edges and <>=(N-1)p Page 9
Classical Random Graph which is an approximate Normal Distribution exp p( )! with <>=(N-1)p Exponential cutoff so no hubs e.g. N=10 6, <>=4.0, typically has <17 Page 10
Example of Classical Random Graph N=200 <>~4.0 < 11 In figure vertex size Diffuse, no tight cores Page 11
Generalised Random Graphs The Molloy-Reed Construction [1995,1998] i. Fix N vertices ii. Attach stubs to each vertex, where is drawn from given distribution p() iii. Connect pairs of stubs chosen at random Page 12
No Vertex-Vertex Correlations Generalised Random Graphs have given p() but otherwise completely random in particular - Properties of all vertices are the same For any given source vertex, the properties of neighbouring vertices independent of properties of the source vertex Page 13
Random Wals on Random Graphs The degree distribution of a neighbour is not simply p() You are more liely to arrive at a high degree vertex than a low degree one p( ) n p( n i n ) 1 2 3 n Degree of neighbour n independent of degree of starting point i i Page 14
A random friend is more popular than you n n p( ) n i 2 (Number of friends neighbour has) - (Number of your friends) n 2 0 Give a random friend that life saving vaccine (if social networs are random and uncorrelated) Page 15
Length of Random Wals on Random Graphs Suppose we follow a random wal where we never go bac along the edge we just arrived on, then for infinite graphs (N ) Wals always end if n < 2 No GCC 1 2 OUT ( n -1) Wals never end if n > 2 GCC n IN (GCC= Giant Connected Component) Page 16
Length of Random Wals on Random Graphs PROVIDED there are no loops. True for sparse random graphs in limit of infinite size (N ) Wals always end if n < 2 No GCC 1 2 OUT ( n -1) Wals never end if n > 2 GCC n IN (GCC= Giant Connected Component) Page 17
GCC (Giant Connected Component) transition GCC= Giant Connected Component, where a finite fraction of vertices in infinite graph are connected GCC exists if z>1 where 2 OUT z n 1 2 1 1 n ( n -1) IN = Fractional measure of how much more popular your friends are Page 18
Other properties of General Random Graphs All global properties depend on same z 1 e.g. GCC size, component distribution, average path lengths 2 Page 19
Average Path Length in MR Random Graph For any random graph has an average shortest length which scales as ln( N ) d c ln( z)
Six Degrees of Separation [John Guare 1990] I read somewhere that everybody on this planet is separated by only six other people. Six degrees of separation.
Small World A Small World networ is one where the average shortest distance is <d> ~ O(ln(N)) All random graphs are small world In fact most complex networs are small world c.f. a regular lattice in d-dimensions where the distance scales as <d> ~ O(N 1/d )
Watts and Strogatz s Small World Model (1998) Start with lattice, pic random edge and rewire move it to two lin two new vertices chosen at random. 1 dim Lattice Small World Random N=20, E=40, =4 0 5 200 Number of rewirings
Clustering and Length Scale in WS networ Average distance drops very quicly, Loss of local lattice structure much slower Cluster coef N=100, =4, 1-Dim lattice start, 100 runs Distance Lattice Classical Random Graph Small World
Ensembles of Graphs Mathematically we do not consider a single instance of a random graph but an ensemble of random graphs Page 25
Ensembles of Graphs e.g. The probability of creating a particular simple graph with E edges and Ē empty edges is P( G) p 1 E p E Classical Random Graphs Page 26
Ensemble Averages Averages of quantities are strictly over both a) different graphs and b) over some element of a graph e.g. vertices G P( G ) 1 N i i V ( G) Page 27
Exponential Random Graphs (p* models) General ensemble of graphs, those with highest probability obey any given constraints f G P( G) f ( G) P( G) 1 Z e H ( G) H(G) chosen so that graphs with preferred properties are most liely Page 28
Example Graph Hamiltonians H(G) H( G) be Classical random graph with p=2e/(n(n-1)) H ( G) vv b v v Random Graph with given degree distribution. In both cases Lagrange multipliers b, b v fixed by specifying desired values of < E > and < v > Page 29
Summary of Random Graphs Calculations wor because lac of correlations between vertices few loops for large sparse graphs, graphs are basically trees Accessible analytically so can suggest typical behaviour even if very wea e.g. diameter vs N These can be reasonable approximations for many theoretical models Probably not for real world so then use these as a null model.
How to excite a physicist give them a power law SCALE FREE MODELS Page 31
log p() Distribution p() log(p()) 0-1 -2-3 -4-5 -6-7 Long Tails in Real Data Period 2 (excluding Haldane), degree distribution 0 0.5 1 1.5 2 2.5 3 Imperial Library Loans (Hoo, Sooman, Warren, TSE) log() Imperial Medical School (URL, 4 wees, < 3rd Jan 2005) N=14805 <>=22.95 slope=-2.4 1.E+00 1 10 100 1000 10000 100000 1.E-01 1.E-02 1.E-03 1.E-04 1.E-05 1.E-06 Imperial Medical School Web Site (Hoo, TSE) p Po 1.E-07 0-1 -2 Degree distribution, ebay Crawl (max 1000) 0 1 2 3 4 Logged user in degree 1.E-08 1.E-09 Equiv. Binomial Degree -3-4 -5-6 -7-8 -9 (Sooman, Warren, TSE) Page 32 log gallery future-i com [Barabasi, Albert, 1999] All log() vs. log(p()) except text log(ran) vs. log(freq.) Text frequency in TSE Contemporary Physics review
Growth with Preferential Attachment [Yule 1925, 1944; Simon 1955; Price 1965,1976; Barabasi,Albert 1999 ] 1. Add new vertex attached to one end of m=½<> new edges 2. Attach other ends to existing vertices chosen with by picing random end of an existing edge chosen randomly, so probability is P() = / (2E) Preferential Attachment Rich get Richer Page 33 2/(2E) P() 2/(2E) 5/(2E) 4/(2E) Result: Scale-Free n() ~ -g g=3
Growth with Preferential Attachment [Yule 1925, 1944; Simon 1955; Price 1965,1976; Barabasi,Albert 1999 ] P() = / (2E) Preferential Attachment Rich get Richer Result: Scale-Free Networ n() ~ -g g=3 P() 2/(2E) 5/(2E) 4/(2E) 2/(2E) Page 34
N=200, <>~4.0, vertex size Classical Random Scale-Free = Power-Law p()~ 1/ 3 Diffuse, small degree vertices max =O(ln(N)) Page 35 Tight core of large hubs max =O(N 1/2 )
Master Equation Approach Let n(,t) represent the average number of vertices at time t. (I should really use <n(,t)>) Again average means we loo at an ensemble of such networs. The master equation the equation for evolution of the degree distribution averaged over different instances of networ in the ensemble n(,t) to n(,t+1) Page 36
Master Equation Processes n(,t) changes in one of three ways:- Increases as we add an edge to existing vertex of degree (-1). Decreases as we add an edge to existing vertex of degree. Number of vertices of degree =m= ½<> always increase by 1 as add new vertex. (-1) (-1) new vertex Page 37
Mean Field Degree Distribution Master Equation n(, t 1) n(, t) n(, m 1, t) m P( n(, t) m P( ) new vertex P() = Probability of attaching to a vertex of degree 1) in simplest preferential attachment models (-1) (-1) Page 38
The Mean Field Approach is an Approximation Distribution n i () different in each instance i Page 39 Ensembles over many instances i at one time t ni ( ) n ( ) i b b n i ( ) b 1 n ( ) Normalisation of probabilities not usually same for different i If P() is a function of degree then normalisation of this probability is different in each instance of a networ in the ensemble at a single time t. i b
Ensemble Invariants n(, t 1) n(, t) n(, m 1, t) P( n(, t) P( ) 1) Adding one vertex and m= ½<> edges at each time means that the number of edges E(t) = mt + E(0) number of vertices N(t) = t + N(0) are the same for all instances of networ in the ensemble at any one time t. Page 40
The Mean Field Approach Can Be Exact Distribution n i () different in each instance i ni ( ) n ( ) i b b n i ( ) b 1 n ( ) i b Ensembles over many instances i at one time t Normalisation of probabilities the same for different i if b=0 or 1 YES only if Page 41 n i ( ) b n i ( ) b b =0 or b=1
Exact Solution of Master Equation Possible if P( ) p p 2E p r 1 N, Note probability so Lowest degree is Thus 0 p p Preferential Attachment 0 P( ) 1 min Random Attachment 1 min m 21 &! p p p 2 r 1 Page 42
Page 43 Loo for asymptotic solutions Find for > m = ½<> Exact Solution of Master Equation m p p m p p N N p p r p r p P P 2) (1 1 1) ( 2) (1 ) (. 1 1) (. 1) ( ) ( ) ( ) ( ), ( p t N t n
Exact Solution of Master Equation Hence ) 1 ( ) ( ) ( b a a A p Large limit:- A p ) ( lim, 2, p p r p b p p a 2 2 1 p p where &
Scale-Free Growing Model comments Illustrates use of master equations and their approximations statistical physics experience Exact solutions for ensemble average asymptotic value of degree distribution p() if Interpretation of parameters p p >1 allowed Finite Size effects? real networs are mesoscopic Fluctuations in ensemble? P( ) (1 [TSE, Saramäi 2004] Networ not essential =frequency of previous choices Growth not essential networ rewiring g ~ 1.0 p 1 N [Moran model, see TSE,Plato, 2008] r ) 2E p r, Page 45
Scale-Free in the Real World Attachment probability used was 1 P( ) p p 2E p r N, BUT if lim P() a for any a 1 then a power law degree distribution is not produced! Page 46
Preferential Attachment for Real Networs [Saramäi, Kasi 2004; TSE, Saramäi 2004] 1. Add a new vertex with ½<> new edges 2. Attach to existing vertices, found by executing a random wal on the networ of L steps Probability of arriving at a vertex number of ways of arriving at vertex =, the degree Preferential Attachment g=3 Page 47 Start Wal Here (Can also mix in random attachment with probability p r )
Preferential Attachment for Real Networs Probability of arriving at a vertex number of ways of arriving at vertex =, the degree Preferential Attachment =3 Can also mix in random attachment with probability p r Start Wal Here Page 48
Naturalness of the Random Wal algorithm Gives preferential attachment from any networ and hence a scale-free networ Uses only LOCAL information at each vertex Simon/Barabasi-Albert models use global information in their normalisation Uses structure of Networ to produce the networs a self-organising mechanism e.g. informal requests for wor on the film actor s social networ e.g. finding lins to other web pages when writing a new one Page 49
Is the Wal Algorithm Robust? I varied: Length of wals <> Starting point of wals Length distribution of wals.. L=0 Pure Random Attachment exponential <d>~5 graph L=7 L=1 YES - Good Power Laws but NOT Universal values - 10% or 20% variation Page 50
Finite Size Effects Networs are mesoscopic systems In practice a networ of N ~ 1 million is still not large since many quatitities scale with the logarithm of system size e.g. Diameter scales as log(n) ~ 6. Page 51
Finite Size Effects for pure preferential attachment Log 10 (F s ) ( ) ( )., 1 / 2 ( 2) p p F S 1 p ( ) 3 N 2( 1)( 2) Scaling Function F S ( x) 1 Pure Pref.Attach. if x 1 Page 52 100 runs to get enough data near 1
Mean Field Exact Finite Size Scaling Function F s (pure pref.attach.) F Can calculate the finite size effects in the mean field approximation to find s ( x) Page 53 erfc( x) exp( x 2 ) (TSE+Saramäi, 2005; 2x m 2 n3 generalisation of Krapivsy and Redner, 2002) 8 n! Asymptotic analytic solution and two numerical solutions to mean field equations n 1 (1 m) x H ( x) 2m=<> m1, n n3 Hermite Polynomials
What can physicsts and mathematicians do well? RANDOM WALKS Page 54
Properties of irreducible non-negative matrices (1) Will phrase this in terms of Adjacency Matrix A ij for a networ A ij = A ji = 1 for edges in simple graphs A ij is the weight of edge from j to i for weighted networ A ij A ji (symmetric matrix) if directed networ Page 55
Definition of irreducible non-negative matrices (1) In terms of an Adjacency Matrix A ij for a networ A non-negative matrix is A ij 0 Irreducible if there is a path from each vertex to every other vertex n A i, j n 0 s. t. ( ) ij 0 Page 56
Properties of irreducible non-negative matrices (2) Largest eigenvalue (l 1 ) is real and positive Largest eigenvalue is bounded by largest and smallest sums of each row and each column Eigenvector of largest eigenvalue has only positive entries Entries in all other eigenvectors differ in sign Page 57
Random Wal Transition Matrix The transition matrix for a simple unbiased random wal on a networ is T where the probability of moving from vertex j to vertex i is i T ij j A with strength j A ij i ij j 0.25 i Probability of following an edge from j to any vertex i is 0.25 j Page 58
Random Wal Transition Matrix (2) Another useful form is T A. D D ij ij j 1 0.25 i Probability of following an edge from j to any vertex i is 0.25 j Page 59
Transition matrix properties (1) Adjacency matrices of networs are nonnegative (almost always) Irreducible if networ fully connected (or add some wea lins to mae it so) Transition matrix is also non-negative and irreducible i T ij i A ij j j j 1 Must go somewhere j Page 60
Transition matrix properties (2) Transition matrix columns always sum to 1 i T ij i A ij j j j 1 Must go somewhere j Transition matrix has unique largest eigenvalue equal to 1=l 1 Page 61
Transition matrix properties (3) Eigenvector of largest eigenvalue of transition matrix, v 1, of undirected networ is just i. ij T v 1 i j Aij j v1 i j A j i.e. Flow in = Flow Out is equilibrium reached if flow along each edge is equal to the weight of the edge j Page 62
Transition matrix properties (4) Flow in = Flow Out is equilibrium reached if flow along each edge is equal to the weight of the edge 1 For simple graph, ONE waler passes along each edge in each direction at each time step walers arrive and leave each vertex *** NOT solution if number of in- and out-edges different Page 63
Random wal as linear algebra Let w i (t) be the number of random walers at vertex i at time t (or the probability of finding one waler at i ) then the evolution is simply w( t 1) T. w( t ) w ( t 1) T w ( t) i ij j j Page 64
Random wal as linear algebra Decompose w i (t) in terms of eigenvectors v n as w ( t 0) then the evolution is simply c n v n n w( t) n c l v n n t n Page 65
Equilibrium Equilibrium reached is eigenvector with largest eigenvalue as w( t ) v 1 l l n 1 1 n 1 So for simple networs we have w( t ) i i Page 66
PageRan Google rans web pages using v 1 Follows lins between web pages lie a random waler Google maes money because the web is a directed graph so largest eigenvector, v 1, is not trivial Page 67
PageRan for Mathematicians Using MacTutor bibliography of over 200 mathematicians finds [Clare, TSE, Hopins, 2010] Page 68
Centrality Measures The closer a vertex is to the centre of a networ, the higher its Centrality Measures:- Degree PageRan Betweenness Simple Betweenness = number of shortest paths passing through each vertex etc. Page 69
Simple Betweenness 1. Calculate the shortest paths between all pairs of vertices. 2. Betweenness = number of shortest paths passing through each vertex Example BUT ONLY defined for simple graphs Page 70
Electric Current Betweenness [Newman 2005] I Treat undirected networ as resistors, with V R Ohm s Law Conductivity of resistor edge weights = A ij =A ij Voltage at vertex i = V i External current flowing into vertex i = I i I i I j A ( V V ) ij i j j I i i j Page 71
Betweenness and Currents (2) j A ij ( V i V D AV I j ) where D ij = i ij a diagonal matrix using degree I i 1 V D A I Wait till later to see why inverse is OK Page 72
Betweenness and Currents (3) where T =AD -1 is the random waler transition matrix Page 73 I T D I AD D I A D V 1 1 1 1 1 1 1 1
Betweenness and Currents (4) Define the net flow of current through a vertex to be F i so F i 1 2 then using j A ij ( V 1 D 1 1 V i T I V j ) we find that i F i j F i 1 T ((1 T) 1 I) T ((1 T) 1 ij j ji 2 j I ) i Page 74
Betweenness and Currents (5) So net flow of current F i through vertex i is F where i ij 1 2 j, ij is the current flowing from j to i due to external current put in at T ij I (1 T) 1 i j ji I ij j I i Page 75
Betweenness and Currents (6) However in terms of random walers (1 pt) 1 pt n So ij counts the number of random walers starting at, arriving at j after n steps, followed by a move to i T T n ij ij j n n if pl 1 1 Page 76
Betweenness and Currents (7) The total current put into the circuit must match the current taen out I 0 i i In terms of the transition matrix, this means this vector I i does not contain the equilibrium eigenvector with eigenvalue 1 Thus ij T n ij T n is well defined j if p<1 or if acts on I. Page 77
Betweenness and Currents (8) Suppose we put one unit of current in at source vertex s we tae one unit of current out at target vertex t I ( st) i 1 is it jis j 1 t i s jit Page 78
Betweenness and Currents (9) The net flow in terms of positive (from s) and negative (from t) random walers is F ( st) i neg 1 2 j ijs ijt jis jit pos t i j s Page 79
Betweenness and Currents (10) Newman suggests a centrality measure of summing over all possible source and sin currents with ( st) F i 1 2 F F i i j ijs ijt s, t ( st) and ij is the number of random walers starting from passing from j to i after n steps jis jit Page 80
Betweenness and Currents Summary Uses negative random walers Random waler picture wors for directed graphs Does not use equilibrium eigenvector unlie PageRan, Modularity for community detection,... Can introduce distance scale d= p/(1-p) Walers move on with probability p, stop with probability (1-p). Replace TpT and (1-T) -1 (1-p)/(1-pT) Can introduce biased random wals e.g. T ij A i z j ij i T, z j i A [Lambiotte et al. 2011] ij Page 81
Betweenness and Currents Summary Newman suggests a centrality measure of summing over all possible source and sin currents with ( st) F i 1 2 F F i i j ijs ijt s, t ( st) and ij is the number of random walers starting from passing from j to i after n steps jis jit Page 82
THANKS Page 83
Bibliography: Free General Networ Reviews Evans, T.S. Complex Networs, Contemporary Physics, 2004, 45, 455-474 [arxiv.org/cond-mat/0405123] Easley, D. & Kleinberg, J. Networs, Crowds, and Marets: Reasoning About a Highly Connected World Cambridge University Press, 2010 [http://www.cs.cornell.edu/home/leinber/networs-boo/ ] van Steen, M. Graph Theory and Complex Networs Maarten van Steen, 2010 [http://www.distributed-systems.net/gtcn/ ] Hanneman, R. A. & Riddle, M. Introduction to social networ methods 2005 Riverside, CA: University of California, Riverside [http://faculty.ucr.edu/~hanneman/ ] Brandes, U. & Erlebach, T. (ed.) Networ Analysis: Methodological Foundations 2005 [http://www.springerlin.com/content/nv20c2jfpf28/] Page 84
Bibliography in more detail Evans, T.S. Complex Networs, Contemporary Physics, 2004, 45, 455-474 [arxiv.org/condmat/0405123] M.Newman, Networs: An Introduction, OUP, 2010 A.Froncza, P. Froncza and J.A. Holyst, How to calculate the main characteristics of random uncorrelated networs in Science of Complex Networs: From Biology to the Internet and WWW; CNET 2004, (ed.s Mendes,J.F.F. et al.) 776 (2005) 52 [cond-mat/0502663] M. Molloy and B. Reed, A critical point for random graphs with a given degree sequence, Random Structures and Algorithms 6 (1995) 161-180. M. Molloy and B. Reed, The size of the giant component of a random graph with a given degree sequence, Combin. Probab. Comput. 7 (1998) 295-305. Watts, D. J. & Strogatz, S. H., Collective dynamics of 'small-world' networs, Nature, 1998, 393, 440-442. Zachary, W. Information-Flow Model For Conflict And Fission In Small-Groups Journal Of Anthropological Research, 1977, 33, 452-473 Evans, T.S. & Lambiotte, R., Line Graphs, Lin Partitions and Overlapping Communities, Phys.Rev.E, 2009, 80, 016105 [arxiv:0903.2181] Evans, T.S. Clique Graphs and Overlapping Communities, J.Stat.Mech, 2010, P12037 [arxiv.org:1009.063] Blondel, V. D.; Guillaume, J.-L.; Lambiotte, R. & Lefebvre, E., Fast unfolding of community hierarchies in large networs, J STAT MECH-THEORY E, 2008, P10008 [arxiv.org:0803.0476] Also try arxiv.org and search for words networs with school or review in whole entry Page 85
Bibliography in more detail (2) Watts, D. J. & Strogatz, S. H. Collective dynamics of 'small-world' networs Nature, 1998, 393, 440-442 T.S.Evans, J.P.Saramäi Scale Free Networs from Self-Organisation Phys.Rev.E 72 (2005) 1 [cond-mat/0411390] Krapivsy, P. L. & Redner, S. Finiteness and Fluctuations in Growing Networs J. Phys. A, 2002, 35, 9517 M.Newman, A Measure of Betweenness centrality based on random wals Social networs 27 (2005) 39-54 Lambiotte, R.; Sinatra, R.; Delvenne, J.-C.; Evans, T.; Barahona, M. & Latora, V., Flow graphs: interweaving dynamics and structure, Phys. Rev. E 84, 017102 (2011) T.S.Evans & J.Saramäi, Scale-free networs from self-organization Phys.Rev.E. 72 (2005) 026138 [arxiv:cond-mat/0411390] T.S.Evans & A.D.K.Plato, Exact Solution for the Time Evolution of Networ Rewiring Models Phys. Rev. E 75 (2007) 056101 [arxiv:cond-mat/0612214] Page 86
Bibliography: General Networ Reviews Newman, M. Networs: An Introduction, OUP, 2010 Page 87
Conclusions Considerable input possible from from a mathematical approach to networs Google Tim Evans Networs to find my web pages on networs Page 88
Average Path Length in MR Random Graph Let p ij (x) be the probability that a random wal (never returning along last step) starting at vertex i passes through vertex j at least once after x steps Number of different wals of length x from i to j, if no loops, is W(i,x) = i ( n -1) x-1 [Froncza et al, 2005] Final vertex j Initial vertex i i n n n
Average Path Length in MR Random Graph (2) Probability of not arriving at j on any one step = 1- ( j /2E) Probability that a random wal does not arrive at j after x steps is 1 ), ( 2 exp 2 1 ) ( x j i x i W j ij z E E x p 1 1 : 2 z n
Average Path Length in MR Random Graph (3) Probability that waler first arrives after x steps is p ij (x-1) - p ij (x) Average path length from i to j is d ij x1 x p ( x 1) p ( x) ij ij x0 Average path length <d> is (after some wor) d ln( N) ln( z) ln( z) ln( ) p ij 1 2 ( x) [Froncza et al,2005]
Average Path Length in MR Random Graph (4) For any random graph has an average shortest length which scales as ln( N ) d c ln( z)