Branching Out: Quantifying Tree-like Structure in Complex Networks

Size: px

Start display at page:

Download "Branching Out: Quantifying Tree-like Structure in Complex Networks"

Cody Thompson
5 years ago
Views:

Science and Mathematics Division Oak Ridge National

1 Branching Out: Quantifying Tree-like Structure in Complex Networks Blair D. Sullivan Complex Systems Group Center for Engineering Science Advanced Research Computer Science and Mathematics Division Oak Ridge National Laboratory MMDS, July 12, 2012 Joint work with Michael Mahoney & Aaron Adcock, Stanford University

structure at very largescale (diameter), small-scale (clustering coefficient); very few to probe

2 Motivation Large networks are becoming ubiquitous in many domains e.g. biology, physics, chemistry, infrastructure, communications, and sociology Many methods to understand structure at very largescale (diameter), small-scale (clustering coefficient); very few to probe intermediate scale (clusters of size 5K in a 5M node network). Can we get good tools to understand and exploit this? A partial map of the Internet, January The US electric transmission system. Courtesy North American Reliability Corporation. 2 Managed by UT-Battelle Drug-Target Network. Nature Biotechnology 25(10), October 2007

3 Intermediate-Scale Structure Ising model (ferromagnetism): Temperature parameter controls scale of local correlations between magnetic spins. 3 Managed by UT-Battelle

4 Intermediate-Scale Structure Ising model (ferromagnetism): Temperature parameter controls scale of local correlations between magnetic spins. The intermediate-scale structure is the coupling of local & global properties. Determines network evolution & dynamics of diffusion, other processes Implicitly affects applicability of common data analysis tools This is where all the interesting stuff happens. 4 Managed by UT-Battelle

5 Prior empirical evidence Claim: Many large complex networks are tree-like when viewed at intermediate scales: The Unreasonable Effectiveness of Tree-Based Theory for Networks with Clustering, Melnik, Hackett, Porter, Mucha, Gleeson. Physical Review E, Vol. 83, No. 3 (2010). Finding Hierarchy in Directed Online Social Networks, Gupta, Shankar, Li, Muthukrishnan, Iftode. WWW2011. "It was noted in recent years that the Internet structure has a highly connected core and long stretched tendrils, and that most of the routing paths between nodes in the tendrils pass through the core. Therefore, we suggest in this work, to embed the Internet distance metric in a hyperbolic space where routes are bent toward the center Shavitt, Tankel Hyperbolic embedding of internet graph for distance estimation and overlay construction. IEEE/ACM Trans. Netw. 16, 1 (2008). However, no consensus has been reached on defining and measuring this treelike structure, making it difficult to exploit algorithmically. 5 Managed by UT-Battelle Image credit: Munzer et al

Prior empirical evidence Claim: Many large complex networks are tree-like when viewed at intermediate scales: The Unreasonable Effectiveness of Tree-Based Theory for Networks with Clustering, Melnik,

6 Prior empirical evidence Claim: Many large complex networks are tree-like when viewed at intermediate scales: The Unreasonable Effectiveness of Tree-Based Theory for Networks with Clustering, Melnik, Hackett, Porter, Mucha, Gleeson. Physical Review E, Vol. 83, No. 3 (2010). Finding Hierarchy in Directed Online Social Networks, Gupta, Shankar, Li, Muthukrishnan, Iftode. WWW2011. "It was noted in recent years that the Internet structure has a highly connected core and long stretched tendrils, and that most of the routing paths between nodes in the tendrils pass through the core. Therefore, we suggest in this work, to embed the Internet distance metric in a hyperbolic space where routes are bent toward the center Shavitt, Tankel Hyperbolic embedding of internet graph for distance estimation and overlay construction. IEEE/ACM Trans. Netw. 16, 1 (2008). However, no consensus has been reached on defining and measuring this treelike structure, making it difficult to exploit algorithmically. 6 Managed by UT-Battelle

7 What do you mean, tree-like? Facebook: Caltech Network Arxiv GR-QC collaboration Image credit: Traub, Kelsic, Mucha, Porter Image credit: Tim Davis Autonomous Systems 7 Managed by UT-Battelle Image credit: Graphics@Illinois

8 Hyperbolic Space Multiple parallel lines pass through a point, and angles in a triangle sum to less than 180. At right, see a {7,3}-tessellation of the hyperbolic plane by equilateral triangles, and the dual {3,7}-tessellation by regular heptagons. All triangles and heptagons are of the same hyperbolic size but the size of their Euclidean representations exponentially decreases as a function of the distance from the center, while their number exponentially increases. In Euclidean space, a circle s area grows polynomially with its diameter; in hyperbolic space, it grows exponentially. Think of growth as in a binary tree. The shortest paths in hyperbolic spaces are arcs through disk, not paths around the exterior (much like travel in a rooted tree) 8 Managed by UT-Battelle Image credit Krioukov et al.

9 Hyperbolic Embedding and Greedy Routing Hyperbolic space gives us extra room to embed networks (as opposed to Euclidean space). A number of algorithms take advantage of this to devise greedy routing schemes Kleinberg uses a minimum spanning tree, embedded as a subset of a d-regular tree, where d is the maximum degree of the MST (d = 4 is shown at right) Image credit Kleinberg 9 Managed by UT-Battelle

10 So is it good or bad? 10 Managed by UT-Battelle Image credit M.C.Escher

11 A generative model Three-parameter model introduced by Krioukov et al uses an underlying hyperbolic geometry and allows us to vary the curvature, degree heterogeneity, and density. (Physicists: this is basically fermions) Idea: place nodes in the hyperbolic plane (Poincare disk) and connect them with a probability which is dependent on their hyperbolic distance. Knob 1: Power law exponent: determines distribution of nodes in the disk the higher the exponent, the more nodes go towards the center. This determines the curvature (and degree heterogeneity) Knob 2: Temperature: determines how much we ignore the underlying geometry in adding edge; at high temperatures, edge connections become essential random (independent of distance). Knob 3: Average degree (target): approximately allows control over density 11 Managed by UT-Battelle Temp. Curv. Finite Infinite Finite Random hyperbolic graphs Random geometric graphs Our test parameters Infinite Classical random graphs (Erdos-Renyi) Random graphs w/given expected deg. Power Law Temperature Avg. Degree

12 Special Thanks Image credit San Diego Reader Special thanks to D. Krioukov for providing us code to generate networks according to the model described on the previous slide. 12 Managed by UT-Battelle

Hyperbolic Embedding for Inference Boguna, Krioukov, Papadopolous have mapped the internet to hyperbolic space, and used the embedding to identify community structure (and offer

13 Hyperbolic Embedding for Inference Boguna, Krioukov, Papadopolous have mapped the internet to hyperbolic space, and used the embedding to identify community structure (and offer suggested routing schemes). Their methods rely on iterative MLE methods, and do not seem to be scalable to examine big data. 13 Managed by UT-Battelle Image credit Boguna, Krioukov, Papadopolous

14 A geometric measure of tree-likeness Gromov s δ-hyperbolicity arises from the geometry of metric spaces and δ measures the extent to which a (geodesic) metric space embeds in a tree metric. u v δ = 0 u v δ = 1 w x w x d(u,v) + d(w,x) = = 2 d(u,x) + d(v,w) = = 2 d(u,w) + d(v,x) = = 2 d(u,v) + d(w,x) = = 2 d(u,x) + d(v,w) = = 4 d(u,w) + d(v,x) = = 2 Note: d(u,v) is the length of the shortest path between u and v in the graph. The minimum δ for which G is δ-hyperbolic can be computed (naively) in O(n 4 ) 14 Managed by UT-Battelle

More on δ-hyperbolicity Viewing graphs as a geodesic metric space (replace edges with length 1 segments intersecting only at endpoints) provides another way to think of δ-hyperbolicity.

15 More on δ-hyperbolicity Viewing graphs as a geodesic metric space (replace edges with length 1 segments intersecting only at endpoints) provides another way to think of δ-hyperbolicity. For a geodesic triangle, there is a unique isometry to a tripod so that except for the leaves, each point on the tripod has two pre-images on the triangle. Image credit: Chepoi, Dragan et al Image credit: Bridson, Haefliger A triangle is δ-thin if the pre-images of every tripod point have distance at most δ. A triangle is δ-slim if each of its sides is contained in the δ -neighborhood of the union of the other two sides. A graph is δ -hyperbolic if all its geodesic triangles are δ -thin (or δ-slim); each results in a slightly different min δ, related to each other by small constant factors. 15 Managed by UT-Battelle

16 Examples: Small world graphs & Ringed Trees Kleinberg s small-world random graphs add long-range edges with probability proportional to 1/d B (u,v) p to a d-dimensional grid. Mahoney et al (2011) showed even at the sweet spot of p = d, the small-world graphs are not logarithmically hyperbolic w.h.p. When p < d, the graphs are not hyperbolic, and for p > 3 and d = 1, the hyperbolic delta is polynomial in the size of graph. Define a ringed tree to be a binary tree plus edges connecting all vertices at a given tree level into a ring (quasi-isometric to the Poincare disk) 16 Managed by UT-Battelle Image credit: Mahoney et al Replace the ringed tree with a pure binary tree: none of the resulting graphs are hyperbolic. Adding long-range edges between the leaves of a ringed tree w/ probability decreasing: exponentially fast with the ring distance produces logarithmic hyperbolicity as a power-law with the ring distance produces non-hyperbolic random graphs

17 Empirical Results: Real Graphs 17 Managed by UT-Battelle

18 Empirical Results: Planar Planar graphs have a very different distribution of delta over their quadruples, and very high diameters. 18 Managed by UT-Battelle

19 Empirical Results: Hyperbolic? Much more subtle differences when looking at non-planar graphs. Density seems to play a role, and most networks considered had very low diameter. 19 Managed by UT-Battelle

Computing δ: Sampling Due to high computational complexity, a number of prior works have used sampling to estimate the hyperbolicity of large networks. Some prior work sampled at a rate of about.

20 Computing δ: Sampling Due to high computational complexity, a number of prior works have used sampling to estimate the hyperbolicity of large networks. Some prior work sampled at a rate of about.0002 percent (on their largest data), and although biased towards pairs at larger distances, this could still easily miss the maximum delta, which is achieved on a very small (in our example 2 x percent) subset of quadruplets. Note that sampling, however, is likely to be sufficient for computing average deltas. Example below is SNAP graph as (about 1600 nodes) delta Fraction of quadruplets: # of quadruplets 0.0: : : : : : Total Managed by UT-Battelle

21 K-core Decompositions Given a graph G = (V,E), the k-core of the graph, denoted H k is the maximal subgraph H of G so that deg H (v) is at least k for all v in H. The core number of a vertex v is defined to be the maximum k so that v is in H k but not H k+1. The set of nodes with core number k is called the k-shell of G. 21 Managed by UT-Battelle Condensed Matter Collaboration Network Image credit: LaNet-vi

22 Empirical Results: Social Graphs Facebook-Texas84 ~36,000 nodes ~3x10^6 edges soc-epinions1 ~47,000 nodes ~730,000 edges 22 Managed by UT-Battelle

23 Empirical Results: Autonomous Systems AS ~5,500 nodes ~22,000 edges AS ~5,500 nodes ~22,000 edges 23 Managed by UT-Battelle

24 Empirical Results: Collaboration Graphs CA-AstroPhysics ~18,000 nodes ~394,000 edges CA-GrQc ~4,000 nodes ~26,000 edges 24 Managed by UT-Battelle

25 Empirical results: Synthetic by power law exponent 25 Managed by UT-Battelle

26 Empirical results: Synthetic by temperature 26 Managed by UT-Battelle

27 Some (oversimplified) Summary Statistics ca-astrophysics: ~0.6% of nodes (113 nodes) in two deepest cores (k = 55,56) ~1.8% of edges (~7,000 edges) leaving the deepest core (k = 56) ~1.8% of edges (~7000 edges) leaving next core (k = 55) Max average k-shell change is +12 (out of k = 56 max shell) Suggests collaborators tend to collaborate with people of similar coreness/peripheryness Typical for collaboration graphs (and other core-periphery graphs) Texas84: ~8% of nodes ( 2400 nodes) in two deepest cores (k = 80,81) ~7% of edges ( 220K edges) leaving the deepest core (k = 81) ~17% of edges ( 510K edges) leaving the next core (k = 80) Max average k-shell change is +50 (out of k = 80 max shell) Suggests that the periphery nodes are more tightly connected to corelike nodes Typical for more social graphs (and Facebook in particular) 27 Managed by UT-Battelle

28 A combinatorial measure of tree-likeness A tree decomposition of a graph G = (V,E ) is a pair (X={X 1, X 2,..., X L }, T) with X i a subset of V, and T a tree with nodes {1,,L} satisfying three conditions: The union of the sets in X is equal to V For every edge (u,v) in G, {u,v} is a subset of some X i For every v in V, the indices of {X i } containing V form a sub-tree of T. We call the sets X i the bags of the decomposition and max( X i ) the width. The tree-width of G is the minimum width over all valid tree decompositions. 28 Managed by UT-Battelle

29 Understanding FPT: problems are easier on trees Many NP-hard problems can be solved in polynomial time on trees (graphs with no cycles) Example: Maximum Weighted Independent Set: Complexity O( V ) (17,15) 2 (8,10) 1 (7,5) 7 (3,6) 3 (4,1) (3,0) (2,0) (3,0) (2,0) (1,0) (1,0) We can generalize this dynamic programming approach to get polynomial algorithms (in graph size) on graphs where tree-width is bounded. 29 Managed by UT-Battelle

30 Heuristics for low-width decompositions In numerical linear algebra, one often wants to permute the rows of a matrix before computing a factorization so that the resulting factors are as sparse as possible. The objective is to minimize the number of fill edges added. For tree decompositions, we instead need to minimize the maximum clique size in the resulting chordal graph. Numerous implementations of common heuristics are available, and we tested several on a large set of random graphs with a fixed maximum width and varying sizes. Comparison of width and fill from 6 heuristics on graphs known to have tw <= Managed by UT-Battelle Min-degree-based heuristics are orders of magnitude faster than min-fill, etc.

31 Empirical results: Synthetic AMD Upper Bounds: MCS Lower Bounds: 31 Managed by UT-Battelle

32 More AMD Upper Bounds: MCS Lower Bounds: 32 Managed by UT-Battelle

33 Empirical Results: Facebook 33 Managed by UT-Battelle

34 Empirical Results: Autonomous Systems A larger AS graph had similar results: 600K nodes resulted in a 200K largest connected component, and the upper bound was 5961, lower bound Managed by UT-Battelle

35 Problems with Using Tree Decompositions Every bag in a tree decomposition is a vertex separator, so a low-width decomposition means many small separators. Treewidth is O(n) w/ high probability for many random graphs (Gao 2009): Erdos-Renyi graphs G(n,m) when m/n > Random intersection graphs G(n,m,p) on universe {1, m} with m=n a, p at least 2/m and a > 0. Barabasi-Albert preferential attachment with at least 12 new edges for each additional vertex. Current heuristics get lost in local noise 35 Managed by UT-Battelle

36 Average k-cores on a tree decomposition 36 Managed by UT-Battelle Temperature: 20 Power law exp: 2.1 Avg deg target: 5

37 Average k-cores on a tree decomposition 37 Managed by UT-Battelle Temperature: 0.5 Power law exp: 2.1 Avg deg target: 20

38 Real Graphs 38 Managed by UT-Battelle

39 What s next? Clustering Diffusions Sparse Dimensionality Reduction Applications to Statistical Inference 39 Managed by UT-Battelle

40 Acknowledgements Primary support for this work through the ORNL Laboratory Directed Research & Development SEED Program. These slides would not have been possible without many hours of hard work by Aaron Adcock. 40 Managed by UT-Battelle

41 41 Managed by UT-Battelle Backup Slides

42 Motivation for some improvements to min-degree Minimum Fill-In Eliminate 2 Minimum Degree Eliminate 9 42 Managed by UT-Battelle

43 Tiebreaking with second neighbors Joint work with Gloria D Azevedo (ORHS student) and Chris Groer (ORNL). Gloria investigated various strategies for breaking ties within min-degree and min-fill algorithms Her hypothesis was that including information about second-neighborhoods could improve the quality of these heuristics Even with optimizations, the running time of the improved algorithms was often significantly slower than random tie-breaks due to computation of additional information (fill or second-neighborhood sizes) 43 Managed by UT-Battelle

44 An example where second neighbors help MIND MIND+(0.5)(SEC) 44 Managed by UT-Battelle

45 45 Managed by UT-Battelle : 0.5: 1.0: 1.5: 2.0: 2.5:

Lesson 4. Random graphs. Sergio Barbarossa. UPC - Barcelona - July 2008

Lesson 4. Random graphs. Sergio Barbarossa. UPC - Barcelona - July 2008 Lesson 4 Random graphs Sergio Barbarossa Graph models 1. Uncorrelated random graph (Erdős, Rényi) N nodes are connected through n edges which are chosen randomly from the possible configurations 2. Binomial