Gaps between theory practice in!!! large scale matrix computations for networks

Transcription

1 Gaps between theory practice in!!! large scale matrix computations for networks David F. Gleich Assistant Professor Computer Science Purdue University

2 Networks as matrices Bott, Genetic Psychology Manuscripts, 1928

3 Networks as matrices

4 Networks as matrices

5 Networks as matrices A =

6 Everything in the world can be explained by a matrix, and we see how deep the rabbit hole goes The talk ends, you believe -- whatever you want to. Image from rockysprings, deviantart, CC share-alike

7

8 Matrix computations in a red-pill Solve a problem better by exploiting its structure!

9 My research! Models and algorithms for high performance! matrix and network computations on data Network alignment methods: j k i Triangle j 0 i 0 Big data methods too k 0 A L B Massive matrix " computations Ax = b min kax bk Ax = x on multi-threaded and distributed architectures Fast & Scalable" Network analysis

10 One canonical problem PageRank ( A T D 1 )x = f Protein function prediction Personalized PageRank A D adjacency matrix degree matrix Gene-experiment association Semi-supervised" learning on graphs regularization Network alignment f prior or givens Food webs

11 One canonical problem ( A T D 1 )x = f Vahab - clustering Karl - clustering Art prediction Jen - prediction Sebastiano ranking/centrality

12 An example on a graph PageRank b B B B B B 2 4 1/ 1/ / 0 0 1/ / 1/2 0 1/ / 0 1/ / 0 1/2 1/ / C C C C C C A 2 4 x 1 x 2 x 3 x 4 x 5 x 3 5 = ( A T D 1 )x = f non-singular linear system ( < 1), non-negative inverse, works with weights, directed & undirected graphs, weights that don t sum to less than 1 in each column,

13 An example on a bigger graph f has a single " one here Newman s netscience graph 39 vertices 1828 non-zeros zero on most of the nodes

14 A special case one column or one node ( A T D 1 )x = e i x = column i of ( A T D 1 ) 1 localized solutions

15 An example on a bigger graph Crawl of flickr from 200 ~800k nodes, M edges, alpha=1/ error x true x nnz plot(x) x nonzeros

16 Complexity is complex Linear system O(n 3 ) Sparse linear sys. (undir.) O(m log(m) ) where is a function of latest result on solving SDD systems on graphs Neumann series O(m log( )/log(tol))

17 Forsythe and Liebler, 1950

18 Monte Carlo methods for PageRank K. Avrachenkov et al Monte Carlo methods in PageRank Fogaras et al Fully scaling personalized PageRank. Das Sarma et al Estimating PageRank on graph streams. Bahmani et al Fast and Incremental Personalized PageRank Bahmani et al PageRank & MapReduce Borgs et al Sublinear PageRank Complexity O(log V )

19 Gauss-Seidel and Gauss-Southwell Methods to solve A x = b Update x (k+1) = x (k) + j e j such that [Ax (k+1) ] j =[b] j In words Relax or free the jth coordinate of your solution vector in order to satisfy the jth equation of your linear system. Gauss-Seidel repeatedly cycle through j = 1 to n Gauss-Southwell use the value of j that has the highest magnitude residual r (k) = b Ax (k)

20 Matrix computations in a red-pill Solve a problem better by exploiting its structure!

21 Gauss-Seidel/Southwell for PageRank w/ access to in-links & degs. PageRankPull j = blue node x (k+1) j x (k+1) j = f j Solve for x (k+1) j X i!j x (k) a / x (k) (k) b /2 x c /3 x (k) i /deg i = f j w/ access to out-links PageRankPush Let b r (k) = f + A T D 1 x (k) a c then x (k+1) j = x (k) j + r j j = blue node Update r (k+1) r (k+1) j =0 r a (k+1) = r a (k) + r (k) j /3 r (k+1) b = r (k) b + r (k) j /3 r (k+1) = r (k) + r (k) /3 x (k)

22 Python code for PPR Push # main loop! while sumr > eps/(1-alpha):! j = max(r.iteritems,! key=(lambda x: r[x])! rj = r[j]! x[j] += rj! r[j] = 0! sumr -= rj! deg = len(graph[j])! for i in graph[j]:! if i not in r: r[i] = 0.! r[j] += alpha/deg*rj! sumr += alpha/deg*rj!! If f 0, this terminates when x true x alg 1 < # initialization! # graph is a set of sets! # eps is stopping tol! # 0 < alpha < 1! x = dict()! r = dict()! sumr = 0.! for (node,fi) in f.items():! r[node] = fi! sumr += fi!

23 Relaxation methods for PageRank Arasu et al. 2002, PageRank computation and the structure of the web Jeh and Widom 2003, Scaling personalized PageRank McSherry 2005, A unified framework for PageRank acceleration Andersen et al. 200, Local PageRank Berkhin 200, Bookmark coloring algorithm for PageRank Complexity O( E )

24 10 5 Monte Carlo Relaxation x true x alg gap Sublinear" in theory 10 5 nnz(a) gap Node degree=155 22k node, 2M edge Facebook graph Number of edges the algorithm touches 10" nnz(a) How I d solve it

25 Matrix computations in a red-pill Solve a problem better by exploiting its structure!

26 Some unity? Theorem (Gleich and Kloster, 2013 arxiv: )" Consider solving personalized PageRank using the Gauss- Southwell relaxation method in a graph with a Zipf-law in the degrees with exponent p=1 and max-degree d, then the work involved in getting a solution with 1-norm error is work = O (1/") 1 1 d(log d) 2 Improve this? * (the paper currently bounds exp(a D -1 ) e i but analysis yields this bound for PageRank) ** (this bound is not very useful, but it justifies that this method isn t horrible in theory)

27 There is more structure The one ring. (See C. Seshadhri s talk for the reference)

28 Further open directions Nice to solve Unifying convergence results for Monte Carlo and relaxation on large networks to have provably efficient, practical algs. Use triangles? Use preconditioning? A curiosity Is there any reason to use a Krylov method? Staple of matrix computations, A! AV k+1 = V k H k with H k small BIG gap Can we get algorithms with top k or ordering convergence? See Bahmani et al. 2010; Sarkar et al (Proximity Search) Important? Are the useful, tractable multi-linear problems on a network? e.g. triangles for network alignment; e.g. Kannan s planted clique problem. Supported by NSF CAREER CCF