CS 5614: (Big) Data Management Systems. B. Aditya Prakash Lecture #21: Graph Mining 2

Transcription

1 CS 5614: (Big) Data Management Systems B. Aditya Prakash Lecture #21: Graph Mining 2

2 Networks & Communi>es We think of networks being organized into modules, cluster, communi>es: VT CS

3 Goal: Find Densely Linked Clusters VT CS

4 Micro-Markets in Sponsored Search Find micro-markets by par>>oning the query-toadver>ser graph: query advertiser [Andersen, Lang: Communities from seed sets, 2006] VT CS

5 Movies and Actors Clusters in Movies-to-Actors graph: [Andersen, Lang: Communities from seed sets, 2006] VT CS

6 TwiRer & Facebook Discovering social circles, circles of trust: [McAuley, Leskovec: Discovering social circles in ego networks, 2012] VT CS

7 We will work with undirected (unweighted) networks How to find communi>es? COMMUNITY DETECTION VT CS

8 Method 1: Strength of Weak Ties Edge betweenness: Number of shortest paths passing over the edge Intui>on: b=16 b=7.5 Edge strengths (call volume) in a real network Edge betweenness in a real network VT CS

9 [Girvan-Newman 02] Method 1: Girvan-Newman Divisive hierarchical clustering based on the nohon of edge betweenness: Number of shortest paths passing through the edge Girvan-Newman Algorithm: Undirected unweighted networks Repeat un>l no edges are Calculate betweenness of edges Remove edges with highest betweenness Connected components are communihes Gives a hierarchical decomposihon of the network VT CS

10 Girvan-Newman: Example Need to re-compute betweenness at every step VT CS

11 Girvan-Newman: Example Step 1: Step 2: Step 3: Hierarchical network decomposition: VT CS

12 Girvan-Newman: Results CommuniHes in physics collaborahons VT CS

13 Girvan-Newman: Results Zachary s Karate club: Hierarchical decomposihon VT CS

14 WE NEED TO RESOLVE 2 QUESTIONS 1. How to compute betweenness? 2. How to select the number of clusters? VT CS

15 How to Compute Betweenness? Want to compute betweenness of paths star>ng at node A Breath first search star>ng from A: VT CS

16 How to Compute Betweenness? Count the number of shortest paths from A to all other nodes of the network: VT CS

17 How to Compute Betweenness? Compute betweenness by working up the tree: If there are mulhple paths count them frachonally The algorithm: Add edge flows: -- node flow = 1+ child edges -- split the flow up based on the parent value Repeat the BFS procedure for each starting node U 1 path to K. Split evenly paths to J Split 1:2 1+1 paths to H Split evenly VT CS

18 How to Compute Betweenness? Compute betweenness by working up the tree: If there are mulhple paths count them frachonally The algorithm: Add edge flows: -- node flow = 1+ child edges -- split the flow up based on the parent value Repeat the BFS procedure for each starting node U 1 path to K. Split evenly paths to J Split 1:2 1+1 paths to H Split evenly VT CS

19 WE NEED TO RESOLVE 2 QUESTIONS 1. How to compute betweenness? 2. How to select the number of clusters? VT CS

20 Network Communi>es Communi>es: sets of >ghtly connected nodes Define: Modularity Q A measure of how well a network is parhhoned into communihes Given a parhhoning of the network into groups s S: Q s S [ (# edges within group s) (expected # edges within group s) ] VT CS 5614 Need a null model! 20

21 Null Model: Configura>on Model VT CS

22 Modularity VT CS

23 Modularity: Number of clusters Modularity is useful for selec>ng the number of clusters: Q VT CS

24 SPECTRAL CLUSTERING VT CS

25 Graph Par>>oning Undirected graph Bi-par>>oning task: Divide verhces into two disjoint groups A Ques>ons: How can we define a good parhhon of? How can we efficiently idenhfy such a parhhon? B VT CS

26 Graph Par>>oning What makes a good par>>on? Maximize the number of within-group connechons Minimize the number of between-group connechons A B VT CS

27 Graph Cuts VT CS

28 Graph Cut Criterion Criterion: Minimum-cut Minimize weight of connechons between groups arg min A,B cut(a,b) Degenerate case: Optimal cut Minimum cut Problem: Only considers external cluster connechons Does not consider internal cluster connechvity VT CS

29 [Shi-Malik] Graph Cut Criteria Criterion: Normalized-cut [Shi-Malik, 97] ConnecHvity between groups relahve to the density of each group cut( A, B) ncut ( A, B) = + vol( A) cut( A, B) vol( B) : total weight of the edges with at least one endpoint in : n Why use this criterion? n Produces more balanced parhhons How do we efficiently find a good par>>on? Problem: CompuHng ophmal cut is NP-hard VT CS

30 Spectral Graph Par>>oning A: adjacency matrix of undirected G A ij =1 if is an edge, else 0 x is a vector in R n with components Think of it as a label/value of each node of What is the meaning of A x? Entry y i is a sum of labels x j of neighbors of i VT CS = n n nn n n y y x x a a a a!!!! = = = E j i j n j ij i x xj A y ), ( 1

31 What is the meaning of Ax? j th coordinate of A x : Sum of the x-values of neighbors of j Make this a new value at node j Spectral Graph Theory: a! a Analyze the spectrum of matrix represenhng x! x Spectrum: Eigenvectors of a graph, ordered by the magnitude (strength) of their corresponding eigenvalues : Λ = 11 n1 a1 n 1 1 { 1 2 n VT CS a! λ, λ,..., λ } λ λ λ n nn A x=λ x n = x λ! x n

32 Example: d-regular graph VT CS

33 d is the largest eigenvalue of A Details! VT CS

34 Example: Graph on 2 components VT CS

35 More Intui>on VT CS

36 Matrix Representa>ons Adjacency matrix (A): n n matrix A=[a ij ], a ij =1 if edge between node i and j Important proper>es: Symmetric matrix Eigenvectors are real and orthogonal VT CS

37 Matrix Representa>ons Degree matrix (D): n n diagonal matrix D=[d ii ], d ii = degree of node i VT CS

38 Matrix Representa>ons Laplacian matrix (L): n n symmetric matrix What is trivial eigenpair? then and so Important proper>es: Eigenvalues are non-negahve real numbers Eigenvectors are real and orthogonal L = D A VT CS

39 Details! Facts about the Laplacian L VT CS

40 λ 2 as op>miza>on problem λ 2 = min x x T x M x T x VT CS

41 Proof VT CS

42 λ 2 as op>miza>on problem VT CS

43 Find Op>mal Cut [Fiedler 73] VT CS

44 Rayleigh Theorem VT CS

45 Details! Approx. Guarantee of Spectral VT CS

46 Details! Approx. Guarantee of Spectral VT CS

47 Details! Approx. Guarantee of Spectral VT CS

48 So far How to define a good par>>on of a graph? Minimize a given graph cut criterion How to efficiently iden>fy such a par>>on? Approximate using informahon provided by the eigenvalues and eigenvectors of a graph Spectral Clustering VT CS

49 Spectral Clustering Algorithms Three basic stages: 1) Pre-processing Construct a matrix representahon of the graph 2) Decomposi>on Compute eigenvalues and eigenvectors of the matrix Map each point to a lower-dimensional representahon based on one or more eigenvectors 3) Grouping Assign points to two or more clusters, based on the new representahon VT CS

50 Spectral Par>>oning Algorithm 1) Pre-processing: Build Laplacian matrix L of the graph ) Decomposi>on: Find eigenvalues λ and eigenvectors x of the matrix L λ= X = Map verhces to How do we now corresponding find the clusters? components of λ VT CS

51 Spectral Par>>oning 3) Grouping: Sort components of reduced 1-dimensional vector IdenHfy clusters by splihng the sorted vector in two How to choose a splisng point? Naïve approaches: Split at 0 or median value More expensive approaches: Ajempt to minimize normalized cut in 1-dimension (sweep over ordering of nodes induced by the eigenvector) Split at 0: Cluster A: PosiHve points Cluster B: NegaHve points A B VT CS

52 Example: Spectral Par>>oning Value of x 2 Rank in x 2 VT CS

53 Example: Spectral Par>>oning Components of x 2 Value of x 2 Rank in x 2 VT CS

54 Example: Spectral par>>oning Components of x 1 Components of x 3 VT CS

55 k-way Spectral Clustering How do we par>>on a graph into k clusters? Two basic approaches: Recursive bi-par>>oning [Hagen et al., 92] Recursively apply bi-parhhoning algorithm in a hierarchical divisive manner Disadvantages: Inefficient, unstable Cluster mul>ple eigenvectors [Shi-Malik, 00] Build a reduced space from mulhple eigenvectors Commonly used in recent papers A preferable approach VT CS

56 Why use mul>ple eigenvectors? Approximates the op>mal cut [Shi-Malik, 00] Can be used to approximate ophmal k-way normalized cut Emphasizes cohesive clusters Increases the unevenness in the distribuhon of the data AssociaHons between similar points are amplified, associahons between dissimilar points are ajenuated The data begins to approximate a clustering Well-separated space Transforms data to a new embedded space, consishng of k orthogonal basis vectors MulHple eigenvectors prevent instability due to informahon loss VT CS

57 ANALYSIS OF LARGE GRAPHS: TRAWLING VT CS

58 [Kumar et al. 99] Trawling Searching for small communi>es in the Web graph What is the signature of a community / discussion in a Web graph? Use this to define topics : What the same people on the left talk about on the right Remember HITS! Dense 2-layer graph Intuition: Many people all talking about the same things VT CS

59 Searching for Small Communi>es A more well-defined problem: Enumerate complete biparhte subgraphs K s,t Where K s,t : s nodes on the lem where each links to the same t other nodes on the right X Fully connected K 3,4 Y X = s = 3 Y = t = 4 VT CS

60 [Agrawal-Srikant 99] Frequent Itemset Enumera>on Market basket analysis. Sehng: Market: Universe U of n items Baskets: m subsets of U: S 1, S 2,, S m U (S i is a set of items one person bought) Support: Frequency threshold f Goal: Find all subsets T s.t. T S i of at least f sets S i (items in T were bought together at least f Hmes) What s the connec>on between the itemsets and complete bipar>te graphs? VT CS

61 [Kumar et al. 99] From Itemsets to Bipar>te K s,t Frequent itemsets = complete bipar>te graphs! How? View each node i as a set S i of nodes i points to K s,t = a set Y of size t that occurs in s sets S i S i ={a,b,c,d} Looking for K s,t à set of frequency threshold to s and look at layer t all frequent sets of size t X Y s minimum support ( X =s) t itemset size ( Y =t) VT CS

62 [Kumar et al. 99] From Itemsets to Bipar>te K s,t View each node i as a set S i of nodes i points to Say we find a frequent itemset Y={a,b,c} of supp s So, there are s nodes that link to all of {a,b,c}: S i ={a,b,c,d} Find frequent itemsets: s minimum support t itemset size We found K s,t! K s,t = a set Y of size t that occurs in s sets S i X VT CS Y

63 a c e f Itemsets: a = {b,c,d} b = {d} c = {b,d,e,f} d = {e,f} e = {b,d} f = {} d b Example (1) Support threshold s=2 {b,d}: support 3 {e,f}: support 2 And we just found 2 bipar>te subgraphs: a c e d b VT CS c e f d

64 Example (2) Example of a community from a web graph Nodes on the right Nodes on the left [Kumar, Raghavan, Rajagopalan, Tomkins: Trawling the Web for emerging cyber-communities 1999] VT CS