Networks and Algebraic Statistics

Transcription

1 Networks and Algebraic Statistics Dane Wilburne Illinois Institute of Technology UC Davis CACAO Seminar Davis, CA October 4th, 2016 (IIT) Networks and Alg. Stat. Oct / 23

2 Outline 1 The basics 2 Exponential random graph models 3 Parameter estimation for networks 4 Goodness-of-fit testing for networks 5 Stochastic block models dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

3 The basics Networks: Beyond i.i.d. Data Network data are everywhere in science! Social networks, supply chain networks, biological networks, chemical reaction networks, etc. From size n = 18 to size n = 10 6 They capture dependence beyond the classical i.i.d. setup We want to actually do statistics, not just summarize In other words, we want to do principled, model-based inference dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

4 The basics Example: The Network Science Co-authorship Dataset (IIT) Networks and Alg. Stat. Oct / 23

5 The basics Mathematical setup Network g: represented by a set of n nodes and e edges. Simple graphs: No self loops or multiple edges. We can represent graphs as 0-1 tables or vectors (e.g. adjacency matrices) Sample Space G n : set of all (labeled) simple graphs on n nodes. G n = 2 (n 2) Network Model P = {P θ }: A (parametric) family of probability distributions supported on G n Analogy: The normal (Gaussian) distribution is a family of (continuous) probability distributions on R indexed by the parameters µ and σ 2 dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

6 The basics What would be like to do? Network data are a new frontier in statistics - they pose many challenges: 1 Parameter estimation: Given an observed network, produce an optimal estimate of the parameters of the model (e.g. use x to estimate µ) 2 Goodness-of-fit testing: Decide whether the model fits the data, i.e. does the model provide a reasonable explanation of the observed data (e.g. eye color college major)? 3 Many other problems: model selection, scaling inference to large networks, noisy data, privacy issues, etc. Challenges: Standard statistical theory fails: Sample size is usually 1, but # of parameters can increase with n, so usual asymptotics do not hold. Many approaches to parameter estimation, goodness-of-fit testing, model selection, etc. are heuristic or ad hoc. dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23 =

7 The basics What would be like to do? Network data are a new frontier in statistics - they pose many challenges: 1 Parameter estimation: Given an observed network, produce an optimal estimate of the parameters of the model (e.g. use x to estimate µ) 2 Goodness-of-fit testing: Decide whether the model fits the data, i.e. does the model provide a reasonable explanation of the observed data (e.g. eye color college major)? 3 Many other problems: model selection, scaling inference to large networks, noisy data, privacy issues, etc. Today s goal: The goal of this talk is to explore the ways in which algebraic and geometric thinking can help help us address some of these challenges. = dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

8 The basics What would be like to do? Network data are a new frontier in statistics - they pose many challenges: 1 Parameter estimation: Given an observed network, produce an optimal estimate of the parameters of the model (e.g. use x to estimate µ) 2 Goodness-of-fit testing: Decide whether the model fits the data, i.e. does the model provide a reasonable explanation of the observed data (e.g. eye color college major)? 3 Many other problems: model selection, scaling inference to large networks, noisy data, privacy issues, etc. Ideas coming from: Commutative algebra Graph theory Combinatorics = Polyhedral geometry Algebraic geometry dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

9 Exponential random graph models Exponential random graph models (ERGMs) Key Idea of an ERGM Specify probability distributions over G n via network summary statistics. Sufficient statistic: t : G n R k, g (t 1 (g),..., t k (g)) t( ) records a quantity that is expressive of the network property of interest Simplest/most important case: t( ) is a linear function of the network (log-linear models) Natural parameters: θ = {θ 1,..., θ k } Θ R k Definition (Linear Exponential Family) P θ (g) = exp{ θ, t(g) ψ(θ)} ψ(θ) = log exp{ θ, t(g) } g G n dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

10 Exponential random graph models Exponential random graph models (ERGMs) Key Idea of an ERGM Specify probability distributions over G n via network summary statistics. Sufficient statistic: t : G n R k, g (t 1 (g),..., t k (g)) t( ) records a quantity that is expressive of the network property of interest Simplest/most important case: t( ) is a linear function of the network (log-linear models) Natural parameters: θ = {θ 1,..., θ k } Θ R k Takeaways: The probability of observing a network depends only on the value of its sufficient statistics Think of this roughly as dimension reduction in a way that captures the essential features of the nework dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

11 Exponential random graph models So many ERGMs! Erdős-Renyi-Gilbert model Total number of triangles k-stars Deg. seq. Directeddeg. seq. Triangle+star β model [RPF 2011; Chatterjee, Diaconis 11] +hypergraphs[sspfr] p 1 model [Holland, Leinhardt 81] Markov graph model [Frank, Strauss 86] degen +edge Degeneracy + number of edges [Kim et al. 16] Shell disrib. k-core ERGM [Karwa, Pelsmajer, Petrović, Stasi, Wilburne 15+] ERGMs are flexible: pick (almost) any t( ) you want! Each ERGM gives us a tool for unique modeling; we need lots of tools Each ERGM presents its own unique set of challenges dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

12 Exponential random graph models Some easy examples Erdős-Renyi random graphs Independently for each pair {i, j}, include the edge e ij with probability p. Sufficient statistic:??? P p (g):??? Exponential family form:??? dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

13 Exponential random graph models Some easy examples Erdős-Renyi random graphs Independently for each pair {i, j}, include the edge e ij with probability p. Sufficient statistic: E(g) P p (g):??? Exponential family form:??? dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

14 Exponential random graph models Some easy examples Erdős-Renyi random graphs Independently for each pair {i, j}, include the edge e ij with probability p. Sufficient statistic: E(g) P p (g): p E(g) (1 p) (n 2) E(g) Exponential family form:??? dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

15 Exponential random graph models Some easy examples Erdős-Renyi random graphs Independently for each pair {i, j}, include the edge e ij with probability p. Sufficient statistic: P p (g): Exponential family form: E(g) p E(g) (1 p) (n 2) E(g) P θ (g) e θ E(g), where p = eθ 1 + e θ dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

16 Exponential random graph models Some easy examples The β model Associated to every vertex i is a parameter β i that measures its friendliness or propensity to attract edges. Each edge {i, j} appears independently with probability proportional to β i β j, so that: P β (g) {i,j} E(g) β i β j = n i=1 β deg(i) i. Sufficient statistic: The degree sequence of g, Exponential family form: d(g) = (d 1, d 2,..., d n ). P β (g) = e n i=1 d iβ i ψ(β) dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

17 Parameter estimation for networks Parameter estimation The parameter estimation problem for networks Given an observed network, estimate the unknown probability distribution from the model that best explains the data. The most common approach to the parameter estimation problem is maximum likelihood estimation: θ MLE := argmax θ ΘP θ (g). (IIT) Networks and Alg. Stat. Oct / 23

18 Parameter estimation for networks Parameter estimation The parameter estimation problem for networks Given an observed network, estimate the unknown probability distribution from the model that best explains the data. The most common approach to the parameter estimation problem is maximum likelihood estimation: θ MLE := argmax θ ΘP θ (g). Problem (MLE existence) Does the maximum likelihood estimate θ MLE exist? (IIT) Networks and Alg. Stat. Oct / 23

19 Parameter estimation for networks Solving MLE existence problem The polyhedral geometry of the model answers the MLE existence question for us. Definition (The model polytope) For a network model M on G n, denote the set of all possible sufficient statistics of the model by T := {t(g) : g G n }. The model polytope P M of M is P M := conv(t ) R k. Theorem (MLE existence) Let g be an observed network. Then, θ MLE exists (and is unique) if and only if t(g) rel int P M. dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

20 Parameter estimation for networks Back to our examples Some model polytopes: Erdős-Renyi: P ER = [0, ( n 2) ] R ( n 2) The β model: P β = conv{d = (d 1,..., d n ) : d is realizable degree sequence} R n Realizable degree sequences characterized by Havel-Hakimi P β well-studied in graph theory literature (vertex description, hyperplane description, characterization of extreme points, etc.) When n = 3, P β = conv{(0, 0, 0), (1, 1, 0), (1, 0, 1), (0, 1, 1), (2, 1, 1), (1, 2, 1), (1, 1, 2), (2, 2, 2)} dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

21 Parameter estimation for networks What else does the model polytope tell us? The facial structure of P M can be used to discern precisely which parameters are estimable P M contains information about the statistical degeneracy of the model Degenerate distributions place a lot of weight on extreme graphs Related problems Which integer points in P M are realizable as sufficient statistics of the model? What are the asymptotic properties of P M? dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

22 Goodness-of-fit testing for networks Does the model fit? Problem (Goodness-of-fit problem for networks) Can P θ be considered as a satisfactory generative model for the observed network g, i.e. does P θ describe the data sufficiently well? A solution to the goodness-of-fit problem for networks is difficult, since: Standard approaches rely on the asymptotic properties of large samples We only get samples of size 1 in network applications Must develop exact tests Each network model P M poses a unique set of challenges dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

23 Goodness-of-fit testing for networks GoF testing for networks Key idea: To test the fit of the model, we compare the observed network g to the other networks in the fiber Performing a GoF test F t(g) := {h G n : t(h) = t(g)}. Choose a test statistic that measures the distance of g from the expected value under the model For contingency tables, one might use expected cell counts Compare this distance to the distance of h from the expected value under the model for all h F t(g) Is g closer or further away to the expected value than most h F t(g)? If g is not close, we reject the model (i.e. we say it is not a good fit for the data) dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

24 Goodness-of-fit testing for networks GoF testing for networks A big problem For any interesting example, F t(g) is far too large to enumerate. We therefore must resort to Markov Chain Monte Carlo methods: Generate a random sample of graphs from F t(g) in a representative way This approach necessitates a technique for moving from graph to graph while staying in F t(g) Must be able to get from any g F t(g) to any h F t(g) without leaving F t(g) along the way, i.e. the moves must connect the fiber A set of moves that connects every fiber of M is called a Markov basis for M Theorem (Diaconis-Sturmfels) For a log-linear model M, a generating set for the toric ideal I M is a Markov basis for M. dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

25 Goodness-of-fit testing for networks The toric ideal I M The A matrix Associated to every log-linear model M is a matrix A M that reads the sufficient statistic off the data, i.e. A M defines a map: A M : G n {suff stats} g t(g). The toric ideal I M is the toric ideal of A. Erdős-Renyi: (1, 1,..., 1)e = E(g), where e {0, 1} (n 2) is the edge-incidence vector of g. The β model: A M = vertex-edge incidence matrix of K n, so A M e = (d 1,..., d n ). dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

26 Goodness-of-fit testing for networks Easy Markov basis examples Erdős-Renyi: I ER = {x ei x ej : 1 i, j ( n 2) } The Markov basis for the Erdős-Renyi model consists of moves of the form add an edge, remove an edge The β model: I β = {x ei x ej x ek x el : 1 i, j, k, l ( n 2) } The Markov basis for consists of all possible two switches A reformulation of the classical theorem in graph theory that two switches connect the space of graphs with a fixed degree sequence dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

27 Stochastic block models Stochastic block models Stochastic block models (SBMs) were introduced and first studied by Holland, Laskey, and Leinhardt in SBMs are useful for community-based network modeling Example: Nodes: UC Davis undergrads Edges: Friendships Communities: Majors Maybe there is reason to believe that Davis undergrads are more likely to be friends with students in their major than with students in other majors. Can we detect/quantify/test this effect by looking at the network? SBM model is suited for this type of application. SMBs are a simple generalization of the famous Erdős-Renyi model for random graphs dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

28 Stochastic block models Erdős-Renyi SBM We view a network as a random graph on n nodes, each of which lies in exactly one of k communities (or blocks) B 1,..., B k. Block assignment function: c : [n] [k] c(i) = l i B l (i.e. node i lies in block B l ) Block sizes: n l = B l k l=1 n l = n Definition: In the Erdős-Renyi-SBM with k (known) blocks, the probability of observing the graph g G n is: P (g) = ij p eij c(i)c(j) (1 peij c(i)c(j) )1 e ij, where e ij is 1 or 0 according to whether the edge ij is present in g. dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

29 Stochastic block models Erdős-Renyi SBM We view a network as a random graph on n nodes, each of which lies in exactly one of k communities (or blocks) B 1,..., B k. Block assignment function: c : [n] [k] c(i) = l i B l (i.e. node i lies in block B l ) Block sizes: n l = B l k l=1 n l = n Definition: We can also specify the ER-SBM in a more compact way by specifying the log-odds: ( ) pij log = α 1 p c(i)c(j) ij dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

30 Stochastic block models Erdős-Renyi SBM We view a network as a random graph on n nodes, each of which lies in exactly one of k communities (or blocks) B 1,..., B k. Block assignment function: c : [n] [k] c(i) = l i B l (i.e. node i lies in block B l ) Block sizes: n l = B l k l=1 n l = n Definition: In words: the probability of edge ij occurring depends only on the blocks c(i) and c(j) in which i and j lie (and in particular it is independent of the other dyads in g). dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

31 Stochastic block models Erdős-Renyi SBM We view a network as a random graph on n nodes, each of which lies in exactly one of k communities (or blocks) B 1,..., B k. Block assignment function: c : [n] [k] c(i) = l i B l (i.e. node i lies in block B l ) Block sizes: n l = B l k l=1 n l = n Note: The most statistically interesting case is when the block assignments are unknown (community detection). This is the subject of a forthcoming paper [Alexeev-Pati-Karwa-Petrović-Raic-Solus.-W.-Williams-Yan, 16+]. dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

32 Stochastic block models Algebraic statistics of the ER-SBM Goal: Given a network, estimate the k k matrix of edge probabilities Sufficient statistics: k k matrix of inter- and intra-block edge counts. Markov basis: Essentially the same as Erdős-Renyi model (add/remove an inter-block edge, add/remove an intra-block edge) Model polytope: Parallelepiped in R (k+1 2 ) with side lengths Bi B j and ( B i ) 2 Latent block model Latent (unknown) block model is a mixture model and thus a secant of toric varities dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23

33 Stochastic block models Additive SBM Idea: Blocks are node degrees Sufficient statistics: k-vector of the degree sum of each block Markov basis: ER-SBM Markov basis + 2-switches of all types Theorem (Add. SBM model polytope, A.-P.-K.-P.-R.-S.-W.-W.-Y., 16+) For the model with blocks B 1,..., B k with n i nodes in block B i, for each K S {B 1,..., B k } with either S = 1 or such that K is a non-empty proper subset of S, the facet supporting hyperplanes of M AddSBM are given by: dwilburne@hawk.iit.edu (IIT) Networks and Alg. Stat. Oct / 23