Faster Random Walks By Rewiring Online Social Networks On-The-Fly

Transcription

1 1 Faster Random Walks By Rewiring Online ocial Networks On-The-Fly Zhojie Zho 1, Nan Zhang 2, Zhigo Gong 3, Gatam Das 4 1,2 Compter cience Department, George Washington Uniersity 1 rexzho@gw.ed 2 nzhang10@gw.ed 3 Compter and Information cience Department, Uniersity of Maca 2 fstzgg@mac.mo 4 Compter cience Department, Uniersity of Texas at Arlington 4 gdas@ta.ed Abstract Many online social networks featre restrictie web interfaces which only allow the qery of a ser s local neighborhood throgh the interface. To enable analytics oer sch an online social network throgh its restrictie web interface, many recent efforts rese the existing Marko Chain Monte Carlo methods sch as random walks to sample the social network and spport analytics based on the samples. The problem with sch an approach, howeer, is the large amont of qeries often reqired (i.e., a long mixing time ) for a random walk to reach a desired (stationary) sampling distribtion. In this paper, we consider a noel problem of enabling a faster random walk oer online social networks by rewiring the social network on-the-fly. pecifically, we deelop Modified TOpology (MTO)-ampler which, by sing only information exposed by the restrictie web interface, constrcts a irtal oerlay topology of the social network while performing a random walk, and ensres that the random walk follows the modified oerlay topology rather than the original one. We show that MTO- ampler not only proably enhances the efficiency of sampling, bt also achiees significant saings on qery cost oer real-world online social networks sch as Google Pls, Epinion etc. I. INTRODUCTION A. Aggregate Estimation oer Online ocial Networks An online social network allows its sers to pblish contents and form connections with other sers. To retriee information from a social network, one generally needs to isse a indiidal-ser qery throgh the social network s web interface by specifying a ser of interest, and the web interface retrns the contents pblished by the ser as well as a list of other sers connected with the ser 1. An online social network not only proides a platform for sers to share information with their acqaintance, bt also enables a third party to perform a wide ariety of analytical applications oer the social network - e.g., the analysis of rmor/news propagation, the mining of sentiment/opinion on certain sbjects, and social media based market research. While some third parties, e.g., adertisers, may be able to negotiate contracts with the network owners to get access to the fll nderlying database, many third parties lack the 1 We crrently focs on the ndirected relationship between sers. resorces to do so. To enable these third-party analytical applications, one mst be able to accrately estimate bigpictre aggregates (e.g., the aerage age of sers, the COUNT of ser posts that contain a gien word) oer an online social network by issing a small nmber of indiidal-ser qeries throgh the social network s web interface. We address this problem of third-party aggregate estimation in the paper. B. Existing ampling Based oltions and Their Problems An important challenge facing third-party aggregate estimation is the lack of cooperation from online social network proiders. In particlar, the information retrned by each indiidal-ser qery is extremely limited - only containing information abot the neighborhood of one ser. Frthermore, almost all large-scale online social networks enforce limits on the nmber of web reqests one can isse (e.g., 600 open graph qeries per 600 seconds for Facebook 2, and 350 reqests per hor for Twitter 3 ). As a reslt, it is practically impossible to crawl/download most or all data from an online social network before generating aggregate estimations. There is also no aailable way for a third party to obtain the entire topology of the graph nderlying the social network. To address this challenge, a nmber of sampling techniqes hae been proposed for performing analytics oer an online social network withot the prereqisite of crawling [9], [11], [12], [15]. The objectie of sampling is to randomly select elements (e.g., nodes/sers or edges/relationships) from the online social network according to a pre-determined probability distribtion, and then to generate aggregate estimations based on the retrieed samples. ince only indiidal local neighborhoods (i.e., a ser and the set of its neighbors) - rather than the entire graph topology - can be retrieed from the social network s web interface, to the best of or knowledge, all existing sampling techniqes withot prior knowledge of all nodes/edges are bilt pon the idea of performing random walks oer the graph which only reqire knowledge of the local neighborhoods isited by the random walks

2 2 In literatre, there are two poplar random walk schemes: simple random walk and Metropolis Hastings random walk. imple random walk (RW) [17] starts from an arbitrary ser, repeatedly hops from one ser to another by choosing niformly at random from the former ser s neighborhood, and stops after a nmber of steps to retriee the last ser as a sample. When the simple random walk is sfficiently long, the probability for each ser to be sampled tends to reach a stationary (probability) distribtion proportional to each ser s degree (i.e., the nmber of sers connected with the ser). Ths, based on the retrieed samples and knowledge of sch a stationary distribtion, one can generate nbiased estimations of AVG aggregates (with or withot selection conditions) oer all sers in the social network. If the total nmber of sers in the social network is aailable 4, then COUNT and UM aggregates can be answered withot bias as well. Metropolis Hastings random walk (MHRW) is a random walk achieing any distribtion (typically niform distribtion) constrcted by the famos MH algorithm. As an extension of MHRW, based on the knowledge of all the ids of a graph, [11] sggests that we can condct random jmp (RJ), which jmps to any random ertex 5 in the graph with a fixed probability in each step when it carries on the MHRW. Althogh MHRW can yield asymptotically niform samples, which reqires no additional processing for sbseqent analysis, it is slower than RW almost for all practical measrements of conergence, sch as degree distribtion distance, K distance and mean degree error. According to [10] and [14], RW is times faster than MHRW. Ths we set the baseline as RW, while we also inclde MHRW in the experimental section. A critical problem of existing sampling techniqes, howeer, is the large nmber of indiidal-ser qeries (i.e., web reqests) they reqire for retrieing each sample. Consider the aboe-described simple random walk as an example. In order to reach the stationary distribtion (and thereby an accrate aggregate estimation), one may hae to isse a large nmber of qeries as a brn-in period of the random walk. Traditional stdies on graph theory fond that the length of sch a brn-in period is determined by the graph condctance - an intrinsic property of the graph topology (formally defined in ection II). In particlar, the smaller the condctance is, the longer the brn-in period will be (i.e., the more indiidal-ser qeries will be reqired by sampling). Unfortnately, a recent stdy [18] on real-world social networks sch as Facebook, Liejornal, etc. fond the condctance of their graphs to be sbstantially lower than expected. As a reslt, a random walk on these social networks often reqires a large nmber of indiidal-ser qeries - e.g., approximately 500 to 1500 single random walk length for a real-world social network Liejornal of one million nodes to achiee acceptable ariance distance [18]. One can see that, in order to retriee enogh samples to reach an accrate aggregate estimation, the existing sampling techniqes may reqire a ery large nmber of indiidal-ser qeries. 4 Which is the case for many real-world social networks whose proiders pblish the total nmber of sers for adertising prposes. 5 It may need the global topology or the whole ser id space for generate random ertex, ths not iable for all online social networks. C. Otline of Technical Reslts In this paper, we consider a noel problem of how to significantly increase the condctance of a social network graph by modifying the graph topology on-the-fly (dring the thirdparty random walk process). In the following, we shall first explain what we mean by on-the-fly topology modification, and then describe the rationale behind or main ideas for topology modification. First, by topology modification we do not actally modify the original topology of the social network graph - indeed, no third party other than the social network proider has the ability to do so. What we modify is the topology of an oerlay graph on which we perform the random walks. Fig 1 depicts an example: if we can decide that not considering a particlar edge in the random walk process can make the brn-in period shorter (i.e., increase the condctance), then we are essentially performing random walks oer an oerlay graph on which this edge is remoed. By doing so, we can achiee same accrate aggregate estimation with lower qery cost. One can see that, with traditional random walk techniqes, the oerlay graph is exactly the same as the original social network graph. Or objectie here is to maniplate edges in the oerlay graph so as to maximize the graph condctance. It is important to note that the technical challenge here is not how edge maniplations can boost graph condctance - a simple method to reach theoretical maximm on condctance is to repeatedly insert edges to the graph ntil it becomes a complete graph. This reqires the knowledge of all nodes in the social network, which a third-party does not hae. The key challenge here is how to perform edge maniplations only based on the knowledge of local neighborhoods that a random walk has passed by, and yet increases the condctance of the entire graph in a significant manner. In the following, we proide an intitie explanation of or main ideas for topology modification. To nderstand the main ideas, we first introdce the concepts of cross-ctting and non-cross-ctting edges intitiely with an example in Fig 1 (we shall formally define these concepts in ection II). Generally speaking, if we consider a social network graph consisting of mltiple densely connected components (e.g., and in Fig 1), then the edges connecting them are likely to be cross-ctting edges, while edges inside each densely connected component are likely non-cross-ctting ones. A key intition here is that the more cross-ctting edges and/or the fewer non-cross-ctting edges a graph has, the higher its condctance is. For example, Graph G in Fig 1 has a low condctance (i.e., high brn-in period) as a random walk is likely to get stcked in one of the two dense components which are difficlt to escape, gien that there is only one cross-ctting edge (, ). On the other hand, with far fewer non-cross-ctting edges and a few additional crossctting edges, G has a mch higher condctance as it is mch easier now for a random walk to moe from one component to the other. With the concepts of cross-ctting and non-cross-ctting edges, we deelop Modify TOpology ampler (MTO- ampler), a topology maniplation techniqe which first de-

3 3 Oerlay graph G* Original graph G Fig. 1. graph. w * r r A concept of a random walk on the topologically modified oerlay termines 6 whether a gien edge in the graph is a cross-ctting edge based solely pon knowledge of the local neighborhood topology, and then remoes the edge if it is non-cross-ctting. MTO-ampler may also moe an edge by changing a node connected to the edge if it is determined that, by doing so, the new edge is more likely to be a cross-ctting edge. We shall show in the paper that MTO-ampler is capable of significantly improing the efficiency of random walks: For the example in Fig 1, MTO-ampler is capable of redcing the pper bond on mixing time (i.e., qery cost of a random walk) by 97%. We also demonstrate throgh experimental reslts the significant improement of efficiency achieed by MTO-ampler for real-world social networks sch as Epinions, Google Pls, etc. The main contribtions of or approach inclde: (Problem Noelty) We consider a noel problem of modifying the graph topology on-the-fly (dring the random walk process) for the efficient third-party sampling of online social networks. (oltion Noelty) We deelop MTO-ampler which determines whether an edge is (non-)cross-ctting based solely pon local neighborhood knowledge retrieed by the random walk, and then maniplates the graph topology to significantly improe sampling efficiency. Or contribtions also inclde extensie theoretical analysis (on arios social network models) and experimental ealation on synthetic and real-world social networks as well as online at Google+ which demonstrate the speriority of or MTO-ampler oer the traditional sampling techniqes. II. PRELIMINARIE A. Model of Online ocial Networks In this paper, we consider an online social network with an interface that allows inpt qeries of the form 6 Note that, as we shall proe in section III-A, it is impossible to assert deterministically that an edge is cross-ctting. Nonetheless, it is possible to assert deterministically that an edge is non-cross-ctting. Ths, or algorithm has two possible otpts: non-cross-ctting or ncertain. We shall show in the paper that it otpts non-cross-ctting for a large nmber of (non-crossctting) edges in real-world social networks. s s * q(): ELECT * FROM D WHERE UER-ID =, and responds with the information abot ser (e.g., ser name, self-description, ser-pblished contents) as well as the list of all other sers connected with (e.g., s friends in the network). This is a model followed by many online social networks - e.g., Google Pls, Facebook, etc - with the interface proided as either an end-ser-friendly web page or a deeloper-specific API call. Consider the social-network topology as an ndirected graph G(V, E), where each node in V is corresponding to a ser in the social network 7, and each edge in E represents the connection between two sers. One can see that the answer to qery q() ( V ) is a set of nodes N() V, sch that N(), there is an edge e : (, ) E. We henceforth refer to N() as the neighborhood of. We se k to denote the degree of - i.e., k = N(). For abbreiation, we also write e : (, ) as e. Rnning Example: We shall se, throghot this paper, the 22-node, 111-edge, barbell graph shown (as the original graph G) in Fig 1 as a rnning example. B. Performance Measres for ampling In the following, we shall discss two key objecties for sampling: (1) minimizing bias - sch that the retrieed samples can be sed to accrately estimate aggregate qery answers, and (2) redcing the nmber of qeries reqired for sampling - gien the stringent reqirement often pt in place by realworld social networks on the nmber of qeries one can isse per day. Bias: In general, sampling bias is the distance between the target (i.e., ideal) distribtion of samples and the actal sampling distribtion - i.e., the probability for each tple to be retrieed as a sample. We shall frther discss a concrete bias measre in the next sbsection and an experimental measre in ection V-A.3. Qery Cost: To this end, we consider the nmber of niqe qeries one has to isse for the sampling process, as any dplicate qery can be answered from local cache withot consming the qery limit enforced by the social network proider. C. Random Walk A random walk is a Marko Chain Monte Carlo (MCMC) method which takes sccessie random steps on the aboedescribed graph G according to a transition matrix P = (p ),, V, where p represents the probability for the random walk to transit from node to. The premise here is that, after performing a random walk for a sfficient nmber of steps, the probability distribtion for the walk to land on each node in G conerges to a stationary distribtion π which then becomes the sampling distribtion 8. There are 7 Note that withot introdcing ambigity, we se node and social network ser interchangeably in this paper. 8 That is, if we take the end node as a sample

4 4 many different types of random walks, corresponding to the following ineqality [3]: different designs of P and different stationary distribtions. In ) t this paper, we consider the simple random walk which has a (1 2Φ(G)) t 2 E (t) (1 Φ(G)2. (3) stationary distribtion of π() = k /(2 E ) for all V. min V k 2 Definition 1: (imple Random Walk). Gien a crrent One can see that the graph condctance Φ(G) ranges between node, a simple random walk chooses niformly at random 0 and 1 - and the larger Φ(G) is, the smaller the mixing a neighboring node N() and transit to in the next step time will be (for a fixed threshold ɛ). Also note from (3) - i.e., { the log scale relationship between Φ(G) and the mixing 1/k if N(), time. This indicates a small change on Φ(G) may lead to a P = (1) 0 otherwise. significant change of the mixing time. For example, increasing One can see that each step of a simple random walk reqires condctance from to will change the pper bond exactly one qery (i.e., q() to identify the neighborhood of mixing time from log(c/ɛ) to log(c/ɛ). of and select the next stop ). Ths, the performance of Rnning Example: The condctance of the barbell sampling - i.e., the tradeoff between bias and qery cost - graph in the rnning example is Φ(G) = 1/( ( ) ) = is determined by how fast the random walk conerges to the The corresponding (and niqe) and are stationary distribtion. Formally, we measre the conergence shown in Fig 1. Correspondingly, the mixing time to speed as the mixing time defined as follows. reach a relatie point-wise distance of (t) ɛ is Definition 2: (Mixing Time) Gien G : (V, E), after t bonded from aboe by log(22.2/ɛ). We shall steps of simple random walk, the relatie point-wise distance show throghot the paper how or on-the-fly topology between the crrent sampling distribtion and the stationary modification techniqes can significantly increase condctance and redce the mixing-time pper-bond for distribtion is { } P t (t) = max π() this rnning example. (2), V, N() π() where P t is the element of P t with indices and. The mixing time of the random walk is the minimm ale of t sch that (t) ɛ where ɛ is a pre-determined threshold on relatie point-wise distance. One can see from the definition that the relatie point-wise distance (t) measres the bias of the random walk after t steps. Mixing time, on the other hand, captres the qery cost reqired to redce the bias below a pre-determined threshold ɛ. In the following sbsection, we describe a key characteristics of the graph which determines the mixing time - the condctance of the graph. D. Condctance: An Efficiency Indicator Intitiely, the condctance Φ, which indicates how fast the simple random walk conerges to its stationary distribtion, measres how well-knit a graph is. pecifically, the condctance is determined by a ct of the graph G - i.e., a partition of V into two disjoint sbsets and - which minimizes the ratio between the probability for the random walk to moe from one partition to the other and the probability for the random walk to stay in the same partition. Formally, we hae the following definition. Definition 3: (Condctance). The condctance 9 of a graph G : (V, E) is {e, Φ(G) = min } V min { {e, V }, {e, V } }. The relationship between the graph condctance and the mixing time of a simple random walk is illstrated by the 9 Rigidly, the condctance is determined by both the graph topology and the transition matrix of the random walk. Here we tailor the definition to the simple random walk considered in this paper. E. Key for Condctance: Cross-Ctting Edges A key obseration from Definition 3 is that the graph condctance critically depends on the nmber of edges which cross-ct and - i.e., {e, }. The more sch cross-ctting edges there are, the higher the graph condctance is likely to be. On the other hand, since a noncross-ctting edge is only conted in the denominator, the more non-cross-ctting edges there are in the graph, the lower the condctance is likely to be. Formally, we define crossctting edges as follows. Definition 4: (Cross-ctting edges). For a gien graph G(V, E), an edge e is a cross-ctting edge if and only if there exists V sch that, where = V \, and ϕ() = {e, } min { {e, V }, {e, V } } takes the minimm ale among all possible V. We note that in large graphs sch as online social networks, it is reasonable to assme that the nmber of cross-ctting edges is relatiely small when compared to total nmber of edges in or. One can see that or objectie of on-the-fly topology modification is then to increase the nmber of cross-ctting edges and decrease the nmber of non-cross-ctting edges as mch as possible. We describe or main ideas for doing so in the next section. Rnning Example: For the barbell graph, adding any edge between the two hales of the graph prodces a new cross-ctting edge, and increases the graph condctance from Φ(G) = to i.e., the mixingtime pper-bond will be redced to / = a significant redction of 75%.

5 5 III. MAIN IDEA OF ON-THE-FLY TOPOLOGY MODIFICATION A. Technical Challenges: Negatie Reslts One can see from ection II-E that the key for increasing the condctance of a social network (and thereby redcing the qery cost of sampling) throgh topology modification is to determine whether an edge is a cross-ctting edge or not. Unfortnately, the deterministic identification of a crossctting edge is a hard problem (in the worst case) een if the entire graph topology is gien as prior knowledge, as shown in the following theorem. Theorem 1: The problem of determining whether an edge is cross-ctting or not is NP-hard. Proof: Consider the case of eqal transition probability for each edge. The problem of finding all cross-ctting edges is eqialent with finding the optimm ct of the graph according to the Cheeger constant - a problem proed to be NP-hard [6]. Gien the worst-case hardness reslt, we now consider the best-case scenario - i.e., is there any graph topology (which is not the worst-case inpt, of corse) for which it is possible to efficiently identify cross-ctting edges? It is easy to see that, if the entire graph topology is gien, then there certainly exist sch graphs - with the original graph in Fig 1 being an example - for which the cross-ctting edge(s) can be straightforwardly identified. Nonetheless, or interest lies on making sch identifications based solely pon local neighborhood knowledge - becase of the aforementioned restrictions of online social-network interfaces. The following theorem, nfortnately, shows that it is impossible for one to deterministically confirm the cross-ctting natre of an edge nless the entire graph topology has been crawled. Theorem 2: Gien the local neighborhood topology of ertices accessed by a third-party sampler, { 1,..., k } V in G(V, E) where k < V, for any gien edge e : ( i, j ), there mst exist a graph G (V, E ) sch that: (1) e : ( i, j ) is not a cross-ctting edge for G, and (2) G and G are indistingishable from the iew of the sampler - i.e., there exists { 1,..., k } V which hae the exactly same local neighborhood as { 1,..., k }. Proof: The constrction of G can be stated as follows: First, insert n extra ertices 1, 0..., n 0 and e extra edges into the graph, sch that e : ( i, j ) E, there is e 0 : (i 0, 0 j ) in the new graph. Note that at this moment, there is no edge between any i and j 0. Then, in the second step, identify from G a ertex w which has not been accessed by the sampler - i.e., w { 1,..., k } - and insert into the graph an edge e : (w, w 0 ). One can see that the only cross-ctting edge in the otpt graph G is (w, w 0 ) - i.e., e : ( i, j ) cannot be a cross-ctting edge for G. An intitie demonstration of the proof is shown in Fig 2. It is important to note from the theorem, howeer, that it still leaes two possible ways for one to increase the condctance of a social network based on only the local neighborhood knowledge: (1) While the theorem indicates that it is impossible to deterministically confirm the cross-ctting natre of an edge, it may still be possible to deterministically disproe an edge from being cross-ctting - i.e., we may proe that an edge is definitely non-cross-ctting based on jst local neighborhood knowledge, and therefore remoe it to increase the condctance deterministically. (2) It is still possible to conditionally or probabilistically ealate the likelihood of an edge being cross-ctting - e.g., we may determine that an edge absent from the original graph is more likely to be a cross-ctting edge (if added) than an existing edge, and thereby replace the existing edge with the new one to increase the condctance in a probabilistic fashion. We consider the remoal and replacement strategies, respectiely, in the next two sbsections. i G j w w 0 Fig. 2. By cloning graph G, we can always constrct graph G sch that simply adding an edge e : ( i, j ) may decrease the condctance. B. Deterministic Identification of Non-cross-ctting Edges To illstrate the main idea of or deterministic identification of non-cross-ctting edges (for remoals), we start with an example in Fig 3 to show why we can determine, based solely pon the local neighborhoods of and as shown in the graph, that e : (, ) (henceforth denoted by e ) in the Fig is not a cross-ctting edge. The intition behind this is fairly simple: When and share a large nmber of common neighbors (e.g., 5 in Fig 3) bt hae relatiely few other edges (e.g., 1 each in Fig 3), it is highly nlikely for the partition to ct throgh e rather than the other edges of and - e.g., (, 0 ) in Fig 3 - if it cts throgh any edges associated with and at all. The rigid (dis-)proof can be constrcted with contradiction. ppose e is a cross-ctting edge between two partitions of the graph, and. One can see that since and belong to different partitions, there mst be at least 6 cross-ctting edges in the sbgraph (Fig 3 (a) depicts an example). We now show in the following discssion that this is actally impossible becase one can always constrct another partition and (by dragging and into the same part) and redce the nmber of cross-ctting edges to at most 5. Note that this contradicts the definition of and being a configration which minimizes the nmber of cross-ctting edges. Ths, e cannot be a cross-ctting edge. To nderstand how the constrction of and works, consider Fig 3 (b) as an example. For the partition illstrated in Fig 3 (a), we can drag into to form the new configration, sch that the nmber of cross-ctting edges associated with and is now at most 5, as shown in Fig 3 (b). Note that the other edges not shown in the sbgraph (no matter cross-ctting or not) are not affected by the re-configration, G'

6 6 ' w w 0 0 ' (a) (b) (a) (b) ' Fig. 4. Replace the edge e with e w 0 0 (c) Fig. 3. A figre shows that the edge e cannot be the cross-ctting edge in theorem 3. Locally, (a) and (c) hae 6 cross-ctting edges, while (b) and (d) only hae 5 of them. becase all ertices associated with are already known in the local neighborhood of (shown in Fig 3). More generally, for the other possible settings of and (sch as Fig 3(c)), one can constrct the re-configration in analogy with the following general principle: First, find the more poplar partition (i.e., either or ) among the 5 common neighbors of and (e.g., in Fig 3 (a) or Fig 3 (c)). Then, drag one of and to ensre that both of them are in this more poplar partition nder the new configration. One can see that, since at most 2 common neighbors of and are in the less poplar partition, the nmber of crossctting edges nder the new configration is at most , where 2 2 is the nmber of cross-ctting edges associated with the 2 common neighbors in the less poplar partition (at most 2 for each), and 1 is the nmber of cross-ctting edge associated with the other (non-common) neighbor of the node being dragged (i.e., 0 in Fig 3 (a)). The following theorem depicts the general case for which we can remoe an edge on-the-fly to increase the graph condctance. Recall that N() and k represent the set of neighbors and the degree of a node, respectiely. Theorem 3: [Edge Remoal Criteria]: Gien G(V, E),, V, if e E and N() N() + 1 > max{k, k }, (4) then e is not a cross-ctting edge. Proof: Let n = N() N(), withot losing generality, assming,, then there mst be n cross-ctting edges in these n disjoint paths of length 2 between and. We denote n, n as the nmber of cross-ctting edges in these n paths connected with and, so n + n = n. One can see that if we try to drag from to, all the edges connected with wold be modified, e.g. flip the edges linked to, which means the old cross-ctting edges will be the new non-cross-ctting edges, and ice ersa. As the assmption from ineqality (4): n 2 (d) ' +1 > 1 2 max{k, k }, so either n + 1 > 1 2 k or n + 1 > 1 2 k holds. Withot losing generality, assming for ertex the ineqality holds, we change from set to, so the nmber of cross-ctting edges mst be strictly decreasing. ince we hae assmed that the nmber of edges in or is mch greater than the nmber of cross-ctting edges, so Φ(G) mst decrease according to the decrease of the nmber of ctting-edges, which leads to the contradiction of e is a cross-ctting edge. De to space limitations, please refer to the technical report [23] for the proofs of all theorems in the rest of the paper. Intitiely, theorem 3 gies s a cle that if two nodes hae enogh common neighbors, then we can deterministically say that the edge between them is non-cross-ctting. Moreoer, (4) is tight - i.e., if it does not hold, then we can always constrct a conter example where e is cross-ctting - as shown in the following theorem. Corollary 1: For all N(), N(), k, k which satisfy N() N() max{k, k }, (5) there always exists a graph G(V, E) in which e is crossctting. Rnning Example: With or on-the-fly edge remoals, any random walk is essentially following an oerlay topology G which can be constrcted by applying Theorem 3 to eery edge in the original graph G. For the bar-bell rnning example, the solid lines in Fig 1 depicts G. The condctance is now Φ(G ) = Compared with the original condctance of 0.018, the corresponding lower bond on mixing time is redced to / = of the original ale - a redction of 89%. C. Conditional Identification of Cross-ctting Edges We now describe or second idea of conditionally identifying cross-ctting edges. We start with an example in Fig 4 to show why we can replace an existing edge with a new one sch that (1) the new edge is more likely to be crossctting, and (2) the replacement is garanteed to not decrease the condctance. pecifically, consider the replacement of e by e w gien the neighborhoods of and. A key obseration here is that e and e w cannot be both cross-ctting edges. The reason is that otherwise we cold always drag into the same partition as and w to redce the nmber of cross-ctting edges by at least 1. Gien this key obseration, one can see

7 7 that the replacement of e by e w will only hae two possible otcomes: if e is a cross-ctting edge, then e w mst also be a cross-ctting edge becase, de to the obseration, e w cannot be a cross-ctting edge. Ths, the replacement leads to no change on the graph condctance. if e is not a cross-ctting edge, then replacing it with e w will either keep the same condctance, or increase the condctance if e w is cross-ctting. As sch, the replacement operation neer redces the condctance, and might increase it when e w is cross-ctting. More generally, we hae the following theorem. Theorem 4: Gien G(V, E), V, if k = 3,, w N(), then replacing edge e with e w will not decrease the condctance, while it also has positie possibility to increase the condctance. Next, we are going to proe that k = 3 is actally the only case when replacement is garanteed to not redce the condctance, as shown by the following corollary. Corollary 2: For V, if k 3, then there always exist a graph G(V, E),, w N(), sch that replacing e with e w will decrease the condctance or hae no effect. Rnning Example: With Theorem VIII, an example of the replacement operations one can perform oer the bar-bell rnning example in Fig 1 is to replace e r with e r, gien that (after edge remoals) has a degree of 3. Compared with the original condctance of Φ(G) = and the post-remoal condctance of Φ(G ) = 0.053, the condctance is now frther increased to Φ(G ) = The corresponding lower bond on mixing time is redced to 416.6/ = 0.25 of the post-remoal bond - a frther redction of 75% - and 416.6/ = of the original bond - an oerall redction of 97%. D. Extension If we know more abot the ser s neighbors, especially the common neighbors of the ser and the random walk s next candidate, we will deterministically identify more non-crossctting edges. When the random walk reaches the nodes we hae accessed before, we can se their degree information withot issing extra web reqests since we cold retriee data from or local database. Fig 5 (a) shows an example that with the extra degree knowledge of and s common neighbor w, e mst be a non-cross-ctting edge. As k w = 3, if we assme e is a cross-ctting edges, then there mst be 3 cross-ctting edges between and. Howeer, there exists another configration Fig 5 (b), which only has 2 cross-ctting edges. Ths, it contradicts the assmption that e is a cross-ctting edge. Noticed that if we do not know the degree of w, we cold not deterministically identify e since theorem 3 does not apply here. Fig. 5. VIII. w (a) w (b) A demo shows that e cannot be a cross-ctting edge in theorem Theorem 5: Gien G(V, E),, V, if e E and N() N() N (4 k w ) > max{k, k }, w N (6) we can assert that e is not a cross-ctting edge. Here we denote N = {w N() N() k w is known, 2 k w 3}. Intitiely, the edge between two nodes which hae many common neighbors has higher probability to be a non-crossctting edge. Also, it is easy for s to find these edges in online social networks. If a friend knows almost eery other friends of a person, then this edge may be considered as noncross-ctting edge according to theorem 3 and VIII. IV. ALGORITHM MTO-AMPLER A. Algorithm implementation Algorithm description. Or local modification does not need to create the oerlay graph before actally carrying ot the random walk. As the random walk proceeds, it will rewire the topology by remoing or replacing edges to make the local neighborhood identical to the corresponding part of its oerlay graph G. To explain how the on-the-fly modification works, we demonstrate an example in Fig 6. Fig 6(a) is an oerlay graph G that has been modified according to former theorems, in which edges A, C and D are remoed and edge B is replaced. Fig 6(b) shows one possible track of or Modified TOpology random walk. For example when the random walk sees a node, and k = 3 (it satisfies the condition of replacement), then it may replace an edge as we described in theorem VIII. The colored area contains all the nodes that the random walk isits. Algorithm 1 depicts the detailed procedre of MTO sampler, and the stopping rle (which indicates that the random walk shold stop and otpt samples) can be any conergence monitor sed in Marko Chain. Aggregate estimation and probability reision. We se Importance ampling to directly estimate the aggregate information throgh the samples from the random walk s stationary distribtion τ. The key challenge for MTO-ampler sing importance sampling is to estimate the stationary distribtion of MTO- ampler random walk τ. ince MTO-ampler modifies the topology, τ may not eqal to the stationary distribtion τ. Here we hae τ () = k 2 E. (7)

8 A B 8 A B C D A B C D (a) Modified oerlay graph G (b) Carry ot the random walk by modify the topology on-the-fly. It is identical to the random walk in oerlay graph G. Fig. 6. A demo shows how the MTO-ampler modifies the topology of the graph on-the-fly. Algorithm 1 MTO-ampler for imple Random Walk for i = 1 sample size do starting from ertex while!(topping rle) do while N() 1 do niformly pick a neighbor, and isse a qery if e is remoable then N() N() {} contine else if e is replaceable then /* Depends on the replacement strategy */ if Option (a) then else if Option (b) then N() N() { } choose or randomly break end if end if if rand(0, 1) < 1/2 then break else contine end if end while end while record sample x i end for Importance ampling: for i = 1 to ample ize N do x i sampling from τ w(x i ) ˆπ(xi) ˆτ(x i) record f(x i ) /*Aggregate Fnction f( ) */ end for Otpt estimation A(f(X)) = 1 N N i=1 f(xi)w(xi) 1 N N i=1 w(xi) ' ' Option (a) Option (b) Fig. 7. Two possible options of the replacement strategy. (a) Replace e with e, and k remains the same. (b) Replace e with e, and k k + 1 Howeer, k is nknown in oerlay graph G. To estimate k, when we try to access one of s neighbor, we can define a random ariable ξ as: If e cold be remoed, let ξ = 0. If e can be replaced to e, or e is jst a reglar edge, let ξ = k. ee Fig 7 option (a). If has another edge which can be replaced, and it happens to link to according to the replacement strategy, let ξ = 2 k. ee Fig 7 option (b). We repeat it for s times or ntil get the first nremoable edge. Then the estimation k is: k ˆ = 1 s s i=1 ξ i. E[ k ˆ ] = E[ξ] = k., so k ˆ is an nbiased estimation of k. B. Theoretical Model Analysis There are some theoretical models for modeling online social networks, and these models are designed to fit some common properties that are shared by almost all online social networks, for example small world phenomenon, power-law distribtion of degree and scale-free property. We wold like to introdce a well known theoretical model - Latent space model - to analysis the performance of MTO-ampler. Latent space model. Latent space graph model [21] proides s with locations in a latent metric space, and the connections are formed between two nodes with high probability when they are close enogh. 1 P (i j d ij ) =, (8) 1 + eα(dij r) ' '

9 9 Dataset #nodes #edges 90% diameter Epinions [19] lashdot A [16] lashdot B [16] TABLE I LOCAL DATAET here d ij is the distance between two nodes i and j; r is an important parameter which controls the leel of sociability of a node in this graph, and α controls the sharpness of the fnction. We will show in the following theorem that if two nodes distance is close enogh (smaller than a threshold d 0 ), then it is likely to be an non-cross-ctting edge. Therefore, we can calclate the probability that two nodes distance is smaller than d 0, and then it is straight forward to theoretically derie the improement of MTO-ampler based on the definition of condctance. Theorem 6: Gien a latent space graph model G(V, E), assme α = +, then the expected nmber of edges we can remoed ( ( ( ) )) 1/D 1 E[R] E P d < V (r) 1 (9) 3 Moreoer, if we assme the dimension D = 2, and nodes are niformly distribted in a rectangle [0, a] [0, b], then for the graph G (after remoing edges from G) is: E[Φ(G )] Φ(G) 1 z 2 1 +z r2 f a (z 1 )f b (z 2 )dz 1 z 2 (10) where z 1 and z 2 are independent niform random ariable spported on [0, a] and [0, b]. For example, from points simlation, one can get the empirical distribtion of point-wise distance. More specifically, If we let r = 0.7, a = 4 and b = 5, D = 2, then E[Φ(G )] 1.052Φ(G) (11) From this lower bond of condctance, one can derie the pper bond of theoretical mixing time. We will compare the experimental reslts together with this theoretical bond of latent space model in section V-B. A. Experimental etp V. EXPERIMENT 1) Hardware and Platform: We condcted all experiments on a compter with Intel Core i3 2.27GHz CPU, 4GB RAM and 64bit Ubnt operating system. All algorithms were implemented in Python 2.7. Or local, synthetic and online datasets are stored in the in-memory Redis database and the MongoDB database. 2) Datasets: We tested three types of datasets in the experiments: local real-world social networks, Google Pls online social network, and synthetic social networks - which we describe respectiely as follows. Local Datasets: The local social networks - i.e., real-world social networks for which the entire topology is downloaded and stored locally in or serer. For these datasets, we simlated the indiidal-ser-qery-only web interface strictly according to the definition in ection 1, and ran or algorithms oer the simlated interface. The rationale behind sing sch local datasets is so as we hae the grond trth (e.g., real aggregate qery answers oer the entire network) to compare against for ealating the performance of or algorithms. Table I shows the list of local social networks we tested with (collected from [1]). All three datasets are preioslycaptred topological snapshots of Epinions and lashdot, two real-world online social networks. ince we focs on sampling ndirected graphs in this paper, for a real-world directed graph (e.g., Epinions), we first conert it to an ndirected one by only keeping edges that appear in both directions in the original directed graph. Note by following this conersion strategy, we garantee that a random walk oer the ndirected graph can also be performed oer the original directed graph, with an additional step of erifying the inerse edge (resp. ) before committing to an edge (resp. ) in the random walk. The nmber of edges and the 90% effectie diameter reported in Table I represent ales after conersion. Google Pls Online ocial Graph: We also tested a second type of dataset: remote, online, social networks for which we hae no access to the grond trth. In particlar, we chose the Google Pls 10 network becase its API 11 is the most generos among what we tested in terms of the nmber of accesses allowed per IP address per day. Using random walk and MTO-ampler random walk, we hae accessed 240,276 sers in Google Pls. We obsered that the interface proided by Google ocial Graph API strictly adheres to or model of an indiidal-ser-qery-only web interface, in that each API reqest retrns the local neighborhood of one ser. We also collected the data of sers self-description. ynthetic ocial Networks: One can see that, for the realworld social network described aboe, we cannot change graph parameters sch as size, connectiity, etc, and obsere the corresponding performance change of or algorithms. To do so, we also tested synthetic social networks which were generated according to theoretical models. In particlar, we tested the latent space model. We note that, since the effectieness of these theoretical models are still nder research/debate, we tested these synthetic social networks for the sole prpose of obsering the potential change of performance for social networks with different characteristics. The speriority of or algorithm oer simple random walk, on the other hand, is tested by or experiments on the two types of real-world social networks. 3) Algorithms Implementation and Ealation: Algorithms: We tested for algorithms, the simple random walk (i.e., baseline), Metropolis Hastings Random Walk (MHRW), Random Jmp (RJ) and or MTO-ampler, and compared their performance oer all of the aboe-described datasets The sorce code of its Python wrapper can be fond at After April 20, 2012, this social graph api will be flly retired.

10 10 Inpt Parameters: Both simple random walk and or MTOsampler are parameter-less algorithms with one exception: They both need a conergence indicator to determine when the random walk has reached (or become sfficiently close to) the stationary distribtion - so a sample can be retrieed from it. In the experiments, we sed the Geweke indicator [8], one of the most poplarly sed methods in the literatre, which we briefly explain as follows. Gien a seqence of nodes retrieed by a random walk, the Geweke method determines whether the random walk reaches the stationary distribtion after a brn-in of k steps by first constrcting two windows of nodes: Window A is formed by the first 10% nodes retrieed by the random walk after the k-step brn-in period, and Window B formed by the last 50%. One can see that, if the random walk indeed conerges to the stationary distribtion after brn-in, then the two windows shold be statistically indistingishable. This is exactly how the Geweke indicator tests conergence. In particlar, consider any attribte θ which can be retrieed for each node in the network (a commonly sed one is degree which applies to eery graph). Let Z = θ A θ B ŜA θ + ŜB θ, (12) where θ A and θ B are means of θ for all nodes in Windows A and B, respectiely, and θ A and B θ are their corresponding ariances. One can see that Z 0 when the random walk conerges to the stationary distribtion. Ths, the Geweke indicator confirms conergence if Z falls below a threshold. In the experiments, we set the threshold to be Z 0.1 by defalt, while also performing tests with the threshold ranging from 0.01 to 1. Performance Measres for ampling: As mentioned in ection II-B, a sampling techniqe for online social networks shold be measred by qery cost and bias - i.e., the distance between the (ideal) stationary distribtion (i.e., p() = deg()/ deg() for a simple random walk) and the actal probability distribtion for each node to be sampled. To measre the qery cost, we simply sed the nmber of niqe qeries issed by the sampler. Bias, on the other hand, is more difficlt to measre, as shown in the following discssions. For a small graph, we measred bias by rnning the sampler for an extremely long amont of time (long enogh so that each node is sampled mltiple times). We then estimated the sampling distribtion by conting the nmber of times each node is retrieed, and compared this distribtion with the ideal one to derie the bias. In particlar, we measred bias as the KL-diergence between the two distribtions, specifically D KL (P P sam ) + D KL (P sam P ), where P and P sam are the ideal distribtion and the (measred) sampling distribtion, respectiely. For a larger graph, one may need a prohibitiely large nmber of qeries to sample each node mltiple times. To measre bias in this case, we se the collected samples to estimate aggregate qery answers oer all nodes in the graph, and then compare the estimation with the grond trth. One can see that, a sampler with a smaller bias tends to prodce an estimation with lower relatie error. pecifically, for the local social networks, we sed the aerage degree as the aggregate qery (as only topological information is aailable for these networks). For the Google ocial Graph experiment, we tested arios aggregate qeries inclding the aerage degree and the aerage length of ser self-description. Finally, to erify the theoretical reslts deried in the paper, we also tested a theoretical measre: the mixing time of the graph. In particlar, we continosly ran or MTO-ampler ntil it hits each node at least once - so we cold actally obtain the topology of the oerlay graph (e.g., as in Fig 1). Then, we compted the mixing time of the oerlay graph (from the econd-largest Eigenale Modls (LEM) of its adjacency matrix 12, see [5]). We wold like to cation that, while we sed it to erify or theoretical reslts of MTO- ampler neer decreasing the condctance of a graph, this theoretically compted measre does not replace the aboedescribed bias s. qery cost tests becase it is often sensitie to a small nmber of badly-connected nodes (which may not case significant bias for practical prposes). B. Performance Comparison Between imple Random Walk and MTO-ampler We started by comparing the performance of imple Random Walk (RW) and MTO-ampler oer real-world social networks sing all three performance measres described aboe - KL-diergence, relatie error s. qery cost, and theoretical mixing time. Local Datasets: We started by testing the relatie error s. qery cost tradeoff of RW, MTO, MHRW and RJ for estimating aggregate qery answers. ince only topological information is aailable for local datasets, we sed the aerage degree as the aggregate qery. Fig 8 depicts the performance comparison for the three real-world social networks. Here each point represents the aerage of 20 rns of each algorithm, and the qery cost (i.e., y-axis) represents the maximm qery cost for a random walk to generate an estimation with relatie error aboe a gien ale (i.e., x-axis). For random jmp in the experiments, we set the probability of jmping as 0.5. One can see that, for all three datasets, or MTO-ampler achiees a significant redction of qery cost compared with the RW sampler, MHRW sampler and Random Jmp sampler. We also tested the KL-diergence measred by performing an extremely long exection of RW and MTO in Fig 9 - with each prodcing samples - to estimate the sampling probability for each node. The Geweke threshold was set to be 0.1 for the test. One can see that or MTO-ampler not only reqires fewer qeries for generating each sample (i.e., conerges to the stationary distribtion faster), bt also prodces less bias than the RW sampler. To frther test the bias of samples generated by or MTO- ampler, we also condcted the test while arying the Geweke threshold from 0.1 to 0.8 on the dataset lashdot B. Fig Typical theoretical mixing time of imple Random Walk can be defined as Θ(1/ log(1/µ)), where µ is LEM of transition matrix P.

11 11 Qery Cost RW MTO MHRW RJ Qery Cost RW MTO MHRW RJ Qery Cost RW MTO MHRW RJ Relatie Error Relatie Error Relatie Error (a) lashdot A (b) lashdot B (c) Epinions Fig. 8. Bias s. Qery Cost tests for local datasets aerage degree. Kllback-Leibler diergence KL_RW KL_MTO QC_RW QC_MTO Qery Cost Mixing Time Original Graph Theoretical Bond MTO_Both MTO_RM MTO_RP Geweke Threshold Nmber of Nodes Fig. 9. Comparison between RW and MTO on qery cost and the Kllback Leibler diergence measre defined in ection V-A.3 oer all three datasets. Fig. 10. Varying Geweke Threshold to get different KL diergence on dataset lashdot B. KL and QC stands for KL diergence and Qery Cost respectiely. Fig. 11. Comparison of theoretical mixing time on latent space graph model. MTO Both: Remoe and replace edges. MTO RM: Only remoe edges. MTO RP: Only replace edges. depicts the change of measred bias for RW and MTO, respectiely. One can see from the figre that or MTO- ampler achiees smaller bias than RW for all cases being tested. In addition, a smaller threshold leads to a smaller bias and larger qery cost, as indicated by the definition of Geweke conergence monitor. Google Pls online social network: For Google Pls, we do not hae the grond trth as the entire social network is too large (abot 85.2 million sers in Feb ) to be crawled. Ths, we performed the tests in two steps. First, we continosly ran each sampler ntil their Geweke conergence monitor indicated that it had reached its stationary distribtion. We then sed the final estimation as the presmptie grond trth which we refer to as the conerged ale. In the second step, we sed the conerged ale to compte the relatie error s. qery cost tradeoff as preiosly described. Fig 12(a) shows the estimated aerage degree when rnning RW and MTO-ampler random walk on Google Pls. It clearly shows that MTO-ampler s ariance is smaller and conerges faster than simpler random walk. Fig 12(b) and 12(c) illstrate the comparison between RW and MTO of the relatie error s qery cost of mltiple attribtes. We note that the self-description length is the nmber of characters in sers self-description. One can see that or MTO-ampler significantly otperforms RW. 13 Estimated by Pal Allen s model, ynthetic ocial Networks: Finally, we condcted frther analysis of or MTO-ampler, in particlar the indiidal effects of edge remoals (RM) and edge replacements (RP), sing the synthetic latent space model described in ection V- A.2. Fig 11 depicts the reslts when the nmber of nodes in the graph aries from 50 to 100 (with the latent space model, we distribted these nodes in an area of [0, 4] [0, 5], and set r = 0.7). We deried the theoretical mixing time from the second largest eigenale modls of the transition matrix. Note that Fig 11 also incldes the theoretical bond deried in ection 4.2. One can see from the figre that or final MTO-ampler achiees better efficiency than the indiidal applications of edge remoal and replacement. In addition, the theoretical model represents a conseration estimation that is otperformed by the real efficiency of MTO-ampler - consistent with or reslts in ection 4.2. VI. RELATED WORK ampling from online social networks. eeral papers [2], [13], [15] hae considered sampling from general large graph, and [10], [12], [18] focs on sampling from online social networks. With global topology, [15] discssed sampling techniqes like random node, random edge, random sbgraph in large graphs. [11] introdced Albatross sampling which combines random jmp and MHRW. [10] also demonstrated tre niform sampling method among the sers id as grond-trth.