The Growth of Diaspora A Decentralized Online Social Network in the Wild

Transcription

1 The Growth of Diaspora A Decentralized Online Social Network in the Wild Ames Bielenberg Swarthmore College Swarthmore, PA abiele1@swarthmore.edu Lara Helm Wellesley College Wellesley, MA lhelm@wellesley.edu Anthony Gentilucci, Dan Stefanescu, Honggang Zhang Dept. of Math & Computer Science, Suffolk University Boston, MA {argentilucci, dstefanescu, hzhang}@suffolk.edu Abstract The Diaspora network [1] is a recently launched decentralized online social network with over 216, 000 users as of November 16, It is a network of independent, federated Diaspora servers that are administrated by individual users who allow Diaspora users profiles to be hosted on their servers. In this paper we take a first look at the Diaspora network s overall growth in terms of number of users, the topology of its interconnected servers, and the reliability of those servers. We also present a simple analysis to explain the growth of the Diaspora network. Our timely measurement study of a real-world decentralized online social network sheds light on the evolution of such a network in practice, and provides valuable observations and insights that can help the future design and implementation of decentralized online social networking. I. INTRODUCTION In spite of the recent tremendous success of major online social networking (OSN) sites (such as Facebook [2], RenRen [3], Vkontakte [4], Orkut [5]) in attracting hundreds of millions of users, they have also raised increasing concerns of privacy and security within the research community (e.g., [6], [7], [8]) because of their centralized design. Specifically, those major OSN sites are logically centralized in the sense that the data of all users of an OSN site are centrally owned by a single administrative domain or a single entity that manages the site, and the same entity handles all communications between its users on the site. To address the problems and concerns associated with data ownership, privacy, and security of centralized OSN sites, several decentralized OSN designs have been recently proposed and experimented in several ongoing projects, such as PrPl [9], Mr. Privacy [10], Diaspora [1], and PeerSon [11]. Among those decentralized design approaches, the Diaspora network [1] is the only real-world Internet-scale decentralized online social network in use today, to the best of our knowledge. It was recently launched on November 23, 2010 [12]. It is a network of independent, federated Diaspora servers that are administrated by individual users who allow other Diaspora users profiles to be hosted on their respective servers. Diaspora is currently in the alpha stage of development and permits the public to join and try the service. Although a decentralized approach to OSN seems a quite promising alternative for potentially much improved privacy and security for users, there is still a lack of basic understanding regarding how a decentralized OSN grows over time in terms of the number of users joining the network, their joining patterns, and the network s connectivity. We believe that the knowledge of such basic network dynamics can significantly help to lay out a solid foundation for the architecture and protocol design of a decentralized OSN. In this work, we take a first step in this direction, by collecting real-world data and attempting to find out how the Diaspora network is structured, how it functions, and how its servers are growing and connecting. The major contribution of this work is the first measurement study (to the best of our knowledge) of the first decentralized OSN in practice, based on our collected data over a span of 150 days on the Diaspora network, and the analysis of the Diaspora network s dynamics based on our collected data. We hope that our work can shed light on the evolution of decentralized OSNs in practice and provide valuable empirical data and lessons to help further improve their design and implementation to grant better privacy to millions of users. The rest of the paper is organized as follows. In Section II, we briefly describe the basic sharing privacy design in the Diaspora network. In Section III, we outline the data collection process and present the summary data of the Diaspora network. In Sections IV and V, we take a detailed look at the growth dynamics of the Diaspora network and its server reliability characteristics. In Section VI, we give an explanation of Diaspora s growth in terms of preferential attachment. Finally, Section VII concludes the paper. II. PRELIMINARIES: PRIVACY DESIGN IN DIASPORA In order to understand the Diaspora network, we first give a brief overview of its design for sharing privacy. It is claimed that the Diaspora network s primary advantage over existing centralized OSNs (e.g., [2]) is its privacy-preserving design, i.e., it allows users to stay in control of their data [1]. Specifically, the Diaspora network is a network of independent, federated Diaspora servers that are administrated by individual users. Rather than forcing users to store all their information on one central server or a collection of servers owned by one single entity, the Diaspora network users decide for themselves on which servers their information will be stored. Some users choose to maintain their own Diaspora servers in order to keep complete control of their data, while others might choose to join an existing server.

2 On the Diaspora network most communication is conducted via posts. A post has two levels of privacy: public and limited. A public post is visible to anyone while a limited post is only shared with a specific group of users. A user organizes his or her contacts into sharing groups, referred to as aspects. A post can be shared with one or more aspects, and only users in those designated aspects will be able to see it. A user can also choose to only view posts from users in certain aspects. The notion of aspect is similar to the circle of Google+ [13]. A Diaspora server uses push design to send data and notifications to other servers. For example, when a user on a server shares a post with his or her friends in an aspect, the post data is pushed out to all other servers hosting his or her friends in that aspect. Once the post is stored on those other servers, the users (or friends) it is shared with will see it in their news stream. In essence, the Diaspora network distributes data replicas to multiple servers. Even though this approach keeps data away from a central server or some central entity, it still leaves the security and integrity of a post in the hands of the administrators of the servers where it is stored. A typical Diaspora server grants its administrator read and write access to unencrypted user information in the database hosted by the server. If a user decides to join an existing server, he must trust the owner of that server with his data. When a user communicates with a user on a different server, data is pushed to and stored in that other server s database. Therefore, a user must entrust his private data to both his own server as well as the servers of the users that he or she communicates with. We think that such a potential privacy leak can easily be fixed in the future by encrypting all users data before it is routed on the Diaspora network or stored in any Diaspora server. III. MAPPING THE DIASPORA NETWORK Recall that we are interested in whether Diaspora, a claimed better-privacy-preserving social network, is well received among users on the Internet. Given the tremendous success and popularity of existing centralized OSNs such as Facebook [2], one certainly does not expect that any decentralized OSN will soon become more popular than those existing centralized OSNs, even though a decentralized OSN might have better privacy design. Nevertheless, the user growth dynamics of the Diaspora network can still shed light on the future of decentralized OSNs. To understand such dynamics, we crawled or collected a large amount of data on users and their friends from the Diaspora network. With the understanding of the basic privacy design outlined in the previous section in mind, we will now describe our data collecting process. A. How to Join Diaspora and Connect with A Friend In order to understand our crawling process, we first briefly describe how two users on the Diaspora network connect with each other as friends. For simplicity, we now refer to two Diaspora users - Alice and Bob. In the Diaspora network, an open server allows anyone to join while a closed server only allows users to join via invitation. There are three different ways that Alice can join Diaspora, i.e., create an account in the Diaspora network: 1) She can join a closed server after receiving an invitation; 2) She can join an open server; 3) She can create her own server and make an account on that server. After creating an account in any of these ways, Alice might want to find her friend Bob on the Diaspora network to share information with him. There are two ways that Alice can find and connect with Bob: 1) If Bob is located on Alice s home server, she can use her home server s search feature. 2) If Bob is located on a different server; Alice must know his entire Diaspora handle to find him. Bob must explicitly give Alice his Diaspora handle via a different medium such as phone, , instant message, etc. A Diaspora handle is similar in form to an address: username@servername, for example alice@example.com. For the following scenario lets say Alice and Bob are located on two different servers. Alice s handle is alice@joindiaspora.com and Bob s is bob@diasp.org. When Alice connects with Bob, Bob s profile is replicated on Alice s home server. No matter whether Alice views Bob s profile on his home server or a replicated copy of his profile on a different server, Bob s profile will still be replicated on Alice s home server. This happens because when Alice connects to a user s profile on a different server using their Diaspora handle, her home server queries the remote server and fetches the user s profile information and replicates it on her home server. B. Basic Crawling Process As we know, most online social networks by default display each user s friend list which represent the links between a user and each of his friends. However, due to its privacy-preserving design, the Diaspora network does not show friend lists to users. This prevented us from collecting data about the links between individual users. So instead, our crawler went to each server and directly accessed each profile hosted on that server. Note that profiles of users on a server can be classified as home profiles and foreign profiles, which correspond respectively to the users that have an account with that server and the users that have accounts on other servers but their profiles are replicated on this server. Accordingly, we use home users of a server to denote those users that have created their accounts on that server. The set of all distinct users on the Diaspora network is the collection of all home users on all servers. Each profile on a server can be accessed by an assigned numeric ID. On most Diaspora servers, these IDs are assigned incrementally, meaning the first profile is 1, the second is 2, and so on. Our crawler started with one server and, by trying IDs incrementally, it tried to find all home profiles and replicated foreign profiles on that server. When the crawler found a foreign profile, it stored that profile s home server in a queue to be crawled next. We found that some servers were down or no longer functioning as Diaspora servers. Other servers were closed, i.e., it required an invitation to join and access the profiles. We were not able to crawl those down or closed servers, with the exception of joindiaspora.com,

3 joindiaspora.com: User Growth joindiaspora.com % users % 3.1% 5.3% 15.1% diasp.org diasp.eu others pod.geraspora.d Fig Number of home users on joindiaspora.com during 150 days joindiaspora.com: Profile Growth Fig. 1. User distribution among servers in the Diaspora network as of November 16, Size distribution of all servers 10 2 profiles number of servers number of users Fig. 2. Size distribution of all servers. to which we received an invite. At the beginning of our data collection, we gathered all available existing profile and server data. After this point we began to scan each server every 20 minutes for new home and foreign profiles. By scanning the servers continuously, we were able to track the growth of individual servers and their connectivity. After crawling for 150 days from June 20 to November 16, 2011, we identified 755 servers, 304 of which we were able to crawl. We were not able to crawl other servers due to the fact that they were closed servers. Overall we have observed over 216, 000 distinct users, and have crawled a total of 535, 785 profiles. Figure 1 shows the distribution of home users across the severs that we crawled. We see that although Diaspora is decentralized, users are not evenly distributed across the servers. Note that joindiaspora.com, a server maintained by the developers of Diaspora software, contains over 70% of the users in the entire network. In total the largest four servers contain 94% of users. In Figure 2, we show the size distribution of servers (in terms of numbers of home users on servers), which approximately follows the power law, that is, the majority of servers only host a small percentage of users, and the several largest servers, or hubs, host the majority of users in the network Fig. 4. Number of replicated profiles on joindiaspora.com during 150 days. IV. DYNAMIC GROWTH PATTERNS OF DIASPORA SERVERS By analyzing the number of users or profiles on a server over time (i.e., growth rate), we can make a few observations. The overall rates of both home users and remote profiles (that are replicated from other servers) varies across the servers that we observed. Since the variations of growth rates indicate variations in activity, we observe that there are periods of fairly constant levels of activity and sudden rapid changes. A. Two Servers in the Diaspora Network Figure 3 shows the growth of the largest Diaspora server, joindiaspora.com in terms of its home users. This growth pattern shows several notable features. Overall the graph 1 shows a linear growth between when we started crawling in late June of 2011 through September 1st of In early September, joindiaspora.com took a step away from being a closed server by handing out invites to all who requested them [14]. This event caused a faster growth than before, as can be seen from September onwards in the figure. Compared with its home user growth pattern, the growth of joindiaspora.com s replicated profiles shows a somewhat different increasing pattern, as can be seen in Figure 4. The replicated profile growth remains linear from mid August until mid September, this is followed by a sharp increase in the rate of replicated profiles which lasts from late September until mid November of Note that the gaps in this graph, and in all other graphs in the rest of the paper, are caused by our crawler failures.

4 Fig. 5. users users diasp.eu: User Growth Number of home users on diasp.eu during 150 days All Servers Fig. 6. Entire Network User Growth. Number of home users on all servers over a span of 150 days. portion replicated Portion Replicated users Fig. 7. Portion Replicated. Number of home users vs. (replicated profiles/total profiles) for each server. In addition, we also observed that diasp.eu s (the third largest server) home user growth pattern is relatively nonlinear, as shown in Figure 5. From when we started crawling until mid September, the new user rate remained relatively low. From this point forward there was a sudden influx in the new user rate during which the server nearly doubled in size between mid September and mid November. B. Growth of Home Users in the Diaspora Network Figure 6 shows the number of home users on all servers over a span of 150 days. Since joindiaspora.com hosts such a large share of the network s users, the graph of the user growth of the entire network looks very similar to that of joindiaspora.com. Figure 7 plots the number of home users on a server against the portion of replicated profiles from remote servers over the total number of profiles on a server for 297 servers. We can see that servers with greater than 100 users tend to follow a pattern where larger servers (those with more home users) have a smaller portion of replicated profiles while smaller servers have a larger portion of replicated profiles. The possible reason for this is that on a larger server, users are more likely to connect with other users on the same server, because the probability of finding a friend on the same server is larger if the server is larger (i.e., hosting larger number of users). Thus, we suspect that on joindiaspora.com a low percentage of replicated profiles is expected. Our data confirmed this hypothesis: only 18% of profiles on joindiaspora are replicated profiles. This trend indicates that after a server reaches a certain size it connects out proportionally less than smaller servers. And, as a home to around 70% of users on the network, joindiaspora.com could not exceed a percentage of replicated profiles greater than 30%. This would be the case in which every user s profile in the network is replicated on joindiaspora.com. V. RELIABILITY OF SERVERS IN THE DIASPORA NETWORK In addition to the growth of users on servers, we also examined the reliability of Diaspora servers. The Diaspora network faces a unique challenge since the reliability of its servers will inevitably vary due to its decentralized nature. As servers are administered by individual users, thus, it is unrealistic to expect that all servers are reliably online all the time. Diaspora s current implementation of how users communicate with each other requires both the sending and receiving server to be online in order for a successful transaction to take place. This means that if a server is down when a message or profile up or post is pushed, the data will never be sent. Therefore, at this stage of Diaspora s development, it is vital to the network s functionality that servers remain reliably online, especially servers which host many users or those that communicate frequently with remote servers. In order to evaluate server reliability, we periodically pinged each server. A standard IP ping was not sufficient for our purposes. It was necessary to not only ensure a web server was online but that it was also hosting a Diaspora server. We verified that a server was both online and hosting a Diaspora server by using HTTP pings to check for the presence of a unique HTML element common to all Diaspora servers. Over 150 days, we sent pings to 764 servers every 26 minutes, collecting a total of over 2, 000, 000 pings. From this data we were able to determine each server s overall uptime percentage, that is, the portion of time that a server is up, as summarized in Figures 8 and 9. As can be seen from these figures, over 35% of the servers we pinged were never online during the 150 day period, while the top 20% of servers had over 90% uptime. About half of the servers have less than 50% percent uptime. For servers in the 50-th to 80-th percentile, the up-times are fairly evenly distributed between approximately 0% and 90% uptime.

5 100 Empirical CDF 2 Server Size vs New User Rate Percent of servers users/hour(log) Fig Fig. 8. Users Percent uptime Distribution of server uptime percentages. Uptime percentage vs. number of home users Percent uptime Uptime percentage vs. number of home users for each server. VI. PREFERENTIAL ATTACHMENT We are also interested in how to explain the growth of the Diaspora network in terms of the sizes of its servers. The Diaspora network is also a type of social network exhibiting scale-free properties, and for these types of networks, there are many existing models that attempt to explain network growth. Among which, we find that the Diaspora network s growth follows a pattern similar to what the preferential attachment model [15] predicts, as described below. In the preferential attachment model, a new node is more likely to connect to an existing node with a higher degree. If we think that a server in the Diaspora network is a node, then a direct application of the preferential attachment model requires us to measure the links among server nodes. Note that a link between two server nodes represent a communication between two users on these two servers nodes. However, in the Diaspora network, due to its privacy-preserving design, we were unable to observe the way that specific users connect to each other, i.e., it is difficult to get the number of links among server nodes. Thus, instead of directly applying the preferential attachment model, we chose to look at the impact of a server s current size (in terms of number of home users) on its growth rate. Analogous to the increase rate of a node s degree in the preferential attachment model [15], we look at the server size growth rate in our analysis of the Diaspora network. In the preferential attachment model, a node with higher degree is more likely to connect to a newly joined node (which is indicated by a higher degree increase rate), whereas in our analysis, we conjecture that a server with a larger number of server size(log) Gradient and intercept: R-squared: p-value: e-11 Fig. 10. The number of home users vs. the number of newly joined users per hour since crawling began. home users is more likely to attract new home users (which is indicated by a higher size increase rate). In fact, our conjecture is shown to be approximately true from the data we collected. Figure 10 (in log scale) shows the number of home users on a server vs. the number of newly joined users per hour on a server since crawling began. There appears to be a strong positive correlation between the current size of a server and its growth rate. We perform a linear regression analysis on this data using initial server size as a predictor variable and new user rate as the response variable. For a server i, we let n (i,start) denote the size of a server at the time when we started crawling, and let n (i,end) denote the size of the server at the end of our data collecting process. Then we can calculate the average increasing rate of the number of users on server i as n (i,end) /n (i,start), denoted as r i. We first find (n (i,start), r i ) for all servers, where i = {1, 2,..., S} (where S denotes the number of servers), and then we find the following statistically significant linear relationship in log scale: log(r) = log(n) (1) for which the p-value is This suggests that the greater the existing number of home users in a server, the higher the rate at which new users join the server. Although current server size is clearly a predictor for new user joining rate, we hypothesize that there are other factors that also influence which server a new user would join. The factor we find most likely to be significant is server uptime. Regardless of how large a server was when we began to crawl, if it was rarely online, it would have a low growth rate. To verify this hypothesis, we performed linear regression on both initial server size and server uptime percentage as predictor variables. Because we previously observed (in Figure 9) that there was some correlation between server size and percent uptime, we initially included a product variable to measure the interaction between these variables. We found, however, that the interaction was not statistically significant. Thus, we only run a multiple regression of server increasing rate r on the number of initial users on a server (denoted as n) and the portion of time that the server is up (denoted as u). We derive the following relationship. log(r) = log(n) log(u) (2)

6 Fig. 11. profiles over users Jun Jul New Profile Rate/New User Rate over Time Jul Aug Aug joindiaspora.com diasp.org diasp.eu Sep Sep time Oct Oct Nov Nov Ratio of new profile rate to new user rate over a span of 150 days. This regression analysis shows a stronger and more statistically significant relationship than the previous regression of rate r only on initial server size n. For regression in (2), the coefficient of determination is 0.863, larger than of (1), showing a stronger correlation. Furthermore, recall that in Figure 6, there are three different time periods (line segments) with different increasing rates of users in the network. To understand each period, we ran the same multiple regressions for data during each period and we find similar results. In summary, a server s size and uptime can be used to predict the rate at which it will attract new users. The more users and higher uptime percentage of a server, the higher the new user join rate for that server. We are also interested in how the relative scale between new home users and new replicated profiles change over time, which indicates the changes in a server s growth compared to its connectivity. Figure 11 shows the ratio of new profile rate to new user rate over a span of 150 days. This plot shows the ratio of new replicated external profiles to new home users over time for three of the four largest servers. We see that joindiaspora.com, the largest server, generally has the lowest rate of new profiles per new user, remaining less than one most of the time. On the other hand diasp.eu, a smaller server, has the highest rate of new profiles per new user which is generally greater than one. VII. CONCLUSION We present the first systematic measurement study of the Diaspora network s implementation, topology and user growth. Despite the fact that Diaspora allows users to host their data on their own server, our data shows that most users choose to join existing servers. These users rely on a server s owner to maintain the security, integrity, and reliability of their data. Furthermore, a Diaspora user must decide whether he trusts his data with the owner of a particular server as well as the owners of all the servers hosting the other users he communicates with. One possible fix is to let a user store only the encrypted copy of his or her personal data on a Diaspora server. According to our study, large servers with consistently reliable uptime are most likely to attract new users. Most Diaspora users are concentrated on a few very large servers that act as network hubs. Although these large servers are tightly connected to each other, they generally have a proportionally low number of outside connections compared to smaller servers. This suggests that many users do not host their own data in the network even though Diaspora provides this option, which distinguishes it from centralized OSNs. One possible explanation for the concentration of users on the few largest servers could be that the Diaspora network is currently in its alpha phase of development and many users are still learning how to use the network and servers. In addition, the features and implementations that we have observed are likely to change in future. It is possible that when Diaspora reaches beta and inter-server connection becomes more reliable, users may become more willing to run their own servers or join small existing servers. This is particularly likely if individual server administrators begin to add attractive, unique features to their servers. However, as the network exists now, most new users choose to join large existing servers with high reliability. Therefore, Diaspora users are likely to remain concentrated on a few prominent servers for the foreseeable future. ACKNOWLEDGMENT The authors would like to thank Tristam MacDonald at Suffolk University. This paper is based upon work supported in part by the National Science Foundation (NSF) under NSF REU Grant CNS and CAREER Grant CNS REFERENCES [1] Diaspora. [Online]. Available: [2] Facebook. (2011, December). [Online]. Available: [3] RenRen. [Online]. Available: [4] Vkontakte. [Online]. Available: [5] Orkut. [Online]. Available: [6] J. Anderson, C. Diaz, J. Bonneau, and F. Stajano, Privacy preserving social networking over untrusted networks, in The Second ACM SIG- COMM Workshop on Online Social Networks, [7] B. Krishnamurthy and C. Wills, On the leakage of personally identifiable information via online social networks, in The Second ACM SIGCOMM Workshop on Online Social Networks, [8] A. Shakimov, A. Varshavsky, L. Cox, and R. Caceres, Privacy, cost, and availability tradeoffs in decentralized osns, in The Second ACM SIGCOMM Workshop on Online Social Networks, [9] S.-W. Seong, J. Seo, M. Nasielski, D. Sengupta, S. Hangal, S. K. Teh, R. Chu, B. Dodson, and M. S. Lam, Prpl: a decentralized social networking infrastructure, in 1st Intl. Workshop on Mobile Cloud Computing Services: Social Networks and Beyond, San Francisco, [10] M. Fischer, T. J. Purtell, R. Chu, and M. S. Lam, Mr. privacy: Open and federated social networking using . in Tech Report, [11] S. Buchegger, D. Schiöberg, L. H. Vu, and A. Datta, Peerson: P2p social networking - early experiences and insights, in The Second ACM Workshop on Social Network Systems 2009, Nuernberg, Germany, [12] Wiki-Diaspora. [Online]. Available: (software) [13] Wiki. (2011) Goolge plus wiki. [Online]. Available: [14] blog diasporafoundation. (2011). [Online]. Available: [15] A.-L. Barabsi and R. Albert, Emergence of scaling in random networks, Science, October 1999.