Comparative Analysis of Internet Topology Data sets

Transcription

1 Comparative Analysis of Internet Topology Data sets M. Abdullah Canbaz, Jay Thom, and Mehmet Hadi Gunes Department of Computer Science, University of Nevada Reno, Reno, Nevada {mcanbaz, jthom, Abstract Several platforms continuously collect and provide Internet topology measurement data to the public. In this paper, we analyze the data provided by such public Internet topology measurement platforms. In particular, we analyze the router level probe data sets from USC/ISI Ant census, CAIDA Archipelago (Ark), UW information Plane (iplane), Measurement Lab (M- Lab), and RE NCC Atlas and Autonomous System level BGP data sets from CAIDA, Internet Research Lab (IRL), and CIDR. We provide insights into the characteristics of the platforms and the properties of the data sets sampled by these platforms. We further compare the topology coverage of these platforms. I. INTRODUCTION The Internet is continuously growing with various entities ranging from networking hardware manufacturers to application developers providing services. Additionally, business relationships between Autonomous Systems (AS) change continually, resulting in a dynamic topology where each AS optimizes its own communication performance. Understanding the network topology dynamics is beneficial to design more efficient protocols and systems [1]. Continued study in the area of network topology is also integral to the efficiency, stability and security of the Internet as it expands to connect a multitude of devices and to provide new services. Measurement studies span from active to passive probing approaches [2] [4]. Researchers have developed approaches to measure the Internet topology, and have built platforms for the collection of Internet topology data [5]. Several groups have developed long-lasting measurement platforms that continuously probe the Internet, and share topology data repositories with network practitioners and researchers [6]. Organizations that provide accurate real-time and historical data play a vital role in understanding the Internet backbone and growth. The provided data sets not only guide network research efforts, but also help in informed regulations and business decisions. As time progresses, it is important that these platforms are maintained, and new ones are built to provide a continued view of the Internet and its functionality. In this study, we analyze public Internet topology measurement platforms and compare the characteristics of their data sets. In particular, we focus on Autonomous System level data from Cooperative Association for Internet Data Analysis (CAIDA) [7], UCLA Internet Research Lab (IRL) [8], and CIDR [9]; and router level data from USC/ISI Ant census [10], CAIDA Archipelago (Ark) [7], University of Washington information Plane (iplane) [11], Measurement Lab (M-Lab) [12], and RE NCC Atlas [13]. Measurement data is available at In Section II, we provide an overview of the measurement platforms that are utilized. In Section III, we analyze BGP data sets (i.e., prefix announcements). In Section IV, we analyze probing data sets (i.e., ping and traceroute). In Section V, we analyze AS coverage of the data sets. Finally, we conclude with Section VI. II. MEASUREMENT PLATFORMS In this section, we briefly summarize features of measurement platforms from which probe and BGP data is analyzed. CAIDA Archipelago (Ark): The Cooperative Association for Internet Data Analysis (CAIDA) introduced their Archipelago measurement infrastructure (Ark) to support large-scale measurements and to collect and distribute data sets to support various research, engineering, and operational interests [14]. Ripe Atlas: Ripe NNC is a more recent measurement platform deployed in Jan 2013 by the RE Network Coordination Centre (RE NCC). The RE measurement platform consists of a large number of probes distributed across the globe capable of performing RTT, ping, traceroute, DNS, SSL, NTP and HTTP measurements. Ant Census: Researchers at the University of Southern California (USC) Information Sciences Institute (ISI) began an effort in 2003 known as the Ant Census to collect data about the Internet address space. This is accomplished by conducting a census of edge hosts in the visible Internet [15]. Ant estimates that 3.6% of the allocated address space is occupied by visible hosts, with approximately 34 million addresses reported as stable and visible to their probes (16% of responsive addresses). From this, they estimate there are 60 million stable Internet-accessible computers on the network [15]. M-Lab: The Measurement Lab is an open, distributed server platform designed to allow researchers to deploy their active Internet measurement tools [12]. M-Lab is run in conjunction with PlanetLab [16] with a few key differences in administration and software [17]. iplane: iplane is a measurement platform hosted by the University of Washinton Computer Science and Engineering Department that utilizes the PlanetLab system. Its goal is to provide a scalable service generating accurate predictions of Internet path performance for emerging overlay services [11]. iplane utilizes the PlanteLab infrastructure to probe the avail-

2 able address space, along with various public Looking Glass/Traceroute servers for low-intensity probing. IRL: Internet Research Lab (IRL) at the University of California Los Angeles hosts an archive of historical Internet AS-level topology data for academic research. Historical BGP data collected by Route Views, RE RIS, PCH and Internet2 are available on a daily and monthly basis till 2015 [8]. CIDR: CIDR-report is an online resource for network data such as BGP updates, v4 and v6 address space reports, AS number space reports, and a variety of useful information for Internet measurement [9].,000 10, caida16 cidr caida15 irl15 III. BGP DATA SETS AS level topological data can deliver insight into the connectivity and relationship among ASes. In order to gain a comprehensive understanding of the BGP data sets, in this section, we analyze AS level prefix announcements of UCSD CAIDA [7], UCLA IRL [8], and CIDR [9]. Table I presents BGP data (i.e., announced ASes, prefixes, and total number of s in prefixes) provided by each platform. Among BGP data providers, UCLA IRL had no new data since 2015 and we were not able to obtain historical data from CIDR. We combined IRL and CIDR s latest data sets with Caida s BGP prefix announcement from the same time frame to obtain 2015 and 2016 data sets, respectively. While CIDR provides much fewer number of ASes, it has comparable number of addresses in the announced prefixes. We then processed the prefix announcements to filter the duplicates and merged consecutive smaller announcements into bigger ones. We identified multiple announcements for AS prefixes, i.e., an range being announced as part of two separate ASes, also called MOAS prefixes [18]. We observed that the IRL and CIDR data contain 51,554 and 946 prefix announcements, respectively, that overlap with each other involving 10,061 and 524 ASes. On the other hand, CAIDA had no such overlapping announcements indicating that such announcements are being filtered. When we consider pairwise overlaps between the data sets for respective years, we observe IRL and CIDR announcements also overlap with CAIDA. In figure 1, we present how the subnet prefix announcements are distributed for each data set. Granularity of the prefix announcements give insight into how the longest prefix matching operations in the border routers result in higher AS coverage for each router level data source. We observe that a majority of the announced prefixes are /24. We observe a drop in the number of unique subnets smaller than /24. Note that we eliminated non-conflicting smaller prefix announcements over the same region. TABLE I BGP ANNOUNCEMENTS ASes Prefixes s CIDR16 1, ,734 2,694,269,384 CAIDA16 55, ,237 3,220,560, , ,507 5,321,550,144 IRL15 51, ,071 3,939,677,760 CAIDA15 49, ,171 3,087,590, , ,956 4,738,394,582 1 /8 /10 /12 /14 /16 /18 /20 /22 /24 /26 /28 /30 /32 Fig. 1. Prefix announcements vs Mask Distribution (log scale) IV. PROBE DATA SETS In this section, we analyze the probe data sets (i.e., ping and traceroute) from USC/ISI Ant [10], CAIDA Ark [7], University of Washington iplane [11], M-Lab [12], and RE NCC Atlas [13]. We collect traceroute data from the four data sources and then perform analysis on this data to better understand the coverage of each platform. We first analyze a 30-day set of data (June 1, 2016 to June 30, 2016) from these platforms and then perform in depth analysis on a single cycle of data. As Ark cycles recurred every 5 days, in detailed analysis, we considered the data from the same 5-day period (June 1, 2016 to June 5, 2016) from all sources. This smaller data set was comparatively consistent with the larger data set. Figure 2 displays the cumulative number of source addresses (i.e., vantage points), destination addresses (i.e., targets), total traceroutes performed, observed addresses, and discovered Edges by each of the four data sources over the 30-day period. We put an edge between two consecutive Addresses assuming a link between them. The Source figure shows that the number of vantage points on the first day is 80%, 55%, 88% and 98% of all vantage points used over the month for Ark, iplane, Ripe, and M-lab, respectively. The Destination figure shows that the number of destinations do not change for iplane and Ripe but the first day of Ark and M-Lab is 6% of the combined destinations, for both. The number of observed addresses do not vary in iplane and Ripe, both observe 89% and 88% of the overall addresses in the first day, while Ark and M-Lab both only observe 6% of month-long addresses in the first day. Similarly, iplane and Ripe observe 49% and 42% of overall edges in first day, whereas Ark and M-Lab only observe 12% and 9% of month-long edges. Finally, we observe all methods have new traces over the time (i.e., there is a unique edge in the path trace). Overall, Ark, iplane, Ripe, and M-Lab observe only 6%, 9%, 13%, and 3% of combined traces in the first day, respectively. Table II outlines the number of sources, destinations, traces, s and edges for the June 1-5, 2016 data set (corresponding to one Ark cycle). It also shows unique data, i.e., data that were not observed in the other platforms, for each category.

3 Edge Trace Source Destination Edge Trace Dest Src TABLE II COVERAGE FOR 5 DAYS DATA RE M-Lab iplane Ark Total 6, Unique 6, Total , ,733 52,790,278 Unique , ,550 52,783,156 Total 883, ,310 45,793,268 54,297,491 Unique 883, ,309 45,793,268 54,297,490 w/repeat 169, , ,454 43,932,702 w/loop 777, ,262 16,844,293 32,152,569 Total 6, , ,313 54,359,540 U.-Other 5, , ,641 54,040,614 U.-Ant 4, , ,789 46,328,571 Total 280, ,721 2,914,809 10,019,704 Unique 170, ,619 1,745,528 8,742,947 Bridge 61,876 80, , ,681,000,000,000 Ark iplane M-Lab Ripe Fig. 2. Cumulative # of Destinations, Sources, Traces, s, and Edges, for a month (log scale) Overall, there are 6516 sources and 53,162,697 destinations when all data sets are combined. We observe that Ark, iplane, M-Lab, and Ripe have 7122, 4183, 3021, and 1 destination, respectively, that is traced by another platform as well (corresponding to 0.01%, 1.9%, 1.9%, and 0.5% of each platforms destinations). As Ripe only allows tracing to its own anchors, we observe that they utilize only 193 destinations. Overall, while others have traces from few sources to a large number of destinations, Ripe has traces from thousands of sources to less than two hundred destinations. Trace rows of Table II present the number of unique traces where traces that have the same sequence of addresses are ignored. When all data sets were combined, there are 101,453,864 unique traces. We observe that Ark, iplane, M- Lab, and Ripe obtain 53.5%, 47.8%, 0.5%, and 0.9% of all traces, respectively. We also identified traces that contain a routing loop and repeated s at the end (most likely caused by a NAT box or firewall). We observe that 1.4%, 49.4%, 80.9%, and 9.6% of iplane, M-Lab, Ark, and Ripe traces, respectively, have a repeating addresses at the end. We believe Ripe and iplane observe fewer border devices as they have selected destinations while Ark and M-Lab trace toward arbitrary addresses that might contain a border device that causes these repetitions. Similarly, we observe 36.8%, 42.0%, 59.2%, and 44.0% of iplane, M-Lab, Ark, and Ripe traces, respectively, have a routing loop in them. Overall, we observe about half of traces have a routing loop in them. Overall, when all data sets are combined there are 54,771,286 addresses. In Table II, we observe that 99.4%, 44.1%, 97.6%, and 82.8% of addresses observed by Ark, iplane, M-Lab, and Ripe are not observed in the other platforms during the same time frame. When compared to Ant census data that has million ping responsive addresses (note that Ant census cycles take about two months to probe all v4 space), we observe Ark, iplane, M-Lab, Ripe, and all platforms combined, has 88.1%, 96.5%, 92.2%, 86.1%, and 88.1%, respectively, addresses that were not observed in the Ant census data. This most likely indicates that these devices ignored the ICMP ping requests but generated an ICMP TTL exceeded message. We observe that each platform is able to obtain unique edges compared to the rest in Edge rows of Table II. When all data sets are combined, there are 12,180,189 edges. In particular, 89.9%, 96.3%, 98.4%, and 61.8% of edges sampled by Ark, iplane, M-Lab, and Ripe, respectively, are unique to that platform and not observed by another one during the same 5-day period. These findings indicate that vantage point location plays an important role in discovering new edges. We also perform an analysis of the relationships between vantage points and data they are able to discover. Figure 3 presents the number of unique traces observed, number of edges discovered, total number of addresses visited, and number of destination addresses targeted per vantage point. We can see from the data that Ripe is utilizing a much larger number of vantage points (6146 in total for the 5-day period), yet they actually see a smaller portion of the network. This is likely because they perform repeated measurements of the same space, since they are primarily concerned with issues of reachability rather than the collection of data for topology study. iplane and M-lab both utilize the PlanetLab nodes for vantage points, but iplane seems to see a much larger part of the network since it scans a much larger block of destination addresses. Ark by far outperforms both with only 112 vantage points, but targets a larger number of addresses.

4 Destinations Edge Trace Fig. 3. Traces, Destinations, s and Edges per Vantage Point (log-log scale) In the destinations subfigure of Figure 3, we observe that a majority of Ripe anchors trace to each other but then Ripe probes trace to a subset of the anchors. We observe a majority of Ark vantage points trace toward a large set of destinations while a few of them trace to a much smaller subset. iplane has a consistent destination tracing with the exception of a few vantage points that are slightly behind the others. M-Lab has a mixed coverage as the trace destinations are not consistent. We observe a similar pattern with the number of traces, i.e., Ark and iplane vantage points have similar workloads while Ripe and M-Lab workloads vary. Most Ark monitors observe around 1 million addresses, while a few of them only observe tens of thousands of s. iplane monitors observe on the order of a half million s. M- Lab monitors observe 25K to 500 addresses, and Ripe monitors observe varying number of addresses. Similarly, Ark monitors observe 1.4M to 30K edges, iplane monitors observe 750K to 30 edges, M-Lab monitors observe 30K to 500 edges, and Ripe monitors observe 9K to 1 edge. Similarly, Figure 4 presents the number of traces, sources, s, and edges per destination address. In the sources subfigure, we observe that only iplane sources are targeting all destinations consistently. Similarly, Ripe vantage points are probing majority of destinations except couple of monitors that are lagging. As Ark sources target destinations with groups, only a few sources reach a destination. M-Lab probes show a decreasing trend, which can be attributed to its measurement campaigns being triggered by users. Additionally, in the traces subfigure that counts the unique traces to a destination, we observe that some destinations of M-Lab and Ark receive couple of traces (as reflected in the sources figure). While there are many destinations that are probed only by ark monitors, they discover the fewest number of s and edges per destination (due to the team configuration of Ark). Overall, per destination, Ripe seem to observe the largest graph area as it has much higher vantage points. V. AS COVERAGE In this section, we analyze the AS coverage of the measurement platforms where we perform look up operations over the AS prefix announcement data. Note that we only utilize combined BGP data sets from Table III presents the detailed coverage of each probe data set with respect to BGP data. We observe that not all addresses found in probes are announced by the BGP data sets. CIDR, CAIDA, and 2016, overall have 25.7%, 23.6%, and 11.2% of addresses not announced in their prefixes, respectively. Note that we identified the AS of the uncovered addresses using DNS lookup. Additionally, AS and Prefix indicate the number of covered ASes and Prefixes that had an address in the probe data set. We observe that CIDR has the fewest ASes and prefixes that are observed in the probe data sets. The source row shows that 20% of Ripe vantage points did not show up in BGP prefix announcements but were located using DNS lookup. The destination row shows the total number of destination s within that data set. After performing lookup, we see that Ark has about 100 times more addresses within their traces than the others.

5 Source Edge Trace Fig. 4. Traces, Sources, s and Edges per Destination (log-log scale) Figure 5 presents the coverage of each AS with respect to the number of sources that reached the AS, the number of trace destinations that either reached the AS or crossed the AS, the number of addresses discovered per AS, and the number of edges discovered per AS. Note that the figure ranks 42K observed ASes independent of data source while combined shows the cumulative discovery of measurement platforms for that AS. In the sources subfigure, we observe the AS rank based on the number of vantage points it contains. For instance, there is one AS that has 384 Ripe vantage points in it. Also, Ripe has a large number of probes in the same AS, while Ark has very few probes located in the same AS. In the destination subfigure of Figure 5, we observe that iplane and Ark trace toward a similar number of ASes, while Ripe traces toward very few. The results show that with much Src Dest TABLE III BGP COVERAGE RE M-Lab iplane Ark Announced 4, Missing 1, AS covered Prefix covered 1, Announced , ,645 52,482,541 Missing 11 5,488 12, ,737 AS covered 156 7,165 42,469 46,068 Prefix covered , , ,872 Announced 4, , ,951 48,225,987 Missing 1,207 5,144 31,362 6,133,553 AS covered 1,611 7,261 43,388 45,032 Prefix covered 2,002 39, , ,050 fewer traces iplane is able to match Ark s AS coverage. This indicates iplane is able to produce a good information plane for end-to-end communications. In and edge distribution of ASes, we observe a power-law like distribution, where there are few ASes from which a large number of and edge are observed, while there are a large number of ASes from which a few number of s and edges are observed. Overall, we observe more edges than s per AS. VI. CONCLUSION In this study, we analyzed the public measurement data sets provided by Ant census, Ark, CAIDA, CIDR, iplane, IRL, M-Lab, and Ripe and provide insights on their coverage of the Internet topology. We first analyzed the BGP data sets and compared three resources in their prefix announcements and address coverage. We observed some data sets contain multiple AS announcements for the same address/prefix. Next, we analyzed the probe data sets from five data sources and compared their topological coverage. We observed each data set provided a unique topological perspective. For instance, the edges discovered by different platforms are often unique. We also observed persistent routing anomalies producing various loop configurations, as well as unresponsive routers in the trace data which will require further study. Regardless of the particular study undertaken, the results indicate that the use of multiple data sets is important for building a comprehensive picture of the Internet topology as they each make a unique contribution.

6 Source Edge Destination Fig. 5. AS rank of Destination, Source,, and Edge (log-log scale) When probing, Ark targets a randomized and increasing set of destinations in each cycle, and achieves the most comprehensive coverage of the public measurement platforms we survey. For research focused on AS reachability and node/edge discovery, Ark provides the most diverse traces. lane and MLab use a significantly smaller destination set than Ark, and rely on a much smaller number of vantage points. Ripe also uses a smaller number of destinations, but has a much higher number of vantage points than any of the other three platforms. While offering less coverage than Ark; iplane, MLab, and Ripe would be beneficial in studies focusing on network latency, throughput, and bandwidth measurements. In terms of vantage points, even though Ripe consists of about 60 times more nodes than the others, their measurement campaigns primarily target only Ripe Atlas anchors. As their objective is network reachability, their data is of limited use for topology research. iplane and Mlab both utilize nodes from the PlanetLab network for vantage points, but this system has not been well maintained and has degraded significantly in recent years (from 1100 nodes at its peak, to less than 100 nodes presently). Ark and Ripe, on the other hand, are distributing their own probing devices to the research community and to the public. Unlike the PlanetLab system, these nodes are not available for individual measurement efforts. From the perspective of vantage point distribution, combining the data from all platforms provides a much wider view of the Internet, as all are revealing unique s, edges, and traces. ACKNOWLEDGMENT This material is based upon work supported by the National Science Foundation under grant number CNS REFERENCES [1] M. B. Akgun and M. H. Gunes, Bipartite internet topology at the subnet-level, in IEEE NSW, [2] B. Donnet and T. Friedman, Internet topology discovery: a survey, IEEE Communications Surveys Tutorials, vol. 9, no. 4, pp , Fourth [3] K. Claffy, Y. Hyun, K. Keys, M. Fomenkov, and D. Krioukov, Internet mapping: from art to science, in In IEEE DHS CATCH, [4] H. Kardes, M. H. Gunes, and T. Oz, Cheleby: A subnet-level internet topology mapping system, in COMSNETS, [5] M. H. Gunes and K. Sarac, Analyzing router responsiveness to active measurement probes, in International Conference on Passive and Active Network Measurement, [6] B. Huffaker, M. Fomenkov, and K. Claffy, Internet topology data comparison, [7] University of california san diego, the Ark measurement platform. Available: [8] Ucla internet research lab. Available: [9] Cidr reports.org. Available: [10] Ant censuses of the Internet address space. Available: https: //ant.isi.edu/address/ [11] H. V. Madhyastha, T. Isdal, M. Piatek, C. Dixon, T. Anderson, A. Krishnamurthy, and A. Venkataramani, iplane: An information plane for distributed services, in USENIX OSDI, [12] Measurement lab. Available: [13] Ripe atlas probes. Available: [14] Y. Hyun, Ark topology query system, February [15] J. Heidemann, Y. Pradkin, R. Govindan, C. Papadopoulos, G. Bartlett, and J. Bannister, Census and survey of the visible internet, in ACM SIGCOMM IMC, [16] B. Chun, D. Culler, T. Roscoe, A. Bavier, L. Peterson, M. Wawrzoniak, and M. Bowman, Planetlab: an overlay testbed for broad-coverage services, ACM SIGCOMM Computer Communication Review, vol. 33, no. 3, pp. 3 12, [17] C. Dovrolis, K. Gummadi, A. Kuzmanovic, and S. D. Meinrath, Measurement lab: Overview and an invitation to the research community, SIGCOMM Comput. Commun. Rev., vol. 40, no. 3, pp , Jun [18] X. Zhao, D. Pei, L. Wang, D. Massey, A. Mankin, S. F. Wu, and L. Zhang, An analysis of bgp multiple origin as (moas) conflicts, in ACM SIGCOMM IMW, 2001.