Understanding Your Network Using NetSage

Transcription

1 Understanding Your Network Using NetSage Jennifer Schopf, Ed Balas, Brian Tierney International Networks, Indiana University Sean Peisert LBNL, and UC Davis Jason Leigh Laboratory for Advanced Visualization & Applications, University of HawaiʻI Mānoa Supported by the National Science Foundation

2 Normal day:

3 Monday Feb 5

4 NetSage for NEAAR link Feb 1-5

5 Zoom in: Gbps vs Time

6 Zoom in: Histogram showing sizes of flows

7 Averages (A-Z and Z-A)

8 Max (A-Z and Z-A)

9 So we looked at our flow analysis prototype

10 Prototype tool NetSage, last 5 days, Top talkers by site pair

11 And then I got an from our ESNet colleagues saying:

12 So what is NetSage?

13 IRNC Monitoring: NetSage Open, privacy-aware, network measurement, analysis, and visualization service Designed to address the needs of today s international networks Focus on work with production networks, not on novel research Tasked by NSF to coordinate monitoring work across the IRNC projects

14

15 IRNC Backbones and Exchange Points Backbones: TransPAC4 (Schopf, IU) AmLIght ExP (Ibarra, FIU) Pacific Islands Research and Education Networks (PIREN) (Lassner, U Hawaii) Network for European, American, and African Research (NEAAR) (Schopf, IU) Exchange Points AtlanticWave (Ibarra, FIU) StarLight (Mambretti, NU) Pacific Wave(Fox, CENIC)

16 Current NetSage Targeted Users Project oversight (NSF, etc) Congressional mandate that NSF projects show societal relevance IRNC NOC and other operators Project planning by backbone and exchange point operators Application engagement staff of IRNC projects But this is adaptable

17 High-Level Project Goals Standard measurement and analysis frameworks Open source tools and archive Use of Grafana for analysis and viz to enable extensive customization Canonical TestPoint for basic info gathering PerfSONAR, SNMP, Tstat, Flow, retransmits, etc

18 Tstat Developed as part of the EU mplane (Measurement Plane) project Captures packets to do generate flow data, TCP retransmission analysis, NOT sampled Able to do analysis of 10G TCP flows using <10% of a single core

19 What does NetSage tell you? Today: How used are the links? Where are congestion points? Soon- Flow Data Analysis (mid April): Top ten sites/countries/protocols using my link? What science domains are using the links? Next Identification of problematic flows Alerts for changing behavior, retransmits

20 Today portal.netsage.global

21

22 Loss Pattern (uses perfsonar) (Similar dashboard for latency)

23 Flow data release mid April

24 Top Talkers Could be by number of flows or size Could be organized by Countries Organizations Ports Projects Want to see volume and rate We will never show between 2 IPs NetSage archive doesn t have that data!

25 Top Talkers Source/Destination

26 Zoom Top Talkers Source

27 Then same view for individual pairs (fix one end) by volume Then a view of a single pair over time (rate)

28 Countries

29 Continent

30 Science Registry Goal: Understanding network traffic by domain Map IP addresses to: AS Numbers Universities Project name Science type Seeding with top talkers from TransPAC and NEAAR Joint work with AARnet Community input will be ongoing

31 Global Science Registry (cont) Needed to answer basic usage questions What percentage used by what science domain? How much of this link used by a specific organization? Needs to be easy for science projects to update Requires manual labor

32 Example Registry Record

33 Registry Entry Form

34 Science Domain Top Talkers

35 Next Steps Expanding deployment Working on DTN instrumentation Additional data sources Extension of Science Registry data Possible file transfer data inclusion Additional capabilities Identification of changes in patterns Alarms and alerts

36 Are you interested in NetSage? NetSage is funded by US NSF award # NetSage Portal: Questions? Contact: Jennifer Schopf