Introduction to. Network Startup Resource Center. Partially adopted from materials by

Size: px

Start display at page:

Download "Introduction to. Network Startup Resource Center. Partially adopted from materials by"

Antonia Matthews
6 years ago
Views:

1 Introduction to Network Startup Resource Center Partially adopted from materials by These materials are licensed under the Creative Commons Attribution-NonCommercial 4.0 International license ( Last updated 18 th October 2016

2 Overview What problem are we trying to solve? What is perfsonar? The core subsystems Acronym busting perfsonar testpoint-bundle A few practical examples bwctl vs. pscheduler

3 Problem Statement The global Research and Education network ecosystem is comprised of hundreds of international, national, regional and local-scale networks.

4 Problem Statement While these networks all interconnect, each network is owned and operated by separate organizations (called domains ) with different policies, customers, funding models, hardware, bandwidth and configurations.

5 Problem Statement In practice, performance issues are prevalent and distributed. When a network is underperforming or errors occur, it is difficult to identify the source, as problems can happen anywhere, in any domain. Local-area network testing is not sufficient, as errors can occur between networks.

6 Problem Statement Hard vs Soft Failures Hard failures are the kind of problems every organization understands Fiber cut Power failure takes down routers Hardware ceases to function Classic monitoring systems are good at alerting hard failures i.e., NOC sees something turn red on their screen Engineers paged by monitoring systems Soft failures are different and often go undetected Basic connectivity (ping, traceroute, web pages, ) works Performance is just poor How much should we care about soft failures?

7 What does it take to transfer data...

8 But, just a little packet loss...

9 A Solution perfsonar and the perfsonar Toolkit perfsonar is a network measurement toolkit that provides end-to-end statistics and helps to establish endto-end usage expectations. How fast is our circuit in reality? Are there problems? How do I fix problems? perfsonar provides a uniform interface that allows for the scheduling of measurements, storage of data in uniform formats, and scalable methods to retrieve data and generate visualizations to help answer the three questions above.

10 perfsonar from above Uses multiple network testing tools (many can be run on the command line). Wraps them to both abstract and schedule them. Can save results of tests to optimized database formats PostgreSQL Cassandra Manages archiving of measurements (esmond) Can display results graphically (perfsonar Toolkit) or Results can be sent to a central store and correlated on a grid (MaDDash)

11 perfsonar backend What do we want to measure? Latency: ping, OWAMP (don t trust Traceroute) Jitter: OWAMP, UDP iperf, iperf, UDP, nuttcp Packet loss, duplication, unordered data: OWAMP, UDP iperf, UDP simplestream, nuttcp Interface utiliization, discards, errors: - SNMP Achievable bandwidth: iperf(2 & 3), nuttcp Traveled route: traceroute, tracepath MTU feedback: tracepath, ping

12 perfsonar version 4 Some transitions: bwctl (bandwidth test controller) migrating to pscheduler. Preferred, but both available. perfsonar Toolkit 4.0.x still uses bwctl. 4.1 will use pscheduler. No more NPAD (Network Path & Application Diagnostics) No more NDT (Network Diagnostic Tool) Both at

13 iperf3 example Standalone iperf3 c remote_host i 2 u b 500M bwctl bwctl -T iperf3 -c receive_host -s send_host -i 2 -u -b 500M pscheduler pscheduler task --tool iperf3 throughput --dest receive_host --interval 2 -u --bandwidth 500M b = bandwidth i = interval to view output in seconds T = tool u = udp

14 Benefit: Multi-domain Insight Approximately 2200 public perfsonar nodes are deployed globally This allows you to test performance between your perfsonar node and any other publicly available perfsonar node.

15 The bits and pieces perfsonar version 4 is Distributed on CentOS and Debian Can be installed for various uses: Tools Basic command line clients Test Point Core Automate tests, join a mesh, join the global directory Esmond measurement archive software Toolkit (usually via ISO) Web interface, scheduling, system-wide tuning and security scripts Central Management Separate and centrally manages multiple perfsonar hosts

16 The bits and pieces flowchart

17 Where our bundle fits

18 Sample perfsonar site

19 Summarized Historical Trends

Benefits: Finding hidden problems Above all, perfsonar allows you to maintain a healthy, high-performing network because it helps identify the soft failures in the network path.

20 Benefits: Finding hidden problems Above all, perfsonar allows you to maintain a healthy, high-performing network because it helps identify the soft failures in the network path. Classical monitoring systems have limitations Performance problems are often only visible at the ends Individual network components (e.g. routers) have no knowledge of end host state perfsonar tests the network in ways that classical monitoring systems do not More perfsonar distributions equal better network visibility.

21 Benefits: Finding hidden problems perfsonar is designed to pinpoint and identify soft failures to accelerate resolution. Example: Find and replace failing optics:

22 Benefits: Finding hidden problems perfsonar is designed to pinpoint and identify soft failures to accelerate resolution. Example: Host Tuning:

23 Benefits: Finding hidden problems Inside the firewall l l One direction severely impacted by firewall Not useful for science data Outside the firewall l Good performance in both directions

24 Firewalls and throughput

25 In summary perfsonar helps you to test data and metrics verifies performance of current investments Can you get full use out of the carrier circuit you re paying for? If performance suddenly changes, what caused it? Data informs future investments Put resources in places with demonstrated need Example: if your firewall is limiting performance, buying more bandwidth won t solve the problem Make the case for architectural improvements How might a Science DMZ or alternate architecture change performance? How to evaluate potential purchases? Put perfsonar hosts on them and run tests Prototype new model

26 Resources Online perfsonar training videos

27 perfsonar Website perfsonar Documents Resources perfsonar mailing lists perfsonar Directory of Users perfsonar Online Training Videos FasterDate Knowledgebase (Energy Sciences Network) Esnet Science DMZ

28 Resources MaDDash Globus GridFTP perfsonar at ES.net pscheduler and bwctl to pscheduler transition guide perfsonar Checklist

29 Primary perfsonar Development For a full list of development collaborators and history of perfsonar, please visit:

30 Questions?

HIDDEN SLIDE Summary These slides are meant to be used as is to give an upper level view of perfsonar for an audience that is not familiar with the

HIDDEN SLIDE Summary These slides are meant to be used as is to give an upper level view of perfsonar for an audience that is not familiar with the concept. You *ARE* allowed to delete things you don t