DYNES: DYnamic NEtwork System

Transcription

1 DYNES: DYnamic NEtwork System Artur Barczyk California Institute of Technology / US LHCNet TERENA e2e Workshop Prague, November 29 th,

2 OUTLINE What is DYNES Status Deployment Plan 2

3 DYNES Overview What is DYNES? A U.S-wide dynamic network cyber-instrument spanning ~40 US universities and ~14 Internet2 connectors Extends Internet2 s dynamic network service ION into U.S. regional networks and campuses; Aims to support LHC traffic (also internationally) Based on the implementation of the Inter-Domain Circuit protocol developed by ESnet and Internet2; Cooperative development also with GEANT, GLIF Who is it? Collaborative team: Internet2, Caltech, Univ. of Michigan, Vanderbilt The LHC experiments, astrophysics community, WLCG, OSG, other VOs The community of US regional networks and campuses What are the goals? Support large, long-distance scientific data flows in the LHC, other programs (e.g. LIGO, Virtual Observatory), & the broader scientific community Build a distributed virtual instrument at sites of interest to the LHC but available to R&E community generally 3

4 DYNES Team Internet2, Caltech, Vanderbilt, Univ. of Michigan PI: Eric Boyd (Internet2) Co-PIs: Harvey Newman (Caltech) Paul Sheldon (Vanderbilt) Shawn McKee (Univ. of Michigan) 4

5 The Problem to be Addressed Sustained throughputs at 1-10 Gbps (and some > 10 Gbps) are in production use today by some Tier2s as well as Tier1s LHC data volumes and transfer rates are expected to expand by an order of magnitude over the next several years As higher capacity storage and regional, national and transoceanic 40G and 100 Gbps network links become available and affordable. Network usage on this scale can only be accommodated with planning, an appropriate architecture, and nationwide and Int l community involvement by The LHC groups at universities and labs Campuses, regional and state networks connecting to Internet2 ESnet, US LHCNet, NSF/IRNC, other major networks in US & Europe Network resource allocation and data operations need to be consistent DYNES will help provide standard services and low cost equipment to help meet the needs 5

6 Addressing the The Problem with Dynamic Network Circuits DYNES will deliver the needed capabilities to the LHC, and to the broader scientific community at all the campuses served, by coupling to their analysis systems: Dynamic network circuit provisioning: IDC Controller Data transport: Low Cost IDC-capable Ethernet Switch; FDT Server for high throughput, h t Low cost storage array where needed d (also non-lhc) End-to-end monitoring services DYNES does not fund more bandwidth, but provides access to Internet2 s dynamic circuit network ( ION ), plus the standard mechanisms, tools and equipment needed To build circuits with bandwidth guarantees across multiple network domains, across the U.S. and to Europe in the future In a manageable way, with fair-sharing Will require scheduling services at some stage To build a community with high h throughput h capability, using standardized, di d common methods 6

7 Leveraging on the Internet2 Dynamic Circuit Network 7

8 DYNES System Description AIM: extend hybrid & dynamic capabilities to campus & regional networks. A DYNES instrument must provide two basic capabilities at the Tier 2S, Tier3s and regional networks: 1. Network resource allocation such as bandwidth to ensure transfer performance 2. Monitoring i of the network and data transfer performance All networks in the path require the ability to allocate network resources and monitor the transfer. This capability currently exists on backbone networks such as Internet2 and ESnet, but is not widespread at the campus and regional level. In addition Tier 2 & 3 sites require: 3. Hardware at the end sites capable of making supports: one Tier2 - Tier3 and optimal use of the available network resources another Tier1-Tier2. Two typical transfers that DYNES The clouds represent the network domains involved in such a transfer. 8

9 DYNES: Regional Network - Instrument t Design Regional networks require 1. An Ethernet switch 2. An Inter-domain Controller (IDC) The configuration of the IDC consists of OSCARS, DRAGON, and perfsonar. This allows the regional network to provision resources on-demand through interaction ti with the other instruments A regional network does not require a disk array or FDT server because they are providing transport for the Tier 2 and Tier 3 data transfers, not initiating them. At the network level, each regional connects the incoming campus connection to the Ethernet switch provided. Optionally, if a regional network already has a qualified switch compatible with the dynamic software that they prefer, they may use that instead, or in addition to the provided equipment. The Ethernet switch provides a VLAN dynamically allocated by OSCARS & DRAGON. The VLAN has quality of service (QoS) parameters set to guarantee the bandwidth requirements of the connection as defined in the VLAN. These parameters are determined by the original circuit request from the researcher / application. through this VLAN, the regional provides transit between the campus IDCs connected in the same region or to the global IDC infrastructure. 9

10 DYNES: Tier2 and Tier3 Instrument t Design Each DYNES (sub-)instrument at a Tier2 or Tier3 site consists of the following hardware, combining low cost & high performance: 1. An Inter-domain Controller (IDC) 2. An Ethernet switch 3. A Fast Data Transfer (FDT) server. Sites with 10GE throughput capability will have a dual-port Myricom 10GE network interface in the server. 4. An optional attached disk array with a Serial Attached SCSI (SAS) controller capable of several hundred MBytes/sec to local storage. The Fast Data Transfer (FDT) server connects to the disk array via the SAS controller and runs FDT software developed by Caltech. FDTi is an asynchronous multithreaded lih dsystem that automatically adjusts I/O and network buffers to achieve maximum network utilization. The disk array stores datasets to be transferred among the sites in some cases. The FDT server serves as an aggregator/ throughput optimizer in this case, feeding smooth flows over the networks directly to the Tier2 or Tier3 clusters. The IDC server handles the allocation of network resources on the switch, interactions with other DYNES instruments related to network provisioning, and network performance monitoring. The IDC creates virtual LANs (VLANs) as needed. 10

11 Local Connectivity Options DYNES offers several connectivity options for the local sites and the RONs. Two examples: Campus using 1 of 2 connections to the regional for DYNES and the other for general purpose IP connectivity Connector and campus incorporating DYNES as part of production infrastructure 11

12 Fast Data Transfer (FDT) DYNES instrument includes a storage element, with FDT as data transfer application FDT is an open source application for efficient data transfers Easy to use: similar syntax with SCP, iperf/netperf Witt Written in java and runs on all major platforms Single.jar file (~800 KB) Based on an asynchronous, multithreaded system Uses the New I/O (NIO) interface and is able to: stream continuously a list of files use independent d threads to read and write on each physical device transfer data in parallel on multiple TCP streams, when necessary use appropriate size of buffers for disk IO and networking resume a file transfer session 12

13 FDT Main Features User defined loadable modules for Pre and Post Processing to provide support for dedicated Mass Storage system, compression, dynamic circuit setup, Pluggable file systems providers (e.g. non-posix FS: FDT/Hadoop, etc.) Dynamic bandwidth allocation Using IDC API Limiting transfer rate on end-host Different transport strategies: blocking (1 thread per channel) non-blocking (selector + pool of threads) On the fly MD5 checksum on the reader side Configurable number of streams and threads per physical device (useful for distributed FS) Automatic updates Can be used as network testing tool (/dev/zero /dev/null memory transfers, or nettest flag) 13

14 DYNES SC10 Demo Successful demonstration setup between Internet2, Caltech and Vanderbilt booths, through SCinet FDT disk-disk throughput on 5 Gbps ION channel (1 host/booth) 5 4 FDT used to automatically request circuit provisioning between booths [Gbps s]

15 DYNES Roadmap and Status ICO and Council notification (August, 2010) Briefings to AOAC, RAC, and ICO Community announcement Internet2 Letter, and Website (August, 2010) 2 Community Calls Held (August 5th and August 17th, 2010) Set up external review panel (September, 2010) Call for Proposals (September, 2010) Calls with individual regionals and with individual campuses (plus associated regional network) upon request DYNES BoF (Internet2 Fall MM in Atlanta in Nov.) Site applications deadline December 15th, 2010 Selection done/announcements by January 31st, 2011 Distributed virtual Instrument building begins in

16 THANK YOU! 16