Building a Digital Bridge from Ohio

Transcription

1 Building a Digital Bridge from Ohio Marcio Faerman, PhD and Alex Berryman Internet2 Global Summit April 8 th, 2014 Slide 1

2 The OH-TECH Consortium Ohio Supercomputer Center provides high performance computing, software, storage and support services for Ohio s scientists, faculty, students, businesses and their research partners. OARnet connects Ohio s universities, colleges, K-12, health care and state and local governments to its high-speed fiber optic network backbone. OARnet services include co-location, support desk, federated identity and virtualization. OhioLINK serves nearly 600,000 higher education students and faculty by providing a statewide system for sharing 50 million books and library materials, while aggregating costs among its 90 member institutions. estudent Services provides students increased access to higher education through e-learning and technology-enhanced educational opportunities, including virtual tutoring. Research & Innovation Center will operate, when opened, as the proving grounds for next-generation technology infrastructure innovations and a catalyst for cutting-edge research and collaboration. Slide 2

3 Context and Motivation NSF OSU CC-NIE Science-DMZ Network Integration with Enterprise Network and University Policies The Performance Engineer OSU International Gateways New OSU Gateway in Brazil Brazil invests in Science DMZ architecture RNP Working Group Competition LARC/USP leads Science DMZs in Brazil Tereza Carvalho, Fernando Redigolo, Dino Magri Deployment in Brazilian Institutions Leverage AmLight Local to Global National to International Slide 3

4 Science DMZ Design at OSU Two Connection Models: Low Friction Layer 3 Layer 2 Circuit (AL2S) Component: Cisco Nexus 7000 (100Gig) NEC PF5820 (40Gig) Campus OpenFlow Switches (10Gig) Data Transfer Node (10Gig) Slide 4

5 Researcher Network-as-a-Service Dashboard Researcher uses Shibboleth credentials to define Application and end points. Researchers also classify the data s privacy type. After approval, Performance Engineer pushes endpoint info into a RESTful API that is polled by OpenFlow controller. Real-time software defined network monitoring of critical network parameters for bottleneck identification and troubleshooting Slide 5

6 Multi-physics Use Case (Low Friction Layer 3) Researchers in OSU s ME Department move large files to and from HPC at WPAFB in Dayton, OH. Problems: 1. Transfers of 80Gb files take over 12 hours 2. Hosts are using shared and firewalled 1Gig connection 3. The DREN path was not being correctly advertised to OSU by OARnet and was going all the way to Los Angeles 4. Researchers are using SCP Planned Actions: 1. Use OpenFlow to switch research traffic onto a non-firewalled dedicated fiber to OSU core 2. Correct BGP Peering so OSU uses OARnet s direct connection to DREN 3. Still using SCP due to lack of control on DoD HPC side Transfers currently take under two hours Slide 6

7 Multi-physics Topology (Low Friction Layer 3) Multi-physics Topology Scott Labs Student Computer All Traffic HP Campus Openflow Traffic approved for DMZ? ME Dept. Firewall OSU Central IT Non-Research traffic Research traffic (Dotted) Science DMZ 1Gb 10Gb 100Gb OARnet Corrected Peering DAY DREN AFRL spirit.afrl.hpc.mil WPAFB, Dayton, OH Incorrect Peering Internet2 LAX DREN Slide 7

8 KernelSpace TCP Streams (Layer3) TCP Socket s TCP/IP Reliabili ty Flow Control Conge st ioncont rol EthernetDrive r EthernetAdapt er KernelSpace TCP Streams (Layer3) TCP Socket s TCP/IP Reliabili ty Flow Control Conge st ioncont rol EthernetDrive r EthernetAdapt er Biomedical Use Case (Layer 2 Circuit) Imaging Microscope Kernel Space TCP Streams TCP Sockets TCP/IP Flow Reliability Control Ethernet Driver Legend: Raw Image Congestion Control Ethernet Adapter IP Network Processed Image MPI Traffic ADTS Receiver/ Sender RDMA Support? No Yes No Yes RDMA Support? ADTS Receiver/ Sender Image Processing Cluster Render Display User Space RDMA Streams Verbs API RDMA RoCE Adapter TCP/IP Flow Reliability Control RoCE Driver Non-IP Network Congestion Control Transfer Neuroblastma cell images from MU to be processed at OSU using custom ADTS tool: Supports RDMA over converged Ethernet (RoCE) Falls back to TCP based transfers if necessary Policy: Matched based on MAC address and Ethernet type Requirements: Layer 2 Connection RDMA compatible network cards 10Gig bandwidth Lowest latency possible Slide 8

9 CEMAS Use Case (Layer 2 Circuit) The Center for Electron Microscopy and AnalysiS wants to share remote access to Electron microscopes using a vendor provided solution Remote control from lecture hall on campus Remote control from other sites in Ohio connected to OARnet (WBAFB, other Universities) These end-points are manufactured single purpose boxes, unlike the first use case we discussed Policy can be defined based on VLAN Tag Requirements: Layer 2 Traffic Low jitter tolerances High continuous bandwidth Slide 9

10 CEMAS Topology CEMAS Topology CEMAS, Kinner Rd Electron Microscope OSU Campus Class Remote Access Box Remote Access Box Layer 2 Layer 2 NEC PF5820F Openflow Cisco Nexus Gb 100Gb OARnet DAY DREN AFRL Remote Access Box WPAFB, Dayton, OH Slide 10

11 Science DMZ Security Real time utilization monitoring using SFlow If there is suspicious traffic the individual flow can be mirrored out to a Bro cluster for deeper packet analysis Maintain an Authentication and Authorization paper trail of policies defined in the portal for auditing purposes. Slide 11

12 Building a Digital Bridge to the Last Mile Across the Globe Ohio-Brazil Multi-Domain Challenges CCE-USP ANSP RNP AmPath Internet2 OSU SDN Multi-domain virtualization End-to-End Connection with Programmability at the Edges Interoperation of software defined clouds End-to-End Sliceable Connection with Centralized Programmability The multiple NOCs and Humans in the loop Where and Who is going to solve my problem? Distributed Responsibility Slide 12

13 Use Cases Ohio-Brazil The Matchmaking Climate High Energy Physics Virtual Collaboration Environments for Biomedics Arts and Dance Remote Instrumentation Energy Agriculture Forensics Slide 13

14 Matchmaking Lessons Learned Match Researchers Timing, Scope and Funding The HEP Inner Tier versus Up-Down Hierarchy Communication Models Climate Match our Science before our Cyberinfrastructures It is not only about Throughput Delay and Jitter sensitive requirements Last Mile and/or Classical Data Transfer Nodes Operation Room Network cannot go down! WiFi for Collaborative Environments Science DMZs at Art Venues Measurements Requested! PerfSONAR all the way Slide 14

15 Questions Marcio Faerman Alex Berryman Like Us on Facebook: Kinnear Road Columbus, OH Phone: (614) Slide 15