Doing Science on GENI GENI and NRP

Transcription

1 Doing Science on GENI GENI and NRP Ilya Baldin, RENCI / UNC Chapel Hill With much help from Paul Ruth (RENCI/UNC), Anirban Mandal (RENCI/UNC), Yufeng Xin (RENCI/UNC) and Jeff Chase (Duke CS)

2 Background Director for Network Research and Infrastructure at RENCI/UNC Chapel Hill A group with broad portfolio of projects from NSF (CC, CICI, NeTS, GENI), DARPA, DoE PI on a number of GENI projects Lead PI for ExoGENI testbed Component of GENI infrastructure 2

3 What is GENI? Federated Cyberinfrastructure Many domains, people, users sharing resources Common API for diverse resources Deeply programmable Originally targeted networking experiments 3

4 ExoGENI

5 (Exo)GENI best-kept secret: L2 Stitching Service Slice Client Slice Client Campus Client Slice

6 RENCI s Work Toward Science on GENI Solar Fuels (2011, unfunded) Chemistry ADAMANT (2013, NSF CC-NIE) Adaptive Data-Aware Multi-Domain Application Network Earthquakes (Cybershake), Astronomy (Montage) GENI Science Shakedown (2013, GENI Spiral 4) Storm surge (ADCIRC), Genomics (MotifNetwork) Panorama, Panorama360 (2015 and 2017, DOE) Evaluating performance of exascale applications Workflows, Physics SAFE SuperFacilities (2016, NSF CICI) Creating Dynamic SuperFacilities the SAFE Way High-energy physics SWIP (Scientific Workflow Integrity with Pegasus) (2016, NSF CICI) Developing tools to validate behavior of Pegasus under adverse infrastructure conditions 6

7 Whereit all began: SC11 Demo - Solar Fuels Workflow

8 Pegasus/ExoGENI Collaboration Network Infrastructure-as-a-Service (NIaaS) for workflow-driven compute applications. Tools for workflows integration with adaptive infrastructure (ExoGENI). Workflows triggering adaptive infrastructure Pegasus workflows using ExoGENI Adapt to application demands Integrate data movement into NIaaS On-ramps Target applications Montage Galactic plane ensemble: Astronomy mosaics CyberShake: Probabilistic Seismic Hazard Analysis MapSeq: High-Throughput Sequencing

9 Scaling Science Experiments GENI Science Shakedown GENI Spiral 4 project Project Goals Apply the GENI experiment workflow to domain science applications and evaluate GENI s ability to run domain science experiments (performance and ease-of-use). Build tools for domain scientists to create slices from high-level descriptions of high-throughput and high-performance applications. ADCIRC application

10 GEC22 Plenary Demo Both GENI Testbeds (ExoGENI & InstaGENI) 10

11 The Future: Multidomain Science DOE coined the term Superfacility Definition Two or more existing facilities (e.g. instruments, compute resources, data repositories) using high-performance networks and data management software in order to increase scientific output. Currently manually created Superfacilities are purpose-built manually for a specific scientific application or community. Trust: handshake model Ideally automated Advanced Science DMZs and federated Infrastructure-as-a-Service provide the technical building blocks to construct dynamic superfacilities on demand.

12 Virtual Science Networks for SuperFaclities (CICI SAFE vsdx) Idea: Use circuits to create a private network provisioned to link multiple resources/subnets. Superfacility Testbed slice Campus Cross-campus collaborations Facility access Examples: cluster, LCF, testbed Resource sharing Virtual data enclaves Live network services Large Computational Facility or Instrument Deep network programmability Routing control Security monitoring / NFV Dynamic peering Topology adaptation Elastic edge clouds 12

13 Challenges to building national scale distributed CI Managing a diverse community with different needs Sharing and clearly delineating infrastructure management responsibilities Sites vs. central authority Different models possible Opt-in Make it easy standard hw, packaged sw Make the ROI clear to different constituencies campus IT, researchers Managing expectations Distributed infrastructure reliability Performance Availability 13

14 Possible collaborations with NRP GENI/Chameleon/NRP hero experiments We re ready to help Using GENI to reach campus infrastructure for getting data in and out Using GENI to reach other national resources via dedicated networks Deploying virtual NRP nodes into GENI 14

15 Thank You 15