SLATE. Services Layer at the Edge. First Meeting of the National Research Platform Montana State University August 7-8, 2017

Transcription

1 SLATE Services Layer at the Edge Rob Gardner University of Chicago Shawn McKee University of Michigan Joe Breen University of Utah First Meeting of the National Research Platform Montana State University August 7-8, 2017

2 What is SLATE? New NSF DIBBs award, "SLATE and the Mobility of Capability" Equip the SciDMZ with a service orchestration platform, potentially federated to create scalable, multi-campus science platforms Underlayment-as-a-service (UaaS) for platform builders & science gateway developers 2

3 Ultimate goal: enabling collaborative science XENON Collaboration 3

4 How we got to SLATE lessons 4

5 Early 'platforms': ivdgl, grid3, open science grid, wlcg Deployments of globus software for gatekeepers and storage interfaces, condor for job submission and routing Since 2003 we've led deployments of integration testbeds and supported production services across ~100 institutions 5

6 CERN LHC computing: global grid of data centers ATLAS Tier2 centers led by UMich and UChicago Multi-site grid federations of distributed storage, job routing, software and data caches in production > 10 years Each supports workflows from >1k users from the international collaboration 6

7 perfsonar meshes & data federations Helping campuses deploy perfsonar instances to optimize data throughput (OSG and WLCG) Managed deployment of xrootd federation network of 66 LHC data centers xrootd data federation 7

8 Friction in deployment of distributed services A broken DevOps cycle! - Latency in updates across sites make it extremely difficult to rapidly innovate platform services VO (domain & application) centric - Requires distributed expertise in specialized software stacks across many institutions - Resource providers need to become experts in a range of VO software services 8

9 SLATE Vision

10 10

11 Services layer at the edge A ubiquitous "underlayment" -- the missing shim A generic CI substrate optimized for hosting edge services Programmable Easy & natural for HPC and IT professionals Tool for creating "hybrid" platforms Potentially federate Centralized deployment & operations possible DevOps friendly (for platform and science gateway developers) quick patches, release iterations, fast track new capabilities 11

12 Concepts & components Containerized services in managed clusters Standalone or federated Federated clusters support multi-site platform systems Widely used open source technologies for growth and sustainability 12

13 WP2: Node Specification & Deployment (SPN) API for local and (potentially) remote admin Monitoring & analytics SLATE SciDMZ node -- to be compatible with emerging national specs "Big", "Small", "Central" types 13

14 WP3: Federated Services Example command line interface to service meshes 14

15 Two important challenges (among many) Cybersecurity and policy On premise services, potentially managed by third parties, present additional risks that need to be understood Engagements with CISO from our campuses Engagement with Center for Trustworthy Scientific Cyberinfrastructure Scientific outreach Underlayment to Science Embedded with science collaborations with multi-institution cyberinfrastructure - distributed data services, software, job routing Developers associated with the Science Gateways Community Institute 15

16 SLATE and the NRP A service for science platform builders Easily compatible with FIONA box and current PRP setups We need partners & volunteers: to help inform technology choices to confront and solve policy & cybersecurity issues focus on usability & integration in HPC environments usability for science gateway developers & distributed service deployment teams align with emergent national SciDMZ patterns 16

17 Timeline - first year Testbed spec and deployment Service orchestration technology evaluations Early multi-site pilot demonstrator with: stateless services stateful services non-trivial HPC resource integration Early federated platform tools Recruit volunteers and build community 17

18 Summary SLIDES: Reduce barriers to supporting collaborative science Give science platform developers a ubiquitous "CI substrate" Change distributed cyberinfrastructure opertional practice by mobilizing capabilities in the edge Achieve scale through multi-site service meshes: ensembles of federated, orchestrated services 18

19 Thank you! contact 19