ADAC Federated Testbed Creating a Blueprint for Portable Ecosystems

Transcription

1 ADAC Federated Testbed Creating a Blueprint for Portable Ecosystems Sadaf Alam, Jeffrey Vetter, Mark Klein, Maxime Martinasso, ExCL ORNL,... ADAC Workshop February 15, 2018

2 January, 2016 June, 2016 January, 2017 July, 2017 Motivation and Use Cases ADAC 1-4 workshops Performance modeling Performance tools Emerging technologies Emerging technologies & testbeds

3 ExCL and CSCS experimental testbeds TokyoTech, ORNL LCCF and CSCS evaluation systems

4 Bridging Gaps & Accelerating Innovations Why? No ADAC Experimental Testbeds Computer science research topics e.g. kernel, runtime, tools, etc. Now what? T2 T1 Production platforms ADAC Benchmarking Testbeds Applications, libraries, and algorithm development and tuning

5 Summary: Motivation & Use Cases Enable evaluation of full ecosystem for emerging technologies Share testbed resources Including experiments requiring privileged access Hardware level testing Memory and storage hierarchies Inventory of resources Federated access Coordinating installation of new hardware Share benchmarking and evaluation results Portfolio of results Collection of tools with support matrix Monthly calls + topic related discussions

6 Inventory (Work in Progress)

7

8 Inventory: CSCS Experimental Testbed Existing infrastructure Traditional with Multiple generation Xeon based systems with GPUs InfiniBand and OPA interconnect and HCAs Power 8 ARM8 nodes Upcoming AMD CPUs and GPUs Intel FPGAs SSD storage such as To be announced...

9 Inventory: Evaluation and Benchmarking Platforms ORNL NCCS Open Resources (2-factor authentication) a. b. c. d. ARM1 system HPE-ARM system (system name unknown still) Cray XC40 Knights Landing (Percival) 168 nodes IBM Minsky (P8+/Pascal) (Summitdev) 54 node TokyoTech a. TSUBAME KFC

10 Challenges: Technical & Policy

11 Challenges Policy: Governance and management Application process Authentication and authorization Access policies Technical: System provisioning and scheduling (isolation and security) Innovative hardware ecosystem Virtualization Metal-as-a-service Access to external environments e.g. quantum systems thru cloud

12 Management of Privileged Access 1 Changes to installed compilers and user tools. 2 3 Changes to OS, Kernel level experimentation Application level benchmarking Changes to installed drivers, low level power measurements 4 5 Pri v (ma ilege nag d ac c ed by ess a n tes tbe d con dp rov trol ide rs) Hardware & firmware changes ) ng ineers i t r o rep d eng d an stbe g e n i tor d by t i n Mo nage (ma

13 Lightweight Application Process for Accessing Testbeds

14

15 ORNL XCAMS & CSCS account application is similar

16 Architecture Addressing Experimental (incl. NDA) and Evaluation Platform needs

17 ExCL 2.0 (ORNL) Exclusive access to machines in this cluster. These nodes are only available once a VM has been launched from the web portal Exclusive Access Cluster Management Server Web portal for bare-metal and gateway VMs management Bare-Metal Node Type A Bare-Metal Node Type A Bare-Metal Node Type A Bare-Metal Node Type B Bare-Metal Node Type C Bare-Metal Node Type D Gateway Gateway Gateway IP/key based restricted access Login Node Special HW 1 Special HW 2 Virtual login nodes on top of VMs (not bare-metal). Only to ensure that 1 user is going to access other HW from here at a time. Gateway machines can have associated metadata to make them unique. Dedicated login node. Creates the idea of an integrated system Hardware where virtualization is not possible. Only accessible from gateways Shared Node Type A Shared Node Type B Compute Nodes Hardware not located at ORNL. ie. quantum system Special HW External Same type of HW available in bare-metal, but shared. No VM needed. Direct access from login node. Multiple concurrent users

18 CSCS Testbed Architecture Hardware connected to Monitored PDUs OpenStack based bare-metal deployments (for power measurements) Operation default for most nodes is shared access (slurm w/container support). Allowed users can re-provision a server into a dedicated resource for various test cases: VMs, Containers, or Bare Metal deployments. Clean up on release & return to shared mode. Users to be able to provide own images for dedicated modes, with some pre-made available for ease of use. NDA hardware can be hidden from certain users/groups (not operated in Shared mode, only available for Dedicated Provisioning)

19 Proposal for Federated AAI Ease of Access & Sharing of Resources

20 Concept Similar to OpenID Connect

21 ADAC Federation with Trusted Identities ADAC User and Project DB ORNL ExCL and CSCS users initially Proof of concept for web based technologies CSCS OpenStack Platform Identity brokering with KeyCloak 1. Access ORNL registered user 4. Authorized to access CSCS service ORNL IdP 2. Select ORNL IdP for authentication 3. Redirected to ORNL

22 Status & Next Steps Finalize federated AAI and central services decisions Introduce a landing page for ADAC testbeds Testing and implementation of Keycloak ADAC user and proj DB central management Links to other sites Inventory Tune testbed implementation for use cases Enable HPC features Metal-as-a-service Verify functionality of OpenStack services Bare Metal (Ironic) VMs (Nova) Container Orchestration (Magnum) Resource Reservation (Blazar) Vendors engagement is critical

23 DOE Workshop on Extreme Heterogeneity Jan 2018

24

25 Status Gov shutdown forced cancellation of physical meeting Moved to virtual meeting Kept to original agenda (with some minor changes for timezones) Approximately 200 participants viewed plenary sessions!! Breakout groups converged on priority research directions

26 Status Initial Priority Research Directions (Categories) Programmability and Software Development Productivity Managing Execution, Scheduling Correctness, Debugging, Reproducibility Modeling and Simulation for Performance, Power, Resiliency Deliverables Slides w/ PRDs be the end of the workshop / Done Brief report for HQ by Mar 1 Final public report by May 1