The new Docker networking put into action to spin up a SLURM cluster

Transcription

1 Multi-host containerised ss HPC cluster The new Docker networking put into action to spin up a SLURM cluster

2 The Bits and Pieces

3 Docker Containers do not spin up their own kernel All containers share the same host kernel they are separated via Kernel Namespaces constrained using CGroups SERVICE SERVICE SERVICE Userland (OS) Userland (OS) Userland (OS) IB KERNEL KERNEL KERNEL SERVICE SERVICE SERVICE HYPERVISOR Userland Ubuntu:14.04 (OS) Ubuntu:15.10 Userland (OS) Userland RHEL7.2 (OS) Userland (OS) HOST KERNEL SERVER Traditional Virtualisation IB Userland (OS) Tiny Core Linux :6.4.1 HOST KERNEL SERVER Containerisation 3

4 Docker as a Container Runtime creates/starts/stops/manipulates/removes containers - all through RESTful API handles Namespaces, CGroups, IP connectivity, bind-mounts, etc. docker0 container2 container1 eth0 SERVER 4

5 Docker Networking Docker Networking spans networks across engines KV-store to synchronise (Zookeeper, etcd, ) VXLAN to pass messages along (macvlan might take over) global container0 container1 containern DC SERVER0 SERVER1 SERVER<n> 5

6 Docker Swarm Docker Swarm proxies docker-engines serves an API endpoint in front of multiple docker-engines does (simple) placement decisions. :2376 :2376 :2376 swarm-client swarm-client swarm-client swarm-master :2375 SERVER0 SERVER1 SERVER<n> 6

7 Docker Swarm [cont] query docker-engine query docker-swarm 7

8 Introduce new Technologies

9 Introducing new Tech credit: Russian Oil Claim 9

10 Introducing new Tech Self-perception when introducing new tech credit: TF2 - Meet the Pyro 10

11 Introducing new Tech not always the same as the perception of others. credit: TF2 - Meet the Pyro 11

12 Docker Buzzword Chaos! Auto-Scaling Distributions On-Premise & OverSpill Orchestration Solutions self-healing enterprise-grade production-ready 12

13 Pitfalls [incomplete list] You say Docker - perception is basically VMware VMs were easy to shoehorn in legacy workflow, containers might break it spans environments: laptop, dev-cluster, staging, prod Not everyone is a unicorn, I do NOT want special distributions useful for elasticity (AWS) and green-flied deployment, not so much for a on-premise datacenter w/ legacy in it. want to leverage existing <stuff> security (ssh infrastructure), user authentication installation workflow, monitoring, logging want to keep up with upstream docker ecosystem networking, volumes features of docker-engine, -compose, -swarm 13

14 Reduce to the max!

15 Put a Docker on it! Leverage existing install/configuration workflow Kickstart + ansible-run Don t focus on corner cases postpone snowflake-container (e.g. needs multi-tenant IB usage) User namespace: we get there (eventually) HPC environments assumptions single-tenant focus on performance 15

16 The Setup

17 Testbed Hardware (courtesy of ) 8x Sun Fire x2250, 2x 4core XEON, 32GB, Mellanox ConnectX-2) Software Base installation CentOS 7.2 base installation (updated from 7-alpha) Ansible consul, sensu docker v1.9.1, docker-compose docker SWARM 17

18 Docker Networking Synchronised by venus008 venus002 venus001 DC 18

19 Docker SWARM Docker SWARM Synchronised by KV-store venus008 swarm master venus002 venus001 swarm swarm SWARM DC 19

20 SLURM Cluster SLURM within SWARM slurmd within app-container pre-stage multiple container venus008 slurmd slurmctld OpenFOAM slurmd hpcg swarm master spawn at job-start (pre-hook) venus002 slurmd OpenFOAM slurmd hpcg swarm venus001 slurmd OpenFOAM slurmd hpcg swarm SLURM SWARM DC 20

21 SLURM Cluster [cont] 21

22 Samza Cluster Distributed Samza Zookeeper and Kafka cluster Samza instances to run jobs venus008 zookeeper samza kafka swarm master venus002 zookeeper samza kafka swarm venus001 zookeeper samza kafka swarm SWARM DC 22

23 Outta time!

24 SENSU 24

25 Metrics 25

26 Recap aka. IMHO Using vanilla docker tech on-top of any distribution keep up with the ecosystem and prevent vendor/ecosystem lock-in 80/20 rule have caveats on the radar but don t bother too much everything is so fast moving - it s hard to predict Don t scare away stakeholders KISS reuse workflow and infrastructure solution and not problem driven 26

27 Q&A egalea Conference (Pisa) 20/21 April