Investigating Containers for Future Services and User Application Support JLAB CNI NLIT 2018 ()
Overview JLAB scope What is a container? Why are we interested? Platform-as-a-Service (PaaS) for orchestration Future Plans 2
JLAB Scope Core Services Predominately RHEL6/7 ~200 web servers (apache/wikis/db) Jupyter Notebooks/JupyterHub (new) Compute Farm CentOS6/7 ~200 nodes 3
What is a Container? A container image is a lightweight, stand alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. An Image is a file, essentially a container snapshot that produces a containerwhen started. Same as VMDK compared to VM. https://docker.com/what-container/ 4
What is a container? Why do it? Containerization is increasingly popular because containers are: Flexible: Even the most complex applications can be containerized. Lightweight: Containers leverage and share the host kernel. Interchangeable: You can deploy updates and upgrades on the fly. Portable: You can build locally, deploy to the cloud, and run anywhere. Scalable: You can increase and automatically distribute container replicas. Stackable: You can stack services vertically and on the fly. https://docs.docker.com/get-started/ 5
What is a container? Lightweight Containers leverage and share the host kernel. Containers vs VMs https://docs.docker.com/get-started/ 6
What is a container? Interchangeable You can deploy updates and upgrades on the fly. 8.5 9.0 jre9 jre10 https://docker.com/what-container/ 7
What is a container? Portable You can build locally, deploy to the cloud, and run anywhere. Host OS dependency is Docker Engine Server dockerd REST API specifies interfaces that programs use to talk to daemon Command line interface (CLI) client docker Security, less is more Host OS can be stripped down Minimal attack vector Less patching involved https://docs.docker.com/get-started/ 8
What is a container? Container Security Kernel namespaces Individual network stacks Creates isolation between host processes and other containers Cross container communication must be specified. Example: web front-end, database back-end Kernel cgroups isolate resource usage Memory share CPU Disk I/O Network Prevents denial-of-service attacks root inside container Whitelisted capabilities Denied mount operations Denied access to raw sockets Denied some file system operations Denied module loading Other restrictions... Content trust Understand where image is coming from Notary options for signing images Image registry scans https://docs.docker.com/engine/security/security 9
What is a container? Architecture https://docs.docker.com/engine/docker-overview Registryis an image store Docker Hub RedHat Registry On premsolution 10
Example Existing Image $> sudo docker run --detach --publish 8080:80 nginx Unable to find image 'nginx:latest' locally latest: Pulling from library/nginx Status: Downloaded newer image for nginx:latest 2296068eda542ec661b8f254756a8f8213f4a542e67e3a871bcd2af98229 NGINX is a web server, can be used as: reverse proxy load balancer HTTP cache CONTAINER ID $> sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 2296068eda54 nginx "nginx -g gdae 'daemon o of " 3seco seconds dsago Up 3 seconds 0.0.0.0:8080->80/tcp80/tcp practical_curie ca 11
Example Custom Image # Dockerfile FROM centos:centos7 Parent image (e.g., latest centos7 image) ENV http_proxy http://jprox.jlab.org:8081 ENV https_proxy https://jprox.jlab.org:8081 Set environmental variables to use our whitelist web proxy, req d for outbound connections WORKDIR /etc/pki/ca-trust/source/anchors RUN curl slo https://pki.jlab.org/jlabca.crt \ && update-ca-trust \ && yum update y \ && yum clean all Install local/jlab certificates Rebuild CA-Trust database Patch base image Remove cached files, helps reduce image size $> sudo docker build --tag jlab/centos7. Fully patched Centos7 $> sudo docker images jlab/centos7 REPOSITORY TAG IMAGE ID CREATED SIZE jlab/centos7 latest 519703e24d68 6 minutes ago 378.2 MB 12
Example # Dockerfile.httpd FROM jlab/centos7 Image Stack # reuse jlab image RUN yum install y httpd # install apache EXPOSE 80 port 80 # listen on CMD [ -D, FOREGROUND ] ENTRYPOINT [ /usr/sbin/httpd ] executable # default args # run as $> sudo docker build -t jlab/httpd. f Dockerfile.httpd Successfully tagged jlab/httpd:latest $> sudo docker run d p 8080:80 jlab/httpd 72deeb856412eb55dc7b4d7941ab81ca6e4e4557240e653df6e4 13
Example Production-ish [Unit] Description=Docker container for GitLab Web UI After=docker.service Requires=docker.service POINTLESS TO USE CONTAINERS THIS WAY Wrapped with systemd Tied to specific host Not scalable [Service] TimeoutStartSec=0 ExecStartPre=-/usr/bin/docker rm gitlab ExecStart=/usr/bin/docker run \ --name gitlab \ --hostname gitlab.jlab.org \ -p 443:443 \ -p 80:80 \ -v /docker/gitlab/config:/etc/gitlab \ -v /docker/gitlab/logs:/var/log/gitlab \ -v /docker/gitlab/data:/var/opt/gitlab \ gitlab/gitlab-ce:latest ExecStop=/usr/bin/docker stop gitlab [Install] WantedBy=multi-user.target 14
PaaS Orchestration Originally designed by Google Provides: Easy scalability Real portability Forced consistency Podis a group of one or more containers, with shared storage/network https://commons.wikimedia.org/w/index.php?curid=53571935 15
PaaS Orchestration Openshift= Kubernetes + security rules + better deployment 16
PaaS App Overview Running on MacOS via minishift& VirtualBox https://github.com/minishift/minishift 17
PaaS Scaling 18
PaaS Routing 19
Deployment Blue-Green Deploy two identical environments Load-balancer points to blue or green, one live and one staged Easy rollback https://opensource.com/article/17/5/colorful deployments 20
Deployment Canary Incremental rollout Route small percentage of users to dev version Gain confidence, route more users Shutdown old version https://opensource.com/article/17/5/colorful deployments 21
Deployment Rolling Openshift default Health check the canary Scale up dev version Scale down old version https://opensource.com/article/17/5/colorful deployments 22
Future Plans Continue to gather intel Communicate with staff & user community What are their expectations? Future needs? Launch full pilot with Openshift/Origin Sort out redundancy (DNS and load-balancers) Test workflows, debugging, scaling A few web servers? JupyterHub? 23
Future plans Jupyter Notebooks Web apps to create and share: live code visualizations documentation Use cases: learning to write code data processing modeling CPU intensive, sometimes Needs scaling & Fast deployment http://jupyter.org/ https://github.com/jupyterhub/jupyterhub deploy docker 24
Future plans JupyterHub Already adapted to container clusters http://jupyter.org/ https://github.com/jupyterhub/jupyterhub deploy docker 25
Questions? Acknowledgements: Marty Wise (JLAB, CNI) Brent Morris (JLAB, CNI) 26