Think Small to Scale Big Intro to Containers for the Datacenter Admin Pete Zerger Principal Program Manager, MVP pete.zerger@cireson.com Cireson Lee Berg Blog, e-mail address, title Company
Pete Zerger Lee Berg @PZerger @LeeAlanBerg 10-Time Microsoft MVP Microsoft MVP 18 Years 10 Years Foods (and people) of the world Internet-of-Things
Goals Familiarize you with container concepts Demonstrate some real world scenarios Explain WHY you should invest time even as a Microsoft-centric administrator
Agenda Container Basics Docker Deep Dive High Availability & Performance Orchestration for Ops Management and Monitoring MS Integration with Docker Resources & Next Steps
Container Basics Windows Containers & Docker
Containers vs Virtual Machines result is significantly faster deployment, much less overhead, easier migration, faster restart How are they different? Containers share a single OS kernel Containers tend to lock you into a single OS When to use containers? When you want to run multiple instances of an app When to use VMs? When you want to run multiple apps
Windows Containers Can be managed with PowerShell or the Docker client Can run in on-premises VMs (any hypervisor) or Azure Today, Docker has richer capabilities than PowerShell for Windows container management //Build 2016: MS announced deprecation of current preview PowerShell module and replacing with a new PowerShell module for Docker.
Demo Deploying & Managing Windows Containers
Docker Deep Dive
Agenda Level 1 Level 2 Level 3
Docker allows you to package an application with all of its dependencies into a standardized unit for software development. Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code runtime system tools system libraries anything you can install on a server This guarantees that it will always run the same, regardless of the environment it is running in.
Docker for Windows What it is It s Docker as you know it Same user experience as on Linux It s Windows as you know it Complete environment inside the container It s native containers Contained processes run on the host system What it s not It s not virtualization Docker for Windows will not run Linux Images It s not a different project / code base The existing Docker tree was ported It s not quite finished Requires Windows Server 2016 (current TP5) It s available for testing
Docker Images, Registries & Containers Images are read-only templates. Images are used to create Docker containers. There are public or private stores from which you upload or download images. Think A Container of Docker Image Hub is as to Github a Container for Docker as a VM Template Container is Images to a VM Containers are similar to a directory. A Docker container holds everything that is needed for an application to run. Registries are stateless, scalable server side application that stores & lets you distribute Docker images. Docker Hub (SaaS, in the cloud) Trusted Registry (you manage, on-prem) Machine, Compose and Swarm work together. They are packaged together in the Docker Toolbox.
Networking for containers and hosts Network Address Translation Mode Each container is connected to an internal virtual switch Receives an internal IP address. NAT (port mapping) translates internal IP of container to IP of container host Transparent Mode Each container is connected to an external virtual switch Receives an IP Address from a DHCP server.
Docker Machine automates container host provisioning on your network or in the cloud. Install Machine on Windows, Mac OS X, or Linux. Tools & Terms Docker Compose defines multi-container applications. You can install Docker Compose on Ubuntu, Mac OS X, and other systems. Docker Swarm is used to host clustering and container scheduling. Machine, Compose and Swarm work together. They are packaged together in the Docker Toolbox.
Demo Exploring the Docker Registry
Images & Layers Each Docker image references a list of read-only layers that represent filesystem differences Layers are stacked on top of each other to form a base for a container s root filesystem Docker storage driver is responsible for stacking these layers and providing a single unified view. When you create a new container, you add a new, thin, writable layer on top of the underlying stack (the container layer ) You Machine, can see Compose the image and Swarm layers work when together. you They download are packaged an together image in from the Docker the Docker Toolbox. Hub
What s in a container? (Namespaces) Namespaces provide processes with their own view of the system. Also limits what process can see (and therefore use). pid. isolate the process ID number space. Processes in different PID namespaces can have the same PID net. provide isolation of the system resources associated with networking mount. isolate the set of filesystem mount points seen by a group of processes. uts. allows each container to have its own hostname and NIS domain name ipc. isolate certain interprocess communication (IPC) resources user. isolate the user and group ID number spaces
Demo Deploying a Docker Container from an Image
High Availability & Performance Clustering & Scale
Docker Swarm Challenge: Containers run on a single host, making scale operations and centralized management challenging. Solution: Docker Swarm enables multiple container hosts to behave like a single, virtual host. Manager. Manager (or Master) is a pre-defined Docker Host, that serves as a single point for all administration. Nodes. Additional Docker hosts (e.g. Ubuntu VMs) to which containers are deployed Node Agent. registers the referenced Docker daemon, monitors it, and updates the discovery service with the node s status. The Docker Swarm API is compatible with the Docker remote API, and extends it with some new endpoints.
Discovery & Scheduling Node Discovery Services Swarm uses hosted discovery service, based on Docker Hub, using tokens to discover nodes that are part of a cluster. etcd, consul, and zookeeper can be also be used for onprem service discovery (for Internet-restricted networks). Schedulers and Scheduler Strategies Determines the best node on which to run your container Several strategies to pick the best node are available (bin, spread and random) Spread is the default, which optimizes by least number of running containers
Docker Client with and without Swarm Without Swarm With Swarm Talks to the individual container hosts (nodes) directly. Talks to the Swarm Manager which communicates with nodes The transition is seamless because the Docker tools (Machine, Compose, Swarm) use the Docker Remote API
DEMO: Hybrid Docker Swarm Cluster docker Docker run docker -d Swarm -p run 3376:3376 -t swarm manage -H 0.0.0.0:3376 -- tls=false docker run -d -d does swarm not join --addr=10.0.0.5:2376 run --rm join differentiate Export DOCKER_HOST=tcp://0.0.0.0:3376 swarm --addr=10.0.0.6:2376 host OS We create token://b4b6180b7b7e4e8f7c4e5585500b64e4 can build this intelligence using labels Each node registers it s docker daemon with the hosted discovery service WIN-NODE-1 Windows Container Host LIN-NODE-1 Linux Container Host Join Windows node to the Swarm Swarm node agent Swarm node agent Join Linux node to the Swarm Sends instructions to Swarm manager, as thought it s a standard container host swarm-master-1 Swarm Manager win1 Docker Client TCP 3376 Swarm node agent Establish the Swarm
Demo Creating a Docker Swarm (Cluster)
Orchestration for Ops with Docker Compose
Docker Compose Compose is an orchestration tool for defining and running multi-container Docker applications. With Compose, you use a Compose file (YAML) to configure your application's services. Using a single command, you then create and start all the services from the configuration in your Compose file. Compose runs on OS X and 64-bit Linux. Currently not supported on Windows.
A Sample Compose (YAML) File Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using Docker build, users can create an automated build that executes several command-line instructions in succession. wordpress: image: wordpress links: - db:mysql ports: - 8080:80 Compose file is a YAML file defining services, networks and volumes Define dependencies, like DNS and DBs May reference multiple dockerfiles. db: image: mariadb environment: MYSQL_ROOT_PASSWORD: 'P@ssw0rd1!' depends_on will not wait for db dependent images (redis & db) to be ready, only to be started.
Demo Creating a multi-container app with Docker Compose
Load Balancing Containers Docker Containers Options for load balancing include: HAProxy a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. Available in a Docker image NGINX Widely used open source web server Can act as a reverse proxy for HTTP and HTTPS, as well as e-mail related protocols Available in a Docker image In ACS, you simply use an Azure load balancer!
Monitoring & Management of hosts & container images
Monitoring and Logging Docker Stats Sort and display container resources utilization and performance stats Container Logging Json-file (default), Syslog, Journald, Gelf, Fluentd and splunk Event Stream Using the Docker API, we can get real-time notifications of everything happening in the Docker engine. cadvisor Using the Docker API, we can get real-time notifications of everything happening in the Docker engine. MS Ops Mgmt Suite (OMS) Using the Docker API, we can get real-time notifications of everything happening in the Docker engine.
Demo Monitoring with MSOMS and cadvisor
Build Automation & Source Integration Source Integration. Docker Hub supports webhooks and source control integration (Github, etc.) Using Docker files, you can automate Container builds with your developers latest builds Automated Build. Enables automated image builds and upload to Docker Hub Orchestration. Advanced orchestration with other tools (Jenkins, Chef, Azure Automation)
Scenario: Org has an existing website running on NGINX (Apache) Dev team is building a new version of the site they want to test Step 1: Update source control (check code into Git repository) Step 2: Git triggers a webhook in Docker Hub This is automatic. Triggers image rebuild via a Dockerfile. Step 4: Remove and redeploy image (using Docker CLI) Step 3: Download new image (using Docker CLI)
Demo Automated Builds with Docker and Github
Resources Docker Documentation https://docs.docker.com/ ACS Github Homepage https://github.com/azure/azure-quickstart-templates/tree/master/101-acs-swarm Docker Swarm Container Service Walkthrough How to create the SSH key https://github.com/azure/azure-quickstart-templates/blob/master/101-acsswarm/docs/swarmpreviewwalkthrough.md https://www.digitalocean.com/community/tutorials/how-to-create-ssh-keys-withputty-to-connect-to-a-vps
Resources Docker Hub Automated Build https://hub.docker.com/r/leeberg/thinksmalltoscalebigmms2016 Github for Dockerfile and website example code https://github.com/leeberg/mms2016
Next Steps Walk through the tutorials Get the sample scripts Get comfortable with Docker (it will help when Windows Containers arrive!) Explore use production use cases in your org!
And Then Discuss: Evaluations: Ask your questions-real world answers! Plenty of time to engage, share knowledge. Please provide session feedback by clicking the EVAL button in the scheduler app (also download slides). One lucky winner will receive a free ticket to the next MMS!
DevOps Process with Containers Developers update, iterate, and deploy updated containers 2 3 Operations collaborates with developers to provide app metrics and insights Developers build and test apps in containers, using development environment i.e. Visual Studio Central 1 2 Repository Operations automates deployment and monitors deployed apps from central repository Containers pushed to central repository