A DEVOPS STATE OF MIND WITH DOCKER AND KUBERNETES Chris Van Tuin Chief Technologist, West cvantuin@redhat.com
Open Source V In short, software is eating the world. - Marc Andreessen, Wall Street Journal, August 2011
UBER, LYFT FALLOUT: TAXI RIDES PLUNGE 65% IN SAN FRANCISCO
THROW IT OVER THE WALL Walled off people, walled off processes, walled off technologies DEV QA OPS 5
THE NEED FOR SPEED THE ACCELERATION OF APPLICATION DELIVERY FOR THE BUSINESS
WHAT CAN I.T. DO? I.T. CAN TURN OPS AND DEV INTO DEVOPS DevOps is a software development method that stresses communication, collaboration and integration between software developers and information PEOPLE technology (IT) professionals. [1] DEVOPS PROCESS TECHNOLOGY Applying many of the principles of Agile software development to the full application lifecycle and incorporating automation and monitoring with just a touch of Lean Manufacturing theory.
BREAK DOWN THE WALLS Cross-organization teams, walled off processes, walled off technologies DEV QA OPS 8
WHAT ENABLES DEVOPS? Configuration in code Standardized environments Linux containers Automated provisioning KEY CONCEPT: FAIL FAST AND RECOVER VS. NEVER FAIL
WHAT ENABLES DEVOPS? CI/CD: Automated testing and deployment Continuous integration Continuous delivery KEY CONCEPT: SMALL CHANGES = LESS RISK
WHAT ENABLES DEVOPS? Continuous innovation Developer self-service Rapid prototyping KEY CONCEPT: CULTURE CHANGE = ACCEPTANCE OF FAILURE
ORGANIZATIONS IMPLEMENTING DEVOPS Better deployment quality 63% DEVOPS VALUE IN ACTION: VELOCITY AT AMAZON AWS MAX DEPLOYMENTS/ HOUR 10,000 Faster release frequency 63% 11.6.001% Improved process visibility 61% MEAN TIME BETWEEN DEPLOYMENTS (SECONDS) SOFTWARE DEPLOYMENTS CAUSING AN OUTAGE Source: 2014 State of DevOps Report Puppet Labs, IT Revolution Press, ThoughtWorks
CASE STUDY
FINANCIAL SERVICES COMPANY It took 2 years after a competitive start-up launch to get a competing product to market. Growth Competition
FINANCIAL SERVICES COMPANY It took 2 years after a competitive start-up launch to get a competing product to market. It could take 6 weeks to get a single word changed on the web site. Growth Competition Agility Predictability
FINANCIAL SERVICES COMPANY It took 2 years after a competitive start-up launch to get a competing product to market. It could take 6 weeks to get a single word changed on the web site. Growth Competition Agility Predictability Productivity When developers work in Node.js, they can change the code they re working on, direct it to run, and see whether it works-in the blink of an eye.
FINANCIAL SERVICES COMPANY It took 2 years after a competitive start-up launch to get a competing product to market. It could take 6 weeks to get a single word changed on the web site. Growth Competition Agility Predictability Productivity Recruiting When developers work in Node.js, they can change the code they re working on, direct it to run, and see whether it works-in the blink of an eye. The environment, while stable, didn t use the sexiest technologies, which made recruiting difficult.
DEVOPS SOLUTION APPROACH Leverage Automation Technologies Combined with Cloud Architecture
CI/CD with Containers
ENABLING TECHNOLOGIES
OPEN SOURCE ENABLING DEVOPS
OPEN SOURCE ENABLING DEVOPS
THE NEW OPERATING SYSTEM APP Orchestrator: Model the app across multiple hosts/containers SERVICE 1 SERVICE 2 SERVICE 3 SERVICE 4 Scheduler: Provide service and APIs for placing the app onto resources SCHEDULER Container pool : Provide resources to run app RED HAT ENTERPRISE LINUX RED HAT ENTERPRISE LINUX RED HAT ENTERPRISE LINUX
WHAT ARE LINUX CONTAINERS? Used to create containers for software applications / microservices CONTAINER Package Once Deploy Anywhere APP LIBS Containers provide lightweight isolation of process, network, filesystem spaces HOST OS SERVER Docker builds on Linux containers, adds an API, image format, runtime, and a delivery and sharing model 24
CONTAINER USE CASES DevOps CI/CD App Modernization Microservices Hardware Modernization 100% Availability Real-time Scale Diet VM Infrastructure Optimization Big Data
26
BUILD, SHIP, RUN Dockerfile Image Container FROM fedora:latest CMD echo Hello Red Hat Certified Private Registry docker.io Registry Physical, Virtual, Cloud Build Ship Run docker build or commit docker push or pull <IMAGE_ID> docker run <IMAGE_ID> 27
BUILD, SHIP, RUN Dockerfile FROM fedora:latest CMD echo Hello Build docker build or commit 28
CONSISTENT PACKAGING FORMAT Docker provides a language agnostic packaging format and runtime API #include<stdio.h> main() { printf("hello World"); } public class HelloWorld { } public static void main(string[] args) { System.out.println ("Hello, World"); } var http = require('http'); var server = http.createserver( function (request, response) { response.writehead(200, {"Content-Type": "text/plain"}); response.end("hello World\n"); }); server.listen(8000); $_ = "hello world"; $_ =~ s/^(\b\w)(\b\w+)\s(\d)(\d+)$/ \U$1\E$2 \U$3\E$4\!\n/; print $_; <?php Print "Hello, World!";?> PHYSICAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUD
PACKAGED DEPENDENCIES Package dependencies ensure consistency and portability* #include<stdio.h> main() { printf("hello World"); } public class HelloWorld { } public static void main(string[] args) { System.out.println ("Hello, World"); } var http = require('http'); var server = http.createserver( function (request, response) { response.writehead(200, {"Content-Type": "text/plain"}); response.end("hello World\n"); }); server.listen(8000); $_ = "hello world"; $_ =~ s/^(\b\w)(\b\w+)\s(\d)(\d+)$/ \U$1\E$2 \U$3\E$4\!\n/; print $_; <?php Print "Hello, World!";?> bash glibc jre nodejs perl php... libssl libv8... bash glibc... bash glibc bash glibc PHYSICAL VIRTUAL PRIVATE CLOUD PUBLIC CLOUD
TRADITIONAL SOFTWARE SUPPLY CHAIN
A CONVERGED SOFTWARE SUPPLY CHAIN
CUSTOM SUPPLY CHAIN
BUILD, SHIP, RUN Dockerfile Image FROM fedora:latest CMD echo Hello Red Hat Certified Private Registry docker.io Registry Build Ship docker build or commit docker push or pull <IMAGE_ID> 34
WHAT S INSIDE THE CONTAINER MATTERS 64% of official images in Docker Hub contain high priority security vulnerabilities examples: ShellShock (bash) Heartbleed (OpenSSL) Poodle (OpenSSL) Source: Over 30% of Official Images in Docker Hub Contain High Priority Security Vulnerabilities, Jayanth Gummaraju, Tarun Desikan, and Yoshio Turner, BanyanOps, May 2015 (http://www.banyanops.com/pdf/banyanops-analyzingdockerhub-whitepaper.pdf)
36
OpenSCAP Scan physical servers, virtual machines, docker images and containers for Compliance (CCEs) and known Vulnerabilities (CVEs) Content Scan Reports SCAP Security Guide for RHEL CCE-27002-5 Set Password Minimum Length
WHAT ARE MY OPTIONS? Security-by-Luck Unsupported distro Public registry End-of-lifed Security-by-Firewall No patch management Untrusted containers selinux disabled Trusted Platform Supported distro Patch management Trusted containers & host Private registry Container scanning selinux enforcing 38
BUILD, SHIP, RUN Dockerfile Image Container FROM fedora:latest CMD echo Hello Red Hat Certified Private Registry docker.io Registry Physical, Virtual, Cloud Build Ship Run docker build or commit docker push or pull <IMAGE_ID> docker run <IMAGE_ID> 39
TRADITIONAL OS VS CONTAINERS Traditional OS Containers CONTAINER CONTAINER APP A APP B APP A APP B LIBS A LIBS B LIBS LIBS LIBS LIBS HOST OS HARDWARE HOST OS HARDWARE
More serious workloads require orchestration like Kubernetes to offload management overhead ADD NAME (View > Master > Slide master)
Kubernetes: Deploy Application 42 ADD NAME (View > Master > Slide master)
Kubernetes: Deploy Application 43 ADD NAME (View > Master > Slide master)
Kubernetes: Deploy Application 44 ADD NAME (View > Master > Slide master)
Kubernetes: Scale Service 45 ADD NAME (View > Master > Slide master)
Kubernetes: Scale Service 46 ADD NAME (View > Master > Slide master)
Once created, Kubernetes will keep the environment online as described in the json file JSON Kubernetes ADD NAME (View > Master > Slide master)
Once created, Kubernetes will keep the environment online as described in the json file JSON Kubernetes ADD NAME (View > Master > Slide master)
Once created, Kubernetes will keep the environment online as described in the json file JSON Kubernetes ADD NAME (View > Master > Slide master)
Once created, Kubernetes will keep the environment online as described in the json file JSON Kubernetes ADD NAME (View > Master > Slide master)
Canary Deployments ` Version 1.2 Tests / CI Version 1 Version 1 Version 1 51
Each container/pod is updated one by one 33% Version 1 Version 1 Version 1.2 52
Canary Deployments 66% Version 1 Version 1.2 Version 1.2 53
Canary Deployments 100% Version 1.2 Version 1.2 Version 1.2 54
Blue / Green Deployments Version 1 ADD NAME (View > Master > Slide master)
Admins won t get stuck in middle of a deployment Version 1 Version 1.2 ADD NAME (View > Master > Slide master)
Tests and certification can be done before customers access it Version 1 Version 1.2 Tests / CI ADD NAME (View > Master > Slide master)
Once ready, the new version is used and the old version can be removed Version 1 Version 1.2 ADD NAME (View > Master > Slide master)
Rollbacks can be done using the same method if desired Version 1.2 ADD NAME (View > Master > Slide master)
DEVOPS with Platform as a Service REDUCE CYCLE TIME FROM IDEA TO FEATURE
OPENSHIFT: The Docker and Kubernetes Container Platform Any App Anywhere Any Time Any Scale Any Docker Image Dev, Test, Prod Self-Service Cloud burst Any Source 2 Image Private Cloud DevOps Multi-DC Big Data Public Cloud CI/CD Scale Up/Down Web, App, DB Physical Auto Build Web Scale Batch Virtual Auto Deploy 1k hosts Legacy, Persistent Laptop Security Scan 30k pods.net Mac,Win,Lin CDK Health Check
Choice of Platforms Add any Docker Image
Automated Build Pipeline
Automated Deployments
DEVOPS ROI Improve business agility Improve developer productivity Improve business predictability Improve operational efficiency and costs
DEVOPS METRICS Deployment Frequency Change Volume Lead Time 99.999 Deployment Failure Rate Mean Time to Recover Service Availability
THANK YOU Chris Van Tuin cvantuin@redhat.com