Arup Nanda VP, Data Services Priceline.com

Similar documents
Data Engineering for Data Science

Who is Docker and how he can help us? Heino Talvik

A Hands on Introduction to Docker

Introduction to containers

An introduction to Docker

Introduction to Containers

Harbor Registry. VMware VMware Inc. All rights reserved.

Red Hat Containers Cheat Sheet

Containers. Pablo F. Ordóñez. October 18, 2018

/ Cloud Computing. Recitation 5 February 14th, 2017

SQL Server inside a docker container. Christophe LAPORTE SQL Server MVP/MCM SQL Saturday 735 Helsinki 2018

Think Small to Scale Big

Best Practices for Developing & Deploying Java Applications with Docker

Docker Cheat Sheet. Introduction

docker & HEP: containerization of applications for development, distribution and preservation

Set up, Configure, and Use Docker on Local Dev Machine

Infrastructure at your Service. Oracle over Docker. Oracle over Docker

Network softwarization Lab session 2: OS Virtualization Networking

Dockerfile & docker CLI Cheat Sheet

agenda PAE Docker Docker PAE

Midterm Presentation Schedule

Container-based virtualization: Docker

Getting Started With Amazon EC2 Container Service

/ Cloud Computing. Recitation 5 September 26 th, 2017

Getting Started With Containers

Docker A FRAMEWORK FOR DATA INTENSIVE COMPUTING

Creating pipelines that build, test and deploy containerized artifacts Slides: Tom Adams

DGX-1 DOCKER USER GUIDE Josh Park Senior Solutions Architect Contents created by Jack Han Solutions Architect

~Deep dive into Windows Containers and Docker~

Code: Slides:

MySQL operations with Docker A quick guide for the uninitiated

Docker task in HPC Pack

Running Splunk Enterprise within Docker

Technical Manual. Software Quality Analysis as a Service (SQUAAD) Team No.1. Implementers: Aleksandr Chernousov Chris Harman Supicha Phadungslip

GitLab-CI and Docker Registry

Docker und IBM Digital Experience in Docker Container

DEPLOYMENT MADE EASY!

Deployment Patterns using Docker and Chef

CS-580K/480K Advanced Topics in Cloud Computing. Container III

Important DevOps Technologies (3+2+3days) for Deployment

S INSIDE NVIDIA GPU CLOUD DEEP LEARNING FRAMEWORK CONTAINERS

Using DC/OS for Continuous Delivery

INDIGO PAAS TUTORIAL. ! Marica Antonacci RIA INFN-Bari

Docker on VDS. Aurelijus Banelis

Fixing the "It works on my machine!" Problem with Docker

Index. Bessel function, 51 Big data, 1. Cloud-based version-control system, 226 Containerization, 30 application, 32 virtualize processes, 30 31

Well, That Escalated Quickly! How abusing the Docker API Led to Remote Code Execution, Same Origin Bypass and Persistence in the Hypervisor via

Run containerized applications from pre-existing images stored in a centralized registry

Docker 101 Workshop. Eric Smalling - Solution Architect, Docker

containerization: more than the new virtualization

/ Cloud Computing. Recitation 5 September 27 th, 2016

Developing and Testing Java Microservices on Docker. Todd Fasullo Dir. Engineering

Follow me!

Containers, Serverless and Functions in a nutshell. Eugene Fedorenko

" Qué me estás container?" Docker for dummies

Zenoss Resource Manager Upgrade Guide

Docker II - Judgement Day

RACifying Multitenant

Tricks of the Captains. Adrian Mouat. Chief Scientist Container Solutions

Zenoss Resource Manager Upgrade Guide

Splunk N Box. Splunk Multi-Site Clusters In 20 Minutes or Less! Mohamad Hassan Sales Engineer. 9/25/2017 Washington, DC

UP! TO DOCKER PAAS. Ming

DevOps in the Cloud A pipeline to heaven?! Robert Cowham BCS CMSG Vice Chair

VNS3 3.5 Container System Add-Ons

DOCKER 101 FOR JS AFFICIONADOS. Christian Ulbrich, Zalari UG

Investigating Containers for Future Services and User Application Support

CONTAINERIZING JOBS ON THE ACCRE CLUSTER WITH SINGULARITY

Cloud providers, tools and best practices in running Magento on Kubernetes. Adrian Balcan MindMagnet Software

Zenoss Core Upgrade Guide

Singularity: container formats

Docker for Sysadmins: Linux Windows VMware

It s probably the most popular containerization technology on Linux these days

SQL Server containers with in-container data

Dockerfile Best Practices

Introduction to Linux

Installing and Using Docker Toolbox for Mac OSX and Windows

Networking for the Cloud DBA. Arup Nanda Longtime Oracle DBA And Explorer of New Things

Get Started with SQL Server containers with cloned databases

Zenoss Resource Manager Upgrade Guide

[Docker] Containerization

USING NGC WITH GOOGLE CLOUD PLATFORM

MySQL and Docker Strategies Patrick Galbraith Giuseppe Maxia

Travis Cardwell Technical Meeting

Blockchain on Kubernetes

Using Docker in High Performance Computing in OpenPOWER Environment

Docker. Master the execution environment of your applications. Aurélien Dumez. Inria Bordeaux - Sud-Ouest. Tuesday, March 24th 2015

Microservices. Chaos Kontrolle mit Kubernetes. Robert Kubis - Developer Advocate,

Building A Better Test Platform:

Docker and Oracle Everything You Wanted To Know

Infrastructure Security 2.0

BEST PRACTICES FOR DOCKER

Leveraging Docker and CoreOS to provide always available Cassandra at Instaclustr

Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2

Containerizing GPU Applications with Docker for Scaling to the Cloud

Copyright Heraflux Technologies. Do not redistribute or copy as your own. 1

Getting Started with Hadoop

CNA1699BU Running Docker on your Existing Infrastructure with vsphere Integrated Containers Martijn Baecke Patrick Daigle VMworld 2017 Content: Not fo

Deploying Rails with Kubernetes

StreamSets Control Hub Installation Guide

ourwiki Documentation

Transcription:

Jumpstarting Docker Arup Nanda VP, Data Services Priceline.com

My application worked in Dev but not in QA Will it work in production? I need an environment right now No, I can t wait for 2 weeks I just need it for a day to test my hypothesis. What do you mean I should have budgeted it?. 2

booking.com priceline.com kayak.com agoda.com rentalcars.com opentable.com 3

Bots Demand Technology Supply 4

WWW Providers Affiliates GDS White labels Affiliates Inventory 250K Hotels X 200 rooms X 365 days X. Demand Technology Supply 5

6

7

Cache Demand Technology Supply Smart Cache Management 8

Bots MySQL MySQL MySQL MySQL Shards 9

Bots Data Pipe based on Kafka named Mississipi has APIs and common libraries MySQL MySQL MySQL MySQL Shards 10

Need: Automatically spin up MySQL servers, with the right metadata! 11

Agenda What s the problem we are trying to solve? How Docker solves the problem? Installing and getting started with Docker How is it different from Virtual Machines? Docker in the Cloud GCP, AWS, Oracle Cloud, etc. Docker Compose, Swarm, etc.. 12

Basic Command Structure The word docker followed by a verb and optionally arguments $ docker ps $ docker run mycontainer Help $ docker help $ docker ps help 13

Coloring Book Example of building a Coloring Book Docker Concepts Image: a set of programs, or building blocks which can be deployed; but not deployed yet Container: when you deploy an image. It s the executable. 14

Pull Image Pull images from docker hub $ docker pull ubuntu Pulls the image from the hub to the docker host Show the images we have on the docker host $ docker images 15

Bring up a container Run a container $ docker run ubuntu See which containers are running $ docker ps See which containers were running earlier; but not now $ docker ps -a Containers do not stay up if they have nothing to do 16

Containers and Images Users Image Container Modified Container on disk 17

Containers Copy 1 Image Container on disk Copy 2 18

Running Apps in Containers Verb run $ docker run --name myhost ubuntu sleep 100 This will not let you come back From a different terminal $ docker ps Stop the container $ docker stop myhost Remove the container $ docker rm myhost 19

Detached mode Run the container in detached mode to run in background docker run -d --name myhost ubuntu sleep 100 20

Execute commands Execute a command in a running container $ docker exec -it myhost /bin/bash This gives the OS prompt of the container Options: -i gives interactive prompt -t gives TTY prompt (shell) 21

Execute in a container remotely From Docker host $ docker exec ContainerName Program So to run docker exec myhost cat /etc/*release* in container myhost: $ docker exec myhost cat /etc/*release* A more practical example: Shell script to compute bonus: /usr/bin/bonus.sh $ docker exec myhost sh /usr/bin/bonus.sh 100 22

Persisting Changes Commit Changes $ docker commit Container Repository Repository, not Image You may add a tag $ docker commit docker commit Container Repository:v1 23

Comparison with VMs Virtual Machine Docker App1 App2 App1 App2 Container Container Linux Windows VM VM Host OS Host OS 24

Volume Databases need persistent data Makes a filesystem persist in the host even after the container is shutdown On the docker host, create a directory /dockervol Add this to the container as a filesystem named /fs $ docker run --name myhost -d -v /dockervol:/fs myubuntu1:v1 sleep 1000 This will create a filesystem on the container named /fs 25

Copying files to/from containers Copying $ docker cp myhost:/var/www/html/index.html. 26

Port Mapping Mycontainer1 Port 80 myhost http://myhost:80 Mycontainer2 Port 80 27

Port Mapping Mycontainer1 Port 80 Mycontainer2 Port 80 myhost 8080 8081 http://myhost:8080 28

Port Mapping Start the container $ docker run -d -p 8080:80 --name myhost ubuntu2 nginx - g "daemon off;" Now you can kick off the webpage http://192.168.56.101:8080/ where 192.168.56.101 is the address of the Dockerhost. 29

Dockerfile Create a file named Dockerfile in the directory /dockerfiles FROM ubuntu:latest RUN apt-get update RUN apt-get install -y vim RUN apt-get install -y nginx RUN echo "My first ubuntu docker image for vim and nginx" > /var/www/html/index.html EXPOSE 80 Create the image from the dockerfile $ docker build -t ubuntu2 /dockerfiles 30

Changes in the file To find out the changes in the files since the container was created, $ docker diff myhost Sample output: $ docker diff myhost C /root C /root/.bash_history C /root/.viminfo C /tmp A /tmp/a C /usr C /usr/bin C /usr/bin/bonus.sh 31

Inspect Containers Get detailed inner settings of a container $ docker inspect myhost2 Shows all the important details 32

Get Usage Stats $ docker stats myhost2 33

Compose Deploy multiple containers on multiple hosts Uses a YAML file to define the various components 34

Swarm Bring up multiple containers A cluster Automatically bring up containers Automatically restart after failures 35

Container Clouds Container2 MyImage Amazon Container Cloud Oracle Container Cloud Service Container1 36

Summary Docker is just a set of processes running inside a host OS; it doesn t have an OS This makes it very quick to bring up and deploy Takes very little memory and CPU Also makes it less secure and isolated Image: the executable programs and processes to be deployed Container: the running processes Docker Hub: the storage for all docker images Dockerfile: a file that instructs what to do when building containers Compose: a method to automate deployment of docker containers Swarm: the automated way to manage a network of containers 37

Thank You! Blog: arup.blogspot.com Tweeter: @ArupNanda Facebook.com/ArupKNanda 38