Dockerize Your IT! Centrale Nantes Information Technology Department Yoann Juet Dec, 2018

Transcription

1 Dockerize Your IT! Centrale Nantes Information Technology Department Yoann Juet Dec,

2 A Brief History of Containers UNIX CHROOT BSD JAIL LINUX VSERVER LINUX NAMESPACES LINUX OPENVZ LINUX CGROUPS LINUX LXC DOCKER KUBERNETES

3 Docker Containers!= Virtual Machines 3 Containers share host ressources (kernel, cpu, memory, network) ; good isolation between Containers VMs have a full copy of an OS, leading to cpu, memory, network overhead ; very good isolation between VMs

4 Docker architecture docker : the Docker user CLI dockerd : engine daemon Create image, pass it to containerd containerd : runtime daemon Core container runtime for Docker Manage the complete container lifecycle (stop, start, transfer, supervision, storage, network) This model gives the ability to restart or upgrade Docker Engine without breaking the running containers 4

5 Docker images, An image is a read-only group of layers of other images. It includes everything an application needs to run: binaries, libraries, config files Each image is made of a base image (e.g. debian, ubuntu, alpine) plus a collection of diffs - intermediate images/layers - that adds the required features (e.g. emacs, apache). Images can be stored on public, private repos, on any host machine that has previously pulled the package from a repo 5

6 Docker Containers A container is a running instance - read-write - of an image You can run containers on Linux, Windows 10, Windows Server 2016, Cloud (AWS, Google ) Containers should be as ephemeral as possible. You should expect them to go down at any time and lose all data stored inside: Don t store data in containers Don t run more than one process in a single container Use custom created volumes or system mounts 6

7 Docker Image!= Docker Container push pull r/w Instance App 1 Container Public or Private Registry App Image r/w Instance App 2 Container Commit Remember A Docker Image is similar to a read-only template A Docker Container is an writable instance of a Docker Image Each Docker Container has its own read-write layer - thus its own data - that sits on one or more Docker image 7

8 Docker Containers - Pros and Cons Lightweight, low resources consumptions Isolation model, default set of capabilities Low attack surface Networking can be tricky Fast boot, removal, reproducibility : the same code runs everywhere Orchestration complexity Large ecosystem - lot of official and unofficial images - Ideal for development team (test, pre-prod, prod) 8

9 Installing Docker Community Edition (CE) On debian 9 (stretch) host - from the netinst image - Docker repository (version 18.09) user@host:~$ sudo apt-get update && apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common user@host:~$ sudo curl -fssl apt-key add - user@host:~$ sudo apt-key fingerprint 0EBFCD88 pub rsa [SCEA] 9DC FC7 DD38 854A E2D8 8D81 803C 0EBF CD88 uid sub [ unknown] Docker Release (CE deb) <docker@docker.com> rsa [S] user@host:~$ sudo add-apt-repository "deb stretch stable" user@host:~$ sudo apt-get update && apt-get install docker.ce user@host:~$ sudo usermod -ag docker <user> && newgrp docker 9

10 Installing Docker Community Edition (CE) On debian 9 (stretch) host - from the netinst image - Check for correct installation user@host:~$ docker info Server Version: Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Runtimes: runc Default Runtime: runc Kernel Version: amd64 Operating System: Debian GNU/Linux 9 (stretch) Docker Root Dir: /var/lib/docker Product License: Community Engine 10

11 Run Your First Docker Application Let s say an nginx http server! Search for an official nginx image user@host:~$ docker search nginx NAME DESCRIPTION STARS OFFICIAL AUTOMATED nginx Official build of Nginx [OK] jwilder/nginx-proxy Automated Nginx reverse proxy for docker con 1473 [OK] richarvey/nginx-php-fpm Container running Nginx + PHP-FPM capable of 652 [OK] Download the image from the official Docker Registry user@host:~$ docker pull nginx In the OFFICIAL column, OK indicates Using default tag: latest an image built by the latest: Pulling from library/nginx company/community behind the project a5a6f2f73cd8: Pull complete 1ba02017c4b2: Pull complete 33b176c904de: Pull complete Digest: sha256:5d32f60db294b5deb55d078cd4feb410ad88e6fe77500c87d3970eca97f54dba Status: Downloaded newer image for nginx:latest 11

12 Run Your First Docker Application Let s say an nginx http server! Verify user@host:~$ docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest 568c4670fa80 46 hours ago 109MB Run a container based on this image user@host:~$ mkdir ~/mydir && echo "Hello, France!" > ~/mydir/index.html && docker run --name myweb -v ~/mydir:/usr/share/nginx/html -p 8080:80 -d nginx 8807f82f280b6dde c414b22b9ac7b77362b39d7c58c94e15d2eedc905 user@host:~$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8807f82f280b nginx "nginx -g " 40 seconds ago Up 39 seconds :8080->80/tcp myweb 12

13 Run Your First Docker Application Let s say an nginx http server! Connect to your App user@host:~$ wget -q localhost:8080 -O /dev/stdout Hello, France! Enter in your App user@host:~$ docker exec -it myweb /bin/bash root@8807f82f280b:/# ls /etc/nginx/ && exit conf.d fastcgi_params koi-utf koi-win mime.types modules nginx.conf Stop and Remove your App user@host:~$ docker rm -f myweb myweb root@host:/tmp# docker ps --all CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 13

14 Focus on Docker Images List all installed images docker images docker image ls [--all] Pull an image from a registry docker pull image docker image pull image Push an image to a registry Remove an image Backup and restore an image (tarball) docker push image[:tag] docker image push image[:tag] docker rmi image docker image rm image docker image prune [--all] docker save -o archive.tar image docker load < archive.tar Inspect the content of an image docker inspect image 14

15 Focus on Docker Containers List all containers docker ps [--all] docker container ls [--all] [re]start an inactive container docker [re]start container docker container [re]start container Stop an active container docker stop container docker container stop container [stop and] Remove a container Execute a command in an active container / Get a shell console docker rm [-f] container docker container rm [-f] container docker exec container command docker exec -it container /bin/bash Inspect the content of a container docker inspect container 15

16 Focus on Docker Containers Create an image from a container docker commit container [REPO[:TAG]] Show the logs of a container docker logs container Create a new container docker create image docker run image Backup and restore a container (tarball) docker export container > archive.tar docker import - container < archive.tar 16

17 What about my data? Remember that each container has is own read-write layer 1 - Get alpine image user@host:~$ docker pull alpine 2 - Run it user@host:~$ docker run -d -it --name myapp alpine 3 - Edit a file in the container user@host:~$ docker exec myapp \ sh -c 'echo "Hello, France!" > /root/msg.txt' 4 - Stop the container user@host:~$ docker stop myapp 5 - Start the container user@host:~$ docker start myapp 6 - Check user@host:~$ docker exec myapp less /root/msg.txt Hello, France 7 - Stop then remove the container user@host:~$ docker rm -f myapp 8 - Run again a container user@host:~$ docker run -d -it --name myapp alpine 9 - Check user@host:~$ docker exec myapp less /root/msg.txt more: can't open '/root/msg.txt': No such file or directory When you instantiate an image, the new container starts with a clean filesystem 17

18 What about my data? Sharing data in your docker host with containers Run it echo "Hello, Nantes!" > ~/msg.txt && docker run -d -it -v ~/msg.txt:/root/hop.txt --name mydata alpine Check docker exec mydata ls /root hop.txt docker exec mydata less /root/hop.txt Hello, Nantes! Run another docker based on alpine docker run -d -it -v ~/msg.txt:/root/hop.txt --name mydatb alpine Edit the message echo "Hello, Centrale!" > ~/msg.txt Check docker exec mydata less /root/hop.txt Hello, Centrale! docker exec mydatb less /root/hop.txt Hello, Centrale! Use the -v option : -v [HOST-DIR]:[CONTAINER-DIR] It creates a bind mount exposing the host directory/host file to the container directory/file 18

19 What about my data? Sharing data in your docker host with containers Get information about volumes docker inspect mydata "Mounts": [ { "Type": "bind", "Source": "/home/user/msg.txt", "Destination": "/root/hop.txt", "Mode": "", "RW": true, "Propagation": "rprivate" } ], Instruction to share data in read-only mode By default, Docker mounts the volume in read-write mode user@host:~$ docker run -d -it -v ~/msg.txt:/root/hop.txt:ro --name mydata alpine 19

20 What about my data? Sharing data between containers Run the yoyo container docker run -d -it -v /sharing --name yoyo alpine docker inspect yoyo "Mounts": [ { "Type": "volume", "Name": "fdc5bd0de90688d590b38f9f931eb011c4de9c4032d49", "Source": "/var/lib/docker/volumes/fdc5bd0de90a54b/_data", "Destination": "/sharing", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ], If the host directory/file is omitted, a data container (read-write mode per default) is created The volume specified, here /sharing, is created inside the container user@host:~$ touch /var/lib/docker/volumes/fdc5bd0d/_data/msg.txt user@host:~$ docker exec yoyo ls /sharing msg.txt 20

21 What about my data? Sharing data between containers Run the dong container docker run -d -it --volumes-from yoyo --name dong alpine docker inspect dong "Mounts": [ { "Type": "volume", "Name": "fdc5bd0de90688d590b38f9f931eb011c4de9c404a54b", "Source": "/var/lib/docker/volumes/fdc5bd0de9054b/_data", "Destination": "/sharing", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ], user@host:~$ docker exec yoyo ls /sharing Msg.txt user@host:~$ docker exec dong ls /sharing Msg.txt user@host:~$ docker exec dong touch /sharing/msg2.txt user@host:~$ docker exec yoyo ls /sharing msg.txt Msg2.txt user@host:~$ ls /var/lib/docker/volumes/fdc5bd0de90688d5/_data msg2.txt msg.txt 21

22 Committing Changes in a Container Turning a container into an image user@host:~$ docker image ls --all REPOSITORY TAG IMAGE ID CREATED SIZE alpine latest 196d12cf6ab1 2 months ago 4.41MB user@host:~$ docker run -d -it --name ding alpine user@host:~$ docker exec -it ding /bin/sh / # apk update && apk upgrade fetch fetch v g80b45d6920 [ v g80b45d6920 [ OK: 9546 distinct packages available OK: 4 MiB in 13 packages / # exit user@host:~$ docker commit ding alpine: sha256:4acef3925b22e668e0e1755a3fa5ab cacee28c8a487c5754ea105be70 user@host:~$ docker image ls --all REPOSITORY TAG IMAGE ID CREATED SIZE alpine acef3925b22 27 seconds ago 5.71MB alpine latest 196d12cf6ab1 2 months ago 4.41MB 22

23 Committing Changes in a Container Display container changes user@host:~$ docker diff ding C /lib C /lib/apk C /lib/apk/db C /var/cache C /var/cache/apk A /var/cache/apk/apkindex.adfa7ceb.tar.gz A /var/cache/apk/apkindex.efaa1f73.tar.gz C /root A /root/.ash_history You see all changes applied to the read-write layer, the container itself - useful before executing the commit instruction First column: A means that the directory/file was added, C means that a change was made, D means that it was removed. 23

24 Docker Networking Services Bridge Host Macvlan Bridge None Overlay Share host interfaces No more network isolation between the host and containers Should be used very carefully Unique MAC address Allows you to configure slave/sub-interfaces of a parent, physical ethernet interface, each with its own unique MAC address No network! Multiple hosts with only L3 connectivity Combines local bridges and VXLAN, GRE to overlay container-to-container Useful to offer network connectivity between containers on multiple hosts using their own IP addresses 24

25 Going deeper in docker bridge default mode docker run --name alice -p 8080:80 -d nginx docker run --name bob -p 8081:80 -d nginx docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ffe72798ac2d nginx "nginx -g 'daemon of " :8081->80/tcp bob 1a2f2f83e044 nginx "nginx -g 'daemon of " :8080->80/tcp alice Expose unique ports on the host - here 8081 and

26 Going deeper in docker bridge default mode brctl show bridge name bridge id STP enabled interfaces docker d1a2630 no veth856faee vetha53c999 ip add 3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:2d:1a:26:30 brd ff:ff:ff:ff:ff:ff inet /16 brd scope global docker0 valid_lft forever preferred_lft forever 7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default link/ether be:71:06:64:64:8f brd ff:ff:ff:ff:ff:ff link-netnsid 1 11: veth856faee@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default link/ether b2:6c:95:08:43:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 2 26

27 Going deeper in docker bridge default mode docker inspect bob "NetworkSettings": { "Bridge": "", "Gateway": " ", "IPAddress": " ", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:04", } user@host:~$ docker inspect alice "NetworkSettings": { "Bridge": "", "Gateway": " ", "IPAddress": " ", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:03", } Docker assigns a dynamic IP address for both containers 27

28 Going deeper in docker bridge default mode iptables -L -n -v Chain DOCKER (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT tcp --!docker0 docker / tcp dpt: ACCEPT tcp --!docker0 docker / tcp dpt:80 user@host:~$ iptables -t nat -L -n -v Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 1 84 RETURN all -- docker0 * / /0 0 0 DNAT tcp --!docker0 * / /0 tcp dpt:8080 to: : DNAT tcp --!docker0 * / /0 tcp dpt:8081 to: :80 28

29 Build your own image docker image ls --all REPOSITORY TAG IMAGE ID CREATED SIZE alpine latest 196d12cf6ab1 2 months ago 4.41MB user@host:~$ vi /tmp/dockerfile FROM alpine:latest RUN apk update && apk upgrade ENTRYPOINT ["/bin/echo","hello, France!"] user@host:~$ cd /tmp && docker build. -t hello Sending build context to Docker daemon 18.94kB Step 1/3 : FROM alpine:latest ---> 196d12cf6ab1 Step 2/3 : RUN apk update && apk upgrade ---> Running in 116d94cb96fa fetch /APKINDEX.tar.gz fetch 86_64/APKINDEX.tar.gz v ge3ed6b4e31 [ v g45bdd0edfb [ OK: 9546 distinct packages available OK: 4 MiB in 13 packages Removing intermediate container 116d94cb96fa ---> fd105816bcb9 Step 3/3 : ENTRYPOINT ["/bin/echo","hello, France!"] ---> Running in 8756eb9e1d37 Removing intermediate container 8756eb9e1d37 ---> b4277a54d78b Successfully built b4277a54d78b Successfully tagged hello:latest 29

30 Sending build context to Docker daemon 18.94kB Step 1/3 : FROM alpine:latest ---> 196d12cf6ab1 Step 2/3 : RUN apk update && apk upgrade ---> Running in 116d94cb96fa fetch /APKINDEX.tar.gz fetch 86_64/APKINDEX.tar.gz v ge3ed6b4e31 [ v g45bdd0edfb [ OK: 9546 distinct packages available OK: 4 MiB in 13 packages Removing intermediate container 116d94cb96fa ---> fd105816bcb9 Step 3/3 : ENTRYPOINT ["/bin/echo","hello, France!"] ---> Running in 8756eb9e1d37 Removing intermediate container 8756eb9e1d37 ---> b4277a54d78b Successfully built b4277a54d78b Successfully tagged hello:latest user@host:~$ docker image ls --all REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> fd105816bcb9 5 minutes ago 5.71MB hello latest b4277a54d78b 5 minutes ago 5.71MB alpine latest 196d12cf6ab1 2 months ago 4.41MB 30

31 Build your own image vi /tmp/dockerfile FROM alpine:latest RUN apk update && apk upgrade RUN apk add openssh ENTRYPOINT ["/bin/echo","hello, France!"] cd /tmp && docker build. -t hello Sending build context to Docker daemon 5.632kB Step 1/4 : FROM alpine:latest ---> 196d12cf6ab1 Step 2/4 : RUN apk update && apk upgrade ---> Using cache ---> fd105816bcb9 Step 3/4 : RUN apk add openssh ---> Running in c0fbeb4c8ea8 fetch fetch (1/6) Installing openssh-keygen (7.7_p1-r3) (6/6) Installing openssh (7.7_p1-r3) Executing busybox r1.trigger OK: 8 MiB in 19 packages Removing intermediate container c0fbeb4c8ea8 ---> 3ba d0 Step 4/4 : ENTRYPOINT ["/bin/echo","hello, France!"] ---> Running in ead00e7c9ada Removing intermediate container ead00e7c9ada ---> 1d451e42e9e2 Successfully built 1d451e42e9e2 Successfully tagged hello:latest 31

32 Build your own image Sending build context to Docker daemon 5.632kB Step 1/4 : FROM alpine:latest ---> 196d12cf6ab1 Step 2/4 : RUN apk update && apk upgrade ---> Using cache ---> fd105816bcb9 Step 3/4 : RUN apk add openssh ---> Running in c0fbeb4c8ea8 fetch fetch (1/6) Installing openssh-keygen (7.7_p1-r3) (6/6) Installing openssh (7.7_p1-r3) Executing busybox r1.trigger OK: 8 MiB in 19 packages Removing intermediate container c0fbeb4c8ea8 ---> 3ba d0 Step 4/4 : ENTRYPOINT ["/bin/echo","hello, France!"] ---> Running in ead00e7c9ada Removing intermediate container ead00e7c9ada ---> 1d451e42e9e2 Successfully built 1d451e42e9e2 Successfully tagged hello:latest root@host:/tmp# docker image ls --all REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> 3ba d0 5 minutes ago 11MB hello latest 1d451e42e9e2 5 minutes ago 11MB <none> <none> b4277a54d78b 24 hours ago 5.71MB <none> <none> fd105816bcb9 24 hours ago 5.71MB alpine latest 196d12cf6ab1 2 months ago 4.41MB root@host:/tmp# docker image prune WARNING! This will remove all dangling images. Are you sure you want to continue? [y/n] y Deleted Images: deleted: sha256:b4277a54d78bc870ba2c1e971f076da6a324cc73f97dead 788eb c9e5e Total reclaimed space: 0B 32

33 Thanks for your attention