CONTAINER AND MICROSERVICE SECURITY ADRIAN MOUAT

CONTAINER AND MICROSERVICE SECURITY ADRIAN MOUAT

Chief Scientist @ Container Solutions Wrote "Using Docker" for O'Reilly 40% Discount with AUTHD code Free Docker Security minibook http://www.oreilly.com/webops-perf/free/dockersecurity.csp @adrianmouat

OVERVIEW The Benefits of Security Container Attack Vectors Security Philosophy Demo Tips & Techniques

THE BENEFITS OF SECURITY

CONTAINER ATTACK VECTORS

KERNEL ATTACKS

DENIAL OF SERVICE

CONTAINER BREAKOUTS

POISONED IMAGES

SNIFFING SECRETS

SECURITY PARADIGMS

DEFENCE-IN-DEPTH Multiple layers of security

LEAST PRIVILEGE Only access data and resources essential to function "Least Privilege Microservices" by Nathan McCauley and Diogo Mónica

DEMO

SO WHAT NOW? Ideally have guidelines for procedure Need to isolate container (and probably host) docker network disconnect Don't delete, preferably don't stop if safe docker diff

HOW TO MITIGATE Run container with less privileges --read-only Use non-privileged user...

NOT A SOLUTION! Still allows remote execution of arbitrary JS Real solution is to replace vulnerable library Image should be scanned for known vulns

IMAGE SCANNING Docker Security Scanning Other solutions Clair from CoreOS Peekr from Aqua Security Twistlock Atomic Scan from Red Hat

DEPENDENCY CHECKERS OWASP Dependency Checker Node Security Project (NSP)

TIPS & TECHNIQUES

USE CONTAINERS AND VMS Use VMs to segregate groups of containers For multitenancy Each user's containers in separate VM For different security levels Containers processing CC details in own VM

ASIDE: DIRTY COW (CVE-2016-5195) Recent vulnerability found in the kernel Allows privilege escalation Can be used to break out of containers Also breaks read-only filesystems https://dirtycow.ninja/ https://blog.paranoidsoftware.com/dirty-cow-cve-2016-5195-docker-container-escape/ docker run --rm amouat/dirty-cow-test

SEGREGATE BY NETWORK Use multiple "logical" networks e.g. backend, frontend frontend should not be able to backend network "link" container will be in both docker network create frontend

DOCKER PRIVILEGES == ROOT PRIVILEGES

Can mount any directory Can create and copy out "backdoors" docker run -v $PWD:/data debian /bin/sh -c \ 'cp /bin/sh /data/ && chown root.root /data/sh && chmod a+s /data/sh'

USER NAMESPACING By default, there is no user namespacing Root in container is root on host Don't run apps in a VM as root Same goes for containers

USER NAMESPACING Can be turned on since 1.10 Maps users in containers to high-numbered users on host Set on daemon, not per container Due to complications with ownership of image layers

GOTCHAS Problems with volumes and plugins Can't use --pid=host or --net=host Can't use read-only Restrictions on some operations (e.g. mknod)

SET A USER Create a user in your Dockerfile Change to the user via USER or su/sudo/gosu RUN groupadd -r user && useradd -r -g user user USER user

BE CAREFUL WHEN DELETING DATA IN DOCKERFILES

THIS DOESN'T WORK FROM debian RUN apt-get update && apt-get install -y curl RUN curl http://sourcecode.com/file.tgz -o /file.tgz RUN tar xzf /file.tgz && make RUN rm /file.tgz

THIS DOES FROM debian RUN apt-get update && apt-get install -y curl RUN curl http://sourcecode.com/file.tgz -o /file.tgz && tar xzf /file.tgz && make && rm /file.tgz

# Copy github ssh key COPY github_rsa /root/.ssh/id_rsa... # Remove ssh key RUN rm /root/.ssh/id_rsa AND THIS IS REALLY BAD

SET CONTAINER FS TO READ-ONLY $ docker run --read-only debian touch x touch: cannot touch 'x': Read-only file system

SET VOLUMES TO READ-ONLY $ docker run -v $(pwd)/secrets:/secrets:ro \ debian touch /secrets/x touch: cannot touch '/secrets/x': Read-only file system

DROP CAPABILITIES $ docker run --cap-drop SETUID --cap-drop SETGID myimage $ docker run --cap-drop ALL --cap-add...

$ docker run -d myimage $ docker run -d -c 512 myimage $ docker run -d -c 512 myimage SET CPUSHARES

$ docker run -m 512m myimage SET MEMORY LIMITS

DEFANG SETUID/SETGID BINARIES Applications probably don't need them So don't run them in production

TO FIND THEM $ docker run debian \ find / -perm +6000 -type f -exec ls -ld {} \; 2> /dev/null -rwsr-xr-x 1 root root 10248 Apr 15 00:02 /usr/lib/pt_chown -rwxr-sr-x 1 root shadow 62272 Nov 20 2014 /usr/bin/chage -rwsr-xr-x 1 root root 75376 Nov 20 2014 /usr/bin/gpasswd -rwsr-xr-x 1 root root 53616 Nov 20 2014 /usr/bin/chfn...

TO DEFANG THEM FROM debian:wheezy RUN find / -perm +6000 -type f -exec chmod a-s {} \; \ true

RESULT $ docker build -t defanged-debian.... Successfully built 526744cf1bc1 $ docker run --rm defanged-debian \ find / -perm +6000 -type f -exec ls -ld {} \; \ 2> /dev/null wc -l 0 $

USE MINIMAL IMAGES Less software Less attack surface

Alpine Linux https://hub.docker.com/_/alpine/ Static binaries Go makes this easy https://medium.com/iron-io-blog/an-easier-way-tocreate-tiny-golang-docker-images-7ba2893b160

USE LINUX SECURITY MODULES

SELINUX By NSA! Policy based MAC not DAC File access, sockets, interfaces

PITA Hard to define own policies Have to use devicemapper Extra work to use volumes

$ sestatus grep mode Current mode: enforcing $ mkdir data $ echo "hello" > data/file $ docker run -v $(pwd)/data:/data debian cat /data/file cat: /data/file: Permission denied

$ ls --scontext data unconfined_u:object_r:user_home_t:s0 file $ chcon -Rt svirt_sandbox_file_t data $ docker run -v $(pwd)/data:/data debian cat /data/file hello

APPARMOR Used by Debian & Ubuntu On by default Limits container access to host files and kernel capabilities Can pass in own policy for a container Process based; not as fine-grained as SELinux

ALSO A PITA, BUT...

BANE Project by Jessie Frazelle Simplifies creating AppArmor profiles

SECURITY HARDENED KERNEL Patched kernel with security enhancements grsecurity PaX Lag behind latest kernel version

VERIFY IMAGES Know what you're running And where it came from Only use automated builds, check Dockerfile Docker Content Trust Pull by digest

AUDITING Immutable infrastructure Audit images, not containers Docker diff Scanning tools scalock, twistlock, clair

SHARING SECRETS

BAKE IT INTO THE IMAGE

ENVIRONMENT VARIABLES $ docker run -e API_TOKEN=MY_SECRET myimage Suggested by 12 factor apps Can be seen too many places linked containers, inspect Can't be deleted Get included in reports

MOUNTED VOLUMES OR DATA VOLUME CONTAINERS $ docker run -v /secretdir/keyfile:/keyfile:ro myimage $ docker run --volumes-from my-secret-container myimage Works, but icky Files can get checked in by accident

SECURE KEY-VALUE STORE Docker 1.13 in Swarm Mode https://github.com/docker/docker/pull/27794 Kubernetes Secrets Vault https://hashicorp.com/blog/vault.html Can control leases, store encrypted

CONCLUSION Containers Add isolation Provide tools for restricting attackers Use with VMs if concerned Think Defence-In-Depth & Least Privilege

THANK YOU!

Chief Scientist @ Container Solutions Wrote "Using Docker" for O'Reilly Free Docker Security minibook https://www.openshift.com/promotions/dockersecurity.html @adrianmouat