JavaOne 2017 CON7957 Best Practices for Developing & Deploying Java Applications with Docker Eric Smalling - Solution Architect, Docker Inc. @ericsmalling
Who Am I? Eric Smalling Solution Architect Docker Customer Success Team ~25 years in software development, architecture, version control admin, etc ~10 years in build & test automation Docker user since pre-1.0 days Java developer since 1.1.x days 2
Agenda Docker 101 Running a simple Java web application in Docker Services, stacks & deploying to clusters Application management & troubleshooting Application Configuration Q&A
Docker 101 A quick overview of Docker
Some Docker vocabulary Docker Image The basis of a Docker container. Represents a full application Docker Container The standard unit in which the application service resides and executes Docker Engine Creates, ships and runs Docker containers deployable on a physical or virtual, host locally, in a datacenter or cloud service provider Registry Service (Docker Hub or Docker Trusted Registry) Cloud or server based storage and distribution service for your images
Docker File System Images, Layers & Containers Logical file system by grouping different file system primitives into branches (directories, file systems, subvolumes, snapshots) Each branch represents a layer in a Docker image Allows images to be constructed / deconstructed as needed vs. a huge monolithic image (ala traditional virtual machines) When a container is started a writeable layer is added to the top of the file system
Docker File System Containers & Copy on Write Super efficient: Sub second instantiation times for containers New container can take <1 Mb of space Containers appears to be a copy of the original image But, it is really just a link to the original shared image If someone writes a change to the file system, a copy of the affected file/directory is copied up
Docker File System What about data persistence? Volumes allow you to specify a directory in the container that exists outside of the docker file system structure Can be used to share (and persist) data between containers Directory persists after the container is deleted Unless you explicitly delete it Can be created in a Dockerfile or via CLI
Dockerfile - Linux + Java Example: Initial state
Image Layers Initial State Start tomcat Copy webapp Install Tomcat Download Tomcat Install JDK and curl Update apt catalogs Ubuntu Linux 16:04 Kernel
Building the image The docker client command build = build an image -t = apply a name and optional build Image name and optional tag Path to build context and Dockerfile
Running the image in a container The docker client command run = start a container --rm = delete container when it exits Image name and optional tag -t = run with a tty (for console i/o) -i = run in interactive mode These often are used in combination like this
Demo Build and run demonstration
Dockerfile - Linux + Java Example: Optimization step 1
Image Layers Optimization Step 1 Start tomcat Copy webapp Install Tomcat Download Tomcat Update apt catalogs, install JDK and curl, clean up Ubuntu Linux 16:04 Kernel
Dockerfile - Linux + Java Example: Optimization step 2
Image Layers Optimization Step 2 Start tomcat Copy webapp Install Tomcat Download Tomcat Update apk catalogs, install curl OpenJDK:8-alpine Kernel
Dockerfile - Linux + Java Example: Fully Optimized
Image Layers Fully optimized Copy webapp tomcat:8.5-alpine Kernel
Deploying to Clusters Services, Stacks and Swarms
More terminology Swarm A group of docker hosts, connected and running as a cluster 1-n managers 1-n workers Service An application (or part of an application) that provides a specific function (catalog lookup, web front end, payment processing) Stack A way of representing multi-service applications Made up of 1-n services
Stack deploy demo Simple J2EE application deployment with 2 containers: React based front end Java based back end
Application Management Monitoring & Troubleshooting
Health Checks Helping Docker help you HEALTHCHECK instruction in DockerFile Tells Docker how to test a container to check that it is still working New status added to container lists Adds (healthy) to Status column in a docker ps response
Health Checks Helping Docker help you Examples: HEALTHCHECK CMD curl --fail http://localhost exit 1 HEALTHCHECK --interval=12s --timeout=12s --start-period=30s \ CMD node /healthcheck.js References: Documentation: https://docs.docker.com/engine/reference/builder/#healthcheck Elton Stoneman blog about not using curl/iwr: https://t.co/zgdd1lyzhk
JVM Memory Tips and tricks Always explicitly specify JVM heap size with -Xmx arguments By default, J2SE 5.0+ will use up to 25% of the host machine s RAM or 1GB (whichever is smaller) Container memory limits (enforced via cgroups) are ignored* (*cgroup awareness is planned for Java 9) It s just a good practice to specify it anyway Do use Docker cpu and memory reservations and limits to avoid over-subscribing your host machines --memory --memory-reservation --cpus etc If limiting cpu, be sure to update GC Thread limiter in JVM -XX:ParallelGCThreads
Logging Dealing with application logs Docker EE Reference Architecture document about this: http://dockr.ly/logging Do not output logs into the container s RW layer slow have to exec or cp out of the container to see them Option 1: send logs to stdout (see logging drivers below) Visible via docker logs command Visible via Docker UCP web console Option 2: send logs to volume Many use a centralized NAS/SAN volume for this Option 3: Docker logging drivers
Docker Log Drivers Log drivers available (as of 9/4/17) Latest always available at: https://docs.docker.com/engine/admin/logging/overview/#supported-logging-drivers
Application Log Drivers Consider the following when selecting application log drivers: syslog and splunk: Good options if log data is highly sensitive since they can be configured to use TLS for transporting logs. journald: great for retaining the usage of docker logs as well as logging Docker daemon logs allows for easier troubleshooting and log portability at the same time logs write first locally, so that there is less reliance on logging infrastructure. awslogs or gcplogs: Only if cluster exist solely on a single cloud provider
Application Log Drivers (continued) Consider the following when selecting application log drivers: gelf and fluentd: good choice if there's a NoSQL database somewhere in the environment where the logs can be stored. Again, see http://dockr.ly/logging for much more detail on logging.
Troubleshooting How to use Java tools with container based JVMs JVM command line tools via docker exec GC Stats: jstat --gcutil Heap dumps/histograms: jmap Expose JMX ports for jconsole or other utilities Intelligent health checks More than just port 8080 is listening Check third party monitoring tools for updated to be container aware i.e. Licensing issues with older monitoring tools because each container appears as a new host Also, docker specific commands/tools: docker stats ctop
Application Configuration Managing multi-environment config s
Application Configuration Deploying to disparate environments with identical images Build artifacts are your Docker images, not.war files or similar Build images in CI, store in registry, deploy same images everywhere Patterns and tools to deal with configuration differences Separate Stack yaml files Docker secrets Application configuration via volume mounts Third party configuration tools such as Consul and/or Vault consul-template Joyent Containerpilot Roll-your-own
Environment specific Stacks Different environment variable values Services that mock production endpoints db web service prod.yml dev.yml
Docker Secrets Stored encrypted in swam Exposed only to nodes that run services that need them Presented in container via RAM only tmpfs files never persisted to disk in encrypted format when container stops, secret is no longer present All communications between swam nodes via TLS, so secret never in the clear on the wire either Different secret values per environment using tags UCP can manage who/where secrets are available
Application configuration in volume mounts Use volumes that are only available in physical environment they apply to Contain environment-specific application configuration properties DO NOT store secrets in these (use Docker Secrets or other secure mechanism) You can bind mount files (doesn t have to be full directory structures)
Resources So much to talk about, so little time to do so!
Resources So much to talk about, so little time to do so! Docker Resources: https://www.docker.com/products/resources Logging Reference Architecture: http://dockr.ly/logging Training: https://training.docker.com Instructor led Self paced with Play With Docker Containerizing legacy applications? https://docker.com/mta SquareSpace Blog: Understanding Linux Container Scheduling (with JVMs) https://engineering.squarespace.com/blog/2017/understanding-linux-container-scheduling
THANK YOU :) @ericsmalling