Solution Brief February, 2019 AMD EPYC Delivers Linear Scalability for Docker with Bare-Metal Performance The AMD EPYC SoC brings a new balance to the datacenter. Utilizing x86 architecture, the AMD EPYC processor, brings together high core counts, large AMD EPYC Processor Meets the Needs of Container Architecture Workloads High Performance: Cores, Memory, I/O memory capacity, ample memory bandwidth and massive I/O with the right ratios to help performance reach new heights. Match core count to the workload Match core count with application needs without compromising processor features. EPYC s balanced set of resources means more freedom to rightsize the server configuration to the workload. Highly Scalable: Scale-up or Scale-out Scale-up or scale-out, AMD and its ecosystem partners offer highperformance network connectivity options for applications at massive scale. The AMD EPYC processor rapidly scales up to thousands of containers on a single Docker Engine node. EPYC System-on-Chip (SoC) performance scales linearly and uniformly across cores minimizing performance variation within applications. Designed from the ground up for a new generation of solutions, AMD EPYC implements a philosophy of choice without restriction. Choose the number of cores and sockets that meet your needs without sacrificing key features like memory and I/O. Each EPYC SoC can have from 8 to 32 cores with access to incredible amounts of I/O and memory regardless of the number of cores in use, including 128 PCIe lanes, and support for up to 2 TB of high speed memory per socket. Linear Scalability Containers on Docker Engine scale linearly on AMD EPYC, enabling admins foresight to balance workload and CPU resources with SLA requirements. Create Agile, Scalable Applications Use the Docker platform to create agile, scalable applications that take full advantage of available hardware without consuming the resources of a full virtual machine stack. The Docker Platform Enables Exceptional Scalability Taking Full Advantage of Resources Scale up or down containers quickly to meet spikes in demand or to repurpose resources for other applications. 2019 Advanced Micro Devices, Inc. EPYC s innovative architecture translates to terrific performance at a low cost. More importantly, the performance you re paying for is appropriate to the performance you need. The Docker Platform Enables New Application Architectures The Docker platform is available as both an open-source and enterpriseready container platform for packaging, distributing, and managing applications within containers. Containers allow developers to package up an application and all of the parts, such as libraries and other dependencies, and deliver it as a single package. Docker Engine, the foundation of the Docker platform, runs on virtually any server or virtualized machine that supports the container's runtime environment. Applications are not tied to the host operating system and can be moved easily from local systems to cloud environments. 1
Docker Platform Architecture The Docker container platform has many advantages, including: o Rapid Application Development: Docker Engine has a minimal runtime environment that does not boot an OS. Multiple containers can be quickly deployed in a matter of seconds. o Portability: A Docker container image is independent of the host operating system version and runs the same no matter which machine, eliminating compatibility issues that can arise from environment settings. o Minimal Overhead: Docker Engine s lightweight runtime environment and typically very small images, compared to a virtualized guest, facilitates rapid development and reduces application deployment time. Without the full virtualized stack, performance for some workloads are close to bare-metal. o Multi-Cloud Platforms: Major cloud computing providers, including Amazon Web Services (AWS) and Google Compute Platform (GCP) support Docker containers. o Isolation: Applications and resources are isolated and segregated. Docker containers only use resources that have been assigned to them, enabling applications to run completely different stacks. o Efficiency and Sharing: The Docker platform enables users to build a container image that can be used across every stage of the deployment process. A container image registry enables users to share images with others in or outside the organization. Users can leverage Docker Hub for a public repository of container images including MySQL-Server, Phoronix, WindowsServerCore, Ubuntu, etc. Organizations can also setup Docker Trusted Registry for a secure and private registry solution designed to meet enterprise needs like access controls, security scanning, image signatures and automated workflows. o Enterprise Security and Access Controls: Enterprise organizations integrating Docker containers into their datacenter can upgrade to Docker Enterprise - a complete container management platform for securing, managing, deploying containerized applications across any infrastructure. Docker Enterprise includes Docker Engine - Enterprise, Docker Trusted Registry, and a universal control plane for managing your Docker environment. With the Docker platform s many key advantages, businesses have been able modernize monolithic or traditional applications and transition to a container based solution. Most business applications consist of several components organized into stacks: web server, database, and inmemory cache. Containers make it possible to compose each component into functional units that can be maintained and updated independently. The Docker platform is a key technology for enabling this type of application design which is called the microservice model. See Figure 1. 2019 Advanced Micro Devices, Inc. 2
Docker Platform Architecture (cont.) Figure 1: Docker Architecture The Docker platform architecture enables rapid application development by making it easy to spawn new containers in parallel or sequential mode to handle spikes in demand. When an application or service running inside separate containers work independently of each other and require no coordination between themselves, scaling up using parallel mode is the quickest method to meet bursts in demand. Examples of such applications include: Stateless web servers and proxies Worker instances that process jobs from a queue Cron -style instances that execute periodic tasks Some services require coordination between different containers to ensure that the service functions correctly. Microservice based solutions typically consist of multiple individual components that can be run across a variety of platforms. Scaling up this solution could involve orchestrating a number of different components and services to ensure that they can still integrate with the rest of the system. For example, many databases, such as MySQL, require that the containers know about each other at startup time so that traffic can be routed to them appropriately. The Docker platform provides the ability to scale this type of service with either Swarm Manager in sequential mode or through Kubernetes autoscaling. 2019 Advanced Micro Devices, Inc. 3
Docker Engine Scales Linearly on AMD EPYC AMD EPYC provides superior core density and flexibility to scale Docker-based applications up or down to meet spikes in demand or conserve system resources. This experiment mimics a Cron type container instance to demonstrate scalability of Docker Engine on EPYC. The container image has a single task that runs a Sysbench CPU test. Memory and I/O transactions were intentionally kept low to focus solely on CPU response time. Sysbench is an open source benchmarking tool that is used for evaluation of databases and general system performance. It is a tool for testing CPU usage, memory usage, and disk IO and MySQL performance. The test selected here is CPU usage: sysbench --test=cpu --cpu-max-prime=200000 In this mode, the test calculates the prime numbers up to the cpu max prime parameter using a single thread. Computing performance can be measured by how many operations the system can process within a given time (events/sec) or by how long a certain task takes to complete. The selected test took approximately 7 minutes to complete on a single CPU core. Docker Swarm was then used as the orchestration mechanism to create and deploy replicas of the image for this scaling test. All containers were scaled in parallel and ran concurrently. We were able to quickly spin up 4, 8, 16, 32, 64, 128, 256, 512, and 1024 containers on a single node Swarm cluster. The value of cpu max prime was purposely selected so that the test does not complete too quickly, ensuring all containers are still running when the last one started. The metric for each scale up run is total execution time for all containers within a batch to complete. The results are shown in Figure 2. Containers Total Execution Time (s) 4 454 8 480 16 509 32 579 64 586 128 1015 256 2095 512 4033 1021 8130 Figure 2: Typical Expected Trend Line Linear Scaling of Docker Containers on AMD EPYC CPU response time increases, in a linear path, when all cores becomes fully saturated and the number of concurrently running containers continue to scale up. For CPU bound workloads on EPYC, this feature enables system administrators to calculate how much to over provision depending on the Server Level Agreement (SLA) for the system. 1024 is not the ceiling for the number of Docker containers on EPYC. Rather for this experiment, a hypothetical SLA requirement was imposed that all tasks should complete within approximately two hours. Interpolating the results further, if the requirement was that all tasks could complete within 5 hours, then the number of containers could scale to 2056 and so on. These tests were performed with Docker s Community Engine, but results are expected to be consistent with Docker s Enterprise Engine as they share a common runtime architecture. 2019 Advanced Micro Devices, Inc. 4
Docker Engine Scales Linearly on AMD EPYC (cont.) The hardware and software configuration for this experiment is shown in Table 1. HARDWARE BIOS CFG MASTER NODE 1 x OEM SERVER PLATFORM WITH: o 2 x AMD EPYC 7601 SOC (64 Cores / 128 Threads) o 512 GB DDR4-2400 MHz (16 x 32 GB) Memory o 1 x 250 GB SATA SSD (Operating System) SMT enabled OPERATING SYSTEM RHEL Server 7.6 SOFTWARE DOCKER ENGINE - COMMUNITY (CE) 18.09 DOCKER IMAGE CONTAINER APP NUMBER of CONTAINERS UBUNTU:16.04 sysbench --test=cpu --cpu-max-prime=200000 run 4, 8, 16, 32, 64, 128, 512, 1024 TEST Table 1: Test Configuration for Linear Scaling https://github.com/akopytov/sysbench#linux 2019 Advanced Micro Devices, Inc. 5
Near Bare-Metal Performance on AMD EPYC and Docker Engine AMD EPYC takes advantage of the Docker platform s minimal overhead and isolation capabilities. Docker Engine, which facilitates deployment of containers, has a small footprint that doesn t boot an operating system. All containers on the same host share the host kernel. With this thin layer between the host and application running inside the container, workloads can take advantage of EPYC s CPU cores, large memory capacity, and I/O bandwidth to achieve near bare-metal performance results. For this experiment, performance results of workloads such as, Phoronix/Stream and MySQL, were compared between two environments, bare-metal and Docker Engine. The Phoronix Test Suite is a benchmarking platform for Linux, Solaris, MacOS, and BSD operating systems. It is also an extensible test framework with more than 200 tests available by default. Of the available default, there are a range of subsystems, such as CPU, memory, file systems, and kernel, that can be tested. Phoronix Stream benchmark is designed to test the sustainable memory bandwidth (MB/s) and corresponding computation rate for four simple vector kernels (Copy, Scale, Add, and Triad). Of the four vectors, this experiment is concerned with the metric for Triad because it is the most complex. For testing MySQL performance, we used Sysbench. The tool covers benchmark cases such as OLTP (Online Transaction Processing) workloads, read-only or read-write, primary key lookups, and primary key updates. For this test, the focus was OLTP. Online transaction processing applications are high throughput and insert or update-intensive in database management. The key goals of OLTP applications are availability, speed, concurrency and recoverability. To eliminate inconsistencies, the software version, configuration files, and test parameters are identical for both bare-metal and Docker Engine test environments. From the results below, there is minimal performance overhead between them. The results are shown in Table 2. Percent Performance Difference of Docker Engine Environments vs. Bare-metal Benchmarks MySQL (QPS) Queries Per Second Bare-metal Raw Value Docker Engine (Default Network) Percent Raw Value Difference Table 2: Bare-Metal Performance vs. Docker Containers on AMD EPYC Docker Engine (No Network) Raw Value Percent Difference 159874.16 148337.15 7.211% 163364.38-2.18% MySQL OLTP.lua Read 26946570 25003454 7.203% 27535032-2.176% MySQL OLTP.lua Write 7607577 7059574 7.206% 7773108-2.176% MYSQL Total 1894063 1757574 1935284 Transactions 7.216% -2.183% Stream - Triad (MB/s) 90768.6 N/A N/A 91268.2-0.55% 2019 Advanced Micro Devices, Inc. 6
A typical use case for MySQL and Docker containers is in the testing and development environment. However, with little performance overhead, MySQL with Docker Engine could run in production as well, especially with the advanced security and management features of Docker Enterprise. The hardware and software configuration for this experiment is shown in Table 3. Memory encryption and SVM is not needed for this test. Disabling these options could lower latency for memory access. HARDWARE CONFIGURATION FOR DOCKER BENCHMARKS COMPARISON HARDWARE 1 x OEM SERVER PLATFORM WITH: o 1 x AMD EPYC 7551 SOC (32 Cores / 64 Threads) o 256 GB DDR4-2666 MHz (8 x 32 GB) Memory o 1 x 950 GB SATA SSD (Operating System) o 1 x 950 GB SATA SSD (Data) BIOS CFG AMD Secure Memory Encryption disabled AMD SVM (Virtualization) - disabled AMD SMT enabled OS RHEL Server 7.6 SOFTWARE Docker Engine Community (CE) 18.09 MySQL Community 8.0.14 DOCKER IMAGE OLTP OPTIONS TEST mysql/mysql-server:8.0.14 ltp_legacy/oltp.lua --threads=100 --oltp-table-size=100000 --maxtime=240 --max-requests=0 --report-interval=2 --oltp-testmode=complex --oltp-read-only=off https://github.com/akopytov/sysbench#linux Table 3: Test Configuration for Bare-Metal vs. Docker Performance Comparison 2019 Advanced Micro Devices, Inc. 7
AMD EPYC and Docker Engine Exhibit Minimal Performance Variation Across CPU Cores In addition to observing a negligible performance drop between bare-metal and Docker containers, AMD EPYC also demonstrates uniform performance across CPU cores. Minimal performance variation among cores is important especially for parallelized workload or Worker type container instances that process jobs from a queue. Both workloads are multi-threaded and span multiple cores. The impact of high CPU performance variation could lead to application inefficiency where a thread could block completion of a higher-level task or function. This is especially true for microservice based architecture where some or many application components are broken down based on the services or functionality they offer. Almost all applications/services, that do anything useful, are integrated with one or more service in this architecture. A microservice based solution has a high number of integration points, which could adversely impact the performance and stability of an application if not designed properly. In addition, large performance variation across CPUs could exacerbate the performance issue. Take for instance an application flow where five microservices are connected via REST over http. The application uses a common thread pool for maintaining their connections. If there s nonuniformity across cores, it could impact any one of the five services by causing it to respond slowly. This could eventually lead to all pool members being exhausted, while waiting for a response from the misbehaved service. For this experiment, the number of container images deployed concurrently was limited to the number of logical cores on the system. Again, the application in the container is a Sysbench CPU test. This Cron type of container instance was chosen because it s easier, from an application perspective, to consistently control the amount of load across all containers. Rather than specifically pinning a container to a CPU core, the kernel scheduler automatically assigns each container to a core. As long as there are available (idle) CPU cores in the system, each container will automatically be assigned a unique core. Sets of 4, 8, 16, 32, 64, and 128 containers were deployed concurrently. Within each set, performance results were individually tracked for all containers and compared. The following observations were made, shown in Table 4. Table 5 shows the test configuration. Number of Containers 4 8 16 32 64 128 % Variation < 1% < 1% < 2% < 9% < 5% < 12% Table 4: Performance Variation Across Cores (% represents difference between highest and lowest performance results among cores tested) 2019 Advanced Micro Devices, Inc. 8
AMD EPYC and Docker Engine Exhibit Minimal Performance Variation Across CPU Cores (cont.) The results show there is minimal performance variation across all CPU cores. Without any NUMA optimization, application/services in a microservice based solution can count on getting equal access to and expect consistent performance from each CPU core. HARDWARE MASTER NODE 1 x OEM SERVER PLATFORM WITH: 2 x AMD EPYC 7601 SOC (64 Cores / 128 Threads) 512 GB DDR4-2400 MHz (16 x 32 GB) Memory 1 x 250 GB SATA SSD (Operating System) OPERATING SYSTEM Ubuntu Server 18.04 BIOS CFG AMD SMT - enabled DOCKER Version: DOCKER ENGINE COMMUNITY (CE) 18.09 Image: Ubuntu:16.04 CONTAINER APP sysbench --test=cpu --cpu-max-prime=200000 run NUMBER of CONTAINERS 4, 8, 16, 32, 64, 128 TEST https://github.com/akopytov/sysbench#linux Table 5: Configuration for Performance Variation Test Software References SOFTWARE TOOLS and BENCHMARK REFERENCES DOCKER TESTS https://www.docker.com https://www.docker.com/hub https://www,phoronix-test-suite.com/ https://dev.mysql.com https://github.com/akopytov/sysbench#linux 2019 Advanced Micro Devices, Inc. 9
Conclusion In summary, businesses can leverage the Docker platform s portability, minimal overhead, isolation, and public images on Docker Hub to modernize their applications. The Docker platform s small footprint enables applications and services to quickly scale up or down to meet changing demands. Because the Docker platform has been embraced by large public cloud vendors such as AWS and GCP, businesses can easily migrate their services to the cloud. AMD EPYC highlights the benefits of Docker Engine through scale and performance. The results from three different experiments demonstrate how AMD EPYC processors can quickly scale thousands of containers on a single Docker Engine node with near bare-metal performance and minimal performance variation. LEARN MORE at amd.com/epyc and https://www.docker.com/ The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes. 2019 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo, EPYC and combinations thereof are trademarks of Advanced Micro Devices, Inc. Docker is a registered trademark of Docker, Inc. in the United States and/or other countries.. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies. 2019 Advanced Micro Devices, Inc. 10