4 Effective Tools for Docker Monitoring. By Ranvijay Jamwal

Transcription

1 4 Effective Tools for Docker Monitoring By Ranvijay Jamwal

2 CONTENT 1. The need for Container Technologies 2. Introduction to Docker 2.1. What is Docker? 2.2. Why is Docker popular? 2.3. How does a Docker Environment look like? Docker monitoring 4. 4 key tools to monitor Docker 4.1. AWS CloudWatch 4.2. Ruxit 4.3. New Relic 4.4. Sysdig Conclusion 6. About the Author 7. About TO THE NEW

3 1. The Need for Container Technologies What are the reasons to opt for container technologies? Faster/easier deployment Flexibility in deployment Better isolation Architectural reasons (microservices) Cost saving Other Share of respondents Image 1: Why to use container technology? Source: RUXIT There are many reasons why one would opt for container technologies. As the graph shows, the most common one is faster/easier deployments. This is definitely what Docker and other container technologies come packed with. Container technologies also bring in the required amount of isolation along with saving costs. For example, you can work inside a Docker container in development environment, once the new code is running successfully push it as an image over Docker Hub. Now, pull it on a different server, maybe production, run the container and the application will be up and running with the same code and configuration. 2. Introduction to Docker 2.1. What is Docker? Docker is defined as - Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries anything you can install on a server. 3

4 This guarantees that it will always run the same, regardless of the environment it is running in. In simple words, it is something that makes everyday tasks easy for a DevOps Engineer. It is basically an isolated environment that can be achieved by launching containers on a host machine. Containers are the heart and soul of Docker. It can be used to work inside containers, have services or packages inside the containers or may be reuse the containers on different host machines using their images. It just quickens up the process of development and deployment Why is Docker Popular? Of what we know about Docker, it just changes the way you work. To start doing any work on a Linux machine you need to install a package or a service and it may have some dependencies. Now, while solving these dependencies, the user might end up spoiling some other services. With Docker you can go ahead run a container, experiment as much as you want and it won t cost the host machine anything other than resources. It makes the entire process from development to production very simple in the form of pushes and pulls and all this is facilitated via the Docker Hub. Which container technologies are you using or do you plan to use? Docker containers LXC (Linux Containers) rkt containers Other Share of respondents Image 2: Popularity of Docker Source: RUXIT The graph (image 2) clearly shows the popularity of Docker over other containers including LXC, and rkt containers. Docker tops the list distinctly with more than 75% opting for it. All the megacorps are investing in Docker because of its ease-of-use, lightweight architecture, security, ease of portability and advanced features like a Dockerfile, central repository - Docker Hub etc. LXC and other container technologies lack many of these features. 4

5 2.3. What does a Docker Environment look like? Docker Environment A Docker environment looks somewhat like this. Containers and images are what make up Docker along with a number of utilities. So, a Dockerized environment starts with pulling an image from the Docker registry and this pull call is made via the host. Once that is done, these images are brought to life using something called containers using Docker commands. So, these containers give an isolated environment on top of the the host machine. All the work is then done inside the containers. Once the work is complete, these containers are pushed to the Docker registry (Docker Hub) to be used again or on another host machine. 3. Docker Monitoring Everyone in the Docker world is concerned about Docker Monitoring. How to monitor Docker container? Before coming to that we should discuss why is Docker monitoring important? Docker Monitoring mainly helps in the following: 1. Monitoring Container status and resources 2. Application performance inside the container 3. Find and fix application performance issues 4. DB container Queries and performance All of the above will help your business to grow. 5

6 What are the biggest challenges in moving to a containerized infrastructure? Technology maturity Orchestration Monitoring Automation Enviornment size Other Share of respondents Image 3: Challenges in moving to a containerized infrastructure Source: RUXIT The graph (image 3) depicts the biggest challenges faced by people when they plan to move to a Dockerized environment. Keeping aside technology maturity and orchestration which are solved if one uses Docker, Docker monitoring is a challenge. Containers are usually large in numbers and they come and go very frequently which sometimes makes it more difficult to manage and monitor them than the host machines. Choosing the right type of monitoring is necessary, whether one wants white box monitoring (all insights of Docker) or black box (Just the outer overview of Docker Resources). Thus, Docker monitoring is something without which you cannot proceed while moving ahead with Docker. 4. Four key tools to monitor Docker Now that we have looked at the importance of Docker monitoring, let us look at the ways in which this can be achieved. There are a lot of ways to monitor one s Docker containers. A few helpful and easy ways are discussed here. AWS CloudWatch New Relic sysdig Ruxit 6

7 4.1. Monitoring Docker using Shell and AWS Cloudwatch. The most simplest way to monitor Docker containers is using the Docker commands like docker stats. While using this method one can understand what values we can get out of the box. $ docker stats redis1 redis2 CONTAINER MEM USAGE / CPU % MEM % NET I/O BLOCK I/O LIMIT redis1 0.07% 796 KB / 64 MB 1.21% 788 B / 648 B MB / 512 KB redis1 0.07% MB / 64 MB 4.29% KB / 648 B 12.4 MB / 0 B Few commands like cat /sys/fs/cgroup/blkio/docker/$container_id/ cat /sys/fs/cgroup/cpu/docker/$container_id/cpu.stat can give values of I/O and CPU usage respectively. You can use these values and push them to AWS CloudWatch to monitor the container s stats. So, this is a method which you can follow if you are working in the AWS cloud, ready for some overhead and want to spend minimal time and cost on Docker monitoring. Other container stats like memory, number of connections, application stats etc. can also be monitored using shell and AWS CloudWatch Now, there are not many tools that monitor whether the Docker container is running or not. You cannot afford not knowing when a container goes down or rather stops. A tool like Monit can be used to keep the Docker service up all the time. Also, containers can be run with the switch --restart= always. For monitoring container status one can check whether the required Docker container is up using a simple bash script. This script would consider that the Docker service is always running. You can then run this script every minute. For each minute, if the container is up, the count remains 1 (if there is a single container) in the use case and is pushed to AWS CloudWatch. In AWS CloudWatch an alarm will be triggered if the count <1 followed by an SNS notification. This is the most simplest way to monitor the Docker containers. You end up paying very minimal but this works only if the servers are in the AWS cloud. Here s a step by step guide on how to monitor Docker container status using AWS CloudWatch. 7

8 4.2. Ruxit A more sophisticated way of monitoring applications in Docker containers doesn t require users to touch their images at all. Ruxit provides such a solution. It seamlessly integrates with existing Docker environments and automatically detects the creation of containers. Upon creation of a new container on a Docker host, Ruxit Agent hooks into the container and provides code for injecting the respective sensor into the containerized processes. So, it gives deep insights of what is running inside containers and how the services are performing. Also, what resources are being used by those containers? It not only monitors Docker containers but can be used for application monitoring as well. Here s a step by step guide on how to monitor Docker using Ruxit New Relic New Relic is a software analytics company that attempts to derive real-time business insights from the metrics produced by software. Some of these include user click streams, mobile activity, end user experiences, and transactions. New Relic is a very popular online application which also provides Docker Monitoring. New Relic provides various features like Alerts, Application performance monitoring(apm), server monitoring, Plugins, Browser, Synthetics (user behavior) etc. Here s a step by step guide on how to monitor Docker using New Relic. 8

9 4.4. Sysdig Sysdig is a simple tool for deep system visibility, with native support for containers. Sysdig not only monitors the basic things like CPU, memory but provides the whole insight of your application with metrics like JVM, connections, DB queries etc. It is an application monitoring tool for Docker containers in all means. Sysdig has been one of the most advanced cloud-based tools for monitoring the infrastructure and application. So, talking about Docker monitoring, Sysdig gives a lot of insights about containers, few of which are going to be discussed. Here s a step by step guide on how to monitor Docker using Sysdig. 5. Conclusion Container technology has gained a lot of popularity in the last couple of years. Containers provide an escape from software dependencies and scaffold for scalable software architectures. One tends to use containers in large numbers which might become operationally difficult to handle. Many stacks today which are using containers do not have proper monitoring set up. This creates a huge blind spot and leaves the systems vulnerable to downtime. Therefore: One should monitor all layers of the stack together, so what is happening everywhere, can be seen at the same time without any gaps One should go ahead with in-depth monitoring of the containers Tag the containers which helps one to filter and categorize them One can restrict the amount of resources which are used by any Docker container Try to use one application per container Docker is the most popular container technology because of it s ease-of-use and pace of development. Many of the Global Fortune 500 companies have invested a lot of money in Docker keeping in mind what it will reap for them in the near future. Docker tries to push new versions every quarter with totally new features. As of today, no other container technology is at par with Docker. 9

10 6. About the Author Ranvijay Jamwal Lead DevOps Engineer TO THE NEW Ranvijay is a lead DevOps Engineer, accredited technical certified professional who has excellent ability to develop & implement technical solutions for any size of business and believes in learning each day. He is a charismatic and cheerful person who loves Linux shell scripting. Ranvijay believes that cloud and container technologies will bring about a revolution. He is also a Microsoft Certified Technology Associate. Besides he is a hardcore cricket fan, who has played state level matches and also loves singing and writing. 7. About TO THE NEW TO THE NEW is a digital technology company that builds disruptive products and transforms businesses. We leverage the power of experience design, cutting-edge engineering, cloud and analytics led marketing to enable digital transformation. Our passionate team of 750+ people includes passionate technologists, digital analytics experts, video specialists and creative mavericks who have transformed businesses of more than 300 companies spread across 30 countries worldwide. We take pride in our culture which is driven by passion for making an impact through technology. 10

11 Explore Other Resources 7 Tools That Will Revolutionize DevOps DevOps Practices And Principles To Improve IT Efficiency Download Whitepaper Download Whitepaper 11

12 Lets Connect