Evaluation of cloud-based infrastructures for scalable applications

Transcription

1 LiU-ITN-TEK-A--17/022--SE Evaluation of cloud-based infrastructures for scalable applications Carl Englund Department of Science and Technology Linköping University SE Norrköping, Sweden Institutionen för teknik och naturvetenskap Linköpings universitet Norrköping

2 LiU-ITN-TEK-A--17/022--SE Evaluation of cloud-based infrastructures for scalable applications Examensarbete utfört i Datateknik vid Tekniska högskolan vid Linköpings universitet Carl Englund Handledare Peter Steneteg Examinator Pierangelo Dell'Acqua Norrköping

3 Upphovsrätt Detta dokument hålls tillgängligt på Internet eller dess framtida ersättare under en längre tid från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida Copyright The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances. The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non-commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: Carl Englund

4 Abstract The usage of cloud computing in order to move away from local servers and infrastructure have grown enormously the last decade. The ability to quickly scale capacity of servers and their resources at once when needed is something that can both be a price saver for companies and help them deliver high end products that will function correctly at all times even under heavy load to their customers. To meet todays challenges, one of the strategic directions of Attentec, a software company located in Linköping, is to examine the world of cloud computing in order to deliver robust and scalable applications to their customers. To this end, this thesis investigates the usage of cloud services in order to deploy scalable applications which can adapt to usage peaks within minutes.

5 Acknowledgments First of all I would like to thank Attentec for giving me the opportunity to perform my thesis at their company. It has been a rewarding journey where I felt extremely welcome by the entire company and I hope this thesis will help them improving their work against present and new customers in the future. Secondly I would like to specially thank my supervisor Martin Andersson at Attentec for interesting discussions and help during the master thesis. I would also like to thank my examiner at Linköpings University, Pierangelo Dell Acqua. Secondly thanks to all the people I have met during my studies. Sara, my classmates, people I have met through various student associations and all other amazing people at late nights and early mornings over a beer or ten. Finally I would like to thank my family for supporting me and encouraging me at all times through my studies. Norrköping, June 2017 Carl Englund v

6 Contents List of Tables Listings ix x 1 Introduction Background Attentec The Cloud Internet of Things Motivation Aim Research questions Delimitations Report structure Theory Cloud Computing Deployment models Service models Google Cloud Amazon Web Services Microsoft Azure Scaling in cloud environments Load testing Software Containers Docker Security in Docker Deploying to the cloud Kubernetes System Architecture Application deployed Related Work vi

7 3 Method Preparing a Docker image Creating Kubernetes Clusters Google Cloud Microsoft Azure Amazon Web Services Deploying the application Using the Kubernetes Dashboard Testing with Locust Results and Discussion Veryfing Autoscaling capabilities Discussion Cloud services Summarizing the Cloud Services Using Docker and Kubernetes Docker Kubernetes Security in Docker Results Conclusion Research questions Aim Future Work Bibliography 39

8 List of Figures 1.1 Gartner hype cycle[gartnerimg] Different service models within a cloud[cloudpyramid] Traditional hypervisor virtualization vs Container virtualization[hypcont] ARP spoofing attack[arpspoof] System Architecture Adding a deployment object in the Kubernetes Dashboard Launching virtual users in Locust Locust dashboard showing requests CPU and Users over Time for Google Cloud Replicas over time for Google Cloud CPU and Users over Time for AWS Replicas over time for AWS CPU and Users over Time for Azure Replicas over time for Azure viii

9 List of Tables.1 Loadtesting on Google Cloud Loadtesting on AWS Loadtesting on Azure ix

10 Listings 2.1 Example Dockerfile Building a Docker Image Running a Docker Image Listing running containers Listing running containers A Kubernetes configuration file Launching a Horizontal Pod Autoscaler Formula used for calculating the amount of Pods needed Dockerfile for the SIMS-application Uploading the Docker image to Google Container Registry Setting a computing zone Launching a cluster on Google Launching a cluster on Azure Launching a cluster on Azure A Kubernetes configuration file Launching a cluster with Kops Launching the Kubernetes dashboard Test cases developed with Locust. Filename loadtest.py Starting up Locust Kubernetes Deployment configuration file for the application replica Kubernetes Service configuration file for the application replica Kubernetes Deployment configuration file for the MongoDB replica Kubernetes Service configuration file for the MongoDB replica x

11 1 INTRODUCTION The following master thesis was carried out at Attentec AB and the Masters program of Media Technology and Engineering at the department for Science and Technology at Linköpings University. This introduction chapter will introduce the company in section In section 1.2 the motivation for the thesis is explained. Section 1.3 describes the aim of this thesis. Furthermore section 1.4 describes the research questions that will be answered at the end of the report. Delimitation s to the research questions are presented in section 1.5. Finally the structure of this report is explained in section Background Attentec Attentec AB[1], from now on known as Attentec is an IT-consulting company based in Linköping. They are focusing on Internet of Things, streaming media and modern software development for their various customers. They offer consulting services that add innovativeness and increased flexibility leading to successful projects and prosperity for their customers The Cloud The cloud and its perks has become a massive term within computer science in the past few years. The concept has however existed for several decades spanning back to when big computer machines were used together with time sharing, a technique still used today as well. Today cloud computing is all about running small and scalable services that have high elasticity and different usage peaks. It has become an alternative to running expensive and space consuming servers locally at companies. 1

12 1.1. Background Internet of Things Internet of Things from now on called IoT is often defined as a new era of computing [2] where technology leaves the often referred to as traditional computing and goes on to having devices everywhere in everyday life connected to a network. There can be connected microwaves, scales, sensor networks and million other devices alike. Cloud computing combined with IoT is a very powerful tool to assess and ease everyday life in where devices and sensors that require very little computing power or storage can be connected to the Cloud and send reports on demand. A survey by a company conducting technology research called Gartner[3] in July 2011 shows that Internet of things is a Technology trigger and will explode in the coming 5-10 years. Where as Cloud computing is on the edge of "Peak of inflated expectations" and only has 2-5 years until massive success. See Figure 1.1 Figure 1.1: Gartner hype cycle[4] 2

13 1.2. Motivation A more recent press release[5] from Gartner in 2016 seems to show that they were right about the massive success of Cloud Computing. It says that "By 2020, a corporate "no-cloud" policy will be as rare as a "no-internet" policy is today". Aswell as "Cloud will increasingly be the default option for software deployment. The same is true for custom software, which increasingly is designed for some variation of public or private cloud.". 1.2 Motivation Cloud computing is a market that has recieved a huge boost in the last couple of years. Imagine being able to scale an application within minutes of a usage peak, and when the peak has faded being able to scale your hardware down to the minimum amount needed. This is possible using today s cloud service techniques. Since most cloud services available today also uses the paradigm "pay for what you use" the costs of hosting applications on the cloud may also be significantly lower than hosting your own servers. Attentec has a wide range of customers with different needs and requirements. They need to be able to build large scalable applications which can handle a varied range of active users. Attentec can at the moment neither scale applications to their current needs nor provide sufficient support for large applications. In order to make it easier for both Attentec and their customers the usage of scalable cloud based solutions will be examined in this thesis. 1.3 Aim Attentec is right now hosting several of their customer s projects on their own locally based servers which have limited bandwidth and scaling capabilities. This produces limitation in terms of how many users Attentec can handle, and what kind of support they can. To overcome these limitations Attentec needs to be able to offer customers scalable applications that can grow in computing power depending on the amount of active users and the computing power they need. There is also the issue of guaranteeing the availability of the servers. An issue that could require Attentec to provide support around the clock. Many big corporations have solved similar problems employing cloud based hosting services, like Microsoft Azure[6], Google Cloud[7] or Amazon Web Services[8]. In order to easily be able to use these cloud services one can take advantage of containers to deploy applications. A container is something that wraps an application with all its dependencies in a box which can be deployed to several cloud services, the container will behave in the same way everywhere and the dependency problems 3

14 1.4. Research questions occurring when using different platforms can be avoided. The choice for this thesis is to use Docker[9] containers. and their flexibility to solve the problem. The aim of this Master Thesis is to provide Attentec guidelines on how to use and when they should deploy their application on a cloud service. And in some terms which hosting service might suit their needs the best. 1.4 Research questions 1. What kind of cloud services can and should be used to host scalable IoT applications? 2. How can these cloud services be tested for performance? 3. Are container techniques such as Docker suitable for deploying applications to the cloud? 4. How secure is it to host applications in Docker? 1.5 Delimitations In order to narrow down the project and make it possible to carry through during the 20 weeks allocated for the thesis, a number of delimitations had to be settled. The delimitations are set since the subject examined is vast and there may be no true answer to the research questions at hand. By narrowing them the hope is that it will be easier to answer the questions in a reasonable way. The cloud services that will be examined for the first issue will be limited to Amazon Web Services, Google Cloud, and Microsoft Azure. When discussing container techniques most time will be put on evaluating Docker. Other techniques may be examined, but may also be excluded from the final evaluation. Security aspects examined in the fourth issue will only be related to Docker. 1.6 Report structure Chapter 1 introduces Attentec, the problem as well as the aim for how to solve it. Chapter 2 describes the theory behind the intended solution. Chapter 3 describes how the the solution was implemented from a theory level to actual results. 4

15 1.6. Report structure Chapter 4 describes the results in the thesis. Chapter 5 offers a discussion of the results and how these were reached. Chapter 6 concludes the thesis, what have been done, what should have been done differently and what can be improved in the future. 5

16 2 THEORY This chapter will explain the theory behind the thesis and how everything correlates to each other. It will pinpoint important aspects in cloud computing, developing for the cloud and security aspects. It will also discuss virtualization techniques and how to deploy to the cloud. 2.1 Cloud Computing Cloud computing is according to the NIST[10] definition: A model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing services that can be rapidly provisioned and released with minimal management effort or service provider interaction. In short it is a way to swiftly provision new server resources when your application is experiencing high traffic, and scale down those resources when they are not needed. NIST mentions five essential characteristics of the cloud computing model which are: On-demand self service. A user can determine the wanted resources for their applications on-demand on their own by clicking a button. Broad network access. Services are available over the network and can be accessed through standard web browsers and do not have to be high resource demanding products. Resource pooling. Means that the provider of the service has pools of computing resources. Each server can serve multiple customers utilizing the computing power of entire machines instead of having one machine per user. 6

17 2.2. Deployment models Rapid elasticity. Resources can be provisioned elastically, this can be done using for example a loadbalancer that performs it automatically and increases the computing power of the customers service when needed. And scaling resources down when not needed. Measured service. Monitoring the customers service in order to display the resources used and when and how peaks of usage happens. 2.2 Deployment models A cloud can have several different deployment models for you to host and deploy your application. According to NIST[10] the following models are the most commonly used. Private Cloud A private cloud is a cloud set up for exclusive use by a single company or organization. It can be owned and operated by this organization, a third party or some sort of combination of these. Community Cloud A community cloud is set up to serve a specific community of organizations that have a common interest in the technique. Public Cloud A public cloud is open for use by the general public, it can be owned by a business, academic or government organization. Hybrid Cloud A hybrid cloud is a mix of two or more of the above deployment models, they can be run differently for each user but are bound together to enable standardized data storage or application portability. 2.3 Service models When deploying or using services in the cloud there are several different service models to cover different users needs and usages. The most common ones being used today can be divided into three different services[11] and their hierarchy can be visualized in a simple pyramid seen in figure

18 2.3. Service models Figure 2.1: Different service models within a cloud[12] Infrastructure as a Service Infrastructure as a service (IaaS) means that the consumer has access to alter the underlying infrastructure as they wish. Customers can choose what environment they want to run on the machine, for example operating system, databases, languages and frameworks. They can also provision processing power, storage, networks and other computing resources to their liking and usage. Companies offering IaaS are for example Amazon through AWS, Google through Google Cloud or Microsoft with their service Azure. Platform as a Service Platform as a service (PaaS) is different from IaaS in that the consumer does not control the underlying infrastructure of the service. They do however have control of deployed applications and possibly configuration settings for the hosting environment. As well as provisioning of processing power, memory, storage and network. Examples of different PaaS providers are Heroku[13] and OpenShift[14]. These providers let users deploy repositories with code of different languages making their services available through the Internet. Software as a Service Software as a Service (SaaS) is when the user does not have control of either the infrastructure or the platform being used. Software as a service can be a software being run in the cloud for the intended customer to use and interact with. Examples of different SaaS providers are Google, Facebook or ShareLatex. Users can for example handle documents, communicate with each other or utilize planning tools available through the service. 8

19 2.4. Google Cloud 2.4 Google Cloud Google Cloud is a cloud service developed by Google. It was ranked the highest in the general ranking of five cloud services from JDN/CloudScreener/Cedexis U.S. Cloud Benchmark[15]. Google also received the highest rank in terms of pricing in the same article, this is considered to be due to the per-minute-billing model that Google employs as well as a automatic discount that occurs for increased usage of the service. In terms of regions available Google has the lowest amount of regions compared to Amazon Web Services and Microsoft Azure with a total of 9 regions available today. But they are expanding and throughout 2017 they will add 11 more regions to this number[16]. Regions are something that can be important in terms of latency for some applications. 2.5 Amazon Web Services Amazon Web Services is a cloud service developed by Amazon. In the JDN/- CloudScreener/Cedexis U.S. Cloud Benchmark[15] it ranked second closely following Google Cloud. It does also rank second in the pricing ranking of the same article, but they do not use the same per-minute-billing model as Google. Instead Amazon are using a per-hour-billing model meaning that every usage is rounded up to the closest hour which might increase the pricing for deploying an application to their cloud if used for a limited amount of time. Amazon has the most regions out of the three cloud services evaluated with 16 regions across the world and 3 more coming up[17]. They are also the only cloud service examined with a region coming up in Sweden in 2017[18]. 2.6 Microsoft Azure Microsoft Azure is a cloud service developed by Microsoft. It ranked fourth in the JD- N/CloudScreener/Cedexis U.S. Cloud Benchmark[15] general ranking. In the pricing ranking it also ranked fourth. Azure however uses the same billing model as Google where users are billed per-minute used of their cloud service. In terms of regions Azure has the most regions with a total of 34 regions all over the world and 6 more regions planned[19]. 9

20 2.7. Scaling in cloud environments 2.7 Scaling in cloud environments In cloud environments there are several ways to scale an application. For example the most common way is probably to scale the infrastructure being used by either scaling vertically which means adding more power to the server. Or scaling horizontally which means adding more servers to serve your application[20]. One can also scale for example containers that are being used on the infrastructure by assigning more power to these, or spinning up more containers. The different scaling techniques can be performed manually or most commonly by using an automated load balancer. With this tool users can define different thresholds for when to scale and how to scale Load testing There are a lot of tools to load test applications on the web. Some of these are wrk[21], Apache JMeter[22], Tsung[23] and Locust[24]. The choice for this thesis was to use Locust since it is easy to setup and has the capability of running the test distributed over multiple machines. Locust uses Python[25] to define user behaviour, and has the capability to swarm a service with million of users. It can be monitored through a web interface. It is designed to help companies test their services before sending them out to production, and identify bottlenecks or problems occurring when the load of the service reaches large levels. Locust will be used to test the load balancing capabilities of all cloud services used in this thesis. 2.8 Software Containers Containers are an effective way of encapsulating an application. It is possible to use the same environment to develop, test and deploy an application. Comparing containers to traditional hypervisor[26] solutions of virtualization there are some significant differences. 10

21 2.8. Software Containers Figure 2.2: Traditional hypervisor virtualization vs Container virtualization[27] In figure 2.2 some differences between a hypervisor virtualization compared to a container based virtualization can be seen. The different parts of the figure are described below. First of all, traditional hypervisors provides access to a system s hardware, meaning the programmer still has to install operating systems and handle the amount of resources being given at a certain time. This represents the hypervisor block in figure 2.2. Containers on the other hand use protected portions of the current operating system, meaning they virtualize on top of an existing system. This means that the container creates an isolated process on the operating system that utilizes the already available operating system functions. Secondly, since hypervisors have access to hardware and will have to install their own operating system it will take up a large chunk of hardware at all times. Since containers virtualize on top of existing systems it is possible to have it only take up space when actually running its desired service, and power down after that. Using a container is similar to starting and shutting down an application, and as such, is very fast compared to having to start an entire operating system stack in the hypervisor approach. Containers do of course have disadvantages compared to hypervisors as well. One disadvantage is that the container has more contact area with the kernel, meaning that an attacker might have an easier time breaking in and getting access to processes they should not. This is something that will be discussed more in section

22 2.8. Software Containers Examples of software container services are Docker[9] and rkt[28] Docker Docker is a container service for containerizing your applications. The Docker client runs natively on Linux, Windows and OS X. By default these clients connects to a local Docker daemon running a virtual environment managed by Docker. This makes Docker able to run Linux-based containers within OS X or Windows, or Windowsbased containers on Windows. When having a big application with several small services running such as an Nginx[29] web server, a Mongo[30] database, a caching system like Redis[31] and a back-end like Flask[32], it becomes very tedious to run everything in the same environment. A single service such as the back-end might end up doing 90% of the work while the database just waits for request. Instead of having everything tightly coupled in the same application, it should be decoupled. This makes it easy to distribute the power needed for the services evenly. For example one could spin up two containers for the back-end and making the load evenly distributed between these two containers if the back-end experiences high load. Docker makes this easy by utilizing something they call Dockerfiles. Dockerfiles is somewhat of a recipe of commands that describe an application, what it should do and how it should be built up. An example Dockerfile can be viewed in listing FROM node:4.5 2 COPY./app/src 3 WORKDIR /app/src 4 RUN curl sh 5 RUN meteor npm install 6 EXPOSE ENTRYPOINT meteor --settings settings.json Listing 2.1: Example Dockerfile The lines in listing 2.1 are explained in the following list: Line 1: Fetches a previously created image. Line 2: Copies your current working directory on your host machine to the Docker directory /app/src Line 3: Changes the working directory of the Docker container to /app/src Line 4: Installs Meteor in the Docker Container Line 5: Installs Meteor dependencies in the Docker Container 12

23 2.8. Software Containers Line 6: Exposes port 3000 on the Docker container so it can be accessed from the host Line 7: Runs a Meteor command in the Docker container to start the Meteor server To use this the Dockerfile has to be built to an image however. This is done by issuing the command seen in listing 2.2, the -t command with the following name tags the image so it can easily be found later. In order to run the image after it has been built we can use the command in listing 2.3. \$> docker build -t exampleimage PATH_TO_DOCKERFILE Listing 2.2: Building a Docker Image \$> docker run -d -p 3000:3000 exampleimage Listing 2.3: Running a Docker Image Listing 2.3 shows the command to run a Docker image. The -d command means that we want to detach the container from our host meaning it will run as a background process. The port 3000 which is exposed in the image is mapped to the port 3000 of the host machine. And the last entry just means that our container image called exampleimage will be started. If everything works it is possible to connect to our Meteor application on the address In order to list running Docker containers one can use the command in listing 2.5. \$> docker ps -a Listing 2.4: Listing running containers By running this command a result could look something like in listing 2.5. Here there are one example container running. The image has a random container id as seen in the first column. The image column displays what image we are running. The container is running the command "meteor settings settings.json" which basically means that it is running some sort of bash script inside the container, the COMMAND column usually represents the ENTRYPOINT issued in the Dockerfile. We can see that the image was created 3 days ago and the current image has been up for 10 seconds. The port 3000 of our host operating system is mapped to the containers port 3000 over TCP. The name of our container if not set is something randomly generated by the Docker application. \$> docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 13

24 2.8. Software Containers 2c8e02 exampleimage "meteor --settings settings. json" 3 days ago Up 10 seconds 3000->3000/tcp random_name Listing 2.5: Listing running containers Security in Docker Security in containerization techniques is a complex matter, this section will focus on Docker, but similar problems may exist in other container techniques as well. Comparing containers to traditional virtual machines, containers have a tighter coupling to the host operating system. According to Combe et al.[33], Docker security relies on three factors. Isolation of processes at the userspace level managed by the Docker daemon, enforcement of this isolation by the kernel, and network operations security. By default Docker containers isolation configuration is quite strict. There are however problems with networking. Since all containers share a common network bridge when using default settings, the container system is open for Address resolution protocol, ARP, spoofing attacks[34] between containers. An ARP spoofing attack is a man in the middle technique used to monitor data being sent between devices in a network. By sending an ARP reply with its own MAC address to two devices. Effectively making all data sent through those two devices being send to the middle man attacker at first. A reference of this can be seen in figure 2.3. There are possibilities to prevent this by disabling network communications between containers, however if this is done a big strength of multicontainer applications disappear since they cannot communicate between each other. 14

25 2.8. Software Containers Figure 2.3: ARP spoofing attack[35] Another issue mentioned by Combe et al. is Host Hardening. Docker supports a few different host hardening modules today. Some of these are SELinux, Apparmor and Seccomp. These are used to limit Docker containers from access right to certain processes and other user related features in the default filesystem, the restrictiveness of these modules differs although. Since Docker images uses some Host OS capabilities to virtualize such as network configurations it is possible to access this in the Docker container, making it possible to change Host OS configurations directly from a Docker container. The different modules differs in their restrictiveness though. For example the default Apparmor profile allows full access to the filesystem, network and all actions that are available in all Docker containers, but not access to actions coming directly from the Host OS. So even if these security models protect the host from containers, it does not protect containers from other containers. To solve this Combe et al. suggest writing your own protocols depending on how much protection your containers need. Furthermore a security issue when using Docker is the distribution of images. Docker has a feature called content trust which is not enabled by default. Content trust in this case means that the publisher of a Docker image needs to sign it before pushing it to the remote registry. When enabling content trust the user can verify that any image pulled from the Docker repository has been signed and verified. If content trust is 15

26 2.9. Deploying to the cloud disabled, which it is by default, the images pulled from the docker hub cannot be trusted. 2.9 Deploying to the cloud The whole idea behind using containers is to be able to use the same environment from development, to testing, to running applications in production. This is where container techniques excels compared to Virtual Machines, by making developers able to easily run the same environment everywhere from development to production. Docker also makes it easy to transfer between different clouds since most cloud providers today provides support for Docker images native. A Docker image being used on Amazon can in theory be deployed to Google Cloud platform seamlessly without many different configuration changes. How seamless this really is will be examined in the implementation part of this thesis project. When deploying a large application with a vast amount of different services utilized, it can be tedious to deploy every container by itself. Docker offers a tool called Docker Swarm which is an orchestrator for organizing and managing several different containers running small services. Another container orchestrator which has received large support lately is Kubernetes[36]. Kubernetes is officially supported by both Google Cloud and Microsoft Azure, therefore chosen to be used in this thesis Kubernetes Kubernetes is an online container orchestration tool for deploying, managing and scaling services. Since containers and especially Docker is recommended to follow a single responsibility principle, users can have several different containers for any given project. In order to organize these in a simple and easy way Google created Kubernetes. While Amazon and Azure does not force you to use Kubernetes it is supported by several large IT-companies such as IBM, Microsoft, Red Hat. According to [37], Kubernetes is becoming the core cloud technology therefore it will be investigated and used in this thesis. Kubernetes utilizes something called Nodes to organize several Pods[38] in order to work together. The Nodes are ran on the actual servers and there is one server instance per Node. Containers are stored in Pods which share the same storage memory and network distributed from the nodes. Kubernetes supports configuration files for easily storing and being able to create deployments without having to remember all commands necessary for this through the command line. The file format for the configuration files Kubernetes uses are YAML since they are according to the Kubernetes community more configuration friendly compared to JSON. A configuration file for a Kubernetes deployment can be 16

27 2.9. Deploying to the cloud seen in listing apiversion: extensions/v1beta1 2 kind: Deployment 3 metadata: 4 name: meteor 5 spec: 6 replicas: 1 7 strategy: {} 8 template: 9 metadata: 10 labels: 11 service: meteor 12 spec: 13 containers: 14 - env: 15 - name: ROOT_URL 16 value: 17 image: exampleimage 18 name: meteor 19 resources: 20 limits: 21 cpu: 200m 22 memory: 100Mi 23 ports: 24 - containerport: protocol: TCP 26 resources: {} 27 restartpolicy: Always 28 status: {} Listing 2.6: A Kubernetes configuration file The configuration file in listing 2.6 starts up a deployment with the name Meteor. The kind of configuration files is defined with the keyword kind. There are two different configurations available and these are either Deployment or Service. A deployment configuration is a way to write declarative updates for Pods. These holds the containers that we will be using. It will collect a Docker container image stored at the address given in the file. This particular container is stored at the Google Container Registry. It also has a restart policy which says "Always". This means that if the con- 17

28 2.10. System Architecture tainer should crash for some reason it will be restarted indefinitely until the pod that the deployment is being hosted on is removed or the deployment file is updated. Horizontal Pod Autoscaler The Horizontal Pod Autoscaler[39] (HPA) is Kubernetes tool for automatically scaling the number of pods based on their Resource usage. Users can set a desired target for CPU or Memory usage, and custom metrics is currently available in alpha support. For example in listing 2.6 the CPU limit is set to 200m. This refers to 200 milicpu[40] and corresponds to 20% of the total CPU of the Node. This means it is possible to have 5 Pods per Node in the case of this thesis since each Node had 1 core. If a Pod breaches the desired target utilization a new pod will be launched and the resource usage can be divided between several pods instead of a single one. A HPA can be launched with the command seen in listing 2.7. The deployment foo is autoscaled setting a minimum of 2 replicas of the deployment and a maximum of 5 replicas. The target CPU-percent is set to 80%. \$> kubectl autoscale deployment foo --min=1 --max=5 --cpupercent=80 Listing 2.7: Launching a Horizontal Pod Autoscaler The way the autoscaler works is by calculating the average resource utilization across every minute divided by the requested resources from a pod. The target number of replicas is then calculated by using the formula in listing 2.8. TargetNumOfPods = ceil(sum(currentpodscpuutilization) / Target ) Listing 2.8: Formula used for calculating the amount of Pods needed While launching or removing replicas noise can occur on the metrics. Therefore scaling up can not happen if any other rescaling has occured in the last three minutes and scaling down will not happen if any rescaling occured within the last five minutes. The scaling will not occur if the pod resource consumption compared to the target consumption is within a 10% tolerance either. This is to avoid problems that might occur while having unstable load System Architecture In order to easier explain the different tools being utilized in this thesis a figure was constructed. See figure

29 2.10. System Architecture Figure 2.4: System Architecture As seen the top hierarchy involves the Cloud Service, Amazon, Azure or Google Cloud in this case. The cloud service launches a series of machines linked together. The machines runs Kubernetes and for this thesis three virtual machines has been used. One for the Kubernetes master which coordinates everything inside the Kubernetes cluster, and two virtual machines running one Node each. Each Node has at least one process called a Kubelet that is responsible for communicating with the Kubernetes master and the other Nodes. It also has one or multiple Pods which run the actual containers where the application lives. The Pods are a Kubernetes abstraction that has one or more application containers running. The containers can also share resources between themselves such as shared storage and networking as stated above. 19

30 2.11. Application deployed 2.11 Application deployed The example application used in this project to test cloud technologies is an application prototype for SIMS Recycling Solutions developed at Attentec during the summer of It is a garbage bin monitoring system utilizing IoT devices on every bin in order to see their position, fill rate and other useful information that can be used to analyze data collected of these bins. It is built for the web using Meteor[41] for the back-end and React[42] for the front-end. In order to sync data from the garbage bins a library called KAA[43] is also utilized. KAA is an open-source IoT platform for monitoring, managing and configuring connected devices. SIMS also uses Jasper[44] which is a platform developed by Cisco for IoT connectivity management. It allows for monitoring devices as well as sending new deployment directly to connected devices Related Work Dick Merkel[45] foresees in his article that Docker is a technique on the uprise and mentions that "It is quickly being adopted by the Linux community and making its way to production environments". He also mentions that Docker has not been developed to replace Virtual Machines, but they can work together having their own strengths and weaknesses. Claus Pahl[46] talks about the benefits of using containerized applications and PaaS clouds to manage and orchestrate applications. He emphasizes the need of a cluster management architeture for handling large clouds with many containers and mentions Kubernetes as a solution to this. Ang Li et al.[47] presents a system for comparing cloud providers based on a few different metrics.they say that the software CloudCmp offers "CloudCmp measures the elastic computing, persistent storage, and networking services offered by a cloud along metrics that directly reflect their impact on the performance of customer applications.". Their software offers companies a possiblity to select the best cloud for their applications and usages. As well as an end-to-end benchmark for cloud providers for how to progress and enhance their clouds perfomance. 20

31 3 METHOD This chapter describes the steps taken in order to deploy the application being used as an example to three different cloud services, Google, Amazon and Microsoft. It discusses how the example application was packaged into a Docker image for later use when deploying to the cloud and how Kubernetes was used to simplify the deployment between these services, and make the application available on the Internet. The chapter also introduces the test cases used in order to test the stability and load balancing methods available through these cloud services. The testing platform used, Locust, is also explained regarding how it was used to setup the tests and running them. All steps will be explained so they can be followed and replicated for other applications. 3.1 Preparing a Docker image In order to prepare the application that is being deployed to the different cloud services a Dockerfile was created with the needed commands. This file can be used by other projects using the same technologies. See listing FROM node:4.5 2 RUN useradd applicationuser 3 RUN mkdir /home/applicationuser 4 RUN chown applicationuser /home/applicationuser 5 USER root 6 COPY./app/src 7 WORKDIR /app/src 8 RUN chown -R applicationuser /app/* 9 EXPOSE ENV PORT USER applicationuser 21

32 3.2. Creating Kubernetes Clusters 12 ENTRYPOINT MONGO_URL=mongodb://\$MONGO_SERVICE_HOST:\ $MONGO_SERVICE_PORT METEOR_SETTINGS="\$(cat settings.json)" node main.js Listing 3.1: Dockerfile for the SIMS-application The Dockerfile above differs from the one given in listing 2.1. Since the application is being deployed and Meteor does not support running in production directly with Meteor commands it has been built into a Node project. A new user called applicationuser has been created for security reasons. In case unauthorized users gets access to the container. If this should happen they can only execute commands that the applicationuser is allowed. This means that they will not have root access but they can still access Node. The MONGO_SERVICE_HOST and MONGO_SERVICE_PORT environment variables are set by Kubernetes and points to the service called Mongo which is ran in the Kubernetes cluster[48]. 3.2 Creating Kubernetes Clusters This section cover how the Kubernetes clusters were created on the different cloud services Google Cloud Google Cloud has a simple dashboard for controlling applications deployed on their service. It is possible to choose by either installing the Cloud SDK locally on your computer making it possible to reach the Cloud API through your terminal, or by using the built in dashboard online. There is also a built in cloud shell available online if one chooses to not install the SDK. The Docker image created for the application described in section 3.1 was uploaded to the Google Container Registry to simplify the deployment. This was done by simply tagging the image with the correct project tag and using Google Cloud SDK to push it to the correct registry. The issued commands for this can be seen in lising \$> docker tag user/example-image gcr.io/your-project-id/ example-image 2 \$> gcloud docker -- push gcr.io/your-project-id/example-image Listing 3.2: Uploading the Docker image to Google Container Registry In order to start setting up a Kubernetes Cluster on Google Cloud one has to choose the desired computing zone first. This basically tells us where the hosting servers are located. There are at the moment 20 different zones available located all over 22

33 3.2. Creating Kubernetes Clusters the world. The zone used for this thesis was Europe-West1-B. To set the zone the command in listing 3.3 was issued. 1 \$> gcloud config set compute/zone europe-west1-b Listing 3.3: Setting a computing zone When the computing zone has been chosen all that is left is to launch the Kubernetes cluster. This was once again done using the Google Cloud SDK and the command issued for this can be seen in listing \$> gcloud container cluster create SIMS-cluster Listing 3.4: Launching a cluster on Google When the cluster had been established the Kubernetes dashboard was used in order to setup our deployments and services from there. This is where configuration files were used in order to easily set up our wanted application Microsoft Azure To install Kubernetes on Azure, the Azure CLI 2.0 can be used. One first needs to create a resource group in a location where Azure has support for the Azure Container Service, see listing \$> RESOURCE_GROUP=my-resource-group 2 \$> LOCATION=westeu 3 \$> az group create --name=$resource_group --location= $LOCATION Listing 3.5: Launching a cluster on Azure Once this has been run, the creation of the Kubernetes cluster can be proceeded by setting a DNS prefix for your cluster and the desired name for your cluster. The desired orchestrator also has to be set which in this case is Kubernetes, see listing \$>DNS_PREFIX=some-unique-value 2 \$>CLUSTER_NAME=any-acs-cluster-name 3 \$> az acs create --orchestrator-type=kubernetes --resourcegroup $RESOURCE_GROUP --name=$cluster_name --dns-prefix= $DNS_PREFIX --generate-ssh-keys Listing 3.6: Launching a cluster on Azure All Kubernetes resources are setup and ready to be used after this command has been completed and we can move on to deploying the application. 23

34 3.2. Creating Kubernetes Clusters Amazon Web Services Since Amazon does not support Kubernetes natively on their platform it has to be installed manually. In order to ease this process a tool called Kops[49] was used. Kops is a tool for deploying production-grade highly available Kubernetes clusters from the command line. When using Kops and AWS together one has to set up a dedicated AWS IAM[50] user for Kops. IAM is AWS Identity and Management access service which lets different tools and other users control your AWS account. Kops can then use this access to create and launch different services that are hosted on AWS. For this case Kops needs access to setup EC2[51] instances, routing in Route53[52], access to creating an S3[53] bucket for storing states, as well as setting up a VPC. All these accesses was setup using the AWS dashboard. Moving on, when setting up a cluster on AWS a hosted zone in Route53 is needed. One can either register a new domain and use this, or move an old domain from somewhere else to AWS. It was chosen to register a new domain to save time completely skipping any configuration step for moving a domain. When the domain has been setup a S3 bucket had to be setup in order to store the states of the Kubernetes Cluster. This was done using the AWS SDK and the command issued can be seen in listing 3.7. The zone chosen for the bucket storage was once again Eu-West-1. Note that even though the name of the zones are the same, the servers can be located at different locations, but should be somewhere in the west of Europe. 1 \$> aws s3api create-bucket --bucket prefix-example-com-state -store --region eu-west-1 Listing 3.7: A Kubernetes configuration file After setting up the S3 bucket and the Route 53 hosted zone it is time to create the cluster. This is done by using Kops and referencing our domain address and S3 bucket through environment variables. After this the cluster can be created through a simple command, see listing \$> export NAME=mycluster.example.com 2 \$> export KOPS_STATE_STORE=s3://prefix-example-com-statestore 3 \$> kops create cluster \ 4 --zones eu-west-1 \ 24

35 3.3. Deploying the application 5 ${NAME} Listing 3.8: Launching a cluster with Kops After having the cluster setup there is only one step left in order to access the dashboard. When using Kops the dashboard is not started by default. The dashboard can be started with the command seen in listing 3.9. This fetches the official dashboard configuration file for Kubernetes and deploys it. 1 \$> kubectl create -f kubernetes/kops/master/addons/kubernetes-dashboard/v yaml Listing 3.9: Launching the Kubernetes dashboard 3.3 Deploying the application In order to deploy the application to the cloud, configuration files for Kubernetes were implemented. There are 4 files which covers a deployment and service object for the application, as well as a deployment and service object for the database used. The application was separated from the database due to it not being deemed important for testing the performance of the application. The files used can be seen in Appendix A. The database instance was started at first in order for the Meteor application to have something to connect to. The database was not automatically scaled at all since the tests were performed to solely focus on the application instance and the load from the HTTP-requests. The mongo-disk that is referenced in the deployment file at line 28 for the database is a drive stored at Google cloud which has 200 gigabytes of memory and will persist after the replicas for the database has been taken down. This is to protect the database in case anything should happen to the MongoDB replicas. From here on the deployment process for the different clouds will be explained. It will go through how to setup the applications through the built in dashboard for Kubernetes. From the dashboard the deployment is the same for all clouds. 3.4 Using the Kubernetes Dashboard When the cluster has been setup and runs on each of the cloud services, one can begin to deploy the desired application. In order to make the deployment replicable and the same on all cloud services the Kubernetes Dashboard was utilized, since it is easy to replicate it what is going on with its graphical user interface. 25

36 3.5. Testing with Locust A deployment and a service was setup, and the chosen way to do this was to use predefined configuration files as previously mentioned. It is also possible to set this up directly in the dashboard with the help of some forms for what should be used. But since the configurations are the same over all cloud platforms it was easier to use a configuration file instead of repeating some steps over all cloud platforms. In order to add a deployment object all that is needed is going to the deployments route accessed on the left menu in the dashboard and uploading a new deployment file. See figure 3.1 Figure 3.1: Adding a deployment object in the Kubernetes Dashboard 3.5 Testing with Locust The test is using simple HTTP-requests such as GET and POST in order to send requests to the site, logging in to the interface using a dummy account and navigating around in the SIMS dashboard. In order to test the cloud services and the load balancing, the test cases were implemented with Locust. The file used for setting up the test cases can be seen in listing An explanation of the test cases can be read below. 1 from locust import HttpLocust, TaskSet 2 3 def indexroute(l): 4 l.client.get("/") 26

37 3.5. Testing with Locust 5 6 def stationroute(l): 7 l.client.get("/stations") 8 9 def usersroute(l): 10 l.client.get("/users") def maproute(l): 13 l.client.get("/map") def loginroute(l): 16 l.client.post("/login", {"login- ":"admin@example.com", "login-password":"admin"}) class UserBehavior(TaskSet): 19 tasks = {indexroute: 1, stationroute: 2, usersroute: 3, maproute: 4} def on_start(self): 22 loginroute(self) class WebsiteUser(HttpLocust): 25 task_set = UserBehavior 26 min_wait = max_wait = 9000 Listing 3.10: Test cases developed with Locust. Filename loadtest.py First off there are 4 different functions for visiting different routes the indexroute, stationroute, userroute, maproute and loginroute. Moving on these routes have different weights set where the index route is visited the most, and the map route has the least weight as seen in line 19. As mentioned earlier the test starts by logging in to the application as seen in the function on_start on line 21. In order to run the test cases above, the command in listing 3.11 is issued. 1 \$> locust -f locust_files/loadtest.py --host= com Listing 3.11: Starting up Locust This command initiates the test cases from the provided file in the Locust program and launches a local website with a dashboard where the tests can be controlled. 27

38 3.5. Testing with Locust When navigating to the website the user is prompted to select how many users should be launched for the test, and the hatch rate, meaning how fast the requests should increase over time until reaching its full potential. An image for this view can be seen in figure 3.2. In Figure 3.3 the dashboard showing the requests happening in real time and their current results for success, errors, and the time per request. Figure 3.2: Launching virtual users in Locust Figure 3.3: Locust dashboard showing requests 28

39 4 RESULTS AND DISCUSSION The main result achieved from this master thesis is an application deployed in the same way on three different cloud platforms. The load testing results found will be presented below. After that there will be a discussion part for the thesis. 4.1 Veryfing Autoscaling capabilities In order to answer the research question stated in the beginning of the report How can these cloud services be performance tested so that the scaling of the applications happens in a suitable way? Locust was setup to launch requests against the sites deployed on the different cloud services. The scaling for the application was setup using the Horizontal Pod Autoscaler (HPA) in Kubernetes with a CPU target of 80% and a maximum of 8 replicas. The results acquired can be seen in the figures below. 29

40 4.1. Veryfing Autoscaling capabilities Figure 4.1: CPU and Users over Time for Google Cloud Figure 4.2: Replicas over time for Google Cloud 30

41 4.1. Veryfing Autoscaling capabilities Figure 4.3: CPU and Users over Time for AWS Figure 4.4: Replicas over time for AWS 31

42 4.1. Veryfing Autoscaling capabilities Figure 4.5: CPU and Users over Time for Azure Figure 4.6: Replicas over time for Azure 32