Hybrid Auto-scaling of Multi-tier Web Applications: A Case of Using Amazon Public Cloud

Transcription

1 Hybrid Auto-scaling of Multi-tier Web Applications: A Case of Using Amazon Public Cloud Abid Nisar, Waheed Iqbal, Fawaz S. Bokhari, and Faisal Bukhari Punjab University College of Information and Technology,Lahore {abid.nisar, waheed.iqbal, fawaz.bokhari, faisal.bukhari}@pucit.edu.pk Abstract In this paper, we propose a hybrid auto-scaling method to dynamically allocate resources to multi-tier web applications hosted on Amazon public cloud. Our proposed auto-scaling method implements horizontal scaling at the web server tier and vertical scaling at the database tier of a web application on varying workloads automatically. To the best of our knowledge, our proposed method is the first attempt to provide Quality of Services (QoS) guarantees to application owners using hybrid auto-scaling approach. The experimental results show the effectiveness of our proposed method in terms of performance and cost compared to current practices of resource over-provisioning in the cloud. Keywords-Cloud Computing, Amazon EC2, Telemedicne, Autoscaling, Restful, Web services I. INTRODUCTION Cloud computing technology has emerged as a promising technology for the provision of low cost, on-demand, and pay-as-you-go services to enterprises, application service providers, and individual users. With the proliferation of such Internet applications, the demand for cloud computing services has seen an unprecedented rise recently. However most of the applications deployed on a cloud follow the model of multitier web architecture, which usually have a web server tier and a database tier. In order to offer better quality of services (QoS), these cloud-hosted web applications need to maintain specific response time guarantees which can be achieved by dynamic detection of bottlenecks and their effective resolution automatically. However, providing such response time guarantees pose a great challenge to the cloud service providers mainly due to the inherent nature of multi-tier applications, which are typically complex and bottlenecks may occur on multiple locations depending upon specific workload patterns at any give time interval. Almost all of the Public Cloud service providers offer services to automatically scale web applications. However, these services do not support multi-tier applications to automatically scale all tiers dynamically on changing workload patterns. For example, Amazon Web Services (AWS) [1] is a public cloud service provider, whose Elastic Compute Cloud (EC2) Autoscaling service [2] enable users to configure their all tiers to horizontally scale based on user define policies and then users have to ensure connectivity configurations among tiers by themselves. Specifically, for a multi-tier web application, using EC2 Auto-scaling service, one can develop a custom autoscaling service to scale database and web tiers horizontally. However, horizontal database scaling is quite challenging mainly due to the use of master-slave architecture which provides limited scaling only on read queries and introduces additional overhead of data synchronization from master to slaves. There has been some research work done in auto-scaling of multi-tier web applications. For example, Urgaonkar et al., [3] describes a queuing network based analytical model to learn working of each tier. Bonvin et al., [4] provides a cost-effective solution for the dynamic provisioning of cloud resources to multi-tier web applications in order to satisfy response time and availability guarantees using horizontal scaling. Furthermore, most of the research in dynamic resource provisioning of web applications [5] [7] has been focused on changing the application workload volume to provision more resources (increase number of allocated virtual machines) to maintain application performance. For example, Singh et al., [8] presents a technique to model dynamic workloads for multi-tier Web applications using k-means clustering on the service time feature that collects logs at each tier. The method uses queuing theory to model the system s reaction to the workload and identifies the number of instances required for an Amazon EC2 cloud to perform well under a given workload scenario. In [9], the authors have purposed a vertical scaling strategy using reinforcement learning method for a local testbed cloud by controlling allocations of CPU and memory to the virtual machines hosting application tiers. Similarly, in [1], the authors describe the horizontal scaling of a multi-tier web application hosted on Amazon Web Service (AWS) using coarse grained access log monitoring techniques. However, none of these explore a possibility to use horizontal and vertical scaling together to provision resources to different tiers of a multi-tier web application. In this paper, we propose and evaluate a new hybrid autoscaling method to dynamically allocate resources to a multitier web application hosted on AWS. Our proposed system horizontally scale web server tier and vertically scale database tier dynamically to offer better response time and overcome database horizontal scaling limitations. To the best of our knowledge, our proposed method is first attempt to propose and evaluate hybrid auto-scaling method for a multi-tier web applications hosed on AWS. In the rest of this paper, we explain our hybrid auto-scaling approach, experimental design, and experimental evaluation of the proposed hybrid auto-scaling method on AWS.

2 II. HYBRID AUTO-SCALING OF MULTI-TIER WEB APPLICATIONS In this section, we explain our proposed hybrid auto-scaling approach for multi-tier web applications hosted on the Amazon Web Service (AWS) cloud architecture. We consider a simple two tiers web application consisting of a web server (web tier) and a database server (database tier) to auto-scale provisioned resources of AWS with dynamic workloads. We assume that each tier is deployed on a separate instance of virtual machines. Our proposed approach, scale-out (horizontal scaling) web tier and scale-up (vertical scaling) database tier whenever a bottleneck which we define as the CPU utilization on a specific tier is detected. AWS auto-scaling services are used to scale-out web tier and Amazon Relational Database Service (RDS) is used for database scaling. In the rest of the section, we briefly explain Amazon auto-scaling service, Amazon RDS service, and our proposed hybrid scaling method. A. Amazon auto-scaling The auto-scaling service of AWS enables users to dynamically control number of Amazon Elastic Compute Cloud (EC2) instances allocated for end users application to maintain performance of the application on varying workloads. It allows users to configure policies (define specific thresholds of CPU, memory, I/O, and disk reads etc.) to scale-out and scalein the number of Amazon EC2 instances allocated to an application. For a single tier simple web application which can be defined as a typical web application consisting of static and dynamic pages and usually does not require database interaction, it is convenient to use AWS auto-scaling service to ensure dynamic horizontal scaling to maintain specific performance constraints. However, for a typical multi-tier web application, AWS auto-scaling does not provide capabilities to automatically manage number of EC2 instances allocated for both the tiers. For example, increasing the number of database tier requires updating configuration of web tier virtual machines to inform them about the new database instances, so that the web tier instances can distribute the database load equally among the database machines that were just instantiated. In addition to that, dynamic horizontal scaling of a database tier also presents challenges to synchronize database updates. B. Amazon RDS Amazon Relational Database Service (RDS) provides capabilities in setting up relational database tier of any application. It provides automatic backup and restore, scaling, and multizone availability. Amazon RDS does not provide any autoscaling feature like AWS Auto-scaling policies that user can configure to scale applications automatically, but it provides facility to scale-up database instance vertically on an explicit user s request. Therefore, RDS does not offer auto-scaling on database tiers. C. Hybrid auto-scaling method In this section, we explain our proposed methodology to combine AWS auto-scaling and Amazon RDS service to build a hybrid auto-scaling service for a two tier web application architecture, consisting of a web server tier and a database tier. Our proposed hybrid auto-scaling service horizontally scale web tier and vertically scale database tier in order to provide better response time and overcoming the challenges of horizontal scaling of database tier in a typical cloud infrastructure. 1) Web tier horizontal scaling:: The web tier of our application is deployed on an EC2 instance. We configure AWS auto-scaling service in such a way that it increases the number of virtual machines allocated to web tier when the average CPU utilization of allocated EC2 instances stay above 7% for at-least 5 minutes consecutively. Similarly, we define a policy in the auto-scaling service to decrease the number of allocated virtual machines whenever the average CPU utilization of all allocated EC2 instances stay less than 3% for minimum 5 minutes consecutively. We strongly believe that using this simple strategy will enable an application to horizontally scale its web tier on dynamic workloads conditions. 2) Database tier vertical scaling:: In the beginning, we deployed a database tier of the application on Amazon RDS instance of type db.m3.medium and dynamically vary RDS instance type to one from db.m3.medium (M), db.m3.large (L), db.m3.xlarge (XL), and db.m3.2xlarge (2XL) depending on the workload and user preferences. We allow user to configure a setting namely (scale-mode) in the service with two possible values; gradual-scaling or rapid-scaling. If user sets scale-mode to gradual-scaling, then we vertically scale RDS instance to next level on 7% CPU utilization of the instance for 1 minutes consecutively. For example, if RDS instance was using db.m3.medium then scaling decision would update the RDS instance to db.m3.large. If user sets scale-mode configuration setting to rapid-scaling, then we vertically scale RDS instance to the maximum allocation level on 7% CPU utilization of the instance for 1 minutes consecutively. For example, if CPU utilization of initial allocation of RDS instance remains above 7% for 1 minutes, then RDS instance will be updated directly to db.m3.2xlarge. Similarly, in this configuration, scale down also directly reduces the allocation of RDS server directly to lowest level of allocation (db.m3.medium) on CPU utilization of less than 2% for consecutive 1 minutes. A typical RDS instance modification call requires 4 to 6 minutes to update the instance type. Figure 1 explains deployment of our proposed hybrid autoscaling of a multi-tier web application over Amazon cloud. In a typical scenario, workload is received by Elastic Load Balancing service of AWS and distribute the allocated instances of web tier and then web tier instances query the database to currently allocated RDS instance.

3 TABLE I SUMMARY OF EXPERIMENTS. Users Users Users Users Users Web-Tier Elastic Load Balancing EC2 Instance EC2 Instance EC2 Instance EC2 Instance Database-Tier db.m3.2xlarge db.m3.xlarge db.m3.large db.m3.medium RDS Instance MySQL Database Instance Availability Zone: us-west-2a Region: us-west-2 Exp. Description 1 Static allocation using 1 Amazon EC2 instance of type m3.medium for Web server tier and 1 EC2 instance of type db.m3.medium for database tier. 2 Horizontal auto scaling enabled for web server tier and a powerful Amazon RDS instance of type db.m3.2xlarge for database tier. 3 Horizontal auto scaling enabled for web server tier and vertical scaling using gradual-scaling setting is enabled for database tier. 4 Horizontal auto scaling enabled for web server tier and vertical scaling using rapid-scaling setting is enabled for database tier. TABLE II INSTANCE TYPE AND RESOURCE ALLOCATION OF INSTANCES USED IN EXPERIMENTS. Instance Type vcpus Memory - Network Performance (GiB) m3.medium Moderate db.m3.medium Moderate (M) db.m3.large (L) Moderate db.m3.xlarge (XL) 4 15 Moderate db.m3.2xlarge (2XL) 8 3 High Fig. 1. Hybrid auto-scaling of a multi-tier web application using Amazon Web Services. III. EXPERIMENTAL DESIGN In this section, we explain the experimental benchmark web application, synthetic workload generation method, and experiments performed to evaluate our proposed hybrid autoscaling approach. A. Benchmark multi-tier web application We use RUBiS [11], a web application which provides auction facility of items including bidding, selling, and browsing of items, as a benchmark web application to evaluate hybrid auto-scaling methods. We have used PHP implementation of RUBiS with MySQL database. B. Workload generation We have used Apache JMeter [12] to generate synthetic workloads for RUBiS benchmark web application in each of the experiment. Our synthetic workload generation emulates specific number of concurrent user sessions per second for RUBiS web application in a step-up fashion. Each user session issues read requests to Categories, Regions, Items, Home, Register, and Sell pages of RUBiS. We have configured JMeter with 75 number of threads (users) and ramp-up period of 12 seconds. We have performed four experiments based on this workload for RUBiS. Experiment 1 profiles the system behavior under static allocation, Experiment 2 profiles the system behavior under over-provisioning of database tier and a dynamic scaling of web tier. In Experiment 3 and 4, we profile our proposed hybrid auto-scaling scheme under gradual-scaling and rapid-scaling for database tier and dynamic scaling is enabled for web tier. We generated the workload for each experiment for a total of 7 minutes which contains ramp-up period of first 55 minutes and then workload starts decreasing rapidly till the end of each experiment. C. Experiment details In Experiment 1 we have installed Apache web server on Amazon EC2 instance of type m3.medium and deployed RUBiS benchmark web application (web tier). We have used MySQL database and deployed RUBiS database (database tier) on Amazon RDS instance of type db.m3.medium. We did not enable any kind of dynamic scaling for both tiers in Experiment 1. In Experiment 2 we enable horizontal scaling on web tier and over-provisioned database tier using db.m3.2xlarge type instance. In Experiment 3, we enable horizontal scaling on web tier and enable vertical scaling on database tier to scale from db.m3.medium to db.m3.large to db.m3.xlarge to db.m3.2xlarge in a step-up fashion whenever database CPU saturates. In Experiment 4, we enable horizontal scaling on web tier and enable vertical scaling on database tier to directly scale from db.m3.medium to db.m3.2xlarge in case of CPU saturation. Table I summarizes the experiments and Table II describes the resource allocation of EC2 instances used in experiments.

4 Fig. 2. Average response time, throughput, and rejections in Experiment 1. The web tier and database tier is statically deployed on m3.medium and db.m3.medium respectively. IV. EXPERIMENTAL RESULTS A. Experiment 1: Static allocation This section explains the results we obtained in Experiment 1. Figure 2 shows average throughput (request/second) and average response time of the application during Experiment 1. The web and database tiers are statically allocated m3.medium and db.m3.medium respectively before starting the experiment. After 22 nd minute of the experiment, response time of the application start growing dramatically and crosses acceptable response time threshold. However, response time start decreasing after 62 nd minute as the amount of workload is reduced by the workload generator. This experiment provides baseline performance and depicts system behavior under traditional deployment of a multi-tier web application. Once CPU resources of web tier and database tier saturate, then throughput of the application stop growing, the response time start increasing, and the number of requests processed by the serves start decreasing. B. Experiment 2: Horizontal web tier scaling and database tier over-provisioning Now, we explain the results obtained in Experiment 2. Figure 3 shows average response time, average throughput, average rejections, and dynamic provisioning of web tier instances during Experiment 2. The number of instances allocated to web tier are dynamically managed using the approach explained in II-C1. The database tier is over-provisioned by allocating db.m3.2xlarge instance. During this experiment, average response time never violates response time threshold and throughput increases gradually till 55 th minute of the experiment. After this, the throughput decreases due to the workload generator also start decreasing the amount of user requests after 55 th minute. One can argue that this approach provides better performance than the rest of the schemes, however, adopting this scheme which is over-provisioning of database tier would be expensive for the application owners and therefore is not a cost effective solution Web tier machines Fig. 3. Average response time, throughput, rejections, and dynamic allocation of instances to web tier in Experiment 2. The web tier horizontally scale on CPU saturation of allocated instances. The database tier over-provisioned by allocating db.m3.2xlarge RDS instance. C. Experiment 3: Horizontal web tier scaling and gradual database tier scaling In this section, we explain the results obtained in Experiment 3. Figure 4 shows average response time, average throughput, average rejections, and provisioning of web and database tier during Experiment 3. The number of instances allocated to web tier are dynamically managed using the approach described in Section II-C1. The database tier instance type is dynamically managed using the approach explained in Section II-C2 with gradual-scaling configuration. The experiment shows the effectiveness of our proposed hybrid auto-scaling method using gradual-scaling configuration for increasing workload. It can be easily seen from the graph that our proposed hybrid auto-scaling method appropriately brings down the response time by automatically adjusting the web and database tier resources. D. Experiment 4: Horizontal web tier scaling and rapid database tier scaling This section explains the results we obtained in Experiment 4. Figure 5 shows average response time, average throughput, average rejections, and provisioning of web and database tier during Experiment 4. The number of instances allocated to web tier are dynamically managed using the approach explained in Section II-C1. The database tier instance type is dynamically managed using the approach explained in Section II-C2 with rapid-scaling configuration. It can be seen that the database scaling strategy rapidly changes the RDS instance from db.m3.medium (M) to db.m3.2xlarge (2XL). Our proposed hybrid auto-scaling method appropriately brings down the response time by adjusting the web and database tier resources automatically.

5 TABLE III EXPERIMENTAL RESULTS SUMMARY. Experiment No. of Requests Processed No. of Requests Rejected % SLA Missing Web Scale Ops DB Scale Ops Experiment Experiment Experiment Experiment DB tier scaling initiated DB tier machine Web tier machines 2XL XL L M DB tier scaling initiated DB tier machine Web tier machines 2XL XL L M Fig. 4. Average response time, throughput, rejections, and dynamic allocation of web tier and database tier in Experiment 3. The bottom graph s y2 title used to show database tier allocation during the experiment. The bottom graph also shows when database scaling is actually triggered and the time it realized in the system. The hybrid auto-scaling method appropriately brings down the response time by adjusting web and database tier resources automatically. Fig. 5. Average response time, throughput, rejections, and dynamic allocation of web tier and database tier in Experiment 4. The bottom graph s y2 title used to show database tier allocation during the experiment. The bottom graph also shows when database scaling is actually triggered and the time it realized in the system. The hybrid auto-scaling method appropriately brings down the response time by adjusting web and database tier resources automatically. E. Experimental Summary Table 3 summarizes experimental results. The table shows total number of requests processed, total number requests rejected, the percentage of requests missing the acceptable response time threshold (%SLA Missing), the number of web scale operations (Web Scale Ops), and number of database scale operations (DB Scale Ops) for each experiment. It appears from the table that over-provisioning of database tier (Experiment 2) outperforms the other experiments including our proposed method, however, as mentioned earlier, this scheme cost more to the application owners and therefore is not very cost effective. On the other hand, our proposed method (Experiment 3 and 4) also came out relatively closer to Experiment 2 in terms of performance because of adaptive provisioning of required resources to the appropriate tiers automatically. We strongly believe that our proposed method provide significant cost benefits to the multi-tier application owners with an acceptable performance compared to the overprovisioning method. V. CONCLUSION AND FUTURE WORK A new hybrid auto-scaling method to dynamically provision resources to multi-tier web applications hosted on Amazon public cloud is proposed and evaluated in this paper. We have shown that our proposed hybrid scaling method dynamically scales the web server tier horizontally and also scales the database tier vertically in order to provide better response time cost effectively. Our experimental evaluation validates the effectiveness of our proposed method compared to current practices of over-provisioning of database tier. To the best of our knowledge, our proposed method is the first attempt to propose and evaluate a custom hybrid auto-scaling method for multi-tier web applications hosted on public clouds. Currently, we are extending our hybrid auto-scaling method to support generic n-tier application architecture and to also incorporate proactive scaling decisions to minimize the percentage of SLA missing requests. ACKNOWLEDGMENTS We would like to thank Amazon Web Services (AWS) for providing us with a generous grant in terms of credits to use its cloud computing resources. The grant was part of its Amazon Educational Grant AWS Fund. REFERENCES [1] Amazon.com, Inc, Amazon Web Services (AWS), 26, available at [2] Amazon Inc, Amazon Web Services auto scaling, 29, available at

6 [3] B. Urgaonkar, G. Pacifici, P. Shenoy, M. Spreitzer, and A. Tantawi, An analytical model for multi-tier internet services and its applications, in SIGMETRICS 5: Proceedings of the 25 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, vol. 33. ACM, 25, pp [4] B. Nicolas, P. Thanasis G., and A. Karl, Automatic SLA-driven provisioning for cloud applications, in Proceedings of the 211 International Symposium Cluster, Cloud and Grid Computing. Newport Beach, CA, USA: IEEE Computer Society, 211. [5] D. Villela, P. Pradhan, and D. Rubenstein, Provisioning servers in the application tier for e-commerce systems, ACM Transaction on Internet Technology, vol. 7, February 27. [Online]. Available: [6] P. Bodik, R. Griffith, C. Sutton, A. Fox, M. Jordan, and D. Patterson, Statistical machine learning makes automatic control practical for internet datacenters, in HotCloud 9: Proceedings of the Workshop on Hot Topics in Cloud Computing, 29. [7] J. Dejun, G. Pierre, and C.-H. Chi, Resource provisioning of Web applications in heterogeneous clouds, in Proceedings of the 2nd USENIX Conference on Web Application Development, 211. [8] R. Singh, U. Sharma, E. Cecchet, and P. Shenoy, Autonomic mix-aware provisioning for non-stationary data center workloads, in ICAC 1: Proceedings of the 7th IEEE International Conference on Autonomic Computing and Communication. Washington, DC, USA: IEEE Computer Society, 21. [9] L. Yazdanov and C. Fetzer, Lightweight automatic resource scaling for multi-tier web applications, in 7th IEEE International Conference on Cloud Computing, IEEE. IEEE, 214. [1] W. Iqbal, M. Dailey, and D. Carrera, Unsupervised learning of dynamic resource provisioning policies for cloud-hosted multi-tier web applications, in Systems Journal, IEEE. IEEE, 215, pp [11] OW2 Consortium, RUBiS: An auction site prototype, 1999, rubis.ow2.org/. [12] Apache Software Foundation, Apache JMeter, 1999, apache.org/.