Automated Control of Internet Services

Transcription

1 Automated Control of Internet Services Jean Arnaud IRIA Grenoble, France ABSTRACT Finding an efficient configuration for cluster-based multi-tier Internet services is often a difficult task. Moreover, even a good configuration could become obsolete, depending on workload evolution. In this paper, we address both problems by dynamically calculating an optimal configuration for multi-tier Internet services and applying this configuration to the managed application. Our approach is based on two main components. A model of the underlying application, and a controller using this model to find the optimal configuration according current environment and performance objectives. We evaluate the model accuracy and the controller efficiency. Experiments show that our solution improves resource consumption, and may lead to significant energy savings, besides matching the performance objectives even with a dynamic workload. Keywords Multi-tier applications, Cost, Availability, Performance, SLA, Modeling, Capacity planning Categories and Subject Descriptors C.2.4 [Distributed Systems]: Client/server, Distributed applications; C.4 [Performance of Systems]: Performance attributes, Reliability, availability, and service ability; D.4.8 [Performance]: Modeling and prediction General Terms Algorithms, Performance 1. ITRODUCTIO As multi-tier architectures become the most used design for Internet services, their configuration poses some significant challenges, in terms of performance, reliability and cost. Performance and reliability are important for the quality of service that clients experience, while cost is important for Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. FeBid 1, april 13, 21, Paris, France Copyright c 21 ACM /1/4...$1.. the application hoster. Finding a configuration that matches all this conflicting objectives could be very difficult. Moreover, the amount of clients concurrently accessing the service is not constant across time, nor the kind of requests they send. Thus, monitoring the client interactions and adapting the service to their needs are real challenges in such dynamic systems. Some multi-tiered Internet services faces large amount of request at the same time. Clusters are used to run Internet services, providing the ability to add or remove resources to the service when load evolves. More client requests, or increased requests heaviness will leads to increased resources consumption. On the opposite, when load decreases, we can save costly resources. Properly adding and removing resources when needed will leads to cost-efficient Internet services. Each server of the application has also several configuration parameters that impact global system performance. As we can see, finding the most efficient configuration could be a very difficult to impossible task to do for a human administrator. Our approach aims to provide a tool for dynamically adapting multi-tier Internet services configuration, through an autonomic controller that handles all tasks from monitoring to applying new configuration. This paper describes the two main components of our approach. First, a model to predict a system performance depending on entering workload and its configuration. Second, a capacity planning algorithm that, given high level performance objectives, uses this model to find an optimal configuration. The remainder of this paper is organized as follows. Section 2 presents the context of multi-tiered Internet services. We introduce our model in section 3. The design principles of our control approach is detailled in section 4. Section 5 provides a brief overview of the related work, and section 6 draws our conclusion. 2. DEFIITIOS 2.1 Multi-tier Internet services A multi-tier system is composed of a series of M tiers T 1, T 2,..., T M. Each tier is tasked with a specific concern. For instance, a three-tier system could consists of tier T 1 responsible of processing the application web content, tier T 2 responsible of the application business logic, and tier T 3 responsible of the storage of non-ephemeral data of the application. When a client issues a request to a multi-tier system, the request first accesses tier T 1, and then may flow through successive tiers T 2, T 3,..., T M. More precisely, when

2 a request is processed by tier T i either a response is returned to tier T i 1 (or to the client if i = 1), or a subsequent request is sent to tier T i+1 (if i < M). Multiple clients may concurrently access a multi-tier system. In its basic form, each tier consists of a single server. To prevent a server from thrashing when the number of concurrent clients grows, a classically used technique is admission control [7]. It consists in fixing a limit for the maximum number of clients allowed to concurrently access a server also known as the Multi-Programming Level (MPL) configuration parameter of servers. Above this limit, incoming clients are abandoned (i.e. rejected). Thus, a client request processed by a multi-tier system either terminates successfully with a response to the client, or is abandoned because of a server s concurrency limit. Moreover, a multi-tier application may face a varying workload amount and a varying workload mix over time. The workload amount denoted is the number of clients that try to concurrently access a multi-tier application. And the workload mix denoted X corresponds to the distribution of different types of interactions issued by clients. Application workload variation reflects different client behaviors at different times; for instance, an service is likely to face a higher workload amount in the morning than in the rest of day. X is a structure containing visit ratio V i which concerns load repartition on each tier and Z, the average client think time between interactions. Multi-tier servers are hosted by machines (i.e. computing units), and a machine is exclusively owned by a server. However, for scalability purposes, a tier is usually provisioned with multiple servers in a cluster built atop replication, partitioning and load balancing techniques. In the following, we consider fair load balancing techniques. If not provisioned adequately, cluster-based multi-tier applications may face a bottleneck on one of the tiers; and bottleneck may occur at most on one tier at a given time [2]. We consider that adding more machines to an overloaded tier reduces load amount on each machine of the tier, thus improving overall performances. We also made the assumption that machines are homogeneous inside each tier of the application, as it is typically the case in computer clusters of that size [1]. 2.2 Quality of Internet services SLA Service Level Agreement is a contract negotiated between clients and their service provider [8]. It specifies service level objectives (SLOs) that the application must guarantee in the form of constraints on performance and availability. Among the key metrics of interest for quantifying the performance and availability of multi-tier applications, we can cite client request latency and client request abandon rate used in the following. The latency of a client request is the necessary time for a multi-tier application to process that request. The average client request latency (or latency, for short) of a multi-tier application is denoted as l. A low latency is a desirable behavior which reflects a reactive application. The abandon rate of client requests is the ratio of requests that are rejected by a multi-tier application compared to the total number of requests issued by clients to that application. It is denoted as α. A low client request abandon rate (or abandon rate, for short) is a desirable behavior which reflects the level of availability of a multi-tier application. Besides performance and availability metrics, the cost of a multi-tier application is another aspect that is taken into account when optimizing application provisioning. The cost of a multi-tier system refers to the economical and energetical costs of this system. Here, cost is defined as the total number of machines that host a cluster-based multi-tier application, and is denoted as ω. Saving energy from running servers is always a good point, even if machines amount are not limited, then a low cost always preferable. 2.3 Configuration of Internet services We define the configuration κ of a cluster-based multitier application with a triplet κ(m, AC, LC), where M is the fixed number of tiers of the multi-tier application, and AC and LC are respectively the architectural configuration and local configuration of the application. We define the architectural configuration of a cluster-based multi-tier application as the distributed setting of the application in the form of the number of replica servers at each tier. It is conceptualized as an array AC < AC 1, AC 2,..., AC M >, where AC i is the number of machines at tier T i of the multi-tier application. We define the local configuration of a multitier application as the local tuning parameter(s) applied on servers at tier T i. Local configuration is conceptualized as an array LC < LC 1, LC 2,..., LC M >; and in the following, LC i represents servers MPL (multi-programmig level) at tier T i of the multi-tier system. 3. MODEL OF ITERET SERVICES 3.1 Model definition We propose a queuing network model of Internet services based on the Mean Value Analysis algorithm [12]. Roughly speaking, we model each server as queue, and requests are going from one queue to another, representing the processing time required on each tier. Figure 1 summarizes the model inputs, outputs and state parameters. Figure 1: Model parameters The model inputs variables are the configuration κ of the system, the workload mix X and amount. The initial κ < M, AC,LC > is provided to the controller at its startup, then maintained until the controller stops. Workload amount is probed on the first tier of the real system. The visit ratios V i from workload mix X < V i, Z > are processed by the controller using relative throughput on each tier. The average client think time Z is measured on the first tier of the application. In addition to input variables, the model has two state variables, called service times S i and communication delays D i.

3 Service time S i is the average incompressible time necessary to process a request on tier T i. Communication delay D i is the average incompressible network communication time between tier T i 1 and tier T i. D 1 is always equal to, since T 1 is the first tier of the application. The internal state variables are calculated using the measured latency and abort rates. We search for the S i and D i values that minimize the deviation between model prediction and current measurements. We used gradient descent algorithm to efficiently find the best internal state parameters. Indeed, preliminary experiments have shown that the space of internal state variables has no local optimal. For example, figure 2 plots the model deviation according to S 1 and S 2 state parameters values (D 2 is not represented) Model prediction deviation (latency) 2 S1 parameter (ms) 3 4 Deviation with 5 clients (%) S2 parameter (ms) Figure 2: Model deviation depending on internal state parameters The proposed model has also 3 output variables, which are the latency l, abort rate α and cost w of the considered application. The output processing is detailed in algorithm 1. We begin by using the local configuration to calculate the admitted clients. Then we process the latency l, injecting admitted clients one by one into the queuing network. Little s law is used to calculate the queue length on each tier. The ratio between rejected request throughput and total request throughput determines the abort rate. The cost of the multi-tier application is the total amount of allocated machines. 3.2 Model validation Experimental setup Model and utility-aware capacity planning and provisioning evaluations have been conducted using the TPC-W benchmark [13], that models a realistic web bookstore. TPC-W comes with a client emulator which generates a set of concurrent clients issuing requests for browsing the content of the bookstore, consulting the best-sellers, etc. The client emulator generates a tunable workload; this allows us to vary the workload amount and workload mix during the experiments. In our experiments, the on-line bookstore was deployed as a cluster-based two-tier system, consisting of a cluster of replicated web/business servers as a front-end and a cluster of replicated database servers as a backend. Our experiments have been conducted on a set of computers organized as follows: a first computer dedicated to the 1 Input: : workload amount X < V i, Z >: workload mix κ < M, AC, LC >: multi tier system configuration Output: l: latency α: abort rate ω: cost R = Z; τ = ; ω= ; 2 /* abort rate (first part) */ 3 for m = 1;m M;i + + do 4 MPL m = AC m LC m 5 ar m = Max(,1 MPLm V m ) 6 α= Max(ar 1...ar M) 7 = (1 α); /* admitted clients */ 8 /* latency */ 9 for m = 1;m M;m + + do 1 Ql m = ; 11 for n = 1;n ;n + + do 12 for m = 1;m M;m + + do 13 R m = Max(V m S m, 14 (1+Ql m) (V m S m V m D m)/ac m)+v m D m n τ = ( R + P M ); /* throughput */ m=1 R m for m = 1;m < M;m + + do Ql m = τ R m; /* Little s law */ l= P 18 M m=1 Rm; /* latency */ 19 /* cost */ 2 for m = 1;m M;m + + do 21 ω=ω+ac m; /* total #resources */ 22 /* abort rate (second part) */ 23 τ failed = α /Z 24 τ success = /(Z + l) τ α= failed 25 τ failed +τ success Algorithm 1: Modeling multi-tier systems client emulator, a second one for our controller, a front-end cluster of replicated servers running Apache Tomcat 5.5 web and application server, and a back-end cluster of replicated servers running MySQL 5. database server. A proxy-based approach was followed in order to implement admission control, and load balancing among the replicas of a tier. Here, load balancing is able to dynamically integrate/remove replicas upon server (un-)provisioning. The experiments were conducted on a cluster of machines with bi-2.33 GHz Intel Xeon CPUs and 4 GB RAM, connected via a 1 Gb/s Ethernet LA Model validation Scenario. To test model accuracy, we consider an experiment with varying workload amount and mix. The workload amount evolves between 25 and 125 clients, with irregular steps to simulate load peaks. The workload mix changes from mix 1 to mix 2 at the middle of the experiment. Mix 2 has more heterogeneous client interactions than Mix 1. Figure 3 presents the workload mix and amount as measured by the controller. We used for our model validation experiments

4 two configurations κ1 and κ2 defined as follow: κ1 : AC =< 1, 1 >, LC =< 5, 5 >: a low-cost configuration, where capacity is increased through high local configuration. κ2 : AC =< 3,6 >, LC =< 2, 1 >: a large configuration where capacity is increased through high architectural configuration, with default values for local configuration, since Tomcat default MPL is 2 and MySQL one is 1. not test a larger configuration). The capacity planning algorithm uses the model to get the performance predictions for a configuration. On a second step, the capacity planning algorithm performs a dichotomic search into this configuration space to find the optimal configuration. The optimal configuration is the less costly configuration that matches the SLA. The capacity planning is automatically triggered when the SLA is not matched by the currently applied configuration. For example, this can happen after the workload amount has increased. The controller also regularly calls the capacity planning algorithm, to check if the configuration is still the optimal one, i.e. if some resources can be saved. Workload amount Workload amount Workload mix Figure 3: Workload variation Figure 4 compares the real latency measured on the system, and the model prediction for the first configuration κ1. The average latency deviation is 1ms, which represents less than 1% of the average measured latency. Figure 5 compares real and measured abort rates for this configuration. The relative average deviation of the abort rate is 1%. As we can see, the model is close to the real system for both latency and abort rate criteria. Figure 6 compares the real latency measured on the system, and the model prediction for the first configuration κ2. The relative deviation between average latencies is under 1%. Figure 7 compares real and measured abort rates for this configuration. The relative average deviation of the abort rate is under 2%. As we can see, the model is close to the real system for both latency and abort rate criteria. 4. AUTOMATED COTROL 4.1 Controller presentation We designed a controller, responsible of monitoring the system, processing the optimal configuration, and applying this configuration to the controlled Internet service. The controller has only two inputs, the Service Level Agreement and a list of available resources. Capacity Planning. We provide a capacity planning algorithm to find the optimal configuration of a multi-tier Internet service. This algorithm has two main steps. Firstly, it bounds the set of all possible configurations, by rejecting the useless ones. There is no need to add some resources to a configuration that already matches the SLA, so an acceptable configuration defines a bound for the configuration space (we will Workload mix Enforcing Optimal Configuration. Once the optimal configuration has been calculated, we have to apply this configuration to the real system. The architectural configuration is applied by adding and/or removing resources from the Internet service tiers. We apply the local configuration by modifying admission control on each tier. The controller provides actuators for the aforementionned operations. Since all the steps from monitoring to configurations are performed automatically, the controller can adapt the system to varying workload without any human intervention. 4.2 Control evaluation We compared three configurations, two of them being static (κ1 and κ2 we defined in section 3.2.2), and a third one κ3 dynamically configured by our controller. For each configuration, we runned the same experiment, consisting in several steps of varying workload, with peaks and drops. The SLA we specified to the controller is requiring a latency under 7 seconds, and an abort rate under 5%. As we can see on figure 9, κ3 is the only configuration to match the SLA requirement concerning abort rates. Both κ1 and κ2 imply large abort rates when workload amount is too high. This is due to a low architectural configuration for κ1, and a low local configuration for κ2. Figure 8 represents the system latency with each configuration. κ1 latency is above SLA objective most of the time, because this configuration allow a lot of requests to run simultaneously on the same server. This induces trashing and high latency for κ1. The latency objective is matched by κ2 and κ3. We can check system cost on figure 1. κ1 is the most economical configuration, but respects neither latency nor abort rate objectives. The controlled system (κ3) achieves to be more economical than κ2, although it is the only configuration to match both latency and abort rate criteria. Figure 11 represents the architectural and the local configurations of the controlled system (κ3). The controller dynamically adapt configuration to the current workload to match SLA constraints. As we can see, even with high load variations, our approach leads to fast system adaptation and preserves quality of service. 5. RELATED WORK Internet services configuration poses significant challenges in terms of performance, availability and cost [9, 3]. Admission control (i.e. MPL control) and server provisionning are used to improve these criteria. Elnikety et. al. apply admission control at the level of a web server [5], Milan- Franco et. al. use it in database servers [11], whereas Menascé et. al. study it at each tier of a multi-tier system [1]. Some ad-

5 14 12 k1 - Real system k1 - Modeled system k1 - Real system k1 - Modeled system Latency (ms) Abandon rate (%) Figure 4: κ1 Latency with workload variation Figure 5: κ1 Abort rates with workload variation 2 k2 - Real system k2 - Modeled system 15 1 k2 - Real system k2 - Modeled system Latency (ms) Abandon rate (%) Figure 6: κ2 Latency with workload variation Figure 7: κ2 Abort rates with workload variation mission control solutions are proposed in the form of heuristics such as the well-known hill-climbing heuristic [6]. Some approches provide quality of service guarantees through application modeling. Diao et. al. use an analytic model to control MPL in multi-tier systems [4]. Zhang et. al. propose a regression-based modeling of multi-tier applications in order to predict the capacity of applications in terms of allowed concurrent clients [16]. Other approaches control the dynamic provisioning of servers in cluster-based systems. Villela et. al. follow a model-based approach for provisioning the business tier in a multi-tier system [15]. Urgaonkar et. al. apply an analytic model calibrated for the underlying workload mix in order to plan the capacity of multi-tier systems in terms of the number of servers to provision at each tier [14]. Both approaches are motivated by performance objectives; however they require calibrating the model with appropriate parameters. Our present work provides a self-calibrating model for multi-tier Internet services. This model is used to dynamically provision the service and adapt admission control on each of its tier. 6. COCLUSIO In this paper, we present an autonomic Internet services controller, for adapting the configuration of a multi-tier application facing a dynamic workload. The controller goal is to match quality of service objectives while reducing the amount of resources allocated to the application. Our approach is based on two main components: A queuing theory based model of the underlying application, capable of predicting the application performance, and a controller which monitors the system, calculates the optimal configuration, and apply this configuration to the system. The experiments conducted on an online multi-tier application show that our solution may improve quality of service while reducing running costs of the application. 7. REFERECES [1] R. Buyya. High Performance Cluster Computing - Volume 1. Prentice Hall, [2] E. Cecchet, A. Chanda, S. Elnikety, J. Marguerite, and W. Zwaenepoel. Performance Comparison of Middleware Architectures for Generating Dynamic Web Content. In ACM/IFIP/USEIX International Middleware Conference, Rio de Janeiro, Brazil, June 23. [3] D. A. Menascé and V. A. F. Almeida. Capacity Planning for Web Services: Metrics, Models, and Methods. Prentice Hall, 21. [4] Y. Diao, J. L. Hellerstein, S. Parekh, H. Shaikh, and M. Surendra. Controlling Quality of Service in Multi-Tier Web Applications. In 26th International Conference on Distributed Computing Systems (ICDCS 26), Lisbon, Portugal, July 26.

6 12 1 SLA k1 k2 k SLA k1 k2 k3 8 7 Latency (ms) 6 Abandon rate (%) Figure 8: Latency with and without control Figure 9: Abort rate with and without control 12 1 k1 k2 k3 1 8 k3 - AC on tier 1 k3 - AC on tier 2 k3 - LC on tier 1 k3 - LC on tier Cost (# resources) Architectural configuration Local configuration Figure 1: System cost with and without control Figure 11: Controlled system configuration [5] S. Elnikety, J. Tracey, E. ahum, and W. Zwaenepoel. A method for transparent admission control and request scheduling in e-commerce web sites. In The 13th international conference on World Wide Web, 24. [6] H.-U. Heiss and R. Wagner. Adaptive load control in transaction processing systems. In VLDB, [7] J. Hyman, A. A. Lazar, and G. Pacifici. Joint Scheduling and Admission Control for ATS-based Switching odes. In The ACM Conference on Communications Architecture and Protocols (SIGCOMM 92), Baltimore, MA, Aug [8] J. Lee and R. Ben-atan. Integrating Service Level Agreements. Wiley, 22. [9] C. Loosley, F. Douglas, and A. Mimo. High-Performance Client/Server. John Wiley & Sons, ov [1] D. A. Menascé, D. Barbara, and R. Dodge. Preserving QoS of E-Commerce Sites Through Self-Tuning: A Performance Model Approach. In ACM Conference on Electronic Commerce (EC 1), Tampa, FL, Oct. 21. [11] J. Milan-Franco, R. Jimenez-Peris, M. Patino-Martinez, and B. Kemme. Adaptive middleware for data replication. In Middleware 4: Proceedings of the 5th ACM/IFIP/USEIX international conference on Middleware, 24. [12] M. Reiser and S. S. Lavenberg. Mean-value analysis of closed multichain queuing networks. J. ACM, 27(2), 198. [13] TPC. [14] B. Urgaonkar, G. Pacifici, P. Shenoy, M. Spreitzer, and A. Tantawi. Analytic modeling of multitier internet applications. ACM Transactions on the Web (ACM TWEB), 1(1):2, 27. [15] D. Villela, P. Pradhan, and D. Rubenstein. Provisioning Servers in the Application Tier for E-Commerce Systems. ACM Trans. Interet Technol., 7(1), 27. [16] Q. Zhang, L. Cherkasova, and. Mi. A Regression-Based Analytic Model for Capacity Planning of Multi-Tier Applications. Journal of Cluster Computing, 11(3), 28.