Distributed Resource Allocation Strategies for Achieving Quality of Service in Server Clusters

Transcription

1 Proeedings of the 45th IEEE Conferene on Deision & Control Manhester Grand Hyatt Hotel an Diego, CA, UA, Deember 13-15, 2006 Distributed Resoure Alloation trategies for Ahieving Quality of ervie in erver Clusters Björn Johansson, Constantin Adam, Mikael Johansson and Rolf tadler Abstrat We investigate the resoure alloation problem for large-sale server lusters with quality-of-servie objetives, where key funtions are deentralized. peifially, the optimal servie seletion is posed as a disrete utility maximization problem that reflets management objetives and resoure onstraints. We develop an effiient entralized algorithm that solves this problem, and we propose three suboptimal shemes that operate with loal information. The performane of the suboptimal shemes is evaluated in simulations, both under idealized onditions and in a full-sale system simulator. I. INTRODUCTION Large-sale web servies, suh as on-line shopping, autioning, and webasting, rapidly expand in geographial overage and number of users. Current systems that support suh servies, inluding ommerial solutions (IBM Webphere [1] and BEA WebLogi [2]) and researh prototypes (e.g., Ninja [3], Neptune [4]), are based on entralized designs. Centralized designs limit the salability in terms of effiient operation, low onfiguration omplexity and robustness. In our reent work on salable web servies with performane objetives, we addressed these limitations through a deentralized design [5]. The design uses peer-to-peer tehnologies, with proven properties of salability, selforganization and fault-tolerane [6], [7], as building bloks. Fig. 1 shows a possible deployment senario. A large number of idential servers, loated in different data enters, form a global luster with multiple entry points. The luster offers several servies, eah with its own Qo objetives in terms of maximum response time and rejetion rate. To aess a servie, a lient sends a request to an entry point, whih forwards it to a server inside the luster that an proess the request, see [5] for details. Fig. 2 shows the three distributed mehanisms that form the ore of the design. Eah server exeutes these mehanisms periodially and asynhronously. First, topology onstrution, based on an epidemi protool [8], organizes the luster nodes into dynami overlays, whih are used to disseminate state and ontrol information in a salable and robust manner. eond, request routing direts servie requests towards available resoures along an overlay. Third, servie seletion dynamially alloates the luster resoures to servies. This mehanism runs a loal algorithm that periodially assigns all loal resoures to a single servie. The resoures should be assigned suh that the quality of servie targets are reahed. The deision whih The authors are with the hool of Eletrial Engineering, Royal Institute of Tehnology (KTH), tokholm, weden. {bjorn.johansson, onstantin.adam, mikael.johansson, rolf.stadler}@ee.kth.se Fig. 1. Data Center Management tation Database Internet Clients Entry Points A large-sale server luster with multiple entry points. (a) ( b) () Fig. 2. Three deentralized mehanisms ontrol the system behavior: (a) topology onstrution, (b) request routing, and () servie seletion. servie to hoose is based on the loal state of the server and the states of its neighbors, as well as on the external load (supplied by the entry points). This paper fouses on the servie seletion mehanism. The paper is organized as follows. etion II develops a simple model that relates the alloation of the server resoures to the Qo objetives. The resoure alloation problem is formalized as a disrete utility maximization problem, and an effiient entralized algorithm for finding the globally optimal resoure alloation is developed. etion III and IV desribe heuristi ontrol mehanisms that mimi the behaviors of the entralized algorithm and disuss possible randomized shemes. Finally, a omparative evaluation of these strategies, both in idealized Matlab simulations and in detailed full-sale system simulations, is performed. II. MODELING AND PROBLEM FORMULATION The objetive of the system is to ontinuously a luster utility funtion that we define as the sum of servie utility funtions, one for eah servie the luster provides. A servie utility funtion speifies the reward for exeeding and /06/$ IEEE. 1990

2 45th IEEE CDC, an Diego, UA, De , 2006 u (q ) α (ρ q ) Fig. 3. ρ q + q The utility funtion. (q ρ ) β the penalty for missing the Qo targets for a given servie. The relative magnitudes of these rewards and penalties are defined by one or more ontrol parameters assoiated with a servie utility funtion. The partiular hoie of these parameters allows the system to differentiate between servies, whih is important in overload situations. In our design, these parameters an be set from the management station, and the system propagates them to all servers [5]. We model the luster as onsisting of servers that proess requests oming from C different lasses of servies. Moreover, we assume that the number of proessed requests, p, depends linearly on the alloation as follows, p = s=1 k sx s, where x s is the amount of resoures assigned to lass for server s, and k s is the apaity of server s for lass. This implies that the rate of rejeted requests, r, is omputed as, r = l s=1 k sx s, where l is the total load. We then define the rejetion ratio, q, as the rate of rejeted requests, r, divided by the total load, l : q = r /l =1 1 l s=1 k s x s. (1) For eah servie lass,, there is a utility funtion, where the utility inreases with dereasing rejetion ratio, q. The utility funtions are defined as u (q )={ α (ρ q ) if q q + (q ρ ) β elsewhere, where α, β, and ρ are lass speifi parameters that ontrol global system behavior with β > 1 and α,ρ > 0, see Fig. 3. The parameter q + is hosen as the intersetion point between the linear part and the nonlinear part (i.e., q + = ρ +α 1/(β 1) ), to make the funtion ontinuous. Thus, the utility funtions are ontinuous and dereasing, and it an also be shown that they are onave. The nonlinear term is introdued as an attempt to prevent starvation of a servie lass; that is, no servie lass should be without resoures in an overload senario. At a first glane, it may seem to be odd that the utility funtion is not inreasing. This is beause the utility funtion is formulated in terms of the rejetion ratio, and the rejetion ratio should be as low as possible. We would like to work with the resoure alloation for eah servie lass instead of rejetion rates. In the rest of the hapter we assume that the servers have the same apaity for eah lass, k s = k, and to redue notation we introdue x = s=1 x s. We use (1) for variable transformation and (2) do the following definition: ũ (x )={ α ((k /l )x χ ) if x x + (χ (k /l )x ) β elsewhere, where χ = 1 ρ and x + = (l /k ) ( χ α 1/(1 β)). The utility funtions ũ ( ) are ontinuous, inreasing, and onave. The ondition that the rejetion ratio always is greater than zero translates into x l x. This leads to the main optimization problem (posed in the the original x s variables): {x 11,...,x C} =1 ũ( s=1 x s) =1 x s =1, s s=1 x s l /k, C x s {0, 1}, s, C, where = {1,..., } and C = {1,..., C}. The objetive is to utility (essentially minimize the rejetion ratios) under the onstraints that eah server is fully utilized, and that the alloated resoures for eah lass is below the requested load. Finally, the resoures are either one or zero, whih in ombination with the resoure sum onstraint imply that a server will handle only one servie. We an hide the individual server alloation and see the problem at a more abstrat sale by rewriting the main optimization problem (3) using the x variables {x 1,...,x C} =1 ũ(x ) =1 x = x l /k, x nonnegative integer, C C. The problem (4) is a disrete resoure alloation problem with a separable and onave objetive funtion, and it an therefore be solved quite easily in its original form. One well-known method for finding the disrete global optimal solution for this type of problems is the greedy algorithm [9]. The greedy algorithm works as follows: start with zero resoures alloated to all servies. In eah step add one resoure to the servie that have the greatest marginal utility (to be defined below). When the number of alloated resoures reah the maximum value, the global optimum is reahed. More effiient algorithms also exist, but they are more ompliated [9]. In this hapter we present a modified version, the Greedy Variation. To present the algorithm, we need to define the marginal utility as, (y) =ũ (y) ũ (y 1), for C and y {1,..., l /k }, where is the value rounded to the losest integer towards minus infinity. Thus, (y) is the inrease in utility when one resoure is added to y 1 to reah y, and ũ (y) =ũ (0) + y i=1 (i). The algorithm, desribed in detail in Algorithm 1, works as follows: starting from a feasible alloation it takes resoures from servies whih have the least marginal utilities and realloates these resoures to servies with the greatest marginal utilities. The main differene between this algorithm and the standard greedy algorithm is that this algorithm an warm start. If (3) (4) 1991

3 45th IEEE CDC, an Diego, UA, De , 2006 Algorithm 1 Greedy Variation. olves (4). 1: tart with feasible x 0 2: notdone:=true 3: k := 0 4: while notdone do 5: x k+1 := x k 6: i := arg min v { v (x k v)} 7: x k+1 i := x k+1 i 1 8: j := arg max v { v (x k+1 v +1) x k+1 v +1 lv k v } 9: x k+1 j := x k+1 j +1 10: if =1 u (x k+1 )= =1 u (x k ) then 11: notdone:=false 12: k := k +1 13: x := x k the load onditions are hanged slightly, then the optimal server alloation will not hange muh. If the original greedy algorithm is used, then the alloation starts from srath. In ontrast, the Greedy Variation algorithm an exploit the previous alloation and will not need as many realloations, exept in the worst ase. Proposition 2.1: Algorithm 1 onverges to the optimal solution of (4). Proof: ee [10, Appendix B.2]. ine all servers are assumed to know the global load, the number of servers, and the server apaity, they an ompute the optimal alloation on their own. Furthermore, if the servers are ordered from 1 to, then a server an deide whih servie to provide, e.g., for one speifi load, the servers one to ten should provide servie 1 et. We all this the tati Approah, sine the server alloation is stati under stati load. This is the optimal solution with our problem formulation, and its performane is beaten only by an ideal system (by whih we mean a super server with resoures, and apability of assigning non-integer resoures to different servies). However, the tati Approah requires additional global information (the ordering), and it is not deentralized, whih we require. In the rest of the hapter, we will fous on how to approximately solve (4). The approximate solutions should avoid that servers unneessarily hange the type of servie they provide (so alled hurn), sine hurn will yield muh overhead. We will do this using two basi approahes, the Diret Approah and the Referene Approah. In the Diret Approah, the servers try to the utility in (4) using loal information. On the other hand, in the Referene Approah, the servers will solve (4) loally and use the optimal alloation as referene values. They will try trak the optimal alloation as lose as possible using loal information. III. DIRECT APPROACH In the Diret Approah we use an algorithm that is inspired by the Greedy Variation algorithm. We annot diretly use the Greedy Variation algorithm sine it requires global oordination. Rather, we try to approximately solve (4) in a Algorithm 2 Diret Approah. elets servie for server z. 1: for := 1 to C do 2: x Neigh := x z + s N (z) x s { ( )} x Neigh v 3: i := arg min v v N +1 4: if i = z then 5: x Neigh i = x Neigh i { 1( (x Neigh 6: j := arg max v v v +1) N +1 7: with to servie j ( x Neigh j ) x Neigh v := x Neigh j +1) +1 lv k v } neighborhood of eah server. The idea is that eah server estimate the global server alloation using information from its neighborhood. Eah server solves x z ( P C (xz+ =1 ũ s N (z) xs) ) N (z) +1 =1 x z 1 (x z + s N (z) x s) N (z) +1 l k x z {0, 1}, C, where N (z) is the set of all neighbors to server z and N (z) is the number of neighbors. The interpretation of the objetive funtion is that the server omputes the server alloation ratio in its neighborhood, and this value is multiplied with the total number of servers to get an approximate value of the global alloation. The optimization problem (5) is a disrete resoure alloation problem with a separable and onave objetive funtion, and inspired by Algorithm 1, we propose Algorithm 2 to approximately solve it. To go through Algorithm 2 in some detail, let r denote line r in the pseudoode. First, the server omputes the urrent resoure alloation in the neighborhood 1 2. Then the servie, denoted i, that gives the smallest marginal ontribution to the total utility is found 3. If the server is providing servie i, then it an swith to another servie to inrease utility. A resoure is subtrated from servie i and a resoure is added to the servie, denoted j, that gives the greatest marginal ontribution to total utility 4 7. It is also possible to let the servers swith to the servie that gives the maximum inrease in utility, but sine we desire to minimize the hurn, a server swithes only when the server is providing a servie with the least marginal utility. The following example illustrates the disussion above using a speial ase. Example 1: Consider a system with three servies and ten servers. The servers are assumed to have global information and the utility is defined as u(x) =x 1 +10x x 3. The onstraints are that x 1 5,x 2 5,x 3 7, and the starting alloation is x = ( ). We assume that the servers selet servie in the following order: first the servie 2 servers, then the servie 3 servers, and finally the servie 1 servers. If Algorithm 2 is used, then servers providing servie 2 will never swith servie, only the servers providing servie 1 will swith to provide servie 3. However, if the servers (5) 1992

4 45th IEEE CDC, an Diego, UA, De , 2006 Algorithm 3 imple tohasti. elets servie for server z. 1: a :=rand(0, 1); 2: for i := 1 to C do 3: if i 1 =1 a< i 4: with to state i =1 then instead are set to always swith to a servie that gives greater marginal utility, then the servers providing servie 2 first swith to servie 3. Finally, the servers providing servie 1 will swith to servie 2. Thus, in the former ase there is 3 state swithes and in the latter ase there is 6 state swithes. In this example, Algorithm 2 is superior in the sense that the hurn is lower. IV. REFERENCE APPROACH An alternative approah is to separate the main optimization problem: first solve (4) loally, then interpret the optimal solution as referene values, denoted, and try to trak them as losely as possible. The separation is suboptimal and an potentially be done in several ways. The global traking problem that we would like to solve is the following minimize x =1 s=1 x s 2 s 2 C =1 x s 1, s (6) x s {0, 1}, s, C. However, this requires global information whih is in onflit with our desire of a deentralized solution. Thus, we have to be ontent with an approximate solution. There are several approahes how to approximately solve (6) without global oordination, and the approahes we will use an be divided into stohasti and deterministi. The simplest ompletely deentralized stohasti approah is that the normalized referene values are onsidered to be the probabilities of providing the orresponding servie. We also onsider a deterministi approah where we minimize the estimated referene deviation using information from a neighborhood for eah server. A. imple tohasti The basi idea is that the expeted proportion of time spent providing the different servies should be the same as the orresponding referene value divided by the total number of servers, /. The servers swith to provide servie with probability /, see Algorithm 3. We then have that E[X s ]= = xref,for C, s=1 s=1 where X s is the stohasti variable with value one if the server is in state and value zero otherwise. The swithing probability for a server is P ( swith state )= C =1 (1 xref ), (7) and the worst ase is found by maximizing the swithing probability under the onstraints that =1 xref / =1and 0. ine this is a onvex optimization problem, we an use the Karush-Kuhn-Tuker-onditions (see e.g., [11]) to find the worst ase. The worst ase referene values are, x worst = /C, and the worst ase swithing probability is (C 1)/C. Thus, we have the following bounds for the swithing probability 0 P ( swith state ) C 1 C. Note that the upper bound tends to 1 as the number of servie C 1 lasses inreases, i.e., lim C C =1. This implies that if we have a large number of servie lasses, we get large hurn if we use the simple stohasti approah. The imple tohasti approah an be interpreted as if the servers deide whih servie to provide using stationary Markov hains, with the invariant distribution set to the referene values divided by the number of servers. It is also possible to devise more elaborate shemes using Markov hains, but we will not pursue this diretion in this paper. B. Minimize Distane Approah The idea of this approah is to minimize estimated referene deviation or distane. The referene deviation is estimated by using information from the neighbors, and we will use the same method as in the Diret Approah (etion III) but with another utility funtion. Eah server, denoted z below, ompute the optimal solution to the following optimization problem P C x =1 (x z+ s N (z) xs) N (z) +1 2 z 2 C =1 x (8) z 1 x z {0, 1}, C. This problem is on the same form as (5) if we replae ũ (y) with f (y) and (y) with Γ (y), defined as f (y) = y x ref 2 2 and Γ (y) =f (y) f (y 1). The minus sign in the definition of f (y) yields a onave funtion sine the norm is onvex. Therefore, we an approximately solve (8) with Algorithm 2. V. EVALUATION We are now ready to evaluate the proposed servie seletion mehanisms. Evaluations will be performed both in Matlab and in a detailed full-sale system simulator. The Matlab simulations allow us to study the distributed ontrol shemes in an idealized setting lose to our model assumptions, while the full-sale simulations demonstrate the feasibility of our proposals in the real system. A. Matlab imulations We have simulated a luster with 400 servers and three different lasses of servie. Every server is onneted to 20 neighbors and the topology is fixed, randomly generated at the simulation start. The utility parameters are shown in Table I. The load is 110% of the apaity of the luster (overload) and is distributed to lass 1,2, and 3 with the 1993

5 45th IEEE CDC, an Diego, UA, De , 2006 TABLE I ERVICE QO OBJECTIVE AND CONTROL PARAMETER. 0.6 Qo Objetives Parameters Resp. Time [se] Rej. Rate (ρ) [%] α β ervie ervie ervie x 10 4 Proportion of alloated servers Diret Minimize Distane imple tohasti tati Utility tati Diret Minimize Distane Time Fig. 4. Trae of the utility for the tati alloation, Diret Approah, Minimize Distane Approah. The performane of the imple tohasti approah is not inluded sine it is several magnitudes worse than the others. following proportions: 3/6, 2/6, and 1/6. With this setup we have evaluated the Diret Approah, the imple tohasti, and the Minimize Distane Approah. The resulting utilities are shown in Fig. 4 and the assoiated server alloations in Fig. 6. The imple tohasti approah is not inluded in the utility plot sine it is several magnitudes worse than the others and its inlusion would ruin the illustration of the others. We an see that although the server alloations are quite lose to the optimal, the total utility appears to be quite sensitive. This is largely due to the high values of the β-parameters in the utility funtion definitions. The Number of swithes Diret Minimize Distane imple tohasti Time Fig. 5. Number of swithes for the Diret Approah, Minimize Distane, and imple tohasti Time Fig. 6. erver alloation for the three approahes. The top bundle is for servie 1, the mid bundle orrespond to servie 2, and the lower bundle orrespond to servie 3. hurn for the different approahes are shown in Fig. 5. The imple tohasti lies as predited by (7), while the two deterministi shemes have substantially lower hurn. B. Full-sale ystem imulations The full-sale simulator implements the protools for distributed topology onstrution, request routing and servie seletion, and takes into aount the routing, proessing and queuing delays. Using this simulator, we have simulated a system with two entry points and 400 servers. In this setup, eah server has 20 neighbors and runs the servie seletion mehanism every 5 simulation seonds. The maximum servie rate of a server is 8 req/se and that of the luster is thus 3200 req/se. Table I shows the Qo objetives and the utility parameters for eah servie. We model the request arrivals as a Poisson proess, and eah servie has the same average arrival rate. For eah simulation run, we start measuring the output metris after a warm-up phase of 100 se. We measure the performane of three resoure alloation strategies: the imple tohasti strategy, desribed in etion IV-A, the Diret approah, desribed in etion III, and the tati alloation strategy. In the stati alloation strategy, servers are assigned to servies before the start of the simulation. In order to better understand the system performane, we ompare the results against an ideal system. uh a system does not inur any routing delays, and an do frational alloation of resoures to servies. The ideal system ahieves the best possible performane under any operating onditions. Tables II-III show the average values of the server alloations to eah servie and the rejetion rates for eah servie, and Fig. 7 shows a trae of the luster utility. These results were olleted under overload onditions, where the average request arrival rate is 110% of the system apaity, or 3520 req/se. Based on this output we draw the following onlusions. First, the utility generated by the diret alloation approah is larger than the utility generated by the simple stohasti approah. Furthermore, 1994

6 45th IEEE CDC, an Diego, UA, De , 2006 TABLE II ERVER ALLOCATION IN OVERLOAD. trategy ervie 1 ervie 2 ervie 3 Ideal tati imple tohasti Diret TABLE III REJECTION RATE IN OVERLOAD. trategy ervie 1 ervie 2 ervie 3 Ideal 0% 0% 27.27% tati 9.20% 5.58% 27.89% imple tohasti 20.25% 7.32% 21.38% Diret 8.46% 4.74% 29.58% the diret approah results in a server alloation that is loser to the ideal and the stati alloations than the alloation produed by the simple stohasti approah. eond, we observe several spikes in the utility graphs represented in Fig. 7. These spikes are likely aused by the server alloation deviating from the optimal alloation in ombination with load flutuations. As mentioned in the Matlab imulations setion, the shape of the utility funtions, with the high value of the β-parameter, makes the utility highly sensitive to rejetion rates exeeding the maximum rejetion rates. Finally, Table III shows that the rejetion rates for servies 1 and 2 for the diret and the stati approahes are higher than the orresponding rejetion rates of the ideal system, despite a near-optimal alloation of resoures to servies. This strongly suggests that mehanisms other than servie seletion, suh as request routing, have a large influene on the system performane. (In fat, inreasing the number of neighbors of a node from 20 to 50 yields the following rejetion rates for the diret approah: 0.19% for servie 1, 0.05% for servie 2 and 29.77% for servie 3 and the resulting system utility was , a value muh loser to the ideal.) Utility Fig. 7. imulation time (se) Ideal tati Diret Trae of the utility of a luster under overload. VI. CONCLUION AND FUTURE WORK This paper has investigated ontrol mehanisms for ahieving quality-of-servie objetives in large-sale server lusters. We have formalized the resoure alloation problem for our distributed servie arhiteture as a disrete utility maximization problem that reflets management objetives and ritial system onstraints. An effiient entralized algorithm for omputing the optimal resoure alloation has been developed, and three suboptimal shemes that operate on only loal information have been proposed. Properties of the suboptimal shemes have been disussed and evaluated in simulations, both under idealized onditions and in fullsale system simulations. The simulations indiate that the Diret Approah is the best sheme. The Minimize Deviation approah is quite lose to the performane of the Diret Approah, but tends to alloate too few resoures to the premier servie resulting in a lower total utility. everal aspets need to be addressed in our future work. A formulation of the resoure alloation problem that inludes the ost of servie hanges (hurn) should be developed, and the interation between the servie seletion and the other ontrol funtions in our arhiteture should be understood in more detail. For example, we have observed that the server onnetivity has a profound impat on the performane in the full system simulations. Finally, performane guarantees or bounds should be developed for the algorithms, to assure effiient and effetive operation under all operating onditions. VII. ACKNOWLEDGMENT The authors wish to thank the anonymous reviewers for their helpful omments. REFERENCE [1] IBM Webphere software, February [Online]. Available: [2] BEA ystems - BEA WebLogi, February [Online]. Available: htm\&fp=/ontent/produts/weblogi [3]. D. Gribble, M. Welsh, R. von Behren, E. A. Brewer, D. Culler, N. Borisov,. Czerwinski, R. Gummadi, J. Hill, A. Joseph, R. Katz, Z. Mao,. Ross, and B. Zhao, The ninja arhiteture for robust internet-sale systems and servies, Journal of Computer Networks, vol. 35, no. 4, Marh [4] K. hen, H. Tang, T. Yang, and L. Chu, Integrated resoure management for luster-based internet servies, in ODI 02, [5] C. Adam and R. tadler, A middleware design for large-sale lusters offering multiple servies, etransations on Network and ervie Management (etnm), vol. 2, no. 2, [6] P. Yalagandula and M. Dahlin, A salable distributed information management system, in ACM IGCOMM, [7] P. T. Eugster, R. Guerraoui, A.-M. Kermarre, and L. Massoulie, From epidemis to distributed omputing, IEEE Computer, vol. 5, no. 37, pp , May [8] M. Jelasity, W. Kowalzyk, and M. van teen, Newsast omputing, Department of Computer iene, Vrije Universiteit, Teh. Rep. IR- C-006, November [9] T. Ibaraki and N. Katoh, Resoure Alloation Problems, ser. Foundations of omputing. MIT Press, [10] B. Johansson, Distributed Resoure Alloation in Networked ystems using Deomposition Tehniques, Royal Institute of Tehnology (KTH), Teh. Rep., Aug. 2006, Lientiate Thesis. [11]. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press,