Achieving class-based QoS for transactional workloads

Transcription

1 Achevng class-based QoS for transactonal workloads Banca Schroeder Mor Harchol-Balter Carnege Mellon Unversty Department of Computer Scence Pttsburgh, PA USA <banca, Arun Iyengar Erch Nahum IBM T.J. Watson Research Center Yorktown Heghts, NY USA Abstract In e-commerce applcatons certan classes of users desre mean response tme guarantees and are wllng to pay for ths preferental level of servce. Unfortunately, todays commercal DBMS, whch le at the heart of most e- commerce applcatons, do not provde adequate support for class-based qualty of servce guarantees. When desgnng methods for provdng such guarantees, t s furthermore desrable that they be effectve across workloads and not rely on changes to DBMS nternals for portablty and ease of mplementaton. Ths paper presents an External Queue Management System (EQMS) that strves to acheve the above goals. 1. Introducton Transacton processng systems le at the core of modern e-commerce applcatons such as on-lne retal stores, banks and arlne reservaton systems. The economc success of these applcatons depends on the ablty to acheve hgh user satsfacton, snce a sngle mouse-clck s all that t takes a frustrated user to swtch to a compettor. Gven that system resources are lmted and demands are varyng, t s dffcult to provde optmal performance to all users at all tmes. However, often transactons can be dvded nto dfferent classes based on how mportant they are to the onlne retaler. For example, transactons ntated by a bg spendng clent are more mportant than transactons from a clent that only browses the ste. A natural goal then s to ensure short delays for the class of mportant transactons, whle for the less mportant transactons longer delays are acceptable. It s n the fnancal nterest of an onlne retaler to be able to ensure that certan classes of transactons (fnancally lucratve ones) are completed wthn some target mean response tme. It s also fnancally desrable for the onlne retaler to be able to offer a Servce Level Agreement (SLA) to certan customers, guaranteeng them some target mean response tme that they desre (wth possble deterorated performance for customers wthout SLAs). Ths paper proposes and mplements algorthms for provdng such performance targets on a per-class bass A guaranteed mean response tme for some class of transactons s one form of a Qualty of Servce (QoS) target. In many stuatons t s useful to provde more general QoS targets such as percentle targets, where x% of response tmes for a class are guaranteed to be below some value y. Percentle targets are often demanded by clents as part of a Servce Level Agreement (SLA), for example to ensure that at least 9% of the clent s transactons see a response tme below a specfed threshold. In addton to perclass response tme and percentle targets, another common QoS target s to provde low varablty n response tmes. The reason s that users may judge a relatvely fast servce stll unacceptable unless t s also predctable [5, 11, 25]. Because the domnant tme assocated wth servng an e-commerce transacton s often the tme spent at the backend database (rather than the front-end web/app server), t s mportant that the QoS be appled to the backend database system to control the tme spent there. Yet, commercal database management systems (DBMS) do not provde effectve servce dfferentaton between dfferent classes of transactons. In desgnng a framework for provdng class-based QoS targets one strves for the followng hgh-level desgn goals: Dverse per-class QoS target metrcs The system should allow for an arbtrary number of dfferent classes, where the classes can dffer n ther arrval rates, transacton types, etc. Each class s assocated wth one or more QoS targets for (per-class) mean response tme, percentles of response tme, varablty n response tme, best effort, or any combnaton thereof. Portablty and ease of mplementaton Ideally the system should be portable across DBMS, and easy to mplement.

2 Self-tunng and self-adaptve The system should deally have few parameters, all of whch are determned by the system, as a functon of the QoS targets, wthout nterventon of the database admnstrator. The system should also automatcally self-adapt to changes n the workloads and QoS targets. ncomng transactons external queue DBMS MPL=4 Effectve across workloads Database workloads are dverse wth respect to ther resource utlzaton characterstcs (CPU, I/O, etc.). We am for a soluton whch s effectve across a large range of workloads. No sacrfce n throughput & overall mean response tme Achevng per-class targets should not come at the cost of an ncrease n the overall (over all classes) mean response tme or a drop n overall throughput. Wth respect to the above desgn goals, the pror work s lmted. Commercal DBMS provde tools to assgn prortes to transactons, however these are not assocated wth any specfc response tme targets. Research on real-tme databases does not consder mean per-class response tme goals, but rather looks only at how an ndvdual transacton can be made to ether meet a deadlne or be dropped (we never drop transactons). The only exstng work on per-class mean response tme guarantees for databases s based on modfed buffer pool management algorthms [6, 7, 14, 22]. These technques are not effectve across workloads, snce they focus only on one resource: Tunng the buffer pool wll for example have lttle effect on CPU-bound or lock-bound workloads. Moreover, they don t cover more dverse QoS goals such as percentle or varablty goals. A major lmtaton of all the above approaches s that they rely on changes to DBMS nternals. Ther mplementaton depends on complex DBMS specfcs and s nether smple, nor portable across dfferent systems. Our approach ams at achevng the above hgh-level desgn goals through an external frontend scheduler. The scheduler mantans an upper lmt on the number of transactons executng smultaneously wthn the DBMS called the Mult-Programmng Lmt, or MPL (see llustraton n Fgure 1). If a transacton arrves and fnds MPL number of transactons already n the DBMS, the arrvng transacton s held back n an external queue. Response tme for a transacton ncludes both watng tme n the external queue (queueng tme) and tme spent wthn the DBMS (executon tme). The mmedately apparent attrbute of our approach s that t lends tself to portablty and ease of mplementaton as there s no dependence on DBMS nternals. Also movng the schedulng outsde the DBMS, rather than schedulng ndvdual DBMS resources (such as the bufferpool or lock queues), makes t effectve across dfferent workloads, ndependent of the resource utlzaton. Fgure 1. Smplfed vew of mechansm used to acheve QoS targets. A fxed lmted number of transactons (MPL=4) are allowed nto the DBMS smultaneously. The remanng transactons are held n an unlmted external queue. Response tme s the tme from when a transacton arrves untl t completes, ncludng queueng tme. Wth respect to obtanng dverse QoS targets, the core dea s that by mantanng a low MPL, we obtan a better estmate of a transacton s executon tme wthn the DBMS, and hence we are able to mantan accurate estmates of the per-class mean executon tmes. Ths n turn gves us an upper bound on the queueng tme for a transacton, whch can be used by the scheduler n order to ensure that QoS targets are met. The actual algorthms that we use are more complex and rely on queueng analyss n order to meet a more dverse set of QoS targets, and behave n a self-adaptve manner. The external scheduler acheves class dfferentaton by provdng short queueng tmes for classes wth very strngent QoS targets, at the expense of longer queueng tmes for classes wth more relaxed QoS targets. There are no transactons dropped. One nherent dffculty n ths approach s that not every set of targets s feasble, e.g., not every class can be guaranteed a really low response tme. An external scheduler therefore also needs to nclude methods for determnng whether a set of QoS targets s feasble. The effectveness of the external schedulng approach and whether t requres sacrfces n overall performance (e.g. throughput or mean response tme) depends on the choce of the MPL. For schedulng to be most effectve a very low MPL s desrable, snce then at any tme only a small number of transactons wll be executng nsde the DBMS (outsde the control of the external scheduler), whle a large number are queued under the control of the external scheduler. On the other hand, too low an MPL can hurt the overall performance of the DBMS, e.g., by underutlzng the DBMS resources resultng n a drop n system throughput. Therefore, another core problem an external scheduler needs to solve s that of choosng the MPL. In ths paper we propose and mplement a unfed external schedulng framework called EQMS (External Queue Management System) that addresses all of the above problems. Fgure 2 gves an overvew of the EQMS archtecture.

3 Fgure 2. Overvew of the EQMS system. The EQMS takes as nput a set of classes wth one or several QoS targets for each class. These are specfed by the onlne retaler and are not part of the EQMS. The core component of the EQMS s the Scheduler whch decdes on the order n whch transactons are dspatched to the DBMS such that the assocated QoS targets are met. The scheduler reles on the MPL Advsor to determne an MPL that provdes suffcent schedulng control, whle keepng performance penaltes, such as loss n throughput, below a threshold defned by the DBA (database admnstrator). The MPL Advsor also checks for the feasblty of a gven set of targets. The EQMS combnes feedback control (based on nformaton collected by the Performance Montor) wth queueng theory to operate n a self-tunng and self-adaptve fashon. We demonstrate the effectveness of our soluton n experments wth two dfferent DBMS, IBM DB2 and PostgreSQL. We create a range of workloads, ncludng CPUbound, I/O-bound, and hgh vs. low lock contenton workloads, based on dfferent confguratons of TPC-C [23] and TPC-W [24]. We show that our solutons apply equally well across all workloads studed. The reason s that the core dea of lmtng the MPL reduces contenton wthn the DBMS at the bottleneck resource, ndependent of what the partcular bottleneck resource s. The paper s organzed as follows: Secton 2 revews related work. Secton 3 descrbes the expermental setup. Secton 4 detals the algorthms used by the Scheduler to acheve class-based mean response tme targets and Secton 5 explans how to schedule for more complex QoS goals, ncludng percentle and varablty goals. Secton 6 descrbes the technques used by the Scheduler and the MPL Advsor to adapt n dynamc envronments. We conclude n Secton Related work When lookng at pror work on provdng QoS guarantees for DBMS transactons two ponts are apparent: Frst, pror work focuses on schedulng datbase nternal resources and hence requres modfcatons to DBMS nternals; our goal s to provde QoS guarantees externally, transparent to the underlyng DBMS. Second, only per-class mean response tme targets have been consdered; our goal s to provde methods for a wder range of QoS targets, ncludng varablty or percentle targets. Below we descrbe pror work on provdng guarantees for DBMS transactons. Most of the work s n the area of real-tme DBMS (RTDBMS), whch s concerned wth deadlnes rather than targets nvolvng mean response tme. Commercal DBMS provde tools to assgn prortes to transactons, however these are not assocated wth any specfc response tme targets. The lttle work that nvolves per-class guarantees s prmarly smulaton-only, and does not cover complex QoS goals such as percentle or varablty goals, and s not portable n that t requres the modfcaton of database nternals (e.g. the bufferpool manager). Work on RTDBMS In Real-tme DBMS, there s a deadlne (typcally a hard deadlne) assocated wth each transacton. The goal of RTDBMS s to mnmze the number of transactons whch mss ther deadlnes. If a hard deadlne s mssed, the transacton s dropped. Examples of work n ths area nclude: [1 4, 12]. Ths work s dfferent from our own n that t does not allow for mean response tme targets or varablty targets. Also, n our work, no transactons are dropped. The RTDBMS typcally nvolves usng a specalzed database engne, and the mechansm s mplemented nternally, makng t less portable. Commercal DBMS As a testament to the mportance of the problem of provdng dfferent servce levels most commercal DBMS provde prorty mechansms n some form. For example, both IBM DB2 [16] and Oracle [2] offer CPU schedulng

4 tools for prortzng transactons. Although dfferent classes are gven dfferent prortes wth respect to system resources, t s not clear how these prorty levels relate to achevng specfc response tme targets. Towards ths end, Krass et al. [15] try to map each class to some fxed prorty such that schedulng based on prortes wll meet the desred response tme targets. Such an assgnment of prortes to classes does not always exst. Towards per-class mean response tme targets Carey et al. [1] consder the stuaton of two classes, where they strve to make the mean response tme for the hgh prorty class as low as possble by schedulng nternal DBMS resources on a read-only workload. Ther work s a smulaton study. In our recent work [18] we consder the same problem for a more general workload (TPC-C and TPC-W) under a varety of DBMS va an mplementaton of (DBMS nternal) lock schedulng and CPU schedulng. More closely related to our current work are the followng papers, [6 8,22], all of whch have multple classes each wth a dfferent mean response tme target. Other QoS targets are not consdered. Ther approach s to schedule nternal memory (buffer pool management). The above are all smulaton studes. 3. Expermental setup As representatve workloads for transactonal web applcatons, we choose the TPC-C [23] and TPC-W [24] benchmarks. The TPC-C workload n ths study s generated usng software developed at IBM. The TPC-W workload s generated usng the TPC-W Kt from PHARM [9], though mnor modfcatons are made to mprove performance, ncludng rewrtng the connecton poolng algorthm to reduce overhead. Dfferent confguratons of these workloads (number of warehouses, number of clents) result n dfferent levels of resource utlzaton for the hardware resources: CPU and I/O. We experment wth 4 dfferent confguratons of TPC- C and TPC-W as shown n Table 1(top). We chose these confguratons n order to cover dfferent combnatons of resource utlzaton levels (see Table 1(bottom)). In addton, varyng the confguraton wll also result n dfferent levels of lock contenton [18]. For example, lock contenton s a large component of a transacton s lfetme n workloads W I and W I>CP U, but not n the other workloads. The TPC-C and TPC-W benchmarks are defned to be used as closed systems, and we use them ths way. We assume a zero thnk tme throughout. All results n the paper have been repeated wth non-zero thnk tmes, and wth open system confguratons and results have been found to be smlar. Due to a lack of space, unless otherwse stated, we show only the results for zero thnk tmes, allowng us Workload Benchmark Confg Data- CPU I/O base load load W IO TPC-C 4 WH, 4GB low hgh 1 clents W IO>CP U TPC-C 1 WH, 1GB med. hgh 1 clents W CP U>IO TPC-W 1 EBs, 3MB hgh med. Shoppng 1K tems, 14K customers W CP U TPC-W 1 EBs, 3MB hgh low Browsng 1K tems, 14K customers Table 1. Descrpton of the expermental workloads. to focus on the effect of varyng the MPL n all the graphs. The DBMS we experment wth are IBM DB2 [16] verson 8.1, and PostgreSQL [19] verson 7.3. Due to lack of space, all results graphs throughout the paper pertan to the IBM DB2 DBMS. Results for PostgreSQL are very smlar and we descrbe these n words only. In all experments the DBMS s runnng on a 2.4-GHz Pentum 4 wth 3GB RAM, runnng Lnux , wth a buffer pool sze of 2GB. The machne s equpped wth two 12GB IDE drves, one of whch we use for the database log and the other one for the data. The clent generator s run on a separate machne wth the same specfcatons as the database server, and s drectly connected to the database server through a network swtch. 4. Achevng response tme targets In ths secton we assume that each of the QoS targets s a specfc mean response tme target for each class. Specfcally, class transactons have a target mean response tme of τ. After ntroducng some notaton, we explan the algorthms used by the Scheduler to acheve the per-class response tme targets and to determne whether a set of targets s feasble Notaton The notaton we use n order to formally explan the external schedulng algorthms s summarzed n Table 2, and s straghtforward. The mean response tme of transactons s denoted by T, and can be dvded nto T Q and, where the former denotes the mean tme the transactons spend queueng externally to the DBMS and the latter quantty s the mean tme that the transactons spend wthn the DBMS. That s, T = T Q +

5 T Q T Q T DBMS T T R R τ p t curr T x% DBMS x% τ x% Mean tme transactons spend watng n external queue n system wth external schedulng Mean tme transactons spend executng n the DBMS n system wth external schedulng Mean tme transactons n class spend watng n external queue Mean tme transactons n class spend executng n the DBMS n system wth external schedulng Overall mean response tme,.e. sum of T Q and Mean response tme n orgnal (no external schedulng) system Mean response tme of class transactons n orgnal (no external schedulng) system Mean response tme target of class Fracton of transactons that are class current tme x-th percentle of T x-th percentle of T DBMS Target for x-th percentle of T Table 2. Notaton Furthermore, we denote the per-class response tmes va a subscrpt denotng class, where T denotes the mean response tme for class transactons, and T = T Q + Notce that the above notaton s dfferent from the τ s whch denotes the th class mean response tme target. Lastly, we defne R to be the mean response tme n the orgnal system, wthout external schedulng. The remanng notaton wll be explaned as needed. Measurements of the quanttes ntroduced above,.e. the queung tmes, executon tmes, and total mean response tmes, both per class and aggregated, are needed by the Scheduler and the MPL Advsor and are therefore tracked by the Performance Montor The basc algorthm The Scheduler reles on the MPL Advsor to choose an MPL so that the tme spent wthn the DBMS s low and predctable. In partcular, snce the Scheduler cannot control the tme a transacton takes to execute nsde the DBMS, the MPL has to be low enough such that for each class the expected tme wthn the DBMS s lower than the class response tme target. Gven n QoS classes wth response tme targets {τ 1,..., τ n }, the MPL needs to ensure that for each class < τ The man queston for the Scheduler s then how to order the transactons wthn the external queue to acheve the targets. Observe that for each class the Scheduler knows the mean target response tme τ and can obtan the mean database executon tme T DBMS from the Performance Montor. It can therefore determne how much slack t has n schedulng transactons from ths class: transactons n class can afford on average to wat up to but not more than s = τ tme unts n the external queue before they should start executng n the DBMS. Based on the slack of a transacton the Scheduler computes a tmestamp for when the transacton should be dspatched out of the external queue and nto the DBMS, whch we call the dspatch target tme. Formally, f a new transacton of class arrves at tme t a ts dspatch target t d s t d = t a + s = t a + τ. Whenever a transacton completes at the DBMS, and we have to pck the next transacton for executon from the external queue, we pck the transactons n ncreasng order of ther dspatch targets (t d value). We demonstrate the vablty of the above algorthm expermentally. The above hgh level descrpton omts an mportant ssue arsng n practce: how does the Scheduler adjust to surges and fluctuatons n system load whch mght make t mpossble to acheve all the targets. Ths practcal concern wll be addressed n Secton Feasblty of assgnment The Scheduler dstngushes two types of nfeasble targets. The frst one has already been explaned above and ncludes per-class targets that are lower than the per-class mean executon tme (.e. the average tme spent nsde the DBMS). The mean response tme of a class s the sum of ts queueng tme and ts executon tme, and can obvously not be smaller than ether one of ts components. The frst condton for a target to be feasble s therefore the followng: < τ

6 It s mportant to note that the mean executon tme T DBMS depends on the MPL: a smaller MPL leads to less contenton at the DBMS and therefore to shorter executon tmes. It s therefore the goal of the MPL Advsor to recommend an MPL that meets the above condton, provded that ths does not come at a performance penalty (e.g. n terms of throughput loss) beyond what the DBA has specfed as tolerable. If the MPL Advsor cannot determne an MPL that satsfes the above condton, then the target τ s not feasble. The detals of how the MPL Advsor works wll be explaned n Secton 6.1. The second type of nfeasble targets comprses those that cannot be acheved due to a smple lack of system resources (e.g. suppose every class desres a really low response tme guarantee). More precsely, we don t expect the overall mean response tme under class-based prortzaton to be lower than for the unprortzed system. That s the weghted average over all per-class mean response tme targets s not expected to be lower than the mean response tme n the orgnal unprortzed system. We descrbe a smple condton for determnng whether a set of per-class mean response tme targets s feasble,.e., whether the set of targets s achevable va some algorthm. We start by defnng the overall target mean response tme (aggregated over all classes): τ overall = n p τ =1 Recall R represents the mean response tme n the orgnal system (wthout schedulng). Obvously a necessary condton for achevng the ndvdual τ s (va some orderng of the external queue) s that τ overall > R We now argue (only ntutvely) that ths also represents a suffcent condton. The crux of the argument s that the external schedulng (wth the lmted MPL) does not ncrease the overall measured mean response tme T as compared wth the orgnal R. That s, when the MPL s chosen carefully (as n Secton 6.1) we have T R Hence the above condton also mples whch s suffcent. τ overall > T 4.4. Results for mean response tme targets We expermentally evaluate the accuracy of the Scheduler n achevng per-class mean response tme targets usng the four workloads n Table 1. In all experments we use an MPL of 2, snce for our workloads ths MPL s hgh enough that nether throughput nor overall mean response tme s sacrfced. Tables 3 and 4 show detaled results for W I and W CP U respectvely under IBM DB2. Results are shown n the Measured column correspondng to the QoS target specfed n the prevous two columns. Mean and max values are specfed for a sequence of 1 expermental runs, each consstng of 25, transactons. At the moment, we are only concerned wth the frst two rows of these tables, whch consder per-class response tme targets. We have expermented wth three dfferent classes wth dfferent targets and frequences. As shown n the tables, we are always able to acheve wthn 8% of the desred per-class response tme targets for W IO (Table 3) and wthn 12% of the desred perclass response tme targets for W CP U (Table 4). Recall that W CP U corresponds to the TPC-W workload whch s more varable. Results for W I>CP U are very smlar to those for W I wth errors rangng between -5% and results for W CP U>IO are very smlar to those for W CP U wth errors rangng between 1-11%. We also repeated all experments for PostgreSQL, where results are slghtly worse but stll wthn 15% of the targets. 5. More complex QoS targets In ths secton we consder more complex QoS targets. These nclude: Reducng overall varance n response tmes (aggregated over all class transactons)(secton 5.1); and achevng targets on the x th percentle of response tme for multple classes (Secton 5.2) Reducng varance In addton to desrng low mean response tme, users are equally desrous of low varablty n response tmes [5]. Both W CP U and W IO benchmarks are composed of a fxed set of transacton types. We fnd that although the varance wthn each transacton type s not too hgh, the overall varance n response tme across all transacton types s qute hgh. Specfcally, for W CP U, the squared coeffcent of varaton (C 2 ) for ndvdual transacton types ranges from C 2 = 2 to C 2 = 5; however over all transacton types, we measure C 2 = 15. W IO s less varable. For ndvdual transacton types we measure values rangng from C 2 =.15 to C 2 =.8, whle lookng across all transacton types we measure C 2 = 2.3. As a reference pont, the exponental dstrbuton has C 2 = 1. Fgure 3(left column) shows the (orgnal) response tmes for the dfferent transactons under W CP U and W IO, for IBM DB2. Our approach to combattng varablty s to decrease the response tme of the long transactons (by gvng them prorty) and n exchange ncrease the response

7 Experment type Response tmes Response tmes Percentles Class Frequency Prorty QoS target Target Measured Mean Max Avg. error C1 1% 1 Resp Tme.7 sec % C2 2% 2 Resp Tme 1 sec % C3 7% N/A Best effort N/A N/A C1 4% 1 Resp Tme.6 sec % C2 4% 2 Resp Tme 1.3 sec % C3 2% N/A Best effort N/A N/A C1 1% 1 8th %tle 1 sec % C2 1% 2 95th %tle 2 sec % 8th %tle: N/A C3 8% N/A Best effort N/A 95th %tle: N/A C1 2% 1 8th %tle 1 sec % Percentles C2 2% 2 95th %tle 2 sec % C3 6% N/A Best effort N/A 8th %tle: N/A 95th %tle: N/A Varablty C1 1% 1 Reduce var N/A C 2 =.18 (before C 2 = 2.3) C1 1% 1 Resp Tme.7 sec % C2 1% 2 Resp Tme 1 sec % Combned C3 1% 3 8th %tle 1 sec % C4 1% 4 95th %tle 2 sec % C5 6% N/A Best effort N/A 1.7 Experment type Response tmes Response tmes Percentles Percentles Table 3. Summary of results for dfferent QoS targets for W IO. Class Frequency Prorty QoS target Target Measured Mean Max Avg. error C1 1% 1 Resp Tme 3 sec % C2 2% 2 Resp Tme 6 sec % C3 7% N/A Best effort N/A N/A C1 25% 1 Resp Tme 2.5 sec % C2 25% 2 Resp Tme 6.5 sec % C3 5% N/A Best effort N/A N/A C1 1% 1 8th %tle 3 sec % C2 1% 2 95th %tle 12 sec % C3 8% N/A Best effort NA 8th %tle: N/A 95th %tle: N/A C1 2% 1 8th %tle 3 sec % C2 2% 2 95th %tle 9 sec % C3 6% N/A Best effort N/A 8th %tle: N/A 9th %tle: N/A Varablty C1 1% 1 Reduce var N/A Combned C 2 =.19 (before C 2 = 15) C1 1% 1 Resp Tme 3 sec % C2 1% 2 Resp Tme 6 sec % C3 1% 3 8th %tle 2.5 sec % C4 6% N/A Best effort N/A 8th %tle N/A 95th %tle N/A Mean N/A Table 4. Summary of results for dfferent QoS targets for W CP U.

8 no QoS 14 QoS Mean Response Tme (sec) Mean Response Tme (sec) Request Type Request Type (W CP U before QoS) (W CP U after QoS) Mean response tme (sec) Mean response tme (sec) Transacton type Transacton type (W IO before QoS) (W IO after QoS) Fgure 3. QoS target reducng varablty: Results for W CP U (top) and W IO (bottom). tme of the short transactons, where the goal s to make all transacton response tmes as close to the overall mean response tme as possble. Ths turns out to be possble because n typcal workloads the fracton of transactons wth very long response tmes s qute small as compared wth the fracton of transactons wth short response tmes (as n Pareto s 8-2 rule ). Fgure 3(rght column) shows the results of equalzng the response tmes, hence greatly decreasng varance. Under IBM DB2, for W CP U we are able to decrease C 2 from 15 to.19. For W IO we are able to decrease C 2 from 2.3 to.11. These results are summarzed n Tables 3 and 4. Under PostgreSQL, for W CP U we are able to decrease C 2 from 14 to.9. For W IO we are able to decrease C 2 from 1.6 to.8. The exact algorthm for reducng varablty s easy to mplement wthn our external schedulng framework. We start wth the measured overall mean response tme of the orgnal system R. We denote the mean response tme for the th transacton type by T. Intally some of the T s are hgher than R and some are lower. To make the system more predctable, we assgn type transactons a target mean response tme of τ = R. We then apply the standard method for achevng per-class target mean response tmes. For ths method to work, t s mportant to note that t s desrable that the varablty wthn each type s low, so that each type s more predctable. For many OLTP servers, e.g. the database backend of a Web ste, ths s the case: There are a lmted number of possble transacton types that the user nterface allows for, e.g. orderng, product search, retrevng shoppng cart contents, and these transacton types are each lmted n scope, resultng n low response tme varablty wthn each type Meetng xth percentle targets Mean target response tmes are loose n that they can be heavly nfluenced by a small percentage of transactons. It s concevable that some customers mght prefer stronger guarantees, namely that 9%, say, of ther transactons have response tmes strctly below some target. In ths secton we descrbe how to obtan per-class percentle target guarantees. Consder the example of settng a 9 th percentle target denoted by τ 9% for the transactons n class. Our approach for mean response tme targets doesn t apply to percentle targets. Thus we need a new approach. Our percentle target approach has two parts: Frst the MPL Advsor determnes an MPL value whch ensures a 9 th percentle target on just the executon tme T DBMS,.e. DBMS 9% T < τ DBMS 9% where T denotes the 9 th percentle of executon tmes. We next defne an algorthm for schedulng the external queue that uses T to acheve a 9 th percentle DBMS 9% target on the response tme for class. The second step of our approach s to convert,9% nto a 9 th percentle result for response tme. Observe that f the queueng tme T Q s bounded by some c, the resultng 9 th percentle response tme s bounded by T 9% c +,9% That means, when schedulng a transacton, n order to ensure a gven percentle target τ 9%,.e. ensure that T 9% τ 9%

9 the amount of slack we have n schedulng ths transacton s τ 9%,9% We can hence translate percentle targets to dspatch targets as follows: assgn a transacton wth target target τ 9% the dspatch target of: t d = t curr + τ 9%,9% As before we schedule transactons from the queue n order of ncreasng dspatch targets. Tables 3 and 4 show results for varous experments wth per-class percentle target targets under IBM DB2 for W IO and W CP U. As shown, n all experments, for both workloads, we are able to acheve our percentle response tme targets usually wthn 3%. Results for W IO>CP U and W CP U<IO are comparable, wth errors n achevng percentle targets rangng from 1-4%. For our experments wth PostgreSQL ths number becomes 1% Combnaton targets Fnally, t s qute plausble that () dfferent customer classes mght desre dfferent types of QoS targets, and () a gven customer class mght smultaneously request multple targets (e.g., a target for the mean and a percentle target). Both of these combnaton scenaros are easy to acheve n our external schedulng framework snce all targets are mmedately mapped to dspatch targets and transactons are then pulled from the external queue n order of these targets. A transacton havng multple QoS targets s assgned the most strngent of all of ts correspondng dspatch targets. Some results nvolvng combnaton targets are shown for IBM DB2 n Tables 3 and 4, and all targets are acheved wth very hgh accuracy. 6. Makng the EQMS self-tunng and adaptve Ths secton detals the self-tunng and self-adaptve features of the EQMS that make t robust to dynamc stuatons. We frst explan how the MPL Advsor tunes and dynamcally adapts the MPL, the most mportant parameter of the EQMS. We then dscuss how the Scheduler copes wth varyng load condtons, especally wth sudden load surges Detals of the MPL Advsor One bg advantage of the EQMS approach s that there are very few parameters to tune. Essentally, the one most mportant parameter s the MPL. The proper choce of the MPL s crucal, though, snce too hgh an MPL wll provde the scheduler wth only lttle control on class dfferentaton, whle a too low MPL can harm the overall system performance. Below we frst descrbe how the MPL Advsor determnes a lower bound on the MPL (to lmt loss n throughput, and ncrease n response tme) and then how t determnes an upper bound on the MPL (to allow suffcent control for achevng a gven set of QoS targets). Determnng a lower bound on the MPL There are two potental rsks nvolved n choosng the MPL too low. Frst, a low MPL may cause the DBMS resources to be underutlzed, leadng to loss n throughput. Second, a low MPL may ncrease overall mean response tme, snce t enforces a strcter queueng of transactons, resultng n short transactons beng forced to queue behnd long ones (Head-of-Lne-Blockng). Our expermental results across varous workloads and system confguratons show that the wrong choce of MPL can result n a drop n throughput by a factor of 1 and a more than tenfold ncrease n overall mean response tme. What the MPL Advsor does s to fnd a lower bound on the MPL whch lmts the above problems such that throughput loss and ncrease n mean response tme are wthn tolerable range (where tolerable s specfed by the DBA). Fndng a good lower bound s a dffcult problem (and s left as an open problem even n recent publcatons [13]). Our basc approach s to use a control loop augmented wth queueng theoretc gudance. We develop queueng theoretc models that capture the basc propertes of the relatonshp between system throughput and response tme and the MPL. Analyss of the models (parameterzed based on the gven system and workload) provdes us wth a good ntal MPL value, whch we then fne-tune through a control-loop. Jump-startng the control-loop wth a close-to-optmal startng value provdes fast convergence tmes, even when usng only small conservatve constant adjustments n each teraton. The detals are nvolved and addressed n a separate paper [21], also n submsson. Determnng an upper bound on the MPL In Secton 4.3 we have dentfed several necessary condtons for a set of QoS targets to be feasble. In partcular, for each class that has a mean response tme target assocated wth t the followng condton needs to hold < τ and for each class j wth a percentle target DBMS 9% Tj < τj 9% DBMS 9% needs to hold. Snce both and Tj are affected by the MPL (a hgher MPL wll lead to hgher contenton at the DBMS and therefore to hgher T DBMS DBMS 9% and T values) the feasblty of a set of targets j

10 Executon tme (sec) W CPU W IO Response tme (msec) th percentle 9th percentle 8th percentle 6th percentle 4th percentle Multprogrammng lmt MPL Fgure 4. The mean executon tme as a functon of the MPL under W IO and W CP U. Fgure 5. Percentle of the executon tme at the server for dfferent MPLs for W IO. depends on the MPL. The goal of the MPL Advsor s to choose an MPL such that the above condtons are met. The basc mechansm s a control loop that dynamcally adjusts the MPL based on measurements provded by the s hgher than desred, the MPL s reduced (provded the lower bound on the MPL determned n the prevous subsecton s not volated). In order to decde how much the MPL needs to be adjusted the MPL Advsor uses queueng theoretc gudance to provde fast convergence. The detals of the algorthm are explaned below, frst for response tme targets then for percentle targets. Determnng the rght MPL for ensurng the feasblty of response tme targets would be trval f we had a functon that descrbes the exact relatonshp between the MPL Performance Montor; f the measured. Whle we cannot know the exact functon, queueng theory reveals a crucal property of ths functon: Accordng to Lttle s law [17] the expected executon tme T s lnear n the MPL value. Fgure 4 verfes 1 ths law n experments for W IO and W CP U. Based on ths observaton, we can tune the MPL n a control loop smlar to the followng: and the mean executon tme 1. Perodcally montor the per-class executon tmes. 2. Adjust the MPL f for any class the mean executon tme ncreases above the per-class target,.e. > τ 3. Assumng that for some class the executon tme s a factor of f > 1 tmes the per-class target target,.e. we adjust the MPL as follows: = f τ MP L new := MP L old 1/f 1 In the case of W CP U, the lne s not as straght as for W IO, snce the workload s created usng TPC-W whch exhbts a very hgh varablty n servce tmes, leadng to hgher varablty n expermental results. provded that MP L new does not volate the lower bound on the MPL. The above algorthm nvolves multplcatve adjustments. In practce, combnng ths wth small constant adjustments, when close to the target, works well. We fnd that for obtanng a good estmate of the mean executon tme n step 1) t suffces for the Performance Montor to sample a few hundred transactons n the case of W IO and a few thousand transactons n the case of W CP U (due to the nherently hgher varablty of workload W CP U ). In our experments the above tunng algorthm fnds the optmal MPL wth at most 1 teratons. Movng from mean response tme targets to percentle targets the above approach does not work anymore, snce Lttle s law does not apply to percentles of response tme. We solve ths problem by approxmatng the DBMS by a Processor-Sharng (PS) server. Whle ths approxmaton mght be too crude for exact predctons, the hope s that t s stll useful for provdng ntuton on the relatonshp DBMS 9% between the MPL and T. From queueng theory 2 t follows that n a PS server the percentles of response tme scale lnearly wth the number of transactons at the server. Ths result enables us to use the same method we used for mean response tme targets for percentle targets as well. The effectveness of the above approach for percentle targets hnges on the assumpton that our DBMS behaves smlarly to a PS server wth respect to the lnear scalng of percentle response tmes as a functon of MPL. Our expermental results n Fgure 5 show that ths assumpton s n fact vald for our workloads runnng under IBM DB2. Although not shown, we fnd that the same result holds for PostgreSQL. We add a dsclamer that the above algorthm works best when all classes have a smlar mx of transactons. Ths was not a necessary condton on all of our 2 Ths s based on the queueng-theoretc result that under M/G/1/PS, for a gven transacton, ts expected response tme s proportonal to both the number of transactons at the server and ts servce demand.

11 Mean response tme (sec) Best effort Target Resp 1.3 sec Target Resp.6 sec Mean response tme (sec) Target Resp 2.4 sec Target Resp 1.3 sec Target Resp.6 sec Load (Number of clents) Load (Number of clents) Fgure 6. The graphs show the response tmes for three classes and workload W IO wth ncreasng load (.e. ncreasng number of clents). prevous QoS algorthms, and s needed only here. If that condton s not met, the control loop that adjusts the MPL needs to be more complex Adaptng to load fluctuatons The results n Sectons 4 and 5 have assumed a system n steady state wth a stable arrval process. Durng the day, however, the system load may fluctuate. Assumng that the system s never n overload, there should be no need to drop transactons, but the load mght rse too hgh for the current set of targets to be feasble,.e. T > τ overall The EQMS offers two approaches for handlng ths case. The frst approach assumes t s more mportant for some classes to stay wthn ther target than for others. The DBA ndcates ths by specfyng a prorty for each class, n addton to specfyng per-class response tme targets. These prortes only become effectve when load condtons make the per-class targets no longer feasble (.e. T > τ overall ). We detect ths stuaton by checkng whether there are any late transactons n the external queue,.e., transactons that have already mssed ther dspatch target. If t curr s the current tme and t d s the transacton s dspatch target tme, then late transactons as those for whom t curr > t d Whenever we have to choose a new transacton for executon n the DBMS from the external queue, we frst check whether there are transactons that are late. If there are, we pck the transacton wth the hghest prorty among the late transactons. If there are no late transactons n the queue, we schedule as usual n the order of the dspatch targets. An alternatve approach s to carry the burden of the excess load equally among all the classes. The burden wll be proportonately shared between the classes n the followng way: For each transacton we compute ts lateness, l, as l = t curr t d When schedulng late transactons, we consder the relatve lateness, l rel, of the transacton normalzed by ts target response tme τ. Relatve lateness s defned as l rel = l/τ = (t curr t d )/τ Whenever we choose a transacton for executon n the DBMS from the external queue, we frst check whether there are transactons that are late. If there are late transactons n the queue, we pck the transacton wth the largest relatve lateness, l rel, for executon. If there are no late transactons n the queue, we schedule as usual solely based on the dspatch targets. We mplement both approaches for dealng wth fluctuatng load descrbed above and expermentally evaluate them on the workload correspondng to the second row n Table 3. In the experments we vary the load by ncreasng the number of clents from 1 to 3. The results are shown n Fgure 6. In the frst approach, the targets for the frst two classes are mantaned despte the load ncrease, whle only the thrd (best effort) class suffers. These results for IBM DB2 are shown n Fgure 6 (left). Observe that the frst two classes have nearly constant mean response tmes across all loads. In the second approach, we share the burden across all the classes. The results are shown n Fgure 6 (rght), for the case where the thrd class has target response tme 2.4 seconds. In ths fgure, the mean response tmes of all classes ncrease by the same factor as load ncreases. 7. Concluson Many tme-senstve applcatons rely on a DBMS backend. From the perspectve of these applcatons, the DBMS s a mysterous black box: Transactons are sent nto the DBMS and may take ether a very short tme (msec) or a

12 very long tme (tens of secs), dependng largely on the other transactons concurrently n the DBMS. The applcaton has no control over whch transactons wll take long and whch wll take a short tme. Ths paper provdes mechansms and algorthms for controllng the tme dfferent transactons spend at a database backend. We provde methods for creatng dfferent QoS classes and for meetng specfed per-class QoS targets. QoS targets can be mean response tme targets, percentle targets, varablty targets, or a combnatons of targets. Our soluton s an external schedulng mechansm whch lmts the number of concurrent transactons (MPL) wthn the DBMS, holdng all remanng transactons n an external queue. We fnd that for our workloads there s a good range of MPL values whch allows us to acheve class targets wthout hurtng overall system performance wth respect to throughput and overall mean response tme. The algorthms needed to acheve the QoS targets are non-obvous and rely on queueng theory results and analyses. We demonstrate the effectveness of the algorthms on several benchmark based workloads, ncludng CPU bound, I/O bound and lock bound workloads, n stuatons wth multple classes and multple targets per class. However, t s desrable to experment wth other real workloads to further valdate the algorthms. Our external schedulng approach s extremely portable, not just to dfferent DBMS, but also to other types of backend servers. The queung theoretc arguments n ths paper do not depend on the server beng a DBMS and apply to general systems as well. References [1] R. K. Abbott and H. Garca-Molna. Schedulng real-tme transactons. In Proceedngs of SIGMOD, pages 71 81, [2] R. K. Abbott and H. Garca-Molna. Schedulng real-tme transactons wth dsk resdent data. In Proceedngs of Very Large Database Conference, pages , [3] R. K. Abbott and H. Garca-Molna. Schedulng I/O requests wth deadlnes: A performance evaluaton. In IEEE Real- Tme Systems Symposum, pages , 199. [4] R. K. Abbott and H. Garca-Molna. Schedulng real-tme transactons: A performance evaluaton. Transactons on Database Systems, 17(3):513 56, [5] A. Bouch and M. Sasse. It an t what you charge t s the way that you do t: A user perspectve of network QoS and prcng. In Proceedngs of IM 99, [6] K. P. Brown, M. J. Carey, and M. Lvny. Managng memory to meet multclass workload response tme goals. In Proceedngs of Very Large Database Conference, pages , [7] K. P. Brown, M. J. Carey, and M. Lvny. Goal-orented buffer management revsted. In Proceedngs of the 1994 ACM SIGMOD Conference on Management of Data, pages , [8] K. P. Brown, M. Mehta, M. J. Carey, and M. Lvny. Towards Automated Performance Tunng For Complex Workloads. In Proceedngs of the Twenteth Internatonal Conference on Very Large Databases, pages 72 84, Santago, Chle, [9] T. Can, M. Martn, T. Hel, E. Weglarz, and T. Bezenek. Java TPC-W mplementaton. pharm/tpcw.shtml, 2. [1] M. J. Carey, R. Jauhar, and M. Lvny. Prorty n DBMS resource schedulng. In Proceedngs of Very Large Database Conference, pages , [11] B. Dellart. How tolerable s delay? Consumers evaluaton of nternet web stes after watng. Journal of Interactve Marketng, 13:41 54, [12] J. Huang, J. Stankovc, K. Ramamrtham, and D. F. Towsley. On usng prorty nhertance n real-tme databases. In IEEE Real-Tme Systems Symposum, pages , [13] W. Jn, J. S. Chase, and J. Kaur. Interposed proportonal sharng for a storage servce utlty. In Proceedngs of ACM SIGMETRICS 4, pages 37 48, 24. [14] K. D. Kang, S. H. Son, and J. A. Stankovc. Servce dfferentaton n real-tme man memory databases. In Ffth IEEE Internatonal Symposum on Object-Orented Real- Tme Dstrbuted Computng, [15] A. Krass, F. Schoen, G. Wekum, and U. Deppsch. Wth heart towards response tme guarantees for message-based e-servces. In VIII. Conference on Extendng Database Technology (EDBT 22), pages , 22. [16] I. T. Lab. IBM DB2 unversal database admnstraton gude verson 5. Document Number S1J-8157-, [17] J. Lttle. A proof of the theorem L = λw. Operatons Research, 9: , [18] D. T. McWherter, B. Schroeder, A. Alamak, and M. Harchol-Balter. Prorty mechansms for OLTP and transactonal web applcatons. In 2th IEEE Conference on Data Engneerng (ICDE 24), 24. [19] PostgreSQL. [2] A. Rhee, S. Chatterjee, and T. Lahr. The Oracle Database Resource Manager: Schedulng CPU resources at the applcaton. Hgh Performance Transacton Systems Workshop, 21. [21] B. Schroeder, M. Harchol-Balter, A. Iyengar, and E. Nahum. How to determne a good mult-programmng level for external schedulng. In submsson to ICDE 6 (Paper 597). [22] M. Snnwell and A. Koeng. Managng dstrbuted memory to meet multclass workload response tme goals. In 15th IEEE Conference on Data Engneerng (ICDE 99), [23] Transacton Processng Performance Councl. TPC benchmark C. Number Revson 5.1., December 22. [24] Transacton Processng Performance Councl. TPC benchmark W (web commerce). Number Revson 1.8, February 22. [25] M. Zhou and L. Zhou. How does watng duraton nformaton nflucence customers reactons to watng for servces. Journal of Appled Socal Psychology, 26: , 1996.