ANALYTIC PERFORMANCE MODELS FOR SINGLE CLASS AND MULTIPLE CLASS MULTITHREADED SOFTWARE SERVERS

Transcription

1 ANALYTIC PERFORMANCE MODELS FOR SINGLE CLASS AND MULTIPLE CLASS MULTITHREADED SOFTWARE SERVERS Daniel A Menascé Mohamed N Bennani Dept of Compute Science Oacle, Inc Geoge Mason Univesity 1211 SW Fifth Avenue Faifax, VA 22030, USA Potland, OR menasce@csgmuedu mohamedbennani@oaclecom Abstact Moden compute systems ae based on a wide vaiety of softwae seves, such as web seves, application seves, database seves, and mail seves The typical softwae achitectue of such seves includes a set of pocesses o that seve submitted to the seve Requests that aive at the seve and find all busy, ae placed in a queue Theads that ae busy executing compete fo hadwae esouces (eg, pocessos and I/O devices) at the machine whee the softwae seve uns It is impotant to be able to model softwae seves in a way that takes into account both softwae contention waiting fo and hadwae contention waiting fo pocessos and I/O devices This pape pesents analytical models fo a wide ange of multitheaded softwae seve achitectues: a) single class (ie, all have simila demands) with unlimited thead queue size, b) single class with limited thead queue size, c) multiclass with unlimited thead queue size, and d) multiclass with limited queue size Numeical esults ae pesented to illustate the use of the models 1 Intoduction Moden compute systems ae based on a wide vaiety of softwae seves that seve fom client pocesses Some examples of softwae seves include web seves, application seves, database seves, and mail seves The typical softwae achitectue of such seves includes a set of pocesses o that seve submitted to the seve Requests that aive at the seve and find all busy, ae placed in a queue Theads that ae busy executing compete fo hadwae esouces (eg, pocessos and I/O devices) at the machine whee the softwae seve uns Theefoe, it is impotant to be able to model softwae seves in a way that takes into account both softwae contention waiting fo and hadwae contention waiting fo pocessos and I/O devices This pape pesents analytical models fo a wide ange of multitheaded softwae seve achitectues: a) single class (ie, all have simila demands) with unlimited thead queue size, b) single class with limited thead queue size, c) multiclass with unlimited thead queue size, and d) multiclass with limited queue size This wok was suppoted in pat by the National Geospatial- Intelligence Agency (NGA) contact NMA The models pesented hee build on well-known modeling techniques: bith-death Makov chains [1], Mean Value Analysis (MVA) [4] fo single class queuing netwok models, and appoximate MVA (amva) [5] fo multiple class queueing netwok models We also use the well-known pinciples of hieachical modeling and iteative models The est of this pape is oganized as follows Section two pesents some backgound and notation Section thee pesents a model fo a single class softwae seve with an unlimited thead queue size The next section modifies the model to account fo thead queues of limited size Section five deals with the issue of multiple classes of fo both unlimited and limited queue size Finally section six pesents some concluding emaks 2 Basic Concepts and Notation The models used in this pape daw on the notion of closed queuing netwok (QN) models used to epesent compute systems Such models compise a collection of K devices visited by that may belong to diffeent classes, o wokloads A class of is typically associated to a collection of that have simila demands on the vaious esouces The sevice demand, D i,,of of class at device i is the total sevice time spent by a equest of class ove all visits to device i duing the (c) 2006 Menasce and Bennani All Rights Reseved 1/7

2 execution of the equest The solution to such QNs can be obtained though Mean Value Analysis [4] o appoximate MVA (amva) fo the case of multiple classes [5] Fo moe details on queuing netwoks, the eade is efeed to [3] The notation given in Table 1 is used thoughout the pape 3 Single-class Multitheaded Seve with Unlimited Queue Figue 1 displays a situation in which fo sevice aive to a multitheaded softwae seve that has M The undelying hadwae platfom used by all has pocessing and I/O esouces Aiving join a queue of waiting fo an available thead Once a thead to execute, it stats to compete with othe fo pocessing and I/O esouces The bottom pat of the figue shows queues fo the CPU and fo I/O It is also assumed in this section that all ae homogeneous in tems of thei sevice demands This assumption will be elaxed late in the pape Thus, we ae dealing with a single-class modeling situation aiving queue of waiting fo 1 completing and completion pocess of The ate at which complete is detemined by the solution of a lowe level model, in the fom of a queuing netwok (QN) Let X(k), k =1,, M, be the ate at which a thead completes its execution when thee ae k in execution This ate can be obtained by solving the closed QN model fo the hadwae subsystem composed of the CPU and disk when thee ae k concuent in execution The solution of this queuing netwok can be obtained though Mean Value Analysis (MVA) [3, 4] A state k in the Makov chain of Fig 2 epesents the numbe of in the system (waiting fo a thead o being executed by a thead) This Makov chain has an infinite numbe of states since thee is no limit on the numbe of in the queue The aival ate in the Makov chain (tansition ate to a highe numbeed neighboing state) is the aival ate of, λ, and the completion ate (tansition ate to a lowe numbeed neighboing state) is the completion ate of Note that since thee can be at most M in execution, the depatue ate is constant and equal to X(M) fo any state k>m The solution of this Makov chain, ie, the set of pobabilities P k,k =0,, can be obtained using the Genealized Bith-Death theoem [1, 3]: k 1 λ i P k = P 0 (1) μ i+1 i=0 Fo the model consideed in this subsection, λ i = λ i and μ i = X(i) fo i =0,,M and μ i = X(M) fo i>m M 0 1 k M M+1 (1) (2) (k) (k+1) (M) (M) (M) CPU Figue 2: Makov Chain fo the unlimited thead queue case The solution to the Makov chain of Fig 2 is given by Disk compute system shaed by all Figue 1: Multitheaded seve with unlimited queue In ode to model this type of softwae seve, we use the well-known pinciple of hieachical modeling At the softwae level, we use a Makov chain to model the aival { P0 λ P k = k /β(k) fo k =1,,M P 0 ρ k X(M) M /β(m) fo k>m whee: β(k) =X(1) X(2) X(k), ρ = λ/x(m), and P 0 =[1+ M λk β(k) + X(M)M β(m) ρm+1 1 ρ ] 1 (2) Note that the solution given above only exists when ρ<1 The aveage esponse time, R 0, can be easily computed fom the state pobabilities and fom Little s Law [1] as (c) 2006 Menasce and Bennani All Rights Reseved 2/7

3 K Numbe of devices (eg, pocessos, disks) in the compute system R Numbe of classes (wokloads) M total numbe of in the system N system size (maximum queue length plus numbe of in system) D i, sevice demand of class at device i U i, utilization of device i by class U i utilization of device i (U i = R =1 U i,) n i, aveage numbe of class queued at device i o using the device aveage numbe of class in execution ( = K i=1 n i,) aveage numbe of class in the system (waiting fo a thead o using a thead) Table 1: Notation R 0 = /λ, whee, the aveage numbe of in the system (waiting fo a thead o using a thead), is given by [ M ] = P 0 k λk β(k) + ρ (X(M)ρ)M (1 + M(1 ρ)) β(m) (1 ρ) 2 (3) The aveage numbe of in execution,,is given by = M 1 M 1 k P k + M [1 k=0 P k ] (4) Figue 3 shows the vaiation of the aveage esponse time R 0 as a function of the aveage aival ate λ The paametes used fo this example ae: Numbe of (M): 20 Sevice demand at the CPU (D CPU ): 0020 sec Sevice demand at the disk (D I/O ): 0015 sec Fo the paametes given above, X(20) = 4996 /sec, close to the maximum bound on thoughput which is 1/D CPU = 1/002 = 50 /sec Since ρ = λ/x(20) < 1, λ<4996 As the figue shows, as λ appoaches its maximum possible value, the esponse time goes to infinity vey fast The esponse time stays below 05 sec fo λ<49 /sec 4 Single-class Multitheaded Seve with Limited Queue Figue 4 displays a multitheaded seve with a limited queue fo stoing incoming The numbe of is M as in the unlimited queue size case but thee is a limit N on the total numbe of in the system (waiting fo a thead o being executed by a thead) The solution fo this case can be obtained following the same appoach as in section 3 The Makov chain model in this case, shown in Fig 5, has N +1 states A state k, (k =0,, N), epesents the numbe of in the Avg Response Time (sec) Aival ate (eq/sec) Figue 3: Aveage esponse time (sec) vs aival ate (eq/sec) fo the unlimited queue case with 20 system (waiting fo a thead o using a thead) The aival ate in the diagam is the aival ate of, λ, and the completion ate is the completion ate of Note that since thee can be at most M in execution, the depatue ate is constant and equal to X(M) fo any state k>m as in the pevious case The solution of this Makov Chain, ie, the values of the pobabilities P k,fok =(0,, N), of finding k in the system, can be obtained using the methods in [1, 3] The solution is given below { P0 λ P k = k /β(k) fo k =1,,M P 0 ρ k X(M) M /β(m) whee: β(k) =X(1) X(2) X(k), ρ = λ/x(m), and P 0 =[1+ M λk fo k = M +1,,N (5) β(k) + ρ λm (1 ρ N M ) β(m) (1 ρ) ] 1 (c) 2006 Menasce and Bennani All Rights Reseved 3/7

4 aiving queue of waiting fo 1 used in the unlimited queue case Thee is one additional paamete in this case, which is the system size N The figue shows two cuves, one fo N =25and anothe fo N =35 Max? Yes ejected No M completing Avg esponse time (sec) CPU Aival ate (eq/sec) n = 25 n = 35 Figue 4: queue Disk compute system shaed by all Multitheaded seve with limited thead The metics of inteest, namely the pobability of ejection P ej, the aveage thoughput X and the aveage esponse time R, can be easily computed fom the state pobabilities and fom Little s Law [1] as follows P ej = P n X = λ (1 P ej )= N X(k) P k R 0 = N k P k/x Figue 6 shows the vaiation of the aveage esponse time as a function of the aveage aival ate fo the limited queue case The paametes fo this example ae the same 0 1 k M N-1 N (1) (2) (k) (M) (M) (M) Figue 5: Makov Chain fo the limited thead queue case Figue 6: Aveage esponse time (sec) vs aival ate (eq/sec) fo the limited queue case with 20 and two values of the maximum numbe of in the system (N =25andN = 35) It is inteesting to obseve the shape of this cuve At low loads, the pobability that ae lost, P ej,is vey low Fo example, fo N = 25 and λ = 45 /sec, P ej =1% Howeve, as the load inceases, moe ae ejected and the effective aival ate to the system (equal to the system thoughput) inceases at asloweatewithλ Fo example, fo N =25and λ =70 /sec, P ej =287% Thus, only 499 /sec ae actually being accepted by the system fo execution This explains why the esponse time does not gow to infinity In fact, fo a vey lage value of λ, accepted will find (N M) 1 in the queue fo and all will be busy seving So, the aveage esponse time will become constant and equal to the sum of a constant aveage waiting time and a constant aveage execution time since all M ae busy and competing fo the hadwae esouces 5 Multi-class Multitheaded Seve We now elax the assumption that all have the same sevice demands Conside R diffeent classes of Class ( =1,,R) has an aival ate equal to λ /sec Thee ae K devices (eg, pocessos and I/O devices) and the sevice demand of a equest of class at esouce i is given by D i, Ifweweetousethesame two-level modeling appoach descibed in Sections 3 and 4 we would have a multidimensional Makov chain in which (c) 2006 Menasce and Bennani All Rights Reseved 4/7

5 the state would have to indicate the numbe of of each class in execution as well as the numbe of of each class in the queue This Makov chain would have a vey complex and lage state space Instead of consideing such a lage state space, we use an appoximation that was used by Lazowska et al [2] to model shaed domains in multipogamming systems The basic appoach woks basically as follows: 1 Initialization: Estimate the aveage numbe of in execution,, fo class and the aveage numbe of class in the system (executing o waiting fo a thead),, fo all classes =1,,R 2 Fo each class compute the aveage numbe, δ exec,of of all othe classes that ae in execution and the aveage numbe, δ syst, of of all othe classes that ae in the system (in execution o waiting fo a thead) 3 Solve a multiclass QN model using appoximate Mean Value Analysis (amva) [3] whee the population vecto is ( 1, 2,,n,, fo n =1to M δ exec In othe wods, since thee ae δ exec in execution fom classes othe than class, the numbe of allocated to class is the diffeence between the numbe of M and the numbe of allocated to the othe classes The solution of this QN model gives the class thoughput X ( 1, 2,,n,, fo n =1to M δ exec 4 A unidimensional Makov chain such as the ones discussed in sections 3 o 4 can now be built fo each class using λ as the aival ate and the thoughputs X ( 1, 2,,n,, as the completion ate The solution of this Makov chain povides new values of the aveage numbe of in execution 5 Repeat steps 2 though 4 until successive values of ae sufficiently close fo all classes 6 Compute the pefomance metics We now povide a detailed desciption of the algoithm to model multiclass multitheaded seves The pseudocode fo the initialization phase (step 1 above) is given in Fig 7 The pupose of this phase is to obtain an estimate fo and fo all classes This is done by fist computing the utilization of all devices using the Sevice Demand Law [3] If all utilizations ae below 100%, the fomula fo multiclass open QNs is used to estimate Othewise, we estimate using an appotionment facto based on the elative aival ate of class In both cases, we assume that the queue fo is initially empty, ie, = Note that the initial values of and will be modified though the iteation pocess in the emaining steps of the algoithm Figs 8 and 9 povide a detailed desciption of the iteation phase of the algoithm fo the unlimited and limited thead queue cases, espectively Note that the computation of the pobability distibutions fo P k follows in each case the fomulas used in Sections 3 and 4 fo evey device i do U i ( λ) R =1 U i,( λ)= R =1 λ D i, ; if (U i ( λ) < 10) fo all i then fo evey class do /* use open QN fomula to estimate */ fo evey device i do n i, U i, /(1 U i ); K i=1 n i,; /* adjust n so that it does not exceed M, the numbe of class estimated to be in execution */ min{,m } whee M = M / R s=1 nexec s ; /* assume no queuing fo at initialization */ else /* altenative initialization: use aival ate atios to estimate n */ fo evey class do M λ / s=r s=1 λ s ; /* assume no queuing fo at initialization */ Figue 7: Initialization Phase of the Pefomance Model fo a Multiclass Multitheaded Seve Table 2 shows the esults afte thee iteations fo a two-class multitheaded case with unlimited queue The following paametes wee used: M =10, λ 1 =25 /sec, λ 2 =20/sec, D CPU,1 =0014 sec, D I/O,1 =0020 sec, D CPU,2 =0018 sec, and D I/O,2 = 0012 sec Iteation 0 shows the esults of the initialization step Columns 7 and 13 show the absolute pecent elative eo between two consecutive values of Note that in iteation 3, this eo is aleady at 03% Moe iteations would yield even smalle eos The esponse time fo classes 1 and 2 at iteation 3 is seconds and (c) 2006 Menasce and Bennani All Rights Reseved 5/7

6 Iteation step: fo evey class =1, 2,, R do δ exec = nexec /*adjustethevalueofm */ M = M δ exec fo n =1, 2,, M do Call amva to compute X ( 1, 2,,n,, if M is not an intege then Call amva to compute X ( 1, 2,,M,, define a single class genealized bith-death model with λ = λ and death ates: μ (k) =X ( 1, 2,,k,,,fo1 k M,and μ (k) =X ( 1, 2,,M,, fo k> M compute the pobability distibution, P k, accoding to Section 3 P k = P 0 λk β(k),fo1 k M,and P k = P 0 ρ k μ (M ) M /β( M ),fok> M whee ρ = λ /μ (M ), β(k) =μ (1) μ (k), and [ P 0 = 1+ M λk /β(k)+ μ( M ) M ρ M +1 β( M ) (1 ρ) compute the pefomance metics as: aveage numbe [ of in the system: M = P 0 kλk /β(k)+ ρ [μ( M )ρ] M aveage thoughput: X = λ aveage esponse time: R 0, = /λ aveage numbe of in execution: = M 1 k P k + M [1 M 1 k=0 P k ] ] 1 (1+(M )(1 ρ)) β( M )(1 ρ) ] 2 Convegence test step: Repeat the iteation step until successive values of Compute final metics Retun the pefomance metics as computed in the last iteation ae sufficiently close fo all classes Figue 8: Iteative Phase of the Pefomance Model fo a Multiclass Multitheaded Seve with Unlimited Queue seconds, espectively 6 Concluding Remaks This pape showed how well-known analytic pefomance models can be used to assess the pefomance of multitheaded softwae seves The models use a combination of bith-death Makov chains, Mean Value Analysis, and hieachical modeling The softwae seve achitectues consideed in this pape cove a wide ange of situations found in moden seve-oiented systems The modeling appoach pesented hee takes into account both softwae and hadwae contention in single and multiclass situations Refeences [1] L Kleinock, Queueing Systems Volume I: Theoy, John Wiley & Sons, New Yok, NY, 1975 [2] E Lazowska, J Zahohan, GS Gaham, and KC Sevcik, Quantitative System Pefomance: compute system analysis using queuing netwok models, Pentice Hall, Uppe Saddle Rive, 1984 [3] DA Menascé, VAF Almeida, and LW Dowdy, Pefomance by Design: Compute Capacity Planning by Example, Pentice Hall, Uppe Saddle Rive, 2004 [4] M Reise and S Lavenbeg, Mean-value analysis of closed multi-chain queuing netwoks, J ACM,vol 27, no 2, 1980, pp [5] P Schweitze, Appoximate analysis of multiclass closed netwok of queues, Poc Intl Conf Stochastic Contol and Optimization, Amstedam, 1979 (c) 2006 Menasce and Bennani All Rights Reseved 6/7

7 Iteation 1 1 δ1 exec M1 R 0,1 % elative 2 2 δ2 exec M2 R 0,2 % elative eo 1 eo Table 2: Results fo multiclass unlimited thead queue case Iteation step: fo evey class =1, 2,, R do δ exec = nexec = nsyst /* Adjust the values of M and N */ δ syst M = M δ exec N = N δ syst fo n =1, 2,, M do Call amva to compute X ( 1, 2,,n,, if M is not an intege then Call amva to compute X ( 1, 2,,M,, define a single class genealized bith-death model with λ = λ and death ates: μ (k) =X ( 1, 2,,k,,fo1 k M,and μ (k) =X ( 1, 2,,M, fo M <k N compute the pobability distibution, P k, accoding to Section 4, as P k = P 0 λ k /β(k),fo1 k M,and P k = P 0 ρ k μ (M ) M /β( M ),fo M <k N whee ρ = λ /μ (M ), β(k) =μ (1) μ (k), and P 0 = [ 1+ M λk /β(k)+ ρ λ M Compute the pefomance metics as: aveage numbe of in the system: = N k P k aveage thoughput: X = N P k μ (k) pobability of ejection: P,ej = P N aveage esponse time: R 0, = / X aveage numbe of in execution: ] (1 ρ N M 1 ) β( M ) (1 ρ) = M 1 k P k + M [1 M 1 k=0 P k ] Convegence test step: Repeat the iteation step until successive values of Compute final metics: Retun the pefomance metics as computed in the last iteation ae sufficiently close fo all classes Figue 9: Iteative Phase of the Pefomance Model fo a Multiclass Multitheaded Seve with Limited Queue (c) 2006 Menasce and Bennani All Rights Reseved 7/7