Quantfyng Performance Models Prof. Danel A. Menascé Department of Computer Scence George Mason Unversty www.cs.gmu.edu/faculty/menasce.html 1 Copyrght Notce Most of the fgures n ths set of sldes come from the book Performance by Desgn: computer capacty plannng by example, by Menascé, Almeda, and Dowdy, Prentce Hall, 2004. It s strctly forbdden to copy, post on a Web ste, or dstrbute electroncally, n part or entrely, any of the sldes n ths fle. 2 1
A Resource and ts Queue W S Customers LINE Resource S : servce tme W : watng tme resource: CPU, dsk, network, etc. 3 Computng Dsk Servce Tmes I/O request I/O result fle system cache devce drver devce drver queue dsk cache bus adaptor dsk controller queue dsk I/O bus dsk controller 4 2
Computng Dsk Servce Tmes s d = ContrTme + P ( Seek + Latency + TransferT) mss TransferT = BlockSze TransferRate 5 Computng Dsk Servce Tmes Types of Workloads Random Workload: 10, 201, 15, 1023, 45, 39, 782 Sequental Workload: 4, 102, 103, 104, 105, 106, 25, 88, 32, 33, 34, 35, 36, 37, 38, 29, 15 run length= 5 run length= 7 6 3
Computng Dsk Servce Tmes Random Workload: P mss = 1 RunLength = 1 SeekTme = S rand Latency = 1 / 2 Re volutontme 7 Computng Dsk Servce Tmes Sequental Workload: P mss = 1/ RunLength SeekTme = S rand / RunLength 1/ 2 + ( RunLength 1)[(1+ U Latency = RunLength RevolutonTme d )/ 2] U = λ S d d D 8 4
Dsk Arrays PA A1 A2 A3 A4 B1 PB B2 B3 B4 C1 C2 PC C3 C4 D1 D2 D3 PD D4 E1 E2 E3 E4 PE 9 Dsk Arrays - Wrte One Strpe Unt Compute PA A2 2 reads 2 wrtes PA A1 A2 A3 A4 B1 PB B2 B3 B4 C1 C2 PC C3 C4 D1 D2 D3 PD D4 E1 E2 E3 E4 PE 10 5
Dsk Arrays - Wrte Two Strpe Unts 2 reads 3 wrtes Compute PA A3 A4 PA A1 A2 A3 A4 B1 PB B2 B3 B4 C1 C2 PC C3 C4 D1 D2 D3 PD D4 E1 E2 E3 E4 PE 11 Dsk Arrays - Wrte Three Strpe Unts 1 read 4 wrtes Compute PA A3 A4 A2 PA A1 A2 A3 A4 B1 PB B2 B3 B4 C1 C2 PC C3 C4 D1 D2 D3 PD D4 E1 E2 E3 E4 PE 12 6
Dsk Arrays - Wrte Four Strpe Unts 0 reads 5 wrtes Compute PA A3 A4 A2 A1 PA A1 A2 A3 A4 B1 PB B2 B3 B4 C1 C2 PC C3 C4 D1 D2 D3 PD D4 E1 E2 E3 E4 PE 13 Network Servce Tmes clent TCP IP Network Layer Ethernet request reply FDDI Rng server TCP IP Network Layer Token Rng 14 7
Network Servce Tmes 18 B (wth traler) 20 B 20 B Frame Header IP Header TCP Header Clent Request Frame Traler MTU=1500 bytes Clent Message Sze = 2500 bytes No Datagrams = 2500 / (1500 20 20) = 2 Τοtal Overhead = 2 * (18+20+20)=116 bytes Message Servce Tme = [2500+116]*8/10,000,000=0.02098 sec 15 Web Page Download Tmes Depend on type of HTTP protocol used page parameters network parameters TCP parameters 16 8
HTTP 1.0 nteracton 0 RTT TCP conn. 1 RTT clent sends HTTP req. 2 RTT clent parses HTML doc. 3 RTT clent sends req. for mage syn syn ack dat ack dat syn syn dat Server tme Server tme 4 RTT mage begns to arrve dat 17 HTTP 1.1 nteracton 0 RTT syn TCP conn. 1 RTT clent sends HTTP req 2 RTT dat syn ack ack dat Server tme clent parses HTML doc. clent sends req. for mage ack dat Server tme 3 RTT mage begns to arrve dat ack 18 9
HTTP 1.0 and 1.1 0 RTT syn TCP conn. 1 RTT syn ack clent sends dat HTTP req. ack 2 RTT dat clent parses HTML doc. syn syn 3 RTT clent sends req. for mage dat 4 RTT dat mage begns to arrve Server tme Server tme 0 RTT TCP conn. 1 RTT clent sends HTTP req 2 RTT clent parses HTML doc. clent sends req. for mage 3 RTT mage begns to arrve syn syn ack dat ack dat ack dat ack dat Server tme Server tme HTTP 1.0 HTTP 1.1 19 Lower Bound on Page Download Tme PageSze: sze, n bytes, of all objects of a page, ncludng the HTTP header (290 bytes). B: effectve network bandwdth (n bps) RTT: network round trp tme (n sec) NObj: Number of embedded objects n a page. 20 10
Lower Bound on Page Download Tme Non-persstent connecton PageSze PDT NP > ( NObj + 1) (2 RTT ) + B Persstent connecton PageSze PDT P > RTT + ( NObj + 1) RTT + B 21 Page Download Tme Example: Smple Page HTML page = 15,650 bytes HTTP header = 290 bytes 10 mages of 4,200 bytes each RTT = 0.05 sec B = 125,000 bytes/sec 15,650+ 11 290 + 10 4,200 PDT NP > 11 2 0.05+ = 1.59 125,000 15,650+ 11 290 + 10 4,200 PDT P > 0.05+ 11 0.05+ = 1.09 125,000 sec sec 22 11
Page Download Tme Example: Elaborate Page HTML page = 15,650 bytes HTTP header = 290 bytes 20 mages of 20,000 bytes each RTT = 0.05 sec B = 125,000 bytes/sec 15,650 + 21 290 + 20 20,000 PDT NP > 21 2 0.05+ = 5.47 125,000 15,650+ 21 290 + 20 20,000 PDT P > 0.05 + 21 0.05+ = 4.47 125,000 sec sec 23 TCP Throughput Depends on: Packet Loss Rato Round Trp Tme Wm: Maxmum Recever Wndow Sze (advertsed by the recever at connecton establshment tme) TCP tmeout Network Bandwth Maxmum Segment Sze 24 12
TCP: Wndow sze vs tme (n RTTs) Wndow Sze (segments) Tme (n RTTs) 25 TCP Throughput RTT=0.04 sec; Bandwdth = 12,500 KB/sec Tmeout = 2 sec; Maxmum Segment Sze = 1,460 bytes 26 13
Servce Demand (D) Servce demand = Total average servce tme over all vsts Arrvng requests LINE S 1 S 2... S k Resource Completng requests S: Servce tme at vst D: Servce demand = S1 + S2 + + Sk 27 Important take home! Servce demands are mportant parameters for performance models Servce demands are easy to measure. Servce tmes are much harder to obtan! Servce demands are assocated wth a type of request and a resource. Servce demands are measured n tme unts (e.g., sec, msec) Servce demands are load ndependent! More on ths to come 28 14
Servce Demand Example Requests to a Web ste use two dsks. The servce tmes at each of the dsks for each I/O carred out by a sngle request are Servce Tme (msec) I/O Dsk 1 Dsk 2 1 12 12 2 20 15 3 15 14 4 18-65 41 Servce demand at dsk 1 Servce demand at dsk 2 29 Queung Tme w1 s1 w2 s2 w3 s3 CPU w4 s4 w5 s5 Dsk Queung tme at the CPU = w1 + w2 + w3 Queung tme at the dsk = w4 + w5 Watng tme Servce tme 30 15
Servce Demand w1 s1 w2 s2 w3 s3 CPU w4 s4 w5 s5 Dsk Servce demand at the CPU = s1 + s2 + s3 Servce demand at the dsk = s4 + s5 Watng tme Servce tme 31 CPU Resdence Tme w1 s1 w2 s2 w3 s3 Dsk w4 s4 w5 s5 Resdence tme at the CPU = w1 + s1 + w2 + s2 + w3 + s3 Resdence tme at the dsk = w4 + s4 + w5 + s5 Watng tme Servce tme 32 16
Response Tme w1 s1 w2 s2 w3 s3 CPU w4 s4 w5 s5 Dsk Response tme = Resdence tme at the CPU + Resdence tme at the dsk Watng tme Servce tme 33 Practce Questons What unts are used to measure servce demands? Is the servce demand a functon of the workload ntensty? What s the relatonshp between servce tme and servce demand? What s the relatonshp between response tme, servce tme, and watng tme? What s the relatonshp between resdence tme and response tme? What s the relatonshp between response tme and resdence tme? 34 17
Computer Systems Have Many Resources! S dsk1 V dsk1 λ requests/sec X 0 Dsk 1 S cpu V cpu CPU Computer System S dsk2 V dsk2 Dsk 2 35 Some Notaton V : average number of vsts to queue by a request (e.g., avg. no. of I/Os to a dsk) S : average servce tme of a request at queue per vst to the resource; (e.g., avg. dsk servce tme) λ : average arrval rate of requests to queue (e.g., number of I/O requests per second arrvng at a dsk). D : servce demand of a request at queue, (e.g., avg. total I/O tme of a request at a gven dsk) 36 18
Notaton (cont d) N : average number of requests at queue, watng or recevng servce from the resource (e.g., avg. no. of I/O requests usng or n the watng queue of a gve dsk) X : average throughput of queue,.e. average number of requests that complete from queue per unt of tme (e.g., avg. no. completed I/O requests/sec at a gven dsk) X o: average system throughput, defned as the number of requests that complete per unt of tme. (e.g., avg. no. of completed HTTP requests/sec) 37 Basc Performance Results Utlzaton Law The utlzaton (U ) of resource s the fracton of tme that the resource s busy. U = X * S = λ * S 38 19
Utlzaton Law: example 1 The bandwdth of a communcaton lnk s 56,000 bps and t s used to transmt 1500-byte packets that flow through the lnk at a rate of 3 packets/sec. What s the utlzaton of the lnk? 39 Utlzaton Law: example 1 The bandwdth of a communcaton lnk s 56,000 bps and t s used to transmt 1500-byte packets that flow through the lnk at a rate of 3 packets/sec. What s the utlzaton of the lnk? Avg Packet Servce (transmsson) Tme = (1500 x 8) / 56000 = 0.214 sec/packet Lnk Throughput = 3 packets/sec Lnk Utlzaton = 0.214 sec/packet x 3 packets/sec = 0.642 = 64.2% 40 20
Utlzaton Law: example 2 A computer system has one CPU and 3 dsks and supports a DB server. All DB transactons have smlar resource demands and the server s under a constant load. Measurements taken durng one hour show that 13,680 transactons were executed. The number of reads and wrtes and the dsk utlzatons are shown n the table. What s the average servce tme per request on each dsk? What s the DB server s throughput? Dsk 1 2 3 Reads/sec 24 28 40 Wrtes/sec 8 8 10 I/Os/sec 32 36 50 Utl. 0.30 0.41 0.54 41 Utlzaton Law: example 2 A computer system has one CPU and 3 dsks and supports a DB server. All DB transactons have smlar resource demands and the server s under a constant load. Measurements taken durng one hour show that 13,680 transactons were executed. The number of reads and wrtes and the dsk utlzatons are shown n the table. What s the average servce tme per request on each dsk? What s the DB server s throughput? Dsk 1 2 3 Reads/sec 24 28 40 Wrtes/sec 8 8 10 I/Os/sec 32 36 50 Utl. 0.30 0.41 0.54 S = U / X S1 = 0.3/32 = 0.0094 sec S2 = 0.41/36 = 0.0114 sec S3 = 0.54 / 50 = 0.0108 sec X0 = 13680 /3600 = 3.8 tps 42 21
Utlzaton Law: example 3 A network segment transmts 1,000 packets/sec. Each packet has an average transmsson tme equal to 0.15 msec. What s the utlzaton of the LAN segment? 43 Utlzaton Law: example 3 A network segment transmts 1,000 packets/sec. Each packet has an average transmsson tme equal to 0.15 msec. What s the utlzaton of the LAN segment? U LAN = X LAN * S LAN = 1,000 * 0.00015 = 0.15 = 15% 44 22
Forced Flow Law John Mary dsk... Each transacton does 3 I/Os on average and Mary measures a throughput equal to 12 tps. How many I/Os per second are seen by John? 45 Forced Flow Law By defnton of the average number of vsts V, each completng request has to pass V tmes, on the average, by queue. So, f X o requests complete per unt of tme, V *X o requests wll vst queue. X = V * X o 46 23
Forced Flow Law: example 1 Database transactons perform an average of 4.5 I/O operatons on the database server. Durng a one-hour montorng perod, 7,200 transactons were executed. What s the average throughput of the dsk? If each I/O takes 20 msec on the average, what s the dsk utlzaton? 47 Forced Flow Law: example 1 Database transactons perform an average of 4.5 I/O operatons on the database server. Durng a one-hour montorng perod, 7,200 transactons were executed. What s the average throughput of the dsk? If each I/O takes 20 msec on the average, what s the dsk utlzaton? X server = 7,200 / 3,600 = 2 tps X dsk = V dsk * X server = 4.5 * 2 = 9 tps U dsk = X dsk * S dsk = 9 * 0.02 = 0.18 = 18% 48 24
Forced Flow Law: example 2 X0 = 13680 /3600 = 3.8 tps Dsk Reads/sec Wrtes/sec I/Os/sec Utl. What s the average number of I/Os made by a transacton I/Os on each dsk? 1 2 24 28 8 8 32 36 0.30 0.41 V = X/X0 V1 = 32/3.8 = 8.4 I/Os V2 = 36/3.8 = 9.5 I/Os V3 = 50/3.8 = 13.2 I/Os 3 40 10 50 0.54 49 Servce Demand Law The servce demand D s gven by: D = V * S = (X /X o )(U /X ) = U / X o 50 25
Measurng Servce Demands The servce demand D s related to the system throughput and utlzaton by: D = U / X o where U s the utlzaton of resource and X o the system throughput. Easy to get! 51 U X D cpu 0 cpu Example of Servce Demand Law: vmstat n sy cs us sy dle = 119 65 24 1 0 99 296 2491 289 13 6 81 260 5586 213 44 7 49 326 2822 474 21 7 72 352 1913 271 13 4 83 304 2058 280 17 5 78 275 3072 506 21 7 72 322 3340 417 18 8 74 301 2000 201 9 3 87 261 1952 282 10 4 86 251 1870 220 9 4 87 412 4646 763 33 12 54 76.83 = 1 0.7683 = 0.232 = 23.2% 20 / 60 U = X cpu 0 = 0.333 requests/sec = 0.232 / 0.333 = 0.695sec Interval: 12*5sec= 60 sec Number of Requests: 20 52 26
Servce Demand Law: example A Web server runnng on top of a Unx system was montored for 10 mnutes. It was observed that the CPU was 90% busy durng the montorng perod. The number of HTTP requests counted n the log was 30,000. What s the CPU servce demand of an HTTP request? 53 Servce Demand Law: example A Web server runnng on top of a Unx system was montored for 10 mnutes. It was observed that the CPU was 90% busy durng the montorng perod. The number of HTTP requests counted n the log was 30,000. What s the CPU servce demand of an HTTP request? U cpu = 90% X server = 30,000 / (10*60) = 50 requests/sec D cpu = V cpu * S cpu = U cpu / X server = 0.90 / 50 = 0.018 sec 54 27
Servce Demand law: example 3 X0 = 13680 /3600 = 3.8 tps Ucpu = 35% Dsk Reads/sec Wrtes/sec I/Os/sec Utl. What are the servce demands at the CPU and dsks? 1 2 24 28 8 8 32 36 0.30 0.41 D = U / X0 Dcpu = 0.35 / 3.8 = 0.092 sec Ddsk1 = 0.3 / 3.8 = 0.079 Ddsk2 = 0.41 / 3.8 = 0.108 Ddsk3 = 0.54 / 3.8 = 0.142 3 40 10 50 0.54 55 Lttle s Law N X The average number of customers n a black box s equal to the average tme each customer spends n the box tmes the throughput of the box. R N = R * X 56 28
Lttle s Law: example 1 An NFS server was montored durng 30 mn and the number of I/O operatons performed durng ths perod was found to be 32,400. The average number of actve requests (N req ) was 9. What was the average response tme per NFS request at the server? 57 Lttle s Law: example 1 An NFS server was montored durng 30 mn and the number of I/O operatons performed durng ths perod was found to be 32,400. The average number of actve requests (N req ) was 9. What was the average response tme per NFS request at the server? black box = NFS server X server = 32,400 / 1,800 = 18 requests/sec R req = N req / X server = 9 / 18 = 0.5 sec 58 29
Lttle s Law: example 2 A large portal servce offers free emal servce. The number of regstered users s two mllon and 30% of them send send mal through the portal durng the peak hour. Each mal takes 5.0 sec on average to be processed and delvered to the destnaton malbox. Durng the busy perod, each user sends 3.5 mal messages on average. The log fle ndcates that the average sze of an e-mal message s 7,120 bytes. What should be the capacty of the spool for outgong mals durng the peak perod? 59 Lttle s Law: example 2 A large portal servce offers free emal servce. The number of regstered users s two mllon and 30% of them send send mal through the portal durng the peak hour. Each mal takes 5.0 sec on average to be processed and delvered to the destnaton malbox. Durng the busy perod, each user sends 3.5 mal messages on average. The log fle ndcates that the average sze of an e-mal message s 7,120 bytes. What should be the capacty of the spool for outgong mals durng the peak perod? AvgNumberOfMals = Throughput x ResponseTme = (2,000,000 x 0.30 x 3.5 x 5.0) / 3,600 = 2,916.7 mals AvgSpoolFle = 2,916.7 x 7,120 bytes = 19.8 MBytes 60 30
Arrvng requests Applyng Lttle s Law to the Watng Lne LINE Resource Completng requests w N = W X X 61 Applyng Lttle s Law to the Queue Arrvng requests LINE Resource Completng requests N = R X X 62 31
Applyng Lttle s Law to the Server Arrvng requests LINE Resource Completng requests s N = S X = U X 63 Interactve Response Tme Law source of requests Z 1 R = M/X 0 - Z M Computer System R X 0 R: avg. response tme Z: avg. thnk tme X 0 : avg. throughput M: number of sources of requests. 64 32
Interactve Response Tme Law source of requests (1) Z 1 M Computer System R N X 0 R: avg. response tme Z: avg. thnk tme X 0 : avg. throughput M: number of sources of requests. Apply Lttle s Law to the box (1): N = X 0 R 65 Interactve Response Tme Law source of requests (2) Z 1 M Computer System R N M X 0 R: avg. response tme Z: avg. thnk tme X 0 : avg. throughput M: number of sources of requests. Apply Lttle s Law to box (2): M = X 0 Z 66 33
Interactve Response Tme Law source of requests (2) Z 1 M Computer System R N M X 0 R: avg. response tme Z: avg. thnk tme X 0 : avg. throughput M: number of sources of requests. Combnng the results: N = X R M = X N + M = M = X R = 0 0 Z M X 0 Z 0 ( R + Z) 67 Interactve Response Tme Law Example A database server s capable of processng 20 requests/sec. The average thnk tme s 15 sec. What s the maxmum number of clent machnes that can be supported so that the average response tme does not exceed 2 seconds? 68 34
Interactve Response Tme Law Example A database server s capable of processng 20 requests/sec. The average thnk tme s 15 sec. What s the maxmum number of clent machnes that can be supported so that the average response tme does not exceed 2 seconds? Z = 15 sec, X 0 = 20 req/sec. So, M = (R + 15) * 20 (2 + 15) * 20 = 340 69 Summary of Basc Results Basc Concept of Queung Theory and Operatonal Analyss termnology and notaton servce tme and servce demand watng tme and queung tme Basc Performance Results and Examples utlzaton law: U = X * S forced flow law: X = V * X 0 servce demand law: D = V * S = U / X 0 Lttle s Law: N = R * X Interactve Response Tme Law: R = M/X 0 - Z 70 35
Practce Questons What s servce demand? What unts are used to measure servce demands? Is the servce demand a functon of the workload ntensty? What s the relatonshp between servce tme and servce demand? What s the relatonshp between response tme, servce tme, and watng tme? What s the relatonshp between resdence tme and response tme? What s the relatonshp between response tme and resdence tme? 71 Bounds on Performance Bounds on response tme and throughput can be computed from the servce demands only. U = D X 0 The resource wth the largest servce demand reaches 100% utlzaton before all others. Ths resource s the bottleneck. 72 36
Throughput Bound The utlzaton of a resource cannot exceed 100%: X U D 0 = 1 D Ths s the upper asymptotc bound on throughput under heavy load condtons. 73 Throughput Bound Apply Lttle s Law to the entre system: N K = R X 0 N X 0 K D = 1 = 1 D X 0 Ths s the upper asymptotc bound on throughput under lght load condtons. 74 37
Throughput Asymptotc Bounds X 1 mn max { D } 0, K = 1 N D 75 Throughput Asymptotc Bound Upgraded system= bottleneck (dsk 3) replaced by a 2x faster devce. 76 38
Lower Bound on Response Tme R = N X 0 mn 1 max = max N max N { D } N D 1 K { D }, D, = = 1 K 77 Response Tme Lower Bound 78 39
Usng QNs to Predct Performance 79 Usng QNs to Predct Performance The followng measurements were taken from a Web server. Compute the servce demands and response tmes for HTML and mage fles for the current load and for a load 5 tmes bgger. Measurement Perod 1 hour Number of HMTL fles 14040 Number of Image fles 1034 CPU tme per KB/read Avg. Sze of HTML fle Avg. Sze of an Image Fle Avg. Dsk Tme per KB/read Fle ndependent CPU Tme/HTTP Request 0.002 sec 3 KB 15 KB 0.012 sec 0.008 sec 80 40
Usng QNs to Predct Performance What knd of model? Open or closed? Sngle-class or multclass? Open snce the workload ntensty s gven as the number of requests processed durng a measurement nterval. Two-class model: HMTL and mages (sgnfcantly dfferent szes) Arrval rates: λ λ HTML mages = 14040 / 3600 = 3.9 = 1034 / 3600 = 0.29 req/sec req/sec 81 Usng QNs to Predct Performance Servce demands: D D D D CPU, HTML CPU, mages dsk, HTML dsk, mages = 0.008+ 0.02 3 = 0.014 = 0.08+ 0.002 15 = 0.038 = 3 0.012 = 0.036 sec = 15 0.012 = 0.18 sec sec sec 82 41
Open Multclass Queung Networks Ths wokbook comes wth the books "Capacty Plannng for Web Servces" and "Scalng for E-Busness" by D. A. Menascé and V. A. F. Almeda, Prentce Hall, 2002 and 2000. No. Queues: 2 No. of Classes: 2 Classes f Arrval Rates: 3.9 0.29 Servce Demand Matrx Classes f Queues fl Type fl (LI/D/MPn) HTML Images CPU LI 0.014 0.038 Dsk LI 0.036 0.180 83 Open Multclass Queung Networks - Utlzatons Ths wokbook comes wth the books "Capacty Plannng for Web Servces" and "Scalng for E-Busness" Classes f Queues fl HTML Images Total CPU 0.05460 0.01102 0.06562 Dsk 0.14040 0.05220 0.19260 by D. A. Menascé and V. A. F. Almeda, Prentce Hall, 2002 and 2000. 84 42
Classes f Queues fl HTML Images CPU 0.01498 0.04067 Dsk 0.04459 0.22294 Response Tme 0.05957 0.26361 Open Multclass Queung Networks - Resdence Tmes Ths wokbook comes wth the books "Capacty Plannng for Web Servces" and "Scalng for E-Busness" by D. A. Menascé and V. A. F. Almeda, Prentce Hall, 2002 and 2000. 85 43