Two-Level Iterative Queuing Modeling of Software Contention

Transcription

1 Two-Level Iterative Queuing Modeling of Software Contention Daniel A. Menascé Dept. of Computer Science George Mason University D. Menascé. All Rigts Reserved. 1

2 Motivation ncs cs Software View CPU Hardware View 2002 D. Menascé. All Rigts Reserved. 2

3 % Software Contention Time vs. Multitreading Level 100% 90% % Software Contention Time 80% 70% 60% 50% 40% 30% 20% 10% 0% Multitreading Level 2002 D. Menascé. All Rigts Reserved. 3

4 %Sofware Contention Time vs Non-CS/CS Ratio 100% 90% % Software Contention Time 80% 70% 60% 50% 40% 30% 20% 10% 0% Non-CS/CS Ratio 2002 D. Menascé. All Rigts Reserved. N=2 N=5 N=10 N=15 N=20 N=25 N=50 4

5 Software Contention example: multitreaded software server Server Tread Pool Waiting Being Processed Daniel A. Menascé 5

6 QN Models tat Capture SW Contention 1. Carleton University s Layered Queuing Networks (LQN) Neilson, J.E., C.M. Woodside, D.C. Petriu, and S. Majumdar, Software bottlenecking in client-server systems and rendezvous networks, IEEE Tr. Software Engineering, 21(9), Rolia, J. and K.C. Sevcik, Te Metod of Layers, IEEE Tr. Software Engineering, 21(8), 1995, pp Franks, G., T. Al-Omari, C.M. Woodside, O. Das, and S. Derisavi, Enanced Modeling and Solution of Layered Queueing Networks, IEEE Tr. Software Engineering, 35(2), Menascé s two-level iterative SQN-HQN model Menascé, D.A., Two-level Iterative Queuing Modeling of Software Contention, Proc. Tent IEEE/ACM Intl. Symp. Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2002),Fort Wort, TX, Oct , Menascé, D.A., V.A.F. Almeida, and L.W. Dowdy. Performance by Design: Computer Capacity Planning By Example, Prentice Hall, Daniel A. Menascé Modeling and Optimization of Multitiered Server-Based Systems 6

7 QN Models tat Capture SW Contention 1. Carleton University s Layered Queuing Networks (LQN) Neilson, J.E., C.M. Woodside, D.C. Petriu, and S. Majumdar, Software bottlenecking in client-server systems and rendezvous networks, IEEE Tr. Software Engineering, 21(9), Rolia, J. and K.C. Sevcik, Te Metod of Layers, IEEE Tr. Software Engineering, 21(8), 1995, pp Franks, G., T. Al-Omari, C.M. Woodside, O. Das, and S. Derisavi, Enanced Modeling and Solution of Layered Queueing Networks, IEEE Tr. Software Engineering, 35(2), Menascé s two-level iterative SQN-HQN model Menascé, D.A., Two-level Iterative Queuing Modeling of Software Contention, Proc. Tent IEEE/ACM Intl. Symp. Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2002),Fort Wort, TX, Oct , Menascé, D.A., V.A.F. Almeida, and L.W. Dowdy. Performance by Design: Computer Capacity Planning By Example, Prentice Hall, Daniel A. Menascé Modeling and Optimization of Multitiered Server-Based Systems 7

8 If we just consider te SQN we are ignoring time spent at te software resources due to contention for ardware resources. ncs Approac cs1 cs2 Software QN 2002 D. Menascé. All Rigts Reserved. 8

9 If we just consider te SQN we are ignoring time spent at te software resources due to contention for ardware resources. ncs Approac cs1 cs2 Software QN Te HQN model must consider tat some processes are not using ardware resources because tey are blocked for software resources. CPU disk 1 disk 2 disk 3 Hardware QN 2002 D. Menascé. All Rigts Reserved. 9

10 SQN-HQN Sceme Software QN N D s j, i s D j Adjust demands - B ' R i ( N ) N j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 10

11 Input Service Demands Software Modules Hardware Devices ( ) NCS CS 1 CS 2 Hardware Demands CPU Disk Disk Disk Software Demands sum across a column sum across a row 2002 D. Menascé. All Rigts Reserved. 11

12 Service Demands Software Modules Hardware Devices ( ) NCS CS 1 CS 2 Hardware Demands CPU Disk Disk Disk Software Demands D s cs 1 (total service time of a software module) 2002 D. Menascé. All Rigts Reserved. D s cs1, disk 1 (total service time of a software module at a pysical device) D disk 1 (total service time of te application at a given device) 12

13 SQN-HQN Sceme Software QN Avg. number of processes blocked due to software contention. N D s j, i s D j Adjust demands - B ' R i ( N ) N j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 13

14 SQN-HQN Sceme Software QN Avg. number of processes blocked due to software contention. N D s j, i s D j Adjust demands - B ' R i ( N ) N j D i Hardware QN Adjusted population for HQN 2002 D. Menascé. All Rigts Reserved. 14

15 SQN-HQN Sceme Adjusted demands for SQN. Software QN Avg. number of processes blocked due to software contention. N D s j, i s D j Adjust demands - B ' R i ( N ) N j D i Hardware QN Adjusted population for HQN 2002 D. Menascé. All Rigts Reserved. 15

16 Basic Idea Iteration between solving te SQN and te HQN. Number B of processes blocked due to software contention computed troug te SQN. Population at HQN is reduced by B. Service demands at SQN are adjusted to account for pysical contention D. Menascé. All Rigts Reserved. 16

17 SQN-HQN: Initialization D s j, i No adjustment at initialization s D j Software QN Adjust demands - B N ' R i ( N ) N j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 17

18 SQN-HQN: Solve SQN No Hardware Contention Software QN N D s j, i s D j Adjust demands - B ' R i ( N ) N = N - B j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 18

19 SQN-HQN: Solve HQN Software QN N D s j, i s D j Adjust demands - B ' R i ( N ) N = N - B j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 19

20 SQN-HQN: Adjust demands for SQN Software QN N D s j, i s D j Adjust demands - B ' R i ( N ) N = N - B j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 20

21 SQN-HQN: Solve SQN Again Software QN N D s j, i s D j Adjust demands - B ' R i ( N ) N = N - B j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 21

22 SQN-HQN: Solve HQN Software QN N D s j, i s D j Adjust demands - B ' R i ( N ) N = N - B j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 22

23 SQN-HQN: Adjust demands for SQN Software QN N D s j, i s D j Adjust demands - B ' R i ( N ) N = N - B j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 23

24 and so on Convergence is cecked on absolute relative error on te number of blocked processes in te SQN D. Menascé. All Rigts Reserved. 24

25 Adjustment of SQN Demands Single class case: D s D j D Multiple class case: i s j, i ' Ri ( N i residence time at device i. ) residence time at device i. for class r. D s j; r i D D s j; i, r i; r R ' i; r! ( N ) 2002 D. Menascé. All Rigts Reserved. 25

26 Example ncs cs1 cs2 Software QN disk 1 CPU disk 2 disk 3 Hardware QN 2002 D. Menascé. All Rigts Reserved. 26

27 Comparison wit oter approaces Absolute % Error N SQN-HQN GB SQN_HQN ASM ASPA GB: global balance equations ASM: Aggregate Server Metod [Agrawal and Buzen 1983] ASPA: Avg. Subsystem Population Approximation [Jacobson & Lazowska 1983] SQN is consistently pessimistic. ASPA is muc more complex to implement. ASM only works for one class D. Menascé. All Rigts Reserved. 27

28 Modeling Non-Software Resources Client tink time ncs network cs1 cs2 disk 1 Software QN B is te avg. no. of processes in te software resource waiting lines. Client tink time network CPU disk 2 disk 3 Hardware QN 2002 D. Menascé. All Rigts Reserved. 28

29 Open QN at te Software Level λ ncs cs1 Software QN cs2 disk 1 CPU disk 2 disk 3 Hardware QN 2002 D. Menascé. All Rigts Reserved. 29

30 SQN-HQN Sceme: Open SQN Software QN λ D s j, i s D j Adjust demands - N, B R ' i ( N ) N = N B j D i Hardware QN 2002 D. Menascé. All Rigts Reserved. 30

31 Results of Iterations for Open SQN Case Adjusted SQN Demands Iteration N Resp. Time Ns B NCS CS1 CS D. Menascé. All Rigts Reserved. 31

32 Modeling and Optimization of Multitiered Server-based Systems

33 Te SQN-HQN Model SQN te software queueing network models te software application using a software network HQN te ardware queueing network models te ardware infrastructure on top of wic te software runs Software modules blocked at te SQN are not counted as active in te HQN. Contention computed by te HQN is used to elongate te execution time of software modules at te SQN. Daniel A. Menascé Modeling and Optimization of Multitiered Server-Based Systems 33

34 Te SQN-HQN Model N SQN - (1) HQN Daniel A. Menascé Modeling and Optimization of Multitiered Server-Based Systems 34

35 A 3-tiered Server-based System: te SQN single-queue multiple-servers / Open 1 λ 1 1 λ m n p WS AS DS Daniel A. Menascé 35

36 Seidmann s Approximation to Model Multi-server Queues load independent 1 Delay device m demand = D/m demand = D(m-1)/m demand = D Daniel A. Menascé 36

37 Residence time at SQN Daniel A. Menascé 37

38 A 3-tiered Server-based System: te HQN A Closed QN Models te ardware infrastructure (processors, storage devices, load balancers, etc.) on top of wic te software runs. Daniel A. Menascé 38

39 Mapping te SQN to te HQN Mapping te service demand at eac SW tier to te HW it uses. Example: Daniel A. Menascé 39

40 Mapping te service demand at eac SW tier to te HW it uses. Example: Mapping te SQN into te HQN f1, f2 and f3 [0,1] indicate te fraction of te demand of a software module spent at CPU. Daniel A. Menascé 40

41 Te SQN-HQN Model λ SQN N - B (1) HQN Daniel A. Menascé 41

42 Modeling te Mulitiered System using SQN-HQN Daniel A. Menascé 42

43 Modeling te Mulitiered System using SQN-HQN Daniel A. Menascé 43

44 Example used parameters Daniel A. Menascé 44

45 Example First iterations of te SQN-HQN metod - te SQN (ε = 0.01): Daniel A. Menascé 45

46 Example First iterations of te SQN-HQN metod - te HQN: Daniel A. Menascé 46

47 Example R SQN as a function of n and p m=30 λ= 3.6 tps Daniel A. Menascé 47

48 Near Optimal Number of Treads Problem: wat is te optimal number of web servers, application servers and database servers tat minimizes te response time? More precisely, find an exact optimization solver is not an option because tere is no closed form expression for R SQN. Instead, tere is an iterative algoritm. Daniel A. Menascé 48

49 Near Optimal Number of Treads A combinatorial searc tecnique can be used to searc te solution space for te near optimal tuple. We used ill-climbing, wic starts from an arbitrary point in te searc space, and builds a relatively small neigborood, N, by perturbing te values of eac element of (m, n, p) For example, Daniel A. Menascé 49

50 Near Optimal Number of Treads Additionally, te number of treads of eac type in eac point in te neigborood also as to satisfy te memory constraint in eac pysical macine. In oter words, were M ws, M as and M ds are te memory footprint for eac tread type, and AM M1 and AM M2 stand for te available memory on macines 1 and 2. Daniel A. Menascé 50

51 Hill Climbing Algoritm Daniel A. Menascé 51

52 Near Optimal Results for Varying Arrival Rates used parameters Daniel A. Menascé 52

53 Near Optimal Results for Varying Arrival Rates Daniel A. Menascé 53

54 Concluding Remarks Simple approac. Open, closed, and multiclass QNs can be used at te SQN. SQNs can include non-software resources tat are not mapped to ardware resources. HQNs are closed and can be multiclass. Any tecnique can be used to solve te SQN and HQN. Tis includes any known approximation to multiple-server devices, priorities, simultaneous resource possession, etc D. Menascé. All Rigts Reserved. 54