Performance Measurement and Queueing Analysis of Parallel Connection Servers with Different Dispatch Probabilities

Transcription

1 Performance Measurement and Queueing Analysis of Parallel Connection Servers with Different Dispatch Probabilities Chung-Ping Chen Graduate Institute of Applied Science and Engineering Fu Jen Catholic University, Taipei, Taiwan Department of Electronic Engineering, National Taipei University of Technology, Taipei, Taiwan Ying-Wen Bai, Bao-Li Hu and Cheng-Yu Chung Department of Electronic Engineering Fu Jen Catholic University, Taipei, Taiwan and Abstract This paper is a description of our efforts to create a queueing model for parallel connection servers with different dispatch probabilities, so that we can estimate the system response time of parallel Web servers. First, we calculate the system response time of the parallel connection servers with different dispatch probabilities. Second, we simulate the queueing model of the parallel connection servers with different dispatch probabilities and find the system response time. Third, we measure the system response time of the parallel connection servers with different dispatch probabilities. Fourth, we compare these simulation and modeling results whilst the CPU loading is low, and we find that the maximum error is about.%. Fifth, we compare the measurement with the modeling results, which shows that the difference is about.4%-.%. Keywords-Equivalent queue model; Service rate; Dispatch probability I. INTRODUCTION As the size of a network becomes bigger, the gap in its system response time generates many results that can t be predicted. To even this difference there are many methods and software algorithms for managing many Web servers to reach a so-called load balance []. The DHT (distributed hash table)- based PP system proposes solving load balancing problem, but never considered the heterogeneous nature of the system. Zhu present an efficient, proximity-aware load balancing scheme by using the concept of virtual servers. It has higher capacity nodes carrying more loads, thereby minimizing the load movement cost and allowing load balancing to perform efficiently []. Chyouhwa proposes the virtual-server-based load balancing method systematically using an optimizationbased approach and derives an effective algorithm to rearrange loads among the peers. It systematically characterizes the effect of heterogeneity on load balancing algorithm performance [3]. Qi Zhang focuses on load balancing policies for homogeneous clustered Web servers that tune their parameters on-the-fly to adapt to changes in the arrival rates and service times of incoming requests [4]. Jiani Guo designs, implements and evaluates three scheduling algorithms, first fit (FF), streambased mapping (SM), and adaptive load sharing (ALS), for multimedia transcoding in a cluster environment. He proposes an online prediction algorithm and two new load scheduling algorithms, prediction-based least load first (P-LLF) and prediction-based adaptive partitioning (P-AP). The performance of the system is evaluated in terms of system throughput, out-of-order rate of outgoing media streams and load balancing overhead through real measurements using a cluster of computers [5]. Ho-Lin Chen shows that the degree of inefficiency in load balancing designs is highly dependent on the scheduling discipline used at each of the back-end servers. It is changed to shortest remaining processing time (SRPT); the degree of inefficiency can be independent of the number of servers. Switching the back-end scheduler to SRPT can provide significant improvements in the overall mean response time of the system as long as the heterogeneity of the server speeds is small [6]. To attain such a balance, the different hardware connections which influence the system response time need to be studied. The FTMP (Fault-Tolerant Multiprocessor) is a digital computer architecture which could have a wide application in several life-critical situations in aerospace. A dispatch probability model is also presented. Experience with an experimental emulation is described [7]. Solving the probabilistic optimal dispatching problem using fixed steady probability and a full contingency-set, an effective contingency-set according to the transmission element outage distribution factors is founded [8]. To simplify the analysis of both network performance, packet flow and congestion control, many people use an equivalent queueing model which can deduce the equivalent equation and represent the system response time of the whole network. We classify the basic network structure as a serial or a parallel connection and analyze the error between a queueing model and the resulting measurement [9]. When several servers are serial, the overall system response time is proportional to the number of servers []. When two server parallel connections of the same service rate have a high CPU loading, the overall system response time is in reciprocal proportion to the amount of servers. When two server parallel connections of //$5. IEEE 9

2 the same service rate have a low CPU loading, the system response time has not change too much []. In addition, we aim at two parallel connection servers with different dispatch probabilities. From the measurement results we learn the error in the system response time between the measurement and analysis []. II. THE QUEUEING MODEL FOR THE PARALLEL CONNECTION SERVERS We use an equivalent model to represent the parallel connection servers. We find the system response time of the equivalent model and verify it by measuring the system response time. Table shows the system definition and the model parameters. We set up the measurement system with msec as the measurement unit for the system response time. TABLE I. SYSTEM DEFINITION AND PARAMETERS OF THE MODEL Parameter Description Definition λ Web request rate Requests/sec μ n Service rate of the n th server Requests/sec Dispatch probability of the n P th parallel n queue EP n ( T ) System response time of the n th parallel queue E ( ) eq T Equivalent system response time of the parallel connection server msec msec We use the idea of a parallel equivalent electric circuit as our analytical foundation and, by measuring, verify the performance. In this paper we use the approximate equations for the different dispatch probabilities of the parallel connection servers. The model representation is shown in Fig.. Pλ λ E ( P ) T P + P = μ P λ P E ( P T ) μ P Figure. Equivalent model of parallel queues λ E P eq μ Peq At the beginning we use queueing models and Markov Chains [] to analyze the parallel connection servers with the following assumptions: All requests are first in first out first in the system. The total of the requests in the system is unlimited. Each node in the system can have Poisson arrivals from an outside node. The request can leave the system from another node. All service times are exponentially distributed. The arrival rate λ enters the node of each parallel connection server as shown in Fig.. Table II shows the measurement environment for deciding the system response times. We analyze these conditions of the parallel connection servers and the major factors, μ, μ, P, P, and λ. We find the equivalent system response time to be Eeq = PE P + PE P, where P and P are the dispatch probability, E ( T ) and E ( T ) are the response time P P of each queue. Therefore the equivalent system response time is the sum of the system response times of all queues. We use six sets of different service rates and nine kinds of different dispatch probabilities, for a total of 54 cases. We measure the system response time of different dispatch probabilities. TABLE II. SPECIFICATIONS OF THE PARALLEL CONNECTION SERVERS Client Server Server CPU 3.GHz.8GHz RAM G G O.S. Windows Advanced Server 3 Network LAN (Mb/sec) Tool/Web Server Webserver Stress Tool IIS 6. The system response time of the first server is as shown in Eq. (). EP = μ Pλ The system response time of the second server is as shown in Eq. (). E P = μ P λ The dispatch probability of the second server is as shown in Eq. (3). P = P (3) The service rate of the first server equals that of the second server n times is as shown in Eq. (4). μ = nμ (4) The system arrival rate equals the service rate of the second server p times is as shown in Eq. (5). λ = pμ (5) The equivalent system response time of the parallel connection servers is as shown in Eq. (6). E = PE + PE (6) eq P P We plug Eqs. (), (), (3), (4) and (5) into Eq. (6) and obtain Eq. (7). () () 93

3 E eq P P = + nμ Ppμ μ ( P) pμ The arrival rate equals p times, from.7 to.9, the service rate of the second server. The service rate of first server equals that of the second server n times, from to 3. The dispatch probability P of the first server is from. to.9. The service rate of the second server μ is set to. We plug n, p, μ and P into Eq. (7) and obtain the system response time as shown in Fig μ =μ, λ=.7μ μ =μ, λ=.8μ μ =μ, λ=.9μ μ =μ, λ=.7μ μ =μ, λ=.8μ μ =μ, λ=.9μ μ =3μ, λ=.7μ μ =3μ, λ=.8μ μ =3μ, λ=.9μ od Server Figure. System response time of different dispatch probabilities (7) System response time (ms) P =. P =. P =.3 P =.4 P =.5 P =.5 P =.6 P =.7 P =.8 P = Arrival Rate (requests/sec) Figure 3. System response time by simulation If the system works at a high CPU loading, we simulate the service queue of the two parallel connection servers. Then we use the equation to calculate the system response time of the two parallel connection servers and set the arrival rate at 3 requests/sec. Fig. 4 shows the system response time of five sets of different dispatch probabilities with an equal service rate of each server. When the arrival rate is low, the system response time is very small. When the service rates of two servers of the simulation are the same, the difference of the two dispatch probabilities increases, and the system response time increases. When the two dispatch probabilities are equal, the system response time decreases. Fig. 4 shows the smallest system response time to be P = P =.5. The system response time increases in P =. and P =.9. III. THE SIMULATION OF TWO PARALLEL CONNECTION SERVERS OF DIFFERENT DISPATCH PROBABILITIES To analyze the error margin of a serial connection network we have to check the accuracy of the previous equivalent serial equation. After modeling we make use of the software network simulation tool to support our research during the simulation stage. For our simulation we use the Queuing Network Analysis Tool (QNAT) [3]; it can have a closed or open queueing network by simulation We focus the parallel connection network simulation and set the arrival rate and the service rate. To find the arrival rate we carry out a measurement from requests/sec to 3 requests/sec with an increment step of requests/sec. The service rate is set at requests/sec. As we set the dispatch probabilities P from. to.5, we increase each step by., P from.9 to.5, and we reduce each step by., and therefore we conclude that P + P =. Fig. 3 shows the system response time of dispatch probabilities. When the dispatch probabilities are smaller than.3, the system response times show small changes. When the dispatch probability equals.4, the system response time just starts rising while the arrival rate reaches 5 requests/sec. When the dispatch probability equals.9, the system response time starts rising while the arrival rate reaches requests/sec, as shown in Fig. 3. Figure 4. System response time (ms) P P P P P =.5, P =.5 =.4, P =.6 =.3, P =.7 =., P =.8 =., =.9 P Arrival Rate (requests/sec) System response time of different dispatch probabilities IV. THE PERFORMANCE MEASUREMENT OF TWO PARALLEL CONNECTION SERVERS OF DIFFERENT DISPATCH PROBABILITIES To verify the performance representation of the parallel connection servers of a real network, we use a local network as the measuring environment and both the Web Server Stress Tool (Webstress Tool) [4] and the ASP Web for the measurement. Then we set up the server IP address as , the server IP address as and the client IP address as For the investigation of the effect of the parallel connection we use the Webserver Stress 94

4 Tool measurement software to carry out the measurement of the system response time of the parallel connection servers. The client is responsible for creating the Web service request to the Web servers while the Web servers then aim to fulfill this client request to provide a Web page service. To increase the accuracy of the measurements we provide the Web page service with regular operations including a large number of mathematical operations. We adjust two servers to different service rates, their dispatch probability ratio to :9, and the Web user number from to, and then carry out the performance measurement. The measurement results change when we adjust the dispatch probabilities each time, from :9, :8, 3:7, and then to 9:. The Web user numbers are from - with a step of. To reduce the measurement error, the computer configurations of all Web servers are set similarly whilst simultaneously the service requests for using our ASP Web page create a similar CPU loading which can be controlled by adjustable multiplication loops. When we use K, K, K and M multiplication loops as the loading to measure the system response time, we find that M multiplication loops give better measurement results. Therefore, in the measuring environment we use M multiplication loops to mimic the system response time for the speed of the specific CPU, and these multiplication loops play the major part in the main loading of the system response time that can cause a delay longer than that of the network. In addition we maintain the system operation at mediumhigh CPU loading with an arrival rate that is equal to or much greater than the service rate, with K, K, K and M multiplication loops. x 4 5 M=. M=. M=.3 4 M=.4 M=.5 M=.6 3 M=.7 M=.8 M= Number of Users Figure 5. ASP=M: The system response time of different dispatch probabilities Two Web servers are parallel and are measured with their system response time as the performance index. Because at a high CPU loading the parallel connection servers generate an irregular reaction, we cannot accurately control either the arrival rate or the service rate. Therefore we compare the system response time of the model, the result of the simulation and the measurement. We adjust the total number of users that can access the Web page and then carry out our analysis, measurement and comparison. Fig. 5 shows the system response time of the Web servers with ASP= M, when the dispatch probabilities have been adjusted from. to.9. From the server point of view the CPU loading of a parallel server is ASP=M, its dispatch probabilities are between.-.9 respective to the variation of the system response time. If the server gets only a % loading, the system response time is lowest. Fig. 6 shows the comparison between the system response time of the model and the actual measurement with different dispatch probabilities. The error margin is.4%-.%. Error of Response Time (%) M=., K=.9 M=., K=.8 M=.3, K=.7 M=.4, K=.6 M=.5, K=.5 M=.6, K=.4 M=.7, K=.3 M=.8, K=. M=.9, K=. 5 Number of Users Figure 6. ASP=M and K: The error margin of the system response time of two parallel connection servers with different dispatch probabilities The performance of two servers with parallel connections of different service rates is investigated. By adjusting the different dispatching probabilities we obtain the best system response time. We use a low CPU loading at CPU loading ASP= K and take its service rate as the base of its parallel connection. Fig. 7 shows that the lowest system response time is at ASP= K with dispatch probability at.9 or at 9: (D9). Because the margin would be too big for displaying the characteristics of the system response time of several experiments here, we indicate the Y shaft use logarithms. 3 ASP=K ASP=K ASP=3K ASP=4K ASP=5K ASP=6K ASP=7K ASP=8K ASP=9K ASP=K D9 D8 D37 D46 D55 D64 D73 D8 D9 Figure 7. ASP=K: The parallel system response time of different dispatch probabilities is from ASP=K to ASP=K with an increment step of ASP=K We use the CPU loadings ASP= Ks to Ks, a total of different server parallel connections of different service 95

5 rates and the dispatch probability at :9, :8, 3:7,, 9:, a total of nine measurements, thus comparing 8 sets of data in all. Each set of data is used by - users, adding 5 users each time, totaling measurements, give us 378 measurements altogether. The experiment finds that the service rate of two parallel servers with CPU loadings ASP= Ks and ASP=9 Ks, Ks and Ks and dispatch probability at 5:5 has the lowest system response time, as shown in Fig ASP=9K ASP=K ASP=K D9 D8 D37 D46 D55 D64 D73 D8 D9 Figure 8. ASP=K: The parallel system response time of different dispatch probabilities with CPU loading ASP=9K to ASP=K with an increment step of ASP=K When the service rates of a single servers are less than the CPU loading ASP= K, the overall system response time of two parallel connection servers improved as shown in Fig ASP=K ASP=K 5 ASP=3K ASP=4K ASP=5K ASP=6K 5 ASP=7K ASP=8K D9 D8 D37 D46 D55 D64 D73 D8 D9 Figure 9. ASP=K: The parallel system response time of different dispatch probabilities with CPU loading ASP=K to ASP=8K with an increment step of ASP=K. Fig. shows that this measurement matches the mathematical model. At a low CPU loading the service rates of two parallel connection servers differ twice, and the best system response time is the dispatch probability of the biggest service rate at 9:, or.9 (D9). At a high CPU loading, the two server service rates differ three times, the service rate of the slow server implicating the system performance and increasing the response time of the system ASP=K ASP=3K ASP=4K ASP=5K ASP=6K ASP=7K ASP=8K ASP=9K ASP=K D9 D8 D37 D46 D55 D64 D73 D8 D9 Figure. ASP=K: The parallel system response time of different dispatch probabilities with CPU loading ASP=K to ASP=K with an increment step of ASP=K. In two parallel connection servers with similar service rate, one can from the ratio of their different dispatch probabilities obtain the lowest system response time. Fig. shows two parallel connection servers with the loadings ASP= Ks and ASP=5 Ks, the dispatch probability ratio at :9, but the system response time isn't lowest. However, it will be the lowest at the ratio of :8 (D8) ASP=K ASP=5K Average D9 D8 D37 D46 D55 D64 D73 D8 D9 Figure. Three system response times of parallel connection servers at two different dispatch probabilities with CPU loading ASP=K, ASP=5K and the average. When the service rate of two parallel connection servers is the same, the system response time is lowest at the probability distribution 5:5 (D55). Fig. shows that the system response time is lowest when the loading of two parallel connection servers is ASP=K with the ratio of the dispatch probability at 5:5. 96

6 5 4 3 ASP=K ASP=K Average D9 D8 D37 D46 D55 D64 D73 D8 D9 Figure. Three system response times of parallel connection servers at two different dispatch probabilities with the CPU loading ASP=K. V. CONCLUSION According to the experiment measurement, the dispatch probability shall be the same as in the model to both obtain the best balance and to reduce the system response time by a maximum value. If the service rate of a server is different, the dispatch probability should be chosen according to the service rate of the server. The parallel connection queue with medium-high CPU loading can improve the system response time because of the system response time s nonlinear characteristics. After investigating the characteristics of the parallel connection servers, we will in the future work towards partitioning complex series-parallel connection servers capable of predicting the system response times. When two parallel connection servers operate, and their service rates differ by less than %, we can fine-tune the dispatch probability to find out the best system response time. If the difference is greater, we can t do this. At a high loading, parallel connection servers with the same service rate can obtain a performance improvement when their ratio of dispatch probability is equal. At a medium loading two parallel connections servers with the same service rate obtain an improvement of their overall performance. At a low loading we need not increase the number of servers, since the performance improvement would be very limited. REFERENCES optimization of the data queueing memory management in high-speed network processors, Proceedings of Design Automation Conference,, 39th, pp , -4 June,. [] Yingwu Zhu, and Yiming Hu, "Efficient, proximity-aware load balancing for DHT-based PP systems," Proceedings of the IEEE Transactions on Parallel and Distributed Systems, vol.6, no.4, pp , April 5. [3] Chyouhwa Chen, and Kun-Cheng Tsai, "The Server Reassignment Problem for Load Balancing in Structured PP Systems," Proceedings of the IEEE Transactions on Parallel and Distributed Systems, vol.9, no., pp.34-46, Feb. 8. [4] Qi Zhang, ALma Riska, Wei Sun, Evgenia Smirni, and Gianfranco Ciardo, "Workload-aware load balancing for clustered Web servers," Proceedings of the IEEE Transactions on Parallel and Distributed Systems, vol.6, no.3, pp. 9-33, March 5. [5] Jiani Guo, and Laxmi Narayan Bhuyan, "Load Balancing in a Cluster- Based Web Server for Multimedia Applications," Proceedings of the IEEE Transactions on Parallel and Distributed Systems, vol.7, no., pp.3-334, Nov. 6. [6] Ho-Lin Chen, Jason R. Marden, and Adam Wierman, "On the Impact of Heterogeneity and Back-End Scheduling in Load Balancing Designs," Proceedings of the IEEE INFOCOM 9 - IEEE 8th Conference on Computer Communications, pp.67-75, 9-5 April 9. [7] Albert L. Hopkins, T. Basil Smith, III, and Jaynarayan H. Lala, "FTMP A highly reliable fault-tolerant multiprocess for aircraft," Proceedings of the IEEE, vol.66, no., pp. -39, Oct [8] H. Zha, X. S. Han, Y. L. Wang, and L. Han, "An effective algorithm of the power system probabilistic optimal dispatching," Proceedings of the Third International Conference on Electric Utility Deregulation and Restructuring and Power Technologies, 8. DRPT 8, vol., no., pp.9-96, 6-9 April 8. [9] F. S. Tu, D. P. Song, and Sheldon X. C. Lou, Steady state performances analysis in stochastic serial production lines, Proceedings of the 3 nd IEEE Conference on Decision and Control, vol.3, pp , 5-7 Dec [] Chung-Ping Chen, Ying-Wen Bai and Yin-Sheng Lee, Performance Measurement and Queueing Analysis with Low Blocking Probability of Serial Connection Networks, Proceedings of the I²MTC 8 IEEE International Instrumentation and Measurement Technology Conference, pp.6-, May -5, 8. [] Chung-Ping Chen, Ying-Wen Bai and Cheng-Hung Tsai, "Performance Measurement and Queueing Analysis at Medium-High Blocking Probability of Parallel Connection Servers with Identical Service Rates", Proceedings of the 8th WSEAS International Conference on Communications on Data Networks, Communications, Computers (DNCOCO '9), pp , November 7-9, 9. [] Gunter Bolch, Stefan Greiner, Hermann de Meer, and Kishor S. Trivedi, Queueing Networks and Markov Chains, Wiley-Interscience, 6. [3] Hema Tahilramani Kaur, D. Manjunath and Sanjay K. Bose, The Queueing Network Analysis Tool (QNAT), Proceedings of International Symposium on Modeling, Analysis and simulation of Computer and Telecommunication Systems, pp ,. [4] [] C. Ykman-Couvreur, J. Lambrecht, D. Verkest, F. Catthoor, A. Nikologiannis, and G. Konstantoulakis, System-level performance 97