A Quantitative Model for Capacity Estimation of Products

Transcription

1 A Quantitative Model for Capacity Estimation of Products RAJESHWARI G., RENUKA S.R. Software Engineering and Technology Laboratories Infosys Technologies Limited Bangalore INDIA Abstract: - Sizing activity estimates the computer resources required to meet the workload demand in a costeffective manner while meeting the performance objectives. While there are several techniques to aid in system sizing, only a few are suitable for product deployments that have clients with different workload profiles, and lab configuration close to deployment site is not available for testing. Here, we propose an approach that uses the resource demands of workload components on the lab configuration, the workload and performance targets of the client deployment to arrive at a suitable target configuration. We validate the approach by applying it to real-life banking product. Key-Words: - Performance Modeling, Workload, Queuing Networks, Scalability, Benchmarks 1 Introduction Sizing activity estimates the computer resources required to meet the workload demand in a costeffective manner while meeting the performance objectives. While there are several techniques to aid in system sizing [1,2] only a few are suitable in practice where a product needs to be deployed for different client workload profiles. Menasce et al [1,2] explain several sophisticated analytical techniques that can be used to build a performance model of an existing system. The predictive nature of these models helps save time, effort and money by avoiding a measurement cycle every time the client workload profile changes. However, most practical sizing exercises have their lab configuration that is much lower compared to real-life site deployments, due to cost constraints. Hence, capacity planners often resort to doing a performance testing in the lab, and intuitively arrive at suitable target environment. There are other techniques proposed by vendors [5,6] who have customized basic operational laws defined in queuing theory [4] to work in-conjunction with their specific scaling strategies to give customized solutions for their specific platform. However, these techniques fail to predict capacity if the workload mix changes. The key requirement for our capacity estimation model were: (a) Predict the capacity with any workload profile, even though lab may be of lower configuration (b) Accurate sizing by accounting for the application scalability [7] (c) Support movement across different hardware configuration, by leveraging the industrystandard benchmarks [8,9]. (d) Estimate processor requirements as well as disk subsystems. And none of the above mentioned techniques address all the requirements completely. Hence, our objective here is to propose a new quantitative model to meet all the requirements mentioned above. The paper is organized as follows. Section 2 does a literature survey. Section 3 proposes the problem solution and formulates the estimated scale factor. Section 4 details the prototype experiments with a real-life Internet banking product, and presents the results of applying the technique. Section 5 offers conclusion and outlines future research. 2 Literature Survey The cornerstone of performance modeling is the concept of a resource (such as, disks, memory and network) and its capacity to service demands placed by the requests. The objective is to predict performance of the system in terms of throughput, response times, and the utilization of the various resources. There are a wide variety of analytical techniques in the literature to solve this network of queues, and some are suggested for sizing studies. Bounding analysis [4] is one technique applicable where the different candidate classes have similar resource requirements, and can therefore be modeled with one representative class. For other cases [10,11,12] techniques based on mean value analysis

2 (MVA) are suggested. There are several MVA algorithms that cover the separable (or product form), non-separable queuing systems, and various devices types like load dependent and load independent (LI) devices. The main input to these models is the service demand that quantifies the demand of the request on a device. These algorithms are built using some fundamental operational laws. One important law is the Utilization law [3] that states the utilization of a resource to be equal to the product of the throughput of the resource and the service demand at the resource. Utilization law is frequently used to model open system, that are characterized by jobs arriving from outside, and the behavior of their arrival (the rate at which they arrive, the distribution of job types etc.) is independent of the way the system processes these jobs. The techniques mentioned above abstract the system as a network of queues of system resources (hardware) only. However, we also need to determine how the following two factors impact the accuracy of prediction of the techniques. They are: (a) Effects of simultaneous resource possession, that is at any given instance there could be multiple requests trying to acquire the system resources. This introduces delays, and as mentioned in [13] predictions from MVA and related techniques could grossly overestimate the throughput and utilization of devices for a given workload. One simple technique for modeling simultaneous resource possession is the method of surrogates [13]. The method suggests adding a delay device referred to as the surrogate delay in the performance models. These delays do not interfere with throughput, but increase the response times of the system for a given utilization. The effect of simultaneous resource possession is difficult to be quantified for different hardware configurations or for a new workload composition. Hence, these could pose a potential problem while estimating target response times. However, estimations of target configuration based on arrival rate to be supported and utilizations of devices will not be impacted. The paper will not deal will target configuration estimations based on response time requirements. (b) Application scalability that determines how the application can leverage its underlying hardware. Jogalekar et al [7] define scalability as not just the ability to operate, but to operate efficiently and with adequate quality of service, over the given range of configurations. Thus, the increased capacity should be in proportion to the cost, and quality of service should be maintained. They quantify this by a scalability metric ψ. Accordingly, applications have been classified as linearly scalable (ψ = 1), super linear (ψ > 1) and not scalable (ψ < 1). We need to account for the scalability metrics while estimating capacity from lab to target configuration. 3 Problem Solution The technique we propose has the following broad steps (see figure 1): (a) Measure the service demands of each individual workload components in the lab configuration. (b) Propose the amount of scaling required in terms of estimated scale factor at each device (i.e. processors and disk) with respect to lab configuration. This estimation would be based on target workload requirement and performance objectives to be met during deployment. (c) The estimated scale factor is a scalar and offers benefits such as platform independence. This can hence be used in conjunction with models [7] that account for application scalability, yielding an effective scale factor. (d) The effective scale factor can be used to arrive at the possible target configuration using industry-standard benchmarks or other vendor specific product data sheets. (e) Further analysis of the target configuration, the effective scale factor and service demands of lab configuration can be used to derive the target configuration performance model. This model will help do several what-ifs on the target configuration with different workload mixes. Service demands on lab configuration Industrystandard Benchmark Workload Model Estimated Scale factor Effective Scale factor Target Configuration Performance SLAs Lab configuration Figure 1. Steps in building a quantitative model

3 In this paper our focus will be to arrive at the estimated scale factor. We consider separable queuing networks, where each device can be evaluated independently to arrive at the overall system performance. Besides, we will consider only load independent (LI) (i.e. those satisfying the service demand homogeneity assumptions [4]). 3.1 Formulation of Estimated scale factor Service demands quantify the demand of the request on a device, and vary if the device configuration is changed. Utilization law indicates that the net effect of device upgrade results in reduction of service demands. Here, we make an assumption that all the components of the application have similar scalability characteristics. Thus, the impact of change in configuration of the i th device impacts the service demands of the M classes of requests passing through it uniformly. Let, {D (1,i) LAB, D (2,i) LAB,D (M,i) LAB } be the set of service demands of requests on the lab configuration. {D (1,i), D (2,i),D (M,i) } be the set of service demands of requests in the target after upgrade. Then, (D (1,i) LAB / D (1,i) ) = (D (2,i) LAB / D (2,i) ) = (D (M,i) LAB / D (M,i) ) = a constant We refer to the constant factor as the estimated scale factor. The notion of estimated scale factor at each device level will be the basis for us to eventually arrive at the target configuration. Consider a lab configuration system with the M request classes passing through K LI devices. Here, we consider only LI devices for upgrade as delay devices have infinite capacity and hence, are never considered for upgrade. For each request class C m (m= 1..M), the inputs and the overall system performance metrics are defined as: X m LAB, the overall system throughput For each device i=1 to K, D (m,i) LAB, the service demand of request class C m, U i LAB, utilization were computed for the K devices as: m=m LAB LAB U i = (X m m=1 * D (m,i) LAB ) (1) The target configuration requirement in the workload model mentions the expected arrival rates of various request classes. Besides, performance objectives related to utilization of the devices will mentioned. The determination of the required target utilization is primarily dependent on the required response time. Since response time increases with increase in the utilization of resources, a suitable utilization that gives the desired response time needs to be obtained. Let us represent the target performance metrics for the M request classes and K LI devices as: X m, the overall arrival to be supported For each device i=1 to K, D (m,i), the service demand of request class C m, U i, utilization were computed for the K devices as: m=m U i = (X m m=1 * D (m,i) ) (2) Now we apply our assumption that the device upgrade reduces the service demand of all the requests classes proportionately. Hence, the estimated scale factor SF i would be D (m,i) LAB / D (m,i) = SF i (constant) for all m request classes passing through the device i (3) Using equations (1) (2) and (3) we get, SF i = ( U LAB i /U i ) * m=m [( (X m * D LAB LAB (m,i) )) / ( (X m m= (4) * D (m,i) LAB ))] Delays due to simultaneous resource possessions may differ in the target configuration. However, since the estimated scale factor computation is based on taking the throughputs and utilization, equation

4 (4) remains unaltered irrespective of the delay encountered in the target configuration. A special case when the workload mix in the lab and target configuration is equal is: X m LAB / X m = k (constant) for all m request classes. The value k represents the intensity increase of the workload mix. Thus equation (4) for this case would be, SF i = ( U i LAB /U i ) * k It is important to note that the equation (4) does not mandate construction of lab performance models where the service demands of the workload components required in target are available along a mixed run involving the workload components. 4 Case Study 4.1 Application Characteristics and Modeling Assumptions The application was an Internet Banking product, used by tellers in the several banks distributed across geographical locations. A load generator was setup to fire requests with a constant arrival rate. The other modeling assumptions were: (a) A set of parallel and equal capacity device was modeled by one equivalent logical device that mimics their behavior. The symmetric multiprocessor (SMP) and disk subsystem configured as RAID was considered as a single logical device. (b) The memory was sufficiently large to avoid memory bottlenecks during the measurement cycle. (c) The measurement test-bed was configured and tuned to eliminate hardware setup problems. (d) For service demand computation the utilization of a device is taken as the sum of the inherent service requirements and the overheads. The target configuration is assumed to have the same proportion of the inherent service requirement and the overheads in the proposed configuration. 4.2 Measurement Test-bed The lab configuration consisted of the architectural components of the application on 2 boxes. The web and application server components were deployed on one Sun Enterprise 6500, 400 MHz, 4 s under Solaris 8. The database was hosted on another machine of the same configuration, however a disk subsystem configured as RAID 0 (with striping) and the database size was about 500 GB. The objective of these tests was to determine service demand values for each component by using the device utilization and throughput processed. (a) Tests were conducted for each component in the workload separately as the monitoring tools such as iostat, sar or perfmon do not report application-level break-up of device utilization. (b) The correctness of configuration of the test bed was ensured. (c) The measurements used to derive service demands were validated using operational laws. (d) The device utilizations for processors were computed in a manner to account for the system overheads and the actual processing time (i.e. %processor time = %sys + %user in iostat). The %sys overhead in the particular case was negligible compared to the user utilization. Similarly, for disks the utilization (%tm_act in iostat) included seek, rotational latency, data transfer and any time lost in contentions of I/O path. (e) Monitoring utilities (like vmstat in Unix and AIX O.S.) were used to ensure that memory was not a bottleneck. Table 1, below indicates the service demands of 7 banking online transactions, such as cash withdraw from account, issue a demand draft etc. However for the sake of confidentiality we name them as C1, C2 etc. The disk utilizations at the database servers were negligible and hence ignored. Request Service demand Service demand Class At Box 1 - At Box 2 C C C C C C C Table 1. Service demands in lab configuration Tests were conducted for a particular mix of these 7 transactions in the lab configuration and table 2 mentions the observed throughput and utilization of the servers.

5 Request Class C C C C C C C Box 1 Throughput (Request/sec) Box 2 Utilization 29% 66% Table 2. Performance in lab configuration For the prototype experimentation, we assumed a target throughput and utilization requirement as displayed in Table 3. The arrivals in the target displayed a different mix compared to the lab configuration. Request Class Throughput (Request/sec) C C C C C C C Box 1 Box 2 Utilization 23% 47% Table 3. Target requirements Using equation (4) we determine the estimate scale factor at the Box 1 and Box 2 as follows: Box 1 Box 2 Estimated Scale factor Table 4. Estimated scale factor We need an estimated scale factor to 2. Using the Sun specific benchmark, we obtain for Enterprise 6500, 400 MHz, with 8 s under Solaris 8 [5] (see Table 5) the Sun s Constant Performance Metric (SCPM) number as The ratio of the SCPM benchmark numbers of lab configuration (with 4 ) and the same series of machine with 8 configuration is 1.95, and this can be associated with the required effective scale factor of the banking application. The SCPM benchmark makes certain assumption about the application scalability ψ, and reports the performance of the two configuration in-terms of the SCPM number. Numbers reported by other benchmarks are also not just the raw hardware processing power but performance with a specific benchmarked application and workload. For example, Transaction Processing Council [8] reports a benchmark for commercial OLTP workloads and reports the benchmark in-terms of number of transactions completed in a minute (tpmc). And SPEC benchmark [9] measures performance of scientific workloads on different configurations. For the same two machines, depending on the benchmark chosen, the ratio of their processing powers could be different, because ψ would be different. In practice it is important to choose an industry-benchmark that bears close resemblance to the applications scalability characteristics. N SCPM Number (Solaris 8, 400 Mhz) Table 5. SCPM benchmark of E6500 on Solaris 8 Here we chose to run target workload tests on 8 machine. We performed tests to validate the accuracy of the procedure. Both the boxes were Sun Enterprise 6500 servers, 400 MHz, with 8 s under Solaris 8. The application was subjected to an arrival as given in the table 3. The table 6 mentions the resulting utilization of the servers. Actual utilization on target Box 1 Box 2 26% 50% Table 6. Resulting utilizations on target The deviation between actual observed utilization (table 6) and target utilization (table 3) was close to 3% on both the boxes. 5 Conclusion The paper proposes a quantitative model for estimating the capacity suitable for product deployments scenarios. The actual target configuration was not available for testing and workload profiles were changing. The approach proposed here follows from application of the Utilization law and introduces the notion of estimated scale factor and effective scale factor. The

6 approach was applied to capacity estimation of an Internet Banking setup. There is scope for more research and practical work in the area. We covered only one type of projection technique based on arrival pattern, a similar relationship can be drawn for a supporting a population of users or achieve response time targets. Such techniques will be more complex as they need to account for effects of simultaneous resource possessions. Besides we need to extend the technique for handling load-dependent devices. [12] M.Reiser and S.S. Lavenberg. Mean value analysis of closed multichain queuing networks. journal of the ACM, 27(2): , April [13] Patricia A. Jacobson and Edward D. Lazowska. Analyzing queueing networks with simultaneous resource possession, Communications of the ACM, 25(2):142{151, February References: [1]Daniel Menasce, Virgilio Almeida and Larry Dowdy. Capacity Planning and Performance Modeling: from Mainframes to Client-Server Systems, Reading, Prentice Hall, [2]Daniel Menasce and Virgilio Almeida. Capacity Planning for Web Services: Metrics, Models, and Methods, Reading, Prentice Hall, 2001 [3] Kleinrock L. Queueing Systems, Vol I: Theory, John Wiley & Sons 1975 [4] Edward Lazowska, John Zahorjan, Scott Graham, and Kenneth Sevcik. Quantitative System performance, Reading, Prentice Hall, [5] Adrian Cockcroft and Bill Walker. Capacity Planning for Internet Services, Sun Microsystem Press, [6] Morgan Oslake, Hilal Al-Hilali and David Guimbellot. Capacity Model for Internet Transactions, Technical Report MSR-TR-99-18, April 1999 [7] Prasad Jogalekar and Murray Woodside. Evaluating the Scalability of Distributed Systems, IEEE Transactions on Parallel and Distributed Systems, Volume 11, No.6, June [8] Transaction Processing Council, at URL [9] Standard Performance Evaluation Corporation, at URL [10] Y. Bard. Some extensions to multi class queuing network analysis. In: M.Arato, A.Butrimenko and E.Gelenbe, eds. Performance of Computer Systems, North-Holland, Amsterdam, Netherlands, [11] P.J.Schweitzer. Approximate analysis of multiclass closed network of queues. Proceedings of International Conference on Stochastic Control and Optimization, 25-29, Amsterday, Netherlands, 1970.