Computational performance and scalability of large distributed enterprise-wide systems supporting engineering, manufacturing and business applications

Transcription

1 Computational performance and scalability of large distributed enterprise-wide systems supporting engineering, manufacturing and business applications Janusz S. Kowalik Mathematics and Computing Technology The Boeing Company Abstract The Boeing Company has been implementing large client/server systems that support all major engineering, manufacturing and business functions. One of the challenges has been computational performance and scalability to a very large number of users and data sets. Modeling and simulation combined with analytical approximations have been used to predict performance, identify bottlenecks and to establish the required system capacity. This approach is highly dependent on the accurate workload predictions which usually are uncertain and changing. To get around this difficulty the system models have been used in conjunction with parametric studies that explore ranges of system performance behavior. Good performance and scalability are important system quality indicators but there are other issues that have to be taken into account such as system's cost and availability. One of our recent research projects aims at designing a method for defining an initial system topology and capacity. This definition is not unique and can be further refined by considering system's costs. The three models: workload model, performance model and cost model together allow trade-off studies and making rational decisions about the final system selection in view of the present and the future technological and business uncertainties. 1 Introduction We consider here large client/server distributed transaction systems that are used by tens of thousands of users and run applications such as BAAN, PeopleSoft and

2 392 Applications 2000 WIT Press, of High-Performance ISBN Computers in Engineering VI CATIA (a CAD including : system). A system of this kind must satisfy several requirements (a) computational performance (e.g. response times and thruputs) (b) scalability (allow new users and new applications), (c) to be built and maintained at an acceptable cost. Since such systems are company strategic assets they have to be reliable with a high uptime to support business processes. It has been observed that systems supporting emergent organizations (dynamically evolving and constantly seeking new business modes of operations) cannot be stable and finished. The authors of a recent paper on this subject (1) stated "Systems should be under constant development, can never be fully specified and are subject to constant adjustment and adaptation". We can add that not only organizational needs are changing but also fast moving technology makes any static system increasingly cost/inefficient and eventually obsolete. It is not easy to build a system that is not only cost effective at the time when it is delivered but it evolves in time and has a long life with economic functionality. We, the IS community do not have at present a proven methodology for designing such systems but we have made first steps that have advanced our understanding of the issues, methods and tools that are needed for accomplishing the goal. In this paper we share some of our lessons learned and describe the related experience. 1.1 Technical challenges This paper also attempts to summarize an approach to designing and reengineering large client/server architectures and make specific suggestions for organizations which require such systems. A brief description of technical challenges is a good starting point. (a) Workload predictions are uncertain and inaccurate. Unless the planned system has been at least partly in production and needs only tuning there is no production workload and utilization data available. Vendor benchmark data may not correspond to the expected system use and typically have been obtained using a scaled down system compared to the target system. (b) Multiple vendors environment tends to have ill-defined responsibility for the total system integration, functionality and performance. (c) System size (the number of users and the total system's workload) is often beyond vendors'experience. (d) Vendors are tempted to provide simple linear extrapolations of performance and capacity data from small systems to larger configurations. This may lead to technical and cost disappointments. Correct extrapolations require complex performance models exhibiting highly nonlinear behavior. (e) We define system scalability as the ability to add system resources in order to handle additional workloads exceeding the workloads used in the design definition, without catastrophic performance degradation or unreasonable expenses. System scalability requires scalable hardware, software an communications. To put it differently the useful notion of scalability includes the entire system not any part of it.

3 2000 WIT Press, Applications of High-Performance ISBN Computers in Engineering VI 393 (f) We have to design a system for all applications running concurrently. It is not enough to consider separate applications even if their independent performances are satisfactory. 1.2 Performance and Scalability Factors Total Hardware/Software Fit Workload Characteristics Application Software: Interconnected COTS systems System software: OS, Schedulers, load balancing Hardware: Servers and networking Figure 1: Factors influencing performance Consider the question: what does determine system performance and scalability? The answer to the question is potentially everything. Any system component can become a bottleneck limiting the system performance. A well balance system does not have any resource device too close to saturation (100% utilization). Otherwise this device is likely to become a bottleneck. (a) a well balanced system for one workload may be out-of-balance for another. (b) but even a perfectly balanced system may be performing poorly for other reasons than device saturation. For example: excessive remote memory accesses, bad thread scheduling algorithms which defeat the memory - CPU affinity principle, etc. (c) While designing a system we tend to select high performance components, but this by itself does not warrant high performance for the integrated system (d) Unfortunately, it is also true that a single weak component that is on the transaction paths can degrade the end to end system performance and scalability. Summing up this section we conclude that a thorough capacity planning exercise which determines required resources such as processing power and memory has to be combined with performance studies to avoid pitfalls of having a system with plenty of resources and poor performance at the same time.

4 394 Applications of High-Performance Computers in Engineering VI 2 Approach The approach to designing and tuning large client/server distributed architectures that has proven to be cost-effective is building and exercising two types of models: workload models and performance models. Two kinds of modeling techniques are combined: analytical and simulation. Analytical modeling helps to evaluate performance of single components or simplified systems in order to identify performance concerns and find bottlenecks. Simulation modeling provides more detailed performance characteristics of the integrated global systems. 2.1 Formal problem definition Formally the system performance problem resembles a complex optimization task: given a workload and service level agreements (SLA) design a system topology and capacity that satisfies SLAs subject to cost constrains. The most desirable system may not be least expensive.since the owner may prefer to minimize the future risks (lack of scalability, redesign disruption, or inability to upgrade the system ) rather than minimize the initial system cost. There are two distinct cases which require different approaches: (a) there is already a production system that can be used to obtain data needed for modeling,and (b) there is no production data source,but some benchmarks are available and additional data can be obtained from simplified analytical models approximating the target system. 2.2 Modeling objectives and tools Our main simulation modeling tools have been Workbench and Strategizer from the SES (Scientific Engineering Systems ) Company. Using these tools together with analytical modeling we have been able to perform the required work: (a) capacity planning studies (b) performance evaluation ( response times and thruputs) (c) what-if studies for different architectural and workload assumptions, (d) identify and remove performance bottlenecks The objectives of our work have depended on the system development stage. The most preferable stage is the very beginning of the project before significant architectural decisions are made. At more mature stages of system development only tuning is an option. It is important to realize that performance models should be kept and exercised after the system is implemented. The idea is to modify the system continuously by modernizing its components rather than wait until the system is obsolete and has to be replaced. This modernization process can greatly benefit from the availability of performance models and help to avoid costly replacement and disrupting company business.

5 Applications of High-Performance Computers in Engineering VI Information required for modeling The first step needed for building a simulation performance model is gathering data that characterize the system including the expected workload. This is summarized in Fig 2. We need to know the types and mix of transactions, transaction flow through the system, arrival rates and utilization of resources such as CPU and memory. We may be interested in average performance or performance for the peak loads. The next type of information required is related to hardware and software that we intend to use. This includes the initial topology of the system (how devices are connected), speeds and capacities of servers and their architectures (e.g. uniprocessors or multiprocessors) and other hardware components such as disks and networks. In the software area we need to know some characteristics of the DBMS and operating systems (OS).OS are responsible for the CPU scheduling methods, load balancing and managing concurrent threads.all these factors impact system computational performance. Some new server architectures such as NCC-NUMA (cache coherent non-uniform.memory access) are very sensitive to these software system features. Given this information we can build within a couple of weeks a medium or a large system model. The greatest impediment to modeling is lack of reliable data. Another not surprisingly is the computational performance of the tools themselves. Good performance and scalability of tools is essential to their efficient and successful use. Workload characteristics types of transactions transaction flow arrival rates mix of transactions intensity: number of active users H/W and SAV characteristics Configuration topology Processor speeds/parallelism Disk parameters Network parameters. OS CPU scheduler Concurrent threads Load balancing algorithms Figure 2: Information needed for modeling

6 396 Applications of High-Performance Computers in Engineering VI 3 Examples To illustrate the usefulness of analytical modeling we provide an example of a model of a commercial UNIX SMP server. The following notation is used: m - The number of processors U- CPU utilization S - Service Time 8-U/m R -Response Time Approx. R =, The response time equation is developed using the well known Little relationships. This equation is approximate and does underestimate time. More accurate response time can be calculated from the Erland function (3). Clearly the system response time is a nonlinear function of the number of processors. Adding more processors to the configuration reaches a point of diminishing return (Fig. 3) m Figure 3: Response time as a function of m

7 Applications of High-Performance Computers in Engineering VI 397 Figure 4 shows that for heavy loads close to 1.00 multiprocessor performance suffers more drastically than the performance of the uniprocessor I i I I i Load 4 Conclusions Figure 4: Another View Adequate resource capacity is required for good performance but it does not guarantee it Therefore adding resources may be counterproductive, i.e. may not improve performance unless we understand performance problems and bottlenecks One of the hardest bottlenecks to find and eliminate are those related to software (system and application) References [1] Truex, D.P, Baskerville, R. & Klein, H. Growing systems in emergent organizations, CACM,Vol.42, No. 8, pp ,1999. [2] Menasce, D.A., Almeida, V. A. F. and Dowdy, L. W. Capacity planning and performance modeling, Prentice Hall, [3] Gunther, N. J. The Practical Performance Analyst, The McGraw Hill Series on Computer Communications, 1998.