VMware and Xen Hypervisor Performance Comparisons in Thick and Thin Provisioned Environments

Transcription

1 VMware and Hypervisor Performance Comparisons in Thick and Thin Provisioned Environments Devanathan Nandhagopal, Nithin Mohan, Saimanojkumaar Ravichandran, Shilp Malpani A capstone paper submitted as partial fulfillment of the requirements for the degree of Masters in Interdisciplinary Telecommunications at the University of Colorado, Boulder, 4 May Project directed by Mark Dehus. 1 Introduction Traditional x86 standalone server systems have a very serious caveat of under-utilization, due to which datacenters incur heavy costs. Research findings report that, on an average only 10 to 15% capacity of a server is utilized. Globally, businesses are dependent on applications that run on these commodity servers, and as the capacity tends to scale out, there is a direct linear implication on costs associated [1] [2]. A transition from traditional servers to consolidation of these servers by using virtualization can be a strategic decision for datacenters. And with increasing dependence of business critical decisions on applications, it becomes imperative to justify these decisions as the decisions relating to underlying platforms that support these applications can have a direct impact on business. Costs are an important factor, but performance is another critical factor that has to be taken into account. And, as with any decisions pertaining to technology, there would be trade-offs associated [3]. Consolidating server footprints by leveraging virtualization technology has been commonly practiced over the past several years. This has eventually helped enterprises to reduce costs like: floor space, power, cooling, hardware, and administration. Virtualization enables the better utilization of server resources by allowing many virtual instances of machines to be created on a server. This technology has been critical to driving IT costs down by enabling applications to be supported on shared resource of servers. Despite the benefits, virtualization presents challenges like capacity planning and workload management [4]. A major caveat here is that the use cases for adopters of virtualization may vary considerably, and with so many different vendors providing virtualization products it becomes difficult to have a standard industry conformed template in place which would guide the adopters of virtualization technology.

2 2 Statement of the Research Virtualization is making rapid progress in its foray into datacenters and enterprise IT systems. On one side, while virtualization is helping enterprises cut down on IT costs, it is not clear if virtualization is a viable solution for all applications. The important components critical to performance of an application are: CPU, Memory, Network, and Disk IO capabilities [5] [6]. In this paper, our focus is to characterize the performance of VMware and hypervisors when a disk intensive application is run on a single virtual machine hosted on each of the hypervisors. We benchmark the performance by using a native system for comparison with VMware and under the same system configuration for all the tests performed. There are many factors that may induce performance overhead; we are more specifically focused on determining the disk IO performance overhead due to storage provisioning. As with deployment of any technology, we believe that the storage provisioning feature of virtualization technology requires a careful evaluation of the trade-offs associated. However, quantifying performance difference of applications in terms of application-level metrics is a non-trivial task, because the use cases can vary between adopters significantly [4]. We aim to evaluate the performance overhead incurred by and VMware under the same consolidation configuration and workload conditions with reference to disk IO for thick and thin provisioned environments while running a disk intensive application. 3 Definitions The following are the definitions for the technical terms used in our research: 3.1. Virtualization: Virtualization is a method of running multiple independent virtual operating systems on a single physical computer. It is a way of maximizing physical resources to maximize the investment in hardware [14] Thin Provisioning: Thin provisioning is a storage feature in virtualization that provides a logical method of allocation storage resources [15] Capacity: The total storage space present in a single storage device or a pool of storage devices [11] Committed: Storage space that is reserved for a virtual machine is referred to as committed storage [11] Used: Storage space that has been utilized by a virtual machine Overcommitted: When the storage capacity in data store is less than the total storage capacity reserved for virtual machines [11] Percent Overcommitted: The extent by which the storage capacity of virtual machines has exceeded the storage capacity of virtual machines [7] [11] Thick Provisioning: Thick provisioning is similar to allocating fixed storage to virtual devices [16] Hypervisor: A software system that enables the x86 server hardware to be utilized for virtualization [17] Virtualization Overhead: The lag induced due to virtualization which results in the virtualized system to be slower than a native system of the same configuration [18].

3 4 Scope Thin provisioning is an important storage feature that can help an enterprise better manage its storage hardware costs. The motivation for this paper germinated out of the imperative importance to understand the performance degradation (if any) that can arise due to utilizing the provisioning feature that are available in virtualization products these days. There are many virtualization products in the market, and evaluation of every one of those products is not feasible given the time frame and scope. The decision to choose VMware hypervisor in the quantitative performance comparison was because of its market leader position. Microsoft s Hyper-V and Citrix s are the next two competitors in the virtualization product market, and we decided to choose because of its open source standard [13]. The main objective of this study is to benchmark the performance of 32bit SQL 2005 database server hosted on each virtual machine running on and VMware. The benchmark test measures the performance of database application on virtual machine primarily with respect to disk IO. We evaluate the performance of two hypervisors: and VMware s, when a SQL 2005 database server is under load in a Windows 2003 guest Operating System [8]. We have used a webserver as another workload hosted on each of the virtual machines, and have run it simultaneously with the database workload. One reason for running the webserver workload simultaneously was to ensure that systems resources like CPU and memory are well utilized (webserver places negligible stress on disk IO), so that the database workload has another workload to contend with (at least for CPU and memory resources). 5 Importance and prior work Similar to the under-utilization of other resources with traditional server deployments, storage is another resource that is seldom completely utilized. Enterprise Strategy Group in their survey of 20 customers published some insightful findings with reference to storage underutilization. The key points of their survey finding are: Among the surveyed customers, 55 percent had up to 50% storage that was unused. Despite 45% of customers having unused storage that was already bought, they were required to buy new storage because the unused storage was stranded with physical servers [9]. Larger pool of shared storage can be provided by thin provisioning when compared to traditional storage methods, thus increasing the efficiency of storage management. This has been depicted in Figure 1. [9]

4 Figure 1: Initial Capacity Requirements In thin provisioning, storage costs are saved because an enterprise need not deploy additional storage for storage capacity that has been allocated but unused. This has been depicted in Figure 2 [9]. Figure 2: Capacity Requirements Qian et al. have published a work with reference to an out-of-band virtualization system called ThinStore. It has been reported in their research that they found storage utilization using thin provisioning improved by as much as 88.7% [12]. Another provisioning technique that is similar to the thin provisioning feature of storage is the resource provisioning in the Infrastructure-as-a-Service (IaaS) cloud computing model. In an IaaS model, clients can request to use a part of that resource. Although cost effective, the performance is found to fall by a factor of five during the I/O intensive applications [10]. While this skepticism regarding the performance of applications in the cloud is a valid argument, the cost benefits of provisioning cannot be ignored. Understanding how provisioning (in our case, storage provisioning) can alter the performance of the applications, is a significant factor to consider while making any trade-off decisions.

5 6 Methodology The methodology used for our research is quantitative comparison and performance characterization using the quantitative analysis. We are using test experiments to quantitatively compare the performances of a disk intensive application running in each virtual machine hosted on VMware and hypervisors in thin and thick provisioned environments. The test set up for the comparison comprised of: a native system, a virtual machine in with thick disk, a virtual machine in with thin disk, a virtual machine in VMware with thick disk, a virtual machine in VMware with thin disk. All the virtual machines were configured with Windows Server 2003 Enterprise Edition 32bit operating system and SQL server A webserver was created using Windows Server 2003 s IIS capabilities. SQLIO was used as a database testing tool to perform the benchmark tests and webserver stress tool was used to stress the webserver. Performance Tuning Corporation has published a work for their performance test of Oracle 10g on Hyper-V virtual machine. Since their test was of similar nature in terms of benchmarking the virtual system with a native system, we have used some of the test methodologies from their paper as a model for our research [6]. Sufficient cool down and warm up periods were provided during each of the test runs. Each test instance lasted 4 hours, and consisted of 2 hours of warm up and at least 4 hours of cool down period. The summary of platforms used for the tests are provided in Table 1. Table 1: Hardware Platform Summary Table Environment Native VMware VMware Thick Thin Thick Thin Machine Dell R805 Dell R805 Dell R805 Dell R805 Dell R805 Physical 32 GB 32 GB 32 GB 32 GB 32 GB Memory Hard Disk 120 GB 120 GB 120 GB 120 GB 120 GB CPUs 2x Quad- Core AMD Opteron(tm) x Quad- Core AMD Opteron(tm) x Quad- Core AMD Opteron(tm) x Quad- Core AMD Opteron(tm) x Quad- Core AMD Opteron(tm) 2350 Operating System (32bit) Workloads Database name Database test tool Webserver test tool Windows Server 2003 Database server, Web server Windows Server 2003 Database server, Web server Windows Server 2003 Database server, Web server Windows Server 2003 Database server, Web server Windows Server 2003 Database server, Web server SQL server SQL server SQL server SQL server SQL server SQLIO SQLIO SQLIO SQLIO SQLIO Webserver stress tool Webserver stress tool Webserver stress tool Webserver stress tool Webserver stress tool

6 IOs/sec 7 Test Results and Analysis Table 2 summarizes disk IO results in thick provisioning write operations for VMware and Citrix. Table 2: IOs/sec Thick Provisioning Write Operation Results Write Native Thick Thick Threads IOs/sec IOs/sec IOs/sec Native Number of threads Figure 1: IOs/sec Thin Provisioning Write Operation The above results show a tabular and pictorial comparison between the performance differences of and with reference to the native system for thick provisioning write operations. The results are respectively placed at 98.78%, 99.26%, and 96.44% of native for threads 16, 32 and 64. For, the results for the respective threads are placed at 42.32%, 40.81% and 41.33% of native. Table 3 summarizes disk IO results in thin provisioning write operations for VMware and Citrix.

7 Ios/sec Table 3: IOs/sec Thin Provisioning Write Operation Results Write Native Thin Thin Threads IOs/sec IOs/sec IOs/sec Number of threads Native Figure 2: IOs/sec Thin Provisioning Write Operation The above results show a tabular and pictorial comparison between the performance differences of and with reference to the native system for thin provisioning write operations. The results are respectively placed at 86.34%, 87.85%, and 86.7% of native for threads 16, 32 and 64. For, the results for the respective threads are placed at 41.21%, 40.59% and 40.69% of native. Table 4 summarizes disk IO results in thick provisioning read operations for VMware and Citrix. Read Table 4: IOs/sec Thick Provisioning Read Operation Results Native Thick Thick Threads IOs/sec IOs/sec IOs/sec

8 IOs/sec Ios/sec Native Number of threads Figure 3: IOs/sec Thick Provisioning Read Operation The above results show a tabular and pictorial comparison between the performance differences of and with reference to the native system for thin provisioning write operations. The results are respectively placed at 93.56%, 93.69%, and 97.06% of native for threads 16, 32 and 64. For, the results for the respective threads are placed at 42.69%, 44.12% and 46.25% of native. Table 5 summarizes disk IO results in thin provisioning read operations for VMware and Citrix. Table 5: IOs/sec Thin Provisioning Read Operation Results Native Thin Thin Threads IOs/sec IOs/sec IOs/sec Read Native Number of threads Figure 4: IOs/sec Thin Provisioning Read Operation

9 The above results show a tabular and pictorial comparison between the performance differences of and with reference to the native system for thin provisioning write operations. The results are respectively placed at 89.5%, 91.1%, and 93.75% of native for threads 16, 32 and 64. For, the results for the respective threads are placed at 43.27%, 44.71% and 46.48% of native. The web server results for the platforms tested are shown in Table 10. Native Thin Thick Thin Thick Table 10: Web Server Results Avg. Click Response URL No. Clicks Errors Errors [%] Time [ms] Hits URL URL URL URL URL URL URL URL URL URL From the analysis of web server results, it can be inferred that the performance degradation is significant for when a CPU intensive workload (web server) is tiered with another workload (in our case, database server). For the four hours test duration, the best average click response time is given by native at 1465 (ms), and it also leads in the number of clicks during the four hour period at shows a stable performance and its best performance numbers are at 1661 (ms) average click response time and clicks during the four hour test run. has an error percentage of up to during its thin disk test run. And its best performance is at a modest 3327 (ms) average click response time during its thick disk test run.

10 8 Conclusion In this paper, we have presented a quantitative analysis representing the impact of storage provisioning on a disk intensive application. For each of thick and thin platform, we have presented the analysis between two virtualization products: VMware s and Citrix s. For the disk IO metric, the performance of VMware for thick disk is found to be as close as 99.26% of native for write operations, whereas for read operations the closest result was 97.06% of native. The best performance of VMware for thin disk was found to be 87.85% of native for write operations, whereas for read operations the best performance was 93.75% of native. In contrast, the best performance of for thick disk was found to be 42.32% of native for write operations, and 46.25% of native of read operations. For thin disk, the best performance of was found to be 41.21% of native for write operations, and 46.48% for read operations. The results for the web server application that was run simultaneously with SQL server are as follows: native system has the least average click response time of 1465 (ms), followed by thick disk at 1661 (ms), thin disk at 1783 (ms), thick disk at 3375 (ms), and thin disk at 9585 (ms).

11 References: [1] Forrester Consulting, The business value of virtualization, July, [2] P. Barham, B. Dragovic, K. Fraser et al., and the art of virtualization, University of Cambridge Computer Laboratory, [3] Citrix Desktop, TCO savings with desktop virtualization, Citrix. [4] P. Padala, X. Zhu, Z. Wang et al., Performance evaluation of virtualization technologies for server consolidation, HP Laboratories, HPL R1, September 30, [5] H. Raj and K. Schwan, High performance and scalable I/O virtualization via selfvirtualized devices, High Performance Distributed Computing, June 25-29, [6] G. W. Parker, Oracle database 10g performance on windows server 2008 Hyper-V virtual machine, Performance Tuning Corporation, October [7] VMware, Performance study of VMware vstorage thin provisioning, VMware vsphere 4.0, November 11, [8] P.M. Perera, and C. Keppitiyagama, A performance comparison of hypervisors, The International Conference on Advances in ICT for Emerging Regions, pp. 120, September 2, [9] T. Asaro, Thin provisioning, Enterprise Strategy Group, October [10] M.F. Rehman and M. F. Sakr, Initial findings for provisioning variation in cloud computing, 2nd IEEE International Conference on Cloud Computing Technology and Science, [11] VMware, Dynamic storage provisioning, VMware. [12] K. Qian, L. Yi, J. Shu, ThinStore: out-of-band virtualization with thin provisioning, Sixth IEEE International Conference on Networking, Architecture, and Storage, [13] Dornan, VMware rivals closing virtualization gap: Gartner, InformationWeek, July 8, [14] G. Ou, "Introduction to server virtualization," Tech Republic, May 22, [15] Thin provisioning, Tech Target, November, [16] Thick provisioning, IBM, January 12, [17] Virtualization Guide, Red Hat Inc., [18] Measuring performance of applications on virtualized systems under test, Intel, October 2, 2008.