Dell PowerEdge R910 SQL OLTP Virtualization Study Measuring Performance and Power Improvements of New Intel Xeon E7 Processors and Low-Voltage Memory A Dell Technical White Paper Dell, Inc. Waseem Raja Systems Performance Analysis (OCTO) John Beckett Systems Performance Analysis (OCTO)
PowerVault DL Backup to Disk Appliance: Key Factors Impacting Performance THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND. Dell, the DELL logo, and the DELL badge, PowerEdge, PowerConnect, EqualLogic, and PowerVault are trademarks of Dell Inc. Microsoft, Windows, Windows Server, SQL Server, Exchange, and Active Directory are either trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. VMware, VSphere, VCenter, and ESX are either trademarks or registered trademarks of VMware, Inc. Intel and Xeon are registered trademarks of Intel Corporation. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell Inc. disclaims any proprietary interest in trademarks and trade names other than its own. April 2011 Page ii
Contents Executive Summary... 2 Introduction... 2 Test Description... 3 Sample Environment Characterization... 4 Test Methodology... 6 Overview... 6 Data Collection Overview... 6 Test Results... 7 Performance Comparison Between Nehalem-EX and Westmere-EX... 7 Power Measurements at 100% and Idle... 7 OPM Per Watt (OPM/watt)... 9 Conclusion... 10 Appendix A: Additional Configuration Details... 11 Page 1
Executive Summary Dell s Systems Performance Analysis team was commissioned by Server Outbound Marketing to measure performance and performance/watt in a virtualized environment to compare the Intel Xeon X7560 (Nehalem-EX) processors on the Dell PowerEdge R910 against the new Intel Xeon E7-4870 (Westmere-EX) processors. The configuration was aimed at exploiting the strengths of the PowerEdge R910 by equipping the system with four processors, a 10Gb network connection to backend SAN, and sufficient memory to run the workload at 100% CPU utilization for both processor architectures tested. The testing indicates that the Xeon E7 processors offer significant performance gains within the same power envelope, while the addition of LV RDIMMs actually lowers system power draw at both idle and 100% CPU utilization. Key Findings: The with Intel Xeon E7-4870 processors provided 34% higher virtualized OLTP performance than the PowerEdge R910 with Intel Xeon X7560 processors. The R910 configuration with Intel Xeon E7-4870 processors obtained 727,049 orders per minute (OPM) in a virtualized environment, as compared to the R910 configuration with Intel Xeon X7560 processors, which scored 541,971 OPM. The with Intel Xeon E7-4870 processors obtained 50% higher performance/watt than the PowerEdge R910 with Intel Xeon X7560 processors. The R910 configuration with Intel Xeon E7-4870 processors obtained a performance/watt score of 825 in a virtualized environment, as compared to the R910 configuration with Intel Xeon X7560 processors, which achieved a performance/watt score of 551. The with Intel Xeon E7-4870 processors used 10% less overall power while running the virtualized OLTP workload than the PowerEdge R910 with Intel Xeon X7560 processors. The R910 configuration with Intel Xeon E7-4870 processors used an average of 881 watts, as compared to the R910 configuration with Intel Xeon X7560 processors which used an average of 984 watts. The with Intel Xeon E7-4870 processors used 2% less overall power at idle than the PowerEdge R910 with Intel Xeon X7560 processors. The R910 configuration with Intel Xeon E7-4870 processors used an average of 552 watts at idle, as compared to the R910 configuration with Intel Xeon X7560 processors which used an average of 563 watts. Introduction In order to assist customers to understand virtualization performance, this study was aimed primarily at characterizing specific application performance in a virtualized environment to characterize the Dell PowerEdge R910 with two generations of Intel Xeon processors. Many other virtualization benchmarks exist, such as VMmark and SPECvirt, but this study is intended to be a means to answer a common question which cannot be answered by other benchmarks; specifically, performance of a virtualized Microsoft SQL Server scenario on a robust quad-socket Dell server. Page 2
Considerations include: Comparing Microsoft SQL Server virtualized performance between the Dell PowerEdge R910 that launched in March of 2010 with Intel Nehalem-EX processors against the second version of the that launched in April 2011 equipped with Intel Westmere-EX processors. Comparing the power utilization of the with Nehalem-EX processors against the with Westmere-EX processors at idle and 100% CPU utilization levels. Comparing the performance/watt of the with Nehalem-EX processors against the with Westmere-EX processors. This study is not intended to be a comprehensive analysis of virtualization performance between Dell PowerEdge platforms, for only the R910 was chosen to characterize the relative gains in performance and power efficiency by comparing two generations of processors. In addition, new supported memory types and other elements of the R910 have been updated during the April 2011 launch of the Westmere-EX processor lineup, so this plays a role as well in affecting the comparison. Test Description DVD Store Version 2 (DS2) is a complete online e-commerce test application, with a back-end database component, a Web application layer, and driver programs. The goal in designing the database component as well as the mid-tier application was to utilize many advanced database features (e.g., transactions, stored procedures, triggers, and referential integrity) while keeping the database easy to install and understand. The DS2 workload may be used to test databases or as a stress tool for any purpose. The distribution includes code for Microsoft SQL Server, Oracle, and MySQL databases. Included in the release are data generation programs, shell scripts to build data for 10 MB, 1 GB, and 100 GB versions of the DVD Store; database build scripts and stored procedures, PHP Web pages; and a C# driver program. 1 Database workload sizes can be small, medium, or large, depending on environment setup. See Table 1 for details. Table 1 DVD Store Database Size Description Database Orders Size Products Customers Small 10 MB 20,000 1,000/month 10,000 Medium 1 GB 2,000,000 100,000/month 100,000 Large 100 GB 200,000,000 10,000,000/month 1,000,000 1 Test Description information from Dell Enterprise Technology Center website: http://www.delltechcenter.com/page/dvd+store+readme Page 3
Sample Environment Characterization To measure consolidated VM infrastructure host performance in a simulated e-commerce scenario, a full end-to-end solution was implemented. This environment consists of a physical server (host) running VMware ESX 4.1. This is referred to as the system under test (SUT) in the diagram. In addition to the SUT, the environment contains an environment controller system and a workload driver system. The environment controller is running Windows Server 2008 SP1. The intent of this system is to manage the VMware vcenter /vsphere client and act as a logging device for the power analyzer. The workload driver system is running Windows Server 2008 R2 Enterprise. This system is used to create simultaneous SQL user workload instances on the VMs that are being tested. In addition to these three physical systems, the environment contains a 10Gb Ethernet PowerConnect 8024F switch, four EqualLogic PS6010 SAN, and a Yokogawa WT210 power analyzer. The PowerConnect switch enables communications between the VMs and controller systems (workload driver and environment controller), and also between the system under test (SUT) and the iscsi-based EqualLogic storage backend. The power analyzer is connected to the environment controller system through a serial cable. The power analyzer takes one-second snapshots of all power readings and the controller logs this data during both the idle and active portions of the benchmark runs. Figure 1 Test Environment Page 4
Virtualized SQL SUT Configurations Configurations used in the test environment are shown in Table 2 and Table 3. Table 2 SUT Configurations Platform R910 Nehalem-EX R910 Westmere-EX Operating System VMware ESX 4.1 Processor Model 4 x Intel Xeon X7560 4 x Intel Xeon E7-4870 Processor Frequency 2267 MHz 2400 MHz Memory Details 32 x 4GB DR 1.5V 32 x 8GB QR 1.35V Memory Frequency 1066 MHz 1066 MHz Internal Storage 2x 2.5 146GB 15k RPM SAS (RAID 1) BIOS Power Management Setting Maximum Performance Physical Cores 32 40 Logical Processors 64 80 # 2-vCPU VMs 32 40 Table 3 Supporting Infrastructure Configurations Environment Controller System PowerEdge 1950 Windows Server 2008 Enterprise SP1 PowerEdge R905 SQL Workload Driver System Windows Server 2008 R2 Enterprise 2 x Intel Xeon processor E5410, 2.33 GHz 4 x AMD Opteron 8435, 2.60 GHz Memory: 16 GB RAM Memory: 96 GB RAM Disk Configuration: 2x 73 GB SAS (RAID 1) Disk Configuration: 2 x 146 GB SAS (RAID 1) PowerConnect 8024F Networking Utilizing 10 Gigabit connection rate 1 network port used per EqualLogic SAN (4 total) Monitors SUT only Power Analyzer Measures all power data during active portion of the benchmark Measurements recorded during official run: 1 network port used for SUT Average watts during active period 1 port used as management port (via vsphere) Average watts during idle period Jumbo frames enabled on switch and on SANs Page 5
Test Methodology Overview The following methodology was used in the testing procedure: Determined overall system performance at an equal CPU utilization (100%) across all logical processors with predetermined configurations and a methodology that produces consistent and repeatable data. Exercised all enabled VMs for a 10 minute run time. Recorded Performance metric, SUT Performance counters, and Power Measurements. Note: The metric recorded is OPM (orders per minute a measure of the number of orders processed during the benchmark runtime). Data Collection Overview The following steps were used in collecting data: 1. Determined the number of VMs to enable during testing, keeping in mind that the VM count should be half the number of physical processor cores available. (Each VM is a two vcpu VM). 2. Built a medium-size SQL database on each VM using DVD Store scripts. 3. Backed up each SQL database to a restore file, allowing the database to be restored from a pristine state, and then deleted all VM databases. 4. Restored all databases from the backup restore file. 5. Rebooted all VMs. 6. Performed a warm-up run to prepare databases for the official run, allowing each database to warm up individually and allowing a 10-minute idle period after warm up to allow VMs to return to a steady state. 7. Performed the official run starting all VMs simultaneously using a script. The power analyzer began measuring the system power at the start of the official run and an ESX host was triggered to start esxtop batch session during the official run. (Note: The VMware esxtop tool gathered system-related performance counters for profiling CPU usage during official run.) 8. Adjusted workload by adding additional VMs after each run to lower or raise CPU usage and then restarted official run to get an official measurement as close to 100% CPU utilization on each. 9. Checked the official run data for performance stability, collecting data if stable and restarting the process if high variation was present. Data collected during the official run included: Average watts used during the official run (power metric) Average of the total esxtop CPU counters during the official run (CPU percentage) VM throughput in OPM (Orders per Minute) (metric used in this study is an aggregate OPM measurement of all enabled VMs and is represented as performance in the test results) Page 6
Performance (OPM) at 100% Dell PowerEdge R910 SQL OLTP Virtualization Study Test Results Performance Comparison Between Nehalem-EX and Westmere-EX Quad-socket performance measurements on the show the increase in performance between the two generations of Intel processors. The Intel Xeon X7560 2.26GHz eightcore processor in the PowerEdge R910 was compared to the Intel Xeon E7-4870 2.4 GHz ten-core processor. The addition of 16 additional logical processors in a Hyperthreaded 4P configuration using the Intel E7-4870 processors necessitated adding 8 more VMs to the configuration in order to saturate the system s processing capabilities while running this benchmark. Additionally, the memory was doubled from 128 GB with the Intel X7560 to 256GB with the Intel E7-4870 to provide sufficient memory to run all VMs. The PowerEdge R910 configuration using the Intel Xeon E7-4870 processor provided 34% more performance, as recorded in aggregate orders per minute (OPM) across all VMs, as compared to the R910 configuration equipped with the Intel Xeon X7560 processor. Virtualized SQL Performance Orders per Minute at 100% (Higher Is Better) 541971 727049 Intel Xeon X7560 Intel Xeon E7-4870 Figure 2 Virtualized SQL Performance Power Measurements at 100% and Idle Power consumption of the SUT during the active states of the benchmark was also measured. The power analyzer used reports 1-second snapshots of the average power being consumed at any given point of the measured sample. The average of these snapshots over the active period of the benchmark was used as the basis for the average power consumed. The R910 configuration using Intel Xeon E7-4870 processors used 10% less power at 100% utilization than the R910 configuration using Intel Xeon X7560 processors. The R910 with E7-4870 processors used an average of 881 watts, and the R910 with X7560 processors used an average of 984 watts. Some of the power delta can be attributed to the capability of the E7-4870 configuration to use Low Voltage DIMMs, even though the memory configuration was twice as large as the X7560 configuration. Page 7
Average Power (Watts) Average Power (Watts) Dell PowerEdge R910 SQL OLTP Virtualization Study At idle, the configuration using Intel Xeon E7-4870 processors used an average of 2% less power than the configuration using Intel Xeon X7560 processors. The configuration with E7-4870 processors used an average of 552 Watts at idle, while the configuration with X7560 processors used an average of 563W. Even lower idle power readings would be expected if the BIOS Setup Program Power Management policy was modified to Dell Active Power Controller from the Maximum Performance mode used in this study. Virtualized SQL Power Utilization Average Power at 100% (Lower Is Better) 984 881 Intel Xeon X7560 Intel Xeon E7-4870 Figure 3 Average Power at 100% (Lower Is Better) Virtualized SQL Power Utilization Average Power at Idle (Lower Is Better) 563 552 Intel Xeon X7560 Intel Xeon E7-4870 Figure 4 Average Power at Idle (Lower is Better) Page 8
Performance/Watt at 100% Dell PowerEdge R910 SQL OLTP Virtualization Study OPM Per Watt (OPM/watt) The overall benchmark score (OPM) is divided by the average power consumed to generate the OPM per watt (OPM/watt) metric shown in Figure 5. This value is considered to be the measure of performance per increment of power used. Higher is better in this metric. Formula: Aggregate Performance (OPM) / Average Power Consumed = OPM per Watt (OPM/watt) In this comparison, the PowerEdge R910 configuration with Intel Xeon E7-4870 processors achieved the highest result with a score of 825 OPM/watt a difference of 50% over the PowerEdge R910 configuration with Intel Xeon X7560 processors. The PowerEdge R910 configuration with X7560 processors scored 551 OPM/watt. Virtualized SQL Performance/Watt OPM / Average Watts at 100% (Higher Is Better) 551 825 Intel Xeon X7560 Intel Xeon E7-4870 Figure 5 Performance Per Watt (Higher is Better) Page 9
Conclusion The goal of this study was to characterize the performance and power impacts of the transition of the platform from Nehalem-EX support at initial platform launch in March 2010 to Westmere-EX support in April 2011 using a virtualized database workload. It should be noted that actual customer environments are unlikely to be running at 100% processor utilization levels, but the relative scaling achieved with the processor transition to Westmere-EX on the R910 platform in this test should still be indicative of the merits of the new incarnation of Intel Xeon processors on the Dell PowerEdge R910. Based on the data contained in this study, the PowerEdge R910 with E7-4870 processors provides considerable performance and performance/watt advantages while reducing the overall system power at idle and at the 100% utilization level over the previous generation Intel Xeon processors in the PowerEdge R910. The PowerEdge R910 with Intel Xeon E7-4870 processors would be the platform of choice for maximum performance and expandability with 64 DIMM slots and 10 PCIe slots, as well as the ability to use low-voltage DIMMs to help reduce overall system power consumption. Page 10
Appendix A: Additional Configuration Details Configuration as Tested Platform PowerEdge R910 PowerEdge R910 Processor Type Intel Xeon X7560 Intel Xeon E7-4870 # Processors Populated 4 4 Clock Speed (GHz) 2.26 2.40 Total Cores/System 32 40 Total Logical Processors 64 80 L3 Cache (MB) 24 30 Memory 128 GB (32 x 4GB 2R) 256 GB (32 x 8GB 4R LV) # Memory Risers 8 8 # HDD/Size 2 x 146GB 15K 2.5" 6GBbps SAS 2 x 146GB 15K 2.5" 6GBbps SAS RAID Type RAID 1 RAID 1 HBA PERC H700 PERC H700 BIOS Version 1.2.2 2.1.0 idrac Version 1.57.12 1.70.20 Total NIC ports used 1 x 10Gb, 1 x 1Gb 1 x 10Gb, 1 x 1Gb # 10GB NICs 1 1 Type of 10Gb NICs 10Gb Intel XF PCI-E FC single port 10Gb Intel XF PCI-E FC single port Total PCIe Slots Used 1 1 Power Supply Qty/Rating 2 x 1100W 2 x 1100W Voltage Used for Testing 208v 208v Page 11
Virtualized Environment Details VMware ESX Version 4.1.0 4.1.0 OS Version Installed/VM SQL Version Installed per VM Windows 2008 Enterprise R2 x64 Microsoft SQL 2008 R2 #Virtual Machines 32 40 #Logical Processors per VM 2 2 EqualLogic Configuration EqualLogic Systems in Use 4 4 EqualLogic System Type 4 x EqualLogic PS6010 (RAID-10) Total Storage per EQL box 3.66TB 3.66TB Windows 2008 Enterprise R2 x64 Microsoft SQL 2008 R2 4 x EqualLogic PS6010 (RAID-10) Size and # HDDs/EqualLogic 16 x 600GB 16 x 600GB R910 BIOS settings used Turbo Enabled Enabled Prefetchers Both Enabled Both Enabled C-states Enabled Enabled C1E Enabled Enabled Logical Processors Enabled Enabled Node Interleave Disabled Disabled Power Management Maximum Performance Maximum Performance Page 12