EMC VPLEX VIRTUAL EDITION: USE CASES AND PERFORMANCE PLANNING

Transcription

1 White Paper EMC VPLEX VIRTUAL EDITION: USE CASES AND PERFORMANCE PLANNING Abstract This white paper provides an overview of VPLEX/VE use cases and performance characteristics

2 Copyright 2014 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are the property of their respective owners. Part Number H For more information: Explore and compare the latest VPLEX products in the EMC Store 2

3 Table of Contents Executive summary... 4 Document scope and limitations... 4 Audience... 4 Introduction... 5 VPLEX/VE Overview... 6 Architecture and Deployment... 7 VPLEX Witness... 8 VPLEX/VE vapp Architecture... 9 VPLEX/VE STORAGE CONCEPTS... 9 VPLEX/VE Use Cases Block Storage High Availability Application HA Migration Dynamic Workload Load Balancing across Sites Immediate Cross-Site vmotion VPLEX/VE Performance Planning Summary Background Hardware Configuration Key Performance Indicators (KPIs) Simulated Application Profiles I/O Profile of Simulated Applications Notes on Selecting the Application Profile Testing Methodology Type of Measurements Data Charts Conclusion VPLEX References Supplemental ware References

4 Executive summary EMC VPLEX/VE delivers software-defined data mobility and availability within and across sites for ware based infrastructures. VPLEX/VE is a unique virtual storage technology that enables mission critical applications to remain up and running during any of a variety of planned and unplanned downtime scenarios. VPLEX/VE permits fluid, non-disruptive data movement, taking technologies that were built assuming a single block storage instance and enabling them to function across disparate iscsi arrays and across distance. EMC VPLEX/VE leverages the proven advantages of the EMC VPLEX family and puts that power in a customer installable, COTS hardware based configuration that is managed using standard ware workflows and interfaces. Document scope and limitations The use cases and performance overview discussed in this white paper are applicable to the VPLEX/VE 2.1 release. The use cases and performance details provided in this white paper are only applicable to environments consisting of the following elements: ESXi 5.1, 5.5 ware Enterprise+ Licensing EMC VPLEX/VE EMC VNXe 3100 and 3200 series iscsi arrays 8-32 ESXi hosts [See the EMC VPLEX/VE Product Guide available at for detailed host requirements] Please consult with EMC sales and support representatives if there is uncertainty as to the applicability of this information for specific VPLEX/VE environments. Audience This white paper is intended for technology architects, storage administrators, and ware system administrators who use or are planning to use EMC VPLEX/VE technology. It is assumed that the reader is familiar with ware and iscsi storage array technologies. 4

5 Introduction Data center infrastructure is undergoing a massive shift. Virtualization in the data center has had a profound impact on customer expectations of flexibility and agility. Especially as customers get to 70+% virtualized, they have the potential to realize tremendous operational savings by consolidating management in their virtualization framework. In this state, customers typically do not want to deploy physical appliances and want everything handled from their virtualization context. Similar changes in networking and storage have meant that the basic infrastructure is now completely in software running on generic hardware. This is the software defined data center. VPLEX has been no stranger to this conversation. Especially given the very strong affinity of VPLEX to ware use-cases, customers have been asking us for a software only version of VPLEX. This is precisely what is being delivered with VPLEX/VE 2.1 software release. This white paper reviews the following topics: VPLEX/VE Technology Overview VPLEX/VE Architecture and Deployment VPLEX/VE Use Cases VPLEX/VE Performance Planning 5

6 VPLEX/VE Overview EMC VPLEX /VE represents next-generation architecture for data mobility and information access. This architecture is based on EMC s 20+ years of expertise in designing; implementing and perfecting enterprise class intelligent cache and distributed data protection solutions. VPLEX/VE for ware vsphere provides simplified block storage management and data mobility for ware vsphere clusters. VPLEX/VE is exclusively managed using the ware vsphere Web Client through a plug-in. The VPLEX/VE virtual appliance runs at each site or fault domain within an ESXi Cluster or ware Metro Storage Cluster (vmsc). VPLEX/VE enables automatic data sharing and workload balancing along with HA and non-disruptive application mobility across sites. These benefits are achieved by providing a highly available shared block storage device to the ESX cluster. By default, the shared block storage device provided by VPLEX/VE will be formatted with FS and then consumed as a Distributed Datastore by the ESX cluster. By eliminating the need to move between datastores to vmotion from one ESXi host to another, otion and HA are now enabled across separate fault domains or sites with up to 10ms RTT latency between them. Site-1 ware Metro Storage Cluster Site-2 vsms vdirector ESXi VPLEX/VE Site-1 vapp vdirector ESXi vdirector ESXi VPLEX/VE Distributed Datastore vdirector ESXi ESXi vsms vdirector ESXi vdirector ESXi VPLEX/VE Site-2 vapp vdirector ESXi vdirector ESXi ESXi iscsi IP SAN iscsi VNXe 3150 / 3200 VNXe 3150 / 3200 Figure 1: VPLEX/VE within a 2 Site vsphere Cluster VPLEX/VE is a two site (fault domain) solution. Comparing the hardware version of VPLEX Metro to VPLEX/VE we can see that the VPLEX directors have been converted into vdirectors. The VPLEX/VE vdirector configuration is called a meaning deployment of four vdirectors at each site. From a configuration standpoint, that is analogous to two VPLEX engines on each site of a VPLEX Metro. 6

7 Architecture and Deployment VPLEX/VE is deployed as vapp into an ESXi cluster managed by a single vsphere Server instance. ware vsphere Web Client and Desktop Client do not have a concept of physical sites or fault domains within a cluster. One suggestion is to use host names that indicate which Site they are located in. It is incumbent on the ware administrator to understand the physical location of each ESXi host in the cluster during VPLEX/VE deployment. The VPLEX/VE installation wizard will organize the vdirectors and vmanagement Server as follows: Figure 2: VPLEX/VE vdirector and vmanagement Server Deployment Within the ESXi cluster: ESXi hosts are organized into 2 Sites, each consisting of 4 or more hosts 1 VPLEX/VE vapp is deployed per Site Each VPLEX/VE Site consists of 4 Director virtual machines (vdirectors) Each VPLEX/VE Site contains 1 Management Server virtual machine 7

8 vcenter Server ESXi Cluster Site-1 ESXi Hosts Site-2 ESXi Hosts Site-1 vapp Site-1 Virtual Directors Site-1 Virtual Mgmt Server Site-2 vapp Site-2 Virtual Directors Site-2 Virtual Mgmt Server Figure 3: VPLEX/VE within vsphere Web Client VPLEX Witness VPLEX/VE provides an intelligent quorum mechanism to allow VPLEX/VE to make a distinction between a site loss and a wan partition. This mechanism is known as VPLEX Witness. It consists of a single virtual machine that is deployed within a 3 rd fault domain. VPLEX Witness communicates with Site 1 and Site 2 via independent IP networks. Figure 4: VPLEX Witness Deployed in a 3rd Fault Domain Technical Note: The VPLEX Witness virtual machine must reside on an ESXi host outside of the ware cluster that contains the two VPLEX/VE Sites. The design goal is to run the VPLEX Witness in an isolated 3 rd fault domain up to 1000ms (1 second) RTT from Site1 and Site 2. 8

9 VPLEX/VE allows the ware administrator to select which Site will continue I/O operations during a dual wan-partition event. Setting the affinity to align with Site Bias for the virtual machines that comprise an application enables them to non-disruptively ride through dual wan partition events. For Site failures, VPLEX/VE automatically guides the surviving/healthy site to continue I/O regardless of the Site Bias setting. No matter which site fails, IO carries on at the surviving site. VPLEX/VE vapp Architecture Each VPLEX/VE Site consists of 4 vdirectors running on independent ESXi hosts and 1 Management Server (vsms). One vdirector per ESXi host ensures the VPLEX/VE vapp can survive at least 2 ESXi host failures before a Site is lost. Site-1 vcenter Server ware ESXi Cluster One Logical Group of Hosts Two Physical Locations / Fault Domains VPLEX vapp 2 Site-2 VPLEX vapp 1 vsms 4 - vdirector s vsms 4 - vdirector s ESXi-1 ESXi-2 ESXi-3 ESXi-4 ESXi-n ESXi-5 ESXi-6 ESXi-7 ESXi-8 ESXi-m VPLEX/VE STORAGE CONCEPTS Figure 5: vdirector Deployment across ESXi Hosts Without VPLEX/VE ESXi hosts in a ware cluster consume shared block storage devices from physical storage arrays that are within the same physical site. The ESXi hosts and the virtual machines at one site do not typically consume the iscsi storage or datastores from another site. When attempting to vmotion a virtual machine across sites, the block storage device containing the datastore and corresponding files that make up the virtual machine would not be available to the ESXi host at the second site. This creates the need for Storage vmotion, to move the data across sites prior to using vmotion to move the virtual machine. VPLEX/VE creates Distributed Datastores with backing physical block storage devices at both sites. VPLEX/VE distributed cache coherence feature ensures that the data is consistent and available in an active-active configuration at both sites. VPLEX/VE enables the creation of Distribute Datastores which are backed by distributed virtual volumes within VPLEX/VE. These Distributed Datastores are accessible to all ESXi hosts participating in the within the vsphere cluster. Virtual machines provisioned with 9

10 Distributed Datastores enable vmotion across sites with no limitation due to local shared physical storage. Figure 6: VPLEX/VE Storage I/O Path 10

11 VPLEX/VE Use Cases VPLEX/VE allows ware admins to combine ESXi infrastructure at disparate data centers or fault domains into a single pool of resources. Storage, CPU, Memory, and Network at two locations now becomes a single, larger, more resilient pool of IT infrastructure. Block Storage High Availability The AccessAnywhere feature of VPLEX/VE ensures cache-coherent active-active access to datastores across VPLEX/VE sites. The features of VPLEX/VE increase resiliency in the event of a site outage. The data is protected in the event of disasters or failure of components within data centers. With VPLEX/VE, the applications can withstand failures of storage arrays and site components. The VPLEX/VE components are not disrupted by a sequential failure of up to two vdirectors in a site. The failure of a VPLEX/VE site or a dual WAN partition event is tolerated to the extent that the site configured with site bias continues to access the storage infrastructure. This means that if a storage array is unavailable, another storage array configured under VPLEX/VE continues to serve the I/O. Application HA Figure 7: VPLEX/VE Provides RAID-1 mirroring across Sites VPLEX/VE provides a shared block storage device that enables a single datastore to be accessible across sites. In doing so, the ESXi cluster can leverage all of the traditional functionality found within a single site across two sites separated by up to 10ms RTT latency. This means ware HA can be applied to virtual machines within the ESXi cluster and provide automatic (hands-off) virtual machine restart in the event of a site loss or hardware failure. 11

12 Migration VPLEX/VE simplifies data center block storage management and eliminates outages during data migrations between arrays or upgrading/maintaining array technology. With VPLEX/VE, a ware Admin is able to: Perform non-disruptive array technology refresh tasks using the two-way data exchange between locations Create an active-active ESXi cluster configurations to achieve active use of resources at both sites Provide instant access to data between data centers Non-disruptively move between Datacenters up with up to 10ms RTT latency Figure 8: Non-Disruptive Array Technology Refresh Dynamic Workload Load Balancing across Sites Dynamically move storage from busy arrays to idle arrays for better asset utilization Use ware DRS to balance virtual machine workloads within and across data centers Figure 9: Cross-Site DRS and vmotion Immediate Cross-Site vmotion Leverage vmotion without the need to use Storage vmotion when moving between sites. 12

13 VPLEX/VE Performance Planning Summary This section provides information about VPLEX/VE performance and some sample performance metrics. The information is straightforward, but fairly simplified. In other words, a number of simplifying assumptions have been made. Individual results WILL vary. Background Adding VPLEX/VE to a vsphere cluster provides increased block storage availability, flexibility, and capability. These benefits come at the expense of adding processing overhead / latencies associated with servicing I/O for the ESXi hosts running VPLEX/VE. Performance of the host system hardware and IP network infrastructure is, therefore, critical to the success of a VPLEX/VE deployment. A representative vsphere cluster environment along with the corresponding performance data charts and tables are provided to illustrate what VPLEX/VE was able to achieve for various IO workloads. All numbers are based on measurements from the test environment with VPLEX/VE running against common IO profiles. Hardware Configuration The example vsphere cluster test configuration, which includes the VPLEX/VE Systems Under Test, or SUTs, has the following components: 4x4 VPLEX VE Metro configuration running on isolated ESX Server. The table below for performance information on the vdirectors. Cluster Storage: VNX iscsi Array with LUNs evenly distributed across the two clusters. IP WAN network consisting of two iscsi Ethernets. Performance Attribute CPU Cores Processor Type Description 12 CPUs X 2.5 GHz Intel Xeon CPU ES-2640 at 2.5 GHz Processor Sockets 2 Cores per Socket 6 Logical Processors 24 Hyper-Threading Active Number of 1 GbE NICs 11 Figure 10: ESXi Test Host Specifications 13

14 Key Performance Indicators (KPIs) Five simulated application workloads were measured. These are the same application IO footprints that are part of the key performance indicators (KPIs) for our VPLEX appliance characterization. Simulated Application Profiles The following simulated application profiles were tested: 1. OLTP1 (mail application) 2. OLTP2 (small oracle application / database transactional) 3. OLTP2-HW (large oracle application / heavy-weight transactions) 4. DSS2 (database decision support) 5. DSS128K (multimedia streaming / large block read and writes) These simulated application profiles are each a composite of five simple I/O profiles: Random Read Hits (rrh), Random Reads (Miss) (rr), Random Writes (rw), Sequential Reads (sr) and Sequential Writes (sw). I/O Profile of Simulated Applications The I/O size and proportion of each component I/O profile varies across the application profiles as detailed in the following tables. I/O Profile OLTP1 I/O Size (KB) % of Total rrh 4 40 rr 4 24 rw 4 16 sr 4 10 sw 4 10 I/O Profile OLTP2 I/O Size (KB) % of Total rrh 8 20 rr 8 45 rw 8 15 sr sw

15 I/O Profile OLTP2HW I/O Size (KB) % of Total rrh 8 10 rr 8 35 rw 8 35 sr 64 5 sw I/O Profile DSS2 I/O Size (KB) % of Total rrh 4 0 rr 4 15 rw 4 5 sr sw I/O Profile DSS128K I/O Size (KB) % of Total rrh rr rw 64 4 sr sw Notes on Selecting the Application Profile In order to select the application profile to use, i.e., which is most like the application that will be used with VPLEX/VE; the following section will serve as a mini guide. The subtitles of the 5 apps are representative of their origin: OLTP1 is based on Exchange; OLTP2 on a typical Oracle OLTP app; OLTP2HW on the TPC-C benchmark (which emulates a heavy-weight OLTP application); DSS2 on a typical Decision Support app; and DSS128k on a multimedia distribution app. 15

16 While OLTP1 is the lightest application type, i.e., an OLTP1 transaction takes the fewest resources to execute, if the workload is specified in terms of KBps, then OLTP1 is also the densest application, i.e., it is able to saturate the CPU resources with the smallest bandwidth. That is because OLTP1 is made up of only small size (4 KB) I/Os. When comparing a given bandwidth s worth of application demand, it should not be surprising when OLTP1 transactions saturate the system earlier than OLTP2HW transactions. The former will be doing many more IOps. The correct application to select is the one that is predominately like the application workload that will be deployed onto VPLEX/VE storage. And the correct metric to size the workload (IOps or KBps) is the one for which the most environment specific data is available. Experience shows that the app estimates tend to be too conservative. Pick the average or (better) median case, not the one-time-highest-peak-ever-seen-in-a-1-minute-interval. Testing Methodology Type of Measurements The types of performance measurements are presented in this document are called Steady-State Performance or Steady-State Latency Curves. Initial peak workload is used to determine the workload range ( peak = 100% duty cycle). A series of offered workloads at various duty cycles (10%, 30%, 50%, 70%, and 99% of peak ) is run, attaining a steady-state measurement of each. This type of run is useful for creating more realistic workloads, or at least for measuring latencies by plotting the observed load against the observed latency. The latency measure at peak is not steady-state, and, therefore not a real or useful latency. Data Charts The charts provide an easy way to lookup the vapp vcpu utilization (%) and the latencies (Average Response Time) by picking either the I/O per second or KB per second on the X-axis and following up to the relevant curve and across to the relevant Y-axis. The default IP network MTU of 1500 is shown. Testing did not show a significant difference in performance when using an MTU of For example: 65,000 OLTP1 I/O per second would run at ~ 40% utilization (green line) and have about 2.3 ms response time (red line). Technical Note: For the following charts, WAN RTT latency between Sites was set to 0 milliseconds. VPLEX/VE uses write-through caching, so the actual WAN RTT latency for the environment should be added each of the Average RT (ms) data points on the chart for the best overall estimates. 16

17 OLTP1 IOps mtu:1500 avgrt mtu:1500 Percent Utilization 100% 80% 60% 40% 20% 0% KB per second ,000 40,000 60,000 80, , , ,000 I/Os per second 10 Average Response Time (ms) OLTP2 IOps mtu:1500 avgrt mtu:1500 Percent Utilization 100% 80% 60% 40% 20% 0% KB per second ,000 1,200 1, ,000 20,000 30,000 40,000 50,000 60,000 70,000 I/Os per second Average Response Time (ms) 17

18 OLTP2HW IOps mtu:1500 avgrt mtu:1500 Percent Utilization 100% 80% 60% 40% 20% 0% KB per second , ,000 20,000 30,000 40,000 50,000 60,000 I/Os per second DSS Average Response Time (ms KBps mtu:1500 avgrt mtu:1500 Percent Utilization 100% 80% 60% 40% 20% 0% I/Os per second - 5,000 10,000 15,000 20,000 25,000 30,000 35, ,000 1,500 2,000 KBs per second 10 Average Response Time (ms) 18

19 Conclusion Adding VPLEX/VE to a new or existing vsphere cluster provides increased storage resiliency, flexibility, and functionality, but it does come with a cost. Though minor, the expense of added I/O processing overhead / latencies when servicing IO from must be accounted for during the initial planning process. Performance of the underlying host hardware system is the key to getting the desired result. This document reviewed VPLEX/VE architecture, use cases, and performance characteristics. Data charts and a test methodology were provided that can be used to estimate VPLEX/VE ability to support a given IO workload. The estimates are based on measurements from a representative host hardware and IP network configuration running various well known IO profiles. A number of environmental and applications assumptions were made to simplify the overall discussion. For this reason, the results indicated are only a guide and may not represent the actual results obtained in specific VPLEX/VE environments. 19

20 VPLEX References The following reference documents are available at EMC VPLEX/VE Site Preparation Guide EMC VPLEX/VE Release Notes EMC VPLEX/VE Security Configuration Guide EMC VPLEX/VE Configuration Worksheet EMC VPLEX/VE CLI Guide EMC VPLEX/VE Product Guide EMC ware ESXi Host Connectivity Guide Supplemental ware References The CPU Scheduler in ware vsphere 5.1: Perf.pdf Performance Best Practices for ware vsphere 5.1: Best Practices for Running ware vsphere on iscsi: Best Practices for Performance Tuning of Latency-Sensitive Workloads in vsphere s: Workloads.pdf Deploying Extremely Latency-Sensitive Applications in ware vsphere Performance Study: For more information: Explore and compare the latest VPLEX products in the EMC Store 20