Reference Architecture EMC VSPEX SERVER VIRTUALIZATION SOLUTION VMware vsphere 5 for 100 Virtual Machines Enabled by VMware vsphere 5, EMC VNXe3300, and EMC Next-Generation Backup EMC VSPEX April 2012
Copyright 2012 EMC Corporation. All rights reserved. Published in the USA. Published April 2012 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date regulatory document for your product line, go to the technical documentation and advisories section on the EMC online support website. Part Number: H10670 2
Table of contents Reference architecture overview...... 5 Document purpose... 5 Solution purpose... 5 The business challenge... 5 The technology solution... 6 Solution benefits... 6 Key components......... 7 Introduction... 7 VMware vsphere 5... 7 EMC VNXe Series... 7 Software suites available... 8 Software packs available... 8 EMC Virtual Storage Integrator for VMware vsphere... 8 EMC Avamar... 8 Solution architecture......... 10 Logical architecture diagram...10 Reference architecture overview...10 Hardware resources...11 Sizing for Validated Configuration...12 Software resources...14 Storage architecture......... 15 Storage layout...15 Storage layout overview...15 High availability and failover...... 16 Introduction...16 Storage layer...16 Connectivity layer...16 Host layer...16 Validated environment profile...... 17 Profile characteristics...17 Backup environment profile...... 18 Backup characteristics...18 Backup layout...18 Conclusion......... 19 3
References......... 20 EMC documentation...20 Other documentation...20 4
Reference architecture overview Document t purpose Solution purpose This document describes the reference architecture of the EMC VSPEX server virtualization solution for 100 virtual machines. This document validates the performance of the solution and provides guidelines for building similar solutions. VSPEX pre-validated and modular architectures are built with proven best-of-breed technologies to create complete virtualization solutions that enable you to make an informed decision in the hypervisor, compute and networking layers. VSPEX eliminates server virtualization planning and configuration burdens. VSPEX infrastructures accelerate your IT Transformation by enabling faster deployments, greater flexibility of choice, efficiency, and lower risk. This reference architecture is not intended to be a comprehensive guide to every aspect of this solution. Server capacity is provided in generic terms for required minimums of CPU, memory and network interfaces; the customer is free to select the server and networking hardware of their choice that meet or exceed the stated minimums. For more detailed information on performance and scalability testing, refer to EMC VSPEX Server Virtualization Solution Enabled by VMware vsphere 5, EMC VNXe3300 Deployment Guide of this solution and associated documentation for best practices and specific usage requirements. The business challenge Customers require a scalable, tiered, and highly available infrastructure on which to deploy their business and mission-critical applications. There are several new technologies available to assist them in consolidating and virtualizing their server infrastructure, but they need to know how to use these technologies to maximize their investment, support service-level agreements, and reduce their total cost of ownership (TCO). This solution addresses the following challenges: Availability: Standalone servers incur downtime for maintenance or unexpected failures. Clusters of redundant standalone nodes are an inefficient in use of CPU, disk and memory resource. Server management and maintenance: Individually maintained servers require significant repetitive activities for monitoring, problem resolution, patching, and other common activities. This is labor intensive, costly, errorprone, and inefficient. Security, downtime, and outage risks are elevated. Ease of solution deployment: While small and medium businesses must address the same IT challenges as larger enterprises, staffing levels, experience, and training are generally more limited. IT generalists are often responsible for managing the entire IT infrastructure, or reliance is placed on third-party sources for maintenance or other tasks. Perceived complexity of the IT function raises fear of risk and may block adoption of new technology. Therefore, simplicity of deployment and management are highly valued. Storage efficiency: Storage added locally to physical servers or provisioned directly from a shared resource or array leads to over-provisioning and waste. 5
Backup: Traditional backup approaches are slow and frequently unreliable. There tends to be inflection points (or plateaus) in the virtualization adoption curve when the number of virtual machines increases. With a few virtual machines, the situation seems manageable and most organizations can get by with existing tools and processes. However, when the virtual environment grows the backup and recovery become the limiting factor in deployment. The technology solution Solution benefits This solution uses the EMC VNXe3300 array and VMware vsphere 5 to provide storage and server hardware consolidation. The new virtualized infrastructure is centrally managed, allowing efficient deployment and management of a scalable number of virtual machines and associated shared storage. This VSPEX solution allows the customer that is new to server virtualization to realize the following benefits: Application availability Highly available, scalable infrastructure that makes most efficient use of CPU, memory, and storage resources and grows with the business Simplified and centralized management of server and storage deployment, monitoring, patching, and other normal activities, resulting in lower management overhead, reduced risk, and reduced downtime Simplicity of solution deployment and use, targeting the IT generalist in the small or medium business (SMB) and providing ease of use, integration points, and ability to mask complex tasks through the use of advanced wizards Storage efficiency EMC VNXe array features optimized storage efficiency, reduced over-provisioning and simplified storage management Backup Integration with EMC s next-generation backup solutions offering virtual machine and application backup and recovery capabilities. 6
Key components Introduction VMware vsphere 5 This section briefly describes the key components of this solution. VMware vsphere 5 EMC VNXe Series EMC Virtual Storage Integrator for VMware vsphere EMC Avamar The Solution architecture on page 10 provides details on all the components that make up the reference architecture. VMware vsphere 5 transforms a computer s physical resources, by virtualizing the CPU, RAM, hard disk, and network controller. This transformation creates fully functional virtual machines that run isolated and encapsulated operating systems and applications just like physical computers The high-availability features of VMware vsphere 5 such as vmotion and Storage vmotion enable seamless migration of virtual machines and stored files from one vsphere server to another with minimal or no performance impact. Coupled with vsphere DRS and Storage DRS, virtual machines have access to the appropriate resources at any point in time through load balancing of compute and storage resources. EMC VNXe Series The EMC VNX family is optimized for virtual applications delivering industry-leading innovation and enterprise capabilities for file, block, and object storage in a scalable, easy-to-use solution. This next-generation storage platform combines powerful and flexible hardware with advanced efficiency, management, and protection software to meet the demanding needs of today s enterprises. The VNXe series is powered by Intel Xeon processor, for intelligent storage that automatically and efficiently scales in performance, while ensuring data integrity and security. The VNXe series is purpose-built for the IT manager in smaller environments and the VNX series is designed to meet the high-performance, high-scalability requirements of midsize and large enterprises. Table 1. Feature VNXe customer benefits Next-generation unified storage, optimized for virtualized applications Capacity optimization features including compression, deduplication, thin provisioning, and application-centric copies High availability, designed to deliver five 9s availability Multiprotocol support for file and block ü ü ü ü 7
Simplified management with EMC Unisphere for a single management interface for all NAS, SAN, and replication needs ü Note: VNXe does not support block compression. Software suites available Local Protection Suite Increases productivity with snapshots of production data. Remote Protection Suite Protects data against localized failures, outages, and disasters. Application Protection Suite Automates application copies and proves compliance. Security and Compliance Suite Keeps data safe from changes, deletions, and malicious activity. Software packs available VNXe3300 Total Protection Pack Includes the Local Protection, Remote Protectuion and Application Protection Suites. EMC Virtual Storage Integrator for VMware vsphere EMC Virtual Storage Integrator (VSI) for VMware vsphere is a plug-in to the vsphere client that provides a single management interface that is used for managing EMC storage within the vsphere environment. Features can be added and removed from VSI independently, which provides flexibility for customizing VSI user environments. Features are managed by using the VSI Feature Manager. VSI provides a unified user experience, which allows new features to be introduced rapidly in response to changing customer requirements. The following features were used during the validation testing: Storage Viewer (SV) Extends the vsphere client to facilitate the discovery and identification of EMC VNXe storage devices that are allocated to VMware vsphere hosts and virtual machines. SV presents the underlying storage details to the virtual datacenter administrator, merging the data of several different storage mapping tools into a few seamless vsphere client views. Unified Storage Management Simplifies storage administration of the EMC VNXe unified storage platform. It enables VMware administrators to provision new Network File System (NFS) and Virtual Machine File System (VMFS) datastores, and raw device mapping (RDM) volumes seamlessly within vsphere client. Refer to the EMC VSI for VMware vsphere product guides on the EMC Online Support website for more information. EMC Avamar EMC s Avamar data deduplication technology seamlessly integrates into virtual environments, providing rapid backup and restoration capabilities. Avamar s deduplication results in vastly less data traversing the network, and greatly reduces 8
the amount of data being backed up and stored translating into storage, bandwidth and operational savings. The following are two of the most common recovery requests made to backup administrators: File-level level recovery: Object-level recoveries account for the vast majority of user support requests. Common actions requiring file-level recovery are, individual users deleting files, applications requiring recoveries, and batch process-related erasures. System recovery: Although complete system recovery requests are less frequent in number than those for file-level recovery this bare metal restore capability is vital to the enterprise. Some common root causes for full system recovery requests are viral infestation, registry corruption, or unidentifiable unrecoverable issues. The Avamar s functionality in conjunction with VMware implementations adds new capabilities for both backup and recovery in both of these scenarios. Key capabilities added in VMware, such as the vstorage API integration and change block tracking (CBT) enables Avamar software to protect the virtual environment more efficiently. Leveraging CBT for backup and recoveries with virtual proxy server pools, this functionality minimizes management needs. Coupling that with Data Domain as the storage platform for image data, this solution enables the most efficient integration with two of the industry leading next-generation backup solutions. 9
Solution architecture Logical architecture diagram The architecture diagram in this section characterizes the infrastructure validated for support of up to 100 virtual machines. Figure 1 shows the overall logical architecture of the solution. Figure 1. Reference architecture overview Network Logical architecture The architecture includes the following components: VMware vsphere 5 Server Provides a common virtualization layer to host a server environment. The specifics of the validated environment are listed in Table 2. vsphere 5 provides highly available infrastructure through such features as: vmotion Provides live migration of virtual machines within a virtual infrastructure cluster, with no virtual machine downtime or service disruption. Storage vmotion Provides live migration of virtual machine disk files within and across storage arrays with no virtual machine downtime or service disruption. vsphere High Availability (HA) Detects and provides rapid recovery for a failed virtual machine in a cluster. Distributed Resource Sc Scheduler heduler (DRS) Provides load balancing of computing capacity in a cluster. Storage Distributed Resource Scheduler (SDRS) Provides load balancing across multiple datastores, based on space use and I/O latency. VMware vcenter Server 5 Provides a scalable and extensible platform that forms the foundation for virtualization management for the VMware vsphere 5 cluster. All vsphere hosts and their virtual machines are managed from vcenter. 10
EMC Virtual Storage Integrator for VMware vsphere EMC Virtual Storage Integrator (VSI) for VMware vsphere is a plug-in to the vsphere client that provides storage management for EMC arrays directly from the client. VSI is highly customizable and helps provide a unified management interface. Microsoft SQL Server 2008 R2 VMware vcenter Server requires a database service to store configuration and monitoring details. A Microsoft SQL 2008 R2 server is used for this purpose. IP Network All network traffic is carried by standard Ethernet network with redundant cabling and switching. User and management traffic is carried over a public network while storage traffic is carried over a private, non-routable subnet. Shared Infrastructure DNS and authentication/authorization services like Microsoft Active Directory can be provided through existing infrastructure or set up as part of the new virtual infrastructure. EMC VNXe3300 array Provides storage by presenting Network File System (NFS) datastores to vsphere hosts. These datastores are created using application aware wizards that are included in the EMC Unisphere interface. Hardware resources Table 2 lists the hardware used in this solution. Table 2. Hardware VMware vsphere servers Solution hardware Configuration Memory: 2 GB RAM per virtual machine (200 GB RAM across all servers) 2 GB RAM reservation per vsphere host CPU: One vcpu per virtual machine Four vcpus per physical core Network: Six 1 GbE NICs per server Notes Configured as a single vsphere cluster. Note: To implement VMware vsphere High Availability (HA) functionality and to meet the listed minimums, the infrastructure should have one additional server. Network infrastructure Minimum switching capacity: Two physical switches Six 1GbE ports per vsphere server One 1GbE port per storage processor for management Two 1GbE ports per storage processor for data Redundant LAN configuration 11
Hardware EMC nextgeneration backup EMC VNXe3300 Servers for customer infrastructure Configuration Avamar One Gen4 utility node One Gen4 3.9TB spare node Three Gen4 3.9TB storage nodes Data Domain 1 Data Domain DD640 1 ES30 15x1TB HDD Two storage processors (active / active) Two 1 GbE interfaces per storage processor Sixty-three 600 GB 15k rpm 3.5-inch SAS disks (9 * 600 GB 6+1 R5 Performance Drive Packs) Three 600 GB 15k rpm 3.5-inch SAS disks as hot spares Minimum number required: Two physical servers 16 GB RAM per server Four processor cores per server Two 1 GbE ports per server Notes This can include the initial disk pack on the VNXe. These servers and the roles they fulfill may already exist in the customer environment Sizing for Validated Configuration When selecting servers for this solution, the processor core shall meet or exceed the performance of the Intel Nehalem family at 2.66Ghz. As servers with greater processor speeds, performance and higher core density become available servers may be consolidated as long as the required total core and memory count is met and a sufficient number of servers are incorporated to support the necessary level of high availability. As with servers, network interface card (NIC) speed and quantity may also be consolidated as long as the overall bandwidth requirements for this solution and sufficient redundancy necessary to support high availability are maintained. A configuration of four (4) servers each with two (2) sockets of six (6) cores each, 72Gb of RAM and six (6) 1Gbe NICs will support this solution for a total of 48 cores and 288Gb of RAM. A as shown in Table 2, a minimum of one (1) core is required to support four (4) virtual machines and a minimum of 2Gb of RAM for each VM. The correct balance of memory and cores for the expected number of virtual machines to be supported by a server must also be taken into account. For example, server expected to support 20 virtual machines requires a minimum of 5 cores but also a minimum of 40Gb of RAM. Additional cores and RAM to support the hypervisor and loss of at least one physical host supporting VMs requiring high availability have also been calculated into this solution. 12
Figure 2. Network diagram IP network switches used to implement this reference architecture must have a minimum backplane capacity of 48 Gb/s non-blocking and support the following features: IEEE 802.1x Ethernet flow control 802.1q VLAN tagging Ethernet link aggregation using IEEE 802.1ax (802.3ad) Link Aggregation Control Protocol SNMP management capability Jumbo frames The quantity and type of switches chosen should support high availability and it is also recommended that a network vendor be chosen based on the availability of parts, service and support contracts. In addition to the above features, the network configuration should include the following: A minimum of two switches to support redundancy Redundant power supplies 13
A minimum of 36 1Gbe ports (distributed for high availability) and the appropriate uplink ports for customer connectivity Use of 10Gbe ports should align with those on the server and storage while keeping in mind the overall network requirements for this solution and a level of redundancy to support high availability. Additional server NICs and storage connections should also be considered based on customer or specific implementation requirements. Disk storage layout is explained in the Storage architecture on page 15. Software resources The following table lists the software used in this solution. Table 3. Solution software Software Next-generation eneration backup Configuration Avamar 6.0.0-592 Data Domain OS 5.1.0.9-282511 VNXe3300 Software Version 2.2.0.16150 EMC VSI for VMware vsphere: Unified Storage Management EMC VSI for VMware vsphere: Storage Viewer Version 5.1 Version 5.1 VMware vsphere vsphere server 5.0 vcenter Server 5.0 Operating system for vcenter Server Microsoft SQL Server Windows Server 2008 R2 Standard Edition Version 2008 R2 Enterprise Edition Virtual machines Base operating system Microsoft Window Server 2008 R2 VDBench 5.02 Note: VDBench was used to validate this solution. However, it s not required in this configuration. 14
Storage architecture Storage layout The architecture diagram in this section shows the physical disk layout. Disk provisioning on the VNXe series is simplified through the use of wizards, so that administrators do not choose which disks belong to a given storage pool. The wizard may choose any available disk of the proper type, regardless of where the disk physically resides in the array. Figure 3 shows the overall storage layout of the solution. Figure 3. Storage layout Storage layout overview The reference architecture uses the following configuration: Sixty-three 600 GB SAS disks are allocated to a single storage pool as nine 6+1 RAID 5 groups (sold as 7-disk packs). EMC recommends that at least one hot spare disk is allocated for each 30 disks of a given type. At least two NFS shares are allocated to the vsphere cluster from the single storage pool to serve as datastores for the virtual servers. 15
High availability and failover Introduction This VSPEX solution provides a highly available virtualized server and storage infrastructure. Each component is configured to provide a scalable, robust architecture for the host, connectivity, and storage layers. Storage layer Connectivity layer Host layer The VNX family is designed for five 9s availability by using redundant components throughout the array. All of the array components are capable of continued operation in case of hardware failure. The RAID disk configuration on the array provides protection against data loss due to individual disk failures, and the available hot spare drives can be dynamically allocated to replace a failing disk. The advanced networking features of the VNX family provide protection against network connection failures at the array. Each vsphere host has multiple connections to user and storage Ethernet networks to guard against link failures. These connections are spread across multiple Ethernet switches to guard against component failure in the network. The vsphere servers have redundant power supplies and network connections to reduce the impact of failure of these components. vsphere HA is configured on the vsphere cluster to help recover virtual servers quickly in case of a complete host failure. 16
Validated environment profile Profile The solution was validated with the following environment profile. characteristics Table 4. Profile characteristics Profile characteristic Value Number of virtual machines 100 Virtual machine OS Windows Server 2008 R2 Processors per virtual machine 1 Number of virtual processors per physical CPU core 4 RAM per virtual machine Average storage available for each virtual machine Average IOPS per virtual machine 2 GB 100 GB 25 IOPS Number of datastores to store virtual machine disks 4 Number of virtual machines per datastore 25 Disk and RAID type for datastores RAID 5, 600 GB, 15k rpm, 3.5-inch SAS disks 17
Backup environment profile Backup The solution was sized with the following application environment profile. characteristics Table 5. Profile characteristics Profile characteristic Value Number of users 1000 Number of virtual machines Exchange data SharePoint data SQL server User data 100 (20% DB, 80% Unstructured) 1TB (1GB mail box per user) 0.5 TB 0.5 TB 5 TB (5.0 GB per user) Daily Change Rate for the applications Exchange data 10% SharePoint data 2% SQL Server 5% User data 2% Retention per data types All DB data User data 14 Dailies 30 Dailies, 4 Weeklies, 1 Monthly Backup layout Avamar provides various deployment options depending on the specific use case and the recovery requirements. In this case, the solution is deployed with both Avamar and Data Domain managed as a single solution. This enables the unstructured user data to be backed up directly to the Avamar system for simple file level recovery. The database and virtual machine images will be managed by the Avamar software, but it is directed to the Data Domain system with the embedded Boost client library. This backup solution unifies the backup process with industry leading deduplication backup software and storage, and achieves the highest levels of performance and efficiency. 18
Conclusion VSPEX have been engineered to enable the small or medium business to simply, quickly, and reliably deploy a broad range of shared storage workloads. Setup wizards enable an IT generalist to provision VMware storage with a short series of mouse clicks and recommended best practices applied automatically. In conjunction with VMware vcenter and vsphere, VNXe provides a proven and economical path to server virtualization for the small or medium business. Table 6. Solution benefits Feature EMC next-generation backup EMC VNXe Series VMware vsphere 5 Benefits Unifies the backup process with industry leading deduplication backup software and storage, and achieves the highest levels of performance and efficiency. Provides a robust, reliable, high-performance, common storage platform for many virtual servers. A single storage platform that is efficient, powerful, and built for the most demanding virtual environments. VMware vsphere, including vcenter and vsphere, provides a well-established virtualization platform for server consolidation. vsphere 5 adds new storage features to an already rich set of capabilities to help increase and scale virtualized environments. This reference architecture provides a blueprint of a validated server virtualization solution enabled by EMC VNXe series, EMC next-generation backup, and VMware vsphere 5 virtualization platform. 19
References EMC documentation Other documentation The following documents, available on the EMC Online Support website, provide additional and relevant information. If you do not have access to a document, contact your EMC representative: EMC VSPEX Server Virtualization Solution VMware vsphere 5 for 50 Virtual Machines Enabled by VMware vsphere 5, EMC VNXe3100, and EMC Next- Generation Backup Reference Architecture EMC VSPEX Server Virtualization Solution VMware vsphere 5 for 125 Virtual Machines Enabled by VMware vsphere 5, EMC VNX5300, and EMC Next- Generation Backup Reference Architecture EMC VSPEX Server Virtualization Solution VMware vsphere 5 for 250 Virtual Machines Enabled by VMware vsphere 5, EMC VNX5500, and EMC Next- Generation Backup Reference Architecture EMC VSPEX Server Virtualization Solution VMware vsphere 5 for 100 Virtual Machines Enabled by VMware vsphere 5 and EMC VNXe3300 Deployment Guide EMC VSPEX Server Virtualization Solution VMware vsphere 5 for 0-100 Virtual Machines Sizing Guide EMC VSI for VMware vsphere: Storage Viewer Product Guide EMC VSI for VMware vsphere: Unified Storage Management Product Guide For VMware documentation, please refer to the VMware web site at http://www.vmware.com. 20