EMC CLOUD-ENABLED INFRASTRUCTURE FOR SAP

Transcription

1 White Paper EMC CLOUD-ENABLED INFRASTRUCTURE FOR SAP Foundation Bundle using EMC Symmetrix VMAX, EMC Storage Resource Management Suite, VMware vcloud Suite, and VMware vcenter Chargeback Manager Build a virtual data center for SAP providing service catalogs, productized SLAs and multitenant resource pools Implement chargeback, based on fully automated cost measurement, cost analysis, and cost reporting Efficiently manage availability, capacity, performance, and health in the SAP landscape EMC Solutions Group Abstract This white paper focuses on the foundation layer of the on-premises Cloudenabled Infrastructure for SAP, which can be enhanced with optional add-on bundles to address enterprise business challenges. This Cloud-enabled Infrastructure is driven by VMware vcloud Suite. End-to-end visibility, management and analysis are provided by the EMC Storage Resource Management Suite. The solution is powered by the EMC Symmetrix VMAX storage platform and technologies. September 2013

2 Copyright 2013 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All trademarks used herein are the property of their respective owners. Part Number H

3 Table of contents Executive summary... 6 Business case... 6 Solution overview... 6 Key benefits... 7 EMC service offering Cloud Readiness Accelerator... 7 Introduction... 9 Purpose... 9 Scope... 9 Audience... 9 Overview... 9 Prerequisites and supporting documentation Terminology Key components Overview EMC Symmetrix VMAX EMC PowerPath/VE EMC VSI EMC Storage Resource Management Suite VMware vcloud Director VMware vcloud Networking and Security VMware vcenter Chargeback Manager VMware vcenter Operations Manager EMC storage platform key values Symmetrix VMAX EMC FAST VP Twenty-two percent CAPEX reductions Thirty-seven percent OPEX reductions Automated and optimized storage management with sustained performance Federated Tiered Storage Host I/O Limits Management VMware integration Virtual provisioning Validated landscape Overview Environment architecture

4 Physical architecture Hardware resources Logical architecture Software resources Key functional design and configuration Overview Virtual data center for SAP Service levels Designing vcloud Director Designing the vsphere Infrastructure chargeback Overview vcenter ChargeBack Manager design Integrated cloud management performance analysis Overview Full visibility of the virtual data center End-to-end performance analysis Configuration validation and compliance Storage capacity analysis Predefined and customized intelligent reports Fully Automated Storage Tiering Cloud networking and security for SAP Overview VMware vcloud networking and security design SAP system design Overview Compute Storage Network Installation type Other Validation of functions Overview Use case 1: Virtual data center for SAP Test scenario Success criteria Validation procedure Summary Use case 2: Automated provisioning, service catalogs, self-services, and chargeback

5 Test scenario Validation procedure Summary Use case 3: Integrated cloud management and performance analysis Success criteria Scenario 1: On-demand resource pooling Scenario 2: Root cause analysis and problem resolution Summary Conclusion Summary Findings Moving forward References White papers Product documentation Other documentation

6 Executive summary Business case Automation and optimization of IT processes running on a virtualized infrastructure are key enablers to fulfill the business needs of IT managers who want to provide information technology as a service. Many case studies exemplify striking results after transforming to the EMC Cloudenabled Infrastructure for SAP. One of the best examples of deploying a private cloud is EMC s own SAP landscape, which became fully operational in The EMC IT approach to the has enabled a global SAP instance supporting over 300 global sites. It has resulted in a 50 percent performance increase, five times the scalability, and twenty times the integration throughput. In the process, EMC IT was able to retire 65 legacy applications and reduce the count of Application Integration Component (AIC) interfaces from 450 to 125. New virtual environment provisioning times have gone from days to hours, and the percentage of time available for primary value chain activities has doubled. To support running your SAP operations as a business, the following basic functions would be required for the on-premises cloud: A virtual datacenter for SAP providing service catalogs, productized service level agreements (SLAs), and multitenant resource pools Chargeback services, based on fully automated cost measurement, cost analysis, and cost reporting Integrated cloud management and performance analysis tools, enabling you to efficiently manage availability, capacity, performance, and health in the SAP landscape Automatic resource management for server, network, and storage A high level of security with simplified compliance This white paper introduces the EMC solution that serves all of these functions. Solution overview VMware vcloud Suite, EMC Storage Resource Management Suite, and EMC Symmetrix VMAX are the primary enabling technologies for this solution: VMware vcloud Director applies the principles of pooling, abstraction, and automation to all data center services such as compute, storage, networking, and security. It enables the multi-tenancy business model for IT operation and provides the portal for self-service provisioning. VMware vcenter Chargeback Manager is used for metering the resource consumption of different organizations. VMware vcenter Operation Manager provides powerful functions to efficiently manage availability, capacity, and performance of the virtual datacenter. 6

7 The EMC Storage Resource Management Suite provides visualizations and reporting for understanding and managing the impact storage has in the onpremise cloud. It helps administrators to manage availability, capacity, and performance at the storage layer and provides powerful views of the relationships among host, virtual machine, network, and storage. The Symmetrix VMAX platform is a powerful, trusted, and smart storage platform for your multiple mission-critical applications and provides automation and efficiency in tiered storage environments with Fully Automated Storage Tiering for Virtual Pools (FAST VP). Key benefits EMC service offering Cloud Readiness Accelerator The solution builds the foundation layer of the EMC Cloud-enabled Infrastructure for SAP by providing the following functions: Virtual datacenter for SAP Infrastructure chargeback Integrated cloud management and performance analysis Fully Automated Storage Tiering Cloud networking and security for SAP The Cloud Readiness Accelerator (CRA) is a fast-track engagement with immediate value and actionable deliverables that allow IT organizations to accelerate their transformation strategies towards cloud computing. CRA uses a proven methodology to examine an IT organization s maturity by analyzing nine fundamental characteristics of the cloud data center. An EMC cloud architect with extensive experience and certifications in virtualization and cloud computing technologies guides the participants through a set of guided workshops. 7

8 The workshops are designed to build a personalized cloud maturity model that describes the existing and desired readiness state of the IT environments on a scale from basic to federated, as shown in Figure 1. Participants define their own roadmaps for building their own Cloud-enabled Infrastructures for SAP. Figure 1. Cloud maturity model 8

9 Introduction Purpose Scope Audience Overview This white paper describes the foundation layer of the EMC Cloud-enabled Infrastructure for SAP solution. This white paper: Introduces the key enabling technologies for this solution Describes the best practice and design consideration for this solution Outlines how the key components are configured Presents the test results performed to validate the solution Identifies the key business benefits of the solution Demonstrates the functionality within predefined use cases This white paper is intended for SAP operations managers, administrators (SAP/server/network/storage), IT architects, and technical managers responsible for designing, creating, or managing mission-critical SAP landscapes. The solution is designed so that customers can flexibly choose the cloud functionalities (or bundles ) they want to enable without losing sight of their end goal an efficient and reliable private cloud. The foundation bundle is a mandatory component upon which all the other cloud bundles are based. After establishing the foundation bundle, customers can plan their own roadmaps to determine their next steps. Table 1 outlines the foundation bundle and its functions. 9

10 Table 1. solution: foundation bundle Bundle Function Benefits Technology On-premises cloud Virtual datacenter for SAP Autonomy of business units and application operations VMware vcloud Suite Enterprise Service catalogs SLAs Management of vcloud tenants Resource pooling Infrastructure chargeback Cost measurement, analysis, and reporting of the use of compute, network, storage, and backup resources (in combination with data protection add-on bundle) VMware vcenter Chargeback Integrated cloud management and performance analysis Manage availability, capacity, performance, and health in the SAP landscape VMware vcenter Operations Manager, EMC SRM, EMC VSI Storage tiering Automatically get the right data to the right place at the right time EMC VMAX, FAST Suite Cloud networking and security for SAP Cloud-enabled infrastructure security framework VMware vcloud Networking and Security Authorization concepts Compliance and noncompliance tracking 10

11 Table 2 describes the add-on bundles that are described in separate solution white papers. Table 2. solution: add-on bundles Bundle Function Benefit Technology High availability and application mobility High availability and application mobility for SAP High availability within one datacenter and across two sites Non-disruptive movement of applications from one datacenter to another EMC VPLEX, VMware HA Disaster recovery Disaster recovery for SAP Disaster recovery protection for cloud management cluster and SAP systems VMware Site Recovery Manager EMC RecoverPoint SAP automation and operations SAP system virtualization Automated SAP system provisioning Any service any time on any server Provision SAP systems on demand with automated end-to-end process SAP NetWeaver Landscape Virtualization Management Mass operations Reduce downtime window during maintenance Data protection Backup and recovery for SAP Provide data protection to cover the cloud management stack and SAP systems Backup/recovery at local and far apart (hundreds of kilometers) sites EMC Avamar, EMC Data Domain, EMC Data Protection Advisor (DPA) Enhanced security Enhanced security and compliance for SAP Efficient, collaborative enterprise governance, risk and compliance (egrc) program across IT, finance, operations and legal domains RSA Archer egrc, RSA Data Loss Prevention (DLP), RSA SecurID Data loss prevention Secure user to network authentication 11

12 Prerequisites and supporting documentation Terminology Read the white paper EMC Transforms IT On-Premise Private Cloud Infrastructure for SAP Reference Architecture, available on EMC.com, for a detailed overview of the Cloud-enabled Infrastructure for SAP, and to find out how enterprises can benefit, from both the technical and business perspectives. Table 3 defines the terms used in this document. Table 3. Term CMCNE Datastore DRS FAST VP LDAP Terminology Definition Connectrix Manager Converged Network Edition A storage location for virtual machine files Distributed Resource Scheduler Fully Automated Storage Tiering for Virtual Pools. A feature of EMC Symmetrix VMAX that automatically optimizes performance in a tiered storage environment. FAST VP reduces costs, footprint, and management efforts and puts the right data in the right place at the right time. FAST VP maximizes the utilization of Flash drive capacity for high IOPS workloads and SATA drives for capacity intensive applications. Lightweight Directory Access Protocol Resource pool Storage Resource Management Suite vapp vcd OvDC PvDC VMware vcenter Chargeback Manager VMware vcops A logical abstraction for the flexible management of CPU and memory resources The Storage Resource Management Suite provides visualizations and reporting solution for optimizing storage resources, managing service levels and improving productivity. Virtual application. A virtual system that contains one or more individual virtual machines and system parameters that define operational details vcloud Director. Software that enables customers to build a secure, multi-tenant cloud by pooling infrastructure resources into virtual data centers and enabling those resources to be consumed by users on demand Organization Virtual Data Center. A vdc abstraction, which is partitioned from a PvDC that maps resources to an organization Provider Virtual Data Center. A vdc abstraction that combines the computer and memory resources of a single vcenter Server resource pool with the storage resources of one or more datastores connected to that resource pool VMware vcenter Chargeback Manager is a software component of vcloud Suite that allows you to customize rate cards and prices according to the processes and policies of different organizations. The VMware vcenter Operations Manager software manages the operation of vsphere and physical and cloud infrastructures 12

13 Term VMware vsphere DRS VSI Definition The VMware vsphere Distributed Resource Scheduler continuously monitors the utilization across vsphere servers and intelligently allocates available resources among virtual machines according to business needs Virtual Storage Integrator, a VMware vcenter server plug-in 13

14 Key components Overview EMC Symmetrix VMAX EMC PowerPath/VE EMC VSI This paper introduces the following technology components: EMC Symmetrix VMAX EMC PowerPath /VE EMC VSI EMC Storage Resource Management Suite VMware vcloud Director VMware vcloud Networking and Security VMware vcenter Chargeback Manager VMware vcenter Operations Manager The EMC Symmetrix VMAX with the Enginuity operational environment is built to support enterprise customers top IT priorities, including the transformation to the private cloud and big data processing. The EMC Symmetrix VMAX system builds on the revolutionary scale-out Virtual Matrix Architecture that features unprecedented performance, availability, functionality, and cost efficiency. EMC PowerPath /VE for VMWARE vsphere delivers multipathing features that optimize VMware vsphere virtual environments. PowerPath/VE installs as a kernel module on the VMware ESXi host and works as a multipathing plug-in (MPP) that provides enhanced path management and load-balancing capabilities for ESXi hosts. EMC VSI provides multiple feature sets including Storage Viewer (SV), Path Management, and Unified Storage Management. It facilitates the discovery and identification of all EMC storage devices allocated to VMware ESXi hosts and virtual machines. Unified Storage Management simplifies the provisioning of Symmetrix VMAX virtual pooled storage for data centers, ESXi server, and resource pools. Path Management allows you to control how users access datastores. 14

15 EMC Storage Resource Management Suite The Storage Resource Management Suite provides visualizations and reporting for understanding and managing the impact storage your application services. Storage Resource Management Suite enables you to better: View the performance and health status of your storage environment in real time Monitor storage service levels by customizing service-level agreement (SLA) dashboards and setting alerts when service levels are about to be breached Optimize capacity to align with performance and business requirements Reduce application downtime by being aware of the relationships and dependencies between applications and storage Align cost of service delivery with business requirements using charge back reports Continuously validate compliance of the storage environment with design best practices and the EMC Support Matrix (ESM) Analyze performance from the application/host perspective and correlate it with historical storage metrics to identify workload patterns and their impact The Storage Resource Management Suite, delivers new insights into the storage environment to better understand the impact storage has on application services. It enables storage administrators to visualize application to storage dependencies, analyze configurations and capacity growth, and optimize their environment to improve return on investment. By using the suite, storage administrators can assure storage services by providing end-to-end performance analysis, monitoring, alerting, and configuration analysis. With this increased visibility, storage teams are in a better position to optimize storage utilization to avoid unnecessary purchases and justify new ones while meeting SLAs. The suite provides this visibility across datacenters for heterogeneous storage (EMC and third party) environments. VMware vcloud Director VMware vcloud Director orchestrates the provisioning of software-defined data center services as complete virtual data centers that are ready for consumption in a matter of minutes. Virtual data centers provide virtualized computing, networking, storage, and security. Administrators can provision the complete set of services necessary to make workloads operational in minutes. Software-defined data center services and the virtual data centers fundamentally simplify infrastructure provisioning, and enable IT to move at the speed of business. By dramatically simplifying the provisioning of these services, VMware enables IT to provision a complete and operationally ready infrastructure without worrying about the physical configuration of its hardware. VMware vcloud Networking and Security VMware vcloud Networking and Security is the leading software-defined networking and security solution that enhances operational efficiency, unlocks agility, and enables extensibility to rapidly respond to business needs. It provides a broad range of services in a single solution, including virtual firewall, VPN, load balancing, and VXLAN extended networks. 15

16 VMware vcenter Chargeback Manager VMware vcenter Chargeback Manager improves utilization of your virtual infrastructure with accurate visibility into the true costs of virtualized workloads. This software enables line-of-business owners to have full cost transparency and accountability for self-service resource requests. VMware vcenter Chargeback Manager allows you to customize rate cards and prices to the process and policies of different organizations. Virtual machine resource consumption data is collected from VMware vcenter Server, ensuring the most complete and accurate tabulation of resource costs. VMware vcenter Operations Manager VMware vcenter Operations Management Suite provides automated operations management using patented analytics and an integrated approach to performance, capacity and configuration management. vcenter Operations Management Suite enables IT organizations to get better visibility and actionable intelligence to proactively ensure service levels, optimum resource usage, and configuration compliance in dynamic virtual and cloud environments. 16

17 EMC storage platform key values Symmetrix VMAX The EMC Symmetrix VMAX with Enginuity operating environment builds on EMC s industry-leading Symmetrix storage array platform for powerful, trusted, smart storage that provides a higher level of performance, availability, and intelligence in the virtual data center, as described in Table 4. Symmetrix VMAX system offers a broad array of functionality and tools that simplify storage management and reduce costs to help accelerate the transformation to the private cloud. Table 4. EMC Symmetrix VMAX with Enginuity Storage feature Powerful Trusted Smart Description Symmetrix VMAX is a distributed multi-engine storage system that can non-disruptively scale from one to as many as eight VMAX Engines. It supports millions of virtual machines, high-demand online transaction processing (OLTP) and online analytical processing (OLAP) for enterprise data warehousing (EDW), and business intelligence (BI) applications. VMAX 10K supports up to 4 engines, 512 gigabyte (GB) memory, and 1.5 PB capacity VMAX 20K supports up to 8 engines, 1 TB memory, and 2 PB capacity VMAX 40K supports up to 8 engines, 2 TB memory, and 4 PB capacity Six-nines application availability 24x7xForever operations Robust security with secure audit, secure access, and data encryption for cloud environments Automation and efficiency in tiered storage environments with FAST VP Trusted Symmetrix services to consolidate and simplify operations with Federated Tiered Storage Ease-of-management and integration into virtual environments with EMC Unisphere for VMAX EMC FAST VP EMC FAST VP optimizes array performance across all drive types (Flash, SAS, and NL SAS disk) in the array to improve system performance while reducing cost. FAST dynamically allocates workloads based on the configured service level, and nondisruptively moves workloads across storage types, optimizing overall system performance. FAST VP moves the most active parts of your workload to highperformance flash disks and the least frequently accessed data to lower cost drives, leveraging the best performance and cost characteristics of the different drive types. Compared to a traditional single-tier Fibre Channel solution, FAST VP provides higher performance but requires fewer drives, reducing energy and space requirements, helping to accelerate the transformation to a cloud environment. FAST VP real-time performance monitoring and intelligent algorithms enable the VMAX storage arrays to relocate chunks of data as small as 7.6 MB. This very granular analysis and data movement enables cost-efficient and high-performing 17

18 configurations, continually tuned to the performance needs of the applications running on the storage array. From previous proven SAP solution, you may see: Twenty-two percent CAPEX reductions A multi-tier Flash/SAS/NL SAS configuration with FAST VP results in a 22 percent cost reduction per gigabyte in storage capital expenditure (CAPEX) without compromising system performance or throughput. This figure was derived by evaluating and comparing the costs of the storage tier configurations with and without the use of FAST VP. Thirty-seven percent OPEX reductions A multi-tier Flash/SAS/NL SAS configuration with FAST VP results in 37 percent power consumption cost reduction per gigabyte in operational expenditure (OPEX). This figure was derived by using the EMC Power Calculator web-based tool to evaluate and compare the costs of the storage tier configurations with and without the use of FAST VP. Automated and optimized storage management with sustained performance FAST VP provides an automated solution to simplify optimization of storage resources while meeting the performance requirements of a realistic SAP workload. Storage is managed as pools instead of LUNs tracked with spreadsheets, and storage can be provisioned quickly with no downtime for data growth. To learn more about the validation, refer to the Increase Efficiency for SAP Landscape with Symmetrix FAST VP White Paper. Federated Tiered Storage Federated Tiered Storage (FTS) enables the on-premises private cloud by creating cooperating pools of resources and by enabling dynamic application and data movement. FTS technology allows you to leverage the benefits of EMC s trusted innovations across the entire private cloud by consolidating, optimizing, and sharing both EMC and non-emc storage resources. Using FTS, you can integrate legacy arrays to maximize efficiency, protect your storage investment, and simplify the management of your storage landscape. FTS provides the capability to put heterogeneous arrays behind VMAX to consolidate and simplify operation in the private cloud. With FTS, customers can use a trusted Symmetrix feature such as FAST VP, Symmetrix Remote Data facility (SRDF), and VLUN to extend the life and value of existing storage arrays in the data center. In a FAST VP configuration, the external array can be used as a lower tier. FAST VP will automatically move data between VMAX and external storage as additional performance or reduced cost is required from the application. 18

19 Host I/O Limits VMAX Host I/O Limits enable users to set maximum performance quotas for IOPS and throughput on a per-storage group basis. This ensures that each application does not affect the performance of other applications. Host I/O Limits enable: Users to provide greater levels of control on performance allocation in multitenant environments Service providers to articulate service-level characteristics, manage expectations, and sell new services The VMAX monitors and manages I/O on each director in the masking view to ensure that the performance limit is not exceeded. When the limits are reached, VMAX slows I/O to discourage additional transactions from that storage group. Management VMware integration EMC Unisphere for VMAX is an intuitive management interface that allows IT managers to dramatically reduce the time required to provision, manage, and monitor Symmetrix storage assets. Unisphere delivers the simplification, flexibility, and automation that are key requirements to accelerate the transformation to the private cloud. With the Performance monitoring options, Unisphere for VMAX provides tools for performing analysis and monitoring historical trending of Symmetrix system performance data. VMAX supports VMware vstorage APIs for Array Integration (VAAI), which offloads virtual machine operation to the array to optimize server performance. In the VMware environments, VMAX 40K supports the following VAAI components: Full copy: Offloads replication to VMAX to enable up to 10 times faster virtual machine deployments, clones, snapshots, and VMware Storage vmotion. UNMAP: Enables more efficient space usage for virtual machines by reclaiming space on datastores that is unused and returns it to the thin provision pool from which it was originally drawn. UNMAP is only supported with vsphere 5.01 and higher. VMware vsphere Storage APIs for Storage Awareness (VASA): Allows VMware administrators to view VMAX driver types (Flash, SAS, or NL SAS) as needed with VASA support. VMware administrators have a single management view into both the virtualization and storage infrastructures. Block zero: Initializes file system block and virtual disk space more rapidly, with as much as 10 times less I/O for virtual machine file system (VMFS) formatting and relocation. Hardware-assisted locking: Enables more efficient Meta data updates and assists virtual desktop deployments, allowing up to 10 times more virtual machines per data store. 19

20 Virtual provisioning Symmetrix virtual provisioning can reduce cost, improve capacity utilization, and simplify storage management. Users can present a large amount of capacity to a host and then consume space only as needed from a shared pool. This improves TCE for reducing initial over-allocation of storage capacity. Symmetrix virtual provisioning also offers automated pool rebalancing, helping to expand thin pools in small increments while protecting performance, as well as non-disruptive shrinking of thin pools to reuse space to improve capacity utilization. Virtual provisioning can reduce labor costs by simplifying data layout and reducing the steps required to accommodate capacity growth. 20

21 Validated landscape Overview We validated the foundation bundle s functions in the infrastructure and software environment shown in Figure 2. Environment architecture Physical architecture The foundation bundle is designed to run in a single site with basic cloud-ready functions that enable the virtual data center. You can seamlessly integrate the add-on bundles to the foundation bundle at a later time to configure a full cloud-enabled infrastructure. Figure 2 illustrates the foundation bundle s physical architecture. Figure 2. Physical architecture of the foundation bundle 21

22 Hardware resources Table 5 lists the hardware resources used in this solution. Table 5. Hardware resources Hardware Quantity Product configuration Storage 1 EMC Symmetrix VMAX Compute 6 ESXi servers: 2 for Management Cluster, 4 for Resource Cluster 2 x four-core CPUs, 128 GB RAM Network switching and routing platform 2 10 GbE (gigabit Ethernet) fabric switch 2 SAN fabric switch Logical architecture Figure 3 depicts the logical architecture diagram. Figure 3. Logical architecture of the foundation bundle 22

23 We suggest separating the virtual infrastructure into two cluster groups: The management group hosts all virtual machines used for cloud-enabled infrastructure management. This enables functions such as multi-tenancy (VMware vcloud Director), monitoring (EMC Storage Resource Management Suite, vcops, and SAP Solution Manager), networking, security (vcloud networking and security), and chargeback (vcenter Chargeback Manager). The resource group hosts all SAP systems consumed by different organizations to support business needs. Software resources Table 6 lists the hardware components used in this solution. Table 6. Software components Software Version Description Active Directory Windows 2008 Central hostname and user management EMC PowerPath/VE 5.8 Multipathing software EMC ProSphere 1.6 Storage management software EMC Enginuity 5876 Operating environment for Symmetrix VMAX EMC Solution Enabler 7.5 Symmetrix command line interface EMC Storage Configuration Advisor Configuration software EMC Watch4Net 6.1u3 Enterprise reporting and analysis tool EMC Unisphere VMAX management software SAP ERP 6.0 EHP 4/NW 7.01 SP 10 IDES/Unicode SAP Solution Manager 7.1/NW 7.3 SAP Management tool Oracle Database Used on all SAP systems VMware vsphere 5.1 Hypervisor hosting all virtual machines VMware vcenter Chargeback Manager Chargeback software Microsoft SQL Server 2008 R2 Used by VMware vcenter Chargeback Manager and vcloud Director VMware vcenter Operations Manager 5.6 User interface and analytics virtual machine VMware vcenter Server 5.1 Managing resource and management clusters VMware vcloud Director Multi-tenancy software component of the VMware vcloud suite 23

24 Software Version Description VMware vcloud Networking and Security Security software SUSE Linux Enterprise 11 SP2 Operating system for SAP 24

25 Key functional design and configuration Overview Virtual data center for SAP The foundation bundle of a cloud-enabled infrastructure is a collaboration of functions that delivers essential characteristics of a private cloud including: Virtual data center for SAP Infrastructure chargeback Integrated performance analysis Fully Automated Storage Tiering Cloud networking and security for SAP The virtual data center aggregates compute, memory, storage, and network resources into resource pools for SAP systems. In this section, we describe the service level and corresponding virtual data center design. Service levels When enterprises transform their traditional data centers into a cloud-enabled data center, the first step is to define service levels that suit various SAP requirements for each organization. The service levels presented in this section serve as a reference for building a differentiated service model. Service levels and their inherent virtual data center constructs provide an effective means of creating service differentiation within a broader vcloud service landscape. They deliver consistent service levels that invariably align with the unique business requirements of individual organizations, which are represented in our solution. According to VMware vcloud Architecture Toolkit (vcat) 3.1, the following is a summary of these service levels: Basic Based on the Pay As You Go allocation model, this service level lends itself to quick-start pilot projects or test and development application workloads that typically do not require persistent resource commitments or upfront resource reservations. Committed Based on the Allocation Pool allocation model, this service level provides consumers with a minimum initial resource commitment and the ability to exceed that minimum, with additional capacity available as needed. This minimum commitment level, expressed as a percentage of overall capacity, accommodates the customer s varying application workloads when needed. Dedicated Based on the Reservation Pool allocation model, this service level provides a reserved resource capacity up front, fully dedicated to an individual organization. This resource level guarantees a high degree of service that assures the customer that the capacity required by their workload is available. 25

26 To help you decide which service level makes the most sense for your (or your customer s) business needs, refer to Table 7. Table 7. Key service level attributes Service attributes Service level Basic Committed Dedicated Target workloads Test and development Tier 2 and tier 3 production Tier 1 production Workload examples QAS and DEV systems Training systems Sandbox systems Lightly used SAP systems Moderate IOPS systems High IOPS systems CPU-intensive systems Standalone database Compute resource Allocation model Pay As You Go Allocation pool Reservation pool Compute resource Limit CPU is not guaranteed, memory is guaranteed at high percentage CPU and memory is guaranteed at high percentage CPU and memory is fully guaranteed as allocated Charge mode Usage Allocation Physical server resource Low-end physical server models Latest high-end physical server models Storage resource Medium performance High performance We designed the following components to implement the service levels: vcloud Director: PvDC, OvDC vsphere: DRS, storage profile, storage DRS, networking Designing vcloud Director The virtual data center is a new construct that represents the standard container for a pool of compute and storage resources. There are two types of virtual data centers: Provider (PvDC) and Organization (OvDC). This section focuses on PvDC and OvDC design. PvDC A PvDC is composed of resource pools and datastores from one or several vcenter Servers. A PvDC is the source for Organization vdcs. This solution consists of two PvDCs: Standard The standard PvDC is built on distributed resource scheduler (DRS) clusters with low-end ESXi servers, which may come from existing physical servers or as replacements for high-end physical servers. Storage in the standard PvDC is categorized as Bronze tier, with relatively low performance compared to advanced PvDC. QAS/DEV SAP systems are hosted on this standard PvDC. 26

27 Advanced The advanced PvDC is built on DRS clusters with high-end ESXi servers. Storage in advanced PvDC is categorized as Silver tier and Gold tier. Both Silver and Gold tier storage offer high performance that is suitable for production SAP systems. Gold tier storage uses more enterprise Flash drive capacity than Silver tier storage, making it best for high-iops SAP systems. The advantages of separating PvDC into Standard and Advanced are: Separation of non-productive SAP systems from productive SAP systems on different DRS clusters ensures enough resources for the PRD SAP systems when needed. You can reuse existing x86 servers to construct a standard PvDC to protect your investment. You can replace older servers in an advanced PvDC with new models. You can then transition the old models to a standard PvDC to support QAS/DEV SAP systems. OvDC An Organization vdc allocates resources from a PvDC and makes it available for a given organization. Multiple Organization vdcs can share the resources of the same PvDC. The Allocation model is a core concept in vcloud Director 5.1 that determines how and when the vcloud compute and memory resources that you allocate are committed to an organization. There are three allocation models: Reservation Pool Model Reservation Pool resources allocated to the organization vdc is completely dedicated. All guarantees are set to 100 percent. Reservation Pool vdcs map to resource pools with the reservations set equivalent to the limits. Allocation Pool Model An Allocation pool is a pool of allocated resources with a certain percentage of resources guaranteed. When an organization vdc is created using the Allocation Pool Model, a dynamic resource pool is instantiated. This resource pool automatically adjusts available resources as new workloads are powered on, based on the values specified within the OvDC. Each value has a direct impact on how the related resource pool dynamically changes as new virtual machines are deployed. Pay As You Go Model The Pay As You Go model provides the illusion of an unlimited resource pool. This model maps to a sub-resource pool with no configured reservations or limits. Resources are committed only when vapps are deployed in the organization vdc. For more information about allocation models, refer to the vcloud Director Administrator's Guide and vcloud Architecture Toolkit 3.1. To demonstrate the vcloud Director design, we introduced the following business scenarios as an example: ABC Corporation adopts the solution to transform their traditional IT infrastructure running SAP to a cloud infrastructure. There is a centralized IT in ABC Corporation that manages the cloud infrastructure and provides IT related services for its organizations. 27

28 Cloud-enabled infrastructure resources are shared between organizations of ABC Corporation. Org A requires Committed and Basic service levels for their PRD/QAS/DEV SAP systems. Org B requires Dedicated and Basic service levels for their PRD/QAS/DEV SAP systems. Figure 4 shows the high-level architecture of the vcloud Director design. Figure 4. vcloud Director design 28

29 OvDCs are mapped to service levels according to organization requirements as shown in Table 8. Table 8. OvDC organizational mapping Organization vdc Service level Compute resource allocation model SAP system OrgA_Basic Basic Pay As You Go QAS/DEV OrgA_Committed Committed Allocation Pool Medium workload OrgB_Basic Basic Pay As You Go QAS/DEV OrgB_Dedicated Dedicated Reservation Pool Dedicated service Designing the vsphere This solution includes the following vsphere design components: DRS cluster Storage profile Storage DRS cluster Networking DRS cluster In this solution, we used two separate DRS clusters to support two PvDCs: one standard cluster and one advance cluster. VMware recommends that for all DRS clusters, you: Enable the High Availability (HA) feature to protect all virtual machines against ESXi host failures. Configure the Automation Level as fully automated so that vsphere can balance the workload automatically. When building DRS clusters for the cloud-enabled infrastructure, you should: Use the same ESXi server model in a cluster with the same type and amount of CPU, memory, NICs, and HBAs to simplify deployment and management. Configure multiple smaller clusters rather than a single cluster to achieve better performance. Avoid setting DRS affinity rules with Virtual Machines to Hosts, as this may cause HA failure. 29

30 Storage profile vcloud Director leverages storage profiles defined in vsphere as storage resources. By using profile-driven storage, you can define various storage characteristics as tiers and apply them to individual virtual machines. FAST VP policies on VMAX provide three storage tiers to meet different performance requirements: Bronze High Capacity datastores leveraging mix of SAS and NL SAS storage Silver High performance datastores with a mix of Flash, SAS, and NL SAS storage Gold Highest performance datastores leveraging Flash and SAS storage The detailed storage and FAST VP design on VMAX is covered in the Fully Automated Storage Tiering section. We defined three storage profiles to correspond with the defined FAST VP policies, Bronze, Silver, and Gold. Each storage profile maps to a user-defined storage capability. You can define the storage profile of a datastore automatically through VMware vstorage APIs for Storage Awareness (VASA) or you can manually define the storage profile for each datastore. In this solution, we manually defined the storage profile of each datastore. Refer to vsphere Storage Guide 5.1 for detailed procedures for configuring storage profiles. We defined the storage profile of the SAP DEV/QAS applications as Bronze because these applications do not typically demand high performance. We used Gold and Silver storage profiles for SAP PRD applications that typically demand higher performance levels. Datastore cluster Datastore clusters form the basis of storage DRS. As with a cluster of hosts, a datastore cluster is used to aggregate storage resources, enabling smart and rapid placement of new virtual machines and virtual disk drives and balancing existing workload. We created three datastore clusters (gold, silver and bronze) in accordance with the storage profiles. We set the DRS storage automation level to NO Automation to avoid automatic storage migration activities that will affect performance. Storage DRS then determines the initial placement of the virtual machine storage, leaving the performance optimization to EMC FAST VP. We disabled the Storage DRS I/O metrics as shown in Figure 5. This allows SDRS to automatically perform capacity balancing and FAST VP to automatically relocate data on the EMC VMAX array. This simplifies the storage view from the vcloud Suite higher up the stack, and enables an efficient provisioning model that is easily consumed by the vcloud Suite. 30

31 Figure 5. Disabled I/O metric in Storage DRS settings Networking vsphere networking is the foundation of vcloud Director. In This solution, each ESXi host has four network adapters: Two 1 Gb Ethernet adapters are teamed together and are configured with a port group for Internet access using a standard switch. Two 10 Gb Ethernet adaptors are teamed together and are configured with multiple port groups for virtual machines, vmotion, management, an external network for client access, and an administration network using a distributed switch. Figure 6 depicts the network design. Figure 6. Network design in vsphere 31

32 Table 9 describes each port group defined in Figure 6. Table 9. Network details Network name Port group name VLAN Traffic type Usage Public network Public Network 10 Virtual machine Internet access vmotion network dvpg_vmotion 11 vmotion vmotion traffic Host network dvpg_host 12 Host Management For communications among ESXi hosts, and between ESXi hosts and the vcenter server Management network dvpg_management 100 Virtual machine Management virtual machine Corp network dvpg_corp 101 Virtual machine Corporate internal network for communication between SAP systems and client hosts Admin Network dvpg_corp_admin 102 Virtual machine Corporate internal network for privileged communications between SAP systems and SAP administrators For better network utilization and balance of network adapter bandwidth, we recommend that you enable Network I/O Control (NIOC) on the distributed switches to guarantee sufficient bandwidth for SAP systems. Specify the following share values for the corresponding traffic types: Fault Tolerance/vSphere Replication/NFS/iSCSI = 1 Management Traffic = 5 vmotion Traffic = 20 Virtual Machine Traffic = 10 We set the FT/VR/NFS/iSCSI share values to the minimum value because this solution does not use them. When a vapp is deployed, its virtual machine network traffic share value is set according to the Virtual Machine Traffic settings. To minimize the latency in the process of DRS cluster balancing, when configuring the Network I/O control settings, be sure to set a higher share value for vmotion traffic and iscsi/nfs traffic than other traffic values. The above configuration is for illustration purposes only, and should not be considered as the default standard configuration used in all environments. To determine the share values, plan the configuration carefully. For more details, refer to vsphere Networking Guide vsphere

33 Infrastructure chargeback Overview In this solution, vcenter Chargeback Manager provides the virtualized infrastructure metering functionality. It integrates with VMware vcloud Director, providing the service provider with the ability to track and charge for the resources used by customers, and provides the serviced consumer with the ability to track resource usage and generate a cost reports when needed. vcenter ChargeBack Manager design In vcenter Chargeback Manager, we defined the cost elements required to support the service level listed in Table 7, including: Billing policy Defines how to measure and charge for each compute resource Pricing model Defines the base computing resource rate and binds that rate with the relevant billing policy Infrastructure specific cost Defines the rate factor on specific infrastructure components, such as different storage tiers Cost template Defines the rate factor on specific compute resources for the purpose of discounts, promotion for an individual vapp, or organizations defined in the cloud hierarchy. Entity cost Binds the price model to a specific entity Fixed cost Defines the costs that Chargeback Manager cannot meter as resource usage but must be factored into the price, such as software licenses and maintenance fees. Defining a billing policy We used vcenter Chargeback Manager to measure usage of the computing resources listed in Table 10. Table 10. Resource CPU Memory Storage vcpu Computing resource and measurement Measurement The CPU usage is measured in GHz The memory usage is measured in GB The storage usage is measured in GB The number of virtual CPUs in the virtual machine External Network Transmit External Network Receive The external network usage for uploads measured in MB The external network usage for downloads measured in MB 33

34 In Chargeback Manager, each computing resource can be charged using different attributes as shown in Table 11. Table 11. Attributes Usage Resource attributes Description The virtual machine s resource utilization. This attribute can be used only with CPU, memory, storage, disk I/O, network I/O, vcpu, and external network transmit and receive computing resources. For storage, the usage attribute indicates the total provisioned storage space for the virtual machine. For vcpu, the usage attribute indicates the vcpu count. Reservation Allocation Size The amount of resources reserved for the virtual machine. The amount of resources allocated for the virtual machine. This attribute is useful when you want to enforce a minimum amount of resource to be charged, irrespective of the actual resource utilization or the resource reservation. This attribute can be used only for memory. You should name your billing policies according to service level, for example: Billing policy Basic: Set CPU=Usage, Memory=Usage Billing policy Committed: Set CPU= Allocation, Memory= Allocation Billing policy Dedicated: Set CPU= Allocation, Memory= Allocation 1 Pricing model We suggest that you associate different pricing models with billing policies and set base rates for CPU/memory/network consumption. The base rate can differ depending on the cost variation of its DRS cluster. Table 12 provides an example pricing model. Table 12. Pricing model settings Pricing model Billing policy Base rate PM_Foundation Basic Billing Policy Basic CPU: $3/GHz Memory: $0.5/GB PM_Foundation Committed Billing Policy Committed CPU: $6/GHz Memory: $1/GB PM_Foundation Dedicated Billing Policy Dedicated CPU: $8/GHz Memory: $1.5/GB Note: The rates are samples and subject to change in each customer environment. 1 The parameter setting is based on VMware recommendation for vcloud director. 34

35 Define infrastructure specific cost We created different storage profiles by associating with different FAST VP policy in the datastore level. You can define different rate factors for each storage profile in addition to the allocation model as shown in Table 13. Table 13. Infrastructure cost setting for datastores Infrastructure cost datastores Rate factor Foundation_Bronze 1 Foundation_Silver 1.5 Foundation_Gold 2 Defining cost template You can define different cost templates with different rate factors for all kinds of resources. Apply the cost template to the entity for a specific period. You can customize it for a promotion or special discount for individual organizations or virtual machines, as shown in Figure 7. Figure 7. Define cost templates Defining entity cost After defining the cost elements, we bound them to a vcloud Director-synchronized hierarchy, as shown in Figure 8. When you bind the pricing model to the OvDC level, it automatically binds to all the associated vapps. The entity cost takes the highest precedence over any rate factors that are defined elsewhere. Figure 8. Defining the entity cost 35

36 Defining fixed cost You can use fixed cost to meter the cost that cannot be measured directly, such as application license fees and hardware maintenance fees. You can define it as a onetime charge or a recurring charge. After defining the elements, you can generate a cost report. A cost report provides the cost and utilization information for each computing resource in the organizational hierarchy. You can also automatically generate on-demand cost reports and distribute them to users through the SMTP interface. Integrated cloud management performance analysis Overview VMware vcenter Operations Manager together with the EMC Storage Resource Management Suite provide role-based dashboards and lead to an optimal team collaboration, as shown in Table 14. The combined software solutions provide efficient management of availability, performance, capacity, and health of the EMC cloud-enabled Infrastructure for SAP in part or as a whole. This leads to fewer incidents, less downtime, fewer penalties and avoids multiple handoffs. In addition, you can identify unused applications and underutilized hardware. Figure 9 presents an overview of the monitoring strategy in the foundation bundle. The SAP administrator monitors the application performance and availability through SAP Solution Manager. The administrator can log in to vcenter Operations Manager and EMC SRM with read-only authorization to speed up the root cause analysis process. The cloud administrator is responsible for monitoring and managing the underlying infrastructure and takes care of the vcloud director configuration. SAP administrators can serve individual or several organizations, as needed. 36

37 Figure 9. Monitoring strategy for the foundation bundle 37

38 Table 14 shows an example of role-based user profiles in the Cloud-enabled Infrastructure. Table 14. Cloud-enabled Infrastructure user profile Component Component authorization Organization end user SAP system X X Enterprise SAP administrator Enterprise Cloud administrator SAP Solution Manager EMC Storage Resource Management Read/report X X X Configure X Alert X X vsphere VM-OS X X Server Network Storage X X X vcloud Director Read X X Ordering resources X X Configure X vcenter Operations Manager Read/report X X Configure X 38

39 Table 15 lists the monitoring functions used in this solution. Table 15. EMC SRM and VMware vcops monitoring functions Function VMware vcops EMC Storage Resource Management Suite Performance Storage capacity Reporting Alert Performance data at the different levels include: Data center Cluster ESXi server Virtual machine level Datastore Metrics: CPU usage Disk I/O (virtual disk) Memory usage Network workload Cluster or host capacity Virtual machine capacity Datastore capacity Metrics: CPU Memory Disk I/O Disk space Network I/O Host utilization report Cluster or host capacity inventory report Virtual machine capacity overview report: Virtual machine list report Waste, stress, and capacity report Oversized virtual machines report Badge score alerts Fault alerts Administrative alerts or SNMP notifications EMC storage performance: LUN throughput, IOPS, response time FE Director, FE IOPS (VMAX) SAN switch performance: Port throughput/port link Port utilization Host performance: Host IOPS, response time, throughput, queue length Port link utilization Host CPU utilization Virtual machine performance: Host devices IOPS, throughput Port link utilization Host CPU utilization Raw capacity usage Pool capacity LUN capacity Service level: By array type By FAST VP By disk characteristics or RAID type Storage real time and historical report Storage capacity and performance report Trending report Enterprise-wide SLA report Change tracking and reporting VMAX health Switch health EMC storage performance thresholds or SNMP notifications 39

40 This chapter discusses the integrated management, which allows: Full visibility of the virtual data center End-to-end performance analysis Configuration validation and compliance Storage capacity analysis Predefined and customized intelligence reports Full visibility of the virtual data center VMware vcenter Operations Enterprise is designed to simplify and automate operations management. It provides the automated performance analytics for enterprise IT systems and business processes. It also integrates patented analytics to provide the intelligence and visibility needed to ensure service levels are met in dynamic virtual and cloud environments. VMware vcenter Operations offers the following values: Real-time performance monitoring and analysis Displays role-based performance dashboards Smart alerts Informs you of impending problems so you can avoid major performance or capacity issues On-demand analytics and reporting Provides capacity planning and optimization for long-term enterprise systems Real-time performance monitoring and analysis The VMware vcenter Operations Enterprise role-based performance dashboards leverage the analysis of individual metric behaviors to determine the health of an enterprise in part or as a whole. The flexibility of the vcenter Operations Enterprise architecture allows you to automatically calculate the health of any desired container of resources (for example, the individual servers and devices or the combined health of a collection of resources, including technology silos such as an SAP tier that contains database, central instance, dialog instance, and so on). Figure 11 shows an example of a customized dashboard for the data center administrator: In the vcenter relationship widget, the environment has two data centers with six ESXi servers hosts, 68 virtual machines, and 30 datastores. The dashboard shows that: One virtual machine is very busy (red) Some virtual machines are shut down (gray) Some datastores are undergoing a heavy workload (yellow) In the ESXi server health scoreboard, the status of ESXi servers is displayed as: Green means a healthy status Orange means there are serious problems 40

41 The virtual machine/datastore workload and response time heat map clearly shows how the workload is allocated across the data center. In this example, size represents workload and color represents the response time. In the vcloud Director heat map, size represents each organization s CPU usage, while color represents the CPU contention. The data center administrator can customize the resource scoreboard and the sapvsedb scoreboard. Figure 11. vcenter Operations Manager performance dashboard Smart alert When vcenter Operations Enterprise detects significant abnormal behavior exceeding its expected noise level, a smart alert pops up to prevent impact to end users. You can monitor Smart Alerts in a variety of ways, including viewing the vcenter Operations Enterprise alert summary dashboards or using notifications. 41

42 Figure 12. Smart Alerts example On-demand analytics and reporting The VMware vcenter Operations on-demand analytics and reporting offers the ability to gain a deep understanding of the known behavior patterns of your applications and supporting IT systems. These features enable the cloud administrator to optimize the operation infrastructure that supports mission-critical applications. Figure 13 shows an example of how the most problematic or most volatile resources manifest in vcenter Operations Enterprise. In this example, the vcenter Operations Manager can determine that the virtual machine SAPVM3SQLCEN is the root cause of the problem. You can use this tool to determine not only the problematic resources, but also the technology tiers and applications that are causing problems. Figure 13. On-demand analytics showing the least healthy or most volatile resources Cloud capacity usage is critical for both real time monitoring and historical reporting that determines future hardware expenditures. Capacity usage reporting can help cloud users determine the status of their resource pools, including: Undersized or oversized virtual machines Virtual machine resources that are nearing depletion 42

43 Figure 14 illustrates the report showing oversized and undersized virtual machines with recommended vcpu and memory. Figure 14. Oversized and undersized virtual machines report End-to-end performance analysis If the underlying infrastructure fails to meet the workload requirements, the performance of the SAP system will be affected. EMC Storage Resource Management Suite enables you to analyze host, application, and storage performance to identify workloads and assess impacts on the application. SAN performance To address the cloud administrator s need for performance management, EMC SRM provides an end-to-end view, allowing insight into the complex relationships in large virtualized environments. EMC SRM automatically collects performance data for the entire path, including virtual machines, hosts, switches, and storage. It also integrates with EMC element managers, including Unisphere for VMAX and Connectrix Manager Converged Network Edition, to improve productivity. 43

44 Figure 15 shows the end-to-end performance view. Figure 15. EMC SRM end-to-end performance view As shown in Figure 15, customers can easily filter a specific SAP host and view its end-to-end topology. Figure 15 presents a compact, high-level topology map showing the SAN elements connected to the host, while filtering out all non-pertinent data. If a customer wants to see detailed performance or connectivity path information, they can select the object of interest such as the ESXi host, switches, or storage. Consolidated Alert Console EMC Storage Resource Management presents a single view of performance and healthy alerts from EMC Unisphere and CMCNE. Alerts that appear or are updated in EMC SRM can be sent to a problem tracking application through SNMP traps for dispatch to the appropriate organization. As shown in Figure 16, the use space of the virtual pool SAS15K_RAID1 in VMAX has crossed a predefined threshold, and the level is above 80 percent. 44

45 Figure 16. EMC SRM alert view EMC SRM performance thresholds enable customers to set the threshold metrics of select Configuration Items (CI) in the storage environment and to keep track of the resource utilization limit and performance. If thresholds are breached, the software generates warnings or critical alerts and s them to the storage administrator, so problems can be proactively resolved before issues affect users. Figure 17 displays the performance thresholds set for this solution. Figure 17. Setting performance thresholds Configuration validation and compliance EMC SRM helps SAP and cloud administrators manage compliance with internal configurations policies, vendor design and interoperability guidelines, and industry best practices across virtual infrastructures. In this solution, EMC SRM helps to improve service levels and operational efficiency by: Providing best practices policy management Tracking changes Flagging configuration breaches for further investigation 45

46 Best practices policy management EMC SRM validates storage configuration for the cloud-enabled infrastructure best practices. As shown in Figure 18, we created a policy to enforce array configuration best practices, validate Symmetrix port flag settings for VMware, and set notifications when customers breach the policy. This enables virtual data center administrators to provide consistent service to the business for the cloud-enabled infrastructure. Figure 18. Creating a policy to validate Symmetrix port flag settings for VMware Change tracking Figure 19 displays how EMC SRM detects the movement of virtual machines across physical servers and virtual disk configuration changes in a virtualized environment. Administrators can keep an audit trail of any change. This historical record enables administrators to analyze incident causes or trends and improve management processes and planning. 46

47 Figure 19. Virtual machine activity detection Investigating configuration breaches EMC SRM continuously monitors the cloud-enabled infrastructure. If a change differs from the desired state for a rule, EMC SRM triggers a configuration breach. Figure 20 shows how EMC SRM identifies areas where the path of one virtual machine is missing, determines which objects caused the breach, and provides the steps needed to resolve the problem. Administrators can drill down into a breach to investigate its cause, view recommended remedial actions, and add or view notes and tags related to the breach. Figure 20. EMC SRM Investigate Breach Storage capacity analysis EMC SRM allows you to report on several capacity metrics based on specific use cases that reflect the everyday needs of storage administrators. It provides a dashboard view that offers insight into where and how capacity is being consumed and when more will be required. You can monitor storage usage information including: Used versus free storage Total raw storage utilization Array utilization by purpose Array utilization by FAST policy 47

48 Array utilization by tier Array-based storage chargeback Total federated storage capacity Figure 21 shows EMC SRM capacity monitoring. Figure 21. Monitoring storage capacity with EMC SRM Predefined and customized intelligent reports EMC SRM provides an enterprise reporting and analysis tool that provides real-time, historical, and projected visibility into the network, data-center, and cloud infrastructure performance. Administrators can have a complete view of the reports from a single interface. EMC SRM helps virtual data center administrators improve performance and capacity management processes, delivering high quality IT services at more-efficient costs. It is packaged with an extensive set of predefined Report Packs. In this case, EMC SRM provides the following reports: Real-time and historical report Investigate trending report SLAs reports 48

49 Real-time and historical report EMC SRM monitors the performance, utilization, and capacity of virtual data center components in real time and provides a historical report as shown in Figure 22. In this case, we can make informed virtualization decisions by gathering configuration and performance data on virtualized environments. Figure 22. Real time and historical reporting We can also schedule reports to send at fixed times to a list of recipients. Administrators can choose a format for the report as shown in Figure 22. Trending report Trending reports based on key metrics can help virtual data center administrators avoid SLA violations. As shown in Figure 23, we can identify potential issues before they occur and plan growth with accurate data using the trending reports. 49

50 Figure 23. Trending and Forecasting report and configuration SLA reports Virtual data center administrators can quickly measure operational results, check compliance with contracted service levels, and document SLA reports through EMC SRM. Figure 24 shows the actual and forecasted SLA value of the devices in the virtual data center. EMC SRM identifies SLA issues with check marks and color changes, and reduces the time spent monitoring service levels. Figure 24. Monthly SLA overview 50

51 Fully Automated Storage Tiering FAST VP allows the cloud administrator to leverage the storage tiering abilities of the storage array. It enables automatic storage tiering below both the virtual machine and VMDK level. It is the storage array rather than vcloud Director that provides the tiering. The storage array is responsible for managing the data placement, moving the data to the appropriate location where the vapps benefit from the performance required. FAST VP maximizes utilization of Flash drive capacity for high IOPS workloads and maximizes utilization of NL SAS drives for capacity-intensive applications. FAST VP provides the following benefits for Cloud-enabled Infrastructures: Service providers can offer a greater range of service levels Service providers can optimize and protect their vcloud Director environments on Flash, SAS, and NL SAS drives Multiple tiers can be aligned and integrated with storage profile VMware vcenter Chargeback Manager can report on and bill for FAST VP enabled storage Service providers can meet a greater number of SLAs by taking advantage of tight integration between EMC and VMware products This section discusses the implementation of FAST VP for the cloud-enabled infrastructure. Table 16 details the tier design. Table 16. Virtual pool configuration and FAST VP tiers Storage pool Disk size and technology RAID protection Tier name Flash drive 200 GB Flash Raid5 (3+1) Flash SAS15K_RAID1 450 GB SAS RAID1 SAS NLSAS_R6 2 TB NL SAS RAID6 (6+2) NL SAS The EMC Solutions Enabler Symmetrix Array Controls CLI v7.5 Product Guide provides detailed information about VMAX virtual provisioning. FAST VP for EMC Symmetrix VMAX Theory and Best Practices for Planning and Performance provides detailed information on best practices and design considerations for FAST VP. We designed FAST VP policies to deliver service levels according to the customer s performance needs. Table 17 shows the FAST VP policies used in this implementation. The FAST VP policy assigned and the storage consumption of a virtual machine serve as the base for the chargeback mechanism described in the Infrastructure chargeback section. 51

52 Table 17. FAST VP policies Policy name Flash tier SAS tier NL SAS tier Gold 15% 85% 0% Silver 4% 28% 68% Bronze 0% 20% 80% Figure 25 shows the storage service-level relationship throughout the cloud-enabled infrastructure. At the storage layer, FAST VP polices define a level of service for sets of LUNs presented to the ESXi clusters. Within VMware vcenter, VMFS datastores are created on the EMC VMAX LUNs and are grouped into storage DRS clusters according to their service-level. VMware ESXi HA/DRS Cluster EMC VMAX with FAST VP FAST Policy High Performance Vmware SDRS Datastore Clusters Gold Medium Performance Redundant FC SAN Silver Bronze Low Performance Figure 25. Service-level relationship throughout the cloud-enabled infrastructure In this solution, we used a standard datastore size of 2-TB thin LUNs as the base configuration for the VMAX storage LUNs presented to ESXi hosts. This provides for up to 1 PB of shared storage per cluster (256 shared datastores). We configured the VMAX volumes as striped Meta devices (128 GB Meta member). We used Unisphere Storage Templates to create the VMAX LUNs. This enables storage administrators to quickly and easily create standard volumes. Figure 26 shows the Unisphere storage template for creating a 2-TB Meta volume. We configured the details of the Meta volume under the Advanced Options, which is a one-time effort that can be reused multiple times. 52

53 Figure 26. Defining a Storage Template in Unisphere for VMAX 53

54 Cloud networking and security for SAP Overview It is a critical requirement for every data center to have a robust networking and security solution. The cloud-enabled infrastructure uses VMware vcloud networking and security, which virtualizes the network and security layer of the IT infrastructure. vcloud networking and security integrates with vcenter and vcloud Director to centralize management and deploy logical constructs for a multi-tenant model, including firewalls and network configurations. In this section, we demonstrate the following design: How SAP system networks are isolated between organizations How client hosts access their own SAP systems while access to other organization s SAP systems is prohibited VMware vcloud networking and security design There are three types of networks in vcloud Director: External network External networks are used to provide a connection outside the vcloud environment. In this solution, external networks are networks that are connected to the corporate network. Through the external network, client hosts, admin hosts, and management servers can connect to SAP systems based on access rules. Organization network We customized the Organization networks according to organization requirements. This configuration ties vapps together to provide interconnection between vapps, outside connections between vapps, and connection to external networks. vapp network We used vapp networks to provide a connection between virtual machines that are within one vapp. Predefined port groups in vsphere back the external networks. Port groups that are created dynamically by vcloud Director back organization networks and vapp networks. You can configure these networks to directly connect to a higher network level, to be routed to a higher network level, or to connect internally only. Figure 27 depicts a reference network design. 54

55 Figure 27. Networking and security design Different installation options are supported in this solution. We used the centralized installation option when deploying SAP systems. Each SAP system is encapsulated into one vapp. Within the vapp, we deployed one virtual machine, with two vnics: one for client access (CL) and one for administration purposes (Admin). The above network configuration is one example of many options and not to be considered the default configuration for all environments. Networking and security should be carefully planned based on the requirements of each organization. The following sections explain how we designed and connected these networks. vapp network Figure 27 shows the vapps directly connected to OrgNet without a vapp network. This is based on the assumption that OrgNet is a trusted network inside the organization and that there is no need to isolate the SAP systems within the organization. VMware recommends that you keep the network configuration simple because the more networks you have, the more complex network management becomes. 55

56 Organization network Table 18 shows the organization networks in this solution along with their network names, IP pools, and gateways. Table 18. Organization networks Organization Network name IP pool Gateway Org A OrgNet Org A OrgNet_Admin Org B OrgNet Org B OrgNet_Admin Two types of organization networks are defined in each organization: OrgNet A network that provides a link between the SAP systems inside the cloud and client hosts outside the cloud. OrgNet also provides interconnections between SAP systems, for example, between DEV and PRD inside the organization. OrgNet is a routed network, which connects to the external network, CorpNet, through the edge gateway. Figure 27 shows Edge 02 and Edge 04 as the firewall and NAT device in each organization respectively. The main functions of these edge gateways are to: Allow only the IP ranges of client hosts from their own organizations to access SAP systems. Block all ports that are not necessary for client hosts to access SAP systems. Open essential ports like SAP message service (port number: 36xx) and SAP dispatch service for client access (port number: 32xx). This protects SAP systems from internal attacks and threats. Translate IP addresses from organization networks to external networks. For example, CL interface of SAP ERP DEV in Org A is configured with IP address on OrgNet. It will be translated to , to which client hosts can connect. OrgNet_Admin A network that provides a link between the SAP systems inside the cloud and admin hosts outside the cloud. The admin host has more privileges than the client host. One of the privileges is direct access to OS through SSH. OrgNet_Admin is a routed network, which connects to the external network CorpNet_Admin through the edge gateway. Figure 27 shows Edge 01 and Edge 03 as the firewall and NAT device in each organization respectively. The main functions of these edge gateways are to: Allow only administrator hosts to access the SAP systems and OS. Block all ports that are not necessary for administrator hosts to access SAP systems. Open essential ports such as SAP message service, SAP dispatch service, and SSH service according to business requirements. 56

57 Translate IP addresses from organization networks to external networks. External network We defined three external networks as shown in Table 19. Table 19. External networks Network name Backed by port group VLAN IP pool Gateway CorpNet dvpg_corp CorpNet_Admin dvpg_corp_ad min CorpNet_Public Public Network The external networks in this solution include: CorpNet A network that allows client hosts to connect to SAP systems. Access rules are defined in Edge 02 and Edge 04, which block interconnections between SAP systems across organizations and allow access from the organization s own client hosts and other management servers like SAP Solution Manager. CorpNet_Admin A network that allows administrator hosts from the SAP basis support team to access restricted resources such as OS and DB. Like CorpNet, access rules are defined in Edge 01 and Edge 03, which block the interconnection between SAP systems across organizations and allow access only from specific admin hosts in centralized IT. CorpNet_Admin is backed by port group dvpg_corp_admin and is a trusted network between the SAP system and administrator hosts. Security devices should be used on this network to protect and isolate them from other external networks on the physical network layer and they should be properly tracked and audited. CorpNet_Public A network that provides Internet access. 57

58 Table 20 describes the IP address range of external networks for each organization. Table 20. External network assignments for organizations Organization Network Name IP Range Org A CorpNet Org A CorpNet_Admin Org B CorpNet Org B CorpNet_Admin Network pool Network pools are used as a template to create networks at the organization and vapp levels. Two types of organization networks require network pools: Routed organization networks that connect to an external network Internal organization networks All vapp networks are built on network pools. See the vcloud Director Administrator's Guide for detailed information about network pools. In this solution, vcloud Director dynamically creates vappnet, OrgNet, and OrgNet_Admin based on the network pool. We used a vcloud Director Network Isolated (VCD-NI) backed network pool to simplify the network design. Due to MAC-in- MAC encapsulation by VCD-NI, be sure to set MTU to 1524 in the following places to avoid package fragmentation and performance downgrade: Distributed switches in vsphere that vcloud networks are using Physical switches that connect ESXi hosts Network pool settings in vcloud Director 58

59 SAP system design Overview Compute This chapter describes the design considerations used to deploy SAP systems in the cloud-enabled infrastructure, including: Compute Storage Network Installation type Allowing CPU and memory over-commitment would boost compute resource utilization but would also result in performance degradation for SAP systems. In general, SAP recommends not overcommitting CPU or memory for the following reasons: For memory resources, the SAP kernel allocates memory permanently and does not release it again. If a memory outage on an ESXi server occurs, vsphere uses the swap file on the datastore as memory, which causes performance degradation. For CPU resources, SAP successfully runs performance tests in virtual machines (utilizing all available virtual CPUs to 100 percent), which overcommits the host system up to 150 to 200 percent. The performance degradation of the virtual machine is linearly reciprocal to the over commitment. In a cloud environment, the different service levels have their own design considerations: For Dedicated service levels, compute resources are fully guaranteed. If the allocation of CPU and memory is 50 GHz and 100 GB, the guaranteed CPU and memory values are 50 GHz and 100 GB. Cloud administrators can set reservations and limitations on each single virtual machine. We recommend you set the "Memory Reservation" to the amount of memory configured for most critical SAP systems running in Dedicated OvDC. For Committed service levels, compute resources are guaranteed at 75 percent of total allocation. This allows 25 percent of over-commitment on CPU and memory. Cloud administrators cannot set reservations and limitations on virtual machines. Since Committed service levels target less critical or low workload production SAP systems, it is reasonable to allow a certain degree of over-commitment, which increases resource utilization and lowers cost. The cloud infrastructure administrator should regularly monitor total resource usage on ESXi servers. If severe resource contention lasts, the cloud infrastructure administrator can increase total allocable resources by assigning additional compute and memory resources. For Basic service levels, memory resource is guaranteed at 75 percent and there is no guarantee on CPU resource availability. This allows maximized compute resource utilization. We recommend you review the-size report in vcops regularly and adjust memory and CPU on each virtual machine to the appropriate value. 59

60 Here are some general guidelines for allocating compute resources: Determine the proper sizing of vcpu before going live. More vcpus in virtual machines can increase scheduling overhead. If the exact workload is unknown, size the virtual machine with a smaller number of vcpus initially and increase the number later if necessary. Storage Even though the current version of vcloud Director provides all storage of an SAP system out of the same storage profile, you still can follow SAP s best practice of file systems set up (especially for critical SAP production systems). The file systems can lie on different VMDK files on single or multiple datastores of the same performance level as shown in Table 21. The additional advantage is: in case of possible changes of vcloud Director in future to also support file systems of an SAP system to be delivered from different performance levels that are already prepared. Table 21. Sample file system configuration for database instance Logical volume Volume group File system fslvorastaging orastagingvg /oracle/stage fslvoraclient oraclientvg /oracle/client fslvorahome orahomevg /oracle/vse fslv vg /oracle/vse/11203 Fslvsaparch saparchvg /oracle/vse/oraarch fslvsaptrace saptracevg /oracle/vse/saptrace fslvorigloga saporalogavg /oracle/vse/origloga fslvoriglogb saporalogbvg /oracle/vse/origlogb fslvmirrloga sapmirrlogavg /oracle/vse/mirrloga fslvmirrlogb sapmirrlogbvg /oracle/vse/mirrlogb fslvsapdata1 sapdata1vg /oracle/vse/sapdata1 fslvsapdata2 sapdata2vg /oracle/vse/sapdata2 fslvsapdata3 sapdata3vg /oracle/vse/sapdata3 fslvsapdata4 sapdata4vg /oracle/vse/sapdata4 Fslvsapmnt sapmntvg /sapmnt/vse Fslvusrsap usrsapvg /usr/sap/vse Fslvtrans transvg /usr/sap/trans fslvusrhome usrhomevg /home/vseadm 60

61 Use multiple virtual SCSI Controllers for the database instance. The use of multiple virtual SCSI Controllers allows several parallel I/O operations inside the guest OS. We also highly recommend you separate the log I/O traffic from the datafile I/O traffic through separate virtual SCSI controllers. As a best practice, you can use one controller for OS and Swap, one controller for database log, and one or more controllers for database data files (depending on the amount and size of database data files). For optimal throughput, use the paravirtual SCSI Controller (PVSCSI). Be sure that you install the VMware Tools first. See VMware KB for details: Configuring disks to use VMware Paravirtual SCSI (PVSCSI) adapters. Network Installation type Other For best network performance, use a vmxnet3 network card. See VMware KB for details: Choosing a network adapter for your virtual machine. Central installation and distributed installation are supported. A High Availability configuration is explained in the HA/Mobility bundle. Install the latest version of VMware Tools in the guest operating system for optimal performance. See SAP Note Linux: VMware vsphere configuration guidelines for detailed guidelines from SAP. 61

62 Validation of functions Overview We tested the following use cases to validate the solution: Use case 1: Virtual data center for SAP Use case 2: Automated provisioning, service catalogs, self-services, and chargeback Use case 3: Integrated cloud management and performance analysis Use case 1: Virtual data center for SAP Test scenario By implementing the foundation bundle, customers can pool different physical and virtual resources into a single abstraction layer that allows resources to be dynamically and elastically assigned and reassigned based on the multi-tenancy model. Success criteria The virtual datacenter for SAP should demonstrate the following benefits: Multi-tenancy Virtual machines from different organizations can occupy the same resource pool. You can regulate resource allocation using a set of policies. Security Using the design in the VMware vcloud networking and security design section, the foundation bundle must be able to: Isolate SAP, DB, and OS data access within an organization Restrict end user access from other organizations Enable Admin access across organizations Validation procedure The validation procedure includes Multi-tenancy and Security testing. Multi-tenancy We validated the following scenarios in the multi-tenancy function: PvDC is configured and ready for consumption Organization vdc is configured with different service levels 62

63 As shown in Table 22, we built Standard and Advanced PvDCs to provide resources for constructing Organization vdcs. Table 22. PvDC resources Standard PvDC Advanced PvDC DRS cluster Standard cluster-1 Advanced cluster-1 Storage profile Bronze Silver Gold Datastore cluster Bronze Tier Silver Tier Gold Tier We built Organization vdcs according to the OvDC design mentioned in the Virtual data center for SAP section. Table 23 depicts the detailed parameters of each OvDC. Table 23. OvDC parameters Org A Org A Org B Org B Basic Committed Basic Dedicated Allocation Model Pay as you go Allocation pool Pay as you go Reservation pool CPU allocation 25 GHz 50 GHz 40 GHz 50 GHz CPU resource guaranteed 0% 75% 0% 100% vcpu speed 1 GHz 2 GHz 1 GHz Any Memory allocation 100 GB 150 GB 100 GB 200 GB Memory resources guaranteed Maximum number of virtual machines 75% 75% 75% 100% Unlimited Unlimited Unlimited Unlimited Storage profile Bronze Silver Gold Bronze Silver Gold Thin provisioning Enabled Enabled Enabled Disabled When thin provisioning is enabled in the OvDC settings, vdisks of virtual machines are created in thin format. Otherwise, vdisks of virtual machines are created in zeroedthick. We disabled thin provisioning in Dedicated OvDC to ensure storage performance. For Basic and Committed OvDCs, we enabled thin provisioning to maximize storage utilization. 63

64 Security We validated the following security scenarios: Client hosts can access their own SAP systems in their organizations through SAPGUI Client hosts cannot access other organizations SAP systems Administrator hosts can access the operating systems of SAP systems through SSH across organizations We built the organizations and external networks according to the network design in the Cloud networking and security for SAP section. Client hosts from different organizations and administrator hosts from centralized IT are external to the cloud environment. They access SAP systems through the external networks CorpNet and CorpNet_Admin. The IP addresses of all SAP systems within the cloud were translated by the vshield edge gateway, which serves as a firewall that controls access rules. Figure 28 shows an overview of the two organizations IP addresses. Figure 28. IP addresses Each SAP system has two IP addresses connected to the organization network; one for client access, and the other for administrator access. IP addresses are translated when the packages leave the cloud environment. Table 24 and Table 25 show the detailed information on NAT. 64

65 Table 24. NAT for Org A SAP system Interface Original IP Translated IP Edge gateway ERP QAS Client Edge 01 ERP QAS Admin Edge 02 ERP PRO Client Edge 01 ERP PRO Admin Edge 02 Table 25. NAT for Org B SAP system Interface Original IP Translated IP Edge gateway ERP DEV CL Edge 03 ERP DEV Admin Edge 04 ERP PRO CL Edge 03 ERP PRO Admin Edge 04 Client hosts and administrator hosts only know about external network IP addresses. Organization networks are transparent to the client hosts. We created firewall rules on each edge gateway to control access. Table 26 to Table 28 show detailed information about our firewall rules. Table 26. Firewall rules for client access on Edge 01 ID Name Source Destination Protocol Action 1 Dispatcher /16:Any /24:3200 TCP Allow 2 Message Server /16:Any /24:3600 TCP Allow 3 Outbound internal:any Any:Any ANY Allow Table 27. Firewall rules for client access on Edge 03 ID Name Source Destination Protocol Action 1 Dispatcher /16:Any /24:3200 TCP Allow 2 Message Server /16:Any /24:3600 TCP Allow 3 Outbound internal:any Any:Any ANY Allow 65

66 Table 28. Firewall rules for admin access on Edge02 and Edge 04 ID Name Source Destination Protocol Action 1 SSH /16:Any /24:22 TCP Allow 2 Dispatcher /16:Any /24:3200 TCP Allow 3 Message Server /16:Any /24:3600 TCP Allow 4 Outbound internal:any Any:Any ANY Allow Verifying network access within an organization Table 29 shows the result of client host in Org A accessing an SAP system in Org A. Table 29. Client host in Org A accesses an SAP system in Org A Client host IP SAP system / IP ERP PRD / Port Result 3200/TCP (SAP dispatcher) Allowed Figure 29 depicts the firewall logs: Figure 29. Firewall logs for client host in Org A accessing SAP system in Org A Access from a client host ( ) in Org A to SAP ERP PRD ( ) in Org A was allowed. IP address of SAP ERP PRD was translated to

67 Verifying secure network in multi-tenancy Table 30 shows the result of client host in Org A accessing an SAP system in Org B. Table 30. Client host in Org A accesses SAP system in Org B Client host IP SAP system/ip Port Result ERP PRD/ /TCP (SAP dispatcher) Denied Figure 30 depicts the firewall logs: Figure 30. Firewall logs for client host in Org A accessing SAP system in Org B Access from a client host ( ) in Org A to SAP ERP PRD ( ) in Org B is denied. The SAP ERP PRD IP address is translated to We performed the same test for the client host in Org B accessing SAP systems in Org B and Org A. Table 31 and Table 32 show the test results. Table 31. Client host in Org B accesses SAP system in Org B Client host IP SAP system/ip Port Result ERP PRD/ /TCP (SAP dispatcher) Allowed 67

68 Table 32. Client host in Org B accesses SAP system in Org A Client host IP SAP system / IP ERP PRD / Port Result 3200/TCP (SAP dispatcher) Denied Administrator host in centralized IT accesses OS in Org A and Org B Table 33 shows the result of the administrator host in centralized IT accessing the OS in Org A and Org B. Table 33. Administrator host in centralized IT accesses the OS in Org A and Org B Client host IP SAP system / IP Org A ERP PRD / Org A ERP QAS / Org B ERP DEV / Org B ERP PRD / Port Result 22/TCP (SSH) Allowed Figure 31 depicts the following firewall logs: The first entry shows that access from the administrator host ( ) to SAP ERP PRD ( ) in Org A is allowed and that the SAP ERP PRD IP address is translated to The second entry shows that access from the administrator host ( ) to SAP ERP QAS ( ) in Org A is allowed and that the SAP ERP QAS IP address is translated to The third entry shows that access from the administrator host ( ) to SAP ERP DEV ( ) in Org B is allowed and that the SAP ERP DEV IP address is translated to The last entry shows that access from the administrator host ( ) to SAP ERP PRD ( ) in Org B is allowed and that the SAP ERP PRD IP address is translated to

69 Figure 31. Firewall logs for the administrator host accessing SAP system in Org A and Org B Summary With vcloud Director, corporate IT can provide a multi-tenancy infrastructure for different organizations running SAP systems. Cloud administrators can setup vdcs with different service levels to meet the organizations business requirements. vcloud networking and security enables fast network provisioning within the cloud environment and secure access for clients outside of the cloud environment. NAT and firewall on the edge gateway provides the flexibility to implement access control. Use case 2: Automated provisioning, service catalogs, self-services, and chargeback Test scenario This use case includes the following scenarios: Simplified provisioning The foundation bundle allows easier service allocation and provisioning. Demonstrate how the services are consumed by: The SAP administrator An SAP developer in Org B Chargeback The foundation bundle can: Accurately measure resources and services consumed by organizations Automatically reflect changing vapp resources in vcenter Chargeback and adjust the cost accordingly Validation procedure Simplified provisioning The SAP administrator for Org B can log on to the organization management link to manage, monitor and control vapps and resources that belong to the organization, as shown in Figure

70 Figure 32. Organization management page in vcd The SAP administrator can also deploy new applications with the vapp template with just a few clicks of a button. In this solution, we prepared a foundation operating system environment preconfigured to meet all the configuration and software requirements needed by SAP. All the agents, such as VMware Tools and backup agents were preinstalled in this environment, so this template was ready to deploy in the new SAP system without any additional configuration. Templates can be imported into vcloud Director either by importing a virtual machine from vcenter, or by importing an OVF file directly. We predefined templates, one for the SAP ABAP stack system and one for the SAP JAVA stack system, as shown in Figure 33. Figure 33. Displaying the templates available in the vcd catalog 70

71 Figure 34 shows the steps used to deploy a new vapp from the catalog. Figure 34. Adding a vapp from the catalog Figure 35 shows the main steps needed to complete the deployment of a vapp configuration. 1. Select template 2. Naming vapp 3. Naming VM 4. Configure Network Figure 35. Configure steps to add a vapp from the catalog 71

72 After the deployment was completed, we powered on the virtual machine to reset the specific static IP to the virtual machines related to the vapp as shown in Figure 36. Figure 36. Managing the new created virtual machine in vcd After deploying the new application, the cloud administrator can find the new entry under the organization of Org B in vcenter Chargeback Manager as shown in Figure 37. Figure 37. Displaying the hierarchy in vcenter Chargeback Manager The newly deployed vapp SAPDM1 was already reflected in the Chargeback hierarchy of Org B. Accurately measure resources and services consumed by organizations We generated a weekly report for OrgA_Basic OvDC. SAP virtual machines under this OvDC were in the service category, Basic. As depicted in Figure 38, OrgA_Basic OvDC contains one vapp named SAP ERP 6.0 EHP QAS. CPU, memory, and storage resources are charged separately. 72

73 Figure 38. Cost report for OrgA_Basic Changing the vapp resource automatically reflects in Chargeback Manager In the virtual machine event, we can find the virtual machine sapvsedb was off at 00:03:23 and was back on at 07:57:12 on April 9 th. Figure 39 shows the event log in vcenter. Figure 39. Event log in vcenter 73

74 The change in the virtual machine s state appears in the cost report as shown in Figure 40. Figure 40. Virtual machine state change appears in the cost report We found two entries for CPU and memory: one entry was made before the power off, and the other entry was made after the power on. The time exactly matched what we observed in the tasks and events change infrastructure. We also observed that from April 27 th to April 28 th, the datastore was changed three times. Figure 41 shows the datastore change reflected in the cost report. Figure 41. Virtual machine datastore change reflected in the Chargeback report 74

75 The first change was to add a 200 GB disk from the silver datastore to extend PSAPTEMP table space to rebuild the index for a large table. The storage profile of this SAP virtual machine was upgraded from silver level to gold level, which is reflected by the rate factor increasing from 1.5 to 2. Everything was done non-disruptively. Summary The SAP administrator or SAP developer can easily provision virtual machines from a catalog in its own organization through vcloud Director. After being authorized by the cloud infrastructure administrator, the SAP developer can log into the vcloud Director portal to provision an SAP standard application on demand through the vcd catalog. The addition or removal of virtual machines is automatically reflected in the Chargeback Manager reports. Billing from the Chargeback Manager is charged according to the resources consumed. Use case 3: Integrated cloud management and performance analysis An SAP system experiences an abnormally high workload. In the first step, we validated VMware's fully automated Distributed Resource Scheduling functionality. In the second scenario, we demonstrated how the SAP administrator performed a high speed root cause analysis, using vcops, EMC Storage Resource Management Suite, and Unisphere to analyze the performance and optimize the storage resources. Success criteria The evaluation is based on the following criteria: Fully automated resource pooling the CPU and storage react automatically by regulating the resources as needed, while allowing enough room for administrators to control the process. Root cause analysis and problem resolution validate the effectiveness of the monitoring tools in determining the root cause and make recommendations based on the data. Scenario 1: On-demand resource pooling For demonstration purposes, we set the VMware DRS Automation level to Manual before simulating a heavy SAP workload. The SAP administrator received an incident report from EMC SRM indicating that the CPU utilization of the virtual machine SAPVSEDB had reached 80 percent, as shown in Figure

76 Figure 42. Virtual machine performance tab By using vcops to investigate further, the SAP administrator observed high CPU usage on the ESXi host where the virtual machine resided, as shown in Figure 43. Figure 43. ESXi CPU usage in vcops We resolved this problem by moving the virtual machine to a host with a lower CPU utilization. We set the VMware DRS to automatically migrate the virtual machine to a different host to balance the workload and automatically avoid contention issues. 76

77 The Cloud administrator changed the Automation level setting from Manual to Fully automated, as shown in Figure 44. Figure 44. DRS setting in vcenter Using EMC SRM, the SAP administrator detected the virtual machine vmotion, as shown in Figure 45. Figure 45. EMC SRM detects virtual machine migration Analysis After the automatic migration, the host CPU usage dropped by 45 percent as shown in Figure 46. Figure 46. CPU usage after automatic migration 77

78 Scenario 2: Root cause analysis and problem resolution In this scenario, we demonstrate how the SAP administrator performed a high-speed root cause analysis, using vcops, EMC Storage Resource Management Suite, and Unisphere to analyze the performance and optimize the storage resources. Validation procedure We used several SAP systems and set up a test environment as follows: 1. We measured the performance of an SAP ECC system under a Silver storage profile by generating an I/O-intensive workload on an SAP system. We used 1,000 dialog SD (Sales and Distribution module) users and 20 resourceintensive background jobs to simulate a real environment. 2. We benchmarked the ST03N response time to fall within 800ms to 1,000ms before the test, such that approximately 40 percent came from the database time. This was to ensure that the database time drove the overall system performance. 3. We doubled the number of users from 1,000 to 2,000 and used exactly the same workload before and after the performance issue was resolved. 4. The SAP administrator used the monitoring tools to analyze the performance and observed how the data center responded to this increase in workload. The administrator then identified the bottleneck and recommended a solution. 5. Following the recommendation, we observed the difference in performance subjected to the same 2,000-user workload, using the same tools. Figure 47 shows the system performance after the host migration. Note that the response time was still high, even if the processing and CPU times were already normal. The database time contributed more than 46 percent to this value. Furthermore, the roll wait time was high. Figure 47. Dialog response time 1. Identify affected area by VMware vcops The SAP administrator logs into vcops to identify the affected area in the cloud environment. As shown in Figure 48, in the Org_B view, the SAPVSEDB virtual machine presents a low healthy index score (35) and the datastore (G86_RG_silver_DS_823) used by this virtual machine had a heavy workload of up to 78

79 100%. We performed deeper analysis on the above affected virtual machines in EMC SRM. Figure 48. Org_B s virtual machine health scoreboard and datastore workload 2. Identify the topology path from virtual machine to storage In the cloud environment, virtual machines float in the ESXi Server to balance the workload. To efficiently locate the ESXi host and the storage of the affected virtual machine, the SAP administrator quickly retrieves an end-to-end view of the relationship between the virtual machine (SAPVSEDB) and storage in EMC SRM, as shown in Figure 49. Figure 49. End-to-end view from the virtual machine to storage 3. End-to-end detail performance analysis on affected virtual machine Next, the administrator drilled down to see more details on the virtual machine, ESxi host, fabric switches, and arrays. By simply clicking on each of these objects and selecting the Performance tab, several KPI charts displayed the trends that enabled the administrator to detect irregularities in performance. 79

80 Table 34. Virtual machine to array performance analysis in EMC Storage Resource Management Suite Measure Value Virtual machine(sapvsedb) CPU utilization 40% IOPS abruptly increased from 2,000 to 4,000 ESXi server CPU utilization 20% SAN switch CPU utilization 12% port utilization 8% Array LUN IOPS Abruptly increased from 2,000 to 4,000 LUN response time Increased from 2ms to 12ms As shown in Table 34, the SAP administrator analyzed virtual-machine-to-array endto-end performance using EMC Storage Resource Management Suite and detected the following performance activities: The virtual machine s I/O rate abruptly increased from 2,000 to 4,000 IOPS. However, the virtual machine s CPU utilization was low (only 40 percent). Therefore, the virtual machine s CPU did not cause the bottleneck and the administrator moved forward to the host level. The ESXi host and SAN switch showed similar behavior as that of the virtual machine chart. The CPU and switch port utilization values were both very low. Neither the ESXi host nor the switches caused the bottleneck and the administrator moved forward to the array level. This virtual machine used LUN As shown in Figure 40, the LUN s IOPS increased from 2,000 to 4,000, and the response time also increased from less than 2ms to 12ms. At this point, the SAP administrator isolated the problem to a storage issue. The SAP administrator escalated the issue to the cloud administrator for further investigation. Figure 50. EMC storage array Performance tab 80

81 The cloud administrator launched EMC Unisphere from Prosphere. In Prosphere, the response time of the Basic_Silver_Pool where LUN0823 resided increased with the IOPS. This indicates that the pool was unable to provide enough IOPS capacity to the virtual machine, which caused degraded performance across the hosts and up through the SAP system. Figure 51. Storage group response time 4. Recommendation to decrease IO contention The cloud administrator recommended that the organization upgrade its storage profile from Silver to Gold. In this case, the organization accepted the recommendation to decrease I/O contention. As shown in Figure 52, the cloud administrator upgraded the storage profile in vcloud Director Console. Figure 52. Change storage profile in vcloud Director After the upgrade, the datastore migrated from VMAX storage pool named Basic_Silver_Pool to Basic_Gold_Pool non-disruptively. As shown in Figure 53 and Figure 54, the storage response time decreased to 8.2ms, with an IOPS increase from 4,000 to 6,182. The database time improved by 28 percent, and roll wait time was reduced by 70 percent a 45 percent improvement overall, as shown in Figure 55 and Figure 56. Figure 53. Virtual machine s IOPS increased after the upgrade 81

82 Figure 54. Virtual machine s response time decreased after the upgrade Figure 55. Storage response time after the upgrade Figure 56. Improved dialog response time 82