BUSINESS CONTINUITY BEST PRACTICES FOR SAP HANA TAILORED DATACENTER INTEGRATION

Transcription

1 BUSINESS CONTINUITY BEST PRACTICES FOR SAP HANA TAILORED DATACENTER INTEGRATION On EMC VNX Storage Systems with EMC MirrorView and EMC SnapView EMC Solutions Abstract This white paper introduces EMC VNX series software and hardware capabilities and provides a comprehensive set of best practices and procedures for high availability and business continuity when using SAP HANA with EMC VNX in a Tailored Datacenter Integration (TDI) deployment. This solution includes EMC MirrorView TM and EMC SnapView TM. February 2015

2 Copyright 2015 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All trademarks used herein are the property of their respective owners. Part Number H

3 Table of contents Executive summary... 5 Business case... 5 Solution overview... 5 Key results... 5 Introduction... 6 Document purpose... 6 Scope... 6 Audience... 6 Technology overview... 7 Introduction... 7 Disaster recovery support in SAP HANA... 7 Backups... 7 HANA System Replication... 7 Storage replication... 8 SAP HANA Storage Replication with VNX series... 9 Enterprise protection and compliance with MirrorView... 9 SAP HANA database snapshots with SnapView... 9 Overview of the VNX replication technologies... 9 MirrorView modes of operation MirrorView/Synchronous mode MirrorView/S write intent log MirrorView/Asynchronous mode Replication bandwidth requirements Consistency groups When to use MirrorView/S and MirrorView/A Consistent data copies with SnapView Operating system and HANA shared file system Business continuity best practices use cases for SAP HANA storage replication Use Cases for SAP HANA General requirements Software requirements Test environment for use cases Maintaining the SAP HANA global.ini file at the remote site Use Case 1: Establishing remote mirroring for disaster protection (synchronous or asynchronous) Use Case 2: Planned failover to the secondary site (maintenance) Use Case 3: Unplanned failover to the secondary site (disaster) Part

4 Part Use Case 4: Failback to the primary site Use Case 5: Asynchronous remote mirroring across multiple VNX arrays Use Case 6: Creating restartable and writable database snapshots for repurposing at the secondary site Conclusion Summary Findings References EMC documentation SAP documentation

5 Executive summary Business case Customers deploy SAP HANA databases and integrated applications in many of the most mission-critical functions, including manufacturing, financial accounting, inventory management, and sales and marketing. These and other functions are the lifeblood of a business, and interruptions or loss of data can be catastrophic. To assure the availability of these systems for businesses that depend on them, a comprehensive approach to business continuity planning and execution is required. Recovering these databases in the event of a disaster, or periodically testing the effectiveness of this recovery for business and audit requirements, is a unique requirement of federated systems such as SAP. Solution overview Key results The EMC VNX series is widely used in SAP landscapes, in mission-critical applications, and with traditional databases. The EMC VNX series now provides the same reliability platform for the software and hardware layer for the SAP HANA database as for traditional databases. In this solution, EMC MirrorView TM with consistency technology combined with EMC SnapView TM snapshots, enables SAP customers to replicate consistently and reliably restore or test the many business functions in the total SAP landscape. MirrorView and SnapView enable complete high availability and business continuity for mission-critical environments for synchronous and asynchronous data protection. The use cases in this white paper provide best practices for implementing SAP HANA storage replication on EMC VNX series arrays, using proven tools that are already in wide use by customers. The use cases illustrate recovery using MirrorView and SnapView and show how to implement recoverable and restartable remote database replicas. 5

6 Introduction Document purpose Scope This white paper introduces SAP HANA replication for the EMC VNX series and provides a comprehensive set of best practices and procedures for business continuity when using SAP HANA with VNX in a Tailored Datacenter Integration (TDI) deployment. The scope of this white paper includes the following: Introduces the key technologies for this solution Describes best practices and design considerations for this solution Outlines how the key components are configured and deployed Presents use cases to show how the solution can be successfully used Audience This white paper is for SAP HANA database and system administrators, storage administrators, and system architects who implement, maintain and protect robust SAP HANA in-memory databases and storage systems. Readers must have some familiarity with SAP HANA in-memory databases, EMC VNX series storage systems, and VNX software, especially MirrorView and SnapView. 6

7 Technology overview Introduction When the SAP HANA in-memory database is used in Tailored Datacenter Integration (TDI) deployments with EMC VNX storage arrays, most customers must protect their mission-critical applications and the SAP HANA in-memory database against disasters, hardware or software failures, or human errors by maintaining a copy of the data either locally or remotely. SAP HANA provides disaster recovery support with backups or HANA System Replication to seamlessly integrate SAP HANA into existing business continuity solutions based on VNX replication technologies and enables federated disaster recovery strategies. Note: The EMC white paper Storage Configuration Best Practices for SAP HANA Tailored Datacenter Integrations on EMC VNX Series Unified Storage Systems provides details about using EMC VNX storage arrays with TDI. Disaster recovery support in SAP HANA SAP HANA offers three levels of disaster recovery support backups, HANA System Replication, and storage replication. Each solution addresses different Recovery Point Objectives (RPO) within the required Recovery Time Objective (RTO) where: RPO denotes the point of consistency to which HANA needs to recover RTO denotes the time allowed for a recovery of the HANA system to a specified point of consistency Backups Backups are used to protect the primary HANA persistence against a storage failure. Therefore, the HANA backup target should never be on the same storage array as the primary HANA persistence. Backup systems can typically replicate the backup storage to a remote system to protect against a site failure. With HANA backups, the RPO depends on the frequency in which the customer is doing HANA backups and can be from minutes to several hours. The RTO with backups can be several hours because a HANA backup needs to be restored to the persistence and from there read into memory before the database is available. EMC offers HANA backup solutions based on EMC NetWorker and EMC Data Domain product lines. HANA System Replication HANA System Replication is an application-based disaster recovery solution where a secondary standby HANA system is configured as an exact copy of the active primary system. Each secondary system must consist of the same number of active HANA nodes. As with storage replication, HANA System replication requires a reliable connection between the primary and secondary sites. The replication technology supports multiple replication modes, including synchronous, synchronous in-memory, and asynchronous modes. With HANA System replication, only the database content is required to be replicated to the secondary site. 7

8 Storage replication HANA storage replication in TDI deployments provides a convenient method to protect HANA against a site failure. The HANA primary persistence is replicated to a secondary site using technologies on the storage arrays. Depending on RTO and RPO requirements and the distance between the primary and secondary site, you can implement synchronous storage replication (RPO = 0) or asynchronous storage replication (RPO 0. If a disaster occurs, the RTO is typically the time it takes to start the HANA database at the secondary site. HANA Storage Replication provides several benefits to customers compared to the HANA System Replication. The storage replication includes the database persistence to ensure that the data is available at a remote site. Storage replication can also include data outside of HANA in the consistency technology of the storage array, creating a consistent point-in-time image of the overall business applications at the secondary site. In addition to the replication of the HANA database persistence, you can include components such as the operating system boot volumes, the HANA shared file system that includes the HANA binaries and configuration files, or non-hana applications, to enable a federated disaster recovery strategy. This white paper covers several use cases for SAP HANA local and remote Storage Replication on VNX Series storage arrays. It covers the HANA database persistence only (data and log volumes). Note: The SAP HANA Administration Guide provides more details about the different disaster recovery solutions and their advantages and disadvantages. 8

9 SAP HANA Storage Replication with VNX series Enterprise protection and compliance with MirrorView SAP HANA database snapshots with SnapView Overview of the VNX replication technologies Data consistency refers to the accuracy and integrity of the data and between copies of the data. VNX offers several solutions for local and remote replication of SAP HANA in-memory databases. With the MirrorView software, mirrors of the SAP HANA database persistence can be created together with their external data and application files, all sharing a consistency group. Replicating the SAP HANA persistence this way creates the point of consistency across business units and applications before any disaster takes place. Failover to the disaster recovery site uses a series of application restart operations that reduce overall complexity and downtime. Every mission-critical system needs multiple copies of data, such as for development, test, quality assurance, backup, and more. Because VNX uses SnapView software in combination with SAP HANA storage snapshots, multiple copies of the SAP HANA database persistence (data and log) can be created or restored in seconds. The examples in this document use SnapView at the remote site to create a consistent point-in-time image of the database persistence that can be used by the HANA servers at the remote site for test or development while replication from the local to the remote site is ongoing. The EMC VNX Series storage arrays support various remote replication technologies developed by EMC, which protect your data from a disaster. These technologies are: MirrorView VNX Replicator EMC RecoverPoint EMC SRDF EMC VPLEX EMC Replication Manager This paper covers SAP HANA storage replication using MirrorView/Synchronous (MirrorView/S) and MirrorView/Asynchronous (MirrorView/A) for the HANA persistence on block devices. Other VNX replication technologies are not covered in this document. In addition to remote replication, the EMC VNX series supports the following two local replication technologies: SnapView VNX Snapshots Both technologies can create snapshots of block LUNs. However, they use different underlying technologies and support the different LUN types in the VNX array. VNX Snapshots can only be used with Pool LUNs, SnapView can only be used on traditional LUNs and MetaLUNs. Since MetaLUNs are recommended in HANA environments, this document describes how to use SnapView with HANA. 9

10 MirrorView modes of operation MirrorView/S and MirrorView/A are the basic operation modes of MirrorView. These two are valid for SAP HANA database protection and maintain dependent write-order consistency. MirrorView/Synchronous mode MirrorView/S is used to create a no data loss solution of committed transactions. It provides the ability to replicate multiple databases and applications data remotely while guaranteeing that the data on both the source and target devices is exactly the same. With MirrorView/S replication, as shown in Figure 1, each write on the local host is written concurrently to the remote site. After the remote VNX unit acknowledges that it received the I/O successfully, the write is acknowledged to the local host. If a disaster occurs at the local site, data at the secondary site is exactly the same as the data at the local site (RPO = 0). Figure 1. MirrorView/Synchronous replication The synchronous replication impacts the latency of write I/O operations. Because SAP HANA requires low latencies for write I/Os to the persistence, the distance between the two sites is limited. When to use MirrorView/S and when MirrorView/A provides more details. MirrorView/S write intent log MirrorView/S uses an optional write intent log that tracks in-flight writes to both the local and the remote array on two dedicated write intent log LUNs. One write is assigned to each storage processor. In recovery situations, the write intent log can be used to determine which extents must be synchronized from the local to the remote storage system. When a single HANA database system is installed across multiple arrays, the write-intent log must be disabled. You can type the following example commands to create the write intent log on a VNX array. A log is required on the primary and the secondary array. The write intent log LUNs must be created on a RAID group and can reside on one of the RAID groups used for the HANA persistence: naviseccli -h <VNX ip> bind r5 0 -rg 31 -cap sq mb -sp a naviseccli -h <VNX ip> bind r5 1 -rg 31 -cap sq mb -sp b naviseccli -h <VNX ip> mirror -sync -allocatelog -spa 0 -spb 1 o Note: In this example the write intent log LUNs are bound in RAID group 31 with LUN IDs 0 and 1. The write intent log is required on the primary and the secondary array. 10

11 Type the following command to list the write intent log configuration: naviseccli -h <VNX IP> mirror -sync -listlog MirrorView/Asynchronous mode MirrorView/A, as shown in Figure 2, uses a periodic update model which provides a consistent point-in-time image on the target devices with an RPO greater than zero. Data changes at the primary site are tracked and applied at the remote site at a user-defined frequency, which can be: 1. Manually 2. Minutes after the last transfer began 3. Minutes after the last transfer ended Option 3 was used in this document in most HANA tests with MirrorView/A. However, in some use cases (such as Use Case 6: Creating restartable and writable database snapshots for repurposing at the secondary site), you must switch to a manually triggered transfer. By tracking data areas that change, it is possible to send less data if writes occur to the same areas of the devices between updates. Updates are smoothed out over the entire update period, regardless of write bursts to the source LUNs, allowing for low bandwidth and low-cost links between sites. MirrorView/A tracks and stores data changes in the global reserved LUN pool, which is required on the local and the remote array. Figure 2. MirrorView/Asynchronous replication The primary benefit of MirrorView/A is that it allows longer distance replication, because SAP HANA response time is not dependent on the latency of the link. Reserved LUN pool for MirrorView/A The reserved LUN pool consists of private LUNs with a total size that is at least 20 percent of the total capacity of the source LUNs. The number of LUNs must at least match the number of source LUNs. This extra capacity is required on the local and the remote array and must be considered when you use MirrorView/A for disaster recovery. You can use the following example commands to create the reserved LUN pool on a VNX array: naviseccli -h <VNX ip> bind r5 <lun id> -rg <raid group> -cap <capacity> -sq gb -sp a 11

12 Repeat the command and create the number of LUNs that you need: naviseccli -h <VNX ip> reserved lunpool addlun <lun_numbers> A reserved LUN pool is also required when you use SnapView, which is described in Use Case 6: Creating restartable and writable database snapshots for repurposing at the secondary site. Note: VNX Command Line Interface Reference for Block 5.33, which is available from EMC Online Support, provides more information about planning and sizing the reserved LUN pool. Replication bandwidth requirements When using synchronous storage replication with MirrorView/S, you can calculate the bandwidth requirement between the primary and the secondary site for peak workload with the following formula: Number of HANA worker nodes x 200 MB/s For example, for a HANA 3+1 scale-out installation, 600 MB/s bandwidth is required between the primary and secondary site. With asynchronous replication, the bandwidth requirement depends on the change rate of the database and the interval in which the tracked changes are applied to the remote site. This interval can be specified when you configure MirrorView/A and defines the RPO of the replication. Consistency groups MirrorView (both /S and /A) includes the storage-system-based consistency group feature. A storage-system-based consistency group is a collection of mirrors that function together as a unit within the storage system. All operations, such as synchronize, promote, or fracture, occur on all members of the consistency group. Consistency groups also manage write-order dependency of all members and ensure that there is always a restartable copy of the database that is available on the secondary system. When to use MirrorView/S and MirrorView/A There are three basic types of disaster recovery solutions: Campus solution Data is typically transmitted with fiber-optic cabling using VNX and SAN equipment with a distance smaller than 66 km using channel extenders or long-distance fiber optics. Metro solution For distances less than 100 km. Geo solution For any distance larger than 100 km. In campus and metro clusters, MirrorView/S is usually used. However, for SAP HANA, it is important that the latency for smaller 4 K I/Os on the log device be in the low single digit milliseconds range. If this cannot be achieved with existing infrastructure and synchronous replication, you can switch to asynchronous replication. Extended Distance Wide Area Network (WAN) provides MirrorView connectivity over long distances using telecommunication networks such as IP, SONET, or ATM. MirrorView/A is best for all WAN implementations. 12

13 Consistent data copies with SnapView Operating system and HANA shared file system You can use SnapView to create consistent copies of the data, which can be used for setting up test or development systems on the secondary site without interrupting the replication. Any work on the snapshot may be rolled back to its original state. The use cases in this white paper describe the replication of the SAP HANA database persistence (data and log devices). SAP HANA also uses a shared NFS /hana/shared file system that resides on the VNX. The replication of this file system, which contains the HANA binaries, configuration files and traces, can be accomplished using VNX Replicator. Note: NFS file system replication is not part of this document but covered in Using VNX Replicator 8.1, which is available from EMC Online Support. If the HANA servers start from a SAN and the operating system is installed on VNX LUNs, these LUNs can be included in the MirrorView replication scenarios by adding them to the consistency groups. However, additional steps may be required when you restart the remote site after a failover, and hostnames and network addresses must be adjusted. 13

14 Business continuity best practices use cases for SAP HANA storage replication The following sections describe use cases with best practices for implementing SAP HANA storage replication and recovery using MirrorView and SnapView technology. IMPORTANT: The commands used in these use cases are examples which must be customized based on the actual customer environment. Create scripts with the customized commands and carefully test and validate them before implementing the use cases in productive customer environments. Use Cases for SAP HANA The following six use cases for SAP HANA and VNX are examples and show how to implement recoverable and restartable local and remote database replicas. Use cases 1-4 and 6 can be implemented using either MirrorView/S or MirrorView/A. Use case 5 is a use case for MirrorView/A only. Use Case 1: Establishing remote mirroring for disaster protection (synchronous or asynchronous) Use Case 2: Planned failover to the secondary site (maintenance) Use Case 3: Unplanned failover to the secondary site (disaster) Use Case 4: Failback to the primary site Use Case 5: Asynchronous remote mirroring across multiple VNX arrays Use Case 6: Creating restartable and writable database snapshots for repurposing at the secondary site General requirements Software requirements Before a SAP HANA Disaster Recovery (DR) solution in a SAP HANA Tailored Datacenter Integration (TDI) scenario can be established, the following requirements must be met: A MirrorView/S or MirrorView/A license are applied on the primary and secondary VNX Series arrays. A MirrorView connection between the source and the destination exists. MirrorView links are logical connections between two VNX arrays, which are physically connected by cables, routers, extenders, switches, and other network devices. Both arrays must be configured with identical HANA volumes as described in the EMC white paper Storage Configuration Best Practices for SAP HANA TDI on EMC VNX Series Storage Systems. A SnapView license must be applied on the local and/or remote array if this technology is used as described in use case 6. Additional disk capacity in the primary and secondary arrays is required for the reserved LUN pool when using MirrorView/A or SnapView. The use cases were tested using the following software versions: SAP HANA 1.0 SPS08 rev82 SUSE Linux SLES11 for SAP Applications SP3 kernel VNX OE Block on the primary and secondary array 14

15 Test environment for use cases The use cases described in this document, as shown in Figure 3, were tested in a SAP HANA scale-out deployment with three worker nodes and one standby node on a primary VNX5400 connected with Fibre Channel (FC) to a secondary VNX7600, as shown in Figure 3. Figure 3. Test environment for use cases Each VNX array has a management server connected with VNX Navisphere command line interface (NaviSecCLI) installed. The use cases described in the following sections use NaviSecCLI commands: The -h <primary VNX> target IP address command must run on the local management server. The -h <secondary VNX> target IP address command must run on the remote management server. SAP HANA servers must be on the two sites. The servers at Site-2 use the secondary devices for their HANA persistence when a failover occurs. Remote snapshots are used when the persistence is used at the secondary site for test or development while the replication is ongoing. Maintaining the SAP HANA global.ini file at the remote site The storage section in the HANA global.ini file contains the references from the HANA storage partitions to the storage LUNs. EMC uses the UUID of the LUN to identify the correct storage devices. The UUIDs can be identified using the following two methods: Method 1 From the SAP HANA server If you need to identify the UUID of a 1.5 TB data LUN, type the following command on the HANA node: multipath -ll grep -B1 1.5T a00c8e8afa7495fe411 dm-10 DGC,RAID 5 size=1.5t features='1 queue_if_no_path' hwhandler='1 emc' wp=rw The string a00c8e8afa7495fe411 is the UUID of the corresponding storage LUN. 15

16 Note: Linux adds a preceding 3 to the storage UUID. Method 2 From a management server using NaviSecCLI The UUIDs from the LUNs can be displayed with this command: naviseccli -h <primary VNX> getlun -uid name LOGICAL UNIT NUMBER 310 UID: 60:06:01:60:32:E0:35:00:CA:C1:DF:69:72:6E:E4:11 Name LUN_Data1 [..] Note: NaviSecCLI commands display the UUID without the preceding 3. The HANA global.ini file will then look like this example: [storage] ha_provider = hdb_ha.fcclient partition_*_* prtype = 5 partition_1_data wwid = e035000b287d720277e411 partition_1_log wwid = a303500b09e9a780277e411 partition_2_data wwid = e eaa0277e411 partition_2_log wwid = e03500da327b790277e411 partition_3_data wwid = e035000c287d720277e411 partition_3_log wwid = a303500b19e9a780277e411 IMPORTANT: Prepare a global.ini file that contains the partition entries of the secondary LUNs. Name the file, for example global.ini_dr, and and keep it in the same directory where the regular global.ini resides. Swap the global.ini files when HANA must be started at the secondary site. The same method can be used when you start HANA at the secondary site using SnapView LUNs for test or development purposes. 16

17 Use Case 1: Establishing remote mirroring for disaster protection (synchronous or asynchronous) This use case shows how to create remote mirrors of the SAP HANA database persistence (data and log) to establish disaster protection using MirrorView (/S or /A). The primary devices in the VNX at site 1 are replicated to the secondary devices in the VNX at site 2. While the replication is active, access to the secondary devices from the HANA Nodes at site 2 is not possible. The basic steps to set up a replication between the primary and the secondary site are identical between MirrorView/S and MirrorView/A. The CLI commands use different options (mirror-sync and mirror-async) Figure 4 provides an overview of this use case. Figure 4. Establish remote mirroring To establish remote mirroring for disaster protection in synchronous or asynchronous mode: 1. Enable a MirrorView connection between the VNX arrays. Before a connection can be established, the MirrorView ports on the two arrays must be connected to each other using a network topology and bandwidth that supports the planned replication method (synchronous or asynchronous). Note: The EMC white paper MirrorView Knowledgebook provides further information on MirrorView ports. Type the following command to establish a MirrorView connection between the primary and the secondary arrays: naviseccli -h <primary VNX> mirror -enablepath <secondary VNX> -connectiontype fibre 17

18 2. Configure replication settings as follows: a. MirrorView/S Configure the write intent log as described in MirrorView/S Write Intent Log. b. MirrorView/A Configure the reserved LUN pool as described in Reserved LUN pool for MirrorView/A. 3. Enable the primary image for each LUN on the primary system: naviseccli -h <primary VNX> mirror -async -create -name Lun_Data1_Mirror -lun <alu> o Repeat the previous command for all LUNs. Use a different name for each mirror. <alu> must be replaced with the array LUN ID. Note: If synchronous replication is configured, use mirror -sync in all remaining commands of this use case. Ensure that the same replication method is used on all persistent devices 4. Add secondary images to the mirrors as follows: naviseccli -h <primary VNX> mirror -async -addimage -name Lun_Data1_Mirror -arrayhost <secondary VNX> -lun <alu> Repeat the previous command for all secondary LUNs. <alu> must be replaced with the array LUN ID on the secondary array. 5. Create a consistency group with name HANA_Volumes as follows: The consistency group manages write-order consistency across all LUNs and allows MirrorView commands to run against the group instead of against individual devices. After the consistency groups are created, add all HANA persistent LUNs (data and log) to the groups. If restarting from a SAN, the operating system LUNs can be added to the consistency group. naviseccli -h <primary VNX> mirror -async -creategroup - name HANA_Volumes enddelay 0 o naviseccli -h <primary VNX> mirror -async -addtogroup name HANA_Volumes -mirrorname Lun_Data1_Mirror Repeat the previous command for all mirrors. For only MirrorView/A, the -enddelay 0 parameter causes the next update cycle to occur quickly. If you need to manually control the update process to the target LUNs, use the -manualupdate parameter instead. The initial synchronization starts automatically and it may take some time until all devices are synchronized. Type the following command to verify the state of the synchronization: naviseccli -h <primary VNX> mirror -async -listgroups - name HANA_Volumes -state Group Name: HANA_Volumes State: Consistent When the state is Consistent or Synchronized, the LUNs at the secondary site are ready to take over the database if a disaster occurs. 18

19 Use Case 2: Planned failover to the secondary site (maintenance) This use case shows how to failover the HANA persistence to site 2 using MirrorView (/S or /A) for maintenance at site 1. For example, failover might be done for a hardware replacement. This use case describes how to make the database persistence (secondary devices) available to the standby HANA installation at site 2 with an automatic personality swap of the devices. (Site 2 becomes primary site and replicates to former primary site that is now secondary). Figure 5 shows the status after a planned failover and a personality swap of the devices when site 1 is in maintenance mode. Figure 5. Enable remote protection while production running on site 2 To failover to the secondary maintenance site: 1. Remove primary LUNs from the local HANA servers. In a scheduled failover scenario, stop HANA at the primary site first and then shut down the HANA servers. Remove the primary LUNs from the storage groups for all HANA servers (worker and standby) because the servers will lose any read or write access. Type the following command to list the LUNs of a storage group: naviseccli -h <primary VNX> storagegroup -list -gname Server01 Then remove all LUNs using the host LUN (HLU) number from the output of the previous command: naviseccli -h <primary VNX> storagegroup -removehlu - gname Server01 -hlu <hlu> o Repeat the previous command for all primary LUNs and storage groups. Prepare a script for this purpose. 19

20 2. Failover consistency group HANA_Volumes to write-enable the destination devices. Type the following command to promote the original secondary LUNs at site 2 to the role of primary images and automatically change the role of the LUNs at site 1 to secondary images: naviseccli -h <secondary VNX> mirror -async -promotegroup -name HANA_Volumes o Note: Use mirror -sync if synchronous replication is configured. 3. Make LUNs available to the HANA servers at the secondary site. This step assumes that storage groups for the HANA servers already exist at the secondary site and must include the HBA initiators of the standby HANA servers. The storage groups must not include any LUNs. They will be added as part of a failover scenario. Type the following command: naviseccli -h <secondary VNX> storagegroup -addhlu -gname server05 -alu <alu> -hlu <hlu> o Repeat the previous command for all storage groups and LUNs and replace <alu> with the array LUN ID and <hlu> with the host LUN ID. Prepare a script for this purpose. The secondary devices in the consistency group HANA_Volumes are now in read/write enabled state and can be discovered by the operating system to start SAP HANA at the remote site. Ensure that the global.ini file of the HANA system at site 2 contains the correct partition entries with the pointers to the new UUIDs of the devices at site 2 before starting HANA. For example, use the previously prepared global.ini_dr. 20

21 Use Case 3: Unplanned failover to the secondary site (disaster) This use case shows how to failover the HANA persistence to site 2 using MirrorView (/S or /A). If a disaster occurs, this use case makes the database persistence available to the HANA installation at site 2, which then becomes the new production site. site 1 is unavailable while production is running at site 2. This use case is split in two parts. Part 1 describes how to failover and run the HANA production at site 2. After site-1 is repaired and is back online, part 2 sets the replication from site 2 to site 1 to enable data protection while running at site 2. Figure 6 shows the status after a disaster when site 2 becomes the new production site. Figure 6. New production site 2 after a failover Part 1 Failover and run the HANA production at site 2: 1. Type the following command to force a failover of the HANA_Volumes consistency group to write-enable the secondary devices. This changes the role to primary: naviseccli -h <secondary VNX> mirror -async -promotegroup -name HANA_Volumes -o When trying to promote the LUNs at the secondary site while the connection between the sites is broken like in this use case, the following error is displayed: Unable to contact MirrorView/A on remote array during group promote (0x715280fd) If the error occurs, create a force promote as follows: naviseccli -h <secondary VNX> mirror -async -promotegroup -name HANA_Volumes -type force o Local Status: Local Promote succeeded. Remote Status: Unable to send a message to the secondary or peer SP. Check to see if the MirrorView connections between secondary and primary arrays have been established. (0x ) 21

22 After the successful force promote, the LUNs are available for read/write operation in the secondary site, which is then running the SAP HANA production. Note: If synchronous replication is configured, use mirror -sync in all remaining commands of this use case. Ensure that the same replication method is used on all persistent devices. 2. Type the following command to make new primary LUNs available to the hosts at site 2: naviseccli -h <secondary VNX> storagegroup -addhlu -gname server05 -alu <alu> -hlu <hlu> o Repeat the previous command for all LUNs and storage groups and replace <alu> with the array LUN ID and <hlu> with the host LUN id. Prepare a script for this purpose. The secondary devices in the HANA_Volumes consistency group are now in read/write enabled state and can be discovered by the operating system to start SAP HANA at the secondary site. Ensure that the global.ini file of the HANA system at site 2 contains the correct partition entries with the pointers to the new UUIDs of the devices at site 2 before starting HANA. For example, use the previously prepared global.ini_dr. Part 2 After the primary site has been recovered and repaired and the primary VNX is available and online, enable the LUNs at site 1 as secondary images: 1. Remove LUNs at site 1 from storage groups as follows: naviseccli -h <primary VNX> storagegroup -removehlu - gname Server01 -hlu <hlu> o Repeat the previous command for all LUNs and storage groups. Use <hlu> with the host LUN ID. Prepare a script for this purpose. 2. Destroy the stale consistency group on site 1 as follows: naviseccli -h <primary VNX> mirror -async -destroygroup - name HANA_Volumes -force o Destroy stale mirrors on site 1 as follows: naviseccli -h <primary VNX> mirror -async -destroy -name Lun_Data1_Mirror -force o Repeat the previous command for all mirrors. 3. Type the following command to destroy the consistency group at site 2 before the new mirrors can be defined: naviseccli -h <secondary VNX> mirror -async -destroygroup -name HANA_Volumes -force o 22

23 4. Type the following command to create new mirrors with primary LUNs from site 2 and secondary LUNs from site 1: naviseccli -h <secondary VNX> mirror -async -addimage - name Lun_Data1_Mirror -arrayhost <primary VNX> -lun <alu> Repeat the previous command for all mirrors. Replace <alu> with the array LUN ID. Create a script for this purpose. 5. Recreate the HANA _Volumes consistency group at site 2: naviseccli -h <secondary VNX> mirror -async -creategroup -name HANA_Volumes enddelay 0 o Add LUNs and wait until initial synchronization is finished: naviseccli -h <secondary VNX> mirror -async -addtogroup - name HANA_Volumes -mirrorname Lun_Data1_Mirror Repeat the previous command for all mirrors. Create a script for this purpose. 23

24 Use Case 4: Failback to the primary site After a disaster and failing over the HANA database to site 2 using the steps in use case 3, repair site 1 and make it accessible and available for MirrorView replication. Before bringing the SAP HANA database back online at the primary site, follow the steps in Part 2 of use case 3. These steps re-establish the replication from site 2, which ran the production while site 1 was down, to site 1. After all devices have a status of Consistent or Synchronized, you can promote the devices at site 1 to the role of primary images. Note: This process is a scheduled event and requires a short production downtime when failing back from site 2 to site 1. To failback to the primary site: 1. Verify the status of replication to ensure that the devices are synchronized or consistent: naviseccli -h <secondary VNX> mirror -async -listgroups - name HANA_Volumes state Group Name: HANA_Volumes State: Consistent Note: Use mirror -sync if MirrorView/S is set up in all use case commands. After the devices have a status of Consistent or Synchronized, the LUNs at site 1 are ready to take over the database. 2. Stop SAP HANA at site Failback the HANA_Volumes consistency group to write-enable the devices at site 1. Promote the LUNs at site 1 to the role of primary images and automatically change the role of the LUNs at site 2 to secondary images: naviseccli -h <primary VNX> mirror -async -promotegroup - name HANA_Volumes o 4. Make primary LUNs available to the HANA servers at site 1. In Part 2 of use case 3, LUNs at site 1 were removed from their storage groups because they were write disabled during replication. Type the following command to add them back to the storage groups so that they will be available for the HANA servers at site 1: naviseccli -h <primary VNX> storagegroup -addhlu -gname Server01 -alu <alu> -hlu <hlu> o Repeat the previous command for all LUNs and storage groups. Replace <alu> with the array LUN ID and <hlu> with the host LUN ID. Prepare a script for this purpose. Primary devices in the HANA_Volumes consistency group are now read/write enabled and can be discovered by the operating system to start SAP HANA at site 1. Ensure that the global.ini file of the HANA system contains the correct partition entries with the pointers to the UUIDs of the primary devices before starting HANA. 24

25 Use Case 5: Asynchronous remote mirroring across multiple VNX arrays This use case shows how to create restartable remote mirrors of the SAP HANA database persistence (data and log). It provides disaster protection using MirrorView/A when the storage persistence of a single HANA database is distributed across multiple VNX arrays. The performance requirements of SAP HANA enable a limited number of HANA worker nodes to be connected to a single VNX array. Note: Refer to the white paper Storage Configuration Best Practices for SAP HANA TDI on EMC VNX Series Storage Systems, on EMC Online Support, for details about scalability. If a single HANA database is distributed across multiple VNX arrays, managing write-order dependency using consistency groups is no longer possible because consistency groups cannot have mirrors that span arrays. In this use case, the SAP HANA database creates a consistent point-in-time image of the persistence on disk using SAP HANA storage snapshots. This feature, in combination with MirrorView/A, ensures that a restartable consistent point-intime image of the database is available site 2. The SAP HANA Storage Snapshots are triggered using a HANA command line and must be synchronized with the mirror updates of the database persistence at the secondary site. Combine the corresponding commands in a shell script and schedule it to run in intervals. The interval time (30 minutes, for example) denotes the RPO of the replication. Figure 7 shows a SAP HANA installation that is deployed across two VNX arrays. Figure 7. Asynchronous replication with multiple primary VNX arrays To create asynchronous remote mirroring across multiple VNX arrays: 1. Enable the MirrorView connection between the VNX arrays (as in use case 1): Note: In this use case, the MirrorView connection is required between VNX1 and VNX3 and between VNX2 and VNX4, as shown in Figure 7. naviseccli -h <primary VNX1> mirror -enablepath <secondary VNX3> -connectiontype fibre 25

26 naviseccli -h <primary VNX2> mirror -enablepath <secondary VNX4> -connectiontype fibre 2. Configure reserved LUN pools in each array for MirrorView/A, as described in Reserved LUN pool for MirrorView/A. 3. Enable the primary image for each LUN on the arrays at site 1: naviseccli -h <primary VNX1> mirror -async -create -name Lun_Data1_Mirror -lun <alu> o Repeat the previous command for all LUNs in VNX1. Replace <alu> with the array LUN ID. naviseccli -h <primary VNX2> mirror -async -create -name Lun_Data3_Mirror -lun <alu> o Repeat the previous command for all LUNs in VNX2. Replace <alu> with the array LUN ID. 4. Add secondary images to the mirrors: naviseccli -h <primary VNX1> mirror -async -addimage - name Lun_Data1_Mirror -arrayhost <secondary VNX3> -lun <alu> Repeat the previous command for all mirrors in VNX1. Replace <alu> with the array LUN ID. naviseccli -h <primary VNX2> mirror -async -addimage - name Lun_Data3_Mirror -arrayhost <secondary VNX4> -lun <alu> Repeat the previous command for all mirrors in VNX2. Replace <alu> with the array LUN ID. 5. Create two consistency groups, HANA_Volumes1 and HANA_Volumes2: naviseccli -h <primary VNX1> mirror -async -creategroup - name HANA_Volumes1 manualupdate o naviseccli -h <primary VNX1> mirror -async -addtogroup name HANA_Volumes1 -mirrorname Lun_Data1_Mirror Repeat the previous command for all mirrors in VNX1 naviseccli -h <primary VNX2> mirror -async -creategroup - name HANA_Volumes2 manualupdate o naviseccli -h <primary VNX2> mirror -async -addtogroup name HANA_Volumes2 -mirrorname Lun_Data3_Mirror Repeat the previous command for all mirrors in VNX2 The initial synchronization is started automatically and it may take some time until all devices are synchronized. Type the following command to verify the status of the synchronization: naviseccli -h <primary VNX1> mirror -async -listgroups - name HANA_Volumes1 state naviseccli -h <primary VNX2> mirror -async -listgroups - name HANA_Volumes2 state 26

27 After the state is Consistent or Synchronized, the LUNs at site 2 are ready to take over the database if a disaster occurs. 6. Create a restartable remote mirror of the database persistence and create a shell script that includes steps 6a to 6d. Schedule the shell script to run in certain intervals, for example every 30 minutes, which denotes the RPO of the replication. The SAP HANA client software is required on the management server to run HANA hdbsql commands. A restartable mirror contains a consistent state of the database across all LUNs on all primary arrays. Use the HANA function Storage Snapshots is used to achieve this state. These Storage snapshots, triggered with hdbsql or the SAP HANA studio, will create a consistent restartable image within the HANA persistence across multiple primary arrays. This consistent state can safely be transported to the secondary arrays. As soon the LUNs at the secondary site are promoted to the role of primary, HANA is able to restart on this consistent state. a. Verify that all HANA nodes at the primary site are up and running by typing the following command on one of the HANA nodes as user <sid>adm: python $DIR_INSTANCE/exe/python_support/landscapeHostConfigur ation.py. Use echo $? to display the return code when you run the command. If the return code is 3 or higher, then HANA is in a valid state and you can continue with the next steps. If the return code is 2 or less, then HANA has a problem which needs to be addressed. In this case, it is not possible to create a consistent restartable image at site 2. b. Prepare the SAP HANA database for a storage snapshot. This HANA command needs to run as <sid>adm user: hdbsql -i <instance number> -u <system user> -p <system password> BACKUP DATA CREATE SNAPSHOT Note: Refer to the SAP HANA Administration Guide for further information on the HANA storage snapshots. c. When finished, manually update each consistency group: naviseccli -h <primary VNX1> mirror -async -syncgroup -name HANA_Volumes1 o naviseccli -h <primary VNX2> mirror -async -syncgroup -name HANA_Volumes2 o Type the following command to verify the status of mirrors: naviseccli -h <primary VNX1> mirror -async -listgroups -name HANA_Volumes1 state naviseccli -h <primary VNX2> mirror -async -listgroups -name HANA_Volumes2 -state 27

28 d. Close the snapshot as soon as the transfer is started or the primary system will use this snapshot in case of a restart. First determine the backup snapshot ID by typing the following HANA command: hdbsql -i <instance number> -j -u <system user> -p <system password> "SELECT BACKUP_ID, SYS_START_TIME FROM "PUBLIC"."M_BACKUP_CATALOG" WHERE ENTRY_TYPE_NAME = 'data snapshot' AND STATE_NAME = 'prepared'" BACKUP_ID,SYS_START_TIME ," :04: " The backup ID is required for closing the snapshot. Ensure that the time shown is for the snapshot that was taken. Then close the snapshot: hdbsql -i <instance number> -u <system user> -p <system password> "BACKUP DATA CLOSE SNAPSHOT BACKUP_ID SUCCESSFUL 'Remote Storage Clone'" If the data transfer failed for some reason, the SAP HANA storage snapshot must be abandoned by using the following command: hdbsql -i <instance number> -u <system user> -p <system password> "BACKUP DATA CLOSE SNAPSHOT BACKUP_ID UNSUCCESSFUL 'Storage Clone Failed'" 7. If a disaster occurs, or during a planned maintenance on the primary site, promote site 2 to the primary role, as described in use cases 2 and 3. Restarting the HANA database at site 2 will use the consistent point-intime image represented by the storage snapshot in the HANA persistence. 28

29 Use Case 6: Creating restartable and writable database snapshots for repurposing at the secondary site This use case, as shown in Figure 8, shows how to create a copy of a HANA system using VNX SnapView snapshots without interrupting the replication. Such a copy can be used for creating test or development systems or just to maintain multiple point-in-time images. VNX SnapView sessions are consistent point-in-time snapshots of the whole HANA system. When a set of snapshot LUNs is created, these LUNs can be associated with a SnapView session. Multiple sessions can be created to keep multiple states of the system. Note: The steps in this use case cannot be applied when HANA is installed across multiple primary VNX arrays. With multiple primary VNX arrays, a SAP HANA storage snapshot is required to create a consistent point-in-time database image in combination with SnapView snapshots. Figure 8. Secondary site using snaps for test or development 1. To create restartable and writable database snapshots for repurposing at the secondary site: Create snapshot LUNs at site 2: Before consistent snapshots can be used, snapshot LUNs must be created. Snapshot LUNs will be allocated in the reserved LUN pool. naviseccli -h <secondary VNX> snapview -createsnapshot <alu> -snapshotname Lun_Data1_Snap Repeat the previous command for all LUNs. Replace <alu> with the array LUN ID. The UUIDs of the snapshot LUNs are required in the HANA global.ini file if you want to start HANA on the secondary site using the snapshots. Type the following command to determine the UUIDs: naviseccli -h <secondary VNX> snapview listsnapshots SnapView logical unit name: Lun_Log2_Snap SnapView logical unit ID: 60:06:01:60:32:E0:35:00:A8:08:FB:F4:46:80:E4:11 Target Logical Unit: 215 State: Inactive 29

30 The output will display all snapshot LUNs. Note: A preceding 3 is added to the UUID by the operating system. 2. Create a SnapView session: naviseccli -h <secondary VNX> snapview -startsession HANA_Snap -lun <LUN list> -consistent Note: You must activate a snapshot on this session before you can access it. The Consistent parameters ensure that HANA is restartable. Replace the <LUN list> option with the list of all LUNs in the consistency group. For example, use Activate the Snapview session and make snapshot LUNs available to the hosts at site 2 as follows: naviseccli -h <secondary VNX> snapview -activatesnapshot HANA_Snap -snapshotname Lun_Data1_Snap Repeat the previous command for all snapshot LUNs. Create a script for this purpose. After the LUNs have been activated, they need to be added to the storage groups at site 2 as Snapshot LUNs using the following command: naviseccli -h <secondary VNX> storagegroup -addsnapshot - gname server05 -hlu <hlu> -snapshotname Lun_Data1_Snap Repeat the previous command for all snapshot LUNs and storage groups. Replace <hlu> with the host LUN ID. HANA can now be started at site 2 using the snapshot LUNs. Ensure that the correct UUIDs of the snapshot LUNs are used in the HANA global.ini file. 4. Delete the SnapView session (optional): The SnapView session can be deleted if the LUNs are no longer required. Type the following command to remove the SnapView session from the storage groups: naviseccli -h <secondary VNX> snapview -stopsession HANA_Snap -o 30