P9000 Virtualization Adapter Administrator Guide

Transcription

1 HP StorageWorks P9000 Virtualization Adapter Administrator Guide This guide contains detailed instructions for installing and removing the P9000 Virtualization Adapter. The intended audience has independent knowledge of related OS software and of HP disk arrays and software. Part Number: Fourth edition: October 2010

2 Legal and notice information Copyright Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR and , Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation.

3 Contents 1 Overview of the P9000 Virtualization Adapter... 7 Discovery of disk arrays and LUNs... 7 Failover testing... 8 Failover recovery... 9 Local configuration... 9 Remote configuration using Telnet RAID Manager and array configurations rmsra command and environmental variables rmsra command Environment variables $HORCMROOT $RMSRATOV $RMSRATMU Setting environment variables Environment variables on the SRM host Environment variables on a UNIX host Modifying environment variables using the Windows GUI Installing and removing the P9000 Virtualization Adapter Installation procedure Verifying the installation Upgrading Removing the product Configuring SRM and the disk array Creating Pair-Volumes for data replication groups Configuring datastores Pairing SRM sites Configuring SRM to communicate with the P9000 Virtualization Adapter Create protection groups Create recovery plans Failover and failback with SRM Failover Failback Failback scenario with SRM Troubleshooting General troubleshooting Error messages on SRM log files Errors in XML passed from SRM P9000 Virtualization Adapter Administrator Guide 3

4 RAID Manager command errors in rmsra.exe Configuration and status errors Troubleshooting information for technical support Error code for multiple errors Support and other resources Related documentation HP technical support Subscription service HP websites Documentation feedback Typographical conventions Glossary Index

5 Figures Configuration using local connection Configuration using remote connection RAID Manager configuration for the P9000 Virtualization Adapter horcmx.conf on protection site horcmx.conf on recovery site horcmx+1.conf on recovery site horcmx.conf on primary site horcmx.conf on secondary site Multi Path I/O connection Environment variables P9000 Virtualization Adapter Administrator Guide 5

6 Tables 1 Document conventions

7 1 Overview of the P9000 Virtualization Adapter HP StorageWorks P9000 Virtualization Adapter (P9000 Virtualization Adapter) is a Perl interface between VMware Site Recovery Manager (SRM) and HP StorageWorks P9000 and XP disk arrays. The interface enables SRM to provide automatic access to remote data copies when virtual machines become unavailable locally. The P9000 Virtualization Adapter performs the following functions for SRM: Discovers disk arrays Discovers replicated LUNs Fails over storage for testing (test a recovery plan) Fails over storage for recovery (execute a recovery plan) Unless otherwise specified, the terms disk arrays, P9000, and XP in this guide refer to the following supported disk arrays: HP StorageWorks P9500 Disk Array HP StorageWorks XP24000 Disk Array HP StorageWorks XP20000 Disk Array HP StorageWorks XP12000 Disk Array HP StorageWorks XP10000 Disk Array Also, the terms Business Copy, Continuous Access, Continuous Access Journal, Snapshot, and RAID Manager, refer to the to the following supported products: HP StorageWorks P9000 Business Copy Software HP StorageWorks XP Business Copy Software HP StorageWorks P9000 Continuous Access Synchronous Software HP StorageWorks XP Continuous Access Software HP StorageWorks P9000 Continuous Access Journal Software HP StorageWorks XP Continuous Access Journal Software HP StorageWorks P9000 Snapshot Software HP StorageWorks XP Snapshot Software HP StorageWorks P9000 RAID Manager Software HP StorageWorks XP RAID Manager Software Discovery of disk arrays and LUNs During SRM configuration, the P9000 Virtualization Adapter is called to discover available arrays and replicated LUNs. The following information is discovered: Storage Array IDs serial number P9000 Virtualization Adapter Administrator Guide 7

8 Model: for example, /FF Product: P9000_RAID Storage Port ID: Port WWN Replicated LUNs and IDs: LDEV number and LUN WWN Remote Storage Array IDs: serial number Remote LUN IDs: LDEV number NOTE: Remote S-VOL is available to the groups through RAID Manager horcm.conf, and are replicated between protected and recovery P9000 or XP disk arrays using Continuous Access and Continuous Access Journal software. Failover testing The P9000 Virtualization Adapter provides information and performs operations for nondisruptive testing of SRM failover recovery plans. SRM uses the information gathered during the discovery process to create point-in-time copies of remote LUNs. After testing verifies that the point-in-time copies of the LUNs can be recovered after failure of the local array (via a recovery plan), SRM uses the P9000 Virtualization Adapter to unpresent and delete the point-in-time copies. Business Copy or Snapshot is used to create point-in-time copies during recovery plan testing. NOTE: For Failover, the ESX LVM settings must be set to default for point-in-time copies (default is disallow) and resignature (default is allow). The failover testing sequence is: SRM makes a request to the P9000 Virtualization Adapter to do test failover to replicated LUNs on a remote (recovery) array. The P9000 Virtualization Adapter connects to a RAID Manager instance and the IDs of all replicated point-in-time copy S-VOLs on the recovery array. The P9000 Virtualization Adapter looks for and checks that the point-in-time copy S-VOLs are members of data replication groups, and then makes them read and write S-VOLs. The P9000 Virtualization Adapter provides information and performs failover operations for nondisruptive testing of SRM failover recovery plans, and returns the following information: Replicated key and point-in-time copy S-VOLs and IDs: LDEV number and LUN WWN CT group IDs for replicated LUNs SRM maps LUNs with datastores using the replicated keys and LUN WWN. SRM verifies that the point-in-time copy S-VOLs can be recovered after failure of the local array (via a recovery plan). SRM uses the P9000 Virtualization Adapter to unpresent and delete the point-in-time copies. NOTE: For failover testing, the point-in-time copy S-VOLs must be created as PAIR state, and must be registered to horcm.conf as a group. 8 Overview of the P9000 Virtualization Adapter

9 Failover recovery During recovery, SRM executes a recovery plan. When SRM reaches the Prepare Storage step in the recovery plan, it provides the LUN information needed for failover to the P9000 Virtualization Adapter. The failover sequence is: SRM makes a request to the P9000 Virtualization Adapter to do failover to replicated LUNs on a remote (recovery) array. The P9000 Virtualization Adapter connects to an RAID Manager instance and requests replicated Continuous Access S-VOLs on the recovery array. The P9000 Virtualization Adapter looks for and checks that Continuous Access S-VOLs are members of data replication groups, and then executes horctakeover to make them read and write S- VOLs. The P9000 Virtualization Adapter provides information and performs failover operations defined in SRM failover recovery plans, and returns the following information: Replicated key and Continuous Access S-VOLs and IDs: LDEV number and LUN WWN CT group IDs for replicated LUNs NOTE: CT Group IDs for replicated LUNs are supported in Business Copy. SRM maps LUNs with datastores using the replicated keys and LUN WWN, and then moves all resources to the recovery site. NOTE: During initial setup, replicated LUNs need to be presented to both the protected and recovery ESX hosts. Local configuration Figure 1 shows the local configuration for the P9000 Virtualization Adapter on a Windows machine. The Virtual Center server (VIC), SRM, and P9000 Virtualization Adapter are configured on Windows P9000 Virtualization Adapter Administrator Guide 9

10 . Figure 1 Configuration using local connection Remote configuration using Telnet Figure 2 shows the Telnet configuration for the P9000 Virtualization Adapter on a Windows machine. The Virtual Center server (VIC), SRM, and P9000 Virtualization Adapter are configured on Windows RAID Manager is on a remote host (for example, Linux, Solaris, or HP-UX). The P9000 Virtualization Adapter is configured using the command.pl Perl script file (see command.pl example on page 12) as an interface adapter and the rmsra command. In a remote connection, command.pl calls the rmsra command on a remote RAID Manager. IMPORTANT: If the RAID Manager instance is on Windows, the P9000 Virtualization Adapter must use the local configuration because the P9000 Virtualization Adapter is called directly, and does not use the LAN/telnet server locally. 10 Overview of the P9000 Virtualization Adapter

11 . Figure 2 Configuration using remote connection P9000 Virtualization Adapter Administrator Guide 11

12 command.pl example #!/usr/bin/perl use Net::Telnet; = <STDIN>; my $input; my $output; my $out; my $host; my $user; my $pass; my $local = 1; my $cmd; : : : my $telnet = Net::Telnet->new(Host=>$host,Cmd_remove_mode=>1,Timeout=>180); $telnet->login($user,$pass); if ($ENV{'HORCMROOT'} ne "") { $cmd = "echo \"". $input. "\" ". $ENV{'HORCMROOT'}. "\\HORCM\\etc\\rmsra -auto timeout=". $ENV{RMSRATOV}. " testmu=". $ENV{'RMSRATMU'}. " -XMLSTDOUT"; } else { $cmd = "rm -f ". $txtname; $telnet->cmd($cmd); foreach my $line (@input) { $cmd = "echo \'". $line. "\' >> ". $txtname; $telnet->cmd($cmd); } $cmd = "cat ". $txtname. " /HORCM/usr/bin/rmsra -auto timeout=". $ENV{'RMSRATOV'}. " testmu=". $ENV{'RMSRATMU'}. " -XMLSTDOUT >". $xmlname. " 2>". $logname; } $telnet->cmd($cmd); $cmd = "cat ". $logname. " >> ". $xmlname; $telnet->cmd($cmd); $cmd = "cat ". $xmlname; my $exit) = $telnet->cmd($cmd); foreach my $line (@stdout) { if ($line =~ /^\s*< wwn=/) { $output = $output. $line; } elsif ($line =~ /^\[RMSRA\]/) { print $line; } } $out = "> ". $out; open(out,$out); print OUT $output; close(out); exit($exit/256); 12 Overview of the P9000 Virtualization Adapter

13 . RAID Manager and array configurations HORCM instances To use the P9000 Virtualization Adapter, HORCM instances X are needed on both sites (primary and secondary), but remote communications are not needed between the instances. However, if Continuous Access paired volumes are created using RAID Manager, remote communications are needed. For failover testing, HORCM instance X+1 is needed. If HORCM X is not running, the P9000 Virtualization Adapter uses the auto option (-auto) to start it. The P9000 Virtualization Adapter then tries to start HORCM X+1. Figure 3 shows the RAID Manager configuration for the P9000 Virtualization Adapter. Figure 3 RAID Manager configuration for the P9000 Virtualization Adapter Fibre Channel Address conversion table Because VMware ESX Server uses WWN and LUN number notation for device discovery, the Fibre conversion table number for HORCM must be changed using $HORCMFCTBL variable: C:\HORCM\etc> set HORCMFCTBL=3 C:\HORCM\etc> horcmstart This variable is set to 3 automatically by the P9000 Virtualization Adapter, but if the auto option (-auto) is used, then the LUN number in horcm.conf must be specified as the same LUN number on ESX Server, P9000 Remote Web Console, or XP Remote Web Console. P9000 Virtualization Adapter Administrator Guide 13

14 . RAID Manager configuration files The examples shown in Figure 4 on page 14 and Figure 5 on page 15 assume the following instance number and the target port: X is the RAID Manager instance number 0 is the LUN number for failover testing CL4-G-1 is the target port for ESX Server Figure 4 horcmx.conf on protection site 14 Overview of the P9000 Virtualization Adapter

15 . Figure 5 horcmx.conf on recovery site NOTE: 1 - MU# must NOT be specified for registering as Continuous Access group. 2 - MU# must be specified for registering Business Copy group. HP recommends that you specify either MU#0,1.2 for Business Copy or Snapshot. Business Copy can be specified either MU#0,1.2. Snapshot can be specified either MU#0, If MU# (except MU#0) is specified, it is needed to set the P9000 Virtualization Adapter variable. (see Environment variables on page 22). The example in Figure 6 on page 16 is required for failover testing. P9000 Virtualization Adapter Administrator Guide 15

16 .. Figure 6 horcmx+1.conf on recovery site The examples shown in Figure 7 on page 16 and Figure 8 on page 17 assume the following instance number and the target port: X is the RAID Manager instance number Business Copy or Snapshot is not used for failover testing CL4-G-1 is the target port for ESX Server Figure 7 horcmx.conf on primary site 16 Overview of the P9000 Virtualization Adapter

17 . Figure 8 horcmx.conf on secondary site NOTE: For failover testing on Continuous Access and Continuous Access Journal S-VOL: If the Business Copy licence is not installed, SRM requires that SplitReplication is set to true. If the Business Copy licence is installed, SRM requires that SplitReplication is set to true and the MU number except 0 is specified as $RMSRATMU variable. Establish communication between the HORCM of the primary site and the HORCM of the recovery site. For non-stop testing of an application, Continuous Access Synchronous CTG or Continuous Access Journal volumes must be used. Snapshot for failover testing The point-in-time copy for failover testing must be created as volume using point-in-time-split (this capability is supported only for Business Copy and Snapshot). C:\HORCM\etc> paircreate g <group> -m grp -fq quick -pid <snapshotpool> Required LUN ID VMware ESX Server supports NAA version 6 information in Page 83 SCSI inquiry as UUID. ESX Server, Datastore, and SRM recognize a LUN using LUN WWN returned by the disk array. Therefore, the P9000 Virtualization Adapter returns the LUN WWN, matching the datastore regardless of Multi-Path I/O (MPIO). The device identification must include NAA version 6 information in Page 83 SCSI. P9000 Virtualization Adapter Administrator Guide 17

18 . Protection groups using Continuous Access Synchronous or Continuous Access Synchronous CTG If the disk pairs for a protection group are Continuous Access Synchronous, use a fence level of DATA. If the fence level of NEVER is used, the internal horctakeover command fails (horctakeover returns EX_VOLCUR). A fence level of NEVER cannot guarantee data consistency among all disks in a device group when a single disk fails (because I/O is not fenced). Virtual machines must be able to boot when a disaster occurs (failover), or when a failover test is executed. When a disaster happens, it is acceptable to use fence level NEVER. Therefore, the P9000 Virtualization Adapter completes failover successfully, but horctakeover returns an EX_VOLCUR error. Fence level NEVER can be used under the following conditions: When using an application such as APP(SQL/Exchange/Oracle/..) The SRA does not swap from S-VOL to P-VOL when the Continuous Access link is active. Instead, the failover succeeds and SRM is notified that all target volumes have changed to the S-VOL SSWS state. Protection groups and CT groups using XP Continuous Access Asynchronous and Continuous Access Journal Software If the P-VOLs for protection groups are CT group volumes using XP Continuous Access Asynchronous or Continuous Access Journal Software, you must split all volumes (LUNs) associated with the protection groups to ensure the P-VOLs of the protection groups are identical to the CT group. MPIO (Multi Path I/O) Connection In a Multi-Path I/O (MPIO) connection, SRM needs to discover both target port WWNs recognizing an ESX Server in discovery phase. The P9000 Virtualization Adapter converts from a target port number in horcm.conf to a target port WWN. Therefore, the P9000 Virtualization Adapter needs to include the alternate path port and LUN number as shown Figure 9. This example assumes that a target volume is connected to CL3-G and CL4-G for MPIO. Figure 9 Multi Path I/O connection 18 Overview of the P9000 Virtualization Adapter

19 Starting HORCM as a service in Windows RAID Manager provides the svcexe.exe command and a sample script file (HORCM0_run.txt), so RAID Manager (HORCM) can be started automatically from the services. See the HP StorageWorks P9000 RAID Manager User Guide. Starting HORCM as a service in Windows 2000/2003 RAID Manager commands and the HORCMgr process normally have the same privileges. However, if the HORCM has Local System privileges, any RAID Manager commands are unable to attach to the HORCMgr process. In this case, the RAID Manager HORCM_EVERYCLI environment variable must be used to enable communications. See the HP StorageWorks P9000 RAID Manager User Guide. P9000 Virtualization Adapter Administrator Guide 19

20 20 Overview of the P9000 Virtualization Adapter

21 2 rmsra command and environmental variables rmsra command Configures the P9000 Virtualization Adapter parameters Description Use the rmsra command to modify the P9000 Virtualization Adapter parameters. Syntax rmsra { h XMLSTDOUT XMLSTDERR [sralog=path_name] auto loglvl= [quiet] [error] [warning] [info] [verbose] [trivia] timeout=value testmu=value} Arguments h Displays command usage. XMLSTDOUT This option is used to change the output destination of XML from the output destination specified by SRM to STDOUT. XMLSTDERR [sralog=path_name] This option is used to change the output destination of XML from the output destination (<OutputFile>) specified by SRM to STDERR. The sralog is used to change the output destination of the error messages from STDERR. auto This option is used to start HORCM instance automatically. If the instance is not running, the P9000 Virtualization Adapter starts HORCM X and HORCM X+1 (but ignores the results of HORCM X+1). In addition to starting, this option executes the horcmshutdown command for the HORCM instance. loglvl= [quiet] [error] [warning] [info] [verbose] [trivia] This option is used to modify the logging level defined by SRM. The value can be: quiet error warning info P9000 Virtualization Adapter Administrator Guide 21

22 verbose trivia timeout=value This option is used to specify the timeout value for failover using for Continuous Access Journal. If the timeout value is specified using the $RMSRATOV variable, then this variable is ignored. The default is 60 seconds. testmu=value This option is used to specify MU number of the Business Copy or Snapshot volume for failover tests. If the MU number value is specified using the $RMSRATMU variable, this variable is ignored. The default MU is 0. Environment variables $HORCMROOT This environment variable identifies the path for HORCM directory. If this variable is set, the path is set to the HORCM root: setx HORCMROOT D: /m %HORCMROOT%\HORCM\etc\raidqry -l If this variable is not set, the path is the same as the P9000 Virtualization Adapter installation directory (see Installation directory on page 27). NOTE: You must restart the Windows server to complete any changes. $RMSRATOV This environment variable specifies the timeout value for failover using Continuous Access Journal and XP Continuous Access Asynchronous. The timeout value is set using the P9000 Virtualization Adapter command [timeout=value]. The default is 60 seconds. The following examples set the timeout to 30 seconds. Windows example: setx RMSRATOV 30 /m UNIX example (bash): export RMSRATOV=30 NOTE: You must restart the Windows server to complete any changes. 22 rmsra command and environmental variables

23 $RMSRATMU This environment variable specifies the MU number of a Business Copy or Snapshot volume for failover testing. If the MU number value is specified through the P9000 Virtualization Adapter command option [testmu=value], this variable is ignored. The default is MU number 0. The following examples specify 1 as the MU number. Windows example: setx RMSRATMU 1 /m UNIX example (bash): export RMSRATMU=1 NOTE: You must restart the Windows server to complete any changes. Setting environment variables The P9000 Virtualization Adapter must be configured to look for MU#1 for the Business Copy volume. If it cannot find MU#1 on virtual machine 1 during failover testing, the Continuous Access pair is suspended and used for failover testing. If it locates MU#1 at HORCM instance <X>+1, the Business Copy volume is suspended and used for the test failover. NOTE: The Business Copy S-VOL must be presented on the same Fibre Channel port as the Business Copy P-VOL or the UUID on the datastore changes. The ESX Server cannot attach it to the shadow virtual machine for test failover unless the UUID matches. Environment variables on the SRM host Follow these steps to set the environment variables on the SRM host: 1. Issue the following command to set the SplitReplication parameter to equal true: setx SplitReplication true /m 2. Issue the following command to set the RMSRATMU parameter to 1: setx RMSRATMU 1 /m 3. Reboot the SRM host. 4. Verify that the variables are set correctly using the Set command. P9000 Virtualization Adapter Administrator Guide 23

24 5. If RAID Manager is installed to another drive, such as the E drive, use the HORCMROOT variable: setx HORCMROOT E: /m The default timeout value for failover using Continuous Access Journal or XP Continuous Access Asynchronous is 60 seconds. To can change the timeout value, use the RMSRATOV variable: setx RMSRATOV 120 /m If RAID Manager is installed on the SRM host, no more changes are required. Follow the steps in Environment variables on a UNIX host on page 24 if RAID Manager is installed on a UNIX host. Environment variables on a UNIX host To execute RMSRA commands, SRM uses telnet to connect as the root user to the UNIX host. Use the root user profile to set these variables. For Linux, use /root/.bash_profile. For HP-UX, use /.profile. Use the appropriate root user profile for your default shell. Insert the following lines in this file: SplitReplication=true export SplitReplication RMSRATMU=1 export RMSRATMU Log out and back in, and use the env command to verify that these variables are set correctly. The configuration is now complete. When testfailover is executed on virtual machine 1, the Continuous Access pairs are suspended and used for testing. When testfailover is done on virtual machine 2, the Business Copy pairs at MU#1 are suspended and used for testing. Modifying environment variables using the Windows GUI In the Windows, environment variables are added, deleted, or modified using the Advanced tab in the Windows Control Panel. Click Environment Variables, and add the HORCMROOT, RMSRATOV, and RMSRATMU to the System Variables list. 24 rmsra command and environmental variables

25 . Figure 10 Environment variables P9000 Virtualization Adapter Administrator Guide 25

26 26 rmsra command and environmental variables

27 3 Installing and removing the P9000 Virtualization Adapter This chapter explains how to install, configure, and remove the P9000 Virtualization Adapter. IMPORTANT: A successful installation depends on using compatible versions of the OS and all installed software. See P9000 Virtualization Adapter Release Notes for version requirements. Installation directory The P9000 Virtualization Adapter is installed in the following directory (default): \Program Files\VMware\VMware Site Recovery Manager\scripts\SAN\RMXP Installation prerequisites To install the P9000 Virtualization Adapter, you need administrator privileges, and VMware Site Recovery Manager must already be installed on the server. The P9000 Virtualization Adapter requires that RAID Manager is installed on the management server or hosts. For RAID Manager installation instructions, see the HP StorageWorks P9000 RAID Manager User Guide. Installation procedure The P9000 Virtualization Adapter is installed using an executable (rmxpsra.exe). The P9000 Virtualization Adapter must be installed on both the local site SRM server (protected site) and the remote site SRM server (recovery site). If you are upgrading to a newer version of the P9000 Virtualization Adapter, you must remove the existing the P9000 Virtualization Adapter version before upgrading. See Uninstalling HWP on page 29. To remove the existing the P9000 Virtualization Adapter version before upgrading: 1. Download rmxpsra.exe from the HP web page ( and save it in a temporary folder. 2. Browse to the temporary folder and run rmxpsra.exe. P9000 Virtualization Adapter Administrator Guide 27

28 3. Restart the SRM service using the vmware-dr command: C:\>net STOP vmware-dr The VMware Site Recovery Manager Service service is stopping. The VMware Site Recovery Manager Service service was stopped successfully. C:\>net START vmware-dr The VMware Site Recovery Manager Service service is starting. The VMware Site Recovery Manager Service service was started successfully. 4. Set any environment variables. See Environment variables on page 22. If you change environment variables, restart the SRM service. See step 3. Verifying the installation After a new installation or update, verify the version information by issuing the help command: For Windows: %HORCMROOT%\HORCM\etc>rmsra h For UNIX: /HORCM/usr/bin/rmsra h The version is displayed as xx.xx.xx in the example below: Ver&Rev: xx.xx.xx Usage: rmsra [-h] [-XMLSTDOUT] [-XMLSTDERR] [-auto] [sralog=path_name] [loglvl=value] [timeout=value] [testmu=value] Help/Usage Change XML_output to STDOUT Change XML_output to STDERR Start HORCM automatically. Change STDERR_output to path_name. Change logging level to ( quiet/error/warning/info/ verbose/trivia ) on STDERR. Specify timeout_value( in sec ) for UR/Async. Specify MU# using for failover testing. Upgrading Upgrade the XP or P9000 Virtualization Adapter using an executable (rmxpsra.exe). You must upgrade the product on both the local site SRM server (protected site) and the remote site SRM server (recovery site). 28 Installing and removing the P9000 Virtualization Adapter

29 To upgrade the product: 1. Confirm that SRM is enabled and that the EventLog, vpxd and vmware-dr startup types are set to automatic. 2. Download rmxpsra.exe from the HP web page ( and save it in a temporary folder. 3. Browse to the temporary folder and run rmxpsra.exe. 4. Restart the SRM service using the vmware-dr command: C:\>net STOP vmware-dr The VMware Site Recovery Manager Service service is stopping. The VMware Site Recovery Manager Service service was stopped successfully. C:\>net START vmware-dr The VMware Site Recovery Manager Service service is starting. The VMware Site Recovery Manager Service service was started successfully. 5. Set any environment variables. See Environment variables on page 22. If you change any environment variables, restart the SRM service. See step 4. Removing the product To remove the product: 1. Remove the product using the Windows Add or Remove Programs window. 2. Restart VMware Site Recovery Manager Service by using the vmware-dr command: C:\>net STOP vmware-dr The VMware Site Recovery Manager Service service is stopping. The VMware Site Recovery Manager Service service was stopped successfully. C:\>net START vmware-dr The VMware Site Recovery Manager Service service is starting. The VMware Site Recovery Manager Service service was started successfully. P9000 Virtualization Adapter Administrator Guide 29

30 30 Installing and removing the P9000 Virtualization Adapter

31 4 Configuring SRM and the disk array This chapter summarizes the steps to set up SRM and the disk array for use with the P9000 Virtualization Adapter. The following procedures assume that the LUNs for ESX Server are already assigned on the protected and recovery sites, and the assigned LUNs for ESX Server are known as Ports, Port WWN, LUN number. For detailed information on these steps, see the documents listed in Related documentation on page 49. To configure the P9000 Virtualization Adapter with SRM: 1. Create pair-volumes for data replication groups. 2. Configure datastores. 3. Pair SRM sites (Protected and Recovery). 4. Configure SRM to communicate with the P9000 Virtualization Adapter. 5. Create Protection Group(s). 6. Create Recovery Plan(s). Creating Pair-Volumes for data replication groups Once LUNs have been created on the local disk array, create data replication groups containing the LUNs. Data replication groups are created on the local disk array and provide for failover between two disk arrays. You can create data replication groups using RAID Manager. For more information on these tasks, see the HP StorageWorks P9000 RAID Manager User Guide. When you put a pair volume into a data replication group, you must replicate (paircreate) it on the remote disk array. At the remote data center, you must then present the S-VOLs to the ESX servers that are hosts to the remote disk array. After you present the LUNs to both local and remote ESX servers, log into the local ESX servers and perform a rescan. There is no need to rescan for LUNs on the remote ESX servers, since SRM does this during failover. Configuring datastores Perform the following tasks on protected and recovery ESX servers after LUN presentation: Create datastores on the ESX server. Install virtual machines onto the datastores. Create resource pools and folders as needed on recovery ESX servers. Perform a rescan to make sure LUNs are visible on the protected site. For more information on these tasks, see the VMware Site Recovery Manager Administration Guide. Pairing SRM sites You should now pair the protected and recovery sites in SRM. For more information, see the VMware Site Recovery Manager Administration Guide. P9000 Virtualization Adapter Administrator Guide 31

32 Configuring SRM to communicate with the P9000 Virtualization Adapter To configure SRM to communicate with the P9000 Virtualization Adapter: 1. Configure Array Manager. When the protected and recovery SRM servers have been paired, click Configure to configure Array Manager to use the P9000 Virtualization Adapter. 2. Add the P9000 Virtualization Adapter to SRM on the protected site. From the Configure Array Managers window, click Add to add connection information for the P9000 Virtualization Adapter. If the SRM and the P9000 Virtualization Adapter are using a remote connection: Select the P9000 Virtualization Adapter from the Manager Type that is connected to the P9000 Virtualization Adapter. Enter the Alias to connect to RAID Manager instance via the P9000 Virtualization. 32 Configuring SRM and the disk array

33 Connecting to RAID Manager on remote UNIX Host: This connects to HORCMINST=X on the Remote UNIX Host without %HORCMROOT%. This connects to $HORCMINST setting on a remote login environment on a UNIX Host without %HORCMROOT%. Connect to RAID Manager on remote windows Host: HORCMINST=X@Host-name This connects to HORCMINST=X on the Remote Windows Host by specifying %HORCM- ROOT%. $HORCMINST@Host-name This connects to %HORCMINST% setting on a remote login environment on a Windows Host by specifying %HORCMROOT%. After you enter the username and password, click Connect and then click OK to discover P- VOL LUNs on protected array. For a local connection, select the P9000 Virtualization Adapter from the Manager type that is connected to the P9000 Virtualization Adapter. Enter the Alias to connect to RAID Manager instance via the P9000 Virtualization Adapter. P9000 Virtualization Adapter Administrator Guide 33

34 Connecting to RAID Manager on the Windows host: HORCMINST=X This connects to HORCMINST=X on Windows host. $HORCMINST This connects to %HORCMINST% setting on this Windows host. Enter the username and password as dummy. After you enter the username and password, click Connect and then click OK to discover P- VOL LUNs on the protected array. Verify that the array managers are discovered the following information. Local Array ID and Remote Array ID information LUN count (equals the number of P-VOLs LUNs) 34 Configuring SRM and the disk array

35 3. Add the P9000 Virtualization Adapter to SRM on the recovery array. From the Configure Array Managers window, click Add to add connection information for the P9000 Virtualization Adapter. Configuration using a remote connection: Select the P9000 Virtualization Adapter from Manager Type that is connected to the P9000 Virtualization Adapter. Enter the Alias to connect to an RAID Manager instance via the P9000 Virtualization Adapter. P9000 Virtualization Adapter Administrator Guide 35

36 Connecting to RAID Manager on remote UNIX Host: This connects to HORCMINST=X on the Remote UNIX Host without %HORCMROOT%. This connects to $HORCMINST setting on a remote login environment on a UNIX Host without %HORCMROOT%. Connecting to RAID Manager on remote windows Host: HORCMINST=X@Host-name This connects to HORCMINST=X on the Remote Windows Host by specifying %HORCM- ROOT%. $HORCMINST@Host-name This connects to %HORCMINST% setting on a remote login environment on a Windows Host by specifying %HORCMROOT%. After you enter a username and password, click Connect and then OK to discover P-VOL LUNs on the protected array. Configuration using local connection: Select the P9000 Virtualization Adapter from the Manager type that is connected to the P9000 Virtualization Adapter. Enter the Alias to connect to the RAID Manager instance via the P9000 Virtualization Adapter. 36 Configuring SRM and the disk array

37 Connecting to RAID Manager on Windows: HORCMINST=X This connects to HORCMINST=X on this Windows Host. $HORCMINST This connects to %HORCMINST% setting to this Windows Host. Enter the username and password as dummy. After entering the username and password, click Connect and then OK to discover P-VOL LUNs on the protected array. Verify that the protection array managers are discovered the remote Array ID as following information. Local and remote Array ID information LUN count - equals the number of P-VOLs LUNs P9000 Virtualization Adapter Administrator Guide 37

38 4. Verify Replicated Datastore. Verify the array managers are configured for both protected and recovery sites, and replicated datastores in the Review Replicated Datastores window. 5. End Array Manager. Verify that SRM is configured in Array Manager for both protected and recovery sites. 38 Configuring SRM and the disk array

39 Create protection groups You are ready to create protection groups in the protected site using SRM. Protection groups (containing virtual machines) fail over together to the recovery site during test and recovery. For more information on creating protection groups, see the VMware Site Recovery Manager Administration Guide. Create recovery plans Create recovery plans on the recovery site. A recovery plan is a list of steps the SRM follows on a protected site to switch operation from the protected site to the recovery site during a disaster (or test). For more information on creating recovery plans, see the VMware Site Recovery Manager Administration Guide. P9000 Virtualization Adapter Administrator Guide 39

40 40 Configuring SRM and the disk array

41 5 Failover and failback with SRM This chapter discusses failover and failback with SRM. It also provides a scenario as a guide for the manual failback process. Failover Failover occurs when an SRM recovery plan is executed, and the SRM recovery plan is configured to fail over SRM protection groups that use replicated P-VOLs and S-VOLs on a disk array as a datastore. During normal operation, the replication path is from a protected site (local site) to a recovery site (remote site). When a failure occurs at the local site (due to hardware failure or the entire site loss), a failover occurs and the replication path is reversed from the recovery site to the protected site. With the P9000 Virtualization Adapter and SRM, the failover process is an automated feature and the process is executed using a recovery plan located at the recovery site. Failback Failback is the process of setting the replication environment back to its original state at the protected site (local site) after a failover. Failback can be managed as a normal server migration process. Managing the failback with SRM is a manual process and the steps vary with respect to the degree of failure at the protected site (local site). For example, the failover could have been due to a hardware error or the loss of the entire data center. Failback scenario with SRM In the following failback scenario, SRM is used as a failback tool to return a protected site (Site A) to its original state after executing a recovery plan (R1) at a recovery site (Site B). To execute a failback manually after site A is recovered: 1. Delete recovery plan at the recovery site (Site B). 2. If the protected site (Site A) still has DR protection configured for the protection groups (P1) in recovery plan (R1), delete protection groups (P1) at the protected site. This could be the case if some of the hardware at the protected site was not replaced after a disaster and you are employing the existing hardware. If new hardware has been installed, this is not necessary. 3. If you have not done so already, install SRM and P9000 SRA on the updated hardware at the protected site (Site A). 4. Using the VI Client at the recovery site (Site B), establish the protected site (Site A) as the secondary site for the recovery site (Site B). This means you create site B as protected site and site A as the recovery site just for the purpose of failing back the Continuous Access device group and the VMs on it. P9000 Virtualization Adapter Administrator Guide 41

42 5. Configure array replication from the recovery site (Site B) to the protected site (Site A) for datastores containing recovered virtual machines. 6. Create any protection groups (P2) at the recovery site (Site B) to protect the recovered virtual machines at the protected site (Site A). 7. On the protected machine (Site A), create a recovery plan (R2) for the protection groups (P2). 8. After the virtual machines have been fully replicated to the protected site (Site A), execute recovery plan R2 at the protected site (Site A) in test mode. 9. If the test is successful, execute the recovery plan (R2) in recovery mode. If needed, you can now protect the recovered virtual machines on the protected site (Site A) back to the recovery site (Site B). To do this: 1. Delete the recovery plan R2 from the protected site (Site A). 2. Delete the protection groups P2 at the recovery site (Site B). 3. Configure array replication from the recovery site (Site B) to the protected site (Site A) for datastores containing recovered virtual machines. 4. Create protection groups P3 at the protected site (Site A) to protect recovered virtual machines from the protected site (Site A) to the recovery site (Site B). 5. Create a recovery plan R3 at the recovery site (Site B) for the protection groups. 42 Failover and failback with SRM

43 6 Troubleshooting General troubleshooting Use the following solutions to troubleshoot general issues: Disk arrays do not appear in Site Recovery Manager. Solutions: If the disk arrays are not appearing in Site Recovery Manager, make sure that Site Recovery Manager has been restarted (see step 3 in Installation procedures on page 27). Make sure that the array manager that SRM is connected to is actively managing arrays. The P9000 Virtualization Adapter does not install. Solution Check that SRM is installed. the P9000 Virtualization Adapter does not install if SRM is not on the disk array. Virtual machines do not start at recovery site after failover. Cause The ESX recovery hosts are not configured with the correct HBA port WWN on the recovery XP. The replicated LUNs used in failover were not presented to the recovery ESX servers. Virtual machines do not start at recovery site during failover testing. Cause The ESX recovery hosts are not configured with the correct HBA port WWN on the recovery XP. Error messages on SRM log files The P9000 Virtualization Adapter error messages are identified in the SRM log files with P9000 Virtualization Adapter shown in the message. To identify the P9000 Virtualization Adapter error messages, search the SRM log files for the P9000 Virtualization Adapter. SRM log file is located in the following directory (default): C:\Documents and Settings\All Users\Application Data\VMware\VMware Site Recovery Manager\Logs\vmware*.log NOTE: Logs roll over after reaching 5 MB by default. The vmware-dr-index contains the most recent Log File number. Errors in XML passed from SRM [P9000 Virtualization Adapter] [Time]: [command_main] : Parameter in XML was NOT enough. P9000 Virtualization Adapter Administrator Guide 43

44 Cause A parameter in XML was entered from SRM to the P9000 Virtualization Adapter, but it could not be found in any parameters. Action to be taken Confirm that SRM has passed appropriate parameters in XML from its own SRM log message. [P9000 Virtualization Adapter] [Time]: [command_discoverluns] : NO arrayid in XML. Cause A parameter in XML (discoverluns) was entered from SRM to the P9000 Virtualization Adapter, but it could not find the Array ID. Action to be taken Confirm that SRM has passed Array ID parameter in XML (discoverluns) from own SRM log message. [P9000 Virtualization Adapter] [Time]: [command_failover] : NO ReplicaLunKey in XML. Cause A parameter in XML (failover) was entered from SRM to the P9000 Virtualization Adapter, but it could not be found ReplicaLunKey (LDEV number of Continuous Access_S-VOL). Action to be taken Confirm that SRM has passed ReplicaLunKey parameter in XML (failover) from own SRM log message. [P9000 Virtualization Adapter] [Time]: [command_testfailover] : Action in XML was NOT start/stop. Cause A parameter in XML failover test was entered from SRM to the P9000 Virtualization Adapter, but it could not be found start/stop as Action. Action to be taken Confirm that SRM has passed start/stop parameter in XML failover test from own SRM log message. [P9000 Virtualization Adapter] [Time]: [command_testfailover] : NO ReplicaLunKey in XML. Cause A parameter in XML failover test was entered from SRM to the P9000 Virtualization Adapter, but it could not be found ReplicaLunKey (LDEV number of Continuous Access_S-VOL). Action to be taken Confirm that SRM has passed ReplicaLunKey parameter in XML failover test from own SRM log message. [P9000 Virtualization Adapter] [Time]: [command_main] : Can't be connected to HORCMINST=X@ with error(0x000000fc). Cause A connection address in XML was entered from SRM to the P9000 Virtualization Adapter, but it could not be found HORCM instance number X. Action to be taken Check that HORCM instance number is running, or a connection address (Alias) specified in configuration of Array Manager is appropriate. RAID Manager command errors in rmsra.exe [P9000 Virtualization Adapter] [Time]: [ XML OUTPUT file name ] : fopen : system error message Cause A parameter in XML was entered from SRM to the P9000 Virtualization Adapter, but it could not be created XML OUTPUT file name. Action to be taken Confirm that SRM has passed appropriate output file in XML from own SRM log message, or take action by referring to system error message. [P9000 Virtualization Adapter] [Time]: [system()] : Command line : system error message Cause An execution of Command line has failed via system() call. Action to be taken Confirm that RAID Manager is installed, or the path of Command line is correct, or %HORCMROOT% ENV has been set, or make action by referring to system error message. [P9000 Virtualization Adapter] [Time]: [ Command line ] : popen : system error message Cause An execution of Command line has failed via popen() call. Action to be taken Confirm that RAID Manager is installed, or the path of Command line is correct, or %HORCMROOT% ENV has been set, or make action by referring to system error message. 44 Troubleshooting

45 [P9000 Virtualization Adapter] [Time]: [ ] : malloc : system error message Cause Could not retain the memory for executing the P9000 Virtualization Adapter. Action to be taken Increase the capacity of virtual memory of the whole system or terminate unnecessary programs or Daemon processes running simultaneously. [P9000 Virtualization Adapter] [Time]: [ ] : Command line failed with RC=XXX. Cause The RAID Manager command Command line has failed with RC=XXX. Action to be taken Remove the cause of the error after confirming the RAID Manager error code and command error log messages output in this log file, as shown in the example below: COMMAND ERROR : EUserId for HORC[24] : root (0) Thu Jul 17 18:38: CMDLINE : pairdisplay -IH -d CLI -l -fwe 18:38: ERROR:cm_sndrcv[rc < 0 from HORCM] 18:38:55-4c5e Could not find a group on configuration file for this LDEV.(Port#?,Seq# 64015,LDEV# 9,mun# 0) 18:38:55-51feb [pairdisplay][exit(239)] [EX_ENOGRP] No such group [Cause ]:The group name which was designated or the device name doesn't exist in the configuration file,or the network address for remote communication doesn't exist. [Action]:Please confirm if the group name exists in the configuration file of the local and remote host Configuration and status errors [P9000 Virtualization Adapter] [Time]: [disarray_exe] : Unknown PWWN. Cause Could not find the Port WWN in the output of pairdisplay fw command via discoverarrays. Action to be taken Confirm that RAID Manager is a version supported by the P9000 Virtualization Adapter. [P9000 Virtualization Adapter] [Time]: [dislun_exe] : invalid arrayid (- ). Cause A parameter in XML (discoverluns) was entered from SRM to the P9000 Virtualization Adapter, but it could not be found correct Array ID. Action to be taken Confirm that SRM has passed an Array ID parameter in XML (discoverluns) from own SRM log message [P9000 Virtualization Adapter] [Time]: [dislun_exe] : Unknown PWWN. Cause Could not find the Port WWN in the output of pairdisplay fw command via discoverluns. Action to be taken Confirm that RAID Manager is a version supported by the P9000 Virtualization Adapter. [P9000 Virtualization Adapter] [Time]: [failover_chk] : Command line? P/S =, Status =, Fence =. Cause The pair status of a target volume specified with failover is inappropriate status (SMPL, P- VOL, or COPY). P9000 Virtualization Adapter Administrator Guide 45

46 Action to be taken Confirm that volume status is appropriate status (Continuous Access is S-VOL and PAIR) using the pairdisplay command. [P9000 Virtualization Adapter] [Time]: [failover_chk] : The output of Command line is missing. Cause Could not find the correct format in the output of Command line command via failover. Action to be taken Confirm that RAID Manager is a version supported by the P9000 Virtualization Adapter. [P9000 Virtualization Adapter] [Time]: [testfailover_chk] : Command line? P/S =, Status =, CTG =. Cause The pair status of a target volume specified with failover testing is inappropriate status (SMPL, P-VOL, or NOT PAIR). Action to be taken Confirm that volume status is the appropriate status using the pairdisplay command (for Business Copy, S-VOL and PAIR). [P9000 Virtualization Adapter] [Time]: [testfailover_chk] : The output of Command line is missing. Cause Could not find the correct format in the output of Command line command via failover testing. Action to be taken Confirm that RAID Manager is a version supported by the P9000 Virtualization Adapter. [P9000 Virtualization Adapter] [Time]: [failover_exe] : invalid arrayid (- ). Cause An XML failover parameter was received from SRM to the P9000 Virtualization Adapter, but the Array ID could not be found. Action to be taken Use the SRM log message file to confirm the Array ID parameter and XML failover parameter. [P9000 Virtualization Adapter] [Time]: [testfailover_exe] : invalid arrayid (- ). Cause A XML failover testing parameter was received from SRM to the P9000 Virtualization Adapter, but the Array ID could not be found. Action to be taken Use the SRM log message file to confirm the Array ID parameter and XML failover testing parameter. Troubleshooting information for technical support Before calling technical support, collect the following information: For SRM and the P9000 Virtualization Adapter local configuration: On Windows where SRM is performing: 1. Collect the SRM log file on Windows on both the protection and recovery sites. Collect the following SRM log file, including the error messages of the P9000 Virtualization Adapter, SRM ERROR messages, and RAID Manager command error log: %ALLUSERSPROFILE%\ Application Data\VMware\VMware Site Recovery Manager\Logs\vmware*.log 2. Collect the outputs of the following command on HORCMINST=XX (instance# for the P9000 Virtualization Adapter) set %HORCMROOT%\HORCM\etc\raidqry -l %HORCMROOT%\HORCM\etc\raidqry -g %HORCMROOT%\HORCM\etc\pairdisplay IH g??? CLI l fwe (where??? is a group name shown by raidqry g ) 46 Troubleshooting