3PAR InForm Operating System Upgrade Pre-Planning Guide. 3PAR, Inc Technology Drive Fremont, CA USA

Transcription

1 3PAR InForm Operating System Upgrade Pre-Planning Guide 3PAR, Inc Technology Drive Fremont, CA USA QL May 2011

2 Changes The material in this document is for information only and is subject to change without notice. While reasonable efforts have been made in the preparation of this document to assure its accuracy, 3PAR Inc. assumes no liability resulting from errors or omissions in this document or from the use of the information contained herein. 3PAR reserves the right to make changes in the product design without reservation and without notification to its users. Updates to the Documentation at 3PAR Central Any updates to this document, or to other 3PAR technical documents, can be found by logging in to 3PAR Central s Document Control System from 3PAR s Support page at 3PAR Technical Support and Services Contact your local service provider for technical support and services at: Sales and Ordering Information For sales and ordering information, contact: 3PAR Inc Technology Drive Fremont, CA USA Telephone: Fax: salesinfo@3par.com Reader Comments and Suggestions Please your comments and suggestions about this document to ReaderComments@3PAR.com. Copyrights PAR Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written consent of 3PAR Inc., 4209 Technology Drive, Fremont, CA The users may print one copy for personal use. Production of multiple copies or copies for sale is strictly prohibited. Trademarks 3PAR, InServ, InForm, InSpire and Serving Information are registered trademarks of 3PAR Inc. All other trademarks and registered trademarks are owned by their respective owners. 2

3 InForm OS Version Release Upgrade Pre-Planning Guide The following topics are covered in this guide: Introduction 3 InForm OS Update Overview 4 Guidelines for Online Updates 5 Planning Information for Online InForm OS Updates 14 Veritas DMP and Online Updates 15 Sun Cluster Configuration Guidelines 16 Exanet ExaStore Clustered NAS System Prerequisites 17 VMware ESX Server 18 Remote versus Local Methods for InForm OS Update Introduction This guide is intended for use as a planning tool for 3PAR customers and provides the following information on 3PAR InForm Operating System (InForm OS) software updates: An overview of the online and offline update processes for the 3PAR InForm OS Pre-requisites for online updates Guidelines for online updates Planning information needed for online updates Veritas DMP prerequisites for online updates Remote versus local methods of upgrading NOTE: In addition to this planning document, the 3PAR InForm OS Configuration Matrix (P/N ) and Implementation Guides are available on 3PAR s Document Control System (DCS). These documents are provided to assist you in setting up host servers with 3PAR supported configuration information. 3

4 Release Upgrade Pre-Planning Guide InForm OS Version NOTE: 3PAR recommends updating host drivers to a supported level on InForm OS before attempting an online update. Refer to the 3PAR InForm OS Configuration Matrix (P/N ) for more information about supported driver levels on InForm OS InForm OS Update Overview The 3PAR InServ Storage Servers support two different methods for updating the InForm OS: online and offline. The updates can be performed with a service representative on-site or remotely (from 3PAR). Remote updates require that remote operations are available. 2.1 Online Updates The InServ Storage Servers support online, non-disruptive updates of the InForm OS. Online updates are achieved by installing the new software on all the nodes in an alternate location designated as the new software repository. The update proceeds by loading the new revision of the software on half of the controller nodes in the system and resetting them to reload the software into memory. Rigorous design and testing assures that this process is non-disruptive to hosts, if all of the prerequisites described in this document are met. Once the first set of nodes have successfully loaded the new software, the second set of controller nodes proceeds to reset and reload the new revision of the software. The update process is considered complete and successful once the second set of nodes has re-initialized and starts using the new revision of the software. 2.2 Online Updates of Maintenance Updates Online updates of InForm OS Maintenance Updates are different from an InForm OS update to a new level (for example, to is a new level). A maintenance update proceeds by loading the maintenance update in an alternate location on the controller nodes internal disk. After the loading of the maintenance update to the alternate location has completed, one node at a time will be reset so that it can load the maintenance update. The update process is successful once all nodes in the system have reinitialized with the maintenance update version of the software. 2.3 Offline Updates Offline updates require that all hosts connected to the InServ Storage Server should be shut down before the InForm OS update. Offline updates are achieved by installing the new 4

5 InForm OS Version Release Upgrade Pre-Planning Guide software on all the nodes in an alternate location designated as the new software repository. The update proceeds by loading the new revision of the software on all of the controller nodes within the system simultaneously. The update process is considered complete and successful once all the nodes have re-initialized using the new revision of the software. After this is complete, hosts connected to the InServ Storage Server can be re-started. 3.0 Guidelines for Online Updates Several guidelines need to be followed for online InForm OS updates. The guidelines are divided into several categories: general, host configuration, and storage server configuration guidelines. 3.1 General Guidelines Observe the following general guidelines in respect to online InForm OS updates: Hardware and configuration changes should not be performed on the InServ Storage Server during the update. For example, software operations (such as servicemag, createaldvv, or use of the 3PAR System Tuner commands) and hardware operations (such as adding/removing drives, cages, etc.) are prohibited during the online update process. All logical disk growth operations for Common Provisioning Groups continue uninterrupted as under normal operation. The storage server must be in a healthy state (e.g. no failed or degraded physical disks, cages, FCALs, nodes, etc.). The InForm OS update software performs an automated check. No host or environment changes are allowed during an online update (for example, rebooting, applying patches, or re-zoning switches). The update software checks for the existence of hosts before and after each step, and if host connections do not reappear after each step, the update process reverts to the original level. Any automated processes that communicate with the InServ Storage Server s management interface (scripting) must be stopped before performing an online update. Online updates should be scheduled and performed during periods of low I/O activity. As system resources are placed offline to be updated, system utilization must be lower than 50% for each node and IO port. All hosts must be compliant with the current applicable Implementation Guides for the target InForm OS level. 5

6 Release Upgrade Pre-Planning Guide InForm OS Version Host Configuration Guidelines Observe the following host configuration guidelines when planning an online InForm OS update: All attached hosts must have multi-pathing software in place and operational for every storage server volume exported, and the host(s) must be connected to two horizontally adjacent nodes on the InServ Storage Server. NOTE: Check the 3PAR Configuration Matrix for supported multi-pathing environments. WARNING: An InServ Storage Server running InForm OS or lower version, utilizing iscsi with CHAP authentication connecting to all supported hosts will be affected in two scenarios:. a revert to or an InServ reboot (shutdownsys reboot) on InForm OS or lower version, will fail to reenable all paths to a host. Advise that all hosts connecting this way should be upgraded to MU6. If during the upgrade a revert does occur, then on the Inserv re-issue the target CHAP password. This must be followed by issuing the discovery of targets command on the host. This will re-enable all previous CHAP enabled iscsi paths to the host. NOTE: For unsupported environments, the hosts should be shut down (and in some cases powered off), or the VLUNs exported to these hosts must be unexported, or FC logically disconnected before execution of the online InForm OS update. Multi-pathing cabling must adhere to 3PAR configuration rules: each defined host must have at least two operational paths attached to two horizontally adjacent nodes (i.e. evennumbered node, followed by a consecutive odd-numbered node. Valid pairings are 0 & 1, 2 & 3, 4 & 5, 6 & 7. Invalid pairings are 2 & 1, 0 & 3.). All host paths must be identified with a single host name. 6

7 InForm OS Version Release Upgrade Pre-Planning Guide All system administration activities must be stopped while performing an Inform OS Online Update. This includes any provisioning activity, physical/virtual copy activity, and dynamic optimization. All active tasks must complete or be stopped before proceeding. Be sure to have the customer stop any automated administration scripts that may be running as well (Recovery Manager and VSS Provider for Microsoft Windows). For VxVM DMP, depending on the level, the customer should set the link restore interval to a value of 1 second before starting the online update. This causes the DMP software to react to the return of missing paths more quickly than it normally would. See the Veritas DMP and Online Updates section for instructions. After the online update, reset the link restore interval to the previously configured value. Follow this procedure for each host that is running VxVM for its DMP. Linux hosts using Veritas VXVM 4.1 with MP4 or higher do not require this change. Linux hosts using Veritas VXVM 5.0 with MP3 or higher do not require this change. Solaris hosts using Veritas VXVM 4.1 with MP2 or higher do not require this change. Solaris hosts using Veritas VXVM 5.0 with MP1 or higher do not require this change. Solaris hosts using Veritas VXVM 5.1_GA or higher do not require this change. AIX hosts using Veritas VXVM 5.0 with MP1 and 3PARODM 1.2 or higher do not require this change. With Windows 2008, using the Microsoft native MPIO driver, the load-balancing policy defaults to Active / Passive. 3PAR requires the load-balancing policy be set to Active / Active. Refer to the 3PAR Implementation Guide for more details on configuring the loadbalancing policy. 7

8 Release Upgrade Pre-Planning Guide InForm OS Version With Windows 2008 and 2008R2 Failover Clusters, including Failover Clusters as guest OS clustering in a virtual environment (for example: Win2008 R2 Failover Cluster as virtual machines on VMware ESX), server node-failover can occur while performing online software updates of the InForm OS. In order to avoid a server node-failover during InForm OS updates, 3PAR requires all data LUNs that are presented to the Microsoft cluster from the 3PAR InServ Storage Server first to be put into Maintenance Mode. Once the InForm OS online update is completed, the maintenance mode must be turned off. To turn ON/OFF maintenance mode for each of the cluster 3PAR disks start the Failover Cluster Manager application. Double-click the storage icon, and a summary of the storage will be displayed on a window. For each of the clustered 3PAR data disks (not including any Witness / Quorum disks), right-click and select the More Actions option for the menu and either turn ON of OFF maintenance for the disk. For details, see the following example: NOTE: Does not apply to Windows 2008 systems running Veritas DMP driver. For Windows 2008 and Windows 2008R2 using MS MPIO, Microsoft Hotfix must be applied before InServ update in accordance with: Microsoft KB for Windows 2008R2 8

9 InForm OS Version Release Upgrade Pre-Planning Guide Microsoft KB for Windows 2008 The SCSI timeout value for RH4, SuSE 9 must be changed from the default of 30 seconds to 60 seconds. Refer to the specific Implementation Guide for changing the timeout value. This needs to be performed for both Fibre Channel and iscsi connected hosts. QLogic drivers for SUSE 9, 10 and RH 4, 5 should be running with driver parameter PORT- DOWN retry count (qlport down retry) set to 1. Refer to the specific Implementation Guide for setting the value. Solaris 10 MU3 with an iscsi configuration is not supported for online updates. For hosts using Oracle Clusterware, make the following required changes or adjustments before performing an update: For 10gR x and 10gR (without bug patched), set css miscount to 195 (# set css miscount=195). For 10gR (with bug patched) and later, 11gR1, and 11gR2, css disktimeout must not be lowered from its default value of 200. For all Linux hosts with Oracle Clusterware, install and adjust Linux Hangcheck tick parameters to 30 seconds. Even if a direct connected HP host running Tachyon based HBAs (example A5158A, A6795A) is configured correctly, if the HP host is not doing I/O during the online update, the update may revert to the previous software level. The work-around for this requires ioscan or I/O to be performed while doing an online update if the direct connect HP host does not have mounted file systems. This is not applicable for QLogic based HP cards (example A6826A, AB3790). VMware ESX Server servers with Emulex HBAs (lpfcdd-v732 driver) that connect to an InServ Storage Server must have VMware patch ESX VMware ESX / ESXi server - Active/active, with a path policy of MRU (Most Recent Used) does not maintain or re-instate balancing of I/O load after a failover/failback multipath event. This could leave I/O in an unplanned for and unbalanced state which may yield significant I/O performance issues. Implementation of a MRU (Most Recent Used) path policy is not recommended by 3PAR. Site Recovery Manager and Adapter requires a newer version to be compatible with InForm OS Review VMware s Site Recovery Manager Storage Partner Compatibility Matrix located here for details: 9

10 Release Upgrade Pre-Planning Guide InForm OS Version Recovery Manager for Oracle, SQL, Exchange or VMWare, may require a newer version to be compatible to InForm OS Refer to the Configuration Matrix for supported levels. Please verify compatibility of any host 3rd party software to the InForm OS release System Reporter may require an update to be compatible to the InForm OS release Refer to the Configuration Matrix for supported levels. The InForm Management Console (IMC) is generally compatible with all and InForm OS levels. It is advisable to upgrade to the current IMC release for availability of the latest features and fixes. 10

11 InForm OS Version Release Upgrade Pre-Planning Guide InForm OS Persona Changes Use Table 1-1 below when migrating from InForm OS 2.2.x to The change occurs automatically as part of the Rolling Upgrade or by Offline Upgrade. NOTE: VCN will be disabled by default on all ports when InServ Storage Server migration to completes. Table 1-1. Persona Translation Table 2.2.x Port Persona - Topology Host Persona - Topology 0 - Loop (No persona) - Loop 1, 2, 3, 4 - Loop 6 (Generic - Legacy) - Loop 5 (HPUX) - Loop 7 (HPUX - Legacy) - Loop 8 (HPUX) - Point 7 (HPUX -Legacy) - Point 7, 9, 10, 15 - Point 6 (Generic - Legacy) - Point 11 (Egenera) - Loop 9 (Egenera) - Loop 12 (Egenera) - Point 9 (Egenera) - Point 13 (AIX) - Loop 8 (AIX - Legacy) - Loop 14 (AIX) - Point 8 (AIX - Legacy) - Point 18 (NetApp) - Loop 10 (ONTAP - Legacy) - Loop 19 (NetApp) - Point 10 (ONTAP - Legacy) - Point 16 (RCFC) - Loop (No persona) - Loop 17 (RCFC) - Point (No persona) - Point All iscsi configurations 6 (Generic - Legacy) 11

12 Release Upgrade Pre-Planning Guide InForm OS Version Once that InServ Storage Server is running InForm OS it is recommended that the Host Persona for some of the hosts be changed. SeeTable 2 below. Table 2. Host Persona Details Table Persona Operating System when migrated to Change to Persona Notes Red Hat 4 and 5, SUSE 10 and 11 - Fibre channel and iscsi (Except SUSE 11 iscsi, which should be left at Persona 6) 6 1 Use of Generic persona allows the Host server to run Host Explorer on the host. It will automatically create LUN 254 as a SCSI Enclosure device for Host Explorer use purpose. W2k3 including iscsi 6 1 Solaris including iscsi 6 1 W2k8 SP1 and SP2 including iscsi 6 2 Enabling Host Persona 2 will enabled A/ A multipath on LUNs by default This change will require a reboot. DO NOT change the Host Persona unless you can reboot the W2k8 server. If you do not reboot the server, the host will lose the mapping of the LUNs. W2k8 R2 including iscsi 6 1 Microsoft enables A/A multipathing by default at Windows 2008 R2 VMware (3.5, 4.0) 6 6 For VMware 4.0 only change the SATP rules on host to enable Active/Active Round Robin by default. Refer to the ESX IG for instruction of how to change the ESX 4.0 SATP rules. To change the Host Persona on a host: cli% sethost persona <persona _number> <host_name> 12

13 InForm OS Version Release Upgrade Pre-Planning Guide To verify the Host Persona setting on a host: cli% showhost persona <host_name> NOTE: If a host server is powered down during an upgrade from 2.2.x to the recommended Host Persona for the type of host that is down should apply, as the upgrade assigns Host Persona 6 to the server. In some cases, it would prevent the host from operating correctly, as is the case with HPUX, AIX, and NetApp. 3.3 Storage Server Configuration Guidelines The following guidelines must be observed when planning an online InForm OS update: VLUN for each exported volume must be exported to a defined host on at least two fully operational paths that span horizontally adjacent storage server nodes (valid pairings are 0 & 1, 2 & 3, 4 & 5, 6 & 7). 3PAR software applications running on the InServ Storage Server must be shut down. 3PAR Remote Copy must be stopped for all systems running below InForm OS 2.2.3, and no full or physical copies may be in progress. Ensure that no 3PAR Virtual Copy promotes are in progress. To confirm, check that no volumes are in the started, cpp state. If any promotes are in progress, wait until the promote finishes to begin the online update. For systems running Remote Copy below InForm OS 2.2.3, stop the synchronization processes for the duration of the update by using the following command: stoprcopy stopgroups. For systems running InForm OS with Remote Copy, only the groups needs to be stopped. During the process of InForm OS upgrades, plan on upgrading both InServ Storage Servers in the Remote Copy pair within a five week period. The difference in the InForm OS Release versions cannot be greater than adjoining release levels,. i.e and (all MU versions are treated as the same OS version). Starting with InForm OS 2.2.2, Remote Copy Ethernet ports must be set to autonegotiate. If they were changed to 100Mbs full duplex, the links may not start when performing a software update to InForm OS The auto-negotiate setting can be changed in advance of a software update by executing controlport rcip speed auto {value} {n:s:p}. 13

14 Release Upgrade Pre-Planning Guide InForm OS Version This should be performed on one RC link at a time, making sure to check that the link is working before setting the other link to autonegotiate. Remote Copy VVs must be the same size on the primary and the secondary system. Earlier releases of InForm OS allowed the secondary VV in a remote copy group to be larger than the primary. Check to insure that all volumes in a remote copy group are equal sizes on the primary and secondary InServ Storage Server. 4.0 Planning Information for Online InForm OS Updates The host connectivity environment must adhere to the qualified and supported environments (see the 3PAR InForm OS Configuration Matrix). 3PAR service representatives will request this information for review before performing an online update. Relevant details include: Host platform Architecture (i.e. SPARC/x86) OS (version and patch level) DMP software HBA (vendor, model, driver, firmware, and version. Also include FCODE and BIOS level if booting from InServ Storage Server) Switch (vendor, model and firmware) Information on host software provided by 3PAR is needed. Examples include: 3PAR Recovery Manager (SQL, Exchange, Oracle), 3PAR VSS Provider, 3PAR MPIO for Microsoft Windows, 3PAR ODM Definition for IBM AIX, 3PAR System Reporter, etc. In addition to verifying the driver/firmware levels match the Configuration Matrix, there may be host configuration settings that are needed as documented in the 3PAR Implementation Guides. These settings need to be verified before an online update is performed. 14

15 InForm OS Version Release Upgrade Pre-Planning Guide 5.0 Veritas DMP and Online Updates For all servers that are connected to the InServ Storage Server and use Veritas DMP for the multi-pathing solution (except as noted on page 6), the default value of the link restore interval must be changed before performing an online update of the InForm OS. 5.1 Changing the Veritas DMP Link Restore Interval The following procedure is required if you are performing an online update to the InForm OS and Veritas DMP is being used for the multi-pathing solution on the host. The procedure must be repeated for each host server connected to the InServ storage server and using Veritas DMP. Use the following procedure to gather the current settings and change the link restore interval: 1 Display the current setting of the link restore interval as follows. Record the interval so that it can be restored later. vxdmpadm stat restored (note the current setting) 2 Stop the daemon. vxdmpadm stop restore 3 Restart the daemon with the interval set to 1. vxdmpadm start restore interval=1 4 Verify the daemon is running with the interval set to 1. vxdmpadm stat restored NOTE: The restore daemon reverts to the default value of 300, if, or when a host reboot occurs. 15

16 Release Upgrade Pre-Planning Guide InForm OS Version Restoring the Veritas DMP Link Restore Interval If you are using Veritas DMP, use the following commands to change the link restore interval to the previous setting on storage server hosts after completing an online update. The following procedure must be repeated for each host server connected to the InServ Storage Server and using Veritas DMP. Use the following procedure to restore the link restore interval to its original setting: 1 Stop the daemon. vxdmpadm stop restore 2 Restart the daemon with the interval set to what it was previously. vxdmpadm start restore interval=<prior setting> 3 Verify the daemon is running with the interval set to the prior setting. vxdmpadm stat restored 6.0 Sun Cluster Configuration Guidelines There are some InForm OS online configuration rules to be aware of for Sun cluster configurations. InForm OS online upgrades with attached Sun cluster hosts are not supported by 3PAR for major InForm OS releases. (Example to is considered a major InForm OS release). As of MU2, online maintenance updates can be successfully performed using the pause option for the upgrade, e.g MU2 to MU4. Implementing the pause option for an online maintenance update will cause a wait for user input in between each node reboot needed during an online InForm OS update. This pause time in the upgrade process is used to allow all of the I/O paths to all of the Sun Cluster nodes, for all of the presented LUNs to recover (failback). In Sun Cluster, this is a function of Solaris SSTM MPxIO. Failure to wait for all I/O paths to return results in fatal I/O errors and off-lining of Sun Cluster resources. 16

17 InForm OS Version Release Upgrade Pre-Planning Guide The recovery of the I/O paths must be monitored at each individual Sun Cluster node. Status of the Sun Cluster I/O paths cannot be assessed from the InServ Storage Server. On each individual Sun Cluster node the command mpathadm list lu is run, and the output is analyzed to see that all paths to all devices are enabled. Identifying all paths as enabled is accomplished by noting that Operational Path Count: equals Total Path Count: for all devices in the list. The command mpathadm list lu outputs the current path status. Running the command multiple times on each Sun Cluster node may be necessary in order to monitor the paths as they come up. The upgrade should be paused until Operational Path Count matches Total Path Count in the mpathadm list lu output for all devices on all Sun Cluster nodes. Note that the pause time necessary may be many minutes, depending on the number of host cluster nodes, I/O paths, and devices. Sample output; in this case all paths are up: # mpathadm list lu /dev/rdsk/c4t50002ac d8d0s2 Total Path Count: 4 Operational Path Count: 4 Sample output; in this case all paths are not up: NOTE: Only two of four paths are up in the output below, continue to wait until Total Path Count equals Operational Path Count. /dev/rdsk/c4t50002ac d8d0s2 Total Path Count: 4 Operational Path Count: Exanet ExaStore Clustered NAS System Prerequisites There are special procedures that will need to be performed on an Exanet ExaStore v4.0.x Clustered NAS system when a 3PAR InForm OS update needs to take place. In the standard ExaStore configuration, NAS clients that connect to the ExaStore might experience excessive periods of I/O quiescence during the InForm OS upgrade. Contact Exanet for details on obtaining this script as well as specific usage instructions. An overview of the process is included for quick reference: 17

18 Release Upgrade Pre-Planning Guide InForm OS Version The procedure includes running two scripts: 1 Pre-upgrade script, which performs the following: Modifies appropriate configuration values. Generates the post-upgrade script so that the system can be returned to the standard configuration after the 3PAR InForm OS software update is completed. 2 Post-upgrade script, which sets all appropriate values that were modified in the Preupgrade script back to their original values. 8.0 VMware ESX Server Once the InServ Storage Server is upgraded to InForm OS 2.3.1, the VMware ESX Server Scsi.ConflictRetries parameter should be returned to its default value or 80 on all VMware ESX servers that connect to the InServ, if previously changed to 200 as directed by 3PAR. InForm OS contains a fix for the issue in previous InForm OS versions where ESX SCSI-2 reservation release commands could be delayed by the Sysmgr configuration lock resulting in virtual machines I/O errors. 9.0 Remote versus Local Methods for InForm OS Update Read this section for information regarding the difference between a remote online update and a local online update. 9.1 Remote InForm OS Updates Remote InForm OS updates are achieved by utilizing 3PAR s Remote Operations Service Tools (RemOps) (e.g., via Ethernet or modem). The process used for remote InForm OS updates depends on the connectivity model allowed for remote updates. There are three connectivity models: Ethernet RemOps - Service processor software updates can be downloaded, staged, and performed remotely. Modem RemOps -The service processor software CD can be supplied to the customer to place in the service processor CD-ROM drive, and the update can be performed remotely. 18

19 InForm OS Version Release Upgrade Pre-Planning Guide Temporary RemOps - In some cases, customers will temporarily open up connectivity to the service processor for Remote Operations for software updates. If temporary Remote Operations access can be granted, we will be able to perform the update remotely. 9.2 Local InForm OS Updates Local InForm OS updates are achieved by utilizing an on-site service representative to perform the update. Typically, local InForm OS updates are performed when there is no remote connection to the service processor. Additionally, a local InForm OS update may be combined with other on-site activities, such as hardware updates or maintenance. 19

20 20 Release Upgrade Pre-Planning Guide InForm OS Version 2.3.1