SSW-1988 Planning for Disaster Recovery and High Availability with IBM Spectrum Protect

Transcription

1 SSW-1988 Planning for Disaster Recovery and High Availability with IBM Spectrum Protect Dave Daun, IBM Solutions Response Team 2016 IBM Corporation

2 Topics Understanding the costs and benefits of various disaster recovery methods Using high availability techniques with Spectrum Protect to recover from localized disasters Using Spectrum Protect Disaster Recovery Manager (DRM) for protection against large-scale disasters Exploring electronic vaulting techniques with Spectrum Protect 1

3 High Availability / Disaster Recovery Techniques Clustering protects against localized failure of a server machine and possibly also against network or software failures Clustering generally does not protect against media failure of the database or storage pools nor against a massive disaster that disrupts the entire data center Disaster recovery provides protection against media failure or massive disasters Clustering Techniques with automatic fail-over AIX PowerHA / HACMP Microsoft Cluster Services (MSCS) Veritas Cluster Service Tivoli System Automation for Multiplatforms Remote recovery via physical tape copies Spectrum Protect Disaster Recovery Manager (DRM) Electronic Vaulting Spectrum Protect Node Replication 2

4 Seven Tiers of Disaster Recovery Tier 7 - Site Mirroring with automated recovery Tier 6 - Storage Mirroring (with or without automation) C O S T Tier 5 - Software Mirroring (transaction integrity, DBMS, Mail, VMware, SONAS, etc.) Tier 4 - Point in Time disk copy, (FlashCopy Manager + TSM) Tier 3 - Electronic Vaulting (TSM) 15 Min. 1 4 Hr. 4 8 Hr Hr Hr. LENGTH OF RECOVERY PTAM=Pickup Truck Access Method Tier 2 - Hot Site, Restore from tape (TSM+PTAM) Tier 1 Restore from tape (TSM+PTAM) 24 Hr. Days Tier 0: No offsite data. 3

5 Overview of Clustering Techniques Characteristics of a clustered environment: Disks are shared between physical hosts, either exclusively or concurrently Disks appear as local disks to the host and not as network resources Mount points are identical on each physical host Resources such as disks, IP addresses, and configuration files can be defined as cluster shared resources and control of these resources can be automatically transferred from one host to another in the cluster Hosts in the cluster are able to detect when other hosts in the cluster have failed and can automatically take control of any shared resources that belonged to that failed host Notes: Spectrum Protect client cluster support provides backup protection in addition to the high availability protection that the cluster provides The client option, CLUSTERNODE, is no longer supported for Unix/Linux environments The Spectrum Protect server can also be protected with cluster techniques: MSCS, PowerHA and Tivoli System Automation for Multiplatforms 4

6 Spectrum Protect Client Cluster Failover Operation 5

7 Configuring for High Availability with Unix/Linux Local Scheduler Shared Scheduler Local Scheduler Node1 Local Local Shared Shared Resource Node Name(s) Shared password file Include-Exclude List dsm.opt file with Domain statement Client error log (named in dsm.sys) Client schedule log (named in dsm.sys) Start/Stop Scripts Shared Scheduler Local Scheduler Shared Scheduler Node2 Node3 Local Local Node Name(s) Local password file Include-Exclude List dsm.opt file with Domain statement dsm.sys file with stanzas for local resources and stanzas for each shared resources identical on each node 6

8 Configuring for High Availability with Windows Local Scheduler Local Resource Group Scheduler Local Scheduler Resource Group Scheduler Node1 Node2 Local Resource Group(s) Resource Group Node Name(s) Shared password file Include-Exclude List dsm.opt file with Domain statement Use clusternode yes Client error log (named in dsm.opt) Client schedule log (named in dsm.opt) Local Scheduler Resource Group Scheduler Node3 Local Local Node Name(s) Include-Exclude List dsm.opt file with Domain statement 7

9 Clustering Support for the Spectrum Protect Server Shared Resources: TSM database, log, disk storage pool and TSM IP address 8

10 Clustering Support for the Spectrum Protect Server Normal Operation: Primary node controls all the shared resources Primary Node Fails: Tivoli System Automation attempts to restart the TSM server Primary Node Fails and restore attempt fails: Tivoli System Automation fails-over to the secondary node 9

11 Overview of Disaster Recovery Techniques Disaster Recovery involves moving the operation of a particular server from one location to another usually as the result of failure at the primary location Typically this involves the movement to a geographically distant location (across town or across the country) sometimes called Big D disaster recovery The key requirement in this type of recovery is the movement of data from the primary site to the recovery site There are many technique including physical cartridge movement and many types of electronic vaulting Notes: Developing and documenting a recovery plan is an essential element of any successful disaster recovery plan Testing the recovery plan is also very important 10

12 Tier 2 - Offsite Vaulting with a Hot-site Datacenter A Offsite Vault Datacenter B (hotsite) Daily Backups At Recovery Time 11

13 Offsite Vaulting of Tape Cartridges Requires a disaster recovery plan and careful management of offsite volumes. Use Disaster Recovery Manager (DRM) to automate the Spectrum Protect server recovery process and to manage offsite volumes. Strategy must include vaulting of Spectrum Protect database backup, volume history information, device configuration information, disaster recovery plan file and copy pools for storage at an offsite location. Consider using tape encryption hardware Ship the Spectrum Protect database backup separately 12

14 DRM Overview What is Spectrum Protect s Disaster Recovery Manager? Manage the tape cartridge life-cycle (MOVE DRMEDIA) Create an offsite disaster recovery plan (PREPARE) Record client machine recovery information (DEFINE MACHINE) Disaster Recovery Planning Develop a documented plan Be sure all operational procedures, especially tape movement, are documented, tested and working Automation is important; use available tools to reduce human error as much as possible: O/S scripts, Spectrum Protect scripts and macros, Spectrum Protect DRM, scheduling tools, electronic transfer when possible Test the recovery plan often 13

15 Understanding the DRM State Values 14

16 The Daily Process Steps Spectrum Protect - backup storage pool BACKUP STGPOOL Spectrum Protect copy to active data pool COPY ACTIVEDATA Spectrum Protect - backup database BACKUP DB Spectrum Protect - identify off-site cartridges QUERY DRMEDIA WHERESTATE=MOUNTABLE Library - eject cartridges MOVE DRMEDIA TOSTATE=VAULT Spectrum Protect create the disaster recovery planfile -- PREPARE Operator - remove cartridges and turn over to courier -- manual Courier - carry to vault -- manual Spectrum Protect - identify empty cartridges in the vault QUERY DRMEDIA WHERESTATE=VAULTRETRIEVE Courier - carry empty cartridges back to on-site manual Operator - insert empty (scratch) cartridges in library CHECKIN LIBVOL 15

17 The PREPARE Output: The Planfile Stanzas Informational Stanzas Custom Instruction Stanzas Recovery Scripts Spectrum Protect Macros Spectrum Protect Configuration Files SERVER.REQUIREMENTS RECOVERY.INSTRUCTIONS.GENERAL RECOVERY.INSTRUCTIONS.OFFSITE RECOVERY.INSTRUCTIONS.INSTALL RECOVERY.INSTRUCTIONS.DATABASE RECOVERY.INSTRUCTIONS.STGPOOL RECOVERY.VOLUMES.REQUIRED RECOVERY.DEVICES.REQUIRED RECOVERY.SCRIPT.DISASTER.RECOVERY.MODE script RECOVERY.SCRIPT.NORMAL.MODE script DB.STORAGEPATHS LICENSE.REGISTRATION macro ACTIVEDATASTGPOOL.VOLUMES.AVAILABLE macro ACTIVEDATASTGPOOL.VOLUMES.DESTROYED macro COPYSTGPOOL.VOLUMES.AVAILABLE macro COPYSTGPOOL.VOLUMES.DESTROYED macro PRIMARY.VOLUMES.DESTROYED macro PRIMARY.VOLUMES.REPLACEMENT macro STGPOOLS.RESTORE macro VOLUME.HISTORY.FILE DEVICE.CONFIGURATION.FILE DSMSERV.OPT.FILE LICENSE.INFORMATION 16

18 Tier 4 - Electronic Vaulting to Hot-site with Offsite Vaulting as a Failsafe Datacenter A High Bandwidth Connections Datacenter B (hot site) Daily Backup Offsite Vault At Recovery Time 17

19 Electronic Vaulting of Spectrum Protect Recovery Data Spectrum Protect Node Replication Server to Server replication of selected nodes addresses some of the issues with virtual volumes Vault Spectrum Protect database backup, volume history information, device configuration information and disaster recovery plan file to an offsite location or to the hot-site. For Spectrum Protect database backup use virtual volumes or ftp the sequential file volumes ftp the configuration files Copy Storage Pool vaulting to an offsite location or to the hot-site Use virtual volumes (not recommended) or use an extended SAN Use a hardware mirroring technique for Spectrum Protect database backups and storage pool volumes (use with care) Use DB2 HADR to replicate a Spectrum Protect V6 database to a remote location Export / Import is another option Use DRM to automate the Spectrum Protect server recovery process Bandwidth is an important concern for any electronic vaulting configuration 18

20 Electronic Vaulting with Spectrum Protect Node Replication Primary Site Recovery Site Storage Pools Storage Pools IP TSM Database Protect StgPool Replicate Node TSM Database StgPool Data TSM Server Meta-Data TSM Server 19

21 Spectrum Protect Node Replication Overview Incremental replication of data belonging to backup/archive client nodes Data is replicated from one Spectrum Protect server to another server Source and target servers can have dissimilar retention policies client node data that is deleted from the source replication server is not necessarily deleted from the target replication server During a replication process, only the data that is not on the target replication server is copied If a disaster occurs, client nodes can recover their data directly from the target server clients can automatically fail-over to the target server for restore/retrieve client nodes can be converted for backup operations on the target server 20

22 Node Replication for Container Pools Container Storage Pools are the deduplication enable storage pools In-line deduplication and in-line server-side compression No Device Class or Volumes to manage Container pools introduce a new replication option: PROTECT STGPOOL Replicates only the deduplication extents (chunks) to the target server Object metadata is replicated via REPLICATE NODE PROTECT STGPOOL can be run multiple times per day Improves recover point Shrinks the window needed for replication REPLICATE NODE runs fast when only replicating metadata 21

23 Electronic Vaulting with Spectrum Protect Node Replication Considerations: No distance limitations -- uses TCP/IP No Spectrum Protect synchronization issues No need for specialized hardware or software all support is built into Spectrum Protect Data transfer can be deduplicated for improve utilization of WAN link No need for space reclamation across the WAN link Must have a second Spectrum Protect server at the recovery site to accept the replicated data Very fast fail-over to disaster site fail-over to the target site is automated for restore purposes 22

24 Electronic Vaulting with a Remote Library 23

25 Electronic Vaulting with a Remote Library Considerations: Most closely matches the Tier 1 method of disaster recovery may be able to reuse many processes Does not require a Spectrum Protect server at the recovery site Requires specialized SAN extension hardware Has distance limitations usually metro distances work well check with SAN vendor Works best with physical tape or VTL device Data transfer is not deduplicated Space reclamation may require large amounts of data transfer 24

26 Electronic Vaulting with Disk Mirroring 25

27 Spectrum Protect and External Disk Mirroring There are certain requirements which must be met in order to ensure that Spectrum Protect will function properly when data replication techniques are used for the Spectrum Protect Database, Recovery Log and disk Storage Pools. Write order must be strictly maintained between the source and target systems The Spectrum Protect database, recovery log and storage pools must be part of a consistency group and write order maintained within the group Writes must be non-volatile writes within a transaction group must all be either permanently resident on the disk or non-resident The hardware must report a successful write and must do so if and only if the data is permanently resident on the device These requirements apply to both hardware and software mirroring techniques These requirements apply to both asynchronous and synchronous mirroring techniques 26

28 Spectrum Protect and External Disk Mirroring Many disk subsystems use write caching to improve performance and provide replication functions. If these devices use battery backups and can ensure that the writes are destaged to disk before complete loss of power then they are acceptable In a remote replication configuration, you should never start the Spectrum Protect server at the target site unless the source site has failed or has been shut down for some reason. If you lose synchronization of the two sites, you should not attempt to start the target server and you should immediately resynchronize If you start the target server for whatever reason, you must have a plan to return to the primary system. Many times the disk subsystem will provide a mechanism to fall back to the primary system after it has been completely synchronized You should be sure your disk vendor understands these issues and has certified it s hardware to satisfy these requirements. 27

29 Electronic Vaulting with Disk Mirroring (Alternate) 28

30 Electronic Vaulting with Disk Mirroring Considerations: Can provide a very favorable recovery point when the Spectrum Protect database is mirrored Does not require a storage pool backup Data can be mirrored in a deduplicated state fewer bytes need to be transferred (best with client-side dedup) Fast recovery of the Spectrum Protect server at the disaster site with no special pre-recovery procedures Data is mirrored byte-by-byte difficult to synchronize with Spectrum Protect Requires careful use of consistency groups when Spectrum Protect database is mirrored May be distance limitations check with disk vendor 29

31 Electronic Vaulting with VTL Replication 30

32 Electronic Vaulting with VTL Replication Considerations: No distance limitations uses TCP/IP Does not require a storage pool backup Data can be replicated in a deduplicated state fewer bytes need to be transferred Replication is done at the virtual cartridge level easier to synchronize with Spectrum Protect than disk replication Does not require use of consistency groups 31

33 Electronic Vaulting with DB2 HADR and VTL Replication 32

34 Electronic Vaulting with DB2 HADR and VTL Mirroring Considerations: Does not require a Spectrum Protect database backup to replicate the Spectrum Protect database Can provide a very favorable recovery point Requires a running (DB2) standby system at the recovery site DB2 High Availability-Disaster Recovery (HADR) is a log-shipping methodology to replicate a DB2 database from one DB2 system to another 33

35 Spectrum Protect Node Replication to a Cloud Target Primary Site Cloud Recovery Site Storage Pools Storage Pools IP TSM Database TSM Database Protect StgPool Replicate Node TSM Server Meta-Data StgPool Data TSM Server 34

36 Electronic Vaulting to a Cleversafe Device (Hybrid Solution) Primary Site Cloud BACKUP DB Cleversafe Replication Client Backup IP TSM Server Primary Storage Pool 35

37 Electronic Vaulting to a Cloud Storage Pool Primary Site Cloud BACKUP DB Cloud Replication Primary Storage Pool Client Backup IP TSM Server 36

38 Electronic Vaulting to a Cloud Based Provider Primary Site Cloud TSM Database BACKUP DB Storage Pools Cloud Gateway Device BACKUP STG IP TSM Server 37

39 Electronic Vaulting to a Cloud Based Provider Considerations: Requires adequate bandwidth at backup Requires adequate bandwidth at recovery time Must rebuild the Spectrum Protect server from scratch at the DR site (similar to a Tier 2 recovery) except when using the Node Replication solution Most closely matches the Tier 1 method of disaster recovery may be able to reuse many processes Does not require a stand-by Spectrum Protect server at the recovery site except for the Node Replication solution 38

40 Comparing Electronic Vaulting Options Node Replication should be considered as the first choice in most cases It is a totally Spectrum Protect-contained solution for replication of both storage pool data and Spectrum Protect database (meta-data) Requires no extra hardware or non-spectrum Protect software set-up Spectrum Protect ensures the replicated storage pool data is always synchronized with the replicated Spectrum Protect database data (meta-data) However, you must plan for appropriate Spectrum Protect server hardware resources to support the replication (and deduplication) workload If Spectrum Protect server hardware resources are limited then consider using a replicating Cleversafe or VTL combined with Spectrum Protect database HADR replication or some other method of getting the database backup offsite Be prepared to deal with synchronization issues: the Database backup/hadr replication will usually complete much faster than the Cleversafe/VTL replication of the storage pool data All other methods require greater amounts of data transfer or introduce more difficult synchronization issues 39

41 Summary Understand how Disaster Recovery fits into the overall Business Continuance strategy Use Spectrum Protect DRM to manage your off-site tape cartridges and to automatically build a disaster recovery plan for your Spectrum Protect server Choose an electronic vaulting methodology that satisfies your recovery needs and is cost effective No disaster recovery plan is complete until it has been tested! 40

42 Please Note: IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice and at IBM s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 41

43 Notices and Disclaimers Copyright 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply. Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law 42

44 Notices and Disclaimers Con t. Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com, Aspera, Bluemix, Blueworks Live, CICS, Clearcase, Cognos, DOORS, Emptoris, Enterprise Document Management System, FASP, FileNet, Global Business Services, Global Technology Services, IBM ExperienceOne, IBM SmartCloud, IBM Social Business, Information on Demand, ILOG, Maximo, MQIntegrator, MQSeries, Netcool, OMEGAMON, OpenPower, PureAnalytics, PureApplication, purecluster, PureCoverage, PureData, PureExperience, PureFlex, purequery, purescale, PureSystems, QRadar, Rational, Rhapsody, Smarter Commerce, SoDA, SPSS, Sterling Commerce, StoredIQ, Tealeaf, Tivoli, Trusteer, Unica, urban{code}, Watson, WebSphere, Worklight, X-Force and System z Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: 43

45 Thank You 2016 IBM Corporation