Accelerate with IBM Storage: Copy Services Manager / Safeguarded Copy Update Matt Houzenga, Lisa Martinez, Randy Blea

Transcription

1 Accelerate with IBM Storage: Copy Services Manager / Safeguarded Copy Update Matt Houzenga, Lisa Martinez, Randy Blea

2 Accelerate with IBM Storage Webinars The Free IBM Storage Technical Webinar Series Continues in Washington Systems Center Storage experts cover a variety of technical topics. Audience: Clients who have or are considering acquiring IBM Storage solutions. Business Partners and IBMers are also welcome. To automatically receive announcements of upcoming Accelerate with IBM Storage webinars, Clients, Business Partners and IBMers are welcome to send an request to acceleratejoin@hursley.ibm.com. Located in the Accelerate with IBM Storage Blog: Also, check out the WSC YouTube Channel here: Webinars: January 9 DS8880 Easy Tier January 17 Start 2018 Fast! What's New for Spectrum Scale V5 and ESS February 8 - VersaStack - Solutions For Fast Deployments February 16 - TS7700 R4.1 Phase 2 GUI with Live Demo February 22 - DS8880 Transparent Cloud Tiering Live Demo March 7 - Spectrum Storage Management, Control, Insights, Foundation March 15 - IBM FlashSystem A9000/R and SVC Configuration Best Practices March 27 - IBM FlashSystem A9000/R Technical Update April 12 - Introducing Spectrum NAS - The Newest Member in the Spectrum Storage Family April 26 - TS7700 Grid Configuration Changes -- Joins, Merges and Removals May 8 - DS8880 Technical Update June 7 Economic Value of Response Time June 21 Deep Dive into Spectrum Scale AFM June 28 - Open System Tape, What s New July 19 - Copy Services Manager Version Technical Update August 2 - Back to Basics - TS7700 Concepts and Operations August 7 - IBM/Broadcom: Why NVMe-Over Fiber should be on your radar! August 30 - Transforming Traditional IT to a Multi-Cloud strategy leveraging IBM Storage September 13 - DS8880 Technical Update September 20 - IBM/Broadcom SAN Extension September 27 Introduction to Open Systems Tape 101 October 2 - Copy Services Manager / Safeguarded Copy Update Register Here: 4b6ad1d220 October 11 Storage Insights & Storage Insights PRO Use Cases at the WSC Register Here: c6352e850 October 18 TS7700 usability enhancements in release Register Here: e44

3 Agenda DS8880 Cyber Resiliency with Safeguarded Copy Matt Houzenga DS8880 Safeguarded Copy Backup Volume Creation Lisa Martinez Copy Services Manager Update Randy Blea 3

4 DS8880 Cyber Resiliency Safeguarded Copy 4

5 Safeguarded Copy - Background Organizations are becoming more concerned about the ability to recover from data corruption events These cover a wide range of possibilities from application corruption and user error through malicious destruction of data and ransomware attacks where data might be encrypted by an attacker Specific guidelines exist for financial services organizations in the United States documented in Appendix J of the FFIEC IT Examination Handbook Providing one / more / many versions of data at previous points in time is seen as a key technical capability to enable recovery use cases 5

6 Use Cases for Protection Copies Catastrophic Recover the entire environment back to the point in time of the copy as this is the only recovery option Forensic Start a copy of the production systems from the copy and use this to investigate the problem and determine what the recovery action is Surgical Extract data from the copy and logically restore back to the production environment Validation Regular analytics on the copy to provide early detection of a problem or reassurance that the copy is a good copy prior to further action 6

7 Challenges with using FlashCopy for Logical Corruption Protection Addressing requirements of multiple FlashCopy target devices Each FlashCopy target consumes a volume on the DS8000 restricting the number of copies that can be created FlashCopy target devices could also consume UCBs although planned LCP enhancements remove this requirement Limit of 12 copies also an issue although other considerations probably mean this limit is not reached in practice Space Requirements using Thin Provisioned devices The 21cylinder allocation unit used for Extent Space Efficient devices is optimised for host and FlashCopy performance and is not as efficient as a smaller (e.g. track level) allocation unit especially for sparse updates Performance impact of maintaining large number of FlashCopy relationships Updates to the production volume are copied to all FlashCopy target devices increasing the impact as more FlashCopies are created Securing the FlashCopy target volumes FlashCopy volumes are normal production volumes and can be mapped to a host so extra efforts need to be made to secure these 7

8 Objectives for Safeguarded Copies Enable the capturing of many (target 500 for initial delivery) Point in Time images of a production environment with optimized capacity usage and minimized performance impact. Enable a previous Point in Time to be either restored to the production volumes or made available on another set of volumes while the production environment continues to run. Secure the data for the Safeguarded Copies to prevent this data being compromised either accidentally or deliberately. 8

9 Safeguarded Copies and FlashCopy Safeguarded Copies do not replace FlashCopy and both technologies remain relevant in Logical Corruption Protection scenarios FlashCopy provides an instantly accessible copy of a production volume or dataset and for multiple FlashCopies each copy is independent from the others from a data perspective Safeguarded Copies could be used to take many frequent copies of a production environment (e.g. hourly copies maintained for a number of days) while FlashCopy continues to be used to take a small number of less frequent copies (e.g. weekly copies maintained for 1-2 weeks) 9

10 Volume Terminology The Production Volume is the source volume for a Safeguarded Copy relationship. Depending on the specific client topology this could be a Metro Mirror, Global Mirror or z/os Global Mirror primary or secondary volume or a Simplex volume. **Maximum Volume Size = 2 TB Production Volume B1 Backup Volume Recovery Volume The Backup Volume is a thin provisioned volume associated on a 1:1 basis with a Production Volume. It is not accessible to hosts and does not consume a host volume number. Updates written to the production volume cause tracks to be copied into the Backup Volume to allow Recovery to a number of specific Backup Copies which are created each time a Consistency Group is formed **Maximum Size = 16 TB The Recovery Volume is the target volume for a Safeguarded Copy Recovery which enables a prior Backup Copy to be accessed by a host attached to this volume. The Recovery Volume would typically be thin provisioned but this is not mandatory 10

11 Operational Action Summary **Restoring from the Backup Volume to the Production Volume is NOT supported in the initial release A Backup Copy can be Recovered from the Backup Volume to the Recovery Volume. This uses the current state of the Production Volume as the baseline for the Recovery with the Consistency Group used to provide a logical view of the volume at the previous point in time. Production Volume B1 Backup Volume Recovery Volume The oldest Consistency Group can be Expired when it is no longer required freeing up the extents in the Backup Volume Consistency Groups are Created on a regular basis creating a new Backup Copy on the Backup Volume and storing copies of updated tracks in the Consistency Group 11

12 Creating a Consistency Group / Backup Copy Create a Reservation on all volumes for the Consistency Group which sets up structures and prepares for the creation of a new Backup Copy. If this fails for any volume we can cancel the Reservation and report an error to the user without affecting the existing Backup Copies. Writes are not prevented during the reservation process Production Systems Write Cache Checkin the Reservation for all LSS involved in the consistency group with a single command for each LSS. Writes are prevented on each volume in the LSS as it is checked in to create a consistent point in time for the Backup Copy so this process is performance critical to minimise impact to production writes H1 As part of the reservation creation a commit scan is performed to harden all tracks currently in the write cache for the volume. While a reservation exists any new writes that require data to be stored in the Backup Volume will perform this copy synchronously SG1 Complete the Checkin on each LSS allowing writes to continue and new updates to be stored in the Undo Log for the new Backup Copy 12

13 Expiring a Consistency Group on the Backup Volume Production Volume Notes: The DS8880 will support an initial maximum number of CGs (500) In normal operation, software should expire the oldest CGs as needed DS8880 may internally expire CGs if resources become constrained B1 Backup Volume The oldest Consistency Group can be Expired when it is no longer required freeing up the extents in the Backup Volume used for the Metadata and Consistency Group Logs 13

14 Scenarios for Accessing a Backup Copy Accessing a Backup Copy on a Recovery Volume Production systems can continue to run while this is done Optional Background Copy to copy all required data out of the Backup Volume Identification of a good Backup Copy is expected to be done on the Recovery Volume Individual objects can be copied from the Recovery Volume to the Production Volume via software Full volume restore to production via Global Copy Recovery of a Backup Copy to the Production Volume is not available at GA 14

15 Accessing a Backup Copy on a Recovery Volume A FlashCopy type relation is created from the Production Volume to the Recovery Volume. Typically this would be a NoCopy to a thin provisioned Recovery Volume but could be with Copy if there was a requirement to keep the Backup Copy permanently A previous Backup Copy is selected and associated with the Recovery Volume. This Backup Copy is now available for access on the Recovery Volume for surgical recovery (by the user) or forensic analysis Production Systems H1 R1 Recovery Systems If the data is not stored in the Backup Volume, then it will be read from the Production Volume unless the data has been copied to the Recovery Volume already SG1 Optional background copy process to copy all required data to the Recovery Volume 15

16 Management Software Copy Services Manager (October) CSM GDPS support planned Used in combination with existing GDPS/CSM Sessions Safeguarded Copy of Local Production Safeguarded Copy of MM or GM Secondary 16

17 Attentions and Alerting In normal operation, the Backup Volume would be able to contain the max number of Consistency Groups (e.g. 500 CGs) Management software will be responsible for rolling off older CGs to keep the Backup volume at less than or equal to max CGs However, there could be conditions where we can approach other resource limits prior to reaching the max CG threshold There is a need to handle the case where the backup volume (or Extent Pool) fills in the middle of a CG. We cannot allow an in-progress CG to overlay any needed data in earlier CGs To manage this condition, the DS8K will internally roll off the oldest CGs to relieve the constraint(s) This is done to prevent I/O from being held up on the production volume 17

18 Attentions and Alerting Alerts: Extent Pool which contains the Backup Volume is approaching full or full This alert is already implemented in DS8K today Backup Volume approaching fully provisioned Alert at 85% provisioned (customer modifiable threshold) Alert at 100% provisioned (CGs will be rolled off via software or internally by DS8K) 18

19 Security The Backup Volume is not addressable or accessible by hosts The Backup Volume cannot be deleted unless all Backup Copies are removed by controlling software A Production Volume cannot be deleted unless the Backup Volume and all Backup Copies have been deleted Custom User roles for DSGUI and DSCLI Policy on retention for Backup Copies will be in the controlling software (GDPS or CSM) NOT in the DS

20 Safeguarded Copy Topology Example 1 Logical Isolation example Example illustration showing a sample topology of a Metro Mirror source set of volumes, possibly in a Hyperswap configuration, replicating to a second DS8880 The target DS8880 subsystem is configured to have Safeguarded Copy backup and recovery volumes The blue outlines surrounding the volumes are intended to represent DS8880 subsystems containing many copysets An even simpler logical isolation example could be drawn to have just a simplex set of volumes (think H1 in the illustration) with backup and recovery volumes in the same subsystem Metro Mirror could just as easily be Global Mirror in this topology 20

21 Safeguarded Copy Topology Example 2 Physical Isolation example Example illustration showing a sample topology of a Metro Mirror source set of volumes, possibly in a Hyperswap configuration, replicating to a second DS8880, and then cascading that replication to a third DS8880 using Global Copy technology The third DS8880 subsystem, potentially in a secure bunker location, is configured to have Safeguarded Copy backup and recovery volumes The blue outlines surrounding the volumes are intended to represent DS8880 subsystems containing many copysets Creation of a SGC consistency group would involve consistently pausing the Metro Mirror relationship, allowing Global Copy to catch up, then capturing the SGC consistency group all coordinated by CSM or GDPS management software Metro Mirror could just as easily be Global Mirror in this topology 21

22 Early engagement process Safeguarded Copy, even when released for General Availability, will be delivered to customers with the function disabled IBM has decided that in the earliest stages of implementation, we would like to assess each customer s plans for testing and deploying Safeguarded Copy At least for October and November, 2018, if you want to test Safeguarded Copy in your environment: You or your TA can open a PMR against the DS8880 that will be used, using the keywords Safeguarded Copy Enablement Interest and provide direct contact information for your technical specialist You will be invited to a meeting with DS8000 development to discuss the environment you would like to test, and if within already tested boundaries, will result in IBM Service logging into the target DS8880 and enabling the capability Any issues discovered during testing would flow through the normal service process IBM has developed several different approaches to sizing the physical space that will be needed to contain SGC consistency groups (the backup volumes) Contact your IBM Technical Advisor or your IBM/BP sales rep to engage IBM in this analysis 22

23 DS8880 Safeguarded Copy Backup Volume Creation 23

24 DSGUI Creating capacity for Safeguarded Copies 24

25 DSGUI 25

26 DSGUI Deleting Safeguarded capacity 26

27 DSGUI Migrating Safeguarded capacity to another pool 27

28 dscli dscli> manageckdvol -multiplier 1 -action mksafeguardedcap -extpool p Date/Time: September 24, :28:48 PM EDT IBM DSCLI Version: DS: IBM FCM01 CMUC00570I manageckdvol: Backup capacity for CKD volume 2101 is successfully created. dscli> showckdvol 2101 Date/Time: September 24, :31:10 PM EDT IBM DSCLI Version: DS: IBM FCM01 Name ckg_mod27 ID 2101 safeguardedcap (cyl) safeguardedloc P1 usedsafeguardedcap (cyl) 0 safeguarded no SGC Recovered no dscli> manageckdvol -action migsafeguardedcap -extpool p dscli> manageckdvol -action rmsafeguardedcap -extpool p dscli> managefbvol -multiplier 1 -action mksafeguardedcap -extpool p Safeguarded Copy dscli actions: mksafeguardedcap rmsafeguardedcap migsafeguardedcap migsafeguardedcapcancel migsafeguardedcapresume migsafeguardedcappause 28

29 Copy Services Manager Update 29

30 CSM Support for DS8000 Safeguarded Copies - Overview Safeguarded Copy is supported under a fully licensed version of CSM running on the DS8000 HMC, z/os or any of the supported Distributed servers At the time of release, CSM support for Safeguarded Copy requires an HMC connection. If only a z/os connection exists to the storage system, APAR OA56173 (not yet available) will need to be applied to the z/os system or an HMC connection will need to be added. Safeguarded Copy is a new two volume session type that supports a host volume (H1) which contains the data you wish to backup, and a corresponding recovery volume (R1), that can be used to recover any of the backups for forensic analysis 30

31 CSM Support for DS8000 Safeguarded Copies Creating the session From the Sessions Panel click on the Create Session button to bring up the create session wizard Select the Hardware type of DS8000 Select Safeguarded Copy At the end of the wizard click on Launch Add Copy Sets Wizard to add copy sets into the new session Copy sets can be added by selecting volumes or using a CSV file. Copy sets are made up of two volumes: H1 (host volume) and R1 (recovery volume) 31

32 CSM Support for DS8000 Safeguarded Copies Preparing to issue backups Use the DSCLI or DSGUI to define Backup Capacity for all H1 volumes in the session Set the Backup Options for the session Minimum Timeframe per backup (mins) This setting defines how frequent backups can be taken. Default is 30 minutes. Use the setting to prevent a malicious user from creating a string of backups that cause all non-corrupted backups to be rolled off Retention Period Since Last Recoverable Backup This setting defines how long backups should be retained in order to meet the customer s SLAs. It is HIGHLY recommended customers set this value. The time set is applied since the last recoverable backup for the session to help ensure there is always at least one recoverable backup Set the H1-R1 Options No Copy This setting defines whether the recovery relationship should use no copy when creating the relationship. Default is to use No Copy. 32

33 CSM Support for DS8000 Safeguarded Copies Creating a backup Backups can be created manually via the CSM GUI/CLI or through a CSM Scheduled Task using the Backup command on the Safeguarded session The Backup command creates a data consistent backup across all volumes in the session Each backup will have an initial preparation time of up to 5 minutes to close out and harden the previous backup The last backup will display as not recoverable until a new backup is taken to close out the previous backup A Safeguarded session will appear in the Protected state after the backup has been issued and all volumes have a backup 33

34 CSM Support for DS8000 Safeguarded Copies Creating a backup via a Scheduled Task CSM Scheduled Tasks can be used to schedule backups to run automatically at a given interval In CSM 6.2.3, Scheduled Tasks were modified so that the schedule will start at the time the task is enabled, allowing customers to stagger the run time for tasks To create a Scheduled Task either click on the Schedule link on the Safeguarded Copy session or go to the Scheduled Task panel and then click the Create Task button. You can name the task, set a schedule (30 minute intervals or run on a daily/weekly schedule), and then create a Command Action to issue the Backup command against the session After creating the Scheduled Task select the task from the task table and issue Enable Task from the Actions menu 34

35 CSM Support for DS8000 Safeguarded Copies Creating a backup via a Scheduled Task Important considerations when choosing a schedule for the Scheduled Task The schedule chosen should not run more frequently than the Minimum Timeframe per Backup value for the session The schedule chosen should not run less frequently than what the retention time is for the session The scheduled time for each backup and the retention period on the session should be directly influenced by the SLA required for the backups The maximum number of backups allowed by the hardware is 500 by default Ensure that you have defined enough backup capacity to meet your SLA requirements Defining a Scheduled Task does not prevent a manual backup from being taken on the session, but the manual backup will count towards the minimum timeframe between backups It is recommended that you do not take backups more frequently than every 30 minutes at this time 35

36 CSM Support for DS8000 Safeguarded Copies Backup Retention and Auto Roll CSM will automatically roll backups based on the retention period When CSM rolls a backup it will no longer be displayed in the list of backups In order to allow new backups to form, the hardware will automatically roll older backups on a volume by volume basis in order to free capacity for the new backup When the hardware rolls a backup, CSM will see that the backup has rolled and mark the backup as not recoverable A customer must manually expire older backups that are not recoverable because the hardware rolled the backup If backups are auto-rolling because of space, the customer should reevaluate the size of the backup capacity as well as the retention and schedule for the backup 36

37 CSM Support for DS8000 Safeguarded Copies Viewing volumes in a backup CSM allows you to see the number volumes in a backup and drill into to see the volumes that are in that backup To view the volumes in the backup click on the link in the backup table for the Backup Time Viewing the backup can also be used to determine which volumes may have auto rolled causing the backup to no longer be recoverable Message IWNR2811E is displayed on volumes that may have auto rolled on the hardware CSM will query the volumes for backups approximately once an hour or at the end of a backup, to determine if there were any backups that have been rolled From the CLI lsbackups will display all backups for the session and lsbackupvols displays the volumes in an individual backup 37

38 CSM Support for DS8000 Safeguarded Copies Manually expiring backups When volumes auto roll causing a backup to no longer be recoverable it may be necessary to manually expire the backup You may also want to auto expire backups to free up additional space Expiring a backup will expire the selected backup AND all backups older than the backup selected To expire a backup from Session Actions click Commands- >Expire Backup From the popup select the backup you wish to expire The popup will highlight the backup selected and all older backups to make it clear that they will all be expired NOTE: Backups cannot be manually nor auto expired if that backup or an older backup is currently in a recovery relationship 38

39 CSM Support for DS8000 Safeguarded Copies Recovering Backups The Recover Backup command allows the customer to recover one of the recoverable backups to the R1 (recovery volumes) From the Session Actions select Commands->Recover Backup and a popup will display all of the recoverable backups for the session Select the backup time to recover to and click Yes to continue After the R1 volumes are recovered the session will change to the Target Available state and the progress on the recovery relationship can be viewed on the Recovery Backup Info tab If No Copy was selected on the session options, the recovery relationship will be established such that a full background copy will not occur unless manually initiated To start the background copy to harden the R1 volume, from Session Actions select Commands->Initiate Background Copy If a background copy completes the recovery relationship will no longer exist on the hardware however CSM will remain in the Target Available state 39

40 CSM Support for DS8000 Safeguarded Copies Terminate H1R1 and Terminate Terminate H1R1 After recovering to the recovery relationships and completing forensic analysis, it may be necessary to free the space taken by the recovery relationship To free the space for the recovery relationship issue the Terminate H1R1 command Terminate H1R1 will only free space if the background copy has not yet completed for the relationship If there are still backups for all volumes in the session, the session will go back to the Protected state Terminate The Terminate command will remove any existing recovery relationships from the hardware and expire all backups for all volumes When the terminate is complete the session will go back into the Defined state 40

41 CSM Support for DS8000 Safeguarded Copies Adding and Removing Copy Sets As with all CSM sessions, copy sets can be added or removed at any time Volumes added to the session will not have backups until the next Backup command is issued on the session After adding copy sets existing backups will show a number of volumes less than the total number of copy sets If you attempt to recover a backup that does not have all the copy sets a warning will appear and the volumes that were not part of that backup will show as not recovered When removing copy sets from the session, all backups for those volumes will be removed from the hardware Care should be taken when removing copy sets as the volume may be required for consistency across the entire backup The keep on hardware option can be used during the remove to remove the copy set from the session but keep the backups on the hardware. The volume can then be added back into the session or another session and the backups will be assimilated during the next hourly query, or if Refresh States is issued 41

42 CSM Support for DS8000 Safeguarded Copies Safeguarded Copy at Remote Site Safeguarded Copy can be combined with remote copy replication such as a Metro Mirror or Global Mirror session in order to take backups at a remote site or bunker site Before creating a backup it is necessary to suspend the remote mirror consistently. For CSM MM and GM sessions this can be done using the Suspend command CSM Scheduled Tasks can be used to easily automate the Suspension of the remote mirror, the backup and the restart of the remote mirror Starting in CSM a Scheduled Task can be created with no schedule and then run manually Also starting in any Scheduled Task can be run from the CSM CLI manually or via a batch job using the runtask CLI command and the task ID 42

43 CSM Support for 4 Site Sessions Metro Mirror - Global Mirror w/ Site 4 Replication is a new session type in The new session type allows customers to manage the replication to 4 separate sites (2 local and 2 remote) The session will use Multi-target MM-GM and provide a cascaded GC replication to a 4 th site as with the existing solution Site 1 and Site 3 will provide primary host support with the ability to failover or HyperSwap to a local MM Solution is completely symmetrical providing same replication options whether you re at the local or remote location z/os HyperSwap will be available on the local Metro Mirror leg 43

44 Questions 44