ETERNUS DX Advanced Copy Functions (Remote Equivalent Copy) 0
Content Overview Modes Synchronous Split and Recovery Sub-modes Asynchronous Transmission Sub-modes in Detail Differences Between Modes Skip Initial Copy Combinations of Advanced Copy 1
Overview Usage Business Continuity: Disaster Recovery Concept Diagram Suspend and Resume Process Concurrent Suspend 2
Usage (1) Remote copy between two ETERNUS systems High Availability Short distances Data transfer can be achieved using FC or iscsi Separate, independent interconnection lines between the ETERNUS systems are necessary Site A Management Server Site B Management Server SAN Brocade Fiber Channel Switch Brocade SAN Fiber Channel Switch Operation Volume Operation Volume ETERNUS DX ETERNUS DX 3
Usage (2) Remote copy with ETERNUS systems Disaster Recovery Very long distances 10+ km data transfer can be achieved by FC-WAN converter or iscsi Separate, independent interconnection lines between the ETERNUS systems are necessary Main site Management Server Advanced Copy Function Backup site Management Server SAN Brocade Fiber Channel Switch WAN/LAN Brocade SAN Fiber Channel Switch Operation Volume Backup Volume ETERNUS DX ETERNUS DX 4
Business Continuity: Disaster Recovery Recovery Point Objective, RPO How much data has been lost? Recovery Time Objective, RTO How soon after the event is the application available again? Last known good image RPO Application Downtime RTO Application is available again Recover Yesterday 24 hrs Now Restore Time All modifications that are lost Analyze 5
Concept Diagram Background copy transfers sequentially all data of the Source Volume to the target site While also updating both the Source and Target with any new write data must be started and reach the equivalent state before the Destination Volume can be detached (suspend or cancel) The Destination Volume is again writeable after suspend or cancel Start Write Initial Copy complete Write Stop (Split) I/O Req. Source at site A Target at site B Background Copy Paired Mirroring After Split, both Volumes can be accessed independently I/O Req. 6
Suspend and Resume Cancelling terminates the session and a complete remirroring will be required when a new session is started All blocks of the Source need to be copied After resuming from Suspend state only the changes made during the suspended state are copied Changes to both the Source and Destination Volumes are recorded in a bitmap during the suspended state S D Equivalent state Cancel S S D D Write to source or destination volume Start Resume Re-mirroring Copy All data Equivalent state Suspend Changes to Source and Destination volumes are recorded when suspended Using the bitmap information, copy changed data only! or overwrite data only! 7
Process is started Replication proceeds in the background Block level copy from the local Source to the remote Destination Copy rate depends on ETERNUS priority setting (high, medium or low) Parameter to be assigned via the ETERNUS GUI or CLI The Source Volume remains available The Destination Volume is inaccessible Throughput is degraded during initial copy Replication is complete The mirrors are kept in Equivalent state Detach request (Cancel or Suspend) Must be coordinated with the application to obtain a logically "consistent" copy For example, flush system buffer, shutdown application, etc. Both mirror Volumes are available for write access while suspended or after cancelling 8
Concurrent Suspend Concurrent Suspend function This allows consistent copies of multi-volume data objects, such as databases All replication pairs that are involved have to be grouped Possible after session has reached Equivalent state Multiple sessions can be suspended at the same point in time 9
Modes Synchronous Modes Asynchronous Modes 10
Synchronous Modes Write confirmation is responded to the host after data transfer to the target ETERNUS DX is completed This mode guarantees that the copy process is fully completed on both sides of the before an acknowledgement of a data write request is returned to the server Application Write Acknowledged Source Volume Site A Copy OK Target Volume Site B 11
Asynchronous Modes Write confirmation is responded to the host after data write to local cache of the ETERNUS DX system This mode will not guarantee that the copy process is fully completed on both sides of the before any update or write process response is returned to the server Application Application Write Acknowledged Write Acknowledged Source Volume Site A Queue Target Volume Site B 12
Synchronous Split and Recovery Sub-modes Transfer Mode "Automatic Split" Transfer Mode "Manual Split" Line error options/recovery with "Automatic Recovery" or "Manual Recovery" 13
Transfer Mode "Automatic Split" During path failure Automatic disconnection of the session might occur For example if the transmission fails during the process Write access to Source Volume can continue even when the session is in Hardware Suspend Changes are tracked by the bitmap mechanism Server continues to write on the copy Source only The source and target data are no longer in Equivalent state Source and Destination LUNs need to be synchronized after the connection problem has been solved Automatic Split mode Write Write Write Write Write Source Target Copy Copy Copy Failure Not Copied Time 14
Transfer Mode "Manual Split During path failure The session stops completely and all writes are suspended Source and Target Volumes remain in Equivalent state If the path failure is a result of a site disaster this prevents data losses but on the other hand means unscheduled downtime for the applications To regain access to the Source LUN of the ETERNUS, "Suspend" needs to be executed Manual Split mode Write Write Write Write Write Source Target Copy Copy Copy Failure Not Copied Time 15
Line Error Options/Recovery Automatic Recovery session is automatically resumed after the data path functionality has been recovered Manual Recovery session must be manually "Resumed" after the data path is restored Before issuing "Resume", this option allows clone creation (OPC, EC) of the mirror Volumes Ensures that the last known good data images are preserved before writing on the Volumes is re-enabled As both mode options, Manual Split and Manual Recovery, require manual or scripted intervention, they are recommended for experienced users only 16
Asynchronous Transmission Sub-modes Stack Mode Example of Using Stack Mode Asynchronous Mode - Consistency Buffer Transfer Parameters Consistency Mode Buffer Configuration Consistency Mode Usage 17
Asynchronous - Stack Mode (1) This mode uses a bitmap Only updated block locations are recorded in the bitmap before sending acknowledgement to the server Data itself is not cached Transfer of the changed blocks is performed by an independent transfer engine that scans the bitmap Write order is NOT maintained in the destination Useful when limited bandwidth is available but a large amount of data needs to be transferred Write I/O Write Write Ack. 1. Write data to disk 2. Record updated block 3. Write acknowledgement Bit map 1 1... 18
Asynchronous - Stack Mode (2) When multiple writes have occurred in the same block address, only the latest data at the time of the next patrol check is sent to the copy destination, helping to minimize the amount of transferred data This mode can lead to a loss of data if a malfunction occurs in the source system This mode is mainly used for backing up data to a remote system Transfer Engine 1. Write 2. Write... Transfer Engine Process 4. Transfer engine search and find an updated block 5. Transfer updated block 1 1 19
Example of Using Stack Mode (1) Example: Multi Copy Two (Stack) sessions are set up and alternately operate as Suspend / Active (2) Example: Cascade Copy Two Local Copy sessions are set up and operate at the (Stack) copy destination Creating a Snapshot to a Remote site (remote backup) When a data failure occurs at source site Restore is performed using the session volume that was in the suspend state at that time Suspending a session not possible in Stack mode, only in Async. mode Stack mode -> Force suspend Switch to through mode Resume -> Suspend 1 2 20
Asynchronous Mode - Consistency (1) This mode works always with Buffer Cache Receive buffer and send buffer as a pair The write order across multiple sessions can be guaranteed at the copy destination i.e. write consistency over multiple LUNs or regions Updates to multiple LUNs on the ETERNUS are accumulated in the buffer and then grouped in buffer units for transfer This reduces the communication sequences with the remote site and enables more efficient use of line capabilities (increased performance) 21
Asynchronous Mode - Consistency (2) This mode uses ETERNUS buffers to store actual data Not only bitmap pointers buffers are configured as either Receive or Transmit At check point, all access to Source Volumes are stopped Less than 100 micro seconds Overhead is less than 0.1% Check point interval can be set between 1 and 120 seconds This has an affect to the RPO Buffer data is applied to the remote site only after the entire buffer is received Write data copies to buffer Source Destination Send Buffer Receive Buffer Check Point Buffer switch, then transfer Transfer complete, then copy to dest. 22
Asynchronous Mode - Consistency (3) In this mode a large amount of data accumulates in the buffer All the accumulated data is transferred to the destination as a block at user definable intervals There are three buffer parameters for the asynchronous consistency mode that are set with the ETERNUS Web GUI Item Forwarding Interval (sec) Watch (Monitor) Time (minutes) Halt Wait Timer (seconds) Specify the data forwarding interval Description Specify the monitoring time before transition changes to Halt status for a session when a buffer shortage occurs. If the buffer remains in high-load state (overload) for the specified time, the status of session will be automatically changed to Halt. When the Watch Time is set to 0, the buffer is not monitored. Specify the maximal non-response time. The ETERNUS monitors the host I/O waiting time. During this time the host I/O is stopped from ETERNUS point of view. This gives priority to perform buffer transfers. If the waiting time exceeds the specified time, session status automatically changes to Halt status. 23
Buffer Transfer Parameters Buffers can be set to either Send or Receive with following setup Send Buffer at source Receive Buffer at destination Up to eight 1) buffers (in total) can be set per system In the asynchronous Consistency mode, there may be Eight connected ETERNUS DX pairs Each pair has one connection channel with a send and receive buffer This only allows copy in one direction But the direction can be reversed from the command line by changing the buffer role (swsrpbuffset) For model specific configuration details please refer to the Remote Advanced Copy Function Guide Many ETERNUS DX models provide also Disk Buffer functionality 1) Four with DX90 and DX90 S2, please refer to product specification for up-to-date information. 24
Consistency Mode Buffer Configuration Multiple pairs of buffers have been configured to enable copying between multiple systems (1) Enables copying between multiple locations (1) Use single buffer pair for unidirectional data transfer only (2) Enables copying in both directions between systems (2) Use two buffer pairs for bidirectional data transfer For forward and reverse data transfer NOTE: It is not possible to configure 2 x send buffers addressing the same destination ETERNUS system SEND buffer 2 Pair 1 Receive buffer 1 SEND buffer Pair 1 Receive buffer Site B Receive buffer Pair 2 SEND buffer SEND buffer Receive buffer Site C Pair 2 Site A Site B Site A 25
Consistency Mode Usage (1) Consistency pairs can be established between two systems and for each transfer direction Multiple applications can share one consistency pair Each consistency pairs can be individually controlled Backups can be made separately for each system App-Sys A App-Sys B App2-Sys C App1-Sys C Site A Site B Restore can be used for recovery, independently of the consistency pair buffer send buffer receive Consistency Pair 1 Consistency Pair 2 buffer receive buffer send App-Sys A App-Sys B App2-Sys C App1-Sys C 26
Consistency Mode Usage (2) Consistency mode preserves write order During the copy process the updated blocks are stored in different sets of buffers located in the CM cache All buffer sets are transmitted in one transaction Either all buffer sets are copied or nothing is copied The receiving ETERNUS system copies the data to the disk only after having received the complete buffer contents The transmitting ETERNUS system will not discard buffer contents until the receiving ETERNUS system acknowledges the reception 27
in Detail Asynchronous Mode Suspend and Resume Asynchronous Mode Line Failure Asynchronous Through Mode 28
Asynchronous Mode - Suspend/Resume (1) During Suspended state, modified blocks are marked at each side in bitmap tables Marked blocks at target side are overwritten (nullified) after resume Suspend Resume Write Write Write Source Volume Site A A A* Bitmap Table A* A** Maintain Copy Keep track of modifications 1 Merge Bitmap Table Re-Sync Copy Maintain Copy Bitmap Table 1 Target Volume Site B A C Overwrite A* A** Write 29
Asynchronous Mode - Suspend/Resume (2) This picture illustrates the bitmap merge after Resume is invoked Source Suspend Destination Source Resume Destination Cache Cache Cache Cache Bitmap Bitmap Bitmap Bitmap Bitmap merge Source physical disk Destination physical disk Source physical disk Destination physical disk 30
Asynchronous Mode - Line Failure During path failure, the modified blocks are marked in bitmap table After path becomes available again, the marked blocks are transferred without preserving the write order First Write after path failure returns sense information Write Write Write Source Volume Site A A A* Keep track of modifications A* A** Bitmap Table Maintain Copy Path Failure Path Available Re-Sync Copy Maintain Copy Target Volume Site B A A* A** Halt State Re-synchronization 31
Asynchronous Through Mode This mode is a special asynchronous mode It is used to flush as-yet-untransferred data when a Stack or Consistency mode session is in Stop state (due to Cancel command) Suspend state This mode should be used when an interconnect path failure occurs after a Cancel or Suspend is initiated After the path is restored the buffered data must be flushed using the Asynchronous Through Mode 32
Through Mode for Sequential Writing This diagram shows the impact to the server I/O when running "Asynchronous Through Mode" The performance of this mode equals that of the Synchronous Mode For these reasons, you should not use the Asynchronous Through Mode under normal circumstances Server 1. Update data ETERNUS DX Copy Source ETERNUS DX Copy Destination 1. Acknowledge 2. Update data ETERNUS 1. Acknowledge 2. Acknowledge Source ETERNUS DX has to wait for the acknowledgement of the previous data transfer. 33
Differences Between Modes Network Bandwidth Differences Between Modes Modes of Operation - Summary 34
Network Bandwidth Synchronous and Consistency modes: Match to peak times Update quantity Required bandwidth Stack mode: Match to average load Update quantity Required bandwidth 35
Differences Between Modes Transfer mode comparison 36
Modes of Operation - Summary Synchronous Application waits until update reflected on local and remote volume Order guaranteed No data loss High impact on performance Automatic or manual split options Asynchronous (Stack mode) Application waits until update reflected locally and placed in move queue Independent process sends updates to destination No update order guarantee Data loss possible Low impact on performance level Asynchronous (Consistency mode) Application waits until update placed into local group send queue After timer interval (1, 2 or 4 seconds) transmitted to remote receive queue Update order guaranteed Data loss possible Low impact on performance level 37
Skip Initial Copy Start+Suspend / Resume+Remain Initial Copy Skip Mode 38
Start+Suspend / Resume+Remain There is a special ACM command option available to support this feature for only Start+Suspend Resume+Remain Write Start + Suspend Write Resume + Remain Time line Source Volume Site A Operation can be continued Tape Backup Suspend Suspend EC Maintain Copy Target Volume Site B Tape Restore 39
Initial Copy Skip Mode When the Start+Suspend is issued the Copy Source data has to be consistent from the application point of view The Copy Destination data has to be 100% identical with the Source data when the Resume+Remain is issued Note, ETERNUS does not verify the data consistency between the two Volumes After the synchronization of the Destination is completed, ETERNUS shows always equivalent state for the Volumes Regardless if the data actually is consistent 40
Combinations of Advanced Copy Cascade Copy Multi Copy 41
Cascade Copy Concatenates several copy sessions An existing copy Destination Volume becomes the copy Source for a new copy session ETERNUS DX-1 ETERNUS DX-2 ETERNUS DX-3 Source-1 OPC, QuickOPC EC Destin-1 Source-2 - Stack Destin-2 Source-1 - Synchronous - Stack - Consistency Destin-1 Source-2 OPC, QuickOPC SnapOPC EC Destin-2 ETERNUS DX-1 ETERNUS DX-2 ETERNUS DX-3 Source-1 - Synchronous - Stack - Consistency Destin-1 Source-2 - Consistency - Stack Destin-2 42
Multi Copy Copies the same copy source area to multiple copy destinations The destination can not be the source for another multi copy session ETERNUS DX-2 ETERNUS DX-1 Destin-1 Destin-1 Source-1 Destin-2 43