DISASTER RECOVERY TECHNICAL OVERVIEW BY GREEN CLOUD OPERATIONS

Transcription

1 DISASTER RECOVERY TECHNICAL OVERVIEW BY GREEN CLOUD OPERATIONS

2 DISASTER RECOVERY - OVERVIEW Green Cloud Technologies, LLC operations@gogreencloud.com 2

3 DISASTER RECOVERY - OVERVIEW How Green Cloud Disaster Recovery Works Backups are made using StorageCraft ShadowProtect and copied to a NAS on the customer site. Incremental updates are sent to Green Cloud s datacenter using StorageCraft ImageManager. We will test a restore of each server to identify potential delays with restoration in the course of a disaster. If a disaster is declared, we will copy the most recent incremental backup to our live VMware environment and spin the servers up as VMs. This is a managed service. Green Cloud will handle any software updates, configuration or service adjustments. Green Cloud Technologies, LLC operations@gogreencloud.com 3

4 DISASTER RECOVERY - OVERVIEW Notes on the Disaster Recovery Service Backups are block-level backups of each volume, which means that the software will not recognize files or directories. Instead it will recognize changed sectors on the drive. This is important because it means we can not exclude specific directories on a drive. We can only choose to backup a drive (for example, we can choose to backup the E: drive, but we cannot choose to backup all of the E: drive except for their E:\DATA_WE_DONT_WANT_SENT_TO_GREEN_CLOUD directory). File restoration is technically possible through the software, but we recommend that our dealers interested in that service become StorageCraft certified, as we offer no specific interface for doing that by default. Green Cloud Technologies, LLC operations@gogreencloud.com 4

5 DISASTER RECOVERY - OVERVIEW Green Cloud Responsibilities vs. Dealer Responsibilities Green Cloud will manage the service. We are responsible for making sure the backups are valid and alerting the dealer/customer if there s anything that we need. We will send weekly status notifications that will offer visibility about the health of the service as well. The dealer is responsible for making sure nothing on the server gets installed that will conflict with our software, notifying us about any public IP changes, new drives that need to be backed up, and anything else that might lead to us not having a good backup of the whole server. They will also need to work with the customer to make sure the mock restore environment is reviewed properly and the Business Continuity Plan is filled out with the customer. Green Cloud Technologies, LLC operations@gogreencloud.com 5

6 DISASTER RECOVERY PROPOSAL MARKED SOLD 1. Proposal Marked Sold Internal documentation, account, and customer collateral created. 2. Site Survey 3. Provisioning 4. Install 5. Synchronization 6. Mock Restore 7. Business Continuity Plan Green Cloud Technologies, LLC

7 DISASTER RECOVERY SITE SURVEY 1. Proposal Marked Sold 2. Site Survey Introduction sent with site survey, dealer/partner completes site survey and makes sure indicated items on site survey get completed (such as creating the greencloud user account and removing other backup software). 3. Provisioning 4. Install 5. Synchronization 6. Mock Restore 7. Business Continuity Plan Green Cloud Technologies, LLC

8 DISASTER RECOVERY SITE SURVEY Gigabit switch and DHCP required. Allows for fast backups and ease of installation/configuration. We need to know the amount of bandwidth so we can properly manage the service during business hours so that the incremental uploads don t negatively affect the business. Port 25 is outbound , and we need a method for receiving alerts from the NAS and software. We *must* have the correct public IP here, or the NAS will not connect to our management server and we will not be able to upload incremental backups to the datacenter. A greencloud user must be created. Note: must be a domain administrator on domain controllers and a local administrator on non-dc s. Whitelisting s from us is important so our notifications don t get blocked. Green Cloud Technologies, LLC operations@gogreencloud.com 8

9 DISASTER RECOVERY SITE SURVEY Hostnames are very important they re how we manage each server, so they must be accurate. Hardware and OS details will ensure we get the virtual server set up properly in case of a disaster. This will let us know the amount of space to provision the NAS to handle. Important to get right. Lets us know what each server does, helps us figure out where to install Image Manager, which server to restore first (confirmed on MRS). Ideally, all servers are licensed using volume licensing. OEM licenses can be tricky and may require additional expense for the customer. Appendix? Since we are doing block level backups, any change made will be copied. Including defrags, which will generate very large updates. No other backup software can be installed, because it may conflict with our software. Green Cloud Technologies, LLC operations@gogreencloud.com 9

10 DISASTER RECOVERY PROVISIONING 1. Proposal Marked Sold 2. Site Survey 3. Provisioning Using the information from the site survey, Green Cloud will provision and ship the customer s equipment Synology NAS and encrypted USB Drive. 4. Install 5. Synchronization 6. Mock Restore 7. Business Continuity Plan Green Cloud Technologies, LLC operations@gogreencloud.com 10

11 DISASTER RECOVERY PROVISIONING Once we ve received the site survey we will: 1. Provision a NAS, using the information from the site survey so we right-size the drive space and set up our secure management connection. 2. Configure an encrypted USB Drive, using a new, randomly generated passcode. 3. Pack everything in one box, which will include the NAS, power adapter and power cable, a pre-labeled return box containing the encrypted USB drive with two-pronged Y-cable and network cable. 4. Ship to customer/dealer site. Send to customer/dealer with shipping information, installation instructions and the unlock code for the encrypted USB drive. Green Cloud Technologies, LLC operations@gogreencloud.com 11

12 DISASTER RECOVERY INSTALL 1. Proposal Marked Sold 2. Site Survey 3. Provisioning 4. Install Dealer installs NAS and connects USB drive. Green Cloud installs software, reboots, runs initial backup, copies seed data to USB drive, and then notifies dealer to return drive. 5. Synchronization 6. Mock Restore 7. Business Continuity Plan Green Cloud Technologies, LLC

13 DISASTER RECOVERY INSTALL Dealer Installation Steps 1. Unbox NAS and encrypted USB drive. 2. Connect NAS to network, using gigabit port. 3. Connect encrypted drive to NAS, using both prongs of Y- cable, unlock using provided passcode. 4. Call Green Cloud support, in order to verify everything is working and connecting properly. 5. Provide location for Image Manager and software installation/reboot window, a time when Green Cloud installers can safely reboot the servers after configuring the software. Green Cloud Technologies, LLC operations@gogreencloud.com 13

14 DISASTER RECOVERY INSTALL Green Cloud Installation Steps 1. Login to servers using management connection and provided greencloud account. 2. Install and configure ShadowProtect software agent on each server. 3. Install and configure Image Manager management/replication software on one server. 4. Reboot all servers. 5. Log back in the following business day and verify everything is working properly, copy initial full backup data to encrypted USB drive. 6. Notify dealer/customer to return USB drive. Green Cloud Technologies, LLC

15 DISASTER RECOVERY INSTALL Installation Notes By default, backups will happen around 7am, 3pm and 11pm each day. This can be adjusted per customer needs, of course. For example, /db servers may be backed up more frequently. Retention schedule: 7 days of intra-daily backups 14 days of consolidated daily backups 30 days of consolidated weekly backups 6 months of consolidated monthly backups It is absolutely vital that SQL backups and defrags are disabled and not running to drives we backup or they will generate extremely large incremental backups. Green Cloud Technologies, LLC operations@gogreencloud.com 15

16 DISASTER RECOVERY INSTALL Common Installation Problems Management connection doesn t work. Usually because incorrect public IP on the site survey and our firewall is blocking the unrecognized address. The local firewall is blocking the outbound connection. Drive not showing as connected to NAS. Drive not connected properly or is not unlocked. Cannot login to server or don t have permissions to install software. Provided greencloud account credentials are incorrect or don t have domain administrator (on DC s) or local administrator (on non-dc s) rights. Green Cloud Technologies, LLC operations@gogreencloud.com 16

17 DISASTER RECOVERY SYNCHRONIZATION 1. Proposal Marked Sold 2. Site Survey 3. Provisioning 4. Install 5. Synchronization Daily incremental updates will start replicating immediately upon install. Once full initial backup arrives on the USB drive and uploads have been stable for 1 week, servers are added to mock restore schedule. 6. Mock Restore 7. Business Continuity Plan Green Cloud Technologies, LLC operations@gogreencloud.com 17

18 DISASTER RECOVERY SYNCHRONIZATION Synchronization is when we match up the regular incremental backups we ve been receiving from the customer since installation to the full backups that were copied to the USB drive. 1. We receive the USB drive and copy the data to our datacenter storage facility. 2. Then we run a verification procedure on every file to make sure that the entire chain is intact and valid. 3. Notification sent to dealer/customer informing them their service is now live and going to billing. 4. Watch everything for 1 week, to make sure there are no problems. 5. Once service has been fully stable for a week, we will add the servers to our mock restore schedule. Green Cloud Technologies, LLC operations@gogreencloud.com 18

19 DISASTER RECOVERY SYNCHRONIZATION Common Synchronization Problems Backups not arriving in a timely fashion Not enough bandwidth available Backups may be too large Other backup software conflicting with ShadowProtect agent Backups being uploaded but are corrupt or incomplete Networking or internet issue NAS showing errors Hard drive issue (usually shipping related) Cable/switch connection bad Green Cloud Technologies, LLC operations@gogreencloud.com 19

20 DISASTER RECOVERY MOCK RESTORE 1. Proposal Marked Sold 2. Site Survey 3. Provisioning 4. Install 5. Synchronization 6. Mock Restore Green Cloud will bring up each server as a VM in a limited testing environment, as well as document any items that need to be addressed by the dealer/customer in a Mock Restore Summary document. This environment will be made available to customer/dealer for testing. 7. Business Continuity Plan Green Cloud Technologies, LLC operations@gogreencloud.com 20

21 DISASTER RECOVERY MOCK RESTORE Green Cloud Mock Restore Steps 1. Using the most recent incremental backup, we will start the mock restore of all servers for that customer, configuring a random high port to allow password-protected console access via Remote Desktop as well as internal network access so servers can communicate with each other. 2. We will create a Mock Restore Summary document and send it to the dealer/customer, containing: Information on how to access the environment for their testing and verification. Estimated Time to Restore in case of a disaster Notes on discovered services/features that may need some attention or changes A pre-generated site to site VPN form, that the customer can use as a template for pre-configuring a site to site VPN tunnel. Green Cloud Technologies, LLC operations@gogreencloud.com 21

22 DISASTER RECOVERY MOCK RESTORE Types of questions the dealers/customers should be asking themselves when they review the environment: Is the data there and confirmed good? Do programs work as expected? If not, what needs to change to make them work? What changes would I have to make to run off this virtual machine if I had a disaster? DNS changes? VPN ready? Do IP address changes affect service configuration (Exchange/IIS)? Do I have software that needs additional licensing to run off a new platform? Green Cloud Technologies, LLC operations@gogreencloud.com 22

23 DISASTER RECOVERY MOCK RESTORE Common Mock Restore Problems Not able to login to server in Safe Mode greencloud account not created as Local Administrator on that server. greencloud account disabled after installation by dealer/customer Windows 2003 server requires immediate activation before booting. 9 times out of 10, this is because the OS is licensed with an OEM license instead of a volume license Green Cloud Technologies, LLC operations@gogreencloud.com 23

24 DISASTER RECOVERY BUSINESS CONTINUITY PLAN 1. Proposal Marked Sold 2. Site Survey 3. Provisioning 4. Install 5. Synchronization 6. Mock Restore 7. Business Continuity Plan Based on the Mock Restore findings, dealer can generate a BCP incorporating information from the MRS as well as any other information at their disposal in order to prepare the customer site for a disaster. Green Cloud Technologies, LLC operations@gogreencloud.com 24

25 DISASTER RECOVERY - OVERVIEW Glossary of Terms Consolidated daily/weekly/monthly backup combined sets of incremental files that are generated every day/week/month Intra-daily the incremental files generated 3+ times a day NAS Network Attached Storage. Essentially means a networked external hard drive Seed Data the initial full backup, which all incremental backups will refer to in order to make a complete chain Green Cloud Technologies, LLC operations@gogreencloud.com 25