Backups to the Cloud Our Journey to Move Beyond Tape pollev.com/ryanbass401 Educause 2018 Presented by: Ryan Bass, Juliana Kutch, Thomas Litterer, Shingi Marangwanda Introductions Ryan Bass Juliana Kutch Thomas Litterer Associate Chief Information Officer, Technology Infrastructure Project Coordinator Associate Director, Enterprise Systems and Research Solutions Agenda Presentation Logistics About Portland State University Disaster Threats and Goals Tape Architecture Budget Planning New Cloud Based Offsite Backup Architecture Our Process and Lessons Learned Shingi Marangwanda Manager, Infrastructure & Cloud Applications
Questions You have the option to ask questions via the URL above, you can also upvote your favorite questions. 4 Poll URL https://pollev.com/ryanbass401 5 Audience CIOs and Senior IT Leadership IT Management Individual Contributor https://pollev.com/ryanbass401 6
Worksheet Portland State University and Disaster Recovery 8
Overview of PSU 10 Previous Data Center Disasters at PSU March 2007 (time to recovery ~8 hours) Failure of redundant power feeds October 2008 (very close call) UPS Room Cooling Failure September 2009 (time to recovery ~12 hours) Building Fire Alarm Disaster November 2012 (time to recovery ~15 hours) UPS Disaster 11 Regional Disasters at PSU May 30, 1948
Cascadia Subduction Zone Last event occurred in 1700 Average recurrence is 240 years 13 Project Goals Evaluate feasibility of using cloud based technology to improve our current backup architecture Improve disaster recovery capabilities, especially for a regional disaster Minimize additional cost as much as possible 14
PSU backups PSU s backup environment: tape architecture Weekly Personnel Media Servers Onsite Disks Tape Library DATA CENTER Commvault License Backup ARCHIVE
Worksheet Example Current Expense Monthly Cost Data Center $300 Investment Cost Type $1,000,000 Fixed Personnel and Training $12,000 $160,000 Fixed Software $8,000 $500,000 Investment Media Servers $900 $40,000 Investment Onsite Disks $2,000 $60,000 Investment Tape Library $2,200 $0 Monthly New Tapes $250 $0 Monthly Offsite Storage $1,200 $0 Monthly Fixed Costs - Budgeting Cloud Backups Fixed costs must be paid regardless of backup method Data Center Space Personnel and Training Other Universities who rent data center space or handle lots of tapes could reduce costs here Current Investments - Budgeting Cloud Backups Current Investments include purchased software, and hardware that is not fully depreciated Commvault ($500k invested) Media Servers ($40k value) Onsite Disks ($60k value) Other considerations might include network, fiber channel, NAS, or virtualization software 21
Monthly Tape Costs - Budgeting Cloud Backups Other costs which are paid for yearly or less. Fully depreciated Tape Library Tapes Offsite Storage (Iron Mntn) Budget for these items can easily convert to Cloud budget Cloud (AWS) Costs - Budgeting Cloud Backups Costs to maintain backups in the public Cloud NetApp Marketplace license EC2 for NetApp S3 for NetApp S3 for Commvault We avoided Commvault growth by utilizing existing NetApp OnTap features Migration from Tape to Cloud Today
Backup Appliance - Budgeting Cloud Backups Replace all Investments and Monthly costs. Requires large up-front capital expense. Rubrik ($400,000 up-front) Rubrik Support S3 (equivalent to Commvault) Good option when starting from scratch or when existing investments are minimal. Worksheet Example Current Expense Monthly Cost Data Center $300 Investment Cost Type $1,000,000 Fixed Personnel and Training $12,000 $160,000 Fixed Software $8,000 $500,000 Investment Media Servers $900 $40,000 Investment Onsite Disks $2,000 $60,000 Investment Tape Library $2,200 $0 Monthly New Tapes $250 $0 Monthly Offsite Storage $1,200 $0 Monthly Introduce yourself to your neighbors Are you considering cloud for offsite backups? If so, what are your current barriers/challenges? If not, why? 28
Technical deployment Our methodologies At PSU, we decided to leverage our existing investment in 2 different backup methodologies: AWS components used for Backups Amazon EC2 virtual computing environment Amazon EBS (elastic block storage) persistent block storage volumes Amazon S3 Object Storage S3 tiers S3IA (infrequent access - cheaper) Glacier (even cheaper cold storage) We decided not to use Glacier because of 90 day policy
Commvault environment Tape Windows Linux Storage etc CV in PSU Data Center OFFSITE DATA CENTER ONSITE Commvault environment Internet2 S3 Object Storage Windows Linux Storage etc CV in PSU Data Center CLOUD DATA CENTER ONSITE Netapp environment SnapMirror Changes AWS Virtual Private Cloud (VPC) VPN Storage Netapp DATA CENTER WAN EC2 VM S3 Object Storage EBS Disk (cache) CLOUD (AWS VPC) ONSITE
CommVault PROS CONS Great compression and deduplication of data Similar process to tape - just change write storage target from tape to S3IA Expensive licensing Restore time is limited due to S3 speed Restore time to the cloud is slower and more complicated than Netapp Backup data is encrypted in flight by software Netapp Cloud Volumes ONTAP PROS CONS Data in AWS can be accessed from prem for recovery or use Does not have a file recovery catalog Data can be made accessible to EC2 instances in the cloud VPN required for transfer Hardware dependency Increased cost because of EC2 VM and EBS Disk requirements in AWS Short Recovery Time Objective (RTO) Current stats 310TB of data backed up to the cloud 5TB/day being backed up to the cloud Almost no more tapes!
The process
Challenges for moving to the cloud Source: RightScale 2018 State of the Cloud Report Overcome lack of experience BEFORE THE PROJECT Training for staff (AWS and Azure) DR Tests in AWS and Azure (multiple teams) DURING THE PROJECT Use existing infrastructure Work closely with vendors
Overcome security and compliance challenge Leverate existing infrastructure (CV and NetApp) Additional Security Review for cloud components Encryption (transport, at rest) Data integrity (verification) System integrity (authentication, logging) Governance and control Figure out what you want to track in terms of cost and configure tags from the start so you can monitor them: Netapp Commvault Research Departments Administrative Manage cloud spend concerns Methodical deployment while observing costs Constant check-ins especially as big volumes get moved Plan for lumps!
Lessons Learned Consider campus priorities for RTO/RPO Archival policies review What is actually a priority in case of a disaster? Consider future campus partnerships Regions, retention needs Many vendors are also new at this Even names are changing (marketing side, technical side) Documentation not 100% Configure cost tracking from the start AWS admin panel allows you to add a tag to different datasets Worksheet Identify what challenges are keeping your organization from moving beyond tape 51 Worksheet How to overcome challenges? Security Security questionnaire -> review Managing cloud spend Phased methodical deployment Lack of experience Training, pre-project projects Governance/control Tagging and monitoring Compliance Using same infrastructure 52
Takeaways 1 2 Cloud backups improve campus resilience to natural disasters. Make sure you know what your fixed costs and current investments are, and what funds are available for cloud backups. 3 4 Explore potential of new and existing infrastructure for better cost and efficiency. Consider the projects before the project. Questions
Thank you. References Cover image: Photo by Samuel Zeller on Unsplash Cascadia Fault image: https://scitechdaily.com/geologists-find-anomalies-pieces-of-mantle-found-risi ng-under-cascadia-fault/ Icons: flaticon.com