I D C T E C H N O L O G Y S P O T L I G H T

Transcription

1 I D C T E C H N O L O G Y S P O T L I G H T AF A Snap shot Implementations W ill Broaden Use of Snapsho t Te chnol og y March 2015 Adapted from A Comparative Look at Disk-Based Data Protection and Recovery Strategies and Cloud Adoption, IDC # Sponsored by EMC Snapshot technologies have long been a part of enterprise-class storage arrays. The ability to quickly create disk-based, point-in-time copies of data enables the data to be more easily accessed and used for business processes and administrative operations across both primary and secondary application environments. This paper examines enterprise-class snapshot technologies, discussing use cases for snapshots as well as classic challenges associated with using snapshot technologies in pure hard disk drive (HDD) based arrays. It then looks at a snapshot implementation from EMC on its XtremIO all-flash array (AFA), exploring the advantages that flash media and a new design point can provide in the use of snapshot technology. Introduction Most enterprises keep between 7 and 10 copies of their production data. Depending upon their intended use, these persistent copies can be colocated with primary storage or kept in secondary data stores. Enterprise workflows that create these copies leverage disk-based, point-in-time copies of production volume data. There are two basic types of persistent, disk-based, point-in-time copies: snapshots and clones. Snapshots are logical copies of data at a specific point in time so they can be created very quickly. They can also be very space efficient because no data is copied up front just metadata representing the logical volume. Snapshots can be read only or read/write. The creation of a read-only snapshot effectively requires little or no space. During the creation of a read/write snapshot, some storage capacity is often reserved to accommodate the expected changes to the writeable snapshot volume. Thus, there is a small amount of up-front space consumption. Two different methods, termed copy on write and redirect on write, are used to accommodate writes to the original volume or the snapshots, or both. Each method has different performance and management implications. With traditional implementations, snapshot performance tends to degrade as more snapshots are created and retained. Clones are full physical copies of a volume at a specific point in time, and their creation time is directly proportional to the amount of data that has to be copied. A full clone will take up the same amount of space as its source volume. However, full clones typically deliver the same performance as the original source volume. The industry term linked clones effectively describes a read/write snapshot. Linked clones are often used where multiple snapshots will be created from a single source volume at a single point in time for example, a golden master desktop template in virtual desktop infrastructure (VDI) environments. IDC 1856

2 There are five basic reasons that enterprises create and use these disk-based, point-in-time copies: to protect against logical data corruption, to meet a wide range of data protection requirements, for use in test and development environments, to offload data processing from production servers, and for bulk provisioning of virtual machines (VMs). Performance is a critical concern for all of these operations. Traditional snapshot technologies have performance, scalability, space-efficiency, and management limitations that limit administrators' ability to effectively build the relevant workflows. These technologies can be evaluated across four metrics: How efficiently copies consume raw storage capacity for both metadata and data How long it takes to create the copy as well as the impact of copy operations on production services Operational performance of the copy and what impact the copy's existence has on the source volume performance Management limitations the copy imposes on administrators (Typical limitations are restrictions on deletion, unacceptable hierarchies [e.g., snapshots of snapshots], refresh of snapshots from source volumes, and which data services [e.g., inline data reduction] can be used.) Some of these limitations are due to the nature of spinning disks. The same physical disk usually hosts both the original source volumes and the point-in-time copies. Snapshot I/O operations increase disk head contention and typically reduce performance for the source volumes. With many applications requiring disk access latencies in the low milliseconds, it wasn't feasible to actively access both source and snapshot volumes. These performance issues resulted in many disk array vendors limiting the number of snapshots an array could retain. There are other design issues as well: how metadata is managed (e.g., by reference versus copied, 100% in-memory versus swapped in from the drives) and how physical data is referenced in snapshot hierarchies (chains of multiple successive snapshots of the same evolving set of source volumes). These implementation choices affect snapshot performance, scalability, space efficiency, and management. Administrators reacted accordingly. When snapshots were needed for a business process, they were often created, used, and then immediately discarded. Administrators did not keep large numbers of snapshots in their systems because of these concerns. In theory, business operations for data protection, test and development, offload processing, and bulk provisioning areas could have benefited from the use of many snapshots. In practice, the previously mentioned implementation issues, compounded by disk drive rotational latency, limited the value that snapshots could realistically provide in datacenter workflows. Administrators want to create high numbers of snapshots rapidly and space efficiently, retain the snapshots for long periods, and do all of this while being able to rely on consistently high performance for all data (production and snapshots). They want the decision of whether snapshots are read only or read/write to be based on policy, not performance concerns. Traditional snapshot implementations could not support all of these requirements simultaneously. With the advent of AFAs, enterprise storage vendors have an opportunity to redesign snapshot implementations to meet all of these requirements. A successful implementation results in simplifying many common datacenter workflows while improving administrator productivity IDC

3 Definitions AFAs are storage platforms that rely entirely on flash media to meet all performance and capacity requirements. EMC XtremIO's AFA enterprise storage platform uses two types of snapshots to serve the five key use cases for persistent, disk-based, point-in-time copies of data. "Read-only" snapshots meet the need for immutable copies of volume data, while "read/write" snapshots meet the need for mutable copies of volume data. As we discuss the XtremIO snapshot implementation, it will become clear that these two types of snapshots better meet the needs of datacenter administrators than traditional disk-based snapshot and clone technologies. XtremIO snapshots enable new workflows that were not feasible in the past. Considering EMC XtremIO The EMC XtremIO product targets mixed workload consolidation in enterprise environments. Its design engineers could assume all-flash media and had the opportunity to implement data services that take advantage of random access and low latency. Additionally, the XtremIO snapshot implementation has been architected from the start for the platform's scale-out, flash-based architecture. The result completely addresses the performance, scalability, space-efficiency, and management limitations of traditional snapshots. EMC XtremIO read-only snapshots provide high performance, space efficiency, and massive scalability and do not limit data services. In cases where writes are useful, EMC XtremIO read/write snapshots also provide all of the previously mentioned benefits without the limitations typically seen in linked or full clone implementations of the past. XtremIO snapshot technology effectively overcomes all challenges of traditional snapshot technologies implemented for use in HDD-only arrays. Administrators gain significantly more flexibility and agility in each of the five key use cases for snapshots. Examples in each area include: Protecting against logical data corruption. Although backups provide data protection against logical corruption as well as comprehensive recovery points, they are generally completed only periodically (say hourly). XtremIO snapshot technology can create and retain hundreds of snapshots seconds apart, providing many successive point-in-time copies. XtremIO actually supports near-continuous data protection if a customer so desired, a rolling record of changed data state for selected volumes can be retained over short periods (e.g., a copy every minute for the most recent 4 hours). Increased granularity minimizes data loss in the event of logical corruption by increasing the likelihood of a good recovery point just before the logical corruption occurred. Alternatively, snapshots can be created and retained at longer intervals, say every hour or so, to provide multiple recovery points over the course of one or more days. These snapshots reside in primary high-performance all-flash storage and are quickly accessible at the same high performance as the source volumes. Improving data protection operations. Key concerns in data protection operations revolve around backup windows, plus recovery point objectives and recovery time objectives (RPOs and RTOs). XtremIO snapshots are created instantaneously, can integrate with popular snapshot backup APIs like vsphere VADP and Microsoft VSS to provide application-consistent recovery points, and can be taken frequently with no noticeable impact on application performance. Selected snapshots can then be mounted on backup servers to perform the physical copy to secondary storage, effectively moving the backup window to zero. The ability to create a large number of snapshots without any impact on performance enables XtremIO to meet extremely stringent RPO requirements. Because of their high performance and instant accessibility, XtremIO snapshots meet extremely stringent RTO requirements IDC 3

4 Injecting more reality into test and development environments. Sometimes it is difficult to create synthetic data sets and workloads that closely match production data in terms of contents, size, and I/O profile. The ability to actually work with copies of production data helps focus development operations and produce better results in less time. The ability to test with these data sets at production performance levels provides accurate feedback about real-world performance with relevant data services enabled. XtremIO can instantly create snapshot copies, thus providing fresh copies of production data that consistently deliver the same performance as the source volumes. The data is inherently deduplicated and compressed, making it economically feasible to provide individual copies of the data set to each test and development engineer. This further parallelizes operations and shortens time to market. If privacy or security is a concern, an initial writeable snapshot can first be sanitized and then repeatedly snapped as a template. Enabling better offload processing. A number of batch-oriented operations, ranging from business analytics to data protection, support business and administrative operations in datacenter environments. The whole point of offload processing is to perform these operations without affecting production performance. This allows them to run concurrently with production operations that are generating revenue, driving manufacturing, or delivering services on a 24 x 7 basis. The ability to effectively create and use as many snapshots as needed (instantaneously and while continuing to consistently provide the same performance levels as source volumes) provides the maximum freedom in leveraging production data sets for offload operations. There is no creation lag time or additional space consumption as there would be with conventional clones, and the XtremIO snapshots benefit from all the same storage efficiency technologies and data services as the source volumes. This allows the results of business analytics to be applied back into the business faster and enables dependent operations in multistage workflows (e.g., periodic replication of snapshots to an offsite location for disaster recovery purposes) to work with more frequent, efficient, and timely copies of production data without impacting production. Optimizing bulk provisioning. Bulk provisioning operations are typically used to create many VMs from a common template and are used for both server and desktop (VDI) provisioning environments. XtremIO snapshots can create hundreds of copies of a single VM (e.g., a server template or a golden master desktop template) in seconds. Each copy will deliver the same performance as the source volumes. The snapshots work off a single logical volume for space efficiency, tracking only the changes for each copy as they occur. One hundred percent of snapshot metadata is held in memory, ensuring that performance stays consistent on reads and writes across all snapshots, regardless of the read or write activity against each. There are no limitations in terms of data services support any data services available for use with the source volumes can be applied equally to the snapshots. In VDI environments, the use of space-efficient bulk provisioning technologies can easily achieve data reduction ratios (and hence storage capacity savings) in the 15:1 to 20:1 range. With XtremIO snapshots, EMC has provided a high-performance, very scalable, and space-efficient implementation that imposes no management limitations in the use of snapshots, regardless of whether they are read only or read/write. Let's turn to a quick review of EMC's technology so we can see how this is done IDC

5 Feature Details The XtremIO array natively supports inline data reduction, and it is important to understand how this operates before we move to a brief overview of the snapshot implementation. XtremIO uses content addressing combined with in-memory, dual-stage metadata management. Writes entering the system are separated into chunks of fixed length and fingerprinted using a very robust cryptographic function. The written logical block address is associated with the fingerprint (metadata stage 1). The physical location of the chunk on nonvolatile flash storage (metadata stage 2) is determined by the configuration of the system and bits in the fingerprint. The mathematical properties of the cryptographic function ensure very even distribution of data across all controllers and solid state disks (SSDs). If the chunk is unique, metadata is updated to reflect its creation and the chunk is then compressed and written to flash. If the chunk is not unique, then only the stage 1 metadata table is updated with the location of the existing chunk in the stage 2 metadata. Deduplication is global across the entire XtremIO system to support maximum data reduction ratios. XtremIO keeps a single logical copy of every chunk, ensuring data integrity through two key data redundancy mechanisms. The data is first mirrored using a low-latency RDMA fabric across two physically separate controllers before acknowledging a successful write back to the application. It is then written to flash using XtremIO Data Protection (XDP) to ensure its resiliency. XDP was specifically developed for flash media. It leverages the random access nature of flash and XtremIO's unique dual-stage metadata engine. The combination delivers the performance of RAID 10, the protection of RAID 6, and only 8% capacity overhead. XDP minimizes the writes to flash (extending flash endurance) and at the same time makes for extremely efficient free space management. XDP does not require performance-blocking garbage collection operations and allows XtremIO to deliver very consistent latencies even as configurations scale. Creating a snapshot is an immediate operation. For each volume, the system allocates two empty metadata containers pointing to the metadata of the source. This happens instantaneously and does not consume any capacity. No data or metadata is copied even for a read/write snapshot. Contrast this with traditional snapshot implementations that must copy the entire volume metadata set in order to create a read/write snapshot. The traditional implementation is time consuming and leads to capacity bloat as snapshots accumulate in the array. XtremIO snapshots can be of a single volume or of groups of volumes (an atomic consistency group). Snapshots of a volume snapshot group are consistent across all volumes. Unlike traditional implementations, snapshots are inherently read/write in the XtremIO system and are immediately available for consumption. To make them read only, the system just disallows writes to the relevant SCSI target(s). Snapshots start to consume capacity only when they or the source volumes are actually written. That space is taken from a global resource pool across the entire scale-out system, eliminating the need for snapshot space reservations. This approach is more space efficient and easier to manage than the traditional approach of preallocating space for writes associated with each individual snapshot. XtremIO supports hundreds of snapshots per snapshot tree and many thousands of snapshots across the entire system. It is critical to the performance and scalability of this implementation that all operations associated with the snapshots occur in main memory at RAM latencies. XtremIO uses a data structure called the existence bitmap for efficient operation. The existence bitmap ensures consistent read performance in snapshot hierarchies regardless of the length of the chain. It is stored in main memory and allows the correct block of data in every snapshot to be directly located without having to reference snapshot ancestors. This is very different from traditional snapshot implementations, which depend upon ancestor references and can significantly slow read performance by having to walk the entire snapshot chain for read requests IDC 5

6 Another key performance and scalability feature of this implementation is that the workload is automatically spread very evenly across all system resources by the cryptographic fingerprinting process. This enables the system to scale in both performance and capacity as additional resources are added. All storage controllers in the XtremIO storage cluster are constantly engaged in managing both the data flow and the metadata. In traditional dual-controller architectures, load is typically spread only across each controller pair, limiting scalability. A number of popular snapshot-based workflows take successive (or cascading) snapshots over time for an evolving data set for example, daily backups or copying a production data set for recurring testing, development, or business analytics operations. The XtremIO snapshot technology includes a refresh feature that makes it simple to refresh volume contents, enabling an immediate and unrestricted refresh of any snapshot to any other snapshots or production volumes. Traditional snapshot-assisted backup workflows have to create a new snapshot, map it to the offload server initiator, mount it, and start the backup. XtremIO can immediately and instantly change the data underneath an existing mounted volume to reflect new contents with a simple SCSI rescan. The XtremIO snapshot implementation includes a scheduler for automation of local protection policies. Scripting snapshot-based workflows is possible via either a REST API or a command line interface (CLI). Native snapshots integrate with virtual host snapshot APIs like VSS to create crash-consistent copies. For the most complete application integration, XtremIO can also be used in conjunction with another EMC software product, AppSync, to create application-consistent snapshots. XtremIO delivers a fully modern snapshot implementation that takes advantage of optimizations possible only with an all-flash array. AFAs can instantly create snapshots with 100% full performance. XtremIO snapshots are metadata and capacity efficient, with in-memory snapshot operations that enable a fully flexible snapshot topology. Volumes can be deleted anywhere in the hierarchy without losing subsequent snapshots. XtremIO offers unrestrictive immediate refresh capabilities. When these capabilities are coupled with the ability to retain large numbers of high-performance, space-efficient snapshots, administrators gain the agility to leverage snapshots for improving business and administrative operations. Challenges The AFA market is on fire, and IDC expects it to grow at a 46.1% CAGR over the next five years. AFAs offer better performance; don't require capacity overprovisioning; enable the use of data services like inline data reduction, thin provisioning, snapshots, and clones that were severely hampered in HDD-based environments; and offer a total cost of ownership (TCO) profile that can be 50 80% lower than that of legacy storage architectures over an enterprise depreciation life cycle. Despite all these advantages, one very familiar storage metric from the past has hindered AFA adoption: the $/GB cost for raw capacity. HDD-based systems still offer a lower overall $/GB acquisition cost. Even though storage decision makers stubbornly cling to it, IDC believes that the use of the acquisition $/GB metric does not accurately reflect the value that AFAs bring to datacenters for use in primary production environments. AFA snapshot implementations like XtremIO tip the scale more heavily in AFA's favor relative to HDD-based snapshot implementations, particularly when you consider the 7 10 copies of production data that are routinely retained by most enterprises. Snapshot implementations that take advantage of features like all in-memory operations, offer equivalent performance on all source as well as snapshot/clone volumes, and impose no management limitations in terms of snapshot topologies or data service usage are much more cost efficient than traditional implementations. Administrators who can't get past the acquisition $/GB comparisons for flash and spinning disk may never see these advantages IDC

7 Hybrid flash arrays (HFAs), storage platforms that can (but do not necessarily) use both flash and spinning disk to meet performance capacity requirements, typically provide traditional snapshot implementations as part of their data services offerings. Due to its lower latencies, flash will allow these implementations to perform better than pure HDD-based configurations. However, they do not offer the performance, scalability, space-efficiency, and management advantages of snapshot implementations like XtremIO (specifically architected for use in flash-based arrays). All snapshot implementations are not born equal, and it is EMC's marketing challenge to make storage administrators understand the performance, scalability, space-efficiency, and management benefits of newer snapshot implementations built specifically for use in AFA environments. The risk is that potential buyers do not understand the distinction and assume that snapshots are snapshots, with all their attendant limitations, regardless of the platform. Conclusion AFA platforms offer significant opportunity to redesign snapshot implementations and deliver on the promise of snapshot technology: to provide read-only and read/write snapshots that can be created very rapidly, are very space efficient, deliver performance consistent with that of source volumes regardless of how many there are, and can be retained for long periods of time without performance impacts. Snapshots that meet these requirements open up new ways for administrators to improve the classic snapshot-based workflows protecting against logical data corruption, improving data protection operations, injecting more reality into test and development environments, enabling better offload processing, and optimizing bulk provisioning. They also enable new workflows that can benefit from these readily available point-in-time copies of production data. Not all AFA vendors have redesigned their snapshot implementations along with the release of their flash-based storage platforms. EMC is one vendor that has, and the company has made several key design decisions (e.g., maintaining all snapshot metadata and performing all snapshot/clone operations in main memory 100% of the time, implementing a "redirect on unique write" method that requires no data movement during snapshot creation or writes, supporting maximum topology flexibility, and enabling an instant refresh capability) that deliver on the promise of snapshot technology in flash-optimized storage platforms. IDC expects other AFA vendors to pursue a similar development path for their AFA-based snapshot technologies as awareness increases around EMC's implementation and benefits. A B O U T T H I S P U B L I C A T I ON This publication was produced by IDC Custom Solutions. The opinion, analysis, and research results presented herein are drawn from more detailed research and analysis independently conducted and published by IDC, unless specific vendor sponsorship is noted. IDC Custom Solutions makes IDC content available in a wide range of formats for distribution by various companies. A license to distribute IDC content does not imply endorsement of or opinion about the licensee. C O P Y R I G H T A N D R E S T R I C T I O N S Any IDC information or reference to IDC that is to be used in advertising, press releases, or promotional materials requires prior written approval from IDC. For permission requests, contact the IDC Custom Solutions information line at or gms@idc.com. Translation and/or localization of this document require an additional license from IDC. For more information on IDC, visit For more information on IDC Custom Solutions, visit Global Headquarters: 5 Speen Street Framingham, MA USA P F IDC 7