MODERN ARCHIVING WITH ELASTIC CLOUD STORAGE (ECS)

Transcription

1 MODERN ARCHIVING WITH ELASTIC CLOUD STORAGE (ECS) Lowering costs while supporting your digital transformation ABSTRACT This white paper explains how ECS helps organizations meet the requirements of modern archiving. October 2016 WHITE PAPER

2 The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC 2, EMC, the EMC logo, and Elastic Cloud Storage are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. Copyright 2016 EMC Corporation. All rights reserved. Published in the USA. 10/16, white paper, H EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice. EMC is now part of the Dell group of companies. ii

3 TABLE OF CONTENTS EXECUTIVE SUMMARY...1 DIGITAL TRANSFORMATION THROUGH MODERN ARCHIVING...2 WHY YOU NEED A MODERN ARCHIVE...3 ECS IS YOUR MODERN ARCHIVING SOLUTION...4 Lower Your Costs... 4 Simplify Your Datacenter... 4 Store All Types of Data... 5 Allow Access by Diverse Applications... 6 Provide Global Access to Data... 7 Offer High Scalability... 7 Ensure Compliance... 8 Replace Tape... 8 Work with Any ISV You Choose... 9 Go Beyond Archiving... 9 CONCLUSION iii

4 EXECUTIVE SUMMARY If you re like most IT leaders, you re thinking about how to guide your organization through a digital transformation. You know that using technology effectively can change how you do business. You also know that making this change isn t optional it has to happen. What you might not know, however, is that changing your approach to data archiving can be an important part of doing this. As in traditional archiving, a modern approach still saves you money by moving less frequently accessed data to lower-cost storage. But rather than let that data sit unused, a modern approach keeps this data available, helping you find new ways to analyze and monetize your information. A media company storing videos, then serving them to users on demand is doing modern archiving, as is an organization creating a data lake that it uses for diverse analytics. As the way we use archived data has changed, so has archiving technology. Dell EMC Elastic Cloud Storage (ECS) provides a solution for modern archiving. Its object-based approach to storage lets it hold all kinds of data, including files, video, and more. Its support for geo-replication of data provides massive scale, along with fast access from anywhere in the world. And once you ve installed ECS for archiving, this storage technology can also be used for other scenarios, such as supporting data-intensive applications. Modern archiving both lowers your costs which helps free up resources for digital transformation and lets you use archived data in new ways. But being successful with modern archiving requires choosing a technology that meets today s requirements. ECS is designed to be this technology. AUDIENCE This white paper is intended for anybody who makes decisions about archiving solutions. It assumes that you have a basic understanding of storage technologies, but it doesn t assume any background in EMC Elastic Cloud Storage. 1

5 DIGITAL TRANSFORMATION THROUGH MODERN ARCHIVING At Dell EMC, we believe technology exists to drive human progress on a global scale to create new markets, reshape industries, and improve the lives of every person on the planet. As part of this, we see organizations undergoing a digital transformation. For some, the goal of this transformation is to use technology to leapfrog the competition by offering innovative products and services. For others, the focus is on avoiding disruption from new market entrants. Whatever your situation might be, it s clear that you can t ignore the change. But for IT leaders, this raises an uncomfortable question: Where will you find the money to make this transformation? You re already under constant pressure to lower IT costs. How can you invest in new technologies while still doing this? How can you fund your digital transformation? One answer is to find ways to spend less on what you re already doing, then use those savings to pay for new efforts. And one way to do that is to lower your cost of storing data. This might seem like an unreachable goal, since we re in the middle of a data explosion. Whether it s more video, giant logs, massive data streams from sensors used in Internet of Things (IoT) scenarios, or something else, virtually every IT shop is required to hold onto more and more data today. Yet there is a way to achieve this goal while lowering your costs and freeing up money for digital transformation. The key is to focus on the fact that most of your data doesn t need to be kept on expensive, tier-one storage. Instead, it can be archived at much lower cost. This doesn t mean using traditional compliance-oriented archiving technologies, however, although those can still be important. What s required is the adoption of modern archiving, bringing a broader and significantly less expensive approach to data storage. Dell EMC s Elastic Cloud Storage (ECS) is an object store designed to support modern archiving. This paper explains what modern archiving is and how ECS supports it. The goal is to make clear how improving your archiving technology can save money, improve long-term access to your valuable data, and become an important part of your organization s digital transformation. With modern archiving, lowering costs remains a primary goal. But modern archiving can do more; it can be the engine that leads your digital transformation. To see why this is true, think about the kinds of data that organizations need to archive today. Examples include the following: A media company might need to archive video of movies, sporting events, and more. To monetize this data, the company also wishes to offer its customers the ability to stream this video on demand. A genomics firm might archive the massive amount of data that its scientists create, then make that data available to global research partners over the Internet. A manufacturing firm might archive large volumes of historical data produced by sensors on the factory floor, then use machine learning to find patterns in this data, such as predictions of when robots need preventive maintenance. A retailer might wish to archive information into a data lake that holds diverse data in its original form. This information could include clickstreams from its e-commerce site, point-of-sale data from its brick-and-mortar stores, and data produced by inventory tracking applications. The retailer might then run various kinds of analytics on this data to learn how to improve its business. For all of these situations, using high-performance, high-cost tier-one storage isn t required. In other words, they re all archiving scenarios. Yet none of them matches the traditional, often compliance-oriented, view of archiving. Instead, they re all examples of modern archiving, with each one needing a way to store data for less while still getting more business value from that data. And more business value for less money is exactly what an organization needs to lead its digital transformation. 2

6 Why You Need a Modern Archive A modern archive can lower your costs. The fundamental goal of archiving is to save money by moving less frequently used data from your highest-cost primary storage to something less expensive. This reduces the amount of high-cost storage you need to buy, and it also shrinks the volume of active data that must be backed up. In fact, most data centers will find that less than 25% of the data they store requires tier-one storage. Even though the other 75% might not fit the traditional view of archiving, it can still be kept on a modern archiving solution. A modern archive can simplify your datacenter. If you re like most organizations, you use different storage technologies for different purposes: files, SAN data, backups, and more. A modern archive can simplify your world by providing a shared archive for all of them. Figure 1 shows how this looks. Figure 1: A modern archive can store all kinds of data. Rather than using different archiving technologies for different kinds of storage, you can use a single modern archive for everything. A modern archive can store all types of data. Given the diversity in use today, a modern archiving technology must be able to store many different types of data, including files, databases, unstructured data such as logs, and more. In other words, a modern archiving solution must be able to act as a data lake. A modern archive allows access by diverse applications. In modern archiving, archived data is accessible on demand by various applications. In fact, one important reason for creating a data lake is to run analysis applications against that data. This implies that a modern archiving technology must support a variety of access protocols. For instance, a common way to access a data lake today is via the protocol defined by the Hadoop Distributed File System (HDFS), so a modern archiving solution should support this protocol. A modern archive offers global access. While archived data isn t accessed as frequently as data kept on primary storage, modern archiving means keeping this data available. Since applications run everywhere, including mobile devices, a modern archiving solution must provide easy and fast access to data from anywhere in the world. It must also offer a multi-tenant solution, one designed to be shared by multiple groups. And as more and more organizations consolidate their datacenters, there s a shift to centralized archiving. Rather than storing archived data in each datacenter, modern archiving must support a shared approach where multiple datacenters rely on a single global archive. A clear implication of this is that a modern archiving technology needs to be securely accessible from anywhere, including outside the four walls of your datacenter. A modern archive provides high scalability. We live in the big data era, and so the amount of data you ll need to store won t shrink anytime soon. Since a majority of your data can be stored in a modern archiving solution, the technology you choose to do this must be very scalable. And since scalability results from design it s hard to add after the fact you should plan for this up front. 3

7 A modern archive ensures compliance. In many industries, archiving is an essential part of meeting an organization s legal requirements. While modern archiving goes well beyond this traditional role, archiving technologies must still provide what s needed to ensure compliance with applicable laws and regulations. A modern archive can replace tape. Tape has long been a popular storage medium for cold archiving and longterm data retention. But as your organization finds more ways to use its archived data, letting any of that data be inaccessible no longer makes sense. There can be real value in making all of your data instantly readable. Think of a sports network, for example, that s kept all of its old games on tape. If these were available on disk in a modern archive, the network could stream them to customers on demand. What was a cost paying for tape storage can instead become a new revenue source. A modern archiving technology can make this possible. A modern archive lets you work with any ISV you choose. Archiving technologies don t exist in a vacuum. Especially when they re used to store many different kinds of data, you might use a variety of products from multiple vendors to move data to and from archives. Because of this, a modern archiving solution must work with solutions offered by a broad range of ISVs. A modern archive supports scenarios beyond archiving. If a technology is general enough to meet the needs of modern archiving, it should also support other scenarios. It might provide a foundation for creating new applications, for instance, that use large amounts of diverse data. If the majority of your data will be kept in a modern archive and it will letting that data be used by applications that don t require tier-one storage performance is important. ECS IS YOUR MODERN ARCHIVING SOLUTION ECS provides all of the benefits that modern archiving can bring. This section walks through these benefits, showing what ECS offers in each area. Lower Your Costs You can buy ECS either as a self-contained appliance or as software that runs on servers you supply. In either case, ECS is running on commodity hardware. This makes ECS less expensive to acquire than solutions that require their own custom hardware. ECS also costs less to run than alternatives. The main reasons for this are the following: ECS can provide built-in disaster recovery. Organizations commonly deploy ECS nodes at two or more geographically separated sites, and ECS automatically replicates data across these sites. This means that all data is available in more than one place, and so organizations aren t required to spend money on separate disaster recovery solutions. ECS uses storage capacity very efficiently. For example, ECS uses erasure coding to achieve high reliability in the face of failures. This approach incurs much less storage overhead than other options, and so lowers the cost of operating ECS. ECS erasure coding is especially cost-efficient for data that s replicated three or more times. Simplify Your Datacenter ECS can act as a common archive for all of your data, whether it s stored on Dell EMC offerings or products from other vendors. Figure 2 illustrates how ECS fits with your existing Dell EMC investments. 4

8 Figure 2: ECS can be used as an archive by many other Dell EMC products. As the figure shows, the technologies ECS can work with include the following: Isilon: This NAS system provides Cloud Pools, an option that can transparently move less frequently used data to and from ECS storage. Data Domain: Focused on storing data for backups, archiving, and disaster recovery, this Dell EMC solution provides Cloud Tier to move infrequently accessed data to ECS and other cloud storage services. Even though Data Domain is also intended to hold archived data, ECS can often be a lower-cost option. Cloud Array: This product can act as a cache for ECS data, making it available via standard protocols such as NFS, CIFS, and iscsi. CloudArray can also be used by other EMC offerings, including VMAX, VNX, VxRail, and VPLEX, to access data stored in ECS. Data Protection Suite: Both of the products in this suite, Avamar and Networker, can use a technology called. CloudBoost to move archive data to and from ECS. The key thing to realize is that ECS can provide a modern archive for all of your data, wherever it s kept and however it s used. Store All Types of Data ECS provides an object store, not a filesystem. Accordingly, it stores data in objects that are grouped into buckets. Figure 3 shows how this looks. Figure 3: Each ECS object has a unique identifier, along with data and metadata that describes that data. 5

9 A bucket can contain an arbitrary number of objects, but it can t contain other buckets there s no hierarchy. Instead, each object has a unique identifier, as the figure shows. All that s required to access an object is to provide its identifier. Each object also contains data, which can be just a single byte, several petabytes, or something in between. Whatever its size, that data can be anything; an object imposes no structure. This generality gives ECS the ability to store all types of data. Each object also contains metadata, which consists of key/value pairs. Some parts of this metadata are defined and maintained by ECS, such as the object s size and creation date. You can also define your own metadata, however. For example, an organization using ECS to archive movies could create metadata for each movie that specified the movie s stars, its Rotten Tomatoes rating, and other information. Because ECS provides the ability to search for objects based on their metadata, this can make archived data much more useful. It s an example of how modern archiving technology can help organizations derive more value from what might otherwise be rarely used data. Allow Access by Diverse Applications Different applications need to access archived data in different ways. Accordingly, ECS exposes objects through several different protocols, as Figure 4 shows. Figure 4: ECS objects can be accessed via several different protocols. The most popular protocol choices for accessing object stores today are the interface defined by Amazon s Simple Storage Service (S3) and the Swift interface defined as part of the OpenStack project. ECS supports both of these options. It also supports protocol interfaces defined for earlier EMC archiving offerings, including the Atmos and CAS APIs. Along with these, ECS allows access to data via NFS. And since applications that analyze big data often access that data through the HDFS protocol, ECS supports this as well. This support, designed to let ECS act as a data lake, provides another example of how a modern archiving technology can help you get more business value from your archived data. 6

10 Provide Global Access to Data To give applications a common view of data from anywhere, ECS groups related buckets and objects into a namespace. A single ECS installation typically has many namespaces, providing built-in multi-tenancy. Figure 5 illustrates this idea. Figure 5: ECS buckets and the objects they contain are grouped into namespaces. This approach gives applications a single, consistent view of the data they access, regardless of where in the world they re running. It also makes it easy for multiple groups to share the same modern archiving technology, since each one gets its own distinct namespaces. Offer High Scalability Scalability has several different aspects. For example, a traditional NAS filesystem might have trouble storing billions of files in a single NFS directory. This isn t a problem with ECS; a bucket can hold billions of objects, and a single ECS installation can contain hundreds of petabytes of data. Yet storing all of this data on a single machine eventually becomes impossible; you can t buy a big enough machine. Instead, scaling effectively with modern archiving requires taking a scale-out approach, where handling more data just means deploying more machines. This is what ECS does, as Figure 6 shows. 7

11 Figure 6: ECS automatically replicates data across geographically distributed nodes. In ECS, different nodes in different locations can each store a copy of the same bucket. Changes made to any copy will be propagated to the others. This approach offers a number of benefits, including the following: Since all copies are read/write, an application commonly uses the one that can be accessed most quickly. This provides fast data access for applications regardless of their location. Because the same data is accessible on multiple nodes, requests from applications can be spread across those nodes. This lets ECS handle a higher volume of requests. Built-in data replication makes a separate disaster recovery mechanism unnecessary. ECS s distributed approach automatically keeps data available if one site goes down. Given the scale demands of modern archiving, you should be able to continually add data to your archive without worrying about whether you re reaching its limits. ECS, which has no design limits on scalability, was created with this in mind. Ensure Compliance ECS is the successor to Dell EMC Centera, an earlier archiving offering. Compliance was a primary focus of this product, and so ECS supports all of the same compliance features. These include options such as litigation hold, event-based retention, and others. For customers that choose to migrate from Centera to ECS, Dell EMC provides a migration tool, making it straightforward to move data into your modern archive. If necessary, Dell EMC also offers professional services to smooth the transition. Even though a modern approach to archiving goes well beyond compliance, ECS still provides what s necessary to meet this essential purpose. Replace Tape ECS D Series, released in 2016, provides object storage that s specifically aimed at being cost competitive with traditional tape archives. To make this possible, this ECS model provides very dense storage. This means that, unlike other ECS offerings, its disks aren t hot-swappable replacing a failed disk requires downtime. While this might not 8

12 be acceptable in some modern archiving scenarios, it s fine for tape replacement, especially considering the cost advantages it brings. Replacing tapes with ECS also eliminates the delays inherent in getting a tape mounted and read (or worse yet, retrieved from a remote location). Work with Any ISV You Choose ECS works with archiving software from a variety of vendors. This includes Dell EMC s own solutions, such as Cloud Protection Suite, as well as products from third parties. ECS also builds on the large ISV ecosystem established by Centera and Atmos. The result is that you can use this modern archiving solution with software from virtually any vendor you choose. Go Beyond Archiving Using ECS for a modern archive is the most common way that organizations begin using this product. Yet ECS is a more general technology, one that can be used in a variety of scenarios. For example, if your organization has custom applications that use large amounts of data, you might benefit from modernizing those applications to use ECS rather than traditional NAS technology. Doing this typically requires changing your applications to access data as objects (such as via S3) rather than files, so it takes some effort. But the benefits of making this change can be considerable. They include the following: Storing the application s data in ECS is likely to be less expensive than using a traditional NAS system, for all of the reasons described earlier. ECS provides a storage-as-a-service model, letting users self-provision rather than wait for a reactive response from storage administrators. ECS gives applications direct Internet access to data via HTTP. This lets applications go right to the information they need without opening a new port in your firewall, something that s not typically possible with NFS. Without ECS, building an application that effectively accesses large amounts of diverse data might require first moving that data into an object store. If the data is already archived in ECS, however, there s no need to do this the data is already where it should be. 9

13 Choosing Archiving Solution: ECS or Isilon? Both ECS and Isilon can support modern archiving. Which one should you choose? While there are no fixed rules, here are some guidelines for making the decision. Use Isilon when: You re already using Isilon for primary storage. You ll mostly be storing and accessing files via file protocols such as NFS. Your modern archive will have only one or two sites. ECS is likely to be a better choice when: You re already using Centera, EMC s earlier archiving solution. You ll be storing diverse data and so can benefit from the flexibility of object storage. Your modern archive will have three or more sites. Dell EMC offers a portfolio of technologies for modern archiving, providing good options for whatever path you choose to take. CONCLUSION Once upon a time, archiving was a graveyard for data. Maybe you were required to keep data for compliance, or maybe your users just couldn t be bothered to delete data that they no longer used. In any case, archiving was a cost you had to bear. But what if archiving could instead be an engine that leads your digital transformation? By enabling new ways to use your data, modern archiving can help your organization get better at what it does. To do this, the modern archiving technology you choose must let you do all these things: Lower your costs Simplify your datacenter Store all types of data Allow access by diverse applications Provide global access to data Offer high scalability Replace tape Work with any ISV you choose Go beyond archiving Modern archiving can be the engine for your digital transformation. ECS has an important role to play in making this happen. 10