REFERENCE ARCHITECTURE Quantum StorNext and Cloudian HyperStore

Transcription

1 REFERENCE ARCHITECTURE Quantum StorNext and Cloudian HyperStore CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 1

2 Table of Contents Introduction to Quantum StorNext 3 Introduction to Cloudian HyperStore 3 Audience 3 Solution Overview 3 Architectural Overview 4 Use Cases 11 Conclusion 13 CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 2

3 Introduction to Quantum StorNext Xcellis from Quantum is high-performance shared storage, with built-in data protection and data management capabilities. Powered by StorNext advanced data management software, Xcellis is independently scalable in performance, capacity, and connectivity to meet any need, from Mom & Pop to Fortune 50. The self-protecting nature of Xcellis eliminates the pain and expense of maintaining separate backup infrastructure, and policy-based tiering functions enable cost control and sharing, connecting your data to object storage devices, cloud destinations, and even robotic tape libraries for maximum cold storage savings. Connectivity options include NAS, SAN, and S3, enabling access to data anywhere and anyhow. For more information go to Introduction to Cloudian HyperStore Cloudian HyperStore is designed for enterprise customers and service providers in data-intensive markets where unstructured data demands double every year, straining the limits of modern data centers. Cloudian HyperStore provides exabyte scalability, up to 14 nines data durability, and powerful data management at costs as low as $0.01 per GB each month. The product offers a rich set of features including quality of service controls, multi-tenancy, and billing, as well as the highest level of S3 API compatibility to ensure plug-and-play interoperability with S3-enabled applications. Being infinitely scalable on demand allows you to seamlessly grow your storage as needed, up to exabytes across multiple data centers to meet increasing demand. S3 compatibility also offers the flexibility to be onpremises object storage as well as a hybrid tier to all the major public cloud platforms. For more information, visit the Cloudian website. Audience This reference architecture is intended to provide CTOs, solutions architects and infrastructure administrators with information about the architecture, implementation and benefits of an integrated Quantum and Cloudian solution. Cloudian HyperStore combined with Quantum StorNext offers the perfect combination of performance, reliability and scalability needed to fulfill the data storage needs for your applications while controlling infrastructure costs. Solution Overview In recent years, there has been a massive shift in the way organizations generate, store, and use the large amounts of data that they generate. In order to extract the maximum value out of this data, it needs to be stored, analyzed, and monetized not just when it is collected but also for the future. To address the challenge of massive data growth for organizations, storage architectures need to provide and manage different tiers based on frequency and performance of access, security needs, and cost considerations. With the right choice of storage architecture, organizations can not only keep their infrastructure costs down, but also capitalize on the enormous revenue opportunities that present themselves with the massive amounts of data. The common practice across the industry has been to store as much data as possible on the primary storage, and then tier it off to either digital tapes, or public clouds for long-term storage. Digital tape offers the lowest-cost solution for long-term and offline storage, but it comes with the trade-off of data access latency. A public cloud archive can provide storage for content off-site, but regularly moving a large number of data assets to and from the cloud can be very time-consuming and expensive. Organizations need a storage tier that can not only provide lower latencies, but also helps keeping infrastructure costs in control. That s where on-premise object storage comes in. It overcomes these challenges by providing lower latency compared to tapes, scales infinitely, protects against data loss, and keeps the budget in control. It also provides the flexibility to add tapes or public cloud as an additional tier for the passive archive tier. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 3

4 The certified solution of Cloudian HyperStore working with Quantum StorNext gives users a costeffective option to expand their Quantum environment with an industry-leading object store that has the largest S3 API interoperability, enabling organizations to efficiently and economically meet their evergrowing data storage needs. Quantum Xcellis is a high-performance, multi-tier shared storage platform designed for large, dataintensive workloads. It is powered by StorNext, the industry s fastest streaming file system and policydriven data management software. Employing policy-based tools, it can automatically copy or move data to Cloudian HyperStore with no scripts, programming, or manual processes. This automated data management capability gives users a single pool of high performance and high capacity accessible storage that comes with comprehensive data protection, on-the-fly scalability, and optimized primary storage utilization. Architectural Overview To address the challenges of exponential data growth and to make the end-to-end workflow more efficient, Cloudian HyperStore leverages Quantum StorNext. Customers can simply configure the HyperStore as a target for policy-based tiering from Xcellis in the StorNext GUI. After the policy is enabled, files stored in Xcellis are automatically archived and retrieved using the industry-standard S3 protocol without any operational overhead. On the front end, StorNext offers multiple connectivity options including SAN, NAS (SMB, NFS), S3, and a unique high-performance IP client known as DLC. The following steps outline the process to connect HyperStore as a tier behind Xcellis, and enable policy-based data movement to HyperStore. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 4

5 Object Storage First step in the process is to configure the HyperStore as the target object storage in the Storage Destination. 1. Put the HyperStore host address, port, HyperStore access key as the User Name, and HyperStore secret key as the Password. 2. Configure the controller name and number of max streams in the Controllers section. 3. Configure the IO path in the IO Paths section. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 5

6 4. Configure the HyperStore bucket information in the Containers section. Before you configure this section, make sure that bucket is created on the HyperStore node. StorNext supports a rich set of features for AWS S3 compatible cloud storage providers including: HTTP and HTTPS access to AWS S3 compatible buckets AWS Signature Version 2 (V2) and Version 4 (V4) AWS Standard authentication that makes use of either your AWS Identity and Access Management (IAM) Access Key Id and Secret Access Key or your user name and password. HyperStore also supports Batch Delete, so you can enable that too. NAS Share Next step in the process is to create a SMB share on StorNext file system using the following command. share add smb testdir /stornext/snfs1/testdir write list = sysadmin If the StorNext Connect administrative tool is installed, it may be used to create NAS shares using the Connect GUI. After that you can mount the share on your client machine. This could be Windows, Linux, UNIX or Mac. In this case, we have chosen Mac. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 6

7 Alternatively, for testing purposes, you can also browse the share using StorNext Storage Manager. Storage Policy Next step in the process is to configure the storage policy in StorNext to tier data from Xcellis to Cloudian HyperStore. Here, you can specify the policy details, such as file age before copy, file age before truncation, number of copies and versions, encryption, retrieve order, and associated directories. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 7

8 Cloudian HyperStore The files are copied from Xcellis to HyperStore based on the schedule defined in the storage policy. After the files are tiered from StorNext to HyperStore, you can login to HyperStore and verify that files are accessible. Additionally, the Cloudian HyperStore provides rich set of features that provide that boost interoperability, data durability, and operational efficiency with StorNext. Modular growth - HyperStore cluster consists of storage nodes that join to form a limitless cluster. You can scale capacity and performance by simply adding more nodes to your existing HyperStore cluster. Native S3 platform - HyperStore is built on native S3 platform. The highest level of S3 API compatibility ensures plug-and-play interoperability with S3-enabled applications. It also makes it easier to integrate HyperStore with all the major public cloud platforms, such as Amazon AWS, Microsoft Azure, and Google Cloud Platform. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 8

9 Multiple protection policies - Storage policies on object storage are ways of protecting data so that it is durable and highly available to users. The HyperStore system lets you pre-configure one or more storage policies. For each storage policy that you create you can choose and configure either of two data protection methods: replication or erasure coding. If your HyperStore system spans multiple data centers, for each storage policy you can also choose how data is allocated across your data centers. These storage policies protect objects from disk, node, rack, or even site failures. HyperStore also allows configuring these policies at the bucket level so that critical data can be given greater data protection without segregating it at the hardware level. Multiple deployment options - Customers can deploy HyperStore in multiple ways. It can be deployed either as on-premises storage (on a single site or across multiple sites) behind StorNext, or as distributed storage across multiple public cloud platforms. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 9

10 Bucket lifecycle policies - HyperStore also supports configuring bucket lifecycle policies so that objects are automatically moved (auto-tiered) to a different object storage system on a defined schedule, or automatically deleted (expired) on a defined schedule. The destination storage system includes Amazon AWS S3, AWS Glacier, Google cloud platform, Microsoft Azure, and any custom S3 endpoint. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 10

11 Use Cases 1. Hierarchical Storage Management - Ideal archive tier for Quantum StorNext. 2. Scalable Storage for Video Surveillance - Ideal storage solution for small and large-scale intelligent video surveillance. The rate at which the fixed CCTV cameras, body-worn cameras, and mobile cameras is growing and the amount of data that they are generating is at an all-time high. Storing this huge amount of data could strain an organization s budget. StorNext primary storage combined with high-capacity HyperStore provides high performance SMB/iSCSI protocols for ingest, policy-driven automated tiering, and metadata tagging for quick search and retrieval. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 11

12 3. Media and Entertainment - Optimized storage solution for modern media workflows. With ever increasing capturing devices, higher resolutions and shooting ratios, and the proliferation of multiple delivery formats, maintaining an efficient media workflow has become a huge challenge for customers. By combining HyperStore with a high-performance file access and management solution like StorNext, you get a complete ingest-to-archive workflow storage solution. It allows better collaboration amongst teams across geographies, and better management of assets by enabling its search and visualization using the associated metadata. 4. Defense & Intelligence End-to-end storage solution for mission information workflows With the advancements in the sensors, the geolocation devices, and the packet capture technologies, the amount of Intelligence, Surveillance, and Reconnaissance (ISR) data has grown exponentially. While these advancements provide better arsenal against cybersecurity threats, they also present the storage and archival challenges for this ever-growing data. The StorNext file system combined with HyperStore provides both performance and scalability along with shared access to this data for global collaboration. CLOUDIAN + QUANTUM REFERENCE ARCHITECTURE 12

13 5. Object Storage as Primary Site with Disaster Recovery on Tape In this case, the Cloudian HyperStore acts as the primary site. Applications can upload assets in the form of objects directly to HyperStore, or in the form of files and folders using Cloudian HyperFile. Assets are protected using either replication or erasure coding across multiple datacenters on the primary site. These assets are then replicated to another HyperStore cluster on a Disaster Recovery (DR) site using Cross Region Replication (CRR). Then, data is written to Xcellis S3 interface which writes data to the tapes for cost-effective long-term storage. Conclusion Most organizations across various sectors that have to deal with multiple petabytes of data often struggle with maintaining an efficient data pipeline and their application workflows. Proliferation of large file sizes and even larger datasets make it extremely difficult to maintain the infrastructure performance without and degradation. A storage solution built with high-performance Quantum StorNext combined with infinitely scalable Cloudian HyperStore helps organizations to overcome these challenges in a costefficient way. The set of rich features supported by both the solutions combined helps organizations to be more productive by making the workflows more efficient, and to be more innovative by modernizing their infrastructure. CLOUDIAN, INC. 177 Bovet Road, Suite 450, San Mateo, CA Tel: cloudian.com 2018 Cloudian, Inc. Cloudian, the Cloudian logo, HyperFile, and HyperStore are registered trademarks or trademarks of Cloudian, Inc. CLOUDIAN All other trademarks + QUANTUM are property of REFERENCE their respective holders. ARCHITECTURE WP-QTM