Scality RING S3 Connector Storage that Powers Digital Business

Transcription

1 Storage that Powers Digital Business

2 Table of Contents I. Introduction: Object Storage for Digital Business... 1 II. S3 Connector Overview... 2 III. S3 Connector Container Architecture... 3 A. S3 Server... 4 B. S3 Vault... 5 C. S3 Metadata... 7 D. Logger... 9 IV. S3 Connector Federated Deployment... 9 V. S3 Connector Deployment Use Cases A. On-Premises Private Storage Cloud B. Support for ISV and Custom Applications C. Geo-Distributed S3 Deployments VI. Summary Table of Figures Figure 1: Market Trending, Adoption of S3 Object Storage... 1 Figure 2: S3 Connector & RING for Secure, Scalable Storage for a Wide Range of Applications... 2 Figure 3: S3 Connector Container Architecture... 4 Figure 4: The S3 Vault Object Model with Accounts, IAM Users and Groups... 6 Figure 5: S3 Metadata with Scale-out Cluster Design... 8 Figure 6: Viewing the S3 Logger Service with Kibana... 9 Figure 7: Six (6) Server RING with S3 Connectors & Store Nodes Figure 8; Active/Active Stretched S3 Deployments Scality. All rights reserved Confidential i

3 I. Introduction: Object Storage for Digital Business Industry observers have long been following the emergence of object storage for large-scale unstructured data, reflected in major public storage cloud offerings from the likes of Amazon, Microsoft, and Google. Far and away, the clear market leader today among these clouds is the Amazon Web Services (AWS) S3 service, which since its launch in 2006 has grown dramatically with EC2 and other value-added services. Figure 1: Market Trending, Adoption of S3 Object Storage Today, the AWS S3 service is used to store trillions of user objects. Its remarkable growth is the result of such popular SaaS solutions as Netflix and Uber, which rely on it to run their key applications. In addition, over the past three years corporate enterprises have dramatically increased the use of S3 with their homegrown media, financial data, and Internet-of-Everything (IoT) applications. Many ISVs are now adopting S3 as their back-end storage API too, including CommVault Simpana and Veritas Netbackup, as well as such emerging players as Ctera. As a result, a defacto object storage API standard has in essence emerged, one that is analogous to SQL for relational databases and NFS for NAS systems. This all serves to create a positive trend for S3, thus increasing application pull for object storage. Many corporations and ISVs are seeking the flexibility that is borne of being able to deploy applications that can run both in the public cloud and in on-premises enterprise cloud solutions, a critical need that is fulfilled by the seamless and highly compatible S3 storage services made available by the Scality RING S3 Connector (known also as S3 Connector). Presently, Scality customers are deploying the RING in mission-critical scenarios, to store over 800 billion objects, and this number is growing fast. The S3 Connector leverages the S3 Server, which provides a Scality-developed open source S3- compatible API. This presents multiple clear advantages for ISVs and enterprise developers, supplying a simplified solution for S3 application development, for instance, while also ensuring the painless transition of applications to the S3 Connector (thus providing high-performance, scale-out storage services for applications that can remain portable across public and enterprise cloud environments) Scality. All rights reserved Confidential 1

4 II. S3 Connector Overview The overarching business goal for Scality s next-generation S3 Connector is to provide scalable, reliable, and highly available storage services to manage the massive volumes of unstructured data that are generated by emerging cloud applications. As the company s new S3-compatible interface to the RING, the S3 Connector succeeds the RS2 Connector, first introduced in The connector provides comprehensive cloud and enterprise security capabilities, scalable performance, advanced data storage and management functionality. For enterprises and service providers, the S3 Connector offers the most secure, scalable and portable storage services for custom and packaged cloud applications on the market today. Figure 2: S3 Connector & RING for Secure, Scalable Storage for a Wide Range of Applications 2016 Scality. All rights reserved Confidential 2

5 The S3 Connector s core proficiencies work in tandem to provide a rich set of storage services with which applications can be deployed seamlessly across public, private and hybrid cloud environments. Rich AWS and Enterprise Security S3 API Compatibility Any-to-Any Scale-Out High-Performance Buckets Geo-Distributed Capabilities Ease of Deployment Support for the full complement of AWS security services, such as multitenant accounts, Identity and Access Management (IAM) for users, groups and roles, AWS-style access keys and secret keys, the latest Signature v4 Authentication mechanism, and data encryption. Also featured is interoperability with such existing enterprise security services as LDAP and Microsoft Active Directory servers. Notwithstanding rapid AWS advancements, a high-degree of S3 API coverage is assured, including core data APIs for Bucket and Object access and Multi-Part-Upload (MPU) for efficient ingest of large objects. S3 Connector development is based on Continuous Integration (CI) and agile delivery of features when ready, which allows Scality to introduce new S3 methods shortly after their AWS publication. This functionality is provided by the S3 Server, which is supported by Scality as an open source project on Github. Applications can access any Bucket or Object from any connector, thus allowing for parallel and multi-user access to data and scaling to billions of buckets and objects. Performance can be scaled-out simply by adding more connectors. Support for low-latency response times and high throughput of reads and writes of Objects in Buckets. Also, performance is optimized for fast Bucket listing operations, including fast partial-path search for selected objects by path prefix. S3 Connector provides integrated geo-replication capabilities for storage across multiple datacenters, supporting Active/Active stretched deployments for site disaster protection with continuous data availability Delivered as a set of easy-to-deploy Docker containers, installation of the S3 Connector is simple, with zero-configuration across the customer s choice of physical, virtual or cloud environments. III. S3 Connector Container Architecture Comprising a set of distributed software-driven services, the S3 Connector offers three (3) complementary functions: S3 API (S3 Server), security (S3 Vault), and a purpose-built metadata engine that is optimized to manage Buckets at cloud-scale (S3 Metadata). These key services are delivered as Docker containers for ease of deployment Scality. All rights reserved Confidential 3

6 Figure 3: S3 Connector Container Architecture Access to S3 Server occurs via standard RESTful HTTP calls over URls, such as S3.scality.com, with all processes deployed on standard servers as Docker containers. Such Docker containers are also available for the Logger (ElasticSearch, aka ELK) and for the Ansible framework components. With the Ansible framework, a federated deployment allows the setup of all Docker containers including S3 Connector components on target servers (refer to S3 Connector Federated Deployment on page 9). A. S3 Server The central component of the S3 Connector, S3 Server supports standard S3 API command URLS, enabling SSL certificate configuration for secure HTTPS access and encryption over-thewire. It supports S3 headers, response codes and error codes. Its processes are stateless, thus applications can access any Bucket or Object resources via multiple servers for requests to ingest (PUT), or access (GET) object data, using standard load balancers. The S3 Server enables scale-out functionality across as many connectors as needed to serve application or user load requirements, even on single Buckets, along with scalable capacity per Bucket. The component provides two (2) means for accessing a Bucket as per the S3 protocol: Path Style and Virtual Hosting. Path Style Using the Bucket name as a path following the domain, in the form For example, with the configured domain as s3.scality.com and the Bucket name testbucket, using path style the URL is Virtual Hosting Using the Bucket name as a subdomain within the configured domain, in the form For example, assuming that the configured domain is s3.scality.com and that the bucket is testbucket, using virtual hosting style the URL will be Scality. All rights reserved Confidential 4

7 To support both Path Style and Virtual Hosting access, the configured domain and any subdomains driving the network traffic must be defined on the S3 machines. S3 Server supports standard S3 API operations on resources, including Buckets and Objects containing data payloads. Addressing schemes with unique key names are constructed on to of Bucket names, as previously described. Object names may include pseudo-path names separated by / delimiters, to provide a path hierarchy akin to that of a file system. The contents of a Bucket can be listed via Bucket GET API calls, including partial path listings of Objects with a common object prefix (e.g., /foo/ ) that lists all Object names starting with indicated path. A wide set of API calls are supported by S3 Server, including standard PUT, GET (including range GETs), DELETE, HEAD, and Multi-Part-Upload (MPU) for efficient, parallel ingest of large objects. The S3 Server component also supports standard and canned Bucket and Object Access Control Lists (ACLs). Advanced API support including Bucket Versioning and Bucket Lifecycle are on Scality s roadmap for the S3 Connector. Offered as an open source product (published on and downloadable from Github), developers and ISVs can download the S3 Server standalone and use it to test and develop S3 compatible applications without the necessity of incurring AWS Account and S3 storage fees. Resulting applications can thus be reliably and compatibly deployed on the S3 Connector and the RING for production use. A full description of the API set supported by the S3 Connector can be found in the Scality S3 Connector API Reference. B. S3 Vault S3 Vault provides identity management and access control security functionality for the S3 Connector. Adhering to the IAM model, the service offers a comprehensive AWS-compatible security service that is able to integrate with external enterprise directory and security services for instance, Active Directory via SAML (Security Assertion Markup Language) version 2.0. More specifically, S3 Vault is responsible for key security and access control capabilities, such as accounts, users, groups, roles, access keys and AWS-style secret keys. ACLs are implemented for arbitrating access to such underlying resources as Buckets and Objects. Thus, the implication is that S3 Vault must reconcile overlapping rules from various roles, ACLs and IAM Policies (to be enabled in a future release), and grant least restrictive access to the resources. S3 Vault supports authentication over the new AWS S3 Signature v4 mechanism as well as the legacy v2 authentication mechanism, and in 2016 follow-on releases the service will also support access control through IAM Policies and data encryption via the S3 SSE-C (Server Side Encryption, with customer-provided Key Management Services). The component was designed to parallel AWS S3 concepts, including: 2016 Scality. All rights reserved Confidential 5

8 Accounts Top logical billing entity associated with multiple tenants (applications or customers); can contain IAM users IAM users Associated with one or more roles (to provide credentials via ACLs); contained in accounts, and can belong to groups Access key IDs and Secret access keys Assigned to accounts or IAM users Authentication Performed through external security services (e.g., Active Directory) over SAML 2.0 Figure 4: The S3 Vault Object Model with Accounts, IAM Users and Groups Two (2) interfaces for managing accounts and users are provided by S3 Vault: a Command Line Interface (CLI) with interactive or scriptable commands, and a documented REST API. Both interfaces provide commands for creating, updating and deleting security objects, including the Account access key IDs and secret access keys. Vault administration itself is authorized via Super-Administrator credentials only, thus ensuring that access to sensitive Account, User and Key management tasks is restricted to only the most privileged administrator. S3 Vault echoes the importance given to identity by AWS S3 with regard to access control, providing authentication of the accessing identity via one of the two AWS Signature schemes Signature v4 or Signature v2 as dictated by the API headers. Requests are allowed or denied in part based on the identity of the requester. For example, the right to create a new Bucket in an account is reserved for registered users (by default), and the right to create Objects in a Bucket is reserved for the owner of the Bucket in question. The Signature v4 and v2 authentication schemes are both based on keyed-hmac (Hash Message Authentication Code) mechanisms. To authenticate a request, the API call must first concatenate selected elements of any API request to form a string, and then use the Account (or User s) secret access key to calculate the HMAC of that string. The HMAC algorithm output termed the signature is added as a parameter of the API request as the final step in signing the request. (The v4 authentication scheme requires an additional header in each API call, and in addition the HMAC is computed on the headers and object payload being passed. This differs from the v2 scheme, which did not require the HMAC signature to include the object payload.) 2016 Scality. All rights reserved Confidential 6

9 Upon receiving an authenticated request, S3 Vault retrieves the AWS secret access key indicated from either the Vault database, or from an external security service via SAML and uses it to compute a signature for the message it received in the same manner as with request authentication. The service next compares the signature it calculated against the signature presented by the requester. If the two signatures match, the system concludes that the requester must have access to the AWS secret access key. If, however, S3 Vault performs the calculations and the two signatures do not match, the request is denied and the system responds with an error message. Initially, S3 Vault implements access control through AWS ACLs on Buckets and Objects, to provide control over basic access privileges to these resources for PUT, GET and DELETE operations. This includes support for standard and canned ACLs, with standard ACLs defining READ, WRITE, READ_ACP, WRITE_ACP, FULL_CONTROL permissions on the underlying resources, whereas canned ACLs serve to define more advanced permissions (e.g., private, public-read, public-read-write, authenticated-read, bucket-owner-read, bucket-owner-full-control, log-delivery-write). In a 2016 follow-on release, S3 Vault will also enable AWS IAM User and Group Policies with the service resolving access control by way of a most restrictive rule among Policies and ACLs, to ensure the highest security. The S3 Connector employs SAML 2.0 as a standards-based XML protocol designed to facilitate the exchange of AAA -related data (authentication, access control, and accounting) between different security domains. SAML is uniquely capable of establishing cross-domain Single-Sign-On (SSO) behavior between S3 Vault and other security services. C. S3 Metadata A distributed metadata database service, S3 Metadata stores system metadata in an ultrareliable, scalable and high-performance manner (e.g., data related to user s Buckets and Object keys, as well as such S3 Vault security-related items as accounts, users and ACLs). Perhaps the most technically challenging aspect of distributed storage solutions is highly scalable metadata management. To that end, Scality invested in a decade s worth of leading distributed systems research both internally, and in collaboration with leading research institutions (e.g., INRIA 1 ) to develop a state-of-the-art Bucket indexing and object metadata engine for the S3 Connector. As a result of the company s diligence, this engine is able to offer a convergence of scalability, resiliency to failures, and high-performance access for high numbers of concurrent requests, resulting in an engine that as the cornerstone of the S3 Metadata service enables the S3 Connector to support extremely large-scale deployments involving millions of Buckets each containing millions of Objects. The engine also serves as the foundational technology for scale-out access and performance support. It is also a key building block in the advanced versioning, lifecycle, and geographically-distributed multiple datacenter deployments planned for future releases. 1 INRIA and Scality distributed systems research 2016 Scality. All rights reserved Confidential 7

10 In delivering its service, S3 Metadata uses a highly available leader/quorum clustered database design based on the Raft 2 distributed consensus algorithm ( with each cluster member managing its own copy of the metadata in a key/value database. Each S3 Bucket is mapped to a database, complete with its own cluster of database server daemons in a Raft session. The underlying key/value database provides the desired properties of fast insertion, scalability to millions of entries (Objects per Bucket), and fast listing operations to present the contents of a Bucket. All of which assures high performance access for applications performing Bucket operations with high rates of PUT, GET and DELETE requests. Figure 5: S3 Metadata with Scale-out Cluster Design As depicted in Figure 5, the key elemental building block is a logical node for storing metadata called a Repd. Multiple Repds form a replication group to tolerate the failure of a minority of Repds, with a consensus protocol in place to achieve consistency. DBAPI is responsible for partitioning and acts as a replication mechanism for sessions and Repds transparent to higherlevel applications (e.g., S3 Connector). The design assigns a set of Repds to each S3 Bucket, and by selecting different sets of Repds from a pool (across separate hosts) for each Bucket, S3 Metadata ensures scalability as the number of Buckets is scaled-out. This also provides highavailability, by allowing full service access to the Bucket as long as a majority of the Repds comprising the session are available. The algorithm is adaptive to failures, including the leader, and provides the ability to dynamically select a new leader of a cluster. To ensure fast response times, the key/value databases underlying the Repds should be stored on flash (SSD) devices. This ensures very low-latency access for Bucket writes, reads, and listing operations on the S3 metadata service. Scality customer service can provide sizing recommendations for CPU and memory, as well as utilities and guidelines covering the SSD requirements for a given number of keys (objects) and the average size of objects expected in the system. 2 Raft distributed Consensus algorithm 2016 Scality. All rights reserved Confidential 8

11 D. Logger S3 Connector offers a logging function that captures and presents log data generated by S3 Connector service events. The Logger channels the comprehensive service logs in a JSONbased format. S3 Vault events are captured in the S3 Connector logs (e.g., account creation, user creation), as are such API interactions as Bucket and Object operations. Viewing and searching the JSONbased logs is easily accomplished via the ElasticSearch tool, which is based on the well-known ELK 3 suite. Figure 6: Viewing the S3 Logger Service with Kibana ELK components can also be included and installed with the S3 Docker containers as part of the standard deployment process. ElasticSearch enables log searching for auditing purposes, and the Kibana tool enables visualization of data through graphs. This can also facilitate performance optimization, as well as drill-down for diagnosis and troubleshooting. A separate tool is also included for converting the JSON logs into a more easily read text format. IV. S3 Connector Federated Deployment The data storage layer for the S3 Connector is supplied by Scality RING. Independently scalable and eminently reliable, the S3 Connector accesses the RING through a native REST interface that is incorporated into the S3 Connector containers. This interface is deployed as part of the federated installation process, providing a simple, high-performance key/value store for access to the powerful data storage mechanisms of Scality RING. Data flows from standard load balancers to one of the S3 Server processes. Each S3 API command is parsed and processed by the S3 Server, which interacts with S3 Vault to 3 ELK suite (ElasticSearch, Kibana and Logstash) 2016 Scality. All rights reserved Confidential 9

12 authenticate the request and ensure that access is granted. Next, the S3 Server contacts the S3 Metadata service to locate or update the requested Bucket and Object key, and once the key is secured the S3 Server sends the data to the RING by passing the key/value pair to the native REST interface for the corresponding read or write action. The S3 Connector s native REST interface is responsible for sending the data to the RING s store nodes over the RING s erasure coding or variable Class-of-Service (CoS) replication mechanisms for high durability data protection. Also, all S3 API, S3 Vault, and S3 Metadata actions are registered to the Logger, thus creating a complete trail of the data and administrative commands associated with the request. For simplicity, the S3 Connectors are deployed uniformly on all RING servers in the physical machine OS or in a VM, as illustrated in Figure 8 in the case of a six (6) server RING. This delivers a simplified uniform (homogeneous) deployment model, both on the initial installation and for new servers added to the RING when scaling out capacity. The S3 Connector optimizes interactions between services to reduce latency in the local (single site) case, as well as to the greatest extent possible in multiple datacenter deployments. S3 container deployment is centralized through Ansible framework scripts, running from a central deployment machine. Ansible does not rely on agents deployed on each of the target hosts, and instead leverages the popular SSH tool for remote access to the target hosts. A set of Ansible federation scripts automates the deployment process, including the installation of Docker components from the deployment machine onto all target hosts. This federated deployment scheme allows for a simple, centralized command line installation of all distributed S3 Connector containers across all RING servers. Figure 7: Six (6) Server RING with S3 Connectors & Store Nodes The standard RING storage processes including storage nodes and IO Daemons are installed as in a typical RING deployment Scality. All rights reserved Confidential 10

13 For a comprehensive description of federated deployment, please refer to the S3 Connector Installation Guide. V. S3 Connector Deployment Use Cases The S3 Connector sustains a wide range of deployment scenarios, supporting the growing hybrid model of public cloud (enabling service providers to create new cloud storage capabilities) and on-premises private enterprise clouds. In addition to the security, performance, and compliance advantages of private cloud deployments, leading industry-analysts state 4 that the use of private clouds at large scale provides a TCO advantage over solely public cloud solutions. In any scenario the need for geo-distributed multi-datacenter deployments is key to supporting access from distributed applications, and to providing site failure protection. A. On-Premises Private Storage Cloud For many enterprises, the deployment of applications in the public cloud advantageous from both a flexibility and economic perspective. Even so, many types of applications require the control, security, and performance advantages of an on-premises enterprise private cloud deployment. Financial Services and Healthcare applications good examples, as the maintaining of large-scale archives of records and compliance data is typically precluded from public cloud storage due to specialized locality and security requirements. The S3 Connector provides a solution for enterprises that are developing new cloud applications designed to use the AWS model and the S3 API for storing large unstructured data. Applications developed and deployed in the AWS public cloud will function without any code changes required in a private cloud environment on the Scality RING, with the S3 Connector providing a totally compatible storage interface. Enterprise multitenancy deployments map well to the AWS security model with accounts and IAM users. Internal departments, applications or workgroups can be assigned their own accounts, users and groups, capable of functioning seamlessly in the on-premises deployment model. The interoperability of the S3 Connector with enterprise directory services (e.g., Active Directory) provides a solution for leveraging existing enterprise assets, rather than forcing a migration to a new proprietary model. B. Support for ISV and Custom Applications Hundreds of applications now leverage the S3 API as the standard protocol for accessing object storage. Many vertical industry ISVs are now delivering products that are S3-compatible, including key business applications for backup and archiving (Veritas, CommVault, IBM Spectrum Protect, ASG/Atempo, QStar, NetApp Altavault), digital media asset management (CatDV), syncand-share (Ctera), and content delivery (Aspera). For storage administrators, this trend virtually ensures that an S3-compatible, on-premises storage solution will be required to support business 4 BTIG Equity Research, April 2016 The Decline and Fall of Enterprise Storage 2016 Scality. All rights reserved Confidential 11

14 processes and workflows in the near-term future. Scality supports application validation through the open source S3 Server, which provides a simplified single Container image. This enables ISVs to develop, test, and certify the functionality of their applications for deployment on the S3 Connector and the Scality RING. The S3 Connector also directly supports custom application development via S3 REST API commands, or through the AWS SDKs for Java, JavaScript, PHP,.NET and many other languages. As such, large enterprises can safely standardize custom applications generating unstructured data for S3 compatibility, without being tied into a specific backend infrastructure. As a side benefit, the application development process itself generates large amounts of unstructured data in the form of images, droplets, etc.). The S3 Connector fully adheres to the S3 specification to provide portability of applications across public and private cloud deployments. The S3 Vault security model maps to the AWS security model for accounts, IAM users and groups, to provide full deployment compatibility and portability. C. Geo-Distributed S3 Deployments The S3 Connector s any-to-any scale-out capabilities extend to geographically-distributed deployments across multiple datacenters (DCs). The S3 Connector supports an Active/Active stretched model for deploying both the data RING and S3 services across two or more DCs connected via metro area networks, such as between San Francisco to San Jose, or between Paris and London. The S3 Metadata service implements the concept of a leader as referred to in S3 Metadata on page 7 and the term Master DC is used to describe the datacenter in which the currentlyassigned leader for the S3 Metadata service resides. In this deployment model, the S3 services comprised of the S3 Servers, S3 Vault and S3 Metadata containers are distributed across DCs, on the RING machines. The data is stored using the Scality RING model stretched across two or more DCs hosted on the same machines as the S3 services. Full any-to-any capability is provided (hence active/active access), and thus any S3 Connector in either DC can access any S3 Bucket resource. The system automatically manages metadata updates to the S3 Metadata service from the connectors, with synchronous updates made to the data RING across DCs for consistency. In some cases, however, the connectors in non-master DCs will incur a remote access penalty while updating metadata across the metro area network. In the event of a network failure or a DC outage, either writes are preserved on the Master DC (when one of the non-master DCs is rendered inaccessible), or read-only access is preserved (should the Master DC become inaccessible). An administrative command allows for switching a non-master DCs role to that of Master DC, which enables read/write access to the surviving DC following inaccessibility or failure of the original Master DC Scality. All rights reserved Confidential 12

15 Figure 8: Active/Active Stretched S3 Deployments As in the case of a typical stretched RING deployment, data updates are written synchronously to all nodes in the RING. This ensures that data is fully protected at the prescribed durability level across DCs, providing full resiliency and consistency. VI. Summary The Scality RING S3 Connector offers a modern and highly compatible S3 API server for Scality RING that provides a state-of-the-art implementation of AWS-compatible security services (e.g., IAM), as well as integration with enterprise-class security servers such as LDAP and Active Directory. In addition, the inclusion of Scale-Out, Geo-Replication, and Lifecycle Management functions provide a powerful solution for customers seeking an in-house S3 storage solution with petabyte-scale capabilities. The S3 Connector enables enterprise-caliber on-premises S3 storage solutions, as well as future enablement of hybrid cloud integration through Bucket Lifecycle support. About Scality Scality powers digital business. The industry leader in software-defined storage, Scality serves over 500 million end-customers worldwide, with over 800 billion objects in production. The Scality RING supports any file and object application, on any hardware, with unlimited capacity and guaranteed reliability. The RING enables organizations to offer always-on public or private services, connecting with applications both old and new, to drive competitiveness and heighten customer satisfaction. Able to seamlessly scale storage to match business growth, the RING runs on all industry standard x86 server platforms (such as those offered by HPE, Dell, Cisco, Seagate, or even mixed form factors, without the necessity for forklift upgrades. With the RING, Scality customers can expect a TCO reduction of up to 90% versus legacy storage. Scality is headquartered in San Francisco, with offices throughout the world. Try the Scality RING online at Scality. All rights reserved Confidential 13