STORAGE MADE EASY ENTERPRISE FILE FABRIC

Transcription

1 SOLUTION GUIDE STORAGE MADE EASY ENTERPRISE FILE FABRIC IBM COS ABOUT STORAGE MADE EASY ENTERPRISE FILE FABRIC The Storage Made Easy File Fabric enables IT to regain control of "cloud data sprawl" by unifying on-premises and cloud storage into a single, easily managed, converged infrastructure with policy-based controls for governance, audit and security. Organizations use the File Fabric to enable workgroup collaboration, file sharing and synchronization through a full suite of desktop, mobile and web applications. The solution supports software-defined storage (SDS) particularly well, enabling end users and legacy applications to interact with object storage through the File Fabric as if it was a regular file system. Businesses are able to monitor, secure and audit company and employee data silos be they be on premise, on the cloud, in file systems, in object storage or in applications. INTRODUCTION This document outlines the solution architecture, configuration steps, functionality and performance characteristics of Storage Made Easy Enterprise File Fabric software with IBM Cloud Object Storage (COS). A separate guide provides information on standalone desktop applications S3 Drive and S3 Explorer. PRODUCTS TESTED Storage Made Easy software: Enterprise File Fabric IBM software: IBM Cloud Object Storage On Premise (version ) IBM Cloud Object Storage US South (version ) IBM Cloud Object Storage US Region (version ) /StorageMadeEasy /company/storage-made-easy The certification exercise included installing products and executing functional and performance tests with IBM Cloud Object Storage - on premise and public cloud. The File Fabric product is shipped as a virtual appliance and was provided as an ova image. Solution Guide Storage Made Easy Enterprise File Fabric 1!

2 SOLUTION ARCHITECTURE HARDWARE Client Load Generators 2 Ubuntu Linux Servers Ubuntu Server VM Specification RAM: 16GB CPUs: 8 HDD: 500GB (SSD) Test data ranging from 64KB to 10GB Load Balancer HaProxy on CentOS 7.2 Standard Configuration VM Specification RAM: 16GB CPUs: 8 HDD: 16GB Solution Guide Storage Made Easy Enterprise File Fabric!2

3 HARDWARE Enterprise File Fabric Services 2 - CentOS 7.2 Linux Virtual Machines SME standard OVA images for VMware Specifications RAM: 12GB CPUs: 8 HDD: 2x 36GB(SSD) OS & Scratch Drive Enterprise File Fabric Metadata Server CentOS 7.2 Linux Virtual Machine MariaDB 5.5 SME standard OVA image for VMware Specifications RAM: 16GB CPUs: 8 HDD: 2x 36GB(SSD) OS & Database Partition IBM CLOUD OBJECT STORAGE Fabric was tested against the following COS storage backends: IBM Cloud Object Storage On Premise (version ) IBM Cloud Object Storage US South (version ) IBM Cloud Object Storage US Region (version ) ENTERPRISE FILE FABRIC CONFIGURATION The Enterprise File Fabric platform is a multi-tenant system allowing for thousands of users across multiple organizations. Testing was of a single organization configured exclusively with IBM COS for user data. Solution Guide Storage Made Easy Enterprise File Fabric!3

4 FILE FABRIC CONFIGURATION ORGANIZATION ACCOUNT CREATION Log in as the Enterprise File Fabric Administrator, select from Menu, Add a User Organizations are Users with an Organization Package specified Solution Guide Storage Made Easy Enterprise File Fabric!4

5 FILE FABRIC CONFIGURATION ADDING IBM COS ON PREMISE OR CLOUD OFFERING AS STORAGE PROVIDER The first time an organization logs into an account, a wizard guides the addition of storage providers approved in the Organization s Cloud Package. The Enterprise File Fabric supports over 60 backend providers. IBM COS endpoint, Access & Secret Keys are entered: Solution Guide Storage Made Easy Enterprise File Fabric!5

6 FILE FABRIC CONFIGURATION Existing buckets/vaults can be indexed by adding now or later. The radio button designates the default vault. A new vault can also be created by using the radio button and filling in the name of the vault at the bottom of the menu. Solution Guide Storage Made Easy Enterprise File Fabric!6

7 FILE FABRIC CONFIGURATION File Fabric will then index existing objects and metadata: Storage is then available for users: Solution Guide Storage Made Easy Enterprise File Fabric!7

8 FILE FABRIC CONFIGURATION CREATING ACCOUNTS FOR LOAD TESTING Storage Made Easy Enterprise File Fabric supports LDAP, Active Directory, SAML, and local user accounts. Local accounts can be created through API, however for testing Active Directory was utilized. Authorization Systems can be added under Organization -> Auth Systems. Active Directory can be configured against a local AD system or remotely via the SME ADproxy. Once added, users can be imported under Organization -> Users. Solution Guide Storage Made Easy Enterprise File Fabric!8

9 FILE FABRIC CONFIGURATION Importing of users: Users and Groups/Roles auto-registered or filtered for import based on name or group membership. Solution Guide Storage Made Easy Enterprise File Fabric!9

10 PERFORMANCE TEST 1-10,000 USER OFFICE ORGANIZATION This test simulated a large typical SME organization. The organization has 10,000 registered users with typical file size falling between 1MB-4MB and 2-4 opens/saves per hour. In order to meet stated requirements, the Enterprise File Fabric will need to respond to between 20,000 and 40,000 requests in an hour. TESTING PARAMETERS Cycle between 1, 2, 4, 8, 16 and 32 open connections File sizes between 256KB & 64MB 256KB 62,000 Reads/Writes - 35 % of Operations 1MB 62,000 Reads/Writes - 35 % of Operations 4MB 31,000 Reads/Writes - 18 % of Operations 16MB 13,000 Reads/Writes - 7 % of Operations 64MB 10,000 Reads/Writes - 5 % of Operations OFFICE RESULTS SET Tested configuration was able to exceed the requirements of a 10,000 user organization. The configuration as tested was suitable for an organization of over 20,000 users. Peak uploads & download of files in the 1-4MB range exceeded 60,000 operations in one hour. Based on the result data, the number of concurrent requests on the File Fabric nodes would be in the range of 8-16 requests for a 10,000 user organization. Bandwidth usage for 4MB files peaked around 75MB/s, allowing for use of 1Gb networking between instances, clients and storage. However, 10Gb networking would be preferred for larger organizations or organizations with a larger average file size. All testing was conducted with encryption both In-Flight and At-Rest. In-Flight encryption was enabled via TLS1.2 using an AES-256 Cipher, and was utilized at all data hops: Client to Load Balancer Load Balancer to File Fabric File Fabric to IBM COS Storage Encryption did add some latency to testing and increased CPU demands. CPU utilization was below 90% on all download tests and mixed workload testing; however, 100% write tests with 32 concurrency did exceed 90% CPU utilization. Since the Enterprise File Fabric has a scale-out web component, accommodating larger organizations is a simple matter of adding more instances. Below are charts for 100% Read and 100% Write testing indicating Operations per Hour and Bandwidth usage in Megabytes per second. Solution Guide Storage Made Easy Enterprise File Fabric!10

11 PERFORMANCE TEST 1-10,000 USER OFFICE ORGANIZATION Solution Guide Storage Made Easy Enterprise File Fabric!11

12 PERFORMANCE TEST 1-10,000 USER OFFICE ORGANIZATION Solution Guide Storage Made Easy Enterprise File Fabric!12

13 PERFORMANCE TEST 2 - MEDIA COMPANIES Some organizations have larger datasets than standard office workers. These organizations may have very large video files, healthcare images, or other atypical workloads. These larger files take advantage of Multipart Uploads when used in conjunction with IBM COS. MPU is an important feature of the S3 API as it allows larger files to avoid being chunked in a proprietary format that could render them unusable to other data accessors. While the Enterprise File Fabric can chunk data if MPU is not enabled, it is preferred use the native functions of IBM COS where available such as: Multipart Uploads (MPU) SecureSlice Server Side Encryption (SSE-C) TESTING PARAMETERS Cycle between 1, 2, 4, 8, 16 and 32 open connections File sizes over 16MB S3 Native MPU Chunking at 100MB increments for files over 105MB M-Stream File Transfer Acceleration, a capability of the Enterprise File Fabric, platform, splits files into parts that are streamed in parallel from one storage server to another. It can dramatically speed up uploads, downloads and server-to-server copying of objects. MEDIA FILES RESULTS SET During testing of larger media files, the client systems became the bottleneck in sending or receiving data. Each workload driver was able to push close to 250MB/s before the workload generator utilized all CPU time. Further testing with the TLS Encryption disabled showed increased network performance with large files. All testing was conducted with encryption both In-Flight and At-Rest. In-Flight encryption was enabled via TLS 1.2 using an AES-256 Cipher, and was utilized at all data hops: Client to Load Balancer Load Balancer to File Fabric Enterprise File Fabric to IBM COS Storage To bypass the client bottleneck, a transfer of a single 4 GB file was also made from IBM COS cloud (US South) to IBM COS on-premises. With M-Stream active 4 threads saturated the 1 GB connection to the Internet - an optimum transfer. Encryption did add some latency to testing and increased CPU demands. CPU utilization was below 90% on all large media tests with the bottleneck coming from the client nodes and not the storage or File Fabric. Testing files above 2GB did not show any significant change in bandwidth vs 2GB. As such, larger files will take more time to transfer and will move at the rates indicated for 2GB. Solution Guide Storage Made Easy Enterprise File Fabric!13

14 PERFORMANCE TEST 2 - LARGE MEDIA COMPANIES Solution Guide Storage Made Easy Enterprise File Fabric!14

15 CONCLUSION The performance characteristics of the Storage Made Easy Enterprise File Fabric are largely dependent on the backend storage provider. Lackluster performance arising from lack of compatibility with the Amazon S3 API, to inability to fulfill concurrent requests often arise when the backend storage is not a mature offering. IBM Cloud Object Storage excelled when configured as a backend provider for Storage Made Easy Enterprise File Fabric. During testing all S3 API based features of the Enterprise File Fabric were enabled and tested satisfactorily against On-Premises and Cloud based COS endpoints. Advanced S3 API features such as multipart upload (MPU) provide native storage of files as objects without the need for proprietary chunking. Ranged read requests allow for enhanced performance during file downloads when used with large objects. IBM COS additionally supports offloading of Data-at-Rest encryption from the File Fabric via S3 API SSE-C or SecureSlice enablements on vaults. Enterprises choose to use IBM Cloud Object Storage because of its enhanced durability, availability and cost per terabyte vs traditional storage. As such, IBM COS is an excellent pairing choice for enterprises with as little as 25 users to over 20,000 users with the Storage Made Easy Enterprise File Fabric. The File Fabric removes all complexity from using object based storage and the associated APIs. Users enjoy simple access to storage on any device in the office or when working remotely. Administrator s have peace of mind from the enhanced governance and compliance features of the File Fabric and the reliability and security of IBM COS based storage. Solution Guide Storage Made Easy Enterprise File Fabric!15

16 ABOUT STORAGE MADE EASY STORAGE MADE EASY HEAD OFFICE 1 Mulgrave Chambers Mulgrave Road. Sutton, London SM2 6LE UK The Storage Made Easy Enterprise File Fabric enables IT to regain control of "cloud data sprawl" to unify on-premises and on-cloud files into a single easily managed converged infrastructure with policy-based controls for governance, audit and security. The File Fabric solution provides a blanket enterprises wrap around all their data: on premises, within public and private clouds, as well as on third-party software vendor clouds (SharePoint Online and Salesforce for example). Customers use the SME File Fabric product to unify corporate data to enable workgroup collaboration with universal controls for security, encryption, audit, and data access. Companies will have access to a greater understanding of where and how sensitive data is stored, transferred, and accessible with particular focus on how companies are able to police and audit data to ensure enforcement of the General Data Protection Regulations (GDPR) in addition to other regional data legislation. USA UK +44(0) EU sales@storagemadeeasy.com Free hosted and enterprise free trial available from Storage Made Easy is the trading name of Vehera LTD. reg no: /StorageMadeEasy /company/storage-made-easy Solution Guide Storage Made Easy File Fabric!16