STORAGE MADE EASY: S3 DRIVE & S3 EXPLORER

Transcription

1 SOLUTION GUIDE STORAGE MADE EASY: S3 DRIVE & S3 EXPLORER IBM COS ABOUT STORAGE MADE EASY FILE FABRIC The Storage Made Easy File Fabric enables IT to regain control of "cloud data sprawl" by unifying on-premises and cloud storage into a single, easily managed, converged infrastructure with policy-based controls for governance, audit and security. Organizations use the File Fabric to enable workgroup collaboration, file sharing and synchronization through a full suite of desktop, mobile and web applications. The solution supports software-defined storage (SDS) particularly well, enabling end users and legacy applications to interact with object storage through the File Fabric as if it was a regular file system. Businesses are able to monitor, secure and audit company and employee data silos be they be on premise, on the cloud, in file systems, in object storage or in applications. INTRODUCTION This document outlines the solution architecture, configuration, functionality and performance characteristics of Storage Made Easy (SME) direct apps with IBM Cloud Object Storage (COS). The direct or standalone desktop applications, S3 Drive and S3 Explorer, work as independent desktop clients directly connecting IBM Cloud Object Storage. Configuration information was verified and performance information collected during a certification process at the IBM Cloud Object Storage Alliance Integration Team certification lab. A separate guide provides information on the full stack Storage Made Easy Enterprise File Fabric product. PRODUCTS TESTED Storage Made Easy software: Storage Made Easy S3 Drive Storage Made Easy S3 Explorer IBM software: IBM Cloud Object Storage On Premise (version ) IBM Cloud Object Storage US South (version ) IBM Cloud Object Storage US Region (version ) S3 DRIVES The S3 Drive enables users to mount and interact with IBM Cloud Object Storage as a local disk. Users can browse and open remote files with desktop applications including the Microsoft Office Suite. The Windows File Explorer can be used to copy, move and rename objects as if they were files and folders. The S3 Drive can also be accessed through Windows batch scripts as if it was a regular local drive. S3 /StorageMadeEasy /company/storage-made-easy The S3 Explorer desktop application allows users to interact with IBM Cloud Object Storage through a familiar file explorer like interface. From the menu, or using drag and drop, users can easily upload, download and delete files and folders. Copying, moving and renaming objects and folders is also supported. Objects can be quickly located through search, and sensitive data automatically encrypted on upload. Solution Guide - Storage Made Easy 1!

2 CERTIFICATION The certification exercise included installing products and executing functional and performance tests with IBM Cloud Object Storage - on premise and public cloud. Functional tests centered around uploading, downloading, copying and deleting files and folders. App specific features such as rename and searching were also tested. Performance testing focused on throughput supported by a single app simulating heavy usage by a single end user. Results were compared between on premise and cloud instances of IBM Cloud Object Storage. SOLUTION ARCHITECTURE The S3 Drive and S3 Explorer were deployed to a Windows 10 virtual machine. Other than endpoints, configuration for the different IBM COS instances was the same. IBM COS Public Cloud For testing with IBM COS in the public cloud over the Internet the apps were configured for an S3 compatible endpoint over HTTPS. IBM COS On-Premises An on premise deployment of IBM COS offers multiple endpoints for high availability. To allow the apps to use both endpoints a load balancer was used. An Nginx load balancer was used during certification but any open source or commercial load balancer (supporting HTTPS) would be appropriate. Solution Guide - Storage Made Easy!2

3 DEPLOYMENT Client The S3 Drive application was installed on two client machines: O/S: Microsoft Windows 10 Pro Standard Install RAM: 16GB CPU: 8 HDD: 500GB (SSD) Other software installed: Microsoft Office used for functional testing Fiddler 4 - used for identifying HTTP calls to IBM COS and for timing tests Python - used for scripting operating system calls Boto 3 - used for scripting S3 calls in Python Load Balancer An NGINX load balancer was used for IM COS On-Premise. STORAGE MADE EASY S3 DRIVE The S3 Drive enables users to mount and interact with IBM Cloud Object Storage as a local disk. For configuration information see Appendix A. FUNCTIONAL TESTING The objective of S3 Drive functional testing was to: Verify compatibility with IBM Cloud Object Storage for both on premise and public cloud storage. Map S3 Drive user interactions to one or more IBM COS HTTP operations for capacity planning and budgeting. Validate and document any recommendations for optimum performance. The S3 Drive tests were performed manually through the Windows 10 File Explorer and with Microsoft Office applications. The S3 Drive Cache was cleared before each test (Settings > Browse > Delete all). S3 Drive Operations Create Root Folder creates bucket Delete Root Folder removes bucket Browse root-level folders list all buckets Create Subfolder Remove Subfolder - hierarchy (with files) Upload File Download File Word - Save new document Word - Edit document Browse Folder Upload Folder Download Folder Copy folder Rename File Rename Folder Refresh (access from tray icon) Excel - Save new document Excel - Edit document Testing was successful against IBM COS - both on premise and public cloud storage. Solution Guide - Storage Made Easy!3

4 STORAGE MADE EASY S3 DRIVE PERFORMANCE TESTING The objective of S3 Drive performance testing was to: Calculate throughput to assist with recommendations of appropriate use cases Validate and document any recommendations for optimum performance. A single workload was creating consisting of 161 in 17 folders, ranging in size from 1MiB to 1GiB. Total workload size was 10.4 GiB (11,161,589,120 bytes). The S3 Drive Cache was cleared before each test (Settings > Browse > Delete all). Transfer and *Effective Transfer rates for S3 Drive S3 Drive Mixed Workload MiB/sec Test On Prem US South US Region Upload Download Rename* Delete* 2, * Effective speed (no data transferred) Rename and Delete operations were executed without data transfer between the S3 Explorer and IBM COS. Observations: Uploads are multi-threaded. Other operations are single-threaded. Solution Guide - Storage Made Easy!4

5 STORAGE MADE EASY S3 EXPLORER The S3 Explorer desktop application allows users to interact with IBM Cloud Object Storage through a familiar file explorer like interface. For configuration information see Appendix B. FUNCTIONAL TESTING The objective of S3 Explorer functional testing was to: Verify compatibility with IBM Cloud Object Storage for both on premise and public cloud storage. Map S3 Explorer user interactions to one or more IBM COS HTTP operations for capacity planning and budgeting. Validate and document any recommendations for optimum performance. Features Create Root Folder - creates bucket Delete Root Folder removes bucket View Root items (buckets) Create Subfolder Remove Subfolder - file hierarchy Upload File Download File Browse Folder Upload Folder Download Folder - file hierarchy Copy file Copy folder Move file Move folder Rename File Rename Folder Bucket Search Sort files (name, modified, size) Properties (root folder or file only) Testing was successful against IBM COS - both on premise and public cloud storage. Solution Guide - Storage Made Easy!5

6 STORAGE MADE EASY S3 EXPLORER PERFORMANCE TESTING 161 files, 17 folders, ranging in size from 1MB to 1GB. Total 10.4 GiB (11,161,589,120 bytes). S3 Explorer Mixed Workload MiB/sec MiB/sec On Prem US South US Region Upload Download Copy* Rename* Delete* 2, * Effective speed (no data transferred) Transfer and *Effective Transfer rates for S3 Explorer Copy, Rename and Delete operations were executed without data transfer between the S3 Explorer and IBM COS. Solution Guide - Storage Made Easy!6

7 APPENDIX A - S3 DRIVE CONFIGURATION To connect to cloud storage the S3 Drive requires: Endpoint - HTTPS was used for testing. HTTP is also supported. Access Key Secret Key Default installation settings were used: Solution Guide - Storage Made Easy!7

8 APPENDIX B - S3 EXPLORER CONFIGURATION To connect to cloud storage the S3 Drive requires: Endpoint - HTTPS was used for testing. HTTP also supported. Access Key Secret Key All other settings were defaults after installation. Solution Guide - Storage Made Easy!8

9 APPENDIX C MAPPING OF S3 DRIVE FUNCTIONS TO IBM COS OPERATIONS S3 Drive Operation IBM COS Operations Create Root Folder creates bucket Delete Root Folder removes bucket (bucket) (contents of bucket to possibly delete) (bucket) Browse root-level folders list all buckets Create Subfolder Remove Subfolder - hierarchy (with files) For each subfolder: /<subfolder>/ /<subfolder> /<bucket> Upload File Download File Browse Folder Upload Folder Download Folder Copy folder Rename File folder folder (x-amz-copy-source) Rename Folder Refresh (access from tray icon) (x-amz-copy-source) Solution Guide - Storage Made Easy!9

10 APPENDIX C MAPPING OF S3 DRIVE FUNCTIONS TO IBM COS OPERATIONS Feature Word - Save new document Word - Edit document Excel - Save new document Excel - Edit document Notes - if in cache, is it out of date? - if not in cache, or cache out of date - if in cache, is it out of date? - if not in cache, or cache out of date APPENDIX D MAPPING OF S3 EXPLORER FUNCTIONS TO IBM COS OPERATIONS Feature Create Root Folder - creates bucket Delete Root Folder removes bucket View Root items (buckets) Create Subfolder Remove Subfolder - file hierarchy Upload File Download File Browse Folder Upload Folder Download Folder - file hierarchy Copy file Copy folder Move file Move folder Rename File Rename Folder Bucket Search Sort files (name, modified, size) Properties (root folder or file only) Notes (bucket) (contents of bucket to possibly delete) (bucket) / (refresh explorer listing) /<bucket> (refresh parent folder) (contents cached for period of time) (refresh parent) (x-amz-copy-source) (x-amz-copy-source) folder (x-amz-copy-source) folder (x-amz-copy-source) folder (x-amz-copy-source) folder (x-amz-copy-source) (for each 1,000 objects) None (client-side only) Solution Guide - Storage Made Easy!10

11 ABOUT STORAGE MADE EASY STORAGE MADE EASY OFFICE 1 Mulgrave Chambers Mulgrave Road. Sutton, London SM2 6LE UK The Storage Made Easy Enterprise File Fabric enables IT to regain control of "cloud data sprawl" to unify on-premises and on-cloud files into a single easily managed converged infrastructure with policy-based controls for governance, audit and security. The File Fabric solution provides a blanket enterprises wrap around all their data: on premises, within public and private clouds, as well as on third-party software vendor clouds (SharePoint Online and Salesforce for example). Customers use the SME File Fabric product to unify corporate data to enable workgroup collaboration with universal controls for security, encryption, audit, and data access. Companies will have access to a greater understanding of where and how sensitive data is stored, transferred, and accessible with particular focus on how companies are able to police and audit data to ensure enforcement of the General Data Protection Regulations (GDPR) in addition to other regional data legislation. USA UK +44(0) EU sales@storagemadeeasy.com Free hosted and enterprise free trial available from Storage Made Easy is the trading name of Vehera LTD. reg no: /StorageMadeEasy /company/storage-made-easy Solution Guide Storage Made Easy!11