Scale-out Object Store for PB/hr Backups and Long Term Archive April 24, 2014

Transcription

1 Scale-out Object Store for PB/hr Backups and Long Term Archive April 24, 2014 Gideon Senderov Director, Advanced Storage Products NEC Corporation of America

2 Long-Term Data in the Data Center (EB) Consumption of Enterprise Disk Capacity by Type 83% CAGR 46.0% 47.5% 28.8% Content Depots & Cloud Unstructured data Replicated data Structured data 22.5% Source: IDC, November 2012 Page 2 NEC Corporation of America 2014

3 PBBA Market Forecast (CY CY2017) Market is growing fast. (Source: IDC) Page 3 NEC Corporation of America 2014

4 Archive Market Trend Expected rapid growth in disk based archive market due to wide usage of active archive, growing with CAGR 35%, and reach $5.7B WW in 2016 File- and Object-Based Storage (FOBS) market exceeding $23B in 2013, projected to grow with CAGR 25% and reach $38B WW in 2017 (IDC, August 2013) WW Disk based Archive sales forecast 6,000 5,000 Sales revenue ($M) 4,000 3,000 2,000 1,000 CAGR 35% Source: IDC, June 2013 Page 4 NEC Corporation of America 2014

5 Storage Infrastructure Spending Page 5 NEC Corporation of America 2014

6 Scale-out Object Storage Object Storage A storage architecture that manages data as objects, as opposed to other storage architectures like file systems which manages data as a file hierarchy and block storage which manages data as blocks within sectors and tracks Object storage systems allow relatively inexpensive, scalable and selfhealing retention of massive amounts of unstructured data. Source: Wikipedia, April 2014 Page 6 NEC Corporation of America 2014

7 Legacy Solutions Scalability Limitations Inadequate scalability of capacity & performance Cannot scale performance to keep up with data growth Multiple products with different architectures More siloed capacity to manage Limited deduplication scope Limited scalability proliferates duplicate data across appliances Lower deduplication ratio for large environments Page 7 NEC Corporation of America 2014

8 Local vs. Global Deduplication Single deduplication repository across entire solution Data deduplication across ALL data from ALL node Cross-node deduplication for greater efficiency Cross application deduplication leveraging application-aware deduplication Page 8 NEC Corporation of America 2014

9 Legacy Solutions Resiliency Limitations Day 1: Full Day 2: Incremental Day 3: Incremental Day 7: Full Q: What data can be restored if block #1 lost? A: NONE! Traditional RAID is not sufficient for deduplicated data Page 9 NEC Corporation of America 2014

10 Scale-out Scalable Grid Storage Architecture Community of Smart Nodes Accelerator Nodes Storage Nodes Hybrid Nodes Nodes Industry-standard servers Multiple types allowed Heterogeneous & open Unrestricted Scalability PERFORMANCE SCALABILITY CAPACITY SCALABILITY Intelligent Management SW Fully distributed system Self-aware & self-organizing Data management services Virtualizes hardware platform Page 10 NEC Corporation of America 2014

11 Adaptive Grid Storage For Long-Term Data Online Upgrade/Expansion with Multi-generation Nodes Non-disruptive Remove Non-disruptive Add/Upgrade V2 Grid + V3 + V4 + Vx = 1 System Enable in-place technology refresh with no data migration Ever greener storage with faster, denser components Enable continuous data availability Reduce CapEx and OpEx with deduplication Non-disruptive scaling from dynamic auto provisioning storage Page 11 NEC Corporation of America 2014

12 Hands-Free Management Simple, fast deployment < 45 minutes to backup & archive Self-discovering capacity No storage provisioning No tape emulation tasks Self-tuning and Resource Management Optimized performance & capacity Self-healing Automatic recovery across resources Web-browser GUI Monitoring and Planning Page 12 NEC Corporation of America 2014

13 Advanced Erasure-Coded Data Resiliency User dialable disk/node protection Default protection against 3 concurrent failures Dynamically allocated Intermix of multiple resiliency levels (1-6) for different applications Greater protection with lower overhead Default setting: 25% capacity overhead 1.5x greater protection than traditional RAID 6 with lower overhead and faster recovery No idle spare drives Faster self-healing with less performance degradation Only data is reconstructed rather than entire drive Data is reconstructed across multiple spindles Page 13 NEC Corporation of America 2014

14 Lab Benchmark Test Page 14 NEC Corporation of America 2014

15 Scale-out vs. Scale-up Data Deduplication Controller AN AN Shelf Shelf Shelf Shelf SN SN SN SN Single or Multi-Controller Modular scale-out front-end Fixed maximum throughput Scalable maximum throughput Scalability limited by controller physical capabilities Scalability independent of physical capabilities with linear performance Page 15 NEC Corporation of America 2014

16 Scale-out Inline Global Data Deduplication Distributed Two-Tier Architecture Independent linear scalability of performance & capacity AN AN Global Deduplication Data deduplication across ALL data from ALL nodes SN SN SN SN Distributed Hash Table Data routed to responsible Storage Node Deduplication & hash table processing scales linearly with Storage Nodes Prevents silos of deduped data Page 16 NEC Corporation of America 2014

17 Linear Performance Scalability Page 17 NEC Corporation of America 2014

18 Scale-out Deduplication Performance Page 18 NEC Corporation of America 2014

19 Application Metadata Impact Clients Files Agent Operation File Aggregation (tar) Blocking Backup Server Storage Media Metadata Filtering Application metadata makes user data appear different Inserted metadata reduces deduplication efficiency Page 19 NEC Corporation of America 2014

20 Application-Aware Deduplication Capacity Optimization through Enhanced Deduplication Generic Dedupe Data (Bytes) Application-Aware Dedupe Time (Weeks) Original Data Generic Dedupe Application-Aware Dedupe Application-aware deduplication leverages format awareness to filter metadata inserted by the application and deduplicate the data payload separately Application-Awareness can increase Reduction Ratio by 130% or more Page 20 NEC Corporation of America 2014

21 Disaster Recovery and Cloud WAN-optimized Replication Geo-Distributed Grid Site A Replication Site B Site A Site C Site B Page NEC Corporation of America 2014

22 WAN-Optimized Replication Accelerator Nodes Storage Nodes Many to one replication Many to many replication Asynchronous grid-to-grid WAN-optimized replication for DR Deduplication across all replicated HYDRAstor grids Minimizes network bandwidth requirements Minimizes DR site capacity requirements Policy-based data selection File System Granularity In-flight encryption Page 22 NEC Corporation of America 2014

23 Customer Configuration Example Replication across primary and secondary datacenter grid systems Replication from remote sites within the U.S., and across the Atlantic and the Pacific Page 23 NEC Corporation of America 2014

24 HYDRAstor Gen-4 Family Performance Single Node Model HS TB/hr, 3.6TB/hr (OST) 8-24TB Raw TB Effective HS TB/hr, 6.3TB/hr (OST) 12-48TB Raw TB Effective Page 24 NEC Corporation of America 2014 HS TB/hr, 12.6TB/hr (OST) 96TB Raw 1.2PB Effective HS TB/hr, 5.6TB/hr (OST) 96TB Raw 1.2PB Effective Grid Model HS TB/hr, 12.6TB/hr (OST) 192TB Raw 2.5PB Effective HS TB/hr, 37.8TB/hr (OST) 720TB Raw 9.4PB Effective Capacity HS TB/hr, 1,040TB/hr (OST) 7,920TB Raw 103PB Effective Node Building Blocks Hybrid Node Storage Node

25 Common Code Base and Functionality Common code/features and modular scalability across models Intermix of multi-generation nodes in the same grids All Software Features supported for entire product line DRD protection Advanced erasure-coded data resiliency DataRedux Inline application-aware global data deduplication and compression Cloning/Snapshot Instant fully deduplicated file system or file R/W copy Dynamic Data Shredding Data shredding for deleted classified data AN Failover Front-end failover for High Availability RepliGrid (Option) WAN-optimized replication with in-flight encryption HYDRAlock (Option) WORM file system functionality Encryption at Rest (Option) Encryption to protect data at rest HYDRAstor OpenStorage Suite (Options) Express I/O Lightweight data transport for high throughput Dynamic I/O Adaptive I/O load balancing across nodes Optimized Copy WAN-optimized copy services Optimized Synthetics Storage-synthesized full backups Page 25 NEC Corporation of America 2014

26 Scale-out Storage for Long-Term Data 1 PB/hr 100 PB Page 26 NEC Corporation of America 2014

27 Potential Enhancement Directions Performance Functionality Page NEC Corporation of America 2014

28