2012 HPC Advisory Council

Transcription

1 Q HPC Advisory Council DDN Big Data & InfiniBand Storage Solutions Overview Toine Beckers Director of HPC Sales, EMEA

2 The Global Big & Fast Data Leader DDN delivers highly scalable & highly-efficient storage solutions that enable customers to accelerate time to results, scale simply with big data growth and gain competitive advantage through resolving performance & capability scaling challenges. Key Statistics Established: 1998 Revenue: $200M+ Per Year, Profitable & Growing Headquarters: Chatsworth, California USA Employees: More than 500 Worldwide Worldwide Coverage: 4 Continents, 15 Countries Installed Base: Over 1,000 Across 50 Countries Go to Market: Global Partners, VARs, Resellers Industry Validation World s Largest Privately-Held Storage Co. Fast500 Technology Company Inc High-Growth Company Best HPC Storage Platforms Hot Technology for World-Leading Customers Key Market Segments: High Performance Computing & Life Science Cloud & Web Content Rich Media Intelligence/Federal Video Surveillance 2

3 DDN High-Speed Networking Background InfiniBand Storage Leader Since 2005 World s Broadest IB/GbE Portfolio Powering New & Exciting HPC Applications Driving Exascale Storage Architecture Forward 3

4 DDN InfiniBand Experience 2005: DDN Releases Fastest SRP Target 2005: Spider File System Is Deployed World s Fastest IB Parallel File System 2010: First IB-Connected Object Store (joint US Department of Defense) 2012: SFA12K Owns All The Metrics Bandwidth & IOPS; File & Block And much more 4

5 SFA12K-40 Highly Parallelized SFA Storage Processing Engine Active/Active Design 1.7 Million Burst IOPS from 32GB Mirrored, Non-Volatile Cache Up to 1.4M Sustained Random Read IOPS from SSDs 40GB/s Raw Sequential Read & Write Speed Internal PCIe Gen 3 bus 280Gb/s ICL RAID Levels 1, 5 and 6 Intelligent Write-Through Striping SATAssure Data Protection GUI, SNMP, CLI 32 x FC-16 ports or 16 x FDR-IB ports 32 x 16Gb Fibre Channel Host Ports or 16 x FDR InfiniBand Host Ports SFA Interface Virtualization 16GB High-Speed Cache 280Gb/s Cache Link SFA Interface Virtualization 16GB High-Speed Cache 960Gb/s Internal SAS2 Storage Management Network P RAID 5, m Internal SAS Switching 40 GB/s RP RP RP RP P RAID 5,6 Q RAID 6 Internal SAS Switching Up to 1680 SAS, SATA or SSD Drives with full redundant paths Q RAID 6 Parallel processing engine Massive I/O Back-End RAID 5,6 RAID 5,6 RAID 1 5

6 SFA12K-20E Highly Parallelized SFA Storage Processing Engine Active/Active Design 1.7 Million Burst IOPS from 32GB Mirrored, Non-Volatile Cache Up to 1.4M Sustained Random Read IOPS from SSDs 20GB/s Raw Sequential Read & Write Speed Internal PCIe Gen 3 bus 280Gb/s ICL RAID Levels 1, 5 and 6 Intelligent Write-Through Striping SATAssure Data Protection GUI, SNMP, CLI 32 x 10GbE or 16 x FDR-IB ports 32 x 10Gb Ethernet Host Ports or 16 x FDR InfiniBand Host Ports SFA Interface Virtualization 280Gb/s Cache Link SFA Interface Virtualization 480Gb/s Internal SAS2 Storage Management Network P RAID 5, m Internal SAS Switching 20 GB/s AP RP AP RP 64GB AP Memory 16GB High- Speed Cache P RAID 5,6 Q RAID 6 16GB High- Speed Cache 64GB AP Memory Internal SAS Switching Up to 1680 SAS, SATA or SSD Drives with full redundant paths Q RAID 6 Parallel processing engine Massive I/O Back-End RAID 5,6 RAID 5,6 RAID 1 6

7 Example: Implementing Lustre Today Infrastructure: Multiple OSS servers 2 Lustre MDS servers 1 Lustre MGS server Multiple RAID Arrays for OSTs RAID Array for MDTs SAN Switching Multiple Racks of equipment requiring power, cooling and floor space 2011 DataDirect Networks. All rights reserved. RESTRICTED COMMERCIAL

8 HPC Storage on the SFA1KE Appliance SFA10000E with embedded File Systems can result in a 10 to 1 or greater reduction in managed systems. Storage Fusion Architecture not only reduces complexity, it streamlines IO by reducing latency and protocol conversions 2011 DataDirect Networks. All rights reserved. RESTRICTED COMMERCIAL

9 SFA12KE Appliances SFA12KE initially available with DataDirect Networks parallel clustered file system solutions Integrate multiple appliances to scale to over 200GB/s and 10 s of Petabytes ExaScaler SFA12KE 20GB/s GridScaler SFA12KE 20GB/s 2011 DataDirect Networks. All rights reserved. RESTRICTED COMMERCIAL

10 Multi-Platform Architecture Block Storage Array Clustered Filer Open Appliance SFA1XK Block Storage Target SFA1XKE DDN File Storage EXAScaler GridScaler SFA1XK Block Storage Target SFA1XKE Customer Applications SFA1XKE Embedded Storage Server SFA1XK Block Storage Target Product Evolution Flexible Deployment Options: 3 System Modalities 2011 DataDirect Networks. All rights reserved. RESTRICTED COMMERCIAL

11 DDN SFA12K 800% Faster Consolidate, Simplify, Save Intelligent Appliances: SFA12K-20E, SFA12K-20, SFA12K-40 40% Denser Reclaim Your Data Center Leading Density: StorageScaler 8460: 84 Disk in 4U (DDN exclusive) 50% Less TCO Smarter Big Data Storage 10+ Years of Innovation: Storage Fusion Processing, Storage Fusion Fabric, QoS, SATAssure, ReACT, more..

12 SFA12K Models Drives Until 2H Gb FC available 2H12 SFA12K-20 SFA12K-20E SFA12K-40 Maximum Drives System Interface Drive Types System Capacity Bandwidth Cache IOPS Flash IOPS In-Storage Processing 1, , ,680 1 FDR IB 16Gb FC 2 FDR IB 10/40GbE FDR IB 16Gb FC & 2.5 SSD, SAS & SATA (inter-mixable) 20GB/s (raw I/O) 6.72PB (w/ 4TB HDDs) 1 20GB/s (file I/O) 40GB/s (raw I/O) 850K 850K 1.7M 700K 700K 1.4M N/A Yes. ExaScaler, GridScaler Customer Provided N/A

13 SFA Products 2012 The SFA10K-M Start with up to 60 disks, grow to 300! SFA 10K-M and 10K-ME March, 2012 A new version of the SFA10K Family that can start small, but grow to full 10K-X performance! Start with as little as 1 disk enclosure o Choose the SS disk enclosure or the SS disk (2.5 SAS) enclosure o Add additional enclosures 1 at a time o Can be added to a live system o Based on SFA OS Depending on the disk type, start at about 3GB/s and grow all the way to 15GB/s Start with up to 24 x 2.5 disks, grow to 120!

14 DDN Application Focus HPC/Life Science Supercomputing Gene Sequencing Seismic Processing Financial Modeling University & Lab Research Professional Media Television Broadcast Post-Production Digital Intermediate Animation Video On Demand Cloud & Internet Web Content Managed Service Providers Storage As A Service Personal Cloud Storage Security Geospatial Intelligence Video Surveillance Massive Interception Rugged Solutions 14

15 New & Exciting HPC Applications Public STAC-M3 Benchmarks using kdb+ (see IBM System x3850 X5 32-core server 512GB RAM 6Gb SAS IBM System Storage DS3524 Express (96) 15K SAS Drives IBM System x3850 X5 32-core server 512GB RAM 40Gb QDR InfiniBand DDN SFA 10K-X (280)10K SAS Drives STAC SUT ID: KDB STAC SUT ID: KDB110913

16 All Storage Systems are not Equal Latency (ms) 3000 Throughput (MB/s) > 350% faster results! Less is better > 750% faster throughput! More is better Hi Bid- Yr 3 DDN SFA IBM SAS STAC-M3.ß1.1T.YRHIBID.LAT2.MEAN STAC-M3.v1.1T.WRITE.LAT2.MEAN STAC-M3.ß1.1T.YRHIBID.BPS.MEAN

17 Hyperscale Storage Web Object Scaler NoFS Hyperscale Distributed Collaboration

18 NoFS Efficiency Defined Management Efficiency: Single Pane of Glass, Global System Fail-In-Place, Zero-Intervention Performance Efficiency: 1 Seek/Operation, 10X Speedup per HDD Linear Global Scale-Out Network-Aware Lowest-Latency Access Data Center Efficiency: 660 HDDs / Rack, Low Power/Cooling

19 WOS Architected for Big Data Hyper-Scale Global Reach & Data Locality 3U, 16-drive WOS Node (SAS/SATA) 4U, 60-drive WOS Node (SAS/SATA) 256 billion objects per cluster Scales to 23PB per cluster Start small, grow to tens of Petabytes Network & storage efficient 2PB / 11 Units per Rack Up to 4 way replication Global collaboration Access closest data No risk of data loss Resiliency with Near Zero Administration Dead Simple to Deploy & Administer Universal Access San Francisco New York London Tokyo Self healing NAS Protocols (CIFS, NFS, etc) Cloud Platform S3 compatibility Native Object Store interface All drives fully utilized Much faster recovery than traditional RAID WOS (Replicated or Object Assure) Reduce or eliminate service calls MultiProtocol Gateway CIFS/NFS protocols LDAP/AD Support Scalable HA & DR Protected Cloud Store Platform S3-Compatible & WebDAV APIs Multi-tenancy Reporting & Billing Native Object Store C++, Python, Java, PHP, HTTP REST interfaces PUT, GET,

20 Requirements HyperScale: DDN s Sweet Spot is BIG! 100Ms: Concurrent Users The Internet of Everything Hyperscale 100Ms: Parallel Threads Exascale Computing CLOUD HPC DDN Is Driving Data-Intensive Computing Systems Toward Convergence

21 Thank You. DataDirect Networks, Information in Motion, Silicon Storage Appliance, S2A, Storage Fusion Architecture, SFA, Storage Fusion Fabric, Web Object Scaler, WOS, EXAScaler, GRIDScaler, xstreamscaler, NAS Scaler, ReAct, ObjectAssure, In-Storage Processing and SATAssure are all trademarks of DataDirect Networks. Any unauthorized use is prohibited. 21