DDN s Vision for the Future of Lustre LUG2015 Robert Triendl

DDN s Vision for the Future of Lustre LUG2015 Robert Triendl

3 Topics 1. The Changing Markets for Lustre 2. A Vision for Lustre that isn t Exascale 3. Building Lustre for the Future 4. Peak vs. Operational Performance 5. Application Optimized Lustre 6. Why Conventional Storage Still Matters

4 Hyperscale Storage Markets HPC Scratch Petabytes Streaming Write Large Files Infiniband Big Data & Data Analytics Cloud WORM(N) Billions of Files Random Read Small Files Ethernet Single Location Distributed

5 Lustre Markets Today Cloud 2% Archive 12% Work 31% Mixed 22% Data 33%

6 Market Diversification Cloud Work Data Mixed Use Archive Weather Climate HPC Work CAE Chemical General Academic HPC Cloud Tier 2 Genomics Finance Cloud Big Data Science Energy Security

7 Lustre Futures Beyond Exascale Manufacturing Genomics General Academic Cloud Archive CIFS/NFS Export, AD Integration, RAS Features, Snapshots, Data Management, etc. Random Performance, Small File & Metadata Performance, Data Management, Security, etc. Broad Application Support, Connectors, User Monitoring, User Access to Snapshot, etc. Virtualization, Snapshots, Small File Read Performance, Data Distribution, etc. Data Management Features, SMR Drive Use, Data Scrubs, Data Distribution, etc.

8 Market Evolution 100% 90% 80% 70% 60% 50% 40% 30% 20% Archive Cloud Mixed Data Work 10% 0% 2012 2013 2014

9 Market Segments 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2011 2012 2013 2014 Industry Government University

10 Disks: Throughput vs. IOPS MB/sec 1800 1600 1400 1200 1000 800 600 400 200 IOPS/10 GB/sec 14,000 12,000 10,000 8,000 6,000 4,000 2,000 0 Lustre 1.8 Lustre 2.4 ExaScaler 2.2 Lustre with BtrFS Next Gen SAS Drives 0

11 Lustre Development at DDN Lustre Usability Features Build-in Reliability and Availability Lustre Recovery Features for a Broader Market Performance for Broad Set of Applications Application-optimized Lustre

12 Lustre Code Contributions 120 100 80 60 40 Other EMC SUSE Bull Cray Seagate DDN 20 0 2.1 2.4.0 2.3.50-2.4.0 2.5.0 2.5.50-2.6.0

13 DDN ExaScaler Software Stack DDN DDN & Intel Intel HPDD Other Data Management Fast Data Copy Tape S3 Cloud Object (WOS) ExaScaler Data Management Framework Monitoring & Management DDN DirectMon DDN ExaScaler DDN ExaScaler Monitor Intel IML DDN Clients NFS/CIFS/S3 DDN IME Intel Hadoop Core FS ldiskfs Intel DSS DDN Lustre Edition OpenZFS btrfs Storage HW DDN Block Storage with SFX Cache Other HW

14 Why BtrFS? Standard Local Filesystem in RHEL7 Better Throughput Performance than ZFS Similar Feature Set, but all Linux No Possible Patent Infringement Simple Integration and Deployment

15 Application-Optimized Lustre Lustre for Specific Applications Workload Profiling Optimization Across I/O Calls Optimizing Application Runtime Working with Customers

16 Genome Pipeline Benchmarks Samtools 20% faster with DDN Lustre optimizations Runtime (Hours) Lustre 2.5 Client Performance Human Genetics samtools workflow 50 40 30 20 10 0 2.5.1 DDN Branch

17 SSD Pools and Caching DSS to Link File Layer to Block Layer Build into the File System Better use of SSDs for I/O Optimization Increased Small File Performance Increased Random Read to Large Files Additional specificity with fadvice()

18 ExaScaler Monitoring Filesystem, OSS, MDS, OST, MDT, etc. JOB ID, UID/GID, application stats, etc. Archive of data by policy Lightweight Near real- Rme Massive scale Burst Buffer Monitoring Server collectd Graphite plugin OSS, MDS Storage UDP(TCP)/IP based small text message transfer graphite

19 TITECH Examples 64 Billion of Lustre Stats in 15 days!

20 Why Block-Level Raid? Best Mixed I/O Performance Consistent Performance Hardware-optimized Performance Best Performance During Failure Integrated Storage Services

21 SFA RAID Stack Performance Above 1 Million 4K IOPS per 8 CPU Cores Above 10 GB/sec per 8 CPU Cores 8 Cores Sufficient for PCI Infrastructure More Cores for File System Services Additional Cores for More Functionality

22 SFA Random Read MB/sec 30 25 20 15 10 5 512K I/O Size 1M I/O Size 2M I/O Size 4M I/O Size 0 1 8 16 24 32 40 48 56

23 Flexible SSU Design S M L XL

24 SFA14K Performance SSU Up to 45 GB/sec Up to 2950 TB External MDT 4-6 OSS Monitoring 2-4 MDS MDT Storage OST Storage

25 Wolfcreek Hardware

26 Wolfcreek Hardware