How to Network Flash Storage Efficiently at Hyperscale Manoj Wadekar Michael Kagan Flash Memory Summit 2017 Santa Clara, CA 1
ebay Hyper scale Infrastructure Search Front-End & Product Hadoop Object Store Deep Learning AI Databases 2
Flash Typical hyper scale servers Performance Needs In Memory/Search Databases Hadoop Object Store FE/Dev Archival/Cold 3
Typical hyper scale servers: Design goals Efficiency: Utilization, commonality Growth: Performance, Capacity 4
Converged Infrastructure: challenges Mismatched App Needs: - Compute/Storage needs can change for different clusters. - Can result in underperformance or wastage Inefficiency: - Spend, infrastructure space, power utilization - Further challenge to justify high density/high performance drives Scale Challenge: - Cattle use cases for data-heavy workloads may result in large data movement - Complicated storage scheduler leads to constrained scaling Server Platform: Shackled to local storage 5
What s needed: Disaggregated Storage Separate out storage and compute resources 6
What s needed: Rack-As-A-Compute Ethernet Node Local Storage Rack Local Storage 7
Why: Rack-As-A-Compute Right Sizing: - Clusters can use optimized ratio of compute and storage. - Allows reducing wastage and improve performance Independent Scaling: Compute and storage capacities can be scaled per need Ethernet 8
Disaggregated Storage: Interconnect Needs Throughput: - Sequential workloads driven by throughput - Aggregated storage drives higher needs Latency: - IOPs sensitive workloads - Appropriate deployment topologies Simplicity: - Known, ubiquitous network Is Ethernet Ready? 9
The Paradigm Shift Resource as a Service 2017 Mellanox Technologies 10
RDMA: Data Center Infrastructure Foundation Direct access to remote resource App App BUF BUF OS NIC NIC OS Kernel Bypass Acceleration RDMA over Converged Ethernet RoCE 2017 Mellanox Technologies 11
RDMA-Enabled Cloud Infrastructure 2017 Mellanox Technologies 12
Storage Media Technology 10000 100,000 1000 100 10 Storage Media Access Time 10,000X Improvement 1 1 0.1 Hard Disk SSD NVDIMM Hard disk SSD NVDIMM 2017 Mellanox Technologies 13
Networked Storage Continues Growth and Movies to Ethernet Networked Storage (SANs) Better utilization: capacity, rack space, power Scalability, management, fault isolation Ethernet growing very rapidly driven by: Cloud & Hyper Converged Infrastructure (HCI) - No Fibre Channel in the Cloud NVMe Over Fabrics Software Defined Storage 2017 Mellanox Technologies 14
Ethernet Storage Fabric We ve Got You Covered Ethernet Storage Fabric Everything a Traditional SAN Offers but Faster, Smarter, & Less Expensive Performance Efficiency Ubiquity 2017 Mellanox Technologies 15
NVMe-oF Performance with Open Source Linux 100 2017 Mellanox Technologies 16
BlueField: High Performance Yet Cost Effective Storage 200 Gb/s of throughput More than 5 million IOPS BlueField Industry-Leading ConnectX Intelligent Offload Tile Multicore ARM Architecture 2017 Mellanox Technologies 17
BlueField Building Blocks for Storage Platforms ASIC Storage Controller Adapter Family of products 4, 8, 16 cores NMVe storage solutions Customized per-oem Smart NIC Standard PCIe slot Open Platform Software Standards Based BlueOS Linux Built from kernel.org Standard Linux development tools to build your application 2017 Mellanox Technologies 18
Storage Class Memory (SCM) 1980 2010 2017+ SCM Over Fabrics Definition WIP in IBTA 2017 Mellanox Technologies 19
Technology Leadership RAID in network BlueField Storage-optimized NVMEoF SoC Target offload Crypto T10DIF offload RDMA Boards Silicon RDMA NAS Systems Boards Silicon RDMA block storage (iser) Cables Systems Boards Silicon Software Cables Systems Boards Silicon Optics Software Cables Systems Boards Silicon Processing Optics Software Cables Systems Boards Silicon 2005 2008 2010 2012 2015 2017 2017 Mellanox Technologies 20
Network is a Computer Deliver Value $$$$$$$/User Users Smart Network Increases Datacenter Value $$/User Users Intelligent Tasks Network functions On Network Offload: 100% for Apps 2017 Mellanox Technologies Leadership in Data Center Networking 21
Questions? Manoj Wadekar Michael Kagan Flash Memory Summit 2017 Santa Clara, CA 22