Near- Data Computa.on: It s Not (Just) About Performance

Transcription

1 Near- Data Computa.on: It s Not (Just) About Performance Steven Swanson Non- Vola0le Systems Laboratory Computer Science and Engineering University of California, San Diego 1

2 Solid State Memories NAND flash Ubiquitous, cheap Sort of slow, idiosyncra0c Phase change, Spin torque MRAMs, etc. On the horizon DRAM- like speed DRAM or flash- like density 2

3 Bandwidth Rela.ve to disk x à 2.4x/yr PCIe- Flash (2012) PCIe- PCM (2010) PCIe- PCM (2015?) DDR Fast NVM (2016?) Hard Drives (2006) PCIe- Flash (2007) 7200x à 2.4x/yr /Latency Rela.ve To Disk 3

4 Programmability in High-Speed Peripherals As hardware gets more sophis0cated, programmability emerges GPUs fixed func0on à Programmable shaders à full- blown programmability NICs Fixed func0on à MAC offloading à TCP offload Storage has been leb behind It hasn t goden any faster in 40 years 4

5 Why Near-data Processing? Well- studied Data Dependent Accesses Internal bandwidth interconnect bandwidth Moving data takes energy Trusted Computa0on NDP compute can be energy efficient Worth a Closer Look Storage latencies Interconnect Latency Storage latencies CPU latency Seman0c Flexibility NDP compute can be trusted Improved Real- 0me capabili0es 5

6 Diverse SSD Semantics File system offload/os bypass [Caulfield, ASPLOS 2012] [Zhang, FAST 2012] Caching support [Bhaskaran, INFLOW 2013] [Saxena, EuroSYS 2012] Database transac0ons support [Coburn SOSP 2013] [Ouyang HPCA 2011] [Prabhakaran OSDI 2008] NoSQL offload Samsung s SmartSSD [De FCCM 2013] Sparse storage address space FusionIO DFS Lessons Learned Our Goal: Make Implementa0on costs are high Programming an SSD Easy and Flexible. Fit with applica0ons is uncertain 6

7 The Sobware Defined SSD 7

8 The Software-defined SSD SDSSD Host Machine Application Generic SSD- APP interface PCIe Ctrl PCIe Bridge Memory Memory Memory Controller SSD CPU Memory Memory 3GB/s 3GB/s 3GB/s Memory Controller SSD CPU Memory Memory Memory Controller SSD CPU 4 GB/s 8

9 SDSSD Programming Model The host communicates with SDSSD processors via a general RPC interface. Host- CPU Send and receive RPC requests SSD- CPU Send and receive RPC requests Manage access to NV storage banks SDSSD CPU RPC Host CPU PCIe Ring Data streamer Memory 9

10 The SDSSD Prototype FPGA- based implementa0on DDR2 DRAM emulates PCM Configurable memory latency 48 ns reads, 150 ns writes 64GB across 8 controllers PCIe: 2 GB/s, full duplex 10

11 SDSSD Case Studies Basic IO Caching Transac0on processing NoSQL Databases 11

12 Basic IO 12

13 Normal IO Operations: Base-IO Base- IO App provides read/write func0ons. Just like a normal SSD. Applica0ons make system calls to the kernel The kernel block driver issues RPCs user accesses 1. SysCall 2. Read/Write RPCs kernel module SDSSD CPU NVM 13

14 Faster IO: Direct access IO OS installs access permissions on behalf of a process. Applica0ons issue RPCs directly to HW The App checks permission user accesses 1. userspace channel kernel module 2. kernel installs perms 3. userspace channel direct IO accesses SDSSD CPU NVM 14

15 Preliminary Performance Comparison 1800 Read 1800 Write Bandwidth (MB/s) Direct- IO Direct- IO- HW Base- IO Direct- IO Direct- IO- HW Base- IO

16 Caching 16

17 Caching NVMs will be caches before they are storage Applica.on file file Cache hits should be as fast as Direct- IO Caching App Tracks dirty data Tracks usage sta0s0cs Userspace Kernel File System system reads/writes Cache Library Cache Miss Cache Manager Device Driver Cache Hit Disk SDSSD Caching 17

18 4KB Cache Hit Latency 18

19 Transac0on Processing 19

20 Atomic Writes Write- ahead logging guarantees atomicity New interface for atomic opera0ons LogWrite(TID, file, file_off, log, log_off) Transac0on commit occurs at the memory interface à full memory bandwidth Logging Module inside SDSSD Transaction Table TID 15 PENDING TID 24 FREE TID 37 COMMITTED " " X " " Metadata File Log File New D New B New A New C Data File Old D Old B Old A Old C 20

21 Atomic Writes SDSSD atomic- writes outperform pure sobware approach by nearly 2.0x SDSSD atomic- writes achieve comparable performance to a pure hardware implementa0on AtomicWrite Bandwidth (GB/s) SDSSD Tx Opt SDSSD Tx Moneta Tx Soft atomic Request Size (KB) 21

22 Key Value Store 22

23 Key-value store Key- value store is a fundamental building block for many enterprise applica0ons Supports simple opera0ons o o o Get to retrieve the value corresponding to a key Put to insert or update a key- value pair Delete to remove a key- value pair Open- chaining hash table implementa0on. 23

24 Direct-IO based Key-Value Store Get K3 Key K3 Hash(K3) 0 1 K1 K4 V1 V4 K2 V2 K3 V3 Compare Keys M- 1 Index Structure I/O K7 V7 K8 V8 K9 V9 Key- value data Mismatch Match Host SSD 24

25 SDSSD Key-Value Store App Get K3 K1 V1 K2 V2 K3 V3 0 1 K4 V4 Key K3 Hash(K3) Hash Idx RPC(Get) M- 1 Index Structure Hash Idx Key K3 K7 V7 K8 V8 K9 V9 Mismatch Match Compare Keys Key- value data K3 V3 DMA Host SDSSD SPU 25

26 MemcacheDB performance 0.6 Throughput (in Million ops/s) Direct- IO Key- Value- HW Key- Value 0 Get Put Workload- A Workload- B Throughput (Million ops/s) Get Avg. chain length Direct- IO Key- Value Key- Value- HW Throughput (Million ops/s) Put Avg. chain length Direct- IO Key- Value Key- Value- HW 26

27 Ease-of-Use 27

28 Conclusion New memories argue for near- data processing But not just for bandwidth! Trust, low- latency, and flexibility are cri0cal too! Seman0c flexibility is valuable and viable 15% reduc0on in performance compared to fixed- func0on implementa0on 6-30x reduc0on in implementa0on 0me Every applica0on could have a custom SSD 28

29 Thanks! 29

30 Ques0ons? 30