Solid State Storage Technologies. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University

Transcription

1 Solid State Storage Technologies Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University

2 NVMe (1) NVM Express (NVMe) For accessing PCIe-based SSDs Bypass block I/O layer Low latency Read, Write, Flush, Format Deallocate, Atomic Write Data Set Management 2

3 NVMe (2) Deep queue: 64K commands/queue, up to 64K queues Streamlined command set: only 13 required commands One register write to issue a command ( doorbell ) Support for MSI-X and interrupt aggregation Doorbell 3

4 All-Flash Array Interfaces 10Gb/40Gb Ethernet (iscsi) or 16Gb Fibre Channel or PCIe SAS or NVMe SSDs Functionalities Volume management Virtualization support RAID Snapshot Deduplication Compression, 4

5 Traditional Block Interface SATA/SCSI/SAS Read (sector #, length) Write (sector #, length, data) No block-level liveness information No high-level semantics on data Several unwritten contracts do not hold for SSDs Sequential accesses are several tens of times better than random accesses Distant LBNs lead to longer seek times Data written is equal to data issued Host File system Block device driver SSD FTL NAND Flash Block I/F Flash I/F 5

6 Extending Block I/F TRIM command The data in the specified sectors is no longer needed ATA interface standard (T13 technical committee) Non-queued command SATA 3.1 introduces the Queued TRIM command Host SSD File system Block device driver FTL NAND Flash Block I/F + SSD-Specific I/F Flash I/F 6

7 Atomic Write Transaction support for multi-block writes Simplifies file systems and DBMSes X. Quyang, et al., Beyond Block I/O: Rethinking Traditional Storage Primitives, HPCA,

8 FusionIO svfsl Virtualized Flash Storage Layer Provides 64-bit virtual block-addressed space Virtual-to-physical block mapping: A variation of B-trees in memory FTL functionalities read, write, trim, and atomic update supported W. Josephson, et al., DFS: A File System for Virtualized Flash Storage, FAST,

9 DFS over VFSL Direct File System Simple metadata and data layout (2TB chunk) 9

10 Annotating Block Semantics For differentiated storage service Class mapped to Group Number field (5 bits) of the SCSI CDB Selective caching on SSDs Similar mechanism in emmc 4.5 ContextID Ext3 Class Group Number Cache priority Unclassified 0 12 Superblock 1 0 Group desc. 2 0 Bitmap 3 0 Inode 4 0 Indirect block 5 0 Directories 6 0 Journal 7 0 File <= 4KB 8 1 File <= 16KB 9 2 File > 1GB M. Mesnier, et al., Differentiated Storage Services, SOSP,

11 Multi-streamed SSD (1) Previous write patterns (= current state) matter 11

12 Multi-streamed SSD (2) Mapping data with different lifetime to different streams Standardized in T10 SCSI (SAS SSDs) 12

13 Multi-streamed SSD (3) Cassandra with Multi-streamed SSD 13

14 Multi-streamed SSD (4) Cassandra s normalized updated throughput with 5 streams 14

15 OSSD: Object-based SSD (1) OSD (Object-based Storage Device) Virtualizes physical storage as a pool of objects Offloads space management to storage devices Standardized as a subset of SCSI command set Block interface Object interface 15

16 OSSD: Object-based SSD (2) OSD storage model Application Application Host System Call Interface File System User Component System Call Interface File System User Component File System Storage Management OSD Interface Storage Device Sector/LBA Interface Block I/O Manager Physical Media OSD Storage Management Block I/O Manager Physical Media Traditional OSD 16

17 OSSD: Object-based SSD (3) Host Target OSD Interface (iscsi) VFS OFS OAQ iscsi Initiator iscsi Target Daemon OSSD Framework OML READ/WRITE/ERASE FML FAL RawSSD TCP 1Gbps SATA-2 OID -Context Hash I/O OAQ Context OML Object Data Buffer Descriptor FML oid = 46 oid = 7 Object Data (Extents) Priority Queue W W 16:8 Object I /O instances μ-tree C Q n Q 1 Q 0 Object Attr. Allocation Bitmap 17

18 OSSD: Object-based SSD (4) Simplified host file system No need for SSD-specific parameter tuning More efficient management of flash storage Block-level liveness Metadata separation Object-aware storage management (allocation, dedup,...) Application-aware storage management Application hints, QoS Storage virtualization Pooling, tiering, caching, backup, replication, etc. 18

19 In-Storage Computing (1) Samsung ISC SSD Prototype Commodity SSD: Samsung PM1725 NVMe with the ISC feature PCIe 3.0x4 800 GB Software C++11 C++STL G++ Software emulator 19

20 In-Storage Computing (2) ISC Application Development Process 20

21 In-Storage Computing (3) ISC Dataflow Programming Model 21

22 In-Storage Computing (4) Example: Simple Key-Value Store 22