THE IN-PLACE WORKING STORAGE TIER OPPORTUNITIES FOR SOFTWARE INNOVATORS KEN GIBSON, INTEL, DIRECTOR MEMORY SW ARCHITECTURE

Transcription

1 THE IN-PLACE WORKING STORAGE TIER OPPORTUNITIES FOR SOFTWARE INNOVATORS KEN GIBSON, INTEL, DIRECTOR MEMORY SW ARCHITECTURE

2 I/O LATENCY WILL SOON EXCEED MEDIA LATENCY 30 NVM Tread 25 NVM xfer Controller ASIC Latency (usecs) The Media I/O Overhead Controller Firmware Platform Link xfer & protocol (NVMe/PCIe) Driver Storage Stack I/O 0 NAND MLC NVMe SSD (4kB read) 3D XPoint NVMe SSD (4kB read) Persistent Memory with In-Place Persistence DIMM Memory (64B read)

3 THE TRADITIONAL STORAGE I/O ARCHITECTURE File APIs Optional File Filters Working Data Optional Block Filters Block Drivers OS Kernel Data I/O DRAM I/O-based Storage The File I/O Programming Model

4 ESTABLISHING THE OPEN INDUSTRY PROGRAMMING MODEL 36+ Member Companies SNIA Technical Working Group Defined 4 programming modes required by developers Spec 1.0 developed, approved by SNIA voting members and published Interfaces for PM-aware file system accessing kernel PM support interfaces for application accessing a PM-aware file system Kernel support for block NVM extensions Interfaces for legacy applications to access block NVM extensions

5 THE OPEN INDUSTRY PROGRAMMING MODEL Mgmt. Management Tools Storage Standard Raw Device Access File Standard File API Memory Standard File API User Load/Store Management Library DAX PM-Aware MMU Mappings Kernel NVDIMM Driver NVDIMM

6 CREATING THE API THE OPEN SOURCE NVM LIBRARY Standard File API Load/Store User pmem-aware MMU Mappings Kernel NVDIMM NVM Libraries Open Source Libpmem libvmem libvmmalloc libpmemobj libpmemblk Atomic & Transactional libpmemlog

7 FROM PERSISTENT TO DURABLE TO COMPLIANT File APIs Optional File Filters Working Data Optional Block Filters Block Drivers Example Data Services Local & Remote Replication Snapshot & Continuous Data Protection Data protection, RAID, etc. Tiering, Caching Data reduction: Dedup, Compression Audit trail, Encryption Virus protection Other Compliance features. OS Kernel Data I/O DRAM I/O-based Storage Intercepting I/O to add Durability and Compliance

8 DURABILITY AND COMPLIANCE FOR PERSISTENT MEMORY User Storage Mode DAX DAX + File APIs Standard File APIs Kernel & Storage I/O Kernel NVML APIs NVML File APIs Direct Load/Store Direct Load/Store Apache Pass Device Driver I/O I/O In-place Persistence DRAM NVDIMM Compatible with Today s Filter Drivers and PaaS Layers NVDIMM Requires New Approach to Durability & Compliance DRAM NVDIMM App-Level Data Copies

9 WRITE I/O REPLACED WITH PERSIST POINTS Standard File API User NVM Library New Durability and Compliance Plug-ins pmem-aware DAX Kernel NVDIMM Traditional APIs msync() FlushViewOfFile() NVML API pmem_persist() Opportunity for NextGeneration DataGrids?

10 THE END STATE MEMORY AS THE NEW PERFORMANCE STORAGE TIER Persistent Memory DRAM SSD HDD Working Data Hot Storage Capacity Storage Today In-Place Working Storage I/O-based Storage The Future Working Memory and Hot Storage Tier Merge

11 JOIN THE DISCUSSION Ø Learn about the Persistent Memory programming model Ø Ø Join the pmem NVM Libraries Open Source project Ø Ø Read the documents and code supporting ACPI 6.1 and Linux NFIT drivers Ø Ø Ø Ø Ø Intel Architecture Instruction Set Extensions Programming Reference Ø Ø Intel 3D XPoint TM Memory Ø 11

12 THANK YOU