Persistent Memory what developers need to know Mark Carlson Co-chair SNIA Technical Council Toshiba

Transcription

1 Pesistent Memoy what developes need to know Mak Calson Co-chai SNIA Technical Council Toshiba 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 1

2 Contents Welcome Pesistent Memoy Oveview Non-Volatile Memoy (NVM) Pogamming Model Non-Volatile DIMM (NVDIMM) 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 2

3 SNIA Legal Notice The mateial contained in this pesentation is copyighted by the SNIA unless othewise noted. Membe companies and individual membes may use this mateial in pesentations and liteatue unde the following conditions: Any slide o slides used must be epoduced in thei entiety without modification The SNIA must be acknowledged as the souce of any mateial used in the body of any document containing mateial fom these pesentations This pesentation is a poject of the SNIA. Neithe the authos no the pesentes ae attoneys and nothing in this pesentation is intended to be, o should be, constued as legal advice o an opinion of counsel. If you need legal advice o a legal opinion, please contact you attoney. The infomation pesented heein epesents the autho(s) pesonal opinion and cuent undestanding of the elevant issues involved. The authos, pesentes, and the SNIA do not assume any esponsibility o liability fo damages aising out of any eliance on o use of this infomation. NO WARRANTIES, EXPRESS OR IMPLIED. USE AT YOUR OWN RISK Stoage Develope Confeence EMEA. All Rights Reseved. 3

4 The Tend: Memoy & Stoage Convegence Today Volatile and non-volatile technologies ae continuing to convege Memoy DRAM DRAM DRAM/OPM** DRAM/OPM** Stoage Disk/SSD PM* Disk/SSD PM* Disk/SSD PM* Disk/SSD New and Emeging Memoy Technologies *PM = Pesistent Memoy **OPM = On-Package Memoy HMC HBM RRAM 3DXPoint TM Memoy MRAM PCM Low Latency NAND Managed DRAM 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 4 Souce: Gen-Z Consotium 2016

5 Pesistent Memoy (PM) Vision Pesistent Memoy Bings Stoage Fast Like Memoy Pesistent Like Stoage To Memoy Slots Fo system acceleation Fo eal-time data captue, analysis and intelligent esponse 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 5

6 Stoage vs. Memoy Context Switch NUMA Typical NUMA ange: ns Typical context switch ange: above 2-3 us 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 6

7 Application Hoizons Until Recently Hoizon 1: PM Middlewae Hoizon 2: PM Libaies Hoizon 3: Languages Application Application Application Compile File System File System PM Libay Application Disk Dive File System File System SSD PM PM PM 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 7

8 Pesistent Memoy (PM) Chaacteistics Byte addessable fom pogamme s point of view Povides Load/Stoe access Has Memoy-like pefomance Suppots DMA including RDMA Not Pone to unexpected latencies associated with demand paging o page caching Think Powe Potected RAM 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 8

9 NVM Pogamming Model Witing Applications fo Pesistent Memoy 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 9

10 Role of the NVM Pogamming Model Rally the industy aound a view of Pesistent Memoy that is: Application centic Vendo neutal Achievable today Beyond stoage Applications Memoy Netwoking/Fabics Pocessos 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 10

11 NVM Pogamming Model TWG - Mission Acceleate the availability of softwae that enables Pesistent Memoy hadwae. Hadwae includes SSD s and PM Softwae spans applications (use-space) and OS s (kenel-space) Ceate the NVM Pogamming Model Descibes application visible behavios Allows API s to align with OS s Exposes oppotunities in netwoks and pocessos 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 11

12 SNIA NVM Pogamming Model Vesion 1.2 appoved by SNIA in June Expose new block and file featues to applications Atomicity capability and ganulaity Thin povisioning management Use of memoy mapped files fo pesistent memoy Existing abstaction that can act as a bidge Limits the scope of application e-invention Open souce implementations available Pogamming Model, not API Descibed in tems of attibutes, actions and use cases Implementations map actions and attibutes to API s 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 12

13 The NVM Pogamming Model Has 4 Modes Block Mode Innovation Emeging NVM Technologies IO Pesistent Memoy Use View NVM.FILE NVM.PM.FILE Kenel Potected NVM.BLOCK NVM.PM.VOLUME Media Type Disk Dive Pesistent Memoy NVDIMM Disk-Like Memoy-Like 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 13

14 Pogamming Model Modes NVM.FILE and NVM.BLOCK modes use IO Data is ead o witten using RAM buffes Softwae contols how to wait (context switch o poll) Status is explicitly checked by softwae NVM.PM.* (FILE and VOLUME) modes enable Load/Stoe Data is loaded into o stoed fom pocesso egistes Pocesso makes softwae wait fo data duing instuction No status checking eos geneate exceptions 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 14

15 File and Block Mode Extensions NVM.BLOCK Mode Tageted fo file systems and block-awae applications Atomic wites Length and alignment ganulaities Thin povisioning management NVM.FILE Mode Tageted fo file based apps. Discovey and use of atomic wite featues Discovey of ganulaities Block awae Applications PM Block Mode Extensions Block Device Dive PM Block Device File Based Applications PM File mode Extensions File Systems Use mode Kenel mode 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 15

16 Pesistent Memoy (PM) Modes NVM.PM.VOLUME Mode Softwae abstaction fo pesistent memoy hadwae Addess anges Thin povisioning management PM Awae Apps File APIs NVM.PM.FILE Mode Mem ops Use mode NVM.PM.FILE Mode Application behavio fo accessing PM Mapping PM files to application addess space PM Awae File Systems PM VOLUME Mode PM capable Dive Kenel mode Syncing PM files PM Device 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 16

17 Map and Sync Map Sync Associates memoy addesses with open file Calle may equest specific addess Flush CPU cache fo indicated ange Additional Sync types Optimized Flush multiple anges fom use space Optimized Flush and Veify Optimized flush with ead back fom media Waning! Sync does not guaantee ode Pats of CPU cache may be flushed out of ode This may occu befoe the sync action is taken by the application Sync only guaantees that all data in the indicated ange has been flushed some time befoe the sync completes 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 17

18 Failue Atomicity Cuent pocesso + memoy systems Guaantee inte-pocess consistency (SMP) But only povide limited atomicity with espect to failue System eset/estat/cash Powe Failue Memoy Failue Failue atomicity is pocesso achitectue specific Pocessos povide failue atomicity of aligned fundamental data types Fundamental data types include pointes and integes PM pogams use these to ceate lage atomic updates o tansactions Fallback is an additional checksum o CRC 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 18

19 Ongoing SNIA NVMP TWG Wok NVM Pogamming Model Specification NVM Intefaces between OS components Application Intefaces to NVM elated OS, hypeviso and hadwae components Remote Access fo High Availability white pape Ceate models and equiements fo communication with emote pesistent memoy fo the pupose of High Availability Asynchonous Flush, Pesistence Failue Descibe consideations fo suppoting atomics and tansactions using extensions to the NVM Pogamming Model Specification PM Secuity fo Multi-Tenancy Descibe models fo PM secuity when multiple tenants ae pesent See companion SNIA Stoage Develope Confeence talk Stoage Develope Confeence EMEA. All Rights Reseved. 19

20 Summay The NVM Pogamming Model is aligning the industy ( Gaining common teminology Not focing specific APIs What ae we doing with it? PM models expose it DAX-awae file-systems in Linux (see FS_DAX fo moe info) New PM models build on existing ones Linux Pmem Examples (see examples folde) New TWG wok items Emeging technologies will dive inceasing wok in this aea as cost comes down (e.g. mateials and memoy-centic fabics) 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 20

21 NVDIMM Example 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 21

22 NVDIMM Types NVDIMM-N NVDIMM-P DRAM FLASH CNTLR DRAM Host has diect access to DRAM CNTLR moves DRAM data to Flash on powe fail Requies backup powe (typically 10 s of seconds) CNTLR estoes DRAM data fom Flash on next boot Communication though SMBus (JEDEC standad) Byte-addessable DRAM fo lowest latency with NAND fo pesistence backup Block diagam example, JEDEC Seve Foum Jul 17 NVDIMM-P inteface specification tageting pesistent memoies and high capacity DRAM memoy on DDR4 and DDR5 channels It extends the DDR potocol to enable tansactional access Host is decoupled fom the media Multiple media types suppoted Suppots any latency (ns ~ us) JEDEC specification publication in Stoage Develope Confeence EMEA. All Rights Reseved. 22

23 NVDIMM-N Applications In Memoy Database: Jounaling, educed ecovey time, Ex-lage tables Taditional Database: Log acceleation by wite combining and caching Entepise Stoage: Tieing, caching, wite buffeing and meta data stoage Vitualization: Highe VM consolidation with geate memoy density High-Pefomance Computing: Check point acceleation and/o elimination 2018 Stoage Develope Confeence EMEA. All Rights Reseved

24 NVDIMM-N Use Case File System Tansaction Log File System With diect access to ecods this emoves disk IO and all the ovehead that involves Tansaction Log I/O on M-Map Device Block Device Laye SAS/SATA/Fibe Channel Laye NVDIMM-N NVDIMM-N NVDIMM-N Data Dive Data Dive Data Dive A tansaction log is a histoy of actions executed by a DBMS used to guaantee Atomicity, Consistency, Isolation, and Duability (ACID) ove a hadwae failue. When these logs can be stoed in NVDIMMs vs stoage then system pefomance can be damatically impoved Stoage Develope Confeence EMEA. All Rights Reseved. 24

25 Linux Kenel 4.4+ NVDIMM-N OS Suppot Linux subsystems added suppot of NVDIMMs NVDIMM modules pesented as device links: /dev/pmem0, /dev/pmem1 QEMO suppot (expeimental) XFS-DAX and EXT4-DAX available DAX BTT (Block, Atomic) PMEM BLK File system extensions to bypass the page cache and block laye to memoy map pesistent memoy, fom a PMEM block device, diectly into a pocess addess space. Block Tanslation Table: Pesistent memoy is byte addessable. Existing softwae may have an expectation that the powe-fail-atomicity of wites is at least one secto, 512 bytes. The BTT is an indiection table with atomic update semantics to font a PMEM/BLK block device dive and pesent abitay atomic secto sizes. A system-physical-addess ange whee wites ae pesistent. A block device composed of PMEM is capable of DAX. A PMEM addess ange may span an inteleave of seveal DIMMs. A set of one o moe pogammable memoy mapped apetues povided by a DIMM to access its media. This indiection pecludes the pefomance benefit of inteleaving, but enables DIMM-bounded failue modes Stoage Develope Confeence EMEA. All Rights Reseved. 25

26 Windows NVDIMM-N OS Suppot Windows Seve 2016 suppots DDR4 NVDIMM-N Block Mode No code change, fast I/O device (4K sectos) Still have softwae ovehead of I/O path Diect Access Achieve full pefomance potential of NVDIMM using memoy-mapped files on Diect Access volumes (NTFS- DAX) No I/O, no queueing, no async eads/wites Moe info on Windows NVDIMM-N suppot: Stoage Develope Confeence EMEA. All Rights Reseved. 26

27 Application Benefits Windows Example Tail of Log in SQL 2016 Wites updates to SQL log though pesistent memoy fist Uses memoy instuctions to issue log updates to pesistent memoy diectly Utilizes memoy-mapped files on NTFS Diect Access (DAX) volume Souce; Micosoft 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 27

28 Summay The NVM Pogamming Model is pefect fo NVDIMMs Block and File mode atomicity featues PM Mode memoy mapped stoage Use the NVM pogamming model with NVDIMMs Enable a path fowad fo applications Lead the way to innovation in NVM optimized softwae 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 28

29 SNIA-at-a-Glance 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 29

30 Join Ou Wok in Access technology visionaies and leading companies in the industy Collaboate to guide technology stategic & technical diections Influence Industy messaging and best pactices to be included in open calls on technology topics and to lean moe about how you company can join SNIA, the SSSI, and a SNIA Regional Affiliate 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 30

31 Thank You! Visit snia.og/pm fo the latest on SNIA Pesistent Memoy activities 2018 Stoage Develope Confeence EMEA. All Rights Reseved. 31