APIs for Persistent Memory Programming
|
|
- Austen Hicks
- 5 years ago
- Views:
Transcription
1 APIs for Persistent Memory Programming MSST 2018 Andy Rudoff NVM Software Architect Intel Corporation Data Center Group
2 A Full-Stack Example Using a key-value store as an example App Unmodified App, uses KV APIs KV Store PCJ Use Java containers to create pmem-aware KV Store Caller just sees the same APIs, uses them as before Provide Java Containers libpmemobj Provide transactions, persistent memory allocator libpmem Abstract away hardware details pmem-aware File System Expose Persistent Memory as Memory-mapped files (DAX) Persistent Memory Data Center Group 2
3 Another Full-Stack Example The app does everything App pmem-aware application Handles its own allocations, transactions, etc pmem-aware File System Expose Persistent Memory as Memory-mapped files (DAX) Persistent Memory Data Center Group 3
4 Transparency Levels for pmem application Application middleware libraries User file systems File System in-kernel usages block storage Driver Kernel Increasing barrier to adoption Persistent Memory Data Center Group 4
5 Transparency Levels for pmem application Application middleware libraries User file systems File System in-kernel usages block storage Driver Kernel Increasing leverage Persistent Memory Data Center Group 5
6 Ancient History June 2012 Formed the NVM Programming TWG Immediate participation from key OSVs, ISVs, IHVs January 2013 Held the first PM Summit (actually called NVM Summit ) January 2014 TWG published rev 1.0 of the NVM Programming Model June 2017 Rev 1.2 published And now Programming model supported & shipping in multiple operating systems APIs built on top of the programming model available 6 Data Center Group 6
7 The SNIA NVM Programming Model Mgmt. Storage File Memory Management UI Application Application Application Management Library Standard Raw Device Access Standard File API Standard File API Load/Store User File System PM-Aware File System MMU Mappings NVDIMM Driver Kernel NVDIMMs 7 Data Center Group 7
8 Must Open File Before Mapping Standard Naming and Permission Model Standard File API Application Load/Store User PM-Aware File System MMU Mappings Kernel NVDIMMs 8 Data Center Group 8
9 Direct Access Definition: no paging, no page cache use Application Standard File API Load/Store User DAX PM-Aware File System MMU Mappings Kernel NVDIMMs 9 Data Center Group 9
10 Direct Access Standard File API Application Load/Store PM-Aware File System MMU Mappings User DAX Windows: DAX Support is shipping NTFS is PM-Aware Some new APIs PMDK support Kernel NVDIMMs 10 Data Center Group 10
11 Direct Access Standard File API Application Load/Store PM-Aware File System MMU Mappings User Kernel DAX Linux: DAX Support is shipping ext4 is PM-Aware XFS is PM-Aware PMDK support More filesystems coming NVDIMMs 11 Data Center Group 11
12 Direct Access Standard File API Application Load/Store PM-Aware File System MMU Mappings User DAX Virtualization of PM: VMware Hyper-V KVM Xen Kernel NVDIMMs 12 Data Center Group 12
13 Applications: Public Demos 2017 was an interesting year for demos SAP SAPPHIRE Oracle OpenWorld Built on the Persistent Memory programming model! 13 Data Center Group 13
14 Intel Persistent Memory New Type of Memory Persistent, Large Capacity & Byte Addressable 6 TB per two-socket system DDR4 Socket Compatible Can Co-exist with Conventional DDR4 DRAM DIMMs Cheaper than DRAM Availability 2018 Data Center Group 14
15 trivial.c fd = open(filename, O_RDWR); pmaddr = mmap(null, 4096, PROT_READ PROT_WRITE, MAP_SHARED, fd, 0); close(fd); strcpy(pmaddr, "Hello, Persistent Memory!"); msync((void *)pmaddr, 4096, MS_SYNC); 15 Data Center Group 15
16 trivial.c fd = open(filename, O_RDWR); pmaddr = mmap(null, 4096, PROT_READ PROT_WRITE, MAP_SHARED, fd, 0); close(fd); strcpy(pmaddr, "Hello, Persistent Memory!"); msync((void *)pmaddr, 4096, MS_SYNC); pmaddr could point to a really huge capacity terabytes! Want some allocator like malloc/free/new or language integration strcpy is not atomic msync is not atomic Basic memory-mapped files are not transactional up to the caller 16 Data Center Group 16
17 Also, you should know msync now just flushes CPU caches no page cache with DAX Platforms may support Optimized Flush Flush changes from using user space instructions for performance Windows supports this Linux supports this with new MAP_SYNC flag Future platforms may have persistent CPU caches ACPI property tells you this Write future-proof code by looking at this property & skipping flushes Persistent Memory errors appear as memory errors For example: SIGBUS on an uncorrectable in Linux Data Center Group 17
18 The Persistent Memory Development Kit PMDK PMDK is a collection of libraries Developers pull only what they need Low level programming support Transaction APIs Fully validated Performance tuned. Open Source & Product neutral Data Center Group 18
19 PMDK Libraries C++ C PCJ* Python Support for volatile memory usage libmemkind In Development * Persistent Collections for Java Interface to create a persistent memory resident log file libpmemlog Low level support for local persistent memory libpmem Interface for persistent memory allocation, transactions and general facilities libpmemobj Low level support for remote access to persistent memory librpmem Interface to create arrays of pmemresident blocks, of same size, atomically updated libpmemblk Data Center Group 19
20 Also, you should know PMDK libraries are validated to product quality Many hundreds of unit tests Many hundreds of system tests We don t think we re all done now Performance work continues, with some significant results Feature development continues RAS More mature language integration (especially around C++) More mature replication Still adding libraries Example: libpmemkv Data Center Group 20
21 PMDK in a Nutshell Ten libraries, tools, examples void push(pool_base &pop, uint64_t value) { transaction::exec_tx(pop, [&] { auto n = make_persistent<pmem_entry>(); } 21 }); n->value = value; n->next = nullptr; if (head == nullptr) { head = tail = n; } else { tail->next = n; tail = n; } Data Center Group 21
22 PMDK in a Nutshell void push(pool_base &pop, uint64_t value) { transaction::exec_tx(pop, [&] { auto n = make_persistent<pmem_entry>(); } 22 }); n->value = value; n->next = nullptr; if (head == nullptr) { head = tail = n; } else { tail->next = n; tail = n; } Memory Allocations & Frees Data Center Group 22
23 PMDK in a Nutshell void push(pool_base &pop, uint64_t value) { transaction::exec_tx(pop, [&] { auto n = make_persistent<pmem_entry>(); } 23 }); n->value = value; n->next = nullptr; if (head == nullptr) { head = tail = n; } else { tail->next = n; tail = n; } Multiple Operations Made Atomic Data Center Group 23
24 PMDK in a Nutshell Complex transactions, allocation handled by libraries No flush calls to manage in most cases Each ISV doesn t have to re-invent Performance tuned (esp for future enhancements) Licensing is very liberal Steal all the code you want! PMDK is a convenience, not a requirement Build your own library if you like! 24 Data Center Group 24
25 Persistent Collections for Java... PersistentIntArray data = new PersistentIntArray(1024); ObjectDirectory.put("My_fancy_persistent_array", data); // no serialization data.set(0, 123);... No flush calls. Transactional. Java library handles it all. See pilot project at: 25 Data Center Group 25
26 A Full-Stack Example Using a key-value store as an example App Unmodified App, uses KV APIs KV Store PCJ Use Java containers to create pmem-aware KV Store Caller just sees the same APIs, uses them as before Provide Java Containers libpmemobj Provide transactions, persistent memory allocator libpmem Abstract away hardware details pmem-aware File System Expose Persistent Memory as Memory-mapped files (DAX) Persistent Memory Data Center Group 26
27 SNIA TWG Ongoing Work Security PM Hardware Security Threat Model (balloting) Remote persistent memory (via RDMA) Ongoing optimizations for RDMA worked in multiple forums Remote asynchronous flush (under discussion) Higher-level Semantics As we learn more.. 27 Data Center Group 27
28 More Information Specs, workgroups, webcasts, videos, presentations PMDK and other persistent memory programming information Links to publications, standards, Windows & Linux info Intel Developer Zone for persistent memory programming 28 Data Center Group 28
Impact on Application Development: SNIA NVM Programming Model in the Real World. Andy Rudoff pmem SW Architect, Intel
Impact on Development: SNIA NVM Programming Model in the Real World Andy Rudoff pmem SW Architect, Intel Agenda What everyone already knows about pmem What everyone forgets Ways to use pmem with no app
More informationPersistent Memory: The Value to HPC and the Challenges
Persistent Memory: The Value to HPC and the Challenges November 12, 2017 Andy Rudoff Principal Engineer, NVM Software Intel Corporation Data Center Group Intel Persistent Memory New Type of Memory Persistent,
More informationThe SNIA NVM Programming Model: Latest Developments and Challenges. Andy Rudoff, Intel Corporation
The SNIA NVM Programming Model: Latest Developments and Challenges Andy Rudoff, Intel Corporation Programming Model Four meanings (at least) 2 Programming Model: SW Interface to HW Core L1 L1 L2 Core L1
More informationTHE IN-PLACE WORKING STORAGE TIER OPPORTUNITIES FOR SOFTWARE INNOVATORS KEN GIBSON, INTEL, DIRECTOR MEMORY SW ARCHITECTURE
THE IN-PLACE WORKING STORAGE TIER OPPORTUNITIES FOR SOFTWARE INNOVATORS KEN GIBSON, INTEL, DIRECTOR MEMORY SW ARCHITECTURE I/O LATENCY WILL SOON EXCEED MEDIA LATENCY 30 NVM Tread 25 NVM xfer Controller
More informationWindows Support for PM. Tom Talpey, Microsoft
Windows Support for PM Tom Talpey, Microsoft Agenda Industry Standards Support PMDK Open Source Support Hyper-V Support SQL Server Support Storage Spaces Direct Support SMB3 and RDMA Support 2 Windows
More informationWindows Support for PM. Tom Talpey, Microsoft
Windows Support for PM Tom Talpey, Microsoft Agenda Windows and Windows Server PM Industry Standards Support PMDK Support Hyper-V PM Support SQL Server PM Support Storage Spaces Direct PM Support SMB3
More informationIN-PERSISTENT-MEMORY COMPUTING WITH JAVA ERIC KACZMAREK INTEL CORPORATION
IN-PERSISTENT-MEMORY COMPUTING WITH JAVA ERIC KACZMAREK INTEL CORPORATION LEGAL DISCLAIMER & OPTIMIZATION NOTICE INFORMATION IN THIS DOCUMENT IS PROVIDED AS IS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL
More informationThe SNIA NVM Programming Model. #OFADevWorkshop
The SNIA NVM Programming Model #OFADevWorkshop Opportunities with Next Generation NVM NVMe & STA SNIA 2 NVM Express/SCSI Express: Optimized storage interconnect & driver SNIA NVM Programming TWG: Optimized
More informationUpdate on Windows Persistent Memory Support Neal Christiansen Microsoft
Update on Windows Persistent Memory Support Neal Christiansen Microsoft 1 Agenda What is Persistent Memory (PM) Review: Existing Windows PM Support What s New New PM APIs Large Page Support Hyper-V Support
More informationPersistent Memory over Fabric (PMoF) Adding RDMA to Persistent Memory Pawel Szymanski Intel Corporation
Persistent Memory over Fabric (PMoF) Adding RDMA to Persistent Memory Pawel Szymanski Intel Corporation 1 Adding RDMA to Persisteny memory Agenda PMoF Overview Comparison with other remote replication
More informationRDMA Requirements for High Availability in the NVM Programming Model
RDMA Requirements for High Availability in the NVM Programming Model Doug Voigt HP Agenda NVM Programming Model Motivation NVM Programming Model Overview Remote Access for High Availability RDMA Requirements
More informationPersistent Memory, NVM Programming Model, and NVDIMMs. Presented at Storage Field Day June 15, 2017
Persistent Memory, NVM Programming Model, and NVDIMMs Presented at Storage Field Day June 15, 2017 Contents Introductions Persistent Memory Overview NVM Programming Model NVDIMM 2 Speakers Rob Peglar,
More informationPM Support in Linux and Windows. Dr. Stephen Bates, CTO, Eideticom Neal Christiansen, Principal Development Lead, Microsoft
PM Support in Linux and Windows Dr. Stephen Bates, CTO, Eideticom Neal Christiansen, Principal Development Lead, Microsoft Windows Support for Persistent Memory 2 Availability of Windows PM Support Client
More informationAccessing NVM Locally and over RDMA Challenges and Opportunities
Accessing NVM Locally and over RDMA Challenges and Opportunities Wendy Elsasser Megan Grodowitz William Wang MSST - May 2018 Emerging NVM A wide variety of technologies with varied characteristics Address
More informationFlash Memory Summit Persistent Memory - NVDIMMs
Flash Memory Summit 2018 Persistent Memory - NVDIMMs Contents Persistent Memory Overview NVDIMM Conclusions 2 Persistent Memory Memory & Storage Convergence Today Volatile and non-volatile technologies
More informationPERSISTENT MEMORY PROGRAMMING
14th ANNUAL WORKSHOP 2018 PERSISTENT MEMORY PROGRAMMING THE REMOTE ACCESS PERSPECTIVE Tom Talpey, Architect Microsoft April 10, 2018 OUTLINE SNIA NVMP Programming Model PMEM Remote Access considerations
More informationSNIA NVM Programming Model Workgroup Update. #OFADevWorkshop
SNIA NVM Programming Model Workgroup Update #OFADevWorkshop Persistent Memory (PM) Vision Fast Like Memory PM Brings Storage PM Durable Like Storage To Memory Slots 2 Latency Thresholds Cause Disruption
More informationWhat You can Do with NVDIMMs. Rob Peglar President, Advanced Computation and Storage LLC
What You can Do with NVDIMMs Rob Peglar President, Advanced Computation and Storage LLC A Fundamental Change Requires An Ecosystem Windows Server 2016 Windows 10 Pro for Workstations Linux Kernel 4.2 and
More informationUsing persistent memory and RDMA for Ceph client write-back caching Scott Peterson, Senior Software Engineer Intel
Using persistent memory and RDMA for Ceph client write-back caching Scott Peterson, Senior Software Engineer Intel 2018 Storage Developer Conference. Intel Corporation. All Rights Reserved. 1 Ceph Concepts
More informationApplication Access to Persistent Memory The State of the Nation(s)!
Application Access to Persistent Memory The State of the Nation(s)! Stephen Bates, Paul Grun, Tom Talpey, Doug Voigt Microsemi, Cray, Microsoft, HPE The Suspects Stephen Bates Microsemi Paul Grun Cray
More informationUsing persistent memory to
Using persistent memory to Talk build Title a high-performant, Here fully Author Name, Company user space filesystem Krzysztof Czuryło, Intel What it is? pmemfile Low-overhead userspace implementation
More informationUsing NVDIMM under KVM. Applications of persistent memory in virtualization
Using NVDIMM under KVM Applications of persistent memory in virtualization Stefan Hajnoczi About me QEMU contributor since 2010 Focus on storage, tracing, performance Work in Red
More informationBuilding on The NVM Programming Model A Windows Implementation
Building on The NVM Programming Model A Windows Implementation Chandra Konamki Sr Software Engineer, Microsoft Paul Luse Principal Engineer, Intel Open NVM Programming Model NVML Overview Abstraction Value
More informationI want to build on Rick Coulson s talk earlier this morning that addressed emerging Persistent Memory technologies. I want to expand on implications
I want to build on Rick Coulson s talk earlier this morning that addressed emerging Persistent Memory technologies. I want to expand on implications to the overall Cloud and Enterprise server storage architecture
More informationPersistent Memory and Media Errors
Persistent Memory and Media Errors Vishal Verma vishal.l.verma@intel.com Vault 2016 1 Or How to have your Poison and (not) consume it too 2 NVDIMM software stack Regular Block IO Application Standard Raw
More informationPersistent Memory over Fabrics
Persistent Memory over Fabrics Rob Davis, Mellanox Technologies Chet Douglas, Intel Paul Grun, Cray, Inc Tom Talpey, Microsoft Santa Clara, CA 1 Agenda The Promise of Persistent Memory over Fabrics Driving
More informationCapabilities and System Benefits Enabled by NVDIMM-N
Capabilities and System Benefits Enabled by NVDIMM-N Bob Frey Arthur Sainio SMART Modular Technologies August 7, 2018 Santa Clara, CA 1 NVDIMM-N Maturity and Evolution If there's one takeaway you should
More informationREMOTE PERSISTENT MEMORY ACCESS WORKLOAD SCENARIOS AND RDMA SEMANTICS
13th ANNUAL WORKSHOP 2017 REMOTE PERSISTENT MEMORY ACCESS WORKLOAD SCENARIOS AND RDMA SEMANTICS Tom Talpey Microsoft [ March 31, 2017 ] OUTLINE Windows Persistent Memory Support A brief summary, for better
More informationRemote Persistent Memory SNIA Nonvolatile Memory Programming TWG
Remote Persistent Memory SNIA Nonvolatile Memory Programming TWG Tom Talpey Microsoft 2018 Storage Developer Conference. SNIA. All Rights Reserved. 1 Outline SNIA NVMP TWG activities Remote Access for
More informationThe Slow NVM (R)evolution in the context of Next Generation Postprocessing
The Slow NVM (R)evolution in the context of Next Generation Postprocessing 5. December 2016, Stuttgart Workshop on Sustained Simulation Performance Erich Focht NEC HPC Europe The K Problem Two things
More informationPlanning For Persistent Memory In The Data Center. Sarah Jelinek/Intel Corporation
Planning For Persistent Memory In The Data Center Sarah Jelinek/Intel Corporation SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies
More informationNVDIMM Overview. Technology, Linux, and Xen
NVDIMM Overview Technology, Linux, and Xen Who am I? What are NVDIMMs? A standard for allowing NVRAM to be exposed as normal memory Potential to dramatically change the way software is written But.. They
More informationPaving the Way to the Non-Volatile Memory Frontier. PRESENTATION TITLE GOES HERE Doug Voigt HP
Paving the Way to the Non-Volatile Memory Frontier PRESENTATION TITLE GOES HERE Doug Voigt HP What is happening to the relationship between storage and memory? 2014 Data Storage Innovation Conference.
More informationExtending RDMA for Persistent Memory over Fabrics. Live Webcast October 25, 2018
Extending RDMA for Persistent Memory over Fabrics Live Webcast October 25, 2018 Today s Presenters John Kim SNIA NSF Chair Mellanox Tony Hurson Intel Rob Davis Mellanox SNIA-At-A-Glance 3 SNIA Legal Notice
More informationLoad-Sto-Meter: Generating Workloads for Persistent Memory Damini Chopra, Doug Voigt Hewlett Packard (Enterprise)
Load-Sto-Meter: Generating Workloads for Persistent Memory Damini Chopra, Doug Voigt Hewlett Packard (Enterprise) Application vs. Pure Workloads Benchmarks that reproduce application workloads Assist in
More informationMicrosoft SMB Looking Forward. Tom Talpey Microsoft
Microsoft SMB Looking Forward Tom Talpey Microsoft Outline A look at SMB3 today A look at things in the works in Windows The SMB1 situation Other uses of SMB3 sambaxp 2018 Göttingen 2 SMB3 Today SMB3 is
More informationWindows Persistent Memory Support
Windows Persistent Memory Support Neal Christiansen Microsoft Agenda Review: Existing Windows PM Support What s New New PM APIs Large & Huge Page Support Dax aware Write-ahead LOG Improved Driver Model
More informationThe Impact of the NVM Programming Model Andy Rudoff Intel Corporation
The Impact of the NVM Programming Model Andy Rudoff Intel Corporation The Impact of the NVM Programming Model and Persistent Memory! Andy Rudoff Intel Corporation Questions Why now? Basic programming model
More informationManaging Persistent Memory Tiffany Kasanicky Intel
Managing Persistent Memory Tiffany Kasanicky Intel 1 Agenda Managing* Be in charge of; administer; run. The process of dealing with or controlling things or people. Persistent Memory Memory that retains
More informationVMware vsphere Virtualization of PMEM (PM) Richard A. Brunner, VMware
VMware vsphere Virtualization of PMEM (PM) Richard A. Brunner, VMware Disclaimer This presentation may contain product features that are currently under development. This overview of new technology represents
More informationJim Pappas Director of Technology Initiatives, Intel Vice-Chair, Storage Networking Industry Association (SNIA) December 07, 2018
Jim Pappas Director of Technology Initiatives, Intel Vice-Chair, Storage Networking Industry Association (SNIA) December 07, 2018 jim@intel.com 1 How did this Effort Start? Memristor MRAM Carbon Nanotube
More informationPMEM Software and Applications
PMEM-201-1 Software and Applications Chairs: Arthur Sainio, SMART Modular Jeff Chang, AgigA Tech Co-Chairs, SNIA Persistent Memory and NVDIMM SIG Sponsored by: 1 Speakers Part 1 Doug Voigt, HPE The SNIA
More informationApplications Take Advantage of Persistent Memory
Applications Take Advantage of Persistent Memory A BrightTalk webcast by the SNIA Persistent Memory & NVDIMM Special Interest Group January 15, 2019 10:00 am PT/1:00 pm ET Today s Presenters Raghu Kulkarni
More informationPersistent Memory Over Fabrics. Paul Grun, Cray Inc Stephen Bates, Eideticom Rob Davis, Mellanox Technologies
Persistent Memory Over Fabrics Paul Grun, Cray Inc Stephen Bates, Eideticom Rob Davis, Mellanox Technologies Agenda Persistent Memory as viewed by a consumer, and some guidance to the fabric community
More informationNew types of Memory, their support in Linux and how to use them with RDMA
14 th ANNUAL WORKSHOP 2018 New types of Memory, their support in Linux and how to use them with RDMA Christoph Lameter, Ph.D., R&D Team Lead Jump Trading LLC April 5, 2018 Overview Why talk about memory?
More informationUpcoming advances in Non-Volatile Memory (NVM) technologies
Programming Models for Emerging Non-Volatile Memory Technologies ANDY RUDOFF Andy Rudoff is an Enterprise Storage Architect at Intel. He has more than 25 years of experience in operating systems internals,
More informationHow Next Generation NV Technology Affects Storage Stacks and Architectures
How Next Generation NV Technology Affects Storage Stacks and Architectures Marty Czekalski, Interface and Emerging Architecture Program Manager, Seagate Technology Flash Memory Summit 2013 Santa Clara,
More informationNon-Volatile Memory Through Customized Key-Value Stores
Non-Volatile Memory Through Customized Key-Value Stores Leonardo Mármol 1 Jorge Guerra 2 Marcos K. Aguilera 2 1 Florida International University 2 VMware L. Mármol, J. Guerra, M. K. Aguilera (FIU and VMware)
More informationDesign Considerations When Implementing NVM
Design Considerations When Implementing NVM Jim Pinkerton Architect Microsoft Windows Server 1/29/2012 Why is NVM Interesting to Microsoft? New levels of performance for applications & OS Lower storage
More informationSMB3 Extensions for Low Latency. Tom Talpey Microsoft May 12, 2016
SMB3 Extensions for Low Latency Tom Talpey Microsoft Problem Statement Storage Class Memory A new, disruptive class of storage Nonvolatile medium with RAM-like performance Low latency, high throughput,
More informationNVDIMM-N Cookbook: A Soup-to-Nuts Primer on Using NVDIMM-Ns to Improve Your Storage Performance
NVDIMM-N Cookbook: PRESENTATION TITLE GOES HERE A Soup-to-Nuts Primer on Using NVDIMM-Ns to Improve Your Storage Performance Jeff Chang VP Marketing and Business Development, AgigA Tech Arthur Sainio Director
More informationJanuary 28-29, 2014 San Jose
January 28-29, 2014 San Jose Flash for the Future Software Optimizations for Non Volatile Memory Nisha Talagala, Lead Architect, Fusion-io Gary Orenstein, Chief Marketing Officer, Fusion-io @garyorenstein
More informationFlavors of Memory supported by Linux, their use and benefit. Christoph Lameter, Ph.D,
Flavors of Memory supported by Linux, their use and benefit Christoph Lameter, Ph.D, Twitter: @qant Flavors Of Memory The term computer memory is a simple term but there are numerous nuances
More informationNVMe SSDs with Persistent Memory Regions
NVMe SSDs with Persistent Memory Regions Chander Chadha Sr. Manager Product Marketing, Toshiba Memory America, Inc. 2018 Toshiba Memory America, Inc. August 2018 1 Agenda q Why Persistent Memory is needed
More informationHow Might Recently Formed System Interconnect Consortia Affect PM? Doug Voigt, SNIA TC
How Might Recently Formed System Interconnect Consortia Affect PM? Doug Voigt, SNIA TC Three Consortia Formed in Oct 2016 Gen-Z Open CAPI CCIX complex to rack scale memory fabric Cache coherent accelerator
More informationPersistent Memory in Linux. Tom Coughlan (with Thanks to Jeff Moyer), Red Hat
Persistent Memory in Linux Tom Coughlan (with Thanks to Jeff Moyer), Red Hat Since our last meeting... Fedora 23 shipped with pmem support (min. kernel version 4.4) RHEL 7.3 shipped with Full support for
More informationNVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory
NVMFS: A New File System Designed Specifically to Take Advantage of Nonvolatile Memory Dhananjoy Das, Sr. Systems Architect SanDisk Corp. 1 Agenda: Applications are KING! Storage landscape (Flash / NVM)
More informationRemote Persistent Memory With Nothing But Net Tom Talpey Microsoft
Remote Persistent Memory With Nothing But Net Tom Talpey Microsoft 1 Outline Aspiration RDMA NIC as a Persistent Memory storage adapter Steps to there: Flush Write-after-flush Integrity Privacy QoS Some
More informationFile Systems: Consistency Issues
File Systems: Consistency Issues File systems maintain many data structures Free list/bit vector Directories File headers and inode structures res Data blocks File Systems: Consistency Issues All data
More informationStorage and Memory Hierarchy in HPC: New Paradigm and New Solutions with Intel Dr. Jean-Laurent Philippe
Storage and Memory Hierarchy in HPC: New Paradigm and New Solutions with Intel Dr. Jean-Laurent Philippe Senior EMEA HPC Technical Specialist Intel Data Center Group Legal Disclaimer Intel may make changes
More informationDistributed Shared Persistent Memory
Distributed Shared Persistent Memory (SoCC 17) Yizhou Shan, Yiying Zhang Persistent Memory (PM/NVM) Byte Addressable Persistent CPU Cache Low Latency Capacity Cost effective PM DRAM 2 Many PM Work, but
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
SER2734BU Extreme Performance Series: Byte-Addressable Nonvolatile Memory in vsphere VMworld 2017 Content: Not for publication Qasim Ali and Praveen Yedlapalli #VMworld #SER2734BU Disclaimer This presentation
More informationArchitectural Principles for Networked Solid State Storage Access
Architectural Principles for Networked Solid State Storage Access SNIA Legal Notice! The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted.! Member companies and individual
More informationCreating Storage Class Persistent Memory With NVDIMM
Creating Storage Class Persistent Memory With NVDIMM PAUL SWEERE Vice President, Engineering paul.sweere@vikingtechnology.com MEMORY/STORAGE HIERARCHY Data-Intensive Applications Need Fast Access To Storage
More informationA New Key-Value Data Store For Heterogeneous Storage Architecture
A New Key-Value Data Store For Heterogeneous Storage Architecture brien.porter@intel.com wanyuan.yang@intel.com yuan.zhou@intel.com jian.zhang@intel.com Intel APAC R&D Ltd. 1 Agenda Introduction Background
More informationREMOTE PERSISTENT MEMORY THINK TANK
14th ANNUAL WORKSHOP 2018 REMOTE PERSISTENT MEMORY THINK TANK Report Out Prepared by a cast of thousands April 13, 2018 THINK TANK ABSTRACT Challenge - Some people think that Remote Persistent Memory over
More informationFujitsu SC 18. Human Centric Innovation Co-creation for Success 2018 FUJITSU
Fujitsu SC 18 Human Centric Innovation Co-creation for Success 2018 FUJITSU A Breakthrough in Non-Volatile Memory Technology & 2018 FUJITSU IT needs to accelerate time-to-market Situation: End users and
More informationKaladhar Voruganti Senior Technical Director NetApp, CTO Office. 2014, NetApp, All Rights Reserved
Kaladhar Voruganti Senior Technical Director NetApp, CTO Office Storage Used to Be Simple DRAM $$$ DISK Nearline TAPE volatile persistent Access Latency 2 Talk Focus: Persistent Memory Design Center DRAM
More informationStrata: A Cross Media File System. Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, Thomas Anderson
A Cross Media File System Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, Thomas Anderson 1 Let s build a fast server NoSQL store, Database, File server, Mail server Requirements
More informationDisclaimer This presentation may contain product features that are currently under development. This overview of new technology represents no commitme
FUT3040BU Storage at Memory Speed: Finally, Nonvolatile Memory Is Here Rajesh Venkatasubramanian, VMware, Inc Richard A Brunner, VMware, Inc #VMworld #FUT3040BU Disclaimer This presentation may contain
More informationNEXTGenIO Performance Tools for In-Memory I/O
NEXTGenIO Performance Tools for In- I/O holger.brunst@tu-dresden.de ZIH, Technische Universität Dresden 22 nd -23 rd March 2017 Credits Intro slides by Adrian Jackson (EPCC) A new hierarchy New non-volatile
More informationProcesses COMPSCI 386
Processes COMPSCI 386 Elements of a Process A process is a program in execution. Distinct processes may be created from the same program, but they are separate execution sequences. call stack heap STACK
More informationAccelerating Microsoft SQL Server Performance With NVDIMM-N on Dell EMC PowerEdge R740
Accelerating Microsoft SQL Server Performance With NVDIMM-N on Dell EMC PowerEdge R740 A performance study with NVDIMM-N Dell EMC Engineering September 2017 A Dell EMC document category Revisions Date
More informationLive block device operations in QEMU
Live block device operations in QEMU Paolo Bonzini Red Hat Yokohama, June 2012 1 Outline What is QEMU? The QEMU block layer Live block operations Q&A 2 What is QEMU? A FAST! processor emulator Started
More informationEnabling Persistent Memory Use in Java. Steve Dohrmann Sr. Staff Software Engineer, Intel
Enabling Persistent Memory Use in Java Steve Dohrmann Sr. Staff Software Engineer, Intel Motivation Java is a very popular language on servers, especially for databases, data grids, etc., e.g. Apache projects:
More informationHewlett Packard Enterprise HPE GEN10 PERSISTENT MEMORY PERFORMANCE THROUGH PERSISTENCE
Hewlett Packard Enterprise HPE GEN10 PERSISTENT MEMORY PERFORMANCE THROUGH PERSISTENCE Digital transformation is taking place in businesses of all sizes Big Data and Analytics Mobility Internet of Things
More informationDemystifying Network Cards
Demystifying Network Cards Paul Emmerich December 27, 2017 Chair of Network Architectures and Services About me PhD student at Researching performance of software packet processing systems Mostly working
More informationJANUARY 20, 2016, SAN JOSE, CA PRESENTATION TITLE GOES HERE. NVDIMM Panel
JANUARY 20, 2016, SAN JOSE, CA PRESENTATION TITLE GOES HERE NVDIMM Panel Panel & Agenda Amit Golander Software Defined Memory (SDM) CTO and VP R&D, Plexistor A technology leadership expert with experience
More informationPCIe Storage Beyond SSDs
PCIe Storage Beyond SSDs Fabian Trumper NVM Solutions Group PMC-Sierra Santa Clara, CA 1 Classic Memory / Storage Hierarchy FAST, VOLATILE CPU Cache DRAM Performance Gap Performance Tier (SSDs) SLOW, NON-VOLATILE
More informationI/O Profiling Towards the Exascale
I/O Profiling Towards the Exascale holger.brunst@tu-dresden.de ZIH, Technische Universität Dresden NEXTGenIO & SAGE: Working towards Exascale I/O Barcelona, NEXTGenIO facts Project Research & Innovation
More informationExploiting the benefits of native programming access to NVM devices
Exploiting the benefits of native programming access to NVM devices Ashish Batwara Principal Storage Architect Fusion-io Traditional Storage Stack User space Application Kernel space Filesystem LBA Block
More informationOvercoming System Memory Challenges with Persistent Memory and NVDIMM-P
Overcoming System Memory Challenges with Persistent Memory and NVDIMM-P JEDEC Server Forum 2017 Bill Gervasi, Discobolus Designs Copyright 2017 Jonathan Hinkle, Lenovo Datacenter Research and Technology
More informationIntroduction to Operating Systems (Part III)
Introduction to Operating Systems (Part III) Amir H. Payberah amir@sics.se Amirkabir University of Technology (Tehran Polytechnic) Amir H. Payberah (Tehran Polytechnic) Introduction 1393/6/29 1 / 43 Operating
More informationPanel: Persistent Memory Adoption in Operating Systems (-ish) Moderator: Jeff Chang AgigA Tech, NVDIMM SIG Co-Chair
Panel: Persistent Memory Adoption in Operating Systems (-ish) Moderator: Jeff Chang AgigA Tech, NVDIMM SIG Co-Chair Meet The Experts Tom Talpey Architect Tom Coughlan Senior Engineering Manager Amit Golander
More informationMemory management. Single process. Multiple processes. How to: All memory assigned to the process Addresses defined at compile time
Memory management Single process All memory assigned to the process Addresses defined at compile time Multiple processes. How to: assign memory manage addresses? manage relocation? manage program grow?
More informationCS24: INTRODUCTION TO COMPUTING SYSTEMS. Spring 2018 Lecture 23
CS24: INTRODUCTION TO COMPUTING SYSTEMS Spring 208 Lecture 23 LAST TIME: VIRTUAL MEMORY Began to focus on how to virtualize memory Instead of directly addressing physical memory, introduce a level of indirection
More informationNVDIMM Block Window Driver Writer s Guide
NVDIMM Block Window Driver Writer s Guide Example NFIT-Based NVDIMM Block Window and Persistent Memory Interface Guide April July 20165 You may not use or facilitate the use of this document in connection
More informationSNIA s SSSI Solid State Storage Initiative. Jim Pappas Vice-Char, SNIA
SNIA s SSSI Solid State Storage Initiative Jim Pappas Vice-Char, SNIA jim@intel.com SNIA at a glance 160 unique member companies 3,500 active contributing members 50,000 IT end users & storage pros worldwide
More informationLinux Memory Management
Linux Memory Management An Overview Anshuman Khandual Contents Introduction Platform Memory MMU Virtual Memory Physical Memory Analyze Memory Performance Introduction Main Memory - A non persistent storage
More informationContiguous memory allocation in Linux user-space
Contiguous memory allocation in Linux user-space Guy Shattah, Christoph Lameter Linux Plumbers Conference 2017 Contents Agenda Existing User-Space Memory Allocation Methods Fragmented Memory vs Contiguous
More informationCS24: INTRODUCTION TO COMPUTING SYSTEMS. Spring 2015 Lecture 23
CS24: INTRODUCTION TO COMPUTING SYSTEMS Spring 205 Lecture 23 LAST TIME: VIRTUAL MEMORY! Began to focus on how to virtualize memory! Instead of directly addressing physical memory, introduce a level of
More informationNOVA-Fortis: A Fault-Tolerant Non- Volatile Main Memory File System
NOVA-Fortis: A Fault-Tolerant Non- Volatile Main Memory File System Jian Andiry Xu, Lu Zhang, Amirsaman Memaripour, Akshatha Gangadharaiah, Amit Borase, Tamires Brito Da Silva, Andy Rudoff (Intel), Steven
More informationAnalyzing I/O Performance on a NEXTGenIO Class System
Analyzing I/O Performance on a NEXTGenIO Class System holger.brunst@tu-dresden.de ZIH, Technische Universität Dresden LUG17, Indiana University, June 2 nd 2017 NEXTGenIO Fact Sheet Project Research & Innovation
More informationIBM PSSC Montpellier Customer Center. Blue Gene/P ASIC IBM Corporation
Blue Gene/P ASIC Memory Overview/Considerations No virtual Paging only the physical memory (2-4 GBytes/node) In C, C++, and Fortran, the malloc routine returns a NULL pointer when users request more memory
More informationvirtio-mem: Paravirtualized Memory
virtio-mem: Paravirtualized Memory, Edinburgh, Scotland David Hildenbrand Software Engineer 26. October 2018 AGENDA Memory Hot(un)plug and Ballooning * Basics Main Differences (Selected) Issues Hyper-V
More informationCHAPTER 16 - VIRTUAL MACHINES
CHAPTER 16 - VIRTUAL MACHINES 1 OBJECTIVES Explore history and benefits of virtual machines. Discuss the various virtual machine technologies. Describe the methods used to implement virtualization. Show
More informationECE 650 Systems Programming & Engineering. Spring 2018
ECE 650 Systems Programming & Engineering Spring 2018 Inter-process Communication (IPC) Tyler Bletsch Duke University Slides are adapted from Brian Rogers (Duke) Recall Process vs. Thread A process is
More informationA Breakthrough in Non-Volatile Memory Technology FUJITSU LIMITED
A Breakthrough in Non-Volatile Memory Technology & 0 2018 FUJITSU LIMITED IT needs to accelerate time-to-market Situation: End users and applications need instant access to data to progress faster and
More informationOperating System Services
CSE325 Principles of Operating Systems Operating System Services David Duggan dduggan@sandia.gov January 22, 2013 Reading Assignment 3 Chapter 3, due 01/29 1/23/13 CSE325 - OS Services 2 What Categories
More informationM 3 Microkernel-based System for Heterogeneous Manycores
M 3 Microkernel-based System for Heterogeneous Manycores Nils Asmussen MKC, 06/29/2017 1 / 35 Outline 1 Introduction 2 Data Transfer Unit 3 Prototype Platforms 4 M 3 5 Summary 2 / 35 Outline 1 Introduction
More information