I want to build on Rick Coulson s talk earlier this morning that addressed emerging Persistent Memory technologies. I want to expand on implications

Transcription

1 I want to build on Rick Coulson s talk earlier this morning that addressed emerging Persistent Memory technologies. I want to expand on implications to the overall Cloud and Enterprise server storage architecture and then dig into the disruptive opportunities that will be created for Storage and Data Management Software as Memory becomes the new Hot Storage Tier. 1

2 As Rick showed, Persistent Memory technology, including Intel s 3D Cross-Point technology is technology is released in a DIMM, it will bring multiple TB of Load/Store accessible Non-Volatile memory subystem at sub-microsecond latencies and one of the ways these latencies are achei NVM Load/Store accessible, eliminating the I/O operation. This placement of data in-memory to avoid the I/O is not an entirely new concept. It aligns with server applications to move more hot data into memory. Examples include the memcached an Value services commonly used in cloud architectures and real-time analytics applications like H MongoDB, the next generation architecture announced by Cloudera, and many others. Where do we see this convergence of High-Capacity, Persistent Memory and In-Memory applic terms of the overall cloud and enterprise storage architecture? 2

3 We think this leads to a merging of the Working Memory and Hot Storage tiers into a combined layer I m calling In-Place Working Storage Today, despite the IMDB trend, most applications still use the decade- old programming model that we all learned in a Programming 101 class. That model is based on placing working data in DRAM and then doing I/O Operations to move data to/from the storage tiers. The future that is implied in the SNIA NVM Programming Model, as has been presented by Andy Rudoff from Intel, Neal Christensen from Microsoft, among others, is a future where the I/O operation is eliminated for performance critical access to hot data. This, of course, offers tremendous Performance and QoS benefits for applications such as real-time analytics, creating a highly disruptive shift for the Multi Billion $ market for data services software that for decades has operated by hooking into various points in the I/O operation. Intercepting that movement of data between working memory and storage. The question, and the disruptive opportunity for SW innovators is: How are those necessary data services and compliance features applied in this new In- Place Working Storage tier? As a background, how does this work in today s 3

4 This picture shows an example of a real-world Street-Legal storage SW stack with examples of the Data Services and Data Compliance features required to meet various Data SLAs. Years ago it was more common to implement these functions in the application server SW stack. For years there has been a trend to move these out to NAS or SAN devices and with today s trend to DAS NVM and Hyperconvergence, that is driving a trend back to the host SW stack in various forms of filter or layered drivers. A first step to getting ready for NVDIMMs and next generation NVM is various optimizations in this stack. NVM Express is just one of those optimizations. But for a street-legal server, that is still a ways from the Direct, sub-us access latecy showed earlier. 4

5 The SNIA NVM Programming Model combined with OS implementations such as DAX and Linux and Windows SCM architecture Neal Christensen has presented enables the ability to map the Persistent Memory directly into the application s address space creating opportunity for a new, optimized I/O architecture. This architecture could leverage user-mode libraries that reduce the disruptive aspect of the Persistent Memory by retaining the familiar I/O model for applications while taking a significant step towards improving SW overhead on Persistent Memory. Such libraries might present new APIs or emulate traditional File APIs transparently for applications. Similar to the way File Filter drivers insert data services such as Snapshots or Tiering, Libraries might implement similar functionality in a user mode library using direct load/store access to Persistent Memory underneath. This is just one example of an area for storage SW innovation. I m sure there are many others. 5

6 Again, the ultimate end-state is In-Place Working Storage 6

7 The question and opportunity for SW innovators is what are the new Software Layers for In-Place Working Storage where the I/O has been eliminated for the Hot Data. 7

8 We don t have the answer, but we are kick-starting innovation with a set of Open Source NVM Libraries located at This is an open source project so anyone can join in. With these libraries, we are starting with the foundational functionily applications will need to work directly with the persistent media. These include basic persistent memory space management, flushing data to insure persistence, and functions to insure data is consistently and transactionaly stored on the media something kernel-based File Systems and I/O drivers have traditionally handled for applications. 8

9 One of the concepts supported by NVML is the concept of Persist Points. This concept is similar to traditional APIs such as msync() in Unix or FlushViewOfFile() in Windows where an application decides its working data is consistent and ready to be pushed out to the storage media. The applicationbased NVM Library equivalent is pmem_persist(). Although not as strict a control point as kernel-based File Systems/Filters, (remember applications have direct load/store access to the Persistent Memory) it may provide a Beachhead for inserting new data services for Persistent Memory. 9

10 So join the collaboration to innovate the new software architecture for In-Place Working Storage! 10