Failure-atomic Synchronization-free Regions

Transcription

1 Failure-atomic Synchronization-free Regions Vaibhav Gogte, Stephan Diestelhorst $, William Wang $, Satish Narayanasamy, Peter M. Chen, Thomas F. Wenisch NVMW 2018, San Diego, CA 03/13/2018 $

2 Promise of persistent memory (PM) Performance Density Non-volatility Byte-addressable, load-store interface to storage 2

3 Persistent memory system Core- 1 Core- 2 Core- 3 Core- 4 L1 $ L1 $ L1 $ L1 $ LLC Recovery DRAM Persistent Memory (PM) Recovery can inspect the data-structures in PM to restore system to a consistent state

4 Memory persistency models Provide guarantees required for recoverable software Academia [Condit 09][Pelley 14][Joshi 15][Kolli 17] Industry [Intel 14][ARM 16] Express primitives to define state of the program after failure Govern ordering constraints on persists to PM Ensure failure atomicity for a group of persists 4

5 Semantics for failure-atomicity Assures that either all or none of the updates visible post failure Guaranteed by hardware, library or language implementations Design space for semantics Individual persists [Intel 14][ARM 16][Joshi 15] Outermost critical sections [Chakrabarti 14][Boehm 16] Transactions [Coburn 11][Volos 11] Do not provide clean failure semantics Have high performance overhead Do not compose well with general sync. primitives Existing mechanisms suffer a trade-off between programmability and performance 5

6 Contributions Failure-atomic Synchronization-free Regions (SFR) Extend clean semantics to post-failure recovery Employ existing synchronization primitives in C++ Propose failure-atomicity as a part of language implementation Build compiler pass that emits logging code for persistent accesses Propose two designs: Coupled-SFR and Decoupled-SFR Achieve 65% better performance over state-of-the-art tech. 6

7 Outline Design space for granularities of failure-atomicity Our proposal: Failure-atomic SFRs Coupled-SFR design Decoupled-SFR design Evaluation 7

8 Language-level persistency models [Chakrabarti 14][Kolli 17] Enables writing portable, recoverable software Extend language memory-model with persistency semantics Persistency model guarantees: Ordering: How can programmers order persists? Failure-atomicity: Which group of stores persist atomically? 8

9 Persist ordering Thread 1 Thread 2 Core- 1 L1.acq(); L1.rel(); A = 100; sw L1.acq(); L1.rel(); B = 200; Core- 2 L1 $ L1 $ LLC b St A < hb St B St A < p St B PM a Ascribe persist ordering using synchronization primitives in language 9

10 Why failure-atomicity? Task: Fill node and add to linked list, safely In-memory data Fence fillnewnode() updatetailptr() 10

11 Why failure-atomicity? Task: Fill node and add to linked list, safely In-memory data Atomic updatetailptr() fillnewnode() Failure-atomicity à Persistent memory programming easier 11

12 Granularity of failure-atomicity - I L1.lock(); x -= 100; y += 100; L2.lock(); a -= 100; b += 100; L2.unlock(); L1.unlock(); Individual persists [Condit 09][Pelley 14][Joshi 16][Kolli 17] Mechanisms ensure atomicity of individual persists Non-sequentially consistent state visible to recovery à Need additional custom logging 12

13 Granularity of failure-atomicity - II L1.lock(); x -= 100; y += 100; L2.lock(); a -= 100; b += 100; L2.unlock(); L1.unlock(); Outer critical sections [Chakrabarti 14][Boehm 16] Guarantees recovery to observe SC state à Easier to build recovery code Require complex dependency tracking between critical sections à > 2x performance cost 13

14 Our proposal: Failure-atomic SFRs Synchronization free regions (SFR) Thread regions delimited by synchronization operations or system calls Persistent state atomically moves from one sync. operation to the next l1.acq(); x -= 100; y += 100; l2.acq(); a -= 100; b += 100; l2.rel(); l1.rel(); SFR1 SFR2 14

15 Failure-atomicity of SFRs Persist SFRs in sequentially consistent order Extends clean SC semantics to post-failure recovery Allow hardware/compiler optimizations within SFR Persist ordering Synchronizing acquire and release ops in C++ Failure-atomicity Undo-logging for SFRs Two logging designs à Coupled-SFR and Decoupled-SFR 15

16 Undo-logging for SFRs createundolog (L) L1.acq(); SFR1 x = 100; L1.rel(); mutatedata (M) persistdata (P) Failureatomic SFR1 commitlog (C) Need to ensure the ordering of steps in undo-logging for SFRs to be failure-atomic 16

17 Design 1: Coupled-SFR Thread 1 Thread 2 Thread 1 Thread 2 L1.acq(); SFR1 x = 100; L1.rel(); sw L1.acq(); SFR2 x = 200; L1.rel(); SFR1 L1 M1 P1 ACQ2 L2 M2 SFR2 + Persistent state lags execution by at most one SFR à Simpler implementation, latest state at failure C1 P2 - Need to flush updates at the end of each SFR à High performance cost REL1 C2 17

18 Design 2: Decoupled-SFR Coupled-SFR has simple design, but lower perf. Persists and log commits on critical execution path L Key idea: Decouple persistent state from program exec. Persist updates and commit logs in background Create undo logs in order Roll back updates in reverse order of creation on failure 18

19 Decoupled-SFR in action Thread 1 Thread 2 Thread 1 Thread 2 L1.acq(); SFR1 x = 100; L1.rel(); sw L1.acq(); SFR2 x = 200; L1.rel(); SFR1 L1 M1 REL1 ACQ2 L2 M2 Create logs in order during execution SFR2 Need to commit logs in order à record order in which logs are created P1 C1 P2 C2 Flush and commit in background 19

20 Log ordering in Decoupled-SFR Init X = 0 Thread 1 Thread 2 X = 100; L1.rel(); sw L1.acq(); X = 200; Thread 1 Header Store X X = 0 Rel L1 Seq = 0 Thread 2 Header Acq L1 Seq = 1 Store X X = 100 Sequence Table L1 0 à 1 Sequence numbers in logs record inter-thread order of log creation Background threads commit logs using recorded sequence no. 20

21 Evaluation setup Designed our logging approaches in LLVM v3.6.0 Instruments stores and sync. ops. to emit undo logs Creates log space for managing per-thread undo-logs Launches background threads to flush/commit logs in Decoupled-SFR Workloads: write-intensive micro-benchmarks 12 threads, 10M operations Performed experiments on Intel E v3 2GHz, 12 physical cores, 2-way hyper-threading 21

22 Better Normalized exec. time Performance evaluation Atlas Coupled-SFR Decoupled-SFR No-persistency [Chakrabarti 14] 65% CQ SPS PC RB-tree TATP LL TPCC Mean Decoupled-SFR performs 65% better than state-of-the-art ATLAS design 22

23 Better Normalized exec. time Performance evaluation Atlas Coupled-SFR Decoupled-SFR No-persistency [Chakrabarti 14] CQ SPS PC RB-tree TATP LL TPCC Mean Coupled-SFR performs better that Decoupled-SFR when fewer stores/sfr 23

24 Conclusion Failure-atomic synchronization-free regions Persistent state moves from one sync. operation to the next Coupled-SFR design Easy to reason about PM state after failure; high performance cost Decoupled-SFR design Persistent state lags execution; performs 65% better than ATLAS More details in our full paper on Persistency for Synchronization-free Regions appearing in PLDI 18 24

25 Thank you! Questions? 25