WORT: Write Optimal Radix Tree for Persistent Memory Storage Systems

Transcription

1 WORT: Write Optimal Radix Tree for Persistent Memory Storage Systems Se Kwon Lee K. Hyun Lim 1, Hyunsub Song, Beomseok Nam, Sam H. Noh UNIST 1 Hongik University

2 Persistent Memory (PM) Persistent memory is expected to replace both DRAM & NAND NAND STT-MRAM PCM DRAM Non-volatility o o o x Read (ns) 2.5 X Write (ns) 2 X Byte-addressable x o o o Density Gbit/cm Gbit/cm Gbit/cm Gbit/cm 2 K. Suzuki and S. Swanson. A Survey of Trends in Non-Volatile Memory Technologies: , IMW 2015 Non-volatile High performance Persistent Memory 2

3 Indexing Structure for PM Storage Systems B+Tree

4 Consistency Issue of B+tree in PM B+tree is a block-based index Key sorting à Block granularity write Rebalancing à Multi-blocks granularity write Persistent memory Byte-addressable à Byte granularity write Write reordering Can result in consistency problem 4

5 Consistency Issue of B+tree in PM Traditional case Volatile CPU Caches Write reordering Not persistent data 3 DRAM Non-volatile Block based storage Block granularity update

6 Consistency Issue of B+tree in PM PM case Volatile CPU Caches Byte granularity update Write reordering Non-volatile Crash Persistent Memory Persistent data Garbage data persistently stored 6

7 Primitives for Data Consistency in PM Durability CLFLUSH (Flush cache line) Can be reordered Ordering MFENCE (Load and Store fence) Order CPU cache line flush instructions Volatile Non-volatile CPU Caches Persistent Memory 7

8 Primitives for Data Consistency in PM Durability CLFLUSH (Flush cache line) Can be reordered Ordering MFENCE (Load and Store fence) Order CPU cache line flush instructions Volatile Non-volatile CPU Serialization of CLFLUSH and MFENCE is known to cause large overhead CPU Caches Persistent Memory 8

9 Primitives for Data Consistency in PM Atomicity 8-byte failure atomicity Need only CLFLUSH Logging or CoW based atomicity Non-volatile (more than 8 bytes) Log area Data area Requires duplicate copies

10 Primitives for Data Consistency in PM Atomicity 8-byte failure atomicity Need only CLFLUSH Logging or CoW based atomicity (more than 8 bytes) Non-volatile Logging increases cache line Log areaflush overhead Data area Requires duplicate copies

11 B+tree Variants for Persistent Memory How can we ensure consistency using failure-atomic writes without logging? Unsorted keys à Append-only with metadata Failure-atomic update of metadata wb+tree (VLDB 15) NVTree (FAST 15) FPTree (SIGMOD 16) Slot array bmp K1 Kn P1 Pn Entry Cnt. Flag (+/-) K1 P1 Flag (+/-) K2 P2 Flag (+/-) K3 P3 bmp Fingerprints K1 P1 Kn Pn Pnext Unsorted key à Decreases search performance 11

12 B+tree Variants for Persistent Memory Logging still necessary Multi-block granularity updates due to node splits and merges Cannot update atomically 35 New key Overflow Split Logging-based solution wb+tree, FPTree Tree reconstruction based solution NVTree large overhead 12

13 B+tree Variants for Persistent Memory Key sorting Fundamental characteristics of B+tree cause problems Rebalancing Overflow New key Split

14 B+tree Variants for Persistent Memory Key sorting Why use B+ trees in the first place? Fundamental characteristics of B+tree cause problems Perhaps Rebalancing there is a better tree data structure more suited for PM? Overflow New key Split

15 Our Contributions Show Radix Tree is a suitable data structure for PM Propose optimal radix tree variants WORT and WOART WORT: Write Optimal Radix Tree WOART: Write Optimal redesigned Adaptive Radix Tree (ART) Optimal: maintain consistency only with single failure-atomic write without any duplicate copies 15

16 Radix Tree Deterministic structure A C C A A C Z C ACA ACC ACZ CAC 16

17 Radix Tree Deterministic structure No key comparison A C C A A C Z C ACA ACC ACZ CAC 17

18 Radix Tree Deterministic structure No key comparison Only 8-byte pointer entries Implicitly stored keys 8-byte pointer A C C A A C Z C ACA ACC ACZ CAC 18

19 Radix Tree Deterministic structure No key comparison Only 8-byte pointer entries Implicitly stored keys No problem caused by key sorting A C N A A C Z C ACA ACC ACZ CAC 19

20 Radix Tree Deterministic structure No key comparison Only 8-byte pointer entries Implicitly stored keys No problem caused by key sorting A C N A No modification of other keys Single 8-byte pointer write per node Easy to use failure-atomic write A C Z C ACA ACC ACZ CAC 20

21 Problem of Deterministic Structure For sparse key distribution Waste excessive memory space à Optimized through path compression High utilization Low utilization key key key key key key key key 21

22 Path Compression in Radix Tree Path compression Search paths that do not need to be distinguished can be removed Unnecessary search path A C C A A C Z ACA ACC ACZ C CAC 22

23 Path Compression in Radix Tree Path compression Common search path is compressed in header Improve memory utilization & indexing performance A Compression header C A C Z ACA ACC ACZ 23

24 Node Split with Path Compression Path compression split Prefix keys are not equal AZ!= AC AZA to be inserted A C A C Z ACA ACC ACZ 24

25 Node Split with Path Compression Path compression split Split A C 1 New parent allocation C Z A C A C Z AZA ACA ACC ACZ 25

26 Node Split with Path Compression Path compression split 2 Decompression of old common prefix AC A A C Z C Z AZA ACA ACC ACZ 26

27 Node Split with Path Compression Path compression split A C However, this split process causes consistency Z 2 Old common prefix update problem in PM. AC A C Z ACA ACC ACZ AZA 27

28 Path compression Problem in PM 28

29 Consistency Issue of Path Compression Path compression split cause updates of multiple nodes have to employ expensive logging methods Consistent state A C Z AZA A C Z Inconsistent state A C Crash ACA ACC ACZ 29

30 Path compression Solution 30

31 WORT (Write-Optimal Radix Tree) for PM Failure-atomic path compression Add node depth field to compression header Compression header (8 bytes) struct Header { unsigned char depth; unsigned char PrefixArr[7]; } 0 A C A C Z ACA ACC ACZ 31

32 WORT (Write-Optimal Radix Tree) for PM Failure-atomic path compression Add node depth field to compression header AZA to be inserted Compression header (8 bytes) 0 A C A C Z ACA ACC ACZ 32

33 WORT (Write-Optimal Radix Tree) for PM Failure-atomic path compression Add node depth field to compression header Compression header (8 bytes) 0 A Consistent state 2 2 Decompression of old common prefix Inconsistent state Crash 0 A C C A C Z ACA ACC ACZ Z AZA 33

34 WORT (Write-Optimal Radix Tree) for PM Failure-atomic path compression Failure detection in WORT Depth in a header Counted depth à Crashed header Compression header (8 bytes) 0 A C Z Inconsistent state 0 A C AZA A C Z Not equal to expected tree depth (2) ACA ACC ACZ 34

35 WORT (Write-Optimal Radix Tree) for PM Failure-atomic path compression Failure recovery in WORT Compression header can be reconstructed à Atomically overwrite Compression header (8 bytes) ACA ACC Consistent state 2 0 A C Z Inconsistent state 0 A C AZA A C Z ACA ACC ACZ 35

36 Write Optimal Data Structure for PM Our proposed radix tree variant is optimal for PM Consistency is always guaranteed with a single 8-byte failure-atomic write without any additional copies for logging or CoW WORT (Write Optimal Radix Tree) WOART (Write Optimal Adaptive Radix Tree) 1. Failure-atomic path compression 2. Redesigned adaptive node 36

37 Evaluation Experimental environment System configuration Description CPU Intel Xeon E5-2620V3 X 2 OS PM Linux CentOS 6.6 (64bit) kernel v4.7.0 Emulated with 256GB DRAM Write latency: Injecting additional stall cycles 37

38 Evaluation Experimental environment Comparison group Radix tree variants B+tree variants WORT wb+tree (VLDB 15) NVTree (FAST 15) FPTree (SIGMOD 16) PM PM DRAM DRAM 38

39 Evaluation Experimental environment Synthetic Workload Characteristics Dense [1 N] Sparse [ ) N Clustered [ ) 39

40 Evaluation Insertion performance WORT outperform the B+tree variants in general 40

41 Evaluation Insertion performance WORT outperform the B+tree variants in general DRAM-based internal node à more favorable performance for FPTree 41

42 Evaluation Insertion performance WORT vs wb+tree Performance differences increase in proportion to write latency 42

43 Evaluation CLFLUSH count per operation B-tree variants incur more cache flush instructions 43

44 Evaluation Search performance WORT always perform better than B+Tree variants 44

45 Evaluation Range query performance Performance gap for range query decreases for PM indexes compared with it between WORT and original B+Tree B+Tree variants do not keep the keys sorted à Rearrangement overhead 45

46 Evaluation MC-benchmark performance on Memcached WORT outperform B+Tree variants in both SET and GET Additional indirection & flush overhead in B-tree variants 46

47 Conclusion Showed suitability of radix tree as PM indexing structure Proposed optimal radix tree variants WORT and WOART Optimal: maintain consistency only with single failure-atomic write without any duplicate copies 47

48 Thank you Any question? 48