Write-Optimized and High-Performance Hashing Index Scheme for Persistent Memory

Transcription

1 Write-Optimized and High-Performance Hashing Index Scheme for Persistent Memory Pengfei Zuo, Yu Hua, Jie Wu Huazhong University of Science and Technology, China 3th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 28

2 Persistent Memory (PM) Non-volatile memory as PM is expected to replace or complement DRAM as main memory Non-volatility, low power, large capacity PCM ReRAM DRAM Read (ns) Write (ns) Non-volatility Standby Power ~ ~ High Density (Gb/cm 2 ) C. Xu et al. Overcoming the Challenges of Crossbar Resistive Memory Architectures, HPCA, 25. K. Suzuki and S. Swanson. A Survey of Trends in Non-Volatile Memory Technologies: 2-24, IMW 25. PCM ReRAM 2

3 Index Structures in DRAM vs PM Index structures are critical for memory&storage systems Traditional indexing techniques originally designed for DRAM become inefficient in PM Hardware limitations of NVM Limited cell endurance Asymmetric read/write latency and energy Write optimization matters The requirement of data consistency Data are persistently stored in PM Crash consistency on system failures CPU Persist 3

4 Tree-based vs Hashing Index Structures Tree-based index structures Pros: good for range query Cons: O(log(n)) time complexity for point query Ones for PM have been widely studied CDDS B-tree [FAST ] NV-Tree [FAST 5] wb+-tree [VLDB 5] FP-Tree [SIGMOD 6] WORT [FAST 7] FAST&FAIR [FAST 8] 4

5 Tree-based vs Hashing Index Structures Tree-based index structures Pros: good for range query Cons: O(log(n)) time complexity for point query Ones for PM have been widely studied CDDS B-tree [FAST ] NV-Tree [FAST 5] wb+-tree [VLDB 5] FP-Tree [SIGMOD 6] WORT [FAST 7] FAST&FAIR [FAST 8] Hashing index structures Pros: constant time complexity for point query Cons: do not support range query Widely used in main memory Main memory databases In-memory key-value stores, e.g., Memcached and Redis When maintained in PM, multiple non-trivial challenges exist Rarely touched by existing work 5

6 Challenges of Hashing Indexes for PM High overhead for consistency guarantee Ordering memory writes Cache line flush and memory fence instructions Avoiding partial updates for non-atomic writes Logging or copy-on-write (CoW) mechanisms CPU Volatile caches Memory Bus 8-byte width Non-volatile memory 6

7 Challenges of Hashing Indexes for PM High overhead for consistency guarantee 2 Performance degradation for reducing writes Hashing schemes for DRAM usually cause many extra writes for dealing with hash collisions [INFLOW 5, MSST 7] Write-friendly hashing schemes reduce writes but at the cost of decreasing access performance PCM-friendly hash table (PFHT) [INFLOW 5] Path hashing [MSST 7] 7

8 Challenges of Hashing Indexes for PM High overhead for consistency guarantee 2 Performance degradation for reducing writes 3 Cost inefficiency for resizing hash table Double the table size and iteratively rehash all items Take O(N) time to complete N insertions with cache line flushes & memory fences Rehash all items Old Hash Table New Hash Table 8

9 Existing Hashing Index Schemes for PM ( : bad, : good, -- : moderate) Bucketized Cuckoo (BCH) PFHT Path Hashing 2 Memory efficiency Search Deletion Insertion NVM writes Resizing Consistency [] B. Debnath et al. Revisiting hash table design for phase change memory, INFLOW, 25. [2] P. Zuo and Y. Hua. A write-friendly hashing scheme for non-volatile memory systems, MSST, 27. 9

10 Existing Hashing Index Schemes for PM ( : bad, : good, -- : moderate) Bucketized Cuckoo (BCH) PFHT Path Hashing 2 Level Hashing Memory efficiency Search Deletion Insertion NVM writes Resizing Consistency [] B. Debnath et al. Revisiting hash table design for phase change memory, INFLOW, 25. [2] P. Zuo and Y. Hua. A write-friendly hashing scheme for non-volatile memory systems, MSST, 27.

11 Level Hashing Write-optimized & High-performance Hash Table Structure x One movement TL: N-4 N-3 N-2 N- BL: One movement Resizing support Consistency support Cost-efficient In-place Resizing Scheme Low-overhead Consistency Guarantee Scheme

12 Write-optimized Hash Table Structure Multiple slots per bucket Two hash locations for each key Sharing-based two-level structure At most one movement for each successful insertion 2

13 Maximum Load Factor Write-optimized Hash Table Structure Multiple slots per bucket % 8% 2 Two hash locations for each key 6% 3 4 Sharing-based two-level structure At most one movement for each successful insertion 4% 2% % 2.2% D D+D2 D+D2+D3 All x TL: N-4 N-3 N-2 N- 3

14 Maximum Load Factor Write-optimized Hash Table Structure Multiple slots per bucket % 8% 2 Two hash locations for each key 6% 47.6% 3 4 Sharing-based two-level structure At most one movement for each successful insertion 4% 2% % 2.2% D D+D2 D+D2+D3 All x TL: N-4 N-3 N-2 N- 4

15 Maximum Load Factor Write-optimized Hash Table Structure Multiple slots per bucket % 8% 82.5% 2 Two hash locations for each key 6% 47.6% 3 4 Sharing-based two-level structure At most one movement for each successful insertion 4% 2% % 2.2% D D+D2 D+D2+D3 All x TL: N-4 N-3 N-2 N- BL: 5

16 Maximum Load Factor Write-optimized Hash Table Structure Multiple slots per bucket % 8% 82.5% 9.% 2 Two hash locations for each key 6% 47.6% 3 4 Sharing-based two-level structure At most one movement for each successful insertion 4% 2% % 2.2% D D+D2 D+D2+D3 All x One movement TL: N-4 N-3 N-2 N- BL: One movement 6

17 Write-optimized Hash Table Structure Write-optimized: only.2% of insertions incur one movement High-performance: constant-scale time complexity for all operations Memory-efficient: achieve high load factor by evenly distributing items x One movement TL: N-4 N-3 N-2 N- BL: One movement 7

18 Cost-efficient In-place Resizing Put a new level on top of the old hash table and only rehash items in the old bottom level TL: 2 3 N-2 N- BL: 8

19 Cost-efficient In-place Resizing Put a new level on top of the old hash table and only rehash items in the old bottom level TL: N-4 2N-3 2N-2 2N- TL: BL: 9

20 Cost-efficient In-place Resizing Put a new level on top of the old hash table and only rehash items in the old bottom level TL: N-4 2N-3 2N-2 2N- BL: IL: (the interim level ) 2

21 Cost-efficient In-place Resizing Put a new level on top of the old hash table and only rehash items in the old bottom level TL: BL: N-4 2N-3 2N-2 2N- Rehashing IL: (the interim level ) 2

22 Cost-efficient In-place Resizing Put a new level on top of the old hash table and only rehash items in the old bottom level TL: N-4 2N-3 2N-2 2N- BL: 22

23 Cost-efficient In-place Resizing Put a new level on top of the old hash table and only rehash items in the old bottom level The new hash table is exactly double size of the old one Only /3 buckets (i.e., the old bottom level) are rehashed TL: N-4 2N-3 2N-2 2N- BL: 23

24 Low-overhead Consistency Guarantee A token associated with each slot in the openaddressing hash tables Indicate whether the slot is empty A token is bit, e.g., for non-empty, for empty A bucket: KV KV Tokens Slots 24

25 Low-overhead Consistency Guarantee A token associated with each slot in the openaddressing hash tables Indicate whether the slot is empty A token is bit, e.g., for non-empty, for empty Modifying the token area only needs an atomic write Leveraging the token to perform log-free operations A bucket: KV KV Tokens Slots 25

26 Log-free Deletion Delete an existing item Delete KV KV 26

27 Log-free Deletion Delete an existing item Delete KV KV Modify the token in an atomic write KV KV 27

28 Log-free Deletion Delete an existing item Delete KV KV Modify the token in an atomic write KV KV Log-free insertion and log-free resizing Please find them in our paper 28

29 Consistency Guarantee for Update If directly update an existing key-value item in place Inconsistency on system failures Update KV KV 29

30 Consistency Guarantee for Update If directly update an existing key-value item in place Inconsistency on system failures Update A straightforward solution is to use logging KV KV Expensive! 3

31 Opportunistic Log-free Update Our scheme: check whether there is an empty slot in the bucket storing the old item Yes: log-free update Update No: using logging KV KV Write KV in an empty slot KV KV KV 2 Modify the two tokens in an atomic write KV KV KV 3

32 Log-free Probability Opportunistic Log-free Update Our scheme: check whether there is an empty slot in the bucket storing the old item Yes: log-free update Update No: using logging KV KV % 8% Write KV in an empty slot 6% 4% 2% % 4 slots/bucket 8 slots/bucket 6 slots/bucket Load Factor KV KV KV KV KV 2 Modify the two tokens in an atomic write KV 32

33 Performance Evaluation Both in DRAM and simulated PM platforms Quartz (Hewlett Packard) A DRAM-based performance emulator for PM Comparisons Bucketized cuckoo hashing (BCH) [NSDI 3] PCM-friendly hash table (PFHT) [INFLOW 5] Path hashing [MSST 7] In PM, implement their persistent versions using our proposed log-free consistency guarantee schemes 33

34 Insertion Latency (ns) Insertion Latency (ns) Insertion Latency 24 6 BCH Path PFHT Level BCH Path PFHT Level Load Factor Load Factor DRAM NVM read/write latency: 2/6 Level hashing has the best insertion performance in both DRAM and NVM 34

35 Update Latency (ns) Update Latency BCH PFHT Path Level Level w/o Opp Load Factor Opportunistic log-free update scheme reduces the update latency by 5% 52%, i.e., speeding up the updates by

36 Search Latency (ns) Search Latency BCH PFHT Path Level Positive Search Negative Search The search latency of level hashing is close to that of BCH, which is much lower than PFHT and path hashing 36

37 The Resizing Time (s) Resizing Time BCH PFHT Path Level-Trad Level DRAM NVM-2ns/6ns Level hashing reduces the resizing time by about 76%, i.e., speeding up the resizing by

38 Throughput (M reqs/s) Concurrent Throughput Libcu-2 Level-2 Libcu-4 Level-4 Libcu-8 Level-8 Libcu-6 Level-6 Concurrent level hashing: Support multiple-reader multiplewriter concurrency via simply using fine-grained locking / 7/3 5/5 3/7 /9 Search/Insertion Ratio (%) Concurrent level hashing has.6 2. higher throughput than libcuckoo, due to locking fewer slots for insertions [] X. Li et al.. Algorithmic improvements for fast concurrent cuckoo hashing, Eurosys,

39 Conclusion Traditional indexing techniques originally designed for DRAM become inefficient in PM We propose level hashing, a write-optimized and highperformance hashing index scheme for PM Write-optimized hash table structure Cost-efficient in-place resizing Log-free consistency guarantee.4 3. speedup for insertion,.2 2. speedup for update, and over 4.3 speedup for resizing 39

40 Thanks! Q&A Open-source code: