Unioning of the Buffer Cache and Journaling Layers with Non-volatile Memory. Hyokyung Bahn (Ewha University)

Transcription

1 Unioning of the Buffer Cache and Journaling Layers with Non-volatile Memory Hyokyung Bahn (Ewha University)

2 Contents Reliability issues in storage systems Consistency problem Journaling techniques Consistency problem with non-volatile memory Non-volatile memory technology overview Data inconsistency with non-volatile memory Unioning i of fthe Buffer cache and djournaling layers In-place commit and system recovery of UBJ Cache performance of UBJ Performance evaluation Conclusion 2

3 A man working hard What the hell? 3

4 A man working hard What the hell? 4

5 What happens? udden power failure may incur file system inconsistency in hierarchical memory systems ystem Crash Main memory (DRAM) EW EA T Volatile Non-volatile N W E G O After reboot.. EW E A T econdary torage (HDD, Flash, etc.) Inconsistent 5

6 How to solve this problem? Prevent data inconsistency i by journaling techniques ext3, ext4, ReiserF, XF, btrf Frequent commit increases write traffic to storage significantly, leading to the performance degradation Buffer Cache Provide consistency at any case Main memory (DRAM) EW A E T Commit Journal Area W E uccess! Transaction Checkpoint G O WE AE T 6 econdary torage 6.8 N Volatile 3 Journal no-journal Non-volatile W Increase storage writes by 2.7 times E

7 Non-volatile memory promises to replace DRAM in main memory 1. caling Limit of DRAM 2. Power consumption (Ming-Hsiu Lee Macronix, NVMT 2011) As much as 40% of the total system energy is consumed by the main memory subsystem in a mid-range IBM eerver machine. (Querish, ICA 2009) DRAM technology is greatly challenged beyond 45nm (NVMW 10, Driskill) Replacing DRAM with TT-RAM in data centers can reduce power by up to 75% (NVMW 10, Driskill) 3. Demand for fast memory access As critical applications are becoming more data-centric, memory performance is fast becoming the key bottleneck 7

8 Non-volatile Memory Technology ource: T. Perez, C. A. F. D Rose, Technical Report, PUCR, 2010 calability Low-power High-performance 8

9 Non-volatile Memory Technology ource: T. Perez, C. A. F. D Rose, Technical Report, PUCR, 2010 v v v calability Low-power High-performance (Optimistic expectations) 9

10 What if non-volatile main memory? Non-volatile memory seems to provide data consistency as data survive after crashes G O Power crash Main memory (TT-MRAM, PCM) WE EA T W After reboot, restore consistency with remaining data EW EA T torage (HDD) WET! Consistent N E 10

11 What if non-volatile main memory? However, inconsistency i can occur with nonvolatile main memory W is evicted by cache replacement ystem crashes during overwriting data in cache Main memory (TT-MRAM, PCM) GW E T If using data in main memory, N G O EW AE T Inconsistent WIf using data in storage E, After reboot G O EW A T G O WE A T Inconsistent torage (HDD) 11

12 Unioning of Buffer cache and Journaling Layers (UBJ) 12 Goal is providing data consistency without sacrificing performance imply adopting non-volatile memory does not suffice Buffer Cache Novel buffer cache architecture Journal Area called UBJ ubsume functions of caching and journaling by using a data block for dual purposes Make a journaling effect just by changing the status of cache block instead of storage writes non-volatile G O Main memory (TT-MRAM, (DRAM) PCM) W E W E W E T T econdary torage Transaction management

13 Workings of UBJ Event sequences Role of data block Cache Cache Commit Checkpoint Final Final Checkpoint update update start start update update end end (W) (E) (W) (E) Cache block (normal) Freeze cache blocks to be write-protected Cache & log block (frozen) Change cache blocks to be writable Main memory Cache (normal) In-place Commit W E E A T Transaction management Checkpoint uccessfully committed data are managed in a transaction G O EW EA T econdary torage (HDD, Flash, etc.) 13

14 Workings of UBJ Event sequences Role of data block Cache Cache Commit Checkpoint Final Final Checkpoint update update start start update update end end (W) (E) (W) (E) Cache Cache + Journal Cache Journal data Managed by a transaction Protected from partial updates due to cache replacement Perform write requests via copy-onwrite to protect commit data safely erve read requests as cache blocks 14

15 ystem recovery of UBJ Cache Cache Commit Checkpoint Final Final Checkpoint update update start start update update end end (W) (E) (W) (E) Case 1. Crash before commit Guarantee reflecting updates to file system Case 2. Crash after commit Main memory EW E A T Inconsistent Commit Main memory W EW E A T Consistent Transaction management Consistent G O E A T econdary torage checkpoint W E G O W E E A T Inconsistent Consistent econdary torage 15

16 Cache performance of UBJ Buffer Cache Buffer Cache Journal Area Buffer Cache+ Journal area NVRAM NVRAM NVRAM Journal area Journal area Journal area econdary storage 1. Original buffer cache econdary storage econdary storage 2. eparate journaling 3. UBJ 16

17 Cache performance of UBJ Buffer Cache Buffer Cache Journal Area Buffer Cache+ Journal area NVRAM NVRAM NVRAM Miss s ratio Original buffer cache eparate journaling 3. UBJ BF Jm BF no UBJ 0.00 UBJ provides nearly same cache performance as original buffer cache cache ratio

18 Performance Evaluation Prototype t of fubj on Linux Compare with ext4 in journal-mode logs both data and metadata Intel Core i CPU 3.1GHz and 4GB of DDR2-800 memory Emulate non-volatile memory with DRAM Three benchmarks Filebench, IOzone, Postmark 18

19 Performance Evaluation Filebench Exec cution tim me (s) BF-ext4 UBJ Throu ughput(m MB/s) BF-ext4 UBJ 0 varmail proxy fileserver webserver 0 varmail proxy fileserver webserver Improve execution time and throughput by 30.7% and 59.8% on average 19

20 Performance Evaluation Iozone Thro oughput(m MB/s) BF-ext4 40 UBJ Fileset size(mb) (a) Random write Thro oughput(m MB/s) BF-ext4 UBJ Fileset size(mb) (b) equential write Improve performance by 110% on average 20

21 Performance Evaluation Postmark Thro oughput(m MB/s) BF-ext4 UBJ 2K 4K 6K 8K 10K Transactions (a) Read Throu ughput(m MB/s) (b) Write BF-ext4 UBJ 2K 4K 6K 8K 10K Transactions Improve performance by 109% on average 21

22 Performance Evaluation Effectiveness of UBJ on performance as the commit period changes Lat tency (ms s) UBJ BF-ext Commit period (s) 22 The latency of ext4 becomes small as the commit period is longer The latency of UBJ is not sensitive to the commit period changes

23 Conclusion Novel non-volatile memory buffer cache architecture t ubsumes the functions of caching and journaling Buffer cache blocks Journal logs Notion of a frozen state In-place Commit Journal log block as well as a cache block Performance results Implemented on Linux Compared to ext4 in journal mode Improve I/O performance by %76 and up to 240% 23