Unioning of the Buffer Cache and Journaling Layers with Non-volatile Memory

Transcription

1 Unioning of the Buffer Cache and Journaling Layers with Non-volatile Memory UENIX FA 13 Eunji Lee (Ewha University, eoul, Korea) Hyokyung Bahn (Ewha University) am H. Noh (Hongik University)

2 Outline Reliability issues in storage systems Non-volatile memory as a solution UBJ: Unioning of Buffer cache and Journaling Performance evaluation 2

3 A man working hard What the #$%^*&? 3

4 A man working hard What the #$%^*&? 4

5 o what happened? udden power failure incurs file system inconsistency Power went off Buffer cache W E E A Volatile Non-volatile N W E G O After reboot.. E W A File system 5

6 Journaling as a solution Prevent data inconsistency through write-twice ext4, ReiserF, XF, btrf Journal Area File system Buffer Cache W E A E Commit Consistency guaranteed! W E Checkpoint G O W E A E Increase storage writes N by 2.7 times 3 Journal no-journal W E 6

7 Non-volatile memory as a solution Non-volatile Memory Buffer cache DRAM Journal area File system 7

8 Non-volatile memory as main memory 1. caling Limit of DRAM 2. Power consumption (Ming-Hsiu Lee Macronix, NVM 2011) As much as 40% of the total system energy is consumed by the main memory subsystem in a mid-range IBM eerver machine. (Querish, ICA 2009) DRAM technology is greatly challenged beyond 45nm (NVMW 10, Driskill) Replacing DRAM with -RAM in data centers can reduce power by up to 75% (NVMW 10, Driskill) 3. Demand for fast memory access As critical applications are becoming more data-centric, memory performance is fast becoming the key bottleneck 8

9 Non-volatile Memory echnology ource:. Perez, C. A. F. D Rose, echnical Report, PUCR, 2010 calability Low-power High-performance 9

10 Non-volatile Memory echnology ource:. Perez, C. A. F. D Rose, echnical Report, PUCR, 2010 v v calability Low-power High-performance v (Optimistic expectations) 10

11 Non-volatile memory as a solution eems to provide data consistency Power crash Buffer Cache N W E E A After reboot, restore consistency with remaining data W E G O W E E A File system 11

12 Non-volatile memory as a solution? Inconsistency problem still exists with NVM W is evicted by cache replacement ystem crashes during overwriting data in cache Buffer cache G W E If using data in main memory, N G O E W A E Inconsistent 12 G O E W A File system After reboot W G O W E A E If using data in storage, Inconsistent Drop-in replacement of non-volatile memory does not suffice

13 Unioning of Buffer cache and Journaling Layers (UBJ) 13 Provide data consistency without sacrificing performance Design a novel buffer cache architecture UBJ ubsume functions of caching and journaling Use data block for dual purposes Provide journaling effect through transition of cache block state Buffer Cache Journal Area G O Non-volatile memory W E W E W E File system ransaction management

14 Workings of UBJ Event sequences Role of data block Cache update (W) Cache update (E) Cache block (normal) Durability & Consistency in-place commit Freeze blocks to be write-protected Commit start end Commit G O Cache & log block (frozen) Change blocks to be Main memory writable W E E A Checkpoint start W E E A econdary torage (HDD, Flash, etc.) Final update (W) ransaction management Final update (E) Checkpoint Checkpoint end Cache (normal) uccessfully committed data are managed in a transaction 14

15 Workings of UBJ Event sequences Cache update (W) Cache update (E) Commit start end Checkpoint start Final update (W) Final update (E) Checkpoint end Role of data block Cache Cache & log Cache Please refer to our paper for details! Log blocks ransaction Management Protected from replacement Copy-on-write for write request erve read requests as cache blocks 15

16 Cache performance of UBJ Buffer Cache Buffer Cache Journal Area Buffer Cache+ Journal area NVRAM Journal area NVRAM Journal area NVRAM Journal area econdary storage 1. Original buffer cache econdary storage econdary storage 2. eparate journaling 3. UBJ 16

17 Miss ratio Cache performance of UBJ Buffer Cache Buffer Cache Journal Area Buffer Cache+ Journal area NVRAM NVRAM NVRAM eparate journaling BF-Jm BF-no UBJ Original buffer cache UBJ UBJ provides nearly same cache performance as original buffer cache cache ratio

18 Hit count distribution Cache hits on frozen data blocks hit_when_frozen hit_when_normal varmail proxy fileserver webserver 18

19 ystem recovery Event sequences Cache update Write () (W) Cache update (E) Commit start end Checkpoint start Final update (W) Final update (E) Checkpoint end Crash before commit Crash during commit Crash after commit Buffer cache W A Inconsistent Buffer cache W E Inconsistent ransaction Buffer cache W E Consistent E A Consistent E A Consistent W E A E Inconsistent Consistent 19

20 Performance Evaluation Prototype of UBJ on Linux Intel Core i CPU 3.1GHz and 4GB of DDR2-800 memory Emulate non-volatile memory with DRAM Compare with ext4 in journal-mode logs both data and metadata hree benchmarks Filebench, IOzone, Postmark 20

21 Execution time (s) hroughput(mb/s) Performance Evaluation Filebench BF-ext4 UBJ 100 BF-ext4 UBJ varmail proxy fileserver webserver 0 varmail proxy fileserver webserver Improve execution time and throughput by 30.7% and 59.8% on average 21

22 hroughput(mb/s) hroughput(mb/s) Performance Evaluation IOzone BF-ext4 UBJ Fileset size(mb) BF-ext4 UBJ Fileset size(mb) (a) Random write (b) equential write Improve performance by 110% on average, up to by 240% 22

23 hroughput(mb/s) hroughput(mb/s) Performance Evaluation Postmark BF-ext4 UBJ BF-ext4 UBJ K 4K 6K 8K 10K ransactions (a) Read K 4K 6K 8K 10K ransactions (b) Write Improve performance by 109% on average 23

24 Latency (ms) Performance Evaluation Effectiveness of UBJ on performance as the commit period changes UBJ BF-ext Commit period (s) Latency of ext4 becomes smaller as the commit period is longer Latency of UBJ is not sensitive to the commit period changes

25 Conclusion Novel non-volatile memory buffer cache architecture ubsumes the functions of caching and journaling Buffer cache blocks Journal logs In-place Commit Notion of a frozen state Performance results Implemented on Linux Compared to ext4 in journal mode Improve I/O performance by 76% and up to 240% 25

26 hank you Eunji Lee Hyokyung Bahn am H. Noh 26