Request-Oriented Durable Write Caching for Application Performance

Transcription

1 Request-Oriented Durable Write Caching for Application Performance Sangwook Kim 1, Hwanju Kim 2, Sang-Hoon Kim 3, Joonwon Lee 1, and Jinkyu Jeong 1 Sungkyunkwan University 1 University of Cambridge 2 Korea Advanced Institute of Science and Technology (KAIST) 3

2 Non-volatile Write Cache Volatile DRAM cache is ineffective for write Writes are dominant I/Os [FAST 09, FAST 10, FAST 14] Non-volatile write cache (NVWC) provides Fast response for write w/o loss of durability NVWC candidates Flash PCM MRAM NV-DRAM GB/$ Performance [Bhadkamkar et al., FAST 09] BORG: Block-reORGanization for self-optimizing storage systems [Koller et al., FAST 10] I/O deduplication: Utilizing content similarity to improve I/O performance [Harter et al., FAST 14] Analysis of HDFS under HBase: a Facebook messages case study

3 Non-volatile Write Cache Simple caching policy Application P1 P2 P3 Write Write Write Operating System NVWC Blindly caching all writes Backing Storage Lazily writing back to storage

4 Non-volatile Write Cache Simple caching policy P1 P2 Operating System Application No consideration for Write Write Write application performance P3 NVWC Blindly caching all writes Backing Storage Lazily writing back to storage

5 Impact on Application Performance Illustrative experiment TPC-C workload PostgreSQL database 2 NVWC devices 32MB NV-DRAM (emulated via ramdisk) 4GB flash SSD

6 Impact on Application Performance Experimental result * System perf. - ~ 2.1X improved 2.1X 1.7X

7 Impact on Application Performance Experimental result Marginal gain * System perf. - ~ 2.1X improved * Application perf. - ~ 50% degraded Performance drop by 50%!

8 What s the Problem? Criticality-agnostic contention

9 Criticality-Agnostic Contention Different write criticality Client Request Response Application Application performance P1 P2 P3 Operating System NVWC Backing Storage

10 Criticality-Agnostic Contention Different write criticality Client Request Response Application P1 P2 P3 Background process/thread Operating System NVWC Backing Storage

11 Criticality-Agnostic Contention Different write criticality Client Request Response Application P1 P2 P3 Critical Non-critical Operating System NVWC Backing Storage

12 Criticality-Agnostic Contention Different write criticality Client Request Response Application P1 P2 P3 Critical Non-critical * Contentions - Capacity contention - Bandwidth contention Operating System NVWC Backing Storage

13 Criticality-Agnostic Contention Capacity contention Client Request Response Application P1 P2 P3 Critical Non-critical Operating System Frequent write stalls Backing Storage Bounded writeback throughput

14 Criticality-Agnostic Contention Bandwidth contention Client Request Response Application P1 P2 P3 Critical Non-critical C NC NC NC NC NC WB WB WB WB Head Excessive queueing delay Sufficient free blocks Backing Storage

15 Our Approach Request-oriented caching policy Client Operating System Application CP P1 NCP P2 NCP P3 Critical Non-critical * Definitions - Critical process (CP): a process handling request - Critical write: a write awaited by a critical proc. NVWC Caching critical writes only Backing Storage

16 Challenge How to accurately detect critical writes Types of critical write Sync. writes from critical processes Dependency-induced critical writes Process dependency-induced I/O dependency-induced

17 Dependency Problem Process dependency NCP Process Dep. Lock B1 Wake B3 B4 B5 CP Wait for B1

18 Dependency Problem I/O dependency NCP CP I/O Dep. B1 B2 Sync Wait for B2 Complete B3 B4 B5 * Example scenarios: - CP fsync() to a block under writeback issued by NCP - CP tries to overwrite fs journal buffer under writeback

19 Critical Write Detection Critical process identification Application-guided identification

20 Critical Process Identification Application-guided identification Client 1 Client 2 Application CP CP API NCP Operating System NCP NCP NVWC Backing Storage

21 Critical Write Detection Critical process identification Application-guided identification Dependency resolution Criticality inheritance protocols Process criticality inheritance I/O criticality inheritance Blocking object tracking

22 Critical Write Detection Critical process identification Application-guided identification Dependency resolution Criticality inheritance protocols Process criticality inheritance I/O criticality inheritance Blocking object tracking

23 Criticality Inheritance Protocols Process criticality inheritance NCP Lock B1 Inherit Wake B2 B3 B4 CP

24 Criticality Inheritance Protocols I/O criticality inheritance NCP B1 B2 Discard B3 B4 B5 CP Sync Reissue B2 Complete Key issue: caching the dependent write outstanding to disk w/o side effects

25 Evaluation Implementation on Linux 3.13 w/ FlashCache 3.1 Application studies PostgreSQL database Client 1 Client 2 Back end1 Back end2 Check pointer Log writer Writer Redis key-value store Client 1,2,3, Master Snap shotter Log rewriter

26 Evaluation Experimental setup PostgreSQL / Redis FlashCache 4GB ramdisk / 256GB SSD 10K RPM HDD x2 1Gbps TPC-C / YCSB Server Machine Client Machine

27 Evaluation Experimental setup PostgreSQL / Redis * Caching policies - ALL (default) FlashCache No discretion 4GB ramdisk / 256GB SSD 10K RPM HDD x2 1Gbps TPC-C / YCSB Server Machine Client Machine

28 Evaluation Experimental setup PostgreSQL / Redis * Caching policies - ALL (default) - SYNC FlashCache Sync. writes 4GB ramdisk / 256GB SSD 10K RPM HDD x2 1Gbps TPC-C / YCSB Server Machine Client Machine

29 Evaluation Experimental setup PostgreSQL / Redis * Caching policies - ALL (default) - SYNC - CP FlashCache CP sync. writes 4GB ramdisk / 256GB SSD 10K RPM HDD x2 1Gbps TPC-C / YCSB Server Machine Client Machine

30 Evaluation Experimental setup PostgreSQL / Redis FlashCache * Caching policies - ALL (default) - SYNC - CP - CP+PI + Process criticality inheritance 4GB ramdisk / 256GB SSD 10K RPM HDD x2 1Gbps TPC-C / YCSB Server Machine Client Machine

31 Evaluation Experimental setup PostgreSQL / Redis FlashCache * Caching policies - ALL (default) - SYNC - CP - CP+PI - CP+PI+IOI + I/O criticality inheritance 4GB ramdisk / 256GB SSD 10K RPM HDD x2 1Gbps TPC-C / YCSB Server Machine Client Machine

32 Evaluation Experimental setup PostgreSQL / Redis FlashCache * Caching policies - ALL (default) - SYNC - CP - CP+PI - CP+PI+IOI - WAL (PostgreSQL) Trx log writes 4GB ramdisk / 256GB SSD 10K RPM HDD x2 1Gbps TPC-C / YCSB Server Machine Client Machine

33 PostgreSQL Performance TPC-C workload w/ ramdisk 80% Same performance w/ 72% less cached writes Our scheme resolves capacity contention & runtime dependencies Scarce Sufficient

34 PostgreSQL Performance TPC-C workload w/ SSD 2.2X Our scheme resolves bandwidth contention & runtime dependencies Sufficient Sufficient

35 Redis Performance Update-heavy workload w/ 16GB SSD 47% better throughput Improved tail latency 13X 99.9 th %ile (50ms vs. 649ms) Our scheme improves request throughput & request latency

36 Conclusions Key observation Each write has different performance-criticality Request-oriented caching policy Solely utilizes NVWC for application performance Improves performance while reducing cached writes Future work System-level critical process identification Application to user-interactive environments

37 Thank You! Questions and comments Contact