The Unwritten Contract of Solid State Drives

Size: px
Start display at page:

Download "The Unwritten Contract of Solid State Drives"

Transcription

1 The Unwritten Contract of Solid State Drives Jun He, Sudarsun Kannan, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau Department of Computer Sciences, University of Wisconsin - Madison

2 Enterprise SSD revenue is expected to exceed enterprise HDD in 2017 Enterprise HDD Enterprise SSD 15 Revenue (Billion Dollars) 10 5 HDD SSD E 2016E 2017E 2018E 2019E Year Source: Gartner, Stifel Estimates

3 Storage stack is shifting from the HDD era to the SSD era App FS

4 Storage stack is shifting from the HDD era to the SSD era App FS

5 Storage stack is shifting from the HDD era to the SSD era App App FS FS

6 Storage stack is shifting from the HDD era to the SSD era App App? FS FS?

7 Storage stack is shifting from the HDD era to the SSD era App App? FS FS??

8 Storage stack is shifting from the HDD era to the SSD era App App? FS FS??

9 Storage stack is shifting from the HDD era to the SSD era for SSD App App? App for SSD FS FS? FS?

10 Storage stack is shifting from the HDD era to the SSD era for SSD App App? App? FS FS? FS? for SSD?

11 Storage stack is shifting from the HDD era to the SSD era for SSD App App? App? FS FS? FS? for SSD??

12 S. Boboila and. Desnoyers. Write Endurance in Flash Drives: Measurements and Analysis. In roceedings of the 8th USENIX Symposium on File and Storage Technologies (FAST 10), San Jose, California, February 2010 The consequences of misusing SSDs

13 S. Boboila and. Desnoyers. Write Endurance in Flash Drives: Measurements and Analysis. In roceedings of the 8th USENIX Symposium on File and Storage Technologies (FAST 10), San Jose, California, February 2010 The consequences of misusing SSDs erformance degradation

14 S. Boboila and. Desnoyers. Write Endurance in Flash Drives: Measurements and Analysis. In roceedings of the 8th USENIX Symposium on File and Storage Technologies (FAST 10), San Jose, California, February 2010 The consequences of misusing SSDs erformance degradation erformance fluctuation

15 S. Boboila and. Desnoyers. Write Endurance in Flash Drives: Measurements and Analysis. In roceedings of the 8th USENIX Symposium on File and Storage Technologies (FAST 10), San Jose, California, February 2010 The consequences of misusing SSDs erformance degradation erformance fluctuation Early end of device life

16 What is the right way to achieve high performance on SSDs?

17 What is the right way to achieve high performance on SSDs? Block Device Interface: read(range), write(range), discard(range)

18 What is the right way to achieve high performance on SSDs? Block Device Interface: read(range), write(range), discard(range) Unwritten Contract of HDDs Sequential accesses are best Nearby accesses are more efficient than farther ones MEMS-based storage devices and standard disk interfaces: A square peg in a round hole? Steven W. Schlosser, Gregory R. Ganger FAST 04

19 What is the right way to achieve high performance on SSDs? Block Device Interface: read(range), write(range), discard(range) Unwritten Contract of HDDs Sequential accesses are best Nearby accesses are more efficient than farther ones MEMS-based storage devices and standard disk interfaces: A square peg in a round hole? Steven W. Schlosser, Gregory R. Ganger FAST 04 Unwritten Contract of SSDs?

20 What is the right way to achieve high performance on SSDs?

21 What is the right way to achieve high performance on SSDs? Existing studies

22 What is the right way to achieve high performance on SSDs? Existing studies Experience of implementing a detailed SSD simulator

23 What is the right way to achieve high performance on SSDs? Existing studies Experience of implementing a detailed SSD simulator Analysis of experiments

24 What is the right way to achieve high performance on SSDs? Existing studies Experience of implementing a detailed SSD simulator Analysis of experiments The Unwritten Contract of SSDs

25 Do the current apps/fses comply with the unwritten contract of SSDs?

26 Do the current apps/fses comply with the unwritten contract of SSDs? 5 apps LevelDB for SSD RocksDB SQLite (RollBack) SQLite (WAL) Varmail

27 Do the current apps/fses comply with the unwritten contract of SSDs? 5 apps LevelDB for SSD RocksDB SQLite (RollBack) SQLite (WAL) Varmail 3 file systems for SSD ext4 F2FS XFS

28 Do the current apps/fses comply with the unwritten contract of SSDs? 5 apps LevelDB for SSD RocksDB SQLite (RollBack) SQLite (WAL) Varmail 3 file systems for SSD ext4 F2FS XFS Contract Rule 1 Rule 2 Rule 3 Rule 4 Rule 5

29 Do the current apps/fses comply with the unwritten contract of SSDs? 5 apps LevelDB for SSD RocksDB SQLite (RollBack) SQLite (WAL) Varmail 3 file systems for SSD ext4 F2FS XFS Contract Rule 1 Rule 2 Rule 3 Rule 4 Rule 5

30 Do the current apps/fses comply with the unwritten contract of SSDs? 5 apps LevelDB for SSD RocksDB SQLite (RollBack) SQLite (WAL) Varmail 3 file systems To study the contract, we built a sophisticated SSD simulator and workload analyzer for SSD ext4 F2FS XFS Contract Rule 1 Rule 2 Rule 3 Rule 4 Rule 5

31 In the paper

32 In the paper We made 24 detailed observations

33 In the paper We made 24 detailed observations We learned several high-level lessons

34 Outline Overview SSD Unwritten Contract Violations of the Unwritten Contract Conclusions

35 Outline Overview SSD Unwritten Contract Violations of the Unwritten Contract Conclusions

36 SSD Background

37 SSD Background

38 SSD Background Block

39 SSD Background Block

40 SSD Background Channel Block

41 SSD Background Channel Channel Block Block

42 SSD Background Channel Channel Channel Block Block Block

43 SSD Background Channel Channel Channel Block Block Block

44 SSD Background Channel Channel Channel Block Block Block

45 SSD Background Controller Channel Channel Channel Block Block Block

46 SSD Background Block Channel Block Channel Block Channel Controller FTL

47 SSD Background Controller FTL - address mapping - garbage collection - wear-leveling Channel Channel Channel Block Block Block

48 SSD Background Controller RAM FTL - address mapping - garbage collection - wear-leveling Channel Block Channel Block Channel Block

49 SSD Background Controller FTL - address mapping - garbage collection - wear-leveling RAM Mapping Table Data Cache Channel Channel Channel Block Block Block

50 Rules of the Unwritten Contract #1 Request Scale #2 Locality #3 Aligned Sequentiality #4 Grouping by Death Time #5 Uniform Data Lifetime

51 Rule #1: Request Scale SSD clients should issue large data requests or multiple outstanding data requests.

52 Rule #1: Request Scale SSD clients should issue large data requests or multiple outstanding data requests. SSD Channel Channel Channel Channel

53 Rule #1: Request Scale SSD clients should issue large data requests or multiple outstanding data requests. Request SSD Channel Channel Channel Channel

54 Rule #1: Request Scale SSD clients should issue large data requests or multiple outstanding data requests. SSD Channel Channel Channel Channel

55 Rule #1: Request Scale SSD clients should issue large data requests or multiple outstanding data requests. SSD High internal parallelism Channel Channel Channel Channel

56 Rule #1: Request Scale Violation

57 Rule #1: Request Scale Violation SSD Channel Channel Channel Channel

58 Rule #1: Request Scale Violation SSD Channel Channel Channel Channel

59 Rule #1: Request Scale Violation SSD Channel Channel Channel Channel Wasted

60 Rule #1: Request Scale Violation If you violate the rule: SSD - Low internal parallelism Channel Channel Channel Channel Wasted

61 Rule #1: Request Scale Violation If you violate the rule: SSD - Low internal parallelism Channel Channel Channel Channel erformance impact: 18x read bandwidth 10x write bandwidth Wasted F. Chen, R. Lee, and X. Zhang. Essential Roles of Exploit- ing Internal arallelism of Flash Memory Based Solid State Drives in High-speed Data rocessing. In roceedings of the 17th International Symposium on High erformance Computer Architecture (HCA-11), pages , San Antonio, Texas, February 2011.

62 Rule 2: Locality SSD clients should access with locality SSD RAM Translation Cache FLASH Logical to hysical Mapping Table

63 Rule 2: Locality SSD clients should access with locality SSD RAM Translation Cache FLASH Logical to hysical Mapping Table

64 Rule 2: Locality SSD clients should access with locality SSD RAM Translation Cache FLASH Logical to hysical Mapping Table

65 Rule 2: Locality SSD clients should access with locality SSD RAM Hit Translation Cache FLASH Logical to hysical Mapping Table

66 Rule 2: Locality SSD clients should access with locality SSD RAM Hit Translation Cache FLASH Logical to hysical Mapping Table High Cache Hit Ratio

67 Rule 2: Locality Violation SSD SSD clients should access with locality RAM Logical to hysical Mapping Table FLASH

68 Rule 2: Locality Violation SSD SSD clients should access with locality RAM Logical to hysical Mapping Table FLASH

69 Rule 2: Locality Violation SSD SSD clients should access with locality If you violate the rule: RAM - Translation cache misses - More translation entry reads/writes Logical to hysical Mapping Table FLASH

70 Rule 2: Locality Violation SSD SSD clients should access with locality If you violate the rule: RAM - Translation cache misses - More translation entry reads/writes FLASH Logical to hysical Mapping Table erformance impact: 2.2x average response time Y. Zhou, F. Wu,. Huang, X. He, C. Xie, and J. Zhou. An Efficient age-level FTL to Optimize Address Translation in Flash Memory. In roceedings of the EuroSys Conference (EuroSys 15), Bordeaux, France, April 2015.

71 Rule 3: Aligned Sequentiality Details in the paper

72 Rule 4: Grouping By Death Time Data with similar death times should be placed in the same block.

73 Rule 4: Grouping By Death Time Data with similar death times should be placed in the same block. 1pm 2pm 1pm 2pm 1pm 2pm 1pm 2pm

74 Rule 4: Grouping By Death Time Data with similar death times should be placed in the same block. 1pm 2pm 1pm 2pm 1pm 2pm Time 1pm 2pm 1pm 2pm

75 Rule 4: Grouping By Death Time Data with similar death times should be placed in the same block. 1pm 2pm 1pm 2pm 1pm 2pm Time 1pm 2pm 1pm 2pm

76 Rule 4: Grouping By Death Time Data with similar death times should be placed in the same block. 1pm 2pm 1pm 2pm 1pm 2pm Time 1pm 2pm 1pm 2pm

77 Rule 4: Grouping By Death Time Data with similar death times should be placed in the same block. 1pm 2pm 1pm 2pm 1pm 2pm Time 1pm 2pm 1pm 2pm

78 Rule 4: Grouping By Death Time Data with similar death times should be placed in the same block. 1pm 2pm 1pm 2pm 1pm 2pm Time 1pm 2pm 1pm 2pm

79 Rule 4: Grouping By Death Time Violation 1pm 1pm 1pm 1pm 2pm 2pm Time 1pm 2pm 2pm 2pm

80 Rule 4: Grouping By Death Time Violation 1pm 1pm 1pm 1pm 2pm 2pm Time 1pm 2pm 2pm 2pm

81 Rule 4: Grouping By Death Time Violation 1pm 1pm 1pm 1pm 2pm 2pm Time 1pm 2pm 2pm 2pm

82 Rule 4: Grouping By Death Time Violation 1pm 1pm 1pm 1pm 2pm 2pm Time 1pm 2pm 2pm 2pm

83 Rule 4: Grouping By Death Time Violation 1pm 1pm 2pm 1pm 1pm 2pm 2pm 2pm Time 1pm 2pm 2pm 2pm

84 Rule 4: Grouping By Death Time Violation 1pm 2pm 1pm 2pm 2pm Time 1pm 2pm 2pm

85 Rule 4: Grouping By Death Time Violation Data movement!!! 1pm 2pm 1pm 2pm 2pm Time 1pm 2pm 2pm

86 Rule 4: Grouping By Death Time Violation Data movement!!! 1pm 2pm 1pm 2pm 2pm Time 1pm 2pm 2pm

87 Rule 4: Grouping By Death Time Violation If you violate the rule: Data movement!!! - erformance penalty - Write amplification 1pm 2pm 1pm 2pm 2pm Time 1pm 2pm 2pm

88 Rule 4: Grouping By Death Time Violation If you violate the rule: Data movement!!! - erformance penalty - Write amplification 1pm 2pm 1pm 2pm Time 1pm 2pm erformance impact: 2pm 4.8x write bandwidth 1.6x throughput 2pm 1.8x block erasure count C. Lee, D. Sim, J.-Y. Hwang, and S. Cho. F2FS: A New File System for Flash Storage. In roceedings of the 13th USENIX Conference on File and Storage Technologies (FAST 15), Santa Clara, California, February J.-U. Kang, J. Hyun, H. Maeng, and S. Cho. The Multi- streamed Solid-State Drive. In 6th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 14), hiladelphia, A, June Y. Cheng, F. Douglis,. Shilane, G. Wallace,. Desnoyers, and K. Li. Erasing Belady s Limitations: In Search of Flash Cache Offline Optimality. In 2016 USENIX Annual Technical Conference (USENIX ATC 16), pages , Denver, CO, USENIX Association.

89 Rule 5: Uniform Data Lifetime Clients of SSDs should create data with similar lifetimes

90 Rule 5: Uniform Data Lifetime Clients of SSDs should create data with similar lifetimes Lifetime 1 Day

91 Rule 5: Uniform Data Lifetime Clients of SSDs should create data with similar lifetimes Lifetime 1 Day SSD Usage Count:

92 Rule 5: Uniform Data Lifetime Clients of SSDs should create data with similar lifetimes Lifetime 1 Day SSD Usage Count:

93 Rule 5: Uniform Data Lifetime Clients of SSDs should create data with similar lifetimes Lifetime 1 Day No wear-leveling needed SSD Usage Count:

94 Rule 5: Uniform Data Lifetime Violation Lifetime 1 Day 1000 Years SSD Usage Count:

95 Rule 5: Uniform Data Lifetime Violation Lifetime 1 Day 1000 Years SSD Usage Count: 365* *10001

96 Rule 5: Uniform Data Lifetime Violation Lifetime 1 Day 1000 Years Some blocks wear out sooner Frequent wear-leveling needed!!! SSD Usage Count: 365* *10001

97 Rule 5: Uniform Data Lifetime Violation If you violate the rule: Lifetime 1 Day - erformance penalty - Write amplification 1000 Years Some blocks wear out sooner Frequent wear-leveling needed!!! SSD Usage Count: 365* *10001

98 Rule 5: Uniform Data Lifetime Violation If you violate the rule: Lifetime 1 Day - erformance penalty - Write amplification 1000 Years Some blocks wear out sooner erformance impact: Frequent wear-leveling needed!!! 1.6x write latency SSD S. Boboila and. Desnoyers. Write Endurance in Flash Drives: Measurements and Analysis. In roceedings of the 8th USENIX Symposium on File and Storage Technologies (FAST 10), San Jose, California, February Usage Count: 365* *10001

99 Outline Overview SSD Unwritten Contract Violations of the Unwritten Contract Conclusions

100 Do applications/file systems comply with the unwritten contract?

101 We conduct vertical analysis to find violations of SSD contract

102 We conduct vertical analysis to find violations of SSD contract App + FS SSD

103 We conduct vertical analysis to find violations of SSD contract App + FS Block Trace SSD

104 We conduct vertical analysis to find violations of SSD contract App + FS Block Trace SSD SSD Simulator: WiscSim

105 We conduct vertical analysis to find violations of SSD contract App + FS Block Trace Analyzer: SSD WiscSee Rule violation? SSD Simulator: WiscSim

106 We conduct vertical analysis to find violations of SSD contract App + Root Cause FS Block Trace Analyzer: SSD WiscSee Rule violation? SSD Simulator: WiscSim

107 2 of Our 24 Observations 1. Linux page cache limits request scale 2. F2FS incurs more GC overhead than traditional file systems

108 2 of Our 24 Observations 1. Linux page cache limits request scale 2. F2FS incurs more GC overhead than traditional file systems

109 We evaluate request scale by request size and number of concurrent requests ext4 f2fs xfs ext4 f2fs xfs leveldb rocksdb sqlite rb sqlite wal varmail leveldb rocksdb sqlite rb sqlite wal varmail read write Num of Concurrent Requests read write 0 0 seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix small large mix seq rand mix seq rand mix small large mix Request Size (KB)

110 We evaluate request scale by request size and number of concurrent requests ext4 f2fs xfs ext4 f2fs xfs leveldb rocksdb sqlite rb sqlite wal varmail leveldb rocksdb sqlite rb sqlite wal varmail read write Num of Concurrent Requests read write 0 0 seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix small large mix seq rand mix seq rand mix small large mix Request Size (KB)

111 We evaluate request scale by request size and number of concurrent requests ext4 f2fs xfs ext4 f2fs xfs leveldb rocksdb sqlite rb sqlite wal varmail leveldb rocksdb sqlite rb sqlite wal varmail 1200 LevelDB & RocksDB: read write read write seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix seq rand Num of Concurrent Requests mix small large mix seq rand mix seq rand mix Request Size (KB) small large mix Insertions Background Compactions

112 We evaluate request scale by request size and number of concurrent requests ext4 f2fs xfs ext4 f2fs xfs leveldb rocksdb sqlite rb sqlite wal varmail leveldb rocksdb sqlite rb sqlite wal varmail 1200 LevelDB & RocksDB: 30 Request Size (KB) read write read write seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix small large mix Num of Concurrent Requests seq rand mix seq rand mix small large mix Insertions Background Compactions Median: ~100KB

113 We evaluate request scale by request size and number of concurrent requests ext4 f2fs xfs ext4 f2fs xfs leveldb rocksdb sqlite rb sqlite wal varmail leveldb rocksdb sqlite rb sqlite wal varmail 1200 LevelDB & RocksDB: 30 Request Size (KB) read write read write seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix seq rand mix small large mix Num of Concurrent Requests seq rand mix seq rand mix small large mix Insertions Background Compactions Median: ~100KB Median: ~2

114 LevelDB and RocksDB can access files in large sizes. Why was the request scale low?

115 Buffered read(): age cache implementation splits and serializes user requests App age Cache Block Layer SSD

116 Buffered read(): age cache implementation splits and serializes user requests App read() 2MB age Cache Block Layer SSD

117 Buffered read(): age cache implementation splits and serializes user requests App read() 2MB age Cache 128KB 128KB 128KB Block Layer SSD

118 Buffered read(): age cache implementation splits and serializes user requests App read() 2MB age Cache 128KB 128KB 128KB One request at a time Block Layer SSD

119 Buffered read(): age cache implementation splits and serializes user requests App read() 2MB age Cache 128KB 128KB 128KB One request at a time Surprise! Even reading 2MB in your Block Layer app will not utilize SSD well. SSD

120 Cause of Violation Large reads are throttled by small prefetching (readahead).

121 2 of Our 24 Observations 1. Linux page cache limits request scale 2. F2FS incurs more GC overhead than traditional file systems

122 2 of Our 24 Observations 1. Linux page cache limits request scale 2. F2FS incurs more GC overhead than traditional file systems

123 We study GC (rule #3: grouping by death time) by zombie curves

124 We study GC (rule #3: grouping by death time) by zombie curves What s a zombie curve??

125 We study GC (rule #3: grouping by death time) by zombie curves What s a zombie curve?

126 We study GC (rule #3: grouping by death time) by zombie curves What s a zombie curve? Run workloads with infinite space over-provisioning

127 We study GC (rule #3: grouping by death time) by zombie curves What s a zombie curve? Run workloads with infinite space over-provisioning

128 We study GC (rule #3: grouping by death time) by zombie curves What s a zombie curve? Run workloads with infinite space over-provisioning Valid

129 We study GC (rule #3: grouping by death time) by zombie curves What s a zombie curve? Run workloads with infinite space over-provisioning Invalid Valid

130 We study GC (rule #3: grouping by death time) by zombie curves Valid ratio

131 We study GC (rule #3: grouping by death time) by zombie curves Valid ratio

132 We study GC (rule #3: grouping by death time) by zombie curves Valid ratio

133 We study GC (rule #3: grouping by death time) by zombie curves Valid ratio

134 We study GC (rule #3: grouping by death time) by zombie curves Valid ratio

135 We study GC (rule #3: grouping by death time) by zombie curves Valid ratio

136 We study GC (rule #3: grouping by death time) by zombie curves Valid ratio

137 What s a good zombie curve? Over-provisioned Over-provisioned

138 What s a good zombie curve? Ready to be used Ready to be used Over-provisioned Over-provisioned

139 What s a bad zombie curve? Over-provisioned

140 What s a bad zombie curve? Move data before use Over-provisioned

141 BTW, zombie curve helps you choose over-provisioning ratio Over-provisioned

142 BTW, zombie curve helps you choose over-provisioning ratio Over-provisioned

143 F2FS incurs a worse zombie curve (higher GC overhead) than ext4 for SQLite 1.0 Over-rovisioned Valid Ratio Animations cannot be displayed in DF. lease see the animations at Flash Space

144 F2FS incurs a worse zombie curve (higher GC overhead) than ext4 for SQLite 1.0 Over-rovisioned Valid Ratio Animations cannot be displayed in DF. lease see the animations at Flash Space

145 F2FS incurs a worse zombie curve (higher GC overhead) than ext4 for SQLite 1.0 Over-rovisioned Valid Ratio ext Animations cannot be displayed in DF. lease see the animations at Flash Space

146 F2FS incurs a worse zombie curve (higher GC overhead) than ext4 for SQLite 1.0 F2FS Over-rovisioned Valid Ratio ext Animations cannot be displayed in DF. lease see the animations at Flash Space

147 F2FS incurs a worse zombie curve (higher GC overhead) than ext4 for SQLite 1.0 F2FS Over-rovisioned Valid Ratio ext Animations cannot be displayed in DF. lease see the animations at Stable-state curves characterize workloads Flash Space

148 F2FS incurs a worse zombie curve (higher GC overhead) than ext4 for SQLite 1.0 F2FS Over-rovisioned Valid Ratio ext Animations cannot be displayed in DF. lease see the animations at Stable-state curves characterize workloads Flash Space

149 Why did F2FS incur a worse zombie curve (GC overhead)?

150 Why did F2FS incur a worse zombie curve (GC overhead)? SQLite fragmented F2FS

151 Why did F2FS incur a worse zombie curve (GC overhead)? SQLite fragmented F2FS F2FS did not discard data that was deleted by SQLite

152 Why did F2FS incur a worse zombie curve (GC overhead)? SQLite fragmented F2FS F2FS did not discard data that was deleted by SQLite F2FS was not able to stay log-structured for SQLite s I/O pattern

153 More Observations

154 More Observations Legacy file system allocation policies break locality

155 More Observations Legacy file system allocation policies break locality Application log structuring does not reduce GC

156 More Observations Legacy file system allocation policies break locality Application log structuring does not reduce GC 24 observations in the paper

157 Lessons Learned

158 Lessons Learned The SSD contract is multi-dimensional Optimizing for one dimension is not enough We need more sophisticated tools to analyze workloads

159 Lessons Learned The SSD contract is multi-dimensional Optimizing for one dimension is not enough We need more sophisticated tools to analyze workloads Although not perfect, traditional file systems perform surprisingly well upon SSDs

160 Lessons Learned The SSD contract is multi-dimensional Optimizing for one dimension is not enough We need more sophisticated tools to analyze workloads Although not perfect, traditional file systems perform surprisingly well upon SSDs Myths spread if the unwritten contract is not clarified Random writes increase GC overhead

161 Conclusions

162 Conclusions Understanding the unwritten contract is crucial for designing high performance application and file systems

163 Conclusions Understanding the unwritten contract is crucial for designing high performance application and file systems System designing demands more vertical analysis

164 Conclusions Understanding the unwritten contract is crucial for designing high performance application and file systems System designing demands more vertical analysis WiscSee (analyzer) and WiscSim (SSD simulator) are available at:

The Unwritten Contract of Solid State Drives

The Unwritten Contract of Solid State Drives The Unwritten Contract of Solid State Drives Jun He Sudarsun Kannan Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau Department of Computer Sciences, University of Wisconsin Madison Abstract We perform

More information

Yiying Zhang, Leo Prasath Arulraj, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. University of Wisconsin - Madison

Yiying Zhang, Leo Prasath Arulraj, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. University of Wisconsin - Madison Yiying Zhang, Leo Prasath Arulraj, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau University of Wisconsin - Madison 1 Indirection Reference an object with a different name Flexible, simple, and

More information

ParaFS: A Log-Structured File System to Exploit the Internal Parallelism of Flash Devices

ParaFS: A Log-Structured File System to Exploit the Internal Parallelism of Flash Devices ParaFS: A Log-Structured File System to Exploit the Internal Parallelism of Devices Jiacheng Zhang, Jiwu Shu, Youyou Lu Tsinghua University 1 Outline Background and Motivation ParaFS Design Evaluation

More information

p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer

p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer p-oftl: An Object-based Semantic-aware Parallel Flash Translation Layer Wei Wang, Youyou Lu, and Jiwu Shu Department of Computer Science and Technology, Tsinghua University, Beijing, China Tsinghua National

More information

Presented by: Nafiseh Mahmoudi Spring 2017

Presented by: Nafiseh Mahmoudi Spring 2017 Presented by: Nafiseh Mahmoudi Spring 2017 Authors: Publication: Type: ACM Transactions on Storage (TOS), 2016 Research Paper 2 High speed data processing demands high storage I/O performance. Flash memory

More information

FlashTier: A Lightweight, Consistent and Durable Storage Cache

FlashTier: A Lightweight, Consistent and Durable Storage Cache FlashTier: A Lightweight, Consistent and Durable Storage Cache Mohit Saxena PhD Candidate University of Wisconsin-Madison msaxena@cs.wisc.edu Flash Memory Summit 2012 Santa Clara, CA Flash is a Good Cache

More information

Getting Real: Lessons in Transitioning Research Simulations into Hardware Systems

Getting Real: Lessons in Transitioning Research Simulations into Hardware Systems Getting Real: Lessons in Transitioning Research Simulations into Hardware Systems Mohit Saxena, Yiying Zhang Michael Swift, Andrea Arpaci-Dusseau and Remzi Arpaci-Dusseau Flash Storage Stack Research SSD

More information

Toward SLO Complying SSDs Through OPS Isolation

Toward SLO Complying SSDs Through OPS Isolation Toward SLO Complying SSDs Through OPS Isolation October 23, 2015 Hongik University UNIST (Ulsan National Institute of Science & Technology) Sam H. Noh 1 Outline Part 1: FAST 2015 Part 2: Beyond FAST 2

More information

Hibachi: A Cooperative Hybrid Cache with NVRAM and DRAM for Storage Arrays

Hibachi: A Cooperative Hybrid Cache with NVRAM and DRAM for Storage Arrays Hibachi: A Cooperative Hybrid Cache with NVRAM and DRAM for Storage Arrays Ziqi Fan, Fenggang Wu, Dongchul Park 1, Jim Diehl, Doug Voigt 2, and David H.C. Du University of Minnesota, 1 Intel, 2 HP Enterprise

More information

FStream: Managing Flash Streams in the File System

FStream: Managing Flash Streams in the File System FStream: Managing Flash Streams in the File System Eunhee Rho, Kanchan Joshi, Seung-Uk Shin, Nitesh Jagadeesh Shetty, Joo-Young Hwang, Sangyeun Cho, Daniel DG Lee, Jaeheon Jeong Memory Division, Samsung

More information

Performance Modeling and Analysis of Flash based Storage Devices

Performance Modeling and Analysis of Flash based Storage Devices Performance Modeling and Analysis of Flash based Storage Devices H. Howie Huang, Shan Li George Washington University Alex Szalay, Andreas Terzis Johns Hopkins University MSST 11 May 26, 2011 NAND Flash

More information

FFS: The Fast File System -and- The Magical World of SSDs

FFS: The Fast File System -and- The Magical World of SSDs FFS: The Fast File System -and- The Magical World of SSDs The Original, Not-Fast Unix Filesystem Disk Superblock Inodes Data Directory Name i-number Inode Metadata Direct ptr......... Indirect ptr 2-indirect

More information

Understanding Write Behaviors of Storage Backends in Ceph Object Store

Understanding Write Behaviors of Storage Backends in Ceph Object Store Understanding Write Behaviors of Storage Backends in Object Store Dong-Yun Lee, Kisik Jeong, Sang-Hoon Han, Jin-Soo Kim, Joo-Young Hwang and Sangyeun Cho How Amplifies Writes client Data Store, please

More information

Using Transparent Compression to Improve SSD-based I/O Caches

Using Transparent Compression to Improve SSD-based I/O Caches Using Transparent Compression to Improve SSD-based I/O Caches Thanos Makatos, Yannis Klonatos, Manolis Marazakis, Michail D. Flouris, and Angelos Bilas {mcatos,klonatos,maraz,flouris,bilas}@ics.forth.gr

More information

vstream: Virtual Stream Management for Multi-streamed SSDs

vstream: Virtual Stream Management for Multi-streamed SSDs vstream: Virtual Stream Management for Multi-streamed SSDs Hwanjin Yong, Kisik Jeong, Joonwon Lee, and Jin-Soo Kim Samsung Electronics Co., South Korea Sungkyunkwan University, South Korea Seoul National

More information

SFS: Random Write Considered Harmful in Solid State Drives

SFS: Random Write Considered Harmful in Solid State Drives SFS: Random Write Considered Harmful in Solid State Drives Changwoo Min 1, 2, Kangnyeon Kim 1, Hyunjin Cho 2, Sang-Won Lee 1, Young Ik Eom 1 1 Sungkyunkwan University, Korea 2 Samsung Electronics, Korea

More information

A Buffer Replacement Algorithm Exploiting Multi-Chip Parallelism in Solid State Disks

A Buffer Replacement Algorithm Exploiting Multi-Chip Parallelism in Solid State Disks A Buffer Replacement Algorithm Exploiting Multi-Chip Parallelism in Solid State Disks Jinho Seol, Hyotaek Shim, Jaegeuk Kim, and Seungryoul Maeng Division of Computer Science School of Electrical Engineering

More information

HashKV: Enabling Efficient Updates in KV Storage via Hashing

HashKV: Enabling Efficient Updates in KV Storage via Hashing HashKV: Enabling Efficient Updates in KV Storage via Hashing Helen H. W. Chan, Yongkun Li, Patrick P. C. Lee, Yinlong Xu The Chinese University of Hong Kong University of Science and Technology of China

More information

D E N A L I S T O R A G E I N T E R F A C E. Laura Caulfield Senior Software Engineer. Arie van der Hoeven Principal Program Manager

D E N A L I S T O R A G E I N T E R F A C E. Laura Caulfield Senior Software Engineer. Arie van der Hoeven Principal Program Manager 1 T HE D E N A L I N E X T - G E N E R A T I O N H I G H - D E N S I T Y S T O R A G E I N T E R F A C E Laura Caulfield Senior Software Engineer Arie van der Hoeven Principal Program Manager Outline Technology

More information

Optimizing Flash-based Key-value Cache Systems

Optimizing Flash-based Key-value Cache Systems Optimizing Flash-based Key-value Cache Systems Zhaoyan Shen, Feng Chen, Yichen Jia, Zili Shao Department of Computing, Hong Kong Polytechnic University Computer Science & Engineering, Louisiana State University

More information

Designing a True Direct-Access File System with DevFS

Designing a True Direct-Access File System with DevFS Designing a True Direct-Access File System with DevFS Sudarsun Kannan, Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau University of Wisconsin-Madison Yuangang Wang, Jun Xu, Gopinath Palani Huawei Technologies

More information

A Self Learning Algorithm for NAND Flash Controllers

A Self Learning Algorithm for NAND Flash Controllers A Self Learning Algorithm for NAND Flash Controllers Hao Zhi, Lee Firmware Manager Core Storage Electronics Corp./Phison Electronics Corp. haozhi_lee@phison.com Santa Clara, CA 1 Outline Basic FW Architecture

More information

Middleware and Flash Translation Layer Co-Design for the Performance Boost of Solid-State Drives

Middleware and Flash Translation Layer Co-Design for the Performance Boost of Solid-State Drives Middleware and Flash Translation Layer Co-Design for the Performance Boost of Solid-State Drives Chao Sun 1, Asuka Arakawa 1, Ayumi Soga 1, Chihiro Matsui 1 and Ken Takeuchi 1 1 Chuo University Santa Clara,

More information

VSSIM: Virtual Machine based SSD Simulator

VSSIM: Virtual Machine based SSD Simulator 29 th IEEE Conference on Mass Storage Systems and Technologies (MSST) Long Beach, California, USA, May 6~10, 2013 VSSIM: Virtual Machine based SSD Simulator Jinsoo Yoo, Youjip Won, Joongwoo Hwang, Sooyong

More information

Warped Mirrors for Flash Yiying Zhang

Warped Mirrors for Flash Yiying Zhang Warped Mirrors for Flash Yiying Zhang Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau 2 3 Flash-based SSDs in Storage Systems Using commercial SSDs in storage layer Good performance Easy to use Relatively

More information

Baoping Wang School of software, Nanyang Normal University, Nanyang , Henan, China

Baoping Wang School of software, Nanyang Normal University, Nanyang , Henan, China doi:10.21311/001.39.7.41 Implementation of Cache Schedule Strategy in Solid-state Disk Baoping Wang School of software, Nanyang Normal University, Nanyang 473061, Henan, China Chao Yin* School of Information

More information

PebblesDB: Building Key-Value Stores using Fragmented Log Structured Merge Trees

PebblesDB: Building Key-Value Stores using Fragmented Log Structured Merge Trees PebblesDB: Building Key-Value Stores using Fragmented Log Structured Merge Trees Pandian Raju 1, Rohan Kadekodi 1, Vijay Chidambaram 1,2, Ittai Abraham 2 1 The University of Texas at Austin 2 VMware Research

More information

LightNVM: The Linux Open-Channel SSD Subsystem Matias Bjørling (ITU, CNEX Labs), Javier González (CNEX Labs), Philippe Bonnet (ITU)

LightNVM: The Linux Open-Channel SSD Subsystem Matias Bjørling (ITU, CNEX Labs), Javier González (CNEX Labs), Philippe Bonnet (ITU) ½ LightNVM: The Linux Open-Channel SSD Subsystem Matias Bjørling (ITU, CNEX Labs), Javier González (CNEX Labs), Philippe Bonnet (ITU) 0% Writes - Read Latency 4K Random Read Latency 4K Random Read Percentile

More information

Application-Managed Flash

Application-Managed Flash Application-Managed Flash Sungjin Lee*, Ming Liu, Sangwoo Jun, Shuotao Xu, Jihong Kim and Arvind *Inha University Massachusetts Institute of Technology Seoul National University Operating System Support

More information

S-FTL: An Efficient Address Translation for Flash Memory by Exploiting Spatial Locality

S-FTL: An Efficient Address Translation for Flash Memory by Exploiting Spatial Locality S-FTL: An Efficient Address Translation for Flash Memory by Exploiting Spatial Locality Song Jiang, Lei Zhang, Xinhao Yuan, Hao Hu, and Yu Chen Department of Electrical and Computer Engineering Wayne State

More information

A Caching-Oriented FTL Design for Multi-Chipped Solid-State Disks. Yuan-Hao Chang, Wei-Lun Lu, Po-Chun Huang, Lue-Jane Lee, and Tei-Wei Kuo

A Caching-Oriented FTL Design for Multi-Chipped Solid-State Disks. Yuan-Hao Chang, Wei-Lun Lu, Po-Chun Huang, Lue-Jane Lee, and Tei-Wei Kuo A Caching-Oriented FTL Design for Multi-Chipped Solid-State Disks Yuan-Hao Chang, Wei-Lun Lu, Po-Chun Huang, Lue-Jane Lee, and Tei-Wei Kuo 1 June 4, 2011 2 Outline Introduction System Architecture A Multi-Chipped

More information

Performance Benefits of Running RocksDB on Samsung NVMe SSDs

Performance Benefits of Running RocksDB on Samsung NVMe SSDs Performance Benefits of Running RocksDB on Samsung NVMe SSDs A Detailed Analysis 25 Samsung Semiconductor Inc. Executive Summary The industry has been experiencing an exponential data explosion over the

More information

Strata: A Cross Media File System. Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, Thomas Anderson

Strata: A Cross Media File System. Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, Thomas Anderson A Cross Media File System Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, Thomas Anderson 1 Let s build a fast server NoSQL store, Database, File server, Mail server Requirements

More information

Optimizing Software-Defined Storage for Flash Memory

Optimizing Software-Defined Storage for Flash Memory Optimizing Software-Defined Storage for Flash Memory Avraham Meir Chief Scientist, Elastifile Santa Clara, CA 1 Agenda What is SDS? SDS Media Selection Flash-Native SDS Non-Flash SDS File system benefits

More information

Divided Disk Cache and SSD FTL for Improving Performance in Storage

Divided Disk Cache and SSD FTL for Improving Performance in Storage JOURNAL OF SEMICONDUCTOR TECHNOLOGY AND SCIENCE, VOL.17, NO.1, FEBRUARY, 2017 ISSN(Print) 1598-1657 https://doi.org/10.5573/jsts.2017.17.1.015 ISSN(Online) 2233-4866 Divided Disk Cache and SSD FTL for

More information

A File-System-Aware FTL Design for Flash Memory Storage Systems

A File-System-Aware FTL Design for Flash Memory Storage Systems 1 A File-System-Aware FTL Design for Flash Memory Storage Systems Po-Liang Wu, Yuan-Hao Chang, Po-Chun Huang, and Tei-Wei Kuo National Taiwan University 2 Outline Introduction File Systems Observations

More information

Page Mapping Scheme to Support Secure File Deletion for NANDbased Block Devices

Page Mapping Scheme to Support Secure File Deletion for NANDbased Block Devices Page Mapping Scheme to Support Secure File Deletion for NANDbased Block Devices Ilhoon Shin Seoul National University of Science & Technology ilhoon.shin@snut.ac.kr Abstract As the amount of digitized

More information

Flash Memory Based Storage System

Flash Memory Based Storage System Flash Memory Based Storage System References SmartSaver: Turning Flash Drive into a Disk Energy Saver for Mobile Computers, ISLPED 06 Energy-Aware Flash Memory Management in Virtual Memory System, islped

More information

LevelDB-Raw: Eliminating File System Overhead for Optimizing Performance of LevelDB Engine

LevelDB-Raw: Eliminating File System Overhead for Optimizing Performance of LevelDB Engine 777 LevelDB-Raw: Eliminating File System Overhead for Optimizing Performance of LevelDB Engine Hak-Su Lim and Jin-Soo Kim *College of Info. & Comm. Engineering, Sungkyunkwan University, Korea {haksu.lim,

More information

Redesigning LSMs for Nonvolatile Memory with NoveLSM

Redesigning LSMs for Nonvolatile Memory with NoveLSM Redesigning LSMs for Nonvolatile Memory with NoveLSM Sudarsun Kannan, University of Wisconsin-Madison; Nitish Bhat and Ada Gavrilovska, Georgia Tech; Andrea Arpaci-Dusseau and Remzi Arpaci-Dusseau, University

More information

Understanding SSD overprovisioning

Understanding SSD overprovisioning Understanding SSD overprovisioning Kent Smith, LSI Corporation - January 8, 2013 The over-provisioning of NAND flash memory in solid state drives (SSDs) and flash memory-based accelerator cards (cache)

More information

Speeding Up Cloud/Server Applications Using Flash Memory

Speeding Up Cloud/Server Applications Using Flash Memory Speeding Up Cloud/Server Applications Using Flash Memory Sudipta Sengupta and Jin Li Microsoft Research, Redmond, WA, USA Contains work that is joint with Biplob Debnath (Univ. of Minnesota) Flash Memory

More information

Replacing the FTL with Cooperative Flash Management

Replacing the FTL with Cooperative Flash Management Replacing the FTL with Cooperative Flash Management Mike Jadon Radian Memory Systems www.radianmemory.com Flash Memory Summit 2015 Santa Clara, CA 1 Data Center Primary Storage WORM General Purpose RDBMS

More information

Clustered Page-Level Mapping for Flash Memory-Based Storage Devices

Clustered Page-Level Mapping for Flash Memory-Based Storage Devices H. Kim and D. Shin: ed Page-Level Mapping for Flash Memory-Based Storage Devices 7 ed Page-Level Mapping for Flash Memory-Based Storage Devices Hyukjoong Kim and Dongkun Shin, Member, IEEE Abstract Recent

More information

Getting it Right: Testing Storage Arrays The Way They ll be Used

Getting it Right: Testing Storage Arrays The Way They ll be Used Getting it Right: Testing Storage Arrays The Way They ll be Used Peter Murray Virtual Instruments Flash Memory Summit 2017 Santa Clara, CA 1 The Journey: How Did we Get Here? Storage testing was black

More information

OSSD: A Case for Object-based Solid State Drives

OSSD: A Case for Object-based Solid State Drives MSST 2013 2013/5/10 OSSD: A Case for Object-based Solid State Drives Young-Sik Lee Sang-Hoon Kim, Seungryoul Maeng, KAIST Jaesoo Lee, Chanik Park, Samsung Jin-Soo Kim, Sungkyunkwan Univ. SSD Desktop Laptop

More information

A Novel Buffer Management Scheme for SSD

A Novel Buffer Management Scheme for SSD A Novel Buffer Management Scheme for SSD Qingsong Wei Data Storage Institute, A-STAR Singapore WEI_Qingsong@dsi.a-star.edu.sg Bozhao Gong National University of Singapore Singapore bzgong@nus.edu.sg Cheng

More information

Linux Kernel Abstractions for Open-Channel SSDs

Linux Kernel Abstractions for Open-Channel SSDs Linux Kernel Abstractions for Open-Channel SSDs Matias Bjørling Javier González, Jesper Madsen, and Philippe Bonnet 2015/03/01 1 Market Specific FTLs SSDs on the market with embedded FTLs targeted at specific

More information

SHRD: Improving Spatial Locality in Flash Storage Accesses by Sequentializing in Host and Randomizing in Device

SHRD: Improving Spatial Locality in Flash Storage Accesses by Sequentializing in Host and Randomizing in Device SHRD: Improving Spatial Locality in Flash Storage Accesses by Sequentializing in Host and Randomizing in Device Hyukjoong Kim 1, Dongkun Shin 1, Yun Ho Jeong 2 and Kyung Ho Kim 2 1 Samsung Electronics

More information

Storage Systems : Disks and SSDs. Manu Awasthi July 6 th 2018 Computer Architecture Summer School 2018

Storage Systems : Disks and SSDs. Manu Awasthi July 6 th 2018 Computer Architecture Summer School 2018 Storage Systems : Disks and SSDs Manu Awasthi July 6 th 2018 Computer Architecture Summer School 2018 Why study storage? Scalable High Performance Main Memory System Using Phase-Change Memory Technology,

More information

arxiv: v1 [cs.db] 25 Nov 2018

arxiv: v1 [cs.db] 25 Nov 2018 Enabling Efficient Updates in KV Storage via Hashing: Design and Performance Evaluation Yongkun Li, Helen H. W. Chan, Patrick P. C. Lee, and Yinlong Xu University of Science and Technology of China The

More information

Optimizing Translation Information Management in NAND Flash Memory Storage Systems

Optimizing Translation Information Management in NAND Flash Memory Storage Systems Optimizing Translation Information Management in NAND Flash Memory Storage Systems Qi Zhang 1, Xuandong Li 1, Linzhang Wang 1, Tian Zhang 1 Yi Wang 2 and Zili Shao 2 1 State Key Laboratory for Novel Software

More information

A Novel On-the-Fly NAND Flash Read Channel Parameter Estimation and Optimization

A Novel On-the-Fly NAND Flash Read Channel Parameter Estimation and Optimization A Novel On-the-Fly NAND Flash Read Channel Parameter Estimation and Optimization Tingjun Xie Staff Engineer VIA Technologies, Inc. TingjunXie@viatech.com Santa Clara, CA 1 Outline Significance of flash

More information

Purity: building fast, highly-available enterprise flash storage from commodity components

Purity: building fast, highly-available enterprise flash storage from commodity components Purity: building fast, highly-available enterprise flash storage from commodity components J. Colgrove, J. Davis, J. Hayes, E. Miller, C. Sandvig, R. Sears, A. Tamches, N. Vachharajani, and F. Wang 0 Gala

More information

A Flash Scheduling Strategy for Current Capping in Multi-Power-Mode SSDs

A Flash Scheduling Strategy for Current Capping in Multi-Power-Mode SSDs A Flash Scheduling Strategy for Current Capping in Multi-Power-Mode SSDs Li-Pin Chang, Chia-Hsiang Cheng, and Kai-Hsiang Lin Department of Computer Science National Chiao-Tung University, Taiwan Presented

More information

A Memory Management Scheme for Hybrid Memory Architecture in Mission Critical Computers

A Memory Management Scheme for Hybrid Memory Architecture in Mission Critical Computers A Memory Management Scheme for Hybrid Memory Architecture in Mission Critical Computers Soohyun Yang and Yeonseung Ryu Department of Computer Engineering, Myongji University Yongin, Gyeonggi-do, Korea

More information

BROMS: Best Ratio of MLC to SLC

BROMS: Best Ratio of MLC to SLC BROMS: Best Ratio of MLC to SLC Wei Wang 1, Tao Xie 2, Deng Zhou 1 1 Computational Science Research Center, San Diego State University 2 Computer Science Department, San Diego State University Partitioned

More information

Reducing Solid-State Storage Device Write Stress Through Opportunistic In-Place Delta Compression

Reducing Solid-State Storage Device Write Stress Through Opportunistic In-Place Delta Compression Reducing Solid-State Storage Device Write Stress Through Opportunistic In-Place Delta Compression Xuebin Zhang, Jiangpeng Li, Hao Wang, Kai Zhao and Tong Zhang xuebinzhang.rpi@gmail.com ECSE Department,

More information

Part II: Data Center Software Architecture: Topic 2: Key-value Data Management Systems. SkimpyStash: Key Value Store on Flash-based Storage

Part II: Data Center Software Architecture: Topic 2: Key-value Data Management Systems. SkimpyStash: Key Value Store on Flash-based Storage ECE 7650 Scalable and Secure Internet Services and Architecture ---- A Systems Perspective Part II: Data Center Software Architecture: Topic 2: Key-value Data Management Systems SkimpyStash: Key Value

More information

Open-Channel SSDs Offer the Flexibility Required by Hyperscale Infrastructure Matias Bjørling CNEX Labs

Open-Channel SSDs Offer the Flexibility Required by Hyperscale Infrastructure Matias Bjørling CNEX Labs Open-Channel SSDs Offer the Flexibility Required by Hyperscale Infrastructure Matias Bjørling CNEX Labs 1 Public and Private Cloud Providers 2 Workloads and Applications Multi-Tenancy Databases Instance

More information

Introduction to Open-Channel Solid State Drives and What s Next!

Introduction to Open-Channel Solid State Drives and What s Next! Introduction to Open-Channel Solid State Drives and What s Next! Matias Bjørling Director, Solid-State System Software September 25rd, 2018 Storage Developer Conference 2018, Santa Clara, CA Forward-Looking

More information

Gecko: Contention-Oblivious Disk Arrays for Cloud Storage

Gecko: Contention-Oblivious Disk Arrays for Cloud Storage Gecko: Contention-Oblivious Disk Arrays for Cloud Storage Ji-Yong Shin Cornell University In collaboration with Mahesh Balakrishnan (MSR SVC), Tudor Marian (Google), and Hakim Weatherspoon (Cornell) FAST

More information

Open-Channel SSDs Then. Now. And Beyond. Matias Bjørling, March 22, Copyright 2017 CNEX Labs

Open-Channel SSDs Then. Now. And Beyond. Matias Bjørling, March 22, Copyright 2017 CNEX Labs Open-Channel SSDs Then. Now. And Beyond. Matias Bjørling, March 22, 2017 What is an Open-Channel SSD? Then Now - Physical Page Addressing v1.2 - LightNVM Subsystem - Developing for an Open-Channel SSD

More information

Fusion Engine Next generation storage engine for Flash- SSD and 3D XPoint storage system

Fusion Engine Next generation storage engine for Flash- SSD and 3D XPoint storage system Fusion Engine Next generation storage engine for Flash- SSD and 3D XPoint storage system Fei Liu, Sheng Qiu, Jianjian Huo, Shu Li Alibaba Group Santa Clara, CA 1 Software overhead become critical Legacy

More information

HPDedup: A Hybrid Prioritized Data Deduplication Mechanism for Primary Storage in the Cloud

HPDedup: A Hybrid Prioritized Data Deduplication Mechanism for Primary Storage in the Cloud HPDedup: A Hybrid Prioritized Data Deduplication Mechanism for Primary Storage in the Cloud Huijun Wu 1,4, Chen Wang 2, Yinjin Fu 3, Sherif Sakr 1, Liming Zhu 1,2 and Kai Lu 4 The University of New South

More information

Optimizing SSD Operation for Linux

Optimizing SSD Operation for Linux ACTINEON, INC. Optimizing SSD Operation for Linux 1.00 Davidson Hom 3/27/2013 This document contains recommendations to configure SSDs for reliable operation and extend write lifetime in the Linux environment.

More information

Benchmarking Enterprise SSDs

Benchmarking Enterprise SSDs Whitepaper March 2013 Benchmarking Enterprise SSDs When properly structured, benchmark tests enable IT professionals to compare solid-state drives (SSDs) under test with conventional hard disk drives (HDDs)

More information

I/O Stack Optimization for Smartphones

I/O Stack Optimization for Smartphones I/O Stack Optimization for Smartphones Sooman Jeong 1, Kisung Lee 2, Seongjin Lee 1, Seoungbum Son 2, and Youjip Won 1 1 Dept. of Electronics and Computer Engineering, Hanyang University 2 Samsung Electronics

More information

Barrier Enabled IO Stack for Flash Storage

Barrier Enabled IO Stack for Flash Storage Barrier Enabled IO Stack for Flash Storage Youjip Won, Jaemin Jung, Gyeongyeol Choi, Joontaek Oh, Seongbae Son, Jooyoung Hwang, Sangyeun Cho Hanyang University Texas A&M University Samsung Electronics

More information

CBM: A Cooperative Buffer Management for SSD

CBM: A Cooperative Buffer Management for SSD 3 th International Conference on Massive Storage Systems and Technology (MSST 4) : A Cooperative Buffer Management for SSD Qingsong Wei, Cheng Chen, Jun Yang Data Storage Institute, A-STAR, Singapore June

More information

Improving throughput for small disk requests with proximal I/O

Improving throughput for small disk requests with proximal I/O Improving throughput for small disk requests with proximal I/O Jiri Schindler with Sandip Shete & Keith A. Smith Advanced Technology Group 2/16/2011 v.1.3 Important Workload in Datacenters Serial reads

More information

Using MLC Flash to Reduce System Cost in Industrial Applications

Using MLC Flash to Reduce System Cost in Industrial Applications Using MLC Flash to Reduce System Cost in Industrial Applications Chris Budd SMART High Reliability Solutions Santa Clara, CA 1 Introduction Component selection: cost versus quality Use same component to

More information

Bitmap discard operation for the higher utilization of flash memory storage

Bitmap discard operation for the higher utilization of flash memory storage LETTER IEICE Electronics Express, Vol.13, No.2, 1 10 Bitmap discard operation for the higher utilization of flash memory storage Seung-Ho Lim 1a) and Woo Hyun Ahn 2b) 1 Division of Computer and Electronic

More information

OS-caused Long JVM Pauses - Deep Dive and Solutions

OS-caused Long JVM Pauses - Deep Dive and Solutions OS-caused Long JVM Pauses - Deep Dive and Solutions Zhenyun Zhuang LinkedIn Corp., Mountain View, California, USA https://www.linkedin.com/in/zhenyun Zhenyun@gmail.com 2016-4-21 Outline q Introduction

More information

Physical Disentanglement in a Container-Based File System

Physical Disentanglement in a Container-Based File System Physical Disentanglement in a Container-Based File System Lanyue Lu, Yupu Zhang, Thanh Do, Samer AI-Kiswany Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau University of Wisconsin Madison Motivation

More information

Differential RAID: Rethinking RAID for SSD Reliability

Differential RAID: Rethinking RAID for SSD Reliability Differential RAID: Rethinking RAID for SSD Reliability Mahesh Balakrishnan Asim Kadav 1, Vijayan Prabhakaran, Dahlia Malkhi Microsoft Research Silicon Valley 1 The University of Wisconsin-Madison Solid

More information

Recent Advances in Analytical Modeling of SSD Garbage Collection

Recent Advances in Analytical Modeling of SSD Garbage Collection Recent Advances in Analytical Modeling of SSD Garbage Collection Jianwen Zhu, Yue Yang Electrical and Computer Engineering University of Toronto Flash Memory Summit 2014 Santa Clara, CA 1 Agenda Introduction

More information

SSD Architecture for Consistent Enterprise Performance

SSD Architecture for Consistent Enterprise Performance SSD Architecture for Consistent Enterprise Performance Gary Tressler and Tom Griffin IBM Corporation August 21, 212 1 SSD Architecture for Consistent Enterprise Performance - Overview Background: Client

More information

Zettabyte Reliability with Flexible End-to-end Data Integrity

Zettabyte Reliability with Flexible End-to-end Data Integrity Zettabyte Reliability with Flexible End-to-end Data Integrity Yupu Zhang, Daniel Myers, Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau University of Wisconsin - Madison 5/9/2013 1 Data Corruption Imperfect

More information

Disks and RAID. CS 4410 Operating Systems. [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse]

Disks and RAID. CS 4410 Operating Systems. [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse] Disks and RAID CS 4410 Operating Systems [R. Agarwal, L. Alvisi, A. Bracy, E. Sirer, R. Van Renesse] Storage Devices Magnetic disks Storage that rarely becomes corrupted Large capacity at low cost Block

More information

Copyright 2014 Fusion-io, Inc. All rights reserved.

Copyright 2014 Fusion-io, Inc. All rights reserved. Snapshots in a Flash with iosnap TM Sriram Subramanian, Swami Sundararaman, Nisha Talagala, Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau Presented By: Samer Al-Kiswany Snapshots Overview Point-in-time representation

More information

ECE 598-MS: Advanced Memory and Storage Systems Lecture 7: Unified Address Translation with FlashMap

ECE 598-MS: Advanced Memory and Storage Systems Lecture 7: Unified Address Translation with FlashMap ECE 598-MS: Advanced Memory and Storage Systems Lecture 7: Unified Address Translation with Map Jian Huang Use As Non-Volatile Memory DRAM (nanoseconds) Application Memory Component SSD (microseconds)

More information

Chapter 12 Wear Leveling for PCM Using Hot Data Identification

Chapter 12 Wear Leveling for PCM Using Hot Data Identification Chapter 12 Wear Leveling for PCM Using Hot Data Identification Inhwan Choi and Dongkun Shin Abstract Phase change memory (PCM) is the best candidate device among next generation random access memory technologies.

More information

FILE SYSTEMS, PART 2. CS124 Operating Systems Fall , Lecture 24

FILE SYSTEMS, PART 2. CS124 Operating Systems Fall , Lecture 24 FILE SYSTEMS, PART 2 CS124 Operating Systems Fall 2017-2018, Lecture 24 2 Last Time: File Systems Introduced the concept of file systems Explored several ways of managing the contents of files Contiguous

More information

Delayed Partial Parity Scheme for Reliable and High-Performance Flash Memory SSD

Delayed Partial Parity Scheme for Reliable and High-Performance Flash Memory SSD Delayed Partial Parity Scheme for Reliable and High-Performance Flash Memory SSD Soojun Im School of ICE Sungkyunkwan University Suwon, Korea Email: lang33@skku.edu Dongkun Shin School of ICE Sungkyunkwan

More information

NAND Flash-based Storage. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University

NAND Flash-based Storage. Jin-Soo Kim Computer Systems Laboratory Sungkyunkwan University NAND Flash-based Storage Jin-Soo Kim (jinsookim@skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Today s Topics NAND flash memory Flash Translation Layer (FTL) OS implications

More information

SMORE: A Cold Data Object Store for SMR Drives

SMORE: A Cold Data Object Store for SMR Drives SMORE: A Cold Data Object Store for SMR Drives Peter Macko, Xiongzi Ge, John Haskins Jr.*, James Kelley, David Slik, Keith A. Smith, and Maxim G. Smith Advanced Technology Group NetApp, Inc. * Qualcomm

More information

CS 318 Principles of Operating Systems

CS 318 Principles of Operating Systems CS 318 Principles of Operating Systems Fall 2018 Lecture 16: Advanced File Systems Ryan Huang Slides adapted from Andrea Arpaci-Dusseau s lecture 11/6/18 CS 318 Lecture 16 Advanced File Systems 2 11/6/18

More information

FlashKV: Accelerating KV Performance with Open-Channel SSDs

FlashKV: Accelerating KV Performance with Open-Channel SSDs FlashKV: Accelerating KV Performance with Open-Channel SSDs JIACHENG ZHANG, YOUYOU LU, JIWU SHU, and XIONGJUN QIN, Department of Computer Science and Technology, Tsinghua University As the cost-per-bit

More information

Design Considerations for Using Flash Memory for Caching

Design Considerations for Using Flash Memory for Caching Design Considerations for Using Flash Memory for Caching Edi Shmueli, IBM XIV Storage Systems edi@il.ibm.com Santa Clara, CA August 2010 1 Solid-State Storage In a few decades solid-state storage will

More information

GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction

GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction GearDB: A GC-free Key-Value Store on HM-SMR Drives with Gear Compaction Ting Yao 1,2, Jiguang Wan 1, Ping Huang 2, Yiwen Zhang 1, Zhiwen Liu 1 Changsheng Xie 1, and Xubin He 2 1 Huazhong University of

More information

ZBD: Using Transparent Compression at the Block Level to Increase Storage Space Efficiency

ZBD: Using Transparent Compression at the Block Level to Increase Storage Space Efficiency ZBD: Using Transparent Compression at the Block Level to Increase Storage Space Efficiency Thanos Makatos, Yannis Klonatos, Manolis Marazakis, Michail D. Flouris, and Angelos Bilas {mcatos,klonatos,maraz,flouris,bilas}@ics.forth.gr

More information

Warming up Storage-level Caches with Bonfire

Warming up Storage-level Caches with Bonfire Warming up Storage-level Caches with Bonfire Yiying Zhang Gokul Soundararajan Mark W. Storer Lakshmi N. Bairavasundaram Sethuraman Subbiah Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau 2 Does on-demand

More information

Don t stack your Log on my Log

Don t stack your Log on my Log Don t stack your Log on my Log Jingpei Yang, Ned Plasson, Greg Gillis, Nisha Talagala, Swaminathan Sundararaman Oct 5, 2014 c 1 Outline Introduction Log-stacking models Problems with stacking logs Solutions

More information

Towards Efficient, Portable Application-Level Consistency

Towards Efficient, Portable Application-Level Consistency Towards Efficient, Portable Application-Level Consistency Thanumalayan Sankaranarayana Pillai, Vijay Chidambaram, Joo-Young Hwang, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau 1 File System Crash

More information

arxiv: v1 [cs.os] 24 Jun 2015

arxiv: v1 [cs.os] 24 Jun 2015 Optimize Unsynchronized Garbage Collection in an SSD Array arxiv:1506.07566v1 [cs.os] 24 Jun 2015 Abstract Da Zheng, Randal Burns Department of Computer Science Johns Hopkins University Solid state disks

More information

Enabling NVMe I/O Scale

Enabling NVMe I/O Scale Enabling NVMe I/O Determinism @ Scale Chris Petersen, Hardware System Technologist Wei Zhang, Software Engineer Alexei Naberezhnov, Software Engineer Facebook Facebook @ Scale 800 Million 1.3 Billion 2.2

More information

UCS Invicta: A New Generation of Storage Performance. Mazen Abou Najm DC Consulting Systems Engineer

UCS Invicta: A New Generation of Storage Performance. Mazen Abou Najm DC Consulting Systems Engineer UCS Invicta: A New Generation of Storage Performance Mazen Abou Najm DC Consulting Systems Engineer HDDs Aren t Designed For High Performance Disk 101 Can t spin faster (200 IOPS/Drive) Can t seek faster

More information

Don t Let RAID Raid the Lifetime of Your SSD Array

Don t Let RAID Raid the Lifetime of Your SSD Array Don t Let RAID Raid the Lifetime of Your SSD Array Sangwhan Moon Texas A&M University A. L. Narasimha Reddy Texas A&M University Abstract Parity protection at system level is typically employed to compose

More information

[537] Flash. Tyler Harter

[537] Flash. Tyler Harter [537] Flash Tyler Harter Flash vs. Disk Disk Overview I/O requires: seek, rotate, transfer Inherently: - not parallel (only one head) - slow (mechanical) - poor random I/O (locality around disk head) Random

More information