Tri-Hybrid SSD with storage class memory (SCM) and MLC/TLC NAND Flash Memories Chihiro Matsui, Tomoaki Yamada, Yusuke Sugiyama, Yusuke Yamaga, and Ken Takeuchi Chuo University, Japan Santa Clara, CA 1
Outline Introduction Hybrid SSDs with 1. MLC and TLC NAND flash memories 2. SCM and MLC NAND flash memory 3. SCM, MLC and TLC NAND flash memories Summary Santa Clara, CA 2
Outline Introduction Hybrid SSDs with 1. MLC and TLC NAND flash memories 2. SCM and MLC NAND flash memory 3. SCM, MLC and TLC NAND flash memories Summary Santa Clara, CA 3
SCM and NAND Flash Memory in Memory Hierarchy Access time 1ns 1ns 1us 1ms CPU SRAM DRAM Storage Class Memory (SCM) MLC NAND flash TLC NAND flash MRAM, ReRAM, PRAM, and 3D XPoint Capacity Santa Clara, CA 4
Current [ua] ReRAM Set/Reset Time Short set/reset time if no verify operation 15 4 1 5-5 -1-15 Reset current Set current Reset voltage Set voltage -4 5 1 15 2 Time [ns] [1] S. Ning et al., IMW, 213, pp. 56-59. Santa Clara, CA 5 2-2 Voltage [V]
Program BER [a.u.] Program Error in ReRAM Program BER increases with set/reset cycles 6 4 Series1 LRS Series2 HRS @RT, 1kbit 2 1 1 1 1 2 1 3 1 4 1 5 Set/Reset cycles [1] K. Maeda et al., IRPS, 217, pp. 5A-4. Santa Clara, CA 6
Write latency [us] Write latency [us] MLC and TLC NAND Flash Latency TLC flash has longer read/write latency than MLC flash 6 MLC flash 6 TLC flash 4 2 Upper page Lower page 1.5x longer 4 2 Upper page Middle page Lower page 2 4 6 Write/Erase cycles 5 1 Write/Erase cycles [1] C. Matsui et al., SSDM, 216, pp. 15-16. Santa Clara, CA 7
Program disturb error (a.u.) Program disturb error [a.u.] Retention error (a.u.) Data retention error [a.u.] Errors in MLC and TLC NAND Flash TLC flash has higher error rate than MLC flash 1 1 1-1 Program disturb error TLC flash 1 Data retention error TLC flash 1-2 1-3 MLC flash 1-1.1 MLC flash 1-4.25.5.75 1 Write/Erase Write/Erase cycle cycle [a.u.].1 1-2 1.E+2 1-2 1.E+3 1-1 1.E+4 1 Time [a.u.] [1] S. Hachiya et al., JJAP, vol. 53, no. 4S, pp. 4EE4-1 - 4EE4-1, 214. Santa Clara, CA 8
Outline Introduction Hybrid SSDs with 1. MLC and TLC NAND flash memories 2. SCM and MLC NAND flash memory 3. SCM, MLC and TLC NAND flash memories Summary Santa Clara, CA 9
Average overwrite Average read frequency Application Characteristics Hot Cold 1 8 6 4 2 prxy_ proj_ hm_ web_1 2 4 6 8 1 Random write request (%) Sequential Write intensive Random Hot Cold 1 8 6 4 2 Sequential Read intensive prxy_1 src2_1 proj_3 2 4 6 8 1 Random read request (%) Random Application characteristics defined by Average overwrite = Total write data size / user data size Average read frequency = Total read data size / user data size Random: data size is 8KB or less (half of page size) [1] S. Okamoto et al., IMW, pp. 157-16, 215. Santa Clara, CA [2] MSR Cambridge Traces, http://iota.snia.org/traces/388. 1
Data Management Strategy Determined by memory characteristics used in hybrid SSD Latency: Endurance: Reliability: Capacity: Cost: Short High High Small High Memory characteristics Long Low Low Large Low SCM MLC flash MLC NAND flash TLC flash TLC NAND flash Hot data Data characteristics to be stored Frozen data Santa Clara, CA 11
1) MLC and TLC NAND Flash Hybrid SSD Frozen data are collected in TLC NAND flash when MLC Flash GC Write operation: (1) Write to MLC NAND flash (2) Return write complete to host Write request from host (a) (2) Frozen data eviction operation: (1) Search frozen and static data during MLC MLC NAND NAND flash flash GC operation (2) Evict to TLC NAND flash (1) MLC NAND flash (2) TLC NAND flash Read request from host (1) (1) (2) (2) Read operation: (1) Read from MLC or TLC NAND flash (2) Return read complete to host [1] S. Hachiya et al., SSDM, 213, pp. 894-895. Santa Clara, CA 12
Write Throughput [MB/s] MLC flash W/E 1) MLC and TLC NAND Flash Hybrid SSD TLC flash improves the hybrid SSD performance and prolongs MLC flash endurance 12 9 6 3 Write performance +3% 15 12 9 6 3 Write energy [J/GB] 25 5 75 1 Optimum TLC flash capacity [%] capacity ratio 2 16 12 Endurance [1] S. Hachiya et al., JJAP, vol. 53, no. 4S, pp. 4EE4-1 - 4EE4-1, 214. Santa Clara, CA 13 8 4 w/ prxy_1 (Hot Random) 25 5 75 1 TLC flash capacity [%] 5 4 3 2 1 NW/E,TLC/N W/E,MLC [%]
2) SCM and MLC NAND Flash Hybrid SSD SCM stores hot or random data. No data deduplication in between SCM and MLC flash Hot or random write request from host Hot and random write operation: (1) Write to SCM and update LRU table (2) Return write complete to host (1) (2) Cold data eviction operation: (1) Search cold and sequential data in SCM (2) Evict to MLC NAND flash DRAM LRU table (1) SCM (Storage) (2) MLC NAND flash Cold and sequential write request from host Cold and sequential write operation: (1) Write to MLC NAND flash (2) Return write complete to host (1) (2) [1] C. Sun et al., TCAS-I, vol. 61, no. 2, pp. 382-391, 214. Santa Clara, CA 14
Normalized IOPS to MLC flash only SSD 1 2) SCM and MLC NAND Flash Hybrid SSD SCM capacity ratio WO-DM 1% WO-DM 5% WO-DM 1% w/ SCM read/write latency 1ns 1 1 prxy_ proj_ I II hm_ III web_1 prxy_1 IV V proj_3 VII src2_1 VIII Hot Hot Cold Cold Hot Cold Cold Random Seq. Random Seq. Random Random Seq. Write intensive Read intensive [1] C. Sun et al., TCAS-I, vol. 61, no. 8, pp. 236-2369, 214. Santa Clara, CA [2] S. Okamoto et al., IMW, pp. 157-16, 215. 15
3) SCM and MLC/TLC NAND Flash Tri-hybrid SSD SCM stores hot or random data, and TLC stores frozen data Hot or random write request from host Hot and random write operation: (1) Read from SCM or MLC NAND flash (2) Return write complete to host Frozen data eviction algorithm: (2) Frozen data (1) eviction Search operation: frozen and static data during (1) Search frozen MLC and NAND static flash data GC in MLC operation NAND flash (1) (2) Evict to (2) TLC Evict NAND to TLC flashnand flash DRAM LRU table SCM (Storage) (1) MLC NAND flash (2) TLC NAND flash Cold and sequential write request from host (1) Cold and sequential write operation: (1) Write to MLC NAND flash (2) Return write complete to host (2) [1] C. Matsui et al., SSDM, 216, pp. 15-16. Santa Clara, CA 16
Normalized IOPS to Normalized MLC flash only IOPS SSD Normalized IOPS to Normalized MLC flash only IOPS SSD 3) SCM and MLC/TLC NAND Flash Tri-hybrid SSD SCM improves SSD performance of hot applications prxy_ workload (Hot Random) w/ SCM read/write latency 1ns proj_ workload (Hot Seq.) 15 125 1 75 5 25 % 5% 75% 6 45 3 15 % 5% 75% [1] C. Matsui et al., JJAP, vol. 56, no. 4S, p. 4CE2-1 - 4CE2-9, 217. Santa Clara, CA 17
Capacity ratio TLC flash MLC flash 3) Optimal Memory Capacity in Tri-hybrid SSD with 1.1-times Cost 1% 8% 6% 4% 2% % prxy_ proj_ hm_ src2_1 web_1 Hot Hot Cold Cold Random Seq. Random Seq. Write intensive w/ SCM read/write latency 1ns [1] C. Matsui et al., JJAP, vol. 56, no. 4S, p. 4CE2-1 - 4CE2-9, 217. Santa Clara, CA 18 SCM Bit cost ratio Storage class memory 1 MLC flash 1 TLC flash 2/3
Capacity ratio TLC flash MLC flash 3) Tri-hybrid SSD Performance with 1.1-times Cost 1% 8% 6% 4% 2% % prxy_ proj_ hm_ src2_1 web_1 Hot Hot Cold Cold Random Seq. Random Seq. Write intensive [1] C. Matsui et al., JJAP, vol. 56, no. 4S, p. 4CE2-1 - 4CE2-9, 217. Santa Clara, CA 19 SCM w/ SCM read/write latency 1ns 1 5 1 Normalized IOPS to MLC flash only SSD
Outline Introduction Hybrid SSDs with 1. MLC and TLC NAND flash memories 2. SCM and MLC NAND flash memory 3. SCM, MLC and TLC NAND flash memories Summary Santa Clara, CA 2
Summary SCM and TLC flash store hot and frozen data, respectively in tri-hybrid SSD If 1% SSD cost increase is allowed: Small capacity of SCM boosts SSD performance by 6 times for hot random application Hot sequential or cold random applications require mix of SCM, MLC and TLC NAND flash memories Small SCM capacity is enough for cold sequential application Santa Clara, CA 21
Acknowledgement This presentation is based on results obtained from a project commissioned by the New Energy and Industrial Technology Development Organization (NEDO). Santa Clara, CA 22