Flash Trends: Challenges and Future

Transcription

1 Flash Trends: Challenges and Future John D. Davis work done at Microsoft Researcher- Silicon Valley in collaboration with Laura Caulfield*, Steve Swanson*, UCSD* 1

2 My Research Areas of Interest Flash characteristics Flash specialization (HW/SW co- Flash- Hardware accelerators (HW/SW co- Computer Architecture 2

3 tape is dead disk is tape flash is disk RAM locality is king - Jim Gray, Dec

4 The Access-Time Pyramid Decreasing Cost On- chip Memory Off- chip Memory On- line Storage Off- line Storage Access Human scale Time (ns) (t x 10 9 ) 1 CPU operations ( 1 ns) second 10 L2 Cache access (10 ns) 100 DRAM access (60 ns) minute 10 3 hour The Gap before NAND Flash! 10 4 NAND Flash read (20 s) 10 5 day 10 6 NAND Flash random write(1 ms) week 10 7 HDD read/write (5 ms) month 10 8 year 10 9 decade TAPE read (40 s) millenium 4

5 NAND Flash by the Data Sheet Block interface(like an HDD): page = 2KB 16KB Asymmetric read/write time (fast/slow) Sequential writes Random write and sequential write almost the same Random read = fast, no seek penalty! Block erase ( pages) No write- in place & slow Low write endurance (<10 6 ) Can wear out fast Scalability beyond 1X nm (at 19 nm now) Toshiba introducing 3D lithography in a couple of years!??? 6

6 The NAND Flash Black Hole in 2008 How do bits fail? What does endurance mean? What about data retention and bit rot? Average/typical/maximum latency? Characteristics change with scaling? Write out- of- order? Program\read disturb? Page layout? 7

7 FLASH BASICS 8

8 Reliability Performance Cost Per Capacity 9

9 Reliability Performance Decreasing Write Budget Increasing Density Cost Per Capacity 10

10 Reliability Performance Decreasing Write Budget Increasing Density Cost Per Capacity Write Latency (µs) Expected Performance Gap 0 Low Density Flash Disk Drives 11

11 Reliability Performance Decreasing Write Budget Increasing Density Cost Per Capacity Will the price decline be enough? Write Latency (µs) Expected Performance Gap What performance & scaling trends can we expect from our SSDs? Low Density Flash Disk Drives High Density Flash 12

12 Flash Chip Operation Page: n- 4 n- 3 n- 2 n- 1 n Block 0 SLC: Single Level Cell Block 1 Block 2 Block n A B C Erase Blocks Before Programming Program Pages In Order == 1 bit MLC: Multi Level Cell == 2 bits TLC: Triple Level Cell == 3 bits 13

13 Collecting Flash Latency Trends Xilinx XUP Board Custom Built Daughter Board Full- fledge Linux Kernel Module 10ns resolution Chip Collection 45 chips 6 companies 25nm- 72nm SLC, MLC, TLC Read Erase Repeat 10x Lifetime Read Program 14

14 Program Operation Latency 2,500 2,000 SLC MLC Program Time(µs) 1,500 1, A- SLC2 A- SLC4 A- SLC8 B- SLC2 50nm B- SLC4 72nm E- SLC8 B- MLC8 72nm B- MLC32 50nm C- MLC32 C- MLC64 43nm D- MLC16 D- MLC32 50nm E- MLC4 E- MLC8 51nm 15

15 Program Latency Anomaly Program Time (µs) 2,500 2,000 1,500 1, Fast 50% of Pages Slow 50% of Pages Page #: - A- SLC2 A- SLC4 A- SLC8 B- SLC2 50nm B- SLC4 72nm E- SLC8 B- MLC8 72nm B- MLC32 50nm C- MLC32 C- MLC64 43nm D- MLC16 D- MLC32 50nm E- MLC8 E- MLC4 51nm 16

16 Paired Pages: High & Low Order Bits Flash State Logical Data High Order (fast) 0 Low Order (slow) 0 Different Bits, Different Behaviors Harey Tortoise [USENIX13] Industry Impact? Datasheets Before and After Fault Tolerance Study 17

17 Flexible Dimensions of Reliability Media Caching: High BER, Many Cycles Caching: Many Cycles, Short Data Life P/E Cycles (Lifetime) Enterpise emlc 3 months, 30,000 cycles Conventional MLC 10 years, 5,000 cycles OS Code: Low BER, Few Cycles Long- Term Cold Storage: Long Data Life, Few Cycles Sensor Node: Low BER, Short Data Life : High BER, Long Data Life 19

18 FLASH TRENDS 20

19 Predicting Future Flash-Based SSDs Fixed SSD Architecture Flash Chip Trends SSD Trends 21

20 The Constant-Die-Count SSD (SSD-CDC) PCIe Link Channel 0 Flash Die 0 Channel 1 Flash Die 1 Flash Die 2 Flash Die 3 Represents High- End (FusionIO, Virident, OCZ) Baseline 96 dies 320 GB 34nm, MLC Controller Flash Die 0... Flash Die 1... Flash Die 2... Flash Die 3... Scaling from Chips SLC, MLC, TLC To 6nm by 2024 (ITRS) Channel 23 Flash Die 0 Flash Die 1 Flash Die 2 Flash Die 3 Assumptions Constant die count Unlimited PCIe Link Channel Speed: 400MB/s 22

21 Capacity Best Possible by SSD Capacity (GB) TLC- 3 MLC- 2 SLC- 1 SSD- CDC 43x Feature Size (nm)

22 Empirical Data Chip Write Latency (ms) TLC- 3 MLC- 2 SLC Feature Size (nm) 8 24

23 Scaling Trends in Empirical Data 3.0 Chip Write Latency (ms) SLC- 1 MLC- 2 TLC- 3 2x 2x Feature Size (nm) 8 25

24 Write Latency of Fixed-Sized SSD SSD Write Latency (ms) 3.0 TLC- 3 MLC SLC x SSD Capacity (GB) 26

25 Reduced Bandwidth SSD- CDC Write Bandwidth (MB/s) Increased Page Size MLC: 4kB, TLC 8kB SSD Capacity (GB) 0.7x SLC- 1 MLC- 2 TLC- 3 27

26 IOPs 512B Random Accesses SSD- CDC Write kiops SLC- 1 MLC- 2 TLC- 3 Fastest HDD: 0.2 kiops Our Slowest SSD: 32.0 kiops 0.4x SSD Capacity (GB) 28

27 SPECIALIZING FOR FLASH 29

28 High Density, Large Variation 8000 Advantages: Two Distinct Performance Bins Write Latency (µs) Expected Performance Gap Low Density Flash Worst Case Best Case Disk Drives High Density Flash Equal Division of Pages Between Bins Regular pattern Consistent Between Vendors Challenges: Pattern of fast and slow areas SSD management algorithms 30

29 Variation-Aware Interface Extend Interface of Page- Mapped FTL fast page data data data data data data data data data + Lower latency when it matters - Increase wear 31

30 Multi-Chip Variability Aware FTL: Many-Write Points for Increased Flexibility This Work: Storage Array Leverage write variability in how WP is chosen Chip Erased Block Written Block Write Point (WP) Slow Page Latency == 4.6x Fast Page Latency 32

31 For Bursts: Return To Fast (RTF) Baseline All Fast Solution for Bursty Traffic 0xC001 0xC001 2x Wear 0xC001 0xBA11 0xBA11 0xGCMv 0xGCMv 0xBEEF 0xBA11 0xBEEF 0xCA11 0xCA11 0xGCMv 0xGCMv 0xAB1E 0xBEEF 0xAB1E 0xFEED 0xFEED 0xGCMv 0xGCMv 0xCA11 Fast Page Slow Page 0xAB1E 0xFEED 33

32 Results: Improved Performance 120 Bandwidth (MB/s) Performance Opportunity Burst Size (kb) Baseline All All Fast Fast 1 WP/Chip 1 x 8 2 WP/Chip 2 x 8 4 WP/Chip 4 x 8 8 WP/Chip 8 x WP/Chip x 8 34

33 Results: Improved Performance 120 Bandwidth (MB/s) More WPs Sustain High Performance for Larger Bursts Wear of All Fast is 2x Baseline RTF is on par with Baseline Burst Size (kb) Baseline All Fast 1 WP/Chip 2 WP/Chip 4 WP/Chip 8 WP/Chip 16 WP/Chip 35

34 Multi-Chip Variability Aware FTL: Many-Write Points for Increased Flexibility This Work: Storage Array Leverage write variability in how WP is chosen Technique for coping with alternating page latencies: More Write Points (WPs) Chip Erased Block Written Block Write Point (WP) Slow Page Latency == 4.6x Fast Page Latency 36

35 Workloads Normalized Burst BW 1.8 Baseline 1 WP/Chip 8 WP/Chip 32 WP/Chip AllFast Build Financial WebIndex Swap DeskDev. Average 37

36 FTL under sustained write load Constant Stream of External Writes Chip Busy with External Write Chip Busy with Online GC Requirement: Match rates of external writes and GC Variability: Which page type for external and GC writes? 38

37 Locality-Dependent Choice Normalized BW Baseline FGC SGC Locality (% of Block Moved By User Writes) 39

38 Page Choice in Busy Array Storage Array More WPs per Chip More page type Options Idle Chip Busy Chip Erased Block Written Block Write Point (WP) 40

39 FGC and SGC FGC and SGC affects amplify with more write points per chip 1.4 Normalized Bandwidth FGC: Fast Garbage Collection, Slow User SGC: Slow Garbage Collection, Fast User Write Points Per Chip (WPs/Chip) 41

40 Workload Performance Normalized Write Bandwidth SGC- 1 SGC- 8 SGC- 32 Baseline FGC- 1 FGC- 8 FGC- 32 AllFast Build Financial Swap DeskDev Average WebIndex 42

41 Flash Conclusion Chip Characterization: Revealing a New Landscape to the NAND Flash Community Leveraging large- scale flash drives: Write point per chip for parallelism Flash Trends: Capacity: 43x, Latency: 2.6x, Bandwidth: 40% Chip & Application Symbiosis: SLC performance in MLC parts > 1 write point per chip Bursts: up to 100% (89% on average) of SLC performance Sustained: up to 95% (65% on average) performance improvement 43

42 QUESTIONS? 44