[537] Flash. Tyler Harter

Transcription

1 [537] Flash Tyler Harter

2 Flash vs. Disk

3 Disk Overview I/O requires: seek, rotate, transfer Inherently: - not parallel (only one head) - slow (mechanical) - poor random I/O (locality around disk head) Random requests take 10ms+

4 Flash Overview Hold charge in cells. No moving parts! Inherently parallel. No seeks!

5 Disks vs. Flash: Performance Throughput: - disk: ~130 MB/s (sequential) - flash: ~4 MB/s Latency - disk: ~10 ms (one op) - flash - read: us - program: 2-5 us - erase: 2 ms types of write, more later

6 Disks vs. Flash: Capacity An obvious question is why are we talking about spinning disks at all, rather than SSDs, which have higher IOPS and are the future of storage. The root reason is that the cost per GB remains too high, and more importantly that the growth rates in capacity/$ between disks and SSDs are relatively close (at least for SSDs that have sufficient numbers of program-erase cycles to use in data centers), so that cost will not change enough in the coming decade. We do make extensive use of SSDs, but primarily for high performance workloads and caching, and this helps disks by shifting seeks to SSDs. ~ Eric Brewer et al. Source:

7 Flash Architecture

8 SLC: Single-Level Cell charge NAND Cell

9 SLC: Single-Level Cell charge 1 NAND Cell

10 SLC: Single-Level Cell charge 0 NAND Cell

11 MLC: Multi-Level Cell charge NAND Cell

12 MLC: Multi-Level Cell charge NAND Cell

13 MLC: Multi-Level Cell charge 10 NAND Cell

14 MLC: Multi-Level Cell charge NAND Cell

15 Single- vs. Multi- Level Cell SLC MLC charge charge

16 Single- vs. Multi- Level Cell SLC MLC charge charge expensive robust cheap sensitive

17 Wearout Problem: flash cells wear out after being overwritten too many times. MLC: ~10K times SLC: ~1K times Usage strategy:???

18 Wearout Problem: flash cells wear out after being overwritten too many times. MLC: ~10K times SLC: ~1K times Usage strategy: wear leveling. - prevents some cells from wearing out while others still fresh.

19 Banks Flash devices are divided into banks (aka, planes). Banks can be accessed in parallel. Bank 0 Bank 1 Bank 2 Bank 3

20 Banks Flash devices are divided into banks (aka, planes). Banks can be accessed in parallel. read read Bank 0 Bank 1 Bank 2 Bank 3

21 Banks Flash devices are divided into banks (aka, planes). Banks can be accessed in parallel. Bank 0 Bank 1 Bank 2 Bank 3

22 Banks Flash devices are divided into banks (aka, planes). Banks can be accessed in parallel. data data Bank 0 Bank 1 Bank 2 Bank 3

23 Banks Flash devices are divided into banks (aka, planes). Banks can be accessed in parallel. Bank 0 Bank 1 Bank 2 Bank 3

24 Writing 0 s: - fast, fine-grained Flash Writes Writing 1 s: - slow, course-grained

25 Writing 0 s: - fast, fine-grained - called program Flash Writes Writing 1 s: - slow, course-grained - called erase

26 Flash Writes Writing 0 s: - fast, fine-grained [pages] - called program Writing 1 s: - slow, course-grained [blocks] - called erase

27 Bank 0 Bank 1 Bank 2 Bank 3

28 each bank contains many blocks Bank 0 Bank 2 Bank 3

29 Block

30 Block one block

31 Block one page

32 Block

33 Block program 11

34 Block 11

35 Block program 11

36 Block 11

37 Block program

38 Block 11 10

39 Block erase

40 Block erase

41 Block

42 APIs disk flash write read

43 APIs disk flash read read sector read page write

44 APIs disk flash read read sector read page write write sector program page (0 s) erase block (1 s)

45 Flash Hierarchy Bank/plane: 1024 to 4096 blocks - banks accessed in parallel Block: 64 to 256 pages - unit of erase Page: 2 to 8 KB - unit of read and program

46 Disk vs. Flash Performance Throughput: - disk: ~130 MB/s (sequential) - flash: ~4 MB/s Latency - disk: ~10 ms (one op) - flash - read: us - program: 2-5 us - erase: 2 ms

47 Working with File Systems

48 Traditional File Systems File System Storage Device Traditional API: - read sector - write sector

49 Traditional File Systems File System Storage Device Traditional API: - read sector - write sector not same as flash

50 Options 1. Build/use new file systems for flash - JFFS, YAFFS - lot of work! 2. Build traditional API over flash API. - use FFS, LFS, whatever we want

51 Traditional API with Flash read(addr): return flash_read(addr) write(addr, data): block_copy = flash_read(all pages of block) modify block_copy with data flash_erase(block of addr) flash_program(all pages of block_copy)

52 Memory: Flash: block 0 block 1 block 2

53 Memory: FS wants to write Flash: block 0 block 1 block 2

54 Memory: Flash: block 0 block 1 block 2

55 Memory: read all other pages in block Flash: block 0 block 1 block 2

56 Memory: Flash: block 0 block 1 block 2

57 Memory: modify target page in memory Flash: block 0 block 1 block 2

58 Memory: Flash: block 0 block 1 block 2

59 Memory: erase block Flash: block 0 block 1 block 2

60 Memory: Flash: block 0 block 1 block 2

61 Memory: program all pages in block Flash: block 0 block 1 block 2

62 Memory: Flash: block 0 block 1 block 2

63 Write Amplification Random writes are extremely expensive! Writing one 2KB page may cause: - read, erase, and program of 256KB block.

64 Write Amplification Random writes are extremely expensive! Writing one 2KB page may cause: - read, erase, and program of 256KB block. Would FFS or LFS be better with flash?

65 File Systems over Flash Copy-On-Write FS may prevent some expensive random writes. What about wear leveling? LFS won t do this. What if we want to use some other FS?

66 Flash Translation Layer

67 Better Solution Add copy-on-write layer between FS and flash. Avoids RMW (read-modify-write) cycle. Translate logical device addrs to physical addrs. FTL: Flash Translation Layer. Should translation use math or data structure?

68 Flash Translation Layer logical: physical: block 0 block 1

69 Flash Translation Layer write logical: physical: block 0 block 1

70 Flash Translation Layer write logical: physical: block 0 block 1

71 Flash Translation Layer write logical: physical: block 0 block 1

72 Flash Translation Layer logical: physical: block 0 block 1

73 Flash Translation Layer logical: must eventually be garbage collected physical: block 0 block 1

74 FTL Could be implemented as device driver or in firmware (usually the latter). Where to store mapping? SRAM. Physical pages can be in three states: - valid, invalid, free

75 States free valid invalid

76 States free program valid erase invalid

77 States free program valid erase relocate invalid

78 States free program valid erase relocate invalid why is file delete problematic?

79 States free program valid erase invalid relocate or TRIM SSD-aware file systems can tell device that page is garbage

80 SSD Architecture SSD: looks like disk FTL SRAM: mapping tbl

81 Problem: Big Mapping Table Assume 2GB device, 2KB pages, 4-byte entries. SRAM needed: (2GB / 2KB) * 4 bytes = 4 MB. Too big, SRAM is expensive!

82 Page Translations logical: physical: block 0 block 1

83 2-Page Translations logical: physical: block 0 block 1

84 Larger Mappings Advantage: larger mappings decrease table size. Disadvantage?

85 2-Page Translations logical: physical: block 0 block 1

86 2-Page Translations write 10 logical: physical: block 0 block 1

87 2-Page Translations write 10 logical: copy physical: block 0 block 1

88 2-Page Translations write 10 logical: physical: block 0 block 1

89 2-Page Translations write 10 logical: physical: block 0 block 1

90 Larger Mappings Advantage: larger mappings decrease table size. Disadvantages? - more read-modify-write updates - more garbage - less flexibility for placement

91 Hybrid FTL Use course-grained mapping for most (e.g., 95%) of data. Map at block level. Use fine-grained mapping for recent data. Map at page level.

92 Hybrid FTL logical: A B C D W X Y Z physical: A B C Z W X Y D block map logical physical page map logical physical

93 Strategy: Log Blocks Write changed pages to designated log blocks. After blocks become full, merge changes with old data. Eventually garbage collect old pages.

94 Merging

95 Merging Merging technique depends on I/O pattern. Three merge types: - full merge - partial merge - switch merge

96 Merging Merging technique depends on I/O pattern. Three merge types: - full merge - partial merge - switch merge

97 logical: physical: A B C D block 0 block 1 (log) block 2

98 write D2 logical: physical: A B C D block 0 block 1 (log) block 2

99 write D2 logical: physical: A B C D D2 block 0 block 1 (log) block 2

100 logical: physical: A B C D D2 block 0 block 1 (log) block 2

101 logical: physical: A B C D D2 block 0 block 1 (log) block 2 eventually, we need to get rid of red arrows, as these represent expensive mappings

102 logical: physical: A B C D D2 block 0 block 1 (log) block 2

103 logical: COPY physical: A B C D D2 A B C D2 block 0 block 1 (log) block 2

104 logical: physical: A B C D D2 A B C D2 block 0 block 1 (log) block 2

105 logical: physical: A B C D D2 A B C D2 block 0 block 1 (log) block 2

106 logical: garbage physical: A B C D D2 A B C D2 block 0 block 1 (log) block 2

107 Merging Merging technique depends on I/O pattern. Three merge types: - full merge - partial merge - switch merge

108 logical: physical: A B C D block 0 block 1 (log) block 2

109 write D2 logical: physical: A B C D block 0 block 1 (log) block 2

110 write D2 logical: physical: A B C D D2 block 0 block 1 (log) block 2

111 logical: physical: A B C D D2 block 0 block 1 (log) block 2

112 logical: physical: A B C D D2 block 0 block 1 (log) block 2 why was this a better pick?

113 logical: physical: A B C D A B C D2 block 0 block 1 (log) block 2

114 logical: physical: A B C D A B C D2 block 0 block 1 (log) block 2

115 logical: physical: A B C D A B C D2 block 0 block 1 block 2

116 logical: garbage physical: A B C D A B C D2 block 0 block 1 block 2

117 Merging Merging technique depends on I/O pattern. Three merge types: - full merge - partial merge - switch merge

118 logical: physical: A B C D block 0 block 1 (log) block 2

119 write A2 logical: physical: A B C D block 0 block 1 (log) block 2

120 write A2 logical: physical: A B C D A2 block 0 block 1 (log) block 2

121 logical: write B physical: A B C D A2 B2 block 0 block 1 (log) block 2

122 logical: write C physical: A B C D A2 B2 C2 block 0 block 1 (log) block 2

123 write D2 logical: physical: A B C D A2 B2 C2 D2 block 0 block 1 (log) block 2

124 logical: garbage physical: A B C D A2 B2 C2 D2 block 0 block 1 (log) block 2

125 Merging Merging technique depends on I/O pattern. Three merge types: - full merge [copy unmodified and modified] - partial merge [copy unmodified] - switch merge [no copy]

126 Merging Merging technique depends on I/O pattern. Three merge types: - full merge [copy unmodified and modified] - partial merge [copy unmodified] - switch merge [no copy] why not always do partial or switch merge?

127 Summary Flash is much faster than disk, but It is more expensive. It s not a drop-in replacement beneath an FS without a complex layer for emulating hard disk API.