Rechnerarchitektur (RA)

Transcription

1 12 Rechnerarchitektur (RA) Sommersemester 2017 Flash Memory 2017/07/12 Jian-Jia Chen (Slides are based on Tei-Wei Kuo and Yuan-Hao Chang) Informatik 12 Tel.:

2 Mögliche Stufen der Speicherhierarchie und derzeit eingesetzte Technologien Tertiärspeicher (Optische Speicher, Bänder) Sekundärspeicher (Platten, Flash) Plattencaches (DRAM, Flash) Haupt- bzw. Primärspeicher (DRAM) Caches, scratch pads, TLBs (SRAM) Register (SRAM) 12,

3 Why Flash Memory Diversified Application Domains Portable Storage Devices Consumer Electronics Industrial Applications Critical System Components Solid State Disks (SSD) 12,

4 Layout of Flash Memory 0 63 Block: basic eraseoperation unit. 0 (2KB + 64 Byte) (2KB + 64 Byte) (2KB + 64 Byte) (2KB + 64 Byte) Page: basic writeoperation unit (2KB + 64 Byte) (2KB + 64 Byte) 128MB Flash Memory 12,

5 Characteristics of Flash Memory Write-Once No writing on the same page unless its residing block is erased! Pages are classified into valid, invalid, and free pages. Bulk-Erasing Pages are erased in a block unit to recycle used but invalid pages. Wear-Leveling Each block has a limited lifetime in erasing cycles. E.g., 10,000 ~ 100,000 erase cycles for each block 12,

6 Terminology Valid data: the latest version of data stored in flash Invalid data: not the latest version of data stored in flash Live page: a page that stores valid data Dead page: a page that stores invalid data Free page: a page that is erased and is ready to store data Free block: a block that is erased and is not allocated to store any data Hot data: frequently updated data Valid hot data might become invalid in the near future. Cold data: non-frequently updated data Valid cold data might stay in the same place for a long time. 12,

7 Management Issues Flash-Memory Example 1: Out-place Update Characteristics A B C D A B Dead pages 12,

8 Management Issues Flash-Memory Example 2: Garbage Collection Characteristics L D D L D D L D L L D L L L F D L F L L L L D F F L L F L L F D This block is to be recycled. (3 live pages and 5 dead pages) A live page A dead page A free page 12,

9 Management Issues Flash-Memory Example 2: Garbage Collection Characteristics D D D D D D D D L L D L L L L D Live data are copied to somewhere else. L F L L L L D L L L L F L L F D A live page A dead page A free page 12,

10 Management Issues Flash-Memory Example 2: Garbage Collection Characteristics F F F F F F F F L L D L L L L D L F L L L L D L The block is then erased. Overheads: live data copying block erasing. L L L F L L F D A live page A dead page 12,

11 Management Issues Flash-Memory Example 3: Wear-Leveling Characteristics L D D L D D L D L L D L L L F D A B Wear-leveling might interfere with the decisions of the blockrecycling policy L F L L L L D F F L L F L L F D C D A live page A dead page A free page Erase cycle counts 12,

12 Single-Level Cell (SLC) vs. Multiple-Level Cell (MLC) A limited bound on erase cycles SLC : 100,000 SLC Flash MLC x2 : 10,000 Bit error probability 64 SLC: 10-9 MB 1 0 MLC: 10-6 a) BILEVEL (1 BIT/CELL) MLC Flash NUMBER OF CELLS LOGIC STATE V T 128 MB NUMBER OF CELLS b) MULTILEVEL (2 BIT/CELL) LOGIC STATE V T 12,

13 MLC vs. SLC (Cont.) Electronic Engineering Times, July ,

14 Price and Read/Write Performance NOR NAND SLC NAND MLC x2 Price $/GB 6.79 $/GB 2.48 $/GB Read MB/sec MB/sec 13.5 MB/sec Write 0.07 MB/sec 4.57 MB/sec 2.34 MB/sec Erase 0.22 MB/sec MB/sec MB/sec *NOR: Silicon Storage Technology (SST). NAND SLC: Samsung Electronics. K9F1G08Q0M. NAND MLCx2: ST STMicroelectronics [1,2] 1. Jian-Hong Lin, Yuan-Hao Chang, Jen-Wei Hsieh, Tei-Wei Kuo, and Cheng-Chih Yang, "A NOR Emulation Strategy over NAND Flash Memory," the 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), Daegu, Korea, August 21-24, Yuan-Hao Chang and Tei-Wei Kuo, A Log-based Management Scheme for Low-cost Flash-memory Storage Systems of Embedded Systems 12,

15 Flash Memory Management Management issues Write constraints imposed by flash memory Scalability issue Garbage collection Performance considerations Reliability issues Cell error rate problem imposed by MLC flash memory Error correction coding vs. wear leveling Read/write disturbance problem Data retention problem 12,

16 Typical System Architecture Application 1 Application 2... Application n fwrite (file, data) Shared Library Virtual File System (VFS)... Native Flash File System Layer (e.g., JFFS2, YAFFS) File-system Management Unit Allocator Address Translation / Block Assignment File Systems (e.g., DOS FAT) Flash Translation Layer (FTL) Allocator Address Translation / Block Assignment flash write (block, page) Control single Cleaner Garbage Collection Wear Leveler Cleaner Garbage Collection Memory Technology Device Layer (MTD) Flash Memory Wear Leveler 12,

17 Flash Translation Layer (FTL) FTL adopts a page-level address translation mechanism. The main problem of FTL is on large memory space requirements for storing the address translation information. Access LBA = 3 Logical Block Address (array index) Physical Block Address (block,page) Address Translation Table (in main-memory)... (1,3) (1,2) (2,1) (1,0) (4,7) (0,4) (0,6) (0,1) (0,3) Physical Block Address (block,page)... 1,3 1,2 1,1 1,0 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0,0 User data Spare data Flash memory Spare data LBA=3; ECC=...; Status=...; 12,

18 NAND Flash Translation Layer (NFTL) A logical address under NFTL is divided into a virtual block address and a block offset. e.g., LBA=1019 => virtual block address (VBA) = 1019 / 8 = 127 and block offset = 1019 % 8 = 3 Write data to LBA=1019 VBA=127 NFTL Address Translation Table (in main-memory)... (9,23)... Block Offset=3 A Primary Block Address = 9 Free Free Free Used Free If the page Free has been Free Write to the used Free first free page Free Free Free A Replacement Block Address = 23 Used Used Used Free Free 12,

19 FTL or NFTL FTL NFTL Memory Space Requirements Large Small Address Translation Time Short Long Garbage Collection Overhead Less More Space Utilization High Low The memory-space requirements for one 1GB NAND (2KB/Page, 4B/Table Entry, 128 Pages/Block) FTL: 2MB (= 4*(1024*1024*1024)/2K) NFTL: 32KB (= 2*4*(1024*1024*1024)/(2K * 128)) 12,

20 Size of Translation Tables 1GB 32GB 1TB 32TB FTL 2MB 64MB 2GB 64GB NFTL 32KB 1MB 32MB 1GB No matter which kind of granularity of address translation is adopted, the fast growing flash memory capacity would eventually make the translation table too large to be fitted in RAM. 12,

21 New Write Constraints of MLC Flash Write constraints Pages can only be written sequentially in a block. Partial page write/programming is prohibited. Impact on NFTL Data can t be written to any free page of primary blocks. The space utilization in primary blocks is even lower. Most writes are forced to be placed in the replacement block. Pages of invalid data can t be marked as dead. Each read operation should scan pages of the replacement block. 12,

22 Intuitive and Practical Solution Level-paging translation tables Pages of translation tables are stored in flash, and are cached in RAM. Problems: Hit ratio of cached pages Extra page reads and writes for translation information Crash recovery for translation table and lost data 9 bits Level 0 paging page Level 1 paging page 3 bits Chip 4 bytes... 2nd outerpage table LBA (4 bytes) 9 bits 4 bytes PBA (4 bytes) 17 bits Block empty... Outer-page table 2 9 bits bits Level 2 paging page Offset 4 bytes... Page table 7 bits Page 256GB Flash memory Page = 2KB 1 Block = 2 7 Pages 1 Chip = 2 17 Blocks Page (2KB) Hsieh, J.-W., Wu, C.-H., and Chiu, G.-M MFTL: A design and implementation for MLC flash memory storage systems. ACM Trans. Storage 8, 2, Article No. 7 12,

23 Performance Considerations MLC flash has growing market share: Reason: low cost and high density Drawbacks: low speed, low endurance, and low reliability Solutions: Hardware multi-channel programming Software multi-bank or multi-channel programming 12,

24 Wear Leveling No Wear Leveling Dynamic Wear Leveling Intuitive Perfect Static Wear Leveling Erase Cycles Physical Block Addresses (PBA) 12,

25 Static Wear Leveling Random policy It randomly select a block to reclaim after a fixed number of block erases or write requests. It doesn t track the locality of data acceses, such that it might move hot data that might turn dead in the near future. 12,

26 Static Wear Leveling (Cont.) Random policy with block-erasing table Each-bit flag of the table is to indicate whether the corresponding blocks have been erased. Whenever the block erases are not even enough, select blocks whose corresponding bit flag are not set. Pros and Cons: Pros: it can identify the locality of data accesses. Cons: the block-erasing table needs extra RAM space even it is comparatively small. Yuan-Hao Chang, Jen-Wei Hsieh, Tei-Wei Kuo: Endurance Enhancement of Flash-Memory Storage, Systems: An Efficient Static Wear Leveling Design. DAC 2007: ,

27 The Block Erasing Table (BET) A bit-array: Each bit is for 2 k consecutive blocks. Small k in favor of hot-cold data separation Large k in favor of small RAM space e cnt =2 =3 =0 =1 f cnt =2 =0 =1 Flash e cnt =1 =0 =2 =3 =4 f cnt =1 =0 = BET k=0 k=2 e cnt : the total number of block erases done since the BET is reset f cnt : the number of 1 s in the BET : an index to a block that the Cleaner wants to erase : a block that has been erased in the current resetting interval 12,

28 A Simple Static-Wear Leveler An unevenness level (e cnt / f cnt ) >= T à Triggering of the SW Leveler Resetting of BET when all flags are set. T: A threshold, T=1000 in this example e cnt : the total number of block erases since the BET is erased e cnt =1998 =1999 =2000 =2004 =2998 =2999 =3000 =3004 =3999 =4000 =0 f cnt =2 =3 =4 = f cnt : The number of 1 s in the BET : An index that SW Leveler triggers the Cleaner to do garbage collection : An index in the selection of a block set : An index to a block that the Cleaner wants to erase : A block that has been erased in the current resetting interval k=2 The Cleaner is triggered to Reset to a randomly 1. Copy valid data selected block set free area, 4000 After a / 4 period = 1000>= Erase block in the selected of of time, selected block the (ethe block set cnt total / total f set, cnt erase >=1000) erase and , 3. count but Inform all / reaches 2 3 the flags = Allocator 1000 in (flag) BET >= to update 1000 are 1à the (Eaddress cnt reset / f cnt mapping >= BET T) between LBA and PBA 12,

29 Main-Memory Requirements 512MB 1GB 2GB 4GB 8GB k=0 256B 512B 1024B 2048B 4096B k=1 128B 256B 512B 1024B 2048B k=2 64B 128B 256B 512B 1024B k=3 32B 64B 128B 256B 512B MLC x2 (1 page = 2 KB, 1 block=128 pages) 12,

30 Worsening Reliability - Narrow Threshold Voltage Window MLC/TLC/QLC technology must squeeze the available window of threshold voltage for each logical state Higher Bit Error Rate Lower Endurance SLC MLC Source: 12,

31 Reliability Issues The low-cost MLC flash Has lower erase cycles Has higher and higher bit error rate. Methods to improve reliability Error correction coding (ECC) à passively Non-erasure code such as BCH and RS Wear leveling à proactively Distribute block erases as even as possible Note: Erasure code is used in communication Non-erasure code is used in storage systems 12,

32 Error Correction Coding Error correction code of a page is stored in its spare area. The time on error correction might be affected by the location of error bits or the number of error bits. The space of the ECC hardware is increased as the number of supported error bits increases. Trend of MLC flash The page size is getting larger The bit error rate is getting higher Fast erasing bits The fast worn-out flash cells V t 12,

33 Data Retention Problem The guaranteed data retention: 10 years As the cell size is getting smaller, the number of electrons in the floating gate of a flash cell is getting smaller. For example: A programmed cell can store 10,000 electrons. A lost of only 10% in this number can lead to a wrong read. à A loss of less than 2 electrons per week can be tolerated. 12,

34 Flash Self-Healing Potential Healing Command [1][2] For high BER (block error ratio) blocks For worn-out blocks Healthy Block Healing Worn-out Block 1. Lue, Hang-Ting. "Method for operating a semiconductor structure." U.S. Patent Application 13/710, Helm, Mark, William Kueber, and Andrei Mihnea. "Non-volatile memory cell healing." U.S. Patent No. 7,701, Apr Q. W. et al. Exploiting heat-accelerated flash memory wear-out recovery to enable self-healing SSDs. HotStorage H.-T. L. et al. Radically extending the cycling endurance of flash memory (to > 100m cycles) by using built-in thermal annealing to self-heal the stress-induced damage. IEDM M. V. et al. refresh SSDs: Enabling high endurance, low cost flash in datacenters. Technical report, FMS S. Lee, T. Kim, K. Kim, and J. Kim. Lifetime management of flash-based SSDs using recovery-aware dynamic throttling. USENIX 12 12,

35 Possible Implementation of Heal Leveler (Open Research Area) Process Process Process Applications Process Process Process Applications Virtual File System (VFS) File System (FAT, EXT2, NTFS, ) File Systems Virtual File System (VFS) File System (FAT, EXT2, NTFS, ) File Systems Flash Translation Layer Address Translator Cleaner Flash Memory Storage System Flash Translation Layer Address Translator Cleaner Heal-Leveler Flash Memory Storage System Heal-Leveler Memory Technology Device (MTD) Memory Technology Device (MTD) Physical Devices (Flash Memory) Physical Devices (Flash Memory) Layer Implementation Module Implementation 12,