ECE 598-MS: Advanced Memory and Storage Systems Lecture 7: Unified Address Translation with FlashMap

Transcription

1 ECE 598-MS: Advanced Memory and Storage Systems Lecture 7: Unified Address Translation with Map Jian Huang

2 Use As Non-Volatile Memory DRAM (nanoseconds) Application Memory Component SSD (microseconds) Application Storage Component HDD (milliseconds) Designed for fast memory Designed for slow disk 2

3 Use As Non-Volatile Memory DRAM (nanoseconds) Application Memory Component SSD (microseconds) 40% HDD (milliseconds) Application Storage Component % Designed for fast memory Designed for slow disk 2

4 Use As Non-Volatile Memory DRAM (nanoseconds) Application Memory Component SSD (microseconds) HDD (milliseconds) Application Storage Component Designed for fast memory Designed for slow disk 2

5 Memory Mapped SSD Application Virtual Memory Filesystem SSD File 3

6 Memory Mapped SSD Application Virtual Memory Filesystem SSD mmap() munmap() msync() File 3

7 Memory Mapped SSD Application Virtual Memory Filesystem SSD mmap() munmap() msync() File YAY! MORE MEM!!! Extended Memory 3

8 Memory Mapped SSD Application Virtual Memory Filesystem SSD mmap() munmap() msync() File YAY! MORE MEM!!! + Extended Memory Minimal Code Modifications 3

9 Memory Mapped SSD Application Virtual Memory Filesystem SSD mmap() munmap() msync() File YAY! MORE MEM!!! + + Extended Memory Minimal Code Modifications Data Durability 3

10 No Free Lunch: Software Overhead Application Virtual Memory System File System Translation Layer 4

11 No Free Lunch: Software Overhead Application page fault Virtual Memory System Virtual Address Page Table & Memory Manager Physical Address/File Offset File System Translation Layer 4

12 No Free Lunch: Software Overhead Application page fault Virtual Memory System Virtual Address Page Table & Memory Manager Physical Address/File Offset File Offset File System File Index Logical Block Address Translation Layer 4

13 No Free Lunch: Software Overhead Application page fault Virtual Memory System Virtual Address Page Table & Memory Manager Physical Address/File Offset File Offset File System File Index Logical Block Address Logical Block Address Translation Layer FTL Physical Block Address 4

14 No Free Lunch: Software Overhead Application page fault Virtual Memory System File System Virtual Address Page Table & Memory Manager Physical Address/File Offset File Offset File Index Logical Block Address Logical Block Address 3 address translations + 2 boundary checks + 2 permission checks = Latency: microseconds Translation Layer FTL Physical Block Address + Metadata Overhead 4

15 Map: Unified Address Translation Application Virtual Memory System File System Translation Layer 5

16 Map: Unified Address Translation Application Unified Address Translation 1 Reduced 2 Reduced Storage, only 1 mapping table Latency only 1 address translation + 1 permission check + 1 boundary check 5

17 Application Virtual Memory System Map Challenges File System Translation Layer 6

18 Application Map Challenges Virtual Memory System Page table is process-specific & private File System File index is system-wide shared resource Translation Layer Mapping table is shared and should be persisted 6

19 Combining Page Table and File System Virtual Memory File File System Translation Layer 7

20 Combining Page Table and File System VM Region Virtual Memory File File System Translation Layer 7

21 Combining Page Table and File System Page Table VM Region Virtual Memory File File System Translation Layer 7

22 Combining Page Table and File System Page Table VM Region Virtual Memory Page table is process-specific & private File File System Translation Layer 7

23 Combining Page Table and File System Page Table VM Region Only for mapped file Virtual Memory File File System Translation Layer 7

24 Preserving File System Permissions Page Table VM Region Process B Page Table VM Region Shared Page Table Mapped File File System Translation Layer 8

25 Preserving File System Permissions Page Table VM Region Process B Page Table VM Region Shared Page Table Mapped File File System Translation Layer 8

26 Preserving File System Permissions Page Table VM Region READ_ONLY Process B Page Table VM Region READ_WRITE Shared Page Table Mapped File File System Translation Layer 8

27 Preserving File System Permissions Page Table VM Region READ_ONLY Permission Conflict!!! Process B Page Table VM Region READ_WRITE Shared Page Table Mapped File File System Translation Layer 8

28 Preserving File System Permissions Page Table VM Region READ_ONLY Process B Page Table VM Region READ_WRITE Only share the leaf-level page table pages Shared Page Table Mapped File File System Translation Layer 8

29 Preserving File System Permissions Page Table VM Region READ_ONLY Process B Page Table VM Region READ_WRITE Shared Page Table Mapped File File System Translation Layer 8

30 Preserving Memory Protections Private Process B Private Shared Leaf-level Page Table Shared Mapped File Private leaf-level page table File System Translation Layer 9

31 Preserving Memory Protections Process B Shared Leaf-level Page Table Private leaf-level page table Custom memory protection? Mapped File File System Translation Layer 9

32 Preserving Memory Protections Process B Private leaf-level page table Shared Leaf-level Page Table Mapped File File System Translation Layer 9

33 Combining FTL and Shared Page Table Process B Shared Leaf-level Page Table Mapped File File System Translation Layer 10

34 Combining FTL and Shared Page Table Process B Shared Leaf-level Page Table Mapped File File System Mapping Table Garbage Collection ECC Wear Leveling 10

35 Combining FTL and Shared Page Table Process B Overloaded PTE Shared Leaf-level Page Table Mapped File File System Garbage Collection ECC Wear Leveling 10

36 Put It All Together Read DRAM Shared Leaf-level Page Table Pages Mapped File 11

37 Put It All Together Read DRAM Shared Leaf-level Page Table Pages Mapped File 11

38 Put It All Together Read DRAM page fault Shared Leaf-level Page Table Pages Mapped File 11

39 Put It All Together Read DRAM Shared Leaf-level Page Table Pages update PTE Mapped File 11

40 Put It All Together Write DRAM Shared Leaf-level Page Table Pages Mapped File 11

41 Put It All Together Write DRAM DRAM hit Shared Leaf-level Page Table Pages Mapped File 11

42 Put It All Together Write DRAM Shared Leaf-level Page Table Pages DRAM miss Mapped File 11

43 Put It All Together Write DRAM Shared Leaf-level Page Table Pages update PTE Mapped File 11

44 Put It All Together Write DRAM Shared Leaf-level Page Table Pages Mapped File 11

45 Put It All Together DRAM Shared Leaf-level Page Table Pages GC Mapped File 11

46 Metadata Size (GB) Metadata Size for 1 TB SSD Baseline (mmap + EXT4 + FTL with page level mapping) FTL + FS (mmap + combined FTL & filesystem) Map 50% Reduction 12

47 Latency (us) Latency Reduction with Map 50 Baseline FTL+FS Map Device latency (microseconds) 13

48 Latency (us) Latency Reduction with Map 50 Baseline FTL+FS Map Device latency (microseconds) The combination of page table and file system contributes mostly to the latency reduction 13

49 Throughput (K TPS) Benefits from Reduced Latency (Database) Baseline FTL+FS Map Device Latency (us) As becomes faster, Map provides more performance benefits for latency-critical apps 14

50 Performance 1.8x Map Impact Latency 53% Metadata Size 50% 15

51 Impact on industry How to use scalable SSD in data center Map Impact Unifying emerging memory and storage e.g., storage-class memory 15