Progress & Challenges for Virtual Memory Management Kathryn S. McKinley

Transcription

1 Progress & Challenges for Virtual Memory Management Kathryn S. McKinley Microsoft Research 1!

2

3 The Storage Gap Petabytes 50,000,000! 40,000,000! 30,000,000! 20,000,000! 10,000,000! 0! Digital Universe Gap Installed Capacity Capacity Shipped Source: IDC

4 Memory capacity for $10,000 Memory size GB TB 64! , , Addressable servers 2 31 Addressable M B Year *Inflation-adjusted 2011 USD, from: jcmit.com!

5 Memory technologies FLASH DRAM Electrode PCM Chalcogenide Resistor (heat) Electrode DNA

6 Heterogeneous Memory Systems Mix of DRAM, Flash, NVM Memory coupled with FPGA & special purpose ASIC

7 Server software Mobile + Web + Cloud traditional batch throughput + interactive latency

8 Scaling up & out Basic Server Architecture aggregator Server workloads scaled to memory sizes & CPUs 8!

9 Applications must fit in memory 1000" Conviva customer search logs Queries """"Queries"per"second" seconds 100" 10" 1" 1" 2" 4" 8" 16" 32" 64" 128" Raw uncompressed data size Raw"input"size" Elas7csearch" MongoDB" Cassandra" 60 GB Amazon EC2 server, one core Corollary no swapping Source: R. Agarwal [NSDI 15, NSDI 16]

10 Road map Fitting in memory Better big memory hardware Implications for operating system memory management

11 Data in memory is compressible It s now cheaper to compress a page and write it out than write an uncompressed page Apple & VMware do it Physical Memory compress swapping! copy

12 Can we compute on compressed data?

13 Java limit study ψ ξ! run program Heap dumps Compression rate 60! 50! 40! 30! 20! 10! 0! Sartor et al. [ISMM 08, PLDI 10] t! ξ ψ! Models savings limit Value set indirection Value set caching Objects Deep equal sharing Arrays Zeros Hybrid

14 Reality of compressed arrays up to 49% compression, but 6% on average Fast indirect slow access Array Spine Header Inlined First N Arraylet Pointers Remaining Elements Arraylet Space Arraylet Arraylet Arraylet... Zero Arraylet

15 Are databases compressible? Data#Scans# Indexes# 0,#10,#14,#16,#19,#26,#29# 1,#4,#5,#8,#20,#22,#24# 2,#15,#17,#27# Search( ) 3,#6,#7,#9,#12,#13,#18,#23#..# 11,#21# Low$storage$ Low$Throughput$ High$storage$ High$Throughput$ Source: R. Agarwal [NSDI 15, NSDI 16]

16 Databases are compressible! Succinct insight Queries'execute'directly'on'compressed'data!' compression index serves as search index Burrows-Wheeler Transform (bzip) suffix arrays New data structures & query algorithms Source: R. Agarwal [NSDI 15, NSDI 16]

17 Source: R. Agarwal [NSDI 15, NSDI 16] Advantages Limitations Compression index is the data index! No scans No decompression Queries search, range, random access, regular expressions More CPU operations for data access No scans No in-place updates Preprocessing time

18 Order of magnitude more data Total footprint (GB) RAM Cassandra MongoDB ElasCcsearch Succinct Raw uncompressed data size Source: R. Agarwal [NSDI 15, NSDI 16]

19 Queries per seconds Throughput DB- X (Scans) ElasCcsearch Succinct Raw uncompressed data size

20 Fitting in memory is necessary, but not sufficient

21 Road map Fitting in memory Better big memory hardware Implications for operating system memory management

22 Scaling up & out Basic Server Architecture aggregator Server workloads scaled to memory sizes & CPUs 22!

23 Hardware organization Virtual Memory Page Table Physical Memory Process Process Page Table

24 Hardware organization Process Virtual Memory Page Table miss TLB fill Physical Memory hits Process TLB Translation Look aside Buffer on CPU critical path

25 Virtualization exacerbates TLB limitations Process Virtual Memory Host Page Table Guest Page Table Physical Memory Process TLB misses incur 4 + 4*5 memory accesses! Shadow page tables [TACO 13]

26 Memory capacity for $10,000 Memory size GB TB 64! , , Addressable servers 2 31 Addressable M B Year TLB L1 Entries *Inflation-adjusted 2011 USD, from: jcmit.com!

27 Bigger pages extend TLB reach Physical Memory 1 MB 1 GB aligned Explicit Programmer burden Transparent Internal fragmentation up to.5 page size depending on policy One size does not fit all!

28 Paged memory Virtual Memory Physical Memory 28!

29 Key observation: ranges of contiguity Virtual Memory Code Heap Stack Shared Lib. How much range contiguity is there?

30 Ranges vs Pages Total Application Address Space (%) ranges! pages! astar mcf cactusadm canneal graph500 memcached tigr #number of entries 30!

31 Idea: Redundant memory mappings Virtual Memory Code Heap Stack Shared Lib. Physical Memory 31!

32 Range translations: Base, Limit, Offset Virtual Memory BASE LIMIT Range Translation BASE OFFSET Physical Memory Range Translation: maps contiguous virtual pages to contiguous physical pages with uniform protection

33 OS memory management support Virtual Memory Demand Paging Physical Memory Does not facilitate physical range creation

34 Eager paging Virtual Memory Physical Memory Increases range sizes, but can waste memory

35 ISCA 15, IEEE MICRO Top Picks It works: eliminates VM overheads 45%! 35%! 25%! 15%! 5%! -5%! 0.00%! 0.25%! 0.40%! 0.00%! 0.14%! 0.06%! 0.26%! 4KB! CTLB! THP! DS! RMM! 4KB! CTLB! THP! DS! RMM! 4KB! CTLB! THP! DS! RMM! 4KB! CTLB! THP! DS! RMM! 4KB! CTLB! THP! DS! RMM! 1.06%! cactusadm! canneal! graph500! mcf! tigr!

36 Road map Fitting in memory Better big memory hardware Implications for operating system memory management

37 Variable page sizes and ranges require contiguity base pages do not

38 Source: codeproject.com OS Buddy allocation Design goals Power of 2 page sizes Fast allocation Fast coalescing

39 Memory management is a space tradeoff Lessons from garbage collection

40 GC Fundamentals Allocation Free List Identification Tracing (implicit) Reclamation Sweep-to-Free Compact Bump Allocation Reference Counting (explicit) 3 1 ` Evacuate 40

41 ! Canonical Garbage Collectors Mark-Sweep [McCarthy 1960]! Free-list + trace + sweep-to-free!!!! Mark-Compact [Styger 1967]! Bump allocation + trace + compact!!! Semi-Space [Cheney 1970]! Bump allocation + trace + evacuate!!!! Sweep-to-Free Compact ` Evacuate

42 Mark-Sweep Free List Allocation + Trace + Sweep-to-Free Mutator Poor locality Minimum Heap Space efficient Time! Space! Space! Time! Garbage Collection Simple fast collection Space! Time! Total Performance Space! SemiSpace MarkCompact MarkSweep

43 Mark-Compact Bump Allocation + Trace + Compact Time! Mutator Good locality Space! Space! Minimum Heap Space efficient Time! Garbage Collection Expensive multi-pass collection Time! Total Performance SemiSpace MarkCompact MarkSweep Space! Space! Actual data, taken from geomean of DaCapo, jvm98, and jbb2000 on 2.4GHz Core 2 Duo!

44 Semi-Space Bump Allocation + Trace + Evacuate Time! Mutator Good locality Space! Space! Minimum Heap Space inefficient Time! Garbage Collection Space inefficient Time! Total Performance SemiSpace MarkCompact MarkSweep Space! Space!

45 Performance Pathologies Mark-Sweep, Mark-Compact, Semi-Space Time! Mutator Mark-Sweep Poor locality Space! Space! Minimum Heap Semi-Space Space inefficient Time! Garbage Collection Mark-Compact expensive multi-pass Time! Total Performance SemiSpace MarkCompact MarkSweep Space! Space! Actual data, taken from geomean of DaCapo, jvm98, and jbb2000 on 2.4GHz Core 2 Duo!

46 Mark-Region with Sweep-To-Region Mark-Sweep Mark-Compact Semi-Space Mark-Region Bump + trace + sweep-to-region Reclamation Sweep-to-Free Compact Evacuate ` Sweep-to-Region

47 Naïve Mark-Region Contiguous allocation into regions Excellent locality Objects cannot span regions Simply mark objects and their region Free unmarked regions

48 Region Size? Large Regions More contiguous allocation! Fragmentation (false marking)! Small Regions Increased metadata! Constrained object sizes! Fragmentation (can t fill blocks)!

49 Hierarchy: lines and blocks N pages Free Recyclable lines Free approx 1 cache line Recyclable lines TLB locality, cache locality Objects span lines Less fragmentation Block > 4 X max object size! Lines marked with objects Fast common case

50 Allocation Policy (Recycling) Recycle partially marked blocks first ü ü ü Minimizes fragmentation Maximizes contiguity Shares free blocks among parallel thread allocators

51 Opportunistic Defragmentation Opportunistically evacuate fragmented blocks Lightweight, uses same allocation mechanism No cost in common case (specialized GC) Identify source and target blocks Evacuate objects in source blocks Allocate into target blocks Opportunistic: Leave in place if no space, or object pinned

52 Immix Mark-Region 128 byte lines, 32 KB blocks Opportunistic defragmentation triggered if there exist partially full blocks at memory exhaustion Overflow allocation of medium objects (> 128 bytes) Implicit marking

53 Immix Bump Allocation + Trace + Sweep-to-Region Time! Time! Locality Garbage Collection Simple Mutator Space! fast collection Space! Space! Total Performance MarkSweep Time! Minimum Heap Excellent Space efficient Space! Actual data, taken from geomean of DaCapo, jvm98, and jbb2000 on 2.4GHz Core 2 Duo! MarkCompact SemiSpace Immix

54 Lessons for OS Metadata Immix is more space & time efficient with 1.5% of metadata than all the zero metadata algorithms. Contiguity Applications allocate & access sequential virtual addresses. Immix exploits this property. Why not the OS?

55 Looking forward Opportunities for OS memory management

56 Mark Region (MR) for Multiple Page Sizes page sizes = region sizes page! huge page big page page allocation big page allocation big page promotion huge page promotion page = MR line big page = multiple pages = MR block (aligned) huge page = multiple big pages = multiple MR blocks, etc. Fragmentation management - better, block statistics always correct prevention allocate into occupied blocks w/ free pages defrag target & source blocks

57 Incremental adoption in buddy allocator buddy sizes = page sizes page! huge page big page page allocation big page allocation big page promotion huge page promotion Eliminate all buddy sizes potential page sizes Add metadata for blocks and pages Add allocation and defragmentation policies

58 What about range allocation? Virtual Memory Physical Memory

59 Mark Region for Ranges max range size = biggest block size big page max range size base range allocation bound partially filled block of pages Range allocation overflow allocation, harder due to variable sizes Fragmentation management prevention same, allocate into occupied blocks w/ free pages defrag same mechanisms with new incremental policies that consider multiple page sizes and maximum range size

60 Lots of challenges Heterogeneous memories memory management CPU constraints Queries'execute'directly'on'compressed'data!' DNA

61 Collaborators maybe YOU! Steve Blackburn Karin Strauss Vasileios Karakostas Jayneel Gandhi Mark D. Hill Michael M. Swift Osman S. Ünsal Mario Nemirovsky Furkan Ayar Adrián Cristal Jennifer Sartor Martin Hirzel Tiejun Gao

62 Lots of challenges Heterogeneous memories memory management CPU constraints Queries'execute'directly'on'compressed'data!' DNA