Linux Memory Management

Transcription

1 Linux Memory Management An Overview Anshuman Khandual

2 Contents Introduction Platform Memory MMU Virtual Memory Physical Memory Analyze Memory Performance

3 Introduction Main Memory - A non persistent storage of memory which requires electricity Joint Electron Device Engineering Council (JEDEC) specifies standards for memory Double Data Rate (DDR) is the prevalent Dynamic Random Access Memory (DRAM) DDR standards DDR3 (Released in 2007) DDR4 (Released in 2014) DDR5 (Will be released around 2020) DRAM chips are placed on Dual In Line Memory Module (DIMM) cards Memory DIMM Types Non ECC DIMM (Laptop, Desktop etc) ECC (Workstations, Server) Industry standard DIMM (Compatible with most systems, cheaper) Proprietary DIMM (Buffered, faster, more RAS features, expensive)

4 Why Memory is Required Working set is fundamental to Jon Von Neuman computer architecture Similar to the concept of stack in push down automation There is a need to have working memory to solve a problem In matrix multiplication, the matrices need to be stored some where In image transformation, the image (it s pixels) needs to be stored some where Main memory (RAM) provides the infrastructure of a working set of problem data Input Device Control Unit Arithmetic and Logic Unit Output Device CPU Memory

5 Platform Memory SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT SMT DIMM DIMM DIMM DIMM L1 CACHE L2 CACHE L1 CACHE L2 CACHE L1 CACHE L2 CACHE DIMM DIMM DIMM L3 CACHE DIMM Buffer MC System BUS MC Buffer HW/FW Enabled Memory RAS CPU MMU MEMORY CONTROLLER MEMORY BUFFER MEMORY DIMM Linux Managed Non Standard Buffered DIMM IS DIMM

6 MMU Address Translation Kernel User Space On POWER load and store instructions are the only way to generate memory access Load & Store instructions Real Address CPU MEMORY CONTROLLER MEMORY BUFFER MMU MEMORY DIMM Virtual Address Physical Address (Transparent to software) PGD PUD PMD PTE Process Page Table

7 Virtual Memory Area Memory Management System Calls mprotect / madvise / mremap / mbind Changes existing VMA mmap / brk / sbrk/ munmap Creates, expands, contracts, destroys VMA task_struct mm_struct Virtual Area VMA VMA VMA Total size of the address space 512 TB

8 Linux Address Translation task_struct mm_struct Page Table Virtual Area Process s faulted physical accounting Need to be attached to CPU task will run on Contains MMU configuration for the VA space Process s view of virtual address space Obtained with system calls Task controls this directly Committed by the kernel PGD PUD PMD PTE VMA VMA VMA This relationship is not permanent Task keeps moving to different CPUs MMU CPU Only text VMA executes here But it can access (load / store) any VMA Task s scheduled context! TLB Entries

9 TLB is not tracked in SW Linux Address Translation Physical Page Physical Page Physical Page Buddy Allocator Memblock Memory RAM Virtual. Physical. Kernel physical memory representation PGD PUD PMD PTE VMA VMA VMA Page faults populate table MMU CPU TLB Entries When task moves, flush TLB entries!

10 Process Page Table System Calls Modifying VMAs System Calls Modifying Mappings Virtual HW Error Virtual Physical Virtual Not allocated Virtual Not allocated... Virtual. Physical Without Valid Physical Page MMU does not have translation Virtual Swapped out Virtual Physical Virtual Migrating Access Causes Page Fault Kernel Handle Pages Fault Allocates Page, Inserts in Page Table Reclaims, Compaction, Migration, HW Error, Hotplug

11 Physical Memory 0x Physical Address Space node0 0x D x D node1 0x F x F node2 0x x x C x node3 0x x x x xFFFFFFFFFFFFFFFF

12 Physical Memory Platform Firmware Scans HW memory DIMM configuration addr, size, node addr, size, node addr, size, node addr, size, node Hotplug In / Out ACPI / Device Tree Memblock Interface Buddy Allocator struct s free_s() alloc_s() Memory Used Kernel, Driver and User Space

13 Buddy System NODE 0 NODE 1 NODE 2 NODE.. NODE N pglist_data pglist_data pglist_data... pglist_data ZONE_MOVABLE ZONE_NORMAL ZONE_DMA32 ZONE_DMA free_area[0] free_list free_area[n] free_list MIGRATE_UNMOVEABLE... MIGRATE_MOVEABLE... MIGRATE_RECLAIMABLE... MIGRATE_PCTYPES... MIGRATE_HIGHATOMIC... MIGRATE_CMA...

14 Physical Memory Allocator Kernel Memory Requirements kmalloc() / kzalloc() Smaller size, cache hot etc cache_object cache_object cache_object User Space Page Faults SLAB Allocator Page Allocator alloc_s() 2 ^ order * PAGE_SIZE granularity Buddy System

15 Page Allocator Kernel Direct Allocations VMA memory_policy GFP_MASK ORDER NODEMASK PREFERRED_NID alloc_s_nodemask() Async Compaction (kcompactd) Fast Path Allocator Async Reclaim (kswapd) Slow Path Allocator Direct Reclaim OOM Allocator

16 Analyze Memory Performance char *ptr; ptr = mmmap(, PAGE_SIZE * NR_PAGES, PROT_READ PROT_READ, MAP_PRIVATE MAP_ANON, -1, 0); memset(ptr, 0, size); munmap(ptr, size); Allocate virtual memory buffer Sequential access Release virtual memory buffer

17 Analyze Memory Performance char *ptr; ptr = mmmap(, PAGE_SIZE * NR_PAGES, PROT_READ PROT_READ, MAP_PRIVATE MAP_ANON, -1, 0); memset(ptr, 0, size); SW cost VMA Virtual. NR_PAGES SW cost NR_PAGES First Byte Access Address fetch retried! Page Fault HW cost RAM Access Cache Access PAGE_SIZE Byte Accesses Address Translation Memory Store RAM Access.. Cache Access.. TLB Hit TLB Miss TLB Load OR RAM Access Cache Access

18 Analyze Memory Performance char *ptr; ptr = mmmap(, PAGE_SIZE * NR_PAGES, PROT_READ PROT_READ, MAP_PRIVATE MAP_ANON, -1, 0); memset(ptr, 0, size); munmap(ptr, size); VMA Virtual. NR_PAGES Tear down mapping for the memory buffer Flush TLB (This is tricky!!!) Local TLB Flush OR Global TLB Flush Free relevant potions of the table Free the faulted in s if any!

19 Thank You