Memory Management Disclaimer: some slides are adopted from book authors slides with permission 1
Demand paging Concepts to Learn 2
Abstraction Virtual Memory (VM) 4GB linear address space for each process Reality 1GB of actual physical memory shared with 20 other processes Does each process use the (1) entire virtual memory (2) all the time? 3
Demand Paging Idea: instead of keeping the entire memory pages in memory all the time, keep only part of them on a on-demand basis 4
Page Table Entry (PTE) PTE format (architecture specific) 1 1 1 2 20 bits V M R P Page Frame No Valid bit (V): whether the page is in memory Modify bit (M): whether the page is modified Reference bit (R): whether the page is accessed Protection bits(p): readable, writable, executable 5
Partial Memory Mapping Not all pages are in memory (i.e., valid=1) 6
Page Fault When a virtual address can not be translated to a physical address, MMU generates a trap to the OS Page fault handling procedure Step 1: allocate a free page frame Step 2: bring the stored page on disk (if necessary) Step 3: update the PTE (mapping and valid bit) Step 4: restart the instruction 7
Page Fault Handling 8
Demand Paging 9
Starting Up a Process Stack Unmapped pages Heap Data Code 10
Starting Up a Process Stack Heap Data Access next instruction Code 11
Starting Up a Process Stack Heap Data Page fault Code 12
Starting Up a Process Stack OS 1) allocates free page frame 2) loads the missed page from the disk (exec file) 3) updates the page table entry Heap Data Code 13
Starting Up a Process Stack Over time, more pages are mapped as needed Heap Data Code 14
Anonymous Page An executable file contains code (binary) So we can read from the executable file What about heap? No backing storage (unless it is swapped out later) Simply map a new free page (anonymous page) into the address space 15
Program Binary Sharing Bash #1 Bash #2 Physical memory Bash text Multiple instances of the same program E.g., 10 bash shells 16
Recap: Demand Paging Idea: instead of keeping the entire memory pages in memory all the time, keep only part of them on a on-demand basis 17
Recap: Page Fault Handling 18
Concepts to Learn Page replacement/swapping Thrashing 19
Memory Size Limit? Demand paging illusion of infinite memory 4GB 4GB 4GB Process A Process B Process C TLB MMU Page Table 1GB Physical Memory 500GB Disk 20
Illusion of Infinite Memory Demanding paging Allows more memory to be allocated than the size of physical memory Uses memory as cache of disk What to do when memory is full? On a page fault, there s no free page frame Someone (page) must go (be evicted) 21
On a page fault Recap: Page Fault Step 1: allocate a free page frame Step 2: bring the stored page on disk (if necessary) Step 3: update the PTE (mapping and valid bit) Step 4: restart the instruction 22
Page Replacement Procedure On a page fault Step 1: allocate a free page frame If there s a free frame, use it If there s no free frame, choose a victim frame and evict it to disk (if necessary) swap-out Step 2: bring the stored page on disk (if necessary) Step 3: update the PTE (mapping and valid bit) Step 4: restart the instruction 23
Page Replacement Procedure 24
Page Replacement Policy Which page (a.k.a. victim page) to go? What if the evicted page is needed soon? A page fault occurs, and the page will be re-loaded Important decision for performance reason The cost of choosing wrong page is very high: disk accesses 25
Page Replacement Policies FIFO (First In, First Out) Evict the oldest page first. Pros: fair Cons: can throw out frequently used pages Optimal Evict the page that will not be used for the longest period Pros: optimal Cons: you need to know the future 26
Random Page Replacement Policies Randomly choose a page Pros: simple. TLB commonly uses this method Cons: unpredictable LRU (Least Recently Used) Look at the past history, choose the one that has not been used for the longest period Pros: good performance Cons: complex, requires h/w support 27
LRU Example 28
LRU Example 29
Ideal solutions Timestamp List Implementing LRU Record access time of each page, and pick the page with the oldest timestamp Keep a list of pages ordered by the time of reference Head: recently used page, tail: least recently used page Problems: very expensive (time & space & cost) to implement 30
Page Table Entry (PTE) PTE format (architecture specific) 1 1 1 2 20 bits V M R P Page Frame No Valid bit (V): whether the page is in memory Modify bit (M): whether the page is modified Reference bit (R): whether the page is accessed Protection bits(p): readable, writable, executable 31
Implementing LRU: Approximation Second chance algorithm (or clock algorithm) Replace an old page, not the oldest page Use reference bit set by the MMU Algorithm details Arrange physical page frames in circle with a pointer On each page fault Step 1: advance the pointer by one Step 2: check the reference bit of the page: 1 Used recently. Clear the bit and go to Step 1 0 Not used recently. Selected victim. End. 32
Second Chance Algorithm 33
Recap: Page Replacement On a page fault Step 1: allocate a free page frame If there s a free frame, use it If there s no free frame, choose a victim frame and evict it to disk (if necessary) swap-out Step 2: bring the stored page on disk (if necessary) Step 3: update the PTE (mapping and valid bit) Step 4: restart the instruction 34
Recap: Page Replacement Policies FIFO Evict the oldest page first. Pros: fair; Cons: can throw out frequently used pages Optimal Evict the page that will not be used for the longest period Pros: optimal; Cons: you need to know the future Random LRU Randomly choose a page. Pros: simple. TLB commonly uses this method; Cons: unpredictable Look at the past history, choose the one that has not been used for the longest period. Pros: good performance; Cons: complex, requires h/w support 35
Recap: Page Table Entry (PTE) PTE format (architecture specific) 1 1 1 2 20 bits V M R P Page Frame No Valid bit (V): whether the page is in memory Modify bit (M): whether the page is modified Reference bit (R): whether the page is accessed Protection bits(p): readable, writable, executable 36
Recap: Clock Algorithm Key idea Replace an old page, not the oldest page Algorithm Step 1: advance the pointer by one Step 2: check the reference bit of the page: 1 Used recently. Clear the bit and go to Step 1 0 Not used recently. Selected victim. End. 37
Implementing LRU: Approximation N chance algorithm OS keeps a counter per page On a page fault Step 1: advance the pointer by one Step 2: check the reference bit of the page: check the reference bit 1 reference=0; counter=0 0 counter++; if counter =N then found victim, otherwise repeat Step 1. Large N better approximation to LRU, but costly Small N more efficient but poor LRU approximation 38
Performance of Demand Paging Three major activities Service the interrupt hundreds of cpu cycles Read/write the page from/to disk lots of time Restart the process again just a small amount of time Page Fault Rate 0 p 1 if p = 0 no page faults if p = 1, every reference is a fault Effective Access Time (EAT) EAT = (1 p) x memory access + p (page fault overhead + swap page out + swap page in ) 39
Performance of Demand Paging Memory access time = 200 nanoseconds Average page-fault service time = 8 milliseconds EAT = (1 p) x 200 + p (8 milliseconds) = (1 p) x 200 + p x 8,000,000 = 200 + p x 7,999,800 If one access out of 1,000 causes a page fault, then EAT = 8.2 microseconds. This is a slowdown by a factor of 40!! If want performance degradation < 10 percent 220 > 200 + 7,999,800 x p 20 > 7,999,800 x p p <.0000025 < one page fault in every 400,000 memory accesses 40
Thrashing A process is busy swapping pages in and out Don t make much progress Happens when a process do not have enough pages in memory Very high page fault rate Low CPU utilization (why?) CPU utilization based admission control may bring more programs to increase the utilization more page faults 41
Thrashing 42
Memory-mapped I/O Concepts to Learn Copy-on-Write (COW) 43
Memory Mapped I/O Idea: map a file on disk onto the memory space 44
Memory Mapped I/O Benefits: you don t need to use read()/write() system calls, just directly access data in the file via memory instructions How it works? Just like demand paging of an executable file What about writes? Mark the modified (M) bit in the PTE Write back the modified pages back to the original file 45
Copy-on-Write (COW) Fork() creates a copy of a parent process Copy the entire pages on new page frames? If the parent uses 1GB memory, then a fork() call would take a while Then, suppose you immediately call exec(). Was it of any use to copy the 1GB of parent process s memory? 46
Copy-on-Write Better way: copy the page table of the parent Page table is much smaller (so copy is faster) Both parent and child point to the exactly same physical page frames parent child 47
Copy-on-Write What happens when the parent/child reads? What happens when the parent/child writes? Trouble!!! parent child 48
Page Table Entry (PTE) PTE format (architecture specific) 1 1 1 2 20 bits V M R P Page Frame No Valid bit (V): whether the page is in memory Modify bit (M): whether the page is modified Reference bit (R): whether the page is accessed Protection bits(p): readable, writable, executable 49
Copy-on-Write All pages are marked as read-only Page tbl Page tbl parent RO RO RO RO RO RO child 50
Copy-on-Write Up on a write, a page fault occurs and the OS copies the page on a new frame and maps to it with R/W protection setting parent Page tbl RO RO RW Page tbl RO RO RW RO child 51
Memory mapped I/O Recap Mapping files on disk to the process s virtual address space. Copy-on-write Copy page table, not actual data pages Copy pages on writes 52
Memory allocator Concepts to Learn 53
Kernel/User Virtual Memory 0xFFFFFFFF 0xC0000000 Kernel Kernel memory Kernel code, data Identical to all address spaces Fixed 1-1 mapping of physical memory 0x00000000 User User memory Process code, data, heap, stack,... Unique to each address space On-demand mapping (page fault) 54
User-level Memory Allocation When a process actually allocate a memory from the kernel? On a page fault Allocate a page (e.g., 4KB) What does malloc() do? Doesn t physically allocate pages Manage a process s heap Variable size objects in heap 55
Kernel-level Memory Allocation Page-level allocator (low-level) Page granularity (4K) Buddy allocator Other kernel-memory allocators Support fine-grained allocations Slab, kmalloc, vmalloc allocators 56
Kernel-Level Memory Allocators Kernel code kmalloc Arbitrary size objects vmalloc (large) non-physically contiguous memory SLAB allocator Multiple fixed-sized object caches Page allocator (buddy) Allocate power of two pages: 4K, 8K, 16K, 57
Buddy Allocator Linux s page-level allocator Allocate power of two number of pages: 1, 2, 4, 8, pages. Request rounded up to next highest power of 2 When smaller allocation needed than is available, current chunk split into two buddies of next-lower power of 2 Quickly expand/shrink across the lists 32KB 16KB 8KB 4KB 58
Buddy Allocator Example Assume 256KB chunk available, kernel requests 21KB 256 Free 128 Free 128 Free 64 Free 64 Free 128 Free 32 Free 32 Free 64 Free 128 Free 32 A 32 Free 64 Free 128 Free 59
Buddy Allocator Example Free A 32 A 32 Free 64 Free 128 Free 32 Free 32 Free 64 Free 128 Free 64 Free 64 Free 128 Free 128 Free 128 Free 256 Free 60
Virtual Memory Summary MMU and address translation Paging Demand paging Copy-on-write Page replacement 61
Quiz: Address Translation 8 bits 1 st level 8 bits 8 bits 2 nd level offset Virtual address format (24bits) 4 bits 3 Frame # Unused 1 V Page table entry (8bit) Vaddr: 0x0703FE Paddr: 0x3FE Vaddr: 0x072370 Paddr:??? Vaddr: 0x082370 Paddr:??? Page-table base address = 0x100 Addr +0 +1 +2 +3 +4 +5 +6 +7 +8 +A +B +C +D +E +F 0x000 31 0x010 0x020 41.. 0x100 00 01 01 00 01.. 0x200 62
Quiz: Address Translation 8 bits 1 st level 8 bits 8 bits 2 nd level offset Virtual address format (24bits) 4 bits 3 Frame # Unused 1 V Page table entry (8bit) Vaddr: 0x0703FE Paddr: 0x3FE Vaddr: 0x072370 Paddr: 0x470 Vaddr: 0x082370 Paddr: invalid Page-table base address = 0x100 Addr +0 +1 +2 +3 +4 +5 +6 +7 +8 +A +B +C +D +E +F 0x000 31 0x010 0x020 41.. 0x100 00 01 01 00 01.. 0x200 63
Quiz. Complete the following with the FIFO replacement policy Reference E D H B D E D A E B E Page #1 E E E Page #2 D D Page #3 H Mark X for a fault X X X 64
Quiz. Complete the following with the FIFO replacement policy Reference E D H B D E D A E B E Page #1 E E E B B B B A A A A Page #2 D D D * E E E * B B Page #3 H H H H D D D D E Mark X for a fault X X X X X X X X X 65