Understanding the Design of Virtual Memory. Zhiqiang Lin

CS 6V8-05: System Security and Malicious Code Analysis Understanding the Design of Virtual Memory Zhiqiang Lin Department of Computer Science University of Texas at Dallas February 27 th, 202

Outline Basic Concepts 2 Why Virtual Memory Caching Memory management Memory protection 3 Address translation 4 Summary

Where we are today Splitting the Ker nel The System Call Interface Process management Memory management Filesystems Device control Networking Kernel subsystems Concurrency, multitasking Virtual memory Files and dirs: the VFS Ttys & device access Connectivity Features implemented Archdependent Code Memory manager File system types Block devices Character devices Network subsystem IF drivers Software support Hardware CPU Memory Disks & CDs Consoles, etc. Network interfaces features implemented as modules Figur e -. A split view of the kernel This diagram is from Linux Device Driver book (page 5) throughout the whole system. In addition, Linux supports multiple filesystem

Outline Basic Concepts 2 Why Virtual Memory Caching Memory management Memory protection 3 Address translation 4 Summary

Von Neumann Architecture Data Software code is loaded memory Program data is also in the memory

A System Using Physical Addressing A System Using Physical Addressing Carnegie Mell CPU Physical address (PA) 4 Main memory 0: : 2: 3: 4: 5: 6: 7: 8:... M-: Data word Used Used in simple in simple systems systems like like embedded embedded microcontrollers microcontrollers in in devices devices like like cars, cars, elevators, elevators, and and digital digital picture picture frames frames

Carnegie Mellon A System Using Virtual Addressing A System Using Virtual Addressing CPU Chip CPU Virtual address (VA) 400 MMU Physical address (PA) 4 Main memory 0: : 2: 3: 4: 5: 6: 7: 8:... M-: Data word Used in all modern servers, desktops, and laptops Used One of inthe all great modern ideas servers, in computer desktops, science and laptops One of the greatest ideas in computer science 4

Virtual Memory In computing, virtual memory is a memory management technique developed for multitasking kernels. This technique virtualizes a computer architecture s various forms of computer data storage (such as random-access memory and disk storage), allowing a program to be designed as though there is only one kind of memory, "virtual" memory, which behaves like directly addressable read/write memory (RAM). http://en.wikipedia.org/wiki/virtual_memory

Who invented this Peter J. Denning, The Working Set Model for Program Behavior, Proceedings of the First ACM Symposium on Operating Systems Principles, October 967, Gatlinburg, TN, USA. This paper introduced the working set model, which has became a key concept in understanding of locality of memory references and for implementing virtual memory. Most paging algorithms can trace their roots back to this work. This paper introduced the working set model, which has became a key concept in understanding of locality of memory references and for implementing virtual memory. Most paging algorithms can trace their roots back to this work. http://cs.gmu.edu/cne/pjd/pubs/bvm.pdf Andre Bensoussan, Charlie T. Clingen, Robert C. Daley, The Multics Virtual Memory: Concepts and Design, Communications of the ACM 5(5):308-38, May 972.

Who invented this Dennis M. Ritchie and Ken Thompson, The UNIX Time-Sharing System, Proceedings of the Fourth ACM Symposium on Operating Systems Principles, October 973, Yorktown Heights, NY, USA. Edsger W. Dijkstra, The Structure of the THE Multiprogramming System, Proceedings of the First ACM Symposium on Operating Systems Principles, October 967, Gatlinburg, TN, USA. The first paper to suggest that an operating system be built in a structured way. That structure was a series of layers, each a virtual machine that introduced abstractions built using the functionality of lower layer. The paper stimulated a great deal of subsequent work in building operating systems as structured systems.

Virtual Memory of a Linux Process Virtual Memory of a Linux Process Carnegie Mellon Different for each process Identical for each process %esp Process-specific data structs (ptables, task and mm structs, kernel stack) Physical memory Kernel code and data User stack Kernel virtual memory brk 0x08048000 (32) 0x00400000 (64) 0 Memory mapped region for shared libraries Runtime heap (malloc) Uninitialized data (.bss) Initialized data (.data) Program text (.text) Process virtual memory 37

How hardware support (MMU) 0-bits 0-bits 2-bits Offset in PGD Offset in PDE Offset in Data Page Page Directory PDE Page Table PTE pte Data Page CR3 2-bits.. X 0 0 X X X 0 0 X cache write present reserved page size cache through writable Page Directory Entry (PDE) disabled accessed 2-bits.. Page Table Entry (PTE) 0 global page U/S R/W present

Basic Concepts space. To understand Why this Virtual in more Memory details, let us first take a look Address at how translation the paging in x86 works. Summary 2. Paging in x86 Processor A more concrete view The x86 processor divides the physical address space (or physical memory) in 4 KB pages. Thus to address 4GB of memory, we will need Mega (024x024) 4KB pages. The processor uses a two level structure to refer to these Mega pages. You can think of it as a two dimensional matrix of 024x024 elements. The first dimension is known as Page Directory and second dimension is known as Page Table. Thus we can create Page directory with 024 entries, each of which points to a Page Table. This will allow us to have 024 page tables. Each page table in turn can have 024 entries, each of which points to a 4 KB page. Graphically it looks something like: Page Directory Index = 0, Address = X Index =, Address = Y Index = 023, Address = Z Page Tables Index = 0, Address = X Index =, Address = X2 Index = 2, Address = X3 Index = 023, Address = Xn Index = 0, Address = Y Index =, Address = Y2 Index = 2, Address = Y3 Index = 023, Address = Yn Index = 0, Address = Z Index =, Address = Z2 Index = 2, Address = Z3 Index = 023, Address = Zn Physical Address Address X 4 KB Page Address Yn 4 KB Page Address X3 4 KB Page Unused Address Xn 4 KB Page Unused Address Y 4KB Page Address X2 4KB Page Address Z2 4KB Page Address Y2 4KB Page Address Z3 4KB Page Unused Unused Address Zn 4KB Page Address Y3 4KB Page Address Z 4KB Page Unused Unused so on Page of 5

Other Relevant Concepts Memory address Address space Virtual address space Page (computing) Page table Paging Working set Memory management unit Cache algorithms Page replacement algorithm Segmentation (memory) Memory management Memory allocation Protected mode, an x86 mode that allows for virtual memory....

Outline Basic Concepts 2 Why Virtual Memory Caching Memory management Memory protection 3 Address translation 4 Summary

Address Spaces Linear address space: Ordered set of contiguous non-negative integer addresses: {0,, 2, 3... } Virtual address space: Set of N = 2 n virtual addresses: {0,, 2, 3,..., N-} Physical address space: Set of M = 2 m physical addresses: {0,, 2, 3,..., M-} Clean distinction between data (bytes) and their attributes (addresses) Each object can now have multiple addresses Every byte in main memory: one physical address, one (or more) virtual addresses

Why Virtual Memory (VM)? Uses main memory efficiently Use DRAM as a cache for the parts of a virtual address space Simplifies memory management Each process gets the same uniform linear address space Isolates address spaces One process can t interfere with another s memory User program cannot access privileged kernel information

Outline Basic Concepts 2 Why Virtual Memory Caching Memory management Memory protection 3 Address translation 4 Summary

VM as a Tool for Caching VM as a Tool for Caching Virtual memory is an array of N contiguous bytes stored Virtual memory is an array of contiguous bytes stored on on disk. disk. The The contents of of the array on disk are cached in in physical memory (DRAM cache) These These cache cache blocks blocks are called are called pages pages (size is (size P = 2 is p bytes) P = 2 p bytes) VP 0 VP VP 2 n-p - Virtual memory 0 Unallocated Cached Uncached Unallocated Cached Uncached Cached Uncached N- Virtual pages (VPs) stored on disk Physical memory 0 M- Empty Empty Empty PP 0 PP PP 2 m-p - Physical pages (PPs) cached in DRAM

DRAM Cache Organization DRAM cache organization driven by the enormous miss penalty DRAM is about 0x slower than SRAM Disk is about 0,000x slower than DRAM Consequences Large page (block) size: typically 4-8 KB, sometimes 4 MB Fully associative Any VP can be placed in any PP Requires a large mapping function - different from CPU caches Highly sophisticated, expensive replacement algorithms Too complicated and open-ended to be implemented in hardware Write-back rather than write-through

Page Tables Page Tables Carnegie Mellon A page table is an array of page table entries (PTEs) that maps virtual pages to physical pages. A page table is an array of page table entries (PTEs) that Per-process kernel data structure in DRAM maps virtual pages to physical pages. Per-process kernel data structure in DRAM Physical page number or Valid disk address PTE 0 0 null 0 0 null 0 PTE 7 Memory resident page table (DRAM) Physical memory (DRAM) VP VP 2 VP 7 VP 4 Virtual memory (disk) VP VP 2 VP 3 VP 4 VP 6 VP 7 PP 0 PP 3 0

Page Hit Page Hit Carnegie Mellon Page hit: reference to VM word that is in physical memory (DRAM Page cache hit: reference hit) to VM word that is in physical memory (DRAM cache hit) Virtual address Physical page number or Valid disk address PTE 0 0 null 0 0 null 0 PTE 7 Memory resident page table (DRAM) Physical memory (DRAM) VP VP 2 VP 7 VP 4 Virtual memory (disk) VP VP 2 VP 3 VP 4 VP 6 VP 7 PP 0 PP 3

Page Fault Page Fault Carnegie Mellon Page fault: reference to VM word that is not in physical memory Page fault: (DRAM reference cache to miss) VM word that is not in physical memory (DRAM cache miss) Virtual address Physical page number or Valid disk address PTE 0 0 null 0 0 null 0 PTE 7 Memory resident page table (DRAM) Physical memory (DRAM) VP VP 2 VP 7 VP 4 Virtual memory (disk) VP VP 2 VP 3 VP 4 VP 6 VP 7 PP 0 PP 3 2

Handling Page Fault Handling Page Fault Page miss causes page fault (an exception) Page miss causes page fault (an exception) Carnegie Mellon Virtual address Physical page number or Valid disk address PTE 0 0 null 0 0 null 0 PTE 7 Memory resident page table (DRAM) Physical memory (DRAM) VP VP 2 VP 7 VP 4 Virtual memory (disk) VP VP 2 VP 3 VP 4 VP 6 VP 7 PP 0 PP 3 3

Handling Page Fault Handling Page Fault Page miss causes page fault (an exception) Carnegie Mellon Page fault handler selects a victim to be evicted (here VP 4) Page miss causes page fault (an exception) Page fault handler selects a victim to be evicted (here VP 4) Virtual address Physical page number or Valid disk address PTE 0 0 null 0 0 null 0 PTE 7 Memory resident page table (DRAM) Physical memory (DRAM) VP VP 2 VP 7 VP 4 Virtual memory (disk) VP VP 2 VP 3 VP 4 VP 6 VP 7 PP 0 PP 3 4

5 Handling Page Fault Page miss causes page fault (an exception) Handling Page Fault Page fault handler selects a victim to be evicted (here VP 4) Page miss causes page fault (an exception) Offending instruction is restarted: page hit! Page fault handler selects a victim to be evicted (here VP 4) Carnegie Mellon Virtual address Physical page number or Valid disk address PTE 0 0 null 0 0 null 0 PTE 7 Memory resident page table (DRAM) Physical memory (DRAM) VP VP 2 VP 7 VP 3 Virtual memory (disk) VP VP 2 VP 3 VP 4 VP 6 VP 7 PP 0 PP 3

Locality Virtual memory works because of locality At any point in time, programs tend to access a set of active virtual pages called the working set Programs with better temporal locality will have smaller working sets If (working set size < main memory size) Good performance for one process after compulsory misses If ( SUM(working set sizes) > main memory size ) Thrashing: Performance meltdown where pages are swapped (copied) in and out continuously

Outline Basic Concepts 2 Why Virtual Memory Caching Memory management Memory protection 3 Address translation 4 Summary

Basic Concepts Why Virtual Memory Address translation Carnegie Summary Mellon VM as a Tool for Memory Management VM as a Tool for Memory Management Key Key idea: idea: each each process process has has its own its own virtual virtual address address space space It can view memory as a simple linear array It can view memory as a simple linear array Mapping function scatters addresses through physical Mapping memory function scatters addresses through physical memory Well chosen Well chosen mappings mappings simplify simplify memory memory allocation allocation and management and management Virtual Address Space for Process : 0 VP VP 2... Address translation 0 PP 2 Physical Address Space (DRAM) Virtual Address Space for Process 2: N- 0 VP VP 2... PP 6 PP 8... (e.g., read-only library code) N- M- 9

Basic Concepts Why Virtual Memory Address translation Carnegie Summary Mellon VM as a Tool for Memory Management VM as a Tool for Memory Management Memory allocation Key idea: Each each virtual process page has can its be own mapped virtual to any address physical space page It can Aview virtual memory pageas can a simple be stored linear inarray different physical pages at Mapping different function times scatters addresses through physical memory Sharing Well chosen codemappings and datasimplify amongmemory processes allocation and management Map virtual pages to the same physical page (here: PP 6) Virtual Address Space for Process : 0 VP VP 2... Address translation 0 PP 2 Physical Address Space (DRAM) Virtual Address Space for Process 2: N- 0 VP VP 2... PP 6 PP 8... (e.g., read-only library code) N- M- 9

Simplifying Linking and Loading Linking Each program has similar virtual address space Code, stack, and shared libraries always start at the same address 0xc0000000 0x40000000 Kernel virtual memory User stack (created at runtime) Memory-mapped region for shared libraries Memory invisible to user code %esp (stack pointer) Run-time heap (created by malloc) brk 0x08048000 0 Read/write segment (.data,.bss) Read-only segment (.init,.text,.rodata) Unused Loaded from the executable file

Simplifying Linking and Loading Linking Each program has similar virtual address space Code, stack, and shared libraries always start at the same address Loading execve() allocates virtual pages for.text and.data sections = creates PTEs marked as invalid The.text and.data sections are copied, page by page, on demand by the virtual memory system 0xc0000000 0x40000000 0x08048000 0 Kernel virtual memory User stack (created at runtime) Memory-mapped region for shared libraries Run-time heap (created by malloc) Read/write segment (.data,.bss) Read-only segment (.init,.text,.rodata) Unused Memory invisible to user code %esp (stack pointer) brk Loaded from the executable file

Outline Basic Concepts 2 Why Virtual Memory Caching Memory management Memory protection 3 Address translation 4 Summary

VM as a Tool for Memory Protection VM as a Tool for Memory Protection Extend PTEs with permission bits Page Extend fault PTEs handler with permission checks these bits before remapping Page Iffault violated, handler sendchecks processthese SIGSEGV before (segmentation remapping fault) If violated, send process SIGSEGV (segmentation fault) Carnegie Mellon Process i: VP 0: VP : VP 2: SUP No No Yes READ WRITE Yes Yes Yes No Yes Yes Address PP 6 PP 4 PP 2 Physical Address Space PP 2 PP 4 PP 6 Process j: VP 0: VP : VP 2: SUP No Yes No READ WRITE Yes Yes Yes No Yes Yes Address PP 9 PP 6 PP PP 8 PP 9 PP

Outline Basic Concepts 2 Why Virtual Memory Caching Memory management Memory protection 3 Address translation 4 Summary

VM Address Translation Virtual Address Space V = {0,,..., N-} Physical Address Space P = {0,,..., M-} Address Translation MAP: V P {Ø} For virtual address a: MAP(a) = a if data at virtual address a is at physical address a in P MAP(a) = Ø if data at virtual address a is not in physical memory Either invalid or stored on disk

Address Translation With a Page Table Address Translation With a Page Table Page table base register (PTBR) Virtual address n- Virtual page number (VPN) p p- Virtual page offset (VPO) Carnegie Mellon 0 Page table address for process Page table Valid Physical page number (PPN) Valid bit = 0: page not in memory (page fault) m- p Physical page number (PPN) Physical address p- Physical page offset (PPO) 0

Address Translation: Page Hit Address Translation: Page Hit Carnegie Mellon CPU Chip CPU VA MMU 2 PTEA PTE 3 PA 4 Cache/ Memory Data 5 ) Processor sends virtual address to MMU ) Processor sends virtual address to MMU 2-3) MMU fetches PTE from page table in memory 4) MMU sends physical address to cache/memory 5) Cache/memory sends data word to processor

Address Translation: Page Hit Address Translation: Page Hit Carnegie Mellon CPU Chip CPU VA MMU 2 PTEA PTE 3 PA 4 Cache/ Memory Data 5 ) Processor sends virtual address to MMU ) Processor sends virtual address to MMU 2-3) MMU fetches PTE from page table in memory 2-3) MMU fetches PTE from page table in memory 4) MMU sends physical address to cache/memory 5) Cache/memory sends data word to processor

Address Translation: Page Hit Address Translation: Page Hit Carnegie Mellon CPU Chip CPU VA MMU 2 PTEA PTE 3 PA 4 Cache/ Memory Data 5 ) Processor sends virtual address to MMU ) Processor sends virtual address to MMU 2-3) MMU fetches PTE from page table in memory 2-3) MMU fetches PTE from page table in memory 4) MMU sends physical address to cache/memory 4) MMU sends physical address to cache/memory 5) Cache/memory sends data word to processor

Address Translation: Page Hit Address Translation: Page Hit Carnegie Mellon CPU Chip CPU VA MMU 2 PTEA PTE 3 PA 4 Cache/ Memory Data 5 ) Processor sends virtual address to MMU ) Processor sends virtual address to MMU 2-3) MMU fetches PTE from page table in memory 2-3) MMU fetches PTE from page table in memory 4) MMU sends physical address to cache/memory 4) MMU sends physical address to cache/memory 5) Cache/memory 5) Cache/memory sends sends data data word word to processor to processor

Address Translation: Page Fault Address Translation: Page Fault Exception Page fault handler 4 Carnegie Mellon CPU Chip CPU VA 7 MMU 2 PTEA PTE 3 Cache/ Memory Victim page 5 New page Disk 6 ) Processor sends virtual address to MMU 2-3) ) MMU Processor fetches sends PTE from virtual page address table in memory to MMU 4) Valid bit is zero, so MMU triggers page fault exception 5) Handler identifies victim (and, if dirty, pages it out to disk) 6) Handler pages in new page and updates PTE in memory 7) Handler returns to original process, restarting faulting instruction 29

Address Translation: Page Fault Address Translation: Page Fault Exception Page fault handler 4 Carnegie Mellon CPU Chip CPU VA 7 MMU 2 PTEA PTE 3 Cache/ Memory Victim page 5 New page Disk 6 ) Processor sends virtual address to MMU 2-3) ) MMU Processor fetches sends PTE from virtual page address table in memory to MMU 4) 2-3) Valid MMU bit is zero, fetches so MMU PTEtriggers from page fault tableexception in memory 5) Handler identifies victim (and, if dirty, pages it out to disk) 6) Handler pages in new page and updates PTE in memory 7) Handler returns to original process, restarting faulting instruction 29

Address Translation: Page Fault Address Translation: Page Fault Exception Page fault handler 4 Carnegie Mellon CPU Chip CPU VA 7 MMU 2 PTEA PTE 3 Cache/ Memory Victim page 5 New page Disk 6 ) Processor sends virtual address to MMU 2-3) ) MMU Processor fetches sends PTE from virtual page address table in memory to MMU 4) 2-3) Valid MMU bit is zero, fetches so MMU PTEtriggers from page fault tableexception in memory 5) 4) Handler Valid identifies bit is zero, victim so MMU (and, if triggers dirty, pages page it out fault to disk) exception 6) Handler pages in new page and updates PTE in memory 7) Handler returns to original process, restarting faulting instruction 29

Address Translation: Page Fault Address Translation: Page Fault Exception Page fault handler 4 Carnegie Mellon CPU Chip CPU VA 7 MMU 2 PTEA PTE 3 Cache/ Memory Victim page 5 New page Disk 6 ) Processor sends virtual address to MMU 2-3) ) MMU Processor fetches sends PTE from virtual page address table in memory to MMU 4) 2-3) Valid MMU bit is zero, fetches so MMU PTEtriggers from page fault tableexception in memory 5) 4) Handler Valid identifies bit is zero, victim so MMU (and, if triggers dirty, pages page it out fault to disk) exception 6) Handler pages in new page and updates PTE in memory 5) Handler identifies victim (and, if dirty, pages it out to disk) 7) Handler returns to original process, restarting faulting instruction 29

Address Translation: Page Fault Address Translation: Page Fault Exception Page fault handler 4 Carnegie Mellon CPU Chip CPU VA 7 MMU 2 PTEA PTE 3 Cache/ Memory Victim page 5 New page Disk 6 ) Processor sends virtual address to MMU 2-3) ) MMU Processor fetches sends PTE from virtual page address table in memory to MMU 4) 2-3) Valid MMU bit is zero, fetches so MMU PTEtriggers from page fault tableexception in memory 5) 4) Handler Valid identifies bit is zero, victim so MMU (and, if triggers dirty, pages page it out fault to disk) exception 6) Handler pages in new page and updates PTE in memory 5) Handler identifies victim (and, if dirty, pages it out to disk) 7) Handler returns to original process, restarting faulting instruction 6) Handler pages in new page and updates PTE in memory 29

Address Translation: Page Fault Address Translation: Page Fault Exception Page fault handler 4 Carnegie Mellon CPU Chip CPU VA 7 MMU 2 PTEA PTE 3 Cache/ Memory Victim page 5 New page Disk 6 ) Processor sends virtual address to MMU 2-3) ) MMU Processor fetches sends PTE from virtual page address table in memory to MMU 4) 2-3) Valid MMU bit is zero, fetches so MMU PTEtriggers from page fault tableexception in memory 5) 4) Handler Valid identifies bit is zero, victim so MMU (and, if triggers dirty, pages page it out fault to disk) exception 6) Handler pages in new page and updates PTE in memory 5) Handler identifies victim (and, if dirty, pages it out to disk) 7) Handler returns to original process, restarting faulting instruction 6) Handler pages in new page and updates PTE in memory 7) Handler returns to original process, restarting faulting instruction 29

Carnegie Me Integrating VM and Cache Integrating VM and Cache PTE CPU Chip CPU VA MMU PTEA PA PTEA hit PTEA miss PA miss PTE PTEA PA Memory PA hit Data Data L cache VA: virtual address, PA: physical address, PTE: page table entry, PTEA = PTE address

Speeding up Translation with a TLB Page table entries (PTEs) are cached in L like any other memory word PTEs may be evicted by other data references PTE hit still requires a small L delay Solution: Translation Lookaside Buffer (TLB) Maps virtual page numbers to physical page numbers Contains complete page table entries for a number of pages

Carnegie M TLB Hit TLB Hit CPU Chip TLB 2 PTE VPN 3 CPU VA MMU PA 4 Cache/ Memory Data 5 A TLB hit eliminates a memory access

Basic Concepts Why Virtual Memory Address translation Carnegie Summary Mellon TLB Miss TLB Miss CPU Chip TLB 4 2 PTE VPN 3 CPU VA MMU PTEA PA Cache/ Memory 5 Data 6 A TLB miss incurs an additional memory access (the PTE) Fortunately, TLB misses are rare. Why? 33

Outline Basic Concepts 2 Why Virtual Memory Caching Memory management Memory protection 3 Address translation 4 Summary

Summary Programmer s view of virtual memory Each process has its own private linear address space Cannot be corrupted by other processes System view of virtual memory Uses memory efficiently by caching virtual memory pages Efficient because of locality Simplifies memory management and programming Simplifies protection by providing a convenient interpositioning point to check permissions

References Dening BEFORE MEMORY WAS VIRTUAL http://cs.gmu.edu/cne/pjd/pubs/bvm.pdf Linux device drivers, Addison-wisely. Understanding Linux virtual memory manager http://www.kernel.org/doc/gorman/pdf/understand.pdf Understanding Linux kernel (3rd edition) http://www.cs.cmu.edu/afs/cs/academic/class/523- f0/www/lectures/5-vm-concepts.pdf