Virtual Memory Management in Linux (Part II)

Transcription

1 Virtual Memory Management in Linux (Part II) Minsoo Ryu Department of Computer Science and Engineering

2 2 1 Page Table and Page Fault Handling Page X 2 Page Cache Page X 3 Page Frame Reclamation (Swapping and OOM) Page X 4 Q & A Page X 2

3 3 Page Table In Linux, the page tables consist of three levels Page global directory (PGD) Consists of an array of pgd_t Entries in PGD point to entries in PMD Page middle directory (PMD) Consists of an array of pmd_t Entries in PMD point to entries in PTE Page table (PTE) Consists of an array of pte_t Page table entries point to physical pages 3

4 4 Virtual Address Structure Not only the length but also the way are different depending on the processor architectures 4

5 5 Three-Level Page Table 5

6 6 Page Fault Handling An assembler routine serves as the entry point for page fault (in arch/x86/kernel/entry_32.s) Invokes do_page_fault()(in arch/x86/mm/fault.c) 6

7 7 Page Fault Handling Complex logic: Easier to read code than read this slide! 7

8 8 Page Fault Handling The page fault is finally handled by the handle_pte_fault() function 8

9 9 Demand Fetching via Page Faults 9

10 Page Cache

11 11 Page Cache The page cache is a cache of pages in RAM that correspond to physical blocks on a disk The pages originate from reads and writes of regular filesystem files, block device files, and memory-mapped files During a page I/O operation, such as read(), The kernel checks whether the data resides in the page cache If the data is in the page cache, the kernel can quickly return the requested page from memory rather than read the data off the comparatively slow disk 11

12 12 The address_space Object Linux page cache uses a new object to manage entries in the cache and page I/O operations That object is the address_space structure The address_space object is the physical analogue to the virtual vm_area_struct While a single file may be represented by 10 vm_area_struct structures (if, say, five processes each mmap() it twice), the file has only one address_space structure A physical page is 4KB in size on the x86 architecture, whereas a disk block on many filesystems can be as small as 512 bytes Therefore, 8 blocks might fit in a single page The blocks need not be contiguous because the files might be laid out all over the disk 12

13 13 The address_space Object The address_space structure is defined in <linux/fs.h>: 13

14 14 Finding a Cached Page Whenever a process issues the read() system call Kernel checks if the requisite data is in the page cache 14

15 15 Radix Tree Each address_space has a unique radix tree stored as page_tree A radix tree is a type of binary tree The radix tree enables quick searching for the desired page, given only the file offset Page cache searching functions such as find_get_page() call radix_tree_lookup(), which performs a search on the given tree for the given object The find_get_page() function receives as its parameters a pointer to an address_space and an offset value The offset value is used as the key for radix tree searching 15

16 16 Binary Trie Trie (comes from retrieval) An ordered tree data structure to store a dynamic set or associative array where the keys are usually strings 16

17 17 Radix Trie (Patricia Trie or Radix Tree) Searching for R = move right (since bit 0 is 1), then left (since bit 4 is 0) On the way down the tree, we check only the key bits indicated in the numbers over the nodes (and ignore the keys in the nodes) Searching for S = or I = 01001? 17

18 18 Radix Trie (Patricia Trie) Inserting I = We add a new node to check bit 4, since H = and I = differ in only that bit (top) 18

19 19 Radix Tree for Page Cache Consists of three data structures radix_tree_root The root node radix_tree_node Node for the intermediate node page The leaf node 19

20 20 Radix Tree for Page Cache The root of the tree: radix_tree_root Holds the height of the tree and a pointer to the first node data structure height specifies the current height of the tree Maximum number of pages that can be stored in a tree can be derived from height rnode points the first node of the tree 20

21 21 Radix Tree for Page Cache The node of the tree: radix_tree_node Basically arrays with 2 RADIX_TREE_MAP_SHIFT entries Typically RADIX_TREE_MAP_SHIFT = 6 (2 6 = 64 entries) Each entry is called slot Points either to other nodes or to data elements (pages) Unused entries have null pointers 21

22 22 Write Caching Data ends up in the page cache via read operations, but what happens when a process writes to disk? Two strategies Write-through: automatically update both the in memory cache and the on-disk file Write-back: perform write operations directly into the page cache, but the backing store is not immediately or directly updated Instead, the written-to pages in the page cache are marked as dirty and are added to a dirty list Periodically, pages in the dirty list are written back to disk 22

23 23 Flusher Threads Dirty page writeback occurs in three situations: When free memory shrinks below a specified threshold When dirty data grows older than a specific threshold When a user process invokes the sync() and fsync() system calls A gang of kernel threads, the flusher threads, performs write-back operations When free memory drops below this threshold, the kernel invokes the wakeup_flusher_threads() A flusher thread periodically wakes up (unrelated to lowmemory conditions) and writes out old dirty pages 23

24 24 Cache Eviction Cache eviction is required either to make room for more relevant cache entries or to shrink the cache to make available more RAM for other uses Linux simply replaces them with something else The ideal eviction strategy evicts the pages least likely to be used in the future Such a strategy is ideal, but impossible to implement LRU (least recently used) is a good strategy However, one particular failure of the LRU strategy is that many files are accessed once and then never again 24

25 25 Two-List Strategy Linux keeps two lists Active list: pages considered hot and not available for eviction Inactive list: pages available for cache eviction Pages are placed on the active list only when they are accessed while already residing on the inactive list Both lists are maintained in a pseudo-lru manner Items are added to the tail and removed from the head, as with a queue The lists are kept in balance: If the active list becomes much larger than the inactive list, items from the active list s head are moved back to the inactive list 25

26 26 Two-List Strategy The two-list strategy solves the only-used-once failure in a classic LRU and also enables simpler, pseudo-lru semantics to perform well This two-list approach is also known as LRU/2; it can be generalized to n-lists, called LRU/n 26

27 27 Read Ahead Predicts the future requests and read ahead the pages that is expected to be read in the next request Usually pages are read sequentially, it therefore makes sense to read ahead the following data Example When a process has read a file linearly from position A to B, this practice will usually continue for a while 27

28 28 Read Ahead The current Linux controls read-ahead mechanism from three places do_generic_file_read() Standard routines of the kernel to read data Triggered by read() system call filemap_fault() Page fault handler to read missing page from memory mapping generic_file_splice_read() Support the splice system call that allows for passing data between two file descriptors directly in kernel space 28

29 29 Read Ahead When process attempts to perform read operation via do_generic_file_read() It firstly find the contents in the page cache It then perform synchronous read-ahead and asynchronous read-ahead consecutively 29

30 30 Read Ahead Synchronous/asynchronous read-ahead File is equipped with file_ra_state structure 30

31 31 Read Ahead 31

32 Page Frame Reclamation (Swapping & OOM)

33 33 Page Frame Reclamation Page frame reclaiming must be performed before all the free memory has been used up Otherwise, the kernel might be easily trapped in a deadly chain of memory requests that leads to a system crash If no free page frame exists, no page frame can be freed To free a page frame the kernel must write its data to disk, the kernel requires another page frame (to allocate the buffer heads for the I/O data transfer) One of the goals of page frame reclaiming is to conserve a minimal pool of free page frames so that the kernel may safely recover from "low on memory" conditions 33

34 34 Target Pages for Reclamation 34

35 35 Entry Points for Page Frame Reclamation 35

36 36 The try_to_free_pages Function The goal of try_to_free_pages is to free at least 32 page frames Repeatedly invoke the shrink_caches( ) and shrink_slab( ) functions shrink_caches( ) reclaims inactive pages shrink_slab( ) reclaims pages from the shrinkable kernel caches One last resort is killing a process to free all its page frames (out_of_memory) 36

37 37 Two Lists and LRU Linux kernel maintains LRU cache Makes use of page vectors to collect page instances Place them block-by-block on the system s active and inactive lists 37

38 38 Two Lists and LRU Two flags PG_active denotes the current activity rating PG_referenced signals if page has been recently referenced 38

39 39 Two Lists and LRU 39

40 40 Out of Memory Killer The out_of_memory( ) function is invoked by alloc_pages( ) when the free memory is very low and other techniques fail to reclaim page frames Invoke select_bad_process( ) to select a victim among the existing processes Invokes oom_kill_process( ) to perform the sacrifice 40

41 41 Out of Memory Killer Criteria for victim process selection The victim should own a large number of page frames Killing the victim should lose a small amount of work It is not a good idea to kill a batch process that has been working for hours or days The victim should be a low static priority process The victim should not be a process with root privileges The victim should not directly access hardware devices (such as the X Window server), because the hardware could be left in an unpredictable state The victim cannot be swapper (process 0), init (process 1), or any other kernel thread 41

42 42 The kswapd Kernel Thread Invoke shrink_zone( ) and shrink_slab( ) to reclaim pages from the LRU lists kswapd works in two different manners When the remaining free page is below WMARK_LOW Swapping daemon is woken up and swaps the pages out When the remaining free page is below WMARK_MIN Kernel directly frees the pages 42

43 43 Swapping Swapping is to use some space on disk as an extension of RAM Swap areas The pages swapped out from memory are stored in a swap area It may be implemented either as a disk partition of its own or as a file Several different swap areas may be defined, up to a maximum number specified by the MAX_SWAPFILES macro (usually set to 32) Each swap area consists of a sequence of page slots: 4,096-byte blocks used to contain a swapped-out page 43

44 44 Swap Area Structure 44

45 45 Swap Cache Transferring pages to and from a swap area is an activity that can induce many race conditions Multiple swap-ins Two processes may concurrently try to swap in the same shared anonymous page Concurrent swap-ins and swap-outs A process may swap-in a page that is being swapped out by the kernel The swap cache has been introduced to solve these kinds of synchronization problems The key rule is that nobody can start a swap-in or swapout without checking whether the swap cache already includes the affected page 45

46 46 Swapping Out Pages Four steps Inserting the page frame in the swap cache The add_to_swap( ) function allocates a new page slot in a swap area and inserts a page frame in the swap cache Updating the Page Table entries Writing the page into the swap area The swap_writepage( ) function invokes submit_bio( ) Removing the page frame from the swap cache Invoke delete_from_swap_cache( ) to remove the page frame from the swap cache 46

47 47 Swapping In Pages Swap-in takes place when a process attempts to address a page that has been swapped out to disk When the page fault is triggered, Chooses page fault handler according to the relevant page table entry If no page table entry is present, Kernel must load the page from scratch via do_linear_fault() or do_anonymous_page() If the page is marked as not present but information on the page is held in the page table, do_swap_page() handles the fault If the page table entry belongs to a nonlinear mapping, do_nonlinear_fault() handles the fault 47

48 48 Procedures for Swapping In Pages At the first time, kernel must check whether the requested page is still in the swap cache by invoking the do_swap_page() Function If it is, read data from the cache 48

49 49 Procedures for Swapping In Pages If the page is not in the swap cache, Cause the page to be read via read_swap_cache_async() Initiate a read-ahead operation to read a few pages in anticipation via swapin_readahead() Read request are issued not only for the desired page but also for a few pages in the adjacent slots 49

50 50 Procedures for Swapping In Pages Code flow for read_swap_cache_async() find_get_page() is invoked to check whether the page is in the swap cache If the page is found in the cache, the desired page is immediately returned 50

51 51 Procedures for Swapping In Pages If the page is not found, page_alloc_vma() is invoked to allocate a fresh memory page to hold the data from the swap area If this function fails to allocate new pages, the higher level code instructs the OOM killer to close the least-important process 51

52 52 Procedures for Swapping In Pages If the page is successfully reserved, Kernel adds the page instance To the swap cache via add_to_swap_cache() To the LRU cache via lru_cache_add_active() The page data are then transferred from the swap area to RAM by means of swap_readpage() 52

53 53 swapin_readahead() As when reading files, the kernel also uses a readahead mechanism to read data from swap areas Ensure that data are read in anticipation so that future page-in requests can be satisfied quickly Improving system performance 53

54 54 Low Memory Killer (LMK) Android has a low-memory-killer which is a in kernel driver that watches memory levels and will kill specified applications before we hit critical OOM levels Avoids poor performance immediately before OOM Uses slab shrinker Particularly useful with no swap Android uses the standard oom_adj process property to prioritize the pool of running processes and choose which to kill to free system memory Enables user-space policies to control which process should be killed on low memory 54

55 55 Low Memory Killer (LMK) oom_adj Represents the importance of process Processes can be persistent, foreground, visible, service, background, and empty process A small value of oom_adj means a more important process min_free Threshold value for the amount of free memory When the free memory drops below a certain min_free value, processes whose oom_adj value is greater than or equal to the specified value will be killed 55

56 56 56