Operating System Design and Implementation Memory Management Part III

Transcription

1 Operating System Design and Implemation Memory Managem Part III Shiao Li Tsao Memory access in a program Cache Managem Regions Virtual to physical address X86/Linux example APIs Page fault handling Segma tion and paging Physical X86/Linux kernel mmap and cache Page table and TLB Internal and external fragma tion Process address space Page frame reclaim 國立交通大學資訊工程學系曹孝櫟老師 1

2 Space vs. User Space Linux User/ Windows, default Windows booted with 3GB switch Space (1GB) Space (2GB) Space (1GB) User Space (3GB) User Space (2GB) User Space (3GB) Virtual Addresses Before/After Context Switches Space (1GB) Process Switch Space (1GB) Process Switch Space (1GB) Firefox /bin/ls Firefox 國立交通大學資訊工程學系曹孝櫟老師 2

3 Per Process Address Space 1GB Stack Memory Mapping 3GB Heap BSS Data Text task_struct and mm_struct Stack Memory Mapping task_struct mm_struct Heap BSS Data Text 國立交通大學資訊工程學系曹孝櫟老師 3

4 struct file (ld.so) struct file (libc.so) struct file VMA Stack Memory Mapping Memory Mapping Heap BSS Data File backed Text File backed mm_struct Segmation Fault Segmation fault Page fault Normal access 國立交通大學資訊工程學系曹孝櫟老師 4

5 Page Fault OS Find page in backing store 3 2 CPU 6 1 v 1 0 Page table 5 Update page table trap Load into 4 Allocation of MM Related Structures 國立交通大學資訊工程學系曹孝櫟老師 5

6 Linux Thread and mm_struct threads do not have a process address space mm field of a kernel thread's process descriptor is NULL Lack of an address space is fine, because kernel threads do not ever access any user space Better performance Memory access in a program Cache Managem Regions Virtual to physical address X86/Linux example APIs Page fault handling Segma tion and paging Physical X86/Linux kernel mmap and cache Page table and TLB Internal and external fragma tion Process address space Page frame reclaim 國立交通大學資訊工程學系曹孝櫟老師 6

7 mm_struct,, and page mm_struct struct file struct address_space struct page VMA, Page and Frame 3GB Virtual User Space =4KB(perpage)* 786,432 (pages) page.. page page page Page 5 Page 4 Page 3 Page 2 Page 1 Frame Frame 2913 Frame 102 國立交通大學資訊工程學系曹孝櫟老師 7

8 Memory Region Linear Address Space Memory Regions Red Black Tree O(log n) By Cburnett,CC BY SA 3.0, via Wikimedia Commons" 國立交通大學資訊工程學系曹孝櫟老師 8

9 VMA Search Using R B Tree vm_end=300 vm_end=100 vm_end=400 vm_end=30 vm_end=150 vm_end=490 Ref: w4118/lectures/l20 adv mm.pdf malloc() and Heap Managem 國立交通大學資訊工程學系曹孝櫟老師 9

10 Process Address Space Memory Access contiguous Memory Region Grow original region Access discrete Create new regions Remove at the back Shrink original region Remove in the middle Split into two regions 19 Enlarge VMA Call brk() to grow its heap Page fault Ref: the kernel manages your / 國立交通大學資訊工程學系曹孝櫟老師 10

11 Memory access in a program Cache Managem Regions Virtual to physical address X86/Linux example APIs Page fault handling Segma tion and paging Physical X86/Linux kernel mmap and cache Page table and TLB Internal and external fragma tion Process address space Page frame reclaim Segmation Fault Segmation fault Page fault Normal access 國立交通大學資訊工程學系曹孝櫟老師 11

12 Page Fault Handler No Belong to this process? Yes No User mode? Yes No Access right granted? Yes Bug Illegal access, send SIGSEGV legal access, allocate page frame Memory access in a program Cache Managem Regions Virtual to physical address X86/Linux example APIs Page fault handling Segma tion and paging Physical X86/Linux kernel mmap and cache Page table and TLB Internal and external fragma tion Process address space Page frame reclaim 國立交通大學資訊工程學系曹孝櫟老師 12

13 Memory Mapped Virtual Memory mapped Backing store Data Structure for File Memory Mapping address_space struct page struct inode Page frame struct struct file Ref: Understanding the Linux, Daniel P. Bovet, Marco Cesati 國立交通大學資訊工程學系曹孝櫟老師 13

14 Two Types of Memory Mapping A file mapping maps a region to a region of a file backing store = file as long as the mapping is established, the cont of the file can be read from or written to using direct access ( as if they were variables ) An mappings maps a region to a fresh virtual area filled with 0 backing store = zero ed area Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli Having mapped pages in common Thanks to virtual, differ processes can have mapped pages in common More precisely, mapped pages in differ processes can refer to physical pages that have the same backing store That can happen in two ways: through fork, as mappings are inherited by children when multiple processes map the same region of a file Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli 國立交通大學資訊工程學系曹孝櫟老師 14

15 Shared vs private mappings With mapped pages in common, the involved processes might see changes performed by others to mapped pages in common, depending on whether the mapping is: private mapping in this case modifications are not visible to other processes. pages are initially the same, but modification are not shared, as it happens with copy on write after fork private mappings are also known as copy on write mappings shared mapping in this case modifications to mapped pages in common are visible to all involved processes i.e. pages are not copied on write Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli File mapping and layout Virtual file Memory mapped Memory mapped portion of file Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli 國立交通大學資訊工程學系曹孝櫟老師 15

16 Effects: Shared file mapping (#2) processes mapping the same region of a file share physical frames more precisely: they have virtual pages that map to the same physical frames additionally, the involved physical frames have the mapped file as ultimate backing store i.e. modifications to the (shared) physical frames are saved to the mapped file on disk Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli Shared file mapping (#2) Process A page table Physical Mapped region Open file Process B page table Mapped pages I/O Managed by Mapped pages Mapped region 國立交通大學資訊工程學系曹孝櫟老師 16

17 Shared file mapping (#2) (cont.) Use cases mapped I/O, as an alternative to read/write as in the case of private file mapping, but here it works for both reading and writing data Inter process communication, with the following characteristics: data transfer (not byte stream) with filesystem persistence among unrelated processes Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli Memory mapped I/O Given that: cont is initialized from file changes to are reflected to file we can perform I/O by simply changing bytes of. Access to file mappings is less intuitive than sequial read/write operations the mal model is that of working on your data as a huge byte array (which is what is, after all) a best practice to follow is that of defining struct s that correspond to elems stored in the mapping, and copy them around with memcpy & co Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli 國立交通大學資訊工程學系曹孝櫟老師 17

18 Memory mapped I/O advantages performance gain: 1 copy with read/write I/O each action involves 2 copies: 1 between user space and kernel buffers + 1 between kernel buffers and the I/O device with mapped I/O only the 2nd copy remains flash exercise: how many copies for standard I/O? performance gain: no context switch no syscall and no context switch is involved in accessing mapped page faults are possible, though reduced usage we avoid user space buffers! less needed if mapped region is shared, we use only one set of buffers for all processes seeking is simplified no need of explicit lseek, just pointer manipulation Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli Memory mapped I/O disadvantages garbage the size of mapped regions is a multiple of system page size mapping regions which are way smaller than that can result in a significant waste of mapping must fit in the process address space on 32 bits systems, a large number of mappings of various sizes might result in fragmation it then becomes harder to find continuous space to grant large mappings the problem is substantially diminished on 64 bits systems there is kernel overhead in maintaining mappings for small mappings, the overhead can dominate the advantages mapped I/O is best used with large files and random access Ref: Programmation Systèmes /Memory Mapping, Stefano Zacchiroli 國立交通大學資訊工程學系曹孝櫟老師 18

19 Page Cache A read(abc.dat, buf, 512) Search for abc.dat #0 libc.so /bin/ls libc.so libc.so abc.dat /bin/ls libc.so disk A read(abc.dat, buf, 512) Ref: cache the affair between and files/ Read only Read/write A abc.dat #2 abc.dat #1 abc.dat #0 B A write abc.dat #2 abc.dat #1 abc.dat #0 B A abc.dat #2 abc.dat #1 abc.dat #0 B A write abc.dat #2 abc.dat #1 abc.dat #0 B Page fault, copy on write Ref: cache the affair between and files/ 國立交通大學資訊工程學系曹孝櫟老師 19

20 stack A B stack libc.so /lib/libc.so, #1 /lib/libc.so, #0 libc.so abc.dat abc.dat, #1 abc.dat, #2 abc.dat abc.dat, #0 heap heap text A, #1 A, #0 text Ref: cache the affair between and files/ Memory access in a program Cache Managem Regions Virtual to physical address X86/Linux example APIs Page fault handling Segma tion and paging Physical X86/Linux kernel mmap and cache Page table and TLB Internal and external fragma tion Process address space Page frame reclaim 國立交通大學資訊工程學系曹孝櫟老師 20

21 The Types of Pages Considered by the PFRA Ref: Understanding the Linux, Daniel P. Bovet, Marco Cesati Ref: CS161: Operating Systems, Matt Welsh 國立交通大學資訊工程學系曹孝櫟老師 21

22 Implemation of PFRA Ref: Understanding the Linux, Daniel P. Bovet, Marco Cesati 國立交通大學資訊工程學系曹孝櫟老師 22