Memory Allocators. Pradipta De

Transcription

1 Memory Allocators Pradipta De

2 Today s Topic How does memory allocators work? Challenges and techniques for memory allocation (Physical) Memory allocation techniques in Linux CSE506: Page Frame Management 2

3 Basic facts Physical memory: actual RAM in machine Virtual Memory: address space size, determined by the machine architecture Early days: VM >> PM On 32 bit arch: VM (4GB) < PM in tens of GBs On 64 bit arch: VM >> PM

4 Fragmentation Single process on a system Allocates and deallocates memory in variable chunk sizes Variable sized holes are created in memory region External fragmentation Assume the chunk size (c) is fixed If memory requirement is [c+1], then two chunks are allocated memory wasted [c-1] Internal fragmentation

5 Memory allocators Memory allocator keeps track of used and free memory, and services memory requests Maintains all free blocks in a doubly linked list Types of memory allocators First fit: allocate the first hole that is sufficient Next fit: start search for hole where the last search ended Best fit: allocate the smallest hole Worst fit: allocate the largest hole

6 Buddy System: Example Let us allocate 128 contiguous blocks of memory using Buddy System Initialization Lists of contiguous memory blocks of sizes 4, 8, 16, Search for space First checks for a free block in the 128 list No free block look in 256 list If free block found, then allocate 128 blocks to the request, and put remaining 128 blocks to 128-list If free block NOT found, look in higher block size list, allocating and redistributing the remaining blocks If no block can be allocated an error is reported

7 Buddy System: Technique Allocates memory from fixed-size segment consisting of physically-contiguous pages Memory allocated using power-of-2 allocator Satisfies requests in units sized as power of 2 Request rounded up to next highest power of 2 When smaller allocation needed than what is available, current chunk is split into two buddies of next-lower power of 2 Continue until appropriate sized chunk available

8 Buddy System: Illustration When two 32 KB C L are deallocated, they can be returned to the memory pool as 1one 64 KB B L

9 Buddy System: Problem Let us allocate a request for 175 contiguous memory blocks Picks from a list 256 blocks Unused memory blocks ( ) = 81 Buddy System prevents external fragmentation, but cannot tackle internal fragmentation When allocation requests are for small memory regions, then memory wastage is significant

10 Object Allocators Many data structures in kernel are frequently allocated and deallocated leads to memory fragmentation Solution generate a collection of objects of the correct size and pre-allocate them

11 SLAB Slab is one or more physically contiguous pages Cache consists of one or more slabs Single cache for each unique data structure type Each cache filled with objects instantiations of the data structure When cache created, filled with objects marked as free When structures stored, objects marked as used If slab is full of used objects, next object allocated from empty slab If no empty slabs, new slab allocated Benefits include no (internal) fragmentation, memory requests are serviced quickly

12 SLAB: Illustration

13 SLAB allocator organization Cache_chain: linked list of slab caches (useful for best-fit algo) kmem_cache: defines a pool of objects of a given size Slabs_full: all objects are used Slabs_empty: all objects are free Slabs_partial: consists of free and used items

14 SLAB Allocator example Request to allocate 1.7 KB of memory Look for an free object in slabs_partial If none exist, then assign from slabs_free If no empty slab, then request for cntiguous memory blocks, and assign to slab_empty Service memory request from slab_empty

15 Recap The virtual address space of a process is divided into two parts 0x to PAGE_OFFSET-1 can be addressed in either user or kernel mode PAGE_OFFSET to 0xffffffff (4GB-1) can be addressed only in kernel mode PAGE_OFFSET is typically 0xc (= 3GB)

16 32-bit machines

17 Linux Kernel Memory Allocators Manages the available page frames Typically 4 KB page frames Need to handle requests for contiguous page frames

18 Zoned Page Frame Allocator Due to hardware peculiarities, page frames are not uniformly equivalent Zone_DMA: first 16 MB to direct map devices ( < 16 MB) Zone_Normal: between 16 MB to 896 MB : for kernel usage Zone_HighMem: above 896 MB

19 Linux Object Allocators Linux slabs are initialized when the kernel boots Allocates slabs for different frequently used data types: process descriptor, inodes, Kmalloc function uses slab allocator If kmalloc called with unknown slab size, function to create new slab size

20 Check: cat /proc/slabinfo

21 SLUB and SLOB Linux default memory allocator is SLUB SLUB: The unqueued slab allocator v6 Better performance and scalability by reorganizing data structures (reduced housekeeping overhead) SLOB: Simple list of Blocks On low memory (embedded) devices Create simplified slob cache (low, medium, high) Abandon cache/slob/object hierarchy Not specific to data structures

22 Putting It Together Summary of memory allocation techniques Hierarchy of calls: Kernel code object allocator page allocator physical page frame Page allocator Buddy System Object allocator (all provides same API) SLAB allocator: traditional allocator SLOB allocator: for embedded system SLUB allocator: default, for large system CSE506: Page Frame Management 22