Advanced Programming

Transcription

1 Advanced Programming Memory Management The Heap Dr. Miri (Kopel) Ben-Nissan Bar-Ilan University

2 The Traditional Heap Manager The standard C library routines for allocating and freeing memory are malloc() and free(). C++ uses the new and delete operators. void* malloc ( size_t size ); void free ( void *memblock ); Void* operator new (size_t); void operator delete (void*); Void* operator new[](size_t); void operator delete[](void*); 2

3 The Traditional Heap Manager (Cont ) Main problem with those interfaces: they assumes the programmer never makes a mistake in calling the memory management functions. #include <stdlib.h> int main(void) { void *pmem=malloc(100); 3 free(pmem); free(pmem); return 0; } <- behavior undefined!

4 The Traditional Heap Manager (Cont ) Besides the interface, there is also general problems with dynamic memory allocations: Allocation and de-allocation are non-deterministic with respect to time. De-allocation: Explicit by the programmer can cause memory leaks if not handled correctly. Implicit by garbage collector, nondeterministic. Fragmentation problem. 4

5 malloc void *malloc(long numbytes): This allocates numbytes of memory and returns a pointer to the first byte. Returns NULL if failed. Guaranties well fragmentation. void free(void *firstbyte): Given a pointer that has been returned by a previous malloc, this gives the space that was allocated back to the process's "free space." 5

6 malloc (cont ) You can imagine the heap as a vector of bytes (characters), starting at address managed_memory_start, and last_valid_address as a pointer to the first available byte in the heap. int has_initialized = 0; void *managed_memory_start; //start of heap void *last_valid_address; //end of heap void *last_allocated= managed_memory_start; //no allocations yet. 6

7 7 malloc (cont )

8 malloc (cont ) A trivial implementation of malloc in C is: void* malloc ( int size ) { void* loc = last_allocated; }; last_allocated += size; return loc; 8

9 malloc (cont ) If you always create dynamic structures but never delete them, you will eventually run out of heap space. In that case, malloc will request the operating system for more heap. This is very expensive, because it may require to move the stack data to higher memory locations. 9

10 malloc (cont ) Alternatively, you can recycle the dynamically allocated data that you don't use. This is done using free in C or delete in C++. How we will know how many bytes to move the pointer back? What if it is in the middle of the heap? 10

11 malloc (cont ) We will add an header to each allocated block in the heap: struct mem_control_block { int is_available; int size; }; 11

12 malloc (cont ) As said previously, the heap is a continues (in terms of virtual address) space of memory with three bounds: Starting point. Maximum limit. End point (break). The break marks the end of the mapped memory spaces, that is, the part of the virtual address space that has correspondence into real memory. 12

13 malloc (cont ) In the Unix operating system, we find the following two system calls that enables us to control the break border: brk places the break at the given address addr and returns 0 if successful, -1 otherwise. sbrk moves the break by the given increment (in bytes). Depending on system implementation, it returns the previous or the new break address. On failure, it returns (void*)-1. On some systems, sbrk accepts negative values (in order to free some mapped memory). 13

14 when increment is null (i.e. sbrk(0)), the returned value is the actual break address. sbrk is thus used to retrieve the beginning of the heap which is the initial position of the break. Accessing addresses above the break should trigger a bus error. The remaining space between the break and the maximum limit of the heap is not associated to physical memory by the virtual memory manager of the system. 14

15 malloc (cont ) #include <unistd.h> /* Include the sbrk function */ int has_initialized = 0; void *managed_memory_start; void *last_valid_address; void malloc_init() { /* grab the last valid address from the OS */ last_valid_address = sbrk(0); /* we assume that managed memory size at the beginning * is empty, so just set the beginning to be last_valid_address */ managed_memory_start = last_valid_address; } 15 has_initialized = 1;

16 malloc (cont ) struct mem_control_block { int is_available; int size; }; void free(void *firstbyte) { struct mem_control_block *mcb; mcb = firstbyte - sizeof(struct mem_control_block); mcb->is_available = 1; } 16 return;

17 void *malloc(long numbytes) { /* Holds where we are looking in memory */ void *current_location; /* This is the same as current_location, but cast to a memory_control_block */ struct mem_control_block *current_location_mcb; void *memory_location; malloc (cont ) 17 if(! has_initialized) { malloc_init(); }

18 THE HEAP malloc (cont ) numbytes = numbytes + sizeof(struct mem_control_block); /* Set memory_location to 0 until we find a suitable location */ memory_location = 0; /* Begin searching at the start of managed memory */ current_location = managed_memory_start; managed_memory_start current_location header numbytes last_valid_address 18

19 19 malloc (cont ) /* Keep going until we have searched all allocated space */ while(current_location!= last_valid_address) { current_location_mcb = (struct mem_control_block *)current_location; if(current_location_mcb->is_available){ if(current_location_mcb->size >= numbytes){ current_location_mcb->is_available = 0; memory_location = current_location; break; } } /* If we made it here, it's because the current memory * block not suitable, move to the next one */ current_location = current_location + current_location_mcb->size; }

20 malloc (cont ) /* If we still don't have a valid location, we'll have to ask the operating * system for more memory */ if(! memory_location) { /* Move the program break numbytes further */ if (sbrk(numbytes) == (void*)(-1) return 0; /*unable to resize heap size.*/ /* The new memory will be where the last valid address left off */ memory_location = last_valid_address; /* We'll move the last valid address forward numbytes */ last_valid_address = last_valid_address + numbytes; } 20 /* We need to initialize the mem_control_block */ current_location_mcb = (struct mem_control_block *)memory_location; current_location_mcb->is_available = 0; current_location_mcb->size = numbytes;

21 malloc (cont ) /* Move the pointer past the mem_control_block */ memory_location = memory_location + sizeof(struct mem_control_block); } /* Return the pointer */ return memory_location; Taken from: 21

22 malloc (cont ) Exercise Write an aligned malloc & free function, which takes two parameters: The number of bytes to allocate. The aligned byte (which is always power of 2). Example: align_malloc (1000,128); will allocate the memory and return a memory address which is a multiplication of 128. aligned_free(); will free memory allocated by align_malloc. 22

23 Allocation Policies Allocation Policy = The concrete policy used by an allocator (memory manager) for choosing a free block to satisfy an allocation request. Two methods: Fixed size blocks. Variable size blocks. Most popular policies (for both methods): First Fit Best Fit Next Fit 23

24 Notice: malloc allocates pages (or chunks ) of fixed size. Than this chunk is split: part of it goes to the user, and the rest goes to the free-blocks list. 24

25 Allocation Policies (Cont ) Best-fit: choose smallest hole that fits creates small holes! First-fit: choose first hole from beginning that fits generally superior Next-fit: variation: first hole from last placement that fits. (also worst fit) 25

26 Allocation Policies (Cont ) First fit simply searches the free list from the beginning, and uses the first free block large enough to satisfy the request. If the block is larger than necessary, it is either used as is (internal fragmentation) or split and the remainder is put on the free list. Easy to implement Memory overhead is small. Internal fragmentation. Lots of small blocks at the beginning 26

27 Allocation Policies (Cont ) Best Fit always allocates from the smallest suitable free block. In theory, best fit may exhibit bad fragmentation, but in practice this is not commonly observed. Minimizes the amount of wasted space Sequential best-fit is inefficient 27

28 Allocation Policies (Cont ) Next Fit is a variant of the first fit allocation mechanism that uses a roving pointer on a circular free block chain. Each allocation begins looking where the previous one finished. No accumulation of small blocks, which would have to be examined on every allocation. Generally increases fragmentation, like first-fit. Tendency to spread related objects out in memory. Gives quite poor locality for the allocator (as the roving pointer rotates around memory, the free blocks touched are those least-recently used). 28

29 Allocation Policies (Cont ) All algorithms can be improved by keeping the information about the free blocks in the heap. There are many techniques for that, like: Free-page bitmap. Free list. And many more Some of the techniques allocates fixed-size blocks and other a variable-sized blocks. 29

30 Allocation Policies (Cont ) Free-page bitmap. Every bit in the bitmap represents one 4096 byte (4KB) block in the heap. If the bit is 0, the block has been allocated. If the bit is 1, then it is free. Assume a 4MB heap. Each allocated block is 4KB. allocation: O(N) free: O(1) The free page bitmap is 1024 bits wide: 4MB = 4096KB => 4096KB/4096B =

31 Allocation Policies (Cont ) Free list is a linked list of freeblock structures. struct freeblock { struct freeblock *next; char garbage[4092]; }; 31 Internal fragmentation! In the data portion of the processes, we store a pointer which points to the first freeblock structure.

32 Allocation Policies (Cont ) Variable Size Allocations allows the allocation of memory of variable sizes. Instead of having several smaller blocks to allocate, we start with one big block of memory and split it as necessary to make smaller blocks. struct freeblock { struct freeblock *next; size_t size; }; 32

33 Allocation Policies (Cont ) Segregated free lists uses a set of lists of free blocks. Each list holds blocks of a given size A freed block is pushed onto the relevant list When a block is needed, the relevant list is used 33

34 Memory Allocation in C++ Even though malloc and free are available in C++, their use is not recommended. It is always preferred to use new and delete, especially when working with objects. new and delete point to the correct memory type (they are type safe), so casts are not necessary. new invokes the constructor, allowing the initialization of objects, and delete invokes the destructor. new figures out the size it needs to allocate, so the programmer doesn't have to specify it. new throws an exception of type std::bad_alloc when it fails, instead of only returning a NULL pointer like malloc. 34

35 new and delete void* operator new(size_t); void operator delete(void*); void* operator new[](size_t); void operator delete[](void*); 35

36 The new & delete operators Do I need to check for NULL after p = new Fred()? 36

37 The new & delete operator (Cont ) failure Attempt to allocate memory success no Handler installed? yes Return the address Throw bad_alloc Call the handler function retry 37

38 The new & delete operators (Cont ) Do I need to check for NULL after p = new Fred()? Do NOT check for NULL when allocating memory with new, since new will never return NULL! The ptr will retain its previous value. Malloc can fail. When you call malloc, or when you get a pointer back from a function that calls malloc, you should check to ensure that the pointer you got back wasn't NULL. 38

39 The new & delete operators (Cont ) Until the early 1990s, operator new behaved very much like standard C's malloc(), as far as allocation failures were concerned. When it failed to allocated a memory block of the requested size, it would return a NULL pointer. This behavior was changed in the early 1990s. Instead of returning a NULL pointer, new throws an exception of type std::bad_alloc to indicate a failure. Testing the return value is utterly wrong. 39

40 The new & delete operators (Cont ) p = new CMyClass; if (!p){ //not to be used under ISO //compliant compilers cout<<"allocation failure! ; exit(1); } The if condition is never evaluated in the event of an allocation failure, because the exception thrown by new has already transferred control to a matching catch() clause, if one exists. If no matching catch() is found, C++ automatically calls function terminate() that terminates the program unconditionally. 40

41 The new & delete operators (Cont ) try { p = new DerivedWind; } catch (std::bad_alloc &ba) { cout<<"allocation failure! ; //...additional cleanup } Embedded systems don't always support exceptions. In such environments, nothrow new can be rather useful. 41

42 The new & delete operator (Cont ) Before operator new throws an exception, it calls a client-specified error-handling function, called a newhandler. To specify the out-of-memory-handling function, clients call set_new_handler, a standard library function declared in <new>: namespace std { typedef void(*new_handler_ptr)(); //ptr to function new_handler set_new_handler(new_handler_ptr p) throw(); } 42

43 The new & delete operator (Cont ) Example: void OutOfMem() { std::cerr<< Failed to allocate memory\n ; std::abort(); } int main() { std::set_new_handler(outofmem); int* pbigdataarray = new int[ l]; // } 43

44 The new & delete operator (Cont ) A well designed new-handler function must do one of the following: 44 Make more memory available. Install a different new-handler. De-install the new-handler, by passing NULL to set_new_handler. Like this the new operator will throw an exception. Throw an exception. No return (abort or exit).

45 The new & delete operator (Cont ) Further reading at home: Handling memory failures in different ways, depending on the class of the object being allocated. (Effective C++, pages ). 45

46 Overloading new & delete When you overload operator new and operator delete, you re changing only the way raw storage is allocated. The compiler will simply call your new instead of the default version to allocate storage, then call the constructor for that storage. So, although the compiler allocates storage and calls the constructor when it sees new, all you can change when you overload new is the storage allocation portion. (delete has a similar limitation). 46

47 Overloading new & delete (Cont ) When you overload operator new, you also replace the behavior when it runs out of memory, so you must decide what to do in your operator new. (return zero, write a loop to call the new-handler and retry allocation, or (typically) throw a bad_alloc exception). 47

48 Overloading global new & delete When the global versions of new and delete are unsatisfactory for the whole system. If you overload the global versions, you make the defaults completely inaccessible you can t even call them from inside your redefinitions. 48

49 Overloading global new & delete (Cont ) The overloaded new must take an argument of size_t (the Standard C standard type for sizes). You must return a pointer either to an object of that size (or bigger, if you have some reason to do so), or to zero if you can t find the memory (in which case the constructor is not called!). However, if you can t find the memory, you should probably do something more drastic than just returning zero, like calling the new-handler or throwing an exception, to signal that there s a problem. The return value of operator new is a void*, not a pointer to any particular type. 49

50 Overloading global new & delete (Cont ) The operator delete takes a void* to memory that was allocated by operator new. It s a void* because you get that pointer after the destructor is called, which removes the object-ness from the piece of storage. The return type is void. 50

51 Overloading global new & delete (Cont ) #include <cstdio> #include <cstdlib> using namespace std; void* operator new(size_t sz) { printf("operator new: %d Bytes\n", sz); void* m = malloc(sz); if(!m) puts("out of memory"); return m; } void operator delete(void* m) { puts("operator delete"); free(m); } 51

52 Overloading global new & delete (Cont ) class S { int i[100]; public: S() { puts("s::s()"); } ~S() { puts("s::~s()"); } }; 52

53 Overloading global new & delete (Cont ) int main() { puts("creating & destroying an int"); int* p = new int(47); delete p; puts("creating & destroying an s"); S* s = new S; delete s; puts("creating & destroying S[3]"); S* sa = new S[3]; delete []sa; } 53

54 Overloading new & delete for arrays (Cont ) After p = new Fred[n], how does the compiler know there are n objects to be destructed during delete[ ] p? The run-time system stores the number of objects, n, somewhere where it can be retrieved if you only know the pointer, p. There are two popular techniques that do this. Both these techniques are in use by commercial-grade compilers, both have tradeoffs, and neither is perfect. 54

55 Overloading new & delete for arrays (Cont ) These techniques are: Over-allocate the array and put n just to the left of the first Fred object. Use an associative array with p as the key and n as the value. 55

56 Overloading new & delete for arrays (Cont ) Using "over-allocation" (header) to remember the number of elements in an allocated array. Each block, allocated or freed, contains a header. Usually the heap simply consists of two lists of cells; one is the list of allocated cells and the other the list of free cells. 56

57 57

58 Overloading new & delete for arrays (Cont ) p = new Fred[n] code: char* tmp = (char*)operator new[](wordsize + n*sizeof(fred)); Fred* p = (Fred*) (tmp + WORDSIZE); *(size_t*)tmp = n; size_t i; try { for (i = 0; i < n; ++i) new(p + i) Fred(); // Placement new } catch (...) { while (i--!= 0) (p + i)->~fred(); // Explicit call to the destructor operator delete[] ((char*)p - WORDSIZE); throw; 58 }

59 Overloading new & delete for arrays (Cont ) Then the delete[] p statement becomes: // Original code: delete[] p; size_t n = *(size_t*)((char*)p-wordsize); while (n--!= 0) (p + n)->~fred(); operator delete[] ((char*)p - WORDSIZE); 59

60 Overloading new & delete for arrays (Cont ) Note that the address passed to operator delete[] is not the same as p. If you make a programming error by saying delete p where you should have said delete[] p, the address that is passed to operator delete(void*) is not the address of any valid heap allocation. This will probably corrupt the heap. 60

61 Overloading new & delete for arrays (Cont ) Using an "associative array" to remember the number of elements in an allocated array In this technique, the code for p = new Fred[n] looks something like this (where arraylengthassociation is the imaginary name of a hidden, global associative array that maps from void* to "size_t"): 61

62 Overloading new & delete for arrays (Cont ) Fred* p=(fred*)operator new[] (n * sizeof(fred)); size_t i; try { for (i = 0; i < n; ++i) new(p + i) Fred(); // Placement new } catch (...) { while (i--!= 0) (p + i)->~fred(); // Explicit call to the destructor operator delete[] (p); throw; } arraylengthassociation.insert(p, n); 62

63 Overloading new & delete for arrays (Cont ) Then the delete[] p statement becomes: size_t n=arraylengthassociation.lookup(p); while (n--!= 0) (p + n)->~fred(); operator delete[] (p); If you make a programming error by saying delete p where you should have said delete[] p, only the first Fred in the array gets destructed, but the heap may survive (unless you've replaced operator delete[] with something that doesn't simply call operator delete, or unless the destructors for the other Fred objects were necessary). 63

64 Overloading new & delete for arrays (Cont ) The Over-Allocation technique is faster, but more sensitive to the problem of programmers saying delete p rather than delete[] p. The associative array technique is slower, but less sensitive to the problem of programmers saying delete p rather than delete[] p. 64

65 coalescence Objects allocated at the same time tend to die at the same time. On average, after 2.5Kb of allocation, 90% of all objects have both neighbors free It pays to wait a short time after an object is freed. 65

66 Constructor calls MyType* f = new MyType; Assume we call new to allocate a MyTypesized piece of storage. This call invokes the MyType constructor on that storage. What happens if all the safeguards fail and the value returned by operator new is zero? 66

67 Constructor calls (Cont ) The constructor is not called in that case, so although you still have an unsuccessfully created object, at least you haven t invoked the constructor and handed it a zero pointer. 67

68 Constructor calls (Cont ) void my_new_handler() { cout << "new handler called" << endl; } class NoMemory { public: NoMemory() { cout << "NoMemory::NoMemory()" << endl; } }; 68 void* operator new(size_t sz) throw(bad_alloc) { cout << "NoMemory::operator new" << endl; //throw bad_alloc(); // "Out of memory" return 0; }

69 Constructor calls (Cont ) int main() { set_new_handler(my_new_handler); NoMemory* pnm = new NoMemory; cout << pnm = " << pnm << endl; } return 0; 69

70 placement new Used to construct an object on a pre-allocated storage. The overloaded operator new can take more than one argument. The first argument is always the size of the object, which is secretly calculated and passed by the compiler. The other arguments can be anything you want: the address you want the object placed at, a reference to a memory allocation function or object, or anything else that is convenient for you. The way you pass the extra arguments to operator new during a call may seem slightly curious at first: You put the argument list ( without the size_t argument, which is handled by the compiler) after the keyword new and before the class name of the object you re creating. 70

71 placement new (Cont ) class X { int i; public: X(int ii = 0) : i(ii) {} ~X() { cout << "X::~X()" << endl; } void* operator new(size_t, void* loc) { return loc; } }; int main() { int vec[10]; example 1: } X* xp = new(vec) X(47); // X at location vec xp->x::~x(); // Explicit destructor call // ONLY use with placement! return 0; 71

72 placement new (Cont ) class Data { public: int x; }; example 2: mini memory pool void* pmempool = malloc(256); void* pnextalloc = pmempool ; Data* pdata = new (pnextalloc) Data(); pnextalloc = (char*)(pnextalloc) + sizeof(data); // Writing to it pdata->x = 10; // Destroying c pdata->~data(); pnextalloc = (char*)(pnextalloc) - sizeof(data); free(pmempool); pmempool=null; pnextalloc=null; 72

73 placement new (Cont ) There s only one version of operator delete, so there s no way to say, Use my special de-allocator for this object. You want to call the destructor, but you don t want the memory to be released by the dynamic memory mechanism because it wasn t allocated on the heap. You can explicitly call the destructor, as in xp->x::~x(); // Explicit destructor call You will have serious problems if you call the destructor this way for an object created on the stack because the destructor will be called again at the end of the scope. If you call the destructor this way for an object that was created on the heap, the destructor will execute, but the memory won t be released, which probably isn t what you want. 73

74 nothrow new and delete extern const nothrow_t nothrow; This constant value is used as an argument for operator new and operator new[] to indicate that these functions shall not throw an exception on failure, but return a null pointer instead. This version cannot be replaced by the user. int main () { cout << "Attempting to allocate 1 MB..."; char* p = new (nothrow) char [ ]; if (p==0) cout << "Failed!\n"; else { cout << "Success!\n"; delete[] p; } return 0; } 74

75 summary The standard-provided overloads of operator new (there are also corresponding ones for array new[ ]): void* ::operator new( std::size_t size) throw(std::bad_alloc); // usual plain old boring new // usage: new T void* ::operator new( std::size_t size, const std::nothrow_t&) throw(); // nothrow new // usage: new (std::nothrow) T void* ::operator new(std::size_t size, void* ptr) throw(); // in-place (or "put-it-there") new // usage: new (ptr) T 75