Advanced Programming & C++ Language ~6~ Introduction to Memory Management Ariel University 2018 Dr. Miri (Kopel) Ben-Nissan
Stack & Heap 2 The memory a program uses is typically divided into four different areas: The code area, where the compiled program sits in memory. The global (data) area, where global variables are stored. The heap, where dynamically allocated variables are allocated from. The stack, where parameters and local variables are allocated from.
3
The stack 4 The stack is a container that holds other variables. It is a last-in, first-out (LIFO) structure. It is a fixed-size chunk of sequential memory addresses. All the item are allocated in advanced and used according to the program s needs. A register (a small piece of memory) in the CPU is used as the stack pointer. The stack pointer keeps track of where the top of the stack currently is.
5 The stack handing function calls When a function is called: 1. Push return address: The address of the instruction beyond the function call is pushed onto the stack. 2. Return value: Room is made on the stack for the function s return type. This is just a placeholder for now. 3. Jump: The CPU jumps to the function s code. 4. Function s frame in stack: The current top of the stack is held in a special pointer called the stack frame. Everything added to the stack after this point is considered local to the function. 5. Passing arguments: All function arguments are placed on the stack. 6. Execute: The instructions inside of the function begin executing. 7. Local variables are pushed onto the stack as they are defined.
The stack handing function calls (cont ) 6 When the function terminates, the following steps happen: 1. Push return value: The function s return value is copied into the placeholder that was put on the stack for this purpose. 2. Clean local frame: Everything after the stack frame pointer is popped off. This destroys all local variables and arguments. 3. Use return value: The return value is popped off the stack and is assigned as the value of the function. If the value of the function isn t assigned to anything, no assignment takes place, and the value is lost. 4. Reconstruct current frame: The address of the next instruction to execute is popped off the stack, and the CPU resumes execution at that instruction.
The stack handing function calls (cont ) 7
The stack stack overflow 8 The stack has a limited size, and consequently can only hold a limited amount of information. Stack overflow is generally the result of allocating too many variables on the stack, and/or making too many nested function calls. Overflowing the stack generally causes the program to crash.
The stack (cont ) 9 The stack has advantages and disadvantages: Memory allocated on the stack stays in scope as long as it is on the stack. It is destroyed when it is popped off the stack. All memory allocated on the stack is known at compile time. Consequently, this memory can be accessed directly through a variable. The stack is relatively small, thus it is generally not a good idea to allocate large arrays, structures, and classes, as well as perform heavy recursion.
The heap 10 The heap (also known as the free store ) is a large pool of memory used for dynamic allocation. In C++, when you use the new operator to allocate memory, this memory is assigned from the heap. int *pvalue = new int; int *parray = new int[10]; Because the precise location of the memory allocated is not known in advance, the memory allocated has to be accessed indirectly which is why new returns a pointer.
11 The heap (cont ) When a dynamically allocated variable is deleted, the memory is returned to the heap and can then be reassigned as future allocation requests are received. The heap has advantages and disadvantages: Allocated memory stays allocated until it is specifically deallocated (beware memory leaks). Dynamically allocated memory must be accessed through a pointer. Because the heap is a big pool of memory, large arrays, structures, or classes should be allocated here.
The heap (cont ) 12
Memory Management 13 The stack is managed by the compiler. We can manage only the heap allocated memory. What does it mean managing memory? Allocate memory. Free memory. Manipulate addresses (pointers). Reuse of free memory.
Memory Management 14 Memory management is one of the most fundamental areas of computer programming. Knowing the abilities and limitations of your memory manager is critical for effective programming. Memory management is usually divided into three areas: Hardware Operating system Application
Memory Management (cont ) Win32 Applications Application Win32 APIs Operating System Virtual Memory Manager 15 Physical memory Hard disk(s) Hardware
Memory Management (cont ) 16 Hardware memory management Electronic devices that actually store data. This includes devices like RAM and memory caches. Operating system memory management Efficiency memory allocation and reuse for processes. Virtual Memory Memory protection, Memory sharing, Security, etc. Application memory management Efficient memory allocation techniques. Reuse of not used allocated memory. Consider CPU overhead.
1) Hardware Memory Management 17
1) Hardware Memory Management (cont ) 18 Computers have several different types of memory. This memory is often viewed as a hierarchy as shown below.
1) Hardware Memory Management (Cont ) 19 Cache is transparent to user; transfers occur automatically Word Line CPU Reg file Cache (fast) memory Main (slow) memory
1) Hardware Memory Management (Cont ) 20
21 1) Hardware Memory Management (Cont ) The cache memory boost s the speed of accessing memory, and managed entirely by the hardware. There are two types of addresses into the memory: logic address and physical address. Processes uses logical addresses. The mapping of logic address into the physical addresses is done inside the hardware, and programmed by the operating system. There are some types of Hardware Accelerators that makes those operations as well as math operations etc Miri much Ben-Nissan faster.
2) OS Memory Management 22
2) OS Memory Management (cont ) 23 Virtual Memory virtual memory is a technique in which the computer simulates having large amounts of contiguous physical memory. A paging table is responsible for moving segments of virtual memory into physical memory as necessary. If the amount of memory demanded by all running processes exceeds the available physical memory (RAM), the paging table stores low-priority processes on the hard drive in the page file, which is much slower than RAM.
2) OS Memory Management - Virtual Memory (cont ) 24 When a program is started (a web browser or a word processor), it runs in its own process. A process contains its own "virtual" memory space and resources. Its memory is "virtual" because memory the process thinks is at address 0x12345678 may actually be at address 0x65f7a678 in physical memory.
2) OS Memory Management - Virtual Memory (cont ) 25 The paging table moves segments of virtual memory into physical memory as needed to provide more memory to running processes.
2) OS Memory Management - Virtual Memory (cont ) 26
2) OS Memory Management - Virtual Memory (cont ) 27 The virtual memory is divided into two parts: user-space and system space. The user space is one global area in which windows runs the user-mode applications. Each user-mode application gets a block of virtual addresses. The system space is the portion of the address space in which the OS and kernel-mode drivers reside. It cannot be directly accessed by the application.
2) OS Memory Management - Virtual Memory (cont ) 2) OS Memory Management - Virtual Memory (cont ) 28 Virtual memory is useful for many reasons: The process cannot access other process' memory, Each page can have different protection settings (read-only or read-write, kernel-mode-only), and Inactive memory regions of the process can be "paged out" (stored) to the pagefile and be retrieved by the operating system when needed. This is also done when the system is low on physical memory.
29 3) Application Memory Management The user has the ability to manage its applications memory usage by himself. Data Structures. Memory Pools. Overloading dynamic memory functionality. Garbage Collections. Recycling.
3) Application Memory Management (Cont ) 30 When talking about application memory management, we usually deal with the following problems that the programmer should deal with: Premature free or dangling pointer. Memory leaks. External fragmentation. Poor locality of reference. Inflexible design. 3) Application Memory Management (Cont ) Interface complexity.
31 3) Application Memory Management (Cont ) 3) Application Memory Management (Cont ) Premature free / Dangling pointer: class Sample { int *ptr; public: Sample(int i) { ptr = new int(i); } ~Sample() { delete ptr; } void PrintVal() { cout<<"the value is "<<*ptr; } }; void SomeFunc(Sample x) { cout << "Say i am in somefunc " << endl; } int main() { Sample s1 = 10; SomeFunc(s1); s1.printval(); // dangling pointer }
3) Application Memory Management (Cont ) 3) Application Memory Management (Cont ) 32 Memory Leaks:
3) Application Memory Management (Cont ) 3) Application Memory Management (Cont ) 33 External Fragmentation:
34 3) Application Memory Management (Cont ) 3) Application Memory Management (Cont ) Poor locality of reference Also known as the principle of locality. The use of data elements within relatively close storage locations. The phenomenon that the collection of the data locations referenced in a short period of time in a running computer often consists of relatively well predictable clusters. for i in 0..n for j in 0..m for k in 0..p C[i][j] = C[i][j] + A[i][k] * B[k][j];
3) Application Memory Management (Cont ) 3) Application Memory Management (Cont ) 35 The larger matrix can be divided into evenly-sized submatrices, so that the smaller blocks can be referenced (multiplied) several times while in memory. for (ii = 0; ii < SIZE; ii += BLOCK_SIZE) for (kk = 0; kk < SIZE; kk += BLOCK_SIZE) for (jj = 0; jj < SIZE; jj += BLOCK_SIZE) for (i = ii; i < ii + BLOCK_SIZE && i < SIZE; i++) for (k = kk; k < kk + BLOCK_SIZE && k < SIZE; k++) for (j = jj; j < jj + BLOCK_SIZE && j <SIZE; j++) C[i][j] = C[i][j] + A[i][k] * B[k][j]; A block can be used several times before moving on, so that it is moved in and out of memory less often. In addition, elements with consecutive memory addresses tend to be pulled up the memory hierarchy together.
3) Application Memory Management (Cont ) 3) Application Memory Management (Cont ) 36 Inflexible design Memory managers can also cause severe performance problems if they have been designed with one use in mind, but are used in a different way. These problems occur because any memory management solution tends to make assumptions about the way in which the program is going to use memory, such as typical block sizes, reference patterns, or lifetimes of objects. If these assumptions are wrong, then the memory manager may spend a lot more time doing bookkeeping work to keep up with what's happening.
37 3) Application Memory Management (Cont ) Interface complexity When one module uses one memory management toolkit and another module uses a different one, it is complicated to pass arguments between those modules.
Managed and Unmanaged Systems 38 Memory management can be automatically managed by the programming language, or either manually managed by the user.
Manual memory management 39 The programmer has direct control over when memory may be recycled. Usually this is either by explicit calls to heap management functions (for example, malloc/free in C), or by language constructs that affect the stack (such as local variables).
Manual memory management (Cont ) 40 The advantages of manual memory management are: It can be easier for the programmer to understand exactly what is going on; Works well in simple situations; Can be ideal for small, constrained problems, especially where the total memory required is close to the total memory available.
Manual memory management (Cont ) 41 The disadvantages of manual memory management are: The programmer must write a lot of code to do repetitive bookkeeping of memory; Memory management must form a significant part of any module interface; Manual memory management typically requires more memory overhead per object; Memory management bugs are common.
3) Application Memory Management (Cont ) Manual memory management (Cont ) 42 Bugs in memory management implementations: Grows over time Out of memory GPF / Bus Error Data Corruption Performance (Thrashing)
Manual memory management (Cont ) 43 Causes of those bugs are usually: Premature FREE (Dangling pointer). Memory leaks. Fragmentation. Poor locality. Solutions Debugging Application (such as Purify). Garbage Collection (Automatic Memory Management).
Manual memory management (Cont ) 44 The following languages use mainly manual memory management in most implementations, although many have conservative garbage collection extensions: Algol; C; C++; COBOL; Fortran; Pascal.
45 Automatic memory management A service, either as a part of the language or as an extension, that automatically recycles memory that a program would not otherwise use again. Automatic memory managers often known as garbage collectors, or simply collectors. Usually do their job by recycling blocks that are unreachable from the program variables (that is, blocks that cannot be reached by following pointers).
Automatic memory management (Cont ) 46 There are two aspects to Automatic Memory Management: Allocation Subdividing the large blocks that the memory manager receives from the operating system into blocks suitable to the application. Static/Dynamic. Recycling Physical memory is a limited resource Virtual memory is a trade-off Object-oriented Languages and Dynamic Languages.
Automatic memory management (Cont ) 47 The advantages of automatic memory management are: The programmer is freed to work on the actual problem; Module interfaces are cleaner; There are fewer memory management bugs; Memory management is often more efficient.
Automatic memory management (Cont ) 48 The disadvantages of automatic memory management are: Memory may be retained because it is reachable, but won't be used again; Automatic memory managers (currently) have limited availability. When a garbage collector bug does arise, since it is a result of a failure of a runtime system, rather than the application programmer s own code, it can be difficult to diagnose and repair.
Automatic memory management (Cont ) 49 Most modern languages use mainly automatic memory management: BASIC, Java TM, JavaScript TM, Perl, the PostScript language, Python, Smalltalk, etc.
Automatic memory management (Cont ) 50 The user has the ability to control the usage of its application s memory even in managed (automatic) systems. One example is the Alignment of data.
3) Application Memory Management (Cont ) Alignment 51 What is the size of the following struct? struct MixedData { char Data1; short Data2; int Data3; char Data4; };
3) Application Memory Management (Cont ) Alignment (cont ) 52 How programmers see memory: How processors see memory:
3) Application Memory Management (Cont ) Alignment (cont ) 53 Data structure alignment is the way data is arranged and accessed in computer memory. It consists of two separate but related issues: data alignment and data structure padding. Data alignment means putting the data at a memory offset equal to some multiple of the word size, which increases the system's performance due to the way the CPU handles memory.
3) Application Memory Management (Cont ) Alignment (cont ) 54 To align the data, it may be necessary to insert some meaningless bytes between the end of the last data structure and the start of the next, which is data structure padding. If the highest and lowest bytes in a datum are not within the same memory word the computer must split the datum access into multiple memory accesses. Additionally the size of the structure must be such that in an array of the structures all the structures are correctly aligned in memory so there may be padding bytes at the end of the structure too.
Alignment (cont ) 55
Alignment (cont ) 56
Alignment (cont ) 57 The MixedData struct, after compilation in 32-bit x86 machine: struct MixedData { char Data1; short Data2; int Data3; char Data4; };
Alignment (cont ) 58 The sizeof a struct is compiler and processor dependent for reasons of both architectural limitations and efficiency. Some processors are unable to read or write multibyte values from certain locations. Almost always they cannot read them from odd addresses. Frequently they cannot read them unless the address is a multiple of the size of the value - so a four-byte long can only be read if the address is a multiple of four, and a two-byte short can only be read if the address is a multiple of two. Single-byte values can be addressed at any location.
Alignment (cont ) 59 Another example: struct example { char c1; //[c1][p] short s1; //[s1][s1] char c2; //[c2][p][p][p] long l1; //[l1][l1][l1][l1] char c3; //[c3][p][p][p] };