DMMU: Dynamic Memory Management Unit

Transcription

1 DMMU: Dynamic Memory Management Unit J. Morris Chang, Witawas Srisa-an and Chia-Tien Dan Lo Department of Computer Science Illinois Institute of Technology Chicago, IL, , USA (chang sriswit Abstract Dynamic memory management has been a performance bottleneck in many operating systems including multithreaded and real-time operating systems. Moreover, recent advances in software engineering, such as graphical user interface and object-oriented programming, have caused applications to become more memory intensive. The growing popularity of these operating systems and applications increases the importance of high-performance dynamic memory management. Currently, the heap management is done through software routines (e.g. malloc and free in C/C++) and handled in a per process base. This paper presents a hardware implementation of DMMU which will be shared by all the processes in the same way that the TLBs (Translation Look-aside Buffers) used in current MMU. This DMMU employs a novel approach to implement an efficient memory management. Allocation and deallocation are performed in hardware domain. The detailed design and evaluation of the proposed DMMU are presented in this paper. Simulation results show that the hit ratio for 1 Kbits and 8 Kbits buffer are ranging from 78-99% and 95-99%, respectively. At the same time, the memory overhead for 1 Kbits is 10% and for the 8 Kbits is 0%. Index Terms Architectural support for operating system, dynamic memory management, bit-map allocation/ deallocation, object-oriented systems, VLSI implementation. 1. Introduction Dynamic memory management has been a performance bottleneck in many operating systems including multi-threaded and real-time operating systems. In multi-threaded system, threads of the same process share and access a common heap space concurrently. While this alleviated the context switching overhead, the allocation and deallocation within that heap space can be very intensive if there are several threads existing in the same process. In real time operating system, meeting the deadline is the utmost important goal. However, functions such as malloc and free yield non-deterministic turnaround time which can cause real-time systems to fail. 1

2 Moreover, recent advances in software engineering, such as graphical user interface and object-oriented programming, have caused applications to become more memory intensive. The growing popularity of these operating systems and applications increases the importance of high-performance dynamic memory management. This document presents a design of Dynamic Memory Management Unit (DMMU) to facilitate the memory management. The memory-management unit (MMU), a hardware device, can be found in almost every modern CPU. The MMU has been used to map virtual address to physical address for more than two decades. This paper presents an extension to the traditional MMU to include hardware support for heap management. The rationale behind this approach is two folds: 1. The functions for dynamic memory allocation and deallocation are one of the most time consuming functions in many applications today. The object-oriented nature of C++ applications results in much more intensive memory usage than in C applications. In particular, a much larger proportion of total memory usage is taken in allocations on the heap compared to a procedural language, where there is greater use of the stack and static data. Moreover, [18] finds that in six C/C++ programs, 23% - 38% of runtime is spent on dynamic memory allocation and deallocation. Our studies in using GNU s gprof to profile the invocation of malloc and free functions in several programs also confirm with Zorn s findings. Moreover, JAVA applications are more dynamic memory intensive. For example, to allocate an array of class object, we need to dynamically allocate each array element. Additionally, operating systems such as multi-threaded or real-time, also generate higher memory intensity. These illustrate the need for a high-performance dynamic memory allocator and deallocator. 2. As the VLSI technology advances, it becomes more and more attractive to map basic software algorithms into hardware. Recent examples are 57 new instructions for multimedia extension (MMX) in Intel s Pentium and 21 new instructions for 3D graphic and multimedia in AMD s K6-2. By 2001, it is expected to see a 100-milliontransistor CPU which is ten times more complex than Pentium II [19]. Currently, the heap management is done through software routines (e.g. malloc and free in C/C++) and handled in a per process base. This paper presents a hardware implementation of DMMU which will be shared by all the processes in the same way that the TLBs (Translation Look-aside Buffers) used in current MMU. This DMMU employs a novel approach to implement an efficient memory management. The binary buddy system, known for 2

3 speed and simplicity in memory allocation, has been in use for nearly three decades. A software realization incurs the overhead of internal fragmentation and of memory traffic due to splitting and coalescing memory blocks. This paper employs a simple hardware design for buddy-system allocation with a size-encoding scheme that takes full advantage of the speed of a pure combinational-logic implementation[1]. This allows dynamic allocation and deallocation to be done in constant time. Moreover, locality will also be improved, since the allocation and deallocation are done by a dedicated hardware unit. This modified buddy system can effectively eliminate internal fragmentation. Simulation results show that the memory utilization can be 25% to 33% better than the conventional buddy system and is very close (within 10%) to the first-fit approach [1]. The hardware complexity of the proposed scheme is O(n), where n is the size of the bit-vector. Recently, much attention has been paid to research in the field of Processor In Memory (PIM) such as David Patterson s IRAM project [14]. The research is intended to close the performance gap between processor and memory. An intelligent memory system should include high-performance strategies to manage memory dynamically. This DMMU will fit into PIM paradigms very nicely. Moreover, the proposed design can also benefit any application that requires high-performance resource allocation. Such applications include IIT s JAVA chip[5], real-time systems, and embedded systems. The remainder of this paper is organized as follows. Section 2 elaborates on the previous work. Section 3 provides the design of the DMMU and the architectural support for the DMMU. Section 4 provides information on operating system support. Section 5 presents the performance evaluation of the DMMU. The last section presents the conclusions of this paper. 2. Previous work Dynamic memory management has been an important topic in computer systems for over three decades. High-performance algorithms for dynamic memory management are of considerable interest 1. The primary goals of these algorithms are to speed up the dynamic memory allocation and deallocation as well as to minimize internal and external fragmentation. Each of the popular algorithms has its drawbacks. The first fit and next fit algorithms achieve good storage utilization but incur a time penalty associated with scanning a free list [3]. The buddy system, 1. Recently, a very extensive survey (78 pages long) was presented by Paul Wilson [17]. 3

4 which allocates memory in blocks whose lengths are powers of 2, is known for its speed and simplicity, but suffers from internal fragmentation [17]. Modern operating systems employ these strategies with several optimizations in the implementation of dynamic memory management. For example, a Cartesian-tree, better fit algorithm is used in the Sun Operating System; a fast buddy algorithm can be found in the BSD Unix implementation of malloc; a hybrid first-fit, buddy algorithm is used in the GNU publicly available malloc/free implementation[15]. In the modern programming languages, dynamic memory management is part of the runtime system to manage the heap memory. It is typically implemented through function calls (e.g. malloc() and free() in C language). The malloc API specified by ANSI is a general-purpose search for an arbitrarily sized free area of memory. Compilers generally implement C++ operator new directly in terms of malloc. Applications written in C and (especially) C++ spend much of their execution time in allocating and deallocating memory. The increasing popularity of C++ also magnifies the importance of efficient heap management. The inefficiency (in terms of time) of malloc and free is due to the search through a list of free blocks. The current approach in dealing with dynamic memory management requires that each process manages its own heap space, which exists in virtual memory domain, on a per process basis. Each process calls its own malloc and free functions and the allocation information is usually kept on linked lists. Once the process heap space is exhausted, operating system s function call such as sbrk() or brk() is made to request for additional memory [16]. The major drawback of this approach is in the allocation information data structure. By using linked list, such as in sequential fit or segregated fit, the search is done sequentially and the number of memory blocks on the available lists can greatly affect the turnaround time. As the memory becomes more scattered, the search time also grows linearly longer. It is worth noting that heap management algorithms have been around long before the popularity of objectoriented programming. In Pascal, memory is allocated and deallocated by new and dispose operators. In C, the allocation and deallocation are done through malloc and free. Even in C++, the new and delete functions are mapped to malloc and free functions from C. This implicates that while the programming trend is moving toward objectoriented approach, dynamic memory management functions still operate based on the functions designed for the procedural approach which yield much less memory intensity than the object-oriented approach. Apparently, new 4

5 approaches to manage dynamic memory are needed in order to cope with the increasing in memory intensity of the heap region. If storage is allocated in contiguous sequences of fixed-sized blocks, allocation information may be held in a separate bit-map [15]. A bit-map approach provides a great opportunity to implement a hardware version of the buddy system [7]. It is worth noting that the proposed hardware approach is rather simple and yet does not have the difficult issues, splitting and coalescing, which does exist in its software counterpart. In this section, the technical rationale and approach are presented from the performance perspectives. Improvement in speed. When a block of a given size is to be allocated, the buddy system locates a block that is at least as large as the allocation request, and whose size is a power of two. The block is split in half as many times as possible, until it can be split no longer yet still satisfy the memory request. When a block is split, its two halves are known as buddies. At the time a block is freed, if the block s buddy is also free, the buddy system coalesces the two buddies immediately, making available a larger piece of memory that can be used later for allocation. The operations of splitting and coalescing memory dominate the cost of the buddy system in its software realizations. This observation has led to research in reducing the frequency of coalescing [10]. A hardware-maintained bit-map approach; however, eliminates the need for splitting and coalescing operations. Splitting becomes unnecessary because allocation is done using a hardware-maintained binary tree that finds free blocks using combinational logic. The bit-vector forms the base of the binary tree. Deallocation is indicated by resetting bits in the bit-vector, eliminating the need for explicit coalescing. In the software approach, a free list for blocks of each size is maintained; thus, after blocks are coalesced, the pointers to two free lists need be updated. The hardware approach, however, requires no coalescing at all. The freed bits are, in effect, combined immediately with adjacent bits [2]. Because the proposed hardware can be realized in pure combinational logic, the time needed for memory management is greatly reduced. Improvement in memory utilization. Binary buddy systems always allocate memory in sizes of powers of 2, so they may leave much space unused at the end of an allocated block. This is known as internal fragmentation. Bitvector allocation raises the possibility of using the otherwise unused portion of the block later, without increasing the allocation/deallocation time. For example, if the buddy system allocates 8 blocks for a 5-block request, the hardware 5

6 can mark exactly 5 of the 8 bits (one bit per block). This allows using the unused 3 blocks in the future. We will call this the modified buddy system. Again, such bit-manipulation is quite easy in a hardware realization. Recalling the day when multiprogramming operating system was first introduced, the virtual memory space was managed by software algorithms. Later on, the hardware implemented memory management unit (MMU) became the first choice in memory management for most of the CPUs available today. This document presents a novel attempt to create a dynamic memory management unit (DMMU) that can collaborate with operating systems in managing heap space. As the trend moves toward object-oriented programming, multi-threaded operating system, and real-time operating system, the demand for high performance dynamic memory management becomes more intense. It is time to look for alternatives to current software approaches that can respond to the needs of today s computing. The DMMU can be a strong candidate to accomplish such goal because it provides improvement in speed, improvement in memory utilization, and constant turn around time which responds to the needs of today s computing. 3. Design of Dynamic Memory Management Unit (DMMU) This section is separated into two sub-sections. First sub-section provides an overview picture of the DMMU, while the second sub-section provides architectural support for the DMMU. 3.1 Top level design The main purpose of the Dynamic Memory Management Unit (DMMU) is to take responsibility for managing heap space for all processes in the hardware domain. The proposed DMMU utilizes the modified buddy system combined with bit-map approach to store allocation and size information. Usually, each process has a heap associated with it. In the proposed scheme, each heap requires two bit-maps, one for allocation status (A bit-map) and one for object size (S bit-map). It is necessary to place these two bit-maps together all the time, since searching and modification to these two bit-maps are required for each allocation and deallocation. The communication between the CPU and the DMMU is done through extended instruction set designed specifically for the DMMU. This instruction set would allow the CPU to make the allocation or deallocation request and pass on the requested size or starting address information to the DMMU. Figure 1 demonstrates the top-level diagram of the DMMU. 6

7 Figure 1. The top-level description of a DMMU A bit-map S bit-map from different processes Allocation / Deallocation CPU Object Size DMMU sbrk/brk Object pointer O.S. Kernel Figure 1 illustrated the basic functionality of the DMMU. First of all, the DMMU provides service to the CPU by maintaining the memory allocation status inside the heap region of the running process. Thus, the DMMU must have access to the A bit-map and S bit-map of the running process. Similar to the TLB, DMMU is shared among all processes. The parameters that the CPU can pass to the DMMU are: the Allocation/Deallocation signal, the Object_size (for the allocation request), and the Object_pointer. The operation of DMMU is very similar to the function calls (i.e. malloc() and free()) in C language. Thus, object_pointer is either returned from DMMU in allocation or passed on to DMMU during deallocation process. If the allocation should failed, the DMMU would make a request to the operating system for additional memory using system call sbrk() or brk(). Since the algorithms used in DMMU is implemented through pure combinational logic, the time to perform a memory request or a free instruction is constant. On the other hand, the time for a software approach in performing a malloc() or a free() function is non-deterministic. By and large, the practical programs spend more than 20% of execution times in dealing with dynamic memory management [18]. This extensive execution time can be eliminated by using DMMU. The DMMU can be implemented as a co-processor or integrated into the CPU. 3.2 Architectural support for DMMU This section summarizes the process of memory allocation and deallocation in the DMMU. Since the bit-map of a given process may be too large to be handled in the hardware domain, the bit_vector, a small segment of the bitmap, is used in the proposed system. This idea is very similar to the idea of using TLB (Translation Look-aside Buffer) in the virtual memory. Due to the close tie between S bit-map and A bit-map, the term of bit-vector used in this section represents one A bit-vector (of A bit-map) and one S bit-vector (of S bit-map). Figure 2 presents the operation of the proposed DMMU. 7

8 When a memory allocation request is received (step 1), the requested size is compared against the largest_available_size of each bit-vector in a parallel fashion. This operation is similar to the tag comparison in a fully associated cache. However, it is not an equality comparison. There is a hit in DMMU, as long so one of the largest_available_size is greater or equal to the request size. If there were a hit, the corresponding bit-vector is read out (step 2) and sent to the CBT (Complete Binary Tree). The CBT [1] is a hardware unit to perform allocation/ deallocation on bit-vector. For the purpose of illustration, we assume that one bit-vector represents one page of the heap. After the CBT identified the free chuck memory from the chosen page, CBT will update the bit-vector (step 3) and the largest_available_size field (step 3*). The object pointer (in terms of page offset address) of the newly created object is generated by CBT (step 4). This page offset combines the page number (from step 2*) into the resultant address. Figure 2.The allocation and deallocation processes of the DMMU starting address to be freed page number page offset page number bit-vectors largest_available_size A 1 page number page offset starting address of new object Allocation steps: 1 free memory space request * bit-vector read out, bit-vector update A* 3 * update largest_available_size 4 offset address of newly created object 4 2 * page # read out 3 C CBT A B 2 B Deallocation steps: select bit-vector A * starting page-offset address bit-vector read out C bit-vector update C * update largest_available_size 3 * C * For the deallocation, when the DMMU receives a deallocation request, the page number of the object pointer (i.e. a virtual address) is used to select a bit-vector (step A). This process is similar to the tag comparison in cache operation. At the same time, the page offset is sent to CBT as the starting address to be freed (step A*). The corresponding bit-vector is read out (step B) and sent to CBT. The CBT will free the designated number of blocks 8

9 (based on the information from S bit-vector) starting at the address provided by the step A* and update the bit-vector (step C) and the largest available size field (step C*). The page number, bit-vectors, and the largest_available_size are placed in a buffer, called Allocation Look-aside Buffer (ALB). Since the DMMU is shared among all processes, the content of ALB will be swapping during the context switching. This issue also exists in TLB. To solve this problem, we can add a process-id field in ALB. This will allow bit-vectors of different processes to coexist in the ALB. We expect the performance of the ALB to be very similar to the much studied TLB. However, further research in the ALB organization, hit ratio and miss penalty is required. In order for the CPU to communicate with the DMMU, an extended instruction set is needed to take advantage of these new features. These instructions (i.e. MALC, MFRE, etc.) allow the CPU to communicate with the DMMU. For example, if the CPU executing the MALC command, the requesting memory size is sent to the DMMU with the allocation signal. In return, the DMMU would send the starting address back to the CPU. In case of failure, the DMMU would return a null pointer back to the CPU. Similarly, when the CPU executes the MFRE command, it simply issues the starting address of the memory to be freed and send out the deallocation signal to the DMMU. 4. Operating system support for DMMU The two main functions of operating system are resource management and providing an interface between bare machines and users. When the user creates a job, the operating system must provide an environment for the job to execute until it finishes. From the user s point of view, how the operating system implements system calls such as malloc or free is irrelevant. By adding a new resource, DMMU, some parts of operating system need to be modified. 4.1 Compiler Like the multimedia extension (MMX) instructions or three-dimension (3D) instruction sets, a modified compiler is required to generate object code that can take advantage of these new instruction sets. Once the compiler recognizes malloc() or free() in a program, it will generate code using DMM instruction set instead of linking malloc() or free() code from the current software library. Some of the DMMU instruction sets are user-level such as MALC and MFRE whereas most of them are supervisor-level such as ALB management instructions. The supervisorlevel instruction sets can only be used by operating system. Since the DMMU is a resource that shares among all processes, it is necessary to prevent some user processes to modify the content of the DMMU. The only job that the 9

10 modified compiler does is to generate code incorporating the DMMU instruction sets. Hence, the executable code can fully utilize the benefits of the DMMU. After compiling a program, the structure of a executable object code is: code segment, data segment, heap segment, and stack segment. Figure 3 shows a typical memory map for a process in a computer system. The process control block (PCB) stores all the information for the process to be executed such as process status, context switch data, and all the bit-vectors needed for a program. The program code is stored in the code segment. Global variables are store in the code segment. Dynamic variables requesting from malloc or realloc functions are stored in the heap segment. The stack segment is used for auto or local variables. Once the executable code is loaded into memory, the size of every segment is determined. Some operating systems may have only one segment for heap segment and stack segment; however, the combined size is still fixed after being loaded into memory. On the other hand, the maximum number of bit-vectors to represent a heap segment can be obtained during compiled time. Consequently, for storing these bit-vectors, the modified compiler has to reserve memory space which can be in PCB. Figure 3. A typical memory map for a process in a computer system Auto Variables Current Top of Stack Dynamic Memory Area Global Variables Program Process Control Information STACK HEAP DATA CODE PCB DMMU From Figure 3, it is worth noting that memory is allocated from the heap region which is above the data segment and can grow to accommodate the memory allocation requests. The address of the first available location address beyond the data segment is called the break address. A process can manipulate its break address by invoking the two system calls, brk() and sbrk(). The library functions such as malloc(), free(), and realloc() are constructed using either brk() or sbrk() system calls. By introducing DMMU, some signals need to be generated in order to invoke brk() or sbrk() when necessary. 4.2 Context-switch In a multi-process operating system, several processes can be executed concurrently. A CPU scheduler decides the states of processes which are ready, running or idle. When a process is scheduled to be in running state, it 10

11 can get control of CPU and the current running process will be swapped to be ready or idle states. To swap these two process, it requires saving the CPU status of the current process and loading the saved CPU status of the new process. This is so-called context-switch. Generally speaking, there are two context switching mechanisms in modern operating systems: segmentation and paging. Segmentation uses a variable length of memory chunk whereas paging uses many small fixed size of memory chunks for a program. In this proposal, only operating systems with paging will be discussed. Context switching has become an overhead because of the time it takes to switch two processes. One solution to alleviate this cost is introducing threads which reduce context-switching frequency. However, the bit-vectors in DMMU also need to be replaced during context-switch. This overhead can not be avoid but can be reduced by introducing ALB which can be multiple sets of registers. If the architecture of the ALB allows multiple sets of process to be kept at one time, during the context-switch, the only change is simply modifying the content of a register which points to the next scheduled process information in the buffer. On the other hand, if the ALB includes process-id (PID), then the PID can be used to compared with the current process PID to identified which bit-vector is valid for current process. There is another case that DMMU fails to allocate a memory request by looking all the bit-vectors in the ALB. This would happen if there are no space in the heap space or other bit-vectors need to be brought into ALB. Next section discusses this issue in greater detail. 4.3 DMMU service routines In order to execute a program, the huge virtual memory space must be mapped into small limited physical memory space. This is the reason why some of the memory contents must be swapped between memory and disks. The job of the DMMU is to keep track of free memory blocks within the process heap segment. Some bit-vectors are used to represent memory allocation status corresponding to the heap space. Inside the ALB, only a limited number of bit-vectors can be held. For a smaller heap space, it is possible to have all the bit-vectors for a process residing in the ALB. However, for a larger heap space, it may not be practical to hold all bit-vectors inside the ALB. In this case, a replacement policy must be selected and swapping mechanism similar to cache must be utilized. Figure 4 demonstrates this strategy. 11

12 Figure 4. DMMU structure for 24-page heap space Heap... Memory structure P0 P23 page # 0 to page # 23 (P0...P23) S & B bit-vectors for P0 S & B bit-vectors for P1 S & B bit-vectors for P2 S & B bit-vectors for P3 S & B bit-vectors for P4 Allocation Look-aside Buffer (5 entries) In Figure 4, the DMMU can only manage 5 bit-vectors at one time. If we need to allocate memory in P5, an approach similar to cache must be used to replace the bit-vectors inside the ALB. Our recent study on the allocation patterns[4] of C++ programs shows a good spatial locality. This is due to most objects have a short life span in objectoriented systems [7]. This suggests that a pseudo least-recently-used (LRU) or first-in-first-out (FIFO) replacement policies may work well with proposed ALB. Moreover, a prefetching-on-miss strategy may be used to optimize the ALB performance. The management of ALB, including the implementation of the replacement policy, will be done by a set of supervisor-level ALB management instructions. These instructions are very similar to the one used to manage the TLB in PowerPC [12]. If the allocation should fail or just a miss in the ALB, the DMMU service routines will make the sbrk()/brk() to the kernel or apply ALB management instructions to manipulate ALB. 4.4 Multi-process system versus multi-thread system In a multi-process system, the proposed ALB is to bit-maps what the TLB is to page table. During the contextswitch, there is overhead associated with maintaining the TLB. Similarly, ALB also adds cost to the context-switch. However, context-switch cost may be reduced through two proposed strategies: 1) adding a process-id field to the ALB or 2) using a multiple set of bit-vectors; each set is pointed by a process-id. The first approach has been used in TLB. The second approach leads to a possibility of loading the bit-vectors for the next scheduled process while system is in non-cpu cycle. The optimal set size will be investigated through simulation. On the other hand, the emergence of the multi-thread system may drastically reduce the cost for context switching. In multi-threaded system, the content of ALB might need to be updated during the process switching. While it is true that threads, in many aspects, operate in the same manner as processes, they do not have their own heap space. In multi-threaded system, threads within a process share a common heap space; therefore, the content of ALB should 12

13 only be recorded and updated during the process switching. That is no context-switch cost for switching from one thread to another within a process. Consequently, the DMMU can operate very well in a multi-threaded system. 5. Simulation Results This section presents detailed simulation results of the proposed DMMU. The simulator accepts memory allocation and deallocation traces as inputs and provides hit ratio as the result. The memory allocation / deallocation traces are obtained by instrumenting the malloc and free functions of the source programs. In the following subsections, the characteristics of the programs we traced and the performance evaluation of DMMU are summarized. 5.1 Application overview: The four programs traced in our experiment are publicly available software applications. These programs include gcc (C++ compiler), Xsnow, Xearth and Xspim (MIPS R2000/R3000 simulator). The first application is GNU product. Xearth (written by Kirk Lauritz Johnson) and Xsnow (written by Rick Jansen) are freeware. Xspim is developed by James Larus and used widely in academia [13]. They represent several different applications. Gcc is a CPU intensive application with no screen interaction. Xsnow and Xearth are X-window applications without screen interaction. Xspim is also an X-window application and has screen interaction through pull down menu and text input. Table 1 summarizes the traces characteristics of these four programs. The gcc was used to compile the Xspim. The Xspim was use to run an assembly program of recursive Ackerman s function. The numbers of malloc and free invocation are ranging from 2500 to 6400 and 1100 to 7300, respectively. The average object size of memory allocation request for each programs is ranging from 94 bytes to 1581 bytes. This shows that our experiment covers a good variety in allocation patterns. Table 1 Characteristics of program traces Name Malloc calls Free calls Avg. Malloc size (bytes) Total Memory Requested (bytes) GCC ,987,707 XSPIM ,238 XEARTH ,567 XSNOW ,203,586 13

14 Figure 5 depicts the histogram of object size for each application. For Xspim and Xearth, small objects are invoked very frequently. In Xsnow, object sizes are distributed quiet evenly. However, they are in the order of 2 n. It is worth noting that gcc invoked more than 900 objects in the size of 4K bytes (i.e. one page worth memory). This is due to the gcc maintained its own free list for certain objects. The burdensome overhead of malloc and free is a well known issue among experienced programmers. The most common way to lower the penalty is to make less frequent calls to malloc. Thus, programmers tend to request a large chunk of memory once, then keep track of their own free list. It may need to request another large chunk of memory if the current one has run out the space. This scheme is used in gcc and the chunk size is 4K bytes. Figure 5. Histogram of object size. Histogram (GCC) Histogram (XSPIM) Frequency Frequency Size (Bytes) Size (Bytes) Histogram (XEARTH) Histogram (XSNOW) Frequency Frequency Size (Bytes) Size (bytes) As we mentioned earlier, the large objects will be handled separately. This is a common practice in many other schemes [9]. In this research, the objects with size greater than 8K bytes are defined as large objects. In the four programs we traced, there are only 43 objects that are large objects. 5.2 Investigating block size Before we can evaluate the system performance, the first parameter need be studied is the block size. Again, in the bit-map, one bit stands for one block worth memory. The block size affects the bit-map size for a give heap memory size. The larger block size will yield a smaller bit-map size. A smaller bit-map size means a lower cost for 14

15 the bit-map. However, the larger block size may lead to higher internal fragmentation during the allocation. The higher internal fragmentation can contribute to a higher watermark (i.e. the highest memory address allocated). Apparently, the higher watermark is considered as the memory overhead in the proposed scheme. Next Table summarizes the memory overhead (through watermark) with block size ranging from 8 bytes to 64 bytes. The smallest block size, 4 bytes for one block, is used as the benchmark. Table 2 Memory overhead as compared to 4 bytes/block Bytes/Block GCC(%) XSPIM(%) XEARTH(%) XSNOW(%) From the table above, 16 bytes/block is the most logical block size. When compare block size of 16 and 8, the overhead in block size of 16 is minimal (5.88%). However, the overall size of bit-map would be reduced by 50% compared to block size of 8. Thus, we will use 16 bytes/block throughout the subsequence simulations. 5.3 Investigating replacement policy Similar to cache, the replacement policy can be a determining factor in the performance of the ALB. The three most common replacement policies, FIFO, Random, and LRU are investigated in the simulation. The two basic buffer configurations used in the simulation are 4 entries x 512 bits and 4 entries x 1Kbits. Figure 6 demonstrates the performance of the buffers with different replacement policies. Figure 6. Performance of the buffers with different replacement policies Comparison between Replacement Policies (4 entries x 512 bits) Comparison between Replacement Policies (4 entries x 1 Kbits) Hit Ratio Hit Ratio GCC XSPIM XEARTH XSNOW Applications 0.9 GCC XSPIM XEARTH XSNOW Applications FIFO RANDOM LRU FIFO RANDOM LRU 15

16 The performances between different policies do not differ much. A closer look reveals that in most instances, FIFO performs a little better than the other policies. The reason why FIFO performs better is because of the object life span may vary (young object dies young while old object continues to live [20]). FIFO guarantees the bit-vector that contains the oldest objects will always be replaced. Thus FIFO will be used as the replacement policy in our simulations. 5.4 Performance Evaluation We investigate the performance of ALB through two approaches. First, we fix the size of the Bit-Vector Length (BVL) and increase the number of entries (this also increases the buffer size). In doing so, we can find a good saturation point where the hit ratio of all or most of the programs begin to stabilize. The result is illustrated in Figure 7. Figure 7. Buffer size VS. Hit ratio Buffer Size VS. Hit Ratio (Fixed BVL at 512 bits) Buffer Size VS. Hit Ratio (Fixed BVL at 1K bits) Hit Ratio # of entries Hit Ratio # of entries GCC XSPIM XEARTH XSNOW GCC XSPIM XEARTH XSNOW Case a Case b In three of the four programs, the good saturation point for case a is at 16 entries. This translates to the buffer size of 8Kbits. For case b, the saturation point is at 8 entries. This also translate to 8Kbits buffer size. In the second approach, we set the buffer size to the value provided by the first approach (in this case 8K bits). Then, we would investigate the effect of buffer configuration (number of entries x BVL) on the hit ratio. Since the buffer size is set to 8K blocks, we can have the following configuration, 2x4K, 4x2K, 8x1K, and 16x0.5K. The result is demonstrated in Figure 8. 16

17 Figure 8. Buffer size VS. Hit ratio (different configurations). Buffer Size VS. Hit Ratio (Fixed Buffer Size at 8K bits) Hit Ratio x4K 4x2K 8x1K 16x0.5K Configurations GCC XSPIM XEARTH XSNOW From the above Figure, hit ratio decreases as the BVL decreases. This phenomenon is similar to caches (i.e. larger cache line size may lead to a higher hit ratio). It is worth noting that hit ratio varies more with gcc which has a larger average object size. This is due to the higher miss probability occurs in a smaller BVL with many objects that are relative large. Obviously, the configuration the allows longer BVL with less entries (2x4K) has the best hit ratio. On the other hand, the configuration that allows more entries with shorter BVL (16x0.5K) would also have smaller miss penalty (i.e. less data need be moved to buffer for each miss). Similar to the cache design, trade-off between lower miss penalty and higher hit ratio must be made by the system architect. 5.5 Investigating Memory overhead with different configuration It is worth noting that the changes in BVL not only affect the hit ratio but the watermark. This is because the proposed scheme does not allocate objects across the boundaries of the bit-vectors. This can lead to higher external fragmentation. Thus, a higher watermark is reached. Table 3 illustrates the effect of memory overhead with BVL. Fortunately, the BVL does not change memory overhead significantly. Programs Buffer size = (1x watermark) Watermark Table 3 BVL VS. Memory overhead Buffer size= (2x4K) Memory overhead(%) Buffer size= (4x2K) Memory overhead(%) Buffer size= (8x1K) Memory overhead(%) Buffer size= (16x0.5K) Memory overhead(%) GCC 108, XSNOW 26, XSPIM 12, XEARTH 9,

18 Another factor that may change the memory overhead is buffer size. Different buffer size may yield to different allocation address. Thus, this would affect watermark. Table 4 shows the simulation results for two buffer sizes: 2x0.5K and 32x0.5K. Programs Buffer size = (1x watermark) Watermark Table 4 Buffer size VS. Memory overhead Buffer size= (2x0.5K) Watermark Memory overhead(%) Buffer size= (32x0.5K) Watermark Memory overhead(%) GCC 108, , , XSNOW 26,687 26, ,687 0 XSPIM 12,473 13, ,473 0 XEARTH 9,215 9, ,215 0 With a larger buffer size, the overhead is less than 0.5%. In the case with smallest buffer size (i.e. 2x0.5K), the memory overhead is less than 10%. The benchmark used in the comparison is the buffer size equal to watermark and with only one entry. This excludes the potential factor from external fragmentation due to the BVL. 5.6 Summary We have presented performance study over four programs with distinct allocation patterns. Obviously, larger buffer size will lead to a higher hit ratio and lower memory overhead. This observation is confirmed by the simulation with the larger buffer size (in Table 4 and Figure 2). Even with the smallest buffer, 2x0.5K (which is only 128 bytes), the hit ratios are ranging from 78% to 99% depending on the application. And, the memory overhead is still within 10%. It is worth noting that gcc yields the worst performance because the programmer s hesitation to use the malloc function. Without maintaining their own free list which leads to many 4 Kilobytes requests, gcc would perform as well as the other applications. With the proposed hardware malloc and free functions, the programmer should not need to maintain their own free list. In such situation, the ALB would perform very well even with the smallest buffer size. 18

19 6. Conclusion Recent advances in software engineering such as graphical user interface and object-oriented programming have caused the applications to become more memory intensive. To facilitate the data intensive computing, this paper presents a novel design of dynamic memory management unit. The time consuming functions such as allocation and deallocation in software can now be processed through hardware components in the DMMU. By caching a small part of bit-map of the current process, the DMMU can operate efficiently in multiprocessor environments. Detailed design and simulation of the proposed DMMU have been presented in this paper. To evaluate the performance of proposed scheme, four commonly used applications are traced. The average object sizes of these traces range from bytes. Even with the smallest buffer, 2x0.5Kbits (which is only 128 bytes), the hit ratios are ranging from 78% to 99% while the memory overhead is within 10%. With the 8x1Kbits buffer, the hit ratios are ranging from 95% to 99%; at the same time, the memory overhead is down to 0%. 7. References [1] M. Chang, C.D. Lo, and W. Srisa-an, A Hardware Implementation for Malloc and Free Functions, submitted to IEEE International Conference on Computer Design, Austin, Texas, USA, October [2] M. Chang and E. F. Gehringer, A High-Performance Memory Allocator for Object-Oriented Systems, IEEE Transactions on Computers. March, pp [3] C. H. Daugherty and J. M. Chang, Common List Method: A Simple, Efficient Allocator Implementation, Proceedings of Sixth Ann. High-Performance Computing Symposium, Boston, Massachusetts, Apr. 5-9, pp [4] J. Morris Chang, Woo Hyong Lee and Yusuf Hasan, "Measuring Dynamic Memory Invocations in Object-Oriented Programs" Proceedings of 18th IEEE International Performance Conference on Computers and Communications, Phoenix, Arizona, Feb , pp [5] A. Kim and J. M. Chang, Designing a Java Microprocessor Core using FPGA Technology, Proceedings of 7th IEEE International ASIC Conference, Rochester, New York, Sep , pp [6] E. F. Gehringer and J. M. Chang, Hardware-Assisted Memory Management, Proc. OOPSLA 93 Workshop on Memory Management, Sep [7] J. M. Chang and E. F. Gehringer, "Evaluation of an Object-Caching Coprocessor Design for Object-Oriented Systems", Proceedings of IEEE International Conference on Computer Design, Oct. 3-6, 1993, pp [8] J. M. Chang and E. F. Gehringer, "Object Caching for Performance in Object-Oriented Systems", Proceedings of IEEE International Conference on Computer Design, Oct. 1991, pp [9] R. Jones, R. Lins, Garbage Collection: Algorithms for automatic Dynamic Memory Management, John Wiley and Sons, 1996, pp.20-28, 87-95, 296 [10] Arie Kaufman, Tailored-list and recombination-delaying buddy system, ACM Transactions on Programming Languages and Systems, Vol. 6, No. 1, Jan. 1984, pp

20 [11] Microsoft Visual C++ programmer s reference manual, Microsoft press, [12] Motorola MPC 750 Technical Summary, [13] D. Patterson and J. Hennessy, Computer Organization and Design, 2nd edition, Morgan Kaufmann, 1998 [14] D. Patterson, T. Anderson, N. Cardwell, R. Fromm, K. Keeton, C. Kozyrakis, R. Thomas, and K. Yelick, A Case for Intelligent RAM: IRAM, IEEE Micro, April [15] A. Silberschatz and P. B. Galvin, Operating System Concepts, 5th edition, Prentice-Hall, 1998 pp. 397 [16] A. S. Tannenbaum and A. S. Woodhull, Operating System: Design and Implementation, 2th edition, Prentice- Hall, 1997 pp [17] Paul Wilson, M. Johnstone, M Neely and D. Boles, Dynamic Storage Allocation: A Survey and Critical Review, Proc Int l workshop on Memory Management, Scotland, UK, Sept , [18] Benjamin Zorn, Custo-Malloc: efficient synthesized memory allocators, Technical Report CU-CS , Computer Science Department, University of Colorado, July [19] Albert Yu, Creating The Digital Future; The Secrets Of Consistent Innovation At Intel, The Free Press, Aug [20] D.M. Ungar and F. Jackson, "An adaptive tenuring policy for generation scavengers", ACM Transactions on Programming Languages and Systems, 14(1): 1-17,