Space and Time-Efficient Hashing of Garbage-Collected Objects

Size: px

Start display at page:

Download "Space and Time-Efficient Hashing of Garbage-Collected Objects"

Shon Carr
5 years ago
Views:

1 SML document # To appear in Theory and Practice of Object Systems, Space and Time-Efficient Hashing of Garbage-Collected Objects Ole Agesen Sun Microsystems 2 Elizabeth Drive Chelmsford, MA 01824, U.S.A. ole.agesen@sun.com Initial version: April Revised: July 1997, February 1998, May Abstract. The hashcode() method found in the Java programming language, and similar methods in other languages, map an arbitrary object to an integer value that is constant for the lifetime of the object. We review existing implementations of the hash operation, specifying the kinds of memory systems for which they work. Then we propose a new implementation of hashing for the hardest case: memory systems with compaction and direct pointers. Our proposal uses just two bits of space per object for the (majority of) objects that are never hashed. 1 Introduction The Java programming language defines the method hashcode() in the topmost class Object [2] (p. 64): public native int hashcode(). For any object, the method returns a 32-bit integer hash value, which programmers can use to build hash tables of objects. Smalltalk [4] (p. 96) and Self [1] (p. 62), as well as other object-oriented languages, define similar methods. The Self version of the method is called _IdentityHash. As this name suggests, hash values reflect object identity (by identity we mean the exact same object, not just two objects which have the same state): if obj1 and obj2 have different hash values, they cannot be identical. The hash operation is typically implemented in the virtual machine or runtime system although many object-oriented languages allow application programmers to override this default implementation for specific classes. For example, the java.lang.string class overrides hashcode() to compute a hash value based on the characters in the string. The resulting hash values, in this case, reflect string equality as defined by the java.lang.string.equals() method: if two strings have different hash values they cannot be equal (and consequently cannot be identical). In this document, we discuss different implemenation strategies for the identity hash operation. For brevity, we shall just refer to it as the hash operation. For correctness and efficiency, the hashcode() operation must: remain constant throughout the lifetime of the object, have good distribution (i.e., two different objects should, with high probability, have different hash values), be efficient to compute, and have low storage overhead. For example, a hashcode() implementation that always returns 0 would be a poor choice since it degrades the performance of data structures that use hashing. For most programs, the vast majority of objects are never hashed using the virtual machine s implementation of hashcode(). First, programs simply don t put each and every object they create into hash tables. Second, as explained above, important classes of objects override the virtual machine s hashcode() implementation. To illustrate, we measured on a version of the source-to-bytecode compiler javac, which is written in the Java programming language. Compiling the java.lang.* package, javac allocated 351,353 objects, hashing only 1,801 of these with the virtual machine s hashcode() operation. In another experiment, we used the hotjava web browser for a period of time. At the end of the run, 1,649,228 objects had been allocated and only 499 objects had been hashed. Since most objects are never hashed, to ensure low space overhead it is essential to use as little space as possible for non-hashed objects. The problem is, of course, that in general it is difficult to predict whether the hashcode() 1

2 operation will be applied to a given object (although for classes that override hashcode(), this prediction may be possible). 2 Traditional implementations 2.1 Non-copying memory systems In systems that use a non-compacting garbage collector, objects never move. Once allocated at a certain address, an object remains there until it become garbage. It is therefore possible to use the address of the object as its hash- Code(). This solution is fast, has no space overhead, and offers an extremely good distribution of hash values: obj1.hashcode() == obj2.hashcode() obj1 == obj2. However, few modern implementations of object-oriented language use non-compacting memory systems, since the induced fragmentation can affect performance negatively. 2.2 Handle-based memory systems The original Java virtual machine [6] and some Smalltalk virtual machines use indirect pointers, called handles in [6], to refer to objects. Handles allow easy relocation of objects during garbage collection since, with handles, there is only one direct pointer to each object: the one in its handle. All other references to the object indirect through the handle. In such handle-based memory systems, while object addresses change over the lifetime of objects and therefore cannot be used for hashing, handle addresses remain constant. Thus, the hashcode() operation can be implemented by returning the address of the object s handle. This implementation, like the object address implementation in non-compacting systems, is fast, has no space overhead, and gives a good distribution (satisfying the relation given above). However, other concerns, including execution efficiency of the system as a whole, may favor memory systems without handles, necessitating a different implementation of the hashcode() operation. 2.3 Direct-pointer, no-handle memory systems Most high-performance implementations of languages use compacting garbage collection algorithms that work with direct pointers. Consequently, hash codes must be implemented in a different way than by using addresses (objects move, so their address is no good, and there are no handles either). A common solution is illustrated by the Self system [3]. Each Self object contains two header words: a map pointer (which is analogous to a class pointer in a classbased language) and a mark word. The mark word contains a number of bit fields related to garbage collection in addition to a 22-bit hash field (22 is not an arbitrary number, but the number of bits remaining once the other needs have been fulfilled). The 22-bit hash field is initialized to zero, but when an object is first hashed a pseudo-random number is generated and stored in the field. While this lazy hash value assignment slows down all hash retrieval operations by an extra test, it is a net win since object allocation is more frequent than hashing for most programs. This hash code implementation has fast retrieval. It also has low storage overhead, if there happens to be enough spare bits in a header word. However, when hashing is the drop that overflows the bucket and necessitates adding an extra header word to every object, the space cost of this implementation of hashcode() is large. Another potential drawback is the temptation to compress the hash function range into whichever number of bits are available to avoid paying the cost of an extra word in every object. While Self s 22 bits probably suffice for most purposes, Squeak s 12-bit hash values may not [5]. Certainly, one could imagine applications for which a full 32-bit (or larger) hash range would improve performance. Some Lisp systems have used a different technique to implement the hashcode() operation. As a way of introducing this technique, reconsider for a moment the very idea of having a hashcode() operation. Instead of defining a hashcode() operation, the virtual machine can provide a built in hash table data type. For specificity, let us call this data type BIHashTable. A BIHashTable maps an arbitrary key object to an arbitrary value object. By virtue of being defined in the virtual machine, BIHashTables can use the key objects addresses for hashing. When the garbage collector moves one or more key objects, it performs extra work to maintain the validity of the BIHashTables. The extra work can be done incrementally by deleting the old (key, value) pair and inserting the new pair, or it 2

3 can be done en masse by building a replacement hash table at the end of each GC, thus saving the cost of the hash table deletion operations. Lisp systems have used variations of BIHashTables for several years; see for example [7] (p ). In a straightforward implementation of BIHashTables, the rehashing imposes extra work at every garbage collection in proportion to the number of key objects that were moved. An optimized implementation may perform the rehashing lazily. Then the garbage collector will update the key addresses in the hash table data structure when keys move, but it will not rehash (i.e., it will not move the (key, value) pairs to their new bins). To detect out-of-date hash tables, a simple GCcount time stamp suffices: is GCcount of last rehash less than current GCcount? The test to determine if rehashing is necessary can be performed after failing lookups (rehash and retry the lookup). BIHashTables, while elegantly solving the problem of using addresses of moving objects as keys in a hash table, have some drawbacks: broader virtual machine interface: instead of providing a simple hashcode() operation, systems with BIHashTables must provide an implementation of hash tables in the virtual machine. high specificity: a mapping from objects to nearly unique integers, as provided by hashcode(), can be used for many purposes other than building hash tables. These disadvantages disappear if BIHashTables are used internally in the virtual machine to implement the hashcode() operation. The idea is to use a BIHashTable, which we will call hashcodetable, to map objects to hash codes. The only access to this table from user code is through the hashcode() operation, implemented in the virtual machine as follows: int hashcode(object *obj) { int h = hashcodetable->lookup(obj); /* Returns NOTFOUND if obj is not in table. */ if (h == NOTFOUND) { /* Assumes NOTFOUND isn t used as a hash code. */ h =...new hash code...; hashcodetable->insert(obj, h); } return h; } The hashcodetable must use weak references for the key objects to allow them to be garbage collected when they are otherwise unreachable. (The need for weak references may add some implementation complexity, unless weak references are already present in the virtual machine for other reasons.) Having reviewed the existing implementation techniques for the hashcode() operation, we now describe a new implementation that has a different and, we believe, competitive time/space trade-off, at least for some systems. 3 New proposal for implementing hashcode() The following proposal for implementing the hashcode() operation works in compacting handleless memory systems and satisfies the four properties mentioned in Section 1. Specifically, it provides a full 32-bit hash range and uses only two bits of space in non-hashed objects. 3.1 Basic idea We initially allocate objects without a field for hash values. Under some circumstances, the object may later be expanded by an extra field, which will hold the hash value of the object. The problem then is to support this expansion without incurring a large overhead or excessive garbage collection and tracing of objects. Indeed, we note that expanding an object is usually not possible without relocating it, since objects are packed tightly to utilize memory efficiently. Moreover, having no handles, it is difficult to relocate an object without scanning large areas of memory to locate and update all pointers to the object. To overcome these difficulties, we reserve two bits in the header of objects, as follows: hasbeenhashed: 0 initially; set to 1 when the hashcode() operation is performed on the object. hashashfield: 0 initially; set to 1 if the object has been expanded with a hash field at the end. 3

4 At object allocation time, both bits are set to 0 and they remain so until the object is hashed. Using these bits, the hashcode() operation is implemented as follows: int hashcode(object *obj) { obj->hasbeenhashed = 1; /* Object has now been hashed. */ if (obj->hashashfield == 0) return (int)obj; /* Use object s address. */ else return obj->hashfield; /* Use "internalized" hash value. */ } This method has two important aspects: it tracks which objects have been hashed, and it selects either the address of the object or a value from a hashfield as the hash value (this field is set by the garbage collector; see below). When the garbage collector relocates objects, it inspects their hash bits. If hasbeenhashed == 0, nothing special needs to be done to the object. If hasbeenhashed == 1 and hashashfield == 0, the object has been hashed, but it has no hash field yet. The garbage collector allocates an extra field, hashfield, at the object s new location, stores the object s old address in the field (which is the object s hash value), then copies the rest of the object and sets hashashfield == 1. From now on, the object will be one field larger than when it was born. We say that the hash value has been internalized. The new field is located at the end of the object so that the original fields offsets remain unchanged. Figure 1 illustrates this process for a system in which objects have a single-word header containing a class pointer and the two hash state bits. hashashfield hasbeenhashed class ptr 10 non-header fields hashashfield hasbeenhashed class ptr 11 non-header fields offsets to these fields remain unchanged hash code In summary, we use the object s address as the hash value for as long as possible (until the object is relocated). When relocation of a hashed object changes its address, we expand the object and internalize the hash value. Finally, it is interesting to note that in memory systems where certain (or all) areas are never compacted, this hash code implementation automatically reverts to a solution that uses just 2 bits of additional space over the space-wise optimal technique of using object addresses in non-relocating systems. 3.2 Generational systems before relocation after relocation Fig. 1. The result of copying an object the first time after it has been hashed. Generational memory systems allocate objects in a small young space. They attempt to gain performance by keeping new and active objects in a small region of memory, which can be garbage collected more efficiently. Restricting new objects to a small area of memory may negatively affect the distribution of hash values. We expect that most objects will be hashed while they are still young and therefore receive a hash value from the limited range of addresses in the young space. For example, a typical young space may be 512 kb. Assuming word-aligned objects, the maximal number of different hash values is 128 k or just 17 bits. To recover a good distribution of hash values we can xor a pseudo-random number onto the object s address and return the result as the hash value. The random number expands the range of hash values to 32 bits (or whatever size is desired). The random number stays constant for the duration of one collection cycle of the young space. After compaction of the young space, the current random number is no longer needed because the garbage collector has internalized the hash values for all objects that used it. The garbage collector then replaces the random number by a new one so that in the next cycle through young space, an object that is hashed for the first time is likely to receive a unique hash value, even if other objects occupied the same address during a previous cycle. 4

5 3.3 Evaluation Table 1 compares the three implementations of the hashcode() operation for handle-less compacting memory systems. Column 1 describes the implementation that uses a hash field in all objects, column 2 describes the implementation that uses a weak BIHashTable, and column 3 describes the technique proposed in this paper. 1. Hash field in every obj. 2. Weak BIHashTable 3. On-demand extension of objs Range of hash codes n bits; often less than full word full word full word Space per unhashed object n bits 0 bits 2 bits Space per hashed object n bits 2 words a 1 word and 2 bits Time, hashcode() load, bit masking, test, branch (optionally set hash code) hash table lookup and perhaps insertion load bit, test, branch, load hashfield or use obj. addr. Time overhead, GC none rehash b extra test to determine obj. size Table 1. Comparison of three techniques for implementing hashcode() in compacting handle-less memory systems. a. We assume that each entry (object address, hash code) in the hash table occupies two words; additional overhead may result from the hash table operating at less than 100% utilization. b. With the optimization described in the text, at most one rehash per GC is required. It can be argued whether this cost should be charged to the GC or the mutator; here we have arbitrarily listed it as a GC cost. First consider space usage. No one implementation technique is always best. If sufficiently many spare bits (n) can be found to hold the hash code within the object, the technique in column 1 wins. Otherwise, if two bits can be found, column 3 wins. Next, consider the speed of the hashcode() operation. Columns 1 and 3 are very fast. Column 1 implements hashcode() with a load, some bit masking, a test to determine if this is the first time the object is hashed (we assume lazy setting of hash codes), and the code to set the hash code if needed. In the common case, when the object has been previously hashed, this amounts to about a handful of instructions. Column 3 s implementation of hash- Code(), shown in Section 3.1, compiles into eight instructions with Sun s cc compiler at optimization level -xo4. The difference between five and eight instructions is probably small when considered in relation to the subsequent use of the hash code (probing some application-level hash table, typically). Column 2 also has a fast hashcode() operation, though not quite as fast as the other two implementation techniques. In many cases, the complexity of hashcode() in column 2 will be comparable to but no worse than the complexity of the subsequent use of the hash value. Essentially, each application-level hash table probe will cost two hash probes: one to look up the hash code followed by one at the application level. Now consider garbage collection overhead. First we compare columns 1 and 3. The technique in column 3 complicates the (copying) collector slightly by imposing extra work on the relocation of objects. For example, to compute the word size of an object about to be relocated, the collector first computes the size without counting the possible extra hash code field. Next, the collector must add one to the size if the hash field is present. This adjustment requires a load of the word containing the hashashfield bit, a bit-wise and to extract the bit, possibly a shift to bring the bit to the right location, and an add to combine the bit and the unadjusted object size. The overhead on computing the object size therefore is three or four instructions. Similarly, computing the after-relocation object size can be done in the same number of extra instructions, but using the hasbeenhashed bit instead of the hashashfield bit. Finally, the collector compares the two sizes to determine if it needs to internalize the hash code (i.e., store the object s old address in the hash field and turn on the hashashfield bit). With careful coding, we expect the total overhead on copying an object to be less than 20 instructions in the case when hash codes are being internalized and less than 12 instructions in other cases. While 12 to 20 extra instructions seem affordable, the bottom-line performance impact, we acknowledge, remains to be determined by measuring on an actual implementation. One important fact was ignored above: the implementation technique in column 3 should be favored over column 1 only when it saves a word per unhashed object. Thus, the system in which the garbage collector performs extra work for each object it relocates (column 3) should be compared against a system in which (most) objects are one word bigger (column 1). The former system can pack more objects in a given size heap, resulting in fewer garbage collections, which in part or perhaps completely offsets the extra work required when relocating objects. While a word per object may not sound like much, the overall effect nevertheless could be significant since the average object size 5

6 in many object-oriented programs is small, perhaps as small as 10 words. In some cases, such as memory-limited embedded systems, a 10% heap size reduction could be crucial. Even for large systems, the resulting locality improvement and d-cache hit rate increase could yield a measurable performance gain. Finally, let us compare garbage collection overhead between column 2 and column 3. The main drawback of column 3, as explained above, is the extra work required when copying objects. Column 2 suffers from two different drawbacks. First, it requires support for weak references. (Of course, for systems that require weak references anyway, this cost should not be taxed to the hashcode() implementation technique). Second, it incurs the cost of rehashing the hashcodetable up to once per garbage collection. The cost of rehashing is proportional to the number of live objects that have been hashed and is being moved. Thus, as the number of hashed objects grows, the performance of column 2 will decrease. 4 Conclusion We have described a new implementation of object identity hashing for handle-less compacting memory systems. The new technique uses on-demand extension of objects, offers a good distribution of hash codes, is simple to implement, and provides a competitive time/space trade-off. It requires only 2 bits of space in the majority of objects that are not hashed, and 2 bits plus a word (or whatever size hash value is desired) in objects that have been hashed and subsequently relocated. Acknowledgments. The main idea in this paper came about after discussions with the other members of the Java Topics Group: David Detlefs, Christine Flood, Steve Heller, Guy Steele, and Derek White. The anonymous reviewers provided comments that significantly improved this paper. References 1. Agesen, O., Bak, L., Chambers, C., Chang, B.-W., Hölzle, U., Maloney, J., Smith, R.B., Ungar, D., and Wolczko, M. The Self 4.0 Programmer s Reference Manual. Available from July Arnold, K. and Gosling, J. The Java Programming Language. The Java Series, Addison-Wesley, Chambers, C., Ungar, D., and Lee, E. An Efficient Implementation of Self, a Dynamically-Typed Object-Oriented Language Based on Prototypes. Lisp and Symbolic Computation 4(3), Kluwer Academic Publishers, June Originally published in Proceedings of the 1989 ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages & Applications (OOPSLA 89), New Orleans, LA, October Goldberg, A. and Robson, D. Smalltalk-80: the Language and its Implementation. Addison-Wesley, Ingalls, D., Kaehler, T., Maloney, J.H. Wallace, S., and Kay, A. Back to the Future: The Story of Squeak, A Practical Smalltalk Written in Itself. Proceedings of the 1997 ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages & Applications (OOPSLA 97), Atlanta, GA, October Lindholm, T. and Yellin, F. The Java Virtual Machine Specification. The Java Series, Addison-Wesley, Symbolics. Reference Guide to Symbolics-Lisp, release 6.0, March Sun, Sun Microsystems, and Java are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. 6

Lecture Notes on Garbage Collection

Lecture Notes on Garbage Collection 15-411: Compiler Design Frank Pfenning Lecture 21 November 4, 2014 These brief notes only contain a short overview, a few pointers to the literature with detailed descriptions,