Towards Garbage Collection Modeling

Transcription

1 Towards Garbage Collection Modeling Peter Libič Petr Tůma Department of Distributed and Dependable Systems Faculty of Mathematics and Physics, Charles University Malostranské nám. 25, Prague, Czech Republic phone , fax Technical Report No. 2011/1, January 2011 Abstract Performance models tend to ignore impact caused by background subsystems like garbage collection. The precision of such models might therefore be questionable. We want to introduce models that capture the performance of the garbage collection. Towards that goal, we propose a set of models to estimate collection frequency and timings and give validation results. We also introduce tools and benchmarks needed to gather the input data and calibrate the models. 1 Introduction Systems with automated memory management became widely used inrecentyears. Suchsystems are very convenient for the application developers because the code does not need to handle deallocations. This convenience, however, comes with a cost. Theinfrastructureintroducesanew subsystem, the garbage collector, which is responsible for reclaiming memory that is not needed anymore by the application. If the application is allocating asignificantnumberofobjects,the garbage collector overhead may be as high as tens of percents [1, 2]. This overhead is not included in performance models and it is implicitly treated as a constant background factor. However, the overhead is not constant it changes with different virtual machines and workload configurations [1, 2]. We believe that the performance models need to incorporate better knowledge of garbage collection performance in order to provide reliable results. In this technical report, we document our approach towards creating a model of garbage collector overhead for Java. In Section 2, we describe the garbage collectors we are interested in. Section 3 describes the workload parameters we believe are responsible for most of the observed overhead. Next, we describe the general model design (Section 4), how we modelthegarbagecollectionpoints (Section 5) and how long the collections take (Section 6). We conclude the report in Section 7. 2 Garbage Collectors The task of the garbage collector is to identify memory that is notusedanymoreandtomakethis memory available for new object allocations. In contemporary virtual machines, tracing garbage collectors are utilized. Here, tracing means the collector traverses the entire object reachability graph and those objects that were not traversed are thus identified as garbage and freed. We focus on stop-the-world collectors, where the application threads are interrupted and not running for the duration of the garbage collection. Stop-the-world collectors are relatively frequent because suspending the application threads makes the algorithm easier to implement and also more effective on the other hand, long garbage collection pauses may be experienced. Many different garbage collector implementations exist across virtual machines. In the following text, we use always Java, however we believe, the results might be applied for different systems, 1

2 also. Here, we describe three collectors from the virtual machines we are using for the experiments described in the following text. 2.1 Jikes Research Virtual Machine With Jikes RVM, we are using the SemiSpace collector. This collector algorithm is using two equally sized heap regions. Objects are allocated only in one oftheregions,andwhenitbecomes full, garbage collection is triggered. The algorithm is tracing the reachability graph and all objects it encounters are copied into the other region, one after another, creating a contiguous area of allocated space. The roles of the two spaces are exchanged after each collection. Since all objects are in one contiguous memory area, new objects can be allocated by simply increasing the pointer pointing to the end of the allocated space. The copying costs for large objects are high, therefore there is one extra space, called Large Object Space, where objects larger than a certain threshold are allocated. This space is collected using the Mark&Sweep algorithm. 2.2 IBM Virtual Machine With IBM VM, we are using the collector that optimizes throughput. It uses one heap space with free lists and collects using the Mark&Sweep algorithm. Allocated objects are also identified in a bitmap. The algorithm operates in two phases. In the Mark phase, objects are marked as live during traversal. The VM is using one more bitmap for the marks, with the same granularity as the allocation bitmap. In the Sweep phase, the collector traverses the allocation and mark bitmaps and when it finds a memory block that is allocated and not marked, the block is freed. The collector creates small areas of free space that cannot be allocated,becausetheyaretoo small. To avoid the depletion of memory due to this effect, the collector compacts the heap from time to time. During compaction, the heap is divided into several areas and all live objects in those areas are copied to the beginning of the area. 2.3 Sun Virtual Machine With Sun VM, we are using a generational stop-the-world garbage collector. It has a young generation with three areas, namely one Eden space and two Survivorspaces. Objectsareallocatedinto Eden by a simple end pointer increment. The two Survivor spaces are forming a SemiSpace area. At collection, which is triggered when Eden is full, live objects from Eden and one Survivor space are copied into the other Survivor space. When the number of times an object has been moved between survivor spaces exceeds certain threshold, or when there is no more space in the Survivor space, the object is copied into the Old generation (it is tenured). The Old generation is using free lists to track the allocated objects. When the Old generation isfull, thefullgarbagecollectionis triggered. It uses a variant of the Mark&Sweep algorithm, which also copies objects traced in the Young generation into the Old generation. In order for the Young generation collection to be correct, references from the Old to the Young generation must be tracked. This is done by instrumenting the pointer updates (called write barriers). The instrumentation keeps track of pointers leading from the Old to the Young generation. At the Young generation collection, these pointers are added to the heap roots for the collection. 2

3 3 Relevant Workload Parameters The series of experiments described in Deliverable 3.3 [1] and [2] has identified several workload parameters that significantly influence the garbage collector overhead. Although there are other workload parameters relevant to performance, those listed here not only have the apparently largest impact, but also are mostly unrelated to the garbage collector internals, and are therefore suitable for devising a platform independent model. The parameters we investigateareobjectlifetimes, object sizes, object connectivity, and allocation speed. 3.1 Object Lifetimes The object lifetime is time that elapses between the moment when the object is allocated and the moment when the object becomes unreachable. It affects the performance of the collection in two major ways: It affects how many live objects are in the heap when an object is allocated,thenumber of objects increases by one, and when it becomes unreachable the number decreases by one. This is important because tracing garbage collectors operate on live objects and lifetimes can tell how many live objects are in the heap at any time. It affects the distribution of the objects between distinct heap generations. The objects with longer lifetimes will be located in higher generations than objects with shorter lifetimes, thus affecting when a collection of a particular generation is triggered. Different units can be used to express lifetime including seconds, allocations, bytes or method invocations. In our case, the suitable unit is one allocation, which means that the lifetime of an object is defined by how many allocations of other objects it survived Lifetime Measurement In systems with explicit memory management, the object lifetime can be determined by intercepting the allocation and deallocation functions (like malloc() and free() in the C language) Unfortunately, this straightforward approach cannot be used in environments with automatic memory management, since there is no deallocation function and the objects aredeallocatedonlyduringgarbage collection. This can happen a long time after the object became unreachable. There are two known approaches to measuring the object lifetimes the bruteforce algorithm and the Merlin algorithm [3]. The first approach forces garbage collection in small, regular intervals, which makes it possible to assess the object lifetimes with the precision equal to the interval. If the collection is triggered every allocation, the lifetime traces are precise. However, this comes with a price of a huge overhead, often in the order of millions of percents. On the other hand, the Merlin algorithm operates only with normally triggered garbage collections. During the run of the application, it records pointer updates and then reconstructs the object lifetimes from this data. The algorithm is faster than the bruteforce approach,however,it requires cooperation of a virtual machine to be effective and there is no working implementation available at present. Since, to our knowledge, there is no suitable tool to measure object lifetimes, we had to create anewone. Wehavedecidedtousethebruteforceapproachsinceitiseasiertoimplementandif we set a lower resolution (higher garbage collection interval), the overhead becomes feasible. The 3

4 trace precision should be set to a value significantly lower than the correspondingnumber of objects needed to fill the smallest generation of the heap. The tool is using instrumentation by bytecode rewriting to keep track of allocations every NEW, NEWARRAY, ANEWARRAY and MULTIANEWARRAY instruction isinstrumentedwith anotificationtoajvmtiagentthatlogstheevents. TheJVMTIagent is also responsible for receiving object deallocation events sent by the virtual machine. These can be requested by the agent and it also logs these events. The two steps produce a trace of object allocations and deallocations, from which it is possible to compute the lifetime for every object. 3.2 Object Sizes The importance of the object size is clear if the application isusinglargerobjects,itneedsfewer allocations to fill the heap and trigger a collection. It also implies smaller size of the free space in the heap after the collection, which again causes more frequent collections. To measure the sizes of objects, the same tool as for lifetime measurements can be used. At every object allocation callback, a JVMTI function can be called that returns the size of the object thisvalueislogged. AccordingtotheJVMTIdocumentation,aproblemwiththeobjectsize measurement is that it is not clear whether the size returned by the JVMTI function will or will not include all the object headers and other types of overhead. Therefore, these numbers must not be trusted entirely, but they still provide information sufficient for our purposes. 3.3 Allocation Speed The allocation speed gives the relationship between object allocations and time, so it is expressed in objects per time unit, for example seconds. Typical applications have allocation speeds in orders higher than hundreds of thousands per second. This influences howquickly(inwallclocktime)the heap will fill up. For a stable application, we can measure the allocation speed by counting allocations(for that, we need bytecode instrumentation), and the total execution time in the period of execution we are interested in. The time spent in GC should not be included in total time for modeling purposes. In this case, the overhead and perturbation by instrumentation should be dealt with. However, the overhead is usually low enough (approximately 2 % of execution time). 3.4 Outgoing References Apropertyofthetracinggarbagecollectionalgorithmistherequirementtovisitallthereachable objects through references pointing from an object that was already proven reachable. This means that the work the algorithm needs to do depends on the total number of non-null references in the heap. Therefore, the number of references stored in an object pointing to other objects is an important parameter of the application. For a stable application, the average number of outgoing references per object can be sampled throughout the execution of the application using JVMTI. At specific points (for example garbage collections), the heap can be traversed, reference by reference, to get the total number of references in the heap. Also in the same traversal, the total number of live objects can be determined. The two numbers together give a sample average number of references per object. 4

5 4 Modeling Assuming a stable application behavior, the model that captures the garbage collector overhead consists of two logical parts: 1. Determining when the garbage collection is triggered. First, the moments on a timeline when a garbage collection is started are identified. By using the allocation speed, it is possible to determine the time (in seconds) when the garbage collections happen. Forastableapplication, the exact moments can be replaced by a garbage collection frequency. 2. Determining the duration of a particular garbage collection. Given the collection from a previous step, the time spent in the collection needs to be identified. If the two steps are completed, the total overhead of a garbage collectionisthesumoftimes of the individual collections. The two steps are described in the following sections: the first one in the Section 5 and the second one in the Section 6. 5 Modeling Garbage Collection Frequency As the first step in garbage collection performance overhead modeling, the moments of the garbage collections occurrence need to be determined. Since we assume stable applications, this can be substituted by collection frequencies, or in the case of an application with a fixed behavior, by the counts of collections. For the sake of limiting the model complexity, we make certain assumptions about the memory management system that might not be true in real applications, but that can be forced by setting the virtual machine parameters. Specifically, we expect the heap and all of its regions to be of fixed size for the entire lifetime of the virtual machine. Other assumptions reflect our understanding of the garbage collection. We know the following assumptions are not always true, but we believe they do not have an impact on the overall garbage collection performance. The collections are triggered when the entire heap or any of the generations is full. The collections are complete no garbage is left in the collected generation. There are two significantly different collection algorithm classes with single generation and with multiple generations. The behavior of the two classes is very different and we therefore handle them separately the approach for a single generation case is covered in Section 5.1, multiple generations are treated in Section Non-generational Garbage Collectors We simplify the memory management system of a single-generational heap in this manner: Objects are allocated into the heap one after another, as long asthereisfreespaceleft. 5

6 If the object cannot be allocated because of the lack of space, the garbage collection is triggered. It de-allocates all the objects that are not reachable anymore. In order to assess the moments where garbage collection occurred, we use a simulation-based approach. We use the measured object lifetimes for that purpose, and we assume that in stable applications, the object lifetimes do not change. The simulation is following the lifetime trace, which also includes the object sizes, and adds every object found in the trace to a simulated heap. When the simulated heap is full, it is reported as a moment of the garbage collection and all objects whose lifetime expired at or before that moment are removed from the simulated heap. This scenario can be expressed using a mathematical equation: n i HS = SIZE [j] + SIZE [j] (1) j=n i 1 +1 The meaning of symbols in the equations is as follows: j {1...n i 1 } DEATH[j] n i The moment when the collection number i occurred. Since the object allocation is a time unit, this value means that collection number i happened after allocating n i objects. SIZE[i] The size of the object that was allocated as the i-th object in the application. DEATH[i] The time when object number i (allocated as the i-th object) became unreachable. Since the time is measured in object allocations, this means the object became unreachable after allocating object number DEATH[i] and before allocating object number DEATH[i]+1. When the object lifetimes are available, this value can be computed as DEATH[i]=LIF ET IME[i]+ i. HS The available size of the heap to model. In most virtual machines, this corresponds to setting the -Xmx and -Xms parameters to this value. Note that for a fixed application and heap size, the n i sequence is the only variable. With knowledge of lifetimes and sizes of the objects, it is possible to compute these values. Also, the equality relation must be understood as approximate only it isunlikelythatthesumofsizes of objects would result in exactly the heap size, but for modeling purposes, we can allow this simplification in order for the formula to be less complicated. The formula from above requires large input data set the lifetime and size information for every object allocated in the application. We therefore make anattempttouseinputdatathat are smaller in size and easier to get. These values are the average lifetime and object size. In this case, the average object size can be measured at allocation time, and the average lifetime can be measured indirectly, by exploiting the fact that the average objectlifetimeisequaltotheaverage number of life objects in the application. This can be measured for example by sampling at garbage collection invocations. Using average lifetime and object size, the formula can be simplified into: n i n i 1 = HS OS LT (2) 6

7 Here, LT denotes the average lifetime, OS denotes the average object size. The left side of the equation gives the (average) number of allocations that take place between collections. In the following section, we show results for the simulation based on equation 1 and the simplified formula Simplified Formula The evaluation of the simplified formula was performed on Jikes RVM, version and BaseBaseSemiSpace configuration [5] with the DaCapo benchmarking suite, version [4]. The results are in Table 1. The last column (Prediction Ratio) shows the ratio of measured and predicted number of allocations between collections a value of 1.0 meansexactprediction,biggervaluesmean we are predicting fewer allocations between collection and therefore more collections. Given the simplicity of the model and data, the results are promising while they are not usable for exact performance prediction, they suffice for better vs worse analysis and other important use cases. Table 1: Simplified formula results for single-generational collection frequency prediction Benchmark -Xmx and -Xms LT OS Prediction Ratio antlr MB bloat MB fop MB hsqldb MB jython MB luindex MB lusearch MB pmd MB Simulation We have performed the simulation on IBM J9 VM, build pxa6460sr8fp (SR8 FP1). The lifetime and object size traces were also collected with this virtual machine. The benchmarks used are from the DaCapo benchmarking suite, version 9.12-bach, with -n 100 and --no-preiteration-gc options. The virtual machine is using a single-generation heap with the throughput collector, which uses the mark & sweep collection algorithm with occasional compaction. In order to get a more predictable behavior, we force the compaction after every collection. We measure the number of garbage collections and compare them with the number of collections predicted by the model. The results are in Fig Generational Garbage Collectors For a generational collector, we need to take into account more heap partitions and the way in which the partition where the object resides is chosen. Our model is aimed at the Sun JVM, version family 6. It uses the collector described in Section 2.3. We model the behavior as follows: 7

8 Comparison of predicted and measured s avrora 16MB avrora 17MB avrora 20MB avrora 24MB avrora 32MB avrora 64MB avrora 128MB batik 120MB batik 132MB batik 150MB batik 180MB batik 240MB batik 480MB batik 960MB fop 64MB fop 70MB fop 80MB fop 96MB fop 128MB fop 256MB fop 512MB pmd 64MB pmd 70MB pmd 80MB pmd 96MB pmd 128MB pmd 256MB pmd 512MB tomcat 24MB tomcat 26MB tomcat 30MB tomcat 36MB tomcat 48MB tomcat 96MB tomcat 192MB xalan 24MB xalan 26MB xalan 30MB xalan 36MB xalan 48MB xalan 96MB xalan 192MB Figure 1: Comparison of measured and predicted garbage collection counts, IBM VM. On the horizontal axis there is combination of benchmark and heap size. 1. New objects are always allocated into Eden space. Eden space and one of the Survivor spaces form the young generation. When the Eden is full, a young generation (or Minor) collection is triggered. 2. During minor collection, all surviving objects from Eden are copied into the second Survivor space. Also, the surviving objects from the first Survivor space are copied into the second one. As an exception, when the number of copying operations on a single object reaches a threshold, the object is copied into the Old generation. We denote this threshold as PCC. Also, if there is no space left in the second Survivor space, all remaining live objects from Eden or first Survivor space are copied into the Old generation. This makes the Eden space and the first Survivor space empty after the collection, and the roles of the two Survivor spaces are switched. 3. If the objects from Eden or Survivor space cannot be copied to Old generation because it is full, Full (or Major)collectionistriggered,causingtraversalofallreachable objects (including those in young generation). Reachable objects from the Young generation are copied into the Old generation, leaving the Young generation empty. Similarly to the single-generation case, the described behavior can be expressed using a set of mathematical equations. However, they are too complicated and give no better insight into the modeling problem, we therefore do not include them here. We have performed the simulation on Sun Java VM, build b02. The lifetime and object size traces were also collected with this virtual machine. The benchmarks used are from the Da- Capo benchmarking suite, version 9.12-bach, with the -n 100 and --no-pre-iteration-gc options. The heuristic for automatically sizing heap spaces was turned offbythe-xx:-usepsadaptivesurvivorsizepolicy parameter. For the model evaluation, we set a fixed heap size using the -Xmx and -Xms parameters, and also the Young generation size using the -XX:NewSize and -XX:MaxNewSize parameters. 8

9 Comparison of predicted and measured Minor s fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop fop Figure 2: Comparison of measured and predicted Young garbage collection counts for Da- Capo fop benchmark, Sun VM. On the horizontal axis, the label format is: Benchmark- With sizing heuristics turned off, this causes the young generation to be of fixed size. Each survivor space takes a constant 1/8 oftheyounggenerationsize,theedenspacetakes6/8 oftheyoung generation size. The size of the Old generation is also fixed by thissetting(old = Xmx Y oung). We measure the number of both Minor and Full collections and compare them with values predicted by the simulation. The results for Minor garbage collections are plotted on Fig. 2forfop, Fig.3fortomcat, Fig.4 for xalan, andfig.5forpmd benchmarks. As exemplified on fop, xalan and tomcat, the prediction is relatively good, however, some exceptions occur, as shown with the pmd benchmark. The results for Full garbage collections are plotted on Fig. 6forfop, Fig.7fortomcat, Fig.8for xalan, andfig.9forpmd benchmarks. The results show the simulation is in many cases imprecise. Even if the simulation usually fits the shape of measured values, the absolute values are imprecise. We speculate that causes for this could originate in pre-tenuring, early garbage collection triggers, or imprecise traces due to service objects on the heap that are notreportedbythevirtualmachine diagnostics. 6 Modeling Garbage Collection Times After determining the moments or frequency of garbage collections, we need to compute how long these collections take in order to figure out the overhead. The simulation from the previous sections provides us with information on what objects are alive in the collected generation. This is important since our previous measurements [2] indicate the time spent in garbage collection is proportional to the amount of live data in the collected generation. Most of the work of a tracing collector is spent traversing live objects. Since only live objects are traversed, the time should depend on the number or live objects, or the number of non-null references in the heap, since the collector is tracing the objects by following the references. 9

10 Comparison of predicted and measured Minor s tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat tomcat Figure 3: Comparison of measured and predicted Young garbage collection counts for Da- Capo tomcat benchmark, Sun VM. On the horizontal axis, the label format is: Benchmark- Comparison of predicted and measured Minor s xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long Figure 4: Comparison of measured and predicted Young garbage collection counts for Da- Capo xalan benchmark, Sun VM. On the horizontal axis, the label format is: Benchmark- 10

11 Comparison of predicted and measured Minor s pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd pmd Figure 5: Comparison of measured and predicted Young garbage collection counts for Da- Capo pmd benchmark, Sun VM. On the horizontal axis, the label format is: Benchmark- Comparison of predicted and measured Full s fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long fop long Figure 6: Comparison of measured and predicted Full garbage collection counts for Da- Capo fop benchmark, Sun VM. On the horizontal axis, the label format is: Benchmark- 11

12 Comparison of predicted and measured Full s tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long tomcat long Figure 7: Comparison of measured and predicted Full garbage collection counts for Da- Capo tomcat benchmark, Sun VM. On the horizontal axis, the label format is: Benchmark- Comparison of predicted and measured Full s xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long xalan long Figure 8: Comparison of measured and predicted Full garbage collection counts for Da- Capo xalan benchmark, Sun VM. On the horizontal axis, the label format is: Benchmark- 12

13 Comparison of predicted and measured Full s pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long pmd long Figure 9: Comparison of measured and predicted Full garbage collection counts for Da- Capo pmd benchmark, Sun VM. On the horizontal axis, the label format is: Benchmark- To determine the dependency of the collection time on the number of live objects and object references, we have created a set of three microbenchmarks: 1. An array of references of the specified length. Every item of the array is pointing to an object on the heap, no two references from the array point to the same object. In each step, one of the objects is replaced, the replacements are done in a sequential order from the first reference to the last one. This is repeated enough times to get stable performance. 2. A binary tree, representing a set of numbers from 1 to a specified number, therefore it has this number of nodes. In each benchmark step, one node is replaced with a newly created one. The order of replacement is by node values, from one to the last numberstoredinthetree, replacing all the nodes eventually. This is repeated enough times to get stable performance. 3. A circle, or a simple linked list with the last item pointing backtothefirstone,andone reference from the local variable to one of the objects. In each benchmark step, tho object the local reference is pointing to is replaced by a new object, andthereferenceismovedto the next object in the list. Eventually, all the nodes are replaced, and this replacement is repeated enough times to get stable performance. In these microbenchmarks, we vary the number of objects or nodes to be able to observe the dependency of the garbage collection times on the number of objects. With the Sun virtual machine, these benchmarks are used to determine the timings in the Young generation we make sure (by sizing the heap appropriately) that no benchmark objects are intheoldgeneration. Theresults are near linear, as showed in Fig. 10. We have computed a linear regression on the data and applied ittothedacapobenchmarks. The results and comparison with measured values are in Fig. 11. It is clear that the results are still 13

14 Microbenchmark timings GC time Circular list Reference array Binary tree 0e+00 2e+05 4e+05 6e+05 8e+05 1e+06 Benchmark object count Figure 10: Dependency of Young garbage collection time and number of object in microbenchmarks rather imprecise for the purpose of predicting exact performance our hypotheses for the reasons include processor caches and remembered set maintenance (responsible for handling references from the Old to the Young generation). We have also made an experiment with a simpler garbage collector on the IBM virtual machine, measuring the garbage collection time per object for the DaCapo benchmarks. As the IBM virtual machine reports the times of multiple collection phases separately, we plot both the marking time and the compaction time. The results are in Fig. 12. It is clear that while the times per object are similar for several benchmark classes, they are not constant in general. Again, this makes the model usable for better vs worse analysis, but not for predicting exact performance. The experiments carried out suggest that the exact garbage collection timing depends on a large number of factors, including (potentially proprietary) optimizations inside the virtual machines. Unless the virtual machine makes an extra effort to make the garbage collection timing predictable, it is unlikely that precise timing prediction could be made at modelingstage. Thebenchmark applications and benchmark methods developed throughout the project can be used to make a more precise prediction at later development stages. 7 Conclusion In this report, we have described a set of models we have examined in order to estimate the garbage collection performance. The models provide varying degrees of complexity, obviously tied to model precision, but also and that more importantly to the amount of input data that the model requires. Dealing with the amount of input data is important from practical perspective, not only because large input data sets require long computation times during prediction, but also because some types of data are very difficult to collect (especially lifetimes). We have developed a 14

15 Comparison of predicted and measured GC times GC times [s] avrora avrora avrora avrora avrora avrora fop fop fop fop fop fop pmd pmd pmd pmd pmd pmd tomcat tomcat tomcat tomcat tomcat tomcat xalan xalan xalan xalan xalan batik batik batik batik batik batik luindex luindex luindex luindex luindex luindex Figure 11: Comparison of measured and predicted Young garbage collection times for Da- Capo benchmarks, Sun VM. On the horizontal axis, the label format is: Benchmark- GC times per object GC time/object [s] Mark time per object Compact time per object Total time per object avrora 16MB avrora 17MB avrora 20MB avrora 24MB avrora 32MB avrora 64MB avrora 128MB batik 120MB batik 132MB batik 150MB batik 180MB batik 240MB batik 480MB batik 960MB fop 64MB fop 70MB fop 80MB fop 96MB fop 128MB fop 256MB fop 512MB pmd 64MB pmd 70MB pmd 80MB pmd 96MB pmd 128MB pmd 256MB pmd 512MB tomcat 24MB tomcat 26MB tomcat 30MB tomcat 36MB tomcat 48MB tomcat 96MB tomcat 192MB xalan 24MB xalan 26MB xalan 30MB xalan 36MB xalan 48MB xalan 96MB xalan 192MB Figure 12: Garbage collection phases times per object, IBM VM. On the horizontal axis there is combination of benchmark and heap size in MB. 15

16 tool to measure this data, however, it is still difficult to use because of significant space and time requirements. The precision of the models varies depending on many circumstances. In particular, the precision of the garbage collection time prediction is not sufficient for exact estimates, however, the shape of the prediction result graphs suggests that applications such as better vs worse analysis or identification of situations with significant garbage collection overhead would work reliably. In any case, the application of the models is not going to be a one-click affair. Apparently, the underlying technologies that the models are predicting are simply too complex to allow for predictions that would be reasonably generic and still completely automated. We believe, however, that it is reasonably easy to identify situations where taking the time to use the resource models pays off in terms of precision these situations include for example architectures with major computationally intensive components or architectures that are required to run in memory constrained environments. References [1] Babka V., Bulej L., Decky M., Kraft J., Libic P., Marek L., Seceleanu C., Tuma P.: Resource Usage Modeling, Q-ImPrESSDeliverable3.3,Feb2009. [2] Libi P., Tma P., Bulej L.: Issues in Performance Modeling of Applications with Garbage Collection, inproceedingsofquasoss2009,amsterdam,netherlands,acm, ISBN , pp. 3-10, Aug [3] Hertz, M., Blackburn, S.M., Moss, J.E.B, McKinley, K.S., Stefanovic, D.: Generating Object Lifetime Traces with Merlin, inacmtransactionsonprogramminglanguagesandsystems, Vol. 28, Issue 3, May [4] Blackburn S.M. et al.: The DaCapo Benchmarks: Java Benchmarking Development and Analysis, inproceedingsofoopsla2006.acm,2006. [5] Alpern, B. et al.: The Jalapeno Virtual Machine, IBM System Journal, Vol. 39, No 1,