Comparison of Garbage Collectors in Java Programming Language

Transcription

1 Comparison of Garbage Collectors in Java Programming Language H. Grgić, B. Mihaljević, A. Radovan Rochester Institute of Technology Croatia, Zagreb, Croatia Abstract - Considering the need for continuous and uninterrupted service of modern software applications, it is valuable to analyze how garbage collection (GC) algorithms are handling memory challenges. Widely adopted generalpurpose programming languages, like Java, represent an inevitable foundation for many modern application developments. In Java Platform, Standard Edition, and accompanying Java Virtual Machine (JVM), several GCs could be used. In the latest version of Java SE Development Kit (JDK) default GC was changed to Garbage-First (G1) GC, now becoming widely adopted in addition to previously used Parallel GC and Concurrent Mark & Sweep (CMS) GC. Since GC is a vital part of the JVM, changes and upgrades to its implementation, which reflect upon performance results, are properties worth exploring. Using benchmarks to create non-trivial memory pressures, and with extensive data monitoring, this paper analyzes insights gathered about critical performance factors across several GC algorithms. With the evaluation of benchmark elements, such as object allocations from young area to old area and the duration of the collection time, it was possible to compare GC behavior and assess the overall memory management. This paper presents our preliminary research performed in an academic environment on several benchmark cases and our conclusion about it. Keywords - Garbage Collector; Memory Management; Garbage-First; G1 I. INTRODUCTION Garbage collection (GC) is an automated process of memory management that scans through the heap space looking which objects are in use, and which are not. In some lower level procedural programming languages, such as C programing language, allocating memory is more of a manual process. However, in modern objectoriented programming (OOP) languages garbage collection is an essential part of memory management. For instance, garbage collectors in Java programming language are embedded as an important component of the runtime environment and Java Virtual Machine (JVM). The main purpose of the GC is to release from the memory the objects that are no longer used, and thus have no active references that would in some part of the application maintain a pointer to that object [1]. These sweeping operations are performed automatically at regular intervals and last for a certain amount of time but result in benefits of reclaimed memory where Java occupies only the necessary amount of space. At the same time, the released memory is available for further usage. Key components of the JVM that are essential to memory management [2], presented in Fig. 1, include: heap a space where objects reside in memory garbage collector an algorithm that directly manages the heap JIT compiler a mechanism in charge of making dynamic memory optimization decision while the application is running. With the current software architectural approaches, many modern software applications significantly depend on the continuous availability of the remotely deployed services. In production environments delaying the processing on the server side, due to any reason, might result from minor issues in reduced user experience, up to more severe critical system failures caused by memory overloads. Therefore, automated memory management, like GC, has to have very low overhead and require only the optimal amount of time for the sweeping operations. In Java, there is a JVM s "stop-the-world" concept that halts all of the non-critical application threads in order to perform GC, which results in interrupted tasks resuming only when GC processes have been completed. Such events occur no matter which GC algorithm is selected but require different durations and a distinctive number of garbage collections [3]. As part of the latest update of Java SE Development Kit (JDK), the default GC algorithm was switched to Garbage-First (G1) Garbage Collector [4]. The main motivation for this decision was the G1 s capability to limit the previously mentioned GC pause time to a predictable duration and to enhance the overall experience by operating concurrently, without requiring much larger heap size or processing power. Because it is difficult to successfully achieve a setting in which memory management is handled in an optimal manner, for this comparison we ve selected three widely used garbage collectors, included in the latest version of JDK. Currently, there several GCs available in the JDK: Serial Garbage Collector Parallel Garbage Collectors in several versions Concurrent Mark Sweep (CMS) Collector Garbage First (G1) Garbage Collector MIPRO 2018/SP 1785

2 Figure 1. Visual representation of the Java Virtual Machine with key memory components (bolded), according to Oracle [2]. For the purpose of this research, we selected Parallel GC, CMS GC, and G1 GC, which were taken into the consideration when performing benchmarks of the nontrivial memory pressures, created by several applications included in the DaCapo benchmark [5]. As the goal of this research was to evaluate how the algorithms perform with out-of-the-box settings, only the utmost tuning of the JVM s parameters has been performed. II. BACKGROUND A. Java Garbage Collectors In September 2017 Java SE Development Kit was updated to the anticipated version 9, delayed mainly because of the Jigsaw Project. There were many significant improvements included in this release that the community was expecting. One of the most anticipated changes was the JDK s modular system aka Project Jigsaw [11]. However, a more interesting feature for our research was the previously mentioned change of the default garbage collector to the freshly updated version of G1 GC algorithm. With G1 the main idea was to distance from the maximization of the throughput. Thus, the primary goal, for the G1, was to achieve the lower pause times, i.e., to minimize the periods when application threads are not progressing. This goal was not largely prioritized in the previously used algorithms in Parallel GC and CMS GC. As Java does not especially denote the amount of memory necessary for the applications, there is no suitable way of performing manual memory cleaning in the source code. However, there is a possibility of dereferencing objects and setting their reference to null. In this case, previous objects reference is lost, and memory is explicitly released. The problem could occur when developers invoke garbage collection method like System.gc() method. Explicitly calling the garbage collection method might decrease performance drastically, and it is not advisable to use it. With these premises in mind, Java organized internal mechanisms to handle memory issues automatically, in a variety of ways. Firstly, Parallel GC [10] uses larger amounts of processing power in order to perform object marking phase in parallel. This approach is very useful as it is faster than a Serial GC would be, which uses only a single thread [3]. But, successfully processing numerous tasks in parallel is only possible with machines that have a significant number of processing cores and substantial amounts of memory. Parallel GC is a typical example of a throughput GC. Secondly, CMS GC [1] is significantly more complex at its core. Despite Parallel GC uses multiple amounts of threads its actions are still somewhat serial in nature, whereas CMS performs both memory sweeping concurrently with the running application. As a result, operations are executed much faster, which is critical when there is a high demand for availability. However, in order to provide such short delay times, CMS can cause memory fragmentation and uses more processing power than any other GC [2]. Lastly, G1 GC functions similarly as CMS. It first starts by doing global marking phase to determine the importance of objects. After global marking, G1 knows which areas are full of reclaimable garbage objects that would yield a lot of free space. By concentrating on such garbage areas first, the algorithm s name was formed. Moreover, G1 uses user-defined acceptable delay times, so that heap targeting is performed in an optimal and predictable manner [4]. Specific characteristics of garbage collectors are compared and presented in Table I. Even though G1 is well tested, making it a default GC might produce some new concerns. Also, if the assumption that latency is more important than throughput proves rather incorrect, there could be a reasonable doubt to reconsider the G1 s default position MIPRO 2018/SP

3 TABLE I. CHARACTERISTICS OF GARBAGE COLLECTORS INCLUDED IN THIS RESEARCH BASED ON A STUDY BY CARPEN-AMARIE, MARLIER, FELBER, AND THOMAS [6]. Old generation Young generation GC Concurrent Concurrent Concurrent Concurrent Parallel Compacting Parallel Copying Marking Compacting Marking Copying Parallel CMS G1 B. The Heap The working hypothesis of the generational garbage collectors is that most of the objects soon become unreferenceable and that there is a rarity of cases when it is necessary to reference specific older object back to the newly created ones [3]. This property is specified in the literature as the "weak generational hypothesis" [7]. Therefore, in the majority of cases, the heap is divided into three major sections of total fixed size. Firstly, the young space is where newly created objects are located. When the allocated space of this generation runs out, minor garbage collection is executed. Even though all minor collections are relatively fast, these still require non-critical application threads to be stopped. Secondly, the old generation stores longer surviving objects, which get placed in this space when certain age criteria are met. When old generation space gets filled, major garbage collections are executed. In general, major GC operations are much slower than minor collections and result in longer pause times. Thirdly, permanent generation gets populated at runtime by metadata necessary to denote various JVM components. As part of the full garbage collection, JVM can remove information from the permanent collection if it no longer finds it important [2]. In contrast to the traditional fixed heaps, G1 takes a completely different architectural approach, and the heap is portioned in a large number of small areas, with the distinctive types of areas, as presented in Fig. 2. Eden space represents a young area where new objects get allocated. Slightly older space with objects that have existed for some period of time is called Survivor space, and the old generation space is where long existing objects are persisting. All allocated memory areas do not have a fixed size, which provides significantly greater flexibility, and creates an opportunity for more suitable memory management environment [4]. C. The Benchmark DaCapo suite represents Java memory and architecture benchmarking tool [5]. It consists of a set of real-world open-source applications. Each application, when run, will create non-trivial memory loads. By measuring the performance details, DaCapo suits offers great insights into the operating details of the Java VM. In this research, we were using DaCapo version 9.12-bach. Only a set of the following applications was included in this research paper. There are more available but were not considered as part of this preliminary research results. The information about the applications included was gathered from the benchmark provider [3], as well as from the related study by Carpen-Amarie, Marlier, Felber, and Thomas [6]. The detailed information regarding the included applications are the following: avrora a single client threaded application (internally multithreaded) in charge of simulating a number of programs run on a grid of AVR microcontrollers fop a single threaded application which takes an XSL-FO file, parses it, and creates a PDF file tomcat a multi-threaded application that sends multiple requests to the server for processing h2 a multi-threaded application which executes JDBC like in-memory benchmarks by executing transactions against a banking model pmd a single thread application (internally multi-threaded) analyses Java classes for source code issues xalan a multi-thread application which transforms XML documents into HTML Figure 2. Visualization of the architecture of traditional heap space and G1 heap space. MIPRO 2018/SP 1787

4 The remaining applications included in the benchmark are batik, eclipse, jython, luindex, lusearch, tradesoap, sunflow, and tradebeans. By default, the selected application from the suit is ran only once. But, it is possible to set via the parameter the desired number of iterations. However, if there is an N amount of iterations passed, the benchmark considers N-1 iterations to be warmup rounds whose purpose is to stabilize JIT compiler and GC ergonomics [8]. The final round is considered to be the actual benchmark test. By default, the benchmark performs system GC after every iteration, but this feature can be disabled by passing additional parameters. III. TESTING ENVIRONMENT A. Testing Environment Setup The environment in which this research has been conducted consists of several elements, out of which the most important is Java Virtual Machine (JVM). The used JVM is Java Hotspot 64-bit Server VM of build , mixed mode. Java SE Runtime Environment (JRE) is also of a build By default, the JVM s GC of choice was G1, but for the testing purposes, additional GC s included have been respectfully activated. The operating system used as hosting environment is Apple s macos version (High Sierra), and all testing was started and performed directly on Apple s Terminal application version 2.8. In order to measure the results of the performed tests, we used Visual VM software version 1.4, supported by Oracle. Visual VM is a simple, but powerful graphical tool, which enables inspection of various performance factors in Java-based applications. However, to successfully monitor precise details about garbage collection, Visual VM needs to be enriched by installing Visual GC plugin [9]. With this plugin, it is possible to attach Visual GC to the running application, and to collect data about garbage collectors, class loaders, and HotSpot s compiler performance. B. Test Cases Our testing setup included a scenario consisting out of 10 iterations, where each scenario was repeated two times. In the end, the results were put into a table so that average duration time and the number of GC collections could be calculated. This course of actions was repeated for each of the respected garbage collector algorithms included in this research. The size of the heap was set in every case to a fixed value of 1 GB. This feature provided greater transparency for the results as limiting the heap memory prevented some of the applications to use more memory than the others. An example of test case command is presented in Fig. 3. java -Xms1G -Xmx1G -jar dacapo-9.12-bach.jar h2 -n 5 --no-pre-iteration-gc Figure 3. The example of the command that would execute 5 iterations of h2 application with limited memory heap to 1 GB, and no system GC between iterations. C. Limitations Although the objective of this preliminary research was to establish a strong line of comparison between different GCs, there are still some limiting factors that necessarily need to be taken into consideration. Namely, the present limitations are: Limited computing power of an academic setting. Overhead created by the measuring tool. Modest number of iterations and overall testing repetitions due to the limited amounts of memory available. Due to the previously mentioned limitations, tested garbage collection algorithms might behave differently in a large enterprise setting, which has substantially more computing resources. IV. RESULTS Preliminary results obtained by executing the described test setup have yielded the results (presented in Table II and Table III) that do not show an obvious advantage for the G1 GC algorithm. TABLE II. TOTAL NUMBER OF GARBAGE COLLECTIONS Benchmarks G1 GC CMS GC Parallel GC avrora fop h pmd xalan tomcat TABLE III. TOTAL DURATION OF GARBAGE COLLECTIONS IN MILLISECONDS Benchmarks G1 GC CMS GC Parallel GC avrora 37,44 45,22 36,66 fop 634,39 279,90 412,55 h pmd 1409,0 1104,8 685,2 xalan 211,63 307,75 237,41 tomcat From the fig. 4, it is obvious that in four out six test cases G1 GC performed in a similar or sometimes even worse manner than the other compared algorithms. With the applications avrora and pmd the number of garbage collections was the same across every tested algorithm. But, with the applications fop and h2 G1 GC executed substantially more garbage collections than Parallel GC or CMS GC. However, in the case of the multi-threaded xalan and multi-threaded tomcat applications, G1 performed the best. G1 GC outperformed Parallel GC and CMS, which conducted almost double the number of collections and ended with requiring from 10% up to 30% more time to finish executing MIPRO 2018/SP

5 Figure 4. Overall numbers of executed garbage collections per application across three GC algorithms. The garbage collection execution time visualization presented in fig. 5 depicts overall time durations for the performed tests. In every benchmark application case, except for the tomcat and xalan application, when compared with CMS GC and Parallel GC, G1 GC utilized the same amount time or more. The most perceptible difference can be noticed in the test case with the h2 application. As presented in Table III, proportional to the larger number of collections, G1 GC operations required 3992 milliseconds, while Parallel GC and CMS required 1724 and 2060 milliseconds, respectfully. Regarding the CPU usage, in most of the cases, there was no critical difference between the tested algorithms. Parallel GC and CMS utilized approximately between 5% to 8% more processing resources than G1 GC. However, each algorithm produced almost the same CPU usage spikes at the similar point in time during the test execution. Also, CMS GC and Parallel GC demonstrated slightly more unpredictable rate of garbage collections, when compared to rather consistent intervals of the G1 algorithm. V. DISCUSSION Although it might seem that G1 Garbage Collector did not present any obvious advantages, as the results were mostly equal or worse, we have noticed interesting facts that possibly yield towards a stirring discovery. In order to interpret it, it is necessary to place a greater emphasis on the architecture of the applications involved in the testing scenarios. In an interaction with fully single threaded applications like fop, and externally single threaded apps lik avrora or pmd, G1 GC performed mostly the same or slightly worse than other two garbage collectors. This could indicate that switching to G1 GC is probably not the Figure 5. Overall execution time of garbage collections per application expressed in milliseconds. best possible option for such environments. Likewise, when G1 GC was put to the test where it was necessary to interact with the embedded h2 database, G1 GC performed significantly worse than any other algorithm involved. But, as this test was the only one that included extensive database communication with a complex banking model, it might not be legitimate to generalize a conclusion based on such single case. Still, an interesting pattern can be noticed in the example of the last two conducted tests. With fully multithreaded applications like xalan and tomcat, G1 GC was an absolute winner. In these tests other garbage collectors required almost double the time and performed substantially larger amount of GC operations. Even more interesting is the fact that in cases with avrora or pmd where their internal architecture and processing is organized in a multi-threaded way, G1 GC performed equally as other algorithms. This demonstrates that in circumstances when multi-threaded systems are involved it might be worthy to test the existing configurations further, and to evaluate if keeping the traditional, generational, garbage collectors yields negative performance effects. Also, by investing more effort into adjusting the large amount of available Java virtual machine parameters, which was out of scope of this research, the positive outcome might be significantly more numerous. An interesting research conducted on this topic by Lengauer and Mössenböck [12] indicates that it is even possible to use an iterative approach to search and custom tailor JVM configuration, so it brings additional performance enhancements. Combining such approach with utilization of G1 s scalable design that splits heap into small manageable regions [13], results is a clear and viable reason to engage into optimization and bring increased quality of service to system s users. VI. CONCLUSION With this version of the G1 GC, and out of the box settings, it might be hard to notice the benefits that have been presented as part of this Java release. Parallel GC and CMS GC have proved to be at least equally good in most of the scenarios, which could indicate that any large production software systems might have to carefully consider widely adopting the G1 garbage collector. But, by possibly optimizing applications from the beginning of the development process, could result in better utilization of the benefits the G1 Garbage Collector brings. Furthermore, if the system at hand is operating with a large amount of threads, it might be useful to create a replica of it, in a testing environment, and to see how G1 GC performs. The justification for such a decision can be based on convincingly good results of G1 GC shown in a multi-threaded environment. According to the presented results, simple utilization of the default settings of the G1 GC might be the reason why critical performance improvements were not consistently observed across all of the tested scenarios. This fact potentially implies that the advanced tuning of the JVM parameters, as well as parameters of the garbage collector itself, might yield much more successful MIPRO 2018/SP 1789

6 achievements in less threaded environments, and environments that include extensive database interactions. Nevertheless, G1 GC algorithm with its stable and predictable usage patterns and a possibility to specify desired pause times represents a technology that in the long run is worth exploring. VII. FUTURE WORK The current research was of relatively narrow scope and leaves much more space for further research. Especially interesting would be to perform a significant amount of much more complex tests in large multithreaded environments. Such results could possibly yield more precise outcomes that would enable to deduce if wide enterprise adoption of G1 is a viable cause. Also, an interesting aspect that was not overly tested in this research is the implantation of benchmarks that are more focused on interaction with a database. It would be interesting to see how memory management is conducted when interacting with commercial databases like Oracle database, instead of using embedded h2 database. Finally, our research could be extended by focusing on a single application, but in a more detailed approach. By fine-tuning all of the applicable JVM parameters, it could be seen how the true potential of G1 GC algorithm looks like. [11] Project Jigsaw [Online]. Available: s/jigsaw/. [Accessed: 04-Feb-2018]. [12] P. Lengauer and H. Mössenböck, "The taming of the shrew," Proc. 5th ACM/SPEC Int. Conf. on Performance Engineering - ICPE '14, [13] C. Hunt, M. Beckwith, P. Parhar and B. Rutisson, Java performance companion, 1st ed. Crawfordsville, Indiana: Pearson Education, Inc., REFERENCES [1] R. Jones and R. Lins, Garbage collection: Algorithms for Automatic Dynamic Memory Management. 1st ed., Chichester: John Wiley, [2] T. Lindholm, F. Yellin, G. Bracha and A. Buckley, The Java Virtual Machine Specification. 7th ed. Upper Saddle River, NJ: Addison-Wesley, [3] R. Jones, A. Hosking, and E. Moss. The garbage collection handbook: the art of automatic memory management. Chapman & Hall/CRC, 1st edition, [4] D. Detlefs, C. Flood, S. Heller and T. Printezis, "Garbage-first garbage collection," Proc. 4th Int. Symp. on Memory Management - ISMM '04, pp , [5] S. Blackburn et al., "The DaCapo Benchmarks: Java Benchmarking Development and Analysis," ACM SIGPLAN Notices, vol. 41, no. 10, pp. 169, [6] M. Carpen-Amarie, P. Marlier, P. Felber, and G. Thomas, "A performance study of Java garbage collectors on multicore architectures," Proc. Sixth Int. Workshop on Programming Models and Applications for Multicores and Manycores - PMAM '15, pp , [7] "Memory Management in the Java HotSpotTM Virtual Machine," Sun Microsystems, [Online]. Available: ymanagement-whitepaper pdf. [Accessed: 12-Jan-2018]. [8] P. Lengauer, V. Bitto, H. Mössenböck, and M. Weninger, "A Comprehensive Java Benchmark Study on Memory and Garbage Collection Behavior of DaCapo, DaCapo Scala, and SPECjvm2008," Proc. 8th ACM/SPEC on Int. Conf. on Performance Engineering - ICPE '17, pp. 3-14, [9] "Visual GC - Visual Garbage Collection Monitoring Tool," Oracle.com, [Online]. Available: com/technetwork/java/visualgc html. [Accessed: 15-Jan- 2018]. [10] F. Siebert, "Limits of parallel marking garbage collection," Proc. 7th Int. Symp. on Memory Management - ISMM '08, pp , MIPRO 2018/SP