COMP 412 FALL 2017 Sustainable Memory Use Allocation & (Implicit) Deallocation (mostly in Java) Copyright 2017, Keith D. Cooper & Zoran Budimlić, all rights reserved. Students enrolled in Comp 412 at Rice University have explicit permission to make copies of these materials for their personal use. Faculty from educational institutions may use these materials for nonprofit educational purposes, provided this copyright notice is preserved.
Preliminaries Today s lecture Automatic memory management Procedure-local memory management Garbage Collection Reference Counting Copying Collectors Short-Pause Collectors How you should program defensively now that you know Second part: Java memory model, allocation, & recycling How it works How it affects your code s runtime How you should program defensively now that you know COMP 412, Fall 2017 1
Where do objects live in Java? Fred Julia Kate System.out. println() Java World COMP 140 & COMP 215 Students encouraged to ignore the issue of where objects, variables, and methods live The implementation (Python or Java) takes care of these details Fundamentally, abstraction is a good thing right up to the point where it causes problems At some point in your Java career, performance will matter COMP 412 labs At that point, you need to pay attention to details Today s lecture is about details COMP 412, Fall 2017 2
Where do objects live? Fred Julia Kate System.out. println() The Java System maps Java World onto Processor Resources Processor has finite resources Java suggests that you have enough resources Mapping enough onto what s there is the job of the Java compiler and runtime (JVM) Java World 0 1 2 k Processor Core Processor Core RAM Processor Core Knowing how that mapping works can help you understand the behavior of your programs, and suggest ways to improve the program s behavior. COMP 412, Fall 2017 3
Fundamentals Class Point { public int x, y; public void draw(); } Class C { int s, t; public void m() { int a, b; Point p = new Point(); a = ; b = ; p.draw(); } } In the example, what needs storage? The two classes (Point & C) Point s local members (x, y, & draw) C s local members (s, t, & m) m s local variables (a, b, & p) A classic example COMP 412, Fall 2017 4
Fundamentals Class Point { public int x, y; public void draw(); } Class C { int s, t; public void m() { int a, b; Point p = new Point(); a = ; b = ; p.draw(); } } A classic example In the example, what needs storage? The two classes (Point & C) Point s local members (x, y, & draw) C s local members (s, t, & m) m s local variables (a, b, & p) Memory in the Java runtime is divided, broadly speaking, into a Heap and a collection of Stacks One heap per program (large) One stack per thread (smaller) STACK 0 STACK 1 Heap String Pool COMP 412, Fall 2017 5 p: a: b: STACK 2 STACK n Point Hello world! C new point
JVM Memory Layout Conceptually, Java memory is laid out along these lines Stacks Growth space for stacks Heap Globals Code When running code creates a variable, it goes into the thread s stack When running code creates a class or an object (e.g., with a new), it goes into the heap Code lives off to the left (might consider it part of the heap) So, can a program run out of heap space? Yes. Emphatically yes What happens? The runtime system tries to recycle space on the heap (too many news) COMP 412, Fall 2017 6
Sustainable Memory Management AKA Garbage Collection When the heap runs out of space, the system copes Scours the heap looking for objects that are no longer of interest Technical term is live An object is considered live iff it can be reached from the running code Start from all the names in the running code Variables are on the stack 1 Global names such as declared or imported classes Each object on the stack has a declaration which reveals its structure You can imagine chasing down chains of references to find all live objects 2 That s how it was done for a long time Modern garbage collectors are more nuanced They still start from the beginning: local & global names Most modern collectors are copying collectors 1 Locals of the current method are on the stack. Locals of the method that called it are below the current method on the stack. Locals of the method that called that method are below, and so on. That s why the runtime uses a stack 2 H. Schorr and W.M Waite, An efficient machine-independent procedure for garbage collection in various list structures, COMP Comm. 412, ACM, Supplemental 10 (8), 1967, pages Material 501 506 7
Taxonomy Garbage Collectors Reference- Counters Trace- Based Stop-the-World Short-Pause Mark-and- Sweep Mark-and- Compact Incremental Partial Basic Baker s Basic Cheney s Generational Train COMP 412, Fall 2017 8
Reference Counting Pros Root Object Simple implementation Conservative Easy to adapt to real-time A(1) B(2) Cons Fragmentation Self-referencing garbage Locality effects C(1) D(2) E(1) Overhead per pointer modification Memory overhead COMP 412, Fall 2017 9
Reference Counting Pros Root Object Simple implementation Conservative Easy to adapt to real-time A(0) B(2) Cons Fragmentation Self-referencing garbage Locality effects C(1) D(2) E(1) Overhead per pointer modification Memory overhead COMP 412, Fall 2017 10
Reference Counting Pros Root Object Simple implementation Conservative Easy to adapt to real-time B(1) Cons Fragmentation Self-referencing garbage Locality effects C(0) D(2) E(1) Overhead per pointer modification Memory overhead COMP 412, Fall 2017 11
Reference Counting Root Object Pros Simple implementation Conservative Easy to adapt to real-time D(1) B(1) E(1) Cons Fragmentation Self-referencing garbage Locality effects Overhead per pointer modification Memory overhead COMP 412, Fall 2017 12
Taxonomy Garbage Collectors Reference- Counters Trace- Based Stop-the-World Short-Pause Mark-and- Sweep Mark-and- Compact Incremental Partial Basic Baker s Basic Cheney s Generational Train COMP 412, Fall 2017 13
Mark and Sweep 1. Free = not holding an object; available for allocation. 2. Unreached = Holds an object, but has not yet been reached from the root set. 3. Unscanned = Reached from the root set, but its references not yet followed. 4. Scanned = Reached and references followed. 14
Marking 1. Assume all objects in Unreached state. 2. Start with the root set. Put them in state Unscanned. 3. while Unscanned objects remain do examine one of these objects; make its state be Scanned; add all referenced objects to Unscanned end; if they have not been there; 15
Sweeping Place all objects still in the Unreached state into the Free state. Place all objects in Scanned state into the Unreached state. To prepare for the next mark-and-sweep. Handles garbage cycles properly 16
Mark and Sweep Problem: Takes time proportional to the heap size. Because you must visit all objects to see if they are Unreached. Baker s algorithm keeps a list of all allocated chucks of memory, as well as the Free list. Key change: In the sweep, look only at the list of allocated chunks. Those that are not marked as Scanned are garbage and are moved to the Free list. Those in the Scanned state are put in the Unreached state. For the next collection. 17
Taxonomy Garbage Collectors Reference- Counters Trace- Based Stop-the-World Short-Pause Mark-and- Sweep Mark-and- Compact Incremental Partial Basic Baker s Basic Cheney s Generational Train COMP 412, Fall 2017 18
Issue: Why Compact? Compact = move reachable objects to contiguous memory. Locality --- fewer pages or cache-lines needed to hold the active data. Fragmentation --- available space must be managed so there is space to store large objects. 19
Basic Mark-and-Compact 1. Mark reachable objects as before. 2. Maintain a table (hash?) from reached chunks to new locations for the objects in those chunks. Scan chunks from low end of heap. Maintain pointer free that counts how much space is used by reached objects so far. 3. Move all reached objects to their new locations, and also retarget all references in those objects to the new locations. Use the table of new locations. 4. Retarget root references. 20
Example: Mark-and-Compact free 21
Example: Mark-and-Compact free 22
Example: Mark-and-Compact free 23
Example: Mark-and-Compact free 24
Example: Mark-and-Compact free 25
Example: Mark-and-Compact free 26
Example: Mark-and-Compact free 27
Example: Mark-and-Compact free 28
Example: Mark-and-Compact free 29
Taxonomy Garbage Collectors Reference- Counters Trace- Based Stop-the-World Short-Pause Mark-and- Sweep Mark-and- Compact Incremental Partial Basic Baker s Basic Cheney s Generational Train COMP 412, Fall 2017 30
Garbage Collection via Copying Copying Collectors A copying collector divides the heap into two or more pools New objects are allocated in the current pool When the current pool is full, execution pauses and the collector: copies all live objects from the current pool to the empty pool swaps the designations current and empty Unreachable objects are not copied, so the new pool has free space 1 4 3 2 5 6 Current Pool Empty Pool HEAP BEFORE COMP 412, Fall 2017 31
Garbage Collection via Copying Copying Collectors A copying collector divides the heap into two or more pools New objects are allocated in the current pool When the current pool is full, execution pauses and the collector: copies all live objects from the current pool to the empty pool swaps the designations current and empty Unreachable objects are not copied, so the new pool has free space 1 4 3 2 5 6 Current Pool Empty Pool HEAP BEFORE Objects left in the empty pool are discarded en masse Empty Pool 1 2 3 4 5 6 Current Pool HEAP AFTER COMP 412, Fall 2017 32
Cheney s Copying Allocator A shotgun approach to GC. 2 heaps: Allocate space in one, copy to second when first is full, then swap roles. Maintain table of new locations. As soon as an object is reached, give it the next free chunk in the second heap. As you scan objects, adjust their references to point to second heap. Advantages: O(live) complexity No unreachable garbage Very fast allocation Issues: Only ½ memory can be used Stop the world to perform garbage collection Objects have to be movable
The Object Life-Cycle Most objects die young. But those that survive one GC are likely to survive many. Tailor GC to spend more time on regions of the heap where objects have just been created. Gives a better ratio of reclaimed space per unit time. 34
Taxonomy Garbage Collectors Reference- Counters Trace- Based Stop-the-World Short-Pause Mark-and- Sweep Mark-and- Compact Incremental Partial Basic Baker s Basic Cheney s Generational Train COMP 412, Fall 2017 35
Generational Garbage Collection Divide the heap into partitions P 0, P 1, Each partition holds older objects than the one before it. Create new objects in P 0, until it fills up. Garbage collect P 0 only, and move the reachable objects to P 1. When P 1 fills, garbage collect P 0 and P 1, and put the reachable objects in P 2. In general: When P i fills, collect P 0, P 1,,P i and put the reachable objects in P(i +1). 36
Summary Reference counting: conservative, small pause, fragmented, cyclic garbage Mark & Sweep: no cyclic garbage but long pause, O(heap) Mark & Compact: no fragmentation, O(heap), long pause, needs movable objects Copying (Cheney s): fast allocation, needs movable objects, wastes ½ heap, O(live), long pause Generational: short(er) pause, needs write barrier, can be used for real time Concurrent: needs thread synchronization Incremental: run in parallel with execution, needs synchronization, can be used for real time Hybrid: copying for the youngest generation, mark&compact for older ones COMP 412, Fall 2017 37
Implications for Programming If you want performance, pay attention to garbage Collector locates live objects by walking out from variables When you are done with an object, set the variable to NULL Leaving the reference to the heap object will keep it live This is the takeaway message! Storage can leak, or become un-recyclable Leave a pointer to a large data structure on the stack, or in a global, or forgotten in another object, that happens to be live Leads to extra collections and, eventually, an out of memory error Stack Globals Hello world! String Pool Heap COMP 412, Fall 2017 38
Implications for Programming If performance really matters, pay attention to size of the pool Java uses a variant of a generational copying collector All new objects are allocated into Eden Eden is copied, when full, into one of Stable 0 or Stable 1 When Stable is too full, it is added to the Long Term Pool Minor collection Major collection swap minor major Eden Stable 0 Stable 1 Long Term Pool HEAP In COMP 412, we offered a 5% bonus for the fastest lab 1 in a language In the Java labs, the top three or four were separated by the behavior of the garbage collector The fastest lab had no major collections, & fewer minor collections COMP 412, Fall 2017 39
Does this stuff matter? Performance of One Student s Java Code, COMP 412, Lab 1 30.000 25.000 Standard Big Heap Small Heap 20.000 15.000 way too many collections 10.000 5.000 0.000-25,000.00 50,000.00 75,000.00 100,000.00 125,000.00 major collection (2x in speed) COMP 412, Fall 2017 40