520 Principles of Programming Languages. Memory Management. Memory Management... 35: Garbage Collection

Transcription

1 Dynmic Memory Mngement 50 Principles of Progrmming Lnguges 35: Grbge Collection Christin Collberg Deprtment of Computer Science University of Arizon The run-time system linked in with the generted code should contin routines for lloction/delloction of dynmic memory. Pscl, C, C++, Modul- Explicit delloction of dynmic memory only. I.e. the progrmmer is required to keep trck of ll llocted memory nd when it s sfe to free it. Eiffel Implicit delloction only. Dynmic memory which is no longer used is recycled by the grbge collector. Ad Implicit or explicit delloction (implementtion defined). Modul-3 Implicit nd explicit delloction (progrmmer s choice). [1] 50 [] Memory Mngement Memory Mngement... In lnguge such s C or Pscl, there re three wys to llocte memory: 1. Sttic lloction. Globl vribles re llocted t compile time, by reserving. Stck lloction. The stck is used to store ctivtion records, which holds procedure cll chins nd locl vribles. 3. Dynmic lloction. The user cn crete new memory t will, by clling new or (in unix) mlloc procedure. The compiler nd run-time system divide the vilble ddress spce (memory) into three sections, one for ech type of lloction: 1. The sttic section is generted by the compiler nd cnnot be extended t run-time. Clled the uninitilized dt section in unix s.out.. The stck. The stck grows nd shrinks during execution, ccording to the depth of the cll chin. Infinite recursion often leds to stck overflow. Lrge prmeters cn lso result in the progrm running out of stck spce. 3. The hep. When the progrm mkes request for more dynmic memory (by clling mlloc, for exmple), suitble chunk of memory is llocted on the hep.

2 Memory Mngement... Interfce to Dynmic lloction Sttic lloction Globl vribles Stck lloction Procedure cll chins, Locl vribles. Dynmic lloction NEW, mlloc, On the hep. Progrm Code Initilized Dt (strings,rels...) Uninitilized Dt (Globl Vribles) Stck ep C, C++: chr* mlloc(size) nd free(chr*) re stndrd librry routines. Pscl: new(pointer vr) nd dispose(pointer vr) re builtin stndrd procedures. Jv: new(clss nme) is stndrd function. LISP: cons cretes new cells: ed Til (cons (b c)) b b c null (b c) ( b c) c null [5] 50 [6] Explicit Delloction Implicit Delloction Pscl s new/dispose, Modul- s ALLOCATE/DEALLOCATE, C s mlloc/free, C++ s new/delete, Ad s new/unchecked delloction (some implementtions). Problem 1: Dngling references: p=mlloc(); q=p; free(p);. Problem : Memory leks, ep frgmenttion. Smll cell free list: ep: LISP, Prolog Equl-sized cells; No chnges to old cells. Eiffel, Modul-3 Different-sized cells; Frequent chnges to old cells. When do we GC? Stop-nd-copy Perform GC whenever we run out of hepspce (Modul-3). Rel-time/Incrementl Perform prtil GC for ech pointer ssignment or new (Eiffel, Modul-3). Concurrent Run the GC in seprte process. Lrge cell free list: 18 51

3 Implicit Delloction... Finding the Object Grph Frgmenttion Compct the hep s prt of the GC, or only when the GC fils to return lrge enough block. Algorithms: Reference counts, Mrk/ssweep, Copying, Genertionl. Finding the roots: The dynmic objects in progrm form grph. Most GC lgorithms need to trverse this grph. The roots of the grph cn be in 1. globl vribles. registers 3. locl vribles/forml prmeters on the stck. ence, the compiler must communicte to the GC which registers/vribles contin roots. [9] 50 [10] Finding the Object Grph... Finding the Object Grph... Finding internl pointers: Structured vribles (rrys, records, objects) my contin internl pointers. These must be known to the GC so tht it cn trverse the grph. ence, the compiler must communicte to the GC the type of ech dynmic object nd the internl structure of ech type. Finding the beginning of objects: Wht hppens if the only pointer to n object points somewhere in the middle of the object? We must either be ble to find the beginning of the object, or mke sure the compiler does not generte such code. AR for P AR for Q AR for R Execution Stck templte A %r8 C Globls G ep O: templte 8: O: templte 8: 1: O: templte 8: Ptrs: [1] Size:96 Templte for P Ptrs: [0] Size:4 Templte for MAIN Ptrs: [4,1] Size:3 Templte for T1 Ptrs: [8] Size: 4 Templte for T

4 Pointer Mps Pointer Mps... The internl structure of ctivtion records & structured vribles is described by run-time templtes. Every run-time object hs n extr word tht points to type descriptor (or Templte), structure describing which words in the object re pointers. This mp is constructed t compile-time nd stored stticlly in the dt segment of the executble. When the GC is invoked, registers my lso contin vlid pointers. The compiler must therefore lso generte (for every point where the GC my be clled) pointer mp tht describes which registers hold live pointers t this point. For this reson, we usully only llow the GC to run t certin points, often the points where new is clled. We must lso provide pointer mps for every function cll point. A function P my cll Q which clls new which invokes the GC. We need to know which words in P s ctivtion record tht t this point contin live pointers. [13] 50 [14] Pointer Mps... Algorithm: Reference Counts ow does the GC look up which pointer mp belongs to prticulr cll to procedure P t prticulr ddress? The pointer mps re indexed by the return ddress of P! So, to trverse the stck of ctivtion records, the GC looks t ech frme, extrcts the return ddress, finds the pointer mp for tht ddress, nd extrcts ech pointer ccording to the mp. An extr field is kept in ech object contining count of the number of pointers which point to the object. Ech time pointer is mde to point to n object, tht object s count hs to be incremented. Similrly, every time pointer no longer points to n object, tht object s count hs to be decremented. When we run out of dynmic memory we scn through the hep nd put objects with zero reference count bck on the free-list. Mintining the reference count is costly. Also, circulr structures (circulr linked lists, for exmple) will not be collected.

5 Algorithm: Reference Counts... Algorithm: Reference Counts... Every object records the number of pointers pointing to it. When pointer chnges, the corresponding object s reference count hs to be updted. GC: reclim objects with zero count. Circulr structures will not be reclimed. Live cells Globl Vribles b Grbge (will be reclimed) 1 Grbge (won t be reclimed) e p NEW(p) is implemented s: mlloc(p); p.rc := 0; p.next:=q is implemented s: z := p.next; if z nil then z.rc--; if z.rc = 0 then reclim z endif endif; p.next := q; q.rc++; This code sequence hs to be inserted by the compiler for every pointer ssignment in the progrm. This is very expensive. [17] 50 [18] Algorithm: Mrk nd Sweep Algorithm: Mrk nd Sweep... The bsic ide behind Mrk-nd-Sweep is to trverse nd mrk ll the cells tht cn be reched from the root cells. A root cell is ny pointer on the stck or in globl memory which points to objects on the hep. Once ll the live cells (those which re pointed to by globl vrible or some other live cells) hve been mrked, we scn through the hep nd seprte the live dt from the grbge. If we re deling with equl size objects only (this is the cse in LISP, for exmple) the we scn the hep nd link ll the unmrked objects onto the free list. At the sme time we cn unmrk the live cells. If we hve cells of different sizes, just linking the freed objects together my result in hep frgmenttion. Insted we need to compct the hep, by collecting live cells together in contiguous memory re on the hep nd doing the sme with the grbge cells in nother re.

6 Algorithm: Mrk nd Sweep... Mrking Phse Mrking Phse: 1. Mrk ll objects unmrked.. Find ll roots, i.e. hep pointers in stck, regs & globls. 3. Mrk rechble blocks using depth first serch strting t the roots. () DFS my run out of stck spce! (b) Use non-recursive (Deutsch-Schorr-Wite) DFS. Scnning Phse: sme-size-cells Scn hep nd put un-mrked (non-rechble) cells bck on free-list. different-size-cells Compct the hep to prevent frgmenttion. [1] 50 A stright-forwrd implementtion of mrk nd sweep my run into memory problems itself! A depth-first-serch mkes use of stck, nd the size of the stck will be the sme s the depth of the object grph. Remember tht the stck nd the hep shre the sme memory spce, nd my even grow towrds echother. So, if we re out of luck we might run into this sitution: the hep is full (otherwise we wouldn t be gc:ing!), the object grph is deep, we run out of stck spce during the mrking phse. We re now out of memory lltogether. Difficult to recover from! [] Mrking Phse... Fortuntely, there is smrt lgorithm for mrking in constnt spce, clled the Deutsch-Schorr-Wite lgorithm. Actully, it ws developed simultneously by Peter Deutsch nd by erbert Schorr nd W. M. Wite. The bsic ide is to store the DFS stck in the object grph itself. When new node (object) is encountered 1. we set the mrked -bit to 1,. the node (object) is mde to point to the previous node, 3. two globl vribles current nd previous re updted. current points to the current cell, previous to the previously visited cell. Mrking: Look M, No Stck! Use pointer reversl to encode the DFS stck in the object grph itself. When the DFS reches new cell, chnge pointer in the cell to point bck to the DFS prent cell. When we cn go no deeper, return, following the bck links, restoring the links. (1) (1) M current () (4) / (3) () (3) / B B (4) M M (5) / / (5) / / M / / B = Bck Pointer M = Mrked previous

7 Sweeping: Compction Copying Collection dt1 dt dt3 A B C D E dt1 dt dt3 Compction A B C D E 1. Clculte the forwrding ddress of ech cell.. Store the forwrding ddress of cell B in B.forw ddr. 3. If p points to cell B, replce p with B.forw ddr. 4. Move ll cells to their forwrding ddresses. F F Even if most of the hepspce is grbge, mrk nd sweep lgorithm will touch the entire hep. In such cses it would be better if the lgorithm only touched the live objects. Copying collection is such n lgorithm. The bsic ide is: 1. The hep is divided into two spces, the from-spce nd the to-spce.. We strt out by llocting objects in the from-spce. 3. When from-spce is full, ll live objects re copied from from-spce to to-spce. 4. We then continue llocting in to-spce until it fills up, nd new GC strts. [5] 50 [6] Copying Collection... Copying Collection... An importnt side-effect of copying collection is tht we get utomtic compction fter collection to-spce consists of the live objects in contiguous piece of memory, followed by the free spce. This sounds relly esy, but : We hve to trverse the object grph (just like in mrk nd sweep), nd so we need to decide the order in which this should be done, depth-first or bredth-first. DFS requires stck (but we cn, of course, use pointer reversl just s with mrk nd sweep), nd BFS queue. We will see lter tht encoding queue is very simple, nd hence most implementtions of copying collection mke use of BFS. This sounds relly esy, but An object in from-spce will generlly hve severl objects pointing to it. So, when n object is moved from from-spce to to-spce we hve to mke sure tht we chnge the pointers to point to the new copy.

8 Copying Collection... Copying Collection... Mrk-nd-sweep touches the entire hep, even if most of it is grbge. Copying collection only touches live cells. Copying collection divides the hep in two prts: from-spce nd to-spce. to-spce is utomticlly compcted. ow to trverse object grph: BFS or DFS? ow to updte pointers to moved objects? 1. Strt llocting in from-spce. Algorithm:. When from-spce is full, copy live objects to to-spce. Trversing the Object Grph: Most implementtions use BFS. Use the to-spce s the queue. Updting (Forwrding) Pointers: When n object is moved its new ddress is stored first in the old copy. from spce to spce Exmple: GC from spce to spce 3. Now llocte in to-spce. [9] 50 roots: [30] roots: Copying Collection Algorithm Copying Collection Exmple... (A) 1. scn := next := ADDR(to-spce) [scn next] hold the BFS queue. from spce A roots from spce A roots to spce D Objects bove scn point into to-spce. Objects between scn nd next point into from-spce.. Copy objects pointed to by the root pointers to to-spce. 3. Updte the root pointers to point to to-spce. B C D B C D B next scn 4. Put ech object s new ddress first in the originl. E E 5. Repet (recursively) with ll the pointers in the new to-spce. F F () Updte scn to point pst the lst processed node. (b) Updte next to pointe pst the lst copied node. Continue while scn < next.

9 Copying Collection Exmple... (B) Genertionl Collection from spce A B C D E roots to spce D B next scn from spce A B C D E roots to spce D B E next scn Works best for functionl nd logic lnguges (LISP, Prolog, ML,... ) becuse 1. they rrely modify llocted cells. newly creted objects only point to older objects ((CONS A B) cretes new two-pointer cell with pointers to old objects), 3. new cells re shorter lived thn older cells, nd old objects re unlikely to die nytime soon. F F [33] 50 [34] Genertionl Collection... Genertionl Collection... Genertionl Collection therefore 1. divides the hep into genertions, G 0 is the youngest, G n the oldest.. lloctes new objects in G GC s only newer genertions. We hve to keep trck of bck pointers (from old genertions to new). Functionl Lnguge: (cons (b c)) t 1 : x new (b c); t : y new ; t 3 : return new cons(x, y) A new object (creted t time t 3 ) points to older objects. Object Oriented Lnguge: t 1 : t : t 3 : T new Tble(0); x new Integer(5); T.insert(x); A new object (creted t time t ) is inserted into n older object, which then points to the news object.

10 Genertionl Collection... Genertionl Collection After GC(G 0 ) Remembered Set: Roots: Remembered Set: Roots: G G 1 G 0 G 1 G G 0 [37] 50 [38] Genertionl Collection... Since old objects (in G n G 1 ) re rrely chnged (to point to new objects) they re unlikely to point into G 0. Apply the GC only to the youngest genertion (G 0 ), since it is most likely to contin lot of grbge. Use the stck nd globls s roots. There might be some bck pointers, pointing from n older genertion into G 0. Mintin specil set of such pointers, nd use them s roots. Occsionlly GC older (G 1 G k ) genertions. Use either mrk-nd-sweep or copying collection to GC G 0. Remembering Bck Pointers Remembered List After ech pointer updte x.f :=, the compiler dds code to insert x in list of updted memory loctions: x.f := x.f := ; insert(updtedlist, x);

11 Remembering Bck Pointers Remembering Bck Pointers... Remembered Set As bove, but set bit in the updted object so tht it is inserted only once in the list: x.f := x.f := ; IF NOT x.inserted TEN insert(updtedlist, x); x. inserted := TRUE; ENDIF Crd mrking Divide the hep into crds of size k. Keep n rry dirty of bits, indexed by crd number. After pointer updte x.f :=, set the dirty bit for crd c tht x is on: x.f := x.f := ; dirty[x div k ] := TRUE; [41] 50 [4] Remembering Bck Pointers... Remembering Bck Pointers... Pge mrking I Similr to Crd mrking, but let the crds be virtul memory pges. When x is updted the VM system utomticlly sets the dirty bit of the pge tht x is on. We don t hve to insert ny extr code! Pge mrking II The OS my not let us red the VM system s dirty bits. Insted, we write-protect the pge x is on. On n updte x.f := protection fult is generted. We ctch this fult nd set dirty bit mnully. We don t hve to insert ny extr code!

12 Uncoopertive Lnguges Uncoopertive Lnguges... There is some informtion which is necessry in order to perform utomtic memory mngement: 1. We need to find the roots of the object grph, i.e. the pointers from the stck, registers, or globl vribles which point to objects on the hep.. We need to know the size, the beginning, nd end of ech object. 3. For ech object we need to find which of its fields re pointers. C nd C++ don t seprte sfe nd unsfe fetures (such s ddress nd bit mnipultion) which re sometimes needed in systems progrmming. Modul-3 hs similr unsfe fetures s C nd C++ but they cn be encpsulted into unsfe modules, which don t mess up the sfety of the min (sfe) prt of the progrm. Unfortuntely, some lnguges hve been designed so tht it is impossible to determine this informtion. C nd C++ re the two most populr such lnguges. [45] 50 [46] Uncoopertive Lnguges... Most GC lgorithms ssume tht there is lwys pointer to the beginning of every object. Depending on the code genertor, tht my or my not be true. f(g,s) chr (*g)(); chr * s; { int i; int l = strlen(s); for (i = 0; i < l; i++) s[i] = (*g)(s[i]); } There my be no pointer to s[0]. Uncoopertive Lnguges... We need to know 1. the roots of the object grph.. the size, the beginning, nd end of ech object. 3. which object fields re pointers. Finding Roots: Foo* f = new foo; // f = 0x53f36 f = NULL; // f* is grbge int i = 0x53f36; // points to f...

13 Uncoopertive Lnguges... Conservtive GC Finding the beginning: chr* str = new chr[6]; strcpy(str, "This is string"); str += 10; // Only ptr to str... Finding pointers: union Unsure {chr* str; int i} x; Works OK for uncoopertive lnguges (C, C++) where we cn t distinguish between pointers nd integers. Sometimes fils to reclim ll grbge. Min Ides: Allocte memory in chunks. Ech chunk holds collection of objects of certin size (i.e. it s esy to find the strt of objects). Chunks re numbered. A pointer consists of 1 bits of chunk number (C) + 0 bits of offset within the chunk (O). [49] 50 [50] Conservtive GC... Conservtive GC... To check whether vlue V = (C, O) is pointer to some object we check tht 1. ep-bottom V ep-top,. FirstChunk# C LstChunk# 3. the offset O is multiple of the object size in chunk C. Chunk List: Chunk 1: 8 bytes ech size = 8 mrk bits..... Objects 4K bytes Chunk 7: size mrk = 3 bits 3 bytes ech Objects V: Chunk number Offset within chunk (1 bits) (0 bits)

14 Unobrusive Grbge Collection Incrementl GC GC Requirements: btch progrms: We wnt short totl GC time. interctive progrms: We wnt unnoticble GCs. Unobtrusive GC: Incrementl Collection Do little GC-work every time n object is llocted, or pointer is chnged. Concurrent Collection Run the collector nd the progrm in different processes, or on different processors. Use copying collection, but rther thn stop when you run out of memory nd then do ll the GC work in one shot, do little bit whenever pointer vrible is referenced or when new object is llocted. We strt out by forwrding (copying) the objects pointed to by globl vribles. Then, insted of continuing forwrding recursively, we resume the progrm. Every time pointer is referenced we check to see whether it is pointing into from-spce. If it is, we forwrd tht object too. [53] 50 [54] Incrementl GC... Incrementl GC... Even objects which re not explicitly referenced hve to be checked, to see if they hve become grbge. Therefore, every time we llocte new object we forwrd k pointers. A good vlue for k hs to be determined by experimenttion. Eventully scn will ctch up with next nd we switch from-spce nd to-spce nd strt n new cycle. Bker s lgorithm (on the next slide) is vrint of copying collection. 1. Copy nd updte objects pointed to by globl pointers to to-spce.. Resume progrm. 3. When n object in from-spce is referenced, first copy it to to-spce. p := x.next; (implemented s) IF x from spce TEN copy x to to-spce; updte x, scn, nd next; x := x s new ddress in to-spce; END; p := x.next; 4. Every time NEW is clled, k pointers re forwrded.

15 Cost of Grbge Collection Cost of GC Mrk-nd-Sweep The size of the hep is, the mount of rechble memory is R, the mount of memory reclimed is R. Wht is the cost of the different GC lgorithms? e p Rechble=R mortized GC cost = = epsize= Reclimed= R time spent in GC mount of grbge collected time spent in GC R [57] 50 e p Rechble=R epsize= Reclimed= R The mrk phse touches ll live nodes. ence, it tkes time c 1, for some constnt c 1. c 1 10? The sweep phse touches the whole hep. ence, it tkes time c R, for some constnt c. c 3? GC cost = c 1R + c R [58] 10R + 3 R Cost of GC Mrk-nd-Sweep... Cost of GC Copying Collection e p epsize= e p epsize= Rechble=R GC cost = c 1R + c R Reclimed= R 10R + 3 R If R we reclim very litte, nd the cost of GC goes up. In this cse the GC should grow the hep (increse ). Rechble=R Reclimed= R The bredth first serch phse touches ll live nodes. ence, it tkes time c 3 R, for some constnt c 3. c 3 10? The hep is divided into from-spce nd to-spce, so ech collection reclims R words. GC cost = c 3R R 10R R

16 Cost of GC Copying Collection... Cost of GC Genertionl Collection GC cost = c 3R R 10R R If there re few live objects ( R) the GC cost is low. If = 4R, we get GC cost = c 3R 4R R 10. This is expensive: 4 times s much memory s rechble dt, 10 instruction GC cost per object llocted. e p Rechble=R epsize= Reclimed= R Assume the youngest genertion (G 0 ) hs 10% live dt, i.e. = 10R. Assume we re using copying collection for G 0. GC cost G0 = c 3R R = c 3R 10R R 10R 4R =.5 [61] 50 [6] st of GC Genertionl Collection... Exm Problem e p GC cost G0 = Rechble=R epsize= c 3R R = Reclimed= R c 3R 10R R 10R 4R =.5 If R 100 kilobytes in G 0, then 1 megbyte. In other words, we ve wsted bout 900 kilobytes, to get.5 instruction/word GC cost (for G 0 ). 1. Why is genertionl collection more pproprite for functionl nd logic lnguges (such s LISP nd Prolog), thn for object-oriented lnguges (such s Eiffel nd Modul-3)?. The hep in the figure on the next slide holds 7 objects. All objects hve one integer field nd one or two pointer fields (blck dots). The only roots re the three globl vribles X, Y, nd Z. Free spce is shded. Show the stte of To-Spce fter copying grbge collection hs been performed on From-Spce. Note tht severl nswers re possible, depending on the visit strtegy (Depth-First or Bredth-First Serch) you chose.

17 Exm Problem I... Exm Problem... From Spce 5 Roots: 6 X Y Z Nme five grbge collection lgorithms!. Describe the Deutsch-Schorr-Wite lgorithm! When is it used? Why is it used? ow does it work? 3. Wht re the differences between stop-nd-copy, incrementl nd concurrent grbge collection? When would we prefer one over the other? [65] 50 [66] Redings nd References Redings nd References... Red Scott, pp Apple s Tiger book, pp Topics in dvnced lnguge implementtion, Chpter 4, Andrew Appel, Grbge Collection. Chpter 5, Dvid L. Detlefs, Concurrent Grbge Collection for C++. ISBN Aho, opcroft, Ullmn. Dt Structures nd Algorithms, Chpter 1, Memory Mngement. Nndkumr Snkrn, A Bibliogrphy on Grbge Collection nd Relted Topics, ACM SIGPLAN Notices, Volume 9, No. 9, Sep J. Cohen. Grbge Collection of Linked Dt Structures, Computing Surveys, Vol. 13, No. 3, pp