DATA STRUCTURES. amortized analysis binomial heaps Fibonacci heaps union-find. Data structures. Appetizer. Appetizer

Transcription

1 Data structures DATA STRUCTURES Static problems. Give a iput, produce a output. Ex. Sortig, FFT, edit distace, shortest paths, MST, max-flow,... amortized aalysis biomial heaps Fiboacci heaps uio-fid Dyamic problems. Give a sequece of operatios (give oe at a time), produce a sequece of outputs. Ex. Stack, queue, priority queue, symbol table, uio-fid,. Algorithm. Step-by-step procedure to solve a problem. Data structure. Way to store ad orgaize data. Ex. Array, liked list, biary heap, biary search tree, hash table, Lecture slides by Kevi Waye Last updated o Apr 8, 0 6: AM Appetizer Appetizer Goal. Desig a data structure to support all operatios i O() time. INIT(): create ad retur a iitialized array (all zero) of legth. READ(A, i): retur i th elemet of array. WRITE(A, i, value): set i th elemet of array to value. Assumptios. true i C or C++, but ot Java Ca MALLOC a uiitialized array of legth i O() time. Give a array, ca read or write i th elemet i O() time. Data structure. Three arrays A[.. ], B[.. ], ad C[.. ], ad a iteger k. A[i] stores the curret value for READ (if iitialized). k = umber of iitialized etries. C[j] = idex of j th iitialized etry for j =,, k. If C[j] = i, the B[i] = j for j =,, k. Theorem. A[i] is iitialized iff both B[i] k ad C[B[i]] = i. Pf. Ahead. Remark. A array does INIT i O() time ad READ ad WRITE i O() time. A[ ] 5 6 8? 55 99??? B[ ]???? C[ ] 6???? k = A[]=99, A[6]=, A[]=, ad A[]=55 iitialized i that order

2 Appetizer Appetizer Theorem. A[i] is iitialized iff both B[i] k ad C[B[i]] = i. Pf. INIT (A, ) k 0. A MALLOC(). READ (A, i) IF (INITIALIZED (A[i])) RETURN A[i]. WRITE (A, i, value) IF (INITIALIZED (A[i])) A[i] value. Suppose A[i] is the j th etry to be iitialized. The C[j] = i ad B[i] = j. Thus, C[B[i]] = i. B MALLOC(). ELSE ELSE C MALLOC(). RETURN 0. k k +. s MALLOC(). A[i] value B[i] k. A[ ]? 55 99??? INITIALIZED (A, i) C[k] i. IF ( B[i] k) ad (C[B[i]] = i) B[ ]???? RETURN true. ELSE C[ ] 6???? RETURN false. k = 5 A[]=99, A[6]=, A[]=, ad A[]=55 iitialized i that order 6 Appetizer Theorem. A[i] is iitialized iff both B[i] k ad C[B[i]] = i. Pf. Suppose A[i] is uiitialized. If B[i] < or B[i] > k, the A[i] clearly uiitialized. If B[i] k by coicidece, the we still ca't have C[B[i]] = i because oe of the etries C[.. k] ca equal i. AMORTIZED ANALYSIS biary couter multipop stack dyamic table A[ ]? 55 99??? B[ ]???? Lecture slides by Kevi Waye C[ ] 6???? k = A[]=99, A[6]=, A[]=, ad A[]=55 iitialized i that order Last updated o Apr 8, 0 6: AM

3 Amortized aalysis Worst-case aalysis. Determie worst-case ruig time of a data structure operatio as fuctio of the iput size. Amortized aalysis. Determie worst-case ruig time of a sequece of data structure operatios as a fuctio of the iput size. ca be too pessimistic if the oly way to ecouter a expesive operatio is if there were lots of previous cheap operatios Ex. Startig from a empty stack implemeted with a dyamic table, ay sequece of push ad pop operatios takes O() time i the worst case. Amortized aalysis: applicatios Splay trees. Dyamic table. Fiboacci heaps. Garbage collectio. Move-to-frot list updatig. Push-relabel algorithm for max flow. Path compressio for disjoit-set uio. Structural modificatios to red-black trees. Security, databases, distributed computig,... SIAM J. ALG. DISC. METH. Vol. 6, No., April Society for Idustrial ad Applied Mathematics 06 AMORTIZED COMPUTATIONAL COMPLEXITY* ROBERT ENDRE TARJANt Abstract. A powerful techique i the complexity aalysis of data structures is amortizatio, or averagig over time. Amortized ruig time is a realistic but robust complexity measure for which we ca obtai surprisigly tight upper ad lower bouds o a variety of algorithms. By followig the priciple of desigig algorithms whose amortized complexity is low, we obtai "self-adjustig" data structures that are simple, flexible ad efficiet. This paper surveys recet work by several researchers o amortized complexity. ASM(MOS) subject classificatios. 68C5, 68E Biary couter AMORTIZED ANALYSIS Goal. Icremet a k-bit biary couter (mod k ). Represetatio. aj = j th least sigificat bit of couter. CHAPTER biary couter multipop stack dyamic table Couter value A[] A[6] A[5] A[] A[] A[] A[] A[0] Cost model. Number of bits flipped.

4 Biary couter Aggregate method (brute force) Goal. Icremet a k-bit biary couter (mod k ). Represetatio. a j = j th least sigificat bit of couter. Aggregate method. Sum up sequece of operatios, weighted by their cost. Couter value A[] A[6] A[5] A[] A[] A[] A[] A[0] Couter value A[] A[6] A[5] A[] A[] A[] A[] A[0] Total cost Theorem. Startig from the zero couter, a sequece of INCREMENT operatios flips O( k) bits. Pf. At most k bits flipped per icremet. Biary couter: aggregate method Accoutig method (baker's method) Startig from the zero couter, i a sequece of INCREMENT operatios: Bit 0 flips times. Bit flips / times. Bit flips / times. Theorem. Startig from the zero couter, a sequece of INCREMENT operatios flips O() bits. Pf. Bit j flips / j times. The total umber of bits flipped is k j=0 j < j=0 j Assig differet charges to each operatio. Di = data structure after operatio i. c i = actual cost of operatio i. ĉi = amortized cost of operatio i = amout we charge operatio i. Whe ĉi > ci, we store credits i data structure Di to pay for future ops. Iitial data structure D0 starts with zero credits. Key ivariat. The total umber of credits i the data structure 0. i= ĉ i i= c i 0 ca be more or less tha actual cost = Remark. Theorem may be false if iitial couter is ot zero. 5 6

5 Accoutig method (baker's method) Biary couter: accoutig method Assig differet charges to each operatio. D i = data structure after operatio i. c i = actual cost of operatio i. ĉ i = amortized cost of operatio i = amout we charge operatio i. Whe ĉi > ci, we store credits i data structure Di to pay for future ops. Iitial data structure D0 starts with zero credits. ca be more or less tha actual cost Credits. Oe credit pays for a bit flip. Ivariat. Each bit that is set to has oe credit. Accoutig. Flip bit j from 0 to : charge two credits (use oe ad save oe i bit j). Key ivariat. The total umber of credits i the data structure 0. c i 0 i= ĉ i i= Theorem. Startig from the iitial data structure D 0, the total actual cost of ay sequece of operatios is at most the sum of the amortized costs. Pf. The amortized cost of the sequece of operatios is: ĉ i c i. i= i= Ituitio. Measure ruig time i terms of credits (time = moey). icremet Biary couter: accoutig method Biary couter: accoutig method Credits. Oe credit pays for a bit flip. Ivariat. Each bit that is set to has oe credit. Credits. Oe credit pays for a bit flip. Ivariat. Each bit that is set to has oe credit. Accoutig. Flip bit j from 0 to : charge two credits (use oe ad save oe i bit j). Flip bit j from to 0: pay for it with saved credit i bit j. Accoutig. Flip bit j from 0 to : charge two credits (use oe ad save oe i bit j). Flip bit j from to 0: pay for it with saved credit i bit j. icremet

6 Biary couter: accoutig method Potetial method (physicist's method) Credits. Oe credit pays for a bit flip. Ivariat. Each bit that is set to has oe credit. Accoutig. Flip bit j from 0 to : charge two credits (use oe ad save oe i bit j). Flip bit j from to 0: pay for it with saved credit i bit j. Potetial fuctio. Φ(D i) maps each data structure D i to a real umber s.t.: Φ(D 0) = 0. Φ(D i) 0 for each data structure D i. Actual ad amortized costs. ci = actual cost of i th operatio. ĉ i = c i + Φ(D i) Φ(D i ) = amortized cost of i th operatio. Theorem. Startig from the zero couter, a sequece of INCREMENT operatios flips O() bits. Pf. The algorithm maitais the ivariat that ay bit that is curretly set to has oe credit umber of credits i each bit 0. Potetial method (physicist's method) Biary couter: potetial method Potetial fuctio. Φ(Di) maps each data structure Di to a real umber s.t.: Φ(D0) = 0. Φ(D i) 0 for each data structure D i. Potetial fuctio. Let Φ(D) = umber of bits i the biary couter D. Φ(D0) = 0. Φ(D i) 0 for each D i. Actual ad amortized costs. ci = actual cost of i th operatio. ĉ i = c i + Φ(D i) Φ(D i ) = amortized cost of i th operatio. Theorem. Startig from the iitial data structure D0, the total actual cost of ay sequece of operatios is at most the sum of the amortized costs. Pf. The amortized cost of the sequece of operatios is: icremet i= ĉ i = = i= (c i + (D i ) (D i ) c i + (D ) (D 0 ) i= c i i=

7 Biary couter: potetial method Biary couter: potetial method Potetial fuctio. Let Φ(D) = umber of bits i the biary couter D. Φ(D 0) = 0. Φ(D i) 0 for each D i. Potetial fuctio. Let Φ(D) = umber of bits i the biary couter D. Φ(D 0) = 0. Φ(D i) 0 for each D i. icremet Biary couter: potetial method Famous potetial fuctios Potetial fuctio. Let Φ(D) = umber of bits i the biary couter D. Φ(D0) = 0. Φ(D i) 0 for each D i. Fiboacci heaps. Φ(H) = trees(h) + marks(h). Splay trees. (T ) = log size(x) x T Theorem. Startig from the zero couter, a sequece of INCREMENT operatios flips O() bits. Pf. Suppose that the i th icremet operatio flips ti bits from to 0. The actual cost c i t i +. The amortized cost ĉi = ci + Φ(Di) Φ(Di ) ci + ti. operatio sets oe bit to (uless couter resets to zero) Move-to-frot. Φ(L) = iversios(l, L*). Preflow-push. (f) = Red-black trees. (T ) = w(x) = v : excess(v) > 0 x T w(x) height(v) 0 x x 0 x x 8

8 Multipop stack AMORTIZED ANALYSIS biary couter multipop stack dyamic table Goal. Support operatios o a set of elemets: PUSH(S, x): push object x oto stack S. POP(S): remove ad retur the most-recetly added object. MULTIPOP(S, k): remove the most-recetly added k objects. MULTIPOP (S, k) FOR i = TO k POP (S). SECTION. Exceptios. We assume POP throws a exceptio if stack is empty. 0 Multipop stack Multipop stack: aggregate method Goal. Support operatios o a set of elemets: PUSH(S, x): push object x oto stack S. POP(S): remove ad retur the most-recetly added object. MULTIPOP(S, k): remove the most-recetly added k objects. Goal. Support operatios o a set of elemets: PUSH(S, x): push object x oto stack S. POP(S): remove ad retur the most-recetly added object. MULTIPOP(S, k): remove the most-recetly added k objects. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTIPOP operatios takes O( ) time. Pf. Use a sigly-liked list. PoP ad PUSH take O() time each. MULTIPOP takes O() time. overly pessimistic upper boud Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTIPOP operatios takes O() time. Pf. A object is popped at most oce for each time it is pushed oto stack. There are PUSH operatios. Thus, there are POP operatios (icludig those made withi MULTIPOP). top

9 Multipop stack: accoutig method Multipop stack: potetial method Credits. Oe credit pays for a push or pop. Accoutig. PUSH(S, x): charge two credits. use oe credit to pay for pushig x ow store oe credit to pay for poppig x at some poit i the future No other operatio is charged a credit. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTIPOP operatios takes O() time. Potetial fuctio. Let Φ(D) = umber of objects curretly o the stack. Φ(D 0) = 0. Φ(D i) 0 for each D i. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTIPOP operatios takes O() time. Pf. [Case : push] Suppose that the i th operatio is a PUSH. The actual cost c i =. The amortized cost ĉ i = c i + Φ(D i) Φ(D i ) = + =. Pf. The algorithm maitais the ivariat that every object remaiig o the stack has credit umber of credits i data structure 0. Multipop stack: potetial method Multipop stack: potetial method Potetial fuctio. Let Φ(D) = umber of objects curretly o the stack. Φ(D0) = 0. Φ(D i) 0 for each D i. Potetial fuctio. Let Φ(D) = umber of objects curretly o the stack. Φ(D0) = 0. Φ(D i) 0 for each D i. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTIPOP operatios takes O() time. Theorem. Startig from a empty stack, ay itermixed sequece of PUSH, POP, ad MULTIPOP operatios takes O() time. Pf. [Case : pop] Suppose that the i th operatio is a POP. The actual cost c i =. The amortized cost ĉi = ci + Φ(Di) Φ(Di ) = = 0. Pf. [Case : multipop] Suppose that the i th operatio is a MULTIPOP of k objects. The actual cost c i = k. The amortized cost ĉi = ci + Φ(Di) Φ(Di ) = k k =

10 Dyamic table AMORTIZED ANALYSIS biary couter multipop stack dyamic table Goal. Store items i a table (e.g., for hash table, biary heap). Two operatios: INSERT ad DELETE. too may items iserted expad table. too may items deleted cotract table. Requiremet: if table cotais m items, the space = Θ(m). Theorem. Startig from a empty dyamic table, ay itermixed sequece of INSERT ad DELETE operatios takes O( ) time. Pf. A sigle INSERT or DELETE takes O() time. overly pessimistic upper boud SECTION. 8 Dyamic table: isert oly Iitialize table to be size. INSERT: if table is full, first copy all items to a table of twice the size. isert old size ew size cost Dyamic table: isert oly Theorem. [via aggregate method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Pf. Let ci deote the cost of the i th isertio. c i = i i Startig from empty table, the cost of a sequece of INSERT operatios is: lg c i + j i= j=0 < + = Cost model. Number of items that are copied. 9 0

11 Dyamic table: isert oly Dyamic table: isert oly Accoutig. INSERT: charge credits (use credit to isert; save with ew item) Theorem. [via accoutig method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Pf. The algorithm maitais the ivariat that there are credits with each item i right half of table. Whe table doubles, oe-half of the items i the table have credits. This pays for the work eeded to double the table Dyamic table: isert oly Dyamic table: isert oly Theorem. [via potetial method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Theorem. [via potetial method] Startig from a empty dyamic table, ay sequece of INSERT operatios takes O() time. Pf. Let Φ(Di) = size(di) capacity(di). Pf. Let Φ(Di) = size(di) capacity(di). umber of elemets capacity of array umber of elemets capacity of array 5 6 Case. [does ot trigger expasio] size(d i) capacity(d i ). Actual cost ci =. Φ(D i) Φ(D i ) =. Amortized costs ĉi = ci + Φ(Di) Φ(Di ) = + =. Case. [triggers expasio] size(di) = + capacity(di ). Actual cost ci = + capacity(di ). Φ(D i) Φ(D i ) = capacity(d i) + capacity(d i ) = capacity(d i ). Amortized costs ĉi = ci + Φ(Di) Φ(Di ) = + =.

12 Dyamic table: doublig ad halvig Dyamic table: isert ad delete Thrashig. Iitialize table to be of fixed size, say. INSERT: if table is full, expad to a table of twice the size. DELETE: if table is ½-full, cotract to a table of half the size. Efficiet solutio. Iitialize table to be of fixed size, say. INSERT: if table is full, expad to a table of twice the size. DELETE: if table is ¼-full, cotract to a table of half the size. Memory usage. A dyamic table uses O() memory to store items. Pf. Table is always at least ¼-full (provided it is ot empty). Theorem. [via aggregate method] Startig from a empty dyamic table, ay itermixed sequece of INSERT ad DELETE operatios takes O() time. Pf. I betwee resizig evets, each INSERT ad DELETE takes O() time. Cosider total amout of work betwee two resizig evets. Just after the table is doubled to size m, it cotais m / items. Just after the table is halved to size m, it cotais m / items. Just before the ext resizig, it cotais either m / or m items. After resizig to m, we must perform Ω(m) operatios before we resize agai (either m isertios or m / deletios). Resizig a table of size m requires O(m) time. 5 6 Dyamic table: isert ad delete Dyamic table: isert ad delete isert Accoutig. INSERT: charge credits ( credit for isert; save with ew item). DELETE: charge credits ( credit to delete, save i emptied slot). discard ay existig credits delete Theorem. [via accoutig method] Startig from a empty dyamic table, ay itermixed sequece of INSERT ad DELETE operatios takes O() time Pf. The algorithm maitais the ivariat that there are credits with each resize ad delete item i the right half of table; credit with each empty slot i the left half. Whe table doubles, each item i right half of table has credits. Whe table halves, each empty slot i left half of table has credit. 8

13 Dyamic table: isert ad delete Theorem. [via potetial method] Startig from a empty dyamic table, ay itermixed sequece of INSERT ad DELETE operatios takes O() time. Pf sketch. Let α(di) = size(di) / capacity(di). (D i )= size(d i) capacity(d i ) / capacity(d i) size(d i ) < / Whe α(d) = /, Φ(D) = 0. Whe α(d) =, Φ(D) = size(d i). Whe α(d) = /, Φ(D) = size(d i).... [zero potetial after resizig] [ca pay for expasio] [ca pay for cotractio] 9