Introduction. Introduction: Multi-Pop Stack Example. Multi-Pop Stack Cost (clever) Multi-Pop Stack Cost (naïve)

Transcription

1 Introduction Today we begin studying how to calculate the total time of a sequence of operations as a whole. (As opposed to each operation individually.) Why and when we care: You have an algorithm that uses a certain data structure. E.g., Kruskal s uses clusters (disjoint sets). Some operation takes a long time. E.g., merging two clusters is sometimes expensive. But the total time turns out OK. E.g., Kruskal s total time O( V lg V + E lg E ) still good. Introduction: Multi-Pop Stack Example Multi-pop stack operations: push(x): Implementation as usual, Θ(1) time. pop(): Implementation as usual, Θ(1) time. multipop(k): Calls pop() up to k times, Θ(k) worst-case time. (Probably silly, but on purpose to show something.) Start from empty and perform m operations. What is the total time? 1 / 20 2 / 20 Multi-Pop Stack Cost (naïve) Start from empty and perform m operations. What is the total time? A naïve calculation may get a gross overestimate: 1. Each operation may take O(k) time. 2. k may be near m because stack size may be near m. 3. Total: m operations take O(m 2 ) time. You may see what s wrong with it: either multipop happens too rarely to matter or multipop happens so often, stack size stays small How to calculate a better answer? Multi-Pop Stack Cost (clever) Starting from empty, perform m operations: 1. At most m pushes. 2. Cannot pop/multipop more than what has been pushed. 3. All pops+multipops together at most m pops. 4. Total: m operations take O(m) time, worst case. (Execise: Why is it also Ω(m)?) You may be tempted to say: Each operation takes O(1) time in some sense. The next slide tells you the magic word for that! 3 / 20 4 / 20

2 Amortized Time m operations take O(m) total time, worst case. Each operation takes O(1) amortized time. This name is inspired by obvious analogies with paying for mortgages etc. Generally: If m operations take O(f (m)) total worst-case time, we can say each operation take O(f (m)/m) amortized time. Amortization Method #0: Aggregate Aggregate method: Like what I just did about multi-pop stacks. 1. At most m pushes. 2. Cannot pop/multipop more than what has been pushed. 3. All pops+multipops together at most m pops. 4. Total: m operations take O(m) time, worst case. In general: Holistically calculate total worst-case time. Usually count important events. (Similar idea outside this context: In BFS, overall count enqueue s, dequeue s, looping over adjacency lists.) Very often ad-hoc arguments and tailor-making to each situation. 5 / 20 6 / 20 Amortization Method #1: Banker/Accounting Banker/Accounting method: Using multi-pop stacks again, imagine: Each operation receives 2 dollars. Push spends 1 dollar; pop does too. Multipop(k) spends min(k, size) dollars. If surplus: Save up for future spending. If deficit: Spend past savings. Prove: savings 0 always (next slide). Conclude: each operation takes O(2) amortized time (i.e., what it receives). Prove: Multi-Pop Stack Savings 0 Actually, prove invariant: savings size. Then invoke size 0. Initially: $0 savings, and size = 0. push: $1 surplus. Both savings and size go up by 1. savings size before, savings size after. Imagine attaching this 1 dollar to the item pushed. pop: $1 surplus. savings increases, size decreases, savings size. multipop: spending = number of items popped, call it j. So just spend the money attached to those items. Recevied $2 unused. savings go down by j 2, size goes down by j. savings size before, savings size after. Remark: could have said: pop receives $0, multipop receives $0. 7 / 20 8 / 20

3 Banker/Accounting Method: Generally Postulate: Each operation receives some dollars. This will become amortized time. (Remark: Different operations may receive different dollars.) Each operation spends some dollars. This should equal actual time (within Θ). If surplus: Save up for future spending. If deficit: Spend past savings. Prove: savings 0 always. (Common technique: Imagine a distribution of savings over data. This helps the deficit case.) Conclude: Amortize time is in O(received dollars). Amortization Method #2: Physicist/Potential Physicist/Potential method: Using multi-pop stacks again: Choose potential Φ(D i ) = stack size after i operations. Let a i = (ith op actual time) + Φ(D i ) Φ(D i 1 ). Use O(a i ) as amortized time upper bound. Legitimate because their sum is total time upper bound: m a i = i=1 m (ith op actual time) + Φ(D i ) Φ(D i 1 ) i=1 = total time + Φ(D m ) Φ(D 0 ) total time + 0 Next slide calculates a i for push, pop, multipop. 9 / / 20 Calculate: Multi-Pop Stack Amortized Time a i = (ith op actual time) + (size after) (size before) If the ith operation is: push: a i = 1 + (s + 1) s = 2 where s is size before pop: a i = 1 + (s 1) s = 0 where s is size before multipop(k): a i = j + (s j) s = 0 where s is size before, j = min(k, s). Conclusion: Each amortized time is in O(1). Physicist/Potential Method: Generally Postulate potential function Φ: function from data structure state to number. Prove: Φ(D m ) Φ(D 0 ) 0. (For all [large] m, for all sequences of m operations.) (D i stands for data structure state after i operations.) (Often abbreviate Φ(D i ) as Φ i.) Calculate: (ith op actual time) + Φ(D i ) Φ(D i 1 ) for amortized time. Remark: Different operations may have different amortized times. 11 / / 20

4 Expandable Array A.k.a. dynamic table. Support array operations, increase size on demand. E.g., Java ArrayList and Vector. private T[ ] A; // A.length = textbook size = Java ArrayList capacity private int end; // textbook num, Java ArrayList size // maintain: end A.length get(i): read A[i], if i < end set(i, x): A[i] := x, if i < end size(): return end add(x): append x : A[end] := x; end += 1 Except: if A is full, first migrate to an array twice as big. (Expensive copying.) (Exercise: Why not to an array 1 unit bigger?) Pseudocode And Accounting Method for add add(x): // receive $3 if A.length = 0 then: // corner case A := new array of length 1 end := 0 if end = A.length then: // migrate, spend $(A.length) past savings newa := new array of length 2 A.length copy A s content to newa A := newa // store x, spend $1, save $2 A[end] := x end += 1 13 / / 20 Expandable Array: Accounting Method get, set, size: receives $1, spends $1. add(x) receives $3. 1. If migration needed: Since last migration, A.length/2 cells have $2 s saved. They have saved $(A.length) Just what you need for copying all of A. 2. Whether migration or not: Use received $3: spend $1 (store x), save $2 at the cell used. Therefore, savings 0 always. Expandable Array: Potential Method Note invariant: A.length/2 end. I.e., A.length 2 end. Choose potential Φ(D) = 2 end A.length. Note Φ(D) 0 for all D. For D 0 : use initial A.length = 0, end = 0. Φ(D 0 ) = 0 Therefore, Φ(D m ) Φ(D 0 ) 0 for all sequences of m operations. Now it is valid to use (ith op actual time) + Φ(D i ) Φ(D i 1 ) for amortized time. 15 / / 20

5 Potential for add Let e be end before, l be A.length before. If no migration now: a i = add s actual time + Φ(D i ) Φ(D i 1 ) = 1 + (2(e + 1) l) (2e l) = 3 If need migration now: a i = add s actual time + Φ(D i ) Φ(D i 1 ) = (1 + l) + (2(e + 1) 2l) (2e l) = 3 Expandable, Shrinkable Array One more operation: remove() discards the last item. remove(): if end > 0: end -= 1 if end < A.length/4: migrate to new array half as big Note: Right after a migration (by add or remove), array is half-empty, half-full. (Exercise: Why not migrate when end < A.length/2?) So add s amortized time is O(3). 17 / / 20 Expandable, Shrinkable Array: Accounting Method add(x): // receive $3 if end = A.length: // since last migration, at least A.length/2 add s // saved $(A.length), now spend it: double array length, copy A.length items // spend $1, save $2 A[end] := x end += 1 Maintains savings 0. Amortized time O(3). Expandable, Shrinkable Array: Accounting Method remove(): if end > 0: // receive $2 // spend $1, save $1 end -= 1 if end < A.length/4: // since last migration, at least A.length/4 remove s // saved $(A.length/4), now spend it: half array length, copy fewer than A.length/4 items Maintains savings 0. Amortized time O(2). 19 / / 20