Graduate Analysis of Algorithms Prof. Karen Daniels

Transcription

1 UMass Lowell Computer Science Graduate Analysis of Algorithms Prof. Karen Daniels Fall, 203 Lecture 3 Monday, 9/23/3 Amortized Analysis

2 Stack (computational geometry application) Dynamic Table Binary Counter (in book but not lecture) 2 Conga Line for Dynamic Closest Pair (computational geometry) Overview Amortize: To pay off a debt, usually by periodic payments [Websters] Amortized Analysis: creative accounting for operations can show average cost of an operation is small (when averaged over sequence of operations, not distribution of inputs) even though a single operation in the sequence is expensive result must hold for any sequence of these operations no probability is involved (unlike average-case analysis) guarantee holds in worst-case analysis method only; no effect on code operation

3 More Motivation Depth-First and Breadth-First Search Dijkstra s Single-Source Shortest Path Fibonacci Heaps Knuth-Morris-Pratt String Matching Red-Black Tree Restructuring (see index) 3

4 Overview (continued) 3 ways to determine amortized cost of an operation that is part of a sequence of operations: Aggregate Method find upper bound T(n) on total cost of sequence of n operations amortized cost = average cost per operation = T(n)/n same for all the (perhaps different types of) operations in the sequence Accounting Method amortized cost can differ across types of operations overcharge some operations early in sequence store overcharge as prepaid credit on specific data structure objects Potential Method amortized cost can differ across operations (as in accounting method) overcharge some operations early in sequence (as in accounting method) store overcharge as potential energy of data structure as a whole (unlike accounting method) 4

5 Aggregate Method: Stack Operations Aggregate Method find upper bound T(n) on total cost of sequence of n operations amortized cost = average cost per operation = T(n)/n same for all the operations in the sequence Traditional Stack Operations PUSH(S,x) pushes object x onto stack S POP(S) pops top of stack S, returns popped object O() time: consider cost as for our discussion Total actual cost of sequence of n PUSH/POP operations = n STACK-EMPTY(S) time in Θ() 5

6 Aggregate Method: Stack Operations (continued) New Stack Operation MULTIPOP(S,k) pops top k elements off stack S pops entire stack if it has < k items MULTIPOP(S,k) while not STACK-EMPTY(S) and k = 0 2 POP(S) 3 k = k - MULTIPOP actual cost for stack containing s items: Use cost = for each POP Cost = min(s,k) Worst-case cost in O(s) in O(n) source: textbook Cormen et al. 6

7 Aggregate Method: Stack Operations (continued) 7

8 Aggregate Method: Stack Operations (continued) Sequence of n PUSH, POP, MULTIPOP ops initially empty stack MULTIPOP worst-case O(n) O(n 2 ) for sequence Aggregate method yields tighter upper bound Sequence of n operations has O(n) worst-case cost Each item can be popped at most once for each push # POP calls (including ones in MULTIPOP) <= #push calls <= n Average cost of an operation = O(n)/n = O() = amortized cost of each operation holds for PUSH, POP and MULTIPOP source: textbook Cormen et al. 8

9 Accounting Method Accounting Method amortized cost can differ across operations overcharge some operations early in sequence store overcharge as prepaid credit on specific data structure objects c i ĉ i Let be actual cost of ith operation Let be amortized cost of ith operation (what we charge) Total amortized cost of sequence of operations must ˆ be upper bound on total actual cost of sequence i= i Total credit in data structure = must be nonnegative for all n n i= n cˆ i c i= i n n c i c = source: textbook Cormen et al. 9 i

10 Accounting Method: Stack Operations Operation Actual Cost Assigned Amortized Cost PUSH 2 POP 0 MULTIPOP min(k,s) 0 Paying for a sequence using amortized cost: start with empty stack PUSH of an item always precedes POP, MULTIPOP pay for PUSH & store unit of credit credit for each item pays for actual POP, MULTIPOP cost of that item credit never goes negative total amortized cost of any sequence of n ops is in O(n) amortized cost is upper bound on total actual cost 0 source: textbook Cormen et al.

11 Potential Method Potential Method amortized cost can differ across operations (as in accounting method) overcharge some operations early in sequence (as in accounting method) store overcharge as potential energy of data structure as a whole (unlike accounting method) Let c i be actual cost of i th operation Let D i be data structure after applying i th operation Let Φ(D i ) be potential associated with D i Amortized cost of i th operation: Total amortized cost of n operations: n i = cˆ i = ci + Φ( Di ) Φ( Di ) n n ˆi = ( ci + Φ ( Di ) Φ ( Di )) = ci + Φ ( D n ) Φ ( D 0 i = i = ( D n 0 c terms telescope Require: Φ ) Φ( D ) so total amortized cost is upper bound on total actual cost Since n might not be known in advance, guarantee payment in advance by requiring Φ( D i ) Φ( D0 ) source: textbook Cormen et al. )

12 Potential Method: Stack Operations Potential function value = number of items in stack guarantees nonnegative potential after ith operation Amortized operation costs (assuming stack has s items) PUSH: potential difference= Φ( Di ) Φ( Di ) = ( s + ) s = amortized cost = cˆ c + Φ( D ) Φ( D ) = + = 2 MULTIPOP(S,k) pops k =min(k,s) items off stack POP amortized cost also = 0 i = i i i potential difference= Φ( Di ) Φ( Di ) = k' amortized cost = cˆ = c + Φ( D ) Φ( D ) = k' k' = 0 i Amortized cost O() total amortized cost of sequence of n operations in O(n) source: textbook i i i 2 source: textbook Cormen et al.

13 Dynamic Tables: Overview Dynamic Table T: array of slots Ignore implementation choices: stack, heap, hash table... if too full, increase size & copy entries to T if too empty, decrease size & copy entries to T Analyze dynamic table insert and delete Actual expansion or contraction cost is large Show amortized cost of insert or delete in O() Load factor α(t) = num[t]/size[t] num[t] = number of items currently stored in table empty table: α(t) = (convention guaranteeing load factor can be lower bounded by a useful constant) 3 full table: α(t) = source: textbook Cormen Cormen et al.

14 Dynamic Tables: Table (Expansion Only) Load factor bounds (double size when T is full): Sequence of n inserts on initially empty table Worst-case cost of insert is in O(n) Worst-case cost of sequence of n inserts is in O(n 2 ) WHY? LOOSE Double only when table is already full. elementary insertion 4 source: textbook Cormen et al.

15 Dynamic Tables: Aggregate Method: c i = i if i- is exact power of 2 otherwise total cost of n inserts = Accounting Method: Table Expansion (cont.) Amortized Analysis charge cost = 3 for each element inserted intuition for 3 each item pays for 3 elementary insertions inserting itself into current table expansion: moving itself n i= insert expansion: moving another item that has already been moved lgn j ci n + 2 < n + 2n = 3n j= 0 count only elementary insertions copy 5 source: textbook Cormen et al.

16 Dynamic Tables: Amortized Analysis Potential Method: Φ T ) = Value of potential function Φ(Τ) 0 right after expansion (then becomes 2 why?) Φ( T ) = builds to table size by time table is full 0 when size[ T ] = 2num[ T ] Φ( T ) = 2num[ T ] size[ T ] always nonnegative, so sum of amortized costs of n inserts is upper bound on sum of actual costs Table Expansion (cont.) ( 2num[ T ] size[ T ] Amortized cost of ith insert Φ i = potential after ith operation Case : insert does not cause expansion cˆ i = ci + Φi Φi = + (2numi sizei ) (2numi sizei ) = Case 2: insert causes expansion ˆi = ci + Φi Φi c use these: 3 functions: size i, num i, Φ i = numi + (2numi sizei ) (2numi sizei ) = num = i sizei num = + i num i size i = 2sizei 3 6 3

17 Dynamic Tables: Table Expansion & Contraction count elementary insertions & deletions (double size when T is full) (halve size when T is ¼ full (why ¼?)): Load factor bounds: DELETE pseudocode analogous to INSERT Amortized Analysis Φ T ) Potential Method: Value of potential function Φ(Τ) = 0 for empty table 0 right after expansion or contraction builds as α(t) increases to or decreases to ¼ always nonnegative, so sum of amortized costs of n inserts is upper bound on sum of actual costs = 0 when α(t)=/2 = num[t] when α(t)= = num[t] when α(t)=/4 same as INSERT ( = 2num[ T ] size[ T ] if α( T ) / 2 size[ T ]/ 2 num[ T ] if α( T ) < / 2 Different from INSERT motivation for choice of potential function: can pay for moving num[t] items 7 source: textbook Cormen et al.

18 Dynamic Tables: Table Expansion & Contraction Amortized Analysis Potential Method 3 functions: size i, num i, Φ i motivation for choice of potential function 8 source: textbook Cormen et al.

19 Dynamic Tables: Table Expansion & Contraction Amortized Analysis Φ( T) = 2numT [ ] size[ T] if α( T) / 2 Potential Method size[ T]/ 2 numt [ ] if α( T) < / 2 Analyze cost of sequence of n inserts and/or deletes Amortized cost of ith operation Case : INSERT Case a: α i- >= ½. By previous insert analysis: cˆ i 3 holds whether or not table expands Case b: α i- < ½ and α i < ½ ˆi = ci + Φi Φi c Case c: α i- < ½ and α i >= ½ no expansion (why?) = + (( sizei / 2) numi ) (( sizei / 2) numi ) = no expansion (why?) 0 ˆi = ci + Φi Φi see derivation in textbook c = + (2numi sizei ) (( sizei / 2) numi 9) 3

20 Dynamic Tables: Table Expansion & Contraction Amortized Analysis Potential Method Amortized cost of ith operation (continued) Φ( T) = 2numT [ ] size[ T] if α( T) / 2 Case 2: DELETE size[ T]/ 2 numt [ ] if α( T) < / 2 Case 2a: α i- >= ½. Case 2b: α i- < ½ and α i < ½ no contraction ˆi = ci + Φi Φi c textbook exercise = + ( sizei / 2 numi ) ( sizei / 2 numi ) = 2 Case 2c: α i- < ½ and α i < ½ contraction ˆi = ci + Φi Φi c = ( numi + ) + ( sizei / 2 numi ) ( sizei / 2 numi ) = Conclusion: amortized cost of each operation is bounded above by a constant, so time for sequence of n operations is in O(n). 20 source: textbook Cormen et al.

21 Example: Dynamic Closest Pair S S = S S S 2 3 S 2 S 3 Goal: Fast maintenance of closest pair in dynamic setting source: Fast hierarchical clustering and other applications of dynamic closest 2 pairs, David Eppstein, Journal of Experimental Algorithmics, Vol. 5, August 2000.

22 Example: Dynamic Closest Pair (continued) S S S source: Fast hierarchical clustering and other applications of dynamic closest 22 pairs, David Eppstein, Journal of Experimental Algorithmics, Vol. 5, August 2000.

23 Example: Dynamic Closest Pair (continued) Rules Partition dynamic set S into k log n subsets. Each subset S i has an associated digraph G i consisting of a set of disjoint, directed paths. Total number of edges in all graphs remains linear Combine and rebuild if number of edges reaches 2n. Closest pair is always in some G i. Initially all points are in single set S. Operations: Create G i for a subset S i. Insert a point. Delete a point. Merge subsets until k log n. We use log base 2. source: Fast hierarchical clustering and other applications of dynamic closest pairs, David Eppstein, Journal of Experimental Algorithmics, Vol. 5, August

24 Example: Dynamic Closest Pair (continued) Rules: Operations source: Fast hierarchical clustering and other applications of dynamic closest pairs, David Eppstein, Journal of Experimental Algorithmics, Vol. 5, August Create G i for a subset S i : Select starting point (we choose leftmost (or higher one in case of a tie)) Iteratively extend the path P, selecting next vertex as: Case : nearest neighbor in S \ P if last point on path belongs to S i Case 2: nearest neighbor in S i \ P if last point on path belongs to S \S i Insert a point x: Create new subset S k+ ={x}. Merge subsets if necessary. Create G i for new or merged subsets. Delete a point x: Create new subset S k+ = all points y such that (y,x) is a directed edge in some G i. Remove x and adjacent edges from all G i. (We also remove y from its subset.) Merge subsets if necessary. Create G i for new or merged subsets. Merge subsets until k log n : Choose subsets S i and S j to minimize size ratio S j / S i. See handout for example. 24

25 Example: Dynamic Closest Pair (continued) Potential Function source: Fast hierarchical clustering and other applications of dynamic closest pairs, David Eppstein, Journal of Experimental Algorithmics, Vol. 5, August Potential for a subset S i : Φ i = n S i log S i. Total potential Φ = n 2 logn - ΣΦ i. Paper proves this Theorem in Section 3: Theorem: The data structure maintains the closest pair in S in O(n) space, amortized time O(nlogn) per insertion, and amortized time O(nlog 2 n) per deletion. HW#3 contains a problem related to this paper. Please read up through Section 3. Later in the semester we will have sufficient background for much of the remainder of the paper. 25