Amortized Analysis and Union-Find , Inge Li Gørtz

Transcription

1 Amortized Analysis and Union-Find 02283, Inge Li Gørtz 1

2 Today Amortized analysis 3 different methods 2 examples Union-Find data structures Worst-case complexity Amortized complexity 2

3 Amortized Analysis Amortized analysis. Average running time per operation over a worst-case sequence of operations. Time required to perform a sequence of data operations is averaged over all the operations performed. Motivation: traditional worst-case-per-operation analysis can give too pessimistic bound if the only way of having an expensive operation is to have a lot of cheap ones before it. Different from average case analysis: average over time, not input. 3

4 Amortized Analysis Methods. Aggregate method Accounting method Potential method 4

5 Aggregate method Aggregate. Determine total cost. Amortized cost = total cost/#operations. 5

6 Dynamic Tables Doubling strategy. Start with empty array of size 1. Insert: If array is full create a new array of double the size and reinsert all elements. Analysis: n insert operations. Assume n is a power of 2. Number of insertions log n = O(n). Total cost: O(n). Amortized cost per insert: O(1). 6

7 Accounting method Accounting. Some types of operations are overcharged. Credit allocated with elements in the data structure used to pay for subsequent operations Total credit non-negative at all times -> total amortized cost an upper bound on the actual cost. 7

8 Dynamic Tables Amortized costs: Amortized cost of insertion: 3 1 for own insertion 1 for its first reinsertion. 1 to pay for reinsertion of one of the items that have already been reinserted once. 8

9 Dynamic Tables Analysis: keep 2 credits on each element in the array that is beyond the middle. table not full: insert costs 1, and we have 2 credits to save. table full, i.e., doubling: half of the elements have 2 credits each. Use these to pay for reinsertion of all in the new array. Amortized cost per operation: x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 9

10 Example: Stack with MultiPop Stack with MultiPop. Push(e): push element e onto stack. MultiPop(k): pop top k elements from the stack Worst case: Implement via linked list or array. Push: O(1). MultiPop: O(k). Amortized cost per operation: 2. 10

11 Stack: Aggregate Analysis Amortized analysis. Sequence of n Push and MultiPop operations. Each object popped at most once for each time it is pushed. #pops on non-empty stack #Push operations n. Total time O(n). Amortized cost per operation: 2n/n = 2. 11

12 Stack: Accounting Method Amortized analysis. Sequence of n Push and MultiPop operations. Pay 2 credits for each Push. Keep 1 credit on each element on the stack. Amortized cost per operation: Push: 2 MultiPop: 1 (to pay for pop on empty stack). 12

13 Potential method Potential functions. Prepaid credit (potential) associated with the data structure (money in the bank). Can be used to pay for future operations. Ensure there is always enough money in the bank. Amortized cost of an operation: potential cost plus increase in potential due to the operation. Di: data structure after i operations Potential function Φ(Di) maps Di onto a real value. amortized cost = actual cost + Δ(Di) = actual cost + Φ(Di) - Φ(Di-1). 13

14 Potential Functions Amortized cost: amortized cost = actual cost + Δ(Di) = actual cost + Φ(Di) - Φ(Di-1). Stack. Φ(Di) = #elements on the stack. amortized cost of Push = 1 + Δ(Di) = 2. amortized cost of MultiPop(k): If k =min(k, S ) elements are popped. if S : amortized cost = k + Φ(Di) -Φ(Di-1) = k - k = 0. if S = : amortized cost = 1 + Δ(Di) = 1. 14

15 Potential Functions Amortized cost: amortized cost = actual cost + Δ(Di) = actual cost + Φ(Di) - Φ(Di-1). Dynamic tables { 2(k L/2) if k L/2 Φ(Di) = 0 otherwise L = current array size, k = number of elements in array. amortized cost of insertion: Array not full: amortized cost = = 3 Array full (doubling): Actual cost = L + 1, Φ(Di-1) = L, Φ(Di)=2: amortized cost = L (2 - L) = 3. 15

16 Amortized Cost vs Actual Cost Total cost: amortized cost = (actual cost + Δ(Di)) = actual cost + Φ(Dn) - Φ(D0). actual cost = amortized cost + Φ(D0) - Φ(Dn). If potential always nonnegative and Φ(D0) = 0 then actual cost amortized cost. 16

17 Potential Method Summary: 1. Pick a potential function, Φ, that will work (art). 2. Use potential function to bound the amortized cost of the operations you're interested in. 3. Bound Φ(D0) - Φ(Dfinal) Techniques to find potential functions: if the actual cost of an operation is high, then decrease in potential due to this operation must be large, to keep the amortized cost low. 17

18 Union-Find Data Structures 18

19 Union-Find Data Structure Union-Find data structure: Makeset(x): Create a singleton set containing x and return its identifier. Union(A,B): Combine the sets identified by A and B into a new set, destroying the old sets. Return the identifier of the new set. Find(x): Return the identifier of the set containing x. Only requirement for identifier: find(x) = find(y) iff x and y are in the same set. Applications: Connectivity, Kruskal s algorithm for MST,... 19

20 A Simple Union-Find Data Structure Quick-Union: Each set represented by a tree. Elements are represented by nodes. Root is also identifier. Make-Set(x): Create a new node x. Set p(x) = x. Find(x): Follow parent pointers to the root. Return the root. Union(A,B): Make root(b) a child of root(a). 20

21 A Simple Union-Find Data Structure Quick-Union: Union(A,B): Make root(b) a child of root(a). Union(7,5) Union(3,1) Union(7,8) Union(3,7)

22 A Simple Union-Find Data Structure Quick-Union: Each set represented by a tree. Elements are represented by nodes. Root is also identifier. Make-Set(x): Create a new node x. Set p(x) = x. Find(x): Follow parent pointers to the root. Return the root. Union(A,B): Make root(b) a child of root(a). Analysis: Make-Set(x) and Union(A,B): O(1) Find(x): O(h), where h is the height of the tree containing x. Worst-case O(n). 22

23 A Simple Union-Find Data Structure Quick Find: Each set represented by a tree of height at most one. Elements are represented by nodes. Root is also identifier. Make-Set(x): Create a new node x. Set p(x) = x and size(x) = 1. Find(x): Follow parent pointer to root. Return root. Union(A,B): Move all elements from smallest set to larger set (change parent pointers). I.e., set p(b) = A and size(a) = size(a) + size(b). 23

24 A Simple Union-Find Data Structure Quick Find: Union(A,B): Move all elements from smallest set to larger set (change parent pointers). Union(7,5) Union(3,1) Union(7,8) Union(3,7)

25 A Simple Union-Find Data Structure Quick Find: Each set represented by a tree of height at most one. Elements are represented by nodes. Root is also identifier. Make-Set(x): Create a new node x. Set p(x) = x and size(x) = 1. Find(x): Follow parent pointer to root. Return root. Union(A,B): Move all elements from smallest set to larger set (change parent pointers). I.e., set p(b) = A and size(a) = size(a) + size(b). Analysis: Make-Set(x) and Find(x): O(1) Union(A,B): O(n) 25

26 Amortized Complexity of Quick-Find Amortized analysis: Consider a sequence of k Unions. Observation 1: How many elements can be touched by the k Unions? Consider an element x: What can we say about the size of the set containing x before and after a union that changes x s parent pointer? How large can the set containing x be after the k Unions? How many times can x s parent pointer be changed? 26

27 Amortized Complexity of Quick-Find Amortized analysis: Each time x s parent pointer changes the size of the set containing it at least doubles. At most 2k elements can be touched by k unions. Size of set containing x after k unions at most 2k. x s parent pointer is updated at most lg(2k) times. In total O(k log k) parent pointers updated in a sequence of k unions. Amortized time per union: O(log k). Lemma. Using the Quick-Find data structure a Find operation takes worst case time O(1), a Make-Set operation time O(1), and a sequence of n Union operations takes time O(n log n). 27

28 A Better Union-Find Data Structure Union-by-Weight or Union-by-Rank. Union-by-Weight. Make the root of the smallest tree a child of the root of the bigger tree. Union-by-Rank. Each node x has an integer rank(x) associated. Make-Set(x): Create a new node x. Set p(x) = x and rank(x) = 0. Find(x): Follow parent pointers to the root. Return the root. Union(A,B): 3 cases: rank(a) > rank(b). Make B a child of A. rank(a) < rank(b). Make A a child of B. rank(a) = rank(b). Make B a child of A and set rank(a) = rank(a)+1. 28

29 Analysis of Union-by-Rank Increasing ranks. rank(x) < rank(p(x)). A root of set containing x: Find(x) takes O(rank(A)+1) time. rank(a) lg n: Show A 2 rank(a) by induction. A=Makeset(x): rank(a)=0 and A = 2 0 = 1. A=Union(B,C): 2 cases rank(a)=rank(b) or rank(a)=rank(c): ok, since set only got larger. rank(b)=rank(c)=k and rank(a)=k+1. A = B + C 2 k + 2 k = 2 k+1. Lemma. Using the Union-by-Rank data structure a Find operation takes worst case time O(log n), and a Make-Set or Union operation takes time O(1). 29

30 Path Compression Path compression. After each Find(x) operation, for all nodes y on the path from x to the root, set p(y) = x. Example. Find(6)

31 Path Compression Path compression. After each Find(x) operation, for all nodes y on the path from x to the root, set p(y) = x. Tarjan: Union-by-Rank and path compression. Starting from an empty data structure the total time of n Makes-Set operations, at most n Union operations, and m Find operations takes time O(n + m α(n)). α(n). Extremely slowly growing inverse Ackermann s function. Analysis complicated, but algorithm simple. 2 one-pass variants path halving and path splitting. Same asymptotic running time. 31

32 Ackermann s function Ackermann s function { j + 1 if k = 0, A k (j) = A (j+1) k 1 (j) if k 1. Inverse Ackermann: Grows extremely slowly. α(n) = min{k : A k (1) n}. 32

33 Path Compression: Analysis Potential function Potential of node x after i operations: Potential of forest after i operations: φ i (x). Φ i (x) = x φ i (x). Auxiliary functions: level(x) = max{k : rank(p(x)) A k (rank(x))}. iter(x) = max{j : rank(p(x)) A (j) level(x) (rank(x))} Properties: (1) 0 level(x) α(x). (2) 1 iter(x) rank(x). 33

34 Path Compression: Analysis Potential function φ i (x) = { α(n) rank(x) if x root or rank(x) = 0 (α(n) level(x)) rank(x) iter(x) if x not root and rank(x) 1. Potential of forest after i operations: Properties: (1) 0 level(x) α(x). Φ i (x) = x φ i (x). (2) 1 iter(x) rank(x). (3) 0 φ i (x) α(n) rank(x). (4) If x not root and rank(x) > 0, φ i (x) < α(n) rank(x). 34

35 Path Compression: Analysis Potential function φ i (x) = { α(n) rank(x) if x root or rank(x) = 0 (α(n) level(x)) rank(x) iter(x) if x not root and rank(x) 1. Potential of forest after i operations: Makeset(x): O(1) Union(A,B): O(α(n)) Find(x): O(α(n)) Φ i (x) = x φ i (x). Lemma (used in analysis of Union and Find). Suppose x not root, and ith operation Union or Find. Then φ i (x) φ i 1 (x). If also rank(x) > 0 and either level(x) or iter(x) changes, then φ i (x) φ i 1 (x) 1. 35

36 Summary Amortized analysis. 2 Examples: Dynamic tables, stack with multipop. Union-Find Data Structure. Union-by-Rank + path compression: worst case + amortized bounds. 36