Amortized Analysis. Ric Glassey

Transcription

1 Amortized Analysis Ric Glassey

2 Overview Amortized Analysis Aim: Develop methods of determining the average cost of operations on a data structure Motivation: We do not always want to think about the worst case scenario, rather what the average or amortized cost of an operations is Key concepts: Amortized Cost Aggregate Analysis Accounting Method Demonstration: insertion into dynamic table 2

3 AMORTIZED COST 3

4 Best to think in terms of money... If you receive 1024kr per month You spend 256kr per month on small things You invest the 768kr left in a savings account Eventually you spend 9216kr on something big The amortized cost is only 1024kr per month Not the worst case single month costing (9216kr) But this is not Financial Management 101 We can use this concept to think about the average time costs of (small and big) operations on data structures 4

5 DYNAMIC TABLES 5

6 Using Dynamic Tables How big should we make a hashtable? We do not know how many entries are required We want to allocate enough space to ensure good performance More space will ensure less hashing collisions Remember we hash then compress to within table size We do not have infinite resources We should aim to maintain a healthy load factor What should happen as load factor approaches 1? table size = N and entries = n load factor is α = n/n 6

7 Dynamic Table in Operation Create a new table of size 1 insert(1) into table Overflow on insert(2) insert(1) 1 insert(2)? 7

8 Dynamic Table in Operation copy 1 insert(1) insert(2) 1 overflow 1 2 insert 2 On insert(2) an overflow occurs, no space. Create a new table twice the size, copy 1 into new table, insert 2 into empty space What now happens if we try to insert(3)? 8

9 Dynamic Table in Operation copy 1,2 insert(1) insert(2) insert(3) 1 2 overflow insert On insert(3) an overflow occurs, no space. Create a new table twice the size, copy 1,2 into new table, insert 3 into empty space How many opera?ons just took place? What now happens if we try to insert(4)? 9

10 Dynamic Table in Operation insert(1) insert(2) insert(3) insert(4) insert 4 On insert(4) there is an available slot, so no overflow occurs. No further opera?ons required How many opera?ons just occurred? What now happens if we try to insert(5)? 10

11 Dynamic Table in Operation copy 1,2,3,4 insert(1) insert(2) insert(3) insert(4) insert(5) overflow insert 5 On insert(5) there was overflow Once again, double the space in the table, copy old values into new table, insert 5 How many opera?ons just occurred? What now happens if we try to insert(6) or insert(7) or insert(8)? 11

12 Dynamic Table in Operation insert(1) insert(2) insert(3) insert(4) insert(5) insert(6) insert(7) insert(8) insert(9) ? As we can see, things are improving, we are gejng more done, with less resizing What happens on insert(9)? How many insert opera?ons before the next table resize? 12

13 Worst Case Analysis of Dynamic Table As we noticed, the worst case insertion caused a table resize, copying all entries, and inserting the new item O(n) For a sequence of n operations, the worst case time will be n. O(n) = O(n 2 ) However... We observed something about the progress of the operation cost and the distance between table doubling 13

14 AGGREGATE ANALYSIS 14

15 Aggregate Analysis We consider T(n) to be the total cost for a sequence of n operations Average cost will simply be T(n)/n We say that the average cost is the amortized cost of each operation All operations have the same amortized cost Think about: why having the same cost for all operations might not be so useful in all contexts? 15

16 Counting the costs table resize event Insertion Op (i) Size of Table In the sequence of opera?ons, there were only 4 table resize events Whilst there were 10 insert operabons Let s assume it costs 1 unit of Bme to insert an item (with no resize) Also, it costs 1 unit of Bme to copy an item from the old table to the new And the cost of alloca?ng the new table in memory comes for free :- ) 16

17 Counting the costs table resize event Insertion Op (i) Size of Table Cost of all Ops (c) avg( ) < 10^2 For each insert opera?on, we can count up all the operabon costs No?ce where we take a hit, and where we do not As a sanity check, we can show that so far, we are no where near O(n 2 ) for cost 17

18 Counting the costs Insertion Op (i) Size of Table Cost of insert (c) Cost of Copy We rarely double the double, only when i-1 is a power of 2 sum of n insertions + sum of [[which]] series? Finally, if we separate the raw inser?on costs from the resize costs, a parern emerges 18

19 Average cost Worst case cost was O(n 2 ), but this cannot hold Total cost c i is in fact much better, n c i n + i=1 lg(n 1) j=0 2 j <n+2n =Θ(n) Therefore, the amortized cost per operation is constant Θ(n)/n =Θ(1) 19

20 ACCOUNTING METHOD 20

21 Accounting Method We imagine that each operation on a data structure can incur a financial cost (nkr) Different operations can cost different amounts Determined by a process of trial and error Overcharge for some operations It only costs 1Kr to insert, but we charge 3Kr anyway This overcharging occurs early in sequence The remaining 2Kr will accrue in credit Whenever a more expensive operation happens, it can draw from this credit Key invariant - we cannot go into debt! 21

22 Counting the cost (II) For our dynamic table, we have the cost model To insert, we charge amortised cost of 3Kr 1Kr of that pays for the insertion 2Kr is stored as credit, when we need to pay for table resize When table is resized 1Kr is charged to move a recent item inserted 1Kr is charged to copy an old item already inserted Invariant: credit 0 In the back of your mind keep asking: why 3Kr? 22

23 Counting the cost (II) Initial state, empty, or just after a resize operation Here, there are 4 items, and we think of them as 0Kr Kr insert( item 5) will charge amortized cost 3Kr It costs 1Kr to insert The left over 2Kr are 'deposited as credit' credit = 2Kr 23

24 Counting the cost (II) Each insertion operation adds to the credit Kr insert (item 5), charge 3Kr, only pay 1Kr, save 2Kr credit = 2Kr Kr 2Kr insert (item 6), charge 3Kr, only pay 1Kr, save 2Kr credit = 4Kr Kr 2Kr 2Kr insert (item 7), charge 3Kr, only pay 1Kr, save 2Kr credit = 6Kr Kr 2Kr 2Kr 2Kr insert (item 8), charge 3Kr, only pay 1Kr, save 2Kr credit = 8Kr 24

25 Counting the cost (II) insert(item 9) causes overflow and requires table resize Kr 2Kr 2Kr 2Kr / Kr 2Kr 2Kr 2Kr We have 8Kr and 8 items at cost 1Kr to be copied insert (item 9), charge 3Kr, only pay 1Kr, save 2Kr credit = 2Kr Kr 25

26 Counting the cost (II) Insertion Op (i) Size of Table Cost of insert (c) Cost of Copy Amortised Cost (ĉi) 2* In the Bank (ci) pay from credit for the resize start building credit we can observe that the amortised cost will be 3n, as long as sum(ĉi) sum(ci) 26

27 Average Cost As shown, the invariant credit 0 is held Amortised cost (ĉ i ) of 3n creates an upper bound on the true cost (c i ): n i=1 ĉ i n i=1 c i For n operations, we find a constant 3n/n Θ(3n/n) =Θ(1) 27

28 Comparison of Aggregate and Accounting Aggregate analysis is intuitive It lacks the precision of accounting method We are only considering single types of operations in the sequences (shown earlier) Accounting method provides more precision, and allows different operations to have specific costs associated with them It relies on good knowledge of the data structure behaviour Initial costs must be determined by trial and error 28

29 Challenge We did not discuss deletion from the dynamic table, which may have different costs We want to maintain a healthy load factor, and also not waste too much memory with unfilled slots What is a good strategy for table contraction? Don t change it (memory is cheap) Divide it in 2 when α = n/n =.5 Something else? 29

30 Readings Introduction to Algorithms, 3 rd Edition Chapter 17: Amortized Analysis Full text available via KTH Library KTH:KTH_SFX Or, watch a great lecture from one of the authors recorded at MIT and part of their opencourseware: 30

31 Feedback! This week, I will mostly be asking about: Topic detail throughout course (P1 to P3) Improvements Tipjar Survey will appear at this link Also on the course web page after this lecture 31