Wrap up Amortized Analysis; AVL Trees. Riley Porter Winter CSE373: Data Structures & Algorithms

Transcription

1 CSE 373: Data Structures & Wrap up Amortized Analysis; AVL Trees Riley Porter

2 Course Logistics Symposium offered by CSE department today HW2 released, Big- O, Heaps (lecture slides ave pseudocode tat will elp a lot) Weekly summaries out soon 2

3 Review: Amortized Complexity For a Stack implemented wit an array can we claim pus is O() Eme if resizing is O(n) Eme? We can t, but we can claim it s an O() amorezed operaeon Wy is tis good? Wy do we care? If amorezed is good enoug for our problem, ten it s great to be able to say O() instead of O(n) 3

4 Review: Amortized Complexity Like an Average. N inserts at unit cost + insert at N unit cost = N * + * N 2N overall cost for N + insereons = 2N cost / (N + ) insereons = O( 2N/(N+) ) = O() amorezed cost 4

5 Not-Amortized Complexity Wat if we only add 3 slots instead of N slots? 3 inserts at unit cost + insert at N unit cost = 3 * + * N = N + 3 overall cost for N + insereons = N + 3 cost / (3 + ) insereons = O( (N+3)/(3+) ) = O(N) Not AmorEzed! Didn t build up enoug credit 5

6 Example #: Resizing stack A stack implemented wit an array were we double te size of te array if it becomes full Claim: Any sequence of pus/pop/isempty is amorezed O() Need to sow any sequence of M operaeons takes Eme O(M) Recall te non- resizing work is O(M) (i.e., M*O()) Te resizing work is proporeonal to te total number of element copies we do for te resizing So it suffices to sow tat: A^er M operaeons, we ave done < 2M total element copies (So average number of copies per operaeon is bounded by a constant) 6

7 Amount of copying After M operations, we ave done < 2M total element copies Let n be te size of te array after M operations Ten we ave done a total of: n/2 + n/4 + n/8 + INITIAL_SIZE < n element copies Because we must ave done at least enoug pus operations to cause resizing up to size n: M n/2 So 2M n > number of element copies 7

8 Have to be careful If array grows by a constant amount (say 000), operaeons are not amorezed O() A^er O(M) operaeons, you may ave done Θ(M 2 ) copies If array doubles wen full and srinks wen /2 empty, operaeons are not amorezed O() Terrible case: pop once and srink, pus once and grow, pop once and srink, If array doubles wen full and srinks wen 3/4 empty, it is amorezed O() Proof is more complicated, but basic idea remains: by te Eme an expensive operaeon occurs, many ceap ones occurred 8

9 Amortized Complexity Summary Like an average You re building up credit wit N ceap tasks proporeonal to expensive N task SomeEmes ard to prove, but useful if your applicaeon doesn t require every single operaeon to be ceap. Tere s anoter example on slides from Wednesday involving Queues, if you re curious 9

10 Review: Balanced BST Observa=on BST: te sallower te beeer! For a BST wit n nodes inserted in arbitrary order Average eigt is O(log n) see text for proof Worst case eigt is O(n) Simple cases, suc as insereng in key order, lead to te worst- case scenario Solu=on: Require a Balance Condi:on tat. Ensures dept is always O(log n) strong enoug! 2. Is efficient to maintain not too strong!

11 Te AVL Balance Condition Le^ and rigt subtrees of every node ave eigts differing by at most Defini=on: balance(node) = eigt(node.le^) eigt(node.rigt) AVL property: for every node x, balance(x) Ensures small dept Will prove tis by sowing tat an AVL tree of eigt must ave a number of nodes exponen=al in Efficient to maintain using single and double rotaeons

12 Te AVL Tree Data Structure Structural proper=es. Binary tree property 2. Balance property: balance of every node is between - and 8 Result: 5 Worst- case dept is O(log n) Ordering property Same as for BST Defini=on: balance(node) = eigt(node.le^) eigt(node.rigt) 5

13 An AVL tree? 6 Heigt = 3 Balance = - 4 Heigt = Balance = 8 Heigt = 2 Balance = - Heigt = 0 Balance = 0 7 Heigt = 0 Balance = Heigt = Balance = 0 Heigt = 0 Balance = 0 Heigt = 0 Balance = 0

14 An AVL tree? Heigt = 2 Balance = -2 Heigt = 3 Balance = Heigt = Balance = 6 Heigt = 0 Balance = 0 Heigt = 4 Balance = 2 7 Heigt = 0 Balance = 0 8 Heigt = Balance = 0 Heigt = 0 Balance = 0 2 Heigt = 0 Balance = 0 No!

15 Intuition: compactness If te eigts differ by at most, your two subtrees are rougly te same size If tis is true at every node, it s true all te way down If tis is true all te way down, your tree winds up compact. Heigt is O(logN) We ll revisit te formal proof of tis soon -2 -

16 AVL Operations If we ave an AVL tree, te eigt is O(log n), so find is O(log n) But as we insert and delete elements, we need to:. Track balance 2. Detect imbalance 3. Restore balance 0 Is tis AVL tree balanced? Yep! 5 5 How about after insert(30)? No, now te Balance of 5 is off 7 30

17 Keep te tree balanced 0 key 3 value 0 3 eigt cildren Track eigt at all times!

18 AVL find: Same as BST find AVL tree Operations AVL insert: First BST insert, ten ceck balance and potentially fix te AVL tree Four different imbalance cases AVL delete: Te easy way is lazy deletion Oterwise, do te deletion and ten ave several imbalance cases

19 Insert: detect potential imbalance. Insert te new node as in a BST (a new leaf) 2. For eac node on te pat from te root to te new leaf, te insereon may (or may not) ave canged te node s eigt 3. So a^er recursive insereon in a subtree, detect eigt imbalance and perform a rota(on to restore balance at tat node Type of rotaeon will depend on te locaeon of te imbalance (if any) Facts about insert imbalances: If tere s an imbalance, tere must be a deepest element tat is imbalanced a^er te insert A^er rebalancing tis deepest node, every node is balanced So at most one node needs to be rebalanced

20 Case #: Example Insert(6) Insert(3) Insert() Tird insereon violates balance property appens to be at te root Wat is te only way to fix tis (te only valid AVL tree wit tese nodes?

21 Fix: Apply Single Rotation Single rota=on: Te basic operaeon we ll use to rebalance Move cild of unbalanced node into parent posieon Parent becomes te oter cild (always okay in a BST!) Oter subtrees move in only way BST allows (next slide) AVL Property violated ere Intuition: 3 must become root New parent eigt is now te old parent s eigt before insert 0

22 Te example generalized Node imbalanced due to insereon somewere in len- len grandcild tat causes an increasing eigt of 4 possible imbalance causes (oter tree coming) First we did te insereon, wic would make a imbalanced + b a +2 a +2 b + Z +3 Z X Y X Y

23 Te general left-left case Node imbalanced due to insereon somewere in len- len grandcild of 4 possible imbalance causes (oter tree coming) So we rotate at a,using BST facts: X < b < Y < a < Z a +2 b + X Y +3 b + Z X Y +2 + a Z A single rotaeon restores balance at te node To same eigt as before insereon, so ancestors now balanced

24 Anoter example: insert(6) Were is te imbalance?

25 Anoter example: insert(6) Were is te imbalance? 22

26 Anoter example: insert(6)

27 Te general rigt-rigt case Mirror image to le^- le^ case, so you rotate te oter way Exact same concept, but need different code X a b + + a b +2 + Z Y Z X Y

28 Case 3 & 4: left-rigt and rigt-left 2 6 Insert() Insert(6) Insert(3) Is tere a single rotaeon tat can fix eiter tree? 6 Insert(6) Insert() Insert(3) 3 0

29 Wrong rotation #: Unfortunately, single rotaeons are not enoug for insereons in te len- rigt subtree or te rigt- len subtree Simple example: insert(), insert(6), insert(3) First wrong idea: single rotaeon like we did for le^- le^

30 Wrong rotation #2: Unfortunately, single rotaeons are not enoug for insereons in te len- rigt subtree or te rigt- len subtree Simple example: insert(), insert(6), insert(3) Second wrong idea: single rotaeon on te cild of te unbalanced node

31 Sometimes two wrongs make a rigt First idea violated te BST property Second idea didn t fix balance But if we do bot single rotaeons, stareng wit te second, it works! (And not just for tis example.) Double rotaeon:. Rotate problemaec cild and grandcild 2. Ten rotate between self and new cild Intuition: 3 must become root

32 Te general rigt-left case X +3 a X U +3 a U + c V +2 c b - V +2 b - Z + X + a Rotation : b.left = c.rigt c.rigt = b a.rigt = c Rotation 2: U a.rigt = c.left c.left = a root = c c +2 - V b + Z Z

33 Comments Like in te le^- le^ and rigt- rigt cases, te eigt of te subtree a^er rebalancing is te same as before te insert So no ancestor in te tree will need rebalancing Does not ave to be implemented as two rotaeons; can just do: X +3 a U + c V b - +2 Z X + a U c +2 b - V + Z Easier to remember tan you may tink: ) Move c to grandparent s posieon 2) Put a, b, X, U, V, and Z in te only legal posieons for a BST

34 Te last case: left-rigt X Mirror image of rigt- le^ Again, no new concepts, only new code to write +2 b U +3 c a + V - Z X + b U c +2 - V a + Z

35 Insert, summarized Insert as in a BST Ceck back up pat for imbalance, wic will be of 4 cases: Node s le^- le^ grandcild is too tall (len- len single rota:on) Node s le^- rigt grandcild is too tall (len- rigt double rota:on) Node s rigt- le^ grandcild is too tall (rigt- len double rota:on) Node s rigt- rigt grandcild is too tall (rigt- rigt double rota:on) Only one case occurs because tree was balanced before insert A^er te appropriate single or double rotaeon, te smallest- unbalanced subtree as te same eigt as before te insereon So all ancestors are now balanced

36 Efficiency Worst-case complexity of find: O(log n) Tree is balanced Worst-case complexity of insert: O(log n) Tree starts balanced A rotation is O() and tere s an O(log n) pat to root (Same complexity even witout one-rotation-is-enoug fact) Tree ends balanced Worst-case complexity of buildtree: O(n log n) Takes some more rotation action to andle delete

37 Pros and Cons of AVL Trees Arguments for AVL trees:. All operaeons logaritmic worst- case because trees are always balanced 2. Heigt balancing adds no more tan a constant factor to te speed of insert and delete Arguments against AVL trees:. Difficult to program & debug [but done once in a library!] 2. More space for eigt field 3. AsymptoEcally faster but rebalancing takes a liele Eme 4. Most large searces are done in database- like systems on disk and use oter structures (e.g., B- trees, a data structure in te text) 5. If amor=zed (later, I promise) logaritmic Eme is enoug, use splay trees (also in text)

38 Today s Takeaways Review of Amortized Analysis: feel comfortable proving a runtime s amortized cost AVL trees: understand te AVL balance condition be able to identify AVL trees intuition on wy te eigt is O(logN) understand AVL inserts and rotations