Lecture 13: AVL Trees and Binary Heaps

Transcription

1 Data Structures Brett Bernstein Lecture 13: AVL Trees and Binary Heaps Review Exercises 1. ( ) Interview question: Given an array show how to shue it randomly so that any possible reordering is equally likely. static void shue(int[] arr) 2. Write a function that given the root of a binary search tree returns the node with the largest value. public static BSTNode<Integer> getlargest(bstnode<integer> root) 3. Explain how you would use a binary search tree to implement the Map ADT. We have included it below to remind you. Map.java //Stores a mapping of keys to values public interface Map<K,V> { //Adds key to the map with the associated value. If key already //exists, the associated value is replaced. void put(k key, V value); //Gets the value for the given key, or null if it isn't found. V get(k key); //Returns true if the key is in the map, or false otherwise. boolean containskey(k key); //Removes the key from the map void remove(k key); //Number of keys in the map int size(); } What requirements must be placed on the Map? 4. Consider the following binary search tree

2 Perform the following operations in order. (a) Remove 1. (b) Remove 1. (c) Add 13. (d) Add 8.. Suppose we begin with an empty BST. What order of add operations will yield the tallest possible tree? Review Solutions 1. One method (popular in programs like Excel) is to generate a random double corresponding to each element of the array, and then sort the array by the corresponding doubles. Here is a dierent method that avoids sorting (used by Collections.shue). import java.util.random; RandomShue.java public class RandomShue { static void swap(int[] arr, int i, int j) { int tmp = arr[i]; arr[i] = arr[j]; arr[j] = tmp; } //Iterative implementation public static void shue(int[] arr) { Random ran = new Random(); for (int i = arr.length 1; i >= 1; i) swap(arr,i,ran.nextint(i)); } //Recursive implementation public static void shuerec(int[] arr) { srhelp(arr, arr.length, new Random()); } public static void srhelp(int[] arr, int len, Random ran) { if (len <= 1) return; swap(arr, len 1, ran.nextint(len)); srhelp(arr,len 1,ran); } } 2

3 Above we give an iterative and a recursive implementation. We rst randomly choose one of the elements and swap it into the nal position. Then we repeat the process on the rst n 1 elements (i.e., randomly choose last element, and then randomly shue rst n 1 elements). 2. public static BSTNode<Integer> getlargest(bstnode<integer> root) { while (root.getright()!= null) root = root.getright(); return root; } 3. We require the keys of the Map to be Comparable or a Comparator to be provided. In each node instead of simply storing a value, store a key-value pair (i.e., an entry). The BST will be ordered by the keys. Here all operations above will be Θ(h) in the worst case (we don't have containsvalue above which would always be Θ(n)). In a moment, we will show how to achieve Θ(lg n) height trees which gives an ecient Map implementation without needing a hash function (but we need ordered keys). 4. (a) (b) (c) (d)

4 Ascending or descending order (i.e., sorted or reverse sorted). Data Structure: AVL Tree We have learned how to store values in a BST data structure that supports adding, removal, and searching in time Θ(h), where h is the height of the tree. We have also seen that in the worst case, the height can be Θ(n), the size of the tree. If we can keep the tree fairly balanced, then we can reduce the height to Θ(lg n) and obtain a fairly ecient data structure. To maintain a balanced structure we will use a technique invented by Adelson-Velskii and Landis (hence the name AVL tree). The idea is to keep a counter in every node (called a balance factor ) that measures the dierence between the heights of the left and right subtrees (right minus left). We will add the following added constraint on our BST ˆ No balance factor will be greater than 1 or less than. Any BST that satises this constraint will have Θ(lg n) height. Every time we add or remove a node we may violate this constraint. To return the tree to balance we will employ a technique called a rotation. Consider the following AVL tree: Each node has the value and balance factor in it. Now suppose we add 16 (just as we would in a BST). 4

5 As you can see, the tree now violates our constraints on the balance factors. To remedy the issue, we nd the closest (lowest) ancestor of the newly added node that is out of balance. In this case, it would be the node containing the 2. We then perform a rotation so that takes the place of 2, and 2 becomes the right child of. The resulting tree is Now the tree satises the constraint again. Let's generalize this example to all possible ways adding a node can imbalance the tree. First notice that if adding a value will cause a node to violate its balance factor constraint, then it must have been 1 or already, since adding a single node can add at most 1 to the height of any subtree. Let's assume a node currently has balance factor 1 and some value X. We will model the possible ways that the node containing X will be the lowest node that violates the balance factor constraint (by adding values).

6 X Y C k k A B k Above Y < X and all of the labeled subtrees have the same height k. The subtree rooted at X has height k + 1. We rst handle the case that a node is added to subtree A that increases its height. X -2 Z Y C k B k A 1 A 2 Here we assumed the value added was smaller than Y. Note that the subtree rooted at X now has height k + 2. We then zoom in on the subtree A by drawing its root Z and subtrees A 1 and A 2. The possible heights of A 1, A 2 are (this will not eect our course of action though): 1. If k = then A 1, A 2 are both empty. Z has a balance factor of 2. If k > and the new value is smaller than Z then A 1 has height k and A 2 has height k 1. Z has a balance factor of If k > and the new value is greater than Z then A 1 has height k 1 and A 2 has height k. Z has a balance factor of. To x this we perform a rotation that puts Y where X is: 6

7 Y Z X A 1 A 2 B C k The balance constraints are no longer violated. Note that the subtree rooted at Y has height k + 1, the same as the subtree rooted at X before a value was added. Thus all nodes above Y in the tree are now balanced as well. This is the so called left-left case since Z is the left child of Y which is the left child of X. The other left-right case occurs when we add a value that is larger than Y to our original tree (the picture with A, B, and C) above. Y W X -2 C k k A B 1 B 2 Here we have zoomed in on the subtree B. We have similar cases as above that determine the heights of B 1, B 2. Here we perform a rotation that puts W in place of Y : W X -2 Y C k B 2 k A B 1 This gives us a case like the left-left situation. Thus we perform a second rotation as above putting W in place of X: 7

8 W Y X k A B 1 B 2 C k Again we have restored balance and the resulting tree has height k + 1. Thus all ancestors of W are balanced too. Using the operations above and their reections (right-right and right-left cases) we can enforce the balance constraints on every add operation. The TreeMap and TreeSet in Java use a related data structure called a red-black tree that uses a slightly dierent constraint for balance, but also uses rotations to enforce the constraint. AVL Tree Exercises 1. Consider the following BST (a) Add balance factors to all of the nodes. (b) Show what happens when we add the node 13 (treating the tree as an AVL tree). (c) Next add 7 to the tree (treating the tree as an AVL tree). (d) ( ) Next delete, 7, and 6 from the tree. balance constraints. Make sure to remedy any violated 2. Give a simple Θ(n lg n) sorting algorithm assuming you have access to an AVL Tree data structure. 3. ( ) The AVL Tree deletion algorithm is similar to the addition algorithm, but can cause as many as Θ(h) rotations to occur where h is the height of the tree. Can you explain why? 4. ( ) Let minnodes(h) denote the minimum number of nodes you need to have an AVL- Tree of height h. For h >= 2 give a recurrence relation satised by minnodes(h). 8

9 . In the Priority Queue ADT elements have an ordering (Comparable or Comparator). When you dequeue an element, instead of removing the earliest added element, you remove the earliest element in the given ordering. You can imagine a situation where you are ordering tasks to work on, but each task has a priority governing when you must work on it. It has the following operations (a) add: Adds an element to the Priority Queue (b) dequeue: Removes the smallest element (with respect to the ordering) Describe an ecient implementation of this ADT. AVL Tree Solutions 1. (a) (b) We perform the left-right sequence of 2 rotations after adding (c) After adding 7 we perform the right-right rotation which is simply the reection of the left-left rotation. 9

10 (d) After the removal the root is unbalanced. We will treat this like the right-right case and rotate the 13 into the root Add all of the elements to the AVL Tree, then perform an inorder traversal. If we want to handle duplicates, we can either extend our AVL Tree/BST to allow duplicates, or instead of storing values, we can store lists of values that are equivalent with respect to the ordering. This method of sorting isn't used. 3. Note that after an add operation, our rotations returned the eected subtree to the same height it was before the add operation. Thus no ancestors above the lowest unbalanced node had to be xed. After a remove operation the eected subtree may have a lower height than before, and thus ancestors of the lowest unbalanced node may need to be xed as well. 4. minnodes(h) = 1 + minnodes(h 1) + minnodes(h 2). To see why, note that we must have a subtree of height h 1 so that the whole tree has height h. Secondly, the smallest we can make the other subtree is height h 2 due to the balance constraint. This can be used to show that minnodes grows faster than the Fibonacci sequence, which grows exponentially. This in turn can be used to show that the height of an AVL tree is Θ(lg n).. Use an AVL Tree. Adds simply add nodes to the tree. To dequeue we simply remove the smallest value. Both operations require Θ(lg n) in the worst-case. 1

11 Data Structure: (Binary) Min Heap Above we saw how to implement a PriorityQueue using an AVL Tree. Here we will give a new data structure that can be used to implement a PriorityQueue and is way more ecient in practice (lower constants embedded in the Θ-terms). We will allow duplicates values. A min heap is a binary tree with the following two constraints: 1. Ordering: Every node is equal to or smaller than its children. 2. Completenees: Every level of the tree is full but the last level. In the last level the nodes ll up the leftmost positions. The second constraint sounds odd, but it will enable a very ecient implementation. Instead of using binary tree nodes to store our values, we will just use an array containing all of the elements as they would appear in a level-by-level (breadth rst) traversal. Consider the following min heap: We then store this in an ArrayList The nice thing about this format is that we can easily nd the children and parent of any node. Suppose you are at the node with index k in the array. 1. Left child: 2k Right child: 2k Parent: (k 1)/2 (Java integer division; gives on root) We will justify the left child formula. The rest will follow from that. Note that level of nodes at depth d contains the indices 2 d 1 through 2 d 2. Thus the kth node in that level has index 2 d 1 + (k 1). Applying the left child formula, we obtain index 2(2 d 1 + (k 1)) + 1 = 2 d 1 + 2(k 1). This is the index in level d + 1 just after the 2(k 1) children of the nodes preceding our original node in level d. 11

12 The nice formulas above are made possible by the array storage format. Since our min heaps have the completeness property, the array format isn't wasteful. What remains is to implement the two main operations of the min heap: add and removemin. The key to all heap operations is to rst guarantee that the completeness property holds. After the shape is correct, we then make a few updates to x the ordering constraint. Suppose we want to add the node 1 to our heap above. We rst add 1 in the next available spot in the lowest level: Next we need to x the ordering constraint. We use an operation called sift-up. Take the newly added node an compare it with its parent. If it is smaller, swap, and repeat the process on the parent. This is depicted below To remove the mininum we rst swap the top value with the last value. Then we can safely remove the last value and maintain the shape. Finally, we correct the ordering constraint by checking if the root is bigger than its smallest child. If so, swap and then repeat on the node you swapped with. This process is sometimes called sift-down. Min Heap Exercises 1. Consider the following min heap (a) What is the index of 11 in the corresponding array? (b) Add an 8 to the min heap. (c) Then add a 1 to the min heap. (d) Then removemin from the min heap.

13 (e) Then removemin from the min heap. 2. A max heap is just like a min heap, but each node must be larger than its children. Explain why a max heap data structure is unnecessary if you already have a min heap. 3. What are the worst-case runtimes of add and removemin? 4. Assuming you have access to a min-heap, show how to sort a list of Comparable values in worst-case time Θ(n lg n).. How long does it take to nd a value in a min heap? 6. ( ) Sometimes it is useful to remove an arbitrary element from a min heap given its index. Explain how to do this in worst-case Θ(lg n) time. 7. ( ) Given an array of n comparable values, show how to turn it into a min heap. There is a Θ(n) worst-case implementation. Min Heap Solutions 1. (a) 4 (b) No sifting is required (c) Here we must sift-up performing 3 swaps (d) We swap the 7 into the root, remove the 1, and then sift-down (swap 7 with then 6)

14 (e) We swap the with the 8, remove the, and then sift-down (swap 8 with 6 then 7) Just use a min heap but reverse the ordering. 3. Both are Θ(lg n) since our tree is always well-balanced. 4. Add all elements to a min heap, and then repeatedly call removemin to pull them out in order. This is called Heapsort (better than our AVL Tree sorting algorithm above due to the low constant on all heap operations).. Θ(n) in the worst-case (consider a really large value that isn't in the heap) since the heap ordering property doesn't aid in searches like the BST property does. 6. We use the following steps. (a) Swap the item to be removed with the last item and then remove the last item. (b) The swap may have broken the ordering property so: i. Check if the swapped item is smaller than its parent. If so, do the sift-up procedure on it. ii. Otherwise, do the sift-down procedure on it. 7. The slow method is to just call add n times giving a worst-case Θ(n lg n) runtime. A better method is to loop backwards through the array and run the sift-down procedure on every value. To see why the runtime is Θ(n) we consider the work done by sift-down at every node. For simplicity, let's assume every level of the heap is full. Let the height of a node be the height of the subtree it is the root of. All nodes will require at most C steps to compare them with their children in the sift-down procedure. Nodes of height at least one will require at most an extra C steps, since they they could undergo a swap. Nodes of height at least two will require an extra C steps on top of that, and so forth. But, each time we increase the height we halve the number of nodes we consider. Since Cn + Cn/2 + Cn/4 + = 2Cn = Θ(n) the result follows. 14