Priority Queues and Heaps. Heaps of fun, for everyone!

Transcription

1 Priority Queues and Heaps Heaps of fun, for everyone!

2 Learning Goals After this unit, you should be able to... Provide examples of appropriate applications for priority queues and heaps Manipulate data in heaps (regardless of any implementation) Describe and apply the Heapify and Heapsort algorithms, and analyze their complexity

3 Priority Queues A priority queue is an ADT that maintains a bag (or multiset) of items, where each item has a priority What is a bag? Priority queues support the following operations: Insert(item): add an item to the bag RemoveMin: remove lowest priority item RemoveMax: remove highest priority item (usually instead of RemoveMin)

4 Binary Heaps We can use a binary heap to implement our priority queue efficiently A minimum binary heap is a nearly complete binary tree such that: 1. The data in every node is less than or equal to the data in each of its child nodes 2. The left and right sub-trees of a node are also minimum heaps We can also define a maximum heap in a similar fashion-- unless otherwise stated, you may assume we are referring to a minimum heap in our exercises/ exams.

5 Binary Heaps

6 Binary Heaps Note that two binary heaps may contain the same data but in different positions: These two binary heaps both contain the data 2, 5, 5, 7, 7, 8. They are both perfectly valid minimum binary heaps!

7 Binary heaps and nearly complete... Because a binary heap is a nearly complete binary tree, we can represent it using an array. Thus, unlike other types of trees, which require noncontiguous data (i.e. linked list structures), heaps can be represented contiguously. F D H K L J D F J H K L

8 Navigating the heap Since we know that it can be store using an array, we can navigate the array using a few simple formulae. Given a node at index k: Index of left child: 2k+1 Index of right child: 2k+2 Index of parent: (k-1) / 2

9 Inserting an item into a heap: Suppose we want to insert an element (indexed by the letter E) into the min. heap below. 1. We first insert E at the bottom of the heap (maintaining nearly complete status) 2. Then perform a ReheapUp operation: I. Compare the new item with its parent II. If smaller, swap and repeat step 1 F D J F D H K L M J H K L M E

10 F D J D J K L M K L M Algorithm: ReheapUp( root, bottom ) // subscripts are passed if ( bottom > root ) set parent to subscript of parent of bottom element if ( data[parent] > data[bottom] ) swap( data[parent], data[bottom] ) ReheapUp( root, parent )

11 Removing the item at the top... It is often the case, since this is a priority queue, that we want to remove the top most element (i.e. the highest priority) Once we remove it, to preserve our heap, we move the rightmost element from the bottom of the heap to the top position Next we perform a ReheapDown operation: 1. Compare the new root with its children 2. If smaller, swap with the smallest child, and repeat step 1

12 D F J F J H K L H K H K Algorithm: ReheapDown( root, bottom ) // subscripts are passed if ( root node is not a leaf ) set minchild to subscript of child s smallest data value if ( data[root] > data[minchild] ) swap( data[root], data[minchild] ) ReheapDown( minchild, bottom )

13 Time complexity of ReheapUp and ReheapDown We performing a re-heap up or down, the number of operations depends on the height of the tree. We only ever traverse one path of the tree If we note that a nearly complete binary tree of height h always has between 2 h and 2 h+1-1 nodes, then we can express the height of the heap in terms of the number of nodes (that is, what is the longest possible path?): heightheap= lg n Thus, the time complexity of ReheapUp and ReheapDown is O(lg n)

14 Priority Queue complexity... Inserting an item into a priority queue is therefore O(lg n)......since this amounts to performing ReheapUp RemoveMin (or Max) is also O(lg n)......since this amounts to performing ReheapDown

15 Heap Sort Heap sort guarantees worst case O( nlgn ) If we think about heaps for a moment, you may start to see that a heap implies a sorting algorithm... Heapsort consists of two phases: Heapify: build a heap using the elements to be sorted Sort: Use the heap to sort the data

16 Building a heap from an arbitrary array: Note that it is easier to picture this process if we represent the heap using a binary tree rather than an array. Algorithm Heapify : Let index be the subscript of the last parent node in the tree. while index 0: Perform a ReheapDown operation starting with the node at index. Decrement index by 1.

17 Example: Convert the following array to a heap To do so, picture the array as a nearly complete binary tree: For an array with 8 elements, the subscript or index of the last parent node in the tree, is...?

18

19 Now we sort the array: Algorithm: Note: In this section, we represent the data in both binary tree and array formats. It is important to understand that in practice, the data is stored only as an array. Let swapindex = N 1. While swapindex is greater than 0: Swap data at position swapindex with data at position 0. ReheapDown between positions 0 and swapindex 1.

20

21

22 Implementation: template<typename type> void sort( type* data, int size ) //PRE: The capacity of the array pointed to by data // is at least size. //POST: The first size elements of data have been // sorted in descending order. { int swpindx; } BuildHeap( data, size ); // Heapify algorithm for( swpindx = size 1; swpindx > 0; swpindx-- ) { swap( data[0], data[swpindx] ); ReheapDown( data, 0, swpindx ); }

23 Implementation template<typename type> void BuildHeap( type* data, int size ) //PRE: data points to an array of data of capacity at // least size. //POST: The first size elements of data are a heap. { int index; for( index = (size 2) / 2; index >= 0; index-- ) ReheapDown( data, index, size ); }

24 Implementation: template<typename type> void ReheapDown( type* data, int top, int size ) // PRE: data between subscript top+1 and size 1 // is a heap. // POST: data between subscript top and size 1 is // a heap. { int leftchild = 2 * top + 1; int rightchild = 2 * top + 2; int minchild; if( leftchild < size ) { // find subscript of smallest child if( rightchild >= size data[leftchild] < data[rightchild] ) minchild = leftchild; else minchild = rightchild;

25 Implementation: ReheapDown con t } } // if data at top is greater than smallest // child then swap and continue if (data[top] > data[minchild]) { swap( data[top], data[minchild] ); ReheapDown( data, minchild, size ); } Note: This function is tail-recursive and so it can easily be replaced with an iterative version having O(1) space requirements. Why might this be important for some implementations?

26 Time complexity of Heap sort We need to determine the time complexity of the BuildHeap (i.e., Heapify) operation, and the time complexity of the subsequent sorting operation. The time complexity of the sorting operation once the heap has been built is fairly easy to determine. For each element in the heap, we perform a single swap and a ReheapDown. If there are N elements in the heap, the ReheapDown operation is O( ) and hence the sorting operation is O( ).

27 Time complexity of heap sort We must now consider the cost of building the heap. Surprisingly, it is an O( ) operation! If we examine the BuildHeap function we see that the ReheapDown function is called O( ) times but we have to realize that the ReheapDown operation does not always start at the top of the heap and that it is not called at all on any of the leaf nodes (which account for roughly half the nodes in the tree!)

28 Time complexity of heap sort The time complexity of the BuildHeap function can be expressed in terms of the total number of comparisons and swaps while building the heap. Let us consider the worst case, which is when the last level in the heap is full. Let s colour all the paths from each node, starting with the lowest parent and working up to the root, each going down to a leaf node. The number of edges on the path from each node to a leaf node represents an upper bound on the number of comparison and swap operations that will occur while applying the ReheapDown operation to that node. By summing the total length of these paths, we will determine the time complexity of the BuildHeap function.

29

30 Worst case complexity: Let H be the height of the tree, N be the number of elements in the tree, and E be the number of edges in the tree, then: Total number of coloured edges or swaps = Hence, in the worst case, the overall time complexity of the Heapsort algorithm is: O(N) + O(N lg N) = O(N lg N)

31 Learning Goals After this unit, you should be able to... Provide examples of appropriate applications for priority queues and heaps Manipulate data in heaps (regardless of any implementation) Describe and apply the Heapify and Heapsort algorithms, and analyze their complexity