Algorithms and Data Structures

Algorithms and Data Structures Dr. Malek Mouhoub Department of Computer Science University of Regina Fall 2002 Malek Mouhoub, CS3620 Fall 2002 1

6. Priority Queues 6. Priority Queues ffl ADT Stack : LIFO. ffl ADT Queue : FIFO. ADT Priority Queue : peek the element with the lowest (or ffl highest) priority. Malek Mouhoub, CS3620 Fall 2002 2

Priority Queues ffl 6.1 Model. ffl 6.2 Implementation. ffl 6.3 Binary Heap. Malek Mouhoub, CS3620 Fall 2002 3

6.1 Model 2 operations : ffl insert : add an element to the priority queue. ffl deletemin : find, return and remove the minimum element. Malek Mouhoub, CS3620 Fall 2002 4

ffl insert at the front of the list : O(1) ffl deletemin requires traversing the list : O(N ) Priority Queues 6.2 Implementation Λ 1st case : use a linked list Malek Mouhoub, CS3620 Fall 2002 5

ffl insertion requires O(N ) Priority Queues Λ 2nd case : use a sorted linked list ffl but only O(1) for deletemin. Malek Mouhoub, CS3620 Fall 2002 6

± Ξ Π Priority Queues 3rd case : use a binary search tree ffl O(log N ) average running time for both insertion and deletion. Elements are deleted at each time from the left subtree which ffl affects the balance of the tree. The binary search tree supports many operations that are not ffl required by priority queue. Malek Mouhoub, CS3620 Fall 2002 7

6.3 Binary Heap A binary heap is a complete binary tree : a tree that is completely filled, ffl with possible exception of the bottom level, which is filled from left to right. Can easily be represented by an array (and an integer representing the ffl current heap size) instead of a linked list. For any element in array position i : ffl the left child is in position 2i, the right child is in position 2i + 1, and the parent is in position [i=2]. The operations required to traverse the tree are very simple and are ffl performed quickly (because of the heap-order property). Malek Mouhoub, CS3620 Fall 2002 8

I Priority Queues A B C D E F G H J A B C D E F G H I J 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Malek Mouhoub, CS3620 Fall 2002 9

Heap-Order Property ffl The smallest element should be at the root. Any node should be smaller than all of its descendants (any ffl subtree should also be a heap). Heap-order property : ffl For every node X, the key in the parent of X is smaller than (or equal to) the key in X, with the exception of the root (which has no parent). Malek Mouhoub, CS3620 Fall 2002 10

13 13 21 16 21 16 24 31 19 68 6 31 19 68 65 26 32 65 26 32 Two complete trees (only the left is a heap) Malek Mouhoub, CS3620 Fall 2002 11

Basic Heap Operations Λ Insert 1. Create a hole in the next available location. 2. Place the element to be inserted in the hole if it does not violate the heap order. 3. Otherwise slide the element that is in the hole s parent node into the hole, thus bubbling the hole up toward the root. Continue the process until the element can be placed in the hole. Malek Mouhoub, CS3620 Fall 2002 12

13 21 16 bubbling the hole up 13 21 16 24 31 19 68 6 19 68 65 26 32 65 26 32 Insert 14 31 Malek Mouhoub, CS3620 Fall 2002 13

13 13 16 14 16 6 21 19 68 6 21 19 68 65 26 32 31 65 26 32 31 Insert 14 Malek Mouhoub, CS3620 Fall 2002 14

Insert /* Inserting item into the priority queue, maintaining the heap order. Duplicates are not allowed */ template <Class Comparable> void BinaryHeap<Comparable>::insert(const Comparable & x) if (isfull()) f throw Overflow(); // Percolate up int hole = ++currentsize; for ( ;hole > 1 && x< array[hole/2]; hole /= 2) array[hole] = array[hole/2]; 10 array[hole] = g x; Malek Mouhoub, CS3620 Fall 2002 15

deletemin 1. Create a hole at the root. 2. The last element X in the heap must move somewhere in the heap. 3. Slide the smaller of the hole s children into the hole, thus pushing the hole down one level. Repeat this step until X can be placed in the hole. Malek Mouhoub, CS3620 Fall 2002 16

13 14 16 deletemin 14 16 19 21 19 68 19 21 19 68 65 26 32 31 65 26 32 31 Creation of the hole at the root Malek Mouhoub, CS3620 Fall 2002 17

14 14 16 19 16 19 21 19 68 21 19 68 26 32 31 65 26 32 31 Malek Mouhoub, CS3620 Fall 2002 18

14 14 19 16 19 16 26 21 19 68 26 21 19 68 32 31 65 31 32 Malek Mouhoub, CS3620 Fall 2002 19

Other heap operations ffl A heap has very little ordering information, so there is no way to find any particular element without a linear scan through the entire heap. ffl Assuming that the position of every element is known by some other method, the following operations run in logarithmic worst-case time. decreasekey(p, ): lowers the value of the item at position p by a positive amount. Apercolate up is required sometimes to maintain the heap order. increasekey(p; ): increases the value of the item at position p by a positive amount. Apercolate down is required sometimes to maintain the heap order. remove : removes the node at position p from the heap. This is done by performing decreasekey(p;1) and then performing deletemin(). Malek Mouhoub, CS3620 Fall 2002 20

± Ξ Π Priority Queues Other heap operations Buildheap ffl Takes as input N items and places them into an empty heap. ffl N successive inserts ) O(N ) average case but O(N log N ) worst-case. Malek Mouhoub, CS3620 Fall 2002 21

± Ξ Π Priority Queues Other heap operations Buildheap /** Establish heap order property from an arbitrary arrangement of items. Runs in linear time. */ template <class Comparable> void BinaryHeap<Comparable>::buildHeap() f for (int i = currentsize / 2; i > 0; i ) perlocatedown(i); g Malek Mouhoub, CS3620 Fall 2002 22

Applications of Priority Queues Λ The selection Problem ffl Given a list of N elements, which can be totally ordered, and an integer k, find the kth largest element. ffl Algorithm 1A : read the elements into an array and sort them, returning the appropriate element. ) assuming a simple sorting algorithm, the running time is O(N 2 ). ffl Algorithm 1B : 1. read k elements into an array and sort them. The smallest of these is in the kth position. 2. Process the remaining elements one by one. As an element arrives, it is compared with the kth element in the array. If it is larger, then the kth element is removed, and the new element is placed in the correct place among the remaining k 1 elements. ) O(N _ k) running time. Why? ffl If k = N=2 then both algorithms are O(N 2 ). k is known as the median in this case. ffl The following algorithms run in O(N log N ) in the extreme case of k = N=2. Malek Mouhoub, CS3620 Fall 2002 23

Algorithm 6A ffl Algorithm for finding the kth smallest element 1. Read N elements into an array. 2. Apply the buildheap algorithm to this array. 3. Perform k deletemin operations. The last element extracted from the heap is the answer. ffl Complexity : O(N + k log N ) in the worst case. k = O(N= log N ) ) O(N ) k = N=2 ) N (N log N ) For large values of k : O(k log N ) k = N ) O(N log N ) (Idea of the heapsort). ffl By changing the heap-order property, we will solve the problem of finding the kth largest element. Malek Mouhoub, CS3620 Fall 2002 24

Algorithm 6B ffl Find the kth largest element 1. Same idea as algorithm 1B. 2. At any point in time, maintain a set S of the k largest elements. 3. After the first k elements are read, when a new element is read it is compared with the kth largest element, which we denote by S k (S k is the smallest element in S). If the new element is larger, then it replaces S k in S. 4. At the end of the input, we find the smallest element in S and return it as the answer. ffl O(k + (N k) log k) = O(N log k) in the worst case. Why? Malek Mouhoub, CS3620 Fall 2002 25