Chapter 9 Priority Queues Sometimes, we need to line up things according to their priorities. Order of deletion from such a structure is determined by the priority of the elements. For example, when assigning CPU to a list of jobs, OS will pick up the job with the highest priority, take it out of the queue and give it to the CPU. The associated operations are insertion and deletion of the element. More specifically, in a min priority queue, the element with minimum priority will be deleted; while in a max priority queue, the element with maximum one will be deleted. 1
Implementation A priority queue can be implemented in various ways, e.g., either an unordered list, or an ordered list, will do. But, if we use these simple structures, we have to spend O(n) for either the insertion, or the deletion, operation. Hence, a priority queue is almost always implemented with a complete balanced binary tree with a heap condition, i.e., the value of each node is less (in a min queue), or larger (in a max queue), than it children, if they do exist. 2
Flash back As we found out earlier, a complete balanced binary tree with n nodes has an efficient implementation in an array of size n. For any node i, 1) if i =1, then it is the root. Otherwise, its parent is assigned 2 i. 2) If 2i >n,then this element has no left child. Otherwise, its left child is assigned the number 2i. 3) If 2i+1 >n, then it has no right child. Otherwise, its right child is assigned the number 2i +1. We will switch back and forth between the heap and its array implementation. Moreover, for a complete balanced binary tree with n nodes, its height is log 2 (n +1). As a result, if we add an element in, or delete one from, a complete balanced tree in O(height), we will do it in Θ(log n). 3
How to insert? We initially add the new item into the array as the last one, and, if the heap condition is not satisfied yet, keep on swapping it with its parent. The following shows an insertion into a max heap. 4
How to delete? If we work with a max heap, and then the element to be deleted is at the root. After this deletion, we will try to fill up the hole left by the one just deleted with the last element. If this element cannot stay there, because of a violation of the heap condition, we will do some adjustment. The following shows such a deletion. Obviously, both deletion and insertion only work with a single path, whose length is bounded by the height of the heap. Hence, the complexity of both operations is Θ(log n). 5
The MaxHeap class The following declares the class for max heap. The declaration of a min heap is similar. class MaxHeap { public: MaxHeap(int MaxHeapSize = 10); ~MaxHeap() { delete [] heap; } int Size() const { return CurrentSize; } T Max() { if (CurrentSize == 0) throw OutOfBounds(); return heap[1]; } MaxHeap<T>& Insert(const T& x); MaxHeap<T>& DeleteMax(T& x); void Initialize(T a[],int size, int ArraySize); private: int CurrentSize, MaxSize; T *heap; // element array }; 6
Define operations The following defines the insert operation, the others are pretty similar. MaxHeap<T>& MaxHeap<T>::Insert(const T& x){ if (CurrentSize == MaxSize) throw NoMem(); // no space // find place for x // i starts at new leaf and moves up tree int i = ++CurrentSize; while (i!= 1 && x > heap[i/2]) { // cannot put x in heap[i] heap[i] = heap[i/2]; // move element down i /= 2; // move to parent } heap[i] = x; return *this; } 7
Homework 9.1. Add ChangeMax(x) to the MaxHeap class that changes the value of the root to x. 9.2. Add ChangePriority(i, x), that changes the priority of node i to x. 8
How to initialize? A simple way is to insert n elements, one by one, into an initially empty heap. The associated time complexity will be n i=1 log i = log(n!) = Θ(n log n). A different strategy will lead to an Θ(n) complexity. Notice that if we bring in the n elements and put them into an array, topologically, we already have a complete balanced tree. The only thing we need to do to have a heap is to adjust the relative ordering. Since there are no need to adjust those with labeling i, such that i> n 2, we can begin with the node i (= n 2 ) and going back to the root. 9
An example 10
Analysis To adjust an element, we have to do 2 comparisons: 1) get the bigger child; 2) compare the element with the bigger child. In the worst case, an adjustment will be involved with readjustments of its descendants. Thus, the total number of comparisons involved for that element could be as much as the sum of the heights of all the elements in the path. Hence, the time complexity of the initialization process is bounded by the sum of the heights of all the nodes. Recall that a full binary tree is a complete binary tree with its bottom level is also completely filled. 11
Theorem: For a full binary tree of height h, containing 2 h+1 1 nodes, the sum of height of all nodes is 2 h+1 (h + 2). Proof: As there are one node at level 0, or height h; 2 nodes at level 1, or height h 1;...,2 h nodes at level h, or height 0; the height sum is the follows: S = 0 i=h i 2 h i = 2 h+1 (h +2). Thus, for a full balanced tree, S = 2 h+1 h 2 = (n +1) log(n +1) 1 = n log(n + 1) = Θ(n). For a complete balanced binary tree, the sum of its heights can not be more than that of the corresponding full balanced tree with the same height. Therefore, the total number of comparisons for this initialization is O(n). 12
Heapsort It is easy to see how to use the PriorityQueue ADT to design another sorting algorithm. All we have to do is to organize all the elements to be sorted into a priority queue, and then repeatedly delete the maximum item from the heap, and put it at the back of the array, since those spaces are gradually faded out. The Heapsort algorithm is divided into two phases: construct a heap, in O(n), and produce the sorted list by deleting in n i=1 log i = log(n!) = Θ(n log n). Although both bin sort and radix sort are in O(n), they come with some restrictions, on the range of the values. In contrast, Heapsort is a general purpose sorting mechanism. 13
An example 14
Machine scheduling Suppose that we have n jobs to be processed in a machine shop with m identical machines. Each job i needs t i time to process. A schedule is an assignment of those jobs to machines such that 1) No machine processes more than one job at one time. 2) No job is processed in more than one machine at the same time. 3) Each job i is assigned for a total of t i units of time. Each machine starts at time 0, and the finish time, or the length, of a schedule is the time at which all jobs are completed. We will only consider non preemptive schedules in which once a machine is assigned to a job, it will not be released until the job is finished. 15
An example The following shows a three-machine schedule for seven jobs with processing time (2, 14, 4, 16, 6, 5, 3). The finish time is 17. Our task is to write a program that will construct a machine schedule that will end up with the minimum finish time. It is pretty hard, no body is able to write a program whose time complexity is polynomial, i.e., in O(n k m l ). It is an example of the (in)famous class of NP-hard problems. 16
NP-complete problems Those problem classes contain problems that no one has developed a polynomial-time algorithms. The problems in NP-complete class are decision problems, i.e., those for which the answer is either yes or no. Although the machine scheduling problem is not a decision problem, we can easily convert it into one, by giving a time T Min, and ask if there is a schedule with finish time TMin or less. This related problem is NP-complete. Many practical problems are NP-hard or NPcomplete. If any of them turns out to be polynomial-time solvable, all the NP-complete problems can be solved in polynomial time. We have yet to prove that NP-complete problems can t be solved in polynomial time, although common wisdom thinks so. So, the best we can do is to apply an approximation algorithm to find a approximated solution for such a problem. 17
The LPT strategy In our case, we can generate schedules whose lengths are at most of 4 3 3m 1 of optimal by applying a simple strategy, the longest processing time first strategy. According to this method, jobs are assigned to machines in descending order of their processing time, t i. When a job is being assigned to a machine, it is assigned to a machine that is idle first. Ties are broken arbitrarily. For the previous example, the ordering of the job indices, in the descending order of t i s, is (4, 2, 5, 6, 3, 7, 1). At t =0, job 4 is assigned to any of the three machines, job 2 and 5 are assigned to any of the remaining two machines, say in M 2 and M 3. When M 3 becomes available at t =6, job 6 is put on, etc.. This discussion leads to the previous schedule. 18
As we always assign the job with the largest t i, it is clear that a heap will be an ideal data structure. When n m, it is trivial. Otherwise, we use Heapsort to sort the jobs into an ascending order. To determine which machine will become available, we set up an min heap for the machines, and initialize it by assigning a job to each machine. We then use DeleteMin to delete the machine that becomes available first and assign the next job to it, if there is anything left. We will increase the available time for the machine and put it back. 19
Implementation The following declares the data structure for job nodes and machines nodes. class JobNode { friend void LPT(JobNode *, int, int); friend void main(void); public: operator int () const {return time;} private: int ID, // job identifier time; // processing time }; class MachineNode { friend void LPT(JobNode *, int, int); public: operator int () const {return avail;} private: int ID, // machine identifier avail; // when it becomes free }; 20
The LPT function void LPT(T a[], int n, int m){ if (n <= m) { cout << "Schedule one job per machine:\n"; return;} HeapSort(a,n); MinHeap<MachineNode> H(m); MachineNode x; for (int i = 1; i <= m; i++) { x.avail = 0; x.id = i; H.Insert(x); } for (i = n; i >= 1; i--) { H.DeleteMin(x); // get first free machine cout << "Schedule job " << a[i].id << " on machine " << x.id << " from " << x.avail << " to " << (x.avail + a[i].time) << endl; x.avail += a[i].time; // new avail time H.Insert(x); } } 21
An example void main(void){ JobNode a[11]; int n = 10; } for (int i = 1; i <= n; i++) { a[i].time = 2 * i * i; a[i].id = i; } LPT(a,n,3); 22
How good is it? Theorem: Let F (I) be the finish time of an optimal m machine schedule for a job set I and let F (I) be the finish time of the LPT schedule for the same job set. Then F (I) F (I) 4 3 1 3m. Regarding to its time complexity, when n m, it takes just Θ(1). Otherwise, it spends Θ(n log n) on Heapsort, O(m) on building up the machine heap. In the for loop, it has to do n insertion and deletion, each of which takes log m, in a total of n log m. Since, n>m,the time complexity of LPT is n log n. 23
Homework 9.3. Compare the worst-case run times of heap sort and insertion sort. For heap sort,use some number of random permutations to estimate the worst-case run time. At what value of n does the run time of heap sort become less than that of insertion sort? 9.4. A sort method is said to be stable if the relative order of records with equal key stays the same after the sort as it was before. Is heap sort a stable sort? How about insertion sort? 24
A coding mechanism In ASCII code, every character is coded in 8 bits. So, if we have a text file with 1,000 characters, we have to use 8,000 bits to store it. In reality, some characters are used more often than the others (Think about Wheel of Fortune). It makes sense to assign shorter codes to those used more often, and longer codes to those used less often. The question is how? One approach is to assign code to symbols based on their frequencies. For example, in the string aaxuaxz, the frequency of a, x, u and z are 3, 2, 1 and 1. When frequency varies a great deal, it makes sense to assign shortest code to the most frequently occurring one, while assigns longest code to the least frequently occurring symbol. 25
For the above example, we can assign the codes as follows: 0=a, 10=x, 110=u, 111=z). Hence, aaxuaxz will be coded as 0010110010111. The length is 13 bits, compared with 14, if we give each of them two bits. On the other hand, if the frequency of the four symbols are (996, 2, 1, 1), then the 2 bits per code method, for a 1,000 character file will lead to 2,000 bits long, while our code will lead to a file of only 1,006 bits. How could we decode? For 0010110010111, since we have no code 00, the first piece of code must be 0, which is an a, the next is also a. As we do have a code 10, we read off an x, etc.. So, we always read off the longest possible piece from the undecoded string. The reason that this method works is because this is a prefix code, i.e., no code is a prefix of another. Such a code is called Huffman code. 26
The construction To encode a string using Huffman code, we need to do the following: 1. Determine the character set, together with their frequencies. 2. Construct a Huffman tree, in which all the leaves are labeled with the original characters, with the respective frequency as their weight. More specifically, beginning with the root, we assign a 0 to every left branch, and assign a 1 to every right branch. 3. Traverse the root to all its leaves to obtain the code. 4. Replace the symbols with their codes. The following slide gives an example. 27
Construct an Huffman tree When we construct the tree, we always want to add a node with the smallest weight, an minheap is an obvious choice. 28