SE-286: Data Structures and Programming g Priority Queue Virendra Singh Indian Institute of Science Bangalore Lecture 18 Courtesy: Prof. Sartaj Sahni Oct 4, 2010 1
Priority Queues Two kinds of priority queues: Min priority queue. Max priority queue. Oct Nov 4, 2010 2009 2
Min Priority Queue Collection of elements. Each element has a priority or key. Supports following operations: isempty size add/put an element into the priority yqueue get element with min priority remove element with min priority Oct Nov 4, 2010 2009 3
Max Priority Queue Collection of elements. Each element has a priority or key. Supports following operations: isempty size add/put an element into the priority yqueue get element with max priority remove element with max priority Oct Nov 4, 2010 2009 4
Complexity Of Operations Two good implementations are heaps and leftist trees. isempty, size, and get =>O(1)time put and remove => O(log n) time where n is the size of the priority queue Oct Nov 4, 2010 2009 5
Sorting Applications use element key as priority put elements to be sorted into a priority queue extract elements in priority order if a min priority queue is used, elements are extracted in ascending order of priority (or key) if a max priority i queue is used, elements are extracted in descending order of priority (or key) Oct Nov 4, 2010 2009 6
Sorting Example Sort five elements whose keys are 6, 8, 2, 4, 1 using a max priority queue. Put the five elements into a max priority yqueue. Do five remove max operations placing removed elements into the sorted array from right to left. Oct Nov 4, 2010 2009 7
After Putting Into Max Priority yqueue 8 1 2 4 6 Max Priority Queue Sorted Array Oct Nov 4, 2010 2009 8
After First Remove Max Operation 1 2 4 6 Max Priority Queue 8 Sorted Array Oct Nov 4, 2010 2009 9
After Second Remove Max Operation 1 2 4 Max Priority Queue 6 8 Sorted Array Nov Oct 4, 2010 2009 10
After Third Remove Max Operation 1 2 Max Priority Queue 4 6 8 Sorted Array Oct Nov 4, 2010 2009 11
After Fourth Remove Max Operation 1 Max Priority Queue 2 4 6 8 Sorted Array Nov Oct 4, 2010 2009 12
After Fifth Remove Max Operation Max Priority Queue 1 2 4 6 8 Sorted Array Nov Oct 4, 2010 2009 13
Complexity Of Sorting Sort n elements. n put operations => O(n log n) time. n remove max operations => O(n log n) time. total time is O(n log n). compare with O(n 2 ) for sort methods Nov Oct 4, 2010 2009 14
Heap Sort Uses a max priority queue that is implemented as a heap. Iiil Initial put operations are replaced dby a heap initialization step that takes O(n) time. Nov Oct 4, 2010 2009 15
Machine Scheduling m identical machines (drill press, cutter, sander, etc.) n jobs/tasks to be performed assign jobs to machines so that t the time at which h the last job completes is minimum Nov Oct 4, 2010 2009 16
Machine Scheduling Example 3 machines and 7 jobs job times are [6, 2, 3, 5, 10, 7, 14] possible schedule A 6 13 B C 2 7 3 13 21 Nov Oct 4, 2010 2009 17 time ----------->
Machine Scheduling Example A B 6 13 2 7 21 C 3 time -----------> 13 Finish time = 21 Objective: Find schedules with minimum i finish i time. Nov Oct 4, 2010 2009 18
LPT Schedules Longest Processing Time first. Jobs are scheduled in the order 14, 10, 7, 6, 5, 3, 2 Each job is scheduled on the machine on which it finishes earliest. Nov Oct 4, 2010 2009 19
LPT Schedule [14, 10, 7, 6, 5, 3, 2] 14 16 A B C 10 15 7 13 16 Finish time is 16! Nov Oct 4, 2010 2009 20
LPT Schedule LPT rule does not guarantee minimum i finish i time schedules. Usually LPT finish time is much closer to minimum finish time. Minimum finish time scheduling is NP-hard. Nov Oct 4, 2010 2009 21
Complexity Of LPT Scheduling Sort jobs into decreasing order of task time. O(n log n) time (n is number of jobs) Schedule jobs in this order. assign job to machine that becomes available first must find minimum of m (m is number of machines) finish times takes O(m) time using simple strategy so need O(mn) time to schedule all n jobs. Nov Oct 4, 2010 2009 22
Using A Min Priority Queue e Min priority queue has the finish times of the m machines. Initial finish times are all 0. To schedule a job remove machine with minimum finish time from the priority queue. Update the finish time of the selected machine and put the machine back into the priority queue. Nov Oct 4, 2010 2009 23
Using A Min Priority Queue e m put operations to initialize priority queue 1 remove min and 1 put tto schedule each hjob each put and remove min operation takes O(log m) time time to schedule is O(n log m) overall time is O(nlogn+nlogm)=O(nlog(mn)) n n (mn)) Nov Oct 4, 2010 2009 24
Huffman Codes Useful in lossless compression. Nov Oct 4, 2010 2009 25
Min Tree Definition Each tree node has a value. Value in any node is the minimum value in the subtree for which that node is the root. Equivalently, no descendent has a smaller value. Nov Oct 4, 2010 2009 26
Min Tree Example 2 4 9 3 4 8 7 9 9 Root has minimum i element. Nov Oct 4, 2010 2009 27
Max Tree Example 9 4 9 8 4 2 7 3 1 Root has maximum element. Nov Oct 4, 2010 2009 28
Min Heap Definition complete binary tree min tree Nov Oct 4, 2010 2009 29
Min Heap With 9 Nodes Complete binary tree with 9 nodes. Nov Oct 4, 2010 2009 30
Min Heap With 9 Nodes 2 4 3 6 7 9 3 8 6 Complete binary tree with 9 nodes that is also a min tree. Nov Oct 4, 2010 2009 31
Max Heap With 9 Nodes 9 8 7 6 7 2 6 5 1 Complete binary tree with 9 nodes that is also a max tree. Nov Oct 4, 2010 2009 32
Heap Height Since a heap is a complete binary tree, the height of an n node heap is log 2 (n+1). Nov Oct 4, 2010 2009 33
A Heap Is Efficiently Represented As An Array 9 8 7 6 7 2 6 5 1 0 9 8 7 6 7 2 6 5 1 1 2 3 4 5 6 7 8 9 10 Nov Oct 4, 2010 2009 34
Moving Up And Down A Heap 1 9 2 3 8 7 4 5 6 7 6 7 2 6 5 1 8 9 Nov Oct 4, 2010 2009 35
Putting An Element Into A Max Heap 9 8 7 6 7 2 6 5 1 7 Complete binary tree with 10 nodes. Nov Oct 4, 2010 2009 36
Putting An Element Into A Max Heap 9 8 7 6 7 2 6 5 1 75 New element is 5. Nov Oct 4, 2010 2009 37
Putting An Element Into A Max Heap 9 8 7 6 2 6 7 5 1 7 New element is 20. Nov Oct 4, 2010 2009 38
Putting An Element Into A Max Heap 9 8 7 6 2 6 5 1 7 New element is 20. Nov Oct 4, 2010 2009 39
Putting An Element Into A Max Heap 9 7 6 8 2 6 5 1 7 New element is 20. Nov Oct 4, 2010 2009 40
Putting An Element Into A Max Heap 20 9 7 6 8 2 6 5 1 7 New element is 20. Nov Oct 4, 2010 2009 41
Putting An Element Into A Max Heap 20 9 7 6 8 2 6 5 1 7 Complete binary tree with 11 nodes. Nov Oct 4, 2010 2009 42
Putting An Element Into A Max Heap 20 9 7 6 8 2 6 5 1 7 New element is 15. Nov Oct 4, 2010 2009 43
Putting An Element Into A Max Heap 20 9 7 6 2 6 5 1 7 8 New element is 15. Nov Oct 4, 2010 2009 44
Putting An Element Into A Max Heap 20 15 7 6 9 2 6 5 1 7 8 New element is 15. Nov Oct 4, 2010 2009 45
Complexity Of Put 20 15 7 6 9 2 6 5 1 7 8 Complexity is O(log n), where n is heap size. Nov Oct 4, 2010 2009 46
Removing The Max Element 20 15 7 6 9 2 6 5 1 7 8 Max element is in the root. Nov Oct 4, 2010 2009 47
Removing The Max Element 15 7 6 9 2 6 5 1 7 8 After max element is removed. Nov Oct 4, 2010 2009 48
Removing The Max Element 15 7 6 9 2 6 5 1 7 8 Heap with 10 nodes. Reinsert 8 into the heap. Nov Oct 4, 2010 2009 49
Removing The Max Element 15 7 6 9 2 6 5 1 7 Reinsert 8 into the heap. Nov Oct 4, 2010 2009 50
Removing The Max Element 15 7 6 9 2 6 5 1 7 Reinsert 8 into the heap. Nov Oct 4, 2010 2009 51
Removing The Max Element 15 9 7 6 2 6 8 5 1 7 Reinsert 8 into the heap. Nov Oct 4, 2010 2009 52
Removing The Max Element 15 9 7 6 2 6 8 5 1 7 Max element is 15. Nov Oct 4, 2010 2009 53
Removing The Max Element 9 7 6 2 6 8 5 1 7 After max element is removed. Nov Oct 4, 2010 2009 54
Removing The Max Element 9 7 6 2 6 8 5 1 7 Heap with 9 nodes. Nov Oct 4, 2010 2009 55
Removing The Max Element 9 7 6 2 6 8 5 1 Reinsert 7. Nov Oct 4, 2010 2009 56
Removing The Max Element 9 7 6 2 6 8 5 1 Reinsert 7. Nov Oct 4, 2010 2009 57
Removing The Max Element 9 8 7 6 7 2 6 5 1 Reinsert 7. Nov Oct 4, 2010 2009 58
Complexity Of Remove Max Element 9 8 7 6 7 2 6 5 1 Complexity is O(log n). Nov Oct 4, 2010 2009 59
Initializing A Max Heap 1 2 3 4 5 6 7 8 9 710 11 8 input array = [-, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] Nov Oct 4, 2010 2009 60
Initializing A Max Heap 1 2 3 4 5 6 7 8 9 710 11 8 Start at rightmost array position that has a child. Index is n/2. Nov Oct 4, 2010 2009 61
Initializing A Max Heap 1 2 3 4 11 6 7 8 9 710 85 Move to next lower array position. Nov Oct 4, 2010 2009 62
Initializing A Max Heap 1 2 3 4 11 6 7 8 9 710 85 Nov Oct 4, 2010 2009 63
Initializing A Max Heap 1 2 3 9 11 6 7 8 4 710 85 Nov Oct 4, 2010 2009 64
Initializing A Max Heap 1 2 3 9 11 6 7 8 4 710 85 Nov Oct 4, 2010 2009 65
Initializing A Max Heap 1 2 7 9 11 6 3 8 4 710 85 Nov Oct 4, 2010 2009 66
Initializing A Max Heap 1 2 7 9 11 6 3 8 4 710 85 Nov Oct 4, 2010 2009 67
Initializing A Max Heap 1 11 7 9 6 3 8 4 710 85 Find a home for 2. Nov Oct 4, 2010 2009 68
Initializing A Max Heap 1 11 7 9 10 6 3 8 4 8 75 Find a home for 2. Nov Oct 4, 2010 2009 69
Initializing A Max Heap 1 11 7 9 10 6 3 8 4 72 85 Done, move to next lower array position. Nov Oct 4, 2010 2009 70
Initializing A Max Heap 1 11 7 9 10 6 3 8 4 72 85 Find home for 1. Nov Oct 4, 2010 2009 71
Initializing A Max Heap 11 7 9 10 6 3 8 4 72 85 Find home for 1. Nov Oct 4, 2010 2009 72
Initializing A Max Heap 11 10 7 9 6 3 8 4 72 85 Find home for 1. Nov Oct 4, 2010 2009 73
Initializing A Max Heap 11 10 7 9 5 6 3 8 4 72 8 Find home for 1. Nov Oct 4, 2010 2009 74
Initializing A Max Heap 11 10 7 9 5 6 3 8 4 72 81 Done. Nov Oct 4, 2010 2009 75
Time Complexity 11 9 7 8 5 6 3 1 4 710 82 Height of heap = h. Number of subtrees with root at level j is <= 2 j-1. Time Oct 4, for 2010 each subtree is O(h-j+1). Nov 4, 2009 76
Complexity Time for level j subtrees is <= 2 j-1 (h-j+1) = t(j). Total time is t(1) + t(2) + + t(h-1) = O(n). Nov Oct 4, 2010 2009 77
Leftist Trees Linked binary tree. Can do everything a heap can do and in the same asymptotic complexity. Can meld two leftist tree priority queues in O(log n) time. Nov Oct 4, 2010 2009 78
Extended Binary Trees Start with any binary tree and add an external node wherever there is an empty subtree. Result is an extended binary tree. Nov Oct 4, 2010 2009 79
ABinaryTree Nov Oct 4, 2010 2009 80
An Extended Binary Tree number of external nodes is n+1 Nov Oct 4, 2010 2009 81
The Function s() For any node x in an extended binary tree, let s(x) be the length of a shortest path from x to an external node in the subtree rooted at x. Nov Oct 4, 2010 2009 82
s() Values Example Nov Oct 4, 2010 2009 83
s() Values Example 2 2 1 2 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 Nov Oct 4, 2010 2009 84
Properties Of s() If x is an external node, then s(x) = 0. Otherwise, s(x) = min {s(leftchild(x)), s(rightchild(x))} + 1 Nov Oct 4, 2010 2009 85
HihtBi Height Biased dlftitt Leftist Trees A binary tree is a (height biased) leftist tree iff for every internal node x, s(leftchild(x)) >= s(rightchild(x)) Nov Oct 4, 2010 2009 86
A Leftist Tree 2 2 1 2 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 Nov Oct 4, 2010 2009 87
LftitT Leftist Trees--Property 1 In a leftist tree, the rightmost path is a shortest root to external node path and the length of this path is s(root). Nov Oct 4, 2010 2009 88
A Leftist Tree 2 2 1 2 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 Length of rightmost path is 2. Nov Oct 4, 2010 2009 89
LftitT Leftist Trees Property 2 The number of internal nodes is at least 2 s(root) - 1 Because levels 1 through s(root) have no external nodes. So, s(root) <= log(n+1) Nov Oct 4, 2010 2009 90
A Leftist Tree 2 2 1 2 1 1 0 1 1 0 0 1 0 0 0 0 0 0 0 Levels 1 and 2 have no external nodes. Nov Oct 4, 2010 2009 91
LftitT Leftist Trees Property 3 Length of rightmost path is O(log n), where n is the number of nodes in a leftist tree. Follows from Properties 1 and 2. Nov Oct 4, 2010 2009 92
Leftist Trees As Priority Queues Min leftist tree leftist tree that is a min tree. Used as a min priority queue. Max leftist tree leftist tree that is a max tree. Used as a max priority queue. Nov Oct 4, 2010 2009 93
A Min Leftist Tree 2 4 3 6 8 5 8 6 9 Nov Oct 4, 2010 2009 94
Some Min Leftist Tree Operations put() remove() meld() initialize() put() and remove() use meld(). Nov Oct 4, 2010 2009 95
Put Operation put(7) 2 4 3 6 8 5 8 6 9 Nov Oct 4, 2010 2009 96
Put Operation put(7) 2 4 3 6 8 5 8 6 9 Create a single node min leftist tree. 7 Nov Oct 4, 2010 2009 97
Put Operation put(7) 2 4 3 6 8 5 8 6 9 Create a single node min leftist tree. 7 Meld the two min leftist trees. Nov Oct 4, 2010 2009 98
Remove Min 2 4 3 6 8 5 8 6 9 Nov Oct 4, 2010 2009 99
Remove Min 2 4 3 6 8 5 8 6 9 Remove the root. Nov Oct 4, 2010 2009 100
Remove Min 2 4 3 6 8 5 8 6 9 Remove the root. Meld the two subtrees. Nov Oct 4, 2010 2009 101
Meld Two Min Leftist Trees 4 3 6 8 5 6 8 6 9 Traverse only the rightmost paths so as to get logarithmic performance. Nov Oct 4, 2010 2009 102
Meld Two Min Leftist Trees 4 3 6 8 5 6 8 6 9 Meld right subtree of tree with smaller root and all of other tree. Nov Oct 4, 2010 2009 103
Meld Two Min Leftist Trees 4 3 6 8 5 6 8 6 9 Meld right subtree of tree with smaller root and all of other tree. Nov Oct 4, 2010 2009 104
Meld Two Min Leftist Trees 4 6 6 8 8 6 Meld right subtree of tree with smaller root and all of other tree. Nov Oct 4, 2010 2009 105
Meld Two Min Leftist Trees 8 6 Meld right subtree of tree with smaller root and all of other tree. Right subtree of 6 is empty. So, result of melding right subtree of tree with smaller root and other tree is the Oct 4, 2010 other tree. Nov 4, 2009 106
Meld Two Min Leftist Trees 8 6 Make melded subtree right subtree of smaller root. 6 8 Swap left and right subtree if s(left) < s(right). 6 Nov Oct 4, 2010 2009 107 8
Meld Two Min Leftist Trees 4 6 4 6 8 6 8 6 6 8 6 8 Make melded subtree right subtree of smaller root. Swap left and right subtree if s(left) < s(right). Nov Oct 4, 2010 2009 108
Meld Two Min Leftist Trees 3 5 4 9 6 6 8 6 8 Make melded subtree right subtree of smaller root. Swap Oct 4, left 2010 and right subtree if s(left) < s(right). Nov 4, 2009 109
Meld Two Min Leftist Trees 3 4 5 6 6 9 8 6 8 Nov Oct 4, 2010 2009 110
Initializing In O(n) Time create n single node min leftist trees and place them in a FIFO queue repeatedly remove two min leftist trees from the FIFO queue, meld them, and put the resulting min leftist tree into the FIFO queue the process terminates when only 1 min leftist tree remains in the FIFO queue analysis is the same as for heap initialization Oct Nov 4, 2010 2009 111