Priority Queues. Reading: 7.1, 7.2

Transcription

1 Priority Queues Reading: 7.1, 7.2

2 Generalized sorting Sometimes we need to sort but The data type is not easily mapped onto data we can compare (numbers, strings, etc) The sorting criteria changes depending on application or even during an application s execution Examples Airplanes landing or taking off Wait lists (passengers, patients in the ER)

3 Keys Key - object/attribute that an application assigns to an element Used to identify, rank, weigh that element Typically assigned by the user or application, not necessarily related to the content of the element In order to identify, rank, weigh elements we need a comparison rule R for keys KEY ELEMENT

4 Entries Entry - association between a key and a value Simplest example of a composition public interface Entry{ public Object key(); public Object value(); } KEY ELEMENT

5 Total Order Relations R must have the following properties: R(k, k)=true for any valid k reflexive property R(k 1, k 2 )=true and R(k 2, k 1 )=true => k 1 =k 2 Anti-symmetric property R(k 1, k 2 )=true & R(k 2, k 3 )=true => R(k 1, k 3 )=true Transitive property R(k 1, k 2 )=true R(k 2, k 1 )=true for any k 1, k 2 Trichotomy law (comparability) - NOT IN THE BOOK

6 Total Order Relations Example: Assume k N, R is 1. k k for every k 2. k 1 k 2 & k 2 k 1 => k 1 =k 2 3. k 1 k 2 && k 2 k 3 => k 1 k 3 4. k 1 k 2 k 2 k 1 for any k 1, k 2

7 Example: Total Order Relations keys w are is alphabetical equal or before 1. w@w for every w 2. w 2 & w 1 => w 1 =w 2 3. w 2 && w 3 => w 3 4. w 2 w 1 for any w 1, w 2

8 Total Order Relations Important property For finite sets, there is a k max and a k min such that R(k,k max )= true for any k R(k min, k)=true for any k

9 Comparator Design Pattern Comparator Encapsulates the action of comparing two objects according to a given total order relation External to the keys being compared Note that any relationship that gives total ordering is acceptable, so by changing comparators we can achieve any sorting desired

10 Comparator ADT Methods of the Comparator ADT, all with Boolean return type islessthan(x, y) islessthanorequalto(x,y) isequalto(x,y) isgreaterthan(x, y) isgreaterthanorequalto(x,y) iscomparable(x)

11 Comparators in Java java.util.comparator Simplified API Provides a comparison function Can be passed to sort methods - e.g. Collections.sort() int compare(object o1, Object o2) boolean equals(object obj)

12 Priority Queues Container of Entries (i.e. key-value pairs) Organization: smallest key - first out Key value is interpreted as priority level Note that smallest is defined by comparator! k=1 e n k=0 e n-1 k=2 e n-2 k=0 e 1 Remove first

13 Priority Queues Keys can be arbitrary objects on which a total order is defined Several items can have same key Remove any of them Remove oldest

14 Priority Queue ADT Collection of (key, element) pairs Main methods insertitem(k,o) Inserts an element o and key k removemin() Removes the item with the smallest key, returns the element minkey() Returns (without removing) the smallest key minelement() Returns (without removing) the element with the smallest key size(), isempty()

15 Priority Queue Example Priority levels on Internet traffic High [0] - streaming video Normal [1] - HTML files Low [2] - traffic

16 Priority Queue - Implementation 1 Implementation using Unsorted Sequence Elements are (k,e) pairs New entry added to the end of the list removemin() requires that all entries be inspected k=1 e n k=0 e n-1 k=2 e n-2 k=0 e 1 insertitem() removemin()

17 Priority Queue - Implementation 1 Running times insertlast() - O(1) removemin() - Θ(n) Fast insertion, slow removal These don t depend on the implementation of the sequence (DLL or array) Actually removal slower for array-based find+remove+shift vs. find+remove

18 Priority Queue - Implementation 2 Implementation using Sorted Sequence Insert requires appropriate location to be found removemin() simple removes first entry removemin() k=4 e 1 k=2 e n-2 k=1 e n k=0 e n-1 insertitem()

19 Priority Queue - Implementation 2 Running times insertlast() - O(n) removemin() - O(1) Slow insertion, fast removal These don t depend on the implementation of the sequence (DLL or array) Actually insert slower for array-based find+shift +insert vs. find+insert

20 Sorting with Priority Queues S k=1 e 1 k=0 e n-2 k=2 e n k=4 e n-1 a=s.remove(first()) Q.insertItem(a.k, a.e) Q k=1 e 1 k=0 e n-2 k=2 e n k=4 e n-1 k=q.minkey(); e=q.removemin() p=(k,e); S.insertLast(p) S k=4 e n-1 k=2 e n k=1 e1 k=0 e n-2

21 Sorting with Priority Queues We can use a priority queue to sort a set of comparable elements 1. Insert the elements one by one with a series of insertitem(k, e) operations 2. Remove the elements in sorted order with a series of removemin() operations The running time of this sorting method depends on the priority queue implementation Algorithm PQ-Sort(S, C) Input sequence S, comparator C for the elements of S Output sequence S sorted in increasing order according to C P priority queue with comparator C while S.isEmpty () e S.remove (S. first ()) P.insertItem(e, e) while P.isEmpty() e P.removeMin() S.insertLast(e)

22 Selection Sort Implementation with unsorted sequence Store the items of the priority queue in a listbased sequence, in arbitrary order Performance: insertitem() takes O(1) time since we can insert the item at the beginning or end of the sequence removemin(), minkey() and minelement() take O(n) time since we have to traverse the entire sequence to find the smallest key

23 Selection Sort Running time of Selection-sort: 1.Inserting the elements into the priority queue with n insertitem() operations takes O(n) time 2.Removing the elements in sorted order from the priority queue with n removemin() operations takes time proportional to n + (n-1) Selection-sort runs in O(n 2 ) time

24 Insertion Sort Implementation with sorted sequence Store the items of the priority queue in a sequence, sorted by key Performance: insertitem() takes O(n) time since we have to find the place where to insert the item removemin(), minkey() and minelement() take O(1) time time since the smallest key is at the beginning of the sequence

25 Insertion Sort Running time of Insertion-sort: 1.Inserting the elements into the priority queue with n insertitem() operations takes time proportional to n 2.Removing the elements in sorted order from the priority queue with a series of n removemin() operations takes O(n) time Insertion-sort runs in O(n 2 ) time

26 In-place Insertion Sort Instead of using an external data structure, we can implement selection-sort and insertion-sort in-place A portion of the input sequence itself serves as the priority queue For in-place insertion-sort We keep sorted the initial portion of the sequence Remove next element Find its destination Shift all above Insert

27 In-place Selection Sort For in-place selection-sort We keep sorted the final portion of the sequence Remove smallest element in the unsorted portion Shift all above Insert

28 Heaps Reading:7.3

29 Heap ADT Heap - binary tree that stores a collection of keys and satisfies two properties Heap-Order: for every node v other than the root key(v) key(parent(v)) Consequences The minimum key is stored at the root The keys encountered on a path from the root to an external node are in non-decreasing order

30 Heap ADT Complete Binary Tree: the heap tree must be a complete binary tree COMPLETE binary tree A binary tree in which every level, except possibly the deepest, is completely filled At the deepest level, all nodes must be as far left as possible last node

31 Complete Binary Tree ADT Adds the following two methods add(o) - add to T a new external node v storing o such that the resulting tree is complete with last node v remove() - remove the last node of T and return its element

32 Complete Binary Tree ADT v v

33 Heap ADT - example last node

34 Height of a Heap Theorem: A heap storing n keys has height O(log n) Proof: (we apply the complete binary tree property) Let h be the height of a heap storing n keys Since there are 2 i keys at depth i = 0,, h 2 and at least one key at depth h 1, we have n h Thus, n 2 h 1, i.e., h log n + 1 depth 0 1 h 2 h 1 keys h 2 1

35 Heaps and Priority Queues We can use a heap to implement a priority queue We store a (key, element) item at each internal node We keep track of the position of the last node For simplicity, we show only the keys in the pictures (2, Sue) (5, Pat) (6, Mark) (9, Jeff) (7, Anna)

36 Note Sometimes heaps use only internal nodes - adding empty externals simplifies some algorithms So you might find a heap represented:

37 Insertion into a Heap Priority Queue ADT method insertitem() corresponds to the insertion of a key k into the heap Two-stage algorithm 1. Add node - normal complete binary tree operation 2. Restore heap-order - up-heap algorithm (upheap bubbling)

38 Up-heap Algorithm (Bubbling) After the insertion of a new key k, the heaporder property may be violated The up-heap algorithm restores the order by swapping k along an upward path from the insertion node Up-heap terminates when k reaches the root or a node whose parent has a key smaller than or equal to k Since heap has height O(log n), up-heap runs in O(log n) time

39 Up-heap Algorithm (Bubbling)

40 Removal from a Heap Priority Queue ADT method removemin() corresponds to the removal of the root key from the heap Two-stage algorithm 1. Replace root key with the key of the last node 2. Restore heap-order - down-heap algorithm (down-heap bubbling)

41 Down-heap Algorithm (Bubbling) After replacing the root key with the key k of the last node, the heap-order property may be violated Algorithm down-heap restores the heap-order property by swapping key k with its smallest child along a downward path from the root Down-heap terminates when key k reaches a leaf or a node whose children have keys greater than or equal to k Since a heap has height O(log n), down-heap runs in O(log n) time

42 Down-heap Algorithm (Bubbling)

43 Heap Sort Priority queue sort using an auxiliary heap Phase 1 Put all elements in S into initially empty PQ n insert operations => O(nlog n) Phase 2 Extract the items using n removemin() O(nlog n) Overall complexity O(nlog n) S S

44 Vector-based Implementation Node at rank i Left child at 2i Right child at 2i+1 Last node at rank n First empty slot n+1 insertitem() = insert at rank n+1 removemin() = remove at rank n

45 Linked Implementation Keep reference to last node 2 It takes O(log n) to find next insertion point Running times: array based slightly faster insertitem()

46 Updating the last node The insertion node can be found by traversing a path of O(log n) nodes If at left child, go to the right child Go up until a left child or the root is reached If a left child is reached, go to the right child Go down left until a leaf is reached Similar algorithm for updating the last node after a removal Note: this algorithm could be simplified with empty external nodes!

47 Merging Two Heaps Given two heaps and a key k 3 2 Create new heap with root node storing k and with two heaps as subtrees Perform down-heap to restore heap-order

48 Heap Construction - Top-Down Using n insertitem() operations On-line approach, keys are added as they become available Overall complexity: O(nlog n)

49 Heap Construction - Bottom-up Recursive procedure based on heap merging Off-line approach - keys must be known in advance To simplify analysis, assume perfect tree n=2 h+1-1 In phase i, pairs of heaps with 2 i 1 keys are merged into heaps with 2 i+1 1 keys 2 i 1 2 i 1 2 i+1 1

50 Heap Construction - Bottom-up 1. Store (n+1)/2 keys in elementary one-node heaps 2. Form (n+1)/4 heaps, each by joining pairs of elementary heaps and adding a new key.. i. Form (n+1)/2 i heaps, each by joining pairs of heaps constructed on the previous step and adding a new key

51 Example Keys:

52 Example (contd.) Keys:

53 Example (contd.) Keys:

54 Example (end) Keys:

55 Analysis Assume n=2 h+1-1 Step 1: (n+1)/2 Step 2: O[(n+1)/4+((n+1)/4)*1] Step i: O[(n+1)/2 i +((n+1)/ 2 i )*(i-1)] Cost of merging+ cost of up-heap (#of subtrees * cost of up-heap on each subtree) RT(n) = h n +1 i = n +1 2 i i=1 ( ) h i = n +1 i=1 2 i ( ) 2 h h 2 n +1 ( ) So RT(n)=O(n)

56 Analysis We visualize the worst-case time of a downheap with a proxy path that goes first right and then repeatedly goes left until the bottom of the heap (this path may differ from the actual downheap path) Since each node is traversed by at most two proxy paths, the total number of nodes of the proxy paths is O(n) Thus, bottom-up heap construction runs in O(n) time Bottom-up heap construction is faster than n successive insertions and speeds up the first phase of heap-sort

57 Exercise: heap union Assume two heaps of sizes m and n. Describe an algorithmfor merging the two into a single heap. What is the running time? 1.Take elements in smaller heap one by one, insert in larger (m<n) RT 1 =log(n)+log(n+1)+ +log(n+m) <mlog(n+m-1)=o(mlog(n+m))

58 Example: heap union (cont) 2. Flatten the two heaps into single unordered list of size n+m, build using bottom-up RT 2 =(n+m) Which solution is better? mlog(n+m) n=100, m<25 n+m>mlog(n+m) n+m n=100