Priority Queue Sorting

Transcription

1 Priority Queue Sorting We can use a priority queue to sort a list of comparable elements 1. Insert the elements one by one with a series of insert operations 2. Remove the elements in sorted order with a series of removemin operations The running time of this sorting method depends on the priority queue implementation Algorithm PQ-Sort(S, C) n Input list S, comparator C for the elements of S n Output list S sorted in increasing order according to C P priority queue with comparator C while S.isEmpty () e S.remove(S.first ()) P.insert (e, ) while P.isEmpty() e P.removeMin().getKey() S.addLast(e) 2014 Goodrich, Tamassia, Goldwasser Heaps 1

2 Selection-Sort Selection-sort is the variation of PQ-sort where the priority queue is implemented with an unsorted sequence Running time of Selection-sort: 1. Inserting the elements into the priority queue with n insert operations takes O(n) time 2. Removing the elements in sorted order from the priority queue with n removemin operations takes time proportional to n + n Selection-sort runs in O(n 2 ) time 2014 Goodrich, Tamassia, Goldwasser Heaps 2

3 Selection-Sort Example Sequence S Priority Queue P Input: (7,4,8,2,5,3,9) () Phase 1 (a) (4,8,2,5,3,9) (7) (b) (8,2,5,3,9) (7,4) (g) () (7,4,8,2,5,3,9) Phase 2 (a) (2) (7,4,8,5,3,9) (b) (2,3) (7,4,8,5,9) (c) (2,3,4) (7,8,5,9) (d) (2,3,4,5) (7,8,9) (e) (2,3,4,5,7) (8,9) (f) (2,3,4,5,7,8) (9) (g) (2,3,4,5,7,8,9) () 2014 Goodrich, Tamassia, Goldwasser Heaps 3

4 Insertion-Sort Insertion-sort is the variation of PQ-sort where the priority queue is implemented with a sorted sequence Running time of Insertion-sort: n Inserting the elements into the priority queue with n insert operations takes time proportional to n n Removing the elements in sorted order from the priority queue with a series of n removemin operations takes O(n) time Insertion-sort runs in O(n 2 ) time 2014 Goodrich, Tamassia, Goldwasser Heaps 4

5 Insertion-Sort Example Sequence S Priority queue P Input: (7,4,8,2,5,3,9) () Phase 1 (a) (4,8,2,5,3,9) (7) (b) (8,2,5,3,9) (4,7) (c) (2,5,3,9) (4,7,8) (d) (5,3,9) (2,4,7,8) (e) (3,9) (2,4,5,7,8) (f) (9) (2,3,4,5,7,8) (g) () (2,3,4,5,7,8,9) Phase 2 (a) (2) (3,4,5,7,8,9) (b) (2,3) (4,5,7,8,9) (g) (2,3,4,5,7,8,9) () 2014 Goodrich, Tamassia, Goldwasser Heaps 5

6 In-place Insertion-Sort Instead of using an external data structure, we can implement selectionsort and insertion-sort inplace A portion of the input sequence itself serves as the priority queue For in-place insertion-sort n We keep sorted the initial portion of the sequence n We can use swaps instead of modifying the sequence Goodrich, Tamassia, Goldwasser Heaps 6

7 Heap Sort Consider a priority queue with n items implemented by means of a heap n n the space used is O(n) methods insert and removemin take O(log n) time Using a heap-based priority queue, we can sort a sequence of n elements in O(n log n) time The resulting algorithm is called heap-sort n methods size, isempty, and min take time O(1) time Heap-sort is much faster than quadratic sorting algorithms, such as insertion-sort and selectionsort 2014 Goodrich, Tamassia, Goldwasser Heaps 7

8 Heap Sort Create a heap Do removemin repeatedly till head becomes empty To do an in place sort, we move deleted element to end of heap Heaps 8

9 Heap Sort Heaps 9

10 Heap Sort Heaps 10

11 Heap Sort Heaps 11

12 Heap Sort Heaps 12

13 Heap Sort Heaps 13

14 Heap Sort Heaps 14

15 Heap Sort Heaps 15

16 Heap Sort Heaps 16

17 Heap Sort Heaps 17

18 Heap Sort Heaps 18

19 Heap Sort Heaps 19

20 Heapsort - Analysis Create a heap - O(n) Do removemin repeatedly till head becomes empty n O(log n) + O(log n-1) + O(log n-2) + O(1) = O(n log n) To do an in place sort, we move deleted element to end of heap Heaps 20

21 Maps

22 Maps A map models a searchable collection of key-value entries The main operations of a map are for searching, inserting, and deleting items Multiple entries with the same key are not allowed Applications: n address book n student-record database 2014 Goodrich, Tamassia, Goldwasser Maps 22

23 The Map ADT get(k): if the map M has an entry with key k, return its associated value; else, return null put(k, v): insert entry (k, v) into the map M; if key k is not already in M, then return null; else, return old value associated with k remove(k): if the map M has an entry with key k, remove it from M and return its associated value; else, return null size(), isempty() entryset(): return an iterable collection of the entries in M keyset(): return an iterable collection of the keys in M values(): return an iterator of the values in M 2014 Goodrich, Tamassia, Goldwasser Maps 23

24 Example Operation Output Map isempty() true Ø put(5,a) null (5,A) put(7,b) null (5,A),(7,B) put(2,c) null (5,A),(7,B),(2,C) put(8,d) null (5,A),(7,B),(2,C),(8,D) put(2,e) C (5,A),(7,B),(2,E),(8,D) get(7) B (5,A),(7,B),(2,E),(8,D) get(4) null (5,A),(7,B),(2,E),(8,D) get(2) E (5,A),(7,B),(2,E),(8,D) size() 4 (5,A),(7,B),(2,E),(8,D) remove(5) A (7,B),(2,E),(8,D) remove(2) E (7,B),(8,D) get(2) null (7,B),(8,D) isempty() false (7,B),(8,D) 2014 Goodrich, Tamassia, Goldwasser Maps 24

25 A Simple List-Based Map We can implement a map using an unsorted list n We store the items of the map in a list S (based on a doubly linked list), in arbitrary order header nodes/positions trailer 9 c 6 c 5 c 8 c entries 2014 Goodrich, Tamassia, Goldwasser Maps 25

26 The get(k) Algorithm Algorithm get(k): B = S.positions() {B is an iterator of the positions in S} while B.hasNext() do p = B.next() { the next position in B } if p.element().getkey() = k then return p.element().getvalue() return null {there is no entry with key equal to k} 2014 Goodrich, Tamassia, Goldwasser Maps 26

27 The put(k,v) Algorithm Algorithm put(k,v): B = S.positions() while B.hasNext() do p = B.next() if p.element().getkey() = k then t = p.element().getvalue() S.set(p,(k,v)) return t {return the old value} S.addLast((k,v)) n = n + 1 {increment variable storing number of entries} return null { there was no entry with key equal to k } 2014 Goodrich, Tamassia, Goldwasser Maps 27

28 The remove(k) Algorithm Algorithm remove(k): B =S.positions() while B.hasNext() do p = B.next() if p.element().getkey() = k then t = p.element().getvalue() S.remove(p) n = n 1 {decrement number of entries} return t {return the removed value} return null {there is no entry with key equal to k} 2014 Goodrich, Tamassia, Goldwasser Maps 28

29 Performance of a List-Based Map Performance: n n n put takes O(1) time since we can insert the new item at the beginning or at the end of the sequence get and remove take O(n) time since in the worst case (the item is not found) we traverse the entire sequence to look for an item with the given key Average case time? The unsorted list implementation is effective only for maps of small size or for maps in which puts are the most common operations, while searches and removals are rarely performed (e.g., historical record of logins to a workstation) Maps 29

30 Different Data Structures to Implement Map ADT arrays, linked lists (inefficient) Binary Trees Hash Tables Red/Black Trees AVL Trees B-Trees Java n java.util.dictionary - abstract class n java.util.map - interface Maps 30

31 Binary Search 2014 Goodrich, Tamassia, Goldwasser Maps 31

32 Direct Addressing an array that is indexed by the key n O(1) - time n O(r) space - where r is the range of numbers n wastage of space A B C D XXXX-XXXX XXXX-XXXX XXXX-XXXX Maps 32

33 Hash Tables

34 Intuitive Notion of a Map Intuitively, a map M supports the abstraction of using keys as indices with a syntax such as M[k]. As a mental warm-up, consider a restricted setting in which a map with n items uses keys that are known to be integers in a range from 0 to N 1, for some N n Goodrich, Tamassia, Goldwasser Hash Tables 34

35 Hash Table Solution O(1) - expected time O(n+m) - space, where m is size of the table instead of a one-to-one map between the key values and array locations, find a function to map the large range into one which we can manage n e.g., key value modulo size of array, and use that as an index n Insert ( , C) into a hashed array of size 5, mod 5 = 3 XXXX-XXXX XXXX-XXXX XXXX-XXXX XXXX-XXXX C Hash Tables 35

36 More General Kinds of Keys But what should we do if our keys are not integers in the range from 0 to N 1? n Use a hash function to map general keys to corresponding indices in a table. n For instance, the last four digits of a Identification number Goodrich, Tamassia, Goldwasser Hash Tables 36

37 Hash Functions and Hash Tables A hash function h maps keys of a given type to integers in a fixed interval [0, N - 1] Example: h(x) = x mod N is a hash function for integer keys The integer h(x) is called the hash value of key x A hash table for a given key type consists of n Hash function h n Array (called table) of size N When implementing a map with a hash table, the goal is to store item (k, o) at index i = h(k) 2014 Goodrich, Tamassia, Goldwasser Hash Tables 37

38 Example (1,D) (25,C) (3,F) (14,Z) (6,A) (39,C) (7,Q) Collision Two different keys resulting in the same hash value Hash Tables 38

39 Hash Functions Need to choose a good hash function n quick to compute n uniform distribution of keys throughout the table n good hash functions are very rare. How to deal with hashing non-integer keys n find some way to turn keys into integers n use standard hash functions on these integers Hash Tables 39

40 Hash Functions (2) A hash function is usually specified as the composition of two functions: Hash code: h 1 : keys integers Compression function: h 2 : integers [0, N - 1] The hash code is applied first, and the compression function is applied next on the result, i.e., h(x) = h 2 (h 1 (x)) The goal of the hash function is to disperse the keys in an apparently random way 2014 Goodrich, Tamassia, Goldwasser Hash Tables 40

41 Hash Codes Integer cast: n n n We reinterpret the bits of the key as an integer Suitable for keys of length less than or equal to the number of bits of the integer type (e.g., byte, short, int and float in Java) For keys of length greater than the number of bits of the integer type, ignore the exceeding bits Component sum: n n n We partition the bits of the key into components of fixed length (e.g., 16 or 32 bits) and we sum the components (ignoring overflows) Suitable for numeric keys of fixed length greater than or equal to the number of bits of the integer type (e.g., long and double in Java) Not a good choice for strings Hash Tables 41

42 Hash Codes (2) Polynomial accumulation: n n n We partition the bits of the key into a sequence of components of fixed length (e.g., 8, 16 or 32 bits) a 0 a 1 a n-1 We evaluate the polynomial p(z) = a 0 + a 1 z + a 2 z2 + + a n-1 z n-1 at a fixed value z, ignoring overflows Especially suitable for strings (e.g., the choice z = 33 gives at most 6 collisions on a set of 50,000 English words) Polynomial p(z) can be evaluated in O(n) time using Horner s rule: n The following polynomials are successively computed, each from the previous one in O(1) time p 0 (z) = a n-1 p i (z) = a n-i-1 + zp i-1 (z) (i = 1, 2,, n -1) We have p(z) = p n-1 (z) 2014 Goodrich, Tamassia, Goldwasser Hash Tables 42

43 Compression Functions Division: n Use the remainder n h 2 (y) = y mod N w y is the key w N is size of the table n how to choose N? Consider (200, 205,210,215,220, 600) N = 100 N = 101 Hash Tables 43

44 Compression Functions (2) Division: n Use the remainder n h 2 (y) = y mod N w y is the key w N is size of the table n how to choose N? n N = b x (bad) Consider (200,205,210,215,220, 600) N = 100 N = 101 w N is a power of 2, h 2 (y) gives the x least significant bits of y. w all keys with the same ending go to the same place n N is prime (good) w helps ensure uniform distribution Hash Tables 44

45 Compression Functions (3) Multiply, Add and Divide (MAD): n h 2 (y) = [(ay + b) mod p ]mod N n a and b are nonnegative integers such that a mod N 0 n Otherwise, every integer would map to the same value b n p is a prime number larger than N n a and b are chosen at random from the interval [0, p-1], with a > 0 Hash Tables 45

46 Compression Functions (4) For any choice of hash function, there always exists a bad set of keys You could choose only these keys, resulting in all of them getting mapped to the same slot. n reduction in performance. Solution n collection of hash functions n a random hash function n choose a hash function that is independent of the keys Hash Tables 46

47 Collision Handling Collisions occur when different elements are mapped to the same cell Separate Chaining: let each cell in the table point to a linked list of entries that map there n complexity depends on load factor Separate chaining is simple, but requires additional memory outside the table Goodrich, Tamassia, Goldwasser Hash Tables 47

48 Map with Separate Chaining Delegate operations to a list-based map at each cell: Algorithm get(k): return A[h(k)].get(k) Algorithm put(k,v): t = A[h(k)].put(k,v) if t = null then n = n + 1 return t Algorithm remove(k): t = A[h(k)].remove(k) if t null then n = n - 1 return t {k is a new key} {k was found} 2014 Goodrich, Tamassia, Goldwasser Hash Tables 48

49 Linear Probing Open addressing: the colliding item is placed in a different cell of the table n load factor is at most 1 Linear probing: handles collisions by placing the colliding item in the next (circularly) available table cell Each table cell inspected is referred to as a probe Colliding items lump together, causing future collisions to cause a longer sequence of probes Hash Tables 49

50 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 50

51 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 51

52 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 52

53 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 53

54 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 54

55 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 55

56 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 56

57 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 57

58 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order w h(x) = (x+1) mod Goodrich, Tamassia, Goldwasser Hash Tables 58

59 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order w h(x) = (x+1) mod Goodrich, Tamassia, Goldwasser Hash Tables 59

60 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 60

61 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order Goodrich, Tamassia, Goldwasser Hash Tables 61

62 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order w h(x) = (x+1) mod Goodrich, Tamassia, Goldwasser Hash Tables 62

63 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order w h(x) = (x+1) mod 13 w h(x) = (x+2) mod Goodrich, Tamassia, Goldwasser Hash Tables 63

64 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order w h(x) = (x+1) mod 13 w h(x) = (x+2) mod Goodrich, Tamassia, Goldwasser Hash Tables 64

65 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order w h(x) = (x+1) mod 13 w h(x) = (x+2) mod Goodrich, Tamassia, Goldwasser Hash Tables 65

66 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order w h(x) = (x+1) mod 13 w h(x) = (x+2) mod Goodrich, Tamassia, Goldwasser Hash Tables 66

67 Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order w h(x) = (x+1) mod 13 w h(x) = (x+2) mod Goodrich, Tamassia, Goldwasser Hash Tables 67

68 Search with Linear Probing Consider a hash table A that uses linear probing get(k) n n We start at cell h(k) We probe consecutive locations until one of the following occurs w An item with key k is found, or w An empty cell is found, or w N cells have been unsuccessfully probed Algorithm get(k) i h(k) p 0 repeat c A[i] if c = return null else if c.getkey () = k return c.getvalue() else i (i + 1) mod N p p + 1 until p = N return null 2014 Goodrich, Tamassia, Goldwasser Hash Tables 68

69 Search with Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order n Search for Hash Tables 69

70 Updates with Linear Probing To handle insertions and deletions, we introduce a special object, called DEFUNCT, which replaces deleted elements remove(k) n We search for an entry with key k n n If such an entry (k, o) is found, we replace it with the special item DEFUNCT and we return element o Else, we return null Hash Tables 70

71 Update with Linear Probing (2) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order n Remove 57 n Search for Hash Tables 71

72 Update with Linear Probing (3) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order n Remove 57 n Search for X Hash Tables 72

73 Updates with Linear Probing (4) To handle insertions and deletions, we introduce a special object, called DEFUNCT, which replaces deleted elements put(k, o) n We throw an exception if the table is full n We start at cell h(k) n We probe consecutive cells until one of the following occurs w A cell i is found that is either empty or stores DEFUNCT, or w N cells have been unsuccessfully probed n We store (k, o) in cell i 2014 Goodrich, Tamassia, Goldwasser Hash Tables 73

74 Update with Linear Probing (5) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order n Remove 57 n Search for 32 n Insert X Hash Tables 74

75 Update with Linear Probing (5) Example: n h(x) = x mod 13 n Insert keys 18, 41, 22, 44, 57, 32, 31, 73, in this order n Remove 57 n Search for 32 n Insert Hash Tables 75

76 Double Hashing Double hashing uses a secondary hash function d(k) and handles collisions by placing an item in the first available cell of the series (i + jd(k)) mod N for j = 0, 1,, N - 1 The secondary hash function d(k) cannot have zero values The table size N must be a prime to allow probing of all the cells 2014 Goodrich, Tamassia, Goldwasser Hash Tables 76

77 Example of Double Hashing Consider a hash table storing integer keys that handles collision with double hashing n N = 13 n h(k) = k mod 13 n d(k) = 7 - k mod 7 Insert keys 18, 41, 22, 44, 59, 32, 31, 73, in this order k h (k ) d (k ) Probes Goodrich, Tamassia, Goldwasser Hash Tables 77

78 Performance of Hashing In the worst case, searches, insertions and removals on a hash table take O(n) time The worst case occurs when all the keys inserted into the map collide The load factor α = n/n affects the performance of a hash table Assuming that the hash values are like random numbers, it can be shown that the expected number of probes for an insertion with open addressing is 1 / (1 - α) The expected running time of all the dictionary ADT operations in a hash table is O(1) In practice, hashing is very fast provided the load factor is not close to 100% Applications of hash tables: n n n small databases compilers browser caches 2014 Goodrich, Tamassia, Goldwasser Hash Tables 78