lgorithms B DGWICK KVIN WYN.4 IIY QUU lgorithms F U H D I I N I and elementary implementations binary heaps heapsort event-driven simulation (see videos) B DGWICK KVIN WYN https://algs4.cs.princeton.edu ast updated on /6/9 6:9
.4 IIY QUU lgorithms I and elementary implementations binary heaps heapsort event-driven simulation (see videos) B DGWICK KVIN WYN https://algs4.cs.princeton.edu
Collections collection is a data type that stores a group of items. data type core operations data structure stack UH, linked list, resizing array queue NQUU, DQUU linked list, resizing array priority queue IN, D- binary heap symbol table U, G, D binary search tree, hash table set DD, CNIN, D binary search tree, hash table how me your code and conceal your data structures, and I shall continue to be mystified. how me your data structures, and I won t usually need your code; it ll be obvious. Fred Brooks
riority queue Collections allow adding and removing items. Which item to remove? tack. emove the item most recently added. Queue. emove the item least recently added. andomized queue. emove a random item. riority queue. emove the largest (or smallest) item. Generalizes: stack, queue, randomized queue. operation argument insert insert insert remove max insert insert insert remove max insert insert insert remove max return value size Q Q 4 5 4 5 6 7 6 sequenc 4
riority queue I equirement. Keys are generic; they must also be Comparable. Key must be Comparable ( bounded type parameter ) public class axq<key extends Comparable<Key>> axq() create an empty priority queue axq(key[] a) create a priority queue with given keys void insert(key v) insert a key into the priority queue Key delax() return and remove a largest key boolean ismpty() is the priority queue empty? Key max() return a largest key int size() number of entries in the priority queue Note. Duplicate keys allowed; delax() picks any maximum key. 5
riority queue: applications vent-driven simulation. [ customers in a line, colliding particles ] Numerical computation. [ reducing roundoff error ] Discrete optimization. [ bin packing, scheduling ] rtificial intelligence. [ * search ] Computer networks. [ web cache ] Data compression. [ Huffman codes ] perating systems. [ load balancing, interrupt handling ] Graph searching. [ Dijkstra s algorithm, rim s algorithm ] Number theory. [ sum of powers ] pam filtering. [ Bayesian spam filter ] tatistics. [ online median in data stream ] 6
riority queue: elementary implementation xercise. In the worst case, what are the running times for IN and D- for a priority queue implemented with an unordered array? an ordered array? operation argument return value size contents (unordered) contents (ordered) insert insert insert remove max insert insert insert remove max insert insert insert remove max Q Q Q Q 4 5 4 5 6 7 6 sequence of operations on a priority queue Q Q 7
riority queues: quiz In the worst case, what are the running times for IN and D- for a priority queue implemented with an ordered array?. and n ignore array resizing B. and log n C. log n and D. n and 0 8 9 0 8
riority queue: implementations cost summary Challenge. Implement all operations efficiently. implementation IN D- unordered array n n ordered array n goal log n log n log n order of growth of running time for priority queue with n items what might this mean? olution. omewhat-ordered array. 9
.4 IIY QUU lgorithms I and elementary implementations binary heaps heapsort event-driven simulation B DGWICK KVIN WYN https://algs4.cs.princeton.edu
Complete binary tree Binary tree. mpty or node with links to left and right binary trees. Complete tree. very level (except possibly the last) is completely filled; the last level is filled from left to right. complete binary tree with n = 6 nodes (height = 4) roperty. Height of complete binary tree with n nodes is lg n. ecursive definition
complete binary tree in nature
Binary heap: representation Binary heap. rray representation of a heap-ordered complete binary tree. Heap-ordered binary tree. Keys in nodes. arent s key no smaller than children s keys. i 0 8 9 0 a[i] - N I H G rray representation. Indices start at. ake nodes in level order. No explicit links needed! N 4 5 N 6 7 I H G 8 9 I 0 H G Heap representations
riority queues: quiz Which is the index of the parent of the item at index k in a binary heap?. k/ - B. k/ C. k/ + D. *k N I 8 9 I 0 H G i 0 8 9 0 a[i] - N I H G 4
Binary heap: properties roposition. argest key is a[], which is root of binary tree. roposition. Can use array indices to move through tree. arent of node at k is at k/. Children of node at k are at k and k+. i 0 8 9 0 a[i] - N I H G N I H G 4 5 N 6 7 8 9 I 0 H G Heap representations 5
Binary heap demo Insert. dd node at end, then swim it up. emove the maximum. xchange root with node at end, then sink it down. heap ordered N H I G N H I G 6
Binary heap: swim / promotion cenario. key becomes larger than its parent s key. o eliminate the violation: xchange key in child with key in parent. epeat until heap order restored. private void swim(int k) { N 5 while (k > && less(k/, k)) { I H G violates heap order (larger key than parent) } } exch(k, k/); k = k/; parent of node at k is at k/ N 5 I H G eter principle. Node promoted to level of incompetence. 7
Binary heap: insertion Insert. dd node at end in bottom level; then, swim it up. Cost. t most + lg n compares. insert N H public void insert(key x) I G key to insert { pq[++n] = x; swim(n); } N H I G add key to heap violates heap order swim up N I G H 8
Binary heap: sink / demotion cenario. key becomes smaller than one (or both) of its children s. o eliminate the violation: xchange key in parent with key in larger child. epeat until heap order restored. why not smaller child? private void sink(int k) { while (*k <= n) children of node at k violates heap order (smaller than a child) H { are at *k and *k+ 5 int j = *k; I N G } if (j < n && less(j, j+)) j++; if (!less(k, j)) break; exch(k, j); k = j; I 0 5 H N G } op-down reheapify (sink) ower struggle. Better subordinate promoted. 9
Binary heap: delete the maximum Delete max. xchange root with node at end; then, sink it down. Cost. t most lg n compares. remove the maximum key to remove public Key delax() { N I G H exchange key with root Key max = pq[]; exch(, n--); sink(); H violates heap order } pq[n+] = null; return max; prevent loitering N I G remove node from heap sink down N H I G 0
Binary heap: Java implementation public class axq<key extends Comparable<Key>> { private Key[] pq; private int n; public axq(int capacity) { pq = (Key[]) new Comparable[capacity+]; } fixed capacity (for simplicity) public boolean ismpty() { return n == 0; } public void insert(key key) // see previous code public Key delax() // see previous code Q ops private void swim(int k) private void sink(int k) // see previous code // see previous code heap helper functions } private boolean less(int i, int j) { return pq[i].compareo(pq[j]) < 0; } private void exch(int i, int j) { Key t = pq[i]; pq[i] = pq[j]; pq[j] = t; } https://algs4.cs.princeton.edu/4pq/axq.java.html array helper functions
riority queue: implementations cost summary implementation IN D- unordered array n n ordered array n binary heap log n log n order of growth of running time for priority queue with n items
Binary heap: considerations Underflow and overflow. Underflow: throw exception if deleting from empty Q. verflow: add no-arg constructor and use resizing array. inimum-oriented priority queue. eplace less() with greater(). Implement greater(). leads to log n amortized time per op (how to make worst case?) ther operations. emove an arbitrary item. Change the priority of an item. can implement efficiently with sink() and swim() [ stay tuned for rim/dijkstra ] Immutability of keys. ssumption: client does not change keys while they re on the Q. Best practice: use immutable keys.
Immutability: implementing in Java Data type. et of values and operations on those values. Immutable data type. Can t change the data type value once created. public final class Vector { private final int n; private final double[] data; } public Vector(double[] data) { this.n = data.length; this.data = new double[n]; for (int i = 0; i < n; i++) this.data[i] = data[i]; } instance methods don t change instance variables Immutable in Java. tring, Integer, Double, Color, File, utable in Java. tringbuilder, tack, U, arrays, instance variables private and final (neither necessary nor sufficient, but good programming practice) defensive copy of mutable instance variables 4
Immutability: properties Data type. et of values and operations on those values. Immutable data type. Can t change the data type value once created. dvantages. implifies debugging. implifies concurrent programming. ore secure in presence of hostile code. afe to use as key in priority queue or symbol table. Disadvantage. ust create new object for each data-type value. 5
6 Binary heap: practical improvement ultiway heaps. Complete d-way tree. arent s key no smaller than its children s keys. Fact. Height of complete d-way tree on n nodes is ~ log d n. -way heap Y Z K I G D B J F H V C W Q N
riority queues: quiz In the worst case, how many compares to IN and D- in a d-way heap?. ~ log d n and ~ log d n B. ~ log d n and ~ d log d n C. ~ d log d n and ~ log d n D. ~ d log d n and ~ d log d n 7
riority queue: implementation cost summary implementation IN D- unordered array n n ordered array n binary heap log n log n d-ary heap log d n d log d n sweet spot: d = 4 Fibonacci log n Brodal queue log n impossible why impossible? amortized order-of-growth of running time for priority queue with n items 8
Impossibility of priority queue with constant-time IN & D- xercise. ssume there is a priority queue which makes a constant number of compares in the worst case for both IN and D-. Design a sorting algorithm that uses this priority queue. How many compares does it perform in the worst case? 9
.4 IIY QUU lgorithms I and elementary implementations binary heaps heapsort event-driven simulation B DGWICK KVIN WYN https://algs4.cs.princeton.edu
riority queues: quiz 4 What are the properties of this sorting algorithm? public void sort(tring[] a) { int n = a.length; axq<tring> pq = new axq<tring>(); for (int i = 0; i < n; i++) pq.insert(a[i]); for (int i = n-; i >= 0; i--) a[i] = pq.delax(); }. n log n compares in the worst case. B. In-place. C. table. D. ll of the above.
Heapsort Basic plan for in-place sort. View input array as a complete binary tree. Heap construction: build a max-heap with all n keys. ortdown: repeatedly remove the maximum key. keys in arbitrary order build max heap (in place) sorted result (in place) 8 9 0 8 9 0 8 9 0 8 9 0 8 9 0
Heapsort demo Heap construction. Build max heap using bottom-up method. for now, assume array entries are indexed to n array in arbitrary order 4 5 6 7 8 9 0 8 9 0
Heapsort demo ortdown. epeatedly delete the largest remaining item. array in sorted order 8 9 0 4
5 Heapsort: heap construction First pass. Build heap using bottom-up method. for (int k = n/; k >= ; k--) sink(a, k, n); 8 9 0 starting point (arbitrary order) sink(, ) sink(5, ) 8 9 0 heap construction starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) 8 9 0 heap construction starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 4 8 9 0 result (heap-ordered) resul sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 4 8 9 0 result (heap-ordered) resul Key insight. fter sink(a, k, n) completes, the subtree rooted at k is a heap.
6 Heapsort: sortdown econd pass. emove the maximum, one at a time. eave in array, instead of nulling out. while (n > ) { exch(a,, n--); sink(a,, n); } starting point (heap-ordered) exch(, ) sink(, 0) exch(, 0) sink(, 9) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) sink(5, ) sink(4, ) sink(, ) sink(, ) sink(, ) exch(, 6) sink(, 5) exch(, 5) sink(, 4) exch(, 4) sink(, ) exch(, ) sink(, ) exch(, ) sink(, ) sortdown exch(, ) sink(, 0) exch(, 0) sink(, 9) exch(, 9) sink(, 8) exch(, 8) sink(, 7) exch(, 7) sink(, 6) Heapsort: constructing (left) and sorting down (right) a heap 8 9 0 8 9 0 heap construction result (heap-ordered) result (sorted) starting point (heap-ordered) starting point (arbitrary order) Key insight. fter each iteration, the array consists of a heap-ordered subarray followed by a sub-array in final order
Heapsort: Java implementation public class Heap { public static void sort(comparable[] a) { int n = a.length; for (int k = n/; k >= ; k--) sink(a, k, n); while (n > ) { exch(a,, n); sink(a,, --n); } } private static void sink(comparable[] a, int k, int n) { /* as before */ } but make static (and pass arguments) } private static boolean less(comparable[] a, int i, int j) { /* as before */ } private static void exch(bject[] a, int i, int j) { /* as before */ } but convert from -based https://algs4.cs.princeton.edu/4pq/heap.java.html indexing to 0-base indexing 7
Heapsort: trace a[i] N k 0 8 9 0 initial values 5 4 heap-ordered 0 9 8 7 6 5 4 sorted result Heapsort trace (array contents just after each sink) 8
Heapsort: mathematical analysis roposition. Heap construction makes n exchanges and n compares. f sketch. [assume n = h+ ] max number of exchanges to sink node 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 binary heap of height h = a tricky sum (see C 40) h + (h ) + 4(h ) + 8(h ) +... + h (0) = h+ h = n (h ) n 9
Heapsort: mathematical analysis roposition. Heap construction makes n exchanges and n compares. roposition. Heapsort uses n lg n compares and exchanges. algorithm can be improved to ~ n lg n (but no such variant is known to be practical) ignificance. In-place sorting algorithm with n log n worst-case. ergesort: no, linear extra space. Quicksort: no, quadratic time in worst case. Heapsort: yes! in-place merge possible, not practical n log n worst-case quicksort possible, not practical Bottom line. Heapsort is optimal for both time and space, but: Inner loop longer than quicksort s. akes poor use of cache. Not stable. can be improved using advanced caching tricks 40
orting algorithms: summary inplace? stable? best average worst remarks selection ½ n ½ n ½ n n exchanges insertion n ¼ n ½ n use for small n or partially ordered merge ½ n lg n n lg n n lg n n log n guarantee; stable quick n lg n n ln n ½ n n log n probabilistic guarantee; fastest in practice -way quick n n ln n ½ n improves quicksort when duplicate keys heap n n lg n n lg n n log n guarantee; in-place? n n lg n n lg n holy sorting grail 4