Prorty queues and eaps Professors Clark F. Olson and Carol Zander Prorty queues A common abstract data type (ADT) n computer scence s te prorty queue. As you mgt expect from te name, eac tem n te prorty queue as a prorty tat determnes te order n wc te tems leave te queue (not frst n frst out, as n te queues we ave seen prevously). Tere are many applcatons of prorty queues. Scedulng prnt jobs and system processes are two smple examples. Ts s also te abstract data type tat we need to mplement Huffman codng. Te prmary operatons tat are necessary for a prorty queue are: nsert (wt prorty) fndmn (or fndmax) deletemn (or deletemax) Of course, we want to be able to mplement tese tree operatons effcently n terms of n (te number of tems n te queue). Ts could be mplemented n many ways, ncludng several tat we ave already seen. Data structure nsert deletemn Unsorted collecton O(1) O(n) Sorted array O(n) O(1) Sorted lnked lst O(n) O(1) Bnary searc tree (balanced) O(log n) O(log n) Only te balanced bnary searc tree s effcent (n te worst case) unless te number of nsertons s very small compared to te number of deletemn operatons. Tere are two types of data structure tat can guarantee O(log n) tme for nsert and deletemn n te worst case. One s te balanced bnary searc tree; te oter (smpler) data structure s a eap. (Note tat ts s not related to te usage of eap to descrbe free memory.) Heaps Bnary eaps are complete bnary trees tat satsfy te eap order property, wc states tat eac node (except te root) must ave a key (or prorty) tat s no less tan te key of ts parent. (Ts s a mneap. A maxeap would be te reverse.) Complete bnary tree means tat all levels are completely full except possbly te last level, wc s flled from left to rgt. Wt ts defnton, eac subtree wtn te eap s also a eap and te mnmum key must be at te root of te tree. Here s an example eap: 17 8 19 6 1 1 7 41 15 Note tat tere s no restrcton on te orderng of te cldren of a node; no tree traversal wll output te tems n sorted order. We can also ave smaller keys at lower levels of te tree tan larger keys,.e., tere s no relatonsp between left and rgt subtrees oter tan te parent s smaller. Page 1 of 7
Usually, tese are mplemented wt an array, snce tey are always complete trees. 0 1 4 5 6 7 8 9 10 11 1 14 15 16 17-17 8 19 6 1 1 7 41 15 Note tat te frst node s often ntalzed to a value known to be smaller tan all values n te eap n order to smplfy te nserton operaton. Te array may be larger tan te eap to allow nserton or because tems ave been deleted from te eap. Wen mplemented, a eap keeps track of bot ow many tems are n te eap and ow muc space as been allocated to te array. Even better s to use a vector, wc keeps track of te allocaton for us, so te declaraton mgt look lke ts. class Heap { publc: // Constructor, copy constructor, destructor, as usual vod nsert(comparable *c); const Comparable * fndmn() const; bool deletemn(); prvate: nt sze; vector<comparable *> tems; ; We can perform a fndmn operaton n O(1) tme on a mneap, snce te mnmum value s always at te root (*tems[1]). Heap nserton To nsert n a eap, you place te value n te next avalable spot and ten percolate t upwards untl t s no longer smaller tan ts parent s. Let s nsert a 5 nto our eap above. Wen we start te structure looks lke ts: 17 8 19 6 1 1 7 41 15 5 Often te tem s not actually n te end array poston yet. Ts way as we percolate te value up, t s one assgnment (one operaton) nstead of one swap (tree operatons). Page of 7
Ts sn t a eap, snce te 5 s smaller tan ts parent. We swap t wt te to fx ts. Snce t stll sn t smaller tan ts parent s, we ten swap t wt te 8 to get te followng eap: 17 5 19 6 8 1 1 7 41 15 Try nsertng,, and 19 (n tat order) to get a new eap. In practce, te nserton works on te array lke ts: vod Heap<Comparable>::nsert(Comparable *c) { // add tem n poston 0 (dummy poston) to prevent percolatng up from root tems[0] = c; // Ensure we ave enoug space sze++; f (tems.sze() == sze) tems.pus_back(null); // Percolate up nt poston = sze; wle (*tems[0] < *tems[poston / ]) { tems[poston] = tems[poston / ]; poston = poston / ; tems[poston] = tems[0]; Te code above uses a couple of optmzatons. One s not to nsert te tem untl te end. In ts case, we just move te necessary elements untl we fnd te fnal restng spot for te new tem. Also, te value at poston zero s ntalzed to te new tem n ts case. Ts s also a useful tecnque wen usng templates, snce we can t create a dummy object for ndex 0 tat we know as a key less tan every object tat wll be placed n te eap. (Note tat vector transparently ncreases te sze of te nternal dynamc array by more tan one at a tme.) Ts operaton s O(log n) n te worst case, but O(1) n te average case (assumng tat te numbers are nserted n random order). Te average number of levels tat a new tem needs to be percolated s only 1.6. Arguably, ts s O(n) n te worst case, f we need to expand te sze of te vector or array, but clever trcks can be used to nsure tat we do no more tan O(log n), f necessary. Page of 7
Te deletemn operaton Wen we want to do a deletemn, we perform a smlar operaton n reverse. So tat tere are no oles n te array, we move te last element to te root and ten percolate t down. Here s te state wen we move te last element to te root: 17 5 19 6 8 1 1 7 41 15 Now we need to reeapfy te tree by percolatng te down. We always replace t wt te smaller of te two cldren (unless tey are bot larger tan t s) and, ten, repeat te process, yeldng: 5 17 8 19 6 1 1 7 41 15 Te code looks lke: bool Heap<Comparable>::deleteMn() { f (sze == 0) return false; delete tems[1]; tems[1] = tems[sze]; sze--; percolatedown(1); return true; vod Heap<Comparable>::percolateDown(nt poston) { nt cld = poston * ; f (cld > sze) return; f (cld!= sze && *tems[cld] > *tems[cld + 1]) cld++; f (*tems[cld] < *tems[poston]) { swap(tems[cld], tems[poston]); percolatedown(cld); Page 4 of 7
Ts operaton requres O(log n) tme n te average case and n te worst case, snce te last tem s usually relatvely large and often needs to be percolated down almost to te bottom. Lnear tme eap buldng We can buld a eap n lnear expected tme by usng nsert repeatedly, but ts as O(n log n) worst case effcency. It turns out tat we can buld a eap n lnear tme n te worst case f we ave all of te elements wen we start. Te basc dea s stragtforward. We call percolatedown for eac non leaf node startng wt te last node and endng wt te root. // te routne assumes tat te objects and sze ave already // been loaded nto te eap data vod Heap<Comparable>::eapfy() { for (nt = sze / ; > 0; --) percolatedown(); How do we know tat sze/ s te rgt place to start? We can t ave more tan sze/ non leaf nodes. Eac non leaf node as two cldren (wt one possble excepton te last non leaf node). If tere are m non leaves, ten tere are eter m 1 or m cldren. Snce m 1 of tese cldren are oter non leaves (one non leaf s te root), tat leaves m or m+1 leaf nodes and tere must be at least as many leaves as tere are non leaves. Does ts result n a eap wen we re done? Let s try an example: 8 4 46 44 41 1 19 10 45 8 Te 41 percolates down one spot, swappng wt te 8. Te 44 swaps wt te 10. Te 46 swaps wt te 19. Essentally we turn te bottom level of trees nto eaps. Ts yelds: Page 5 of 7
8 4 19 10 8 1 46 44 45 41 At te next level, te 4 swaps wt te. Te swaps wt te 10. Now tese subtrees are eaps. Fnally, at te root, te 8 swaps wt te 10, and ten te 19, and ten te 1, yeldng: 10 19 1 8 4 8 46 44 45 41 Two questons: Do we know tat ts always results n a eap? Does ts really take lnear tme n te worst case? For te frst queston: f we start wt two eaps, jon tem togeter wt any root and, ten, percolate te root down, we end up wt a eap. Inducton usng ts dea proves tat we get a eap n te end wt eapfy. For te second queston: wat s te worst tat can appen? At every non leaf node, we could ave to percolate t down to a leaf. Ts means tat te maxmum number of swaps s te same as te sum of te egts of eac of te nodes n te tree. Provng te followng teorem, proves eapfy s O(N) were N = (+1) 1. N s te number of nodes n a complete bnary tree. (Assume =0 for a tree wt just a root.) Teorem: For a complete bnary tree of egt contanng (+1) 1 nodes, te sum of te egts s (+1) 1 (+1). (Note tat snce N = (+1) 1, (+1) 1 (+1) s O(N).) Page 6 of 7
Proof: Sum te egt of te tree. Here we use te defnton tat says an empty tree as egt = 1; a tree wt just a root s egt zero. For example, wen s, te sum of te egts s: 1() + () + 4(1) + 8(0) Now generalze and wrte as a formula: 1() + () + 4(1) + 8(0) = 0 (-0) + 1 (-1) + (-) + (-) = 0 (-0) + 1 (-1) + (-) + (-) // \ / \ / \ // \ / \ // \ // \ / \ / \ // \ /\ /\ /\ // \ /\ // \ /\ //\\//\\ //\\//\\ 1 branc, brances, 4 brances, of lengt of lengt of lengt 1 Te double lnes sow te egt count. So te sum, S, s as follows: S = = 0 0 1 ( ) = + ( 1) + ( ) +... + + 1 1+ 0 We can use a neat trck and compute S = S S as follows: S= 1 () + (-1) + (-) +... + (-) () + (-1) () + -S= - 0 () - 1 (-1) - (-) - (-) -... - (-) () - (-1) (1) - 0 ------------------------------------------------------------------------- S= - + 1 + * + * +... + (-) (-) + (-1) + - * 1 - * \ / \ / = =... Subtractng term by term, te terms wt '' drop out and a power of s left. Ten subtract and add one, so our summaton starts at zero: S S = S = + + + +... + = + = 1 = 1+ = 0 Now, usng a well known formula: = 0 = + 1 1 proves te teorem: S = 1+ + 1 1 = O( N) Heap sort A fnal note tat s wort mentonng s tat we can easly sort usng a eap, by buldng te eap and repeatedly deletng te mnmum value. We use te delete operaton, but store te deleted value at te end of te array. Interestng, ts yelds an algortm tat s O(n log n) n te worst case (unlke qucksort/oaresort) and tat doesn t requre extra memory (unlke mergesort). Qucksort s stll used most often n practce. Page 7 of 7