Desgn and Analyss of Algorthms Heaps and Heapsort Reference: CLRS Chapter 6 Topcs: Heaps Heapsort Prorty queue Huo Hongwe
Recap and overvew The story so far... Inserton sort runnng tme of Θ(n 2 ); sorts n place. Merge sort runnng tme of Θ(n lg n); needs auxlary storage Θ(n). Next... Heapsort runnng tme of Θ(n lg n); sort n place. Qucksort runnng tme of Θ(n lg n) on average; most practcal (and hence wdely- used) sortng algorthm. Sortng n lnear tme Huo Hongwe 2
Heaps Data structure ndexed by ntegers, 2,..., n. Each element supports the operaton PARENT, LEFT, RIGHT. Easly mplemented usng an array wth PARENT() = /2, LEFT() = 2, and RIGHT() = 2 +. max heap A[PARENT()] A[]. mn heap A[PARENT()] A[]. The root of the heap s A[]. The heght of a node s the longest downward path from the node to a leaf. The heght of the heap s the heght of the root. Huo Hongwe
Heaps Vewed as a bnary tree, t s completely flled on all levels except possbly the last. 6 2 4 0 8 7 2 8 0 6 4 0 8 7 2 4 8 0 2 4 Huo Hongwe 4
Heaps Exercse. The heght of heap wth n elements s lg n. Operatons supported by a heap: MAX-HEAPIFY ensures that a heap s max heap. O(log n) BUILD-MAX-HEAP produces a max heap from an unordered array. Θ(n) To sort an array, we can frst convert t nto a max heap, repeatedly extract the root (the largest element by defnton) and MAX-HEAPIFY the rest. Θ(n log n) Huo Hongwe 5
2 4 4 7 8 6 2 MAX-HEAPIFY 0 8 0 2 4 8 7 4 6 2 0 8 0 (a) (c) 2 4 4 7 8 6 2 0 8 0 (b) (a) The ntal wth A[2] at node =2 volatng the max-heap property (b) by exchange A[2] wth A[4], whch destroys the max-heap property for node 4. Recursve call MAX- HEAPIFY(A, 4) has = 4., (c)max-heapify(a, ) yelds no further change to the data structure. Huo Hongwe 6
MAX-HEAPIFY heap-sze[a] keeps track of the sze of the heap stored n the array A. MAX-HEAPIFY MAX-HEAPIFY(A,) l LEFT() 2 r RIGHT() f l < heap-sze[a] and A[l]> A[] 4 then largest l 5 else largest 6 f r < heap-sze[a] and A[r]> A[largest] 7 then largest r 8 f largest then exchange A[] A[largest] 0 MAX-HEAPIFY(A,largest) Runnng tme: O(h), where h s the heght of element A[]. Huo Hongwe 7
BUILD-MAX-HEAP BUILD-MAX-HEAP BUILD-MAX-HEAP(A) heap-sze[a] length[a] 2 for length[a]/2 downto do MAX-HEAPIFY(A,) Loop nvarant. At the start of the for loop, the nodes +, +2,...,n each roots of a max- heap. Intalzaton. Before the frst teraton, = n/2. Snce the nodes n/2 +,...,n are leaves, they are each roots of a max- heap. Mantenance. The chldren of node are numbered hgher than so by the loop-nvarant, they are both roots of a max-heap. MAX- HEAPIFY(A, ) wll then make the tree rooted at node a max-heap. Termnaton. = 0. Huo Hongwe 8
Acton of BUILD-MAX-HEAP 4 2 6 0 4 8 7 4 8 2 6 7 4 2 8 0 4 0 4 6 0 2 8 7 2 8 0 4 8 4 2 6 0 7 2 8 0 4 4 6 2 8 7 2 0 8 0 Huo Hongwe
Acton of BUILD-MAX-HEAP 2 6 8 0 4 4 7 2 8 0 2 4 8 0 6 8 7 2 4 0 Runnng tme. The naïve analyss leads to O(n lg n). Key observaton. An n-element heap has at most n/2 h+ nodes of heght h (Exercse) and MAX-HEAPIFT requre O(h) tme to run on a node of heght h. The runnng tme s thus bounded by. lg n h= 0 n h O( h) O = O( n) h + h h= 2 2 0 Huo Hongwe 0
HEAPSORT HEAPSORT HEAPSORT(A) BUILD-MAX-HEAP(A) 2 for length[a] downto 2 do exchange A[] A[] 4 heap-sze[a] heap-sze[a]- 5 MAX-HEAPIFY(A,) Runnng tme. O(n lg n). The call to BUILD-MAX-HEAP takes tme O(n) and there are n- calls to MAX-HEAPIFY, each takng O(lg n) tme. In-place. Not Stable. Note that even f BUILD-MAX-HEAP ran n O(n lg n) tme, we d get the same runnng tme. Huo Hongwe
An Example for HEAPSORT 6 4 0 4 0 8 0 8 8 7 4 7 4 7 2 4 2 6 2 4 6 (a) (b) (c) 8 7 8 7 4 4 7 2 4 2 2 8 0 4 6 0 4 6 0 4 6 (d) (e) (f) Huo Hongwe 2
An Example for HEAPSORT 2 4 2 2 7 8 4 7 8 4 7 8 0 4 6 0 4 6 0 4 6 (g) (h) () 2 4 7 8 A 2 4 7 8 0 4 6 0 4 6 (j) (k) the resultng sorted array A Huo Hongwe
Prorty Queues Heapsort s an excellent algorthm, but a good mplementaton of qucksort usually beats t n practce. Nevertheless, the heap data structure tself has enormous utlty. Prorty Queues: a data structure for mantanng a set S of elements, each wth an assocated value called a key. There are two knds of prorty queues: max-prorty queues and mn-prorty queues. We ll focus on here how to mplement max-prorty queues, whch based on maxheaps. Huo Hongwe 4
Prorty Queues Applcatons Max-prorty queues applcatons. Schedule jobs on a shared computer» Keepng track of the jobs and ther relatve prortes Heapsort» mplementng a prorty queue Mn-prorty queues applcatons. Djkstra's shortest path algorthm. Prm's MST algorthm. Event-drven smulaton.» prortzng events to be processed based on ther predcted tme of occurrence Huffman encodng.... Huo Hongwe 5
Prorty Queues Supports the followng basc operatons. INSERT(S, x)» nserts the element x nto S MAXIMUM(S)» returns the element of S wth the largest key EXTRACT-MAX(S)» removes and returns the element of S wth the largest key INCREASE-KEY(S, x, k)» Increase the value of key of element x to the new value k. Huo Hongwe 6
Max Bnary Heaps 6 2 4 0 8 7 2 8 0 6 4 0 8 7 2 4 8 0 2 4 Operatons supported. MAX-HEAPIFY ensures that a heap s max heap. O(log n) BUILD-MAX-HEAP produces a max heap from an unordered array. Θ(n) We wll use these operatons to smulate a prorty queue. Huo Hongwe 7
Prorty queue operatons usng bnary heaps We have constant tme access to the maxmum. MAXIMUM MAXMUM(A) return A[] Extractng the max element takes O(log n) tme. HEAP-EXTRACT-MAX HEAP-EXTRACT-MAX(A)// A[..m] has n < m elements stored. f heap-sze[a]< 2 then error heap underflow max A[] 4 A[] A[heap-sze[A]] 5 heap-sze[a] heap-sze[a] 6 MAX-HEAPIFY(A, ) 7 return max Huo Hongwe 8
INCREASE-KEY HEAP-INCREASE-KEY HEAP-INCREASE-KEY(A,,key) f key < A[] 2 then error new key s small than current key A[] key 4 whle > and A[PARENT()] < A[] 5 do exchange A[] A[PARENT()] 6 PARENT() The runnng tme proportonal to the depth (dstance from root) of the node, whch s O(log n). Huo Hongwe
HEAP-INCREASE-MAX 6 6 4 0 4 0 8 7 8 7 2 4 2 5 4 6 0 5 6 0 5 7 4 7 2 8 2 8 Huo Hongwe 20
HEAP-INSERT MAX-HEAP-INSERT MAX-HEAP-INSERT(A, key) heap-sze[a] heap-sze[a] + 2 A[heap-sze[A]] - HEAP-INCREASE-KEY(A,heap-sze[A],key) Runs n Θ(log n) tme. All operatons on an n-element heap take O(log n). Huo Hongwe 2
Bnary Heap: Inserton Insert element x nto heap. Insert nto next avalable slot. Bubble up untl t's heap ordered.» Peter prncple: nodes rse to level of ncompetence 2 4 6 8 7 0 8 0 2 4 5 next free slot Huo Hongwe 22
Bnary Heap: Inserton Insert element x nto heap. Insert nto next avalable slot. Bubble up untl t's heap ordered.» Peter prncple: nodes rse to level of ncompetence swap wth parent A[5] 6 2 4 0 8 7 8 0 2 4 5 Huo Hongwe 2
Bnary Heap: Inserton Insert element x nto heap. Insert nto next avalable slot. Bubble up untl t's heap ordered.» Peter prncple: nodes rse to level of ncompetence swap wth parent A[2] 6 2 4 0 8 5 8 0 2 4 7 Huo Hongwe 24
Bnary Heap: Inserton Insert element x nto heap. Insert nto next avalable slot. Bubble up untl t's heap ordered.» Peter prncple: nodes rse to level of ncompetence O(lg n) operatons. Stop: heap ordered 2 8 5 8 0 4 6 2 4 7 0 Huo Hongwe 25
Runnng Tme Runnng tme of HEAP-EXTRACT-MAX s O(lg n). Performs only a constant amount of work on top of Heapfy, whch takes O(lg n) tme Runnng tme of HEAP-INSERT s O(lg n). The path traced from the new leaf to the root has length O(lg n). Huo Hongwe 26
Mergeable (mn) heaps A data structure that supports the followng operatons: MAKE-HEAP: creates an empty heap. INSERT(H, x): nserts the element x nto heap H. MAXIMUM(S): returns (a ponter to) the node n heap H whose key s mnmum EXTRACT-Mn(S): deletes the node whose key s mnmum, returnng (a ponter to) the node. UNION-(H,H 2 ): return a new heap that contans all the nodes of heaps H and H 2, destroyng H and H 2. We ll also see: DECREASE-KEY(H,x,k): decreases the key of node x to k. DELETE(H, x): deletes node x from the heap H. Huo Hongwe 27
Operaton MAKE-HEAP INSERT MINIMUM EXTRACT-MIN UNION DECREASE-KEY DELETE Comparson of runnng tmes Lnked Lst N N N Bnary log N log N N log N log N Bnomal log N log N log N log N log N log N Heaps Fbonacc * log N log N Relaxed IS-EMPTY log N log N Djkstra/Prm MAKE-HEAP V INSERT V EXTRACT-MIN E DECREASE-KEY If we don t need the UNION operaton, then bnary heaps are good enough n the worst-case. Huo Hongwe 28
Is heap heght lg n? Suppose the heght of a heap of n elements s h. 6 4 7 5 8 Level 0: node Level : 2 nodes Level 2: 4 nodes Level : 2 nodes Level h: 2 h nodes Ex. 6.-2 +2+4+ +2 h- + n +2+4+ +2 h- +2 h 2 h n 2 h+ - Huo Hongwe 2