9/5/009 Algorithms Sortig 3- Sortig Sortig Problem The Sortig Problem Istace: A sequece of umbers Objective: A permutatio (reorderig) such that a ' K a' a, K,a a ', K, a' of the iput sequece The umbers are called keys Data Structures ad Algorithms Adrei Bulatov Algorithms Sortig 3-3 Algorithms Sortig 3-4 Isertio Sort The most atural sortig algorithm Iput: array A of legth Output: sorted array A for j= to do set key:=a[j] set i:=j- while i>0 ad A[i]>key do set A[i+]:= A[i], i:=i- set A[i+]:=key edwhile edfor Isert A[j] ito sorted sequece A[ j-] Isertio Sort: Soudess Isertio Sort cosists of a sigle loop We use techique called loop ivariat that is a property P such that (Iitializatio) it is true before the first iteratio of the loop (Maiteace) if it is true before a iteratio of the loop, it remais true before the ext iteratio (Termiatio) whe the loop termiates, the property helps to establish the correctess of the algorithm Ivariat for Isertio Sort: At the start of each iteratio of the for loop, the subarray A[ j ] cosists of the elemets origially i A[ j ] but i sorted order Algorithms Sortig 3-5 Algorithms Sortig 3-6 Isertio Sort: Soudess (Iitializatio) iitially j =, so the subarray of iterest cotais oly oe elemet, A[]. The ivariat is true (Maiteace) Suppose the property is true before iteratio j of the loop, i.e. the array A[ j ] is sorted. If A[i ] A[j] < A[i], the all the elemets A[i],, A[j ] are moved to the ext positio, ad A[j] is iserted i place of A[i], maitaiig the order (Termiatio) Obvious. Whe the loop termiates, all the A[ ] elemets are properly ordered. Isertio Sort: Ruig Time Iput: array A of legth Output: sorted array A for j= to do set key:=a[j] set i:=j- while i>0 ad A[i]>key do set A[i+]:= A[i], i:=i- set A[i+]:=key edwhile times j times edfor ( ) ( + 3( j )) = 3 j ( ) = 3 j= j= = O( )
9/5/009 Algorithms Sortig 3-7 Algorithms Sortig 3-8, Divide ad Coquer Recursive algorithms: Call themselves o subproblem Divide ad Coquer algorithms: Split a problem ito subproblems (divide) Solve subproblems recursively (coquer) Combie solutios to subproblems (combie) 5 4 7 3 6 5 4 7 3 6 Divide: Split a give sequece ito halves Coquer: By callig itself sort the two halves Combie: Merge the two sorted arrays ito oe 5 4 7 3 6 5 4 7 3 6 Algorithms Sortig 3-9 Algorithms Sortig 3-0 (A,p,r) Iput: array A, positios p,r Output: array A such that etries A[p],,A[r] are sorted if p<r the do set q:= (p+r)/ (A,p,q) (A,q+,r) Merge(A,p,q,r) Merge The Merge procedure is applied to array A ad three positios p, q, r i this array Assume p q < r A[p],, A[q] ad A[q+],, A[r] are ordered Outputs ordered sequece i positios A[p],, A[r] This sequece is geerated by comparig the two elemets o the top of subarrays ad movig the smaller oe Algorithms Sortig 3- Algorithms Sortig 3- Merge Merge: Soudess Merge(A,p,q,r) set u:=q-p+, set v:=r-q set L[...u]:=A[p...q] set R[...v]:=A[q+...r] set i:=, j:= for k=p to r do if l[i] R[j] the set A[k]:=L[i], i:=i+ else set A[k]:=R[j], j:=j+ edfor set L[u+]:= set R[v+]:= setiel cards Ivariat for Merge: At the start of each iteratio of the for loop, the subarray A[p k-] cotais the k p smallest elemets of L[ u+] ad R[ v+] i sorted order. Moreover, L[i] ad R[j] are the smallest elemets of the correspodig arrays that have ot bee copied to A (Iitializatio) Iitially k = p, so the subarray A[p k ] is empty. It cotais the k p = 0 smallest elemets of L ad R. Sice i = j =, both L[i] ad R[j] are the smallest elemets i the correspodig arrays. The ivariat is true
9/5/009 Algorithms Sortig 3-3 Algorithms Sortig 3-4 Merge: Soudess Example (Maiteace) Suppose the property is true before iteratio k of the loop. If L[i] R[j], the L[i} is the smallest elemet ot yet copied ito A. Sice A[p k ] cotais the k p smallest elemets, after the iteratio L[i] is copied ito A[k], ad A[p k] cotais the k p + smallest elemets. New top elemets of L ad R are clearly the smallest oes After icremetig k the loop ivariat is true agai If L[i] < R[j] the argumet is similar (Termiatio) Obvious. Whe the loop termiates, A[p r] cotais the k p smallest elemets from L ad R, that is all but the setiels. 5 4 7 3 6 Algorithms Sortig 3-5 Algorithms Sortig 3-6 : Soudess returs a sorted array : Ruig Time The ruig time of Merge whe applied to two arrays of total size is Θ() Follows from the soudess of Merge. Fiish the proof yourself QED The ruig time, T(), of is T() = C + T(/) + T(/) + D If = the T() = C divide recursio Merge Recursio tree Algorithms Sortig 3-7 Algorithms Sortig 3-8 : Soudess Recursio tree i There are odes o level i C + D Each ode requires i work Total work o each level: There are log levels (C+D) The ruig time of is Θ( log ) Heaps The mai setback of the IsertioSort is that to isert it eeds to sca a substatial part of the array. Ca it be sped up? Yes! Usig biary trees --- heaps keys 8 4 7 6 4 0 9 3 A heap is a early complete biary tree Recall trees etc.
9/5/009 Algorithms Sortig 3-9 Algorithms Sortig 3-0 Heap Property Let Paret(i) deote the paret of the vertex i Max-Heap Property: Key(Paret(i)) Key(i) for all i Mi-Heap Property: Key(Paret(i)) Key(i) for all i 6 Heaps Nearly complete biary tree meas that the legth of ay path from the root to a leaf ca vary by at most oe The height of a vertex i is the legth of the logest simple dowward path from i Therefore the height of the root is aroud log 4 0 8 7 9 3 4 Algorithms Sortig 3- Algorithms Sortig 3- Heap Operatios Creatig a max-heap Accessig the miimal elemet (root) Isertig a elemet Deletig a elemet Goal ruig time O() O() O(log ) O(log ) Implemetig Heaps ad Operatios Heap ca be implemeted by a array 6 4 0 8 7 9 3 4 Childre: leftchild(i) = i rightchild(i) = i + Paret: paret(i) = i / Legth: legth(h) = the umber of elemets i H Algorithms Sortig 3-3 Algorithms Sortig 3-4 Isertio Isert(H,key) set :=legth(), set H[+]:=key HeapifyUp(H,+) HeapifyUp(H,i) if i> the set j:=paret(i)= i/ if Key[H[i]]>Key[H[j]] the swap array etries H[i] ad H[j] HeapifyUp(H,j) HeapifyUp: Soudess The procedure HeapifyUp(H,i) fixes the heap property i O(log i) time, assumig that the array H is almost a heap with the key of H[i] too large. The ruig time of Isertio is O(log ) Iductio o i. Base Case i = is obvious Iductio Case: Swappig elemets takes O() time It remais to observe that after swappig H remais a heap or almost heap
9/5/009 Algorithms Sortig 3-5 Algorithms Sortig 3-6 Deletio Delete(H,i) set :=legth(), set H[i]:=H[] if Key[H[i]]>Key[H[paret(i)]] the HeapifyUp(H,i) if Key[H[i]]<Key[H[leftChild(i)]] or Key[H[i]]<Key[H[rightChild(i)]] the HeapifyDow(H,i) Deletio (ctd) HeapifyDow(H,i) set :=legth(h) if i> the Termiate with H uchaged else if i< the do set left:=i, right:=i+ let j be the idex that miimizes Key[H[left]] ad Key[H[right]] else if i= the set j:=i if Key[H[j]]>Key[H[i]] the swap array etries H[i] ad H[j] HeapifyDow(H,j) Algorithms Sortig 3-7 Algorithms Sortig 3-8 HeapifyDow: Soudess The procedure HeapifyDow(H,i) fixes the heap property i O(log i) time, assumig that the array H is almost a heap with the key of H[i] too small. The ruig time of Deletio is O(log ) Homework Explai how to implemet creatig a heap, accessig ad deletig the maximal elemet DIY