having any value between and. For array element, the plot will have a dot at the intersection of and, subject to scaling constraints.

Transcription

1 02/10/ :42 AM Class 7 From Wiki6962 Table of contents 1 Basic definitions 2 Bubble Sort 2.1 Observations 3 Quick Sort 3.1 The Partition Algorithm 3.2 Duplicate Keys 3.3 The Pivot element 3.4 Size of the input array 3.5 Performance numbers for Quick sort 3.6 Observations 4 Heap Sort 4.1 MAX-HEAPIFY Procedure 4.2 BUILD-MAX-HEAP Procedure 4.3 HEAP-SORT Procedure 4.4 Analysis 4.5 Performance numbers for Heap sort 4.6 Observations 5 Merge Sort 5.1 Top-Down Merge sort 5.2 Bottom-Up Merge sort 5.3 Analysis 5.4 Performance numbers for Bottom-Up Merge sort 5.5 Observations Basic definitions Adaptive Sort : A sorting algorithm that can take advantage of existing order in the input, reducing its requirements for computational resources as a function of the disorder in the input is called an Adaptive sorting algorithm. In other words, an adaptive sorting algorithm will complete sorting faster by performing a different sequence of operations if most of the input data is already sorted. Non-adaptive Sort : A non-adaptive sorting algorithm on the contrary will perform the same sequence of operations irrespective of any order in the input. It fails to take advantage of any underlying order in the input data. Such sorts are well suited for a hardware implementation. Adaptive sorts represent an optimization of the sorting process and prove advantageous in computationally expensive operations. Stable sort : A stable sorting algorithm is one that maintains the relative order of elements with equal keys (i.e. values). That is, a sorting algorithm is stable if whenever there are two elements and with the same value and with appearing before in the original unsorted list, will appear before in the sorted list. Bubble Sort We use a diagrammatic notation to understand the execution of Sorting algorithms. It has been taken from the book Algorithms in C++ (3rd edition) by Robert Sedgewick[1] ( Here each rectangular box represents a graphical plot of the execution of a sorting algorithm. The represents the array index and the represents the value stored at that position in the array. So for example, assume we have an array of elements having any value between and. For array element, the plot will have a dot at the intersection of and, subject to scaling constraints. Assuming a random distribution of elements in the array we follow the following notation for interpreting the diagrams. Page 1 of 18

2 02/10/ :42 AM Graphical representation of the execution of Bubble sort Column of the diagram shows the execution of Shaker sort which is an optimization of Bubble sort. Both sorts have the same complexity, but Shaker sort coverges faster than Bubble sort. Observations 1. Bubble sort is an example of an inplace, stable, non-adaptive sort. It is stable because the algorithm maintains the original relative order among duplicate elements. 2. Bubble sort has a best and worst case complexity of. Quick Sort The following diagram shows pseudo-code for the Quick sort algorithm. Page 2 of 18

3 02/10/ :42 AM Quick sort pseudo-code Important criteria which affect the execution time of the Quick sort algorithm can be listed as : 1. The choice of the Partition algorithm. 2. How to handle duplicate elements/keys present in the input array? 3. The choice of the Pivot element. 4. The size of the input array. The Partition Algorithm The following diagrams show the working of the Partition algorithm from the pseudo-code above. Page 3 of 18

4 02/10/ :42 AM Working of Partition Algorithm The following is an example showing an array with unsorted elements. the correct position. is chosen as the pivot element. The last array shows the partitioned array with the pivot element finally placed in Page 4 of 18

5 02/10/ :42 AM Example showing Partition Algorithm in action An alternate way to implement the Partition algorithm can be as follows. 1. is the Pivot element. 2. Scan the array from the left end going to the right till you find an element greater than the Pivot, using pointer. 3. Scan the array from the right end going to the left till you find an element lesser than the Pivot, using pointer. 4. Swap the elements in the positions of and. 5. Continue the scan till the pointers cross each other and finally insert the pivot in its correct position. An alternative Partition Algorithm This algorithm has a slightly better performance than the previous one but still needs to address the issues of the choice of the Pivot element and the presence of duplicate keys. The following diagram shows the execution of Quick sort for two different randomly chosen pivot elements. After the sort, the Pivot element ends up on the diagonal while the remaining elements are partitioned into two subarrays. Page 5 of 18

6 02/10/ :42 AM Execution of the Quick sort algorithm for 2 different Pivots Duplicate Keys The presence of duplicate elements affects the performance of Quick sort since time is wasted in repeatedly moving duplicate elements around. A possible optimization to the second partitioning algorithm outlined in the previous subsection was proposed by Bentley and McIlroy. The idea was : 1. Keep keys equal to the Pivot that are encountered in the left subarray to the left end of the array and vice versa for keys encountered in the right subarray. 2. When the pointers cross and the precise location of the Pivot is known, swap all the equal keys around the Pivot element. The following diagram shows the working of the algorithm : Algorithm Duplicate keys and the Partition Observations : 1. The extra overhead for duplicate keys is proportional to the number of duplicate keys found. So, if there are no duplicate keys in the input array, there is no overhead. 2. The method is linear time when there is only a constant number of key values. Each partitioning phase removes from the sort all the keys with the same value as the Pivot, so each key can be involved in atmost a constant number of partitions. The Pivot element In Quicksort, a good choice is to choose the Pivot element that is more likely to divide the file near the middle. This can be achieved if the Pivot is chosen in a truly random manner. Page 6 of 18

7 02/10/ :42 AM However since random number generators are computationally expensive, simpler solutions can also be used. One such solution is the Median-of-Three partitioning. Here three elements are sampled from the left, right and middle of the unsorted array/subarray. The median of the three is chosen as the Pivot. It has been found that this technique reduces total average running time of Quick sort by percent. Further details can be found in Sedgewick's book in this matter. Size of the input array Since Quick sort has recursive calls, such calls can be avoided for subarrays which are small in size. This avoids the overhead of a recursive procedure. Insertion sort can be used to sort such small subarrays. Performance characteristics of Quick sort for a variety of input files is shown in the following two diagrams, using random Pivot selection and Median-of-Three method. Quick sort with random Pivot selection Page 7 of 18

8 02/10/ :42 AM Median-of-Three Pivot selection Performance numbers for Quick sort Page 8 of 18

9 02/10/ :42 AM Quick sort Performance System is found to be relatively lacking in performance due to the use of function pointers in its calls. Check the man pages for for additional information. Observations 1. Quick sort is an inplace, adaptive, unstable sort. It is unstable because the partitioning algorithm does not maintain the relative order among duplicate elements. It is inplace because the sorting is performed on the input array without requiring an additional array for swap or storage. 2. Quick sort has a best case complexity of and worst case complexity of. 3. Sorting networks are also examples of adaptive sorts. See CRLS textbook page 704 for more on Sorting networks. Heap Sort Heap : A heap is a data structure which is a complete tree where every node has a key more extreme (greater or less) than or equal to the key of its parent. Here a complete tree is a tree in which all leaf nodes are at some depth or, and all leaves at depth are toward the left. If a new leaf node is to be added to a complete tree, it will be added to the immediate right of the leftmost node at level, or as the first leftmost leaf node on level. For example, the following diagram shows the result of adding a new node to a max-heap. Page 9 of 18

10 02/10/ :43 AM A heap with elements has a height of. Creating an unsorted heap from an unsorted array can be done in linear time since there are no constraints on the children and parents. MAX-HEAPIFY Procedure Pseudo-code showing the working of Heap sort is as follows. The main procedure for Heap sort is the MAX-HEAPIFY procedure. Pseudo code and working of the procedure is shown in the following diagrams. Page 10 of 18

11 02/10/ :43 AM Working of MAX-HEAPIFY Page 11 of 18

12 02/10/ :43 AM BUILD-MAX-HEAP Procedure MAX-HEAPIFY is in turn called by the BUILD-MAX-HEAP procedure and pseudo-code and working of the same is shown below. Page 12 of 18

13 02/10/ :43 AM Working of BUILD-MAX-HEAP Page 13 of 18

14 02/10/ :43 AM HEAP-SORT Procedure Finally the Heap sort algorithm can be summarized in the following pseudo-code. Analysis MAX-HEAPIFY is a complexity of procedure. Please see the explanation for the same provided below this class log's link in the Tentative Schedule page. Heap sort is an algorithm having. This is due to the following reason. The first max-heap is created. The element at the top of the heap is the largest element of the heap. Since Heap sort is an inplace sort, the heap is maintained within the input array itself. Due to the this the last element of the array is also the last (lowest level and rightmost) leaf node in the heap. For example Page 14 of 18

15 02/10/ :43 AM The top-of-heap element and the last heap element are swapped. As a result the last element in the array becomes the top-of-heap element (the largest element in the array). MAX-HEAPIFY is now called again since a new max-heap has to be created. This is called on elements, since the largest element is already in place. The heap has a height of. Finding the correct position of the top element (which resulted due to the swap) in the heap can take operations in the worst case. Since there are a total of elements in the array, the complexity of Heap sort becomes. Performance numbers for Heap sort Observations 1. Heap sort is an inplace, unstable sort. It is unstable because the max-heap creation and the final swaps do not maintain the relative order among duplicate elements. It is inplace because the sorting is performed on the input array without requiring an additional data structure for swap or storage. 2. Heap sort can be made adaptive using a randomized binary search tree. Further details will be addressed in class or you can Google for it. 3. Heap sort has a complexity of. The choice between Heap sort and Quick sort boils down to the choice between average-case speed and worst-case speed. Merge Sort There are two versions of Merge sort, Top-Down and Bottom-Up. Page 15 of 18

16 02/10/ :43 AM Top-Down Merge sort Top-Down Merge sort works by recursively splitting the input array and one-half of the subarrays down the middle and sorting them before proceeding to sort the other half. This is an example of the Divide-and-Conquer strategy. Bottom-Up Merge sort Bottom-Up Merge sort on the other hand considers pairs of elements, sorts them and proceeds to merge them, doubling their size everytime. Bottom-Up Merge sort thus displays a parallel sorting behavior. This is an example of Combine-and-Conquer strategy. The following diagram shows the working of Bottom-Up Merge Sort and it's comparison with Top-Down Merge Sort. Analysis Merge sort has a complexity of. Top-Down or Bottom-Up Merge sort split or merge an array of elements times. Every time a merge is performed between two subarrays, there could be comparisons performed to get the merged elements in sorted order. This provides a complexity of for Merge sort. Performance numbers for Bottom-Up Merge sort Page 16 of 18

17 02/10/ :43 AM Page 17 of 18

18 02/10/ :43 AM Observations 1. Merge sort is not inplace and requires additional buffers for the merging process. Merge sort running time is insensitive to the input since the number of comparisons and other operations on the input is not dependent on how the input is ordered. However the sequence of comparisons does depend on the input order, which makes Merge sort an adaptive algorithm. The next lecture is likely to address this topic. 2. Merge sort has the advantage over Quick sort and Heap sort, wherein Merge sort is a stable sorting algorithm. Stable sort algorithms are used where data is indexed using multiple keys and the relative order between duplicate keys must be preserved. Radix sort is a sorting algorithm used to sort on multiple keys, provided there is an stable sort algorithm on a single key. 3. Merge sort has a best and worst case complexity of. Merge sort has thus advantage of being guaranteed to run fast irrespective of the input over Quick sort. Retrieved from "" This page was last modified 05:40, 13 Oct Page 18 of 18