Sorting Algorithms. + Analysis of the Sorting Algorithms

Transcription

1 Sorting Algorithms + Analysis of the Sorting Algorithms

2 Insertion Sort What if first k elements of array are already sorted? 4, 7, 12, 5, 19, 16 We can shift the tail of the sorted elements list down and then insert next element into proper position and we get k+1 sorted elements 4, 5, 7, 12, 19, 16 2

3 Divide and Conquer Very important strategy in computer science: Divide problem into smaller parts Independently solve the parts Combine these solutions to get overall solution Idea 1: Divide array into two halves, recursively sort left and right halves, then merge two halves known as Mergesort Idea 2 : Partition array into small items and large items, then recursively sort the two sets known as Quicksort Idea 3 : Bubble Sort 3

4 (1) Merge Sort Merge sort is a sorting technique based on divide and conquer technique. With worst-case time complexity being Ο(n log n), it is one of the most respected algorithms. Merge sort first divides the array into equal halves and then combines them in a sorted manner. 4

5 How it works To understand merge sort, we take an unsorted array as the following. We know that merge sort first divides the whole array iteratively into equal halves unless the atomic values are achieved. We see here that an array of 8 items is divided into two arrays of size 4. 5

6 How it works This does not change the sequence of appearance of items in the original. Now we divide these two arrays into halves. We further divide these arrays and we achieve atomic value which can no more be divided. 6

7 How it works Now, we combine them in exactly the same manner as they were broken down. Please note the color codes given to these lists. We first compare the element for each list and then combine them into another list in a sorted manner. We see that 14 and 33 are in sorted positions. We compare 27 and 10 and in the target list of 2 values we put 10 first, followed by 27. We change the order of 19 and 35 whereas 42 and 44 are placed sequentially. 7

8 How it works In the next iteration of the combining phase, we compare lists of two data values, and merge them into a list of found data values placing all in a sorted order. After the final merging, the list should look like this 8

9 MergeSort Steps Step 1 if it is only one element in the list it is already sorted, return. Step 2 divide the list recursively into two halves until it can no more be divided. Step 3 merge the smaller lists into new list in sorted order. 9

10 Merge Sort Coding procedure mergesort( var a as array ) if ( n == 1 ) return a var l1 as array = a[0]... a[n/2] var l2 as array = a[n/2+1]... a[n] l1 = mergesort( l1 ) l2 = mergesort( l2 ) return merge( l1, l2 ) end procedure procedure merge( var a as array, var b as array ) var c as array while ( a and b have elements ) if ( a[0] > b[0] ) add b[0] to the end of c remove b[0] from b else add a[0] to the end of c remove a[0] from a end if end while while ( a has elements ) add a[0] to the end of c remove a[0] from a end while while ( b has elements ) add b[0] to the end of c remove b[0] from b end while return c end procedure 10

11 Mergesort Divide it in two at the midpoint Conquer each side in turn (by recursively sorting) Merge two halves together 11

12 Mergesort Example Divide Divide Divide element Merge Merge Merge

13 Auxiliary Array The merging requires an auxiliary array Auxiliary array 13

14 Auxiliary Array The merging requires an auxiliary array Auxiliary array 14

15 Auxiliary Array The merging requires an auxiliary array Auxiliary array 15

16 Mergesort Analysis Let T(N) be the running time for an array of N elements Mergesort divides array in half and calls itself on the two halves. After returning, it merges both halves using a temporary array Each recursive call takes T(N/2) and merging takes O(N) 16

17 Mergesort Recurrence Relation The recurrence relation for T(N) is: T(1) < c base case: 1 element array constant time T(N) < 2T(N/2) + dn Sorting n elements takes the time to sort the left half plus the time to sort the right half plus an O(N) time to merge the two halves T(N) = O(N log N) 17

18 Properties of Mergesort Not in-place Requires an auxiliary array Very few comparisons Iterative Mergesort reduces copying. 18

19 (2) Quick Sort Quicksort is a fast sorting algorithm, which is used not only for educational purposes, but widely applied in practice. On the average, it has O(n log n) complexity, making quicksort suitable for sorting big data volumes. The idea of the algorithm is quite simple and once you realize it, you can write quicksort as fast as bubble sort.

20 Algorithm The divide-and-conquer strategy is used in quicksort. Below the recursion step is described: Choose a pivot value. We take the value of the middle element as pivot value, but it can be any value, which is in range of sorted values, even if it doesn't present in the array. Partition. Rearrange elements in such a way, that all elements which are lesser than the pivot go to the left part of the array and all elements greater than the pivot, go to the right part of the array. Values equal to the pivot can stay in any part of the array. Notice, that array may be divided in non-equal parts. Sort both parts. Apply quicksort algorithm recursively to the left and the right parts.

21 In Details There are two indices i and j and at the very beginning of the partition algorithm i points to the first element in the array and j points to the last one. Then algorithm moves i forward, until an element with value greater or equal to the pivot is found. Index j is moved backward, until an element with value lesser or equal to the pivot is found. If i j then they are swapped and i steps to the next position (i + 1), j steps to the previous one (j - 1). Algorithm stops, when i becomes greater than j. After partition, all values before i-th element are less or equal than the pivot and all values after j-th element are greater or equal to the pivot.

22 Example. Sort {1, 12, 5, 26, 7, 14, 3, 7, 2} using quicksort. Notice, that we show here only the first recursion step, in order not to make example too long. But, in fact, {1, 2, 5, 7, 3} and {14, 7, 26, 12} are sorted then recursively.

23 void quicksort(int arr[], int left, int right) { int i = left, j = right; int tmp; int pivot = arr[(left + right) / 2]; /* partition */ while (i <= j) { while (arr[i] < pivot) i++; while (arr[j] > pivot) j--; if (i <= j) { tmp = arr[i]; arr[i] = arr[j]; arr[j] = tmp; i++; j--; } }; Quick Sort Code in C++ /* recursion */ if (left < j) quicksort(arr, left, j); if (i < right) quicksort(arr, i, right); }

24 How it works, again! Quicksort uses a divide and conquer strategy, but does not require the O(N) extra space that MergeSort does Partition array into left and right sub-arrays the elements in left sub-array are all less than pivot elements in right sub-array are all greater than pivot Recursively sort left and right sub-arrays Concatenate left and right sub-arrays in O(1) time 24

25 Four easy steps To sort an array S If the number of elements in S is 0 or 1, then return. The array is sorted. Pick an element v in S. This is the pivot value. Partition S-{v} into two disjoint subsets, S 1 = {all values x v}, and S 2 = {all values x v}. Return QuickSort(S 1 ), v, QuickSort(S 2 ) 25

26 The steps of QuickSort S select pivot value S 1 S 2 partition S S 1 S QuickSort(S 1 ) and QuickSort(S 2 ) S [Weiss] Presto! S is sorted 26

27 (3) Bubble Sort Bubble sort is a simple and wellknown sorting algorithm. It is used in practice once in a blue moon and its main application is to make an introduction to the sorting algorithms. Bubble sort belongs to O(n 2 ) sorting algorithms, which makes it quite inefficient for sorting large data volumes. Bubble sort is stable and adaptive. Algorithm Compare each pair of adjacent elements from the beginning of an array and, if they are in reversed order, swap them. If at least one swap has been done, repeat step 1.

28 Example. Sort {5, 1, 12, -5, 16} using bubble sort. Complexity analysis Average and worst case complexity of bubble sort is O(n 2 ). Also, it makes O(n 2 ) swaps in the worst case. Bubble sort is adaptive. It means that for almost sorted array it gives O(n) estimation. Avoid implementations, which don't check if the array is already sorted on every step (any swaps made). This check is necessary, in order to preserve adaptive property.

29 Comparison of Sorting Algorithms Time Sort Average Best Worst Space Stability Remarks Bubble sort O(n^2) O(n^2) O(n^2) Constant Stable Always use a modified bubble sort Modified Bubble sort O(n^2) O(n) O(n^2) Constant Stable Stops after reaching a sorted array Selection Sort O(n^2) O(n^2) O(n^2) Constant Stable Even a perfectly sorted input requires scanning the entire array Insertion Sort O(n^2) O(n) O(n^2) Constant Stable Heap Sort O(n*log(n)) O(n*log(n)) O(n*log(n)) Constant Instable Merge Sort O(n*log(n)) O(n*log(n)) O(n*log(n)) Depends Stable Quicksort O(n*log(n)) O(n*log(n)) O(n^2) Constant Stable In the best case (already sorted), every insert requires constant time By using input array as storage for the heap, it is possible to achieve constant space On arrays, merge sort requires O(n) space; on linked lists, merge sort requires constant space Randomly picking a pivot value (or shuffling the array prior to sorting) can help avoid worst case scenarios such as a perfectly sorted array.