Lecture 7: Sorting Techniques. Prakash Gautam 26 April, 2018

Transcription

1 Lecture 7: Sorting Techniques Prakash Gautam 26 April, 2018

2 Agenda Introduction to Sorting Different Sorting Techniques Bubble Sort Selection Sort Insertion Sort Merge Sort Quick Sort Shell Sort 2

3 3

4 4

5 5

6 Sorting An operation that segregates items into groups according to specified criterion Input: A = { } Output: A = { } Sorting: Ordering Sorted: Ordered based on a particular way Examples Words in the dictionary 6

7 It is arranging the elements in a list or collection in increasing or decreasing order of some property We may have list of any data types Strings or Words: Lexicographical List of Integers: Increasing order of value 2, 3, 9, 4, 6 2, 3, 4, 6, 9 [Increasing order of value] 9, 6, 4, 3, 2 [Decreasing order of value] 2, 3, 9, 4, 6 [Increasing order of # factors] 7

8 Sorted data are useful not only for representation & retrieval of data It also significantly helps to improve the computational power Unsorted: Linear Search Sorted: Binary Search Goal: Study, Analyze & Compare the various sorting algorithms 8

9 Sorting Algorithms Bubble Sort Selection Sort Insertion Sort Merge Sort Quick Sort Shell Sort Radix Sort Swap Sort Heap Sort 9

10 Classification of Sorting Algorithms Time Complexity Rate of growth of time taken by an algorithm with respect to input size, n Space Complexity In-Place Algorithm or not? Stability Does it preserve the relative order of key values? Internal or External Sort (RAM or Disks) Recursive or Non-Recursive 10

11 Bubble Sort Bubble sort: Sinking sort It is a simple sorting algorithm that works by repeatedly stepping through the list to be sorted, comparing each pair of adjacent items and swapping them if they are in the wrong order The pass through the list is repeated until no swaps are needed, which indicates that the list is sorted 11

12 12

13 Pass 1 13

14 Pass 2 14

15 Pass 3 15

16 Pass 4 16

17 The algorithm gets its name from the way smaller elements "bubble" to the top of the list As it only uses comparisons to operate on elements, it is a comparison sort Notice that at least one element will be in the correct position each iteration Although the algorithm is simple, it is too slow for practical use 17

18 Bubble Sort: Algorithm Bubble_Sort(A,n) for k=1 to n-1 for i=0 to n-2 if(a[i]>a[i+1]) Swap(A[i], A[i+1]) Swap(A[i],A[i+1]) temp=a[i] A[i]=A[i+1] A[i+1]=temp 18

19 Bubble Sort: Improvement-I Bubble_Sort(A,n) for k=1 to n-1 for i=0 to n-k-1 if(a[i]>a[i+1]) Swap(A[i], A[i+1]) 19

20 Bubble Sort: Improvement-II Bubble_Sort(A,n) for k=1 to n-1 for i=0 to n-k-1 flag=0 if(a[i]>a[i+1]) Swap(A[i], A[i+1]) flag=1 if(flag==0) break; 20

21 Bubble Sort: Complexity Analysis Best Case: [ O(n)] Worst Case: [ O(n2)] Average Case: [ O(n2)] 21

22 Selection Sort Array is imaginary divided into two parts - sorted one & unsorted one At the beginning, sorted part is empty, while unsorted one contains whole array At every step, algorithm finds minimal element in the unsorted part and adds it to the end of the sorted one When unsorted part becomes empty, algorithm stops 22

23 23

24 24

25 25

26 Selection Sort: Algorithm Selection_Sort(A,n) for i=0 to n-1 imin=i for j=i+1 to n-1 if(a[j]<a[imin]) imin=j Swap(A[i], A[iMin]) 26

27 Selection Sort: Complexity Analysis O(n2) It minimizes # of swaps 27

28 Insertion Sort Array is imaginary divided into two parts - sorted one & unsorted one At the beginning, sorted part is empty, while unsorted one contains whole array It keeps a prefix of the array sorted This prefix is grown by inserting the next value into it at the correct place Eventually, the prefix is the entire array, which is therefore sorted 28

29

30

31 Insertion Sort: Algorithm Insertion_Sort(A,n) for i=1 to n-1 Value=A[i]; hole=i; while(hole>0 && A[hole-1]>Value) A[hole]=A[hole-1] hole=hole-1 A[hole]=Value 31

32 Insertion Sort: Complexity Analysis Best Case: [ O(n) ] Worst Case: [O(n2)] Average Case: [O(n2)] It minimizes # of swaps Practical comparisions & swaps are much less than Bubble & Selection sort. 32

33 Merge Sort Divide & Conquer DIVIDE: Partition the n-element sequence to be sorted into two subsequences of n/2 elements each CONQUER: Sort the two subsequences recursively using the merge sort COMBINE: Merge the two sorted subsequences of size n/2 each to produce the sorted sequence Note that, Recursion "bottoms out" when the sequence to be sorted is of unit length 33

34 Since every sequence of length 1 is in sorted order, no further recursive call is necessary The key operation of the merge sort algorithm is the merging of the two sorted sub sequences in the "combine step" 34

35 35

36 Merge Sort: Algorithm Merge_Sort(A) n=length(a); if(n<2) {Its Sorted} mid=n/2; left=array of size(mid); right=array of size(n-mid) for i=0 to mid-1 left[i]=a[i] for i=mid to n-1 right[i-mid]=a[i] Merge_Sort(left); Merge_Sort(right); Merge(left, right, A) 36

37 Merge(L, R, A) nl=length(l); nr=length(r); i=j=k=0; while(i<nl && i<nr) if(l[i]<=r[j]) A[k]=L[i]; i++; else A[k]=R[j]; j++; k++; while(i<nl) A[k]=L[i]; i++; k++; while(j<nr) A[k]=R[j]; j++; k++; 37

38 Merge Sort: Analysis Time Complexity: O(n logn) Space Complexity Non In-Place Algorithm.WHY? If we don t clear extra memory for left & right: O(n logn) If we clear extra memory in each call: O(n) 38

39 Quick Sort It is the currently fastest known sorting algorithm and is often the best practical choice for sorting Pick an element, called a pivot, from the array Reorder the array so that all elements with values less than the pivot come before the pivot, while all elements with values greater than the pivot come after it (equal values can go either way). After this partitioning, the pivot is in its final position. This is called the partition operation 39

40 Recursively apply the above steps to the sub-array of elements with smaller values and separately to the sub-array of elements with greater values Divide & Conquer Divide: Partition T[i..j] = T[i.. k-1] & T[k+1... j] such that each element of T[i..k-1]<=Pvt & T[k+1.j]>Pvt Conquer: Sort the two sub arrays T[i.. k-1] & T[k+1... j] by recursive calls to quicksort Combine: Since the sub arrays are sorted in place, no work is needed to combining them: the entire array T[i..j] is now sorted 40

41 41

42 42

43 43

44 44

45 Quick Sort: Algorithm Quick_Sort(A, Start, End) if (start<end) Pindex=Partition(A, Start, End) Quick_Sort(A, Start, Pindex-1); Quick_Sort(A, Pindex+1, End) 45

46 Partition(A, Start, End) pivot=a[end] ; Pindex=Start; for i=start to End-1 if (A[i]<=pivot) swap(a[i], A[Pindex]); Pindex=Pindex+1; Swap(A[Pindex], A[End]); return Pindex; 46

47 Quick Sort: Analysis Time Complexity Best Case(Balanced): O(n logn) Worst Case(If already sorted/unbalanced): O(n2) Solution: Randomized Partition Average Case: O(n logn) Space Complexity An In-Place Algorithm Worst Case: O(n) 47

48 Randomized_Partition(A, Start, End) pivotindex=random(start, End) Swap(A[pivotIndex], A[End]) Partition(A, Start, End) 48

49 Shell Sort Donald L. Shell (1959) Generalization of the Insertion Sort We compare elements that are distant apart rather than adjacent Comparison of elements: If there are N elements then we start with a value gap<n In each pass, we keep reducing the value of gap till we reach the last pass when gap is 1 49

50 In last pass: Shell Sort = Insertion Sort [ ] - A[ ] Index Total Elements (N) = 9 gap must be less than N gap = Floor[N/2] 50

51 Here gap = 4 { Floor[9/2] } So, Pass=1 & gap=4 First element at Index 0 Second at Index, 0+4=4 Third at Index, 4+4=8 [ ] - A[ ] Index 51

52 [ ] - A[ ] Index Is 14 > 23...?, Is 18 > 40?, Is 19 > 29?, Is 37 > 30 (Now Swap) Is 23 > 11?(Now Swap), Is 14 > 11? [ ] - A[ ] 52

53 Pass=2 & gap=2 gap = Floor [gap/2] = 2 [ ] - A[ ] Index [ ] - A[ ] Index 53

54 Pass=3 & gap=1 gap = Floor [2/2] = 1 Equivalent with Insertion Sort when gap = 1 [ ] - A[ ] Index FINALLY SORTED: [ ] - A[ ] Index 54

55 Shell Sort: Algorithm Shell_Sort(A, Size) gap = Size/2; While(gap > 0) i = gap while(i < Size) temp = A[i] for(j=i; j>=gap && A[j-gap]>temp; j=j-gap) A[j]=A[j-gap] A[j]=temp i=i+1 gap=gap/2 55

56 Shell Sort: Algorithm 1. Calculate gap 2. While gap>0 FOR each element in the list, gap apart Extract the current item Locate the position to insert Insert the item to the position END FOR 3. Calculate gap 4. END While 56

57 Shell Sort: Analysis Time Complexity Average Case: O(n5/4) O(n3/2) Worst case: Insertion Sort O(n2) Exact Complexity of this algorithm is still being debated Space Complexity In Place Stable Sorting? No, It doesn t preserve the relative order of duplicates Experience: Not better than O(nlogn) 57

58 Heap Sort 58

59 59

60 60

61 61

62 Heap Data Structure A special tree-based data structure It must be a complete binary tree 62

63 Heapify Heapify (A) Root = A[0] Largest=largest(A[0],A[2i+1], A[2i+2]) If (Root!= Largest) Swap(Root, Largest) 63

64 heapify(int arr[], int n, int i) { int largest = i; int l = 2*i + 1; int r =2*i+ 2; if (l < n && arr[l] > arr[largest]) largest = l; if (r < n && arr[r] > arr[largest]) largest = r; if (largest!= i) {swap(arr[i], arr[largest]); heapify(arr, n, largest)} } 64

65 65

66 66

67 67

68 Heap Sort Max-Heap: largest item is stored at the root node Remove the root node & put at end of the array Reduce the size of the heap by 1 and heapify the root element again so that we have highest element at root The process is repeated until all the items of the list is sorted 68

69 69

70 70

71 71

72 72

73 73

74 74

75 75

76 Heap Sort: Algorithm for (int i=n-1; i>=0; i--) { swap(arr[0], arr[i]); //call max heapify on the reduced heap heapify(arr, i, 0); } 76

77 Heap Sort: Analysis Time Complexity O(nlogn) The height of a complete binary tree containing n elements is log(n) Space Complexity In Place Stable Sorting? No, It doesn t preserve the relative order of duplicates Recursive 77

78 78

79 Radix Sort Counting Sort Topological Sort Bucket Sort Comb Sort Cycle Sort Cocktail Sort Bitonic Sort Gnome Sort Sleep Sort 79

80 Thank You!? 80