Programming II (CS300)

Transcription

1 1 Programming II (CS300) Chapter 12: Sorting Algorithms MOUNA KACEM Spring 2018

2 Outline 2 Last week Implementation of the three tree depth-traversal algorithms Implementation of the BinarySearchTree class (contains and insert operations) Today Exam2 questions review BST: implementation of remove operation Sorting Algorithms Analysis of the selection sort Analysis of the insertion sort Heap sort (may be) Thursday Merge Sort Quick Sort Bucket Sort

3 Sorting Algorithms - Introduction 3 Many operations are more efficient by invoking a sorting procedure initially (such as look-up and search operations) Many applications have an output arranged in some sorted order Dictionary: words in a dictionary are sorted Files in a directory are often listed in a sorted order Index of a book is sorted Card catalog in a library is sorted by author and/or title Calendar of events in a schedule generally sorted by date A listing of course offering at a University is generally sorted by department, and then by course number or identifier

4 Sorting Algorithms 4 Input: an array containing N unsorted elements Only objects that implement the Comparable interface can be sorted Output: an array containing the N elements sorted

5 Selection Sort - Overview 5 Selection Sort: Combination of Searching and Sorting Basic idea Starting from position 0 to the index of the last element of the array 1, an inner loop finds the smallest element in the rest of the array (next smallest element), and the outer loop (i.e. first loop) places it in its proper position

6 Selection Sort - Overview 6 a: array of integers Array Positions (indices)

7 Analysis of the Selection Sort 7 Selection sort Short and simple algorithm Looking for the lowest element requires scanning all n elements (this operation takes N 1 comparisons) and then swapping it into the first position of the array. Finding the next lowest element requires scanning the remaining n 1 elements and so on, T(N) = (N 1) + (N 2) = N(N 1) / 2 O(N 2 ) Time complexity: quadratic (O(N 2 )) where N represents the number of elements in the array Simple sort algorithm appropriate for small inputs (for a need to sort a few elements)

8 Insertion Sort - Overview 8 a: array of integers Array Positions (indices)

9 Analysis of the insertion sort 9 Insertion sort Short and simple algorithm Time complexity: quadratic (O(N 2 )) in the worst case, where N represents the number of elements in the array Simple sort algorithm appropriate for small inputs (for a need to sort a few elements)

10 Merge Sort 10 Divide and conquer algorithm Two half size problems are solved recursively. Then, the result is merged to compose the solution for the original problem. When an original problem is divided into two half-size subproblems and each is solved recursively with an O(N) overhead, the resulting divide and conquer algorithm will have a running time of O(N log(n)), where N represents the size of the original problem

11 Merge sort 11 Basic idea: The mergesort algorithm involves three steps 1. If the number of items to sort is 0 or 1, return 2. Recursively sort the first and second halves separately 3. Merge the two sorted halves into a sorted group To claim an O(N log(n)) algorithm, the merging procedure for two sorted arrays should be performed in linear time (O(N))

12 Merge Sort Illustrative example 12 Divide Conquer

13 Outline 13 Last Week Insertion Sort Selection Sort Merge Sort Introducing Heap Sort Tuesday Heap Sort Quick Sort Bucket Sort Thursday Review for Exam 3

14 Heap Sort 14 The heap-sort algorithm consists of two phases 1. Construction phase: Construct a max-heap given N elements Start with an empty heap and move the boundary between the heap and the sequence from left to right, one step at a time. At step i, for i = 1,...,n, expand the heap by adding the element at index i 1 2. Extraction phase Pull out the value in the root successively, creating a new heap with one less element after each extraction step. Start with an empty sequence and move the boundary between the heap and the sequence from right to left, one step at a time. At step i, for i = 1,...,n, remove the maximal element (root) from the heap and store it at index n i.

15 Illustration of Second phase of Heap-Sort 15 Michael T. Goofrich, et al., "Data Structures & Algorithms", Wiley, six edition, 2014, pp

16 Quick Sort 16 Quick-sort algorithm sorts an array or sequence of comparable items using a simple recursive approach. The basic quicksort algorithm is recursive. The main idea is to apply the divide-and-conquer technique consisting of 3 steps: 1. Divide the array into two sub-arrays 2. Recur to sort each sub-array 3. Combine the sorted subsequences by a simple concatenation

17 Quick Sort 17 Input: array of N comparable items Details include choosing the pivot, deciding how to partition, and dealing with duplicates. The pivot divides array elements into two partitions: those smaller than the pivot, and those larger than the pivot.

18 Quick Sort Divide: If the number of elements in the array or sub-array A is 0 or 1, then return (base case). If A has at least two elements, select a specific element x from A, which is called the pivot. As is common practice, choose the pivot x to be the median element in S. Remove all the elements from A and put them into three partitions: Low partition, storing the elements in A smaller than x Equal, storing the elements in A equal to x High partition, storing the elements in A larger than x If the elements of A are distinct, then Equals holds just one element the pivot itself 2. Conquer: Recursively sort sequences Low and High. 3. Combine: Put back the elements into A in order by first inserting the elements of Low, then those of Equal (elements equals to the pivot), and finally those of High partitions.

19 Quick Sort 19 Input: array of N comparable items Wrong decisions (for instance an inappropriate choice of the pivot) give quadratic running times O(N 2 ) for a variety of common inputs At worst case, the running time for quick-sort algorithm is O(N 2 ) At average case, the running time for quick-sort algorithm is O(N*log(N))

20 Sorting Algorithms - Terminology 20 In-place sorting algorithm: a sorting algorithm is in-place if it uses only a small amount of memory in addition to the sequence storing the objects to be sorted; instead of transferring elements out of the sequence and then back in Comparison-based sorting algorithm An algorithm that makes ordering decisions only on the basis of comparisons

21 Sorting Algorithms - Time Complexity 21 Sort an array of N comparable items Insertion sort : O(N 2 ) at worst case Selection sort : O(N 2 ) at worst case Merge sort : O(N*log(N)) at worst case Heap sort: O(N*log(N)) at worst case Quick sort: O(N 2 ) at worst case; O(N*log(N)) at average case

22 Bucket Sort 22 Consider a sequence S of n entries whose keys are integers in the range [0, N-1], Suppose that S should be sorted according to the keys of the entries In that case, it is possible to sort S in O(n+N) running time if N is O(n), so the running time of a bucket sort will be O(n) bucket-sort algorithm is not based on comparisons

23 Bucket Sort 23 Main idea Use keys as indices into a bucket array B that has cells indexed from 0 to N - 1. An entry with key k is placed in the bucket B[k], which itself is a sequence (of entries with key k). After inserting each entry of the input sequence S into its bucket, Put the entries back into S in sorted order by enumerating the contents of the buckets B[0],B[1],...,B[N-1] in order

24 Running Times for Moderate Inputs 24

25 Costs for growth rates = 2 6