Copyright 2009, Artur Czumaj 1

Transcription

1 CS 244 Algorithm Design Instructor: Artur Czumaj Lecture 2 Sorting You already know sorting algorithms Now you will see more We will want to understand generic techniques used for sorting! Lectures: Monday 13:00 14:00 Tuesday 13:00 14:00 Friday 9:00-10:00 www2.warwick.ac.uk/fac/sci/dcs/teaching/material/cs244 : Insertion Sort Bubble sort Merge sort Counting sort Bucket sort Which algorithm is the best? Bubble sort for j=i-1 downto 0 if A[j+1] < A[j] then swap A[j] with A[j+1]; Insertion sort(a[0 n-1]: real) { Quick insert sort A[i] into A[0..i-1] so that Invariant: At the beginning of the loop with i: A[0 i-1] is sorted Corollary: Algorithm will sort Copyright 2009, Artur Czumaj 1

2 : Insertion Sort Bubble sort for j=i-1 downto 0 if A[j+1] < A[j] then swap A[j] with A[j+1]; Insertion sort(a[0 n-1]: real) { Quick insert sort A[i] into A[0..i-1] so that Invariant: At the beginning of the loop with i: A[0 i-1] is sorted Corollary: Algorithm will sort We want to insert 2 into previously sorted part We want to insert 1 into previously sorted part Copyright 2009, Artur Czumaj 2

3 We want to insert 1 into previously sorted part We want to insert 5 into previously sorted part We want to insert 4 into previously sorted part We want to insert 4 into previously sorted part And so on Copyright 2009, Artur Czumaj 3

4 Bubble sort Insertion sort(a[0 n-1]: real) { insert A[i] into A[0..i-1] so that for j=i-1 downto 0 if A[j+1] < A[j] then swap A[j] with A[j+1]; Invariant: At the beginning of the loop with i: A[0 i-1] is sorted Corollary: Algorithm will sort Bubble sort for j=i-1 downto 0 if A[j+1] < A[j] then swap A[j] with A[j+1]; Insertion sort(a[0 n-1]: real) { Quick insert sort A[i] into A[0..i-1] so that Invariant: At the beginning of the loop with i: A[0 i-1] is sorted Corollary: Algorithm will sort Running time: Number of comparisons: Number of times we re moving an element: Insertion-Sort: running time Bubble sort Insertion sort(a[0 n-1]: real) { insert A[i] into A[0..i-1] so that for j=i-1 downto 0 if A[j+1] < A[j] then swap A[j] with A[j+1]; We can write a recurrence if we start from the end! We first sort A[0 n-2] And then insert A[n-1] Recurrence: T(n) = T(n-1) + O(n) Same as before! T(n) = O(n 2 ) Worst-case number of comparisons: for inversely sorted sequence (decreasing) it s: n-1 = n (n-1)/2 Number of moves : for inversely sorted sequence (decreasing) it s: n-1 = n (n-1)/2 Copyright 2009, Artur Czumaj 4

5 Sorting algorithms Bubble sort Merge sort Counting sort Bucket sort Which algorithm is the best? Performs only O(n) moves of the elements Obvious worst-case lower bound (because for some inputs all n elements must be moved) Merge sort & Make more than O(n) moves But perform only O(n log n) operations May need O(n 2 ) time in the worst-case Typically does only O(n log n) operations Can beat all above. but only in the special case There is no universally best algorithm All depend on the problem instance, type of the input etc Merge sort Basic divide-and-conquer paradigm 1. Partition the array A into two halves 2. Recursively sort each half 3. Merge the two sorted sequences Merge sort(a[0 n-1]: real) { recursively sort A[0.. b(n-1)/2 c ]; recursively sort A[ b(n-1)/2 c +1.. n-1]; merge the two sorted sequences into one sorted sequence } 4 Running time: T(n) = T(dn/2e) + T(bn/2 c) + cost-of-merging(n) How to merge? Copyright 2009, Artur Czumaj 5

6 4 4 How to merge? How to merge? Merging is not hard Merging is not hard We can merge two sorted arrays of length n/2 each in O(n) time, with at most n-1 comparisons We can merge two sorted arrays of length n/2 each in O(n) time, with at most n-1 comparisons Single comparison to move one element down Two pointers (one for each part): smallest element which is not yet sorted Which element can be the smallest? Either the smallest from left or the smallest from right Check which one Copyright 2009, Artur Czumaj 6

7 Merging is not hard Merging is not hard We can merge two sorted arrays of length n/2 each in O(n) time, with at most n-1 comparisons We can merge two sorted arrays of length n/2 each in O(n) time (with at most n-1 comparison) Number of comparisons: 1 per each element moved (except last) n-1 Running-time of Merge sort T(n) = T(dn/2e) + T(bn/2 c) + cost-of-merging(n) = 2 T(n/2) + O(n) = 2 (2 T(n/4) + O(n/2)) + O(n) = 4 T(n/4) + 2 O(n) = 4 (2 T(n/8) + O(n/4)) + 2 O(n) = 8 T(n/8) + 3 O(n) = = 2 k T(n/2 k ) + k O(n) = O(n log n) Number of comparisons of Merge sort T(n) = T(dn/2e) + T(bn/2 c) + n-1 n log 2 n Basic divide-and-conquer paradigm Divide-and-Conquer method Almost same as Merge-sort but sometimes it s better and sometimes it isn t Divide-and-Conquer Pick a pivot element (say, A[0]) Partition A into three parts: Elements smaller than pivot Elements equal to pivot Elements greater than pivot Recursively sort 1 st and 3 rd part Merge the obtained sorted parts (A[0 n-1]: real) { choose a pivot element A[i] and let x = A[i] partition and rearrange the elements in A into elements smaller than x; elements equal to x, and elements greater than x: let A[0.. p-1] contain all elements smaller than x let A[q.. n-1] contain all elements greater than x recursively sort A[0.. p-1]; recursively sort A[ q.. n-1]; } Copyright 2009, Artur Czumaj 7

8 Running time T(n) = O(n) + T(p) + T(q), with p+q < n Number of comparisons: T*(n) = n-1 + T*(p) + T*(q), with p+q < n Worst-case Bad If pivot is poorly chosen: T(n) = max k {T(n-k-1) + T(k)} + (n) = T(n-1) + T(0) + (n) = (n 2 ) Copyright 2009, Artur Czumaj 8