LECTURE NOTES OF ALGORITHMS: DESIGN TECHNIQUES AND ANALYSIS

Transcription

1 Department of Computer Science University of Babylon LECTURE NOTES OF ALGORITHMS: DESIGN TECHNIQUES AND ANALYSIS By Faculty of Science for Women( SCIW), University of Babylon, Iraq

2 Outlines Motivation Master Method Some of Examples Proof I Interpretation of the 3 Cases Proof II Quickly Sort Analysis Example of best Performance Example of average Performance Example of worst Performance 19 March 2017

3 Motivation Integer Multiplication Revisited A Recursive Algorithm

4 Motivation A Better Recursive Algorithm

5 Motivation Which recurrence best describes the running time of the Gauss' algorithm for integer multiplication? T n 2T(n/2) + O(n 2 ) 3T(n/2) + O(n) 4T(n/2) + O(n) 4T(n/2) + O(n 2 ) Master Method Correct answer

6 Recurrence Format

7 The Master Method If T n at n b + O(nd ) then O n d log n if a = b d (Case 1) T n = O(n d ) if a < b d (Case 2) O(n log b a ) if a > b d (Case 3)

8 Example #1 Example #2: Where are the respective values of a, b, d for a binary search of a sorted array, and which case of the Master Method does this correspond to? 1, 2, 0 [Case 1] 1, 2, 1 [Case 2] 2, 2, 0 [Case 3] 2, 2, 1 [Case 1]

9 Example #3 Example #4: Where are the respective values of a, b, d for Gauss s recursive integer multiplication algorithm, and which case of the Master Method does this correspond to? 2, 2, 1 [Case 1] 3, 2, 1 [Case 1] 3, 2, 1 [Case 2] 3, 2, 1 [Case 3]

10 Example #5: Example #6:

11 Proof I If T n at n b + O(nd ) then O n d log n if a = b d (Case 1) T n = O(n d ) if a < b d (Case 2) O(n log b a ) if a > b d (Case 3) Preamble

12 Proof I What is the pattern? Fill in the blanks in the following statement: at each level j=0,1,2,, log b n, there are <blank> subproblems, each of size <blank>. a j and n/a j, respectively. a j and n/b j, respectively. b j and n/a j, respectively. b j and n/b j, respectively. The Recursion Tree

13 Work at a Single Level Total Work

14 How To Think About (*) Our upper bound on the work at level j: cn d ( a b d)j

15 How To Think About (*) Which of the following statements are true? (Check all that apply.) If RSP < RWS, then the amount of work is decreasing with the recursion level j. If RSP > RWS, then the amount of work is increasing with the recursion level j. No conclusions can be drawn about how the amount of work varies with the recursion level j unless RSP and RWS are equal. If RSP and RWS are equal, then the amount of work is the same at every recursion level j.

16 Intuition for the 3 Cases

17 The Story So Far/Case 1 Basic Sums Fact 19 March 2017

18 Case 2 19 March 2017

19 Case 3

20 The Master Method The Master Method If T n at n b + O(nd ) then O n d log n if a = b d (Case 1) T n = O(n d ) if a < b d (Case 2) O(n log b a ) if a > b d (Case 3) 19 March 2017

21 Quicksort Like merge sort, quicksort uses divide-and-conquer, and so it's a recursive algorithm. The way that quicksort uses divide-and-conquer is a little different from how merge sort does. In merge sort, the divide step does hardly anything, and all the real work happens in the combine step. Quicksort is the opposite: all the real work happens in the divide step. In fact, the combine step in quicksort does absolutely nothing. Quicksort has a couple of other differences from merge sort. Quicksort works in place. And its worst-case running time is as bad as selection sort's and insertion sort's: \Theta(n^2)Θ(n2 ). But its average-case running time is as good as merge sort's: \Theta(n \lg n)θ(nlgn). So why think about quicksort when merge sort is at least as good? That's because the constant factor hidden in the big-θ notation for quicksort is quite good. In practice, quicksort outperforms merge sort, and it significantly outperforms selection sort and insertion sort. 19 March 2017

22 Quicksort Here is how quicksort uses divide-and-conquer. As with merge sort, think of sorting a subarray array[p..r], where initially the subarray is array[0..n-1]. 1. Divide by choosing any element in the subarray array[p..r]. Call this element the pivot. Rearrange the elements in array[p..r] so that all other elements in array[p..r] that are less than or equal to the pivot are to its left and all elements in array[p..r] are to the pivot's right. We call this procedure partitioning. At this point, it doesn't matter what order the elements to the left of the pivot are in relative to each other, and the same holds for the elements to the right of the pivot. We just care that each element is somewhere on the correct side of the pivot.as a matter of practice, we'll always choose the rightmost element in the subarray, array[r], as the pivot. So, for example, if the subarray consists of [9, 7, 5, 11, 12, 2, 14, 3, 10, 6], then we choose 6 as the pivot. After partitioning, the subarray might look like [5, 2, 3, 6, 12, 7, 14, 9, 10, 11]. Let qbe the index of where the pivot ends up. 2. Conquer by recursively sorting the subarrays array[p..q-1] (all elements to the left of the pivot, which must be less than or equal to the pivot) and array[q+1..r] (all elements to the right of the pivot, which must be greater than the pivot). 3. Combine by doing nothing. Once the conquer step recursively sorts, we are done. Why? All elements to the left of the pivot, in array[p..q-1], are less than or equal to the pivot and are sorted, and all elements to the right of the pivot, in array[q+1..r], are greater than the pivot and are sorted. The elements in array[p..r] can't help but be sorted! 19 March 2017

23 Quicksort Think about our example. After recursively sorting the subarrays to the left and right of the pivot, the subarray to the left of the pivot is [2, 3, 5], and the subarray to the right of the pivot is [7, 9, 10, 11, 12, 14]. So the subarray has [2, 3, 5], followed by 6, followed by [7, 9, 10, 11, 12, 14]. The subarray is sorted. The base cases are subarrays of fewer than two elements, just as in merge sort. In merge sort, you never see a subarray with no elements, but you can in quicksort, if the other elements in the subarray are all less than the pivot or all greater than the pivot. Let's go back to the conquer step and walk through the recursive sorting of the subarrays. After the first partition, we have have subarrays of [5, 2, 3] and [12, 7, 14, 9, 10, 11], with 6 as the pivot. To sort the subarray [5, 2, 3], we choose 3 as the pivot. After partitioning, we have [2, 3, 5]. The subarray [2], to the left of the pivot, is a base case when we recurse, as is the subarray [5], to the right of the pivot. To sort the subarray [12, 7, 14, 9, 10, 11], we choose 11 as the pivot, resulting in [7, 9, 10] to the left of the pivot and [14, 12] to the right. After these subarrays are sorted, we have [7, 9, 10], followed by 11, followed by [12, 14]. Here is how the entire quicksort algorithm unfolds. Array locations in blue have been pivots in previous recursive calls, and so the values in these locations will not be examined or moved again: 19 March 2017

24 Quicksort Algorithm Worst-case performance O(n 2 ) Best-case performance Average performance O(n log n) O(n log n) (simple partition) or O(n) (three-way partition and equal keys) 19 March 2017

25 Two Examples of best & average performance of Quicksort For best case scenario.. I came up with (bold for pivot, italic for bigger and smaller than markers): March 2017

26 Example of worst performance of Quicksort Full example of quicksort on a random set of numbers. The shaded element is the pivot. It is always chosen as the last element of the partition. However, always choosing the last element in the partition as the pivot in this way results in poor performance (O(n²)) on already sorted arrays, or arrays of identical elements. Since subarrays of sorted / identical elements crop up a lot towards the end of a sorting procedure on a large set, versions of the quicksort algorithm that choose the pivot as the middle element run much more quickly than the algorithm described in this diagram on large sets of numbers. 19 March 2017