Analysis of Algorithms - Quicksort -

Transcription

1 Analysis of Algorithms - Quicksort - Andreas Ermedahl MRTC (Mälardalens Real-Time Reseach Center) andreas.ermedahl@mdh.se Autumn 2003 Quicksort Proposed by C.A.R. Hoare in 962. Divide- and- conquer algorithm Sorts in place Like insertion sort, but not like merge sort. Very efficient in practice. Few instructions in inner loop. Rather difficult to code Quicksort Divide-and-conquer algorithm Quicksort an n-element array:. Divide: Partition the array into two subarrays around a pivot element x. All elements x All elements x Pivot element 2. Conquer: Recursively sort the two subarrays. 3. Combine: Not needed. Key for performance: Linear-time partitioning subroutine

2 Quicksort Quicksort is very easily defined in a functional language, like Erlang: Base: empty list, just return an empty list quicksort([]) -> []; all elements in L quicksort([pivot L]) -> quicksort([y Y <- L, Y < Pivot]) ++ [Pivot] ++ quicksort([z Z <- L, Z >= Pivot]). Note that recursive calls are on strictly shorter lists, ensures termination This is a very good specification. However, it is not maximally efficient, since: it operates on lists, and it builds new lists for each recursive call Rest: nonempty list Returns sorted list of smaller than Pivot Returns sorted list of all elements in L larger than or equal to Pivot Quicksort A more efficient version of quicksort uses array A and sorts in-place: A call to quicksort(a, p, r) sorts A[p... r] in-place This quicksort is a recursive procedure that destructively updates A as a side-effect of the call Since we always sort distinct segments of A this is OK Initial call: quicksort(a,, length[a]) Partitioning subroutine partition(a, p, r) sends all elements < A[p] to the left, all elements A[p] to the right, and returns the new position of rightmost element < A[p]: Pivot = x = A[r] Loops through n- elements: O(n) 2

3 Example of partitioning Example of partitioning Pivot element Elements Pivot Elements Pivot Analysis of Quicksort Assume all input elements are distinct. There are better ways to partition when input has duplicates. Let T(n) be worst-case running time on an array of n elements. Cost: Cost for sorting n numbers, T(n) = cost for partitioning + cost for recursive calls Question: how many recursive calls are made? 3

4 Worst-Case of Quicksort Occurs when input sorted or reverse sorted Produces unbalanced partitioning Partition around min or max element One side of partition always has no elements. Time to sort n elements Time to sort elements Time to sort n- elements Time to partition n elements Best case of Quicksort Occurs when PARTITION always split array evenly Always perfectly balanced partitioning Time to sort n elements Cost for partitioning n elements Two new subproblems n/2 size Master-theorem (case 2) directly gives us: Average case analysis Actually: Quicksort is T(n) = Θ(n log n) also in average case Average-case running time more close to best-case than to worst-case To understand why, we need to understand:. How the balance of the partitioning is reflected in the recurrence 2. How likely a certain partitioning is relative other partitionings 4

5 Constant split analysis What if we always get a certain partitioning of elements? For example, assume we always get a ( / 0) and (9 / 0) split in the partitioning Result in recurrence: Asymptotic bound for this recurrence? Can be derived by using the recurrence tree method 9-to- Split Recursion-tree cost n cn log 0/9 n log 0 n n/0 9n/0 n/00 9n/00 9n/00 8n/00 cn cn Cost for each level is cn for some constant c Shortest recursion branch terminates at depth log 0 n 8n/ n/000 cn cn cn Cost for each level is cn Deepest recursion branch terminates at depth log 0/9 n Total cost: cost per tree- level * number of tree- levels cn * log 0/9 n Constant split analysis We got that the total cost: T(n) cn log 0/9 n This gives that: Even in a seemingly unbalanced case! Actually, any split of constant proportionality yields: Recursion tree of depth Θ(lg n) With a O(n) cost of each level Conclusion: Running time is O(n lg n) whenever the split has constant proportionality 5

6 Mixed type of splits What if the partitioning creates different splits in different steps? We expect some splits to be reasonably balanced and some splits fairly unbalanced Consider an unbalanced split followed by a balanced split: then Thus, balanced splits seems to annihilate the effects of earlier unbalanced splits Likely that average cost is Θ(n log n) Gives that T(n) = O(n log n) Randomized Quicksort Problem: Not always true that all input sequences are equally likely For example, already almost sorted sequences might occur often is certain applications Idea: Partition around a random element. Running time is independent of the input order. No assumptions need to be made about the input distribution. No specific input triggers the worst-case behavior. The worst case is determined only by the random-number generator. Analysing Randomized Quicksort will help us understand average behavior of original Quicksort Randomized Quicksort Chooses randomly a pivot- element from subarray A[p..r] Equally likely that pivot is any of the r- p+ elements Main algorithm very similar to original Quicksort 6

7 Proving worst-case O(n 2 ) bound Worst-case: Parameter q ranges from 0 to n- We use substitution method to show that T(n)=O(n 2 ) We guess that T(n) cn 2 for some constant c Substituting into recurrence yields: Maximum is obtained for q= or q=n- We obtain: Giving that T(n) = O(n 2 ) Average case analysis Observation for avarage case analysis Running time of Quicksort dominated by time spent in partition procedure First, an upper bound on the number of partition calls Each call to partition results in a pivot element gets selected This element can never be included in future recursive calls of Quicksort Thus, we can have at most n calls to partition Second, an upper bound on the total number of comparasions One call to partition takes O() time + amount of time proportional to number of pivot comparasions Most time of algorithm spent in comparing elements Thus, a total upper bound for number of comparasions gives an upper bound for the running time of Quicksort. Average case analysis We must understand when the algorithm compares two elements and when it does not Rename elements in A as z, z 2,..., z n where z i is the i:th smallest element in array A Let Z ij = {z i, z i+,..., z j } The set of elements between z i and z j (inclusive) The total number of comparasions made = n n X = X ij i= j= i+ where X ij is the number of times z i is compared to z j 7

8 Example Assume A is array contaning elements,2...0 The particular input order is not given Then Z 2 9 = {2,3,4,5,6,7,8,9} The set of elements between 2 and 9 Total number of comparasions made: X = 0 0 i= j= i+ comparasions between element i and j The actual number of comparasions made depends on the order of the particular input sequence Average case analysis Value of X depends on probability of comparing different elements: Probability of comparing z n n i and z j Expected value of X E ( X ) = Pr{ z i is compared to z j } i= j= i+ A comparasion is only made when one of the elements is the pivot element Two elements z i and z j can be compared at most once Investigating set Z ij = {z i, z i+,..., z j } in more detail we note that If first pivot element selected from Z ij is x such that z i < x < z j then z i and z j wili never be compared If first pivot element selected from Z ij is z i or z j then z i and z j will be compared Example Z 2 9 = {2,3,4,5,6,7,8,9} Element 2 and 9 will only be compared if 2 or 9 is one of the first two elements selected as pivot from the Z 2 9 set Assume 4 is element first selected as pivot Gives that set Z 2 3 = {2,3} will be sorted in isolation from Z 5 9 = {5,6,7,8,9} Element 2 and 9 will never be compared 8

9 Average case analysis This gives us that: Pr{z i is compared to z j } = Pr{z i or z j is first pivot element chosen from Z ij } Set Z ij contains j i + elements Each element is chosen randomly and independently Probability to choose a given element is therefore: / (j i + ) This gives us that: Pr{z i is compared to z j } = 2 * ( / (j i + )) Example Z 2 9 = {2,3,4,5,6,7,8,9} Probability to chose an element from Z 2 9 is: / (9 2 + ) = / 8 We therefore get: Pr{element 2 is compared to element 9} = Pr{element 2 or element 9 is first selected from Z 2 9 } = 2 * ( / 8) = 2 / 8 = / 4 Average case analysis To summarize we get: Expected value of X Set variable k = j - i A.7: Harmonic series rule Upper bound for average case E( X ) = = = < = n i= j= i+ n i= j= i+ n i= k = n i= k = n i= n i n n n 2 k + 2 k O(lg n) = O( n lg n) Pr{ z is compared to z } 2 j i + i j 9

10 Running time of Quicksort Summary: Best Case: Θ(n log n) Average Case: Θ(n log n) Worst Case: Θ(n 2 ) Lower avarage bound: Ω(n log n) should also be shown! Quicksort in practice Quicksort is a great general- purpose sorting algorithm. Quicksort is typically over twice as fast as Mergesort. Quicksort behaves well even with caching and virtual memory We can make smaller modifications to enhance performance further Modifications of Quicksort Note: Costly to perform quicksort on small inputs Idea: Take advantage of the fact of fast running time of insertion-sort for almost sorted input Modifications: When quicksort is called with fewer than k elements, let it return without sorting subarray After top-level call to quicksort returns, run insertion sort on entire array to finish sorting Sorting algorithm runs in O(nk + n lg(n/k)) See exercise

11 Modifications of Quicksort Note: Quicksort behavior is rather sensitive to unbalanced partitionings Idea: Somehow reduce probability that a bad pivot element is selected Modification: (Randomly) select three different elements from subarray instead of just one element Let the median of the three elements be the pivot Gets the same asymptotic bounds as original Quicksort, but with smaller constants Modifications of Quicksort Note: Quicksort contains two recursive calls to itself Forces us to save a lot of function contexts on stack Idea: Rewrite algorithm to be tail- recursive Removes the need for saving function context on the stack Smart compilers might sometimes help you do this See 7-4for a tail- recursive version of Quicksort