Analysis of Algorithms - Medians and order statistic -

Transcription

1 Analysis of Algorithms - Medians and order statistic - Andreas Ermedahl MRTC (Mälardalens Real-Time Reseach Center) andreas.ermedahl@mdh.se Autumn 2003 Order statistics Select the i:th smallest element of n elements, that is the element with rank i. i = 1: minimum i = n: maximum i = (n + 1)/2 or (n + 1)/2 : median Naive algorithm: Worst-case running time = Θ(n lg n) We can do better!!! Order statistics: min Special case: find minimum (or maximum) Θ(n) if we have no information about the ordering of the elements in the set All elements must be checked The simple, naive algorithm one immediately comes up with is the best: 1

2 Order statistics: min & max A twist: compute simultaneous minimum and maximum Still a Θ(n)-problem Simple solution: Use previous algorithm twice (one modified that computes max) Requires 2n - 2 comparisons We can lower the leading constant somewhat Idea: for each two new elements A[i], A[i+1] from input set do 1. Compare A[i] and A[i + 1] 2. Compare smallest with min 3. Compare biggest with max Uses 3 comparisons for every 2 elements ï in total 3 n/2 comparisons Interesting if comparisons are expensive Selection in expected linear time Problem: How to find the i:th order statistics (not only max or min) in O(n) expected time Idea: Use the (randomized) partitioning routine from (randomized) QUICKSORT Keep track of the size of the partitions When a partition is made, we know in which partition the i:th element is Apply procedure recursively on this partition until termination Randomized divide-and-conquer We want to find the i:th smallest element among elements A[p..r] Partition A[p..r] into two subarrays A[p..q-1] and A[q+1..r] Element A[q] is the k:th smallest element in A[p..r] If i < k then find the i:th smallest element in A[p..q-1] If i > k then find the i- k:th smallest element in A[q+1..r] 2

3 Example: randomized divide-and-conquer Analysis of RAND-SELECT Lucky: T(n) = T(9n/10) + Θ(n) = Θ(n) By case 3 of the Master method Most of the cases (see similarity to analysis of Quicksort!) Unlucky: T(n) = T(n - 1) + Θ(n) = Θ(n 2 ) Happens when we make really bad partitioning all the time (see similarity to worst-case analysis of Quicksort) This is worse than Naive-Select! Summary: Linear expected time. Excellent algorithm in practice. But worst case is very bad. Imagine spending Θ(n 2 ) time to find the minimum element :-( Discussion Exercise Suppose we use RANDOMIZED-SELECT to select the minimum element of the array A = 3, 2, 9, 0, 7, 5, 4, 8, 6, 1 Ú. Describe a sequence of partitions that results in a worst-case performance of RANDOMIZED-SELECT 3

4 Selection in Worst-Case Linear Time Algorithm due to Blum, Floyd, Pratt, Rivest, and Tarjan [1973]. Idea: somehow guarantee a good split SELECT(i, n) 1. Divide the n elements into groups of Find the median of each 5-element group. 3. Use SELECT recursively to find the median x of the n/5 group medians to be the pivot. 4. Partition around the pivot x. 5. Let k = rank(x). 6. If i = k, then return x. 7. If i < k, then recursively call SELECT to find the i:th smallest element in first part. 8. If i > k, then recursively call SELECT to find the (i - k):th smallest element in last part. We have n input elements Each represented by a green circle We want to find the i:th element by choosing a good pivot element x Step 1: Divide the n elements in groups of 5 Each group occupies a column 4

5 Step 2: Find the median of each 5- group. element Step 3: Use SELECT recursively to find the median x of all the n/5 group medians Set x to be the pivot. At least half the group medians are x, which is at least n/5 /2 = n/10 group medians. 5

6 At least half the group medians are x, which is at least n/5 /2 = n/10 group medians. Therefore, at least 3 n/10 elements are x (assuming all elements are distinct) Similarly, at least 3 n/10 elements are x. Minor simplification For n 50, we have 3 n/10 n/4 Therefore, for n 50 the recursive call to SELECT in step 7 8 is executed recursively on 3n/4 elements Thus, the recurrence for running time can assume that step 7 8 takes time T(3n/4) in the worst case For n < 50, we know that the worst- case time is T(n) = Θ(1) 6

7 Analysis of SELECT SELECT(i, n) 1. Divide the n elements into groups of Find the median of each 5-element group. 3. Use SELECT recursively to find the median x of the n/5 group medians to be the pivot. 4. Partition around the pivot x. 5. Let k = rank(x). 6. If i = k, then return x. 7. If i < k, then recursively call SELECT to find the i:th smallest element in first part. 8. If i > k, then recursively call SELECT to find the (i - k):th smallest element in last part. Θ(n) T(n/5) Θ(n) T(3n/4) T(3n/4) Solving the recurrence The resulting recurrence: We show T(n) = O(n) with substitution: T(n) cn: if c is chosen large enough to handle both the Θ(n) and the initial conditions. Conclusions Since the work at each level of recursion is a constant fraction (19/20) smaller, the work per level is a geometric series dominated by the linear work at the root In practice, this algorithm runs slowly because the constant in front of n is large The randomized algorithm is far more practical 7

8 Exercise on the white-board Exercise In the algorithm SELECT, the input elements are divided into groups of 5. Will the algorithm work in linera time if they are divided into groups of 7? Argue that SELECT does not run in linear time if groups of 3 are used. 8