Quicksort. Part 3: Analyzing the Running Time

Transcription

1 Quicksort Part 3: Analyzing the Running Time

2 Choosing a Pivot What is Quicksort s running time? (can we use master theorem?) It depends on the pivot What is the worst case for Quicksort, and what is its running time? Always select the smallest (or largest) possible pivot and it takes O(n 2 ) Think of a one-sided tree What is the best case for Quicksort, and what is its running time? Always select the median element as a pivot leading to O(n lg n) Think of a balanced tree

3 Draw a recursion tree for the best and worst cases of Quicksort n n n-1 n-1 2 n-1 2 n-2 n-2 4 n-2 4 n-2 4 n-2 4 n

4 Draw a recursion tree for the best and worst cases of Quicksort Let s assume the cost of Partition is 5m (n-1) = 35 5n = 40 5n = (n-1) = (n-2) = (n-3) = (n-3) = (n-7) = (n-4) = 20 T(n) = (n-5) = 15 T(n) = (n-6) = (n-7) = 5

5 How would you select a pivot? If pivot selection is so important, how should we do it? Shouldn t we take great care in selecting the pivot? Key idea for Quicksort: select the pivot uniformly at random! Easy Fast Gets good results as long as the pivots are decent fairly often

6 Random Pivots Some foreshadowing: If we get something close to the median (in middle % range) we will get an average running time of O(n lg n) This is not easy to prove as you cannot use the master theorem We are going to show the runtime of quicksort another way

7 Quicksort Theory For every input of the array of length n, the average running time of quicksort with random pivots is O(n lg n). This is a big deal, it means that the average running time is closer to the bestcase than it is to the worst-case. Note: here, average refers to the algorithm itself it does not depend on the input. If we re-run quicksort on the same input we will get different pivots each time, and we are talking about the average running time of quicksort for these different sequences of pivots on the same input array.

8 Quicksort Where is the work being done? What can we count inside partition that would represent the total running time? QUICKSORT (A, left_index, right_index) if left_index >= right_index return PARTITION (A, left_index, right_index) pivot = A[left_index] i = left_index + 1 // Can be done in different ways // Needs to be at most a O(n) procedure, // but O(1) methods work very well MOVE-PIVOT-TO-LEFT (A) pivot_index = PARTITION (A, left_index, right_index) QUICKSORT (A, left_index, pivot_index 1) QUICKSORT (A, pivot_index + 1, right_index) // Up to and including right_index for j = (left_index + 1).. right_index if A[j] < pivot swap A[j] and A[i] i += 1 swap A[left_index] and A[i - 1] return i - 1

9 Counting the total number of comparisons For a fixed input array of length n The sample space (Ω) for quicksort includes all possible sequences of pivots (SOP) The cost of a given SOP is directly proportional to the total number of comparisons required: C(SOP) = total # of comparisons Our goal is to show that:! " #$% = $(( lg () The expected value for the cost of a given sequence of pivots is O(n lg n)

10 Some notation Let Z i = i th smallest element of A (not the i th element)

11 Z i Z 5 Z 4 Z 1 Z 8 Z 7 Z 2 Z 6 Z

12 Some notation Let Z i = i th smallest element of A (not the i th element) Let X ij (SOP) be a random variable for the number of times Z i and Z j get compared during a call to quicksort with a given SOP How many times can Z i and Z j possibly be compared? Can only be compared 0 or 1 times! Every comparison involves the pivot, but the pivot is excluded from recursive calls.

13 Counting the total number of comparisons Let Z i = i th smallest element of A (not the i th element) Let X ij (SOP) be a random variable for the number of times Z i and Z j get compared during a call to quicksort with a given SOP X ij is an indicator variable (it shows if a comparison happened or if it did not)

14 Counting the total number of comparisons Given a sequence of pivots (SOP) what is total number of comparisons? BCA B C SOP = > > F?D (GHI)?@A D@?EA J K GHI = H(N lg N) B CA B J K = > > J[F?D ] Linearity of expectations?@a D@?EA

15 Counting the total number of comparisons *+) C SOP = & '() * &,('-). ', (012) * +) 4 5 = & * & 4[. ', ] Pr. ', = Pr. ', = 0 0 '(),('-) Pr(. ', = 1)

16 Probability that Z i, Z j get compared Pr($ %& = 1) Consider any Z i, Z i+1,, Z j-1, Z j from the array Remember that these are not contiguous in the array, they are a numbers in increasing order What can you tell me about this group of numbers? (Hint: consider different values for the pivot element) As long as none of these are chosen as a pivot, all are passed to the same recursive call.

17 Z i, Z j Z 5 Z 4 Z 1 Z 8 Z 7 Z 2 Z 6 Z What is the probability that Z 3 (37) and Z 7 (79) are compared?

18 Probability that Z i, Z j get compared Pr($ %& = 1) Consider any Z i, Z i+1,, Z j-1, Z j from the array Among these values, consider the first one that gets chosen 1. If Z i or Z j are chosen first, then Z i and Z j are compared. 2. If one of Z i+1,, Z j-1 is chosen, then Z i and Z j are NEVER compared. Why?

19 Probability that Z i, Z j get compared Pr # $% = 1 = 2 )*)+, # *. /h*1/23 = What does this mean for two values that are close to each other? What does this mean for two values that are far from each other?

20 Z i, Z j Z 5 Z 4 Z 1 Z 8 Z 7 Z 2 Z 6 Z What is the probability that Z 3 (37) and Z 7 (79) are compared?

21 Probability that Z i, Z j get compared Pr # $% = 1 = 2 )*)+, # *. /h*1/23 = < =; < <=; < 7 8 = 9 9 Pr # $% = 1 = 9 9 $:; %:$>; $:; %:$>; < @ 9 C 2@ D A:B ; < 1 E FE = 2@ ln = ln 1 ) = 2@ln(@) 1

22 Probability that Z i, Z j get compared * 1! " 2% &, 2% - '(). * 1 / 0/ = 2% ln / % 1 = 2%(ln % ln 1 ) = 2%ln(%)! " 2%ln(%)! " = 7(% lg %)! " 97: = 7(% lg %) What does this really mean?![< % =>?@'ABCD ] = 7(% lg %)