CS240 Fall Mike Lam, Professor. Quick Sort

Transcription

1 ??!!!!! CS240 Fall 2015 Mike Lam, Professor Quick Sort

2 Merge Sort Merge sort Sort sublists (divide & conquer) Merge sorted sublists (combine) All the "hard work" is done after recursing Hard to do "in-place" The sublists need to be interleaved during merging Doing this cleanly requires O(n) extra space at minimum We'd like an O(n log n) algorithm that works "in-place" No extra space required

3 Quick Sort Quick sort Choose a pivot value Partition into sublists (divide) Sort sublists (conquer) Merge sorted sublists (combine) All the "hard work" is done before recursing O(n) at each level Some work (combining) is done after recursing Still O(n) at each level This is actually unnecessary in the in-place version

4 Partitioning Choose a pivot value Easy choices: first, middle, or last More complicated: random, median of three Split list into two or three sublists 1) Less than and 2) Equal or Greater than 1) Less than, 2) Equal to, and 3) Greater than This operation can be done in-place or with auxiliary lists Often implemented in a separate function Like the merge() operation in merge sort

5 Quick Sort Implementation def quick_sort(items): n = len(items) # base case: 0 or 1 items (already sorted) if n < 2: return # choose pivot pivot = items[0] # use first item # divide (a.k.a. partition) less = []; equal = []; greater = [] for elem in items: if elem < pivot: less.append(elem) elif elem > pivot: greater.append(elem) else: equal.append(elem) # conquer (recurse) quick_sort(less) quick_sort(greater) # combine i = 0 for elem in less: items[i++] = elem for elem in equal: items[i++] = elem for elem in greater: items[i++] = elem

6 Quick Sort Implementation Good: Relatively simple and easy to understand Bad: Uses lots of extra lists (similar to merge sort) Alternative: "in-place" implementation Instead of building new lists during partitioning, use swapping to re-arrange sublists in the original list This can be a little difficult to get exactly right It's worth practicing we'll do this in a lab!

7 In-place Quick Sort

8 In-place Quick Sort i i+n

9 In-place Quick Sort i i+n p q (Can also be done in C without i use pointers! This also means you don't need a recursive helper.)

10 Quick Sort Analysis O(n) time per level How many levels?? levels (this is the key difference between analysis of merge sort and analysis of quick sort) O(n) time

11 Quick Sort Analysis O(n) time per level How many levels? Best case: ~ log 2 n ~ log n levels Input size is halved each time T(n) = 2T(n/2) + n Overall: O(n log n) Worst case: ~ n O(n) time Input size decreases by O(1) each time T(n) = T(n-1) + T(1) + n Overall: O(n 2) Average/expected: ~ log 4/3 n Collapse T(n-1) case into T(n/2) case T(n) T(n-1) T(n) = 2T((n-1)/2) + T(1) + 2n Overall: O(n log n) T(n/2) T(n/2)

12 Quick Sort Analysis O(n) time per level How many levels? Best case: ~ log 2 n ~ log n levels Input size is halved each time T(n) = 2T(n/2) + n Overall: O(n log n) Worst case: ~ n O(n) time Input size decreases by O(1) each time T(n) = T(n-1) + T(1) + n Overall: O(n 2) Average/expected: ~ log 4/3 n Collapse T(n-1) case into T(n/2) case T(n) T(n-1) T(n) = 2T((n-1)/2) + T(1) + 2n Overall: O(n log n) T(n/2) T(n/2)

13 Quick Sort Analysis O(n) time per level How many levels? Best case: ~ log 2 n ~ log n levels Input size is halved each time T(n) = 2T(n/2) + n Overall: O(n log n) Worst case: ~ n O(n) time Input size decreases by O(1) each time T(n) = T(n-1) + T(1) + n Overall: O(n 2) T(n) Average/expected: ~ log 4/3 n Collapse T(n-1) case into T(n/2) case T(n) = 2T((n-1)/2) + T(1) + 2n Overall: O(n log n) T((n-1)/2) T((n-1)/2)

14 Pivots Choice of pivot is extremely important! Determines the size of the two sublists And therefore (indirectly) the recursion depth Optimal: median of all values in the list Sublists will be of equal length Guarantees O(n log n) sort (just like merge sort) Chicken-and-egg problem: calculating the median requires the list to be sorted!

15 Pivots Choice of pivot is important! Non-optimal: deterministic selection Choose first item, middle item, or last item Picking the last item simplifies some implementations Picking the middle item works well for most lists All three have pathological cases that are O(n 2) Picking the first or last is particularly problematic because the pathological case is a sorted list! Other options: random or median-of-three Random guarantees O(n log n) with high probability Median-of-three is often cheaper to compute and similar in practice

16 Stability Quick sort is not stable Partition operation re-orders items within sublists Stable variant requires O(n) extra space This erases the largest advantage of quick sort over merge sort

17 Conclusions Quick sort is often the fastest comparative sort in practice Expected O(n log n) running time in most cases Requires no extra space Except for log n stack frames for recursion Watch out for pathological cases! Often can be avoided easily enough in practice Many common tweaks to improve quick sort Random or median-of-three pivot selection Switch to a different sort for pathological cases (introsort) Switch to a different sort when n is small

18 Conclusions Merge sort O(n log n) worst-case time Not in-place Stable Variants (e.g., Timsort) are widely used Quick sort O(n log n) average/expected time In-place Not stable Variants (e.g., introsort) are widely used

19 Minimum Worst Case Question: "Can we sort faster than O(n log n)?"

20 Minimum Worst Case Question: "Can we sort faster than O(n log n)?" Not if we're using comparisons! Lower bound on worst-case comparison-based sorting: Ω(n log n) Justification involves a binary decision tree Each node represents the result of a comparison Each leaf node (or path through the tree) represents a possible permutation of the original list

21 Minimum Worst Case Question: "Can we sort faster than O(n log n)?" Not if we're using comparisons! Lower bound on worst-case comparison-based sorting: Ω(n log n) Justification involves a binary decision tree Each node represents the result of a comparison Each leaf node (or path through the tree) represents a possible permutation of the original list there are n! such leaves Height of the list is at least log(n!) (n/2)log(n/2)

22 Minimum Worst Case Question: "Can we sort faster than O(n log n)?" Kind of, if we're not using comparisons How do you sort without comparing items directly? Multiple cycles of splitting items into bins Preserve ordering within bins Need to make restrictions on item domains Examples: Integer numbers < 10,000 Five-letter character strings

23 Non-Comparative Sorting Bucket "sort" (not really a sort in the general sense) Create N buckets Separate all elements into buckets: O(n) Concatenate all buckets: O(N) Requires some knowledge about the domain to be efficient Goal: items are evenly distributed across all buckets Stable when implemented carefully "Sorts" items by a single digit/character Running time: O(n + N)

24 Non-Comparative Sorting Radix sort Represent elements as ordered tuples (e.g., array of digits or characters) With O(1) access to elements by index Perform repeated bucket sorts One sort per index Start with least-significant index Running time is O(d(n+N)) d is the dimensionality of the input domain (i.e., number of bits or digits or characters) Observation: d log n Because it takes ceil(log n) bits to represent n unique items! So this is effectively just O((n+N) log n) Still, it's nice that we can sort without doing comparisons

25 Sorting Visualizations

26 Sort Algorithm Comparison Big-O Analyses Worst Case Comparisons Worst Case Assignments Worst Case Time Best Case Time Average / Expected Time In Place? Stable? Selection Sort Insertion Sort Merge Sort Quicksort Heapsort