Lower bound for comparison-based sorting

Transcription

1 COMP3600/6466 Algorithms 2018 Lecture 8 1 Lower bound for comparison-based sorting Reading: Cormen et al, Section 8.1 and 8.2 Different sorting algorithms may have different running-time complexities. So how is it possible to know whether the running time of an algorithm is the best possible (at least asymptotically)? We know of several sorting algorithms, e.g., merge sort and heapsort (next lecture), that take time O(n log n) in the worst case to sort an array of n elements. Can this be improved? All the sorting algorithms we have considered so far have the property that the sorted order they determine is based only on comparisons between input elements. Such algorithms are called comparisons sorts. 1

2 COMP3600/6466 Algorithms 2018 Lecture 8 2 Comparison sorts In a comparison sort, we use only comparisons between elements to acquire order information about an input sequence a 1, a 2,..., a n. That is, we only use tests of the form a j < a j, a i a j, a i = a j, a i a j, or a i > a j. Make the simplifying assumption that all input elements are distinct. Then tests of the form a i = a j are useless and comparisons of the form a j < a j, a i a j, a i a j, and a i > a j are equivalent in that they provide identical information about the relative order of a i and a j. Thus we make the assumption that all comparisons have the form a i a j. 2

3 COMP3600/6466 Algorithms 2018 Lecture 8 3 Decision-tree model Comparison sorts can be viewed abstractly in term of decision trees. A decision tree is a full binary tree that represents the comparisons between elements that are performed by a sorting algorithm. (A full binary tree is one that has the property that each node is either a leaf node or has degree exactly 2.) Control, data movement, and all other aspects of the algorithm are ignored. 3

4 COMP3600/6466 Algorithms 2018 Lecture 8 4 Decision-tree model 2 Cormen et al, p.192 The input is an array of 3 elements: a 1, a 2, a 3. 4

5 COMP3600/6466 Algorithms 2018 Lecture 8 5 Decision-tree model 3 a1=15, a2=19, a3=14 < = 1:2 > < = a1=17, a2=12, a3=25 1:2 > 2:3 > < = 1:3 > 1:3 <2,1,3> 2:3 < = > 12, 17, 25 <3,1,2> 14, 15, 19 5

6 COMP3600/6466 Algorithms 2018 Lecture 8 6 Decision-tree model 4 Let n be the number of elements in the input array. Each internal node is annotated by i : j, for some i and j such that 1 i, j n. Each leaf node is annotated by a permutation π(1), π(2),..., π(n). (A permutation π is a bijection from {1,..., n} to {1,..., n}.) The execution of the sorting algorithm corresponds to tracing a path of the decision tree from the root down to a leaf node. Each internal node indicates a comparison a i a j. The left subtree then dictates subsequent comparisons if a i a j and the right subtree dictates subsequent comparisons if a i > a j. 6

7 COMP3600/6466 Algorithms 2018 Lecture 8 7 Decision-tree model 5 At a leaf, the sorting algorithm has established the ordering a π(1) a π(2) a π(n). Each (correct) sorting algorithm must be able to produce each permutation of its input. Hence all n! permutations on n elements must appear as one of the leaf nodes of the decision tree. Furthermore, each of these leaves must be reachable from the root by a downward path corresponding to an actual execution of the comparison sort. Thus we consider only decision trees in which each permutation appears as such a reachable leaf. 7

8 COMP3600/6466 Algorithms 2018 Lecture 8 8 Lower bound for the worst case The length of the longest path from the root of a decision tree to any of its reachable nodes is the worst-case number of comparisons that the corresponding comparison sort executes. That is, the worst-case number of comparisons for a given comparison sort algorithm equals the height of its decision tree. A lower bound on the height of all decision trees in which each permutation appears as a reachable leaf node is therefore a lower bound on the running time of any comparison sort algorithm. 8

9 COMP3600/6466 Algorithms 2018 Lecture 8 9 Lower bound for the worst case 2 Theorem: Any comparison sort algorithm requires Ω(n lg n) comparisons in the worst case. To prove the theorem, it suffices to determine height of a decision tree for which each permutation appears as a reachable leaf node. So consider a decision tree of height h with l reachable leaf nodes corresponding to a comparison sort on n elements. Because each of the n! permutations of the input array appears as some leaf, we have n! l. Since a binary tree of height h has no more that 2 h leaves, we have n! l 2 h. Thus h lg(n!) [lg is monotonically increasing] = Ω(n lg n) [lg(n!) = Θ(n lg n)] 9

10 COMP3600/6466 Algorithms 2018 Lecture 8 10 Asymptotically optimal algorithm Let P be a problem class. An algorithm for solving problems in P is asymptotically optimal (for P) if its running time is O(f (n)), for some f (n)), and the worst-case lower bound for algorithms for solving problems in P is Ω(f (n)). Thus an asymptotically optimal algorithm is at most a constant factor worse than the best possible algorithm (for the problem class). Merge sort and heapsort are asymptotically optimal comparison sorts, since the O(n lg n) upper bound on their running times match the Ω(n lg n) worst-case lower bound for comparison sorts. Quicksort is not asymptotically optimal. Once an asymptotically optimal algorithm for a problem class has been discovered, the focus moves from asymptotical issues to practical issues like finding algorithms with smaller (hidden) constants and better implementations. 10

11 COMP3600/6466 Algorithms 2018 Lecture 8 11 Counting sort As we have seen, comparison sorts require Ω(n lg n) comparison in the worst case. But this result depends on assuming that only comparisons are allowed. If we assume more about the input data, then faster sorting algorithms are possible, even linear ones. We now examine a linear sorting algorithm called counting sort that is not a comparison sort. Its assumption is that each of the n input elements is an integer in the range 0 to k, for some positive integer k. When k = O(n), counting sort runs in Θ(n) time. (Two other (mostly) linear sorting algorithms, radix sort and bucket sort, each based on different assumptions about the input data, are discussed in Cormen et al, Sections 8.3 and 8.4.) 11

12 COMP3600/6466 Algorithms 2018 Lecture 8 12 Counting sort 2 Cormen et al, p

13 COMP3600/6466 Algorithms 2018 Lecture 8 13 Counting sort 3 Cormen et al, p

14 Counting sort 4 How much time does counting sort require? The first for loop takes time Θ(k), the second for loop takes time Θ(n) the third for loop takes time Θ(k), and the fourth for loop takes time Θ(n). Thus the overall time is Θ(k + n). Usually in applications, k = O(n), in which case the running time is Θ(n). Counting sort is a stable sorting algorithm, which means that numbers with the same value appear in the output array in the same order as they do in the input array. Often stability is not important, but counting sort is often used as a subroutine in radix sort and, for this application, stability is absolutely crucial. 14 COMP3600/6466 Algorithms 2018 Lecture 8 14