Computer Science 331

Transcription

1 Computer Science 331 Merge Sort Wayne Eberly Department of Computer Science University of Calgary Lecture #23

2 Introduction Merge Sort Merging Conclusions Outline

3 Introduction Merge Sort is is an asymptotically faster algorithm than the sorting algorithms we have seen so far: It can be used to sort an array of length n using Θ(n log 2 n) operations in the worst case. Presented Here: A version that takes an input array A and produces another sorted array B (containing the entries of A, rearranged). To simplify presentation this will be shown as an algorithm to sort arrays of integers. A solution to the Merging Problem (introduced next) is a subroutine that is used to do much of the work. Reference: Textbook, Section 10.7

4 The Merging Problem Signature: int[] merge(int[] A 1, int[] A 2 ) Precondition: A 1 is a sorted array of lengthn 1 for some positive integer n 1, so that A 1 [h] A 1 [h+1] for0 h n 1-2 A 2 is a sorted array of lengthn 2 for some positive integer n 2 N, so that A 2 [h] A 2 [h+1] for0 h n 2-2 For the signature used above, entries ofa 1 anda 2 are integers. More generally, they should be objects from the same ordered class

5 The Merging Problem Postcondition: Value returned is a sorted array of lengthn 1 +n 2 Entries of the output array are the entries ofa 1 together with the entries ofa 2, reordered but otherwise unchanged A 1 anda 2 have not been modified

6 Example and Cost A 1 A B Worst-Case Cost: This computation can be carried out in linear time, ie, using Θ(n 1 + n 2 ) steps

7 Suppose we Merge Sort: Idea for an Algorithm 1. Split an input array into two (roughly equally sized) pieces. 2. Recursively sort each piece. 3. Merge the two sorted pieces. This sorts the originally given array. Note: This is an example of a Divide and Conquer algorithm: Instance is used to form several smaller instances of the same problem The smaller instances are each solved recursively (that is, using the same algorithm) The solutions of the smaller instances are combined to form a solution of the given instance

8 Pseudocode int[] mergesort (int[] A) { } if (A.length == 1) { int[] B = new int[1]; B[0] = A[0]; return B; } else { int n 1 = (int)math.ceil(a.length/2); int n 2 = A.length - n 1 ; int[] A 1 = new int[n 1 ]; int[] A 2 = new int[n 2 ]; Insert pseudocode from next slide here! };

9 Pseudocode, Continued // Split into array into pieces; recursively // sort each, and merge the results for (int i=0; i < n 1 ; i++) { A 1 [i] = A[i]; }; for (int i=0; i < n 2 ; i++) { A 2 [i] = A[n 1 +i]; }; return merge(mergesort(a 1 ), mergesort(a 2 ));

10 Example

11 Example, Continued

12 Correctness Claim: If this algorithm is run on an input arrayaof length n 1, then the algorithm eventually halts, producing the desired sorted array as output. Proof of Claim: Induction onn(using the strong form of mathematical induction) Basis: Suppose n = 1

13 Correctness, Continued Inductive: Suppose that n 1 and suppose the algorithm terminates, producing the required sorted array, whenever it is run with an array with length at mostnas input. LetAbe an array with lengthn SinceA.length 2, theelse part of the program is executed. An arraya 1 withn 1 containing the leftmost n 1 entries ofa, and an arraya 2 of lengthn 2 containing the remaining entries of A are created. Note thatn 1 = (n+1)/2 nandn 2 = (n+1)/2 n.

14 Termination and Efficiency Let T(n) be the number of steps used by this algorithm when given an input array of length n, in the worst case. We can see the following by inspection of the code: { c 0 if n = 1 T(n) T( n/2 ) + T( n/2 ) + c 1 n if n 2 for some constants c 0 and c 1. Special Case: If n = 2 k is a power of two, we can rewrite this as { c 0 if n = 1 T(n) 2T(n/2) + c 1 n if n 2

15 Termination and Efficiency Claim: If n = 2 k is a power of two, and c = max(c 0, c 1 ), then T(n) cn log 2 (2n) = cn(k + 1). Method of Proof: Induction on k, using the fact that log 2 (n/2) = (log 2 n) 1 for any integer n 1. Proof:

16 General Case: Termination and Efficiency Consider the function L(n) = log 2 n for n 1 Useful Property: L( n/2 ) = L(n) 1 and L( n/2 ) L(n) 1 for every integer n 2 Claim: If n 1 then T(n) cnl(2n) cn(log 2 n + 2). Method of Proof:

17 Analyzing the Running Time Another Way Consider the various arrays that are formed, recursively sorted, and then merged together as this algorithm is used. Total Cost To Form and Merge Arrays at Any Level: Number of Levels:

18 The Merging Problem Signature: int[] merge(int[] A 1, int[] A 2 ) Precondition: A 1 is a sorted array of lengthn 1 for some positive integer n 1, so that A 1 [h] A 1 [h+1] for0 h n 1-2 A 2 is a sorted array of lengthn 2 for some positive integer n 2 N, so that A 2 [h] A 2 [h+1] for0 h n 2-2 For the signature used above, entries ofa 1 anda 2 are integers. More generally, they should be objects from the same ordered class

19 The Merging Problem Postcondition: Value returned is a sorted array of lengthn 1 +n 2 Entries of the output array are the entries ofa 1 together with the entries ofa 2, reordered but otherwise unchanged A 1 anda 2 have not been modified

20 Idea for an Algorithm Output will be constructed as an array B Maintain indices into each input array (each initially pointing to the leftmost element) repeat Compare the current elements of each input array Append the smaller entry onto the end ofb, advancing the index for the input array from which this entry was taken until one of the input arrays has been exhausted Append the rest of the other input array onto the end ofb

21 Pseudocode int[] merge (int[] A 1, int[] A 2 ) { int[] B = new int[a 1.length + A 2.length]; int i 1 = 0; int i 2 = 0; int j = 0; while ((i 1 < A 1.length) && (i 2 < A 2.length)) { if (A 1 [i 1 ] <= A 2 [i 2 ]) { B[j] = A 1 [i 1 ]; i 1 ++; } else { B[j] = A 2 [i 2 ]; i 2 ++; }; j++; };

22 } Pseudocode, Continued while (i 1 < A 1.length) { B[j] = A 1 [i 1 ]; i 1 ++; j++; }; while (i 2 < A 2.length) { B[j] = A 2 [i 2 ]; i 2 ++; j++; }; return B;

23 Example A 1 A B

24 Loop Invariant for Loop #1 At the beginning of each execution of the body of the first loop 1. i 1 andi 2 are integers such that0 i 1 <A 1.length and 0 i 2 <A 2.length 2. a) j =i 1 +i 2 b) B is an array such that B.length =A 1.length +A 2.length c) B[h] B[h+1] for every integerhsuch that0 h j 2 d) B[0],B[1],...,B[j-1] are the values A 1 [0],A 1 [1],...A 1 [i 1-1] anda 2 [0],A 2 [1],...,A 2 [i 2-1], reordered but otherwise unchanged; e) Ifj 1andi 1 <A 1.length thenb[j-1] A 1 [i 1 ] f) Ifj 1andi 2 <A 2.length thenb[j-1] A 2 [i 2 ] g) The arraysa 1 anda 2 have not been changed.

25 Loop Invariant for Loop #1 Proof That This is a Loop Invariant: Induction on the number of executions of the loop body, so far. Basis:

26 Loop Invariant for Loop #1 Inductive Step: Suppose thati 0, the loop is executed at leastitimes, and the loop invariant is satisfied at the beginning of thei th execution of the loop body. Then the following conditions are satisfied at the end of this execution of the loop body: 1. i 1 andi 2 are integers such that 0 i 1 A 1.length and 0 i 2 A 2.length 2. Condition 2 of the loop invariant is satisfied. Explanation: This, and success of the loop test, ensures that the loop invariant is satisfied, once again, at the beginning of thei + 1 st execution of the loop body.

27 Analysis for Loop #1, Concluded True at the End: At the end of every execution of the body of the first loop (notably including the final execution) i 1 andi 2 are integers such that0 i 1 A 1.length and 0 i 2 A 2.length Part 2 of the loop invariant is satisfied, once again The failure of the loop test ensures that the above properties hold when the first loop terminates and, furthermore, that either i 1 =A 1.length ori 2 =A 2.length at this point. Loop Variant for Loop #1: f(a 1,A 2,j) =A 1.length +A 2.length j Loop invariant implies thatj=i 1 +i 2, so that f(a 1,A 2,j) = (A 1.length i 1 ) + (A 2.length i 2 ); Initial value of this function isa 1.length +A 2.length

28 Analysis for Loop #2 Loop Invariant for Loop #2: At the beginning of each execution of the body of the second loop 1. i 2 =A 2.length andi 1 is an integer such that 0 i 1 <A 1.length 2. Part 2 of the loop invariant for the first loop is satisfied here as well. At the end of each execution of the body of this loop 1. i 2 =A 2.length andi 1 is an integer such that 0 i 1 A 1.length 2. Part 2 of the loop invariant for the first loop is satisfied once again.

29 Analysis for Loop #2, Continued Failure of the loop test implies that, on termination of the second loop, 1. i 1 =A 1.length andi 2 is an integer such that 0 i 2 A 2.length 2. Part 2 of the first loop invariant is satisfied Note: We must consider the possibility that loop #2 was skipped when considering the value ofi 2 at this point in the code! Loop Variant for Loop #2: Same as for Loop #1. Note that the value of this function has not been changed between the end of loop #1 and the beginning of loop #2. Conclusions:

30 Analysis for Loop #3 Loop Invariant for Loop #3: At the beginning of each execution of the body of the second loop 1. i 1 =A 1.length andi 2 is an integer such that 0 i 2 <A 2.length 2. Part 2 of the loop invariant for the first loop is satisfied here as well. At the end of each execution of the body of this loop 1. i 1 =A 1.length andi 2 is an integer such that 0 i 2 A 2.length 2. Part 2 of the loop invariant for the first loop is satisfied once again.

31 Analysis for Loop #3, Continued Failure of the loop test establishes the following on termination of loop #3: 1. i 1 =A 1.length andi 2 =A 2.length 2. Part 2 of the loop invariant for the first loop is satisfied. These properties establish the postcondition for the problem being solved. Loop Variant for Loop #3: Same as for loop #1. Note the the value of this function has not changed between the end of loop #2 and the beginning of loop #3. Conclusions:

32 Conclusions It can be shown (by consideration of particular inputs) that the worst-case running time of MergeSort is also in Ω(n log 2 n). It is therefore in Θ(n log 2 n). This is preferable to the classical sorting algorithms, for sufficiently large inputs, if worst-case running time is critical The classical algorithms are faster on sufficiently small inputs because they are simpler Alternative Approach: A hybrid algorithm: Use the recursive strategy given above when the input size is greater than or equal to some (carefully chosen) threshold value Switch to a simpler, nonrecursive algorithm (faster on small inputs) as soon as the input size drops to below this This is almost as fast as the simpler algorithms on small inputs, with worst-case cost in Θ(n log 2 n).

33 Conclusions Unfortunately this algorithm has a significant flaw that the classical algorithms do not: It cannot be used easily to sort in place, so that the amount of additional storage space that it must use is considerably higher than these slower algorithms This will be considered in more detail in the tutorial exercise on MergeSort Next: Another algorithm whose worst-case cast cost is also in Θ(n log n), that can be used to sort in place