1 The sorting problem

Transcription

1 Lecture 6: Sorting methods - The sorting problem - Insertion sort - Selection sort - Bubble sort 1 The sorting problem Let us consider a set of entities, each entity having a characteristics whose values are can be ordered (there exists a relation of total ordering on the set of these values). This characteristics is called sorting key. Sometimes the entire entity is the sorting key (for instance in the case of a set of numerical values). The sorting problem refers to rearranging the elements of the set such that the values of the key are in a given order (an increasing or a decreasing order). Example 1. Let us consider the following set of integers: (5,8,3,1,6). In this case the sorting key is right the element. By an increasing sorting one obtains (1,3,5,6,8) while by decreasing sorting one obtains (8,6,5,3,1). Example. Let us consider an array of entities consisting of two fields: a student name and a grade (integer number). For instance the array is: ((Peter, 9), (John, 10), (Victor, 8),(Adam, 9)). In this case the sorting key could be either the name or the grade. By increasingly sorting by name one obtains: ((Adam, 9),(John, 10),(Peter, 9), (Victor, 8)). On the other hand by decreasingly sorting by the grade, one obtains ((John,10),(Peter,9),(Adam,9),(Victor,8)). In the following we shall consider that the set to be sorted consists of values belonging to a set on which one can define a total order relationship and the aim is to increasingly sorting the array. Sorting the array (x 1, x,..., x n ) is equivalent with finding a permutation (p(1), p(),..., p(n)) such that x p(1) x p()... x p(n). On the other hand we shall consider that the elements of he set are stored on a random access memory. This is the case of internal sorting. The other case (that of external sorting) is used when the data to be sorted are stored in files on hard disks. In the following we shall consider that the set to be sorted is stored in an array. Depending on the extra memory which the sorting process requires there are two main types of methods: Sorting methods which uses an extra memory having a size proportional to the size of the data set: the initial array is x[1..n], while the sorted one will be y[1..n]. Sorting which doesn t require extra space (sorting in place). The elements of the array x[1..n] change their position such that at the end the elements are sorted. Swapping two elements needs an extra memory but its size is small (the size of an element and not of the entire array). Some properties of the sorting methods are: Simplicity. A sorting method is considered to be simple if it is intuitive and easy to understand. 1

2 Efficiency. A method is considered efficient if it doesn t require a large amount of resources. From the point of view of space efficiency an in-place method is more efficient than a one which uses extra memory. From the point of view of time efficiency a sorting method is better if its running time is smaller. Usually the efficiency analysis of a sorting algorithm refers to operations like comparison or swapping. Naturalness. A sorting method is considered natural if the number of operations decreases as the distance between the initial array and the sorted one also decreases. A measure for this distance could be the number of inversions. Stability. A sorting method is stable if it preserves the relative order of any two equal elements in its input. For instance if we apply a stable sorting method to sort in alphabetical order the array from Example we obtain ((John,10),(Paul,9),(Adam,9),(Victor,8)) while if we apply a non-stable method we obtain ((John,10),(Adam,9),(Paul,9),(Victor,8)). In the following we shall analyze some basic sorting methods. These methods are simple, easy to implement and analyze but are not the most efficient. However they can be successfully applied for small size arrays. On the other hand they represent the starting point for some advanced methods. Insertion sort The basic idea of this method is: Starting with the second element of the array, each element is inserted in the sub-array which precede it such that this sub-array is kept sorted. For each step of the sorting process we search for the right position of x[i] in the target sub-array x[1..i 1] (which is already sorted) by comparing x[i] with the elements of x[1..i 1] starting from the right. While x[i] is smaller than x[j] the elements are moved one position to right (in order to create space to insert the element x[i]). The general structure of the algorithm is: insertion(x[1..n]) for i, n do < insert x[i] in x[1..i 1] such that x[1..i] is kept sorted > A more detailed description is: insertion1(x[1..n]) for i, n do j i 1 { i 1 is the index from where we start the search } aux x[i] { saving the value of x[i] } while aux < x[j] j 1 do { searching the insertion position } x[j + 1] x[j] { moving x[j] one position to the right } j j 1 endwhile x[j + 1] aux { inserting the value on the right position }

3 There are different ways of implementing this method. Thus, in order to avoid the comparison j 1 for each execution of the inner loop we can put the value x[i] before the first position of the array (position 0). This element plays the role of a flag. In this variant, we arrive to x[j] = x[i] at least when j = 0. This variant can be described as follows: insertion(x[1..n]) for i, n do x[0] x[i] { placing the value x[i] on position 0 } j i 1 while x[0] < x[j] do x[j + 1] x[j] j j 1 endwhile x[j + 1] x[0] { inserting the value on the right position} Correctness analysis. The problem s precondition is {n } while the postcondition is {x[1..n] is increasingly sorted}. We will identify a loop invariant for each loop. For the external loop (on i) we prove that an invariant property is {x[1..i 1] is increasingly sorted }. At the beginning of the loop i =, thus x[1..i 1] = x[1..1] and it is sorted. At the end of the loop (i = n + 1) the property implies the postcondition. We only have to prove that the property remains true after each loop execution. To do this it suffice that for the inner loop (on j) the assertion {x[1..j] is increasingly sorted and aux x[j + 1] = x[j + ]... x[i]} is invariant. At the end of the WHILE loop this property would imply one of the following statements: x[1]... x[j] aux x[j + 1] = x[j + ]... x[i] when the stopping condition is aux x[j]; aux x[1] = x[]... x[i] when the stopping condition is j = 0. Each relation would lead by x[j + 1] aux to x[1]... x[j] x[j + 1] x[j + ]... x[i] and by i i + 1 to {x[1..i 1] is increasingly sorted}. Thus it remains only to prove that {x[1..j] is increasingly sorted and aux x[j +1] = x[j +]... x[i]} is an invariant of the while loop. At the beginning of the loop the following relations hold: j = i 1, aux = x[i] thus aux = x[j + 1] = x[i] and since x[1..i 1] is increasingly sorted it follows that analyzed property is true. We prove now that this property is not altered by the processing steps of the loop: if aux < x[j], by x[j + 1] x[j] one obtains: aux < x[j] = x[j + 1]... x[i] and by j j 1 the following assertion will become true: { x[1..j] is strictly increasing aux < x[j + 1] = x[j + ]... x[i]. Since x[j + 1] = x[j + ] by the assignment x[j + 1] x[j] we do not lose elements of the array. The algorithm termination is assured by using a counting variable for each loop. Efficiency analysis. We will analyze only the comparisons and moving operations. Let T C (i) and T M (i) be the number of comparisons and of moving operations. The best case is when the array is increasingly sorted while the worst one is when the array is decreasingly sorted (when the purpose is to increasingly sorting the array). In the best case: T C (i) = 1 and T M (i) = (it refers to the operations involving the auxiliary variable aux which in fact are useless in this case), thus T (n) n i= (T C (i) + T M (i)) = 3(n 1). 3

4 In the worst case T C (i) = i and T M (i) = i 1+ = i+1, thus T (n) n i= (i+1) = n +n 3. It follows that 3(n 1) T (n) n + n 3 meaning that the insertion sort belongs to Ω(n) and to O(n ). Other properties. The insertion sort is a natural method and as long as the continuation condition of the inner loop is aux < x[j] it is also stable. However if this condition is aux x[j] the insertion sort is no more stable. 3 Selection sort The basic idea of this method: For each position, i (starting with the first one) the subarray x[i..n] is scanned in order to identify the minimum. This minimum is swapped with the ith element. The general structure of the algorithm: selection(x[1..n]) < find the minimum of x[i..n] and swap it with x[i] > The upper limit of the FOR loop is n 1 since the subarray x[n..n] has only one element which is placed on the final position (due to previous swaps). A more detailed description of the algorithm is: selection(x[1..n]) k i { the current index of the potential minimum } for j i + 1, n do { loop for searching the index of the minimum } if x[k] > x[j] then k j endif {the new index of the minimum} if k i then x[k] x[i] endif {placing the minimum on the right position} Correctness analysis. We will prove that {x[1..i 1] is increasingly x[i 1] x[j] for j = i, n} is an invariant for the external loop. At the beginning i = 1 thus x[1..0] is an empty array. The inner for loop (on j) find the index of the minimum of x[i..n]. The minimum is then placed on position i. Thus we obtain that x[1..i] is increasingly sorted and x[i] x[j] for j = i + 1, n. After incrementing i (at the end of the loop on i) we obtain again the invariant. At the end, i = n and the loop invariant leads us to x[1..n 1] increasingly sorted and x[n 1] x[n] thus x[1..n] is sorted. Efficiency analysis. For any initial arrangement of the elements the number of comparisons is: T C (n) = n 1 n i=1 j=i+1 n 1 1 = (n i) = i=1 = In the best case (that of increasingly sorted arrays) the number of swaps is T M (n) = 0. In the worst case (that of a decreasingly sorted array) for each value of i one swap (three assignments) is executed. Thus T M (n) = 3 n 1 i=1 1 = 3(n 1). 4

5 Taking into account both the comparisons and the swaps we can say that the selection sort belongs to Θ(n ). Other properties. The above algorithm (based on swapping the minimum with the current position) is not stable. If instead of the swap operation, all elements of x[i..k 1] would be moved one position to the right (as in insertion sort) and x[k] (previously saved in an auxiliary variable) would be transferred in x[i] then the selection sort will be a stable sorting method. 4 Bubble sort The basic idea of this method: The array is scanned from left to right and the adjacent elements are compared. If they are out of order then they are swapped. The scanning process is repeated until the array is sorted. The general structure of the algorithm is: bubblesort (x[1..n]) repeat < scan the array and if two adjacent elements are out of order swap them > until < no swap is necessary > Let us analyze what s happening during one scan of the array: if x[i] > x[i + 1] then x[i] x[i + 1] endif Using as a loop invariant the assertion: {x[i] x[j], j = 1, i} it is easy to prove that this loop leads us to the postcondition {x[n] x[i], i = 1, n}. For i = 1 the assertion is true since x[1] x[1]. Let us suppose that the property is true for i. If x[i] x[i + 1] then no operation is executed and it is true also for i + 1. If on the other hand x[i] > x[i + 1] then a swap is executed and one arrives to x[i] < x[i + 1] thus the property becomes true for i + 1. Thus the assertion above is indeed a loop invariant. Based on this property it follows that it suffices to apply that sequence successively for x[1..n], x[1..n 1],..., x[1..]. It follows that the algorithm can be described as follows: bubblesort1(x[1..n]) for i n,, 1 do for j 1, i 1 do if x[j] > x[j + 1] then x[j] x[j + 1] endif Correctness analysis. Since it has been already proven that the inner loop brings the maximum on the last position of the (sub)array x[1..i] it follows that for the outer loop we can consider as invariant the following assertion: {x[i + 1..n] is increasingly sorted and x[i + 1] x[j] for j = 1, i}. 5

6 Efficiency analysis. The number of comparisons do not depend on the initial configuration and it is: n i 1 n n 1 T C (n) = 1 = (i 1) = i =. i= j=1 i= On the other hand, the number of swaps depends on the initial configuration as follows: in the best case (already sorted array) we have T M (n) = 0 while in the worst case (decreasingly sorted array) we have: T M (n) = 3/ (a swap is equivalent with three assignments). Thus the number of operations satisfy: / T (n) i.e. the algorithm belongs to Θ(n ). The number of comparisons can be reduced by modifying it such that the scan is repeated only as many times as necessary (until no swaps is executed in the last iteration. The algorithm becomes: bubblesort(x[1..n]) repeat inter F alse if x[i] > x[i + 1] then x[i] x[i + 1] inter T rue endif until inter = F alse return x[1..n] In this case the number of comparisons is in the best case T C (n) = n 1 and in the worst case T C (n) = n j=1 (n 1) =. The number of swaps is the same as for the first variant: 0 T M (n) 3/. Taking into account both the comparisons and the swaps we obtain: n 1 T (n) i=1 5 thus the algorithm is slightly better than the first variant and it belongs to Ω(n) and to O(n ). On the other hand it suffices to scan the array until the position corresponding to the last swap. Thus the algorithm becomes: bubblesort3(x[1..n]) m n { at the beginning the entire array is scanned } repeat t 0 { t will contain the index of the last swap } for i 1, m 1 do if x[i] > x[i + 1] then x[i] x[i + 1] t i endif m t until t 1 return x[1..n] 6

7 As long as the swap is executed only when x[i] > x[i + 1] all variants of the bubble sort are stable. The running times corresponding to the above presented algorithm are grouped in the following table: Algorithm Best case Worst case Efficiency class T C (n) T M (n) T C (n) T M (n) T (n) = T C (n) + T M (n) n + n n + 3n 4 insertion1 n 1 (n 1) 3(n 1) T (n) n + n 3 selection bubblesort (n 1) 3 bubblesort n (Ω(n), O(n )) T (n) n + 5n 6 (Θ(n )) T (n) (Θ(n )) 5 n 1 T (n) (Ω(n), O(n )) 7