Smart Sort and its Analysis

Transcription

1 Smart Sort and its Analysis Varun Jain and Suneeta Agarwal Department of Computer Science and Engineering, Motilal Nehru National Institute of Technology, Allahabad , Uttar Pradesh, India. Abstract. Sorting is a well-known computational problem. Sorting means arranging a set of records (or a list of keys) in some (increasing or decreasing) order. A new algorithm for sorting, Smart Sort is presented here, which smartly determines the sorted sub lists already present in the input. It then merges these sub lists with the help of heap. The best-case time complexity of Smart Sort is O(n) while the worst-case time complexity is O(n log n). Practically, it is also found to be more efficient than merge sort or heap sort (both comparison based sorting algorithms). Key words: Algorithm, sorting, min heap, merge, heapsort, mergesort. 1 Introduction A file of size n is a sequence of n items r{1, r{2,, r{n. Each item in the file is called a record. A key, a{i, is associated with each record r{i. The key is usually (but not always) a sub field of the entire record. The file is said to be sorted on the key if i < j implies that a{i precedes a{j in some ordering on the keys [1]. The following is the sorting problem. Input: A sequence of n numbers (a1, a2, a3 an). Output: A permutation (or reordering) (a1, a2, a3 an ) of the input sequence such that a1 <= a2 <= a3 <= <= an.[2]. Some of the most common sorting algorithms are merge sort[3], heap sort[4,5], quick sort[6,7], insertion sort[8]. Smart sort is explained below. A new algorithm, smart sort is presented here, which utilizes the sorted parts of the input which are merged with the help of heap. The paper is organized as follow: Section 2 explains how the sorted sub lists are obtained. The next section deals with creating heap from the top element of these sub lists and how the elements from the heap are deleted to get a sorted output. An alternative way to merge the sub lists to get a stable sorted output is given in Section 4. The next section gives the smart sort algorithm in pseudo code form. Section 6 discusses the complexity of algorithm in three cases namely best-case (O(n)), worst-case (O(n log n)) and average-case (O(n log n)). Section 7 illustrates the situations in which this algorithm is very useful.

2 2 First Step: The Smart Step Input array is divided into sub lists. The sub lists may be non-decreasing or nonincreasing. These sub lists are obtained in one scan on the input. The information about these sub lists is stored in a heap of headers. The header has three fields: 1. integer: starting index: It points to the smallest element of each nondecreasing or non-increasing sub list. 2. integer: sign: It can be 0, +1 or -1. Sign = +1 indicates that the sub list is nondecreasing. Sign = -1 indicates that the sub list is non-increasing. Sign = 0 indicates that the next element will decide the sub list to be non-decreasing or non-increasing. 3. integer: size: It stores the number of elements in the sub list. The header is initialized as starting index = 0, sign = 0 and size = 1. Each successive element of input is either attached with the present header or a new header is created for it. This decision is based on the sign of the current header (the header in which the previous element is saved). (i) If sign was 0: and the current data is greater than or equal to the previous element then sign of the current header is updated as 1 and the size is also incremented by one. If current data is less than or equal to the previous element, sign is set as -1 and size is incremented by one, index is also incremented by one. (ii) If the sign was 1(-1): and current data is less (greater) than the previous element, then a new header is created.the sign of this new header is set as 0. Index of the present element is stored in its index field. Size field is initialized to be one. If sign was 1 and the current data is greater than or equal to the previous one, then only the size of the present header is incremented. If sign was -1 and the current data is less than or equal to the previous one, then not only the size of the present header is incremented but also the index is incremented by one. This helps in treating even the non-increasing sub lists as non-decreasing sub lists because now the starting index of the header always points to the smallest element of the sub list. An example: Input: 5, 7, 8, 9, 3, 2, 1, 6, 5, 2. The following headers are created at the end. Table 1. Headers created in the example. Heap{1 Heap{2 Heap{3 Index Sign Size Thus the following three sorted sub lists are found in the original data set. {5,7,8,9,{3,2,1,{6,5,2. Each sub list will be treated as non-decreasing one. So these sub lists are starting from indices 0, 6, 9 respectively. Sign of the list indicates whether it is non-decreasing or non-increasing. These sub lists are stored in an array Heap through headers. Heap{1, Heap{2, Heap{3. Details of any sub list can be

3 obtained from its header list. Sign of Heap{1 is +1, means it is a non-decreasing sub list. Size field of it is 4, means number of elements in it are four, index field is 0, which means this sub list starts from the index 0 of the input and continues till index 3 of input. Sign of Heap{2 is -1, which means second sub list is a nonincreasing one. Heap{2.index is 6, which means that the smallest element of this sub list is at index 6 of input. Heap{2.size is 3, which means it contains three elements starting from 6 th element to 4 th element of the input. Similarly, third sub list is again a non-increasing list of size three. Every non-increasing sub list can be treated as a nondecreasing one, starting from input{index and ending at input{index size Second Step: Heapification 3.1 Creation Thus, the whole input data has been divided into sub lists stored in an array Heap{ of headers, w.r.t. the starting (smallest) element of non-decreasing sub lists. Next, Heap{ is converted into a min heap. Here min heap is formed w.r.t. the least value of each sub list at Heap{i.index whose position is available in its header. Heap can also be created while the headers are being formed in the smart step. Whenever a new header is created, we insert the previous header in the min heap and increment the heap_size by 1. The last header is inserted in the min heap when the scan of input is complete. We keep a heap_size variable that will store the current number of sub lists in the min heap. We also keep a headersize variable that will keep the current number of headers. They are initialized by 0. Whenever a new header is created, the headersize is incremented by one and whenever a new header is inserted in min heap, we increment the heap_size by 1. After the scan of input is complete, the headersize and heap_size equals. Before completion of scan heap_size is always one less than headersize. Initially, headersize = heap_size = 0. When Heap{1 is created in the example, we increment the headersize by 1 and heap_size remains 0. When Heap{2 is created, both headersize and heap_size is incremented by 1. so, min heap now contains only one element. When Heap{3 is created, again headersize and heap_size is incremented by 1 and Heap{2 is suitably placed in the heap according to the value input_data{heap{2.index. When scan of input is over, heap_size is incremented by 1. Now, heap_size and headersize are equal. The Heap{ now looks as: Table 2. Headers after Creation step of Heapification. Heap{1 Heap{2 Heap{3 Index Sign Size 3 4 3

4 3.2 Deletion Once the min heap has been created, the location and the value of least element of the present data set can be obtained from the root of the heap. The location of least element is at Heap{1.index and its value is input{heap{1.index. This element is now saved in a new array output{. This element is then removed from its sub list. For this purpose, index of the header is updated as index + sign while the size is always reduced by one. Thus, the next element of this sub list becomes the root of heap. If the property of min heap is violated by this change, then the heap is updated to retain its property. In case a sub list becomes empty after this deletion, then its header is swapped with the header present at the end of the heap and size of the heap is also reduced by one. If this swapping upsets the min heap, Heap{ is updated to retain its property. This process is continued till heap contains one header only. Now all the elements of this sub list are appended in the array output{. Output is the required sorted elements of input{. 4 Alternative to Second Step: Merging for stability The sorted output obtained in the above method may not be stable. However, a stable sorting can be obtained by combining successive pair of sub lists with simple merging process. 5 Smart Sort Algorithm The pseudo code of Smart Sort is as follow: Input: input_data[] = R0, R1, R2 Rn-1 Initialisation: Take heap[], maximum size = n/2. heap_size = 0 (number of elements in heap[] ). header_size = 0 (stores number of current headers while scanning). structure of heap[]: index (of_input), sign (default = 0), size (default = 0). output_data[], size n complexity = 0 /* it is used in the graph */ Smart step and min heap creation: { j = 1 heap[j].index = 0 (index of first data element) heap[j].size = 1; header_size = 1; last = 0; present = 1; complexity = complexity + n for (i = 1 to n-1) { present = i. if heap[j].sign equal to 0 { if (input_data[last] less than or equal to input_data[present]) { heap[j].sign = 1

5 else if(input_data[last] greater than or equal to input_data[present]){ heap[j].sign = -1 heap[j].index = present else if ((input_data[last] less than input_data[present] and heap[j].sign equal to -1) or ( input_data[last] greater than input_data[present] and heap[j].sign equal to 1)) { increment header_size by 1 complexity = complexity + log2(heap_size) call insert_heap(heap, heap_size) increment j by 1 heap[j].index = present heap[j].sign = 0 heap[j].size = 1 increment heap_size by 1 else if(input_data[last] greater than or equal to input_data[present] and heap[j].sign is equal to -1) { increment heap[j].size by 1. heap[j].index = present. else if(input_data[last] less than or equal to input_data[present] and heap[j].sign is equal to 1){ increment heap[j].size by 1. last = present complexity = complexity + log2(heap_size) call insert_heap(heap, heap_size) increment heap_size by 1 /* insert_heap procedure */ insert_heap(heap,i) { take a temporary variable temp of type heap[]. temp = heap[i]. while(( i!=1 ) and ( input_data[ temp.index ] > input_data[ heap[i/2].index ])) { heap[i] = heap[i/2] i = i/2 heap[i] = temp Deletion and sorting: { for (i = 0 to n-1) { /* get the least element from the heap. */

6 output_data[i] = input_data[heap[1].index] /* update first element of heap[] i.e. heap[1] */ decrement heap[1].size by 1 heap[1].index = heap[1].index + heap[1].sign if heap[1].size equal to 0 { heap[1] = heap[heap_size] decrement heap_size by 1 /* rearrange the heap */ key = heap[1] parent = 1 child = 2 while (child is less than or equal to heap_size) { if child less than or equal to heap_size and input_data [heap[child].index]] greater than input_data[heap[child+1].index]] { increment child by 1 if input_data[key.index] less than or equal to input_data [heap[child].index] { break heap[parent] = heap[child] parent = child child = child * 2 increment complexity by 1 heap[parent] = key 6 Analysis of Algorithm 6.1 Smart Step It will take one scan i.e. n number of comparison to make all the headers, where n is number of elements to be sorted. 6.2 The Heap Best-case: The best-case is when all the elements are already sorted either in non-increasing or non-decreasing order. In best-case only one header will be formed in the Smart Step (First Step). Complexity of creating a min heap from one header is

7 log 1 i.e. 0. And also the complexity of deleting is log 1 i.e. 0. So, in the best-case Smart Sort will sort the elements in O( n ) Worst-case: In the worst-case in Smart Step, number of headers formed would be n/2 (when the input data is such that one element is in increasing order and next element is in decreasing order). It is assumed that the number of elements is an even number; otherwise number of sub lists would be n/ Creation of Heap: In heap creation, each element is first placed in the end of the heap and then compared with parents. In the worst-case, each element becomes the root when it is inserted. So, each element will be compared height (of heap created till then) times. Then, total number of comparisons in the heap creation in worst-case will be as follow: ( log n/2 1).2 log n/2 1 + ( log n/2 ) (n/2 2 log n/2 + 1). (1) First element will be the root. The next two elements will be compared 1 time each. The next four elements will be compared 2 times each. The last level in the heap might not be full. Number of elements in the last level of the heap of x elements will be ((x 2 log x + 1). The summation of above series (with the help sum of geometric series [9]) is (n/2+1)( log n/2 ) 2.2 log n/ (2) Let us suppose n/2 will make a full heap. Then, the last term would be ( log n/2 ).2 log n/2. The summation would be then: 2.2 log n/2 log n/2 2.2 log n/ Deletion from Heap: In the worst-case in deletion, whenever an element is deleted from the sub list pointed by the root of heap, the next element pointed by the root upsets the min heap property. The pointer to the sub list then becomes the leaf of the heap. There are two elements in each sub list and there are n/2 sub lists. In the worst-case first n/2 elements deleted from the heap leaves the heap of size n/2, i.e. n/2 sub lists with each sub list containing one element. In the worst-case, therefore, to get the first n/2 elements in the sorted output, the number of comparisons will be (n/2)( log n/2 ). Now, the remaining heap will contain only one element in each node. So, in worst-case, to get the rest n/2 elements in the sorted output, the comparisons will be same as when creating heap, which is (n/2+1)( log n/2 ) 2.2 log n/ Thus, in the worst-case, total number of comparisons taken by smart sort is the summation of smart step, creation of heap and deletion from heap: n + 2(n/2+1)( log n/2 ) 4.2 log n/ n/2( log n/2 ). = (3n/2 + 2) log n/2 + n 2 log n/ (3) = O((3n/2 + 2) log n/2 ). = O( n log n ). (4)

8 6.2.3 Average Case: For the average case, first the average number of sub lists that can be formed is calculated. 1. Average number of sub lists: Let there are n numbers and they are divided into b sub lists. Let the a1, a2, a3 ab, be the number of elements in the b sub lists. Then, a1 + a2 + a3 + + ab = n, where a1, a2, a3 ab >= 2. (5) Total number of solutions of the equation following above constraint is n 2b + b 1 C b 1 = n b 1 C b 1.[10], where 1<= b <= n/2. (6) The average number of sub lists (using weighted arithmetic mean [11]) will be { n 2 C n 3 C n 4 C n/2 1 C n/2 1. n/2 / { n 2 C 0 + n 3 C 1 + n 4 C n/2 1 C n/2 1. (7) The numerator is summation of product of number of sub lists (b) and the number of ways in which b sub lists can be formed. The denominator is summation of number of ways in which b sub lists can be formed. The value of expression (6) (= {(n 2b + b 1)! / {(b 1)! (n 2b)!) will increase up to a maximum and then decrease back to 1, while value of n varies from 1 to n/2. The maximum value of the expression will be when the two terms of denominator will be equal. i.e. b 1 = n 2b, which implies b = (n + 1)/3. But this is not the average, because the terms on the left side have more weight than that on the right side of b = (n+1)/3. So, the average is less than (n+1)/3. Using the above formula, (Fig. 1.) is computed. We can say that average number of sub lists is always less than n/3 and more accurately ((n/3+n/4)/2 =) 7n/24. The average number of sub lists is considered as n/3 for the rest of the paper. SMART STEP ANALYSIS NUMBER OF LISTS Worst (n/2) Average n/3 Best n/ NUMBER OF ELEMENTS

9 Fig. 1. Smart Step Analysis. 2. Creation of Heap: Whenever an element is added in heap, it has equal probability to be at any node in the heap. The number of comparisons required in adding an element in average-case (using weighted arithmetic mean [11]) will be given by expression (8). The numerator in expression (8) is the summation of product of number of comparison required for an element to add to a certain level of heap and the number of elements at that level of heap which can be replaced by the added element to satisfy min heap property. The denominator is total number of elements in the heap which can be replaced by the added element to make min heap. { ( log x 1).2 log x 1 + log x (x 2 log x + 1) / (8) { log x 1 + (x 2 log x + 1). = {(x+1)( log x ) 2.2 log x + 2 /x. = log x + (1/x)( log x ) 2.2 log x / x + 2/x. (9) This is the average number of comparisons to add one element to the min heap. So, average number of comparisons of adding n elements in heap will be summation of average of adding all the elements in the heap. log x + (1/x)( log x ) 2.2 log x / x + 2/x. (10) x is varying from x = 1 to y. The average of number of sub lists formed is always less than n/3, so y can be taken as n/3. Expression (10) contains four parts. (a) The first part of the expression (10): Using Stirling s approximation [2] to get expression (11). log x <= log ( y! ) = O (y log y ) = O (n/3 log n/3 ). (11) (b) The second part of the expression (10): (1/x)( log x ) = (1/2 + 1/3) + (2/4 + 2/5 + 2/6 + 2/7) + (3/8 + +3/15) ( log y /2 log y + + log y /y)). (c) The third part of the expression (10): (1/x)( log x ) <= y = n/3. (12) 2.2 log x /x = (2/2 + 2/3) + (4/4 + 4/5 + 4/6 + 4/7) + + (2 log y /2 log y log y /y). 2.2 log x /x >= H y = ln y + γ + O( 1/y ) [12]. (13) H y = ln n/3 + γ + O( 3/n ). (14) The term (13) will be deducted therefore, lower bound is taken. In H y γ = is Euler s constant.[12] (d) The fourth part of the expression (10):

10 2/x = 2H y = 2 ln n/3 + 2γ + 2O( 3/n ). (15) Thus, total number of comparisons of heap creation in average case (using (11), (12), (14) and (15)) is O (n/3 log n/3 ) + n/3 (ln n/3 + γ + O( 3/n )) + 2 ln n/3 + 2γ + 2O( 3/n ). = O (n/3 log n/3 ) + n/3 + ln n/3 + γ + O( 3/n ). = O (n/3 log n/3 ). (16) 3. Deletion from Heap: Let us suppose, after getting first 2n/3 elements from the heap to the sorted output, we are left with the heap with n/3 elements. Now, there is only one element in each node of the heap. The average case is always better than this case because while deleting elements in the average case, the heap size can reduce. Thus, the average case to get first 2n/3 sorted elements will be always better than (2n/3) log n/3. So, to get the rest n/3 elements in the sorted output, the comparisons will be same as when creating heap, which is O (n/3 log n/3 ) + n/3 + ln n/3 + γ + O( 3/n ) = O (n/3 log n/3 ). (17) Thus, in the average-case, total number of comparisons is summation of smart step, creation of heap (using (16)) and deletion from heap (using (17)): n + O (n/3 log n/3 ) + O (n/3 log n/3 ) = O (n/3 log n/3 ) = O (n log n ). (18) The best-case complexity of heap sort is O( n log n ), whereas for Smart Sort it is O( n ). The worst-case complexity of heap sort is O( n log n ), same as for Smart Sort it is O( n log n ). But practically, smart sort is better than heap sort because height of heap in case of Smart Sort is often smaller than the height of heap in Heap Sort (log n). 6.3 The Merge In Merge Sort, when merging is done, first single elements are merged to get sub lists of two elements. Then these two elements sub lists are merged to get 4 elements sub lists. This process continues till we get all elements in one list. The complexity in this case is O( n log n ). In Smart Sort, we already have sub lists of size greater than or equal to 2. Worstcase is sub list of size 2, and best-case is when all elements are in one sub list. In the worst-case, the time for making headers is compensated by the merging of single elements in the Merge Sort. Generally, these sub lists will contain more than 2 elements; therefore, time of merging will be reduced effectively. Still we can say that worst-case complexity of Smart Sort is O( n log n). But practically, Smart Sort is better. In the best-case (all elements are already sorted either in increasing order or decreasing order), when only one list is formed, no need of merging. So, best-case complexity will be O( n ).

11 Table 3. Complexities of Smart Sort and some most common sorting algorithms. Name Best Average Worst Smart Sort O( n ) O( n log n ) O( n log n ) Merge Sort O( n log n ) O( n log n ) O( n log n ) Heap Sort O( n log n ) O( n log n ) O( n log n ) Quick Sort O( n log n ) O( n log n ) O( n 2 ) Insertion Sort O( n ) O( n 2 ) O( n 2 ) 6.4 Computation of Complexity COMPLEXITY COMPARISON OF SMART SORT AND MERGE SORT ON A DATA OF 1000 ELEMENTS NUMBER OF ELEMENTS OUT OF ORDER simple merge Smart Sort with merge Smart Sort with Heap Fig. 2. Comparison of Smart Sort and Merge Sort on a data of 1000 elements. The complexity used in the graph (Fig. 2.) is computed as follow. Complexity is initialised by 0. For the smart step, the complexity is incremented with the number of input elements, as each of them is compared to the previous one. In case of smart sort with heap, during heap creation, the complexity is incremented each time a new list is added to the heap by log (to the base of 2) of the number of elements already added in

12 the heap. When rearranging the heap during deletion from the heap, the complexity is incremented by one for each comparison with the parent element. In case of smart sort with merge the complexity is incremented each time the merge is called, by the size of the two sub lists being merged. In simple merge sort, complexity is computed by incrementing it each time by the size of sub lists being merged. In simple merge sort it comes out to be n log n, where n is the number of elements in the input. The graph (Fig. 2.) is depicting the comparison of smart sort and merge sort on a data of 1000 elements with different number of elements out of order in the input. The graph shows that the more the input is sorted the better is the smart sort. 7 Applicability When the elements are randomly arranged, Smart Sort is as good as Heap Sort or Merge Sort. When the elements are already sorted or sorted in chunks, smart sort is efficient. In networking, if one wants to send a large sorted database to another computer. The database is divided into small chunks, each containing many records. These chunks may travel through different routes and finally reach the destination. Thus these chunks can reach out of order. Smart Sort will sort these chunks faster than Heap Sort or Merge Sort. If a company is maintaining a large database and it is sorted after say every week or month. New set of records are inserted, deleted or updated in the database. So, these updating leaves the database not sorted. Only a set of data is not sorted. Smart Sort can sort these databases faster. 8 Conclusion Smart Sort is better than Heap Sort or Merge Sort. Its best-case complexity is O( n ), but that of heap sort is O( n log n ), merge sort is O( n log n ) and quick sort is O( n log n ). And worst-case complexity O( n log n ) is practically better than Heap Sort because the height of heap is less in case for smart sort than heap sort. Worst-case complexity is also better than or as good as merge sort because after smart step we get lists of size greater than or equal to 2. Worst-case of smart sort is better than quick sort, whose worst-case complexity is O( n 2 ). Generally, when the data is already sorted, quick sort does not perform efficiently. There is no such restriction for the use of smart sort. Smart sort is better than the best-case of quick sort and is as good as it is in the worst-case. For millions of data heapification step can be used in smart sort. But this is not stable. For stability, merging can be applied after the Smart Step.

13 References 1. Yedidyah, L., Augenstein, M. J., and Tanenbaum, A.M.: Data Structures using C and C++. Prentice-Hall of India Pvt. Ltd., New Delhi (2005) 2. Cormen, T. H., Leiserson, C. E., and Rivest, R. L.: Introduction to Algorithms. Prentice-Hall of India Pvt. Ltd., New Delhi (2006) 3. Knuth, D. E.: The Art of Computer Programming, Vol. 3: Sorting and Searching, 2nd ed. Addison-Wesley (1998) 4. Horowitz, E., Sahni, S., and Anderson-Freed, S.: Fundamentals of Data Structures in C. Computer Science Press, New York (1998) 5. Williams, J. W. J.: ACM algorithm 232, Heapsort. In: Communications of the ACM, 7(6), (1964) 6. Hoare, C.A.R.: Quicksort. In: Computer Journal 5(1) (Apr. 1962) 7. Hoare, C.A.R.: Algorithm 64: Quicksort. In: Communications of the ACM, 4(7) (July 1961) 8. Horowitz, E., Sahni, S., and Rajasekaran, S.: Computer Algorithms. Galgotia Publications Pvt. Ltd. (2003) 9. Aufmann, R. N., Barker, V. C.: Intermediate Algebra with Application. Houghton Mifflin Company (1986) 10. Liu, C. L.: Elements of Discrete Mathematics. McGraw-Hill (1985) 11. Spiegel, M. R., Stephens, L. J.: Schaum's Outline of Theory and Problems of Statistics. McGraw-Hill Professional (1999) 12. Knuth, D. E.: The Art of Computer Programming, Vol. 1: Fundamental Algorithms, 3rd ed. Addison-Wesley (1997)