Use of Tree-based Algorithms for Internal Sorting

Transcription

1 Use of Tree-based s for Internal Sorting P. Y. Padhye (puru@deakin.edu.au) School of Computing and Mathematics Deakin University Geelong, Victoria 3217 Abstract Most of the large number of internal sorting algorithms are based on the use of arrays or linked lists. This report highlights the use of tree structures for internal sorting. It is shown that the ordered binary tree can be used very effectively for sorting data, whatever the initial ordering of the input may be. It is also shown that the B-Trees, normally used to process large files on disks, can also be effectively employed to sort data in main memory. In both cases, a simple inorder traversal of the tree can retrieve data in sorted order. s based on binary- and B-tree structure are compared with Quick Sort and Heap Sort algorithms in terms of times required to sort various input lists. Input lists of various sizes and degrees of 'presortedness' were used in the comparison tests. It is shown that the times required to build balanced ordered binary trees are the same as or less than the times required to sort the data using Quick Sort and always better than the times required using Heap Sort. The time required to build a B- Tree varies with the order of the B-Tree. B-Trees of order 1, 2, 4, 8 and 16 have been compared to find that B-Trees of order 4 are most suitable. The times to build B-Trees of order 4 are shown to be same as or better than Heap Sort and slightly slower than Quick Sort. Further tests conducted to compare the times required for a complete processing task also found the ordered binary tree performance close to Quick Sort and the B-Tree performance as good as or better than Heap Sort. 1. Introduction "Sorting is the most-studied and most-solved problem in the theory of s" [Kingston, 1990]. "Sorting is an almost universally performed, fundamental, relevant and essential activity, particularly in data processing" [Wirth, 1976]. The topic has been researched for many years and several books are written on this topic [Knuth, 1973], [Mehlhorn, 1984], [Sedgewick, 1980]. Sorting and searching are also important topics in the area of data structures. Hence research in this area is still lively and relevant. By far the most commonly used data structure for internal sorting is the array. Tree structures like ordered binary trees, B-Trees are usually considered for applications which involve searching. It is, however, not widely recognised that these tree structures are suitable for sorting as well as searching. The tree structures are useful for sorting because they partition the data and retrieving data in key sequence is effected by a simple inorder traversal. Simply building the tree is thus tantamount to sorting. Further, the tree structures provide faster random access and maintain volatile data better than arrays or linked lists. One more advantage of using tree structures for sorting is that the sorting is progressive. At any intermediate stage the sorted output of the data seen up to that stage can be produced. The entire data does not have to be read before starting the sorting. Quick Sort, Heap Sort and most of the array based sorting algorithms except Insertion Sort, need to read the entire data first into an array.

2 2 The best case for building an ordered binary tree occurs when the input data is in random order. The time to build the tree in this case is O(nlogn). If the input is already nearly in the right or reverse order the ordered binary tree degenerates into a linear linked list and the time to build is O(n^2). To maintain the desirable O(nlogn) performance the tree has to be balanced. This can be done by first reading the input into an array and then building the binary tree by reading randomly from the array. In this case, however, the building of the tree can only progress when the entire data is available. Alternatively, a B-Tree may be used. Though the B-Tree structure is often used with large files on disks with two-level storage it can also be used for small files in one-level main storage. The B-Tree is always balanced, and it has a larger branching factor than a binary tree, resulting in a shorter average search path length. It is practically insensitive to the 'presortedness' of the input data. It is also more suitable than arrays or binary tree to handle any number of subsequent additions or deletions. This report assesses the use of ordered binary trees and B-Trees to sort and process input varying in size from 64 to 4096 integers. The input also varied in the initial ordering: ascending, largely ascending, random, largely descending, and descending orders were tested. B-Trees of order 1, 2, 4, 8, and 16 were compared before choosing order 4 as the suitable one for the tests. The times required to build the ordered binary trees and B- Trees are comparable with times required to sort the input using the better array sorting methods (O(nlogn)) such as Quick Sort and Heap Sort. It may be noted that the Heap Sort algorithm is also based on a tree structure, the complete binary tree, implemented as an array. The Quick Sort algorithm also works in a manner similar to an ordered binary tree and may be considered to be a pseudo-tree based algorithm. The different versions of binary tree algorithms used in the tests are described in Section 2. Three versions of B-Tree implementation are compared in Section 3. Effect of the order of a B-Tree on the building time is considered in Section 4. Section 5 reports the measured times to build binary- and B-Trees and compares these with the times required to sort the data using Quick Sort and Heap Sort. Section 6 describes similar comparisons of the 'task times'. The task was to read the input and printout the data in ascending order to the screen, formatted 8 integers per line and including checking for error in sorting. In all of the following sections it is assumed that the sorting is required in ascending (non decreasing) order with duplicates allowed. 2. Binary Tree s The conventional ordered binary tree building algorithm, called here Bintreel, uses non recursive search, dynamic pointer variables for the links, and the Pascal procedure new( ) to get the tree nodes. A slight improvement in the building time is achieved by implementing the binary tree as an array of records, getting the new nodes from the static array, and using array subscripts as the links. This version based on the static array is called Bintree2 here. Both Bintree1 and Bintree2 are unsuitable if the input is not in random order. Tests reported in Section 5 confirm the dramatic increase in building times when the input is in ascending or descending order. To overcome this drawback a version called here Bintree3 is developed. In this version the input is first read into an array to be used for the tree. At this stage the root and left/right links are not assigned. Then this array is read randomly. The root and left/right links of the nodes are set up during this random reading of the array. Tests described in Section 5 confirm that this approach is very successful, even allowing for the extra time required to read the data into an array, and the time required to generate an array of random subscripts to be used to read the data array randomly.

3 3 One drawback of the Bintree3 algorithm is that the binary tree is nearly but not perfectly balanced. A better alternative is to use the B-Tree structure. 3. B-Tree s The B-Tree presents an ideal solution to overcome the drawbacks of the ordered binary tree. It maintains a perfect balance for input in any order. The balance is retained under any number of subsequent additions or deletions. Its search paths are also shorter due to its large branching factor. In order to test its use for sorting and processing data in main memory the following versions of the B-Tree implementation were considered:. B-Tree1 : The B-Tree is implemented as an array of nodes, each node holding arrays of data and pointers (node-subscripts). The B-tree is built using nonrecursive search procedure. Backtracking up the tree in case of node overflow during insert operation is aided by maintaining a stack of pointers to nodes on the search path.. B-Tree2 : This is similar to B-tree1 except for one feature. In addition to the pointer to each node on the search path, the rank of the child to which the parent pointed during the search was also pushed on to the stack. This meant that the parent node when visited again due to the splitting of the child node, was not searched again for the position to place the overflown key.. B-Tree3 : This version uses dynamic implementation. The B-Tree nodes are obtained as required using the Pascal new() procedure. The records within each node are also maintained as a dynamic ordered linked list. Whereas the static array implementations B-Tree1, B-Tree2 may waste some space (some nodes may be only half full), this version uses only as much space as required. The list-links in each node, however, do take up some space. Times required to build a B-tree using the above versions B-Tree1/2/3, reading input from the file rndnos.4k containing integers in random order, are given in Table 1. Table 1. Comparison of different versions B-Tree (of order 4) Version Sorting Time (seconds) Input in random order No. of integers sorted in ascending order B-Tree B-Tree B-Tree Results of Table 1 show that there is not much difference between B-Tree1 and B- Tree2, and both are slightly faster than B-Tree3. Hence, B-Tree1, being also the simplest of the three, was chosen for the rest of the tests reported here.

4 4 4. Effect of the order of the B-Tree The order of a B-Tree is defined differently by different people. In this report the following definition ([Wirth, 1976]) is used: A B-Tree of order d has at the most 2d keys in any node. Its root node may have a minimum of one key, while all other nodes have a minimum of d keys. The order of a B-Tree has some impact on all the B-tree operations. To study this effect, times required to build B-Trees of order 1, 2, 4, 8, and 16 were measured using files containing integers as input. The times are compared in Table 2. Table 2. Effect of the order of a B-Tree on the time for building the B-Tree. Building Time (seconds) No. of integers inserted in the B-Tree Version B-Tree1 A. Input in ascending order (input file ascdnos.4k) order order order order order B. Input in random order (input file rndnos.4k) order order order order order C. Input in descending order (input file descdnos.4k) order order order order order Table 2 shows that the effect of the order of the B-Tree on building time is not pronounced when the number of elements to be inserted is small ( <= 256 ). For moderate to large input size (between 256 to 4096 elements) the building times seem to be influenced by the order of the B-Tree as well as the order of the input data. When the input is in ascending or random order, times for order 2 and 4 are better than those for order 1, 8 and 16. When the input is in descending order, times for order 4 and 8 are better than the rest. When the order of a B-Tree is large the search path length is reduced, but the time required to insert data in a node is increased. When the B-Tree order is small the search path length is increased, but the time to insert data in a node is decreased. Hence a choice of order 4 seems to be a good compromise for a B-Tree in main memory for input data in any order.

5 5. Comparison of the performance of tree-based algorithms 5 The algorithms Bintree1/2/3 (Section 2) and B-Tree1 (Section 3 and 4) were tested on several input lists with varying degrees of presortedness. The lists contained integers in ascending, descending, random, roughly ascending or roughly descending order. To supply the test lists the following files containing integers were generated.. rndnos.4k 4096 random integers were generated using the Sun Pascal function random( ) on eros, a SUN-4 minicomputer, and written to this file.. ascdnos.4k 4096 integers from rndnos.4k file were sorted in ascending order and written to this file.. descdnos.4k 4096 integers from rndnos.4k file were sorted in descending order and written to this file.. incnos.4k contained 4096 random integers in roughly ascending order. The integers were generated using equation 1: n = 1024*k + r (1) for k = 1,2,3,...,4096 where r is a random integer in the range 0 to [Hale,1992].. decnos.4k contained 4096 random integers in roughly descending order. The integers were generated using equation 2: n = 1024*(4097-k) + r (2) for k = 1,2,3,...,4096 with r as above [Hale, 1992]. The input lists were also sorted using.heap Sort [Williams, 1964], and.quick Sort [Sedgewick, 1980]. A non-recursive, fine-tuned version of Quick Sort, which uses Insertion Sort when the size of a sub-array is 20 or less and uses a random element as a pivot during the partitioning process, is implemented. In all the Tables 3 to 7 in this section the B-Tree referred to is of order 4. The times measured in the tests were obtained on an IBM PC compatible using Turbo Pascal programs. In each case the time reported is the average time of ten execution runs. The measurements include time required for the all the operations for building the trees (Bintree1/2/3 and B-Tree1) or sorting the data (Heap Sort, Quick Sort). The times measured in seconds are given in the Tables 3 to 7. They may be considered to be adequate for comparison purposes only. The programs ran very much faster when executed on eros, a Sun-4 minicomputer.

6 Table 3. Comparison of Tree-based Sorting s (Input file ascdnos.4k) _ Sorting Time (seconds) Input in ascending order 6 No. of integers sorted in ascending order _ Bintree Bintree Bintree B-Tree Heap Sort Quick Sort _ The results shown in Table 3 confirm that the times required to build an ordered binary tree (Bintree1) are very large when the input is already ordered. Static array implementation of the binary tree (Bintree2) is only marginally better. The version Bintree3, which randomizes input before use, improves the performance very significantly to the level of Quick Sort and better than Heap Sort. Times for building B-Tree are comparable to Heap Sort. Table 4. Comparison of Tree-based Sorting s (Input file incnos.4k) Sorting Time (seconds) Input in roughly ascending order No. of integers sorted in ascending order Bintree Bintree Bintree B-Tree Heap Sort Quick Sort Check Sort The results of Table 4 also confirm that the input in roughly ascending order is not suitable for building an ordered binary tree. The static array version Bintree2 is somewhat better than the dynamic version Bintree1. The version Bintree3, which randomizes input before use, improves the performance very significantly to the level of Quick Sort and much better than B-Tree and Heap Sort. Times for building B-Tree are comparable to Heap Sort.

7 Table 5. Comparison of Tree-based Sorting s (Input file rndnos.4k) 7 Sorting Time (seconds) Input in random order No. of integers sorted in ascending order Bintree Bintree Bintree B-Tree Heap Sort Quick Sort Table 5 indicates that when the data is in random order all the methods have comparable times when the input size is not very large (up to 512 elements). For large inputs Bintree3 and Quick Sort continue to be the best, but the ordinary binary tree versions Bintree1 and Bintree2 are very close to them. Table 6. Comparison of Tree-based Sorting s (Input file decnos.4k) Sorting Time (seconds) Input in roughly descending order No. of integers sorted in ascending order _ Bintree Bintree Bintree B-Tree Heap Sort Quick Sort Table 6 shows that when data is in roughly descending order, Bintree1 and Bintree2 perform poorly. Bintree3 continues to perform very well, marginally better than Quick Sort. B-Tree1 performance is close to Quick Sort and somewhat better than Heap Sort.

8 Table 7. Comparison of Tree-based Sorting s (Input file descdnos.4k) _ 8 Sorting Time (seconds) Input in descending order No. of integers sorted in ascending order _ Bintree Bintree Bintree B-tree Heap Sort Quick Sort _ Table 7 demonstrates that when input data is already sorted in reverse order the results are similar to the case of data in roughly descending order. Bintree1 and Bintree2 perform very poorly. The best performance is of Bintree3, ahead of even Quick Sort. B- Tree1 is close to Quick Sort and better than Heap Sort in this case. 6. Comparison of times required for a complete 'task' The tree-based algorithms 'sort' the input by building the tree. In the case of array based sorting algorithms the sorted data is output apparently more easily as the tree traversal algorithms are not as simple as the array traversal. To test this factor, and to give the comparison a more practical basis, times for a complete task were measured and compared. The task involved reading the input file from a disk, sorting the data into an array or building a binary or B-Tree, and output the data in ascending order to screen. The output was to be checked for any sorting error and formatted 8 integers to a line. The tree traversal part for Bintree3 and B-Tree1 was tackled using both the recursive and non-recursive options. The recursive algorithms are labeled Bintree3r and B-Tree1r. The non-recursive counter parts are labelled Bintree3nr and B-Tree1nr. The results are given in Tables 8 to 12. In all these tests the B-Tree referred to is of order 4.

9 Table 8. Task Times Comparison of Tree-based s (Input file ascdnos.4k) 9 Task Time (seconds) Task involved sorting given input on disk file and printing output on screen in ascending order 8 integers per line, including checking for sorting error. Input in ascending order No. of integers printed out in ascending order Bintree Bintree3r Bintree3nr B-Tree1r B-Tree1nr Heap Sort Quick Sort Table 8 results confirm that the conventional ordered binary tree (Bintree1) performs very poorly when the input is already ordered. However, when the input is randomized before building the tree (Bintree3r/Bintree3nr) the performance is very good, matching the performance of Quick Sort. The B-Tree times are slightly slower than Quick Sort but comparable to Heap Sort. The times for recursive and non-recursive versions of the binary tree are almost the same. Similarly there is hardly any difference between the times for recursive and non-recursive versions of the B-Tree. Table 9. Task Times Comparison of Tree-based s (Input file incnos.4k) Task Time (seconds) Task involved sorting given input on disk file and printing output on screen in ascending order 8 integers per line, including checking for sorting error. Input in roughly ascending order No. of integers printed out ascending order Bintree Bintree3r Bintree3nr B-Tree1r B-Tree1nr Heap Sort Quick Sort

10 10 Table 9 results confirm that the conventional ordered binary tree (Bintree1) performs poorly when the input is already roughly ordered. However, when the input is randomized before building the tree (Bintree3r/Bintree3nr) the performance is very good, even slightly better than Quick Sort. The B-Tree times are slightly slower than Quick Sort but comparable to Heap Sort. The times for recursive and non-recursive versions of the binary tree are almost the same. Similarly there is hardly any difference between the times for recursive and non-recursive versions of the B-Tree. Table 10. Task Times Comparison of Tree-based s (Input file rndnos.4k) Task Time (seconds) Task involved sorting given input on disk file and printing output on screen in ascending order 8 integers per line, including checking for sorting error. Input in random order No. of integers printed out ascending order Bintree Bintree3r Bintree3nr B-Tree1r B-Tree1nr Heap Sort Quick Sort Table 10 results confirm that the conventional ordered binary tree (Bintree1) performs well when the input is in random order. However, when the input is randomized again before building the tree (Bintree3r/Bintree3nr) the performance is improved, and is even slightly better than Quick Sort. The B-Tree times are slightly slower than Quick Sort but comparable to Heap Sort. For both the binary tree and the B-Tree, the recursive versions are as fast as their non-recursive counter parts.

11 Table 11. Task Times Comparison of Tree-based s (Input file decnos.4k) 11 Task Time (seconds) Task involved sorting given input on disk file and printing output on screen in ascending order 8 integers per line, including checking for sorting error. Input in roughly descending order No. of integers printed out in ascending order Bintree Bintree3r Bintree3nr B-Tree1r B-Tree1nr Heap Sort Quick Sort Check Sort Table 11 results confirm that the conventional ordered binary tree (Bintree1) performs poorly when the input is already roughly ordered. When the input is randomized before building the tree (Bintree3r/Bintree3nr) the performance is improved, and is even slightly better than Quick Sort. The B-Tree times are also comparable to Quick Sort and slightly better than Heap Sort. For both the binary tree and the B-Tree, the recursive versions are as fast as their non-recursive counter parts. Table 12. Task Times Comparison of Tree-based s (Input file descdnos.4k) Task Time (seconds) Task involved sorting given input on disk file and printing output on screen in ascending order 8 integers per line, including checking for sorting error. Input in descending order No. of integers printed out in ascending order Bintree Bintree3r Bintree3nr B-Tree1r B-Tree1nr Heap Sort Quick Sort Check Sort

12 12 Table 12 results confirm that the conventional ordered binary tree (Bintree1) performs very poorly when the input is already ordered. When the input is randomized before building the tree (Bintree3r/Bintree3nr) the performance is improved, and is even slightly better than Quick Sort. The B-Tree times are also comparable to Quick Sort and slightly better than Heap Sort. For both the binary tree and the B-Tree, the recursive versions are as fast as their non-recursive counter parts. 7. Conclusions The report highlights the use of ordered binary tree and B-Tree structures in main memory for sorting and processing data in key sequenced order. Array implementation of the ordered binary tree is seen to be a little faster than the alternative based on dynamic variables. It is seen that B-Tree of order 4 performs slightly better than B-Trees of order 1, 2, 8 or 16. Also, the array implementation of a B-Tree, works faster than the alternative based on dynamic variables. It is shown that an ordered binary tree built by randomizing the input is a very effective method for sorting/processing the input. The Bintree3 algorithm using this approach handles input data in any initial order as fast as the fine-tuned Quick Sort and better than Heap Sort or B-tree1 algorithm. The search time for a given key is somewhat smaller in Bintree3 than the logically equivalent binary search of a sorted array, as Bintree3 uses a simple key comparison while the latter needs the integer divide operation to find the 'middle' position during any search. It is shown that the array based implementation, B-Tree1, builds a B-Tree in memory as fast as the Heap Sort algorithm sorting it in an array, and only marginally slower than the fine-tuned Quick Sort and Bintree3. Times required to complete a 'task' were also compared: the task included all the operations required to print the sorted, formatted output to screen. When outputting the data in sorted order, there is not much difference between the recursive and non-recursive inorder traversals. This applies to both the ordered binary tree and the B-Tree. The simplicity of recursive version makes tree traversal almost as easy to code as array or linked list traversal. The Bintree3 algorithm is again as fast as Quick Sort and better than Heap Sort to do the task. The B-Tree1 algorithm is somewhat slower than Bintree3 but comparable with Heap Sort. Since any number of subsequent additions or deletions will not affect its balance, the B-Tree1 algorithm may be preferred where the data is volatile. B-Tree may also be handy in applications where intermediate results are to be output, where the entire input is not available in one go or its size unknown.

13 13 Acknowledgements The author would like to thank Prof. Andrzej Goscinski and Mr.Bob Hale for their encouragement, criticism and useful comments. Thanks also to Judith Jamieson for all the secretarial help. References Hale, R.P., Teague, G.J., "SCP760 Data Structures Study Guide", Session 8, p. 14. Deakin University. Kingston, J.H "s and Data Structures: Design, Correctness, Analysis" Addison-Wesley, Sydney. Knuth, D.E., "The Art of Computer Programming, Vol. 3: Sorting & Searching" Addison-Wesley, Reading, Mass. Mehlhorn, K., "Data Structures & s, Vol. 1: Sorting & Searching" Springer-Verlag, Berlin Heidelberg 1984 Sedgewick, R., "Quicksort." Garland Publishing Inc., New York & London. Williams, J.W.J., " 232: Heapsort." Communications of the A.C.M. Vol. 7, p Wirth, N., "s + Data Structures = Programs' Prentice-Hall, Inc. Englewood Cliffs, N.J.