Optimum Alphabetic Binary Trees T. C. Hu and J. D. Morgenthaler Department of Computer Science and Engineering, School of Engineering, University of C

Transcription

1 Optimum Alphabetic Binary Trees T. C. Hu and J. D. Morgenthaler Department of Computer Science and Engineering, School of Engineering, University of California, San Diego CA 92093{0114, USA Abstract. We describe a modication of the Hu{Tucker algorithm for constructing an optimal alphabetic tree that runs in O(n) time for several classes of inputs. These classes can be described in simple terms and can be detected in linear time. We also give simple conditions and a linear algorithm for determining, in some cases, if two adjacent nodes will be combined in the optimal alphabetic tree. 1 Introduction Binary trees and binary codes have many applications in various branches of science and engineering, and the subjects have been studied extensively for more than forty years. In 1952, Human [] discovered the algorithm for constructing an optimum variable-length binary code or the binary tree with minimum weighted path length known as the Human's tree. In 1959, Gilbert and Moore [3] considered an alphabetic constraint which restricts the ordering of leaves in the binary tree and they found an algorithm for constructing an optimum alphabetic binary tree in O(n 3 ) time. Their technique is based on dynamic programming and is illustrated in many computer science textbooks. In 191, their solution technique was rened to O(n 2 ) time by Knuth [9], and a new algorithm was discovered by Hu and Tucker [6] which needs O(n log n) time and O(n) space. The implementation of the Hu-Tucker algorithm in O(n log n) time was pointed out by Knuth [10], and several improvements over the original algorithm was reported in Hu [4]. A similar algorithm was discovered by Garsia and Wachs in 19 [2]. But the time complexity remained at O(n log n). The applicability of Human's algorithm and the Hu-Tucker algorithm were generalized to other cost functions in [5]. Recently, the equivalence of alphabetic binary search tree and alphabetic binary code is noted by [1, 13]. Actually, Human's algorithm can be implemented in linear time if the weights of leaves are already sorted as noted by Larmore [12]. Also, testing the optimality of a given alphabetic binary tree is shown to be linear in several cases by Ramanan [15]. Thus, the search for a new linear algorithm for constructing an optimum alphabetic binary tree seems to be a natural one. Recently Klawe and Mumsy [8] and Przytycka and Larmore [14] have both obtained algorithms that run in less than O(n log n) time for special cases of weight sequences. Here we basically achieve the same results without introducing as much new terminology. Also, we have found simple conditions for two adjacent nodes to be combined in the optimal alphabetic tree which can be detected in O(n) time.

2 2 Problem and Known Results We use the denitions in Knuth [10] and consider an extended binary tree where every internal node has two children and every leaf has no children. The internal node is represented by a circle and the leaf by a square. The root of the extended binary tree (binary tree for brevity from now on) is said to be at the level zero, and its two children at the level one. Given a sequence of n leaves with positive weights w i, the problem is to construct an alphabetic binary tree with min P w i l i where l i is the level of the leaf with weight w i, and the ordering of leaves in the sequence is to be maintained. Due to the alphabetic constraint, a square node (a leaf) can only be combined with its left or right neighboring square, or a circular node whose descendants are a subsequence of squares next to the given square. Two circular nodes can be combined only if they are both roots of subtrees with leaves from two adjacent subsequences. A related problem is to nd the min-cost forest built on a sequence of positiveweight leaves where the forest has exactly k circles. (The alphabetic tree is a special case with k = n? 1.) We shall use w i to denote the node with weight w i or the node itself. Also, w ij to denote the parent of w i and w j with its weight w i + w j. The parent will occupy the position of the left child. A node w bc is said to be crossed over by another node w ad if one of the children of w bc is between the children of w ad. Two nodes in a sequence are called a compatible pair if they are adjacent in the sequence or if all nodes between them are circular nodes. Among all compatible pairs in a weight sequence, the one with minimum weight is called the minimum compatible pair (mcp). To break the ties among weights, we shall adopt the convention that the node on the left has smaller weight. A pair of nodes (w b ; w c ) is a local minimum compatible pair (lmcp) if w a > w c for all nodes w a compatible with w b w b w d for all nodes w d compatible with w c Hu-Tucker algorithms can be described in three steps: 1. Combination. Keep combining lmcp until a tree T is obtained. 2. Level Assignment. Find the level number of every leaf in tree T, say l 1 ; l 2 ; : : : ; l n. 3. Reconstruction. Use a stack algorithm to construct an alphabetic binary tree based on l 1 ; l 2 ; : : : ; l n. Note that both steps 2 and 3 take linear time while step 1 takes O(n log n) time. A weight sequence is called an increasing sequence if A decreasing sequence if w 1 w 2 : : : w n w 1 > w 2 > : : : > w n

3 A valley sequence if A bimonotonal increasing sequence if w 1 > w 2 > : : : > w i w i+1 : : : w n w 1 + w 2 w 2 + w 3 w 3 + w 4 : : : w n?1 + w n A bimonotonal decreasing sequence if w 1 + w 2 > w 2 + w 3 > w 3 + w 4 > : : : > w n?1 + w n A bimonotonal valley if it contains a single lmcp. Lemma 1. Let w a be an arbitrary node in a weight sequence consisting of circles and squares, and w i be the smallest node compatible to w a. If any lmcp is combined in the weight sequence and w d becomes compatible with w a, then w i w d. In other words, the smallest node compatible to any node w a will always remain the same. In particular, an lmcp in a weight sequence will remain as other lmcp are successively combined. Proof. See Hu [4]. Lemma 2. If in a weight sequence : : : w a ; w b ; w c ; w d ; : : : where w bc is crossed over by w cd in the construction of T, then l bc l ad, i.e., the four nodes are either at the same level or the two nodes w b and w c are at lower levels than w a and w d. Proof. See Hu [4]. 3 Summary of Key Observations We show that for weight sequences in several classes, the optimum alphabetic binary tree can be computed in linear time. We present these classes from the simplest to the most complicated. A binary tree is called a complete binary tree where all its leaves are at the same level or two adjacent levels. Obviously, if all leaves are of the same weight in a weight sequence and n = 2 k, then the optimum alphabetic binary tree is a complete binary tree with all leaves at the same level. If 2 k < n < 2 k+1, then all the leaves will be at two adjacent levels with 2(n? 2 k ) leaves at the lower level and (2 k+1? n) leaves at the higher level. Our rst observation is that for n leaves of almost the same weight, the optimum alphabetic binary tree is again a tree whose leaves are at two adjacent levels. We call such a weight sequence an almost uniform sequence. (Precise denition to be given later in Section 4.) In addition, given a sequence of n nodes of almost uniform weight (circle or square), with n = 2 k, the optimum alphabetic tree is again a complete binary tree where these circles and squares occupy a single level.

4 The second observation is that if the n leaves form an increasing sequence, we can perform the combination phase of the Hu-Tucker algorithm in linear time by queuing the circular nodes as they are created. Compatible circular nodes are placed at the end of a sorted queue as they are formed. Only the bottom two nodes of this queue need to be examined to determine the next lmcp. This is essentially the same modication reported by Larmore [12] that allows Human's algorithm to run in linear time on a sorted sequence. We can also perform the combination phase for a bimonotonal increasing sequence in linear time, since the bimonotonal condition guarantees that the leftmost square node will always be smaller than the node two to the right. Finally, a bimonotonal valley sequence consisting of a bimonotonal decreasing sequence followed by a bimonotonal increasing sequence is still linear, since we are guaranteed a single queue of circular nodes by the presence of a single initial lmcp. Figure 1 shows the operation of the modied combination phase on a bimonotonal valley sequence. At each combination in Fig. 1, the lmcp is found among the six nodes consisting of: the bottom two circular nodes in the queue, the two square nodes to the left of the queue, or the two square nodes to the right of the queue. For example, after the nal step shown in Fig. 1, the next lmcp cannot include the circular nodes with weights 11 or 13 due to construction of the queue. Thus, determining which nodes to combine at each step requires only constant time for a bimonotonal valley sequence. Any weight sequence can be broken into a set of bimonotonal valley sequences in linear time. The only thing that prevents the modied algorithm from running in linear time for an arbitrary weight sequence is the need to \merge" two queues when the mountaintop (dened in Section 4) between their valleys combines. Each merge requires linear time, but as O(n) queues could be merged at one time, the merging may require O(n log n) time. In the next section, we describe a class of initial weight sequences that do not require O(n) queues to be merged, and thus require only linear time for construction of their optimum alphabetic binary tree. 4 Linear Classes of Weight Sequences The previous section gave a modication of the Hu-Tucker algorithm that runs in linear time for a bimonotonal valley sequence. By restricting the number of queues of circular nodes that need to be merged at one time, the algorithm also runs in linear time for a larger class of initial sequences. When two valleys are adjacent, the point at which the bimonotone sequence changes from increasing to decreasing is called the mountaintop. This concept is the dual of lmcp, and could be called the locally maximum compatible pair. More formally: Denition 3. A mountaintop is the point between the two adjacent square nodes whose combined weight is locally the greatest. The weight of the mountaintop is the greater weight of these two nodes.

5 Fig. 1. First seven steps in the combination phase for the initial bimonotonal valley sequence, 3, 4, 1, 3, 2, 4, 3, 5, 3, 6

6 Adjacent mountaintops are the closest mountaintop on either the left or right of a node. If we extend the initial sequence by adding two special nodes of innite weight, one at each end, then every node, circular or square, has two adjacent mountaintops (except these two new innite weight nodes). A mountaintop combines (forming circular nodes) when both square nodes have combined. It is at the point in the algorithm when a mountaintop combines that queues of circular nodes are merged. Clearly, if the number of valleys in the weight sequence is bounded by a constant k, the modied Hu-Tucker algorithm runs in O(n log k) = O(n) time. We may be required to merge all k queues at once if the k? 1 mountaintops have the same weight. An interesting property of a queue of circular nodes is that they are all almost uniform. We can use this property to broaden the class of sequences where Hu- Tucker runs in linear time. Denition 4. A set of nodes is almost uniform if any two nodes have combined weight greater than or equal to any weight in the set. Node set W is almost uniform if and only if 8w 1 ; w 2 ; w 3 2 W; w 1 + w 2 w 3 Lemma 5. A set of compatible circular nodes created during phase I of the Hu- Tucker algorithm is almost uniform. Proof. Assume that the two smallest weight circular nodes sum to a total weight less than the weight of the largest circular node. Then the largest node was not created from an lmcp, since the two smallest weight nodes are locally minimum, contrary to the Hu-Tucker algorithm. The queues of two adjacent valleys combine when the largest square node (w T ) adjacent to their common mountaintop combines. When w T combines, a new circular node with weight greater than that of the last node in either queue is formed. In fact, the largest circular node on either queue (w c ) must have weight less than or equal to twice the weight of the mountaintop. w c 2w T Let the children of w c be w x and w y. Then by the fact that w T has not yet combined and the denition of mountaintop, w x w T and w y w T. After w T combines, all circular nodes in the two queues will have weight at least w T. It is well known that an optimum alphabetic binary tree can be created in linear time from a valley sequence. Now, let us consider two adjacent valley sequences with three mountaintops, e.g. w L > w 1 > > w i < < w T > w j > > w k < < w R Then, a queue of circular nodes will be created in linear time in the valley between w L and w T and similarly a queue of circular nodes will be created in the valley between w T and w R.

7 If w L and w R are both much larger than w T, then w T is combined into a circular node rst. And we will have two queues of circular nodes between w L and w R, with the parent of w T being the largest circular node. When this happens, the next combination is the smallest node in one queue with the smallest node in the other queue (or possibly with the second smallest node in its own queue). In other words, each combination takes constant time inside the valley bounded by the two mountaintops w L and w R. Eectively, we have a single valley, if w T min(w L =2; w R =2). If we have k adjacent valleys between w L and w R, then we may have k queues of circular nodes. To nd the next combination, we can select the two smallest nodes among the front elements in the k queues, say 1 and 1 (with 1 < 1 ) and then compare the next node in the queue ( 2 ) with 1 to determine min( ; ). In other words, as long as k is constant and is not a function of n, we can successively combine nodes in linear time. We say that the k valleys are nested inside the big valley w L and w R if the largest mountaintop among the inside mountaintops is less than half min(w L ; w R ). Of course, all the comments about valley sequences apply equally to bimonotonal valley sequences. Denition 6. Two valleys v 1 and v 2 are nested inside a larger valley V with mountaintops of weight w L and w R if the mountaintop separating v 1 and v 2 has weight w min(w L ; w R )=2. Denition. A proper valley has no nested valleys. A k-proper valley has at most k nested valleys, either proper valleys or k-proper valleys. Dened recursively, a k-proper valley is either: 1. a proper valley 2. a valley with at most k nested adjacent proper valleys 3. a valley with at most k nested adjacent k-proper valleys We now expand the class of weight sequences for which the modied Hu- Tucker algorithm requires only linear time to the set of all k-proper valleys. Theorem 8. The optimum alphabetic binary tree for an initial weight sequence W of n nodes that forms a k-proper valley can be computed in O(n) time for any constant k. Further, we can determine whether a sequence is a k-proper valley is O(n) time. Proof. We consider each case separately. Case 1: W is a proper valley. Since it has no nested valleys, W consists of a single bimonotonal valley, with a single lmcp. At each step in the combination phase, only one lmcp exists. When newly created circular nodes are placed in a queue, only six nodes (four square, two circular) could participate in that lmcp. Thus, each combination requires constant time, and the entire combination phase requires O(n) time. As the remaining phases are also linear, the entire tree is constructed in linear time.

8 Case 2: W is a valley with at most k nested adjacent proper valleys. Each nested proper valley requires linear time to combine all contained square nodes. When a mountaintop is combined, the adjacent compatible queues are merged. This requires at most k merges, each needing at most O(n) comparisons. Once all queues have been merged, this case resembles Case 1. Case 3: W is a valley with at most k nested adjacent k-proper valleys. Recursively, each nested k-proper valley can be handled in time linear in its size. Again, combining adjacent valleys by merging queues of circular nodes is linear, and must only be done k times. 4.1 Permanent Circular Nodes During the construction of an optimum k-circle forest, a given circle in the (k?1) circle forest may be split up in the optimum k-circle forest. For example, consider the initial sequence 4, 2, 3, 4. The optimum 1-circle forest combines the nodes weighing 2 and 3; these are not combined in the 2-circle or 3-circle forests. Would there exist a circle which will never be split up once it is formed? We shall call such a circle a permanent circular node (a permanent node). When we can detect such a circular node, then the circular node can be treated as a single square node in the weight sequence during later combinations. Lemma 9 Gilbert & Moore. Let w a ; w 1 ; w 2 ; : : : ; w k ; w d be a weight sequence where kx i=1 w i min(w a ; w d ) then the subsequence w 1 ; w 2 ; : : : ; w k will form a binary tree whose root is a permanent node. Proof. See [3]. Lemma 10. Let w a ; w b ; w c ; w d be any consecutive four nodes in a sequence and 1. w a > w c 2. w b w d 3. (w b + w c ) < max(w a ; w d ) Then the parent of w b and w c is a permanent node. Proof. Conditions 1 and 2 imply that w b and w c are a local minimum compatible pair. Without loss of generality, assume that w a > w d. Once w bc is formed as a circular node, both w a and w bc are compatible with w d and w a > w bc. Thus, w d combines with w bc before combining with w a. If w a should be combined with another node to its left, there cannot be a new node with its weight smaller than w bc which would be compatible with w d, by lemma 1. If w d should be combined with another node to its right, that circular node would also be compatible with w a and w bc.

9 Thus, the node w d or its ancestor will never cross over w bc to combine with w a or its ancestor. Node w bc will combine with a node adjacent to itself, and will not be crossed over. In other words, w bc is a permanent node. To show the power of identifying permanent nodes, consider the following weight sequence. 1; 2; 3; 4; 2; 1; 8; 10; 1 Here w a = 1, w b = 2, w c = 3, w d = 4, and we shall show it as After 2 and 3 are combined into 5, we move forward until the condition of permanent node is again satised And then move backward (after = 3) Another example is And at this stage, the sequence is almost uniform and no more permanent nodes can be created. Lemma 11. We can identify and combine all permanent nodes in O(n) time. Proof. In a scan from left to right of the initial weight sequence, at each node B, label its neighbors A B C D and do the following: If A > C, B D and B + C < max(a; D) replace nodes B and C with new square node B + C and back up to consider node A again, else do nothing. If the node has no left neighbor or less than two right neighbors, do nothing. At each step, the number of nodes to the right of the node being considered either decreases by one (do nothing case), or remains unchanged. If it remains unchanged, the number of nodes to the left decreases by one. Thus, we can perform at most 2n total steps before the scan terminates. Since permanent nodes can be found and combined in linear time, we can expand our class of weight sequences with optimum alphabetic trees computable in linear time to any sequence which after nding and combining permanent nodes is a k-proper valley.

10 5 Conclusion We have shown that a modication of the Hu{Tucker algorithm will run in linear time for many classes on input sequences. By scanning a given input sequence for mountaintops and counting adjacent groups of almost uniform mountaintops, membership in such a class can also be determined in linear time. We have also given a linear algorithm for nding and combining permanent circular nodes detectable on the initial sequence in a four node moving window. References 1. Arne Andersson. A note on searching in a binary search tree. Software{Practice and Experience, 21(10):1125{1128, A. M. Garsia and M. L. Wachs. A new algorithm for minimum cost binary trees. SIAM Journal on Computing, 6(4):622{642, E. N. Gilbert and E. F. Moore. Variable length binary encodings. Bell System Technical Journal, 38:933{968, T. C. Hu. Combinatorial Algorithms. Addison-Wesley, Reading, MA, T. C. Hu, D. J. Kleitman, and J. K. Tamaki. Binary trees optimum under various criteria. SIAM Journal on Applied Mathematics, 3(2):246{256, T. C. Hu and A. C. Tucker. Optimal computer search trees and variable-length alphabetic codes. SIAM Journal on Applied Mathematics, 21(4):514{532, D. A. Human. A method for the construction of minimum redundancy codes. Proceedings of the IRE, 40:1098{1101, M. M. Klawe and B. Mumey. Upper and lower bounds on constructing alphabetic binary trees. In Proceedings of Fourth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 185{193, D. E. Knuth. Optimum binary search trees. Acta Informatica, 1:14{25, D. E. Knuth. The Art of Computer Programming, Volume III: Sorting and Searching. Addison-Wesley, Reading, MA, L. L. Larmore. A subquadratic algorithm for constructing approximately optimal binary search trees. Journal of Algorithms, 8(4):59{591, L. L. Larmore. Height restricted optimal binary trees. SIAM Journal on Computing, 16(6):1115{1123, N. Nakatsu. An alphabetic code and its application to information retrieval. Transactions of the Information Processing Society of Japan, 34(2):312{19, T.M. Przytycka and L.L. Larmore. The optimal alphabetic tree problem revisited. In Proceedings of 21st International Colloquium on Automata, Languages, and Programming, pages 251{262. Springer-Verlag, July P. Ramanan. Testing the optimality of alphabetic trees. Theoretical Computer Science, 93(2):29{301, This article was processed using the LATEX macro package with LLNCS style