Funnel Heap - A Cache Oblivious Priority Queue

Transcription

1 Alcom-FT Technical Report Series ALCOMFT-TR Funnel Heap - A Cache Oblivious Priority Queue Gerth Stølting Brodal, Rolf Fagerberg Abstract The cache oblivious model of computation is a two-level memory model with the assumption that the parameters of the model are unknown to the algorithms. A consequence of this assumption is that an algorithm efficient in the cache oblivious model is automatically efficient in a multi-level memory model. Arge et al. recently presented the first optimal cache oblivious priority queue, and demonstrated the importance of this result by providing the first cache oblivious algorithms for graph problems. Their structure uses cache oblivious sorting and selection as subroutines. In this paper, we devise an alternative optimal cache oblivious priority queue based only on binary merging. We also show that our structure can be made adaptive to different usage profiles. Keywords: Cache oblivious algorithms, priority queues, memory hierarchies, merging BRICS (Basic Research in Computer Science, funded by the Danish National Research Foundation), Department of Computer Science, University of Aarhus, Ny Munkegade, DK-8000 Århus C, Denmark. {gerth,rolf}@brics.dk. Partially supported by the Future and Emerging Technologies programme of the EU under contract number IST (ALCOM-FT). Supported by the Carlsberg Foundation (contract number ANS-0257/20). 1

2 1 Introduction External memory models are formal models for analyzing the impact of the memory access patterns of algorithms in the presence of several levels of memory and caches on modern computer architectures. The cache oblivious model, recently introduced by Frigo et al. [10], is based on the I/O model of Aggarwal and Vitter [1], which has been the most widely used model for developing external memory algorithms; see the survey by Vitter [11]. Both models assume a two-level memory hierarchy where the lower level has size M and data is transfered between the two levels in blocks of B elements. The difference is that in the I/O model the algorithms are aware of B and M, whereas in the cache oblivious model these parameters are unknown to the algorithms and I/Os are handled automatically by an optimal off-line cache replacement strategy. Frigo et al. [10] showed that an efficient algorithm in the cache oblivious model is automatically efficient on an each level of a multi-level memory model. They also presented optimal cache oblivious algorithms for matrix transposition, FFT, and sorting. Cache oblivious search trees which match the search cost of the standard (cache aware) B-trees [3] were presented in [4, 5, 6, 8]. Cache oblivious algorithms for computational geometry problems were developed in [4, 7]. The first cache oblivious priority queue was recently developed by Arge et al. [2], and gave rise to several cache oblivious graph algorithms [2]. Their structure uses existing cache oblivious sorting and selection algorithms as subroutines. In this paper, we present an alternative optimal cache oblivious priority queue, Funnel Heap, based only on binary merging. Essentially, our structure is a single heap-ordered tree with binary mergers in the nodes and buffers on the edges. It was inspired by the cache oblivious merge-sort algorithm Funnelsort presented in [10] and simplified in [7]. As for the priority queue of Arge et al., our data structure supports the operations Insert and DeleteMin using amortized O( 1 B log M/B N B ) I/Os per operation, under the so-called tall cache assumption M B 2. Here, N is the total number of elements inserted. For a slightly different algorithm we give a refined analysis, showing that the priority queue adapts to different profiles of usage. More precisely, we show that the ith insertion uses amortized O( 1 B log M/B N i B ) I/Os, with N i defined in one of the following three ways: (a) the number of elements present in the priority queue when performing the ith insert operation, (b) if the ith inserted element is removed by a DeleteMin operation prior to the jth insertion then N i = j i, or (c) N i is the maximum rank that 2

3 the ith inserted element has during its lifetime in the priority queue, where rank denotes the number of smaller elements present in the priority queue. DeleteMin is amortized for free since the work is charged to the insertions. These results extends the line of research taken in [9], where (a) and (c) are called size profile and max depth profile, respectively. This paper is organized as follows. In Section 2 we introduce the concept of mergers and in Section 3 we describe our priority queue. Section 4 gives the analysis of the presented data structure. Finally, Section 5 gives the analysis based on different profiles of usage. 2 Mergers Our data structure is based on binary mergers. A binary merger takes as input two sorted streams of elements and delivers as output the sorted stream formed by merging of these. One merge step moves an element from the head of one of the input streams to the tail of the output stream. The heads of the input streams and the tail of the output stream reside in buffers holding a limited number of elements. A buffer is simply an array of elements, plus fields storing the capacity of the buffer and pointers to the first and last elements in the buffer. Binary mergers may be combined to binary merge trees by letting the output buffer of one merger be an input buffer of another in other words, binary merge trees are binary trees with mergers at the nodes and buffers at the edges. The leaves of the tree contains the streams to be merged. Procedure Fill(v) while v s output buffer is not full if left input buffer empty Fill(left child of v) if right input buffer empty Fill(right child of v) perform one merge step Figure 1: The merging algorithm An invocation of a merger is a recursive procedure which performs merge steps until its output buffer is full (or both input streams are exhausted). 3

4 Input buffers Output buffer Figure 2: A 16-merger consisting of 15 binary mergers. Shaded regions are the occupied parts of the buffers. If during the invocation an input buffer gets empty (but the corresponding stream is not exhausted), the input buffer is recursively filled by an invocation of the merger having this buffer as its output buffer. If both input streams of a merger get exhausted, the corresponding output stream is marked as exhausted. The procedure (except for the issue of exhaustion) is shown in Figure 1 as the procedure Fill(v). A single invocation Fill(r) on the root r of the merge tree will produce a stream which is the merge of the streams at the leaves of the tree. One particular merge tree is the k-merger. In this paper, k = 2 i for some positive integer i. A k-merger is a perfect binary tree of k 1 binary mergers with appropriate sized buffers on the edges, k input streams, and an output buffer at the root of size k 3. A 16-merger is illustrated in Figure 2. The sizes of the buffers are defined recursively: Let the top tree be the subtree consisting of all nodes of depth at most i/2, and let the subtrees rooted by nodes at depth i/2 + 1 be the bottom trees. The edges between nodes at depth i/2 and depth i/2 + 1 have associated buffers of size k 3/2, and the sizes of the remaining buffers is defined by recursion on the top tree and the bottom trees. To achieve I/O efficiency in the cache oblivious model, the layout of k- merger is also defined recursively. The entire k-merger is laid out in memory in contiguous locations, first the top tree, then the middle buffers, and finally the bottom trees, and this layout is applied recursively within the top and 4

5 I A 1 v 1 B 1 S 11 S 12 A i v i A i+1 B i K i S i1 S i2 S iki Figure 3: The priority queue based on binary mergers. each of the bottom trees. The k-merger structure was defined by Frigo et al. [10] for use in their cache oblivious mergesort algorithm Funnelsort. The algorithm described above for invoking a k-merger appeared in [7], and is a simplification of the original one. For both algorithms, the following lemma holds [7, 10]. Lemma 1 The invocation of the root of a k-merger uses O(k+ k3 B log M/B k 3 ) I/Os. The space required for a k-merger is O(k 2 ), not counting the space for the input and output streams. 3 The priority queue In this section, we present a priority queue based solely on binary mergers. Our data structure consists of a sequence of k-mergers of doubleexponentially increasing k, linked together in a list by binary mergers and buffers as depicted in Figure 3. In the figure, circles denote binary mergers, rectangles denote buffers, and triangles denote k-mergers. Note that the entire structure constitutes a single binary merge tree. More precisely, let k i and s i be values defined inductively as follows, (k 1,s 1 ) = (2,8), s i+1 = s i (k i + 1), k i+1 = s i+1 1/3, (1) 5

6 where x denotes the smallest power of two above x, i.e. x = 2 log x. Link i in the linked list consists of a binary merger v i, two buffers A i and B i, and a k i -merger K i with k i input buffers S i1,...,s iki. The output buffer of v i is A i, and its input buffers are B i and the buffer A i+1 from the next link. The output buffer of K i is B i. The size of both A i and B i is k 3 i, and the size of each S ij is s i. Link i has an associated counter c i, which is an integer between one and k i + 1 (inclusive), initially of value one. It will be an invariant that S ici,...,s iki are empty. Additionally, the structure contains one insertion buffer I of size s 1. All buffers contain a (possibly empty) sorted sequence of elements. The structure is laid out in memory in the order I, link 1, link 2,..., and within link i the layout order is A i, B i, K i, S i1, S i2,..., S iki. The linked list of buffers and mergers constitute one binary tree T with root v 1 and with sorted sequences of elements on the edges. We maintain the invariant that this tree is heap-ordered, i.e. when traversing any path towards the root, elements will be passed in decreasing order. Note that the invocation of a binary merger maintains this invariant. The invariant implies that if buffer A 1 is non-empty, the minimum element in the queue will be in A 1 or in I. To perform a DeleteMin operation, we first check whether buffer A 1 is empty, and if so invoke Fill(v 1 ). We then compare the smallest element in A 1 to the smallest element (if any) in I, remove the smaller of these from its buffer, and return it. To perform an Insert operation, the new element is inserted in I while rearranging the contents of the buffer to maintain sorted order. If the number of elements in I is now seven or less, we stop. If the number of elements in I becomes eight, we perform the following sweep operation, where i is the lowest index for which c i k i. The sweep operation will move the content of links 1,...,i 1 to the destination buffer S ici. It traverses the path p in T from A 1 to S ici and for each buffer on this path records how many elements it currently contains. It forms a sorted stream σ 1 of the elements in the buffers on the part of p from A i to S ici by traversing that part of the path. It forms a sorted stream σ 2 of all elements in links 1,...,i 1 and in buffer I by marking A i as exhausted and calling DeleteMin repeatedly. It then merges σ 1 and σ 2 into a single stream σ, inserts the front elements of σ in the buffers on the path p during a downwards traversal in such a way that all these buffers contain the same numbers of elements as before the insertion, and inserts the remaining part of σ in S ici. Finally, it resets c l to one for l = 1,2,...,i 1 and increments c i. 6

7 4 Analysis 4.1 Correctness Correctness of DeleteMin is immediate by the heap-order invariant. For Insert we must show that the invariant is maintained and that S ici does not overflow during a sweep ending in link i. After an Insert, the new contents in the buffers on the path p are the smallest elements in σ, with a distribution exactly as the old contents. Hence, an element on this path can only be smaller than the element occupying the same location before the operation. It follows that heap-order is maintained. Call B l, K l, and S l1,...,s lkl the lower part of link l. The lower part of link l is emptied each time c l is reset, so it never contains more than the number of elements inserted into S l1,s l2,...,s lkl since last time c l was reset. If follows by induction on i that the number of elements inserted into S ici during a sweep ending in link i is at most s 1 + i 1 j=1 k js j = s i. 4.2 Complexity Most of the work performed is the movement of elements upwards in the tree T during the invocations of the binary mergers in T. In the analysis, we account for the I/Os incurred during the filling of an output buffer of a merger by charging them evenly to the elements filled into the buffer. The exception is when an A i or B i buffer is not filled completely due to exhaustion of the corresponding input buffers, where we account for the I/Os by other means. We claim that the number of I/Os charged to an element during its ascent in T from an input stream of K i to the buffer A 1 is O( 1 B log M/B s i ), if we identify elements residing in buffers on the path p before a sweep with those residing at the same positions in these buffers after the sweep. To prove the claim, we assume that the maximal number of small links are kept in cache always the optimal cache replacement strategy of the cache oblivious model can only incur fewer I/Os. More precisely, let i be the space occupied by links 1 to i. From (1) we have s i 1/3 k i < 2s i 1/3, so the Θ(s i k i ) space usage of S i1,...,s iki is Θ(k i 4 ), which dominates the space usage of link i. Also from (1) we have s i 4/3 < s i+1 < 3s i 4/3, so s i and k i grows doubly-exponentially with i. Hence, i is dominated by the space usage of link i, implying i = Θ(k i 4 ). We let i M be the largest i for which i M and assume that links 1 to i M are kept in cache always. 7

8 Consider the ascent of an element from K i to B i for i > i M. By Lemma 1, each invocation of K i incurs O(k i + k i 3 B log M/B k 3 i ) I/Os. From M < im +1 and the above discussion, we have M = O(k 4 i ). The tall cache assumption B 2 M gives B = O(k 2 i ), which implies k i = O(k 3 i /B). As we are not counting invocations of K i where B i is not filled completely, i.e. where K i is exhausted, it follows that each element is charged O( 1 B log M/B k 3 i ) = O( 1 B log M/B s i ) I/Os to ascend through K i and into B i. The element can also be charged during insertion into A j for j = i M,...,i. The filling of A j incurs O(1 + A j /B) I/Os. From B = O(k im +1 2 ) = O(k 8/3 im ) and A j = k 3 j, we see that the last term dominates. Therefore an element is charged O(1/B) per buffer A j, as we only charge when the buffer is filled completely. From M = O(k im +1 4 ) = O(s 16/9 im ), we by the doubly-exponentially growth of the s j values have i i M = O(log log M s i ) = O(log M s i ) = O(log M/B s i ), where the last equality follows from the tall cache assumption. Hence, the ascent through K i dominates over insertions into the A j s, and the claim is proved. To prove the complexity stated in the introduction, we note that by induction on i, at least s i insertions take place between each sweep ending in link i. A sweep ending in link i inserts at most s i elements in S ici. We let the last s i insertions preceeding the sweep pay for the I/Os charged to these elements during their later ascent through T. By the claim above, this cost is O( 1 B log M/B s i ) I/Os per insertion. We also let these insertions pay for the I/Os incurred by the sweep during the formation and placement of streams σ 1, σ 2, and σ, and for I/Os incurred by filling buffers which become exhausted. We claim that these can be covered without altering the O( 1 B log M/B s i ) cost per insertion. The claim is proved as follows. The formation of σ 1 is done by a traversal of the path p. By the specified layout of the data structure (including the layout of k-mergers), this traversal is part of a linear scan of the part of memory between A 1 and the end of K i. Such a scan takes O(( i 1 + A i + B i + K i )/B) = O(k 3 i /B) = O(s i /B) I/Os. The formation of σ 2 has already been accouted for by charging ascending elements. The merge of σ 1 and σ 2 into σ and the placement of σ are not more costly than a traversal of p and S ici, and hence also incur O(s i /B) I/Os. To account for the I/Os incurred when filling buffers which become exhausted, we note that B i, and therefore also A i, can only become exhausted once between each sweep ending in link i. From A i = B i = k 3 i = Θ(s i ) it follows that charging each sweep ending in link i an additional cost of O( s i B log M/B s i ) will cover all such fillings, and the claim is proved. 8

9 In summary, charging the last s i insertions preceeding a sweep ending in link i a cost of O( 1 B log M/B s i ) I/Os each will cover all I/Os incurred by the data structure. Given a sequence of operation on an initial empty priority queue, let i max be the largest i for which a sweep ending in link i takes place. We have s imax N, where N is the number of insertions in the sequence. An insertion can be charged by at most one sweep ending in link i for i = 1,...,i max, so by the doubly-exponentially growth of s i, the number of I/Os charged to an insertion is O ( k=0 ) ( ) 1 1 B log M/B N (3/4)k = O B log M/B N 5 Profile Adaptive Performance To make the performance dependent on N l, we make the following changes to our priority queue. Let r i denote the number of elements residing in the lower part of link i. The value of r i is stored at v i and will only need to be updated when removing an element from B i and when a sweep operation creates a new S ij list (in the later case r 1,...,r i 1 are reset to zero). The only other modification is the following change to the sweep operation of the insertion algorithm. Instead of finding the lowest index i where c i k i, we find the lowest index i where either c i k i or r i k i s i /2. If c i k i, the sweep operation proceeds as described Section 3, and c i is incremented by one. Otherwise c i = k i + 1 and r i k i s i /2, in which case we will recycle one of the S ij buffers. If there exists an input buffer S ij which is empty, we use S ij as the destination buffer for the sweep operation. If all S ij are nonempty, the two input buffers S ij1 and S ij2 with the smallest number of elements have a total of at most s i elements. Assume without loss of generality min S ij1 min S ij2. We merge the content of S ij1 and S ij2 into S ij2. By merging from the largest elements this merge only requires reading and writing each element one. Since mins ij1 min S ij2 the heap order remains satisfied. Finally we apply the sweep operation with S ij1 as the destination buffer. 5.1 Analysis The correctness follows as in Section 4. For the complexity, we as in Section 4 only have to consider the case where i > i M. We note that in the modified algorithm, the additional number of I/Os required by a sweep operation is O(k i + s i /B) I/Os, which is dominated by the O( s i B log M/B s i ). 9

10 I/Os required to perform the sweep operation. We will argue that the sweep operation collects Ω(s i ) elements from links 1,...,i 1 which have been inserted since the last sweep ending in link i or larger. Furthermore, for half of these elements the value N l is Ω(s i ). The claimed amortized complexity O( 1 B log M/B N l ) then follows as in Section 4, except that we now charge the cost of the sweep operation to these Ω(s i ) elements. The main property of the modified algorithm is captured by the following invariant: For each i, the links 1,...,i contain in total at most ij=1 A j = i j=1 kj 3 elements which have been removed from A i+1 by the binary merger v i since the last sweep ending a link i + 1 or larger. After a sweep ending at link i + 1, we define all elements in A j to have been removed from A l, where 1 j < l i + 1. When an element e is removed from A i+1 and is output to A i, then all elements in the lower part of link i must be larger than e. All elements removed from A i+1 since the last sweep operation ending at link i+1 or larger were smaller than e. These elements must either be stored in A i or have been removed from A i by the merger in v i 1. It follows that at most i 1 j=1 A j + A i 1 elements removed from A j+1 are present in links 1,...,i. Hence, the invariant remains valid after moving e from A i+1 to A i. By definition, the invariant remains valid after a sweep operation. A sweep operation ending at link i will create a list with at least s 1 + i 1 j=1 k js j /2 s i /2 elements. From the above invariant at least t = s i /2 i 1 j=1 k3 j = Ω(s i ) elements must have been inserted since the last sweep operation ending at link i or larger. Finally, for each of the three definitions of N l in Section 1 we for at least t/2 of the elements have N l t/2: (a) For each of the t/2 most recently inserted elements, at least t/2 elements were inserted when these elements where inserted. (b) For each of the t/2 earliest inserted elements, at least t/2 other elements have been inserted before they themselves get deleted. (c) The t/2 largest elements each have (maximum) rank at least t/2. This proves the complexity stated in Section 1. References [1] A. Aggarwal and J. S. Vitter. The input/output complexity of sorting and related problems. Communications of the ACM, 31(9): , Sept

11 [2] L. Arge, M. A. Bender, E. D. Demaine, B. Holland-Minkley, and J. I. Munro. Cache-oblivious priority queue and graph algorithm applications. In Proc. 34th Ann. ACM Symp. on Theory of Computing. ACM Press, To appear. [3] R. Bayer and E. McCreight. Organization and maintenance of large ordered indexes. Acta Informatica, 1: , [4] M. Bender, R. Cole, and R. Raman. Exponential structures for cacheoblivious algorithms. In Proc. 29th International Colloquium on Automata, Languages, and Programming (ICALP), To appear. [5] M. A. Bender, E. Demaine, and M. Farach-Colton. Cache-oblivious B- trees. In Proc. 41st Ann. Symp. on Foundations of Computer Science, pages IEEE Computer Society Press, [6] M. A. Bender, Z. Duan, J. Iacono, and J. Wu. A locality-preserving cache-oblivious dynamic dictionary. In Proc. 13th Ann. ACM-SIAM Symp. on Discrete Algorithms, pages 29 39, [7] G. S. Brodal and R. Fagerberg. Cache oblivious distribution sweeping. In Proc. 29th International Colloquium on Automata, Languages, and Programming (ICALP), To appear. [8] G. S. Brodal, R. Fagerberg, and R. Jacob. Cache oblivious search trees via binary trees of small height. In Proc. 13th Ann. ACM-SIAM Symp. on Discrete Algorithms, pages 39 48, [9] M. J. Fischer and M. S. Paterson. Fishspear: A priority queue algorithm. Journal of the ACM, 41(1):3 30, [10] M. Frigo, C. E. Leiserson, H. Prokop, and S. Ramachandran. Cacheoblivious algorithms. In 40th Annual Symposium on Foundations of Computer Science, pages IEEE Computer Society Press, [11] J. S. Vitter. External memory algorithms and data structures: Dealing with massive data. ACM Computing Surveys, 33(2): , June