On the Efficient Implementation of Pipelined Heaps for Network Processing. Hao Wang, Bill Lin University of California, San Diego

Transcription

1 On the Efficient Implementation of Pipelined Heaps for Network Processing Hao Wang, Bill Lin University of California, San Diego

2 Outline Introduction Pipelined Heap Structure Single-Cycle Operation Memory Management Conclusion

3 Outline Introduction Pipelined Heap Structure Single-Cycle Operation Memory Management Conclusion

4 Introduction Priority queues widely used in many network processing applications Weighted Fair Queuing for advanced scheduling Queue memory management using DRAMs Network measurements Key design challenge: priority queues expected to operate at very high speed, e.g. 40 Gb/s and beyond

5 Introduction The conventional binary heap is widely used to implement priority queues, but requires O(log N) time complexity However, too slow for 40 Gb/s and beyond To address the performance requirements, a new data structure called the Pipelined Heap (P-Heap) was proposed [Bhagwan & Lin, INFOCOM 2000] with only O(1) time complexity, at the expense of O(log N) pipelined stages

6 Introduction In this talk, we describe two sets of improvements to the original work First, we describe explicitly how we achieve single-cycle pipelined operations (which was not detailed in original work) Second, we describe new memory organizations to enable more efficient use of memories

7 Outline Introduction Pipelined Heap Structure Single-Cycle Operation Memory Management Conclusion

8 Pipelined Heap Unlike conventional heap, all operations proceed top-down In addition, each node in a pipelined heap has a capacity field value capacity

9 Pipelined Heap Each pipeline stage has memory module M k that stores corresponding nodes in k th layer of pipelined heap All operations proceed top-down thru pipelined stages input output 1 16 op1 stage 1 arg1 M op2 p2 arg2 stage 2 M op3 p3 arg3 stage 3 M op4 p4 arg4 stage 4 M4

10 Outline Introduction Pipelined Heap Structure Single-Cycle Operation Memory Management Conclusion

11 Single-Cycle Operation Use ENQUEUE operation as example Ideas: Prepare for all possible future events Drop events that are not going to happen in a later time Insert new operation as frequently as possible Achieve twice the original speed, i.e. single-cycle per operation

12 Read A and A L Compare A and A A =min{a, A } Write A ENQUEUE Operation B L+1 B =min{b, B } Read B and C Compare B and B Write B D L+2 D =min{d, D } Read D and E Compare D and D Write D Clock Cycles L A L+1 B C L+2 D E F G

13 Single-Cycle Operation Result is that single-cycle operation can be realized for all of ENQUEUE, DEQUEUE, and ENQUEUE- DEQUEUE operations Inter-operation management Treat DEQUEUE operation as ENQUEUE-DEQUEUE operation that enqueues a VOID The only exception is DEQUEUE following another DEQUEUE, which results in two-cycle per operation

14 Outline Introduction Pipelined Heap Structure Single-Cycle Operation Memory Management Conclusion

15 Memory Size Disparity Problem Because memory module M k stores corresponding nodes in k th layer of pipelined heap, there will be huge disparity between memory sizes of first thru last stages input output op1 arg1 stage 1 op2 p2 arg2 stage 2 op3 p3 arg3 stage 3 op4 p4 arg4 stage 4 M1 M2 M3 M4 Huge Memory Size Parity

16 Memory Management Optimization We need more efficient way to balance the memory size requirements without interfering with pipelined operations We present two optimizations First, we consider a Forest-of-Heaps Next, we consider memory optimization for a single heap

17 Forest-of-Heaps Optimization In many applications, we may have many logical priority queues that share same memory buffer We call this a Forest of Heaps. e.g. consider 10 heaps, each with 6-levels For the 10 heaps together, the memory size for each of the 6 stages are 1x10, 2x10, 4x10, 8x10, 16x10, 32x10 = 10, 20, 40, 80, 160, 320, respectively Memory size disparity between first and last stage is 320/10 = 32x......

18 Forest-of-Heaps Optimization Can balance memory requirement by alternately inverting half the heaps Heap 2 Heap 10 Heap 1 Heap 3 Heap 9 For 10 heaps, each with 6-levels, with optimization, memory size for each of the 6 stages are 165, 90, 60, 60, 90, 165, respectively Memory size disparity between largest and smallest stage is 165/60 < 3x

19 Single-Heap Optimization For single heap Can divide heap in half Then, alternately invert half the sub-heaps e.g. heap with 6 stages: Original: 1, 2, 4, 8, 16, 32 Disparity = 32x Optimized: 21, 18, 32 Disparity < 2x This idea can be combined with forestof-heaps optimization

20 Detailed op1 input arg1 output Realization stage 1 memory module 1 op2 p2 arg2 The combination of heap with an inverted heap stage 2 op3 p3 arg3 stage 3 memory module 2 memory module 3 An outer control unit is needed : : : : opn-2 pn-2 argn-2 stage N-2 memory module N-2 opn-1 pn-1 argn-1 stage N-1 memory module N-1 opn pn argn stage N memory module N output

21 Detailed Realization (cont) To implement stage i this scheme, each memory structure needs to support potentially twice as many memory operations controller controller memory module memory module memory module memory module Therefore, to support one stage per cycle, each memory structure needs to support a read throughput of 8 and a write throughput of 6 per cycle

22 Outline Introduction Pipelined Heap Structure Single-Cycle Operation Memory Management Conclusion

23 Conclusion A more aggressive pipelining structure is introduced Memory management techniques aimed at resolving the disparity in storage requirements between top and bottom layers of pipelined heap are presented These new proposed techniques enable a faster priority queue implementation

24 Questions? Thank You