Design and Evaluation of a Parallel-Polled Virtual Output Queued Switch *

Transcription

1 Design and Evaluation of a Parallel-Polled Virtual Output Queued Switch * K. J. Christensen Department of Computer Science and Engineering University of South Florida Tampa, FL 3360 Abstract - Input-buffered switches with virtual output queueing require crossbar switch matrix scheduling algorithms. Existing scheduling algorithms are non-deterministic and are based on parallel and iterative request-grant-accept arbitration schemes. This presents challenges to flow-level scheduling for guaranteed throughput and bounded delay services and also to scalability. In this paper, the Parallel-Polled Virtual Output Queued (PP-VOQ) switch is presented. Using parallel token passing, the PP-VOQ switch has deterministic and bounded scheduling delay and is implementable for 6 or 3 ports of 0- Gigabit Ethernet. The PP-VOQ switch is shown, via simulation, to perform very similar to an islip switch. The PP-VOQ switch is extended to a cube switch design that decouples input port VOQ selection from output port selection to reduce the scheduling delay and improve scalability. This decoupled parallel polling is very scalable and is shown to result in better performance than an islip switch. I. INTRODUCTION Input buffering in a packet or cell switch is required when the speed of the internal switch fabric and/or output buffering is less than the sum of the data rates of all switched links. In an output-buffered-only switch, the memory speed of the output buffers must be N times the link speed (for N links). Existing memory speeds and bus widths limit switch buffers to about 40-Gbps transfer rate (based on buffer memory with 3-nanosecond access time and a 8-bit wide bus). Thus, an output-buffered-only switch can support only two to four 0-Gbps ports. It is expected that increases in link speeds will continue to outpace improvements in memory speed. Thus, input buffering will continue to be of significance in the future whether as input-buffered-only or combined input and output buffered switch designs. Most existing methods for flow-level scheduling for Quality of Service (QoS) are based on output buffered switches. Input buffering introduces new challenges for supporting QoS for both guaranteed throughput and bounded delay services. Very high throughputs can be achieved with inputbuffered-only switches if virtual output queueing is used to eliminate head-of-line blocking. In a Virtual Output Queued (VOQ) switch, first proposed in [8] and further developed in [, 4], each of N input ports has N queues, one for each output port. Incoming packets to input port i destined for output port j are queued in VOQ i, j for i, j =,,N. An input buffered VOQ switch, or simply VOQ switch, requires a switch matrix scheduling algorithm to match input ports that have packets queued to output ports. Existing VOQ scheduling algorithms, as exemplified by PIM [] and islip [4], implement a maximal match using an iterative, parallel request-grant-accept algorithm between input and output ports. Maximal matching can result in unfairness and stability problems. In [0] it is shown that stability can be guaranteed for unweighted maximal matching with an internal switch speed-up of two. In [5] a cycles for islip is proven, and an improved algorithm, FIRM, is proposed that more closely approximates FCFS. In [7] stability ranges are studied and standard benchmarking methods are proposed. The non-determinism of existing VOQ scheduling algorithms complicates the implementation of any kind of flow-level scheduling at an input port. To support constant bit rate traffic flows in a VOQ switch, input-output port scheduling can be pre-computed on scheduling bound of N + ( N ) a per session basis [, 8]. This requires the switch to be involved in session set-up and tear-down. Delay bounds cannot be guaranteed during the transition when sessions are added or removed. Priority mechanisms can be implemented by weighting requests as implemented in prioritized islip [4], FARR [], and weighted arbitration [6]. However, maximum weighted matching cannot tightly bound scheduling delay. In [] a centralized two-dimensional round-robin (DRR) scheduler is studied which provides a fairness guarantee (bound) of N time slots for each of the N VOQs. The centralized nature of the DRR scheduler makes it difficult to implement for very high-speed switches. In this paper, a deterministically scheduled VOQ switch is designed and evaluated. The target size, link speed, and link protocol are 6 to 3 ports of 0-Gbps Ethernet. The remainder of this paper is organized as follows. Section II describes the new Parallel-Polled VOQ (PP-VOQ) switch. A hardware design is presented. In Section III the performance of the PP-VOQ switch is compared against an islip scheduled VOQ switch for cell transport. Section III also presents performance of the PP-VOQ switch for variable-length Ethernet packets. Section IV presents the cube switch with decoupled selection for improved performance. Performance comparisons with islip and output buffering are presented in Section IV. Section V is a summary and describes future work. * This material is based upon work supported by the National Science Foundation under Grant No

2 Input port,,, 3 Inputport,,, 3 Control and data lines Crossbar Output port Output port Clock Q T_in Latch T_in, Hold_next, andbusy_next Busy_last T_in Hold_last Block # Logic for A and Busy_next A Block # Logic for Hold_next Busy_next Hold_next Input port 3 3, 3, 3, 3 Output port 3 Block #3 Logic for T_out T_out (to next input port as T_in) Token lines Fig.. Example 3-port PP-VOQ switch II. THE PP-VOQ SWITCH In this section, the Parallel-Polled Virtual Output Queued (PP-VOQ) switch is described. Fig. shows a 3-port switch. A. Parallel polling of VOQs Input port logic with rotating selection Parallel polling via synchronized token passing can be used to establish a match between input ports that have packets or cells buffered in their VOQs and the output ports. In [4] a parallel token ring LAN is described. A PP-VOQ switch [3] is implemented with N parallel polling lines interconnecting the N input ports into a parallel ring. Each of the N tokens (one per line) corresponds to an output port j for matching a VOQ in input port i to output port j. Overlapped scheduling and forwarding phases are used. In the forwarding phase, fixed-size blocks of packet or cell data are transferred between matched input and output ports. The transfer time is the same as the transmission time on the output link (i.e., there is no speed-up in the internal switch crossbar). In a scheduling phase, each token circulates through at most all N input ports or until a match can be made at an input port. In each scheduling phase, tokens begin circulating from the input port at which they were last held. When a synchronized group of tokens arrives at an input port, the next non-empty VOQ (with a corresponding arriving token) following the VOQ that last matched in a previous scheduling cycle is matched. This rotating selection assures fairness by limiting the maximum delay to N transmission times for any input port and transmission times for any VOQ in any input port. N Fig.. Hardware block diagram of input port logic The time required for a parallel polling of all ports (i.e., switch matrix scheduling) affects the delay performance of the switch. In the following subsection, the design and timing analysis of the input port logic that implements the token processing for parallel polling is described. B. Hardware design and timing analysis Fig. shows the hardware block diagram. Fig. 3 is a behavioral description and includes a definition of the logic variables shown in Fig.. A token is the presence of a on a token line and is latched at each input port for the duration of a clock cycle. A 0 on a token line indicates that the token has already been held in an upstream input port. The input port logic determines which token will be held for the next forwarding phase (i.e., the next input port to match to an output port) and releases a token held from the last forwarding phase. Thus, T_out is a masking out (in T_in) of the next token to be held and a masking in of a released token. Logic block # implements lines through 3 of the behavioral description, logic block # implements lines 4 through 7, and logic block #3 implements lines 8 through 0. Lines and are the latching of the present Busy and Hold states for the next clock cycle. The clock rate is determined by the latch delay plus the propagation delay of the three logic blocks. Using the BOOL high level description language and compiler [9], the logic design for the input port logic was completed. The BOOL descriptions are freely available for download at []. For a 6-port switch, an equivalent of,083 two-input logic gates are required per input port logic. Thus, the scheduling logic for a 6-port switch can be implemented in 44,848 two-input gates. This compares favorably to the estimated 35,000 two-input gate equivalents

3 input T_in[N]; input Q[N]; output T_out[N]; boolean A[N]; boolean Busy_last; boolean Busy_next; integer Hold_last; integer Hold_next; // Boolean vector for arriving tokens // Boolean vector for non-empty VOQs // Boolean vector for departing tokens // Boolean vector for eligible VOQs // if hold a token in this phase // if will hold a token in the next phase // VOQ index of token hold for this phase // VOQ index of token hold for next phase with less than a block size of data makes the scheduling nonwork conserving. This can result in an unstable system. In this paper, we consider the more conservative case of matching a VOQ when it contains at least a block size of data. In the next section, delays due to scheduling and transfer blocking are evaluated.. A=T_in&Q;. Busy_next = 0; 3. if (A contains at least one ) Busy_next = ; 4. Hold_next = Hold_last; 5. if (Busy_next == ) 6. Hold_next = position of first bit in A after Hold_last; 7. establish a cross-bar connection to output port Hold_next; 8. T_out = T_in; 9. if (Busy_last == ) set bit position Hold_last in T_out to ; 0. if (Busy_next == ) set bit position Hold_next in T_out to 0 ;. Busy_last = Busy_next;. Hold_last = Hold_next; Fig 3. Behavioral description of PP-VOQ input port logic required for a 6-port islip implementation [3] and is thus easily implementable on a single chip. For a 3-port PP- VOQ switch, a total of about 75,000 two-input gate equivalents are required and is also implementable on a single chip. Logic blocks # and #3 (in Fig. ) are simple two-level logic. Logic block # implements a priority encoding of the next non-empty VOQ with a matching token (i.e., a priority encoding of the A vector). In [5] a transistorlevel priority encoder implementation in -micron CMOS is presented that has a propagation delay of 4.4 nanoseconds for a 3-bit encode. Thus, with a technology that supports gate delays of 0.5 nanoseconds and latch delays of nanoseconds, the token processing (scheduling) delay for an input port is approximately 0 nanoseconds. Thus, the total scheduling delay for a 6-port switch is 60 nanoseconds, and for a 3-port switch it is 30 nanoseconds. C. Cell blocking and support for variable length packets For a total scheduling delay that is greater than the transmission time of single cell or packet, blocking of multiple cells or packets into a transfer block can be implemented. Thus, a block of cell or packet data is forwarded in each cycle and fragments are re-assembled at the output port. For a scheduling delay of 60 nanoseconds for a 6 port ATM OC-9 switch, a block size of four cells is needed. For 0-Gbps Ethernet, the block size is 00 bytes. With transfer blocking, a VOQ with less than a block size of data can be handled in two ways. The VOQ can ) be matched and transfer less than a block data, or ) be prevented from matching until it contains a full block size of data. If the former approach is used, care must be taken to insure stability. In the case of an input port with some VOQ s containing less than and other VOQs containing greater than block size amount of data, matching the VOQ III. PERFORMANCE EVALUATION A simulation model was used to evaluate the performance of the PP-VOQ switch. The simulation model, in source code form, is freely available for download at []. CSIM8 function libraries [7] are used in the implementation. A. PP-VOQ comparison to islip for cell switching Two experiments are defined to compare PP-VOQ and islip for cell switching. A 6-port cell switch with infinitesize VOQs is modeled. For islip, four iterations are used per scheduling cycle. For PP-VOQ, transfer block sizes of, 4, and 8 cells are used. A VOQ is not matchable (i.e., the Q bit in Fig. does not become a ) until it contains at least a transfer block number of cells. For all of the experiments, switch queueing delay is measured. For the experiments, Bernoulli and Interrupted Bernoulli Process (IBP) arrivals are used. For the IBP, α = Pr arrivalat t IBPisin on state, [ ] p = Pr [ IBP is in on state at t + IBPisin on state at t],and q = Pr [ IBP is in off state at t + IBP is in off state at t]. The mean length of an on state is ( p) and the mean length of an off state is ( q). In an off state there arenoarrivals. An off stateisatleastoneslotinlength, therefore a.0 arrival rate (or % offered load) is not possible. The mean arrival rate, ρ, is α ( q) ρ =, () p q and the Coefficient of Variation (CoV) is, ( p) ( p + q) ( ) CoV = + α. () p q ) Experiment #A: Bernoulli arrivals with uniformly selected outputs. Offered load is ranged from 50% to 98%. This is the classic experiment for evaluating switch performance and also enables a validation of the islip simulation model in [3]. ) Experiment #A: IBP arrivals with uniformly selected outputs for each on burst (i.e., all cells in a burst are forwarded to the same output port). The mean burst length is fixed at 0 cells with all slots in an on burst always filled with a cell (i.e., α = ). The mean off, or silence, period is varied to achieve an offered load of 50% to 95%. This experiment evaluates performance for bursty traffic.

4 0 0 Queueing delay (cells) 0 PP-VOQ IBP (block = 4, 8) PP-VOQ Bernoulli (block = 4, 8) PP-VOQ and islip IBP PP-VOQ Bernoulli Queueing delay (microsec) 0 IEEE distribution (block =, 00 bytes) 500 byte pkts islip Bernoulli Fig. 4. Results for PP-VOQ - islip experiments #A and #A Fig. 4 shows the mean queueing delay for experiments #A and #A (the solid lines are for a transfer block of one cell). PP-VOQ queueing delay is slightly greater than islip queueing delay at high offered loads. At 90% offered load, the mean queueing delay for Bernoulli arrivals are, PP-VOQ =.4 and islip = 9.76 cell times. The Bernoulli arrival islip results (experiment #A) in Fig. 4 exactly match the results shown in Fig. 0 of [3]; this validates the islip simulation model. Transfer blocking significantly increases queueing delay for the Bernoulli arrivals, slightly less so for IBP arrivals. At high offered loads, the added delay from transfer blocking is reduced. The reason for this is that at low loads, a VOQ waits for subsequent arrivals to achieve its minimum transfer size (i.e., before turning on the Q bit). B. PP-VOQ performance for 0-Gbps Ethernet Three experiments are defined to measure the performance of a 6-port PP-VOQ switch for 0-Gbps Ethernet. The transfer block size is 00 bytes for all experiments, unless otherwise noted. For all experiments, offered load is ranged from 50% to 98%. ) Experiment #B: Poisson arrivals of 64-byte packets with uniformly selected outputs. ) Experiment #B: Poisson arrivals of 500-byte packets with uniformly selected outputs. 3) Experiment #3B: Poisson arrivals of variable length packets with packet sizes independently taken from the IEEE workgroup average empirical distribution [6]. Transfer block sizes of and 00 bytes are used. For the IEEE workgroup average empirical distribution, all packet lengths are a multiple of bytes. The mean packet length is 590 bytes with about 0% of the packets 00 bytes, 0% 600 bytes, and 0% bytes. This representative packet length distribution, measured from -Mbps Ethernet workgroups at Sun Microsystems, Inc., was originally used by the IEEE 80.3z task force for evaluating Gigabit Ethernet performance. 64 byte packets Fig.5. Results for PP-VOQ Ethernet experiments #B, #B, and #3B Fig. 5 shows the mean queueing delay in microseconds for experiments #B, #B, and #3B. It can be seen that queueing delays remain well under microseconds for offered loads of less than 95% for 64-byte, 500-byte, and IEEE distribution packet lengths. The effects of transfer blocking are again significant at lower offered loads, but decrease at higher loads. IV. THE PP-VOQ CUBE SWITCH By decoupling the rotating selection of the PP-VOQ input port logic, pipelining of the selection can be implemented. The decoupled selection simplifies the critical path logic and thus decreases the scheduling time. The first round selects only a VOQ at an input port; the second round selects only the output port. Transfer blocking, which increases queueing delay, would not be required for 6 or 3 port 0- Gbps designs and thus scalability is increased. One possible arrangement of the input and output ports is to stack the first round logic horizontally and stand-up the second round logic vertically resulting in a cube switch. Fig. 7 shows an example 3-port cube switch. One method of decoupling the selection is by adding buffering to the internal crosspoints (in a crossbar switch fabric) and then selecting VOQ buffers in an input port independently from crosspoint buffers. Polling is used for both independent selections. The feasibility of such a switch design is currently being studied. Experiments #A and #A were repeated for a PP-VOQ cube switch, an islip switch, and an output buffered switch for cell switching. The output buffered switch gives a lower bound on queueing delay. For the PP-VOQ cube switch, buffering within the crossbar crosspoints was implemented with independent polling of these buffers. The results are shown in Figure 7. It can be seen that the PP-VOQ cube switch results in lower delay than an islip switch for both Bernoulli and IBP arrivals (except for at low utilizations).

5 Inputport,,, 3 Inputport,,, 3 Inputport3 3, 3, 3, 3 Logic for first round of rotating selection Logic for second round of rotating selection a a a b 3a c b a b b 3b c c 3a c 3b 3c 3c Fig. 6. Example 3-port PP-VOQ decoupled selection cube switch V. SUMMARY AND FUTURE WORK In the PP-VOQ switch, parallel polling by N tokens results in high throughput, good delay behavior, and deterministic scheduling delay. Scheduling delay for an input port is bounded to N scheduling cycle times. For a 6- port PP-VOQ switch with a 00-byte delay per scheduling cycle, each input port is guaranteed to have a scheduling opportunity every 60 nanoseconds for a 0-Gbps link speed. Thus, each VOQ in each input port is guaranteed a scheduling opportunity every.56 microseconds. Flow-level scheduling at the input ports can now be investigated. The implementable scheduling delay of a PP-VOQ switch is greater than a single cell transmission time at OC-9 or 64-byte packet at 0-Gbps Ethernet. The use of transfer blocking to transfer groups of cells or packets was shown to increase queueing delay. Decoupling the selection of queues in a PP-VOQ switch was shown to result in better performance than that of an islip switch. Future work will study how to implement a PP-VOQ cube switch and investigate early and eligible deadline scheduling in the VOQ s to achieve flow-level scheduling for QoS classes. REFERENCES To crossbar (see Fig. ) Control line a connects toa,btob,andsoon [] T. Anderson, S. Owicki, J. Saxe, and C. Thacker, High-Speed Switch Scheduling for Local-Area Networks, ACM Transactions on Computer Systems, Vol., No. 4, pp , November 993. [] K. Christensen, Home Page for Kenneth J. Christensen, 00. URL: [3] K. Christensen, A Parallel-Polled Virtual Output Queued (PP-VOQ) Switch, IEE Electronics Letters, Vol. 36, No., pp , October 6th, 000. Queueing delay (cells) 0 0 IBP results Output Bernoulli results PP-VOQ cube islip PP-VOQ cube islip Output Fig. 7. Results for PP-VOQ cube - islip experiments #A and #A [4] K. Christensen and F. Noel, Parallel Channel Token Ring Local Area Networks, Proceedings of the 7th IEEE Conference on Local Computer Networks, pp , September 99. [5] J. Delgado-Frias and J. Nyathi, A VLSI High-Performance Encoder with Priority Lookahead, Proceedings of the Great Lakes Symposium on VLSI, pp , February 998. [6] H. Frazier, Jr., Scaling CSMA/CD to 0-Mbps: An Update, IEEE P80.3z Gigabit Task Force Meeting, July 9, 996. URL: HFcarext.pdf. [7] M. Goudreau, S. Kollioulos, and S. Rao, Scheduling Algorithms for Input-Queued Switches: Randomized Techniques and Experimental Evaluation, Proceedings of IEEE INFOCOM, pp , March 000. [8] A. Huang, G. Kesidis, and N. McKeown, ATM Input-Buffered Switches with the Guaranteed-Rate Property, Proceedings of the 3rd IEEE Symposium on Computers and Communications, pp , July 998. [9] A. Kfir, BOOL Logic Design Software, Cornell Design Tools, 997. URL: [0] E. Leonardi, M. Mellia, M. Marsan, and F. Neri, Stability of Maximal Size Matching in Input-Queued Cell Switches, Proceedings of the International Conference on Communications, June 000. [] R. LoMaire, and D. Serpanos, Two-Dimensional Round-Robin Schedulers for Packet Switches with Multiple Input Queues, IEEE/ACM Transactions on Networking, Vol.., No. 5, pp , October 994. [] C. Lund, S. Phillips, and N. Reingold, Fair Prioritized Scheduling in an Input-Buffered Switch, Proceedings of the International IFIP-IEEE Conference on Broadband Communications, pp , 996. [3] N. McKeown and T. Anderson, A Quantitative Comparison of Iterative Scheduling Algorithms for Input-Queued Switches, Computer Networks and ISDN Systems, Vol. 30, No. 4, pp , December 998. [4] N. McKeown, The islip Scheduling Algorithm for Input-Queued Switches, IEEE/ACM Transactions on Networking, Vol. 7, No., pp. 88-0, April 999. [5] D. Serpanos and P. Antoniadis, FIRM: A Class of Distributed Scheduling Algorithms for High-Speed ATM Switches with Multiple Input Queues, Proceedings of IEEE INFOCOM, pp , March 000. [6] R. Schoenen, G. Post, and G. Sander, Prioritized Arbitration for Input- Queued Switches with % Throughput, Proceedings of the IEEE ATM Workshop 99, pp , May 999. [7] H. Schwetman, CSIM8 - The Simulation Engine, Proceedings of the 996 Winter Simulation Conference, pp. 57-5, December 996. URL: [8] Y. Tamir and G. Frazier, High Performance Multi-Queue Buffers for VLSI Communications Switches, Proceedings of the 5th Annual Symposium on Computer Architecture, pp , June 988.