SIMULATION BASED ANALYSIS OF THE INTERACTION OF END-TO-END AND HOP-BY-HOP FLOW CONTROL SCHEMES IN PACKET SWITCHING LANS

Transcription

1 SIMULATION BASED ANALYSIS OF THE INTERACTION OF END-TO-END AND HOP-BY-HOP FLOW CONTROL SCHEMES IN PACKET SWITCHING LANS J Wechta, A Eberlein, F Halsall and M Spratt Abstract To meet the networking requirements of the newer high-bandwidth multimedia applications, many LAN managers are starting to adopt switches as the basis of their network architectures. However, although introducing switches makes the networks faster, they can also introduce some new problems. These are caused by the interaction of the existing end-to-end flow control in end systems with the hop-by-hop flow control that is used by switches. An example is a phenomenon known as head-of-line blocking. The paper illustrates how this can occur and, for an example topology, quantifies its effect using simulation modelling. Keywords: End-to-end and hop-by-hop flow control, TCP 1. Introduction An ever growing demand for more bandwidth is the driving force for the introduction of more efficient and faster network technologies [Ban91]. Switching technology [Hel97] has gained huge popularity over the last few years. Switches are in fact multiport bridges which offer much higher capacity compared to bridges. Switches are equipped with buffered input and output ports [Hac96], [Els96], and optionally with shared memory. The internal structure of switches allows to relay many concurrent flows over the very fast internal bus. The model of the switch used during the simulation also has incorporated the switch hop-by-hop flow control. In order to investigate the interaction of the two flow control schemes, models of the TCP version Reno and the switch have been developed using C++ and the BONeS Designer simulation tool. Throughout the simulations duplex links have been used. The investigated topology shown in Figure 1, consists of two switches interconnecting the server to a set of clients. The data flows originate at the server and are sent to each client. All clients, except client C1, are connected to switch 2 using 100Mb/s links. C1, however, is assumed to be a legacy client and hence is connected to switch 2 using a 10Mb/s link. Server 1Gb/s L1 Switch Mb/s L2 10 Mb/s 100 Mb/s Switch 2 L3 C1 C2 Cn Figure 1: The topology used to illustrate head-of-line blocking effect This paper is divided into four further parts. Section 2 describes the TCP end-to-end flow control scheme that is typical of that used in end systems while Section 3 gives insight into the hop-by-hop flow control scheme used by switches. Simulation results are presented in Section 4, followed by the discussion. The paper is concluded in Section 5. J. Wechta, A. Eberlein and F. Halsall are with the University of Wales, Swansea, UK, eewechta@swansea.ac.uk M. Spratt is with Hewlett Packard Labs, Bristol, UK

2 2. Transport layer end-to-end flow control Examples of the popular transport layer protocols are the Transmission Control Protocol, widely known as TCP [Ste95], DECnet and SNA. All of them use positive acknowledgement, window-based schemes with retransmission as a foundation for controlling data flow. Additive increase and multiplicative decrease methods are used to change the window size. TCP well represents protocols which are implemented as a transport layer protocol and hence the TCP has been used in the simulations described as a provider of end-to-end data flow control. The philosophy of TCP is relatively simple. On receipt of an acknowledgement (ACK) the sender knows that a data packet successfully reached its destination, and hence, network resources for a new data packet have become free. Furthermore, a returned ACK proves that the packet was not rejected by the network and there is a chance that the network can admit even more packets. As a result TCP increases its window allowing more packets to be sent within one round trip time (RTT) period. When a packet is finally dropped the number of data segments which can be sent during one RTT is decreased and the above described procedure is repeated. Two components of the TCP flow control play exceptionally important roles. They are the TCP window and the ACK frames. The window responsibility is to restrict the number of outstanding packets in the network. The TCP flow control mechanisms (slow start, congestion avoidance [Ste97]) influence the size of the window to regulate the amount of currently outstanding data offered by the sender to the network. Flow control mechanisms tend to increase the window size when the network is able to accept more traffic and decrease it when contention/congestion occurs and the loss of data packets is registered. It is a characteristic of end-to-end flow control schemes that only end-station participate in flow control directly and congestion is detected by observed packet loss. That is also why TCP flow control is known as a reactive one. TCP sender and receiver applications are not aware of what kind of underlying system is used. Also partly because of the layered structure of the network they are not aware of emerging congestion. The major problem with reactive end-to-end flow control in high-speed LANs is slow feedback [Jun91] because congestion can dissipate or become severe before a feedback message can reach the source [Fen93]. And that is why the packet switches have been equipped with the hop-by-hop flow control which enables the switch to cope with the effect of short lasting congestion with much shorter delay and hence is more efficient. 3. Hop-by-hop flow control Hop-by-hop flow control schemes [Ger88] are implemented on the data-link layer of the OSI reference model. This kind of flow control is known as a backpressure flow control [Jun91], [Mis96], [Omi96], [Yan95]. A flow control mechanism is used to allow the congested switch to request the switches that are connected to its input ports to temporarily stop sending any new frames until send XOFF frame send XON frame Input Port FIFO queue Head of FIFO queue high-water mark XOFF threshold current FIFO queue occupancy low-water mark XON threshold filling Figure 2: Basics of the switch flow control mechanism emptying the congestion has passed. This switch flow control mechanism is built into the input port. It uses a combination of XOFF/XON thresholds and XOFF/XON control frames to ensure that no data frames are dropped due to a lack of free memory in the switch [Wan91]. The basic idea of the backpressure flow control is shown in Figure 2. When an outgoing link of a switch becomes congested, frames coming into this switch cannot be sent via the congested link, and hence they need to be stored inside the switch. If this situation persists the storage within the switch (including the input port buffer) are likely to overflow. The switch flow control compares the amount of data stored so far in the input port buffer (FIFO

3 queue) with the XOFF threshold value (high-water mark [Wan91]). If the XOFF threshold is reached, an XOFF control frame is sent to the adjacent switch (or the source) which in turn stops sending any new data frames. This causes the FIFO queue to release data until the current queue occupancy reaches the XON threshold (low-water mark). The flow of data frames is only resumed after receiving an XON control frame which is generated by the same switch which issued the previous XOFF. 4. Results and discussion The simulation have been run with various values for the TCP window size, input buffer size, output buffer size and shared memory size. Throughout the investigations the TCP window was always smaller than the storage offered by the switch. Each graph (Figure 3 and Figure 4) contains results of 9 separate simulations. The parameter of the simulations was the number of clients being changed in the range from 2 to 10. One flow is always directed to the client connected to the slow link th node Maximum available bandwidth - 100Mb/s Used bandwidth [Mb/s] th node 8th node 7th node 6th node 5th node 4th node 3rd node 2nd node 1st node 0 +1 fast +2 fast +3 fast +4 fast +5 fast +6 fast +7 fast +8 fast +9 fast node Figure 3: Bandwidth taken by saturated traffic end on the inter-switch link Remarks: A slow node is a sender which transmits data over a path containing a 10 Mb/s link. A fast node is a sender which transmits data over a path containing only 100 Mb/s links. 100 Maximum available bandwidth - 100Mb/s Used bandwidth [Mb/s] fast +2 fast +3 fast +4 fast +5 fast +6 fast +7 fast +8 fast +9 fast node Figure 4: Bandwidth taken by TCP end on the inter-switch link

4 To illustrate the head-of-line blocking phenomenon [Mol96], consider the topology shown in Figure 1. In this investigation, first no end-to-end (TCP) flow control is applied and all traffic sources can therefore produce saturated traffic level, while in the second case, TCP flow control is applied to all transfers. Two points should be noted in the above topology. Firstly, the throughput of all senders is limited by the bandwidth of the shared link L2. Secondly, one of the end is connected to switch 2 via a slower link (10Mb/s). A number of traffic sources send data to switch 1. Switch 1 relays this traffic to switch 2, which accepts the packets and sends them down the links to the end. However, packets destined for the end node on the slow link arrive with a higher rate at switch 2 than they can be delivered to C1. Packets that cannot be sent are stored in switch 2. This effect of data accumulation within the storage in front of the bottleneck is an important issue in the process of analysing the interaction between flow control schemes. The simulation results shown in Figure 3 relate to the case in which no end-to-end flow control is applied. The results show the bandwidth taken by saturated traffic end on the inter-switch link. Saturated traffic results in one packet being sent after another. The available bandwidth on the inter-switch link is 100Mb/s. Packets sent to client C1, which is connected to switch 2 by a slow link, travel with the rate of 10Mb/s. However, since there is a mismatch between the rate at which packets originated by the server process (sending data to C1) are relayed onto the shared link L2 and the rate they are relayed to C1, the difference between the number of segments admitted to and released from switch 2 must be accumulated in buffers within it. This effect causes the switch flow control to be triggered and results in an XOFF control frame being sent to the preceding switch 1. The aim of an XOFF frame is to stop the transmission of new data packets into switch 2. However, if an XOFF frame is sent over the link between the two switches, the upstream switch will stop transmitting all packets. That is, it will not only stop sending packets which are addressed to the end node on the slow link (i.e., C1), but all packets that are sent to the second switch. An XOFF frame causing this behaviour, is said to be source blind. The packet transfer rate of the slower link (10Mb/s) therefore dictates the packet transfer rate over all the other higher rate (100Mb/s) links connecting the receivers to switch 2. The presence of the 10Mb/s link causes all the other links to the receivers to also operate at 10Mb/s even though their links could support a higher rate. This is the head-ofline blocking effect, and the results in Figure 3 show that this effect will persist until the number of receivers connected by the higher bit rate links reaches 9. Beyond this, the capacity of the interswitch link L2 will start to regulate the flow. The results shown in Figure 4 relate to the case in which end-to-end flow control is also being used. As can be seen in this case, the aggregate bandwidth of the link connecting the receivers to switch 2 is determined by that of the 100Mb/s capacity of the interswitch link L2. Firstly, the TCP window restricts the number of outstanding packets in the network and prevents the storage in the switch to overflow which would trigger the switch flow control. Secondly, the amount of data accumulated in front of the bottleneck link is similar to the TCP window size. Thirdly, since TCP operates with an end-to-end flow control mechanism, the source receives information about the traffic conditions on the entire path between the sender and the receiver. As a result, the sending rate of server process sending data to C1 is limited to 10Mb/s by the TCP flow control. Since the packet transfer rate to C1 is limited to 10Mb/s, the remaining 90Mb/s will be shared by the other receivers that are connected to switch 2. This will continue until the total number of receivers reaches 10 after which all bit rates will reduce proportionally. This means that the available bandwidth will be fully utilised when the TCP end-to-end flow control is used. During this investigation it has become obvious that a big TCP window size offers two attractive features for those who use it. Firstly, a big TCP window imposes higher throughput in comparison to a small one. Secondly, it offers a potentially smaller delay of the file transfer. In both cases however gains are made on other users cost and such dishonest behaviour leads to unfair bandwidth sharing. A big TCP window also allows the bandwidth of fast links to be fully utilised. Additionally, increasing the TCP window size to the value which exceeds the capacity of the storage offered by the switch leads to the effect of head of line blocking and to throughput degradation similar to that shown in the graph with saturated traffic sources (Figure 3). Furthermore, the size of the storage offered by the switch is also an important issue to be considered. In general, the higher the ratio of storage capacity to TCP window size, the smaller the probability of head of line blocking.

5 5. Conclusions Switch flow control proves to be an efficient tool which allows switches to support each other by means of exchanging XOFF/XON messages. Receiving XOFF message results in the data packets forwarding process being delayed and in data packets being stored in two (or more) instead of only one storage when contention persists. This is a much better reaction to congestion than dropping packets. More conclusions related to this topic are to be found in [Wec98]. The problem of the backpressure switch flow control is that XOFF/XON frames are not sent directly to the sender which causes buffer overflow. The output port of a switch is forced to interrupt any transmission of packets on the link after receiving an XOFF frame. Since the link transmits packets from all sender processes, not only the packets from the sender process causing the problem are stopped but also all other packets. Possible consequences of using switch flow control are on the one hand the elimination of data losses which could be caused by storage overflow, and on the other hand the degradation of the throughput of some flows due to the head of line blocking effect as described in this paper. The relationship between the TCP window size, storage offered by the switch, and the process of triggering the switch flow control has been shown. The smaller the ratio of the TCP window size to the offered storage the smaller the chance for filling the storage and triggering the switch flow control. One solution to avoid head of line blocking is to increase the size of storage offered by the switch, which is an expensive solution and introduces higher possible delays. The second solution is to keep the TCP window size small which potentially can cause underutilisation of fast links. 6. References [Ban91] Bandula W. High-Speed Local Area Networks and Their Performance: A Survey, ACM Computing Surveys, Vol. 23, No. 2, pp , June 1991 [Els96] Elsaadany A., Singhal M. and Liu M.T. Performance Study of Buffering within Switches in LANs, Computer Communication, pp , 1996 [Fen93] Fendick K. and Rodrigues M. An Adaptive Framework for Dynamic Access to Bandwidth at High Speeds, ACM Sigcomm, pp , 1993 [Ger88] Gerla M. Congestion Control in Interconnected LANs, IEEE Network, Vol. 2, No. 1, pp , January 1988 [Hac96] Hac A. Bandwidth Management and Switch Buffer Allocation in High-Speed Networks with Bursty Traffic, International Journal of Network Management, pp. 2-16, January 1996 [Hel97] Held G. High-Speed Networking with LAN Switches, Wiley Computer Publishing, 1997 [Jun91] Jungok J. Survey of Traffic Control Schemes and Protocols in ATM, Proceedings of the IEEE, Vol. 79 No. 2, pp , February 1991 [Mis96] Mishra P., Kanakia H. and Tripathi S. On Hop-by-Hop Rate-Based Congestion Control, IEEE ACM Transactions on Networking, Vol. 4, No. 2, pp , April 1996 [Mol96] Molle M. and Watson G. 100Base-T / IEEE / Packet Switching, IEEE Communications Magazine, pp , August 1996 [Omi96] Omidyar C. and Pujolle G. Introduction to Flow and Congestion Control, IEEE Communication Magazine, pp , November 1996 [Ste95] Stevens W. TCP/IP Illustrated, Vol 1, The Protocols, Addison Wesley, 1995 [Ste97] Stevens W. TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery RFC 2001, January 1997 [Yan95] Yang C.-Q. and Reddy A. A Taxonomy for Congestion Control Algorithms in Packet Switching Networks IEEE Network, pp , July/August 1995 [Wan91] Wang Y. and Sengupta B. Performance Analysis of a Feedback Congestion Control Policy Under Non- Negligable Propagation Delay, ACM Sigcomm 1991 [Wec98] Wechta J., Eberlein A., Halsall F. and Spratt M. The Interaction of the TCP Flow Control Procedure in End Nodes on the Proposed Flow Control Mechanism for Use in IEEE Switches, submitted to 8th IFIP Conference on High Performance Networking (HPN'98), Vienna, Austria, 1998