An Agent Based Congestion Control and Notification Scheme for TCP over ABR

Transcription

1 An Agent Based Congestion Control and Notification Scheme for TCP over ABR (This work is funded in part by the Engineering and Physical Science Research Council under project GR/L86937) K. Djemame, M. Kara and R. Banwait ATM-Multimedia Group - School of Computer Studies University of Leeds Leeds LS2 9JT, UK {karim,mourad}@scs.leeds.ac.uk Abstract We overview in this paper the enhancement of TCP s congestion control mechanisms using Explicit Congestion Notification () over ATM networks. Congestion is indicated by not only packet losses as is currently the case but an agent implemented at the network s edge as well. The agent bridges the gap between the ATM layer and the TCP layer in the protocol stack at the receiver end and coordinates the congestion control algorithms of the TCP transport protocol and the ATM cell-oriented switching architecture. The novel idea uses ABR rate-based flow control to notify congestion and adjust the creditbased window size of TCP. The effects of running TCP over ABR (Available Bit Rate) are studied with the help of two simulation models (LAN and WAN). The simulation results indicate that LANs having a single switch and WANs with multiple switches benefit most from TCP. In almost all scenarios, TCP achieved significantly lower cell loss, packet retransmissions, buffer utilisation and exhibited better throughput than TCP Reno. Keywords: Transmission Control Protocol, Explicit Congestion Notification, Asynchronous Transfer Mode, Agents, Performance Evaluation. 1 Introduction Integrating ATM (Asynchronous transfer Mode) technology have revealed poor throughput performance of TCP (Transmission Control Protocol) over ATM [18]. This poor performance is mainly due to packet fragmentation which occurs when a TCP packet flows into an ATM Virtual Circuit (VC) through the AAL5 (ATM Adaptation Layer 5). AAL5 is responsible for dividing TCP s packets into sets of 53 bytes (48 bytes data units and 5 bytes header) called cells. Fragmentation at the AAL5 is necessary since the typical size of a TCP packet is much larger than that of a cell. The loss of even a single cell in any of the ATM network s switches results in the corruption of the entire packet to which that cell belongs, and consequently causes its 1

2 retransmission and severe window reduction actions taken in TCP, leading to bandwidth wastage and underutilisation. Techniques to improve cell discard behaviour such as Partial Packet Discard (PPD) and Early Packet Discard (EPD) are aimed at reducing the transmission of useless cells [18]. Performance degradation in TCP over ATM may also be caused by the following [15]: (1) dynamics of TCP; (2) behaviour of ATM, and (3) interaction between TCP and ATM layer congestion control schemes. In this paper we investigate the interaction between ATM and TCP congestion control mechanisms. We examine the enhancement of TCP s congestion control mechanisms using Explicit Congestion Notification () over ATM networks. Congestion is indicated by not only packet losses as is currently the case but an agent implemented at the network s edge as well. The agent bridges the gap between the ATM layer and the TCP layer in the protocol stack at the receiver end and coordinates the congestion control algorithms of the TCP transport protocol and the ATM cell-oriented switching architecture. The novel idea uses ABR (Available Bit Rate) rate-based flow control to notify congestion and adjust the credit-based window size of TCP. ABR is a service category defined by the ATM Forum [3] and supports applications such as those handling data transfer which have the ability to reduce their sending rate if congestion is experienced in the network. Agents are an important area of research and development in various domains such as artificial intelligence, high-performance computing and communication networks [1] Our proposal is driven from our interest in proposing a framework for a coherent approach for better coordination between TCP and ATM congestion control algorithms [8] The effects of running TCP over ABR are studied with the help of two simulation models (LAN and WAN). Simulations under various scenarios then allowed the comparison of TCP Reno and the new implementation based on four performance metrics. The simulation results indicate that LANs having a single switch and WANs with multiple switches benefit most from TCP, and that makes better use of bandwidth. In almost all scenarios, TCP achieved significantly lower cell loss and packet retransmissions. Throughput with TCP was found to be better to that of TCP Reno and buffer utilisation was generally lower. We begin section 2 by discussing some introductory material on the behaviour of TCP traffic with controlled windows and background material for ABR. Some related work to the subject of congestion notification in TCP is presented in section 3. Integrating TCP and ABR and its effect is described in section 4. The environment, network configurations and parameters used in the simulation are presented in section 5. Simulation results over LAN as well as WAN distances are discussed in section 6. Some concluding remarks and perspectives of the research for future developments are given is section 7. A list of acronyms is also included at the end of the paper. 2 Issues of Congestion Control 2.1 TCP Control TCP is a very popular transport protocol [20], and is the most widely used for today s Internet data applications. It combines logic for routing through an internet with end-to-end control. TCP is actually a collection of algorithms that send packets into the network without a reservation and then react to observable events to occur. Such algorithms include congestion control and recovery from loss of packets [2]. Each sender maintains two windows: the window RWND the receiver has granted and a second window CWND called the congestion window. The minimum of the congestion window and the receiver s window is used when sending packets. The congestion control mechanism has two distinct phases: Slow Start and Congestion Avoidance. Upon starting a connection or restarting after a packet loss, the congestion window size is set to one packet, and then doubled once every Round-Trip Time (RTT), i.e., upon the receipt of an 2

3 acknowledgement (ACK). Congestion avoidance takes care of violation which would occur if the retransmit time is too short. During this phase, CWND is linearly increased as opposed to exponential increase during slow start phase. A third parameter SSTHRESH is used to start the congestion phase and is usually initially set to 64kB (kilo Bytes). Upon notification of a network congestion, i.e., a timeout, SSTHRESH is set to half the current CWND and CWND is set to one and the slow start phase restarts once again. There are several ways to detect a packet loss: (1) a timer: when the timer is greater than the Retransmission Time Out (RTO), a packet is assumed to be lost, and (2) when a packet has been acknowledged more than 3 times, the next packet is assumed to be lost. Earlier implementations of TCP were distributed in releases corresponding to the original implementation of Jacobson [12]. However, over the years, there have been some fine-tuning, improvements and additions to the original TCP algorithms. Some of the proposed changes have been widely adopted and are part of TCP implementations today. TCP-Tahoe and TCP-Reno are the (most) deployed current versions. Other revisions for TCP have been proposed and include TCP Vegas [6] and Selective Acknowledgements [16]. 2.2 ABR Mechanism The ABR control unit adjusts the rate of ATM cells into the network dynamically according to the congestion status of the network. This rate is called Actual Cell Rate (ACR). The source is not allowed to send cells faster than ACR to avoid network congestion. The rate-based control scheme (explicit rate or relative rate marking (EFCI, Explicit Forward Congestion Indication)) uses special Resource Management (RM) cells [3]. In the Explicit Rate (ER) scheme, the source sends an RM cell once in every N rm (default N rm = 32) cells or in T rm (default T rm = 100 ms) time units along the VC to the destination. Each cell contains three fields that provide feedback to the source: Congestion Indication (CI) bit, a No Increase (NI) bit, and an Explicit Rate field. The source sets the RM cell s ER field to the rate at which it likes to send and transmits the Forward RM (FRM) Cell. As each FRM cell is received by the destination, it is turned around and transmitted back to the source as a Backward RM (BRM) Cell. Any of the CI, NI and ER fields may be changed by an ATM switch along the VC or the destination before the corresponding BRM cell returns to the source. The calculation of the fair share can be done in accordance with any congestion control scheme recommended by the ATM Forum such as ERICA (Explicit Rate Indication for Congestion Avoidance) [14]. ACR always varies between the limits of MCR (Minimum Cell Rate) and PCR (Peak Cell Rate). Several rules are defined in [3] for modifying ACR according to the received informations. The major ones are: NI bit = 0 and CI bit = 0, ACR max[mcr, min[er, PCR, ACR + RIF x PCR]] NI bit = 1 and CI bit = 0, ACR max[mcr, min[er, ACR]] CI bit = 1, ACR max[mcr, min[er, ACR(1 - RDF)]] RIF (Rate Increase Factor) and RDF (Rate Decrease Factor) are defined in [3] (default value: /16). 3 Explicit Congestion Notification in TCP The TCP congestion control algorithms need to infer the state of the network from lost segments and variations in the RTT or throughput. The reliance on dropped packets to detect congestion affects the performance of delay sensitive applications. This is due to the time required to retransmit lost packets and the need to buffer segments until missing data has arrived. A second disadvantage is the slow response to congestion as a result of the TCP sender waiting for three duplicate ACKs or a retransmit timeout before reacting [9]. As an alternative, Source Quench and fields in packet headers can be used to obtain information on the level of congestion in the network [9]. The proposal by Ramakrishnan and Floyd in [17] uses bits in the Internet Protocol (IP) and TCP headers to inform the TCP source about congestion in the network. In the 3

4 proposal, routers mark the Congestion Experienced (CE) bit in IP headers in response to incipient congestion based on average queue lengths, using the RED (Random Early Detection) algorithm [10,5]. Instead of dropping the packet, RED sets the CE bit in the packet header if such a bit is provided by the IP header and understood by the transport protocol. When an IP packet with a CE bit set reaches its destination, the TCP receiver sets the -Echo flag in the header of the next outgoing ACK segment. The TCP receiver continues to set the -Echo flag until it receives a TCP data segment with a Congestion Window Reduced (CWR) flag set in its header. This protects against ACKs that may be dropped by the network. In delayed ACK implementations, the -Echo flag is set if any of the IP packets received in that interval have their CE bit set. The response to is further complicated by Fast Retransmit and Retransmit Timeout events. Another congestion control method called Binary Congestion Notification (BCN) in TCP also obtains feedback from the network through bits in the IP header. As BCN attempts to integrate TCP and ATM s ABR service, its response to congestion differs from that of [19]. Integration is achieved by getting TCP to reduce its window by a multiplicative decrease factor (MDF) when congestion is experienced. This reaction to congestion is similar to the ABR response thus attempting to match the data rate of both layers based on feedback from the network. 4 Integrating TCP and ABR TCP and ABR use two different ways to detect congestion in the network: TCP uses packet losses to regulate its rate (window) whereas ABR uses the feedback information transported by RM cells to control its ACR. It is worth mentioning that ABR does not wait for a packet loss to reduce its rate. Also, in case of congestion, the ABR control loop is designed to slow down the ABR source. With current implementations of TCP, ATM networks are limited to packet drops as the only viable mechanism to inform TCP sources of congestion. Thus, notification is implicit. The current TCP proposal for TCP/IP networks relies on modifications to the IP header by elements in the network [17]. When TCP runs over ATM, access to the IP header by switches can incur high buffering and processing overheads. Hence we look to the ATM cell for a congestion indication mechanism. The ABR service provides this through the CI field in its RM cell feedback scheme. The EFCI bit in all ATM cell headers can be used to obtain congestion feedback at the receiver for TCP to operate, even when another ATM service such as UBR (Unspecified Bit Rate) is employed. Having chosen the feedback mechanism, we are left with the problem of processing the feedback and informing the TCP receiver about congestion in the network. We achieve this through the use of an agent as discussed in the next two sections. 4.1 Role of the Agent The agent bridges the gap between the ATM layer and the TCP layer in the protocol stack at the receiver end. Each RM cell received by the ABR layer results in the CI value of that cell being sent to the agent. The end of the (AAL5) Protocol Data Unit (PDU) indicator received at the ATM layer is also signalled to the agent. The agent can then decide if the network has experienced congestion during the newly received packet s life. If it is deduced that congestion was experienced, the agent informs the TCP receiver and the next outgoing ACK will have its -Echo bit set. Procedures in [17] can then be used to respond to the congestion at the TCP sender. The implementation proposed requires that changes be made at the receiver s ATM and TCP layers. This can be further simplified if the agent sets the IP header s CE bit, thus making TCP totally unaware about the congestion feedback source which could have come from an ATM network or a TCP/IP network. Figure 1 4

5 shows the location of the agent in the protocol stack and the logical link between the TCP receiver and TCP sender. Application TCP Sender -Echo Indicates Congestion Application TCP Receiver Indication of Congestion IP IP AAL5 Sender AAL5 Receiver Agent ABR Source ABR Sink CI information and end of PDU indication ATM Network Alternative is to set CE bit in IP header. Network will be transparent to TCP. Figure 1: Location of Agent and Operation 4.2 Algorithms Variations in ABR s ACR occur at a very high rate (small time scales) while TCP s window adjustments operate over longer time scales. Initial efforts in developing an algorithm to decide if congestion was experienced in the network centred on calculating a value for ACR at the receiver based on CI, NI and ER from FRM cells. The goal was to observe trends in the ATM cell rate and attempt to adjust TCP s congestion window accordingly. A weighted scheme was tried with more recent ACR values being given higher importance: from the beginning until the detection of the end of a packet, ACRi in RM celli received at time ti has a weight (wi) greater than ACRj in RM cellj received at tj for ti > tj. Several reasons why this failed are: (1) the ERICA algorithm resulted in ER variations due to fair sharing of bandwidth and not just congestion; (2) variations in ATM s ACR occur at a very high rate (small time scales) while TCP s window adjustments operate over longer time scales, and (3) TCP s window adjustment causes a drastic reduction in packet flow while ATM makes fine adjustments to its flow rate. It was then decided that only CI feedback will be used to determine if the network is experiencing congestion. The following criteria has been adopted: if half or more of the RM cells in a packet have the CI bit set, then the packet is said to have experienced congestion. 5

6 Since a relative and not absolute measurement is used, this rule is fair to both large and small packets. In cases where the last cell of the packet is lost, the agent will perform the measurement over two packets. This cannot be avoided because the agent does not know the maximum packet size. One interesting observation was that during congestion, RM cells tend to arrive in batches and often without data cells in between them. This is due to preferential treatment given to RM cells by switches. The Agent The agent counts the number of RM cells received since the end of the last packet. It also accumulates the number of CI bits observed for the current packet. The end of the packet (Payload-Type field (PTI) set to 1) is indicated by the ABR sink through a trigger mechanism without any data transfer. On detecting the end of the packet, the agent sends a signal to the TCP receiver to indicate congestion based on the following simple test: if (CI_received RM\_cells\_received / 2 and RM\_cells\_received > 0) Send a signal to the TCP receiver to indicate congestion. The CI and RM cell counters (CI_received and RM_cells_received respectively) are then reset to prepare for a measurement on the next packet. TCP Receiver On receiving a congestion signal from the agent, the TCP receiver begins setting -Echo bits in outgoing ACKs. This continues until a data segment with the CWR bit set to one is received. Special care is taken to ensure that if a segment carrying the CWR flag experienced congestion, it is taken as a new instance of congestion in the network and hence the -Echo bit is set again. TCP Sender The TCP sender s response to the -Echo bit is complex, as it needs to merge s window adjustments with TCP s existing congestion control algorithms. Detailed testing was done to ensure compliance with the proposal in [17]. Flags are used in the source code to prevent window adjustments due to Fast Retransmit/Fast Recovery or RTO from operating at the same time as s window adjustments. The indication of congestion is treated just as a congestion loss in non--capable TCP. That is, the TCP sender halves the congestion window CWND and reduces the slow start threshold SSTHRESH. A variable records when the action should terminate, which is approximately one RTT after an - Echo is received and the window reduced. To inform the TCP receiver about responses to congestion, the CWR bit is set when the TCP sender reduces CWND. Finally, the TCP sender does not increment CWND for acknowledgements that have the -Echo bit set. 5 Experimental Design 5.1 Simulation Experiments and Objectives We describe in this section the network configurations and parameters used in the simulation. Experiments were performed on two network configurations (LAN and WAN). A comparison between between LAN and WAN sized networks was achieved by varying delays between senders, switches and receivers. With each scenario, simulations were run for: (1) constant number of sources and varying buffer size, and (2) constant buffer size and varying number of sources. The objective of the experiments is to observe the behaviour of TCP relative to that of TCP Reno in different circumstances, to do an analysis over the set of experiments when running simulations with TCP Reno and TCP, and to show that the Agent Based algorithm (TCP ) under various scenarios exhibits a better performance in the evaluation of the simulation tests. 6

7 5.2 Topologies LAN Model Simulation is performed on the N source configuration consisting of N identical TCP sources that send data whenever allowed by the window (Figure 2). All traffic is unidirectional (discounting the TCP ACKs and the BRM cells in the reverse direction). N TCP sources and N TCP destinations are connected to the switch. Source 1 Switch Destination 1 ABR Mux Buffer Data Demux Source N Destination N Figure 2: LAN Simulation Model (configuration 1) WAN Model Grp 2 REC Grp 1 REC Switch 0 Switch 1 Switch 2 Switch 3 Grp 3 REC Grp 1 SRC Grp 2 SRC Grp 3 SRC Grp 4 SRC Grp 4 REC Figure 3: WAN Simulation Model (configuration 2) The WAN simulation model, borrowed from [11], is a multistage wide area network with four switches SW i (i=0..3) (Figure 3). The sources are divided into four groups. The purpose of choosing such a complex model is to make it as realistic as possible: Group Source 1 to Destination 1: end-to-end traffic Group Source 2 to Destination 2: two-hop traffic Group Source 3 to Destination 3: two-hop traffic 7

8 Group Source 4 to Destination 4: one-hop traffic 5.3 Simulation Tool The YATS simulation package is used in the simulation study [4]. YATS is a detailed simulator for TCP over ATM. Most of common features of TCP such as slow-start, congestion detection, congestion avoidance, fast retransmit and fast recovery are included in the simulator. Several objects were modified to incorporate TCP into YATS. The agent was developed from scratch while major changes were made to the TCP receiver and TCP sender according to the proposal in [17]. Minor changes to the ABR sink object allowed CI information and the end of packet indication to be sent to the agent. An -Echo bit was added to the TCP ACK data unit and the CWR bit to the TCP data frame. Traffic is generated by YATS specific objects using a TCP/IP/AAL5/ATM protocol stack. 5.4 Simulation Scenario Experiments are performed using the basic ERICA algorithm implemented at the ATM switch. The default maximum TCP window is 64kB which is sufficient to fill the network pipe in LAN. The TCP sources are greedy sources with infinite supply of data and always have data ready to send as and when permitted by the TCP flow control. A large bulk data transfer application runs on top of TCP sources. and recognises the start-stop protocol. The effects of TCP packet size are studied using sources that generate 512 and 9180 bytes frames. We take into account 20 bytes of TCP header and 20 bytes IP header. This was implemented by altering the Maximum Transfer Unit (MTU) parameter of the TCP sender. The timestamp option (provided by YATS) appears in any data or ACK segment, adding 12 bytes to the 20 bytes TCP header, and gives the sender an accurate RTT measurement for every ACK segment [13]. To study the congestion behaviours at ATM and TCP layers, we consider that another VBR (greedy) application is running in the background. VBR application has higher priority than ABR. We assume the ATM switch has a buffer shared by all VCs passing through. The scheduling policy at the buffer is FIFO. In order to induce variable congestion in the switches, the size of the shared buffer is varied for the simulation cases. The buffer sizes chosen are in the range [ ] cells. The earlier part of the range offers very tight buffer constraint, while on the other hand the latter part of the range offers much more relaxed buffer constraints. The switches are configured with ERICA and binary feedback (CI) but with no specific cell drop policy. A threshold at the ATM switch is set to half the buffer size to turn congestion indication on and change the CI field along the VC or the destination before the corresponding BRM cell returns to the source. All links run at Mbps. We consider a LAN scenario (Scen.LAN) where the delay (dlan + dswitch + dlan) is set to a typical value of 100 µs (Figure 2). The number of sources N is set in the range [ ]. The values of N are arbitrarily selected, but nonetheless deemed sufficient for producing traffic flow required in the simulation study of high-speed networking. In the WAN model (Scen.WAN), the TCP sources each establish a single connection with the similarly numbered destination across the ATM switches and the bottleneck links (Figure 3). As in the LAN model, the internal delay in each ATM switch is set to 10 µs. We consider delay (dwan*5 + dswitch*4) is set to a typical value of 10 ms. Finally, the number of sources in each group is set in the range [20..50]. In both scenarios all the sources start transmitting at the same time. The analysis of the performance results is based on long-term congestion behaviour of the network. 6 Results and Evaluation 8

9 6.1 Performance Metrics Four performance metrics were used to compare TCP Reno and TCP for ATM. It was desirable to observe performance at both the TCP and ATM layers. The metrics chosen for ATM are the Cell Loss Ratio (CLR) and the average buffer utilisation whereas TCP s metrics are the Packet Retransmitted Ratio (PRR) and the average Throughput per connection. The average buffer utilisation is the percentage of the buffer that has been occupied on average. It is measured in each switch to show if the RM cell feedback mechanism and TCP are able to keep queue lengths below the threshold specified (50% lower and 75% upper). The TCP window size also affects queue lengths since it is able to regulate data flow over longer durations when compared to ATM. A larger buffer utilisation results in poorer delay characteristics for connections with cells in the buffer. 6.2 LAN Configuration The following sections discuss simulation variables and their effects on the congestion control methods. From the simulation results, it was observed that segment size had minimal influence on the performance of TCP Reno and TCP. On the whole, TCP produced favourable results for this configuration with regards to CLR, PRR and switch buffer utilisation. Improvements or degradation in throughput was less noticeable especially in the WAN configurations. It must be noted that throughput for varying number of sources spans several orders of magnitude and small differences between TCP and TCP Reno may not be visible Varying the CLR and PRR TCP consistently achieved lower cell loss and packet retransmissions regardless of the number of sources present. Simulations with fewer than 140 sources recorded zero cell loss. PRR exhibited similar characteristics to CLR as expected. The following graph shows the relative performance of the two methods for a LAN-512byte environment. ER I C A - B uf f e r S i z e TC P P a c k e t S i z e E-03 RE NO 1.60E E-03 CLR (%) 1.20E E E E E The feedback mechanism appears to reduce CLR and PRR by making the source window-limited when congestion is noticed on the network. Throughput Both congestion control methods gave comparable throughput measurements. Closer examination of larger source simulations shows this to be true as illustrated in the following graph for the WAN-9128byte scenario. 9

10 ER I C A - B uf f e r S i z e TC P P a c k e t S i z e E+06 RE NO 1.20E E E E E E Buffer Utilisation In almost all the experiments, TCP maintained lower average buffer lengths. The graph below shows a LAN-9128byte environment with TCP following the general trend of TCP Reno but with a bias downwards. Switch Average Buffer ER I C A - B uf f e r S i z e TC P P a c k e t S i z e E+01 RE NO 6.00E E E E E+01 E CN Changes in window size due to the TCP mechanism provides a longterm congestion control strategy when compared to ATM s ACR control. This has resulted in the lower buffer lengths observed here. Maintaining a lower buffer utilisation is desirable because it reduces the chances of cell loss and gives the traffic better delay characteristics. TCP is able to provide significant improvements in PRR, CLR and buffer utilisation with minimal impact on throughput. This was found to be true in all the scenarios investigated Varying the CLR and PRR As expected, the results show that cell loss decreases when larger buffer sizes are used. CLR and PRR measurements were significantly lower in TCP simulations when compared to TCP Reno. This is shown in the results for a WAN-512byte environment below. 10

11 3.50E E-03 ER I C A - S our c e s TC P P a c k e t S i z e 5 12 RE NO 2.50E E E-04 Throughput Throughput was seen to improve with larger buffer sizes due to the increased ability of the switch to cope with congestion. LAN simulations for TCP showed noticeable improvements in throughput for medium sized buffers (32kB to 128kB) as shown in the graph below (LAN-512byte). 3.75E E+06 ER I C A - S our c e s TC P P a c k e t S i z e 5 12 RE NO 3.65E E E E E E E E+06 Buffer Utilisation The graph shows TCP s ability to maintain lower queue lengths in a WAN-9128byte environment. Similar results were achieved for LAN. Switch Average Buffer 8.00E E E E E E E+01 ER I C A - S our c e s TC P P a c k e t S i z e RE NO E CN 11

12 By matching CLR, throughput and buffer utilisation graphs the following can be seen: Cell loss is greatest for small buffer sizes. Buffer utilisation is below the 10% mark for a buffer size of 4kB and increases after that. Note that the lower threshold is set at 50%. The lower throughput at small buffer sizes (number of sources constant) indicates longer link idle times. In most cases, TCP showed a significantly lower CLR/PRR for small buffer sizes. All these points indicate bursty traffic with long idle times and that the TCP advantage is most striking for such traffic characteristics. The 9128 bytes scenario achieved 10\% higher throughput when compared with 512 bytes simulations. 6.2 WAN Configuration Analysis of results in the four switch configuration is more complex due to the number of switches and groups involved. The vast difference in characteristics of the groups makes comparisons between different groups difficult. A brief general comparison of results between the various groups is given below: CLR and PRR WAN scenarios resulted in groups 1 and 2 having similar results. Group 3 encountered the highest losses and retransmissions while group 4 had the lowest. In LAN simulations, groups 1 and 3 had similar results while group 2 had the lowest and group 4 the highest losses. Throughput In all experiments, throughput measurements were similar for group 1 and group 3. Groups 2 and 4 had higher values. This is reasonable because groups 1 and 3 both experience bottlenecks at switch 2 and again at switch 3. Buffer Utilisation Switch 0 consistently encountered a zero average queue length and switch 2 had the highest. This indicates that switch 0 is not experiencing congestion and switch 2 is most likely a bottleneck. TCP and TCP Reno metrics did not differ significant in most cases. The following sections will concentrate on major differences that were observed between the two methods. The few instances where 512 byte simulations exhibit different characteristics from 9128 byte simulations will also be noted Varying the CLR and PRR LAN simulation results showed little sign of cell loss or packet retransmission except for very large source numbers. In WAN simulations, TCP out performed TCP Reno for 512byte simulations. TCP failed to perform well when using 9128byte segments. Throughput Both methods achieved identical throughputs in all cases. 12

13 Buffer Utilisation No significant differences in performance were observed between TCP and TCP Reno except in switch 2 (WAN-512byte) when utilisation exceeded 20%. The follow graph shows TCP maintaining average buffer utilisation below 15%. Switch 2 Average Buffer 2.50E E E E+00 ER I C A - B uf f e r S i z e TC P P a c k e t S i z e 5 12 RE NO E CN Varying the CLR and PRR The LAN scenarios did not generate significant losses except for very small buffer sizes. TCP consistently outperformed TCP Reno in the WAN-512byte environment. TCP results for WAN-9128byte simulations are mixed as shown in the group 1 PRR graph below. 1.40E E-02 ER I C A - S our c e s TC P P a c k e t S i z e RE NO 8.00E-03 Throughput The LAN environment shows throughput increasing with buffer size for groups 1,3 and 4. Throughput for group 2 decreases as the buffers increase. No conclusion can be drawn on the relative performance of TCP and TCP Reno. In WAN scenarios, TCP achieved better throughput for groups 2 and 4 while results for the remaining groups were poorer. The following graph shows improvements to group 2 in the WAN-9128byte scenario. 13

14 9.20E E+05 ER I C A - S our c e s TC P P a c k e t S i z e RE NO 9.00E E E E E E E E+05 Buffer Utilisation TCP lowered buffer utilisation in switches 2 and 3 for WAN experiments. The graph below shows TCP maintaining the average buffer length of switch 2 in the WAN-512byte scenario under the 50% mark. Switch 2 Average Buffer 7.00E E E E E E+01 ER I C A - S our c e s TC P P a c k e t S i z e 5 12 RE NO E CN 7 Summary and Conclusion In this paper the enhancement of TCP s congestion control mechanisms using Explicit Congestion Notification over ATM networks has been overviewed. In addition to packet losses as an indication of congestion, an agent at the network s edge that bridges the gap between the ATM layer and the TCP layer in the protocol stack notifies congestion which results in an adjustment of the credit-based window size of TCP. To do so, the agent uses ABR s rate-based flow control. The simulation environment and simulation results over LAN as well as WAN distances have been presented. An overall comparison of the LAN results indicates that TCP performs better, relative to TCP Reno. This is clearly seen from all CLR, PRR and in throughput results for varying switch buffer size. Buffer utilisation was kept below 50% in the LAN scenario. Average switch buffer lengths were always maintained at lower levels in TCP implementation thus giving the traffic better delay characteristics. However throughput achieved while varying the number of sources was almost identical for TCP Reno and TCP. Performance differences between TCP and TCP Reno were more pronounced for WAN simulations with TCP outperforming TCP Reno. Further work must be done in investigating the 14

15 fairness of this explicit notification scheme more thoroughly as various studies [19] have shown that using for example binary congestion notification results in a bad fairness behaviour. The EFCI bits (available in every ATM cell header) can be used to generalise the response to the ATM UBR service. UBR is of particular interest since it does not have any ATM guarantees or feedback mechanism. Through the use of an agent at the network s edge, TCP s congestion control may become sufficiently effective to overcome UBR s shortcomings [7]. Unlike the use of the CI bit in ABR RM cells to determine congestion, EFCI provides more up to date information on the network since it is received with every ATM cell. References [1] Proceedings of the 3rd International Conference on Autonomous Agents (Agents 99), Seattle, Washington, May ACM. [2] M. Allman, V. Paxson and W. Stevens. TCP Congestion Control. RFC 2581, Apr [3] ATM Forum Technical Committee. Traffic Management Specification. Version 4.1, AFTM , Mar 1999 [4] M. Baumann. Yet Another Tiny Simulator Version 0.3. Communications Laboratory, Dresden University of Technology, Jan [5] B. Braden, D. Clark, J. Crowcroft, B. Davie, S. Deering, Estrin, S. Floyd, V. Jacobson, G. Minshall, C. Partridge, L. Peterson, K. Ramakrishnan, S. Shenker, J. Wroclawski and L. Zhang. Recommendations on Queue Management and Congestion Avoidance in the Internet. RFC 2309, Apr [6] L.S. Brakmo, S.W. O'Malley, and L.L. Peterson. TCP Vegas: New techniques for congestion detection and avoidance. In Proceedings of the SIGCOMM 94 Symposium, Aug [7] K. Djemame, M. Kara. An Agent Based Congestion Control and Notification Scheme for TCP over UBR. (In preparation). [8] K. Djemame and M. Kara. Proposals for a Coherent Approach to Cooperation between TCP and ATM Congestion Control Algorithms. In J.T. Bradley and N.J. Davies, editors, Proceedings of the 15th Annual UK Performance Engineering Workshop (UKPEW 99), pages , Bristol, UK, Jul [9] S. Floyd. TCP and Explicit Congestion Notification. ACM Computer Communication Review, 24(5):8-23, Oct [10] S. Floyd and V. Jacobson. Random Early Detection gateways for Congestion Avoidance. IEEE/ACM Transactions on Networking, 1(4): , Aug ftp://ftp.ee.lbl.gov/papers/early.pdf. [11] M. Hashmani, K. Kawahara, H. Sunahara and Y. Oie. Issues of Congestion Control 15

16 and Notification Schemes in ATM Networks and Proposal of EPRCAM. In Proceedings of ICC98, pp , Atlanta, GA, Jun IEEE. [12] V. Jacobson. Congestion Avoidance and Control. ACM Computer Communication Review, 18: , August Proceedings of SIGCOMM 88 Symposium, Stanford, CA. ftp://ftp.ee.lbl.gov/papers/congavoid.ps.z. [13] V. Jacobson, R. Braden and D. Borman. TCP Extensions for High Performance. RFC 1323, May [14] S. Kalyanaraman, R. Jain, S. Fahmy, R. Goyal, and B. Vandalore. The ERICA Switch Algorithm for ABR Traffic Management in ATM Networks. Submitted to IEEE/ACM Transactions on Networking, Nov [15] M. Kara and M.A. Rahin. Towards a Framework for Performance Evaluation of TCP Behaviour over ATM. In Proceedings of the 13th IFIP International Conference on Computer Communication (ICCC 97), pages 49-60, Nov [16] M. Mathis, J. Mahdavi, S. Floyd and A. Romanow. TCP Selective Acknowledgement Options. RFC 2018, April [17] K. Ramakrishnan, S. Floyd. A Proposal to add Explicit Congestion Notification () to IP. RFC 2481, January [18] A. Romanow and S. Floyd. Dynamics of TCP Traffic over ATM Networks. IEEE Journal on Selected Areas in Communications, 13(4): , May [19] D. Sisalem and H. Schulzrinne. Binary Congestion Notification in TCP. In Conference Record of the International Conference on Communications (ICC), Dallas, Texas, Jun IEEE. [20] W.R. Stevens. TCP/IP Illustrated, Vol.1. Addison Wesley, [21] W. Stallings. High-Speed Networks. TCP/IP and ATM Design Principles. Prentice- Hall,

17 Acronyms AAL5 ATM Adaptation Layer 5 ABR Available Bit Rate ACK Acknowledgement ACR Allowed Cell Rate ATM Asynchronous Transfer Mode BCN Binary Congestion Notification BRM Cell Backward Resource Management Cell CE Congestion Experienced CI Congestion Indication CLR Cell Loss Ratio CWND Congestion Window CWR Congestion Window Reduced Explicit Congestion Notification EFCI Explicit Forward Congestion Indication EPD Early Packet Discard ER Explicit Rate ERICA Explicit Rate Indication for Congestion Avoidance FRM Cell Forward Resource Management Cell LAN Local Area Network IP Internet Protocol MCR Minimum Cell Rate MTU Maximum Transfer Unit NI No Increase PCR Peak Cell Rate PPD Partial Packet Discard PRR Packet Retransmission Ratio PTI Payload-Type RDF Rate Decrease Factor RIF Rate Increase Factor RTT Round Trip Time RTO Retransmission Time Out SSTHRESH Slow Start Threshold TCP Transmission Control Protocol UBR Unspecified Bit Rate VBR Variable Bit Rate VC Virtual Circuit WAN Wide Area Network 17

18 CLR (%) 1.20E E-03 ERICA - Sources 30 - TCP Packet Size 512 Switch Average Buffer 6.00E E E E E+01 ERICA - Sources 30 - TCP Packet Size E E+06 ERICA - Sources 30 - TCP Packet Size E E-04 ERICA - Sources 30 - TCP Packet Size E E E E E E E E E E E E E E E+06 Configuration 1 - LAN - Varying CLR (%) 1.80E E E E E E E E-04 ERICA TCP Packet Size Switch Average Buffer 7.00E E E E E E+01 ERICA TCP Packet Size E E E E E E E+06 ERICA TCP Packet Size E E E E E E E E-06 ERICA TCP Packet Size Configuration 1 - LAN - Varying 18

19 CLR (%) 1.20E E-03 ERICA - Sources 30 - TCP Packet Size 9128 Switch Average Buffer 7.00E E E E E E+01 ERICA - Sources 30 - TCP Packet Size E E E E E E E E E+06 ERICA - Sources 30 - TCP Packet Size E E E E E-04 ERICA - Sources 30 - TCP Packet Size E+06 Configuration 1 - LAN - Varying CLR (%) 1.80E E E E E E E E-04 ERICA TCP Packet Size Switch Average Buffer 7.00E E E E E E+01 ERICA TCP Packet Size E E E E E E E E E+06 ERICA TCP Packet Size E E E E E E-04 ERICA TCP Packet Size Configuration 1 - LAN - Varying 19

20 CLR (%) 2.00E E E E E E-03 Switch 0 Average Buffer 1.00E E E E E E E E E E E E E E E E E E E E E E E E E+05 Configuration 2 - Group 1 - WAN - Varying CLR (%) 2.50E E E E-03 ERICA TCP Packet Size 512 Switch 0 Average Buffer 1.00E E E E E E E E E E-01 ERICA TCP Packet Size E E E E E E E+06 ERICA TCP Packet Size E E E-04 ERICA TCP Packet Size 512 Configuration 2 - Group 1 - WAN - Varying 20

21 CLR (%) 2.50E E E E-03 Switch 1 Average Buffer 7.00E E E E E E E E E E E E E E E E E E E+05 Configuration 2 - Group 2 - WAN - Varying CLR (%) 2.50E E E E-03 ERICA TCP Packet Size 512 Switch 1 Average Buffer 3.00E E E E E E-01 ERICA TCP Packet Size E E E E E E E+06 ERICA TCP Packet Size E E E E-04 ERICA TCP Packet Size 512 Configuration 2 - Group 2 - WAN - Varying 21

22 CLR (%) 6.00E E E E E-02 Switch 2 Average Buffer 7.00E E E E E E E E E E E E E E E E E E E E E E+05 Configuration 2 - Group 3 - WAN - Varying CLR (%) 6.00E E E E E-02 ERICA TCP Packet Size 512 Switch 2 Average Buffer 2.50E E E E+00 ERICA TCP Packet Size E E E E E E E+06 ERICA TCP Packet Size E E E E E E-04 ERICA TCP Packet Size 512 Configuration 2 - Group 3 - WAN - Varying 22

23 CLR (%) 3.50E E E E E-04 Switch 3 Average Buffer 3.00E E E E E E E E E E E E E E E E E E E E E E E E E+05 Configuration 2 - Group 4 - WAN - Varying CLR (%) 7.00E E E-03 ERICA TCP Packet Size 512 Switch 3 Average Buffer 9.00E E E E E E E E E+00 ERICA TCP Packet Size E E E E E E E+06 ERICA TCP Packet Size E E E E E E-04 ERICA TCP Packet Size 512 Configuration 2 - Group 4 - WAN - Varying 23

24 CLR (%) 9.00E E E E E-03 Switch 0 Average Buffer 1.00E E E E E E E E E E E E E E E E E E E E E E+05 Configuration 2 - Group 1 - WAN - Varying CLR (%) 8.00E E E E-03 ERICA TCP Packet Size 9128 Switch 0 Average Buffer 1.00E E E E E E E E E E-01 ERICA TCP Packet Size E+07 ERICA TCP Packet Size E-02 ERICA TCP Packet Size E E E E E-03 Configuration 2 - Group 1 - WAN - Varying 24

25 CLR (%) 9.00E E E E E-03 Switch 1 Average Buffer 3.50E E E E E E E E E E E E E E E E E E E E+05 Configuration 2 - Group 2 - WAN - Varying CLR (%) 9.00E E E E E-03 ERICA TCP Packet Size 9128 Switch 1 Average Buffer 3.00E E E E E E-01 ERICA TCP Packet Size E E E E E+06 ERICA TCP Packet Size E E E-03 ERICA TCP Packet Size 9128 Configuration 2 - Group 2 - WAN - Varying 25

26 CLR (%) 2.50E E E E-03 Switch 2 Average Buffer 7.00E E E E E E E E E E E E E E E E E E E E-03 Configuration 2 - Group 3 - WAN - Varying CLR (%) 2.50E E E E-03 ERICA TCP Packet Size 9128 Switch 2 Average Buffer 1.40E E E E E E+00 ERICA TCP Packet Size E E E E E+06 ERICA TCP Packet Size E E E E E E E-02 ERICA TCP Packet Size 9128 Configuration 2 - Group 3 - WAN - Varying 26

27 CLR (%) 1.60E E E E E E E-04 Switch 3 Average Buffer 3.50E E E E E E E E E E E E E E E E+05 Configuration 2 - Group 4 - WAN - Varying CLR (%) 5.00E E-03 ERICA TCP Packet Size 9128 Switch 3 Average Buffer 9.00E E E E E E E E E+00 ERICA TCP Packet Size E+07 ERICA TCP Packet Size 9128 ERICA TCP Packet Size E E E E E E-03 Configuration 2 - Group 4 - WAN - Varying 27

28 28