Packet Marking for Web traffic in Networks with RIO Routers

Transcription

1 Packet Marking for Web traffic in Networks with RIO Routers Marco Mellia- Politecnico di Torino, Torino - Italy Ion Stoica- University of California, Berkeley - CA 97 Hui Zhang- Carnegie Mellon University - PA, Abstract RIO routers, initially proposed in the context of Differentiated Services networks, distinguish between two classes of packets In-profile and Outprofile based on a one-bit marking in packet headers. Traditionally, packetmarking is performed by ingress routers according to a user-profile specified in terms of bandwidth. In this paper, we consider an alternative strategy in which packets are marked based on the state of individual TCP flows. This marking scheme can be implemented either at the end systems or ingress routers (thus avoiding changes at end systems). In addition, we present a variant of our scheme that integrates the Diffserv style packet-marking based on the user-profile. By extensive simulations, we demonstrate that the proposed scheme can significantly reduce the completion time of TCP flows. In particular, the completion time of short TCP flows is halved. These results are consistent over different TCP variants and in the presence of ECN. I. INTRODUCTION In today s Internet, routers process all packets uniformly. As a result, only one basic service is provided: best-effort datagram delivery. There have been a number of efforts to enhance the Internet to support QoS and service differentiation [], []. To support this, routers need to implement more sophisticated scheduling and buffer management policies that discriminate packets according to the service classes they belong to. While these more sophisticated router functionalities are designed mainly to support QoS and service differentiation, in this paper, we study whether it is possible to exploit the new router functionalities to improve the performance of the best-effort TCP traffic, and in particular on WWW-traffic, which is the dominant type of the traffic in today s Internet. We consider a simple buffer management algorithm called RIO, which distinguishes between two classes of packets, Inprofile (IN) and Out-profile (OUT), based on a one-bit marking in packets headers. RIO was proposed to support one type of Differentiated Services called Assured Service or Expected Capacity Service [], []. When the buffer starts to build up, a RIO router always drops the OUT before the IN ones. In this way, IN packets will receive a better service than OUT packets. In this paper, we study the feasibility of improving the performance of TCP flows in a network with RIO routers by marking packets according to per flow TCP state. The key observation is that TCP performance suffers significantly when it operates in the small window regime. This is because losses during the small window regime may cause retransmission timeouts (RTOs), which will ultimately result in TCP entering the Slow Start phase. The objective of the TCP-aware marking algorithm is then to selectively mark packets in order to reduce the possibility of TCP entering this undesirable state. We propose a simple scheme and evaluate its performance by simulation. The proposed scheme can reduce the completion time of short lived TCP flows by half. These gains are consistent over different TCP variants ranging from TCP-RENO to the combination of Early Congestion Notification with TCP-SACK. The scheme can be easily implemented in end systems. In addition, we describe an implementation in which only ingress routers mark the packets. In this way, the marking algorithm can be implemented as a value-added service at the network edge without changing the end systems. The rest of the paper is organized as follows: the following two sections review the RIO buffer management scheme and the TCP congestion control algorithm. In Section IV we present a packet marking scheme that improves TCP performance under a wide range of scenarios. We then evaluate the proposed scheme through simulations in Section V. In Section VI we discuss implementation and deployment issues. Finally, we summarize our findings in Section VII. II. THE RIO ALGORITHM The RIO algorithm distinguishes between two classes of packets, IN and OUT. While both type of packets share the same FIFO queue, they are dropped according to different policies. Intuitively, RIO can be viewed as the combination of two RED algorithms [] with different drop probabilities curves such that the OUT packets are dropped earlier than the IN packets. Thus, two sets of REDlike parameters are used: fmin th in;max th in; max p in;avg ing, which are used to compute the dropping probability of IN packets, and fmin th out;max th out; max p out;avg total g, which are used to compute the dropping probability of OUT packets. There are several points worth noting. First, since max th out and min th in are set such that max th out < min th in, a RIO router will first drop OUT packets when queue starts to build up, then drop all OUT packets if the queue continues to build up, and finally drops IN packets as a last resort. Second, avg in, which is used to compute the dropping probability of IN packets, accounts for the IN packets only, whereas avg total, which is used to compute the dropping probability of OUT packets, accounts for both IN and OUT packets. Note that if all packets are of the same type (i.e., either IN or OUT), then RIO reduces to RED. Unless otherwise specified, in the case of RED we use the parameters suggested in [], i.e., (min th, max th, max p) (; ; :). In the case of RIO we use the same set of parameters for the OUT packets, and we use (; 7; :) for the IN packets. Note that as long as only a small fraction of traffic is marked, the average queue size of IN packets will be virtually zero, which means that the actual values of the parameters used by RIO-IN will have little impact on the marked traffic. This allows us to study our marking algorithms independent of the RIO-IN parameters. Finally, we assume a total buffer size of packets, i.e., three times the (min th threshold. III. TCP CONGESTION CONTROL Due to the predominance of TCP traffic, the TCP congestion control mechanism plays a major role in the dynamics of today s Internet traffic. While there are many variants of TCP [], [], [7], [], they share a common design principle: the loss of a packet is interpreted by the source as a congestion notification. When such an event occurs, the source simply slows down. TCP infers that a packet is lost whenever one of the following two events occurs: a Retransmission Time Out (RTO), or the arrival of three duplicate ACKs. Of these two events, RTO is the most undesirable one as the timeout period is usually very large. There are two reasons for this. First, the computation of the timeout period takes into account the variance of the Round Trip Time (RTT) which can be quite high. Second, most of today s operating systems use a very coarse timer granularity (i.e., ms). In practice, this leads to a minimum timeout period of about ms. In addition, once an RTO occurs, TCP also resets -7-7-//$7. (C) IEEE

2 Group [bytes] Length [packets] Group 7 9 [bytes] Length [packets] 9 the congestion window to one and re-enters the Slow Start phase. As a result, RTO events can significantly affect the TCP throughput. In contrast, three duplicate ACKs trigger Fast Retransmit and Fast Recovery, which is by far a more efficient way of detecting and handling the congestion. Unfortunately, there are many cases in which Fast Retransmit cannot be triggered. As a result, RTO remains the last resort to detect packet losses, which significant affects the TCP performance. In the next section we illustrate this point by presenting a common scenario that lead to RTO: the small window regime. A. Small window regime TCP performance degrades greatly when the congestion window is small. This is commonly known as the small window regime. This can happen either when the data transfer is small or when the loss rate is relatively high. Previous studies have shown that these situations are quite common due to the large fraction of short WWW transfers, and due to the relatively large number of highly congested links in today s Internet [], []. In the case of small transfers, the added latency due to RTO can be significant. This timer is initialized to seconds, and then updated based on the Round Trip Time (RTT) measurements, by using an exponential weighted moving average algorithm. Thus, if the first packet of a TCP connection is lost, the source has to wait for seconds before retransmitting it. Note that even if the RTT estimation is accurate, the coarse timer granularity can force the sender to be idle for a time period comparable to the actual completion time. To illustrate this, consider a scenario in which FTP flows and WWW-like traffic share a bottleneck link of Mbps with a propagation delay of ms. The router that manages the bottleneck link implements a RED buffer management scheme. Ten ingress routers are connected to it by Mbps links with ms of propagation delays. Ten egress router are connected to the other end of the bottleneck link. Between each ingress and egress pair we assume one continuous backlogged TCP flow. To simulate WWW-like traffic pattern, a number of TCP-RENO sources(receivers) are connected to each ingress(egress) router. The flows arrival rates are modeled by Poisson processes. The flows lengths are drawn from the distribution shown in Table I. To derive this distribution we use the AT&T Labs recent measurements of the Internet traffic []. In particular, we divide the flows in identical groups starting from the shortest to the longest flow, and then we compute the average flow length in each group. For simplicity, in our simulations we use a fixed packet size of 7 bytes. The average length over all flows is, bytes, or.7 packets. Note that, as expected, a large percentage of these flows are short: almost % of flows last less than five packets. We conduct experiments for seconds simulation time, and we set the average flow arrival rates such that the offered load corresponds to 9% of the bottleneck link capacity. In Figure we plot the completion times for all the WWW-like sources, using different patterns to differentiate the flow length. For easy visualization flows are grouped in the increasing order of their packet length. 9 Completion Time [s] TABLE I AVERAGE FLOW LENGTH PER GROUP (MTU7 BYTES ). Flow Number Fig.. Completion times for different flow length sources: TCP-RENO and RED buffer management. The layers of points shown in Figure correspond to different completion times. For example, the completion times for flows with only packet can be (a) between. and. seconds if no packet is lost; (b) about seconds if the first packet is lost; and (c) about 9 seconds if both the first packet and its retransmission are lost. The results for -packet flows are similar, except the fact that when the second or the third packets are lost, the timer calculation is more accurate, allowing the source to retransmit the dropped packet faster. Moreover, for flows that are more then packet long, the Fast Recovery algorithm can be triggered, allowing the source to quickly retransmit the dropped packet. IV. TCP AWARE PACKET M ARKING SCHEME In this section we propose a generic TCP aware packet marking scheme which is designed to be used in a network in which routers implement the RIO buffer management scheme. The key idea is to use the packet marking in order to avoid the RTO to occur. To achieve this goal we use the fact that with RIO, IN packets are delivered with a very high probability. More precisely, our algorithm takes advantage of this by marking packets as IN in order to allow TCP to exit as fast as possible from the undesirable states. In particular: mark the first several packets of the flow. This will protect the first packet against loss, and it will allow TCP to safely exit the small window regime. mark several packets after a RTO occurs. The purpose of this is to make sure that TCP exits the small window regime, and, if possible, the Slow Start phase. mark several packets after receiving three DUP-ACKs. The idea here is to allow TCP to exit the Fast Recovery phase, before losing another packet. We present two packet marking schemes. The first one is tightly integrated with the TCP protocol and takes full advantage of the TCP status; the second one is instead decoupled from the TCP status, and thus can be implemented in the edge node. Moreover, the schemes differ in () how many marked packets a source is allowed to send when it starts transmission or when it receives a congestion notification event, and () which packets are marked. With each source we associate a status variable N that represents the number of IN packets that the source is allowed to send. When the source starts transmission, N is initialized to a default value Ns. Every time the source detects a congestion notification event, N is increased by a constant additional number of IN packets Na. Finally, each time a marked packet is sent, N is decremented //$7. (C) IEEE

3 We first propose a packet marking scheme tightly integrated with the TCP protocol. In particular, we assume that when a new TCP flow starts, the source is allowed to send up to Ns marked packets. sstresh Subsequently, the source is allowed to send up to Na marked packets at the beginning of a Slow Start phase, and up to Na cwnd packets at a beginning of a Fast Recovery phase. The intuition behind the above scheme is the following. By allowing a source to send up to sstresh marked packets at the beginning of a Slow Start phase, we increase the probability that the source exits this phase and enters the Congestion Avoidance. Similarly, allowing the source to send cwnd marked packets at the beginning of a Fast Recovery phase, will protect the entire window of packets during this phase. 9 Completion Time [s] A. PM: Packet Marking scheme integrated with TCP Flow Number Fig.. Completion time for the different flow length sources: TCP-RENO and RIO buffer management with PM. B. S-PM: Simplified Packet Marking scheme The packet marking scheme described in the previous section has one major drawback: since it requires to know the state variables sstresh and cwnd to perform packet marking, this scheme has to be tightly integrated with the TCP algorithm. This would require to change the TCP implementation on every platform, which at the minimum it will seriously slow down the pace at which such a scheme can be adopted. To avoid this problem, we propose a second scheme which does not need to know the values of the internal TCP state variables. The key change from the previous scheme is to use a constant value for Na (instead of a value that depends on sstresh and cwnd). We call the new scheme the Simplified Packet Marking scheme, or S-PM, for short, and we discuss it in more details in Section implementation. For the sake of simplicity, here we assume that only data packets can be dropped by the network; ACKs are always received by the sender. In fact, note that we can employ a symmetric scheme, in which the receiver will always send a marked ACK in response to a marked packet. (see section VI). C. Algorithm interpretation The network signals the congestion to a TCP source either implicitly by dropping a packet, or explicitly by setting ECN bits. Allowing a TCP source to send marked packets after it receives the congestion notification closes the feed-back loop. From the network perspective the marked packets mean that the TCP source has received the congestion notification and now it is adapting its rate. By protecting the marked packets the network tries to avoid punishing the flows that are already adapting, and instead signal the congestion to other flows. The rule of allowing an initial number of marked packets has a similar interpretation. When a new connection enters the network, it doesn t have enough information about the level of congestion in the network. Then marking the first packets of a flow notifies the network that the new flow is too young and potentially fragile for congestion notification. After the initial small window regime is ended, the TCP flow stops sending IN packets, and allows the network to give feedback about congestion. In summary, our packet marking scheme can be seen as a part of a control loop that is established between the source and the network: using the marking scheme, the source can reply to the network con In this paper we use the same variable notations, as found in the BSD. source code: sstresh denotes the Slow Start Threshold, while cwnd denote the current value of the Congestion Window. Early Congestion Notification (ECN) [9] has been proposed to provide better support for the TCP congestion control. The basic idea is to explicitly advertise the congestion to end-hosts by using a bit in the IP header, instead of a packet drop. gestion notification message; in turn, the network can use this information to choose the next flow to be notified. V. S IMULATION RESULTS We present simulation results to asses the performance of the proposed algorithms. In the first set of simulations we consider the scenario presented in Section III-A. We repeat the simulation with the same parameters used to obtain the results plotted in Figure. The only difference is that now we assume that each TCP source employs PM, and the bottleneck router implements the RIO buffer manage, and, according to ment scheme. In addition, we assume that Ns sstresh after a RTO, the description of PM in section IV-A, Na and Na cwnd after entering the Fast-Recovery status. Figure plots the distribution of the flows completion times. By comparing Figures and, it can be immediately noticed that the layer corresponding to the first packet being dropped is no longer present. In addition, note that all flows that send a number of packets less or equal to five appear to not lose any packets. In fact, the first flows that lose packets are the ones that send eight packets (notice the layer at the first RTO value in this case). The reason for this is simply because a TCP source implementing PM sends the first Ns packets always marked as IN, and therefore the probability that these packets are dropped is negligible. In contrast, the following packets are marked as OUT, and as a result the probability that they are dropped is much higher. This explains why eight packet flows are the first to incur losses. It is interesting to note that sending eight packets represents a worst case scenario. In particular, dropping one of the last three packets will force a RTO, as the sender will receive at most two duplicated ACKs. To gauge the effectiveness of our scheme we define a general metric, called Relative Gain and denoted by, as the relative improvement obtained by using PM. More precisely, ) F (x) F (ref F (ref ) () where F represents the measured quantity (e.g., completion time, throughput), ref represent the reference system employing the RED buffer management discipline, and x represents the system employing the RIO buffer management discipline, and either PM or S-PM packet marking schemes. A. Completion Time vs. Offered Load We consider how does the average completion time of a flow varies as a function of the offered load, and what are the improvements that can be achieved by using PM. In doing this we consider four versions of TCP: RENO, NEWRENO, SACK, and RENO FS(), the last being -7-7-//$7. (C) IEEE

4 Average Completion Time lower_bound RENO NEWRENO SACK RENO FS() ECN-SACK Normal PM Gain versus RENO Normal RENO NEWRENO SACK RENO FS() ECN-SACK Normal PM Web Offered Load Fig.. Average completion time for WWW-like traffic versus the offered load. a modified version of RENO proposed in []. In addition, we run a set of simulations with TCP-SACK in a network which provides support for ECN [9]. Figure shows the average completion time versus the offered load. As a base line comparison we plot the minimum completion time, evaluated by assuming that no packets are ever dropped and ignoring the queueing delay. In all the experiments presented in this section the average confidence interval is equal to. Mbps and the maximum is equal to. Mbps, with a confidence level of 9%. As expected, the completion time increases dramatically as the offered load approaches the link capacity. This is mainly due to the packet loss that forces TCP to scale down its transmission rate, and therefore increase its completion time. Note that as the loss number increases, the probability to have several packets dropped in the same congestion window also increases. Ultimately, this will force TCP to enter in the Slow Start phase, which will significantly increase the completion time. As shown in Figure, by using PM we are able to consistently reduce the completion time across all TCP versions (black dots). On the top of Figure we report the Relative Gain in terms of completion time for various TCP versions with or without PM, as compared to a reference network in which each TCP source implements RENO, and each router implements RED. There are two things worth noting. First, the PM is again effective in improving the performance of every TCP version. Second, these improvements are significantly larger than the improvements achieved by the more sophisticated versions TCP-NEWRENO and SACK over TCP-RENO. Note that even using ECN does not improve the average completion time, as the dropping rate is still very high due to the high offered load and bursty TCP behavior. Moreover, the ECN settings are more reliable for the long lived flows, where the high number of packets sent provides the network a high rate of feedback to the sources. This is also confirmed by Table II, which reports the average completion times for different flow lengths with a network load of 9%. Note that by using PM we reduce the completion times for all flows. In the case of short flows we achieve a gain close to 7%. Note also that long lived flows are able to reach % of gain. Using ECN instead, will reduce only the completion time for the long lived flows, while only slightly improving the performance of the short lived one. It must also be noted that RENO FS() guarantees better performance when the network load is relatively low, but, at the same time, it is also the one which performs the worse when the network is heavy loaded. This is due to the fact that sending at minimum three packets can be a wrong decision when the networks is heavy loaded, and can potentially cause instability. Instead, the gain obtained using PM is always greater than % and increases up to % when the network is heavy loaded. Gain: Normal versus PM Web Offered Load RENO NEWRENO SACK RENO FS() ECN-SACK Web Offered Load Fig.. Completion Time Relative Gain: with respect to TCP-RENO on the top and same TCP with or without PM on the bottom ṪABLE II AVERAGE COMPLETION TIME PER GROUP AND SPEEDUP GAIN. flow length Comp Time [s] [%] [pkts] Normal ECN PM ECN PM The bottom part of Figure plots the gain in terms of completion time obtained in the case of each TCP version with and without PM. What is important to note here is that by using PM we achieve similar improvements in all cases. This shows again that PM is equally effective in improving the performance of each TCP version. Also note that the gain increases with the offered load, as long as the offered load is smaller than 9%. However, when the offered load exceeds 9%, the gain starts to decrease. This is because in a highly loaded network, the number of marked packets generated due to the packet losses can become large enough to overflow the buffer. As a result the marked packets are no longer protected. VI. IMPLEMENTATION ISSUES In this section we consider three possible variants of our marking scheme, based on where the packet-marking is performed: (a) integrated with the TCP protocol stack, (b) at the ingress router, or (c) at the source by a proxy module. A. Integrating PM into the TCP protocol stack In this case, the marking scheme is integrated in the TCP stack, as described in section IV-A. The advantage of this scheme is that -7-7-//$7. (C) IEEE

5 Average Completion Time [s] AM PFM Completion time [s] AM: Marked flows AM: Normal flows PFM: Marked flows PFM: Normal flows Toker rate [Mbps] Fig.. Completion time of WWW-like sources versus the token rate T. the entire TCP state is exposed, and therefore it can be used to make optimal marking decision. The problem with this approach is that it is hard to limit the offered fraction of IN packets based only on these state variables. There are at least two solutions that address this problem. The first is to use a signaling protocol to inform the source of how many IN packets it can send. The second is to have the ingress router remarking some of the IN packets as OUT. While the second approach is simpler, as it does not require a signaling protocol, un-marking packets without any knowledge of the TCP state can significantly degrade the performance. B. Deploying S-PM at ingress routers In this case, we implement S-PM at the ingress routers, without any changes at the end-hosts. Note that this implementation is fully compatible with the Differentiated Services architecture. In particular, the ingress router has to () identify the beginning of a new flow, () mark the first number of packets according to section V, () identify packet losses in the network, and then () mark the following packets as IN. The flow identification operation can be done very efficiently using a prefix matching scheme or snooping TCP signaling packets, while identifying a packet loss can be done looking at the flow sequence number in the TCP header. Indeed, if the sequence number is not greater than the previous one, and the TCP payload is not empty, then the source is retransmitting old data. As a result the router can conclude that the flow enters either the Fast Recovery or the Slow Start phase. However, since the ingress router does not have the possibility to know the internal TCP state, it cannot differentiate between the two phases. As result, we use the simplified version of packet marking algorithm described in IV-B. Note that the same scheme can be used on the reverse path, by using the ACK sequence number. The values N s;n a can be set by the network manager to provision the resources. Each ingress node can have an IN ratio that is allowed to spend, and it can distribute them to all flows that are present. Using a S-PM scheme that is implemented in the network, will guarantee all the end systems better performance, whichever TCP version they are using, and will also simplify the control upon the congestion in the IN part of the network. C. Deploying S-PM at end-hosts An alternative to deploying S-PM at the ingress routers is to implement a simple module at the end-host which snoops the TCP packets and mark them according to the S-PM scheme. Since this approach does not require changes in the TCP stack, it is relatively simple to develop and deploy. However, note that similar to the case in which PM is integrated into the TCP stack, the ingress router has to be able 7 Toker rate [Mbps] Fig.. Completion time of WWW-like sources versus the token rate T : % S-PM aware and % best effort flows. to unmark packets if needed, in order to avoid congestion of the IN portion of the traffic. Since the performance of this scheme is basically identical to the one in which the S-PM is fully implemented at ingresses, next we present simulation results only for the former one. D. Simulation Results with S-PM deployed at ingress To compare the different solutions, we implemented two types of packet markers as queue objects in the ns- simulator. Both markers use a leaky bucket model to mark packets as IN. The token depth is denoted by T max, the current token level is denoted by T, and the token arrival rate is denoted by T. While in both cases tokens are shared between all connections, the two markers differ in the way the token are shared. In particular, we have: ffl Aggregate Marker (AM): the node does not perform a per flow marking, and a packet that enter the queue is marked as IN as soon as there are tokens in the leaky bucket profile. ffl Per Flow Marker (PFM): as described in section VI-B, the marker is able to recognize new flows and packet drops per flows. Flow i that has been granted IN packets can use them only if there are tokens in the aggregated leaky bucket, i.e., T >. Note that N (i) is decremented even if the packet can not be marked as IN when T<. Figure reports the average completion time in the same network scenario as the one described in section III-A, with WWW offered load equal to % of the bottleneck capacity. We assume a token bucket with the depth T max, and a token arrival rate T that varies from to Mbps. This results in an aggregate rate of the IN traffic on the bottleneck link of up to Mbps. When PFM scheme is used, the completion time is decreasing as the IN load is increasing up to a threshold (about Mbps), after which the performance degrades. This is because as N s increases, the portion of the buffer dedicated to the IN packets becomes eventually congested, which negates the benefit of sending marked packets. In turn this will cause the OUT packets to be dropped with high probability. As a result, once a flow ceases to send marked packets, its subsequent packets are dropped with high probability. This will likely cause TCP to lose multiple packets within the same window, which will significantly affect its performance. In contrast, by using AM the completion time increases with the aggregate token rate profile. This is not surprising, since the longer the flow is, the higher the probability to have a larger congestion window is. This will result in sending more packets per unit of time and then having a larger probability of using an IN token in the aggregate marker. This makes short lived flows more sensitive to a higher dropping probability, which explains the significant increase in the average completion time //$7. (C) IEEE

6 TABLE III COMPLETION TIME WITH DIFFERENT RTTS. Mbps ms Mbps ms Web Web Web Node RTT Completion time [s] Gain Id [ms] Normal PFM % D. PM-aware and Best-Effort Flows A natural question is what happens when traditional best-effort TCP flows compete with PM-aware flows. This scenario will likely happen in the initial deployment stages in which only a subset of ingress routers and/or end-hosts will be PM-aware. To answer this question, we modify the network bottleneck scenario such that only the first five ingress routers implement PFM; the last five router are normal routers. As a result only half of the traffic will be S-PMaware. Figure shows the completion time for the S-PM-aware and best-effort flows as a function of the token arrival rate at the PFM ingress routers. The dashed lines correspond to the scenario where PFM is used, while the solid lines show the results when AM is used. Note that when the token rate increases, the completion times of the PFM aware flows decrease irrespective of the marking scheme. In contrast, the completion times of the best-effort flows increase with the token rate. Further, note that when using AM, the completion times of the best effort flows increase very fast. In contrast, PFM does not exhibit this problem. This makes PFM the natural choice in a heterogeneous network where PM is not completely deployed, as it can avoid starvation of the best-effort traffic. D. Simulations with more complex topologies To gauge the effects of different RTTs on the TCP performance, we consider a topology similar to the one presented in section III-A, where the propagation delay of ingress link i is i ms, i.e., the first ingress router has ms delay, while the last has ms delay. The total aggregate IN profile token rate is set to Mbps. The offered load is set to % of the bottleneck link capacity. Since PFM consistently outperforms AM, in this section we present only the results obtained by using PFM. We report the simulation results in Table III. When PFM is used, the average completion time gain for short flows is in the range %- %. Note that the flows with a smaller RTT obtain better gains, as the impact of eliminating the RTOs has a much higher impact in these cases. Table IV reports the average completion times measured on the parking lot topology in Figure 7. The topology consists of five RIO routers connected by Mbps links with ms delay. Two ingress routers are connected to each RIO router by Mbps links with ms delay. One FTP source and a number of WWW-like traffic sources are connected to each ingress router. All the receivers are connected to the last router. Each ingress router has an IN token arrival rate of. Mbps, which corresponds to a total aggregate IN rate of Mbps on the last bottleneck link. The offered load of the WWW-like traffic on the last link is %. As shown in Table IV, when ingress routers implement PFM, the average gain in the completion time is % %. These results suggest that our packet-marking scheme will be also highly effective in more realistic scenarios. Web Web R R R R R Web Web Web 7 Web Fig. 7. Parking-lot topology. TABLE IV COMPLETION TIME ON A PARKING LOT TOPOLOGY. Node Completion time [s] Gain Id Normal PFM % VII. SUMMARY 9 Receivers In this paper, we study several TCP-aware packet marking schemes in networks with RIO routers. By marking packets based on the state of TCP congestion control algorithms, it is possible to significantly reduce the probability of TCP entering undesirable states, such as retransmission timeouts and small window regime. This directly translates into performance improvements of TCP. Through extensive simulation experiments, we show that the proposed scheme can reduce the completion times of short lived TCP flows by half. We propose a strategy to implement the packet marking scheme at ingress routers. This allows the marking scheme to be offered by network providers as a value-added service. In addition it can be integrated together with the profile-based marking scheme, which is also implemented at ingress routers. REFERENCES [] R. Braden, D. Clark, S. Shenker, Integrated Services in the Internet Architecture: an Overview, IETF Request For Comment, June., 99 [] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, W. Weiss, An Architecture for Differentiated Services, IETF Request For Comment 7, Dec. 99 [] David D. Clark and Wenjia Fang, Explicit Allocation of Best Effort Packet Delivery Service, IEEE/ACM Transaction On Networking Vol:, NO:, pp. -7, Aug. 99 [] S. Floyd, V. Jacobson, Random Early Detection gateways for Congestion Avoidance, IEEE/ACM Transaction on Networking, Vol:, NO:, Aug. 99 [] V. Jacobson, Congestion Avoidance and Control, Sigcomm, Standford, CA, pp. -9, Aug. 9 [] K. Fall, S. Floyd, Simulation-based Comparison of Tahoe, Reno, and SACK TCP, Computer Communication Review, Vol:, NO:, pp. -, Jul. 99 [7] S. Floyd, T. Henderson, The NewReno Modification to TCP s Fast Recovery Algorithm, RFC, Apr. 999 [] M. Mathis, J. Mahdavi, S. Floyd, S., A. Romanow, TCP Selective Acknowledgement Options, RFC, Apr. 99 [9] Ramakrishnan, K.K., and Floyd, S., A Proposal to add Explicit Congestion Notification (ECN) to IP RFC, Jan. 999 [] Hashem, E., Analysis of random drop for gateway congestion control, Report LCS TR-, Laboratory for Computer Science, MIT, Cambridge, MA, pp., 99 [] UCB/LBNL/VINT Network Simulator - ns (version ) [] A. Feldmann,, Rexford, J., Caceres, R. Efficient policies for carrying Web traffic over flowswitched networks, IEEE/ACM Transactions on Networking, Vol:, NO:, pp. 7-, Dec. 99 [] K.C. Claffy, H.-W. Braun, G.C. Polyzos, A parameterizable methodology for Internet traffic flow profiling Selected Areas in Communications, IEEE Journal on, Vol:, NO:, pp. -9, Oct. 99 [] M. Allman, S. Floyd, C. Partridge, Increasing TCP s Initial Window, RFC, Sep. 99 [] V. N. Padmanabhan, R. H. Katz, TCP Fast Start: A Technique for Speeding Up Web Transfers, Proceedings of the IEEE Globecom 9 Internet Mini-Conference, Sydney, Australia, Nov //$7. (C) IEEE