Delay-Based False Congestion Detection on Groups of TCP Flows

Transcription

1 Delay-Based False Congestion Detection on Groups of TCP Flows Soohyun Cho and Riccardo Bettati Computer Science Department Texas A&M University College Station, TX USA {sc646, Technical Report TAMU-CS-TR--2-4 February 28, Abstract By opening a number of concurrent parallel TCP flows, a node can achieve high utilization of available link bandwidth in high-bandwidth long-delay network without modification of TCP itself. However, parallel TCP flows are inherently unfair with respect to normal TCP flows. There have been several proposals to improve performance of parallel TCP flows. CM, COCOON, Fractional/Combined TCP, and TCP-P are some examples. In this paper, we first assess fairness of these schemes with extensive experiments. Then, we propose TCP-P +, which augments TCP-P with a delay-based false congestion detection mechanism to improve performance in high bandwidth-delay product networks where non-congestion related packet losses are the limiting factor for TCP performance. The proposed scheme effectively utilizes available bandwidth in random, systemic packet loss situations while maintaining fairness, i.e., without stealing bandwidth from normal TCP flows. We implemented TCP-P + as part of Linux and experimental results show that the proposed scheme indeed achieves high utilization without penalty to single flows. I. INTRODUCTION Recently, evolution in transmission technology and the increasing demand for very-high bandwidth services pose a number of new challenges to TCP. Although TCP evolved during the last decades [],[2] and works well in most situations, the limited achievable throughput over networks with high bandwidth-delay product has become increasingly an issue. TCP s flow control and error control mechanisms have shown to be a limiting factor over such networks: Given TCP s lacks of ability to distinguish non-congestion packet loss from congestion related packet loss, its throughput is inherently inversely proportional to the round trip time. Recently, several proposals to modify the behavior of TCP for high bandwidthdelay product networks have been introduced. These proposals include HS-TCP [3], STCP [4], HTCP [], FAST TCP [6], and BIC-TCP [7]. Differently from these approaches, there is another way to achieve high utilization over high bandwidth-delay product connections, which does not require modification of TCP itself since it works around the limitations of TCP by opening multiple concurrent parallel TCP flows. When an end-host establishes multiple TCP flows simultaneously to the same destination, it can achieve higher throughput than with a single TCP flow because each loss of a packet in a set of parallel TCP flows causes the congestion window of only a single TCP flow to half instead of halving all parallel TCP flows windows [8]. Also, if an end-host opens N parallel TCP flows to the same destination, its congestion window increase and recovery are N times faster than for a single flow. As a result, the achievable throughput of parallel TCP flows given the same packet loss probability is bigger than that of a single TCP flow. GridFtp [] and XFTP [] are examples of the use of parallel flows at application level to achieve high throughput. PSockets [] is a library that supports applications to use parallel connections. However, opening multiple parallel TCP flows is unfair in terms of throughput with respect to hosts who open a single TCP flow when they compete for the same bottleneck link. There have been several efforts to achieve high performance with parallel TCP flows while constraining the unfairness of parallel TCP flows. The Congestion Management (CM) architecture [2], COCOON [3], Fractional/Combined TCP flows [4], and TCP-P [] are some examples. However, we show through extensive experiments that improving performance of parallel TCP flows while maintaining fairness (or TCP-friendliness) to single TCP flows is not easy to achieve, especially in high-speed long-delay network. Over high bandwidth-delay product links, TCP requires the packet loss probability to be very small in order to achieve high utilization of the links. However, at this level non-congestion related errors caused by system or transmission link errors can dominate packet losses [3]. This becomes problematic for a single TCP flow to effectively utilize available bandwidth, especially over high-speed long-distance paths, because of the long time required to recover congestion window and, hence, the sending rate. From this observation, we suggest a new fair parallel TCP scheme called TCP-P +, which augments TCP- P with a delay-based false congestion detection mechanism in order to achieve high utilization of available bandwidth in noncongestion related packet loss dominant situation. Meanwhile TCP-P + regulates the aggressiveness of parallel TCP flows to be comparable to that of a normal single TCP flow for fair sharing of bandwidth. There are a number of ways to group of TCP flows. In this

2 paper, we group active TCP flows with the same source and destination pairs and call this a group of parallel TCP flows. The remainder of this paper is organized as follows: Section II shows fairness evaluation methodologies and experimental results for most of the parallel TCP schemes in the literature. Section III proposes TCP-P+, and Section IV shows that TCP-P+ indeed effectively utilizes available bandwidth while maintaining fairness comparable to that of a single TCP flow. Section V concludes this paper. II. FAIRNESS IN EXISTING PARALLEL TCP SCHEMES In this section, we describe the mechanisms of the existing parallel TCP schemes and assess their fairness (or unfairness ) through extensive experiments. By the term fairness (unfairness) we mean how fairly (unfairly) a group of parallel TCP flows from a node share network resources - such as bandwidth - with TCP flows from other nodes. The parallel TCP schemes we evaluate here include unmodified TCP, Fraction/Combined TCP, COCOON, CM, and TCP-P. Although not all the parallel TCP schemes are specifically designed for fairness with TCP flows from other nodes, we believe that our experiments can reveal important aspects of parallel TCP schemes. A. Experiment Setup To measure how fair or unfair a parallel TCP scheme is against competing TCP flows, we use the test-bed shown in Fig.. In this test-bed, we use NIST Net Emulator [6] to emulate delays and packet losses in the Internet. The NIST Net Emulator is implemented on a Linux machine and emulates the Internet by appropriately delaying and dropping packets if it is configured to do so. All the end-host nodes and the NIST Net Emulator are running on Linux PCs. The PCs we use for experiments are Pentium 4 or 3 machines with /Mbps Fast Ethernet network interface cards. Each Fast Ethernet card has an output queue of length packets by default. Because the network links we experiment with are fairly high-bandwidth (Mbps) and the NIST Net delay parameters are fairly large (msec round-trip time) we set the TCP parameters of the Linux end systems - such as tcp wmem and tcp rmem - according to the recommendations in [7], rather than using system defaults. In a sender Node, we installed kernel version of Redhat. and modified Linux kernels for the parallel TCP schemes. The other sender Node also runs on kernel except when we test CM. We use Redhat 7. with kernel in both Node and Node for CM. For COCOON we use kernel in Node according to the scheme s kernel patch. NIST Net Emulator and the TCP sink, Node 2, are running on Redhat Linux 7.2 with kernel For traffic generation and throughput measurement we use iperf Version.7. [8], which supports parallel TCP flows and offers great flexibility for measurements. By default, Linux TCP behavior is based on TCP-Sack [2]. In all experiments in this paper, every experiment was done for sec to get an average value and repeated times with sec waiting u ŒG W j ššg{ ˆ ŠG z œ ŠŒ X X u u Œ X UX XWWt šsgwuz š X`YUX]_UYUWVY[ UY\ [ UY\ [ XWWt šsgwuz š X`YUX]_UZUWVY [ UX wˆ ˆ Œ {jwgš œ ŠŒš Fig.. \W š UY\[ UX XWWt šsgwuz š X`YUX]_UXUWVY[ Experiment Network wˆ ˆ Œ G{jw X u Œ Y z š X u j ššg{ ˆ ŠG time after each experiment unless told otherwise. Error bars at figures of this paper represent % Confidence Interval of the data. To evaluate fairness (or TCP-friendliness) property of the parallel TCP schemes, Node opens from zero to parallel TCP flows of each scheme to the destination Node 2 while another sender Node opens one unmodified TCP connection to the same destination for the same time. When the number of parallel TCP flows from Node is zero, Node does not open any TCP flow to Node 2, so that only the unmodified single TCP flow from Node consumes all the network resources without competition. A single unmodified TCP flow of kernel 2.4. alone in our test-bed achieves about 8Mbps in second average. By increasing the number of parallel TCP flows from Node, we measure how much bandwidth Node gains at the cost of Node. In this experiment we do not introduce any random packet losses in the test-bed to study fairness in response to congestion only. B. Experiment Results We first show that how much bandwidth unmodified parallel TCP flows can steal from a competing single TCP flow. Fig. 2(a) shows the experiment results with an increasing number of unmodified parallel TCP flows from Node. From the figure we can clearly see that how unfairly a node can achieve more bandwidth by opening more connections to the same destination. By opening more TCP connections to a destination, the parallel TCP flows achieve higher throughput by stealing bandwidth from the competing TCP flow of Node. A steady-state throughput ratio model of N parallel TCP flows and competing single TCP flow has been derived and interested readers can refer to []. Fractional and Combined TCP schemes [4] were proposed to mitigate unfairness of unmodified parallel TCP flows. Fractional TCP scheme controls increase speed of congestion window sizes of parallel TCP flows. In slow-start mode, a Fractional TCP flow j in a group of N Fractional TCP flows increases its congestion window W j by one after it received N acknowledgement packets (ACKs). In congestion-avoidance mode, the flow increases its congestion window W j by one after it received NW j ACKs. Combined TCP scheme is suggested to include an unmodified TCP to a group of Fractional TCP flows to compensate the conservativeness of Fractional TCP flows. Combined parallel TCP flows with group size N include one unmodified TCP and N Fractional TCP flows. z

3 8 7 3 Total Parallel TCP Total Single TCP Total Frac Total Comb Parallel Frac TCP Total Parallel Comb TCP Total Single TCP Frac Single TCP Comb Total Parallel COCOON TCP Total Single TCP (a) Unmodified TCP (b) Fractional/Combined TCP (c) COCOON TCP Total Parallel CM TCP Total Single TCP Total Parallel TCP P Total Single TCP Total Parallel TCP P+ Total Single TCP (d) CM TCP (e) TCP-P (f) TCP-P+ Fig. 2. Throughput of TCP Flows in Different Schemes and Single Unmodified TCP Flow We implemented Fractional and Combined TCP in kernel 2.4. according to [4]. Fig. 2(b) shows that Fractional and Combined TCP flows are less aggressive than unmodified TCP flows, while Combined TCP flows show higher throughput than Fractional TCP, at the cost of more throughput decrease in the competing TCP flow. As we can see from the figures, although the unfairness is reduced compared to that of unmodified parallel TCP flows, this scheme still steal significant amounts of bandwidth from the competing TCP flow. COCOON [3] employs a coordinated congestion control to a group of TCP flows. Specifically, when a flow in the group experiences congestion, it adjusts the congestion windows of member TCP flows whose outstanding packet sizes are larger than the congestion window size of the TCP flow. It also uses a watermark method to avoid multiple reductions of congestion windows of the TCP flows in one round-trip time. However, Fig. 2(c) shows that COCOON becomes unfair when it opens more than one TCP connection to the destination, since it is stealing bandwidth from the competing TCP flow. Furthermore, the figure also shows that parallel COCOON TCP flows do not effectively utilize this bandwidth. Congestion Management (CM) architecture [2] proposes an integrated congestion control and loss recovery architecture for parallel flows. A group of parallel TCP flows in the CM architecture uses a single congestion window for all the member flows of the group. This scheme adjusts the total amount of unacknowledged data that the group of flows can have. Since CM is available on Linux 2.2.8, we run this kernel on both Node and Node. For and CM kernels, we increased the socket buffer size of TCP to 2MB because it could not achieve throughputs of more than Mbps in our test-bed with the default 64KB socket buffer size. Fig. 2(d) shows the experiment results with parallel CM TCP flows. The figure shows that the aggregated throughput of parallel CM TCP flows from Node and throughput of the competing TCP flow from Node are not much changed even if Node opens CM TCP flows, so that CM can be considered to be a fair scheme. Although it is not shown in this paper, our experiments indicate that unmodified parallel TCP flows when the both sender nodes use kernel, were very similar to Fig. 2(a) that was with unmodified parallel TCP flows from 2.4. kernels. TCP-P [] was proposed to realize controllable aggressiveness of a group of parallel TCP flows with runtime adjustable strength parameter k, while allowing each flow to maintain its own congestion window. The strength k is a scalar value that describes how big the group (in number of flows) is perceived by other TCP flows. The goal of TCP-P is to make a group of N TCP-P flows appear to other TCP flows like k separate TCP flows. We say that such a group of flows has the aggressiveness of k TCP flows. A group of parallel TCP-P flows realizes this controllable aggressiveness through appropriate manipulation The CM kernel implementation from the authors web-site had some bugs that cause TCP hang. Also, we disabled flood of kernel debugging messages, which significantly slowed down system speed.

4 8 Aggregate Throughput of Parallel TCP Flows Aggregate Throughput of Parallel Fractional TCP Flows Aggregate Throughput of Parallel COCOON TCP Flows (a) Unmodified TCP (b) Fractional TCP (c) COCOON TCP Fig. 3. Throughput of TCP Flows (Top: Node, Bottom: Node ) in Random Loss Environment of the congestion window increase and decrease behaviors of the parallel TCP-P flows in the group. In slow-start mode, a group of N TCP-P flows with strength k increases the congestion window of each TCP-P flow, W j, by k N per non-duplicate ACK as shown in the following equation: W j (t+) = W j (t) + k N. () In congestion avoidance mode, each TCP-P flow in a parallel TCP-P group of size N increases its congestion window by one after receiving the following amount of non-duplicate ACKs: N i= ( N j= W j + i). (2) k N When a congestion event, such as three duplicate ACKs, is detected at time t, TCP-P with strength k and group size N adjusts the total amount of congestion windows of the group according to the following equation: N N W j (t+) = W j (t) ( 2k ). (3) 2k j= j= One possible way to distribute the adjustment amount to the member flows is to decrease every member flow s congestion window by the same proportion. We use this method to TCP- P and to its extension TCP-P + of this paper. Furthermore, to avoid unnecessary reduction of congestion windows in bursty packet losses, TCP-P flows skip adjusting congestion windows if each flow s elapsed time after adjustment by other member TCP-P flows is less than the minimum of its smoothed (i.e., weighted moving average) round-trip time. As a result, the increase and decrease behavior of a group of N TCP-P flows closely reflect that of k TCP flows. Also note that when there is only one TCP-P flow in the group, the single TCP-P flow with strength k becomes similar to a MulTCP [] flow with multiplicity of k. MulTCP is proposed to achieve throughput of multiple TCP flows using one TCP flow by modifying increases and decreases amount of its congestion window size as if it consisted of multiple TCP flows. However, in this paper we only consider TCP-P and its modification TCP-P + for strength k =, so that the total aggressiveness of a group of parallel TCP-P or TCP-P + flows should be comparable to that of a single unmodified TCP flow. Fig. 2(e) shows experiment results with TCP-P. From the figure we can see that a group of parallel TCP-P flows with strength k = has similar fairness property as CM. As discussed in Sec. I, the unfairness of parallel TCP flows stems from its behavior during both congestion window increase and decrease. In contrast the Fractional TCP, which controls only increase behavior, and COCOON, which controls only decrease behavior, TCP-P controls both of them. As a result it controls the aggressiveness of parallel TCP flows more accurately. Furthermore, in contrast to CM, which uses only one congestion window for all the parallel TCP flows in the group, TCP-P allows each TCP-P flow to maintain its own congestion window. III. TCP-P + Groups of parallel TCP flows can measure network status more accurately than a single TCP flow because the former have more feedback loops than the latter. Thus, by leveraging the benefit of individual congestion windows in TCP-P, we propose TCP-P +, which improves performance of TCP-P in utilization of available bandwidth in non-congestion-related packet loss dominant network. TCP-P + does not deteriorate the fairness property of TCP-P. TCP-P + augments TCP-P with a delay-based false congestion detection mechanism to distinguish non-congestion related packet losses from congestion related packet losses. As discussed in Sec. I, non-congestion related packet losses caused by system or transmission link errors become a limiting factor for TCP performance, especially over high-speed longdistance paths. TCP-P + achieves higher utilizations of network resources in these environment by overcoming the blindness of TCP about non-congestion related packet losses. It does so by using group-wide network measurement information and a false congestion detection mechanism.

5 8 Aggregate Throughput of Parallel CM TCP Flows Aggregate Throughput of Parallel TCP P Flows Aggregate Throughput of Parallel TCP P+ flows (a) CM TCP (b) TCP-P (c) TCP-P+ Fig. 4. Throughput of TCP Flows (Top: Node, Bottom: Node ) in Random Loss Environment For the detection of false congestion events, we use intuition and observation [] that it is unlikely that congestion events, such as duplicated ACKs, happen without increased roundtrip time. Therefore, in contrast to TCP-P, which adjusts congestion windows of member TCP flows when a TCP-P flow in the group detects a congestion event, TCP-P + does not adjust congestion windows of member TCP flows if it decides the congestion event is false (i.e., it is due to an systemic error, and not to congestion.) To decide whether a congestion event is false or not, all TCP-P + flows in the group monitor their round-trip time changes and use a thresholdbased approach: Each time a TCP-P + encounters a congestion event, it compares its current smoothed round-trip time (srtt) with the following threshold level T: T = gsrtt min + γ (gsrtt max gsrtt min ). (4) Here, gsrtt min and gsrtt max stand for the maximum smoothed round-trip time and minimum smoothed round-trip time, respectively, experienced by all member TCP flows in the group. The factor γ is a design parameter of the TCP-P+ scheme, and in this paper is set to 3. The smoothed round-trip time srtt is readily available in standard TCP implementations and it is updated whenever a new round-trip time rtt is available using the following equation: srtt(t+) = ( w) srtt(t) + w rtt(t). () A weight parameter w is used to assign a weight to the new round-trip time measurement. For example, in Linux it is set to 8. After updating its own srtt, each member TCP-P+ flow in a parallel TCP group updates gsrtt min and gsrtt max of the group if its srtt is less than gsrtt min or greater than gsrtt max. In this way, a group of parallel TCP-P + flows obtains more accurate measurement information about network status than individual TCP flows could. The delay-based method used in TCP-P + is similar to that used in TCP-Nice [2], however, which uses the method to decide impending congestion instead of false congestion. Also, TCP-P + is based on the measurement data gathered by entire flows in the group rather than by an individual flow. We implemented TCP-P + in Linux kernel by augmenting TCP-P to (a) manage those group-wide measurement data, gsrtt max and gsrtt min, and (b) not to reduce congestion windows of member TCP flows to false congestion events using Eq. 4. However, note that both TCP-P and TCP-P + flows become normal (unmodified) single TCP flows when there is only one flow in the group and the group s strength k =. IV. EVALUATION A. Evaluation with TCP Cross Traffic First, we repeat the experiments of Section II-B to show that TCP-P + indeed maintains the fairness property of TCP-P. Fig. 2(f) shows experiment results with various numbers of parallel TCP-P + flows. The results indicate that parallel TCP- P + flows maintain fairness to the competing TCP flow just as TCP-P in Fig. 2(e) and CM in Fig. 2(d) do. Next, we test how well parallel TCP schemes utilize available bandwidth, and how fair or unfair they are in noncongestion related packet loss dominant situations. To this end, we evaluate the performance of TCP-P + and other parallel TCP schemes by introducing random packet drops in the NIST Net test-bed depicted in Fig.. In these experiments we introduce random drop probabilities at a rate of.%,.3%,.%,.7%, and.%. The achievable bandwidth by TCP was significantly reduced when random packet losses were introduced. For example, when we configure NIST Net to drop.% of packets going to the destination Node 2, the achievable throughput for a single TCP flow alone drops to Mbps, despite the plentifully available bandwidth. This value is close to the theoretical average throughput of a TCP flow in [22] if we consider delayed-ack of Linux TCP and a maximum segment size of bytes. In Fig. 3 and Fig. 4, we show the achieved throughput by Node (Top) and that of Node (Bottom). Node opens various numbers of parallel TCP flows (i.e.,,,, and flows) to the destination Node 2 for seconds, while Node opens a single unmodified TCP flow to Node 2 for the same time. In these figures, the lines labeled as in the bottom

6 Aggregate Throughput of Parallel CM TCP Flows Aggregate Throughput of Parallel TCP P Flows Aggregate Throughput of Parallel TCP P+ Flows (a) CM TCP (b) TCP-P (c) TCP-P+ Fig.. Aggregate Throughput of Parallel TCP flows in Random Loss Environment with No Cross Traffic figures are for the case when there is no flow from Node. This illustrates the bandwidth utilization ability of a single unmodified TCP flow from Node with the given packet loss probability. The lines labeled as,, and illustrate the achieved throughput by each sender node when there are,, and flows from Node, respectively. Fig. 3(a) shows that unmodified parallel TCP flows can realize almost full utilization of the Mbps link by increasing the number of parallel TCP flows even if there are severe random packet drops. However, again, Node s throughput drops when Node opens more parallel TCP flows. Fig. 3(b) and Fig. 3(c) show experiment results when Node opens Fractional TCP and COCOON TCP flows, respectively. The figures show that both Fractional and COCOON TCP achieve less throughput than unmodified parallel TCP flows with the same number of parallel flows and for a given loss probability. Further, the figures also show how the single TCP flow s throughput drops when Node opens more flows. Although it is not shown in this section (see Fig. (a) in the Appendix), the experiment results with Combined TCP were similar to that of Fractional TCP. Fig. 4(a) shows experiment results with CM TCP flows in the random packet loss situation. The results show that the competing single TCP flow does not suffer by the increased number of CM TCP flows. However, the results also show that CM TCP is unable to utilize the available bandwidth by increasing the number of CM TCP flows. These results are expected because CM architecture employs only one congestion window for all the flows in the group. Fig. 4(b) shows the experiment results with TCP-P in the random packet loss situation. The figure shows that parallel TCP-P scheme has similar performance to CM in the utilization of network, and fairness to the competing TCP flow. Fig. 4(c) shows experiment results with TCP-P +. The figure clearly shows that TCP-P + is better than both CM and TCP- P in the utilization of available bandwidth with the same number of flows and the given loss probability. Further, the result also shows that parallel TCP-P + flows maintain fairness comparable to those of TCP-P and CM. The throughput of the competing TCP flow is almost not affected by the increased number of TCP-P + flows. From the figures of this section we can clearly see the benefit of TCP-P +, which utilizes available bandwidth effectively by increasing the number of parallel TCP flows, while maintaining its fairness against the competing TCP flow in non-congestion related packet loss dominant network. B. Evaluation with No Cross Traffic For more evaluation on the performance of TCP-P + in the utilization of available bandwidth, we repeat the previous experiment without cross traffic, i.e., Node does not participate in experiments. By doing this, we measure how well parallel TCP schemes utilize available bandwidth by increasing the number of parallel flows in random loss environment. In this section, we only consider and evaluate CM, TCP-P, and TCP- P + because only these schemes proved to support fairness in the previous experiments. Fig. shows the experiment results with parallel TCP schemes when there is no cross traffic. All experiments are repeated times and points in the figures represent average values of the results. The labels in the figures are the number of parallel TCP flows opened by Node. Fig. (a) shows experiment results with CM TCP. It shows that the increased number of CM parallel flows does not result in higher throughput. (We ignore the minor throughput increase for the case of % packet loss because single unmodified TCP from kernel could achieve that throughput. This is shown with a graph labeled as in Fig. 4(a).) Fig. (b) shows that TCP-P can achieve slightly higher throughput by opening multiple, parallel flows. Fig. (c) shows the experiment result with TCP-P +. The figure clearly shows the effectiveness of the false congestion detection mechanism and group-wide network measurements of TCP-P + flows in random loss dominant network, since the scheme achieves high utilization of available bandwidth by opening multiple, parallel TCP-P + flows. C. Single TCP with False Congestion Detection In order to compare the effectiveness TCP-P + against single-flow based schemes, we need to separately evaluate the

7 Throughput of a TCP D Flow 7 Sinusoidal Wave Square Wave Fig. 6. Throughput of a TCP-D Flow in Random Loss environment with No Cross Traffic UDP Sending Rate [Mbps] Time [sec] (a) Period: sec, Max: Mbps, Min: 2Mbps Fig. 7. UDP Sending Rate [Mbps] Time [sec] (b) Period: sec, High: Mbps, Low: Mbps Sinusoidal and Square-waved UDP Cross Traffic effectiveness of of TCP-P + s delay-based congestion detection scheme and TCP-P + s use of parallel connections. For the forme, we define delay-based, single-flow based scheme which we call TCP-D, which uses only its own round-trip time information to detect false congestion events. For the decision of false congestion events, a TCP-D flow uses srtt max and srtt min, instead of gsrtt max and gsrtt min in Eq. 4. The values of srtt max and srtt min represents the maximum and minimum smoothed round-trip time experienced by the TCP-D flow during its connection. In Fig. 6 we show the experiment results with a TCP-D flow from Node in a non-congestion related packet loss environment. The figure shows that a single TCP-D flow does not effectively utilize available bandwidth although it uses almost the same false congestion detection mechanism as TCP-P +. Compared to the throughput of single TCP flow case shown in Fig. (b) and (c), the achieved throughput of TCP- D remains unchanged. This is mainly because TCP-D is not able to accurately measure network status, such as bottleneck buffer sizes, through its srtt max and srtt min because of the random losses that occurs in the test-bed network. D. Evaluation with UDP Cross Traffic In order to evaluate the responsiveness of the various TCP schemes to changing network bandwidth availability, we perform a suite of measurements with UDP cross traffic. We test TCP-P + and other parallel TCP schemes with two kinds of UDP traffic, sinusoidal and square-waved. By introducing sinusoidal or square-waved UDP traffic, we test how well different parallel TCP schemes from Node compete with non-responsive, time-varying cross traffic from Node. We generate this UDP traffic from Node by modifying the iperf source codes. Fig. 7 shows examples of sinusoidal (Fig. 7(a)) and square-waved (Fig. 7(b)) traffic with sec period generated from Node for a duration of 3 sec. The sinusoidal UDP traffic has maximum sending rate of Mbps, and minimum 2Mbps, so that the average sending rate of the sinusoidal UDP traffic is 36Mbps. The square- waved UDP traffic shown in the figure has sending rate of maximum Mbps in High state and minimum Mpbs in Low state. The periods of the sinusoidal and square-waved UDP used in experiments include. sec,. sec, sec, sec, sec, and 3 sec. These UDP cross traffic from Node result in serious congestion in the bottleneck link of the test-bed network when its period is small. All experiments are repeated times to get an average value. ) Sinusoidal UDP Cross Traffic: Fig. 8 shows the average of the aggregate throughput of various parallel TCP schemes when Node increases the number of parallel TCP flows from one to and while Node is sending non-responsive sinusoidal UDP traffic with varying periods. Fig. 8(a) shows the results with unmodified parallel TCP flows. The figure shows that, even with non-responsive UDP cross traffic, if a node opens multiple unmodified TCP flows, it can achieve higher throughput. Fig. 8(b) shows that Combined parallel TCP scheme can achieve higher throughput when it competes with unresponsive UDP flows although it achieves less throughput than that of unmodified TCP flows. The experiment results with Fractional TCP is shown in Fig. (b) of the Appendix. Fractional TCP shows similar results to Combined TCP but with slightly less throughput. Fig. 8(c) shows that a group of parallel COCOON TCP flows achieves less throughput when it opens more TCP flows, except when the periods are and sec. Fig. 8(d) shows the achieved aggregate throughput of parallel CM TCP flows with the sinusoidal UDP cross traffic. The figure shows that the achieved throughput by multiple, parallel CM TCP flows are not much changed regardless of the number of CM TCP flows. Fig. 8(e) shows the experiment results with varying numbers of parallel TCP-P flows from Node. The figure shows that parallel TCP-P flows with different group size maintain their aggregated throughput comparable to that of a single unmodified TCP flow. Fig. 8(f) shows that the achieved aggregated throughput of parallel TCP-P + flows is not much different from that of parallel TCP-P flows with the same number of parallel flows, given the period of the sinusoidal UDP cross traffic. 2) Square-Waved UDP Cross Traffic: Fig. shows the experiment results of unmodified TCP, Fractional TCP, CO- COON, CM, TCP-P, and TCP-P + with non-responsive square-

8 Aggregate Throughput of TCP Flows Aggregate Throughput of Combined TCP Flows Aggregate Throughput of COCOON TCP Flows (a) TCP (b) Combined TCP (c) COCOON TCP Aggregate Throughput of CM TCP Flows Aggregate Throughput of TCP P Flows Aggregate Throughput of TCP P+ Flows (d) CM TCP (e) TCP-P (f) TCP-P+ Fig. 8. Aggregate Throughput of Parallel TCP Flows with Sinusoidal Waved UDP Cross Traffic waved UDP cross traffic of varying periods. Because the square-waved UDP cross traffic generates more serous congestion than the previous sinusoidal cross traffic (i.e., Mbps UDP traffic from Node when it is in High state as shown in Fig. 7(b)), all the graphs in the figures show less throughput than those in Fig. 8. The figures for the unmodified TCP (Fig. (a)) and Fractional TCP (Fig. Fig. (b)) show that these parallel TCP schemes can increase throughput by increasing the number of parallel flows against the square-waved UDP cross traffic. Although the experiment results with Combined TCP were not shown in this section (see Fig. (c) in the Appendix), Combined TCP showed similar results to Fractional TCP. Fig. (c) shows the experiment results with COCOON. The figure shows that COCOON achieves higher throughput by increasing the number of parallel COCOON TCP flows when the periods of the square-waved UDP cross traffic are longer than one second. The experiment results of CM (Fig. (d)), TCP-P (Fig. (e)), and TCP-P + (Fig. (f)) show that the achieved aggregate throughput of these parallel TCP schemes are almost not affected by the group size of the parallel flows, given the periods of the square-waved UDP cross traffic. The experiment results of these schemes suggest that the three schemes, indeed, maintain the performance of a single unmodified TCP flow even with non-responsive UDP cross traffic. In this section, using the experiment results with nonresponsive, time-varying sinusoidal and square-waved UDP cross traffic, we showed that the performance of parallel TCP- P + flows remain almost unchanged from that of an unmodified single TCP flow, even in seriously congested network situations caused by non-responsive cross traffic. V. CONCLUSION In this paper, through extensive experiments on various parallel TCP schemes, we first showed that achieving high throughput while maintaining fairness to competing TCP flows is not an easy task. Then, we proposed TCP-P +, which augments TCP-P with a false congestion detection method based on delay variations measured group-wide. We showed that TCP-P + can realize higher throughput than TCP-P or CM in random packet loss dominant environment by avoiding response to the detected false congestion events while maintaining fairness property of TCP-P. This is useful characteristic specially in high-bandwidth long-delay network because noncongestion related, systemic packet losses becomes a limiting factor of the performance of traditional TCP. The strong point of TCP-P + is that it can achieve higher throughput by opening more parallel flows when there is available bandwidth while not stealing bandwidth from competing normal TCP flows. REFERENCES [] M. Mathis, J. Mahdavi, S. Floyd, and A. Romanow, TCP Selective Acknowledgement Options, RFC-8, October 6.

9 3 Aggregate Throughput of TCP Flows 3 Aggregate Throughput of Fractional TCP Flows 3 Aggregate Throughput of COCOON TCP Flows (a) TCP Aggregate Throughput of CM TCP Flows (b) Fractional TCP Aggregate Throughput of TCP P Flows (c) COCOON TCP Aggregate Throughput of TCP P+ Flows (d) CM TCP.. 3 (e) TCP-P.. 3 (f) TCP-P+ Fig.. Aggregate Throughput of Parallel TCP Flows with Square Waved UDP Cross Traffic [2] M. Allman, V. Paxson, and W. Stevens, TCP Congestion Control, RFC-28, April. [3] S. Floyd, HighSpeed TCP for Large Congestion Windows, RFC 364, December 3. [4] Tom Kelly, Scalable TCP: Improving Performance in Highspeed Wide Area Networks, in Workshop on Protocols for Fast Long-Distance Networks, Februrary 3. [] D.J. Leith and R. Shorten, H-TCP Protocol for High-Speed Long Distance Networks, in Workshop on Protocols for Fast Long-Distance Networks, Februrary 3. [6] Cheng Jin, David X. Wei, and Steven H. Low, FAST TCP: motivation, architecture, algorithms, performance, in Infocom, March 4. [7] Lisong Xu, Khaled Harfoush, and Injong Rhee, Binary Increase Congestion Control for Fast Long-Distance Network, in Infocom, March 4. [8] S. Floyd and K. Fall, Promoting the Use of End-to-End Congestion Control in the Internet, IEEE/ACM Transactions on Networking, August. [] Jason Lee, Dan Gunter, Brian Tierney,Bill Allcock, Joe Bester, John Bresnahan, and Steve Tuecke, Applied Techniques for High Bandwidth Data Transfers across Wide Area Networks, in International Conference on Computing in High Energy and Nuclear Physics, September. [] Mark Allman, Hans Kruse, and Shawn Ostermann, An Application- Level Solution to TCP s Satellite Inefficiencies, in The First International Workshop on Satellite-based Information Services (WOSBIS), November 6. [] H. Sivakumar, S. Bailey, and R. L. Grossman, PSockets: The Case for Application-level Network Striping for Data Intensive Applications using High Speed Wide Area Networks, in IEEE/ACM SC Conference, November. [2] H. Balakrishnan, H. Rahul, and S. Seshan, An Integrated Congestion Management Architecture for Internet Hosts, in SIGCOMM, September. [3] Y. Gao, G. He, C. Hou, and S. Paul, COCOON: an alternate approach to end-host congestion management, submitted to IEEE Trans. on Computers, April 2, URL: stat.bell-labs.com/who/ yuangao/papers/cocoon.pdf. [4] Thomas Hacker, Brian Noble, and Brian Athey, Improving Throughput and Maintaining Fairness using Parallel TCP, in Infocom, March 4. [] Soohyun Cho and Riccardo Bettati, Aggregate Control of Parallel TCP flows, Technical Report TAMU-CS-TR-4--, November 4. [6] M. Carson and D. Santay, Tools: NIST Net: a Linux-based network emulation tool, ACM CCR, July 3. [7] M. Mathis, J. Heffner, and R. Reddy, Web: Extended TCP Instrumentation for Research, ACM CCR, July 3. [8] Ajay Tirumala, Feng Qin, Jon Dugan, Jim Ferguson, and Kevin Gibbs, Iperf Version.7., March 3, URL: net/projects/iperf/. [] J. Crowcroft and P. Oechslin, Differentiated end-to-end Internet services using a weighted proportionally fair sharing TCP, ACM CCR, July 8. [] S. Moon, J. Kurose, P. Skelly and D. Towsley, Correlation of Packet Delay and Loss in the Internet, UMass CMPSCI Technical Report 8-, January. [2] Arun Venkataramani, Ravi Kokku, and Mike Dahlin, System Support for Background Replication, in OSDI, December 2. [22] M. Mathis, J. Semke, J. Mahdavi, and T. Ott, The Macroscopic Bahavior of the TCP Congestion Avoidance Algorithm, ACM CCR, July 7. APPENDIX In the appendix, we show the experiment results not shown in the related sections. Fig. includes Combined TCP s performance in random packet loss situations (Fig. (a)), Fractional TCP s aggregate throughput with sinusoidal UDP cross traffic (Fig. (b)), and Combined TCP s aggregate throughput with square-waved UDP cross traffic (Fig. (c)).

10 8 Aggregate Throughput of Parallel Combined TCP Flows Throughput of the Single TCP Flow (a) Combined TCP 3 Aggregate Throughput of Fractional TCP Flows.. 3 (b) Fractional TCP Aggregate Throughput of Combined TCP Flows.. 3 (c) Combined TCP Fig.. (a) Combined TCP in random loss (b) Fractional TCP with Sinusoidal UDP Cross Traffic (c) Combined TCP with Square-Waved Cross Traffic