Oscillation of RED with 2way TCP bulk data traffic

Transcription

1 First Author: Oscillation of RED with way TCP bulk data traffic Thomas Ziegler, Salzburg Polytechnic University; Second Author: Bernhard Hechenleitner, Salzburg Polytechnic University; Date: --8 Document type: Technical Report Status: Keywords: Draft Congestion Control, TCP, RED Gateways Abstract/Introduction This report aims at proofing that our simulation results [Zieg99] can be applied to the real network situation. By simulation we found that RED s queuesize oscillates heavily in scenarios with bi-directional TCP bulk-data traffic. These oscillations happen independently of the setting of RED parameters (maxp, wq, maxth, minth) and scenario-parameters (distribution of propagation delay, number of TCP flows, MTU size and bottleneck bandwidth). Furthermore, [Zieg99] proposes models (verified by simulation) how to set RED s parameters as a function of the scenario-parameters. The measurements in this report show that the simulation-results are indeed valid for the real Internet, resulting in severe headaches regarding the deployment of RED. It is questionable that RED parameters can be set properly in the case of uni-directional TCP flows as this would require knowledge of the flow RTT distribution and the number of flows. In case of bi-directional TCP flows (which is the standard-situation in the Internet) oscillations can not be avoided. These queue-size oscillations cause periods of linkunder-utilization followed by periods of vast packet drops due to buffer-overflow, i.e. sub-optimal network-power. Additionally, due to these oscillations, RED-based preferential drop-mechanisms (WRED, RIO) required for the Differentiated Services Internet provide poor service-discrimination among in-profile and out-of-profile packets. In order to enable comfortable measurements with minimum errors a generic measurement environment has been developed. This software is written in TCL, uses standard UNIX commands for process-generation on remote-hosts and standard tools like ttcp and tcpdump to generate traffic and retrieve results. In section, we describe the configuration of this software to perform one-way delay measurements with a GPS clock. The one-way delay of packets directly corresponds to the queueing-delay at the congested router output-port, i.e. the instantaneous RED queue-size. Generic Measurement Environment (GME) The GME enables flexible and comfortable performance evaluation by measurement in the Internet. The main-program (a TCL script) is running at the GME-Server, controlling the entire measurement-process. This script in turn creates and distributes scripts for flow-generation and traffic monitoring, starts flows and traffic-monitors on remote hosts, and finally gathers the measurement results at the GME-server after the measurement has terminated. For generation of flows the "ttcp" tool is used: first a ttcp server (receiver) has to be started at the receiving host with "ttcp -rs -p x". The option "-rs" tells ttcp to operate as data-receiver tcp and to discard all incoming data; "-p x" specifies the portnum- - -

2 ber to listen at. then the ttcp sender is started with "ttcp -ts -p x receiving-host". The "-ts" option tells ttcp the operate as transmitter and to source arbitrary data into the net to the host running the ttcp receiver with portnumber "x". The ttcp sender is aware of several other TCP-specific options and additionally allows creation of UDP flows. Traffic monitoring is performed with tcpdump. Tcpdump switches a host s ethernet-interface into promiscious mode and dumps header-information of all packets broadcasted at the Ethernet Segment matching a filter condition. For instance, in order to filter all packets of one TCP flow from host A to host B with source port-number 7, the call to tcpdump is as follows: "tcpdump src host A and dst host B and proto TCP and src port 7" For remote procedure calls and file-copying the standard Unix-commands rsh and rcp are used. Note that for proper operation of the rsh and rcp commands ".rhosts files" are required at remote hosts enabling access from the GME-server. The main TCL script running at the GME-server can be outlined as follows:. create traffic-files, send traffic-files to hosts: there are two types of traffic-files: receiver-files: shell scripts starting ttcp receivers and sender-files starting ttcp senders. Among a sender-file on a host A and a receiverfile on a host be exists a one-to-one relation determining the traffic-flows: for each ttcp-receiver process on host B listening on port C, a ttcp sender is created (with adjustable options) sending to host B, port C. In the current version of GME, all ttcp senders in one sender-file start immediately. However, in future versions the individual ttcp sender processes can be scheduled at arbitrary points time (this would be facilitated by running TCL not only at the CME-Server but also at the remote hosts running the trafficfiles). After creation of the sender- and receiver-files for host A and B the sender file is copied to host A; the receiver file is copied to host B with the Unix command "rcp".. create tcpdump-files, send tcpdump files to hosts. Tcpdump files are shell scripts containing one line of code, starting tcpdump with a filter-function (see above).. Start tcpdump files: the tcpdump-files are started with the "rsh" command as background processes on remote hosts.. start ttcp receivers: the ttcp receiver-files are started with the "rsh" command as background processes. Note that it is important to start the ttcp receivers earlier than the ttcp senders as ttcp receivers and senders have a client-server relationship. 5. start ttcp senders: the ttcp sender-files are started with the "rsh" command as background processes. As the remote hosts are not time-synchronized it may happen that two sender-files at two distinct remote hosts are not started at the same point in time. Assume that the delay from the CME-server to a host A is longer than the delay between CME-server and a host B or, assume that some rsh-packets are lost on the way to host A and all rsh.packets to host B arrive. In both cases the sender-file at host B (and thereby the ttcp flows at host B) will be started before the sender file at host A. It is the responsibility of the engineer to estimate whether the connectivity between CME-caller and the remote hosts is sufficient to satisfy the time-accuracy required for a specific measurement. For future versions of the tool an intelligent agent would be desirable, detecting the delayed start of ttcp flows and terminating the measurement in case accuracy requirements are not met. 6. wait for the estimated duration of the measurements 7. gather ttcp results: each ttcp-result contains the measurement results for a single tcp (or udp) flow. All ttcp-result files are copied from the remote hosts to the CME-sever by rcp. - -

3 8. gather tcpdump results. TCPdump result files are copied from the remote hosts to the CME-sever by rcp. Network Configuration host host host hub router router hub host5 host host6 router host7 Figure Measured network Hardware: hosts-6: PC Pentium, 5MHz, 8MB RAM host7: PC Pentium MMX, MHz hub,: Bay Networks / Mbps Ethernet Hub router,: Cisco 6, IOS. T router: Cisco 5, IOS. T Links: hosts -6 to Hubs:Mbps Ethernet host7 to router: Mbps Ethernet router to hub: Mbps Ethernet router to hub: Mbps Ethernet router to hubs: Mbps Ethernet router to router: Serial link running HDLC, bandwidth 5kbps The propagation delay of all links can be assumed zero (all equipment is located in one lab). The configuration is slightly non-symmetric as router is connected to hub with Mbps Ethernet, whereas router is connected to hub with Mbps Ethernet. This asymmetry could not be avoided as only one Mbps Ethernet port has been available for the 6er routers. However, if the bottleneck capacity (router- router) is sufficiently low (e.g. 5kbps for the two-way TCP scenarios), this asymmetry does not influence the measurement results as collisions at the Mbps Ethernet are negligible. In earlier scenarios the bottleneck capacity has been set to Mbps, causing a significant amount of collisions (showing TCP s burstiness) and thereby falsifying the measurement results. GME usage for one-way Delay Measurements in the Testbed In this section we show a sample application of GME; one-way delay measurements in the testbed specified in section. - -

4 The ttcp flows are created (as explained in section ) from host to host, host to host 5, host to host and host 5 to host. Host and host 6 are time-synchronized by a GPS clock. Additionally these hosts are running tcpdump. Two one-way delay measurements are performed: ttcp flow A from host to host is tcpdumped at host and host ttcp flow B from host to host is tcpdumped at host and host Tcpdump stores the time a packet is received and the TCP sequence number in a result-file. By relating the sequence numbers of ttcp flow A in the result-files at host and host and subtracting the arrival time of a TCP segment at host from the arrival time of the TCP segment at host the one-way delay can be measured. This equally valid for ttcp flow B. This configuration minimizes measurement errors as there is no traffic created during the execution of the measurement but the flows to be measured. Additionally, tcp-dumping is completely separated from the the task of traffic-generation as this happens on different hosts. Hence the GME does not at all influence the measured system (falsifications due to Heisenberg s uncertainty principle are minimized). 5 Measurements with two-way TCP traffic Common parameters for all measurements: Bottleneck bandwidth 5kbps; switching delay in routers ms, propagation delay ms. All TCP flows start at the beginning of the measurement Measurement: Traffic: tcp flows host -> host, host -> host5, host5 -> host, host -> host (8 in total) Red parameters: minth, maxth 6, buffersize, wq., maxp / Figure one-way delay in forward and backward direction Figure shows that the one-way delay oscillates heavily between and.5 seconds. The oscillations at the two RED-gateways are phase-shifted by exactly 8 degree. The one way delay is directly related to the instantaneous queue-size at the RED gateway: queue-size - - =

5 bottleneck-bandwidth / one-way delay. As shown by figure the measurement confirms the simulation results: RED oscillates with way long-living TCP traffic, causing undesirable behavior as explained in and [Zieg99]. Measurement: Traffic: 7 tcp flows host -> host, host -> host5, host5 -> host, host -> host (8 in total) Red parameters: minth, maxth 6, buffersize, wq., maxp / Figure one-way delay in forward and backward direction Measurement: Traffic: tcp flows host -> host, host -> host5, host5 -> host, host -> host ( in total) Red parameters: minth, maxth 6, buffersize, wq., maxp / Figure one-way delay in forward and backward direction - 5 -

6 Figure and figure show that the frequency of oscillation is less constant in case of higher per-tcpflow throughput. However, the basic behavior remains (8 degree phase shift, oscillations between zero and seconds - corresponding to queue-oscillations between zero and the total buffersize) In summary, measurements - show that oscillations happen independently of the number of TCP flows. Measurements and 5 repeat measurements and, but with wq set to.8. Measurement: Traffic: tcp flows host -> host, host -> host5, host5 -> host, host -> host (8 in total) Red parameters: minth, maxth 6, buffersize, wq.8, maxp / Figure 5 one-way delay in forward and backward direction Measurement5: Traffic: tcp flows host -> host, host -> host5, host5 -> host, host -> host ( in total) Red parameters: minth, maxth 6, buffersize, wq.8, maxp / - 6 -

7 Figure 6 one-way delay in forward and backward direction As shown in figure 5 and figure 6, changing wq to.8 does not significantly affect the result. Measurement6: Traffic: 7 tcp flows host -> host, host -> host5, host5 -> host, host -> host (8 in total) Red parameters: minth 7, maxth, buffersize 5, wq.8, maxp / Figure 7 one-way delay in forward and backward direction - 7 -

8 Measurement7: Traffic: tcp flows host -> host, host -> host5, host5 -> host, host -> host ( in total) Red parameters: minth 7, maxth, buffersize 5, wq.8, maxp / Figure 8 one-way delay in forward and backward direction As shown in figure 7 and figure 8, the oscillations persist in case of lower minth, maxth and buffersize parameters. Obviously, the maximum delay decreases compared to measurements -5, as the queueing delay is shorter. 6 Conclusions The measurements in this report verify the simulation results in [Zieg99]. Oscillation of the RED queue is indeed a problem in case of way TCP traffic and places severe concerns on the deployment of RED. Future work: proposal of queue-management algorithms avoiding the queue-size oscillations quantitative evaluation of the influence of queuesize oscillations on the service-discrimination among in-profile and out-of-profile packets with WRED or RIO more investigations on RED parameter settings 7 Acknowledgements The authors would like to thank Florian Guma from Salzburg Polytechnic University for his greatful technical support. 8 References [Zieg99] T. Ziegler, S. Fdida, "Stability Criteria for RED with bulk-data TCP traffic", technical report,