Non-Cooperative End-to-End Path Characterisation

Transcription

1 Non-Cooperative End-to-End Path Characterisation Kapil Bajaj, Balaji Kasal, Harshal Ninawe, Sriharsha and D. Manjunath Indian Institute of Technology, Bombay Powai Mumbai INDIA 1 Abstract We describe algorithms to obtain one way bandwidth and loss characteristics of Internet paths in the forward and reverse direction from a measuring node to target Internet host. A non cooperative framework is assumed where the target host is not aware of the measurement process. Bandwidth measurements use the well known packet-train method and loss measurements are performed on TCP type or rate controlled flows. These techniques are used in the tools ipathmeter-b and ipathmeter-l that is now being distributed. We also describe some of our early experiences in using these tools. Keywords Non Cooperative Measurement, Path Characterisation, Loss Characterisation, Bandwidth Characterisation, Packet Pair. I. Introduction We consider the problem of estimating one way bandwidth and loss characteristics of Internet path in a non-cooperative setting. To estimate bandwidth we use the packet-pair technique which was originally described [4] for rate controlled networks. [2] uses the packet-pair technique in bprobe to estimate the bottleneck bandwidth and cprobe to estimate the available bandwidth. Though these tools work in a noncooperative environment, they assume symmetric network behavior, which is rarely the case. [3] discusses pathrate, a cooperative bandwidth measurement tool. These tools measure bandwidth in only one direction. [6] describes a passive bottleneck bandwidth measurement tool Nettimer. Here it is necessary to filter out bad samples which gives inconsistent bandwidth estimation. ICMP based loss measurements may not reflect loss characteristics of the paths because many nodes rate limit ICMP responses. Further, it is important to characterise paths as they would be seen by applications, much of which have TCP/UDP for their trans- This work was funded by a grant from the Department of Science and Technology. Balaji Kasal was supported by this grant. Kapil Bajaj and Harshal Ninawe are with Dept of Comp. Sc & Engg. Balaji Kasal, Sriharsha and D. Manjunath are with Dept of Elecl Engg port protocol. In this paper we also address the problem of one way loss characterisation [1], [5] of Internet paths for TCP/UDP type and rate controlled traffic. The rest of the paper is organised as follows: In Section II we describe bandwidth measurement using packet-pair and packet-train techniques and its implementation in ipathmeter-b. In Section III we describe the measurement technique for packet loss statistics and its implementation in ipathmeter-l. Section IV describes some sample results. In Section V we discuss limitations of ipathmeter-b and ipathmeter-l and future work. II. Bandwidth Measurement The principle behind the packet-pair technique is that if two large back-to-back packets (assuming no time delay between them) traverse the bottleneck link, they will be queued and their time dispersion will increase according to the relation (refer Fig. 1), t d sec = P bytes B bl bytes/sec (1) where t d is the time dispersion after the passing bottleneck link, P is the size of the packet and B bl is Packet-1 Packet-2 td Bottleneck Link Packet-1 Packet-2 Time Fig. 1. Timing diagram for bottleneck bandwidth measurement using packet pair. Two back-to-back packets from the source get dispersed at the bottleneck link because of queuing. This dispersion is maintained up to the target host. td

2 2 the bottleneck link bandwidth. It is assumed that this dispersion is maintained as they traversed higher bandwidth links further along the path. Thus the dispersion measured at the destination will be t d and Eqn. 1 can be used. First we describe the packet-pair technique for the forward (from source to target) path. We send two back-to-back TCP SYN packets with large payload to increase the probability of the packets getting queued at the bottleneck link. The destination port number of these packets is some unlikely value, assumed to be not in use at the destination. Target host will respond with TCP RST packets which, being small packets, is assumed to traverse the path without being queued. This implies that the time dispersion between the received RST packets is the same as that of the time dispersion of the SYN packets at the bottleneck link. This dispersion can be used in the Eqn. 1. This packet-pair technique can be extended to a packettrain of length N (N > 2) packets. Using the experimental setup shown in Fig. 2 we sent eight back-toback SYN packets and received the eight RSTs. The resulting trace is shown in Fig. 3. The dispersion between first and last packet of the train will account for competing traffic and we can estimate available bandwidth along the path using relation [2] T sec = (N 1)P bytes B a bytes/sec (2) where T is the time dispersion between first and last packet measured at the source host, N is train length, P is the size of packet and B a is available bandwidth. To measure the bottleneck bandwidth in the reverse path (from target to source) it is necessary to invoke two back-to-back packets from the target host. We will make the target host send these packets from a specified URL using a HTTP style mechanism. Since the target host uses TCP, this task becomes nontrivial and is accomplished as follows. We first set up a TCP connection with a three way handshake. We start receiving base page of the target host after sending HTTP GET. To receive packets of the required size we use the Maximum Segment Size (MSS) parameter. The target host is made to send these packets back-to-back by controlling the advertised window of Experimental Setup 10Mbps 10Mbps 1Gbps 10Mbps Router - 1 Router - 2 Router - 3 Fig. 2. Experimental setup (msec) Time SYN (500) SYN (500) RST (0) Fig. 3. Timing diagram of packet train generation in forward path. Source host sends eight back-to-back large SYN packets to an unused port in the target host. For each SYN, the target host responds with a RST. The times shown are relative to the first packet. the TCP. To make the target host transmit a packettrain of the required length we group the s appropriately. We each received data packet from target host with zero advertised window, but for last we maximize the advertised window size so that target host sends all the packets back-to-back. Fig. 4 describes the algorithm to generate a packet-train on the reverse path. Program runs for user specified number of rounds. For any abnormal response we break the connection and discard those corresponding measurement. Fig. 5 is an example output. This technique assumes FIFO scheduling at the routers, no competing traffic in between packet pair and no downstream congestion. In the presence of competing traffic, Eqn 1 can underestimate the bottleneck link bandwidth. Further, if there is congestion after the bottleneck link, it can overestimate the bottleneck link bandwidth. A. ipathmeter-b ipathmeter-b estimates bottleneck bandwidth and available bandwidth in both reverse and forward direction for a given target host. It takes remote host name/ip address, packet size (in Bytes), train length (number of packets), number of measurements and direction as the command line parameters. For forward measurements, program first sets the BPF [7] filter for the target host, sends back-to-back TCP SYN packets equal to the train length using raw sockets, to some random port uses libpcap to capture the returning

3 3 MSS specifiedsize {Set the option} seqno random() Send TCP SYN to target host recd pkt = grab one packet() {capture packet using libpcap} if recd pkt!= SYN/ then increase seq no send and HTTP GET recd pkt grab one packet() if recd pkt == for GET then for all i such that 1 i rounds do slow win 2 i for all j such that 0 j slow win do recd pkt grab one packet() if (recd pkt) == (RST FIN) then if ((length of train! = req train length) && (pkt recd is last in round)) then send for data pkt with adv window size 0 with all except last. send with req window size for next round. if (pkts reordered retransmission) then save time stamps for future analysis send RST Fig. 4. Pseudocode for reverse packet-train generation SYN (0) SYN/ (0) GET/ (102) P1 (500) P2(500) 1 Wd(0) 2 Wd(32120) P3 (500) P4 (500) P5 (500) P6 (500) 3 Wd(0) 4 Wd(0) 5 Wd (0) 6 P7 (500) P14 (500) Wd(32120) RST Fig. 5. Timing diagram of packet train generation in reverse direction. Program at source host overrides kernel TCP and controls the packet exchange with target host. After handshaking with target host, it asks for base page with a HTTP GET. To get a packet train from the target host each packet is ed with a zero advertised window except for the last in the train. TCP RST packets. For reverse measurements, a firewall using ipchains bypasses the kernel, sets a BPF filter for the target host and opens a connection with its WWW server. It then sends a HTTP GET for the base page and s the data packets received until the specified packet train length is received. Using HTTP GET for the base page makes ipathmeter-b generic for WWW servers. Error conditions out-of-order packets, packet loss, packet retransmission and unexpected packets terminate the program. ipathmeter-b estimates the bandwidth from the time dispersions and displays these estimates as histograms. III. Loss Pattern Measurement Loss pattern analysis is based on [1], [5]. The forward path measurement procedure is straightforward and is similar to the bandwidth measurement process. For the reverse path, we follow the same initial sequence as that for bandwidth measurement and a HTTP GET request is sent. The packet flow is controlling the times that the s are sent. To be able to identify the lost packets, it is necessary to prevent the target host from retransmitting lost packets. Therefore if an expected packet is not received within a time interval of 1.5RT T, an for that packet is sent to prevent the target from timing out and retransmitting. A rate controlled path is emulated by controlling the times of the s and making the target host send according to a specified rate. See

4 4 Open Raw socket Open libpcap descriptor Set firewall to prevent packets from target host reaching local host kernel for i = 1 to i = No.ofIterations do Send TCP SYN to target host Target host IP address,port Number, MSS, Window one segment if Target host sends SYN then Send and HTTP GET request if for request is received then repeat if Sequence Numbers in correct order then for next packet Wait for RTO duration and for next packet. Assume packet is Lost until FIN sent by target host Send for target s FIN and send own FIN Send RST Send RST Fig. 6. Pseudocode for loss pattern measurement. Fig. 6 for details of the algorithm. Metrics like loss distance, loss period, and loss rate can be easily determined from the sample. A. ipathmeter-l ipathmeter-l is a tool that uses the algorithm of Fig. 6 described above. A raw socket is opened and a limited TCP functionality is implemented. To avoid the local host kernel reset connection to target host, an IP chain filter for the desired IP address and port number is introduced. Packets from the target host are captured via libpcap implementation. ipathmeter- L reports loss rate, loss distance and loss period during a HTTP like transaction with a target WWW server. It can be configured to perform these measurements over TCP like or rate controlled flow from the target host. Input parameters are the target host name or the IP address, packet size in bytes, number of packets, number of iterations and mode of data flow control (TCP-like or rate control). Fig. 7 shows a sample sequence from an experiment. We must mention here Local Host lost SYN(0) SYN/(0) GET/(102) P1(90) P2(1460) P3(1460) P4(1460) P5(1460) P6(1460) P7(1460) P20(1460) RST Remote Host Fig. 7. Timing diagram of data flow from WWW server to ipathmeter-l and corresponding s. Program at source host masks the kernel TCP and assumes control of data exchange with WWW server. After establishing connection with the WWW server, requests the base page with a HTTP GET. To enforce rate control the s may be sent after a chosen delay after the data packet is received. One packet marked with dashed line was lost and is thus indicated. that we also need to continuously estimate the RTT and hence the target host s RTO. Estimates of the loss distance and loss period are displayed as histograms. IV. Sample Results Fig. 8 (a),(b) shows measurements taken in forward path from measuring (source) host to target host for bottleneck and available bandwidth respectively. Similarly, Fig. 8 (c),(d) shows the measurement for the reverse path. It is observed that forward measurement results are more prone to errors than reverse measurement results. This may be because of returning s from target host got queued. Fig. 9 shows loss measurement in which vertical bar indicate loss of a particular packet.

5 5 4 No. of Iterations Packet Sequence Fig. 9. Loss pattern for four iterations with as target, observed 50 packets of size 400 bytes. Each box indicates the loss pattern observed for single iteration. Fig. 8. Sample results with as target host, 50 iterations of 16-packet-trains, packet size of 500 bytes. (a) Bottleneck bandwidth in forward path, (b) Available bandwidth in forward path, (c) Bottleneck bandwidth in reverse path and (d) Available bandwidth in reverse path. V. Discussion and Future Work We have introduced the concepts of bandwidth and loss measurements of Internet paths in a noncooperative setting and described algorithms to perform these measurements. ipathmeter-b and ipathmeter-l that can perform such measurements is also described. The ipathmeter has been tested over a number of experiments carried out with different WWW servers and the results thus obtained were consistent. This tool can be downloaded from balaji/ipathmeter/. In the current implementation of ipathmeter-b redirected WWW servers, web pages having frames and servers for which cookies need to set return an error condition. We are addressing this at this time. We have also observed that for forward measurements, some of the servers do not respond to TCP SYN for random ports. This may be because servers use a specially compiled kernel that does not respond to requests other than that to port 80. We are also working towards making ipathmeter-l characterise paths with traffic shapers. Various models of traffic shaping may be employed which generally involve allowing a burst of the users traffic every few seconds (or milliseconds). This requires buffering the users traffic until his time slot and thereafter transmit the buffered data. The measures of interest will be the size of the buffer and the policy for handling users data when the buffer size is exceeded. If these packets are dropped, the loss pattern observed would be definitely different from what would have been otherwise. The ipathmeter-l in its present form would handle the study of traffic shapers for outgoing traffic from the user and not the other way around. References [1] G. Almes, S. Kalidindi, and M. Zekauskas, A One-Way Packet Loss Metric for IPPM, IETF RFC 2680, Sept. 1999, [2] R. Carter and M. Crovella, Measuring Bottleneck Link Speed in Packet-Switched Networks, Tech. Report , Computer Sc Dept, Boston University, [3] C. Dovrolis, P. Ramanathan, and D. Moore, What do Packet Dispersion Techniques Measure? in Proc of IEEE INFOCOM, pages , [4] S. Keshav, Packet-pair flow control, Technical report, AT&T Bell Labs, Murray Hill, New Jersey, [5] S. Koodli. and R. Ravikanth, One Way Loss Pattern Sample Metrics, IETF RFC 3357, August 2002, [6] K. Lai and M. Baker, Nettimer: A Tool for Measuring Bottleneck Link Bandwidth, in Proc. of USENIX Symposium on Internet Technologies and Systems, March [7] S. McCanne, V. Jacobson, The BSD Packet Filter: A New Architecture for User-level Packet Capture, in Proc. of Winter USENIX, pp , [8] J. Padhye and S. Floyd, On Inferring TCP Behavior, in Proc. of ACM SIGCOMM 01, pp 27 31, San Diego, Cal. USA, August [9] V. Paxson, G. Almes, J. Mahdavi, M. Mathis, Framework for IP Performance Metrics, IETF RFC 2330, May 1998,