Traffic Characteristics of Bulk Data Transfer using TCP/IP over Gigabit Ethernet

Transcription

1 Traffic Characteristics of Bulk Data Transfer using TCP/IP over Gigabit Ethernet Aamir Shaikh and Kenneth J. Christensen Department of Computer Science and Engineering University of South Florida Tampa, Florida {ashaikh, This material is based upon work funded by the National Science Foundation under grant no KJC001 (ipccc01a.ppt - 03/28/01) Topics Introduction and questions asked Previous work by others Experiments and characterization Some insights Shaping of TCP/IP traffic Summary and future work KJC002 1

2 Introduction and questions asked Knowledge of traffic characteristics needed for Capacity planning of networks Design of network devices including switches and routers Bulk data transfer applications are increasing Napster is one example Image and video files are other examples TCP/IP is expected to remain the dominant protocol for these apps KJC003 Introduction and questions asked continued Five questions related to 100 and 1000-Mbps Ethernet TCP/IP for bulk data transfer 1) How do traffic characteristics change as a function of link speed, disk or memory access, server load, and OS? 2) How are packets interleaved as a function of the number of connections? 3) What are the expected queuing delays and losses? 4) What are the root causes of interesting observed behaviors? 5) What can be done to reduce queueing delays? KJC004 2

3 Previous work by others Much work done in studying aggregated traffic [4, 10, 12] Aggregated traffic is bursty and LRD Web traffic studied [15] Heavy-tailed file sizes contribute to LRD traffic TCP performance on Gigabit Ethernet studied [6] Effects of processor speed, Linux kernel version, tuning TCP pacing of return ACKs studied [1] Too much pacing can lower throughput in some cases by causing synchronized losses and congestion control» Not clear how much pacing is too much KJC005 Experiments and characterization Testbed and software tools Server (700) Trace collector (700) Alteon ACEswitch Software: WinNT and Linux FTP server/client netperf http_load windump Load generator (dual-300) Clients (1x700, 3x866) KJC006 3

4 Experiments and characterization continued Trace collection experiments - Disk-to-disk transfer FTP of 200 Mbyte file All experiments run for , , and link speeds Memory-to-memory transfer Netperf of 200 Mbyte Operating system WindowsNT and Linux Server load Use http_load to put 20% representative load on server Multiple clients Disk-to-disk with multiple clients KJC007 Experiments and characterization continued Characterization methods - Packet interarrival mean, CoV, and P/M (peak-to-mean) Queueing behavior for single server queue (90% offered load) Trace-driven CSIM18 simulation model Effects of spreading ( for i = 1,..., N ) t i ( ti tclip ) if ti tclip N clip and then ti = ti + i= 1 N S clip = > Effects of smoothing Simulated leaky bucket in sender (byte-based tokens) S KJC008 4

5 Experiments and characterization continued Queueing model for characterization - Single server queue driven by trace Trace is series of <delta_time, pkt_len> tuples Implemented in CSIM18 trace loss Set link speed to achieve target offered load Set buffer size to study loss KJC009 Experiments and characterization continued Characterization results for single client Memory-to-memory Transfer rate (Mbps) WindowsNT 2 1 Linux Time (sec) Snapshot KJC010 5

6 Experiments and characterization continued Characterization results for single client - continued Disk-to-disk Mean CoV P/M Q Delay Linux WinNT No server load Q delay = 5.83 ms Mem-to-mem Mean CoV P/M Q Delay Linux WinNT x lower thruput Q delay = 0.30 ms KJC011 Experiments and characterization continued Characterization results for single client - continued Disk-to-disk Mean CoV P/M Q Delay Linux WinNT w/ server load Q delay = ms Mem-to-mem Mean CoV P/M Q Delay Linux WinNT x lower thruput Q delay = 0.28 ms KJC012 6

7 Experiments and characterization continued Characterization results for multiple clients - Disk-to-disk Mean CoV P/M Q Delay Same - 1 client clients clients Diff - 1 client clients clients Aggregate flow For multiple same file requests, burstiness decreases as number of file requests increases. For multiple different file requests, burstiness increases as number of file requests increases KJC013 Experiments and characterization continued Characterization results for multiple clients - continued Autocorrelation clients, diff file 3 clients, diff file Different files 1 client Lag Lag 6 = window Lag 45 = block KJC014 7

8 Experiments and characterization continued Effects of 1% spreading on disk-to-disk - Packet loss (%) Buffer = 128KB Linux Linux w/ 1% spread WindowsNT Wow! WindowsNT w/ 1% spread Offered load (%) Queueing loss KJC015 Some insights Burstiness - Burstiness is a function of all parameters studied Neither CoV or P/M are consistent (or good) predictors for queueing delay Queueing delay can very by three to four orders of magnitude For same offered load! Both data packets and ACKs were seen to be delayed By close look at a trace Interrupt coalescing is the cause for this KJC016 8

9 Some insights continued Interrupt coalescing - Driver holds multiple packets before sending or receiving Reduces the number of interrupts Maximizes the work done per interrupt Decreasing coalescing time or count decreases burstiness But, increases CPU utilization KJC017 Shaping of TCP/IP traffic Timed socket send() call - Add delay between packets at the sockets send() or sendto() This delay smoothes the transmitted stream If delay is same as mean interarrival time Have something close to deterministic traffic But, complex interactions may TCP will occur Trade-off of delay in sending host versus in network KJC018 9

10 Shaping of TCP/IP traffic Timed socket send() call - continued Our implementation used software spin loop delay Hardware support might held reduce CPU load Timer set and timer interrupt Application Sockets shim TCP (or UDP) IP Device driver Sockets interface Management software Link Possible hardware support for timers KJC019 Ethernet adapter with microsecond timer support Shaping of TCP/IP traffic continued Timed socket send() call - continued Results for memory-to-memory transfer for WinNT Without timed send() With timed send() Mean CoV P/M Q Delay Mean CoV P/M Q Delay x reduction in queueing delay with virtually no change in rate KJC020 10

11 Shaping of TCP/IP traffic continued Timed socket send() call - continued Results for disk-to-disk transfer for WinNT Without timed send() With timed send() Mean CoV P/M Q Delay Mean CoV P/M Q Delay x reduction in queueing delay with no change in rate Not as much reduction as for memory-to-memory KJC021 Summary and future work We studied characteristics of TCP/IP bulk data transfers Focused on effects of source parameters Used a single server trace-driven queueing simulation to study behavior in network bottleneck queues Delay and loss Disk-to-disk transfer are especially bursty Both window and disk block size can be seen in the bursts Interrupt coalescing used to reduce CPU load increases burstiness Prototyped shaping at the sockets layer Improves traffic characteristics» Reduces queueing delays KJC022 11

12 Summary and future work continued Future work should study trade-off of host vs. network delay Should study effects to TCP/IP congestion control Also, further investigate hardware support for shaping KJC023 12