Layer 4 TCP Performance in carrier transport networks Throughput is not always equal to bandwidth

Transcription

1 Layer 4 TCP Performance in carrier transport networks Throughput is not always equal to bandwidth Roland Stooss JDSU Deutschland GmbH Senior Consultant Data/IP Analysis Solutions Mühleweg 5, D Eningen u. A. Tel. +49 (0)7121/ roland.stooss@jdsu.com

2 OSI Model: Responsibility and recommended tests Customer s IT department responsibility Shared responsibility Network Provider s responsibility Application Application Presentation Presentation Session Session Transport Network Network Datalink Datalink Physical Physical HTTP, FTP, , etc. TCP IP Ethernet RFC6349 RFC2544 and/or Y

3 Throughput is not always equal to available bandwidth Layer 4/TCP really outside of the responsibility of the carrier. Layer 2/3 throughput (bandwidth) test 100% Across test link 100 % available Layer 4 throughput test (upload) Back-to-back 100 % available Layer 4 throughput test (upload) Across test link 40 % throughput -> Why? Typical customer complains Bad application responds times Total throughput is not like total available bandwidth -> Why? LAN 40%? 100% 40%? 3

4 TCP throughput the limiting factors (1) Performance even if there is no JAM in the network Bandwidth Latency Error Rate Performance Upload Max. Segment Size Retransmission Timer Window Size Sender can use full bandwidth and transmit at 100 %. Thus the transmit buffer is quickly used up. Latency is too large. Therefore the ACKs of the receiver are coming too late. Sender may already start sending unnecessary retransmissions. Cause Window Size does not fit to bandwidth and latency. Only one TCP session with high available bandwidth. 4

5 TCP throughput the limiting factors (2) Performance Bandbreite Bandwidth Laufzeit Latency Error rate Performance MTU Upload Policing/Shaping Retransmission Timer Window Size Responsibility with carrier Responsibility with customer Shared responsibility: Bandwidth: Customer may ordered too low BW or carrier doesn t deliver BW Latency: Customer didn t fix RTT in SLA. Carrier selected long path MTU: Customer configured too small MTU or carrier has packet size limitations because of used encapsulation method (MPLS) Policing/Shaping: At the critical rate limitation interfaces (in customers or carriers network) Basic policing is extreme critical for TCP performance. 5

6 Why does a TCP traffic get less throughput? Performance of TCP client and server and Round Trip Time As higher the performance and as lower the RTT as faster the Slow Start TCP Traffic rate [1] Available Bandwidth Rate limit by policing [2] t High performance TCP client and low RTT [1]: Session could go above available BW. It always runs in extreme congestion. Final averaged throughput is very low High performance TCP client and large RTT [2]: Session can t go above available BW, therefore no congestion and stable traffic rate. Multiple TCP sessions with possible cumulated TCP traffic above BW may not reach the BW, because of congestion effect. 6

7 Where might be the network problems for TCP? At customers LAN to WAN CE interface Total LAN rates are cumulated higher than WAN access interface (1 GigE) Access Metro/Core 1GigE SDH/OTN Fixed BW LAN CE PE IP/MPLS PE CE LAN 600Mb Eth/VLAN Dynamic BW At carrier access with rate limit function Most critical point, especially when Traffic Policy instead of Traffic Shaping is used At carrier metro/core SDH/OTN This is mainly an end-to-end fixed and 100% available bandwidth No further congestion situations (loss, delay) -> fine for TCP traffic At carrier metro/core IP/MPLS or Eth(VLAN) This is packet based switching with dynamic and possible over booked BW Congestion situations can occur (loss events and increasing delay) -> can be stress for TCP sessions. Probably less like with policing at the access. 7

8 Carrier Access When is their rate limit? The carrier access port is limiting the accepted traffic to the available transport rate. The carrier access port is limiting the accepted traffic per service to their available service rate (CIR) Total access bandwidth might be 100%. Single service rate limit Total BW 100%, but each service has its CIR rate limit Multiple services with different CIRs has to be considered as point of rate limit like single service with rate limit. Recommended test application to detect TCP performance problems -> RFC6349 Frame Work for TCP Throughput testing Not really helpful RFC2544 and/or Y

9 Lack of TCP Testing is a Service Activation Gap Turn-up Related Problem RFC2544 Y.1564 RFC 6349 Single Service, Layer 2/3 SLA Issues (loss, jitter, etc.) Multi-service, Layer 2/3 SLA Issues (service prioritization, loss, jitter, etc.) N/A N/A Demonstrate the effect of End customer TCP Window size on throughput (CPE issue) Inadequate device buffers to handle bursty applications Policing effects to TCP performance 9

10 Application aware turn-up testing workflow Traditional L2/3 Turnup RFC-2544 Class of Service verification ITU-T Y.1564 SAMComplete Application level testing RFC-6349 / TrueSpeed Verify basic performance for single services Prove L2/3 KPIs for differentiated services Experience Your Network Like Your Customer Do 10

11 Automated test routines help ensure quality testing Quality of testing = Reliability + Repeatability independent of individual skill level TrueSpeed automated RFC-6349 test in <5 min 11

12 TCP Performance Tests What has to be tested? Can the maximum packet size be used? Path MTU (IP level) should be 1500 bytes = 1460 bytes MSS on TCP level What is the best Round Trip Time (RTT)? Measured with low load to avoid influence of filled buffers Show influence of TCP Window Size Run 4 TCP sessions with different WS values and compare actual measured versus ideal TCP throughput. This helps to explain the TCP WS influence and will already show critical problems. Run a typical TCP session below the Bottleneck Bandwidth Test duration based on bytes to transfer instead of fixed test time. Compare actual versus ideal rate and transfer time Graphical Results to correlate possible TCP performance issues due to retransmissions and/or congestive RTT. New RFC 6349 Run multiple TCP sessions to force congestion situation This test will conduct multiple TCP connection transfers to test whether the link fairly shares (traffic shaping) or unfairly shares (traffic policing) the bandwidth (MTU: Maximum Transfer Unit; MSS: Maximum Segment Size) 12

13 Why does TCP traffic not reach the full bandwidth? Why do customer only sometimes complain about not getting the committed throughput? Few main factors if rate limit is in place: TCP load is mainly based on large file transfers and less short sessions Short sessions are not all active at the same time, they better share the bandwidth With large file transfers a bad throughput is more likely recognized. How is the performance of TCP client/server and how is Round Trip Time High-performance servers or old & weak PCs As higher the performance and as lower the RTT as faster the Slow Start Possible total TCP rates are higher then available transport rate. The TCP session(s) may run into congestion As larger the difference as more critical (E.g. 100Mbps interface to 2 Mbps transport) TCP window size is larger then ingress buffer of the network port During slow start the buffer is quickly filled and results in packet loss If there is also large RTT the sessions runs into Dup Ack situation Policing function at the network access port Is the throughput rate done in slots of available bandwidth 13

14 TCP performance and congestion in the metro/core Access Core Congestion in the core is different to congestion (policy based rate limitation) at customer access This is packet based switching with dynamic and possible over booked BW. Congestion may cause loss events and increasing delay. Recommended test application to detect TCP performance problems -> RFC6349 Frame Work for TCP Throughput testing -> Layer 2/3 traffic test with focus on packet delay and delay variation PDV shows short burst situations and PD shows longer high load and overload situations, which may result in loss. 14

15 Carrier Access Where and how to limit the rate? Who is managing the bandwidth limitation customer or/and carrier? Essential criteria: Access rate = or > as transport rate N x Access rate Customer CPE Access rate Provider CE Transport rate Access rate even as transport rate Nothing Policing Shaping Policing Shaping Access rate larger as transport rate Nothing Policing Nothing Shaping Shaping Nothing Nothing Policing Policing Nothing Policing Policing Shaping Shaping Policing The above examples are only exemplary and not complete. They shall only show the complexity 15

16 Why layer 4 TCP full state test stream emulation? Customer requires proof of the transfer capacity with real layer 4 TCP load (According RFC 6349) Mostly as second step after failed self-test with laptop file transfer. Quite meaningfully, if the connection goes through a packet oriented (Ethernet, PBB, MPLS) core network. UDP displaces TCP traffic. This is not detected with pure L3 or state less L4 load. Low packet loss events can extremely reduce the TCP throughput if there is also high RTT. To determine the best TCP window size configuration for max. throughput. Performance tests across full state fire walls. When layer 4 TCP state less load is enough? When the transport link has fixed bandwidth and no rate limit at the access (Fiber, optical channel, SDH/OTN) Only RTT and TCP Windowsize is influencing the final throughput 16

17 Estimated, ideal max TCP throughput Calculation based on ideal network situations The test application is to supply additional information, which points the protocol specific throughput limits out. What is the max. throughput for a TCP session, which can be expected? Depends on the standard TCP window size values (8/16/32/64 kbyte) The table shows the max. throughput rates, dependent on the TCP window size, RTT and number of TCP sessions. Note: Calculation is based on ideal network situations and is done with statically test load 17