TCP CONGESTION SIGNATURES. Srikanth Sundaresan (Princeton Univ.) Amogh Dhamdhere (CAIDA/UCSD) kc Claffy (CAIDA/UCSD) Mark Allman (ICSI)

Transcription

1 TCP CONGESTION SIGNATURES Srikanth Sundaresan (Princeton Univ.) Amogh Dhamdhere (CAIDA/UCSD) kc Claffy (CAIDA/UCSD) Mark Allman (ICSI) 1

2 Typical Speed Tests Don t Tell Us Much 2

3 Typical Speed Tests Don t Tell Us Much 2

4 Typical Speed Tests Don t Tell Us Much 2

5 Typical Speed Tests Don t Tell Us Much Upload and download throughput measurements: no information beyond that 2

6 Typical Speed Tests Don t Tell Us Much What type of congestion did the TCP flow experience? 2

7 Two Potential Sources of Congestion in the End-to-end Path 3

8 Two Potential Sources of Congestion in the End-to-end Path Self-induced congestion - Clear path, the flow itself induced congestion - eg: last-mile access link 3

9 Two Potential Sources of Congestion in the End-to-end Path Self-induced congestion - Clear path, the flow itself induced congestion - eg: last-mile access link External congestion - Flow starts on an already congested path - eg: congested interconnect 3

10 Two Potential Sources of Congestion in the End-to-end Path Self-induced congestion - Clear path, the flow itself induced congestion - eg: last-mile access link External congestion - Flow starts on an already congested path - eg: congested interconnect Distinguishing the two cases has implications for users / ISPs / regulators 3

11 How can we distinguish the two? Cannot distinguish using just throughput numbers - Access plan rates vary widely, and are typically not available to content / speed test providers - eg: Speed test reports 5 Mbps is that the access link rate (DSL), or a congested path? 4

12 How can we distinguish the two? Cannot distinguish using just throughput numbers - Access plan rates vary widely, and are typically not available to content / speed test providers - eg: Speed test reports 5 Mbps is that the access link rate (DSL), or a congested path? We can use the dynamics of TCP s startup phase, i.e., Congestion Signatures 4

13 TCP s RTT Congestion Signatures 5

14 TCP s RTT Congestion Signatures Flows experiencing self-induced congestion fill up an empty buffer during slow start - Hence increase the TCP flow RTT 5

15 TCP s RTT Congestion Signatures Flows experiencing self-induced congestion fill up an empty buffer during slow start - Hence increase the TCP flow RTT Externally congested flows encounter an already full buffer - Less potential for RTT increases 5

16 TCP s RTT Congestion Signatures Flows experiencing self-induced congestion fill up an empty buffer during slow start - Hence increase the TCP flow RTT Externally congested flows encounter an already full buffer - Less potential for RTT increases Self-induced congestion therefore has higher RTT variance compared to external congestion 5

17 TCP s RTT Congestion Signatures Flows experiencing self-induced congestion fill up an empty buffer during slow start - Hence increase the TCP flow RTT Externally congested flows encounter an already full buffer - Less potential for RTT increases Self-induced congestion therefore has higher RTT variance compared to external congestion We can quantify this using Max-Min and CoV of RTT 5

18 Example Controlled Experiment CDF CDF External Self Max-Min RTT External Self CoV RTT Mbps access link with 100 ms buffer 1 Gbps interconnect link with 50 ms buffer Self-induced congestion flows have higher values for both metrics and are clearly distinguishable 6

19 Example Controlled Experiment CDF CDF External Self Max-Min RTT External Self CoV RTT Mbps access link with 100 ms buffer 1 Gbps interconnect link with 50 ms buffer Self-induced congestion flows have higher values for both metrics and are clearly distinguishable The two types of congestion exhibit widely contrasting behaviors 6

20 Model Max-min and CoV of RTT derived from RTT samples during slow start We feed the two metrics into a simple Decision Tree - We control the depth of the tree to a low value to minimize complexity We build the decision tree classifier using controlled experiments and apply it to real-world data 7

21 Validating the Method: Step 1- Controlled Experiments Server 1 Server 2 Pi 1 Pi Mbps Shaped access R1 1 Gbps R2 Internet Server 3 Server 4 8

22 Validating the Method: Step 1- Controlled Experiments Pi 1 Pi Mbps Shaped access R1 Background cross-traffic 1 Gbps Interconnect cross-traffic R2 Server 1 Server 2 Internet Server 3 Server 4 8

23 Validating the Method: Step 1- Controlled Experiments Pi 1 Pi Mbps Shaped access Throughput tests R1 Background cross-traffic 1 Gbps Interconnect cross-traffic R2 Server 1 Server 2 Internet Server 3 Server 4 8

24 It s Real 9

25 It s Real Fantastic Cabling effort Post-it defined networking 9

26 Validating the Method: Step 1- Controlled Experiments Pi 1 Pi Mbps Shaped access Throughput tests R1 Background cross-traffic 1 Gbps Interconnect cross-traffic R2 Server 1 Server 2 Internet Server 3 Server 4 Emulated access link + core link - Wide range of access link throughputs, buffer sizes, loss rates, crosstraffic (background and congestion-inducing) - Can accurately label flows in training data as self or externally congested 10

27 Validating the Method: Step 1- Controlled Experiments Pi 1 Pi Mbps Shaped access Throughput tests R1 Background cross-traffic 1 Gbps Interconnect cross-traffic R2 Server 1 Server 2 Internet Server 3 Server 4 High accuracy: precision and recall > 80% robust to model settings 11

28 Validating the Method: Step 2 ISP B Ark VP ISP A From Ark VP in ISP A identified congested link with ISP B using TSLP* *Luckie et al. Challenges in Inferring Internet Interdomain Congestion, IMC

29 Validating the Method: Step 2 ISP B congested link Ark VP ISP A From Ark VP in ISP A identified congested link with ISP B using TSLP* *Luckie et al. Challenges in Inferring Internet Interdomain Congestion, IMC

30 Validating the Method: Step 2 M-lab NDT server ISP B congested link Ark VP ISP A Periodic NDT tests from Ark VP to M-Lab NDT server behind the congested interdomain link 13

31 TSLP latency (far side) d/l Mbps Validation of the Method: Step /18 02/25 03/04 03/ /18 02/25 03/04 03/11 Strong correlation between throughput and TSLP latency: flows during elevated TSLP latency labeled as externally congested 14 w w w.caid a.or

32 TSLP latency (far side) d/l Mbps Validation of the Method: Step /18 02/25 03/04 03/ /18 02/25 03/04 03/11 Externally congested Strong correlation between throughput and TSLP latency: flows during elevated TSLP latency labeled as externally congested 14 w w w.caid a.or

33 d/l Mbps Validation of the Method: Step 2 TSLP latency (far side) self congested /18 02/25 03/04 03/ /18 02/25 03/04 03/11 Externally congested Strong correlation between throughput and TSLP latency: flows during elevated TSLP latency labeled as externally congested 14 w w w.caid a.or

34 TSLP latency (far side) d/l Mbps Validation of the Method: Step /18 02/25 03/04 03/ /18 02/25 03/04 03/11 75%+ accuracy in detecting external congestion, 100% accuracy for self-induced congestion 15 w w w.caid a.or

35 Validation of the Method: Step 3 We use Measurement Lab s NDT test data for real-world validation Cogent interconnect issue in late 2013/early NDT tests to Cogent servers saw significant drops in throughput during peak hours - Several major U.S. ISPs were affected, except Cox - The problem was identified as congested interconnects 16

36 Using the M-lab Data Comcast Cox TimeWarner Verizon January 2014 Mbps Hour of day (local) Comcast Cox TimeWarner Verizon April 2014 Mbps Hour of day (local) 17

37 Using the M-lab Data Comcast Cox TimeWarner Verizon January 2014 Mbps Drop in peak-hour throughput for for Comcast, TWC, Verizon Hour of day (local) Comcast Cox TimeWarner Verizon April 2014 Mbps Hour of day (local) 17

38 Using the M-lab Data Comcast Cox TimeWarner Verizon January 2014 Mbps Cox not affected Drop in peak-hour throughput for for Comcast, TWC, Verizon Hour of day (local) Comcast Cox TimeWarner Verizon April 2014 Mbps Hour of day (local) 17

39 Using the M-lab Data Comcast Cox TimeWarner Verizon January 2014 Mbps Cox not affected Drop in peak-hour throughput for for Comcast, TWC, Verizon Hour of day (local) Comcast Cox TimeWarner Verizon April 2014 Mbps Interconnection dispute resolved; no diurnal effect Hour of day (local) 17

40 Using the M-lab Data Comcast Cox TimeWarner Verizon Peak hour tests in Mbps 20 Jan/Feb 2014 are likely 10 externally congested Hour of day (local) Comcast Cox TimeWarner Verizon Off-peak tests in Mbps Mar/Apr 2014 are likely self congested Hour of day (local) 18

41 But didn t you just say it s hard to infer congestion using throughput tests?? 19

42 But didn t you just say it s hard to infer congestion using throughput tests?? Yes :) 19

43 But didn t you just say it s hard to infer congestion using throughput tests?? Yes :) For that reason, our labeling is broad and coarse. All tests labeled external may not be traversing congested interconnects 19

44 But didn t you just say it s hard to infer congestion using throughput tests?? Yes :) For that reason, our labeling is broad and coarse. All tests labeled external may not be traversing congested interconnects We do not expect the technique to identify all peak hour tests as externally congested, and vice versa - Looking for qualitative differences 19

45 But didn t you just say it s hard to infer congestion using throughput tests?? Yes :) For that reason, our labeling is broad and coarse. All tests labeled external may not be traversing congested interconnects We do not expect the technique to identify all peak hour tests as externally congested, and vice versa - Looking for qualitative differences The general observations about congestion were verified by other sources, e.g., CAIDA s TSLP measurements 19

46 Applying the Model to M-lab data Jan-Feb Mar-Apr %self-inducedcongestion Cogent (LAX) Cogent (LGA) Level3 (ATL) Comcast TimeWarner Verizon Cox Comcast TimeWarner Verizon Cox Comcast TimeWarner Verizon Cox 20

47 Applying the Model to M-lab data Jan-Feb Mar-Apr %self-inducedcongestion Cogent (LAX) Cogent (LGA) Level3 (ATL) Comcast TimeWarner Verizon Cox Comcast TimeWarner Verizon Cox Comcast TimeWarner Verizon Cox Much lower incidences of self-induced congestion for Cogent in Jan/Feb 2014 as compared to Mar/Apr 20

48 Applying the Model to M-lab data Jan-Feb Mar-Apr %self-inducedcongestion Cogent (LAX) Cogent (LGA) Level3 (ATL) Comcast TimeWarner Verizon Cox Comcast TimeWarner Verizon Cox Comcast TimeWarner Verizon Cox Level3 does not show significant differences, was not affected by interconnection disputes 21

49 Applying the Model to M-lab data Jan-Feb Mar-Apr %self-inducedcongestion Cogent (LAX) Cogent (LGA) Level3 (ATL) Comcast TimeWarner Verizon Cox Comcast TimeWarner Verizon Cox Comcast TimeWarner Verizon Cox Cox does not show significant differences, was not affected by interconnection disputes 22

50 Looking at Throughput What throughput should we observe for self and external congested flows? With congested interconnects affecting many flows, both self and external should see similar throughput Without congested interconnects affecting many flows, self congested throughput should follow access link speeds, generally higher than externally congested 23

51 Looking at Throughput Jan-Feb Self Jan-Feb External Mar-Apr Self Mar-Apr External Throughput (Mbps) Comcast TimeWarner Verizon Cox Avg. throughput of self-induced congestion flows significantly higher than externally congested in Mar-Apr (no interconnection disputes) 24

52 Looking at Throughput Both self and external get similar throughput Throughput (Mbps) Jan-Feb Self Mar-Apr Self Jan-Feb External Mar-Apr External Comcast TimeWarner Verizon Cox Avg. throughput of self-induced congestion flows significantly higher than externally congested in Mar-Apr (no interconnection disputes) 24

53 Looking at Throughput Jan-Feb Self Jan-Feb External Mar-Apr Self Mar-Apr External Throughput (Mbps) Comcast TimeWarner Verizon Cox Avg. throughput of self-induced congestion flows significantly higher than externally congested in Mar-Apr (no interconnection disputes) 25

54 Looking at Throughput Throughput (Mbps) Jan-Feb Self Mar-Apr Self Jan-Feb External Mar-Apr External Comcast TimeWarner Verizon Cox Self get higher throughput than external Avg. throughput of self-induced congestion flows significantly higher than externally congested in Mar-Apr (no interconnection disputes) 25

55 Takeaways It is possible to distinguish two kinds of congestion: selfinduced vs. externally congested The difference is important to identify the solution - Upgrade service plan? Or talk to ISP? - Also for regulatory purposes Simple, accurate technique using RTT during TCP slow start dynamics - Can be easily computed using packet captures or other tools such as Web100 (future work) 26

56 Limitations Relies on buffering effect - May not work on TCP variants that minimize buffer occupancy, e.g., BBR Only uses slow start dynamics - Might be confounded by flows that perform one way during slow start but differently afterward Real-world validation relies on coarsely labeled data - It would be great to validate on more real-world data! 27

57 Thanks! Questions? 28