Advanced Computer Networks. Datacenter TCP

Transcription

1 Advanced Computer Networks Datacenter TCP Patrick Stuedi, Qin Yin, Timothy Roscoe Spring Semester Oriana Riva, Department of Computer Science ETH Zürich

2 Last week Datacenter Fabric Portland VL2 Facebook Altoona 2

3 Today Datacenter traffic analysis TCP incast Fine-grained TCP retransmissions Data Center TCP (DCTCP) Deadline-aware TCP Multipath TCP 3

4 Why bother about TCP? Applications require reliable end-to-end data transfer Typically implemented using TCP Problem: TCP designed for the Internet, performance in other networks often suboptimal Example: High bandwidth/delay product networks Two options: Improve TCP for other networks: SnoopTCP for wireless, DCTCP for datacenter Use other protocol instead How easy is it replace TCP? Easy if application is written from scratch Possible if we control both ends of the communication (like in a datacenter) Difficult if we are not controlling both ends: protocol often dictated by the server (e.g., HTTP/TCP webserver) 4

5 The Partition/Aggegrate Pattern 5

6 How does search work? Picasso Picasso

7 How Does Search Work? Picasso Picasso source: stanford CS244

8 How Does Search Work?

9 Partition / Aggregate Foundation of many large web applications Web-search, social networks, ad selection Example: Facebook Aggregators: web servers Workers: Memcached servers Memcached source: stanford CS244

10 Measureing Traffic of a Datacenter 10

11 Two interesting studies The Nature of Data Center Traffice: Measurement & Analycis, IMC, 2009 Looks at MapReduce Datacenters Measuring traffic on servers (intercept socket calls) Network Traffic Characteristics of Data Centers in the Wild, IMC 2010 Wide range of data centers (Universities and Commercial) Measurements using SNMP counters at switches and packet traces Measurement difficulties: make sure measurement technique does not affect performance (e.g., CPU utilization) All datacenters 2-tiered or 3-tiered topologies 11

12 Who talks to whom? Two patterns dominate Most traffic stays within the rack Scatter/gather 12

13 Intra-rack versus extra-rack communciation Extra-Rack Inter-Rack EDU1 EDU2 EDU3 PRV1 PRV2 CLD1 CLD2 CLD3 CLD4 CLD5 Clouds: most traffic stays within rack Other DCs: > 50 % of the traffic leaves the rack 13

14 Are the links busy? Many links experience occasional congestion 86% of the links observe congestion of at least 10s 15% of the links observe congestion of at least 100s 14

15 Link utilization across DC layers Utilization: core > aggr > edge Cloud DCs: core links often loaded > 70% (e.g., CLD5) 15

16 Congestion length Contiguous Duration of >70% link utilization (seconds) Most periods of congestion tend to be short lived 86% of all congestion events last at less than 10s 16

17 Flow size 65% of Flows are < 1MB 1 CDF 0.8 Flow Size Total Bytes Flow Size (Bytes) % of Bytes from Flows > 1MB 10 8

18 Datacenter Workloads Mice & Elephants Short messages (50KB-1MB) (query, coordination, control state) Delay sensitive Large Flows (1MB-100MB) (data update, backup) Throughput sensitive 18

19 Measurement Take Away Cloud DCs: Large parts of the traffic stays within a rack University DCs: 50% of the traffic leaves the rack Link utilization low at edge and aggregation layer Core most utilized, > 70% in Cloud DCs Congestion of flows typically last a few seconds Most flows are small (< 1MB) But it is the large flows that make up for most of the bytes Workload composed of short delay-sensitive message, and large throughput-sensitive flows 19

20 Refresh: TCP Congestion Control 20

21 Remember: TCP congestion control Congestion control got added to TCP to in attempt to reduce congestion inside the network Must rely on indirect measures of congestion Implemented at the sender Src Dest Attempts to reduce buffer overflow inside the network 21

22 Remember: TCP Slow Start Congestion window (CW) Number of bytes in TCP that can be transmitted without waiting for the ACK (CW always smaller than receiver window, flow ctrl) Initially set to 1 TCP segment SSThresh Initially set to 64 KB TCP congestion control: After all ACKs corresponding to one CW have been received (typically after one RTT), the window is doubled - slow start (actually quite fast) If CW >= SSThresh increase CW by 1 TCP segment after all ACKs corresponding to one CW have been received - linear increase (congestion avoidance) On a timeout: Set SSThresh to half of the current CW, then set CW back to 1K 22

23 Example: Slow start 23

24 Incast 24

25 Problem: TCP Incast Worker 1 Synchronized fan-in congestion: Caused by Partition/Aggregate. Worker 2 Worker 3 Aggregator RTOmin = 300 ms Worker 4 TCP timeout 25

26 TCP Incast (2) Incast event measured in a production environment Request forwarded in over 0.8 ms (800 microseconds) All but one response returning in 12.4 ms Retransmission after RTO: 300 ms 26

27 Throughput collapse Collapse! Cause of throughput collapse: coarse-grained TCP timeouts 27

28 Approach: Fine-grained timeouts Roundtrip timeout typically set based on measured RTT+X, with RTO >= RTO_min Two problems: Most Linux TCP implementations do not measure RTT as fine granular as needed for datacenters (Linux jiffies updated per second) RTO_min typically too large Idea: Reduce RTO_min Measure RTT using high-resolution timers in us granularity 28

29 Lower timeout helps microsecond TCP + no minrto 1ms minrto Unmodified TCP (200ms minrto) more servers High throughput for up to 47 servers 29

30 Data Center TCP (DCTCP) 30

31 Queue Buildup Queue Buildup Sender 1 Large flows buildup queues: Increase latency for short flows. Receiver Sender 2 drop

32 Approach: Datacenter TCP Datacenter TCP (DCTCP): Mark packets in switches using Explicit Congestion Notification (ECN) if they experience congestion Scale the TCP window down proportionally to the number of packets with ECN bit set ECN Marks TCP DCTCP Cut window by 50% Cut window by 40% Cut window by 50% Cut window by 5% 32

33 DCTCP Algorithm Switch-side: Mark packets if Queue length > K using ECN bit Receiver-side: Echo bit back to sender with delayed ACKs Sender-side: Maintain running average a of fraction of packets marked (value of 'a' between 0 and 1) - a = (1-g)*a + g*f F: fraction of packets marked in last window 0 > g < 1 : weight given to new samples - a close to 0 means low congestion - a close to 1 means high congestion Window decrease in case of ECN-marked ACK: w = w x (1-a/2) 33

34 DCTCP in Action DCTCP achieves full throughput (not shown in Figure) while taking up a very small footprint in the switch 34

35 Deadline-aware TCP 35

36 User-facing online services Partition/aggregate workflow Application SLAs Cascading SLAs Network SLAs Flow deadlines A flow is useful if and only if it satisfies its deadline 36

37 Limitations of fair sharing (1) Flows f1 and f2 get a fair share of bandwidth Flow f1 misses its deadline (incomplete response to user) Case for unfair sharing: Flows get bandwith in accordance to their deadlines Deadline awareness ensures both flows satisfy deadlines 37

38 Limitations of fair sharing (1) Flows f1 and f2 get a fair share of bandwidth Flow f1 misses its deadline (incomplete response to user) Case for unfair sharing: Flows get bandwith in accordance to their deadlines Deadline awareness ensures both flows satisfy deadlines 38

39 Limitations of fair sharing (2) Insufficient bandwidth to satisfy all deadlines With fair share, all flows miss the deadline (empty response) Case for flow quenching: With deadline awareness, one flow can be quenched All other flows make their deadline 39

40 Limitations of fair sharing (2) Insufficient bandwidth to satisfy all deadlines With fair share, all flows miss the deadline (empty response) Case for flow quenching: With deadline awareness, one flow can be quenched All other flows make their deadline 40

41 D3: Deadline driven delivery Main idea: make the network aware of flow deadlines Prioritize flows based on deadlines Key insight: Rate required to satisfy a flow deadline: r = s / d s: flow size d: deadline 41

42 How it works (1) Application exposes (s,d) Desired rate r = s / d Routers allocate rates (α) based on traffic load Sending rate for next RTT : sr = min(α1,α2) s: flow size d: deadline RRQ: rate request α: allocated rate One of the packets contains and updated RRQ based on the remaining flow size and the deadline 42

43 How it works (2) Application exposes (s,d) Desired rate r = s / d Routers allocate rates (α) based on traffic load Sending rate for next RTT : sr = min(α1,α2) s: flow size d: deadline RRQ: rate request α: allocated rate One of the packets contains and updated RRQ based on the remaining flow size and the deadline 43

44 Flow microbenchmarks Experiment: multiple workers sending traffic to aggregator (within a single rack) How many workers can be supported while satisfying deadlines? D3 can support roughly twice as many workers than RCP (~DCTCP) while satisfying application deadlines 44

45 Flow quenching Terminate useless flows when: Desired rate exceeds link capacity Deadline has expired 45

46 D3 Summary Traditional TCP flow-sharing leads flows missing deadlines D3 allocated rates at switches based on the deadlines of the flows D3 can support many more flows with deadlines than TCP 46

47 Limitations of D3 Needs router support User-space PC-based implementation for paper Violates end-to-end argument Greedy rate allocation leads to priority inversion and later to missed deadlines 47

48 Deadline-aware Datacenter TCP 2 (D TCP) Key idea: Vary sending rate based on both deadline and extent of congestion Built on top of DCTCP Per-flow state at end-hosts (not routers/switches) Reactive: senders react to congestion No knowledge of other flows 48

49 D2TCP Congestion Avoidance (1) Remember: DTCP congestion control TCP window: w = w x (1-a/2) Running average of marked packets: a = (1-g)*a + g*f (a ~ 0: low congestion, a ~ 1: high congestion) D2TCP extends DTCP to integrate deadline Deadline factor d: larger d implies closer deadline Penalty function: p = ad TCP window: w = w * (1 p/2) if p > 0 w=w+1 if p = 0 Note: a = 0 => p = 0 => w = w +1 (similar to TCP) a = 1 => p = 1 => w = w/2 (similar to TCP) 49

50 D2TCP Congestion Avoidance (2) d < 1 for far deadline flows => p large => shrink window d > 1 for near deadline flows => p small => retain window d = 1 for long lived flows => DTCP behavior Near deadline flows back off less while far deadline flows back off mor 50

51 How to determine d? d = Tc / D Tc: time needed for a flow to complete under deadlineagnostic congestion behavior (based on the current window w) D: remaining time until deadline expires Flow is on track: TC D => d 1 Flow is about to miss deadline: TC > D => d > 1 Flow is ahead of deadline TC < D => d < 1 51

52 D2TCP versus DCTCP Flow sizes: 150MB, 220MB, 350MB, 500MB Flow deadlines: 1000ms, 1500ms, 2500ms, 4000ms DTCP: all flows get same b/w irrespective of deadline D2TCP: Near deadline flows get more bandwidth 52

53 Multipath TCP 53

54 Modern datacenters provide many parallel paths Traditional topologies are treebased Poor performance Not fault-tolerant Shift towards multipath topologies FatTree (Portland, VL2) BCube 54

55 Modern datacenters provide many parallel paths Traditional topologies are treebased Poor performance Not fault-tolerant Shift towards multipath topologies FatTree (Portland, VL2) BCube How to effectively use all the bandwidth in a multipath topology? 55

56 Equal-cost multipath routing (ECMP) ECMP Multipath routing strategy that splits traffic over multiple paths for loadbalancing Why not just round-robin packets? Reordering: triple TCP ACKs, TCP fast retransmit, reduced TCP window Different RTT per path Different MTUs per path Path selection via hashing #buckets = #outgoing links Hash network information (src address/port, dst address/port, protocol type) to select outgoing link: preserves flow affinity 56

57 ECMP in Fat Tree Network Hash flows to different paths 57

58 ECMP collision Multiple flows may hash to same path 58

59 Limitations of ECMP ECMP may not utilize the links uniformly Many flows hashed to same link ECMP is static No knowledge about current traffic on a link 59

60 Limitations of ECMP (2) 8192 node fat tree toplogy Random traffic pattern: Flow: every host chooses a random destination No destination is used twice Figure: flows ranked according to their throughput 60

61 Limitations of ECMP (2) 8192 node fat tree toplogy Random traffic pattern: Flow: every host chooses a random destination Vast majority of flows does not achieve No destination is used twice more than 50% of the throughput possible Figure: flows ranked according to their throughput 61

62 Multipath TCP (MPTCP) Instead of using one path for each flow, many random paths Don't worry about collisions Just don't send (much) traffic on colliding path 62

63 Multipath TCP (2) MPTCP establishes mutiple subflows on different paths With many random paths, MPTCP will find at least one good unloaded path and moves most of its traffic on that path Drop-in replacement Improves... Bandwidth (aggregation) Fairness Robustness 63

64 MPTCP in Fat Tree Network Vast majority of flows using MPTCP achieve % throughput 64

65 MTCP: Connection Management 65

66 MTCP: Connection Management 66

67 MTCP: Connection Management 67

68 MTCP: Connection Management 68

69 MTCP: Connection Management client learns about additional interfaces of server 69

70 MTCP: Connection Management Works in data centers, problem when using MPTCP across the Internet: 6% of access networks 70remove unknown options

71 MTCP: Connection Management 71

72 MTCP: Connection Management 72

73 MTCP: Connection Management Subflows can be between different interfaces or between the same pair of IP addresses but different ports 73

74 MTCP: Connection Management 74

75 MTCP: Connection Management 75

76 MTCP: Connection Management MPTCP relies on ECMP to hash different subflows to different paths 76

77 MTCP: Sending Data MPTCP stripes TCP across the subflows Additional TCP options allow the receiver to reconstruct the received data in the original order 77

78 MTCP: Sending Data ACK1 ACK3 ACK2 ACK4 MPTCP stripes TCP across the subflows When used over the Internet, middleboxes may drop ACKs of78unseen data packets

79 MPTCP Congestion Control Each path runs its own congestion control, to detect and respond to the congestion it sees But link congestion control parameters, so as to move traffic away from the congested paths 79

80 How many subflows are needed? 80

81 Next week Flow control Infiniband Converged enhanced Ethernet 81