Utilizing Datacenter Networks: Centralized or Distributed Solutions?

Transcription

1 Utilizing Datacenter Networks: Centralized or Distributed Solutions? Costin Raiciu Department of Computer Science University Politehnica of Bucharest

2 We ve gotten used to great applications

3 Enabling Such Apps is Hard Apps Process huge amounts of data Are fast Are reliable One machine is not enough Limited reliability, speed Super computers are expensive Use many commodity machines instead

4 Data Centers Rule the World Cloud computing Economies of scale: networks of tens of thousands of hosts Datacenter apps support web search, online stores Web search, GFS, BigTable, DryadLINQ, MapReduce Dense traffic patterns Intra datacenter traffic is increasing in volume

5 Flexibility is Important in Data Centers Apps distributed across thousands of machines. Flexibility want any machine to be able to play any role.

6 Traditional Data Center Topology Internet Core Switch 10Gbps 10Gbps 1Gbps Aggrega5on Switches Top of Rack Switches Racks of servers

7 Fat Tree Topology [Fares et al., 2008; Clos, 1953] K=4 Aggrega5on Switches 1Gbps 1Gbps K Pods with K Switches each Racks of servers

8 VL2 Topology [Greenberg et al, 2009, Clos topology] 10Gbps 10Gbps 20 hosts

9 BCube Topology [Guo et al, 2009] BCube (4,1)

10 How Do We Route Packets in Data Centers? Traditional Routing Spanning Tree Protocol - kills all redundancy Instead datacenters use one of the following techniques: Multiple VLANs OSPF TRILL (new IETF standard)

11 How Do We Use this Capacity? Need to distribute flows across available paths. Basic solution: Random Load Balancing. Use Equal-Cost Multipath (ECMP) routing (OSPF, TRILL) Hash to a path at random. Sources randomly pick a VLANs. In practice sources have multiple interfaces pick a random source address for the flow

12 Collisions 1Gbps 1Gbps Racks of servers

13 Single-path TCP collisions reduce throughput

14 How bad are collisions? Capacity wasted (worst case): FatTree 60% BCube 50% VL2 25%

15 How do we address this problem? I will discuss two solutions Flow scheduling Multipath TCP

16 Flow Scheduling Hedera Fares et al. NSDI 2010

17 Solving Collisions with Flow Scheduling Centralized Controller 2. Compute flow demands 1. Pull stats, detect large flows 3. Compute placement 4. Place flows 1Gbps OF OF OF OF 1Gbps OF OF OF OF OF OF OF OF OF OF OF OF OF OF OF OF Racks of servers

18 Hedera Main Idea Schedule elephant flows They carry most of the bytes ECMP deals with short flows

19 Detecting Elephants Pull edge switches for byte counts Flows exceeding 100Mb/s are large What if only short flows? ECMP should be good enough

20 Demand Estimation Current flow rates are a poor indicator of flow demand Network could be the bottleneck Hedera s approach: what would this flow get if the network was not a bottleneck?

21 Demand estimation: simple example 1Gb/s 500Mb/s 500Mb/s General Approach: Iterative algorithm

22 Allocating Flows to Paths Multi-Commodity Flow Problem Single path forwarding Expressed as Binary Integer Programming NP-Complete Solvers give exact solution but are impractical for large networks

23 Approximating Multi-Commodity Flow Global First Fit Linearly search all paths until one that can accommodate the traffic is found Flows placed upon detection, are not moved Simulated Annealing Probabilistic search for good solutions that maximize bisection bandwidth

24 Fault Tolerance Scheduler failure all soft state, just fall back to ECMP Link, switch failures Portland notifies the scheduler

25 Does it work?

26 Hedera: One Flow, One Path Centralized Can it scale to really large datacenters? Needs a very tight control loop How often does it need to run to achieve these benefits? Strong assumption: traffic is always bottlenecked by the network What about app-bound traffic, e.g disk reads/writes?

27 Hedera: One Flow, One Path Centralized Can it scale to really large datacenters? MAYBE Needs a very tight control loop FIXABLE This is the wrong place to start How often does it need to run to achieve these benefits? Strong assumption: Only Hosts Know traffic is always bottlenecked by the network What about app-bound traffic, e.g disk reads/writes?

28 Multipath topologies need multipath transport Multipath transport enables better topologies

29 Collision

30

31

32 Not fair

33 Not fair

34

35

36 No matter how you do it, mapping each flow to a path is the wrong goal

37 Instead, we should pool capacity from different links

38 Instead, we should pool capacity from different links

39 Instead, we should pool capacity from different links

40 Instead, we should pool capacity from different links

41 Multipath Transport

42 Multipath Transport can pool datacenter networks Instead of using one path for each flow, use many random paths Don t worry about collisions. Just don t send (much) traffic on colliding paths

43 Multipath TCP Primer [IETF MPTCP WG] MPTCP is a drop in replacement for TCP Works with unmodified applications Over the existing network

44 MPTCP Operation SYN MP_CAPABLE X

45 MPTCP Operation SYN/ACK MP_CAPABLE Y

46 MPTCP Operation STATE 1 CWND Snd.SEQNO Rcv.SEQNO

47 MPTCP Operation STATE 1 CWND Snd.SEQNO Rcv.SEQNO SYN JOIN Y

48 MPTCP Operation STATE 1 CWND Snd.SEQNO Rcv.SEQNO SYN/ACK JOIN X

49 MPTCP Operation STATE 1 CWND Snd.SEQNO Rcv.SEQNO STATE 2 CWND Snd.SEQNO Rcv.SEQNO

50 MPTCP Operation SEQ 1000 options DSEQ DATA STATE 1 CWND Snd.SEQNO Rcv.SEQNO STATE 2 CWND Snd.SEQNO Rcv.SEQNO

51 MPTCP Operation SEQ 1000 options DSEQ DATA STATE 1 CWND Snd.SEQNO Rcv.SEQNO STATE 2 CWND Snd.SEQNO Rcv.SEQNO

52 MPTCP Operation SEQ 1000 options DSEQ DATA STATE 1 CWND Snd.SEQNO Rcv.SEQNO SEQ 5000 options DSEQ DATA STATE 2 CWND Snd.SEQNO Rcv.SEQNO

53 MPTCP Operation SEQ 1000 options DSEQ DATA STATE 1 CWND Snd.SEQNO Rcv.SEQNO SEQ 5000 options DSEQ DATA STATE 2 CWND Snd.SEQNO Rcv.SEQNO

54 MPTCP Operation SEQ 1000 options DSEQ DATA STATE 1 CWND Snd.SEQNO Rcv.SEQNO SEQ 5000 options DSEQ DATA STATE 2 CWND Snd.SEQNO Rcv.SEQNO

55 MPTCP Operation SEQ 1000 options DSEQ DATA STATE 1 CWND Snd.SEQNO Rcv.SEQNO SEQ 5000 options DSEQ DATA STATE 2 CWND Snd.SEQNO Rcv.SEQNO

56 MPTCP Operation ACK 2000 STATE 1 CWND Snd.SEQNO Rcv.SEQNO STATE 2 CWND Snd.SEQNO Rcv.SEQNO

57 MPTCP Operation SEQ 2000 options DSEQ DATA STATE 1 CWND Snd.SEQNO Rcv.SEQNO STATE 2 CWND Snd.SEQNO Rcv.SEQNO

58 Multipath TCP: Congestion Control [NSDI, 2011]

59 MPTCP better utilizes the FatTree network

60 MPTCP on EC2 Amazon EC2: infrastructure as a service We can borrow virtual machines by the hour These run in Amazon data centers worldwide We can boot our own kernel A few availability zones have multipath topologies 2-8 paths available between hosts not on the same machine or in the same rack Available via ECMP

61 Amazon EC2 Experiment 40 medium CPU instances running MPTCP For 12 hours, we sequentially ran all-to-all iperf cycling through: TCP MPTCP (2 and 4 subflows)

62 MPTCP improves performance on EC2

63 Where do MPTCP s benefits come from?

64 Allocating Flows to Paths Multi-Commodity Flow Problem Single path forwarding Expressed as Binary Integer Programming NP-Complete Solvers give exact solution but are impractical for large networks

65 Allocating Flows to Paths Multi-Commodity Flow Problem Single path forwarding Expressed as Binary Integer Programming NP-Complete Solvers give exact solution but are impractical for large networks Multipath forwarding Expressed as Linear Programming problem Solvable in polynomial time

66 How many subflows are needed? How does the topology affect results? How does the traffic matrix affect results?

67 At most 8 subflows are needed Total Throughput TCP

68 MPTCP improves fairness in VL2 topologies VL2 Fairness is important: Jobs finish when the slowest worker finishes

69 MPTCP improves throughput and fairness in BCube BCube Single path TCP optimum

70 Oversubscribed Topologies To saturate full bisectional bandwidth: There must be no traffic locality All hosts must send at the same time Host links must not be bottlenecks It makes sense to under-provision the network core This is what happens in practice Does MPTCP still provide benefits?

71 Performance improvements depend on traffic matrix Underloaded Sweet Spot Overloaded Increase Load

72 MPTCP vs. Centralized Scheduling

73 MPTCP vs Hedera First Fit Centralized Scheduling MPTCP Infinite Scheduling Interval

74 Centralized Scheduling: Setting the Threshold Throughput 1Gbps Hope 17% worse than multipath TCP 100Mbps App Limited

75 Centralized Scheduling: Setting the Threshold Throughput 1Gbps 21% worse than multipath TCP 100Mbps App Limited Hope

76 Centralized Scheduling: Setting the Threshold Throughput 1Gbps 500Mbps 100Mbps 51% 17% 45% 21%

77 MPTCP vs. Hedera MPTCP HEDERA Implementation Distributed Centralized Network changes No Yes, upgrade all switches to OF Hardware needed No Centralized Scheduler Software changes Yes host stack No Scope Schedules more flows Large flows only Convergence Time Scale Invariant, RTTs Tight Control Loop Limits Scalability Fairness Fair Less fair

78 What is an optimal datacenter topology for multipath transport?

79 In single homed topologies: Hosts links are often bottlenecks ToR switch failures wipe out tens of hosts for days Multi-homing servers is the obvious way forward

80 Fat Tree Topology

81 Fat Tree Topology Upper Pod Switch ToR Switch Servers

82 Dual Homed Fat Tree Topology Upper Pod Switch ToR Switch Servers

83 Is DHFT any better than Fat Tree? Not for traffic matrices that fully utilize the core Let s examine random traffic patterns

84 DHFT provides significant improvements when core is not overloaded Core Underloaded Core Overloaded

85 Summary One flow, one path thinking has constrained datacenter design Collisions, unfairness, limited utilization Fixing these is possible, but does not address the bigger issue Multipath transport enables resource pooling in datacenter networks: Improves throughput Improves fairness Improves robustness One flow, many paths frees designers to consider topologies that offer improved performance for similar cost

86 Backup Slides

87 Effect of MPTCP on short flows Flow sizes from VL2 dataset MPTCP enabled for long flows only (timer) Oversubscribed Fat Tree topology Results: TCP/ECMP Completion time: 79ms Core Utilization: 25% MPTCP 97ms 65%

88 Effect of Locality in the Dual Homed Fat Tree

89 Overloaded Fat Tree: better fairness with Multipath TCP

90

91

92 VL2 Topology [Greenberg et al, 2009, Clos topology] 10Gbps 10Gbps 20 hosts

93 BCube Topology [Guo et al, 2009] BCube (4,1)