T Computer Networks II Data center networks

Transcription

1 T Computer Networks II Data center networks Matti Siekkinen (Sources: S. Kandula et al.: The Nature of Datacenter: measurements & analysis, A. Greenberg: Networking The Cloud, M. Alizadeh et al: Data Center TCP(DCTCP), C. Kim: VL2: A Scalable and Flexible Data Center Network )

2 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 2

3 What is a data center? Contains servers and data Has a network Connect servers together Runs applications and services Internal and external Centrally managed Operated in controlled environment Can have very different sizes SME datacenter vs. Google 3

4 Applications and services External facing Search, Mail, Shopping Carts, Internal to the company/institution E.g. ERP (Financial, HR, ) Services internal to the data center Those necessary for the data center to work E.g. network operations (DNS, NFS, DHCP), backup Building blocks for external facing apps MapReduce, GFS, BigTable (Google), Dynamo (Amazon), Hadoop (Yahoo!), Dryad (Microsoft) Often distributed 4

5 Multi-tier architecture E.g. 3-tiers Front end servers Handles static requests Handles dynamic content Handles database transactions Applications servers Backend database servers Advantages Performance & scalability Security 5

6 What does it look like? Servers in racks Contains commodity servers (blades) Connected to Top-Of-Rack switch Aggregated traffic to next level From Microsoft Chicago data center Modular data centers Shipping containers full of racks Inside a container 6

7 Large data center requires a lot of... Power Cooling Some statistics Google: 450,000 servers in 2006, estimated over a million by now Microsoft is doubling the number of servers every 14 months Photos from Microsoft Chicago data center 7

8 Cloud computing Cloud computing Abstract underlying resouces from the service provided Abstraction on different levels: IaaS, PaaS, SaaS Virtualization enables cloud s many properties Elastic resource allocation Of course limited by number of physical servers One users resources limited by SLA, not by single piece of hardware Efficient use of resources Don t need to run all servers full speed all the time Client s VMs can run on any physical server 8

9 Data center vs. Cloud Data center is physical Physical infrastructure that runs services Cloud is not physical Offers some service(s) Physical infrastructure is virtualized away Cloud usually needs to be hosted in a data center Depends on scale Data center does not need to host cloud services Private cloud vs. own data center Not the same thing 9

10 Cloud DC vs. Enterprise DC Traditional enterprise DC: IT staff cost dominates Human to server ratio: 1:100 Less automation in management Scale up: a few high priced servers Cost borne by the enterprise Utilization is not critical Cloud service DC: other costs Human to server ratio: 1:1000 Automation is more crucial Distributed workload, spread out on lots of commodity servers High upfront cost amortized over time and use Pay per use for customers Utilization is critical 10

11 What is a data center network (DCN)? Enables communication within DC Among the different servers In practice HW: switches, routers, and cabling SW: communication protocols (layers 2-4) Principles evolved from enterprise networks 11

12 What is a data center network (DCN)? Both layers 2 (link) and 3 (network) present Not only L3 routers but also L2 switches Layer 2 subnets connected with layer 3 Layer 4 (transport) needed similar to any packet networks Note: does not have to be TCP/IP! Not part of routed Internet Cannot resolve DC server s address directly from Internet, only front end servers But often is TCP/IP WWW phone..." SMTP HTTP SIP..." TCP UDP " IP" Eth PPP WiFi 3GPP " copper fiber radio OFDM FHSS..." 12

13 What makes DCNs special? Just plug all servers to an edge router and be done with it? Several issues with this approach Scaling up capacity Lots of servers need lots of switch ports E.g.: State of the art Cisco Nexus 7000 modular data center switch (L2 and L3) supports max /10GE ports Switch capacity and price Prices goes up with nb of ports E.g.: List price for 768 ports with 10GE modules somewhere beyond $1M Buying lots of commodity switches is an attractive option Potentially majority of traffic stays within DC Server to server 13

14 What makes DCNs special? (cont.) Requirements different from Internet applications Large amounts of bandwidth Very, very short delays Still, often Internet protocols (TCP/IP) used Management requirements Incremental expansion Should be able to withstand server failures, link outages, server rack failures Under failures, performance should degrade gracefully Requirements due to expenses Cost-effectiveness; high throughput per dollar Power efficiency DCN topology and equipment matter a lot 14

15 Data Center Costs Amortized Cost* Component Sub-Components ~45% Servers CPU, memory, disk ~25% Power infrastructure UPS, cooling, power distribution ~15% Power draw Electrical utility costs ~15% Network Switches, links, transit Total cost varies Upwards of $1/4 B for mega data center Server costs dominate Network costs also significant Network should allow high utilization of servers *3 yr amortization for servers, 15 yr for infrastructure; 5% cost of money Source: Greenberg et al. The Cost of a Cloud: Research Problems in Data Center Networks. Sigcomm CCR

16 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 16

17 Switch vs. router: What s the difference? Switch is layer 2 device Does not understand IP protocol Does not run any routing protocol Router is layer 3 device Speaks IP protocol Runs routing protocols to determine shortest paths OSPF, RIP, etc. Terminology not so clear L2/3, i.e. multi-layer switches 17

18 Switch vs. router: Difference in basic functioning Router Forwards packets based on destination IP address Prefix lookup against routing tables Routing tables built and maintained by routing algorithms and protocols Protocols exchange information about paths to known destinations Algorithms compute shortest paths based on this information Broadcast sending usually not allowed Switch Forwards frames (packets) based on destination MAC address Uses switch table Equivalent to routing table in router Broadcast sending is common How is switch table built and maintained since there is no routing protocol? 18

19 Switch is self learning When frame is received from one port Switch learns that sender is behind that port Switch adds that information to switch table Soft state: forget after a while If destination not (yet) known Flood to all other ports Flooding can lead to forwarding loops Switches connected in cyclic manner These loops can create broadcast storms Spanning tree protocol (STP) used to avoid loops Generates loop-free topology Avoid using some ports when flooding Rapid Spanning Tree Protocol (RSTP) Faster convergence after a topology change CC Hub Port 2 Port 2 AA 21 AA 21 Port 1 Port 1 Hub DD AA BB <Src=AA, Dest=DD> And so on No TTL in L2 headers! 19

20 Layer 2 vs. Layer 3 in DCN Management L2 close to plug-and-play L3 usually requires some manual configuration (subnet mask, DHCP) Scalability and performance L2 broadcasting and STP scale poorly L2 forwarding less scalable than L3 fwding L2 based on flat MAC addresses L3 based on hierarchical IP addresses (prefix lookup) L2 has no such load balancing over multiple paths as L3 L2 loops may still happen in practice, even with STP 20

21 Layer 2 vs. Layer 3 in DCN Flexibility VM migration may require change of IP address in L3 network Need to conform to subnet address L2 network allows any IP address for any server Some reasons may prevent using pure L3 design Some servers may need L2 adjacency Servers performing the same functions (load balancing, redundancy) Heartbeat or application packets may not be routable Dual homed servers may need to be on same L2 domain Connected to two different access switches Some configurations require both primary and secondary to be in same L2 domain 21

22 VLAN VLAN = Virtual Local Area Network Some servers may need to belong to same L2 broadcast domains See previous slide VLANs overcome limitations of physical topology Run out of switch ports VLAN allows flexible growth while maintaining layer 2 adjacency L2 domain across routers VLAN can be port-based 22

23 Port-based VLAN router Traffic isolation Frames to/from ports 1-8 can only reach ports 1-8 Can also define VLAN based on MAC addresses of endpoints, rather than switch port Dynamic membership Ports can be dynamically assigned among VLANs Forwarding between VLANS done via routing 1 2 VLAN1 (ports 1-8) VLAN2 (ports 9-15) 23

24 VLANs spanning multiple switches VLAN1 (ports 1-8) VLAN2 (ports 9-15) Ports 2,3,5 belong to VLAN1 Ports 4,6,7,8 belong to VLAN2 VLANs can span over multiple switches Also over different routed subnets Routers in between 24

25 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 25

26 Design Alternatives for DCN Two high level choices for Interconnections: Specialized hardware and communication protocols E.g. Infiniband seems common Can provide high bandwidth & extremely low latency Custom hardware takes care of some reliability tasks Relatively low power physical layer Expensive Not natively compatible with TCP/IP applications Commodity (1/10 Gb) Ethernet switches and routers Compatible Cheaper We focus on this 26

27 Conventional DCN architecture Internet Topology: Two- or threelevel trees of switches or routers Multipath routing High bandwidth by appropriate interconnection of many commodity switches Redundancy Layer-3 router Layer-2/3 aggregation switches Layer-2 Top- Of-Rack access switches Servers 27

28 Issues with conventional architecture Bandwidth oversubscription Total bandwidth at core/aggregate level less than summed up bandwidth at access level Limited server to server capacity Application designers need to be aware of limitations No performance isolation VLANs typically provide reachability isolation only One server (service) sending/receiving too much traffic hurts all servers sharing its subtree There are more 28

29 One solution to oversubscription FAT Tree topology with special look-up scheme Add more commodity switches Carefully designed topology All ports have same capacity as servers Enables Full bisection bandwidth Lower cost because all switch ports have same capacity Drawbacks Need customized switches Special two level look-up scheme to distribute traffic Lot of cabling Core Switches Aggregation Switches Edge Switches M. Al-Fares et al. Commodity Data Center Network Architecture. In SIGCOMM FAT Tree 29

30 One solution to performance isolation: VLB Random flow spreading with Valiant Load Balancing (VLB) Similar FAT Tree topology with commodity switches Every flow bounced off a random intermediate switch Provably hotspot free for any admissible traffic matrix No need to modify switches (std forwarding) Relies on ECMP and clever addressing Requires some changes to servers D ports D/2 switches... Intermediate node switches in VLB D/2 ports D/2 ports 20 ports 10G D switches... Top Of Rack switch [D 2 /4] * 20 Servers Aggregation switches A. Greenberg et al. VL2: A Scalable and Flexible Data Center Network. In SIGCOMM,

31 DCN architectures in research Lots of alternative proposed architectures in recent years Goals Overcome limitations of typical architectures of today Use commodity standard equipment VL2 & Monsoon & CamCube (MSR) Portland (UCSD) Dcell & Bcube (MSR, Tsinghua, UCLA) 31

32 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 32

33 TCP in the Data Center TCP rules as transport inside DC 99.9% of traffic DCNs different environment for TCP compared to normal Internet e2e transport Very short delays Specific application workloads How well does TCP work in DCNs? Several problems 33

34 Partition/Aggregate Application Structure Internet Deadline = 250ms The foundation for many largescale web applications Web search, Social network composition, Ad selection, etc. Time is money -> strict deadlines Missed deadline means lower quality result Deadline = 50ms Deadline = 10ms Worker Nodes 34

35 Partition/Aggregate Application Structure Internet Deadline = 250ms Deadlines in lower hierarchy must meet with all-up deadline Iterative requests common 1-4 iterations typical Workers have tight deadlines 99.9 th percentiles of delay matter for companies 1 out of 1000 responses Can potentially impact large number of customers Deadline = 50ms Deadline = 10ms Worker Nodes 35

36 Workloads Query-response traffic Partition/Aggregate Part of the mice flows Background traffic Short messages [50KB-1MB] Coordination, control state Part of the mice flows Large flows [1MB-50MB] Updating data on each server The elephant flows Problem: All this traffic goes through same switches Requirements are conflicting Requires minimal delay Requires high throughput 36

37 Traffic patterns from one cluster of Microsoft s DCN Traffic exchanged between server pairs in 10s period Servers within a rack are adjacent on axis Work-Seeks-Bandwidth (W-S-B) Small squares around diagonal Scatter-Gather (S-G) Horizontal and vertical lines ln(bytes) exchange d per 10s

38 Traffic patterns from one cluster of Microsoft s DCN (cont.) Work-seeks-bandwidth Need to make efforts to place jobs under the same ToR Scatter-gather-patterns Server pushes/pulls data to/from many servers across the cluster Distributed query processing: map, reduce Data divided into small parts Each servers works on particular part Answers aggregated Need for inter-tor communication Computation constrained by the network

39 DCN characteristics Network characteristics Large aggregate bandwidths Very short round trip time delays (<1ms) Typical switches Use large numbers of commodity switches Typically commodity switch has shared memory Common memory pool for all ports Why not separated memory spaces? Cost issue for commodity switches 39

40 Resulting problems with TCP in DCN Incast Queue Buildup Buffer Pressure 40

41 Problems: Incast Worker 1 Worker 2 Synchronized mice collide. Caused by Partition/ Aggregate Aggregator Worker 3 Worker 4 41

42 Incast What happens next? TCP timeout Default minimum values of timeout ms depending on OS Why is that a major problem? Several order of magnitude longer than RTT -> huge penalty Fail to meet deadlines in all levels 42

43 Problems: Incast Worker 1 Worker 2 Aggregator Worker 3 RTO min = 300 ms Worker 4 A TCP timeout 43

44 Problems: Queue Buildup Remember the different workloads Small mice flows Large elephant flows Large flows can eat up the shared buffer space Same outgoing port Result is similar than with incast 44

45 Problems: Queue Buildup Sender 1 Big flows build up queues Increased latency for short flows Packet loss Receiver Sender 2 45

46 Problems: Buffer pressure Kind of generalization of the previous problem Increased queuing delay and packet loss due to long flows traversing other ports Shared memory pool Packets incoming and outgoing different ports still eat up each common buffer space 46

47 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 47

48 Data Center Transport Requirements 1. High Burst Tolerance Cope with the Incast problem 2. Low Latency Short flows, queries 3. High Throughput Continuous data updates, large file transfers We want to achieve all three at the same time 48

49 Exploring the solution space Proposal Throughput Burst tolerance Latency (Incast) Deep switch buffers Can achieve high throughput Tolerates large bursts Queuing delays increase latency Shallow buffers Can hurt throughput of Cannot tolerate bursts well Avoids long queuing delay elephant flows Jittering :/ No major impact Prevents Incast Increases median latency Shorter RTO min :/ No major impact Helps recover faster Doesn t help queue buildup Nw assisted congestion ctrl (ECN style) High throughput with high utilization Helps in most cases Problem if only 1 pkt is too much Reacts early to queue buildup 49

50 Jittering MLA Query Completion Time (ms) Jittering on Requests are jittered over 10ms window Jittering off Add random delay before responding Desynchronize the responding sources to avoid buffer overflow Jittering trades off median against high percentiles 50

51 Exploring the solution space Proposal Throughput Burst tolerance Latency (Incast) Deep switch buffers Can achieve high throughput Tolerates large bursts Queuing delays increase latency Shallow buffers Can hurt throughput of Cannot tolerate bursts well Avoids long queuing delay elephant flows Jittering :/ No major impact Prevents Incast Increases median latency Shorter RTO min :/ No Improves major impact Helps recover Doesn t help throughput faster queue buildup Nw assisted congestion ctrl (ECN style) High throughput with high utilization Helps in most cases Problem if only 1 pkt is too much Reacts early to queue buildup 51

52 Review: TCP with ECN Sender 1 ECN = Explicit Congestion Notification Q: How do TCP senders react? A: Cut sending rate by half ECN Mark (1 bit) Receiver Sender 2 52

53 DCTCP: Two key ideas 1. React in proportion to the extent of congestion, not just its presence Reduces variance in sending rates, lowering queuing requirements ECN Marks TCP DCTCP Cut window by 50% Cut window by 40% Cut window by 50% Cut window by 5% 2. Mark based on instantaneous queue length Fast feedback to better deal with bursts Q: Why normal TCP with ECN does not behave like DCTCP? A: Fairness 53

54 Data Center TCP Algorithm Switch side: Mark packets when Queue Length > K Sender side: Maintain moving average of fraction of packets marked (α). In each RTT: Mark K Don t mark Adaptive window decreases: Note: decrease factor between 1 and 2. 54

55 DCTCP in Action (Kbytes) 55

56 Why does DCTCP work? High Burst Tolerance Aggressive marking sources react before packets are dropped Large buffer headroom bursts fit Low Latency Small buffer occupancies low queuing delay High Throughput ECN averaging smooth rate adjustments, low variance Leads to high utilization 56

57 Completely solves the Incast problem? Remember Incast: large number of synchronized small flows hit the same queue Depends on the number of small flows Does not help if so high that even 1 packet from each flow is sufficient to overwhelm the buffer on a synchronized burst No congestion control helps Only solution is to somehow schedule responses (e.g. jittering) Helps if each flow has several packets to transmit Windows build up over multiple RTTs Bursts in subsequent RTTs would lead to packet drops DCTCP sources receive enough ECN feedback to prevent buffer overflows 57

58 Comparing TCP and DCTCP Emulate traffic within 1 Rack of Bing cluster 45 1G servers, 10G server for external traffic Generate query, and background traffic Flow sizes and arrival times follow distributions seen in Bing Metric: Flow completion time for queries and background flows RTOmin = 10ms for both TCP & DCTCP More than fair comparison 58

59 Comparing TCP and DCTCP (cont.) Background Flows Query Flows 59

60 Comparing TCP and DCTCP (cont.) Background Flows Query Flows Low latency for short flows 60

61 Comparing TCP and DCTCP (cont.) Background Flows Query Flows High throughput for long flows 61

62 Comparing TCP and DCTCP (cont.) Background Flows Query Flows High burst tolerance for query flows 62

63 DCTCP summary DCTCP Handles bursts well Keeps queuing delays low Achieves high throughput Features: Simple change to TCP and a single switch parameter Based on existing mechanisms 63

64 TCP for DCN research Data transport in DCN has received attention recently Several solutions proposed just this year Deadline-Aware Datacenter TCP (D 2 TCP) (Purdue, Google) DeTail (cross layer solution) (Berkeley, Facebook) 64

65 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 65

66 Wrapping up Data center networks provide specific networking challenges Potentially huge scale Different requirements than with traditional Internet applications Recently a lot of research activity New proposed architectures and protocols Big deal to companies with mega-scale data centers: $$ Popularity of cloud computing accelerates this evolution 66

67 Want to know more? 1. M. Arregoces and M. Portolani. Data Center Fundamentals. Cisco Press, Kandula, S., Sengupta, S., Greenberg, A., Patel, P., and Chaiken, R The nature of data center traffic: measurements & analysis. In Proceedings of IMC Vasudevan, V., Phanishayee, A., Shah, H., Krevat, E., Andersen, D. G., Ganger, G. R., Gibson, G. A., and Mueller, B Safe and effective fine-grained TCP retransmissions for datacenter communication. In Proceedings of the ACM SIGCOMM A. Greenberg et al. VL2: A Scalable and Flexible Data Center Network. In SIGCOMM, C. Guo et al. DCell: A Scalable and Fault Tolerant Network Structure for Data Centers. In SIGCOMM, M. Al-Fares, A. Loukissas, and A. Vahdat. A Scalable, Commodity Data Center Network Architecture. In Proceedings of the ACM SIGCOMM Niranjan Mysore, R., Pamboris, A., Farrington, N., Huang, N., Miri, P., Radhakrishnan, S., Subramanya, V., and Vahdat, A PortLand: a scalable fault-tolerant layer 2 data center network fabric. In Proceedings of the ACM SIGCOMM Joseph, D. A., Tavakoli, A., and Stoica, I A policy-aware switching layer for data centers. In Proceedings of the ACM SIGCOMM Guo, C., Lu, G., Li, D., Wu, H., Zhang, X., Shi, Y., Tian, C., Zhang, Y., and Lu, S BCube: a high performance, server-centric network architecture for modular data centers. In Proceedings of the ACM SIGCOMM Abu-Libdeh, H., Costa, P., Rowstron, A., O'Shea, G., and Donnelly, A Symbiotic routing in future data centers. In Proceedings of the ACM SIGCOMM Check SIGCOMM 2012 program as well 67