T Computer Networks II Data center networks

Transcription

1 T Computer Networks II Data center networks Matti Siekkinen (Sources: S. Kandula et al.: The Nature of Datacenter: measurements & analysis, A. Greenberg: Networking The Cloud, M. Alizadeh et al: Data Center TCP(DCTCP), C. Kim: VL2: A Scalable and Flexible Data Center Network )

2 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 2

3 What is a data center? Contains servers and data Has a network Connect servers together Runs applications and services Internal and external Centrally managed Operated in controlled environment Can have very different sizes SME datacenter vs. Google 3

4 Applications and services External facing Search, Mail, Shopping Carts, Internal to the company/institution E.g. ERP (Financial, HR, ) Services internal to the data center Those necessary for the data center to work E.g. network operations (DNS, DHCP), backup Building blocks for external facing apps MapReduce, Colossus, GFS, Spanner, BigTable, Dynamo, Hadoop, Dryad Often distributed 4

5 Multi-tier architecture E.g. 3-tiers Front end servers Handles static requests Handles dynamic content Handles database transactions Applications servers Backend database servers Advantages Performance & scalability Security 5

6 What does it look like? Servers in racks Contains commodity servers (blades) Connected to Top-Of-Rack switch Aggregated traffic to next level From Microsoft Chicago data center Modular data centers Shipping containers full of racks Inside a container 6

7 Large data center requires a lot of... Power Cooling Some statistics Microsoft was running about 1 million servers in 2013 Google probably quite a bit more Photos from Microsoft Chicago data center 7

8 Cloud computing Cloud computing Abstract underlying resources from the service provided Abstraction on different levels: IaaS, PaaS, SaaS Virtualization enables cloud s many properties Run many Virtual Machines (VM) on single physical machine Or a single image on a cluster of machines Elastic resource allocation Of course limited by number of physical servers One users resources limited by SLA, not by the amount of hardware Efficient use of resources Don t need to run all servers at full steam all the time Client s VMs can run on any physical server 8

9 Data center vs. Cloud Data center is physical Physical infrastructure that runs services Cloud is an abstraction of the physical Offers some service(s) Physical infrastructure is virtualized Large clouds are usually hosted in data centers Depends on scale Data center does not need to host a cloud 9

10 Different kinds of data centers Traditional enterprise DC: IT staff cost dominates Human to server ratio: 1:100 Less automation in management A few high priced servers Cost borne by the enterprise Utilization is not critical Cloud service DC: other costs Human to server ratio: 1:1000 Automation is more crucial Distributed workload, spread out on lots of commodity servers High upfront cost amortized over time and use Pay per use for customers Utilization is critical 10

11 What is a data center network (DCN)? Enables DC communication Internally from server to server From/to outside of the DC In practice HW: switches, routers, and cabling SW: protocols 11

12 What is a data center network (DCN)? Both layer-2 (link) and 3 (network) devices I.e. L3 routers and L2 switches Layer 2 subnets connected with layer 3 Layer 4 (transport) needed similar to any packet networks Note: does not have to be TCP/IP! Not part of routed Internet Cannot resolve DC server s address directly from Internet, only front end servers But often is TCP/IP WWW phone..." SMTP HTTP SIP..." TCP UDP " " IP" " Eth PPP WiFi 3GPP " copper fiber radio OFDM FHSS..." 12

13 What makes DCNs special? Just plug all servers to a single switch and that s it? Several issues with this approach Scaling up capacity Lots of servers need lots of switch ports E.g.: Cisco Nexus 7000 modular data center switch (L2 and L3) supports max /10GE ports Switch capacity and price Prices goes up with nb of ports E.g.: List price for 768 ports with 10GE modules over $1M (2013) Buying lots of commodity switches is an attractive option Potentially majority of traffic stays within DC Server to server 13

14 What makes DCNs special? (cont.) Requirements different from Internet applications Large amounts of bandwidth Very, very short delays Still, often Internet protocols (TCP/IP) used Management requirements Incremental expansion Should be able to withstand server failures, link outages, server rack failures Under failures, performance should degrade gracefully Requirements due to expenses Cost-effectiveness; high throughput per dollar Power efficiency DCN topology and equipment matter a lot 14

15 Data Center Costs Amortized Cost* Component Sub-Components ~45% Servers CPU, memory, disk ~25% Power infrastructure UPS, cooling, power distribution ~15% Power draw Electrical utility costs ~15% Network Switches, links, transit Total cost varies Upwards of $1/4 B for mega data center Server costs dominate Network costs also significant Network should allow high utilization of servers *3 yr amortization for servers, 15 yr for infrastructure; 5% cost of money Source: Greenberg et al. The Cost of a Cloud: Research Problems in Data Center Networks. Sigcomm CCR

16 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 16

17 Switch vs. router: What s the difference? Switch is layer 2 device Does not understand IP protocol Does not run any routing protocol Router is layer 3 device Speaks IP protocol Runs routing protocols to determine shortest paths OSPF, RIP, etc. Terminology often not so clear E.g. multi-layer and layer 3 switches also exist 17

18 Switch vs. router: Difference in basic functioning Router Forwards packets based on destination IP address Prefix lookup against routing tables Routing tables built and maintained by routing algorithms and protocols Protocols exchange information about paths to known destinations Algorithms compute shortest paths based on this information Broadcast sending usually not allowed Switch Forwards frames (packets) based on destination MAC address Uses switch table Equivalent to routing table in router Broadcast sending is common How is switch table built and maintained since there is no routing protocol? 18

19 Switch is self learning When frame is received from one port Switch learns that sender is behind that port Switch adds that information to switch table Soft state: forget after a while If destination not (yet) known Flood to all other ports Flooding can lead to forwarding loops Switches connected in cyclic manner These loops can create broadcast storms Spanning tree protocol (STP) used to avoid loops Generates loop-free topology Avoid using some ports when flooding Rapid Spanning Tree Protocol (RSTP) Faster convergence after a topology change CC Hub Port 2 Port 2 AA 21 AA 21 Port 1 Port 1 Hub DD AA BB <Src=AA, Dest=DD> And so on No TTL in L2 headers! 19

20 Layer 2 vs. Layer 3 in DCN Management L2 close to plug-and-play L3 usually requires some manual configuration (subnet mask, DHCP) Scalability and performance L2 broadcasting and STP scale poorly L2 forwarding less scalable than L3 fwding L2 based on flat MAC addresses L3 based on hierarchical IP addresses (prefix lookup) L2 has no such load balancing over multiple paths as L3 L2 loops may still happen in practice, even with STP 20

21 Layer 2 vs. Layer 3 in DCN Flexibility VM migration may require change of IP address in L3 network Need to conform to subnet address L2 network allows any IP address for any server ARP will learn new mapping from IP to MAC address Some reasons may prevent using pure L3 design Some servers may need L2 adjacency, e.g.: Servers performing the same functions (load balancing, redundancy) Non-IP traffic Dual homed servers may need to be on same L2 domain Connected to two different access switches Some configurations require both primary and secondary to be in same L2 domain 21

22 VLAN VLAN = Virtual Local Area Network VLANs overcome limitations of physical topology Run out of switch ports VLAN allows flexible growth while maintaining layer 2 adjacency L2 domain across L3 device VLAN can be port-based port-based VLAN: switch ports grouped so that a single physical switch works like multiple virtual switches

23 Port-based VLAN routing Traffic isolation Frames to/from ports 1-8 can only reach ports 1-8 Can also define VLAN based on MAC addresses of endpoints, rather than switch port Dynamic membership Ports can be dynamically assigned among VLANs Forwarding between VLANS happens on L3 1 2 VLAN1 (ports 1-8) VLAN2 (ports 9-15) 23

24 VLANs spanning multiple switches VLAN1 (ports 1-8) VLAN2 (ports 9-15) Ports 2,3,5 belong to VLAN1 Ports 4,6,7,8 belong to VLAN2 VLANs can span over multiple switches Also over different routed subnets Routers in between 24

25 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 25

26 Design Alternatives for DCN Two high level choices for interconnections: Specialized hardware and protocols E.g. Infiniband J Can provide high bandwidth & extremely low latency Custom hardware takes care of some reliability tasks Relatively low power physical layer L Expensive Not natively compatible with TCP/IP applications Commodity (1/10 Gb) Ethernet switches and routers Compatible Cheaper Let s look at this a bit more 26

27 Conventional DCN architecture Internet Topology: Two- or threelevel trees of switches or routers Multipath routing High bandwidth by appropriate interconnection of many commodity switches Redundancy Layer-3 router Layer-2/3 aggregation switches Layer-2 Top- Of-Rack access switches Servers 27

28 Issues with conventional architecture Bandwidth oversubscription Total bandwidth at core/aggregate level less than summed up bandwidth at access level Limited server to server capacity Application designers need to be aware of limitations No performance isolation VLANs typically provide reachability isolation only One server (service) sending/receiving too much traffic hurts all servers sharing its subtree There are more 28

29 One solution to oversubscription FAT Tree topology with special lookup scheme Add more commodity switches Carefully designed topology All ports have same capacity as servers Enables Full bisection bandwidth Lower cost because all switch ports have same capacity Drawbacks? Lot of switches and cabling Need also a way to distribute traffic load Core Switches Aggregation Switches Edge Switches FAT Tree M. Al-Fares et al. Commodity Data Center Network Architecture. In SIGCOMM

30 One solution to performance isolation: VLB Random flow spreading with Valiant Load Balancing (VLB) Similar FAT Tree topology with commodity switches Every flow bounced off a random intermediate switch Provably hotspot free for any admissible traffic matrix No need to modify switches (std forwarding) Relies on existing protocols and clever addressing Requires some changes to servers D ports D/2 switches... Intermediate node switches in VLB D/2 ports D/2 ports 20 ports 10G D switches... Top Of Rack switch [D 2 /4] * 20 Servers Aggregation switches A. Greenberg et al. VL2: A Scalable and Flexible Data Center Network. In SIGCOMM,

31 DCN architectures in research Lots of alternative proposed architectures in recent years Goals Overcome limitations of typical architectures of today Use commodity standard equipment VL2 & Monsoon & CamCube (MSR) Portland (UCSD) Dcell & Bcube (MSR, Tsinghua, UCLA) 31

32 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 32

33 TCP in the Data Center TCP typically used as transport inside DC DCNs different environment for TCP compared to normal Internet e2e transport Very short delays Specific application workloads How well does TCP work in DCNs? Several problems 33

34 DCN: latency matters Online data-intensive applications (OLDI) Web search, retail, advertisement, Latency is critical for OLDI applications Direct impact on user perceived QoE Page load time is a key metric for customer retention à direct impact on revenue How important is it? Amazon.com: revenue decreased by 1% of sales for every 100ms latency Objectives: Complete data flows as quickly as possible Meet flow deadlines 34

35 Generating a Web page in DCN Facebook as an example Creating a Page lots of components Internet Datacenter Network Front End News Feed Search Ads Chat 3 35

36 Generating a Web page in DCN (cont.) Servers may need to perform 100 s of data retrievals Many of which must be performed serially Overall page deadline of ms à Only have 2-3ms per data retrieval Including communication and computation High percentiles of delay are important If single data retrieval is unlikely to miss a deadline (median) but 1 of 100 retrievals is likely to miss a deadline (99 th percentile) à deadline miss happens on every page Data retrieval dependencies can magnify impact 36

37 Partition/Aggregate Application Structure Internet Deadline = 250ms The foundation for many large-scale web applications Web search, Social network composition, Ad selection, etc. Deadlines in lower hierarchy must meet with all-up deadline Iterative requests common à workers have tight deadlines Deadline = 50ms Deadline = 10ms Worker Nodes 37

38 Workloads Query-response traffic Partition/Aggregate Small flows ( mice ) Background traffic Short messages [50KB-1MB] Coordination, control state Small flows Large flows [1MB-50MB] Updating data on each server Large flows ( elephants ) Challenge: All this traffic goes through same switches Requirements are conflicting Requires minimal delay Requires high throughput 38

39 DCN characteristics Network characteristics Large aggregate bandwidths Very short round trip time delays (<1ms) Typical switches Use large numbers of commodity switches Typically commodity switch has shared memory Common memory pool for all ports Why not separated memory spaces? Cost issue for commodity switches 39

40 Resulting problems with TCP in DCN Incast Incoming reply traffic gets synchronized Buffer overflow at a switch Queue Buildup Large flows eat up buffer space at switches Small flows suffer Buffer Pressure Special case of queue buildup with switches having shared buffers 40

41 Problems: Incast Worker 1 Worker 2 Synchronized mice collide. Ø Caused by Partition/ Aggregate Aggregator Worker 3 Worker 4 41

42 Incast What happens next? TCP timeout Default minimum values of timeout ms depending on OS Why is that a major problem? Orders of magnitude longer than RTT (<1ms) à huge penalty Fail to meet deadlines 42

43 Problems: Queue Buildup Different workloads compete for bandwidth Small mice flows Large elephant flows Large flows can eat up the shared buffer space Same outgoing port Result is similar than with incast 43

44 Problems: Queue Buildup Sender 1 Big flows build up queues Ø Increased latency for short flows Ø Packet loss Receiver Sender 2 44

45 Problems: Buffer pressure Kind of generalization of the previous problem Increased queuing delay and packet loss due to long flows traversing other ports Shared memory pool Packets incoming and outgoing different ports still eat up each common buffer space 45

46 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 46

47 Data Center TCP (DCTCP) TCP version particularly designed to transport data center traffic Partition-aggregate applications By Microsoft Research & Stanford University in

48 Data Center Transport Requirements 1. High Burst Tolerance Cope with the Incast problem 2. Low Latency Short flows, queries 3. High Throughput Continuous data updates, large file transfers We want to achieve all three at the same time 48

49 Exploring the solution space Proposal Throughput Burst tolerance Latency (Incast) Deep switch buffers J Can achieve high throughput J Tolerates large bursts L Queuing delays increase latency Shallow buffers L Can hurt throughput of L Cannot tolerate bursts well J Avoids long queuing delay elephant flows Jittering :/ No major impact J Prevents Incast L Increases median latency Shorter RTO min :/ No major impact J Helps recover faster L Doesn t help queue buildup Nw assisted congestion ctrl (ECN style) J High throughput with high utilization J Helps in most cases L Problem if only 1 pkt is too much J Reacts early to queue buildup 49

50 Jittering MLA Query Completion Time (ms) Jittering on Requests are jittered over 10ms window Jittering off Add random delay before responding Desynchronize the responding sources to avoid buffer overflow Jittering trades off median against high percentiles 50

51 Exploring the solution space Proposal Throughput Burst tolerance Latency (Incast) Deep switch buffers J Can achieve high throughput J Tolerates large bursts L Queuing delays increase latency Shallow buffers L Can hurt throughput of L Cannot tolerate bursts well J Avoids long queuing delay elephant flows Jittering :/ No major impact J Prevents Incast L Increases median latency Shorter RTO min :/ J No Improves major impact J Helps recover L Doesn t help throughput faster queue buildup Nw assisted congestion ctrl (ECN style) J High throughput with high utilization J Helps in most cases L Problem if only 1 pkt is too much J Reacts early to queue buildup 51

52 Review: TCP with ECN Sender 1 ECN = Explicit Congestion Notification Q: How do TCP senders react? A: Cut sending rate by half Sender 2 ECN Mark (1 bit) marking threshold Receiver 52

53 DCTCP: Two key ideas 1. React in proportion to the extent of congestion, not just its presence ü Reduces variance in sending rates, lowering queuing requirements ECN Marks TCP DCTCP Cut window by 50% Cut window by 40% Cut window by 50% Cut window by 5% 2. Mark based on instantaneous queue length ü Fast feedback to better deal with bursts Q: Why normal TCP with ECN does not behave like DCTCP? A: Fairness with conventional TCP 53

54 DCTCP Algorithm Switch side: Mark packets when Queue Length > K Sender side: Mark Maintain moving average of fraction of packets marked (α). In each RTT: K Don t mark Adaptive window decreases: Note: decrease factor between 1 and 2. 54

55 Why does DCTCP work? High Burst Tolerance Aggressive marking sources react before packets are dropped Large buffer headroom bursts fit Low Latency Small buffer occupancies low queuing delay High Throughput ECN averaging smooth rate adjustments, low variance Leads to high utilization 55

56 DCTCP in Action (Kbytes) 56

57 Does it completely solve the Incast problem? Remember Incast: large number of synchronized small flows hit the same queue Depends on the number of small flows: If nb of synchronized flows so high that just 1 packet from each enough to overflow buffer à DCTCP does not help No chance to give feedback to senders before damage is done No congestion control mechanism can help Only solution is to somehow schedule responses (e.g. jittering) Helps if each flow has several packets to transmit Windows build up over multiple RTTs Bursts in subsequent RTTs would lead to packet drops DCTCP sources receive enough ECN feedback to prevent buffer overflows 57

58 Comparing TCP and DCTCP Emulate traffic within 1 Rack of Bing cluster 45 1G servers, 10G server for external traffic Generate query, and background traffic Flow sizes and arrival times follow distributions seen in Bing Metric: Flow completion time for queries and background flows RTOmin = 10ms for both TCP & DCTCP More than fair comparison 58

59 Comparing TCP and DCTCP (cont.) Background Flows Query Flows 59

60 Comparing TCP and DCTCP (cont.) Background Flows Query Flows Short flows finish quicker 60

61 Comparing TCP and DCTCP (cont.) Background Flows Query Flows Throughput remains as high for long flows 61

62 Comparing TCP and DCTCP (cont.) Background Flows Query Flows Better burst tolerance for query flows 62

63 DCTCP summary DCTCP Handles bursts well Keeps queuing delays low Achieves high throughput Features: Simple change to TCP and a single switch parameter Based on existing mechanisms 63

64 TCP for DCN research Data transport in DCN has received attention recently Many solutions proposed over the last three years Deadline-Driven Delivery control protocol (D 3 ) (UCSB, Microsoft) Deadline-Aware Datacenter TCP (D 2 TCP) (Purdue, Google) DeTail (Berkeley, Facebook): cross layer solution PDQ (UIUC): flow scheduling solution, not transport protocol pfabric (Insieme Networks, Google, Stanford, Berkeley/ICSI) Kwiken (UIUC, Steklov Math Inst., Microsoft): framework including policing and resource reallocation, not a transport protocol The story is not completely written yet Important to understand the problem and how the solution space differs from that applicable with Internet transport DCTCP is only one possible approach to mitigate TCP performance issues in DCN 64

65 Outline What are data center networks? Layer 2 vs. Layer 3 in data center networks Data center network architectures TCP in data center networks Problems of basic TCP Data Center TCP (DCTCP) Conclusions 65

66 Wrapping up Data center networks provide specific networking challenges Potentially huge scale Different requirements than with traditional Internet applications Recently a lot of research activity New proposed architectures and protocols Big deal to companies with mega-scale data centers: $$ Popularity of cloud computing accelerates this evolution 66

67 Want to know more? 1. M. Arregoces and M. Portolani. Data Center Fundamentals. Cisco Press, Kandula, S., Sengupta, S., Greenberg, A., Patel, P., and Chaiken, R The nature of data center traffic: measurements & analysis. In Proceedings of IMC Vasudevan, V., Phanishayee, A., Shah, H., Krevat, E., Andersen, D. G., Ganger, G. R., Gibson, G. A., and Mueller, B Safe and effective fine-grained TCP retransmissions for datacenter communication. In Proceedings of the ACM SIGCOMM A. Greenberg et al. VL2: A Scalable and Flexible Data Center Network. In SIGCOMM, C. Guo et al. DCell: A Scalable and Fault Tolerant Network Structure for Data Centers. In SIGCOMM, M. Al-Fares, A. Loukissas, and A. Vahdat. A Scalable, Commodity Data Center Network Architecture. In Proceedings of the ACM SIGCOMM Niranjan Mysore, R., Pamboris, A., Farrington, N., Huang, N., Miri, P., Radhakrishnan, S., Subramanya, V., and Vahdat, A PortLand: a scalable fault-tolerant layer 2 data center network fabric. In Proceedings of the ACM SIGCOMM Joseph, D. A., Tavakoli, A., and Stoica, I A policy-aware switching layer for data centers. In Proceedings of the ACM SIGCOMM Guo, C., Lu, G., Li, D., Wu, H., Zhang, X., Shi, Y., Tian, C., Zhang, Y., and Lu, S BCube: a high performance, server-centric network architecture for modular data centers. In Proceedings of the ACM SIGCOMM Abu-Libdeh, H., Costa, P., Rowstron, A., O'Shea, G., and Donnelly, A Symbiotic routing in future data centers. In Proceedings of the ACM SIGCOMM Check SIGCOMM program as well 67