TCP Incast problem Existing proposals

Transcription

1 TCP Incast problem & Existing proposals

2 Outline The TCP Incast problem Existing proposals to TCP Incast deadline-agnostic Deadline-Aware Datacenter TCP deadline-aware

3 Picasso Art is TLA 1. Deadline = 250ms 2. Art is a lie 3... Picasso Time is money Strict deadlines (SLAs) 1. Missed deadline Lower quality result.. MLA MLA Deadline = 50ms 1. Art is a lie 2. The chief.. 3. Deadline = 10ms The It Everything I'd is Art Computers Inspiration chief your like Bad is to a work enemy lie live you artists that in are as does can of life a makes copy. useless. creativity poor imagine exist, that man is the is They but can it with ultimate Good must realize only good lots find artists real. give seduction. of sense. the you money. you steal. truth. working. answers. Worker Nodes

4 The TCP Incast problem Incast:TCP Throughput Collapse Drastic reduction in application throughput when simultaneously requesting data from many servers using TCP Leading to Gross under utilization of link capacity in many- to-one communication networks, like Data Center networks

5 The TCP Incast problem Server Request Unit (SRU) Caused by Partition/Aggregate. Such as web research, MapReduce, Dryad Switch link idle! R Client TCP RTO min timeout = 200 ms Data Block Storage servers

6 The TCP Incast problem Collapse! Cause of throughput collapse coarse-grained TCP timeouts

7 Methods of solving the TCP Incast There are two main methods of solving the TCP Incast issue Reduce packet loss Many data senders send data simultaneously, and the bottleneck switch buffers are overloaded, which leads to packet loss Quick recovery Let TCP Sender enter into retransmission more quickly

8 Existing proposals to TCP Incast By the aspects of consideration, they can be mainly divided into Link layer proposals Congestion control Transport layer Reducing the minimum retransmission timeout timer Enhanced Transmission Control Protocols Application layer Increasing server request unit size Other proposals Larger switch buffer Probabilistic retransmission

9 Link layer proposals - QCN (quantized congestion notification) QCN: Congestion Control Mechanism at Layer2 Terminology: Congestion point where congestion occurs, mainly switches Reaction point source of traffic The QCN Algorithm: Congestion Point (CP) Algorithm Reaction Point (RP) Algorithm

10 The CP Algorithm w : fixed parameter (= 2) Q : Current Queue Length Q eq : Q len in equibrium (20% of buffer) Q old : Queue length at previous processing Calculate F b = (Q off + wq δ ) Q δ = Q Q old n If F b 0, nop Time to mark a packet? If F b < 0, send a congestion message back with the quantized F b value y Q off = Q Q eq

11 The RP Algorithm

12 Link layer proposals - QCN (quantized congestion notification) Unmodified TCP with QCN QCN with Reduced minrto (10ms minrto TCP)

13 Link layer proposals - QCN (quantized congestion notification) Without QCN Over 200KB with 256KB buffer size (~80%) With QCN Only 70KB with 256KB buffer size (~30%) QCN does keep queue lengths controlled, but the performance in an incast setup is not good

14 Transport layer - Reducing the RTOmin The default value of the TCP minimum RTO timer is 200ms This value is greater than the RTT in general data center networks, which is typically around 100μs It leads to a spurious retransmission Reducing the RTO min from the default 200ms to 200μs improves goodput by an order of magnitude

15 Transport layer - Reducing the RTOmin Implementation Challenges A TCP timer in microseconds needs 1) Hardware support that does not exist 2) Efficient software timers that are not available in most OS

16 Enhanced Transmission Control Protocols - Data Center TCP Algorithm Sender 1 ECN = Explicit Congestion Notification ECN Mark (1 bit) Receiver Sender 2

17 Enhanced Transmission Control Protocols - Data Center TCP Algorithm Switch side: Mark packets when Queue Length > K Sender side: Maintain running average of fraction of packets marked (α) In each RTT: B Mark K Don t Mark Adaptive window decreases:

18 Enhanced Transmission Control Protocols - Data Center TCP Algorithm DCTCP satisfies some requirements for Data Center packet transport Handles bursts well Keeps queuing delays low Achieves high throughput Limitations Deadline-unawareness: 7% ~ 25% of flow deadline are missed Fair sharing may delay the flow completion time

19 Enhanced Transmission Control Protocols - Incast Congestion Control for TCP ICTCP designed at the receiver side by adjusting the TCP receive window proactively before packet losses occur The ICTCP algorithm has two main components: Control trigger by evaluating available bandwidth Window adjustment on single connection

20 Enhanced Transmission Control Protocols - Incast Congestion Control for TCP 1) Control trigger by evaluating available bandwidth 2) Window adjustment on single connection CC Link capacity of the interface on receiver BBBB Increase receive window if there is enough quota of available bandwidth dd bb TT : The bandwidth of total incoming traffic BBBB AA : Available γγ1 bandwidth on the network interface αα [0,1]: A parameter Decrease to absorb the quota potential correspondingly oversubscribed if bandwidth the receive during window window is increased adjustment bb mm Measured throughput dd bb Decrease receive window by one MSS2 if this condition holds for three bb ee > The γγ1 expected throughput continuous RTT. The minimal receive window is 2*MSS dd bb The ratio of throughput difference of measured and expected throughput over the expected one for connection Keep ii current receive window γγ1 < dd bb < γγ2

21 Enhanced Transmission Control Protocols - Incast Congestion Control for TCP

22 Application layer - Increasing server request unit size With larger SRU sizes, active servers will use the spare link capacity made available by any stalled flow waiting for a timeout event Unfortunately, an SRU size of 1 megabyte is quite impractical: Most applications ask for data in small chunks, corresponding to an SRU size range of 1-256KB The storage system needs to allocate larger space in the client kernel memory

23 Application layer - Increasing server request unit size

24 Other proposals - Larger switch buffer Incast can be avoided with a large enough buffer space Doubling the size of the switch s output port buffer, doubles the number of servers that can supported before the onset of incast Unfortunately, switches with larger buffers tend to cost more

25 Other proposals - Larger switch buffer

26 Other proposals - Probabilistic retransmission A kernel thread is the lightest unit of kernel scheduling 1) Retransmit the highest unacknowledged segment with probability p, which is marked in one of six reserved bits in the segment header 2) The receiver will return a normal ACK followed by dupackthresh ACKs 3) The sender will automatically enter Fast Retransmit without waiting for retransmission timeouts when it receives dupackthresh ACKs

27 Other proposals - Probabilistic retransmission How to choose P is still a question to be considered If P is set too low The technique will provide no significant benefit If P is set too high Cause unnecessary retransmission, which contributes further to the congestion at the switch

28 The challenges in the data center Many flows in data centers have deadlines and missing deadlines would hurt application performance such as affecting response quality in Web applications Existing solutions now cannot provide deadline-aware transmission services (non deadline-aware or non TCP Incast)

29 Meeting deadlines in datacenter networks Each sender calculates and requests its sending rate from routers along its flow path, while routers perform rate allocations, prioritizing the flows with deadlines to satisfy as many of them as possible [1][2] PDQ flow scheduling is another deadline aware protocol that fixes the priority inversion problem in D3 by enabling flow preemption [2] D3 and PDQ, switches become the critical controllers that proactively allocate sending rates to flows [1][2] The essential difference between DCTCP and D2TCP is deadline awareness [3] 1. Better never than late: Meeting deadlines in datacenter networks, in Proc. ACM SIGCOMM, Finishing flows quickly with preemptive scheduling, in Proc. ACM SIGCOMM, Deadline-aware datacenter TCP (D2TCP), in Proc. ACM SIGCOMM, 2012

30 Deadline-Aware Datacenter TCP Many flows in data centers have deadlines and missing deadlines would hurt application performance such as affecting response quality in Web applications Above solutions are deadline-agnostic Objective 1) Meet OLDI(Online Data Intensive) deadlines 2) Achieve high bandwidth for background flows 3) Working with existing switch hardware 4) Be able to coexist with legacy TCP

31 Main Method Start with DCTCP and build deadline awareness on top of it. Recall how we calculate α in DCTCP: Instead of adjusting cwnd according to α, D2TCP uses p, computed as where d is a deadline imminence factor, defined as Then adjust cwnd as follows: F : The fraction of packets that were marked with CE bits in the most recent window T c : The time needed for a flow to complete transmitting all its data

32 Deadline-Aware Datacenter TCP Reduces the fraction of missed deadlines compared to DCTCP by 75% Coexists with TCP flows without degrading their performance Throughput for DCTCP (left) vs. D2TCP (right)

33 Conclusions Several protocols have been proposed for data center applications employing a Partition/Aggregate workflow structure These transport protocols fall into three categories Fair-sharing protocols Data Center TCP (DCTCP), Rate Control Protocol (RCP) Deadline-aware protocols Deadline-Aware TCP (D 2 TCP), D 3 AFCT-minimizing protocols average flow completion times (AFCT) PDQ, RACS