Application-defined Multipath TCP Scheduling

Transcription

1 Application-defined Multipath TCP Scheduling Frühjahrstreffen der GI Fachgruppen BS und KuVS Amr Rizk Userland Kernel Application Send Data Load MPTCP Scheduler Sending Queue Network Stack Programmable Scheduler A version of this work was published at ACM/USENIX Middleware 2017 Subflows Alexander Frömmgen Amr Rizk Tobias Erbshäußer Max Weller Boris Koldehofe Alejandro Buchmann Ralf Steinmetz

2 Multipath TCP in a Nutshell 1 Congestion Control 2 Fairness on Joint Paths Path Management Trigger Subflow Establishment Server Sending Queue p 6 Scheduler p 5 Subflow 2 LTE Subflow 1 WiFi MPTCP Packet Client Application p 2 p 1 Receive Queue 4 Scheduler 3 TCP Options for MPTCP Map Packets to Subflows Control Information, e.g., Data Sequence Numbers KOM Multimedia Communications Lab 2

3 Multipath TCP Scheduling Server Packet 2 Packet 1 RTT = 30ms BW = P_IN_FLY = 0 CWND = 10 Subflow 2 LTE Client Scheduler Subflow 1 WiFi RTT = 10ms BW = P_IN_FLY = 0 CWND = 10 Images: KOM Multimedia Communications Lab 3

4 Multipath TCP Scheduling Intuition: Server Schedule on Subflow with minimum round-trip time (minrtt). Packet 2 Packet 1 RTT = 30ms BW = P_IN_FLY = 0 CWND = 10 Subflow 2 LTE Client Scheduler Subflow 1 WiFi RTT = 10ms BW = P_IN_FLY = 0 CWND = 10 Images: KOM Multimedia Communications Lab 4

5 Multipath TCP Scheduling Server Schedule on Subflow with minimum round-trip time (minrtt). Packet 11 Scheduler RTT = 30ms BW = P_IN_FLY = 0 CWND = 10 Subflow 2 LTE Subflow 1 WiFi RTT = 10ms BW = P_IN_FLY = 10 CWND = 10 Packet Packet 5 Packet 41 Client Images: KOM Multimedia Communications Lab 5

6 Multipath TCP Scheduling Server Schedule on Subflow with minimum round-trip time (minrtt). Packet 11 Scheduler which is not saturated (Congestion window > packets in flight). RTT = 40ms BW = P_IN_FLY = 0 CWND = 10 Subflow 2 LTE Subflow 1 WiFi RTT = 10ms BW = P_IN_FLY = 10 CWND = 10 Packet Packet 5 Packet 41 Client Images: KOM Multimedia Communications Lab 6

7 Multipath TCP Scheduling Real-world Experiment Server Sending Queue p 6 Scheduler p 5 ~RTT 30ms Subflow 2 LTE ~RTT 10ms Subflow 1 WiFi Client Application p 2 p 1 Receive Queue Constant Bitrate Stream Home Network WiFi and LTE Germany KOM Multimedia Communications Lab 7

8 Multipath TCP Scheduling Real-world Experiment Server Sending Queue p 6 Scheduler p 5 ~RTT 30ms Subflow 2 LTE ~RTT 10ms Subflow 1 WiFi Client Application p 2 p 1 Receive Queue Constant Bitrate Stream Home Network WiFi and LTE Germany KOM Multimedia Communications Lab 8

9 Multipath TCP Scheduler Overview Name Domain Preferences Available in Linux (Implementation) Min RTT (Default) General Purpose Binary Round-Robin General Purpose Binary Redundant Thin Flows Binary Compensate Loss Short Datacenter Flows? No Energy preserving? No DASH Video Yes No... KOM Multimedia Communications Lab 9

10 Multipath TCP Scheduler Overview Great Concepts without MPTCP Implementation X. Corbillon, R. Aparicio-Pardo, N. Kuhn, G. Texier, and G. Simon. Crosslayer scheduler for video streaming over MPTCP. In ACM MMSys, 2016 H. Xu and B. Li. RepFlow: Minimizing flow completio replicated flows in data centers. In IEEE INFOCOM, 2014 N. Kuhn, E. Lochin, A. Mifdaoui, G. Sarwar, O. Mehani, and R. Boreli. DAPS: intelligent delay-aware packet scheduling for multipath transport. In IEEE ICC, KOM Multimedia Communications Lab 10

11 Goals (of this talk) Systematically specify and execute MPTCP schedulers Enable application-defined MPTCP scheduling KOM Multimedia Communications Lab 11

12 Part I: Towards a Programmable Scheduler in the Network Stack Userland Kernel Application Send Data Network Stack Load MPTCP Scheduler Sending Queue Programmable Scheduler Subflows KOM Multimedia Communications Lab 12

13 Scheduler Model of the Scheduler Environment Userland Kernel Application PUSH PUSH op* Sending Queues *operation, e.g., POP Subflows KOM Multimedia Communications Lab 13

14 Scheduler Model of the Scheduler Environment Userland Kernel Application PUSH Sending Queue Q p 4 p 3 p 2 Packets in Flight QU p 1 Reinjection Queue RQ op* op* op* PUSH PUSH Subflow sbf1 p 1 Queue Q sbf1 Properties: rtt, cwnd Subflow sbf2 Queue Q sbf2 Prop.: rtt, cwnd, Sending Queues *operation, e.g., POP Subflows KOM Multimedia Communications Lab 14

15 Specifying Multipath TCP Schedulers Requirements Expressiveness Timely Execution Isolation Design Decisions 1. Modelled Elements as Entities: Set of Subflows with their properties, Queues of Packets with their Properties 2. Declarative Packet and Subflow Selection (Filter, Min, Max) 3. No Recursion, No Functions, Limited Loops 4. Variables with Single Assignment, Implicit and Static Type System 5. No, One, or Multiple Packets per Scheduler Execution KOM Multimedia Communications Lab 15

16 Back to the Motivating Example Server Sending Queue p 6 Scheduler p 5 ~RTT 30ms Subflow 2 LTE ~RTT 10ms Subflow 1 WiFi How to use the Backup Flow? Client Application p 2 p 1 Receive Queue Constant Bitrate Stream Home Network WiFi and LTE Germany KOM Multimedia Communications Lab 16

17 Systematically Specify MPTCP Schedulers Domain Specific Specification Language Example: Preference-aware RTT-sensitive Scheduler 1 VAR sbfcandis = SUBFLOWS.FILTER( 2 sbf => sbf.cwnd > sbf.skbs_in_flight + sbf.queued 3 AND!sbf.TSQ_THROTTLED AND!sbf.LOSSY); 4 5 VAR backsbf = sbfcandis.filter( 6 sbf => sbf.is_backup).min(sbf => sbf.rtt); 7 VAR nonbacksbf = sbfcandis.filter( 8 sbf =>!sbf.is_backup).min(sbf => sbf.rtt); 9 10 IF (nonbacksbf.rtt_ms > 100 AND backsbf.rtt_ms < 80) { 11 backsbf.push(q.pop()); 12 } ELSE { 13 nonbacksbf.push(q.pop()); } KOM Multimedia Communications Lab 17

18 Systematically Specify MPTCP Schedulers Domain Specific Specification Language Example: Preference-aware RTT-sensitive Scheduler 1 VAR sbfcandis = SUBFLOWS.FILTER( 2 sbf => sbf.cwnd > sbf.skbs_in_flight + sbf.queued 3 AND!sbf.TSQ_THROTTLED AND!sbf.LOSSY); 4 5 VAR backsbf = sbfcandis.filter( 6 sbf => sbf.is_backup).min(sbf => sbf.rtt); 7 VAR nonbacksbf = sbfcandis.filter( 8 sbf =>!sbf.is_backup).min(sbf => sbf.rtt); 9 10 IF (nonbacksbf.rtt_ms > R1 AND backsbf.rtt_ms < R2) { 11 backsbf.push(q.pop()); 12 } ELSE { 13 nonbacksbf.push(q.pop()); } KOM Multimedia Communications Lab 18

19 Systematically Specify and Execute MPTCP Schedulers ProgMP Write to /proc/ /schedulers Compiler Frontend Ahead of Time Compiler Userland Kernel Lexer Parser Control Flow Graph Optimizer Native Code ebpf Compiler Interpreter Uses Network Stack Scheduler Specified schedulers are executable in the Linux Kernel KOM Multimedia Communications Lab 19

20 Abstraction vs. Overhead KOM Multimedia Communications Lab 20

21 Abstraction vs. Overhead link saturation at 1 Gbit/s KOM Multimedia Communications Lab 21

22 Abstraction vs. Overhead link saturation at 1 Gbit/s The runtime environment induces an overhead, which is acceptable for most application scenarios. KOM Multimedia Communications Lab 22

23 Part II: Design of Novel Multipath TCP Schedulers Round-trip Time-aware Constant Bitrate Stream Scheduling Redundant Scheduling HTTP/2-aware Scheduling More in Paper and Under Review KOM Multimedia Communications Lab 23

24 A Close Look at Redundant Schedulers Sender Subflow 1 (RTT 50ms) Receiver Q Seq 6 Seq 7 In Flight Seq 2 Seq 3 Seq 4 Seq 4 Ack 2 Subflow 2 (RTT 100ms) Seq 5 Seq 3 Seq 3 Seq 2 Seq 5 When the acknowledgement Ack 2 arrives at the sender: should we send the fresh packet Seq 6 or the old packet Seq 5 KOM Multimedia Communications Lab 24

25 A Close Look at Redundant Schedulers Q Seq 6 Seq 7 Sender In Flight Seq 2 Seq 3 Seq 4 Seq 5 Prefer Per Subflow Fresh Packets 1 Subflow FOREACH(VAR 1 (RTT 50ms) sbf IN sbfcandidates) { Receiver 2 VAR skb = QU.FILTER(s =>!s.sent_on(sbf).top 3 IF(skb!= NULL) Seq { 4 Seq 3 4 Ack 2 sbf.push(skb); 5 Subflow } ELSE 2 (RTT { 150ms) Seq 2 6 sbf.push(q.pop()); 7 } Seq 5 Seq 3 8 } Prefer Global Fresh Packets IF(!sbfCandidates.EMPTY) { FOREACH(VAR sbf IN sbfcandidates) { sbf.push(q.top); Seq 6 Seq 5 } DROP(Q.POP()); } KOM Multimedia Communications Lab 25

26 A Close Look at Redundant Schedulers Prefer Per Subflow Fresh Packets Sender 1 Subflow FOREACH(VAR 1 (RTT 50ms) sbf IN sbfcandidates) { Receiver 2 VAR skb = QU.FILTER(s =>!s.sent_on(sbf).top Q In Flight 3 IF(skb!= NULL) Seq { 4 Seq 3 4 Ack 2 sbf.push(skb); Seq 6 Seq 2 5 Subflow } ELSE 2 (RTT { 150ms) Seq 2 Seq 7 Seq 3 6 sbf.push(q.pop()); 7 } Seq 4 Seq 5 Seq 3 8 } Seq 5 Prefer Global Fresh Packets 1 IF(!sbfCandidates.EMPTY) { 2 FOREACH(VAR sbf IN sbfcandidates) { 3 sbf.push(q.top); Seq 6 Seq 5 4 } ProgMP enables 5 rapid DROP(Q.POP()); specification and evaluation of schedulers 6 } KOM Multimedia Communications Lab 26

27 A Close Look at Redundant Schedulers Better ProgMP enables novel redundant schedulers, which outperform established approaches. KOM Multimedia Communications Lab 27

28 HTTP/2-aware Scheduling ProgMP enables HTTP/2-aware Scheduling. KOM Multimedia Communications Lab 28

29 HTTP/2-aware Scheduling KOM Multimedia Communications Lab 29

30 Conclusion We The presented a programming model enables: for Multipath TCP scheduling Specification and execution of MPTCP schedulers Application-defined MPTCP scheduling We proposed and evaluated novel MPTCP schedulers RTT-aware scheduler Constant bitrate schedulers Flavors of redundant schedulers HTTP/2-aware scheduler KOM Multimedia Communications Lab 30

31 Conclusion We The presented the programming first programming model enables: model for Multipath TCP scheduling. Specification and execution of MPTCP schedulers Application-defined MPTCP scheduling We proposed and evaluated sophisticated novel schedulers. RTT-aware scheduler Constant bitrate schedulers Flavors of redundant schedulers HTTP-aware scheduler KOM Multimedia Communications Lab 31

32 Environ. How can we systematically compare and evaluate scheduler design decisions? Web Frontend to Manage Exp. Add Config. and Env. Variations Add Protocols and Algorithms Fix / Improve Implementation Scalable Experiment Execution Exp. Study Config. x Env. Exp. Exp. Inspect Results: Interactive Data Analysis and Exploration Single Experiment Result Configurations Iterative Refinements KOM Multimedia Communications Lab 32

33 Environ. How can we systematically compare and evaluate scheduler design decisions? Web Frontend to Manage Exp. Add Config. and Env. Variations Add Protocols and Algorithms Fix / Improve Implementation Scalable Experiment Execution Exp. Study Inspect Results: Interactive Data Analysis and Exploration Single Experiment Result Config. A Framework x for the Management, Exp. the Scalable Execution and Interactive Analysis of Extensive Network Experiments Env. Exp. Configurations Iterative Refinements KOM Multimedia Communications Lab 33