Mahout: Low-Overhead Datacenter Traffic Management using End-Host-Based Elephant Detection Vasileios Dimitrakis Vasileios Dimitrakis 2016-03-18 1
Introduction - Motivation (1) Vasileios Dimitrakis 2016-03-18 2
Introduction - Motivation (1) Datacenters have enormous demands for bandwidth Vasileios Dimitrakis 2016-03-18 2
Introduction - Motivation (1) Datacenters have enormous demands for bandwidth Large fraction of data center traffic is carried in small number of flows Vasileios Dimitrakis 2016-03-18 2
Introduction - Motivation (1) Datacenters have enormous demands for bandwidth Large fraction of data center traffic is carried in small number of flows Management of large flows could lead to better utilization of datacenter fabric Vasileios Dimitrakis 2016-03-18 2
Introduction - Motivation (2) Current elephant detection methods suffer from limitations Vasileios Dimitrakis 2016-03-18 3
Introduction - Motivation (2) Current elephant detection methods suffer from limitations Solution-Mahout: A traffic management method for Elephant Flows Vasileios Dimitrakis 2016-03-18 3
Current Approaches Vasileios Dimitrakis 2016-03-18 4
Current Approaches Applications identify their flows as elephant Vasileios Dimitrakis 2016-03-18 4
Current Approaches Applications identify their flows as elephant Maintain per-flow statistics (Hedera Approach) Vasileios Dimitrakis 2016-03-18 4
Current Approaches Applications identify their flows as elephant Maintain per-flow statistics (Hedera Approach) Sampling Method Vasileios Dimitrakis 2016-03-18 4
Proposed Solution: Mahout Vasileios Dimitrakis 2016-03-18 5
Proposed Solution: Mahout A shim layer on each end host monitors flows Vasileios Dimitrakis 2016-03-18 5
Proposed Solution: Mahout A shim layer on each end host monitors flows It detects elephant flows and marks their packets Vasileios Dimitrakis 2016-03-18 5
Proposed Solution: Mahout A shim layer on each end host monitors flows It detects elephant flows and marks their packets Switches forward marked packets to controller Vasileios Dimitrakis 2016-03-18 5
Proposed Solution: Mahout A shim layer on each end host monitors flows It detects elephant flows and marks their packets Switches forward marked packets to controller Mahout controller computes the best path only for the elephant flows Vasileios Dimitrakis 2016-03-18 5
Detection of elephant flows in end host: Mahout approach Advantages of elephant flow detection in end-hosts: Vasileios Dimitrakis 2016-03-18 6
Detection of elephant flows in end host: Mahout approach Advantages of elephant flow detection in end-hosts: End host OS has better visibility into the applications behavior Vasileios Dimitrakis 2016-03-18 6
Detection of elephant flows in end host: Mahout approach Advantages of elephant flow detection in end-hosts: End host OS has better visibility into the applications behavior Feasible deployment on end-hosts Vasileios Dimitrakis 2016-03-18 6
Detection of elephant flows in end host: Mahout approach Advantages of elephant flow detection in end-hosts: End host OS has better visibility into the applications behavior Feasible deployment on end-hosts Very low overhead on commodity servers Vasileios Dimitrakis 2016-03-18 6
Mahout Architecture Vasileios Dimitrakis 2016-03-18 7
In-band Signaling When an elephant flow is detected, controller is informed! Vasileios Dimitrakis 2016-03-18 8
In-band Signaling When an elephant flow is detected, controller is informed! The packets are marked using the Differentiated Services (DS) field Vasileios Dimitrakis 2016-03-18 8
Mahout Controller The controller computes the best path for the packet marked as elephant Vasileios Dimitrakis 2016-03-18 9
Mahout Controller The controller computes the best path for the packet marked as elephant An example flow table setup at a switch by Mahout controller: Vasileios Dimitrakis 2016-03-18 9
Analytical Evaluation Hedera Vasileios Dimitrakis 2016-03-18 10
Analytical Evaluation Hedera Table entries need to be maintained for all flows Vasileios Dimitrakis 2016-03-18 10
Analytical Evaluation Hedera Table entries need to be maintained for all flows No OpenFlow switch can support this high number of flows Vasileios Dimitrakis 2016-03-18 10
Analytical Evaluation Hedera Table entries need to be maintained for all flows No OpenFlow switch can support this high number of flows Sampling Vasileios Dimitrakis 2016-03-18 10
Analytical Evaluation Hedera Table entries need to be maintained for all flows No OpenFlow switch can support this high number of flows Sampling Adds low overhead Vasileios Dimitrakis 2016-03-18 10
Analytical Evaluation Hedera Table entries need to be maintained for all flows No OpenFlow switch can support this high number of flows Sampling Adds low overhead However, things are getting really bad, when network utilization increases Vasileios Dimitrakis 2016-03-18 10
Analytical Evaluation Hedera Table entries need to be maintained for all flows No OpenFlow switch can support this high number of flows Sampling Adds low overhead However, things are getting really bad, when network utilization increases Mahout Vasileios Dimitrakis 2016-03-18 10
Analytical Evaluation Hedera Table entries need to be maintained for all flows No OpenFlow switch can support this high number of flows Sampling Adds low overhead However, things are getting really bad, when network utilization increases Mahout Statistics only gathered for elephant flows Vasileios Dimitrakis 2016-03-18 10
Analytical Evaluation Hedera Table entries need to be maintained for all flows No OpenFlow switch can support this high number of flows Sampling Adds low overhead However, things are getting really bad, when network utilization increases Mahout Statistics only gathered for elephant flows Significantly lower number of controllers needed compared to Hedera Vasileios Dimitrakis 2016-03-18 10
Experimental Results (1) Mahout s detection time of elephant flows is significantly lower than the one of Hedera! Vasileios Dimitrakis 2016-03-18 11
Experimental Results (2) Mahout detects elephant flows 3 times sooner than in-network schemes do Vasileios Dimitrakis 2016-03-18 12
Experimental Results (3) Vasileios Dimitrakis 2016-03-18 13
Strong aspects of Mahout Vasileios Dimitrakis 2016-03-18 14
Strong aspects of Mahout Controller handles less flows Vasileios Dimitrakis 2016-03-18 14
Strong aspects of Mahout Controller handles less flows Reduces the in-switch resource requirements Vasileios Dimitrakis 2016-03-18 14
Strong aspects of Mahout Controller handles less flows Reduces the in-switch resource requirements Significant throughput improvement compared to static load balancing techniques Vasileios Dimitrakis 2016-03-18 14
Strong aspects of Mahout Controller handles less flows Reduces the in-switch resource requirements Significant throughput improvement compared to static load balancing techniques Sooner detection of elephant flows Vasileios Dimitrakis 2016-03-18 14
Weak aspects of Mahout Vasileios Dimitrakis 2016-03-18 15
Weak aspects of Mahout DSCP bits may be needed for other uses in some datacenters Vasileios Dimitrakis 2016-03-18 15
Weak aspects of Mahout DSCP bits may be needed for other uses in some datacenters Mahout shim layer needs to be deployed in every virtual machine in virtualized datacenters Vasileios Dimitrakis 2016-03-18 15
Weak aspects of Mahout DSCP bits may be needed for other uses in some datacenters Mahout shim layer needs to be deployed in every virtual machine in virtualized datacenters No specific way to propose certain thresholds for the detection of elephant flows Vasileios Dimitrakis 2016-03-18 15
Conclusion Mahout is a low overhead yet effective traffic management system Vasileios Dimitrakis 2016-03-18 16
Conclusion Mahout is a low overhead yet effective traffic management system Manages elephant flows based on an end-host detection scheme Vasileios Dimitrakis 2016-03-18 16
Conclusion Mahout is a low overhead yet effective traffic management system Manages elephant flows based on an end-host detection scheme Experimental results showed the feasibility of its deployment Vasileios Dimitrakis 2016-03-18 16
Q & A Thank you very much for your attention! Vasileios Dimitrakis 2016-03-18 17
References [1] Mahout: Low-Overhead Datacenter Traffic Management using End-Host-Based Elephant Detection. A. Curtis, W. Kim, P. Yalagandula [2] Hedera: Dynamic Flow Scheduling for Data Center Networks. M. Al-Fares, S. Radhakrishnan Vasileios Dimitrakis 2016-03-18 18