Dynamic Distributed Flow Scheduling with Load Balancing for Data Center Networks

Transcription

1 Available online at Procedia Computer Science 19 (2013 ) The 4th International Conference on Ambient Systems, Networks and Technologies. (ANT 2013) Dynamic Distributed Flow Scheduling with Load Balancing for Data Center Networks Sourabh Bharti, K.K.Pattanaik Information and Communication Technology Atal Bihari Vajpayee-Indian Institute of Information Technology and Management, Gwalior, India Abstract Current Flow Scheduling techniques in Data Center Networks(DCN) results in overloaded or underutilized links. Static scheduling techniques such as ECMP and VLB use hashing techniques for scheduling the s. In case of hash collision a path gets selected number of times resulting overloading of that path and underutilization of other paths. Dynamic scheduling techniques like global first fit employ centralized scheduler and always selects first fittest candidate path for scheduling. Thus in addition to single-point-of-failure the overall link utilization also remains a problem as the s are not scheduled on the best available candidate path. This paper presents firstly a Dynamic Distributed Flow Scheduling(DDFS) mechanism that will lead to fair link utilization in globally used fat-tree topology of DCN. Secondly, it presents a mechanism to restrict the scheduling decisions to the lower layers thus avoiding saturation of core switches. The entire DCN is simulated using Colored Petri Nets (CPN). The load measured at the aggregate switches for various patterns in DCN reveals that the load factors at the aggregate switches vary by at most 0.11 which signifies the fair utilization of links The Authors. Published by Elsevier B.V. Open access under CC BY-NC-ND license. Selection c 2013 Published and peer-review by Elsevier under Ltd. responsibility Selection and/or of Elhadi peer-review M. Shakshuki under responsibility of [name organizer] Keywords: Dynamic Flow Scheduling, Load Balancing, Link Utilization, Data Center Networks, Colored Petri Nets. 1. Introduction In the present time DCNs, fat tree topology is being used globally. As we move up from top-of-rack (ToR) switches to Core switches (CS) the link over-subscription ratio becomes 16:1. Due to this bandwidth variation, the focus is on scheduling the s effectively. To deal with this problem, many static and dynamic scheduling techniques are proposed like Equal Cost Multiple Path (ECMP)[1], Valiant Load Balancing (VLB)[4], Global First fit [1]. Although the focus of these algorithms is to schedule the s in order to increase the overall network bandwidth utilization, often it causes some links to get highly overloaded while others underutilized. The gross outcome remains unfair utilization of links. As given in Hedera [1], a packet s path is non-deterministic and chosen on its way up to the core, and is addresses: bharti.sourabh90@gmail.com (Sourabh Bharti), kkpatnaik@iiitm.ac.in (K.K.Pattanaik) The Authors. Published by Elsevier B.V. Open access under CC BY-NC-ND license. Selection and peer-review under responsibility of Elhadi M. Shakshuki doi: /j.procs

2 Sourabh Bharti and K.K.Pattanaik / Procedia Computer Science 19 ( 2013 ) deterministic from the core switches to their destination edge switch. The fat-tree topology of DCN may cause all the traffic to approach core switches thus saturating them. In this paper, DDFS mechanism is proposed that will mitigate the core switch saturation problem by employing a deterministic scheduling at lower layer switches. The scheduling decision is thus distributed among all the layers instead of only at the core switches thus enhancing the immunity of core switches from failures and saturation. Load on a switch is defined as the ratio of incoming traffic to outgoing traffic in a defined time period. Load information at the aggregate switches are of interest as these form the junction connecting the two remaining different types of nodes (ToR and Core) in a fat-tree structured DCN. Thus a closely related load information among aggregate switches represents their incoming and outgoing traffic ratios too are closely related. This signifies fair-utilization of links connected to the corresponding aggregate switches. To mitigate the problem of long-waits for small s link reservation has been done. Long-wait is a scenario that arise while a link is occupied by very large s and small s wait for the availability of link. 55% of the average link capacity is reserved [8] for the long-lived s and remaining for the others. The rest of this paper is organized as follows. Section 2 discuss the related work. Proposed mechanism is presented in Section 3. Section 4 discusses the experiment design and simulation. Results and analysis are discussed in Section 5. Finally Section 6 concludes the paper. 2. Related Work Mainly there are two categories of scheduling techniques. One is static scheduling and the other is dynamic. ECMP and VLB come under static scheduling techniques. Whereas the global first fit come under dynamic scheduling techniques. For path selection, ECMP takes the hash value of selected fields of a packet s header modulo the number of equal cost paths. It then forwards the to that path which corresponds to the hash value. As the number of s per host and the size of the increases, there is a good chance of the collision of their hash values. This results in poor link utilization. VLB delivers the load in two steps:it selects the intermediate nodes randomly and then use these nodes for forwarding the load to the destination node. This may result in selecting the same intermediate node multiple times which further lead to the poor utilization of links. Hedera [1] employs centralized scheduler for scheduling. The scheduler uses global first fit for scheduling the s. This sometime cause waste of link capacity due to non effective path allocation. To the best of our knowledge no report is found in the literature regarding DDFS for DCNs for fair link utilization. 3. Dynamic Distributed Flow Scheduling(DDFS) In the fat-tree structured DCN with k source-destination pairs, n s between all source-destination pairs, f S D sized s between any source-destination pair, and any given link constraints the objective function can be described using linear programming model as: With constraints maximum k i=1 n f S D (1) j=1 i U l F i C l (2) U l = set of all s traversing link l, C l = capacity of link l, L = set of all links in the network, F i = requirement of i

3 126 Sourabh Bharti and K.K.Pattanaik / Procedia Computer Science 19 ( 2013 ) Proposed mechanism The proposed mechanism is to schedule the to improve overall link utilization while achieving closely related load values among all aggregate switches. We begin our discussion with the assumption that the load at any aggregate switch i is represented by a nonnegative integer scalar value w i. Each aggregate switch has four interfaces for connecting two core switches and two ToR switches (Fig 1). Each interface is equipped with one input and one output queue. The load on each interface is represented by the ratio of incoming and outgoing traffic over a period [9]. The total load on an aggregate switch can thus be represented by Eq.(3) w i = Load on each inter f ace (3) At time t the system s load distribution is represented by the vector W t = {w 1 t, w 2 t, w 3 t...w n t } (4) Wide variation among w i for i=1 ton signifies that the load on the aggregate switches are not uniformly distributed. Different values of w i is the consequence of different ratios of incoming and outgoing traffic at aggregate switches. The inference here is that due to improper scheduling on the available links results in wide variation among w i values. Our objective is to schedule the s such a way that will make this load distribution converge towards closely related load values and is represented by Eq.(5) W = {w, w, w...w} (5) Under the assumption of homogeneous aggregate switches and link bandwidths w can be represented as w = Algorithm 1: scheduling(destination address, source address, size, link information) Procedure to schedule the Data: Source address, Dest. address, Flow size, Link state information Result: Balanced Load on Aggregate Switches begin; if dest.edge=source.edge then return to dest from Edge; else aggregate SELECT-SWITCH(a,b,fls) if dest.edge is reachable from selected Aggregate switch then return to dest.edge else core SELECT-SWITCH(x,y,fls) get pod number from dest. Address return to the pod number.agg if dest.edge is reachable from selected Aggregate switch then return to dest.edge deliver to dest. n i=1 w i n (6)

4 Sourabh Bharti and K.K.Pattanaik / Procedia Computer Science 19 ( 2013 ) Algorithm 2: SELECT-SWITCH(link 1, link 2, size) Data: Link 1 capacity, Link 2 capacity, fls Result: selected link begin; if fls > (link1,link2) then wait for the link else if (link1 > fls) and (link1 > link2) then send by link 1 if (link2 > fls) and (link2 > link1) then send by link 2 return selected switch 4. Experiment design and simulation The proposed mechanism is simulated using CPN. We explain the experiment design and simulation setup in the following subsections Modeling DCN traffic Since DCNs traffic traces are not publicly available due to privacy and security concerns,we model patterns that characterize DCN traffic. Data center traffic s are characterized in two categories; small or short-lived s and large or long-lived s [1]. Large s are very less in number as compared to small s[5, 7]. Flow arrivals are Poisson distributed with an average number of s in a time-frame. Packet size is application dependent [10]. Similar to Internet traffic, DCN traffic may consist different application specific s andthe packet size in a is specific to an application. Packet size variation follows discrete random distribution. We first focus on generating DCN traffic patterns that stress and saturate the network, and then apply our dynamic distributed scheduling mechanism to achieve fair-link utilization. As a bi-product the mechanism prevents the core switch from saturation and failure. In our traffic data, size distribution was dominated by short-lived s as evident from Fig 6. Our DCN is structured around the fat-tree topology. Fig 1 shows the net for the topology used in DCN. It comprise of four pods and a pod is further comprise two ToRs and one aggregate switch. Each core switch is connected to all four pods. The link capacity between a node or server to ToR switch links is different from all other links. [2] 4.2. Simulation The Hierarchical net of our simulation is shown in Fig 1 and the corresponding simulation parameters are in Table 1. The logical and important subnets of this hierarchical net are shown in Fig 2 through Fig 5 and the firing rules are in Table 2.. Table 1. Simulation parameters Parameters Description Topology Fat-Tree Capacity Partition 55% of the average link capacity to large s Flow Arrivals Poisson Distributed Flow bandwidth 1-2% of average link capacity Scheduling Dynamic distributed Small size MB Large size >70000 MB Maximum no of s 500 s by one node Packet range poisson distributed with mean 100 Packet size range distributed between 1 to 1000

5 128 Sourabh Bharti and K.K.Pattanaik / Procedia Computer Science 19 ( 2013 ) Table 2. Firing rules Label Places Transitions Functionality Input data p1,,p3,p4 t1,t2 Generates s randomly Link capacity p1,,p3,p4,p5 t1,t2,t3,t4 Assigns link capacity dynamically Path selection a,b,p1,,p3 t1 Selects the link according to its available capacity Queue p1,,p3,p4 t1,t2 Implements a queue. C1 C2 C3 C4 C1 C2 C3 C4 v q r s t u w x y z q1 r2 s3 t4 u5 v6 A1 A2 A3 A4 A5 A6 A7 A8 A1 A2 A3 A4 A5 A6 A7 A8 a b f g j k n o c d e h i l m p E1 E2 E3 E4 E5 E6 E7 E8 E1 E2 E3 E4 E5 E6 E7 E8 a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12 a13 a14 a15 a16 Node1 Node1 Node2 Node2 Node3 Node3 Node4 Node5 Node6 Node7 Node8 Node9 Node10 Node11 Node12 Node13 Node14 Node15 Node16 Node4 Node5 Node6 Node7 Node8 Node9 Node10 Node11 Node12 Node13 Node14 Node15 Node16 Fig. 1. Fat-Tree Topology of DCN p1 a 1`0 1`0 j j+1 srange.ran() input (fls,lsize); output (lsize2); action let val x= Int.toLarge lsize; val y= Int.toLarge fls; p4 in (Int.fromLarge (x+y)) end; fls lsize2 1` p1 t1 p3 t4 lsize fls lsize lsize2 t1 nfrange() p3 nf t2 p4 t2 input (fls,lsize); output (lsize2); action let val x= Int.toLarge lsize; val y= Int.toLarge fls; in (Int.fromLarge (x-y)) end; t3 p5 Fig. 2. Input data Fig. 3. Link capacity a x y b p1 if(y>x)then 1` else empty t1 p3 if(y>x)then 1` else empty lu t2 lu::li li p1 1`[] li li^^[lu] li e t1 e lu p4 p3 E Fig. 4. Path selection Fig. 5. Queue

6 Sourabh Bharti and K.K.Pattanaik / Procedia Computer Science 19 ( 2013 ) Results and analysis Flow size distribution for the average number of s generated by any single node/server in a defined time period is shown in Fig 6. The number of large or long lived s are very less as compared to small or short-lived s which characterize the nature of DCN traffic. small s large s No of s t1 t2 t3 t4 t5 t6 t7 t8 t9 Time Fig. 6. Flow size distribution 1.5 DDFS NDFS 140 C1(NDFS) C2(NDFS) C3(NDFS) C4(NDFS) C1(DDFS) C2(DDFS) C3(DDFS) C4(DDFS) Load factor No of s a1 a2 a3 a4 a5 a6 a7 a8 t1 t2 t3 t4 Aggregate switch Time Fig. 7. Load factor variation at aggregate switches Fig. 8. Saturation of core switches Load measurement at aggregate switches was one of our important requirements in order to study the effect of our mechanism. The study was aimed at first to see how the present non deterministic scheduling(ndfs) effects the link utilization, and second to see the level of impact of our mechanism on the link utilization. Fig 7 shows the plot of average load factor estimated at aggregate switches. Load information at an aggregate switch represents the incoming and outgoing traffic rate ratio which signifies load factor on the switch. Ideally, the estimated load factor at an aggregate switch should be close to 1 for it to say that the associated link(s) are fairly utilized. Wide variation in load factor values among aggregate switches represent their incoming and outgoingtrafficrates are widely differing, which signifies unfair link utilization. For the sake of clarity about the traffic we presented in Fig 6 the different s and their population at different times. From Fig 7 it is evident that by using our mechanism, load factor across the aggregate switches vary

7 130 Sourabh Bharti and K.K.Pattanaik / Procedia Computer Science 19 ( 2013 ) between 0.80 and 0.91 with an average load factor of 0.85 per aggregate switch. This is an indication that all the links connected to aggregate switches have been effectively utilized. Whereas in NDFS the load factor across the aggregate switches varied between 1.05 and 1.50, with an average load factor of 1.27 per aggregate switch. This indicates outgoing traffic rate is lower than the incoming rate thus saturating aggregate switch and causing unfair link utilization. Comparison between the maximum variation in load factors obtained in each case demonstrates that our mechanism has been able to address the identified shortcomings of NDFS. Monitoring arrivals at core switch layer was another important requirement to study about how restricting the scheduling decisions to the lower layers enable avoiding saturation of core switches. Fig 8 represents the incoming s over an observation period. The plot shows a comparison between the s at each core switch for both NDFS and DDFS. It is a clear indication that the traffic on which scheduling decisions are to be taken at core switch is greatly reduced. Further analysis reveals that in the case of NDFS as the traffic increases over time the number of s accumulating at the core switches shows an increasing trend. This signifies core switch is tending towards saturation. Whereas the outcome of DDFS shows a decreasing trend of incoming s, which is the direct consequence of scheduling at the lower layers. 6. Conclusion The major findings of our work are that in the pursuit of fair utilization of available links for Data Center Networks, proposed DDFS can outperform NDFS and it is resilient to switch saturation when the network is stressed with more number of s. Another finding of our work is by taking care of scheduling s right from the edge switch level we are able to distribute traffic going towards the core switch fairly across the available links. As an outcome, the findings can be summarized as: fair utilization of links, fairly uniform load at aggregate switches, and finally preventing core switches from getting saturated. Due to the simple and easily deployable approach, we conclude that DDFS has the potential to produce better link utilization with moderate additional cost. References [1] Mohammad Al-Fares, Sivasankar Radhakrishnan, Barath Raghavan, Nelson Huang, Amin Vahdat, Hedera:dynamic scheduling for data center networks, in 7th USENIX Symposium on Networked Systems Design and Implementation, [2] Mohammad Al-Fares, Alexander Loukissas, Amin Vahdat, A scalable, commodity data center network architecture, in Proceedings of the ACM SIGCOMM 2008 Conference on Data Communication Seattle, WA, USA, August 17-22, [3] Albert Greenberg, Srikanth Kandula, David A. Maltz, James R. Hamilton, Changhoon Kim, Parveen Patel, Navendu Jain, Parantap Lahiri, Sudipta Sengupta, VL2:A scalable and flexible data center network, in Proceedings of the ACM SIGCOMM 2009 conference on Data communication, [4] Rui Zhang-Shen and Nick McKeown, Designing a predictable internet backbone with valiant load-balancing, in Thirteenth International Workshop on Quality of Service (IWQoS), Passau, Germany, [5] Theophilus Benson, Ashok Anand, Aditya Akell, and Ming Zhang, M. Un- understanding Data Center Traffic Characteristics. in Proceedings of ACM WREN, [6] G Reenberg. et al, VL2: A Scalable and Flexible Data Center Network. in Proceedings of ACM SIGCOMM, [7] Srikanth Kandula, Sudipta Sengupta, Albert Greenberg, Parveen Patel, Ronnie Chaiken, The Nature of Data Center Traffic: Measure- ments and Analysis. in Proceedings ACM IMC, 2009 [8] Anees Shaikh, Jennifer Rexford, and Kang G. Shin, Load sensitive routing of long-lived ip s, in Proceedings of ACM SIGCOMM, [9] Kulvinder Singh, Router buffer traffic load calculation based on a TCP congestion control algorithm, International Journal of Computational Engineering & Management, Vol. 15 Issue 1, [10] Wang, Xiaoming, Parish, David J.;, Optimized Multi-stage TCP Traffic Classifier Based on Packet Size Distributions, Third International Conference on Communication Theory, Reliability, and Quality of Service (CTRQ), June 2010