Engineering Traffic Uncertainty in the OpenFlow Data Plane

Transcription

1 Engineering Traic Uncertainty in the OpenFlow Data Plane Fei Chen, Chunming Wu, Xiaoyan Hong, Zhouhao Lu, Zhouhao Wang, Changting Lin College o Computer Science and Technology, Zhejiang University Department o Computer Science, University o Alabama ei chen, wuchunming, bctlzh, wzh1994, linchangting}@zju.edu.cn hxy@cs.ua.edu Abstract This paper is driven by a simple question o whether traic engineering in Sotware Deined Networking (SDN) can react quickly to bursty and unpredictable changes in traic demand. The key challenge is to strike a careul balance between the overhead (requently involving the SDN controller) and perormance (the degree o congestion measured as the maximum load and the balance between the minimum and the maximum loads). Exploiting OpenFlow (OF) eatures, quick shit o routing paths or unpredictable traic bursty is the ocal point o this work. It is achieved by using a dual routing scheme and letting the data plane to select the appropriate path in reacting to uncertainty in traic load. The proposed work is called DUCE (Demand Uncertainty Coniguration selection). Further, we describe a traic distribution model, an optimization solution that calculates congestion-ree traic distribution plan which guarantees that each switch can select one o the paths in a distributed way, and moreover, OF details about detaching the unctionality o responding to the demand uncertainty rom the control plane and delegating it to the data plane. Simulations are perormed validating the eiciency o DUCE under various network scenarios. I. INTRODUCTION A. Traic Engineering in SDN and Demand Uncertainty The simplicity o SDN can alleviate the complexities o traditional Traic Engineering (TE) mechanisms [1]. The latter oten relays on proactive methods, which irst collect a set o traic demands, and then compute optimal routes or these traic demands. A straightorward use o SDN has been a centralized traic engineering paradigm. Existing work has shown that centralized TE reduces network congestion and increases eiciency [2] [6]. In such systems, a TE controller typically takes the orm o a control loop that measures the current traic demand and network topology, solves an optimization problem, and then reconigures the network to match current traic demand. While centralized TE can be highly eective, it is still worthy to ask the question o whether centralized TE can work eectively in the presence o bursty and unpredictable changes in traic demands (reerred to as demand uncertainty). Understanding the characteristics o the traic demand is the key to eective and eicient utilization o link capacities [4], [7]. Theoretically, i the traic demand is predictable, optimal routes can be obtained to organize the traic lows in the network to minimize network utilization [8]. However, this approach can only be used i the traic demand is stable. Unortunately, traic can be highly dynamic and may contain unpredictable shits. Prior studies [1], [9], [10] have shown that the shits in traic may leave no time or a proactive TE algorithm to re-compute or adjust. Consequently, packet losses occur due to congestion. One way to deal with the unpredictable traic shits is oblivious routing [7]. In oblivious routing, a robust routing or a class o traic demand is computed, and thus has the potential to handle traic spikes well. A potential drawback o oblivious routing is that optimizing or the worst-case perormance may incur a high cost when traic is predictable and stable, which may account or a majority o time. These early work show that it is desirable or centralized TE to have strategies able to capture the uncertainty in traic demands and to balance the traic load accounting or the uncertainty with adequate link utilization. B. Distributed TE in SDN and Implications Given a TE strategy dealing with demand uncertainty, we can plot its cost (e.g., the requency o involving the SDN controller) and perormance (e.g., the eiciency o responding to congestion) on a 2-D plane. The desirable strategy, i.e., real-time strategy, will require constantly monitoring network conditions and rapidly reacting to problems [2], [3], [11]. As an example, load balancing via a controller involves timely collecting statistics about lows in the data plane [2], then adaptively rerouting traic rom over-utilized to under-utilized paths. However, requent low installation and network-wide statistics collection, may yield signiicant overhead on both the control plane and the data plane, and consequently limit the scalability o SDN [12] [14]. Implication: we should adjust network conigurations when demands shit while reducing the overhead on the control plane. A natural strategy to reduce overhead would be a distributed one, which moves control unctionalities toward distributed control planes [12], [15], where each controller has a partial view o the network. However, applications, such as traic engineering that requires an up-to-date view o the network to optimize their objective unctions, would have to cope with a network o distributed controllers [16]. Implication: we should deal with demand uncertainty in traic demand in a distributed and simple way that does not require collaborating and exchanging inormation with each other. The main problem is how to deal with unpredictable traic spikes in a timely way with less control traic overhead. Inspired by oblivious routing, we can proactively compute a routing or normal traic demand and an alternative routing or unpredictable traic demand, in a way that the latter can handle demand uncertainty. The basic idea, thus, is to reduce the time needed in responding to the sudden changes o traic load with minimum control overhead.

2 C. Proposed Approach We attempt to ind a balance between the requency o involving the SDN controller (i.e., control overhead) and the eiciency o responding to congestion (i.e., network perormance), synthesizing existing routing strategies or SDN. Our approach is to consider both proactive and reactive methods as complementary with each other. The proactive methods, e.g., prediction-based TE and oblivious TE, ocus on minimizing the maximum link load observed over a period o time. However, they rely on precise knowledge o the traic demand, and thus may not prepare or handling demand uncertainty. The reactive methods, e.g., real-time routing and distributed control plane, attempt to adjust routing conigurations as demands shit. However, their eectiveness is tied to how quickly they adapt to changing demands. The key method we use is to delegate the control unction o responding to demand uncertainty to the data plane, with the guideline presented by SDN controller. Thus, while traic lows are passing through the switches, reactions to changes in traic demands will occur directly without passing traic statistics to the controller and waiting or a low table update. Based on this insight, we propose a new scheme called DUCE (Demand Uncertainty Coniguration selection). DUCE will solve the ollowing two problems: Can responding to bursty and unpredictable changes in traic demand be done in the data-path without adding new data-path mechanisms to the switches? How such a distributed data-path-based approach achieve acceptable perormance globally? For the irst problem, the underlying issue is that OpenFlow (OF) switch under SDN is dumb. They cannot dynamically change the proportion o traic that is routed along each path according to the network states. They only explicitly react to commands rom the controller. When bursty traic occurs, they may ail to adjust network conigurations in time, and thus cause serious network congestion. To solve this problem, DUCE uses a simple yet eective way: perorming only a selection unction at each OF switch based on traic marks. Speciically, DUCE ully exploits the eatures o OpenFlow, including the meter table and group table. The ormer measures the rate o packets and remarks the DSCP (Dierentiated Services Code Point) ield. The latter provides the ability to deine multiple orwarding behaviors according to the DSCP mark. The issues relating to measure the bandwidth consumed by each low and to make decisions based only on these local states will be studied in this paper. For the second problem, the underlying issue is that there is a potential risk o adjusting the traic distribution leading to network congestion, because the adjustment o traic distribution is perormed at each switch in an online and distributed ashion. The distributed adjustment has the advantage o responding quickly to changes in traic, but is limited by lacking o global knowledge o the network state. The oblivious method may have the over-provisioning problem i they attempt to ully cover real-time traic spikes with ixed bandwidth reservation in their oline approach. To this problem, instead o providing a single routing or all possible uncertain scenarios, DUCE precomputes and uses two routes or each low in the network: one route is or normal traic conditions and the other is or unpredictable conditions. The second route is a congestion-ree traic distribution plan, which guarantees that each switch can adjust the distribution o its lows in a distributed way and irrespective o the order in which the distributions are adjusted. The route calculation directs the unpredictable low demands to links that can accommodate their bandwidth requirements. The solutions to aorementioned irst problem allow the installations o these routing conigurations to the switches, so the switches handle traic uncertainty in the data plane. In this paper, we study issues relating to estimating traic load and compute the congestion-ree routes. DUCE is evaluated through simulations. First, we compare DUCE to standard oline TE approaches, i.e., prediction-based TE and COPE, based on Abilene topology and traic traces, and show that it can eiciently utilize network bandwidth. Second, we compare DUCE to an online TE approach which periodically shits lows to least loaded paths, and show that with bursty traic demand, the traic distribution under DUCE is stable and load balancing is achieved. This paper makes the ollowing key contributions. Propose the idea o detaching the unctionality o responding to demand uncertainty rom the control plane and delegating it to the data plane. Moreover, exploit the eatures o OF to make it happen, so to achieve the goal o enabling ast reaction to uncertainty in traic load. Use a dual routing strategy at the data plane which handles normal case and bursty case separately by two separate policies. Moreover, present an optimization solution that calculates congestion-ree traic distribution plan which guarantees that each switch can select one o the policies in a distributed way and irrespective o the order in which the distributions are adjusted at dierent switches. Evaluate DUCE with comparisons with both oline optimal TE schemes and an online greedy TE scheme. Comparisons to the ormer show that DUCE can reduce maximum load on bottleneck links, while keep the degree o imbalance over all the links low. Results also show that DUCE ollows the traic demand changes quicker by showing low requency o traic luctuations. The rest o the paper is organized as ollowings. Section II presents case studies about traic uncertainty using Abilene data. Section III introduces the two key ideas o DUCE, namely, selection unction and traic distribution plan. Section IV describes the model and optimization problem or the alternative path that handles traic uncertainty. Section V presents simulation based evaluations. Sections VI and VII are related work and conclusions respectively. II. MOTIVATION Our work is motivated by the ollowing observations, or examples, the demand uncertainty occurs requently in the network, and TE solutions have to consider how to minimize the impact o demand uncertainty on a speciic perormance goal, such as minimizing bandwidth consumption and balancing the load distribution. A. Demand Uncertainty Understanding the characteristics o the traic demand is the key to eicient utilization o link capacities. We briely

3 Demand rom Los Angeles to Chicago (Mbps) Time interval (5min) (a) Traic dynamics CDF dynamic peak multi Relative error (b) CDF o relative error Fig. 1: Traic dynamics in Abilene data (a) and relative error between the predicted demand and the real demand (b). present the properties o Abilene traces [17] and MapReduce bandwidth requirement, highlighting speciic eatures that inluence DUCE design. First, we examine the Abilene (Internet2) data to show the traic demand uncertainty. Figure 1(a) shows the existence o variations in traic demand over time (the red solid line) and huge traic spikes while the rest remains relatively stable most o the time (the blue dashed line). Second, we examine the time-varying traic o MapReduce in Figure 8(a) according to the traic pattern depicted in [18]. The igure shows that using a ixed-bandwidth reservation can potentially waste the resources o the datacenter. These observations validate the need to quickly react to demand uncertainty, and the need to avoid over-provisioning. In this work, we strive to achieve a balance between the two while also to oer optimization or bursty traic. B. Analysis o Existing Methods We classiy exiting TE approaches in terms o the availability o traic demand or the timescale o operations in two classes: oline TE and online TE. The oline TE class conigures the set o link weights that minimize the maximum network utilization with an available and ixed traic demand matrix. In contrast, the online TE class sets link weights adaptively based on current network conditions, reacting quickly to demand uncertainty. Most o the proposed oline TE methods are predictionbased. The prediction-based TE methods can be urther categorized into the ollowing classes [1]: (1) dynamic, methods based on the traic demand in the previous interval; (2) peak, methods based on the traic demand in the peak interval (in terms o the total volume o traic) o the previous day and the same day o the previous week; and (3) multi, methods based on a set o traic matrices in the previous day and the same day o the previous week. We would like to understand the gap between predicted traic demand and real traic demand. Figure 1(b) shows a comparison in terms o relative error (in the -1 to 1 range) between predicted traic demand (e.g., dynamic, peak, and multi) and real traic demand or the Abilene traic trace. For the multi method, we plot the minimum relative error or each IE-pair. The igure shows that both peak and multi methods have high relative error. For example, the relative error greater than 0.5 or the peak and multi approaches are 52.5% and 31.4% respectively. As a result, peak and multi methods typically require bandwidth over-provisioning. On the other hand, the dynamic method has a lower raction above the 0.5 threshold (nearly 10%) than peak and multi methods. In act, the raction o relative error in the range o [-0.2, 0.2] constitutes 63%. Thus, the dynamic method has a tight relative error, and may ail to cope with bursty and unpredictable changes in traic demands. To account or the gap between predicted traic demand and real traic demand, another type o oline TE methods, which reerred to as oblivious routing, computes routes that are optimized or the worst-case perormance. A potential drawback o oblivious routing is that optimizing or the worstcase perormance may incur a high waste when traic is predictable and stable, which may account or a majority o time. COPE (cope) is proposed to combine the best o both prediction-based TE and oblivious routing. It optimizes routing based on the convex hull constructed rom the set o traic matrices collected rom the previous day and the same day last week, subjected to a perormance-ratio penalty envelope on all nonnegative traic demands. Similar to multi, cope also requires bandwidth over-provisioning. The major weakness o oline TE is the lack o adaptive traic manipulation according to traic and network dynamics. In order to rapidly respond to demand uncertainty, online TE class typically perorms on a timescale o minutes or even milliseconds in order to evenly distribute traic within the network promptly. Online TE methods aim to dynamically split traic among multiple paths, i.e., decides on when and how to shit traic among the paths, based on the view o the load situation in the network. Speciically, an online TE solution takes the orm o a control loop that periodically polls the switches or collecting low statistics, and then tires to moves traic rom the over-utilized paths to the under-utilized paths until the utilization is balanced. However, since online TE mechanism is implemented at (logically) centralized controller, this would require continually distribute the rules across the aected switches to dynamically adaptive to time-varying network states. This orwarding rule installation process may take time and yield signiicant overhead at both the control plane and data plane. Indeed it is challenging to build an online TE scheme or SDN that respond quickly to changes in traic, yet requires no additional interactions between the controller and switches during demand uncertainty. Thereore, there is a need or an online TE method that combines practical implementation with clear perormance advantage. Again, the goal o our work is to rapidly respond to demand uncertainty by, perorming traic distribution on a timescale o minutes or even milliseconds. III. DUCE CORE IDEAS As analyzed above, the two undamental limitations (i.e., high latency and reduced scalability) o the existing SDNbased TE approaches stem rom the act that they process the demand uncertainty in the control plane. In this paper, we propose a new design called DUCE which deals with the demand uncertainty in the data plane. DUCE is based on two key ideas. The irst idea is to perorm a selection unction at each OF switch to adjust traic distribution in an online and distributed ashion. However, adjusting the traic distribution locally without incurring network congestion is challenging. Thus, our second idea is to combine the selection unction with a traic distribution plan, which guarantees that adjusting traic distribution locally will not incur network congestion.

4 Algorithm 1 Path selection At each interval o t: 1: Calculate the moving average rate H or every low X. 2: I rate H is less than a pre-deined threshold value T, select route r = 0. Otherwise, select route r = 1. 3: I route is ind, using the corresponding weight to conigure path selection unction Q. At each packet o low X arrival: 4: Forward packet to path p using traic splitting algorithm. 5: Record the number o packets that have passed through or each time interval o t. Selected route r= 1 >0 H T ( ) Flow X Route Selection S Rate Counter H A. Selection Function Route set R route 0 route 1 Path Selection Q Weight set W Fig. 2: Path selection. Path set P p0 p1 p2 For the selection unction, three components are added to the switches as shown in Figure 2. First, a rate counter, which monitoring the rate o incoming lows and determining whether there is a spike or not. Then, a route selection, when there is a spike in traic demand, it will switch to alternative route. Finally, a globally optimized route table, provides two routes or each source-destination pair. One or normal traic conditions and the other or unpredictable conditions. As a result, load assigned on each path can be adapted to dynamic changes o the traic condition. Selection unction is supported using path selection as shown in Figure 2. The input to the path selection unction Q is each packet o low X. The output is a path to which the packet should be assigned according to the load. Path selection or each low is decided independently. The path selection unction can be implemented by using a weight-based traic splitting algorithm (e.g., WRR, THR, etc [19]). Each path is associated with a weight that determines how much traic load should be sent via that path as compared with other paths, i.e., traic splitting ratio. The weights among the available paths are adjusted according to real-time low rates by the route selection unction S. The assignment o weight is based on the arrival rate o low X: whenever the rate o low X exceeding a predeined rate threshold T, the weight will be assigned with route 0; otherwise, it will be assigned with route 1. When a low arrival rate changes, the weight is adjusted. Thereore, load assigned on each path can be adapted to dynamic changes o the traic condition. Congestion induced by bursty and sudden spikes in traic rate can be mitigated. The value o low rate H can be ound by using an exponentially weighted moving average unction: H = (1 α) H old + αh new. The value o T determines the requency o updating weight assignment. The procedure or this basic idea is summarized in Algorithm 1. First, the weight assignment will be updated at each time interval t: calculates the moving average rate or low X (Step 1), determines which weight to be chosen (Step 2) and updates the weight assignment (Step 3). Second, when a B A E1 10 D C E2 10 I I2 10 E1 20 E2 (a) common case N M B A E1 13 D C E I1 I2 16 E1 26 E2 (b) bursty case N M B A E1 4 D C E I1 I2 6 E1 6 E2 (c) alternative routing Fig. 3: This example shows how to perorm a dynamic traic distribution through incremental router reconiguration. packet o low X arrives, it will be sent via a path selected according to its weight (Step 4). The corresponding counter will be updated (Step 5). B. Traic Distribution Plan We illustrate by example that how changes in traic lead to congestion and how to prevent the congestion through a careully designed traic distribution plan. Figure 3(a) shows a simple network where the capacity o each link is 15. The numbers besides the ingress switches are traic demands and the arrow indicates the traic direction. The number on each link is the traic load. This igure shows a normal traic demand and the optimal traic distribution where an oline algorithm uses a multi-commodity low linear programming to optimally split the traic among the links. For example, the load on link I1 A is 10, all the demand o I1 E1, while the load on link I2 C and I2 M are 10, hal o demand o I2 E2. No link is congested. Suppose there is a sudden spike in traic demand, as shown in Figure 3(b). The demand o I1 E1 and I2 E2 increases to 16 and 26, respectively. However, applying the routing plan o common case to the bursty traic demand, i.e., the splitting ratio used in Figure 3(a) to the case in Figure 3(b), will break the load balance o the network. As a result, link I1 A will have a load o 16, exceeding its capacity. To prevent the preceding congestion, we need to change the routing, i.e., the traic split ratio on both I1 and I2. While a traditional solution is to change the splitting ratio o the common case in order to handle the bursty case when it occurs, our approach is to introduce a separate traic distribution plan or increased demand. Figure 3(c) shows such an alternative routing, where I1 splits traic by 4:2 and I2 splits traic by 2:4. It is easy to veriy that no link is congested. IV. DUCE DESIGN DUCE separates the unctionality o responding to demand uncertainty rom the control plane and delegates to the data plane. Thus, DUCE consists o two layers as shown Figure 4: the irst layer, traic distribution plan (TDP), computes routing or the common case demand and the unpredictable case demand; while the second layer, route selection (RS), achieves dynamic transition between two dierent routes (orwarding behaviors). TDP uses a simple model to capture the traic demand o each low : the traic demand o is T at common case, while it is upper bounded by Φ. Instead o optimizing the routing or all possible traic demand, TDP seeks to provide dierent routings or dierent traic demand. Given a predicted traic demand T and a possible peak traic demand N M

5 Traic Distribution Plan Layer Modeling Demand Uncertainty T,Φ low ( ) Monitoring Rate T / Φ Route Selection Layer Conining Overall Penalty ( ε,π ) conigure Selecting Route a / b Control Plane Computing Traic Distribution a,b Enorcing Route e.g., WCMP ( ) port o Data Plane Fig. 4: The high-level working process o DUCE. Φ, TDP will attempt to compute a common case traic distribution a or T and a worst case traic distribution b or Φ T. Considering that optimizing perormance or worst case may come at the expense o poor perormance or common case, TDP ensures the perormance o routing a to be slightly higher than optimal under common case, speciied by parameter ε; and uses the parameter π to control how ar the worst case traic demand is away rom the common case. The TDP layer eeds the computed routing a and b to the RS layer. For each low, the RS layer on the OF switch uses a rate-limiter to limit the traic. The rate-limit is assigned by the TDP layer, i.e., T and Φ. RS aims to transition rom a to b when traic rate changes rom T to Φ. Speciically, when the rate o low is greater than T, RS will change the corresponding rules rom a to b on involved switches. To enorce traic distribution (i.e., a or b ), Weighted-Cost-Multi-Path (WCMP) is used to direct a matching packet to an output port. DUCE leverages the hash unction (i.e., WCMP) on the OF switches. Doing so, DUCE can adjust the bandwidth among dierent lows to achieve high network utilization, while avoiding the congestion caused by uneven low assignment. A. Network Model We irst describe a network model under which we ormally deine the traic demand as the input to DUCE TDP layer. 1) Traic Demand: Let G = (V, E) be a network under consideration, where V is the set o switches, and E is the set o links connecting the switches. Let c e or c ij denote the capacity o a directed link e ij rom switch i to switch j. Let F = } be the set o lows aggregated by ingress-egress switches. A low enters the network rom an ingress to an egress switch through one or multiple paths (H ). A traic demand T deines the size o each low T. 2) Traic Distribution: Let a,h be the bandwidth allocated or low on path h. A traic distribution, denoted by A, is speciied by a set o values a,h F, h H }. Given a traic demand T, the traic distribution A can be computed by using a multi-commodity low linear program, as ollows: min MLU T (1a) subject to : F,h H e E : a,h δ [h, e] MLU T (1b) c e F : h H a,h = T F, h H : 0 a,h T (1c) (1d) Where the binary indicator δ [h, e] denotes i path h traverse link e. The objective in the ormulation is to minimize the max- Rate (Mbps) real dynamic absolute error bursty Time interval (5min) Fig. 5: The bursty line captures the burst traic demand when using predicted traic demand (dynamic) to approximate the real traic demand (real). imum link utilization (M LU). The second condition speciies that all demands o a low should be routed. 3) Flow Rule: Given a traic distribution A, we can compute the rule set R or A. Let r ij be the rule or low on link e ij, where r ij = / h H a,h δ [h, e ij ] T. A rule } set R is speciied by a set o values r ij F, e ij E. B. Modeling Demand Uncertainty We assume that the demand o each low is unknown and varies with time, but that always lies inside an explicitly deined region, i.e., [T, Φ ]. Note that Φ is the bound or the unpredictable traic, and independent o time; i we have a better understanding o the traic, we can have a tight bound. Let θ be the set o time when there exists changes in low. Given θ, we can model demand uncertainty o low F as ollow: X : t θ, X (T, Φ ] (2a) S (t) = T : otherwise (2b) Function 2 attempts to model demand uncertainty into pulse unctions with varying height and width peaks. To see how this model capture traic demand uncertainty, we examine the Abilene data. Figure 5 plots the real traic demand and predicted traic demand (by using dynamic method) rom Los Angeles to Indianapolis. To capture the traic demand, we can use the predicted traic demand as the lower bound, i.e., T ; and use the absolute error o dynamic method as the unpredictable demand, i.e., X T, where absolute error is deined as the predicted demand minus the real demand. Since the value o X T in our model is always greater than 0, we then set the value o X T as max absolute error, 0} (the bursty line in Figure 5). However, Function 2 needs prior knowledge on when bursty will happen. To reduce this assumption, we introduce a vector ϕ (t) = [ϕ t, F ] that indicates the status o each low at time t, where ϕ t, = S (t)/φ ; and a slave variable π that bounds the total number o changes in lows. Then we covert unction 2 into the ollowing constraint: X : ϕ (t) S (t) = 1 π, X (T, Φ ] (3a) T : otherwise (3b) where ϕ (t) 1 = F ϕ t,. C. Computing Traic Distribution The basic idea is that we do not change the distribution o a low across its available paths as long as the bandwidth consumed by this low below a certain threshold, i.e., T. Only

6 when the bandwidth consumption o this low exceeds T, will the distribution or it be adjusted. The main reason or limiting the number o routings to 2 is that we want to limit the amount o rules installed in the memory used in OpenFlow switch. A key challenge is to implement these adjustments without causing congestion. The underlying problem is that the adjustment o traic distribution is perormed at each switch in an online and distributed ashion. This has the advantage o responding quickly to changes in traic, but is limited by lacking o global knowledge o the network state. Hence, there is a risk o collision, moving traic to over-utilized paths, and thus leading to congestion. To avoid congestion distribution adjustments, DUCE computes a congestion-ree traic distribution plan, which guarantees that each switch can adjust the distribution on its lows in a distributed way and irrespective o the order in which the distributions are adjusted. To compute such a distribution plan, we irst remove the time variable in Function 3. Since both Φ and T are independent o time, we can convert demand uncertainty model into time independent demand set as ollow: Lemma 1: Function 3 can be convert into the sum o T and π = = X T X [T, Φ ], X } T π F Φ T where T is normal traic demand, and π is bursty demand. Proo: From the deinition o ϕ t, and S (t), we have: X T Φ T X = S (t) = ϕ t, (4) Φ Φ Thus, captures the increased demand o low. Then we can convert Function 3 into: S (t) = T +. Given π, we denote B the traic distribution or traic demand π, where B = b,h F, h H, } h H b,h =, b,h 0. Then, we can replace constraints 1a and 1b with: min MLU S (5a) F,h H e E : (a,h + b,h ) δ [h, e] MLU S (5b) c e To solve (5), we then bound the load on each link oered by increased demand with the ollowing lemma. Lemma 2: Let ψ e = max b,h δ[h, e], i the π distribution a,h, b,h } F,h H can guarantee that: e E : a,h δ [h, e] + ψ e c e, then each switch can independently adjust its distribution on its lows without causing congestion. Proo: Denote ω,h = b,h / h H b,h. Let d and d h be the instantaneous rate o low and path h respectively. When d exceeds T, i.e., T d Φ, the switch adjusts the distribution o low rom a,h to b,h, the traic load o low on a path h is: d h = a,h + (d T ) ω,h = a,h + ω,h (6) which is directly derived rom the deinition o π, i.e., π : (d T ) π. Thereore, the total traic on a link e is: d h δ [h, e] a,h δ [h, e] + ψ e c e which inishes the proo. Using linear programming duality, we can show that the ψ e ϑ i and only i the ollowing set o constraints can be satisied: µ e, + λ π ϑ (7a) F F : µ e, + λ e b,h δ [h, e] Φ T h H (7b) F : µ e, 0 (7c) λ e 0 (7d) The variables µ e, and λ e are dual multipliers on the constraint Φ T and F /(Φ T ) π, respectively. We can then covert (1) into a set o linear constraints as ollowing: min MLU S (8a) subject to : e E : a,h δ [h, e] + F µ e, + λ π MLU S c e (8b) F,h H e E : a,h δ [h, e] ε MLU T (8c) c e e E, F : µ e, + λ e Φ T h H b,h δ [h, e] (8d) F : h H a,h = T ; h H b,h Φ T F, h H : 0 a,h T ; 0 b,h Φ T F, e E : µ e, 0 e E : λ e 0 (8e) (8) (8g) (8h) Constraint (8c) bounds the penalty on normal traic demand by ε MLU T, and (8e) guarantees that all the traic o T is ully delivered and total traic delivered should not exceed Φ. Constraint (8b) guarantees the worst case perormance. D. Route Selection and Implementation The output o DUCE TDP layer is a sequence o traic distributions, each o which can be implemented by installing its corresponding low table entries into the switches. Given a traic distribution A, we then compute the rule set R 1 or A. As deined beore, the } rule set R 1 is speciied by a set o values r ij F, e ij E, where r ij = / h H a,h δ [h, e ij ] T. Then the rule set should be installed } into a switch i is given by r ij F, j V, e ij E. Similarly, we compute a rule set R 2 or distribution B, where R 2 = r ij = } h H b,h δ [h, e ij ]/ h H b,h e ij E. We assume that each switch has a group table and a meter table. Rules in the group table include an ordered list o action buckets. Each action bucket contains a set o actions to execute, and provides the ability to deine multiple orwarding

7 Algorithm 2 Route Selection on Switch i When switch i receives a packet o low : 1: A meter measures the rate o low. 2: I current rate is greater than T, remark the DSCP ield. 3: I DSCP is remarked, select route R 1 ; otherwise, R 2. 4: I route is R 1, select outport 1; otherwise 2. MPB (%) dynamic duce cope (a) MPB CDF cope dynamic 0.2 duce Absolute relative error (b) CDF o absolute relative error Table 0 Match Instructions SrcPrx = I1, DstPrx = E1 Meter 1, Table 1 Match Table 1 SrcPrx = I1, DstPrx = E1, DSCP = 0 SrcPrx = I1, DstPrx = E1, DSCP = 1 Group ID 1 2 WCMP 2/3 Instructions Group 1 Group 2 Group Type Bucket1 Weight Bucket2 Weight WCMP 0.0 Meter Table Meter ID Band Type Rate Parameter 1 DSCP remark 10 1 Group Table Bucket1 Action Fwd A, Vlan = 1 Bucket2 Action Fwd C, Vlan = 1 1/3 Fwd A, Vlan = 2 Fwd C, Vlan = 2 Fig. 6: Implementing DUCE on OpenFlow switch. behaviors. r ij can be implemented on switch i by using group table with type WCMP. The meter table measures the rate o packets and remarks the DSCP iled. Algorithm 2 describes how a packet o low will be processed within a switch i. When a packet o low arrives, it will be processed as ollows: First, the meter table decides whether DSCP ield o this packet should be remarked (Step 1 and Step 2). Second, the group table select an outport, which depends on whether DSCP ield has been remarked. Figure 6 shows an example o how to implement DUCE on an OF switch. For simplicity, we have replaced the names o ports with the switches they are connected to, e.g., in place o the name o the port connecting I1 to A, we simply write A. The OF pipeline processing always starts at the irst low table (i.e., Table 0): the packet must be irst matched against low entries o low table 0. It consists o a match that speciies packet attributes (e.g., packet aggregated by ingressegress switch I1 E1) and a list o instructions that speciy how to process matching packets. In this case, the rule states that all packets coming rom ingress switch I1 to egress switch E1 should be processed by the entry with identiier 1 in meter table and then processed by Table 1. Thus, when a packet arrives, it is irst processed by meter band with type DSCP remark speciied by the 1-the meter entry, which will increase the value o DSCP to 1 i the current rate is higher than 10. Then, the packet header is matched against low entries o low table 1. Table 1 determines which orwarding behavior (deined by group table) to adopt by using DSCP ield. Finally, group table uses the WCMP group type to orward the packet to the next hop. As a result, the orwarding behavior o the packet is adjusted when the low arrival rate changes. For ease o explanation, we assume that the deault value o DSCP iled in the header o each packet is 0. It can be easily extend to general cases. V. EVALUATION In this section, we evaluate DUCE by simulations. We irst compare DUCE with oline TE approaches and show that it can eiciently utilize network bandwidth. Then we compare DUCE with an online TE approach and show that within a network that has bursty traic demand, the traic distribution Maximum link load (100Mbps) cope dynamic duce Time interval (5min) (c) Time series o maximum link load Fig. 7: Comparison with oline TE. under DUCE is stable and load balancing is achieved. A. Comparison with Oline TE 1) Traic demands and topologies: Similarly to [1], we use the real topology and traic demand o Abilene [17] or comparison with oline TE methods. 2) Perormance metrics: We use the ollowing two perormance metrics to compare dierent oline algorithms: maximum provided bandwidth (MPB), deined as ( ) MP B A = max e E a,h δ [h, e] (9) and maximum link load [10], deined as ( ) MLL A, D a D = max,h δ [h, e] (10) e E h H a,h where D is the set o real traic demand o each low D. Given the maximum provided bandwidth MPB and the maximum link load MLL, the maximum link utilization is given by MLU = MLL/MP B (or simplicity we assume here that the capacity o each link is given by MPB). Instead o using MLU as perormance metric, we use MLL and MPB separately because each reveals unique perormance aspects that matter to DUCE design. Speciically, MPB captures the ability o a TE scheme or capacity planning (i.e., bandwidth reservation), while MLL captures the extent o congestion given the MPB. Minimizing the MPB leads more available bandwidth, which can be used by other lows. Conining the MLL helps reducing the occurrence o packet losses and retransmissions. 3) Alternative algorithms: We consider dynamic, peak, multi, and cope, as discussed earlier. We solve the corresponding linear programming problems with MOSEK [20]. Our simulations show that both peak and multi algorithms perorm poorly, while dynamic achieves close-to-optimal perormance most o the time. Thus in the ollowing presentations, we omit the results o both algorithms, and report only the results o dynamic and cope algorithms. 4) Bandwidth provisioning: Figure 7(a) compares the algorithms in terms o MPB (i.e., reserved bandwidth). Since cope attempts to reduce the worst-case penalty, and thus requiring a high bandwidth provisioning ( 16% o link capacity); while duce is only slightly higher than dynamic.

8 5) Maximum link load: The traic o 10 May is used as the input o the maximum link load or the comparisons o the algorithms. Figure 7(c) plots the maximum link load versus the time interval. For clarity, we zoom in to a ew intervals in the day. We make the ollowing observations. First, duce show lower MLL than dynamic at most o the time intervals. As we saw beore (Figure 1(b)), the predicted error greater than 0.2 o dynamic can be as high as 36.5% by combining 16.1% (relative error less than -0.2) and 20.4% (relative error greater than 0.2), thus resulting a higher perormance penalty than duce. Second, the maximum link load o duce and cope are close beore the time interval o 140. However, ater the time interval o 140, duce achieves better perormance than cope. For clarity, we plot the CDF o absolute relative error between MPB and MLL or these algorithms, as shown in Figure 7(b). The absolute relative error o a given MLL with respect to a certain MPB measures how ar is the MLL rom being optimal, i.e., minimizing the impact o demand uncertainty given the constraint o bandwidth consumption. It is deined as the relative error between MPB and MLL divided by the MPB. Formally, the absolute relative error is (MLL MP B)/MP B. Correspondingly, the closer to 0 the value o absolute relative error means the better o the perormance. We can ind that duce has better perormance in terms o absolute relative error than both dynamic and cope. In summary: (1) For the same traic demands, duce can deal with traic uncertainty compared to cope, but requires only one-ourth bandwidth provisioning. (2) For the same bandwidth provisioning, duce can adapt to traic uncertainty when dynamic cannot. B. Comparison with Online TE 1) Traic demands and topologies: We use a simple topology as shown in Figure 3, where link (C,D) is shared between paths rom pairs (I1,E1) and (I2,E2). Thus, there is an interaction between the pairs, which captures the main eature o online algorithms. We write a packet level discreteevent simulator in Python to conduct this experiment. Figure 8(a) shows the packet arrival rates in our experiments, which is generated according to the traic pattern depicted in [18], which captures the bandwidth requirement o MapReduce appellations with a coarse-grained pulse unctions with dierent widths and heights. For clarity, we only show a ew intervals rom 20s to 80s o our experiments. 2) Alternative algorithms: We consider a greedy online algorithm (greedy): At each time interval t, or each low, greedy searches all possible paths linearly to ind one which is least loaded. I such a path is ound, then that low is placed on the path. This algorithm models online load balancing schemes used in MATE [10], TeXCP [9], Hedera [2], and CONGA [21]. 3) Perormance metrics: We use the ollowing two metrics to compare duce with greedy: throughput imbalance, deined as the maximum throughput minus the minimum divided by the average, i.e., (M AX M IN)/AV G; and stability, deined by the requency o load changes. 4) Overall perormance: Figure 8 compares the algorithms in terms o link load. Figure 8(b) shows that as traic demand CDF duce greedy-0.1s greedy-0.5s greedy-1s Throughtput Imbalance (MAX-MIN)/AVG(%) (a) Extent o imbalance Fourier transorm o throughtput imbalance(%) duce greedy 1s greedy 0.5s (A,B) (C,D) (b) Standard deviation duce greedy-0.5s greedy-1s Frequency (c) Fourier transorm o throughput imbalance Fig. 9: Throughput imbalance and stability. (M,N) changing, greedy with the requency o 1s cannot react to changing demand. It is expected that greedy method increases the requency o path adjustment. As shown in Figure 8(c), when greedy increases its requency to 0.5s, the perormance o greedy improves signiicantly. In contrast, duce (Figure 8(d)) reacts in realtime to the actual demand and perorms comparably with greedy o 0.5s. It means that greedy need to run every 500ms to approach the perormance o duce. However, rerouting traic to the least loaded link too requently may cause oscillations. Figure 9(b) shows the standard deviation o load on each link. The result indicates that duce can achieve comparable and better perormance with greedy or 0.5s and slower update requencies while prevent oscillations. 5) Throughput imbalance: Figure 9(a) shows the CDF o the throughput imbalance across the 3 links (e.g., (A,B), (M,B), (C,D)). This is calculated during the time interval (10s, 100s). The results show that duce is slightly better than greedy with 0.5 seconds adjustment interval. 6) Stability: To quantiy the stability, we use the Fourier transorm to convert the results o throughput imbalance into the requency domain, see Figure 9(c). The igure shows that the oscillation requencies o duce are closely centered together while those o the two cases o greedy are scatted widely. The observation suggests that duce produce more stable traic lows than the greedy algorithm. In all, considering achieving the same throughput imbalance, duce can produce stable lows than greedy. Moreover, duce can achieve acceptable perormance under bursty traic. VI. RELATED WORK For brevity, we ocus below on centralized low management techniques supported by SDN. Existing traic engineering schemes can be mainly classiied into two types: the proactive methods and the reactive methods. Each type o method achieves a dierent tradeo between data-plane perormance and control-plane overhead. Reactive. The reactive methods will constantly monitoring the network conditions and reacting to traic changes accordingly. Hedera [2] detects elephant low at the edge switches and inds an alternative path with suicient capacity or each elephant low. The perormance o Hedera is constrained by

9 Demand (Mbps) Demand (I1,E1) Demand (I2,E2) Time (s) Link load (Mbps) Link (A,B) Link (M,N) Link (C,D) Time (s) Link load (Mbps) Link (A,B) Link (M,N) Link (C,D) Time (s) Link load (Mbps) Link (A,B) Link (M,N) Link (C,D) Time (s) (a) demand (b) greedy 1s (c) greedy 0.5s (d) duce Fig. 8: Comparison with online TE. Synthesized traic demand (a) and overall perormance in terms o link load (b, c, and d). the rates and durations o lows. DevoFlow [3] delegates the handling o mice lows to the data plane, and thereore reduces the number o interactions between the controller and switches. Both Hedera and DevoFlow involve controller in maintaining an accurate and global view o the traic demands, and rerouting the traic. However, it is diers rom the existing work by perorming these all in the data plane, without involving the controller. Proactive. The proactive methods typically rely on prior knowledge o the traic demand observed over a period time. Prediction based methods are used to optimize the overall perormance over most o traic demands [1]. The oblivious routing [7] optimizes the worst-case perormance over all traic demands. MicroTE [4] perorms multipath routing by leveraging the short term and partial predictability o the traic demand. SWAN [5] ocuses on control when and how much traic each service sends. Both MicroTE and SWAN perorm traic distribution based on a global view o traic demands and use OF to coordinate scheduling. As a hybrid approach, DUCE proactively computes two routing plan: one or normal conditions and the other or unpredictable situations. However, its path selection is reactive, triggered by and perormed at the data plane. VII. CONCLUSION We have presented the design and evaluation o DUCE. DUCE delegates the unctionality o responding to the bursty and unpredictable changes in traic demands to the data plane. It does not require any changes to the OF switch by ully utilizing the meter table and group table. In comparison to the existing approaches, DUCE achieves a better utilization than standard oline TE approaches; and it also produces a more stable and load balanced network when the bursty traic demands occur. In our uture work, we plan to realize it in a deployed network testbed to urther validate its beneits. VIII. ACKNOWLEDGEMENT This research is supported by the National Basic Research Program o China (2012CB315903), the Program or Key Science and Technology Innovation Team o Zhejiang Province (2011R , 2013TD20), 863 Program o China (2015AA015602, 2015AA016103), the National Natural Science Foundation o China ( ), and the Research Fund o ZTE Corporation. REFERENCES [1] H. Wang, H. Xie, L. Qiu, Y. R. Yang, Y. Zhang, and A. Greenberg, COPE: Traic Engineering in Dynamic Networks, in SIGCOMM. ACM, Aug. 2006, pp [2] M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat, Hedera: Dynamic Flow Scheduling or Data Center Networks, in NSDI. USENIX Association, Apr [3] A. R. Curtis, J. C. Mogul, J. Tourrilhes, P. Yalagandula, P. Sharma, and S. Banerjee, DevoFlow: Scaling Flow Management or Highperormance Networks, in SIGCOMM. ACM, 2011, pp [4] T. Benson, A. Anand, A. Akella, and M. Zhang, MicroTE: Fine Grained Taic Engineering or Data Centers, in CoNEXT, Dec. 2011, pp. 8:1 8:12. [5] C.-Y. Hong, S. Kandula, R. Mahajan, M. Zhang, V. Gill, M. Nanduri, and R. Wattenhoer, Achieving High Utilization with Sotware-Driven WAN, in SIGCOMM, New York, Aug. 2013, p. 15. [6] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh, S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla, U. Hölzle, S. Stuart, and A. Vahdat, B4: Experience with a Globally-Deployed Sotware Deined WAN, in SIGCOMM, New York, Aug. 2013, p. 3. [7] D. Applegate and E. Cohen, Making Routing Robust to Changing Traic Demands: Algorithms and Evaluation, IEEE/ACM Transactions on Networking, vol. 14, no. 6, pp , Dec [8] A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexord, and F. True, Deriving Traic Demands or Operational IP Networks: Methodology and Experience, IEEE/ACM Transactions on Networking (ToN), vol. 9, no. 3, pp , [9] S. Kandula, D. Katabi, B. Davie, and A. Charny, Walking the Tightrope: Responsive Yet Stable Traic Engineering, in SIGCOMM, New York, Aug. 2005, pp [10] A. Elwalid, C. Jin, S. H. Low, and I. Widjaja, MATE: MPLS Adaptive Traic Engineering, INFOCOM, pp , [11] J. Rasley, B. Stephens, C. Dixon, E. Rozner, W. Felter, K. Agarwal, J. Carter, and R. Fonseca, Planck: Millisecond-scale Monitoring and Control or Commodity Networks, in SIGCOMM, Aug [12] S. Hassas Yeganeh and Y. Ganjali, Kandoo: A Framework or Eicient and Scalable Oloading o Control Applications, in HotSDN. ACM, 2012, pp [13] S. H. Yeganeh, A. Tootoonchian, and Y. Ganjali, On Scalability o Sotware-Deined Networking, Communications Magazine, IEEE, vol. 51, no. 2, pp , [14] S. Schmid and J. Suomela, Exploiting Locality in Distributed SDN Control, in HotSDN. ACM, 2013, pp [15] T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu, R. Ramanathan, Y. Iwata, H. Inoue, T. Hama et al., Onix: A Distributed Control Platorm or Large-scale Production Networks, in OSDI, vol. 10, 2010, pp [16] D. Levin, A. Wundsam, B. Heller, N. Handigol, and A. Feldmann, Logically Centralized? State Distribution Trade-os in Sotware Deined Networks, in HotSDN. ACM, 2012, pp [17] Abilene, yzhang/research/abilenetm/. [18] D. Xie, N. Ding, Y. C. Hu, and R. Kompella, The Only Constant is Change: Incorporating Time-Varying Network Reservations in Data Centers, in SIGCOMM, New York, Aug. 2012, p [19] S. Prabhavat, H. Nishiyama, N. Ansari, and N. Kato, On Load Distribution over Multipath Networks, IEEE Communications Surveys and Tutorials, vol. 14, no. 3, pp , [20] MOSEK, [21] M. Alizadeh, T. Edsall, S. Dharmapurikar, R. Vaidyanathan, K. Chu, A. Fingerhut, V. T. Lam, F. Matus, R. Pan, N. Yadav, and G. Varghese, CONGA: Distributed Congestion-aware Load Balancing or Datacenters, in SIGCOMM. ACM, Aug. 2014, pp