On the Fairness-Efficiency Tradeoff for Packet Processing with Multiple Resources

Transcription

1 On the Farness-Effcency Tradeoff for Packet Processng wth Multple Resources We Wang, Chen Feng, Baochun L, and Ben Lang Department of Electrcal and Computer Engneerng, Unversty of Toronto {wewang, cfeng, bl, ABSTRACT Mddleboxes are wdely deployed n today s networks. They apply a varety of complex network functons to transform, flter, and optmze ncomng traffc based on the payload of packets. These functons requre the support of multple types of resources, such as CPU and lnk bandwdth, for processng ncomng packets. Hence, a mult-resource packet schedulng algorthm s needed to allow flows to share these resources farly and effcently. However, unlke tradtonal far queueng where bandwdth s the only concern, we show n ths paper that farness and effcency are conflctng objectves that cannot be acheved smultaneously n the presence of multple resources. Ideally, a schedulng algorthm should allow network operators to flexbly specfy ther farness and effcency requrements, so as to meet the Qualty of Servce demands whle keepng the system at a hgh utlzaton level. Yet, exstng mult-resource schedulng algorthms focus on farness only, and may lead to poor resource utlzaton. In ths paper, we propose a new schedulng algorthm to acheve a flexble tradeoff between farness and effcency for packet processng, consumng both CPU and lnk bandwdth. Expermental results based on both real-world mplementaton and trace-drven smulaton suggest that tradng off a modest level of farness can potentally mprove the effcency to the pont where the system capacty s almost saturated. Categores and Subject Descrptors C.2.6 [Computer-Communcaton Networks]: Internetworkng General Terms Schedulng Theory Keywords Far Queueng; Mddleboxes; Farness-Effcency Tradeoff 1. INTRODUCTION Queueng algorthms determne the order n whch packets n varous ndependent flows are processed, and serve as a fundamen- Permsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. Copyrghts for components of ths work owned by others than ACM must be honored. Abstractng wth credt s permtted. To copy otherwse, or republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. Request permssons from permssons@acm.org. CoNEXT 14, December 2 5, 2014, Sydney, Australa. Copyrght 2014 ACM /14/12...$ tal mechansm for allocatng resources n a network applance. Tradtonal queueng algorthms [1, 9, 21, 28] make schedulng decsons n network swtches that smply forward packets to ther next hops, and lnk bandwdth s the only resource beng allocated. In modern network applances, e.g., mddleboxes [25, 27], lnk bandwdth s no longer the only resource shared by flows. In addton to packet forwardng, mddleboxes perform a varety of crtcal network functons that requre deep packet nspecton based on the payload of packets, such as IP securty encrypton, WAN optmzaton, and ntruson detecton. Performng these complex network functons requres the support of multple types of resources, and may bottleneck on ether CPU or lnk bandwdth [10, 13]. For example, flows that requre basc forwardng may congest the lnk bandwdth [13], whle those that requre IP securty encrypton need more CPU processng tme [10]. A queueng algorthm specfcally desgned for multple resources s therefore needed for sharng these resources farly and effcently. Farness offers predctable servce solaton among flows. It ensures that the servce a flow receves (.e., number of packets processed per second) n an n-flow system s at least 1/n of that t acheves when the flow monopolzes all resources, ndependent of the behavor of other rogue flows. The noton of Domnant Resource Farness (DRF) [14, 22] embodes ths solaton property, wth whch each flow receves approxmately the same processng tme on ts domnant resource, defned as the one that requres the most packet processng tme [13]. Effcency serves as another mportant metrc measurng the resource utlzaton acheved by a queueng algorthm. Hgh resource utlzaton naturally translates nto hgh traffc throughput. Ths s of partcular mportance to enterprse networks, gven the surgng volume of traffc passng through mddleboxes [27, 35]. Both farness and effcency can be acheved at the same tme n tradtonal sngle-resource far queueng, where bandwdth s the only concern. As long as the schedule s work conservng [38], bandwdth utlzaton s 100% gven a non-empty system. That leaves farness as an ndependent objectve to optmze. However, n the presence of multple resources, farness s often a conflctng objectve aganst effcency. To see ths, consder two schedules shown n Fg. 1 wth two flows whose packets need CPU processng before transmsson. Packets that fnshes CPU processng are placed nto a buffer n front of the output lnk. Each packet n Flow 1 has a processng tme vector 2, 3, meanng that t requres 2 tme unts for CPU processng and 3 tme unts for transmsson; each packet n Flow 2 has a processng tme vector 9, 1. The domnant resource of Flow 1 s lnk bandwdth, as t takes more tme to transmt a packet than processng t usng CPU; smlarly, the domnant resource of Flow 2 s CPU. To acheve DRF, the transmsson tme Flow 1 receves should be ap-

2 CPU Lnk CPU Lnk p1 p2 p3 p1 p2 p3 q1 p4 p5 p6 q1 p4 p5 q2 p (a) A packet schedule that s far but neffcent. p1 p2 p3 p4 p5 p6 p7 p8 p1 p2 p3 p4 p5 p q1 p Tme p9 p10... p8 q1p9... Tme (b) A packet schedule that s effcent but volates DRF. Fgure 1: An example showng the tradeoff between farness and effcency for mult-resource packet schedulng. Packets that fnshes CPU processng are placed nto a buffer n front of the output lnk. Flow 1 sends packets p1, p2,..., each havng a processng tme vector 2, 3 ; Flow 2 sends packets q1, q2,..., each havng a processng tme vector 9, 1. Schedule (a) acheves DRF but s neffcent; Schedule (b) s effcent but unfar. proxmately equal to the CPU processng tme Flow 2 receves. In ths sense, Flow 1 should schedule three packets whenever Flow 2 schedules one, so that each flow receves 9 tme unts to process ts domnant resource, as shown n Fg. 1a. Ths schedule, though far, leads to poor bandwdth utlzaton the lnk s dle for 1/3 of the tme. On the other hand, Fg. 1b shows a schedule that acheves 100% CPU and bandwdth utlzaton by servng eght packets of Flow 1 and one packet of Flow 2 alternately. The schedule, though effcent, volates DRF. Whle Flow 1 receves 24/25 of the lnk bandwdth, Flow 2 receves only 9/25 of the CPU tme. The farness-effcency tradeoff shown n the example above generally exsts for mult-resource packet schedulng, but t has receved lttle attenton before. Exstng mult-resource queueng algorthms focus solely on farness [13, 31, 34]. However, for applcatons havng a loose farness requrement, tradng off a modest degree of farness for hgher effcency and hgher throughput s well justfed. In general, dependng on the underlyng applcatons, a network operator may wegh farness and effcency dfferently. Ideally, a mult-resource queueng algorthm should allow network operators to flexbly specfy ther tradeoff preference and mplement the specfed tradeoff by determnng the rght packet schedulng order. However, desgnng such a queueng algorthm s non-trval. It remans to be seen how effcency can be quanttatvely defned. Further, t remans open how the tradeoff requrement should be approprately specfed. But most mportantly, gven a specfc tradeoff requrement, how can the schedulng decson be correctly made to mplement t? Ths paper represents the frst attempt to address these challenges. We clarfy the effcency measure as the schedule makespan, whch s the completon tme of the last flow. We show that achevng a flexble tradeoff between farness and effcency s generally NPhard. We hence lmt our dscusson to a typcal scenaro where CPU and lnk bandwdth are the two types of resources requred for packet processng, whch s usually the case n mddleboxes. We show that the farness-effcency tradeoff can be strctly enforced by a GPS-lke (Generalzed Processor Sharng [9,21]) flud model, where packets are served n arbtrarly small ncrements on both resources. To mplement the dealzed flud n the real world, we desgn a packet-by-packet trackng algorthm, usng an approach smlar to the vrtual tme mplementaton of Weghted Far Queueng (WFQ) [9, 16, 21]. We have prototyped our tradeoff algorthm n the Clck modular router [19]. Both our prototype mplementaton and trace-drven smulaton show that a 15% 20% farness tradeoff s suffcent to acheve the optmal effcency, leadng to a nearly 20% mprovement n bandwdth throughput wth a sgnfcantly hgher resource utlzaton. 2. FAIRNESS AND EFFICIENCY Before dscussng the tradeoff between farness and effcency, we shall frst clarfy how the noton of farness s to be defned, and how effcency s to be measured quanttatvely. We model packet processng as gong through a resource ppelne, where the frst resource s consumed to process the packet frst, followed by the second, and so on. A packet s not avalable for the downstream resource untl the processng on the upstream resource fnshes. For example, a packet cannot be transmtted (whch consumes lnk bandwdth) before t has been processed by CPU. 2.1 Domnant Resource Farness Farness s one of the prmary desgn objectves for a queueng algorthm. A far schedule offers servce solaton among flows by allowng each flow to receve the throughput at least at the level when every resource s evenly allocated. The noton of Domnant Resource Farness (DRF) embodes ths solaton property by achevng the max-mn farness on the domnant resources of packets n ther respectve flows [13]. The domnant resource of a packet s defned as the one that requres the maxmum packet processng tme. In partcular, let τ r(p) be the tme requred to process packet p on resource r. The domnant resource of packet p s r p = arg max r τ r(p). Gven a packet schedule, let D (t 1, t 2) be the tme flow receves to process the domnant resources of ts packets n a backlogged perod (t 1, t 2). The functon D (t 1, t 2) s referred to as the domnant servce flow receves n (t 1, t 2). A schedule s sad to strctly mplement DRF f for all flows and j, and for any perod (t 1, t 2) they backlog, we have D (t 1, t 2) = D j(t 1, t 2). (1) In other words, a strct DRF schedule allows each flow to receve the same domnant servce n any backlogged perod. However, because packets are scheduled as separate enttes and are transmtted n sequence, strctly mplementng DRF at all tmes may not be possble n practce. For ths reason, a practcal far schedule only requres flows to receve approxmately the same domnant servces over tme [13, 31, 34], as shown n the prevous example of Fg. 1a. 2.2 The Effcency Measure In addton to farness, effcency s another mportant concern for a mult-resource schedulng algorthm, but has receved no sgnfcant attenton before. Even the defnton of effcency needs clarfcaton. Perhaps the most wdely adopted effcency measure s system throughput, whose conventonal defnton s the rate of completons [17], computed as the processed workload dvded by the elapsed tme (e.g., bts per second). Whle ths performance metrc s well defned for sngle-resource systems, extendng ts defnton to multple types of resources leads to a throughput vector, where each component s the throughput of one type of resource (e.g., 10 CPU nstructon completons per second and 5 bts transmtted through the output lnk per second), and dfferent throughput vectors may not be comparable.

3 Another possble effcency measure s resource utlzaton gven non- empty system, or smply resource utlzaton n the remander of ths paper. 1 However, n a mddlebox, dfferent resources may see dfferent levels of utlzaton. The queston s: how should the system utlzaton be properly defned? One possble defnton s to add up the utlzaton rates of all resources. Ths defnton mplctly assumes exchangeable resources, say, 1% CPU usage s equvalent to 1% bandwdth consumpton, whch may not be well justfed n many crcumstance, especally when one type of resource s scarce n the system and s valued more than the other. In ths paper, we measure effcency wth the schedule makespan. Gven nput flows wth a fnte number of packets, the makespan of a schedule s defned as the tme elapsed from the arrval of the frst packet to the tme when all packets fnsh processng on all resources. One can also vew makespan as the completon tme of the last flow. Intutvely, gven a fnte traffc nput, the shorter the makespan s, the faster the nput traffc s processed, and the more effcent the schedule s Tradeoff between Farness and Effcency Wth the precse measure of effcency, we are curous to know how much effcency s sacrfced for far queueng. To answer ths queston, we frst generalze the defnton of work conservng schedules from tradtonal sngle-resource far queueng to multple resources. In partcular, we say a schedule s work conservng f at least one resource s fully utlzed for packet processng when there s a backlogged flow. In other words, a work conservng schedule does not allow resources to be wasted n dle f they can be used to process a backlogged packet. Exstng mult-resource far queueng algorthms [13,31,34] use the goal of achevng work conservaton as an ndcaton of effcency. However, n the theorem below, we observe that such an approach s neffectve. Theorem 1. Let m be the number of resource types concerned. Gven any traffc nput I, let T σ (I) be the makespan of a work conservng schedule σ, and T (I) the mnmum makespan of an optmal schedule. We have T σ (I) mt (I). (2) PROOF. Gven a traffc nput I, let the work conservng schedule σ consst of n b busy perod. A busy perod s a tme nterval durng whch at least one type of resource s used for packet processng. When the system s empty and a new packet arrves, a new busy perod starts. The busy perod ends when the system becomes empty agan. We consder the followng two cases. Case 1: n b = 1. Let traffc nput I consst of N packets, ordered based on ther arrval tmes, where packet 1 arrves frst. For packet, let τ r () be ts packet processng tme on resource r. It s easy to check that the followng nequalty holds for the optmal schedule wth the mnmum makespan: T (I) max r N =1 τ () r. (3) 1 Ths defnton s dfferent from that of queueng theory, where the utlzaton s defned as the fracton of tme a devce s busy [17]. Under ths defnton, hgh utlzaton usually means a hgh congeston level wth a large queue backlog and long delays [36], and s usually not desred. 2 In general, makespan s not the only effcency measure that one can defne. For example, we can also measure effcency wth the average flow completon tme. We choose makespan as the effcency measure n ths paper because t leads to tractable analyss. More mportantly, makespan closely relates to system utlzaton and s conceptually easy to understand. The dscusson of other possble effcency measures s out of the scope of ths paper. On the other hand, for work conservng schedule σ, ts makespan reaches the maxmum when packet processng does not overlap n tme, across all resources,.e., T σ (I) N m =1 r=1 Ths leads to the followng nequaltes: T σ (I) N m τ r () =1 r=1 N =1 τ () r (4) m max τ r () mt (I). (5) r Case 2: n b > 1. Gven traffc nput I, let I(t + ) be the packets that arrve on or after tme t. For schedule σ, let t 0 be the tme when ts second last busy perod (n b 1) ends, and t 1 the tme when the last busy perod (n b ) starts. Because schedule σ s work conservng, no packet arrves between t 0 and t 1. We have and T σ (I) = t 1 + T σ (I(t + 1 )), (6) T (I) = t 1 + T (I(t + 1 )). (7) Note that gven traffc nput I(t + 1 ), schedule σ conssts of only one busy perod. By the dscusson of Case 1, we have T σ (I) = t 1 + T σ (I(t + 1 )) t 1 + mt (I(t + 1 )) mt (I), where the last nequalty s derved from (7). We make the followng three observatons from Theorem 1. Frst, the tradeoff between farness and effcency s a unque challenge facng mult-resource schedulng. When the system conssts of only one type of resource (.e., m = 1), work conservaton s suffcent to acheve the mnmum makespan, leavng farness as the only concern. For ths reason, effcency has never been a problem for tradtonal sngle-resource far queueng. Second, whle work conservaton also provdes some effcency guarantee for multresource schedulng, the more types of resources, the weaker the guarantee. Thrd, even wth a small number of resource types, the effcency loss could be qute sgnfcant. Snce bandwdth throughput s nversely proportonal to the schedule makespan, Theorem 1 mples that solely relyng on work conservaton may ncur up to 50% loss of bandwdth throughput when there are two types of resources. Whle ths s based on the worst case, as we shall see later n 6, our experments confrm that a throughput loss of as much as 20% s ntroduced by the exstng far queueng algorthms. Tradng off some degree of farness for hgher effcency s therefore well justfed, especally for applcatons wth loose farness requrements. 2.4 Challenges Unfortunately, strkng a desred balance between farness and effcency n a mult-resource system s techncally non-trval. Even mnmzng the makespan wthout regard to farness a specal case of farness-effcency tradeoff s NP-hard. In partcular, we note that mnmzng the makespan of a packet schedule can be modeled as a mult-stage flow shop problem [6, 20, 23] studed n operatons research, where the equvalent of a packet s a job, and the equvalent of a type of resource s a machne. However, flow shop schedulng s a notorously hard problem, even n ts offlne settng where the entre nput s known beforehand. Specfcally, when all jobs (packets) are avalable at the very begnnng, fndng the mn- (8)

4 mum makespan s strongly NP-hard when the number of machnes (resources) s greater than two [12]. Gven the hardness results above, n ths paper, we lmt our dscusson to two types of resources, CPU and lnk bandwdth, as these are the two most concerned mddlebox resources [13,25]. We note that even wth two types of resources, mnmzng the schedule makespan remans a hard problem. Because packets arrve dynamcally over tme, the problem resembles a 2-machne onlne flow shop schedulng problem where jobs (packets) do not reveal ther nformaton untl they arrve. For ths problem, only a lmted amount of negatve results s known [6,20,23,26,30]. Specfcally, no onlne algorthm can ensure a makespan wthn a factor of of the optmum n all cases [24]. We also notce that no exstng work gves a concrete soluton, even a heurstc algorthm, that jontly consders both makespan and farness. Table 1: Man notatons used n the flud model. The superscrpt t s dropped when tme can be clearly nferred from the context. Notaton Explanaton n maxmum number of flows that are concurrently backlogged α farness knob specfed by the network operator B (or B t ) set of flows that are currently backlogged (at tme t) d (or d t ) domnant share allocated to flow (at tme t) d (or d t ) far domnant share (at tme t), gven by (16) τ,r (or τ,r) t packet processng tme on resource r requred by the head-of-lne packet of flow (at tme t) τ,r (or τ,r) t normalzed τ,r (or τ,r), t defned by (12) 3. FAIRNESS, EFFICIENCY, AND THEIR TRADEOFF IN THE FLUID MODEL The dffculty of makespan mnmzaton s manly ntroduced by the combnatoral nature of mult-resource schedulng. One approach to crcumvent ths problem s to consder a flud relaxaton, where packets are served n arbtrarly small ncrements on all resources. For each packet, ths s equvalent to processng t smultaneously on all resources wth the same progress, and head-of-lne packets of backlogged flows can also be served n parallel, at (potentally) dfferent processng rates. Such a parallel processng flud model elmnates the need for dscussng the schedulng orders of flows. Instead, t allows us to focus on the resource shares allocated to flows, hence relaxng a combnatoral optmzaton problem to a smpler dynamc resource allocaton problem. Whle n general, optmally solvng such a dynamc problem requres knowng future packet arrvals, we show n ths secton that, under some practcal assumptons, a greedy algorthm gves an optmal onlne schedule wth the mnmum makespan. We can then strke a balance between effcency and farness by mposng some farness constrants to the flud schedule. We shall dscuss later n 4 and 5 how ths flud schedule s mplemented n practce wth a packet-by-packet trackng algorthm at acceptable complexty. 3.1 Flud Relaxaton In the flud model, a flow s relaxed to a flud where each of ts packets s served smultaneously on all resources wth the same progress. Packets of dfferent flows are also served n parallel. The schedule needs to decde, at each tme, the resource share allocated to each backlogged flow. In partcular, let B t be the set of flows that are backlogged at tme t. Let a t,r be the fracton (share) of resource r allocated to flow at tme t. The flud schedule determnes, at each tme t, the resource allocaton a t,r for each backlogged flow and each resource r. Two constrants must be satsfed when makng resource allocaton decsons. Frst, we must ensure that no resource s allocated more than ts total avalablty: B t a t,r 1, r = 1, 2. (9) The second constrant ensures that a packet s processed at a consstent rate across resources. In partcular, for a backlogged flow and ts head-of-lne packet at tme t, let τ t,r be ts packet processng tme on resource r, and r = arg max τ,r t (10) r be ts domnant resource. The processng rate that ths packet receves on resource r s computed as the rato between the resource share allocated and the processng tme requred: a t,r/τ t,r. To ensure a consstent processng rate, we have a t,r/τ t,r = a t,r /τ t,r, for all r and r. Substtutng r nto r above, we see a lnear relaton between the allocaton share of resource r and that of the domnant resource: where a t,r = τ t,r τ t,r a t,r = τ t,rd t, (11) τ t,r = τ t,r/τ t,r (12) s the normalzed packet processng tme on resource r, and d t = a t,r (13) s the domnant share allocated to flow at tme t. Pluggng (11) nto (9), we combne the two constrants nto one feasblty constrant of a flud schedule: B t τ t,rd t 1, r = 1, 2. (14) Before we dscuss the tradeoff between farness and effcency, we frst consder two specal cases, where ether farness or effcency s the only objectve to optmze n the flud model. For ease of presentaton, we drop the superscrpt t when tme can be clearly nferred from the context. Table 1 summarzes the man notatons used n the flud model. 3.2 Flud Schedule wth Perfect Farness We frst consder the farness objectve. To acheve perfect DRF, the flud schedule enforces strct max-mn farness on flows domnant shares, under the feasblty constrant. Specfcally, the flud schedule solves the followng DRF allocaton problem [14, 22] at each tme t: max d s.t. mn B d τ,rd 1, r = 1, 2. B (15) Let n be the number of backlogged flows. The optmal soluton, denoted by d = ( d 1,..., d n), allocates each backlogged flow the same domnant share,.e., d = d = 1/ max { τ,1, τ,2 }. (16)

5 CPU p1 p2 p3 p4 p5 p6 q1 q2 p1 p2 p3 p4 p5 p6 Lnk q1 q Tme Fgure 2: The DRGPS flud that mplements the perfect farness n the example of Fg. 1. Flow 1 sends packets p1, p2,..., and receves 3/5 CPU, 1/15 bandwdth ; Flow 2 sends packets q1, q2,..., and receves 3/5 CPU, 1/15 bandwdth. Only 2/3 of the lnk bandwdth s utlzed. In any backlogged perods, because flows are allocated the same domnant shares, they receve the same domnant servces, achevng strct DRF at all tmes. The resultng flud schedule s also known as DRGPS [32], a mult-resource generalzaton to the wellknown GPS [9, 21]. Any dscrete far schedule s essentally a packet-by-packet approxmaton to DRGPS. For nstance, applyng DRGPS to the example of Fg. 1 leads to a flud schedule shown n Fg. 2, where the normalzed packet processng tmes of Flow 1 and Flow 2 are τ 1,1, τ 1,2 = 2/3, 1 and τ 2,1, τ 2,2 = 1, 1/9, respectvely. By (16), both flows are allocated the same domnant share d = 3/5. Specfcally, Flow 1 receves 2/5 CPU, 3/5 bandwdth ; Flow 2 receves 3/5 CPU, 1/15 bandwdth. In total, only 2/3 of the bandwdth s utlzed, the same as the dscrete far schedule shown n Fg. 1a. 3.3 Flud Schedule wth Optmal Effcency We next dscuss the effcency objectve. Whle there are some schedules proposed n the operatons research lterature that can acheve the mnmum makspan for a flow shop problem, none of them apples n the context of packet schedulng: they ether assume no packet arrvals (e.g., [29]) or requre full knowledge of future nformaton (e.g., [5]). We propose a smple greedy flud schedule as follows. For a gven tme nstant, we defne the system s nstantaneous domnant throughput as the sum of the domnant share allocated,.e., B d. Intutvely, by maxmzng B d at all tmes, one would expect a hgh average domnant throughput B D/T, where T s the schedule makespan and D s the total domnant servces (processng tme) requred by flow. Gven domnant workload B D, maxmzng the average domnant throughput s equvalent to mnmzng the schedule makespan T. Followng ths ntuton, we propose a greedy flud schedule that solves the followng resource allocaton problem to maxmze the nstantaneous domnant throughput at every tme: max d 0 s.t. B d τ,rd 1, r = 1, 2. B (17) In case that the optmal soluton, denoted d = (d 1,..., d n), s not unque, the schedule chooses the one wth the maxmum overall utlzaton: max τ,rd d. (18) r B In the example of Fg. 1, solvng (17) allocates Flow 1 the domnant share d 1 = 9/25 and Flow 2 the domnant share d 2 = 24/25. It s easy to check that both CPU and lnk bandwdth are fully utlzed. Compared to those schedules proposed n the operatons research lterature, the greedy schedule defned by (17) s partcularly attractve for packet schedulng due to the followng three propertes. Frst, t s an onlne algorthm wthout any a pror knowledge of future packet arrvals. Further, among all packets that are backlogged, only the nformaton regardng head-of-lne packets s requred. Ths suggests that the schedule only needs to mantan a very smple per-flow state. Most mportantly, the greedy schedule s more than a smple heurstc. Below we show that under some practcal assumptons, greedly maxmzng the domnant throughput gves the mnmum makespan. Our analyss requres the followng lemma, where we show that the schedule wll not waste any resource n dle, unless all flows bottleneck on the same resource, n whch case the other resource cannot be fully utlzed anyway. The proof s gven n Appendx A. Lemma 1. The flud schedule defned by (17) fully utlzes both resources f there are two head-of-lne packets wth dfferent domnant resource,.e., there exst two flows j and l, such that τ j,1 = 1 > τ j,2 and τ l,1 < τ l,2 = 1. Wth Lemma 1, we analyze the makespan of the flud schedule defned by (17). Followng [13], we say a flow s domnantresource monotonc f t does not change ts domnant resource durng backlogged perods. To make the analyss tractable, we assume that flows are domnant-resource monotonc. Ths s often true n practce as packets n the same flow usually undergo the same processng, and hence have the same domnant resource. The followng lemma, whose proof can be found n Appendx B, states the optmalty of the flud schedule n a statc scenaro wthout dynamc packet arrvals. Lemma 2. For domnant-resource monotonc flows, the flud schedule defned by (17) gves the mnmum makespan f all packets are avalable at the begnnng. We now extend the results of Lemma 2 to an onlne case where packets dynamcally arrve over tme. The followng theorem gves the optmalty condton of the flud schedule. The proof can be found n Appendx C. Theorem 2. For domnant-resource monotonc flows, the flud schedule defned by (17) gves the mnmum makespan among all schedules, f after the system has two flows wth dfferent domnant resources, whenever a new flow arrves, there exst two backlogged flows wth dfferent domnant resources. The optmalty condtons requred by Theorem 2 can be easly met n practce. Because the number of backlogged flows s usually large, t s almost true that we can always fnd two flows wth dfferent domnant resources. In fact, even n a very unfortunate case where all flows bottleneck on the same resource, the greedy flud schedule does not devate far away from the optmum: no matter what flud schedule s used, the bottleneck resource s always fully utlzed when the system s non-empty and hence has the same backlog, whch s a domnant factor n determnng the schedule makespan. The sgnfcance of Theorem 2 s that t connects makespan, a measure defned n the tme doman, to the nstantaneous domnant throughput, a measure defned n the space doman. More mportantly, t shows that mnmzng the former s, n a practcal sense, equvalent to maxmzng the latter at all tmes, wthout the need to know future packet arrvals. We shall use ths ntuton to strke a balance between farness and effcency n the next subsecton.

6 3.4 Tradeoff between Farness and Effcency When both farness and effcency are consdered, we express the tradeoff between the two conflctng objectves as a constraned optmzaton problem mnmzng makespan under some specfed farness requrements. Recall that when perfect farness s enforced, all flows receve the same domnant share d computed by (16),.e., d = d for all. When farness s not a strct requrement, we ntroduce a farness knob α [0, 1] to specfy the farness degradaton. In partcular, an allocaton d s called α-porton far f d α d for all backlogged flow. In other words, each flow receves at least an α-porton of ts far domnant share d. A flud schedule s called α-porton far f t acheves the α-porton far allocaton at all tmes. By choosng dfferent values for α, a network operator can precsely control the farness degradaton. As two extreme cases, settng α = 0 means that farness s not consdered at all; settng α = 1 means that perfect farness must be enforced at all tmes. Gven the specfed farness knob α, the flud schedule tres to mnmze makespan under the correspondng α-porton farness constrants. Snce mnmzng makespan s, n a practcal sense, equvalent to maxmzng the system s domnant throughput, we obtan a smple tradeoff heurstc that maxmzes the domnant throughput, subject to the requred α-porton farness at every tme t: max d s.t. B d τ,rd 1, r = 1, 2, B d α d, B, (19) where the far share d s gven by (16). We see that the flud schedule captures both DRGPS and the greedy schedule defned by (17) as specal cases wth α = 1 and 0, respectvely. Specal Soluton Structure. The tradeoff problem (19) has a closed-form soluton, based on whch the tradeoff schedule can be easly computed. We frst allocate each flow ts guaranteed porton of domnant share α d. We then denote d = d α d (20) as the bonus domnant share allocated to flow. Substtutng (20) nto (19), we equvalently rewrte (19) as a problem of determnng the bonus domnant share receved by each flow: where max d 0 s.t. d + B α d B τ,r d µ r r = 1, 2, B µ r = 1 α d B (21) τ,r, r = 1, 2, (22) and s the remanng share of resource r after each flow receves ts guaranteed domnant share α d. Wthout loss of generalty, we sort all the backlogged flows based on the processng demands on the two types of resources requred by ther head-of-lne packets as follows: τ 1,1/ τ 1,2 τ n,1/ τ n,2. (23) The followng theorem shows that at most two flows are awarded the bonus share at a tme. Its proof s gven n Appendx D. Theorem 3. There exsts an optmal soluton d to (21) where d = 0 for all 2 n 1. In partcular, d s gven n the followng three cases: Case 1: µ 1/µ 2 < τ n,1/ τ n,2. In ths case, resource 1 s fully utlzed, wth d n = µ 1/ τ n,1 and d = 0 for all < n. Case 2: µ 1/µ 2 > τ 1,1/ τ 1,2. In ths case, resource 2 s fully utlzed, wth d 1 = µ 2/ τ 1,2 and d = 0 for all > 1. Case 3: τ n,1/ τ n,2 µ 1/µ 2 τ 1,1/ τ 1,2. In ths case, both resources are fully utlzed, and we have (µ 1 τ n,2 µ 2 τ n,1)/( τ 1,1 τ n,2 τ 1,2 τ n,1), = 1; d = (µ 2 τ 1,1 µ 1 τ 1,2)/( τ 1,1 τ n,2 τ 1,2 τ n,1), = n; 0, o.w. Once the optmal bonus domnant share has been determned as shown above, the optmal soluton d to (19), whch s the domnant share allocated to each flow, can be easly computed as the sum of the bonus share and the guaranteed share: d = d + α d, for all. (24) We gve an ntutve explanaton of Theorem 3 as follows. The frst two cases of Theorem 3 correspond to the scenaro where after each flow receves ts guaranteed share, the remanng amounts of the two types of resources are unbalanced and cannot be fully utlzed smultaneously. In ths case, the schedule awards the bonus share to the flow (ether Flow 1 or Flow n) whose processng demands can better utlze the remanng resources. The thrd case covers the scenaro where the remanng amounts of the two types of resources are balanced, and can be fully utlzed when the system s non-empty. In ths case, they are allocated to two flows (Flow 1 and Flow n) wth complementary resource demands as ther bonus shares. Theorem 3 reveals an mportant structure, that at most two flows are allocated more domnant shares than others. We refer to these flows as the favored flows and all the others as the regular flows. We shall show n 5 that ths structure leads to an effcent O(log n) mplementaton of the flud schedule. 4. PACKET-BY-PACKET TRACKING So far, all our dscussons are based on an dealzed flud model. In practce, however, packets are processed as separate enttes. In ths secton, we present a dscrete trackng algorthm that mplements the flud schedule as a packet-by-packet schedule n practce. We show that the dscrete schedule s asymptotcally close to the flud schedule, n terms of both farness and effcency. We start wth a comparson between two typcal trackng approaches. 4.1 Start-Tme Trackng vs. Fnsh-Tme Trackng Two common trackng algorthms may be used to mplement a flud schedule n practce, start-tme trackng and fnsh-tme trackng. The former tracks the order of packet start tmes among all packets that have already started n the flud schedule, the one that starts the earlest s scheduled frst. Fnsh-tme trackng, on the other hand, assgns the hghest schedulng prorty to the packet that completes servce the earlest n the flud schedule. In tradtonal sngle-resource far queueng, FQS [16] uses the former approach to track GPS, whle WFQ [1, 9, 21] adopts the latter approach. Whle both algorthms closely track the flud schedule of far queueng, only start-tme trackng s well defned for the tradeoff schedule gven by (19). Ths s due to the fact that, n the tradeoff schedule, future traffc arrvals may lead to a dfferent allocaton of packet processng rates and may subsequently change the packet

7 fnsh tmes of current packets. As a result, determnng the order of fnsh tmes requres future traffc arrval nformaton and hence s unrealstc. 3 Start-tme trackng avods ths problem as packets are scheduled only after they start n the flud schedule. For ths reason, we use start-tme trackng to mplement the flud schedule. We say a dscrete schedule and a flud schedule correspond to each other f the former tracks the latter by the packet start tme. Specfcally, we mantan the flud schedule n the background. Whenever there s a schedulng opportunty, among all head-of-lne packets that have already started n the flud schedule, the one that starts the earlest s chosen. Below we show that ths dscrete schedule s asymptotcally close to ts correspondng flud schedule. 4.2 Performance Analyss To analyze the performance of start-tme trackng, we ntroduce the followng notatons. Let τ max be the maxmum packet processng tme requred by any packet on any resource. Let n be the maxmum number of flows that are concurrently backlogged. Let T F be the makespan of the flud schedule, and T D the makespan of ts correspondng dscrete schedule. All proofs are gven n Appendx E. The followng theorem bounds the dfference between the makespan of the flud schedule and ts correspondng dscrete schedule. Theorem 4. For the flud schedule wth α > 0 and ts correspondng dscrete schedule, we have T D T F + nτ max. (25) The error bound nτ max can be ntutvely explaned as the total packet processng tme requred by all n concurrent flows, each sendng only one packet. In practce, the number of packets a flow sends s usually sgnfcantly larger than one. As a result, the traffc makespan s sgnfcantly larger than the error bound,.e., T F nτ max. Theorem 4 essentally ndcates that n terms of makespan, the two schedules are asymptotcally close to each other. We next analyze the farness performance of the dscrete schedule by comparng the domnant servces a flow receves under both schedules. In partcular, let D F (0, t) be the domnant servces flow receves n (0, t) under the flud schedule, and D D (0, t) the domnant servces flow receves n (0, t) under the correspondng dscrete schedule. The followng theorem shows that flows receve approxmately the same domnant servces under both schedules. Theorem 5. For the flud schedule wth α > 0 and ts correspondng dscrete schedule, the followng nequalty holds for any flow and any tme t: D F (0, t) 2(n 1)τ max D D (0, t) D F (0, t) + τ max. (26) In other words, the dfference between the domnant servces a flow receves under the two correspondng schedules s bounded by a constant amount, rrespectve of the tme t. Over the long run, the dscrete schedule acheves the same α-porton farness as ts correspondng flud schedule. To summarze, start-tme trackng retans both the effcency and farness propertes of ts correspondng flud schedule n the asymptotc regme. 5. AN O(log n) IMPLEMENTATION 3 Ths s not a problem of sngle-resource far queueng as dfferent flows are allocated the same processng rate, so that future traffc arrvals wll not affect the order of fnsh tmes of current packets. To mplement the aforementoned start-tme trackng algorthm, two modules are requred: packet proflng and flud schedulng. The former estmates the packet processng tme on both CPU and lnk bandwdth; the latter mantans the flud schedule as a reference system based on the packet proflng results. We show n ths secton that packet proflng can be quckly accomplshed n O(1) tme usng a smple approach proposed n [13]. The man challenge comes from the complexty of mantanng the flud schedule, where drect mplementaton requres O(n) tme. Here, n s the number of backlogged flows. We gve an O(log n) mplementaton based on an approach smlar to vrtual tme. We shall show n 6 that the mplementaton can be easly prototyped n the Clck modular router [19]. 5.1 Packet Proflng As ponted out by Ghods et al. [13], any mult-resource far queueng algorthm, ncludng our flud schedule, requres knowledge of the packet processng tme on each resource. Fortunately, as shown n [13], CPU processng tme can be accurately estmated as a lnear functon of packet sze. Specfcally, for a packet of sze l, the CPU processng tme s estmated as al +b, where a and b are the coeffcents dependng on the type of packet processng (e.g., IPsec). We have valdated ths lnear model through an upfront experment usng Clck [19]. For each type of packet processng, we measure the exact CPU processng tme requred by packets of dfferent szes. Ths allows us to determne the coeffcents a and b. We ft such a lnear model to the scheduler and use t to estmate the CPU processng tme requred by a packet. As for the packet transmsson tme, the estmaton s smply the packet sze dvded by the outgong bandwdth, whch s known a pror. 5.2 Drect Implementaton of Flud Schedulng Based on the packet proflng results, the flud schedule s constructed and s mantaned by the flud scheduler. In partcular, we need to determne the next packet that starts n the flud schedule. Ths requres trackng the work progress of all n flows. Below we gve a drect mplementaton that wll be used later n our vrtual tme mplementaton. For each flow, we record d, whch s the domnant share the flow receves n the flud schedule at the current tme and s computed by (24). We also record R, the remanng domnant processng tme requred by the head-of-lne packet of the flow at the current tme. For flow, ts head-of-lne packet wll fnsh n R /d tme f no event occurs then. An event s ether a packet departure or a packet beng the new head-of-lne n the flud schedule. Ether of them may change the head-of-lne packet of a flow, leadng to dfferent coeffcents of the tradeoff problem (19). Wth d and R, we can accurately track the work progress of flow n an event-drven bass. Specfcally, upon the occurrence of an event, let t be the tme elapsed snce the last update. If t < R /d, meanng that the event occurs before the head-of-lne packet fnshes, we update R R d t. If t = R /d, meanng that the event occurs at the tme when the head-of-lne packet fnshes, we check f flow has a next packet p to process. If t does, then packet p becomes the new head-of-lne and should start n the flud schedule. We update R as the domnant processng tme requred by p. Otherwse, we reset R 0, and flow leaves the flud system. We also recompute d after R s updated. (Note that t s mpossble to have t > R /d.) However, purely relyng on the approach above to track the work progress of all n flows s hghly neffcent. Whenever an event

8 occurs, each flow must be updated ndvdually, whch requres at least O(n) tme per event and s too expensve. We next ntroduce a more effcent mplementaton that requres the above procedure for at most two flows. 5.3 Vrtual Tme Implementaton of Flud Schedulng To avod the hgh complexty requred by the drect mplementaton above, we have noted, by Theorem 3, that at most two flows are favored and are allocated more domnant shares than others. Therefore, t suffces to mantan at most three domnant shares at a tme two for the favored flows and one for the other regular flows. For regular flows, we track ther work progress usng an approach smlar to the vrtual tme mplementaton of GPS [9, 21]. Our ntuton s that, by Theorem 3, all the regular flows are allocated the same domnant share, and ther schedulng resembles far queueng. For favored flows, snce there are at most two of them, we track ther work progress drectly, usng the drect mplementaton above. Our approach s detaled below Identfyng Favored and Regular Flows We frst dscuss how favored and regular flows can be quckly dentfed upon the occurrence of an event. By Theorem 3, t suffces to sort flows n order (23) and examne the three cases. Flows that receve the bonus share (.e., d > 0) are favored. Note that the entre computaton requres only nformaton regardng the headof-lne packet of the frst and the last flow n order (23) ( τ 1,r and τ n,r, the normalzed domnant processng tme). We store all the head-of-lne packets n a double-ended prorty queue mantaned by a mn-max heap for fast retreval, where the packet order s defned by (23). Ths allows us to apply Theorem 3 and dentfy the favored and regular flows n O(log n) tme Trackng Favored Flows For favored flows, because there are at most two of them, we track ther work progress usng the drect mplementaton mentoned n 5.2, where we record d and R for each favored flow. It s easy to see that the update complexty s domnated by the computaton of d. As mentoned n the prevous dscusson, ths can be done n O(log n) tme by Theorem 3. Also, snce there are at most two favored flows, the overall trackng complexty remans O(log n) per event Trackng Regular Flows For regular flows, snce they receve the same domnant share, ther schedulng resembles far queueng. We hence track ther work progress usng vrtual tme [1, 9, 21]. Specfcally, we defne vrtual tme V (t) as a functon of real tme t evolvng as follows: V (0) = 0, V (t) = α d t, t > 0. (27) Here, d t s the far domnant share computed by (16) at tme t, and s fxed between two consecutve events; α d t s the domnant share each regular flow receves. 4 Thus, V can be nterpreted as ncreasng at the margnal rate at whch regular flows receve domnant servces. Each regular flow also mantans vrtual fnsh tme F, ndcatng the vrtual tme at whch ts head-of-lne packet fnshes n the flud schedule. The vrtual fnsh tme F s updated as follows when flow has a new head-of-lne packet p at tme t: F = V (t) + τ (p), (28) 4 We restore the superscrpt t here to emphasze that the far domnant share computed by (16) may change over tme. where τ (p) s the domnant packet processng tme requred by p. Among all the regular flows, the one wth the smallest F has ts head-of-lne packet fnshng frst n the flud schedule. Unless some event occurs n between, at tme t, the next packet departure for the regular flows would be n t N = (mn F V (t))/α d tme. Usng vrtual tme defned by (27), we can accurately track the work progress of regular flows n an event-drven bass. Specfcally, upon the occurrence of an event at tme t, let t 0 be the tme of the last update, and t = t t 0 the tme elapsed snce the last update. If t < t N, meanng that the event occurs before the next packet departure of regular flows, we smply update the vrtual tme followng (27): V (t) = V (t 0) + α d t. (29) If t = t N, then the event occurs at the tme when a packet of a regular flow, say flow, fnshes n the flud schedule. In addton to updatng the vrtual tme, we check to see f flow has a next packet p to process. If t does, meanng that the packet p should start n the flud schedule, we update ts vrtual fnsh tme F followng (28). Otherwse, flow departs the system. We also recompute d by (16). The trackng complexty s domnated by the computaton of the mnmum vrtual fnsh tme,.e., mn F. By storng F s n a prorty queue mantaned by a heap, we see that the trackng complexty s O(log n) per event Handlng Identty Swtchng We note that the dentty of a flow s not fxed: upon the occurrence of an event, a favored flow may swtch to a regular flow, and vce versa. We show that such dentty swtchng can also be easly handled n O(log n) tme. We frst consder a favored flow swtchng to a regular one at tme t, whch requres the computaton of the vrtual fnsh tme F. Recall that we have recorded R, the remanng domnant processng tme requred by the head-of-lne packet, for flow as t s prevously favored. By defnton, the vrtual fnsh tme F can be smply computed as F = V (t) + R. (30) Addng F to the heap takes at most O(log n) tme. We next consder a regular flow swtchng to a favored one at tme t, whch requres the computaton of R. Recall that we have recorded the vrtual fnsh tme F for flow. By defnton, the remanng domnant processng tme requred by ts head-of-lne packet s smply R = F V (t), (31) whch s a dual of (30). We also need to remove the vrtual fnsh tme, F, from the heap. To do so, we mantan an ndex for each regular flow, recordng the locaton of ts vrtual fnsh tme stored n the heap. Followng ths ndex, we can easly locate the poston of F and delete t from the heap, followed by some standard trckle-down operatons to preserve the heap property n O(log n) tme. To summarze, our approach mantans the flud schedule by dentfyng favored and regular flows, trackng ther work progress, and handlng the potental dentty swtchng. We show that any of these operatons can be accomplshed n O(log n) tme. As a result, mantanng the flud schedule takes O(log n) tme per event. 5.4 Start-Tme Trackng and Complexty Wth the flud schedule mantaned as a reference system, the mplementaton of start-tme trackng s straghtforward. Whenever a packet starts n the flud schedule, t s added to a FIFO queue.

9 Upon a schedulng opportunty, the scheduler polls the queue and retreves a packet to schedule. Ths ensures that packets are scheduled n order of ther start tmes n the flud schedule. To mnmze the update frequency, the scheduler lazly updates the flud schedule only when the FIFO queue s empty. We now analyze the schedulng complexty of the aforementoned mplementaton. The schedulng decsons are made by updatng the flud schedule n an event-drven bass. For each event, the update takes O(log n) tme, where n s the number of backlogged flows. Note that there are only two types of events n the flud schedule, new head-of-lne and packet departure. Because a packet served n the flud schedule trggers exactly these two events over the entre schedulng perod, schedulng N packets trggers 2N updates n the flud schedule, wth the overall complexty O(2N log n). On average, the schedulng decson s made n O(2 log n) tme per packet, the same order as that of DRFQ [13]. 6. EVALUATION We evaluate the tradeoff algorthm va both our prototype mplementaton and trace-drven smulaton. We use a prototype mplementaton to nvestgate the detaled functonng of the algorthm, n a mcroscopc vew. We then take a macroscopc vew to evaluate the algorthm usng trace-drven smulaton, where flows dynamcally jon and depart the system. 6.1 Expermental Results We have prototyped our tradeoff algorthm as a new scheduler n the Clck modular router [19], based on the O(log n) mplementaton gven n the prevous secton. The scheduler classfes packets to flows (based on the IP prefx and port number) and dentfes the types of packet processng based on the port number specfed by a flow class table. The scheduler also exposes an nterface that allows the operator to dynamcally confgure the farness knob α. Our mplementaton conssts of roughly 1,000 lnes of C++ code. We run our Clck mplementaton n user mode on a Dell PowerEdge server wth an Intel Xeon 3.0 GHz processor and 1 Gbps Ethernet nterface. To make farness relevant, we throttle the outgong bandwdth to 200 Mbps whle keepng the nbound bandwdth as s. We also throttle the Clck module to use only 20% CPU so that CPU could also be a bottleneck. We confgure three packet processng modules n Clck to emulate a mult-functonng mddlebox: packet checkng, statstcal montorng, and IPsec. The former two modules are bandwdth-bound, though statstcal montorng requres more CPU processng tme than packet checkng does. The IPsec module encrypts packets usng AES (128-bt key length) and s CPU-bound. We confgure another server as a traffc source, ntatng 60 UDP flows each sendng byte packets per second to the Clck router. The frst 20 flows pass through the packet checkng module; the next 20 flows pass through the statstcal montorng module; and the last 20 flows pass through the IPsec module Farness-Effcency Tradeoff We frst evaluate the acheved tradeoff between schedule farness and makespan. To farly compare the makespan at dfferent farness levels, t s crtcal to ensure the same traffc nput when runnng the algorthm wth dfferent values of farness knob α. Therefore, we ntally consder an dealzed scenaro where each flow queue has nfnte capacty and never drops packets. Table 2 lsts the observed makespans wth varous farness requrements, n an experment where each flow keeps sendng packets for 10 seconds. We see that, as expected, tradng off some level of farness leads to a shorter makespan and hgher effcency. Furthermore, the Table 2: Schedule makespan observed n Clck at dfferent farness levels. The queue capacty s nfnte. Farness knob α Makespan (s) Normalzed Makespan % % % % % % % CPU Utlzaton (%) B/W Utlzaton (%) α = 0.85 α = 0.90 α = 0.95 α = Tme (s) Fgure 3: Overall resource utlzaton observed n Clck. No packet drops. margnal mprovement of effcency s decreasng. Ths suggests that one does not need to compromse too much farness n order to acheve hgh effcency. In our experment, tradng off 15% of farness shortens the makespan by 15.3% from the strctly far schedule (α = 1), whch s equvalent to a 18.1% bandwdth throughput enhancement and s near-optmal as seen n Table 2. Fg. 3 gves a detaled look nto the acheved resource utlzaton over tme, at four farness levels. We see that strctly far queueng (α = 1) wastes 30% of CPU cycles, leavng the bandwdth as the bottleneck at the begnnng. Ths stuaton remans untl bandwdth-bound flows fnsh, at whch tme the bottleneck shfts to CPU. By relaxng farness, CPU-bound flows receve more servces, leadng to a steady ncrease of CPU utlzaton up to 100%. Meanwhle, bandwdthbound flows experence slghtly longer completon tmes due to the farness tradeoff. We now verfy the farness guarantee. We run the scheduler at varous farness levels. At each level, for each flow, we measure ts receved domnant share every second for the frst 20 seconds, durng whch all flows are backlogged. Fg. 4 shows the results, where each cross ( x ) corresponds to the domnant share of a flow measured n one second. As expected, under strct farness (α = 1), all flows receve the same domnant share (around 2%). As α decreases, the farness requrement relaxes. Some flows are hence favored and are allocated more domnant share, whle others receve less. However, the mnmum domnant share a flow receves s lower bounded by the α-porton of the far share, shown as the sold lne n Fg. 4. Ths shows that the algorthm s correctly operatng at the desred farness level. We next extend the experment to a more practcal setup, where each flow queue has a lmted capacty and drops packets when t s full. We set the queue sze to 200 packets for each flow and repeat the prevous experments. In ths case, comparng makespan