A Quantitative Assured Forwarding Service

Transcription

1 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE A Quanttatve Assured Forwardng Servce Ncolas Chrstn, Jörg Lebeherr, and Tarek F. Abdelzaher Department of Computer Scence Unversty of Vrgna P.O. Box 0070 Charlottesvlle, VA 90-70, U.S.A. Abstract The Assured Forwardng (AF) servce of the IETF DffServ archtecture provdes a qualtatve servce dfferentaton between classes of traffc, n the sense that a low-prorty class experences hgher loss rates and hgher delays than a hgh-prorty class. However, the AF servce does not quantfy the dfference n the servce gven to classes. In an effort to strengthen the servce guarantees of the AF servce, we propose a Quanttatve Assured Forwardng servce wth absolute and proportonal dfferentaton of loss, servce rates, and packet delays. We present a feedback-based algorthm whch enforces the desred classlevel dfferentaton on a per-hop bass, wthout the need for admsson control or sgnalng. Measurement results from a testbed of FreeBSD PC-routers on a 00 Mbps Ethernet network show the effectveness of the proposed servce, and ndcate that our mplementaton s sutable for networks wth hgh data rates. I. INTRODUCTION The Assured Forwardng (AF, []) servce of the Dfferentated Servces (DffServ, []) archtecture s an attempt to provde a scalable soluton to the problem of servce dfferentaton n the Internet. In the AF servce, flows wth smlar QoS requrements are grouped nto classes, usng the DffServ CodePont feld (DSCP, []) n the IP header. An attractve feature of the AF servce s that t does not requre admsson control or per-flow classfcaton, and s therefore scalable on both the control and data paths. However, the AF servce only provdes qualtatve dfferentaton between classes, n the sense that some classes receve lower delays and a lower loss rate than others, but the dfferentaton s not quantfed, and no absolute servce bounds are offered. Recently, research efforts have tred to strengthen the guarantees that can be provded wthn the context of the AF servce wthout sacrfcng ts scalablty and ts smplcty, ether by tryng to quantfy the dfference n the level of servce receved by dfferent classes, or by offerng absolute bounds on servce parameters, e.g., delays, to a specfc set of classes. For nstance, the proportonal Ths work s supported n part by the Natonal Scence Foundaton through grants NCR-9606 (CAREER), ANI-9700, and ANI servce dfferentaton model [], [5] quantfes the dfference n the servce by makng the ratos of delays or loss rates of dfferent classes roughly constant. Ths type of servce can be mplemented through schedulng algorthms [], [5], [6], [7], [8], [9], [0], [] and/or buffer management algorthms [5], []. Recent works have tred to combne the schedulng and droppng decsons n a sngle algorthm [], []. Most schedulng and/or buffer management algorthms am at proportonal dfferentaton, but do not support absolute servce guarantees. In a dfferent approach to strengthenng the AF servce, the Alternatve Best-Effort (ABE) servce consders two traffc classes. The frst class obtans absolute delay guarantees, and the second class has no delay guarantees, but s gven a better loss rate than the frst class. Schedulng and buffer management algorthms for the ABE servce are presented n [5]. The servce model n [6] also supports absolute delay bounds, and qualtatve loss and throughput dfferentaton, but no proportonal dfferentaton. These recent efforts to strengthen the AF servce rase questons on the best possble class-based servce model that can be acheved by entrely relyng on schedulng and droppng algorthms at routers, and wthout admsson control, traffc polcng, or sgnalng. In an attempt to explore the lmts of such a class-based servce, we defne n ths paper a Quanttatve Assured Forwardng servce that offers, on a per-hop bass, both absolute and proportonal guarantees to classes. Each node enforces any mx of absolute and proportonal guarantees. Absolute guarantees apply to loss rates, delays, or throughput, and defne a lower bound on the servce receved by each class. Proportonal guarantees apply to loss rates and queueng delays. As an example of the guarantees n the Quanttatve Assured Forwardng servce for three classes of traffc, one could specfy servce guarantees of the form Class- Delay ms, meanng that no Class- packet should experence a queueng delay greater than two mllseconds, Class- Delay Class- Delay, meanng that Class- packets should experence queueng delays The name quanttatve dfferentated servce was recently used n [6].

2 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE roughly twce as large as Class- packets, Class- Loss Rate %, meanng that the loss rate of Class should never exceed %, Class- Loss Rate Class- Loss Rate, and Class- Servce Rate Mbps, meanng that the aggregate of flows belongng to Class should get a throughput of at least Mbps. The Quanttatve Assured Forwardng servce supports any mx of such servce guarantees, and the QoS parameters (e.g., delay bound of ms) are confgurable by the network operator. Clearly, wthout admsson control, t s not feasble to satsfy all absolute guarantees at all tmes. Thus, when absolute constrants cannot be satsfed, we allow that some servce guarantees can be temporarly relaxed accordng to a specfed order. We present a formal descrpton of the Quanttatve Assured Forwardng servce, and we devse an algorthm that enforces guarantees on loss, delay and throughput for classes by adjustng the servce rate allocaton to classes and by selectvely droppng traffc. We apply lnear feedback control theory for the desgn of the algorthm, and, to ths effect, make assumptons whch approxmate the non-lneartes n the system of study, smlar to [7], [8], [9]. Ths paper s organzed as follows. In Secton II, we defne the Quanttatve Assured Forwardng servce. In Sectons III and IV, we descrbe the algorthms whch provde the Quanttatve Assured Forwardng servce. In Secton V, we present an mplementaton of these algorthms n FreeBSD PC-routers. We evaluate the algorthms usng the mplementaton n Secton VI, and present bref conclusons n Secton VII. II. THE QUANTITATIVE ASSURED FORWARDING SERVICE In ths secton, we descrbe the Quanttatve Assured Forwardng Servce, and outlne a soluton for an algorthm that realzes ths servce. A. Formal Descrpton We assume that all traffc that arrves to the transmsson queue of the output lnk of a router s marked to belong to one of N classes. We use a conventon whereby a class wth a lower ndex receves a better servce. We consder a dscrete event system, where events are traffc arrvals. We use t to denote the tme of the n-th event n the current busy perod, and t to denote the tme elapsed between the n-th and (n + )-th events. We use a and l, respectvely, to denote the class- arrvals and the amount of class- traffc dropped ( lost ) at the n- th event. We use r to denote the servce rate allocated The begnnng of the current busy perod s defned as the last tme when the transmsson queue at the output lnk was empty. Class- Traffc D Dropped B t(n ) t(n ) t A tme R n R out Fg.. Delay and backlog at the transmsson queue of an output lnk. A s the arrval curve, R n s the nput curve and R out s the output curve. to class- at the tme of the n-th event. The servce rate of a class s a fracton of the output lnk capacty, whch can vary over tme, and s set to zero f there s no backlog of class- traffc n the transmsson queue. For the tme beng, we assume bursty arrvals wth a flud-flow servce, that s, the output lnk s vewed as smultaneously servng traffc from several classes. Such a flud-flow nterpretaton s dealstc, snce traffc s actually sent n dscrete szed packets. In Secton V, we dscuss how the fludflow nterpretaton s realzed n a packet network. All servce guarantees are enforced over the duraton of a busy perod. An advantage of enforcng servce guarantees over short tme ntervals s that the output lnk can react quckly to changes of the traffc load. Further, enforcng guarantees only wthn a busy perod requres lttle state nformaton, and, therefore, keeps the mplementaton overhead lmted. As a dsadvantage, at tmes of low load, when busy perods are short, enforcng guarantees only wth nformaton on the current busy perod can be unrelable. However, at underloaded lnks transmsson queues are mostly dle and all servce classes receve a hgh-grade servce. The followng presentaton specfes the servce dfferentaton ndependently for each busy perod. Let t(0) defne the begnnng of the busy perod. The arrval curve of class at the n-th event, A, s the total traffc that has arrved to the transmsson queue of an output lnk at a router snce the begnnng of the current busy perod, that s, A = n a (k). k=0 The nput curve, R n, s the traffc that has been en-

3 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE tered nto the transmsson queue at the n-th event, R n = A n l (k). k=0 The output curve s the traffc that has been transmtted snce the begnnng of the current busy perod, that s, n R out = r (k) t(k). () k=0 In Fgure, we llustrate the concepts of arrval curve, nput curve, and output curve for class- traffc. At any tme t, the servce rate s the slope of the output curve. In the fgure, the servce rate s adjusted at tmes t(n ), t(n ) and t. As llustrated n Fgure, for event n, the vertcal and horzontal dstance between the nput and output curves, respectvely, denote the class- backlog B and the class- delay D. For the n-th event, we have and B = R n R out, D = t t ( max{k < n R out R n(k)}). () Eqn. () characterzes the delay of the class- traffc that departs at the n-th event. We defne the loss rate to be the rato of dropped traffc to the arrvals. That s p = A R n. (5) A Snce, from the defnton of A and R n, the p are computed only over the current busy perod, they correspond to long-term loss rates only f busy perods are long. We justfy our choce wth the observaton that traffc s dropped only at tmes of congeston,.e., when the lnk s overloaded, and, hence, when the busy perod s long. Wth these metrcs, we can express the servce guarantees of a Quanttatve Assured Forwardng servce. An absolute delay guarantee on class s specfed as n : D d, (6) where d s the delay bound of class. Smlarly, an absolute loss rate bound for class s defned by n : p L. (7) An absolute rate guarantee for class s specfed as n : B > 0, r µ. (8) Class- Traffc Fg.. R out R n D d slope = r,mn B t d -D Determnng servce rates for delay guarantees. tme The proportonal guarantees on delay and loss, respectvely, are defned, for all n such that B > 0 and B + > 0, as and D + D p + p = k, (9) = k, (0) where k and k are constants that quantfy the proportonal dfferentaton desred. B. Rate Allocaton and Drop Decsons We now sketch a soluton for provdng the servce guarantees specfed n Eqs. (6)-(0) at the output lnk of a router wth capacty C and buffer sze B. We assume per-class bufferng of ncomng traffc, thus, each class s transmtted n a Frst-Come-Frst-Served manner. In the proposed soluton, the servce rates r and the amount of dropped traffc l are adjusted at each event n so that the constrants defned by Eqs. (6)-(0) are met. If not all constrants n Eqs. (6)-(0) can be met at the n-th event, then some servce guarantees need to be temporarly relaxed. We assume that the order n whch guarantees are relaxed s gven. The absolute delay guarantee on class, d, mposes a mnmum requred servce rate n the sense that all backlogged class- traffc at the n-th event wll be transmtted wthn ts delay bound f r B d D. Ths condton can be verfed by nspecton of Fgure. If the condton holds for any n, the delay bound d s never volated. If class has, n addton, an absolute rate

4 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE guarantee µ, the expresson for the mnmum rate needed by class at the n-th event, becomes { } B r,mn = max d D, µ χ B>0. () The servce rate that can be allocated to class s upper bounded by the output lnk capacty mnus the mnmum servce rates needed by the other classes, that s, r,max = C j r j,mn. Therefore, the servce rate can take any value r wth r,mn r r,max, subject to the constrant r C. Gven ths range of feasble values, r can be selected to satsfy proportonal delay dfferentaton. We vew the computaton of r n terms of the recurson r = r (n ) + r, (5) where r s selected such that the constrants of proportonal delay dfferentaton are satsfed at event n. From Eqs. () and (), the delay D at the n-th event s a functon of r (k) wth k < n. By montorng D we can thus determne the devaton from the desred proportonal dfferentaton resultng from past servce rate allocatons, and nfer the adjustment r = f(d ) needed to attenuate ths devaton. If no feasble servce rate allocaton for meetng all delay guarantees exst at the n-th event, or f there s a buffer overflow at the n-th event, traffc must be dropped, ether from a new arrval or from the current backlog. The loss guarantees determne whch class(es) suffer(s) traffc drops at the n-th event. To enforce loss guarantees, we rewrte the loss rate, defned by Eqn. (5), as a dfference equaton p = p (n ) A (n ) A + l A. (6) From Eqn. (6), we can determne how the loss rate of class evolves f traffc s dropped from class at the n-th event. Thus, we can determne the set of classes that can suffer drops wthout volatng absolute loss guarantees. In ths set, we choose the class whose loss rate dffers by the largest amount from the objectve of Eqn. (9). Havng expressed the servce rate and the loss rate n terms of a recurson, we can characterze the servce rate allocaton and droppng algorthm as feedback control For any expresson expr, we defne χ expr = f expr s true and χ expr = 0 otherwse. problems. In the next sectons, we wll descrbe two feedback problems: one for delay and absolute rate dfferentaton ( delay feedback loop ), and one for loss dfferentaton ( loss feedback loop ). We descrbe the nteracton of the two feedback problems n Secton V. III. THE DELAY FEEDBACK LOOP In ths secton, we present feedback loops whch enforce the desred delay and rate dfferentaton gven by Eqs. (6), (8), and (9). We have one feedback loop for each class wth proportonal delay guarantees. In the feedback loop for class, we characterze changes to servce rate r by approxmatng the non-lnear effects of the servce rate adjustment on the delays by a lnear system, and derve stablty condtons for the lnearzed control loop. A. Objectve Let us assume for now that all classes are offered proportonal delay guarantees. Later, ths assumpton wll be relaxed. The set of constrants gven by Eqn. (9) leads to the followng system of equatons: D = k D,. D N = ( N ) j= k j D. (7) Let m = j= k j for >, and m =. We defne a weghted delay of class at the n-th event, denoted by D, as D = N D. k=, k m k By multplyng each lne of Eqn. (7) wth j m j, we see that the desred proportonal delay dfferentaton s acheved for all classes f Eqn. (9) s equvalent to where, j, n : D = D j. (9), n : D = D, D := D. N We set D to be the set pont common to all delay feedback loops. The feedback loop for class reduces the dfference D D of class from the common set pont D. Remark: We vew event numbers, n, as samplng tmes on a vrtual tme axs n whch events are equdstant. Hence,

5 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE 5 convergence of the control loop apples to vrtual tme. However, the relatonshp between delay and rate s ndependent of the tme axs chosen. By vrtue of ths ndependence, and snce real-tme s monotoncally ncreasng wth vrtual tme, we make the assumpton that the skew between vrtual-tme and real-tme can be neglected, and that the convergence condton we present later apples to real-tme as well. B. Servce Rate Adjustment Next, we determne how to adjust the servce rate to acheve the desred delay dfferentaton. Let e, referred to as error, denote the devaton of the weghted delay of class from the set pont,.e., e = D D. () Note that the sum of the errors s always zero, that s, for all n, e = ND D = 0. If proportonal delay dfferentaton s acheved, we have e = 0 for all classes. We use the error e to compute the servce rate adjustment r needed for class to satsfy the proportonal delay dfferentaton constrants. From Eqn. (), we note that f e < 0, D > D, class delays are too hgh wth respect to the desred proportonal delay dfferentaton. Therefore, r must be ncreased. Conversely, e > 0 ndcates that class delays are too low, and r must be decreased. Hence, the rate adjustment r s a decreasng functon of the error e, wrtten as r = f(e ), where f(.) s a monotoncally decreasng functon. We choose r = K e, () where K < 0, whch, n feedback control termnology, s the controller. An advantage of ths controller s that t requres a sngle multplcaton, and hence s easly mplemented n a real system. Another advantage s that, at any n, we have r = K e = 0. (5) Therefore, the controller produces a work-conservng system, as long as the ntal condton r (0) = C s satsfed. Note that systems that are not work-conservng,.e., where the lnk may be dle even f there s a postve backlog, are undesrable for networks that need to acheve a hgh resource utlzaton. We then express lmts on K so that the feedback loops are stable. Let us defne r as the average servce rate experenced by the class- traffc departng at the n- th event over the tme ths class- traffc was backlogged. Under the assumpton that the backlog B does not change sgnfcantly durng the tme a partcular traffc arrval s backlogged, we can wrte D B /r. Further, f we can assume that changes to the average servce rate, defned as r = r r (n ), are small compared to the average servce rate,.e., r r, then we can approxmate the effects of changes to the rate allocaton on the changes to the delay by a lnear relatonshp. We refer to [0] for detals of these arguments. Wth the above approxmatons, we can desgn K so that the feedback loop, composed of the controller and the effects of the servce rate adjustment on the delay, s lnear and tme-nvarant. We can then derve a stablty condton on the worst-case of the approxmate, lnearzed model. The stablty condton on the lnearzed approxmate model, presented n detal n [0], results n the followng condton { } mn B j m j D K 0. (6) We emphasze that the assumptons made do not hold n general. Thus, whle we cannot make any clam as to the stablty of the delay feedback loops resultng from the analyss presented here, the numercal data n Secton VI suggests that the loops converge adequately well. To satsfy the constrants r r,mn, we may need to clp r when the new rate s below the mnmum. Ths, however, may volate the work-conservng property resultng from Eqn. (5). Hence, we use the followng to compute K that would satsfy the saturaton constrant r (n ) + Ke r,mn, and apply that K to all control loops. The above mples that we must have ( ) r,mn r (n ) K max. (8) e ( ) r,mn r If max (n ) e > 0, we see that we cannot have K < 0. In other words, we cannot satsfy absolute delay and rate guarantees and proportonal delay dfferentaton at the same tme. In such a case, we relax ether Eqn. (6) or (8) accordng to the gven precedence order on the servce guarantees. Remark: If proportonal delay dfferentaton s requested for some, but not for all classes, constrants as n Eqn. (7) can be defned for each group of classes wth contguous ndces. Then, the feedback loops are constructed ndependently for each group.

6 6 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE IV. THE LOSS FEEDBACK LOOP We now descrbe the feedback loop whch controls the traffc dropped from a class to satsfy proportonal loss dfferentaton wthn the lmts mposed by the absolute loss guarantees. As before, we assume that all classes have proportonal loss guarantees. The assumpton s relaxed smlarly as descrbed n the remark at the end of Secton III. Traffc must be dropped at the n-th event ether f there s a buffer overflow or f absolute delay guarantees cannot be satsfed gven the current backlog. To prevent buffer overflows at the n-th event, the followng condton must hold: B N ( Bk (n ) + a k l k ) k= t(n )C. (9) To provde absolute delay and rate guarantees, the followng condton must be satsfed C N k= { Bk (n ) r k (n ) t(n ) max d k D k + a k l k d k D k, µ k χ Bk >0 }. (0) To choose the amount of traffc to drop from each class so that Eqs. (9) and (0) hold, we defne the weghted loss rate to be p = N p, j=, j m j where m = j= k j for > and m =. Wth ths defnton, Eqn. (0) s equvalent to (, j), n : p = p j. We choose the followng set pont for the loss feedback loop p = p, N and we use the set pont to descrbe an error e = p p. To reach the set pont, the error s decreased by ncreasng p for classes that have e > 0 as follows. Let,,..., R be an orderng of the class ndces from all backlogged classes, that s, B k > 0 for k R, such that e s e r f s < r. Traffc s dropped n the order of,,..., R. Absolute loss guarantees mpose an upper bound, l, on the traffc that can be dropped at event n from class. The value of l s determned from Eqs. (7) and (6) as l = A L p (n )A (n ). If the condtons n Eqs. (9) and (0) are volated, traffc s dropped from class untl the condtons are satsfed, or untl the maxmum amount of traffc l has been dropped. Then traffc s dropped from class, and so forth. Suppose that the condtons n Eqs. (9) and (0) are satsfed for the frst tme f lj traffc s dropped from classes j =,,..., ˆk, and ˆx l ˆk traffc s dropped from class ˆk, then we obtan: l = l f =,,..., ˆk, ˆx f = ˆk, 0 otherwse. (6) If l k = lk for all k =,,..., R, we allow absolute delay and rate condtons to be volated. In other words, condton (0) s relaxed. The loss feedback loop never ncreases the maxmum error e, f e > 0 and more than one class s backlogged. Thus, the errors reman bounded and the algorthm presented wll not engage n dvergent oscllatons around the target value p. Addtonally, the loss feedback loop and the delay feedback loops are ndependent of each other, snce we always drop traffc from the tal of each per-class buffer, losses do not have any effect on the delays of traffc admtted nto the transmsson queue. V. IMPLEMENTATION We mplemented the algorthms presented n Sectons III and IV on PC-routers runnng the FreeBSD v. [] operatng system, usng the ALTQ v.0 package []. ALTQ allows programmers to modfy the operatons of the transmsson queue n the IP layer of the FreeBSD kernel. Our mplementaton s avalable to the publc at edu/software.html. For a detaled dscusson of the mplementaton ssues, we refer the reader to []. In ths paper, we wll only dscuss the operatons performed n our mplementaton when a packet s entered nto the transmsson queue of an IP router (packet enqueueng) and when a packet s selected for transmsson (packet dequeueng). We use the DSCP feld n the header of a packet to dentfy the class ndex of an IP packet. The DSCP feld s set by the edge router; n our testbed mplementaton, ths s the frst router traversed by a packet. In our mplementaton, we chose the followng precedence order for relaxng constrants. Absolute loss guarantees have hgher precedence than absolute delay and

7 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE 7 rate guarantees, whch have n turn hgher precedence than proportonal guarantees. Source Source A. Packet Enqueueng The enqueue procedure are the operatons executed n the IP layer when a packet s entered nto the transmsson queue of an output lnk. Snce the FreeBSD kernel s sngle-threaded, the executon of the enqueue procedure s strctly sequental. The enqueue procedure performs the droppng decsons and the servce rate allocaton. We avod floatng pont operatons n the kernel of the operatng system, by expressng delays as machne clock cycles, servce rates as bytes per clock cycle (multpled by a scalng factor of ), and loss rates as fractons of. Then, 6-bt (unsgned) ntegers provde a suffcent degree of accuracy. In our modfed enqueue procedure, the transmsson queue of an output lnk has one FIFO queue for each class, mplemented as a lnked lst. We lmt the total number of packets that can be queued to B = 00. Whenever a packet s entered nto the FIFO queue of ts class, the arrval tme of the packet s recorded, and the watng tmes of the packets at the head of each FIFO queue are updated. The enqueue procedure uses the loss feedback loop descrbed n Secton IV to determne f and how much traffc needs to be dropped from each class. In our mplementaton, the algorthm of Secton IV s run twce. The frst tme, buffer overflows are resolved by gnorng condton (0); The second tme, volatons of absolute delay and rate guarantees are resolved by gnorng condton (9). Next, the enqueue procedure computes new values for r,mn from Eqn. (), and determnes new servce rates, usng Eqs. (5) and (), wth the constrants on K gven n Eqs. (6) and (8). If no feasble value for K exsts, Eqn. (6) s gnored, thereby gvng absolute delay guarantees precedence over proportonal delay guarantees. B. Packet Dequeueng The dequeue procedure selects one packet from the backlog for transmsson. In our mplementaton, dequeue selects one of the traffc classes, and pcks the packet at the head of the FIFO queue for ths class. The dequeue procedure uses a rate-based schedulng algorthm to adapt the transmsson rates r from a flud-flow vew to a packet-level envronment. In our mplementaton, we use a modfed Defct Round Robn (DRR, []) schedulng algorthm. Let Xmt denote the number of bytes of class- traffc that have been transmtted n the current busy perod, the scheduler selects a Source Router Bottleneck Router Snk Bottleneck Router Snk Snk Fg.. Network Topology. All lnks have a capacty of 00 Mbps. We measure the servce provded by Routers and at the ndcated bottleneck lnks. Class Servce Guarantees d L µ k k 8 ms % 5 Mbps N/A N/A TABLE I SERVICE GUARANTEES. THE GUARANTEES ARE IDENTICAL AT EACH ROUTER. packet from class for transmsson f { = arg max R out k Xmt k }. k In other words, the dequeue procedure selects the class whch s the most behnd ts theoretcal output curve. VI. EVALUATION We present expermental measurements of our mplementaton of the Quanttatve Assured Forwardng servce on a testbed of PC routers. The PCs are Dell PowerEdge 550 wth GHz Intel Pentum-III processors and 56 MB of RAM. The system software s FreeBSD. and ALTQ.0. Each system s equpped wth fve 00 Mbps- Ethernet nterfaces. In our experments we determne f and how well our algorthm provdes the desred servce dfferentaton on a per-node bass. In addton, we want to observe the stablty of the feedback loops. We use a local network topology usng pont-to-pont Ethernet lnks as shown n Fgure. All lnks are fullduplex and have a capacty of C = 00 Mbps. Three PCs are set up as routers, ndcated n Fgure as Router, and. Other PCs are actng as sources and snks of traffc. The topology has two bottlenecks: the lnk between Routers and, and the lnk between Routers and.

8 8 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE Rato of Delays Class /Class Class /Class (a) Ratos of Delays. Rato of Delays (a) Ratos of Delays. Class /Class Class /Class Delay (ms) Rato of Loss Rates Loss Rate (%) Throughput (Mb/s) 8 Delay Bound Class (b) Class- Delays. Class /Class Class /Class (c) Ratos of Loss Rates. Class (d) Class- Loss Rate Class Guarantee Total Class 60 Class Class Class (e) Throughput. Fg.. Router. The graphs show the servce obtaned by each class at the output lnk of Router. Delay (ms) Rato of Loss Rates Loss Rate (%) Throughput (Mb/s) 8 Delay Bound Class (b) Class- Delays. Class /Class Class /Class (c) Ratos of Loss Rates. Class (d) Class- Loss Rate Class Guarantee Total Class Class Class Class (e) Throughput. Fg. 5. Router. The graphs show the servce obtaned by each class at the output lnk of Router. As mentoned earler, the buffer sze at the output lnk of each router s set to B = 00 packets. We consder four traffc classes wth servce guarantees as summarzed n Table I. Recall that all servce guarantees are per-node guarantees. They are enforced ndependently at each router. Sources, and send traffc to Snks, and, respectvely. Each source transmts traffc from all four classes. The traffc mx, the number of flows per class, and the characterzaton of the flows, s dentcal for each source, and as shown n Table II. Each source transmts 6 flows from each of the classes. Class traffc conssts of on-off UDP flows, and the other classes consst of greedy TCP flows. All sources start transmttng packets wth a fxed sze of 0 Bytes at tme t = 0 untl the end of the experments at t = 60 seconds. Traffc s generated usng the netperf v.pl tool [5]. The network load s ntally zero and quckly ramps up to generate an overload at the bottleneck lnks of Fgure. Congeston control at the TCP sources mantans the total load at a level of about 99% of the lnk capacty [0]. We measure the delay, the loss rate, and the through-

9 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE 9 Class No. of Type flows Protocol Traffc 6 UDP On-off 6 TCP Greedy 6 TCP Greedy 6 TCP Greedy TABLE II TRAFFIC MIX. THE TRAFFIC MIX IS IDENTICAL FOR EACH SOURCE-SINK PAIR. THE ON-OFF UDP SOURCES SEND BURSTS OF 0 PACKETS DURING AN ON-PERIOD, AND HAVE A 50 MS OFF-PERIOD. ALL TCP SOURCES ARE GREEDY, I.E., THEY ALWAYS HAVE DATA TO TRANSMIT, AND RUN THE NewReno CONGESTION CONTROL ALGORITHM. put of each traffc class at the output lnks of Routers and (the lnks whch go to the bottleneck lnks). Delays are measured as the watng tme of a packet n the transmsson queue,.e., as the dfference of the tmes read of the machne clock when the packet enters and departs the transmsson queue. Throughput and loss rates are obtaned from reports generated every 0.5 sec by the OS kernel. In the plots, whch summarze our measurements, we depct delay measurements of ndvdual packets. Measurement of delay ratos, loss rates, ratos of loss rates and throughput are shown as averages over a sldng wndow of sze 0.5 sec. In Fgures and 5, we present our measurements of the servce receved at the bottleneck lnks of Routers and, respectvely. Fgs. (a) and 5(a) depct the ratos of the delays of Classes and, and the delays of Classes and. The plots show that the target value of k = (from Table I) s acheved. The plots ndcate that the delay feedback loops appear to be stable, despte the smplfed model we used for determnng K n Secton III. In Fgs. (b) and 5(b) we show the delay of Class- packets at Router and Router. The delay bound of d = 8 ms s satsfed, wth few (<.5%) exceptons at tmes when t s not possble to satsfy smultaneously absolute loss and delay guarantees; as dscussed n Secton V, such a conflct s resolved by gvng precedence to the loss guarantee. Note that even f delay bounds are volated, no class- packet experences a delay whch exceeds 0 ms at ether Router or. Delay values, averaged over sldng wndows of sze 0.5 s, of other classes (not shown) are n the range -50 ms. In Fgs. (c) and (d), and Fgs. 5(c) and (d), we show the measurements of the loss rates. Fgs. (c) and 5(c) depct the ratos of loss rates for Classes and, and for Classes and. The desred ratos of k = k = are mantaned most of the tme. Snce the buffers at the routers are empty at the begnnng of the experment, there are no losses ntally. As Fgs. (d) and 5(d) ndcate, the bound on the loss rates for Class of L = % s always kept (Recall that we gve hghest precedence to absolute loss guarantees.) We note that the maxmum loss rate of classes (not shown) s below % over the entre experment. Fnally, n Fgs. (e) and 5(e) we nclude the throughput measurements of all classes. We observe that the rate guarantee for Class of µ = 5 Mbps s mantaned. The total throughput of all classes, labeled n Fgs. (e) and 5(e) as Total, s close to the lnk capacty of 00 Mbps at each router. In summary, the measurement experments of an overloaded network wth multple bottlenecks show that our feedback algorthms acheve the desred servce dfferentaton, and utlze the entre avalable bandwdth, whle mantanng stablty throughout. We present a bref evaluaton of the overhead of the feedback-based algorthms. We have measured the number of CPU cycles consumed by the enqueue and dequeue procedures, by readng the tmestamp counter regster of the Pentum processor. We measured the average and standard devaton of the number of cycles over 500,000 packet transmssons on a heavly loaded lnk, usng the topology and traffc pattern descrbed n Fgure and Table II. We compare measurements for a set of four classes wth constrants gven n Table I, to a system of four classes wthout any guarantees. The measurements of the number of cycles, collected for Router, are shown n the followng table. Guaran- Enqueue Dequeue tees Avg. Std. Dev. Avg. Std. Dev. wth wthout The table shows that the overhead for the enqueue operaton, whch mplements the feedback algorthms, s sgnfcant. At the same tme, the numbers ndcate that a GHz PC can enqueue and dequeue more than 50,000 packets per second. Consderng that the average sze of an IP packet on the Internet s P = 5. bytes [6], ths results n a maxmum throughput of 86 Mbps. VII. CONCLUSIONS We presented the Quanttatve Assured Forwardng servce, whch provdes proportonal dfferentaton on loss and delay and absolute servce guarantees on loss, throughput and delay for classes of traffc. We proposed a feedback based algorthm for realzng the Quanttatve Assured Forwardng servce at a router. The algorthm does not requre pror knowledge of traffc arrvals, and does not rely on sgnalng. At tmes when not all

10 0 TO APPEAR IN PROCEEDINGS OF IEEE INFOCOM 00, c IEEE absolute servce guarantees can be satsfed smultaneously, the algorthm relaxes some of the guarantees by usng a prorty order. The algorthm has been mplemented n FreeBSD PC-routers and the mplementaton s avalable at software.html. Through experments n a network of PC-routers, we showed that the proposed algorthm could fully utlze the avalable capacty of 00 Mbps. The measurements showed that the servce guarantees of the Quanttatve Assured Forwardng servce are enforced. In ongong work, we are conductng experments for an emprcal evaluaton of the robustness of the proposed feedback algorthms, where we vary the network topology, the servce guarantees, and the network load. REFERENCES [] J. Henanen, F. Baker, W. Wess, and J. Wroclawsk, Assured forwardng PHB group, IETF RFC 597, June 999. [] S. Blake, D. Black, M. Carlson, E. Daves, Z. Wang, and W. Wess, An archtecture for dfferentated servces, IETF RFC 75, December 998. [] K. Nchols, S. Blake, F. Baker, and D. Black, Defnton of the dfferentated servces feld (DS feld) n the IPv and IPv6 headers, IETF RFC 7, December 998. [] C. Dovrols, Proportonal dfferentated servces for the Internet, Ph.D. thess, Unversty of Wsconsn-Madson, Dec [5] C. Dovrols and P. Ramanathan, Proportonal dfferentated servces, part II: Loss rate dfferentaton and packet droppng, n Proceedngs of IWQoS 000, Pttsburgh, PA, June 000, pp [6] C. Dovrols, D. Stlads, and P. Ramanathan, Proportonal dfferentated servces: Delay dfferentaton and packet schedulng, n Proceedngs of ACM SIGCOMM 99, Boston, MA, Aug. 999, pp [7] Y. Moret and S. Fdda, A proportonal queue control mechansm to provde dfferentated servces, n Proceedngs of the Internatonal Symposum on Computer and Informaton Systems (ISCIS), Belek, Turkey, Oct. 998, pp. 7. [8] T. Nandagopal, N. Venktaraman, R. Svakumar, and V. Barghavan, Delay dfferentaton and adaptaton n core stateless networks, n Proceedngs of IEEE INFOCOM 000, Tel-Avv, Israel, Apr. 000, pp. 0. [9] S. Bodamer, A schedulng algorthm for relatve delay dfferentaton, n Proceedngs of the IEEE Conference on Hgh Performance Swtchng and Routng (ATM 000), Hedelberg, Germany, June 000, pp [0] L. Essaf, G. Bolch, and H. de Meer, Dynamc prorty schedulng for proportonal delay dfferentated servces, Tech. Rep. TR-I- 0-0, Unversty of Erlangen, Mar. 00. [] H. Sato, C. Lukovszk, and I. Moldován, Local optmal proportonal dfferentated servces scheduler for relatve dfferentated servces, n Proceedngs of Nnth IEEE Internatonal Conference on Computer Communcatons and Netowrks (ICCCN 000), Las Vegas, NV, Oct. 000, pp [] U. Bodn, A. Jonsson, and O. Schelen, On creatng proportonal loss dfferentaton: predctablty and performance, n Proceedngs of IWQoS 00, Karlsruhe, Germany, June 00, pp [] J. Lebeherr and N. Chrstn, JoBS: Jont buffer management and schedulng for dfferentated servces, n Proceedngs of IWQoS 00, Karlsruhe, Germany, June 00, pp [] A. Stregel and G. Manmaran, Packet schedulng wth delay and loss dfferentaton, Computer Communcatons, vol. 5, no., pp., Jan. 00. [5] P. Hurley, J.-Y. Le Boudec, P. Thran, and M. Kara, ABE: provdng low delay servce wthn best effort, IEEE Networks, vol. 5, no., pp , May 00. See also abeservce.org. [6] R. R.-F. Lao and A. T. Campbell, Dynamc core provsonng for quanttatve dfferentated servce, n Proceedngs of IWQoS 00, Karlsruhe, Germany, June 00, pp [7] C. V. Hollot, V. Msra, D. Towsley, and W. Gong, On desgnng mproved controllers for AQM routers supportng TCP flows, n Proceedngs of IEEE INFOCOM 00, Anchorage, AK, Apr. 00, vol., pp [8] C. Lu, J. A. Stankovc, G. Tao, and S. H. Son, Feedback control real-tme schedulng: Framework, modelng and algorthms, Journal of Real Tme Systems, Mar. 00, Specal Issue on Control-Theoretcal Approaches to Real-Tme Computng. In press. [9] Y. Lu, A. Saxena, and T. F. Abdelzaher, Dfferentated cachng servces; A control-theoretcal approach, n Proceedngs of the st Internatonal Conference on Dstrbute d Computng Systems, Phoenx, AZ, Apr. 00, pp [0] N. Chrstn, J. Lebeherr, and T. F. Abdelzaher, A quanttatve assured forwardng servce, Tech. Rep. CS-00-, Unversty of Vrgna, Aug. 00, ftp://ftp.cs.vrgna.edu/ pub/techreports/cs-00-.pdf. [] The FreeBSD project, [] K. Cho, A framework for alternate queueng: towards traffc management by PC-UNIX based routers, n Proceedngs of USENIX 98 Annual Techncal Conference, New Orleans, LA, June 998. [] N. Chrstn and J. Lebeherr, The QoSbox: A PC-router for quanttatve servce dfferentaton n IP networks, Tech. Rep. CS- 00-8, Unversty of Vrgna, Nov. 00, ftp://ftp.cs. vrgna.edu/pub/techreports/cs-00-8.pdf. [] M. Shreedhar and G. Varghese, Effcent far queueng usng defct round-robn, IEEE/ACM Transactons on Networkng, vol., no., pp , June 996. [5] R. Jones, netperf: a benchmark for measurng network performance - revson.0, Informaton Networks Dvson, Hewlett-Packard Company, Feb See also netperf.org. [6] Packet szes and sequencng, May 00, http: // packetszes.