Avodng congeston through dynamc load control Vasl Hnatyshn, Adarshpal S. Seth Department of Computer and Informaton Scences, Unversty of Delaware, Newark, DE 976 ABSTRACT The current best effort approach to qualty of servce n the Internet can no longer satsfy a dverse varety of customer servce requrements, and that s why there s a need for alternatve strateges. In order to solve ths problem a number of servce dfferentaton models have been proposed. Unfortunately, these schemes often fal to provde proper servce dfferentaton durng perods of congeston. To deal wth the ssue of congeston, we ntroduce a load control mechansm that elmnates congeston based on the feedback from the network core by dynamcally adjustng traffc load at the network boundary. We ntroduce four methods for calculatng load dstrbuton among the ngress routers and among dfferent flows n each ngress router, and we evaluate these proposed methods through smulaton. Keywords: Qualty of servce, dynamc admsson control, load dstrbuton, network feedback. INTRODUCTION As a dverse varety of applcatons wth dfferent customer servce requrements are begnnng to access the Internet, the current best effort approach to qualty of servce n the Internet s no longer suffcent. As people become wllng to pay more for servces that satsfy ther applcaton requrements, the one-servce-for-all approach of today s Internet wll become obsolete, creatng a need for alternatve strateges. In order to solve ths problem, a number of servce dfferentaton models have been proposed. Dfferentated Servce archtecture [] ntroduced by IETF s DffServ workng group, core-stateless far queung [3] proposed by Stoca et al, and proportonal servce dfferentaton framework [4-6] proposed by Dovrols et al are currently among the most popular approaches. Unfortunately, these schemes often fal to provde proper servce dfferentaton durng perods of congeston. For example, IETF s DffServ model fals to provde far resource allocaton and far servce degradaton durng the perods of congeston [7,0-] because of statc resource allocaton. On the other hand, the proportonal servce dfferentaton model does not volate relatve guarantees under any network condton. However n ths model, the lack of mechansms for lmtng the amount of data njected nto the network can reduce the absolute level of QoS below user expectatons [4-6]. We beleve that one way to deal wth ths problem s to ntroduce a dynamc load control mechansm at the boundary routers. The dea behnd dynamc load dstrbuton s dfferent from the dea of admsson control. Although both mechansms adjust themselves based on network feedback, they determne ther adjustments dfferently. Whle admsson control decdes whether or not to admt users nto the network, the dynamc load control mechansm admts all the users and only when congeston arses does t adjust user sendng rates. Fgure llustrates the dea behnd the dynamc load control. As the fgure shows, the traffc enters the network doman at the boundary router B, traverses the network n some fashon, and then exts ths network doman at the boundary router B. If congeston arses, the core routers provde feedback to the ngress routers. Based on ths feedback, the ngress routers adjust the amount of traffc admtted nto the doman. The dea of adjustng admsson control based on network feedback s not. Chow et al also used such an dea n ther work []. They proposed to perodcally probe the network and, based on the obtaned results, adjust admsson control at the ngress nodes. However, they only ntroduced a general framework for the feedback-based dynamc admsson control and dd not pursue further nvestgaton of ths dea. The framework for Explct Congeston Notfcaton [9] also reles on a smlar feedback mechansm. In ths approach, when the core routers experence congeston, they set Congeston Experenced (CE) bts n the IP headers of packets flowng through them n order to Throughout ths paper we wll also refer to the boundary nodes at whch traffc enters a doman as ngress routers and we wll call the boundary nodes where traffc leaves a doman as egress routers.
notfy the sources to reduce ther rates. The Explct Congeston Notfcaton model assumes that the sources wll reduce ther rates upon recepton of the CE marked packets, whch may not always be true. Incomng traffc Dynamc Load Control Boundary Router B Feedback Core router C Feedback Core router C3 Core router C Boundary Router B Outgong Traffc Fgure. Scenaro for Dynamc Load Control In ths paper, we extend these deas and provde a protocol for dynamc load dstrbuton based on network feedback. The rest of the paper s organzed as follows. Secton ntroduces the general dea of our model, ncludng a detaled explanaton of the message exchange mechansm used n ths model. Secton 3 descrbes possble technques by whch each ngress node can reduce ts sendng rate to elmnate congeston at the congested nterface. Secton 4 provdes an ntal expermental evaluaton of our scheme. We nclude concludng remarks and present drectons for future work n Secton 5.. DYNAMIC LOAD DISTRIBUTION FEEDBACK CONTROL SCHEME.. General Idea The man dea of the feedback-based dynamc load control scheme s to dynamcally adjust the traffc sendng rates at the network boundares. When the network s congeston-free, the boundary nodes allow users to nject as much data as they desre. However, when the network experences congeston, the boundary nodes start lmtng the amount of traffc that can be admtted nto the network. Durng these perods of congeston, the core routers provde ndcatons to the boundary nodes to slow down. We use an explct message passng mechansm for provdng congeston notfcatons to the boundary nodes. In our model, these congeston notfcaton messages contan nformaton about whch core router nterface s congested and ts congeston level. Ths nformaton allows each boundary node to determne by how much t should reduce the overall traffc rate to elmnate congeston. Once the overall reduced rate s calculated, the boundary router can calculate correspondng sendng rates for all user flows that contrbute to the congeston. F F B C B C C3 From F and F F3 B4 B3 Fgure. Congeston-free network From F3 Consder the stuaton shown n Fgure. Ths fgure shows a congeston-free network, where the boundary node B accepts traffc flows F, F, and F3 and forwards them towards ther correspondng destnatons. In partcular, flows F and F pass through core routers C and C and then ext ths doman at the egress node B. Flow F3 passes through the routers C and C3 and then exts the network doman through the boundary node B3. Fgure depcts a stuaton where all the nterfaces of the core routers are congeston-free and no load control at the boundary nodes s
requred. Fgure 3 shows what happens when one of the core lnk nterfaces becomes congested. Suppose that flow F4 becomes actve and enters the doman at boundary node B4. Flow F4 follows the path C3, C, B and creates congeston on the lnk between C and B. C dentfes ths stuaton and generates congeston notfcatons to all the boundary nodes that send ther traffc through the congested lnk. Congeston notfcatons travel through the routers C and C3 to reach boundary nodes B and B4 respectvely. F F F3 B F4 cn B4 C cn cn C3 cn C Fgure 3. Node actons durng the congeston When B and B4 receve ther congeston notfcatons, they calculate by how much they should reduce ther traffc rate to elmnate congeston. Then B and B4 dentfy the flows that contrbute to the congeston between C and B (F, F and F4). After that, B and B4 dstrbute the calculated rate reducton among these flows, by controllng each flow s load accordngly. Fgure 4 llustrates how the network stuaton changes after the actvaton of load control for the selected flows. B3 B From F, F, and F4 From F3 Lnk Interface Congested Not Congested F F F3 B B4 C C3 C B3 B From F, F, and F4 Load Control -- Inactve -- Actve F4 From F3 Fgure 4. Congeston-free network after the adjustng load control. Overvew of the message exchange mechansm In our model, we assume that for each source host that sends traffc through a gven ngress node, the ngress node knows the lst of destnaton hosts for whch the traffc s ntended. We record ths nformaton n the Servce Level Agreement (SLA) establshed between each source and the network doman. The SLA wll also nclude nformaton about the mnmum provsoned rate for each source-destnaton host par. Durng congeston, the boundary node wll not reduce a flow s sendng rate below ths mnmum provsoned rate specfed n the SLA. In our scheme we assume that the network s well provsoned, and that congeston does not occur when all the users send traffc at ther mnmum provsoned rates. Throughout the paper, whenever we refer to a flow, we wll mean packets flowng between a sourcedestnaton host par specfed n the SLA, as contrasted wth the tradtonal defnton of a flow, as stream of packets flowng between a par of applcatons. Table shows an example of Servce Level Agreements establshed between users (source hosts) and a network doman. When congeston occurs, the boundary node receves a congeston notfcaton that specfes the address of the nterface where congeston occurred and the level of congeston on that nterface. Not all the flows that enter the doman at a partcular ngress node travel through the congested nterface; therefore, t s mpossble to determne whch flows
should slow down based only on the nformaton provded n the congeston notfcaton message. To remedy ths stuaton, we mantan two data structures n addton to the SLA n each boundary node, called path table and router table. These tables allow the boundary node to dentfy the flows that should be slowed down when congeston occurs. These data structures are generated and mantaned based on nformaton collected by probe messages, whch the ngress nodes perodcally transmt on each currently actve path, n order to dscover the path changes. Probe messages are also generated when a flow becomes actve for determnng the flow s path wthn ths doman. The probes collect the lst of routers they traversed wthn the doman and the ngress nodes use ths nformaton to mantan the path and router tables. The path table contans a complete route wthn the doman that the flows follow to ther destnaton. The router table keeps the router nterface addresses and the lst of flows that pass through those nterfaces. Thus, whenever a flow becomes actve or the path tmer expres, the ngress boundary node generates a probe message that follows the route toward a partcular destnaton. The probe termnates ts progress at the egress node, whch s the last router wthn the doman on the path to the destnaton. The egress node forwards the probe back to the ngress node that generated t. We refer to the message forwarded by the egress node as a probe reply message. Flow ID Source Egress Mn. Provsoned Rate 9.0.. B 00 Kbps 9.0.. B3 50 Kbps 3 9.0.3.5 B 90 Kbps 4 9.0.3.5 B3 300 Kbps Table. Example of Servce Level Agreement Ingress B F, F C C Egress B 9.0.. F3 From F and F3 9.0.3.5 C3 Egress B3 Fgure 5. Network Topology From F Destnaton Path Lst of Flows Tmer Mn. Provsoned Rate B B, C, C, B F, F3 0.39 390 Kbps B3 B, C, C3, B3 F.7 550 Kbps Table. Path Table for the boundary node B. Router Interface B-C C-C C-C3 C-B C3-B3 Lst of Flows F,F,F3 F, F3 F F, F3 F Table 3 Router Table for the boundary node B. Tables and 3 show an example of the path and router tables respectvely at node B for actve flows wthn the network of Fgure 5. We ndex the path table usng the full path between the current ngress node and the egress router where the flow exts the doman. In addton, each entry n the path table contans a tmer and a lst of actve flows that follow that path. The tmer specfed n the path table s used to determne when the next probe for that path should
be generated. A router table kept n each ngress node s ndexed by the router nterface address and contans the lst of the flows that pass through that nterface. Both of these tables are bult based on the data collected by the probe messages. When a probe reply message arrves at the ngress node, t contans a lst of the nterface addresses and ther current arrval rates. Interface arrval rates are used to update load control for the flows that pass through those nterfaces. The nterface addresses are used to update the path and router tables. If the probe reply message was generated due to tmer expraton and contans a route, then the old route entry n the path table should be replaced wth the route nformaton. Moreover, the router table should be updated. Frst, we dentfy the lst of flows that follow a path, let us call t flows-to-update lst. Then, we remove the lst of flows-to-update from each router nterface entry of the router table that was part of the old route. Smlarly, we add the lst of flows-to-update to each router nterface entry of the router table that s part of the route. In the case when the probe message was generated due to actvaton of a flow, we update the path table as follows. If returned path already exsted we smply add a flow to the lst of current flows for that entry, otherwse we create a entry n the path table. Smlarly, we update the router table by ether modfyng the lst of flows for all exstng nterfaces, or by addng entres nto the router table f unknown nterfaces were encountered. Now, let us consder an example of path and router tables updates. Consder that, n addton to the actve flows shown n Fgure 6, flow F4 becomes actve and starts sendng traffc. When the frst packet of the flow F4 arrves at the boundary node B, a probe s generated and forwarded towards ts destnaton. When the egress node B3 receves the probe generated by the ngress node B, t copes collected data nto the probe reply message and sends t to B. Upon arrval of the probe reply message at the boundary node B, the correspondng entres n the path and router tables are updated. Fgure 6 llustrates how the probe message exchange works and Tables 4 and 5 show how path and router tables are updated. Ingress B F, F Probe C C 9.0.. F3, F4 Probe Probe Reply Probe Probe Reply C3 9.0.3.5 Probe Reply Egress B From F and F3 Egress From F, F4 B3 Fgure 6. Network Topology Destnaton Path Lst of Flows Tmer Mn. Provsoned Rate B B, C, C, B F, F3.09 390 Kbps B3 B, C, C3, B3 F, F4 0.00 550 Kbps Table 4. Updated path table for the boundary node B. In general, the message exchange mechansm of the feedback-based dynamc load control scheme conssts of two parts. The frst part called probe message exchange keeps track of the routng changes and allows boundary routers to dentfy the flows to whch load control should be appled. The second part called congeston notfcaton exchange, provdes ndcaton to the boundary nodes about when and how to adjust the load control. The probe message passng accomplshes two goals. Frst of all t allows core routers to dentfy whch ngress routers should be notfed durng the congeston, and t allows ngress routers to dentfy the flows that should reduce ther rates. Snce boundary nodes generate probe messages each tme a flow becomes actve, these messages allow a core router to keep track of the boundary nodes that are sendng traffc through t. When the core router receves a probe message, t records the address of the boundary node that generated ths message and the address of the outgong nterface on whch the probe message departed. So for each outgong nterface the core router keeps a lst of ngress
routers that send traffc through ths nterface. Durng congeston, the core routers consult ths table and send congeston notfcatons to all the boundary nodes that sent the traffc through the congested nterface. Router Interface B-C C-C C-C3 C-B C3-B3 Lst of Flows F,F,F3,F4 F, F3 F, F4 F, F3 F, F4 Table 5. Updated router table for the boundary node B. The core nodes generate congeston notfcaton when an outgong nterface becomes congested. Core nodes estmate ncomng traffc rate for each outgong nterface. If the estmated rate on a partcular outgong nterface reaches a certan threshold then t sgnfes congeston and notfcaton messages are sent to correspondng boundary nodes. A congeston notfcaton message carres nterface address, estmated arrval rate on that nterface, and the capacty of that nterface. Based on ths nformaton along wth the local flow nformaton, the boundary nodes can calculate the sendng rate for each of the flows that send ts traffc on the congested nterface. 3. RATE REDUCTION TECHNIQUES 3. Calculaton of the congeston reduced aggregated rate The problem of adjustng traffc load at the boundary nodes may be dvded nto two parts. The frst part deals wth the problem of calculatng the total load that a partcular ngress router can generate wthout causng congeston n the network core. The second part of the problem deals wth the queston of how the total load should be dvded among the ndvdual flows. In ths secton we wll ntroduce three smple mechansms for solvng the frst problem. Each ngress router must compute the aggregate rate at whch t transmts traffc through the congested lnk. Ths rate, whch we call Congeston Reduced Aggregated Rate (CRAR), should satsfy two requrements:. The sum of all CRARs from all the ngress nodes should not exceed the congested lnk s capacty.. The congested lnk s utlzaton should reman hgh after all the ngress nodes reduce ther sendng rates. It s very dffcult for the ngress nodes to compute the CRARs that satsfy the above requrements wthout addtonal nformaton about the rate dstrbuton among the varous ngress nodes that send traffc on the congested nterface. Unfortunately, ths nformaton s not avalable to the ngress nodes, whch only have the estmated value of ther own sendng rate and the nformaton about the arrval rate and capacty of the congested nterface. A possble soluton to ths problem s to have the core router montor the rates of traffc reachng t from each ngress node and then provde ths nformaton to the boundary nodes when congeston arses. In order for the core router to estmate the arrval rate of the boundary node, t should know whch flows enter the doman through that ngress node. One way to acheve ths s for the core router to pre-compute and keep nformaton about all the flows that enter ths doman and ther correspondng ngress nodes. Another approach would be to requre all the packets to be marked n such a way that the core router could unquely dentfy the ngress node through whch the packet entered ths doman. Both of these approaches seem to be too heavyweght and would ntroduce too much addtonal complexty nto the core router processng. Because of ths addtonal complexty assocated wth dstrbuton of the boundary node traffc sendng rates among the ngress routers, we decded to estmate the value of the Congeston Reduced Aggregated Rate. We wll frst examne a set of nave methods for calculatng the CRAR at each boundary node. In the frst method, we reduce the rate at each ngress node proportonally to ts sendng rate. We wll use the followng notaton n our subsequent dscusson: R total traffc rate on the congested nterface from all ngress nodes. C capacty of the congested nterface. E excess traffc receved at the congested nterface. R CRAR that arrves at nterface.
R j total traffc rate from the ngress node j to the nterface. S j total provsoned traffc rate from the ngress node j to the nterface. E j excess traffc sent from the ngress node j to the congested nterface. R j CRAR from ngress node j to nterface. In the frst method, called nave method, we calculate the CRAR R j, as follows, reducng sendng rates only of those ngress nodes that send traffc above ther mnmum provsoned rate. E R j = R j, only f R j > S j (3..) R Usng ths method, the boundary nodes wll reduce ther aggregated sendng rates to R. However such rate reducton may not completely elmnate congeston. Consder the stuaton where some ngress nodes send traffc below ther mnmum provsoned rate and therefore do not slow down durng the congeston. For ths case, we can show that reducng the sendng rate n accordance wth nave method wll not elmnate the congeston. We defne congeston as the stuaton when the estmated arrval rate at the lnk s above the lnk s capacty, R > C. The excess traffc s gven by: E = R C (3..) The total rate U from all nodes sendng traffc below ther provsoned rates s: U = (3..3) R j j R j S j Whereas the total rate O from all nodes that exceed ther provsoned rates s: O = (3..4) Obvously, R j j R j > S j R = R = O + U (3..5) j When load control s appled accordng to nave method, the rate from the prevously over-provsoned nodes s adjusted to: O = R j, where R j < R j (3..6) j R j > S j Thus, the CRAR s: R = R = U + O (3..7) j j By re-wrtng (3..7) and by usng defntons (3..) (3..6) we wll obtan that: E O R = O + U = R E > C R (3..8) R Inequalty (3..8) holds snce E s the exact amount by whch we should reduce rate R n order to elmnate congeston. O However, snce the rato s smaller than because of the assumpton that there are some ngress nodes that send R traffc below ther mnmum provsoned rate, we have reduced the rate R by an amount smaller than E. Therefore, n ths case, we wll not elmnate congeston at the core router. However, f all the ngress nodes send above ther mnmum provsoned rates, such rate reducton wll elmnate congeston. Another problem of ths approach s that t j
computes rate reducton based only on the ngress node s sendng rate and congeston level n the core router. Because of that, ths rate reducton method mght cause certan ngress nodes to reduce ther sendng rates below ther correspondng mnmum provsoned rate S j. To elmnate the last problem, we propose two other rate-reducton technques, whch we wll call nave methods and 3. Smlarly to the nave method, we wll reduce sendng rate only at those ngress nodes that send traffc above ther mnmum provsoned rate. Both of these approaches guarantee that the ngress nodes wll not reduce ther sendng rates below ther mnmum provsoned rates. In nave method, the ngress nodes reduce ther sendng rate only by a fracton of the excess rate, and therefore can reduce ther sendng rate by excess rate at the most. E R j = R j E j, only f R j > S j (3..9) R On the other hand, n nave method 3, we explctly prohbt reducng sendng rates below correspondng mnmum provsoned rates. R j = E max R j, S j, only f R j > S j (3..0) R Unfortunately, none of the nave methods can reduce sendng rates to a level that wll elmnate congeston after only a sngle congeston notfcaton. Usng these rate reducton approaches would requre multple congeston notfcaton messages before congeston n the core router s nterface s eradcated. 3. Two-teraton method for computng CRAR We expect that none of the nave technques for computaton of the CRAR at the ngress nodes can guarantee fast convergence to the sendng rate that would elmnate congeston. That s why we propose an alternatve method that s able to reach a sendng rate that wll elmnate congeston after the second congeston notfcaton. In ths scheme, when the frst congeston notfcaton arrves at the ngress nodes, the sendng rate s lowered to a level whch reduces but does not elmnate the congeston. Because of that, the congested nterface of the core router generates another congeston notfcaton. In our scheme, after recevng the second congeston notfcaton, the ngress nodes are able to calculate the exact value of the CRAR, such that congeston n the core router s nterface wll be completely elmnated. In ths secton we wll use the same notatons as n Secton 3., except that we wll dstngush between the rates before the frst and the second computaton of the CRAR. For example, we wll call the amount of excess traffc that nterface receves before the frst and the second rate reductons as E and E respectvely. Also we wll use symbol d to denote the rate reducton rato used to compute the rate after recevng the frst congeston notfcaton. Thus, when the frst congeston notfcaton s generated, the congested nterface has an amount of excess traffc that s equal to E. After the rate s reduced usng reducton rato d, the congested nterface wll have an amount of excess traffc that s equal to E. When an ngress node receves a congeston notfcaton, t computes the frst rate reducton rato as follows: E d = (3..) R Those ngress nodes that send traffc below ther provsoned rates have excess traffc rate equal to zero. Then each ngress router that receves a congeston notfcaton wll compute the CRAR as follows: R j = R j E jd (3..) The excess traffc after ths rate reducton s: E j = E j ( d ) (3..3) We can easly show that ths reducton wll not elmnate congeston and therefore a second congeston notfcaton wll be generated. The frst rate reducton caused the followng decrease n the arrval rate at the congested nterface:
d j E j = E E Based on the knowledge that, for ngress node, the reducton by the congested nterface n the amount of E E overall rate reducton at the congested nterface n the amount of (3..4) d caused an overall rate reducton at E j, we can calculate the reducton rato d, such that t wll cause an E. Based on ths observaton and equaton (3..3), d. In order to be able to construct ths equalty, we assume that we can construct the followng equalty and solve t for the number of the ngress nodes that adjust ther rates remans the same for the perod of the rate reducton. E ( ) j d E j d = d (3..5) E E E d E d = (3..6) d E E Selectng the value of d accordng to equaton (3..6) ensures that the prevously congested nterface wll become congeston-free. To verfy the correctness of equaton (3..6), we wll show that the overall rate reducton after the second congeston notfcaton equals the amount of excess traffc at the congested nterface. d E j = E (3..7) j By applyng equaton (3..3) to the left sde of equaton (3..7) and then by usng equatons (3..4) and (3..6) we obtan: E E d E j = d ( d ) E j = d ( d ) d j j E E ( d ) E d E d E j = = j d E E d Therefore, equaton (3..7) holds, and selecton of the reducton rato for the second teraton accordng to equaton (3..6) wll elmnate congeston on the nterface that generated the congeston notfcaton. 3.3 Reducton of the flow rates durng the congeston When the boundary node receves a congeston notfcaton, t calculates the CRAR, usng one of the methods presented n Sectons 3. and 3.. After that, the boundary node calculates the traffc rate, r, for each flow that contrbutes nto the aggregate that passes through the congested lnk. In ths secton, we wll use a slghtly dfferent notaton n order to make the formulas more readable. R Congeston Reduced Aggregated Rate. R estmated traffc rate generated by the flows passng through the congested nterface. S sum of the mnmum provsoned traffc rates for all the flows that travel to ther destnaton through the congested nterface. k r estmated traffc rate of flow k that passes through the congested nterface. k s mnmum provsoned traffc rate of flow k that passes through the congested nterface. k r traffc rate of flow k that passes through the congested nterface. The dea of ths approach s very smple: we calculate a far share for each flow and we reduce the flow s sendng rate only f t sends traffc above ts far share. We calculate the far share of the flow as follows: k n
R S f = s + (3.3.) S where, ( R S ) s the amount of excess bandwdth avalable for all the flows. Then, we calculate a sendng rate for each flow that sends above ts far share as follows: under over R R S r = + s (3.3.) over S where, under R = r, s the total sendng rate of those flows that send below ther far share. r f over S = s r > f under over ( R R S ), s the total provsoned rate of those flows that send above ther far share. s the excess bandwdth left for sharng among the flows that send above ther far share. We assert that reducng flow rates accordng to equaton (3.3.) wll reduce the aggregated rate to the calculated value of R. To prove ths asserton, we need to show that: r = R (3.3.3) u o By applyng the defnton of R, S, and (3.3.) to (3.3.3) we get: under over under R R S under r = R + s + = R + over S s + over R R S S under over under over over r = R + S + S = over R R R S 4. EVALUATION OF THE LOAD DISTRIBUTION FEEDBACK CONTROL SCHEME 4. Smulaton set-up To evaluate the proposed load control scheme, we bult a smulaton model usng the OPNET [7] network smulator. We mplemented the message exchange mechansms descrbed n Secton, and compared the performance of the dfferent rate reducton schemes for the network topology shown n Fgure 7. under S over s User B C User Bottleneck B3 Server User 3 User 4 B C C3 Fgure 7. Smulaton topology.
We connected all the nodes n ths topology wth T lnks wth a capacty of.544 Mbps. Only the bottleneck lnk between nodes C3 and B3 was provsoned wth a dfferent amount of bandwdth, varyng ts capacty to smulate dfferent congeston stuatons. We smulated one-drectonal traffc from the user nodes (sources) to the server node (destnaton). All the sources generated FTP traffc wth an average nter-request tme of 0. second and data sze of 800 bytes. Includng all the headers, ths added up to a total sendng rate of 66,40 bts/second for each user. Also the boundary nodes B and B establshed SLAs wth ther correspondng users as shown n Table 6. We used the Tme Sldng Wndow Meterng (TSW) mechansm [3] for estmatng the arrval rate n the core and boundary nodes. The boundary routers estmated a value of the arrval rate for calculaton of reduced rates, whle the core router used the estmated rates for determnng f there was congeston. We confgured the TSW meter wth a wndow sze of 5 seconds. Durng the congeston, we passed user traffc through a smple token bucket that reduces the sendng rate of the user as needed. We ran our smulatons for 300 seconds and actvated users accordng to the followng schedule: User and user 3 start sendng traffc at tme nstant chosen randomly durng the frst 40 seconds of smulaton. User 4 starts sendng traffc at tme 00 seconds. User starts sendng traffc at tme 300 seconds. User Server User Server User 3 Server User 4 Server Mnmum Provsoned Rate 0 Kbps 0 Kbps 30 Kbps 40 Kbps Boundary node B B B B Table 6. Servce Level Agreement 4. Evaluaton of the -step rate reducton method for computng CRAR. Durng our expermental evaluaton, we observed that the -step rate reducton method was not able to properly estmate the CRAR. After close examnaton of the results, we notced that the value of the aggregated arrval rate at the congested nterface as reported to the ngress nodes was not very accurate. The -step rate reducton method s based on the assumpton that we can clearly observe the results of the frst rate reducton. Any naccuracy of the current arrval rate values causes the estmate of the CRAR to fal durng the second teraton. For ths reason, we evaluate the performance of the -step rate reducton method separately from the other methods. We further observed that the value of the arrval rate at the congested lnk reported durng the frst teraton was usually smaller than the value reported durng the second teraton. Closer examnaton of that phenomenon revealed that the Tme Sldng Wndow rate estmator was not reportng the current rate accurately enough and requred addtonal tme to converge to the actual value of the arrval rate. Therefore, the ngress node was observng that the rate reducton after the frst congeston notfcaton caused an ncrease nstead of decrease n the arrval rate at the core lnk. Because of that we slghtly modfed our mplementaton of the -step rate reducton mechansm. Instead of generatng congeston notfcaton mmedately after the congeston was observed, we delay ths notfcaton. Introducton of the delay sgnfcantly mproved the performance of the -step rate reducton mechansm. Ths method was now able to elmnate congeston n exactly two steps. A second problem wth ths method was that t reduced total sendng rates of the ngress nodes below ther mnmum provsoned rates. Agan, the reason for ths behavor was the naccuracy of the reported values by the TSW rate estmator. In one case, we notced that although all the ngress nodes reduced ther total load by 4 Kbps, the TSW rate estmator reported a rate reducton of only 8 Kbps. Such naccuracy forced the ngress nodes to reduce ther loads by an amount larger than was needed to elmnate the congeston. We then explctly added a condton that does not allow the reducton of the total load at the ngress node to be below the mnmum provsoned rate. Ths solved the problem partally but stll resulted n larger than necessary rate reducton, causng the congested lnk to become underutlzed. In our smulaton, the -step method wth these modfcatons was properly reducng the ngress node sendng rates only when the nterval between the congeston notfcatons was about 7 seconds.
4.3 Evaluaton of the nave rate reducton methods for computng CRAR. In our evaluaton, we used the number of congeston notfcatons needed to completely elmnate congeston n the core router as a measure of how well the method performs. For each method, we ran the experment 0 tmes, counted the number of notfcatons requred to completely elmnate congeston, and averaged these counts over the 0 runs. In ths set of smulatons, we provsoned the bottleneck lnk wth 05 Kbps of bandwdth, whch s only 5 Kbps above the arrval rate at the lnk n the case when all of the users send ther traffc at the mnmum provsoned rate. We also set the mnmum tme nterval between two consecutve congeston notfcatons to be second. Ths nterval was ntroduced n order to avod sendng too many congeston notfcatons. Fgure 8. Nave method : ngress node sendng rate varaton Fgure 9. Nave method 3: ngress node sendng rate varaton Due to the user actvaton schedule, our smulaton had three dstnct perods of congeston. The frst perod of congeston occurred at tme 30 seconds, when both users and 3 became actve. Snce all users send ther traffc at the rate of 66.4 Kbps, the core router C, had ( * 66.4 05) = 7.48 Kbps of excess traffc arrvng on ts nterface. The second perod of congeston stated at tme 05 seconds, when the user 4 started sendng traffc. At ths pont, core router C had about 66.4 Kbps of traffc arrvng above the capacty of the nterface. The thrd congeston perod started at tme 0 seconds when user began sendng traffc. The core router C agan was observng 66.4 Kbps of excess traffc arrvng on ts lnk. Table 7 dsplays the number of congeston notfcatons requred to elmnate congeston n the core for each perod of congeston. As expected, nave method performs very poorly. Ths method requres about two and a half tmes as many congeston notfcatons as nave methods and 3. The man reason for that s the fact that nave method always reduces sendng rate only by a fracton of the excess rate. As experments showed, ths reducton technque s unable to converge to the value of the sendng rate that would elmnate congeston completely. As a result, wth nave method the ngress nodes receve congeston notfcatons almost throughout the whole smulaton. Nave methods and 3, on the other hand, converge to the correct sendng rate farly quckly, and as expected they requre almost the same number of congeston notfcatons to converge.
Rate Reducton Congeston Perod Method -st -nd 3-rd Nave 6.6 4.3 0. Nave 4.0 40.8 5.3 Nave 3 6.6 5.7.4 Table 7. Comparson of the methods for rate reducton Also, as we expected, nave method faled to avod reducng aggregated rates below the mnmum provsoned rates. However, both nave methods and 3 were able to elmnate ths problem. Fgures 8 and 9 llustrate ths by showng how the sendng rates of the ngress nodes change durng the congeston. As Fgure 8 shows, nave method reduces traffc load at ngress node to 59 Kbps, whle ths node s provsoned for 70 Kbps on the path to the server. Nave method 3 on the other hand, does not reduce traffc loads below ther mnmum provsoned rates. As shown n Fgure 9, both, ngress node and ngress node send traffc at rates slghtly above ther provsoned rates. 4.4 Evaluaton of the per-flow rate dstrbuton. As we expected, usng the method proposed n Secton 3.3 allows the ngress nodes to farly dstrbute excess bandwdth among ther users. To llustrate how the ngress nodes dstrbute excess bandwdth among the users, we conducted a dfferent set of smulatons. In ths smulaton set, we allocated 40 Kbps of bandwdth at the bottleneck lnk, and we used nave method 3 for CRAR computaton. In ths scenaro, we had only two congeston perods because the bottleneck lnk had more bandwdth allocated to t. As the results n Tables 8 and 9 show, the excess bandwdth s always dstrbuted farly among the users of the ngress nodes n proporton to ther mnmum provsoned rates. It should be noted that the far share dstrbuton among flows s only across flows n a gven ngress node and s not vald for flows n dfferent ngress nodes. Fgures 0 and llustrate these results n a graphcal form. Fgure 0: Nave method 3: Per-flow rate dstrbuton. Fgure : Nave Method 3: Ingress node sendng rates varaton.
Mnmum Frst Congeston Perod Second Congeston Perod Provsoned Rates Sendng rate Excess BW Sendng Rate Excess BW Ingress Node 30 Kbps 47.3 Kbps 7.3 Kbps 68.5 Kbps 38.5 Kbps Ingress Node 70 Kbps 9.6 Kbps.6 Kbps 7.4 Kbps.4 Kbps Table 8. Rate dstrbuton among ngress nodes. Provsoned Frst Congeston Perod Second Congeston Perod Rate Sendng rate Far share Sendng rate Far share Ingress User 0 Kbps 47.3 Kbps 37.3 Kbps.8 Kbps.83 Kbps Node User 0 Kbps -- -- 45.7 Kbps 5.67 Kbps Ingress User 3 30 Kbps 39.7 Kbps 9.68 Kbps 30.6 Kbps 0.6 Kbps Node User 4 40 Kbps 5.9 Kbps.9 Kbps 40.8 Kbps 0.8 Kbps Table 9. Rate dstrbuton among the users. 5. CONCLUSIONS AND FUTURE WORK In ths paper we ntroduced an archtecture for load dstrbuton n the ngress nodes of a network doman durng perods of congeston. We also presented and evaluated four methods for dstrbuted computaton of the sendng rates at ngress nodes. We found that the complex method for precsely estmatng sendng rates does not work well because of the naccuracy n the reported arrval rate at the congested nterface. However, smple methods for reducng rate can elmnate congeston wthn a reasonable tme and would not reduce traffc rates at ngress nodes below ther correspondng mnmum provsoned rates. Furthermore, we proposed a mechansm for far rate dstrbuton among the ndvdual flows of the ngress node, and we showed usng the OPNET [7] network smulator that ths mechansm acheves far load dstrbuton among the flows. However, to better understand possble advantages of the proposed dynamc load control protocol, we need to nvestgate further the performance of the rate reducton mechansms under a varety of network condtons. 6. ACKNOWLEDGEMENTS We would lke to thank Dr. Constantnos Dovrols for the long hours of dscusson and for helpng to realze ths dea. 7. REFERENCES. S. Blake, D. Black, M. Carlson, E. Daves, Z. Wang, W. Wess. "An Archtecture for Dfferentated Servces", December 998. IETF RFC 475. H. Chow, A. Leon-Garca, "A Feedback Control Extenson to Dfferentated Servces", March 999. Internet Draft: draft-chow-dffserv-fbctrl.txt 3. D. Clark, W. Fang, "Explct Allocaton of Best Effort Packet Delvery Servce," IEEE/ACM Transactons on Networkng, vol. 6, no. 4, pp. 36-373, August 998 4. C. Dovrols, P. Ramanathan, "A Case for Relatve Dfferentated Servces and Proportonal Dfferentaton Model," IEEE Network, Vol. 3, No. 5, pp. 6-34, Sep./Oct. 999. 5. C. Dovrols, D. Stllads, "Relatve Dfferentated Servces n the Internet: Issues and Mechansms," In ACM SIGMETRICS, May 999. 6. C. Dovrols, D. Stllads, P. Ramanathan, "Proportonal Dfferentated Servces: Delay Dfferentaton and Packet Schedulng," Proc. ACM SIGCOMM 99 Conference, Cambrdge, MA, Sep. 999, pp.09-0. 7. Wu-chang Feng, Dlp D. Kandlur, Dabanjan Saha, and Kang G. Shn "Understandng and Improvng TCP Perfromance over Networks wth Mnmum Rate Guarantees," IEEE/ACM Transactons on Networkng, Vol. 7, No., pp. 73-87, Aprl 999.
8. OPNET Modeler. OPNET Technologes Inc. http://www.ml3.com 9. K. K. Ramakrshnan, Sally Floyd, D. Black, "The Addton of Explct Congeston Notfcaton (ECN) to IP", March 00. Internet Draft: draft-etf-tsvwg-ecn-03.txt 0. Rezende, J. F., "Assured Servce Evaluaton", In Proceedngs Globecom 99, March 999.. B. Nandy, N. Seddgh, P. Peda, J. Ethrdge, "Intellgent Traffc Condtoners for Assured Forwardng Based Dfferentated Servces Networks, In Proceedngs of IFIP Hgh Performance Networkng (HPN 000), June 000.. Peter Peda, Nabl Seddgh, Bswajt Nandy, "The Dynamcs of TCP and UDP Interacton n IP-QOS Dfferentated Servces Networks," In Proceedngs of the 3rd Canadan Conference on Broadband Research, November 999. 3. Ion Stoca, Scott Shenker, Hu Zhang, "Core-Stateless Far Queung: Achevng Approxmately Far Bandwdth allocatons n Hgh Speed Networks," Proc. ACM SIGCOMM 98 Conference, Vancouver, B.C., Sept. 998, pp.8-30. 4. A.F. Lobo and A.S. Seth, ''A cooperatve congeston management scheme for swtched hgh-speed networks,'' Proc. ICNP-96, Internatonal Conference on Network Protocols, Columbus, Oho (Oct.-Nov. 996), pp. 90-98.