Faster Network Design with Scenario Pre-filtering

Transcription

1 Faster Network Desgn wth Scenaro Pre-flterng Debojyot Dutta, Ashsh Goel, John Hedemann Informaton Scences Insttute School of Engneerng Unversty of Southern Calforna 4676 Admralty Way Marna Del Rey, CA 9292 (ISI-Techncal-Report-55) November 15, 21 Abstract Desgn and engneerng of networks requres the consderaton of many possble confguratons (dfferent network topologes, bandwdths, traffc and polces). Network engneers may use network smulaton to evaluate changes n network confguraton, but detaled, packet-level smulaton of many alternatves would be extremely tme consumng. Ths paper ntroduces the concept of scenaro preflterng rather than perform detaled smulaton of each scenaro, we propose to quckly evaluate (pre-flter) all scenaros n order to select only the relevant scenaros and dscard those that are clearly too over- or under-provsoned. To rapdly evaluate scenaros, we have developed several new analytcal technques to quckly determne the steadystate behavor of the network wth both bulk and short term TCP flows. These technques apply to arbtrary topologes and routers that use both drop-tal and RED queung polces. Snce we are only nterested n selectng the nterestng scenaros for detaled smulaton, the answers need only be approxmate. However, we show that accuracy s typcally wthn 1% of detaled smulaton. More mportantly, these technques are 1 3 faster than detaled smulaton, and, hence, pre-flterng s a promsng technque to reduce the total smulaton tme when many scenaros must be consdered. Ths work s part of the SAMAN project, supported by DARPA and the Space and Naval Warfare Systems Center San Dego (SPAWAR) under Contract No. N661--C-866. Any opnons, fndngs and conclusons or recommendatons expressed n ths materal are those of the author(s) and do not necessarly reflect the vews of DARPA or SPAWAR. The authors are also at the Department of Computer Scence, Unversty of Southern Calforna 1 Introducton The desgn and engneerng of networks s a challengng task. Interactons between traffc load, topologes, and protocols create a huge parameter space that must be understood. Network smulaton can play an mportant role n ths understandng and n the desgn of better networks. For example, consder a smple lnk havng a varable number of FTP connectons gong through t. A network engneer mght want to fnd the pont when the capacty of a certan lnk would be nsuffcent to meet certan goals. Or she mght want to obtan the maxmum number of bulk connectons that can be supported by a partcular network topology. To explore the network behavor, the engneer mght employ a network smulator to evaluate a number of scenaros wth dfferent traffc characterstcs. Packet level smulators, such as ns-2 [25], are dscrete event drven and ther granularty s a sngle packet. They are wdely used to understand network and network protocol behavor, partcularly by protocol desgners. They are less commonly used to understand the behavor of operatonal networks n scenaros such as the ones just descrbed for at least two reasons. Frst, t s not always easy to represent the current status of an operatonal network n a smulator because ts topology or traffc may not be well known or they may be dynamc. Second, although smple smulatons can be run qute quckly, smulatng scenaros wth many nodes and at hgh traffc rates can easly become qute tme consumng. Understandng the behavor of the network wll requre many scenaros to consder alternate traffc or confguraton choces. Yet often many of these scenaros are not nterestng, ether because the networks are sgnfcantly over-provsoned, and, hence, do not pro- 1

2 vde an understandng of the network bounds, or, they are under-provsoned. Only a few scenaros are crtcal.e. provde a good balance to defne the operatng lmts of the network. Another fact about smulatons wth protocols such as TCP [23] s that they often need to be run for several seconds n order to reach a steady state. Such restrctons further add to the smulaton tme for each scenaro. Ths paper addresses the second problem,.e. to evaluate a wde range of smulaton scenaros to fnd the relevant ones and dscard the undesred ones quckly. Our work s based on the observaton that there s no need to smulate the unnterestng scenaros n detal. We propose Approx-sm, a desgn tool that can very quckly evaluate the steady-state behavor of scenaros and pre-flter them by user-suppled crtera. It allows unnterestng scenaros to be dsmssed quckly and the nterestng ones to be evaluated n detal. Approx-sm dentfes the steady-state behavor of scenaros usng a hybrd queung theoretc approach for droptal routers, a new approach to RED modelng, and, an approxmate fxed-pont algorthm. It makes use of well known equatons for bulk TCP behavor [22] and a new approach to approxmate the short lved TCP flows. We show that approx-sm can evaluate scenaros an order of magntude faster than what s possble wth packet-level smulators. At the same tme, the accuracy s typcally wthn 1% wth the largest observed error beng 3%. In approx-sm, scenaros consst of a mx of long and short lved TCP traffc ( elephants and mce respectvely), and, have both drop-tal and RED queung at routers over arbtrary topologes. Packet level smulators are nherently determnstc. Ths s approprate for smulaton studes. However, ths can lead to synchronzaton of traffc. Our analytcal smulaton engne, approx-sm, does not run nto such problems. In fact, comparsons wth approx-sm helped us to dentfy scenaros where the ns-2 smulatons were gettng synchronzed. In later sectons, we demonstrate how we removed the synchrony n ns-2 by addng short lved flows and by usng RED gateways. 2 Related Work Our work s related to other approaches for fast smulaton, ether through parallelsm or approxmaton. It also bulds on analytcal approaches to understandng network performance. 2.1 Rapd smulaton Parallelsm has been used for many years to mprove smulaton performance [4, 16]. Several parallel network smulators are currently avalable, such as Parsec [1], SSFNET [6], and parallel versons of ns-2 [13, 24]. Our work s complementary to these efforts whle parallelsm can mprove the performance of detaled smulaton by up to the number CPUs devoted to the task (typcally 4 8 today), pre-flterng can mprove performance many-fold by never smulatng the unnterestng scenaros n detal. RPI has proposed the use of expermental factorng, that combnes multple sequental smulatons on a network of workstatons wth search algorthms to choose the scenaros that should be consdered [26]. Ths work s smlar to ours n goal (rapdly explorng the desgn space), and largely complementary n result: ther approach could gan further performance mprovement by usng pre-flterng technques. In contrast to our approach, they do a random search of the parameter space whle we have a determnstc algorthm. 2.2 Analytcal approaches Queung theoretc approaches have long been used to evaluate network performance (for example, [18]). Although t s necessary to understand fundamental performance lmts, these approaches must be appled to the Internet wth care because of the complexty of the protocols and the networks n use there. Our approach seeks to get the best from both queung theory and detaled packet-level smulaton by usng the former for a rapd approxmaton whle usng the later for detaled evaluaton. Recently, there has been extensve work n flud-flowbased approaches to network smulaton [19, 2, 21]. These approaches are promsng, and, some such as Msra et al. s [2] approach can capture the transent behavor. However, further work s needed to understand the performance of these approaches for large networks. Our work complements ther strategy; we look at a dfferent problem of fndng the steady state values of the network state. 2.3 Fxed pont Fxed pont approxmatons have been studed by Bu et al. [2]. They present formalsms to compute the fxed pont for a sngle congested RED router wth reasonable accuracy. They do not, however, have strong results for complex scenaros wth both RED and drop-tal routers, and, wth both long and short lved TCP flows (elephants and mce). We show that our approach allows complex mxed scenaros to be solved, approxmately. Also, the formalsms used n our approach are extremely smple. 2.4 Modelng Recent work has developed ncreasngly accurate analytcal models for the steady state of bulk TCP [22]. Our work 2

3 NS scrpt Parser get topology and traffc detals Approx. Solver (approx sm) Decder (user scrpt) Present steady state to the user NS2 engne Dont do a packet level smulaton Fgure 1: The structure of the pre-flterng tool bulds upon these results. Other work has modeled short TCP flows; we buld upon the work there by Cardwell et al. [3] and Huang et al. [15]. More recently, Ben Fredj et al. [12] descrbe short flows as nelastc traffc and demonstrate that smple queung models lke are reasonably accurate for modelng drop-tal routers. We buld upon ther results and extend them to RED routers. Although RED has been studed n detal [1, 7, 9], and Hollot et al. presented a control theoretc model of RED [14], we beleve that our work s the frst to evaluate RED usng a Markovan queung model. 3 Our Approach We desgned a pre-flterng framework where we ntegrated a a fast approxmate network smulator, a decder (to determne whether to dscard a scenaro) and a detaled packetlevel smulator usng the structure shown n Fgure 1. The user feeds a regular ns-2 scrpt nto our verson of ns-2 that has the embedded approx-solver, approx-sm. Approx-sm would do a fast approxmate smulaton of the network scenaro and would present to the user the drop probabltes of the routers, the delays and the approxmate aggregate throughput of the lnks. The user can then decde to ether smulate the scenaro n detal by usng the packet level engne n ns-2 or dscard t. The data structures of approx-sm are populated by a module wthn the Tcl space of ns-2 [25]. The pre-flterng tool thus reduces to a smple Tcl scrpt that runs the approxsm module wthn the ns-2 framework. The output of the approx-sm would be detaled steady state statstcs of the network that can be accessed by the Tcl scrpts. Ths nformaton can be used to wrte smple pre-flterng tools n Tcl tself. Currently, the approx-sm tool s not tghtly ntegrated nto the ns-2 framework. It runs n a stand-alone mode. There were certan desgn choces that went nto the desgn of the pre-flterng framework. Our ntal approach was to buld a pre-flterng tool along wth approx-sm for the user. We wanted t to have features such as to determne whether a scenaro has congeston above a threshold. That needed a careful desgnng of the query language that the user would use and features that we could possbly support. Ths approach could have lmted the capabltes of the tool tself. As a better desgn choce, we decded to buld just the approx-sm and a good user nterface to the data structures and the results of the smulaton. The user would query the data structure and would take decsons based on the response. Ths approach has a couple of advantages. Frst of all, the user has complete flexblty to desgn the query that would sut hs desgn. Also, the user would not have to learn yet another query-language. From the desgners standpont, t s redundant to redesgn a query language that would mtate the functonalty of a standard scrptng language lke Perl or Tcl, and, at the same tme be dfferent. Another mportant desgn choce was to buld the whole smulator n a plug-and-play fashon so that t s easly scalable and t s easy to ncorporate newer models of network elements and traffc agents. Also, ths ensures that ntegratng approx-sm nto ns-2 s easy. We hope that such a modular desgn wll make the system more sutable for rapd valdaton of new protocols. 4 Solvng for the Steady-state behavor The approxmate-solver s the heart of our pre-flterng tool. It evaluates characterstcs of long-lved and shortlved TCP connectons, ncludng the throughput of the flows and the delays, drop probabltes and the aggregate throughputs at each router. We next descrbe how these are accomplshed, quckly and approxmately. Fgure 2 shows the flowchart we use to solve for the steady-state behavor of the network. We begn wth a topology and the detals of traffc agents n the topology from the user n the form of a ns-2 scrpt. Our module would parse the ns-2 scrpt and populate the nternal data structures of approx-sm. The user can nvoke the approxsm module by a smple Tcl command from hs ns-2 scrpt. In step 1, the engne, n ts smulate procedure, frst calculates the drop probablty and the queung delays at each router from the prevous teraton or the ntal condtons (See Fgure 2). In step 2, t uses these router statstcs to calculate the end-to-end drop probablty and delays encountered by each flow whch are then used to obtan the per-flow throughput. Step 3 calculates the total throughput for each lnk by addng the per-flow throughputs of each flow that pass through t. Then the algorthm checks 3

4 1 Intal Condtons 4. Scale flows, check constrants scaled and constrant satsfed network state calculate agrgegate throughout at each router NO Calculate state of each 1. Drop Tal / RED router 2. Calculate state of each of the long / short TCP flow 3. Convergence? YES drop probabltes queung delays long flows: throughput short flows: aggregate throughput Fgure 2: The structure of the Approx-sm smulator whether convergence of a new network state s acheved. If the network state converges, we termnate. Otherwse we use a scalng algorthm n step 4 to scale the flows to meet the network constrants, and, we use the results to run another teraton of the procedure, smulate. In step 1, we use smple queung theory for drop tal routers and a new, very smple analytcal model for RED [1] gateways. We descrbe these models below n Secton 4.1. At the end of ths step, we know the drop probabltes,, and the average queung delay, at each router. The state of router s defned by. The calculatons n step 2 are dependent on whether the flows are short lved or not. For bulk flows, we calculate the throughput of each flow (Secton 4.2.1). For short lved flows, we calculate ther aggregate throughput (Secton 4.2.2). An mportant queston s how to guarantee that ths process converges and leads to the steady state. We dscuss ths ssue n Secton Modelng network elements Approx-sm consders models of two dfferent packet queung dscplnes: drop-tal and RED [1]. For drop-tal routers, we use the M/M/1/K model [17]. Let us consder a server processng packets from the TCP connecton. Packets arrve at the server at a partcular rate ( ). The server consumes packets at the rate of, whch s gven by the bandwdth of the lnk. Packet losses are gven by the of the system. Hence we can plot a curve!#"%$& of throughput versus drop probablty. Call ths. Note that we could have used the more accurate ' or (' but that would have added to the complexty to our formalsm. We have found the smple ( models to be farly accurate as long as the network s not very heavly loaded. RED gateways represent a common AQM strategy used n today s routers. We have developed a smple model to descrbe the steady state behavor of a RED router. We assume that the traffc that flows nto ths gateway has also reached ts steady state. In Bu et al. s work [2] the authors used the RED model drectly nto the TCP equatons of Padhye et al. [22]. Our approach s dfferent. We determne the drop probablty and the average queue length as a functon of the steady-state utlzaton at the router. Wthout loss of generalty, f the servce rate of a RED router s unty, the throughput at the router,, wll be the same as the utlzaton of the router queung system ),.e. +*,). Let the queue length at the router be - and the drop probablty at the router be. The RED characterstcs can be expressed by./ f EGFIHIJLKNMPO8Q#RTSU VWYX[Z\ 687:9<;>=@?BADC M^]L_LRDS`KaMPO8Q#RDS f ;>=@?BADCcb<6d7:be;gf(hNADC otherwse As long as the queue length s below jk l, the drop probablty s zero. If queue length s between jn l and jporq, drop probablty ncreases lnearly between and Bs:tLu. After ths lmt s crossed, drop probablty becomes unty. Lemma 4.1. The steady state average value of the queuelength wth utlzaton ) and drop probablty s gven by vxw * )zy :{n :{}) y :{nz Lemma 4.2. In the steady state, the average queue length wll never be larger than jporq Theorem 4.3. If the drop probablty seen at ths router s and the steady state queue length s between the nterval jn ~l and jporq, the drop probablty at ths router s gven by * y ƒ D Y xˆ {}jk l > s:tlu L ƒ D L xˆ jpo q {}jn ~l (1) (2) (3) The above theorem ensures that we can obtan a quadratc equaton n. Hence, gven a value of (or ) ), we can fnd a drop probablty. Then, usng the value of, we can use Lemma 4.1 to calculate the queue length and the queung delay. Next, we defne, for every RED router, two parameters )Ns dš and )as:tlu. )as dš s the soluton to the equaton 2 and corresponds to the throughput that causes the buffer length of the RED router to be jn ~l and the drop probablty to be. Smlarly, we can defne ) s tœu to be the throughput that causes the buffer length of the RED router to be jno q and the drop probablty to 4

5 $ $ * $ $ $ $ be s tœu. By Theorem 4.3 and the above defntons of ) s tœu and ) s TŠ, we can easly calculate the queue length and the drop probablty n the followng way: If the lnk throughput s less than )Ns TŠ, the drop probablty s and the queue length s gven by the ( model. Smlarly, when the throughput s greater than ) s:tlu, the queue length s exactly jporq and the drop probablty can be calculated from Theorem 4.3. Note that we cannot use Equaton 1 to obtan the drop probablty because we are nterested n the average drop probablty and not the nstantaneous one. Instead, we are try to fnd the average state. If ) s TŠ ) ) s:tlu, the drop probablty can be computed usng Theorem 4.3. We should note that we can adapt the above analyss to obtan an teratve method n order to calculate the state of RED routers that use more complex varatons lke Gentle RED [8]. 4.2 Modelng the flows In our work, we assume that all flows are TCP flows. Ths s reasonable because TCP traffc s known to account for the bulk of the traffc n the Internet. In ths secton, we refer to the analytcal models that have been used n our approx-sm Modelng elephants Padhye et al. [22] gave the throughput of bulk TCP flows ( yi ), or elephants), as a functon of the probablty of a loss event ( ) and the round trp tme as yi p* where s a constant and the tmeout Modelng mce yig (4) s the maxmum value of Short lved TCP flows, or mce, have been extensvely studed n [3] and [15]. These efforts have focused on fndng very detaled models to accurately depct the behavor of ndvdual short term flows. In contrast, we concentrated on a much smpler model for aggregates of short term flows. We have refned Ben Fredj et al. [12] model of mce to ncorporate the drop probabltes. They valdated ther work on smple topologes whle we valdated ther model (and our refnement) on more general topologes. We approxmate aggregates of short flow between the same source-destnaton par as a smooth flud. The ratonale for ths knd of dea s that aggregates beng nelastc traffc wll have less correlaton to the complex feedback mechansm and wll be easer to model at a hgher level. It s also much faster to determne the behavor of the aggregate. Lemma 4.4. If the rate of arrval of short lved flows between any source-destnaton par s t t w and the data transferred by the flow s, then the rate of short flow traffc between the same source-destnaton par s gven by s * t t w (5) For now, we assume that short TCP connectons come accordng to some arrval pattern that s Markovan and each connecton transfers a constant amount of data. Lemma 4.5. Let the end-to-end drop probablty between any source-destnaton par be and the arrval rate for short flows be s. The throughput of the mce s gven by s s :{n where =drop probablty (6) Proof. The total short lved traffc that arrves at a router s gven by Lemma 4.4. Now, f drop probablty s, the traffc generated by short flow request n one second s gven by s Hence the lemma. * s y (7) Note that long lved bulk TCP traffc s sad to be elastc snce the closed loop congeston control algorthms can adjust the sendng wndow and utlze the avalable bandwdth. In contrast, short lved TCP flows can be consdered to be nelastc. The dynamcs of the network wll be heavly nfluenced by these mce. Ths s also emphaszed n [12]. The ntutve dea s to assume that the long lved flows n presence of these short flows do not contrbute much to ncreasng the load on the network. Hence we can calculate the short flows throughput frst and use the remanng bandwdth for the long lved flows. 4.3 Does a fxed pont exst? It s not clear whether a fxed pont exsts between the router characterstcs and the flow characterstcs n all network scenaros. Bu et al. [2] prove that t does for a sngle congested lnk. They do not have conclusve results for complex networks. We use an teratve scheme and we have found the approxmate fxed pont for all our experments because we relaxed the condtons of convergence to nclude errors. Next, we present an alternate proof of the exstence of the fxed pont wth drop-tal routers. Bu et al. [2] prove the same facts for a AQM router lke RED. Let us assume we have a sngle congested lnk. We use only long-lved TCP flows (elephants) n our subsequent 5

6 m m m proofs and arguments. Consder the throughput characterstc $ of each TCP flow, y8. We call ths curve w $ t Š ". It s natural to queston the exstence of a fxed pont as the throughput y8 s a surface. We now show that a fxed pont ndeed exsts. Lemma 4.6. When the probablty of loss ( ) or the ncreases, the throughput of bulk TCP flows on a sngle lnk y8 decreases. Proof. Follows from Equaton 4. Lemma 4.7. When the aggregate throughput on a lnk ncreases, both and ncrease. Proof. Increasng the throughput mples a hgher rate of arrval nto the queue. That results n hgher queung delays and ncreases the and the drop probablty. :Y!#"%$& Theorem 4.8. The ntersecton of the curve of the " has a unque fxed pont. router wth $ w $ t Š Y!#"%$& Proof. Assume the contrary. Snce our curve assumes a fxed $ RTT, t s a straght lne. It must ntersect the curve w $ t Š " at least once. Assume that t ntersects at two ponts and. Lemmas 4.6, 4.7 ndcate the monotoncty of TCP performance. Assume. By Lemma 4.7, drop probabltes ncrease as we ncrease from to. But, by Lemma 4.6, as drop probabltes ncrease throughput should ncrease. Thus there s a contradcton. Hence the theorem. Note that we cannot say that the above holds for a network wth many congested lnks. Now we dscuss the exact procedure to calculate the fxed pont of each TCP flow n the network. Let there be a network wth edges/lnks and TCP connectons. Let each connecton be and let each lnk be denoted as -. Assume that connecton passes through edges and y% & denotes the edge set of ths connecton. Let the delays of each lnk be, the drop probablty be. Also f we assume that the connecton goes through the lnk, t wll occur a drop of and a delay a of ' and ths lnk s denoted by. Now s equal to, ' s equal to for some lnk -. Intutvely, the procedure s as follows: we use the approprate model at each router to calculate the lnk delays and drops experenced at each router by all the flows gong through that lnk. These lnk characterstcs are used to estmate the end-to-end round trp tme and the drop probabltes seen by each flow at the sender. We can then use ether Equaton 4 or Lemma 4.5 to calculate the throughput of the TCP flows. Hence we have * o l ' * (8) For example, s the, and, hence, t can be denoted by - can wrte ' as ' *Y v where v Y lnk n the path of connecton for some. Now we for some (9) denotes the queung delay of the lnk - and s the propagaton delay of the lnk -. The RTT seen from the end-pont,.e. the sender, for a connecton s denoted by and s gven by * 'g (1) where s the number of lnks traversed by connecton. If we make the assumpton that packet losses are ndependent on each lnk, the followng theorem s obvous Lemma 4.9. The drop probablty seen by the connecton, ' s gven by ' p* { y :{ G (11) Hence, the throughput of a TCP flow, can now be calculated by * y' (12) 4.4 Intal condtons and convergence It s trval to desgn an algorthm to calculate the approxmate steady state throughput from the dscusson n the precedng sub-secton. Thus the algorthm s shown n Fgure 3. The above algorthm requres us to start wth the correct ntal values of B for each lnk. But, we do not want to make any assumptons apror on the state of the network. Wthout such a restrcton we can always solve the network n the followng fashon: Run ns-2 for a few seconds n vrtual tme. The throughput of each lnk wll gve us the ntal s for approx-sm. A more elegant soluton s not to use any pror knowledge of the ntermedate ns-2 results. Ths s our approach Our approach to convergence Intally we assume that the lnks are not loaded when there are only bulk flows or elephants present. We argue that the load due to the elastc flows s such that they wll share all the avalable bandwdth. In ths frst teraton of our fxed pont algorthm, the bulk TCP flows get what Equaton 4 wth low drop probabltes. That may result n wndow lmted or large throughput. Now we run the algorthm and 6

7 y set ntal condtons whle(convergence not reached) do f (not ntal) then scale connectons for = 1 to nlnks do calculate the queueng delays and drop probabltes endfor for j = to nconnectons do sum the drops and delays from the lnk lst of edge for connecton calculate throughput from the TCP equatons endfor for = 1 to nlnks do calculate the total throughput of each lnk endfor endwhle Fgure 3: Aggregaton algorthm after every round of fxed pont compute the new throughput of each connecton and sum them up to fnd the new s of each of the lnks. Ths gves us the ntal throughput. The throughput of a bulk connecton (elephant) s very senstve to small changes n probablty, whch makes t hard to acheve convergence usng the teratve process descrbed earler. Specfcally, f the drop probablty s very low, then the computed throughput of the bulk connectons on a lnk can be much hgher than the capacty of the lnk. To speed up convergence, we scale down the computed throughput of bulk connectons so that lnk capactes are not exceeded. A bref descrpton of the scalng algorthm s gven below. The scalng algorthm: BŠ represent the computed throughput of the l bulk flows after each teraton of the fxed pont algorthm. Intally, we mark each bulk flow as beng unscaled. For each lnk - defne w, the unscaled capacty, as the capacty of the lnk mnus the throughput of all the short flows (mce) on ths lnk. Also, for each lnk -, defne w to be the combned throughput of all the unscaled bulk flows on the lnk. Defne the congeston w as w w. Now, we repeat the followng process. whle there exsts a lnk - wth w : Let - denote the lnk wth the largest value of w. Scale down the throughput of all the unscaled bulk flows usng ths lnk by a factor w, and mark all these flows as beng scaled. Now the total throughput of ths lnk exactly matches the capacty of the lnk and hence w = 1. For each newly scaled flow, and each lnk * - such : that flow uses lnk, we reduce the unscaled capacty of lnk by the new throughput of Let flow and the combned throughput of lnk by the old unscaled throughput of flow.. When the above algorthm termnates, the throughput on any lnk does not exceed ts capacty. In practce, we found the scalng step to be crtcal for fast convergence. We call ths step Lnk cappng. Ths step ensures that a partcular lnk s put back nto a stable state before the averagng process n the convergence algorthm dscussed n the prevous subsecton. The performance of the scalng algorthm s gven by the followng theorem: Theorem 4.1. The worst case runnng tme,, of the scalng algorthm on a network of sze connectons, lnks s gven * y e (13) where s the average number of lnks traversed by each connecton Proof. Fndng the most congested lnk takes y tme wth sutable data structures. When we scale each connecton, we need to change the unscaled capacty of wth sutable data lnks. Ths takes tme structures. where s the number of lnks the connecton traverses. Hence the theorem. 5 Evaluaton and Results We next evaluate how well approx-sm meets ts three goals: speed, accuracy, and generalty. Frst, we consder ts performance relatve to packet-level smulaton. Second, we show that t s reasonably accurate, typcally wthn 1 15% of packet-level smulaton for the scenaros we consder. Only some scenaros were 2% accurate but they were under very heavy load. A very hgh level of accuracy s not requred for approx-sm because we expect fnal smulaton results to be done wth packet-level smulaton; approx-sm merely selects those scenaros. Fnally, 7

8 4 A B 3 NS2 Approx-sm Fgure 4: The lne topology Smulaton tme 2 1 TCP flows TCP flows Bottleneck Sze of the tree (heght) TCP flows TCP flows TCP Snks All lnks are of 1 Mbps have prop. delay = 5ms Fgure 6: Runnng tme comparsons between approx-sm and ns-2 for the symmetrc tree topology wth elephant traffc only Fgure 5: The symmetrc tree topology: a sample bnary tree of heght wth four clents at each leaf. we evaluate the generalty of approx-sm by showng that t s applcable to ncreasngly complex scenaros n terms of traffc mx, topology and network elements. In ths entre secton, we use a partcular termnology. Long lved flows and elephants are used nterchangeably. Smlarly we refer to short lved TCP flows as mce. For throughput, unts of packets/s and kb/s are used nterchangeably snce all our smulatons use a packet sze of 1kB. We start wth smple topologes topologes (lnes and symmetrc trees) and move to more complex topologes (asymmetrc trees and crcular topologes) to valdate approx-sm progressvely. 5.1 Elephant traffc alone Frst we consder results that we obtaned for the experments wth elephant-only traffc. We evaluated approx-sm on the lne topology (Fgure 4) as well as symmetrc (Fgure 5) and asymmetrc trees (Fgure 8). Ths gradual ncrease n the complexty of the topologes wll help us to evaluate approx-sm wth just bulk flows. The lne topology shows good accuracy between ns-2 and approx-sm so we jump drectly to symmetrc trees. Symmetrc trees were ntally chosen because t allows us to study the effect of many smlar flows passng through a bottleneck lnk. Fgure 5 shows the symmetrc tree topology. We place the TCP snks at the bottleneck lnk.e. at the root of the tree, and four bulk TCP sources at each of the leaves of the tree. All the lnks are assumed to have a capacty of 1Mb/s. Fgure 6 shows the run-tme performance of approx-sm compared to packet-level smulaton wth ns-2 for symmetrc trees as a functon of tree heght. Approx-sm s 1-3 faster than packet-level smulaton. Although the performance of both approx-sm and ns-2 s lnear wth network sze (and ncreases exponentally as a functon of tree heght), the very large dfference n constant factor makes approx-sm one to two orders of magntude faster than packet-level smulaton. Speed s not useful f the smulaton s completely naccurate. Fgure 7 compares approx-sm and ns-2 accuracy by evaluatng mean flow bandwdth for the bottleneck lnk. (No error bars are shown n ths case because approx-sm s determnstc and the standard devaton between the ns-2 flows s less than 5%.) Ths graph shows that approx-sm s qute accurate compared to ns-2. The smulators are typcally wth 1 2%; the worst case s wth a hght of 8 when the network s very heavly loaded where they are 4% apart. approx-sm s more accurate when we look at aggregates of many flows. The accuracy s much hgher for the lnks close to the root. At the root bottleneck lnk, the accuracy was 7.6%. We have also conducted experments for hgh lnk capactes and the results have been better wth less utlzaton. Next we consder asymmetrc trees (as shown n Fgure 8) to avod bases n evaluaton due to symmetry. We examne asymmetrc trees of varyng heghts. Fgure 8 shows a tree wth heght two. In general, we construct an asymmetrc tree of heght by expandng the leftmost node of a tree of heghtn{ to have two chldren. All traffc termnates at the lower-left-most node of the tree; traffc begns at all the other leaves of the tree wth elephants. Early comparsons of results for asymmetrc trees show large dfferences between approx-sm and ns-2. In ns-2 all long RTT flows (eg. between nodes,, Fgure 8) had very low throughput whle short RTT flows (eg. between 8

9 Bandwdth of the 1st hop from the source (packets/s) NS2 Approx-sm Throughput of longest connectons (packets/s) NS2 Approx-sm Sze of the tree (heght) Heght of the asymmetrc tree Fgure 7: Accuracy of approx-sm throughput compared to ns-2 for the symmetrc tree topology wth elephant traffc only Fgure 9: Comparson between approx-sm and ns-2: throughput of the longest TCP connectons n several asymmetrc trees - all lnks have a capacty of 1MB/s Throughput of longest connectons (packets/s) NS2 Approx-sm Fgure 8: The asymmetrc tree topology, n Fgure 8) had hgh throughput. Although t s well known that TCP s unfar to flows wth dfferent RTT and Equaton 4 predcts a throughput rato of between the short RTT and long RTT flows respectvely (assumng no queung delay), we observed a rato of more than. We beleve that ths dsparty occurs due to synchronzaton n ns-2. Because packet-level smulators are purposefully determnstc, packets from dfferent senders can arrve at queues at exactly the same vrtual tme, and the flows can reman synchronzed because there s no varaton n the smulated envronment. In real-world experments, nevtable tmng varatons prevent consstent, fne-graned synchronzaton. 1 Ths problem wth packetlevel smulaton has been recognzed, both at small scale where the ns-2 TCP model ncludes optonal jtter, and at larger scales where researchers add a small amount of addtonal background traffc to the smulaton to desynchronze flows [12]. Snce approx-sm predcts the steady-state behavor, t 1 Although at coarse scales, some protocol synchronzaton has been observed [11] Heght of the asymmetrc tree Fgure 1: Comparson between approx-sm and ns-2: throughput of the longest TCP connectons n several asymmetrc trees - all lnks have a capacty of 45MB/s s mmune to such artfcal synchronzaton. To avod synchronzaton n our ns-2 smulatons, we ntroduced a small amount of background traffc. For the asymmetrc tree, we flled 1% of the bottleneck lnk bandwdth wth randomly generated web-lke traffc. For our elephant-only experments approx-sm does not have ths traffc, therefore we expect t to slghtly overestmate performance. An nterestng fact s that flows n approx-sm can never get synchronzed unlke n ns-2. Hence, engnes lke our approx-sm could be useful to to get an alternate opnon of a large class of scenaros. Fgure 9 compares approx-sm to ns-2 wth ths background traffc as the heght of the tree vares. We see that the results of approx-sm are accurate wthn 2% of the ns- 2 results. Ths s good accuracy gven that the network s very heavly loaded and approx-sm s approxmatons are least accurate under heavy load. We expect approx-sm 9

10 to be more accurate when the network s less loaded. We therefore also consdered the same scenaro wth 45Mb/sbandwdth lnks (See Fgure 1). Ths graph shows that n less loaded networks approx-sm s even closer to ns- 2, wthn 2-1%. Agan, approx-sm underestmates bandwdths compared to ns-2 because t does not consder background traffc. 5.2 Mxed mce and elephants In ths secton, we evaluate approx-sm wth a mx of traffc sources.e. wth both mce and elephant traffc. Ths s crucal because Internet traffc conssts of both short and long lved flows. Lke the prevous subsecton, we evaluate approx-sm by gradually ncrease the complexty n topology Lne topology We start our experments wth the smple lne topology because t s easy to hand-verfy our results. Consder a lne as n Fgure 4. We vary topology wth two nodes and the lnk bandwdth, the mean arrval rate (exponental arrvals) and length of short flows ( per second and kb/s), and the number of long flows. Frst, we observe that both approx-sm and ns-2 get very smlar values for aggregate throughput of mce (wthn 1%). Both smulators predct smlar values for elephants as well (wthn 8.3%). From now, we wll focus more on the accuracy of the elephant-flows snce n the scenaros we consder, the load of the mce s small compared to the elephants. Fnally, these values also match hand calculatons as well Symmetrc Trees We now move on to valdate approx-sm on a more complex topology. We choose symmetrc trees as they ensure aggregaton n the network. Further, these topologes are very smple for hand-verfcaton too. Consder a lne as n Fgure 5. We vary topology wth two nodes and the lnk bandwdth, the mean arrval rate (exponental arrvals) and length of short flows ( per second and kb/s), and the number of long flows. The results for ths experment are shown n Fgure 12. We observe that as we ncrease the the lnk bandwdth from 1MB/s to 2MB/s, the accuracy of approx-sm drops from 5% to 27%. The man reason for ths drop s that traffc at the bottleneck lnk ncreases due to aggregaton and approx-sm bounds the maxmum throughput to be y < KB, s the lnk capacty and, the drop probablty on that lnk. But the end-to-end drop probablty may be greater than. But, when we decrease the amount Per Flow Throughput (packets/s) Offered mce-load (fracton of capacty) NS smulaton Approx-sm Fgure 13: Comparsons between approx-sm and ns-2: bandwdth acheved by the flows havng longer RTT.e. around 3ms of mce (or the nelastc traffc), there s a hgher correlaton between the values obtaned from ns-2 and approxsm. Hence approx-sm s more accurate wth lght load Asymmetrc Trees Now we compare results for asymmetrc trees to avod symmetry. Fgure 13 shows the bandwdth of bulk TCP flows between nodes and (the longest path). We observe that approx-sm s predctons are close to what ns-2 outputs wth an accuracy that vared from 13 to 16%. Fgure 14 shows bulk TCP flows between nodes and, the short RTT path. Agan, we see that approx-sm results are smlar to those n Fg 13. The accuracy of approx-sm vared from 13 to 17% for Fg. 14 and from 6-2% n Fg. 15. Comparng Fgures 13, 14, 15, we observe that for long RTTs approx-sm estmates larger throughput than ns-2 whle for shorter RTTs ts estmate s lower. For aggregate throughput of short flows, the ns-2 results are typcally 7 1% hgher than the those predcted by approx-sm. We beleve that ths dfference s related to synchronzaton n ns-2 (as descrbed n Secton 5.1). Mce provde some level of desynchronzaton, but some dfference between approx-sm and ns-2 remans. We plan to nvestgate ths hypothess further. To consder cases wth lower load, we also examned scenaros wth lnk bandwdths of 1 and 1Mb/s. We do not report detaled results here due to space constrants, but we observed hgher accuraces at lower utlzatons as n the all-elephant case (Secton 5.1). Snce approx-sm and ns-2 results are qute smlar, these experments suggest that approx-sm s model s approprate: one can model short flows as nelastc and bulk flows as fllng out the rest of the traffc, at least for the traffc loads we consder. 1

11 C # of # of Mce Elephants Mce (MB/s) elephants (/s), (kb/s) approx-sm ns-2, ( ) approx-sm ns-2 (kb/s) (kb/s), (kb/s) (kb/s) (kb/s) 1 4 2, (9 ) , (9 ) , (9 ) , ( ) 4 4 Fgure 11: Comparson of results wth drop-tal routers on a Lne topology as shown n Fgure 4 Heght C # of elephants # of Mce Elephants of tree (MB/s) per leaf (conn/s), kb/s approx-sm (kb/s) ns-2 (kb/s), (kb/s) , ( ) , (9 ) , , (9 ) Fgure 12: Comparson of results wth drop-tal routers on a symmetrc tree topology as shown n Fgure NS smulaton Approx-sm NS smulaton Approx-sm Per Flow Throughput (packets/s) Per Flow Throughput (packets/s) Offered mce-load (fracton of capacty) Offered mce-load (fractonof capacty) Fgure 14: Comparsons between approx-sm and ns-2: bandwdth acheved by the flows havng short RTT.e. around 2ms Fgure 15: Comparsons between approx-sm and ns-2: Aggregate short flow throughput Crcular topologes We next consdered the rng topology shown n Fgure 16. Between each alternate node (eg. between A, C), we vary the lnk bandwdth, the mean arrval rate (exponental arrvals) and length of short flows ( per second and kb/s), and the number of long flows. Snce there s overlappng traffc, we beleve that ths scenaro provdes a more dffcult case for convergence n approx-sm. Fgure 17 presents the throughput of short and long flows between nodes A and C. Agan, we observe a good match between approx-sm and ns-2, wth the bulk flows wthn 6.1%. More mportantly, even wth ths crcular topology approx-sm converges wthn 5 teratons. 5.3 Experments wth RED Fnally we evaluate routers wth RED queung polces. We have examned some scenaros of each of the topologes (lne, symmetrc and asymmetrc tree, and the crcle) B A C E D Fgure 16: The rng topology wth RED, but here we summarze only the lne and crcle topologes. In each topology we consder RED routers wth the parameters y@jn l jno q s:tlu = y # wth the thresholds n 1KB packets. We make no clams 11

12 C # of # of Mce Elephants (MB/s) elephants (conn/s), kb/s approx-sm (kb/s) ns-2 (kb/s), (kb/s) , (9 ) , (9 ) , ( 4 ) , ( 4 ) , ( ) , ( ) Fgure 17: Comparson of results wth drop-tal routers on a crcular topology as shown n Fgure 16 about these parameters beng deal (n fact, there s some evdence that t s qute dffcult to tune RED [5]), they are merely the defaults n our smulator. If we look at Fgure 18, we see that wth lght load, approx-sm s agan very accurate whle accuracy decreases wth load. Ths further justfes our clam of approx-sm beng sutable for approxmate pre-flterng. Although our prelmnary evaluaton of approx-sm wth RED routers s promsng, a more thorough examnaton s needed and n progress. 6 Concluson Our method solves for the approxmate operatng pont of many TCP flows usng analytcal technques. To acheve ths, we use a combnaton of exstng and new models for network elements and TCP flows coupled wth an approxmate fxed pont teraton algorthm. Our work led us to some nce observatons about modelng and smulaton of networks. S. Ben Fredj et al. [12] clam that for a sngle congested lnk wth a drop-tal router, short flows (mce) add to the load and that the elephants adjust accordng to the avalable bandwdth. Our results show that these results are true even n complcated networks and also n the presence of RED gateways. A very mportant observaton s that scenaros wth TCP flows n packet level smulators (such as ns-2) can easly be dragged nto synchronzatons. Such phenomenon s very msleadng and can gve us unrealstc results. One must take adequate care and nterpret these smulaton results. Tools such as approx-sm can used to dentfy scenaros that are prone to such phenomenon. If the results of approx-sm are farly close to the ns-2 results, we can be confdent that such synchronzatons were not seen n the ns-2 smulatons. Also, one can remove such synchronzaton by addng some background random traffc. Another approach would be to add short term TCP flows between each source destnaton par. Snce approx-sm s fast and scalable, we feel that t may be used for a wde varety of applcatons other than flterng of scenaros. One possble applcaton s to help n convergng on a correct SLA between 2 network provders. There s a lot of work that needs to done. One drect extenson would be get a more accurate model of the network elements and flows wthout compromsng on the smplcty. Then we need to ensure that our approx-sm results are accurate for networks wth thousands of nodes. Also, the ntegraton of our approx-sm engne wth ns-2 s under development (currently, we have a smple module n ns- 2 that outputs a descrpton that approx-sm can read and populate ts data structures). References [1] R. Bagroda, R. Meyer, M. Taka, Y. Chen, X. Zeng, J. Martn, and H. Y. Song. PARSEC: A parallel smulaton envronment for complex systems. IEEE Computer, 31(1):77 85, October [2] T. Bu and D. Towsley. Fxed pont approxmaton for TCP behavor n an AQM network. In Proceedngs of the ACM SIGMETRICS, June 21. [3] N. Cardwell, S. Savage, and T. Anderson. Modelng TCP latency. In Proceedngs of the IEEE Infocom, page to appear, Tel-Avv, Israel, March 2. IEEE. [4] K. M. Chandy and J. Msra. Asynchronous dstrbuted smulaton va a sequence of parallel computatons. Communcatons of the ACM, 24(11):198 25, Aprl [5] M. Chrstansen, K. Jeffay, D. Ott, and F.D. Smth. Tunng RED for web traffc. In Proceedngs of the ACM SIGCOMM, pages , Stockholm, Sweden, September 2. ACM. [6] J. H. Cowe, D. M. Ncol, and A. T. Ogelsk. Modelng the global nternet. Computng n Scence & Engneerng, pages 3 36, January [7] S. Floyd. RED: Dscussons of settng parameters, [8] S. Floyd. Recommendaton on usng the gentle varant of RED, [9] S. Floyd. RED (random early detecton)

13 Topology, # of # of Mce Elephants C (MB/s) elephants (conn/s), kb/s approx-sm (kb/s) ns-2 (kb/s), (kb/s) Lne, 1 4 2, (9 ) Lne, , (9 ) Lne, , (9 ) Crcle, , (9 ) Crcle, , ( ) Fgure 18: Comparson of results wth RED routers [1] S. Floyd and V. Jacobson. Random early detecton gateways for congeston avodance. ACM/IEEE Transactons on Networkng, 1(4): , August [11] S. Floyd and V. Jacobson. The synchronzaton of perodc routng messages. ACM/IEEE Transactons on Networkng, 2(2): , Aprl [12] S. Ben Fredj, T. Bonald, A. Proutere, G. Rgn, and J. W. Roberts. Statstcal bandwdth sharng: a study of congeston at flow level. In Proceedngs of the ACM SIGCOMM, pages ACM, 21. [13] M. H. Ammar G. F. Rley, R. M. Fujmoto. A generc framework for parallelzaton of network smulatons. In Proceedngs of the Seventh Internatonal Symposum on Modellng, Analyss, and Smulaton of Computer and Telecommuncaton Systems, (MASCOTS) College Park, MD. October 1999, October [14] C. Hollot, V. Msra, D. Towsley, and W. Gong. A control theoretc analyss of RED. In Proceedngs of the 21 IEEE Infocom, Aprl 21. [15] P. Huang and J. Hedemann. Capturng TCP burstness n lght-weght smulatons. In Proceedngs of the SCS Conference on Communcaton Networks and Dstrbuted Systems Modelng and Smulaton, pages 9 96, Phoenx, Arzona, USA, January 21. USC/Informaton Scences Insttute, Socety for Computer Smulaton. [2] V. Msra, W. Gong, and D. F. Towsley. Flud-based analyss of a network of AQM routers supportng TCP flows wth an applcaton to RED. In SIG- COMM, pages , 2. [21] D. Ncol, M. Goldsby, and M. Johnson. Flud-based smulaton of communcaton networks usng SSF. In Proceedngs of the European Smulaton Symposum, Erlangen-Nuremberg, Germany, October [22] J. Padhye, V. Frou, D. Towsley, and J. Kurose. Modelng TCP throughput: A smple model and ts emprcal valdaton. In Proceedngs of the ACM SIG- COMM, Vancouver, Canada, September ACM. [23] J. Postel. Transmsson control protocol. Internet Request for Comments RFC 793, September [24] G. Rley, M. Ammar, R. Fujmoto, D. Xu, and K. Perumalla. Dstrbuted network smulatons usng the dynamc smulaton backplane, 21. [25] UCB/LBNL/VINT. The NS2 network smulator, avalable at [26] T. Ye and S. Kalyanaraman. An adaptve random search alogrthm for optmzng network protocol parameters. Techncal report, Rensselaer Polytechnc Insttute Computer Scence Department, June 21. [16] D. R. Jefferson. Vrtual tme. ACM Transactons on Programmng Languages and Systems, 7(3):44 425, July [17] L. Klenrock. Queueng Theory Volume 1. John Wley & Sons, Inc., [18] L. Klenrock. Queueng Theory Volume 2. John Wley & Sons, Inc., [19] V. Msra., W. Gong, and D. F. Towsley. Stochastc dfferental modellngand analyss of TCP wndow sze behavor. In ECE-TR-CCS , October