Modeling the Bandwidth Sharing Behavior of Congestion Controlled Flows

Transcription

1 Modelng the Bandwdth Sharng Behavor of Congeston Controlled Flows Kang L A dssertaton presented to the faculty of the OGI School of Scence & Engneerng at Oregon Health & Scence Unversty n partal fulfllment of the requrements for the degree Doctor of Phlosophy n Computer Scence and Engneerng ovember 00

2 The dssertaton Modelng the Bandwdth Sharng Behavor of Congeston Controlled Flows by Kang L has been examned and approved by the followng Examnaton Commttee: Dr. Jonathan Walpole Professor Thess Research Advsor Dr. Molly H. Shor Assstant Professor Thess Research Advsor Dr. Calton Pu Professor Georga Insttute of Technology Dr. Wuchang Feng Assstant Professor Oregon Graduate Insttute

3 Acknowledgement I have been fortunate enough to have the help and support of a large number of people. I would especally lke to thank my advsors Jonathan Walpole and Molly H. Shor, who have been extremely helpful n dentfyng my thess topc, developng the deas, publshng paper, construct the structure of ths thess, and for all other gudance throughout many years. I would also lke to thank x x x x x The members of my thess commttee members, Professor Calton Pu and Wuchang Feng, for ther nsghtful comments on my proposals, and the dssertaton. I would lke to thank Calton for hs helps on many strategy ponts. Wthout hs help, I would not fnsh ths thess. Charles Buck Krasc, Ashvn Goel that have been workng wth me through a long way form DSRG, DISC, to SYSL, and all the members of the system software group, Jonathan Walpole, Wuchang Feng, Wuch Feng, Davd Steere, Dylan Mcamee, Andrew Black, Perry Wagle, Je Huang, Francs Chang, Mke Shrea, Jn Cho, Jm Snow and many others. Mark Morrssey and Jm Bnkley from Portland State Unversty for ther many useful and nsghtful dscussons about networkng and other random topcs. Raner Koster, Tong Zhou, Yong Xu, Erk Walthnsen, Peter Geb, Anne- Francose Lemeur, Dan Revel, Songtao Xa, We Tang, We Han, Henrque Paques and others that brought a lot of fun to my student lfe n OGI. Shanwe Cen, who recommended me to ths graduate school. Fnally, I would lke to thank atonal Scence Foundaton, and the Defense Advance Research Project Agency, for ther fnancal support.

4 Table of Contents Acknowledgements Lst of Fgures v Abstract x 1 Introducton The Research Problem Our Approach Contrbutons Dssertaton Overvew TCP Overvew TCP Congeston Control Acknowledgements Congeston Wndow Congeston Sgnals Slow Start and AIMD TCP Flavors TCP-frendlness TCP-style Farness TCP-Frendlness State of the Art AIMD-based Algorthm A State-space Model for TCP State-Space Modelng v

5 3.1.1 A State-Space Model System Stablty A State-Space Model for A Bandwdth Sharng System Target System System State Dfferental Equatons and State-Jumps Smulaton System Dynamcs Dscussons Lnear versus onlnear System Modelng Competng Traffc Assumptons about Congeston Sgnal State-space Modelng Results and Analyss Farness TCP-style Farness Farness of Flows wth dfferent AIMD Parameters Farness under on-unversal Congeston Sgnals Dscussons Bufferng requrements for CBR nteractve applcatons Bufferng and Adaptatons Bufferng Requrement for AIMD Congeston Control Real World Experments Adaptve Control Motvaton Lnux Implementaton Experment Setup Experments and Results Farness Tme-scales Share among TCP-frendly AIMD Flows Share among Unfrendly AIMD Flows v

6 5.3.4 Other Farness Ratos Summary Related Work Theoretcal Studes Mathematcal Modelng of Bandwdth Sharng Farness Models for TCP Congeston Control Feedback Control Analyss of Computer Systems etwork Smulatons System Works: TCP-frendly Congeston Control Protocols Concluson and Future Work Summary of Contrbutons Future Work Stochastc Stablty Dynamc Congeston Control Adaptatons Bblography Appendx A Appendx A Bographcal ote v

7 Lst of Fgures Fgure.1: TCP Slow Start and AIMD rate control Fgure 3.1: A Bandwdth Sharng System Fgure 3.: AIMD Rate Control Subsystem Fgure 3.3: The Bandwdth Sharng System wth Key Informaton only Fgure 3.4: A Snapshot of a Smulnk Smulaton Block Fgure 3.5: Trajectory of a System wth a Sngle Flow Fgure 3.6: Trajectory of a System wth Two Flows Fgure 3.7: Trajectory of a System wth Four Flows Fgure 4.1: Farness between two TCPs wth the Same RTT Fgure 4.: Farness between two TCPs wth equal BD but dfferent FD Fgure 4.3: Farness between two TCPs wth equal FD but dfferent BD Fgure 4.4: Farness between unfrendly Flows Fgure 4.5: Farness between TCP-frendly Flows Fgure 4.6: State-Space Trajectory wthout a Clear Lmt Cycle Fgure 4.7: Average Throughput of two dfferent AIMD flows Fgure 4.8: State-Space Trajectory wthout a Clear Lmt Cycle Fgure 4.9: Average Throughput versus AIMD Parameters Fgure 4.10: Average Congeston Sgnal Rate versus AIMD Parameters Fgure 4.11: Orgnal TCP-style Farness Fgure 4.1: Unform Farness Fgure 4.13: A Modfed State-space wth Only Vald Ponts Fgure 4.14: A QoS-Adaptve Applcaton over the Internet v

8 Fgure 4.15: Bufferng Requrement of an AIMD-based Congeston Control Fgure 4.16: Bufferng Requrement for Two Contnuous Back-offs Fgure 5.1 Slow Start and Addtve Increase Fgure 5. Multplcatve Decrease Fgure 5.3: Socket Opton API of the Adaptve AIMD Fgure 5.4: Congeston wndow sze of an Adaptve AIMD flow and a TCP flow Fgure 5.5: Congeston wndow of more Adaptve AIMD flows Fgure 5.6: Experment etwork Topology Fgure 5.7: The System Trajectory Montored n a Real etwork Fgure 5.8: Farness for two dentcal competng flows n dfferent tmescales Fgure 5.9: Farness for 0 dentcal competng flows Fgure 5.10: Farness for 00 dentcal competng flows Fgure 5.11: Farness Index versus RTT Fgure 5.1: Bandwdth Share Rato of TCP-frendly Flows durng a Lght Congeston 99 Fgure 5.13: Lnk Utlzaton durng a Lght Congeston Fgure 5.14: Congeston Sgnals Experenced by TCP-frendly Flows durng a Lght. 100 Fgure 5.15: Bandwdth Share Rato of TCP-frendly Flows wth Severe Congeston. 104 Fgure 5.16: Lnk Utlzatons of TCP-frendly Flows durng a Severe Congeston Fgure 5.17: Congeston Sgnals of TCP-frendly Flows durng a Severe Congeston. 104 Fgure 5.18 Tmeouts of TCP-frendly Flows durng a Severe Congeston Fgure 5.19: Tmeouts versus congeston levels Fgure 5.0: Share Rato of 14 TCP-Frendly Flows durng a Lght Congeston Fgure 5.1: Share Rato Impact of D Parameter, Lght Congeston Fgure 5.: Share Rato Impact of small D Parameter, Lght Congeston Fgure 5.3: Utlzaton Impact of D Parameter, Lght Congeston Fgure 5.4: Congeston Sgnal Impact of D Parameter, Lght Congeston Fgure 5.5: Share Rato Impact of D Parameter, Severe Congeston Fgure 5.6: Utlzaton Impact of D Parameter, Severe Congeston Fgure 5.7: Congeston Sgnal Impact of D Parameter, Severe Congeston Fgure 5.8: Tmeout Impact of D Parameter, Severe Congeston v

9 Fgure 5.9: Share Rato Impact of the E Parameter, Lght Congeston Fgure 5.30: Utlzaton Impact of the E Parameter, Lght Congeston Fgure 5.31: Congeston Sgnal Impact of the E Parameter, Lght Congeston Fgure 5.3: Share Rato Impact of the E Parameter, Severe Congeston Fgure 5.33: Utlzaton Impact of the E Parameter, Severe Congeston Fgure 5.34: Congeston Sgnal Impact of the E Parameter, Severe Congeston Fgure 5.35: Tmeout Impact of the E Parameter, Severe Congeston Fgure 5.36: etwork Topology Fgure 5.37: Adjustng AIMD parameters for RTT compensatons x

10 Abstract Modelng the Bandwdth Sharng Behavor of Congeston Controlled Flows Kang L OGI School of Scence & Engneerng at Oregon Health & Scence Unversty August 00 Thess Advsors: Dr. Jonathan Walpole and Dr. Molly H. Shor Multmeda applcatons have become ncreasngly popular n the Internet. TCP s the domnant Internet congeston control protocol, but t does not serve all applcatons well. Thus, many new congeston control protocols have been proposed recently, n partcular for multmeda applcatons. To ensure that these flows share bandwdth farly wth TCP flows, TCP-frendlness s proposed as a crteron for desgnng new protocols. Currently, the TCP-frendlness crteron s defned based on the assumpton that all flows experence the same statc congeston sgnal. However, the bandwdth sharng and congeston sgnal s a result of the dynamc behavor of all partcpatng flows. The clam of ths thess s that the bandwdth sharng behavor among competng flows should be studed n a dynamcal envronment. To understand a dynamc phenomenon one needs a theoretcal model that adequately descrbes the behavor of the system beng studed. In ths dssertaton, we propose a state-space model to study the dynamcs of the bandwdth competton, n partcular among AIMD-based TCP-frendly flows. It characterzes a dynamc system by a set of related state varables, whch can change wth tme n a manner that s predctable x

11 provded that the external nfluences actng on the system are known. We use the model to descrbe the stablty of bandwdth compettons, whch s characterzed as convergence to a dynamcally oscllatng lmt cycle n the state space. Ths stablty descrpton clearly dstngushes transent and long-term farness. Along wth the state-space modelng, we buld an adaptve AIMD-based congeston control protocol that exposes ts parameters to applcatons. Ths dssertaton presents some example uses of ths adaptve protocol to verfy the results derved from the statespace model. As an example, we adjust the AIMD parameters to acheve a unform farness that s ndependent of round-trp-tmes. x

12 Chapter 1 Introducton At the heart of the success of the Internet s ts congeston control behavor. The Internet serves flows n a best-effort way, whch does not use bandwdth reservaton mechansms. Careful desgn thus s requred to prevent applcatons from congestng the network wth a heavy load. Untl now, t has been the congeston control protocol at each end-host that has prevented applcatons from overloadng the network. A congeston control protocol lmts a flow s rate to ts porton of the avalable bandwdth and prevents t from sendng too fast and causng network overload. A congeston control protocol does not know the avalable bandwdth a pror. Thus t has to probe the avalable bandwdth by ncreasng ts rate untl t detects sgns of congeston, and then decreasng ts rate. A congeston control protocol detects congeston by ndcatons of packet loss (tmeouts or duplcated acknowledgements) and explct marks (wth support from actve queue management mechansms [BCC+98]). We call these ndcatons the congeston sgnals. Transmsson Control Protocol (TCP) [JK89, APS99] s the de-facto standard congeston control protocol n the Internet. It uses an addtve-ncrease and multplcatve-decrease (AIMD) algorthm, whch probes the avalable bandwdth by ncreasng ts transmsson rate addtvely, and responds to a congeston sgnal by decreasng ts rate by half. 1

13 TCP, although powerful and effectve, s not suffcent to satsfy applcatons that requre dfferent rate behavors than TCP. For nstance, TCP s behavor of cuttng ts rate by half upon a congeston sgnal could cause too much rate varaton for a streamng meda applcaton that prefers a smooth rate [BBFS01, FF99]. Fortunately, TCP s not the only possble congeston control protocol. It uses one specfc algorthm wth specfc parameters for the congeston control, whereas many alternatve algorthms wth dfferent rate behavors are avalable. Recently, many congeston control protocols [BMP94, CPW98, RHE99a, TCPF] have been proposed, partcularly for streamng meda applcatons n the Internet. A congeston control protocol, dependng on ts manner of probng the avalable bandwdth and respondng to congeston, produces varous rate behavors. In general, applcatons care about the followng aspects of a congeston control protocol s rate behavor: x x x x Rate Smoothness: how large the magntude of rate varatons s and how often the rate vares. Responsveness: how fast the congeston control protocol responds to changes n network condtons. Farness: what the bandwdth share rato s when t competes wth other flows. Average throughput: whether there are underutlzatons of resources caused by the congeston control scheme. Typcally, applcatons prefer a congeston control that has a smooth rate and a fast responsveness. They also prefer a congeston control that shares bandwdth farly wth other flows whle stll mantanng a hgh average throughput. However, not all of these preferred behavors can be acheved at the same tme. For nstance, a tradeoff exsts between responsveness and smoothness. When a protocol s sendng rate s smooth and less senstve to the varatons n network condtons, t wll be less responsve to changes n the network. Dfferent applcatons have dfferent preferences on ths tradeoff based on

14 3 ther requrements, thus each of them could prefer a dfferent congeston control protocol that meets ts preferred rate behavor. Although many congeston control protocols have been bult to produce desred rate behavors, an mportant aspect, the stablty of systems composed by new protocols and TCP, has been largely gnored by most of the protocol desgners. Understandng the system stablty s mportant because many, f not all, of the above rate behavors are defned based on the system stablty. For example, responsveness s actually the tme for a system to reach a stable state upon a certan nput. Wthout understandng a system s stablty, there s no bass for dscussng responsveness. Smlarly, farness s mostly only nterestng when competng flows acheve a stable state. If a system has no stable state, farness s just a transent aspect and thus s not very meanngful as a rate behavor. Interestngly, the stable state of a system doesn t necessarly need to be a statc state, t could be an oscllaton through a cycle of system states. In the latter case, the farness behavor should also nclude the devatons and oscllatng frequences, etc. Besdes understandng the system stablty, protocol desgners have the problem of predctng the rate behavor of the new congeston control protocols. Most new protocols rate behavors, for example the average throughput, are desgned based on a statc assumpton that the congeston control protocol wll perceve the same congeston sgnals no matter how t behaves. We use statcally derved protocols to denote those desgned based on the assumpton that congeston sgnals are ndependent of the rate behavors of the congeston control protocols. Recent experments [BBFS01, FHP00, LSW01] have shown that some of the statcally derved protocols that are supposed to share bandwdth equally wth TCP flows do not share bandwdth equally wth TCP flows, or wth each other, as predcted. Although Bansal et. al. [BBFS01] show that ther statcally derved protocols tend to use less bandwdth than TCP (whch means they are more frendly to TCP flows) and can thus be safely deployed n the Internet, they do not erase the queston of how to predct accurately the rate behavor of a congeston control protocol n a dynamc envronment.

15 4 These statcally derved protocols do not produce the expected rate behavors because the rate behavors of congeston control protocols are dynamcally determned. By dynamcally determned, we mean that ther rate behavors and congeston sgnals are nter-dependent. The rate behavors of a congeston control protocol are drectly related to the congeston sgnals perceved by t, whereas ts rate behavors drectly contrbute to the producton of the congeston sgnal. Because of ths nter-dependency, adjustng a congeston control protocol, such as changng ts parameters or usng alternatve algorthms, mples that the resultng rate behavor, and congeston sgnal behavor, must be studed n a dynamc envronment. We refer to the rate behavor of a congeston control protocol as ts dynamc behavor, because t must be characterzed n a dynamc envronment. 1.1 The Research Problem The topc of ths thess s to understand the dynamc rate behavor of congeston control protocols. It s mportant to understand system stablty because a lot of rate behavors are related to t. It s also mportant to study the dynamc behavors of newly developed congeston control protocols, whch are desgned for applcatons that are not served well by certan dynamc rate behavor. Any efforts to predct accurately the dynamc rate behavors must take nto consderaton the nterdependency between a congeston control protocol s nput (congeston sgnal) and ts output (the rate behavors). 1. Our Approach Instead of usng an approach that separates the rate behavors and the congeston sgnals, we study them wthn one bandwdth sharng system, whch conssts of a bottleneck lnk used by all competng flows. In a bandwdth sharng system, congeston sgnals are generated by the rate behavors of all competng flows.

16 5 Our goal s to understand the stablty and dynamc rate behavors of a system. Ths naturally rases the need for a theoretcal model that adequately descrbes the behavor of the system beng studed. Generally, modelng work can choose ether a determnstc or a stochastc approach. A determnstc approach tends to produce an accurate descrpton of the system behavor but requres determnstc nformaton of all the nputs. Whereas a stochastc approach requres only statstcally based values, but produces only a hgh level descrpton of the system. We choose a determnstc approach because ths thess work focuses on the rate behavors of ndvdual flows n varous tme scales, rather than the aggregate behavor of many flows over a large tme scale. The latter ssue s also nterestng but s outsde of the scope of ths thess. We model the bandwdth sharng system usng the state-space modelng technque of modern control theory [B91, C86]. The goal of ths modelng approach s to produce a descrpton of the dynamc rate behavors. In a state-space model, a dynamc system s characterzed by a set of related state varables, whch change wth tme n a manner that s predctable provded that the external nfluences actng on the system are known. In partcular, the dynamc behavors of a system are descrbed by a group of dfferental equatons and state jumps n our model. Usng these mathematcal descrptons of the model, we can study theoretcally the system stablty. Along wth the mathematcal descrpton of system states, our state-space model of bandwdth sharng ncludes a state plot analyss that help one vsualze the transent and stable behavors. In addton, we desgn a general adaptve congeston control protocol that s based on the general AIMD algorthm [LSW01, YL00]. We buld smulatons and real-world mplementatons so that we can verfy the outcomes from the state-space model, whch descrbes the rate behavors of ndvdual flows.

17 6 1.3 Contrbutons Ths study of dynamc behavors makes the followng contrbutons: x State Space Model We desgn a state-space model for the bandwdth sharng system. It s the frst model that defnes the dynamc stablty of the bandwdth competton, whch s a lmt cycle n the system state-space. We use ths model to show that statcally derved TCP-frendly protocols actually do not share bandwdth equally n dynamcal envronments. Wth ths state-space model, we also derve many mplcatons for real systems, such as the approprate tme-scale for measurng TCP-frendly farness, the quantzaton effect of packet szes, and the mnmal bufferng delay requrements for delay-senstve applcatons. x Controllable Farness Usng the mplcatons from the state-space model, we develop the mechansm to acheve varous sharng ratos by tunng the parameters of AIMD algorthms. In partcular, a unform farness, ndependent of the RTTs, can be acheved rather than the TCP-style farness. x Adaptve Congeston Control We buld an adaptve congeston control protocol that exposes the control parameters to applcatons rather than usng a fxed set of parameters. Applcatons can adjust the behavor of the congeston control, such as the tradeoff between responsveness and smoothness, based on applcaton requrements. In addton, the current defnton of TCP-frendly behavor can be satsfed by constranng the relatonshp between parameters.

18 7 1.4 Dssertaton Overvew The dssertaton s organzed as follows: Chapter revews TCP and ts AIMD algorthm. Chapter 3 descrbes the state-space modelng technque. It contans the defntons of system state and system dynamcs n general, and t ntroduces the dfference between statc and dynamc equlbrums. It then presents the result of applyng the state-space model to a system of competng AIMD-based flows. Ths model shows that the system has a dynamc equlbrum and descrbes ts stablty by a lmt cycle n the state space. Through the model s state plots, we also nvestgate the mpacts of varous parameters on a sngle flow, and demonstrate dfferent dynamcs of bandwdth sharng behavors among varous competng flows. Among them, we show how the changes of a sngle flow s behavor over short tme scales can change the behavor of the whole system over long tme scales. In addton, we show a way to aggregate states of many AIMD flows to one flow, whch can help us overcome the scalablty lmtaton of keepng per-flow state n a state-space model. Chapter 4 presents a lst of mplcatons from the state-space model for AIMD-based flows. It ndcates that the tme-scale of the farness measurement must be larger than the perod of the stable lmt cycle. It also ndcates that an AIMD flow s rate oscllatons at ts stable state cause a bufferng delay that s quadratc to the flow s round-trp-tme and s proportonal to ts average rate. In addton, the state-space model shows the relatonshp between a flow s AIMD parameters and ts average bandwdth share. We use the relatonshp to produce a dfferent farness paradgm other than TCP-style farness among competng flows. Chapter 5 descrbes our work on buldng an adaptve AIMD congeston control protocol that exposes the AIMD control parameters to applcatons. By exposng the parameters of the AIMD algorthm, applcatons can talor the congeston control protocol based on

19 8 ther requrements. Wth ths adaptve AIMD congeston control, we conduct experments n a controlled real-world setup to verfy the mplcatons from the state-space model. These real-world experments verfy the followng aspects of the bandwdth sharng behavors: (1) the tmescale of farness among AIMD flows, () the unfarness between TCP-frendly AIMD congeston control protocols, and (3) the possblty of adjustng AIMD parameters to acheve dfferent share ratos other than TCP-style farness. Chapter 6 revews some related work and addresses the dfferences between prevous work and the work presented n ths dssertaton. Chapter 7 concludes the dssertaton and outlnes some of our future plan.

20 Chapter TCP Overvew The man concern of ths thess s the rate behavor of congeston control protocols. TCP s the domnant congeston control protocol that s used n almost every computer n the Internet, and thus t s mportant frst to understand TCP and ts congeston control algorthm. TCP as a transport layer protocol has many functons, and congeston control s only one of them. The major functons of TCP are: relablty control, flow control, and congeston control. x TCP Relablty Control provdes an n-order relable data transmsson. In each TCP connecton, packets are marked wth ncreasng sequence numbers. The recever sends acknowledgements to the sender for the packets that have arrved correctly n-order. When packet losses happen, the sender retransmts the lost packets to the recever. x TCP Flow Control prevents a fast sender from overflowng the recever buffer of a slow recever. In TCP, the recever reports ts open buffer space to the sender n each acknowledgment. The sender s only allowed to send as much data as ndcated by the acknowledgements. When the recever s buffer s full, the sender wll stop sendng any data untl the recever acknowledges t wth new open space ndcatons. 9

21 10 x TCP Congeston Control prevents a fast sender from overflowng the buffer n the bottleneck router on the path between the sender and the recever. In ths thess, we smply use TCP to refer to ts congeston control. We revew the TCP congeston control protocol and ts well-known AIMD algorthm n the followng sectons..1 TCP Congeston Control Congeston control s mperatve n order to allow the network to recover from congeston and operate n a state of low delay and hgh utlzaton. Ideally, to acheve hgh utlzaton, end systems need to send as fast as they can. However, f ther sendng rates exceed the network capacty, data accumulates n buffers n the network, whch can cause long delay. Furthermore, routers have lmted buffer space to tolerate temporary overloadng. When the network becomes overloaded, the buffer n the bottleneck router starts to fll up, and eventually overflows. The network overloadng stage s generally called congeston, whch can cause packet losses and long delays to applcatons. Most lost packets are detected by end systems and retransmtted. But f end systems dd not slow down ther transmsson rates durng congeston, most of the bandwdth would be used to transmt packets that would be dropped before reachng the recevers. Ths behavor s called pourng gasolne on fre n a computer network [JK88]. The deal behavor, whch s also the goal of congeston control, s to keep end systems sendng as fast as the network capacty for hgh bandwdth utlzaton wthout creatng too much congeston. TCP congeston control attempts to acheve ths goal as follows. It starts wth a low rate and probes for the exstence of addtonal unused lnk bandwdth on ts path by progressvely ncreasng ts rate. It contnues to ncreases ts rate untl a congeston sgnal occurs. When TCP detects congeston, t reduces ts rate to a safe level and begns probng agan. In the followng subsectons, we revew the way TCP detects congeston

22 11 and lmts ts rate, and we dscuss the algorthm TCP uses for probng and respondng to congestons..1.1 Acknowledgments TCP uses acknowledgments to carry feedback nformaton for all three control functons mentoned above. Each tme a recever gets a packet 1, t nforms the sender of the sequence number of the next n-sequence packet. The packet used to nform the sender s called an acknowledgement. Acknowledgments can be pggybacked on data packets when the recever has data packets to send back to the sender. When there are no packet reorderng events or losses, the acknowledgment contans the sequence number of the packet followng the one that just arrved. If there s a packet loss, the acknowledgments of later packets contan the sequence number of the lost packet, whch s the sequence number of the next n-sequence packet n the data stream..1. Congeston wndow TCP lmts ts sendng rate by controllng ts congeston wndow sze, whch s the number of packets that may be transmtted-but-yet-to-be-acknowledged n a flow. ormally, the tme between delverng a packet and recevng ts acknowledgement s one round-trp-tme (RTT). A TCP sender can send up to the congeston wndow sze of data packets durng one RTT. Once TCP sends out a wndow sze of data packets, t can send new data packets only after some acknowledgements arrve. Thus the average rate of a TCP over one RTT s roughly the wndow sze dvded by the RTT. 1 When TCP s delay acknowledgement s enabled, the TCP recever could send one acknowledgement only after recevng multple packets.

23 1.1.3 Congeston sgnals In most wre-connected modern networks, packet losses due to lnk-level nose have become very rare because of technology mprovements. Loss typcally results from the overflowng of router buffers as the network becomes congested. TCP uses packet losses as an ndcaton of congeston. TCP detects packet losses wth two mechansms. The frst one s the tmeout. A TCP sender starts a tmer when t sends a packet to a recever. If the tmer expres before the sender receves the packet s correspondng acknowledgement, TCP thnks the packet s lost. Clearly, the tmeout nterval should be larger than TCP s RTT. Actually, TCP adapts ths nterval dynamcally based on ts RTT estmatons. In most of the TCP mplementatons, the tmeout nterval s set to the average of RTT estmatons plus 4 tmes the devaton. A detaled study of the effect of varous tmeout settngs s presented n [AP99]. The second way that TCP detects packet losses s through duplcate acknowledgments. A TCP recever only acknowledges the sequence number of the next n-sequence packet. A packet loss causes the recever to re-acknowledge the sequence number of the lost packet when the next packet arrves. The sender thus receves duplcate acknowledgements for the same sequence number. Snce packet reorderng n the network can also cause duplcated acknowledgements, TCP uses a threshold to avod treatng reorderng as packet losses. Typcally, TCP sets the threshold to three. Only when a TCP receves three or more duplcated acknowledgements does t consder that a packet s lost and thus generates a retransmsson. Ths mechansm to detect packet losses s also referred as the Trple Duplcated ACK Hack. Duplcated acknowledgements may detect packet losses earler than the tmeout tmer. Thus detectng congeston by duplcated acknowledgements s called Fast Retransmsson. otce that packet losses are not always caused by congeston. ose, especally n wreless networks, can cause a sgnfcant amount of packet loss. Takng these random

24 13 packet losses by nose as ndcatons of congeston can sgnfcantly mpact the performance of TCP congeston control. Many recent research studes have addressed ths ssue, but, snce our focus s not on congeston detecton, we assume TCP s congeston detecton s adequate. Recently, explct congeston notfcatons (EC) [RFB01] may be used to detect congestons. EC capable routers can mark packets wth a congeston notfcaton when they experence congeston. In ths way, TCP can be nformed of congeston earler and more accurately than usng tmeouts or duplcate acknowledgements. TCP can thus adjust ts rate accordng to the congeston sgnal wthout gettng dropped packets. In ths thess, both ndcatons of packet losses and explct congeston marks are called congeston sgnals..1.4 Slow Start & AIMD Besdes congeston sgnals and how TCP uses a congeston wndow to lmt ts rate, the remanng aspect of TCP congeston control s ts dynamcal wndow adjustment algorthms. The basc rate control mechansms are an exponental ntalzaton stage called Slow Start and an addtve-ncrease-multplcatve-decrease (AIMD) steady-state stage. An example of a TCP flow s two stages s shown n Fgure.1. The Slow Start s used when TCP s n the ntal stage or after a tmeout. Durng a Slow Start stage, TCP starts wth an ntal wndow sze (typcally one or two packets) and ncreases ts wndow sze by one packet upon the recept of each acknowledgment. Ths behavor leads to an exponental ncrease n sendng rate. Slow Start s called slow, compared to jumpng to a fast rate mmedately, but t actually accelerates very quckly (exponentally).

25 14 The AIMD algorthm s used n TCP s steady-state stage. Durng an AIMD stage, TCP ncreases ts current wndow by one packet for each full wndow of data acknowledged. Ths s the Addtve Increase (AI) part of AIMD. Once a Fast Retransmsson (duplcate acknowledgements) happens, TCP cuts ts wndow by half and then restarts the addtve ncrease. The halvng of wndow sze s the Multplcatve Decrease (MD) part of AIMD. The procedure of cuttng the wndow by half and then mmedately gong back to addtve ncrease s also called Fast Recovery, whch s fast compared to the alternatve of cuttng the wndow to the ntal value. Rate Slow Start AIMD 1 MSS/RTT 1 RTT The Avalable Bandwdth Tme Fgure.1: TCP Slow Start and AIMD rate control (RTT s the TCP flow s round-trp-tme, MSS s the TCP flow s packet sze) The transtons between Slow Start and AIMD are controlled by a threshold and tmeout events. TCP starts wth a Slow Start, and once the congeston wndow goes across a threshold, t swtches to the AIMD stage. Durng an AIMD stage, f a tmeout event happens, TCP sets ts wndow back to the ntal value and enters Slow Start agan. Both the threshold and the tmeout nterval are adjusted dynamcally. Detals can be found n [APS99]..1.5 TCP Flavors TCP has evolved n the last decade, and thus many dfferent TCP flavors are deployed n the Internet today. Here we lst the key features of a few TCP flavors:

26 15 x Tahoe: TCP Tahoe detects congeston only by tmeouts and has only Slow Start and AI stages. It starts wth a Slow Start. Once the wndow sze passes a threshold, TCP Tahoe swtches to the addtve ncrease stage. Once a tmeout happens, t sets the wndow sze back to the ntal value and does Slow Start agan. x Reno: TCP Reno adds both Fast Retransmsson and Fast Recovery to TCP Tahoe, and thus ncludes both Slow Start and AIMD. TCP Reno does the same thng as TCP Tahoe upon tmeouts. In addton to the tmeouts used n TCP Tahoe, TCP Reno also detects congestons by duplcated acknowledgements (Fast Retransmsson). Upon the trple-duplcated-acknowledgements, TCP Reno cuts the congeston wndow by half and contnues the addtve ncrease stage (fast recovery) rather than resettng to ntal wndows sze of one and dong slowstart. x ew Reno: TCP ew Reno treats multple packets losses n one RTT as one congeston sgnal nstead of several as n TCP Reno, and thus does at most one multplcatve decrease per RTT. x SACK (Selectve Acknowledgments): TCP SACK s recever nforms the sender wth sequence numbers of multple mssng packets, rather than only acknowledgng the sequence number of the frst mssng one. Thus TCP SACK can retransmt the lost packets, earler than TCP Reno f more than one packet s lost. x FACK (Forward Acknowledgments): TCP FACK s recever nforms the sender of the hghest sequence number that has arrved (even out of order). Wth ths nformaton, the sender can estmate accurately how many packets have left the network for ths partcular flow, and thus could make a better control on ts wndow sze.

27 16 x Vegas: TCP Vegas s generally not vewed as a member of the classc TCP famly. However, t s as well-known as any of the above flavors, and we lst t here. The dfference between Vegas and other TCP flavors s that TCP Vegas does not rely on AIMD as the major congeston avodance algorthm. Instead, TCP Vegas montors the RTT of each packet and adjusts ts rate to control the router queue length, whch s approxmately derved from the RTT measurements.. TCP-frendlness Congeston control protocols not only prevent flows from overloadng the Internet, but also determne the farness, whch s the bandwdth sharng rato among them. Because the bandwdth resource s shared among all partcpatng users, a stable bandwdth sharng farness s desred n the heterogeneous Internet, and the case of one user ganng most of the bandwdth and starvng others should not happen...1 TCP-style Farness TCP s congeston control algorthm ensures that smlarly stuated TCP flows (same RTT, same packet sze) receve roughly equal throughput. We call t TCP-style farness. otce that t does not assure equalty of throughput between flows wth dfferent roundtrp-tmes (RTT), or usng dfferent packet szes. More than a decade of deployment of TCP n the Internet has proven that TCP congeston control mantans ths TCP-style farness across a very wde range of network envronments... TCP-Frendlness Recently, many new congeston control protocols [BMP94, CPW98, RHE99a] have been proposed for applcatons that are not served well by TCP. Ths emergence of new congeston control protocols rases the ssue of nter-flow farness across dfferent protocols. Flows usng newly proposed congeston control protocols may not preserve the bandwdth sharng equalty wth smlarly stuated TCP flows.

28 17 The Internet communty has struggled wth ths tenson between preservng TCP-style farness, and meetng the demands of applcatons for whch TCP s a far-from-deal soluton. A recently proposed resoluton s the TCP-frendlness paradgm [FF99, TCPF]. A congeston control protocol s called TCP-frendly when t uses the same amount of bandwdth on average as a smlarly stuated TCP flow...3 State of the Art Currently the noton of TCP-frendlness s defned n terms of the average throughput over a long tme nterval (many seconds to mnutes). The cornerstone of ths approach s the observaton [PFTK98] that one can roughly characterze the average throughput r of a TCP flow n the presence of a constant packet loss rate p wth the followng equaton: r 1.MSS RTT * p (.1), n whch, MSS s the flow s packet sze and RTT s the round-trp-tme. The model s based on the followng assumptons: x TCP congeston control s workng n the AIMD steady state, whch means TCP s assume to detect congestons by duplcate acknowledgements. Ths assumpton s reasonable when the packet loss rate s low. When the loss rate s hgh, tmeout becomes the domnant mechansm for congeston detectons, and a more complex throughput model [PFTK99] s needed. x Congeston sgnals are ndependent from the sendng rate, whch means a congeston control protocol s assumed to experence the same congeston sgnal no matter how ts transent rate behaves. The detaled dervaton of the throughput equaton (.1) can be found n [PFTK98]. Ths TCP-frendlness defnton enables a wde varety of TCP-frendly congeston control protocols that can be talored to dfferent applcaton requrements. The work [TCPF] has a rch collecton of exstng TCP-frendly congeston control protocols.

29 18 Although many TCP-frendly protocols have been proposed recently, several aspects of the bandwdth sharng behavor between varous congeston-controlled flows are stll unclear. Examples of these ssues are: whether the bandwdth sharng among competng TCP-frendly flows s stable, how long t takes the system to converge to ts stable behavor, and n what tme scale the average throughput should be measured to judge a flow s TCP-frendlness, and whether transent sendng rate behavor affects congeston sgnals. The research presented n ths thess wll shed some lght on these ssues..3 AIMD-based Algorthm The steady-state stage of the TCP congeston control protocol uses the AIMD algorthm. TCP s AIMD algorthm s actually a specfc example of an AIMD-based algorthm. A AIMD-based algorthm can be descrbed as: Increase (Addtve): W ( t RTT ) W ( t) D D! 0 Decrease (Multplcatve): W ( t G ) m W ( t) EW ( t) 0 E 1 (.) where W(t) s the wndow sze at tme t, and RTT s the round-trp-tme. In the absence of congeston sgnals, the algorthm uses the Increase rule n (.), whch ncreases ts wndow by a constant D n every RTT. When the congeston control detects congestons at tme t, the algorthm uses the Decrease rule n (.), whch decreases ts wndow by a constant factor E. The new wndow sze s denoted as W(t+G), n whch t+g ndcates the tme nstance just after tme t. TCP s algorthm can be vewed as a specal case of the AIMD-based congeston control algorthm wth D =1 (packet) and E =1/. Throughout ths paper, we use AIMD(D, E ) to denote an AIMD-based algorthm usng parametersd and E, and thus use AIMD(1,1/) to denote TCP s algorthm. For an AIMD-based algorthm to be TCP-frendly, D and E are not ndependent, but have to follow the relatonshp

30 19 D 3E E (.3) The dervaton s based on the throughput model (.1) wth the assumpton that the AIMD-based protocol wll experence the same congeston sgnal as a smlarly stuated TCP flow. The dervaton of ths relaton can be found n [LSW01, YL00] 3. Intutvely, n order to make a flow wth a smaller D parameter get the same throughput as a TCP flow, t must back off less than the TCP flow. By choosng a smaller D and a smaller E, an AIMD-based algorthm s rate vares by a smaller magntude than a normal TCP does. 3 In some work, the multplcatve decrease part of AIMD s descrbed as W(t) Å E W(t) nstead of W(t) Å W(t) - E W(t). Thus the TCP-frendly relatonshp s presented n a slghtly dfferent form.

31 Chapter 3 A State-space Model Ths chapter presents our work on applyng the state-space control modelng technque [B91, C86] to a bandwdth sharng system. State-space modelng s one modern control technque that has been developed to study the dynamc behavors of systems n areas of physcs, mechancs, electroncs, aerospace, etc. The term "dynamc behavors" s a famlar concept n these areas. In ths chapter, we use a state-space model to capture the dynamc rate behavors of congeston-controlled flows. The chapter starts wth a general descrpton of the state-space control modelng technque, and then descrbes a state-space model for a bandwdth sharng system that s composed of congeston-controlled flows. At the end of ths chapter, some examples are gven of usng ths model to study the dynamc behavors of bandwdth sharng systems. 3.1 State-Space Modelng A state-space model for a system s a representaton that descrbes the evoluton of the system state, whch captures the dynamc behavors of the system. Before explanng the detals, we frst ntroduce some terms used to descrbe a state-space model. x State Varables State varables [C86] can be vewed as a runnng collecton of a system s ntal condtons. Knowledge of these condtons at a gven tme 0

32 1 together wth a fxed explct nput s all that s necessary to specfy future behavors. For convenence of notaton, we usually collect the state varables nto a vector, called a state vector. In ths document, we smply use system state to refer to a system's state vector. x Event-drven State Jumps Events n a system are generated based on certan condtons of the system state (for example, when a state varable exceeds a predefned value). Event-drven state jumps are state transtons that are assocated wth those events and can't be descrbed as dfferental equatons. A state jump s always assocated wth an event, or wth a certan delay after an event. x State Space A state space s a mult-dmensonal space n whch each dmenson represents a state varable. Thus any system state can be represented as a pont n the state space 4. x State-Space Plot A state-space plot s a geometrcal representaton of how a system s state evolves over tme n a state-space model. The soluton of the dfferental equatons and state jumps s vsualzed as a trajectory n the state space. State space plots are also called phase plots n the lterature A State-Space Model A state-space model conssts of a set of equatons descrbng the evoluton of the system state. The future behavor resultng from a partcular nput can be calculated from the state-space model once the current state s known. Our state space models use dfferental equatons and state jumps to descrbe the evoluton of the system state. We use x(t) to represent the system state at tme t and u(t) to represent the nput to the system at tme t. The dervatve of x(t) s represented by xt ( ). The dfferental equatons are wrtten n the form of (3.1), and state jumps are n the form of (3.). In ths mathematcal

33 representaton, h(x(t)) s the event trgger functon, whch returns zero when no statejump events should happen. Durng ths perod, the functon f(x(t),u(t)) of the dfferental equaton governs the system state evoluton. Functon h(x(t)) returns a non-zero value when some state-jump events should happen. At ths nstance, the functon g(x(t)) s the functon that controls the state-jumps. We use t - and t + to represent the tmes mmedately before and after the tme t. x ( t) f ( x( t), u( t)) when h( x( t)) 0 x( t ) m g( x( t )) when h( x( t)) z 0 (3.1) (3.) In addton to the above mathematcal representaton, a state-space model can also have a geometrcal representaton that helps one vsualze the system state transtons and shows the relatonshp among the state varables. The geometrcal vew of the system states s a state-space trajectory n a state-space plot. An mportant property of the geometrcal representaton s the unqueness of ts states n the state-space. Snce the system state s supposed to contan suffcent nformaton about the system at any gven tme for ts subsequent behavor to be predcted f the future nput s known, t s necessary that the dfferental equaton and state-jumps for x(t) should have a unque soluton for every ntal state x(t 0 ) and nput u(t), t W 0. Because of ths unqueness property, there s one and only one trajectory from any gven pont 5 n the geometrcal representaton, for a partcular nput u(t), t W System Stablty Capturng a system s dynamc behavor s the goal of state-space modelng. A system s dynamc behavors can be dvded nto steady-state behavors and transent behavors. The steady state behavors are how a stable system behaves once t converges to ts steady state, and the transent behavors are how the system behaves on ts way from an 4 otce here that not every pont n the state space s necessary a vald state n the system. 5 Because of the state jumps, we could have state transtons from many ponts to one pont.

34 3 ntal state to steady state. Generally, steady-state behavors get more focus because they are the domnant behavors f a system s stable. To study the steady-state behavors, or even the transent behavors, we need to frst study the system s stablty. In ths subsecton, we present a bref revew of the system stablty n the context of a state-space model Equlbrum Equlbrum descrbes possble a steady-state behavor of the system. Equlbra are generally classfed as statc or dynamc. Here we assume the nput u s fxed at u=0. A statc equlbrum s commonly referred to n the classcal control theory as an equlbrum pont. An equlbrum pont s a state xˆ that f once the system state x(t) s equal to xˆ, t remans equal to xˆ for all future tme and a fxed nput û. Mathematcally, t means that h ( xˆ) 0 (3.3) and x ( t) f ( xˆ, uˆ) 0 (3.4), n whch equaton (3.3) guarantees no state-jumps and (3.4) guarantees that the system state can not leave xˆ under the control of the dfferental equaton. Statc equlbra do not cover all the nterestng steady state behavors of a system. A common feature of nonlnear systems wth state jumps s the occurrence of a specal type of trajectory that takes the form of a closed curve. Ths s known as a lmt cycle and represents a perodc soluton of the system equatons snce, when the system state returns to ts ntal value, t must necessarly repeat ts prevous moton and so contnue ndefntely. We call a lmt cycle a dynamc equlbrum state of a system. It represents an oscllaton that s ntrnsc to the system and s not caused by external nput varatons.

35 Stablty The exstence of an equlbrum s a necessary condton for system stablty but not suffcent. Stablty requres a stable equlbrum, n whch a small perturbaton won t cause the system to leave the neghborhood of the equlbrum state or trajectory. Roughly, an equlbrum pont s asymptotcally stable n a regon around the equlbrum f whenever the system starts from any place wthn the regon, t ends up returnng to the equlbrum. It s unstable f t moves away when startng at some poston n the regon. A lmt cycle s asymptotcally stable n a nearby regon f all trajectores startng n that regon approach t asymptotcally and s unstable f some trajectory startng n the regon moves away. An asymptotcally stable lmt cycle s also called a perodc attractor. Above s a geometrc descrpton of the stablty of a system. Mathematcally, system stablty s defned n the followng way: An equlbrum state xˆ (or dynamc lmt cycle) s sad to be stable, f, for any arbtrarly small number H>=0, there exsts real numbers G > 0, and T>0, such that, f x(0)- xˆ <G, then x(t)- xˆ < H for all t>t. Otherwse, the equlbrum state (or the lmt cycle) s unstable. It s asymptotcally stable f t s stable and, n addton, x(t)- xˆ Æ 0 as tæ nf. Here x(t)- xˆ s defned as the dstance between the state x(t) and the equlbrum state xˆ when xˆ s an equlbrum pont. If xˆ s a lmt cycle, the dstance s the shortest dstance from x(t) to any ponts on the lmt cycle. 3. A State-Space Model for a Bandwdth Sharng System After brefly descrbng the state-space modelng technque, now we are ready to present a state-space model for a bandwdth sharng system.

36 Target System We are generally nterested n the rate behavors of flows n the Internet. However, the Internet s too bg and has too many factors to be characterzed by a sngle state-space model. To smplfy the study, we choose a target system that s composed of a fxed number of flows that all use AIMD-based congeston control algorthms. To focus on the steady state congeston control algorthm, we gnore all tmeouts and other components n a congeston control protocol, and we make no dstnctons between retransmtted data and new data, and abstract all of them just as a data rate. In addton, we assume a sngle bottleneck lnk for all competng flows n the system. Fgure 3.1 llustrates the target system of our study, n whch congeston controlled flows are competng for the same bottleneck lnk L, whch has a fxed rate R and a lmted queueng capacty B. S 1 R 1 S Bottleneck lnk: L Lnk Rate: R R 1 Delay: D R R S -1 S Queue Sze: B R -1 R Fgure 3.1: A Bandwdth Sharng System Models for a network system can generally be dvded nto packet-based and flud-based ones. A packet-based model uses packets as the basc unts and the system s behavor s related to the detaled character of each packet, such as ts sze and tmng nformaton. A flud-based model abstracts a system s behavor by rates, and thus gnores the nformaton of varous packet szes and the packet nterval tmes. In general, packetbased models are close to realty but are harder to use for any theoretcal studes than flud-based models. We choose to use a flud-based approach because t requres fewer state varables. In later chapters, we address the quantzaton effect and randomness caused by packet szes.

37 6 To construct a model for the target system, we frst dvde t nto a bottleneck subsystem and AIMD rate control subsystems. We then locate the key factors n each knd of subsystem to determne the system state Bottleneck Subsystem We abstract a bottleneck lnk as a leaky bucket that has a constant leak rate (the bottleneck rate) R and a bucket (the bottleneck queue) wth a lmted sze B. The bucket fll-level fl(t) s controlled by the leak rate and the nput rate r S (t) to the bucket, whch s the sum of nput rates of all the flows to the queue. The equatons for the evoluton of fl(t) are summarzed n (3.5). dfl( t) dt dfl( t) dt dfl( t) dt mn( r r ( t) R S S ( t) R,0) max( r ( t) R,0) S f fl( t) B f 0 fl( t) B f fl( t) 0 (3.5) A congeston sgnal s produced when fl(t) = B and r s (t)>r. However, t s not necessary that every flow perceves the congeston sgnal. Determnng whch flow should perceve the congeston sgnal s not smple. In realty, some flows mght be unfortunate and get ther packet dropped when the bottleneck queue s full, whle some flows do not. Ths random phenomenon happens n the taldrop queue, and could be exaggerated by some advanced queue management schemes [BCC+98] such as RED [FJ93]. We address ths ssue when we defne the event trgger functon h(x) AIMD Subsystem We vew the AIMD rate controller of each flow n the target system as a feedback subsystem, whch outputs a sgnal onto the network to probe the bottleneck state and uses the probe result to control the data output rate. Ths feedback subsystem s llustrated n

38 7 Fgure 3.. An AIMD rate controller probes the network's state wth the data t sends. Data packets travel from the sender to the recever, and acknowledgments for each packet travel back from the recever to the sender. The tme from sendng a packet to recevng ts acknowledgment s the round-trp tme (RTT). The RTT s mportant because t s the delay around the feedback loop. AIMD Rate Controller FD etwork Congestonprobng BD QD Fgure 3.: AIMD Rate Control Subsystem We dvde the RTT nto three parts: the forward delay FD, the bottleneck queueng delay QD, and the backward delay BD (that s: RTT = FD + QD + BD). The forward delay s the tme between the nstant that a flow ncreases ts rate to the nstant that the ncrement starts contrbutng to the nput rate to the bottleneck queue. The bottleneck queueng delay s the tme that s taken by a packet to go through the bottleneck queue. The backward delay s the tme between the nstant that a congeston sgnal s generated at the bottleneck to the nstant that the flow receves t. Snce we assume a sngle bottleneck lnk, packets should only accumulate at the bottleneck and not at other lnks. Thus, both the forward delay and backward delay are composed by lnk propagaton delays, and are therefore treated as constants n our study. Snce the queueng delay s related to the amount of data n the queue, we smply let QD = fl(t)/r. ow we start to abstract the behavor of an AIMD rate controller. An AIMD rate controller lmts the rate at whch data s sent out on the network by usng a congeston wndow. The congeston wndow sze defnes the maxmum amount of outstandng data, data that has been sent but not yet acknowledged; hence, the amount that s sent out n one RTT. If no congeston sgnals have arrved at the AIMD controller, t should have receved acknowledgments for all packets that were sent durng the last RTT. An AIMD rate controller uses packet losses or explct notfcatons as congeston sgnals. For

39 8 detals of packet loss detectons, please refer to Chapter. Under the absence of congeston sgnals, an AIMD controller ncreases ts congeston wndow sze by D packets n every RTT (that s ncreasng ts rate r by D packets/rtt n every RTT); otherwse t decreases ts congeston wndow by E tmes the current wndow sze. Equaton (3.6) and (3.7) summarze an AIMD controller s behavors. When no congeston occurs (uncongested state): dr ( t) dt D u MSS RTT n whch MSS s the packet sze. (3.6), When congeston occurs: r m ( 1 E ) u (3.7). r Wth the above abstractons of the bottleneck lnk and AIMD controllers, the target system can be represented as a decentralzed control system, whose behavor s controlled by AIMD controllers and one bottleneck queue. The lnk between these dstrbuted controllers and the key varables of the system s presented n Fgure 3.3. Our studes of bandwdth sharng systems are based on ths abstracted system. BD AIMD 1 r 1 (t) FD 1 BD 1 Congeston Sgnals AIMD r (t) FD r 1 (t-fd 1 ) + fl(t) R r -1 (t) AIMD -1 AIMD r (t) FD -1 FD r (t-fd ) BD BD -1 Fgure 3.3: The Bandwdth Sharng System wth Key Informaton only

40 9 3.. System State The essental feature of the state of a state-space model for a system s that t contans all nformaton about the past hstory of the system that s relevant to ts future behavor. In the target system, the mportant nformaton that captures the state of the system s the nstantaneous transmsson rate of each flow and the queue fll-level n the bottleneck router at any nstant. Each flow adjusts ts rate based on ts current rate and the congeston sgnals from the bottleneck lnk, whch are determned by the aggregated rate of all competng flows. Thus, the transent transmsson rates of all competng flows and the queue fll-level of the bottleneck router hold all the hstory nformaton that s relevant to determne the future behavor of the system. Therefore, we choose the transmsson rate of every competng flow and the fll-level of the bottleneck router queue as the state varables. We use r (t) to denote the transmsson rate of flow, and fl(t) to denote the bottleneck queue fll-level at tme t. For a system wth competng flows, the system s state at tme t n our model s a vector x )] T ( t) [ r1 ( t), r ( t),, r ( t), fl( t. The target system has a few other key factors that affect the system behavors but are not counted as state varables because they are statc or beng assumed statc by us. These factors are the bottleneck rate R, the queue sze B at the bottleneck router, and the forward and backward delay between each sender and the bottleneck queue. In a real system, these factors mght be vared by router software dong custom queung or dfferental servces Dfferental Equatons and State-Jumps Based on the above system state defnton, we now present the rules that control the state evoluton. We frst look at the event trgger functon h(x), and then dscuss the functons g(x) and f(x, u) for the system evoluton wth and wthout state-jumps.

41 30 The event trgger functon h(x) s not straghtforward. As mentoned n subsecton , one ssue that needs to be addressed by the trgger functon s whch flow perceves the congeston sgnal. We have a few choces on the selecton of the trgger functon. One choce s a pure determnstc approach n whch every flow perceves all congeston sgnals. Ths choce maps to the realty when severe congestons happen. An alternatve choce s usng a random process that lnks the possblty of percevng a congeston sgnal to the transent rate of the flow. The goal of buldng the model s to study the dynamc rate behavors. Ths goal ncludes two tasks: understandng the system stablty and understandng the nteracton between controller behavor and ts congeston perceptons. For the task of studyng system stablty, the frst choce (every flow percevng a congeston sgnal) provdes an easy start. For the task of studyng the nteracton between rate and congeston perceptons, we can no longer assume that all flows perceve all congeston sgnals. In later chapters, we wll show through experments that flows wth dfferent parameters n ther rate controller have dfferent congeston perceptons. In ths chapter, snce the major theme s to use the state-space model to study the system stablty, we decde to start wth the smple assumpton that every flow perceves a congeston sgnal whenever congeston happens. We wll extend our model to more general cases and nvestgate the effects of these choces n the next two chapters usng smulatons and real-world experments. The event trgger functon h(x) also needs to address the effect of feedback delays on AIMD controllers. An AIMD controller wll not back off contnuously durng queueng overflows at the bottleneck. Because RTT s an AIMD controller s feedback delay, an AIMD controller lets back offs happen no more than once n a sngle RTT perod 6. In addton, dfferent AIMD controllers could perceve congeston sgnals at dfferent tmes when they have unequal backward delays. For ths delay ssue, and snce the backward delay could dffer among flows, we defne a separate event trgger functon h (x) for each flow as:

42 31 h (x) = 1 f fl(t-bd )=B, h (x) = 0 r S ( t BD)! R, and h (V) s 0 for all V such that t-rtt<v<t (3.8) otherwse. Ths defnton makes sure that, once the fll-level reaches ts lmt B and the total nput rate from all flows r s (t) s hgher than the lnk rate, the flow receves a congeston sgnal after a delay BD (ndcated by the condton fl(t-bd )=B and r S ( t BD)! R ). The defnton n (3.8) also makes sure that the flow receves at most one congeston sgnal per RTT, otherwse h (V) would not be zero for all V such that t-rtt<v < t. Up to now, we know the trgger functons for all the state jumps. To defne the functons that control the system behavor other than state-jumps, we want a functon that ndcates the absence of state-jumps. Therefore, we defne a global event trgger functon h(x). h(x) = 0 h(x) = 1 f all h (x) = 0 (no state-jumps) otherwse (3.9) When the event trgger functon h(x) = 0, the system state evoluton s under the control of dfferental equatons, n whch the evoluton functon f(x,u) s presented as the followng: f ( x( t), u( t)) f ( x( t), u( t)) f ( x( t), u( t)) D1 umss1 D umss D umss [,,, RTT ( t) RTT ( t) RTT ( t) 1,mn( r ( t) R,0)] D1 umss1 D umss D umss T [,,,, r ( ) ] S t R f RTT1 ( t) RTT ( t) RTT ( t) D1 umss1 D umss D umss T [,,,,max( r ( t) R,0)] S RTT ( t) RTT ( t) RTT ( t) In f(x,u), flow ncreases ts rate by 1 S T f fl( t) 0 fl( t) B f fl( t) D u MSS, n whch D RTT s the AIMD ncrement ( t) parameter of flow, and RTT (t) s ts round-trp-tme. Also n (3.10), B 0 (3.10) 6 Ths clam s rght for TCP ewreno, TCP SACK, but t s not rght for TCP Tahoe and TCP Reno. We make ths clam because TCP ewreno and TCP SACK are the domnant flavors of TCP n the Internet today [PF01].

43 3 rs ( t) r ( t FD ), whch s the total rate of all competng flows at the bottleneck, 1 where FD s the forward delay of flow. otce that (3.10) s dvded nto three cases based on the value of fl(t). When fl(t) s larger than zero but less than B, then the dervatve of fll-level can be ether postve, zero, or negatve. When t s equal to zero, snce the fll-level cannot become negatve, the dervatve of fl(t) can only be postve or zero. When h(x) s not zero, that means at least one state-jump happens. For every h (x) = 1, we have a state transton under the control of g (x): g ([ r t r t r t fl t r1 t T T 1( ), ( ),, ( ), ( )] ) [ ( ),,(1 E ) ( ),, ( ), ( )] (3.11). In (3.11), only flow 's rate s reduced by a factor of system are unchanged. Ths s the AIMD s rate behavor n (3.7). r t r t E and all other state varables of the fl t The state defnton n subsecton 3.. and the equatons (and transtons) (3.8) ~ (3.11) together comprse the state-space model for our target system Smulaton Fgure 3.4 A Snapshot of a Smulnk Smulaton Block