End-to-end available bandwidth estimation Constantinos Dovrolis Computer and Information Sciences University of Delaware Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 1 of 28%
The big picture The Internet is really a big black cloud A Internet? B But.. End-systems can infer what is in the box through end-to-end measurements Why bother? Improved transport, QoS, overlay routing, server selection, etc This work focuses on end-to-end bandwidth estimation Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 2 of 28%
Overview Bandwidth metrics: capacity and available bandwidth Available bandwidth measurement methodology: SLoPS Available bandwidth measurement tool: pathload Using pathload to understand the dynamics of available bandwidth Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 3 of 28%
Part I Capacity and Available Bandwidth Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 4 of 28 %
Capacity Fast Ethernet DS3/ATM Fast Ethernet Capacity: maximum possible end-to-end throughput S 100 45 100 36 (IP) + 9 (ATM) R End-to-end capacity C is limited by narrow link n: C = min fc i g = C n i=0:::h See www.pathrate.org Pathrate: measurement tool based on packet pairs/trains (Infocom 01) Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 5 of 28%
Available bandwidth Fast Ethernet DS3 Ethernet Avail-bw: maximum end-to-end throughput without reducing cross traffic rate S 100 36 10 R Cross-Traffic 30Mbps Network managers monitor avail-bw with MRTG (based on router statistics) Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 6 of 28%
Avail-bw of link i during T : A i = C i (1 ; u i ) A = A i = C i (1 ; u i ) Definition of avail-bw C i : capacity of link i u i : utilization of i link in time T (0 u interval i 1) C 1 C 2 C 3 min i=0:::h min i=0:::h A Source Sink Tight link Avail-bw is limited by tight link Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 7 of 28%
Part II Available Bandwidth Estimation Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 8 of 28 %
Applications of avail-bw estimation Congestion control and TCP: measure Bandwidth-Delay-Product Streaming applications: adjust encoding rate SLA and QoS verification: monitor path load Content distribution networks: select best server Overlay networks: configure overlay routes End-to-end admission control: check for sufficient bandwidth But how can we measure end-to-end avail-bw? Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 9 of 28%
Previous work on available bandwidth estimation 1. Measure throughput of large TCP transfer TCP does not get available bandwidth in under-buffered paths TCP gets more than available bandwidth in over-buffered paths TCP saturates the path (intrusive measurements) 2. Carter Crovella: dispersion of long packet trains (cprobe) Does not measure available bandwidth (see Infocom 01) 3. Banerjee Agrawala (ICN 00): define available capacity as: Amount of data that can be sent to path with OWD of less than 4. Melander et.al. (Global Internet 00) and Ribeiro et.al. (ITC 00) Correct estimation when queueing only at single link in path Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 10 of 28%
Does bulk-tcp measure avail-bw? Conventional wisdom: bulk-tcp oscillates around avail-bw Perform bulk-tcp transfer during (B) and (D) (5-min intervals) 9 UoI (Ioannina) to UDel (Newark) 400 UoI (Ioannina) to UDel (Newark) TCP throughput avail bw (Mbps) 8 7 6 5 4 3 2 1 MRTG avail bw (5 min avg) TCP throughput (5 min avg) TCP throughput (1 sec avg) (A) (B) (C) (D) (E) RTT measurement (msec) 350 300 250 200 150 (A) (B) (C) (D) (E) 0 0 300 600 900 1200 1500 Time (sec) 100 0 300 600 900 1200 1500 Time (sec) Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 11 of 28%
OWD: D i = T R arrive ; T S send = T arrive ; T send + Clock Offset(S R) Interested in OWD variations: D i ; D i+1 Self-Loading Periodic Streams (SLoPS) SLoPS requires access at both ends S and R of path S sends periodic UDP packet streams to R (timestamped) SLoPS analyzes One-Way Delays (OWDs) of packets from S to R So, S and R do NOT need to have synchronized clocks Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 12 of 28%
SLoPS: Basic idea Periodic stream: K packets, size L bytes, rate R = L=T T = L / R 1 2 3 4 K=4 At sender D1 1 2 3 4 D2 D3 D4 At receiver when R>A 1 2 3 4 D1 D2 D3 D4 At receiver when R<A If R > A, OWDs gradually increase due to self-loading of stream % Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 13 of 28
V (t i t i + T ) + Q(t i ) = S(t i t i + T ) + Q(t i + T ) E[V (t i t i + T )] > S(t i t i + T )! Q(t i + T ) > Q(t i ) Analytical model Flow conservation at tight link in interval (t i t i+1 = t i + T ) with T = L R : Arrivals + Queue = Departures + Queue 0 Expected arrivals: E[V (t i t i + T )] = u t C t T + L = (C t ; A)T + RT = [C t +(R ; A)]T Upper bound on departures: S(t i t i + T ) C t T If R > A But, Q(t i + T ) > Q(t i )! D i+1 > D i i.e., Packet stream has increasing OWDs Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 14 of 28%
Increasing trend: R > A A=74Mbps, R=96Mbps, (K = 100 packets, T =100s, L=1200B) Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 15 of 28 %
Non-increasing trend: R < A A=74Mbps, R=37Mbps, (K = 100 packets, T =100s, L=462B) Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 16 of 28 %
Grey-region: R./ A A=74Mbps, R=82Mbps, (K = 100 packets, T =100s, L=1025B) Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 17 of 28 %
Part III Pathload Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 18 of 28 %
Iterative algorithm in SLoPS max R > A Increasing trend: R(n) > A R max = R(n) Grey region G G max min R(n + 1) = (G max + R max )=2 Non-increasing trend: R(n) < A min R < A R min = R(n) Terminate if: max G G min R R < w or max min max min R R Grey region R(n) > G max : G max = R(n) + 1) = (G max + R max )=2 R(n Grey region R(n) < G : min R(n + 1) = (G min + R min )=2 G min = R(n) R(n + 1) = (G min + R min )=2 Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 19 of 28%
Pathload features not covered in this talk Statistical tests for detection of increasing trend Clock-skew compensation Detection of context switches at sender receiver Fleets: a number of streams of same rate (spaced by one RTT) Selection of packet size L and period T Dealing with losses and congestion responsiveness Initialization of rate adjustment algorithm For more details, see PAM 2002 publication on pathload Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 20 of 28%
Verification results 100 U Oregon to U Delaware Verify avail-bw using MRTG graphs for path routers Available bandwidth (Mbps) 90 80 70 60 MRTG measurement Pathload measurement 50 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Measurement # Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 21 of 28 Tight link: U-Oregon GigaPoP link (C=155Mbps),!=3Mbps %
Is pathload intrusive? Does pathload decrease avail-bw? Does it increase delays losses? Run pathload during (B) and (D) (5-min intervals) 6 UoI (Ioannina) to AUTH (Thessaloniki) 175 UoI (Ioannina) to AUTH (Thessaloniki) Available bandwidth (Mbps) 5 4 3 2 1 MRTG avail bw (5 min avg) (A) (B) (C) (D) (E) RTT measurement (msec) 150 125 100 75 50 25 (A) (B) (C) (D) (E) 0 0 300 600 900 1200 1500 Time (sec) 0 0 300 600 900 1200 1500 Time (sec) Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 22 of 28%
Part IV Variability of available bandwidth Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 23 of 28 %
Variability and load conditions Relative variation of avail-bw: = ;R min Rmax max +R min )=2 (R Heavier utilization at tight link causes higher variability Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 24 of 28%
Variability and statistical multiplexing Traffic aggregation reduces avail-bw variability Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 25 of 28 %
Variability and stream duration Variability decreases as stream duration (averaging timescale) increases Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 26 of 28 %
Summary Avail-bw estimation has numerous applications SLoPS: fast, accurate, and non-intrusive measurements Implemented in pathload (to be released in Spring 02) Evaluation of avail-bw variability using pathload Future work: incorporate avail-bw estimation in transport, QoS, and routing Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 27 of 28%
Thank you! Constantinos Dovrolis - dovrolis@cis.udel.edu, IPAM workshop, March 2002 28 of 28%