Analysisof constraintsin Internet Speed Measurements. Fidel Liberal

Transcription

1 Analysisof constraintsin Internet Speed Measurements Fidel Liberal

2 Outline Background Motivation Implementations alternatives and constraints Case Study: Technical constraints in multiple TCP flows based measurement Conclusions 2

3 Background Internet Speed measurements (for End Users) De-facto standards Lack of unified approach Lack of replicability/reliability Fulfilment of advertised speed/sla ITU-T Q15/11 Framework + methodology More info 3

4 Background: Motivation Download Upload 70 Measured Speed (mbps) Different QoS measurement platforms 4

5 Implementation alternatives Typically 2 approaches A) Operator/Regulator driven Large Scale Measurement Platforms Coordinated Active Statistical value only B) End Users Unmanaged Active/Passive For both approaches HW (probe/stb SW (web based/app) Measurement mechanism Single/multiple TCP flows More sophisticated metrics No clear winner 5

6 Implementation alternatives Approach A Large Scale Platforms Regulator Reliable Comparable tests SLA s and customer protection Temperature of the broadband market Approach B End Users Examples: SamKnows RIPE Atlas Bismark Dasu Examples: Speedtest RTR Operator Controllable Capability to incorporate multiple metrics Technically sound Ability to identify network problems and their causes End Users Availability( time to market ) Simplicity/understandable to users Technically feasible to deploy in multitechnologyues(includingweb browsers, apps, etc). Comparison with de facto standards 6

7 Selection criteria/constraints Depending on the final target Accuracy Replicability Adoptability Deployment Cost Criteria or Constraint? No accuracy & replicability No SLA check Usable by End Users Accuracy without buying or installing anything Replicability Cost Deployment Adoptability Web based App End User HW probe Operator/Regulator platform 7

8 Case study: technicalconstraintsin Multiple TCP stream based measurement Current de-facto standard Constraints Conditions that may prevent achieving maximum speed Technical hard constraints Sliding window => max. gput<(max. in-flight pkts./rtt)» Limitations due to Maximum Effective Window Size Measurement Interval Time to reach maximum speed Significativeperiod TCP dynamics Network evolution 8

9 Constraint analysis: Maximum Effective TCP Window Size MEWS=min {STATIC, DYNAMIC variables } STATIC (OS dependant -configurable-flow Control related) min(negotiated WS, MaxSenderTxBuffer, MaxReceiverRxBuffer) DYNAMIC (-TCP flavour dependant-congestion and Flow control) min(cwnd(t),awnd(t)) Not only due to capacity» CWND(t)=f(t e2e available capacity, TCP dynamics, channel errors, competing flows ) Actual maximum available e2e capacity Speed MEWS limitation for a certain BDP Evolution of TCP Window Time 9

10 Constraint analysis: MEWS => WS limitation Different Window Scale appearances within 24 hours (Client => Server) Window Scale values Hours of day High percentage of WS=0, resulting in a maximum achievable capacity of 64KB in the receiving buffer. Essential assessment to ensure measurement reliability. 10

11 Constraint analysis: MEWS => WS limitation 58 Option a) Use 58 connection for the test => close to e2e capacity but unrealistic(no user uses 58 conns) Option b) Use realistic#conns => User will never be able to get advertised speed from his connection WS and Tx/Rx buffer sizes result in a fixed and unsolvableconstraint to the maximum achievable goodput/connection for a certain RTT Although most modern OS have either big enough or autotuningbuffers, access to remote resources may still result in a bottleneck for very high RTTs (not than unusual) Considered Access and Internet Resources tests show very different constraints 11

12 Constraint analysis: TCP Window evolution& multiple flows More stable evolution (and resistance to competing flows) Quicker Slow Start Resulting aggregated speed Speed Single threaded speed Connection 2 Connection 3 Connection 4 Connection 1 Time 12

13 Constraint analysis: TCP Window evolution& measurement period Speed Speed5 Speed3 Speed4 Speed2 Speed1 Time 13

14 Constraint analysis: TCP Window evolution& measurement period Speed Speed Time 14

15 Constraints analysis: Measurement period (example 1) Samples of Goodput evolution for single TCP connections Amsterdam=>Bilbao during 24 hours Once the pipe is full the obtained goodputis stable regardless CWND evolution. goodput (mbps) Measurement interval 5-10 secs Time(s) 15

16 Constraints analysis: Measurement period (example 2) 100 Samples of Goodput evolution for single TCP connections USA=>EUR during 24 hours Goodputfollows CWND evolutions goodput (mbps) Time(s) Measurement period proportional to epoch time =f (TCP, e2e delay, network dynamics, cross traffic) 16

17 Conclussions Technical and non technical constraints Stakeholder dependant Criteria become constraints as soon as accuracy and replicabilityare mandatory. For example Avoid dependence on user equipment/cross traffic HW probe or STB based Publicly available + no need to install + usable in your own connection WEB/TCP based Key findings multiple TCP case Relevance of End User s OS/SW Configuration Problem of number of concurrent connections High enough but unrealistic vs. realistic but under-performing (bug or feature?) Warning upon suspicious tests» Not big enough MEWS Reasonable epoch-time related test duration Statistical post-processing Pending issues Effect of competing flows Mobile network effects Other metrics 17

18 Questions? Fidel Liberal Eva Ibarrola