Measuring quality of user experience in communication networks

Transcription

1 Measuring quality of user experience in communication networks Pravir Chawdhry European Commission - Joint Research Centre ITU Regional Conference on Broadband Mapping Warsaw, Poland April 2016

2 Quality of User experience One of the most important aspects that affects the Internet consumer experience is the quality of network performance. Traffic growth is dominated by on-line video and audio-streaming so low latency and high speed add great value to user experience.

3 Quality of User experience One of the most important aspects that affects the Internet consumer experience is the quality of network performance. Traffic growth is dominated by on-line video and audio-streaming so low latency and high speed add great value to user experience. Speed test applications have become widely used tools for customers.

4 Quality of User experience One of the most important aspects that affects the Internet consumer experience is the quality of network performance. Traffic growth is dominated by on-line video and audio-streaming so low latency and high speed add great value to user experience. Speed test applications have become widely used tools for customers. But how reliable are these tools?

5 Quality of User experience One of the most important aspects that affects the Internet consumer experience is the quality of network performance. Traffic growth is dominated by on-line video and audio-streaming so low latency and high speed add great value to user experience. Speed test applications have become widely used tools for customers. Are these tools reliable? What do these tools really measure?

6 Measuring network performance ICT Technicians can use advance tools and use them with their domain competences.

7 Measuring network performance ICT Technicians can use advance tools and use them with their domain competences. Know-how to check network activity.

8 Measuring network performance ICT Technicians can use advance tools and use them with their domain competences. Know-how to inspect network characteristics with command like tracert and nmap. Know-how to check network activity. And run specialized applications like speedtest-cli, curl, etc..

9 Measuring network performance Nontechnical citizens search for a Speed test tool over Internet.

10 Measuring network performance Nontechnical citizens search for a Speed test tool over Internet.

11 Measuring network performance Nontechnical citizens search for a Speed test tool over Internet.

12 Measuring network performance But results can be very confusing.

13 Measuring network performance How can we know if these tools are reliable?

14 Measuring network performance How can we know if these tools are reliable? App Here is an example of speed test measures over the Internet from the same Workstation using 4 different tools Ping ms Download Speed Mb/s Upload Speed Mb/s SpeedTest ,49 83,36 mlab 22 9,75 13,19 netbravo 71 15,74 59,63 RTR-NetTest ,00 42,00

15 Measuring network performance How reliable are these tools? And what are they measuring?

16 Measuring network performance How reliable are these tools? And what are they measuring? UDP

17 Measuring network performance How reliable are these tools? And what are they measuring? UDP TCP

18 Measuring network performance How reliable are these tools? And what are they measuring? UDP TCP HTTP

19 Measuring network performance The discrete behaviour of the NETWORK is very complex and fairly unpredictable Compression Server Switch Router Firewall Internet Every transition can have effect on the final resulting speed due to: High bandwidth usage on any route segment Router, firewall or switch misconfigurations CPU overhead on any used device Client network activity Compressible or uncompressible content.

20 What about benchmarking? To reduce the complexity of the network and test the reliability of Speed Test tools we can create a reference framework to test and benchmark the applications using a simulated simplified network in which we can operate on various parameters: Up and Down stream Bandwidth Latency Packet loss Round time trip. Following the directive of IPPM (IETF)

21 Benchmarking: an example Previous speed test tools tested on a simulated network that cut download and upload bandwidth to 1, 2, 5, and 10 Mb/s WebApp Latency Download Upload 1Mbs NetBravo 23 ms 0,90 Mb/s 1,02 Mb/s M-Lab 410 ms 0,88 Mb/s 0,91 Mb/s Ookla 30 ms 0,97 Mb/s 0,98 Mb/s Z-Test 72 ms 0,94 Mb/s 0,94 Mb/s 2Mbs NetBravo 23 ms 1,74 Mb/s 2,08 Mb/s M-Lab 230 ms 1,84 Mb/s 1,88 Mb/s Ookla 62 ms 1,92 Mb/s 1,93 Mb/s Z-Test 52 ms 1,90 Mb/s 1,90 Mb/s 5Mbs NetBravo 53 ms 4,38 Mb/s 4,96 Mb/s M-Lab 102 ms 4,63 Mb/s 3,84 Mb/s Ookla 30 ms 4,80 Mb/s 4,81 Mb/s Z-Test 48 ms 4,80 Mb/s 4,70 Mb/s 10Mbs NetBravo 18 ms 8,59 Mb/s 9,33 Mb/s M-Lab 76 ms 9,22 Mb/s 7,91 Mb/s Ookla 31 ms 9,61 Mb/s 9,31 Mb/s Z-Test 51 ms 9,70 Mb/s 4,60 Mb/s

22 JRC experience: netbravo JRC developed a mobile crowd sourcing application that measures quality of user experience: Speed test: Download and upload data transfer rates. Latency, ping and DNS lookup test Local GSM / UMTS / LTE signal strength Local WiFi 2.4 and 5 GHz access point signal strength, sid and characteristics (used channel, security protocols, sid, etc..) Network neutrality test (voip, nat, p2p, etc..)

23 ios Application screen-shots

24 ios Application screen-shots

25 Android Application screen-shots

26 Android Application screen-shots

27 netbravo Mapping: netbravo.org

28 netbravo Application optimizations The application is: Optimized for battery usage Optimized for Bandwidth consumption Takes measurements only when a location is available Measurements are made based on battery state and location availability Uploads measurements to the server when the preferred connection is available Application user data is fully anonymized.

29 Thank you Further information: