QOE: A MICRO AND MACRO PERSPECTIVE

Size: px

Start display at page:

Download "QOE: A MICRO AND MACRO PERSPECTIVE"

Arnold West
6 years ago
Views:

1 QOE: A MICRO AND MACRO PERSPECTIVE Henning Schulzrinne (with Kyung-Hwa Kim, Hyunwoo Nam, Vishal Singh, Daniel Song) Columbia University

2 Micro vs. Macro Time & space local (user, HH) Internet-wide short-term diagnostics Internet weather long-term user satisfaction Internet climate (QoS and usage changes over time)

3 QoS à QoE à QX QoE: focus on individual applications (VoIP, video) QX: focus on overall user experience interaction with customer support installation billing perception of security price-value perception speed of restoration after service outages Example: ACSI survey (June 2015)

4 4 Micro: circle/cycle of blame probably packet loss in your Internet connection à reboot your cable modem OS ISP probably a gateway fault à choose us as provider Voice service provider must be a Windows registry problem à re-install Windows app vendor must be your software à upgrade

5 5 Micro: DYSWIS and WiSlow DYSWIS ( Do you see what I see? ) IoT Diagnosti c Platform WiSlow ( Why is my Wi-Fi Slow? ) Packet Analyzer Non-WiFi Interference Detector Peer Network Fault Filtering Crowdsourcing Remote Probing DYSWIS Framework Rule System

6 6 Crowdsourced Rules Symptom Probe Rule #1 Probe Rule #2 Probe Rule #3 Simple & independent Who? What? Options If: results Then: likely causes & unlikely causes Scoring system Likely causes Probable causes when a particular type of node runs a specific probing module and returns the specified result Unlikely causes Causes that are believed to be irrelevant to the returned result Receive responses from others asynchronously and update the score +1 for likely causes, - 1 for unlikely causes The cause with the highest score is considered the most probable root cause

7 7 DYSWIS Framework Collaboration of peers Network Experts Crowdsourcing of modules and rules Admins or Vendors upload Rules Modules upload AS 5000 AS 1781 Diagnosis Rules & Probe Modules Repository Remote Probe Request Overlay Network (DHT) download History lookup Rules Modules AS 14 Peer-to-peer network Request probes More clues when probe results from various environments are obtained Overcome limitations of single-user investigation Automatic detection Filtering mechanism Distinguishes meaningful failures Crowdsourced rules Small and independent Cooperate to build probe modules and diagnostic rules. Distributed probes Parallel and systemic approach Problem detected è Ask others: TCP connection? DHCP? DNS? Traceroute Ping HTTP? Port blocking

8 8 Problem ID C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 Description Misconfiguration on the user s computer A problem on the link to a router Misbehavior of the local router ISP outage Link between the ISP and the Internet Remote service provider network outage Remote server down The service provider blocks your ISP The server blocks your ISP The service provider blocks your IP address The server blocks your IP address (a) Possible causes Rule ID Requesting probing to: Probing module If response is: Likely cause Unlikely cause R1.1 Sister node TCP connection Yes C1 C2 C11 R1.2 Sister node TCP connection No C3 C11 C1, C2 R1.3 Sister node TCP connection No response C2 - R1.4 Near node TCP connection Yes C10, C11 C1 C9 R1.5 Near node TCP connection No C5, C7 C9 C1 C3 R1.6 Near node TCP connection No response C2 C4 - R1.7 Internet node TCP connection Yes C8 C11 C1 C7 R1.8 Internet node TCP connection No C6, C7 C1 C5 R1.9 Internet node TCP connection No response C1 C5 - R1.10 Far node TCP connection Yes C11 C1 C8, C10 R1.11 Far node TCP connection No C7 C1 C6, C8 C11

9 9 Windows 7 Mac OS X Network HomeNet No. Injected faults DYSWIS diagnostic tool diagnostic tool Magic Pro Manager 1 Ethernet cable unplugged O O O O O 2 Network adapter disabled O O O O O 3 IP address conflicts O O X X Δ (Invalid IP address) 4 Incorrect gateway address Δ (DHCP is not enabled) Δ (reboot the router) X X Δ (Your local gateway router is down or refusing your request) 5 DNS address misconfigured O X X X O (no proper DNS server. Other nodes do not observe the problem on their DNS servers. We recommend an alternative DNS server: xx.xx.xx.xx) 6 Server down (Web or SSH server) Δ (No connection) X X X 7 A NAT blocks a server Δ (No connection) X X X 8 An ISP blocks a server Δ (No connection) X X X 9 Port blocking by NAT (e.g., SSH and BitTorrent) O (Nobody can connect to the server at present. It is very likely that the server is not working at present) O (It is very likely that your router blocks the server) O (It is very likely that your ISP blocks the server) X X X X O (The port X is blocked by your router) 10 Port blocking by ISP X X X X O (The port X is blocked by your ISP) 11 A web server is too slow X X Δ Δ Δ (Pinpoint possible congestion points with additional step) Identified the root causes in 70% of test failure scenarios

10 YouSlow: Video QoE monitoring End-user s browser YouSlow Core YouSlow

edu/ Monitoring users youslow/ Obtain various playback stats directly from

, startup latency, quality changes, rebuffering status, approximate location

o What is the average rebuffering time I usually experience?

10 10 YouSlow: Video QoE monitoring End-user s browser YouSlow Core YouSlow monitoring server Monitoring users youslow/ Obtain various playback stats directly from video players (e.g., startup latency, quality changes, rebuffering status, approximate location and local ISP info.) o Micro: What bitrates did I watch? o What is the average rebuffering time I usually experience? o How is my ISP performance compared to my neighbor s ISPs? Related to net neutrality issues Related work o Google video quality report - o Conviva -

11 YouSlow extension Background scripts Chrome local storage Chrome Web request Inject a script when a viewer watches video at www.youtube.

11 11 YouSlow extension Background scripts Chrome local storage Chrome Web request Inject a script when a viewer watches video at YouSlow core content scripts YouTube player End-user s Chrome browser Internet YouSlow monitoring server YouSlow has obtained more than 500,000 YouTube s playback views from about 1,000 viewers in more than 100 countries We have recently released the beta versions of ios and Android applications

12 Monitoring site at 12

13 Conclusion larger crowds better geographic & ISP locality better statistical validity better diagnostics more users micro macro

Correlating Network Congestion with Video QoE Degradation - a Last-Mile Perspective

Correlating Network Congestion with Video QoE Degradation - a Last-Mile Perspective Correlating Congestion with Video QoE Degradation - a Last-Mile Perspective Francesco Bronzino, Paul Schmitt, Renata Teixeira, Nick Feamster, Srikanth Sundaresan AIMS 13 March 2018 Last-Mile Measurement: