NEW Smartphone Based QoS/QoE Validate and benchmark OTT services vs. telco services e.g. VoLTE vs. WhatsApp call etc.

Transcription

1 NEW Smartphone Based QoS/QoE Validate and benchmark OTT services vs. telco services e.g. VoLTE vs. WhatsApp call etc. Your Bassem Name Youssef

2 SIGOS Entire Solutions for Business Optimization Technical Perspective in Business Assurance Active Testing & Monitoring in heterogenous networks Active Testing & Monitoring across countries End User Perspective to Assure Your Business Service Management with Quality of Service End-To-End Service Management with Customer Experience Service Management with App Experience in enterprise and telco Commercial Business Assurance Revenue Assurance with Fraud Management Revenue Assurance with Billing & Tariff Verification Expertise & Know How to Assure Your Business Success Utilization of your tools with SIGOS senior experts Page 2

3 SIGOS Testing & Monitoring for Business Optimization Roaming and Interconnect World s largest roaming test system GSMA 4G reference system #1 roaming rollout and services Interconnect Fraud Detection #1 fraud detection solution used Most innovative & certified solution Fastest fraud detection available End-to-End QoS & QoE Testing SITE: world leading active test system Testing & monitoring QoS and QoE All technologies, all services App Experience Testing & Monitoring Apps Largest global device bank Used by over 600 enterprises and mobile network operators Billing & Tariff Verification Most comprehensive active RA Any stage, any tariff, any promo National and roaming RA OPERATIONS REVENUE Page 3

4 Agenda Smartphones Adoption Importance of QoE Testing SIGOS Solution for Smartphone QoS/QoE Monitoring OTT Services vs Telco Services Qos/QoE Smartphone Testing Suite with Hybrid LU5 Mobile App Testing with AppEx CLU Sample Reports Smartphone Testing on GlobalRoamer Success Stories Outlook Conclusion

5 No More Smartphones!! while Roaming Page 5

6 Smartphones Adoption by 2025 Smartphone Adoption is reaching 77% globally by 2025 Development, Maintenance and Monitoring of the smartphone ecosystem is a mandate for Mobile Network Operators Page 6

7 Based on Real Events 2017 Various OTT outage Incidents Due to an unknown issue affecting the smartphone app, users were seeing the clock symbol after attempting to send a message, meaning that the message hadn't been sent to WhatsApp's servers. Downtime reached 1 hour globally Facebook and Instagram users in parts of USA, Europe and Canada were unable to access their accounts 12,000 incidents were reported on DownDetector.com (a website that monitors outages) We can help our customers pro-actively detect such events SITE Regression Testing and Proactive QoS/QoE Performance Monitoring Is a key for maintaining high level of customers satisfaction Page 7

8 Quality of Experience (QoE) Testing with SITE Tests, Monitors, and Benchmarks Telco classic services with service specific test cases and measurements/kpis on dedicated Samsung Galaxy X family Smartphones Tests and Monitors any App on both Android and ios with user defined test cases and measurements that reflect actual user interaction. The LU uses the App Experience Studio Page 8

9 Introducing the Hybrid CLU5 Robust hardware chassis for QoS and QoE testing with increased resiliency via virtualisation of the CLU Full support for monitoring network services in 2G/3G/4G such as Voice, SMS, Video and Data Speed, as well as OTT App Monitoring Capable of hosting two smartphones and two wireless modules Out-of-band power cycling of the smartphone via mechanical fingers Leverages existing SITE feature licenses and test cases Flexible Benchmarking enabled with Smartphone SIM Multiplexing LTE Advanced Pro throughput measurements Page 9

10 Hybrid CLU5 Specifications SIMMUX enabled Out-of-band control via mechanical fingers, for easier maintenance and support USB connector for communication with SITE and powering of the device Audio connector for Voice Quality Page 10

11 OTT vs IMS - Service Availability Service Success Rate (%) IMS OTTs ViLTE VoWiFi VoLTE RCS-MSRP SMSoIP Whatspp IM WhatsApp Voice Skype IM Skype Voice SkypeVide o Viber IM Viber Voice 3G Voice Sucess (%) 99,2 98,9 99,1 99,4 99, ,5 93,5 98,9 DropedCalls (number) 10,752 14,784 12,096 8,064 2,688 13,44 80,64 13,44 67,2 80,64 6,72 87,36 14,784 Page 11

12 QoE Call Setup vs Time to Ring CallSetup vs. Time to Ring Durations Instant Message Delivery Overall Call Setup Duration (s) Time to Ring Duration (s) Message E2E duration (s) 3G-Legacy 5,89 6,32 3G 5,3 Viber 3,41 5,61 Viber 1,44 Skype 1,85 3,62 Skype 1,06 WhatsApp 1,25 2,18 Whatspp 0,91 VoWiFi 1,80 2,60 SMSoIP 0,63 VoLTE 1,50 1,20 RCS-MSRP 0,43 Average P.863 POLQA MOS vs. Average Bitrate VoLTE (AMR-WB) 3,8 13 3G-Legacy (AMR) 3,6 Skype 3,2 48 WhatsApp 3,1 36 Viber 2,9 22 Page 12 Average MOS Average Kbps

13 Smartphone Network Basic Services Relevant VQ KPIs: Call Setup Time Call Time to Ring POLQA MOS A-->B, A<--B Codec used NB, WB or SWB Audio_Delay Voice Quality including HD CSFB Relevant CSFB KPIs: Overall_Call_Establishment_Durati on Overall_Time_To_Ring_Duration Return_to_LTE_duration IP_Timeout_Duration POLQA_MOS VoLTE Relevant VoLTE KPIs: Call_Establishment_Duration Time_To_Ring_Duration POLQA_MOS IMS and EPC Troubleshooting via: PCAP Page 13

14 Smartphone Data Speed Testing TCP Session SITE UI SITE CU SITE SIMMUX Supports LTE Advanced Pro with Samsung S9 integration up to Cat.18 2G/3G/4G Internet FTP Server FTP/HTTP/Speedtest Test Cases Smartphone_FTP/HTTP_DL/UL Smartphone_SpeedTest This test case initiates an FTP/HTTP UL/DL session This test case initiates a speedtest session KPIs: HTTP/FTP/Speedtest throughput in DL/UL Supported Network Types: GSM, UMTS, LTE Page 14

15 Multi RAB and E-RAB Testing on Smartphone Multi RAB and Multi E-RAB Active Data UL/DL Active Voice Call/SMS Internet FTP Server 2G/3G/4G SITE UI SITE CU Test Cases Smartphone_MutiRAB_Voice_MX_w_backgrd_FTP Smartphone_MutiRAB_SMS_MX_w_backgrd_FTP SITE SIMMUX This test case initiates an FTP UL/DL session, once established, it Originates/Terminates a Voice Call. This test case initiates an FTP UL/DL session, once established, it Originates/Terminates an SMS. KPIs: POLQA MOS (optional), FTP DL/UL Throughput, VoiceCall establishments durations, E2E_SMS_Durations Supported Network Types: GSM, UMTS, LTE Page 15

16 OTT QoE Services Testing Supported Apps and Use Cases WhatsApp messaging, voice calls and file share YouTube YouTube video playback via the YouTube app Facebook newsfeed access, status change, image sharing Instagram image sharing Viber Viber to Viber messaging, Viber to Viber calls, Viber to/from 2G/3G calls Twitter direct messaging DropBox access a folder, upload/download/delete a file Skype voice calls and text messaging Speedtest Ookla Speedtest App WeChat voice call, messaging, file share, post image/message Line voice call, messaging, file share Page 16

17 HCLU Example Reports Smartphone Voice Quality measurements HTTP Data measurements Speed Test Measurements Youtube Video Streaming OTT Services measurements Page 17

18 OTT Monitoring through Hybrid LU + Smartphones - Customized Customer s Application Customer Native Applications or specific desired applications can be developed to be monitored through SITE using our Hybrid LU with Real Smartphones Page 18

19 Mobile App Testing on Real Devices with App Experience Monitor Your Critical Applications Any transaction on any mobile app, hybrid app or website Real Android smartphones On any network Understand Your Customers Experience Exactly what the user experiences while using an app on a device Page 19

20 Introducing the App Experience CLU Create your unique customer flows Ad hoc testing anytime with one or more devices Easy point-and-click recorder to create your unique customer flows Validate success of each step via object, text and image Place timers to measure any step Single and multi-device test cases Page 20

21 Smartphone Interactive and Visual Scripting Users can easily create any test scenario by using the intuitive scripting workflow Object level scripting allows users to build scripts which work across different devices and OS versions Users can directly interact with the selected Smartphone Page 21

22 Example Customer Flow, Scripted on App Experience Example workflow of app user changing subscription plan via app: 1. Initialize Device 2. Launch App 3. Login to Account 4. Select My Plan and Features 5. Select Change Plan and Make Change 6. Submit Change and Review 7. Logout 8. Close App START TIMER STOP TIMER START TIMER KPI 1 STOP TIMER KPI 2 Page 22

23 Object-Level Scripting on App Experience Studio ios Devices Android Devices Easily and quickly build scripts that work across devices form factors and OS versions Scripts are robust and resilient; reduce script maintenance costs and time Faster script execution; test across more devices in less time Page 23

24 App Experience Report and KPIs Example NEW App Experience DB that Dinamicaly displays KPIs according to their definition in the Studio SpeedTest App Monitoring: AppLaunch Duration, UL/DL Speed, and RTT Page 24

25 Examples of QoE Testing and Monitoring Use Cases Social Media Apps Facebook, Instagram, etc. Mobile Browsing Chrome, ios Safari, etc. OTT App Messaging / Voice WhatsApp, Skype, Line, Viber, WeChat, etc. Live/VoD Streaming Apps: YouTube, ClaroVideo, TIMvision, Entertain to go, HOOQ, go90, Mobile TV apps, embms apps, etc. (excluding video play & quality) Music Streaming Apps Spotify, apple music, google music, Pandora, etc. Online Shopping Apps Amazon, Ebay, etc. Smartphone Apps Gmail, ios mail, etc. Smartphone Location based services Google Maps, Apple Maps, etc. Smartphone Specific Firmware Network Settings Assurance call forwarding, Internet APN, VoLTE IMS APN, etc. And more Page 25

26 QoE Testing on GlobalRoamer Outbound Roaming Multiplex your SIMs in a smartphone located in Nuremberg Germany or San Mateo USA Execute Roaming calls from and to any GlobalRoamer location in the world or between two roamers in Germany or USA Test Voice, SMS, Data and OTT Apps with a real Android Smartphone GlobalRoamer San Mateo Nuremberg Samsung S6, S7 Page 26

27 Success Story Complete QoS/QoE Testing Suite A Tier-1 operator in GCC monitors QoE using SIGOS SITE with 23 Hybrid WLU5s spread across 6 different cities covering airports, shopping centers, theme parks and touristic places. Combined with SITE native interfaces in the same Hybrid LU5, the operator is capable of testing the customer s journey from welcome SMS to subscribing to different bundles to QoS monitoring for Voice, SMS and Data to QoE monitoring for OTT Apps including messaging, File/Image sharing, Data throughput and Video streaming. 23 Hybrid WLU5s with 1x Smartphone, 1x LTE/VoLTE and 1x2G/3G interface Page 27

28 Outlook - Smartphone Testing with SITE Enhancements At the moment SITE offers two solutions for Smartphone Testing with Telco customers: Hybrid CLU AppExperience LU Each of the solutions has its own advantages and limitations The key features required for Telco: Voice Quality ( available on HCLU) SIMMUX (available on HCLU) Interactive recording / scripting (available on AppEXLU) To address these requirements a New Solution for Telco will be developed, which includes the features above in one box first phase planned for June 2019 Page 28

29 Outlook - Smartphone Testing with SITE Enhancements Features BLUE LU - Basic Grade BLUE LU Telco Grade Integration of new device for Testing 4 weeks 4 weeks Integration of new device for Monitoring 1-2 months (if new OS) 1-2 months (if new OS) Testcase Recorder Yes Yes Flagship Devices Supported All Major Vendors Samsung Galaxy and Apple iphone Smartphone Diversity Yes Limited Android Support Yes Yes ios Support Yes Yes SITE Reporting and Alarming Yes Yes Interactive Audio Yes Yes Voice Quality No Yes Video Quality No Yes SIMMUX/SIM-Switch No Yes SITE OTT and Standard Testcases* No TBC SITE Interfaces* No TBC *Final feature set to be decided Page 29

30 Use Case Overview Use Case Application Testing Enterprise Blue LU Telco OTT Application Testing Telco Service Testing OTT Service Testing 5G Service Testing Specific Features Interactive Recorder SIMMUX Voice Quality Page 30

31 Time to Start is a mandate now, tomorrow and in the future helps to monitor all network basic services along with OTT Apps including network KPIs like data throughput, voice quality, video streaming quality and messaging E2E duration is a unique solution to monitor any App on Android and ios smartphones Guaranteed high level of is the key for a distinguished top service provider is your tailored tool for maintaining high level of quality all the time so save the time and get in touch with us Page 31

32 Thank you Page 32