TAKING NETWORK TESTING TO THE NEXT LEVEL

Transcription

1 TAKING NETWORK TESTING TO THE NEXT LEVEL

2 WELCOME TO THE NEXT LEVEL OF NETWORK TESTING. Do you understand the performance and customer experience of your mobile network? P3 does. Our holistic approach is the first in the world combining drive- and walk-testing with crowdsourced data of countless billions of real customer experiences which means we match in-depth network quality and performance analysis with real end-user experience. Today, more than 180 mobile networks in more than 80 countries are being 2 evaluated by our unique scoring methodology, the P3 Score. With crowdsourcing we are covering 465 networks in 147 countries. For more than a decade we have been measuring network quality, identifying potential areas for network improvement and supporting the optimization process. Our comprehensive way of finding solutions helps our clients to achieve the best performance and the highest quality.

3 P3 SCORE IS THE DE-FACTO INDUSTRY STANDARD IN MORE THAN 80 COUNTRIES WORLDWIDE TELECOMMUNICATION COMPANIES WORLDWIDE RELY ON P3 S EXPERTISE OUR GLOBAL FRAMEWORK IS HIGHLY TRANSPARENT AND IS ACKNOWLEDGED AS THE BEST IN THE INDUSTRY INTEGRATED IN 800+ APPLICATIONS WORLDWIDE 3

4 CROWDSOURCING END USERS MEASURE THEIR EXPERIENCE 24/7 Crowdsourcing is a significant addition to classical methods of collecting network performance data. Our app-based, on-device data collection engine is working passively in the background of a user s smartphone. With this app-based approach, end-user smartphones become measurement devices delivering invaluable knowledge on how customers use and experience mobile communications, both, on mobile networks as well as on Wi-Fi networks. Our analyses are therefore based on proprietary crowdsourced data, presenting a holistic view across individual voice calls, data and app usage and the related technical network parameters. P3 can access the data of countless millions of real customer experiences worldwide a footprint that is unrivalled. P3 CROWDSOURCING The data collected is anonymous No personal identifiers, i.e. MSISDN, IMSI, IMEI or IP addresses, are collected Data cannot be manipulated by tested networks Passive approach has higher sample rate compared to traditional speed test solutions No user interaction with measuring component is necessary P3 framework can be integrated into any application (data is gathered from more than 800 worldwide) Overhead of artificial data is removed Data collection is strongly tied to explicit user consent 4

5 CROWDSOURCING CATEGORIES All metrics are determined in a full benchmarking scenario, i.e. we ensure that the reported metrics are collected in fully overlapping areas for all operators. This does not mean that they are only collected where all operators have coverage, but where crowd panelists from all operators have been. This fully overlapping area is also called the tested area. COVERAGE FOR VOICE Assesses the percent of the tested area where available network coverage was sufficient for a voice call and the quality of that coverage. USER DOWNLOAD SPEED Assesses which of the data-consuming apps that a panel user actively uses create the highest data throughputs and create spatial statistics on it. COVERAGE FOR DATA Assesses the percent of the tested area where available network coverage was sufficient for a good data experience (i.e. 3G or 4G) and the reliability of that coverage. OPERATIONAL EXCELLENCE Our crowd apps regularly perform a mini-connectivity test to check for availability of the mobile data service. Our sophisticated statistics allows to determine network wide anomalies. We are able to analyze the duration of service degradations which indicates how excellent operators perform their network operations. COVERAGE FOR 4G Assesses the percent of the tested area where 4G network coverage was available and the reliability of that coverage. 5

6 COMBINED ANALYTICS HOLISTIC APPROACH Our crowdsourced data complements the drive- and walk-testing activities and adds important dimensions (e.g. time, geography or variety in devices and tariff plans), which cannot be covered with reasonable effort using traditional testing methodologies. It enables us to gain market intelligence beyond the aspect of network performance, such as user behavior, network properties and features, usage and performance of over-thetop services, infrastructure and device performance, security and more. ANALYTICS UNPRECEDENTED LEVEL OF DETAIL With drive testing in more than 80 countries and worldwide crowdsourcing, our engineers are working on measurement solutions every day. We have gained extensive experience in collecting, storing and analyzing huge amounts of data. By analyzing data according to predefined KPIs, we can provide information for the optimization of networks and also show if networks live up to expectation. Our data scientists can drill down project-specific data sets and analyze them on a global scale. TRUE DATA ANALYTICS EXPERTS All collected data are transferred to the P3 Big Data Platform and processed continuously. Automated aggregation with P3-developed in-house software provides flexibility. Experienced engineers interpret the data according to the P3 Score ensuring high quality at all times. In constant exchange with the industry, the test evaluation algorithms are updated every year to ensure they continue to reflect both the latest technology and the way mobile services are consumed by end users. All of this has led to a wide acceptance of our tests and analytics in the industry. We are a trusted partner for all market players who need detailed insights into the wireless market, including analysts, utilities, mobility services, corporate users of wireless networks, infrastructure vendors and others. 6

7 It s one thing to gather billions of data sets, but even more important to have the skills to derive insights from them. We do both. 7

8 UNIQUE METHODOLOGY We combine in-depth knowledge of network performance with insights into real end-user usage patterns and data analytics expertise and skills. A unique approach that is a clear benefit to all market players who need detailed insights into mobile performance drivers. DATA P3 SCORE COMPOSITION VOICE 51 % 34 % P3 is industry first in combining drive testing and crowdsourcing in a single benchmarking framework. Web Page Download Success Ra o Youtube Quality File Up- and Download Coverage for 4G Coverage for Data Call Setup Time Call Success Ra o Speech Quality User download speeds Coverage for Voice COMBINING DRIVE-TESTING, WALK-TESTING AND CROWDSOURCING We measure smartphone voice and data performance based on extensive drive-tests from major metropolitan areas to smaller cities, roads and railways, complemented by walk-tests in big cities with special focus on busy areas, on public transport and in frequently visited buildings, like shopping malls, rail stations, etc.. We objectively define the routes and test methodology and publish the results through certificates or public benchmark reports. To complete the network performance picture, we access the most comprehensive set of global wireless market intelligence data provided by more than a constantly growing 800 apps. We look at more than 100 KPIs including long-term service availability, network coverage and data speeds. This approach reflects the high importance of smartphone data service just as well as the users requirement for coverage and availability of their networks for any use case. Opera onal Excellence CROWD 15 % P3 SCORE As the de-facto industry standard, our benchmarking methodology focuses on customer-perceived network quality and covers a wide range of mobile services. It allows a technical analysis that is unprecedented in its level of detail and enables comparisons between the network performance and capability of each mobile network. Our public benchmarks as well as the certification benchmarks help network operators to demonstrate how well they are delivering wireless connections to consumers, business users and enterprises and reveals the areas of improvement. 8

9 DRIVETEST WALKTEST CROWD XXX DATA XXX VOICE CROWD ROADS DATA VOICE CROWD SCORE BREAKDOWN Ci es - Drivetest TOWNS Ci es - Walktest XXX Towns - Drivetest DATA VOICE CROWD DATA VOICE CROWD Roads - Drivetest CITIES TRAINS Train - Walktest Crowdsourcing Total The weight of voice tests in the total result is 34%, the weight of data service tests is 51% while the crowd-based evaluation contributes the remaining 15% of the overall 1000 points. DATA Browsing the most popular websites Uploading and downloading social media Transferring files (evaluated on access success rate, stability and the data transfer rate) Playing YouTube videos (reporting interruptions, buffering times and video quality) VOICE Assessing smartphone-to-smartphone calls (including background data traffic in parallel) Evaluating success of the call setup, call setup time and user triggered termination Evaluating voice quality (perceived over the mobile connection) DRIVE TEST Two or more cars In cities, small towns and on connecting roads Cars are equipped with arrays of selected measurement devices Devices call counterpart in another car Two main categories which are respectively, smartphone voice and data services. WALK TEST Walk-test teams visit larger cities, take the trains between cities and into more rural areas Backpacks and/or trolleys equipped with arrays of selected measurement devices Devices call stationary-receivers Crowdsourcing The P3 crowd score is an autonomous component of the P3 Score and covers the crowdbased assessment of coverage for 4G, coverage for data, coverage for voice, user download spreeds, quality operational excellence. 9

10 CONTACT Hakan Ekmen CEO, P3 communications GmbH P3 communications GmbH Am Kraftversorgungsturm Aachen Germany info@p3-group.com