Measurements and QoS Analysis of Live-World Mobile Telecommunication Networks

Size: px

Start display at page:

Download "Measurements and QoS Analysis of Live-World Mobile Telecommunication Networks"

Lynette Wilcox
5 years ago
Views:

1 Measurements and QoS Analysis of Live-World Mobile Telecommunication Networks Charalampos N. Pitas, Eleni D. Avgeri, Marilena P. Kallenou, and Philip Constantinou Mobile Radiocommunications Laboratory School of Electrical and Computer Engineering, National Technical University of Athens {chpitas, lena, marilena, fkonst}@mobile.ntua.gr Abstract The scope of this paper is to study the performance of contemporary live-world mobile telecommunication networks. The survey is focused on High Speed Packet Access technology which is based on Wideband CDMA. The measurement platform allows QoS assessment, radio access networks audit and benchmarking. Specifically, services are evaluated via extensive mobile and static measurements campaigns in both indoor and various outdoor environments. Emphasis is given to QoS categorization and characterization. Accessibility, retainability and integrity are thoroughly examined by Key Performance Indicators. Enhanced drill down analysis, measurements filtering and reporting is executed on a spatial relational database management system. Index Terms Mobile Telecommunication Systems (GSM / GPRS / UMTS / HSPA), Quality of Service (QoS), Key Performance Indicators (KPIs). I. INTRODUCTION Nowadays 3rd generation wireless cellular networks have already met a great success and have been spread to several new services. Some of these services require higher bit rates, reduced latency, and increased spectral efficiency. High Speed Packet Access represents a new high-speed data transfer feature released by the 3rd Generation Partnership Project (3GPP) with the aim of empower UMTS downlink and uplink data rates and improve support of high dada-rate packet-switched services. Commercial mobile telecommuni-cation networks in Greece have already introduced HSPA technology in coexistence with UMTS R.99 and GSM/GPRS radio access networks. GSM GPRS WCDMA R.99 HSDPA Release 5 Fig. 1 Mobile Telecommunication Systems evolution up to 3.5G. HSUPA Release 6 At the beginning of this paper, SECTION II, a short introduction of the HSPA technology is presented with new channel types details. Then, in SECTION III, QoS KPIs (Key Performance Indicators) are defined for network evaluation. Afterwards, in SECTION IV, the measurement equipment is presented as well as the tests which are carried out. Analysis and measurements post-processing follow in Section V and finally, SECTION VI, several conclusions are exposed. II. HIGH SPEED PACKET ACCESS NETWORKS. HSPA, which is known as 3.5G, is composed of two technologies, High Speed Downlink Packet Access (HSDPA) and High Speed Uplink Packet Access (HSUPA) [1]. Services of 3G wireless cellular networks, such as video telephony, voice over IP (VoIP), gaming, file downloading and video/picture sharing, require different Quality of Service (QoS) with respect to data rate, delay, error rate, capacity and coverage. HSDPA is targeting increased capacity, reduced round trip delay, higher peak data rates up to 10 Mbps, higher user/cell throughput and lower latency. For this scope, the 3GPP has developed the HSDPA technology, an evolution of UMTS based on Wideband Code Division Multiple Access (WCDMA). It is important to note that HSDPA is a pure access evolution without any core networks impacts, except for minor changes due to the higher bandwidth access. For example, in the 3GPP Rel. 5 the maximum throughput set into the signaling protocol has been increased from 2 Mbps to 16 Mbps in order to support the theoretical maximum limit of HSDPA data rate (14.4 Mbps). It follows that the deployment of HSDPA is very cost effective since the incremental cost is mainly due to Node B and Radio Network Controller (RNC) hardware/software upgrade while the operator cost to provide data services is significantly reduced. This is due to the high improvements in spectral efficiency introduced by HSDPA. In order to achieve these goals, HSDPA technology has introduced new channels and new techniques. HSDPA optimizes downlink performance. The HSDPA functionalities define three new channel types: High-Speed Downlink Shared Channel (HS-DSCH), High-Speed Shared Control Channel (HS-SCCH) and High-Speed Dedicated Physical Control Channel (HS-DPCCH). Three new features of HSDPA are Adaptive Modulation and Coding (AMC), Hybrid Automatic Repeat Request (HARQ) and Fast Scheduling. High Speed Uplink Packet Access (HSUPA) uses the Enhanced Dedicated Channel (E-DCH) and constitutes a set of improvements that optimize uplink performance. Among

2 these improvements are higher throughputs, reduced latency, and increased spectral efficiency. HSUPA is standardized in Release 6. Performance gains of HSUPA are achieved through the following approaches an enhanced dedicated physical channel, a short TTI, as low as 2 ms, which allows faster responses to changing radio conditions and error conditions, fast Node-B-based scheduling, which allows the base station to efficiently allocate radio resources and fast Hybrid ARQ, which improves the efficiency of error processing. Theoritical peak data rate Typical downlink peak data rate Typical uplink peak data rate TABLE I 2G TO 3G SUPPORTED BIT RATES EVOLUTION UMTS GSM GPRS HSPA (R.99) 7.2 Mbps Mbps Up to Kbps Kbps 14.4Mbps 9.6 Kbps 9.6 Kbps 40 Kbps 384 Kbps 5.8 Mbps 20 Kbps 64 Kbps 2Mbps III. QUALITY OF SERVICE KEY PERFORMANCE INDICATORS According to ETSI s standards [2] QoS KPIs can be categorized in five phases during service use from the customer's point of view: 1. Network Availability: Probability that the mobile services are offered to a user. 2. Network Accessibility: Probability that the user performs a successful registration on the PLMN. 3. Service Accessibility: If the user wants to use a service, the network operator should provide him as fast as possible access to the service. Includes KIP s that describe how successful is the service access (call setup rate) as well as KPI s that describe setup time (call setup time). 4. Service Integrity: This describes the Quality of Service during service use. Includes KPI s that describe the quality of the service (MOS for voice quality). 5. Service Retainability: Service retainability describes the termination of services (in accordance with or against the will of the user).includes KPI s that count the time that a service can retain (Drop Call Rate or call holding time). Analysis is concentrated on QoS for PS data. Crucial KPIs are comprehensively studied: A. Accessibility 1. PDP Context Activation Duration and Success Rate: indicates the time required for the service access as well as the success rate. 2. Setup Time: The interval of time from the moment the user service requests until the first byte is sent or received. The KPI does not include the duration of PDP Context activation. 3. Setup Failure Rate: Tests percentage that failed at the setup phase. B. Retainability 1. Download Time: The interval of time between the first and the last byte. 2. RTT (Round Trip Time): The interval of time between a packet is sent and received. 3. Drop Rate: Tests percentage that failed and had already started normally. C. Integrity 1. Throughput: The sum of bytes that had been sent or received to the duration it took. 2. Overall Service Success: Tests percentage that have been successfully completed. It is calculated by excluding from the sum of tests the ones that failed either at the setup phase either they dropped. Furthermore, voice quality is of the first consideration for packet switched mobile telephony. Speech Quality indicator is the major factor determining customer satisfaction, since it represents the quality of listener s reception. It is generally expressed as a Mean Opinion Score (MOS) which is valued between 1 (bad level) and 5 (excellent level). Hereupon speech quality is measured and assessed by relative algorithm as well as statistics are presented. IV. MEASUREMENT SYSTEM A. System Configuration The measurements and analysis have been conducted using SwissQual Diversity equipment. Control & Analysis Backend Probes Voice Server Test Manager CS Network Mobile Network Diversity Benchmarker Fig. 2. Measurement System Architecture NQDI IP Network SQL Server Media Server Diversity is a portable, highly scalable platform for drivetest systems that allows the user to measure more than one cellular network of different technology. The system is mobile

in automotive setting or carried by a pedestrian. The interface to the mobile network is through special models of cellular phones that give access to internal network parameters [4],[5].

using Media Server. Diversity provides QoS evaluation, benchmarking, and RF and network optimization. It is designed for both wireless and fixed networks.

Tests execution is configured and controlled by Test Manager and also measurements data can be displayed live on NQView.

3 in automotive setting or carried by a pedestrian. The interface to the mobile network is through special models of cellular phones that give access to internal network parameters [4],[5]. The measurement system sets up calls, records cellular network parameters, measures voice quality calls using Voice Server as well as the quality of other payload such as data, , ping and video using Media Server. Diversity provides QoS evaluation, benchmarking, and RF and network optimization. It is designed for both wireless and fixed networks. The quality of service in voice and data tests is performed through related speed and audio quality algorithms. Tests execution is configured and controlled by Test Manager and also measurements data can be displayed live on NQView. Test Manager and NQView are software applications running on the laptop during the measurements. In this scenario, laptop user in the vehicle monitors the process of measurements. B. Measurement Campaign The measurement system is placed in a car while executing a measurement campaign [6]. In the campaign a session of tests is consecutively repeated. The data session contains the following actions and tests: 1. Activation Packet Data Protocol (PDP Context Activation) 2. Reception of 10 MB file (FTP Download) from a public server broadband connected to the internet. 3. Sent a 3 MB file to the public server (FTP Upload). 4. Web browsing at NTUA ( website of size of 1 MB Ping with 400 Bytes Packet Size. 6. Deactivation Packet Data Protocol (PDP Context Deactivation). The session of voice tests contains both mobile phone to mobile phone test as well as mobile to PSTN phone test. In both cases A-side and B-side are defined and exchange short voice recordings alternately in order to emulate a call conversation. The conversation schema is presented at Fig.2. Fig. 4 Quality of voice measurement tasks configuration. V. ANALYSIS POST PROCESSING RESULTS A. Analysis Tool Measurements are imported in the post processing application based on SwissQual s NQDI (NetQual Database Interface) which is connected with a relational time-spatial database. NQDI offers enhanced drill down analysis, filtering and data mining, KPI statistics, map plotting, database querying/mining and reporting. Using NQDI the measurement data can be filtered by using complicated queries in order to analyze specific part of all the data located in the database. A typical structure of a query based filter applied in the data base aims to show up performance parameter values and compute statistics from a set of well organized measurements or/and network information tables where crucial criteria and combined conditions take place. A general query form is presented: SELECT <List_of_Parameters>, <Statistics_Functions> FROM <List_of_Measurements_Tables> OR/AND <List_of_Radio_Network_Parameters> WHERE <List_of_Combined_Criteria/Conditions> The QoS statistics analysis is focused on significant KPIs, especially, achieved download and upload throughput, overall http success rate and round-trip-time [7]. Fig. 3 Measurements Scenario

In Table II, a statistics analysis of the achieved

A percentage of 42.59% is remarked between 1.

71% is provided up to 7.2Mbps. It is obvious that 14.

Download throughput statistics functions are plotted

It is remarked that only 8,33% of download

the maximum typical throughput for HSDPA 7.

2Mbps is applied in the network even though 3.

FUNCTION PDF CDF TABLE II FTP DOWNLOAD THROUGHPUT

3600<DL<=7200 28,70% 1800<DL<=3600 42,59% 384<DL<=1800

1800<DL<=3600 71,30% 384<DL<=1800 91,67% DL<=384

achieved upload bitrates is presented in details.

19% is remarked between 384kbps and 2Mbps.

and is traced in the majority of the measurement

Besides, 21,02% of the relevant samples are detected

So, broadband access is well supported for demanding

TABLE III FTP UPLOAD THROUGHPUT STATISTICS FUNCTION UL

4 In Table II, a statistics analysis of the achieved download bitrates is presented in details. A percentage of 42.59% is remarked between 1.8Mbps and 3.6Mbps when a worthy 28.71% is provided up to 7.2Mbps. It is obvious that 14.4Mbps technology has not introduced yet. Download throughput statistics functions are plotted at the following figure. It is remarked that only 8,33% of download measurements are below 384kbps while 91,67% are fine supported by HSDPA. Fig. 6. FTP Download Throughput PDF and CDF graph. Fig. 5. Analysis process diagram. B. Results 1) Packet Switched Services It is known that the maximum typical throughput for HSDPA 7.2Mbps technology is 5.8Mbps while for HSDPA 3.6Mbps technology is 3Mbps. As it is arisen from the following diagram, HSDPA 7.2Mbps is applied in the network even though 3.6Mbps technology is used in more samples. FUNCTION PDF CDF TABLE II FTP DOWNLOAD THROUGHPUT STATISTICS DL throughput (kbps) PERCENTAGE >7200 0,00% 3600<DL<= ,70% 1800<DL<= ,59% 384<DL<= ,37% DL<=384 8,33% >7200 0,00% 3600<DL<= ,70% 1800<DL<= ,30% 384<DL<= ,67% DL<= ,00% In Table III, a statistics analysis of the achieved upload bitrates is presented in details. A worthy percentage of 77.19% is remarked between 384kbps and 2Mbps. It is clear that HSUPA technology is well introduced and is traced in the majority of the measurement samples. Besides, 21,02% of the relevant samples are detected between 128kbps and 384kbps. So, broadband access is well supported for demanding upload use. TABLE III FTP UPLOAD THROUGHPUT STATISTICS FUNCTION UL throughput (kbps) PERCENTAGE >2000 0,00% 384<UL<= ,19% PDF 128<UL<=384 21,02% 64<UL<=128 1,66% UL<=64 0,13% >2000 0,00% 384<UL<= ,19% CDF 128<UL<=384 98,21% 64<UL<=128 99,87% UL<=64 100,00% Upload throughput statistics functions are plotted at the following figure. It is mentioned that 98,21% of the upload measurement samples are located above 128kbps and up to 2Mbps owing to HSUPA technology. Less than 2% of the samples are lower than 128kbps.

following diagram (Fig.8) we present a total time distribution of each technology. Fig. 7. FTP Upload Throughput PDF and CDF graph.

In the diagram, the average and minimum download time is presented, as well as the Overall Service Success, tests that have successfully completed without drop or setup failure.

2) Circuit Switched Services Telephony Speech Quality indicator is the major factor determining customer satisfaction, since it represents the quality of listener s reception.

Most objective algorithms [8][9][10] are based on a comparison between a reference sample and a coded version of the reference. The most widely used models of algorithms are ITU-T P.

862 algorithm are displayed in the following figures. Fig. 8. Total HTTP session time and Success Rate.

5 following diagram (Fig.8) we present a total time distribution of each technology. Fig. 7. FTP Upload Throughput PDF and CDF graph. Download time is used to evaluate HTTP performance, which is considered to be more efficient than throughput. In the diagram, the average and minimum download time is presented, as well as the Overall Service Success, tests that have successfully completed without drop or setup failure. The average download time 7,1sec is capable for loading a website of 1 MB. Fig. 9. Data Serving Technology distribution. 2) Circuit Switched Services Telephony Speech Quality indicator is the major factor determining customer satisfaction, since it represents the quality of listener s reception. It is generally expressed as a Mean Opinion Score (MOS) which is valued between 1, bad level, and 5, excellent level. Most objective algorithms [8][9][10] are based on a comparison between a reference sample and a coded version of the reference. The most widely used models of algorithms are ITU-T P.862 standardized in In contrast with subjective experiments, this model enables extensive testing to be performed over short periods. The results of ITU-T P.862 algorithm are displayed in the following figures. Fig. 8. Total HTTP session time and Success Rate. Based on the results of the Round Trip Time KPI and the following table, the network performance considered to be very good with average RTT 194msec. TABLE III PING RTT STATISTICS Count 528 % 99,62% Avg [ms] 194 Min [ms] 101 Max [ms] 3454 StDev [ms] % Cut-Off 345 For each test, Diversity stored the duration of the provided data service technology. Combining all types of tests, in the Fig. 10 CDF of Listening Quality (P.862.1) Fig. 11 Listening Quality distribution. 3) KPIs Map Plotting Finally, Fig.12 and Fig.13 display crucial QoS parameters

Consequently, it is necessary and obligatory (by independent national regulatory authority [11]) for mobile operators to perform continuously measurements so as to audit and optimize the efficiency

6 Channel Quality Indicator and Listening Quality - having applied a spatial filter on a polygon which contains a route of the NTUA Campus. The result of the spatial query has been plotted on a GIS Viewer (Google Earth) giving us an opportunity to locate problematic areas regarding performance or RF coverage. Consequently, it is necessary and obligatory (by independent national regulatory authority [11]) for mobile operators to perform continuously measurements so as to audit and optimize the efficiency of radio access networks as well as for benchmarking reasons. Open and frequent publication of crucial KPIs will lead mobile operators to robust competition introducing innovative and qualitative services. Contemporary technologies like Long Term Evolution UMTS, also known as HSPA+, will explicitly uplift the provided bitrates. Future work may be performing large scale measurements campaigns in differentiated environments of interest in order to extensively examine QoS parameters. Fig. 12 Channel Quality Indicator (CQI) plotted on Google Earth Fig. 13 Listening Quality (LQ P.862.1) plotted on Google Earth. VI. CONCLUSIONS REFERENCES [1] H.Holma and A.Toskala, HSDPA/HSUPA FOR UMTS, High Speed Radio Access for Mobile Communications, Ed. John Wiley & Sons, LTD, [2] ETSI TS , Speech Processing, Transmission and Quality networks; Part 1: Identification of Quality of Service aspects, V1.2.1, Technical Specification, [3] ETSI TS , Speech Processing, Transmission and Quality networks; Part 2: Definition of Quality of Service parameters and their computation, V1.6.2, Technical Specification, [4] ETSI TS , "Speech Processing, Transmission and Quality networks; Part 3: Typical procedures for Quality of Service measurement equipment", V1.3.2, Technical Specification, [5] ETSI TS , Speech Processing, Transmission and Quality networks; Part 4: Requirements for Quality of Service measurement equipment, V1.2.1, Technical Specification, [6] ETSI TS , Speech Processing, Transmission and Quality networks; Part 5: Definition of typical measurement profile, V1.5.1, Technical Specification, [7] ETSI TS , "Speech Processing, Transmission and Quality networks; Part 6: Post processing and statistical methods", V1.2.1, Technical Specification, [8] ITU-T Rec. P.862, Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrowband telephone networks and speech codecs, 2/2001. [9] ITU-T Rec. P.862.1, Mapping function for transforming P.862 raw result scores to MOS-LQO, 11/2003. [10] ITU-T Rec. P.862.2, Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs, 11/2007. [11] Hellenic Telecommunications and Post Commission (EETT), Desicion, No.480/017, "Definition of quality indicators of public provided electronic communication services and specification of the content and the form of the reported information as well as the method and the time of the publication by the electronic communication services providers", Official Government Gazette of Hellenic Republic, OGG/1153/B/2008, In this paper, it is concluded by performance measurements that nowadays mobile telecommunication networks meet a great evolution, integration and interoperability in Greece. Mobile broadband access supports ubiquitously QoS demanding services which are well described by KPIs.

NetQual NQDI Classic TRANSFORM DATA INTO INSIGHTS TO SUPPORT BUSINESS DECISIONS

NetQual NQDI Classic TRANSFORM DATA INTO INSIGHTS TO SUPPORT BUSINESS DECISIONS 1 NETQUAL NQDI CLASSIC Transform Data into Insights to Support Business Decisions Today s increasing network complexity requires