Analyzing Throughput and Stability in Cellular Networks

Transcription

1 Analyzing Throughput and Stability in Cellular Networks Ermias Walelgne 1 Jukka Manner 1 Vaibhav Bajpai 2 Jörg Ott 2 April 25, 2018 NOMS 18, Taipei 1 (ermias.walelgne jukka.manner)@aalto.fi,aalto University, Finland 2 (bajpaiv ott)@in.tum.de, Technical University of Munich, Germany April 25, 2018 NOMS 18, Taipei

2 Introduction

3 Introduction Introduction Motivation Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 1 / 20

4 Introduction Introduction Motivation Mobile-broadband subscriptions have grown more than 20% annually in the last five years (ITU 2017) [1] Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 1 / 20

5 Introduction Introduction Motivation Mobile-broadband subscriptions have grown more than 20% annually in the last five years (ITU 2017) [1] Smartphones (in 2017) have 70% share of the total market of all device types (IDC) [2]. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 1 / 20

6 Introduction Introduction Motivation Mobile-broadband subscriptions have grown more than 20% annually in the last five years (ITU 2017) [1] Smartphones (in 2017) have 70% share of the total market of all device types (IDC) [2]. Quality & performance of the cellular network depends: radio technology, limitations of device hardware wireless link characteristics (e.g interference, fading, etc.) mobility, location and time of the day Infrastructure of Mobile Network Operators (MNO) Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 1 / 20

7 Introduction Introduction Motivation Mobile-broadband subscriptions have grown more than 20% annually in the last five years (ITU 2017) [1] Smartphones (in 2017) have 70% share of the total market of all device types (IDC) [2]. Quality & performance of the cellular network depends: radio technology, limitations of device hardware wireless link characteristics (e.g interference, fading, etc.) mobility, location and time of the day Infrastructure of Mobile Network Operators (MNO) About 30% of cellular measurements from netradar [3] experience sudden drops to zero bitrate for( 200 ms). Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 1 / 20

8 Introduction Introduction Research Question What are the factors affecting the throughput and stability of cellular networks? Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 2 / 20

9 Introduction Introduction Contribution Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 3 / 20

10 Introduction Introduction Contribution 1 Time of a day and the location affect the throughput: Throughput drops during peak hours in cities as congestion increases. Radio technology switches from legacy (UMTS) to advanced (HSDPA) during the course of a day. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 3 / 20

11 Introduction Introduction Contribution 1 Time of a day and the location affect the throughput: Throughput drops during peak hours in cities as congestion increases. Radio technology switches from legacy (UMTS) to advanced (HSDPA) during the course of a day. 2 Predict the probability sudden drops in bit rate in a cellular network with 90% accuracy relying on easily accessible information (e.g device model, location, network technology). Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 3 / 20

12 Introduction Introduction Contribution 1 Time of a day and the location affect the throughput: Throughput drops during peak hours in cities as congestion increases. Radio technology switches from legacy (UMTS) to advanced (HSDPA) during the course of a day. 2 Predict the probability sudden drops in bit rate in a cellular network with 90% accuracy relying on easily accessible information (e.g device model, location, network technology). 3 Predict the average TCP download speed based on the first 5 seconds of median bit rate value of throughput. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 3 / 20

13 Methodology

14 Methodology Time (Sec.) Methodology Measurement platform TCP downlik speed (Mbps) It measures & collects information including throughput (TCP), signal strength, radio technology type, RTT( UDP ) etc. towards Amazon Cloud instances & Aalto University servers. The measurement server tests the download speed by sending a random data over TCP for 10 seconds. During the measurement session, both the client and the server record the number of bytes transferred every 50 ms. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 4 / 20

15 Methodology Methodology Data Set and Measurement Trials Based on a longitudinal dataset collected using the netradar measurement platform [3]. Focused on stationary nodes during a ten second measurement session to minimize the variability that might arise from mobility. A year-long measurement data ( 750K ) from 3 Finnish MNO. Network Total # Of LTE HSPA HSPA+ HSDPA UMTS Others Operator Measurements Elisa 373K 45.75% 8.49 % 14.70% 1.12% 25.43% 5% DNA 235K 63.30% 7.52% 8.17% 10.69% 0.9% 9.42% TeliaSonera 140K 34.74% 1.04% 14.27% 1.29% 33.60% 15.06% Table: Measurement Distribution per radio Technology for each MNO Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 5 / 20

16 Data Analysis

17 Data Analysis Data Analysis Device Model EDGE GPRS HSDPA HSPA HSPA+ LTE UMTS Download speed (Mbps) Release year of the device Figure 1: TCP download speed of different device models per network technology. The release year of the device models does not correlate to TCP download speed. It is not always the newest device model whose TCP download performance is best. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 6 / 20

18 Data Analysis Data Analysis Mobile Network Operator CDF DNA Elisa TeliaSonera TCP Download Average (MBps) CDF DNA Elisa TeliaSonera TCP Upload Average (MBps) Figure 2: Mean TCP throughput for LTE networks of 3 MNO downlink (left) and uplink (right) speed. Clear variation between MNOs on mean uploading speed for LTE. MNO s are significant for network performance variation. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 7 / 20

19 Data Analysis Data Analysis Subscribers Location Elisa (Network Operator With LTE) DNA (Network Operator With LTE) Sonera (Network Operator With LTE) 40 Mbps 110 Mbps 90 Mbps 35 Mbps 30 Mbps 100 Mbps 90 Mbps 80 Mbps 80 Mbps 70 Mbps 25 Mbps 70 Mbps 60 Mbps 60 Mbps 50 Mbps 20 Mbps 50 Mbps 40 Mbps 15 Mbps 10 Mbps 40 Mbps 30 Mbps 20 Mbps 10 Mbps 30 Mbps 20 Mbps 10 Mbps Figure 3: Mean TCP throughput distribution by area in Finland for LTE networks of three MNOs: Elisa (left), DNA (middle), TeliaSonera (right). The comparison across MNOs shows a large variation in throughput per locations. Users in a metropolitan area & (subscribed to DNA or Elisa) get better throughput than urban areas. Better infrastructure provisioning base station density sufficient core network capacity. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 8 / 20

20 Data Analysis Data Analysis Radio Technology Changes and Time of Day Handover frequency 6% 4% 2% 0% 8% 6% 4% 2% 0% Time of a day HSPA to HSPA+ UMTS to HSDPA Figure 4: Frequency of radio technology switches over time of day, for the TeliaSonera network (similar to other MNOs). The occurrence of switches (from legacy to more advanced technology e.g UMTS to HSDPA) increases during peak hours. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 9 / 20

21 Data Analysis Data Analysis Network Stability 5 TCP downlik speed (Mbps) Time (Sec.) Classified the data into two groups: dropped: measurement sessions that experience a sudden dropout >200 ms. non-dropped: sessions without this phenomenon. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 10 / 20

22 (Mbps) Data Analysis Data Analysis Received Data per Recording Interval After sudden dropout Before sudden dropout 'O HSPA Q) 175 Q) o.tll tll LTE 'O <11:S: o- 75 D UMTS rl l l Download oa s speed Sudden dropout duration (sec) Figure 5: TCP maximum download rate observed before and after a sudden dropout of a certain duration ( 200 ms) per radio technology. A sudden dropout duration that stayed for at least 200 ms does have an impact on download bit rate. The impact is visible especially after the dropout period is over (left side of the figure). Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 11 / 20

23 Data Analysis Data Analysis Received Data per Recording Interval Measurements with NO sudden dropout mode Bytes downloaded at every 50 ms sample time span Measurements with sudden dropout mode Bytes downloaded at every 50 ms sample time span Mean & median of dropped diverge with a relatively higher standard deviation than the non-dropped measurements. Network inconsistency and jitter is present in the dropped measurements than in non-dropped ones. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 12 / 20

24 Data Analysis Data Analysis Received Data per Recording Interval Average of the first 3 bit-rates samples before and after dropout Average of the first 3 bit-rates before dropout Average of the first 3 bit-rates after dropout CDF Download Speed (Mbps) Figure 6: Average of the first 3 bit rates samples before and after a sudden dropout. The effect of sudden dropout is reflected even after the sudden dropout (zero bit rate) is over. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 13 / 20

25 Data Analysis Data Analysis Sudden Dropout by Network Technology Sudden dropout frequency 6% 4% 2% 0% 6% 4% 2% 0% 6% 4% 2% 0% 8% 6% 4% 2% 0% 6% 4% 2% 0% 6% 4% 2% 0% 6% 4% 2% 0% Time of a day EDGE GPRS HSDPA HSPA HSPA+ LTE UMTS The sudden dropouts is distributed in all radio technology that has been used. Some technologies such as UMTS and HSPA show frequent sudden dropouts during daytime. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 14 / 20

26 Data Analysis Data Analysis Switching Radio Technology 1.00 CDF Unknown to LTE Unknown to UMTS HSPA HSPA to HSPA+ LTE HSPA+ UMTS to HSPA+ UMTS HSDPA Download speed (Mbps) Figure 7: Impact of radio technology switches for download speed. When sudden dropout happens a switch in radio technology causes a significant variation in TCP download speed. E.g. a change from UMTS to HSPA+ has better download speed than from HSPA to HSPA+. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 15 / 20

27 Data Analysis Data Analysis Feature Importance and Selections Downlink inter-arrival(variance) Average throughput(tcp) Device Model Area province Radio type at the beginning Latency(UDP) Radio type at the end Uplink speed Signal strengths Mobile Network Operator Time of a day(hours) Battery level Handover Cell ID Changes Day of week Frequency of handover Platform Byte inter-arrival(mean) Mean Decrease Accuracy Downlink (TCP) Downlink inter-arrival(variance) Area province Uplink speed Device Model latency Latency(UDP) carrier Radio type at the beginning signalstrengths Day of week hours Battery level DModel Time of a day(hours) weekday Signal strengths battery_level Radio type at the end CellIDChanged Handover devicevendor RadioHandover Mobile Network Operator radioevolution Cell ID Changes EndingNetTechName Frequency of handover StartingNetTechNames Byte inter-arrival(mean) RadioHandoverStartEnd Platform Mean Decrease Mean Gini Decrease Accuracy Ra S (a) with TCP throughput (b) without TCP throughput Latency, carrier network, signal strength radio network technology, time of the day and device model found to be important predictive variables for classification. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 16 / 20

28 Data Analysis Data Analysis Classification model True positive rate o.o ROC Models: Conditional Inference Tree Bagging RandomForest False positive rate Figure 9: True positive and false positive rates of the three classification models; random forest shows the best Receiver Operating Characteristic (ROC) curve. Random forest based classification produces better prediction with accuracy of 90% & error rate of 10.2% on the testing dataset. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 17 / 20

29 Data Analysis Data Analysis Predicting the Average Throughput Prediction of througput is useful (eg. to improve video performance in cellular networks [4]). How to predict the overall mean throughput only using the first five seconds of TCP bit rate measurement? Train a model using Randome Forest algorithm from caret [5] package. The model predict average throughput: using only the first 5 sec. download rates (which would be easily available right after startup) with Root-Mean-Square Error (RMSE) ( Mbps) & 98% R-squared. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 18 / 20

30 Conclusion

31 Conclusion Conclusion Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 19 / 20

32 Conclusion Conclusion 1 Throughput observed by a cellular network user depends on various factors. E.g location and time of the day where metropolitan areas during peak hours showed more drops in throughput. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 19 / 20

33 Conclusion Conclusion 1 Throughput observed by a cellular network user depends on various factors. E.g location and time of the day where metropolitan areas during peak hours showed more drops in throughput. 2 Network stability: TCP being sensitive to losses and jitter 30% sudden drops to zero bitrate - could create performance degradations. classify stability based on sudden dropouts only relying on easily accessible information. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 19 / 20

34 Conclusion Conclusion 1 Throughput observed by a cellular network user depends on various factors. E.g location and time of the day where metropolitan areas during peak hours showed more drops in throughput. 2 Network stability: TCP being sensitive to losses and jitter 30% sudden drops to zero bitrate - could create performance degradations. classify stability based on sudden dropouts only relying on easily accessible information. 3 Predicting the average throughput: useful to anticipate future performance & to adjust application demands. trained a model that predicts average throughput based on throughput of TCP slow start phase. Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 19 / 20

35 Conclusion Conclusion 1 Throughput observed by a cellular network user depends on various factors. E.g location and time of the day where metropolitan areas during peak hours showed more drops in throughput. 2 Network stability: TCP being sensitive to losses and jitter 30% sudden drops to zero bitrate - could create performance degradations. classify stability based on sudden dropouts only relying on easily accessible information. 3 Predicting the average throughput: useful to anticipate future performance & to adjust application demands. trained a model that predicts average throughput based on throughput of TCP slow start phase. 4 Future work extending our predictive approach : Using Crowdsourcing Data for Adaptive Video Streaming in Cellular Network. ermias.walelgne@aalto.fi Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 19 / 20

36 Conclusion References ITU, ITU releases 2017 global information and communication technology facts and figures. IDC, IDC idc 50th anniversary transformation everywhere, Netradar measurement platform. X. K. Zou, J. Erman, V. Gopalakrishnan, E. Halepovic, R. Jana, X. Jin, J. Rexford, and R. K. Sinha, Can Accurate Predictions Improve Video Streaming in Cellular Networks?, HotMobile, pp , M. Kuhn, Building Predictive Models in R Using the caret Package, Journal of Statistical Software, vol. 28, no. 5, pp. 1 26, Analyzing Throughput and Stability in Cellular Networks April 25, 2018 NOMS 18, Taipei 20 / 20