ETSI Workshop on Telecommunication Quality beyond Wajdi Garali Department head of Internet QoS National Regulation Authority in Tunisia (INTT)

Transcription

1 ETSI Workshop on Telecommunication Quality beyond 2015 Wajdi Garali Department head of Internet QoS National Regulation Authority in Tunisia (INTT) Vienna, Austria October

2 Outline INTT presentation Regulatory Frameworks Tunisia Internet market structure Measurement methodology based on ETSI Guide Internet QoS project QoE / QoS assessment KPI description Some results New Experimentation 2

3 INTT : Presentation The National Telecommunications Regulation Authority (INTT) is a specialized authority set up by Article 63 of Law n in QoS Internet department Looks into almost technical issues regulated by INTT Led QoS project in 2G and 3G networks Leads QoS project for fixed broadband since 2010 Sets up a large consensus measurements methodology Performs monthly campaigns measurements and annual surveys to assess the quality of service of telco Follows up insights and enhancements performed by ISP and operators 3

4 Frameworks (1/2) Regulatory framework Article 63 in Telecommunications Code : «INTT ensures compliance with the obligations resulting from legislation and regulations», Decree N of 18 September 2012 underwrites technical thresholds necessary to guarantee quality and security for Voice over IP (VoIP)services Decree N of 26 December 2014 : INTT sets standards and Internet services quality criteria in force at the national level and ensures the monitoring and evaluation of their compliance by ISPs. 4

5 Frameworks (2/2) International Norms and Standards ITU T Recommendations : ITU T Y1540 ITU T Y1541 related to network performance ETSI Guide : ETSI EG related to techniques for measuring the quality of Internet access and the definition of quality parameters perceived by end users. ETSI Technical specification : ETSI TS monitoring architecture for QoE ITU T recommendations : E 800 defining quality of services concepts and models Some drafts SG11 TD (Q15: Unified methodology of Internet speed quality 5 measurement usable by end users )

6 Tunisian Internet structure 6

7 Measurement methodology (1/3) xxx Source : ETSI EG ETSI EG recommended architecture 7

8 INTT measurement architecture 8

9 Measurement methodology (2/3) Level 32 1: : End International Network user QoE QoSaccess QoS 9

10 Measurement methodology (3/3) QoS assessment in different levels of the value chain : Level 1 : Measurements reflecting End user QoE perception Software probes : third party Portal will launch NDT, Neubot and Shaperprobe tools of M LAB (open platform) Level 2 : Measurements of core network from DSLAM / BRAS Hardware probes : third party Covering all 24 states in Tunisia Level 3 : Measurements of International access from IXP: Hardware probes : third party Hardware probes : RIPE Atlas will assets IP Transit, fair routing and traffic prioritization 10

11 QoE / QoS assessment (1/2) Correlation between levels: Level 2 Level 1 In time : identify source problem and accountability In space: detect service performance (get info about End user behavior) Level 3 QoS Assessment Level 1 reflects the QoE as defined in ETSI TS is the degree of delight or annoyance of the user of an application or service Level 2 provides more samples and analysis 11

12 QoE / QoS assessment (2/2) QoE = QoU + service behavior Quality of Use 12

13 KPIs description KPIs* Data Service : Capacity TCP Throughput HTTP Throughput Traffic Shaping Data Service: Latency Network RTT DNS resolution Time IP Transit Graph VoIP Service MOS (Downstream/Upstream) Packet loss, Jitter, delay Video Streaming Service Buffering Delay and Jitter Broadband Map *All detailed KPIs: Methods of calculation Referred norms Thresholds could be seen in INTT decision and public consultation ( 13

14 Some results (1/2) Overall results (April 2014) Web browsing : Throughput (kbps) vs page downlaoded time (ms) Débit Throughput vers national to national Temps Time to vers national Débit Throughput vers to international Temps Time to vers international 14

15 Some results (2/2) Overall results (April 2014) VoIP : MOS Vs Packet loss 2 1,8 1,6 1,4 1,2 1 0,8 0,6 0,4 0,2 0 4,45 4,4 4,35 4,3 4,25 4,2 4,15 VoIP Packet Loss Down MOS DL 15

16

17 Trial : M Lab Results* Guidelines: This is a set of tools carried on the multi probe system We embedded tools and run the tests on the same hardware probes placed at the level 2 (MSAN) which assets the same ADSL access and rate (2 Mbps) The tests were synchronized with Third party (IpQ) tests during the same period of measurements The M Lab reference servers are located in IXP, while third party references servers were placed in ISPs PoP The same KPI that IPQ are evaluated: TCP Throughput, HTTP throughput, RTT latency M lab offers additional KPI relating to traffic shapping practices. *INTT research study report on M-Lab 17

18 Trial : M Lab Results* 2 1,8 1,6 1,4 1,2 1 0,8 0,6 0,4 0,2 0 TCP Throughput (Mbps/24H Average): M Lab NDT tool vs Third party tool (IpQ) Moyenne NDT Upload Moyenne IPQ_Upload Moyenne de IPQ Download Moyenne de NDT Download The performance is virtually identical for both platforms : M Lab and third party *INTT research study report on M-Lab 18

19 Trial : M Lab Results* 140 RTT (ms/24h Average) : M LAB Neubot tool vs Third party tool (IpQ) Moyenne de Neubot RTT Moyenne de IPQ RTT The result show similar measurements in both platforms: M Lab and Third party with some differences due to hopes between the location of references servers *INTT research study report on M-Lab 19

20 Trial : M Lab Results* Traffic Shapping (Mbps): M Lab Shaperprobe tool 2 1,8 1,6 1,4 1,2 1 0,8 0,6 0,4 0,2 0 15/11/ :48 15/11/ :53 15/11/ :13 15/11/ :29 16/11/ :36 16/11/ :40 16/11/ :44 19/11/ :07 19/11/ :16 19/11/ :28 19/11/ :38 19/11/ :51 19/11/ :12 20/11/ :23 20/11/ :29 20/11/ :32 21/11/ :46 21/11/ :49 21/11/ :55 21/11/ :01 21/11/ :06 21/11/ :14 21/11/ :20 22/11/ :25 22/11/ :28 22/11/ :31 25/11/ :23 25/11/ :31 25/11/ :37 25/11/ :48 25/11/ :53 25/11/ :20 25/11/ :28 26/11/ :34 26/11/ :37 26/11/ :40 26/11/ :00 26/11/ :49 26/11/ :57 26/11/ :03 26/11/ :09 26/11/ :15 26/11/ :41 (vide) Download Estimated throughput estimé Measured Download throughput mesuré Shapping Débit de throughput shaping The M Lab platform can accurately detect traffic shaping practices *INTT research study report on M-Lab 20

21 New experimentation We obtained very promising results! (see INTT trials) Question: why not deploying this software on routers and networks nodes? Good idea! but it might influence routers/node performance Solution Use a new protocol dedicated to QoS measurement which could be enabled on each node This is the IETF LMAP expecting out coming works! 21

22 «Thank you...» 22

23 ETSI Workshop QoS and beyond 2015 Vienna October 2015 Wajdi Garali Department head of Internet QoS National Regulation Authority in Tunisia (INTT) INTT 2015 #INTT Instance Nationale des Télécommunications 23