Deliverable D7.4 Field Validation Test Report

Transcription

1 VIVALDI PROJECT DOCUMENT WORK PACKAGE 7 Deliverable D7.4 Field Validation Test Report Revision 1.2 Project full title: Advancing interactive Broadband satellite access by optimal convergence of session based services over DVB-RCS Proposal/Contract no.: FP IST SIXTH FRAMEWORK PROGRAMME PRIORITY 2 IST-Information Society Technologies

2 Revision Record Revision Author Page Description of Revision Date 01 P. Johnson All Document creation (first draft) G. Wilkinson Various Internal review

3 ACRONYMS AND ABBREVIATIONS ACRONYMS AND ABBREVIATIONS BE DVB DVB-RCS GSM FTP IDC IP ISP MAC NAT PEP PLR PSTN QoS RTP RTT SIP VoIP VPN Full Description Best Effort Digital Video Broadcasting DVB-Return Channel by Satellite Global System for Mobile Communications File Transfer Protocol Internet Data Centre Internet Protocol Internet Service Provider Medium Access Control Network Address Translation Performance Enhancing Proxy Packet Loss Rate Public Switched Telephone Network Quality of Service Real Time Protocol Round Trip Time Session Initiation Protocol Voice over Internet Protocol Virtual Private Network

4 1 TABLE OF CONTENTS 1 TABLE OF CONTENTS TABLE OF FIGURES TABLE OF TABLES Purpose and Structure of the Document Applicable Documents Incorporation of the Test Bed Validation Results Test Setup and Location Test Setup Hardware Configuration Software and Server Configuration Testing Philosophy and Overview Testing of System Requirements Scenarios User Plane Validation Control and Management Plane Validation Generation of Test Data Streams VoIP Streams VPN Stream Representing Critical Data Real-Time Non-Interactive Stream FTP Data Stream Classification of Data Streams Quality of Service Mechanisms Provisioning of Quality of Service in the Forward Link Provisioning of Quality of Service in the Return Channel Measurement Methods Packet Loss Jitter Round Trip Time (RTT) VIVALDI Field Validation Test Results User Plane Test Results Test Scenario Test Scenario Test Scenario

5 9.1.4 Test Scenario Test Scenario Test Scenario Test Scenario Test Scenario Test Scenario Control Plane Test Results Management Plane Test Results TABLE OF FIGURES Figure 7-1: Vivaldi Test Setup...8 Figure 9-1: PC A Screenshot for Scenario Figure 9-2: Traffic Shaper Screenshot for Scenario Figure 9-3: WireShark Statistics for the VoIP Connection Figure 9-4: PC A Screenshot for Scenario 2 without VoIP Figure 9-5: Traffic Shaper Screenshot for Scenario 2 without VoIP Figure 9-6: Traffic Shaper Screenshot for Scenario 2 with VoIP Figure 9-7: WireShark Statistics for the VoIP Connection Figure 9-8: Traffic Shaper Screenshot for Scenario 3 without VoIP Figure 9-9: Traffic Shaper Screenshot for Scenario 3 with VoIP Figure 9-10: WireShark Statistics for the VoIP Connection Figure 9-11: PC A Screenshot for Scenario 4 without VoIP Figure 9-12: Traffic Shaper Screenshot for Scenario 4 without VoIP Figure 9-13: Traffic Shaper Screenshot for Scenario 4 with VoIP Figure 9-14: WireShark Statistics for the VoIP Connection Figure 9-15: PC A Screenshot for Scenario 5 with all three Service Classes Figure 9-16: Traffic Shaper Screenshot for Scenario Figure 9-17: Traffic Shaper Screenshot for Scenario 5 Large Plot Figure 9-18: Traffic Shaper Screenshot for Scenario 5 Large Table Figure 9-19: WireShark Statistics for the VoIP Connection Figure 9-20: PC A Screenshot for Scenario Figure 9-21: Traffic Shaper Screenshot for Scenario Figure 9-22: WireShark Statistics for the VoIP Connection Figure 9-23: Traffic Shaper Screenshot for Scenario Figure 9-24: WireShark Statistics for the VoIP Connection Figure 9-25: Traffic Shaper Screenshot for Scenario Figure 9-26: WireShark Statistics for the VoIP Connection Figure 9-27: Traffic Shaper Screenshot for Scenario Figure 9-28: WireShark Statistics for the VoIP Connection TABLE OF TABLES Table 7-1: Vivaldi Hardware Configuration...9 Table 7-2: Vivaldi PCs Software Configuration...9 Table 7-3: Vivaldi HUB / Server Software Configuration...9 Table 8-1: Traffic Classes and Data Stream Classification Table 9-1: User Plane Test Results for Test Scenario

6 Table 9-2: User Plane Test Results for Test Scenario Table 9-3: User Plane Test Results for Test Scenario Table 9-4: User Plane Test Results for Test Scenario Table 9-5: User Plane Test Results for Test Scenario Table 9-6: User Plane Test Results for Test Scenario Table 9-7: User Plane Test Results for Test Scenario Table 9-8: User Plane Test Results for Test Scenario Table 9-9: User Plane Test Results for Test Scenario Table 9-10: Control Plane Test Results Table 9-11: Management Plane Test Results... 42

7 4 PURPOSE AND STRUCTURE OF THE DOCUMENT This document presents the results of the Field Validation required under work package deliverable 7.4. The purpose of this document is to summarise the performance of the system during a field trial under real-world conditions in a live network. This document roughly follows the structure of the Validation Plan, WP7.2 [ AD1], but contains some deviations, the reasons for which are explained in the text. The incorporation of the Test Best Validation results of WP7.3 are explained in Section 6 The document contains minimal assessment or analysis of the results which is included in deliverable APPLICABLE DOCUMENTS Applicable documents are references that have a direct input to this document. AD1. D7.2 Vivaldi Final Test and Validation Plan - VIVALDI_D7.2_WP7_V2.0 AD2. D7.3 Vivaldi Test Bed Validation Results - VIVALDI_D7.3_WP7_V1.0 AD3. D2.3 Vivaldi Technical Requirements Specification VIVALDI_D2.3_WP2_V1.0 6 INCORPORATION OF THE TEST BED VALIDATION RESULTS The Test Bed Validation Results [ AD2] were deemed to be of limited relevance to the production of this report and the contents of this report were not used during the test programme. Testing conducted as part of this report was designed as a field trial and under real-world scenarios in a live network. 7 TEST SETUP AND LOCATION 7.1 Location of Testing Testing was undertaken at various locations, but the results presented here were all measured at the following location: Avanti Communications plc 74 Rivington Street London, EC2A 3AY

8 7.2 Test Setup Hardware Configuration The test setup used for almost all testing is shown below in Figure 7-1. Note that on the PC B end of the setup the ADSL router interfacing between the internet and the test PC is not shown. Any deviations from this configuration will be explicitly described when applicable. Figure 7-1: Vivaldi Test Setup Note that in Figure 7-1, the Forward Link is defined as the incoming traffic into PC A, i.e. PC A s download. The Return Channel is the traffic being uploaded from PC A. The Forward Link has been set to a maximum throughput of 512kbps. The Return Channel has been set to a maximum throughput of 128kbps.

9 Device Manufacturer Model Serial / Identification # PC A Hewlett Packard Compaq 2510p CNF7342PWM PC B Hewlett Packard Compaq dc7600 CZC6201FQY Terminal Newtec STMi NTC2107 SatLink 1000 SIP Server Hewlett Packard BL25P N/A Hub GEO Satellite Newtec STMi Space Systems/Loral 2 way Sat R.4.1 SatLink Carrier Class 12.1 Intelsat Table 7-1: Vivaldi Hardware Configuration Software and Server Configuration / / 1.10 N/A IS903@34.5W 1. PC A and PC B Software Configuration Purpose Publisher Product Name Version Operating System Microsoft Windows XP SP2 Web Browser Microsoft Internet Explorer VoIP Client CounterPath EyeBeam Network Protocol Analyser Gerald Combs Wireshark Network Traffic Monitor Delphi Programming Network Traffic Monitor 1.0 Table 7-2: Vivaldi PCs Software Configuration 2. HUB and Server Software Configuration Purpose Publisher Product Name Version HUB Operating System Microsoft Windows Server 2003 R2 Enterprise Edition SP2 SIP Server Digium Asterix G.729 Codec Digium G.729 Codec 8G729CODEC Traffic Shaper Allot NetXplorer Build 34 Table 7-3: Vivaldi HUB / Server Software Configuration

10 8 TESTING PHILOSOPHY AND OVERVIEW 8.1 Testing of System Requirements Scenarios Nine scenarios, some of which are proposed in AD1, are used as a means to replicate typical network conditions and subsequently illustrate the correct operation of the Vivaldi System. However, some scenarios have been altered from those described in AD1. The changes were implemented for technical reasons and also to better represent typical network conditions. Each scenario is detailed in Section User Plane Validation For each scenario, the parameters required to test the system s performance against the User Plane requirements stated in Section 4 of AD3 were collected. Section 9.1 lists the results and the details of each scenario. Screenshots of each scenario are included to illustrate the network conditions under which the test was conducted and the performance within these scenarios Control and Management Plane Validation As each of the Control and Management Plane requirements stated in Section 5 and Section 6 respectively of AD3 are independent of the Test Scenario used they are only summarised once in Section 0 and 9.3. These results were collected mostly via analysis of the test setup. 8.2 Generation of Test Data Streams VoIP Streams All VoIP data was generated by making a call from PC A to PC B using the EyeBeam VoIP client detailed in Table 7-2. The codec used in both clients and the SIP server was G.729. The VoIP stream was marked with a TOS value of 0x10. The stream is bi-directional at 31.2kbps VPN Stream Representing Critical Data Iperf was used to generate traffic representing VPN data for both the Forward Link and the Reverse Channel. The command used to generate the Forward Link data stream was: PC A: iperf s S 0x08 PC B: iperf c S 0x08 t 2000 i 5 The above commands generated a 2000 second long TCP data stream marked with 0x08 TOS markings. Every 5 seconds the throughput was displayed for debugging purposes Real-Time Non-Interactive Stream The non-interactive real-time (NIRT) data stream was the CNN.com live data stream of 384kbps. The link for this stream is This traffic was received without any QoS markings and was classified and subsequently prioritised by the traffic shaper which classified the stream by the port number.

11 8.2.4 FTP Data Stream The FTP streams were obtained from the Avanti FTP server at The server contains multiple 50Mb test files which can be downloaded in parallel, providing the ability to totally saturate the Forward Link with Best Effort data. The FTP server also limits each download stream to 256kbps, allowing FTP downloads to be done that won t saturate the forward link. The TOS marking for this stream was 0x00. For Reverse Channel testing, 50Mb files were uploaded in a similar manner. 8.3 Classification of Data Streams Forward Return Weighting TOS Traffic Traffic Class Bandwidth Bandwidth (FL Only) Marking Streams Expedited 32kbps 32kbps 10 0x10 VoIP Forwarding (EF) Assured VPN 256kbps 64kbps 8 0x08 Forwarding (AF) NIRT FTP Best Effort (BE) N/A N/A 4 0x00 HTTP Table 8-1: Traffic Classes and Data Stream Classification Table 8-1 outlines the 3 distinct Traffic Classes of EF, AF, and BE. The Vivaldi System considers VoIP to be of the highest priority and assigns it a guaranteed 32kbps (Forward and Return) upon detection of VoIP packets. Upon termination of the call this bandwidth is free to be used for AF and/or BE traffic. 31.2kbps is the required bandwidth for an 8kbps G.729 VoIP call 1. The AF class is assigned half of the available bandwidth and any traffic classified as AF is given priority over BE traffic, but not any EF traffic, if present. If the channel is not saturated then the AF can consume additional bandwidth. However, should the AF traffic exceed the assigned bandwidth in a channel saturated with BE traffic, it is allocated further capacity in line with the weightings listed in Table 8-1. EF traffic takes precedence at all time times. 8.4 Quality of Service Mechanisms Provisioning of Quality of Service in the Forward Link The DVB-RCS Hub does not provide QoS capabilities for the Forward link. To implement this facility a traffic shaper (Allot NetXplorer) installed on the RCS hub provided this capability. Traffic was first identified by its port number and assigned to a traffic class and these classes were then prioritised in accordance with Table 8-1. Traffic classification by port identification was required as the QoS markings applied to the outgoing test packets (either TOS or DSCP) were sometimes removed when they were applied to AF traffic passing out into the internet. Whilst the 0x10 tags on VoIP traffic always passed through successfully, 0x08 tags were often removed. 1

12 8.4.2 Provisioning of Quality of Service in the Return Channel The RCS Terminal and Hub provide the QoS capability in the Return Channel. Traffic classification and allowable volumes of each traffic class is defined in the terminal settings. The traffic request is then sent to the hub which grants bandwidth according the priorities assigned to each class of traffic. The traffic is monitored, but not controlled, by the traffic shaper. 8.5 Measurement Methods Packet Loss The packet loss statistics for the VoIP stream was measured by Wireshark running on PC A, the PC from which all VoIP calls originated. Screenshots are included in the test report Jitter The jitter statistics for the VoIP stream was measured by Wireshark running on PC A, the PC from which all VoIP calls originated. Screenshots are included in the test report Round Trip Time (RTT) The RTT was measured by using the ICMP ping command from PC B, the PC that received the voice calls from PC A. To configure the test setup to give accurate results, all ICMP commands were treated as VoIP or EF traffic. Screenshots are not included in the test report. The results reported are the average of at least 10 pings usually many more.

13 9 VIVALDI FIELD VALIDATION TEST RESULTS 9.1 User Plane Test Results Test Scenario 1 This scenario aims to demonstrate the correct behaviour of the system in the absence of Forward or Return link saturation. EF, AF, and BE traffic classes are present in the form of a VoIP call, a VPN connection, and a FTP download respectively. The following steps were undertaken to simulate Test Scenario 1 as described in Section of the Final Test and Validation Plan [ AD1]. 1. PC A connected to the internet via a SSL server at - This generates a small amount of idle AF traffic. 2. PC A connected to the Avanti FTP site and began to download a single 50MB file. - A large (256k) but non-saturating amount of BE traffic is generated. 3. PC A made a VoIP call to PC B - This generated VoIP traffic which the shaper should class as EF. Figure 9-1 shows the screen from PC A. Visible are the VoIP call, the FTP download, and the VPN connection. Figure 9-1: PC A Screenshot for Scenario 1 Figure 9-2 shows the traffic shaper monitor which is running on PC B. Clearly visible are the three streams.

14 Figure 9-2: Traffic Shaper Screenshot for Scenario 1 Figure 9-3 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A. Figure 9-3: WireShark Statistics for the VoIP Connection Table 9-1 summarises the relevant User Plane statistics for Scenario 1. Test Requirement ID Relevant Measured Parameter Expected Result Measured or Observed Result

15 SYS-REQ-1 RTT (Round Trip Time) <1200ms 742ms SYS-REQ-2 Jitter <100ms 90% of the time. <150ms 100% of the time. <29.17ms 100% of the time. SYS-REQ-3 Packet Loss <5% 0.4% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-1: User Plane Test Results for Test Scenario 1

16 9.1.2 Test Scenario 2 This scenario aims to demonstrate the behaviour of the system when saturated with BE traffic. Saturation of the forward link with BE traffic is obtained by commencing multiple simultaneous FTP downloads and then placing a VoIP call. Multiple downloads are required as the FTP server caps each stream at 256kbps which is not sufficient to saturate a 512kbps forward link. This scenario aims to verify that the Vivaldi System correctly allocates bandwidth to the EF traffic in a channel saturated with BE traffic. The following steps were undertaken to simulate Test Scenario 2 as described in Section of the Final Test and Validation Plan [ AD1]. 1. PC A connected to the Avanti FTP site and began to download three 50MB files simultaneously. - This saturated the forward link with BE traffic. 2. PC A made a VoIP call to PC B - This generated VoIP traffic which the shaper should class as EF. Figure 9-4 shows the screen from PC A with the three simultaneous FTP downloads. The VoIP call has not been made yet. Figure 9-4: PC A Screenshot for Scenario 2 without VoIP Figure 9-5 shows the traffic shaper monitor which was running on PC B. Clearly visible is the saturated forward link.

17 Figure 9-5: Traffic Shaper Screenshot for Scenario 2 without VoIP Figure 9-6 shows the traffic shaper monitor once the VoIP call was made. It can be seen that the system is allocating bandwidth to the VoIP call in preference to the BE traffic. Figure 9-6: Traffic Shaper Screenshot for Scenario 2 with VoIP Figure 9-7 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A.

18 Figure 9-7: WireShark Statistics for the VoIP Connection Table 9-2 summarises the relevant User Plane statistics for Scenario 2. Test Requirement ID Relevant Measured Parameter Expected Result SYS-REQ-1 RTT (Round Trip Time) <1200ms 739ms SYS-REQ-2 Jitter <100ms 90% of the time. <150ms 100% of the time. Measured or Observed Result <37.19ms 100% of the time. SYS-REQ-3 Packet Loss <5% 1.8% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-2: User Plane Test Results for Test Scenario 2

19 9.1.3 Test Scenario 3 This scenario aims to demonstrate the behaviour of the system when saturated with AF traffic. Saturation of the forward link with AF traffic is obtained by generating multiple iperf traffic streams which are identified by the traffic shaper and treated as AF traffic. This scenario aims to verify that the Vivaldi System correctly allocates bandwidth to the EF traffic in a channel saturated with AF traffic. The following steps were undertaken to simulate Test Scenario 3 as described in Section of the Final Test and Validation Plan [ AD1]. 1. The following iperf command was run on PC A. iperf s S 0x08 - This set up PC A to receive AF traffic from PC B. 2. The following iperf command was run twice from two prompts on PC B. iperf c S 0x08 t 2000 i 5 - This saturated the forward link with AF traffic. 3. PC A made a VoIP call to PC B - This generated VoIP traffic which the shaper should class as EF. Note that the traffic generated with the 0x08 TOS tags often did not arrive with these tags still present. As stated in Section 8.4.1, the AF traffic was identified by port number (5001). Figure 9-8 shows the screen from PC B with the forward link saturated with AF traffic. The VoIP call has not been made yet.

20 Figure 9-8: Traffic Shaper Screenshot for Scenario 3 without VoIP Figure 9-9 shows the traffic shaper monitor once the VoIP call is made. The system is correctly decreasing AF traffic in the presence of higher-priority EF traffic from the VoIP call. Figure 9-9: Traffic Shaper Screenshot for Scenario 3 with VoIP Figure 9-10 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A.

21 Figure 9-10: WireShark Statistics for the VoIP Connection Table 9-3 summarises the relevant User Plane statistics for Scenario 3. Test Requirement ID Relevant Measured Parameter Expected Result SYS-REQ-1 RTT (Round Trip Time) <1200ms 752ms SYS-REQ-2 Jitter <100ms 90% of the time. <150ms 100% of the time. Measured or Observed Result 26.25ms 100% of the time. SYS-REQ-3 Packet Loss <5% 0.1% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-3: User Plane Test Results for Test Scenario 3

22 9.1.4 Test Scenario 4 This scenario aims to demonstrate the behaviour of the system when saturated with mixed AF traffic. In this scenario, the AF is a mixture of both Non-Interactive Real-Time (NIRT) media in the form of a streaming online video and an iperf/vpn link. The traffic shaper will assign both to the AF class. This scenario aims to verify that the Vivaldi System correctly allocates bandwidth to the EF traffic in a channel saturated with mixed AF traffic. The following steps were undertaken to simulate Test Scenario 4 as described in Section of the Final Test and Validation Plan [ AD1], although with the addition of the VPN traffic. 1. PC A was set to stream a 384kbps video from the following location: - This generated a significant but non-saturating AF traffic in the forward link. 2. The following iperf command was run on PC A. iperf s S 0x08 - This set up PC A to receive AF traffic from PC B. 3. The following iperf command was run from a single prompt on PC B. iperf c S 0x08 t 2000 i 5 - This saturated the forward link with additional AF traffic. 4. PC A made a VoIP call to PC B - This generated VoIP traffic which the shaper should class as EF. Figure 9-11 shows the screen from PC A with the streaming video and iperf data stream. The VoIP call has not been made yet. Note that the screenshot doesn t capture the streaming video itself.

23 Figure 9-11: PC A Screenshot for Scenario 4 without VoIP Figure 9-12 shows the traffic shaper monitor, illustrating the video and VPN data being allocated to the AF traffic class and the bandwidth being split evenly between the two. Figure 9-12: Traffic Shaper Screenshot for Scenario 4 without VoIP Figure 9-13 shows the traffic shaper monitor once the VoIP call has been made. The bandwidth available to the AF class has again been reduced to make way to the EF class.

24 Figure 9-13: Traffic Shaper Screenshot for Scenario 4 with VoIP Figure 9-14 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A. Figure 9-14: WireShark Statistics for the VoIP Connection Table 9-4 summarises the relevant User Plane statistics for Scenario 4. Test Requirement ID Relevant Measured Parameter Expected Result SYS-REQ-1 RTT (Round Trip Time) <1200ms 761ms. Measured or Observed Result SYS-REQ-2 Jitter <100ms 90% of the 34.01ms 100% of the

25 time. <150ms 100% of the time. time. SYS-REQ-3 Packet Loss <5% 0.4% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-4: User Plane Test Results for Test Scenario 4

26 9.1.5 Test Scenario 5 This scenario aims to demonstrate the behaviour of the system when saturated with BE traffic and AF traffic. Saturation of the forward link with BE and AF traffic is obtained by commencing multiple simultaneous FTP downloads, adding multiple AF streams, and then placing a VoIP call. This scenario aims to verify that the Vivaldi System correctly allocates bandwidth according to Table 8-1 in the presence of all three traffic classes. Test Scenario 5 is effectively the combination of Test Scenario 2 and Test Scenario 3 and illustrates that the Vivaldi system dynamically allocates bandwidth between the three service classes. Note that the SLA RC was set to 140k, not 128k. The following steps were undertaken to simulate Test Scenario PC A connected to the Avanti FTP site and began to download three 50MB files simultaneously. - This saturated the forward link with BE traffic. 2. The following iperf command was run on PC A. iperf s S 0x08 - This set up PC A to receive AF traffic from PC B. 3. The following iperf command was run twice from two prompts on PC B. iperf c S 0x08 t 2000 i 5 - This saturated the forward link with AF traffic. 4. PC A made a VoIP call to PC B - This generated VoIP traffic which the shaper should class as EF. Figure 9-15 shows the screen from PC A with three simultaneous FTP downloads, the two incoming iperf streams, and the VoIP call.

27 Figure 9-15: PC A Screenshot for Scenario 5 with all three Service Classes Figure 9-16 shows the traffic shaper monitor which was running on PC B. Clearly visible is the correctly allocated traffic between the three service classes. Figure 9-16: Traffic Shaper Screenshot for Scenario 5 Figure 9-17 and Figure 9-18 show the details of the QoS being provided to the three traffic classes. It can be seen that the QoS required in Table 8-1 is be adhered to.

28 Figure 9-17: Traffic Shaper Screenshot for Scenario 5 Large Plot Figure 9-18: Traffic Shaper Screenshot for Scenario 5 Large Table Figure 9-19 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A.

29 Figure 9-19: WireShark Statistics for the VoIP Connection Table 9-5 summarises the relevant User Plane statistics for Scenario 5. Test Requirement ID Relevant Measured Parameter Expected Result SYS-REQ-1 RTT (Round Trip Time) <1200ms 747ms SYS-REQ-2 Jitter <100ms 90% of the time. <150ms 100% of the time. Measured or Observed Result 27.66ms 100% of the time. SYS-REQ-3 Packet Loss <5% 0.2% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-5: User Plane Test Results for Test Scenario 5

30 9.1.6 Test Scenario 6 Whereas the previous scenarios have involved a saturated forward link, this scenario saturates the return channel (RC) with BE traffic. Once the RC is saturated a VoIP is made. This scenario aims to verify the correct operation of the Vivaldi System on the RC. The system should assign the required bandwidth to the EF in preference to the BE traffic. However, unlike the previous scenarios where the traffic shaper facilitated the QoS mechanism, for the RC the QoS is provided by the RCS Terminal and Hub. The terminal classifies incoming traffic into the three traffic classes, based on the class volume limits stated in Table 8-1. The Hub then prioritises these traffic classes accordingly. Any excess AF traffic is treated as BE. 1. PC A connected to the Avanti FTP site and began to upload three 50MB files simultaneously. - This saturated the RC with BE traffic. 2. PC A made a VoIP call to PC B - This generated VoIP traffic which the shaper should class as EF. Figure 9-20 shows the screen from PC A with three simultaneous FTP uploads and the VoIP call. Figure 9-20: PC A Screenshot for Scenario 6

31 Figure 9-2 shows the traffic shaper monitor which is running on PC B. Clearly visible are the VoIP and FTP streams. Note that the traffic shaper is only being used as a monitoring tool for the return channel. Figure 9-21: Traffic Shaper Screenshot for Scenario 6 Figure 9-22 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A. Figure 9-22: WireShark Statistics for the VoIP Connection

32 Table 9-6 summarises the relevant User Plane statistics for Scenario 6. Test Requirement ID Relevant Measured Parameter Expected Result SYS-REQ-1 RTT (Round Trip Time) <1200ms 799ms SYS-REQ-2 Jitter <100ms 90% of the time. <150ms 100% of the time. Measured or Observed Result 45.78ms 100% of the time. SYS-REQ-3 Packet Loss <5% 0.3% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-6: User Plane Test Results for Test Scenario 6

33 9.1.7 Test Scenario 7 This scenario saturates the return channel (RC) with AF traffic generated by a VPN connection. Once the RC is saturated a VoIP is made. This scenario aims to verify the correct operation of the Vivaldi System on the RC. The system should assign the required bandwidth to the EF in preference to the AF traffic. 1. The following iperf command was run on PC B. iperf s S 0x08 - This sets up PC B to receive AF traffic from PC A. 2. The following iperf command was run twice from two prompts on PC A. iperf c S 0x08 t 2000 i 5 - This saturated the RC with AF traffic. 3. PC A made a VoIP call to PC B - This generated VoIP traffic which the shaper should class as EF. Figure 9-23 shows the traffic shaper with two simultaneous AF streams and the VoIP call. Figure 9-23: Traffic Shaper Screenshot for Scenario 7 Figure 9-24 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A.

34 Figure 9-24: WireShark Statistics for the VoIP Connection Table 9-7 summarises the relevant User Plane statistics for Scenario 7. Test Requirement ID Relevant Measured Parameter Expected Result SYS-REQ-1 RTT (Round Trip Time) <1200ms 723ms SYS-REQ-2 Jitter <100ms 90% of the time. <150ms 100% of the time. Measured or Observed Result 34.93ms 100% of the time. SYS-REQ-3 Packet Loss <5% 0.7% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-7: User Plane Test Results for Test Scenario 7

35 9.1.8 Test Scenario 8 This scenario aims to demonstrate the behaviour of the system when saturated with BE traffic and AF traffic. Saturation of the RC with BE traffic is obtained by commencing multiple simultaneous FTP uploads, multiple AF streams, and then placing a VoIP call. This scenario aims to verify that the Vivaldi System correctly allocates bandwidth according to Table 8-1 in the presence of all three traffic classes. It is the RC equivalent of Scenario 5. Test Scenario 8 is effectively the combination of Test Scenario 2 and Test Scenario 3 and tests the Vivaldi system dynamically allocates bandwidth between the three service classes. The following steps were undertaken to simulate Test Scenario PC A connected to the Avanti FTP site and began to upload three 50MB files simultaneously. - This saturated the forward link with BE traffic. 6. The following iperf command was run on PC B. iperf s S 0x08 - This set up PC B to receive AF traffic from PC A. 7. The following iperf command was run twice from two prompts on PC B. iperf c S 0x08 t 2000 i 5 - This saturated the RC with AF traffic. 8. PC A made a VoIP call to PC B - This generated VoIP traffic which the shaper should class as EF. Figure 9-25 shows the traffic shaper monitor which was running on PC B. Clearly visible is the correctly allocated traffic between the three service classes.

36 Figure 9-25: Traffic Shaper Screenshot for Scenario 8 Figure 9-26 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A. Figure 9-26: WireShark Statistics for the VoIP Connection

37 Table 9-7 summarises the relevant User Plane statistics for Scenario 7. Test Requirement ID Relevant Measured Parameter Expected Result SYS-REQ-1 RTT (Round Trip Time) <1200ms 785ms SYS-REQ-2 Jitter <100ms 90% of the time. <150ms 100% of the time. Measured or Observed Result 22.55ms 100% of the time. SYS-REQ-3 Packet Loss <5% 0.1% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-8: User Plane Test Results for Test Scenario 8

38 9.1.9 Test Scenario 9 This scenario aims to demonstrate the behaviour of the system when totally saturated in both directions with BE traffic and AF traffic. Saturation of the RC with BE traffic is obtained by commencing multiple simultaneous FTP uploads, multiple AF streams, and then placing a VoIP call. Scenario 9 is the combination of Scenario 5 and Scenario 8 and the setup techniques are identical. Figure 9-27 shows the traffic shaper monitor which was running on PC B. Clearly visible is the correctly allocated traffic between the three service classes for both the Forward Link and the Reverse Channel. Figure 9-27: Traffic Shaper Screenshot for Scenario 9

39 Figure 9-28 shows the WireShark statistics for the VoIP stream. WireShark is running on PC A. Figure 9-28: WireShark Statistics for the VoIP Connection Table 9-7 summarises the relevant User Plane statistics for Scenario 7. Test Requirement ID Relevant Measured Parameter Expected Result SYS-REQ-1 RTT (Round Trip Time) <1200ms 787ms SYS-REQ-2 Jitter <100ms 90% of the time. <150ms 100% of the time. Measured or Observed Result 35.21ms 100% of the time. SYS-REQ-3 Packet Loss <5% 0.8% Return Channel 0.0% Forward Channel SYS-REQ-4 Bandwidth Available for a VoIP Call. 24kbps 31.2kbps SYS-REQ-5 QoS Mechanism Used DSCP TOS / Traffic Shaper. Table 9-9: User Plane Test Results for Test Scenario 9

40 9.2 Control Plane Test Results Test Requirement ID SYS-REQ-6 Relevant Measured Parameter Support of SIP Signalling Expected Result SIP Signalling Supported Measured or Observed Result SIP Signalling Supported SYS-REQ-7 Support of H.323 H.323 Supported The Eyebeam client does not support the H.323 protocol. SYS-REQ-8 Support of Skype Skype Supported Skype is supported over the system. However, the QoS system was not tested with Skype. SYS-REQ-9 Call Setup Time <10s for 90% of the time when using SIP signalling SYS-REQ-10 Call Setup Success Ratio Call setup time was typically 1-2 seconds for over 30 calls >98% 100% for 30 successive calls. SYS-REQ-11 Call Setup Time 5-10 Seconds. 1-2 seconds for 30 calls. The system cannot differentiate between emergency and nonemergency calls. SYS-REQ-12 SYS-REQ-13 SYS-REQ-14 Support of Voice Calls over DVB- RCS System Support of Voice Calls over the Internet. Support of Voice Calls to/from the PSTN. Voice calls can be made between two independent end-point SITs on the same DVB-RCS system Bi-directional voice calls can be made between an end-point behind a SIT to another end-point connected to the internet or private IP backbone. Bi-directional voice calls can be made between an end-point behind a SIT to another end-point connected to the PSTN. Table 9-10: Control Plane Test Results Calls can be made between two independent SITs on the same DVB-RCS system. Yes this report was conducted on a system with calls made between an endpoint SIT and another endpoint on the internet (Avanti SIT ). Calls can be made between an end-point SIT and another end-point connected to the PSTN.

41 9.3 Management Plane Test Results Test Requirement ID SYS-REQ-15 SYS-REQ-16 SYS-REQ-17 SYS-REQ-18 SYS-REQ-19 SYS-REQ-20 SYS-REQ-21 SYS-REQ-22 SYS-REQ-23 SYS-REQ-24 Relevant Measured Parameter Fault Management Fault Management Configuration Management Configuration Management Configuration Management Configuration Management Account Management Account Management Account Management Performance Management Expected Result Fault Management exists on system layers 1, 2, and 3. The impact of faults on session based services is reported. The system supports and allows configuration of service classes. Bandwidth can be configured on demand. Facilities exist for configurable bandwidth management. The system provides a call blocking method. The system can measure forward and return IP volume on a per terminal basis. The system can measure forward and return VoIP volume on a per terminal basis. The system can measure call duration and IP volume on a per session basis. The system monitors and reports performance on a per SIT and per link basis on a forward and return basis. Measured or Observed Result The system does provide Fault Management on layers 1, 2, and 3. The impact of faults on session based services is not reported. However, faults will manifest themselves as poor VoIP performance. The system does support the configuration of service classes. The classes are BE, AF, and EF. The system does not provide on- demand configurable bandwidth. The system must be reset between different reconfigurations. The system provides the ability for configurable bandwidth. The EyeBeam VoIP client provides both domain and individual based call blocking facilities. The system can measure forward and return IP volume in the IDC. The system can measure forward and return VoIP volume using the traffic shaper. The system can measure call duration and IP volume on a per session basis on the Asterix server and or the traffic shaper. Performance on a per SIT basis can be monitored via the traffic shaper. The system can monitor performance on a per link basis. SYS-REQ-25 Security Provision of additional The Asterix server has the

42 Test Requirement ID SYS-REQ-26 Relevant Measured Parameter Expected Result Measured or Observed Result Management security for VoIP calls. ability to implement additional security measures for VoIP calls. Security Management The system can encrypt all IP & VoIP traffic. The system cannot encrypt IP of VoIP traffic but it could be implemented using external applications. Table 9-11: Management Plane Test Results