Acceptance & Regression Testing

Size: px

Start display at page:

Download "Acceptance & Regression Testing"

Elmer French
6 years ago
Views:

1 Compelling Reasons for performing Acceptance and Regression Tests Nexus8610 Traffic Simulation System Acceptance & Regression Testing Guarantee the reliability of your network

2 The Challenge Progressive liberalization in the telecommunications market together with the evolution of technology, challenges the operators of wireless and wireline networks every day. The highly competitive market situation forces the frequent launch of new services and features in order to keep pace with their competitors. To operate a network of high quality and reliability is a key success factor for every operator. If the operators cannot fulfill these expectations, their image and reputation is negatively affected resulting in an increase in customer churn. Launching new Services in a live network is always a very costly task, combined with the risk that previously working functionality could be adversely affected by new software that has not had the time to be proved in the field. The most proactive way of avoiding unexpected or unwanted behavior of the live network is by doing on any new SW releases before deployment in the live network. Performing Acceptance and Regression tests on new SW releases provides the following benefits: The reliability of the live network and therefore the revenue stream is not affected, the current SW release remains in service as long as the new SW release is not finally tested and released. Churn rate reduction; the new SW release is only deployed after its reliability has been proven. Reduced repair costs; detected bugs have to be fixed only once and the deployment starts only after all tests have been finally been passed. Once the new software has passed all tests, it can be deployed with confidence in the network. There are different types of test methods widely used in the communications industry, each focusing to a specific aim: Interworking Testing - Verification of the correct interworking of different protocols and interfaces. Regression Testing: Verification of existing services and / or new services supported by a new software release. Interoperability Testing: Verification of the correct interworking behavior of network elements that are supplied by different vendors. Application Note Page 2

3 There are different possible methods how to perform such tests in order to verify whether the SUT (System Under Test) behaves as specified and expected. At the first glance, using regular phone sets such as 3G-2G mobile handsets, ISDN phones or VoIP terminals to set up calls can be a method to doing initial tests. But as soon as it comes to the question whether the SUT can cope with real conditions, this method is no longer sufficient. Depending on the type of SUT, realistic conditions can mean thousands of calls need to be setup in parallel. This is a typical task for a bulk Call Simulator. A Call Simulator replaces real phone sets and subscribers or parts of the entire network by simulating their behavior. Compared to testing with real handsets or actual network elements, using a Call Simulator brings the following advantages Tests are completely reproducible; they can be performed as many times as required. All tests performed are documented in detail, allowing fast and precise analysis of results. The Number of calls simulated can be freely defined and easily adjusted; from a few calls only up to hundreds of thousands of parallel calls thus allowing to verify the behavior of the System Under Test under various load conditions. Both the control plane (for signaling) and user plane (e.g. switch fabric, encoding- decoding units) are tested. Payload analysis e.g. Speech Quality is tested, to verify whether the operators high quality standards are being fulfilled. Proven test cases can be re-used and automatically executed on future SW releases, dramatically lowering test effort. Network elements e.g. Switches are very expensive, to replace them with a Call Simulator is a more cost effective solution. Application Note Page 3

4 Nexus8610 The test solution for Acceptance and Regression tests The 8610 Telecom Test System perfectly matches all these requirements by providing All physical interfaces from STM-1, Ethernet (10/100/1000 BaseT), TDM (E1/T1), ISDN BA (S0- T0 / Uk0) to Analogue POTS. All protocols to simulate individual wireless or wireline network users and complete core networks elements. Multi protocol multi interface capability to test different protocols running on different interfaces simultaneously. Multi-user capability, to execute different tests by different users in parallel. OPTEC: an easy to learn and use methodology of writing and executing test cases. OPTEC allows the handling of all different protocols such as: UMTS, GPRS, GSM, VoIP, SS7, ISDN and Analogue all in the same way. OPTEC supports fast and efficient test case generation without requiring programming skills. Call profile alteration in order to simulate real subscriber behavior under different load / traffic conditions. Powerful test functionality for in-depth user plane testing Speech Recognition for voice announcement verification Audio Monitor for direct audio access to individual selectable voice channels Speech channel verification with tones / bit patterns Speech Quality Analysis of individual selectable speech channels Test Automation application to minimize test time and costs as well as increasing the overall test coverage. Tests can be scheduled and executed 24x7. The test pilot can control different test equipment connected via telnet and/or RS-232. Application Note Page 4

Nexus8610 - Use cases The following chapter looks at typical use case of network operators that successfully use the Nexus8610 to perform Acceptance and Regression tests.

Task a manufacturer of 3G Mobile network elements needs to test the SGSN in order to be sure that all appropriate interfaces and functionality of the SGSN are implemented as specified see Figure 1.

5 Nexus Use cases The following chapter looks at typical use case of network operators that successfully use the Nexus8610 to perform Acceptance and Regression tests. Interworking Test Testing the interworking behavior of a network element by simulating all appropriate logical interfaces around the SUT. Task a manufacturer of 3G Mobile network elements needs to test the SGSN in order to be sure that all appropriate interfaces and functionality of the SGSN are implemented as specified see Figure 1. Nexus8610TU3 SGSN IuPS Gn Gf,Gd,Gr Gb Nexus8610TU Figure 1: Interworking test on a SGSN by using Nexus8610 Test Setup - The Nexus8610, configured with the OPTEC applications for GPRS Gb, Gf, Gd, Gr, Gn and UMTS IuPS is connected to the SGSN that is the System Under Test. The Nexus8610 simulates on the GPRS Gb interface GPRS mobile subscribers doing data transfer. Gn interface the Internet GPRS Gf, Gd and Gr the related network elements such as EIR, SMSC and HLR UMTS IuPS 3G mobile subscribers doing data transfer. Application Note Page 5

6 With this test setup, the SGSN can completely be tested without using expensive real network elements and components such as RAN, BSS, and HLR. Result performing regression tests with Nexus8610 provides the answers to the following questions Has the new functionality been implemented as specified? Does the functionality, taken form the earlier releases, still work correctly? Can the upgraded SUT manage real load conditions? Regression Test Testing the entire functionality of a network element in the case where a new SW release has been installed. The installation of a new SW release in a network element can affect the availability of existing functionality. Task a mobile operator needs to upgrade the SW of the MSC in the GSM network in order to provide further functionality to the mobile subscribers. Before the deployment of the new SW release 3.6 in the live network the operator wants to prove the new SW release. Nexus8610TU MAP Nexus8610TU GSM-A SS7 MSC Figure 2: MSC Regression test setup using Nexus8610 Test Setup - the Nexus8610, configured with the OPTEC applications for SS7 - ISUP, GSM - A and GSM MAP simulation, is connected to the MSC that is System Under Test. The Nexus8610 simulates on these interfaces the behavior of Application Note Page 6

7 GSM mobile subscribers on the GSM A interface(s). PLMN PSTN subscribers on the SS7 ISUP interface(s). HLR, SMSC and EIR on the GSM MAP interface(s). With this test setup, the MSC can completely be tested without using expensive real network components such as PSTN Switches, HLR or BSS or other costly elements. Regression testing of the new SW release 3.6 for the MSC is split in two parts; see Figure 3. The features and functionality of the current SW release 3.5 that are still part of the new SW release 3.6 are tested with the set of test cases that initially have been used to test the MSC SW release 3.5. Testing this part of the SW release 3.6 can be very efficient by using the Test Automation application of the Nexus8610 that allows fully unmanned automated test execution 24 X 7 For testing the additional features and functionality of the new SW release 3.6 the related test cases have to been added. # of testcases to test the MSC Testing MSC Rel 3.7 with - existing testcases from some new testcases Testing MSC Rel 3.6 with - existing testcases from some new testcases Features provided with Rel 3.7 Features provided with Rel 3.6 Features provided with Rel SW Rel of MSC Figure 3: Principle of Regression testing Application Note Page 7

8 Result doing regression tests provides the answers to the following questions Has the new functionality been implemented as specified? Does the functionality, taken form the earlier release, still work correctly? Interoperability Test Testing the interworking behavior of network elements supplied by different vendors to ensure that all different network elements are correctly working together. Task a wireline network operator needs to upgrade the SSP (Service Switching Point) of the IN (Intelligent Network) with a new SW Release. To ensure the acceptability of the SW upgrade, the operator wants to test the functionality of the new SW release in a test lab environment before the new SW release is deployed in the live network. Test Setup The Nexus8610 configured with the OPTEC INAP application is used to simulate the behavior of the SCP towards SSP (Service Switching Point) and the IP (Intelligent Peripheral), see Figure 4. The Nexus8610 simulates the SCP by receiving/sending the appropriate transactions from/ to the real SSP / IP. Figure 4: SSP-IP Interoperability verification in a multivendor IN environment with Nexus8610 Application Note Page 8

9 Result By simulating the behavior the SCP on the INAP interfaces, the following questions can be answered: Does the upgraded SSP provide the new functionality as expected? Does the upgraded SSP still correctly interwork with the SCP? Does the upgraded SSP still correctly interwork with the IP? Abbreviations ATM Asynchronous Transfer Mode EIR Equipment Identification Register GPRS General Packer Radio Service GSM Global System for Mobile Communications HLR Home Location Register ISDN Integrated Services Digital Network MSC Mobile Switching Center OPTEC Optimized Testcase Composer SGSN Serving GPRS Support Node SMSC Short Message Service Center SS7 Signaling System #7 STM-1 Synchronous Transfer Mode - 155Mbps SUT System Under Test TDM Time Division Multiplex UMTS Universal Mobile Telecommunications System VoIP Voice over IP Application Note Page 9

10 Notice Every effort was made to ensure that the information in this document was accurate at the time of printing. However, information is subject to change without notice, and Nexus Telecom reserves the right to provide an addendum to this document with information not available at the time that this document was created. Copyright Copyright 2005 Nexus Telecom. All rights reserved. No part of this guide may be reproduced or transmitted electronically or otherwise without written permission of the publisher. Trademarks Nexus Telecom and its logo are trademarks of Nexus Telecom AG. All other trademarks and registered trademarks are the property of their respective owners. Revision History Nexus8610 Acceptance & Regression Testing Application Note Version Date Author Editon 2.0 May 2005 Franz Neeser All General Inquiries: Nexus Telecom AG System Solutions Feldbachstrasse 80 P.O. Box 215 CH-8634 Hombrechtikon Switzerland Tel Fax Nexus Telecom AG Wireless Network Systems Muertschenstrasse 27 P.O. Box 1413 CH-8048 Zürich Switzerland Tel Fax Nexus Telecom (Americas) Inc. (NA and CALA) Suite Prince of Wales Drive Ottawa, Ontario Canada K2C 3W7 Tel Fax Application Note Page 10

The most sophisicated and versatile VoIP test solution on the market. Nexus8610 Traffic Simulation System. Nexus8610 VoIP

The most sophisicated and versatile VoIP test solution on the market Nexus8610 Traffic Simulation System Nexus8610 VoIP The telecommunications industry is undergoing a fundamental shift in technology as