An overview on the. version 3 (TTCN-3)

Transcription

1 An overview on the Testing and Test Control Notation version 3 (TTCN-3) Helmut Neukirchen Í Háskóli Íslands helmut@hi.is

2 Outline 1. About me 2. What is TTCN-3? 3. TTCN-3 3example 4. Test automation with TTCN-3 5. Conclusions Helmut Neukirchen: TTCN-3 2

3 1. About me Helmut (í. Helmút) Neukirchen: Born and educated in Germany: University of Aachen, University of Lübeck, University of Göttingen. PhD in Computer Science. Since 2008: Associate professor for Computer Science and Software Engineering at University of Iceland. Working in the field of software quality and distributed systems, in particular testing distributed systems and quality of tests. Test methodology, test languages, test tools, test metrics, test smells, test refactoring, test patterns, standardisation of tests. If you have interesting testing problems for students to work on as part of their BSc, MSc, PhD project: contact me! Office in Tæknigarður, best contacted via Helmut Neukirchen: TTCN-3 3

4 2. What is TTCN-3? Testing and Test Control Notation version 3 Test specification and test implementation language. Test cases specified at an abstract level. Hardware and operating system independent. d Can be turned into automated test cases by compiling into executable code, adding an adaptation layer. Standardised technology: European Telecommunication Standards Institute (ETSI). International Telecommunication Union (ITU). Strength in functional black-box testing of distributed systems. However, applicable to various other domains, levels, and types of tests: Embedded d systems, distributed ib t d systems, pure software systems. From unit test to acceptance test level. From functional to non-functional (real-time, load, performance) tests. Helmut Neukirchen: TTCN-3 4

5 3. TTCN-3 example: Scenario Black-box test t of a message-based weather service: Test System System Under Test (SUT) 1. Weather request( Reykjavik ) Test case 2. Weather response ( Reykjavik, , 3 C, 9m/s) Weather service Purpose of a very simple example test case: Send weather request for Reykjavik to SUT. Check that weather response is received containing: location Reykjavik any date, temperature within a reasonable range ( C) wind speed within a reasonable range (0..100m/s). Helmut Neukirchen: TTCN-3 5

6 TTCN-3 example: Test architecture Test Sy ystem Test component Out Ports In Test behaviour is executed by a test component. Test component Test component Abstract Test System Interface Real Test System Interface System Under Test (SUT) TTCN-3 supports distributed, parallel l test t components. Specified and implemented using TTCN-3 language Test adapter (implemented using Java, C, C++ etc.) Black-box Weather service example: SUT=Weather service, Simple example test case: only one test component needed. However, for some load test, we could, e.g., run the same test case in parallel on multiple test components to generate some load. Helmut Neukirchen: TTCN-3 6

7 TTCN-3 example: Definition of data types and test data module WeatherServiceTestSuite { type record weatherrequest { charstring location Definition of data types. template t weatherrequest t ReykjavikWeatherRequest R t := { location := "Reykjavik" type record weatherresponse { charstring location, charstring date, integer temperature, integer windvelocity it Templates= Definition of test data. template weatherresponse ReykjavikWeatherResponse := { location := "Reykjavik", date := *, Definition iti of test t data: powerful temperature := ( ), matching mechanism allows windvelocity := (0..100) fuzziness (wildcards, ranges, regular expressions). Helmut Neukirchen: TTCN-3 7

8 TTCN-3 example: Definition of ports and interfaces type port weatherport message { in weatherresponse; out weatherrequest; type component TestComponentType tt { port weatherport weatheroffice; Definition of a port used for communication: allowed messages (=data types). Definition of a test component type: ports that may be used by test behaviour running on an instance of that component type. Helmut Neukirchen: TTCN-3 8

9 TTCN-3 example: Definition of test behaviour testcase testweather() runs on TestComponentType { weatheroffice.send(reykjavikweatherrequest); Send message specified by template via port. alt { [] weatheroffice.receive(reykjavikweatherresponse) i ikw R { setverdict(pass) [] weatheroffice.receive { setverdict(fail) t(f il) // End of testcase Test component type used to run test case behaviour. Two expected alternatives: Either expected response (i.e. matched by template) is received (PASS) or any other response is receive (FAIL). control { execute(testweather()) Specification of order (and, e.g., conditional execution) of test cases. // End of module Helmut Neukirchen: TTCN-3 9

10 TTCN-3: Further concepts Not presented in the previous example: Procedure-based communication, Timer and catching timeouts, Distributed testing using parallel test components, Concepts from general-purpose programming languages: Conditional statements (if-then-else, for, while, ), Local variables, Functions, Parametrisation. Helmut Neukirchen: TTCN-3 10

11 4. Test automation ti with TTCN-3 TTCN-3 3test tcases are abstract: t weatheroffice.send(reykjavikweatherrequest) makes no statement how the actual communication shall be implemented: UDP datagram? SOAP over http? SMS over GSM?? how the data to be communicated shall be encoded: XML? Proprietary ASCII-based format? Proprietary binary format?? Helmut Neukirchen: TTCN-3 11

12 Test adaptation ti Automatically ti executable TTCN-3 test suite = Compiled TTCN-3 test cases + CoDecs (encode data to be sent to SUT / decode data received from SUT) + SUT adapter (communication with SUT) + Platform adapter (operating system specific) Test Control [compiled] Test tool user interface SUT Adapter Logging TTCN-3 Test System TTCN-3 Executable Codecs Platform Adapter System Under Test (SUT) Note: TTCN-3 execution tools come with default test control, logging, and platform adapter. Helmut Neukirchen: TTCN-3 12

13 TTCN-3 3&R Re-usability Separation of concerns (abstract t test t case description vs. concrete adaptors): Re-use TTCN-3 test cases for different SUTs: Same, unmodified weather service test case can be used for SOAP/http-based system under test, for SMS/GSM-based system under test, etc. Re-use adaptors for different test cases: A SOAP/http SUT adaptor can be used for testing all kinds of Web services, an SMS/GSM SUT adaptor can be used for testing all kinds of mobile bl services, etc. Helmut Neukirchen: TTCN-3 13

14 TTCN-3 3tools Tools for, e.g., editing, compiling, deploying, executing, debugging TTCN-3: Telelogic Tester (IBM/Rational/Telelogic, Sweden), TTworkbench (Testing Technologies, Germany), OpenTTCN Tester for TTCN-3 (OpenTTCN, Finland) TTCN-3 toolbox (Danet, Germany), Exhaustif/TTCN (Métodos y Tecnología, Spain), MessageMagic (ELVIOR, Estonia), TTCN-3 Express (Fraunhofer FIRST/Metarga, Germany). TRex TTCN-3 Refactoring and Metrics tool (University of Göttingen/University of Iceland, Germany/Iceland) Standardised language and test-tool tool interfaces: No vendor lock-in! Helmut Neukirchen: TTCN-3 14

15 5. Conclusions TTCN-3 benefits: Standardised test technology supporting automated tests. Well documented, many tools, no vendor lock-in. Compatible with any test methodology and test management approach. Fosters re-use. Mature technology used by industry and standardisation. Nokia, Ericsson, Siemens, Motorola, Huawei, ETSI, WiMax forum, Open Mobile Alliance, Automotive Open System Architecture, TTCN-3 standard freely available (via ETSI European Standard , ITU-T Recommendation Z.140 TTCN-3 risks: Heavyweight approach (may only pay off for big, distributed systems): Compilers are expensive (>1000 ). Another language and technology to learn. While Unit Testing is possible (and done) with TTCN-3, I would in most cases recommend a xunit framework for Unit Testing. Helmut Neukirchen: TTCN-3 15

16 Thank you for your attention! ti Make sure to use the right tools! Helmut Neukirchen: TTCN-3 16