Software. Test. Architecture. Dr Sigrid Eldh. Ericsson AB Stockholm, Sweden. Public Ericsson AB Page 1

Transcription

1 Software Test Architecture Dr Sigrid Eldh Radio System & Technology Software Ericsson AB Stockholm, Sweden Public Ericsson AB Page 1 Twitter: DrSEldh

2 The Bug Busters 127 Automated Test Cases Public Ericsson AB Page 2

3 Why do we need a Test Architecture? To be able to Maintain Test Cases efficiently Public Ericsson AB Page 3

4 Why do we need a Test Architecture? To be able to Conquer Complexity Public Ericsson AB Page 4

5 Why do we need a Test Architecture? To be able to Make enhancements to Test, Verification and Validation Public Ericsson AB Page 5

6 Why do we need a Test Architecture? To be able to Provide information & support for Fault diagnoses, Quality information other Metrics Public Ericsson AB Page 6

7 Why do we need a Test Architecture? To be able to Support a large number of people in the test realization Public Ericsson AB Page 7

8 Why do we need a Test Architecture? To be able to To handle a variety of tools, test environment, quality criteria, the SUT & changes in these Public Ericsson AB Page 8

9 Why do we need a Test Architecture? Protect our test effort from a changing world Public Ericsson AB Page 9

10 What is Test Architecture? Test Execution model Test Case generation model Interconnect w. requirements/ and other design models, error handling systems, built systems etc Test System configuration Re-execution of same test case in a variety of versions Re-execution of same test case in simulators, desktop env. or target environment Languages (TCL/TK, TTCN, Erlang, Perl, Java etc), and execution style Public Ericsson AB Page 10

11 What is Test Architecture? Test System with general abstractions and patterns Supporting Test Management A Component model Groupings of TC reflecting system Test Systems internal communication generation of test cases, test oracle, Progress Managing Test Environment e.g. Test suite start, stop, failover and restart during execution Availability/persistence mechanisms for data and processing Monitoring of Test and Maintenance Support for Fault and Error handling Log-files, Automatic reports, Verdicts Public Ericsson AB Page 11

12 Is Architecture Organization = Structure? Architecture is Topological Organization [Bhansali92] Software Architecture in Practice [Bass/Clements/Kazman03] The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them. Ok and for TEST????? Public Ericsson AB Page 12

13 The Evolution of Test Automation Organizing Test Cases Public Ericsson AB Page 13 Public Ericsson AB Page 13

14 Conquer Complexity? Divide et impera Divide and Conquer Julius Caesar BC Public Ericsson AB Page 14 Public Ericsson AB Page 14

15 Test (case) Structure Division of functions (elements) Layers e.g. System, integration, component, unit Group/classes Based on type of test Based on software architecture/org - e.g. object/component The phases of the execution (start, execute, verdict) Public Ericsson AB Page 15 Public Ericsson AB Page 15

16 Test (case) Structure Representation of shared information Communications between functions (or their relationship) This Environment Externally visible properties of elements Meta information about the test cases Parameter boundaries Public Ericsson AB Page 16 Public Ericsson AB Page 16

17 Architecture II An architecture, as we intend to use the term, consists of (a) a partitioning strategy and (b) a coordination strategy The partitioning strategy leads to dividing the entire system in discrete, non-overlapping parts or components. The coordination strategy leads to explicitly defined interfaces between those parts [Crispen 94] Public Ericsson AB Page 17 Public Ericsson AB Page 17

18 Test Strategies? Process & Plan & Test Architecture Public Public Ericsson Ericsson AB AB Page Page 18 18

19 Test Tool Frameworks Partitioning of execution handling in agile CI Different goals Different times (fast feedback) Different issues Prioritization of Test Tool Frameworks and support Templates is not enough If you do not know, understand and can understand the consequences and limits, what difference it makes Competence, competence, competence Public Ericsson AB Page 19

20 The Agile Story of Automated Build and Test Public Ericsson AB Page 20

21 Common misconceptions Architecture and design are the same thing Test architectures constraints the test design Architecture and infrastructure are the same thing Test framework is not sufficient (Test) Architecture is just structure No, it is not ONLY the order and WHEN the test cases are executed (or generated for that matter) Public Ericsson AB Page 21 Public Ericsson AB Page 21

22 Public Ericsson AB Page 22 Public Ericsson AB Page 22 Public Ericsson AB Page 22

23 Characteristics Of a Test System Usability Maintainability Performance Installability/Configurability Stability - Robustness Dependability Supportability Flexibility over time Public Ericsson AB Page 23

24 Other things to consider System size and lifespan Team size Team geographical distribution Team willingness Explainability Extensibility and Evolvability Componentization Price - Performance Public Ericsson AB Page 24 Public Ericsson AB Page 24

25 Increasing issue in Industry Lack of Test Architects University Courses? Need a balance of formalism, rigor and flexibility Modeling at the right level & still retain realization support We need to evaluate, understand & better define Test Architectures to solve the ISSUES Public Ericsson AB Page 25 Public Ericsson AB Page 25 Public Ericsson AB Page 25

26 Public Ericsson AB Page 26 Public Ericsson AB Page 26 Twitter: DrSEldh

27