Variants and Traceability as the Challenge

Size: px

Start display at page:

Download "Variants and Traceability as the Challenge"

Derrick Wilcox
5 years ago
Views:

1 Variants and Traceability as the Challenge Model-Based Test Design as an Answer V

Motivation We are talking about automated ECU testing: Module tests, sub system tests, MIL, SIL, HIL Challenges for the test designer: Traceability Short change request cycles

2 Motivation We are talking about automated ECU testing: Module tests, sub system tests, MIL, SIL, HIL Challenges for the test designer: Traceability Short change request cycles ECU variants Maintainability Test variants Expandability Reuse High test coverage Reviews Low budget Detailed reporting Distributed development Reproducibility Role separation 2

3 Requirements Initially there is a list of requirements All requirements have to be covered by the test design Traceability has to be ensured Requirements differ in some points dependent on ECU variant 3

State Diagrams: Notation The expected behavior of the SUT is modeled as state diagram Test model to be created separately from functional model to find errors in the functional model Extension of

4 State Diagrams: Notation The expected behavior of the SUT is modeled as state diagram Test model to be created separately from functional model to find errors in the functional model Extension of model by test specific (self-)transitions, such as unspecified scenarios boundary value checks ambient light = threshold for 5s ambient light > threshold for 5s switch off switch on initialize cleanup Off switch on switch off On ambient light > threshold for <1s ambient light < threshold for 2s ambient light < threshold for 2s +/- 1s 4

State Diagrams: Test Case Generation Automatic generation of test cases, based on transition coverage Transitions are mapped to stimuli, states are mapped to checks Different algorithm to control

5 State Diagrams: Test Case Generation Automatic generation of test cases, based on transition coverage Transitions are mapped to stimuli, states are mapped to checks Different algorithm to control generated test cases, e.g. few ones vs. short ones Example 1: Chinese postman algorithm initialize check Off ambient light < threshold for 3s check On ambient light = threshold for 5s check Off Example 2: Breadth search algorithm Example 3: One test case for each transition switch off check Off initialize check Off switch on check On switch off check Off cleanup 5

State Diagrams: Separation of Graphic and Test Implementation The graphical model is separated from the test implementation The graphic itself is independent of the concrete SUT variant,

Parameters Reuse of graphic with different test implementations and parameter values void SetAmbientLight() { AmbientLight = LightThreshold + 500; } Different roles for the test design

6 State Diagrams: Separation of Graphic and Test Implementation The graphical model is separated from the test implementation The graphic itself is independent of the concrete SUT variant, SUT access, test implementation and HIL environment On Graphic ambient light > threshold for <1s Easily to be reviewed by developers, customers, quality management Test Implementation Parameters Reuse of graphic with different test implementations and parameter values void SetAmbientLight() { AmbientLight = LightThreshold + 500; } Different roles for the test design possible: test designer vs. responsible person for library functions and parameters void CheckOn() { Check Voltage > 10V; } void CheckOn() { } Check 10Hz < PWMFreq < 30Hz; Check 45% < PWMDC < 55%; 6

Test Sequence Diagrams: Notation In difference to state diagrams dedicated test sequences are modeled Direct specification of chronological command sequence Specific order of commands possible to

7 Test Sequence Diagrams: Notation In difference to state diagrams dedicated test sequences are modeled Direct specification of chronological command sequence Specific order of commands possible to stimulate a dedicated state of the SUT Variant dependent test sequences supported by dedicated graphical element Background observations may be activated for a specific period of time during a test sequence Check indicator frequency Init system Set rpm = 1300 Set rpm = 1000 Set rpm = 3000 Indicator left on If cornering light available Operate indicator lever left Cornering light activated Switch ignition on Indicator right on Operate indicator lever right Else Indicator off Switch indicator off 7

Test Sequence Diagrams: Test Case Generation Automatic generation of test cases, one test case

Variant 1: Cornering light available Variant 2: Cornering light not available Init system Switch

light activated Switch indicator off Check indicator off Init system Switch ignition on Set rpm =

8 Test Sequence Diagrams: Test Case Generation Automatic generation of test cases, one test case for each path through the diagram Transitions are mapped to stimuli, states are mapped to checks Variant 1: Cornering light available Variant 2: Cornering light not available Init system Switch ignition on Set rpm = 1000 Operate indicator lever left Check indicator left on Check cornering light activated Switch indicator off Check indicator off Init system Switch ignition on Set rpm = 1000 Operate indicator lever left Check indicator left on Switch indicator off Check indicator off 8

Comparison of Different Approaches State Diagrams Test Sequence Diagrams State-oriented model Sequence-oriented model

Several algorithm-based approaches for test One straightforward approach case generation for test case generation Both

design Whereas the notation and test case generation algorithm is different, orthogonal concepts like abstraction,

9 Comparison of Different Approaches State Diagrams Test Sequence Diagrams State-oriented model Sequence-oriented model Descriptive Specifying Concise, compact Stating, expressing What shall be tested? How shall be tested? Several algorithm-based approaches for test One straightforward approach case generation for test case generation Both approaches are valuable, they may co-exist Best suitable approach depends on SUT characteristics and requests on the test design Whereas the notation and test case generation algorithm is different, orthogonal concepts like abstraction, separation from test implementation etc. are the same Also the type of generated test cases is identical, no difference at execution time, in reporting, 9

10 Traceability Traceability is required for: Overview of test design coverage by the test designer himself Internal and external reviews Connection to test management tools Link requirements to tests directly in the graphical notation Each generated test case is automatically linked to all requirements covered by any of the contained graphical elements switch off ambient light = threshold for 5s ambient light > threshold for 5s Switch off dependent on ambient light switch on initialize cleanup Off switch on Switch on/off processing time switch off On ambient light > threshold for <1s Switch on dependent on ambient light ambient light < threshold for 2s ambient light < threshold for 2s +/- 1s 10

Traceability Matrix Overview of current test design coverage given by a traceability matrix Coverage indicated for specific variants as well as

11 Traceability Matrix Overview of current test design coverage given by a traceability matrix Coverage indicated for specific variants as well as for superset of variants Easy navigation from a requirement to related test case implementations Change management of requirements simplified 11

integration of vteststudio and CANoe allows direct access to complete system environment in vteststudio: signals,

12 Vector Tool Integration Editors for state diagrams and test sequence diagrams contained in the Vector test design tool vteststudio (*) Generated test cases can be executed with CANoe as test execution environment Close integration of vteststudio and CANoe allows direct access to complete system environment in vteststudio: signals, XCP variables, VT system, interfaces to 3 rd party tools, (*) State diagrams will be available with next main version Q1/

Connection to Test Management Tools Connection of

export exchange formats Test Management Tool

Switch on/off processing time Indicator frequency

light<threshold Failed run initialize, switch off,

13 Connection to Test Management Tools Connection of any test management tool possible by import and export exchange formats Test Management Tool Requirements: Test Cases: Test Execution Results: Switch on/off processing time Indicator frequency initialize, switch off, cleanup initialize, ambient light<threshold Failed run initialize, switch off, cleanup Passed run initialize, ambient light<threshold vteststudio CANoe 13

Summary Model-based test design as one solution for the handling of variants and traceability in the test design vteststudio and CANoe as the solution for model-based test design Fun for the test

14 Summary Model-based test design as one solution for the handling of variants and traceability in the test design vteststudio and CANoe as the solution for model-based test design Fun for the test designer: Traceability ECU variants Test variants Reuse Reviews Detailed reporting Reproducibility Short change request cycles Maintainability Expandability High test coverage Low budget Distributed development Role separation 14

For more information about Vector and our products please visit www.vector.com Author: Katranski, Ute Vector Germany 15 2016.

15 For more information about Vector and our products please visit Author: Katranski, Ute Vector Germany Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. V

Concept Manual vteststudio. Version 2.2 English

Concept Manual vteststudio. Version 2.2 English Concept Manual vteststudio Version 2.2 English Imprint Vector Informatik GmbH Ingersheimer Straße 24 D-70499 Stuttgart The information and data given in this user manual can be changed without prior notice.