Benefits of Collecting Code Coverage Metrics during HIL/ECU Testing Jeffrey Fortin Product Manager VectorCAST V0.1 2018-10-30
Agenda 1. How VectorCAST fits into HIL/ECU Testing 2. Demo 3. Questions and Answers 2
Software System Link to Requirements VectorCAST Manage and Analytics How VectorCAST fits into HIL/ECU Testing Vector Testing Solution Test Tools for all Test Phases System Validation System Integration Test SW Integration Test System validation + code coverage on ECU Change-Based Testing SW integration testing + code coverage on PC CANoe, vteststudio VT System VectorCAST/QA CANoe, vteststudio vvirtualtarget VectorCAST/C++/QA SW Unit Test White-Box testing on host / on target VectorCAST/C++ VectorCAST/RSP SW Implementation Benefits Full support in the development process, from SW unit test to system validation Uniform test management, test automation (CI), result analysis and traceability 3
How VectorCAST fits into HIL/ECU Testing Software Unit Test VectorCAST/C++ (VectorCAST/Ada) Environment to perform software unit and integration tests Automatically generates an executable test harness including test drivers + stubs Instrument software to be tested to collect coverage date for coverage analysis purposes Define test cases to stimulate the functions of the UUT, provide stub results to the UUT and check the expected results Analyze test results and code coverage VectorCAST/RSP Execute test cases in an embedded target or simulator environment Supports 35+ compiler families and hundreds of chip/board/rtos combinations Benefits Provide capabilities for white box testing on the host and on the target 4
How VectorCAST fits into HIL/ECU Testing Software Integration Test CANoe and vvirtualtarget Execute ECU software on simulated environment (PC) Simulate ECU software, e.g. for partly simulated systems Test simulated ECU software, e.g. for integration tests SWC A SWC B SWC C Use CANoe as test execution system, apply all CANoe test and analysis features Design tests with vteststudio RTE / VFB Service-Emulation VectorCAST/QA Environment to perform software system tests (functional tests) Instrument source code of the system for code coverage analysis Analyze code coverage Benefits Enable software integration testing with code coverage on the PC 5
How VectorCAST fits into HIL/ECU Testing System Test CANoe and VT System Real-time execution of tests Quasi industry standard on many test benches Many open interfaces for testing purposes are available Design tests with vteststudio Modular system of interface hardware to access the I/O and bus network lines All necessary functions integrated > I/O and measurement hardware > Relay switches and fault injection > Signal conditioning and signal processing > Dedicated test functions (decade resistor, ) VectorCAST/QA Instrument source code for code coverage analysis of embedded software Allows Change-Based Testing Benefits Get code coverage data during system tests and provide capabilities to perform Change-Based Testing 6
How VectorCAST fits into HIL/ECU Testing VectorCAST and CANoe Complements Each Other System Testing CANoe System Testing CANoe and VectorCAST/QA Unit Testing VectorCAST/C++ ECU I/O (Bus, A/D) ECU I/O (Bus, A/D) Test Driver CANoe is used to perform testing of an ECU via the ECU s external interfaces. Adding VectorCAST/QA exposes the completeness of testing, providing a detailed look on the internal structure of the embedded software. Adding VectorCAST/C++ allows testing of hard-to-reach sections of the application. 7
How VectorCAST fits into HIL/ECU Testing Improve Test Execution Efficiency with Change-Based Testing Instrumented Source Source Code Code Source code traceability Test Cases Do we have to execute all test cases if the source code changes? Code Change Without Change-Based Change-Based Testing Testing Running Code coverage all tests provides take a long traceability time between source Many code and tests test are cases: complicated to run Which source code is covered by which test case Test results are difficult to interpret VectorCAST determines which test cases have been affected by a source code change Only test what has changed to reduce test time, find errors faster and improve testing efficiency 8
How VectorCAST fits into HIL/ECU Testing Support for Continuous Testing (CT) and Continuous Integration (CI) Objective Achieve fast turnaround cycles Find errors as fast as possible Support for parallel/virtualized and headless test execution Automatically identify TCs that have to be executed because: > Code changes > Previous Test Failures > Changed and new requirements > Safety critical requirements > VectorCAST Enterprise Testing Create suites of atomic tests that can be easily run in multiple configurations Support massively parallel testing via CI servers such as Jenkins Support Change-Based Testing to run only the tests affected by code changes CI support already available Start Test COM Script 9 In CANoe via the COM interface In VectorCAST via command line and Python scripting CANoe Test Result
How VectorCAST fits into HIL/ECU Testing Using VectorCAST with CANoe Objective: Provide a detailed look on the internal structure of the embedded software during CANoe system tests Instrument source code of SUT for code coverage with VectorCAST/QA Automatically execute tests with VectorCAST/QA and CANoe, system test cases are created with vteststudio Analyze code coverage with VectorCAST/QA ECU System under Test Functionality on the granularity of test runs already available Tight integration to fully support test case oriented coverage planned 10
Agenda 1. How VectorCAST fits into HIL/ECU Testing 2. Demo 3. Questions and Answers 11
Demo Benefits of Collecting Code Coverage Metrics during HIL/ECU Testing 12
Demo Summary Provide a comprehensive test solution based on well established tools for all kind of industries using and developing embedded systems Enhance system testing with a more detailed look on the software structure including code coverage Combine the strengths of system testing with unit testing, e.g. combine test results for an overall test analysis Provide easy-to-use tools for all testing activities Become the major player for test and validation solutions for embedded software and embedded systems (Automotive, IoT and other industries) Integrate VI s system and function testing portfolio with the code based portfolio of VSW Extend the existing testing solution to increase efficiency increase quality 13
Agenda 1. How VectorCAST fits into HIL/ECU Testing 2. Demo 3. Questions and Answers 14
Questions and Answers Benefits of Collecting Code Coverage Metrics during HIL/ECU Testing?! 15
More Information! Visit our Website for: > News > Products > Demo Software > Support > Workshops > Contact Addresses www.vector.com Author: Jeffrey Fortin Vector 16 2018. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. V0.1 2018-10-30