By V-cubed Solutions, Inc. Page1
Purpose of Document This document will demonstrate the efficacy of CODESCROLL CODE INSPECTOR, CONTROLLER TESTER, and QUALITYSCROLL COVER, which has been developed by V-cubed Solutions, Inc., and can be used to satisfy the software verification and validation requirements documented in the ISO 26262 standard. For more information about how to support ISO 26262, please contact us. Introduction Currently in the automotive industry market, software quality has become very important. Successful car Original Equipment Manufacturer (OEMs) must innovate by introducing new technologies, many of which contain ever more complex embedded software systems. The automobile has been transformed from a mechanical device into an integrated machine with embedded software in all major parts, including: engine control unit, power train, braking, power steering, infotainment system and telematics. The exponential growth of software has brought a dramatic increase in software defects. Software testing has been considered a very high cost task, but the cost of detecting and fixing software bugs before incidents occur versus the direct costs and damaged product branding associated as a result of product recalls makes thorough software testing a necessity in the automotive industry. To help address vehicle safety, improved quality and reliability, ISO 26262 for road vehicle functional safety was developed and released in 2013. This industry standard was created to provide guidance to avoid the risk of systematic failures and random hardware failures through feasible requirements and processes. In terms of the process of software development, this standard provides very detailed levels of process with activities and evidence from the first phase of software development: concept development to encompass all phases of vehicle level testing. The descriptions for the standards required regarding design principles in unit design and implementation, unit testing, integration testing are very specific. In practical real world scenarios it is very difficult to develop software that complies with each of the requirements of the standard, especially during the software testing phase. This step alone takes a great deal of time and investment. Even under the best circumstances, when all of the resources and cost issues are solved, there are still problems from lack of expertise in software testing. V-cubed Solutions, Inc. solves these kinds of problems by using automated software testing tools, named CODESCROLL CODE INSPECTOR, CONTROLLER TESTER and QUALITYSCROLL COVER, that are extremely efficient and effective, when compared to other methods. Page2
How CODE INSPECTOR, CONTROLLER TESTER and COVER Supports Compliance with the ISO 26262 Standard CODE INSPECTOR, CONTROLLER TESTER and COVER for ISO 26262 satisfy the most of requirements for software testing and verification specified in Part 6 of the standard for software development by supporting the code inspection and creation/execution of test cases. The code inspection can be useful to prove whether the software complies with the designs principle requirements and the creation/execution of test cases can be useful to prove whether all of software requirements have been tested or not. CONTROLLER TESTER and COVER can support this effort by measuring the structural coverage through software testing for all Automotive Safety Levels() required by ISO 26262. CONTROLLER TESTER also supports with its robust testing activities the fault injection testing for each unit and for the integrated modules. ISO 26262 uses one of four s (A, B, C or D) to specify the item s or element s necessary safety requirements for achieving an acceptable level of risk, with D representing the highest risk and A the least risky level. Elements with an of D are expected to be tested with the highest level of rigor. Page3
Introduction of CODESCROLL CODE INSPECTOR, CONTROLLER TESTER and QUALITYSCROLL COVER CODESCROLL CODE INSPECTOR can enable organizations to define the kinds of coding guidelines that will be applied to the software source code. It provides the pre-defined coding guideline, such as MISRA C:2004, MISRA C:2012 and MISRA C++:2008 and some high severity rules which have been proven in the real field. Moreover, CODE INSPECTOR can analyze the software source code with simple click on the icon or menu that will show detailed information regarding violations. These violations can be one of the main causes for very critical failures of vehicles. It is really hard task to remove all the violations, which have been detected. To make it easier, CODE INSPECTOR guides the user how to fix them through example codes. CODESCROLL CODE INSPECTOR Detecting Violation Page4
CODESCROLL CONTROLLER TESTER can help the tester design and execute the test cases for unit testing and integration testing. CONTROLLER TESTER analyzes the software source code line by line and automatically generates the test cases based on the result of analyzing the data (global variable, parameters, local variables, etc.) and control flow of each function. CONTROLLER TESTER can measure the structural coverage, such as statement coverage, branch coverage and MC/DC. It shows all of results for executing test cases with coverages. And if there is any lack of structural coverage, the tester can design more test cases with the help of CONTROLLER TESTER by using the control flow graph with coloring block and flow-edges. And it provides the feature for using the mass actual data as the test cases and signal generation as like what HILs(Hardware-in-the-loop simulation) testing tools does. Both features are very helpful to test the software as much similar as possible with actual vehicle level testing environment. CODESCROLL CONTROLLER TESTER Unit/Integration Testing Page5
QUALITYSCROLL COVER can enable the tester measure the completeness of testing in the integration level testing, system level testing, and even vehicle level testing. It measures the function coverage and call coverage, which are required in integration level testing by ISO 26262. It also accumulates all of the history of the previous testing results, so the engineer related to the functional safety, software testing and software quality, can know how much testing has been done and what status of quality has been achieved. QUALITYSCROLL COVER Measuring Structural Coverage Page6
ISO 26262 Part 6 Compliance Tables ISO 26262 Part 6 addresses product development at the software level, including several requirements that must be considered in order to achieve compliance with ISO 26262. The following tables describe where CODESCROLL CODE INSPECTOR, CONTROLLER TESTER and QUALITYSCROLL COVER can be used to ensure and demonstrate compliance. Each of requirements is described through the tables below along with the recommendation to use the corresponding method depends of the. The categories are as follows: Symbols Description ++ Indicates that the requirement is highly recommended for the identified. + Indicates that the requirement is recommended for the identified. Indicates that the requirement has no recommendation for or against its usage for the o identified. Software Modeling and Coding Guidelines As part of the initiation of the development phase at the software level, ISO 26262 created a set of coding guidelines which are described in Table 1 s to be covered by modelling and coding guidelines. The CODESCROLL CODE INSPECTOR supports these guidelines in the following manner: ISO 26262 Part 6 Table 1 - s to be covered by modelling and coding guidelines 1a Enforcement of low complexity ++ ++ ++ ++ 1b Use of language subsets ++ ++ ++ ++ 1c Enforcement of strong typing ++ ++ ++ ++ Use of defensive implementation 1d techniques o + ++ ++ 1e Use of established design principles + + + ++ CODE INSPECTOR measures the cyclomatic complexity for each function with other kinds of metrics. CODE INSPECTOR detects non-standard language constructs. CODE INSPECTOR detects unsafe casting and flag the occurrence as a defect. CODE INSPECTOR enforces defensive programming by detecting the violation such as, whether all value of parameters, variables are within valid range or not before using it. CODE INSPECTOR basically provides suitable rules to check the source code has no violation based on the design Page7
1f Use of unambiguous graphical representation + ++ ++ ++ 1g Use of style guides + ++ ++ ++ 1h Use of naming conventions ++ ++ ++ ++ principle. Through the feature of userdefined rules, custom rule can be created to test for specific violations of select design principles. This requirement is not applicable to C/C++, Java. CODE INSPECTOR can detect the violation related to convention. Through the feature of user-defined rules, custom rule can be created to test for specific violations of style guidelines Software Unit Design and Implementation The standard supplies numerous guidelines for software design and implementation to ensure the correct order of execution, consistency of interfaces, correctness of data flow and control flow, simplicity, readability, and comprehensibility and robustness. ISO 26262 Part 6 Table 8 Design principles for software unit design and implementation One entry and one exit point in 1a subprograms and functions ++ ++ ++ ++ No dynamic objects or variables, or else 1b online test during their creation + ++ ++ ++ 1c Initialization of variables ++ ++ ++ ++ 1d No multiple use of variable names + ++ ++ ++ Avoid global variables or else justify their 1e usage + + ++ ++ 1f Limited use of pointers o + + ++ Page8 to check whether a function shall have only one exit point or not. to check that dynamic memory assignment related functions such as malloc(), calloc(), realloc() and free(), shall not be used. Also, it can check whether there are inspection before using the variable assigned dynamically. to check whether variables are assigned a value before using them or not. to check whether the variable names have been duplicated or not. to check whether this used variable are used. to check whether usage of pointer is existed or not.
1g No implicit type conversions + ++ ++ ++ to check an implicit type conversion in expression or not. 1h No hidden data flow or control flow + ++ ++ ++ to check whether there are hidden data flow or control flow in function or not. 1i No unconditional jumps ++ ++ ++ ++ to check whether there are statement goto with unconditional jumps or not. 1j No recursions + + ++ ++ to check whether any function is called by itself directly/indirectly or not. ISO 26262 Part 6 Table 9 Methods for the verification of software unit design and implementation 1a Walk-through ++ + o o Not applicable 1b Inspection + ++ ++ ++ Not applicable 1c Semi-formal verification + + ++ ++ Not applicable 1d Formal verification o o + + Not applicable 1e Control flow analysis + + ++ ++ CODE INSPECTOR uses the method of control flow analysis. 1f Data flow analysis + + ++ ++ CODE INSPECTOR uses the method of data flow analysis. 1g Static code analysis + ++ ++ ++ CODE INSPECTOR uses the method of static code analysis. 1h Semantic code analysis + + + + Not applicable Software Unit Testing Software unit testing is an important requirement in the ISO 26262 standard. Software unit tests must be planned, specified and executed based on the procedure. ISO 26262 Part 6 Section 9 describes the objectives of software unit testing as to demonstrate that the software units fulfill the software unit specifications and do not contain undesired functionality. To fulfill the requirement, the standard recommends the following unit testing methods to be implemented. ISO 26262 Part 6 Table 10 Methods for software unit testing 1a Requirements-based test ++ ++ ++ ++ Page9 CONTROLLER TESTER can provide efficient requirements-based testing
environment as possible. It consists of the source code under test with traceability to the requirements and the associated test code, usually in the form of test driver and test stubs. CONTROLLER TESTER generates this environment automatically. 1b Interface test ++ ++ ++ ++ CONTROLLER TESTER generates the test cases with various way, such as boundary value analysis for each interface, illegal values, min/middle/max values. Here, the goal is to prevent the errors through ensuring that the interface is robust. 1c Fault injection test + + + ++ CONTROLLER TESTER offers the facilities for introducing faults, including corrupt values of specified variables, in order to test the safety mechanisms in the function. As CONTROLLER TESTER supports the test stubs, the stub function can be used to introduce intentional errors in the middle of the function. 1d Resource usage test + + + ++ Not applicable CONTROLLER TESTER can use the data from the model simulation as a test 1e data. It supports to use the external Back-to-back comparison test between + + ++ ++ mass data and signal generator as like model and code, if applicable HILS operation. Therefore, it is available to compare the result of test between using the model and using source code. Also the standard asks the following methods for deriving test cases for software unit testing. ISO 26262 Part 6 Table 11 Methods for deriving test cases for software unit testing 1a Analysis of requirements ++ ++ ++ ++ As explained in the previous table, the requirements-based testing is supported by CONTROLLER TESTER. Usually, the test cases for unit testing can be derived from low-level software requirements which normally are defined at the critical design phase. If the source code can be linked Page10
1b Generation and analysis of equivalence classes + ++ ++ ++ 1c Analysis of boundary values + ++ ++ ++ 1d Error guessing + + + + to specific requirements, the test cases can be linked to requirements, too. CONTROLLER TESTER supports the type and variable partitioning through using equivalence classes analysis. The most of automatically generated test data are based on ranges and lists of values for local/global variable, constants and parameters. These test data can be combined in linear mode, pair-wise and full mode. Execution of these complex combination test cases is done automatically. CONTROLLER TESTER can prevent the potential software errors at parameter limits or boundaries where it is most likely to fail. CONTROLLER TESTER can find automatically the value at, approaching and exceeding the boundaries of the interface. Also CONTROLLER TESTER can test illegal values, and even special values such as Not-A-Number, positive and negative infinity on floating-point variables. Thus, using CONTROLLER TESTER will guarantee that the range of boundary values tested and it does not matter with type of bits. CONTROLLER testers supports that tester can insert easily the additional test cases, guessing the error condition. There are several ways to edit the test cases, such as inserting, deleting and changing value. User can export the previously run test cases and open it with spread sheet tools or text editor, then simply modifying it. Another way is just simply using UI directly. It provides a grid type UI to edit the test cases. During the software unit testing, the standard asks to evaluate the completeness of test cases and to demonstrate that there is no unintended functionality. And the coverage of requirements at the software unit level shall be determined and the structural coverage shall be measured as like following. Page11
ISO 26262 Part 6 Table 12 Structural coverage metrics at the software unit level 1a Statement coverage ++ ++ + + 1b Branch coverage + ++ ++ ++ MC/DC(Modified Condition/Decision 1c Coverage) + + + ++ CONTROLLER TESTER provides the simple to use structural coverage viewer with colored source code editor. And also it provides control flow graph with coloring to show the covered flow and uncovered flow. Through looking at the colored control flow graph and source code editor, it helps developer and tester to design additional test cases to meet the requirement of structural coverage. Also, it provides the MC/DC table to show which condition and decision has been tested. Software Integration and Testing Software integration and testing is the next phase after software unit testing is done. And like the software unit testing, software integration and testing is also must be planned, specified and executed based on the procedure. ISO 26262 Part 6 Section 10 describes the objectives of software integration and testing to integrate the software elements to demonstrate that the software architectural design is realized by the embedded software. To fulfill the requirement, the standard recommends the following integration testing methods to be implemented. ISO 26262 Part 6 Table 13 Methods for software integration testing 1a Requirements-based test ++ ++ ++ ++ Page12 CONTROLLER TESTER can provide efficient requirements-based testing environment as possible. It consists of the source code under test which is the set of several units to be one component or module with traceability to the requirements and the associated test code, usually in the form of test driver and test stubs. CONTROLLER TESTER generates this environment automatically.
1b Interface test ++ ++ ++ ++ CONTROLLER TESTER generates the test cases with various way, such as boundary value analysis for each interface of component or module and their illegal values, min/middle/max values. Here, the goal is to prevent the errors through ensuring that the interface is robust. 1c Fault injection test + + ++ ++ CONTROLLER TESTER offers the facilities for introducing faults, including corrupt values of specified variables for the component or module, in order to test the safety mechanisms in the component or module. As CONTROLLERT TESTER supports the test stubs, the stub function can be used to introduce intentional errors in the middle of function of a component or module. 1d Resource usage test + + + ++ Not applicable CONTROLLER TESTER can use the data from the model simulation as a test data. It supports to use the external Back-to-back comparison test between 1e + + ++ ++ mass data and signal generator as like model and code, if applicable HILS operation. Therefore, it is available to compare the result of test between using the model and using source code. Also the standard asks the following methods for deriving test cases for software integration testing. ISO 26262 Part 6 Table 11 Methods for deriving test cases for software integration testing 1a Analysis of requirements ++ ++ ++ ++ As explained in the previous table, the requirements-based testing is supported by CONTROLLER TESTER. Usually, the test cases for integration testing can be derived from high-level software requirements which normally are defined in the architectural design phase. If the source code can be linked to specific high-level requirements, the test cases can be linked to requirements, too. Page13
1b Generation and analysis of equivalence classes + ++ ++ ++ 1c Analysis of boundary values + ++ ++ ++ 1d Error guessing + + + + CONTROLLER TESTER supports the type and variable partitioning through using equivalence classes analysis. The most of automatically generated test data are based on ranges and lists of values for local/global variable, constants and parameters which usually are for entry point of the component. These test data can be combined in linear mode, pairwise and full mode. Execution of these complex combination test cases is done automatically. CONTROLLER TESTER can prevent the potential software errors at parameter limits or boundaries where it is most likely to fail. CONTROLLER TESTER can find automatically the value at, approaching and exceeding the boundaries of the interface for components or modules. Also CONTROLLER TESTER can test illegal values, and even special values such as Not-A-Number, positive and negative infinity on floating-point variables. Thus, using CONTROLLER TESTER will guarantee that the range of boundary values tested and it does not matter with type of bits. CONTROLLER TESTER supports that tester can insert easily the additional test cases, guessing the error condition. There are several ways to edit the test cases, such as inserting, deleting and changing value. User can export the previously run test cases and open it with spread sheet tools or text editor, then simply modifying it. Another way is just simply using UI directly. It provides a grid type UI to edit the test cases. During the software integration testing, the standard asks to evaluate the completeness of each test and to obtain evidence that there is no unintended functionality. The coverage of requirements at the software architectural level by test cases shall be determined and the structural coverage shall be measured as follows. Page14
ISO 26262 Part 6 Table 15 Structural coverage metrics at the software architectural level 1a Function coverage + + ++ ++ 1b Call coverage + + ++ ++ CONTROLLER TESTER provides the simple to use function and call coverage viewer with colored source code editor. And also it provides call graph with different color to show the covered calling and uncovered calling. COVER provides the feature to measure the structural coverage, such as function coverage, call coverage. Also, it traces the history of how the result of coverage has been changed through several testing. The result of coverage is shown as the percentage for each function (method), class, module, files, and the whole project with tree map. Certification CODESCROLL CODE INSPECTOR, CONTROLLER TESTER and QUALITYSCROLL COVER have been certified for compliance activities with ISO 26262. Please contact V-cubed Solutions, Inc. for more details. Conclusion CODESCROLL and QUALITYSCROLL provides full line-up tools for automating software testing, including static analysis, software unit testing, and software integration testing, in a way that makes complying with MISRA and ISO 26262 requirements much more efficient. All of the tools can generate detailed documents for reporting the testing result in HTML, MS-Excel, MS-Word, and PDF format, which have been successfully used in the actual field to comply with a number of international industries standards related to functional safety. CODESCROLL CODE INSPECTOR is used to perform static analysis on C/C++ software source code to validate whether the software source code fulfill with the requirement of ISO 26262, such as coding guidelines, design principles for software unit design. It is configured to check the MISRA C:2004, MISRA C:2012 and MISRA C++:2008. Page15
CODESCROLL CONTROLLER TESTER enables the execution of software unit testing and software integration testing on a simulator or a target board. It can measure the structural coverages, which are required by ISO 26262. QUALITYSCROLL COVER is used to measure the structural coverage, such as function coverage and call coverage at the software integration level testing and system level testing such as the ISO 26262 standard requires. Achieve Higher Quality of Software in Functional Safety Industry V-cubed Solutions, Inc. has been a key partner to improve our customer s software quality so that it will meet the highest possible standards in the Functional Safety Industry. For over thirteen years with the fully automated software testing tools, CODESCROLL and QUALITYSCROLL, tools have been used by the following industries: Defense, Nuclear Power Plant, Avionics, Automotive, Railway and Industrial Controls. Hundreds of companies and organization already use CODESCROLL and QUALITYSCROLL to improve the quality of their embedded software About V-cubed Solutions, Inc. V-cubed Solutions, Inc. is the leader of software testing industries, especially automated software testing tools for embedded software developers. Also V-cubed Solutions, Inc. provides the independent V&V services for the customer with the state-of-art testing technologies. V-cubed Solutions, Inc. 3003 North First Street, Suite 336 San Jose, CA 95134 USA Tel: +1(408) 519 5777 Fax: +1(408) 519 5784 Email: sales@v-cubed.net Suresoft Technologies, Inc. 3F, 536 Eonju-ro, Gangnam-gu, Seoul, 135-917, Korea Tel: +82 2 3446 5462 Fax: +82 2 3446 5463 Email: global@suresofttech.com Page16