Automotive Software Security Testing

Size: px

Start display at page:

Download "Automotive Software Security Testing"

Byron Mason
5 years ago
Views:

1 Detecting and Addressing Cybersecurity Issues V

2 Code ahead! 2

3 Automated vulnerability detection and triage + = 3

4 How did we get here? Vector was engaged with a large, US Tier 1 and we were addressing software quality They acknowledged they had software quality issues Concern was related to how these quality issues could affect security Project goals morphed into low-hanging security fruit (for both the customer and the attacker)! Our goal was more along the lines of robustness! 4

5 VectorCAST test automation platform 5

Software System Link to Requirements VectorCAST Analytics Automotive Software Security Testing Vector testing solution System Validation System Integration Test

VectorCAST/QA CANoe, vteststudio vvirtualtarget VectorCAST/C++ and /QA Software Unit Test White-Box testing on host / on target VectorCAST/C++ Software Implementation

6 Software System Link to Requirements VectorCAST Analytics Automotive Software Security Testing Vector testing solution System Validation System Integration Test Software Integration Test System validation + code coverage on ECU Change-Based Testing SW integration testing + code coverage on PC CANoe, vteststudio VT System VectorCAST/QA CANoe, vteststudio vvirtualtarget VectorCAST/C++ and /QA Software Unit Test White-Box testing on host / on target VectorCAST/C++ Software Implementation Benefits Full support in the development process, from software unit test to system validation Uniform test management, test automation (CI), result analysis and traceability 6

7 Vulnerability detection via dynamic analysis The idea The approach To be able to identify and automatically test for undiagnosed security vulnerabilities Utilizes MITRE s classification of CWEs (common weakness enumeration) Once an instance of a generic CWE is found in the software, that issue is then classed as a CVE (common vulnerability and exposure) Automatically interrogate the code and identify possible weaknesses (a la static analysis) Once a potential CWE is found, generate a test exploiting the identified issue and execute it (dynamic execution) After execution, analyses the execution trace and decide if the potential CWE is a genuine threat Code CWEs Tests Execution Analysis CVEs 7

8 Vulnerability detection via dynamic analysis The benefits Weaknesses identified Unlike static analysis, this method will only flag an issue if we can generate an exploit, eliminating the false-positive issue plaguing static analysis The generation of test artefacts allows for their future re-execution to demonstrate the mitigation of a potential issue after software redesign Can be used for both on-host and on-target execution (think security validation for embedded systems) Via the analysis of open-source projects, a number of API-usage related issues have been identified A large US automotive Tier 1 has used it to find security-specific reuse issues on their software platform Able to automatically find issues such as NULL pointer dereference (CWE-476), classic buffer overflow (CWE-120) and improper resource shutdown/release (CWE-404) Automated Validation 8

Two technical approaches Mutational (test-suite) fuzz testing Take an existing test-suite Modify the values to be randomly erroneous Run it with coverage Does it crash? If yes: potential weakness!

9 Two technical approaches Mutational (test-suite) fuzz testing Take an existing test-suite Modify the values to be randomly erroneous Run it with coverage Does it crash? If yes: potential weakness! Directed ( intelligent ) security testing Identify an expression of interest > E.g., pointer deref., divide by zero Generate a test reaching that line with erroneous values Run it with coverage Does it crash? If yes: potential weakness! 9

10 Security weaknesses of interest The approach is focused on automatically generating tests for a number of classifications of vulnerabilities according to MITRE At the highest level, we look to address the general banner CWE-398 ( indication of poor code quality ) Some examples of issues we aim to detect Hard errors Use of a NULL pointer (CWE-476) Buffer {under,over}flow (stack corruption) (CWE-124) Divide by zero (CWE-369) Mismatched calls malloc/free, fopen/fclose, pthread_mutex_lock/pthread_mutex_unlock (CWE- 401/404/413/415/590) Bad arguments memcpy (CWE-120/130) Unchecked return malloc (CWE-252/690) 10

11 Automated (mutational) fuzz testing for unit testing Existing test-case: > TEST.VALUE:buffer.buffer_copy_string_buffer.src:<<malloc 1>> > TEST.VALUE:buffer.buffer_copy_string_buffer.src.used:0 > TEST.VALUE:buffer.buffer_copy_string_buffer.b:<<malloc 1>> Manipulate the values: > TEST.VALUE:buffer.buffer_copy_string_buffer.src:<<malloc 1>> > TEST.VALUE:buffer.buffer_copy_string_buffer.src.used:0 > TEST.VALUE:buffer.buffer_copy_string_buffer.b:<<null>> Execute! 11

12 From software to mathematics Replace x with input and 0 with null pointer dereference 12

13 Directed test-case generation for weaknesses VectorCAST combines in-depth static analysis with constraint solving to identify more complex weaknesses: > param_2->x += 3; > param_3->y += 2; > return param_1->z / (param_2->x - param_3->y); Fuzz testing has to get lucky here, but using test-case generation we can directly generate a test such that: > (param_2->x 3) (param_3->y 2) 0 This gets fed to a black box oracle that can provide the answer! 13

14 Take home Process Identify portfolio Assess vulnerabilities Manage risk Some of the issues we find you might consider are non-issues or are mitigated against as part of your software architecture That s great be wary about software re-use across projects! Mainly: no one size fits all solution use multiple tools! Dynamic execution can find certain vulnerabilities more definitively Need to always consider DP-E ratio (damage potential vs. effort) 15

For more information about Vector and our products please visit www.vector.com Dr Andrew V. Jones andrew.jones@vector.com 16 2018.

15 For more information about Vector and our products please visit Dr Andrew V. Jones Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. V

Putting the dynamic into software security testing

Putting the dynamic into software security testing Ptting the dynamic into software secrity testing Detecting and Addressing Cybersecrity Isses V1.1 2018-03-05 Code ahead! 2 Atomated vlnerability detection and triage + = 3 How did we get here? Vector was