Ptting the dynamic into software secrity testing Detecting and Addressing Cybersecrity Isses V1.1 2018-03-05
Code ahead! 2
Atomated vlnerability detection and triage + = 3
How did we get here? Vector was engaged with a large, US Tier 1 and we were addressing software qality They acknowledged they had software qality isses Concern was related to how these qality isses cold affect secrity Project goals morphed into low-hanging secrity frit (for both the cstomer and the attacker)! Or goal was more along the lines of robstness! 4
VectorCAST test atomation platform 5
Vector testing soltion Software System Link to Reqirements System Validation System Integration Test Software Integration Test Software Unit Test System validation + code coverage on ECU Change-Based Testing SW integration testing + code coverage on PC White-box testing on host / on target CANoe, vteststdio VT System VectorCAST/QA CANoe, vteststdio vvirtualtarget VectorCAST/C++ and /QA VectorCAST/C++ VectorCAST Analytics Software Implementation Benefits Fll spport in the development process, from software nit test to system validation Uniform test management, test atomation (CI), reslt analysis and traceability 6
Vlnerability detection via dynamic analysis The idea The approach To be able to identify and atomatically test for ndiagnosed secrity vlnerabilities Utilizes MITRE s classification of CWEs (common weakness enmeration) Once an instance of a generic CWE is fond in the software, that isse is then classed as a CVE (common vlnerability and exposre) Atomatically interrogate the code and identify possible weaknesses (a la static analysis) Once a potential CWE is fond, generate a test exploiting the identified isse and execte it (dynamic exection) After exection, analyses the exection trace and decide if the potential CWE is a genine threat Code CWEs Tests Exection Analysis CVEs 7
Vlnerability detection via dynamic analysis The benefits Weaknesses identified Unlike static analysis, this method will only flag an isse if we can generate an exploit, eliminating the false-positive isse plaging static analysis The generation of test artefacts allows for their ftre re-exection to demonstrate the mitigation of a potential isse after software redesign Can be sed for both on-host and on-target exection (think secrity validation for embedded systems) Via the analysis of open-sorce projects, a nmber of API-sage related isses have been identified A large US atomotive Tier 1 has sed it to find secrity-specific rese isses on their software platform Able to atomatically find isses sch as NULL pointer dereference (CWE-476), classic bffer overflow (CWE-120) and improper resorce shtdown/release (CWE-404) Atomated Validation 8
Two technical approaches Mtational (test-site) fzz testing Take an existing test-site Modify the vales to be randomly erroneos Rn it with coverage Does it crash? If yes: potential weakness! Directed ( intelligent ) secrity testing Identify an expression of interest > E.g., pointer dereference, divide by zero Generate a test reaching that line with erroneos vales Rn it with coverage Does it crash? If yes: potential weakness! 9
Example from lighttpd int bffer_copy_string_bffer(bffer *b, const bffer *src) { if (!src) retrn -1; } if (src->sed == 0) { b->sed = 0; retrn 0; } retrn bffer_copy_string_len(b, src->ptr, src->sed - 1); Not detected: CppCheck, Facebook s Infer, Uno Possible error: Lint, CodeHawk Programmatic error detected (SIGSEGV): VectorCAST 10
Secrity weaknesses of interest The approach is focsed on atomatically generating tests for a nmber of classifications of vlnerabilities according to MITRE At the highest level, we look to address the general banner CWE-398 ( indication of poor code qality ) Some examples of isses we aim to detect Hard errors Use of a NULL pointer (CWE-476) Bffer {nder,over}flow (stack corrption) (CWE-124) Divide by zero (CWE-369) Mismatched calls malloc/free, fopen/fclose, pthread_mtex_lock/pthread_mtex_nlock (CWE- 401/404/413/415/590) Bad argments memcpy (CWE-120/130) Unchecked retrn malloc (CWE-252/690) 11
12 Technical Approach
Atomated (mtational) fzz testing for nit testing Existing test-case: > TEST.VALUE:bffer.bffer_copy_string_bffer.src:<<malloc 1>> > TEST.VALUE:bffer.bffer_copy_string_bffer.src[0].sed:0 > TEST.VALUE:bffer.bffer_copy_string_bffer.b:<<malloc 1>> Maniplate the vales: > TEST.VALUE:bffer.bffer_copy_string_bffer.src:<<malloc 1>> > TEST.VALUE:bffer.bffer_copy_string_bffer.src[0].sed:0 > TEST.VALUE:bffer.bffer_copy_string_bffer.b:<<nll>> Execte! 13
From software to mathematics Replace x with code and 0 with nll pointer dereference 14
Directed test-case generation for weaknesses We combine in-depth static analysis with constraint solving to identify more complex weaknesses: > param_2->x += 3; > param_3->y += 2; > retrn param_1->z / (param_2->x - param_3->y); Fzz testing has to get lcky here, bt sing test-case generation we can directly generate a test sch that: > (param_2->x 3) (param_3->y 2) 0 This gets fed to a black box oracle that can provide the answer! 15
16 Real World Examples
Real examples fond (divide by zero) atomotive extern VEHICLE_T Vehicle; void check_speed(int8_t speed_thomph) { int32_t temp,tho_mph,tho_rpm; if(vehicle.wheelspeed<150) {} else { if(speed_thomph>1000) {} else { Tho_MPH=0; } } if(tho_mph==0) { // no change to Tho_MPH! } else {} There exists a nmber of paths throgh the code where Tho_MPH is nassigned (so ndefined behavior) or is assigned to zero What is srprising is that Tho_MPH is checked against 0 and then sed in a divide at the same scope-level No corrective action taken, even thogh the corrective condition is already detected! 17 } temp=(100*tho_rpm)/tho_mph;
Real examples fond (NULL pointer) medical STATUS process_lamp_event(lamp_pattern_t patternid, LAMP_TASK_DATA_t *ptrtaskdata) { DRV_RET_CODE_t drvretcode = DRV_RC_ERROR; STATUS retcode = OK; LAMP_PATTERN_t tmppatternid = patternid; A lot of the code for provided in this project was extremely jdicios in checking all parameters Their style of coding made this crash stand-ot, as ptrtaskdata is never checked for nll! if (((LAMP_FAST_BLINKING == ptrtaskdata->previospatternid) (LAMP_SLOW_BLINKING == ptrtaskdata->previospatternid)) && (LAMP_PATTERN_NONE == tmppatternid)) { tmppatternid = ptrtaskdata->previospatternid; } } ptrtaskdata->conter += ptrtaskdata->timeot_ms; 18
19 Actionable Intelligence
Actionable Intelligence Software metrics An approach to ascertaining qickly Chess Morningstar for Software Secrity absence of obvios reliability isses This similar to CWE-398 for poor code qality The easy ones Defect density Defects/SLoC Lines free from obvios isses (via code coverage) Confidence of defect freedom (bt not garanteed!) Ratio of secrity tests free of defects Higher ratio => more secre 20
Actionable Intelligence Open Sorce analysis Project Metric LIGHTTPD ZLIB LIBXML2 Version 1.4.20 1.2.8 2.9.4 # files 89 16 84 SLoC⁶ 36,605 6,726 184,179 Uniqe # isses 709 113 2,926 Defect density (defects/line) 1/52 1/60 1/63 Avg. # of tests per defect 11 7 12 Tests hitting defects 69% 28% 40% Fnctions with defects 44% 44% 29% Fnctions with vg 20 and defects⁷ 51% 55% 66% ⁶measred with cloc 21 ⁷Jones 08: [complexity] levels greater than 20 are considered hazardos
Take home Process Identify portfolio Assess vlnerabilities Manage risk Some of the isses we find yo might consider are non-isses or are mitigated against as part of yor software architectre That s great be wary abot software re-se across projects! Mainly: no one size fits all soltion se mltiple tools! Dynamic exection can find certain vlnerabilities more definitively Need to always consider DP-E ratio (damage potential vs. effort) 22
For more information abot Vector and or prodcts please visit www.vector.com 23 2018. Vector Informatik GmbH. All rights reserved. Any distribtion or copying is sbject to prior written approval by Vector. V1.1 2018-03-05