Verification & Validation of Open Source 2011 WORKSHOP ON SPACECRAFT FLIGHT SOFTWARE Gordon Uchenick Coverity, Inc
Open Source is Ubiquitous Most commercial and proprietary software systems have some open source component 2
Open Source in Embedded Software According to Gartner, by 2012, 80% of commercial software development projects will include open-source components Popularity of Linux kernel Even proprietary operating systems have absorbed some open source components 3
The Challenges When Using Open Source Incorporating open-source isn t free or effortless: 1. Quality Risk each developer tests according to his own requirements, probably limiting scope 2. Unit testing and integration testing have unknown rigor and coverage 3. Two ways of getting issues resolved: 1. Report issues upstream and wait for patches 2. Dedicate internal resources to maintain open source and then feed fixes back into the community 4
How Static Analysis Can Help Fewer defects escape dev Reduced Risk wrt Quality, Budget, & Schedule Design Development Quality Assurance Product Release Automated, Scalable, Fast Finds and reports defects in all parts of the code as well as defects due to integration
How Static Analysis Works Build Analyze Present & Manage Integrates with existing build systems Mimicks the behavior of dozens of compilers Statically tests all execution paths Finds defects and inconsistent coding patterns Explains the location and root cause of defects Manage and share triage of defects across teams
Static Analysis for Managing Risk from Open Source 1. Use static analysis to automate defect detection across the entire code base, including open source components 1. Having all source code instead of just API contracts enables a more complete analysis 2. Fixes can be verified 1. Did I fix the problem? 2. Did I break anything else? 7
Most Commonly Found Defects In Open Source Frequency in SCAN projects Risk NULL Pointer Dereference 27.60% Medium Resource Leak 23.19% High Unintentional Expressions 9.76% Medium Uninitialized Values Read 8.41% High Use After Free 5.91% High Buffer Overflow 5.52% High 8
Coverity SCAN: Accelerating Open Source Software Integrity Established in 2006 in collaboration with US Department of Homeland Security http://scan.coverity.com 2009 2010 Total LOC scanned 11.5 billion 14.5 billion Total Open Source Projects analyzed 280 291 Total Defects Found 38,453 49,654 Total Defects Fixed 11,246 15,278 9
Resources Coverity SCAN project: http://scan.coverity.com Software Integrity Risk Report: http://www.coverity.com/forrester-software- Integrity-Risk/ 2010 SCAN Report: http://softwareintegrity.coverity.com/2011scana ndroidreg.html 10
Thank You
Appendix A few Static Analysis examples
C/C++ Defects That Coverity Can Find Part 1 Resource Leaks Memory leaks Resource leak in object Incomplete delete Microsoft COM BSTR memory leak Uninitialized variables Missing return statement Uninitialized pointer/scalar/array read/write Uninitialized data member in class or structure Concurrency Issues Deadlocks Race conditions Blocking call misuse Integer handling issues Improper use of negative value Unintended sign extension Improper Use of APIs Insecure chroot Using invalid iterator printf() argument mismatch Memory-corruptions Out-of-bounds access String length miscalculations Copying to destination buffers too small Overflowed pointer write Negative array index write Allocation size error Memory-illegal access Incorrect delete operator Overflowed pointer read Out-of-bounds read Returning pointer to local variable Negative array index read Use/read pointer after free Control flow issues Logically dead code Missing break in switch Structurally dead code Error handling issues Unchecked return value Uncaught exception Invalid use of negative variables
C/C++ Defects That Coverity Can Find Part 2 Program hangs Infinite loop Double lock or missing unlock Negative loop bound Thread deadlock sleep() while holding a lock Null pointer differences Dereference after a null check Dereference a null return value Dereference before a null check Code maintainability issues Multiple return statements Unused pointer value Insecure data handling Integer overflow Loop bound by untrusted source Write/read array/pointer with untrusted value Format string with untrusted source Performance inefficiencies Big parameter passed by value Large stack use Security best practices violations Possible buffer overflow Copy into a fixed size buffer Calling risky function Use of insecure temporary file Time of check different than time of use User pointer dereference
C/C++ Resource Leaks Resource leaks occur when variables go out of scope while owning a resource Memory leaks are one of the most common kinds of resource leaks Technical Impact Crashes, inability to allocate more resources, Vulnerability to denial of service attacks 15
Example C++ Memory Leak Allocating memory into member field Destructor does not free member field 16
Example Memory Leak Allocated names Allocated other variables Checking for allocation failures for all variables Freeing the wrong variable in cleanup code. Cut and paste error? names leaked Bonus: potential double free 17
C++ Memory Leak with Incorrect Delete Constructor for each object allocates field Allocating array of objects Using delete instead of delete[] means destructor is not called, leaking the memory in the fields. 18
Other C/C++ Resource Leaks Coverity also detects incorrect delete patterns such as: delete a, b; // comma operator means only a is deleted Resource leaks on handles that refer to files, sockets, and other system resources 19
C/C++ Memory Corruption Memory corruption occurs when programs write to memory outside of the bounds of memory buffers Buffer overflow is another common name for memory corruption Technical Impact Crashes, unexpected behavior, security vulnerabilities 20
Example Memory Corruption Table has 6 ints, valid indices are 0..5 Loop from 0 to sizeof(table) Table indexed past the end of the array because sizeof(table) is measured in bytes 21
Example Memory Corruption Assigning ident to a constant string Freeing constant string causes memory corruption 22
Example Double Free Bonus: memory leak on buf3 Freeing buf1 Freeing buf1 again, copy and paste error 23
Example Buffer Overflow Possible security vulnerability through stack buffer overflow 24
Example String Buffer Escape C++ string s is destroyed when function returns, making the pointer returned from c_str() invalid. The caller of this function will find stack garbage there, caused unexpected behavior and possibly a crash. 25
Other Memory Corruption Defects Coverity Can Detect Coverity looks for over a dozen different patterns of memory corruption including: String length miscalculations Copying to too small destination buffers Negative array index write Allocation size error Integer overflow 26