Struggles at the Frontiers: Persistent Pursuit of Software Assurance in the Development and Sustainment of Defense Systems Dr. Kenneth E.

Transcription

1 Struggles at the Frontiers: Persistent Pursuit of Software Assurance in the Development and Sustainment of Dr. Kenneth E. Nidiffer Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Struggles Struggles at the at Frontiers: the Frontiers: Persistent Persistent Pursuit Pursuit of Software of Assurance Software in the Assurance Development in the Development and Sustainment and of Defense Sustainment Systems of [[Distribution 1

2 Copyright 2017 Carnegie Mellon University Copyright 2017 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN AS-IS BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.. Please see Copyright notice for non-us Government use and distribution. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu. Carnegie Mellon and CERT are registered marks of Carnegie Mellon University. TSP SM DM

3 Software Enabled Systems are Today s Strategic Resource Software is the building material for modern society Software Oil Dr. Bill Scherlis* Steam Manual Labor Water Source: SEI Increasing globalization 3

4 SEI Works in Several Core Technical Areas Autonomy & Counter- Autonomy C4ISR Mission Assurance Cyber Missions Data Modeling & Analytics Human- Machine Interactions Software & Information Assurance Systems Verification & Validation Combine to Provide Software and Cyber Capabilities 4

5 Context Definition*: The level of confidence that software functions as intended and is free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software throughout the life cycle. Perspective: Persistent pursuit of software quality assurance is a constant struggle Future: Software enabled systems underpin the cyber environment, enabling explorations into new frontiers *Source: DoDI Protection of Mission Critical Functions to Achieve Trusted Systems and Networks (TSN), and 2013 NDAA S933 Source: SEI 5

6 Software Assurance is a Constant Purpose, and Software- Enabled Systems are Moving Targets Constant purpose Software assurance provides the required level of confidence that software functions as intended (and no more) and is free of vulnerabilities, either intentionally or unintentionally designed or inserted in software, throughout the lifecycle* Moving target The changing and expanding role that software plays in cyberspace means that the development of software-enabled systems must continue to evolve while we pursue software quality *Source: DoDI Protection of Mission Critical Functions to Achieve Trusted Systems and Networks (TSN), and 2013 NDAA S933 6

7 Software Assurance Struggles at the Frontiers 1. Increasing complexity of software assurance 2. Satisfying unique operational mission and business needs 3. Solving the SDLC* software tool chasm 4. Addressing limited spectrum availability 5. Handling the expanding code base 6. Understanding attack patterns, vulnerabilities, and weaknesses 7. Increasing vulnerabilities 8. Designing-in software quality over the lifecycle 9. Reducing technical debt 10. Working in the infancy of software engineering discipline * SDLC: System Development Life Cycle 7

8 Increasing Complexity of Software Assurance First Line of Defense in Software Assurance is the Application (Software) Layer 84% of breaches exploit vulnerabilities in the Application 1 Yet, funding for IT defense vs. software assurance is 23-to Clark, Tim, Most cyber Attacks Occur from this Common Vulnerability, Forbes Feiman, Joseph, Maverick Research: Stop Protecting Your Apps; It s Time for Apps to Protect Themselves, Gartner G

9 SEI Technical Themes SEI Technologies Address DoD Software Assurance Challenges In the broad sense as defined by the DoD challenges and underscored by the SEI Technical Insight. Focus Areas: the behavior as built: architecture evaluation, code analysis, security patterns, etc. the behavior during evolution: system monitoring and self-adaptation, etc. the affordability: cost estimation, sustainment work, etc. the quality: defect modeling, architectural fault modeling, requirements engineering, etc. the mission success: acquisition support, acquisition dynamics, acquisition and architecture alignment, etc. 9

10 Satisfying Unique Operational Mission and Business Needs Practical Answers for a Complicated World Intelligent Integrated Solutions Source: SEI 10

11 Addressing Limited Spectrum Availability Spectrum-based Technologies - Key to DoD Mission and Commercial Business Needs DoD Mission Increasing spectrum needs Commercial Business Increasing spectrum needs Exploding requirements for information and mobility! CAGR = compound annual growth rate Source: Kenneth R. Turner, Dep. Director, Spectrum Policy and International Engagements, DoD Chief Information Officer 11

12 Handling the Expanding Code Base Software Is Dramatically Expanding with Limited Natural Governance Size of Codebase (SLOC) Google = 2,000 MSLOC Healthcare.gov Debian 5.0 (all software in package) Software in typical new car, 2013 Mac OS X 10.4 Debian 5.0 codebase US Army's Future Combat System Facebook (without backend code) Windows Vista (2007) Microsoft Visual Studio 2012 Large Hadron Collider Microsoft Office (2013) Windows XP (2001) Windows 7 Symbian Microsoft Office for Mac (2006) Windows 2000 (2000) Microsoft Office (2001) F-35 Fighter Apache Open Office Linux 3.1 (recent version, 2013) Android (upper estimate) Boeing 787, total flight software Source: David McCandless, Information is Beautiful, 21 September 2016 web retrieval Millions of Source Lines of Code 12

13 Understanding Attack Patterns, Vulnerabilities, and Weaknesses water known threat actors flooding attack patterns (CAPEC) hatch door in sub weaknesses & vulnerabilities (CWE/CVE) close door actions* System & System Security Engineering Trades sub fills w/water technical impacts sub sinks operational impacts Attack Weakness Item Impact Asset Attack Weakness Item Impact Function Attack Weakness Impact Asset Weakness/ Vulnerability Reported Item Source: Bob Martin, MITRE Actions include architecture choices; design choices; added security functions, activities, and processes; physical decomposition choices; static and dynamic code assessments; design reviews; dynamic testing; and pen testing. Vulnerability is the intersection of three elements: a system susceptibility or flaw, attacker access to the flaw, and attacker capability to exploit the flaw. 13

14 Increasing Vulnerabilities Reported Common Vulnerabilities and Exposures (CVE) CVE 1999 to WEAKNESSES VULNERABILITIES CVEs (reported, publicly known vulnerabilities and exposures with patches) Unreported or undiscovered Vulnerabilities Zero-Day Vulnerabilities (previously unmitigated weaknesses that are exploited with little or no warning) Uncharacterized Weaknesses CWEs (characterized, discoverable, possibly exploitable weaknesses with mitigations) Source: Dr. Robert A. Martin, MITRE Corporation, August MITRE 14

15 Designing-in System Acquisition Quality Over the Lifecycle Continuous Engineering Over The Life Cycle Understand SwA Risks Design Principles Design Inspection Secure Coding Practices Code Inspection Test Coverage Metrics Pen Test Monitor Threats/ Attacks Respond to Incidents Defects Upgrade SwA Requirements Decision Developmental RFP Decision Full Rate Production Decision Review A B C MDD Materiel Solution Analysis AOA Technology Maturation & Risk Reduction Engineering & Manufacturing Development ICD Production and Deployment Operation and Sustainment ASR SRR SFR PDR CDR TRR SVR OTRR IOT&E IOC FOC Criticality Analysis Threat Vuln Analysis SwA Counter- Measures RFP Eliminate Design Flaws Sound Arch & Design Identify Code Flaws More Secure Code Identify Test Gaps Better Test Suites Vuln Root Cause Fewer Process Vuln Effective Threat Response Cyber Situational Awareness Cyber Hardened Systems Software assurance must be baked-in throughout entire system lifecycle Source: OSD/DASD/SE Distribution Statement A Approved for public release. Distribution is unlimited. 15

16 SEI Supporting JFAC* Pilot Program Focused on Solving the SDLC Software Assurance Tool Chasm The JFAC is a federation of DoD organizations that have a shared interest in promoting software and hardware assurance in defense acquisition programs, systems, and supporting activities. SEI provides experience SEI technical staff to support the dynamic and evolving needs of the JFAC, such as the SDLC software assurance tool chasm *JFAC: D0D Joint Federated Assurance Center 16

17 Reducing Technical Debt Engineering Assurance into the Fabric of Programs Right Direction Source: PERFORMANCE OF THE DEFENSE ACQUISITION SYSTEM 2015 ANNUAL REPORT Sept 16,

18 Reducing Technical Debt Aircraft Software Development and Rework Cost Reference: U.S. Air Force Scientific Advisory Board. Sustaining Air Force Aging Aircraft into the 21st Century (SAB-TR-11-01). U.S. Air Force,

19 Working in the Infancy of the Software Engineering Discipline Integrate Software and Systems Engineering with a Focus on Cyber Physical Science Bioscience Computer/Software/Cyber Science Origins/History Begun in antiquity Begun in antiquity Mid-20th century Enduring Laws Framework of Scientific Study R&D and Launch Cycle Laws are foundational to furthering exploration in the science Four main areas: astronomy, physics, chemistry, and earth sciences Laws are foundational to furthering exploration in the science Science of dealing with health maintenance and disease prevention and treatment Only mathematical laws have proven foundational to computation Several areas of study: computer science, software/systems engineering, IT, HCI, social dynamics, AI All nodes are attached to and rely on a netted system years years Significantly compressed; solution time to market must happen very quickly HCI: human computer interaction; AI: artificial intelligence Source: SEI 19

20 Software Assurance Struggles at the Frontiers 1. Increasing Complexity of Software Assurance 2. Satisfying unique operational mission and business needs 3. Solving the SDLC Software Tool Chasm 4. Addressing limited spectrum availability 5. Handling the expanding code base 6. Understanding attack patterns, vulnerabilities, and weaknesses 7. Increasing vulnerabilities 8. Designing-in software quality over the lifecycle 9. Reducing Technical Debt 10. Working in the infancy of software engineering discipline 20

21 SEI Technologies Address DoD Software Assurance Challenges Today s Presentations of SEI s Contributions Software Vulnerabilities Track - Relationship Between Design Flaws and Software Vulnerabilities - Predicting Software Bug Closure Rates and Reliability Growth with Weibull Modeling - Using Malware Analysis to Identify Overlooked Security Requirements Modernization and Sustainment Track - Case Study - Successful Government Roadmap Modernization Project - Panel - Software Sustainment: Continuous Engineering to Deliver Warfighter Capability - Methodology for the Cost Benefit Analysis of a Large Scale Multiphasic Software Enterprise Migration 21

22 Questions? 22

23 Contact Information Dr. Kenneth E. Nidiffer, Director of Strategic Plans for Government Programs Software Engineering Institute, Carnegie Mellon University Office: Fax: nidiffer@sei.cmu.edu 23

24 References Gallagher, B.; Nidiffer, K. & Saga, R. The Ordered Process for Improving Agile Engineering Outcomes, CrossTalk, Nov/Dec 2016 JFAC Collaboration Portal SEI Training: Manage Cybersecurity Risk Across the Lifecycle - The CERT Software Assurance Framework (SAF) Assessment SEI Training: Architecture Tradeoff Analysis Method (ATAM) SEI Training: Security Requirements Engineering Using the SQUARE Method SEI Training: Team Software Process (TSP) Source Code Analysis Laboratory (SCALe), SEI Technical Note, CMU/SEI-2012-TN- 013, 2013 Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE), Technical Report, CMU/SEI-2011-TR-026, 2011 Alberts, C.; Woody, C.; & Dorofee, A. Evaluating Security Risks using Mission Threads(CMU/SEI-2014-TN-025),