Software Security and CISQ. Dr. Bill Curtis Executive Director

Transcription

1 Software Security and CISQ Dr. Bill Curtis Executive Director

2 Why Measure IT Applications? Six Digit Defects now affect Board of Directors CEO, COO, CFO Business VPs Corporate Auditors CIO accountable for Governance Risk management Risk measurement Brand protection Customer experience Evaluate Application Quality with CISQ Measures Consortium for IT Software Quality 1

3 What is CISQ? Co-founders IT Executives CISQ Technical Experts OMG Special Interest Group CISQ is chartered to define automatable measures of software size and quality that can be measured in the source code, and promote them to become Approved Specifications of the OMG CISQ Sponsors 2

4 CISQ/OMG Standards Process Automated Function Points Approved Measure Specifications Reliability CISQ Exec Forum Performance Efficiency OMG ISO Fasttrack Security Maintainability Deployment Workshops 3

5 Content of CISQ Measures CISQ Quality Characteristic Measures Example architectural and coding violations composing the CISQ measures Security 22 violations (Top 25 CWEs) SQL injection Cross-site scripting Buffer overflow Reliability 29 violations Empty exception block Unreleased resources Circular dependency Performance Efficiency 15 violations Expensive loop operation Un-indexed data access Unreleased memory Maintainability 20 violations Excessive coupling Dead code Hard-coded literals 5

6 How Do CISQ Measures Relate to ISO? ISO series replaces ISO/IEC 9126 (Parts 1-4) ISO defines quality characteristics and sub-characteristics CISQ conforms to ISO quality characteristic definitions ISO defines measures, but not at the source code level CISQ supplements ISO with source code level measures Software Product Quality Functional Suitability Reliability Performance Efficiency Operability Security Compatibility Maintainability Portability Functional appropriateness Accuracy Maturity Availability Fault tolerance Recoverability Time behavior Resource utilization Appropriateness Recognizability Learnability Ease of use Confidentiality Integrity Non-repudiation Accountability Co-existence Interoperability Modularity Reusability Analyzability Changeability Adaptability Installability Replaceability Attractiveness Technical Accessability Authenticity Modification stability Testability CISQ automated quality characteristic measures highlighted in blue 9

7 The 22 CWEs in the Security Measure CWE-22 Path Traversal Improper Input Neutralization CWE-78 OS Command Injection Improper Input Neutralization CWE-79 Cross-site Scripting Improper Input Neutralization CWE-89 SQL Injection Improper Input Neutralization CWE-120 Buffer Copy without Checking Size of Input CWE-129 Array Index Improper Input Neutralization CWE-134 Format String Improper Input Neutralization CWE-252 Unchecked Return Parameter of Control Element Accessing Resource CWE-327 Broken or Risky Cryptographic Algorithm Usage CWE-396 Declaration of Catch for Generic Exception CWE-397 Declaration of Throws for Generic Exception CWE-434 File Upload Improper Input Neutralization CWE-456 Storable and Member Data Element Missing Initialization CWE-606 Unchecked Input for Loop Condition CWE-667 Shared Resource Improper Locking CWE-672 Expired or Released Resource Usage CWE-681 Numeric Types Incorrect Conversion CWE-706 Name or Reference Resolution Improper Input Neutralization CWE-772 Missing Release of Resource after Effective Lifetime CWE-789 Uncontrolled Memory Allocation CWE-798 Hard-Coded Credentials Usage for Remote Authentication CWE-835 Loop with Unreachable Exit Condition ('Infinite Loop') Robert Martin MITRE Common Weakness Enumeration cwe.mitre.org 7

8 Factors and non-factors in CWE density per KLOC Factor Non-Factors 30 Releases per Year 35 Source 30 Shore CWEs per KLOC CWEs per KLOC CWEs per KLOC

9 CWE-89 SQL injection # of weaknesses Total SQL Injection weaknesses Apps ordered by # of weaknesses # of opportunities Total SQL Injection opportunities No weaknesses Apps ordered by # of weaknesses Statistic Value Mean 1.3 Median 0 Mode 0 Std. Deviat. 5.9 Range 4 Minimum 0 Maximum 4 Count 139 Range Freq. % to to to >50 0 SQL opportunities occurred in 9% of all apps SQL weaknesses occur in 9% of checked apps 3% of checked apps have extensive weaknesses Weaknesses unrelated to # of opportunities

10 CWE-79 Cross-site scripting # of weaknesses Total Cross-site Scripting weaknesses Apps ordered by # of weaknesses Statistic Value Mean 2.8 Median 0 Mode 0 Std. Deviat. 9.8 Range 86 Minimum 0 Maximum 86 Count 135 Range Freq. % to to to > # of opportunities Total X-Site Scripting Opportunities No weaknesses Apps ordered by # of weaknesses X-site opportunities occurred in 9% of all apps X-site weaknesses occur in 1/3 of checked apps 5% of checked apps have extensive weaknesses Weaknesses unrelated to # of opportunities

11 Modern Apps Are a Technology Stack 1 Unit Level Code style & layout Expression complexity Code documentation Class or program design Basic coding standards Developer level Architectural APIs Web Services Spring Oracle Struts EJB Transaction Risk JSP Java PL/SQL Sybase SQL Server Java T/SQL ASP.NET COBOL DB2 Java IMS Data Flow Java Java Java Java Single language/technology layer Intra-technology architecture Hibernate Intra-layer dependencies Messaging Inter-program invocation.net Security vulnerabilities Development team level 3 2 Integration quality Architectural compliance Risk propagation Application security Resiliency checks Transaction integrity Technology Level System Level Function point Effort estimation Data access control SDK versioning Calibration across technologies IT organization level 8

12 Emerging CISQ Measures Automated Function Points Must measure functional and non-functional code segments Automated Enhancement Points Extensions to Embedded Software Must add future effort to fix bugs into productivity Quality- Adjusted Productivity Four Quality Characteristic Measures Must estimate the corrective costs in in future releases Automated Technical Debt

13 App Certification Using CISQ CISQ measures CISQconformance assessment Technology vendors CISQ-conformant technology used in CISQ service process Vendor authorized service providers CISQ-conformant service process to provide CISQ/OMG only assess vendor conformance do not certify applications program initiates in 2017 Service providers use CISQ-conformant technology in a CISQ-conformant service process to provide application certifications Application Certification Security Xσ Reliability Xσ Performance Xσ Maintainability Xσ 12

14 Join CISQ! 15