WHITEPAPER THE CONTRAST ASSESS COST ADVANTAGE APPLICATION SECURITY TESTING COSTS COMPARED WELCOME TO THE ERA OF SELF-PROTECTING SOFTWARE CONTRASTSECURITY.COM
EXECUTIVE SUMMARY Applications account for nearly 40% of enterprise IT expenses. 1 This is not too surprising given that most of today s businesses run on applications. What is surprising is that only about 10% of business applications get any significant security testing, 2 even though they are the number one source of successful data breaches. 3 An analysis of the cost to secure even a single application reveals the culprit: conventional application security products and methods have huge human capital cost components. Multiplying those costs across an entire application portfolio would quickly exceed any organization s application security budget. The result is that only a handful of applications ever get assessed and/or protected. Contrast Assess changes that equation by dramatically reducing laborrelated costs, which are the largest cost component of application security testing and remediation. Contrast Assess delivers a 65% annual cost reduction compared to manual approaches and a 60% reduction versus static application security testing tools. Those figures are based on the conservative set of assumptions presented below, including just a single application analyzed once per year. Analyzing more applications or analyzing applications more frequently produces even greater costs savings. These savings span multiple organizations, including security, development, and operations. This document presents a model for comparing costs across different approaches to application security testing, and provides a sample comparison for a single application. Businesses can use this model and data from their own experience to tailor the comparison to their environments. 1 Source: Apptio, IT Economics Insights. Unpacking the Application IT Tower 2 Source: Contrast prospects 3 Source: 2016 Verizon Data Breach Investigations Report 2
COST ANALYSIS This cost analysis compares three different application security testing approaches: performing a manual vulnerability assessment, using a Static Application Security Testing (SAST) product, and using Contrast Assess the leading Interactive Application Security Testing (IAST) solution. The same cost analysis can also be used with Dynamic Application Security Testing (DAST) tools (readers can simply substitute DAST product costs and associated process costs where SAST is referenced). While each of these three approaches is different, the overall process can be broken down into seven common steps. The overall cost, and the cost for each step, can then be compared across all approaches. The Seven-Step Application Security Testing Process 1 Product Licensing 2 Vulnerability Analysis 3 Triage 4 Reporting 5 Fix-Test-Redeploy 6 Security Retest 7 Program Management Tailoring the Comparison The analysis in this document uses the assumptions presented in the Primary Assumptions section below. To get custom results, it is necessary to replace these assumptions with your own details. The most significant assumption presented here is that only one (1) vulnerability assessment takes place per year. Organizations that perform testing more than once per year will need to multiply the costs for Vulnerability Analysis, Triage, Reporting, Fix-Test-Redeploy and Security Retest based on the number of assessments done. Performing multiple assessments per year makes the Contrast Assess cost advantage even greater. Please contact a Contrast Security Sales Representative for assistance in building a custom analysis. PRIMARY ASSUMPTIONS The analysis presented below uses the assumptions listed here. Organizations can easily modify and tailor these assumptions and any other assumptions used in this analysis to reflect their experience, as well as their unique situations and costs. ASSUMPTION USED IN THIS DOCUMENT YOUR COMPANY Number of vulnerability assessments per year 1 Serious vulnerabilities per application 22.4 4 Skilled software developer, hourly rate US$ 80 Skilled in-house application security expert, hourly rate US$ 120 Program administrator, hourly rate US$ 200 Commercial application security tool Price per application, per year False positive rates US$ 5,000 10,000 SAST Very High Contrast Very Low 5 4 Source: Aspect Security 5 Source: OWASP Benchmark Project 3 WELCOME TO THE ERA OF SELF-PROTECTING SOFTWARE CONTRASTSECURITY.COM
OVERALL COST COMPARISON Figure 1 and Table 1 (below) summarize the costs for each of the seven steps across the different testing methods Manual, SAST and Contrast IAST for doing a single vulnerability assessment on a single application. Both Figure 1 and Table 1 clearly show that Fix-Test-Redeploy is the largest cost-contributor, and that it is the one step where Contrast Assess with IAST capabilities has the greatest overall impact. Contrast Assess also completely eliminates both the need for a separate Security Retest step and the costs associated with onboarding applications at the Vulnerability Analysis step. Contrast Security also reduces Triage and Reporting costs significantly. Because most organizations conduct more than one vulnerability assessment per application, per year, Figure 1 is a relatively conservative estimate of the cost savings that organizations experience using Contrast Assess. Still, using the assumptions described above and doing a single vulnerability assessment for one application Contrast Assess represents a 65% cost savings compared to Manual Testing, and a 60% cost savings compared to using a SAST Service. Figure 1: Annual Cost Comparison Contrast Assess IAST SAST Product Manual PROGRAM MANAGEMENT SECURITY RETEST FIX-TEST-REDEPLOY REPORTING TRIAGE VULNERABILITY ANALYSIS PRODUCT LICENSING 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 Table 1: Summarized Annual Cost Table PRODUCT LICENSING $ $10,000 $10,000 VULNERABILITY ANALYSIS $19,200 $1,920 $30 TRIAGE $ $6,000 $480 REPORTING $2,640 $2,640 $720 FIX-TEST-REDEPLOY $35,200 $35,200 $8,800 SECURITY RETEST $4,800 $960 $ PROGRAM MANAGEMENT $4,000 $4,000 $4,000 SUMMARY RESULTS $65,840 $60,720 $24,030 4
ACTSOA: Annual Cost to Secure One Application This document enables organizations to calculate the total annual cost required to secure a single application a metric worthy of its own acronym: ACTSOA, the Annual Cost to Secure One Application. It s a metric that organizations can use to measure application security effectiveness for an application: over time, between different applications, and between peer organizations. The total budget required to secure all applications in an enterprise portfolio can be calculated by multiplying an organization s average ACTSOA by the number of applications in the portfolio. Legacy approaches have such a high ACTSOA that they are impractical methods to use at an enterprise portfolio scale. In fact, for an organization using legacy approaches the cost may be many times the total allocated application security budget. Therefore, reducing the ACTSOA without compromising security is the key to a successful application security program. In the example scenarios described in this document, the ACTSOA using Contrast Security is less than half of the ACTSOA using legacy tools and methodologies, demonstrating the financial advantage of using Contrast Assess for even one application. COST COMPONENTS EXPLAINED IN DETAIL Each of the seven steps in testing for vulnerabilities is explained below, along with the key cost contribution for each step. The Product Licensing step has an obvious cost, but it is far from the only or largest cost associated with the application security testing process. Because of the need for trained experts, the other six steps have human resources costs that can equal, and even exceed, the product licensing cost. And, it s these other costs that are typically repeated multiple times per year, per application. 1. Product Licensing Manual vulnerability analysis makes sense in a few, limited circumstances, such as when there isn t much code to analyze or when looking for certain types of vulnerabilities (e.g., vulnerabilities in custom authentication and access control code). For the other over 90% of application security testing scenarios, commercial application security testing solutions are the way to go. These tools are designed to automate what otherwise becomes a taxing and repetitive process. However, the effectiveness of commercial tools can have a major cost impact, as discussed in the other steps below. There are a large number of commercial tool choices available, but they fall into three broad categories: SAST, DAST, and IAST products. While pricing and pricing models vary, businesses can expect to pay in the range of US$5,000 to US$10,000 on average, per application, per year. The analysis below assumes the use of a SAST or IAST product. PRODUCT LICENSING $ $10,000 $10,000 5 WELCOME TO THE ERA OF SELF-PROTECTING SOFTWARE CONTRASTSECURITY.COM
2. Vulnerability Analysis Vulnerability Analysis is the process of examining an application manually or with an automated solution to verify that the proper security measures are in place and working as intended. The output of a Vulnerability Analysis is often a list of possible application vulnerabilities, because when the tools are inaccurate the results need significant validation. For automated tools, Vulnerability Analysis costs include installing, configuring, tailoring, and running the tools. For SAST tools, a couple of days are typically needed for onboarding the application into the tool and generating a report. Contrast Assess users incur minimal onboarding costs due to the simplicity of its agent-based instrumentation. It takes just a few seconds to download the instrumentation agent, and only a few minutes to integrate the agent with the application server. For manual code reviews, the cost includes the time Application Security experts spend performing the reviews. The Manual model below estimates two Application Security experts, working two, forty-hour work weeks each, at the rate of $120 per hour (2 x 2 x 40 x $120) for a total of $19,200. For SAST, the estimate is two days of onboarding (2 x 8 x $120 for a total of $1,920), and for Contrast Assess this model estimates 15 minutes of an Application Security expert s time for downloading and integration, for a total of $30. VULNERABILITY ANALYSIS $19,200 $1,920 $30 6
3. Triage Triage is the process of evaluating the findings from the Vulnerability Analysis step and determining which vulnerabilities need to be fed into an organization s defect management system. The primary issue addressed during the Triage step is determining whether a finding represents a valid security issue (i.e., is it a True Positive?). See the OWASP Benchmark Project for supporting materials on tool accuracy. Both SAST and DAST products have serious accuracy problems. First, SAST products generate large numbers of False Positives, each of which requires significant time to triage. Second, SAST and DAST have significant problems with False Negatives, because both SAST and DAST fail to discover many real vulnerabilities in applications. In this analysis, we have ignored the costs of risk associated with False Negatives. For manual code reviews, the Triage step is an integral part of the Vulnerability Analysis process, so the Manual cost for the Triage step is zero. For SAST tools, this analysis assumes 200 possible vulnerabilities that need to be investigated, and 16 on the Contrast Assess side 6. The assumption is that an Application Security expert can triage these types of issues at a pace of 15 minutes each, or 4 per hour. This places the SAST cost at 50 hours (200 4) at $120 per hour for a total of $6,000; and while the 200 figure is conservative, it highlights an important intangible issue. That is, in general, SAST tools also produce such a high number of False Positives that they introduce an element of fatigue into the process. In turn, this leads to some True Positives being tossed out with the False Positives. Ignoring True Positives is equivalent to having False Negatives; that is, businesses are missing real vulnerabilities, which increase their risk of being breached. Contrast Assess strikes an impressive balance when it comes to vulnerability analysis, delivering highly accurate results and comprehensive coverage. The OWASP Benchmark Project documents these findings, confirming what many professionals have known intuitively from their use of SAST and DAST solutions. In this analysis, the Triage cost for Contrast Assess cost is 4 hours (16 vulnerabilities 4 vulnerabilities per hour) at $120 per hour for a total of $480. TRIAGE $ $6,000 $480 6 Based on 100% accuracy rate, per OWASP Benchmark Project 7 WELCOME TO THE ERA OF SELF-PROTECTING SOFTWARE CONTRASTSECURITY.COM
4. Reporting Reporting is the process of recording every True Positive vulnerability by giving it a risk rating (e.g., using a system such as the OWASP Risk Rating Methodology), and capturing the vulnerability details in a vulnerability or defect tracking system (e.g., Bugzilla, JIRA, MantisBT, etc.). This enables organizations to report on open vulnerabilities until they are fully resolved and closed out in the tracking system. Businesses also use this data to build application security dashboards so they can track the overall effectiveness of their program. Using the assumption of 22 serious vulnerabilities per application, and approximately an hour per vulnerability for thoroughly documenting and communicating the issue, we estimate 22 hours for reporting with both the Manual approach and with SAST tools (22 hours x $120 for a total of $2,640). With Contrast Assess, the majority of vulnerabilities are resolved very early in the development process, while code is being written and tested in the developer s environment. Contrast Assess addresses 75% of vulnerabilities this way, leaving only 25% of the 22 serious vulnerabilities, or a total of about 6, to be reported at a rate of $120 per hour for a total of $720. REPORTING $2,640 $2,640 $720 5. Fix-Test-Redeploy The Fix-Test-Redeploy step is where software developers recode the relevant portions of an application to fix its security vulnerabilities. The time to fix a vulnerability can vary, but based on estimates from Aspect Security and WhiteHat 7, our analysis uses 20 hours for a typical vulnerability, such as Cross-Site Scripting (XSS) or SQL injection (SQLi). Those 20 hours (this is a conservative estimate) include fix-test-redeploy, testing, standard QA processes, staging, and redeployment. Using the estimate of 22 critical vulnerabilities per application, and 20 hours to fix each of those, the cost to fix the vulnerabilities with the Manual and SAST approaches is 440 hours, times a rate of US$ 80 per hour, for a total of $35,200. Using the IAST capabilities of Contrast Assess, organizations find vulnerabilities much earlier in the process. As they test their code, developers can see exactly where to the line of code vulnerabilities exist, and get detailed information on how to remediate them. This enables developers to eliminate the vulnerabilities as part of their normal workflow, before they have moved on both literally and mentally to other work. As with Reporting, the estimate for Contrast Assess is 25% of the SAST tool, due to the smaller number of unresolved vulnerabilities. In this case, that Contrast Assess total is US$ 8,800. FIX-TEST-REDEPLOY $35,200 $35,200 $8,800 7 Source: http://blog.jeremiahgrossman.com/2009/05/mythbusting-secure-code-is-less.html 8
6. Security Retest Once vulnerabilities have been remediated through re-coding efforts, organizations need to confirm that the fix is working. This typically requires retesting the application from an application security perspective. Application security consultants typically charge 25% ($4,800) of the original Vulnerability Assessment cost ($19,200), so we use that same estimate for Manual retesting. For Static retesting, the assumption is 50% ($960) of the Vulnerability Assessment cost ($1,920). For Contrast Assess there is no cost, because the Vulnerability Assessment is ongoing and continuous. SECURITY RETEST $4,800 $960 $0 7. Program Management Application security Program Management varies widely among organizations, and typically correlates to the number of applications being assessed and the maturity of the program. At a minimum, Program Management includes managing work that comes from new development, making changes to existing applications, and/or assessing third-party applications. Each of these streams can have separate program or project managers. In this model, the estimate for Program Management is 20 hours per application, per year, for a program manager earning US$ 200 per hour, for a total of $4,000. PROGRAM MANAGEMENT $4,000 $4,000 $4,000 9 WELCOME TO THE ERA OF SELF-PROTECTING SOFTWARE CONTRASTSECURITY.COM
CONCLUSION Contrast Assess helps businesses dramatically reduce the total cost of application security by reducing costs across all application security testing steps, especially the most expensive: the fix-test-redeploy process. Using the assumptions and model outlined in this document, Contrast Assess represents a 60% annual cost savings over traditional SAST approaches, and 65% savings over manual analysis. By using their own data, businesses can leverage the model presented here to estimate their potential savings over traditional approaches. SUMMARY RESULTS $65,840 $60,720 $24,030 Using an approach and technology that are unique in the industry, Contrast Assess produces highly accurate results that enable organizations to find and fix vulnerabilities early in the software development lifecycle when fix-test-redeploy costs are lowest. Note that while this analysis quantifies the tangible costs, it does not attempt to include the benefits resulting from reduced risks. Contrast Assess identifies a broader range of vulnerabilities earlier in the software lifecycle with far fewer False Negatives than legacy SAST and DAST tools. The expected value of this risk reduction is significant, yet impossible to calculate in the abstract without knowing the details of a business. Contact Contrast Security at salesinfo@contrastsecurity.com for a free demonstration, an evaluation, or for a customized cost comparison using the model detailed in this document. 240 3rd Street Los Altos, CA 94022 888.371.1333 121916 Contrast Security is the world s leading provider of security technology that enables software applications to protect themselves against cyberattacks. Contrast s patented deep security instrumentation is the breakthrough technology that enables highly accurate analysis and always-on protection of an entire application portfolio, without disruptive scanning or expensive security experts. Only Contrast has intelligent agents that work actively inside applications to prevent data breaches, defeat hackers and secure the entire enterprise from development, to operations, to production.