MANAGING APPLICATION SECURITY INSIGHTS FROM FINANCIAL INSTITUTIONS

Transcription

1 MANAGING APPLICATION SECURITY INSIGHTS FROM FINANCIAL INSTITUTIONS 2017 APPLICATION SECURITY SURVEY BY SECURITY COMPASS

2 TABLE OF CONTENTS 2 EXECUTIVE SUMMARY 4 INTRODUCTION 8 SETTING THE CONTEXT: BUSINESS TRENDS 12 DRIVING THE APP SEC PROGRAM 14 Importance of Application Security 17 Regulatory Drivers 18 Organizational Structure 21 Three Lines of Defense 25 Application Security Teams 21 Common Challenges 28 METRICS 30 What s Measured Matters 34 APPLICATION SECURITY ACTIVITIES 36 Secure SDLC Frameworks 39 Scope of Analysis 40 Training 41 Key Activities 43 Risk Analysis 45 Security Testing 49 Requirements and Design 50 Other Activities 51 Tools Used 52 THIRD-PARTY SOFTWARE 56 SOFTWARE DEVELOPMENT 58 Development Processes 60 Application Lifecycle Management 62 CONCLUSIONS 66 APPENDIX: RESEARCH METHODOLOGY MANAGING APPLICATION SECURITY 3

3 THREE KEY BUSINESS TRENDS IN FINANCIAL SERVICES INCREASING SPEED OF BUSINESS FINANCIAL INSTITUTIONS WHICH HAVE TRADITIONALLY OPERATED WITH COMPLEX RISK MANAGEMENT PROCESSES ARE FACING STIFF COMPETITION FROM SMALLER, MORE NIMBLE FINANCIAL TECHNOLOGY STARTUPS AND PRODUCTS. INCREASING SOPHISTICATION OF RISK MANAGEMENT THERE IS INCREASING PRESSURE FROM BOARDS OF DIRECTORS FOR FINANCIAL INSTITUTIONS TO ADDRESS CYBER SECURITY RISKS. THE IMPORTANCE OF APPLICATION SECURITY 17% 33% 8% 75% OF FINANCIAL INSTITUTIONS STATED THAT APPLICATION SECURITY IS A HIGH OR CRITICAL PRIORITY WITHIN THEIR ORGANIZATION. 42% KEY DRIVERS OF APPLICATION SECURITY 77% OF FINANCIAL INSTITUTIONS STATED THAT GENERAL RISK MANAGEMENT WAS THE KEY DRIVER FOR THEIR ORGANIZATION S APPLICATION SECURITY. GENERAL RISK MANAGEMENT COMPLIANCE REQUIREMENTS BREACHES/INCIDENTS AT OWN OR OTHER ORGANIZATIONS 38% 62% 77% ENSURING THE SECURITY OF THIRD-PARTY SOFTWARE VENDORS ONLY 46% OF FINANCIAL INSTITUTIONS STATED THAT THEY REQUIRE VENDORS TO HAVE AN APPLICATION SECURITY POLICY. ONLY INCREASING PRESSURE ON COST CONTROL AS SUCCESSFUL FINANCIAL INSTITUTIONS EXPAND GLOBALLY, THERE IS INCREASED PRESSURE ON MAINTAINING A COMPETITIVE COST TO INCOME RATIO. INFORMATION SECURITY BUDGETS MUST BE ALLOCATED CAREFULLY TO AVOID OVERSPENDING IN INAPPROPRIATE PLACES. IT IS CRITICAL/ALWAYS A TOP PRIORITY IT IS CURRENTLY A HIGH PRIORITY (E.G. BECAUSE OF AN AUDIT DEFICIENCY) IT IS NOT PARTICULARLY IMPORTANT COMPARED TO OTHER AREAS APPLICATION SECURITY IS NOT (YET) IMPORTANT TO OUR ORGANIZATION AT ALL COMPETITIVE NEED CUSTOMER DEMAND 15% 15% 8% OF FINANCIAL INSTITUTIONS STATED THAT THEY PROVIDE DETAILED APPLICATION SECURITY REQUIREMENTS AS PART OF THEIR CONTRACTS WITH THIRD-PARTY SOFTWARE VENDORS. EXECUTIVE SUMMARY TRACKING THE EFFECTIVENESS OF AN APPLICATION SECURITY PROGRAM NUMBER OF VULNERABILITIES FOUND COMPLIANCE/ADHERENCE TO COMPANY POLICIES/STANDARDS LENGTH OF REMEDIATION NUMBER OF DEVELOPMENT TEAMS USING TOOLS/TOOL ADOPTION WE DO NOT TRACK THE EFFECTIVENESS OF OUR APPLICATION SECURITY PROGRAM MONEY SPENT ON PATCHING IN PRODUCTION DELAYS TO DEADLINES DUE TO SECURITY FIXES COMPLETION OF SECURITY REQUIREMENTS MONEY SPENT ON REMEDIATION 8% 15% 15% 15% 15% 38% 46% 62% 77% KEY SECURITY ACTIVITIES PERFORMED APPLICATION RISK CLASSIFICATION THREAT RISK ASSESSMENTS (NOT FOCUSED SPECIFICALLY ON APPLICATION SECURITY) MANUAL PENETRATION TESTING/ VULNERABILITY ASSESSMENTS DYNAMIC ANALYSIS (DAST) APPLICATION SECURITY REQUIREMENTS STATIC ANALYSIS (SAST) SECURE CODING STANDARDS/GUIDELINES WEB APPLICATION FIREWALLS (WAFS) MANUAL CODE REVIEWS THREAT MODELING/DESIGN REVIEW (APPLICATION SECURITY FOCUSED) OPEN SOURCE LIBRARY SCANNING SECURITY TESTING PERFORMED BY QA TESTERS FUZZ TESTING REAL-TIME APPLICATION SECURITY PROTECTION (RASP)/INTERACTIVE APPLICATION SECURITY TESTING (IAST) 1 = WE DON T PERFORM THIS ACTIVITY 5 = PERFORMED ON ALL APPLICATIONS % OF APPLICATION-LEVEL RISKS ARE NOT COVERED BY SAST & DAST TOOLS. BY FOCUSING ON THE NUMBER OF VULNERABILITIES, REMEDIATION BECOMES FOCUSED ON WHAT THE TOOLS CAN FIND RATHER THAN WHAT MATTERS MOST WHAT YOU DID NOT TEST FOR OR VALIDATE. A FRAMEWORK FOR APPLICATION SECURITY STRATEGY DRIVERS & GOALS METRICS & GOVERNANCE ORGANIZATIONAL STRUCTURE IN HOUSE TRAINING REQUIREMENTS & DESIGN TESTING EXECUTION 50% OF FINANCIAL INSTITUTIONS REPORTED THAT THEY PROCURE AT LEAST HALF OF THEIR SOFTWARE FROM THIRD PARTIES. THIRD PARTY PROCUREMENT SECURITY QUESTIONNARIES REQUIREMENTS TESTING WHEN DESIGNING THE STRATEGY TO DRIVE A SUCCESSFUL APPLICATION SECURITY PROGRAM, ORGANIZATIONS SHOULD ENSURE APPROPRIATE DRIVERS, GOALS, METRICS, GOVERNANCE, AND ORGANIZATIONAL STRUCTURES ARE CONSIDERED, THEN PUT INTO PLACE DURING THE EXECUTION PHASE, ORGANIZATIONS SHOULD ENSURE THAT SECURITY RISKS ARE MANAGED BY A VARIETY OF SECURITY ACTIVITIES TO ADDRESS BOTH IN-HOUSE AND THIRD-PARTY SOFTWARE. SECURITY AWARENESS TRAINING ADOPTION BY DEVELOPERS ACROSS THE ORGANIZATION 1 = NOT BROAD 5 = VERY BROAD AVERAGE RATING BY FINANCIAL INSTITUTIONS 3.5 OUT OF 5 4 SECURITY COMPASS MANAGING APPLICATION SECURITY 5

4 INTRODUCTION 6 SECURITY COMPASS MANAGING APPLICATION SECURITY 7

5 INTRODUCTION Inadequately secured software ranks amongst the most significant root cause issues in cybersecurity. Security products and services designed to detect, defend, and respond to increasingly sophisticated attacks continue to proliferate across the industry. Meanwhile, less than 4% 1 of information security budgets and a commensurate amount of mindshare is being spent on the securing of software. Ensuring the security of software is difficult. Unlike network security, where a single control like patch management can impact large parts of a company s infrastructure, the security of software differs with each unique application. Improving software security involves interacting directly with internal development teams, business stakeholders and third-party vendors, rather than simply the back-end IT team. When we asked how important application security is relative to other areas of information security, 75% of respondents reported that application security is either currently or always high priority. This shows that financial services firms are largely taking the subject seriously. The 25% of firms who are not emphasizing application security risk falling behind their industry peers. The relative importance of application security comes from its drivers. As we discuss in the business trends section, enterprise risk management is becoming increasingly technically sophisticated. Nearly all financial institutions have formal enterprise risk management, with teams focused on operational risk and others further specialized in technology risk. These specialized risk professionals tend to have deep knowledge of information technology risk and broadly identify software security as a key risk for their organization. Because they ultimately report up to oversight committees in the board of directors, they are partially freed from the inherent business pressures that often impact the Chief Information Security Officer s (CISO s) group. As we will see, financial institutions are especially prone to risks from insecure software. In the face of increasingly comprehensive regulatory audits, financial institutions have an obligation to address insecure software in greater breadth and depth than most other industries. Over the years, several organizations have conducted benchmark studies in broad cyber security practices for financial institutions, the effectiveness of application security testing practices, and descriptive/prescriptive maturity models for the secure software development life cycle (SDLC). However, we saw that some organizations in the field struggled to get their application security programs off the ground at all. From our observations, many large organizations felt overwhelmed when trying to grasp the enormity of securing their software portfolios. From these observations, we realized that the industry was lacking a comparative study that analyzed the following: What is driving application security programs for financial institutions? How are financial institutions structuring their application security programs? Which activities are working at scale for financial institutions? We embarked on a research study that included financial institutions, as well as other industries, to help answer these questions. While some of our findings were expected, others were surprising. We believe that the analysis presented below can help financial institutions understand how to effectively build their own application security programs and avoid costly lessons learned by their peers. 8 SECURITY COMPASS MANAGING APPLICATION SECURITY 9

6 SETTING THE CONTEXT: BUSINESS TRENDS 10 SECURITY COMPASS MANAGING APPLICATION SECURITY 11

7 SETTING THE CONTEXT: BUSINESS TRENDS To understand the nature of secure software development within financial institutions, it is imperative to first examine broad business trends that set the context. Three of these trends are detailed here INCREASING SPEED OF BUSINESS Financial services firms which have traditionally operated with complex risk management processes are facing stiff competition from smaller, more nimble financial technology (fintech) startups 2 and products. Larger firms are facing increasing pressure to move faster. The rise of global financial services firms means no country is isolated from this pressure. This imperative for speed has a major effect on the technology that support the business, and particularly the software development teams that build products servicing end customers. For internal IT systems, cloud-based software offerings allow business teams to benefit from tool support without the cost and time burden of IT department infrastructure. INCREASING SOPHISTICATION OF RISK MANAGEMENT Financial services firms are facing increasing pressure from boards of directors to address cyber security risks 3. Risk and audit committees provide broad oversight and thereby raise the priority of information security across the organization. Sophisticated strategies at financial services organizations often link executive variable pay to effective risk management. At the same time, regional regulations are exerting pressure on firms. Regulatory examiners, such as those from the Office of the Comptroller of Currency (OCC) in the United States, are becoming increasingly sophisticated in their knowledge of technical information security controls. In turn, risk management and internal audit functions need to become more aware of information security risks in order to ensure regulatory compliance. Conferences and industry publications such as the ISACA journal educate practitioners on complex information security topics. INCREASING PRESSURE ON COST CONTROL As successful financial services firms expand globally, they are constantly exposed to risks from geopolitical events and local economic uncertainty. This places increasing pressure on maintaining a competitive cost to income ratio 4. Despite its strategic importance, information security budgets must be allocated carefully to avoid overspending in inappropriate places. Financial service firms often use a combination of real threat data, known vulnerabilities, broad information security frameworks such as ISO and benchmarking data to understand how to effectively allocate budgets for information security. Sophisticated firms often attempt to quantify risk and use this as a major driver of spending. 12 SECURITY COMPASS MANAGING APPLICATION SECURITY 13

8 DRIVING THE APP SEC PROGRAM 14 SECURITY COMPASS MANAGING APPLICATION SECURITY 15

9 DRIVING THE APP SEC PROGRAM WEB APPLICATION ATTACKS CONTINUE TO BE THE LARGEST SOURCE OF INCIDENTS FOR FINANCIAL INSTITUTIONS AS PER THE 2016 VERIZON DATA BREACH INVESTIGATIONS REPORT. IMPORTANCE OF APPLICATION SECURITY Web application attacks continue to be the largest source of incidents for financial institutions as per the 2016 Verizon Data Breach Investigations Report 5. Moreover, many other forms of hacking take advantage of Common Vulnerabilities & Exposures (CVEs) in software products such as server software, operating systems, and desktop products. The existence of CVEs gives rise to vulnerability management, which is often on the front lines of incident prevention for organizations. Incident data isn t the only driver of software security for financial institutions. One respondent with a robust application security program phrased it best: When we asked how important application security is relative to other areas of information security, 75% of firms reported that application security is either currently or always high priority. This shows that financial services firms are largely taking the subject seriously. The 25% of firms who are not emphasizing application security are at risk of falling behind their industry peers. The relative importance of application security is driven by a number of factors. 77% 62% THE BUSINESS UNDERSTANDS THAT APPLICATIONS ARE OUR CROWN JEWELS. THUS, WE SIMPLY HAVE MADE IT A BUSINESS PRIORITY TO ENSURE THAT THOSE APPLICATIONS ARE SECURE. GENERAL RISK MANAGEMENT COMPLIANCE REQUIREMENTS FIGURE 2: What drives the priority of application security? 38% BREACHES/INCIDENTS AT OWN OR OTHER ORGANIZATIONS 15% 15% COMPETITIVE NEED CUSTOMER DEMAND SURVEY RESPONDENT 17% 33% 8% 42% IT IS CRITICAL/ALWAYS A TOP PRIORITY IT IS CURRENTLY A HIGH PRIORITY (E.G. BECAUSE OF AN AUDIT DEFICIENCY) IT IS NOT PARTICULARLY IMPORTANT COMPARED TO OTHER AREAS FIGURE 1: What is the relative importance of application security? APPLICATION SECURITY IS NOT (YET) IMPORTANT TO OUR ORGANIZATION AT ALL As discussed in the business trends section above, enterprise risk management continues to increase in technical sophistication. Nearly all financial institutions possess formal enterprise risk management, with teams focused on operational risk and others further specialized in technology risk. These specialized risk professionals tend to have deep knowledge of information technology risk and broadly identify software security as a key risk for their organization. Because they ultimately report to a board of directors oversight committee, they are partially freed from the inherent business pressures that often impact the Chief Information Security Officer s (CISO s) group. Typically, financial institutions maintain risk registers that both illustrate and often attempt to quantify risk to the executive team and risk and/or audit committees of the board of directors. This level of visibility means financial institutions receive attention and funding directed toward application security that few other industries can match. Thus, financial institutions are often the early adopters of application security technologies and serve as a bellwether for broader mass industry adoption. 16 SECURITY COMPASS MANAGING APPLICATION SECURITY 17

10 WHEN WE ASKED HOW IMPORTANT APPLICATION SECURITY IS RELATIVE TO OTHER AREAS OF INFORMATION SECURITY, 75% REGULATORY DRIVERS Unlike years past where compliance was the primary driver for information security spending, many firms reported it as secondary. Despite its lower emphasis, compliance still affects application security programs. Region-specific compliance requirements such as the FFIEC guidelines 6 in the US, the European Banking Authority (EBA) 7, the OSFI standards in Canada, and guidelines from the Monetary Authority of Singapore (MAS) 8 have detailed requirements and/or auditors who investigate application security programs in detail. Other regulations, such as the Gramm Leach Bliley Act (GLBA) and US state privacy laws, may not specifically call out application security controls but tend to have an impact on application security programs. US STATE PRIVACY REQUIREMENTS (E.G. CALIFORNIA PRIVACY ACT) OSFI REQUIREMENTS FFIEC GUIDANCE GLBA 62% 62% 62% 69% OF FIRMS REPORTED THAT APPLICATION SECURITY IS EITHER CURRENTLY OR ALWAYS HIGH PRIORITY. THIS SHOWS THAT FINANCIAL SERVICES FIRMS ARE LARGELY TAKING THE SUBJECT SERIOUSLY. THE 25% OF FIRMS WHO ARE NOT EMPHASIZING APPLICATION SECURITY ARE AT RISK OF FALLING BEHIND THEIR INDUSTRY PEERS. PCI DSS SARBANES OXLEY EUROPEAN PRIVACY DIRECTIVE FISMA/NIST HIPAA EBA SECURITY OF INTERNET PAYMENTS 15% 15% 23% 23% 46% 62% PCI PA DSS (SPECIFICALLY FOR PAYMENT APPLICATIONS) 8% FIGURE 3: What compliance regulations drive application security? Auditors haven t always had a deep understanding of application security, but this is changing. One of the survey respondents shared a story of how an auditor at the Office of Comptroller of Currency (OCC) in the United States looked deeply into all the activities of their SDLC. The auditor went as far as reviewing detailed tool output to discover any evidence of built-in security. The overall lesson is clear: financial services firms that do not adequately address application security expose themselves to increased regulatory risk. 18 SECURITY COMPASS MANAGING APPLICATION SECURITY 19

11 DRIVING THE APP SEC PROGRAM ORGANIZATIONAL STRUCTURE SENIOR MANAGEMENT BOARD OF DIRECTORS INTERNAL AUDIT Moreover, nearly all CISOs report to a Chief Information Officer (CIO), Chief Technology Officer (CTO), or other business function. 8% of respondents state that their CISO reports to both the CIO and Chief Risk Officer. Even though CISOs are being tapped to present to the board of directors, they rarely report independently of the rest of the business. CORPORATE LINE OF BUSINESS A LINE OF BUSINESS B ADDITIONAL LINES OF BUSINESS 8% CIO/CTO/OTHER BUSINESS ROLE CTO/CIO CRO ISO APPLICATION DEVELOPMENT ISO APPLICATION DEVELOPMENT ISO APPLICATION DEVELOPMENT DUAL: CIO & CHIEF RISK OFFICER CISO OTHER RISK GROUPS OPERATIONAL RISK SECURITY CHAMPIONS SECURITY CHAMPIONS SECURITY CHAMPIONS 92% APPLICATION SECURITY OTHER SECURITY TEAMS TECHNOLOGY RISK FIGURE 6: Who does the CISO report to? FIGURE 4: Simplified example of a financial institution s organizational structure with key application security players illustrated. It should be no surprise that all financial institutions surveyed have a Chief Information Security Officer (CISO) or equivalent. YES 100% NO 0% FIGURE 5: Does the company have a CISO or equivalent? These two data points have an impact on the structure of overall application security programs: 1. The CISO provides a vital role in securing executive buy-ins for application security programs. Many respondents cited executive support as crucial to driving the success of their program. 2. The CISO usually reports to a business function that is responsible for application development and/or support. It is possible, and sometimes the case, that the business accepts risks in the name of delivering functionality. Indeed, unless the technology heads are also held accountable for risk management, there is little structural incentive for them to prioritize information security ahead of delivering functionality. While this second point has broad implications across information security, the impact is particularly high in application security. Network and infrastructure security occur independently at financial institutions and for the most part does not get in the way of business processes. However, application security does have an impact on core business processes. Many large financial institutions have mitigated this concern by tying risk management to executive variable compensation. 20 SECURITY COMPASS MANAGING APPLICATION SECURITY 21

12 DRIVING THE APP SEC PROGRAM Likely due to higher visibility in the public eye and the need to fiercely protect a reputation, the financial industry enjoys broader company-wide support for application security, including independent software vendors (ISVs) who are pressured by their customers to prioritize security. We asked respondents to rate their organization s internal support for application security on a scale of 1.0 to 5.0, with 1.0 indicating no support and 5.0 indicating support across the board. On average, financial institutions rated their support for application security at 3.7 compared to 3.5 for software companies, and 3.4 for all other industries. THREE LINES OF DEFENSE Most financial services participants have adopted the Three Lines of Defense Model 9 for risk management. BOARD SENIOR MANAGEMENT FINANCIAL INSTITUTIONS LINE 1: MANAGEMENT CONTROLS LINE 2: RISK MANAGEMENT & INFORMATION SECURITY LINE 3: INTERNAL AUDIT FIGURE 8: Three Lines of Defense Model. INDEPENDENT SOFTWARE VENDORS Applied to application security, the model generally works in the following way: Line 1: Management Controls Business owners of applications and local information security teams in larger ALL OTHERS organizations are responsible for integrating security controls into their operational processes, including developing and acquiring software. FIGURE 7: How broad is your organizational support for application security? (1 = no support, 5 = support across the board) Financial institution application security teams lacking a requisite buy-in are faced with a severe uphill battle. These teams would do well to temper their expectations about the speed of their application security roadmaps execution, as well as the breadth of its impact. They should focus attention towards helping emphasize the priority of their efforts amongst executives, for example, by highlighting benchmark data like this report. Even for those that have secured an executive buy-in, grassroots resistance continues to be a challenge. Most respondents say that it is easier to get technology executives to agree to the idea of building security in, rather than getting individual software development teams to buy in. We will cover this topic in more depth later in the report. Line 2: Risk Management & Information Security Risk Management and/or Information Security takes responsibility for establishing frameworks, measuring risk and general compliance to these processes. They are not typically the people adding security into software development or acquisition processes on the tactical level, but they assist and ensure that application security risk is adequately defined and addressed. Line 3: Internal Audit Internal Audit is tasked with ensuring compliance to regulations, standards and policies, and is not involved in the implementation of controls. In the context of application security, this means that internal auditors are ensuring that operational management is following through on adoption of application security controls. 22 SECURITY COMPASS MANAGING APPLICATION SECURITY 23

13 DRIVING THE APP SEC PROGRAM APPOX. 50% OF VULNERABILITIES NOT REMEDIATED DAYS TO REMEDIATE CRITICAL VULNERABILITIES = SIGNIFICANT RISK EXPOSURE STAGES OF APPLICATION SECURITY MATURITY IN THE THREE LINES OF DEFENSE Through interviews, we observed a three-stage evolution in application security. In each stage, a specific subset of the Three Lines of Defense Model is engaged. Stage 1: Isolated Application Security LINE 1: MANAGEMENT CONTROLS Stage 2: Integrated Application Security LINE 1: MANAGEMENT CONTROLS LINE 2: RISK MANAGEMENT & INFORMATION SECURITY LINE 3: INTERNAL AUDIT LINE 2: RISK MANAGEMENT & INFORMATION SECURITY LINE 3: INTERNAL AUDIT During the early stages of an application security program, the information security team is primarily responsible for both operational and risk management activities related to application security. Counter to the Three Lines of Defense Model, only Information Security (Line 2) is involved in securing applications. The business groups and Internal Audit are not yet involved. This usually means Information Security runs tools or performs services to assess the security posture of applications as a gating process. At this stage, the business teams have assumed no responsibility for integrating application security into their daily activities. Grassroots efforts by Line 2 Risk Management to get software development teams to increase their adoption of application security activities are usually ineffective because their key stakeholders the business owners are not incentivized or measured in their ability to prioritize security. The only effective control most organizations have at this stage is to stop applications from going into production with high risk vulnerabilities. However, exemption processes exist and are used liberally in many cases. A study from Veracode 10 shows a 54% rate of remediation for vulnerabilities across all industries. Another study from WhiteHat Security shows a lower remediation rate of 42 to 48% in financial services industries 11. Moreover, the WhiteHat Security study also shows that the average critical risk vulnerability takes 316 days to remediate 12. Once organizations understand the significant risk exposure of having Information Security be solely responsible for application security, they enlist the support of Line 1: business units and the software development teams that support them. These groups accept responsibility for adopting application security activities. Generally, this stage involves development teams beginning to perform their own tests, for example, integrating static analysis products into the software build process. This is also a natural stage for organizations to shift left, which is a term commonly used in the industry to describe the adoption of security activities earlier in the software development process to reduce the number of vulnerabilities in their code. Many survey respondents explain that while they have an executive buy-in to implement management controls within business units, the actual adoption of those controls is lagging. In other words, Information Security teams are successful at mandating the usage of controls, but they are less effective at enforcing that mandate. This is a common challenge in risk management and information security, and part of the reason why leveraging all three lines of defense is more effective. The most mature respondents typically move to Stage 3 when they make this realization. Even though shift left activities benefit business units in the long run, they rarely take priority over other pressing business issues, such as building features. 24 SECURITY COMPASS MANAGING APPLICATION SECURITY 25

14 DRIVING THE APP SEC PROGRAM Stage 3: Optimized Application Security LINE 1: MANAGEMENT CONTROLS LINE 2: RISK MANAGEMENT & INFORMATION SECURITY APPLICATION SECURITY TEAMS All financial institution respondents in our study have centralized application security teams. This contrasts with other industries, such as Independent Software Vendors (ISVs), which may have separate product security teams for each business unit. Moreover, just under 70% of financial institution respondents have champions in individual business units and/or software development teams. LINE 3: INTERNAL AUDIT 31% CENTRAL GROUP OF APPLICATION SECURITY EXPERTS WITH CHAMPIONS IN INDIVIDUAL TEAMS OR BUSINESS UNITS Once management controls are in place for an organization and Information Security is effectively monitoring the adoption and usage of these controls, the most progressive financial institutions typically engage Line 3 (Internal Audit). Because of their independence and exclusive focus on non-compliance, internal auditors can elevate mandates from Risk Management to key stakeholders. Internal auditors can assess business units for non-compliance to internal standards and highlight application security risk. This doesn t mean that financial institutions must have Internal Audit engaged to be successful. Indeed, some organizations have succeeded at securing their highest risk applications at Stage 2. However, without the authority of Internal Audit, organizations often struggle to make progress on advancing their application security programs in the face of other business pressures. 69% FIGURE 9: Composition of application security teams. CENTRAL GROUP OF APPLICATION SECURITY EXPERTS, NO CHAMPIONS ON INDIVIDUAL TEAMS OR BUSINESS UNITS In many cases, large financial institutions reported a model of having a central application security team, with individual Information Security Officers (ISOs) who are responsible for information security in their respective business unit. In some cases, these ISOs are further supported by champions in individual development teams. In other cases, the ISOs are the sole representatives of security in the team. Respondents using this model stated that the ISOs wield significant influence on the effectiveness of their application security program. ISOs generally have a nuanced understanding of their business unit and its associated risk appetite. In the case of a large bank, an ISO may effectively be the CISO of a unit with tens of thousands of employees. They are often responsible for localizing corporate security policies, processes and tools to their own unit. Some respondents reported political challenges where strained relationships between an ISO and the corporate security group negatively impacted the overall structure. 26 SECURITY COMPASS MANAGING APPLICATION SECURITY 27

15 DRIVING THE APP SEC PROGRAM Those financial institutions that have security champions embedded in individual development teams have advocates nested deeply in the organization. This structure is successfully used by many large software vendors, such as Adobe 13. Most respondents indicated that it was a successful strategy to employ. DEVELOPMENT TEAMS ARE TOO BUSY TO PERFORM APPSEC ACTIVITIES STANDARD TOOLS DO NOT WORK WELL IN OUR ENVIRONMENT 46% 100% COMMON CHALLENGES Rolling out an application security program is rarely straightforward. All respondents reported that they faced organizational challenges when starting their application security programs. While some organizations continue to face these challenges, others faced them only early on. It is unclear why some stopped facing these challenges, although we suspect this is linked with broad company awareness and support for application security as a priority. The first and most common challenge organizations faced was that development teams were too busy to perform application security activities. Every financial institution surveyed reported that their development teams are under enormous pressure to deliver business features in compressed timelines. This means they are often resistant to new application security initiatives that may slow down their development speed. This, of course, is not unique to the financial services industry, as 86% of respondents across all industries reported the same challenge. The implication here is that if you are beginning to craft an application security program you should consider the impact on developer timelines as a primary constraint. INTRODUCING ACTIVITIES THAT SIGNIFICANTLY IMPACT DEVELOPERS USUALLY COMES WITH SIGNIFICANT RESISTANCE. RESPONDENTS TOLD US THAT THE RESISTANCE OFTEN STEMS FROM A LACK OF EDUCATION AS TO WHY APPLICATION SECURITY IS IMPORTANT, WHICH 46% CITED AS A CHALLENGE. THIS IS LIKELY THE ROOT CAUSE FOR THE COMMON CHALLENGE OF DEVELOPMENT TEAMS NOT BUYING IN TO PERFORMING APPLICATION SECURITY ACTIVITIES AT THE GRASSROOTS LEVEL, WHICH WAS ALSO CITED BY 46% OF FINANCIAL INSTITUTIONS. POLITICAL CHALLENGES (E.G. REACHING ACROSS ORGANIZATIONAL/UNIT BORDERS) LACK OF UNDERSTANDING/ EDUCATION ON WHY APPLICATION SECURITY IS IMPORTANT LACK OF GRASS-ROOTS BUY-IN TO PERFORMING APPLICATION SECURITY ACTIVITIES STANDARD APPLICATION SECURITY BEST PRACTICE DOES NOT ALIGN WITH OUR PROCESSES AND/OR CULTURE FIGURE 10: Which of the following challenges did/do you face when rolling out your application security program? 31% Introducing activities that significantly impact developers usually comes with significant resistance. Respondents told us that the resistance often stems from a lack of education as to why application security is important, which 46% cited as a challenge. This is likely the root cause for the common challenge of development teams not buying in to performing application security activities at the grassroots level, which was also cited by 46% of financial institutions. While the risks of insecure applications may be immediate, financial institutions may slow down their programs by forcing development teams to take on too much at once. Successful application security programs make use of best practices for organizational change management and emulate previous successful organizational changes. Dealing with politics, which 46% of respondents listed as a challenge, is a classic obstacle in any organizational change. Respondents frequently favored a phased deployment of all activities that impact developer throughout. 46% 46% 46% 28 SECURITY COMPASS MANAGING APPLICATION SECURITY 29

16 METRICS 30 SECURITY COMPASS MANAGING APPLICATION APPLICATION SECURITY SECURITY AT SCALE 31

17 METRICS WHAT S MEASURED MATTERS Metrics and Key Performance Indicators (KPI) are critical components for application security programs. Large companies rely on metrics to drive behavior and to demonstrate the effectiveness of programs and projects. NUMBER OF VULNERABILITIES FOUND COMPLIANCE/ADHERENCE TO COMPANY POLICIES/STANDARDS LENGTH OF REMEDIATION NUMBER OF DEVELOPMENT TEAMS USING TOOLS/TOOL ADOPTION WE DO NOT TRACK THE EFFECTIVENESS OF OUR APPLICATION SECURITY PROGRAM MONEY SPENT ON PATCHING IN PRODUCTION DELAYS TO DEADLINES DUE TO SECURITY FIXES COMPLETION OF SECURITY REQUIREMENTS MONEY SPENT ON REMEDIATION 8% FIGURE 11: Key application security metrics. 15% 15% 15% 15% Many organizations reported that they are still trying to get a handle on application security metrics. To track their programs, 77% of organizations use the number of vulnerabilities found. Typically, these vulnerabilities come from different sources, such as Static Analysis Security Testing (SAST) and Dynamic Analysis Security Testing (DAST) tools. Because vulnerabilities are not specific to application security, companies often use vulnerability management products to aggregate and track data from disparate sources. 38% 46% 62% 77% SAST and DAST tools are limited in their capability to look for risks. They have certain strengths and weaknesses; our own analysis suggests that 46% of application-level risks are typically not covered by SAST & DAST tools. Thus, by focusing on the number of vulnerabilities, financial institutions are naturally biasing their programs toward a subset of real risks. Remediation efforts become focused on what the tools can find rather than what matters most. Moreover, because one can only find vulnerabilities through testing, focusing on the number of vulnerabilities found may have the unintended consequence of emphasizing security testing over shift left activities occurring earlier in the software development process. The overemphasis of a vulnerability count also fails to provide a holistic picture of risk in other ways. For example: How long does it take to fix vulnerabilities (that is, the length of remediation)? Only 46% of respondents across all industries track this. How much money is spent on fixing vulnerabilities? Only 8% of respondents across all industries track this. Compliance and/or adherence to company standards is tracked by 62% of respondents. This is an essential governance step to understand whether software development teams are following corporate security policies. Many organizations which don t track this metric explain they have a difficult time enforcing application security processes. A lack of governance also limits the positive impact of internal auditors as we described in the Three Lines of Defense section above. The next most common metric is the number of development teams using application security tools (38%). This includes tracking developer security awareness training. In a sense, this is a subset of adherence to company standards, since most financial institutions have mandated tool usage. The fact that organizations call this metric out specifically points to large organizations relying heavily on tools for data collection and tracking metrics. The manual collection of metrics simply does not scale. 32 SECURITY COMPASS MANAGING APPLICATION SECURITY 33

18 METRICS AS PER OUR INTERVIEWS, THE MOST SOPHISTICATED ORGANIZATIONS USE A BLEND OF SEVERAL METRICS TO PROVIDE A SINGLE SECURITY SCORE, WHICH THEY AGGREGATE AT THE APPLICATION, TEAM, AND BUSINESS UNIT LEVELS. GENERALLY, THIS INCLUDES AT LEAST A COMBINATION OF SECURITY CODING MEASURES, VULNERABILITY COUNTS, AND INFRASTRUCTURE SECURITY DATA. No other metric has significant adoption across our sample. The adage what s measured matters rings true here. Organizations that only track scanning tool & penetration test results along with adherence to policies are missing out on a more holistic picture. Organizations can t convince a software development team that it s worth investing in building security into software when they aren t being measured and rewarded for doing so. Some organizations have run studies to map vulnerability reduction to early-phase secure SDLC activities. They use these to build awareness and to start focusing on and measuring secure SDLC activities. One organization provided us with data where they linked the usage of an early-phase secure SDLC tool to significant vulnerability reduction. The bars indicate the average number of high-risk and medium-risk vulnerabilities found by DAST & SAST tools. They then used this to build awareness for tracking early-phase secure SDLC activities within their organization. This is an effective workaround, though difficult to track, for organizations that focus myopically on vulnerability counts. To capture a holistic view and incentivize the right behaviors, financial institutions should consider capturing and aggregating data with the appropriate tool support in the following areas, some of which were cited by survey respondents: Completion of work required to build security applications (e.g., building in application security controls). This includes controls that you may not be able to easily test with standard scanning technologies (for example, the absence of hard-coded security credentials in code) Validation of whether these controls have been addressed, which may simply be a vulnerability count Accepted risks in production Remediation length Infrastructure security metrics, including network vulnerability counts Adherence to information security policies Adoption of security tools & training AVERAGE NUMBER OF VULNERABILITIES FOUND 32.8% Over time, financial institutions will need to prove the efficiency of their spending against other risk reduction measures. To do this effectively, financial institutions will need to capture key financial metrics: 13.2% 0% 0.4% HIGH RISK MEDIUM RISK Expected losses for open risks Cost of using preventative security measures Cost of remediation & production patching BEFORE TOOL ADOPTION AFTER TOOL ADOPTION FIGURE 12: Vulnerability reduction attributed to shift left activities. As per our interviews, the most sophisticated organizations use a blend of several metrics to provide a single security score, which they aggregate at the application, team, and business unit levels. Generally, this includes at least a combination of security coding measures, vulnerability counts, and infrastructure security data. Respondents tell us that these metrics must receive high-level visibility to be successful. For this reason, a simple view as to whether the application, team, and unit have met an acceptable threshold works best. The data should not reside solely in isolated application security tools or a spreadsheet on an individual desktop. Consider integrating this data into the organization s business intelligence or reporting platform such as Tableau. Alternatively, if it receives high level visibility, consider leveraging a Governance, Risk and Compliance (GRC) tool like EMC Archer for reporting metrics. 34 SECURITY COMPASS MANAGING APPLICATION SECURITY 35

19 APPLICATION SECURITY ACTIVITIES 36 SECURITY COMPASS MANAGING APPLICATION APPLICATION SECURITY SECURITY AT SCALE 37

20 APPLICATION SECURITY ACTIVITIES The core of our research on how organizations secure their applications at scale focuses on specific security activities. SECURE SDLC FRAMEWORKS Before diving into analyzing individual activities, we first consider how financial institutions select the type of security activities to pursue. Several frameworks, standards, and maturity models exist as best-practices guidelines for application security. Our respondents cited three primary sources for structuring their application security program: Building Security In Maturity Model (BSIMM) 14, Open Web Application Security Project s Open Software Assurance Maturity Model (OWASP SAMM) 15, and Microsoft s Secure Development Lifecycle (SDL) 16. For simplicity, we refer to all three as frameworks. Regarding effectiveness, respondents ranked BSIMM the highest at 7.8 out of 10 and OWASP SAMM the lowest at 6.0 out of 10. Generally, financial institutions found that the frameworks are useful for building a roadmap, securing a budget, and critically, in the case of BSIMM, providing benchmark data to help secure budgets. BSIMM OWASP SAMM % SDL % 33% ADOPTION OF BSIMM ADOPTION OF OWASP SAMM ADOPTION OF SDL FIGURE 14: Average effectiveness of Secure SDLC frameworks. (1 = not effective, 10 = very effective) FIGURE 13: Adoption of Secure SDLC frameworks among respondents who use a framework. While many respondents have not yet adopted a framework, almost 90% of those who have report that they used BSIMM. Respondents reported that the breadth of adoption in the industry and the ability to provide benchmark data were primary reasons for selecting BSIMM. Respondents also rated the effectiveness of the frameworks that they had worked with. We defined effectiveness as follows: Ability to use the framework to generate a business case for application security Usefulness of the framework for building a roadmap for application security Ability to execute on the activities described or prescribed by the framework With a handful of notable exceptions, most financial institutions did not make significant progress against the maturity models/core activities described by the frameworks at scale. These range from 17 key activities described in SDL to 113 described in BSIMM. While our analysis of activities at scale did not include release, deployment, or incident response phase activities, there is still a wide gulf between a financial institution s capability to execute at scale and the highest levels of maturity described in the frameworks. This is revisited in more detail later in this report. This highlights a key motivator for our research: given the business constraints facing financial institutions and the sheer volume of their application portfolios, we need to understand which activities are effectively scaling to define a minimum bar for the industry s approach to application security and identify areas to improve. 38 SECURITY COMPASS MANAGING APPLICATION SECURITY 39

21 DRIVING APPLICATION THE APP SECURITY PROGRAM ACTIVITIES SCOPE OF ANALYSIS Applications are only one component of a typical information technology environment. The security posture of an application is heavily reliant on the security of its underlying infrastructure, databases, users, and administration. One could argue that infrastructure security is part of an application security program. To focus our scope for analysis we specifically excluded the following areas which impact software security but are often addressed outside of application security programs: Operational and network/infrastructure security Data security and database security User security awareness training, except for developer awareness training General security policy & standards Identity and access management Vulnerability & patch management, except for third-party libraries in applications WHEN GIVEN THE BUSINESS CONSTRAINTS FACING FINANCIAL INSTITUTIONS AND THE SHEER VOLUME OF THEIR APPLICATION PORTFOLIOS, WE NEED TO UNDERSTAND WHICH ACTIVITIES ARE EFFECTIVELY SCALING TO DEFINE A MINIMUM BAR FOR THE INDUSTRY S APPROACH TO APPLICATION SECURITY AND IDENTIFY AREAS TO IMPROVE. Critically, application security programs cannot be effective in isolation. Information security teams must consider all the areas above and several other aspects of information security to effectively safeguard applications. 40 SECURITY COMPASS MANAGING APPLICATION SECURITY 41

22 APPLICATION SECURITY ACTIVITIES TRAINING All respondents have deployed developer security training to some extent. Practitioners told us that educating developers is a prerequisite to reducing the grassroots challenges we described above. 0% 31% 23% AVERAGE RATING: 3.5 FIGURE 15: How broad is the adoption of developer security awareness training at your organization? (1 = no training, 5 = all developers are trained) 8% 38% Security training for third-party developers is an area of concern for financial institutions. Most organizations will not pay for training for staff at outsourced development firms. While some outsourced companies have responded in turn by creating internal developer security training programs, financial institutions report that they are not necessarily confident in the quality of the training. Inconsistency in training quality is a primary driver of developer security certification and accreditation programs. Major training providers offer certificates in secure developer awareness to meet industry needs 17,18. Financial institutions should consider requiring industry standard certification from third-party software developers to ensure a consistent level of quality. We discuss third-party application security in more depth below. Financial institutions should also consider role-based training to ensure that each student receives the correct depth and breadth of material commensurate with their needs. The average rating from financial institutions on a scale of 1 to 5 regarding the broadness of security training adoption is 3.5, the same average rating reported by Independent Software Vendors (ISVs). KEY ACTIVITIES We asked participants how broadly they have deployed key application security activities. For each activity, respondents provided a score of 1 to 5 according to the following scale: 8% 1. We do not perform this activity today 2. We perform this activity on a very small number of applications today 31% 61% E-LEARNING/COMPUTER BASED TRAINING (CBT) COMBINATION OF E-LEARNING AND IN-PERSON IN-PERSON (e.g., only the most critical Internet-facing applications) 3. We perform this activity on a large subset of applications (e.g., all high-risk applications) 4. We perform this activity on most applications 5. We perform this activity on all or nearly all applications FIGURE 16: Which training format do you use? While 31% of respondents use both e-learning and in-person training, 61% exclusively use e-learning. Practitioners cited cost and logistical challenges as the reason for preferring to deploy elearning. Those who used a blended model often used in-person training to educate their security champions. 42 SECURITY COMPASS MANAGING APPLICATION SECURITY 43

23 APPLICATION SECURITY ACTIVITIES APPLICATION RISK CLASSIFICATION THREAT RISK ASSESSMENTS (NOT FOCUSED SPECIFICALLY ON APPLICATION SECURITY) MANUAL PENETRATION TESTING/VULNERABILITY ASSESSMENTS DYNAMIC ANALYSIS (DAST) APPLICATION SECURITY REQUIREMENTS The analysis revealed that the following activities were most broadly adopted: Classifying applications by risk (4.6) Threat risk assessments, not limited to application security (4.3) Manual penetration testing (3.0) Dynamic analysis (2.9) Application security requirements (2.8) Static analysis (2.8) STATIC ANALYSIS (SAST) SECURE CODING STANDARDS/GUIDELINES WEB APPLICATION FIREWALLS (WAFS) MANUAL CODE REVIEWS THREAT MODELING/DESIGN REVIEW (APPLICATION SECURITY FOCUSED) OPEN SOURCE LIBRARY SCANNING RISK ANALYSIS Financial institutions deploy two common risk management activities widely, application risk classification and threat risk assessments. Neither is inherently an application security activity, though both can be critical components of an application security program. Many financial institutions reported that they conduct both activities on all or nearly all applications. Classifying applications by risk is critical to the effective execution of an application security program. Absent of a formal risk classification activity, security professionals and/or software development teams are left to make largely subjective and often inconsistent choices about which activities and policies to apply to a specific application. When organizations scale to manage hundreds or thousands of applications it is practically impossible to execute an application security program without formal classification. Most financial institutions use a simple questionnaire to ascertain a risk classification. Below is an example excerpt of a risk assessment questionnaire: SECURITY TESTING PERFORMED BY QA TESTERS FUZZ TESTING REAL-TIME APPLICATION SECURITY PROTECTION (RASP)/INTERACTIVE APPLICATION SECURITY TESTING (IAST) QUESTION 1. IS THE APPLICATION INTERNET-FACING? 2. DOES THE APPLICATION HANDLE CREDIT CARD DATA IN ANY WAY? 3. DOES THE APPLICATION HANDLE ANY PERSONALLY-IDENTIFIABLE INFORMATION (PII) DATA? 4. DOES THE APPLICATION HOST ANY LEGALLY-PROTECTED DATA (CLASSIFIED, PATENTED, OR OTHERWISE)? 5. WILL THE LOSS OF AVAILABILITY TO THE APPLICATION CAUSE ANY HARM TO HUMAN LIFE? 6. IS THE APPLICATION CURRENTLY COMPLIANT WITH AN INDUSTRY-RECOGNIZED STANDARD (PCI, HIPAA, ETC.)? RESPONSE YES/NO FIGURE 17: Key activities performed by financial institutions. (1 = activity is not performed, 5 = performed on all applications) FIGURE 18: Example of a risk classification questionnaire. 44 SECURITY COMPASS MANAGING APPLICATION SECURITY 45

24 APPLICATION SECURITY ACTIVITIES Extending the questionnaire format, financial institutions also conduct formal Threat Risk Assessments (TRAs) on most applications. Most TRAs encompass a set of common steps seen in Figure 19. IDENTIFY ASSETS ASSESS THREATS ASSESS VULNERABILITIES FIGURE 19: Example steps of a Threat Risk Assessment (TRA). IDENTIFY RESIDUAL RISKS RECOMMEND Many financial institutions reported having a set of specific application security activities, such as security requirement analysis and/or threat modeling in addition to TRAs. Given that development teams are being pushed to deliver more functionality faster, information security can in fact encumber business goals by introducing multiple overlapping activities (i.e., threat risk assessments and threat modeling). Information security teams can help support their business by harmonizing these risk assessment steps to minimize the impact on development teams. Paper-based exercises to assess risk are common at all levels for financial institutions. For example, see the FFIEC Cybersecurity Assessment Tool 19 to assess the overall effectiveness of a financial institution s cybersecurity program. These activities often predate the introduction of a formal application security program at financial institutions and are intended to cover any kind of risk to an asset. Several popular frameworks for TRAs exist, such as the Canadian Communications Security Establishment Harmonized Threat and Risk Assessment Methodology (CCSE Harmonized TRA) 20. The intent of a TRA is to identify risks and provide recommendations at the onset of a project. This process is an ideal opportunity to introduce shift left practices to prevent application security vulnerabilities. However, respondents reported several challenges with the common practice of TRAs: In most organizations, TRAs are performed by a different team than application security. Application security teams have a limited ability to influence a TRA without support from information security leadership. TRAs are usually done at the end of an application development/maintenance project rather than the onset, as project teams often see these assessments as bureaucratic overhead and avoid doing them early on. The quality of the TRA tends to vary by the specific person who performs them. Many TRAs are performed by people with a background in network security or security policy but they have limited domain knowledge of application development. The questions and recommendations related to application security in a TRA tend to reflect this limited domain knowledge. SECURITY TESTING The next three most common activities are manual penetration testing/vulnerability assessments, dynamic analysis security testing (DAST), and static analysis security testing (SAST). These are also the first three activities specifically focused on application security. The average financial institution performs all three of these activities on nearly all high-risk applications. Manual source code reviews are surprisingly rare with an average score of 2.2. Manual penetration testing is more widely deployed than SAST or DAST. Respondents reported that while vulnerability assessments/penetration tests are relatively expensive and require manual effort, they are also effective and have relatively little setup time. Moreover, manual assessments do not suffer the same restrictions as automated tools with respect to specific programming languages or application types (e.g., web vs. mobile vs. client/server applications). 46 SECURITY COMPASS MANAGING APPLICATION SECURITY 47

25 APPLICATION SECURITY ACTIVITIES Regarding the carrying out of penetration testing, 77% of financial institution respondents utilize third parties, and 69% utilize in-house information security/ application security teams. There appears to be a trend for large financial institutions to insource a portion of security testing as they see it as a critical business process. The internal analysis of security scanner libraries shows that 46% of potential security flaws are detectable by neither SAST nor DAST packages without rule customization. EXTERNAL/THIRD PARTY 77% 46% 54% FOUND BY SCANNERS NOT FOUND BY SCANNERS INFORMATION SECURITY/ APPLICATION SECURITY TEAM 69% FIGURE 21: Security Compass s analysis of SAST and DAST coverage. FIGURE 20: Groups that perform penetration testing within financial institution respondents. While it is encouraging to see that financial institutions are serious about security testing, it is also alarming that many organizations appear to be satisfied with deploying security testing as their only activity for specifically addressing application security at scale. SAST and DAST tools are effective at finding certain classes of vulnerabilities, but are incapable of finding other ones absent of significant customization. For example, the NIST Static Analysis Tools Exposition (SATE) project discovered that static analysis tools found only 20% of the common vulnerabilities in the popular open source tools that they analyzed 21. Gartner analysts have suggested that they believe a SAST and DAST combination cover 40% of all risks 22. While the reasons for this low coverage vary, one of the key reasons cited by the NIST paper was the presence of a significant number of design level flaw[s] [ ] that are very hard to detect by computer analysis. If we combine this limited coverage with the fact that a maximum of 48% of vulnerabilities identified are remediated 23, this means that the average financial institution is only effectively remediating 26% of the potential vulnerabilities in an application; the rest are either undetected or unfixed. This should cause concern for most financial institutions that adopt security testing as their primary and/or only method to secure applications. 48% 52% NOT REMEDIATED REMEDIATED FIGURE 22: Remediation rates of vulnerabilities in financial services. 48 SECURITY COMPASS MANAGING APPLICATION SECURITY 49

26 APPLICATION SECURITY ACTIVITIES While penetration testing does not suffer from the same coverage gaps as SAST & DAST, most of these assessments are time boxed and limited in scope. Moreover, it is often impossible to know exactly which security vulnerabilities a penetration test looked for and which ones it did not look for. The combination of these factors exacerbates the need for some of the other activities in the Secure SDLC. Interactive Application Security Testing (IAST)/Real-time Application Security Protection (RASP) had the lowest adoption rate of all activities. With a score of 1.0, the average financial institution has not deployed IAST or RASP at all. Several organizations noted that they are currently piloting IAST solutions and are optimistic about its capabilities. Given the enormity of challenges with SAST tools false positives, we believe that IAST will rise in popularity in the coming years. Fuzz testing, apart from web/web services testing, yielded an average score of 1.4. This is the natural result of financial institutions having a greater emphasis on webbased technologies, as opposed to technology vendors who produce C/C++ applications. However, several financial institutions do use non-web-based third-party client/server applications. These applications can expose financial institutions to significant risk, particularly if they are built on unmanaged programming languages, like C/C++. Financial institutions should consider using contractual obligations to ensure that software providers include fuzz testing on their non-web-based applications. Security testing by QA testers yielded a score of 1.7. Unlike RASP/IAST, which respondents expressed optimism for, many organizations have unsuccessfully deployed application security in QA. Those who were successful in this activity typically limited the technical complexity of QA tests or had very technical QA testers with software development backgrounds. The latter is rare in financial institutions, who often hire QA testers to perform basic step-by-step functionality tests. In both cases, QA testing was always supplemented with additional automated or manual testing. The idea of having QA handle security testing appears to be a pipe dream for the average financial institution. While organizations would do well to realize the efficiencies of QA-supported security testing, they should calibrate expectations to include only the most basic security tests until they increase the technical capabilities of their QA teams. REQUIREMENTS AND DESIGN The next most common security activity is application security requirements (ranked at 2.8). Used properly, security requirements provide a structured way to embed security into a software development project at its very onset. Security requirements are a repeatable method to build security into applications, but they do not offer the same depth of analysis as threat modelling. Respondents cited speed and simplicity as key to the wide deployment of security requirements relative to other early-phase secure SDLC activities. Despite being a standard best practice, threat modeling had a lower score (1.8). Most organizations cited the long duration and dependency on security expertise as factors which prevent them from using threat modeling at scale. Organizations that did successfully deploy threat modeling usually reported being satisfied with the results. Both security requirements and threat modeling can alleviate the shortcomings of a penetration-testing-only approach. Tools exist to facilitate the wide-scale deployment of both activities. Linking testing activities to requirements or controls allows organizations to better understand their testing coverage while decreasing the chance of a vulnerability being produced in the first place. Organizations adopting either one or both controls typically followed the steps in Figure 23 to obtain software security assurance. IDENTIFY THREATS IDENTIFY REQUIREMENTS/ CONTROLS FIGURE 23: Four steps to software assurance. IMPLEMENT CONTROLS VALIDATE CONTROLS Respondents reported success with secure coding standards (ranked at 2.7). While nearly all respondents confirmed having these standards, most could not validate how widely the standards were being followed. Those that were successful often combined secure coding standards with security requirements or threat modeling automation. 50 SECURITY COMPASS MANAGING APPLICATION SECURITY 51

27 APPLICATION SECURITY ACTIVITIES OTHER ACTIVITIES Web application firewalls (WAFs), ranked at 2.6, are used less frequently than other technologies with the same maturity, such as DAST and SAST. WAFs provide a critical capability to protect web applications when secure coding or remediation is unavailable or impractical, such as legacy third-party software. Without WAFs and/or RASP technologies, financial institutions are often solely reliant on network approaches, such as Intrusion Prevention Systems (IPS), to respond to real-time attacks. When discussing WAFs, the single biggest complaint financial institutions cited was the lack of skillset to properly configure devices. Some respondents admitted to having WAFs but not actually leveraging them to block traffic. The last activity we measured was open source library scanning (ranked at 1.8). Like IAST/RASP, most respondents were relatively new to open source library scanning. Third-party libraries present significant risk to applications, and most of the other activities cited above do not cover security risks from such libraries. Respondents reported that certain SAST vendors have begun to offer open source scanning as part of their offering. This is a positive development and we expect the adoption of open source library scanning to increase in the future. While some respondents described additional activities, we could not find any other activities that were performed by more than a single respondent. This validates our assumption that only a small number of common application security best practices are widely deployed at scale. TOOLS USED HP FORTIFY IBM APPSCAN SECUREASSIST VERACODE BURP SUITE BLACK DUCK SOFTWARE HP WEBINSPECT IBM SECURITY APPSCAN SOURCE 23% 23% 23% 23% 31% 31% FIGURE 24: Which application security tools do you use? * This chart only displays data for the most common responses. Tool support is a critical aspect of large-scale success for application security activities. Many respondents reported that next to executive support, automation was the most important factor in a successful application security program. We asked about tool usage to understand which specific tools were being deployed. Only HP Fortify (SAST) is deployed by more than three quarters of financial institution respondents, with 77% adoption. Many organizations reported having a basket of tools with limited licenses, usually supplemented with larger licenses for a single SAST and a single DAST tool. 46% 77% 52 SECURITY COMPASS MANAGING APPLICATION SECURITY 53

28 THIRD-PARTY SOFTWARE 54 SECURITY COMPASS MANAGING APPLICATION SECURITY 55

29 THIRD-PARTY SOFTWARE Third-party software is pervasive within financial institutions. As we see below, 58% of financial institution respondents deploy some amount of third-party software, with 17% of financial institutions that primarily rely on it. Attackers do not differentiate between third-party and in-house software. As we discussed at the start of this paper, software of any type presents significant risk to financial institutions. 17% 17% 25% 41% BUILD IN-HOUSE FIGURE 25: Do you primarily build in-house, outsource, or buy & configure third-party software? (ROUGHLY) EQUAL MIX OF ALL THREE BUY & CONFIGURE COTS (ROUGHLY) EQUAL MIX OF BUILD IN-HOUSE AND COTS Primarily, financial institutions are relying on vendor security questionnaires to assess the vulnerability of their vendors, including the critical software that runs the core processes of their applications. These vendor security questionnaires are holistic in nature, not aimed specifically at software providers, and often do not go into great depth about application security. This lack of emphasis on application security also explains why 62% of respondents specifically cited ISO and/or SSAE16/SOC II audits as a means for assessing third-party risk. Like standard threat risk assessments, these audits include a slight coverage of application security at most. THIRD-PARTY SOFTWARE IS PERVASIVE WITHIN FINANCIAL INSTITUTIONS. 58% OF FINANCIAL INSTITUTION RESPONDENTS DEPLOY SOME AMOUNT OF THIRD-PARTY SOFTWARE, WITH 17% OF FINANCIAL INSTITUTIONS THAT PRIMARILY RELY ON IT. ATTACKERS DO NOT DIFFERENTIATE BETWEEN THIRD-PARTY AND IN-HOUSE SOFTWARE. AS WE DISCUSSED AT THE START OF THIS PAPER, SOFTWARE OF ANY TYPE PRESENTS SIGNIFICANT RISK TO FINANCIAL INSTITUTIONS. While many of our respondents described elaborate application security programs focused on in-house development, third-party software security programs rarely received parity. DETAILED VENDOR SECURITY QUESTIONNAIRE (NOT SPECIFIC TO APPLICATION SECURITY) REVIEW OF SECURITY CERTIFICATION NOT SPECIFIC TO APPLICATION SECURITY (E.G. SSAE16/SOC II TYPE 2/3, ISO 27001) REQUIRE VENDORS TO HAVE A SECURE SDLC/APPLICATION SECURITY POLICY PENETRATION TESTING AND/OR DYNAMIC ANALYSIS ON THIRD PARTY SOFTWARE CODE REVIEW, STATIC AND/OR BINARY ANALYSIS ON THIRD PARTY SOFTWARE THREAT MODELLING OR OTHER DESIGN-LEVEL ANALYSIS PROVIDE DETAILED APPLICATION SECURITY REQUIREMENTS (E.G. "PERFORM INPUT VALIDATION") AS PART OF CONTRACT RELY ON CYBERSECURITY INSURANCE 0% 8% 15% 38% 38% 46% 62% 92% Less than half the respondents require their vendors to have a secure SDLC and/or application security policy. Large financial institutions should consider using their leverage to improve the state of application security amongst their vendors. Most organizations had not yet heard of or adopted the draft ISO , which offers a robust standard for software development firms to demonstrate building security in. We strongly suggest financial institutions consider pushing their vendors to adopt the ISO as Microsoft 25 and SAP 26 have done. Other proprietary models, such as the vbsimm 27, can also serve this purpose. Financial institutions reported having difficulty performing penetration testing or code review, whether it s manual or automated, on third-party applications due to entrenched relationships and/or their vendors lack of willingness to cooperate. Only 38% of the respondents were able to perform penetration tests/dynamic analysis on third-party applications, and only 38% were able to perform static/binary testing. Fewer still were willing to perform threat modeling or design reviews (15%). 8% of respondents provide detailed application security requirements as part of a contract. This is one area where financial institutions can drastically decrease their application security risk with relatively little cost or organizational change. FIGURE 26: Third-party software security activities. 56 SECURITY COMPASS MANAGING APPLICATION SECURITY 57

30 SOFTWARE DEVELOPMENT 58 SECURITY COMPASS MANAGING APPLICATION SECURITY 59

31 SOFTWARE DEVELOPMENT A review of application security is not complete without an examination of software development. In this section, we examine software development and its impact on security. Looking in detail at how financial institutions integrate security into a typical scrum development practice, typical application security activities fall into three periods of frequency: DEVELOPMENT PROCESSES AGILE SCRUM 85% Periodically activities teams perform on a periodic basis, such as adding security stories to a backlog Every sprint activities that teams perform every sprint, such as penetration testing Every story or build activities that teams perform on each story or build, such as running automated security tests WATERFALL/CMMI 77% AGILE KANBAN 8% PERIODICALLY EVERY SPRINT EVERY STORY/BUILD FIGURE 27: Which of the following industry standard development processes do you use at your company? Most financial institutions surveyed reported using both agile scrum and waterfall ACQUIRE/PLAN TO BUILD APPLICATION Application risk classification DEVELOP Complete security stories and adhere to security constraints CODE REVIEW Review code for security vulnerabilities software development processes. Many organizations are transitioning from one SDLC to the other, using terms like scrum-fall to describe their hybrid application DEFINE PRODUCT BACKLOG SPRINT PLANNING MEETING RELEASE development process. Primarily due to the organization s size and emphasis on risk management, financial institutions find it challenging to eschew phase gates completely. This dampens their ability to be truly agile. Financial institutions broadly reported that they found it challenging to retrofit application security best practices to agile development practices. Threat risk assessment Security requirements and/or threat modeling Add security stories to backlog Select security stories for current sprint Add security constraints to functional stories MANUAL TEST Manual security testing by developers/qa AUTOMATED TEST DAST and SAST Security unit tests Penetration testing In particular, requiring security sign-off and performing threat risk assessments for each project contrasts sharply with the concept of 1 to 2-week development iterations. Information security and, more broadly, risk management professionals will need to innovate and offer lower friction solutions for their businesses to remain competitive. FIGURE 28: Depiction of typical security activities in an agile development process. 60 SECURITY COMPASS MANAGING APPLICATION SECURITY 61

32 SOFTWARE DEVELOPMENT Many organizations also reported the adoption of DevOps practices, with an emphasis on allowing automated integration testing and deployment, thereby reducing the cost of making changes to code. DevOps initiatives in and of themselves do not necessarily impact up-front SDLC practices, but a logical extension of adopting DevOps is adopting a continuous delivery process 28. Very few traditional application security tools or processes can effectively work in a continuous delivery process, lacking upfront planning and manual sign-off. Another useful data point is understanding how frequently developers adhere to standard development processes. 46% of financial institution respondents reported that developers consistently followed a standard process. 27% of respondents said that developers followed a standard process most of the time, and an equal proportion of respondents said that developers followed a standard process some of the time. Security teams are challenged when attempting to embed security into the SDLC if teams do not follow the process. APPLICATION LIFECYCLE MANAGEMENT As financial institutions move to adopt agile practices, the application lifecycle management (ALM) tool is quickly becoming the hub of application development activity. Developers have optimized workflows that take advantage of ALM features such as marking a ticket ready for code review. Application security teams that do not adapt their activities to ALM tools risk increased resistance from development teams. Security teams need to customize ALM tools to cover security activities and processes. Within ALMs, JIRA is the clear front-runner despite broad competition from several entrenched players. In many cases, development teams are acquiring JIRA through Atlassian s simple SaaS model, and it is becoming an enterprise standard after an increasing number of disparate teams adopt it independently at the grassroots level. 27% 46% 27% ALWAYS MOST OF THE TIME SOME OF THE TIME FIGURE 30: How closely do developers follow the standard SDLC? JIRA 69% HP ALM MICROSOFT TEAM FOUNDATION SERVER IBM RATIONAL 15% 23% 46% PRIMARILY DUE TO THE ORGANIZATION S SIZE AND EMPHASIS ON RISK MANAGEMENT, FINANCIAL INSTITUTIONS FIND IT CHALLENGING TO ESCHEW PHASE GATES COMPLETELY. THIS DAMPENS THEIR ABILITY TO BE TRULY AGILE. FINANCIAL INSTITUTIONS BROADLY REPORTED THAT THEY FOUND IT CHALLENGING TO RETROFIT APPLICATION SECURITY BEST PRACTICES TO AGILE DEVELOPMENT PRACTICES. RALLY (CA AGILE CENTRAL) 15% FIGURE 29: Which Application Lifecycle Management (ALM) tools do you use? 62 SECURITY COMPASS MANAGING APPLICATION SECURITY 63

33 CONCLUSIONS 64 SECURITY COMPASS MANAGING APPLICATION SECURITY 65

34 CONCLUSIONS We believe that the findings from this report are important and can help drive application security programs forward in the financial services industry. The key business drivers discussed at the beginning have ripple effects throughout the report. Agility and a move towards third-party & cloud-based software, along with increased global regulatory scrutiny, challenge application security teams within financial institutions. At the same time, increased awareness and visibility of these giant organizations within the public eye are helping teams push their agendas forward. Application security teams within financial institutions need to design their security programs with the appropriate goals, governance and metrics. They should leverage third-party data like this report, the BSIMM, and the annual Verizon Data Breach Investigations Report to enhance their business cases. Firms should select security activities that meet their risk reduction and scalability goals. Simply selecting a set of best practices from a secure SDLC framework may not result in an ability to execute. The top security activities described by our study provide clues as to what peer organizations in the industry have scaled successfully. 66 SECURITY COMPASS MANAGING APPLICATION SECURITY 67