DOT Cybersecurity Summit FISMA Cybersecurity Performance Metrics and Scoring Office of the Federal Chief Information Officer, OMB OMB Cyber and National Security Unit, OMBCyber@omb.eop.gov
2. Cybersecurity Metrics OMB receives agency FISMA metrics through DHS s CyberScope System. Metrics reporting schedule per M-17-05 1. Quarterly CIO FISMA metrics (CFO Act only) 2. Monthly PIV/CAC submissions (CFO Act Only) 3. Annual IG, CIO, and SAOP Metrics (All Agencies)
3. Cybersecurity Metrics in Action OMB uses FISMA metrics for nine processes and products that drive agency Performance: 1. President s Management Council (PMC) Assessment 2. Cybersecurity Cross-Agency Priority (CAP) Reports 3. Annual FISMA Report to Congress 4. CyberStat Reviews 5. PortfolioStat Reviews 6. FedStat Reviews 7. President s Budget Cybersecurity Crosscut 8. Cabinet Engagements 9. Policies and Guidance
4. Annual FISMA Report to Congress FY 2016 Annual FISMA Report to Congress One Pagers Agency one-pagers provide greater context for FY 2016 cybersecurity performance data Goal of improving readability and look and feel of report. FY 2016 IG Metrics provide independent assessment of FY 2016 CIO metrics. OMB anticipates a decrease in IG scores due scoring methodology changes. One-pagers will also serve as one-stop summary for future OMB, DHS and IG oversight discussions (e.g., CyberStat, etc.).
5. PMC Assessment Background In the wake of the OPM incidents, OMB recognized the need for a cybersecurity performance assessment that provides agency Deputy Secretaries and EOP with an understand of the agency s hygiene. OMB developed the PMC Cybersecurity Assessment in late 2014 using agencies FISMA metrics data and assessment criteria from the NIST Cybersecurity Framework. This assessment is a vehicle for driving agency performance and ensuring accountability from the Deputy Secretary on down through the organization. OMB has matured the assessment process and products to ensure clear and effective communication to agencies, and uses the output from this process to inform its oversight and budget processes.
6. Leveraging the NIST Cybersecurity Framework The 23 Civilian CFO-Act Agencies receive a quarterly assessment from OMB that provides overall rating based on performance across the five NIST Cybersecurity Framework function areas: Identify Develop the organizational understanding to manage cybersecurity risk to systems, assets, data, and capabilities. Detect Develop and implement the appropriate activities to identify the occurrence of a cybersecurity event. Protect Develop and implement the appropriate safeguards to ensure delivery of critical infrastructure services. Respond Develop and implement the appropriate activities to take action regarding a detected cybersecurity event. Recover Develop and implement the appropriate activities to maintain plans for resilience and to restore any capabilities or services that were impaired due to a cybersecurity event.
7. Rating Standards EOP provides ratings to agencies for each NIST framework function area on a 0-3 scale: 0 1 2 3 Agency has not met foundational targets Agency has met foundational targets Agency has met government-wide targets Agency has exceeded government-wide targets In first quarter of FY 2016, only five agencies had information security programs that met or exceeded government-wide performance goals. By the end of FY 2016, 13 agencies had met these targets and all others were making significant progress toward this end as a direct result of PMC Assessment process.
8. Sample of Individual Agency Scorecard 1 2 2 Overall Example Justice has agency met government-wide has met government-wide targets targets Overall rating is the average of the component scores, rounded to the nearest whole number. (1+2+2+3+3)/5 = 2 Identify 1 2 3 Hardware Asset Management (CAP) 100% Software Asset Management (CAP) 97% Unclassified systems with security ATO 87% Review of contracts with sensitive information Policy empowering incident commanders Protect 1 2 3 Vulnerability Management (CAP) 100% Secure Configuration Management (CAP) 95% Unprivileged PIV logical access (CAP) 95% Privileged PIV logical access (CAP) 100% Remote access security Insider Threat Program Media destruction policy 2 3 3 Detect 1 2 3 Anti-Phishing and Malware Defense (CAP) Anti-Phishing 5 of 7 Malware Defense 3 of 5 Other Defense 2 of 4 EINSTEIN Program Attempts to access data detected and investigated Test exfiltration attempts are caught Respond 1 2 3 Incident response plan Participating in C-CAR protocol Roles and responsibilities are verified No active critical vulnerabilities > 30 days Recover 1 2 3 Disaster and incident recovery plans Incident notification 100% Credit monitoring BPA Credit repair contract Legend Criteria Met Criteria Not Met Criteria NA To achieve a particular rating in a framework area, an agency must meet all criteria within both the given level and all prior levels. Level 1 - Foundational targets Level 2 - Government-wide targets Level 3 - Exceeding government-wide targets
9. Baseline Metrics In addition to the metrics underlying the PMC Assessment, OMB also analyzes agency performance on measures not used to calculate agency scores. The intent is to better understand current agency performance so ambitious but realistic targets can be set for future assessments. Current Baseline Metrics include: HTTPS implementation (M-15-13) Endpoints with data encrypted at rest (FIPS 140-2) Users with significant security responsibilities who have completed role-based security training Time to revoke role-based credentials following the termination of employees/contractors
10. Appendix: EOP Cybersecurity Assessment Criteria Identify Protect Detect Respond Recover Hardware Asset Management (CAP) Level 1 80% Level 2 95% (CAP Goal) Level 3 100% Software Asset Management (CAP) Level 1 80% Level 2 95% (CAP Goal) Level 3 100% Unclassified information systems with a security ATO Level 1 80% Level 2 95% Level 3 100% Review of contracts with sensitive information Level 1 Review of key contracts is in progress Level 2 Review of key contracts is completed Level 3 All contracts contain clauses on protection, detection, reporting of information Policy empowering incident commanders Level 1 In place Vulnerability Management (CAP) Level 1 80% Level 2 95% (CAP Goal) Level 3 100% Secure Configuration Management (CAP) Level 1 80% Level 2 95% (CAP Goal) Level 3 100% Remote access security Level 1 FIPS 140-2 validated Level 2 30 minute time out Level 3 prohibit split tunneling Unprivileged PIV logical access (CAP) Level 2 85% (CAP Goal) Privileged PIV logical access (CAP) Level 2 100% (CAP Goal) Insider Threat Program Level 1 Initial operating capability Level 2 Full operating capability Media destruction policy Level 1 In place Anti-Phishing and Malware Defense (CAP) Level 1 1 of 3 key indicators 90% Level 2 All key indicators 90% (CAP Goal) Level 3 All key indicators 100% EINSTEIN Program Level 2 Fully Implemented Attempts to access large volumes of data are detected and investigated Level 2 Detected and investigated Test exfiltration attempts are caught Level 3 Test conducted in past year and attempt was caught Incident response plan Level 1 Developed, tested once annually Level 2 Tested twice annually Level 3 No more than 180 days old Participating in C-CAR protocol Level 1 Participated in most recent C-CAR call Roles and responsibilities are verified Level 2 Verified during incident response testing No active critical vulnerabilities > 30 days Level 2 No vulnerabilities identified Disaster and incident recovery plans Level 1 Developed, but not tested regularly Level 2 Tested annually Level 3 Less than one year old Incident notification Level 1 Policy that establishes timeline for public or internal notifications after the detection or discovery of a compromise of PII is in place Level 2 Metrics tracking for notifications in place Level 3 Metrics indicate 100% compliance Credit monitoring BPA Level 2 In place Credit repair contract Level 3 In place
11. Appendix: Sample Action Items The following items require additional information/details on how the agency is working to meet government-wide targets: Category Criteria Questions Actions Needed Identify CAP Goal: Hardware Asset Management 1.2, 1.4, 3.16 Identify CAP Goal: Software Asset Management 1.5, 3.17 Detail actions to improve Hardware Asset Management capabilities or explain impediments to OMB Detail actions to improve Software Asset Management capabilities or explain impediments to OMB Identify Review of key contracts with sensitive information 1.8 Complete review of key prioritized contracts or explain impediments to OMB Protect CAP Goal: PIV logical access (unprivileged users) 2.4, 2.4.1 Protect CAP Goal: PIV logical access (privileged users) 2.5, 2.5.1 Detail actions to improve PIV usage amongst unprivileged users or explain impediments to OMB Detail actions to improve PIV usage amongst privileged users or explain impediments to OMB Protect Privileged user count has achieved target 2.5 Reduce number of privileged users where possible Protect Insider Threat Program, per E.O. 13587 2.30 Provide a status update of program implementation in next submission Respond Recover Incident response plan developed and tested biannually Recovery plans have been developed and tested annually 4.7 5.5 Increase specificity regarding the frequency of incident response plan testing and updating Ensure that enterprise-wide incident recovery plan is in place and updated on annual basis
12. Annual Assessment Timeline March 1 Annual FISMA report to congress released (from previous FY) January 1-15 CFO Act Agencies complete Q1 FISMA CIO assessment April 1-15 CFO Act Agencies complete Q2 FISMA CIO assessment July 1-15 CFO Act Agencies complete Q3 FISMA CIO assessment October 1-31 All agencies complete annual FISMA CIO assessment February 15 Q1 PMC assessments delivered to agency CIOs May 15 Q2 PMC assessments delivered to agency CIOs August 15 Q3 PMC assessments delivered to agency CIOs December 15 Q4 PMC assessments delivered to agency CIOs EOP conducts FY Q1 PMC assessments EOP conducts FY Q2 PMC assessments EOP conducts FY Q3 PMC assessments EOP conducts FY Q4 PMC assessments
IG FISMA Metrics - History Collaboration with OMB, DHS, CIGIE, and other stakeholders Historically, the OIG FISMA metrics were mostly yes/no questions that did not enable an easy determination of Effectiveness In 2015, a FISMA metrics subcommittee of the FAEC IT Committee was formed to develop effectiveness based measures
Maturity Model Approach Maturity model incorporates Federal requirements and maps to best practices (e.g., CMMI, CoBIT, NIST, C2M2) Maturity indicators map to CIO metrics, NIST 800-53 and supporting special publications, President s Management Council, and other governmentwide focus areas/initiatives FISMA Metrics Subcommittee has incorporated comments from various stakeholders including FAEC, CIGIE, and the CIO/CISO community 14
Effectiveness within the Maturity Model Level 1 Ad-hoc Level 2 Defined Level 3 Consistently Implemented Level 4 Managed & Measurable Operate Implement Level 5 Optimized Desired Results Effectiveness
FISMA IG and CIO Metrics FY 2017 FISMA IG Metrics are broadly aligned with the FY 2017 FISMA CIO Metrics Function (Section) IG Metrics CIO Metrics Identify (Risk Assessment) X X Protect (Configuration Management) X X Protect (Identification and Authentication) X X Protect (Security Training) X X Detect (ISCM) X X Respond (Incident Response) X X Recover (Contingency Planning) X X
Scoring Methodology FY 2017 FISMA IG Metrics scoring methodology will seek to provide a balanced assessment of agency information security capabilities Agency IGs will assess capabilities on a spectrum of potential maturity levels Overall maturity for each NIST Function will be recommended based on its average maturity level Goal is to provide a representative maturity level, but IGs can substitute a different score if they choose Overall agency maturity will be determined by the IG with no automatically generated recommendation This allows IGs to customize their assessments based on agency circumstances
Next Steps Access the metrics at DHS.gov/FISMA Hold follow-on training session in July Work with DHS to make changes to the Cyberscope application For 2018, develop a review guide or companion document to the metrics for IG use
Q&A Sample of the common questions received by the FISMA Metrics Subcommittee When is the due date for FISMA this year? Will the scoring methodology be different this year? Will specific questions be weighted differently than others? Is Level 4 still considered to be the bar for Effectiveness? Can an agency have an Effective program at Level 3? Will IGs be required to provide comments in Cyberscope for all responses not at a Level 4?