Cybersecurity T&E and the National Cyber Range

Transcription

1 Cleared for Public Release 23 March 2017 Case # 17-S-1267 Cybersecurity T&E and the National Cyber Range Prepared for 2nd ITEA Cyber Security Workshop "Challenges Facing Test and Evaluation 24 March 2017 Prepared by National Cyber Range Team Peter H. Christensen Range Director, NCR Cleared for Public Release 23 March 2017 Case # 17-S-1267

2 What, Why, How? What do we want to accomplish? Look back to where we were Provide insight into TRMC Cybersecurity Test and Evaluation Infrastructure and the National Cyber Range Highlight Lessons learned and NCR successes Make some predictions about the future! Why is this important? Cyberspace T&E is extremely challenging TRMC has Operationalized the NCR over six years of intense operations NCR Team has delivered significant value added to DOD customers The future looks promising! How will we do it? Look back to 2009 IA Policy Crosswalk and 2010 IT Acquisition Reform Highlight TRMC/NCR progress and new T&E capabilities Look ahead to the future 2

3 Looking Back 2009: Information Assurance (IA) Policy Cross Walk Proposed by DISA T&E Exec Chartered by DOT&E and DASD DT&E Examine current IA related T&E policies, directives, instructions and guidance Provide findings and recommendations 3

4 Key Findings Key Findings from 2009 Information Assurance Policy Cross Walk Focus on coordinating IA Test and Evaluation Activities Form Integrated Test Teams Early and enable test by one, use by many Promote Collaboration among Acquisition, Engineering and Test Teams Operationally Realistic IA Test Environment is Crucial to Successful Testing Threat Portrayal During Testing Must Reflect Current Threat Information Provide Adequate Resources & Expertise Essential for Testing Promote Acquisition-Related IA and Computer Network Defense (CND) T&E as Critical to Ensure Secure DOD Systems IA and CND testing in acquisition is a critical activity and can only be accomplished with adequate resources and threat characterization! 4

5 2010 NDAA Section 804 IT Acquisition Reform Congress directed DoD to develop a new IT Acquisition Process DCMO Section 804 Task Force established T&E and Certification Government Lead Principal Deputy DASD DT&E Test Evaluation and Certification (TE&C) Process collaboratively developed Supported an 18-month release goal, embraced both SE discipline and Agile Methods Delivered to DCMO 01 Oct 2010 Industry and DOD have embraced Agile DOD 5000 incorporates Incrementally Fielded Software Intensive Programs Agile methods influenced proposed Cybersecurity T&E process 2010: Strong Leadership is required to progress forward! 5

6 Key Elements of 2010 Draft TE&C Policy/Process Agile TE&C execution Tailored to the IT acquisition Responsive to evolving requirements Risk based and mission focused TE&C infrastructure and tools Provides testing as a service Development, deployment, and sustainment Verified, Validated, and Accredited (VV&A) Infrastructure Replicates an operational/production environment Provides repeatable and defensible test results 6

7 March 2017 TRMC Organization Under Secretary of Defense for Acquisition, Technology & Logistics Mr. Jimmy MacStravic (Acting) Dir, TRMC Updated 12 Dec 2016 Chief Financial Officer Chief Operating Officer ** Mr. Derrick Hinton (Acting) Principal Deputy, TRMC Mr. Paul Mann (Acting) Deputy EA for Cyber Test Ranges Range Director, NCR Sr. Security Advisor Admin. Officer Dr. Robert N. Tamburello Deputy Range Director, NCR DD, T&E Range Oversight * PM, CTEIP DD, Major Initiatives and Technical Analyses * Agency RO ** AF RO ** PM, REP Deputy PM, CTEIP PM, T&E/S&T PM, JMETC Army RO ** Navy RO ** * Supervisor ** Team lead Deputy PM, T&E/S&T Director, TENA SDA 7

8 TRMC Team Includes Government, FFRDC, SETA And Contractor Infrastructure and Services Include: Test Bed Design Support Integration of Custom Assets Software Hardware Wired and Wireless Remote Red/Blue Team Support Cyber and Testing Expertise Threat Vector Development Custom Traffic Generation Custom Sensor and Visualization Support Custom Data Analysis End-to-End Test Support Customers Identify Cyber T&E Requirements TRMC Provides Infrastructure And Services To Satisfy Them! 8

9 TRMC Cybersecurity T&E Infrastructure Components: JMETC MILS and Regional Service Delivery Points (RSDPs) JMETC MILS provides secure connectivity for Cybersecurity T&E MILS VPN hosted on Defense Research and Engineering Network Peers with Joint Information Operations range Regional Service Delivery Points (RSDP) Computing and Storage Assets that host virtual Cyberspace Environments TRMC provides tools and services and instrumentation for traffic generation, visualization, integrated event management, collaboration Geographically dispersed to minimize latency and maximize usability Modular/Flexible architecture evolves with requirements TRMC provides Support Staff Available to help plan, design and execute events TRMC will provision the right Cyber T&E Infrastructure based upon your needs! 9

10 What Is The National Cyber Range? NCR Provides Cybersecurity Testing And Training As A Service! 10

11 NCR Executed 140 Events in Six Years Note: FY-16: NCR unavailable for 9 Weeks during A&A NCR 2016 Aviation Week Program Excellence Winner! Helping developing programs understand, mitigate cyber-attack and prepare people to defend U.S. systems 11

12 1. Start Small and grow Cybersecurity Testing Lessons Learned 2. Testing is an important Engineering and Design Tool that can be used to refine requirements 3. Cyber Table Top is an effective tool to understand Mission Risks and prioritize testing 4. Focus Cybersecurity Testing on the Mission! 5. Cybersecurity Testing must be executed with key IT Staff, Incident Responders and Protection Teams 6. Customers need Cybersecurity T&E As a Service 7. Collaborative Approach to Coordinate Test Planning Critical 8. Effective Test Teams understand Cyber Offense and Defense 9. Automated Tool Suite creates efficiencies in event design, development and deployment 10. Connectivity makes range location irrelevant Lessons Learned from NCR Events 12

13 Cyber Testing Engineering and Design Tool Cybersecurity Testing is Systems Engineering Tool Reduces technical debt, Identifies exposed vulnerabilities and provides engineering alternatives Identifies New Cyber Requirements, exposes Residual Vulnerabilities! No System will ever be 100% Secure! What is the Mission Risk? 13

14 Cybersecurity T&E Helps Manage Mission Risk Phases are iterative and incremental! Initial Phases reduce Type 1 Debt! Complements System Security Engineering and Risk Management Framework SE and RMF Activities Later Phases reduce Type 2 Debt! Promotes Understanding of Mission Risk! RMF Manages Acquisition Program Risk.Cybersecurity T&E Manages Mission Risk 14

15 Cyber Table Top (CTT) Effective Tool To Prioritize Cybersecurity T&E What is a Cyber Table Top? Low technology, low cost, intellectually intensive wargame Introduces and explores Mission Effects of Offensive Cyber Ops Helps estimate Mission Risk to System, SoS or FoS Why is it used? Identifies Threat Vectors, Potential Vulnerabilities and Mission Risk Identifies new Functional and Non Functional Requirements Scopes the size and scale of Cybersecurity Testing Provides actionable recommendations What does it produce? Prioritization of vulnerabilities based upon likelihood of exploitation and consequences to the mission High Risk Vulnerabilities must be evaluated first Medium Risk Vulnerabilities can be evaluated subsequently Low Risk Vulnerabilities may be deferred for later Cybersecurity Risk Matrices and Recommendations for Cybersecurity Testing and Vulnerability Mitigation CTTs Informed Several Events At The NCR! 15

16 Customers Need Cybersecurity T&E As A Service Cyber Testing and Training Demand is for Customer Centric Services (predominantly) Customers come to the TRMC/NCR for Higher Tier Live Cyber Environments, Cyber T&E expertise, and event support 16

17 Automated Tools Simplify Event Design, Development, Deployment/Redeployment! Reusable Content Includes: ADNS Emulation Round-robin NTP Full DNS infrastructure Whois Various Exchange Server versions and architectures DNS registrar Webmail ecommerce sites Content Management Systems (CMS) Model Reuse Is Creating Efficiencies! 17

18 Connectivity Makes Range Location Irrelevant! NCR demonstrated ability to support Major Training Exercises! Remotely supported 1000 s of Users Connected numerous Logical Ranges 100 s of Enclaves & Subnets Thousands of Nodes RSDPs PSDPs Realistic Mission Environments JMN NCR demonstrated ability to support remote Testing NCR has both JMN and JIOR Connectivity! Used remotely for multiple customers Connectivity Enables Customers To Create A Virtual Enterprise For Testing And Training! 18

19 NCR Objective Pathfinder Event: Avionics Bus Architectures Emulate Non IP Based Networks, Bus Architectures: MIL Std 1553, ARINC 429 for Cyberspace T&E Relevance Cyber Threats demonstrated ability to exploit vulnerabilities in systems, subsystems and components Some testing cannot/should not be conducted w/ actual aircraft, particularly in-flight Outcomes TRMC/NCR collaborated with NAVAIR to conduct a Cyber Table Top to understand Cyber Risks NCR built-out Avionics Management System to execute Risk Reduction Event Collaborated with Aircraft Cyber Threat Working Group 19

20 Pathfinder Event: Control Systems Cyber Security (CS 2 ) Challenge NCR Objective Demonstrate ability to emulate representative Industrial Control Systems Architecture Relevance Internet of Things makes Control Systems connected and exploitable Developers fail to consider Cybersecurity Control Systems need to be tested in real-world environments Outcome OSD Energy, Installations and Environment (E&EI) requested NCR implement a representative DOD Building Environmental Control System NCR successfully developed environment Follow on demonstrations will evaluate commercial products Image Source: Mario Morales (IDC) The Internet of Everything Exposes Control Systems to Attacks Not Previously Considered! 20

21 What Does the Future Look Like for Infrastructure? Demand for Cybersecurity T&E Infrastructure will continue to increase NCR/RSDP/JMN Complex being provisioned to satisfy increasing demand Way Ahead: Leverage TRMC Investment Ensure future investments in automation tools are Interoperable/Compatible/Un-encumbered Move Legacy SILs HW&SW, ICS/SCADA Labs, etc. to JMN 21

22 What Does the Future Look Like for Workforce? Demand for Cybersecurity T&E Workforce will continue to increase People are the most valuable and limited resource Workforce must have Acquisition and Cyberspace Skills Capabilities Development, Program Management, Contracting, Systems Security Engineering, Risk Management Framework and Cybersecurity T&E, Cyber Defense and Offense Way Ahead: Invest in Wet Ware Enhance Government Cybersecurity T&E Workforce Enable them with robust FFRDC, SETA and Industry Personnel Procuring Cybersecurity T&E Infrastructure Is Easy Greater Challenge: Developing The Cybersecurity T&E Workforce! 22

23 What Does the Future Look Like In Practice? Investments in T&E Infrastructure reduce Technical Debt : PMs realizing schedule/performance improvements Way Ahead: Provision infrastructure to support full spectrum testing Not just one Program of Record Use incremental/iterative T&E to evaluate System Functional and Non Functional Requirements Autonomous Systems could adopt a similar approach Adversarial Assessments should be Mission Based Events Fully exercise Systems of Systems Include Cybersecurity Defense Providers TRMC Is making Essential Investments! 23

24 What Does the Future Look Like For The Adversary? Adversaries are sophisticated, persistent and getting better Conduct extended and sophisticated Cyber Campaigns Speed and agility enables them to get inside the Acquisition Lifecycle DOD Acquisition systems and approaches lack agility to keep up with the threat Industry has adopted Agile Methods/DEVOPS! Robust DEV/Test Infrastructure essential to support continuous innovation Increases efficiency and Delivery Velocity enhances Cybersecurity Posture and reduces ownership costs Enables Industry to evolve at the same pace as the adversary Infrastructure Investments Enable Agile Methods/DEVOPS! 24

25 What Does the Future Look Like For The Adversary? (cont.) Way Ahead: Investment in Cybersecurity T&E Infrastructure is having positive impact TRMC S&T and CTEIP Programs enablers to help disrupt the Adversary Lifecycle Enable Agile Methods and DEVOPS with High Fidelity reusable emulations/models of Systems and Enterprise Exploit virtualization, Software Containers Consider provisioning Digital Twins Evolve with system as it matures Promote closer community engagement Development Community, Cybersecurity Defense Providers and Cyber Mission Forces Effort Needed To Bring Together Development, Testing And Training Communities! 25

26 Summary Since 2009: DOD T&E Community has significantly advanced the practice of Cybersecurity T&E Past approaches to address Cybersecurity have created Technical Debt! Shift Left is Helping Programs improve Cybersecurity Posture Other Federal Agencies are adopting similar approaches! TRMC and the NCR are helping Testing and Training Customers! Deliver unique cybersecurity test, evaluation, and training capabilities Enable DOD to conduct focused cybersecurity test and evaluation Events are tailored to meet program requirements throughout the systems acquisition lifecycle TRMC is investing in the future! Workforce and Infrastructure investments are key enablers Without them advances in practice process cannot be achieved! TRMC/NCR Team Are Making It Harder For The Adversary! 26

27 Cleared for Public Release 23 March 2017 Case # 17-S-1267 Special Thanks! PROGRAM CHAIR - Ms. Chris Susman SURVICE Engineering Company PROGRAM TECHNICAL CHAIRS Mr. Robert Laughman, US Army Evaluation Center Duane Wilson, Ph.D., Wilson Innovative Solutions LLC EXHIBITS & SPONSORSHIPS Ms. Cathy Pritts and Mr. Jim Myers Superb effort organizing this workshop! Cleared for Public Release 23 March 2017 Case # 17-S

28 The 2017 CG Classic In Honor of Seaman Aaron N. Redd, USCG Please join the Chief Petty Officers Association Alexandria Chapter and the family & friends of Aaron N. Redd, on Friday, June 16th, as we host a fun filled day of golf at the Potomac Shores Golf Club. All proceeds will be donated to the Coast Guard Enlisted Memorial Foundation. 28

29 Cleared for Public Release 23 March 2017 Case # 17-S-1267 Questions? Peter H. Christensen Range Director, National Cyber Range TRMC Office Phone: TRMC peter.h.christensen.civ@mail.mil Dr. Robert N. Tamburello Deputy Range Director, National Cyber Range TRMC robert.n.tamburello.civ@mail.mil Address: 4800 Mark Center Drive Suite 07J22 Alexandria, Va Cleared for Public Release 23 March 2017 Case # 17-S-1267