Nov 2012 Page-1 Dr. Steven J. Hutchison Principal Deputy Developmental Test and Evaluation November 2012
Nov 2012 Page-2 DT&E for Complex Systems Performance Reliability Interoperability Information Security Operations Test & Evaluation Training Experimentation Modeling & Simulation System Integration Labs JIOR Cyber Range JMETC Persistent, rapidly composable, secure representation of the Joint Information Environment
Nov 2012 Page-3 The DoD Acquisition Model
Nov 2012 Page-4 Test, Evaluation, Certification Security T&E DIACAP Late to Need!
Hindsight is 20-20 Compliance with IA Controls and Interoperability Standards and Profiles are necessary but not sufficient DISTRIBUTION STATEMENT A Cleared for Open Publication by OSR on What did we test? What did we know? Fielded systems: Interoperability issues IA Vulnerabilities To reduce discovery late in the acquisition lifecycle, test in mission context, against realistic threat, and..! DOT&E COCOM/Service Interop & IA Assessments Nov 2012 Page-5
Net Ready KPP New Role for DASD(DT&E) Nov 2012 Page-6 New Language DISA will ensure JITC leverages previous, planned and executed DT&E and OT&E tests and results to support joint interoperability test certification and eliminate test duplication. DASD(DT&E) shall approve Developmental Test and Evaluation plans in support of Joint Interoperability Test Certification as documented in the TEMP. JITC shall advise DASD (DT&E) regarding the adequacy of test planning in support of Joint Interoperability Test Certification. CJCSI 6212 DASD(DT&E) approves adequacy of Interoperability test planning
Nov 2012 Page-7 Information Assurance Policy Information Assurance compliance activities need to be integrated into DT&E and included in the TEMP
Information Assurance What s Changing? Nov 2012 Page-8 Implements Risk Management Framework (RMF) instead of Mission Assurance Category/Confidentiality Level (MAC/CL) Adopts new guidance from the National Institute of Standards and Technology (NIST) and Committee on National Security Systems Instruction (CNSSI) documents on Cybersecurity Goes beyond IA and adopts the term: Cybersecurity Lexicon Changes Certification and Accreditation becomes Assessment and Authorization Designated Approving Authority (DAA) becomes Authorizing Official (AO) Certifying Authority becomes Security Control Assessor Threat = Any event with potential to cause harm to the network Vulnerability = Absence/weakness of safeguards to protect the network Risk = Likelihood that a threat will realize or exploit a vulnerability
Implementing Cybersecurity What s Being Proposed? Nov 2012 Page-9 DASD(DT&E): Oversight of test planning in support of Cybersecurity C&A(A&A) Establish procedures to ensure that DT&E authorities for acquisition programs verify that adequate DT&E is planned and resourced to address Cybersecurity Confirm DT&E can be executed in a timely manner prior to approval of program Test and Evaluation Master Plans (TEMPs) DASD(DT&E) will ensure adequate Cybersecurity test planning
Nov 2012 Page-10 DT&E in the Cyberspace Domain Process Methodology Desired Federated Cyberspace T&E Capability Systems Under Test Instrumentation ACETEF BAF SDREN JPRIMES CDS TSMO IO Range Test Tools Cyberspace Threat Representations Workforce Infrastructure An Integrated T&E Enterprise Capable of Creating a Realistic Cyberspace Test Environment at All Required Security Levels Persistent, rapidly composable, secure representation of the Joint Information Environment
DT&E Cybersecurity Process Summary Nov 2012 Page-11 Step 1 Cybersecurity Test Requirements Evaluation Focus on initiating an approach to Cybersecurity DT&E at Milestone A or B, with update at Milestone C. Step 2 Cybersecurity System Integration Evaluation Focus is assessment of Cybersecurity in component and system integration vulnerability testing, between MS B and C. Step 3 Cyber Kill Chain Evaluation Focus is assessment of Cybersecurity of the system under test, in a realistic mission and cyber environment, using exploitation testing techniques, post-cdr. Step 4 Cybersecurity Test in Realistic Cyber Environment Focus is on Cybersecurity readiness in an operational mission environment to understand capabilities and limitations of the SUT and interconnections against a cyber threat using Red Team testing.
Nov 2012 Page-12 Cybersecurity Testing in the Acquisition Lifecycle AOTR MS A MS B MS C Full Rate Production Decision Review Strategic Guidance (OSD/JCS) Joint Concepts (COCOMs) CBA ICD MDD Materiel Solution Analysis Technology Development CDD Engineering & Manufacturing Development CPD Production and Deployment O&S JCIDS Process AoA ASR SRR SFR PDR CDR TRR SVR IOT&E TDS TEMP SEP SRD STAR TRA AS *TEMP *SEP PPP OTR STAR * *PPP TEMP * STAR * Cyber Test Step 1 Cyber Test Step 1 Step 2 Cyber Test Step 1 Step 2 Step 3 Cyber Test Step 1 Step 2 Step 3 Step 4 Reduce the Cyber Attack Surface
Nov 2012 Page-13 Conclusion DT&E in mission context Improve Interoperability Improve Cybersecurity Reduce discovery in IOT&E Improve Acquisition Outcomes To ensure rapid fielding of enhanced capabilities to the Warfighter!
Nov 2012 Page-14 Questions?
DoD Test, Evaluation, & Certification Nov 2012 Page-15 Multiple Test Orgs DT, OT, Iop, IA Multiple Decision Makers MDA, CIO, DAA DT&E Test Concept Brief Operational Test Plan User Training AOTR OTRR Test Plan Approved Tester Training Support Implemented DIACAP Pilot IAC&A OT&E OTRR Interop Testing Record Interop Cert Eval Report Full Deployment Decision Review 60 days 60 days 14 days 60 days T&E Plan Test Report cycle can exceed six months!