Leveraging Data Provenance to Enhance Cyber Resilience

Transcription

1 Leveraging Data Provenance to Enhance Cyber Resilience Thomas Moyer Karishma Chadha, Robert Cunningham, Nabil Schear, Warren Smith, Adam Bates, Kevin Butler, Frank Capobianco, Trent Jaeger, and Patrick Cable IEEE SecDev Nov 2016 This material is based upon work supported by the Assistant Secretary of Defense for Research and Engineering under Air Force Contract No. FA C-0002 and/or FA D Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Assistant Secretary of Defense for Research and Engineering. DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited.

2 battles, campaigns, and even wars have been won or lost primarily because of logistics. - General Dwight D. Eisenhower Div. 5 Provenance - 2

3 Data Resilience for US Transportation Command USTRANSCOM challenges 70% of transportation work done by 3 rd party contractors APTs have targeted and attacked USTRANSCOM directly and via contractors Goal: Ensure integrity of logistics planning operations as they transition to the cloud Monitor and inform users of data integrity during logistics planning USTRANSCOM needs resilient systems to ensure DoD mission success Div. 5 Provenance - 3

4 Federal Cybersecurity Research and Development Strategic Plan What is Resilience? Malicious Cyber Activities Div. 5 Provenance - 4 Figure 1. Continuously strengthening defensive elements improves success in thwarting malicious cyber activities. Federal Cybersecurity Research and Development Strategic Plan, National Science and Technology Council, February 2016 Critical Dependencies Data provenance provides detection and adaptation capabilities for a resilient system Advancements in the following six areas are critical to developing the S&T for the four elements: Scientific foundation. The Federal Government should support research that establishes the theoretical, empirical, computational, and data mining foundation needed to address future threats. A strong,

5 Outline National Need: Resilient Systems Data Provenance Use Case: Data Integrity for USTRANSCOM Future Work and Summary Div. 5 Provenance - 5

6 Data Provenance Enables Resilience Data provenance is the history of ownership/processing to guide authenticity wasderivedfrom Processes Data Entity used Activity wasgeneratedby Entity wasattributedto wasassociatedwith wasattributedto Users, groups, other systems, etc Agent World Wide Web Consortium Data provenance helps to answer: Where are all my data? Where did they come from? Are the data secure and trustworthy? How to recover after being attacked? Div. 5 Provenance - 6

7 Secure Data Provenance Challenges Granularity Collection Encoding Storage Analysis Adaptation Granularity How much detail to collect? Collection Where to collect provenance data? Encoding Do current standards allow the system to fully express the semantics of the data? Storage How is the provenance data protected against malicious modifications? Analysis What can the collected data tell system users? Adaptation What actions are possible, based on the analysis of the provenance data? Answering these questions incorrectly leads to a provenance system that will not achieve the desired goals Div. 5 Provenance - 7

8 Lincoln Secure Data Provenance Technology Granularity Collection Encoding Storage Analysis Adaptation Context Mission System Applications Developer Annotation Library Active Response Coverage Software Infrastructure Operating Systems Provenance Collector Linux Provenance Modules Entity Activity Agent Entity World Wide Web Consortium Secure Graph Ancestors Descendants Anomaly Detection Operator Interfaces End-to-end integrated provenance system enabling mission system resilience Div. 5 Provenance - 8

9 Outline National Need: Resilient Systems Data Provenance Use Case: Data Integrity for USTRANSCOM Future Work and Summary Div. 5 Provenance - 9

10 Operational Architecture OV-1: Distribution High Level View Plan Order Ship Pay Protecting the integrity of the data and processing in the planning pipeline is critical to ensuring mission success for USTRANSCOM Div. 5 Provenance - 10

11 Provenance for the Planning Process Plans produced by the correct processing pipeline are required for mission success Requirements are generated from a request A plan is generated from requirements Ultimately, a plan is derived from a request wasderivedfrom Request used Requirements Collection wasgeneratedby Requirements used Planning Service wasgeneratedby Plan wasassociatedwith wasassociatedwith Analyst Planner Div. 5 Provenance - 11

12 Anomalies in the Planning Process If data used to create a plan is unexpectedly modified, the mission is at risk Requirements generated from a request, and modified by malicious software, or attacker A plan is generated from modified requirements Plan no longer derived from the expected requirements Request used Requirements Collection wasgeneratedby Requirements used Planning Service wasgeneratedby Plan wasassociatedwith Modified Reqs. wasassociatedwith Analyst Planner wasderivedfrom Div. 5 Provenance - 12

13 Lincoln Secure Data Provenance Technology Granularity Collection Encoding Storage Analysis Adaptation Context Mission System Applications Developer Annotation Library Active Response Coverage Software Infrastructure Operating Systems Provenance Collector Linux Provenance Modules Entity Activity Agent Entity World Wide Web Consortium Secure Graph Ancestors Descendants Anomaly Detection Operator Interfaces End-to-end integrated provenance system enabling mission system resilience Div. 5 Provenance - 13

14 Example Code Annotation Added instrumentation: 300 lines of code for 311K lines of code in system Attributes.add( RequirementsId, requirementsid ); Attributes.add( TransactionId, transactionid ); Requirements = newentity( Requirements, Attributes ); Create provenance graph entries StoreProvenance( transactionid, Requirements ); Store provenance Data provenance collection requires very little instrumentation to provide high impact on system resilience Div. 5 Provenance - 14

15 Seconds Data Storage in MB Overhead Collection Overhead Storage Overhead % Overhead <1% Overhead Req. Collect Req. Store Gen. Plan Operation 0 Data Execution Time Prov. Collection Time Planning Data Provenance Collection and storage overheads are a tiny fraction of the overall computation and storage costs for the system. Div. 5 Provenance - 15

16 USTRANSCOM Provenance Integration Provenance Analysis Integrated in Operator User Interface Good plan, using requirements from known sources Insecure plan using requirements from an unknown source Provenance-based data resilience is integrated into mission operators workflow Div. 5 Provenance - 16

17 Provenance Collector Overhead (%) Lincoln Secure Data Provenance Technology Granularity Collection Standard Encoding Secure Storage Detect Adapt Context Mission System Built Secure and Efficient OS-level Provenance Collector Coverage Applications Software Infrastructure Operating Systems Developer Annotation Library Collector Linux Provenance Modules Graph Graph Analysis Ancestors Descendants Anomaly Detection Compilation Mail Server DNA Search System Workload Active Response Operator SA Displays End-to-end integrated provenance system enabling mission system resilience Div. 5 Provenance - 17

18 Cumulative Density Lincoln Secure Data Provenance Technology Granularity Demonstrated Efficient Queries Standard on Collection Large Provenance Graphs Encoding Secure Storage Detect Adapt Context Mission System Applications 99% of queries return in less than 2.5ms Developer Annotation Library Active Response Coverage 0.96 Software Infrastructure Collector Response Time (Milliseconds) Ancestry queries on 6.5M node provenance graph using SNAP in-memory graph Linux database Operating Systems Provenance Modules Graph Graph Analysis Ancestors Descendants Anomaly Detection Operator SA Displays End-to-end integrated provenance system enabling mission system resilience Div. 5 Provenance - 18

19 Lincoln Secure Data Provenance Technology Granularity Collection Standard Encoding Secure Storage Using Data Provenance to Prevent Data Exfiltration Detect Adapt Mission System Context Applications Developer Annotation Library Active Response Coverage Software Infrastructure Operating Systems Collector Linux Provenance Modules Provenance Collector Web Server Graph? Graph Analysis Ancestors Guard Descendants Anomaly Detection 6.1ms latency to prevent SQL command injection attacks Operator SA Displays End-to-end integrated provenance system enabling mission system resilience Div. 5 Provenance - 19 W3C World Wide Web Consortium SA Situational Awareness SQL Structured Query Language

20 Outline National Need: Resilient Systems Data Provenance Use Case: Data Integrity for USTRANSCOM Future Work and Summary Div. 5 Provenance - 20

21 Future Work Challenge Current analytics only provide awareness of problems Solution Active response mechanisms to recover from anomalies Legacy code requires manual provenance instrumentation Code analysis to automatically retrofit legacy code Current analysis is tailored to workflow for a specific system Enhanced analytics that detect deviations in an automated way Systems and analytics rely on a single provenance sensor Integration of multiple provenance sensors for a holistic view of data processing Div. 5 Provenance - 21

22 Summary Granularity Collection Encoding Storage Analysis Adaptation Mission System Context Applications Developer Annotation Library Active Response Coverage Software Infrastructure Operating Systems Provenance Collector Linux Provenance Modules Entity Activity Agent Entity World Wide Web Consortium Secure Graph Ancestors Descendants Anomaly Detection Operator Interfaces Active work to transfer this technology to other mission areas within the lab ISR, Space, Ballistic Missile Defense Provenance is capable of providing resilience for data processing to ensure mission success Div. 5 Provenance - 22

23 Getting Linux Provenance Modules Linux Provenance Modules is available for download from Div. 5 Provenance - 23

24 Acknowledgements University Collaborations Kevin Butler Patrick Cable Karishma Chadha Rob Cunningham Jeff Diewald Ben Kaiser Bryan Richard Robert Rudd Nabil Schear Warren Smith Adam Bates Trent Jaeger Frank Capobianco Informatics and Decision Support Group Information Integration and Decision Support Group Michael Calder Christopher Botaish George Heineman Div. 5 Provenance - 24

25 Legal Notices 2016 Massachusetts Institute of Technology. Delivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS or DFARS as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work. Div. 5 Provenance - 25