Automated Attack Framework for Test & Evaluation (AAFT)

Transcription

1 Automated Attack Framework for Test & Evaluation (AAFT) 34 th International Test and Evaluation Association Symposium October 4, 2017 Mr. Andrew Shaffer The Applied Research Laboratory The Pennsylvania State University

2 Outline Test & Evaluation Needs AAFT Overview Development Approach New Technologies System Tools and Capabilities Potential Use Cases Summary & Future Work 2

3 Test & Evaluation Needs X X System Accredited Following Red Team Testing New System Delayed By Limited Red Team Availability Deployed System Compromised Before Red Team Testing Completed DoD requires Red Team cyber penetration testing - Continuous and comprehensive monitoring are recommended Red Teams lack availability for all required system testing - Limited availability delays system accreditation and increases risk - Tools to improve Red Team utilization and efficiency are needed Blue Team validation and training require additional support - Automated Red Team tools are needed to free up testing personnel Red Team training is time-consuming and highly specialized - Knowledge of one attack tool often does not translate to others - Decision aids are needed to improve and accelerate training Automated tools are needed to improve Red Team utilization & efficiency 3

4 AAFT Overview Few available skilled Red Team testers must: Emulate all threats Train Blue Teams Validate vulnerabilities Penetration Testing Techniques Advanced Intermediate Basic Skilled Red Team testers can focus on advanced threats: AAFT emulates basic/intermediate threats AAFT trains Blue Teams AAFT validates vulnerabilities Current Practice AAFT Provides automated basic and intermediate-level penetration testing - Enables DoD comprehensive continuous monitoring - Provides decision aid for attack selection - Facilitates training and improves entry-level Red Team tester effectiveness - Generates comprehensive attack logs to support Red Team post-attack analysis Supports easy-to-use framework to facilitate integration of new attacks - Supports easy integration with new open-source and custom tools - Framework comparable to Metasploit with support for multiple attack tools Allows skilled Red Teams to focus on advanced threat emulation - Takes over less challenging penetration testing tasks - Takes over Blue Team training and rapid vulnerability verification AAFT provides an integrated framework to automate cyber attacks 4

5 Development Approach Develop database of real-world attacks - Specify attack preconditions, postconditions, timing requirements, risk of detection, and risk of damaging target Create modular attack scripts - Programmatically invoke attacks from command line - Capture, parse and store attack output Develop attack tree generation algorithms - Generate attack trees offline using threat representative attack scripts that meet timing and risk tolerances - Algorithms automatically reconfigure attack tree whenever new attack scripts become available Develop Autonomous Attack Engine & GUI - Create new algorithms to optimize attack selection based on attack objective, timing, and risk parameters Incremental development ensures ongoing capability improvement 5

6 New Technologies Attack Scripts Attack Timing Detection Risk Attack Command Damage Risk Attack parameters Attack script issues attack command, captures and parses attack tool output, and updates fulfilled conditions Attack tool output and obtained target information is stored for future use Target knowledge store Attack scripts encode simple attack commands and parameters - Specify preconditions (i.e. knowledge required) for attack to proceed - Specify estimates of time required, risk of detection, and risk of damaging target - Specify expected postconditions (i.e. knowledge gained) if attack is successful Scripts capture attack tool output following attack execution - Parse tool output and update list of conditions that have been fulfilled Simple well-defined interfaces facilitate community creation of new attack scripts - Community involvement supports broad coverage of current and new attack tools Attack scripts encode individual attack tool commands 6

7 New Technologies Attack Tree Generation Start IP Address Port Scan IP Address Open Ports List * Italicized text indicates knowledge to be gained by attack Initial Exploit 1 Initial Exploit N IP Address Open Ports List Exploit 1 Successful Escalate Privileges IP Address Open Ports List Exploit N Successful IP Address Open Ports List Exploit 1 Successful Privileges Escalated Root Access Root Access AAFT attack trees encode attack sequences and target knowledge - Overall attack initially starts with minimal target knowledge - Each attack script attempted gains additional target knowledge - Attack tree nodes encode specific attack scripts and knowledge about target at specific points during attack Attack trees are generated offline using all available threat representative attack scripts that meet timing and risk tolerances - Attacks are added to the tree wherever their preconditions are fulfilled and the attack has the potential to gain additional target knowledge - Attacks are added to tree recursively until no further additions are possible - Branches not reaching the objective are pruned once tree generation is complete Attack Trees are generated offline to match specific threat profiles 7

8 Start: Full threatrepresentative attack tree w/minimal target knowledge Update Attack Tree to Reflect Latest Target Knowledge Select & Execute (or Recommend) Next Attack Update Target Knowledge Autonomous Attack Engine Processing Cycle New Technologies Autonomous Attack Engine Finish: Attack transcript showing how objective achieved or determined unreachable Autonomous Attack Engine iteratively selects and executes optimal attack, updates target knowledge, and updates attack tree until attaining objective or exhausting all available attacks - Optimally selects next attack based on currently available information - Continually updates alternative attack options as target knowledge increases - Changes attack paths whenever a better attack option becomes available Autonomous Attack Engine can also recommend attacks as a decision aid instead of autonomously executing them Autonomous Attack Engine provides high-level direction for AAFT attack 8

9 System Tools Initial AAFT development is focused on common open-source tools - Nmap (network mapper) - Metasploit (integrated attack framework) - THC Hydra (online login cracker) - John the Ripper (offline password cracker) Attack scripts and target knowledge encoded using Extensible Markup Language (XML) - XML is portable and highly extensible - Open-source and custom attack tools use the same interfaces Python facilitates rapid software development - Highly portable high-productivity language that is widely used VirtualBox supports software portability and easy installation - AAFT system will ship as a ready-to-run virtual machine image - Simplifies end user configuration and maintenance requirements AAFT framework supports both open-source and custom tools 9

10 System Capabilities Autonomous Attack Engine Automation enables continuous comprehensive monitoring Open-Source Attack Tool #1 New Custom Attack Tool Open-Source Attack Tool #2 Open-Source Attack Tool #3 Framework design enables easy new tool integration Custom Tools Automation enables highlevel direction of attacks Automation supports DoD continuous monitoring - Increases testing rigor and reduces reauthorization costs Framework enables easy integration with current and future tools - Leverages developments in open-source attack tools - Supports customized/proprietary attack tools - Supports community involvement to provide broad coverage of attack tools Automation and decision aids expand Red Team capabilities - Allow skilled testing personnel to focus on more challenging tasks - Increase capabilities of entry-level cyber testing personnel - Improve Blue Team vulnerability validation and training AAFT automation, integration framework, and decision aids improve cyber test and evaluation capabilities 10

11 Potential Use Cases Skilled Red Team focuses on advanced attacks AAFT facilitates and automates basic and intermediate attacks Allow skilled Red Team testers to focus on advanced attacks Perform basic and intermediate-level Red Team attacks using an innovative Autonomous Attack Engine Accelerate vulnerability validation and remediation for test & evaluation Support Blue Team training X X Attacks blocked by accelerated Blue Team remediation Automated and advanced attacks discover new vulnerabilities AAFT accelerated vulnerability validation AAFT-enabled training improves Blue Team efficiency Legend: Red: AAFT supports Red Team Blue: AAFT supports Blue Team AAFT supports both Red Team and Blue Team cybersecurity efforts 11

12 Summary Skilled Red Team Testing Penetration Testing Techniques Advanced AAFT Automated Testing Intermediate Basic Systems Under Test Automatic attack tree generation and Autonomous Attack Engine are new technologies that advance cybersecurity test and evaluation - Applying autonomy research to cybersecurity improves state-of-the-art in Red Team and Blue Team testing AAFT addresses critical cybersecurity Test and Evaluation needs - Enables better utilization and efficiency of available Red Team testers - Increases scope of cybersecurity testing that can be performed System design and development is ongoing - Created attack tool test lab and system prototype development lab - Developed initial attack scripts and algorithms for attack tree generation - Began initial development of Automated Attack Engine AAFT advances state-of-the-art in Red Team and Blue Team testing 12

13 Future Work Skilled Red Team Testing Penetration Testing Techniques Advanced AAFT Automated Testing Intermediate Basic Incrementally increase tool and attack sophistication IP & Non-IP Systems Under Test - Add support for concurrent, persistent, and social engineering attacks Legend: Yellow: Future Work Focus Add support for automated analysis of harvested target information - Apply automated text analysis and natural language processing to identify key IP addresses, port numbers, and user information that may enable greater penetration of target networks Automate and accelerate non-ip testing - Non wireless, acoustic, serial, USB, MIL-STD-1553, ARINC-429, PROFIBUS, Modbus, etc also require penetration testing - Standardized protocol decoding and attacks can significantly accelerate testing Seeking transition sponsors & funding to support further maturation 13

14 Andrew Shaffer Penn State Applied Research Laboratory Bruce Einfalt Penn State Applied Research Laboratory Points of Contact 14

15 Questions 15