Case Study Experiences from the DIAMONDS Project 8 th ETSI Security Conference January, Sophia Antipolis - France

Transcription

1 Case Study Experiences from the DIAMONDS Project 8 th ETSI Security Conference January, Sophia Antipolis - France Ina Schieferdecker

2 DIAMONDS Project In six countries Project Duration: October 2010 March 2013 Project Partner: Large companies (6) Small companies (9) Universities (3) Research institutes (4)

3 Introduction & Relevance Effective and Efficient Security Testing DIAMONDS will enable efficient and automated security testing methods of industrial relevance for highly secure systems in multiple domains. Objectives: Security test methodologies and test patterns Automatic monitoring techniques Open source platform for security test tool integration Business Impact: 6 different industrial domains Pre-standardization work Novel integration of testing, security analysis and risk orientation

4 Innovation & Expected Results Combination of Approaches Security Testing Model-based Testing

5 DIAMONDS Innovation Expected Results Achieved Advanced model-based security testing methods: Modelbased fuzz testing Autonomous testing techniques based on automatic monitoring techniques: Passive symbolic monitoring, Integration of monitoring and model-based test generation Pre-standardization work on multi-domain security test methodologies and test patterns: Risk-based security testing methodology, security test pattern Open source platform for security test tool: Traceability platform for security testing, Malware reverse engineering and its application to testing

6 Case Studies Six Industrial Domains Security testing solutions for six industrial domains Banking Automotive Radio protocols Smart cards Telecommunication Industrial automation

7 DIAMONDS Results So Far Techniques and their Application Risk-Based Testing (Banking, Automotive): Risk-based test identification & risk-based test selection (FOKUS, SINTEF) Advanced Fuzz Testing (Banking, Radio Protocols, Automotive, Telecommunication): Model-based behavioural fuzzing (FOKUS) Model inference assisted smart fuzzing (INPG) Active Testing Techniques (Banking, Radio Protocols) Model-based security testing from behavioral models and test purposes (SMARTESTING) Active Intrusion Testing (FSCOM) Passive testing techniques (Radio Protocols, Industrial Automation): Events-based passive testing (monitoring) (MONTIMAGE) Anomaly detection with Machine Learning (INPG)

8 G&D Banking Case Study Case Study Characterization Banknote processing machine that counts, sorts and assess banknotes by their currency, denomination, condition and authenticity external peripherals external peripherals CP = Currency Processor RS = Reconciliation Station CC = Control Center VMS = Vault Management System CP CP CP CC WAN LAN CC Firewall CC / GW RS RS VMS

9 G&D Banking Case Study Case Study Characterization Security challenges Restricted access to functions: The access to functions is restricted to authorized users. Operation system access restriction: The access to the operation system, i.e. file system, or process monitor is restricted to authorized users. Prevent Admin Hijacking: Hijacking an administrator account is used to get the privileges of an administrator account as a user that is not assigned to the administrator group. Prevent infiltration/manipulation of software: Software manipulation can be used to fake data or to provoke errors on the currency processor application. Prevent manipulation of application configuration: Manipulation could possibly change the classification of banknotes.

10 G&D Banking Case Study Approach: Risk-based Security Testing CORAS Risk Analysis Deliverable D1.WP2* Behavioural Fuzzing Deliverable D2.WP2* (see also next slide), D3.WP2* Data Fuzzing with TTCN-3 Deliverable D3.WP3* Risk Analysis Security Test Test Code Test Pattern Generation (CORAS) Generation Identification (TTCN-3) Test Execution Pattern name Context Problem/Goal Solution Usage of Unusual Behavior Sequences Test pattern kind: Behavior Testing Approach(es): Prevention Security of information systems is ensured in many cases by a strict and clear definition of what constitutes valid behavior sequences from the security perspective on those systems. For example Test procedure template: Known uses Model-based behavioural fuzzing of sequence diagrams is an application of this pattern Security Test Pattern Catalogue Deliverable D3.WP4.T1* *project deliverables are available at publications

11 G&D Banking Case Study Approach Behavioural Fuzz Testing Test cases are generated by fuzzing one or more valid sequences. This concrete fuzzing of behaviour is realized by changing the order and appearance of messages in two ways: By rearranging messages directly. This enables straight-lined sequences to be fuzzed. Fuzzing operators are for example remove, move or repeat a message. By utilising control structures of UML 2.x sequence diagrams, such as combined fragments, guards, constraints and invariants. This allows more sophisticated behavioural fuzzing that avoids less efficient random fuzzing. By applying one ore more fuzzing operators to a valid sequence, invalid sequences (= behavioural fuzzing test cases) are generated. valid sequence Behavioural Fuzzing invalid sequence TC SUT Fuzzer Apache 1: logon("op1") 2: selectdenomination( ) Remove Message 1: logon 1: selectdenomination( )

12 G&D Banking Case Study Results Focus on risks related to unauthorized access machine/configuration modification Until now, no weaknesses were found confidence in the security of the system is strengthened Metrics different security levels depending on the covered risks/vulnerabilities by number of test cases (one or more) per risk/vulnerability unauthorized access, configuration modification: more number of test methods to generate these test cases data fuzzing and behavioural fuzzing: 2 test methods

13 G&D Banking Case Study Exploitation CORAS method for risk analysis has been proven to be of value graphical modelling specification of assets to be protected Saved resources due to reuse of functional test cases and reuse of test execution environment for non-functional security testing integration of data fuzzing in the TTCN-3 execution environment keeps the behavioural model clean and concise allows easy combination of data and behavioural fuzzing Standardization of DIAMONDS results provides certification options for products with security requirements

14 Automotive Case Study Case Study Characterization Bluetooth connectivity module for mobile devices that allows direct communication between car s head unit and a mobile phone Security challenges: Access to the car s infrastructure by malfunctioning or hostile mobile phones or by misuse of the Bluetooth interface Modification of the Bluetooth module in order to interfere with the car s normal operation and its security and safety Technical challenges: Simulation of Bluetooth device/mobile phone and integration of CAN bus specialized Bluetooth stack for security testing

15 Automotive Case Study Approach: Risk-based Security Testing Security Risk Analysis Functional test cases Fuzzing techniques System Model Test Model Security Test Case Templates

16 Automotive Case Study Approach: Data Fuzzing Fuzzing Library developed by Fraunhofer FOKUS Library is called by FuzzingContainer to inject fuzzed test data Improved fuzzing heuristics based on Peach and Sulley Interface uses XML for requests and generated fuzz test data Example: Device name and PIN was fuzzed within this case study Generators:

17 Automotive Case Study Results So far, about 150 test cases have been executed Test purposes break Bluetooth connectivity module compromise the head unit by anomalous Bluetooth messages Until now, a few anomalies were found need further investigation Metrics several vulnerabilities resulted from risk analysis were covered further metrics have to be found

18 Radio Protocol Case Study Case Study Characterization DEMONSTRATOR DESCRIPTION OMNeT++ simulation platform of mobile ad-hoc networks. Vulnerability analysis based on overthe-air exchanged PDU at mac and physical layers. Model-based generation of test cases (Smartesting and FSCOM) and their execution based on OMNeT++. Online analysis of captured traces in order to detect security flaws (Montimage) OBJECTIVES Security and risk analysis Formal security flaws identification Testing tools chain complementarity mitigation strategies (on-going)

19 Radio Protocol Case Study Security Testing Approach

20 Radio Protocol Case Study Results Integration of the tools in the TCS validation framework, use of standardized API to help on the integration on different validation environment and industrial domains. Validation of the framework with the validation of 19 security properties. Implementation of 7 intrusion attacks. Further work distributed detection of several attackers at routing layer genetic testing for static analysis of memory overflow.

21 Radio Protocol Case Study Exploitation DIAMONDS satisfies the requirements of higher security testing, in particular on Over The Air threats. Evolution of security testing from the critical components to the whole parts of the radio equipment (Hw platform, midleware and radio protocol application). DIAMONDS is a first response the security testing analysis of these applications for which tools and methodologies are lacking. Next step might be the integration of Intrusion Detection & Prevention System in the radio equipments.

22 Overall Exploitation Results in DIAMONDS From Case Studies to Industry Montimage, Codenomicon Product update that integrate features that have been developed in DIAMONDS itrust consulting Malwasm, an open-source tool allowing to monitor a executable during execution by stepping forward and back like in a video and observing and tracing changes to all kind of system parameter. Trick-tester, a Linux distribution allowing Pentesters to have all kind of open source tools corrected configured with tailored script, and perform efficient pentests. Testing Technologies TTworkbench will integrate the TTCN-3 Fuzzing Support Smartesting Security test purpose language and a test generation mechanism extend the current Smartesting product with a dedicated feature for model-based security test generation. Giesecke & Devrient Adoption of the Risk Analysis method CORAS for the product development life cycle.

23 DIAMONDS Standardisation Work Standardisation Bodies Standardisation levels: International: ISO, ITU, European: ETSI, ENISA, National: NIST, AFNOR, DIN, Industrial communities: IEEE, OMG, DIAMONDS focusses on ETSI: TC MTS: Methods for testing and specification, Model-based testing, Security Special Interest Group; TC TISPAN/E2NA: Threat, vulnerability and risk analysis (TVRA) TC INT: IMS network testing (concrete test case catalog) ISG ISI: Operational Security Indicators measuring IT security policy enforcement & effectiveness (in cooperation with national R2GS Clubs)

24 DIAMONDS Standardisation Work Standardisation Approach System (risk) analysis methods & models: e.g. CORAS, UMLsec Test tools & techniques: e.g. fuzzing, partitioning usecase testcase TOE 1, TOE 2, TOE n identifying SFRs (specification) TSFI (realisation) testing TCL, JUnit, C++, TTCN-3, manual tests System definition & analysis 1) TOE, subjects, assets, 2) threats, policies, assumptions 3) security objectives 4) Security functional requirements enforcing TSFI supporting SFRs non-interfering Test developer plan a) concepts/architecture b) purposes c) Test suite structure coverage of security relevant TSFI

25 Co-summit 2011 and 2012 ITEA Exhibition Award

26 Contact Fraunhofer Institute for Open Communication Systems FOKUS Innovation Center for Cost-Effective Systems Quality Kaiserin-Augusta-Allee Berlin, Germany Tel. +49 (30) Fax +49 (30) Prof. Dr.-Ing. Ina Schieferdecker Tel. +49 (30)