Security Testing Improvement Profile (STIP) An evaluation scheme for security testing

Transcription

1 Security Testing Improvement Profile (STIP) An evaluation scheme for security testing SASSI13 Security Assessment for Systems, Services and Infrastructures September 2013 at the Technical University (TU) in Berlin Jürgen Großmann, Fraunhofer FOKUS,

2 Motivation Technical Guide to Information Security Testing and Assessment NIST Special Publication

3 TMMi, TPI and TPI NEXT \ TPI, TPI Next are registered trademarks of Sogeti TMMi is based on CMM, and developed by the Illinois Institute of Technology The TMMi Model from

4 TPI and TPI NEXT Analysis with respect of the key areas Levels are used to assign a degree of maturity to each key area Checkpoints are defined to determine the level for each key area Each higher level is better than its prior level in terms of time (faster), money (cheaper) and/or quality (better). Maturity Scale Staged representation: Initial Controlled Efficient Optimizing Continuous representation A M or 1-13 Checkpoints Key Areas Maturity Levels Improvement Suggestions

5 Security Testing Improvement Profile (STIP) Evaluation of the DIAMONDS Case Studies Security Testing Improvement Profiles (STIP) enables an objective, detailed analysis and evaluation of security testing processes First introduced to evaluate the case studies of the DIAMONDS project Provide a detailed analysis and evaluation of our research & development Show how tools & techniques have evolved Provide a template for other on how to pragmatically integrate the DIAMONDS results to improve security testing processes on hand. Analysis with respect of the key areas Levels are used to assign a degree of progress to each key area Each higher level is considered better than its prior level in terms of quality (e.g. exactness of the outcome) or effectiveness (e.g. automation of activities).

6

7 STIP key areas Inception and target analysis Elaboration and execution Test Techniques Artifact consistency and tool support Information gathering Security risk assessment technique Test depth Generation of security test models Security functional testing Fuzzing Security test tool integration Security risk assessment scope Security test identification Security test generation Security test execution automation Security passive testing/security monitoring Static security testing Traceability & test coverage

8 STIP level definition Key area: Risk Assessment Technique A: Informal security risk assessment At this level, the security risk assessment is conducted in an unstructured manner without a specific notation/language for document risk assessment results or a clearly defined process for conducting the security risk assessment. B: Model-based security risk assessment At this level, the security risk assessment is conducted with a language for documenting assessment results and a clearly defined process for conducting the assessment. C: Model and testbased security risk assessment At this level, the model-based security risk assessment uses testing for verifying the correctness of the risk assessment results.

9 STIP level definitions Key area: Security Test Identification A: Identification based on requirements analysis B: Identification based on threat/vulnerability models C: Identification based on threat/vulnerability models and test pattern D: Risk-based security test identification + prioritization Test identification can be based on the analysis of the functional security requirements (SFR) and their coverage through testing. Often these requirements have priority numbers that additionally provide guidance on the importance of a requirement and the related test purpose. Security threat/vulnerability models additionally allow for the identification of penetration tests that are based on estimations on potential threats and potential vulnerabilities. This allows testing for unwanted incidents that are not covered by the security functional requirements. The combination of threat/vulnerability models and test pattern additionally provides best practices for the identification and selection of testing means dedicated to well-known classes of threats or vulnerabilities. This approach provides extensive guidance to identify adequate test purposes and to apply approved security testing methods, techniques and tools. Risk-based security test identification and prioritization combines the advantages of Level 3 with a prioritization of the test purposes by considering probabilities of the unwanted incident and estimations on their consequences (quantified security risks). The integration of test identification with security risk assessment allows for a problem and business specific prioritization of the identified tests purposes and testing approaches.

10 Analysis and improvement suggestions A security testing matrix defines the current state of a process (blue background). Profiles define optimal and well aligned security testing levels (red line). Improvements suggestions are to be defined on basis of dependencies between key areas and their levels (red background) e.g. Security test identification B requires Security risk assessment technique B (green arrow)

11 Application of STIP Evaluation of the DIAMONDS case studies Security testing solutions for six industrial domains in 8 case studies Banking Automotive Radio protocols Smart cards Telecommunication Industrial automation

12

13 Evaluation of the DIAMONDS Case Studies STIP results for the international case studies

14 Evaluation of the DIAMONDS Case Studies Progress in all case studies

15 Banknote processing machine case study

16 Summary & Conclusion STIP is an evaluation and improvement scheme for security testing processes First introduced to evaluate the case studies of the DIAMONDS project Provide a detailed analysis and evaluation of security testing processes on hand Provide a template to pragmatically improve security testing processes on hand First version is available at Can be used in addition to TMMi or TPI to emphasize security testing aspects. FOKUS plans to offer consultancy and certification optinos on basis of STIP in the near future Contact: Jürgen Großmann Fraunhofer Institute for Open Communication Systems FOKUS MOTION Modeling and Testing for System and Service Solutions Kaiserin-Augusta-Allee 31, Berlin, Germany juergen.grossmann@fokus.fraunhofer.de