Integration of Formal Methods and Testing for Model-Based Systems Engineering NII Shonan Meeting, Japan, Nov. 30 Dec. 4, 2014

Transcription

1 Alexandre Petrenko Lead Researcher Computer Research Institute of Montreal CRIM, Canada Curiosity driven and industrial research: FranceTelecom, Siemens, Bombardier, SAP, CAE, Ericsson, GM Integration of Formal Methods and Testing for Model-Based Systems Engineering NII Shonan Meeting, Japan, Nov. 30 Dec. 4, 2014

2 Requirement-based Modelling and Verification A requirement that cannot be tested is not a requirement (a company in automotive industry) Requirements are used to define test purposes and/or scenarios (sequence diagrams) for test generation Consistency of the obtained scenarios needs to be verified before tests are generated Generating Asynchronous Test Cases from Test Purposes, Information and Software Technology, 2011 From Scenarios to Test Implementations via Promela, ICTSS 2010 Implementing MSC Tests with Quiescence Observation, TESTCOM

3 Fault-based Modelling and Testing Fault model is <Spec, Set-of-Imp, Conf-Rel> where Spec and Imp are FSM, EFSM, or IOTS; Set-of-Imp is a subset of the universe of them, explicitly or implicitly specified Generating Complete and Finite Test Suite for ioco: Is It Possible?, MBT 2014 Fault Coverage-Driven Incremental Test Generation, The Computer Journal, 2010 Testing from Partial Deterministic FSM Specifications, IEEE TC, 2005 Confirming Configurations in EFSM Testing, IEEE TSE, 2004 Fault Model-Driven Test Derivation from Finite State Models: Annotated Bibliography, MOVEP, 2000 Test Generation Driven by User-defined Fault Models, IWTCS'99 Test Suite Generation for a FSM with a Given Type of Implementation Errors, PSTV,

4 Compositional Approaches to Testing (I) Compositional testing is to test components in a system containing COTS which include queues The approach by composing components tested in isolation fails for communications via queues Integration Testing of Communicating Systems with Unknown Components, Annals of Telecom., Springer, 2014 Transition Covering Tests for Systems with Queues, Software Testing, Verification and Reliability, 2009 Test Generation for CEFSM Combining Specification and Fault Coverage, TestCom, 2002 Testing Strategies for Communicating FSMs, IWPTS,

5 Compositional Approaches to Testing (II) Given a global model Spec of a system composed of component(s) to test and the rest of the system, Context, the model of component(s) to test is a largest solution Context Spec to the equation Context X = Spec The solution is nondeterministic, modelling component(s) to test; even if they are perfectly deterministic The Unknown Component Problem: Theory and Applications, (Villa, T., Yevtushenko, N., Brayton, R. K., Mishchenko, A., Petrenko, A., Sangiovanni- Vincentelli A. L.), Springer, 2012 Testing in Context and Synthesis of the Unknown Component: Two Faces of the Same Coin, IEEE/ACM IWLS, 2009 Solving Asynchronous Equations, FORTE, 1998 Testing in Context: Framework and Test Derivation, Computer Communications,

6 Learning Approaches to Testing Learning without the Teacher answering your queries YES/NO is inference, isn't? Test suite is complete for a given fault model iff only Spec can be inferred from the test suite Inferring Approximated Models for Systems Engineering, IEEE HASE, 2014 Checking Completeness of Tests for Finite State Machines, IEEE TC, 2010 Inferring Behavioral Models from Traces of Business Applications, IEEE Web Services, 2009 Modular System Verification by Inference, Testing and Reachability Analysis, TESTCOM 2008 On Fault Coverage of Tests for Finite State Specifications, Computer Networks and ISDN Systems,

7 Adding to Technology Vectors Interactive tools for test development Combining model and code-based testing 7