APPROACHES TO AUTOMATED TEST IMPLEMENTATION IN MODEL DRIVEN TEST AUTOMATION ARCHITECTURES

Transcription

1 Sophia Antipolis, French Riviera October 2015 APPROACHES O AUOMAED ES IMPLEMENAION IN MODEL DRIVEN ES AUOMAION ARCHIECURES Presented by Marc Florian Wendland

2 C C C SU SU SU What is test implementation? Definition ISQB he process of developing and prioritizing test procedures, creating test data and, optionally, reparing test harnesses and writing automated test scripts. Logical est Cases echnical/executable est Cases est Implementation Activities 2

3 C C C C C C SU SU SU SU SU SU he ISQB fundemantal test process Management Automated test design (aka model-based testing) Knowledge est model est generator est generator est Analysis Control est Design est Implementation est Execution est Evaluation est Closing Activities Automated test execution is state of the practice (if ever) in industry est Scripts Model transformation est execution tool Automated test execution (KD, DD, CN-3) SU System under test 3

4 Using abtraction levels for automation Question How to increase the degree of automation from automated test execution to automated test design by remaining being immediately executable? (Unsatisfying) Answer Depends on the project circumstances Idea Leverage the notion of abstraction levels 4

5 Optimal abstraction levels in test automation KD, DD, CN 3 CN 3, XML, Excel UML esting Profile, DL est modeling and generation tool est Execution System est model (ABS. LEVEL 3) est analysis activities echnical test cases est design activities Executable test cases requires Actual implementation (ABS. LEVEL 0) Logical actions the system can perform Logical test cases (ABS. LEVEL 3) est implementation activities est execution activities Sequence diagram, est Descriptions Implementation, RI, SA, CoDEC 5

6 Approaches to automated test implementation (High Level) test model (ABS. LEVEL 3) N numbers of intermediate models (ABS. LEVEL < 3 AND > 2) est model (ABS. LEVEL 3) est model (Low Level) test model echnical test cases echnical test cases echnical test cases op-down approach (i.e., model-is-master) Bottom-up approach (i.e., is-master) Meet-in-the-middle approach (i.e., and Model are equal) 6

7 op down approach Model is master est design not imposed by adaptation layer or test execution system No constraint on the test execution system Early testing Often used in academic prototypes echnical test cases est model (ABS. LEVEL 3) Recommendation Feasible for proof of concepts, limited use for industry projects 7

8 Bottom up approach is master Ensures immediate automated test exection Requires available adaption layer est model derived from the adaption layer Poorly abstracted adaptation layer propagtes ist poorness est model echnical test cases Recommendation Applicable if abstraction of adaptation layer is appropriate for domain experts 8

9 Meet in the middle approach Combines advantages of top down and bottom approach Fosters early testing and ensures being immediately executable Any number of intermediate models possible Highest engineering complexity, but most flexible (High Level) test model (ABS. LEVEL 3) N numbers of intermediate models (ABS. LEVEL < 3 AND > 2) (Low Level) test model echnical test cases Recommendation For long running projects with frequently changing requirements 9

10 MIDAS Example: Bottom up approach Platform mapping (vertical) Platform mapping Platform (horizontal) mapping (vertical) Usage-based Fuzzing import UP Platform mapping WSDL/XSD 2 UML (horizontal) Import Fuzzing Resolution Standard Platform CN 3 mapping (horizontal) est Gen. UML (PIM) Layer wb Adoption UML Resolution Characteristics Characteristics Characteristics 2 layered import of WSDL/XSD Characteristics UP Platform Characteristics mapping (vertical) Resolution Mapping of Fuzzing from strategies UP to CN 3 (MIDAS Resolved UML (PIM) High level: only UML est (+ Model UP) Initial resolution of elements to be CN 3 Adoption code generation to wb handling of WSDL and test model) Mapping to CN 3 fromfuzzinglib XSD to CN 3 MIDAS est Model XSD Low level: WSDL/XSD details invocations (according deployed to XSD 2 CN 3 UML representation of WSDL service definitions Resolution WSDL/XSD utilizing ofconjugated est SoaML Definition Ports Characteristics Deployment of WSDL definitions Adoption standard) of test control part (Activity) to UML (PSM) Layer UML representation of XSD type system Completion used in WSDL of Generation Adoption according of CN 3 tomodules proprietary XSD incorporate Mapping fuzzed from test data SoaML asparameters XSD toprofile CN 3 Rigorous tracing among libraries using Deployment InstanceSpecification MIDAS MDA oftrace CN 3 framework (templates) according mapping interpretation to deploymentof spec. wb Refinement of deployment according modules to CN-3 WSDL/XSD UML (PSM) Import of already deployed the predefined FuzzingLib library wb Import Involved Standard lanugaes CN-3 Fuzzing CN-3 wb Involved CN-3 Involved profiles modules profiles (built in libs) WSDL/XSD CN 3 PSM UML Involved profiles UP CN-3 import CN-3 exhibits UP CN-3 Fuzzing Involved profiles (view) Strategies MIDAS Extension (part of MIDAS UP Code est Execution CN 3 PSM SoaML Extension) XSDp Layer CN-3 XSD profile CN 3 PSM SoaML CN-3 Execution WSDL /XSD CoDec CN-3 Executable RI CN 3 PSM SOA System UML Libraries Imports ransformation/ Generation race Link 3rd Party xxx 3rd Party Component 10

11 Conclusion hree different approaches have been shown Based on an (informal) notion of abstraction levels Model transformations in between represent the magical red button (Initial) engineering effort can be quite high Bottom up approach was realized in MIDAS Meet in the middle approach reported by SIEMENS in the ICX project Future work will focus automated generation of test models from test suites 11

12 Contact Fraunhofer FOKUS Kaiserin Augusta Allee Berlin, Germany Marc Florian Wendland Senior Researcher, SQC marc Phone +49 (0)