Framework for Constructing Context- Specific Migration Methods for Test Cases

Transcription

1 for Constructing Context- Specific s for Cases Presented by Ivan Jovanovikj All rights reserved

2 Agenda 1 Challenges in Case Reuse in Projects 2 Construction of Context- Specific s 3 Outlook and Conclusion 2 All rights reserved

3 1. CHALLENGES IN TEST CASE REUSE IN MIGRATION PROJECTS All rights reserved

4 Problem Statement Problem Statement How to systematically reuse existing test cases in a software migration scenario? Reverse Engineering Restructuring Software Changes from the software migration must be considered Forward Engineering Requirements must be considered Cases test System System test Cases the desired functionality of the source system Case Design is costly and time consuming 4 All rights reserved

5 Solution Approach Solution Approach Development of a Context-Specific Case In this talk: for Constructing Context-Specific s for Cases Software Cases test System System test Cases 5 All rights reserved

6 2. Construction of Context-Specific s All rights reserved

7 Approach Overview Pre- Post- Context Adapted Method Context Method Method Reference Method Contextspecific s 7 All rights reserved

8 Context System Context 1. What was migrated? 2. How was it migrated? ing Context 3. What is the current state of the test cases? 4. What are the testing requirements? Context 8 All rights reserved

9 Context Factors Context Factors System Context Factors ing Context Factors Source Environment Characteristics Target Environment Characteristics Characteristics Source ing Environment Characteristics Target ing Environment Characteristics Language Language Degree of Automation ing ing Degree of Formality Case Type Case Type Architecture Architecture Process Support Case Anatomy Case Anatomy Change Type Case Quality Case Quality Context 9 All rights reserved

10 System Context - Case Study Eclipse ing Project & Eclipse ling Development s ling EMF Ecore EMF.Edit OCL BPMN2 ecore OCL EMF.Gen Just-in-Time Compilation (JIT) MoDisco Ahead-of-Time Compilation (AOT) Context 10 All rights reserved

11 ing Context - Case Study Source ing Environment Characteristics Cases 2 of Cases Target ing Environment Characteristics Cases OCL 1 of OCL OCL ing (JUnit) ing (NUnit) Language (Java) Language (C#) Context 11 All rights reserved

12 Reference Metamodel Language Metamodel instance of instance of UTP Meta- (Source) of the Abstraction Case Understanding Discovery instance of Platform Independent Language Restructuring () of the Generation Activities Text to Text Text to / to Text to Endogeneous Tranformation Exogeneous Tranformation Artifacts Textual Artifact Meta Context 12 All rights reserved

13 Method Reductive Remove Activity Remove Artifact (Source) of the Abstraction Case Understanding Discovery Platform Independent Language Restructuring () of the Generation Context 13 All rights reserved

14 Method (Source) Abstraction Platform Independent Restructuring () of the Case Understanding Language of the Discovery Generation Context C1 specific Context C2 specific Context C3 specific Context C4 specific Context C5 specific Context C6 specific Context C7 specific Context C8 specific Context C9 specific Context C10 specific Context Cn specific Context 14 All rights reserved

15 Method Case Reductive Remove Activity Remove Artifact Abstraction (Source) Platform Independent Restructuring () of the Case Understanding Language of the Discovery Generation Context 15 All rights reserved

16 Method : Case Reductive Remove Activity Remove Artifact (Source) of the Case Understanding Discovery () + Generation Context 16 All rights reserved

17 Method : OCL Reductive Remove Activity Remove Artifact Abstraction (Source) Platform Independent Restructuring () of the Case Understanding Language of the Discovery Generation OCL OCL Context 17 All rights reserved

18 Method : OCL Reductive Remove Activity Remove Artifact OCL OCL Context 18 All rights reserved

19 Method 2 Case (Source) () of the Case Understanding Discovery + Generation 1 OCL OCL OCL Context 19 All rights reserved

20 Case2 2 TesCase JDT Core (Source) () Case Understanding of the Discovery + Generation supports Context 20 All rights reserved

21 2 Case Method Specification (Source) () s Case Understanding of the Discovery + Generation Case2 2 TesCase 1 OCL OCL OCL Context 21 All rights reserved

22 Case2 2 TesCase JDT Core (Source) () of the Case Understanding Discovery + Generation Context 22 All rights reserved

23 Case2 2 TesCase JDT Core xunit model AST Case Understanding Discovery + Generation xunit model Context 23 All rights reserved

24 Case2 2 TesCase JDT Core xunit model AST Case Understanding Discovery + Generation xunit model Context 24 All rights reserved

25 Case2 2 TesCase JDT Core AST xunit model Case Understanding Discovery + Generation xunit model Context 25 All rights reserved

26 Case2 2 TesCase JDT Core xunit model xunit model AST Case Understanding Discovery + Generation Context 26 All rights reserved

27 Case2 2 TesCase JDT Core xunit model AST Case Understanding Discovery + Generation xunit model Context 27 All rights reserved

28 Case2 2 TesCase JDT Core xunit model AST Case Understanding Discovery + Generation xunit model Context 28 All rights reserved

29 How to validate a test case migration? What is a valid test case migration? case migration is a process of transferring test cases into new environments without changing their functionality, i.e., without changing what they test. How to ensure behavioral equivalence in test case migration? Context 29 All rights reserved

30 Cases Cases Execution Report Passed Cases (negatives) Failed Cases (positives) True negatives False negatives True positives False positives Problem: Hides potential errors Consequence: System malfunctioning/crashes Problem: Detects non-existing errors Consequence: Waste of time by looking for non-existing error system works as expected Errors in the migrated system Context 30 All rights reserved

31 How to ensure behavioral equivalence in test case migration? How to avoid/detect false positives and false negatives? Constructive approaches Analytical Approaches Bottom Line Bi-simulation Mutation ing Cost of Case Cost of of Case + <= Cost of Developing New Cases Context 31 All rights reserved

32 OUTLOOK AND CONCLUSION All rights reserved

33 Overview and Conclusion Domain Problem Statement How to systematically reuse existing test cases in a software migration scenario? Cases test System Case Software System Changes from the software migration must be considered Requirements must be considered test Cases Conceptual solution Pre- Post- Context Characterisation Context Method Adapted Method Method the desired functionality of the source system Case Design is costly and time consuming Reference Method Contextspecific s System Context Method Specification Eclipse ing Project Eclipse ling EMF Ecore EMF. Gen EMF.Edit Just-in-Time Compilation (JIT) Development s OCL MoDisco BPMN2 ling ecore OCL & Ahead-of-Time Compilation (AOT) (Source) of the Case Understanding Discovery + Generation () ing Context Source ing Environment Characteristics Cases OCL ing (JUnit) Language (Java) 2 1 Target ing Environment Characteristics Cases OCL ing (NUnit) Language (C#) s Case2 2 TesCase Comprehensive discussion on the practical application 33 All rights reserved

34 Thank you for your attention s-lab Software Quality Lab Paderborn University Zukunftsmeile Paderborn Ivan Jovanovikj Tel.: (05251) ivan.jovanovikj@s-lab.uni-paderborn.de 34 All rights reserved