model centric Testing (.mzt)

Transcription

1 model centric Testing (.mzt) Challenges and Solutions in Test Design, Test Management and Test Execution Georg Götz (sepp.med gmbh) Dr. Armin Metzger (sepp.med gmbh)

2 Agenda TestNGMed project Traditional approach to quality assurance Challenges in the medical domain model centric Testing (.mzt) TestNGMed process and tool chain 2

3 sepp.med gmbh Medium-sized IT service provider More than 28 years experience Our domains Medical Pharmaceutical Automotive Expertise: complex and safety relevant systems 3

4 TestNGMed Project Test of Next Generation Medical IT ProInno / EUREKA-Project Σ! 4053 RETEMES Goal: Tool supported validation and verification for medical IT systems: Test workflows, based on clinical workflows, HL7 and IHE (medical standards) model centric Test-ing using UML profiles Test automation, based on TTCN-3 4

5 Testing traditional Approach Document based test bed Arbitrary test scenarios Redundancies but no guarantee of completeness Challenges Rising complexity Budgets Time to market Maintenance Test Cases TC_1 SC TF_2 Communicability? Taxonomy? Re-use and maintenance? Usage of test/domain standards? Test Suite Test Management Automation? Test automation always fails (in Med) 5

6 Integration und QS in der Medizintechnik Gap between manufacturer and clinical environment Growing Many different stakeholders Complexity in clinical environment Heterogenous systems and networks Different vendors and product families Configurations, individual solutions Standards (IHE / HL7) vs. reality Standard conformance and interoperability Processes Safety relevance Heavily regulated (FDA, MDD, ) Test Lab Tested Untested! Customer User workflows 6

7 Goals of model centric Test (.mzt*) Better collaboration and unified documentation Usage oriented tests Metrics Increased efficiency in test execution Process and tool chain for: Modeling of requirements and workflows Automated generation from test models Automated test execution TestNGMed: Using the standards HL7/IHE and TTCN-3/UML *: from German modellzentrierter Test, English: model centric test 7

8 model centric Testing Intuitive and exact models Goals Collaboration over domain borders Visualization of workflows and context Maintainable and adaptable test designs Metrics from designs Quantifiable coverage metrics Realized in.mzt Higher product quality More effective testing 8

9 Test Design & Generation Architecture Test Mgmt Information IHE Scenarios TC_1 SC TF_2 Test UC Design Tester s Mindset & Adaptions.getmore systematic generation Test Design TTCN-3 Test Suite Test Management 9

10 Test model with Management and Tracing Information Priorities Visualize relevant information in test designs Controlled generation of test cases (coverage, time, ) Requirements Prio 1 Req._AB Prio 2 Implementation independent Test management in the test design Safety Hazard Data Tracing of requirements to test cases One repository for all test related data 10

11 Features of.mzt Tester Hierarchies and abstraction Stakeholders Developer Visualization and communication Use Cases Workflow details techn. Details Model repository Reuse and adaptability 11

12 IHE Workflow Definitions as Test Basis Basis: IHE Workflows Deficits Actors Test systems Systems under Test Persons / ext. actions Messages HL7 DICOM Generic Linear! Good Case! No Tester s Mindset! Variations (real world) need specific workflows! Only a sample scenario 12

13 Test Workflow Modeling IHE workflow description Registration Test model with all possible workflows Order Change data Combination Adaption Tester s Mindset 13

14 TTCN-3 Code Generation Model Test parameters Template definitions Test case generation Abstract test cases TTCN-3 export TTCN-3 compiling TTCN-3 test cases Test campaign Executable test suite 14

15 TTCN-3 Code Generation Mapped to a test component or system Template reference Multiple test components Multiple SuT Everything talks to everything Different communication protocols concurrently Synchronization must be possible Data must be manipulated on the fly (timestamps ) Asynchronous behavior 15

16 .mzt: Process and Tool Chain Business/User workflows Tester s Mindset model centric Te esting process & tool chain Use Cases Functional Require- ments Change Requests based on Tracing based on Visual test models & designs Autom. generatio ion using.getmore e Manual test cases Automation code Test Management Test Execution Regulatory conformance Efficient test phase Better product quality Priorities, Savety,... 16

17 TestNGMed: model centric Workflow Usage oriented hierarchical test models Med. standards (IHE) Efficient maintenance and adaptability Economical Communicable Generation TM aspects Strategies Coverage Test case generation using.getmore Modeling Automated generation SuT Test Suite Clinical systems: KIS, RIS, PACS, modalities Test procedures Test of conformance and interoperability System validation and verification TTCN-3 Med. protocol standards HL7/DICOM Test automation Test execution using TTworkbench 17

18 Thank you for your attention. Any questions?