Experiences with Action Based Testing a Case Study. Flavio Kubagawa, QAD Inc Hans Buwalda, LogiGear Corporation

Size: px

Start display at page:

Download "Experiences with Action Based Testing a Case Study. Flavio Kubagawa, QAD Inc Hans Buwalda, LogiGear Corporation"

Monica Rogers
6 years ago
Views:

1 Experiences with Action Based Testing a Case Study Flavio Kubagawa, QAD Inc Hans Buwalda, LogiGear Corporation

3 Action Based Testing (ABT) Test Planning and Control Global Test Design Breakdown Analysis Clustering "Test Clusters" Navigation Scheme Physical Navigation Report End users Management QA/Auditors System Development Input data Expected outcomes Documentation SEPARATION System(s) Under Test Actual results Comparison with expectations Management information (c) 2002, QAD, LogiGear, all rights reserved 3

4 Test Lines cluster EXAMPLE OF A TEST CLUSTER version 1.0 author Hans Buwalda section 1. Entering clients and balances last name first name account nr balance enter client Green John enter client Wood Anna section 2. Money Transfers from to sum transfer transfer section 3. Checking names and numbers account nr last name first name check name Green John check name Wood Anna account nr sum check balance check balance (c) 2002, QAD, LogiGear, all rights reserved 4

5 Test Objectives... TO-3.51 The exit date must be after the entry date... test objective TO-3.51 name entry date exit date enter employment Bill Goodfellow check error message The exit date must be after the entry date. (c) 2002, QAD, LogiGear, all rights reserved 5

7 Navigation Scheme CONTROL Reports High-level High-level High-level Intermediate Level(s) Low Level Action Layer Intermediate Level(s) Templates Tables Scripts Application (c) 2002, QAD, LogiGear, all rights reserved 7

8 About QAD Producer of advanced enterprise software for manufacturers "collaborative commerce" concept, focusing on integration between the diverse business processes Manufacturers of automotive, food and beverage, consumer, electronics, industrial and medical products use QAD applications 5,200 licensed sites in more than 80 countries and in as many as 26 languages for more information, please see: (c) 2002, QAD, LogiGear, all rights reserved 8

9 About MFG/PRO Enterprise software Menu with over 2,700 screens ("programs") For the test automation project, used version with a textual interface using a terminal emulator there is also a newer, web based, version, but the text version had priority for the test automation (c) 2002, QAD, LogiGear, all rights reserved 9

10 Project Objectives Create an automated regression test that supports: Integration testing against MFG/PRO baseline for: "Projects staged" "Maintenance fixes staged" Service pack testing Possible localization testing Absorbs project changes Represents long term payoff for QAD (c) 2002, QAD, LogiGear, all rights reserved 10

11 Criteria for Automation Priorities First priorities: Modules most impacted by new projects Legacy maintenance against baseline Prioritize next modules with highest defect-density Based on previous release Remain responsive Be flexible enough to reorder priorities as needed (c) 2002, QAD, LogiGear, all rights reserved 11

12 Action Based Testing The navigation scheme is basically an interpreter Core is the "ABT Engine", with a library of functions that is loaded from a testing tool Reads the test lines from the Excel file Calls functions created by the user to execute the actions and checks Processes the results into easy to read reports Test step >> action line Test script >> set of action lines >> cluster MFG/PRO interface >> interface layer (c) 2002, QAD, LogiGear, all rights reserved 12

13 Test Automation: Development Process Test case descriptions are created and streamlined from system test scripts by a test specialist (Robin Kronk) 700 pages = 1,600 test steps On basis of descriptions ABT test clusters are made and the actions are implemented with an ABT navigation scheme plus generic Winrunner functions by two test engineers (Flavio Kubagawa and Edward Duran) 600 Action Words, 65 scripts in Winrunner Tools used to support test automation: Winrunner ABT Engine MS Excel (c) 2002, QAD, LogiGear, all rights reserved 13

15 Test Automation: Challenges MFG/PRO ChUI (Character User Interface) interface Created WinRunner functions to work with ChUI Maximize reusability Developed methodology to create generic action words to increase reusability Minimize need for additional scripts Created generic functions to decrease need of coding (c) 2002, QAD, LogiGear, all rights reserved 15

16 Test Automation: Generic Functions Layer Implements the middle layers of the ABT Navigation Scheme (without the use of templates) Virtually eliminates the need to create new WinRunner scripts/functions to implement a new action word Speeds up the process of automating test scripts. Eliminates need for large number of automation engineers (c) 2002, QAD, LogiGear, all rights reserved 16

23 cluster Manufacturing sheet scenarios version 1.0 author date Jun/ section Backlog Flavio Y. Test KubagawaCluster Enter actuals that are lower than the plan Backlog Plan Maintenance Site Product Line Year create backlog plan D>Year+1 January February March April >>> May June July August >>> September October November December >>> (c) 2002, QAD, LogiGear, all rights reserved 23

25 Example Interface Definition create interface mfg_awinfo AW prg menu title frames interface entity create actual pay rate rwemmt Actual Pay Rate Maintenan 01,02,03 interface entity browse approval pcbr Approval Browse 01,Filter,02 interface entity create backlog plan plppmt Backlog Plan Maintenance 01,02 interface entity inquire backlog plan plppiq Backlog Plan Inquiry 01,PRT,SB interface entity create bom registration wlbmmt BOM Registration Maintena01,02,03,0 interface entity build calendar cross-reference spglclmt Calendar Cross-Reference B01 interface entity create commodity code ppcommt Commodity Code Maintena 01,02,END interface entity inquire co/by-product work ordwojpiq Co/By-Product Work Order 01,PRT,SB interface entity initialize compliance clclpm Compliance Control 01,02 interface entity copy plan to simulation spfpcp Copy Plan to Simulation 01,PRT,SB interface entity copy simulation to plan spfpcp Copy Simulation to Plan 01,PRT,SB interface entity cost roll-up freeze/unfreeze bmfrzmt Cost Roll-Up Freeze/Unfree01,02,03,P save interface mfg_awinfo c:\actionwords\data\mfg_awinfo.dat (c) 2002, QAD, LogiGear, all rights reserved 25

27 create interface plppmt05 interface entity interface entity setting title fields_plppmt05 Interface Layer Backlog Plan Maintenance name class nbr init endkey condition value interface element frame01 frame 01 3,9 name class tab row col len frame interface element Site edit interface element Product Line edit interface element Year edit > name class nbr init endkey condition value interface element frame02 frame 02 7,17-23 name class tab row col len frame interface element Begin edit interface element January edit interface element February edit interface element March edit interface element April edit interface element May edit interface element June edit interface element July edit interface element August edit interface element September edit interface element October edit interface element November edit interface element December edit > save interface plppmt05 C:\ActionWords\data\fields_plppmt05.dat (c) 2002, QAD, LogiGear, all rights reserved 27

29 Very Good Reusability Interface layer and action words: one size fits all Once created, can be reused as often as needed. Clusters: as reusable as the related manual test. New clusters can be created reusing existing action words Different business cycles. (c) 2002, QAD, LogiGear, all rights reserved 29

30 Reusability, Some Numbers Top 5% most used Action Words Suite 1 = 27% of Total Steps: 7 Action Words = 86 Steps Suite 2 = 27% of Total Steps: 10 Action Words = 113 Steps Top 10% most used Action Words Suite 1 = 39% of Total Steps: 14 Action Words = 123 Steps Suite 2 = 39% of Total Steps: 19 Action Words = 168 Steps 50% of the Test Suites were completed with: Suite 1: 23 Action Words (17%) = 159 Steps (50%) Suite 2: 30 Action Words (16%) = 214 Steps (50%) (c) 2002, QAD, LogiGear, all rights reserved 30

31 Reusability Example Suite 1 Incorporating Project Enhancements added 2 new sections to the manual tests 22 pages 44 steps 12 new Action Words Reused 73% of existing Action Words Quickly implemented new Action Words using the layered structure (c) 2002, QAD, LogiGear, all rights reserved 31

32 Overall Experiences ABT process is a major improvement maintainability re-usability manageability (clear tests and test results) LogiGear's support is very effective training and coaching quick and appropriate answers to all questions QAD expects to extend the use of ABT to other systems and other tests (c) 2002, QAD, LogiGear, all rights reserved 32

Example. Section: PS 709 Examples of Calculations of Reduced Hours of Work Last Revised: February 2017 Last Reviewed: February 2017 Next Review:

Example. Section: PS 709 Examples of Calculations of Reduced Hours of Work Last Revised: February 2017 Last Reviewed: February 2017 Next Review: Following are three examples of calculations for MCP employees (undefined hours of work) and three examples for MCP office employees. Examples use the data from the table below. For your calculations use