Topaz for Total Test User Guide

Transcription

1 Topaz for Total Test User Guide

2

3 Table of Contents Welcome to Topaz for Total Test... 1 Introduction... 2 Performance... 2 Intended Audience... 3 How This Guide is Organized... 3 Product Support... 3 Overview - Topaz for Total Test - COBOL Batch Programs... 4 Introducing I/O Call Stubbing... 4 Benefits... 5 Benefits - Topaz for Total Test - COBOL Batch Programs... 6 Unique Differentiation... 6 Audience and Savings... 6 Open a Topaz for Total Test Project... 7 Topaz for Total Test Projects Upgrade Window... 8 Related Topics... 8 Topaz for Total Test Sample Training Programs... 9 CWKTCOBX Test Program...10 CWKTDB2X Test Program...10 CWKTVSKS Test Program...11 Process Flow - Topaz for Total Test - COBOL Batch Programs...12 Summary Unit Test Creation Scenarios...18 Basic Background Knowledge...20 Topaz for Total Test Integration with Xpediter/Eclipse...20 Interface Testing...20 Test Scenario...21 Test Case...22 Test Suite...23 Test Run...23 Creating And Preparing A Test Project...24 Four Steps For Testing...26 I

4 Table of Contents Step 1: Import Structures...26 Step 2: Define Interface...27 Step 3: Create Test Scenario...28 Step 4: Execute Test...28 How To Reflect Interface Changes...29 Topaz for Total Test Reference...31 Related Topics...31 Context Menus from Xpediter Debug Session...32 Generate Unit Test Generation of Unit Test Data...35 Considerations When Setting Limits for Stub Data Collection...35 Generate Program Stub Related Topics...36 Perspectives and Views...37 Open Perspective...37 Toolbar Icons...38 Project Explorer View...39 Outline View...39 Add Check Conditions View...39 Host Explorer View...39 Progress View...39 General Editor Behavior...40 Preferences...40 Project...43 Creating a New Project...43 Importing an Existing Project...43 Project Structure...43 Project Properties...45 Structures...50 Importing Structures...50 Update Structures...55 Viewing a Structure...55 II

5 Topaz for Total Test User Guide Interfaces...57 Creating a New Interface...57 Editing an Interface...58 Test Scenarios...62 Test Suites...69 Creating A Test Suite...69 Editing Test Suites...69 Running Test Suites...69 Stubs...71 Stub Creation During Xpediter Debug Session...71 Creating A New Stub...71 Editing Stubs...72 Exporting Stub To Excel...76 Importing Stub From Excel...79 Run Test Dialog...81 Host Connection...82 JCL template...83 Execution Options...83 Test Aids...83 Enable Code Coverage...84 OK...85 CANCEL...85 Command Line Interface (CLI)...87 Prerequisites...87 Usage...87 Configuring Topaz for Total Test Jenkins Plugin Prerequisites Import Your Topaz for Total Test Project Into ISPW When you are finished with your Testing Automated Testing Workflow Message Codes Overview III

6 Table of Contents Current Topaz for Total Test Limitations COBOL Limitations VSAM Limitations DB2 Limitations IMS Limitations Miscellaneous Limitations Glossary Overview Index IV

7 Welcome to Topaz for Total Test Topaz for Total Test is a product that can automate unit testing. It is designed to speed testing development and provide higher confidence during program deployment process. Topaz for Total Test provides these features: Collection of unit test data for selected unit test cases by using the new Generate Unit Test feature in Xpediter. Easy identification and setting of where in a COBOL program a unit test case should be created. Collected Xpediter information is used to generate the unit test case and format the data collected so it can be used by the test cases. A method to stub out parts of the program, by using the new Generate Program Stub feature in Xpediter, allow tests runs to be almost independent of the original program and realize less system resource usage during testing. Test case execution can be automated using a Command Line Interface (CLI) so the unit test cases can be run as a regression test suite. By running these test cases, the user can verify that previously resolved issues stay fixed. Can be run as part of the normal product build and deploy process using the CLI. CLI can be executed as part of a continuous integration and build process. You may view this introductory video on YouTube or look for the Automated Unit Tests on the Mainframe with Topaz for Total Test video and other related Topaz for Total Test videos on the Compuware channel. 1

8 Introduction When you use this document, it is presumed that Topaz for Total Test has already been properly installed, and, if necessary, customized for your specific environment. For details please refer to the separate document Topaz Workbench Installation Guide. Topaz for Total Test mainframe test runner modules included in the Enterprise Common Components (ECC) product have already been properly installed with current maintenance applied, and, if necessary, customized for your specific environment. For details please refer to the separate document Enterprise Common Components Installation and Configuration Guide for Release To use the features of Topaz for Total Test: Compuware Host Explorer, Xpediter, and Xpediter/Eclipse features must also be installed For Topaz Workbench users, the Host Explorer is included with the kit and is installed accordingly. IMPORTANT: To use Topaz Workbench, you must define an HCI connection and connect Topaz Workbench to a mainframe system. Connections are defined within the Preferences dialog box, which is available from the Window menu by selecting Preferences> Compuware>Host Explorer>Host Connections. A Host Communication Interface (HCI) connection between the mainframe host and Topaz Workbench should have already been established during the installation and configuration of Topaz Workbench. Contact your system administrator to obtain specific connection credentials for your site for establishing Host Connections. (For system administrators, refer to the appropriate installation guide for further information on installing and configuring the HCI.) Performance Note: Generating Topaz for Total Test unit test cases for very large programs may require increasing the memory allocated to Topaz Workbench based on the available memory of the system. It always helps to improve the performance by providing more memory if it is possible. To do this edit the topaz.ini file in your installation directory of Topaz Workbench (by default the path is C:\Program Files\Compuware\Topaz Workbench\eclipse): Find the entries: -Xms256m -Xmx1536m Change them to: -Xms512m -Xmx3072m 2

9 Topaz for Total Test User Guide This change doubles the amount of memory available to the Topaz Workbench and Topaz for Total Test. Save the topaz.ini file and restart Topaz Workbench. Intended Audience This User Guide and Reference is provided for the users of Topaz for Total Test who actually create and execute tests. Its purpose is to help them in using Topaz for Total Test in an optimum way. How This Guide is Organized This guide contains the following: Process Flow - Topaz for Total Test - COBOL Batch Programs Basic Background Knowledge Topaz for Total Test Reference Command Line Interface Message Codes Limitations Glossary Product Support Visit the Compuware Support Center at to find product documentation, knowledge articles, and other technical resources. You can open a case with the Customer Solutions team, order products, and more. Contact Customer Solutions by Phone USA and Canada: or All other countries: Contact your local Compuware office. Contact information is available at Corporate Website Visit Compuware on the Web, go to for additional product information. 3

10 Overview - Topaz for Total Test - COBOL Batch Programs Topaz for Total Test is a new Unit Testing Product for COBOL batch programs. It utilizes Xpediter to collect unit test data and select the location in a COBOL program to create a unit test. Topaz for Total Test is flexible with these features: Collection of unit test data for selected unit test cases by using the new "Generate Unit Test" feature in Xpediter. Easy identification and setting of where in a COBOL program a unit test case should be created. Collected Xpediter information is used to generate the unit test case and format the data collected so it can be used by the test cases. A method to stub out parts of the program, by using the new "Generate Program Stub" feature in Xpediter, allowing test runs to be almost independent of the original program and realizing less system resource usage during testing. Introducing I/O Call Stubbing Another important feature of Topaz for Total Test is that it can use data collected during an Xpediter Debug session to create an I/O call data stub. Currently, Topaz for Total Test supports QSAM, VSAM (KSDS, ESDS, and RRDS) Read and Write stubs Db2 SQL statement stubs IMS DL/I statement stubs 4

11 Topaz for Total Test User Guide Stubbing out calls to I/O calls in your program makes it easier to run your unit test in isolation. This capability makes it easier to run your test case on more systems since it reduces your dependencies on data files In the case of a Read I/O call stub you no longer require the input file. Instead of the input file a record is read from the Read I/O Stub. In the case of a Write I/O call stub you no longer have to worry about changing the output file. You can run the program tests over and over again without affecting the file. Benefits The test case execution can be automated so the unit test cases can be run as a regression test suite to verify that previously resolved issues stay fixed. Changes to the typical development process to automatically run the unit tests as the last phase of the build process (SCM such as ISPW doing a build (generate), deploying the software to a test system) provide much faster feedback to any problems introduced into the software. 5

12 Benefits - Topaz for Total Test - COBOL Batch Programs Can import data collected from Xpediter Debug Sessions and automatically create Unit Test Cases. Utilize the data collected by Xpediter to create functional unit test cases which should significantly reduce the amount of work to create a Unit Test Case. Automatically create Unit Test Assertions based on the assumption that collected data is correct data (a possibly wrong assumption). The assertions will provide detailed messages when the Test Case fails because of an invalid assertion. Can be executed as part of a customer's continuous integration and build process. Provides testing stubs for I/O calls to help isolate the COBOL program from its environment. Unique Differentiation Includes some mock / stub libraries that will eliminate dependencies on QSAM and VSAM files and Db2 tables. These mock / stub libraries will allow specific small sets of consistent data to be used in the unit test case. This approach eliminates most of the test setup required for COBOL batch programs. Introduces a Test Scenario to help in the typical COBOL environment where a program may have many other program dependencies. Test Scenarios allow multiple dependent programs to be tested as a set because one program may create data required by subsequent execution of the next program. Automated collection and creation of Unit Tests from Xpediter data. Works with Xpediter Code Coverage to show code executed by the Unit Test run. Audience and Savings Topaz for Total Test is targeted at COBOL developers and provides automatic generation and execution of unit tests that are typically written by developers to verify that their code is correct. Time and expenses to create tests are reduced and benefits are realized by the fact that testing is thorough as developers can deliver by automating tests that only the developers can create. These tests are usually at a very detailed level and thus focus on specific parts of the program that are known only to the COBOL developer. 6

13 Open a Topaz for Total Test Project Topaz for Total Test does not have its own perspective. You can access it through context menus and menu choices in the Host Explorer perspective. To open a Compuware Topaz for Total Test project 1. From the Compuware menu, select Host Explorer. 2. Click on the Project Explorer tab. The Project Explorer lists all of your defined projects, including your Topaz for Total Test projects. You recognize Topaz for Total Test projects based on each project's folder structure when expanded. The Topaz for Total Test projects appears in the Project Explorer. To create a new Compuware Topaz for Total Test project 1. Right click in the Project Explorer view. 2. Select New > Project. The Select a Wizard opens. 3. Expand the Compuware folder. 4. Select Topaz for Total Test Project. 5. Click Next>. This opens the New Topaz for Total Test Project window. 6. In the Project Name field, enter a project name. You should choose a name that makes it easily recognized as a Topaz for Total Test project. 7. Click Finish. The new Topaz for Total Test project appears in the Project Explorer. 7

14 Topaz for Total Test Projects Upgrade Window The Project Explorer displays all your current projects. The sample screen shot shows the new Topaz for Total Test project. When creating a new Topaz for Total Test project, the sample Runner.jcl, RunnerBmp.jcl, RunnerBmpDB2.jcl, and RunnerDB2.jcl are already included. Modify the JCL as needed for your test environment. Topaz for Total Test Projects Upgrade Window When Topaz for Total Test detects that projects were created with an earlier version of Topaz for Total Test, you will be prompted to upgrade. The Upgrade dialog may include a list of all projects in need of the upgrade. 1. Click OK to start the upgrade. A message informs you when the upgrade has completed. 2. Click OK to close the message. When trying to open a project component from an earlier version, for example a structure, you may receive an upgrade message in the editor. 1. Click the Upgrade project button in the top right corner of the Editor window to start the upgrade. A message informs you when the upgrade has completed. 2. Click OK to close the message. Related Topics Create JCL Template Project 8

15 Topaz for Total Test Sample Training Programs The Enterprise Common Components (ECC) sample library (hlq.cpwr.mlcx170.slcxcntl) includes sample programs you can use to demonstrate Topaz for Total Test functionality and to verify the Topaz for Total Test installation. Compuware strongly recommends that, in addition to these procedures, you test your own application programs before considering the installation complete. Topaz for Total Test Sample Training Files Member Function CWKTCDB2 Sample JCL used to prepare the Db2 test program and also Create, and Grant a Db2 table. The last step also executes the program. CWKTDB2X COBOL source code for use with Db2 table. CWKTCOBX COBOL source code for use with QSAM input file. CWKTDATE COBOL source code for sub program used with test programs CWKTCOBX, CWKTDB2X, and CWKTVSKS. CWKTSUBC COBOL source code for sub program used with test programs CWKTCOBX, CWKTDB2X, and CWKTVSKS. CWKTVSKS COBOL source code for use with VSAM input file. LAUNCHDB Sample JCL for use with Xpediter/Eclipse launch configuration for test program CWKTDB2X. LAUNCHCX Sample JCL for use with Xpediter/Eclipse launch configuration for test program CWKTCOBX. LAUNCHKS Sample JCL for use with Xpediter/Eclipse launch configuration for test program CWKTVSKS. RUNNERDB Information on how to create a RunnerDB2.jcl file for use with test program CWKTDB2X. RUNNERCX Information on how to create a Runner.jcl file for use with test program CWKTCOBX. 9

16 Topaz for Total Test Sample Training Programs RUNNERKS Information on how to create a Runner.jcl file for use with test program CWKTVSKS. TTTDATD Used as a QSAM input file for test program CWKTDB2X. TTTDATH Used as a QSAM input file for test program CWKTCOBX. VSAMKSDS Sample JCL used to create a VSAM KSDS input file for test program CWKTVSKS. Use the Compuware Shared Services COBOL (CSS) Language Processor (LP) to compile the sample programs. Some main routines statically link to subroutines. Refer to the Compuware Shared Services User/Reference Guide for information on using the COBOL Language Processor. Optionally, you can use the online Compile Facility provided by Xpediter/TSO by selecting Option 1 (PREPARE) from the Xpediter/TSO Primary Menu. Notes: 1. Xpediter requires specific compile options which may vary depending on the language type and actual version being used. Always refer to the appropriate sections in the Compuware Shared Services User/Reference Guide for information related to any required compiler options. For example, options NOTEST and NONUMBER are required for COBOL. Refer to the Compuware Shared Services User/Reference Guide to assist in manually converting Compile and Link JCL to execute the CSS LP. The guide also provides information on how to set up the LP parameters to write a matching source listing to a DDIO file. 2. The SYSLMOD output from these compiles should not be directed to the ECC LOADLIB library, but to a dataset available to your application programmers for training sessions. CWKTCOBX Test Program CWKTCOBX is a COBOL QSAM demo program that utilizes sub programs CWKTDATE and CWKTSUBC. Compile CWKTDATE and CWKTSUBC first before CWKTCOBX. The LAUNCHCX member contains sample JCL. Use it in an Xpediter/Eclipse Batch debug launch configuration to execute the CWKTCOBX program and to easily generate Topaz for Total Test unit tests. The RUNNERCX member contains information on how to create a Runner.jcl file for use with test program CWKTCOBX in a Topaz for Total Test Project JCL folder. CWKTDB2X Test Program CWKTDB2X is a COBOL demo program that accesses a Db2 table and utilizes sub programs CWKTDATE and CWKTSUBC. Compile CWKTDATE and CWKTSUBC first before CWKTDB2X. The CWKTCDB2 member contains sample JCL used to prepare the test program and also Create and Grant a Db2 table. The last step also executes the program. The LAUNCHDB member contains sample JCL. Use it in an Xpediter/Eclipse Batch debug launch configuration to execute the CWKTDB2X program and to easily generate Topaz for Total Test unit tests. 10

17 Topaz for Total Test User Guide The RUNNERDB member contains information on how to create a RunnerDB2.jcl file for use with test program CWKTDB2X in a Topaz for Total Test Project JCL folder. CWKTVSKS Test Program CWKTVSKS is a COBOL VSAM KSDS demo program that also utilizes sub programs CWKTDATE and CWKTSUBC. Compile CWKTDATE and CWKTSUBC first before CWKTVSKS. Use the VSAMKSDS member to create a sample VSAM KSDS input file. The dataset created from this member is used in the LAUNCHKS and RUNNERKS JCL members. The LAUNCHKS member contains sample JCL. Use it in an Xpediter/Eclipse Batch debug launch configuration to execute the CWKTVSKS program and to easily generate Topaz for Total Test unit tests. The RUNNERKS member contains information on how to create a Runner.jcl file for use with test program CWKTVSKS in a Topaz for Total Test Project JCL folder. 11

18 Process Flow - Topaz for Total Test - COBOL Batch Programs Generate Unit Test from an Xpediter/Eclipse Debugging Session When developers need to test any COBOL program behavior changes, they would initiate an Xpediter debug session to generate a unit test and validate their change. 1. Restart debugging the program in Xpediter which stops at the main program entry. 2. Find the sub-program for which to collect test data. 3. Right click and select Generate Unit Test... on the Procedure section of the program on which to collect data. Refer to Context Menus from Xpediter Debug Session for more details. 4. In the Generate Unit Test window, select an existing project or create a new project. 12

19 Topaz for Total Test User Guide 13

20 Process Flow - Topaz for Total Test - COBOL Batch Programs 5. After you select the project, in the Scenario drop-down, the last scenario specified for this program is pre-selected. Otherwise, a new scenario is displayed, using the name of the program with the suffix _Scenario (<program>_scenario), for example, CWXTCOB_Scenario. 6. Optionally, you may specify the project's test suite into which the generated test scenario will be copied. 7. Check the appropriate Target Program Type boxes: DB2 if the Target Program Type is Db2 IMS if the Target Program Type is IMS NOTE: Generate program stubs are not available for IMS program type. 8. By default, all Generate stubs options are selected. Only uncheck the checkboxes if stubs aren't wanted. For more information on stubs refer to Stubs. a. Generate program stubs. Uncheck if program stubs should NOT be created. 1. Check Create assert condition for data passed into sub-programs for Topaz for Total Test to generate sub-program assertions for stubbed out sub-programs. Sub-program assertions validate all of the data that is being passed into a sub-program call just like a Write Assertion validates all the data that is passed into a Write I/O call. NOTE: You cannot define a Program Entry Assertion for a live sub-program. 2. Update the Maximum number of calls to generate for each program. Default is 100. NOTE: Generate program stubs are not available for IMS program type. b. Generate I/O stubs. Uncheck only if no I/O stubs should be created. Check or uncheck the desired data types and Operation checkboxes. Update the Maximum number of records to generate for each file field. Default is 100. Refer to Generate IO stubs for description of the context menu. c. Generate DB2 stubs. Uncheck if Db2 Statement stubs should NOT be created. Update the Maximum number of SQL statements allowed in each stub. Default is 100. d. Generate IMS stubs. Uncheck if IMS DL/I call stubs should NOT be created. Update the Maximum number of IMS calls allowed in each stub. Default is Check the Overwrite existing stubs checkbox if you want to replace the contents of previously created, same named, stubs with the data to be collected from this unit test data collection. If left unchecked, Topaz for Total Test will create a copy of the same named stubs, and identify each new set of Data Stubs with a 2 digit set identifier, for example: Existing stub: CWXTCOB_EMPFILE_READ.stub New stub: CWXTCOB_EMPFILE_01_READ.stub 10. Check or uncheck the Advanced performance options. Consider selecting this option when collecting unit test data for large programs with big and complex structures. a. Create test case check conditions only for program call parameters changed during execution. Check if you want limit the creation of test case check conditions. Note: If you only want to create program stubs, right click and select Generate Program Stub... on the Procedure section of the program on which to collect data. Generate program stub is not available for IMS program type. 14

21 Topaz for Total Test User Guide 11. Click OK to create the test case. 12. If so desired, use the Xpediter session to change variable values at this point. 13. Issue the appropriate Run commands in your Xpediter debug session. Xpediter collects and saves the program entry data from the structure specified by the Using Statements. 14. Xpediter continues executing the rest of the program. For a Data Stub to be created, at least one Read or one Write must be executed in Xpediter during data collection. When the program ends or is stopped, Xpediter transfers the collected data to Topaz for Total Test. 15. Once the Test Case data has been imported, the new Test Case(s) appear in the Test Project previously specified. Select the new Test Case(s) in the Test Project Explorer View. Topaz for Total Test populates the following project folders with the transferred data: Interfaces: program interface names using the program name. Scenario: test case Structures: COBOL layouts identified by their level 01 name. Stubs: Program, QSAM Read, QSAM Write VSAM KSDS Read, VSAM KSDS Write VSAM ESDS Read, VSAM ESDS Write VSAM RRDS Read, VSAM RRDS Write. SQL, SQL CURSOR Suite: test case Capture specific sets of data from a program There may be instances when you want to capture specific sets of data from a program when it is called for the "n th " time or for a specific value. This use case can be supported by using the Xpediter Feature: INTERCEPT to stop at the n th iteration of loop or the WHEN command to stop when a specific value is assigned to a variable. To use this: Set up the Intercept / WHEN command using the Xpediter/Eclipse interface after the Debug session has initially started. Set these commands on a line prior to Call of the Program. Once Xpediter has stopped you can then set the Program for Test Collection Data collection. 15

22 Process Flow - Topaz for Total Test - COBOL Batch Programs This approach allows collecting a specific set of data even though it may take many call iterations to get to the set of data you want to collect for the test. If needed, add some more sophisticated test assertions, for example: 1. Open the Test Case in the Scenario folder with a double click and select the Check Conditions tab in the Editor window. 2. Find the variable you want to base a range test on. 3. Change the operator to less than and enter the high boundary value in the assertion. 4. Add another assertion for the variable, change the operator to greater than and enter the low boundary value in the assertion. 5. Save your changes. Run a Test Suite/Scenario 1. Select the specific Test Suite/Scenario you want to run. 2. Right click on the selected Test Suite/Scenario and select Run Test Scenario..., or simply click on the Run Test Suite button in the upper right area of the Scenario editor. The Test Run window opens. 16

23 Topaz for Total Test User Guide 3. Select the host where you want to execute the Test Suite/Scenario from the list of Hosts in the Target Environment drop down list. 4. Select the JCL to use for the test execution from the JCL template view. Make sure your steplib DD points at your site's Enterprise Common Components (ECC) Release or newer load library (CPWR.MLCXnnn.SLCXLOAD), for example hlq.cpwr.mlcx170.slcxload. For details please refer to the Enterprise Common Components Installation and Configuration Guide for Release or newer. ${TOTALTEST_JOBCARD} will load the Topaz for Total Test default jobcard specified in Window > Preferences > Compuware > Topaz for Total Test. 17

24 Summary Unit Test Creation Scenarios 5. Select the appropriate Execution Options, Test Aids, and Code Coverage Options. For details on these options, please refer to Run Test Dialog. 6. Click OK to execute the test case. 7. The progress bar shows the test assets moving to the Host that will execute the Test Suite/Scenario. 8. A notification in the Console informs that the Test Suite/Scenario has completed and the Result Report opens in the Editor area. Review Test Results 1. After a test run the Result Report opens in the Editor area. Otherwise, double-click the desired Result Report in the Last Execution Output folder. 2. When the selected Result Report opens, look for failed tests. 3. Should some of the test cases have failed, follow the link in the Result Report to open the Test Details Report. 4. In the Test Details Report identify the specific test cases that failed and the specific test assertions that failed. Since this test had just been created and it was successful during the test data collection, an assertion test had probably been changed incorrectly. 5. Change the assertion test. 6. Rerun the test until all of the test cases complete successfully. Summary Unit Test Creation Scenarios Unit tests should be created to test any program behavior changes. There are three scenarios (existing test case, create a test case PRIOR to changes and create a test case AFTER modifications. 1. Unit test case already exists PRIOR to any modifications to the COBOL program. a. Check the unit test case assertions with the existing values. b. Run the unit test case to get a baseline for assertions. c. Make changes to the COBOL program, then rerun the unit test case created above. d. Validate that the changes and assertions for the new values are correct and transfer the new values to the test case. e. Rerun the unit test case and validate the assertions. 2. Create a unit test case PRIOR to any modifications to the COBOL program. a. Check the unit test case assertions with the existing values. b. Run the unit test case to get a baseline for assertions. c. Make changes to the COBOL program, then rerun the unit test case created above. d. Validate that the changes and assertions for the new values are correct and transfer the new values to the test case. e. Rerun the unit test case and validate the assertions. 3. Create a unit test case AFTER modifications to the COBOL program 18

25 Topaz for Total Test User Guide a. Check that the unit test case assertion values. b. Run the unit test case and validate the assertions for the unit test case are correct. There should be no reason to have to create the unit test case after the code modification unless it does not exist prior to the code change (#3 above). 19

26 Basic Background Knowledge In order to be able to use Topaz for Total Test efficiently, we recommend you become familiar with the basic principles and some elementary terms used in Topaz for Total Test. For more information, see Wiki: testing. Topaz for Total Test Integration with Xpediter/Eclipse When debugging a COBOL Batch Program using the Xpediter/Eclipse debugger, Topaz for Total Test provides a context menu (Generate Unit Test...) to automatically generate unit test case(s) from the debugging session. This eliminates creating the following elements manually: Interface Structures Test Scenario Test Case Stubs (optional) Suite (optional) Refer to Process Flow - Topaz for Total Test - COBOL Batch Programs and Context Menus from Xpediter Debug Session for more details. Interface Testing The concept of component interface testing, or just the shortened name interface testing, is a wellknown testing methodology that can be used to check the handling of data passed between COBOL programs. Topaz for Total Test Exploits Interfaces Using Topaz for Total Test, an interface consists of the called program and the data structures passed to it (also called the signature). As an example, in COBOL this may look like this: CALL PROGRAM1 USING PARM-1 PARM-2 The data structures passed in the above USING clause are usually copybooks. By importing these using Topaz for Total Test, they are translated to a language independent XML format simply called a structure in Topaz for Total Test. That way you can create and perform tests irrespective of the target object's language. Topaz for Total Test Adds Usage Information 20

27 Topaz for Total Test User Guide However, technically describing your interface by the target and the structures it uses is not enough. Still missing is information about which structures are used for input and which for output. This is because their usage is not contained in the structures nor does the call statement imply this, but this is merely a convention between calling program and called program. Topaz for Total Test provides a way to add this semantic information, called usage information, to the interface definition. Summary In a nutshell, the Topaz for Total Test test interface is a technical description (call interface) plus semantic information (usage). This provides a business view on the interface because it eventually masks out the technical details which are not relevant for the business functionality. We will need the argument structures in the specific order expected by the target. the usage of the structures or attributes (fields) contained therein. With this information, Topaz for Total Test knows how to (technically) call the target and how the data are (substantially) used by the target. Test Scenario The concept of a Test Scenario is used to define and describe what to test, which input to use, and which output to expect to consider the test successful. In terms of Topaz for Total Test, you need to provide the following information in order to create a test case for a test scenario: Which interface is being called? What is the name of the target that you want to test (program or load module)? Which input data need to be passed? Which check conditions must be met if the target functions as expected? 21

28 Basic Background Knowledge Check conditions are used and are essential to successful testing. Not only are they required to properly document the test scenario, but they are crucial for the re-use of the test scenario. Check conditions are needed to: Automate the tests Re-use the scenario for regression testing. A test scenario is aimed to be a reproducible test of one or more specific requirements implemented by an interface. Test Case Each time an interface is called during the test run, it represents a test case. The simplest variant of a test scenario contains just one test case, and it is thus referred as a simple test scenario. However, it is often required to create test scenarios that consist of more than one test case. There may be correlations that need to be reproduced in the test scenario. As an example, you may want to process contract data, but the function requests a contract ID that you must first read using another function. Or you may want to store certain data in a database in order to provide a reproducible data environment for your actual test interface and become independent from the environment. To accomplish such tasks, Topaz for Total Test allows you to combine multiple test cases in one test scenario, and to pass data from one test case to another by using references. 22

29 Topaz for Total Test User Guide As a rule, a test scenario consists of any of those test cases required to check one or more requirements of one specific interface. The test cases belong to this test scenario in the specific order. Please note that a test scenario should not test multiple interfaces, but use the additional test cases only for so-called setup and tear down purposes to support the test of one interface. Test Suite Opposite to the close relations of the test cases within a test scenario, a test suite is just a loose bundle of test scenarios. The test scenarios in a test suite are not related to each other and may even be executed in any order without interfering with each other. The test scenarios are bundled in a test suite mostly for practical reasons. For example, you may have a collection of test scenarios that you want to run regularly as a regression test. Test Run When you run a test scenario or a test suite, in this order: all required information and data are collected from the test scenario and the test interfaces it uses. Data is converted into a format required by the target platform, copied over to the target platform, and then the tests are executed on the target platform. Results data is converted back, parsed and the test results are displayed in a results report. 23

30 Creating And Preparing A Test Project Steps and procedures follow to create and prepare a test project. Topaz for Total Test users need to execute these steps for proper test setup. Create New Topaz for Total Test Project To start with a brand-new Topaz for Total Test project: Click on the Create a new Topaz for Total Test project toolbar icon or select from the menu File > New > Project > Compuware > Topaz for Total Test Project. Alternatively, just click on the New Project symbol available in the toolbar and select Compuware > Topaz for Total Test Project. The New Topaz for Total Test Project wizard will appear. Simply enter a project name and click Finish. If you already have a Topaz for Total Test project that you want to import into your workspace: Simply use the standard project import mechanism. From the menu, select File > Import, then in the list select General > Existing Projects into Workspace. Once imported, Topaz for Total Test will recognize the project as a Topaz for Total Test project. Create JCL Template To run tests, Topaz for Total Test requires at least one JCL template in the JCL folder of the project to be used for execution on the target platform. Sample JCL templates (Runner.jcl, RunnerBmp.jcl, RunnerBMPDB2, and RunnerDB2.jcl) are automatically included when creating a test project. Use Runner.jcl for tests that do not use live Db2 or live IMS BMP. Notes: If your test case includes live DB2, use JCL based on the RunnerDB2.jcl template. If your test case includes live IMS BMP, use JCL based on the RunnerBmp.jcl template. If your test case includes both live DB2 and live IMS BMP, use JCL based on the RunnerBmpDB2.jcl template. JCL Customization is Required Follow the instructions inside the sample JCL as they require some customization to meet the requirements of your specific test environment. Make sure your STEPLIB DD points at your site's Enterprise Common Components (ECC) Release or newer load library (CPWR.MLCXnnn.SLCXLOAD), for example hlq.cpwr.mlcx170.slcxload. For details please refer to the Enterprise Common Components Installation and Configuration Guide for Release Notes: 24

31 Topaz for Total Test User Guide Make sure a job or step REGION= parameter specifies an appropriate value. REGION=0M allows Test Runner to obtain all the storage it needs. When specifying any other value for REGION=, be aware that Test Runner requires 120 MB more, in most cases, than the unit test target program would require on its own. In our testing, the sample CWXTCOB Xpediter program requires a minimum of REGION=130M, but yours may be different due to z/os system setup/customization. For example, if a target program requires REGION=200M to run on its own, then REGION=320M should be specified. ${TOTALTEST_JOBCARD} will load the Topaz for Total Test default jobcard specified in Window > Preferences > Compuware > Topaz for Total Test. See also Topaz for Total Test Preferences. Adjust Project Properties To do this: Right-click on your project's name in the Project Explorer to the left, then, from the context menu, select Properties. From the list shown, select Topaz for Total Test. Usually, you do not need to make any modifications in the general Topaz for Total Test entry. Please refer to Project_Properties for a description of the options there. If you expand the Topaz for Total Test entry on the left, you will see one more entry: COBOL Copybook Importer. COBOL Copybook Importer You do not need to make any modifications in the COBOL Copybook Importer section unless your copybooks contain special characters that need to be replaced or removed. If so, click Add to add one or more string replacements. In order to remove strings, just leave the replacement entry empty. You are now ready to go. 25

32 Four Steps For Testing To get started quickly, this topic provides a four-basic step overview which you need to know to create and run tests with Topaz for Total Test. Please see the Reference part of this document, if and when you want to know more details. At this point, it is presumed that you have already created and configured a Topaz for Total Test project as described in Creating And Preparing A Test Project. Additionally, you need access to the copybooks that you are going to use either on a local file system or on a remote system. The four basic steps for manually creating and running a test are: Step 1: Import Structures Step 2: Define Interface Step 3: Create Test Step 4: Execute Test Note: When debugging a COBOL Batch Program using the Xpediter/Eclipse debugger, Topaz for Total Test provides a context menu (Generate Unit Test...) to automatically generate unit test case(s) from the debugging session. This eliminates creating the following elements manually: 1. Structures Interface Test Scenario Test Case Step 1: Import Structures The structures used need to be converted to the language independent format Topaz for Total Test uses. For COBOL copybooks, this is done by the Unit Test Structure from Copybook Import Wizard. 1. Right-click in your Topaz for Total Test Project Structure folder, then from the context menu, select Import Structure. The Import Unit Test Structure window opens. Alternatively, from the menu, select File > Import The Import Wizard opens. Select Topaz for Total Test > Unit Test Structure from Copybook, then click Next. The Import Unit Test Structure window opens. 2. Click Select... The file selection window opens. 3. Select one or more copybook(s) and click Open. 26

33 Topaz for Total Test User Guide 4. If necessary, select alternate character encoding from the Encoding list. 5. Select the structure folder in which to store the structure(s). 6. If desired, change the name of imported structure in the Filename for structure field. By default, the structure name is derived from the imported copybook file name(s). 7. If necessary, tick Overwrite existing structures and click Finish. The selected structures are now imported and can be used in Topaz for Total Test interfaces. The structures are usually imported only once unless their sources change. Alternatively, you can import structures from a Topaz Editor session. Refer to Unit Test Structure from Selection Import for details. Step 2: Define Interface The basic definition of an interface is the name of the program to be called (referred to as the test target) and the data structures passed to it as call arguments. This corresponds to the technical interface as specified in a CALL statement (program name and USING parameters). The technical interface is completed by adding the type of usage of the structures and/or attributes within them. The usage is: Usually a convention between caller and called program on which structures or attributes are used as input data or output data. Other types of usage are: fixed technical data, or no usage at all (for details see Usage for Structures and or Attributes). We recommend that you perform this step thoroughly, because it increases the efficiency of creating test scenarios. 1. Right-click in your Topaz for Total Test project, then from the context menu, select New > Interface... (alternatively, from the menu, select File > New > Other > Topaz for Total Test > Interface... The Interface Wizard opens. 2. Enter Name, Alias name, Type, and Description, then click Next >. 3. Add the required call parameters (structures) for this program by moving the desired items from the Available to the Selected column. 4. Click Finish. The Interface Editor opens with the newly created interface. 5. Select usage for structures and/or attributes. Usage for Structures and/or Attributes Usage Meaning Visibility in Test Scenario Editor in Used as input data In Input Data out Used as output data In Simple Check Conditions inout Used as input and output data as well Both in Input Data and Simple Check Conditions 27

34 Four Steps For Testing fix Fixed (technical) data, contents set in the No interface only none Not used and never filled with any data A No A Please see Edit Parameters for details on Usage none. Step 3: Create Test Scenario In a test scenario, you select the: Interface to be called in the test Input data to be passed Check conditions required to decide if the test output conforms to the expected results. Check conditions are often also referred to as assertions. Follow these steps: 1. Right-click on the interface for which you want to create a test scenario, then in the context menu, select New > Other > Topaz for Total Test > Scenario (alternatively, from the menu, select File > New > Other > Topaz for Total Test > Scenario 2. Click Next > The Test Scenario Wizard opens. 3. Enter Name (required), Alias name and Description. 4. If appropriate, check the appropriate Program Type box: DB2 if the Target Program Type is Db2 IMS if the Target Program Type is IMS 5. Click Next >. 6. Select interface (if not selected already). 7. Enter the name of the program to be tested in the Target field. 8. Click Finish. The Test Scenario Editor opens with the new test scenario just created. 9. Enter any input data required for the test. 10. Select tabulator Check Conditions and add the check conditions to verify the expected results. 11. Save the test scenario. If a test scenario requires more than one test case, it is called a complex test scenario. You can add additional test cases by: Clicking on the Add test case... icon above the list of test cases on the left of the editor. This opens the Add Test Case Wizard which allows you to add test cases just the same way as described above. Please refer to reference part for more details on how to create complex test scenarios. Step 4: Execute Test 28

35 Topaz for Total Test User Guide For test execution: All required specifications are collected, and transferred to the target platform. Then a job will be created based on the selected JCL template and submitted to the target platform. After the execution of the job has finished, the result data are transferred back, get analyzed. Finally, a result report is created and displayed. To execute the test, follow these steps: 1. Open test scenario, then click on the Run test scenario icon located in the upper right corner of the editor (alternatively, right-click on the test scenario in the Project folder, then select Run test scenario... from the context menu) The Run Test dialog opens For details on this dialog, please refer to Run Test Dialog. 2. Optionally, change the pre-selected Host Connection, JCL Template, Execution Options, Test Aids, Enable Code Coverage, and Code Coverage Options. 3. If your test case includes live Db2 or live IMS BMP, make sure to use JCL based on the RunnerDB2.jcl or RunnerBmp.jcl template. 4. Xpediter/Code Coverage must be installed at your site. Refer to the Xpediter/Code Coverage documentation for more details. 5. Click OK. The test will be run, then, after it has finished, the Result Report will be displayed. Note: You can close the Run Test dialog by clicking on Run in Background. If you do, the dialog will be closed, but the test continues to run in background. Its status can be seen in the Progress view which is usually found in the lower middle area. Once the test execution has finished, you can open the result report from this view by clicking on the link provided there. The Result Report offers two different views: Overview Displays the results reported for the test suite, all test scenarios, and all test cases within the scenarios. For each test case, all check conditions and their results are listed. In addition to that, miscellaneous data about the run environment are displayed. Details Displays all data in the parameter structures before and after the execution of the target in each test case. This detail view is intended for problem analysis. How To Reflect Interface Changes During ongoing development, an interface may change either due to changes in the structures it uses or changes in type or number of argument structures passed. Of course, this affects any test scenarios that have been created for this interface. However, due to the intelligent automatic migration mechanism built into Topaz for Total Test, there are just two steps required to reflect the interface changes in the associated test scenarios: 1. Re-import changed structures Only if changes were made to a structure, a new import is required to announce the changes to Topaz for Total Test. To do so, simply repeat Step 1: Import Structures and check Overwrite existing structures. 29

36 How To Reflect Interface Changes 2. Update test interface Simply opening the affected test interface which will automatically trigger the intelligent migration. Changes in structures will be recognized and migrated. If new structures or attributes were added, their usage and default values may then be changed, if necessary. To do so, please refer to Step 2: Define Interface. Finally, save the updated interface. After that, all test scenarios based on this interface will use the new interface when executed, migrating the test data accordingly. Note: If the interface changes require modifications in the test scenarios, possibly because of additional input data or new check conditions are to be defined/added, these modifications have to be made manually. 30

37 Topaz for Total Test Reference The following topics describe all components and user interfaces in detail. This includes all available options as well as the collaboration of the components. Related Topics Context Menus from Xpediter Debug Session Perspectives and Views Project Structures Interfaces Test Scenarios Test Suites Stubs Run Test Dialog Command Line Interface Configuring Topaz for Total Test Jenkins Plugin Import Total Test Project Into ISPW Messages Limitations 31

38 Context Menus from Xpediter Debug Session When you debug a COBOL Batch Program using the Xpediter/Eclipse debugger and come to the Procedure section of the program on which to collect data, Topaz for Total Test provides these context menus via right-click: Generate Unit Test... Generate Program Stubs... Generate Unit Test... When selecting Generate Unit Test... the Generate Unit Test dialog displays. Here you specify the project and scenario where the new unit test for the selected program will be generated. This eliminates creating the following elements manually: Structures Interface Test Scenario Test Cases Stubs Suite (optional) The dialog contains the following fields under the Location heading. Project From the Project drop-down, select a project for which the new unit test will be generated. The last specified Topaz for Total Test project should be pre-selected. The drop-down should list of all Topaz for Total Test projects in your current workspace. If you need to create a new project, click on the NEW... button to the right of the Project drop-down and the New Topaz for Total Test Project Wizard will open. See also Creating A New Project. Scenario After you select the project, in the Scenario drop-down, the last scenario specified for this program is pre-selected. Otherwise, a new scenario is displayed, using the name of the program with the suffix _Scenario (<program>_scenario), for example, CWXTCOB_Scenario. For more information refer to Test Scenarios. 32

39 Topaz for Total Test User Guide Note: You can change the scenario name to any value. See also Renaming Moving and Deleting Resources. Suite Optionally, you may specify the project's test suite. The generated test scenario will be copied into the specified test suite. For more information refer to Test Suites. The dialog contains the following check boxes under the Target Program Type heading. DB2 if the Target Program Type is Db2 IMS if the Target Program Type is IMS NOTE: Generate program stubs are not available for IMS program type. The dialog contains the following fields under the Stubs heading. Generate program stubs Check the Generate program stubs checkbox to indicate that a Program stub will be added to the scenario for each program call in the program. The stub will have the same name as the DD. Only uncheck the checkboxes if stubs aren't needed. For more information on stubs refer to Stubs. Remember: For Program stubs to be created, the sub-program(s) must be called while Xpediter is gathering the data to send to Topaz for Total Test. In order for Xpediter to get call data, the call parameters and results data must actually execute the sub-program during the captured debug session. Create assert condition for data passed into sub-programs Check the Create assert condition for data passed into sub-programs checkbox for Topaz for Total Test to generate sub-program assertions for sub-program stubs. Sub-program assertions validate all of the data that is being passed into a sub-program call just like a Write Assertion validates all the data that is passed into a Write I/O call. Note: You cannot define a Program Entry Assertion for a live sub-program. Maximum number of calls to generate for each program: Update the Maximum number of calls to generate for each program. Default is 100, maximum is Generate I/O stubs Check the Generate I/O stubs checkbox to indicate that I/O stubs will be added to the scenario as selected with the following Data Types and Operations checkboxes. An I/O stub will be added to the scenario for each QSAM or VSAM DD READ or WRITE in the program. The stub data includes the first record read for each DD, unless you specify a value greater than one (1) in the Maximum number of records to generate for each file. The stub will have the same name as the DD. Note: For a Data Stub to be created, at least one Read or one Write must be executed in Xpediter during data collection. When unchecked, I/O stubs will NOT be generated. 33

40 Context Menus from Xpediter Debug Session Data Types Check the Data Types checkboxes, QSAM and VSAM, for which to generate I/O stubs. I/O stubs will only be generated for the checked data types. Operations Check the Operations checkboxes, READ, WRITE for which operation to generate I/O stubs and check the Create I/O assert conditions for writes checkbox, if the generated WRITE stubs should include generated write check conditions. Maximum number of records to generate for each file: Update the Maximum number of records to generate for each file. Default is 100, maximum is Generate DB2 stubs Check the Generate DB2 stubs checkbox to indicate that Db2 statement stubs will be added to the scenario for Db2 SQL call in the program. Calls for Db2 SQL statements using a cursor are grouped into a single stub by cursor name, and the stub will have the same name as the cursor name. Calls for all other Db2 SQL statements are grouped by individual statement, and each stub will have a name identified by SQL statement type. Only uncheck the checkboxes if Db2 stubs aren't needed. For more information on stubs refer to Stubs. Maximum number of SQL statements allowed in each stub: Update the Maximum number of SQL statements allowed in each stub. Default is 100, maximum is Generate IMS stubs Check the Generate IMS stubs checkbox to indicate that IMS DL/I statement stubs will be added to the scenario for IMS DL/I calls in the program. Calls for IMS DL/I statements are grouped into a single stub by PCB name associated with the call, and the stub will have the same name as the PCB name. Only uncheck the checkboxes if stubs aren't needed. For more information on stubs refer to Stubs. Maximum number of IMS calls allowed in each stub: Update the Maximum number of IMS calls allowed in each stub. Default is 100, maximum is The dialog contains the following fields under the Processing heading. Overwrite existing stubs Check the Overwrite existing stubs checkbox if you want to replace the contents of previously created, same named, stubs with the data to be collected from this unit test data collection. If left unchecked, Topaz for Total Test will create a copy of the same named stubs, and identify each new set of Data Stubs with a 2 digit set identifier, for example: Existing stub: CWXTCOB_EMPFILE_READ.stub New stub: CWXTCOB_EMPFILE_01_READ.stub 34

41 Topaz for Total Test User Guide Note: If you have already assigned a stub in a Test Scenario, remember to update the Test Scenario if you want to use more recently created stubs (see also Stubs Input Data). Advanced performance options Consider selecting the Advanced performance options when collecting unit test data for large programs with big and complex structures. Create test case check conditions only for program call parameters changed during execution. Check if you want limit the creation of test case check conditions. Note: Users should set proper memory setting in the Topaz.ini based on the available memory of the system. It always helps to improve the performance by providing more memory if it is possible. Refer to Performance for how to increase memory. Generation of Unit Test Data After selecting all the appropriate Generate Unit Test dialog items, click OK to request unit test data generation from the debug session. Continue the debug session. When the debugger completes, a message similar to the following displays in the Console view: Test case created at exit of program program_name. Project: project_name: CWXTCOB_Scenario TestCase: CWXTCOBCase1 Considerations When Setting Limits for Stub Data Collection Test case execution may produce failures when stubbing files where the number of records processed is greater than the number of records collected in the stubs. Example 1: Target program ABC called sub-program XYZ 149 times. However, the stub for sub-program XYZ was limited to collecting data for only 100 calls. To avoid erroneous unit test assertion failures and possible improper processing by the target program, either use Xpediter to re-generate the test case, increasing the stub collection limits, or manually edit the stub to add the appropriate data for the additional sub-routine calls. Example 2: Target program ABC read 299 records from file INPUT01. However, the stub for file INPUT01 was limited to collecting data for only 200 I/O operations. To avoid erroneous unit test assertion failures and possible improper processing by the target program, either use Xpediter to regenerate the test case, increasing the stub collection limits, or manually edit the stub to add the appropriate data for the additional file records. Note: If it is impractical to increase the limits or manually edit the stub, when possible, use smaller sets of input data when using Xpediter to generate the test case. 35

42 Context Menus from Xpediter Debug Session Generate Program Stub... When selecting Generate Program stubs... from the Xpediter debug session context menu, the Generate Program Stub dialog displays. The Generate Program Stub dialog contains the following fields. Project From the Project drop-down, select a project for which the program stub will be generated. The last specified Topaz for Total Test project should be pre-selected. The drop-down should list of all Topaz for Total Test projects in your current workspace. If you need to create a new project, click on the NEW... button to the right of the Project drop-down and the New Topaz for Total Test Project Wizard will open. See also Creating A New Project. Stub Check the Generate Program stubs checkbox to indicate that a Program stub will be added to the scenario for each program call in the program. The stub data includes the first record read for each DD only. The stub will have the same name as the DD. Note: You can change the stub name to any value. See also Renaming Moving and Deleting Resources. Remember: For Program stubs to be created, the sub-program(s) must be called while Xpediter is gathering the data to send to Topaz for Total Test. In order for Xpediter to get call data, the call parameters and results data must actually execute the sub-program during the captured debug session. Related Topics Perspectives and Views Project Structures Interfaces Test Scenarios Test Suites Stubs 36

43 Perspectives and Views The installation of Topaz for Total Test provides a set of views within the Host Explorer perspective, arranged with defined positions within the client GUI. You may re-arrange the positions. You may even add or remove additional views. The idea is to switch between different perspectives optimized for different tasks. In addition to that, a perspective displays a custom toolbar. Perspectives and views are a basic Eclipse concept. Please refer to the corresponding chapter in the Workbench User Guide for Eclipse, if you want to learn more about it. 1 Open Perspective In order to open the Host Explorer perspective, from the menu select Window > Perspective > Open perspective > Other, then select Host Explorer (default) from the list. The default Host Explorer perspective provides a set of views. You may add other views that you need or want to see in your perspective, and you may re-arrange the views, if you prefer. If you want to reset the perspective to the default arrangement and set of views, from the menu select Window > Perspective > Reset perspective... Click on the Project Explorer tab to view or create Topaz for Total Test projects. The middle position of the Host Explorer perspective is confined to the different types of editors. For Topaz for Total Test, you have interface editor test scenario editor 37

44 Perspectives and Views test suite editor stub editor result report. The result report is not really an editor in the sense of editing data as it is display only. Some views depend on the product for which Topaz for Total Test has been installed and may not be visible in your product. Some of the other views may display data only when a certain editor is active for which the data makes sense. Toolbar Icons The Host Explorer perspective includes 2 Topaz for Total Test action icons in the toolbar. To manage the Topaz for Total Test icons in the toolbar, select Window > Customize Perspective. Then, in the Customize Perspective - Host Explorer window click on the Command Groups Availability tab. Scroll to the bottom of the Available command groups column and click on the check boxes for the Topaz for Total Test items to place a check-mark. Click OK to activate the icons and close the window. The toolbar will then show the following Topaz for Total Test action icons: Topaz for Total Test Toolbar Icons Icon Action Corresponding menu entry Create a new Topaz for Total Test project File > New > Topaz for Total Test Project Alternatively, these actions may be selected from the context menu displayed when right-clicking within the navigator view. Other icons for creating new Topaz for Total Test objects are listed in the following table: Other Topaz for Total Test Icons Icon Action Corresponding menu entry Unit Test Scenario from Excel File File > Import > Compuware > Topaz for Total Test > Unit Test Scenario from Excel File Unit Test Scenario to Excel File File > Export > Compuware > Topaz for Total Test > Unit Test Scenario to Excel File Create Unit Test Structure from Copybook File > Import > Compuware > Topaz for Total Test > Create Unit Test Structure from Copybook Create a new interface File > New > Interface... 38

45 Topaz for Total Test User Guide Extract Check Conditions File > New > Extract Check Conditions... Create a new stub File > New > Stub... Create new test scenario File > New > Scenario... Create a new test suite File > New > Suite If an appropriate element within the navigator has been selected prior to the action, this element will be used in the action. For example, if you right-click on an interface file, then select New > Other > Topaz for Total Test Test > Scenario, the new test scenario will be created with a test case for the selected interface. Although in this document, only one of these ways to select an action is shown, the alternatives are equivalent. It is up to the individual users to decide which variant they prefer. Project Explorer View This view, by default located to the left of the editor, provides a list of projects and their contents. This is an Eclipse standard view. Outline View This view, by default located to the right of the editor, provides a structural view of the data currently displayed in the editor. Using this view, it is possible to navigate quickly within the structure. If you click on an element in the outline view, the editor will automatically position to this element. The outline view is only available for editors that display data organized in some type of structure. For example, the outline view is not available when using the test suite editor. This is an Eclipse standard view. Add Check Conditions View This view, by default located to the right of the editor, displays a list of attributes available to create check conditions for. This view is only available in the test scenario editor while the Simple Check Conditions tab is selected. Host Explorer View This view, by default located to the left of the Project Explorer View, displays the remote systems connections. Progress View This view, by default located in the area below the editor, displays a list of test runs that are executed in the background by selecting Run in Background. The result report can be opened from here once execution has finished. All test runs stay on the list until explicitly removed by the user. 39

46 Perspectives and Views Note: If you have ticked Always run in background when sending a test run to the background, all subsequent test runs will be run in background without asking you first. In order to reestablish the default behavior, from the menu select Window > Preferences, then in General, untick the option Always run in background and click OK. This view is always active. It is an Eclipse standard view. General Editor Behavior Collapsing And Expanding Data And Structures The editors for interfaces and test scenarios display data structures and enable you to enter data values for attributes. Structures may contain attributes as well as other structures. The contents of structures can be displayed or hidden. This action is usually referred to as expanding and collapsing structures, respectively. It can be done in one the following ways: To expand just one level of a structure, click on the triangle on the left of the structure name. A second click will collapse the structure. To expand all levels within a specific structure, right-click on the structure and, from the context menu, select Expand levels. To collapse the structure, select Collapse levels. To quickly expand or collapse all levels of all structures displayed in the editor, click on the buttons Expand levels (plus sign) or Collapse levels (minus sign) located in the upper right of the editor view. This works in the outline view, too. Positioning Within Data and Structures You can use the Outline view to position the editor on a certain attribute or structure. If you click on an attribute or structure in the outline view, the editor will be positioned accordingly to display this attribute or structure. If necessary, the containing structures will be expanded. Filtering Data and Structures The scenario editor displays a Quick Filter field in the lower part. If you type in the name of an attribute or structure, or part thereof, it will quickly filter the displayed data to show only those attributes or structures whose name match the entered name or fragment thereof. If necessary, the containing structures will be expanded. Displaying Additional Information If you move the mouse over the name of attributes or structures, the full name including the containing structure names will be displayed in a pop-up. If you move the mouse over the value field of an attribute, the type and length of the attribute will be displayed in a pop-up. Preferences Topaz for Total Test preferences allow you to control settings and defaults for the various functions in Topaz for Total Test. To view or change your preferences, click Window>Preferences. The Preferences dialog box appears. Expand Compuware and select Topaz for Total Test. 40

47 Topaz for Total Test User Guide When you change your preferences settings, the updated preferences will be remembered the next time you start Topaz for Total Test. To restore the default settings, click Restore Defaults. Then click Apply or OK to accept your changes. Preferences Settings Logging format: Select the logging format from these choices: Console logging Eclipse logging Topaz logging The default is Topaz logging. Logging level: Select the logging level from these choices: INFO DEBUG WARNING ERROR FATAL The default is INFO. Report Generator: Select the only choice: Default Topaz for Total Test Result Report Generator Default JCL: In the JCL Jobcard field, specify the default job card information for Topaz for Total Test batch jobs on the mainframe. The variable ${TOTALTEST_JOBCARD} in the Topaz for Total Test JCL templates will then load the default jobcard information you specify here. Dataset High Level Qualifier: (Optional) Specify a high level qualifier other than the default TSO User ID to be used when Topaz for Total Test allocates datasets on the mainframe during test execution. The default is your TSO User ID. Additional Preferences Settings When you expand the Topaz for Total Test entry in the Preferences dialog box, these additional preference pages appear: Test Interface Editor Test Scenario Editor Test Structure Editor Test Stub Editor Each of these Editor preference pages specify the default column width for each of the display columns in the Topaz for Total Test editors. 1. Workbench User Guide for Eclipse. Concepts and usage of the Eclipse workbench explained 41

48 Perspectives and Views 42

49 Project Creating a New Project In order to start with a brand-new project, click on the toolbar icon. Create a new Topaz for Total Test project Alternate ways to create a new project: Select from the menu File > New > Project > Compuware > Topaz for Total Test Project. Click on the New Project symbol available in the toolbar and select Compuware > Topaz for Total Test Project. Right click in the Project Explorer view, select New > Project > Compuware > Topaz for Total Test Project. Click the NEW... button in the Generate Unit Test window. See also Context Menus from Xpediter Debug Session. After that, the New Topaz for Total Test Project Wizard will open. This is an Eclipse standard wizard which you can go through without any special requirements. You just have to enter the new project's name. Only if the project data should not be saved in your current workspace, uncheck Use default location and provide the folder where you want the project data to be saved. Importing an Existing Project If you already have an existing Topaz for Total Test project which you want to import into your workspace, invoke the Eclipse standard wizard for the import of existing projects. 1. Form the menu, select File > Import. 2. Choose General > Existing Project into Workspace from the list. 3. Select the folder containing the project data. The wizard will display a list of all projects found in the folder. 4. Select the one(s) you want to import and click Finish. All projects found after having successfully imported the project, will automatically be recognized as a Topaz for Total Test project. If you are managing your projects with an SCM product, the import process may differ depending on the SCM product you use. Often the SCM products provides its own import wizard. Please refer to the SCM product manual for details. Project Structure Several folders are created for a Topaz for Total Test project. 43

50 Project Their purposes are described in Table Project Folders. Project Folders Folder Description Unit Test Topaz for Total Test Unit Test Contains the Unit Test folder structures for unit tests for the Topaz for Total Test project. Interfaces Test Interfaces Contains the interface files which define the technical and functional aspects of a call interface. JCL JCL templates Contains the templates used to create the job submitted to execute tests on the target platform. See also Create JCL Template. Output Unit Test Output folders Contains the History, JUnit, Last Execution, and Reports sub-folders holding all information of a test run. When opened, a Result Report will be displayed. 44

51 Topaz for Total Test User Guide History Archived Tests Contains test archives of earlier test runs, if the option Save time stamped archive files has been select in the Topaz for Total Test project properties. When opened, a Result Report will be displayed. JUnit Jenkins Report Contains the Result Reports that can be copied into Eclipse or any other tool that supports JUnit. Last Test Results from Last Execution Execution Contains the plain results of the check conditions in a test run. Can be used to import results into any quality management tool. Reports Static Report Contains the Result Reports as displayed when opening a test archive, but saved as static HTML file. Sonar SonarQube Report Contains the Result Reports that can be imported into SonarQube or any other tool that supports Sonar files. Scenarios Test Scenarios Contains test scenarios holding all information necessary to perform a test: test cases with the interfaces used, input data, and check conditions. Structures Parameter Structures Contains structures used as parameters in interfaces in a language independent format. Stubs Test Stubs Contains data and program stubs defining the simulated return data for calls to a given interface. Suites Test Suites Contains test suites defining a collection of test scenarios to be executed with this suite. Project Properties Before you can start to work with your Topaz for Total Test project, you have to configure the project properties. To do so, open the project properties by right-clicking on the project, then select Properties from the context menu. 45

52 Project General Settings In the list to the left, select Topaz for Total Test to display the general Topaz for Total Test properties. Save Time Stamped Archive Files After executing a test, all information regarding this test is collected in an archive file, including input data, output data, check conditions and their results, as well as some additional runtime information. If you activate this option, archives from previous runs are retained by adding a timestamp to the file name and moving them to the History folder of the project. If this option is not active, archives are always overwritten if you re-run a test. Usually, the recommended solution is to use an SCM product for the purpose of archiving. If you do not have an SCM product available or, for some reason, do not want to use it for Topaz for Total Test archives, you can activate this option to obtain at least a simple versioning. This option is inactive by default. Use Stubs With the aid of test stubs, instead of actually calling a program, a simulation is performed at runtime, acting as if the called program processed the call and returns a pre-defined set of return data. This options controls if stubs defined in a test scenario should be used or not. If this option is active, all stubs defined in a test scenario will be activated during the test which means that the stubbed 46

53 Topaz for Total Test User Guide interfaces will be simulated as defined. If this option is not active, stubbing will not be used during the test run, even if the scenario contains stubs. This enables you to switch off stubbing without having to remove all stubs from the test scenarios. This option is active by default. Its project setting can be overridden in the Test Run dialog. Stubbing is an optional feature. Refer to Stubs for more information on stubs. Delete Temporary Files Some temporary files are created during the execution of a test on the client and on the target platform. These files are needed to collect the scenario specification into an archive and to transfer the input and output data between client and target platform. Usually, these temporary files are not needed any more after the execution has completed. However, if you want to re-submit the test job for use with a debugger, the temporary files are required to resume the test execution. In this case, switch this option to off to maintain the temporary files after execution. If this option is active, then all temporary files will be deleted once the execution has been completed. If this option is not active, all temporary files will be kept. This option is active by default. Its project setting can be overridden in the Test Run dialog. Initialization with byte value (hex string) The initialization of empty attributes with specific values is a test aid which helps to identify problems caused by the use of uninitialized data by the test target. This may apply to the use of data not passed in a call as well as missing return values. Such problems are often difficult to identify because during testing, there may be no difference between uninitialized and initialized data. Therefore, problems usually occur later, maybe even in production, and are usually not easy to reproduce. With the help of this test aid, uninitialized data can be distinguished from initialized data. In order to activate this option, in Initialization of empty fields enter a string to be used for the initialization of all fields with no defined value. The string must be in hexadecimal representation. Choose a value that is unique so it differs significantly from common initialization values. A good and proven example is x'ee' as it largely improbable to appear within real data. In the Test Run dialog, you can decide if this option is used or not. If not used, all fields without defined values will be initialized with a value of zero for numeric data types, and with a value of spaces for character data types. Note that this "clean" initialization renders it impossible to identify initialization problems. Repeat Execution Default The repeated execution of test cases is a test aid which helps to identify problems caused by missing or inappropriate internal initialization of data in the test target. Such problems are often overlooked during the test phase, but occur in production due to optimization like resident loading and reusing programs. Because these problems are often difficult to identify, Topaz for Total Test provides this option to verify that a repeated execution does not affect a programs functionality. Please note that not all interfaces may support repeated execution by design. If this option is not active, each test cases will be executed just once. This option is inactive by default. Its project setting can be overridden in the Test Run dialog. Repeated Execution Count If the repeated execution is active, each test case can be executed multiple times. Specify the actual number of times in this field. 47

54 Project Dataset allocation overrides Use this section to override the default number of blocks allocated for the binary input (BININP) and binary results (BINRES) datasets on the mainframe. When many records are present in a testcase/stub, it might be necessary to increase the primary and/or secondary extends for these files. Override Default Allocation Sizes Check the Override Default Allocation Sizes check box to activate the override values you specify for the primary and secondary extends for the BININP and BINRES datasets. The value 0 (zero) specifies using the default allocation. COBOL Copybook Importer On the left of the project properties dialog, extend the Topaz for Total Test entry and select COBOL Copybook Importer to display the COBOL Copybook Importer properties. Copybook Replacements If you use text replacements in your copybooks which are resolved using a COPY REPLACING statement in your COBOL programs, it is necessary to specify replacement strings in this properties dialog. If you do not use replacements, it is not required to provide any settings here. 48

55 Topaz for Total Test User Guide Add replacements by clicking Add, then enter the From text string as found in the copybook, and the To text string which will replace the former during import. Add more replacements if necessary. Edit or remove replacements by selecting the entry, then clicking on the appropriate button on the right. During import, multiple replacements can take place. All defined replacements are used. Longer strings are replaced first to avoid unwanted side effects. For example, if you specify that ABC be replaced by XXX and ACBDE be replaced by YYY, during import, a text ACBDEF will be replaced by YYYF, not by XXXDEF which is probably just what you would expect. Refer to Importing Structures for more information. Renaming, Moving and Deleting Resources If you rename a structure, interface, scenario or stub file, all resources that reference it will be updated accordingly. Similarly, if you move a file, all references it contains as well as references to it in other files will be updated accordingly, unless you are just dragging and dropping it. You must use the Move entry from the context menu in order to move a file and at the same time update all references. In order to do so, right-click on the resource you want to move, then select Move from the context menu and, in the pop-up window, select the target folder you want to move it to. If you delete a structure, interface, scenario or stub file, it will be removed from all resources that reference it. Before the actual update is performed, you can always click on the Preview button in order to see which resources will be affected by the update. 49

56 Structures Structures, or layouts, in Topaz for Total Test are descriptions of the data areas passed when calling a test target. This description is in an language-independent format based on XML. This allows you to create tests independent from the programming language of the test target. In COBOL, these are the USING parameters in the CALL statement of the caller, or the USING parameters of the PROCEDURE DIVISION statement in the called program. If your COBOL structures include REDEFINES, they are created as overlapping structures and require special attention in the interface usage definition. Refer also to Usage none for more information. Importing Structures Before a test interface can be defined, the structures used in the interface must be imported into Topaz for Total Test. During the import of a structure, the language specific description of the structure is transformed into the language independent format used in Topaz for Total Test. The import input may not even be in a programming language format, but can be virtually any format for which an import plug-in is installed. Currently, a Unit Test Structure from Copybook Import plug-in is included with Topaz for Total Test. Other import plug-ins can be made available on request. Unit Test Structure from Copybook Import You can import COBOL Copybooks using the Import facility. 1. Right click in the Project Explorer and select Import from the context menu or Select File > Import from the menu. This opens the import wizard. 50

57 Topaz for Total Test User Guide Note: To quickly display matching entries, enter a filter text in the input field located in the top area of the wizard window. For example, to find the COBOL copybook import, enter copy. 2. From the list, select Topaz for Total Test > Unit Test Structure from Copybook, then click Next >. This opens the Import Unit Test Structure wizard. 51

58 Structures 3. Now click on the Select button, navigate to the where your copybooks are stored, and select one or more copybooks that you wish to import. If necessary, change the Encoding to match the character encoding used in your copybooks. 4. In the middle area, select the structures folder (or a subfolder) of the Topaz for Total Test project into which you want to import. By default, the name of the output structure file is derived from the file name of the input copybook. This may optionally be changed. 5. If the structure files already exist in the project, you must select Overwrite existing structures in order to replace them during the import. This is usually the case, when you re-import copybooks because they have been modified. 6. Click Finish. Note: Importing structures from COBOL copybooks does not require the test targets to be written in COBOL, either. This is in fact completely independent. Create Unit Test Structure from Selection Context Menu 52

59 Topaz for Total Test User Guide Topaz for Total Test also provides context menus to import COBOL layouts from these other Topaz programs: Topaz Editors Topaz Editors Topaz Host Explorer When you are editing a COBOL layout or program with one of the Topaz Editors, you can create unit test structures from the selected COBOL layouts. 1. Select the COBOL layout(s) in your Edit session, starting with Level 01 and including all of the associated layout elements. 2. Right click in the selected Level 01 data structure of the layout and select Create Unit Test Structure from Selection... from the context menu. This opens the Create Structure from Selection wizard with the Structures folder highlighted and the structure name pre-filled as it was derived from the Level 01 description. 3. If the structure files already exist in the project, you must select Overwrite existing structures in order to replace them during the import. This is usually the case, when you reimport copybooks because they have been modified. 4. Click Finish. 53

60 Structures Topaz Host Explorer When navigating in the Host Explorer to a COBOL layout member in a z/os dataset, you can create a unit test structure from the selected layout. 1. Select the COBOL layout in your Host Explorer session. 2. Right click the selected layout member and select Create Unit Test Structure from Copybook... from the context menu. This opens the Create Structure from Copybook wizard with the Structures folder highlighted and the structure name pre-filled with the member name. 3. If the structure files already exist in the project, you must select Overwrite existing structures without warning in order to replace them during the import. This is usually the case, when you re-import copybooks because they have been modified. 4. Click Finish. Import Processing in Detail Usually, it is assumed that each copybook file contains the description of exactly one level 01 data structure. In such a case, one copybook file corresponds to one Topaz for Total Test structure. However, if, during import, this is found not to be true, the following actions are taken: 54

61 Topaz for Total Test User Guide If a copybook file contains multiple level 01 data structures, the import will automatically create multiple structure files. The file names will be suffixed by a consecutive number. This does not apply for level 01 REDEFINES structures. If a copybook file does not contain a level 01 structure at all, a surrounding structure will be automatically created based on the copybook file name. During import, the following information will be taken from the COBOL copybook: Structure names Structure alignment Attribute names Attribute data type (from USAGE clause) Attribute default values (from VALUE clause) REDEFINES structures are recognized, too, creating overlapping structures. Update Structures If structures change due to ongoing development or error fixing, these changes must be made available in Topaz for Total Test, too. Simply re-import the affected structures. Of course, the option Overwrite existing structures in the import wizard must be active. After that, open the interfaces which use the changed structures, and set default values and usage of any new fields, if applicable. Thanks to the automatic intelligent migration technique in Topaz for Total Test, it is not necessary to manually update all test scenarios. Instead, they will be migrated on the fly when they are opened or executed in a test run. Naturally, if new attributes have been added to a structure, and these new attributes must be provided in an existing scenario, there is no way to avoid a manual update of that test scenario. The same applies if the name of an attribute has changed, so the migration cannot automatically match old name with new name. Viewing a Structure The Structure Editor has two tabs that can be selected in the lower area of the editor view: Edit Parameters to view the type and values of the layout structure. Information to display the name of the structure. Edit Parameters The Edit Parameters tab of the Structure Editor is used to view the structure definition. The Structure column displays the fields of the structure. Use the Expand buttons to expand and collapse the levels. The following icons identify the structure types: identifies 01 LEVEL identifies GROUP identifies NUMERIC identifies CHARACTER identifies REDEFINES 55

62 Structures identifies KEYS identifies ARRAYS The Type column displays the data type of the fields. The Input Length column displays the maximum input length for the field. To copy the value, right click on the field and select Copy value. 56

63 Interfaces An interface describes how to call a program interface that is to be tested. It is based on the technical parameters which correspond to those used in a call instruction: the called program and the data areas passed to it. In Topaz for Total Test, we speak of the target program and the argument structures, respectively. In addition to this purely technical information, Topaz for Total Test requires you to add some semantic information, that is, information about how the content of the data area is used by the interface. Structures and attributes may be used as input, output or other, like control purposes. This information is usually defined outside the program source as a convention between the target program and its callers. By adding this information to a Topaz for Total Test interface, defining test cases based on it becomes a much easier and more efficient task. Because an interface is defined once, but usually used multiple times, the additional effort is well invested. Creating a New Interface In order to create a new interface, select the project folder in which you want to create the interface, right-click to open the context menu and select New > Interface.... The Interface Wizard opens. On the first page, enter the following data (obligatory fields listed in bold): Field Description Name File name for the interface Alias NameMore descriptive name Type Select the interface type, either: Default, Keyed (KSDS) DescriptionDetailed description If you have not selected a Topaz for Total Test project before invoking the wizard, a list of Topaz for Total Test projects displays in the Project list at the bottom of the wizard pages. You must select a project to continue to the next wizard page. Click Next > to proceed to the next page. For z/os, this is usually the name of the load module. Then select the Call Parameters by selecting the required structures from the list of Available (previously imported) structures on the left, and clicking on the middle arrow in order to copy them to the list of Selected structures on the right. A structure may be copied to the Selected structures multiple times, if necessary. The Selected structures must be in the exact order in which they are expected by the target when invoking this interface. This means, the structures must exactly match the data areas passed in a call. You can re-arrange the Selected structures by selecting a structure and clicking the Up or Down buttons on the right of the list. Click the Remove button to remove a structure from the Selected structures. 57

64 Interfaces If you select a structure, additional information regarding this structure will be displayed in the lower area. Although displayed, an Alias is not used for structures. Once you have provided all required information, click Finish. The interface will be created and opened in the Interface Editor where you can complete the interface description by providing the usage of the structures and attributes. Optionally, default values can be provided. For details on how to use the Interface Editor, please refer to Editing an Interface. Editing an Interface The Interface Editor has two tabs that can be selected in the lower area of the editor view: Edit Parameters to edit the type and values of argument structures. Information to edit the alias name and the description, but also the target name, as well as the structures used in the interface. Edit Parameters The Edit Parameters tab of the Interface Editor is used to complete the interface description to provide a business view on the interface. The Structure column displays a list of the structures and attributes used in the interface. The Usage column allows you to change the usage of structures and attributes. The Default Value column allows you to enter default values for attributes. Structure The Structure column displays a list of the structures used in the interface, including their substructures and attributes. Please refer to General Editor Behavior for a detailed description of all options provided to navigate in this list. You can change the names of structures and attributes in this list by simply overwriting them in the Structure column. The new names will be used in all test cases based on this interface. This can be very useful to change cryptic technical names into more descriptive names. Changing names like this will even affect test cases that have been created before. When the structure includes keys, for example VSAM KSDS or VSAM RRDS structures, the primary key field is identified with the key icon. To set a field as primary key, right-click the field in the structure and select Set as Primary key. This update will also be reflected in the associated stub(s) and test cases for this interface. Usage In the Usage column, you can select the type of usage for a structure or field from a drop-down list. The usage of a structure or attribute is information about how it is used in the interface. This information is required to later display only relevant attributes, hence enabling an easy and efficient creation of tests. Because this information is not available from the source code, it must be added here. If you change the usage type of a structure, all attributes and substructures contained therein will be changed to the same usage type. It is therefore best to proceed from top down, that is, first define the main usage of the argument structures, then go down and modify its substructures and attributes as required. The meaning of the different types of usage is explained below. 58

65 Topaz for Total Test User Guide Note: Providing this information is an important step in order to facilitate the use of the interface. Select input usage only for attributes which require data input in a test case, saving time when creating scenarios. Usage "in" Attributes for which data have to be entered when defining a test, must be defined as usage in (input data). This usage type allows data to be entered in a test case. Usage "out" Attributes in which the interface target returns data, must be defined as usage out (output data). This usage type allows data to be used in check conditions in a test case, and to be passed on to subsequent test cases. Usage "inout" Some interfaces may use the same attributes for input and output at the same time. (Although this is not a good interface design, these interfaces must also be tested.) Attributes for which data have to be entered when defining a test, but at the same time are filled with a return value, the usage type must be set to inout (input and output data). Usage "fix" Attributes may provide input data to the interface target, but at the same time, this input need not be entered in a test because, for this interface, the data are always the same. In such cases, the usage type should be set to fix (fixed data). This usage type effectively hides such attributes in test cases, keeping them as simple as possible. Usage "none" This usage type is required for overlapping structures as defined by a REDEFINES clause in COBOL. You specify which one of the overlapping fields to use when processing these fields for this interface, and which not to use. All overlapping structures that are not used for this interface target, must be set to usage none (not used at all). Usage type none not only hides such attributes, but effectively prevents them from being assigned any data. Default Value The Default Value column allows you to change or add the default value of an attribute. The default value is used in all test cases based on these interfaces. If you change a default value in an interface, this will even change the default value for existing test cases. Even a newly created interface can have default values. These default values are taken from the structures. Default values from structures are displayed in blue. If you change or enter new default values in the Interface Editor, these values are displayed in black. Values for text attributes can alternatively be entered in hex format, for example as x'61a5'. The hex value must be given in the target platform encoding because for hex values, no character conversion is performed. If you want to reset a default value entered in an interface to the default value from the structure, simply clear the input field. If you want to override a text default value taken from the structure to spaces, simply enter space. Besides entering values for attributes, you may enter values for structures, too. However, as structures cannot really hold values, the value entered will be propagated to the attributes contained 59

66 Interfaces in the structure. Values entered for structures are always treated as character data. Please note that character data propagated to numeric data types may lead to unexpected values. The values used in a test case are determined in the following order for input attributes (usage in and inout): 1. Value entered in the test case, if existing 2. Default value entered in the interface, if existing 3. Default value stored in the structure, if existing 4. Empty according to project settings (see Building/Parsing Settings for details) Table_Processing Order provides an overview of the processing order. Processing Order Usage Test case Interface Structure Empty A in out in/out fixed none n/a B A Depending on the project settings see General Settings for details. B Attributes without a defined usage may occur due to new attributes in reimported structures, if the interface has not been updated. If an input attribute is later changed to a non-input usage type, values previously entered in test cases based on this interface are not used any more (but they are still present in the test case). Information The Information tab allows you to change the Alias Name and Description of the interface. Additionally, you can change the Selected structures. Changes will affect all test cases based on this interface. You can add, remove or re-arrange the Selected structures. However, any modifications that lead to changed positions of the structures, will cause an Automatic Intelligent Migration to take place for the affected test cases. This will usually work as expected, but under some circumstances the Automatic Intelligent Migration may not be able to migrate all values. 60

67 Topaz for Total Test User Guide 61

68 Test Scenarios Creating A New Test Scenario In order to create a new test scenario, right-click on the Scenario folder in the project where you want to create the scenario, and select New > Scenario (or from the File menu select New > Other > Topaz for Total Test > Scenario). The Scenario Wizard opens. On the first page, enter the following data (obligatory fields listed in bold): Field Description Name File name for the test scenario Alias Name More descriptive name Description Detailed description Program Type Check the appropriate box: DB2 if the Target Program Type is Db2 IMS if the Target Program Type is IMS If you have not selected a Topaz for Total Test project before invoking the wizard, a list of Topaz for Total Test projects displays in the Project list at the bottom of the wizard pages. You must select a project to continue to the next wizard page. Click Next > to continue to the next page where you select an interface. You can only select a program interface; non-program interfaces aren't included in the selection list. If no interfaces are listed, click < Back. The selected interface is also used as input for the Test case name field. If so desired, change the Test case name. Add the name of the program to be tested in the Program field. Optionally, you can enter values in the Csect, Alias, and Description fields. Click Finish to create the scenario. Or optionally, click Next > instead to proceed to the final page where you can select another destination folder for the test scenario. After completing the wizard, the newly created test scenario opens in the Test Scenario Editor where you can specify input data, check conditions, stubs, and also add more test cases. Note: If the test scenario is an IMS BMP test scenario, then there can only be one test case per scenario and suite. Also, exits aren't currently supported. For details on how to use the Test Scenario Editor, please refer to Editing Test Cases. Editing Test Cases Test case specifications are entered in the Test Scenario Editor. On the left side of the editor view, all test cases contained in the test scenario are listed. By selecting a test case in this list, its data are displayed in the editor and can be entered or updated. 62

69 Topaz for Total Test User Guide To add a test case, click on the Add test case... icon above the Test cases listing. This opens the Add Test Case Wizard which allows you to add test cases. The Test Scenario Editor has four tabs which can be selected in the lower area of the editor view: Input_Data to enter values or select field references for input attributes. Stubs to select one or multiple stubs to be used in this test case. Assertions to specify check conditions in a simple and efficient way. Information to change alias name and description and obtain information about the interface of a test case. Input Data The Selection tab of the Test Scenario Editor is used to provide the necessary input values for a test case. The Structure column displays a list of the structures and attributes used for input to the test case. The Location column describes the position values for the specific field. The Value column allows you to enter or change the values of attributes. The Assignation from column allows you to select a reference from preceding test cases to assign its value as input to an attribute or structure. Structure The Structure column displays a list of the structures used as input in the test case, as specified in the interface definition, including their substructures and attributes. Please refer to General Editor Behavior for a detailed description of all options provided to navigate in this list. If the attribute is contained in an array, by default, the first array element will be used for the input value. If you need an input value for another array element, right-click on the array, then select Add array element from the context menu. A new array element will be added to the array which you can now enter input values for. Alternatively, you can right-click on an array element and select Duplicate array element from the context menu to create a copy of the array element and add it to the array. In order to remove an array element, right-click on the array element, then select Remove array element from the context menu. Other right-click options for the Structure elements include: Copy value Redefine element Replace element Insert new field after element Delete element Revert to original interface Location The Location column describes the position values for the specific field. 63

70 Test Scenarios Value The Value column allows you to enter values for input attributes. Default values derived from the structures and interface are displayed in blue. Values entered in the test case are displayed in black. If you enter values that are not suitable for the attribute, for example, strings that are too long, they are displayed in red. Values for text attributes can alternatively be entered in hex format, for example as x'61a5'. The hex value must be given in the target platform encoding because for hex values, no character conversion is performed. If you want to reset a default value entered in a test case to the default value from the structure or interface, simply clear the input field. If you want to override a text default value taken from the structure to spaces, simply enter space. Besides entering values for attributes, you may also enter values for structures. However, as structures cannot really hold values, the value entered will be propagated to the attributes contained in the structure. Values entered for structures are always treated as character data. Please note that character data propagated to numeric data types may lead to unexpected values. Assignation From The Assignation from column allows you to specify that a value should be assigned from the output of a preceding test case at runtime. In order to do so, click on the Assignation from column in the line of the attribute for which you want to select a reference. A button with three dots appears. Click this button, then the Select Reference pop-up will appear where you can select the reference test case from the drop-down list, then select the desired attribute from this test case. Because (naturally) you cannot have forward references, only test cases preceding the current one are displayed in the dropdown list. Note that the source and destination attributes must be data format compatible. Otherwise, problems may occur during execution. Any numeric data types are compatible and a proper conversion will be performed during the assignation at runtime. Alphanumeric and numeric data formats can be compatible, too, if the alphanumeric attribute contains an all-numeric character value. If the source attribute is larger than the destination attribute, truncation will occur and a warning will be issued by the Runner. Besides using assignations for attributes, you may specify an assignation for structures, too. An assignations on structure level will always be treated as character data. The assigned value will propagate to the attributes contained in the destination structure. If source and destination structures have different lengths, the data will be truncated or padded with spaces accordingly. Keep in mind that character data propagated to numeric data types may lead to unexpected or invalid values. If a reference has been selected for an attribute, the test case and attribute name will be displayed in the Assignation from column. If you specified both, a value and an assignation at the same time, the assignation will not be used during test execution, and the reference displayed in the Assignation from column will be grayed out as a hint. However, if you remove the value, the assignation will be reactivated immediately. This allows you to use assignations in a test, but to occasionally overwrite them easily. In order to change a reference, invoke the Select Reference pop-up again in the same way as when you added the new reference, then select another reference attribute to be used. In order to remove a reference, invoke the Select Reference pop-up again in the same way as when you added the new reference, then check Remove reference and click OK. Stubs The Stubs tab of the Test Scenario Editor is used to assign stubs to a test case. 64

71 Topaz for Total Test User Guide Please refer to the Stubs section for information about how to create stubs. In the Stubs column, you can select one or more stubs to be used in the test case. On the left side, all Available stubs are listed. Select a stub in this list and click on the arrow in the middle to copy it to the Selected list on the right. You can copy the same stub multiple times. During execution, the Selected stubs are used to simulate calls to the targets of their interface. If the same target is contained multiple times in the Selected stubs, the stubs will be used in exactly the order provided. This means that the first stub is used for the first call, the second stub is used for the second call, and so on. If there are more calls to the stub target than you have stubs in your test case, the last stub will be re-used indefinitely. This allows you to easily use one stub for multiple calls, if applicable. Stubs apply only to the test case they are selected for. Their target will not be stubbed in other test cases unless you have specified stubs there, too. Note that stubs can be turned off completely in the project settings or in the Test Run dialog. This allows you to use stubs for tests during development, but later use the same tests with real calls without the need to go through all affected test scenarios to remove them first. Assertions The Assertions tab allows you to define the conditions under which a test case is considered successful in a most simple way. Attribute The Attribute column displays a list of output attributes that can be used for check conditions. In order to add additional attributes, right-click in the list and select Create new assertion from the context menu, or click on the button in the upper right area of the editor. Then click on the Attribute column of the newly created assertion to select the attribute to be used. Alternatively, you can use the Create new assertion icon to add a check condition for an attribute by double-clicking it. If the attribute is contained in an array, by default, the first array element will be used for the assertion. If you need an assertion for another array element, right-click on the array, either in the pop-up list or in the Create new assertion view, then select Add array element from the context menu. A new array element will be added to the array which you can now select for an assertion. In order to remove an assertion, right-click on the assertion, then select Delete from the context menu. If you want to just temporarily disable an assertion, you can change its comparison operator to noassert. Comparison The Comparison column allows you to specify the comparison operator used for the assertion. The following operators are available: Comparison Operators Operator Description noassert Check condition not active will not be used = Attribute must be equal to the given value 65

72 Test Scenarios <> Attribute must be not equal to the given value > Attribute must be greater than the given value >= Attribute must be greater than or equal to the given value < Attribute must be less than the given value <= Attribute must be less than or equal to the given value same Attribute must be unchanged (output value equal to input value) Value The Value column allows you to enter the value to be used in the an assertion. Values for text attributes can alternatively be entered in hex format, for example as x'61a5'. The hex value must be given in the target platform encoding because for hex values, no character conversion is performed. Label The Label column allows you to enter a descriptive name for the assertion which is displayed in the Result Report. When creating a check condition, it defaults to "Check for attribute name", but you can change it to a more descriptive text. Failure Message The Failure Message column allows you to enter a descriptive name for a check condition failure which is displayed when the check condition fails (does not evaluate as expected). When creating a check condition, it defaults to "Check for attribute name failed", but you can change it to a more descriptive text. Information The Information tab allows you to change the Alias Name and Description of the test scenario. Additionally, details are displayed about the test case selected in the test case list on the left. You can see the name, program, CSECT, alias, description, interface name, and structure for the interface on which the test case is based. The interface and structure(s) can be quickly opened by clicking on the listed interface or structure name. Adding A New Test Case If more test cases are required to define the test scenario, additional test cases can be added by rightclicking in the test case list area on the left and selecting Add test case... from the context menu, or by clicking on the Add test case... icon above the Test cases listing. The Add Test Case wizard 66

73 Topaz for Total Test User Guide opens, allowing you to select the interface for the new test case. The selected interface is also used as input for the Test case name field. If so desired, change the Test case name. Add the name of the program to be tested in the Target field. Alternatively, you can use an existing test case as a base copy for the new test case. To do so, in the test case list, right-click the test case name you want to copy, then select Duplicate test case... from the context menu. Enter the new name and click OK. Modifying Test Cases You can change the name of a test case by right-clicking on the test case in the test case list, then selecting Rename from the context menu. Enter the new name, then click OK. You can remove a test case from the test scenario by right-clicking on the test case in the test case list, then selecting Delete from the context menu. Confirm the deletion by clicking OK. Note that all data entered for the test case will be deleted with the test case. You can re-arrange the order of the test cases by simply moving them up or down using your mouse pointer. Note that re-arranging test cases can lead to orphaned assignation references, if a referenced test case is moved after the test case that references it. Running Test Scenarios In order to run a test scenario, right-click the test scenario in the Project Explorer view, then select Run Test Scenario... from the context menu. If you want to run the test scenario that you are just editing, simply click on the Run Test Scenario icon in the upper right area of the editor. The Test Run dialog opens, allowing you to set the Target Environment, as well as to override the Used JCL Template, Execution Options, Test Aids, and Code Coverage options for this run. For details on these please refer to General Settings. Click OK to start the test run. The test run will be started and monitored. After the execution has finished, the results will be analyzed and displayed in the Result Report. While the test is being executed, you can click Run In Background to close the dialog. The test will continue to be executed and monitored, and you can view it in the Progress view. In the meantime, you can continue to use your development environment. Once the test run has completed and the results are ready to be displayed, this will be reported in the Progress view, and you can open the Result Report with a click from there. Extracting Check Conditions This feature offers a quick and easy way to automatically create check conditions for a test case or a whole test scenario. It allows you to use the output data from a previous run to create simple check conditions from it. To do so, right-click on the archive file (usually the one in the output folder of your project), then select Test > Extract Check Conditions... from the context menu. Alternatively, if the archive is opened in the Result Editor, simply click on the Extract check conditions from this output button located in the upper right area of the Result Editor. The wizard opens, asking you to select the destination test scenario which you want the check conditions created for. The archive of a test suite may contain several test scenarios in which case you can select multiple test scenarios. By default, the original test scenarios used in the archive will be selected. If you decide to select another test scenario, it must contain the same test cases as the original. Click Clear for test scenarios that you do not want to extract check conditions from the archive. 67

74 Test Scenarios Click Next > to continue to the data selection. Here you can select the attributes for which you want check conditions to be created using the values from the output. For each test case in the test scenario, you will see a tab by which you can go through all test cases of the test scenario. Above the test case tabs, you will see a tab for each test scenario found in the archive. Once you have completed your selection of attributes, click Finish to create the check conditions. The affected test scenarios will automatically be opened for you to review them. Note that the attributes you select will overwrite any check conditions for the same attributes in the destination test cases. Check conditions for attributes that you did not select will be preserved in the destination test cases. Exporting To Excel Test scenarios can be exported to the extended Excel file format (file suffix xlsx) to allow test specifications to be made outside of Topaz for Total Test. The great advantage of using an Office application is that this enables users to contribute to the tests without having to know Topaz for Total Test or requiring to have a development environment installed. To export a test scenario, right-click on the test scenario in the Project Explorer view, then select Export from the context menu. From the list select Compuware > Topaz for Total Test > Unit Test Scenario to Excel File, then click Next >. On the next page, select the test scenario (unless selected already), then again click Next >. On the final page, select the destination folder and enter a file name, check or uncheck Open after export as you prefer, then click Finish. The Excel file will be created, and if so requested, it will be opened in your Excel application. Note: The Excel file might open in the Eclipse editor by default. This behavior can be changed in the Eclipse settings. To do so, select Window > Preferences, then navigate to General > Editors and uncheck Allow in-place system editors. In the Excel file, one Excel sheet is created for each test case, and one for the check conditions of each test case. The order and names of data sheets correspond with the order and names of the test cases in the test scenario. For the input data, the attributes are found in the columns while the data values are below. It is imported to preserve the order of the attributes because it is checked when re-importing the data. For the check conditions, the layout is very close to the one in Topaz for Total Test. Importing From Excel In order to import the test scenario data into Topaz for Total Test, right-click on the destination test scenario, then select Import from the context menu. Then select Topaz for Total Test > Unit Test Scenario to Excel File from the list, and click Next >. Now select the Excel file to be imported and click Finish. Input values and (simple) check conditions will be imported from the Excel file into the corresponding test cases of the destination test scenario. All assignations will be preserved. Note that the destination test scenario and the source Excel file must be structurally matching. During import, a check on the test cases as well as the attributes is performed. If they do not properly match the structure of the Excel file, a message is displayed. Depending on the severity of the problem, the import may optionally be continued or it can only be aborted. 68

75 Test Suites Test suites allow you to execute multiple test scenarios in one test run. This is useful for larger test runs in integration or regression tests. Test suites are intended to just loosely bundle test scenarios. The test scenarios contained in a test suite should not have any dependencies to each other. This condition is met if the test scenarios in a test suite could be run in any order and always lead to the same results. Creating A Test Suite In order to create a new test suite, select the project in which you want to create the suite, right-click to open the context menu and select New > Suite. The Test Suite wizard opens. On the first page, enter the following data (obligatory fields listed in bold): Field Description Name File name for the test suite Alias Name More descriptive name Description Detailed description If you have not selected a project before invoking the wizard, you have to select one now from the Project list below. Click Next> to open the Available Test Scenarios dialog. To add test scenario(s) to this test suite, select one ore more test scenarios from the Available list on the left and click on the arrow in the middle to copy them to the Selected list on the right. If you had selected one or more test scenarios before invoking the wizard, they will already have been added to the test suite. Note: If the test scenario is an IMS BMP test scenario, then there can only be one test case per scenario and suite. Click Finish, then the newly created test suite will open in the editor. Or optionally, click Next > instead to proceed to the final page where you can select another destination folder for the test suite. Editing Test Suites You can edit a test suite by opening it in the editor. You can now add or remove test scenarios just like when you created it. Running Test Suites In order to run a test suite, right-click the test suite in the Project Explorer view, then select Run Test Suite from the context menu. If you want to run the test suite that is just opened in the editor, simply click on the Run this Test Suite icon in the upper right area of the editor. 69

76 Test Suites Besides from executing multiple test scenarios in one test run, running a test suite does not differ from running a single test scenario with one exception: it is possible to select multiple test suites in the Project Explorer and start the test run for all of them with one click. If you start multiple test suites, they are always run in background. You are informed about their progress and completion in the Progress view. 70

77 Stubs Stubs are very useful for simulation of programs or functions which are not yet implemented or fully functional isolated unit testing without unwanted side effects from sub-programs emulation of environment data that are costly to establish emulation of error conditions that are too complex to provoke These problems can be solved by assigning and using stubs in test scenarios. A stub simulates a program call by returning a given set of return values. The actual program is not called, but for the caller, this is transparent which means, it cannot see a difference. With Topaz for Total Test, it is not necessary to modify the called program, neither is a special linkage step required. The simulation is performed at runtime. A stub affects those test cases that include it, but will have no effect on other test cases of the test scenario. Optionally, stubs can be switched off during runtime without having to remove them from the test scenario. Stub Creation During Xpediter Debug Session When debugging a COBOL Batch program you can request that Topaz for Total Test automatically create stubs from the captured debug session. Refer to Context Menus from Xpediter Debug Session for more details. Creating A New Stub In order to manually create a new test stub, select the project in which you want to create the stub, right- click to open the context menu and select New > Stub (or from the File menu, select New > Other > Topaz for Total Test > Stub). The Test Stub wizard opens. On the first page, enter the following data (obligatory fields listed in bold): Field Description Name File name for the test stub Alias Name More descriptive name Type Entry Select the stub type, either: Program, QSAM Read, QSAM Write VSAM KSDS Read, VSAM KSDS Write VSAM ESDS Read, VSAM ESDS Write VSAM RRDS Read, VSAM RRDS Write. Check the Entry Assertion checkbox to enable sub-program assertions when Type is 71

78 Stubs Assertion Program. Sub-program assertions validate all of the data that is being passed into a sub-program call just like a Write Assertion validates all the data that is passed into a Write I/O call. Subprogram Specify the name of the sub-program if the Stub Type is Program. File DD Name Specify the file name (DD name) of the Read or Write file for the stub. Description Detailed description If you have not selected a Topaz for Total Test project before invoking the wizard, a list of Topaz for Total Test projects displays in the Project list at the bottom of the wizard pages. You must select a project to continue to the next wizard page. If you have selected an appropriate interface for the stub type before invoking the wizard, this interface will automatically be used for the test stub and you can just click Finish to create the stub. Click Next > to continue to the next page where you select an interface for the stub. Only interfaces that match the stub type are included in the selection list. Click Finish to create the stub. Or optionally, click Next > instead to proceed to the final page where you can select another destination folder for the stub and enter another name for the stub. If there are no interfaces of the appropriate type then you need to create an interface of the correct type in the Topaz for Total Test project or change to a Topaz for Total Test project that contains an interface of the correct type. After completing the wizard, the newly created test stub opens in the Test Stub Editor where you can specify the simulation data to be used by the stub. The name listed in the Simulation Data tab's Description column for the stub was derived from the interface name. Note: Db2 and IMS stub types can only be created using the Generate Unit Test feature during an Xpediter Debugging Session. Refer to Generate DB2 stubs and Generate IMS stubs for details. Editing Stubs Editing a stub means specifying the simulation data to be used when the stub target is invoked during the execution of a test case. A stub only has two tabs: Simulation Data for entering the values to be returned by the stub. Information to change alias name and description and obtain information about the interface of a test case. Simulation Data The Simulation Data tab allows you to enter the values to be returned when the stub target is invoked during the execution of a test case that includes the stub. The name listed in the Description column for the stub was derived from the interface name for which it was created. The Key column only appears for keyed VSAM RRDS stubs and displays the relative record number (RRDS) for the selected record. The records will be automatically sorted in key order when the Stub 72

79 Topaz for Total Test User Guide definition is saved. You can also explicitly sort the records by using the choose not to include a key, the records will be read sequentially. Key Sort Button. If you To update a key value, right click in the Description column and select Update Key. In the dialog specify the new Key value and click OK. Entering stub data values is basically identical with entering data for a test case. However, opposite to the attributes in test cases, the attributes used for entering simulation data values for stubs are the output (with usage out or in/out) attributes of an interface. When a stub is active, the input data it receives are not evaluated in any way. Instead, just the given set of simulation data is returned to the caller. If you modify the simulation data of a stub, the change will affect all test cases that include this stub. Therefore, subsequent execution of test scenarios using the stub will be performed with the new simulation data. To add Simulation Data, right click in the Description column and select Add Simulation Data..., or click on the icon above the Description column, specify the new name and click OK. Then just add new values as needed. When multiple names are listed, the sequence number for each name indicates the sequence order of the simulation data. Use drag and drop or Move Up and Move Down to change the sequence order. Hover over the name to view the initial sequence number of the entry. Other right-click options in the Description column are: Duplicate Simulation Data... Rename Delete Move Up Move Down Update Key Structure The Structure column displays a list of the structures used as input in the test case, as specified in the interface definition, including their substructures and attributes. Please refer to General Editor Behavior for a detailed description of all options provided to navigate in this list. If the attribute is contained in an array, by default, the first array element will be used for the input value. If you need an input value for another array element, right-click on the array, then select Add array element from the context menu. A new array element will be added to the array which you can now enter input values for. Alternatively, you can right-click on an array element and select Duplicate array element from the context menu to create a copy of the array element and add it to the array. In order to remove an array element, right-click on the array element, then select Remove array element from the context menu. Other right-click options for the Structure elements include: Redefine element Replace element Insert new field after element Delete element 73

80 Stubs Revert to original interface Location The Location column describes the position values for the specific field. Comparison When the stub type is WRITE, for example, QSAM Write, the Simulation Data includes three additional columns, Comparison, Label, and Failure Message. They allow you to specify write assertions (check conditions) for the stub. These check conditions can be created for WRITE operations and Topaz for Total Test will compare the value of the stub to the intercepted write operation in the program. Note: Write assertions are only executed when WRITE Stubs are assigned to a test case and stubs are enabled for test execution (see Running Test Scenarios). Note: When generating I/O stubs from an Xpediter Debug Session and you check the Create I/O assert conditions for writes checkbox, the generated WRITE stubs will include generated write check conditions. Refer to Generate Unit Test for a description of the context menu. The Comparison column allows you to specify the comparison operator used for the check condition. The following operators are available: Comparison Operators Operator Description noassert Check condition not active will not be used = Attribute must be equal to the given value <> Attribute must be not equal to the given value > Attribute must be greater than the given value >= Attribute must be greater than or equal to the given value < Attribute must be less than the given value <= Attribute must be less than or equal to the given value 74

81 Topaz for Total Test User Guide same Attribute must be unchanged (output value equal to input value) Expected Value The Expected Value column allows you to enter the value to be used in the assertion or check condition. Values for text attributes can alternatively be entered in hex format, for example as x'61a5'. The hex value must be given in the target platform encoding because for hex values, no character conversion is performed. Label The Label column allows you to enter a descriptive name for the check condition which is displayed in the Result Report. When creating a check condition, it defaults to "Check for attribute name", but you can change it to a more descriptive text. Failure Message The Failure Message column allows you to enter a descriptive name for a check condition failure which is displayed when the check condition fails (does not evaluate as expected). When creating a check condition, it defaults to "Check for attribute name failed", but you can change it to a more descriptive text. Db2 Simulation Data The Simulation Data tab for Db2 stubs allows you to enter the values to be returned when the stub target is invoked during the execution of a test case that includes the stub. The name listed in the Description column for the stub identifies the SQL statement type, for example Insert, Select, etc.. To view the captured SQL Statement click on the SQL Statement or SQL Cursor button on the top of the window. It expands the SQL Statement partition of the Editor area and displays the SQL statement in a scrollable area. If the SQL Statement allows editing, the area below is filled with SQL Selection Criteria. You may change values in the Value and NULL Indicator columns where applicable. Doubleclick the displayed value of the Value field to edit the value. When you double-click the value in the NULL Indicator field, click on the checkbox to toggle between NULL and Non-NULL. Changing the field value from Non-NULL to NULL will automatically remove the corresponding value in the Value column. The original value will be restored when toggling the NULL field back to Non-NULL. The SQL Execution partition displays the SQL Result, Value, and NULL Indicator columns, with the Return Code and Values columns at the bottom. The SQLSTATE and SQLCODE entries can be edited to force the program into the error handling code. For additional information on the SQL Result and SQL error codes, refer to the IBM Db2 for z/os documentation. The only right-click option in the Description column is: Rename IMS Simulation Data The Simulation Data tab for IMS stubs allows you to enter the values to be returned when the stub target is invoked during the execution of a test case that includes the stub. 75

82 Stubs The name listed in the Description column for the stub identifies the IMS call type, for example GU (Get Unique), ISRT (Insert), etc.. The data area to the right displays the information for the selected call and is divided into these partitions, where applicable: IOAREA SSA PCB AIB The only right-click option in the Description column is: Rename Sub-program Entry Assertion When during unit test generation you checked Create assert condition for data passed into subprograms, the Simulation Data presents in two tabbed displays. The tabs Entry Assertions and Returned Simulation Data are located above the simulation data area. When opening a stub with generated entry assertions in the stub editor, the Entry Assertion displays first. Sub-program assertions validate all of the data that is being passed into a sub-program call just like a Write Assertion validates all the data that is passed into a Write I/O call. Click on Returned Simulation Data to show the returned simulation data. The returned simulation data is the same as the simulation data generated when you didn't check Create assert condition for data passed into sub-programs during unit test generation. Information The Information tab displays the Stub Name and Type information and allows you to change the Alias Name and Description of the stub. To quickly open the interface, click on the interface name listed under Current Simulation Record Interface. Exporting Stub To Excel Data stubs can be exported to the extended Excel file format (file suffix xlsx) to allow stub data modifications to be made outside of Topaz for Total Test. The advantages of using an Office application are: Familiarity with Office applications. Ability to use existing test data. Easy way to modify multiple data at once. Import test data based on modified structure. Note: The Excel Import / Export feature in Topaz for Total Test is not supported for DB2 SQL Stubs. To export a data stub, right-click on the stub in the Project Explorer view, then select Export from the context menu. From the list select Compuware > Topaz for Total Test > Unit Test Stub to Excel File, then click Next >. On the next page, select the stub (unless selected already), then again click Next >. On the final page, select the destination folder and enter a file name, check or uncheck Open after 76

83 Topaz for Total Test User Guide export as you prefer, then click Finish. The Excel file will be created, and if so requested, it will be opened in your Excel application. Note: The Excel file might open in the Eclipse editor by default. This behavior can be changed in the Eclipse settings. To do so, select Window > Preferences, then navigate to General > Editors and uncheck Allow in-place system editors. One Excel file will be generated for each stub. When an Excel file contains multiple data sheets, the TotalTestInterfaceCatalog sheet contains mapping information between the sheet and interface file. The order and names of data sheets correspond with the order and names of the records in the data stub. For the input data, the attributes are found in the columns while the data values are below. It is imported to preserve the order of the attributes because it is checked when re-importing the data. For the check conditions, the layout is very close to the one in Topaz for Total Test. Column Naming in Data Sheets The column names identify the Interface and the field names, including any sub-level field, for example, column name EMPLOYEE_WORK_AREA.WA_SALES_EMPLOYEE_DATA.WA_SALES_SALARY identifies: 01 Level: EMPLOYEE_WORK_AREA Sub-level: WA_SALES_EMPLOYEE_DATA Field: WA_SALES_SALARY The screenshot shows a sample structure in Topaz for Total Test. 77

84 Stubs Here is the corresponding COBOL layout: 01 EMPLOYEE-WORK-AREA. 05 WA-EMP-NUM PIC 9(5). 05 WA-EMP-TYPE PIC X. 88 HOURLY VALUE 'H'. 88 SALES VALUE 'S'. 88 MANAGEMENT VALUE 'M'. 05 WA-EMP-REGION PIC NORTH VALUE SOUTH VALUE EAST VALUE 3. 78

85 Topaz for Total Test User Guide 88 WEST VALUE WA-EMP-NAME PIC X(15). 05 WA-EMP-ADDRESS. 10 WA-EMP-STREET PIC X(15). 10 WA-EMP-CITY PIC X(8). 10 WA-EMP-STATE PIC XX. 10 WA-EMP-ZIP PIC X(9). 05 WA-HOURLY-EMPLOYEE-DATA. 10 WA-EMP-HOURS PIC 9(2). 10 WA-EMP-RATE PIC 9(3)V99 COMP FILLER PIC X(8). 05 WA-SALES-EMPLOYEE-DATA REDEFINES WA-HOURLY-EMPLOYEE-DATA. 10 WA-SALES-SALARY PIC 9(5)V99 COMP WA-SALES-AMOUNT PIC 9(5)V FILLER PIC X(2). 05 WA-MGMT-EMPLOYEE-DATA REDEFINES WA-SALES-EMPLOYEE-DATA. 10 WA-MGMT-SALARY PIC 9(5)V99 COMP FILLER PIC X(9). 05 WA-EMP-HIRE-DATE. 10 WA-EMP-HIRE-YY PIC 9(2). 10 WA-EMP-HIRE-MM PIC 9(2). 10 WA-EMP-HIRE-DD PIC 9(2). 05 FILLER PIC X(5). When changing the Interface you must also reflect the change in the column name(s). Steps to Update the REDEFINED Active Status Users decide which REDEFINED field is active and used during the execution. The active status of the REDEFINED fields is saved as an Excel comment when exporting data. Users can update the Excel cell comment to choose the REDEFINED field. How to Use the Excel Catalog Sheet to Define the Interface The first Excel sheet generated (TotalTestInterfaceCatalog) contains mapping information between the sheet and interface file. The column named Sheet identifies the names of the other sheet(s) included in the file and the column named TotalTestInterface identifies the corresponding Interface for each sheet. You should update the catalog data when adding or deleting data sheets or when changing the corresponding interface file. Importing Stub From Excel Note: The Excel Import / Export feature in Topaz for Total Test is not supported for DB2 SQL Stubs. In order to import the stub data into Topaz for Total Test, right-click on the destination stub, then select Import from the context menu. Then select Topaz for Total Test > Unit Test Stub to Excel File from the list, and click Next >. Now select the Excel file to be imported and click Finish. Input values and (simple) check conditions will be imported from the Excel file into the corresponding records of the destination stub. All assignations will be preserved. 79

86 Stubs Select the interface for Each Sheet When Importing Data When selecting the Excel data file during import, the interface mapping table displays in the same page. Clicking the data from the interface mapping table, pops up the Interface Selection dialog where you can select a different Interface file for each Excel sheet. Note that the structure of the interface and the source Excel file must be structurally matching. During import, a check on the test cases as well as the attributes is performed. If they do not properly match the structure of the Excel file, a message is displayed. Depending on the severity of the problem, the import may optionally be continued or it can only be aborted. 80

87 Run Test Dialog In order to run or execute a test scenario or test suite, right-click the test scenario or test suite in the Project Explorer view, then select Run Test Scenario... from the context menu. Or, if you want to run the test scenario that is open in the editor, simply click on the right area of the editor. The Test Run dialog opens. Run Test Scenario icon in the upper 81

88 Run Test Dialog Host Connection Execution of the unit tests occurs on the mainframe, the Target Environment. Select the host connection where you want to execute the Test Suite/Scenario from the list of Hosts in the Target Environment drop down list. If the desired host connection isn't listed, click on the Configure button to define a new Host Communication Interface (HCI) connection. Refer to the Host Explorer Connections help topic in 82

89 Topaz for Total Test User Guide the Topaz Workbench Host Explorer User Guide for detailed instructions on how to add a new connection. JCL template Select the JCL to use for the test execution from the Used JCL template view. The view lists all JCL files stored for the current project. Make sure your steplib DD points at your site's Enterprise Common Components (ECC) Release or newer load library (CPWR.MLCXnnn.SLCXLOAD), for example hlq.cpwr.mlcx170.slcxload. For details please refer to the Enterprise Common Components Installation and Configuration Guide for Release or newer. The variable ${TOTALTEST_JOBCARD} in the JCL will load the Topaz for Total Test default jobcard specified in Window > Preferences > Compuware > Topaz for Total Test. Use JCL based on the Runner.jcl for tests that do not use live Db2 or live IMS BMP. If your test case includes live DB2, use JCL based on the RunnerDB2.jcl template. If your test case includes live IMS BMP, use JCL based on the RunnerBmp.jcl template. If your test case includes both live DB2 and live IMS BMP, use JCL based on the RunnerBmpDB2.jcl template. Execution Options The defaults for the Execution Options have been set in Project_Properties. Use Stubs With the aid of test stubs, instead of actually calling a program, a simulation is performed at runtime, acting as if the called program processed the call and returns a pre-defined set of return data. This options controls if stubs defined in a test scenario should be used or not. Check this option to use (activate) all stubs defined in a test scenario during the test which means that the stubbed interfaces will be simulated as defined. Uncheck this option to specify that stubbing will not be used during the test run, even if the scenario contains stubs. This enables you to switch off stubbing without having to remove all stubs from the test scenarios. Stubbing is an optional feature. Refer to Stubs for more information on stubs. Delete Temporary Files Some temporary files are created during the execution of a test on the client and on the target platform. These files are needed to collect the scenario specification into an archive and to transfer the input and output data between client and target platform. Usually, these temporary files are not needed any more after the execution has completed. However, if you want to re-submit the test job for use with a debugger, the temporary files are required to resume the test execution. In this case, switch this option to off to maintain the temporary files after execution. Check this option to specify that all temporary files will be deleted once the execution has been completed. Uncheck this option to specify that all temporary files will be kept. Test Aids The defaults for the Test Aids have been set in Project_Properties. 83

90 Run Test Dialog Initialization with byte value The initialization of empty attributes with specific values is a test aid which helps to identify problems caused by the use of uninitialized data by the test target. This may apply to the use of data not passed in a call as well as missing return values. Such problems are often difficult to identify because during testing, there may be no difference between uninitialized and initialized data. Therefore, problems usually occur later, maybe even in production, and are usually not easy to reproduce. With the help of this test aid, uninitialized data can be distinguished from initialized data. In order to activate this option, in Initialization of empty fields enter a string to be used for the initialization of all fields with no defined value. The string must be in hexadecimal representation. Choose a value that is unique so it differs significantly from common initialization values. A good and proven example is x'ee' as it largely improbable to appear within real data. Check this option to specify to use this option. Uncheck this option to specify that all fields without defined values will be initialized with a value of zero for numeric data types, and with a value of spaces for character data types. Note that this "clean" initialization renders it impossible to identify initialization problems. Repeat Execution The repeated execution of test cases is a test aid which helps to identify problems caused by missing or inappropriate internal initialization of data in the test target. Such problems are often overlooked during the test phase, but occur in production due to optimization like resident loading and reusing programs. Because these problems are often difficult to identify, Topaz for Total Test provides this option to verify that a repeated execution does not affect a programs functionality. Please note that not all interfaces may support repeated execution by design. Note: This check option is not available if the selected scenario or test suite includes live IMS processing. Check this option to repeat the execution as many times as specified with the Repeated Execution Count option in Project_Properties. Uncheck this option to specify that each test cases will be executed just once. Enable Code Coverage Xpediter/Code Coverage must be installed at your site for this feature. Refer to the Topaz Workbench Code Coverage/Eclipse User Guide. Check this option to enable Code Coverage for this unit test execution. Uncheck this option to NOT enable Code Coverage for this unit test execution. If unchecked, the following Code Coverage items are greyed out. Repository DSN Specify the dataset where Xpediter will write coverage information. This dataset will later be fed into the Code Coverage reporting system to report on program invocations that occurred during the test session. Members with like systems, names, and compile date and times will automatically be merged together. If the desired repository dataset isn't listed in the drop down list, click on the Browse button to select a Code Coverage repository dataset. System name Specify or select a system name for this test. The System name drop down contains a list of recently used system names. Code Coverage keeps statistics by programs separated by system name. This 84

91 Topaz for Total Test User Guide name can be any user-specified name. If the desired system name isn't listed in the drop down list, click on the Browse button to select one from the available system names in the repository. Test ID Specify or select a test identification to be added to the Code Coverage test. This information can be useful when reviewing test reports from Xpediter/Code Coverage. If the Test ID isn't listed in the drop down list, click on the Browse button to select one from the available Test IDs in the repository. Main Program Type Select the main executable program (this is the program specified on the 'EXEC PGM=' JCL statement in runner*.jcl) for Code Coverage: Live DB2 - IKJEFT01 when the main program is IKJEFT01 or IKJEFT1B for live Db2 processing Live IMS - DFSRRC00 when the main program is DFSRRC00 for live IMS processing TOTALTEST - TTTRUNNR when the main program is TTTRUNNR. Clear existing statistics before running the test The "Clear existing statistics before running the test" checkbox indicates if you want the repository statistics cleared before the test is run. Check this option to clear the repository statistics cleared before the test is run. Uncheck this option to keep any existing repository statistics. Display a report after running the test The "Display a report after running the test" will cause the Code Coverage report to be displayed after the test is run. Check this option to display the Code Coverage report after the test is run. After the test run is complete, a prompt will display asking if you want to switch to the Code Coverage perspective. Select Yes to display the Code Coverage report. Uncheck this option and the Code Coverage report will not be displayed. OK Click OK to execute the test case. The progress bar shows the test assets moving to the Host that will execute the Test Suite/Scenario. A notification in the Console informs that the Test Suite/Scenario has completed and the Result Report opens in the Editor area. CANCEL Click Cancel to cancel the test run request and close the Test Run dialog. 85

92 Run Test Dialog 86

93 Command Line Interface (CLI) The CLI tools allow you to run test suites in batch with no interactive client required and/or from an integration server or within a build tool (like Hudson or Jenkins). Prerequisites The Topaz for Total Test CLI components are installed as part of the Topaz Workbench CLI installation. A 64-bit Java Runtime Environment (JRE) is required for the Topaz for Total Test CLI. Refer to the Topaz Workbench Installation Guide on how to install the Topaz Workbench CLI. The test suite must have been defined in the Topaz for Total Test Eclipse client. The command line tools expect a complete Topaz for Total Test project structure. Usage You have the option to run a complete test project with one command, runtest, or in individual steps. Complete End to End Run Use the runtest command to perform a complete end to end run of a Topaz for Total Test test suite or test scenario. See Test_Runner for more information. For example: TotalTestCLI.bat -cmd=runtest -h=mfhost -pt= u=tsoab71 -pw=akznxy -p=c:\users\username\compuware\workbench\workspace\totaltestproject -ts=cwxtcob_scenario.testscenario -te=1047 -j=runner.jcl This example will execute all the test cases in "CWXTCOB_Scenario.testscenario" for project "TotalTestProject" on host "mfhost" using port "10239" and log on information UserID "tsoab71" with password "AKznXY" and submit "Runner.JCL". Execute Individual Steps If you want to execute the individual steps, one at a time, to run a test, execute them in this order: 1. Use the build command to create the binary input files for the test case(s) using the Binary Builder. See Binary Builder for more information. 2. Use the upload command to upload the binary files created by the Binary Builder to the mainframe. See File Transfer for more information. 3. Use the submit command to submit the JCL to execute the Topaz for Total Test Runner on the mainframe. Note that the binary files need to be uploaded before submitting the JCL to execute the Test Runner. See JCL Submit for more information. 4. Use the download command to retrieve the result from the mainframe Topaz for Total Test Test Runner execution. See File Transfer for more information. 5. Use the parse command to unpack the binary test results file created by the Topaz for Total Test Runner. See Binary Parser for more information. 6. Use the resultcheck command to analyze the test results against the test check conditions. The result checker will output test archive files containing the test results. See Result Checker for more information. To run the CLI components, use this command syntax: TotalTestCLI.bat -cmd=<command> options 87

94 Command Line Interface (CLI) The batch script is in the directory where the Topaz Workbench CLI was installed. To execute the script, the configuration directory must be writable. CLI Commands The following table lists the valid commands to invoke them from the command line. For options see the parameters explained below for each CLI component. CLI Commands Command Component Description build Binary Builder Parses a test suite or test scenario and creates the required binary files. upload File Transfer Uploads the parsed local binary files to the target host system. submit JCL Submit Submit JCL to run a test. download File Transfer Downloads the test results from the target host to a local file. parse Binary Parser Parses the local binary result file and updates the archive (history). resultscheck Result Checker Parses the updated archive (history), applying the check conditions to the output data and updating the archive accordingly. runtest Test Runner Performs all the above commands. All options have the following format: parameter=value If the value contains spaces, the entire option must be enclosed in quotes, for example: -t=c:\totaltestproject\unit Test\Suites\SimpleTest.testsuite To set the logging level of the CLI tools logging output, see the Debug Options below. Binary Builder The Binary Builder component parses a test suite or test scenario and creates the binary input file (containing the binary input data), and the reference file (containing the test case references). The following table lists the input parameters that must be provided. Binary Builder Input Parameters Parameter Abbr. Description 88

95 Topaz for Total Test User Guide -testsuite -t Test suite or test scenario name, if project is specified. Otherwise must be the full path name to the test suite or test scenario. -targetencoding -te The character encoding (charset) used on the target platform. Default is '1047'. The following table lists the output parameters that must be provided. Binary Builder Output Parameters Parameter Abbr. Description -archive -a The archive created containing all test data. See Optional Parameters for more information. -binput -bin The binary input data file. See Optional Parameters for more information. -binref -brf The binary reference file. See Optional Parameters for more information. The following table lists optional output parameters. Binary Builder Optional Output Parameters ParameterAbbr. Description -project -p The Topaz for Total Test project folder. If specified, the archive, binary input file and reference file do not need to be specified. The files will be placed on the associated project folders. File Transfer File Transfer transfers binary files to and from the target host. The following table lists the input parameters that must be provided: File Transfer Input Parameter Parameter Abbr. Description 89

96 Command Line Interface (CLI) -command -cmd The command to be executed. Must be either upload or download. The following table lists the common parameters that must be provided: File Transfer Common Parameters Parameter Abbr. Description -host -h The host name or IP address of the target host. -port -pt The port the host is listening on. -user -u The name of the user (userid) to connect to the host with. -password -pw The password associated with the userid. -targetencoding -te The character encoding (charset) used on the target platform. Default is '1047'. The following table lists the parameters that must be provided for the upload command: File Transfer Upload Parameters Parameter Abbr. Description -bininp -bin The binary input data file produced by the builder. -binref -brf The binary reference file produced by the builder. -dsnhlq -hlq Optional high level qualifier to be used when allocating datasets. The following table lists the parameters that must be provided for the download command: File Transfer Download Parameters Parameter Abbr. Description -binres -brs The binary result file to be written to. 90

97 Topaz for Total Test User Guide -binresdsn -brsdsn The BINRES dataset name on the target host to be downloaded. JCL Submit The JCL submit component submits JCL to run the test(s) on the target host. The JCL can reside on the local files system (use parameter -j) or as a dataset on the target host (use parameter -jdsn). The following table lists the parameters that must be provided: JCL Submit Parameters Parameter Abbr. Description -host -h The host name or IP address of the target host. -port -pt The port the host is listening on. -user -u The name of the user (userid) to connect to the host with. -password -pw The password associated with the userid. -targetencoding -te The character encoding (charset) used on the target platform. The default is bininpdsn -bindsn The BININP dataset name on the target host, to use for the test run. This is only required if the JCL requires substitution. -binrefdsn -brfdsn The BINREF dataset name on the target host, to use for the test run. This is only required if the JCL requires substitution. -binresdsn -brsdsn The BINRES dataset name on the target host, to use for the test run. This is only required if the JCL requires substitution. -ccrepository -ccrepo The name of the Code Coverage repository dataset. Must be specified 91

98 Command Line Interface (CLI) to enable Code Coverage. -ccsystem -ccsys Code Coverage system. If not specified with '-ccrepo', defaults to the test suite or test scenario name. -cctestid -cctid Code Coverage test id. If not specified with '-ccrepo', defaults to the test suite or test scenario name. name. Should be used if the Topaz for -ccstepname -ccstep Specifies the Topaz for Total Test step Total Test step is contained in a cataloged procedure. -ccprogramtype -cctype Specifies the main executable program (this is the program specified on the 'EXEC PGM=' JCL statement in runner*.jcl) for Code Coverage. Specify: -cctype=db2 when the main program is IKJEFT01 or IKJEFT1B for live Db2 -cctype=totaltest when the main program is TTTRUNNR -cctype=ims when the main program is DFSRRC00 for live IMS -ccclearstats -ccclear Specifies whether the Code Coverage repository statistics should be cleared before running the test. Valid values are 'true' or 'false'. The default value is 'true'. The following table lists the parameter for submitting JCL from local host: JCL Submit Parameter from Local Host Parameter Abbr. Description -jcl -j The file name containing the JCL to submit. The following table lists the parameter for submitting JCL from a z/os dataset: JCL Submit Parameter from z/os Dataset 92

99 Topaz for Total Test User Guide Parameter Abbr. Description -jcldsn -jdsn The name of a dataset containing the JCL to submit. Note: -jcl and -jcldsn cannot both be used in the same statement. Specify either -jcl or - jcldsn. The following table lists the optional parameters that can be specified: JCL Submit Optional Parameters Parameter Abbr. Description -wait -w Indicates if component waits for test completion. Must be true or false. Defaults to true. -maxwait -mw The number of minutes to wait for the test to complete, The default is 20 minutes. Binary Parser The Binary Parser component parses the binary result file from an executed test and updates the archive file. The following table lists the input parameters that must be provided. Binary Parser Input Parameters Parameter Abbr. Description -binresult -brs The binary result data file which was the output of the test execution on the target platform. -targetencoding -te The character encoding (charset) used on the target platform. Default is '1047'. -archive -a The archive (history) created by the Builder which will be updated. There are no explicit output parameters. The Parser output data will be saved in the updated archive file. Result Checker The Result Checker component parses the archive, applying the check conditions to the output data and updating the archive accordingly. In addition to that, the Result Checker will create a separate 93

100 Command Line Interface (CLI) results file and report file. For details about the contents of these file, please refer to the section about Project Structure in this document. The following table lists the input parameters that must be provided. Result Checker Input Parameters Parameter Abbr. Description -archive -a The archive from the Parser which will be updated (unless the output parameter is used see below). The following table lists the optional input parameter. Result Checker Optional Input Parameters Parameter Abbr. Description -targetencoding -te The character encoding (charset) used on the target platform. Defaults to the encoding stored in the archive. The following table lists the output parameters that must be provided, if these output types are required: Result Checker Output Parameters Parameter Abbr. Description -result -rs The name of the XML result fle to be created. (Not required when project parameter is provided.) -noresult If specified with the -p option, no result file will be created. -report -rp The name of the HTML result report to be created. (Not required when project parameter is provided.) -noreport If specified with the -p option, no report file will be created. -nojunit If specified with the -p option, no JUnit file will be created. -nosonar If specified with the -p option, no Sonar file will be created. -externaltoolsworkspace -etws Specifies the workspace of an external tool. This argument requires the 'postruncommands' argument be specified. -postruncommands -prc Specifies the commands to be run after test completion. Currently only 'CopyJUnit' and 'CopySonar' are supported. If 94

101 Topaz for Total Test User Guide both are specified, they should be separated by a comma. This command requires the 'externaltoolsworkspace' argument be specified. 'CopyJUnit' will copy the JUnit results to the directory TTTJUnit, in the location specified by the external tools workspace argument. 'CopySonar' will copy the Sonar results to the directory TTTSonar, in the location specified by the external tools workspace argument. The following table lists optional output parameters. Result Checker Optional Output Parameters Parameter Abbr. Description -outputarchive -oar A new archive to be created instead of updating the input archive. Default is to update the input archive. -project -p Path to the project folder. If provided, the result and report parameters can be omitted, and their file names will be automatically derived. This parameter has no default. -fileencoding -fe The character encoding to be used for output files. Defaults to UTF- 8. -save -s Save output or not. Set to TRUE to write output, or FALSE for no output. When set to FALSE, no files will be created and the archive will not be updated. Defaults to TRUE. Test Runner The Run Test component com.compuare.unittestrun.jar allows for the complete end to end run of a Topaz for Total Test test suite or test scenario. The following table lists the parameters that must be provided: Test Runner Parameters Parameter Abbr. Description -host -h The host name or IP address of the target host. 95

102 Command Line Interface (CLI) -port -pt The port the host is listening on. -user -u The name of the user (UserID) to connect to the host with. -password -pw The password associated with the user. -project -p The Total Test project folder. -testsuite -ts The name of a test suite from the Suites folder or the name of a test scenario from the Scenario folder. test scenarios/suites names to be run. -testsuitelist -tsl Specifies a comma delimted list of Test scenarios/suites names can contain the wildcard characters asterisk (*) to indicate any characters or a question (?) to indicate a single character. 'All_Scenarios' can be used to run all scenarios. 'All_Suites' can be used to run all test suites. -jcl -j The name of a JCL file from the JCL folder. -jcldsn -jdsn The name of a dataset containing the JCL to submit. -targetencoding -te The character encoding (charset) used on the target platform. Default is '1047'. -noreport If specified with the -p option, no report file will be created. -noresult If specified with the -p option, no result file will be created. -nojunit If specified with the -p option, no JUnit file will be created. -nosonar If specified with the -p option, no Sonar file will be created. -ccrepository -ccrepo The name of the Code Coverage repository dataset. Must be specified 96

103 Topaz for Total Test User Guide to enable Code Coverage. -ccsystem -ccsys Code Coverage system. If not specified with '-ccrepo', defaults to the test suite or test scenario name. -cctestid -cctid Code Coverage test id. If not specified with '-ccrepo', defaults to the test suite or test scenario name. name. Should be used if the Topaz for -ccstepname -ccstep Specifies the Topaz for Total Test step Total Test step is contained in a cataloged procedure. -ccprogramtype -cctype Specifies the main executable program (this is the program specified on the 'EXEC PGM=' JCL statement in runner*.jcl) for Code Coverage. Specify: -cctype=db2 when the main program is IKJEFT01 or IKJEFT1B for live Db2 -cctype=totaltest when the main program is TTTRUNNR -cctype=ims when the main program is DFSRRC00 for live IMS -ccclearstats -ccclear Specifies whether the Code Coverage repository statistics should be cleared before running the test. Valid values are 'true' or 'false'. The default value is 'true'. -externaltoolsworkspace -etws Specifies the workspace of an external tool. This argument requires the 'postruncommands' argument be specified. -postruncommands -prc Specifies the commands to be run after test completion. Currently only 'CopyJUnit' and 'CopySonar' are supported. If both are specified, they should be separated by a comma. This command requires the 'externaltoolsworkspace' argument be specified. 'CopyJUnit' will copy the JUnit results to the directory TTTJUnit, in the location specified by the external tools workspace argument. 97

104 Command Line Interface (CLI) 'CopySonar' will copy the Sonar results to the directory TTTSonar, in the location specified by the external tools workspace argument. Note: -jcl and -jcldsn cannot both be used in the same statement. Specify either -jcl or - jcldsn. The following table lists the optional parameters that can be specified: Test Runner Optional Parameters Parameter Abbr. Description -dsnhlq -hlq High level qualifier to be used when allocating datasets. -wait -w Indicates if component waits for test completion. Must be TRUE or FALSE. The default is TRUE. -maxwait -mw The number of minutes to wait for the test to complete. The default is 20 minutes. -usestubs -us Indicates if the test should use stubs. Must be TRUE or FALSE. The default is TRUE. -repeat -r The number of time to repeat the test. The default is 0. -backuparchive -ba Indicates if the archive file should be backed up. The file will be time stamped and placed in the History project folder. Must be TRUE or FALSE. The default is TRUE. -deletetemp -dt Indicates if temporary files are to be deleted. Must be TRUE or FALSE. The default is TRUE. Debug Options The following table lists the debug parameters that can be used on by components. Debug Parameters Parameter Abbr. Description -loglevel -log The logging level. Must be INFO, WARN, DEBUG or ERROR. The default is INFO. 98

105 Topaz for Total Test User Guide -logparms -lp Indicates if the various components run parameters should be logged before the run. Must be TRUE or FALSE. The default is TRUE. 99

106 Configuring Topaz for Total Test Jenkins Plugin The batch execution of Topaz for Total Test test scenarios/test suites can be initiated from Jenkins. The Topaz for Total Test Jenkins plugin makes it easy to configure Topaz for Total Test in your CI/CD process driven by Jenkins. Topaz for Total Test is a Unit Testing Tool for Mainframe COBOL Batch Programs. Using the Topaz for Total Test Jenkins plugin allows you initiate Test Suites and associated Test Cases for your mainframe programs. The plugin communicates with a TestRunner Component that executes your program on the mainframe. The test results can then be viewed with Jenkins (and SonarQube, if installed). Prerequisites The following are required to use the Topaz for Total Test Jenkins plugin: Jenkins Jenkins Credentials Plugin Topaz Workbench CLI installed on the Jenkins server. Refer to Topaz Workbench CLI Installation in the Topaz Workbench Installation Guide for instructions. Topaz for Total Test licenses Host Communications Interface (HCI) Connection Enterprise Communication Components (ECC) which contains the Topaz for Total Test TestRunner Mainframe component. Please make a NOTE of the location where you installed Topaz CLI as you need to provide this information to Jenkins later. The test suite must have been defined in the Topaz for Total Test Eclipse client. The command line tools expect a complete Topaz for Total Test project structure. Prepare Jenkins and Install the Topaz for Total Test Jenkins Plugin Follow these steps to prepare Jenkins: 1. Install Jenkins 2. Login to Jenkins as Admin 3. Install the Topaz for Total Test Plugin from list of available plugins. 4. Select Manage Jenkins 5. On the Manage Jenkins page, click on Configure System 6. Locate the Compuware Configuration section 100

107 Topaz for Total Test User Guide 7. Enter the location where you installed the Topaz Workbench CLI into the appropriate field: "Windows Topaz Workbench CLI Home" if you are on Windows system "Linux Topaz Workbench CLI Home" if you are on a Linux system 8. Use the Add Host Connection to define a new host: In the Description field, enter a description for the host. a. In the Host:port field, enter the host name or IP address of the target host and the port the host is listening on (from Topaz for Total Test Run Test Scenario dialog) b. In the Code Page field, enter the code page to use for the host connection c. In the Read/write timeout(minutes) field, enter the timeout for read and writes to the host in minutes. A value of zero indicates no read/write timeout. 9. Click the Save button. Now Jenkins can execute Topaz CLI including the Topaz for Total Test CLI from the Topaz for Total Test Jenkins plugin. Create a Freestyle Project Follow these steps to create a Freestyle Project and name it the same as the Topaz for Total Test Project. 1. Login as Admin 2. Select New Item 101

108 Configuring Topaz for Total Test Jenkins Plugin 3. Create a new Freestyle Project 4. In the Enter an item name field, enter an appropriate project name which can be the same item name as the name of the Topaz for Total Test Project 5. Click OK. The General tab displays. 6. In the Description field, enter a comment describing the project. 7. In the Build Triggers panel, scroll down to the Add Build Step drop-down list. 8. In the Build panel, select Topaz for Total Test from the Builder drop-down list. 102

109 Topaz for Total Test User Guide 9. Click on the Code Coverage button to display the Code Coverage options. Optional, only required if Code Coverage is requested. 10. Click on the Execute Options button to display the Execute options: Use Stubs, Delete temporary files, and High level qualifier. 103

110 Configuring Topaz for Total Test Jenkins Plugin 11. Gather the information for the required fields from the Topaz for Total Test project and enter it in the fields: Host Connection: Select the desired host connection, from the drop-down list. The Manage Connections link can be used to define a new host connection. Login Credentials : Select or add your mainframe login credentials (userid and password) to connect to the host with. Test Project Folder : The Topaz for Total Test project folder path (from Topaz for Total Test Project Folder Properties). This location must be accessible to Jenkins. Read/write access is required to all files in this location. Test Scenario/Suite : Test suite or test scenario name for the specified project (from Topaz for Total Test Project Folder) or select all_scenarios from the list. JCL : The file name containing the JCL to submit (from Topaz for Total Test Run Test Scenario dialog). NOTE: If you use the Topaz for Total Test Preference option to hold the JobCard 104

111 Topaz for Total Test User Guide 12. Press SAVE. information you need to have executed the Test Suite or Test Scenario at least once in the Topaz for Total Test interface for this file to get generated. Main executable program type : Select the main executable program (this is the program specified on the 'EXEC PGM=' JCL statement in runner*.jcl) for Code Coverage: Live DB2 when the main program is IKJEFT01 or IKJEFT1B TOTALTEST when the main program is TTTRUNNR Live IMS when the main program is DFSRRC00 Code Coverage Repository : The name of the Code Coverage repository dataset. Must be specified to enable Code Coverage (from Topaz for Total Test Run Test Scenario dialog). System Name : Code Coverage system. If not specified, defaults to the test suite or test scenario name (from Topaz for Total Test Run Test Scenario dialog). Test ID : Code Coverage test id. If not specified, defaults to the test suite or test scenario name (from Topaz for Total Test Run Test Scenario dialog). Clear existing statistics before the test: Check if existing Code Coverage statistics should be deleted before running the test. Use Stubs : Check if the test should use stubs (from Topaz for Total Test Run Test Scenario dialog). Delete temporary files : Check if temporary files are to be deleted (from Topaz for Total Test Run Test Scenario dialog). High Level Qualifier : Optional high level qualifier to be used when allocating datasets during test execution (from Topaz for Total Test Preferences). Create a Pipeline Project Follow these steps to create a Pipeline Project and name it the same as the Topaz for Total Test Project. 1. Login as Admin 2. Select New Item 3. Create a new Pipeline Project 4. In the Enter an item name field, enter an appropriate project name which can be the same item name as the name of the Topaz for Total Test Project 5. Click OK. The General tab displays. 6. Copy the following information and paste it into the Script field: stage("run Topaz for Total Test") { // This stage uses the Topaz for Total Test Plugin (general post build step) to run a Topaz for Total Test Unit Test node{ // This line pulls the Topaz for Total Test Unit Test Results in Junit format into Jenkins } } junit 'TTTUnit/*.xml' 105

112 Configuring Topaz for Total Test Jenkins Plugin 7. Press SAVE 8. Select Configure 9. Select Pipeline Syntax 10. Select step: General Build Step from the Sample Step drop-down list. 11. Select Topaz for Total Test from the Build Step drop-down list. 12. Click on the Code Coverage button to display the Code Coverage options. Optional, only required if Code Coverage is requested. 13. Click on the Execute Options button to display the Execute options: Use Stubs, Delete temporary files, and High level qualifier. 14. Gather the information for the required fields from the Topaz for Total Test project and enter it in the fields: Host Connection: Select the desired host connection, from the drop-down list. The Manage Connections link can be used to define a new host connection. Login Credentials : Select or add your mainframe login credentials (userid and password) to connect to the host with. Test Project Folder : The Topaz for Total Test project folder path (from Topaz for Total Test Project Folder Properties). This location must be accessible to Jenkins. Read/write access is required to all files in this location. Test Scenario/Suite : Test suite or test scenario name for the specified project (from Topaz for Total Test Project Folder). JCL : The file name containing the JCL to submit (from Topaz for Total Test Run Test Scenario dialog). NOTE: If you use the Topaz for Total Test Preference option to hold the JobCard 106

113 Topaz for Total Test User Guide information you need to have executed the Test Suite or Test Scenario at least once in the Topaz for Total Test interface for this file to get generated. Main executable program type : Select the main executable program (this is the program specified on the 'EXEC PGM=' JCL statement in runner*.jcl) for Code Coverage: Live DB2 when the main program is IKJEFT01 or IKJEFT1B TOTALTEST when the main program is TTTRUNNR Live IMS when the main program is DFSRRC00 Code Coverage Repository : The name of the Code Coverage repository dataset. Must be specified to enable Code Coverage (from Topaz for Total Test Run Test Scenario dialog). System Name : Code Coverage system. If not specified, defaults to the test suite or test scenario name (from Topaz for Total Test Run Test Scenario dialog). Test ID : Code Coverage test id. If not specified, defaults to the test suite or test scenario name (from Topaz for Total Test Run Test Scenario dialog). Clear existing statistics before the test: Check if existing Code Coverage statistics should be deleted before running the test. Use Stubs : Check if the test should use stubs (from Topaz for Total Test Run Test Scenario dialog). Delete temporary files : Check if temporary files are to be deleted (from Topaz for Total Test Run Test Scenario dialog). High Level Qualifier : Optional high level qualifier to be used when allocating datasets during test execution (from Topaz for Total Test Preferences). 15. Click the Generate Pipeline Script button. 16. Copy the generated script step into your clipboard. 17. Return to the Project and select Configure. 18. In the Script field, generate a new line after the node{ line. 19. Paste the generated script step from the clipboard into the new line. 20. Press SAVE. Run a Test Suite 1. Return to the Jenkins Home page. 2. In the list of projects, find the Topaz for Total Test project you want to run. 3. Click on the Topaz for Total Test project. The selected project page opens. 4. Click Build Now. You will see a new Build start in the Build History area. 5. When the built has completed, click on the Build # for the completed build in the Build History area. 6. Click on Console Output to review the Console Output for the Build #. 7. Scroll to the bottom and look for SUCCESS or FAILURE. Add additional information in the console output 107

114 Configuring Topaz for Total Test Jenkins Plugin If you are having problems with getting a Test Suite to run from Topaz for Total Test Jenkins Plugin you may find it helpful to put additional information in the console output. Also if your Test Case isn t running it may be helpful to turn debug information on the TestRunner component by modifying a Runner.jcl file or your test jcl file for your Test Suite. TotalTestCLI.bat 1. Find the TotalTestCLI.bat file in the location where the Topaz for Total Test CLI was installed. 2. Open TotalTestCLI.bat. 3. off on. 4. Save TotalTestCLI.bat. Runner.jcl 1. Find the TestRunner JCL for the Test Run 2. Open the TestRunner JCL file 3. Change DEBUG(OFF) to DEBUG(ON). 4. Save the TestRunner JCL file. Add step for Jenkins to process the test results files 1. Open the Project page for the Jenkins project. 2. Click Configure. 3. In the Post-build Actions, click Add post-build action. 4. Select Publish Junit test result report. 5. In the Test report XML field, enter the location of the TTTUnit and TTTSonar folders holding the Topaz for Total Test XML results files, for example: TTTUnit\*.xml TTTSONAR\*.xml 6. Press SAVE. 7. Click the Build Now button to ensure the Test Results files are copied correctly into the Jenkins WORKSPACE TTTUnit Folder. 8. Notice whether the build completed with Success Status (blue ball), Warning Status (yellow ball), or Error Status (red ball). 9. Click on the Build # to review build results. 10. Notice the new Test Results. 11. Click on Test Result to display the results. 12. Drill down to view the details of the individual test case. This screenshot shows the detailed information for a failed test. 108

115 Topaz for Total Test User Guide 109

116 Import Your Topaz for Total Test Project Into ISPW Developers will want to retain and improve their Topaz for Total Test assets. They can store their complete Topaz for Total Test project in ISPW along with their code and other tasks in their Assignment. Total Test projects stored at the PROD level can be brought down for your use as you modify your program. You can then promote them up as your changes go through the lifecycle. The Topaz for Total Test projects stored in ISPW can also be downloaded using a Jenkins plugin and then run in your DevOps tool-chain. This topic explains the workflow to import an existing Topaz for Total Test project into ISPW. It is also presumed that ISPW has already been properly installed and, if necessary, customized for your specific environment, including the required configuration by the ISPW Administrator for Topaz for Total Test Component Types. For details please refer to the separate ISPW Technical Reference Guide. Preliminary Steps 1. Create your Topaz for Total Test Project. If you haven't already created your project, do that and execute the test. See Process Flow - Topaz for Total Test - COBOL Batch Programs. It is a good naming convention for your Topaz for Total Test projects to use a consistent name that includes your ISPW Application name (program name). This will make it easier to find your related Topaz for Total Test project. Make sure your ISPW Root in your Connection dialog points to your current Topaz Workspace. IMPORTANT: If you intend to execute your unit test in Jenkins, your project name MUST NOT contain any SPACES. If your project name contains spaces, then you must rename it. To do that, select it in the Topaz Project View and press F2, or right-click on it and choose Rename. 2. Copy the project location to your clipboard. In the Topaz Project View, select the project and press Alt+Enter, or rightclick and click Properties. Select the entry in the Location field and copy it to your clipboard. Import Project into ISPW Start importing the Topaz for Total Test project into an ISPW Assignment at the DEV Level. 1. Go to the ISPW Repository Explorer view and right-click the top node Application where you wish to store your project. This will be where you have your source which you should already have in an Assignment where you have been making your changes. Note, if you select another folder, it will make it a subfolder under it. 110

117 Topaz for Total Test User Guide 2. Select Import > Local File Select the top node Application, your folder will go under that. 111

118 Import Your Topaz for Total Test Project Into ISPW 3. In the Import Local File dialog: a. In From directory, paste the directory of your project, which you previously copied onto your clipboard. b. Press Enter. c. Click the checkbox on the left-hand side to select your project. d. Below select your Assignment, the Path will be your DEV level path. Or, if necessary, create a new assignment: Click on the New Assignment button. Add a Description Select the desired Assignment prefix Click on the OK button. e. You usually keep the Import new or changed files only option checked, this will save time when importing by avoiding importing existing files. This isn't an issue for a first-time import, but it will be when you use this to bring a Test Project back into ISPW. f. Many Types will not be defined - have them default to your TTT Type, select TTT for default 112

119 Topaz for Total Test User Guide g. Press Finish. This brings your Topaz for Total Test project into ISPW. You can see it in the Task View. Promoting your Assignment It is easiest to just Promote the entire Assignment from the Container View. You can do this as long as all Tasks are at the same Level, if not you then have to go to Task View, highlight the desired Tasks and Promote them as a group. 1. From the ISPW Containers view, right-click the desired container and select Promote. This will bring up your source and the Test Project. 2. If it is determined that the tasks will be promoted in a Set process, the Set dialog box appears. Otherwise, when processing has completed, the Promote Failure dialog box appears if there were any failures. 3. If the Set dialog box appeared, modify the fields on the Set dialog box as necessary and click OK. When processing has completed, the Promote Failure dialog box appears if there were any failures. Exporting Test Assets from ISPW 1. From the ISPW Repository Explorer, right click on the Topaz for Total Test project folder you want to export. 113

120 Import Your Topaz for Total Test Project Into ISPW 2. Select the Copy to Workstation option. 3. In the Copy to Workstation dialog, specify the directory path to a folder on your Workstation. 4. Press OK. ISPW copies the selected Topaz for Total Test project to the specified folder. 5. In Topaz, select File > Import. The Import dialog displays. 6. In the Import dialog, select General > Existing Project into Workspace 7. Click Next>. 114