IBM Software Group. Code Coverage

Size: px

Start display at page:

Download "IBM Software Group. Code Coverage"

Aubrey Benson
5 years ago
Views:

1 IBM Software Group Code Coverage Jon Sayles/IBM October 1 st, 2018

2 Code Coverage Overview Extension to Debugging: Tracks lines of code that have been executed during test Improves application testing quality Focuses testing resource usage Numerous reports on tested code and trends Supports: Batch, CICS and IMS TM Many options: Exclude/Include specific modules Filter and Export Reports Integrates with ADDi Detailed/Source Line Coverage Module Level Code Coverage 2

3 Code Coverage Setup Process: Compile/Link with standard Debug Tool JCL Parms (TEST etc.) Submit run JCL passing in a PARM with: Your I/P address and port# ENVAR( EQA_STARTUP_KEY=CC") Make sure that the Debug Tool runtime is in STEPLIB, JOBLIB or installed into Link Pack Run your debug session. Instead of dropping into interactive Debug, your program will execute After the job finishes a Code Coverage report opens in the editor (this takes a few seconds) Double-click a program to view the detailed lines The report will provide varying levels of Code Coverage. NOTES: //STEPLIB reference to Debug Tool runtime is only required if not installed in Link Pack You can also add the ENVAR parameter into //CEEOPTS DD card 3

4 Code Coverage Specifying ENVAR Specifying the ENVAR parameter Batch: PARM card //CEEOPTS DD card TEST parameter Online DTCN Profile DTCN Transaction PF9 Options 4

Code Coverage Reports 1 of 2 Paragraph coverage When the Code Coverage batch job or transaction finishes and you re running RDz interactively: The current Code Coverage report

let you know of any modules and/or paragraphs that were accessed at below a specified coverage threshold You set the threshold for coverage warnings in Preferences Individual

5 Code Coverage Reports 1 of 2 Paragraph coverage When the Code Coverage batch job or transaction finishes and you re running RDz interactively: The current Code Coverage report opens in the editor window The Compiled Code Coverage Results view is opened Code Coverage Reports present: Coverage Totals The lines that were executed A warning symbol to let you know of any modules and/or paragraphs that were accessed at below a specified coverage threshold You set the threshold for coverage warnings in Preferences Individual program paragraph coverage statistics Program Entry Compiled Code Coverage Results view allows you to see previous Code Coverage Reports. These are stored in your Workspace Opens 5

Code Coverage Reports 2 of 2 Opens From the report window, you

) and double-click the entry to open a hyper-linked source

PROCEDURE DIVISION (as well as the procedural logic in PL/I

6 Code Coverage Reports 2 of 2 Opens From the report window, you can drill-down to a Program or Function (COBOL paragraph, etc.) and double-click the entry to open a hyper-linked source file Executable statement/lines are annotated within the PROCEDURE DIVISION (as well as the procedural logic in PL/I programs) Red == this line was not executed Green == this line was executed 6

differences Deltas are calculated at various levels Totals Module Paragraph In

7 Compare Reports Can compare two Code Coverage Reports Later (Date/Timestamped) report is compared to earlier selected report Colored deltas highlight differences Deltas are calculated at various levels Totals Module Paragraph In this example a batch job is run twice using two different input datasets Reports are compared 7

8 Filtering Detailed Reports You can manipulate the combination of Threshold, Above and Below then click: Refresh to filter the paragraphs displayed in the report... 8

9 Merge Code coverage Results Combine multiple results Hover over area of code to see individual test result 9

10 Save Code Coverage to Local Workstation Can Export reports to a local file system, and them back as a Result Location... 10

Code Coverage Options Large number of options for Code Coverage. Detailed information https://www.ibm.

11 Code Coverage Options Large number of options for Code Coverage. Detailed information c/topics/tccstartup.html 11

12 Code Coverage Options Filtering out Modules You can exclude modules from Code Coverage by adding a parm to the ENVAR: EQA_STARTUP_KEY=CC,,moduleexcludelist=PGM) Notes Two (2) commas before the parm lowercase You can also use moduleincludelist= 12

Code Coverage Options Using tag= If you wish to

Add a tag=nnn specification to your JCL 4.

13 Code Coverage Options Using tag= If you wish to selectively cover only code that you ve added you can pinpoint specific lines as follows: 1. Add a source Tag to the line(s) 2. Compile/Link 3. Add a tag=nnn specification to your JCL 4. Run Code Coverage 5. Open the tag= Report as a Filtered File Report 6. Specify the same Filter as your tag 13

14 Running Code Coverage in Batch (headless) Mode Small footprint non-gui code coverage collector Runs native on target system Requires Java 6 or higher Collects CC results and optionally generates Source that matches CC results HTML reports Zip of results Results can be imported into RDz for further analysis Comparisons View source lines annotated with hit information Integrate into automated builds and testing Run as continuous daemon that collects all data OR Run instance per build/test Can be invoked from JCL Supports adding tags for future reference e.g. testcase ID 14

Running Code Coverage in Batch Mode Example JCL //GYOUNGG JOB XXXXXXXX,CLASS=A,MSGCLASS=H,MSGLEVEL=(1,1), // REGION=256M,NOTIFY=&SYSUID,TIME=1 //STEP1 EXEC

100%8005:*), ENVAR("EQA_STARTUP_KEY=CC,COB01A, CCLEVEL=LINE,MODULEINCLUDE=COB01A") Start daemon COB01 Load Module COB01A - Subroutine GYOUNG.CC5.

15 Running Code Coverage in Batch Mode Example JCL //GYOUNGG JOB XXXXXXXX,CLASS=A,MSGCLASS=H,MSGLEVEL=(1,1), // REGION=256M,NOTIFY=&SYSUID,TIME=1 //STEP1 EXEC PGM=COB01 //STEPLIB DD DISP=SHR,DSN=GYOUNG.CC5.LOADPDSE //SYSOUT DD SYSOUT=* //CEEOPTS DD * TEST(ALL,*,PROMPT,TCPIP& %8005:*), ENVAR("EQA_STARTUP_KEY=CC,COB01A, CCLEVEL=LINE,MODULEINCLUDE=COB01A") Start daemon COB01 Load Module COB01A - Subroutine GYOUNG.CC5.LOADPDSE (Load library) Code Cover COB01A Save Coverage Report Can specify DIRECT instead of TCPIP Excellent White Paper here 15 Windows Server

reports for: Specific programs Specific areas within specific

Group IDs Involves setting up JCL that includes an EQAOPTS DD

16 Debug Tool Additional C.C. Functionality Debug Tool allows you to specify Code Coverage reports for: Specific programs Specific areas within specific programs Single lines Ranges of lines Optional authorization by Group IDs Involves setting up JCL that includes an EQAOPTS DD used to specify input/options and output/observations files For more information check out the Debug Tool Users Guide 16

Debug Tool Additional Functionality: Code Coverage Utility There is a Debug Tool Code Coverage Utility that provides a considerable number of code coverage metrics through reports.

17 Debug Tool Additional Functionality: Code Coverage Utility There is a Debug Tool Code Coverage Utility that provides a considerable number of code coverage metrics through reports. Each instruction line of the listing has a character that annotates what happened during the test run: & A conditional branch instruction that has executed both ways > A conditional branch instruction that has branched but not fallen through V A conditional branch instruction that has fallen through but not branched : Non-branch instruction that has executed Instruction that has not executed The reports are available for green-bar printout (!) and as HTML. There is a bit more setup to do to use the Utility. For more information check out the Debug Tool Users Guide 17

IBM Rational Developer for System z PROC Validation

RDz Workstation / Host Component Validation IBM Rational Developer for System z PROC Validation Checklist #3 Version V1.3 February 27, 2012 The home location for this document is in the RDz Server Health