QMetry Test Managment Plugin

Size: px

Start display at page:

Download "QMetry Test Managment Plugin"

Nicholas Ward
5 years ago
Views:

QMetry Test Managment Plugin About the Plugin Pre-requisites How to install QMetry Test Management Plugin for Jenkins Configuring a Standard Jenkins job Freestyle project Configuring QMetry Test

1 QMetry Test Managment Plugin About the Plugin Pre-requisites How to install QMetry Test Management Plugin for Jenkins Configuring a Standard Jenkins job Freestyle project Configuring QMetry Test Management Plugin as a Post-build Action Pipeline Define Parameters QMetry Configuration for Specflow Part A : Make the following changes in your Specflow Project to generate result file format as Cucumber JSON Part B : Make the following Configurations in Jenkins About the Plugin QMetry Test Management plugin for Jenkins has been designed to seamlessly integrate your CI/CD pipeline with QMetry. Easily configure Jenkins to submit your test results to QMetry without needing to write any code or deal with REST API. Your Test Results could be from any automation framework like Cucumber, Test NG, JUnit, QAF and more. For more info visit QMetry Test Management. Read the following topics on this page: Pre-requisites How to install QMetry Test Management Plugin for Jenkins Configuring a standard Jenkins Job. Configuring QMetry Plugin job as a post build action. Triggering the job and publishing results in QMetry. Pre-requisites QMetry Test Management installed on Server or Cloud Jenkins installation How to install QMetry Test Management Plugin for Jenkins 1. Login to Jenkins and go to Manage Jenkins. 2. Locate the Manage Plugins section.

2 3. Open the Available tab and search for "QMetry Test Management". 4. Locate and select the QMetry Test Management Plugin. Choose install option either Install without Restart OR Download Now and install after restart.

3 Configuring a Standard Jenkins job You can configure Jenkins job through either of the following options: Freestyle Project Pipeline Freestyle project Let s create a standard Freestyle job in Jenkins. 1. To create a new Jenkins job, click on the New Item option on the Jenkins menu. 2. Provide a name for your job. Select Freestyle project on the screen and then click OK.

On the Source Code Management tab, select the appropriate option in which you have your source code.

4 3. Next, the configuration page appears where you can configure your job by various categories: General, Source Code Management (SCM), Build Triggers, Build, and Post build actions. 4. On the Source Code Management tab, select the appropriate option in which you have your source code. For example, here it is Git. So, select Git and provide the necessary information. (Note: You need to configure GitHub plugin in global configuration). 5. Optionally, you can set Build Triggers.

Select the Build Triggers tab. Let s just add a periodic trigger which triggers build every minute every hour every day (Schedule: * * * * *). 6. Next, Select a build step.

5 Select the Build Triggers tab. Let s just add a periodic trigger which triggers build every minute every hour every day (Schedule: * * * * *). 6. Next, Select a build step. You can select from various options: Ant, Maven, Windows batch process, etc. For example, for a maven project, trigger a maven build and specify goals. Next step is to add Post-build Actions. Here, add the plugin: QMetry Test Management Plugin.

Configuring QMetry Test Management Plugin as a Post-build Action Jenkins works as a bridge between QMetry Test Management and automation frameworks.

6 Configuring QMetry Test Management Plugin as a Post-build Action Jenkins works as a bridge between QMetry Test Management and automation frameworks. To publish results in QMetry, define another post-build action. 1. On the Add post-build Actions drop-down, select the "Publish test result to QMetry" option. 2. Fill in the required data. You can get more information regarding fields by clicking on the Help icon for respective fields. It explains what to enter in that particular field.

7 Parameters Parameter Type Required Description QMetry URL string Yes QMetry Test Management URL Automation API Key string Yes Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key. Automation Framework string Yes Format of result file to be imported. Supported formats: Cucu mber Tes tng J U nit Q AS HP UFT Automation Hierarchy string No Hierarchy which will be used to parse test result files on QTM. Supported formats are :- JUnit 1 - Use current Hierarchy in which JUnit Testcase is treated as TestStep and Testsuite is treated as Testcase 2 - Use Junit Testcase as Testcase and link all those (from all testsuites and all files of Junit) to one Testsuite 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites TestNG 1 - Use class as Testcase and testmethod as TestStep 2 - Use test-method as Testcase Result File(s) Path/Directory string Yes Path wherein your result file to be uploaded. Supported file extensions:.json,.xml and.zip (zip file must contain files of the format given in the 'Automation Framework' param). Test Suite ID string No Key of the test suite exists in QMetry. In the absence of Key, a new test suite will be created. Note: If for Junit Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then the parameter Test Suite ID will be ignored. Test Suite Name string No If existing "testsuiteid" is not being used and auto-generated name for the new test suite is not desired, then a custom test suite name can be entered for this parameter. Note: If for Junit Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then this parameter will be ignored. Project string Yes ID/Key/Name of the Project within which the test suite will be created. Release string *No ID/Name of the Release in which the test execution will be created. If Release is not mentioned, then Default Release will be considered.

8 Cycle string No ID/Name of the Cycle in which the test execution will be created. If Cycle is not mentioned, then Default Cycle will be considered. Build string No Build Name/Id. In the absence of value, it will create the default build. Platform string No Name of Platform on which test run to be executed. * Release is mandatory if Cycle is mentioned. 3. On successful completion of the process, you can see the console output. It displays success message for uploading the result file. 4. Finally check test execution page in QMetry and review your test run results. Pipeline QMetry Test Management supports Jenkins Pipeline for implementing and integrating continuous delivery pipelines into Jenkins. Let s create a standard Pipeline job in Jenkins. 1. To create a new Jenkins job, click on the New Item option on the Jenkins menu.

9 2. Provide a name for your job. Select Pipeline on the screen and then click OK. 3. Next, the configuration page appears where you can configure your job by various categories: General, Build Triggers, Advanced Project Options, and Pipeline.

10 4. Optionally, you can set Build Triggers. Select the Build Triggers tab. Let s just add a periodic trigger which triggers build every minute every hour every day (Schedule: * * * * *). 5. Advanced Project Options: This is optional. You can give Display Name which will replace the name you gave earlier to the Pipeline project. 6. Jenkins works as a bridge between QMetry Test Management and automation frameworks. To publish results in QMetry, provide Pipeline script. Enter Pipeline script to define your code repository, generate test result and upload test results into QMetry Test Management. Code Snippet Pipeline Script step([$class: 'QTMReportPublisher', disableaction:true, qtmurl: ' automationframework: 'JUNIT', automationhierarchy: '2', testresultfilepath: 'junit/test-com.examples. ex2.appcustomtest.xml/ ', qtmautomationapikey: 'ctmmb6wyxo7sj0ychpjygttzcmtsdwd', project: 'GPT', release: 'Release 1', cycle: 'Cycle 1', buildname: 'testbuild1', platformname: 'Jenkins', testsname: 'junit pipeline'])

11 Syntax of automationframework: CUCUMBER TESTNG JUNIT QAS HPUFT Parameters Parameter Type Required Description QMetry URL string Yes QMetry Test Management URL Automation API Key string Yes Your API Key. API Key is unique for a specific user in a single project. The result will be imported for the project associated with the given API Key. Automation Framework string Yes Format of result file to be imported. Supported formats: Cucu mber Tes tng J U nit Q AS HP UFT Automation Hierarchy string No Hierarchy which will be used to parse test result files on QTM. Supported formats are :- JUnit 1 - Use current Hierarchy in which JUnit Testcase is treated as TestStep and Testsuite is treated as Testcase 2 - Use Junit Testcase as Testcase and link all those (from all testsuites and all files of Junit) to one Testsuite 3 - Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites TestNG 1 - Use class as Testcase and testmethod as TestStep 2 - Use test-method as Testcase

12 Result File(s) Path/Directory string Yes Path wherein your result file to be uploaded. Supported file extensions:.json,.xml and.zip (zip file must contain files of the format given in the 'Automation Framework' param). Test Suite ID string No Key of the test suite exists in QMetry. In the absence of Key, a new test suite will be created. Note: If for Junit Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then the parameter Test Suite ID will be ignored. Test Suite Name string No If existing "testsuiteid" is not being used and auto-generated name for the new test suite is not desired, then a custom test suite name can be entered for this parameter. Note: If for Junit Create Multiple Testsuites and then link their respective testcases in corresponding Testsuites - is chosen, then this parameter will be ignored. Project string Yes ID/Key/Name of the Project within which the test suite will be created. Release string *No ID/Name of the Release in which the test execution will be created. If Release is not mentioned, then Default Release will be considered. Cycle string No ID/Name of the Cycle in which the test execution will be created. If Cycle is not mentioned, then Default Cycle will be considered. Build string No Build Name/Id. In the absence of value, it will create the default build. Platform string No Name of Platform on which test run to be executed. * Release is mandatory if Cycle is mentioned. 7. On successful completion of the process, you can see the console output. It displays success message for uploading the result file. 8. Finally check test execution page in QMetry and review your test run results.

Define Parameters To define parameters for your job, select the This project is parameterized check box. The Add Parameter drop-down is enabled with the list of parameter types.

13 Define Parameters To define parameters for your job, select the This project is parameterized check box. The Add Parameter drop-down is enabled with the list of parameter types. Use the drop-down button to add as many parameters as you need. There are different parameter types available. The way parameters take effect according their respective parameter type. For example, Choice Parameter and String Parameter are selected, and corresponding parameters are added as shown below.

14 The above parameters will be parsed dynamically at the run time. The field values should be entered in the following syntax: ${ParameterName} For example, For the parameters created above, the field value would be ${API} and ${TRID} respectively. The parameter values will be parsed accordingly for the corresponding fields. The values can also be changed before triggering the Jenkins build.

QMetry Configuration for Specflow Specflow by default generates the output results in Specflow JSON format. QMetry can consume it as Cucumber JSON format.

15 QMetry Configuration for Specflow Specflow by default generates the output results in Specflow JSON format. QMetry can consume it as Cucumber JSON format. The following solution will generate the output result in Cucumber JSON format in parallel and the result could imported to QMetry. Part A : Make the following changes in your Specflow Project to generate result file format as Cucumber JSON Step 1 : Add Dependencies Step 2 : Add the below code in.cs of feature file [BeforeTestRun] public static void BeforeTestRun() { SpecNuts.Reporters.Add(new JsonReporter()); SpecNuts.Reporters.FinishedReport += (sender, args) => { String pathname = "specflow_cucumber.json"; System.IO.File.WriteAllText(pathName, args.reporter. WriteToString()); Console.WriteLine("Result File: " + System.IO.Directory. GetCurrentDirectory().ToString() + System.IO.Path. DirectorySeparatorChar + pathname); } }; Step 3 : Open Text Explorer in Visual Studio by Test > Windows > Test Explorer -> Choose Run All Step 4 : In Visual Studio console, select show output from Test and open Report File as path given in Console

16 Part B : Make the following Configurations in Jenkins Configuration Steps Install "MSBuild Plugin" and "VSTest Runner Plugin" Install Visual tools and Set path for MSBuild.exe and vstest.console.exe files in Manage Jenkins > Global Tool Configuration Configure the following parameters in the Jenkins Job Step 1 : In the Build Step >> Choose - Build a Visual Studio project or solution using MSBuild Step 2 : Provide the MsBuild Version and relative path to solution file Select "MSBuild Version" which is configured in Global Tool Configuration Specify the relative path for solution file in "MSBuild Build File"

17 Step 3 : Select a build step for VSTest in Build Section - Run unit tests with VSTest console Select "VSTest Version" which is configured in Global Tool Configuration Specify the relative path to your VSTest compiled assemblies in "Test Files"

18 Step 4 : In post build action setup QMetry Test Management plugin Choose "Cucumber" format to upload Relative Path to result file "specflow_cucumber.json" Choose other relevant parameters QMetry Privacy Policy Automation API URL

Continuous Integration with Selenium and Jenkins

Continuous Integration with Selenium and Jenkins Continuous Integration Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several