RemoteLaunch Automated Testing Integration & User Guide Inflectra Corporation

Transcription

1 RemteLaunch Autmated Testing Integratin & User Guide Inflectra Crpratin Date: Nvember 22nd, 2017

2 Cntents Intrductin RemteLaunch Guide HP UFT / QTP Engine SmarteScript Engine TestCmplete Engine Selenium Engine Squish Engine Cmmand-Line Engine LadRunner 11 Engine SapUI Engine FitNesse Engine NeLad Engine TestPartner Engine BadBy Engine JMeter Engine Intrductin SpiraTest prvides an integrated, hlistic Quality Assurance (QA) management slutin that manages requirements, tests and incidents in ne envirnment, with cmplete traceability frm inceptin t cmpletin. SpiraTeam is an integrated Applicatin Lifecycle Management (ALM) system that manages yur prject's requirements, releases, test cases, issues and tasks in ne unified envirnment. SpiraTeam cntains all f the features prvided by SpiraTest - ur highly acclaimed quality assurance system and SpiraPlan - ur agile-enabled prject management slutin. This guide utlines hw t use the RemteLaunch add-n fr SpiraTest r SpiraTeam t remtely schedule and launch autmated tests using a variety f cmmercial and pen-surce test autmatin engines. Fr infrmatin regarding hw t use SpiraTest itself, please refer t the SpiraTest User Manual. The first sectin utlines hw t use the RemteLaunch applicatin, and the subsequent sectins describe each f the plugins that supprt a different test autmatin tl. 15. Ranrex Engine Ratinal Functinal Tester TestingAnywhere Page 1 f 144

3 1. RemteLaunch Guide There are actually tw separate versins f RemteLaunch that are available frm Inflectra: 1. The Micrsft Windws cmpatible Spira RemteLaunch applicatin that prvides a graphic user interface applicatin fr executing autmated tests n remte cmputers using varius plugins fr different testing technlgies and have the results be sent t the cnfigured SpiraTest/SpiraTeam server. 2. The crss-platfrm Spira RemteLaunchX Java applicatin that prvides a lightweight cnsle applicatin that can execute simple cmmand line scripts n the target cmputer and send the results back t the cnfigured SpiraTest/SpiraTeam server. This applicatin can be used in Micrsft Windws, Linux r Apple MacOS X cmputers prvided that they have the Java 1.7 (r later) runtime installed. The first part f this sectin will describe hw t use the Windws-nly RemteLaunch GUI applicatin and the secnd part will describe hw t use the crss-platfrm RemteLaunchX cnsle applicatin Installing RemteLaunch It is required that yu install the prgram befre cpying r installing any test extensins fr the prgram. Testing applicatins, like Selenium and QuickTest Pr can be installed with n regards t the client applicatin if they are nt installed by the time a test requiring them needs t be executed, the test extensin will simply reprt an errr r blck fr the specified test set. There are n ptins t the installer except fr installatin path. If yu d nt use the default installatin path (typically C:\Prgram Files\Inflectra\Spira RemteLaunch\), then make a nte f where the installatin path is, because it will be needed t install test extensins later Installing a Test Extensin A test extensin is a single r a set f DLLs that the prgram will read upn startup and prvides a link in which testing applicatins (like TestCmplete and Squish) t reprt test infrmatin and status back t SpiraTeam. When yu dwnlad a test extensin, the ZIP file shuld cntain at least ne DLL file. Unless therwise specified by a readme.txt file included in the cmpressed file, cpy the DLL file t the \extensin directry lcated within Spira RemteLaunch installatin directry. (If n such flder exists, yu must create it.) If an extensin is remved r added, the prgram must be restarted fr the any changes t take effect. The prgram will nly lad up t the first number f extensins that the license allws. Additinal extensins will nt be laded r used during testing Registratin Spira RemteLaunch has its wn License key needed fr using the prgram. Yu cannt use yur existing SpiraTest/Plan/Team key in Spira RemteLaunch. Upn the first launch f the prgram, yu will be asked t update yur license infrmatin. Enter in yur rganizatin name and license key in the that was sent when yu purchased the license, r as listed n yur custmer infrmatin page at Page 2 f 144

4 Trial licenses are gd until the 28 th day f the listed mnth. The next time the prgram is run after the 28 th f the mnth, yu will be prmpted t re-enter a new permanent license key, r the prgram will be unusable. The license key can be updated at any time by ging t the Tray Menu and select Help -> Abut. Once the Abut screen pens up, click the Update buttn in the license details sectin t update r change license infrmatin Using RemteLaunch Basic Unattended Operatin When run, the prgram will start minimized t the system tray and will start its plling f the server. Plling will ccur every x minutes (60 by default) fr any autmated test sets that are scheduled t be run. When time cmes fr a test t be launched, it will start the test extensin. The installed test extensin will then perfrm the test and reprt results back t SpiraTeam. At the end f the test, the prgram will g back and resume scanning fr tests that need t be executed. N user input is ever needed frm the applicatin itself. Hwever, testing applicatins may pp up dialgs needing user input. Fr existing Inflectra testing extensins, effrt was put in t avid as much userinteractin as pssible, but in sme cases it is unavidable Client Cnfiguratin By right clicking n the system tray icn and selecting Cnfiguratin, the applicatin s windw will pen t the cnfiguratin panel. The panel has the fllwing ptins: SpiraTeam Server Cnfiguratin: Server URL: This is the URL f the SpiraTeam installatin. Be sure t nt put /Lgin.aspx r any ther page in the string, this shuld be just the rt URL f the applicatin s install. Lgin Username: This is the SpiraTeam lgin id f the user that yu want the tests reprted as. Nte that while the applicatin is plling and updating test results, if the user is lgged int a web brwser sessin, they will get kicked ut. Lgin Passwrd: The passwrd t the Username abve. Page 3 f 144

5 Test: Clicking this will test the lgin t make sure the applicatin can cnnect t the server prperly. Server Plling: Autmatin Hst Tken: This field is required, and uniquely identifies the lcal testing machine. Any scheduled tests assigned t the Autmatin Hst n SpiraTeam will get plled fr this machine. Except in special circumstances, this ID shuld be unique amng all testing machines. Imprtant: This field must match the string that is entered int the Autmatin Hst Details screen in the Tken: field, r scheduled tests will nt be recgnized. Autmatically Run Overdue Tests: When this is checked, any tests that are pulled frm the SpiraTest server that has a scheduled date in the past will be marked as Overdue. Nrmally, verdue tests will nt be executed. With this check, they will be executed as sn as the pll is finished. Cntinue plling server after a cnnect errr: When this is checked, if RemteLaunch receives an errr cnnecting t the SpiraTest server, it will cntinue plling in the future. If this is unchecked, RemteLaunch will switch t the errr status upn encuntering a cnnectin errr. It is imprtant t check this ptin if yur SpiraTest server will be peridically unavailable fr server maintenance. Plling Frequency: Hw ften in minutes the applicatin will pll the SpiraTeam server fr updates t the autmatin hst s schedule. The default is 60 (1 hur), and shuld be fine fr mst installatins. Nte that tests will still be executed n their scheduled time, this is simply hw ften the prgram will talk t the SpiraTeam server t detect schedule changes. Updating the plling frequency will reset the currently running timers. Plling Read Ahead: Hw far ahead in minutes the prgram shuld read the schedule fr the Autmatin hst. Tests that are scheduled farther in advance will nt shw up as a pending test n the status screen Extensin Cnfiguratin If an extensin has custm cnfiguratin ptins, they will appear as separate tabs lcated after the Client Setup tab. The cntents f each tab will vary depending n the extensin. View the extensin s dcumentatin fr ptins given in thse extensins Status Screen The status screen is usually hidden, but can be brught up fr display by duble-clicking n the system tray icn. The tp f the screen shws the current status, whether it s running a test r waiting t pll the server fr an update. It will als shw any errrs present n the applicatin, like a registratin errr r cnfiguratin issue. Under the status bar is a list f any pending r executing tests that are scheduled fr this testing machine. The list will get cleared at every pll, s tests that have executed since the previus pll will still be n the list, and will shw their executin status: Page 4 f 144

6 Green Arrw: A green arrw indicates that the test is still running, r RemteLaunch is waiting fr a reply frm the testing engine / test applicatin. Blue Checkbx: A blue checkbx indicates that the test is cmpleted, regardless f status f the individual test steps in the scheduled test set. Red Errr: A red errr indicatr indicates that the test extensin r the testing applicatin ran int an issue (utside f test results). In this case, any further tests that require the extensin will be marked as blcked, as the issue needs t be crrected within the extensin settings r testing applicatin. N Indicatin: N indicatin means that the test is currently awaiting fr its scheduled date t start. Nte that nly ne test will be launched at a time, s that if tw tests are scheduled at the same time, the ne with the lwer TestSet ID will be executed first, then as sn as it s finished, the secnd scheduled test will be run. By highlighting a test that has nt been executed yet, yu can click the Frce Execute buttn. This will cause the selected test t have its scheduled date t the current time, causing it t be immediately executed (r, if anther test is already running, next in line fr executin). At any time the Frce Pll buttn can be clicked, causing RemteLaunch t initiate an immediate pll f the SpiraTeam server t check fr pending runs. The timers fr the next server pll will be reset when the buttn is clicked Tray Icn Menu Instead f perating frm the applicatin windw, all functins exist n the tray icn menu as well, as well as sme additinal cmmands: Page 5 f 144

7 Pause / Resume: The Pause/Resume ptin pauses r resumes the timers fr plling and executing tests. If a test r server pll is already in prgress, it will nt cancel these. Hwever, after they are finished, n further plls r tests will be run. Pll Nw: This will frce a server pll fr upcming tests, and reset the pll timer. Cnfiguratin: Opens the main windw t the Cnfiguratin page. Help -> Abut: Opens the Abut windw, which displays the current license infrmatin and any laded extensins. Help -> View Help: Opens this PDF file in a brwser. Exit: Will cmpletely exit the prgram. Ding this will cancel any tests currently running and shut dwn the prgram. Any tests that were waiting t be executed will nt execute until the prgram is restarted and the plling is resumed. Yu can duble-click the try icn t bring up the main windw n the Status page. 1.3 Test Executin and Reprting All test handling is perfrmed by the extensin that the autmated tests are cnfigured fr. Test Sets that have multiple Test Cases, the Test Cases will all be executed in rder, sequentially. (N parallel executing.) At the start f executin fr a Test Set, the test set will be updated in SpiraTeam as In Prgress. As tests are perfrmed, the Test Cases will be updated with their status. The Test Set n the status screen will be marked with the executing icn. Once the Test Set is cmpleted, the status f the Test Set will be changed t Cmpleted, and will be marked n the status screen with a cmpleted icn. In case f an uncaught exceptin that is thrwn by the testing extensin, the Test Set will be marked Blcked, and the Test Case will be recrded as Blcked. All ther fllwing tests will nt be run and remain as Nt Run. The Test Set must be reset t be executed again, and it is recmmended t lk int the cause f the errr (recrded in the Blcked Test Case results) and crrect it befre rescheduling the test. This Test Set will be marked with and errr icn. The same results are applied in the case where a Test Set cntains a Test Case that references a testing extensin that is nt installed. Install the extensin and re-run the Test Set. Executing, Cmpleted, and Errr Test Sets are marked with the icns next t their scheduled date in the Status screen. They will stay in the list until the next scheduled server pll. Yu cannt manually re-run them Running RemteLaunch frm a Build Script Nrmally yu schedule tests in SpiraTeam using the Planned Date field f the test sets and let the varius instances f RemteLaunch pll SpiraTeam fr upcming tests. In additin (as described in the SpiraTeam User Manual) yu can execute a test set n the lcal machine immediately by clicking the Execute buttn within SpiraTeam. Hwever there are situatins where yu want t be able t launch an autmated test script using ne f the supprted engines frm an external batch file r build script (e.g. as part f a cntinuus integratin envirnment) and have thse tests reprt their results back int SpiraTeam. Yu can achieve this by using the special cmmand-line argument testset which is passed t RemteLaunch. Fr mre details n this parameter see the next sectin. Page 6 f 144

8 1.5 Cmmand line arguments Fr debugging and additinal ptins when running the prgram, the fllwing cmmand-line arguments are available: -status -paused -pll -trace -lgfile -testset:[test Set ID] <filename> Shws the Status screen upn startup. (Nrmal actin is t run minimized t the system tray. Starts the applicatin with timers Paused instead f active. Frces the prgram t d an initial pll upn startup. (Nrmal actin is t wait the pending time befre ding the initial pll.) Enables tracelgging t the EventLg fr debugging and watching tests execute. Frces events t be written t a text file instead f the Applicatin EventLg. This ptin enables trace as well. Files are lcated in the Lcal Applicatin Data flder. (C:\Users\<user>\AppData\Lcal n Vista/Win7). Allws yu t tell RemteLaunch t execute a specific test set n the remte cmputer (e.g. -testset:45 runs test set TX00045) Must be the last item n the cmmand line. This is a TST file dwnladed frm SpiraTeam t start immediate executin n. Page 7 f 144

9 1.6. Using RemteLaunchX When yu need t run autmated tests n a variety f different platfrms (Windws, MacOS X, Linux, Unix, etc.) the RemteLaunchX crss-platfrm autmated testing agent is a better chice than the standard RemteLaunch GUI applicatin. T start using RemteLaunchX, please g t the Custmer Area f the Inflectra website and dwnlad the latest versin f the RemteLaunchX applicatin. It will be packaged as a simple.zip cmpressed flder that yu can extract nt the target cmputer: The fllwing fur files are included: RemteLaunchX.jar this is the main applicatin, packaged as a Java JAR file. This versin f RemteLaunch requires Java 1.7 SE r later t be installed. cnfig.prperties this cntains all the settings used by RemteLaunchX. Yu will need t edit this file in a text editr t cnfigure RemteLaunchX fr use. RemteLaunchX.bat this is a sample Windws batch file that can be used t simplify running RemteLaunchX n Windws systems. RemteLaunchX.sh this is a sample UNIX/Linux/MacOS X shell script that can be used t run RemteLaunchX n UNIX, Linux r Mac OS X Cnfiguring RemteLaunchX Once yu have extracted the files listed abve, pen up the cnfig.prperties file in a text editr: #This file cntains the cnfiguratin data used by the RemteLaunch-X applicatin #Spira cnnectin infrmatin server-url = server-lgin = fredblggs server-passwrd = PleaseChange #The autmatin hst tken hst-tken = MyHst1 #The license key license-rganizatin: TBD license-key: TBD #The regular expressins fr each f the pssible executin statuses pass-regex =.* fail-regex =.*(Errr Fail Fatal).* cautin-regex =.*(Warning Cautin).* blcked-regex =.*(Blcked).* The fllwing changes need t be made t this cnfiguratin file: Page 8 f 144

10 server-url This is the URL f the SpiraTest r SpiraTeam installatin (hereafter referred t as just SpiraTest). Be sure t nt put /Lgin.aspx r any ther page in the string, this shuld be just the rt URL f the applicatin s install. server-lgin This is the SpiraTest lgin id f the user that yu want the tests reprted as. Nte that while the applicatin is plling and updating test results, if the user is lgged int a web brwser sessin, they will get kicked ut. server-passwrd The passwrd f the SpiraTest lgin listed abve. hst-tken This field is required, and uniquely identifies the lcal testing machine. Any scheduled tests assigned t the Autmatin Hst n SpiraTest will get plled fr this machine. Except in special circumstances, this ID shuld be unique amng all testing machines. Imprtant: This field must match the string that is entered int the Autmatin Hst Details screen in the Tken: field, r scheduled tests will nt be recgnized. license-rganizatin The name f the Organizatin that yur RemteLaunch license key was issued t. This is listed in the Custmer Area f the Inflectra website alngside the license key. Nte: RemteLaunch and RemteLaunchX use the same license keys, s yu dn t need t have a separate RemteLaunchX ne. license-key The RemteLaunch license key that is listed in the secure Custmer Area f the Inflectra website Yu shuld leave the fur regex settings alne fr nw, they can be changed when yu start executing tests and need t fine-tune hw RemteLaunchX interprets the results. Nw that yu have cnfigured the plugin, yu can execute the RemteLaunchX cnsle applicatin by either running the prvided batch / shell cmmand r just executing the JAR file directly: Java jar RemteLaunchX.jar When yu run the applicatin, the fllwing shuld be utput t the cnsle: Starting RemteLaunch... ======================== Server URL: Server Lgin: fredblggs Autmatin Hst: MyHst1 Checking License Key fr: Inflectra Crpratin Prductin License Key in Use. Testing cnnectin t Spira... Successfully cnnected t Spira. WARNING: Unable t retrieve test runs fr SpiraTest prject PR2, s skipping this prject - Autmatin Hst with tken 'MyHst1' desn't exist in prject PR2. WARNING: Unable t retrieve test runs fr SpiraTest prject PR3, s skipping this prject - Autmatin Hst with tken 'MyHst1' desn't exist in prject PR3. Retrieved 0 test run(s) frm SpiraTest. Exiting RemteLaunch... ======================== The system will reprt back zer Test Runs at this pint because nthing has been scheduled in SpiraTest. In the next sectin we shall setup an autmated test set that cntains an autmated test case. Page 9 f 144

11 Setting up Autmated Tests in SpiraTest This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunchX n the varius test autmatin hsts fllwing the instructins abve. Once thse prerequisites are in place, please fllw these steps: Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Cmmand-Line this shuld be simply CmmandLine. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with Cmmand-Line listed as an available autmatin engine. Next yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Page 10 f 144

12 Yu need t enter the fllwing fields: Autmatin Engine - Chse the Cmmand-Line Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This can be set t Attached r Linked (see belw fr the difference). Filename This needs t cnsist f the fllwing three sectins separated by a pipe ( ) character: 1. The full path t the cmmand-line tl being executed. 2. Any arguments fr the cmmand-line tl. In additin, yu can use the fllwing additinal tkens fr sme f the special RemteLaunchX values: [TestCaseId] the ID f the test case [TestSetId] the ID f the test set [ReleaseId] the ID f the release (if specified) [Filename] - This special tken will be replaced by the actual filename f the test script when RemteLaunchX dwnlads it frm SpiraTeam. 3. The mask fr cnverting any parameter values frm SpiraTeam int valid cmmand line arguments. If parameters are nt accepted by the cmmand-line tl, yu can leave this sectin ut. The mask can include any symbls tgether with name t refer t the parameter name and value t refer t the parameter value. Example 1: If yu want parameters t be prvided in the frm: -param1=value1 param2=value2 yu wuld use the fllwing mask: -name=value Example 2: If yu want parameters t be prvided in the frm: /param1:value1 /param2:value2 yu wuld use the fllwing mask: /name:value Sme example filenames wuld be: C:\Temp\TestApp.exe -arg1 -arg2 -name=value C:\Temp\TestApp.exe -arg1 -arg2 "-arg3=[filename]" where the first ne is fr a Linked test and the secnd ne is fr an Attached test. Dcument Type Yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder Yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script Fr Attached test scripts, this needs t cntain the cmplete test script in whatever language and syntax is being expected by the cmmand-line applicatin. Fr Linked test scripts, yu shuld leave this blank. If yu wuld like t have SpiraTeam pass any parameter values t this test script yu can specify them by using the syntax ${parametername} inside the test script. Page 11 f 144

13 here is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur cmmand-line autmated testing tl. This is very useful if yu want t have a data-driven test script that be executed multiple times with different parameter values. T setup the autmated test case fr parameters, click n the Edit Parameters hyperlink abve the Test Script bx: The name f the parameter ${lgin} needs t match the name f a parameter accepted by the cmmand-line tl. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin. G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunchX applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case. Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Page 12 f 144

14 Create a new Test Set t hld the Cmmand-Line autmated test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Page 13 f 144

15 Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Running RemteLaunchX Once yu have set the varius test set fields (as described abve), yu are nw ready t execute RemteLaunchX. Yu can execute the RemteLaunchX cnsle applicatin by either running the prvided batch / shell cmmand r just executing the JAR file directly: Java jar RemteLaunchX.jar When yu run the applicatin, the fllwing shuld be utput t the cnsle: Starting RemteLaunch... ======================== Server URL: Server Lgin: fredblggs Autmatin Hst: MyHst1 Checking License Key fr: Inflectra Crpratin Prductin License Key in Use. Testing cnnectin t Spira... Successfully cnnected t Spira. WARNING: Unable t retrieve test runs fr SpiraTest prject PR2, s skipping this prject - Autmatin Hst with tken 'MyHst1' desn't exist in prjec PR2. WARNING: Unable t retrieve test runs fr SpiraTest prject PR3, s skipping this prject - Autmatin Hst with tken 'MyHst1' desn't exist in prjec PR3. Retrieved 1 test run(s) frm SpiraTest. Executing test case TC18 with filename 'C:\Windws\System32\ipcnfig.exe /all' This is a Linked test script Executing cmmand 'C:\Windws\System32\ipcnfig.exe' with arguments '/all' Executin Status = Passed Exiting RemteLaunch... ======================== The cnsle utput will indicate which test sets are being executed and what the final result was. Inside SpiraTest, nce executin begins the status f the test set will change frm Nt Started t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted (passed r failed) r Blcked RemteLaunchX was nt able t execute the test. Page 14 f 144

16 In additin, the individual test cases in the set will display a status based n the results f the cmmandline test that was executed: Passed The autmated test ran successfully and matched the PASS regular expressin. Failed The autmated test ran successfully, and matched the FAIL regular expressin. Cautin The autmated test ran successfully, and matched the CAUTION regular expressin. Blcked The autmated test did nt run successfully r it matched the BLOCKED regular expressin. If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Test Run that was recrded and the Cnsle utput sectin will cntain the underlying errr message(s). Once the tests have cmpleted, yu can lg back int SpiraTest and see the executin status f yur test cases. If yu click n a Test Run that was generated by the cmmand-line tl, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm cmmand-line tl tgether with any messages r ther infrmatin. The executin status will be set accrding t the rules described abve, the Message field will cntain the first line f cnsle utput and the large details bx will cntain the full cnsle utput frm the cmmand-line tl. Cngratulatins Yu are nw able t run a custm cmmand-line test, and have the results be recrded within SpiraTest / SpiraTeam Scheduling RemteLaunchX Unlike the main RemteLaunch applicatin, RemteLaunchX des nt have a built-in timer and s when executed it will run nce, check fr pending test sets and then exit. If yu want t have it run n a peridic basis, yu will need t schedule it externally. If yu are using Micrsft Windws yu wuld use the Windws Task Scheduler and in ther perating systems yu wuld setup a CRON jb. We recmmend scheduling RemteLaunchX t run every 5 minutes. Page 15 f 144

17 Custmizing the Reprting By default, RemteLaunchX will use the fllwing rules t determine if a test has passed, failed, blcked r passed with warnings (cautin): Passed The test cmpleted and the cnsle utput didn t cntain any f the errr phrases listed in the ther rules (belw). Failed The test cmpleted and the cnsle utput cntained the phrases Errr, Fail r Fatal. Cautin The test cmpleted and the cnsle utput cntained the phrases Warning, r Cautin. Blcked The autmated test did nt run successfully r the cnsle utput cntained the phrase Blcked. Yu can custmize the reprting by changing the Regular Expressins (Regex) stred in the cnfig.prperties files: #The regular expressins fr each f the pssible executin statuses pass-regex =.* fail-regex =.*(Errr Fail Fatal).* cautin-regex =.*(Warning Cautin).* blcked-regex =.*(Blcked).* Page 16 f 144

18 2. HP UFT / QTP Engine HP QuickTest Prfessinal (hereafter QTP) is an autmated functinal test autmatin system that lets yu recrd applicatin peratins and generate VBA test autmatin scripts that can be used t playback the test script against the test applicatin. HP Unified Functinal Testing (hereafter UFT) is an updated versin f QTP that als includes functinality fr web service testing. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f QTP and UFT n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated QTP and UFT tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 9.0 f Quick Test Prfessinal. Fr accessing UFT, yu d need at least versin 4.0 f SpiraTest/Team and versin 11.0 f UFT Installing the UFT/QTP Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the QuickTestPrAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate QuickTestPrX.dll r UftX.dll fr the versin f QTP r UFT that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file QuickTestPrX.dll r UftX.dll (where X is the apprpriate versin) int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Page 17 f 144

19 Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr QTP this shuld be QuickTestPrX where X is the versin number f the DLL file that yu are using. Fr UFT this shuld be UftX where X is the versin number f the DLL file that yu are using. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with QTP listed as an available autmatin engine Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated QTP r UFT test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Overview tab and scrll dwn t the Autmatin sectin: Yu need t enter the fllwing fields: Autmatin Engine - Chse the QTP/UFT Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with QTP/UFT nly supprts referencing QTP/UFT test script flder paths and nt physically uplading the test scripts int SpiraTeam. Filename This needs t be the full path t the QTP/UFT test script flder (i.e. the flder that yu pen in QTP/UFT t run the test). T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins (see example in screensht): [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. Page 18 f 144

20 [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the QTP/UFT Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur QTP/UFT autmated test script. This is very useful if yu have a data-driven QTP/UFT test script that accepts input parameters frm an external data surce. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${lgin} needs t match the name f the input parameter defined within the QTP/UFT script in its input parameters cnfiguratin Executing the QTP/UFT Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Page 19 f 144

21 Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the QTP/UFT autmated test cases and click n its hyperlink t display the test set details page. Yu need t add at least ne autmated test case t the test set: Page 20 f 144

22 Then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Page 21 f 144

23 Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the QTP/UFT test: Page 22 f 144

24 Passed The QTP/UFT autmated test ran successfully and all the test cnditins in the test script passed Failed The QTP/UFT autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The QTP/UFT autmated test did nt run successfully If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser r applicatin windws launch as QuickTest Pr (QTP) r Unified Functinal Testing (UFT) executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by QTP/UFT, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm QTP/UFT tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside QTP/UFT, and the executin status crrespnds the matching status inside QTP/UFT as illustrated belw: QTP/UFT Status Passed Failed Warning Stpped Nt Applicable (Any ther status) SpiraTeam Status Passed Failed Cautin Blcked N/A Nt Run Page 23 f 144

25 In additin, the detailed test reprt frm QTP/UFT is available in the large text-bx belw. It will cntain messages such as: QuickTest Prfessinal Test: Test1 Results Name: Res21 Run Started: 10/22/ :49:06 Run Ended: 10/22/ :49:10 Summary Results ======= ======= Passed: 0 Failed: 0 Warnings: 0 Detailed Results ======= ======= Iteratin: 1 ============= Actin: Lg In T Flight ============= Step: Lgin: Dialg Step: Agent Name:.SetText: "Bbba Fett" Step: Agent Name:.Type: "&lt MicTab&gt" Step: Passwrd:.SetSecureText: "4cc08e b35bb8a7dab8442c69b8441f3e" Step: OK.Click: Step: Flight Reservatins: Dialg Step: OK.Click: Cngratulatins Yu are nw able t run QTP/UFT autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 24 f 144

26 3. SmarteScript Engine SmarteSft SmarteScript (hereafter SmarteScript) is a Graphic User Interface (GUI) script-free functinal test autmatin system that lets yu recrd applicatin peratins by capturing the varius testable bjects f the applicatin and then playback the peratins t autmatically test the applicatin. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f SmarteScript n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated SmarteScript tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 5.0 f SmarteScript Installing the SmarteScript Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the SmarteScriptAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate SmarteScriptX.dll fr the versin f SmarteScript that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file SmarteScriptX.dll (where X is the apprpriate versin) int the extensins subflder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Page 25 f 144

27 SmarteScript this shuld be SmarteScriptX where X is the versin number f the DLL file that yu are using. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with SmarteScript listed as an available autmatin engine Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated SmarteScript test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the SmarteScript Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with SmarteScript nly supprts referencing SmarteScript test script file (.ses) and nt physically uplading the test scripts int SpiraTeam. Filename This needs t be the full path t the SmarteScript test script (i.e. the.ses file that yu pen in SmarteScript t run the test). T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins (see example in screensht): [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Page 26 f 144

28 Test Script This is nt used with the SmarteScript Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases SmarteScript des nt supprt the passing f input test parameters s the SmarteScript autmatin engine des nt supprt this feature f SpiraTeam r RemteLaunch Executing the SmarteScript Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the SmarteScript autmated test cases and click n its hyperlink t display the test set details page: Page 27 f 144

29 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the SmarteScript test: Passed The SmarteScript autmated test ran successfully and all the test cnditins in the test script passed Failed The SmarteScript autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The SmarteScript autmated test did nt run successfully Page 28 f 144

30 If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser r applicatin windws launch as SmarteScript executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by SmarteScript, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm SmarteScript tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside SmarteScript, and the executin status crrespnds the matching status inside SmarteScript. Cngratulatins Yu are nw able t run SmarteScript autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 29 f 144

31 4. TestCmplete Engine SmarteBear TestCmplete (hereafter TestCmplete) is an autmated functinal test autmatin system that lets yu recrd applicatin peratins and generate test autmatin scripts in a variety f languages (JavaScript, C#, VBScript). These test scripts can then be used t playback the test script against the test applicatin and verify that it wrks crrectly. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f TestCmplete n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated TestCmplete tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 8.0 f TestCmplete Installing the TestCmplete Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the TestCmpleteAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate TestCmpleteX.dll r TestExecuteX.dll fr the versin f TestCmplete r TestExecute that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file TestCmpleteX.dll r TestExecuteX.dll (where X is the apprpriate versin) int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Page 30 f 144

32 Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr TestCmplete this shuld be TestCmpleteX where X is the versin number f the DLL file that yu are using. Fr TestExecute this shuld be TestExecuteX where X is the versin number f the DLL file that yu are using. Nte: We nly use the majr versin numbers fr the tken name. S the DLLs TestCmplete9.0.dll and TestCmplete9.1.dll wuld bth use Tken = TestCmplete9. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with TestCmplete listed as an available autmatin engine Advanced Settings Yu can mdify the TestCmplete cnfiguratin fr each f the specific autmatin hsts, by right-clicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The TestCmplete engine adds its wn tab t this page which allws yu t cnfigure hw TestCmplete perates: The fllwing fields can be specified n this screen: Wait Time This shuld be set t the amunt f time TestCmplete needs n this wrkstatin t clse the currently pen test. The default value is 10000ms (10 secnds). If yu get errr messages that TestCmplete is still pen, yu need t increase this value. Applicatin Visible This allws yu t cnfigure whether the TestCmplete applicatin is displayed during test executin r is kept hidden. The default is fr it t be hidden. Clse TC After Each Test When this is selected, the plugin will clse the TestCmplete applicatin after each test executes. We generally recmmend leaving this disabled as the startup and clsedwn f TestCmplete can smetimes interfere with the tests being executed. Page 31 f 144

33 4.2. Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated TestCmplete test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the TestCmplete Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with TestCmplete nly supprts referencing TestCmplete test prject/suite file paths and nt physically uplading the test scripts int SpiraTeam. Filename This is actually a cmpund f several different cmpnents that need t be entered, separated by the pipe ( ) symbl. The syntax depends n whether we want t assciate the SpiraTeam test case with a specific prject item r with a specific test rutine. If yu want t use parameterized test cases, yu need t link it with a specific rutine (see belw fr mre details n parameters). If yu want t execute a specific prject item, the filename shuld cnsist f Suite Filename Prject Name Prject Item Name (e.g. [CmmnDcuments]\TestCmplete 7 Samples\Open Apps\OrdersDem\C#\TCPrject\Orders.pjs Orders_C#_C#Script Prjec ttestitem1) If yu want t execute a specific test rutine, the filename shuld cnsist f Suite Filename Prject Name Unit Name Rutine Name (e.g. [CmmnDcuments]\TestCmplete 7 Samples\Scripts\Hell\Hell.pjs Hell_C#Script hell_cs Hell) In the case f executing a specific test rutine, the last cmpnent (the rutine name) is actually the name f the functin in the test script itself. As illustrated in the examples, fr the Test Suite filename, yu can use several cnstants fr standard Windws lcatins t make things easier: [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. Page 32 f 144

34 [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the TestCmplete Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur TestCmplete autmated test script. This is very useful if yu have a data-driven TestCmplete test script that accepts input parameters. T use this feature yu need t use the ptin described abve t link the SpiraTest test case t an explicit test rutine inside TestCmplete. If yu chse the ptin t link t a Prject Item, any parameters passed will be ignred. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : Since the parameters in SpiraTeam map t the functin arguments inside a TestCmplete test script the parameter names need t match the rder f the arguments inside TestCmplete (i.e. they are matched by psitin/rder nt by name). Therefre we recmmend using numbers fr the parameter names s that it s easy t see which parameter value will be passed t which argument in the test script functin Executing the TestCmplete Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: Page 33 f 144

35 3. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 4. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the TestCmplete autmated test cases and click n its hyperlink t display the test set details page: Page 34 f 144

36 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Page 35 f 144

37 Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Page 36 f 144

38 Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the TestCmplete test: Passed The TestCmplete autmated test ran successfully and all the test cnditins in the test script passed Failed The TestCmplete autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The TestCmplete autmated test did nt run successfully If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser r applicatin windws launch as TestCmplete executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by TestCmplete, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm TestCmplete tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside TestCmplete, and the executin status crrespnds t the matching status inside TestCmplete as illustrated belw: Page 37 f 144

39 TestCmplete Status Passed Failed Warning SpiraTest Status Passed Failed Cautin In additin, the detailed test reprt frm TestCmplete is available in the Cnsle Output text-bx belw. It will cntain messages such as: Fr the mst detail, the Test Steps sectin will cntain a step-by-step breakdwn f each actin taken in the autmated test: Screensht Capture During the executin f the test, TestCmplete will capture screenshts f the applicatin being tested. These screenshts are upladed t SpiraTest s that yu have a cmplete recrd f the testing activities: Cngratulatins Yu are nw able t run TestCmplete autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 38 f 144

40 5. Selenium Engine Selenium Remte Cntrl (RC) is a test tl that allws yu t write autmated web applicatin user interface tests in any prgramming language against any HTTP website using any mainstream JavaScript-enabled brwser 1. Selenium RC cmes in tw parts. A server which can autmatically launch and kill supprted brwsers, and acts as a HTTP prxy fr web requests frm thse brwsers. Client applicatins that send cmmands t the Selenium-RC server in a special language (called Selenese) that tell it what peratins t perfrm n the launched web brwser. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f Selenium-RC (hereafter just referred t as Selenium) n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated Selenium web tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 1.0 f Selenium- Remte Cntrl Installing the Selenium Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the SeleniumAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate SeleniumX.dll fr the versin f Selenium that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file SeleniumX.dll (where X is the apprpriate versin) int the extensins sub-flder f the RemteLaunch installatin. Als cpy the ThughtWrks.Selenium.Cre.dll frm the zipfile int the extensins sub-flder. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. 1 Selenium RC Hme Page. OpenQA < Page 39 f 144

41 Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Selenium this shuld be SeleniumX where X is the versin number f the DLL file that yu are using. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with Selenium listed as an available autmatin engine Advanced Settings Yu can mdify the Selenium cnfiguratin fr each f the specific autmatin hsts, by right-clicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. a) Selenium-RC 1.0 The Selenium 1.0 engine adds its wn tab t this page which allws yu t cnfigure hw Selenium perates: Page 40 f 144

42 The fllwing fields can be specified n this screen: Server Hst This shuld be the name / IP address f the Selenium server. Typically this will be lcalhst because RemteLaunch is usually installed n the Selenium server itself. Server Prt This shuld be set t the custm prt that the Selenium server uses as a prxy when intercepting requests t the brwser. The default value is Brwser String This needs t be the name f the brwser that the Selenium server will launch. Cmmn values include: *firefx This will launch the Firefx web brwser *iexplre This will launch the Micrsft Internet Explrer web brwser *safari This will launch the Apple Safari web brwser Brwser URL This needs t be the initial URL that yu want the brwser t pen t a) Selenium WebDriver 2.x The Selenium 2.x engine adds its wn tab t this page which allws yu t cnfigure hw Selenium perates: Page 41 f 144

43 The fllwing fields can be specified n this screen: Brwser Type This needs t be set t the type f brwser that the Selenium webdriver will launch. Brwser URL This needs t be the initial URL that yu want the brwser t pen t 5.2. Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and either linking it t an existing Selenium test script file r entering a Selenium test script directly int SpiraTeam Attaching a Selenium Test Script First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the Selenium Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Attached fr this case Filename Since the test script is ging t be entered directly int SpiraTeam yu can enter any name yu like fr the filename as lng as it s lgical and memrable. Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This needs t cntain the cmplete Selenium test script written in Selenium IDE Selenese. Selenium IDE test scripts cnsist f three parts: The cmmand The target f the cmmand The data t be used Yu shuld enter the three cmpnents n each line separated by the Pipe ( ) character. If yu need t use a pipe character inside any f the cmpnents yu can escape it with a backslash (\ ). Page 42 f 144

44 An example cmmand wuld be type q hell If the cmmand desn t need all three cmpnents, yu can simply leave it ut (fr example pen If yu wuld like t have SpiraTeam pass any parameter values t this test script (this will be discussed in mre detail later) yu can specify them by using the syntax ${parametername}. An example parameterized cmmand wuld be pen ${url} A cmplete sample script is illustrated belw: pen asserttitle Ggle type q ${query} click btng waitfrpagetlad 5000 istextpresent ${matchtext} Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Linking a Selenium Test Script First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the Selenium Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked fr this case Filename This needs t be the full path t the Selenium IDE test script file. T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins: [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Page 43 f 144

45 Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used when yu are using the linked test script ptin The linked test script needs t be an HTML dcument that cntains a table with three clumns. Each rw crrespnds t a single Selenium actin. Each f the clumns in the rw crrespnds t the three Selenium cmmand cmpnents: The cmmand The target f the cmmand The data t be used An example Selenium test script is illustrated belw: <html> <bdy> <table> <tr> <td> pen </td> <td> </td> <td> </td> </tr> <tr> <td> asserttitle</td> <td> </td> <td> Ggle</td> </tr> <tr> <td> type</td> <td> q</td> <td> ${query}</td> </tr> <tr> <td> click</td> <td> btng</td> <td> </td> </tr> <tr> <td> waitfrpagetlad</td> <td> </td> <td> Page 44 f 144

46 5000</td> </tr> <tr> <td> istextpresent</td> <td> </td> <td> ${matchtext}</td> </tr> </table> </bdy> </html> When pened in an HTML editing tl it lks like: pen asserttitle Ggle type q ${query} click btng waitfrpagetlad 5000 istextpresent ${matchtext} If yu wuld like t have SpiraTeam pass any parameter values t this test script (this will be discussed in mre detail later) yu can specify them by using the syntax ${parametername}. An example parameterized cmmand is displayed in the third and sixth rws f the table abve (${query} and ${matchtext}). Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur Selenium autmated test script. This is very useful if yu want t have a data-driven Selenium test script that be executed multiple times with different parameter values. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : Page 45 f 144

47 The name f the parameter ${lgin} needs t match the name f the input parameter defined within the Selenium script Executing the Selenium Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the Selenium autmated test cases and click n its hyperlink t display the test set details page: Page 46 f 144

48 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Page 47 f 144

49 Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their Page 48 f 144

50 list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the Selenium test: Passed The Selenium autmated test ran successfully and all the test cnditins in the test script passed Failed The Selenium autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The Selenium autmated test did nt run successfully If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser windws launch as the Selenium server executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by Selenium, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm Selenium tgether with any messages r ther infrmatin. The executin status will be set t PASSED if all the Selenium cmmands reprt back OK and all the tests passed. If any f the cmmands failed r the tests dn t pass, the verall executin status will be listed as FAILED. Page 49 f 144

51 The Message field will cntain a summary f the number f cmmands executed and the number f failed cmmands, with the large details bx cntaining the full cmmand executin lg as reprted back frm Selenium: pen (, - OK asserttitle (, Ggle) - OK type (q, Philmene Lng) - OK click (btng, ) - OK waitfrpagetlad (, 5000) - OK istextpresent (, Philmene Lng) - OK,true Cngratulatins Yu are nw able t run Selenium autmated web tests and have the results be recrded within SpiraTest / SpiraTeam. Page 50 f 144

52 6. Squish Engine Frglgic Squish (hereafter Squish) is a functinal test autmatin system that lets yu recrd applicatin peratins and generate test autmatin scripts in a variety f different scripting languages (JavaScript, Tcl, Pythn) that can be used t playback the test script against the test applicatin. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f Squish n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated Squish tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 4.0 f Squish running n a Windws platfrm Installing the Squish Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the SquishAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate SquishX.dll fr the versin f Squish that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file SquishX.dll (where X is the apprpriate versin) int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Page 51 f 144

53 Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Squish this shuld be SquishX where X is the versin number f the DLL file that yu are using. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with Squish listed as an available autmatin engine Squish RemteLaunch Settings Yu will need t mdify the Squish cnfiguratin fr each f the specific autmatin hsts, by rightclicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The Squish engine adds its wn tab t this page which allws yu t cnfigure hw Squish perates: The fllwing fields can be specified n this screen: Squish Lcatin This shuld be flder cntaining the SquishRunner executable that will be used t actually run the autmated tests. Server Hst This field can be set t the name f a remte Squish server if yu did nt install RemteLaunch n the machine running the Squish server (ptinal). Server Prt This field can be set t the prt being used by a remte Squish server if yu did nt install RemteLaunch n the machine running the Squish server (ptinal). Trace Lgging This checkbx can be selected if yu need t prvide debugging infrmatin t Inflectra supprt persnnel. Nrmally this shuld remain unchecked Nte: In mst cases, the secnd and third fields can be left empty Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and either linking it t an existing Squish test suite, test case r entering a Squish test script directly int SpiraTeam. Nte: that the Squish engine nly supprts passing parameters t an attached test script and nt t a linked test script. Page 52 f 144

54 Attaching a Squish Test Script First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the Squish Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Attached fr this case Filename Since the test script is ging t be entered directly int SpiraTeam yu can enter any filename yu like as lng as the file extensin matches the scripting language that yu re using. After that yu need t add any cmmand-line parameters after the filename, separated by a pipe ( ) symbl. Fr example, t launch a web test using Javascript, yu d use: address_test.js --wrapper Web Fr example, t launch an applicatin test using Pythn, yu d use: address_test.py --aut <applicatin> Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This needs t cntain the cmplete Squish test script. Squish test scripts can be written in JavaScript, Pythn r TCL. If yu wuld like t have SpiraTeam pass any parameter values t this test script (this will be discussed in mre detail later) yu can specify them by using the syntax ${parametername}. Page 53 f 144

55 A cmplete sample script (illustrating the use f parameters) is illustrated belw: functin main() { // pen URL ladurl(": // wait fr the first entry bject t be available waitfrobject(":_id0:title_select-ne"); // check that the submit buttn is disabled test.cmpare(findobject(":_id0:submit_image").disabled, true); // enter data selectoptin(":_id0:title_select-ne", "${title}"); settext(":_id0:firstname_text", "${firstname}"); settext(":_id0:lastname_text", "${lastname}"); settext(":_id0:city_text", "${city}"); // check that after entering city, the state is autmatically chsen crrectly var state = "${state}"; setfcus(":_id0:state_text"); if (!test.verify(waitfr("findobject(':_id0:state_text').value == state", 10000))) { clickbuttn(":_id0:reset_image"); cntinue; } // input ZIP selectoptin(":_id0:zipselect_select-ne", "${zip}"); // check that submit buttn is enabled nw setfcus(":_id0:lastname_text"); if (!test.verify(waitfr("findobject(':_id0:submit_image').disabled == false", 10000))) { clickbuttn(":_id0:reset_image"); } // submit clickbuttn(":_id0:submit_image"); // wait fr results page waitfrcntextexists(":respnse.iface"); waitfrobject(":_id1:_id3_span"); // verify that data is stred and displayed crrectly test.cmpare(findobject(":_id1:_id3_span").innertext, firstname); test.cmpare(findobject(":_id1:_id6_span").innertext, state); // clse brwser clsewindw(":[windw]"); } Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Linking a Squish Test Script First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Page 54 f 144

56 Yu need t enter the fllwing fields: Autmatin Engine - Chse the Squish Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked fr this case Filename This needs t be the full path t the Squish test case r test suite flder. If specifying a test case flder, yu need t als prvide the cnfiguratin cmmand-line parameters after the filename, separated by a pipe ( ) symbl. These are nt needed if executing a test suite, since they are cntained in the suite.cnf file instead. Fr example, t launch a web test case yu d use: [PrgramFiles]\Frglgic\squish webwin32\examples\web\suite_examples\tst_icefaces_addressbk_ datadriven --wrapper Web Fr example, t launch a web test suite yu d simply use: [PrgramFiles]\Frglgic\squish webwin32\examples\web\suite_examples Fr example, t launch a web test case within a test suite yu d use the path f the test suite, fllwed by the pipe ( ) symbl, fllwed by the test case name: [PrgramFiles]\Frglgic\squish webwin32\examples\web\suite_examples tst_icefaces T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins: [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Page 55 f 144

57 Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used when yu are using the linked test script ptin Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur attached (nt linked) Squish autmated test script. This is very useful if yu want t have a data-driven Squish test script that be executed multiple times with different parameter values. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${city} needs t match the name f the parameter defined within the attached Squish script Executing the Squish Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Page 56 f 144

58 Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the Squish autmated test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Page 57 f 144

59 Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Page 58 f 144

60 Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the Squish test: Passed The Squish autmated test ran successfully and all the test cnditins in the test script passed Failed The Squish autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The Squish autmated test did nt run successfully If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see applicatin r brwser windws launch as the Squish server executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by Squish, yu will see the fllwing infrmatin: Page 59 f 144

61 This screen indicates the status f the test run that was reprted back frm Squish tgether with any messages r ther infrmatin. The executin status will be set accrding t the wrst-case assessment reprted back frm Squish. The varius Squish statuses are mapped t their nearest equivalent SpiraTeam statuses as illustrated belw: Squish Status PASS FAIL WARNING FATAL ERROR SpiraTeam Status Passed Failed Cautin Blcked Failed In additin, the Message field will cntain a summary f the number f tests cmpleted and the number f tests that reprted an errr, fatal, fail, pass r warning status T16:50:01-04:00 START_TEST suite_examples T16:50:01-04:00 START_TEST tst_icefaces_addressbk_datadriven T16:50:05-04:00 PASS Cmparisn 'true' and 'true' are equal T16:50:25-04:00 PASS Verified True expressin T16:50:30-04:00 PASS Verified True expressin T16:50:32-04:00 PASS Cmparisn 'Reginald' and 'Reginald' are equal T16:50:33-04:00 PASS Cmparisn 'CA' and 'CA' are equal T16:50:40-04:00 PASS Cmparisn 'true' and 'true' are equal T16:50:59-04:00 PASS Verified True expressin T16:51:09-04:00 PASS Verified True expressin T16:51:12-04:00 PASS Cmparisn 'Tanja' and 'Tanja' are equal T16:51:12-04:00 PASS Cmparisn 'NY' and 'NY' are equal Cngratulatins Yu are nw able t run Squish autmated web tests and have the results be recrded within SpiraTest / SpiraTeam. Page 60 f 144

62 7. Cmmand-Line Engine In additin t the varius pre-built plug-ins fr different test autmatin engines, there is a generic cmmand-line engine available that lets RemteLaunch execute an arbitrary cmmand-line prgram, capture the cnsle utput and send the utput back t SpiraTeam as the test results. This is useful when yu want t be able t use SpiraTeam t manage the scheduling and executin f autmated testing using an in-huse tl r a third-party tl that Inflectra has nt yet built a plug-in fr. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f a cmmand-line applicatin n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmatin Nte: This integratin requires at least versin 4.0 f SpiraTest/Team and RemteLaunch Installing the Cmmand-Line Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the CmmandLineAutmatinEngine.zip file frm the Inflectra website and lcate the CmmandLine.dll Cpy the file CmmandLine.dll int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Cmmand-Line this shuld be simply CmmandLine. Page 61 f 144

63 Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with Cmmand-Line listed as an available autmatin engine Cmmand-Line RemteLaunch Settings Yu may need t mdify the Cmmand-Line cnfiguratin fr sme f the specific autmatin hsts, by right-clicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The Cmmand-Line engine adds its wn tab t this page which allws yu t cnfigure hw the Cmmand-Line engine perates: The fllwing fields can be specified n this screen: RunAs Administratr This nrmally shuld nt be checked. Hwever if yur autmatin tl requires Windws UAC elevatin t perate, yu will need t select this ptin. We recmmend initially trying yur tl with the value unchecked. Then, if yu get an errr message requires elevatin in the test results yu will need t select the ptin. Lg Results Nrmally the cmmand-line engine will capture the utput results frm the cmmand-line and send the results back t SpiraTeam as the test result. When yu are executing a tl that directly integrates with SpiraTeam (e.g. a NUnit test suite that is already integrated with SpiraTeam) yu dn t want tw different results t be sent back. In such a scenari, deselecting this ptin will prevent the cmmand-line engine frm sending back its wn test result. Default Status This specifies the executin status that will be returned t SpiraTeam in the event that nne f the regular expressins (Regex) specified match the results returned frm the test applicatin. By default, the system will return Passed if nne f the ther regular expressins match crrectly. Pass Regex This is the regular expressin that is used t match a passed test result. By default the system will search fr the phrase Pass in the test utput and return a Passed status if the match is successful. Fail Regex This is the regular expressin that is used t match a failed test result. By default the system will search fr the phrases Fail, Errr and Fatal in the test utput and return a Fail status if any f the matches are successful. Page 62 f 144

64 Cautin Regex This is the regular expressin that is used t match a cautin test result. By default the system will search fr the phrases Warning and Cautin in the test utput and return a Cautin status if any f the matches are successful. Blcked Regex This is the regular expressin that is used t match a blcked test result. By default the system will search fr the phrase Blcked in the test utput and return a Blcked status if the match is successful. Test Regular Expressins This text bx lets yu enter in sme sample text and see hw the Cmmand-Line extensin wuld interpret it. Once yu have entered in the text, click Test Regular Expressin and the system will display a ppup message bx letting yu knw what the utcme f such a test wuld be interpreted as: 7.2. Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and either linking it t an existing test script file r entering a test script directly int SpiraTeam Attaching a Cmmand-Line Test Script First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the Cmmand-Line Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Attached fr this case Filename This needs t cnsist f the fllwing sectins separated by a pipe ( ) character: The full path t the cmmand-line tl. T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins: [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. Page 63 f 144

65 [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Any arguments fr the cmmand-line tl, with the filename specified as ${filename}. This special tken will be replaced by the actual filename f the test script when RemteLaunch dwnlads it frm SpiraTeam. In additin, yu can use the fllwing additinal tkens fr sme f the special SpiraTeam ID values: [TestCaseId] the ID f the test case [TestSetId] the ID f the test set [ReleaseId] the ID f the release (if specified) [PrjectId] the ID f the prject An example filename wuld be: C:\Temp\TestApp.exe -arg1 -arg2 "-arg3=${filename}" Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This needs t cntain the cmplete test script in whatever language and syntax is being expected by the cmmand-line applicatin If yu wuld like t have SpiraTeam pass any parameter values t this test script (this will be discussed in mre detail later) yu can specify them by using the syntax ${parametername} inside the test script. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Linking a Cmmand-Line Test Script First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Page 64 f 144

66 Autmatin Engine - Chse the Cmmand-Line Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked fr this case Filename This needs t cnsist f the fllwing sectins separated by a pipe ( ) character: The full path t the cmmand-line tl. T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins: [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Any arguments fr the cmmand-line tl, including the filepath f the test script file that the cmmand-line tl will be executing. In additin, yu can use the fllwing additinal tkens fr sme f the special SpiraTeam ID values: [TestCaseId] the ID f the test case [TestSetId] the ID f the test set [ReleaseId] the ID f the release (if specified) [PrjectId] the ID f the prject The mask fr cnverting any parameter values frm SpiraTeam int valid cmmand line arguments. If parameters are nt accepted by the cmmand-line tl, yu can leave this sectin ut. The mask can include any symbls tgether with name t refer t the parameter name and value t refer t the parameter value. Example 1: If yu want parameters t prvided in the frm: -param1=value1 param2=value2 yu wuld use the fllwing mask: -name=value Example 2: If yu want parameters t prvided in the frm: /param1:value1 /param2:value2 yu wuld use the fllwing mask: /name:value An example filename wuld be: C:\Temp\TestApp.exe -arg1 -arg2 -name=value Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Page 65 f 144

67 Test Script This is nt used when yu are using the linked test script ptin Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur cmmand-line autmated testing tl. This is very useful if yu want t have a datadriven test script that be executed multiple times with different parameter values. T setup the autmated test case fr parameters, click n the Edit Parameters hyperlink abve the Test Script bx: The name f the parameter ${lgin} needs t match the name f a parameter accepted by the cmmandline tl Executing the Cmmand-Line Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field Page 66 f 144

68 must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the Cmmand-Line autmated test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Page 67 f 144

69 Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Page 68 f 144

70 Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the cmmand-line test: Passed The autmated test ran successfully and the results utput t the cnsle did nt include any f the phrases FAIL, ERROR, FATAL, WARNING, CAUTION Failed The autmated test ran successfully, but ne f the phrases FAIL, ERROR, FATAL was included in the cnsle utput Cautin The autmated test ran successfully, but ne f the phrases WARNING, CAUTION was included in the cnsle utput Blcked The autmated test did nt run successfully If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see applicatin windws launch as the cmmand-line tl server executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by the cmmand-line tl, yu will see the fllwing infrmatin: Page 69 f 144

71 This screen indicates the status f the test run that was reprted back frm cmmand-line tl tgether with any messages r ther infrmatin. The executin status will be set accrding t the rules described abve, the Message field will cntain the first line f cnsle utput and the large details bx will cntain the full cnsle utput frm the cmmand-line tl. Cngratulatins Yu are nw able t run a custm cmmand-line run tests and have the results be recrded within SpiraTest / SpiraTeam. Page 70 f 144

72 8. LadRunner 11 Engine HP LadRunner is a lad testing system that lets yu recrd applicatin perfrmance by a number f virtual users. This sectin cvers installing and using the Engine t reprt back statistics f run scenaris. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 11 f LadRunner. The extensin has nt been tested n previus versins f LadRunner due t lack f availability f previus released versins Installing the LadRunner11 Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the LadRunner11Engine.zip file frm the Inflectra website. Cpy the files in the ZIP file int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users, and will be displayed in the drpdwn when the user selects the Tester. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr LadRunner, it needs t be LadRunner11. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with LadRunner listed as an available autmatin engine. Page 71 f 144

73 8.2. Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t a Lad Runner Scenari. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the LadRunner Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type Fr LadRunner, all scenaris must be stred n the lcal testing machine s Linked must be selected. If yu select Attached, when the scenari is attempted t be executed it will be marked as blcked and skipped. Filename This needs t be the full path t the LadRunner Scenari (*.lrs) file. Certain tkens are allwed t be able t specify cmmn lcatins acrss different perating systems. Nte that the tkens are case-sensitive, and there are n spaces in them. A list f tkens are: [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated scenari will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated scenari will be stred in. Versin The versin f the scenari (1.0 is used if n value specified) Test Script Nt used. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin. Page 72 f 144

74 Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur LadRunner scenari. This is very useful if yu have a data-driven LadRunner scenari that has custm parameters used that yu wuld like t change based n the test. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${lgin} needs t match the name f the custm parameter defined in the LadRunner scenari. Invalid parameters will be silently ignred by the LadRunner engine. Parameters must have a unique name Executing the LadRunner Scenari frm SpiraTeam There are tw ways t execute a scenari in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Page 73 f 144

75 Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified as the Hst name in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the LadRunner test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Page 74 f 144

76 Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Page 75 f 144

77 Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be executed. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the LadRunner executin: Passed The Scenari ran and reprted n errr messages. Warning, Debug, and Infrmatinal messages may have been lgged, hwever. Failed The Scenari ran and at least ne errr message was reprted. Blcked There was an errr with the Test Set r LadRunner applicatin. If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser r applicatin windws launch as LadRunner runs the scenari and cnnects VUsers t their tasks. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by LadRunner, yu will see the fllwing infrmatin: Page 76 f 144

78 This screen indicates the status f the scenari that was reprted back frm LadRunner tgether with any messages r ther infrmatin. Because LadRunner nly reprts statistics n the scenaris that was run, the test set will always be marked as passed regardless f hw lng it tk, unless there were errrs reprted. If any errrs are reprted, the test will be marked as Failed. The Message f the test will reprt the duratin the scenari tk, alng with the cunt f VUsers that reprted an errr, the number that reprted a pass, and the hits per minute n the applicatin. A mre detailed reprt will be included in the test run s details the infrmatin abve, and then any added Executin Messages and messages lgged by the script in time rder. Nte: LadRunner s engine is very basic at this stage. If yu have issues with a scenari nt reprting r executing prperly, please let Inflectra s supprt team knw. Page 77 f 144

79 9. SapUI Engine SmartBear SapUI (hereafter SapUI) is an pen surce Web Service testing tl fr Service Oriented Architectures (SOAs). There is als a Pr versin that is released as a cmmercial prduct. Its functinality mainly cvers Web Service Inspectin, Invking, Develpment, Simulatin and Mcking, Functinal testing, Lad and Cmpliance testing. Prductivity enhancement features can be fund in the sapui pr versin. 2 This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f sapui n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated web service testing. Nte: This integratin requires at least versin 4.0 f SpiraTest/Team and an instance f SapUI r SapUI Pr running n a Windws platfrm Installing the SapUI Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the SapUIEngine.zip file frm the Inflectra website. Extract the file sapuiengine.dll frm the cmpressed archive int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. 2 Page 78 f 144

80 Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr sapui this shuld be SapUI. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with SapUI listed as an available autmatin engine SapUI RemteLaunch Settings Yu will need t mdify the SapUI cnfiguratin fr each f the specific autmatin hsts, by rightclicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The SapUI engine adds its wn tab t this page which allws yu t cnfigure hw SapUI perates: The fllwing fields can be specified n this screen: SOAP-UI Lcatin This shuld be SOAP-UI Bin flder that cntains the TestRunner.bat batch file that will be used t actually run the autmated tests. Installatin Type This allws yu t take advantage f the enhanced reprting available in the cmmercial Pr editin f SapUI. Check the SOAP-UI Pr Installatin bx nly if yu are using the cmmercial versin f SapUI (knwn as SapUI Pr). Executin Type If this is a LadUI perfrmance test rather than a standard SapUI functinal test, check the bx and RemteLaunch will knw t parse the lad-test reprt frmat. Trace Lgging Nrmally this can be left unchecked unless yu are diagnsing cnfiguratin issues and need additinal lgging Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an existing SapUI test suite and test case. Page 79 f 144

81 Linking a SapUI Test Script First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and expand the Autmatin sectin f the Test Case Overview tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the SapUI Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked fr this case Filename This needs t be the full path t the SapUi test prject XML file r cmpsite flder tgether with the test suite name and test case name separated by the pipe ( ) symbl. Yu can als pass custm cmmand line switches as an ptinal final segment Fr standard tests, yu use the frmat: Prject XML File Test Suite Name Test Case Name Switches Fr cmpsite flder tests, yu use the frmat: Test Flder Test Suite Name Test Case Name Switches Fr example if the test suite was named Requirements Testing and the test case was named Get Requirements yu d use: [MyDcuments]\SpiraTest-4-0-Web-Service-sapuiprject.xml Requirements Testing Get Requirements T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins: [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type Yu can chse which dcument type the autmated test script will be categrized under. Page 80 f 144

82 Dcument Flder Yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used when yu are using the linked test script ptin Nte: The example filename shwn abve was taken frm a test prject in SapUI that has the fllwing structure: Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur SapUI autmated test. This is very useful if yu have a data-driven SapUI test that has custm prject prperties used that yu wuld like t change based n the test. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${lgin} needs t match the name f the custm parameter defined in the SapUI prject prperties. Invalid parameters will be silently ignred by the SapUI engine. Parameters must have a unique name. Nte that the plugin currently nly supprts Prject Prperties and nt Glbal r System Prperties. Page 81 f 144

83 9.3. Executing the SapUI Test Sets frm SpiraTeam There are three ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. 3. Execute the test cases frm the cmmand-line r a build script We shall utline each f these three scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the SapUI autmated test cases and click n its hyperlink t display the test set details page: Page 82 f 144

84 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Page 83 f 144

85 Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their Page 84 f 144

86 list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. If yu want t run the tests as part f a build script, just call RemteLaunch.exe with the apprpriate test set id passed int the cmmand-line: RemteLaunch.exe testset:18 In all cases, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the SapUI test: Passed The SapUI autmated test ran successfully and all the assertins in the test script passed Failed The SapUI autmated test ran successfully, but at least ne assertin in the test script failed. Blcked The SapUI autmated test did nt run successfully If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see applicatin r brwser windws launch as the SapUI server executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by SapUI, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm SapUI tgether with any messages r ther infrmatin. The executin status will be set accrding t the wrst-case assessment reprted back frm SapUI. If yu have zer(0) failures, then the status will display as Passed, therwise it will display as Failed. Under Cnsle Output sectin yu will see mre detailed lgging infrmatin (in bth SapUI and SapUI Pr): Page 85 f 144

87 The Message field will cntain a summary f the number f test steps cmpleted, the number f assertins and the number f failed assertins. The Details field will cntain the detailed trace f what happened, captured frm the summary utput lg that is generated by SapUI. SapUI Pr If yu have the cmmercial SapUI Pr prduct and have cnfigured RemteLaunch s that it knws t use SapUI Pr, in additin, the Test Steps sectin f the test run will cntain mre detailed reprting: Where each test step crrespnds t a step recrded in the SapUI Pr results file. Cngratulatins Yu are nw able t run SapUI autmated web-service tests and have the results be recrded within SpiraTest / SpiraTeam. Page 86 f 144

88 10. FitNesse Engine FitNesse is a lightweight, pen-surce autmated sftware testing framewrk that uses web-based Wikis t define the inputs and expected results frm different cmbinatins f input values and then cmpare the results with what is actually generated during testing. Fr mre details n FitNesse, please refer t the FitNesse website: This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f FitNesse n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated FitNesse acceptance tests. Nte: This integratin requires at least versin 4.0 f SpiraTest/Team and RemteLaunch Installing the FitNesse Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the FitNesseEngine.zip file frm the Inflectra website and cpy the file FitNesseEngine.dll int the extensins sub-flder f the RemteLaunch installatin. Yu may als need t verify that yu have the full Micrsft.NET Framewrk 4.0 installed since that is needed by the FitNesse engine. RemteLaunch itself nly needs the.net 4.0 Client Prfile, s make sure yu have the.net 4.0 Framewrk Extended entry listed in the Prgram & Features sectin f the Windws Cntrl Panel. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr FitNesse this shuld be simply FitNesse. Page 87 f 144

89 Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with FitNesse listed as an available autmatin engine Advanced Settings Yu can mdify the FitNesse cnfiguratin fr each f the specific autmatin hsts, by right-clicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The FitNesse engine adds its wn tab t this page which allws yu t cnfigure hw FitNesse perates: The fllwing fields can be specified n this screen: Server Hst This shuld be the base URL fr accessing the installatin f FitNesse. Each f the FitNesse test cases will be a URL relative t this base URL. Server Prt This shuld be set t the TCP prt that the FitNesse web server uses fr displaying the FitNesse wiki web pages. FitNesse Timeut This allws yu t extend the timeut fr executing FitNesse tests. This is useful if yu find that the FitNesse tests take a lng time t execute and RemteLaunch is abrting the executin befre they are finished Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an existing FitNesse test case wiki page. Nte: The FitNesse autmatin engine nly supprts Linked test scripts in SpiraTeam (nt Attached). First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and g t the Autmatin sectin lcated in the Overview tab: Page 88 f 144

90 Yu need t enter the fllwing fields: Autmatin Engine - Chse the FitNesse Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked fr FitNesse tests. Filename This needs t be the relative URL f the FitNesse test case. I.e. if the FitNesse URL is and the base URL setup in RemteLaunch is then the filename wuld be just FitNesse.UserGuide.TwMinuteExample. Dcument Type Yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder Yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used when yu are using the linked test script ptin Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases The FitNesse autmatin engine des nt currently supprt the passing f parameter values frm SpiraTeam t the FitNesse test Executing the FitNesse Test Sets frm SpiraTeam There are three ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. 3. Execute the test cases frm the cmmand-line r a build script We shall utline each f these three scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Page 89 f 144

91 Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the FitNesse autmated test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Page 90 f 144

92 Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be executed. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the FitNesse test: Passed The FitNesse autmated test ran successfully and all the test cnditins in the test script passed Failed The FitNesse autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The FitNesse autmated test did nt run successfully If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see cmmand windws appear as the FitNesse server executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by FitNesse, yu will see the fllwing infrmatin: Page 91 f 144

93 This screen indicates the status f the test run that was reprted back frm FitNesse tgether with any messages r ther infrmatin. The executin status will be set t PASSED if all the FitNesse rws reprt back OK and all the tests passed. If any f the rws failed r the tests dn t pass, the verall executin status will be listed as FAILED. Yu can see a step-by-step recrd f what happened by scrlling dwn t the Test Steps sectin: In additin, yu can scrll dwn t the Cnsle Output sectin t get the FitNesse specific infrmatin: The Message field will cntain a summary f the number f tests executed and the number f wrng results and exceptins. The large details bx cntains the full cmmand executin lg as reprted back frm FitNesse: Cngratulatins Yu are nw able t run FitNesse autmated acceptance tests and have the results be recrded within SpiraTest / SpiraTeam. Page 92 f 144

94 11. NeLad Engine Netys NeLad is a perfrmance and lad testing system that lets yu recrd applicatin perfrmance by a number f virtual users and measure the perfrmance against specified Service Level Agreement (SLA) metrics fr the applicatin. When yu use NeLad with SpiraTest yu can reprt back pass/fail/cautin by cmparing the actual results against the specified SLA metrics. This sectin cvers installing and using the Engine t reprt back statistics f run scenaris as well as the results f the test cmpared t the required SLAs. Nte: This integratin requires at least versin 4.0 f SpiraTest/Team and has been tested against versin 5.0 f NeLad Installing the NeLad Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the NeLadEngine.zip file frm the Inflectra website. Cpy the files in the ZIP file int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users, and will be displayed in the drpdwn when the user selects the Tester. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr NeLad, it needs t be simply NeLad. Page 93 f 144

95 Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with NeLad listed as an available autmatin engine NeLad RemteLaunch Settings Yu will need t mdify the NeLad cnfiguratin fr each f the specific autmatin hsts, by rightclicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The NeLad engine adds its wn tab t this page which allws yu t cnfigure hw NeLad perates: The fllwing fields can be specified n this screen: NeLad Lcatin This shuld be flder cntaining the NeLadCmd.exe executable that will be used t actually run the autmated tests. Attach PDF Reprt NeLad has a built-in reprt generatr that can create detailed Acrbat (PDF) frmat reprts. Enabling this ptin will attach these reprts t the test runs recrded in SpiraTeam Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t a NeLad prject and scenari. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and g t the Autmatin sectin in the main Overview tab: Page 94 f 144

96 Yu need t enter the fllwing fields: Autmatin Engine - Chse the NeLad Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type Fr NeLad, all scenaris must be stred n the lcal testing machine s Linked must be selected. If yu select Attached, when the scenari is attempted t be executed it will be marked as blcked and skipped. Filename This needs t be the full path t the NeLad prject file (*.nlp) file fllwed by the name f the NeLad scenari. The tw cmpnents need t be separated by a pipe ( ) character. Certain tkens are allwed t be able t specify cmmn lcatins acrss different perating systems. Nte that the tkens are case-sensitive, and there are n spaces in them. A list f tkens are: [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type Yu can chse which dcument type the autmated scenari will be categrized under. Dcument Flder Yu can chse which dcument flder the autmated scenari will be stred in. Versin The versin f the scenari (1.0 is used if n value specified) Test Script Nt used. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases Currently the NeLad autmatin engine des nt supprt the passing f parameter values frm SpiraTeam t NeLad. Page 95 f 144

97 11.3. Executing the NeLad Scenari frm SpiraTeam There are three ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. 3. Execute the test cases frm the cmmand-line r a build script We shall utline each f these three scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified as the Hst name in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the NeLad test cases and click n its hyperlink t display the test set details page: Page 96 f 144

98 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be executed. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the NeLad executin: Passed The scenari ran and reprted n errr messages and all SLAs were passed. Cautin The scenari ran and at least ne SLA reprted back as acceptable Failed The scenari ran and at least ne errr message was reprted r at least ne SLA was reprted back as failed. Blcked There was an errr with the Test Set r NeLad applicatin. Page 97 f 144

99 If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser r applicatin windws launch as NeLad runs the scenari and cnnects VUsers t their tasks. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by NeLad, yu will see the fllwing test run summary infrmatin: This sectin f the screen indicates hw lng the test tk t execute, the verall status, which release was being executed, which test set it was a part f and each f the key summary statistics, tgether with infrmatin n hw they cmpared t the defined SLA: N/A There was n SLA defined fr this metric Passed There is an SLA defined fr this metric and it was passed. Cautin There is an SLA defined fr this metric and it was cnsidered less than a pass, but still acceptable. Failed There is an SLA defined fr this metric and it was nt met successfully. In additin, if yu scrll dwn, in the Cnsle Output sectin f the reprt there is mre detailed infrmatin: Page 98 f 144

100 The Message f the test will reprt the number f ttal pages, number f ttal hits, number f ttal users, number f errrs as well as the ttal cunt f virtual users. In additin, mre detailed infrmatin is displayed in the test run details: Tp 5 errrs by page Tp 5 alerts by page Tp 5 average respnse times by page Tp 5 maximum respnse times by page Finally, if yu have chsen the ptin t attach the NeLad PDF reprt, in the Attachments sectin f the Test Run, that will be listed: Cngratulatins Yu are nw able t run autmated NeLad perfrmance scenaris and have the results be recrded within SpiraTest / SpiraTeam. Page 99 f 144

101 12. TestPartner Engine Micr Fcus TestPartner (hereafter TestPartner) is a Graphic User Interface (GUI) functinal test autmatin system that lets yu recrd applicatin peratins by capturing the varius testable bjects f the applicatin and then playback the peratins t autmatically test the applicatin. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f TestPartner n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated TestPartner tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 6.0 f TestPartner Installing the TestPartner Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the TestPartnerAutmatinEngine.zip file frm the Inflectra website and lcate the TestPartner.dll inside the zip archive. Cpy the file TestPartner.dll int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr TestPartner this shuld just be TestPartner. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with TestPartner listed as an available autmatin engine. Page 100 f 144

102 12.2. Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated TestPartner test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the TestPartner Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with TestPartner nly supprts referencing TestPartner test scripts (stred in the internal database) and nt physically uplading the test scripts int SpiraTeam. Filename This needs cntain the prject and test name frm TestPartner with the apprpriate parameter name describing which is the prject name and which is the test name. The test name can be either a test script f a visual test. The syntax is: -visualtest <test name> -prject <prject name> r -testscript <script name> -prject <prject name> Dcument Type If using SpiraTeam (nt SpiraTest) yu can chse which dcument type the autmated test script will be categrized under. Dcument Flder If using SpiraTeam (nt SpiraTest) yu can chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the TestPartner Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases TestPartner des nt supprt the passing f input test parameters s the TestPartner autmatin engine des nt supprt this feature f SpiraTeam r RemteLaunch. Page 101 f 144

103 12.3. Executing the TestPartner Test Sets frm SpiraTeam There are three ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. 3. Execute the test cases frm the cmmand-line r a build script We shall utline each f these three scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the TestPartner autmated test cases and click n its hyperlink t display the test set details page: Page 102 f 144

104 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the TestPartner test: Passed The TestPartner autmated test ran successfully and all the test cnditins in the test script passed Failed The TestPartner autmated test ran successfully, but at least ne test cnditin in the test script failed. Blcked The TestPartner autmated test did nt run successfully Page 103 f 144

105 If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu may see brwser r applicatin windws launch as TestPartner executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by TestPartner, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm TestPartner tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside TestPartner, and the executin status crrespnds the matching status inside TestPartner. Cngratulatins Yu are nw able t run TestPartner autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 104 f 144

106 13. BadBy Engine Badby is an autmated website functinal test autmatin system that lets yu recrd website peratins in Internet Explrer and generate test autmatin scripts that can be used t playback the test script against the website. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f Badby n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated Badby tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 2.1 f Badby Installing the Badby Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the BadbyAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate BadbyX.dll fr the versin f Badby that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file BadbyX.dll (where X is the apprpriate versin) int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Page 105 f 144

107 Badby this shuld be BadbyX where X is the versin number f the DLL file that yu are using. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with Badby listed as an available autmatin engine Advanced Settings Yu can mdify the Badby cnfiguratin fr each f the specific autmatin hsts, by right-clicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The Badby 2.x engine adds its wn tab t this page which allws yu t cnfigure hw Badby perates: The fllwing fields can be specified n this screen: Trace Lgging When selected, this will lg additinal trace and debugging infrmatin t the Windws Event Lg. This shuld nt be selected in a prductin envirnment Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated Badby test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Page 106 f 144

108 Yu need t enter the fllwing fields: Autmatin Engine - Chse the Badby Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with Badby nly supprts referencing Badby test script files and nt physically uplading the test scripts int SpiraTeam. Filename This needs t be the full path t the Badby test script. T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins (see example in screensht): [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Dcument Type This allws yu t chse which dcument type the autmated test script will be categrized under. Dcument Flder This allws yu t chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the Badby Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur Badby autmated test script. This is very useful if yu have a data-driven Badby test script that defines input variables frm an external data surce. Page 107 f 144

109 T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${lgin} needs t match the name f the variable defined within the Badby script in its variables cnfiguratin Executing the Badby Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Page 108 f 144

110 Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the Badby autmated test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Page 109 f 144

111 Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Page 110 f 144

112 Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the Badby test: Passed The Badby autmated test ran successfully and all the test steps in the test script passed and n assertins were thrwn. Failed The Badby autmated test ran successfully, but at least ne test step failed r at least ne assertin failed. Cautin The Badby autmated test run successfully, but at least ne warning was lgged in ne f the test steps. Blcked The Badby autmated test did nt run successfully r at least ne timeut errr was recrded. If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu will see brwser windws launch as Badby executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by Badby, yu will see the fllwing infrmatin: Page 111 f 144

113 This screen indicates the status f the test run that was reprted back frm Badby tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside Badby and the executin status crrespnds the matching status inside Badby as illustrated belw: Badby Status Succeeded Failure Warning Assertin Timeut SpiraTeam Status Passed Failed Cautin Failed Blcked In additin, the detailed test reprt frm Badby is available in the large text-bx belw. It will cntain messages such as: Suite: Test Suite 1 ============================================== Test: Test played, 12 succeeded, 0 failures, 0 assertins, 0 warnings, 0 timeuts Step: Step played, 12 succeeded, 0 failures, 0 assertins, 0 warnings, 0 timeuts Cngratulatins Yu are nw able t run Badby autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 112 f 144

114 14. JMeter Engine Apache JMeter is a free, pen surce Java desktp applicatin designed t lad test functinal behavir and measure perfrmance. It was riginally designed fr testing Web Applicatins but has since expanded t ther test functins. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f JMeter n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated JMeter perfrmance tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 2.5 f JMeter Installing the JMeter Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the JMeterEngine.zip file frm the Inflectra website and lcate the apprpriate JMeterX.dll fr the versin f JMeter that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file JMeterX.dll (where X is the apprpriate versin) int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr JMeter this shuld be JMeterX where X is the versin number f the DLL file that yu are using. Page 113 f 144

115 Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with JMeter listed as an available autmatin engine Advanced Settings Yu can mdify the JMeter cnfiguratin fr each f the specific autmatin hsts, by right-clicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The JMeter 2.x engine adds its wn tab t this page which allws yu t cnfigure hw JMeter perates: The fllwing fields can be specified n this screen: JMeter Lcatin This shuld pint t the lcatin n the hst cmputer where JMeter is installed. Yu can click n the brwse buttn and navigate t the lcatin f the JMeter.bat file. Trace Lgging When selected, this will lg additinal trace and debugging infrmatin t the Windws Event Lg. This shuld nt be selected in a prductin envirnment Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated JMeter test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Page 114 f 144

116 Yu need t enter the fllwing fields: Autmatin Engine - Chse the JMeter Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with JMeter nly supprts referencing JMeter test script files and nt physically uplading the test scripts int SpiraTeam. Filename This cnsists f the fllwing elements: The full path t the JMeter test script. T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins (see example in screensht): [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. Optinally yu can include JMeter cmmand-line arguments by separating them with a pipe ( ) character. Examples f Filenames yu can enter in SpiraTeam include: [MyDcuments]JMeter\JMeter-SampleScript.jmx [MyDcuments]JMeter\JMeter-SampleScript.jmx -P 81 [MyDcuments]JMeter\JMeter-SampleScript.jmx -P 81 -H Dcument Type This allws yu t chse which dcument type the autmated test script will be categrized under. Page 115 f 144

117 Dcument Flder This allws yu t chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the JMeter Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur JMeter autmated test script. This is very useful if yu have a data-driven JMeter test script that expects specific JMeter prperties t be passed t the test script. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${lgin} needs t match the name f the prperty defined within the JMeter script Executing the JMeter Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Page 116 f 144

118 Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the JMeter autmated test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Page 117 f 144

119 Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Page 118 f 144

120 Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be executed. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the JMeter test: Passed The JMeter autmated test ran successfully and n failures r errrs were lgged. Failed The JMeter autmated test ran successfully, but at least ne errr r failure was lgged. Blcked The JMeter autmated test did nt run successfully. If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu will see a Windws cmmand prmpt pen as JMeter executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by JMeter, yu will see the fllwing infrmatin: Page 119 f 144

121 This screen indicates the status f the test run that was reprted back frm JMeter tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside JMeter and the executin status crrespnds the rules described abve. In additin, the detailed test reprt frm JMeter is available in the large text-bx belw. It will cntain messages such as: Respnse Assertin ( failure=true, errr=false, message='test failed: text expected t cntain /(?i)purchase Our Prducts Online/' Respnse Assertin ( failure=true, errr=false, message='test failed: text expected t cntain /(?i)purchase Our Prducts Online/' Respnse Assertin ( failure=false, errr=false, message='' Respnse Assertin ( failure=false, errr=false, message='' Cngratulatins Yu are nw able t run JMeter autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 120 f 144

122 15. Ranrex Engine Ranrex is an autmated functinal test autmatin system that lets yu recrd applicatin peratins and generate.net language (C#, VB.NET) test autmatin scripts that can be used t playback the test script against the test applicatin. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f Ranrex n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated Ranrex tests. This plugin was develped by ne f ur partners (step2it GmbH) but has been frmally tested by Inflectra and is fully supprted by Inflectra. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team and versin 3.0 f Ranrex Installing the Ranrex Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the RanrexAutmatinEngine.zip file frm the Inflectra website and lcate the apprpriate RanrexAutmatinEngine.dll fr the versin f Ranrex that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file RanrexAutmatinEngine.dll int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Page 121 f 144

123 Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr Ranrex this shuld always be RanrexEngine. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with Ranrex listed as an available autmatin engine: Advanced Settings Yu can mdify the Ranrex cnfiguratin fr each f the specific autmatin hsts, by right-clicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The Ranrex engine adds its wn tab t this page which allws yu t cnfigure hw Ranrex perates: The fllwing fields can be specified n this screen: Result Path This is the flder where the results f Ranrex tests will be stred. The currently lgged-in user needs t have Read/Write permissins ver this flder. Trace Lgging When selected, this will lg additinal trace and debugging infrmatin t the Windws Event Lg. This shuld nt be selected in a prductin envirnment. Page 122 f 144

124 15.2. Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated Ranrex test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the Ranrex Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with Ranrex nly supprts referencing Ranrex test script files and nt physically uplading the test scripts int SpiraTeam. Filename This needs t be the full path t the Ranrex test suite. T make this easier acrss different machines, yu can use several cnstants fr standard Windws lcatins (see example in screensht): [MyDcuments] The user s My Dcuments flder. The user indicated is the user that ran RemteLaunch. [CmmnDcuments] The Public Dcument s flder. [DesktpDirectry] The user s Desktp flder. The user indicated is the user that ran RemteLaunch. [PrgramFiles] Translated t the Prgram Files directry. Fr 64-bit machines, it s the 64-bit directry. [PrgramFilesX86] Translated t the 32-bit Prgram Files directry. If yu d like t pass parameters t Ranrex yu can specify them by separating them frm the filename with a pipe ( ) character. Fr example t run a specific Ranrex test case yu can use the fllwing filename : c:\test\mytestsuit.exe /testcase:adddatatest Dcument Type This allws yu t chse which dcument type the autmated test script will be categrized under. Dcument Flder This allws yu t chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the Ranrex Engine since it nly supprts linked test scripts. Page 123 f 144

125 Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur Ranrex autmated test suite. This is very useful if yu have a data-driven Ranrex test suite that defines input variables frm an external data surce. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${lgin} needs t match the name f the variable defined within the Ranrex script in its variables cnfiguratin Executing the Ranrex Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Page 124 f 144

126 Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the Ranrex autmated test cases and click n its hyperlink t display the test set details page: Lwer dwn, the list f test cases in the test set are displayed: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Page 125 f 144

127 Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu can set their values in three different ways: Test Set Parameter Values this lets yu set the same value f a parameter fr all the test cases in the test set: Test Case Parameter Values this lets yu set a specific value fr a parameter fr a particular test case in the test set: Page 126 f 144

128 Yu set these values, by right-clicking n a rw and chsing Edit Parameters : Test Cnfiguratins this lets yu create a data grid f pssible test parameters and execute the test set multiple times, nce fr each unique cmbinatin: Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the Ranrex test: Page 127 f 144

129 Passed The Ranrex autmated test ran successfully and all the test steps in the test script passed and n assertins were thrwn. Failed The Ranrex autmated test ran successfully, but at least ne test step failed r at least ne assertin failed. Cautin The Ranrex autmated test run successfully, but at least ne warning was lgged in ne f the test steps. Blcked The Ranrex autmated test did nt run successfully r at least ne timeut errr was recrded. If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu will see brwser windws launch as Ranrex executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by Ranrex, yu will see the fllwing infrmatin: This screen indicates the status f the test run that was reprted back frm Ranrex tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside Ranrex and the executin status crrespnds the matching status inside Ranrex as illustrated belw: Ranrex Status Success Failed SpiraTeam Status Passed Failed Page 128 f 144

130 In additin, the detailed test reprt frm Ranrex is available in the large text-bx belw. It will cntain messages such as: [2011/10/14 14:08:32.795][Debug ][Lgger]: Cnsle lgger starting. [2011/10/14 14:08:32.874][Inf ][Test]: Test Suite 'WinFrms Test' started. [2011/10/14 14:08:32.889][Inf ][Test]: Test Case 'VS2005_Applicatin_Test' started. [2011/10/14 14:08:33.467][Success][Test]: Test Mdule 'StartWinfrmsSample' cmpleted with status 'Success'. [2011/10/14 14:08:33.467][Inf ][Test]: Test Mdule 'CheckIfApplicatinAlive' started. [2011/10/14 14:08:33.608][Inf ][Validatin]: Validating Exists n item 'WinFrmsApp.ButtnTest_PushButtn'. [2011/10/14 14:09:55.718][Failure][Test]: Test Case 'VS2005_Applicatin_Test' cmpleted with status 'Failed'. [2011/10/14 14:09:55.718][Failure][Test]: Test Suite 'WinFrms Test' cmpleted with status 'Failed'. Cngratulatins Yu are nw able t run Ranrex autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 129 f 144

131 16. Ratinal Functinal Tester IBM Ratinal Functinal Tester (hereafter RFT) is sftware test autmatin tl used by quality assurance teams t perfrm autmated regressin testing. Testers create scripts by using a test recrder which captures a user's actins against their applicatin under test. The recrding mechanism creates a test script frm the actins. The test script is prduced as either a Java r Visual Basic.net applicatin. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f RFT n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated RFT tests. Nte: This integratin requires at least versin 3.0 f SpiraTest/Team Installing the RFT Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the RFTEngine.zip file frm the Inflectra website and lcate the apprpriate RFTAutmatinEngine.dll fr the versin f RFT that yu are using. If yu dn t see the versin listed, just use the nearest versin that is lwer than yur current versin. Cpy the file RFTAutmatinEngine.dll int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Page 130 f 144

132 Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr RFT this shuld always be RFTAutmatinEngine. Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with RFT listed as an available autmatin engine Advanced Settings Yu can mdify the RFT cnfiguratin fr each f the specific autmatin hsts, by right-clicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The RFT engine adds its wn tab t this page which allws yu t cnfigure hw RFT perates: The fllwing fields can be specified n this screen: RFT Lcatin this is where the installatin f RFT can be fund. Typically it s C:\Prgram Files\IBM\SDP\FunctinalTester\bin Wrkspace Lcatin This is the flder where the RFT test scripts and generated lg files will be stred. The currently lgged-in user needs t have Read/Write permissins ver this flder. Typically it s: Page 131 f 144

133 C:\Dcuments and Settings\[User Name]\IBM\ratinalsdp\wrkspace n a Windws XP wrkstatin r Windws 2003 server. C:\Users\[User Name]\IBM\ratinalsdp\wrkspace n a Windws Vista, 7, 2008 r 2008 R2 cmputer. Trace Lgging When selected, this will lg additinal trace and debugging infrmatin t the Windws Event Lg. This shuld nt be selected in a prductin envirnment Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated RFT test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Autmatin tab: Yu need t enter the fllwing fields: Autmatin Engine - Chse the RFT Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with RFT nly supprts referencing RFT test script files and nt physically uplading the test scripts int SpiraTeam. Filename This needs t cnsist f the fllwing three cmpnents separated by a pipe ( ) character (see the screensht fr an example): The name f the RFT prject that the test is mapped t The name f the RFT script in the prject that the test is mapped t Either java r net depending n whether yu have a Java r.net test script Dcument Type This allws yu t chse which dcument type the autmated test script will be categrized under. Dcument Flder This allws yu t chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the RFT Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin. Page 132 f 144

134 Using Parameterized Test Cases There is an advanced feature f SpiraTest/Team and RemteLaunch that lets yu pass parameters frm SpiraTeam t yur RFT autmated test suite. This is very useful if yu have a data-driven RFT test suite that defines input variables frm an external data surce. T setup the autmated test case fr parameters, click n the Test Steps tab and click n Edit Parameters : The name f the parameter ${lgin} is actually nt used when passing the data t RFT, nly the values are passed. Therefre it s imprtant that the parameters are stred in the rder they are expected by yur RFT test script Executing the RFT Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Page 133 f 144

135 Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the RFT autmated test cases and click n its hyperlink t display the test set details page: Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing If yu have parameterized test cases inside the autmated test set yu need t set their values by rightclicking n the test case and chsing Edit Parameters : Page 134 f 144

136 Enter the parameter values and click Update t cmmit the change. This allws yu t have the same test case in the test set multiple times with different data fr each instance Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be execute. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the RFT test: Passed The RFT autmated test ran successfully and all the test steps in the test script passed and n assertins were thrwn. Failed The RFT autmated test ran successfully, but at least ne test step failed r at least ne assertin failed. Cautin The RFT autmated test run successfully, but at least ne warning was lgged in ne f the test steps. Blcked The RFT autmated test did nt run successfully. If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu will see brwser windws launch as RFT executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by RFT, yu will see the fllwing infrmatin: Page 135 f 144

137 This screen indicates the status f the test run that was reprted back frm RFT tgether with any messages r ther infrmatin. The Test Name indicates the name f the test inside RFT and the executin status crrespnds the matching status inside RFT as illustrated belw: RFT Status PASS FAIL WARNING SpiraTeam Status Passed Failed Cautin In additin, the detailed test reprt frm RFT is available in the large text-bx belw. It will cntain messages such as: 07-Nv :00: PM: Script Start - INFORMATION - Script start [Script1] 07-Nv :00: PM: Simplified Script Grup - INFORMATION - firefx.exe: self imprvement - QuickStart Tutrials fr Ratinal Functinal Tester (RFT) - Stack Overflw - Mzilla Firefx 07-Nv :00: PM: Timer Start - INFORMATION - Start timer: firefxexeselfimprvementquicksta_1 07-Nv :00: PM: General - WARNING - Object Recgnitin is weak (abve the warning threshld) 07-Nv :00: PM: General - FAIL - Script1.testMain had an unhandled exceptin. 07-Nv :00: PM: Script End - FAIL - Script end [Script1] Exceptin ccurred during playback f script [Script1] [CRFCN0019E: RatinalTestScriptExceptin n line 49 f script Script1 - cm.ratinal.test.ft.objectntfundexceptin: CRFCN0661W: The recgnitin scre f the fund bject des nt qualify the bject as a match. Lking fr [GuiSubitemTestObject(Name: gtawebsitetext, Map: GTAWebSite)], best failing candidate scre was [22500] with best failing descriptin [{.class=.text,.name=g t a Web Site,.classIndex=0}].]. Cngratulatins Yu are nw able t run RFT autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 136 f 144

138 17. TestingAnywhere TestingAnywhere is a pwerful and easy t use autmated sftware testing tl that allws users t autmate any type f testing. Pwerful GUI based recrding capabilities and a n-prgramming required user interface allws testers t quickly set up even cmplex test cases. A built-in editr allws users t build tests that can be easily edited t allw fr changes in the test cases. This sectin describes hw yu can use SpiraTest / SpiraTeam (hereafter SpiraTeam) tgether with RemteLaunch t schedule and remtely launch instances f TestingAnywhere n different cmputers and have the testing results be transmitted back t SpiraTeam. This allws yu t extend yur SpiraTeam s test management capabilities t include autmated TestingAnywhere tests. Nte: This integratin requires at least versin 4.0 f SpiraTest/Team and RemteLaunch Installing the TestingAnywhere Engine This sectin assumes that yu already have a wrking installatin f SpiraTest r SpiraTeam and have installed RemteLaunch n the varius test autmatin hsts fllwing the instructins in Sectin 1 (abve). Once thse prerequisites are in place, please fllw these steps: Dwnlad and extract the TestingAnywhereEngine.zip file frm the Inflectra website and lcate the TestingAnywhereAutmatinEngine.dll file cntained within. Cpy the file TestingAnywhereAutmatinEngine.dll int the extensins sub-flder f the RemteLaunch installatin. Lg in t SpiraTeam as a system administratr and g int SpiraTeam main Administratin page and click n the Test Autmatin link under Integratin. Click the Add buttn t enter the new test autmatin engine details page. The fields required are as fllws: Name: This is the shrt display name f the autmatin engine. It can be anything that is meaningful t yur users. Descriptin: This is the lng descriptin f the autmatin engine. It can be anything that is meaningful t yur users. (Optinal) Active: If checked, the engine is active and able t be used fr any prject. Tken: This needs t be the assigned unique tken fr the autmatin engine and is used t tell RemteLaunch which engine t actually use fr a given test case. Fr TestingAnywhere this shuld simply be TestingAnywhere. Page 137 f 144

139 Once yu have finished, click the Insert & Clse buttn and yu will be taken back t the Test Autmatin list page, with TestingAnywhere listed as an available autmatin engine Advanced Settings Yu can mdify the TestingAnywhere cnfiguratin fr each f the specific autmatin hsts, by rightclicking n the RemteLaunch icn in the system tray and chsing Cnfiguratin. That will bring up the RemteLaunch cnfiguratin page. The TestingAnywhere engine adds its wn tab t this page which allws yu t cnfigure hw TestingAnywhere perates: The fllwing fields can be specified n this screen: Player Lcatin this is the flder where the TestingAnywhere player (TAPlayer.exe) can be fund. Typically it s C:\Prgram Files (x86)\testing Anywhere 9.0\Testing Anywhere Files Lcatin This is the flder where the TestingAnywhere test scripts and generated lg files will be stred. The currently lgged-in user needs t have Read/Write permissins ver this flder. Typically it s: C:\Dcuments And Settings\[UserName]\My Dcuments\Testing Anywhere Files n a Windws XP wrkstatin r Windws 2003 server. Page 138 f 144

140 C:\Users\[UserName]\Dcuments\Testing Anywhere Files n a Windws Vista, 7, 2008 r 2008 R2 cmputer. Trace Lgging When selected, this will lg additinal trace and debugging infrmatin t the Windws Event Lg. This shuld nt be selected in a prductin envirnment Setting up the Autmated Test Cases This sectin describes the prcess fr setting up a test case in SpiraTeam fr autmatin and linking it t an autmated TestingAnywhere test script. First yu need t display the list f test cases in SpiraTeam (by clicking Testing > Test Cases) and then add a new test case. Once yu have added the new test case, click n it and select the Overview tab, and scrll dwn t the Autmatin sectin: Yu need t enter the fllwing fields: Autmatin Engine - Chse the TestingAnywhere Autmatin Engine that yu created in the previus sectin frm the drp-dwn list. Script Type This shuld be set t Linked as the integratin with TestingAnywhere nly supprts referencing TestingAnywhere test script files and nt physically uplading the test scripts int SpiraTeam. Filename This needs t cnsist f the relative lcatin f the TestingAnywhere test script t the test script rt flder. Dcument Type This allws yu t chse which dcument type the autmated test script will be categrized under. Dcument Flder This allws yu t chse which dcument flder the autmated test script will be stred in. Versin The versin f the test script (1.0 is used if n value specified) Test Script This is nt used with the TestingAnywhere Engine since it nly supprts linked test scripts. Once yu are happy with the values, click [Save] t update the test case. Nw yu are ready t schedule the autmated test case fr executin Executing the TestingAnywhere Test Sets frm SpiraTeam There are tw ways t execute autmated test cases in SpiraTeam: 1. Schedule the test cases t be executed n a specific cmputer (lcal r remte) at a date/time in the future Page 139 f 144

141 2. Execute the test cases right nw n the lcal cmputer. We shall utline bth f these tw scenaris in this sectin. Hwever first we need t setup the apprpriate autmatin hsts and test sets in SpiraTeam: Cnfiguring the Autmatin Hsts and Test Sets G t Testing > Autmatin Hsts in SpiraTeam t display the list f autmatin hsts: Make sure that yu have created an Autmatin Hst fr each cmputer that is ging t run an autmated test case. The name and descriptin can be set t anything meaningful, but the Tken field must be set t the same tken that is specified in the RemteLaunch applicatin n that specific machine. Once yu have at least ne Autmatin Hst cnfigured, g t Testing > Test Sets t create the test sets that will cntain the autmated test case: Nte: Unlike manual test cases, autmated test cases must be executed within a test set they cannt be executed directly frm the test case. Create a new Test Set t hld the TestingAnywhere autmated test cases and click n its hyperlink t display the test set details page: Page 140 f 144

142 Yu need t add at least ne autmated test case t the test set and then cnfigure the fllwing fields: Autmatin Hst This needs t be set t the name f the autmatin hst that will be running the autmated test set. Planned Date The date and time that yu want the scenari t begin. (Nte that multiple test sets scheduled at the exact same time will be scheduled by Test Set ID rder.) Status This needs t be set t Nt Started fr RemteLaunch t pick up the scheduled test set. When yu change the Planned Date, the status autmatically switches back t Nt Started Type This needs t be set t Autmated fr autmated testing Executing the Test Sets Once yu have set the varius test set fields (as described abve), the Remte Launch instances will peridically pll SpiraTeam fr new test sets. Once they retrieve the new test set, they will add it t their list f test sets t be executed. Once executin begins they will change the status f the test set t In Prgress, and nce test executin is dne, the status f the test set will change t either Cmpleted the autmatin engine culd be launched and the test has cmpleted r Blcked RemteLaunch was nt able t start the autmatin engine. If yu want t immediately execute the test case n yur lcal cmputer, instead f setting the Autmatin Hst, Status and Planned Date fields, yu can instead click the [Execute] icn n the test set itself. This will cause RemteLaunch n the lcal cmputer t immediately start executing the current test set. In either case, nce all the test cases in the test set have been cmpleted, the status f the test set will switch t Cmpleted and the individual test cases in the set will display a status based n the results f the TestingAnywhere test: Passed The TestingAnywhere autmated test ran successfully and all the test steps in the test script passed and n assertins were thrwn. Failed The TestingAnywhere autmated test ran successfully, but at least ne test step failed r at least ne assertin failed. Cautin The TestingAnywhere autmated test run successfully, but at least ne warning was lgged in ne f the test steps. Blcked The TestingAnywhere autmated test did nt run successfully. If yu receive the Blcked status fr either the test set r the test cases yu shuld pen up the Windws Applicatin Event Lg n the cmputer running RemteLaunch and lk in the event lg fr errr messages. Nte: While the tests are executing yu will see brwser windws launch as TestingAnywhere executes the apprpriate tests. Once the tests have cmpleted, yu can lg back int SpiraTeam and see the executin status f yur test cases. If yu click n a Test Run that was generated by TestingAnywhere, yu will see the fllwing high-level test infrmatin: Page 141 f 144

143 This screen indicates the status f the test run that was reprted back frm TestingAnywhere tgether with the executin date/time. If yu scrll dwn t the Cnsle Output sectin, yu will see: Finally, t see the detailed test steps, lk under the Test Steps sectin: Cngratulatins Yu are nw able t run TestingAnywhere autmated functinal tests and have the results be recrded within SpiraTest / SpiraTeam. Page 142 f 144