BASIC USER TRAINING PROGRAM Module 5: Test Case Development

BASIC USER TRAINING PROGRAM Module 5: Test Case Development

Objective Student will have an understanding of how to create, edit and execute a Test Case from Develop a Test Case Activity Page. Student will learn the various properties and components of the test case. Outline Test case from Add Steps by capturing Editing test case and manipulating Steps Executing Test Case and Response View Review Test Reports Lab Quiz Questions TEST CASE FROM ADD STEPS BY CAPTURING 1. Go to Manage Workspace and create a folder under my_project/test_cases. Name it Module 5--Test Case Development Click Finish. 2. Let s create another folder called saved_test_reports under my_project for reporting purposes.

3. Click Finish 4. To create a test case, go back to the Home page and click Develop a Test Case. The Develop Test Case wizard starts. You first specify whether to work on an existing test case or to create a new one. To work on an existing test case, you will navigate to the document and click OK. 5. For this module, let s select Create a new test case. The dialog box will change based on your selection. Specify the following settings -

1. Save in - Navigate to the project and folder to create the test case in. Create a test case in the default project named my_project > default folder named test_cases > in the folder named Module 5 Test Case Development. 2. File name - Type a name for the file name. For the name of the test case, enter module5_testcase. 3. Owner It is optional. Type the unique identifier of the person responsible for developing and/or maintaining the test case (typically, the name, login name, or email address) Default is current username. Let s type Fanfare Training 4. Headline- Type Module 5 Test Case. This is optional and generally a one line description that documents the usage and function of the test case. 5. Description Type This test case is developed in Module 5 This is optional and to type additional text that describes the test case to make its usage clear to coworkers. Note: This is an excellent place to paste a copy of the test plan. 6. Associate a topology with the test case - Specify a topology to use for the sessions in the test case. When you specify a topology, test documentation includes the topology. When a topology is used for execution, an informational execution message will appear immediately after the execution start

message to help you keep track of which topology was used. The message identifies the fully qualified URI of the topology that is being used. In addition, the URI appears in the header section of the test report. Let us specify ff_device_topology to use the Router Telnet Session and Linux SSH Session that were created in the previous modules and added into the devices. 7. Click OK. The Develop a test case activity page opens. Notice that the names of the Test Case and the Topology used for the test case appears on the top left side of the Develop a Test Case activity page. Test case and topology tabs will appear at the top of the open area. If the tabs get closed, you can double click the filename to open the file in the appropriate editor Click Change the topology to associate a different topology. The Develop a Test Case page provides quick access to the following operations:

1. Adding steps by capturing (start sessions with devices and save the captured steps into the test case) 2. Execute the test case 3. Add analysis rules that validate a value in a response or store it into a variable 4. View test reports for the current test case Adding steps by capturing In this module, we will use the Capture view to create and add steps to our test case. In order to build our test case lets first prototype the test manually. We will use the following test as our example: Open a connection to Router Open a connection to Linux box Check to see who is logged in on the Linux box Check the following on the Router o version and flash on Router o Check the arp table o Display the IP Interface table o Show the status of vlan1 o Display the ICMP received echo counter on the Router Use the Linux box to send 5 pings to the router Recheck the ICMP received echo counter on the router to ensure the count has increased appropriately Close the connection to the Router Close the connection to the Linux box We have associated the Test Case to ff_device_topology to use the router telnet session and Linux ssh session that were created in the previous modules and added into the devices. 1. On the Develop a test case activity page, click Add steps by capturing. The Add steps to Test Case page opens. 2. The current test case module5_testcase is identified at the top of the page. You will notice that the topology ff_device_topology file is associated with the test case is also identified on the left. Two new views open on the left side of the Start a Session page Sessions view - Captured sessions will be listed in the Sessions table Steps view - Captured steps are listed in the Steps table. Follow these steps to prototype your test: 1. Notice that the module5_testcase has already been opened in the Test Case editor. 2. Go to the General tab of the open test case and confirm that the ff_device_topology topology has been associated with the test case.

3 Start a session with a device: Click Start a Session. The Start a Session dialog box displays all devices defined in the topology that you associated with the test case. Navigate the tree to select the linux_ssh session and then click Start. 4. Using the session profile settings that are defined for the device, itest launches a session on the device and you will notice the captured sessions are listed in the Sessions table. Captured steps are listed in the Steps table.

5. To insert the comments while capturing the commands, select a step and click to add a Comment step after the selected step. The comment step is added to the test case along with the captured steps. a. Select the step and click the Add Comment button b. In the Add Comment dialog box, type the comment and click OK

4. Execute the whoami command 5. You can open as many sessions on as many devices as you like. Let s click Start a Session in the Sessions window The Start a Session dialog box displays all devices defined in the topology that you associated with the test case. Navigate to select the cisco_telnet session and then click Start. 3. Using the session profile settings that are defined for the device, itest launches a session on the device. Log in to router and enter fanfare at the Password prompt. Notice that the captured steps are listed in the Steps table and captured sessions appear in the Sessions table.

4. Execute the following commands as listed below on the Router session and add the comments before sending each command (refer to the below screen shot)- a. show version b. show flash c. show arp d. show ip int brief e. show ip int Vlan1 f. show ip traffic On the show ip traffic command, if you see the More prompt, you can use the spacebar to page through to the completion of the response. 5. In the Linux_box session, execute the following command: ping c 5 ffcisco1.fnfr.com 6. In the Router session, execute the following commands:

a. show ip traffic b. exit The capture history will look like this after you insert the comments before the commands: Note: For maintainability and good communications, it is important that you use comments in your Test Case. You can add comment steps manually in the test case or you can capture the comments while capturing the commands. itest captures any commands that you perform whenever the capture indicator displays Capturing To pause the capturing, click Pause the capture indicator changes to While capture is paused, any comments that you submit in any session are not captured. To restart capture, click Capture button again and itest resumes adding steps to the Steps list. Click the Clear Steps button to clear all captured steps from the list, close all open sessions, and cancel the capture activity.

Create the test case from captured steps and comments When you are ready to add the captured steps from the Steps list into the test case, click Add Steps icon To not add a particular step or session into the test case, right-click the step and select Exclude Item. Excluded steps are not added to the test case. By default, excluded steps are not displayed in the list. To display excluded steps (shaded to indicate that they will not be added to the test case), right-click in the table and select Show Excluded Items. For new test cases, steps are added to the end of the test case. For existing test cases, the Insert Steps wizard prompts you to specify the location in the test case to add the steps.

Follow these steps to add captured steps into the Test Case- 1. Click Add Steps icon 2. When you click Finish, the steps are added to the test case. You have now successfully created your Test Case. You should see the following for your Test Case: Once again, to optimize the interface we will do two things. First, we auto-resize the steps grid in the Test Case editor:

1. Click the word procedure in the Action column. This is a read-only field so you should not notice anything change in the interface; we are simply ensuring that the interface is currently focused on the grid portion of the screen. 2. Press the F8 key (a shortcut key that resizes the test case grid. Next, you will notice the section labeled Procedure Properties. To minimize this section, click the words Procedure Properties. Note:It is critical to understand how the itest interface works so you can keep things simple and optimized for the task you are currently conducting. There is no need to keep all the views open when you are only using one or two of them for any given task. EDITING TEST CASE AND MANIPULATING STEPS Editing steps There are several ways to edit a step. First you must understand what you have selected when you click a step. Some fields are read-only while others are editable. If you click anywhere to the left of the step icon or on the step ID number you will see this part of the line is read-only. This is where you click when you want to select the entire step (for copy, cut, paste, etc.). If you click the action, session or description fields you will notice a cursor is displayed. You are now in edit mode on the specific cell. If you use copy at this point you are copying only the contents of the cell and not the entire step. For a menu of step editing options, right-click a step, use the Test Case menu, or use the toolbar. The tools help you to quickly insert new steps, move steps (up, down or indent), delete steps, add rules (discussed later), skip steps, apply a breakpoint, and so on. Shortcut keys are associated with these actions; for example Ctrl+Enter inserts a new test case step.

--------------------------------------------------------------------------------------------------------------------------------- Editing Step properties Each step in the test case has associated properties. Some property pages are common to all steps and some are added based on the session type for the selected step. If a step is selected you will notice the header text below the steps is labeled Step Properties (the header label/property section) is context sensitive to what you have selected in the test case grid. Click the header to expand or collapse the Step Properties section. If you select the show version step and expand the Step Properties section, you should see the following: Session-specific properties appear as the last item in the tree. In the example, the properties in the Telnet command Properties are identical to the properties in the session profile. If you set any of these values here you override the setting in the session profile. The session-specific properties for the step are inherited from the associated topology, testbed, or session profile and can be overridden in this section. For now we will focus on the General properties associated with a step.

The Session, Action, and Command properties are all currently displayed in the steps grid. Notice that you can skip the step, launch named threads, mask the command field and enable/disable substitution for the step. The Context and Target properties are specifically associated with GUI session types (Web, Swing, and Flex for example). You can use the Details button to add multi-line entries into the Command field. Wrapping associated steps under a comment step To organize steps and make the test case more readable, you can group associated steps under a comment step (the steps execute normally; they are merely indented under the comment step to improve readability). As a result, you can collapse or expand a group of steps under the parent comment step using and. When you collapse all steps, the test case reads like pseudo code so that a coworker can easily understand the operation of the test case. Let s indent the command steps under the comments added during the capture. 1. Click the step to highlight it 2. Click Increase Indent icon on the Test Case editor menu bar 3. Collapse the steps under the parent comment step using The Test Case would like this

You can also wrap the certain steps into the comment step using right click menu; Select group of steps, rightclick, and then select Wrap in Comment. After grouping the steps, type appropriate text for the comment into the Description cell.

Using comments to message out to the Execution view For a comment step, the system interprets the text in the Description field as a comment to the user. You have the option to specify that, at runtime, all comments in the test case should appear as execution messages in the Execution view and in the test report, but no other action is performed by the step. Go to the Test Case editor General page and select Generate an execution issue for every comment step executed. Note: Comments can also include field replacements. This is an easy way to send data to the Execution view. Comment steps are a great way to outline or pseudo-code a test case before adding actual commands. To add comments quickly, select a comment step and then press Ctrl-Enter. Once your outline is complete, you can go back and insert commands.

Skipping Steps While developing and troubleshooting a test case, it is often helpful to skip execution for particular steps (like steps that you know work or do not work or that take a long time to complete). Skipped steps are not executed and do not appear in reports. itest ignores breakpoints on skipped steps the steps are skipped without pausing execution. We will talk more about the break points in the advance modules. Note: Add a comment step and indent any number of related steps under the comment. You can then skip all of the steps by skipping only the comment step. In addition, you can collapse (fold) the comment step to temporarily hide the related steps. To skip and unskip steps 1 In the Test Case editor, select the steps (use Ctrl+click or Shift+click for multi-select). 2 Click Skip 3. Click the button again to unskip the step. In the steps grid, skipped steps are dimmed (grayed-out). In the example, steps 5 and 6 are skipped. EXECUTING TEST CASE AND RESPONSE VIEW Executing the test case 1 1. After opening or creating the test case, on the Develop a test case activity page, click Execute test case.

2. The Execute Test Case activity page opens. The current test case module5_testcase is identified at the top 2of the page and a topology ff_device_topology file associated with the test case, is also identified. 3To start execution, click. Use the following tools to control execution. You will notice that two sessions are executing the commands from our Test Case

When execution finishes, you will notice that both the sessions get closed. If you want to keep the sessions open after the execution completes, then uncheck the following setting under Windows>Preference>iTest>Execution

During execution, the Message view displays the severity and execution message that is generated for each execution issue. Icons represent the severity of the execution issues: Information, Warning, OK, and Error. Execution messages appears in the Message cell and describe execution events like test case pass/fail, results of analysis rules, execution pause/resume, and so on. Some execution messages are built-in (identifying the topology used for execution or generated by events for example, Execution started); others are defined by the test case developer (generated by analysis rule actions we will cover the analysis rules in the next module). Execution messages also appear in the Execution view and Step Issues view. As you notice the execution message tells us that Execution has started Which testbed used in execution The comments appear in the messages as we had checked the box Generate an execution issue for every comment step executed on Test Case editor General page. There is a warning saying that pass/fail criteria has not been set. The reason is because we have not set the analysis rules yet Click the message in the Message View, the execution window will open. For each execution issue, the Execution View and the Test Report view displays the following information

The time remaining bar displays the best estimate of the time remaining until the test finishes. The estimate is based on previous execution times and takes into account the optional execution speed setting. By default, tests run at the fastest possible speed. By default, the speed control does not appear in the Execution view. To display the control, set the preference setting as described in the itest > Execution section of the Preferences page. On the Preferences page, click itest > Views > Execution View. Check the box to display the speed control in the Execution view.

Default: unchecked You can change the execution speed setting during execution to any one of the following settings or any setting in between. Fast: Submit commands as quickly as the computer can send them while allowing each step run to completion before starting the next step. Asynchronous (concurrent) steps, in contrast, are executed at original speed. Original: Execute no faster than the original capture. Execution may be slightly slower because each step will start no sooner than the original delay from the previous step end, but may be later because it will wait for the previous step to complete before starting. Slow: Use this setting to give you a chance to observe behavior that you might miss at a higher speed. The Response view The Response view is one of the most useful views for developing test cases. As you select any step, the response returned during the most recent execution of the test (if any) is displayed in the Response view. Due to screen space you will generally not want to have the properties section expanded and the Response view docked. For now, collapse the Properties section of the Steps page and dock the Response view (which should be part of the view group that is minimized on the bottom of the interface).

Restore the view group and ensure that the Response view is selected. Select the show version step from the test case Open the Response view to see the response from the commands, For example, click the show version command and you can see the response in the response window.

The command response may not be populated. If this step has never been executed as part of a test case or was not associated with a step that was captured on this system the response may not currently be known. The Response view is populated by a response that is derived from either the last test report for this test case (if there is one) or the capture database (if this step was brought in from the capture history). This will most likely occur when you add steps manually to the test case. To populate the response if it is blank or to update it, just execute the test case. If executing the entire test case is too much, you can leverage the skip step functionality to only execute the minimum number of steps to harvest the response. REVIEW TEST REPORTS By default, itest saves test reports to a database within your workspace. itest users that use other workspaces might find it cumbersome to access test reports in your workspace. To make it easier for everyone on your team to share report data, you can configure itest to save test reports to an external centralized database that is accessible to all itest and itestrt users rather than to the default standalone database within your workspace. Note: To use an external database, your system administrator must first set up a database server. For instructions, see the chapter on Setting up an external database for storing itest Test Reports in the itest Installation Guide. Configure itest to save reports to the external database For this module, you can install a local database and configure itest to save reports to the external database Follow these instructions after you have configured a local database server for use with itest. 1 1. Install itest on a client computer. 2 2. Start itest and click Window > Preferences. 3 3. On the Preferences page, in the itest group, navigate to General > Test Reports > Test Report Database. 4. Select Use an external database to store test reports and then specify the following settings (the 4 information should be available from your system administrator).

5. You can configure the connection in one of the following ways: For any database type other than Other : Specify the Database type, hostname or IP address and IP port of the database server, the database/catalog name or SID and, if required, the User ID / Password credentials. or For a database type of Other: Specify the Java class for the custom JDBC driver, JDBC connection string, and, if required, the User ID / Password credentials. 6. In any case, you must specify the User ID / Password credentials for the account Here is how the settings from a working system: 7. Click Test Connection to check the connectivity with the database server. You should be able to get the message as below - Use the Review test reports page to fetch test reports from the database for review. The view varies slightly to support the internal (default) or an external test report database: Click Develop Test Case to go back to the Develop Test Case activity page.

Click Review test reports Wherein the example, the database server is installed on the local machine.

Search the database Use these settings to perform a database search that populates the list of reports (that is, reports that meet the search criteria are listed). Host - External database only. The search returns reports from the specified DBMS host. Result - The search returns reports with the specified execution result: Pass, Fail, Abort, Indeterminate, or All (all of the result types). Show only latest - If there is more than one test report for a test case that matches the other criteria, then fetch only the most recent test report. Fine-tune the database search criteria Click the More search options arrow to access additional search settings. If you populate any of the following options, then the view is populated with the reports that meet both sets of search criteria. Specify any or all of the following optional search criteria.

Group - External database only. Optional. The search returns reports from the specified workspace Subgroup - External database only. Optional. The search returns reports from the specified project Time period - Specify the time period during which the test case was executed. Default: Last week. Apply a filter to the reports listed on the Test Reports view Once the database search fetches the reports that meet the search criteria, you can further limit the reports that appear in the view by typing part or all of the text in any of the reported values, for example, the Subgroup name, the into the text box. Only reports for test cases that pass the filter now appear in the list. To reset so that no filter is applied, delete the text. Right-click the Test Reports view to see more options.. The warning icon on each report gives us the indication that analysis rule has not been set.

Test Report editor Double-click a report to view it in the Test Report editor. By default, the Test Report editor displays the report as soon as execution ends. For older test reports, double-click the report in the Test Reports view to open the report in the Test Report editor. In the report, the steps are organized into a hierarchy matching the procedures and steps that were executed. To make it easier for you to troubleshoot execution, the report looks very much like the Test case editor: a grid of steps and analysis rules with events occurring during execution nested under step row. n n 1. While working on a test case in the Test Case editor, the fastest way to view a recent test report is to click in the toolbar. Click to open the most recent report in the Test Report editor. 2. The Test Report editor displays only executed steps, skipped steps are not included. 3. You can click and to open and close procedures, QuickCalls, analysis rules, and nested step constructs in the report. 4. You can specify a preference for how long to store test reports in itest s built-in database (that is, when to start discarding old reports). To make it easy to distinguish between a test report and its associated test case, the background color for test nreports is blue by default. You can configure a different background color under preferences Summary information on the executed test case This section (collapsible by clicking the arrow ) displays the URI of the test case, the URI of the topology or local or global testbed used for execution, time and duration of execution, counts of execution issues, final test result, and a Report ID that uniquely identifies the report. Toolbar: Read more about the tools on the toolbar and the various columns of the test report in the online help

You can save the test report as an HTML, XML, or text file. This is useful for sharing with someone who does not use itest. Sharing test reports with others To ensure high performance, itest does not, by default, automatically save test reports as files. So, to share a test report, you must first export it as a file. There are two options: 1. Export or auto-export the report as an HTML, XML, or text file. This is useful for sharing with someone who does not use itest. Manually saving a test report as an HTML, XML, or text file. You can save a test report from either of the following starting points: a. While viewing a test report in the Test Report editor, click the Save Test Report As button. b. Let s save it as module5_testcase_year-month-date_time.html. It will save it under the folder that we created earlier under - /my_project/saved_test_reports

Or you can set a preference to auto-export a test report whenever a test case executes. On the Preferences page, navigate to itest > General > Test Reports > Auto Export Test Reports 2. Export the report as a compressed itest Test Report file (fftz format) When you export a report as a compressed file, other itest users can open it in the Test Report editor. You can email the resulting file to a coworker or tell the coworker the path of the file (typically, in a shared workspace under revision control). The coworker then uses itest to import the file into their workspace. The person importing the compressed file will follow the instructions in importing itest test reports that are in compressed file format (fftz). Follow these steps to export a report in fftz format - 1 Select the report to export and start the Export wizard. Use any of the following methods: In the Test Reports view, select the report. On the main menu, click File > Export. While working on a test report in the Test Report editor, click File > Export on the main menu. For a report saved in any of the other formats (HTML, XML, or text): In the itest Explorer, right-click the report and select Export. Lets select itest > Test Reports as Compressed File. Click Next.

2. Specify the location in your workspace to export the file to. Click OK. 3. On the next page, click Finish. When you export a report as a compressed file, you can share it with other itest users for opening in the Test Report editor. To share a report with persons who do not have access to itest, save a test report as an HTML, XML, or text file. To import an.fftz file into itest Copy the fftz file on file system and follow the steps. 1. Go to file and click Import. Import window will pop up. 2. Select option Test Report and click Next.

3. Click the Browse option. 4. Select the fftz file that you copied on file system and then click OK. 5. Click Finish. 6. Finally you will see result in Test Reports View with time stamp. Find in Test Report Did you ever want to find something you were looking for in an itest test report but because the test report is so long, it is difficult to find? Suppose, for example, that you d like to find a certain step during execution where one of your devices produced a certain message? Press Ctrl+F in the Test Report editor (or select Find/Replace from the main Edit menu). The Find in Test

Report dialog lets you specify the text you are looking for. You can even narrow your search into certain fields and can provide patterns to match. Use the toolbar buttons to jump to the next (or previous) step that has an issue. For long test reports, this can be a very quick way to navigate through steps of interest.

Lab Add more steps to the existing test Create a new test case with different commands