DESIGNED BY DAVID C. JACOBS, PH.D.

Transcription

1 DESIGNED BY DAVID C. JACOBS, PH.D. HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Contents HOME LESSON ONE LESSON TWO LESSON THREE Introduction This tutorial has been designed to teach the new WinRunner user how to test GUI software with this automated testing tool. It is assumed that you have knowledge of manual testing procedures and are using WinRunner 6.0. (I have not used the sample tests on WinRunner 7.0.) LESSON FOUR LESSON FIVE LESSON SIX Go Straight to the Lesson Go Straight to the Review LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Mercury Interactive Corporation How to Use this Tutorial You can use this tutorial in any way you wish (except from selling it, etc.), but it has been designed to be used in the following manner: It is best viewed with Internet Explorer, with Medium size text (set this by going to View / Text Size / Medium), and with the resolution set at 800 by 600 Since WinRunner is not going to function well with a browser running that is not part of the test, either print out each lesson and then follow along or in a classroom have two computers running (one running WinRunner and another viewing this tutorial) Follow each lesson from the beginning to the end (and it is best to start at the beginning of the tutorial and proceed through each lesson until you have finished all lessons -- at least for the first time through) Printer-Friendly Version of Complete Tutorial What You Will Learn from This Tutorial In this tutorial, each lesson will be presented to teach you how to operate one aspect of the WinRunner automated testing environment. Except for the first lesson, each lesson will include explanation and an exercise where you will apply what you have learned. (1 of 2) [12/9/ :39:57 AM]

2 Your Responsibilities Your job while using this tutorial is to report any bugs that you might come across to the author at HOME FIRST LESSON WinRunner and the Mercury Interactive logo are trademarks of Mercury Interactive Corporation. Copyright 2001 David C. Jacobs (2 of 2) [12/9/ :39:57 AM]

3 DESIGNED BY DAVID C. JACOBS, PH.D. HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Contents HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN Lesson One: Introduction In this lesson, you will learn: where the tester fits within software development different types of testing (white box, manual, and automated testing) where automated testing is beneficial and where it is not beneficial how automated testing follows the steps of manual testing some basics about WinRunner LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE A Basic Overview of Software Development (Software Life Cycle) 1. Client defines what the software should be like (the expected object or the requirements) 2. Developer produces software according to client's requirements 3. Tester tests software against requirements and reports bugs to developer Mercury Interactive Corporation 4. Developer fixes bugs and modifies software according to latest requirements Printer-Friendly Version of Complete Tutorial Types of Testing 1. White Box Testing: The tester examines the code of the AUT (Application Under Testing) to see if the code is written correctly 2. Manual Testing / Black Box Testing: The tester uses the application in order to test it (testing for properties, appearance, and functionality) 3. Automated Testing: The tester creates test scripts that test the application similar (1 of 6) [12/9/ :39:59 AM]

4 to manual testing (testing for properties, appearance, and functionality) Excellent for regression testing, because the same scripts can be run again and again What are the benefits of automated testing over manual and white box testing? Compared to white box testing: not as expensive in the long run better for functionality and regression testing Compared to black box testing: more reliable more consistent reusable and repeatable inherent programming capabilities Which software testing should be automated? Tests that need to be run for every build of the application Tests that use multiple data values for the same actions (data-driven tests) Tests that require detailed information from application internals (e.g., GUI attributes) Stress/load testing Any test that has a repetitive feature (test one object multiple times in one test or test the same objects through multiple builds of the software) Which software testing should NOT be automated? (2 of 6) [12/9/ :39:59 AM]

5 Usability testing (how easy is the software to use?): the tester needs to use the software as a user One-time testing: automated will be too time consuming ASAP or Quick-look testing: automated will be too time consuming Ad hoc or random testing (testing the software based on hunches): automated will be too time consuming to set up General rule: if improvisation is required or little time is available, avoid automated testing and use manual testing The Steps of Manual and Automated Testing Automated testing follows the steps of manual testing Manual Testing Steps 1st Step 2nd Step 3rd Step 4th Step Perform user actions Wait for process to complete Verify AUT Repeat for all actions of AUT Automated Testing Steps 1st Step 2nd Step 3rd Step 4th Step Generate automated test script Synchronize script and AUT Add verification tests Run test scripts About WinRunner Created by Mercury Interactive, Inc. in 1992 WinRunner is an automated testing tool that tests GUI software and web pages WinRunner combines a GUI software package and scripting language (3 of 6) [12/9/ :39:59 AM]

6 GUI software package: Integrated Development Environment (IDE) allows the tester to record, write, edit, run test scripts in one environment Scripting language: Test Script Language (TSL) is based on C Three aspects of GUI software to test with WinRunner Functions Properties Bitmap features How WinRunner Basically Tests The tester creates a test script by recording the interaction with the AUT The tester can insert "checkpoints" to test the properties, appearance, and text of objects The tester can enhance test scripts with code The tester can enhance test scripts with data for a data-driven test which inserts data to determine if the AUT will accept it The tester easily runs the test by pressing an icon The tester reads the results A temporary GUI Map file is created while recording; the GUI Map file acts as an interpreter between application and test script Supported Platforms Windows 95 Windows 98 Windows NT Windows ME Windows (4 of 6) [12/9/ :39:59 AM]

7 Supported Environments Netscape, Microsoft Internet Explorer Java applets, Java applications, JavaScript ActiveX, OCX Controls WAP ERP/CRM applications (Oracle applications, PeopleSoft, SAP, Siebel) Terminal emulators (mainframe, AS/400, Unix servers) Visual Basic C/C++ PowerBuilder Delphi WinRunner Resources These sources were consulted in the construction of this tutorial and are an execellent guide for further research. WinRunner User's Guide WinRunner Tutorial TSL Online Reference WinRunner Student Handbook (from Mercury Interactive classes) HOME REVIEW THIS LESSON NEXT LESSON WinRunner and the Mercury Interactive logo are trademarks of Mercury Interactive Corporation. Copyright 2001 David C. Jacobs (5 of 6) [12/9/ :39:59 AM]

8 (6 of 6) [12/9/ :39:59 AM]

9 DESIGNED BY DAVID C. JACOBS, PH.D. HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Contents HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE Lesson Two: Recording Tests, Playback, Context Sensitive and Analog Modes In this lesson, you will learn: how WinRunner tests are recorded how to playback a test the differences between analog and context sensitive recording LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Creating a Test Script There are two ways to create a test script: 1. Recording the test script as you interact with the application 2. Writing or scripting the test script in test script language (TSL) Mercury Interactive Corporation How to Record a Test Script 1. Start WinRunner Start / Programs / WinRunner / WinRunner Printer-Friendly Version of Complete Tutorial Close the Add-In Manager, by clicking OK If the Welcome to WinRunner dialog appears, uncheck the Show at Startup check box, then close the Welcome to WinRunner dialog by pressing the X in the top right 2. Start the AUT 3. Start a new test script (by pressing the New icon selecting File / New) (1 of 8) [12/9/ :40:02 AM]

10 4. Activate the recording function (by pressing the Record icon or selecting Create / Record - Context Sensitive) 5. Interact with the AUT 6. Stop recording (by pressing the Stop icon) 7. Save the test script Test 1 Execute the following steps to learn how to record a test script and to test the search function of WordPad: 1. Put a floppy in the a:\ drive 2. Start WinRunner (if you have not already done so) Start / Programs / WinRunner / WinRunner 3. Close all other applications (even the internet browser you are using; this is why it is best to print out this tutorial) 4. Start our AUT (WordPad) Start / Programs / Accessories / WordPad 5. Position the two applications so that they do not overlap Why? So that you can see both applications clearly and so that you can interact with both applications and the actions do not get executed by the incorrect application Position WinRunner on the left and the AUT on the right 6. Start a new test script in WinRunner File / New or press the New icon 7. Start recording Click the red Record icon Or Create / Record - Context Sensitive Or press F2 8. Click inside of WordPad and type these lines: (2 of 8) [12/9/ :40:02 AM]

11 This is the first line. This is the second line. This is the third line. This is the fourth line. 13. Click on the Search icon (binoculars) in WordPad In the Find What text field, type: line Click on Find Next four times Press Cancel 14. In WordPad, click the New icon or select File / New Press OK Do not save changes, by selecting No 15. Stop Recording, by pressing the Stop icon in WinRunner 16. Save the test script in WinRunner File / Save Save In: a:\ drive File Name: test1 Press Save 17. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top 18. Accept the Test Run Name of res1, by pressing OK 19. Examine the Results and then close Results 20. Maximize WinRunner's window and look at the test script; these are our recorded movements interacting with WordPad 21. Click on the Help Icon (arrow and question mark) and then click on the first line (this will explain the syntax of the script) 22. set_window ("Document - WordPad", 3); (3 of 8) [12/9/ :40:02 AM]

12 set_window activates the window to receive input "Document - WordPad" is the logical name of WordPad 3 is the number of seconds before the next function in the test script will run (plus the default time) 23. Close the TSL Online Reference window, WordPad, and the test script What did we learn from this test script? Recording a test script in WinRunner is as simple as pressing the Record icon and interacting with the AUT, and running or playing back the test script will execute the movements that were recorded. In this way, we can test the functionality of the AUT. Two Recording Modes In WinRunner, when we record a test script, there are two recording modes; test scripts can be in both modes or in either mode. The two modes are: 1. Context Sensitive Mode (the default mode) Context Sensitive statements describe actions made to GUI objects and are recommended for most situations, where the test scripts are: object-based not affected by user interface changes (i.e., movement of objects) Software examples: menus, icons, check boxes 2. Analog Mode Analog statements are useful for literally describing the keyboard, mouse, and mouse button input of user Test scripts are: screen-coordinate dependent based on mouse and keyboard activities (e.g., mouse movement, press, and release, and keyboard input) (4 of 8) [12/9/ :40:02 AM]

13 Software examples: virtual reality environment, drawing, paint brush strokes How to Choose Between Context Sensitive and Analog Modes Choose Context Sensistive if: The application contains GUI objects Exact mouse movements are not required You plan to reuse the test in different versions of the application Why? Movements of the GUI objects in different builds will not affect the test. Choose Analog if: The application contains Bitmap areas (e.g., a drawing area) Exact mouse movements are required You do not plan to reuse the test in different versions of the application Why? Movements of the GUI objects in different builds will affect the test. Test 2: Recorded in Context Sensitive mode, but will not function properly 1. Start a new test in WinRunner (by pressing the New icon) 2. Open Flight Reservation application Start / Programs / WinRunner / Sample Applications / Flight 1A 3. Log on the Flight Reservation application Type your first name in the Agent Name field (your name must be at least four characters long) Type mercury in the Password field and click OK 4. Position the two applications so that they do not overlap 5. Start recording Press the Record icon or select Create / Record - Context Sensitive 6. Open order number 1 in the Flight Reservation application Select File / Open Order Check the Order No. check box in the Open Order window (5 of 8) [12/9/ :40:02 AM]

14 Type 1 in the text box and press OK 7. Open the Fax Order window Select File / Fax Order 8. Enter in the Fax Number field 9. Sign your first name in the Agent Signature area by holding down the left mouse button and drawing your name 10. Press the Send icon 11. Stop Recording 12. Save as test2 on your a:\ drive 13. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top 14. Accept the Test Run Name of res1, by pressing OK 15. Notice that WinRunner will not sign your name; it finds Agent Signature area but not the exact place to start writing 16. Examine the Results and then close Results 17. Close the test script What did we learn from this test script? Given the instructions from the test script, WinRunner can find the object to write in but not the exact place. Because of this, the writing of our name did not take place. Thus, analog mode is required for this to work properly. Test 3: Same test in Analog Mode 1. Start a new test in WinRunner (by pressing the New icon) 2. Start recording in Analog mode Press the Record icon and F2 (6 of 8) [12/9/ :40:02 AM]

15 Or select Create / Record - Analog 3. Open order number 1 in the Flight Reservation application Select File / Open Order Check the Order No. check box in the Open Order window Type 1 in the text box and press OK 4. Open the Fax Order window Select File / Fax Order 5. Enter in the Fax Number field 6. Sign your first name in the Agent Signature area by holding down the left mouse button and drawing your name 7. Press the Send icon 8. Stop Recording 9. Save as test3 on your a:\ drive 10. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top 11. Accept the Test Run Name of res1, by pressing OK 12. Examine the Results and then close Results 13. Close the test script and the Flight Reservation application What did we learn from this test script? With the test script recorded in analog mode, WinRunner can precisely place the mouse pointer in the exact location inside of the drawing window. Because of this, the test script can write our signature and hence perform the test properly. HOME PREVIOUS LESSON REVIEW THIS LESSON NEXT LESSON (7 of 8) [12/9/ :40:02 AM]

16 WinRunner and the Mercury Interactive logo are trademarks of Mercury Interactive Corporation. Copyright 2001 David C. Jacobs (8 of 8) [12/9/ :40:02 AM]

17 DESIGNED BY DAVID C. JACOBS, PH.D. HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Contents HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE Lesson Three: Initial and End Conditions, Organizing Files In this lesson, you will learn: why there is a need to set initial and end conditions how to set initial and end conditions for your tests how WinRunner organizes your files LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Why Create Initial and End Conditions? When you record a test script and when you playback a test script, the AUT is run or interacted with; this can change the AUT: by entering data into text fields by enabling icons that were grayed out before the test or graying out buttons that were enabled before the test Mercury Interactive Corporation Printer-Friendly Version of Complete Tutorial Why Is This a Problem? When WinRunner runs a test script, the AUT may not be in the same condition it was before the test The test may not be able to interact with the AUT as the test script describes; thus, the test script will not function properly and will not generate a good test of the AUT Creating Initial or End Conditions Solves this Problem What is an initial condition? An initial condition is the state of the AUT at the beginning (1 of 5) [12/9/ :40:03 AM]

18 of the test. What is an end condition? The end condition is the state of the AUT after a test has run. Important Note: The end condition of an AUT after a test has run should not affect a subsequent test upon the AUT. If the subsequent test is affected by the earlier test, the AUT is not being tested properly. In other words, the subsequent test is not reliable or useful. For example: 1. If a test script instructs WinRunner to open an application to test it, make sure the end condition closes the application (otherwise, running the same test twice may create problems). 2. If a test enters data into text fields, have the end conditions clear the data (as we did with Test 1). Test 4: A test without proper end conditions 1. Start a new test in WinRunner (by pressing the New icon) 2. Start WordPad Start / Programs / Accessories / WordPad 3. Position the two applications so that they do not overlap 4. Start Recording in WinRunner (in Context Sensitive Mode) Press the Record icon or select Create / Record - Context Sensitive 5. Type this in WordPad: WinRunner was created by Mercury Interactive. Mercury Interactive makes a variety of testing tools. Mercury Interactive trades stock with the symbol MERQ. 6. Select the Find icon (binoculars); search for the word: interactive 7. Select Find Next three times and press Cancel (2 of 5) [12/9/ :40:03 AM]

19 8. Stop recording 9. Save as test4 on your a:\drive 10. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top 11. Accept the Test Run Name of res1, by pressing OK 12. Look at the text in WordPad; the same text was entered twice (once when we recorded the test script and another time when we ran the test script) What did we learn from this test script? The running of this test script entered the same data as the recording of our interaction with the AUT, because it did not clear the data already entered. This could become a problem in a more complicated test. 13. Close the Results, the test script, and WordPad In WordPad, select No to avoid saving changes Test 5: Same test as Test 4 but with end conditions 1. Start a new test in WinRunner (by pressing the New icon) 2. Start WordPad Start / Programs / Accessories / WordPad 3. Position the two applications so that they do not overlap 4. Start Recording in WinRunner (in Context Sensitive Mode) Press the Record icon or select Create / Record - Context Sensitive 5. Type this in WordPad: WinRunner was created by Mercury Interactive. Mercury Interactive makes a variety of testing tools. (3 of 5) [12/9/ :40:03 AM]

20 Mercury Interactive trades stock with the symbol MERQ. 6. Select the Find icon (binoculars); search for the word: interactive 7. Select Find Next three times and press Cancel 8. Create end conditions to clear the data Select the New icon in Word Pad (this will create end conditions by clearing the data for the next test of WordPad) In the New window of WordPad, select OK Select No for saving changes in WordPad 9. Stop recording 10. Save as test5 on your a:\ drive 11. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top 12. Accept the Test Run Name of res1, by pressing OK 13. Examine and then close the Results 14. Run test script again 15. Accept the Test Run Name of res2, by pressing OK 16. Close the Results, the test script, and WordPad What did we learn from this script? The end conditions clear the data from the AUT during the running of the test, so that the data will not get in the way of the next running of the test. How WinRunner Organizes Your Files When you create and run a test script, WinRunner creates a directory with the name of the test. (4 of 5) [12/9/ :40:03 AM]

21 For example, for Test 5, there would be a diretory named Test5. Depending upon configuration of WinRunner, the Test5 directory may contain these subdirectories and files: a script file a backup script file (if WinRunner is configured to do this) a header file a lock file containing the name of the user and a process id exists while the a test script is open to prevent others from opening it, but it is automatically deleted when the test script is closed a db subdirectory which contains files which influence playback and which should not be changed a res1 subdirectory which contains the first set of results of your test a exp subdirectory which contains the expected results for some tests A directory for a test could also contain: a chklist subdirectory which contains the checklists for objects to be checked for GUI tests a debug subdirectory which contains the results of running your test script in debug mode a table file which contains the data for a data-driven test HOME PREVIOUS LESSON REVIEW THIS LESSON NEXT LESSON WinRunner and the Mercury Interactive logo are trademarks of Mercury Interactive Corporation. Copyright 2001 David C. Jacobs (5 of 5) [12/9/ :40:03 AM]

22 DESIGNED BY DAVID C. JACOBS, PH.D. HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Contents HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR Lesson Four: Synchronizing Tests In this lesson, you will learn: when you should synchronize a test how to synchronize a test LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE The Problem that Synchronizing Solves Actions in a test script either set a time for the next action to be executed or they have a default time of 10 seconds. If the AUT does not respond quickly enough to a command in the test script, WinRunner will move on to the next command; a problem may arise because the outcome of the last command did not occur. There are a variety of objects and functions that may cause these; the following are some common objects and functions that may cause synchonization problems: Software retrieving information from a database Mercury Interactive Corporation Waiting for a window to pop up Waiting for a progress bar to reach 100% Waiting for a status message to appear Printer-Friendly Version of Complete Tutorial If WinRunner waits for these objects and functions for the full ten seconds of the default interval time (or the set time by the function in the script), then it will move on to the next action in the script which may not execute correctly. (1 of 7) [12/9/ :40:05 AM]

23 Entering a Synchronization Point Solves the Problem A synchronization point is a line in the test script that instructs WinRunner to wait for a certain response from the AUT during playback before proceeding to the next line in the test script. Types of Synchronization Cue Outcome Test Script Function Window Wait for window set_window Object State Wait for an object state obj_wait_info Bitmap Wait for a bitmap to refresh obj_wait_bitmap Elapsed Time Wait for a set time to elapse wait Important Note: Our software does not currently have a synchronization problem. We will create one by changing the default interval between actions in a test script. This changes the default time for all subsequent test scripts, so you must change it back after you run this test script. Test 6 1. Start WinRunner (if it is not already running) 2. Start a new test in WinRunner (by pressing the New icon) 3. Start the Flight Reservation application and log in Start / Programs / WinRunner / Sample Applications / Flight 1A In login window, type your name and enter mercury for the password and click OK 4. Position the two applications so that they do not overlap 5. Start Recording Press the Record icon or select Create / Record - Context Sensitive 6. Create a new order in Flight Reservation application (2 of 7) [12/9/ :40:05 AM]

24 File / New Order 7. Fill in flight and passenger information Date of Flight: 02/02/02 Fly From: Los Angeles Fly To: San Francisco Click the Flights button and double click on any flight Name: Steve Johnson Select First Class Click Insert Order 8. Delete the order (this will set the end conditions) Click the Delete Order button Click Yes in the message window to confirm 9. Stop Recording 10. Save as test6 on your a:\ drive Changing the Default Interval Between Actions in a Test Script Since there is currently no synchronization problem with our test script and AUT, we are going to create one by changing the default time interval between actions. To change this, we do the following steps: 1. Settings / General Options 2. Click the Run tab 3. For Timeout for checkpoints and CS statements, change the value to 1000 milliseconds (which equals 1 second) 4. Click OK Run Test 6 (3 of 7) [12/9/ :40:05 AM]

25 1. Make sure the test6 test window is active in WinRunner, by clicking on the title bar 2. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top 3. Accept the Test Run Name of res1, by pressing OK 4. Notice that WinRunner fails to click the Delete Order button because it is still disabled (WinRunner did not wait until the Insert Order operation was completed) 5. Select Continue in Warning Message Box 6. The test fails again because it cannot find the Flight Reservations Message Box 7. Select Cancel in the Run Wizard dialogue box Synchronizing the Test 1. Make sure the test6 test window is active in WinRunner, by clicking on the title bar 2. Place the cursor at the point where you want to synchronize the test Add a blank line after the button_press ("Insert Order"); statement Place the cursor in the blank line 3. Synchronize the test so that the test script waits for the "Insert Done" message to appear in the status bar Create / Synchronization Point / For Object/Window Bitmap Use the hand pointer to click the status bar in the Flight Reservation window (where the phrase, "Insert Done...", is) WinRunner will insert an obj_wait_bitmap synchronization point in the test script 4. Manually change the 1 second wait in the script to 10 seconds Change the following line: (4 of 7) [12/9/ :40:05 AM]

26 obj_wait_bitmap ("StatusBar95", "Img1", 1); To read: obj_wait_bitmap ("StatusBar95", "Img1", 10); 5. Save the test script again 6. Maximize WinRunner The Syntax of the obj_wait_bitmap Statement To get information on the syntax of the this statement, click on the help icon (arrow and question mark) and then click on the obj_wait_bitmap statement in the test script. obj_wait_bitmap ("StatusBar95", "Img1", 10); obj_wait_bitmap: the function in the test script which waits for ten seconds for the image to appear before moving on to the next action "StatusBar95": the logical name of the image (your test script may have "Insert Done..." as the logical name) "Img1": the file name of the bitmap (the image) 10: the amount of seconds to wait for the image to appear (plus the default which we changed to one second) Close the TSL Reference window Run the Synchronized Test 1. Position the two applications so that they do not overlap 2. Make sure the test6 test window is active in WinRunner, by clicking on the title bar 3. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top (5 of 7) [12/9/ :40:05 AM]

27 4. Accept the Test Run Name of res2, by pressing OK 5. Watch how WinRunner waits for the "Insert Done" message to appear in the status bar 6. Review the results Double click on the "wait for bitmap" event to see the bitmap image WinRunner was waiting for 7. Close the Results, the test script, and the Flight Reservation application What did we learn from this test script? When testing an AUT with synchronization problems, we must insert a synchronization point to stop WinRunner momentarily in its running of the test script so that the AUT can perform its actions. If we don't do this, the test will not run properly. Change Time Interval Back to Default 1. Settings / General Options 2. Select Run tab 3. For Timeout for checkpoints and CS statements, change the value to milliseconds 4. Click OK Additional Exercise 1. Maximize WinRunner 2. Use the Help icon (arrow and question mark) to determine the action and syntax 3. Write down in your notes answers to the following questions about these lines from our test script: (6 of 7) [12/9/ :40:05 AM]

28 set_window ("Flight Reservation", 2); What is set_window? What is its action? What is "Flight Reservation"? What is 2? edit_set ("Name:", "Steve Johnson"); What is edit_set? What is its action? What is "Name:"? What is "Steve Johnson"? button_set ("First", ON); What is button_set? What is its action? What is "First"? What is ON? HOME PREVIOUS LESSON REVIEW THIS LESSON NEXT LESSON WinRunner and the Mercury Interactive logo are trademarks of Mercury Interactive Corporation. Copyright 2001 David C. Jacobs (7 of 7) [12/9/ :40:05 AM]

29 DESIGNED BY DAVID C. JACOBS, PH.D. HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Contents HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR Lesson Five: GUI Verification In this lesson, you will learn: what verification is how to insert GUI checkpoints to verify GUI objects LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Verification and Manual Testing In manual testing, the tester tests the AUT in order to determine if it meets the specified requirements; this is called "verification." Verification is the process where the tester determines if the actual object (AUT) matches (or is the same as) the expected object (specifications or requirements). If the actual object matches the expected object, the AUT passes the test; if the actual object does not match the expected object, the AUT fails the test. Mercury Interactive Corporation Actual Object should match... Expected Object The functions, properties, and appearance of the actual object should match the functions, properties, and appearance of the expected object. Printer-Friendly Version of Complete Tutorial WinRunner and Verification Definition: Verification is the process by which WinRunner reports whether expected results are actualized by the application. In other words, verification in WinRunner is checking to see if the actual object matches the expected object when the actual object is driven by a test script in WinRunner. (The verb actualize means to realize in action or to make real.) (1 of 6) [12/9/ :40:07 AM]

30 Functionality testing of a GUI object can be performed by recording the interactions with the AUT and then playing back the test script. The appearance of an AUT can be verified through Bitmap verification. Many of the properties of a GUI object can be verified through inserting GUI checkpoints. The Four Steps of How WinRunner Verifies 1. The tester stores an expected result with the recorded test script (known as a "Checkpoint") 2. WinRunner captures an actual result during playback 3. WinRunner compares the actual result with the stored, expected result 4. WinRunner reports the comparison outcome as PASS or FAIL Definition of a Checkpoint: A TSL statement that verifies that the actual object matches the expected object. GUI Object Verification To verify a GUI object, we use GUI checkpoints; GUI checkpoints are placed in a test script and test a GUI object according to a specific checklist of properties that we create. With GUI checkpoints we can test to determine, for example: if a window is the correct size if a button is enabled or disabled if the correct text is in a text field if boxes are checked or unchecked if a list has the correct number of items The Process for Establishing GUI Checkpoints 1. Determine the visual cues to verify (2 of 6) [12/9/ :40:07 AM]

31 2. Create the GUI checkpoint 3. Test the checkpoint to ensure that it works for PASS and FAIL conditions (we will not be doing this) 4. Run the recorded test script to verify correct execution Examples of Visual Cues These examples are changes to the AUT during our interaction with it: a window appears during interaction with AUT an object changes state (for example, from enabled to disabled, from focused to unfocused) output data is returned from the system (for example, the total price of a sale or invoice number) a message is returned from the system (for example, "Purchase completed") Test 7 1. Start WinRunner 2. Start a new test in WinRunner (by pressing the New icon) 3. Start the Flight Reservation application and log in Start / Programs / WinRunner / Sample Applications / Flight 1A In login window, type your name and enter mercury for the password and click OK 4. Position the two applications so that they do not overlap 5. Start Recording Press the Record icon or select Create / Record - Context Sensitive 6. Open the Open Order dialog box in Flight Reservation File / Open Order Move the Open Order dialog box so that it does not overlap with WinRunner (3 of 6) [12/9/ :40:07 AM]

32 7. Create a GUI checkpoint for the Order No. check box Create / GUI Checkpoint / For Object/Window Use the hand pointer to double-click on the Order No. check box The Check GUI dialog box opens and displays the available checks Accept the default check ("State"); this will capture the current state ("OFF") of the check box and stores it as expected results Click OK The obj_check_gui statement will appear in the test script 8. Check the Order No. check box and enter 4 as the Order No. 9. Create another GUI checkpoint for the Order No. check box Create / GUI Checkpoint / For Object/Window Use the hand pointer to single click on the Order No. check box WinRunner inserts a checkpoint for the default check ("State") by inserting the obj_check_gui statement in the test script 10. Create a GUI checkpoint for the Customer Name check box Create / GUI Checkpoint / For Object/Window Use the hand pointer to double click on the Customer Name check box Accept the default check "State" Also, select "Enabled" as an additional check The State check captures the current state ("OFF") of the check box; the enabled check captures the current condition ("OFF") of the check box Click OK 11. Click OK in the Open Order dialog box to open the order 12. Stop recording 13. Save as test7 on your a:\ drive 14. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top (4 of 6) [12/9/ :40:07 AM]

33 15. Accept the Test Run Name of res1, by pressing OK 16. Examine the Results Click on end GUI checkpoint They show the expected value and the actual value 17. Close the GUI Checkpoint Results window 18. Close the Results and the Flight Reservation application The Syntax of the GUI Checkpoint Statement To get information on the syntax of the this statement, click on the help icon (arrow and question mark) and then click on the obj_check_gui statement in the test script. obj_check_gui("order No.","list1.ckl","gui1",1); obj_check_gui: the function in the test script that performs the GUI verification; it compares the actual GUI object and the expected GUI object "Order No.": logical name of GUI object to be verified "list1.ckl": name of checklist which defines the GUI checks "gui1": captured data in file used as the expected results 1: time interval in seconds (plus default time) to capture data from GUI object 19. Close the TSL Reference window 20. Close the test script What did we learn from this test script? In order to verify a property of a GUI object, we can insert GUI checkpoints; these checkpoints will verify the properties of the GUI object according to the checklist that we create. (5 of 6) [12/9/ :40:07 AM]

34 HOME PREVIOUS LESSON REVIEW THIS LESSON NEXT LESSON WinRunner and the Mercury Interactive logo are trademarks of Mercury Interactive Corporation. Copyright 2001 David C. Jacobs (6 of 6) [12/9/ :40:07 AM]

35 DESIGNED BY DAVID C. JACOBS, PH.D. HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Contents HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR Lesson Six: Bitmap Verification In this lesson, you will learn: how to test Bitmap images in your application how to insert Bitmap checkpoints to verify Bitmap images LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN Bitmap Verification Recall that verification is the process by which WinRunner reports whether expected results are actualized by the application With Bitmap verification, WinRunner is comparing a Bitmap image (the actual object) with the expected object pixel by pixel to verify that the actual object matches the expected object LESSON TWELVE Bitmap Image (Actual Object) should match... Bitmap Image (Expected Object) Mercury Interactive Corporation The Process for Establishing Bitmap Checkpoints Printer-Friendly Version of Complete Tutorial 1. Determine the Bitmap object to verify 2. Create the Bitmap checkpoint 3. Test the Bitmap checkpoint to ensure that it works for PASS and FAIL conditions (optional) 4. Run the recorded test script (1 of 4) [12/9/ :40:09 AM]

36 Test 8 1. Start WinRunner 2. Start a new test in WinRunner (by pressing the New icon) 3. Start the Flight Reservation application and log in Start / Programs / WinRunner / Sample Applications / Flight 1A In login window, type your name and enter mercury for the password and click OK 4. Position the two applications so that they do not overlap 5. Start Recording Press the Record icon or select Create / Record - Context Sensitive 6. Open order #6 File / Open Order Select the Order No. check box and type 6 Click OK 7. Open the Fax Order dialog box File / Fax Order 8. Enter in the Fax Number field 9. Move the Fax Order dialog box so that it does not overlap WinRunner 10. Switch to Analog mode Press the F2 function key 11. Sign your first name in the Agent Signature box with the mouse pointer 12. Switch back to Context Sensitive mode Press the F2 function key 13. Insert a Bitmap checkpoint that verifies your signature Create / Bitmap Checkpoint / For Object/Window Use the hand pointer to click on the Agent Signature box (2 of 4) [12/9/ :40:09 AM]

37 WinRunner will capture the Bitmap and will insert the obj_check_bitmap statement in the test script 14. Click the Clear Signature button 15. Insert another Bitmap checkpoint that checks the Agent Signature box Create / Bitmap Checkpoint / For Object/Window Use the hand pointer to click on the Agent Signature box WinRunner will capture the Bitmap and will insert the obj_check_bitmap statement in the test script 16. Click the Cancel button on the Fax Order dialog box 17. Stop recording 18. Save as test8 on your a:\ drive 19. Don't run the test; instead, lets view the expected results first Open the Test Results Window Tools / Test Results View the captured Bitmaps Double click on the first "capture bitmap" event Double click on the second "capture bitmap" event Close the Test Results window File / Exit 20. Playback or Run the test script Click the Run from Top icon Or Run / Run from Top 21. Accept the Test Run Name of res1, by pressing OK 22. Examine the Results and then close the Results The Syntax of the obj_check_bitmap Statement To get information on the syntax of the this statement, click on the help icon (arrow and question mark) and then click on the obj_check_bitmap statement in the test script. (3 of 4) [12/9/ :40:09 AM]

38 obj_check_bitmap: the function in the test script that performs the Bitmap verification h "(static)": the logical name of the object "Img1": the file name of the captured Bitmap (the image) 10: the amount of seconds to wait for the image to appear (plus the default time) 22. Close the TSL Online Reference window 23. Close the test script and the Flight Reservation application What did we learn from this test script? To verify a Bitmap image, we insert a Bitmap checkpoint in the test script to compare the actual Bitmap object with the expected Bitmap object. WinRunner will then compare the actual object and the expected object pixel by pixel to determine if the actual object matches the expected object. HOME PREVIOUS LESSON REVIEW THIS LESSON NEXT LESSON WinRunner and the Mercury Interactive logo are trademarks of Mercury Interactive Corporation. Copyright 2001 David C. Jacobs (4 of 4) [12/9/ :40:09 AM]

39 DESIGNED BY DAVID C. JACOBS, PH.D. HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN LESSON ELEVEN LESSON TWELVE Contents HOME LESSON ONE LESSON TWO LESSON THREE LESSON FOUR Lesson Seven: Text Verification In this lesson, you will learn: how to test text elements in an application how to insert text checkpoints to verify text elements LESSON FIVE LESSON SIX LESSON SEVEN LESSON EIGHT LESSON NINE LESSON TEN When to Use Text Verification When text is not part of a GUI object (e.g., a label or caption) When text is part of a bitmap image When text is part of a non-standard, custom GUI object LESSON ELEVEN LESSON TWELVE Functions Used to Read Text Mercury Interactive Corporation Printer-Friendly Version of Complete Tutorial If text is: A property of a standard GUI object Part of custom GUI object which cannot be read from the property Part of Bitmap or in ASCII (not part of GUI object) Use: Context Sensitive functions (e.g., obj_get_info, button_get_info) obj_get_text text reading feature (e.g., win_get_text) Process to Verify Text (1 of 11) [12/9/ :40:11 AM]

40 1. Declare a variable in the script and assign it the expected value (in this test, we will compare the actual text object with the expect text object) 2. Insert a function to read (or get) the actual text from the AUT during playback 3. Insert code to compare the actual value with the expected value 4. Insert code to report the comparison outcome to the WinRunner Test Results (e.g., PASS or FAIL) Declaring a Variable In TSL, the syntax for declaring a variable is: VariableName="value"; VariableName is the name of the variable. "value" is the value of VariableName as a string item. The variable that we will declare in our next test script will be: VersionNum="Version 1.0"; If-Else Statement We will also include an if-else statement in our test script. This statement will be used to compare the actual text object and the expected text object and to report to us if the actual text object passed or failed the test. The syntax for an if-else statement in TSL is: if (condition) action1 else action2 If the condition is TRUE, then action1 will execute and action2 will be ignored. If the condition is FALSE, then action1 will be ignored and action2 will execute. The code that we will insert in our test script is the following: if (VersionNum == text) (2 of 11) [12/9/ :40:11 AM]

41 tl_step("check Version",PASS,"Version is correct."); else tl_step("check Version",FAIL,"Version is incorrect."); This code explained: VersionNum: the variable that will hold the expected value text: the variable that will hold the actual value tl_step: function that inserts a status message in the test results for a single section of the test script The syntax for the tl_step function is: tl_step (step_name, status, description); tl_step: function step_name: the name of the test step status: PASS or FAIL status of test step description: description of status to be printed in results Passing the Test If the expected text object (expected text value stored in VersionNum) is the same as the actual text object (actual text value) stored in text, then the test step will have the status of PASS and the phrase ("Version is correct.") will print in the results. Failing the Test If the expected text object (expected text value stored in VersionNum) is NOT the same as the actual text object (actual text value) stored in text, then the test step will have the status of FAIL and the phrase ("Version is incorrect.") will print in the results. How to Read or Get the Text In order to verify the text, we need to get the text. The following describes how to get the text: 1. Go to the window or object in the AUT where you need to get the text (3 of 11) [12/9/ :40:11 AM]

42 2. Use WinRunner to get the text Create / Get Text / From Screen Area Three Modes to Run a Script In Recall the three modes that you can run a script in: Verify Debug (we will use debug the first time after we add lines of code to our test script to make sure the code is correct) Update Test 9: This test will read text from a Bitmap area and then verify it 1. Start WinRunner 2. Start a new test in WinRunner (by pressing the New icon) 3. Start the Flight Reservation application and log in Start / Programs / WinRunner / Sample Applications / Flight 1A In login window, type your name and enter mercury for the password and click OK 4. Position the two applications so that they do not overlap 5. Start Recording Press the Record icon or select Create / Record - Context Sensitive 6. Open the About window in Flight Reservation application Help / About 7. Read the version number text Create / Get Text / From Screen Area The pointer becomes crosshairs Move the cursor slightly above and to the left of what you want to capture (a quarter inch away from "Version 1.0") (4 of 11) [12/9/ :40:11 AM]

43 Hold down the left mouse button and drag the cursor so that the text you want to capture (and no other text) is enclosed by a rectangle, then release the mouse button Press the right mouse button to complete the operation 8. Close the About window by pressing the OK button 9. Stop recording 10. After the last line of the test script, insert the variable that holds the expected text value VersionNum="Version 1.0"; 12. After the last line of the test script, type in the following code which will verify that the actual text matches the expected text: if (VersionNum == text) tl_step("check Version",PASS,"Version is correct."); else tl_step("check Version",FAIL,"Version is incorrect."); 12. Save as test9 on your a:\ drive 13. Check the test script by running it in debug mode Change mode to debug Run from Top If there are problems, fix your script and run in debug again 14. Playback or Run the test script in verify mode Click the Run from Top icon Or Run / Run from Top 15. Accept the Test Run Name of res1, by pressing OK 16. Examine the Results 17. Close the Results and the Flight Reservation application 18. Open the Flight Reservation (1B) Start / Programs / WinRunner / Sample Applications / Flight 1B (5 of 11) [12/9/ :40:11 AM]

44 19. Playback or Run the test script again with this application in verify mode Click the Run from Top icon Or Run / Run from Top 20. Accept the Test Run Name of res2, by pressing OK 21. Examine the Results This test should have failed 22. Close the Results, the Flight Reservation application, and the test script What did we learn from this test script? To verify text that is part of a Bitmap area, the tester must get the text (or read the text), and insert code to verify the text. Reading Text from an Application If your AUT has an area that generates a total number of orders, etc., you can read the text, place an order, and verify that the order number has gone up by one. To do this, you must: 1. Get the text 2. Add an order 3. Synchronize the test (to ensure the test script waits for order to take place) 4. Get the text again 5. Change the name of the variables 6. Insert an if-else statement Why Change the Name of a Variable? When we get or read the text twice (once before a new order and once after a new order). The order should increase the number of orders by 1. However, when we get the text, the variable text will be used to hold both values. The first time we get the text we will: (6 of 11) [12/9/ :40:11 AM]