1. Overview Marathon is a general purpose tool for both running and authoring acceptance tests geared at the applications developed using Java and Swing. Included with marathon is a rich suite of components to help you interact with your application at the User Interface Level (GUI). Marathon tests are scripted using Jython, a 100% pure java implementation of the programming language Python -- the emphasis being on a extremely simple, highly readable syntax that customers, analysts, testers and developers can all be comfortable with. But don't let the simplicity fool you. Python is a mature, full-featured programming language, so testers and developers don't need to sacrafice anything in terms of the power and creativity with which they can express their tests. To aid with the regression testing of existing applications, Marathon comes bundled with a recorder to capture events as you use and interact with your application. These events are then converted into a valid Marathon test which can subsequently be played back. 2. Marathon Usage Using marathon depends on your organization and how the software is developed. We found Marathon useful for the following purposes. Smoke testing Developers write very cursory, very coarse grained tests to make sure the application meets basic functionality and they have not introduced some glaring errors into the application. Regression testing Marathon is used to write test cases and these test cases are run to ensure that no existing functionality is broken in the application being developed. Formal acceptance testing Marathon test cases are written at a very abstract level and these test cases are used to reflect user requirements before any code is developed. Once the development of any part is complete, Marathon is used to re-record the test case. A feature is complete only when the Page 1
test cases passes against the application. While nothing in Marathon stops developers from using it for GUI unit-tests, Marathon is not developed for such a use. There are other projects that are aiming at developer level unit testing and may be they should be given a try first. Check out JFCUnit and Abbot for excellent unit testing projects. 3. Concepts 3.1. Application Under Test This is the application which is being tested using Marathon. Marathon can be used to test any application that is developed using Java/Swing technology. 3.2. Test Case A Marathon test case is a recorded test script that tests a function of the AUT. A Test Case is basically a python script with a test function. 3.3. Capture Scripts A capture script is a recorded script. A capture script is a python script. The function name is as selected by the user. 3.4. Fixtures If you are familiar with any of XUnit test frameworks, you are already familiar with Fixtures. A Fixture provides a mechanism for running a testcase with the same environment. In marathon a Fixture is a Python class. This class provides two functions setup and teardown. The framework executes setup before starting the testcase and teardown at the end of the testcase. The fixture is responsible for starting the application and passing the appropriate parameters to it. A fixture is generated by Marathon that in-effect does not perform any setup or teardown tasks except to start the application. This fixture is written in default.py file in Fixtures directory. Marathon can facilitates of selection of fixtures for individual testcases. Marathon fixtures can also be written in Java (instead of Python). The Java fixture should provide setup and teardown functions. The fixture should be given under the marathon.fixture property of the MPF file. This feature need to be used Page 2
only in rare occasions and as such discouraged. 3.5. Project File A Marathon Project File(MPF) is a set of Java properties that define the structure for a Marathon project. The relevant information includes The project directory, test case directory, the fixture etc. 4. Downloading and Installing Marathon The latest version of Marathon is available at the SourceForge project site. Download the latest marathon-<version>-tar.gz and support-<version>-tar.gz files. You need GNU compatible tar and gunzip programs to extract the files. WinZip also works fine for windows. Untar the marathon-<version>-tar.gz into a directory. This will create a directory marathon-<version>. Untar the support-<version>-tar.gz file in this directory. Windows 95/98 users On some windows machines (notably 95/98 etc.) the marathon.bat file might not work as it is provided. You can set the environment variable MARATHON_HOME to installation directory and use marathon.bat file. Please add a trailing \ to the installation directory. 5. Invoking Marathon Marathon can be invoked at the command prompt by providing the options. Marathon itself is a Java application and can be invoked using java command. For easier use the distribution includes a script file (marathon.bat for Windows and marathon for Linux/Unix) that should be used to invoke Marathon. The following is a description of marathon command in Unix man-page format. 5.1. NAME marathon - Marathon GUI testing framework 5.2. SYNOPSIS marathon marathon MPF marathon -batch [-text <file>] [-html <file>] [-xml <file>] Page 3
MPF [ TestCase...] 5.3. DESCRIPTION In the first two forms marathon is launched in interactive mode. In the first case the user is prompted to select a project file. The third form is used to invoke marathon in non-interactive mode. The result output can be captured by using the output options. If no TestCases are given on command line all tests in the TestCases directory from the MPF file will be executed. A TestCase is the python filename (without the.py extension) from the TestCases directory. The directories in the file name path are seperated by '.'. For example, a test case TestCases/SubDir1/test1.py will be represented as 'SubDir1.test1'. 5.4. OPTIONS -help Displays a help message -text Instructs marathon to write the test results in text format into the given file -html Instructs marathon to write the test results in HTML format into the given file -xml Instructs marathon to write the test results in XML format into the given file 6. MPF Selection An MPF can be passed to Marathon on command-line. In GUI mode, when Marathon is invoked without a MPF on the command-line, a selection dialog is displayed. The same dialog is also used to modify a MPF. For creating a new MPF or modifying an existing one, invoke Marathon without any command-line parameters. Select MPF Page 4
The 'File name:' combo box displays the file currently selected. The last 5 MPF names selected are available in the drop-down list. Use the 'Browse' button to select a file from the file system. The MPF selection dialog contains the following command buttons. New... Selecting this option lets you create a new MPF. Modify... Selecting this option lets you modify an existing MPF. The file should have been selected into the 'File name:' combo box. OK Launches Marathon using the currently selected MPF configuration. Cancel Exits Marathon. 7. MPF Configuration The MPF configuration dialog is launched when you select either a 'New...' or 'Modify...' option from the MPF selection dialog. The MPF configuration dialog consists of multiple tabs. We need to enter information in the tabs to create a MPF. 7.1. Project tab In this tab we enter the information related to the project as a whole. Name: Project tab MPF The name of the project. The name is displayed as part of the window title by Marathon. Directory: The project directory. Marathon stores the test cases, capture scripts, fixture etc. in this directory. You can use the 'Browse' button to select a directory from the file system. Description Free form text description of the project. Page 5
7.2. Main tab This tab allows you to select the application that is configured in this project to test with Marathon. Main class: Main tab MPF This is the java class that needs to be launched. This class should have been declared public and should define a public static void Main(String[] args) function. run Main() in seperate thread option In cases where the Main() function displays a modal dialog or does not return due to other reasons, select this option. This option will enable Marathon to generate a fixture that can execute Main() in a seperate thread. Program arguments: The arguments that are passed to the application. These arguments are built into a String array and passed to the Main class's main() function. The respective code is in the Fixture that is created. VM arguments: The arguments that are passed to the java command. Working Directory: The working directory under which the application is launched. You can use the 'Browse' button to select a directory from the file system. 7.3. Class path tab The Class path tab is used to define the folders and the Jar/Zip files that should be added to the Java class path for successful launching of the application. Class path tab MPF Use the 'Up', 'Down', 'Add folders', 'Add Archieves' and 'Remove' commands to manage the class path. Page 6
7.4. Python path tab The python path tab allows you to add folders to add to the python path. This allows to make use of external modules in the test and capture scripts. Python path tab MPF Use the 'Up', 'Down', 'Add folders' and 'Remove' commands to manage the python path. The Marathon distribution contains the standard python modules. You need to set the python path only if you want to use python modules other than the standard ones. You do not need to add the CaptureScripts, Project directory, Test cases directory to the python path. 7.5. Resolvers tab The Resolvers tab is used to enter the class names of the custom resolvers that might be needed by the application. The custom resolvers are called in the order given to resolve a component. As a last resort, the default component resolver is used to resolve the component. The details of developing custom resolvers and deploying them is given in the Developer Guide. 7.6. Property variables tab Enter the properties that should be used by Marathon to expand the values given in other tabs. The property variables are useful to provide paths that are not hard-coded. While entering the path enter the values as %variable_name% and provide the value in this tab. The variables given here overrides the values computed by Marathon for marathon.capture.dir and marathon.test.dir. Hence these variables can be used to provide non-standard project structure to Marathon. Look at a generated MPF file to see which other variables might be of use to override in this tab. As of now, these variables are not visible to applications that are launched. For that purpose use "-Dvariable_name=value" in the vmargs section of the Main tab. The main configuration dialog has the following command buttons. Test Page 7
Use this option to test the launching of the application given in the Main tab. This option opens a dialog that displays the command being executed along with the stdout and stderr streams. The 'Close' button closes this dialog and stops the application that is launched. This command is helpful in finding out whether all the elements in the classpath have been set properly. Save Selecting this command saves the MPF. You will be prompted to provide a filename, in case a New MPF is being created. Save As This option is available only when a MPF is being modified. Lets you save the MPF information into a new MPF. You will be prompted to provide a filename. Cancel Cancels the New or Modify option. 8. Marathon Main Window Most of the action in Marathon takes place in the Main Window. The project name (as given in the MPF configuration) along with the test file currently opened in the editor is shown in the title of the window. The main-window consists of a menu, a toolbar. The content pane is divided into three parts. The navigator/junit panel, the editor panel and the console panel. 8.1. Main Menu and Toolbar Main Window Most actions of Marathon are performed either using the menu or the toolbar. Accelerator keys have been provided for some actions and can be used to perform the action. The assigned keys can be seen from the drop down menus. 8.1.1. Menu and Toolbar options Action Menu Toolbar Description New TestCase File-->New-->Test Case New TestCase Creates a new test case. Prompts you for saving the existing testcase (if modified). Creates a test template in the editor panel. Page 8
New Capture Script File-->New-->Capture Script New Capture Script Creates a new capture script. Prompts you for saving the existing testcase (if modified). Prompts for the function name of the capture script and creates a capture template in the editor panel. New Fixture File-->New-->Fixture None Creates a new fixture. The editor panel will be filled with a default Fixture using the mainclass provided in the MPF configuration. You can update the script and save it into a file in the Fixtures directory. Save File-->Save Save Saves the current script in the editor pane. If the file being edited is a new file, prompts for a file name. Save As File-->Save As None Prompts for a file name and saves the current script into the file. Record Marathon-->Record Record Starts recording. Launches the application. All actions performed in the application are converted as appropriate marathon scripting elements and updated in the editor pane. Pause None Pause Pauses recording. The actions performed in the application are not recorded till a resume Page 9
is undertaken. The file can't be edited in this mode. Resume None Resume Resumes recording. It is the users responsibility to bring the application back to the same state (the UI wise) as when pause is selected. Insert Script None Insert Script Inserts a capture script into the recording session. If the capture script contains marathon scripting elements, the appropriate actions are taken onto the recorded application. Stop Marathon-->Stop Stop Stops the current recording or playing session. Play Marathon-->Play Play Plays the current script in the editor pane. If the current script is not a test script (identified by test() function), displays an error. Open Application Marathon-->Open Application Open Application Starts up the application. It is useful for starting a capture script after bringing up the application to some state. Close Application Marathon-->Close Application Close Application Closes the application. It is necessary to use close application when a test case fails, before start playing or recording. In case of test case failures the application is kept open to identify the Page 10
Select Fixture Marathon-->Select Fixture state of the application and find the cause for failure. None Select a fixture. The fixture will become the default fixture. Any testcases that are recorded from now onwards uses the new fixture. The fixture can be changed also by double clicking on the fixture shown in the status bar. About Marathon Help-->About None Displays the current version information and credit information about Marathon. Exit Marathon File-->Exit None Exits marathon. Prompts to save the file if the script in editor pane has been modified. 8.2. Editor Panel The editor panel provides the facilities to edit test scripts, capture scripts and fixtures. The editor panel provides syntax coloring to the marathon scripts loaded into it. The script in the editor panel can not be modified while the recording or playing is going on. The editor supports cut, copy, paste and undo, redo capabilities. Use the shortcuts CTL-X, CTL-C, CTL-V, CTL-Z and CTL-Y to use these facilities. Marathon scripts use TAB as an indent. When modifying the scripts use TAB as indent as python is very finicky about the indentation. You can load a file into the editor panel by selecting it from either the Navigator panel or JUnit panel. You can edit the files that are recorded using the recorder. Jython needs init.py files to identify modules. Saving a capture script or a fixture from the editor updates the init.py files in the directory hierarchy. Page 11
8.3. Navigator Panel The navigator panel displays the directory hierarchy and files from the project's testcases and capture script directories. Double clicking on a file opens it up in the editor panel. Right click on the navigator panel to display a popup menu. The following table summarizes the available options in the popup menu. Some of these options are also available on the navigator panels toolbar. 8.3.1. Menu options Action Menu Description Create new folder New-->Folder Creates a new directory Create a new test case New-->Test Case Creates a new test case Create a new capture script New-->Capture Script Creates a new capture script Create a new fixture New-->Fixture Creates a new fixture Play Play Play the script in the editor File operations Copy, Paste, Delete, Move, Rename Performs the corresponding operations on files Expand and Collapse Expand all, Collapse all Expands and collapses the directory tree in the panel View options Go into, Go up, Home Changes the view of the panel. Go into displays the directory only from the current directory node. Go up displays one node up and home resets the selection. Refresh Refresh Reads the file system again to display the directory structure in the navigator panel. This is required when a file has been created outside of marathon. 8.4. JUnit Panel The JUnit panel provides facilities to run multiple test cases from marathon. Select junit tab below the navigator panel to display the JUnit panel. Page 12
The up and down arrows on the panel allows to search for the next and previous failures from the test cases. clicking on the Run all tests button runs all test cases. You can select few testcases using the mouse click and use the 'Run selected test cases' button to run only the selected test cases. Currently the JUnit functionality does not support creation of a test report. A future version of Marathon will support generation of reports. Double clicking on a test opens the corresponding test case in the editor. 8.5. Console Panel Console panel displays the output (standard error and standard output) from running the test case. 9. Recording test cases It is simple to record testcases under marathon. Launch marathon and press record button. This will execute the setup of the fixture that will launch the application. Any actions performed on the application are captured by Marathon and corresponding script elements are entered into the editor. You can use CTL-Right click to open up Marathon popup menu on any component of the application. Depending on the type of component a menu is displayed with different type of assertions that are available for that component. Selecting any option from the menu will insert the corresponding assertion script element into the editor. Switch to Marathon and select the stop recording option to stop the recording. The script can be executed by using the Play button on the toolbar (or CTL-P shortcut). If the playing encounters any problems a dialog is displayed giving an appropriate error message. In case of successful completion of the test a dialog with a success message is displayed. 9.1. Proper recording on components For effective use of Marathon it is necessary to record all actions from the application. Unfortunately, since Marathon works at the semantic level - some actions can't be recorded properly. The following sections provide hints about effective use of Marathon for some of the components. 9.1.1. Buttons Use mouse-click to select a button. Pressing spacebar after focusing does not record as an Page 13
action in Marathon. 9.1.2. Menus When there are child menus, they are displayed when the mouse hovers on the parent node. However marathon treats Menu items as buttons and needs clicks on each item in the tree to properly function. Example: For recording File->New->Testcase, perform mouseclicks on each of the 'File', 'New' and 'Testcase' items (even when they are already displayed). 9.1.3. Tool Bars Just like buttons. use Mouse to click on the buttons. 9.1.4. Trees For expanding a tree element use doubleclick on the node. Clicking on the expand-button ('+') gets recorded as a click and when playing the tree wouldn't be expanded. 9.1.5. Tables Do not use Tab to move between the cells. Use mouse to change the focus to the cell before editing. Page 14