AUTOMATED GUI TESTING OF SOFTWARE APPLICATIONS USING UML MODELS

Transcription

1 AUTOMATED GUI TESTING OF SOFTWARE APPLICATIONS USING UML MODELS Robertas Jasaitis, Dominykas Barisas, Eduardas Bareisa Kaunas University of Technology, Department of Software Engineering Studentu st. 50, LT Kaunas, Lithuania Abstract. In many cases, testing is an essential, but time and resource consuming activity in the software development process. In the case of model-based development, test construction and test execution can be partially automated. As the application size is constantly growing, the need for automated testing frameworks comes into place, particularly frameworks for automated testing of user interaction and graphical user interface. This article describes an implementation of the GUI test generator framework based on UML models where specific UML activity diagrams are used for test case generation. This article covers all the elements of the diagrams, usability and notation. An example of a software system is analyzed in detail and experimental results provided. Keywords. GUI testing, model based testing, automated test generation. 1 Introduction Model-based testing has become popular not only in software design and development, but is widely used for testing as well [1-4]. There is a number of advantages, but also difficulties and shortcoming of various model-based approaches, however it can be employed for graphical user interface testing as well. Developers and testers of custom graphical user interface use a lot of human resources to test their applications. One of the reasons for this is a large number of different devices (PC's, mobile devices) in the market which have different screen resolution, operating system, architecture etc. Manual testing is much more time consuming process and less reliable than automated testing. There is a lack of an automated testing of the system which would be capable to generate test cases from UML and execute them automatically. This would make testing faster, more effective and precise device processor instead of human resource. Tests can be run in parallel on different devices, it makes testing procedure significantly faster. Precision is achieved by checking each graphical element drawn on the screen whether their attribute values are correct and if not, the error is found. All views would be tested throughout the whole testing process. Test cases would be generated automatically using UML. Specific test cases would be created by tester and recorded into memory. All test cases would usually be executed during the testing process. 2 Problems of GUI Testing This section mostly addresses mobile application GUI testing problems as this process is more complicated in many ways and the example provided in section 4.2 is related to mobile devices as well. Current devices with UI in the market are different compared to regular desktop software in several ways, for example, mobile software behavior can be slightly different on emulator and the actual device, different vendor devices support different APIs, Java Virtual Machine details differ, impossible to record GUI events, each of them may have different operating system [5]. Furthermore, software testing on every device is time consuming (uploading and running the application), problems may be introduced due to obfuscation, each device has a different hardware configuration, display size, number of colors, memory size, processor speed etc. All these reasons make testing process more complicated comparing to a regular desktop software applications as the system need to be tested on all possible configurations. The same application looks and sometimes behaves a bit different depending on the device it is installed on, that s why software for different hardware devices should be tested on each device separately. The application deployment process takes some time, testing quickly becomes a routine requiring a lot of patience of software developers and testers. A common strategy is capture-replay testing methodology. The advantage of the proposed technique is possibility to capture once and replay on many devices. A test script is recorded once and should run in the same way on other devices. The tests can be executed both on real devices and emulators of different hardware devices. In such case, the proposed solution would make the testing shorter, easier and repeatable

2 3 Existing solutions There are various automated testing tools for software systems GUI supporting a wide range of testing methods including mobile device application testing [6]. Existing tools usually provide only record-playback functionality, therefore tester has to create many tests manually. This increases the risk of making a mistake and is time consuming. One of the existing testing tools is JUnit or J2MEUnit framework [7]. J2MEUnit is Java 2 Micro Edition (J2ME) library including unit testing framework for J2ME applications. The good thing is that this tool can be used with standard desktop applications as well. It is based on JUnit framework, which is used to test Java Standard Edition (J2SE) applications. J2MEUnit is suitable to test functionality of the system, which can be done by having initial data and a set of expected results, but not as good for the GUI. There is a possibility to test some parameters of graphical objects, their states, rendering methods etc. However, it is difficult to test the view visible on the screen at any given time when there is no access for the GUI management. Moreover, J2MEUnit has no test generation feature, which is a shortcoming comparing to the presented approach. All tests, initial data and expected results have to be written by tester manually, which takes more time to reach the same code coverage. J2MEUnit is a unit testing framework and not really suitable for the GUI testing. Another alternative is to use Record/Playback tools ( Marathon ) [8]. Such tools are suitable for GUI testing automation, but test cases have to be generated manually, which is time consuming and potential risk of fault. After GUI changes occur (new graphical elements are added) test cases have to be recorded manually again, and the old test cases become useless. Therefore this kind of testing is suitable only for a complete software system and is partly manual. There are various researches for automated GUI testing based on UML models. Our approach is simpler and requires less effort to model the UML diagrams. The defined stereotypes of proposed UML extension eases the GUI modeling and testing comparing to other approaches where standard UML Activity diagrams are used [9]. Proposed UML extension provides both: GUI structure and possible navigation between UI elements. 4 The Testing Technique Proposal The proposed testing model is created by generating activity diagrams [10, 11]. It is not a usual case to use UML activity diagrams for UI modeling. However the existing stereotypes of activity diagram elements are not suitable for UI modeling. With usual activity diagram it is complicated to define buttons, containers, pages and other UI elements in the diagram and find differences between them. Even more complicated is to model the navigation of the testing application. On some specific devices it can be keyboard driven navigation, touch screen events and sometimes 3 rd part events like incoming call, incoming SMS or similar events which is not a case in usual desktop applications. So in general case the UI can be defined in a set of UI elements along with a set of UI navigation elements. This is an optimal and suitable approach in most cases. 4.1 The proposed UML Activity diagram extension The extension is made by introducing the following new stereotypes [12, 13]: Action is the primary diagram element. This icon represents activities performed by the System or Actor. Since it is the most common icon, it typically has either a neutral color or no color at all. Presentation activities are indicated by the <<presentation>> stereotype. This stereotype indicates that there is a conversation between the use-case Actor and the System. It represents a special category of Action activities and is used to abstract user interface details. So in UI modeling this is used in much more cases than the action activity. The Connector stereotype represents connections to flows diagrammed elsewhere. The use of Activity Diagrams often leads to the creation of large and complex models, so it is useful to indicate alternate navigational flows. The connector icons for this stereotype can be used to automatically link to another diagram. Exception activity occurs when there is an exceptional flow in the use case, and is indicated by the <<exception>> stereotype. This usually represents an error condition but may also represent unusual or unexpected system behavior. The Frame and Page stereotypes seem very similar, but there is a little difference. Frame is used to represent areas of the presentation that are logically or physically separated. Page is used to indicate separate display pages on a software application. A Frame is often logically associated with a Page and is not compulsory. It is possible define the UI model of some application without frame, but it is impossible to do this without page

3 It would be easy to expand our list of Activity Diagram stereotypes and icons. Figure 1 represents a fairly complete and simple set for modeling user interfaces. Figure 1. A UML activity diagram extension elements 4.2 UML Test Model When the rules of UI modeling are defined we can check the concrete example in more details. Let s say we have the following application with the given navigation as shown in Figure 2. Figure 2. An example of application with UI elements This is a simple testing application. Let s look at the main navigation without looking at the application logic. Figure 3 illustrates the application which is modeled using proposed UML extension

4 Figure 3. Activity diagram of example application As we can see from the images above and UML model the application is fully described in both cases. Let s take the main page as the focus. We can navigate to all other pages from there by activating the given presentations (buttons) or quit by activating Exit presentation. From other pages we can go back to the main page by using the same presentation. It is very handy to use the same presentation in several pages, because the model looks smaller and easier to read. This is common practice in development as well. 4.3 Test case generation We now have a clear view how the user interface of any application can be modeled using UML activity diagrams. The next step is to define the way to generate test cases from this model. Till this point described model doesn't have full information about the UI structure of each page. Besides the navigation between pages are not fully defined as we know where we can go from witch page but we do not know what user actions causes any of the navigation. To make model full enough to contain all the information we need we have a proposal to expand it. We suggest adding two constrains to the model: OCL constrains to each <<presentation>> element containing the constrains in which application navigates to the new page We need these constrains in order to know which user actions causes the navigation. As an example it could be button click action, some key press action, or any other user action. By having this, the test case generator or the test case executor knows how to activate the navigation to the next page. It allows us to have full access to user navigation handling. The structure of constrains depends on the implementation of generator or test case executor, so these constrains are not analyzed in this article. base structure of the UI of the page to each <<page>> element We need the UI structure of each page in order to know what the correct UI state of the page is in order to evaluate the correctness of the application. The correct UI state in this context means the expected UI state of the page. We suggest using the XML format for UI structure realization. It is becoming a commonly used practice worldwide in different areas. itself. Note that other techniques can be used there as well as this is not a main factor of the testing technique After we described constrains we can now define how the test case generator is working. The test case generator analyzes the given UML diagram searching for <<presentation>> elements. Test case generator collects constrains of each <<presentation>> element in order to know how to navigate from one page to another. The aim of the generator is to activate each <<presentation>> element at least once in order to visit all the <<page>> elements of the UML model and collect constrains from each page. Having this, generator imitates the navigation threw the application and in each step makes a test case which is constructed of the activation constrain and the structure of the current page

5 5 Test execution and results Test execution process diagram is shown in Figure 4. Figure 4. Test execution process The process is begun by taking navigation paths from given UML which have been generated during test case generation. Navigation paths consist of the action inputs. Action input here means any action which causes the navigation. It is usually some user action, like key press event or similar. After that the inputs are passed one by one to the application. Application reacts to the inputs and does the navigation inside it. The UI state (the UI state description in this context is described in section 4.3) is taken between each input. These states later are passed to Oracle, which simply compares two given UI states and generates the result: weather the states are equal or not. In the Oracle the expected states (which have been taken from the model while generating test cases) are compared with the actual states from the application. We get the results for each state between inputs and pass them to report generator. Report generator takes the results and converts them to human readable form. The given example in the previous section was passed through the process. Four different paths were generated from the UML model and this is clearly seen in the model. Each of the paths visited different page (or exit state in the path threw the exit presentation). These paths generated thirteen different inputs. All of them were navigation key events or the fire event. So there were thirteen UI states tested in the application with just five different buttons to cover all the paths and it is easy to find them from the model. If the application with at least minor UI problem have been passed threw the process at least one test case found the issue which proves that process is working correctly. Test results of the analyzed application are given in Table 1. Table 1. Measured test statistics Criteria The amount of generated test cases 13 The amount of passed test cases 13 The amount of failed test cases 0 Pages visited 4/4 Buttons activated 7/7 Result Code coverage (focusing on UI classes only) 90.7% 6 Conclusions and Future work This paper presented a formal technique to the GUI testing process based on UML model consisting of specific activity diagrams with pre-defined elements, notations and usage. An example of application provided and testing process analyzed in detail covering the following steps: Definition of activity diagram elements Test model construction consisting of activity diagrams

6 References Test execution process Result storage and assessment Future improvements: Find the solution to identify a set of optimal paths covering all elements in the diagram Implement mechanism that allows user to set priorities to the pages in diagram and inspect the most important pages only while generating the test cases Improve test case generator by implementing the automated notification of input fields and ability to generate user specific inputs to that input field. [1] Briand L., Labiche Y. A UML-Based Approach to System Testing. Fourth International Conference on the Unified Modelling Language [2] Offutt J, Abdurazik A. Generating test cases from UML specifications. In: Proceedings of 2nd international conference on UML [3] Menon A. A comprehensive framework for testing graphical user interfaces. PhD Dissertation, University of Pittsburgh, Pittsburgh [4] Barisas D., Bareiša E. A Software Testing Approach Based on Behavioral UML Models. Information Technology And Control, Kaunas, Technologija, Vol. 38, No. 2, [5] Zduniak M. Automated GUI Testing of Mobile Java Applications. Master thesis, Poznan University of Technology, Poznan, Poland [6] Jasaitis R. Mobile applications UI testing framework based on UML models. INFORMATION TECHNOLOGIES 2009, 14th Master and PhD students conference on Informatics [7] Karlsson J. Unit Testing in J2ME. JayView magazine, Java in the Öresund region, No [8] Marathon 2.0 User's Guide. Jalian Systems Private Ltd [9] Vieira M., Leduc J., Hasling B, Subramanyan R., Kazmeier J. Automation of GUI Testing Using a Model-driven Approach. Proceedings of the 2006 International Workshop on Automation of Software Test, at ICSE p [10] Hartmann J., Vieira M., Foster H., Ruder A. A UML-based approach to system testing Innovations. Syst Softw Eng, pp , Springer-Verlag [11] Lieberman B. UML Activity Diagrams: Detailing User Interface Navigation. IBM developerworks. 2004, [12] Kruchten P. The 4+1 View Model of Architecture. IEEE Software, 12 (6), pp [13] Ambler S. User Interface Design: Tips and Techniques. The Object Primer: Building Object Applications That Work, and Process Patterns, 2e. Cambridge University Press