Automated Testing Frameworks: Test Automation with CodedUI

Transcription

1 Automated Testing Frameworks: Test Automation with CodedUI CodedUI Introduction CodeUI is one of the important new features in Visual Studio 2010 s Premium and Ultimate versions. It helps users to create their own automation test scripts for testing user interfaces of an application. With the help of CodedUI, many test cases can be automated and executed during regression testing therefore reducing the effort of manual testing while increasing testing accuracy and coverage. Compared to other test automation tools, CodedUI provides some unique benefits: 1. Support for Microsoft technologies (e.g. Dot net): With the application developed with Microsoft technology, the objects contained in the application can be easily recognized by CodedUI and the properties of objects are visible. 2. Friendly UI for automation testers to create scripts: Not only because it is based on C# and Visual Studio IDE, but also it delivers an additional GUI to help testers modify the existing scripts (this feature is in service pack 2). 3. Easily integrated into TFS server and controlled by Test Manager. By using Test Manager, testers can decide when the automation scripts are to be executed and which machine (physical/virtual) to execute them on. The automation scripts will be executed automatically based on the schedule and then the results will be reported. However, CodedUI also has some limitations: 1. Too many codes may be generated by CodedUI for describing the UI control. The controls are represented by different level of elements. Some controls are in the very deep level therefore described by long piece of elements. If there are some changes in the elements (e.g., the name of the elements), it may cause a lot of effort to maintain the controls whose description contains the changed elements. 2. Sometimes the same code may be generated more than once by CodedUI. 3. Right now, CodedUI only supports IE and FireFox. In addition, it does not support Flash and Flex. Page 1 of 10 Contact Us at info@xbosoft.com

2 Before we start talking about the framework, there are still some useful tips for using CodedUI: Install Visual Studio service pack2. Be sure to install this as it contains the graphic editor to edit CodedUI tests and enables creating automation tests for Silverlight 4 and Firefox. Use Test Manager to record the steps and then import the recorded scripts into the framework. By doing this all imported scripts contain the same coding standards which make it easier to read, track and maintain. If you are using Test Manager to manage the testing process, try to integrate CodedUI with Test Manager. By doing this all automation tests are connected to the related test cases or defects which make the testing process more effective. The Importance of an Automation Framework In some automation testing projects, the team starts directly coding. They record the controls and actions, create some new methods by themselves and then combine them into an integrated test case. Everything seems to work well, however there are some potential risks if the SUT (system under test) is complicated or the amount of the coding work is large: There may be a bunch of duplicated code in the automation scripts. Such duplication will cause trouble for future maintenance. Also it will affect the test execution speed. Hard code is not handled very well which means that if some hard coded objects need to be changed, they have to be changed in every place they are used. If new test cases need to be added, testers may have to record all the controls and actions again. If some existing functions are out of date and need to be deleted, it is hard to do this because it is not clear which other parts of the scripts are affected by the deleted functions. Without a good framework, it will be difficult to hand-over the scripts to new team members and difficult for new testers to get involved in the automation project. Considering these factors, we recommend building a suitable framework before starting the detailed coding work. A good framework will bring a lot of benefits to the project even if the application is small and simple. Some main advantages include: Flexibility: If there is more than one automation tester writing the automation scripts at the same time, a flexible framework will guarantee the effectiveness of the team work. Page 2 of 10 Contact Us at info@xbosoft.com

3 Maintainability: In reality, the UI or features of the SUT (System Under Test) sometimes are updated. The test environment may also be changed. A good framework will allow the testers maintain the existing scripts easily. Extendibility: When new product module(s) need to be automated, a well structured framework will enable efficient integration of new scripts. Example of a standard framework In this section, we introduce a standard framework developed by XBOSoft (evolved from the default CodedUI framework) which can be used flexibly for most of automation projects. Figure 1 is the default CodedUI framework created by Visual Studio 2010 when you start a new CodedUI test project: Figure 1: Initial Framework Created By CodedUI The above default framework is very simple. CodedUITest1.cs file is used to store the test cases. In building a real framework, more components need to be added in order to fulfill the testing requirements as most applications are more complex than this example. The following will describe steps to build a general extended framework that can be applied for most automation projects. In the next section, we also provide scale down/up suggestions to help adapt the framework based on your own situation. Page 3 of 10 Contact Us at info@xbosoft.com

4 Step 1: Create a test case folder and add test case files based on modules If there are thousands of test cases that need to be automated, it is impossible to put all of them into a single file. Instead, it s better to split them based on the module to which they belong. By storing test cases into different files, this also allows multiple automation testers to work on the scripts simultaneously without contention. Figure 2 shows the result of this step, under the folder Test Case, multiple c# files were created, and then the test cases can be stored in these files separately based on the module that it belongs to. Figure 2: Multiple Test Cases Step 2: Use the UI Map class to store controls and actions. CodedUI builder is the tool provided by Visual Studio which is used to record the user s actions and controls of the application. When using the CodedUI builder, the scripts are generated automatically by Visual Studio 2010 and put into the UIMap.Designer.cs file. Usually, the recorded script contains the controls in the application and the simple actions performed by the tester. In most cases, automation testers need to add more other actions or modify the existing scripts to make the automation scripts more effective. For example when recording the log in action, the specific username and password is remembered and stored in the recorded scripts. If the tester needs to log in via another account, rather than record it again, parameterizing the recorded login actions and passing the different parameters to that Page 4 of 10 Contact Us at info@xbosoft.com

5 function saves time and can be used conditionally as well depending on the user id. In Visual Studio 2010 Service Pack 2, a graphic UI editor is provided which can save testers a lot of effort (Figure 3). However for adding new actions or doing complicated modification for the existing control, testers also need to create their own code in another file called UIMap.cs. Please see the Figure 4. Figure 3: Graphic UI for Modifying Controls Figure 4: Adding UI Map File Page 5 of 10 Contact Us at info@xbosoft.com

6 Step 3: Create a class for common functions (methods) used in the entire project This class is usually used to store the common functions or processes which are used in the project and reused by more than one module. For example, the Login process is usually the first step for each test case and is reused many times. Therefore this process (method) can be stored in the common method c# file and called by each test case. See Figures 5&6. Figure 5: Common Method File Page 6 of 10 Contact Us at info@xbosoft.com

7 Figure 6: Adding Common Method File Step 4: Put hard code (constants) into the project properties setting file: Figure 7 is the screenshot showing the properties setting window. Hard code and constants refers to parameters that are not changed often and appear many times in the scripts. By putting such parameters into a single place, this helps the automation tester to maintain them. If the value needs to be changed, the automation tester just needs to change it in the setting file. Figure 7: Properties Setting File Page 7 of 10 Contact Us at info@xbosoft.com

8 Step 5: Create a common setting file used to store the hard coded (constants) values in properties setting file This file is related to the properties setting file. Figure 8 shows how to define and construct the parameters in the setting file and Figure 9 shows how to use it. Figure 10 shows the final version of the recommend framework. Figure 8: Common Setting File Figure 9: Apply Common Setting File Page 8 of 10 Contact Us at info@xbosoft.com

9 Figure 10: Final Version of Framework 4. Scale up and down to your framework The framework provided in the previous section is a standard framework. Usually, the additional components added in that framework are useful for most automation projects. However, each automation project or application has its own unique characteristics and requirements. The standard framework can be easily scaled up and down to match them. The following are some useful tips to help you tailor your framework: 1. Data driven test: In some applications, there is a requirement that a lot of different input data need to be tested against a same test scenario. If that happens, it will be very convenient to add a data driven test component to your framework. Input data can be stored in an Excel file and called by some additional lines of code in the correct places. You can refer to which shows the details of how to create a data driven test. 2. Store information in XML file: In the standard framework, all hard code is stored in the properties setting file. One of the benefits is that if we want to change that information, we just need to change it one time. However, sometimes it is not enough because those with no have programming skills may also need to use the automation scripts. In this case, they should not need to open Visual Studio 2010, find the properties and Page 9 of 10 Contact Us at info@xbosoft.com

10 change them. Some may not even have Visual Studio on their computer. In this case, we need to store some important configuration information into an external XML file. Such information usually includes URL, saved folder path, database location, etc. Then, the user only needs to open the XML file in the notepad and change values based on their needs. The changed XML file will be passed to the automation scripts. By doing so, the effort and the difficulty of the maintenance work is significantly reduced. 3. Multiple UI Maps vs. Single UI Maps: In the standard framework, all recorded and manual created scripts are stored in a single UI Map. When the application is very complicated, the size of that UI Map file will be very large and there will be many lines of code in the file which makes it difficult to manage and reuse it. Additionally, if there are many testers working on the same UI Map, version control becomes an issue, leading to the risk of losing code. One way to solve this issue is to create the multiple UI Maps based on the different pages or modules. Then one tester works on one UI Map and when the tester checks in their work, only the code created/edited by that tester itself is updated thereby avoiding this situation. However, this may increase the coding effort and require more time for coordination. 5 Conclusion No matter how complicated or simple the application, and regardless of the size of the automation team, building your own suitable automation framework for your CodedUI project is always a good choice to start because it provides some very useful benefits: You are not only designing the framework, you are also creating the automation plan for the whole project. You need to analyze the application, divide them into different modules. You also need to think about the end user of the application and the user who will use the automation scripts in order to decide whether to apply an external XML file. In general, designing the automation framework also help you to organize the whole automation project. Automation scripting is a type of development work. We need to consider the flexibility, extendibility and maintainability of the scripts (code). We also need to consider who may use and maintain the code in the future. Therefore designing a good framework is essential. In this paper, we described how to create a standard CodedUI framework which can be applied to most automation projects. Additionally, we have provided some useful tips to tailor the standard framework under different circumstances. We hope that you ve gotten a general idea about the importance of an automation framework and are ready to start building your own. Page 10 of 10 Contact Us at info@xbosoft.com