1 Capgemini India Private Limited July 2013 Prepared by: Ashish Mishra
2 Table of Contents 1. Abstract... 3 2. Introduction... 3 3. Location and Nature of Applications under Test... 3... 4 4. Location of the Testing Team... 4... 5 5. Geographical Locations of Production Users... 5... 6 6. How to Connect to Test Machines or Devices... 6... 7 7. Test Machines Platforms and Performance... 7... 7 8. Conclusion... 7 9. Author s Biography... 8
3 1. Abstract Setting up an effective QA environment is a key to achieve successful and efficient testing. Choosing an accurate Testing environment can help to simulate the end-user Environment and a key to avoid false alarms due to QA Environment issues. Test labs must be equipped with the required software/hardware to provide adequate test environment coverage for diversified business users across various locations. Although there are many providers offering mobile and desktop test labs on the cloud, organizations in critical sectors like Banking / Insurance / Equities Trading are seriously concerned about the secured access and in many cases prefer Private Cloud for Test Lab. 2. Introduction Test Environment setup is necessary for Testing Projects when testing needs to be performed on multiple platforms irrespective of whether Applications are for Desktop Computer or for any handheld / Mobile device. Most of the traditional Testing Teams have their own Test Machines. Test Lab Sharing and cloud based setup are the available options, which provide significant benefits like cost effective, hassle-free test solution and complete test coverage. Emerging area like Mobile Apps Testing, also requires a Test Lab to make sure that test coverage is sufficient and the application is ready to be deployed for users on all the targeted Devices, OS & Firmware. Otherwise, it is a challenge to test Mobile Apps on all the targeted handheld devices without planning an established Test Environment. 3. Location and Nature of Applications under Test Applications may be hosted on Servers or locally on the system. It is one of the major criteria to consider before setting up of the test machines. For example, if the Application is hosted on the server in New York, then it is always better to use the Test Lab in some nearby location so that the network dependencies like network downtime or latency problems can be avoided as much as possible. There should not be any redundant network setup between the Server and the client until it is required by one of the use cases. One question that may come in mind is, why we need to consider the Network, while testing desktop applications or native applications of Mobile Devices, which will be installed and used only on local resources of the system.
4 The reason is Installation Testing of the Application, which will use the resources from a remote location during installation. One of my team members was installing an Application on his local machine and Installable was located on a Remote Server. The installation failed due to some Network latency, so he raised a defect that Installable Package is broken, though failure was due to poor connectivity between the Server and the Client systems, which was not a targeted use case to test. By selecting the right set of test machines, one can avoid false alarms during the Testing. Try to maintain Low Network Dependency between the test machine and Server or package installable location, while selecting any Test Lab. Server and Client systems should be from the same geographical location in order to avoid latency related issues, if it is not a use case to test. 4. Location of the Testing Team Software Testing is one of the major outsourced businesses. A tester located in India can test an application by remotely accessing the US test machine. Due to network latency/ issues, lots of false functional and user experience issues may result during the testing. Roaming User profile or Redirected Documents and other user related Folders are the concerned criteria when a user of a specific region accesses the machine in another region. In this case, the user data is getting stored on some remote location on another region, which further results in slow response time of the system and apps. Such type of setup should be avoided; otherwise, the testing team will struggle with performance issues due to slowness of the system and may end up raising various environment related defects. A solution is to setup a QA environment locally for the testing team. Our team is using Mobile Devices services from one of the reputed providers of US whereas the team is located in India. The testers were reporting slowness and very poor response on any user action on Devices. So, we proposed to have the Test Lab & environment setup in India. The Team productivity has increased dramatically after setting up of a test lab in local environment.
5 Fig. 1: Mobile Local Test Lab solution for better productivity QA Environment should be setup according to the Testing Team location. Some key points to consider while doing Test Lab Setup are Tester s machine location, Tester s roaming Profile Location, Folder redirection policy, Test Machine Location. If the user remotely accesses another machine and opens the test application, the response may be unnecessarily delayed, leading to some non-reproducible/invalid defects and wastage of efforts/time. 5. Geographical Locations of Production Users While selecting test machines and test environment setup to test an application, one should keep in mind the users scenarios like a user may access some US based application from Asia or some other geographical locations. In this case the application should be tested on machines from different regions to simulate the real time scenarios. In many firms, users are located in different locations of the world. Applications may behave differently from different locations due to some parameters like Firewall settings of different region, Network Providers and Roaming Network in case of cellular networks, User and Computer level Region based policies and settings. Applications developed on English build operating systems may not work properly on non English operating system. Application behavior depends on the System settings and policies set, Fonts Installed & various other parameters. Even a simple PDF opened in a China based machine may throw Fonts not found message on a US based machine.
6 Fig. 2: Geographical Locations of Users If business requirement is to deploy Applications on different regions, which maintain different System Build, Policies and Settings, then it will be a wise decision to test Application from all the intended Regions, otherwise production users may report issues after Applications deployment. 6. How to Connect to Test Machines or Devices If Test machines are located remotely, the tester needs to connect to them using different authentication mechanisms, Applications invoked on a Test Machine may behave differently depending on the type of connectivity being used to connect to the Test Machine. The best way to connect to a Remote machine is by using the method, which a user will be using in the production environment to connect to the machines remotely. Most of the organizations provide facilities to their users to connect to their Office Desktop from home system using different protocols and multiple Browsers. When using Cloud based Systems and Mobile Devices from some third party service provider, make sure that you have some alternative system and devices in hand to cross- check any issue at least in the initial phase. For mobile apps testing, Devices may
7 be placed on the cloud by the service provider himself. Network selection is also essential while choosing the Test setup to test Mobile Applications; otherwise Production users may experience unexpected behavior of Apps in low Network coverage or no coverage area. The tester should make sure that the connectivity method is appropriate and similar to production user environment. If possible, testers should try to simulate PROD environment. 7. Test Machines Platforms and Performance It is necessary to consider the real users environment while selecting test machines, Operating Systems, Device model, Firmware and machine types like physical machines, Virtual machines or Citrix published Desktops. Application developed for one Operating system, Browser or Device model, may not install / launch on the other environment. Applications use the resources of test Machine or Device, so machine configurations also need to be considered for Test Lab Environment Setup. Test Lab should be well equipped with all the required Operating Systems, Browsers and Mobile Devices along with suitable configuration for Test Applications like System Memory, Processer Speed, Network Bandwidth and technologies such as Wi-Fi, 3G, 2G etc. 8. Conclusion By considering key parameters during QA Environment setup, one can mimic the Production Environment successfully, which can further reduce the false issues and enhance the Quality of Testing. Otherwise, teams shall end up spending a lot of time in Reporting & Analyzing environment related issues that got reported during testing phase. Test Machines, handheld devices, Tester and Applications are the key resources for the testing Projects. So we must give a thought while considering Physical location and configurations of these resources to help in building effective Test Lab Environment.
8 Fig. 3: Key Criteria for Effective Test Lab environment 9. Author s Biography Ashish Mishra has more than 10 Years of Experience in Software Testing industry. He Specialized in Infrastructure Software Testing for Mobile Devices, Desktop and Server Applications along with implementation of Test Lab Environment setup as per requirements.