USTGlobal INNOVATION INFORMATION TECHNOLOGY Using a Test Design Tool to become a Digital Organization
Overview: Automating test design reduces efforts and increases quality Automated testing resolves most challenges created by the traditional, manual method of test suite design. However, the focus of test automation has traditionally been on automating test management and test execution, not test design. Test design involves deciding what all must be included and excluded in the test. During test design, other test aspects such as stimulating the system with productionlike test data and verifying how the system responds to the various inputs are also covered. Despite advancements in areas such as functional test automation and cloud testing, most organizations design tests manually. The prevailing approach for testing new functionality still involves manual test design and execution. By automating test design, organizations can reduce testing efforts significantly while increasing the quality of testing. Organizations can streamline and automate test design using test design tools. This whitepaper discusses UST Global s point of view on automated test design and our approach to test case design using test design tools. Challenges of manual test design The evolution of applications and software makes maintenance of test suites and the management of the related change impact extremely challenging. Manual test design is very time consuming and resource intensive. It can be best compared to using a hammer to wrench a screw into the wall when it can be done in more efficient ways. Testers create test cases based on their skills and knowledge rather than using industry-specific test design techniques. Although many test designers do not consciously use standard design techniques, they might use such techniques inadvertently. Without standard test designs, testers expose organizations to major risks with regard to the quality, coverage, and optimization of test packs. During the test design phase, testers tend to focus more on requirements thereby overlooking the business logic in test scenarios. Business rules and validations form a major part of the requirements provided by customers. Logical process flows comprise multiple branches, nodes, and decision boxes. Testers are expected to cover all branches of such a complex logic tree. The major challenges of manual test design include: Large number of test cases Lack of test case reusability Inability to measure test coverage Requires longer time to derive test cases Focuses more on requirements than work flows Lack of test design process standardization Inability to manage changing requirements and to trace test data to test cases Dependence on test design experts UST GLOBAL I USING A TEST DESIGN TOOL TO BECOME A DIGITAL ORGANIZATION I JUNE 2016 2
Manual test design versus test design with a tool Manual test design introduces a lot of risks and it is effort intensive, especially when the requirements changes. Test design tools primarily help decide which tests must be run. They generate test data and test cases. Test design tools provide an effective and disciplined way of addressing these shortcomings. They improve quality, business scenarios, fault detections, traceability, maintenance, model reuse, and requirements while reducing cost and time. Automated test design process is conducted in a series of five phases: Creating a test workflow model, Generating test cases and test data automatically Selecting test cases to create a test pack Running the test cases Comparing results with expected results from the model These five phases together form the test design tool. Figure 1 compares manual test design with tool-based test design. Intuitive decomposition test design Test design tool 50% More Effort Test design specification Generate detailed test case (data steps) Functional documentation Test execution Generate test design models Generate detailed test case (data steps) Test case & procedure procedure Figure 1. Comparison of traditional test design with test design using a tool. UST GLOBAL I USING A TEST DESIGN TOOL TO BECOME A DIGITAL ORGANIZATION I JUNE 2016 3
Test design tools: The benefits Using a test design tool, organizations can achieve business-it alignment in testing by capturing requirements in the form of process models. This helps identify incomplete or inconsistent requirements much earlier in the requirement gathering or analysis phase to facilitate effective and efficient reviews. Early identification of defects in the test cycle accelerates the time to market while containing costs and reducing total cost of quality. Testers can build robust and optimized test packs using test design tools. Test design tools help design clear and unambiguous logic, user stories, and test cases all at once. After designing an accurate set of test cases, the tool helps build the right data and link it directly to the requirements and test cases. With a test design software, organizations can ensure the coverage of all requirements using the minimum number of tests. Organizations can also accurately estimate costs with minimum effort, optimize How to select a test design tool? While selecting a test design tool, the extent of the tool s test design capabilities and ease of use must be the main criteria that must be assessed. The techniques and algorithms used by the tool for test modelling must also be considered. Four industry standard black box techniques are used for test designing: cause-effect graphing, state transition diagrams, classification tree method and flow chart modelling techniques. Cause effect graphing helps in selecting a set of high-yield test cases out of a very large set of input combinations, systematically. It also delineates incompleteness and ambiguities in the specifications. Flowchart modelling helps derive and maintain the smallest number of test cases needed to provide maximum coverage. The flowcharts contain all the qualitative information about a system. Users can leverage technical methods, without the technical skill or deep knowledge of a system. They can also reduce the number of test cases and maintain functional coverage. Test data designing mostly makes use of pair-wise testing, equivalence partitioning, and boundary value analysis techniques. Pairwise Testing is a test design technique that is used to ensure maximum test coverage. It helps design tests for applications that have multiple parameters. The tests are designed ensuring that for each pair of input parameters to a system, there are also all possible discrete combinations of those parameters. In the boundary value analysis technique, the tester tests the input values at the boundaries. Because input values recognized at extreme ends can cause errors in the functionality of the system, the input values are tested at the initial stages. Boundary value analysis eliminates common user input errors that can disrupt proper program functionality. UST GLOBAL I USING A TEST DESIGN TOOL TO BECOME A DIGITAL ORGANIZATION I JUNE 2016 4
It reliably exposes potential user interface or user input problems, enables clear guidelines on determining test cases and generates minimal test cases. Equivalence partition testing reduces the number of tests by dividing the different tests. The system will behave in the same way for all the tests with equivalence partition after the division. Equivalence partitioning also reduces the number of test cases and helps testers focus on smaller data-sets thereby increasing the probability of exposing defects. It also helps cover a large domain of inputs. The most effective combination of techniques to generate test cases linked by right test data is using cause and effect graphing and flow chart modelling along with pairwise test techniques. With this combination, test designers can ensure mapping of product requirements to easy-to-build and unambiguous flow charts thereby systematically optimizing test cases. By evaluating modelling expressivity of the tool to measure its capacity, the end users can rigorously express system behavior using its models. Such tools are easy to use and they reduce the dependency on test design experts. These tools generate models across industries and domain and they help align business and IT. Ideally a test design tool must provide the following capabilities: Optimize test cases with maximum coverage in minimal time Enable easy to understand and business oriented designs and test cases Manage change requests easily and ensure test data traceability to test cases Detect test cases that are impacted by a change in the requirements, fix broken test cases and create new ones Sequence and prioritize the test cases Import existing test cases from various other tools, find duplicate, invalid, or redundant test cases and build more efficient test packs Map the requirements to unambiguous flow charts and test cases Build the right test data and link it directly to requirements, tests, and expected results Export test cases to popular test manage ment tools With the move to digital platforms the use of test design tools becomes inevitable The increasing adoption of agile testing approaches that provide limited time for test design and execution make the case for test design tools stronger. As organizations move to a digital platform, the need for quality and faster time to market become even more pronounced. In fact, it becomes the key distinguishing factor between brands. In such circumstances test design tools become a natural choice for testers to ensure completeness, effectiveness, and efficiency. Test design tools help think through business cases and test scenarios in advance while providing new insights. Also, the modelling flexibility of the tools help users to design applications across various domains seamlessly. Although operating with these tools require a different mindset than with the traditional approach, once organizations overcome the initial hurdles, they stand to gain a competitive advantage because of the benefits of the tools. Author - Sreeja Raju Sreeja.Raju@ust-global.com UST GLOBAL I USING A TEST DESIGN TOOL TO BECOME A DIGITAL ORGANIZATION I JUNE 2016 5
ABOUT UST GLOBAL UST Global is a digital technology services company that provides next generation digital solutions for Global 1000 companies. Our mission is to Transform Lives using the power of digital technologies and the focus is on digital services and solutions. With a business model of fewer CLIENTS, more ATTENTION, UST Global strives for excellence in providing our clients with the best service and commitment to long-term client success. Headquartered in Aliso Viejo, California, UST Global has over 15,000 associates operating in 25 countries across four continents. For more information please visit www.ust-global.com