The Role of Web Services Testing

Transcription

1 A White Paper Analysis from Orasi Software The Role of Web Services Testing in Achieving Application Performance and User Satisfaction

2 Page 2 Contents 3 Introduction Web Services: They re Everywhere, But What Are They? 4 Testing in a Service- Oriented Environment Conventional Testing versus Web Services Testing 5 Approaches to Web Services Testing 6 The Benefits of Web Services Testing 7 Achieving Best-in-Class Web Services Testing 8 Web Services Testing Tools: A Primer Graphic: HPE UFT API canvas Factors to Consider When Selecting a Testing Tool 9 Should You Accelerate? Graphic: HPE UFT Results with API2Go Graphic: Orasi API2Go web utilities 10 Conclusion About Orasi Software, Inc.

3 Page 3 Introduction In today s application development environment, few applications are monolithic (completely standalone). Rather, they have become composite applications their functionality integrates with and depends upon service-oriented architectures (SOAs) collections of externally connected software components (services) whose sole function is to provide information or perform operations for the benefit of other systems. Web services are the mechanism by which these applications interact successfully with those services. Essentially, they are the glue that unites a client (e.g. the application) with a service. The only way to ensure that an application can connect to and successfully interoperate with each web service, every time, is through adequate testing of the web services. Yet, the sheer volume of testing such an approach makes the process daunting to most firms. Adopting practices and methods that create a workable web services testing framework is the key to resolving this challenge. It encourages teams to test web services thoroughly and at the appropriate time. That, in turn, helps ensure application reliability and performance and the subsequent user satisfaction that results from a good product. Achieving that goal is the subject of this white paper. Web Services: They re Everywhere, But What Are They? For those who are not intimately familiar with them, web services can be somewhat confusing. This short primer will help clear the confusion and pave the way for our web services testing discussion. Composite applications rely upon an array of services to perform activities, from authorizing a transaction to retrieving a concert ticket. Web services provide the connecting links between the client (application) and the services. Because of this, web services have become ubiquitous in composite application development literally, they are fundamental building blocks of application function. The W3C (Worldwide Web Consortium; a global standards body for the web) defines a web service as A software system designed to support interoperable machine-to-machine [e.g. mobile device, personal computer, server etc.] interaction over a network. More specifically, a web service negotiates and enables, through an instruction set described in a specific language, communication between two software systems running on disparate hardware devices. This communication makes it possible for some activity or information exchange to take place. There are numerous important concepts in the ecosystem of web services and SOAs, but five fundamental ones are: XML (extensible markup language), a language that can encode documents in a format that both humans and machines can read. Many web services are written using XML. WSDL (web services description language), a language that defines the rules for communication (as well as any relevant error messages that might display) between the system that makes a request (the service requester) and the one that processes and responds to the request (the service provider). The information in the.wsdl file also enables the provider to validate each data request. UDDI (universal description, discovery and integration), a directory that outlines which software systems provide which types of data, enabling the service requester to reach the correct provider.

4 Page 4 SOAP (simple object access protocol), a protocol by which a requesting system can contact the service provider, and by which the provider would then respond. It is a universal protocol that supports communication between applications running on different OSs, technologies and programming languages. Initially, SOAP was part of the specification that included WSDL and UDDI, but it can now be used alone through hard coding of messages. REST (representational state transfer), a type of computing architecture whose interaction constraints enable each request and response in a data exchange to happen in isolation. This approach eliminates the need for servers to retain session or status information. HTTP (on which the Internet is built) is a RESTful protocol. RESTful APIs (application programming interfaces) do not require the use of SOAP or WSDL. Testing in a Service-Oriented Environment SOAs make it easier for applications and machines to interact and exchange data with one another. However, they also make testing more complex and differentiate it from conventional software testing. The service function consists of myriad services, database extraction layers, message queues, and other headless entities (no GUI) that provide important information and business logic to applications. Not only must an application and its functionality be thoroughly tested, end to end, but each web service must also be tested and its functionality and connectivity trusted before the developing organization should integrate it into the application. Testing of web services and their integration with an application generally encompasses functionality, interoperability, security, boundaries, standards compliance and performance testing. Some of these tests are nearly absent from conventional testing. Others apply to conventional testing, but for web services testing they take on an enhanced role. Web services testing requires extensive and intensive interoperability testing, which isn t common in conventional testing. In the course of standard operation, an application might interact with dozens if not hundreds of web services, strung together in a complex web. Ensuring that all functions perform as anticipated, not only by themselves and in coordination with other web services and service APIs is a process that must be given appropriate attention. Likewise, performance testing also becomes more complicated with web services testing. In a sophisticated operating environment where multiple players interact across disparate networks, latency and connection disruption can severely impact delivery of service. Conventional Testing versus Web Services Testing There are other differences between conventional testing and web services testing, in addition to the types of testing and their importance. Web services testing is often left to developers, if it is conducted at all. Web services do not have a GUI and they are not dependent upon the availability of the application GUI. Because developers have more knowledge of application APIs and are able to work outside a GUI-based framework, the assumption is that they should handle web services testing. The result can be that they overlook web services testing in the rush to complete the project. Impediments can dramatically impact the completion of web services testing. For example, testing can be hampered by obstructions such as the unavailability of third-party services at specific times or the cost of connecting to those services hundreds or thousands of times.

5 Page 5 Web services testing includes (or should include) checking the performance of individual SOA tasks, such as discovery and binding (confirming that a web service can be found and reached). This is a process that is not applicable to conventional testing. Automation, while valuable in all testing, essentially becomes an obligation with web services testing. Automation is the only practical means for organizations to sufficiently test all integration points and verify the accuracy and quality of the end-to-end business process across all devices. All of these issues, and more, render some (but not all) conventional testing approaches unfeasible. They also make it more difficult for conventional testers to make the leap to web services testing. Nevertheless, Orasi believes that testing is best accomplished by quality-focused testers. When organizations structure their web services testing programs properly, there is no reason why testers cannot run the effort. Approaches to Web Services Testing Like conventional testing, web services testing has evolved over time. Today, there are a variety of approaches to web services testing that organizations use. Service Mocking: Service mocking is the process of building (using a simple testing tool) a mock or simulated web service to stand in for one that has not yet gone live. Creating a mock web service is a multi-step process where the tester creates the service, assigns a communication port, and then creates a list of operations to approximate how the real web service might act. The tester also creates the responses that the real web service should provide. Service mocking can be an invaluable tool when the service is physically completely unavailable, but it should never be used as a testing shortcut if other avenues exist. Parsing: With parsing, a test tool analyzes data, such as the characters in a WSDL file, to determine if they are logical and conform to specifications (e.g. syntax compliance). Parsing plays a fundamental role in web services testing, but it provides no significant insight into how a web service will actually function in a live environment. Manual: With manual web services testing, a tester stands in for a user and tests the web service connection and function manually by running the application on a device. (Depending on the web service and its accessibility, manual testing can also take place at the server or web site level, but this does not provide any insight into how the web service will function with the application.) Users may employ a variety of devices, depending on the breadth of the application s device support. Even so, it is essentially impossible for companies to cost effectively test how all web services respond for all devices on which an application could potentially be installed, even in the limited range of an operating system like ios. Furthermore, testing a web service on a device is

6 Page 6 cumbersome, and it does not provide sufficient insight into how a web service will react or respond across a variety of network and operating conditions. Orasi does not recommend manual web service testing except as a component of user testing after thorough, individual testing of web services is complete. Automation: As we mentioned earlier, automation is a key to effective, accurate web services testing. Historically, testers have written automated testing scripts manually, often with the help of recorded web service calls and responses. Once the scripts were complete, testers could use various tools to automate running of the scripts against the live services. Today, organizations can leverage much more sophisticated automation tools to test not only the web services themselves, but also the integration points and interoperability of those components with each other and the composite application. Automating web services testing and testing at the API level lets organizations isolate the API layer and test integration points, then verify and validate the quality of the end-to-end business process. The Benefits of Web Services Testing While service mocking, parsing and manual testing have limitations when used manually or independently, they all can play a role in effective web services testing. When used in concert with automation, they become part of a complete web services testing framework that can propel an enterprise to much greater application quality, reliability and user adoption/satisfaction. Two major benefits to utilizing a web services testing framework include testing faster and testing more. In a proper automation framework, an API test can run and complete in a matter of seconds. Since API tests execute so quickly, the team has more time to test more scenarios, which provides more opportunities to find and locate defects within the system under test. When combined with a Test Data Management Strategy, web services testing can be taken even further to include hundreds or thousands of additional test cases or business processes. In addition, the high ROI of testing in particular automation testing is well documented. Based on pure financial calculations alone (the cost of automation tools, licensing, training, etc., per build, compared with the cost of equivalent manual testing resources), the ROI for automation testing is more than 160%. (Curious if these figures apply to you? Estimate your own testing ROI using a calculator from the Automated Testing Institute.) For web services functional testing, where large numbers of repetitive tests are the only way to ensure accuracy and test consistency, the ROI of automated web services testing becomes even more impressive. Unfortunately, these ROI numbers don t always add up if organizations are not engaging in defined and managed testing. That s a regrettable outcome that pragmatic organizations will correct.

7 Page 7 While service mocking, parsing and manual testing have limitations when used manually or independently, they all can play a role in effective web services testing. Finally, with user expectations at an all-time high (and increasing), companies cannot afford to place insufficient emphasis web services testing. Users abhor slow-loading web sites (2-3 seconds is a common expectation), so they look to applications to satisfy their information needs instead. Yet, more than 90% of web and mobile applications are abandoned within a month. Mobile apps undergo the greatest scrutiny 24% of users gain a negative perception of a company if its mobile app doesn t function well and load quickly. In such an environment, companies that do not expend the time and effort to perform sufficient web services testing are not only lowering the quality of their composite applications; they are likely hurting their bottom lines and potentially their business continuity, as well. Achieving Best-in-Class Web Services Testing By this point in our paper, hopefully you are on board with the premise that robust web services testing is a chief component of superior application performance and user satisfaction. That doesn t mean this type of testing owns the development lab or that it requires its own ecosystem or platform. Rather, web services testing works best when it is integrated into a properly managed and executed overall testing strategy. Some factors to consider when accomplishing this goal include: Crafting a test plan and test data management plan that incorporates web services testing is the best way to promote its complete integration into the testing process. Testing should start early in the project, once stakeholders have agreed upon business processes (e.g. use cases) and interface definitions. However, that does not mean that testing should be left to developers. With a properly structured framework, QA-focused testers can and should be performing web services testing. Quality assurance tests should occur at multiple (key) phases of the project. It s important to share these results with others quickly and early as part of the test plan. Test coverage analysis is especially important in web services testing to ensure reliability of the results. The use of web services testing tools, including automation tools and accelerators, greatly increases efficiency and reduces the chances that tests will be overlooked or improperly executed.

8 Page 8 Web Services Testing Tools: A Primer We have alluded to testing tools several times in this white paper. As most developers and testers learn early on, tool is a generic term that can describe a very wide array of helper applications, from simple test case editors to sophisticated automated testing platforms. For example, here is how a simple web services tool might work: Generate a project from the web service s WSDL code (file or end point) Build, with tester-provided information, test cases and test suites (groupings of interrelated test cases) Run the tests by sending requests and receiving responses (when connection can be achieved) Analyze and determine each test s success or failure based upon how the response relates to user-provided assertions and provides results. Micro Focus UFT API canvas There are more sophisticated yet easier-to-use web services testing tools, as well. Consider, for example, a functional testing tool for web services that could eliminate scripts in favor of a completely graphical interface. Such a tool would visually construct tests on a digital canvas. The result would be a nearly codeless testing approach that eliminates the objection that testers and QA teams cannot perform web services testing. The tool we have just described exists, today. Factors to Consider When Selecting a Testing Tool Every organization must decide how it will approach testing. For those wishing to embrace worldclass testing with automation, we recommend using these criteria: Does the tool offer a fully, or at least partially, visual interface (GUI)? Does it offer out-of-the-box support for XML, WSDL, HTTP, REST, JMS, etc.? Does it offer drag-and-drop functionality for creating and editing test cases? Is the tool simple to use and learn, with minimal training required? Does it reduce repetitive activities, for example enabling the user to create new tests from a template created by an earlier effort? Does it support text-based tables and Excel files? Can it be customized, for example, with custom code or event handlers? Does it support web services security configuration (X509, Kerberos, etc.) Does it support API testing? Does it offer test management features?

9 Page 9 Should You Accelerate? Earlier in the white paper, we mentioned accelerators. These types of testing tools enhance and extend the functionality of other testing tools to accelerate test execution and validation. One example is the Orasi accelerator, API2GO, which provides a standardized, stable and integrated web services testing framework. API2Go creates requests and validates responses more completely and faster than an automated testing tool alone. It drives more data and data combinations, validates services responses, and performs data and structural validation. It can also expand the scope of the data iterations and be leveraged to populate needed data for UI tests. With API2Go, script Micro Focus UFT Results with API2Go development times are reduced by an average of 50%, while maintenance is decreased by an average of 62.5%. API2Go also reduces production release risk, reducing test execution time by 70% over UI automation. Orasi API2Go web utilities

10 Page 10 Conclusion As many companies begin to realize after their first software release, correcting defects in a production environment is more expensive and time consuming than correcting them with thorough testing. This is especially true for composite applications that rely significantly on web services. Here, poor connectivity and interoperability can render an application essentially useless. Poor performance also reduces user satisfaction and adoption, which statistics show can lead to app abandonment and even diminished customer loyalty. Yet, many organizations continue to test web services inadequately, if at all. Appropriately structured and implemented with quality tools, web services testing can be started as soon as services are available, enabling QA teams to discover defects sooner, when they are less expensive and faster to remediate. Thorough web services testing shortens testing cycles, speeds development of quality products, and when combined with a test data management strategy, provides the ability to fully and effectively test the business process. The bottom line is that prudent companies can no longer ignore or incompletely address web services testing. They must adopt the approaches, tools and procedures to ensure web services testing occurs at the right time and with the appropriate emphasis in the application lifecycle. Whether they create this testing framework in-house or outsource it to third-party consultants and advisors like Orasi, there is no time to waste. About Orasi Software, Inc. Orasi is an award-winning software reseller and provider of software training, support and professional services. To help companies focus on a complete software quality lifecycle, Orasi offers market-leading consulting services and solutions to support automated testing, application performance management/intelligence, mobile technologies, DevOps pipeline efficiency, and operational excellence. Orasi continues to expand its offerings across the entire software delivery spectrum, from data analytics to continuous delivery and open source tooling. Orasi maintains strategic partnerships with Micro Focus (formerly HPE), Chef, Delphix, SAP, XebiaLabs, and others. For more information, please visit For more information, contact Orasi today Orasi Software, Inc. The information contained herein is subject to change without notice. Orasi is a trademark of Orasi Software, Inc. All other product and company names are trademarks or registered trademarks of their respective owners. Document #: 01WP0215A R1116