D45.4 Test assertions

Transcription

1 Document Identification Date 22/04/2015 Status Final Version 1.1 Related SP / WP SP4 / WP45 Document Reference Related Deliverable(s) D45.03, D45.5 Dissemination Level Lead Participant USTUTT Lead Author Janina Hofer Contributors NRS Reviewers ATOS, AG properties/ PU This document is issued within the frame and for the purpose of the FutureID project. This project has received funding from the European Unions Seventh Framework Programme (FP7/ ) under grant agreement no This document and its content are the property of the FutureID Consortium. All rights relevant to this document are determined by the applicable laws. Access to this document does not grant any right or license on the document or its contents. This document or its contents are not to be used or treated in any manner inconsistent with the rights or interests of the FutureID Consortium or the Partners detriment and are not to be disclosed externally without prior written consent from the FutureID Partners. Each FutureID Partner may use this document in conformity with the FutureID Consortium Grant Agreement provisions Document name: Page: 0 of 46

2 1. Abstract This deliverable gives an overview of the server testbed. The testbed is based on the first internal version of the testbed that is described in D In this deliverable we want to outline the further development of the testbed. The usage of the virtual machine is no longer required in order to write, implement, execute and analyze server tests and assertions. However, this deliverable focuses on the process of implementing server tests and assertions. This process comprises the implementation of tests and assertions using the tool SoapUI, the integration of the tests into the code repository as well as the automatic execution of integrated tests by the main Jenkins platform (ci.futureid.org). Server tests and corresponding assertions have been developed based on existing specifications (e.g. D41.3). Document name: Page: 1 of 46

3 2. Document Information 2.1 Contributors Name Janina Hofer Trenton Schulz Lothar Fritsch Partner USTUTT NRS NRS 2.2 History Version Date Author Changes Janina Hofer First draft of the document and its structure, Chapter 1. Abstract Janina Hofer Chapters: 4. Introduction, 8. Summary, 6. Server testbed, Chapter 7.1 Test Case description Lothar Fritsch Trenton Schulz Chapter 7.2: Test assertions Completion of chapter 6: Server testbed Janina Hofer Chapter 6: test assertion creation in FutureID Janina Hofer incorporation of the review recommendations and finalization Document name: Page: 2 of 46

4 3. Table of Contents 1. Abstract 1 2. Document Information Contributors History Table of Contents 3 List of figures 5 List of tables 6 Abbreviations 7 Project Description 8 4. Introduction Scope Limitations Server testbed SoapUI Git repository Maven and the Project Object Model (POM) Execution of Tests in Jenkins Result presentation Test assertion creation in FutureID Analysis and creation of descriptive test cases Development of test messages Implementation of the test cases Implementation of a test cases and corresponding assertion Test case: FutureID Authenticate Interface Writing and implementing test cases and corresponding assertions in SoapUI Test Case 1: Authenticate Function - Asynchronous with valid RequestID Test Case 2: Authenticate Function - Asynchronous with invalid RequestID Test Case 3: Authenticate Synchronous with valid SessionIdentifier Test Case 4: Authenticate Synchronous with both valid SessionIdentifier and valid RequestID Test Case 5: Authenticate Synchronous with invalid SessionIdentifier Summary 44 Document name: Page: 3 of 46

5 8.1 Conclusion Outlook References 45 Document name: Page: 4 of 46

6 List of figures Figure 1: Server testbed components and connections Figure 2 - SoapUI Navigator Figure 3 - How to access the latest code from the Git repository Figure 4 - pom.xml file with the link to two SoapUI projects in the tag <executions> Figure 5 - Build overview in Jenkins Figure 6 - Output from a Jenkins run of the server tests Figure 7 - Results page for all builds in the Server Testing project Figure 8 - Display individual build results in Jenkins Figure 9 - An example SOAP message Figure 10 - An example REST message with XML representation Figure 11 - An example HTTP GET message Figure 12 - An example HTTP POST message Figure 13 - Context menu for creating a new request Figure 14 - Example of an implemented SOAP message Figure 15 - Creation of a HTTP POST or HTTP GET request Figure 16 - Example of a Property Transfer between a SOAP response and a SOAP request Figure 17 - Example of an Assertions Section with two implemented Assertions Figure 18 - Authentication with FutureID Client present Figure 19 - Overview on the message format for the authenticate function Document name: Page: 5 of 46

7 List of tables Table 1 - Template for developing descriptive test cases Table 2 - Overview on applicable test assertion types in SoapUI Document name: Page: 6 of 46

8 Abbreviations eid SVN SOAP WSDL REST electronic identity Apache Subversion (often abbreviated SVN, after the command name svn) is a software versioning and revision control system Simple Object Access Protocol is a messaging protocol that allows programs that run on disparate operating systems (such as Windows and Linux) to communicate using Hypertext Transfer Protocol (HTTP) and its Extensible Markup Language (XML) Web Service Definition Language is an XML format for describing network services as a set of endpoints Representational State Transfer is a software architecture style SCM SSH POM XML HTTP GET HTTP POST WADL XPath AB UA BC Source Control Management Secure Shell is a cryptographic (encrypted) network protocol for initiating text-based shell sessions Project Object Model is an XML representation of a Maven project held in a file named pom.xml Extensible Markup Language is a markup language that defines a set of rules for encoding documents in a format which is both human-readable and machine-readable. Hypertext Transfer Protocol is an application protocol for distributed, collaborative, hypermedia information systems. The method GET requests a page from a server. The method POST requests that the server accepts the entity enclosed in the request as a new subordinate of the web resource. Web Application Description Language is a machine-readable XML description of HTTP-based web applications (typically REST web services). XML Path Language, is a query language for selecting nodes from an XML document Authentication Backend User Agent Broker Client Document name: Page: 7 of 46

9 Project Description The FutureID project builds a comprehensive, flexible, privacy-aware and ubiquitously usable identity management infrastructure for Europe, which integrates existing eid technology and trust infrastructures, emerging federated identity management services and modern credential technologies to provide a user-centric system for the trustworthy and accountable management of identity claims. The FutureID infrastructure will provide great benefits to all stakeholders involved in the eid value chain. Users will benefit from the availability of a ubiquitously usable open source eid client that is capable of running on arbitrary desktop PCs, tablets and modern smart phones. FutureID will allow application and service providers to easily integrate their existing services with the FutureID infrastructure, providing them with the benefits from the strong security offered by eids without requiring them to make substantial investments. This will enable service providers to offer this technology to users as an alternative to username/password based systems, providing them with a choice for a more trustworthy, usable and innovative technology. For existing and emerging trust service providers and card issuers FutureID will provide an integrative framework, which eases using their authentication and signature related products across Europe and beyond. To demonstrate the applicability of the developed technologies and the feasibility of the overall approach FutureID will develop two pilot applications and is open for additional application services who want to use the innovative FutureID technology Future ID is a three-year duration project funded by the European Commision Seventh Framework Programme (FP7/ ) under grant agreement no Document name: Page: 8 of 46

10 4. Introduction For the proper development of the server backend, testing of the different components is very important and is not to be neglected. Especially towards the end of the development of such a complex backend infrastructure that is constantly updated by new functions, testing and verification of existing and new developed components is essential. This deliverable gives instructions on how to use the server testbed and how to install and use the delivered toolset for writing tests and assertions. 4.1 Scope This server testbed pursuits to serve as an easy to use and holistic set of tools that enables each interested partner to test certain server components and functions. Each partner can easily access and use the provided toolset to write and execute own server tests and assertions. This testbed is developed as a modular and thus flexibly expandable infrastructure which is a very helpful toolset for each developer within this project and beyond. 4.2 Limitations The creation of server tests and implementation of test assertions are limited by several factors. Due to not yet implemented endpoints it cannot be assured that each server component is tested in the same detailed way. However, endpoints that cannot be used by the time of this deliverable can either be integrated in a later step or a mockup, e.g. for different credential authentication, can be used as a workaround. Furthermore, at the present time unclear or not existing specifications cannot be used as a basis for server tests and assertions However, these limiting factors have minor impact on the overall structure and handling of the server testbed. Document name: Page: 9 of 46

11 5. Server testbed The server testbed is administered by the University Stuttgart. However, partners who need access to the server testbed in order to write and execute own tests and assertions will basically need the following tools and methods that are presented in this chapter. The server testbed essentially consists of two components, which work together. One component, the SoapUI tool, is mainly used to create and execute tests right away. The second component consists of an installation of Jenkins, which controls the execution of the SoapUI tests and gathers test results accordingly. Besides these main components of the server testbed, it is also necessary to use the FutureID SVN repository to save and versioning the data. The illustration below gives a brief overview about the used tools and the required steps to follow, in order to use the server testbed properly. Figure 1: Server testbed components and connections The following sections will describe the general process beginning at the creation of test cases and assertions over their execution in Jenkins until the presentation of the result. Besides the description of the used tools and methods, important steps will be explained in more detail. Document name: Page: 10 of 46

12 1. Figure SoapUI 2 - SoapUI Navigator Navigator 5.1 SoapUI opened SoapUI projects including their test suites, test cases, and test steps. 5.1 Git repository SoapUI 1 is a free and open source functional testing tool for server components and web services. It provides testing means for SOAP/WSDL based services as well as for REST services and comes along with a graphical user interface. With the use of SoapUI, automated functional tests of the different components developed within FutureID can be created and executed. SoapUI provides complete test coverage and supports all standard protocols and technologies. The testing process in SoapUI starts with the creation of a new project or with the opening of an already existing project. By adding a WSDL, requests for all operations in that service can be created. Once the project is created, the navigator in form of a tree structure shows all Many distributed teams contribute to the FutureID infrastructure. This is also the case for the testbed development, which e.g. includes already implemented test cases and assertions in SoapUI. Eventually, it is necessary to follow a structured approach to save and share files and data among the team in order to avoid unnecessary double work or in a worst case the loss of important data. Therefore, a Git repository is used as the central tool to avoid the above mentioned problems. Git is a distributed source control management (SCM) written for performance and easy branching in mind. It is an open source system that was created after the Linux project lost the ability to use the BitKeeper SCM. Linux founder, Linus Torvalds, wanted to have the same capabilities that he had with BitKeeper, but there was no comparable open source variant (at least performance-wise). So, he wrote the initial version of Git 2. Other systems, like Mercurial, appeared soon after Git that have very similar performance and features, but Git is the most popular system. One of the advantages of Git is that it is distributed and fast. It makes it very easy to branch code and merge changes from other branches, regardless of where the changes live. This means that it can be adopted into any workflow that is required by the project needs. Successful branching models have been 1 Download and tutorials: 2 Torvalds, L. (2007). Tech Talk: Linus Torvalds on git. Document name: Page: 11 of 46

13 introduced for development 3 to ensure that there is always a known good branch that can be built (for example for a release). In the same vein, Git is also well suited to a continuous integration system. An example system is outlined by Tvete 4 ; in his post he outlines how the main release branch is open to only an integration script. Developers commit changes to an integration branch. The script integrates the changes into a test branch based on the current release branch and sees if there are any regressions in tests. If there are no regressions, the test branch is then integrated to the release branch. The result should be that the number of test failures for a branch should go down and one could create a release at any time. FutureID uses a similar idea for its continuous integration system. There are multiple Git branches for development. Jenkins can monitor the different branches and quickly pull and run tests against the different branches. This can go either way (that is, new tests are added or code for other modules is changed). The main repository for the testbed is available via ssh by fidgit@git.futureid.org:testbed. Jenkins can be set up to monitor branches in Git. The figure below describes the initial step to load the latest testbed code Document name: Page: 12 of 46

14 Figure 3 - How to access the latest code from the Git repository 5.2 Maven and the Project Object Model (POM) Jenkins runs on top of established Java technologies like Maven. Maven is a tool for software management and comprehension. Using the project object model (POM), an XML file that describes the project and actions that can be performed, Jenkins can execute all included test cases and assertions. Maven can do different tasks (or goals in its own language), such as building a project, downloading dependencies, generating documentation, or executing scripts. Jenkins can call maven directly and execute goals that are in a POM file. To run the SoapUI test cases, one simply includes them in the POM file under the proper goal, which is in this case the tag <executions>. (See Figure 4 on page 14) In order to make sure that Jenkins runs the latest code, the updated pom.xml as well as new or renamed SoapUI projects that are linked in the pom.xml need to be uploaded to the FutureID testbed repository, as described in Chapter 5.1. Document name: Page: 13 of 46

15 Figure 4 - pom.xml file with the link to two SoapUI projects in the tag <executions> This tag can hold different test suites that will be automatically executed in Jenkins. Both the test cases and the POM file need to be under the source code management. Thus, to make sure that Jenkins always uses the latest code, each new or renamed SoapUI project needs to be added and committed to the repository. Additionally, the pom.xml always needs to be adapted if a new file has been added to the repository or if a file has been renamed. 5.3 Execution of Tests in Jenkins The tests are executed as part of the server tests view in Jenkins. Jenkins then monitors the branches it is supposed to pull from and when a change happens, it pulls the changes, attempts to build the new release, and runs the tests against it. Document name: Page: 14 of 46

16 Figure 5 - Build overview in Jenkins Each of these builds is considered a job and each gets a unique ID to refer to it later. If there are problems these are flagged so that they can be fixed and so that developers know when they were introduced. Jobs can also be executed at will if desired by a user and the output from a job run can also be checked (Figure 6). Figure 6 - Output from a Jenkins run of the server tests. Document name: Page: 15 of 46

17 5.4 Result presentation Jenkins has its own presentation of the results that can provide a view at a glance (Figure 7) or one can drill down to each individual job (Figure 8). Figure 7 - Results page for all builds in the Server Testing project. This works well if one has access to Jenkins, but requires having access to Jenkins. This might not be desirable for non-developers. The work in D45.5 also details how it is possible to pull XML versions of the results from Jenkins and transform them into to a new format that makes it possible to include them into the FutureID wiki for general distribution among the consortium. Document name: Page: 16 of 46

18 Figure 8 - Display individual build results in Jenkins. Document name: Page: 17 of 46

19 6. Test assertion creation in FutureID The creation of test assertions for server components is based upon three phases: (1) Analysis and creation of descriptive test cases, (2) development of test messages, and (3) implementation of the respective test cases. The following sections provide a detailed overview about each phase and all necessary steps for each phase will be outlined. 6.1 Analysis and creation of descriptive test cases All server development efforts in FutureID are accompanied by requirements and specifications. In order to create server test cases, these deliverables serve as the starting point for analysis and eventually for the creation of server tests. In the example FutureID Authenticate Interface, which is described throughout the chapter 7.1, a test case was derived from the Deliverable D41.3 [1]. That deliverable provided a concrete description of the necessary in- and outputs for the creation of abstract test cases. The respective test cases developer will hereby need to ask the following questions: What are the in- and outcoming messages supported by the component? The test case developer requires a list of expected messages of the server component that is going to be tested. Hereby the test case developer must distil both the mandatory and optional attributes for each message as well as their format. Additionally, all messages must be distinguished by either being an input or an output message. In the case of an output message, the test case developer must distil the expected attributes as well as their expected contents. What does the information flow surrounding the server component looks like? Server components are often orchestrated among other server components in order to provide a flexible approach for authentication. The test case developer must therefore identify the information flow of the regarded components and their endpoints. The test case developer should in any case identify the respective information flow of the corresponding components. In the case of the example shown in chapter 7.1, the specification showed that the endpoints of the corresponding components (Authenticate and Activate) were both orchestrated with an abstract authentication process. Hereby, the authentication process immediately follows after the activate function is called. This is the case, because the Authenticate endpoint may only be called for attribute provision after the authenticate process has been fully executed. The authenticate call, both after successful and unsuccessful authentication, needs to be fully tested regarding error handling and functionality. This example shows that the information flow regarding the orchestration of a component s endpoints provides critical information for the test case developer. If the test case developer follows both questions along the specification of a component, he/she has now identified all necessary information to develop descriptive test cases. These descriptions will be Document name: Page: 18 of 46

20 integrated in the second and third phase of the test assertion creation. The test case developer can hereby utilize the template shown in Table 1 for development of the descriptive test cases. Test case title: Affected endpoint(s): Description: Type (tick): Error Handling Functionality Message 1 Title: Description: Affected Endpoint: Operation: Type: SOAP REST HTTP GET HTTP POST Description of required Input: Description of expected Output:. Message n Title: Description: Affected Endpoint: Document name: Page: 19 of 46

21 Operation: Type: SOAP REST HTTP GET HTTP POST Description of required Input: Description of expected Output: Table 1 - Template for developing descriptive test cases This template is separated into two different sections. The upper section contains descriptive information on the test case itself. As the test case may include orchestrated endpoints of the component, this includes a list of all affected endpoints along with the title. The endpoint developer must hereby list all endpoints which are included in the bottom message section as affected endpoints. If endpoints are not listed in the upper test case section, they cannot be included in the bottom message section. The test case title should be unique per component. Therefore, two test cases with the same title may exist for different components. The title should briefly describe the test case, enabling an easy identification and recognition of the test case in later phases. <soapenv:envelope xmlns:soapenv=" xmlns:urn= > <soapenv:header> <! Header Data --> </soapenv:header> <soapenv:body> <urn:function someattribute= somevalue > <urn:someelement> somevalue </urn:someelement> </soapenv:body> </soapenv:envelope> Figure 9 - An example SOAP message If two or more test cases of a component are named identically, and if the titles briefly reflect the test cases contents, this may imply that only one test case would be required in that case. By following the affected endpoints, the test case developer must provide a description of the test case. The description should include the focus of the test case, e.g. a function of a component, or error handling capabilities, as well as a brief description of the message flow, while including the a brief description of the expected Document name: Page: 20 of 46

22 input at the beginning, along with the expected output at the end of the message flow. Finally, an abstract typology of the case concludes the upper test case section. This typology enables the test case developer to indicate whether the case aims at testing the error handling (Error Handling) of a component or its successful functioning (Functionality). By including this information, the developer ensures that the test case coverage both for successful and unsuccessful functioning of the component may be monitored. POST Host: SomeService Content-Type: text/xml; charset=utf-8 Content-Length: 123 <?xml version="1.0" encoding="utf-8"?> <function> <someattribute>somevalue</someattribute> <someelement>somevalue</someelement> </function> Figure 10 - An example REST message with XML representation Below the upper test case section, the message section includes all messages required in that test case. Hereby the messages are not readily implemented as they are in phase 2, but briefly described in order to enable a structured approach on the test case implementation both in phase 2 and phase 3. Each message block hereby includes the affected endpoint, whereas each message block cannot include multiple endpoints. Please note, that all endpoints used in the message area must be included in the upper test case area. Further attributes for a message include the title, whereas each message title must be unique for each component. The title should indicate required input for the message. The output must not be included in the title, as one message may produce multiple outputs (e.g. in different situations inside the component s information flow). Figure 11 - An example HTTP GET message This enables the test case developer to aggregate and reuse these messages in other test cases for the component, similar to the approach taken for the test cases. Additionally, each message should include a brief description of the message, providing context on what the message sends and receives as well as why this message is required. The test case developer should also indicate for each message, which protocol is being used for communicating with the endpoint. Hereby, the test case developer may choose between SOAP (see Figure 9), REST (see Figure 10), HTTP GET (see Figure 11) and POST (see Figure 12), whereas SOAP refers to SOAP messages, REST to REST messages, HTTP GET to attributes delivered via HTTP GET Forms, and HTTP POST to attributes delivered via HTTP POST Forms. Document name: Page: 21 of 46

23 POST /someendpoint HTTP/1.1 Host: SomeService do=function&attribute=somevalue Figure 12 - An example HTTP POST message Finally, the required input lists all attributes that are required in the message block, whereas the expected output describes the result that should be concluded by the endpoint from the given input. Each test case may hereby include multiple messages. 6.2 Development of test messages The functional testing tool SoapUI enables the testing of services by using SOAP messages, REST messages or classical HTTP GET and POST parameters. Thus, to develop the test messages, the test case developer included the message protocol while describing the test messages. Further, the test case developer can now use the test case description and implement each test message based on the expected input, output, and protocol. Hereby, one message per unique test message should be implemented. The implementation of test messages requires the test case developer to choose a test case, integrate the corresponding endpoint of the first message that uses the SOAP or REST protocol. When these steps are done, the test project has been generated in SoapUI. If the first message of the test case indicates a SOAP or REST message, the developer is able to automatically create empty test messages by importing the Web Service Description Language file (wsdl) or the Web Application Description Language (wadl). Both are capable of generating test messages separately from the test cases. However, in case of HTTP GET or HTTP POST messages, a workaround is required, which will be described later. If a new SOAP or REST project has been created, the test case developer may implement the test message by choosing the Service Binding and selecting New request in the context menu of the respective operation indicated in the message template. The test case developer now uses the expected input section of the message template to create the message. Hereby, dynamic input (e.g. inputs required from the output of previous messages) may be left blank. The messages created in this step should only provide templates during the implementation of the test cases and therefore only include static values, which are required during the requests. Figure 13 - Context menu for creating a new request Document name: Page: 22 of 46

24 Once the message is implemented the test case developer proceeds with further SOAP and REST messages of all test cases. If a SOAP or REST message includes an endpoint different from the previously implemented SOAP or REST message, the test case developer may add a new endpoint by importing a WSDL or WADL. This can be done by using the context menu of the test project and choosing Add WSDL or Add WADL. 6.3 Implementation of the test cases Once the test case developer has implemented all SOAP and REST messages, the test cases can be fully implemented. Hereby the test case developer uses the context menu on the test project in order to create a new test suite. SoapUI arranges multiple test cases in one test suite. The test case developer may hereby name the new test suite according to the service binding. Once the test suite has been created, the test case developer can create new test cases by choosing New Test Case from the context menu of the test suite. This is done by the developer for each documented descriptive test case. For each test case, the developer may now create new test steps. This is done by implementing each test message in the order indicated in the descriptive test cases. If the message indicates a SOAP or REST protocol, the request has already been implemented. In this case, the test case developer may choose the respective message from the service binding and choose Add to Test Case from the context menu of the respective message template. Figure 14 shows a screenshot of an implemented SOAP message. Figure 14 - Example of an implemented SOAP message Document name: Page: 23 of 46

25 If the message indicates a HTTP POST or HTTP GET, the test case developer may use the context menu in the Test Steps section of the dedicated test case, and choose HTTP Test Request. The message can now be implemented by defining the endpoint, naming the test message according to the title in the message description, and including input parameters (see Figure 15). Finally, the test case developer may choose between the POST and GET method, depending on whether the protocol of the message indicates HTTP POST or HTTP GET. Figure 15 - Creation of a HTTP POST or HTTP GET request Finally, if the expected input described in a test message indicates dynamic input, depending on the output of the previous message (e.g. by passing on a session identifier), the test case developer creates a transfer property element, which can be found in the context menu Add Step of the test suite. This element enables the developer to invoke the transfer of a property in response of a previous message to the request of a following message. If the messages provide a XML based response, e.g. in the case of SOAP message or REST message with XML resource presentation, the developer can use XPath and XQuery to select the contents of the response and the request (see Figure 16). However, if one or both of the messages use HTTP POST or HTTP GET, the test case developer can choose the targeted property of both the response and the request by using the property selection. Document name: Page: 24 of 46

26 Figure 16 - Example of a Property Transfer between a SOAP response and a SOAP request Once all test messages, and required property transfers have been implemented, the last step towards the successful implementation of a test case is provided by including the automatic validation of the response. Therefore, the test case developer may now consult the expected output section of the message description. The information indicated in the template can now be implemented by using the Assertion section inside SoapUI (see Figure 17). Hereby the test case developer should implement an assertion for each listed expected output in the described message. Figure 17 - Example of an Assertions Section with two implemented Assertions The following Table 2 on page 20 provides an overview on the available and applicable assertions in FutureID. The test case developer may choose from these types in order to implement an assertion for the expected output described in the descriptive test cases for each message. Document name: Page: 25 of 46

27 Assertion in SoapUI Description Applicable to Contains Not Contains XPath Match XQuery Match Valid HTTP Status Codes Invalid HTTP Status Codes Not SOAP Fault SOAP Fault Schema Compliance Script Assertion Enables the test case developer to search for a certain content in the response using Regular Expressions Enables the test case developer to validate the absence of certain content in the response using Regular Expressions Enables the test case developer to validate the existence of certain content using XPath Enables the test case developer to validate the existence of certain content using XQuery Enables the test case developer to validate the existence of selfdefined HTTP Status Codes Enables the test case developer to validate the absence of selfdefined HTTP Status Codes Enables the test case developer to validate that the response is actually a SOAP response Enables the test case developer to validate the response is not a SOAP response Enables the test case developer to validate that the response is valid against the associated WSDL or WADL Enables the test case developer to implement a script for All protocols All protocols SOAP and REST messages with XML resource presentation SOAP and REST messages with XML resource presentation All protocols All protocols SOAP HTTP GET, HTTP POST, REST SOAP, REST All Protocols Document name: Page: 26 of 46

28 checking the validity of the response Response SLA Enables the test case developer to define service levels regarding the response (e.g. response time) All Protocols Table 2 - Overview on applicable test assertion types in SoapUI Once all messages of the described test cases have been implemented, and every message uses an assertion to automatically validate the success of a test step, the test project with all included test cases is considered implemented. The project can now be included into the Jenkins project, as described throughout chapter 5.2, and uploaded to the repository (see chapter 5.1), and will be executed after the next invocation of a Jenkins build of the respective component. Document name: Page: 27 of 46

29 7. Implementation of a test cases and corresponding assertion In order to verify the setup and interplay of the server testbed one test suite containing five test cases including several assertions has been implemented. The FutureID Authenticate Interface being a central component of the FutureID backend has been selected to test the functionality of the provided server testbed. The test case and the corresponding assertions have been derived from D41.3: Implementation of the Identity Broker in Dispatcher Mode [2] and focuses on error handling and messages. 7.1 Test case: FutureID Authenticate Interface The FutureID Authenticate interface is the first interface of an Authentication Backend (AB) [2]. The following Test Case includes two different scenarios, Authentication with FutureID Client present and Authentication with FutureID Client not present. The first scenario is shown in Figure 18. The red rectangle depicts the tested scenario, in which a User Agent (UA) requests the Broker Core (BC) to perform an authentication. The BC then calls the Authentication Backend (AB), using the Authenticate interface. The AB responds with a message, containing the RequestID and status information, which are both described further below. Figure 18 - Authentication with FutureID Client present Document name: Page: 28 of 46

30 For this scenario, the Authenticate interface receives a message, whose structure is shown in Figure 19. The Authenticate element hereby indicates with two distinct attributes (Profile and RequestID), the usage of synchronous or asynchronous communication, and the ID to correlate results with previous responses. <Authenticate> <dss:optionalinputs> <Session Identifier> <wsp:policy> <RequestedAttributes> n <isoext:issuer> <isoext:redirecturl> <saml:attribute> Figure 19 - Overview on the message format for the authenticate function 7.2 Writing and implementing test cases and corresponding assertions in SoapUI First of all, the requirements had to be derived from specifications. In deliverable D41.3: Implementation of the Identity Broker in Dispatcher Mode [2] the interfaces of the broker service are described in detail. This document consists of the most important information about the test services and broker interfaces, which are to be tested within the FutureID project. Furthermore the so called authentication backends are listed on the FutureID wiki [3]. The request message calls the FutureID broker through its Authenticate web service. With the general perspective, a test runs between the BC and the AB. The BC sends a request message to the AB, in return, AB responds to the BC with a response message. The key point is the information in the response message. In that point, XPath allows us to capture desired information in the response message. XPath reads the information with respect to our XPath Expression. It then checks the information which is captured from response message with the desired information, whether it matches or not. Additionally, the transfer functionality in SoapUI allows us to use information (e.g. RequestID) in the second response message within same test case which was obtained from the first response message. The following chapters document five example test cases that were implemented using SoapUI. The test cases are explained, and then listed step-by-step with the relevant XML code for requests and result testing. Document name: Page: 29 of 46

31 7.2.1 Test Case 1: Authenticate Function - Asynchronous with valid RequestID Step 1: RetrieveRequestID Input: Message without RequestID and profile set to asynchronous mode Expected Output: ResultMajor ok and RequestID as attribute to AuthenticateResponse (MajorResult pending when attributes are retrieved) RequestMessage: <soapenv:envelope xmlns:soapenv=" xmlns:urn="urn:iso:std:iso-iec:24727:tech:schema" xmlns:urn1="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:xd=" xmlns:ws=" xmlns:urn2="urn:oasis:names:tc:saml:2.0:assertion" xmlns:ns24="urn:iso:std:iso-iec:24727:ext:tech:schema" xmlns:ns4="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:ns7="urn:oasis:names:tc:saml:2.0:assertion"> <soapenv:header/> <soapenv:body> <urn:authenticate Profile="urn:iso:std:isoiec:24727:tech:authenticate:profiles:asynchronous"> <ns4:optionalinputs> <ns24:redirecturl> </ns4:optionalinputs> </urn:authenticate> </soapenv:body> </soapenv:envelope> Assertion 1: ResultMajor equals ok (XPath Expression) declare namespace S=" declare namespace ns3="urn:iso:std:iso-iec:24727:tech:schema" declare namespace ns4="urn:oasis:names:tc:dss:1.0:core:schema" /S:Envelope/S:Body/ns3:AuthenticateResponse/ns4:Result/ns4:ResultMajor Expected Result: urn:iso:std:iso-iec:24727:tech:resultmajor:ok Assertion 2: RequestID available (XPath Expression) declare namespace S=" declare namespace ns3="urn:iso:std:iso-iec:24727:tech:schema" declare namespace ns4="urn:oasis:names:tc:dss:1.0:core:schema" Document name: Page: 30 of 46

32 Expected Result: * Step 2: SoapUI Transfers the RequestID to the RetrieveAttributes Message Source: declare namespace soapenv=" declare namespace urn="urn:iso:std:iso-iec:24727:tech:schema" /soapenv:envelope/soapenv:body/urn:authenticateresponse/@requestid Target: declare namespace soapenv=" declare namespace urn="urn:iso:std:iso-iec:24727:tech:schema" /soapenv:envelope/soapenv:body/urn:authenticate/@requestid Step 3: Message for obtaining the Attributes <soapenv:envelope xmlns:soapenv=" xmlns:urn="urn:iso:std:iso-iec:24727:tech:schema" xmlns:urn1="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:xd=" xmlns:ws=" xmlns:urn2="urn:oasis:names:tc:saml:2.0:assertion" xmlns:ns24="urn:iso:std:iso-iec:24727:ext:tech:schema" xmlns:ns4="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:ns7="urn:oasis:names:tc:saml:2.0:assertion"> <soapenv:header/> <soapenv:body> <urn:authenticate RequestID="8F8C2AF03756A11D41E440A864476D1A" Profile="urn:iso:std:isoiec:24727:tech:authenticate:profiles:asynchronous"> <ns4:optionalinputs> <ns24:redirecturl> </ns4:optionalinputs> <urn:requestedattributes> <ns7:attribute Name=" urn:isrequired="true"/> <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" Document name: Page: 31 of 46

33 <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" urn:isrequired="true"/> <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" </urn:requestedattributes> </urn:authenticate> </soapenv:body> </soapenv:envelope> Assertion 1: ResultMajor equals pending (XPath Expression) declare namespace S=" declare namespace ns3="urn:iso:std:iso-iec:24727:tech:schema" declare namespace ns4="urn:oasis:names:tc:dss:1.0:core:schema" /S:Envelope/S:Body/ns3:AuthenticateResponse/ns4:Result/ns4:ResultMajor Document name: Page: 32 of 46

34 Expected Result: urn:iso:std:iso-iec:24727:tech:authenticate:resultmajor:pending Test Case 2: Authenticate Function - Asynchronous with invalid RequestID Step 1: Send request with RequestID Input: Message with an arbitrary (non-existent) RequestID and Profile set to asynchronous mode Expected Output: ResultMajor:error RequestMessage: <soapenv:envelope xmlns:soapenv=" xmlns:urn="urn:iso:std:iso-iec:24727:tech:schema" xmlns:urn1="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:xd=" xmlns:ws=" xmlns:urn2="urn:oasis:names:tc:saml:2.0:assertion" xmlns:ns24="urn:iso:std:iso-iec:24727:ext:tech:schema" xmlns:ns4="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:ns7="urn:oasis:names:tc:saml:2.0:assertion"> <soapenv:header/> <soapenv:body> <urn:authenticate RequestID="789ß ABABAHODFJ" Profile="urn:iso:std:isoiec:24727:tech:authenticate:profiles:asynchronous"> <ns4:optionalinputs> <ns24:redirecturl> </ns4:optionalinputs> </urn:authenticate> </soapenv:body> </soapenv:envelope> Assertion 2: ResultMajor equals error (XPath Expression) declare namespace S=" declare namespace ns3="urn:iso:std:iso-iec:24727:tech:schema" declare namespace ns4="urn:oasis:names:tc:dss:1.0:core:schema" /S:Envelope/S:Body/ns3:AuthenticateResponse/ns4:Result/ns4:ResultMajor Expected Result: urn:iso:std:iso-iec:24727:tech:resultmajor:error Document name: Page: 33 of 46

35 7.2.3 Test Case 3: Authenticate Synchronous with valid SessionIdentifier Step 1: Retrieve SessionIdentifier by opening a session Input: Message without SessionIdentifier, and profile set to synchronous mode Expected Output: ResultMajor:ok and presence of a SessionIdentifier RequestID RequestMessage: <soapenv:envelope xmlns:soapenv=" xmlns:urn="urn:iso:std:iso-iec:24727:tech:schema" xmlns:urn1="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:xd=" xmlns:ws=" xmlns:urn2="urn:oasis:names:tc:saml:2.0:assertion" xmlns:ns24="urn:iso:std:iso-iec:24727:ext:tech:schema" xmlns:ns4="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:ns7="urn:oasis:names:tc:saml:2.0:assertion"> <soapenv:header/> <soapenv:body> <urn:authenticate Profile="urn:iso:std:isoiec:24727:tech:authenticate:profiles:synchronous"> <ns4:optionalinputs> <ns24:redirecturl> </ns4:optionalinputs> </urn:authenticate> </soapenv:body> </soapenv:envelope> Assertion 1: ResultMajor equals ok (XPath Expression) declare namespace S=" declare namespace ns3="urn:iso:std:iso-iec:24727:tech:schema" declare namespace ns4="urn:oasis:names:tc:dss:1.0:core:schema" /S:Envelope/S:Body/ns3:AuthenticateResponse/ns4:Result/ns4:ResultMajor Expected Result: urn:iso:std:iso-iec:24727:tech:resultmajor:ok Assertion 2: Sessionidentifier in Response (XPath Expression) declare namespace S=" declare namespace ns3="urn:iso:std:iso-iec:24727:tech:schema" Document name: Page: 34 of 46

36 declare namespace ns4="urn:oasis:names:tc:dss:1.0:core:schema" /S:Envelope/S:Body/ns3:AuthenticateResponse/ns3:SessionIdentifier Expected Result: * Assertion 3: XPath Match (XPath Expression) declare namespace S=" declare namespace ns3="urn:iso:std:iso-iec:24727:tech:schema" declare namespace ns4="urn:oasis:names:tc:dss:1.0:core:schema" /S:Envelope/S:Body/ns3:AuthenticateResponse/@Profile Expected Result: *urn:iso:std:iso-iec:24727:tech:authenticate Step 2: SoapUI transfers the received SessionIdentifier to the RetrieveAttributes message Source: declare namespace S=" declare namespace ns3="urn:iso:std:iso-iec:24727:tech:schema" declare namespace ns4="urn:oasis:names:tc:dss:1.0:core:schema" /S:Envelope/S:Body/ns3:AuthenticateResponse/ns3:SessionIdentifier Target: declare namespace soapenv=" declare namespace urn="urn:iso:std:iso-iec:24727:tech:schema" /soapenv:envelope/soapenv:body/urn:authenticate/urn:sessionidentifier Step 3: Sending a request message for obtaining the attributes <soapenv:envelope xmlns:soapenv=" xmlns:urn="urn:iso:std:iso-iec:24727:tech:schema" xmlns:urn1="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:xd=" xmlns:ws=" xmlns:urn2="urn:oasis:names:tc:saml:2.0:assertion" Document name: Page: 35 of 46

37 xmlns:ns24="urn:iso:std:iso-iec:24727:ext:tech:schema" xmlns:ns4="urn:oasis:names:tc:dss:1.0:core:schema" xmlns:ns7="urn:oasis:names:tc:saml:2.0:assertion"> <soapenv:header/> <soapenv:body> <urn:authenticate Profile="urn:iso:std:isoiec:24727:tech:authenticate:profiles:synchronous"> <ns4:optionalinputs> <ns24:redirecturl> </ns4:optionalinputs> <urn:sessionidentifier>78ef269e8cd1bab8c488ea4412c942ce</urn:sessionidentifier> <urn:requestedattributes> <ns7:attribute Name=" urn:isrequired="true"/> <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" urn:isrequired="true"/> <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" <ns7:attribute Name=" Document name: Page: 36 of 46