DEVELOPER GUIDE PIPELINE PILOT INTEGRATION COLLECTION 2016
Copyright Notice 2015 Dassault Systèmes. All rights reserved. 3DEXPERIENCE, the Compass icon and the 3DS logo, CATIA, SOLIDWORKS, ENOVIA, DELMIA, SIMULIA, GEOVIA, EXALEAD, 3D VIA, BIOVIA and NETVIBES are commercial trademarks or registered trademarks of Dassault Systèmes or its subsidiaries in the U.S. and/or other countries. All other trademarks are owned by their respective owners. Use of any Dassault Systèmes or its subsidiaries trademarks is subject to their express written approval. Acknowledgments and References To print photographs or files of computational results (figures and/or data) obtained using BIOVIA software, acknowledge the source in an appropriate format. For example: "Computational results obtained using software programs from Dassault Systèmes BIOVIA. The ab initio calculations were performed with the DMol 3 program, and graphical displays generated with Pipeline Pilot." BIOVIA may grant permission to republish or reprint its copyrighted materials. Requests should be submitted to BIOVIA Support, either through electronic mail to biovia.support@3ds.com, or in writing to: BIOVIA Support 5005 Wateridge Vista Drive, San Diego, CA 92121 USA
Contents Chapter 1: SOAP Web Services Guide 1 SOAP Web Services on the Pipeline Pilot 1 Chapter 2: Protocol-based SOAP services 2 Blocking and Polling Modes 2 SOAP Service Options for Blocking Web Services 2 Design Model: Pre-defined WSDL or Generated WSDL 2 Pre-defined WSDL (Document interface) 3 Generated WSDL (Parameterized interface) 3 Endpoints 4 SOAP Service Security Model 4 Single Protocol or Folder-based Service 4 SOAP URLs for Blocking Protocols 5 SOAP Service Options for Polling Web Services 5 Polling Web Service Endpoint 6 Polling Web Service Methods 6 Chapter 3: URL Routing for Blocking SOAP Web Services 7 Chapter 4: The SOAP Web Service Inspector 8 Appendices 11 Appendix A: SOAP Services reference 11
Chapter 1: SOAP Web Services Guide SOAP Web Services on the Pipeline Pilot This document describes how protocols are exposed as web services accessed using the SOAP protocol. SOAP Web Services Guide Page 1
Chapter 2: Protocol-based SOAP services When a user saves a protocol into the Pipeline Pilot protocol repository, a number of web services become available which can be used to execute the protocol. This document describes the SOAP-based web services that can be used to execute a given protocol. There are multiple service endpoints for one protocol to reflect the fact that protocols can be executed in several ways - blocking or polling, different user authentication models and different WSDL-design scenarios. These options are described in more detail in the sections below. When working with Pipeline Pilot Client, you can find information on some of the SOAP services associated with a specific protocol or folder of protocols. Right click on the protocol or folder of protocols and select the "Inspect Web Service" option. The SOAP web service viewer tool opens up in your default web browser. Here you can explore some SOAP web services generated for the protocol or folder of protocols. Blocking and Polling Modes The published protocol WSDLs support the execution of protocols in a blocking (synchronous) mode and in a "launch and poll" (asynchronous) mode. The advantage of the blocking mode is its simplicity. After logging in, the single Execute method takes care of passing in required parameters and returning a result structure, which may contain result data or error information. This mode of operation is best suited to short jobs that do not time out before completion, and that clients do not disconnect from or cancel. Asynchronous operations require various SOAP calls to put into effect. After logging in, the client creates and starts the job, and then polls for job status until it is complete. At this point, you can retrieve job results or error data. In this case, job results may consist of any number of file references and other more atomic data values. In this mode, a client can poll at any frequency or not at all. If needed, a client program can shut down and test for status later, perhaps from another client computer. Additionally a client may terminate a job during its run. Since the job is running without the need for a connection, there are no timeout issues; each SOAP call returns without any delay, whatever the scale of the job itself. SOAP Service Options for Blocking Web Services This section confines itself to the execution of protocols in a blocking mode. In this mode, the client sends the SOAP request and waits for the response from the job to indicate that the protocol job has completed; the response includes the protocol results, and there is no further interaction with the protocol job. There are a number of options for defining blocking SOAP services based on Pipeline Pilot protocols. The set of options employed to invoke the protocol dictate the details of the endpoint URL for the service. So, for this reason, one protocol can be invoked via multiple different endpoints. Design Model: Pre-defined WSDL or Generated WSDL A protocol authored to implement a SOAP web service will generally be written to conform to one of two design models. Protocol-based SOAP services Page 2
Pre-defined WSDL (Document interface) In the case of a Pre-defined WSDL, you already have a SOAP service defined. Perhaps you wish to migrate an existing service to an Pipeline Pilot protocol implementation. In this mode, the SOAP request body content is passed as an XML document to the protocol where it is parsed using standard XML components. The SOAP response body content is constructed by the protocol using XML components. The approach allows you to define arbitrarily complex data structures to pass in and out of the protocol. When this style of SOAP service is invoked (identified by the URL endpoint, discussed in a section below), the Pipeline Pilot web service handler extracts the SOAP request body and passes it to the protocol. Therefore a protocol supporting the pre-defined WSDL model is written using a document-centric interface where it expects a single input parameter containing the XML from the SOAP request body and outputs a single valid XML document output result that will be inserted into the SOAP response body. Generated WSDL (Parameterized interface) In the Generated WSDL approach, the WSDL for a protocol is constructed as a web service by the Pipeline Pilot based on the parameters and results defined for that protocol. So the protocol author uses standard techniques in Pipeline Pilot for defining the protocol interface, with multiple parameters and multiple result data items, if necessary. A WSDL can be generated for any protocol, including those not even originally authored with a SOAP service in mind. The SOAP client makes a request based on the generated protocol WSDL, using the URL path for this type of SOAP service (URL endpoints are described in a section below). The web handler parses the request and invokes the target protocol with the parameter values extracted from the request. The protocol results are converted back into a SOAP response for the client. Page 3 Pipeline Pilot Developer Guide
Endpoints Each of these approaches employs a different SOAP endpoint that indicates the desired behavior to the SOAP handling framework in the Pipeline Pilot web service layer. Details of endpoint construction are provided in a later section. Whether the SOAP service is based on a predefined WSDL or based on a WSDL that reflects the protocol parameterization, the service definition can then be utilized by a standard SOAP client tool to parameterize the request and execute it. soapui from smartbear.com is a good example of tool to test out SOAP services. SOAP Service Security Model There are 3 different security models for executing a protocol as a SOAP service: WS-Security SOAP header Standard Pipeline Pilot authentication based on the server's authentication mode. Anonymous access, if configured on the server. The security mode is reflected in the SOAP endpoint used to execute the protocol. Based on the details of the endpoint, the Pipeline Pilot web service layer passes the request to the appropriate authentication subsystem. Details of endpoint construction are provided in a later section, where you will see how the security model plays a role. Single Protocol or Folder-based Service A SOAP service can be invoked using an endpoint that references a single protocol. In this case, the endpoint itself indicates which protocol should be run. Alternatively, a more multi-functional service can be defined at the folder level. In this case, each protocol within that folder is used to implement a method on the service. The name of which protocol is to be run is extracted from the outer element of the SOAP body of the request, or from the SOAPAction header. The endpoint of a SOAP-based service includes the fact of whether the service path is a reference to a protocol or to a folder of protocols. Protocol-based SOAP services Page 4
SOAP URLs for Blocking Protocols The URL path for a protocol SOAP service is constructed around each of the 3 options above. This is followed by the path of the protocol itself. See the figure below which indicates how 12 possible paths can be constructed, followed by the folder or protocol path itself. Examples (using the color coding from the URL construction figure): The service to execute a protocol as a blocking job using standard Pipeline Pilot authentication: https://server:port/auth/p/wservice/archivedata/search The service representing a folder of protocols executed as blocking jobs using standard Pipeline Pilot authentication: https://server:port/auth/p/fservice/labnotes/queries A pre-defined WSDL example a service implemented by a protocol that processes the SOAP body: https://server:port/auth/fservice/calculators Another pre-defined WSDL example; and in this case the client should pass WS-Security SOAP headers: https://server:port/wsse/fservice/calculators Note: The protocol path can include the 'Protocols/' prefix or omit it. SOAP Service Options for Polling Web Services This section describes working with protocols as SOAP web services based on a polling mode of interaction (aka asynchronous web service). In this model, the service includes several methods to support the launching of the job, polling for status and the retrieval of the job results. Due to the overhead inherent in the polling model, this approach is best-suited to the execution of protocols that is Page 5 Pipeline Pilot Developer Guide
expected to take more than a few seconds. Since the client thread is not blocked, the client software can provide progress updates and messages to the user. Polling Web Service Endpoint The options for an asynchronous protocol web service are more limited than the blocking mode. Due to the very particular nature of a polling service: A polling client must use the WSDL generated from the protocol (i.e. there is no "pre-defined" option) This form of WSDL is only applicable for addressing a single protocol, and not a folder of protocols. WS-Security is not supported Anonymous authentication is not supported. The form of the SOAP endpoint is: https://server:port/scitegic/service/async/protocol-path where the protocol-path identifies the protocol location within the Protocols area of the protocol database. Example: https://localhost:9043/scitegic/service/async/web%20services/examples/generic/stock%20statistics Polling Web Service Methods Each protocol polling web service has the methods listed below. The details of the parameters and data types depend on the specific parameterization and results defined for each protocol. To reference such details, follow the "Inspect Web Service" option for protocols in Pipeline Pilot Client, described later in this document. ListMethods Login Execute CreateJob StartJob StopJob GetJobStatus GetJobResults GetJobErrorStack DeleteJob Lists all operations available through the Web service interface. Creates a server session with the user name and password and returns a SessionID, to be used with the other methods of this Web service. Launches a protocol job on the Pipeline Pilot server and waits for the job to complete or timeout. This is a convenience method, to use as an alternative to the more fine-grained methods below. Creates a job directory and returns a new JobID. Launches a protocol job and returns the initial job state (Running, etc). Terminate a running job. Query the execution state of a job. Query the results (data and file references) for a completed job. Retrieves error messages for a job that completes with an error. Remove the job and its results from the server. Protocol-based SOAP services Page 6
Chapter 3: URL Routing for Blocking SOAP Web Services The previous section includes information on the construction of URL endpoints for SOAP services implemented by a specific protocol or folder of protocols. This section describes how to hide the details of the protocol by creating a mapping between a URL of your choice and the protocol to be executed (blocking mode only). The SOAP endpoints described in the previous section are created when the protocol is saved in the protocol database. To create a URL mapping to the protocol, you must choose to add a URL routing definition to a package configuration. This uses the same mechanism described for URL routing for RESTful services described in the RESTful Web Services Guide document, which provides more details. The URL mapping is defined by a URL directive in a package.conf file. The directive should include the SOAP flag to indicate that this is a SOAP endpoint mapping. The following lists the relevant flags on the URL directive. Flag SOAP ANON/WSSE/AUTH (default) PARAMETERS/XML (default) FOLDER Description Always use this for SOAP endpoint URL mapping. Defines the security model to be employed, and represent the models described in the SOAP Service Security Model section. Use the PARAMETERS flag when the protocol is designed to be invoked using the "Generated WSDL" model where the protocol is fully parameterized. Use the XML flag when the protocol is designed to accept and output an XML document under the "Pre-defined" WSDL model. Include this flag if the referenced protocol path (see below) is a folder of protocols, and not an individual protocol. The URL directive requires a request and a protocol setting to define the mapping from the chosen endpoint URL and the underlying protocol. Examples: Map a protocol with a document interface to a specialized endpoint URL: <url SOAP > request /demografix/cities protocol Web Services/prod/demographics/getCitiesByCountry </url> Map the URL path /demografix/anon to a SOAP service supporting anonymous access, implemented by a folder of parameter-centric protocols: <url SOAP FOLDER PARAMETERS ANON > request /demografix/anon protocol Web Services/prod/demographics/services </url> URL Routing for Blocking SOAP Web Services Page 7
Chapter 4: The SOAP Web Service Inspector The Pipeline Pilot includes a web application for studying the details of the SOAP web services that are generated for a protocol or folder of protocols. You can see the raw WSDL XML, an easier-to-read reference and you can test out the web service form a form interface and see diagnostic information that may help you to build your own SOAP client program. To access the details of a generated web service for a protocol or folder of protocols: From the Protocols tab in Pipeline Pilot Client, right-click the protocol or folder and select Inspect Web Service. You may be asked to log into the server if you do not have a current session on that Pipeline Pilot server. A browser window opens and displays the WSDL URL at the top of the page. This is the WSDL reference that you can use for programming against the protocol web service. Client IDE programs require this URL to generate appropriate proxy code for client development. The SOAP Web Service Inspector Page 8
Generated protocol WSDL with tabs for viewing XML, details in report format, and test data. Tabs are also available for viewing the raw WSDL text, a more easy-to-read version of the method list, and a test page. Page 9 Pipeline Pilot Developer Guide
Notes: Protocols used for SOAP web services are not allowed to have parameters of type ExpressionType defined as Web Service parameters. Using ExpressionType parameters for SOAP web services results in improper WSDL definition and the SOAP calls will not function properly returning HTTP response codes of 500. Any protocol can be invoked as a web service. However, treating a protocol as a web service is most useful if the protocol has input parameters and returned results literal values or file references. An example of an appropriate web service protocol is one that is designed to run from a client-side application, such as Web Port. Spaces in protocol parameter names are converted into hyphens in the WSDL. Protocol names in folder-based services are converted to double underscores. These conversions are necessary to create a valid WSDL file defined by the rules of SOAP and XML. This is evident in the WSDL descriptions. Running the protocol from the Test page allows you to interact with the web service interface and examine the sequence of method calls made to Pipeline Pilot when calling the service. For more information about the WSDL standard, see the Web Services Description on the W3C Web site. The SOAP Web Service Inspector Page 10
Appendices Appendix A: SOAP Services reference This list includes the SOAP interfaces described in this document, plus some of the interfaces supported by previous versions of the server that you may find in older client programs and scripts. Pipeline Pilot Document-centric SOAP services for blocking protocol execution /auth/wservice/[protocols/]<protocol-path> /auth/fservice/[protocols/]<protocol-path> /auth/anon/fservice/[protocols/]<protocol-path> /auth/anon/wservice/[protocols/]<protocol-path> /wsse/wservice/[protocols/]<protocol-path> /wsse/fservice/[protocols/]<protocol-path> Pipeline Pilot Parameter-centric SOAP services for blocking protocol execution /auth/p/wservice/[protocols/]<protocol-path> /auth/p/fservice/[protocols/]<protocol-path> /auth/anon/p/fservice/[protocols/]<protocol-path> /auth/anon/p/wservice/[protocols/]<protocol-path> /wsse/p/wservice/[protocols/]<protocol-path> /wsse/p/fservice/[protocols/]<protocol-path> Legacy SOAP interfaces /scitegic/service/sync/<protocol-path> /scitegic/service/async/<protocol-path> document) /scitegic/service/m/sync/<protocol-path> /scitegic/service/m/async/<protocol-path> WS-Security support /scitegic/xservice/sync/<protocol-path> /scitegic/xservice/m/sync/<protocol-path> Older RPC SOAP interface supporting blocking and polling methods /scitegic/protocol/<protocol-path> (Folder-based) Pipeline Pilot 3.0 SOAP interface with methods for protocol database queries and protocol execution /scitegic/soap (Blocking) (Polling see earl (Folder-based) (Folder-based) Appendices Page 11