Introduction to Web Services

Transcription

1

2 Introduction to Web Services Peter Fischer Hallin, Center for Biological Sequence Analysis Comparative Microbial Genomics Workshop Bangkok, Thailand June 2nd 2008

3 Background - why worry... Increasing size and number of public sequence databases - faster sequencing methods Increasing number of bioinformatic prediction tools Requirements for data integration, spanning several types of data, different databases and physical locations. Solution: Web Services

4 Topics covered Web Services vs. Web Applications Why WS? Advantages / disadvantages Design of web services: interoperability Implementation case stories Invoking a web service WS and it s future role

5 Web applications HTML based software designed to interact with the user - in most cases this involves human interpretation and navigation. If designed well, they are user friendly Do not require special skills to operate In near every case, web pages do not make sense to computers (wget?)

6 Web Applications... you use them everyday... plus a billion more...

7 Web Services (on the other hand) - are software designed to enable computerto-computer interaction - should aim to enhance interoperability between different systems - exchange objects which are well defined - consist of methods / operations and are well defined - exchanging data using SOAP over HTTP.

8 Interoperability

9 <soap:envelope xmlns:soap=" <soap:body> <getproductdetails xmlns=" <productid>827635</productid> </getproductdetails> </soap:body> </soap:envelope> <soap:envelope xmlns:soap=" <soap:body> <getproductdetailsresponse xmlns=" <getproductdetailsresult> <productname>toptimate 3-Piece Set</productName> <productid>827635</productid> <description>3-piece luggage set. Black Polyester.</description> <price currency="nis">96.50</price> <instock>true</instock> </getproductdetailsresult> </getproductdetailsresponse> </soap:body> </soap:envelope>

10 Basic concepts of the web service: XML (Extensible Markup Language): Format for input and output. Request: An XML message generated by the user/client that are uploaded to a service Response: An XML message generated by the server as an answer to the request. WSDL (Web Services Description Language) file: Most often published by the service provider. Describes all aspects of the service

11 Basic concepts of the web service: Message: A detailed declaration of how input and output looks. XSD (XML Schema Definitions) is the language for defining the types that form the messages. Types: Declarations of input and output objects can be within the WSDL file itself, or linked externally. Endpoint: A URL pointing the client to a location which will read the request and respond. Documentation: Every object element/attribute can be documented

12 How does the WSDL look? Written in XML Looks complicated - most developers fear to write them manually Personally, I believe they should be written by hand in a simple text editor... But let s first look at an XSD file - defining objects used for the EasyGene service, which is an ORF finder

13 The XSD: Declaring types used for messages

14

15

16

17

18 Data types

19 Detailed documentation

20 The WSDL: Importing types, defining messages

21 The WSDL: Linking operations and messages

22 The WSDL: Linking operations to endpoints

23 The WSDL: Service documentation

24 The WSDL: Endpoint

25 Web Services design considerations Common data types Granularity Typing

26 Granularity, Granularity, and Granularity Our choice of technology sets standards for typing any element in input/output - and these standards should be used! To a certain extent, Web Services is all about plumbing - connecting objects (pipes) from different operations to build a workflow and finally to generate a result for you This plumbing gets increasingly difficult, the poorer the granularity

27 Tropomyosin isoforms

28 Tropomyosin isoforms

29 Tropomyosin isoforms

30 Tropomyosin isoforms

31 1 2 3 Granularity

32 Typing

33 array id: string Typing sequence: string - restrictions? /^[ACDEFGHIKLMNPQRSTVWY]+$/

34

35 Requirements You will not need to examine the WSDL and XSD files during this workshop. You should just know of their existence and idea behind the WSDL

36 3 methods to invoke web services: Workflow editors: Graphical. Less user un-friendly (supposedly) Text/XML editors: Easy-to-use. Less features. For exploring services Programmatic access: For writing clients and automating tasks.

37 Graphical workflows E.g. Taverna:

38 Raw text/xml SoapUI:

39 Perl/Python/C SOAP modules exist for most programming languages: Perl, Python, C, Java...

40 Perl/Python/C SOAP modules exist for most programming languages: Perl, Python, C, Java...

41 Demonstration using SoapUI SoapUI is... - Java based stand-alone software for inspecting, invoking, and testing Web Services. - Downloadable from free of charge. - pro edition available for purchase having extended features.

42 Ideal for development/testing of Web Services: Provide the raw XML request/response to/from your server. It allows you to view HTTP headers, attachments etc. Construct template/default request messages based on the WSDL/XSD SoapUIs strength is inspection and manually invoking of operations, however the Pro edition supports workflows/ test cases! Handling multiple WSDLs with a project, and multiple projects within a session Why SoapUI

43 Example: Inspecting two services Genome Atlas ( ws/genomeatlas/) Various database tools accessing prokaryotic genomes sequences RNAmmer ( RNAmmer) Predicts ribosomal RNA genes in full genome sequences.

44

45 Creating a new project

46 Labeling the project and adding WSDL

47 Default request are made automatically

48 We provide a genbank accession to getseq

49 RNAmmer online documentation

50 Adding a new WSDL to the project

51 Adding a new WSDL to the project

52 Copy the genome sequence from getseq

53 Paste genome sequence in RNAmmer runservice

54 Adding a new WSDL to the project

55 Submit and acquire jobid

56 Poll queue until job finished

57 Poll queue until job finished

58 Fetch the result, suing fetchresult

59 Asynchronous operations

60 EasyGene - the programmatic way Similar example, this time programmed in Perl Using Genome Atlas getseq to obtain genome sequence, and submit this to the ORF finder EasyGene. Poll for the job status and obtain result.

61 First, obtaining a Genome Sequence

62 EasyGene client : load fasta from STDIN

63 EasyGene client : function to predict ORFs

64

65

66

67

68 Case story: meta genomic application of BLAST atlases A service which allows visualization of homology between a reference genome and any number of genomes, metagenomic samples, or sequence databases.

69 Case-story: the BLASTatlas WS

70 Case-story: the BLASTatlas WS

71 Example: Seven ocean samples from various depths (surface to 4km): 63,837,557 nucleotides, in 65,674 sequences. 24,978 proteins from 12 fully sequenced Prochlorococcus marinus genomes

72 Surface M km 0M 2M M P. marinus str. MIT ,682,675 bp 1M 1.5 M green P. marinus genomes blue=ocean samples

73

74 Conclusionpros and cons Web services are (almost guaranteed) to be more cumbersome to access than its web application counterpart. Web services allow automation and integration into your programming language. You need only a single (WSDL) file to allow your client to connect to the service

75 Pros and cons Input (request) can be validated - remember the XSD! Should failures occur

76 Looking into the crystal ball... Will WS replace the bioinformatician or require more of them? Will WS ease the access to tools and databases? Will WS save time? Will WS allow more complex analysis that were impossible before? Will WS take over - and what will they take over?

77 Acknowledgements David W. Ussery Craig Benham Karin Lagesen Jan Christian Bryne Francisco Roque Kristoffer Rapacki Hans Henrik Stærfeldt