Web Services Architecture and Programming

Transcription

1 Web Services Architecture and Programming Working remotely and distributable Association For Information Security Nikolay Nedyalkov

2

3 What are Web Services? Depends on whom you ask A revolutionary new way of building distributed applications The natural evolution of distributed programming APIs Sockets RPC Distributed Objects Web Services To simplify application integration and interoperability Main ideas Applications structured as lightweight components, which expose services Example: A Weather component, which offers a GetTemperature service Input parameter: Zip code Output response: An integer that represents the temperature Services discovered, described, and interacted with using standard protocols UDDI, WSDL, SOAP, all of which make heavy use of XML Goal: Provide a simple application-to-application interface just like the web has provided a simple human-to-application interface Specifications such as HTML and HTTP, servers and browsers

4 Why Study Web Services? Most new applications have a distributed component Accessible across wide-area networks Themselves need to access other services across networks Need to understand what is involved in writing distributed programs Both from the point of view of what is in use/proposed earlier Sockets, Remote Procedure Calls, Distributed Objects Standard protocols, data formats such as HTTP and XML As well as how these approaches have evolved Web services architecture Protocols such as SOAP, WSDL, UDDI, Easier to construct, more interoperable Are we giving up anything.

5 Agenda Precursors/Alternatives to Web Services Networking fundamentals Sockets RPC SUNRPC XMLRPC Distributed Objects RMI.NET Remoting Web Services Architecture Overview Web Services Architecture Details SOAP, WSDL, UDDI, State Management, Intermediaries, GXA

6 What This Presentation is About Understanding the general issues that must be addressed while constructing distributed applications from component pieces Discovery: How do components learn about each other? [UDDI] Description: How does a component learn about another s interface? [WSDL] What services does it provide? Interaction: How are these services invoked? [SOAP] How are messages transferred from one component to another? [HTTP, TCP] How do we encode service parameters, return values in these messages? [XML] How do the components at either end know which encoding to use? Understanding how different approaches deal with these issues Sockets, Remote Procedure Calls, Distributed Objects, Web Services Getting some practical experience in using these approaches

7 Microsoft s.net Framework

8 Communication in Distributed Applications

9 Network Programming Builds on top of networking protocols (primarily: UDP, TCP, HTTP) Lowest-level API just provides user-level abstractions for TCP and UDP Sockets: Application-level end-point of communication Operations often described by drawing analogy of a telephone Call (Connection) setup Conversation (Sending and receiving data packets) Hangup (Disconnection)

10 Sockets API What does the API include? A data structure, called a socket, that serves as an application-level endpoint for networking operations A set of functions for operating on this data structure Operation Create a socket Specify the kind of communication operation (e.g., TCP, UDP) Setup the socket In case of a protocol like TCP, establish the underlying connection Which requires OS involvement, hence involves a system call Use the socket Provides a file-like interface: read and write bytes against the socket Close the socket Release any OS resources, free up memory used by the structure

11 Programming With The Sockets API Byte streams, no boundaries preserved Send s and Recv s can line up arbitrarily Therefore, need a convention about data format Agreeing on this convention is one of the hardest things For our StringServer app: <length> <sequence of bytes> For HTTP packets: server parses client requests HTTP standard defines the format of these requests Your networking programs need to work in heterogeneous environments Byte-order (Endian-ness) matters: network byte order is big-endian HostToNetworkOrder, NetworkToHostOrder functions Errors can arise because of a number of reasons Connect request to a socket that is not being listened to, early close, disconnected hosts,

12 Remote Procedure Calls (RPC) Writing distributed applications using the Sockets API is complicated Program needs to explicitly send/receive messages Violates programming transparency Local application components Remote application components RPC: Remote Procedure Call Original goal: Provide complete programming transparency Interaction with procedures on remote hosts as if they were local However, needed to deal with several issues Different memory spaces, Parameter passing, Binding, Failures Now: A higher-level abstraction for distributed programming Provides near-transparency

13 RPC History Original idea described in a 1983 paper by A. Birell and B. J. Nelson Several implementations Xerox SunRPC: widely used (NFS) DEC DCE (OpenGroup): basis for Microsoft s implementation of RPC in COM XML-RPC same underlying ideas, but leverages web standards

14 Overall Structure of RPC Client process blocks for duration of the call Just like in a local procedure call Asynchronous RPC: early reply from server RPC package is at the session layer Can work with different transports Shared Memory, UDP or TCP has to be specified at setup time Message passing completely hidden from programmer

15 Understanding RPC: Local Procedure Calls Steps in a local procedure call Caller pushes parameters, return address on a stack Control transferred to procedure Procedure returns values in registers, removes return address and passes control back Call-by-value Call-by-reference Caller cleans up stack frame How can we emulate these in a network setting? Parameter passing across address spaces Pointers are not valid across machines Binding Static: compile/link time (as in the local procedure case) Dynamic: using an intermediary service, requires registration and lookup Dealing with failures Client and server crashes

16 Understanding RPC: Client and Server Stubs We want to make RPC s look like local procedure calls Client stubs allow callers to make remote calls that look like local calls Server stubs allow callees to respond to remote calls as if they were from a local caller

17 Issue 1: RPC Parameter Passing Client and server stubs need to ensure that parameters are correctly passed between address spaces Value parameters Big-endian versus little-endian issues Different sizes of types on different machines E.g., int is 32-bits on x86 platforms and 64-bits on Itanium Reference parameters (pointers) Pointers are invalid, so entire data structure must be sent What happens if client process updates the structure being pointed to? What should you do with an IN-OUT parameter? Thus, a need for standard data types and structures ( wire format )

18 Interface Definition Language (IDL) One way for client and server stubs to agree upon parameter passing is to employ a higher-level definition of the procedure s interface Definition in a separate language: Interface Definition Language (IDL) Restricted set of data types Encoding of these data types into messages is standardized call-by-value is straightforward call-by-reference implemented using copy of structure/restore Example struct DateTime { long date; long time; }; DateTime getdatetime( void );

19 Translating IDL to Wire Format Two options Implicit typing Both the sender and receiver know in advance the type and ordering of data (interface fully defines encoding) E.g., XDR (external Data Representation), NDR (Network Data Repr.) Specifies what byte order is used, what the basic types are, how they are transferred on the wire E.g., string type is transferred as an int (length) followed by the ASCII bytes Explicit typing Encoding includes two things a specification of the type and its encoding, and the value in that encoding E.g., ASN.1 (Abstract Syntax Notation 1), BER (Basic Encoding Rules)

20 Issue 2: RPC Binding Static RPC server must be running at a well-known port number Interaction between clients and servers as in the sockets API Dynamic Use an intermediate program called a nameserver Nameserver must be running at a well-known port Permits binding of server program to port number to be deferred Server: RPC server registers with nameserver Nameserver allocates a port, and associates it with the server Server listens to a socket bound to this port Client: RPC client looks up the server by contacting the nameserver Nameserver returns port where server is listening Client sends request to specified port

21 Dynamic Binding Illustrated

22 Issue 3: Dealing with Failures (State Management) Client cannot locate the server Lost request Server crashes Problem: can crash after processing of request, or before Solutions: at least once - retry until a reply is received requires idempotence (server must generate same reply) at most once - return immediately, client rebinds to new server ID Client crashes and restarts Problem: computation finished, but client crashed before return (orphan) Solutions: RPC at the client gives a new incarnation ID to the client Client has to rebind to the service Server uses client id to distinguish this instance from the previous one

23 SunRPC Most common implementation of RPC and built into most UNIX OSes Used for Network File System (NFS) XDR is used for data description and encoding More about this in the next slides A compiler, rpcgen, translates SunRPC IDL to C, automatically generating Client and server stubs Client and server sample code Header files containing XDR data structure declarations A daemon program, portmapper, that provides nameserver functionality Port # 111

24 Example: A SunRPC Program

25 Example sources Generated test files date_proc.c 1-src.jpg date.x 2-src.jpg date.h 3-src.jpg rdate.c 4-src.jpg date_clnt.c 5-src.jpg date_svc.c 6,7,8,9-src.jpg

26 external Data Representation (XDR) A standard for the description and encoding of data Corresponds to the Presentation layer of the ISO protocol stack Representation of all items requires a multiple of 4 bytes of data Padding of 0-3 bytes to ensure this condition XDR data types integer, unsigned integer, enumeration, boolean, 64-bit signed an unsigned integers, single- and double-precision floating point, fixed-length opaque data Encoding contains only data, no additional information Big-endian byte order Variable-length opaque data, string 4-byte length, followed by the bytes making up the data

27 XDR (cont d)

28 Example of XDR Encoding

29 DCE RPC Distributed Computing Environment RPC Open Group standard (also standardized CORBA) DCE: RPC + security, namespace, and network time services Most implementations provide rpcgen-like stub compiler Underlying model for Microsoft s COM implementation Data description and encoding: Network Data Representation (NDR) Key difference from XDR is the receiver-makes-right model Sender encodes data in format most suited to own architecture Supplies information about encoding in an architecture tag Receiver uses information about encoding to interpret stream Benefit: No translation on homogeneous architectures (LANs)

30 (Review) Remote Procedure Calls A procedure-call like request-reply abstraction built on top of lower-level networking protocols

31 XML-RPC A procedure-call like request-reply abstraction built on top of lower-(higher-)level networking protocols

32 XML XML is a standard for describing structured documents Uses tags to define structure: <tag> </tag> demarcates an element Tags have no predefined semantics except when document refers to a specific namespace Elements can have attributes, which are encoded as name-value pairs A well-formed XML document corresponds to an element tree

33 XML-RPC Wire Format Scalar values Represented by a <value><type> </type></value> block

34 XML-RPC Wire Format (cont d)

35 XML-RPC Request HTTP POST message URI interpreted in an implementation-specific fashion Method name passed to the server program POST /cgi-bin/xmlrpc/sumanddiff.pl HTTP/1.1 Content-Type: text/xml User-Agent: XML-RPC.PERL Content-Length: 278 Expect: 100-continue Connection: Keep-Alive Host: localhost:8080 <?xml version="1.0"?> <methodcall> <methodname>sumanddifference</methodname> <params> <param><value><i4>40</i4></value></param> <param><value><i4>10</i4></value></param> </params> </methodcall>

36 XML-RPC Response HTTP Response Lower-level error returned as an HTTP error code Application-level errors returned as a <fault> element (next slide) HTTP/ OK Date: Mon, 22 Sep :52:34 GMT Server: Microsoft-IIS/6.0 Content-Type: text/xml Content-Length: 467 <?xml version="1.0"?> <methodresponse> <params> <param><value> <struct> <member><name>sum</name><value><i4>50</i4></value></member> <member><name>diff</name><value><i4>30</i4></value></member> </struct> </value> </param> </params> </methodresponse>

37 XML-RPC Fault Handling Another kind of a MethodResponse <?xml version="1.0"?> <methodresponse> <fault> <value><struct> <member> <name>faultcode</name> <value><i4>500</i4></value> </member> <member> <name>faultstring</name> <value><string>arg `a out of range</string></value> </member> </struct></value> </fault> </methodresponse>

38 XML-RPC: Discussion Very simple specification Large number of implementations for every conceivable language No surprises during integration XML wire-format Human readable Somewhat verbose Binary encoding possible, but additional layer of specification HTTP transport Widespread use Firewalls already permit HTTP traffic, so no reconfiguration required Main drawback Leaves a lot unspecified Mapping of URL to server handler State management at the server,

39 XML-RPC Example

40 Distributed Objects Rationale: Object-orientation applied to distributed programming Hiding of differences between local and remote interactions By defining notion of remote/distributed objects Call site looks the same in local and remote case All necessary information encapsulated in the object reference XML-RPC proxies already provide this benefit, unlike SunRPC handles Better support for state management at client and server Object reference identifies necessary state Transmission of object references avoids (expensive) state transfer Closer integration of type system with distributed architecture Compile-time checking of message format, other errors Run-time inspection of object reference to determine appropriate format Example systems: CORBA, DCOM, Java RMI,.NET Remoting

41 Distributed Objects Definitions Object Encapsulates data (state) and operations on that data (methods) External access to object state only via its public methods (interface) An object may implement multiple interfaces An interface can be implemented by multiple objects Distributed object Interface exists separate from the implementation (on separate hosts) Implementation can involve one or more objects When implementation contained in one object: remote object Reasons for splitting implementation across multiple objects Modularity, convenience, security Load-balancing

42 Programming with Distributed Objects Overview

43 CORBA Common Object Request Broker Architecture A language-neutral distributed object architecture Standardized by the Object Management Group (OMG) Same folks who standardized DCE RPC Microsoft s answer to CORBA: DCOM Built on top of Microsoft RPC and COM (Common Object Model) Efficient support for interactions between objects located on the same machine Features Parameter passing: Uses own wire format, called CDR Object implementation: Associated with IDL interface at run time Object binding: Explicit, via separate function calls Object references: Refer to a nameserver, support dynamic invocation Permit queries about implemented interfaces Object persistence: Persistent, Activatable

44 CORBA Architecture

45 CORBA Object Services A standard set of services (also implemented as CORBA objects) that provide commonly-required low-level and basic functionality Collection Service Concurrency Service Event Service Externalization Service Licensing Service Life Cycle Service Naming Service Persistence Service Properties Service Query Service Relationship Service Security Service Time Service Trader Service Transaction Service

46 Java Remote Method Invocation (RMI) Java language-level support for remote objects Parameter passing: Uses own wire format Parameters to RMI methods extend a predefined interface (Serializable) serializing = marshalling Possible to define custom serialization routines Object implementation: Compile-time definition RMI interfaces extend a predefined interface (java.rmi.remote) Implementation class implements RMI interface Typically by extending a predefined class java.rmi.activatable, java.rmi.unicastremoteobject) Client and server stub code automatically generated by invoking rmic Additional exceptions defined for remote interactions

47 Java RMI (cont d) Object binding: Both explicit and implicit Explicit: Using static methods in the java.rmi.naming class Server binds/rebinds name (string) with a nameserver (rmiregistry) Client looks up name with the nameserver Implicit: Whenever a client receives a remote object reference Can invoke methods on it as if it were a local interface What happens if the client does not have the stub classes for the reference? RMI implementation includes a dynamic class-loading feature A reference to the bytecode is sent along with the reference Cannot really do this in a language-neutral fashion Object references (RemoteRef) Stores information about the server name, unique object ID, codebase Distributed GC using reference counts Object persistence: Both transient and persistent objects supported

48 .NET Remoting Provides (almost all of) the functionality of Java RMI Except for activation of a persistent object with the following additional benefits Remoting framework works at the level of the CLR Hence, able to interoperate across all.net languages Extensible wire-formats and transports Comes with Binary Encoding/TCP and SOAP/HTTP Application developers can build their own Multiple activation modes Activation here refers to creation of a new object instance (different from passivating/activating a persistent object) Server Activation Client Activation Lifetime management (GC) of remote objects using leases Involve less traffic as compared to reference counting-based schemes

49 .NET Remoting: Architecture Functionality Server makes available a type at a well-known end-point An operation against this type results in an instance being created (unless one already exists) Client makes a request for the type Obtains a proxy that provides the same interface as the type Method invocations against the proxy are forwarded to the server object Can forward the proxy on to other objects Details What kind of types can be made available using.net remoting? What does an end-point look like? How does a client make a request against an end-point? How are parameters/return values passed in method invocations? What information is contained in a proxy? When are new object instances created at the server? When do these die?

50 Web Services Web Services architecture provides XML-based, language-neutral standards for Discovery [ UDDI, WS-Inspection ] Approximate location-independent nature of object references in distributed object systems by relying on intermediate brokers, who store/categorize/provide information about services Description [ WSDL ] Approximate run-type type inspection by encoding the service types/interface into an XML document that can be interpreted by clients Interaction [SOAP] RPC-like procedure calls + asynchronous invocations Implementation uses standard, interoperable protocols (HTTP) Goes back to stateless nature of RPC systems Simpler to support, particularly when loosely-coupled services come from multiple owners

51 Global XML Web Services Architecture (GXA) An attempt by Microsoft, IBM, BEA, others to define a set of higher-level specifications on top of the core web services specifications Core specifications define client-service and client-broker-service interactions SOAP, WSDL, UDDI GXA specifications build on above to define how groups of web services can interact with each other Need these specifications to allow construction of more complex web service applications E.g., an online book store that requires to interact with a credit card web service to verify the user s credit card number Database updates to the book service

52 GXA Specifications (Still Evolving) WS-Inspection A simpler UDDI-like discovery protocol Caters to scenarios where source can directly announce availability WS-Routing, WS-Referral WS-Security Specifies how security credentials are passed in SOAP messages, how SOAP actors should act on them BPEL4WS Encoding of business process activities First invoke this service, then use its results to invoke other services, WS-Transaction(old), WS-Coordination (September 2003) Atomic actions involving multiple services

53 WS-Routing and WS-Referral WSDL specifies end-to-end connection (from client to service URL) Need for more general structures Peer-to-peer and store-and-forward networking Should be possible to send messages to distributed processing nodes (despite being named by the same URL) WS-Routing Enables specification of a complete message path for the message (including its return path) WS-Referral Permits routing between SOAP nodes on a message path to be dynamically configured Allows delegation of part or all of processing responsibility to other nodes Together, permit use of intermediaries in web services applications Caches, load-balancing agents, transcoders,

54 Introducing SOAP, WSDL, and UDDI SOAP: Simple Object Access Protocol XML-RPC-like request/response protocol + Support for asynchronous invocations Encoding of additional information in the message Security tokens for authentication/encryption, Message route information, WSDL: Web Services Description Language RPC, Distributed Objects-like common structs/interface + Support for asynchronous invocations Possibility of language-neutral (and automatic) interpretation Web-services tools use WSDL description of a service to automatically generate a SOAP-capable proxy UDDI: Universal Description, Discovery, and Integration Defines ways of mapping service characteristics to service providers characteristics generalize names

55 More Information