DISTRIBUTED SYSTEMS [COMP9243] Lecture 7: Middleware MIDDLEWARE. Distributed Object based: Slide 1. Slide 3. Message-oriented: Slide 4

Transcription

1 KINDS OF MIDDLEWARE DISTRIBUTED SYSTEMS [COMP9243] Lecture 7: Middleware Distributed Object based: Objects invoke each other s methods Server Slide 1 ➀ Introduction ➁ Distributed Object Middleware Remote Objects & CORBA Distributed Shared Objects & Globe ➂ Message-Oriented Middleware ➃ Publish/Subscribe Middleware Slide 3 Object MIDDLEWARE Message-oriented: Messages are sent between processes Message queues Machine A Machine B Machine C Sender A Distributed applications Application Application Slide 2 Network OS services Middleware services Network OS services Network OS services Slide 4 Receive queue Send queue Message R2 Kernel Kernel Kernel Application R1 Network Application Router Receiver B KINDS OF MIDDLEWARE 1 KINDS OF MIDDLEWARE 2

2 Coordination-based: Tuple space Transaction Processing Monitors: Slide 5 A Write A B Write B T Read T Insert a copy of A Tuple instance Insert a copy of B A B B B A C Look for tuple that matches T C Return C (and optionally remove it) Slide 7 Transaction Client application Requests Reply Reply Request Request TP monitor Reply Request Reply Server Server Server A JavaSpace Publish/Subscribe Web Services: Publisher Subscriber Subscriber Stock Servicequery_stock buy sell Client XML RPC Bank Servicebalance tansfer HTTP HTTP Slide 6 01 Data Item 01 Subscription Slide 8 XML RPC Match Match Publish/Subscribe Middleware Auction Servicesearch get_auction manage_auction bid SOAP Photo Servicesearch add_photo delete_photo update_photo HTTP KINDS OF MIDDLEWARE 3 DISTRIBUTED OBJECTS 4

3 DISTRIBUTED OBJECTS OBJECT MODEL Classes and Objects Class: defines a type Slide 9 Slide 11 Object: instance of a class Interfaces Object references Active vs Passive objects Persistent vs Transient objects Static vs Dynamic method invocation CHALLENGES Transparency INTERFACES Failure transparency Reliability Define an object s methods Composition of interfaces Dealing with partial failures Inheritance of interfaces Slide 10 Scalability Number of clients of an object Distance between client and object Design Must take distributed nature into account from beginning Performance Flexibility Slide 12 Versions of interfaces Interface Definition Language: Types of objects, Public methods, Exceptions that may occur etc. OBJECT MODEL 5 REMOTE OBJECT ARCHITECTURAL MODEL 6

4 REMOTE OBJECT ARCHITECTURAL MODEL OBJECT SERVER Client Interface Server Object State Methods Object: State & Methods Implements a particular interface Server Slide 13 Proxy Skeleton Run Time System Run Time System Client OS Server OS Remote Objects: Single copy of object state (at single object server) All methods executed at single object server All clients access object through proxy Object s location is location of state Interface Slide 15 Skeleton: Server stub Static vs dynamic skeletons Run-Time System: Dispatches to appropriate object Invocation policies Object Server: Hosts object implementations Transient vs Persistent objects Concurrent access Support legacy code Skeleton Run Time System Server OS CLIENT Slide 14 Client Process: Binds to distributed object Invokes methods on object Proxy: Proxy: RPC stub + destination details Binding causes a proxy to be created Responsible for marshaling Static vs dynamic proxies Usually generated Client Proxy Run Time System Client OS Slide 16 OBJECT REFERENCE Local Reference: Language reference to proxy Proxy Run-Time System: Provides services (translating references, etc.) Send and receive OBJECT SERVER 7 OBJECT REFERENCE 8

5 Remote Reference: Server address + object ID OBJECT REFERENCE Object name (human friendly, object ID, etc.) Naming Service Slide 17 OR address id type Remote Object Slide 19 OR Proxy name Remote Object Proxy What are the drawbacks and/or benefits of each approach? BINDING AND NAME RESOLUTION Reference to proxy code (e.g., URL) & init data Name Resolution: Name remote reference Slide 18 OR URL to proxy code init Remote Object Slide 20 Reference info contained in name (e.g., URL), or Naming service stores name to reference mappings Binding: Remote reference local reference Create a proxy Connect proxy to object server OBJECT REFERENCE 9 REMOTE METHOD INVOCATION (RMI) 10

6 REMOTE METHOD INVOCATION (RMI) STATIC VS DYNAMIC INVOCATION Standard invocation (synchronous): Client invokes method on proxy Proxy performs RPC to object server Slide 21 Skeleton at object server invokes method on object Object server may be required to create object first Slide 23 Other invocations: Asynchronous invocations Persistent invocations Notifications and Callbacks INVOCATION SEMANTICS Local method invocation: executed exactly once Problem: Cannot make such a guarantee for remote invocations! Features: CORBA Object Management Group (OMG) Standard (version 3.1) Slide 22 Fault Tolerance Measure Retransmit Filter Re-execute, Invocation Request Duplicates or Re-reply Semantics No n/a n/a Maybe Yes No Re-execute procedure At-least-once Yes Yes Retransmit reply At-most-once Slide 24 Range of language mappings Transparency: Location & some migration transparency Invocation semantics: at-most-once semantics by default; maybe semantics can be selected Services: include support for naming, security, events, persistent storage, transactions, etc. What s the difference between Maybe and At-most-once? STATIC VS DYNAMIC INVOCATION 11 CORBA ARCHITECTURE 12

7 Example: A Simple File System: CORBA ARCHITECTURE module CorbaFS { interface File; // forward declaration Slide 25 Static IDL proxy Client machine Client application Dynamic Invocation Interface Client ORB ORB interface Object adapter Server machine Object implementation Skeleton Dynamic Skeleton Interface Server ORB ORB interface Slide 27 // a super-simple file system interface FileSystem { exception CantOpen {string reason; enum OpenMode {Read, Write, ReadWrite File open (in string fname, in OpenMode mode) raises (CantOpen); Local OS Local OS Network // an open file interface File { string read (in long nchars); void write (in string data); void close (); INTERFACES: OMG IDL ELEMENTS OF IDL Slide 26 Components of an interface definition: Defines class attributes and methods (Classes are called interfaces and methods are called operations) Method arguments are annotated as in, out, and inout Errors/exceptions Interface inheritance Characteristics of OMG IDL: Grammar is a subset of ANSI C++ plus additional constructs for object invocation No control structures, only declarations Primitive and complex data types, but no pointers No templates and no overloading Includes support for pre-processor Slide 28 Basic types:: The usual short to long long types (size fixed) Latin 1 & wide characters boolean & octet any type Constructed types:: C/C++-style enumerations & structures Other types:: Sequence types e.g.,sequence<octet> Strings, Arrays Exceptions: Standard exceptions User-defined exceptions: struct-like data type INTERFACES: OMG IDL 13 ELEMENTS OF IDL 14

8 Modules & Interfaces: LANGUAGE MAPPINGS Slide 29 An interface defines the public interface of a CORBA object Interfaces can be defined using (multiple) inheritance A module bundles related declarations and interfaces module Foo { typedef long bar; interface A { enum baz { interface B {... interface C: A, B {... Valid names: Foo::bar, Foo::A::baz, and Foo::C::baz Slide 31 Accessing interfaces from "real" programming languages?: Standardised language mappings define Host language representation for basic and constructed IDL types, References to constants and objects, Mechanisms for invoking operations, Behaviour in case of an exceptions, and Signatures for standard operations like those of the object adapters. Ideally, client and (to a lesser extent) object implementation code should be portable across different IDL compilers and ORBs. Note: Defines only how an IDL interface appears in a given language, but not how it is implemented (e.g., OR). Slide 30 Operations & Attributes: All operations contained in the interface of an object can be invoked on that object interface Echo { void doecho (in string msg) raises(echoexception); oneway operations Attributes specify publicly accessible data items interface Counter { readonly attribute long value; void inc (); readonly attributes Slide 32 But C does not have: objects, inheritance, exceptions, and nested name spaces. The C language mapping: Objects: each object has type CORBA_Object Operations: function gets object as additional first argument Inheritance: method vectors and object adapter Exceptions: each operation gets a reference to CORBA_Environment as additional last argument Poor man s name spaces: :: is replaced by _ LANGUAGE MAPPINGS 15 LANGUAGE MAPPINGS 16

9 Slide 33 interface Echo { void doecho (in string msg); typedef CORBA_Object Echo; void Echo_doEcho (Echo _obj, CORBA_char *msg, CORBA_Environment *ev); Slide 35 Object Reference (OR): OBJECT REFERENCE (OR) Refers to exactly one object, but an object can have multiple, distinct handles A language mapping provides an opaque representation ORs are implementation specific interface Counter { readonly attribute long value; void inc (); Interoperable Object Reference (IOR) Can be shared between different implementations Tagged Profile Interoperable Object Reference (IOR) Slide 34 typedef CORBA_Object Counter; CORBA_long Counter get_value (Counter _obj, CORBA_Environment *ev); void Counter_inc (Counter _obj, CORBA_Environment *ev); Slide 36 Repository identifier IIOP version Profile ID Profile Host Port Object key Components Note: No definition for the attribute itself, only the _get_value operation. POA identifier Object identifier Other serverspecific information OBJECT REFERENCE (OR) 17 CLIENT 18

10 CLIENT Slide 37 Binds to remote object Static proxy Dynamic invocation Client-Side Dynamic Invocation: Dynamic Invocation interface (DII) Request is dynamically created Interface repository: Persistent storage of interface definitions Dynamic type checking and checking of inheritance graph Interface browser May be queried by language bindings Slide 39 OBJECT REQUEST BROKER (ORB) Provides run-time system Translate between remote and local references Send and receive messages Maintains interface repository Enables dynamic invocation (client and server side) Locates services OBJECT Slide 38 Also known as Servant Can be proper object (e.g., C++, Java) Can be state and procedures (e.g., C) Can be legacy code (with a wrapper) Slide 40 ORB In the same process ORB Separate processes ORB Object Adapter: Dispatcher (OR Servant) Invokes skeleton (static or dynamic) Possibly creates objects Portable Object Adapter Proxy Skeleton ORB Server based ORB OBJECT REQUEST BROKER (ORB) 19 OBJECT REQUEST BROKER (ORB) 20

11 Slide 41 CORBA Leaves Implementations a Lot of Freedom: Advantages ORB implementations can more easily be optimised for size, speed, or functionality Wide applicability; allows extreme cases like use in real-time systems (TAOS) Future improvements in network & compiler technology can be exploited more easily Vendors can specialise Disadvantages Slide 43 Direct Binding: Create proxy BINDING ORB connects to server (using info from IOR) Invocation requests are sent over connection Indirect Binding: Client IOR IOR refers to implementation repository 5. Actual invocation 4. Redirect message Object server Complexity of the specification; more difficult to understand Incompatibility of different implementations This is in contrast to COM/DCOM. 1. First invocation or binding request 3. Ack object is active Implementation repository 2. Activate/start object INTERCEPTORS COMMUNICATION: GENERAL INTER-ORB PROTOCOL (GIOP) Protocol framework for communicating between ORBs Client application Does not specify actual transport protocol Slide 42 Request-level interceptor Message-level interceptor Client proxy Client ORB Invocation request Slide 44 Components of the GIOP specification: Transfer syntax Message formats Transport layer assumptions Goals of GIOP: Local OS To server Simplicity & ease of implementation Scalability Generality & architecture neutrality BINDING 21 COMMON DATA REPRESENTATION (CDR) 22

12 COMMON DATA REPRESENTATION (CDR) neutral, bi-canonical, on-the-wire representation variable byte addressing (receiver might swap) data aligned at natural boundaries (improves efficiency) covers all data types that can be expressed in OMG IDL MESSAGE TYPES Request message:: Invocation of an operation or use of attribute accessor Request header and request body For example: boolean: True; long: 132; short: 56; char: Q; long long: 234,663 Reply message:: Reply to a request (that requires a reply) Slide 45 Value - 0x01 0x00 0x00 0x00 0x84 0x00 0x38 0x51 0x00 0x00 Address x00 0x00 0x00 0x03 0x94 0xA Slide 47 Reply header and reply body CancelRequest message:: Request not needed anymore (advisory) Transfers the request ID Value - 0x01 0x20 0x20 0x20 0x00 0x00 0x00 0x84 0x00 0x38 0x51 Address x20 0x20 0x20 0x20 0x20 0x00 0x00 0x00 0x00 0x00 0x03 0x94 0xA LocateRequest message:: Check object validity and capabilities Object of interest GIOP MESSAGES Slide 46 GIOP message header (1.1): module GIOP { struct Version {octet major; octet minor; enum MsgType_1_1 { Request, Reply, CancelRequest, LocateRequest, LocateReply, CloseConnection, MessageError, Fragment struct MessageHeader_1_1 { char magic[4]; Version GIOP_version; octet flags; // byte order, last fragment octet message_type; unsigned long message_size; Slide 48 LocateReply message:: Reply to locate request Object might be unknown, available, or forwarded CloseConnection message:: Server terminates connection MessageError message:: Error in message version or format Fragment message:: Follow up to incomplete message MESSAGE TYPES 23 TRANSPORT LAYER (GIOP REQUIREMENTS) 24

13 DISTRIBUTED SHARED OBJECT (DSO) MODEL TRANSPORT LAYER (GIOP REQUIREMENTS) Slide 49 Transport must be connection-oriented Transport must be reliable Unbounded stream of bytes Transport must notify in case of connection loss TCP s connection initiation model must be implementable TCP/IP fits very well Slide 51 State Interface Network Local Representative (Local Object) Process From GIOP to Internet Inter-Orb Protocol (IIOP): Distributed Shared Object A small step: IOR definition for TCP/IP Connection handling Distributed Shared Objects: Object state can be replicated (at multiple object servers) Object state can be partitioned Methods executed at some or all replicas Object location no longer clearly defined CORBA BIBLIOGRAPHY Slide 50 [1] IIOP Complete, W. Ruh, T. Herron, and P. Klinker, Addison Wesley, [2] The Common Object Request Broker: Architecture and Specification (2.3.1), Object Management Group, [3] C Language Mapping Specification, Object Management Group, [4] CORBAservices: Common Object Services Specification, Object Management Group, Play with CORBA. Many implementations available, including ORBit: Slide 52 CLIENT Client has local representative (LR) in its address space Stateless LR Equivalent to proxy Methods executed remotely Statefull LR Full state Partial state Methods (possibly) executed locally Client app LR DISTRIBUTED SHARED OBJECT (DSO) MODEL 25 OBJECT 26

14 OBJECT GLOBE (GLOBAL OBJECT BASED ENVIRONMENT) Scalable wide-area distributed system: Wide-area scalability requires replication Wide-area scalability requires flexibility Slide 53 Remote Object Replicated Object Slide 55 Features: Per-object replication and consistency Per-object communication Mechanism not policy Transparency (replication, migration) Dynamic replication Partitioned Object Replicated and Partitioned Object LOCAL REPRESENTATIVE OBJECT SERVER Slide 54 Server dedicated to hosting LRs Provides resources (network, disk, etc.) Static vs Dynamic LR support Transient vs Persistent LRs Security mechanisms LR Obj A LR Obj B Slide 56 Replication subobject Control subobject Same interface as implemented by semantics subobject Semantics subobject Location of LRs: Communication subobject LRs only hosted by clients Statefull LRs only hosted by object servers Statefull LRs on both clients and object servers Communication with other local objects GLOBE (GLOBAL OBJECT BASED ENVIRONMENT) 27 SEMANTICS SUBOBJECT 28

15 CONTROL-REPLICATION SUBOBJECT INTERACTION Control Subobject: SEMANTICS SUBOBJECT Equivalent to CORBA Servant Implements object interface (Globe IDL) INVOKE State stored in semantics subobject START RETURN Slide 57 Written as though non-distributed (in theory!) Problems to look out for: Slide 59 SEND Concurrent access No references to other Globe objects No complex parameters (lists, other Globe objects, etc.) No assumptions about local environment Replication Subobject: Read methods Write methods Replication interface: start(), send(), invoked() REPLICATION AND CONTROL SUBOBJECTS REPLICATION EXAMPLE: ACTIVE REPLICATION Slide 58 Control Subobject: Implements standard control interface Control Mediates between replication and semantics subobjects Does marshaling/unmarshaling Replication Semantics Slide 60 Repl Local object process B Ctrl Sem Local object process A Repl Ctrl Process A has no semantics subobject Repl Local object process C Ctrl Sem Replication Subobject: Comm Comm Comm Implements standard replication interface Responsible for replication and consistency Decides how to handle local invocations and incoming requests Request path Reply path Replication subobject forwards invocation to A and B and later merges results Implementation independent of semantics subobject CONTROL-REPLICATION SUBOBJECT INTERACTION 29 BINDING 30

16 BINDING Slide 61 Client 3. Select address Class object Local object 1. Name Object handle 2. Object handle Addresses & protocols Naming service Location service Register contact address Slide 63 MESSAGE-ORIENTED MIDDLEWARE (MOM) Middleware services to provide message passing. 5. Make contact 4. Load & instantiate class(es) (Trusted) class repository Distributed shared object Slide 62 Location Service: NAMING AND LOCATION SERVICES Object handle Contact Address Contact Address: Contact Point + LR implementation info Contact Point: Address Hierarchy of regions Pointer caching A0 region A0 A1 region A1 PA region A2 A2 R B2 B1 B0 region B0 PB region B1 region B2 C0 region C0 region PA region PB region PC region R PC C1 region C1 Slide 64 Provides: Persistent communication Message Queues MESSAGE QUEUING SYSTEMS Transfer of messages between queues Model: Application-specific queues Messages addressed to specific queues Only guarantee delivery to queue. Not when. Message transfer can be in the order of minutes Very similar to but more general purpose (i.e., enables communication between applications and not just people) MESSAGE-ORIENTED MIDDLEWARE (MOM) 31 MESSAGE QUEUING SYSTEMS 32

17 Message Queue Architecture Example: Sender A Basic queue interface: Application Application Receive queue Message R2 Slide 65 Slide 67 Send queue Application R1 Application Router Receiver B Message Queue Architecture Concepts: Examples: Slide 66 Queue Source queue (local) Destination queue (remote) Queue Manager Direct (manages queues on its machine) Relay/Router (routes to given destination queue s machine) Multicast support (routes to multiple destination queues) Message Broker Translates between application specific message formats Application gateway Slide 68 IBM MQSeries Message channels connect queue managers Message Channel Agent (MCA) manages a channel end (sends and receives transport level messages) Source and destination MCAs must be running to use channel Queue manager responsible for routing Java Message Service Java API to messaging system (e.g., MQ Series) Implementation of own messaging system Provides Point-to-point (usual message queues) and Publish/Subscribe MESSAGE QUEUING SYSTEMS 33 PUBLISH/SUBSCRIBE (EVENT-BASED) MIDDLEWARE 34

18 EXAMPLES Slide 69 PUBLISH/SUBSCRIBE (EVENT-BASED) MIDDLEWARE Publisher Subscriber Subscriber 01 Data Item 01 Subscription Match Match Publish/Subscribe Middleware Slide 71 Real-time Control Systems: external events (e.g. sensors) event monitors Stock Market Monitoring: stock updates traders subscribed to updates Network Monitoring: status logged by routers, servers monitors screen for failures, intrusion attempts Enterprise Application Integration: independent applications produce output as events consume events as input decoupled CHALLENGES MESSAGE FILTERING Transparency: loose coupling good transparency Publisher Subscriber Slide 70 Scalability: Potentially good due to loose coupling X In practice hard to achieve Number of subscriptions Number of messages Slide 72 Topic-based 01 Data item: comp.os.distributed Match comp.os.unix Publish/Subscribe Middleware Subscription: comp.os.* Flexibility: Loose coupling gives good flexibility Language & platform independence Policy separate from mechanism Programmability: Inherent distributed design Doesn t use non-distributed concepts Content-based Publisher 01 Data item: name=john age=30 Match name=john gender = male Publish/Subscribe Middleware Subscriber Subscription: name=john EXAMPLES 35 ARCHITECTURE 36

19 ARCHITECTURE Centralised: Multicast-based: Slide 73 Publisher Send Event Broker Send Subscribe Send Event Subscriber Slide 75 Publisher Subscriber Subscriber Publisher 01 Send Event No Match 01 Match 01 No Match 01 Distributed: Peer-to-Peer: Publisher Local Broker Subscriber Local Broker Publisher Subscriber Publisher Subscriber Slide 74 Border Broker Broker Border Broker Slide Send Event 01 Send Event Publisher Subscriber Local Broker Broker Broker Subscriber Publisher Subscriber ARCHITECTURE 37 COMMUNICATION 38

20 Slide 77 COMMUNICATION Point-to-point Multicast hard part is building appropriate multicast tree Content-based routing point-to-point based router network make forwarding decisions based on message content store subscription info at router nodes Slide 79 READING LIST Globe: A Wide-Area Distributed System An overview of Globe CORBA: Integrating Diverse Applications Within Distributed Heterogeneous Environments An overview of CORBA New Features for CORBA 3.0 More CORBA EXAMPLE SYSTEMS TIB/Rendezvous: Topic-based Multicast-based Java Message Service (JMS): API for MOM Slide 78 Topic-based centralised or peer-to-peer implementations possible Scribe: Topic-based Peer-to-peer architecture, based on Pastry (DHT) Topics have unique IDs and map onto nodes Multicast for sending events Tree is built up as nodes subscribe READING LIST 39 READING LIST 40