Distributed Systems. Communication (2) Schedule of Today. Distributed Objects. Distributed Objects and RMI. Corba IDL Example

Transcription

1 1 Overview Distributed Systems Communication (2) Lecture 4 Schedule of Today Remote Object (Method) Invocation Binding Client to an Object Static versus Dynamic Binding Basics MPI, Sockets, Distributed Event Based Communication Hint: Details on Sockets later on by Jan Stöss For a better understanding: read Coulouris et al: Distributed Systems: Concepts and Design, ch Universität Karlsruhe, System Architecture Group 2003 Universität Karlsruhe, System Architecture Group 2 and RMI Objects that can receive remote method invocations are called remote objects and they implement a remote interface Remote Due to possibility of independent failures of invoker and invoked object RMIs differ from local invocations Code for marshalling etc. can be generated automatically by an IDL compiler from the definition of the remote interface.! Organization of a remote object with client-side proxy! Real distributed objects may have a distributed state 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 4 Distributed Object System Architecture Corba IDL Example Applications RMI, RPC and events Request reply protocol External data representation Operating System Middleware layers // In file Person.idl struct Person { string name; string place; long year; ; interface PersonList { readonly attribute string listname; void addperson(in Person p) ; void getperson(in string name, out Person p); long number(); ; 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 6

2 2003 Universität Karlsruhe, System Architecture Group 7 Communication between! Object model (brief repetition)! Distributed objects! Distributed object model! Design Issues! Implementation! Distributed garbage collection The Object Model (1)! Objects! Encapsulate data, offering methods! Object references! Variable holding an object actually holds its reference to that object! Object references may be assigned to variables, passed as arguments and returned as results! Interfaces! Provides a definition of the signatures of a set of methods (i.e. types of their arguments, return values, and exceptions)! In Java, a class may implement several interfaces 2003 Universität Karlsruhe, System Architecture Group 8 The Object Model (2)! Actions! Initiated by an object invoking a method of another object! State of invoked object may be changed, and! further invocations of methods on other objects may take place! Exceptions! Programs may encounter many sorts of errors and unexpected conditions of varying seriousness, e.g.! Failures attempting to read or write to a file or socket! Inconsistent values in the object s variables! Block of code may be defined to throw an exception whenever unexpected conditions or errors occur! Another block of code catches the exception, control does not return to the place where the exception was thrown! Garbage collection State of object = values of its instance variables State of program = partitioned into different parts, each of which associated with an object physical distribution of objects = a natural extension Objects = primary candidates for being replicated and migrated, in order to achieve! Fault tolerance! Performance! Availability 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 10 Local and Remote Method Invocation Remote Object and its Remote Interface Remote remoteobject A remote invocation B local C invocation local E invocation local invocation D remote invocation F Data remote interface m1 m2 implementation { m3 of methods m4 m5 m6! Local invocation iff object is in the same task! Remote invocation if object belongs to a! different task in the same node or! different task in another node Representation of a remote object reference 32 bits 32 bits 32 bits 32 bits Internet address port number time object number interface of remote object 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 12

3 2003 Universität Karlsruhe, System Architecture Group 13 RMI Semantics Implementation of RMI Retransmit request message No Yes Yes Fault tolerance measures Duplicate filtering Not applicable No Yes Invocation semantics Re-execute procedure or retransmit reply Not applicable Maybe Re-execute procedure At-least-once Retransmit reply At-most-once client server remote object A proxy for B skeleton object B Request & dispatcher for B s class Reply Remote Communication Communication Remote reference reference module module module module! Role of a proxy and skeleton in an RMI 2003 Universität Karlsruhe, System Architecture Group 14 Request-Reply Message Structure Implementing Proxies messagetype requestid objectreference methodid arguments int (0=Request, 1= Reply) int RemoteObjectRef int or Method array of bytes Instead of executing the method-code a proxy forwards a message to the remote object hiding all the details between the different nodes. There is 1 proxy for each remote object for which a task holds a remote object reference Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 16 Implementing Dispatcher A server has 1 dispatcher and skeleton for each class representing a remote object. Dispatcher receives the request message from the communication module. It uses methodid to select the appropriate method in the skeleton, passing Implementing Skeleton Each class of a remote object has a skeleton which implements the methods of the remote interface. A skeleton method unmarschals arguments in the request message and finally invokes the server method in the remote object It waits for the invocation to complete and the marshals the result together with any exceptions in a reply message to the client s proxy 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 18

4 2003 Universität Karlsruhe, System Architecture Group 19 Compile-time versus Runtime-Objects Compile-time objects! Instances of a class! Interfaces can be compiled into (client-) proxies and server-side stubs (skeletons)! Disadvantage = language dependency (Java, C++, ) Runtime objects! Object adapter acting as a wrapper Persistent and Transient Objects! Persistent object: may exist, even if currently not mapped to an address space (task)! Not dependent on its current server, i.e. if current server exits it may store the object s state to disk! Later a new server may map the state of the persistent object from disk to its address space! Transient object: exists only as long as its server 2003 Universität Karlsruhe, System Architecture Group 20 Object References Binding a Client to an Object! System wide object handles! Can be freely passed as parameters! Include! Globally unique object id! Network address of server! Protocols required to access object (opt.)! Pointer to proxy implementation (opt.) Distr_object* obj_ref; //Declare a systemwide object reference obj_ref = ; //Initialize reference to distributed object obj_ref-> do_something(); //Implicitly bind and invoke a method (a) Distr_object obj_ref; //Declare a systemwide object reference Local_object* obj_ptr; //Declare a pointer to local objects obj_ref = ; //Initialize reference to distributed object obj_ptr = bind(obj_ref); //Explicitly bind and obtain a pointer //to the local proxy obj_ptr -> do_something(); //Invoke a method on the local proxy (b) a) An example with implicit binding using only global references b) An example with explicit binding using global and local references 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 22 Parameter Passing Examples! DCE Remote Objects! Java RMI! Events and!! Situation when passing an object by reference or by value 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 24

5 2003 Universität Karlsruhe, System Architecture Group 25 DCE Distributed-Object Model Remote Object Invocation Java RMI Remote Object Invocation 2-19 a) Distributed dynamic (private) objects b) Distributed named (shared) objects! Distributed objects provided at language level! Goal is to achieve transparency! Give up transparency if impractical or inefficient! Remote objects = only allowed form of distributed objects! Local versus remote objects differences when! Cloning! Cloning the actual object only, not the proxies! Synchronizing! Synchronized methods! Only proxy synchronization is allowed See: Universität Karlsruhe, System Architecture Group 26 Remote Interfaces Shape, ShapeList import java.rmi.*; import java.util.vector; public interface Shape extends Remote { int getversion() throws RemoteException; GraphicalObject getallstate() throws RemoteException; 1 public interface ShapeList extends Remote { Shape newshape(graphicalobject g) throws RemoteException; 2 Vector allshapes() throws RemoteException; int getversion() throws RemoteException; Naming class of Java RMIregistry void rebind (String name, Remote obj) This method is used by a server to register the identifier of a remote object by name, as shown in Figure 15.13, line 3. void bind (String name, Remote obj) This method can alternatively be used by a server to register a remote object by name, but if the name is already bound to a remote object reference an exception is thrown. void unbind (String name, Remote obj) This method removes a binding. Remote lookup(string name) This method is used by clients to look up a remote object by name, as shown in Figure line 1. A remote object reference is returned. String [] list() This method returns an array of Strings containing the names bound in the registry Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 28 Main of class ShapeListServer import java.rmi.*; public class ShapeListServer{ public static void main(string args[]){ System.setSecurityManager(new RMISecurityManager()); try{ ShapeList ashapelist = new ShapeListServant(); 1 Naming.rebind("Shape List", ashapelist ); 2 System.out.println("ShapeList server ready"); catch(exception e) { System.out.println( "ShapeList server main " + e.getmessage()); 2003 Universität Karlsruhe, System Architecture Group 29 ShapeListServant implements interface ShapeList import java.rmi.*; import java.rmi.server.unicastremoteobject; import java.util.vector; public class ShapeListServant extends UnicastRemoteObject implements ShapeList { private Vector thelist; // contains the list of Shapes 1 private int version; public ShapeListServant()throws RemoteException{... public Shape newshape(graphicalobject g) throws RemoteException { 2 version++; Shape s = new ShapeServant( g, version); 3 thelist.addelement(s); return s; public Vector allshapes()throws RemoteException{... public int getversion() throws RemoteException { Universität Karlsruhe, System Architecture Group 30

6 2003 Universität Karlsruhe, System Architecture Group 31 Java Client of ShapeList import java.rmi.*; import java.rmi.server.*; import java.util.vector; public class ShapeListClient{ public static void main(string args[]){ System.setSecurityManager(new RMISecurityManager()); ShapeList ashapelist = null; try{ ashapelist = (ShapeList)Naming.lookup ("//bruno.shapelist"); 1 Vector slist = ashapelist.allshapes(); 2 catch(remoteexception e) {System.out.println(e.getMessage()); catch(exception e) {System.out.println("Client: " + e.getmessage()); Classes supporting Java RMI RemoteObject RemoteServer Activatable UnicastRemoteObject <servant class> 2003 Universität Karlsruhe, System Architecture Group 32 Message Passing System * Basic Communication Message Passing System Basic Communication! Implements data transfer via a network! Offers communication primitives at API:! at least a send(...) and a receive(...) operation Distributed Application API Network Node 1 Node 2 Node 3 Functionality of a Message Passing system:! Uses standard protocols or implements new ones! Guarantees specific properties according to semantic! e.g. order of messages! Abstracts from implementation details! e.g. buffering, low-level addressing! Masks certain failures! e.g. automatic repetition after timeout! Hides heterogeneous HW and SW components improving portability *Very simple form of a middleware 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 34 Pragmatic Design Parameters Overview Message Passing (1) Basic Communication! Length of message! Constant or fixed! Variable, but limited in size! Unlimited! Loss of messages! Not noticed! Suspected and notified! Avoided! Integrity of messages! Not noticed! Detected and notified! Automatically corrected! Message: piece of information passed between processes/objects! Mailbox: place holding messages! Primitives:! send() place message into a mailbox, if there is space! receive() take next message from mailbox, if not empty! select() check for messages on multiple mailboxes 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 36

7 2003 Universität Karlsruhe, System Architecture Group 37 Message Passing (2) Basic Communication Message Passing (3) Basic Communication Relationship between mailboxes + processes! 1:1 (one mailbox per process)! 1:n! m:1! m:n many:many! Extent of buffering (in mailbox)! Limited buffering (typical)! None ( rendez-vous ) Synchronization! Blocking receive: wait for message! Non-blocking receive: return empty! Blocking send: wait for space in mailbox! Non-blocking send: return full in case of a fill mailbox Note: Either sender or receiver must wait! 2003 Universität Karlsruhe, System Architecture Group 38! To overcome synchronous behavior of RPC/RMI! Persistence and Synchronicity in Communication! Message-Oriented Transient Communication! Berkeley sockets! MPI! Message-Oriented Persistent Communication! Message-Queuing Model! General Architecture of a Message-Queuing System! Message Brokers Orthogonal Design Parameters! Synchronous versus asynchronous! Placing the message buffers! Persistent versus transient communication! Addressing the communicating instances! 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 40 Persistence in Communication (1) Persistence in Communication (2)! Organization of a communication system in which hosts are connected through communication servers via a network! Persistent communication of letters back in the ancient days of the Pony Express Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 42

8 2003 Universität Karlsruhe, System Architecture Group 43 Communication Models Persistence and Synchronicity (1)! Persistent versus Transient! Persistent messages stored as long as necessary by the communication system (e.g. )! Transient messages are discarded when they cannot be delivered (e.g. TCP/IP)! Synchronous versus Asynchronous! Asynchronous implies sender proceeds as soon as it sends the message, i.e. no blocking! Synchronous implies sender blocks until the receiving host buffers the message a) Persistent asynchronous communication b) Persistent synchronous communication 2003 Universität Karlsruhe, System Architecture Group 44 Persistence and Synchronicity (2) Persistence and Synchronicity (3) c) Transient asynchronous communication d) Receipt-based transient synchronous communication e) Delivery-based transient synchronous communication at message delivery f) Response-based transient synchronous communication 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 46 Basic Communication Example: UNIX Sockets and Ports UNIX Sockets (1) Basic Communication! Communication endpoint Primitive Socket Bind Listen Accept Connect Send Meaning Create a new communication endpoint Attach a local address to a socket Announce willingness to accept connections Block caller until a connection request arrives Actively attempt to establish a connection Send some data over the connection! Identified by IP address + port number! Many-to-many relationship with processes! TCP, UDP, IP multicast, etc. Receive Receive some data over the connection Select Check which sockets have data Close Release the connection! Socket primitives 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 48

9 2003 Universität Karlsruhe, System Architecture Group 49 UNIX Sockets (2)! Connection-oriented communication pattern using sockets. UNIX Communication Primitives Send! send a message to a socket! blocking or non-blocking Receive! receive a message on a socket! blocking or non-blocking Select (Poll)! check for events on a set of ports! blocking or non-blocking 2003 Universität Karlsruhe, System Architecture Group 50 Berkeley Sockets (1) Primitive Meaning Socket Create a new communication endpoint Bind Attach a local address to a socket Create a new endpoint Berkeley Sockets (2) Associate endpoint Reserve buffers Block waiting for requests Listen Accept Connect Send Receive Announce willingness to accept connections Block caller until a connection request arrives Actively attempt to establish a connection Send some data over the connection Receive some data over the connection Close Release the connection! Socket primitives for TCP/IP. Automatic binding after connection! Connection-oriented communication pattern using sockets Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 52 Message Passing Interface (1)! Overcome disadvantages of sockets:! Wrong level of abstraction being implemented at a too low level with only very primitive operations! Designed for to communicate across networks using general-purpose protocol stacks such as TCP/IP Not well suited for high-speed interconnection networks used in COW* (Myrinet) or MPPs Message Passing Interface (2) Assumptions: 1. Communication only within a group of processes 2. Each group has a unique identifier 3. Each process in a group has a (local) identifier (groupid,processid) uniquely identifies source or target of a message 4. Support diverse forms of buffering and synchronization (over 100 functions) 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 54

10 2003 Universität Karlsruhe, System Architecture Group 55 Message-Passing Interface (3) Persistent Communication Primitive MPI_bsend MPI_send MPI_ssend MPI_sendrecv MPI_isend MPI_issend MPI_recv MPI_irecv Meaning Append outgoing message to a local send buffer Send a message and wait until copied to local or remote buffer Send a message and wait until receipt starts Send a message and wait for reply Pass reference to outgoing message, and continue Pass reference to outgoing message, and wait until receipt starts Receive a message; block if there are none Check if there is an incoming message, but do not block! Application communicate by inserting messages in specific queues! Loosely coupled communication! Support for! persistent asynchronous communication! Larger message transfer (e.g. systems)! The most intuitive message-passing primitives of MPI 2003 Universität Karlsruhe, System Architecture Group 56 Message-Queuing Model (1) Message-Queuing Model (2) Primitive Put Get Poll Notify Meaning Append a message to a specified queue Block until the specified queue is nonempty, and remove the first message Check a specified queue for messages, and remove the first. Never block. Install a handler to be called when a message is put into the specified queue.! 4 combinations for loosely-coupled communications using queues.! Basic interface to a queue in a message-queuing system Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 58 Architecture of a Message-Queuing System (1) Architecture of a Message-Queuing System (2) 2-29! The relationship between queue-level addressing and network-level addressing.! General organization of a message-queuing system with routers Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 60

11 2003 Universität Karlsruhe, System Architecture Group 61 Message Brokers Example: IBM MQSeries! General organization of a message broker in a message-queuing system.! Organization of IBM's MQSeries message-queuing system 2003 Universität Karlsruhe, System Architecture Group 62 Channels Message Transfer (1) Attribute Transport type FIFO delivery Message length Setup retry count Delivery retries Description Determines the transport protocol to be used Indicates that messages are to be delivered in the order they are sent Maximum length of a single message Specifies maximum number of retries to start up the remote MCA Maximum times MCA will try to put received message into queue! Attributes associated with message channel s! General organization of an MQSeries queuing network using routing tables and aliases Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 64 Message Transfer (2) Simple Mail Transfer Protocol (SMTP) Primitive Description MQopen Open a (possibly remote) queue MQclose Close a queue MQput Put a message into an opened queue MQget Get a message from a (local) queue! Primitives available in an IBM MQSeries MQI! Processes! s (mail readers)! Eudora, pine, elm, outlook, messenger! Mail servers! Store messages! SMTP! Uses TCP/IP! Uses DNS! Client-to-server protocols! Pop (post office protocol)! Imap (internet mail access protocol) 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 66

12 2003 Universität Karlsruhe, System Architecture Group 67 SMTP Mail server SMTP Mail server Network News Protocol News client 1.1 News client 1.2 News server 1 News server 3 News client 1.3 News client 3.1 News client 3.2 mailbox Mail server Queue of outgoing messages News client 2.1 News client 2.2 News server 2 News server 4 Network News Reader Protocol (NNRP) Network News Transport Protocol (NNTP) Uses TCP Servers flood fill their peers with new postings News client 4.1 News client 4.2 News client Universität Karlsruhe, System Architecture Group 68 Event & s A different IPC style! Interaction is specified in terms of topics, rather than senders and receivers! Senders and receivers need not be aware of each other! Asynchronous! Loosely coupling! Events cause changes in the state of an object! s asynchronously inform other interested objects of the occurrence of events Example: Dealing Room Systems Dealer s computer Dealer Dealer s computer Dealer Information provider External source Information provider External source Dealer s computer Dealer Dealer s computer Dealer 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 70 Event (2) Event (3) Publish-subscribe! Objects publish events of certain types (on certain topics)! Objects subscribe to the event types (topics) they are interested in! Systems delivers notifications of events to all subscribed objects In general, events have! a type! type-specific attributes Subscriptions are in terms of! event type! optional predicate on the value of event attributes 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 72

13 2003 Universität Karlsruhe, System Architecture Group 73 Event (4) Distributed Event Architecture Example: distributed multi-player game:! Object move in game space, publish events that carry object s position as attribute Event service object of interest 1. object of interest observer notification subscriber subscriber! Object are interested in events generated by object within their visible range, subscribe to event with appropriate position attributes object of interest notification observer notification notification subscriber 2003 Universität Karlsruhe, System Architecture Group 74 Event System Implementation:! Centralized database of events and subscribers! Distributed ENS:! General Approach: spanning tree connects publishers and subscribers 1. One tree per event type, subscribers filter based on predicates, or 2. Subscriptions propagated up the tree, interior nodes filter as appropriate. Could rearrange tree for efficiency (but difficult without global info!)! Support for continuous media that have temporal requirements! Asynchronous transmission! Ordering is important (e.g. file transfer)! Synchronous transmission! Ordering and max delay is important! Isochronous transmission! Man and min delay is important (i.e. bounded delay or jitter)! Simple versus complex streams! Relationship between substreams are also timedependent (consider synchronizing audio and video for a movie) 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 76 Data Stream (1) Data Stream (2) ! Setting up a stream between 2 tasks across a network! Setting up a stream directly between two devices 2003 Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 78

14 2003 Universität Karlsruhe, System Architecture Group 79 Data Stream (3) Specifying QoS (1) Characteristics of the Input maximum data unit size (bytes) Token bucket rate (bytes/sec) Toke bucket size (bytes) Maximum transmission rate (bytes/sec) Service Required Loss sensitivity (bytes) Loss interval (µsec) Burst loss sensitivity (data units) Minimum delay noticed (µsec) Maximum delay variation (µsec) Quality of guarantee! An example of multicasting a stream to several receivers.! A flow specification 2003 Universität Karlsruhe, System Architecture Group 80 Specifying QoS (2) Setting Up a Stream! The principle of a token bucket algorithm! The basic organization of RSVP for resource reservation in a distributed system Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 82 Synchronization Mechanisms (1) Synchronization Mechanisms (2)! Principle of explicit synchronization on the level data units! Principle of synchronization as supported by high-level interfaces Universität Karlsruhe, System Architecture Group Universität Karlsruhe, System Architecture Group 84