Distributed Systems. Communication (2) Lecture Universität Karlsruhe, System Architecture Group

Transcription

1 Distributed Systems Communication (2) Lecture Universität Karlsruhe, System Architecture Group 1

2 Overview Schedule of Today Remote Object (Method) Invocation Distributed Objects Binding Client to an Object Static versus Dynamic Binding Message Oriented Communication Basics MPI, Sockets, Distributed Event Based Communication Stream Oriented Communication Hint: Details on Sockets later on by Jan Stöss For a better understanding: read Coulouris et al: Distributed Systems: Concepts and Design, ch Universität Karlsruhe, System Architecture Group 2

3 Distributed Objects Objects that can receive remote method invocations are called remote objects and they implement a remote interface Due to possibility of independent failures of invoker and invoked object RMIs differ from local invocations Code for marshalling etc. can be generated automatically by an IDL compiler from the definition of the remote interface Universität Karlsruhe, System Architecture Group 3

4 Distributed Objects Distributed Objects and RMI Remote! Organization of a remote object with client-side proxy! Real distributed objects may have a distributed state 2003 Universität Karlsruhe, System Architecture Group 4

5 Distributed Object System Architecture Applications RMI, RPC and events Request reply protocol Middleware layers External data representation Operating System 2003 Universität Karlsruhe, System Architecture Group 5

6 Corba IDL Example // In file Person.idl struct Person { string name; string place; long year; } ; interface PersonList { readonly attribute string listname; void addperson(in Person p) ; void getperson(in string name, out Person p); long number(); }; 2003 Universität Karlsruhe, System Architecture Group 6

7 Communication between Distributed Objects! Object model (brief repetition)! Distributed objects! Distributed object model! Design Issues! Implementation! Distributed garbage collection 2003 Universität Karlsruhe, System Architecture Group 7

8 The Object Model (1)! Objects! Encapsulate data, offering methods! Object references! Variable holding an object actually holds its reference to that object! Object references may be assigned to variables, passed as arguments and returned as results! Interfaces! Provides a definition of the signatures of a set of methods (i.e. types of their arguments, return values, and exceptions)! In Java, a class may implement several interfaces 2003 Universität Karlsruhe, System Architecture Group 8

9 The Object Model (2)! Actions! Initiated by an object invoking a method of another object! State of invoked object may be changed, and! further invocations of methods on other objects may take place! Exceptions! Programs may encounter many sorts of errors and unexpected conditions of varying seriousness, e.g.! Failures attempting to read or write to a file or socket! Inconsistent values in the object s variables! Block of code may be defined to throw an exception whenever unexpected conditions or errors occur! Another block of code catches the exception, control does not return to the place where the exception was thrown! Garbage collection 2003 Universität Karlsruhe, System Architecture Group 9

10 Distributed Objects State of object = values of its instance variables State of program = partitioned into different parts, each of which associated with an object physical distribution of objects = a natural extension Objects = primary candidates for being replicated and migrated, in order to achieve! Fault tolerance! Performance! Availability 2003 Universität Karlsruhe, System Architecture Group 10

11 Distributed Objects Local and Remote Method Invocation Remote A remote invocation B local C invocation local E invocation local invocation D remote invocation F! Local invocation iff object is in the same task! Remote invocation if object belongs to a! different task in the same node or! different task in another node 2003 Universität Karlsruhe, System Architecture Group 11

12 Remote Object and its Remote Interface remoteobject Data remote interface m1 { implementation m2 m3 of methods m4 m5 m6 Representation of a remote object reference 32 bits 32 bits 32 bits 32 bits Internet address port number time object number interface of remote object 2003 Universität Karlsruhe, System Architecture Group 12

13 RMI Semantics Fault tolerance measures Invocation semantics Retransmit request message Duplicate filtering Re-execute procedure or retransmit reply No Not applicable Not applicable Maybe Yes No Re-execute procedure At-least-once Yes Yes Retransmit reply At-most-once 2003 Universität Karlsruhe, System Architecture Group 13

14 Implementation of RMI object A client proxy for B Request server skeleton & dispatcher for B s class remote object B Reply Remote Communication reference module module Communication module Remote reference module! Role of a proxy and skeleton in an RMI 2003 Universität Karlsruhe, System Architecture Group 14

15 Request-Reply Message Structure messagetype requestid int int (0=Request, 1= Reply) objectreference methodid arguments RemoteObjectRef int or Method array of bytes 2003 Universität Karlsruhe, System Architecture Group 15

16 Implementing Proxies Instead of executing the method-code a proxy forwards a message to the remote object hiding all the details between the different nodes. There is 1 proxy for each remote object for which a task holds a remote object reference Universität Karlsruhe, System Architecture Group 16

17 Implementing Dispatcher A server has 1 dispatcher and skeleton for each class representing a remote object. Dispatcher receives the request message from the communication module. It uses methodid to select the appropriate method in the skeleton, passing 2003 Universität Karlsruhe, System Architecture Group 17

18 Implementing Skeleton Each class of a remote object has a skeleton which implements the methods of the remote interface. A skeleton method unmarschals arguments in the request message and finally invokes the server method in the remote object It waits for the invocation to complete and the marshals the result together with any exceptions in a reply message to the client s proxy 2003 Universität Karlsruhe, System Architecture Group 18

19 Distributed Objects Compile-time versus Runtime-Objects Compile-time objects! Instances of a class! Interfaces can be compiled into (client-) proxies and server-side stubs (skeletons)! Disadvantage = language dependency (Java, C++, ) Runtime objects! Object adapter acting as a wrapper 2003 Universität Karlsruhe, System Architecture Group 19

20 Distributed Objects Persistent and Transient Objects! Persistent object: may exist, even if currently not mapped to an address space (task)! Not dependent on its current server, i.e. if current server exits it may store the object s state to disk! Later a new server may map the state of the persistent object from disk to its address space! Transient object: exists only as long as its server 2003 Universität Karlsruhe, System Architecture Group 20

21 Distributed Objects Object References! System wide object handles! Can be freely passed as parameters! Include! Globally unique object id! Network address of server! Protocols required to access object (opt.)! Pointer to proxy implementation (opt.) 2003 Universität Karlsruhe, System Architecture Group 21

22 Distributed Objects Binding a Client to an Object Distr_object* obj_ref; //Declare a systemwide object reference obj_ref = ; //Initialize reference to distributed object obj_ref-> do_something(); //Implicitly bind and invoke a method (a) Distr_object obj_ref; //Declare a systemwide object reference Local_object* obj_ptr; //Declare a pointer to local objects obj_ref = ; //Initialize reference to distributed object obj_ptr = bind(obj_ref); //Explicitly bind and obtain a pointer //to the local proxy obj_ptr -> do_something(); //Invoke a method on the local proxy (b) a) An example with implicit binding using only global references b) An example with explicit binding using global and local references 2003 Universität Karlsruhe, System Architecture Group 22

23 Distributed Objects Parameter Passing! Situation when passing an object by reference or by value 2003 Universität Karlsruhe, System Architecture Group 23

24 Examples! DCE Remote Objects! Java RMI! Events and Notification! 2003 Universität Karlsruhe, System Architecture Group 24

25 DCE Distributed-Object Model Remote Object Invocation 2-19 a) Distributed dynamic (private) objects b) Distributed named (shared) objects 2003 Universität Karlsruhe, System Architecture Group 25

26 Remote Object Invocation Java RMI! Distributed objects provided at language level! Goal is to achieve transparency! Give up transparency if impractical or inefficient! Remote objects = only allowed form of distributed objects! Local versus remote objects differences when! Cloning! Cloning the actual object only, not the proxies! Synchronizing! Synchronized methods! Only proxy synchronization is allowed See: Universität Karlsruhe, System Architecture Group 26

27 Remote Interfaces Shape, ShapeList import java.rmi.*; import java.util.vector; public interface Shape extends Remote { int getversion() throws RemoteException; GraphicalObject getallstate() throws RemoteException; 1 } public interface ShapeList extends Remote { Shape newshape(graphicalobject g) throws RemoteException; 2 Vector allshapes() throws RemoteException; int getversion() throws RemoteException; } 2003 Universität Karlsruhe, System Architecture Group 27

28 Naming class of Java RMIregistry void rebind (String name, Remote obj) This method is used by a server to register the identifier of a remote object by name, as shown in Figure 15.13, line 3. void bind (String name, Remote obj) This method can alternatively be used by a server to register a remote object by name, but if the name is already bound to a remote object reference an exception is thrown. void unbind (String name, Remote obj) This method removes a binding. Remote lookup(string name) This method is used by clients to look up a remote object by name, as shown in Figure line 1. A remote object reference is returned. String [] list() This method returns an array of Strings containing the names bound in the registry Universität Karlsruhe, System Architecture Group 28

29 Main of class ShapeListServer import java.rmi.*; public class ShapeListServer{ public static void main(string args[]){ System.setSecurityManager(new RMISecurityManager()); try{ ShapeList ashapelist = new ShapeListServant(); 1 Naming.rebind("Shape List", ashapelist ); 2 System.out.println("ShapeList server ready"); }catch(exception e) { System.out.println( "ShapeList server main " + e.getmessage());} } } 2003 Universität Karlsruhe, System Architecture Group 29

30 ShapeListServant implements interface ShapeList import java.rmi.*; import java.rmi.server.unicastremoteobject; import java.util.vector; public class ShapeListServant extends UnicastRemoteObject implements ShapeList { private Vector thelist; // contains the list of Shapes 1 private int version; public ShapeListServant()throws RemoteException{...} public Shape newshape(graphicalobject g) throws RemoteException { 2 version++; Shape s = new ShapeServant( g, version); 3 thelist.addelement(s); return s; } public Vector allshapes()throws RemoteException{...} public int getversion() throws RemoteException {... } } 2003 Universität Karlsruhe, System Architecture Group 30

31 Java Client of ShapeList import java.rmi.*; import java.rmi.server.*; import java.util.vector; public class ShapeListClient{ public static void main(string args[]){ System.setSecurityManager(new RMISecurityManager()); ShapeList ashapelist = null; } try{ ashapelist = (ShapeList)Naming.lookup ("//bruno.shapelist"); 1 Vector slist = ashapelist.allshapes(); 2 } catch(remoteexception e) {System.out.println(e.getMessage()); }catch(exception e) {System.out.println("Client: " + e.getmessage());} } 2003 Universität Karlsruhe, System Architecture Group 31

32 Classes supporting Java RMI RemoteObject RemoteServer Activatable UnicastRemoteObject <servant class> 2003 Universität Karlsruhe, System Architecture Group 32

33 Basic Communication Message Passing System *! Implements data transfer via a network! Offers communication primitives at API:! at least a send(...) and a receive(...) operation Distributed Application API Network Node 1 Node 2 Node 3 *Very simple form of a middleware 2003 Universität Karlsruhe, System Architecture Group 33

34 Basic Communication Message Passing System Functionality of a Message Passing system:! Uses standard protocols or implements new ones! Guarantees specific properties according to semantic! e.g. order of messages! Abstracts from implementation details! e.g. buffering, low-level addressing! Masks certain failures! e.g. automatic repetition after timeout! Hides heterogeneous HW and SW components improving portability 2003 Universität Karlsruhe, System Architecture Group 34

35 Overview Pragmatic Design Parameters! Length of message! Constant or fixed! Variable, but limited in size! Unlimited! Loss of messages! Not noticed! Suspected and notified! Avoided! Integrity of messages! Not noticed! Detected and notified! Automatically corrected 2003 Universität Karlsruhe, System Architecture Group 35

36 Basic Communication Message Passing (1)! Message: piece of information passed between processes/objects! Mailbox: place holding messages! Primitives:! send() place message into a mailbox, if there is space! receive() take next message from mailbox, if not empty! select() check for messages on multiple mailboxes 2003 Universität Karlsruhe, System Architecture Group 36

37 Basic Communication Message Passing (2) Relationship between mailboxes + processes! 1:1 (one mailbox per process)! 1:n! m:1! m:n many:many! Extent of buffering (in mailbox)! Limited buffering (typical)! None ( rendez-vous ) 2003 Universität Karlsruhe, System Architecture Group 37

38 Basic Communication Message Passing (3) Synchronization! Blocking receive: wait for message! Non-blocking receive: return empty! Blocking send: wait for space in mailbox! Non-blocking send: return full in case of a fill mailbox Note: Either sender or receiver must wait! 2003 Universität Karlsruhe, System Architecture Group 38

39 Message Oriented Communication Message Oriented Communication! To overcome synchronous behavior of RPC/RMI! Persistence and Synchronicity in Communication! Message-Oriented Transient Communication! Berkeley sockets! MPI! Message-Oriented Persistent Communication! Message-Queuing Model! General Architecture of a Message-Queuing System! Message Brokers 2003 Universität Karlsruhe, System Architecture Group 39

40 Message Oriented Communication Orthogonal Design Parameters! Synchronous versus asynchronous! Placing the message buffers! Persistent versus transient communication! Addressing the communicating instances! 2003 Universität Karlsruhe, System Architecture Group 40

41 Message Oriented Communication Persistence in Communication (1)! Organization of a communication system in which hosts are connected through communication servers via a network 2003 Universität Karlsruhe, System Architecture Group 41

42 Message Oriented Communication Persistence in Communication (2)! Persistent communication of letters back in the ancient days of the Pony Express Universität Karlsruhe, System Architecture Group 42

43 Message Oriented Communication Communication Models! Persistent versus Transient! Persistent messages stored as long as necessary by the communication system (e.g. )! Transient messages are discarded when they cannot be delivered (e.g. TCP/IP)! Synchronous versus Asynchronous! Asynchronous implies sender proceeds as soon as it sends the message, i.e. no blocking! Synchronous implies sender blocks until the receiving host buffers the message 2003 Universität Karlsruhe, System Architecture Group 43

44 Message Oriented Communication Persistence and Synchronicity (1) a) Persistent asynchronous communication b) Persistent synchronous communication 2003 Universität Karlsruhe, System Architecture Group 44

45 Message Oriented Communication Persistence and Synchronicity (2) c) Transient asynchronous communication d) Receipt-based transient synchronous communication 2003 Universität Karlsruhe, System Architecture Group 45

46 Message Oriented Communication Persistence and Synchronicity (3) e) Delivery-based transient synchronous communication at message delivery f) Response-based transient synchronous communication 2003 Universität Karlsruhe, System Architecture Group 46

47 Basic Communication Example: UNIX Sockets and Ports! Communication endpoint! Identified by IP address + port number! Many-to-many relationship with processes! TCP, UDP, IP multicast, etc Universität Karlsruhe, System Architecture Group 47

48 Basic Communication UNIX Sockets (1) Primitive Socket Bind Listen Accept Connect Send Receive Select Close Meaning Create a new communication endpoint Attach a local address to a socket Announce willingness to accept connections Block caller until a connection request arrives Actively attempt to establish a connection Send some data over the connection Receive some data over the connection Check which sockets have data Release the connection! Socket primitives 2003 Universität Karlsruhe, System Architecture Group 48

49 UNIX Sockets (2)! Connection-oriented communication pattern using sockets Universität Karlsruhe, System Architecture Group 49

50 UNIX Communication Primitives Send! send a message to a socket! blocking or non-blocking Receive! receive a message on a socket! blocking or non-blocking Select (Poll)! check for events on a set of ports! blocking or non-blocking 2003 Universität Karlsruhe, System Architecture Group 50

51 Message Oriented Communication Berkeley Sockets (1) Primitive Socket Bind Listen Accept Connect Send Receive Close Meaning Create a new communication endpoint Attach a local address to a socket Announce willingness to accept connections Block caller until a connection request arrives Actively attempt to establish a connection Send some data over the connection Receive some data over the connection Release the connection! Socket primitives for TCP/IP Universität Karlsruhe, System Architecture Group 51

52 Message Oriented Communication Create a new endpoint Berkeley Sockets (2) Associate endpoint Reserve buffers Block waiting for requests Automatic binding after connection! Connection-oriented communication pattern using sockets Universität Karlsruhe, System Architecture Group 52

53 Message Oriented Communication Message Passing Interface (1)! Overcome disadvantages of sockets:! Wrong level of abstraction being implemented at a too low level with only very primitive operations! Designed for to communicate across networks using general-purpose protocol stacks such as TCP/IP Not well suited for high-speed interconnection networks used in COW* (Myrinet) or MPPs 2003 Universität Karlsruhe, System Architecture Group 53

54 Message Oriented Communication Message Passing Interface (2) Assumptions: 1. Communication only within a group of processes 2. Each group has a unique identifier 3. Each process in a group has a (local) identifier (groupid,processid) uniquely identifies source or target of a message 4. Support diverse forms of buffering and synchronization (over 100 functions) 2003 Universität Karlsruhe, System Architecture Group 54

55 Message-Passing Interface (3) Message Oriented Communication Primitive MPI_bsend MPI_send MPI_ssend MPI_sendrecv MPI_isend MPI_issend MPI_recv MPI_irecv Meaning Append outgoing message to a local send buffer Send a message and wait until copied to local or remote buffer Send a message and wait until receipt starts Send a message and wait for reply Pass reference to outgoing message, and continue Pass reference to outgoing message, and wait until receipt starts Receive a message; block if there are none Check if there is an incoming message, but do not block! The most intuitive message-passing primitives of MPI 2003 Universität Karlsruhe, System Architecture Group 55

56 Message Oriented Communication Persistent Communication! Application communicate by inserting messages in specific queues! Loosely coupled communication! Support for! persistent asynchronous communication! Larger message transfer (e.g. systems) 2003 Universität Karlsruhe, System Architecture Group 56

57 Message-Queuing Model (1) Message Oriented Communication! 4 combinations for loosely-coupled communications using queues Universität Karlsruhe, System Architecture Group 57

58 Message-Queuing Model (2) Message Oriented Communication Primitive Put Get Poll Notify Meaning Append a message to a specified queue Block until the specified queue is nonempty, and remove the first message Check a specified queue for messages, and remove the first. Never block. Install a handler to be called when a message is put into the specified queue.! Basic interface to a queue in a message-queuing system Universität Karlsruhe, System Architecture Group 58

59 Message Oriented Communication Architecture of a Message-Queuing System (1)! The relationship between queue-level addressing and network-level addressing Universität Karlsruhe, System Architecture Group 59

60 Message Oriented Communication Architecture of a Message-Queuing System (2) 2-29! General organization of a message-queuing system with routers Universität Karlsruhe, System Architecture Group 60

61 Message Oriented Communication Message Brokers! General organization of a message broker in a message-queuing system Universität Karlsruhe, System Architecture Group 61

62 Example: IBM MQSeries Message Oriented Communication! Organization of IBM's MQSeries message-queuing system 2003 Universität Karlsruhe, System Architecture Group 62

63 Message Oriented Communication Channels Attribute Transport type FIFO delivery Message length Setup retry count Delivery retries Description Determines the transport protocol to be used Indicates that messages are to be delivered in the order they are sent Maximum length of a single message Specifies maximum number of retries to start up the remote MCA Maximum times MCA will try to put received message into queue! Attributes associated with message channel agents 2003 Universität Karlsruhe, System Architecture Group 63

64 Message Oriented Communication Message Transfer (1)! General organization of an MQSeries queuing network using routing tables and aliases Universität Karlsruhe, System Architecture Group 64

65 Message Oriented Communication Message Transfer (2) Primitive MQopen MQclose MQput MQget Description Open a (possibly remote) queue Close a queue Put a message into an opened queue Get a message from a (local) queue! Primitives available in an IBM MQSeries MQI 2003 Universität Karlsruhe, System Architecture Group 65

66 Simple Mail Transfer Protocol (SMTP) Message Oriented Communication! Processes! User agents (mail readers)! Eudora, pine, elm, outlook, messenger! Mail servers! Store messages! SMTP! Uses TCP/IP! Uses DNS! Client-to-server protocols! Pop (post office protocol)! Imap (internet mail access protocol) 2003 Universität Karlsruhe, System Architecture Group 66

67 Message Oriented Communication User agent User agent User agent SMTP Mail server SMTP Mail server User agent User agent User agent Mail server User agent User agent User mailbox Queue of outgoing messages 2003 Universität Karlsruhe, System Architecture Group 67

68 Message Oriented Communication Network News Protocol News client 1.1 News client 1.2 News client 3.1 News client 1.3 News server 1 News server 3 News client 3.2 News client 2.1 News client 2.2 News server 2 News server 4 Network News Reader Protocol (NNRP) Network News Transport Protocol (NNTP) News client 4.1 News client 4.2 News client 4.4 Uses TCP Servers flood fill their peers with new postings 2003 Universität Karlsruhe, System Architecture Group 68

69 Distributed Objects Event & Notifications A different IPC style! Interaction is specified in terms of topics, rather than senders and receivers! Senders and receivers need not be aware of each other! Asynchronous! Loosely coupling! Events cause changes in the state of an object! Notifications asynchronously inform other interested objects of the occurrence of events 2003 Universität Karlsruhe, System Architecture Group 69

70 Example: Dealing Room Systems Dealer s computer External source Dealer s computer Dealer Notification Notification Dealer Notification Information provider Notification Dealer s computer Dealer Notification Notification Notification Information provider External source Notification Dealer s computer Notification Notification Dealer 2003 Universität Karlsruhe, System Architecture Group 70

71 Distributed Objects Event Notification (2) Publish-subscribe! Objects publish events of certain types (on certain topics)! Objects subscribe to the event types (topics) they are interested in! Systems delivers notifications of events to all subscribed objects 2003 Universität Karlsruhe, System Architecture Group 71

72 Distributed Objects Event Notification (3) In general, events have! a type! type-specific attributes Subscriptions are in terms of! event type! optional predicate on the value of event attributes 2003 Universität Karlsruhe, System Architecture Group 72

73 Distributed Objects Event Notification (4) Example: distributed multi-player game:! Object move in game space, publish events that carry object s position as attribute! Object are interested in events generated by object within their visible range, subscribe to event with appropriate position attributes 2003 Universität Karlsruhe, System Architecture Group 73

74 Distributed Event Notification Architecture object of interest Event service subscriber 1. notification object of interest observer subscriber 2. notification notification object of interest observer subscriber 3. notification 2003 Universität Karlsruhe, System Architecture Group 74

75 Distributed Objects Event Notification System Implementation:! Centralized database of events and subscribers! Distributed ENS:! General Approach: spanning tree connects publishers and subscribers 1. One tree per event type, subscribers filter based on predicates, or 2. Subscriptions propagated up the tree, interior nodes filter as appropriate. Could rearrange tree for efficiency (but difficult without global info!) 2003 Universität Karlsruhe, System Architecture Group 75

76 Stream Oriented Communication Stream Oriented Communication! Support for continuous media that have temporal requirements! Asynchronous transmission! Ordering is important (e.g. file transfer)! Synchronous transmission! Ordering and max delay is important! Isochronous transmission! Man and min delay is important (i.e. bounded delay or jitter)! Simple versus complex streams! Relationship between substreams are also timedependent (consider synchronizing audio and video for a movie) 2003 Universität Karlsruhe, System Architecture Group 76

77 Stream Oriented Communication Data Stream (1)! Setting up a stream between 2 tasks across a network 2003 Universität Karlsruhe, System Architecture Group 77

78 Stream Oriented Communication Data Stream (2) ! Setting up a stream directly between two devices 2003 Universität Karlsruhe, System Architecture Group 78

79 Stream Oriented Communication Data Stream (3)! An example of multicasting a stream to several receivers Universität Karlsruhe, System Architecture Group 79

80 Stream Oriented Communication Specifying QoS (1) Characteristics of the Input Service Required maximum data unit size (bytes) Token bucket rate (bytes/sec) Toke bucket size (bytes) Maximum transmission rate (bytes/sec) Loss sensitivity (bytes) Loss interval (µsec) Burst loss sensitivity (data units) Minimum delay noticed (µsec) Maximum delay variation (µsec) Quality of guarantee! A flow specification 2003 Universität Karlsruhe, System Architecture Group 80

81 Stream Oriented Communication Specifying QoS (2)! The principle of a token bucket algorithm 2003 Universität Karlsruhe, System Architecture Group 81

82 Stream Oriented Communication Setting Up a Stream! The basic organization of RSVP for resource reservation in a distributed system Universität Karlsruhe, System Architecture Group 82

83 Stream Oriented Communication Synchronization Mechanisms (1)! Principle of explicit synchronization on the level data units 2003 Universität Karlsruhe, System Architecture Group 83

84 Stream Oriented Communication Synchronization Mechanisms (2)! Principle of synchronization as supported by high-level interfaces Universität Karlsruhe, System Architecture Group 84