Dr Markus Hagenbuchner markus@uow.edu.au CSCI319 SIM Distributed Systems Chapter 4 - Communication CSCI319 Chapter 4 Page: 1
Communication Lecture notes based on the textbook by Tannenbaum Study objectives: 1. Understand the role of communication in DS 2. Explain message oriented communication, stream oriented communication, and multi-casting. 3. Understand the role and limitations of RPC. 4. Explain role and mechanisms of message queuing and message broker systems. 5. Explain QoS in stream oriented communication. 6. Understand communication in wireless mesh networks CSCI319 Chapter 4 Page: 2
Content Communication protocols Types of communication Remote Procedure Calls (RPC) Streaming Multicasting Synchronization Quality of service Introduction to Wireless Mesh Networks CSCI319 Chapter 4 Page: 3
Layered Protocols (1) Layers, interfaces, and protocols in the OSI model: CSCI319 Chapter 4 Page: 4
Layered Protocols (2) Typically, each layer adds a header to the message containing information relevant to the layer. Example, a typical message as it appears on the network: CSCI319 Chapter 4 Page: 5
Interactive slide What does the acronym OSI stand for? Open System Interconnect, provides communication standards. Give examples to what do the following layers in the OSI model typically refer to: Application: FTP, HTTP (application side protocol determines what to do with an incoming message) Transport: Transport Control Protocol (TCP) adds some flow control (sequencing) Network: Network/Internet Protocol (IP) contains information about unique sender and receiver Data link: Frames for error detection. Physical: Refers to wire, line speed, voltage, plugs, etc. CSCI319 Chapter 4 Page: 6
Interactive slide What does the acronym OSI stand for? Open System Interconnect, provides communication standards. Give examples to what do the following layers in the OSI model typically refer to: Application: FTP, HTTP (application side protocol determines what to do with an incoming message) Transport: Transport Control Protocol (TCP) adds some flow control (sequencing) Network: Network/Internet Protocol (IP) contains information about unique sender and receiver Data link: Frames for error detection. Physical: Refers to wire, line speed, voltage, plugs, etc. CSCI319 Chapter 4 Page: 7
Middleware Protocols The standard OSI model does not provision a layer for the middleware protocol. The presentation layer and the session layer are hardly ever used in practice (their corresponding headers are of zero-size). Distributed system leave the presentation layer empty, and use of the session layer for the injection of middleware layer protocols. Hence, the OSI model for distributed systems is defined more specifically as follows: CSCI319 Chapter 4 Page: 8
Middleware Protocols An adapted reference model for Distributed Systems. CSCI319 Chapter 4 Page: 9
Interactive slide What is the role of the middleware layer in the Open System Interconnection (OSI) model? Offers high level communication services and is placed as an independent protocol layer between transport and application layer. Helps to achieve transparency (from the application point of view). Allows the realization of RPC CSCI319 Chapter 4 Page: 10
Interactive slide What is the role of the middleware layer in the Open System Interconnection (OSI) model? Offers high level communication services and is placed as an independent protocol layer between transport and application layer. Helps to achieve transparency (from the application point of view). Replaces the session and presentation layer which are rarely ever used (if needed, the middleware layer can simulate these layers in a transparent fashion). Allows the realization of RPC CSCI319 Chapter 4 Page: 11
RPC Many older distributed systems have been based on explicit message exchange between processes (i.e. using send and receive). Transparency is not achieved An idea was introduced to allow calling of procedures located on other machines. The method is commonly known as Remote Procedure Calls (RPC) CSCI319 Chapter 4 Page: 12 of 47
RPC Local procedure call: Parameters are passed via a stack. Remote procedure call: Parameters are passed via a message over a network. In both cases: the receiver needs to know the type, format, and purpose of the parameters. Main difference: With RPC, the receiver may be of a differing hardware architecture. Type conversion may be necessary. Example: Assume that a function add() is defined to accept two integer parameters such as in add(int i, int j). Then the differences between calling a local procedure and RPC can be shown as follows: CSCI319 Chapter 4 Page: 13 of 47
Conventional Procedure Call Parameter passing when calling a local procedure add(int, int). Left, the stack before the call to add(), right the stack while the called procedure is active. Main program s local variables Main program s local variables int i int j Return address Local variables of add CSCI319 Chapter 4 Page: 14 of 47
Remote Procedure Calls (1) In general, a remote procedure call occurs in the following steps: 1. The client procedure calls the client stub in the normal way. 2. The client stub builds a message and calls the local operating system. 3. The client s OS sends the message to the remote OS. 4. The remote OS gives the message to the server stub. 5. The server stub unpacks the parameters and calls the server. Continued CSCI319 Chapter 4 Page: 15 of 47
Remote Procedure Calls (2) A remote procedure call occurs in the following steps (continued): 6. The server does the work and returns the result to the stub. 7. The server stub packs it in a message and calls its local OS. 8. The server s OS sends the message to the client s OS. 9. The client s OS gives the message to the client stub. 10. The stub unpacks the result and returns to the client. CSCI319 Chapter 4 Page: 16 of 47
Passing Value Parameters (1) A visualization of the steps involved in a doing a remote computation through RPC: But there is a problem in heterogeneous systems: CSCI319 Chapter 4 Page: 17 of 47
Passing Value Parameters (2) But there is a problem: The byte order can differ between architectures. This is called endianess. Example: A integer consists of 4 bytes. Say, we store the value 5 in an integer, which of the bytes in the integer will carry the value? In Big endian (i.e. Intel): the (last) forth byte. Little endian (i.e. Sun sparc): the first byte. Thus, an RPC issued by an intel based computer would send the byte sequence 0005. But a sun workstation would interpret this value as 5*2 24. Solution: Reverse byte order if endianess differ between sender/receiver. This worked for the integer but not for the string. Hence, we need to understand what needs to be reverted and what not (since we cannot ask the RPC system to guess). CSCI319 Chapter 4 Page: 18 of 47
Communication RPC is one example of process communication between two machines. In general, Process Communication can be divided into two types: (1) Message oriented (2) Stream oriented. We will have a closer look at these two. CSCI319 Chapter 4 Page: 19 of 47
Message Oriented Communication We will look at two message oriented communication and their protocols: 1. The message passing interface (MPI) and the message passing protocol. 2. Message Queuing (MQ) and the message queuing protocol. CSCI319 Chapter 4 Page: 20 of 47
The Message-Passing Interface (1) MPI is a well established protocol commonly used for communication in homogeneous systems (i.e. clusters). Some of the primitives of MPI: MPI avoids low level primitives which are irrelevant to the task of message passing -> More intuitive, more transparent CSCI319 Chapter 4 Page: 21 of 47
Message Oriented Communication However, MPI requires that both, sender and receiver are available at the time of communication. This is OK in cluster environments. But it is unsuitable in DS which are distributed over a larger area. Message Queuing allows communication in loosely coupled systems. Through a buffering mechanism within the middleware layer. CSCI319 Chapter 4 Page: 22 of 47
Message-Queuing Model (1) With message queuing, there are four combinations for loosely-coupled communications using queues. CSCI319 Chapter 4 Page: 23 of 47
General Architecture of a Message- Queuing System (1) There can be multiple queues in a MQ system. Messages may be passed via a set of routers. Thus, the path of a message sent through a queue must be defined: Routing tables CSCI319 Chapter 4 Page: 24 of 47
Message Transfer in MQ systems The general organization of a message queuing network using routing tables and aliases. CSCI319 Chapter 4 Page: 25
IBM s WebSphere Message-Queuing System General organization of IBM s message-queuing system: CSCI319 Chapter 4 Page: 26
Channels IBMs message-queuing systems uses channel agents (MCA). Channels avoid the need of re-establishing a connection when sending several messages through a queue. The collective of all channels define the topology of a system. It becomes necessary to define routes to ensure that a message can find a path to any destination. This too is achieved through routing tables. CSCI319 Chapter 4 Page: 27
Message Brokers Large scale DSs are often heterogenous. Messages send be one node may not be understood by the receiving node. This can occur, for example, if sender and receiver use differing software versions. This issue can be addressed i.e. through the use of message brokers as in WebSphere. Message brokers are commonly found in message queuing systems of large enterprises. CSCI319 Chapter 4 Page: 28
Message Brokers The general organization of a message broker in a message-queuing system. CSCI319 Chapter 4 Page: 29
Message Transfer (2) We have seen that MQ systems can be powerful. Yet, the primitives required can be kept quite simple. In practice, the primitives available in the message-queuing interface are: CSCI319 Chapter 4 Page: 30
Next: Stream Oriented Communication in distributed systems CSCI319 Chapter 4 Page: 31
Data Stream A general architecture for streaming stored multimedia data over a network when using Quality of Service (QoS) module. CSCI319 Chapter 4 Page: 32
Streams and Quality of Service Properties that define Quality of Service (QoS): The required bit rate at which data should be transported. The maximum delay until a session has been set up The maximum end-to-end delay. The maximum delay variance, or jitter. The maximum round-trip delay. CSCI319 Chapter 4 Page: 33
Enforcing QoS (1) A strategy to reduce jitter is by using a buffer as is illustrated in the following: The loss of a packet can be a problem since we often can not wait for a packet to be re-sent. So what can be done to reduce the effect of packet loss? CSCI319 Chapter 4 Page: 34
Enforcing QoS (2) The effect of packet loss in (a) non interleaved transmission and (b) interleaved transmission. CSCI319 Chapter 4 Page: 35
Enforcing QoS (3) Note that on the previous slide: 1.Send refers to data sent by a multimedia server 2.Delivered refers to data as is delivered to the client-side application layer by the client-side middleware. Thus, buffering, and the recreating of the frame sequence is handled by the middleware layer. CSCI319 Chapter 4 Page: 36
Multicast Communication With Multicast communication: One sender, multiple receivers Network link between sender/receiver may run at different speeds, produce different cost, etc. Strategies for efficient message propagation? (Network) overlay construction (a structure that defines how a message travels across the network). Dissemination models CSCI319 Chapter 4 Page: 37
Overlay Construction Aim of overlay construction: Reduce transport cost Reduce time requirement. The relation between links in an overlay and actual network-level routes will be demonstrated in an example: A system consisting of four hosts, four routers, a cost (i.e. time, transport cost) associated with each network link, and network level topology. CSCI319 Chapter 4 Page: 38
Example 1 overlay construction Assume sender A multicasts a message by sending the message to each received one at a time. What would the total cost be? Cost: Cost A->B + Cost A->C + Cost A->D 9 + 53 + 57 = 119 CSCI319 Chapter 4 Page: 39
Example 2 overlay construction Sender A attaches ID of all receivers. Routers chose the connection of lowest cost to deliver the message to the first receiver. The receiver removes its own ID from message, then becomes the new sender. Repeat until no more IDs in the message. Cost? Cost: Cost A->B + Cost B->C + Cost C->D 9 + 60 + 6 = 75 CSCI319 Chapter 4 Page: 40
Overlay construction Thus, the overlay network defines the efficiency of a multi-cast system. A well designed overlay can reduce costs significantly. But what is the optimal cost? In the example, the optimal cost is: 9 + 41 + 6 = 56 In dynamic systems it is not generally possible to achieve optimal cost. In static systems it is possible to define static routing tables which can achieve optimal cost. CSCI319 Chapter 4 Page: 41
Information Dissemination Models (1) An alternative to spreading information (i.e. as found in P2P) is to spread information like an epidemic: 1. Anti-entropy propagation model Node P picks another node Q at random Subsequently exchanges updates with Q 2. Approaches to exchanging updates P only pushes its own updates to Q P only pulls in new updates from Q P and Q send updates to each other In both situations, P forgets Q with probability (i.e. 1/k), then picks another node at random. The approach in (1), and (2) does not guarantee that all nodes receive the message. CSCI319 Chapter 4 Page: 42
Wireless Mesh Networks (1) A wireless mesh network (WMN) is a communications network made up of radio nodes organized in a mesh topology. Wireless mesh networks often consist of Nodes have two functions: Generate/terminate traffic Route traffic for other nodes Therefore, MMNs are: Multihop Wireless network. Support for adhoc networking and capability of self forming, self healing and self organization. CSCI319 Chapter 4 Page: 43
Wireless Mesh Networks (2) SS = Subscriber Station; BS = BaseStation CSCI319 Chapter 4 Page: 44
Wireless Mesh Networks (3) Security enforcement and topology management is much easier to achieve in Point-to-multipoint communication. Whereas robustness and range is improved vastly in WMN. This makes mesh communication particularly attractive for distributed pervasive systems. WMN is standardized (IEEE802.16a) for WIFI networks. There is no such standard yet for 3G type of networks. CSCI319 Chapter 4 Page: 45
Wireless Mesh Networks (4) Note: Wireless mesh networks differ from Ad-hoc networks and from Wireless sensor networks. You will learn more about these differences during the tutorial. CSCI319 Chapter 4 Page: 46
Summary Standard protocols are an essential requirement. Types of communication (i.e. RPC) Synchronous vs asynchronous RPC Message distribution models. These are.? Streaming = information dissemination Quality of Service Introduced Wireless Mesh Networks CSCI319 Chapter 4 Page: 47