ICT 6544 Distributed Systems Lecture 5 Hossen Asiful Mustafa
Message Brokers Figure 4-21. The general organization of a message broker in a message-queuing system.
IBM s WebSphere Message-Queuing System Figure 4-22. General organization of IBM s message-queuing system.
Message Transfer (1) Figure 4-24. The general organization of an MQ queuing network using routing tables and aliases.
Message Transfer (2) Figure 4-25. Primitives available in the message-queuing interface.
Data Stream Figure 4-26. A general architecture for streaming stored multimedia data over a network.
Streams and Quality of Service Properties for Quality of Service: The required bit rate at which data should be transported. The maximum delay until a session has been set up The maximum end-to-end delay. The maximum delay variance, or jitter. The maximum round-trip delay.
Enforcing QoS (1) Figure 4-27. Using a buffer to reduce jitter.
Enforcing QoS (2) Figure 4-28. The effect of packet loss in (a) non interleaved transmission and (b) interleaved transmission.
Synchronization Mechanisms (1) Figure 4-29. The principle of explicit synchronization on the level data units.
Synchronization Mechanisms (2) Figure 4-30. The principle of synchronization as supported by high-level interfaces.
UNIX Network Programming
Types of Internet Sockets Different types of sockets implement different communication types (stream vs. datagram) Type of socket: stream socket connection-oriented two way communication reliable (error free), in order delivery can use the Transmission Control Protocol (TCP) e.g. telnet, ssh, http Type of socket: datagram socket connectionless, does not maintain an open connection, each packet is independent can use the User Datagram Protocol (UDP) e.g. IP telephony 13
Network Programming Programmer should consider the following: Byte Ordering Naming Addressing 14
Byte Ordering of Integers Different CPU architectures have different byte ordering Stored at little-endian computer Integer representation (2 byte) Stored at big-endian computer memory address A +1 high-order byte D3 low-order byte memory address A low-order byte F2 high-order byte 15
Byte Ordering Problem What would happen if two computers with different integer byte ordering communicate? Nothing if they do not exchange integers! But: If they exchange integers, they would get the wrong order of bytes, therefore, the wrong value! Message is: [Hello,1] Message is: [Hello,256] Message in Memory of little-endian Computer 48 45 4C 4C 6F 01 00 Message is sent Message in Memory of of big-endian Computer across Network 48 45 4C 4C 6F 01 00 16
Byte Ordering Solution There are two solutions if computers with different byte ordering system want to communicate They must know the kind of architecture of the sending computer (bad solution, it has not been implemented) Introduction of a network byte order. The functions are: uint16_t htons(uint16_t host16bitvalue) uint32_t htonl(uint32_t host32bitvalue) uint16_t ntohs(uint16_t net16bitvalue) uint32_t ntohs(uint32_t net32bitvalue) Use these function for all integers (short and long), which are sent across the network User the function for port numbers and IP addresses 17
Naming and Addressing Host name identifies a single host variable length string (e.g. www.berkeley.edu) is mapped to one or more IP addresses IP Address written as dotted octets (e.g. 10.0.0.1) 32 bits. Not a number! But often needs to be converted to a 32-bit to use. Port number identifies a process on a host 16 bit number 18
Client-Server Architecture response Client Server request Client requests service from server Server responds with sending service or error message to client 19
Simple Client-Server Example Client socket() connect() send() response request Connection establishment Data request Server socket() bind() listen() accept() recv() recv() close() Data response End-of-file notification send() recv() close() 20
Example: Client Programming Create stream socket (socket() ) Connect to server (connect() ) While still connected (loop): send message to server (send() ) receive (recv() ) data from server and process it Close TCP connection and Socket (close()) 21
socket(): Initializing Socket Getting the file descriptor int chat_sock; if ((chat_sock = socket(af_inet, SOCK_STREAM, 0)) < 0) { perror("socket"); printf("failed to create socket\n"); abort (); 1.parameter specifies protocol/address family 2.parameter specifies the socket type Other possibilities: SOCK_DGRAM 3.parameter specifies the protocol. 0 means protocol is chosen by the OS. 22
IP Address Data Structure struct sockaddr_in { short int sin_family; // Address family ; unsigned short int sin_port; // Port number struct in_addr sin_addr; // Internet address unsigned char sin_zero[8]; struct in_addr { ; unsigned long s_addr; // 4 bytes Padding of sin_zeros: struct sockaddr_in has same size as struct sockaddr 23
struct sockaddr_in sin; connect(): Making TCP Connection to Server struct hostent *host = gethostbyname (argv[1]); unsigned int server_address = *(unsigned long *) host->h_addr_list[0]; unsigned short server_port = atoi (argv[2]); memset (&sin, 0, sizeof (sin)); sin.sin_family = AF_INET; sin.sin_addr.s_addr = server_address; sin.sin_port = htons (server_port); if (connect(chat_sock, (struct sockaddr *) &sin, sizeof (sin)) < 0) { perror("connect"); printf("cannot connect to server\n"); abort(); 24
send(): Sending Packets int send_packets(char *buffer, int buffer_len) { sent_bytes = send(chat_sock, buffer, buffer_len, 0); if (send_bytes < 0) { perror ( send"); return 0; Needs socket descriptor, Buffer containing the message, and Length of the message Can also use write() 25
Receiving Packets: Separating Data in a Stream Fixed length record Fixed length record A B C D 0 1 2 3 4 5 6 7 8 9 receive buffer slide through Use records (data structures) to partition the data stream 26
Receiving Packets int receive_packets(char *buffer, int buffer_len, int *bytes_read) { int left = buffer_len - *bytes_read; received = recv(chat_sock, buffer + *bytes_read, left, 0); if (received < 0) { perror ( recv"); buffer if (received <= 0) { return close_connection(); *bytes_read += received; while (*bytes_read > RECORD_LEN) { process_packet(buffer, RECORD_LEN); *bytes_read -= RECORD_LEN; memmove(buffer, buffer + RECORD_LEN, *bytes_read); return 0; Can also use read() *bytes_read buffer_len 27
Server Programming: Simple Create stream socket (socket() ) Bind port to socket (bind() ) Listen for new client (listen() ) While (true){ accept user connection and create a new socket (accept() ) data arrives from client (recv() ) data has to be send to client (send() ) 28
bind(): Assign IP and Port struct sockaddr_in sin; struct hostent *host = gethostbyname (argv[1]); unsigned int server_address = *(unsigned long *) host->h_addr_list[0]; unsigned short server_port = atoi (argv[2]); memset (&sin, 0, sizeof (sin)); sin.sin_family = AF_INET; sin.sin_addr.s_addr = server_address; sin.sin_port = htons (server_port); if (bind(chat_sock, (struct sockaddr *) &sin, sizeof (sin)) < 0) { perror("bind"); printf("cannot bind server application to network\n"); abort(); 29
bind(): bind() tells the OS to assign a local IP address and local port number to the socket. Many applications let the OS choose an IP address. Use wildcard INADDR_ANY as local address in this case. At server, user process must call bind() to assign a port At client, bind() is not required since OS may assign available port and IP address The server will get the port number of the client through the UDP/TCP packet header Note: Each application is represented by a server port number 30
listen(): Wait for Connections int listen(int sockfd, int backlog); Puts socket in a listening state, willing to handle incoming TCP connection request. Backlog: number of TCP connections that can be queued at the socket. 31
Server Example #define MYPORT 8081 // the port users will be connecting to #define BACKLOG 10 // how many pending connections queue will hold int main(void) { int sockfd, new_fd; // listen on sockfd, new connection on new_fd struct sockaddr_in my_addr; // my address information struct sockaddr_in their_addr; // connector's address information int sin_size; if ((sockfd = socket(af_inet, SOCK_STREAM, 0)) == -1) { perror("socket"); exit(1); my_addr.sin_family = AF_INET; // host byte order my_addr.sin_port = htons(myport); // short, network byte order my_addr.sin_addr.s_addr = INADDR_ANY; // auto. filled with local IP memset(&(my_addr.sin_zero), '\0', 8); // zero the rest of the struct 32
if (bind(sockfd, (struct sockaddr *)&my_addr, sizeof(struct sockaddr)) == -1) { perror("bind"); exit(1); if (listen(sockfd, BACKLOG) == -1) { perror("listen"); exit(1); while(1) { // main accept() loop sin_size = sizeof(struct sockaddr_in); if ((new_fd = accept(sockfd, (struct sockaddr *)&their_addr, &sin_size)) == -1) { perror("accept"); continue; printf("server: got connection from %s\n", inet_ntoa(their_addr.sin_addr)); if (send(new_fd, "Hello, world!\n", 14, 0) == -1) perror("send"); close(new_fd); return 0; 33
#include <netinet/in.h> #include <sys/socket.h> Client Example #define PORT 3490 // the port client will be connecting to #define MAXDATASIZE 100 // max number of bytes we can get // at once int main(int argc, char *argv[]) { int sockfd, numbytes; char buf[maxdatasize]; struct hostent *he; struct sockaddr_in their_addr; // server's address information if (argc!= 2) { fprintf(stderr,"usage: client hostname\n"); exit(1); if ((he=gethostbyname(argv[1])) == NULL) { // get the host info perror("gethostbyname"); exit(1); if ((sockfd = socket(af_inet, SOCK_STREAM, 0)) == -1) { perror("socket"); exit(1); 34
their_addr.sin_family = AF_INET; // host byte order their_addr.sin_port = htons(port); // short, network byte order their_addr.sin_addr = *((struct in_addr *)he->h_addr); // already network byte order memset(&(their_addr.sin_zero), '\0', 8); // zero the rest of the struct if (connect(sockfd, (struct sockaddr *)&their_addr, sizeof(struct sockaddr)) == -1){ perror("connect"); exit(1); if ((numbytes=recv(sockfd, buf, MAXDATASIZE-1, 0)) == -1) { perror("recv"); exit(1); buf[numbytes] = '\0'; printf("received: %s",buf); close(sockfd); return 0; 35
I/O Blocking socket(); bind() ; listen(); While(true){ accept(); recv() ; send() ; Simple server has blocking problem Suppose 5 connections accepted. Suppose next accept() blocks. Other connections cannot send and receive. Cannot get keyboard input either. SOCK_NONBLOCK can be used to make accept() non-blocking 36
select() :I/O Multiplexing waits on multiple file descriptors and timeout returns when any file descriptor is ready to be read or written or indicate an error, or timeout exceeded advantages simple application does not consume CPU cycles while waiting disadvantages does not scale to large number of file descriptors 37
Example: Server Programming create stream socket (socket() ) Bind port to socket (bind() ) Listen for new client (listen() ) While Wait for (select() ) (depending on which file descriptors are ready) accept user connection and create a new socket (accept() ) data arrives from client (recv() ) data has to be send to client (send() ) 38
Server: Alternative Ways of Handling Many Clients Forking a new process for each client: fork() But, creating new process is expensive. Multithreaded implementation: have one thread handling each client. Thread is like a process but light-weighted. 39
Network Programmer s Mistakes byte ordering separating records in streams use of select() misinterpreting the project specification not knowing all available system calls 40
There are more System Calls Depends on communication type Datagram sockets use recvfrom() and sendto() for receiving and sending data Closing connection: close(), shutdown() Convenient functions (on UNIX) inet_aton, inet_ntoa inet_pton, inet_ntop 41
HOMEWORK Write a simple TCP client and TCP server from the where the server reverses the client data and send back. Example: client sends hello, server replies olleh. Use linux gcc library for compiling the programs and execute (you can use a ubuntu vm) You can run the client and server in the same machine in different terminal. Run the server first and then the client.