CONCURRENT PROGRAMMING Lecture 5
Last lecture TCP I/O RIO: buffered I/O. Unix IO Question: difference between send() and write()? MSG_DONTWAIT, MSG_NOSIGNAL 2
Concurrent Programming is Hard! The human mind tends to be sequential The notion of time is often misleading Thinking about all possible sequences of events in a computer system is at least error prone and frequen tly impossible 3
Concurrent Programming is Hard! Classical problem classes of concurrent programs: Races: outcome depends on arbitrary scheduling decisio ns elsewhere in the system Example: who gets the last seat on the airplane? Deadlock: improper resource allocation prevents forwar d progress Example: traffic gridlock Livelock / Starvation / Fairness: external events and/or s ystem scheduling decisions can prevent sub-task progre ss Example: people always jump in front of you in line 4
Iterative Echo Server Client socket Server socket bind open_listenfd open_clientfd connect Connection request listen accept Client / Server Session rio_writen rio_readlineb rio_readlineb rio_writen Await connection request from next client close EOF rio_readlineb close 5
Iterative Servers Iterative servers process one request at a time client 1 server client 2 connect write call read ret read accept read write connect write call read close close accept Wait for Client 1 read write ret read 6
Where Does Second Client Block? Second client attempts to conne ct to iterative server Client open_clientfd socket connect rio_writen Connection request Call to connect returns Even though connection not yet accepted Server side TCP manager que ues request Feature known as TCP listen backlog Call to rio_writen returns Server side TCP manager buff ers input data Call to rio_readlineb blocks Server hasn t written anything for it to read yet. rio_readlineb 7
Fundamental Flaw of Iterative Servers User goes out to lunch connect write call read ret read Client 1 blocks waiting for user to type in data client 1 server client 2 accept read write Server blocks waiting for data from Client 1 connect write call read Client 2 blocks waiting to read from server Solution: use concurrent servers instead Concurrent servers use multiple concurrent flows to serve multip le clients at the same time 8
Creating Concurrent Flows Allow server to handle multiple clients simultaneously 1. Processes Kernel automatically interleaves multiple logical flows Each flow has its own private address space 2. Threads Kernel automatically interleaves multiple logical flows Each flow shares the same address space 3. I/O multiplexing with select() Programmer manually interleaves multiple logical flows All flows share the same address space Relies on lower-level system abstractions 9
Concurrent Servers: Multiple Processes Spawn separate process for each client client 1 server client 2 call connect ret connect call fgets User goes out to lunch Client 1 blocks waiting for user to type in data child 1 call read fork... call accept ret accept fork call accept ret accept child 2 call read write close call connect ret connect call fgets write call read end read close 10
Review: Iterative Echo Server int main(int argc, char **argv) { int listenfd, connfd; int port = atoi(argv[1]); struct sockaddr_in clientaddr; int clientlen = sizeof(clientaddr); } listenfd = Open_listenfd(port); while (1) { connfd = Accept(listenfd, (SA *)&clientaddr, &clientlen); echo(connfd); Close(connfd); } exit(0); Accept a connection request Handle echo requests until client terminates 11
Process-Based Concurrent Server int main(int argc, char **argv) { int listenfd, connfd; int port = atoi(argv[1]); struct sockaddr_in clientaddr; int clientlen=sizeof(clientaddr); } Fork separate process for each client Does not allow any communication between different client handlers Signal(SIGCHLD, sigchld_handler); listenfd = Open_listenfd(port); while (1) { connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); if (Fork() == 0) { Close(listenfd); /* Child closes its listening socket */ echo(connfd); /* Child services client */ Close(connfd); /* Child closes connection with client */ exit(0); /* Child exits */ } Close(connfd); /* Parent closes connected socket (important!) */ } 12
Process-Based Concurrent Server (cont) void sigchld_handler(int sig) { while (waitpid(-1, 0, WNOHANG) > 0) ; return; } Reap all zombie children 13
Process Execution Model Connection Requests Client 1 data Client 1 Server Process Listening Server Process Client 2 Server Process Client 2 data Each client handled by independent process No shared state between them Both parent & child have copies of listenfd and connfd Parent must close connfd Child must close listenfd 14
Concurrent Server: accept Illustrated Client clientfd listenfd(3) Server 1. Server blocks in accept, waiting for connection request on listening descriptor listenfd Client Connection request clientfd listenfd(3) Server 2. Client makes connection request by calling and blocking in connect Client clientfd listenfd(3) Server Child connfd(4) Server 3. Server returns connfd from accept. Forks child to handle client. Client returns from connect. Connection is now established between clientfd and connfd 15
Implementation Must-dos With Process-Based Designs Listening server process must reap zombie children to avoid fatal memory leak Listening server process must close its copy of con nfd Kernel keeps reference for each socket/open file After fork, refcnt(connfd) = 2 Connection will not be closed until refcnt(connfd) == 0 16
View from Server s TCP Manager Client 1 Client 2 Server srv>./echoserverp 15213 cl1>./echoclient greatwhite.ics.cs.cmu.edu 15213 srv> connected to (128.2.192.34), port 50437 cl2>./echoclient greatwhite.ics.cs.cmu.edu 15213 srv> connected to (128.2.205.225), port 41656 Connection Host Port Host Port Listening --- --- 128.2.220.10 15213 cl1 128.2.192.34 50437 128.2.220.10 15213 cl2 128.2.205.225 41656 128.2.220.10 15213 17
Pros and Cons of Process-Based Designs + Handle multiple connections concurrently + Clean sharing model file tables (yes) global variables (no) + Simple and straightforward Additional overhead for process control Nontrivial to share data between processes Requires IPC (interprocess communication) mechanisms FIFO s (named pipes), System V shared memory and semaphores 18
#2) Event-Based Concurrent Servers Using I/O Multiplex ing Use library functions to construct scheduler within sing le process Server maintains set of active connections Array of connfd s Repeat: Determine which connections have pending inputs If listenfd has input, then accept connection Add new connfd to array Service all connfd s with pending inputs 19
Adding Concurency: Step One Start with allowing address re-use int sock, opts; sock = socket( ); // getting the current options setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opts, sizeof(opts)); Then we set the socket to non-blocking // getting current options if (0 > (opts = fcntl(sock, F_GETFL))) printf( Error \n ); // modifying and applying opts = (opts O_NONBLOCK); if (fcntl(sock, F_SETFL, opts)) printf( Error \n ); bind( ); 20
Adding Concurrency: Step Two Monitor sockets with select() int select(int maxfd, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, const struct timespec *timeout); So what s an fd_set? Bit vector with FD_SETSIZE bits maxfd Max file descriptor + 1 readfs Bit vector of read descriptors to monitor writefds Bit vector of write descriptors to monitor exceptfds Read the manpage, set to NULL timeout How long to wait with no activity before retur ning, NULL for eternity 21
How does the code change? if (listen(sockfd, 5) < 0) { // listen for incoming connections printf( Error listening\n ); init_pool(sockfd, &pool); while (1) { pool.ready_set = &pool.read_set; pool.nready = select(pool.maxfd+1, &pool.ready_set, &pool.write_set, NULL, NULL); if (FD_ISSET(sockfd, &pool.ready_set)) { if (0 > (isock = accept(sockfd, ))) printf( Error accepting\n ); add_client(isock, &caddr, &pool); } check_clients(&pool); } // close it up down here 22
What was pool? A struct something like this: typedef struct s_pool { int maxfd; // largest descriptor in sets fd_set read_set; // all active read descriptors fd_set write_set; // all active write descriptors fd_set ready_set; // descriptors ready for reading int nready; // return of select() int maxi; /* highwater index into client array */ int clientfd[fd_setsize]; /* set of active descriptors */ rio_t clientrio[fd_setsize]; /* set of active read buffers */ } pool; 23
void init_pool(int sockfd, pool * p); { int i; p->maxfd = -1; for (i=0;i<fd_setsize;i++) p->clientfd[i]=-1; p->maxfd = sockfd; FD_ZERO(&p->read_set); FD_SET(listenfd, &p->read_set); } // close it up down here 24
void add_client(int connfd, pool *p) { int i; p->nready--; for (i = 0; i < FD_SETSIZE; i++) /* Find an available slot */ if (p->clientfd[i] < 0) { /* Add connected descriptor to the pool */ p->clientfd[i] = connfd; Rio_readinitb(&p->clientrio[i], connfd); /* Add the descriptor to descriptor set */ FD_SET(connfd, &p->read_set); /* Update max descriptor and pool highwater mark */ if (connfd > p->maxfd) p->maxfd = connfd; if (i > p->maxi) p->maxi = i; break; } if (i == FD_SETSIZE) /* Couldn't find an empty slot */ app_error("add_client error: Too many clients"); }// close it up down here 25
void check_clients(pool *p) { int i, connfd, n; char buf[maxline]; rio_t rio; for (i = 0; (i <= p->maxi) && (p->nready > 0); i++) { connfd = p->clientfd[i]; rio = p->clientrio[i]; /* If the descriptor is ready, echo a text line from it */ if ((connfd > 0) && (FD_ISSET(connfd, &p->ready_set))) { p->nready--; if ((n = Rio_readlineb(&rio, buf, MAXLINE))!= 0) { byte_cnt += n; //line:conc:echoservers:beginecho printf("server received %d (%d total) bytes on fd %d\n", n, byte_cnt, connfd); Rio_writen(connfd, buf, n); //line:conc:echoservers:endecho } } } } /* EOF detected, remove descriptor from pool */ else { Close(connfd); //line:conc:echoservers:closeconnfd FD_CLR(connfd, &p->read_set); //line:conc:echoservers:beginremove p->clientfd[i] = -1; //line:conc:echoservers:endremove } 26
So what about bit vectors? void FD_ZERO(fd_set *fdset); Clears all the bits void FD_SET(int fd, fd_set *fdset); Sets the bit for fd void FD_CLR(int fd, fd_set *fdset); Clears the bit for fd int FD_ISSET(int fd, fd_set *fdset); Checks whether fd s bit is set 27
What about checking clients? The code only tests for new incoming connections But we have many more to test! Store all your client file descriptors In pool is a good idea! Several scenarios Clients are sending us data We may have pending data to write in a buffer Keep the while(1) thin Delegate specifics to functions that access the appropriate d ata Keep it orthogonal! 28
Back to that lifecycle Client(s) socket() Server socket() bind() listen() connect() Connection Request select() FD_ISSET(sfd) accept() write( ) read() close( ) check_clients() main loop Client / Server Session(s) EOF read() write() read() close() 29
I/O Multiplexed Event Processing Active Descriptors listenfd = 3 Pending Inputs listenfd = 3 Read clientfd clientfd 0 10 10 1 7 Active 7 2 4 4 3 4-1 -1 Inactive -1-1 5 6 7 8 9 12 5-1 -1-1 Active Never Used 12 5-1 -1-1 30
Time out? int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout); 31
Pros and Cons of I/O Multiplexing + One logical control flow. + Can single-step with a debugger. + No process or thread control overhead. Design of choice for high-performance Web servers and search engines. Significantly more complex to code than process- o r thread-based designs. Hard to provide fine-grained concurrency E.g., our example will hang up with partial lines. Cannot take advantage of multi-core Single thread of control 32