Chapter 6 I/O Multiplexing: select and poll function

abstract
Introduction I/O Models(5 ) Synchronous I/O versus Asynchronous I/O select function batch input shutdown function pselect function poll function

Introduction

TCP client is handling two inputs at the same time: standard input and a TCP socket
when the client was blocked in a call to read, the server process was killed server TCP sends FIN to the client TCP, but the client never see FIN since the client is blocked reading from standard input => the capability to tell the kernel that we want to be notified if one or more I/O conditions are ready. : I/O multiplexing (select and poll)

When:
client is handling multiple descriptors (interactive input and a network socket). Client to handle multiple sockets(rare) TCP server handles both a listening socket and its connected socket. Server handle both TCP and UDP. Server handles multiple services and multiple protocols

I/O Models
Blocking I/O nonblocking I/O I/O multiplexing(select and poll) signal driven I/O(SIGIO) asynchronous I/O(posix.1 aio_ functions)

Two distinct phases for an input operations

1. Waiting for the data to be ready 2. Copying the data from the kernel to the process

Blocking I/O
application kernel

System call
recvfrom No datagram ready
Wait for data

Process blocks in a call to recvfrom

Datagram ready copy datagram

Copy data from kernel to user

Return OK Process datagram Copy complete

nonblocking I/O
application
System call

kernel No datagram ready

EWOULDBLOCK System call

recvfrom recvfrom
Process repeatedly call recvfrom wating for an OK return (polling)

No datagram ready datagram ready copy datagram

EWOULDBLOCK

Wait for data

recvfrom

System call

Return OK Process datagram application

Copy data from kernel to user

I/O multiplexing(select and poll)

application
Process block in a call to select waiting for one of possibly many sockets to become readable Process blocks while data copied into application buffer

kernel
System call

select

No datagram ready
Wait for data

Return readable

recvfrom

System call

Datagram ready copy datagram

Copy data from kernel to user

Return OK Process datagram Copy complete

signal driven I/O(SIGIO)

application
Process continues executing

kernel
Sigaction system call

Establish SIGIO
Signal handler

Return

Wait for data

Deliver SIGIO Signal handler Process blocks while data copied into application buffer

recvfrom

System call

Datagram ready copy datagram

Return OK Process datagram Copy complete

Copy data from kernel to user

asynchronous I/O
application kernel

System call
aio_read Return
Process continues

No datagram ready
Wait for data

executing

Datagram ready copy datagram

Copy data from kernel to user

Signal handler Process datagram

Delever signal

Copy complete
Specified in aio_read

Comparison of the I/O Models

I/O signal-driven asynchronous I/O blocking nonblocking multiplexing I/O initiate check check check check check check check blocked ready initiate blocked notification initiate blocked copy data from kernel to user initiate wait for data

complete complete

ist phase handled differently, 2nd phase handled the same

blocked

complete

complete

notification

handles both phases

Synchronous I/O , Asynchronous I/O

Synchronous I/O : cause the requesting process to be blocked until that I/O operation (recvfrom) completes.(blocking, nonblocking, I/O multiplexing, signal-driven I/O) Asynchronous I/O : does not cause the requesting process to be blocked (asynchronous I/O)

Select function

Allows the process to instruct the kernel to wait for any one of multiple events to occur and to wake up the process only when one or more of these events occurs or when a specified amount of time has passed. (readable ,writable , expired time)

#include <sys/select.h> #include <sys/time.h> int select (int maxfdp1, fd_set *readset, fd_set *writeset, fd_set *exceptset, const struct timeval *); struct timeval{ long tv_sec; /* seconds */ long tv_usec; /* microseconds */ }

Condition of select function

Wait forever : return only descriptor is ready(timeval = NULL) wait up to a fixed amount of time: Do not wait at all : return immediately after checking the descriptors(timeval = 0)

wait: normally interrupt if the process catches a signal and returns from the signal handler

Readset => descriptor for checking readable writeset => descriptor for checking writable exceptset => descriptor for checking two exception conditions :arrival of out of band data for a socket :the presence of control status information to be read from the master side of a pseudo terminal

Descriptor sets

Array of integers : each bit in each integer correspond to a descriptor.

fd_set: an array of integers, with each bit in each integer corresponding to a descriptor.

Void FD_ZERO(fd_set *fdset); /* clear all bits in fdset */ Void FD_SET(int fd, fd_set *fdset); /* turn on the bit for fd in fdset */ Void FD_CLR(int fd, fd_set *fdset); /* turn off the bit for fd in fdset*/ int FD_ISSET(int fd, fd_set *fdset);/* is the bit for fd on in fdset ? */

fd_set rset;

Example of Descriptor sets function

FD_ZERO(&rset);/*all bits off : initiate*/ FD_SET(1, &rset);/*turn on bit fd 1*/ FD_SET(4, &rset); /*turn on bit fd 4*/ FD_SET(5, &rset); /*turn on bit fd 5*/

Maxfdp1 argument
specifies the number of descriptors to be tested. Its value is the maximum descriptor to be tested, plus one.(hence our name of maxfdp1)(example:fd1,2,5 => maxfdp1: 6) constant FD_SETSIZE defined by including <sys/select.h>, is the number of descriptors in the fd_set datatype.(1024)

Condition that cause a socket to be ready for select

Condition
Data to read read-half of the connection closed new connection ready for listening socket Space available for writing write-half of the connection closed Pending error TCP out-of-band data

Readable? writable? Exception?

Condition handled by select in str_cli(section5.5)

could be blocked in the call to fgets when something happened on the socket

blocks in a call to select instead, waiting for either standard input or the socket to be readable.

Condition handled by select in str_cli(section5.5)

client Data of EOF

stdin
Socket

Select for readability on either standard input or socket EOF

error

TCP

RST

data

FIN

Three conditions are handled with the socket

Peer TCP send a data,the socket becomr readable and read returns greater than 0 Peer TCP send a FIN(peer process terminates), the socket become readable and read returns 0(end-of-file) Peer TCP send a RST(peer host has crashed and rebooted), the socket become readable and returns -1 and errno contains the specific error code

Implimentation of str_cli function using select

Void str_cli(FILE *fp, int sockfd) { int maxfdp1; fd_set rset; char sendline[MAXLINE], recvline[MAXLINE]; FD_ZERO(&rset); for ( ; ; ) { FD_SET(fileno(fp), &rset); FD_SET(sockfd, &rset); maxfdp1 = max(fileno(fp), sockfd) + 1; Select(maxfdp1, &rset, NULL, NULL, NULL); Continue..

if (FD_ISSET(sockfd, &rset)) { /* socket is readable */ if (Readline(sockfd, recvline, MAXLINE) == 0) err_quit("str_cli: server terminated prematurely"); Fputs(recvline, stdout); }
if (FD_ISSET(fileno(fp), &rset)) { /* input is readable */ if (Fgets(sendline, MAXLINE, fp) == NULL) return; /* all done */ Writen(sockfd, sendline, strlen(sendline)); } }//for }//str_cli

client
time0 time1

request request

Stop and wait sends a line to the servertime2 and then waits for the reply
time3
time4

request request server

reply time5 reply time6 reply time7

server

reply

Batch input
Time 7:
request8 request7 request6 request5

reply1
Time 7:

reply2

reply3

reply4

request9 request8 reply2 reply3

request7 request6 reply4 reply5

The problem with our revised str_cli function

After the handling of an end-of-file on input, the send function returns to the main function, that is, the program is terminated. However, in batch mode, there are still other requests and replies in the pipe.

A way to close one-half of the TCP connection

send a FIN to the server, telling it we have finished sending data, but leave the socket descriptor open for reading <= shutdown function

Shutdown function

Close one half of the TCP connection (example:send FIN to server, but leave the socket descriptor open for reading)
Close function : decrements the descriptors reference count and closes the socket only if the count reaches 0, terminate both direction(reading and writing)
Shutdown function : just one of them(reading or writing)

Calling shutdown to close half of a TCP connection

client server

write write shutdown

data
data FIN Ack of data and FIN data write write close Read returns > 0 Read returns > 0

Read returns 0

Read returns > 0 Read returns > 0

Read returns 0

data FIN Ack of data and FIN

#include<sys/socket.h> int shutdown(int sockfd, int howto); /* return : 0 if OK, -1 on error */ howto argument
SHUT_RD : read-half of the connection closed SHUT_WR : write-half of the connection closed SHUT_RDWR : both closed

Str_cli function using select and shutdown

#include "unp.h" void str_cli(FILE *fp, int sockfd) { int maxfdp1, stdineof; fd_set rset; char sendline[MAXLINE], recvline[MAXLINE]; stdineof = 0; FD_ZERO(&rset); for ( ; ; ) { if (stdineof == 0) // select on standard input for readability FD_SET(fileno(fp), &rset); FD_SET(sockfd, &rset); maxfdp1 = max(fileno(fp), sockfd) + 1; Select(maxfdp1, &rset, NULL, NULL, NULL); Continue..

if (FD_ISSET(sockfd, &rset)) { /* socket is readable */ if (Readline(sockfd, recvline, MAXLINE) == 0) { if (stdineof == 1) return; /* normal termination */ else err_quit("str_cli: server terminated prematurely"); } Fputs(recvline, stdout); } if (FD_ISSET(fileno(fp), &rset)) { /* input is readable */ if (Fgets(sendline, MAXLINE, fp) == NULL) { stdineof = 1; Shutdown(sockfd, SHUT_WR); /* send FIN */ FD_CLR(fileno(fp), &rset); continue; } Writen(sockfd, sendline, strlen(sendline)); } }

TCP echo server

Rewrite the server as a single process that uses select to handle any number of clients, instead of forking one child per client.

Data structure TCP server(1)

Before first client has established a connection Client[] [0] [1] [2] -1 -1 -1 fd0 rset: 0 fd1 0 fd2 0 fd3 1

Maxfd + 1 = 4

[FD_SETSIZE -1]

-1

fd:0(stdin),1(stdout),2(stderr) fd:3 => listening socket fd

Data structure TCP server(2)

After first client connection is established Client[] [0] [1] [2] 4 -1 -1 fd0 rset: 0 fd1 fd2 fd3 fd4 0 0 1 1 Maxfd + 1 = 5

[FD_SETSIZE -1]

-1

* fd3 => listening socket fd *fd4 => client socket fd

Data structure TCP server(3)

After second client connection is established Client[] [0] [1] [2] 4 5 -1 fd0 rset: 0 fd1 fd2 fd3 fd4 fd5 0 0 1 1 1 Maxfd + 1 = 6

[FD_SETSIZE -1]

-1

* fd3 => listening socket fd * fd4 => client1 socket fd * fd5 => client2 socket fd

Data structure TCP server(4)

After first client terminates its connection Client[] [0] [1] [2] -1 5 -1 fd0 rset: 0 fd1 fd2 fd3 fd4 fd5 0 0 1 0 1 Maxfd + 1 = 6

[FD_SETSIZE -1]

-1

*Maxfd does not change * fd3 => listening socket fd * fd4 => client1 socket fd deleted
* fd5 => client2 socket fd

TCP echo server using single process

#include "unp.h"
int main(int argc, char **argv) { int i, maxi, maxfd, listenfd, connfd, sockfd; int nready, client[FD_SETSIZE]; ssize_t n; fd_set rset, allset; char line[MAXLINE]; socklen_t clilen; struct sockaddr_in cliaddr, servaddr; listenfd = Socket(AF_INET, SOCK_STREAM, 0); bzero(&servaddr, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_addr.s_addr = htonl(INADDR_ANY); servaddr.sin_port = htons(SERV_PORT); Bind(listenfd, (SA *) &servaddr, sizeof(servaddr)); Listen(listenfd, LISTENQ);

maxfd = listenfd; maxi = -1; for (i = 0; i < FD_SETSIZE; i++) client[i] = -1; FD_ZERO(&allset); FD_SET(listenfd, &allset);

/* initialize */ /* index into client[] array */ /* -1 indicates available entry */

for ( ; ; ) { rset = allset; /* structure assignment */ nready = Select(maxfd+1, &rset, NULL, NULL, NULL); if (FD_ISSET(listenfd, &rset)) { /* new client connection */ clilen = sizeof(cliaddr); connfd = Accept(listenfd, (SA *) &cliaddr, &clilen); NOTDEF printf("new client: %s, port %d\n", Inet_ntop(AF_INET, &cliaddr.sin_addr, 4, NULL), ntohs(cliaddr.sin_port));

#ifdef

#endif

for (i = 0; i < FD_SETSIZE; i++) if (client[i] < 0) { client[i] = connfd; /* save descriptor */ break; } if (i == FD_SETSIZE) err_quit("too many clients");

FD_SET(connfd, &allset); /* add new descriptor to set */ if (connfd > maxfd) maxfd = connfd; /* for select */ if (i > maxi) maxi = i; /* max index in client[] array */
if (--nready <= 0) continue; }

/* no more readable descriptors */

for (i = 0; i <= maxi; i++) { /* check all clients for data */ if ( (sockfd = client[i]) < 0) continue; if (FD_ISSET(sockfd, &rset)) { if ( (n = Readline(sockfd, line, MAXLINE)) == 0) { /*4connection closed by client */ Close(sockfd); FD_CLR(sockfd, &allset); client[i] = -1; } else Writen(sockfd, line, n);
if (--nready <= 0) break; /* no more readable descriptors */ } } } }

Denial of service attacks

If Malicious client connect to the server, send 1 byte of data(other than a newline), and then goes to sleep. =>call readline, server is blocked.

Solution use nonblocking I/O have each client serviced by a separate thread of control (spawn a process or a thread to service each client) place a timeout on the I/O operation

pselect function
#include <sys/select.h> #include <signal.h> #include <time.h> int pselect(int maxfdp1, fd_set *readset, fd_set *writeset, fd_set *exceptset, const struct timespec *timeout, const sigset_t *sigmask)
pselect function was invented by Posix.1g.

pselect function
struct timespec{ time_t tv_sec; /*seconds*/ long tv_nsec; /* nanoseconds */ sigmask => pointer to a signal mask.

Poll function
Similar to select, but provide additional information when dealing with streams devices #include <poll.h> int poll(struct pollfd *fdarray, unsigned long nfds, int timeout); /*return : count of ready descriptors, 0 on timeout, -1 on error*/

Struct pollfd{ int fd; /* descriptor to check */ short events; /* events of interest on fd */ short revents;/*events that occurred on fd*/ } specifies the conditions to be tested for a given descriptor fd events: the conditions to be tested revents:the status of that descriptor

Input events and returned revents for poll

Constant
POLLIN POLLRDNORM POLLRDBAND POLLPRI
POLLOUT POLLWRNORM POLLWRBAND POLLERR POLLHUP POLLNVAL

Input to Result from events ? revents ?

Description
Normal or priority band data can be read normal data can be read priority band data can be read high-priority data can be read normal data can be written normal data can be written priority band data can be written An error has occurred hangup has occurred descriptor is not an open file

Timeout value for poll

Specifies how long the function is to wait before returning Timeout value INFTIM 0 >0 Description Wait forever Return immediately, do not block Wait specified number of milliseconds

If we are no longer interested in particular descriptor, just set the fd member of the pollfd structure