Chapter 5

TCP Client-Server Example

A simple echo server performs the following steps:

The client reads a line of text from its standard input and writes the line to the
server.
The server reads the line from its network input and echoes the line back to
the client.
The client reads the echoed line and prints it on it standard output.

stdin
stdout

fgets
fputs

TCP
Client

writen
readline

readline
writen

TCP
Server

TCP Echo Server

int main (int argc, char **argv)
{
int listenfd, connfd;
pid_t childpid;
socklen_t clilen;
struct sockaddr_in cliaddr, servaddr;
char buff[80];
listenfd = socket(AF_INET, SOCK_STREAM, 0);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons(1234);
servaddr.sin_addr.s_addr = htonl (INADDR_ANY); //wildcard
bind(listenfd, (SA*)&servaddr, sizeof(servaddr));
listen(listenfd, LISTENQ);
for ( ; ; ) {
clilen = sizeof (cliaddr);
connfd = accept (listenfd, (SA*)&cliaddr, &clilen);
if ((childpid = fork( )) == 0) {
// child process
close (listenfd);
for ( ; ; ) {
recv (connfd , buff , sizeof(buff) , 0);
send (connfd , buff , sizeof(buff) , 0)
}
close (newsockfd);
exit(0);
}
close(connfd);
}
close (listenfd);
return(0);
}

In accept function call, where does the cliaddr variable gets the clients address
from?
From clients SYN segment
After fork, Child closes the listening socket and parent closes the connected
socket.

TCP echo Client

int main (int argc, char **argv)
{
int sockfd;
char buff[80];
struct sockaddr_in servaddr;
if (argc !=2)
err_quit (usage: a.out <IP Address of the server> );
if ((sockfd = socket(AF_INET, SOCK_STREAM,0)) < 0)
err_sys (socket error);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons(1234);
inet_pton(AF_INET,argv[1], &servaddr.sin_addr);
connect( sockfd, (SA *)servaddr, sizeof(servaddr));
for ( ; ;) {
printf ("Enter the message: ");
scanf ("%s", buff);
int len = sizeof(buff);
send(sockfd , buff , len , 0);
recv(sockfd , buff , len , 0);
printf("%s\n",buff);
}
close (sockfd);
return(0);
}

TCP Echo Server - Revisited

int main (int argc, char **argv)
{
int listenfd, connfd;
pid_t childpid;
socklen_t clilen;
struct sockaddr_in cliaddr, servaddr;
char buff[80];
listenfd = socket(AF_INET, SOCK_STREAM, 0);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons(1234);
servaddr.sin_addr.s_addr = htonl (INADDR_ANY); //wildcard
bind(listenfd, (SA*)&servaddr, sizeof(servaddr));
listen(listenfd, LISTENQ);
for ( ; ; ) {
clilen = sizeof (cliaddr);
connfd = accept (listenfd, (SA*)&cliaddr, &clilen);
if ((childpid = fork( )) == 0) {
// child process
close (listenfd);
str_echo (connfd);
exit(0);
}
close(connfd);
}
close (listenfd);
return(0);
}

void str_echo (int sockfd)

{
ssize_t n;
char line[MAXLINE];
for ( ; ; ) {
if( ( n = readline (sockfd, line, MAXLINE)) == 0)
return;
write (sockfd, line, n);
}
}

TCP echo Client - Revisited

int main (int argc, char **argv)
{
int sockfd;
char buff[80];
struct sockaddr_in servaddr;
if (argc !=2)
err_quit (usage: a.out <IP Address of the server> );
if ((sockfd = socket(AF_INET, SOCK_STREAM,0)) < 0)
err_sys (socket error);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons(1234);
inet_pton(AF_INET,argv[1], &servaddr.sin_addr);
connect( sockfd, (SA *)servaddr, sizeof(servaddr));
str_cli(stdin, sockfd);
return(0);
}

void str_cli (FILE *fp, int sockfd)

{
char sendline[MAXLINE], recvline[MAXLINE] ;
while (fgets (sendline, MAXLINE, fp) != NULL) {
//get data from fp(stdin)
write (sockfd, sendline, strlen(sendline)); //send to server
if (readline(sockfd, recvline, MAXLINE) == 0) //read whatever is echoed
from server
err_quit (server terminated);
fputs (recvline, stdout);
//whatever is read, write to stdout
}
}

Normal TCP Startup

We start the server first, and server calls socket(), bind(), listen() and then
accept(). The server gets blocked in the call to accept().
The server socket is in listening mode.
We then start the client program on the same host specifying the servers IP
Address.
The client calls socket( ), and then connect( ).
Connect( ) function initiates the TCP handshake.
After TCP three way handshake completes, connect( ) returns in the client and
accept( ) returns in the server.
Now the following will take place:
Client calls str_cli, which will block in the call to fgets( ), until the user types in
something.
When accept() returns in the server, it calls fork() and child calls str_echo.
This function calls readline, which in turn calls read, which blocks, waiting for
a line to be sent from the client.
The server parent calls accept again and blocks itself waiting for the next
client connection.
Thus now we have three processes and they all are blocked.

Server and client are run on the same machine.

$ netstat a
Proto Recv-Q Send-Q Local Address Foreign Address
(state)
Tcp
0
0
localhost.9877 localhost.1052
ESTABLISHED
Tcp
0
0
localhost.1052 localhost.9877
ESTABLISHED
Tcp
0
0
*.9877
*.*
LISTEN
The first row above is for the server-child.
Second row is for the client.
Third row is for server-parent.
Netstat command displays active TCP connections.
Netstat -a : Displays all active TCP connections and the TCP and UDP ports on
which the computer is listening.

Normal Termination
When the client types EOF(Contl-D), fgets returns a NULL pointer and the
function str_cli returns.
Str_cli function returns to the main() function and main calls exit().
When the client socket is closed by the kernel, FIN is sent to the server, to which
the TCP server responds with an ACK. This is the first half of the TCP connection
termination sequence.
At this point, the server socket is in the CLOSE_WAIT state and client socket is
FIN_WAIT_1 state.
When the server received the FIN from the client, the server child was blocked in
call to readline. And now readline returns 0. This causes str_echo function to
return to the server child main.
And the server child main terminates itself by calling exit.
As a final step, FIN from the server is sent to the client, and an ACK is received
from the client. At this point the connection is completely terminated. The client
socket enters the TIME_WAIT state.
Another part of the process termination is the SIGCHLD signal to be sent to the
parent when the server child terminates. Ideally this signal should be caught and
the process details should be removed from the process table.
But in our program, we are not catching the signal, thus by default the signal will
be ignored and the child enters the zombie state.

$ ps
PID
19130
21130
21132
21167

TT
p1
p1
p1
p1

STAT
Ss
I
Z
R+

TIME
05:08
07:08
07:28
08:00

COMMAND
ksh
./tcpserv
(tcpserv)
ps

In the output of ps command shown above:

First row represents the shell

Second row represents the server (parent)

Third row represents server (child)

Forth row is for the current command ps

Status I means ideal (parent is blocked in accept call)

Status Z means zombie coz the server child has terminated but its signal
was not handled by the server parent.

Status R means running, and + means running in foreground.

Status Ss is for the shell

A process goes in zombie state, when it has terminated but its information is
not removed from the process table by its parent proces.

Signal Handling
A signal is a notification to a process that an event has occurred. Signals are also
called software interrupts.
Signals are asynchronous, i.e. the process doesnt know ahead of time exactly
when a signal will occur.
Signals can be sent :
By one process to another process (or to itself)
By the kernel to the process.
SIGCHLD signal is the one that is send by the kernel to parent of the

terminating process.
Every signal has a disposition, which is also called the action
associated with the signal. We have three choices for the disposition:
1. We can provide a function, called the signal handler and this action is called
catching the signal.
2. We can ignore a signal by setting its disposition to SIG_IGN.
3. We can set the default disposition for a signal by setting its disposition to
SIG_DFL. The default is normally to terminate a process on the receipt of a
signal.

The function sigaction() is used to set the disposition of a signal.

The signals SIGKILL and SIGSTOP can not be caught and cannot be ignored.
The default disposition for SIGCHLD signal is to ignore it.

For setting the disposition for a signal, sigaction() function has to be called, but
this function needs a structure to be initialized and passed as argument, so a
wrapper function signal() is used.
The first argument to the signal() function is the signal number and the second
argument is the pointer to the signal handler function or constant SIG_IGN or
SIG_DFL.

Sigfunc* signal(int signo, Sigfunc *func)

{
struct sigaction act, oact; //action , old_action
act.sa_handler = func;
sigemptyset(&act.sa_mask);
act.sa_flags=0;
if(signo == SIGALRM)
act.sa_flags |= SA_INTERRUPT;
else
act.sa_flags |= SA_RESTART;
if (sigaction(signo, act, &oact) < 0)
return(SIG_ERR);
return (oact.sa_handler);
}
sa_mask contain the signals that will be blocked when our signal handler is
called.
Any signal that is blocked can not be delivered to the process.
Sa_mask is set to emptyset saying that no additional signals are blocked.
Additional options are specified using sa_flag;

If the signal that is caught is SIG_ALRM, we want to run the signal handler, but
along with that we want to interrupt the current running process.
So we set the sa_flags to SA_INTERRUPT
SIG_ALRM signal is used to notify a timer time-out. etc
If the signal is not SIG_ALRM, i.e. any other signal that is caught, then we must
make sure that after the signal handler function is executed, the current running
process restarts itself.
So for that sa_flags is set to SA_RESTART.

Signal handling:
Once a signal handler is installed, it remains installed.
While a signal handler is executing, the signal being delivered is blocked.
For blocking any other signals, sa_mask is used.
If a signal is generated one or more times while it is blocked, it is delivered only
once after the signal I unblocked. i.e by default, unix signals are NOT queued.

Handling SIGCHLD signal

The purpose of the zombie state is to maintain information about the child to be
fetched at some later stage.
The information that is stored is the process ID of the child, its termination status,
and information about resource utilization of the child (CPU time, memory, etc)
If a process terminates, and it has children in the zombie state, the parent
process ID of the zombie process is set to 1 (the init process)
The init process inherits the children and clean them up.

Whichever process calls fork(), should also make sure to clean up the child
process before terminating itself.
Zombies should not be left around, coz otherwise we may run out of space for
any new processes in the system.
To handle zombies, the parent process must catch the SIGCHLD signal and
within the handler, call wait().
#include<sys/wait.h>
pid_t wait (int *statloc);
Returns the process id of the child that has terminated.

SIGCHLD signal handler that calls wait()

void sig_chld(int signo)
{
pid_t pid;
int stat;
pid = wait(&stat);
printf (child %d terminated\n, pid);
return;
}
In our program we must include
signal(SIGCHLD, sig_chld)
to assign the signal handler to the SIGCHLD signal.

Assume that the TCP echoserver and client are currently running.
Also SIGCHLD signal has been assigned the signal handler sig_chld( )
$ ./client
Hi
client types Hi and sends to the server
Hi
echo
Hello
Hello
echo
^D
client wants to end the connection
child 12245 terminated
printf instruction of the signal handler
accept error : Interrupted system call

The parent is delivered the signal SIG_CHLD when it was blocked in accept( )
Sig_chld() function executes, but though the parent is not handling the signal, and
its accept() function blocking state was being interrupted, that is why the error
accept error : Interrupted system call

Sequence of steps of accept error : interrupted system call

1.
2.

3.

4.

We terminate the client by typing EOF (^D). The client sends a FIN to the
server child and the server child responds with an ACK.
The server child was blocked in readline() function when the client had send
FIN. That FIN in turns delivers EOF to the server childs pending readline().
And the server child terminates.
The server parent is blocked in call to accept() when the SIGCHLD signal is
delivered. The sig_chld signal handler executes, wait( ) fetches the childs PID
and termination status. The signal handler then returns.
The parent is delivered the signal SIG_CHLD when it was blocked in accept( )
Sig_chld() function executes, but though the parent is not handling the signal,
and its accept() function blocking state was being interrupted, that is why the
error
accept error : Interrupted system call

Handling Interrupted system calls

Slow system calls this term is used for those system calls that can block
forever, i.e. the system call need never return.
Eg accept(). There is no guarantee that a servers call to accept will ever return.
Other slow system calls are readline(), read and write of pipes and terminal
devices.
When a process is blocked in a slow system call and the process catches a signal
and the signal handler is executed and returns, the system call can return an
error of EINTR.
Some kernels automatically restart some interrupted system calls and in some,
we have to restart it explicitly.
Code that handles interrupted accept( ):
for( ; ; ) {
clilen = sizeof(cliaddr);
if ((connfd = accept(listenfd, (SA)&cliaddr, &clilen)) < 0) {
if (errno == EINTR)
continue;
else
err_sys(accept error);
}

#include <sys/wait.h>
pid_t wait(int *statloc);
pid_t waitpid (pid_t pid, int *statloc, int options);
Both functions return process id of the terminated child if OK, -1 on error.
Both functions return 2 values:
Process ID of the terminated child process.
Termination status of the child is returned through the statloc pointer.
There are three termination status :
Child terminated normally
Was killed by a signal
Job-control stopped
If there are no terminated children for the process calling wait(), but the process
has one or more children still executing, wait() blocks until the first of the existing
children terminate.
Waitpid() gives us more control over which process to wait for. First argument to
waitpid() is pid, that lets us specify the process for which we want to wait for.
-1 in place of pid says to wait for the first of our children to terminate.
Options argument lets us specify additional options.

Difference between wait() and waitpid() when used to clean up terminated children
Suppose there is a client that has made a connect() call to the same server 5
times.
The server will call fork() 5 times and thus there will be one server parent and 5
server children.
When the client terminates, all 5 connections will send FIN at about the same
time to their respective server child, i.e. 5 FIN are sent, one on each connection.
This causes 5 SIGCHLD signals to be delivered to the parent at about the same
time.
The SIGCHLD signal that reaches the first calls the signal handler and the rest 4
SIGCHLD signals are not queued up and thus after the execution of signal
handler, only 1 SIGCHLD is again sent.
As a result, out of the 5 server child processes, 2 SIGCHLD signals are caught by
the server parent and handled. And the rest of the 3 server child processes
become zombies.

The correct way for the above kind of situation would be to all waitpid() instead of
wait()

We will call waitpid() with WNOHANG option, this tells waitpid not to block if
there exist running children that have not yet terminated.
We can not call wait() in a loop coz we cannot prevent wait from blocking.

Final version of SIGCHLD signal handler that calls wait()

void sig_chld(int signo)
{
pid_t pid;
int stat;
while ( (pid = waitpid( -1, &stat, WNOHANG)) > 0)
printf (child %d terminated\n, pid);
return;
}
We must remember to handle three scenarios:
1.
We must catch the signal SIGCHLD when forking child process.
2.
We must handle (restart) interrupted system calls when we catch signals
3.
SIGCHLD signal handler must be coded properly with waitpid() to prevent any
zombies from being left.

TCP Echo Server that handles an error of EINTR from accept

int main (int argc, char **argv)
{
int listenfd, connfd;
pid_t childpid;
socklen_t clilen;
struct sockaddr_in cliaddr, servaddr;
char buff[80];
listenfd = socket(AF_INET, SOCK_STREAM, 0);
bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_port = htons(1234);
servaddr.sin_addr.s_addr = htonl (INADDR_ANY); //wildcard
bind(listenfd, (SA*)&servaddr, sizeof(servaddr));
listen(listenfd, LISTENQ);
signal (SIGCHLD , sig_chld);
for ( ; ; ) {
clilen = sizeof (cliaddr);
if ((connfd = accept (listenfd, (SA*)&cliaddr, &clilen) ) < 0 ) {
if (errno == EINTR)
continue;
else
err_sys(accept error);
}
if ((childpid = fork( )) == 0) {
// child process
close (listenfd);
str_echo (connfd);
}
close(connfd);
}
close (listenfd);
}

Connection Abort before accept( ) returns

Socket, bind, listen

Passive open
Socket
Connect (blocks)

SYN

SYN_RCVD

ack
SYN,

ack

Connect returns

ESTABLISHED

RST
accept returns

client

Server

Three-way handshake completes, the connection is established, and then the

client TCP sends a RST (reset)
On the server side, the server kernel has completed the three-way handshake but
the function call accept( ) has not returned in the user program on the server side.
Sending RST means the previous connection has been aborted from client side.
What happens to the aborted connection is implementation dependent: In some implementation (systems), the aborted connection is handled within
the kernel, and the server process never sees it.
In some implementations, an error is returned, EPROTO (protocol error)
In some implementations, an error is returned, ECONNABORTED
(connection abort error). In these systems, EPROTO is different from
ECONNABORTED.
EPROTO is a protocol-related fatal error whereas ECONNABORTED is a nonfatal error.
In case, ECONNABORTED error is returned, the server ignores the error and
calls accept again.

Termination of server Process

Steps :1.
Start the server and the client and type one line on the client to verify that all is
OK. The line will be echoed back by the server child.
2.
Kill the server child. As part of this, all open file descriptors of the server child
will be closed. This causes a FIN to be sent to the client, and the client TCP
responds with an ACK.
3.
SIGCHLD signal is sent to the server parent and handled correctly.
4.
TCP client kernel has received a FIN and has sent an ACK, but the client user
process is blocked in the call to fgets( )
5.
At this time, client is in CLOSE_WAIT state (has received FIN, but has to send
its own FIN) and server is in FIN_WAIT_2 state (its FIN has been
acknowledged, but has to receive FIN)
6.
$ ./echocli
hello
start client
hello
echo from server
kill server child on server host
again
type second line on the client
str_cli: server terminated prematurely

When we type again, str_cli calls write( ) and the client TCP send the data to
the server. This is allowed by TCP coz the connection is in half-close state, i.e.

the client can send data to the server.

When the server TCP receives the data from the client, it responds with a RST since
the socket on the server side has terminated.
7.
Client process will not see the RST, it will see the FIN, because the client calls
readline( ) immediately after the call to write( ).
readline( ) returns 0(end-of-file) because of the FIN received by the client
kernel.
Client is not expecting to receive an EOF, so it quits with the error message
server terminated prematurely
8.
When the client terminates, all its open descriptors are closed.
9.
If in the client, there is no immediate call to readline( ), then the client will not
receive FIN, but RST and would return the error ECONNRESET (connection
reset by peer)

SIGPIPE Signal
Suppose there are two write( ) instructions in str_cli one after the other, i.e. the
client has to perform 2 write( ) before reading anything back.
The rule is : when a process writes to a socket that has received an RST, the
SIGPIPE signal is sent to the process. The default action of this signal is to
terminate the process, so the process must catch the signal to avoid being
terminated.
Void str_cli(FILE *fp, int sockfd)
{
char sendline[MAXLINE], recv[MAXLINE];
while( fgets(sendline, MAXLINE, fp) != NULL) {
writen(sockfd, sendline,1); //writing one byte of data
sleep(1);
writen(sockfd,sendline+1 ,strlen(sendline)-1); //writing rest of the data
readline(sockfd, recv, MAXLINE);
fputs(recv, stdout);
}
}

We write one byte of data first, and the rest after a pause of 1 second, the
intent is to write first to a closed socket and thus receive a RST. And then write
to the same socket again to generate SIGPIPE.
$ ./tcpcli
Hi
we type this line
Hi
echoed back by the server
we kill the server process
Hello
we type this and send to the server
$
no error is shown, just the command prompt comes
$ echo $?
To see the return value of the last command
269
256 + 13

Echo $? prints (Shell return value + signal number).

Shell return value is 256 and 13 signal number is for the SIGPIPE signal.
This is one of the main problems of the SIGPIPE signal that no error is shown
on the prompt, just the process is terminated.
If nothing else has to be done, SIGPIPE signal should be caught and
disposition should be set to SIG_IGN. So that the process is not terminated.

1.

2.
3.

Crashing of server host

To simulate this, well run the server and client on different hosts. Then we type
some text on the client and it should be echoed back to verify that everything is
working fine.
We now disconnect the server from the network and type in a new line on the
client to be sent to the server.
This also covers the scenario of the server host being unreachable when
the client sends data (some intermediate router is down after connection
establishment)
The following steps take place:
When the server host crashes, nothing is sent out on the existing network
connection. That is we are assuming that the host crashes and is not shut
down. (coz shut down means, it will close the connection in proper manner)
A line is sent from the client using writen( ) to the server. The client then blocks
in the call to readline( ), waiting for the echoed reply.
If well check using TCPDUMP command, well see that the client continually
retransmits the data, trying to receive an ACK from the server.
After the given number of retransmissions, the client gives up and an error is
returned to the client process. The error sent is ETIMEDOUT.
If some intermediate router is unreachable, the error sent is EHOSTUNREACH
and ENETUNREACH.


1.
2.
3.
4.

5.

Crashing and rebooting of the server host

The following steps take place:
We start the server and the client. We type a line on the client and it is echoed
back to verify that everything is working fine.
The server host crashes (not shut down) and reboots.
We type a line of input on the client, which is sent to the server host.
When the server host crashes and reboots, it looses all the information about
the connection that existed before the crash. Therefore the server TCP
responds to the received data segment from the client with an RST.
Our client is blocked in the call to readline( ) when the client kernel receives a
RST from the server. This causes the readline( ) function to return with the
error ECONNRESET.

Shutdown of server host

When a UNIX system shut downs, the init process normally sends the SIGTERM
to all processes, waits some fixed amount of time, and then sends the SIGKILL
signal to any process still running.
This gives all running processes a short amount of time to clean up and
terminate.
If our server will not catch the SIGTERM signal and terminate itself, the server will
be terminated by the SIGKILL signal.
When a process terminates, all open descriptors are closed and the sequence of
steps will the same as for termination of server process.