CS495P MAIN PROJECT REPORT

Intranet Ca hing Proto ol

SUBMITTED IN PARTIAL FULFILMENT OF THE DEGREE OF

BACHELOR OF TECHNOLOGY

by

Ajth T Y2088

Salih K A Y2095

Under the guidan e of

Mr. Vinod Pathari

Department of Computer Engineering

National Institute of Te hnology, Cali ut

2005,Monsoon Semester
 National Institute of Te hnology, Cali ut
Department of Computer Engineering

Certied that this Main Proje t Report entitled

Intranet Ca hing Proto ol

is a bonade report of the work done by

Ajth T Y2088
Salih K A Y2095

in partial fullment of the

Ba helor of Te hnology Degree

Mr. Vinod Pathari Dr. M.P. Sebastian

Le turer Professor and Head

Dept. of Computer Engineering Dept. of Computer Engineering

 A knowledgement
We thank Mr. Vinod Pathari, Le turer, Department of Computer S ien e and Engineering,
NIT Cali ut for his guidan e and o-operation. We also a knowledge the advi e and help given
to us by our friends. We would like to extend our gratitude to the entire fa ulty and sta of
CSE Department of NITC, who stood by us in all di ulties we had to fa e during our work
on this proje t.

Ajth T Y2088
Salih K A Y2095
 Abstra t
Downloading les from the internet is a most frequently o uring pro ess in a Lo al Area
Network(LAN).It may ause a 'bottlene k' of identi al requests for an externel web obje t.This
proje t is an attempt to provide a solution by developing a LAN based proto ol.
This report ontains details on the motivation, design and implementation behind the pro-
vision of an intranet a hing proto ol.
Contents

1 Introdu tion 6
1.1 Problem Spe i ation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Design 7
2.1 Indexing the les . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Server Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.2 Client Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2.1 Proto ol Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2.2 Category Based Sear h . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3 Proxy Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3 Platform and Environment 13

4 Implementation and Testing 14

4.1 Proto ol Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.1.1 Server Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.1.2 Client Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.2 Category Sear h Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.3 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5 Performan e Analysis 19

6 Con lusion 19
1 Introdu tion

1.1 Problem Spe i ation

The aim of the proje t is to develop an intranet a hing proto ol for downloading les. Requests
for previously a hed obje ts result in a a hed opy of the obje t being returned to the user
from the Lo al Area Network rather than from the host server. Also a ategory sear h fa ility
whi h helps the users to see and retrieve the les shared in dierent ma hines. When a request
for a web obje t omes from a browser, the proto ol rst sear hes its availability in the LAN
before sending the request to the origin server. If so, the request is satised from the intranet
itself. The sear h fa ility asks the user for the le type and an optional size-range, and displays
the results satisfying these riteria for retrieval.

1.2 Motivation
Downloading les from the internet is a most frequently o urring pro ess in Lo al Area Net-
work. As bandwidth inside a lo al network (LAN) is signi antly greater than Internet a ess
bandwidth, a hing an prevent a 'bottlene k' of identi al requests for an external web obje t.
This results in little or no extra network tra over the external link and maintains a higher
speed of delivery. Ca hing an therefore be seen to be simultaneously making more e ient
use of Internet bandwidth and exploiting the availability of LAN bandwidth.

1.3 Literature Survey

For the Proto ol design, C. Gray, D. Leases [1℄ and Li Fan, Pie Cao, Jussara Almeida, Andrei
Z. Border [2℄ were beni ial. For the various system alls and network programming, Graham
Glass, King Ables [3℄ and W. Ri hard Stevens [4℄ provided us the ne essary information. We
used the web a he proxy [7℄ for the proto ol implementation. For the database design and
for the C Appli ation Programming Interfa e, SQL Tutorial [5℄ provided good do umentation.
For the GUI reation, Qt Tutorial [6℄ proved to be beni ial.
2 Design

Design in ludes mainly three parts. Indexing the shared les, developing the algorithm and
hoosing the proxy server. In the rst part we are dis ussing the pro edure for making the
indexing information. In the se ond part, we des ribes how to retrieve the les from the LAN,
upon re eiving a request from any lient. In the third part, we des ribes the main features of
the proxy server we took for the implementation of the algorithm.

2.1 Indexing the les

It in ludes indexing at server level and lient levels.

2.1.1 Server Indexing

There are two tables maintained in the server in a database. One table named URL_Table,
will ontain the URL of the requested resour e, two Client IP addresses where the resour e is
supposed to be a hed, two ags representing whether the above spe ied lients shared the
downloaded resour e or not. The other table named Server_IndexTable ontains the path of
the a hed resour e in the lient, its IP address and the name of the le with whi h it is lo ally
stored. The entries to the table are added periodi ally.

2.1.2 Client Indexing

Ea h lient maintains an index le for the resour es it shared. It in ludes the path of the
resour e and the size of those resour es. The shared region is periodi ally monitored and
entries are added to this le.
Pi torial View

LAN view

Server_IndexTable View

URL_Table View
2.2 Algorithm
It in ludes algorithms for the Proto ol and for the Category based sear h.

2.2.1 Proto ol Design

Server Side
At the Server Side there are two algorithms, one for building the Server_IndexTable the
other for the ow of lient web request.
Building Server Index Table

Begin
For ea h lear flag in the URL_Table
Begin
Bring the Client_IndexFile from the asso iated lient in the URL_Table
If (url file is in Client_Index file)
Begin
Add entries to Serever_IndexTable
Set orresponding URL_Table flag
End
End
End
Request Flow

Begin
Get the URL from lient request
If(!(Request URL is in URL_Table ))
Begin
If( lient request is in the downloadable ategories)
Begin
Add entries to URL_Table
End
Pass the lient request to webserver
End
Else
Begin
For ea h set flag in the URL_Table orresponding to the URL
Begin
Get the orresponding IP address and the path inormation from
the Server_IndexTable.
Get the resour e from the orresponding ma hine(if a tive)and
send to the requester.Set the variable su ess.
If (su ess) break
End
If (!su ess)
Begin
Pass the lient request to webserver
End
If(any IP field NULL in URL_Table )
Begin
Update URL_Table with the IP address of the requester.
End
End
End

Client Side
1. Periodi Updation of Client_Index file.
2. The ftp program whi h runs on ea h lient sends the requested file
to the server.
2.2.2 Category Based Sear h

A user interfa e is provided for this pupose.

Begin
Retrieve Client_Index files of all a tive lients
For ea h file retrived
Begin
Sear h for spe ified ategory within the size limit provided by
the user
If(mat h found)
Begin
Display on the interfa e
End
End
If(user needs any displayed resour e)
Begin
Retrieve the resour e from the orresponding IP and path
spe ified
End
End
2.3 Proxy Design
The proxy we hosen is a web- a he proxy.Ca he servers have a proxy fun tion with regard to
requests for ertain ontent from the World Wide Web. When a lient passes all their requests
for web obje ts via a a he, this a he is ee tively a ting as a proxy server. Ca hing is a
ommon fun tion of proxy servers.We implemented our algorithm on top of this web- a he to
provide a se ondary level of a hing.

How does a web- a he work ?

3 Platform and Environment

The proje t was developed under UNIX (LINUX) using the g ompiler, as this system was
available and we had usage experien e. The UNIX environment with it's various tools also
proofed to be very useful for testing purposes. For example, the text-only browser LYNX was
used to easily debug a problem related to the HTTP/MIME Header:
useralinux:/home/user > lynx -mime_header http://lo alhost/
4 Implementation and Testing

4.1 Proto ol Implementation

It in ludes implementation at server side and lient side.

4.1.1 Server Side

Proxy-Part

Here we made use of a minimalist a hing proxy supporting HTTP/1.1 named HTTPEyes[7℄.
Its sour e onsists of around 2.300 lines of C ode. It uses POSIX threads to handle multi-
ple simultaneous onne tions. It is li ensed under the GNU GPL Li ense and runs in Unix
ma hines.
In GNU/Linux i386 with no debugging symbols the exe utable is about 25k ( ompiling it
without assertions and verbose messages). No other les are required for its operation.
One of the basi problems when running several threads that use the same memory spa e, is
making sure they don't "step on ea h other's toes". By this we refer to the problem of using a
data stru ture from two dierent threads. Mutex-a basi me hanism supplied by the pthreads
library to solve this problem.

Database-Part

My-Sql is the database used in the table onstru tion in server side. In order to make queries
into the database Mysql - C API is made used.

HTTP Response

Depending on the type and size of the resour e a hed in the LAN, the proxy generates the
HTTP Response headers before sending the resour e to the lient web browser. It makes use of
the ftp lient in- ooperated in the proxy to get the ne essary information. The ommon types
supported by the proxy are mp3, mpeg, avi, iso, tar and exe. Below is thet list of MIME Types
supported by this server.The "mime-type" of an en oded le refers to the type of data that has
been en oded.
 The table representing the various ategories and their orresponding MIME types.

Category MIME Type

iso appli ation/x-iso9660-image
exe appli ation/x-msdos-program
mpeg video/mpeg
mp3 audio/mpeg
tar appli ation/x-tar
avi video/x-msvideo

4.1.2 Client Side

Indexing the les in Clients

A le s anning utility is used to ontinuously s an the shared region in lients. It will reate
and monitor Client_Index le in lients. It make use of basi unix system alls to get the size
and type of les in the s anned region and to traverse inside dire tories.

Transferring the les

The ftp lient whi h runs on the server requests for the Index le , size of a resour e and
the les spe ied by the user. The ftp server running on ea h lient sends the resour es on
request. It make use of basi so ket programming.
4.2 Category Sear h Implementation
The implementation of ategory sear h is done in three phases. In the rst phase, program
for nding the IP address of all the ma hines whi h are urrently a tive in LAN is done. In
the se ond phase, program to transfer les between two ma hines is developed. And in the
nal phase a graphi al user interfa e is built using Qt Designer. The above mentioned phases
involved in the development are explained below.

First phase:

When the sear h button provided in the GUI is li ked, a broad asting program is exe uted
and it broad asts an AreYou_Up message in to the network. The lient program running in all
a tive ma hines will respond to this message by sending an AreYou_UpAnswer to the server.
Then the server stores the IP address of these ma hines in a le named 'upnode'.

Se ond phase:

An ftp lient is developed in ea h lient using the basi so ket programming and it will
retrieve any requested le from an ftp server running on other ma hines. So every ma hine has
an ftp lient and an ftp server running on them inorder to request and retrieve les.

Third phase:

The main window of the GUI asks the user to enter the type of the le to be sear hed and a
lower limit for the le size. Then the sear h result is displayed in the window. Double li king
the entry a tivates the ftp lient on that ma hine and it retrieves the le from the user having
that le with the help of ftp server running on that ma hine.
4.3 Testing
We have tested the proje t in the CSED LAN by setting up the proxy server in Athena server.
The lient side programs are installed in other ma hines. Initially the URL_Table and the
Server_IndexTable maintained in the server were empty. When the lient1 is requested to
download a le 'X1' from the URL 'url1' URL_Table was empty and the le is downloaded
from the internet and the orresponding URL is a hed in the URL_Table as shown in the
URL_Table View1.

URL_Table View 1

The s anning program whi h is monitoring the URL_Table periodi ally, he ked the avail-
ability of the le in the ma hine and inserted the entry into the Server_IndexTable if present
as in Server_IndexTable View 1.

Server_IndexTable View 1

If le is present, we set the ag in URL_Table as in URL_Table View 2.

URL_Table View 2

Then the next request from the lient2 for the same le is dire tly taken from the lient1
and the updation of the URL_Table is done as in URL_Table View 3.

URL_Table View 3

If the le is available in the se ond ma hine spe ied in the URL_Table then the URL_Table
and Server_IndexTable are modied as shown in Server_IndexTable View 2 and URL_Table
View 4 respe tively.

Server_IndexTable View 2
URL_Table View 4

If subsequent request for the same le omes fronm the LAN, it an be satised from any
of the ma hine spe ied in the URL_Table.

Category sear h is tested in four ma hines. We have given the type 'type1' and size 's1' as
sear h lters so that only the les of type 'type1' having size greater than 's1' will be displayed
in the window. A double li k on the entry retrieved the le from the orresponding ma hine.

GUI of Category Sear h

5 Performan e Analysis

We had anaysed 50 requests from dierent ma hines in the CSED LAN inorder to estimate the
per entage of requests whi h are satised from the LAN.Out of these requests we had found
that on an average 30 downloading requests were satised from the LAN and 20 requsts were
send to the Internet. Sin e sixty per entage of these requests were downloaded les from the
LAN, only the remaining fourty per entage of these requests were send to the origin servers.The
large number of redu tion in the request to the internet is highly improved the usage of available
bandwidth provided to our institution.

6 Con lusion

The proto ol we developed improved the download time for su essive web requests and Internet
bandwidth usage was redu ed signi antly. The perfoman e of this proto ol is very good in a
kind of network in whi h the people are having similar ideas and thinkings. Also the Category
sear h fa ility serves as a good tool for users in the LAN, to see the resour es in the intranet
before sending request to origin servers.
Referen es

[1℄ C. Gray, D. Cheriton, Leases: an e ient fault-tolerent me hanism for distributed le a he
onsisten y, Pro eedings of the twelfth ACM symposium on Operating sytem prin iples,
p.202-210, November 1989.

[2℄ Li Fan, Pei Cao, Jussara Almeida, Andrei Z. Broder, Summary a he: a s alable wide-
area Web a he sharing proto ol, Pro eedings of the ACM SIGCOMM '98 onferen e on
Appli ations, te hnologies, ar hite tures, and proto ols for omputer ommuni ation, p.254-
265, August 31-September 04, 1998, Van ouver, British Columbia, Canada.

[3℄ Graham Glass, King Ables, UNIX for Programmers and Users, Pearson Edu ation, 2003.

[4℄ W.Ri hard Stevens, UNIX Network Programming

[5℄ SQL Tutorial, http://www.mysql. om

[6℄ Qt Tutorial, http://www.qtforum.org/

[7℄ HTTPEyes - Web Proxy Ca he, http://ba hue. om/httpeyes/httpeyes-0.9.tar.gz/