521150A Introduction to Internet - Course Work

Contents 1. 2. 3. 4. 521150A Introduction to Internet - Course Work Preface General Instructions Passing the course work 1. One person groups 2. Three person groups 3. Four person groups Course Work Definition 1. Proxy server 2. Multipart messages 3. Encryption 4. Robust code 5. Integrity The TCP Protocol 1. Baseline example 2. Example with multipart support 3. Example with encryption support The UDP Protocol 1. Multipart messages 1. Pseudocode example 2. Encryption 1. Example 3. Integrity 4. Robustness Proxy Testing

5.

6.

7.

8. 9.

Preface
This course work aims to give a basic understanding of how networking applications are programmed using sockets. Course work simulates the situation where client and server define communication parameters using reliable TCP connection and the actual data exchange uses UDP. Similiar implementation can be found from for example many VoIP applications. There are quite simple requirements to pass the course work but if you want extra points or are in a group, you need to add some features to the application. For those who are willing to endure pain, there will be rewards. Please note that this document is still under construction. You should always check the newest version when coding. We try to inform you about the changes also in optima general discussion.

1 of 7

General Instructions
There are groups of three distinct sizes attending the course. To be completely fair, groups of one person will only need to complete simpliest part of the work. Groups of three people will need to add some extra functionality and groups of four even more. Points will be the same for every group member. To pass the programming exercise, your group must deliver their work as a zip file to a submission box in the course workspace in Optima before the deadline (2013-09-30). There are no exceptions to this rule. The file must be named in the following way: II_2013_groupname.zip, if the exercise is made by a group II_2013_Lastname_Firstname.zip, if the exercise is made by an individual The following are to be included in the zip file: A README.txt file which includes the group's name and the names, e-mail addresses and student ID's of the group members. A directory for a completed task, named task1. This directory will hold the source code and any possible makefiles or other necessary files for the task. A directory called doc, under which you will include all the relevant documentation for the tasks. DO NOT include any compiled executables (i.e. pyc objects) in the zip file! DO NOT include any files provided by the teaching assistants in the zip file! The work is done with the Python programming language. The software must work on the department Linux servers on command line. Supported servers are st-cn0001.oulu.fi and st-cn0002.oulu.fi. st-cn0002 has a newer version of python (currently 2.7.3) so we encourage you to use that. If you submit a work that does not work on the Linux platform, you will fail the programming exercise. So even if you want to code from home you should take a ssh connection to linux server and use text editor there so you can test the code in correct environment. External documentation is not required in this work, but you have to comment your code so that even a less intelligent assistant can understand what you have done. It does not hurt to have some kind of text documentation also. If assistants do not get what you have tried to do in your code, it might result in a bumerang. If you have enough courage you can try Doxygen ( http://www.doxygen.org/) or Epydoc ( http://epydoc.sourceforge.net/) to create your documentation. This might help you to comment your code because you need to generate the documentation during the coding. Each program must have some kind of help. Assistants do not waste their time trying to figure out how to run the program. You can help us few different ways: Program prints help if no arguments are given from the command line. Program prints help with -h or --help option. You can use for example optparse (which is deprecated since version 2.7 replaced by argparse). External readme-file where you spesify how to run the program Some other way which you find suitable Your submissions will be graded as soon as possible, and a feedback e-mail will be sent to all group members. If the program is not of acceptable quality, a list of things to be fixed will be included. Once the final deadline is within a week or has passed already, and your work has not been accepted, you have then exactly one week (7 days) from the feedback e-mail to fix your program and resubmit a working version. If your course work is still unacceptable after the second submission following the deadline, it will be regarded as failed. This means that you may fix and resubmit your work N times before the deadline, but only twice after the deadline. Submissions will be tested against the example programs that the TA's have created. You must code the protocol yourself. You are not allowed to copy code from any other source, it is considered plagiarism. Any student caught with plagiarising some other student or any other source will be kicked out of the course with all accumulated points nulled. Cases of plagiarism will also be reported to the head of the department for any further actions. Submissions will be tested against each other with a plagiarism detection tool. It will show if you have copied code from someone else.

2 of 7

Passing the course work

More detailed course work spesification will follow but here are the requirements to pass the course work for different group sizes. Early Bird deadline is 2013-05-27. If you want to get a EB bonus point you have to return your work before that. After we have reviewed your code and there is some flaws in it, we will give you one week to fix the program to get a bonus point. If it still doesn't work you can forget the EB point. Please test your code before returning it. You can easily miss two extra points if you get a boomerang and miss the EB. "No boomerang" bonus should encourage you to return finished work. Not a version 0.02 just to get some hints how to continue. You can add features to your program to get bonus points. For example one person group can choose two of the five additional features to get two extra points.

One person groups

If you are not in a group you only need to return acceptable program which uses clear text messages. No additional features are needed. 0p Clear text message exchange +1p Accepted work before EB deadline +1p No boomerang +1p One random feature added by your choice +1p Another random feature added

Three person groups

In addition to the clear text message exchange, groups will have to create the proxy feature to pass the course. 0p Clear text + proxy +1p Accepted work before EB deadline +1p No boomerang +1p One random feature added by your choice +1p Another random feature added

Four person groups

In addition to cleax text message exchange groups need to implement both proxy and multipart features. 0p Clear text + proxy + multipart +1p Accepted work before EB deadline +1p No boomerang +1p One random feature added by your choice +1p Another random feature added

3 of 7

Course Work Definition

In this course work you will generate a program which can receive and send messages according the protocol spesified. Program must first exchange some basic information with server using TCP and then use UDP for sending and receiving actual data. You will only create a client, not the server. Server will be made by TAs. There will also be a small header added to each package. Spesification will follow later. Default program should only receive plain text questions and reply correctly to those. Additional features such as encryption or proxy will be spesified later. Questions and answers will be given by TAs. Default requirements are as follows: Program must take server address and port as command-line arguments Program can take additional arguments such as list of extra features Program uses TCP to connect to server, exchanges port numbers, possible features and possible encryption keys with server Program uses UDP to send a "Hello" message to a server Server sends a question Program uses a method provided by assistants to find out the correct answer Program replies with correct answer Server sends a new message predefined number of times After all questions are answered, close the program

Proxy server
Proxy server acts as a man-in-the-middle for each packet sent or received. It listens for incoming messages and forwards them to a correct host. Proxy must work both ways but with only one client at the time. It must be listening both TCP and UDP connections and forward the message when one arrives. Proxy is a separate program. Proxy must listen for TCP connections in a defined port Proxy must read the port number from initial message and forward it to correct host Proxy must read the the answer, store the server port and forward the message back to client Proxy must open the UDP sockets to a correct ports and forward any message to a correct host If the header of the message says EOM, proxy should close UDP sockets

Multipart messages
In default functionality, the messages are under 64 characters long. Each message therefore results in a one packet which has a content length under 64 chars. Multipart messages have a longer length and are divided into multiple packets. Program should read the content length and decide whether there will be more data coming. After receiving the whole message, program should read the question and answer correctly. Answers may also be longer than 64 chars. Program must handle content longer than 64 characters Program must join all the data received in multiple packets Program must be able to send answers longer than 64 chars (more than one packet per message)

Encryption
In this feature you will communicate more securely using encryption. We will use so called one-time pad encryption, where client and server exchange list of cipher keys and use one cipher for encryption just once. After cipher has been used it has to be deleted properly. More detailed information about encryption will follow. Program must be able to generate cipher keys and send them to server Server sends it's own cipher keys Program must crypt/decrypt every message sent and received Program must delete the cipher key after used

Robust code
When creating applications we don't want the program to end in a failure. Things like giving a port number as a char may cause your program to end in a stack trace if we do not handle the possible errors. And users can make huge amount of different kinds of erros. In this feature you should create your code so that any anomaly fed to your program do not cause program to stop unexpectedly. Mostly with try-except structure. We will give different kinds of out-of-the-protocol-spesification messages to your program and see what happens. Each time you receive something from the server you should think what if the data is something other than what protocol says.

Integrity
Integrity is a part of information security. We must be sure that the message received came from the correct sender and it was not modified. In this feature we have to check few basic things before proceeding. Check that the content length in header matches the actual content length. Check that there is a answer found to a question. Check that the question came from the correct host and port.

4 of 7

The TCP Protocol

Client sends the initial connection request to server along with a UDP port it wants to use for data communication. Server replies with own UDP port. Each message should end in carriage return and newline (\r\n).

Baseline example
Client HELO 10000\r\n > < HELO 12345\r\n Server

When implementing extra features, capabilities of the client must be included in the HELO message as a string after the UDP port. The string may contain one or more of the following characters: M C I A Multipart message support Confidentiality (encryption support) Integrity (message integrity checking) Availability (client software robustness)

Example with multipart support

Client HELO 10000 M\r\n > < HELO 12345 M\r\n Server

Example with encryption support

If we use encryption, we will have to exchange encryption keys. The keys are 64 byte strings of hexadecimal characeters. The keys are sent after the HELO message, each key on its own line, followed by a single perdiod on a line. Client HELO 10000 C\r\n cAf90e6fa33feA392d73fBe3fab36243Ae96A7CC0EABb7cEf080dACdEf91A119\r\n ... more keys ... cEEf6A57D50A422Abf2855AD06EAb7DE74Bf4b70Ad8b0725fbe1bfa4f9db51F6\r\n .\r\n > > > > > < < < < < HELO 12345 C\r\n 96B2DcA1Abcf9DffBCd4d688bB232f5eFafb1893fdABdDBd19ce2469edDF087a\r\n ... more keys ... 926D3a513cCacEA63a1C0B47bBf9f1E75Aa8e4dEfC9dBCf9c7F8f7F65Da6A9A9\r\n .\r\n Server

5 of 7

The UDP Protocol

After setting up features and ports with TCP, we use UDP for actual data communication. There is a small header added to each message. Structure of the header is following: EOM Bool ACK Bool Content length unsigned short Data remaining unsigned short Content char[64]

EOM: If you receive end of message, you know that the program can finish. Boolean value True == end. ACK: Boolean value whether the last message received correctly or not. Boolean value. True = ok. Content lenght: Length of the actual content, header is not calculated. Data remaining: Length of the data still remaining (in multipart messaging). Content: 64 byte string of message content. Note that content less than 64 bytes is padded with null characters to fill the 64 byte space. They should be removed from received messages, i.e. message.strip('\x00'). Messages are structured to binary values before sending (you also receive binary data). This can be done using python's struct module.
Example: data = struct.pack('!??H', eom, ack, length)

This would pack eom as a boolean, ack as a boolean and length as an integer. After the TCP connection, client must send the initial message to the server. Message should say "The eagle has landed". Then the client should wait for the first question. Answer for the question can asked from the program created by TAs like this:
from questions import answer message_out = answer(message_received)

Note that we are still making changes to a questions so you should download a new version every now and then. After a number of questions, client should receive EOM and exit nicely.

Multipart messages
When sending messages larger than 64 bytes, the message must be split into chunks of 64 bytes and each send separately. The data remaining field in the header tells how much data is left unsent. For example, given a message content of 128 bytes, two messages must be sent. In the first message, the remaining field is 64, in the second it is 0.

Pseudocode example
def pieces(message, length=64): return list of message pieces remain = len(message) message = pieces(message) while message: subtract piece size from remain pack data send data

6 of 7

Encryption
Messages are encrypted by adding the numeric value of each character in the message to the numeric value of the corresponding charcater in the key Decryption is done by subtracting the values of the key's characters from the encrypted message's characters ord() returns the numeric value of a character, chr() return the character from a numeric value All messages are assumed to include only ASCII characters (numerical value 0-255) Each message is encrypted with a new key, keys are never reused Nubmber of keys exchanged is 20

Example
Given the message "Hello" and key "12345", the encrypted message is "Igopt". Message Numeric Key Sum Encrypted H 72 1 73 I e 101 2 103 g l 108 3 111 o l 108 4 112 p o 111 5 116 t

Integrity
For message integrity, the following checks should be performed: Header length field matches the actual message content length Valid answer is found for the question sent by the server Source address of the message is the server address given at startup and the port negotiated in the HELO process If the integrity check fails, the client must send a message to the server with the ACK field set to false, the message content should be "Send again.". The server will then re-send the message.

Robustness
Some of the anomalies the client should handle gracefully might include: Missing or extra fields in the message Incorrect field value, size or order in the message Non-ASCII characters in the message Caught exceptions should cause a request to re-send the message as specified in the ingetrity section.

Proxy Testing
You can test your implementation with a server created by TAs. Server is running in a host ii.virtues.fi port 10000 (not working yet). One of the bonus points is "no boomerang" point so you should really test your program before uploading it to optima. If you need to find out for example what kind of responses the server gives, you can use telnet and/or netcat to connect to server and send correct messages. For example use nc -u -l 10000 to open UDP socket to port 10000 and then telnet ii.virtues.fi 10000 and give the first HELO with port 10000 as UDP port.

7 of 7