CSE-506 (Fall 2008) Homework Assignment #2 (100 points, 18% of your overall grade) Version 2 (10/13/2008) Due Sunday

11/9/2008 @ 11:59pm * PURPOSE: To become familiar with the VFS layer of Linux, and especially with extensible file systems APIs. To build a useful file system using stacking technologies that implements transparent file integrity checking: ifs. * DETAILS: Integrity is an important principle of information assurance, which states that the data you read, must be the same data that you wrote originally. Often, integrity is guaranteed using a checksumming algorithm such as MD5 or SHA1. Your job is to build a transparent integrity-checking stackable file system, called ifs. Ifs will be able to layer itself on top of any other file system and transparently create and check the integrity of files. If a file and its integrity don't match, your file system should return an appropriate error message back to the user application. For each file "F", ifs will maintain a file named ".F.md5" which contains the MD5 checksum of file "F". When you create a new file, you must also create the checksum file to go with it. When you delete a file, you have to delete its checksum file as well. If you update an existing file, you have to also update its checksum file. Study the Linux kernel CryptoAPI to find out how to checksum files using the MD5 algorithm. Use the latest 2.6.26.x vanilla kernel please. Checksum files, however, should be invisible from user's view through the ifs mount point: a readdir() or stat() should not find them. The file system must be mountable with the following command: # mount -t ifs -o dir=<lowerfs> ifs /mnt/ifs You're free to support additional mount options. When a user opens a file for reading, you must read the file, compute its checksum, and compare the checksum to what's stored in the .F.md5 file. If the checksums match, proceed. If they don't match, return an error (perhaps EPERM?) ** Checksumming Existing Files One problem you'll have to worry about is what happens if you mount ifs on top of an existing directory of files, and you don't have the checksum files there already. If a user tries to access a file for which there is no checksum file, it should be considered an error. For that reason, you have to provide an alternative administrative method to create checksums for files for the first time. Write a user-level program, called ifs_ctl, that could be run periodically to create the checksum for your file(s). The usage for ifs_ctl is: $ ifs_ctl [-haR] file [file file/s...]

-h: print help and exit -a: checksum the file(s) asynchronously [default: synchronous] -R: apply operation recursively on file/s [default: non-recursive] (Add any other flags as necessary, but document them.) Ifs should support an ioctl(2) that, when invoked, will create a checksum file for a given file "F". The ioctl should operate one file at a time, and perform the checksumming right then (synchronously), returning an appropriate status code back to ifs_ctl. If the -a option is given, then the checksumming should be done asynchronously. ** Kernel Thread Checksumming can be an expensive operation consuming a lot of I/O and CPU. Therefore, ifs should support a kernel thread (kthread) to perform such checksumming asynchronously. Create a kernel thread called [kifsd] which will act as a producer-consumer queue: your ioctl can add files to that queue, and something else will dequeue entries from the queue, then checksum them. You have to ensure that the kthread won't consume unbounded resources; its size should be limited. You have to worry what happens when the queue is empty or completely full (e.g., when to block). See kthread_run, kthread_stop, etc. You also have to worry about what locks you need and when. Clearly your queue has to have a lock: but which type of lock is best? What happens if a file is in your queue, then a user tries to read/write/unlink/etc. that file? How can you detect this and address it? What kind of locking, if any, is needed here? ** Miscellaneous Write a man page for ifs_ctl, using the nroff(1) language. The man page file should be named ifs_ctl.8. See any other similar man pages as an example of how to write man pages. You should thoroughly check your code. There are several common ways in which people test that a file system works. You're welcome to use any one or more of those: - compile some large software (linux kernel, gcc, openssh, etc.) inside your file system - run Postmark, a tool to test I/O throughput - run fsx, a tool intended to exercise file system read/write operations - use a free POSIX compliance testing suite for Linux called the Linux Testing Project (LTP). See http://ltp.sourceforge.net/. - write small C programs to exercise specific system calls - use basic tools like ls, cat, rm, mv, strace, etc. ** Freedom and README While making the assignment you will encounter various issues which are not discussed in this document. Most of these issues have more than one

reasonable solutions. It means that you are free to select the design that you think is appropriate. Taking that into account README file becomes extremely valuable. So try not to write it at the very last day. Describe in the README all the problems you've encountered, how does your design solve them and why do you think such design is appropriate. Some of the issues that you perhaps want to address in the README: - Directories and other non-regular files on ifs - If a file is a hardlink, do you always need to compute MD5 checksum? - Various permissions-related problems - Do you implement all file/inode/dentry operations for ifs? - When do you compute a checksum for an updated file? - Everything related to the asynchronous execution of ioctl() - Is it always possible to run an executable on ifs? - What happens if one process updates the file, which another process reads? * RESOURCES It is vital that you read Chapter 12 of the textbook, as well as the paper "A Stackable File System Interface For Linux" which is available here: http://www.fsl.cs.sunysb.edu/all-pubs.html#papers I have provided you with a blank stackable file system template based on Wrapfs. Download it from here: http://www.cs.sunysb.edu/~ezk/cse506-f08/ifs-2.6.26.tar.gz Then untar it into your hw2 directory. To do this assignment, modify the ifs sources as needed. Please keep the name of the file system as "ifs" and the name of the module (produced by the `make`) as "ifs.ko". Also notice that running make in the hw2 directory should produce "ifs_ctl" executable. As it stands, the sources there should compile and run in your vanilla kernel, implementing a non-fanout "null layer" functionality. A "null layer" file system simply passes all of the file system operations to the layer below it. You can compile the ifs.ko module, then load it into the kernel to enable a new file system. There are several useful user-land utilities (esp. for enabling debugging levels) in the tarball as well as a couple of shell scripts. Study all of these files carefully. Note: some of those files are not needed for HW2, and so you should remove them from the distro and not commit them. However try to commit your changes frequently to avoid troubles. We recommend you to implement ifs step by step, starting from the simple functionality. After each step test your code. Then continue to more intricate stages. * SUBMISSION You will need to submit all of your sources, headers, scripts, Makefiles,

man pages, README, and whatever other files are needed. Submit all of your files using CVS. See general CVS submission guidelines on the class Web site. PLEASE test your submission before submitting it, by unpacking it in a separate directory, compiling it cleanly, and testing it again. This is described on the class Web site. DO NOT make the common mistake of writing code until the very last minute, and then trying to figure out how to use CVS and skipping the testing of what you submitted. You will lose valuable points if you do not get to submit on time or if you submission is incomplete. For this assignment, you may work alone or with ONE OTHER partner (i.e., in pairs). You will be graded the same whether you work alone or in pairs; however, at the end of the semester, I may give extra consideration to students who worked alone and their grade may be a borderline one. If you work in pairs, you MUST declare that in your README (full names and emails of both team members). Lastly, if you wish to use a shared CVS repository, ask me and I'll create one for you (give me the OSLAB user-names of both partners and full names). * EXTRA CREDIT (OPTIONAL) [10 pts] Support multiple checksum files. Support all known checksum files in the Linux CryptoAPI. Your ioctl should instruct ifs to create one or more checksum files, using different algorithms. Also, support a mount time option "-o cksum=ALG" which sets the default checksum algorithm to "ALG". Good luck. * ChangeLog: a list of changes that this description had v1: original version v2: switch IFS template from 2.6.23 to 2.6.26