An Online System for Assignment Marking

Greg Baker
Simon Fraser University
Abstract
In an attempt to streamline the process of marking electronically submitted
assignments for large classes, an online marking system has been
implemented that can be used by instructors and teaching assistants. This
system has had several benefits in terms of efficiency of marking time and for
student feedback.
The marking system itself is described, along with a summary of experiences
using it and possible improvements that can be made.
Keywords: grading, marking automation, web application

1. Introduction
The School of Computing Science at Simon Fraser University has for some time had an
Assignment Submission Server available for use in classes. This allows students to
submit files over the Internet to the instructor and teaching assistants. When received,
the submitted code or other work can be evaluated.
Getting feedback to students has been tricky with this system. When written assignments
are submitted, the marker can annotate and give feedback on the question paper itself,
which is then returned to the student. With electronic submission, the common solutions
are to print out and annotate the assignment, fill out a marking form with comments, or
send email to the student with their marks.
Printing out assignments that has been submitted electronically seems counter
productive and returning printed assignments is problematic. It also leaves the instructor
with no copies of the marks and comments once they are returned to the students.
Sending email to each student individually takes a lot of timethe marker must not only
send each email, but also record the marks separately.

2. Automated Marking
In order to address this problem, an online marking system was created. This is a web
based application created in Perl and using MySQL for data storage. Being a webbased
system, it use is not limited to a particular location or group of machines. All

administration and configuration of the system can also be done with a webbased
interface.
Markers can log into the system and access any courses for which they have been granted
access. When they select a course, they are presented with a list of assignments.
Assignments can be marked by any combination of authorized users. To mark a student's
assignment, the marker enters the student's userid (which is assumed to uniquely identify
a student), their student number and email address.
Each assignment has a list of criteria which the students are expected to meet. These
can be configured by the instructor or any of the markers using an online interface. For
each criterion the marker can enter a mark and comments. The comments are arbitrary
text and can be quite long (up to 64k characters). This allows students to get very
detailed feedback on their work.
It was recognized that often many students make the same mistakes on their
assignments. This means that the marker will often have to enter the same comment
many times. There are also occasionally problems with consistencya marker can change
the mark penalty for a particular error part way through the marking or different
markers could apply different standards. To address both of these problems, the concept
of common problems was added to the system.
Common problems are configurable on a percriteria basis. The marker can indicate a
mark penalty and comment for a particular problem. Then on the marking screen, a
button appears associated with that problem. When clicked, it deducts the mark penalty
and inserts the comment for the student. Once a marker has configured a reasonably
complete set of common problems, an individual assignment can be completed quite
quickly. This uses less marking time but also provides better feedback to studentsthe
marker only has to type each comment once, so can take more time in crafting useful
feedback for the problem.
Below is a screenshot of the marking system with a sample assignment. At the top are
the fields to enter student information, ie. their userid, student number and email
address. Following are the assignment criteria. For each criterion, its name and
description are displayed. The marker can enter a mark for that component and a
comment. Finally, the common problems that the marker has configured for this
assignment are represented by the buttons to the right of the criteria. The marker can
also enter a late penalty and overall comments on the assignment (not shown).

Along with simple marks and comments, a file can be stored along with each marked
assignment, allowing for return of annotated code. A student's code can be converted to
an Acrobat PDF file and the marker can add comments with Acrobat. This file can then
be returned to the student, allowing the same kinds of comments that might be made on
printed code. Below is part of an example PDF file.

While this was created for annotating code, any other type of file could be included
along with the marking as well. When the data are submitted, they are then stored in the
system for later processing.

3. Output
The most visible output from the marking system is the email that is sent to the student.
This is used to give feedback, as well as to report marks to the students. A sample email
is shown below.

To:thestudent@sfu.ca
From:JohnDoe<jdoe@cs.sfu.ca>
Subject:CMPT000Assignment1results
SFUUserid:thestudent
Studentnumber:000000000
=================================================================
ProgramCreation:2outof2
Criteria:Thefirst.cprogramwascreatedproperly.
CompilingandExecution:0outof3
Criteria:Theprogramwascompiledandruncorrectly.
Comments:nooutputprovided
Modifiedprogram:5outof5
Criteria:Theprogramwasmodifiedandcorrectlycalculates
thearea.
Algorithm:3.5outof5
Criteria:Thealgorithmdescriptioniscompleteandunambiguous.
Comments:DecompositionOutlineisnotcomplete.Howdoyouknow
thattheequationhas0,1or2realroots?
=================================================================
TOTAL:10.5outof15
[MarkedJan292003,11:43byJohnDoe.]

Typically, all of the emails to the class are sent when the marker has completed the
marking. The system can also be configured on a perassignment basis to send emails
immediately when individual marks are submitted.
Other forms of output can be added to the system as necessary. At the moment the
following are available:
an import file for WebCT
an import file for the SFU Computing Science Gradebook system
a spreadsheet import file
These files can be used to export marks from this system and into another course
management system.

4. Experiences
My experience using the system has been good. The TAs have been happy with it because
it streamlines the feedback process and eliminates some of the repetitiveness of marking
many assignments. It also keeps all of the marking information organized, ie. there is no
possibility of misplacing assignments or completed marking.
I have found that having the marks stored electronically is very valuable when students
come with concerns about their marking. I can find their marks in the system and quickly
review the comments the TA made. This does not rely on the students to bring in a
marking summary or on their memory of the comments they received (which is often
inaccurate).

If I want to change the marks for a student, I can quickly load the data currently in the
system, edit the marks and comments and then resubmit. The entire process can also be
done asynchronously, by email.
An unexpected benefit, as an instructor, is that I can keep track of the marking while it
is in progress. I can easily check to see how many assignments the TAs have marked and
thus have a good idea of when they will be finished. I can also spotcheck a few
assignments early in the process to make sure the marking is being done with due care
and attention. I have also found it useful to be able to look back on the details of a
student's performance over the semester when making difficult pass/fail decisions for
borderline students, and when writing letters of reference.

5. Future Work
While the marking system is currently in a usable state, there are still improvements to
be made.
The user interface needs improvement to allow the most common tasks to be performed
more quickly. In particular, it should be easier to add common problems directly from
the marking screen and the process of entering and looking up marks should be more
tightly tied together.

5.1. Feature Additions

Currently, the system does not import any data about the class. It simply records marks
for the userids that the marker enters. It is easy to enter a userid incorrectly, which then
causes failed emails and missing marks. It would be useful to incorporate a class list into
the system and warn when information is entered incorrectly, or could allow them to
choose from a list of students. This would also allow markers to know how many students
do not have marks and who they are.
The current authentication method is HTTP basic authentication. This is handled by the
web server, so that users arrive into the system with an authenticated userid. This may
not be possible for all systems. It would be possible to extend the system to keep track
of user's passwords and authenticate internally.
The system was designed to focus on mark recording and giving feedback to the students
by email. It would be possible to add student access to the system, allowing them to see
their marks via a web interface. This would require students to be given accounts and
passwords for the system; at SFU their Gradebook accounts could be used. In general, I
have found that email feedback is sufficient.

5.2. Portability
The current system makes several assumptions that are local to SFU. These include the
format of student numbers and userids. There are also several places where an @sfu.ca
email address is assumed for the students and markers. The code has been written so it
can be easily generalized to other institutions. However, there still remains some work to

be done to implement this.

5.3. Release
I would like the code to be reviewed by a third party for security before it is released.
Once this is done, the code will be released under the GNU Public License. I am currently
using the software in several courses and consider it secure Anyone wishing to
experiment with the system before its general release can contact me for more
information.

6. Conclusion
Overall experiences with online marking have been positive, particularly when coupled
with online assignment submission. It has eliminated the problem of processing and
returning hard copies of assignments.
As an instructor, it is one more tool that helps in keeping track of large classes. With
many students and TAs, the online marking system provides a central location for all of
the assignment marks and comments.
Greg Baker
School of Computing Science
Simon Fraser University
Burnaby, BC, Canada
V5A 1S6
ggbaker@cs.sfu.ca
http://www.cs.sfu.ca/~ggbaker/
Return to Greg's Home Page.
Copyright Greg Baker, last modified April 12, 2003.