EVE ONLINE

KILLBOARD USABILITY
STUDY
Eve-Kill.net

Z-Killboard.com

Page 1 of 32

Table of Contents
INTRODUCTION

METHODS

RESULTS

TASK #1: FINDING THE MOST VALUABLE SHIP LOSS IN THE LAST 7 DAYS

EVE-KILL.NET

Z-KILLBOARD.COM

TASK #1 SUMMARY

TASK #2: FINDING THE CHARACTER RANKED 3RD FOR MOST KILLS

EVE-KILL.NET

Z-KILLBOARD.COM

TASK #2 SUMMARY

TASK #3: FINDING THE MOST RECENT KILL FROM A MEMBER OF TEST ALLIANCE PLEASE IGNORE 9
EVE-KILL.NET

Z-KILLBOARD.COM

TASK #3 SUMMARY

TASK #4: FINDING THE THREE MOST RECENT SHIP LOSSES FOR THE CHARACTER KILLER GX,
9

AND NAME THOSE THREE SHIPS

EVE-KILL.NET

Z-KILLBOARD.COM

TASK #4 SUMMARY

10

TASK #5: FINDING THE MOST RECENT AVATAR SHIP LOSS

EVE-KILL.NET

10
10

Page 2 of 32

Z-KILLBOARD.COM

11

TASK #5 SUMMARY

11

TASK #6: FINDING THE MOST VALUABLE CAPSULE LOSS IN THE LAST 7 DAYS

11

EVE-KILL.NET

11

Z-KILLBOARD.COM

11

TASK #6 SUMMARY

11

TASK #7: CHECKING RECENT ACTIVITY LEVELS IN SOLAR SYSTEM AOK-WQ

12

EVE-KILL.NET

12

Z-KILLBOARD.COM

12

TASK #7 SUMMARY

12

TASK #8: ENEMY ACTIVITY LEVELS IN SOLAR SYSTEM F4R2-Q

12

EVE-KILL.NET

12

Z-KILLBOARD.COM

12

TASK #8 SUMMARY

13

TASK #9: RED VS BLUE OCTOBER KILL COUNT

13

EVE-KILL.NET

13

Z-KILLBOARD.COM

13

TASK #9 SUMMARY

13

TASK #10:THE MOST EXPENSIVE LOSS FOR CHARACTER PEDROSUAVE

13

EVE-KILL.NET

13

Z-KILLBOARD.COM

14

TASK #10 SUMMARY

14
Page 3 of 32

RECOMMENDATIONS

16

Page 4 of 32

*note*
As of 10/14/2014, I have been approved to use an external source to do the usability test. I made this
decision because I wanted to see how intuitive these two sites are to someone not familiar with the
game they are based on.

Introduction
This white paper compares the usability of two websites, eve-kill.net and z-killboard.com. Both of these
websites are used in conjunction with a popular online PC game, eve online. In eve online, a large virtual
world filled with players is in constant combat with space ships. Ship losses are permanent, and the game
records every detail of these losses. Websites like eve-kill.net and z-killboard.com comb through the
game for these details, acting as a data cache for players to use.
Methods
Testing was done with both websites in identical conditions, in an attempt to cancel out as many variables
as possible. The conditions are as follows:
-Testing was done in my own apartment, in order to remove environmental variables such as doors
opening and people entering/leaving testing area
-Two screens are connected to my computer, however, this is distracting as only one screen is needed.
Therefore one screen was turned off during the test
-For both websites, I have kept the applications running on my computer consistent. This eliminates
variables such as the computers running speed.
-The computer desk was clear of distracting objects, and remained the same during both tests
-The lighting and temperature of my apartment remained consistent
-The time of testing was consistent (7pm PST)
The individual tasks were chosen based on typical use of a website such as eve-kill or z-killboard. Some
of these tasks are meant to be quick checks; gathering intel about an enemy nearby and using that
information to decide whether or not to engage that individual. Other tasks are more involved, pertaining
to relative activity levels of entire enemy groups (helping to determine what level of defense is needed in
the near future). The task list is shown below:
-Task #1:
-Task #2:
-Task #3:
-Task #4:
-Task #5:
-Task #6:
-Task #7:
-Task #8:

Finding the most valuable ship loss in the last 7 days

Finding the character ranked 3rd for most kills
Finding the most recent kill from a member of test alliance please ignore
Finding the three most recent ship losses for character killer gx, and naming those ships
Finding the most recent loss of an avatar (one of the largest ships in the game)
Finding the most valuable capsule loss of the last 7 days
Checking recent activity levels in the solar system AOK-WQ
Checking enemy activity levels in the solar system F4R2-Q
Page 5 of 32

-Task #9:
-Task #10:

Red vs Blue October kill count

The most expensive loss for character pedrosuave

Results
Results are organized with a list of characteristics:
Constraint:
Range of values: None evident/mild constraint/moderate constraint/heavy constraint/major constraint
Explanation: what does the product prevent you from doing? Is there a task you could not do?
Ease-of-use:
Range of values: Very easy/fairly easy/moderately easy/difficult/extremely difficult
Explanation: How easy or difficult it is to perform a task
Aesthetics:
Range of values: Very aesthetic/moderately aesthetic/mildly aesthetic
Explanation: How the style and appearance of the website enhances or inhibits the task
Time:
Range of values: 0-60 seconds
Explanation: The time it takes to complete a task
Mouse Clicks:
Range of values: 0-infinity
Explanation: The number of mouse clicks performed to complete a task

Page 6 of 32

Task #1: Finding the most valuable ship loss in the last 7 days
Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Extremely easy
Very aesthetic
5 seconds
0

Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Extremely easy
Very aesthetic
4 seconds
0

Task #1 Summary
This task is a very common one-people are always searching for the big kills in eve. Because of this
fact, this task was easy and fast for both websites.

Figure 1: Eve-Kill.nets front page. Both websites have the same background on the top of the page, with
the format for most expensive kills being virtally identical.

Page 7 of 32

Task #2: Finding the character ranked 3rd for most kills
Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Mildly aesthetic
10 seconds
0

Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Very aesthetic
5 seconds
0

Task #2 Summary
Here we see a distinct difference between the two websites: background colors. Z-killboard.com uses a
blue background, with Eve-Kill using a red background with white lettering:

Up close these color differences are not particularly hard to see, but at a normal difference from a monitor
(as was the case in the testing), the red background prevents the user from quickly discerning information.
Page 8 of 32

As a result, the user took more time to gather information from Eve-Kill.net.

Task #3: Finding the most recent kill from a member of test alliance please ignore
Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Mildly aesthetic
25 seconds
4

Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Difficult
Very aesthetic
37 seconds
2

Task #3 Summary
This task produced conflicting results. Although Z-killboard required less clicks to perform the task, it did
not have clear wording to indicate how to search for an alliance. Eve-Kill begins by clicking the search
button, then having a list next to the search window asking what youre searching for. Z-Killboard has no
list, just a simple window with no wording next to it. This problem is similar to what google encountered
in its early beta stage: google was so simple that people didnt know how to initiate a search. In other
words, Eve-Kill is easier to learn, while Z-Killboard is faster once you learn how its search function
works.

TASK #4: FIND THE THREE MOST RECENT SHIP LOSSES FOR THE CHARACTER KILLER GX,
AND NAME THOSE THREE SHIPS

Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Moderately Easy
Mildly aesthetic
20 seconds
4

Z-Killboard.com
Page 9 of 32

Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Very aesthetic
15 seconds
3

Task #4 Summary
Now that the user has understood both search functions from task #3, the initial search on both websites
went much faster. The main issue was with Eve-Kill on locating ship types-the red background made it
challenging to discern all of the information on the page.

Note the color change on each level-why change the shading? This gave the user a confusing visual
stimuli. From this area it is not clear what ships are losses and what are kills-and it is not especially easy
to know what the ship type is unless you are familiar with the game.

Task #5: Finding the most recent loss of an avatar (one of the largest ships in the game)
Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:

Major Constraint
Extremely difficult
Mildly aesthetic
60 seconds
Page 10 of 32

Mouse Clicks: 12
Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Very aesthetic
20 seconds
6

Task #5 Summary
Task #5 proved nearly impossible on Eve-Kill, due to the fact that the site does not have any specific
search function for ship type. The only way to find an avatar ship loss was to manually search through the
entire website for the ship, which would take a very long time since an avatar only dies once every month
or so. The user gave up around one minute on Eve-Kill, whereas on z-killboard the search function
included ship-types. The only issue on z-killboard was an accidental click on kills by an avatar instead
of avatar losses. This primarily seemed to stem from the links only being titled kills and losses. This
is unclear-do they mean kills from an avatar or avatar kills?

Task #6: Finding the most valuable capsule loss of the last 7 days
Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Moderately aesthetic
15 seconds
0

Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Very aesthetic
10 seconds
0

Task #6 Summary
Task #6 went well overall, with capsule losses being on both websites front page. Eve-Kill took a bit
longer to locate, stemming from the red background that makes it hard to read.

Page 11 of 32

Task #7: Checking recent activity levels in the solar system AOK-WQ
Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

Mild constraint
Moderately easy
Fairly aesthetic
13 seconds
6

Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Very aesthetic
9 seconds
3

Task #7 Summary
Specifically, the task was defined as: Find the ship losses from September 28th in solar system AOK-WQ,
and compare to the ship losses in the same system on October 6th. Z-killboard has an easy to read kill list,
making it easy to view losses chronologically. Eve-Kill does not have any separation of kills by date, with
only a small time stamp on the bottom right. This made Z-killboard much easier to use and required less
mouse clicks.

Task #8: Checking enemy activity levels in the solar system F4R2-Q
Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Moderately easy
Fairly aesthetic
23 seconds
9

Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:

None evident
Fairly easy
Very aesthetic
18 seconds
Page 12 of 32

Mouse Clicks: 7
Task #8 Summary
Specifically, the task was to find the amount of kills an alliance called curatores veritatis alliance was
responsible for in the solar system F4R2-Q on October 6th. Now that the user has performed previous
tasks locating an alliance, the initial search went easy enough for both websites. The one hangup was on
Eve-Kill, namely having no indication that more ship losses from the same date are on the next page. The
user initially thought the website might have accidentally left some ship losses out, and after a 6 second
delay realized it may be on the following page.

Task #9: Red vs Blue October kill count

Eve-Kill.net
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Fairly aesthetic
14 seconds
6

Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Very aesthetic
18 seconds
5

Task #9 Summary
The specific instructions were to look at the kills from two different alliances: Red Federation and Blue
Republic. These two alliances are in constant war with each other on purpose, and try to keep the
fighting as even as possible. I did not anticipate both of these alliances being on the front page of both
websites, which made the search relatively fast. Instead of instinctively going to the search bar, the user
looked over the front page, where he eventually found both alliances on the bottom right.
Task #10: The most expensive loss for character pedrosuave
Eve-Kill.net
Constraint:
Ease-of-use:

Mild constraint
Fairly easy
Page 13 of 32

Aesthetics:
Fairly aesthetic
Time:
42 seconds
Mouse Clicks: 15
Z-Killboard.com
Constraint:
Ease-of-use:
Aesthetics:
Time:
Mouse Clicks:

None evident
Fairly easy
Very aesthetic
32 seconds
12

Task #10 Summary

This character has quite a few ship losses over his entire game history, so I did inform the user that the
most expensive loss was in June of 2013. Z-killboard was more intuitive here, specifically with ship
prices.

Page 14 of 32

However, Eve-Kill did not have the prices listed, requiring the user to click on individual kill logs before
finding the price:

Page 15 of 32

Recommendations
Both sites store virtually identical information, but organize this data in different ways. As a typical user
of these types of websites my highest priority is the time taken to achieve a task, and in this case zkillboard offers the best results. Not only is z-killboard usually faster, it is more intuitive, easier to read,
and contains the important information in better locations (primarily on the left, near the top).

Page 16 of 32

Page 17 of 32

Figure 1: Eve-Kill.nets front page. Both websites have the same background on the top of the page, with
the format for most expensive kills being virtally identical.

Page 18 of 32

Page 19 of 32

TASK #1: FINDING THE MOST VALUABLE SHIP LOSS IN THE LAST 7 DAYS

Page 20 of 32

Page 21 of 32

Page 22 of 32

Page 23 of 32

Page 24 of 32

Page 25 of 32

Page 26 of 32

Page 27 of 32

Page 28 of 32

Page 29 of 32

Page 30 of 32

Page 31 of 32

Page 32 of 32