1

Lego Instructions User Test Report

Group: Big Builders
Group members; Jake Owens, Caleb Collins, Will Erwin, Brock Downey
March 31st, 2017

Table of Contents
I. Cover Page: Page 1
 2

II. Table of contents: Page 2

III. Introduction: Page 3
IV. Executive Summary: Page 3-4
V. Participants & Methodology: Page 4
VI. Evaluation Tasks: Page 5
VII. Results: Page 5
VIII. Task Completion Success Rate: Page 5
IX. Errors: Page 5-6
X. Test Reader Comments and Recommendations: Page 6
XI. Conclusions: Page 7
XII. Likert-Scale: Page 8-9
XIII. Observation Sheet: Page 9
XIV. Follow-up interview: Page 9

Introduction:
The goal of this project was to create a lego instruction set from scratch and to improve
the instruction set through the use of user tests. The instruction set is meant to guide a reader to
 3

the efficient assembly of a lego vehicle. The instruction set includes worded instructions,
diagrams, visuals, and can include warnings to aid the reader in the building of the lego.
Our group made up 4 members worked closely to create an instruction set based on the
known final lego vehicle design. The group members split tasks by separating observations and
conducting post-test questions, while overall working collaborating on as a group on all aspects
of the project. The user test primarily consisted of an observation sheet, user feedback, follow
up questions, and a likert scale. The observation sheet and follow up questions were completed
by our group members. The user feedback and the likert scale were completed by the user test
participants upon completion of the user test group build.

Executive Summary:
The usability test took place in class at a table. The participants were provided with
supplies and the instruction set. No further help was provided and they were asked to build the
lego structure as best as they could. The test was conducted by the participants as our group
analyzed and observed the process. During the usability test, any trouble that the participants
experienced was observed and reported.
The number of participants included four college aged students. The group consisted of
all girls. The length of time it took them to complete the build was approximately 15 minutes.
The post survey session lasted about five minutes. The results obtained from the usability test
and survey questions helped our group to improve and add to our instruction set.
The participants were able to construct the lego vehicle; however, they did not build the
exact model that was the goal of the instruction set. The model that they built was very similar in
construction, except they were missing a few small details. These issues came about mostly
from the participants inability to understand our instructions. We believe that the group of girls
had very little experience with lego building based on the feedback they gave. This must be
considered because the lego instructions should be understandable and readable by anyone.
The list of issues the participants encountered was compiled in a list:
The instructions did not include enough steps
The pictures were, at times, somewhat difficult to see
There was a lack of expertise in the field of constructing legos
The resources used by the participant group consisted of an instruction set with pictures
and short captions describing the pictures. Individual lego pieces within the pictures were
described within the text to the left of the picture. Aside from the pictures and text, the only
additional information provided was the final lego vehicle picture from the box. The goal of the
 4

usability test was to let the tested group try to build the lego set with only the aid of the
instructions we created. This was done successfully with only one minor error.

Participants and Methodology:

The User Test Report participants were selected by switching our lego instruction set
with another groups lego instruction set. The group who participated in our user test included 4
white girls, college students with a basic understanding of the user test process. The only other
notable feature is that the group of girls has had some experience creating their own lego set
and therefore have some perspective to help them build our lego vehicle. These girls used our
lego instructions set to build the lego vehicle. We analyzed the group as they built the lego
vehicle with our created instruction set.
The user test took approximately 15 minutes and was met with some difficulty. The
participants began with an error by excluding to add a brick. The test continued with little issue
except that the girls struggled to build the lego fast. We took notes on the process and this
helped us in the analysis of our instruction set. The participants completed the lego set and
were then asked interview questions.
The participants were asked to construct a lego car using the instructions we provided
for them. At first, they had difficulty getting started but once they were able to build the frame
pieces started falling into place. For better understanding, the participants were asked to follow
the pictures within the instructions. In the post test sessions, the participants were asked to give
us feedback about the instruction set. Theyre advise included: adding pictures between certain
steps for clarification, labeling the lego pieces more accurately, and providing additional steps
when two or more lego pieces needed to be put together before being added to whole build.
Another post test session asked the participant to take a survey about the ease of build and
general feedback. In this section, most of the feedback was verbal and contributed to positive
criticism of the instruction set.
Evaluation Tasks:
1. Constructing the Lego vehicle: Brock, Caleb, Jake
2. Writing verbal instructions: Caleb, Will
3. Organizing Instruction Set: Jake
4. Clarity of visual design: Jake, Brock
5. Editing: Brock, Will

Results:
 5

Jake and Will filled out the observation sheet as well as basic mental observations in
order to determine what errors or flaws exist in the instruction set.

Task Completion Success Rate:

Tasks Easy to Somewhat Neutral Somewhat Very difficult
comprehend easy difficult to to
comprehende comprehend comprehend
d
Task 1 X

Task 2 X

Task 3 X

Task 4 X

Task 5 X

Time on Task: The participants took approximately 10-15 minutes to complete the lego build
with a small error in the build.

Errors:
Jake Owens recorded the one error when it occurred when our test users were putting
the car together. Jake was observing the test users as they were using our instructions and
noticed that in one of the first steps, they group conducting the trial had missed a step. Jake
realized that our instructions may have been a little unclear and that is why they missed the
combination of the first two lego pieces.
The task that participants made the most errors involved the first step of putting the lego
car set together. On our instruction steps, we didnt add a picture that displayed what two pieces
to put together first, but they were written in our instructions. Because there was no picture
attached with the step, it was apparent how the participants were confused and missed that vital
step. Since they missed the first step, the end result of the car were not 100% percent correct.
The frame of the body when they finished was too low which meant the tired were not able to
turn and the car was not able to move.
The one error the user test group made did impair the car in the way was meant to be
used. The issue was that the frame of the leg car was too low and did not allow the car to roll on
 6

the tires. Once we identified the problem all we had to do was remove the frame of the car from
the base of the car and add a 2x2 lego ontop the rear axle to give the frame and tire some
space, which allowed the car to roll.

Test Reader Comments and Recommendations:

A few comments from test readers were not critical but instead things that our instruction
set did well. The readers all agreed that the instruction set was overall satisfactory. They liked
the clarity of most pictures and how they were aided by a caption next to the picture. They
agreed that the instructions were very well organized and had a very nice flow in the order with
which they were presented.
Some of the comments received helped our group narrow down which parts of the
instruction set could use improvement. The readers prefered that we use clearer pictures and or
more detailed written captions. Some parts between certain steps were not clear enough or
were not mentioned at all. The best advice they had and a good middle ground was to have our
written instructions clarify what our images can not show well enough.
The readers all recommended that we add more, smaller steps between the overarching
steps of the build. They also recommended that we describe the individual lego pieces in more
detail throughout the instruction set. The group suggested we should use more useful words for
the lego pieces. This criticism was determined invalid because the words they told us to use
would not make sense to someone familiar with legos.

Conclusion:
In conclusion, the general consensus was to elaborate more on our written instructions
and to adjust some of the pictures in the instruction set. These critiques helped us to add and
edit some of the aspects of the instruction set. This allowed us to experience the process of
creating a formal instruction set and then edit and improve the instructions based on a user test.
 7

Attachments-Details

Likert-Scale Test Questions:

Strongly Disagree Slightly Neither Slightly Agree Strongly Agree

Disagree Disagree Agree nor Agree
Disagree

The
instructions
were clear
 8

and easy to
follow.

I would
recommend
these
instructions to
a friend.

Using the
instructions
was easier
than figuring it
out on my
own.

The
instructions
had a user-
friendly
interface.

The
instructions
were missing
key
components.

Observation Sheet for User Test:

Observation Sheet

Problem Interpretation Solution

(What issues did you (What may have caused (What might help avoid these
experience?) these issues?) issues?)
 9

Follow Up Interview Questions:

What was the most appealing part(s) of the instruction set?
What part(s) of the instruction set were unclear/needed more work?
Did the instructions take too long or not cover enough steps?
Was there a specific instruction that had you confused, or a specific instruction
that seemed out of place?
Any other comments/concerns?