Memorandum

To: Professor Malesh

From: Kayla Smith
Date: 03/10/2017
Topic: User Test Data Analysis For wikiHow Instruction Set

Instruction Set Topic: How to Start a Theatre Group

Professor Malesh, I am writing to update you on the user testing I performed on my instruction
set and the changes I made to better them.

I wrote instructions on how one could start their own theatre group. While looking for a topic on
wikiHow, I also made sure to look for a need for answers in areas I was familiar with. I went
straight to the Arts and Entertainment category and found the need for an instruction set on How
To: Start a Theatre Group. I have actually created my own non-profit, volunteer theatre group,
so I knew I could efficiently and accurately create an instruction set for that particular need in the
wikiHow community.

Objectives

The objectives of my user tests were very simple. I wanted to make sure a wide variety of
readers could understand my instruction set. I wanted to work out any confusing terms or steps
so that people in varying level of expertise could all equally understand the instructions.

I needed to pick three users with those varying degrees of expertise. My novice user would help
me make sure the wording was simple enough to understand. My intermediate user would help
more so with the layout of my instruction set and see if any further detail was needed. My expert
user would be able to tell me if my instruction set is like what they used to create their own
theatre group and validate the legitimacy of my instruction set.

All the input from my test readers would help me better my instruction set and to get the
instruction set successfully published on wikiHow. Conducting user testing would be the best
possible way to do this.

Methods

I chose to perform three different user tests with each of three users. I made sure to choose my
test readers based off of varying experience levels. I needed to test for the wide scope of readers
on wikiHow, so choosing my test readers based off of varying experience levels was the best
choice to make sure my instruction set was successful.

My novice user had no prior experience in starting a group of any kind and had no knowledge at
all of practicing theatre arts. My intermediate user had no prior experience in starting a group of
any kind, but knew the elements of theatre and had knowledge of practicing theatre arts. My
expert user had aided in the start-up of a theatre group and has a history of practicing performing
theatre arts.

I wrote out questions for myself to keep in mind while administering an observation usability
test. To further clarify these observations, I wrote questions to ask in an interview while
conducting my readability test. I also set up a location test for each test reader to ensure that all
readers could navigate my instruction set in a timely fashion.

I conducted three user tests. With each user, I first administered a usability test using
observation/concurrent think aloud method testing. I made note of any weird looks the test
readers made and made sure to ask what they thought was wrong with that section. To further
clarify my observations, I asked my test readers the questions I was basing my observations on,
and then wrote down the simple yes or no answers.

Second, I administered a readability test using an interview to draw out explanations on my

observations from the test readers. After my test readers had completely read through the
instruction set, I asked them each a list of questions. I was looking to even further explain my
observations and make substantial improvements in my instruction set.

Last, I administered a location test in which I asked the test readers to navigate my instruction set
and read specific parts and their steps. I asked them to find a specific Part of my instruction
set, timed them, and then asked them to read all of the steps in that part, and timed them again.

Results

The results for my usability testing with the observation/concurrent think aloud test, for all three
users, were relatively consistent. None of my users had trouble understanding the wording of my
instruction set nor thought any terms needed to be further defined. According to all three users,
there were no unnecessary steps.

While my novice and intermediate users did not see the need for any steps to be added, my
expert user did. They mentioned the need for steps saying to actually make choreography,
schedule practices, and schedule performance dates. The addition of these steps helped me
complete my instruction set and get it published.

The results for my readability test were collected through interviews with each of my users. All
of my users gave the same basic answers, even though some of their explanations were different.
All three users said that the steps flowed well, no steps needed to be switched around, the
instruction set was easy to follow, the instruction set looked nice, and that the instruction set
could actually be used as an outline to start other groups. (See Appendix A-C for full user
explanations)

The results for my location test were extremely close from user to user. The time taken to find
the named Parts and Steps was within a second or two of difference while comparing the
data collected (See Appendix A-C for the exact times). This showed that multiple people could
navigate my instruction set with great ease and consistency.
Along with my three test readers, I was frequently contacting and being contacted by a wikiHow
editor named Anna. At first, I was having a difficult time in finding out exactly what the
wikiHow community wanted from my instruction set. The messages I received from Anna are
attached in Appendix D. If I had not tried to relate what she said in her messages back to the
notes from my expert user, my instruction set would not have been published. I used the advice I
received from my expert user to answer Annas concerns and was published.

Conclusion

Each of the user tests conducted illustrated the points in my instruction set that needed to be
improved.

I added the suggested steps, tips, and warnings my test readers suggested. I went into the testing
thinking my instructions were solid, but I was completely open to any suggestions they had for
improvement.

Through the advice of my test readers, I added a tip (Keep lots of lists to keep yourself
organized), a warning (You will need to have experience in theatrical performance, dance,
and/or choir), and three steps (Make your choreography, Schedule practices with your cast,
and Schedule performances with the facilities you wish to perform at) to make my instruction
set as complete and efficient as possible.

If you have any questions please contact me at fv7946@wayne.edu or

KaylaRSmith42@gmail.com
 Appendix A: User One Data
User Testing Tests: Novice
UsabilityConcurrent Think Aloud
Make Note:
o Do they understand the wording of the instructions?
Yes. There appears to be no trouble with saying the instructions out loud.
o Should you further define any terms?
No. None of the terms were too difficult or not explained enough.
o Was any step unnecessary?
No. All steps seemed to be necessary for my novice user to follow.
o Do any steps need to be added?
No. There were no areas that needed an extra step or to be broken down.
Listening to my novice user, nothing was too long or jumped topics.

ReadabilityInterview
Do you think any of the steps should be switched around for convenience?
No. The steps seem to flow in the order they are in now.
Following these instructions, do you think you know what you have to do to start your
own theatre group?
Yes. It gives the basic idea of what to do. I did notice, though, that you dont
include all the professional background in the steps.
Do you feel the instruction set was easy to follow?
Yes. I had no issues.

Does the instruction set look nice?

Yes. It looks neat.
Do you think this instruction set would be an effective outline to start any other groups?
Yes, if you only look at the basic steps and Parts heading names.
Should I add any other disclaimers (warnings)?
If any, state that not all groups will be immediately successful, but I dont think
you need to say that.

LocationThink Aloud/Find Parts

Find The Part (In Seconds) List The Steps (In Seconds)
Part 1 2 7
Part 2 2 10
Part 3 1.5 12
Part 4 2 13
 Appendix B: User Two Data
User Testing Tests: Intermediate
UsabilityConcurrent Think Aloud
Make Note:
o Do they understand the wording of the instructions?
Yes. There appears to be no trouble with saying the instructions out loud.
o Should you further define any terms?
No. None of the terms were too difficult or not explained enough.
o Was any step unnecessary?
No. All steps seemed to be necessary for my novice user to follow.
o Do any steps need to be added?
No. There were no areas that needed an extra step or to be broken down.
Listening to my intermediate user, nothing was too long or jumped topics.

ReadabilityInterview
Do you think any of the steps should be switched around for convenience?
No. The steps flow.
Following these instructions, do you think you know what you have to do to start your
own theatre group?
Yes.
Do you feel the instruction set was easy to follow?
Yes. Nothing seemed to be out of place.
Does the instruction set look nice?
Yes. It looks very professional.
Do you think this instruction set would be an effective outline to start any other groups?
Yes, it could be.
Any re-wording necessary?
-Change Part 4, Step 1 change List of Songs to Possible Performance
Pieces
-Change Part 4, Step 2 cut off for each song
-Add tip Keep lots of lists to keep yourself organized.
-Fix warning need to may need and say you can rent a space, then if
needed you can fundraise for the money.

LocationThink Aloud/Find Parts

Find The Part (In Seconds) List The Steps (In Seconds)
Part 1 1 8
Part 2 2 9
Part 3 2 11
Part 4 2.5 11
 Appendix C: User Three Data
User Testing Tests: Expert
UsabilityConcurrent Think Aloud
Make Note:
o Do they understand the wording of the instructions?
Yes. There appears to be no trouble with saying the instructions out loud.
o Should you further define any terms?
No. None of the terms were too difficult or not explained enough.
o Was any step unnecessary?
No. All steps seemed to be necessary for my novice user to follow.
o Do any steps need to be added?
Yes. Add steps at the end for making choreography, practices, and the
scheduling of performances.
ReadabilityInterview
Do you think any of the steps should be switched around for convenience?
No. I mean you could create a t-shirt at any point, but it wouldnt matter.
Following these instructions, do you think you know what you have to do to start your
own theatre group?
Yes I do.
Do you feel the instruction set was easy to follow?
Yes, no issues.
Does the instruction set look nice?
Yes, visually pleasing to the eye.

Do you think this instruction set would be an effective outline to start any other groups?
Yes, as long as you drop the theatre elements and replace them with elements
related to your new group.

LocationThink Aloud/Find Parts

Find The Part (In Seconds) List The Steps (In Seconds)
Part 1 1 7
Part 2 3 9
Part 3 2 10
Part 4 3 12
Appendix D: User One Raw Data
Appendix E: User Two Raw Data
Appendix F: User Three Raw Data
Appendix G: Messages From
wikiHow Editor