The will help to in order to how?

Tech, biz or people

based?
Range How to test?
How do we know it's a
good test?
1
Page with all categories all forumusers navigate the forum find the information they need
In a quick and efficient manner; simple and
engaging design; will use tiles and minimal
information, as well as images
Tech (design and
implementation)
At least 70% of users should be
able to choose the right category
in 10 to 20 seconds
Ask user to find the right place to
ask question
Ask user if he's happy with
outcome; if our test result is
positive and coincides with
the user's impression, then
it's a good one.
2
Page with list of topics of
given category
all forumusers see topics for each category
select which ones they want to
read
simple but with all the minimumdata; when
hovering over certain elements, see additional
info to help decide
Tech (design and
implementation)
At least 70% of users should be
satisfied with the topics they
selected
Ask user to find answer to a
problemand provide feedback on
UI
Ask user if he's happy with
outcome
3
Page for each discussion all forumusers read discussion solve or prevent security problems;
easy to read: give priority to the relevant
information in the discussion (Q & A), comments
are secondary
Tech (design and
implementation)
Users should be able to decide
whether content is relevant in
between 1 to 3 minutes.
Ask user to find answer to
problemand say when he's found
it.
Ask user if he's happy with
outcome
3.1
Answer form
knowledge
providers
answer questions
build online reputation / showcase
skills; promote brand if they are
service or product provider (see
item22)
Part of discussion page; easy to fill; visually
simple; whenever possible: "tick the box" rather
than "fill in the box"; provide user with feedback
about the quality of his post - reflect this on his
profile and use for reputation purposes
Tech (design and
implementation)
at least 70% of users should be
able to understand how to use
the form
Ask user to answer a question
and tell us their thoughts as they
do so
Ask user if he's happy with
outcome
3.2
Comments all forumusers
comment without formally
answering question
clarify or ask for clarification on
question / answer
Part of discussion page; easy to fill; visually
simple; desincourage long conversations: small,
discrete
Tech (design and
implementation)
first time users find where to
post a comment 5 to 15 seconds;
at least 70% of users find the
experience easy and pleasant
Ask user to add a comment and
talk as they do so; see how long it
takes
Ask user if he's happy with
outcome
3.3
Best Answer Ranking knowledge seekers see best answer first
speed up reading and selecting
information
combine user manual feedback with automatic
quality analysis
Tech (algorithm, design
and implementation)
at least 50% of automated
ranking should correspond to
human manual ranking
get fake data and ask humans to
rank answers; then, run the
algorithmon the data and
compare
test with real data once the
forumis live
4
Ask question form/ page knowledge seekers ask a question solve or prevent security problems
easy to fill; visually simple; whenever possible:
"tick the box" rather than "fill in the box"
Tech (design and
implementation)
User completes formin 3 to 5
minutes
Ask users to post a question and
check how long they take; ask
themto say what they think each
part of the formis for
Ask user if he's happy with
outcome
5
Breadcrumb all forumusers navigate the forum find the information they need simple and easy; use standard design
Tech (design and
implementation)
Users should be able to figure
out switch fromone category to
another in between 5 to 10 secs
Ask user to switch categories and
measure how long it took
Ask user if he's happy with
outcome
6
Settings page all forum users customise their settings
control how they experience the
forum
provide relevant default settings; simple and easy
UI
Tech (design and
implementation)
At least 70% of users should
understand all the settings; at
least 60% of users should be
happy with the default settings
Ask user to find and edit settings;
ask if (s)he understood all
elements; ask if happy with
defaults
Ask user if he's happy with
outcome
M
u
s
t

h
a
v
e

-

t
h
i
s

v
e
r
s
i
o
n
!
7
User profile / identification all forumusers
assert own identity and get to
know others'
build public presence / reputation;
promote business / brand; inspire
trust / formalliances or sub-groups
provide both short/relevant info on discussion
page and a user profile page; users are
encouraged (not obliged) to provide real
information rather than aliases, by having the
possibility to mention their business name and
add a logo; users may add more descriptive
information about who they are, their likes and
dislikes or hobbies, and the projects they are
working on; users may add links to their website
or to their profiles in other social media websites
Tech (design and
implementation)
At least 70% of users should
understand the information
provided; at least 50% of users
should trust user profiles of
active contributors enough to ask
thema question
Ask user to analyse mockup user
profile and describe how they see
the user: frequent user? Would
they trust them? Do they need
more information? Less?
Ask user if he's happy with
outcome
8
Tags all forumusers label content
contribute to effective indexation /
content organisation / facilitate
SEO
Mandatory requirement: at least 1 tag; simplify
experience: provide pre-set list of tags to choose
from; recommend relevant tags based on content
keywords; call them"keywords" so it's easily
recognisable
Tech (design, data
projection)
At least 60% of users should be
able to choose one or more
relevant tags; at least 70% of
those should find it easy;
performing a search after tagging
should bring up more relevant
results than before
Ask users to create and tag a post
and provide feedback
Once we have live data,
performseveral searches to
compare having / not having
tags.
9
Tags list all forumusers filter out the content they need find relevant content quicker
small simple list of tags, different in each
category; each tag is a link that brings up all
related content in the search results page; call
them"keywords" so it's easily recognisable
Tech (design and
implementation)
At least 70% of users should
know what this is for; 60% should
be able to choose what's
relevant for them; 30% should be
inclined to try it without need for
explicit request
Ask users to find the answer to a
problem, register who uses tags;
next, encourage themexplicitly
and note what they choose; ask if
they know what it's for.
Ask user if he's happy with
outcome
10
standard search engine
use pre-existing standard search engine available
in development platform to be used
Tech (implementation)
At least 50% of the results shown
in the first page should be
relevant to the search query
Perform test searches on mockup
data or live data from another
website
Perform test searches on live
data
11
Search bar
default search in current category (all categories
if we are in the categories list page); drop-down
allows choosing other categories; simple,
standard design; present in all the forumpages -
only the default search context changes
Tech (design and
implementation)
At least 70% of users should
understand how the search bar
works and be happy with it
Ask users to performa search and
provide feedback
Ask user if he's happy with
outcome
12
Search results page
analyse available relevant
content
choose which content to read
Display results for all types of content (forum
Q&A, learning modules, useful links, news and
events) in separate tabs; consistent with
discussions list (item2)
Tech (design and
implementation)
at least 60% of users should find
the page easy to read be able to
choose the right content
Ask users to performa search and
provide feedback (we need real
users search queries)
Ask user if he's happy with
outcome
13
Usage statistics page
Project manager,
moderators
analyse user participation and
needs
tailor content and strategies to
promote community growth and
security awareness
detailed and graphic; customisable
Tech (design and
implementation)
At least 70% of community
managers should consider the
information relevant and easy to
understand
Ask users to lookup community
statistics and provide feedback
Ask user if he's happy with
outcome
search forum find relevant content quicker
all forumusers
M
u
s
t

h
a
v
e

-

t
h
i
s

v
e
r
s
i
o
n
!
knowledge seekers
provide feedback on content
they access; know what
feedback others provided
decide which content to read or
trust; express their, opinions
Tech (design and
implementation)
At least 40% of users are inclined
to use at least one formof
content quality feedback; at
least 70% of the above find the
features easy to use and the
information understandable
Ask users to look at pages where
content feedback exists and ask
themto share their thoughts: do
they understand what they see?
what would they do?
knowledge
contributors
receive feedback on content
they create
feel rewarded for their
contributions and engage more
Tech (design and
implementation)
at least 60% of users are happy
to receive feedback;
provide mockup feedback on a
user's post and ask how they feel
about it - do they want to post
again? Is it personal or is it the
UI?
15
notifications of new posts
on subscribed topics
(discussions where they've
participated before)
all forum users
be informed of other's reactions;
stay informed on relevant
updates
stay connected; solve problems or
continue to contribute
simple, graphic and with engaging language;
possibility to opt out / choose which notifications
to receive
Tech (design and
implementation)
At least 80% of users are happy to
receive notifications and find
them clear
Give users sample notification and
ask for feedback: do they
understand it? What would they
do next?
After community has been
launched for a while; analyse
usage statistics - how often
are these features used?
16
Opt in / out of subscription
to specfic discussion
all forumusers
control which updates they want
to receive
stay informed / connected; not
receive irrelevant updates
default is automatically subscribed to discussions
where they previously joined; simple access to
feature, easy to use
Tech (design and
implementation)
At least 60% of users find this
option when needed, in 5 to 10
seconds
Ask user to subscribe, unsubscribe
froma topic; how long did it take;
do they like it?
Ask user if he's happy with
outcome
17
Moderation
all forum users /
Organisation that
owns the forum
have an organised and healthy
community
provide sense of security and trust;
ensure positive relationships
Initially moderators are appointed by owner
organisation and later they can be recruited from
the community; we should always keep
organisational moderators - they are responsible
for managing the others
People (human resources)
At least 80% of users appreciate
and respect moderators; after
maximum one year, some users
are suitable to have some
moderation permissions
simulate user interaction and
moderation intervention, analyse
reactions and ask for feedback
Analyse user behaviour after
community is launched - how
are they responding to
moderation?
18
Moderation features (needs
more detail before
implementing)
Moderators effect their permissions
manage community / organise
contents
provide several levels of moderation; efficient and
useful, easy to use
Tech (design and
implementation)
All moderators should find they
have all the essential features
they need; at least 80% should
find they have all the features
they need (essential and non
essential); at least 80% of all
moderators should find that the
UI is easy to use
Provide sample data and set of
features to moderators and ask
them to try them: is something
missing? How important is it?
How easy is it to use? Do they
understand it? How long does it
take to learn?
Re-test with live data?
all forum users know their rights and obligations
Moderators
know their rights and
obligations; legitimate their
interventions
20
Learning modules list knowledge seekers
see which modules are available
for particular category
select a relevant module if needed
small UI element with simple list, living in the
category page; preview of content; rating
information; most liked ones come first; "see all"
link to a new page with full list and more details
Tech (design and
implementation)
at least 80% of users should find
the list useful and clear, and
know how to go on fromthere
Ask users to analyse and test
feature - what do they think?
What is it for? What can we do
with it?
Ask user if he's happy with
outcome
M
u
s
t

h
a
v
e

-

t
h
i
s

v
e
r
s
i
o
n
!
After community has been
launched for a while; analyse
usage statistics - how often
are these features used?
Participation rules,
guidelines
19
have mutual understanding and
respect
Have a clear and detailed set of guidelines and
terms and conditions, legally valid; provide both
detailed and simplified version
Business (strategy, legal
requirements)
All users should understand at
least the simplified version of the
information; 80% of users should
understand both versions
Ask users to read guidelines and
provide feedback; provide users
with practical behaviour examples
and ask if they are accepted
Observe user behaviour after
community is launched
14
content quality feedback
(ratings; likes or thanks;
accept solution; how many
shares; how many links to
their content - see item29)
relevant information/functionality shown in the
right places and in a clear and visually engaging
way; use images / icons
21
Clear presence of owner
organisation
all forumusers
know that there is someone
who's responsible for managing
the community
inspire trust, especially in the initial
stages of community development
Header of every page contains reference to the
organisation; as well at the bottom; "contact us"
page and "about" make reference to the same;
moderators are properly identified and join
discussions when relevant
Business (organisational
information), people
(moderators), tech
(design and
implementation)
At least 80% of users should feel
confident that the forumis
legitimate and safe
Ask users who they think is
responsible for the forumand
what would they do if they had
an issue; do they trust that things
will go well? If not, is it personal
or is there something missing?
Ask user if he's happy with
outcome
knowledge seekers get expert advice
solve / prevent IT security
problems more efficiently; get a
quicker response; trust
at least 80% of users should find
this content useful and to have
good quality;
provide mockup content - do
users appreaciate it? Would they
follow that advice?
Analyse usage statistics with
live data
IT product / service
providers
showcase their knowledge and
skills
inspire trust and increase average
quality content; promote their
brand and help get more business
at least 60% of users should be
happy to use this resource to
promote their brand
explain concept to users and ask if
they would be happy to
participate
Analyse usage statistics with
live data
23
Option to report abuse all forumusers
report innapropriate behaviour,
spam, etc.
keep the community safe; feel safe quick, simple link or button available in each post
Tech (design and
implementation)
at least 90% of users should find
the feature useful, easy to find
and use
show sample content with
inapropriate behaviour - what
would the user do or look for?
Does it work well?
Analyse usage statistics with
live data
24
User register page all forumusers register into the forum
be able to ask a question or access
special content (ex. Learning
modules)
simple and easy to fill in; allows disclosure of
identity
Tech (design and
implementation)
at least 80% of users should find
it easy to create an account
ask user to create account and
provide feedback
Once we have live data,
track now many users
abandon the register process
25
Special content and
features for registered
users
owner organisation get users to register
get users engaged and be able to
track individual behaviour
when user tries to access special content (ex.
Some learning modules), invite themto register
Tech (design and
implementation)
at least 70% of users should feel
inclined to register
ask user to access special content -
they see the prompt to register -
how do they feel about it? Would
they do it?
Analyse usage statistics with
live data
26
links to the forum/
invitations in other areas of
the homepage
all potential users
become aware that the forum
exists
attract users and grow the
community
strategically placed links, simple and engaging
Tech (design and
implementation)
at least 60% of users should feel
inclined to visit the forum
show homepage and ask user to
say what they'd do; ask themso
say what they are thinking, where
do they think those links take
them?
Analyse usage statistics with
live data
27
Ask a question button knowledge seekers ask a question prevent or solve security issues
well visible, attractive; placed in home, topics list
and discussion page
Tech (design and
implementation)
at least 30% of users should feel
inclined to ask a question; 70% of
those should find the button easy
to find
ask users to solve a problem- do
they want to ask a question? Ask
themto ask a question - can they
find the button
Analyse usage statistics with
live data
28
Private messaging
all forumusers and
moderators
communicate in private
discuss / share information that
cannot or should not be discussed
in public, either for security reasons
or behaviour control; contact one
single user rather than the group
simple access and easy to use
Tech (design,
implementation)
at least 70% of users should find
this feature useful and easy to
use
ask users to contact one particular
user - can they figure out how?
How did they find the
experience?
Analyse usage statistics with
live data
product / service providers are allowed to post
solutions, so long as they don't directly sell or
advertise (see item36)
22
IT product / service
providers contribute to
forumcontents / posts /
solutions
M
u
s
t

h
a
v
e

-

t
h
i
s

v
e
r
s
i
o
n
!
Business (marketing
strategy), people
(contributor availability)
29
Email with summary email
of personal contribution:
posts / solutions statistics,
content quality stats,
rankings, badges (see item
19)
all forum users
be aware of their level of
contribution and their
achievements
Feel rewarded for their
contributions and engage more
simple, graphical information when suitable; link
to personal page for more info; regular
information sent by default once a month; user
can opt out or change regularity; integrate in the
newsletter?; do not send if there are no changes in
relation to last email / if user hasn't visited
community replace "with we missed you" email,
sent every 3 months until opt out
Tech (design and
implementation)
At least 30% of users should be
happy to receive this information;
at least 10% should continue
coming back to the forum
because of it
Give users sample email and ask
for feedback: how do they feel?
Would they want to continue
receiving it? Implement this
feature at a later stage, compare
usage statistics before and after:
how many users have opted out?
Is there an increase in returns?
If statistics show continued
higher engagement then our
test was good?
30
email notifications of
content quality feedback
actions (ex. "someone liked
your post")
all forum users
be aware of their level of
contribution and their
achievements
Feel rewarded for their
contributions and engage more
email sent each time there is something new;
possibility to opt out or configure which
notifications to receive; simple and engaging
language, congratulating user; use images, icons
Tech (design and
implementation)
At least 50% of users should be
happy to receive this information;
at least 10% should continue
coming back to the forum
because of it
Give users sample email and ask
for feedback: how do they feel?
Would they want to continue
receiving it? Implement this
feature at a later stage, compare
usage statistics before and after:
how many users have opted out?
Is there an increase in returns?
If statistics show continued
higher engagement then our
test was good?
31
badges (example: if a user
shares a lot of posts, he
gets the "sharer", or
"giver" badge)
all forumusers display their favourite activities
assert own identity; build
reputation / inspire trust; promote
empathy through identification of
common activities
simple icon associated to personal / profile
information; fun and engaging
Tech (design and
implementation)
At least 30% of users should be
happy to receive this
information; at least 10% should
continue coming back to the
forumbecause of it
Give users a badge thanks to
some action performduring
another test (surprise them); how
did they feel? Would they feel
encouraged to repeat the
behaviour?
If statistics show continued
higher engagement then our
test was good?
32
Invite someone to answer
(frominside or outside of
the community)
all forumusers
contribute to the community
content
grow the community, grow its
content; help users feel useful and
engage more
simple icon or list option, easy to use
Tech (design and
implementation)
At least 10% of users find the
feature helpful and would use it;
80% of those should find it easy
to use
Ask users to look at a question
they don't have an answer to;
what would they do? If not done,
ask themto test the feature and
ask for feedback
Once community takes off,
see how many people use
the feature
33
useful links list all forumusers
find additional content /
solutions
solve or prevent security problems
small simple list of links different in each
category; allow users to suggest useful links;
option to "see all" links brings up all links page
with search bar on it, default search to useful
links
Tech (design and
implementation)
At least 10% of users find the
feature helpful and would use it;
80% of those should find it easy
to use
Ask user to look for solution to
problem- does he use useful
links?; ask himto and ask for
feedback - it it useful? Easy to
use?
Once community takes off,
see how many people use
the feature
34
"recommended for you" all forumusers
find additional content /
solutions
solve or prevent security problems
based on usage statistics, recommend learning
modules, useful links, news.; display
information in forumhomepage; use engaging
language
Tech (algorithm, design
and implementation)
At least 50% of users find the
feature useful; 70% find the
proposed content relevant and
the information to be displayed
clearly and in engaging way
Give users sample information
and ask for feedback: is it useful,
easy to read? Would they click on
the links?
Analyse usage statistics with
live data
35
Off-topic discussions
category
all forum users
discuss issues outside of security:
information, business, chill out /
social
be able to find solutions for other
types of problems or simply
socialize; leverage people's needs
outside of security to keep them
engaged in this forum
relaxed informal context; fun
Tech (design and
implementation)
At least 30% of users should find
this category useful; at least 70%
of those should find it easy to use
Show users the global list of
categories and see if they're
curious about this one; ask them
to check if out and provide
feedback - would they join any of
the discussions or start a new
one?
Analyse usage statistics with
live data
N
i
c
e

t
o

H
a
v
e

-

t
h
i
s

v
e
r
s
i
o
n

o
r

n
e
x
t
?
36
Automatically pick up
domain of the most popular
suggested links and add
them to useful links list
all forum users have additional links find additional information
automatic, send to moderators next who will
confirm addition; only pick up really popular links -
we want to keep the list relatively small and down
to the essential
Tech (algorithm and
implementation)
at least 80% of automatically
picked up links should be
considered good by the
moderators
run test and ask users to analyse
results
Analyse usage statistics with
live data
37
RSS Feed more active users
receive the latest updates on
subscribed categories
stay connected; solve problems or
continue to contribute
simple and easy; standard
Tech (design and
implementation)
at least 50% of users should find
this feature useful and easy to
find
ask users to find and activate RSS
feed
Analyse usage statistics with
live data
38
Video tour of the forum first time users
learn what the forum is and how
to use it
feel comfortable and motivated to
start using the forum
Short, simple video that introduces concept and
shows main functionality; engaging language;
visible in the forum homepage
Tech (media development;
design and
implementation)
at least 80% of the users should
find the video attractive; at least
60% should find it useful
take users to homepage - do they
click on the video? Ask them to
watch - ask what is the forum for?
Ask them to test some features - is
it easier after watching the video
Allow rating of the video;
analyse usage statistics once
the site is live
39
Points / reputation system all forumusers
be rewarded for their
contributions; understand the
value of contributions; identify
other strong contributors in the
community
keep themengaged and continuing
to contribute; allow users to know
who they can trust more to give
themgood information
points for anything that shows positive
contribution; directly reflected on user's profile,
visible to the rest of the community; when points
are attributed - convey message in an
encouraging way
Tech (algorithm; design
and implementation)
at least 10% of users should feel
inclined to contribute more
because of this system; at least
60% of users should trust users
with high reputation
same as user profile /
identification test; also reward
user for having positive
contribution and ask how he
feels, would be contribute again
Analyse usage statistics with
live data
40
Sharing feaures: social
media; send by email
option
all forumusers
share relevant information in
other social platforms; with
other people
touch base with the wider
community; attract new users to
this community
simple and easy; standard
Tech (design and
implementation)
10 to 20 secs to share, depending
on internet connection speed;
50% feel inclined to use it; 80%
of those should find the feature
easy to use;
ask users to try to share
something and measure how long
it takes; ask for feedback on
quality
Analyse usage statistics with
live data
41
Automatic ranking of users all forum users
get feedback on their
contributions; understand other
people's level of contribution
assert own identity; build reputation
/ inspire trust; promote empathy
through identification of common
rankings
fun and engaging rank label; automatic attribution
according to usage statistics; criteria are manually
set by community managers
Tech (design and
implementation)
At least 50% of users should be
happy with the ranking received;
at least 10% should come back
because of the new ranking
received
Give users a new rank thanks to
some action perform during
another test (surprise them); how
did they feel? Would they feel
encouraged to repeat the
behaviour?
If statistics show continued
higher engagement then our
test was good?
42
connect with Facebook all forumusers connect with existing account
facilitate registering and connecting
to the site
convenient, easy to use, standard
Tech (design and
implementation)
user should be able to register in
20 to 40 secs depending on
internet speed; at least 60% of
users should find the feature
useful; 80% of those should find
it easy to use
ask user to connect and check
how long it takes; ask for
feedback
Analyse usage statistics with
live data
43
Manualy create links
between contents
all forumusers
mark correlation between two
pieces of content
help others find additional
information; useful to rank search
results
simple, easy to use, attractive
Tech (design and
implementation)
at least 30% of users should find
this feature useful; 70% of those
should find it easy to use
ask user to look at UI - does he go
for that feature: if not, ask them
to use and provide feedback
Analyse usage statistics with
live data
44
IT Security news and events all website users stay up to date prevent or solve security issues
short, relevant news updates; engaging language;
trusted sources; images and graphs when
relevant; homepage element (both website home
and forumhome) that links to new page or full
news list
Tech (design); people
(content writers),
business (event
organisation)
at least 50% of users should find
the content useful to them; at
least 70% of users should find the
content is presented in a visually
pleasant way
ask users to analyse content and
UI; ask for feedback
Analyse usage statistics with
live data
M
u
s
t

h
a
v
e

-

f
u
t
u
r
e

v
e
r
s
i
o
n
s
N
i
c
e

t
o

H
a
v
e

-

t
h
i
s

v
e
r
s
i
o
n

o
r

n
e
x
t
?
45
Newsletter all forum users stay up to date
prevent or solve security issues; stay
connected and come back to the
forum
short, relevant news updates; engaging language;
trusted sources; images and graphs when relevant
Tech (design and
implementation); people
(content writers)
at least 30% of users should find
the newsletter useful; 70% of
those should find it engaging and
fun to read
ask users to read sample
newsletter and provide feedback:
would they want to receive it?
Was it clear?
Analyse usage statistics with
live data
46
"Subscribe to newsletter"
form
all website users subscribe to newsletter stay up to date
simple, strategically placed, enticing; preview of
latest titles
Tech (design and
implementation)
at least 30% of users should feel
inclined to subscribe; at least
80% of those should find it easy
to register
ask users to look at pages where
element is and say what attracts
themin that page; ask users to
register and provide feedback
Analyse usage statistics with
live data
47
new search engine all forumusers search forum find relevant content quicker
a novel search engine that takes into account
titles, text, tags but also linked content and usage
statistics (how many views), content quality
feedback (see item14) and date of publication -
in order to find and rank content
Tech (algorithmand
implementation)
At least 70% of the results shown
in the first page should be
relevant to the search query and
have good quality
Performtest searches on our
existing live data and compare
with previous search engine
results
Analyse usage statistics
N
e
v
e
r

H
a
v
e
48
Never allow users to direcly
promote / advertise their
products or services in
forum discussions
all forum users
not be bothered with
scaremongers; being sold
something they don't need
keep the community engaged and
keep information / solutions free of
charge
make this part of the rules / T & C's, monitor
behaviour
Business (strategy, legal
requirements)
at least 95% of all posts should
follow this rule and not require
moderation
ask users of they'd be happy to
follow the rule
Analyse usage statistics
M
u
s
t

h
a
v
e

-

f
u
t
u
r
e

v
e
r
s
i
o
n
s