SOFTWARE DESIGN: SESSION 5

FINDING THE POINTS OF LIGHT-

ACTIONABLE EXPERIMENTATION
Alex Cowan

2015 COWAN+
THE VALUE HYPOTHESIS
A certain PERSONA exists VALUE HYPOTHESIS
and they have a certain QUESTIONS
PROBLEMS(S) How much better than the best alternative
is your product?
How obvious is that to the customer?

where theyre currently using certain TOOLS
ALTERNATIVE(S) Testable Propositions
Structured Assumptions
and I have a VALUE PROPOSITION Structured Experiments (including MVP)
thats better enough than the alternatives
to cause the persona to act (purchase,
use, etc.).
2015 COWAN+
HYPOTHESIS-DRIVEN INNOVATION VIA LEAN STARTUP
Do I have real evidence from my buyer that 01 IDEA!
this is compelling?

What is our value hypothesis and what are 02 HYPOTHESIS

its key assumptions?
6.a YES
results
How do I definitely prove or disprove the disprove 03 EXPERIMENTAL DESIGN
assumptions with a minimum of time and hypothesis
effort?
Am I reacting or am I focused on 04 EXPERIMENTATION
validating my pivotal assumptions?
Pivot or persevere? 05 REVISE?

6.b NO
we appear to VALUE
have a valid HYPOTHESIS
hypothesis
source: adapted from The Lean Startup 2015 COWAN+
WHERE YOU SHOULD BE NOW

Personas, Problem Scenarios

including Think- Alternatives
See-Feel-Do Value Propositions
VALUE
PROPOSITIONS & Summary Value
ASSUMPTIONS Hypothesis
MVP Ideas
Assumptions for Value
Hypothesis

Do we understand
this person? What ?
AT
makes them tick? WH
PROBLEM Interview Guide
PERSONAS WHO? SCENARIOS &
ALTERNATIVES
2 (or more) Interviews

2015 COWAN+
REVIEW STUDENT WORK!
If we [do something] for [persona], they will [respond in a certain way]
MVP Archetype Example Assumptions (Enable Quiz)
Wizard of Oz If we get HR managers to a landing page with a demo, 10% will sign up for
our email product announcements.
If we create position-specific quizzes for HR managers, theyll use them
Concierge
~100% of the time and, after two positions, be willing to pay.
If we offer the service at [x] price with [y] supplemental assistance,
Sales
companies that hire a lot of engineers will pay [z].

2015 COWAN+
WHAT WELL COVER TODAY
CUSTOMER
DISCOVERY &
EXPERIMENTS

WH
AT
?IF
VALUE
PROPOSITIONS &
ASSUMPTIONS

2015 COWAN+
UNBUNDLING THE VALUE HYPOTHESIS INTO ASSUMPTIONS
If we [do something] for [persona], they will [respond in a certain way]
MVP Archetype Example Assumptions (Enable Quiz)
Wizard of Oz If we get HR managers to a landing page with a demo, 10% will sign up for
our email product announcements.
If we create position-specific quizzes for HR managers, theyll use them
Concierge
~100% of the time and, after two positions, be willing to pay.
If we offer the service at [x] price with [y] supplemental assistance,
Sales
companies that hire a lot of engineers will pay [z].

2015 COWAN+
REVIEW: AIDAOR

Attention
Interest
Desire
Action
Onboarding
Retention
2015 COWAN+
STORYBOARDING AIDAOR- UNITED CHILDRENS THEATER

I noticed Carlos doesn't Here's that email I'd love to see

do soccer. Maybe he'd from CynthiaOh, Carlos spend more
like to try theater, like this looks great, time with other
my Ricky. easy to try. And it's boys, good boys
affordable. that study and stay
out of trouble.
Oh yeah- can you
send me a link or
something?

ATTENTION INTEREST DESIRE

This looks like the Hi there, To

Action, Onboarding,
right program, I'm Retention Carlos. We're Sub

eligible for aid so so excited to

United Newsletter:
this is what I have you! Let's
Come to our openCOWAN+
2015
pay.Paid. See get started.
house to check out
you Tuesday! the year 2 program!
Bring friends!
 I noticed Carlos doesn't Here's that email I'd love to see
do soccer. Maybe he'd from CynthiaOh, Carlos spend more
like to try theater, like this looks great, time with other
my Ricky. easy to try. And it's boys, good boys
affordable. that study and stay
out of trouble.
Oh yeah- can you
STORYBOARDING AIDAOR- UNITED CHILDRENS THEATER
send me a link or
something?
Attention,ATTENTION
Interest, Desire INTEREST DESIRE

This looks like the Hi there, To

right program, I'm Carlos. We're Sub

eligible for aid so so excited to

United Newsletter:
this is what I have you! Let's
Come to our open
pay.Paid. See get started.
house to check out
you Tuesday! the year 2 program!
Bring friends!

ACTION ONBOARDING RETENTION

Create your own at www.StoryboardThat.com

2015 COWAN+
 I noticed Carlos doesn't Here's that email I'd love to see
Ido soccer.
noticed Maybe
Carlos he'd
doesn't from
Here'sCynthiaOh,
that email Carlos spend
I'd love more
to see
like to try theater,
do soccer. Maybe he'd like this looks great,
from CynthiaOh, time with other
Carlos spend more
like tomy
try Ricky.
theater, like easy
this to try. great,
looks And it's boys, goodother
time with boys
my Ricky. affordable.
easy to try. And it's that study and
boys, good boys stay
affordable. out
that of trouble.
study and stay
Oh yeah- can you out of trouble.
send
Oh me acan
yeah- linkyou
or
something?
send me a link or
something? INTEREST
ATTENTION DESIRE
ATTENTION INTEREST DESIRE

This looks like the Hi there, To

right looks
This program, I'm
like the Carlos. We're
Hi there, Sub
To
eligible for aid
right program, I'm so so excited
Carlos. to
We're Sub
United Newsletter:
this isfor
eligible what
aid Iso have you! Let's
so excited to Come to our open
United Newsletter:
pay.Paid.
this is whatSeeI get you!
have started.
Let's house
Come to to check out
our open
you Tuesday!
pay.Paid. See get started. the year 2 program!
house to check out
you Tuesday! theBring
year 2friends!
program!
Bring friends!

ACTION ONBOARDING RETENTION

ACTION ONBOARDING RETENTION
Create your own at www.StoryboardThat.com
Create your own at www.StoryboardThat.com
2015 COWAN+
YOUR VENTURE, THE EXPERIMENT
Component Notes

What assumption will this test? How?

What is/are the pivotal metric(s)?

What is the threshold for true (validated) vs.

false (invalidated)?

What will you do next if the result is true?

False?

How much time, money will it take to set up?

Roughly, what will it take for each individual
test? Subsequent tests?
Roughly, how long will it take for each test to
run and produce definitive, actionable results?

2015 COWAN+
EXAMPLE: ENABLE QUIZ CONCIERGE MVP
Component
What assumption will this test? How?
What is/are the pivotal metric(s)?
What is the threshold for true (validated) vs.
false (invalidated)?
What will you do next if the result is true?
False?
How much time, money will it take to set up?
Roughly, what will it take for each individual
test? Subsequent tests?
Roughly, how long will it take for each test to
run and produce definitive, actionable results?
Notes
If we create position-specific quizzes for HR managers, theyll use them ~100% of
the time and, after two positions, be willing to pay.
If the HR managers use the quiz, theyll send through <1/2 as many candidates.
If we offer the service at [x] price with [y] supplemental assistance, companies that
hire a lot of engineers will pay [z].
[quizzes administered] / [candidates interviewed]
comparison on candidates screened out (historical)
ability to sell
True: Build a simple1.0
1 & 2 True: Look at customer creation hyp., value hypothesis
False: Consider value hyp., problem hyp.
see bit.ly/exp-test
see bit.ly/exp-test
see bit.ly/exp-test
2015 COWAN+
EXAMPLE: ENABLE QUIZ ADWORDS MVP
Component Notes

What assumption will this test? How? If Enable Quiz offers its quizzing solution in the area of [x], customers will subscribe.

What is/are the pivotal metric(s)?

absolute click-through-rate; wed like to see >=2%
What is the threshold for true (validated) vs. relative to CTR (on this basis well select our top 10 topics)
false (invalidated)?
True (along with previous): Build a simple1.0 with selected topics.
What will you do next if the result is true?
False (all CTRs <2%): Revise customer creation plan & test an alternate topic
False?
assessment strategy
How much time, money will it take to set up? see bit.ly/exp-test
Roughly, what will it take for each individual
see bit.ly/exp-test
test? Subsequent tests?
Roughly, how long will it take for each test to
see bit.ly/exp-test
run and produce definitive, actionable results?

2015 COWAN+

EXAMPLE: ENABLE QUIZ CONCIERGE MVP
Component
What assumption will this test? How?
What is/are the pivotal metric(s)?
What is the threshold for true (validated) vs.
false (invalidated)?
What will you do next if the result is true?
False?
How much time, money will it take to set up?
Roughly, what will it take for each individual
test? Subsequent tests?
Roughly, how long will it take for each test to
run and produce definitive, actionable results?
Draft a set of experiments to verify
your top assumption
Notes
If we create position-specific quizzes for HR managers, theyll use them ~100% of
the time and, after two positions, be willing to pay.
If the HR managers use the quiz, theyll send through <1/2 as many candidates.
If we offer the service at [x] price with [y] supplemental assistance, companies that
hire a lot of engineers will pay [z].
[quizzes administered] / [candidates interviewed]
comparison on candidates screened out (historical)
ability to sell
True: Build a simple1.0
1 & 2 True: Look at customer creation hyp., value hypothesis
False: Consider value hyp., problem hyp.
see bit.ly/exp-test
see bit.ly/exp-test
see bit.ly/exp-test
2015 COWAN+
ASSIGNMENTS
1. Do/Individual: Finish Design (an) Experiment(s) for Your
Assumption(s) (15-30minutes)

v5 2015 COWAN+