The Statistical Imagination

Chapter 9.
Hypothesis Testing I: The Six
Steps of Statistical Inference
2008 McGraw-Hill Higher Educa
tion

A Hypothesis
A hypothesis is a prediction about the
relationship between two variables
that asserts that differences among
the measurements of an independent
variable will correspond to differences
among the measurements of a
dependent variable
2008 McGraw-Hill

Using a Hypothesis to
Test a Theory
The hypothesis is stated before we
gather data
The theoretical purpose of a hypothesis
test is to corroborate theory by testing
preconceived ideas against facts
Theory motivates or pushes us to expect
certain empirical outcomes
2008 McGraw-Hill

Statistical Inference
Statistical inference is drawing conclusions
about a population on the basis of sample
statistics
The logic of hypothesis testing involves
deciding whether to accept or reject a
statement on the basis of observations of data
Accounting for sampling error with sampling
distributions is key to the process
2008 McGraw-Hill

The Test Effect of the

Hypothesis Test
The difference between what is observed
in a sample and what is hypothesized is
called the test effect
We ask: What is the probability that the test
effect is simply the result of sampling
error?
The sampling distribution provides a
measuring stick to answer the question
2008 McGraw-Hill

The Statistical Purpose

of a Hypothesis Test
The statistical purpose of a
hypothesis test is to determine
whether statistical test effects
computed from a sample indicate
(1) real effects in the population
or (2) sampling error
2008 McGraw-Hill

Making Empirical Predictions

To prove a hypothesis we must predict two
things:
1. A mathematical prediction of a
parameter outcome
2. A sampling distribution,
a prediction of all possible sampling
outcomes factoring in sampling error
With these predictions, we determine the
probability that our single sample outcome
differs significantly from the predicted
outcome (i.e., if the effect is real)
2008 McGraw-Hill

Basic Logical Procedure

of a Hypothesis Testing
a. A question is raised
b. Predictions are made on the basis of
probability theory
c. An event is observed and its effects are
measured
d. The probability of the test effect occurring
is computed
e. A conclusion is drawn
2008 McGraw-Hill

Two Sets of Tasks When

Testing a Hypothesis
1. Test preparation deciding
what test to use, and organizing
the data
2. Test the hypothesis following the
six steps of statistical inference
2008 McGraw-Hill

Test Preparation
State the research question: A goal that
can be stated in terms of a hypothesis
Draw a conceptual diagram depicting
givens (population under study, sample
size, variables and their levels of
measurement, provided and calculated
parameters and statistics)
Select the statistical test
2008 McGraw-Hill

Step 1 of The Six Steps

of Statistical Inference
Step 1: State the null hypothesis (H0).
State the alternative hypothesis (HA)
and stipulate the direction of the test

2008 McGraw-Hill

The Null Hypothesis

The null hypothesis, H0, is a hypothesis
stated in such a way that we will know
what statistical outcomes will occur in
repeated random sampling if this
hypothesis is true
It is a statistical hypothesis: it directs
us to the sampling distribution, which
provides sampling predictions
2008 McGraw-Hill

What does null mean?

Null means none
H0 predicts sampling outcomes
assuming no effect or no difference
Often we can nullify (negate the
wording) of the research question to
determine the H0

2008 McGraw-Hill

The Alternative
Hypothesis (HA)
HA is the statement we accept if H0 is
rejected
HA is often a direct statement of the
research question

2008 McGraw-Hill

The Direction of a
Hypothesis Test
Test direction refers to whether we
are able to predict the direction our
observed sample statistic will fall
Direction must be specified before we
observe data

2008 McGraw-Hill

The Direction of a
Hypothesis Test (cont.)
Three possible directional
statements:
1. Nondirectional (two-tailed test)
2. Positive direction (one-tailed test)
3. Negative direction (one-tailed test)
2008 McGraw-Hill

The Direction of a
Hypothesis Test (cont.)
In Step 1, the HA, we specify whether we
expect the outcome in our observed
sample to fall above (positive, one-tailed)
or below (negative, one-tailed) the
hypothesized parameter of the H0
For a nondirectional, two-tailed test, we do
not predict a direction and simply assert
that the outcome is expected to differ from
the hypothesized parameter
2008 McGraw-Hill

When to State a Positive,

One-Tailed Test
When the content of research
question includes terms such as
greater than, more, increase, faster,
heavier, and gain

2008 McGraw-Hill

When to State a Negative,

One-Tailed Test
When the content of research
question includes terms such as less
than, fewer, decrease, slower, lighter,
and loss

2008 McGraw-Hill

When to State a Nondirectional,

Two-Tailed Test
When the content of the research
question includes no statements
about direction, or simply asserts
inequality

2008 McGraw-Hill

Step 2 of The Six Steps of

Statistical Inference
Step 2: Describe the sampling
distribution and draw its curve
The sampling distribution is a
description of all possible sampling
outcomes and a stipulation of the
probability of each outcome assuming
that the H0 is true
It is built around the H0
2008 McGraw-Hill

Step 3 of The Six Steps of

Statistical Inference
Step 3: State the chosen level of
significance, alpha (), and indicate again
whether the test is one-tailed or two-tailed.
Specify the critical test value
The level of significance, alpha, is the
amount of sampling error we are willing to
tolerate in coming to a conclusion
Critical test values are obtained from the
statistical tables in Appendix B
2008 McGraw-Hill

Step 4 of The Six Steps

of Statistical Inference
Step 4: Observe the actual
sample; compute the test
effects, the test statistic, and
the p-value

2008 McGraw-Hill

Step 4 (cont.): The

Test Effect
The test effect is the difference
between the value of the sample
statistic and the parameter value
predicted by the null hypothesis (H0 in
Step 1)
It is a deviation score on the sampling
distribution curve
2008 McGraw-Hill

Step 4 (cont.):
The Test Statistic
The test statistic is a formula for
measuring the likelihood of the
observed effect
It transforms the effect into standard
error units so that the result may be
compared to critical scores of the
statistical tables in Appendix B
2008 McGraw-Hill

Step 4 (cont.)
The p-Value
The p-value is a measure of the
unusualness of a sample outcome when the
H0 is true. E.g., Is it unusual to roll four 7s in
a row with honest dice?
Calculation: p-value = probability (p) of
sampling outcomes as unusual as or more
unusual than the outcome observed under
the assumption that the H0 is true
An area in the tail(s) of the curve in Step 2
2008 McGraw-Hill

Step 5 of The Six Steps of

Statistical Inference
Step 5: Make the rejection decision
by comparing the p-value to
If p < , reject the H0 and accept the
HA at the 1- level of confidence
If p > , fail to reject the H0

2008 McGraw-Hill

Step 6 of The Six

Steps: Interpretation
Step 6: Interpret and apply the results, and
provide best estimates in everyday terms
Fit the interpretation to either a professional
or public audience: use as little statistical
jargon as possible
Frame the interpretation around the H0 or
the HA, whichever survived the hypothesis
test
2008 McGraw-Hill

Probability Theory in
Hypothesis Testing

Computing probabilities is the essential

mathematical operation in hypothesis
testing
Hypothesis testing is based on comparing
two probabilities:
1. What actually occurs in our single observed
sample
2. What we expect to occur in repeated
sampling
2008 McGraw-Hill

A Focus on p-Values:
When the p-Value is Large
When p > , we fail to reject the H0
A large p-value tells us that our observed
sample outcome is not much different or far
off from the outcome predicted by the H0
A large p-value occurs when the test effect is
small, and this suggests that the effect could
easily be the result of expected sampling
error
2008 McGraw-Hill

A Focus on p-Values:
When the p-Value is Small
When p < , we reject the H0
A small p-value tells us that assuming the
H0 is true, our sample outcome is unusual
or far off from the outcome predicted by
the H0
A small p-value occurs when the test effect
is large leading us to conclude that the test
effect did not result from sampling error
2008 McGraw-Hill

Inverse Relationship Between

Effect Size and p-Value
A small test effect = a large
p-value = fail to reject the H0
A large test effect = a small
p-value = reject the H0 and
accept the HA
2008 McGraw-Hill

The Level of Significance ()

in Hypothesis Testing
The level of significance () is the
critical probability point at which we are
no longer willing to say that our
sampling outcome resulted from
random sampling error
is stated in Step 3 and compared in
Step 5 to the p-value. This comparison
is called the rejection decision
2008 McGraw-Hill

Critical Test Scores

The critical test score (Z) is the statistical
test score that is large enough to indicate a
significant difference between the
observed sample statistic and the
hypothesized parameter
The critical region is the area in the tail(s)
of the probability curve that is beyond the
critical test score of the stated level of
significance
2008 McGraw-Hill

Critical Test Scores (cont.)

Zobserved is a test statistic
Z is a critical score
If Zobserved > Z , then p < ; reject
H0
If Zobserved < Z , then p > ; fail to
reject H0
2008 McGraw-Hill

Critical Z-scores
on the Normal Curve
Critical Z-scores are ones of great
importance in statistical procedures
and are used very frequently
Some widely used critical Z-scores
are 1.64, 1.96, 2.33, 2.58, 3.08, and
3.30
See if you can match these scores to the
level of significance and direction of a
hypothesis test
2008 McGraw-Hill

Choosing the Level

of Significance
Setting the level of significance ()
allows us to control the chances of
making a wrong decision or error
Short of double-checking against
data for the entire population, we will
never know for sure whether we
made the correct rejection decision or
made an error
2008 McGraw-Hill

Possible Results of a
Rejection Decision
Correct decision: Fail to reject a true
H0
Type I error: Rejecting a true H0
Correct decision: reject a false H0
Type II error: Failing to reject a false H0
2008 McGraw-Hill

Managing and Controlling

Rejection Decision Errors
When we reject H0, we either made a
correct decision or made a Type I error; we
could not have made a Type II error
When we fail to reject H0, we made either
a correct decision or a Type II error; we
could not have made a Type I error

2008 McGraw-Hill

Controlling Type I
and Type II Errors
Type I error is easily controlled by setting the
level of significance (), because it turns out
that = p [of making a Type I error]
= p [of making a Type II error]; controlling
beta () is difficult
is indirectly controlled when we set
because the two are inversely related; is
also minimized by using a large sample size
2008 McGraw-Hill

Four Conventional
Levels of Alpha ( )
=.10: High likelihood of rejecting the H0. Used in
exploratory research, where little is known about a
topic
=.05: Moderate likelihood of rejecting the H0.
Used in survey research
=.01 and =.001: Low likelihood of rejecting the
H0. Used in biological, laboratory, and medical
research, especially when a Type I error is lifethreatening

2008 McGraw-Hill

The Level of Confidence

(LOC) for a Hypothesis Test
The LOC is the confidence we have that
we did not make a Type I error
LOC = 1 - level of significance = 1 -
E.g., the .05 level of significance
corresponds to a 95% LOC
The only time we have 100% confidence in
a conclusion is when every subject in a
population is observed
2008 McGraw-Hill

Selecting Which
Statistical Test to Use
Ask: How many variables are we observing
for this test?
What are the levels of measurement of the
variables?
Are we dealing with one representative
sample from a single population or more?
What is the sample size?
Are there peculiar circumstances to
consider?
2008 McGraw-Hill

When to Use a Large

Single-Sample Means Test
1. One variable
2. Interval/ratio level of measurement
3. One representative sample from one
population
4. n > 121 cases
Sampling distribution will be the normal
curve (See Chapter 7)
2008 McGraw-Hill