Introduction to Hypothesis Testing

1 Introduction
The purpose of hypothesis testing is to determine whether there is enough statistical evidence in favor of a certain belief about a parameter. Examples
Is there statistical evidence in a random sample of potential customers, that support the hypothesis that more than 10% of the potential customers will purchase a new products? Is a new drug effective in curing a certain disease? A sample of patients is randomly selected. Half of them are given the drug while the other half are given a placebo. The improvement in the patients conditions is then measured and compared.
2

What is a hypothesis?
A tentative statement about a population parameter that might be true or wrong

Types of hypotheses
Null hypothesis Research/alternative hypothesis

2 Concepts of Hypothesis Testing

The critical concepts of hypothesis testing.
Example:
An operation manager wants to determine if the mean demand during lead time is greater than 350. If so, the ordering policy should be changed.

The two hypotheses about a population mean:

H0: The null hypothesis m = 350 H1: The alternative hypothesis m > 350
5

2 Concepts of Hypothesis Testing

Assume the null hypothesis is true (m= 350).

m = 350

Sample from the demand population, and build a test statistic related to the parameter hypothesized (the sample mean). Decide, is the value obtained big enough to reject the null hypothesis?

2 Concepts of Hypothesis Testing

Assume the null hypothesis is true (m= 350).

x 355

m = 350

x 450

Since the x is much larger than 350, the mean m is likely to be greater than 350. Reject the null hypothesis. In this case the mean m is not likely to be greater than 350. Do not reject the null hypothesis.
7

Types of Errors
Two types of errors may occur when deciding whether to reject H0 based on the statistic value. Type I error: Reject H0 while the truth is, it is true. Type II error: Do not reject H0 while the truth is, it is false. Example continued Type I error: Reject H0 (m = 350) in favor of H1 (m > 350) while the truth is, the real value of m is 350. Type II error: Do not reject H0 (m = 350) while the truth 8 is, the real value of m is greater than 350.

Controlling the probability of conducting a type I error

Recall:
H0: m = 350 and H1: m > 350. H0 is rejected if x is sufficiently large

Thus, a type I error is made if x critical value when m = 350. By properly selecting the critical value we can limit the probability of conducting a type I error to an acceptable level. Critical value
m = 350

What is a critical value?

A value needed to determine weather to reject or not to reject the null hypothesis.

10

3 Testing the Population Mean When the Population Standard Deviation is Known
The manager of a department store is thinking about establishing a new billing system for the stores credit customer. After a through financial analysis, she determines that the new system will be cost effective only if the mean monthly account is more than $170. A random sample of 400 monthly account is drawn, for which the sample mean is $178. The manager knows that the accounts are approximately normally distributed with standard deviation $65. Can the manager conclude from this available information that the new system will be cost-effective?
11

3 Testing the Population Mean When the Population Standard Deviation is Known
Example 1
A new billing system for a department store will be costeffective only if the mean monthly account is more than $170. A sample of 400 accounts has a mean of $178. If accounts are approximately normally distributed with s = $65, can we conclude that the new system will be cost effective?
12

Testing the Population Mean (s is Known)

Example 1 Solution
The population of interest is the credit accounts at the store. We want to know whether the mean account for all customers is greater than $170. H1 : m > 170 The null hypothesis must specify a single value of the parameter m, H0 : m = 170
13

Approaches to Testing
There are two approaches to test whether the sample mean supports the alternative hypothesis (H1)
The rejection region method is mandatory for manual testing (but can be used when testing is supported by a statistical software) The p-value method which is mostly used when a statistical software is available.
14

The Rejection Region Method

The rejection region is a range of values such that if the test statistic falls within that range, the null hypothesis is rejected in favour of the alternative hypothesis.

15

Steps in rejection region method

Construct appropriate hypotheses Determine a test statistics to be used Determine the critical value Compare the test statistic with the critical value. Reject the null hypothesis if the former is greater than the latter. Make an appropriate conclusion.
16

The Rejection Region Method for a Right - Tail Test

Example 1 solution continued

Recall:
therefore,

H0: m = 170 H1: m > 170

It seems reasonable to reject the null hypothesis and believe that m > 170 if the sample mean is sufficiently large.
Reject H0 here Critical value of the sample mean
17

The Rejection Region Method for a Right - Tail Test

Example 1 solution continued
Define a critical value x L for to reject the null hypothesis. Reject the null hypothesis if

x that is just large enough

x xL
18

Determining the Critical Value for the Rejection Region

Allow the probability of committing a Type I error be a (also called the significance level). Find the value of the sample mean that is just large enough so that the actual probability of committing a Type I error does not exceed a. Watch

19

Determining the Critical Value for a Right Tail Test

Example 1 solution continued

za

x L 170 65 400

a
m x 170

xL

P(commit a Type I error) = P(reject H0 given that H0 is true) = P( x xL given that H0 is true) is allowed to be a.

Since P( Z Z a ) a we have:

20

Determining the Critical Value for a Right Tail Test

Example 1 solution continued

a = 0.05
m x 170

xL

za

x L 170 65 400

65 x L 170 z a . 400 If we select a 0.05, z .05 1.645. 65 x L 170 1.645 175.34. 400
21

Determining the Critical value for a Right - Tail Test

Re ject the null hypothesis if x 175.34
Conclusion
Since the sample mean (178) is greater than the critical value of 175.34, there is sufficient evidence to infer that the mean monthly balance is greater than $170 at the 5% significance level.
22

The standardized test statistic

Instead of using the statistic x, we can use the standardized value z.

x m s n
One tail test

Then, the rejection region becomes

z za

23

The standardized test statistic

Example 1 - continued
We redo this example using the standardized test statistic. Recall: H0: m = 170 H1: m > 170 Test statistic: x m 178 170 z 2.46 s n 65 400 Rejection region: z > z.05 1.645.
24

The standardized test statistic

Example 1 - continued
Re ject the null hypothesis if Z 1.645

Conclusion
Since Z = 2.46 > 1.645, reject the null hypothesis in favor of the alternative hypothesis.
25

P-value Method
The p-value provides information about the amount of statistical evidence that supports the alternative hypothesis.

The p-value of a test is the probability of observing a test statistic at least as extreme as (greater or equal) the one computed, given the null hypothesis is true. Let us demonstrate the concept on Example 11.1

26

P-value Method
The probability of observing a test statistic at least as extreme as 178, given that m = 170 is

P( x 178 when m 170) 178 170 P( z ) 65 400 P( z 2.4615) .0069

m x 170

x 178

The p-value

27

Interpreting the p-value

Because the probability that the sample mean will assume a value of more than 178 when m = 170 is so small (.0069), there are reasons to believe that m > 170.
Note how the event x 178 is rare under H0 when m x 170, but... it becomes more probable under H1, when m x 170

H0 : m x 170 H1 : m x 170

x 178

28

Interpreting the p-value

We can conclude that the smaller the p-value the more statistical evidence exists to support the alternative hypothesis.

H0 : m x 170 H1 : m x 170

x 178

29

Interpreting the p-value

Describing the p-value
If the p-value is less than 1%, there is overwhelming evidence that supports the alternative hypothesis. If the p-value is between 1% and 5%, there is a strong evidence that supports the alternative hypothesis. If the p-value is between 5% and 10% there is a weak evidence that supports the alternative hypothesis. If the p-value exceeds 10%, there is no evidence that supports the alternative hypothesis.
30

The p-value and the Rejection Region Methods

The p-value can be used when making decisions based on rejection region methods as follows:
Define the hypotheses to test, and the required significance level a. Perform the sampling procedure, calculate the test statistic and the p-value associated with it. Compare the p-value to a. Reject the null hypothesis only if p-value <a; otherwise, do not reject the null hypothesis.

a = 0.05

The p-value
31

m x 170
x L 175.34

x 178

Conclusions of a Test of Hypothesis

If we reject the null hypothesis, we conclude that there is enough evidence to infer that the alternative hypothesis is true. If we do not reject the null hypothesis, we conclude that there is not enough statistical evidence to infer The alternative hypothesis that the alternative hypothesis is true. is the more important
one. It represents what we are investigating.
32

A Left - Tail Test

The SSA Envelop Example.
The chief financial officer in FedEx believes that including a stamped self-addressed (SSA) envelop in the monthly invoice sent to customers will decrease the amount of time it take for customers to pay their monthly bills. Currently, customers return their payments in 24 days on the average, with a standard deviation of 6 days.
33

A Left - Tail Test

The SSA envelop example continued
It was calculated that an improvement of two days on the average will cover the costs of the envelops (checks can be deposited earlier). A random sample of 220 customers was selected and SSA envelops were included with their invoice packs. The times customers payments were received were recorded (SSA.xls) Can the CFO conclude that the plan will be profitable at 10% significance level?
34

A Left - Tail Test

The SSA envelop example Solution
The parameter tested is the population mean payment period (m). The hypotheses are: H0: m = 22 H1: m < 22 (The CFO wants to know whether the plan will be profitable)

35

A Left - Tail Test

The SSA envelop example Solution continued
The rejection region: It makes sense to believe that m < 22 if the sample mean is sufficiently smaller than 22. Reject the null hypothesis if

x xS
36

Left-tail test

A Left -Tail Test

The SSA envelop example Solution continued
The standardized one tail left hand test is:
z xm 21.63 22 6 220 .91

Define the rejection region

z za z.10 1.28

Since -.91 > 1.28 do not reject the null hypothesis. The p value = P(Z<-.91) = .1814 Since .1814 > .10, do not reject the null hypothesis

37

A Two - Tail Test

Example 2
AT&T has been challenged by competitors who argued that their rates resulted in lower bills. A statistics practitioner determines that the mean and standard deviation of monthly long-distance bills for all AT&T residential customers are $17.09 and $3.87 respectively.

38

A Two - Tail Test

Example 2 - continued
A random sample of 100 customers is selected and customers bills recalculated using a leading competitors rates (see Xm11-02). Assuming the standard deviation is the same (3.87), can we infer that there is a difference between AT&Ts bills and the competitors bills (on the average)?
39

A Two - Tail Test

Solution
Is the mean different from 17.09?
H0: m = 17.09
H1 : m 17.09

Define the rejection region

z za / 2 or z za / 2

40

A Two Tail Test

Solution - continued
a/2 0.025
a/2 0.025 17.09

x
We want this erroneous rejection of H0 to be a rare event, say 5% chance.
41

If H0 is true (m =17.09), x can still fall far above or far below 17.09, in which case we erroneously reject H0 in favor of H1

(m 17.09)

A Two Tail Test

Solution - continued
z xm

a/2 0.025 17.55

17.55 17.09 3.87 100

1.19

17.09

a/2 0.025

From the sample we have:

x 17.55

a/2 0.025

a/2 0.025
0 za/2 = 1.96
42

-za/2 = -1.96

Rejection region

Two-tail test

A Two Tail Test

There is insufficient evidence to infer that there is a difference between the bills of AT&T and the competitor.
Also, by the p value approach: The p-value = P(Z< -1.19)+P(Z >1.19) = 2(.1173) = .2346 > .05 a/2 0.025
-1.19 0 1.19

a/2 0.025

xm

17.55 17.09 3.87 100

1.19

-za/2 = -1.96

za/2 = 1.96
43