Topic Notes

QM161

Topic 2 - Probability

Learning Outcomes

In this topic, you will learn:

basic probability concepts; and

conditional probability.

QM161 - Business Statistics 1

Topic Notes

Topic 2 Probability

2.1

Introduction

In this topic we will be covering most of the material in LSS Chapter 4, Sections 4.1 and
4.2. I will not necessarily deal with the subject in the same order, or in the same way as
LSS, but I will give appropriate references to LSS as we proceed.
The key concept is that probability is about measuring uncertainty. Even though
probability is at the very heart of statistics, you do not have to be good at probability in
order to be good at statistics. For future work it is only necessary to gain a few basic
notions about probability. Some students find probability difficult and are tempted to
give up, assuming the unit can only get more difficult. Don't! You will probably find
subsequent topics much easier, and will come to realise that your limited grasp of
probability will be quite adequate.
I believe that almost everyone has a basic intuitive notion of probability, but almost
nobody can obtain correct answers to anything except the simplest of problems. Let me
illustrate with two examples.

Example 1
Three marbles are in a hat. One is red, one white and one black. A marble is chosen at
random. What is the probability that it is red? I would be very surprised if anyone got an
answer different from 1/3. If that is your answer you are right, and you have some notion
of probability.

Example 2
Three cards are in a hat. One card is red on both sides, one is white on both sides and
the third card is red on one side and white on the other. A card is chosen at random and
placed on the table. The side facing up is red. Obviously, the white/white card was not
chosen! If the card is turned over, what is the probability that the other side is also red? If
your answer is 1/2 you are wrong. If your answer is not 1/2 you may be right. Stay tuned
for more on this problem!
If you got 1/2, you (along with almost everyone else) have illustrated the need for a
theory of probability to enable you to obtain correct answers.

QM161 - Business Statistics 1

Topic Notes

In the following sections we will be developing a few simple rules to help you measure
uncertainty correctly.

2.2

Some Basic Definitions

2.2.1 Random Experiment

A random experiment is any mechanism that produces an outcome that cannot be
predicted with certainty in advance. I will usually just refer to a random experiment as an
experiment. Most people are happy with the idea of a scientific/medical experiment. Here
are a few other examples of experiments:

Tossing a coin

Drawing a card from a deck

Throwing a die

A horse race

Asking a randomly selected person their income or political preference

2.2.2 Sample Space

Sample space is the collection (or set) of all possible outcomes of an experiment.
Example
If we toss two dice, what are the possible outcomes?
When two dice are thrown, the sample space consists of 36 outcomes (e.g. die 1 = 5, die
2 = 3). The sample space for this experiment can be tabulated as below.
(1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6)
(2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6)
(3, 1) (3, 2) (3, 3) (3, 4) (3, 5) (3, 6)
(4, 1) (4, 2) (4, 3) (4, 4) (4, 5) (4, 6)
(5, 1) (5, 2) (5, 3) (5, 4) (5, 5) (5, 6)
(6, 1) (6, 2) (6, 3) (6, 4) (6, 5) (6, 6)
Figure 1

QM161 - Business Statistics 1

Topic Notes

You should be aware of the fact that the sample space of most experiments is too
complicated to be able to be represented in this way. However, this is a useful example
for illustrating basic concepts.

2.2.3 Elementary (or Simple) Event

This is an event that contains only one outcome. For example, a double six is an
elementary event, whereas a double is not. It contains six outcomes [(1,1), (2,2), (3,3),
(4,4), (5,5) (6,6)]. It is important to note that elementary events are always mutually
exclusive. That is, a single experiment cannot give rise to two elementary event
outcomes. For example, let A = event (3, 4) and B = event (2, 6) when two dice are
thrown. If the event A occurred then B could not have occurred.

2.2.4 Event
An event is a subset of the sample space. For example, throwing a double is an event. It
is made up of six simple events as illustrated in the table below.

(1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6)

(2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6)
(3, 1) (3, 2) (3, 3) (3, 4) (3, 5) (3, 6)
(4, 1) (4, 2) (4, 3) (4, 4) (4, 5) (4, 6)
(5, 1) (5, 2) (5, 3) (5, 4) (5, 5) (5, 6)
(6, 1) (6, 2) (6, 3) (6, 4) (6, 5) (6, 6)
Figure 2

If we define A as 'Throwing a Double', then we can see the probability of A denoted by

P(A) is equal to 6/36. We write this in the following way:
P(A) =

6
1
=
36 6

This should enable you to make the connection between an event (which we normally
take to be a 'happening') and a subset in a certain sample space.
A tree diagram is a device, which is often useful for depicting all the simple events in the
sample space.

QM161 - Business Statistics 1

Topic Notes

Example
A coin is tossed three times. The sample space may be depicted as follows.

Outcome
H

HHH

HHT

HTH

HTT

THH

THT

TTH

H
H
T

T
T
Throw 1

Throw 2

TTT

Throw 3

Figure 3

This experiment gives rise to 8 simple outcomes, namely, HHH, HHT,..., TTT as shown in
the end branches of the tree diagram above.
Assuming that the coin is 'fair', that is the probability of a head = probability of a tail =
0.5, the probability of each simple event is 1/8 (i.e. 0.5 0.5 0.5).
The probability of the event '2 tails' would be
P(2 tails) = P(HTT) + P(THT) + P(TTH)
= 1/8 +1/8 + 1/8
= 3/8

2.2.5 Complementary Event

The complementary event to event A includes all the events not in A. We use A to
indicate the complementary event.
If A = event A does not occur, then:
P( A ) = 1 P(A).

QM161 - Business Statistics 1

Topic Notes

This relationship is often useful when P(A) is difficult to find directly, whereas P( A ) is
relatively easy.
A useful visual tool sometimes used to illustrate probability concepts is the Venn
diagram. The rectangle represents the sample space and the oval labelled A indicates the
simple events in the sample space that are included in event A. The area outside the oval,
but within the rectangle, contains the simple events that belong to the complementary
event A .

2.2.6 Joint Event

A joint event is an event that has 2 or more characteristics. If we define A as the first
characteristic and B as the second characteristic, we can write the probability of the joint
event as P(A and B). This can be depicted in a Venn diagram as the intersection of two
overlapping ovals, as seen below.

P(A and B)
A

2.2.7 Union Event / Compound Event

A union or compound event is the probability of observing event A or event B. We use the
notation P(A or B) for this probability.
The union event could look like this if A and B are mutually exclusive.

P(A or B)
A

QM161 - Business Statistics 1

Topic Notes

Or, if A and B are not mutually exclusive, the union event would look like this.

P(A or B)
B

Note the overlap, which is the probability of the joint event A and B.

2.3

Definition of Probability and Some Rules

Definition of Probability
To every event A associated with an experiment, we attach a number P(A), known as the
'probability of A'. This number is a measure of how 'likely' it is that A will occur.

Some Rules of Probability

1. Probabilities lie between 0 and 1, that is, 0 P(A) 1.
A probability near 1, say 0.9, indicates that A is 'highly likely' whereas a low
number, e.g. 0.1, indicates that A is not likely to occur. If P(A) = 1, A is certain to
occur, and if
P(A) = 0, A cannot occur.
2. The sum of the probability of all simple events in the sample space is one.
3. Complementary Events A and A have probabilities that add to one.
P(A) + P( A ) = 1 or

P( A ) = 1 P(A)

4. Addition Rule
If A and B are mutually exclusive events (that is, A and B cannot both occur), then
P(A or B) = P(A) + P(B).

Example
Suppose a family has three children. Let A = event of exactly one girl and B = event
of exactly two girls. Then, because the events are mutually exclusive,
P(one or two girls) = P(A or B)
= P(A) + P(B).

QM161 - Business Statistics 1

Topic Notes

But, if A and B are not mutually exclusive events, the Venn diagram has
overlapping ovals.

P(A and B)
A

If, we add P(A) and P(B), we will double count the intersection or overlap of the two
ovals. So, we need to subtract this area that has been counted twice (the hashed
area). This area is P(A and B), the probability of the joint event.
The General Addition Rule is
P(A or B) = P(A) + P(B) P(A and B).

5. Probability Values for Equally Likely Outcomes

The above rules have given us some relationships among probabilities, but in no
case are we able to actually determine the value of P(A). We will now provide a
method, which in some circumstances enables us to put a value on P(A).
It sometimes happens that an experiment is such that the simple event outcomes
are all equally likely. Suppose A = A1 or A2 or ... or Am, where each of the Ai is an
elementary event. Then, by Rule 3
P(A) = P(A1) + P(A2) + ... + P(Am).
If there are n equally likely events A1, A2, . . ., An which constitute all the sample
space, then
P(A1) + P(A2) + ... + P(An) = 1 (by Rule (2))

P(A1) =P(A2) = ... = P(An) =

1
n

1 1
1 m
+ + ... + =
n n
n n
number of ways A can occur
In words, P(A) =
total number of outcomes

P( A) =

LSS uses the term 'a priori classical probability' for this approach, see p. 186.

Example 1
8

QM161 - Business Statistics 1

Topic Notes

A fair coin is tossed. What is the probability of a Head? We all know the answer is 1/2.
This result comes from the following reasoning:
There are two simple outcomes H, T that are equally likely because the coin is fair. The
outcome we are interested in occurs in only one way.

m = 1 and n = 2

P(H) = 1/2.

Notice that if the coin is not fair, the outcomes are not equally likely, and P(H) will not
equal 1/2.

Example 2
Two fair dice are tossed. What is the probability of a double?
There are 36 equally likely outcomes. A double can occur in 6 ways

P(double) = 6/36 = 1/6.

Notes
In using the 'equally likely outcomes' approach, there are some common pitfalls:
(i)

the sample space may be divided up into events that are not simple events; and

(ii)

these events may not be equally likely.

Example 3
Let us revisit the case of the three cards that are red-red, red-white and white-white. You
will remember that I asked you what was the probability of the second side being red,
given the first side was red. I assume most (or all) of you said 1/2. We can now formalise
your reasoning. You would have conceptualised the sample space as
R

QM161 - Business Statistics 1

Topic Notes

As the white-white card clearly could not have been chosen, I have struck it out. From the
above diagram, there appear to be 2 outcomes where the card is turned over, both
equally likely (n = 2). The event we are interested in (namely red when turned over) can
occur in only one way (m = 1)

P(red) = 1/2.

Why is this answer incorrect?

Suppose now that the red-red card has 1 put on one side, and 2 on the other side, and
likewise for the white-white card. The sample space is now seen to be
R1

R2

W1

W2

R2

R1

W2

W1

Once more I have struck out those outcomes that gave 'white side up'.
It is now apparent that there are three outcomes when the card is turned over, and two
ways red can occur. In this case the outcomes are equally likely

P(red) = 2/3.

This is the correct answer.

To use the rule

P( A) =

number of ways A can occur

number of outcomes

we must be able to count!

In the examples used so far counting has been trivial. However, even some very simple
examples can give rise to severe counting problems. Consider this example.

10

QM161 - Business Statistics 1

Topic Notes

Example 4
4 cards are drawn from a standard 52-card pack. What is the probability that exactly three
are spades?
Here

number of ways of getting exactly three spades in four draws

11,154

number of ways four cards can be drawn

270,725

P(exactly three spades) =

11,154
= 0.041.
270,725

Not quite so easy!

If you are wondering about how m and n were found, I used a branch of mathematics
known as 'counting techniques'. This is not necessary for the unit, but if you are
interested, many mathematics books will cover this under the general heading of
'Combinations and Permutations'.

2.4

Relative Frequency Approach to Probability

(LSS uses the term 'empirical probability', see pp. 186-187)

The rule we have been using so far, namely

P( A) =

number of ways A can occur

number of outcomes

depends crucially on the simple events being equally likely. If this is not the case, the rule
is invalid. A very simple example illustrates this. If we have a fair coin P(H) = 1/2. If the
coin is not fair, what is P(H)? The answer is that we don't know, as the rule does not
apply, because the outcomes are not equally likely.
Does this mean that if a coin is not fair we cannot determine P(H)? Unfortunately, we
cannot find P(H), but we are able to obtain a very good estimate of P(H). The relative
frequency approach to probability enables us to estimate probabilities in situations in
which the 'equally likely outcomes' rule cannot be used.

11

QM161 - Business Statistics 1

Topic Notes

In the case of the unfair coin, a sensible approach to getting a handle on the probability
of a head occurring would be to throw the coin (say) 100 times. If 73 heads occurred, the
proportion (or relative frequency) of heads would be 73/100 = 0.73.
This proportion is an estimate of the probability. If the coin were thrown 1000 times, the
relative frequency of heads would be a better estimate of P(H). In general, if an
experiment is conducted n times, and a particular event A occurs m times, then an
approximation to P(A) is m/n. We write this as
P(A) m/n
where '' means 'approximately equal to'.
At this early stage of your study of statistics it is a good idea to identify proportions (or,
relative frequencies) and probabilities as being almost the same, in the sense that a
proportion is an estimate of an underlying probability. The more times an experiment is
conducted the better will be the estimate of the probability.

You should now read LSS Section 4.1

2.5

Conditional Probabilities

Consider the following (hypothetical) result of a survey of 2000 people on how they view
the economy.

Men
Women

Very Favourably
0.2
0.15
0.35

Favourably
0.22
0.16
0.38

Unfavourably
0.12
0.15
0.27

0.54
0.46
1.00

This cross-classification means, for example, that 0.2 of the sample (that is, 400 people)
were both men and 'very favourable' in their view of the economy. Again, 0.15 (or 300
people) were both women and 'unfavourable' in their view of the economy. The
proportions in the table are relative to the whole sample.
Often our interest lies in a proportion of a subset of the sample. For example, we may be
interested in the proportion of men (subset) who view the economy 'very favourably'. This
proportion is 0.2/0.54 = 0.37. As another example, we may be interested in the

12

QM161 - Business Statistics 1

Topic Notes

proportion of those who view the economy 'very favourably' (a subset of the sample) who
are men. This proportion is 0.2/0.35=0.57
If we now view proportions as probabilities, we have calculated two 'conditional'
probabilities. The proportion 0.37 is the probability of viewing the economy 'very
favourably' conditional on being a man.
The second example shows 0.57 to be the probability of being a man, conditional on
viewing the economy 'very favourably'.
If

A = event of 'man'
B = event 'very favourably'.

Then the probability of B conditional on A, denoted by P(B|A), is given by

P( B A) =

P( Aand B)
, provided that P(A) is not zero.
P( A)

This can be depicted in the Venn diagram.

P(A and B)
B

Given that event A has already occurred, the new sample space is (the spotty oval) A. The
probability of observing B within this new (reduced) sample space is P(A and B).
Similarly, the probability of A, conditional on B, denoted by P(A|B), is given by

P( A B) =

P( A and B)
, provided that P(B) is not zero.
P( B)

The concept of a conditional probability, that is a probability relative to a reduced sample

space, is very important for two main reasons.

13

QM161 - Business Statistics 1

Topic Notes

(i) It leads to the multiplicative rule that can result in easy solutions to otherwise
difficult problems.
(ii) It guides us to a natural definition of 'independent events', a concept that is crucial
in statistics.

2.6

The Multiplicative Rule

Given the conditional probability,

P ( B A) =

P( A and B )
,
P ( A)

we can multiply both sides of the equation by P(A) to obtain

P(A and B) = P(B|A)P(A).
Also, because

P( A B) =

P( A and B)
,
P( B)

we obtain
P(A and B) = P(A|B)P(B).
Therefore,
P(A and B) = P(B|A)P(A) =P(A|B)P(B).
This is known as the multiplicative rule.
You should note the following:
(i) It is a rule that enables us to find the probability of the joint event A and B. That is,
the probability that both A and B occur.
(ii) There are two alternative expressions for P(A and B). The first uses P(B|A) and the
second P(A|B). In any given context, one of these is much easier to find than the
other, and this consideration leads to the choice of which expression to use.
(iii) The multiplicative rule is very useful for solving questions where the experiment
consists of two stages, and the outcome at the second stage depends on what
happened at the first stage.

14

QM161 - Business Statistics 1

Topic Notes

Example 1
Two cards are drawn from a pack. What is the probability that both are aces?

Method 1
Use 'equally likely outcomes':
Let A be the event that both cards are aces.
Then,

number of ways of obtaining 2 aces

number of ways of drawing 2 cards
m
=
n

P ( A) =

Here m = 6, n = 1326 (using counting rules)

P( A) =

6
= 0.00452.
1326

Method 2
Using conditional probability:
Let

A1 = event of ace at first draw

A2 = event of ace at second draw.

We want the probability that both A1 and A2 occur. That is, we want P(A1 and A2). By the
multiplicative rule,
P(A1 and A2) = P(A1) P(A2|A1).

4 3
= 0.00452 .
52 51

Note that in this case we would not use the expression P(A1 and A2) = P(A2) P(A1|A2)
because P(A1|A2) is not easy to find.

Example 2
Let us revisit the problem of the three cards that are red-red, red-white and white-white.
The question was, given a red side was up after a draw from the hat, what is the
probability that when it is turned over it is also red? You will recall that the obvious
answer is 1/2 (wrong!), and the correct answer 2/3. We will now approach the problem
using conditional probabilities.
15

QM161 - Business Statistics 1

Topic Notes

Let

R1 = event that red is up after draw

R2 = event of red, when turned over.

The probability we want is that given red is up after draw, it is red when turned over.
This is the conditional probability P(R2|R1).
Now

P( R2 R1 ) =

P( R2 and R1 )
P( R1 )

The only way we can get the event R1 and R2, is if the red-red card was drawn.
P(R2 and R1) = 1/3.
Also P(R1) = 1/2 because the number of red and white faces is equal.

P( R2 R1 ) =

2.7

1/ 3
= 2 / 3 , as before.
1/ 2

Independence

In ordinary usage, we say that two events A and B are independent if they 'have nothing
to do with each other'. As an example, consider the two events of a person preferring tea
over coffee, and the person obtaining a university degree.
Let us define the following two events:
A = event the person prefers tea over coffee
B = event this person gains a university degree.
It seems reasonable to me to assume these events are independent. Therefore, whether
or not someone prefers tea will not influence whether they gain a degree, and vice-versa.
In probability terms, we could write
P(A|B) = P(A).
That is, the probability they prefer tea, given they graduate from a university, is the same
as the probability that they prefer tea. Another way of saying this is that the proportion of
university graduates who prefer tea to coffee is exactly the same as the proportion in the
whole population.
From the definition of conditional probability

16

QM161 - Business Statistics 1

Topic Notes

P( A B) =

P( A and B)
.
P( B)

Therefore, if A and B are independent

P( A) =

P( A and B)
.
P( B)

It follows that, if A and B are independent

P(A and B) = P(A)P(B).
We adopt this as our definition of independence, and say two events are independent if,
and only if, the probability of both occurring equals the product of their
probabilities. To check for independence calculate both sides of this equation. If both
sides are equal then A and B are independent events. If the two sides are not equal then
A and B are dependent events.
The generalisation of this rule is: If the m events, A1, A2,..., Am, are independent then
P(A1 and A2 and..., Am) = P(A1) P(A2) .. P(Am).

You should now read LSS Section 4.2.

17

QM161 - Business Statistics 1

Topic Notes

2.8

Worked Examples

Example 1
In this example a classification table is used.
A random sample of 500 people is taken and each person is asked their gender and
whether they enjoy shopping. The responses are reported in the table below.
Enjoys Shopping Male Female Total
Yes

136

224

360

No

104

36

140

Total

240

260

500

From the table we can deduce the following information:

1. Simple Probabilities
P(Male) = 240/500
P(Female) = 260/500
P(Yes) = 360/500
P(No) = 140/500
2. Joint Probabilities
P(Male and Yes) = 136/500
P(Male and No) = 104/500
P(Female and No) = 36/500
P(Female and Yes) = 224/500

Note that you could use the Multiplication Rule

P(Male and Yes) = P(Male|Yes) P(Yes) = 136/360 360/500 =136/500.
However, in these circumstances the answer can be obtained fairly quickly straight
from the table. But it is nice to link the two approaches!

18

QM161 - Business Statistics 1

Topic Notes

3. Probability of a Union Event

Remember that we use the Additive Rule.
P(Male or Yes) = P(Male) + P(Yes) P(Male and Yes)
= 240/500 + 360/500 136/500
=464/500
I will leave you to calculate the remaining union events.
Number of males in
the new sample space

4. Conditional Probability

P(Male | Yes) =

P(Male and Yes)

=
P(Yes)

136
360

500

500

136
360

New sample space

You can calculate some of the other conditional probabilities.

5. Independence
Are the events 'Male' and 'Enjoying Shopping' independent events?
If they are independent, then
P(Male and Yes) must equal P(Male)P(Yes).
We have already found that P(Male and Yes) = 136/500 = 0.272.
P(Male)P(Yes) = 240/500 360/500 = 0.3456.
As 0.272 0.3456, the events 'Male' and 'Enjoying Shopping' are not independent
events. This may be a surprising finding to some people!

Example 2
For some examples it is useful to use a probability (or decision) tree. If the information
you are given contains a conditional probability, then it may be a useful approach.
Consider the following example.
Your doctor sends you for a test for a serious disease. You are told that 10% of the test
results are positive and if the test result is negative, there is still a 5% chance that you
have the disease. However, if the test is positive, there is still a 70% chance that you do
not have the disease.

19

QM161 - Business Statistics 1

Topic Notes

1. What is the probability that you test positive and have the disease?
2. What is the probability that you have the disease?
To answer these questions we draw a probability tree that starts on the left of the page
and then branches according to test result and disease status. The probabilities are
placed on the branches. We use A to indicate a positive test result and B to represent that
you have the disease.
B - have the disease

.3

P(A and B)

= (.1)(.3)
= 0.03

A - Test +ive

.7

.1

B - no disease

P(A and B)

= (.1)(.7)
= 0.07

.9

B - have the disease

.05

P(A and B)

= (.9)(.05)
= 0.045

A - Test -ive
.95
B - no disease

P(A and B) = (.9)(.95)

= 0.855

Note

The joint probabilities are calculated by multiplying the probabilities on the

relevant branches (pathway). This is the multiplication rule in operation.

The joint probabilities sum to one. That is, 0.03 + 0.07 + 0.045 + 0.855 = 1

From the information on P(A) we can deduce P(A) and from the information on
P(B|A) we can deduce the P(B|A) etc.

20

QM161 - Business Statistics 1

Topic Notes

We can now answer the two questions.

1. The probability that you test positive and have the disease is
P(A and B) = (0.1)(0.3)

= 0.03.

2. The probability that you have the disease is

P(B) = P(B and A) + P(B and A)
= 0.045 +0.03
= 0.075.

2.9

A Final Comment

It is important that at each Section you try a few problems from the Exercises in LSS. Now
that you have covered the material in probability, I will repeat my earlier comment. I only
require you to understand the basic concepts. I do not expect you to be an expert in
solving difficult probability questions.

You can now complete Assignment 1

21

QM161 - Business Statistics 1